The Beautiful Lie

This is the most important chapter in the school. Not the most exciting — the most important. Everything before it produces a strategy; everything after it deploys one. This chapter decides whether the thing you deploy is real.

Start with the student.

A student gets hold of last year's exam paper and memorises every answer, perfectly. On last year's exam, he scores 100%.

Is he ready for this year's exam?

Obviously not — he learned the answers, not the subject. And yet his practice score says 100%, and practice scores feel like proof.

A backtest is running your rules against last year's exam. It's genuinely powerful — the testability superpower from Chapter 1, the only way to examine a strategy before it touches money. And it is also the most fluent liar in finance, because of one seductive mechanic:

You can adjust the rules and re-run the past as many times as you like. Change the average from 50 days to 43. Add a filter. Nudge the stop. Each tweak, the past-score improves... and each tweak, you drift from learning the subject toward memorising the answers. The industry's word for it is overfitting (or curve-fitting), and it is the graveyard of retail algo trading: strategies sculpted into perfect agreement with a past that will never repeat in its details. Look at the figure — the green in-sample beauty, the red live collapse. Every quant recognises that chart. Most paid tuition to recognise it.

The cruellest part: overfitting doesn't feel like cheating. It feels like research. Every tweak has a story ('markets respect the 43-day average because...'). You've met this machine before — Chapter 4 of the Theories school showed you a brain finding patterns in coin flips. A backtester is that brain, given a re-run button.

Overfitting is lie number one. Three siblings complete the family:

Lie two — look-ahead bias: using tomorrow's newspaper. The subtlest bug in the field: your rules accidentally use information that didn't exist at decision time. Classics: acting on today's close at today's open; using financial data on the calendar date it covers rather than the (later) date it was actually published; 'buying the day's low' as if the low announces itself in the morning. One leaked day of hindsight can make a worthless strategy look brilliant — and the bug hides in a single innocent line of logic.

Lie three — survivorship bias: testing only on the winners. Test a strategy on 'the current index members over 20 years' and you've quietly excluded every company that collapsed, got delisted, or faded away — the very stocks that would have wrecked the strategy. You're studying lottery winners to prove lottery-playing works. Honest tests use the universe as it stood at the time, corpses included.

Lie four — the frictionless world: ignoring costs. Backtests execute instantly, at the exact printed price, for free. Reality charges commissions, taxes, the bid-ask spread, and slippage — the gap between the price that triggered you and the price you actually got. For fast-trading strategies these frictions aren't rounding errors; they're the whole verdict. A rule of thumb worth tattooing: the more often a backtest trades, the more its profits are fiction until costs are modelled brutally. Many a 'profitable' system is, after honest friction, a machine for donating spreads.

Now the defence — the honest-testing checklist. Five habits, and they're the difference between research and self-deception:

1. Split the exam. Before touching anything, divide history: tune your rules on the first part (in-sample) and only then — once, solemnly — run them on the untouched remainder (out-of-sample). The out-of-sample result is the only score that counts. And it's a one-use item: peek, tweak, re-run, and it has quietly become in-sample too.

2. Prefer ugly simplicity. Every rule and parameter you add is another opportunity to memorise the past. A strategy with three rules that tests decently is worth ten of one with fifteen rules that tests perfectly. When two versions are close, the dumber one is telling more truth.

3. Demand robustness, not perfection. Wiggle every parameter: if 50 days works but 45 and 55 collapse, you haven't found an edge — you've found a coincidence with an address. Real effects are neighbourhoods, not pinpoints. Test across different instruments and different years; an edge that exists only in one stock in one era is memorised answers.

4. Charge yourself unfairly. Model costs and slippage on the pessimistic side, always. If the strategy survives being over-charged, reality will feel like a bonus. If it only survives being under-charged, you already have your answer.

5. Count the evidence. Thirty trades of backtest history is an anecdote (the coin-flip chapter told you how far chance stretches). Demand enough trades, across enough different market weathers — including at least one storm — before the word 'edge' is allowed in the room.

And carry the paradox that governs this whole craft, the one that separates professionals from victims:

In backtesting, beauty is evidence against you. A too-smooth equity curve, a 90% win rate, returns that embarrass the greatest funds in history — these are not achievements. They're symptoms. The honest backtest of a real edge usually looks... decent. Bumpy. Survivable. Mildly disappointing. That's what truth tends to look like before costs of living.

Your recipe has now faced the past honestly. Next problem: the present — and everything in it that a backtest cannot contain.

🇮🇳 The India Angle

• Survivorship bias bites hard in Indian data: indices reshuffle regularly and delistings are common — free datasets typically contain today's survivors only. Ask any data source the corpse question before trusting a multi-year test.
• Cost modelling must include the full Indian friction stack — brokerage, exchange charges, stamp duty, and the securities transaction tax — plus real spreads on option strikes, which widen dramatically away from the money. Many published 'profitable' intraday backtests die on STT alone.
• Expiry-day regime changes (weekly expiries moving days/products in recent years) are a look-ahead trap of their own: a backtest spanning regime changes must use each era's actual rules, not today's.