Backtesting

A backtest asks a simple question: if I had followed these exact rules in the past, what would have happened? It's the closest thing trading has to a laboratory — and, run naively, the most efficient lie-generating machine ever pointed at a P&L. The lies have names. Learn all four and you're ahead of most people selling "94% accurate strategies" online.

Lie 1 — Overfitting (torturing the data). Start with a simple rule: mediocre results. Add a filter: better. Add four more filters, tune each number, exclude "that weird year": suddenly the equity curve is a staircase to heaven. What happened? You stopped discovering a pattern and started memorizing the past — encoding history's random noise as if it were signal. The tell is fragility: change a parameter slightly (a 20-day rule to 22) and an overfit strategy's results collapse; a real edge degrades gracefully. Your screener chapter met the junior version (Trap 2); your FA school's Simons chapter met the professional standard: patterns must survive brutal testing across data they've never seen. Rule of thumb: every parameter you tune is a confession you might be extracting — fewer, rounder, more principle-driven numbers age better than precisely optimized ones.

Lie 2 — Survivorship bias (testing on the winners' graveyard-free list). Backtest "buy quality smallcaps" on today's stock list and you've quietly excluded every company that went bankrupt, got delisted, or collapsed along the way — you tested on the survivors, the exact stocks that, by definition, didn't die. Results inflate automatically. The same bias runs deeper than stock lists: strategies you hear about survived to be talked about (nobody posts threads about their deleted systems); fund performance tables drop closed funds. Honest testing needs point-in-time universes (what was actually tradeable then) — and honest skepticism knows most free data can't provide them, so smallcap backtests especially deserve a permanent mental haircut.

Lie 3 — Lookahead bias (accidentally using tomorrow's newspaper). The subtlest lie: your rules quietly use information that didn't exist at decision time. Classics: acting on a day's close price with an order placed during that day (the close wasn't known yet — your Chapter 3 forming-candle lesson, weaponized); screening on annual results as of January when they were published in May; using split-adjusted prices whose adjustments encode future knowledge. Lookahead doesn't feel like cheating — it feels like a coding detail — and it routinely turns dead strategies into stars. The audit question for every rule: at the exact moment this decision fires, was every input already public and final?

Lie 4 — Ignored costs (the frictionless fantasy). A backtest that fills every order at the exact signal price, free, has assumed away your entire Market Structure school: no spread (the toll, Chapter 4), no slippage (Chapter 9), no impact, no costs. For a monthly-rebalance investing strategy the fantasy is mildly optimistic; for anything intraday it's fatal — high-frequency backtest profits are routinely a costs-shaped illusion. Honest tests charge realistic per-trade tolls (your own measured slippage from the journal is the gold input) — and if profits vanish under real costs, the strategy was never there.

The honest workflow, assembled: (1) hypothesis first, written, with a reason the edge should exist (behavioral or structural — your other schools are the source of legitimate reasons); (2) simple rules, few parameters; (3) test on one slice of history (in-sample), then verify untouched on another (out-of-sample — data the strategy has never seen; the single most important habit in this chapter); (4) charge full costs; (5) stress the parameters — graceful degradation or collapse?; (6) expect the live version to underperform the test (it nearly always does — every remaining hidden bias points the same optimistic direction). Tools note: TradingView's strategy tester covers rule-based technical ideas at retail level (with its own fill assumptions — read them); spreadsheet-level testing works for slower strategies; and any tool obeys the same four lies, because the lies live in the method, not the software.