How to backtest a trading strategy without fooling yourself

A backtest is running a strategy's rules on historical data to see what would have happened. It's the only tool that separates a method from a hypothesis, and also the easiest tool to fool yourself with: almost every first backtest comes out profitable, and most of that profit disappears the moment you add costs, realistic execution, and stocks that no longer exist. This page is the protocol we work by, including the mistakes we caught in our own measurements.

The rule above all others: if you change the rules after seeing the result, you're no longer testing, you're fitting. A method fitted to the past explains the past perfectly and the future not at all.

Seven mistakes that turn a backtest into a story

• Look-ahead bias: Using information that wasn't available at the moment of decision. The common example: a signal calculated from a candle that hasn't closed yet, or an average that includes today's close in this morning's decision. For us, this was fixed when we switched to generating signals from closed candles only.
• Imaginary fills: Assuming you bought at the trigger price even when the price only "passed near it." In our measurement, skipping the check that the candle's range actually contained the entry price produced a fake profit of hundreds of risk units: losses always get filled (the stop requires passing through the entry) but runners don't always.
• No costs: Commission, bid-ask spread, and slippage. A method with thousands of small trades can flip from profit to loss from this alone. Also run a scenario with doubled costs.
• Survivorship bias: Today's universe is applied to the past. Stocks that crashed and were dropped from the index aren't included, so any method that buys "winners" looks too good. For us, this is noted next to every table, because we haven't solved it.
• Too few trades: Below two hundred trades, the difference between two versions is usually noise. 28 golden-cross trades in our tournament are an example: a row you shouldn't draw conclusions from.
• Overfitting: Testing twenty configurations and publishing the best one. If configuration 14 works and 13 and 15 don't, that's not an edge, it's luck selected in hindsight.
• One window: Three years of a bull market tell one story. Everything we measured from 2023 to 2026 was measured in a bull market, and that's part of the answer.

the protocol

• Rules written before the data. Entry, exit, size, filters, and what happens on a tie. A rule that isn't written doesn't exist.
• Realistic execution: Signal at close, execution at the next open; stop filled at the stop price or at the open, whichever is worse; target only if the candle actually reached it; on a tie, the loss. Every uncertainty against the method.
• Split over time: Development on one period, validation on a period you haven't touched, and then a final test period seen only once. If the result is positive only in development, that's overfitting.
• Pre-set acceptance gates: At least two hundred trades, positive expectancy after costs, profit factor above 1.25, and a result that remains positive under doubled costs and in at least two market regimes.
• Paper trading: A hundred trades with documentation of the trigger, actual fill, slippage, and rule violations. If paper trading doesn't match the backtest, the problem is in execution, not the method.
• Real money, small size: A tenth of a percent risk per trade for the first fifty trades. Only then do you scale up.

What to do with the result

A method that passed the gates isn't a promise, it's a license to trade it small and keep measuring. A method that failed at one gate isn't "improved from memory": you formulate one hypothesis for the change, and test it on new data. TechAnalyzer Pro's live track record is exactly this stage of our method: after the backtest, before any claim. All the hypotheses added to the engine this year (minimum risk-reward ratio, regime gate, closed-candle signal generation, extreme RSI blocking) went through the same process, and what was measured and didn't improve wasn't included.