# Backtesting Methodology

> A backtest simulates a strategy on historical data. Learn the steps, the key performance metrics, common biases and a checklist for backtests you can trust.

Source: https://learn.tradelabsai.com/research/backtesting-methodology/  
Track: Research and Backtesting · Level: Intermediate · Updated: 2026-10-03  
Publisher: TradeLabs AI (https://tradelabsai.com). Education, not financial advice.  
Cite as: TradeLabs Learn, "Backtesting Methodology", https://learn.tradelabsai.com/research/backtesting-methodology/

A backtest simulates how a trading strategy would have performed on historical data. It is the main tool for evaluating ideas before risking money. Done carefully, backtesting filters out bad ideas and gives a realistic sense of what to expect. Done carelessly, it produces impressive numbers that mean nothing. The difference lies in methodology: clean data, realistic assumptions, honest validation and awareness of the many biases that can creep in.

## The steps of a backtest

1. **Define the rules precisely:** entries, exits, position sizing and risk limits, with no room for interpretation.
2. **Prepare data:** clean, adjusted for corporate actions, free of survivorship bias, with correct timestamps. See [Historical Data for Backtesting](https://learn.tradelabsai.com/research/historical-data-for-backtesting/).
3. **Simulate trading** through time, using only information available at each moment.
4. **Apply costs and realistic fills.** See [Costs and Slippage in Backtests](https://learn.tradelabsai.com/research/costs-and-slippage-in-backtests/) and [Fill Models, Partial Fills and Order Queues](https://learn.tradelabsai.com/research/fill-models/).
5. **Calculate performance metrics.**
6. **Validate** out of sample and test robustness. See [In-Sample vs Out-of-Sample Testing](https://learn.tradelabsai.com/research/out-of-sample-testing/).
7. **Document** everything for reproducibility. See [Backtest Reproducibility](https://learn.tradelabsai.com/research/backtest-reproducibility/).

## Key performance metrics

| Metric | What it shows | Lesson |
|---|---|---|
| Total and annual return (CAGR) | Growth | [Measuring Returns and CAGR](https://learn.tradelabsai.com/portfolio/measuring-returns-and-cagr/) |
| Volatility | Variability of returns | [Variance and Standard Deviation](https://learn.tradelabsai.com/math/variance-and-standard-deviation/) |
| Sharpe and Sortino ratios | Return per unit of risk | [Sharpe Ratio](https://learn.tradelabsai.com/portfolio/sharpe-ratio/), [Sortino Ratio](https://learn.tradelabsai.com/portfolio/sortino-ratio/) |
| Maximum drawdown | Worst peak to trough loss | [Maximum Drawdown](https://learn.tradelabsai.com/portfolio/maximum-drawdown/) |
| Win rate and payoff ratio | Trade level behaviour | [Win Rate and Payoff Ratio](https://learn.tradelabsai.com/portfolio/win-rate-and-payoff-ratio/) |
| Profit factor | Gross profit / gross loss | [Profit Factor](https://learn.tradelabsai.com/portfolio/profit-factor/) |
| Number of trades | Statistical reliability | [Statistical Significance in Trading](https://learn.tradelabsai.com/math/statistical-significance/) |
| Turnover and costs | Trading intensity | [Signal Turnover, Breadth and Neutralization](https://learn.tradelabsai.com/research/signal-turnover/) |
| Exposure | Time and capital in the market | |

## Event driven vs vectorised backtests

| Approach | How it works | Pros | Cons |
|---|---|---|---|
| Vectorised | Calculate signals and returns for all dates at once with arrays | Fast, simple | Harder to model complex order logic; easier to introduce look ahead bias. See [Event-Driven vs Vectorized Backtesting](https://learn.tradelabsai.com/research/vectorized-backtesting/) |
| Event driven | Process data one event at a time, as in live trading | Realistic; same code can run live | Slower, more complex |

## Avoiding biases

| Bias | Prevention |
|---|---|
| Look ahead bias | Lag signals; use point in time data. See [Look-Ahead Bias](https://learn.tradelabsai.com/research/look-ahead-bias/) |
| Survivorship bias | Include delisted securities. See [Survivorship and Selection Bias](https://learn.tradelabsai.com/research/survivorship-and-selection-bias/) |
| Data snooping | Limit variations; record all tests. See [P-Hacking and Multiple Testing](https://learn.tradelabsai.com/research/p-hacking-and-multiple-testing/) |
| Overfitting | Few parameters; robustness tests. See [Overfitting and Curve Fitting](https://learn.tradelabsai.com/research/overfitting-and-curve-fitting/) |
| Unrealistic execution | Conservative costs and fill assumptions |

**Example: A lag that changes everything**
A strategy buys at today's close when today's close is above the 20 day moving average. A careless backtest computes the signal from today's close and also fills at today's close, implicitly knowing the close before it happens. Realistically, the order can only fill at the next day's open or a later price. Changing to next day fills cuts the backtest's annual return from 18% to 6%. Many impressive backtests hide errors like this. See [Look-Ahead Bias](https://learn.tradelabsai.com/research/look-ahead-bias/).

## A backtest checklist

1. **Are rules fully specified** before testing?
2. **Is every input available at the time of each decision?**
3. **Does the universe include delisted and failed securities?**
4. **Are costs, slippage and borrowing included,** conservatively?
5. **Is position sizing realistic** relative to liquidity?
6. **How many variations were tested?**
7. **Does it hold out of sample and across periods and markets?**
8. **Are results driven by a few trades or one period?**
9. **Can someone else reproduce the results from your code and data?**

## Interpreting results

A backtest is a hypothesis about the future, not a promise. Treat headline numbers as optimistic upper bounds and focus on robustness, drawdowns and the range of plausible outcomes. Monte Carlo resampling helps show that range. See [Monte Carlo Simulation](https://learn.tradelabsai.com/research/monte-carlo-simulation/).

## Frequently asked questions

### What is backtesting?

Simulating a trading strategy on historical data to estimate how it would have performed.

### What makes a backtest reliable?

Clean, point in time data without survivorship bias, precise rules, realistic costs and fills, few tested variations, and validation on data not used to build the strategy.

### Is a good backtest enough to trade a strategy?

No. Backtests are usually optimistic; strategies should also pass out of sample tests, robustness checks and small live trading.

Next, learn how to choose and prepare data in [Historical Data for Backtesting](https://learn.tradelabsai.com/research/historical-data-for-backtesting/).

## Continue learning

- Next lesson: [Historical Data for Backtesting](https://learn.tradelabsai.com/research/historical-data-for-backtesting/)
- Previous lesson: [Why Strategies Fail](https://learn.tradelabsai.com/research/why-strategies-fail/)
- Related: [Why Strategies Fail](https://learn.tradelabsai.com/research/why-strategies-fail/): Most strategies that look good in backtests fail live. Learn the main causes, from overfitting and costs to regime changes, and how to guard against each.
- Related: [Historical Data for Backtesting](https://learn.tradelabsai.com/research/historical-data-for-backtesting/): Backtests are only as good as their data. Learn data types and sources, quality checks, corporate action adjustments and how to avoid survivorship traps.
- Related: [In-Sample vs Out-of-Sample Testing](https://learn.tradelabsai.com/research/out-of-sample-testing/): Out of sample testing checks a strategy on data not used to build it. Learn train, validation and holdout splits, common mistakes and how to read results.
- Related: [Costs and Slippage in Backtests](https://learn.tradelabsai.com/research/costs-and-slippage-in-backtests/): Ignoring costs is the fastest way to fool yourself in a backtest. Learn the costs to include, how to estimate slippage and market impact, and conservative rules.
- Related: [Look-Ahead Bias](https://learn.tradelabsai.com/research/look-ahead-bias/): Look ahead bias happens when a backtest uses information that was not available at the time. Learn common sources, real examples and how to prevent it in code.
- Related: [Event-Driven vs Vectorized Backtesting](https://learn.tradelabsai.com/research/vectorized-backtesting/): Vectorised backtests compute signals and returns for all dates at once using arrays. Learn how they work, a pandas example, their speed advantages and their traps.