# Sampling and Standard Error

> Standard error measures how much an estimate like a win rate or average return varies between samples. Learn the formulas and what they mean for backtests.

Source: https://learn.tradelabsai.com/math/sampling-and-standard-error/  
Track: Math and Statistics · Level: Intermediate · Updated: 2026-10-03  
Publisher: TradeLabs AI (https://tradelabsai.com). Education, not financial advice.  
Cite as: TradeLabs Learn, "Sampling and Standard Error", https://learn.tradelabsai.com/math/sampling-and-standard-error/

Every statistic you calculate from trading data, such as a win rate, an average return or a Sharpe ratio, is an estimate based on a sample. If you had a different sample of trades or a different period, you would get a different number. The standard error measures how much that estimate would typically vary from sample to sample. It is the key to knowing whether a result is meaningful or just noise, and it explains why small backtests can be so misleading.

## Population vs sample

- **Population:** all possible outcomes, such as every trade a strategy would ever take.
- **Sample:** the trades you actually observe.

We use the sample to estimate population values. The standard error tells us how precise that estimate is.

## Standard error of the mean

```
SE(mean) = s / √n
```

where s is the sample standard deviation and n is the sample size.

**Example: Average return per trade**
A strategy's 64 trades average +0.5% with a standard deviation of 2.0%.

SE = 2.0% / √64 = 2.0% / 8 = 0.25%.

The true average could plausibly be anywhere from about 0% to 1.0% (roughly ±2 standard errors). If the strategy had only 16 trades with the same numbers, SE would be 0.5%, and the true average could be anywhere from about minus 0.5% to +1.5%, which includes losing. See [Confidence Intervals](https://learn.tradelabsai.com/math/confidence-intervals/).

## Standard error of a proportion (win rate)

```
SE(win rate) = √(p × (1 - p) / n)
```

| Trades | SE of a 55% win rate |
|---|---|
| 25 | About 9.9% |
| 100 | About 5.0% |
| 400 | About 2.5% |
| 1,600 | About 1.2% |

Quadrupling the number of trades halves the standard error. See [Law of Large Numbers](https://learn.tradelabsai.com/math/law-of-large-numbers/).

## Standard error of the Sharpe ratio

For roughly normal returns, an approximation from Andrew Lo (2002) is:

```
SE(annual Sharpe) ≈ √((1 + 0.5 × SR²) / years)
```

A Sharpe ratio of 1.0 measured over 3 years has a standard error of about √(1.5 / 3) ≈ 0.71, so the true Sharpe could plausibly be anywhere from below zero to above 2. Over 10 years, the standard error falls to about 0.39. See [Sharpe Ratio](https://learn.tradelabsai.com/portfolio/sharpe-ratio/).

## Sampling problems in trading

| Problem | Effect | Lesson |
|---|---|---|
| Small samples | Large standard errors; luck dominates | |
| Overlapping data | Returns measured over overlapping windows are not independent; true sample size is smaller | |
| Autocorrelation | Correlated returns understate standard errors | [Autocorrelation and Partial Autocorrelation](https://learn.tradelabsai.com/math/autocorrelation/) |
| Survivorship bias | Samples missing failed assets | [Survivorship and Selection Bias](https://learn.tradelabsai.com/research/survivorship-and-selection-bias/) |
| Selection bias | Choosing the best of many tests | [P-Hacking and Multiple Testing](https://learn.tradelabsai.com/research/p-hacking-and-multiple-testing/) |
| Regime dependence | The sample period may not represent the future | [Structural Breaks and Regime Changes](https://learn.tradelabsai.com/math/regime-changes/) |

## Effective sample size

If trades or returns are correlated, the effective number of independent observations is smaller than the count. Ten positions in highly correlated stocks taken on the same day are closer to one bet than ten. Using the raw count makes results look more reliable than they are.

## Practical guidelines

1. **Always report standard errors or confidence intervals** with backtest statistics.
2. **Be sceptical of results from fewer than about 30 to 50 independent trades.**
3. **Prefer longer histories** across different market conditions.
4. **Adjust for autocorrelation** and overlapping data when needed.
5. **Use bootstrapping** when distributions are unusual. See [Bootstrap and Permutation Tests](https://learn.tradelabsai.com/math/bootstrap-and-permutation-tests/).

## A quick sanity check

Before trusting any backtest figure, divide it by its standard error. A ratio below about 2 means the result could easily be chance. A ratio above 3, from data you did not use to design the strategy, is much more convincing. See [Statistical Significance in Trading](https://learn.tradelabsai.com/math/statistical-significance/).

## Frequently asked questions

### What is standard error?

A measure of how much a sample estimate, such as an average or win rate, would vary if you took different samples.

### How does sample size affect standard error?

Standard error shrinks with the square root of the sample size, so four times as many observations halves it.

### Why does standard error matter for backtests?

Because it shows how much of a backtest's result could be due to chance, helping traders avoid trusting lucky results.

Next, learn why averages tend toward a normal shape in [Central Limit Theorem](https://learn.tradelabsai.com/math/central-limit-theorem/).

## Continue learning

- Next lesson: [Central Limit Theorem](https://learn.tradelabsai.com/math/central-limit-theorem/)
- Previous lesson: [Covariance and Correlation](https://learn.tradelabsai.com/math/covariance-and-correlation/)
- Related: [Covariance and Correlation](https://learn.tradelabsai.com/math/covariance-and-correlation/): Covariance and correlation measure how two assets move together. Learn the formulas, how to read them, why correlations change in crises and their portfolio role.
- Related: [Confidence Intervals](https://learn.tradelabsai.com/math/confidence-intervals/): A confidence interval gives a range of plausible values for a statistic. Learn how to calculate them for returns and win rates and how to read them in backtests.
- Related: [Law of Large Numbers](https://learn.tradelabsai.com/math/law-of-large-numbers/): The law of large numbers says averages converge to the true value as samples grow. Learn what it means for judging strategies and how many trades you need.
- Related: [Central Limit Theorem](https://learn.tradelabsai.com/math/central-limit-theorem/): The central limit theorem says averages of many independent values tend toward a normal distribution. Learn what it means for trading statistics and when it fails.
- Related: [Statistical Significance in Trading](https://learn.tradelabsai.com/math/statistical-significance/): Statistical significance helps judge whether trading results reflect a real edge or luck. Learn the t statistic rule of thumb, sample size and multiple testing.
- Related: [Backtesting Methodology](https://learn.tradelabsai.com/research/backtesting-methodology/): A backtest simulates a strategy on historical data. Learn the steps, the key performance metrics, common biases and a checklist for backtests you can trust.