# Optimization

> Optimisation finds inputs that maximise or minimise an objective, from portfolio weights to strategy settings. Learn the methods and how to avoid overfitting.

Source: https://learn.tradelabsai.com/math/optimization/  
Track: Math and Statistics · Level: Advanced · Updated: 2026-10-03  
Publisher: TradeLabs AI (https://tradelabsai.com). Education, not financial advice.  
Cite as: TradeLabs Learn, "Optimization", https://learn.tradelabsai.com/math/optimization/

Optimisation is the process of finding the inputs that produce the best value of some objective: the portfolio weights with the lowest risk for a target return, the model parameters that best fit the data, or the strategy settings with the highest risk adjusted return. Optimisation is everywhere in quantitative trading. It is also dangerous: an optimiser will happily find settings that exploit noise in historical data, producing beautiful backtests that fail live. Knowing how optimisation works, and how to constrain it, is essential.

## The parts of an optimisation problem

| Part | Example |
|---|---|
| Objective function | Maximise Sharpe ratio; minimise portfolio variance; maximise likelihood |
| Decision variables | Portfolio weights; moving average lengths; model coefficients |
| Constraints | Weights sum to 1; no short positions; maximum 10% per asset |
| Data | Historical returns, prices or features |

## Types of problems and methods

| Problem type | Example | Common methods |
|---|---|---|
| Linear | Some allocation and scheduling problems | Linear programming |
| Quadratic | Mean variance portfolio optimisation | Quadratic programming. See [Portfolio Optimization](https://learn.tradelabsai.com/portfolio/portfolio-optimization/) |
| Smooth nonlinear | Maximum likelihood fitting | Gradient based methods (Newton, BFGS). See [Maximum Likelihood](https://learn.tradelabsai.com/math/maximum-likelihood/) |
| Non smooth or discrete | Choosing indicator lengths | Grid search, random search |
| Expensive and noisy | Tuning complex backtests | Bayesian optimisation, genetic algorithms |

## Grid search in strategy design

**Example: Optimising a moving average crossover**
A trader tests fast moving averages from 5 to 50 days and slow ones from 50 to 250 days, about 2,000 combinations, on 15 years of data. The best combination shows a Sharpe ratio of 1.4, while the median combination shows 0.4. The best result is likely inflated by luck: with 2,000 trials, some combinations will look excellent by chance. A better approach looks at the whole surface of results, prefers broad regions of good performance and tests the chosen settings on data not used in the search. See [Parameter Optimization](https://learn.tradelabsai.com/research/parameter-optimization/) and [P-Hacking and Multiple Testing](https://learn.tradelabsai.com/research/p-hacking-and-multiple-testing/).

## Overfitting: the core danger

The more parameters and combinations you optimise, the more the optimiser fits noise. Signs of overfitting:

- **Sharp peaks** in performance at specific parameter values, with poor results nearby.
- **Big gaps** between in sample and out of sample performance.
- **Many parameters** relative to the number of trades.
- **Rules that only make sense in hindsight.**

See [Overfitting and Curve Fitting](https://learn.tradelabsai.com/research/overfitting-and-curve-fitting/).

## Making optimisation more robust

1. **Limit free parameters** and search ranges.
2. **Prefer plateaus over peaks:** choose settings where nearby values also work well. See [Robustness and Stress Testing](https://learn.tradelabsai.com/research/robustness-and-stress-testing/).
3. **Use walk forward optimisation:** optimise on a past window, test on the next, then roll forward. See [Walk-Forward Analysis](https://learn.tradelabsai.com/research/walk-forward-analysis/).
4. **Hold out data** that is never used during development. See [In-Sample vs Out-of-Sample Testing](https://learn.tradelabsai.com/research/out-of-sample-testing/).
5. **Penalise complexity:** use regularisation or information criteria.
6. **Adjust for the number of trials** when judging results. See [Statistical Significance in Trading](https://learn.tradelabsai.com/math/statistical-significance/).

## Optimising portfolios

Mean variance optimisation is notoriously sensitive to inputs. Small changes in expected returns can produce extreme, concentrated weights, an effect Richard Michaud called "error maximisation". Practitioners use constraints, shrinkage of inputs, robust optimisation and simpler methods such as risk parity. See [Portfolio Optimization](https://learn.tradelabsai.com/portfolio/portfolio-optimization/) and [Risk Budgeting and Risk Parity](https://learn.tradelabsai.com/portfolio/risk-budgeting-and-risk-parity/).

## Local vs global optima

Complex objective functions can have many peaks. Gradient methods may get stuck at a local optimum. Starting from several points, using global methods or simplifying the problem helps. See [Calculus for Traders](https://learn.tradelabsai.com/math/calculus-for-traders/).

## Tools

In Python, SciPy's `optimize` module handles many problems, cvxpy handles convex problems such as portfolio optimisation, and libraries such as Optuna provide Bayesian optimisation for parameter tuning. See [Python for Trading](https://learn.tradelabsai.com/programming/python-for-trading/).

## Frequently asked questions

### What is optimisation in trading?

Finding the inputs, such as strategy parameters or portfolio weights, that maximise or minimise an objective like Sharpe ratio or risk.

### Why is optimisation dangerous for traders?

Because it can fit noise in historical data, producing settings that look great in backtests but fail in live trading.

### How can I optimise strategies without overfitting?

Limit parameters, prefer broad stable regions of good results, use walk forward and out of sample testing, and adjust for the number of combinations tried.

Next, learn the statistics of economic data in [Econometrics](https://learn.tradelabsai.com/math/econometrics/).

## Continue learning

- Next lesson: [Econometrics](https://learn.tradelabsai.com/math/econometrics/)
- Previous lesson: [Calculus for Traders](https://learn.tradelabsai.com/math/calculus-for-traders/)
- Related: [Calculus for Traders](https://learn.tradelabsai.com/math/calculus-for-traders/): Calculus describes how quantities change. Learn derivatives, integrals and Taylor approximations, and how they underpin Greeks, duration and optimisation.
- Related: [Parameter Optimization](https://learn.tradelabsai.com/research/parameter-optimization/): Choosing strategy parameters by optimisation risks overfitting. Learn grid and random search, robustness surfaces, walk forward optimisation and sensible defaults.
- Related: [Portfolio Optimization](https://learn.tradelabsai.com/portfolio/portfolio-optimization/): Portfolio optimisation uses maths to choose weights that best meet a goal. Learn mean variance, minimum variance, constraints and how to handle estimation error.
- Related: [Overfitting and Curve Fitting](https://learn.tradelabsai.com/research/overfitting-and-curve-fitting/): Overfitting means a strategy fits noise instead of a real pattern. Learn the warning signs, why it happens, how to measure it and practical ways to avoid it.
- Related: [Maximum Likelihood](https://learn.tradelabsai.com/math/maximum-likelihood/): Maximum likelihood estimation finds the model parameters that make observed data most probable. Learn the idea, simple examples, its use in GARCH and its limits.
- Related: [Walk-Forward Analysis](https://learn.tradelabsai.com/research/walk-forward-analysis/): Walk forward analysis repeatedly optimises a strategy on past data and tests it on the next period. Learn how it works, window choices, efficiency ratios and limits.