Regression and Classification Models

Classification models answer questions with categories: will this stock go up or down tomorrow, will this trade reach its target before its stop, is the market in a calm or volatile regime? Most classifiers output a probability for each class, which makes them natural tools for trading decisions: trade only when the model is confident enough. Getting value from a classifier requires choosing the right evaluation measures, because accuracy alone often misleads.

Common classification models#

The confusion matrix#

From this table come the key measures:

• Accuracy: share of all predictions that were correct.
• Precision: of the times the model said up, how often it was right.
• Recall: of all the actual up moves, how many the model caught.
• Log loss: how good the predicted probabilities are, not just the labels.

See Statistical Power and Type I and II Errors.

Choosing a probability threshold#

Classifiers typically output a probability, and the default rule is to predict "up" above 50%. In trading you can raise the threshold, trading only when the model is more confident. Higher thresholds usually mean fewer trades with higher precision, up to a point. Choose the threshold on validation data, based on expected profit after costs rather than accuracy.

Calibration#

A well calibrated model's 60% predictions come true about 60% of the time. Many models, especially tree ensembles, produce poorly calibrated probabilities. Calibration methods such as Platt scaling or isotonic regression fix this, which matters if you size positions by probability. See Kelly Criterion.

Pitfalls#

1. Class imbalance: rare events make accuracy meaningless; use precision, recall and balanced weights.
2. Leakage: features calculated with future data. See Data Leakage.
3. Random train and test splits on time series, which leak future information. See Model Evaluation and Cross-Validation.
4. Ignoring move size: a model right on small moves and wrong on large ones can lose money with high accuracy.
5. Too many features relative to data. See Overfitting and Curve Fitting.

Frequently asked questions#

What is a classification model in trading?#

A model that predicts a category, such as up or down or target hit versus stop hit, usually with a probability for each outcome.

Is accuracy a good measure for trading models?#

Not on its own. Precision on trades taken, the size of wins and losses, and costs determine profitability.

What is model calibration?#

How well predicted probabilities match actual frequencies; a calibrated model's 70% predictions come true about 70% of the time.

Next, learn one of the most reliable model types in Random Forests and Gradient Boosting.