Market Data Replay

Market data replay means recording the exact data your system received live and later feeding it back through the same code, as if the market were happening again. It sits between a backtest and live trading: more realistic than a backtest because it uses the real message stream, with its gaps, bursts and timing, and safer than live trading because no real orders are sent. Replay is one of the best tools for finding bugs, reproducing incidents and checking that a new version of a bot behaves exactly like the old one.

Replay versus backtesting#

See Backtesting Methodology.

What to record#

• Raw messages from every feed, with the receive timestamp on each.
• Periodic snapshots of order books so replay can start mid day. See Order Book Feeds: Snapshots and Incremental Updates.
• Your own order events: orders sent, acknowledgements, fills and rejects.
• Configuration and code version running at the time. See Data Versioning, Lineage and Schemas.

Recording is cheap compared with the cost of an incident you cannot explain.

Replay modes#

Determinism#

A replay is only useful if the same input produces the same output every time. Sources of non determinism:

• Wall clock time: code that calls the system clock behaves differently on replay. Use a clock driven by the replayed timestamps instead.
• Random numbers without fixed seeds.
• Multithreading where event order depends on scheduling.
• External calls to live services during replay.

Designing the system so all time and input come through one event stream makes replay deterministic. See Backtest Reproducibility.

Regression testing with replay#

Before releasing a new version, replay several recorded days through both the old and new versions and compare every signal and order. Differences should be intended changes only. This catches accidental behaviour changes that unit tests miss. See Developing, Testing and Monitoring Algorithms.

Simulating fills during replay#

Replay of market data does not tell you how your own orders would have filled. A simulated exchange can match your orders against the recorded book, using queue position assumptions. This is still an estimate: your orders would have changed the market slightly. See Fill Models, Partial Fills and Order Queues and Fill Probability and Queue Position.

Frequently asked questions#

What is market data replay?#

Feeding recorded market data back through a trading system to test, debug or benchmark it without sending real orders.

How is replay different from backtesting?#

Backtests usually run research code on cleaned data; replay runs the production system on the raw messages it actually received, including timing and gaps.

Why must replay be deterministic?#

So the same input always produces the same output, which makes bugs reproducible and comparisons between versions meaningful.

You have finished the Programming and Data track. Continue with the systems that run it in Trading Infrastructure Explained.