Feed Handlers and Normalization

Every exchange publishes market data in its own format. One sends JSON over WebSocket with prices as strings; another sends compact binary messages over multicast; a third names the same instrument differently and uses a different time unit. A feed handler is the component that connects to one venue's feed, decodes its messages and converts them into a single standard internal format. Everything downstream, from bar builders to strategies, then works with one consistent model regardless of where the data came from.

What a feed handler does#

Normalisation: the details that matter#

Price representation#

Floating point numbers cannot represent many decimal prices exactly, so 0.1 plus 0.2 does not equal exactly 0.3. For research this rarely matters, but order prices must be exact multiples of the tick size. Many systems store prices as integers in tick units or as scaled integers and convert only for display. See Tick Size and Tick Value.

Symbol and reference data#

A reference data service holds each instrument's details: venue symbols, tick size, lot size, contract multiplier, trading hours and status. Feed handlers use it to map and scale values. Keeping it current is critical, because venues add, delist and change instruments frequently. See Contract Specifications.

Design principles#

• One handler per venue, isolating venue specific quirks.
• Fail loudly on unknown message types or fields rather than guessing.
• Keep the raw message or a raw log for debugging and replay. See Market Data Replay.
• Timestamp on arrival as well as keeping the venue's timestamp. See Timestamps, Time Zones and Daylight Saving.
• Measure latency from venue time to publish time. See Exchange vs Receive Timestamps and Latency Measurement.

Build or buy#

Writing feed handlers for a few crypto exchanges is a manageable project. Covering many traditional exchanges with binary multicast feeds is a large engineering effort, which is why firms often buy normalised data from vendors or use open source libraries that already support many venues, such as CCXT for crypto REST and WebSocket APIs.

Frequently asked questions#

What is a feed handler?#

Software that connects to a venue's market data feed, decodes its messages and converts them into a standard internal format for the rest of a trading system.

Why is data normalisation needed?#

Each venue uses different symbols, units, number formats and timestamps; normalisation lets strategies treat data from all venues the same way.

Should prices be stored as floats?#

For research floats are usually fine, but for order prices many systems use integers in tick units to avoid rounding errors.

Next, learn how sequence numbers keep feeds reliable in Sequence Numbers, Dropped Packets and Out-of-Order Messages.