Binary Protocols

Text formats such as JSON and FIX are readable but relatively slow to parse and large on the wire. For their fastest market data and order entry, many exchanges use binary protocols: messages laid out as fixed fields of raw bytes, each at a known position and size. A program can read a price from a binary message by looking at specific bytes, without searching for delimiters or converting text to numbers. This makes binary protocols smaller, faster and more predictable, at the cost of being unreadable to humans without a decoder.

Text versus binary#

Well known binary protocols#

How a binary message is laid out#

A simplified "add order" message might look like this:

This mirrors the style of Nasdaq ITCH, though the real specification has more fields. Prices stored as integers avoid floating point errors: 1,855,000 in units of one ten thousandth of a dollar is $185.50.

import struct
# '>' big endian, c=char, Q=8 byte uint, c, I=4 byte uint, 8s=8 bytes, I
fmt = ">cQQcI8sI"
msg_type, ts, ref, side, shares, sym, px = struct.unpack(fmt, raw[:34])
price = px / 10_000

Byte order and versions#

• Endianness: multi byte numbers can be stored big endian or little endian; the specification says which.
• Versioning: venues update protocols; decoders must check version fields and handle new message types.
• Schemas: SBE uses XML schemas from which encoders and decoders can be generated, reducing hand written errors.

Working with binary protocols#

1. Read the venue specification carefully; it is the only reliable source.
2. Use generated or proven decoders where available.
3. Test against sample captures the venue provides.
4. Track sequence numbers for gap detection. See Sequence Numbers, Dropped Packets and Out-of-Order Messages.
5. Log raw bytes for replay and debugging. See Market Data Replay.

Frequently asked questions#

What is a binary protocol in trading?#

A message format where fields are stored as raw bytes at fixed positions, making messages smaller and faster to process than text formats.

What is Nasdaq ITCH?#

Nasdaq's binary market data feed, which reports every order added, executed, modified or cancelled, allowing a full order book to be built.

What is SBE?#

Simple Binary Encoding, a FIX Trading Community standard for compact binary messages, used for example in CME's market data and order entry.

Next, learn how trading systems keep their clocks precise in Clock Synchronization and PTP.