Trading Infrastructure Explained
A tour of trading infrastructure: servers, networks, data feeds, order gateways, storage and monitoring, and how needs differ for retail bots, funds and HFT firms.
Trading infrastructure is everything that sits between a trading idea and a filled order: the computers that run strategies, the networks that carry data and orders, the feed handlers and gateways that talk to venues, the databases that store history and the monitoring that keeps it all honest. A retail trader running one bot needs a small slice of this; a high frequency firm spends tens of millions on it. Understanding the pieces helps you choose what you actually need and avoid spending on speed that will not change your results.
The main layers#
| Layer | Purpose | Lesson |
|---|---|---|
| Compute | Servers that run strategies and services | VPS, Cloud and Bare-Metal Servers |
| Operating system | Usually Linux for servers | Linux for Traders |
| Network | Connectivity to venues and data | Networking for Traders |
| Market data | Feed handlers and normalisation | Feed Handlers and Normalization |
| Messaging | Moving events between components | Message Queues |
| Storage | Databases and caches | Redis and PostgreSQL for Trading |
| Order gateway | Risk checks and connections to brokers or venues | Risk Controls and Kill Switches |
| Deployment | Packaging and running services | Docker and Kubernetes |
| Observability | Logs, metrics, alerts | Monitoring and Logging Systems |
| Resilience | Failover and recovery | Fault Tolerance, High Availability and Redundancy |
Three scales of infrastructure#
| Retail bot | Systematic fund | HFT firm | |
|---|---|---|---|
| Compute | One cloud server or VPS | Clusters for research, dedicated servers for trading | Tuned bare metal in exchange data centres |
| Network | Internet | Leased lines, cloud interconnects | Co location, microwave links |
| Data | Broker or exchange APIs | Vendor feeds, consolidated data | Direct exchange feeds |
| Latency target | Seconds to hundreds of milliseconds | Milliseconds | Microseconds to nanoseconds |
| Key tech | Python, a database, alerts | Python and C++, orchestration | C++, FPGAs, kernel bypass. See FPGAs and Hardware Acceleration |
A sensible retail architecture#
- A small Linux cloud server in a region near your broker or exchange.
- Your bot as a service that restarts automatically on failure.
- PostgreSQL or SQLite for orders, fills and state.
- Logs and a heartbeat alert to your phone. See Alerts and Webhooks.
- Broker side stop orders as a safety net. See Failover, Backups and Disaster Recovery.
- Version control and backups of code and configuration.
This costs little per month and covers most needs. See Building Trading Bots.
Reliability before speed#
For most traders, the main infrastructure risks are outages, crashes and silent data failures, not microseconds. Prioritise:
- Automatic restart and state recovery.
- Monitoring and alerts. See Monitoring Positions, P&L and Risk.
- Reconciliation with the broker. See Trade Accounting and Reconciliation.
- Tested backups.
When speed starts to matter#
Latency becomes important for strategies that react to short lived opportunities: arbitrage, market making, news reaction and very short term momentum. There, infrastructure is part of the edge, and competition is intense. See Latency in Trading and High-Frequency Trading.
Security#
Trading systems hold keys that can move money. Basic security matters at every scale: limited API key permissions, secrets kept out of code, firewalls, software updates and multi factor authentication on broker and server accounts.
Frequently asked questions#
What is trading infrastructure?#
The computers, networks, data feeds, software services, storage and monitoring that support trading from data to execution.
Do I need a server to run a trading bot?#
Not strictly, but a cloud server is more reliable than a home computer and costs little, so it is the usual choice for bots that run unattended.
How much does trading infrastructure cost?#
From a few dollars a month for a retail bot on a small cloud server to millions per year for high frequency firms with co location and private networks.
Next, compare where to run your systems in VPS, Cloud and Bare-Metal Servers.
3 quick questions on this lesson. Get them all right to finish it.
Turn on JavaScript to take the quiz.
Mentioned in
- CPU Affinity, NUMA and Cache OptimizationTrading Infrastructure
- Market Data ReplayProgramming and Data
- Quant Developer and Trading EngineerThe Trading Industry