Every trading system eventually confronts the same architectural fork in the road: should I pull data with HTTP requests, or maintain a persistent connection?
The answer seems simple on the surface. Historical data retrieval calls for REST. Real-time price feeds call for WebSocket. Yet the reasoning behind this conventional wisdom runs deeper than convention—it touches on protocol design philosophy, resource constraints, failure modes, and the economics of network traffic. Understanding why each protocol fits its domain explains not just how to use TickDB's dual interface, but how to architect systems that remain resilient under load.
This article dissects the technical underpinnings of both approaches, examines the specific trade-offs as they apply to financial market data, and provides production-ready code patterns for integrating each protocol effectively.
The Fundamental Difference: Request-Response vs Push Architecture
At the most basic level, REST and WebSocket represent two distinct models of communication.
REST (Representational State Transfer) follows the request-response pattern. A client initiates a connection, sends a request, receives a response, and the connection closes. Every transaction is self-contained. There is no notion of a "session" at the protocol level. The client must poll—repeatedly ask—"Is there new data?"—even when nothing has changed.
WebSocket establishes a persistent, bidirectional tunnel. Once the handshake completes, either party can send messages at any time without re-establishing the connection. The server can push data to the client the instant it arrives, eliminating the latency penalty of polling.
This distinction has profound implications for market data scenarios.
| Characteristic | REST | WebSocket |
|---|---|---|
| Connection lifecycle | One request, one connection (typically) | Persistent across many messages |
| Data flow direction | Client pulls | Server pushes (bidirectional after handshake) |
| Latency | Request round-trip time | Near-instant on new data |
| Network overhead | Headers on every request | Handshake once, then minimal overhead |
| Scalability for server | Higher CPU cost per message (HTTP parsing) | Lower per-message cost (direct socket write) |
| Client simplicity | Easier to implement, debug, cache | More complex state management |
| Failure recovery | Stateless—retry any request | Must re-establish connection, replay state |
Neither protocol is universally superior. The choice depends entirely on what kind of data you need and how time-sensitive it is.
Scenario Analysis: When Each Protocol Shines
Historical Data Retrieval: REST is the Right Tool
Consider the task of backtesting a mean-reversion strategy on three years of NVIDIA hourly data. You need 1,576 trading days × 7 hours = roughly 11,000 hourly candles. This is a large, bounded dataset that must be retrieved, stored locally, and processed in batch.
REST is the correct choice here for several reasons:
Bounded, complete datasets favor request-response semantics. When you request /v1/market/kline for a specific symbol and time range, you expect a complete, ordered response. WebSocket's streaming model—where the server pushes incremental updates—is conceptually misaligned with "give me all the data from January 2022 to December 2024."
HTTP caching infrastructure accelerates retrieval. Proxies, CDNs, and browser caches all understand HTTP semantics. If multiple clients in your organization request the same historical range, caching eliminates redundant network calls and server load.
Idempotent requests enable safe retries. A GET request for /v1/market/kline with identical parameters always returns the same result. If a network timeout occurs mid-response, you can safely reissue the request. WebSocket message delivery, while reliable at the transport layer, does not provide the same idempotent guarantees without application-level sequencing.
Debugging and testing simplify. You can copy a REST URL into a browser, share it with a colleague, or load it in Postman. WebSocket connections require a client library and maintain internal state that complicates isolated testing.
Real-Time Market Monitoring: WebSocket Earns Its Overhead
Now consider monitoring NVIDIA's price during the 30 minutes surrounding an earnings release. Price action is continuous. Liquidity fluctuates by the second. The bid-ask spread may widen from $0.02 to $0.15 within 5 seconds of the announcement.
REST is the wrong tool here. To maintain a "live" feed via polling, you would need to request /v1/market/kline/latest every 500 milliseconds—or faster. At that polling frequency:
- You generate approximately 172,800 requests per day per symbol.
- Each request incurs full TCP handshake + TLS negotiation overhead.
- Your server pays HTTP parsing costs on every response.
- You will frequently receive identical responses (no new data) while burning bandwidth and CPU.
WebSocket eliminates this inefficiency. After a single handshake, the server pushes every price update the instant it occurs. If no new trades occur for 3 seconds, zero bytes cross the wire. When a trade fires, you receive it within milliseconds of execution.
The contrast in resource consumption is stark:
| Metric | REST (500ms polling) | WebSocket (event-driven) |
|---|---|---|
| Requests per hour per symbol | 7,200 | ~60–600 (event-driven) |
| Average payload per request | Full candle JSON | Incremental delta |
| Network overhead | High (repeated headers) | Low (persistent tunnel) |
| Server CPU per symbol | High (HTTP parsing) | Low (direct socket write) |
| Data freshness | Up to 500ms stale | Near-instant |
For any scenario where time sensitivity matters—live trading signals, alert triggers, real-time visualization—WebSocket is the appropriate choice.
The Cost of Mismatches: What Happens When You Get It Wrong
Architectural mistakes compound over time. Using the wrong protocol for your use case creates debt that becomes increasingly expensive to service.
Using REST for Real-Time Feeds: The Polling Tax
Teams that implement live feeds via REST polling typically discover the problem in production, under load. The symptoms are recognizable:
- API rate limits hit during volatile periods (when you need data most).
- Latency spikes coincide with market open (when everyone polls simultaneously).
- Server costs balloon as the polling volume scales with symbol count and update frequency.
The root cause is often a desire for simplicity. REST is familiar. WebSocket feels like overengineering. But "simple" code that generates 100× the necessary network traffic is not simple—it is expensive.
Using WebSocket for Historical Retrieval: The Wrong Mental Model
Equally problematic is teams attempting to "stream" historical data over WebSocket for backtesting purposes. This approach fails because:
- WebSocket message ordering is not guaranteed at the application layer without sequencing metadata.
- Streamed data requires client-side buffering and reassembly before analysis.
- Reconnection logic during a long stream creates gaps that require application-level弥补 (gap-filling) logic.
- The debugging surface expands dramatically—you cannot replay a stream in a text editor.
If you are building a backtest, you want a complete, ordered, self-contained dataset. REST's request-response model maps directly to this requirement.
TickDB's Dual-Interface Design: Rationale and Trade-offs
TickDB exposes both REST and WebSocket endpoints deliberately. This is not feature bloat—it reflects the genuine duality of market data access patterns.
REST Endpoints in TickDB
TickDB's REST API (/v1/*) handles all historical and snapshot data retrieval:
GET /v1/market/kline— Historical OHLCV candles for backtestingGET /v1/market/kline/latest— Current incomplete candle (useful for dashboards)GET /v1/symbols/available— Enumerate tradable symbols
These endpoints share common characteristics:
- All use the
X-API-Keyheader for authentication - All return bounded, complete JSON responses
- All support pagination for large datasets
- All are idempotent (safe to retry)
WebSocket Interface in TickDB
TickDB's WebSocket interface handles all real-time streaming:
wss://api.tickdb.ai/v1/market/stream?api_key=...— Real-time price streamwss://api.tickdb.ai/v1/market/depth— Order book depth updates- Ping/pong heartbeat for keepalive
code: 3001rate limiting withRetry-Afterheader support
The WebSocket interface prioritizes low latency and push efficiency. The REST interface prioritizes completeness, debuggability, and cacheability.
Production-Grade Code: REST Implementation
When integrating TickDB's REST API for historical data, use environment-variable-based authentication and appropriate timeouts. The following example demonstrates a robust pattern for retrieving hourly candles:
import os
import requests
import time
from typing import Optional, Dict, Any, List
BASE_URL = "https://api.tickdb.ai/v1"
def get_api_key() -> str:
"""Retrieve API key from environment variable."""
api_key = os.environ.get("TICKDB_API_KEY")
if not api_key:
raise EnvironmentError(
"TICKDB_API_KEY not set. "
"Generate a key at https://tickdb.ai/dashboard"
)
return api_key
def handle_api_error(response_data: Dict[str, Any], symbol: Optional[str] = None) -> Any:
"""
Standard TickDB error handler.
Maps error codes to actionable exceptions.
"""
code = response_data.get("code", 0)
message = response_data.get("message", "Unknown error")
if code == 0:
# Success — return the data payload
return response_data.get("data")
error_map = {
(1001, 1002): (
"Invalid or missing API key. "
"Verify TICKDB_API_KEY is set correctly in your environment."
),
2002: f"Symbol '{symbol}' not found. Check /v1/symbols/available for valid symbols.",
}
if code == 3001:
# Rate limited — extract Retry-After and raise a retriable exception
retry_after = 5
raise RetryableError(
f"Rate limited (code 3001). Retry after {retry_after}s. " + message,
retry_after=retry_after
)
for codes, context in error_map.items():
if code in (codes if isinstance(codes, tuple) else (codes,)):
raise ValueError(f"{context} (API message: {message})")
raise RuntimeError(f"Unexpected API error {code}: {message}")
class RetryableError(Exception):
"""Exception indicating a retriable failure (e.g., rate limiting)."""
def __init__(self, message: str, retry_after: float):
super().__init__(message)
self.retry_after = retry_after
def fetch_historical_klines(
symbol: str,
interval: str = "1h",
start_time: Optional[int] = None,
end_time: Optional[int] = None,
limit: int = 1000,
max_retries: int = 3
) -> List[Dict[str, Any]]:
"""
Fetch historical OHLCV klines from TickDB REST API.
Handles pagination automatically for large date ranges.
Implements exponential backoff on rate limit errors.
Args:
symbol: Trading symbol (e.g., "NVDA.US")
interval: Candle interval (e.g., "1m", "1h", "1d")
start_time: Unix timestamp in milliseconds (optional)
end_time: Unix timestamp in milliseconds (optional)
limit: Maximum candles per request (max 1000)
max_retries: Number of retry attempts on rate limiting
Returns:
List of OHLCV candles sorted chronologically
"""
headers = {"X-API-Key": get_api_key()}
all_candles = []
current_end = end_time
while True:
params = {
"symbol": symbol,
"interval": interval,
"limit": limit,
}
if current_end:
params["end_time"] = current_end
if start_time:
params["start_time"] = start_time
response = requests.get(
f"{BASE_URL}/market/kline",
headers=headers,
params=params,
timeout=(3.05, 10) # (connect_timeout, read_timeout)
)
response.raise_for_status()
data = response.json()
try:
candles = handle_api_error(data, symbol=symbol)
except RetryableError as e:
if max_retries > 0:
print(f"Rate limited. Waiting {e.retry_after}s before retry...")
time.sleep(e.retry_after)
max_retries -= 1
continue
else:
raise
if not candles:
break
# Candles are returned newest-first; append in chronological order
all_candles.extend(reversed(candles))
if len(candles) < limit:
# Server indicates no more data available
break
# Set end_time to oldest candle timestamp for next page
current_end = candles[-1].get("open_time")
# Safety valve: prevent runaway loops
if not current_end:
break
return all_candles
# Example usage
if __name__ == "__main__":
# Fetch 6 months of NVIDIA hourly data
nvidia_candles = fetch_historical_klines(
symbol="NVDA.US",
interval="1h",
limit=1000
)
print(f"Retrieved {len(nvidia_candles)} candles")
Key engineering considerations in this implementation:
- Timeout enforcement:
(3.05, 10)sets a 3.05-second connection timeout and a 10-second read timeout. The specific 3.05 value prevents a known edge case in some HTTP libraries where sub-second precision causes premature timeout triggers. - Pagination loop: Historical data requests exceeding 1,000 candles require multiple requests. The loop uses
end_timepagination to fetch consecutive pages without gaps. - Exponential backoff on rate limits: When the API returns
code: 3001, the client waits and retries, decrementingmax_retriesto prevent infinite loops. - Environment-variable authentication: The API key is never hardcoded or logged.
Production-Grade Code: WebSocket Implementation
For real-time streaming, the WebSocket implementation requires persistent connection management, heartbeat handling, reconnection logic with exponential backoff and jitter, and rate-limit awareness.
import json
import os
import time
import threading
import random
from typing import Callable, Optional
from websocket import (
WebSocketApp,
WebSocketException,
WebSocketConnectionClosedException
)
class TickDBWebSocketClient:
"""
Production-grade WebSocket client for TickDB real-time data.
Features:
- Automatic reconnection with exponential backoff and jitter
- Ping/pong heartbeat management
- Rate-limit handling (3001 error codes)
- Thread-safe message dispatch
- Graceful shutdown
⚠️ For ultra-low-latency HFT workloads, consider a C++ or Rust
implementation with kernel-bypass networking.
"""
def __init__(
self,
api_key: Optional[str] = None,
on_message: Optional[Callable[[dict], None]] = None,
on_error: Optional[Callable[[str], None]] = None,
on_connect: Optional[Callable[[], None]] = None,
):
self.api_key = api_key or os.environ.get("TICKDB_API_KEY")
if not self.api_key:
raise EnvironmentError(
"TICKDB_API_KEY not set. "
"Generate a key at https://tickdb.ai/dashboard"
)
self.on_message = on_message
self.on_error = on_error
self.on_connect = on_connect
self._ws: Optional[WebSocketApp] = None
self._running = False
self._reconnect_delay = 1.0
self._max_reconnect_delay = 60.0
self._jitter_factor = 0.1
self._thread: Optional[threading.Thread] = None
self._last_pong_time: float = 0
self._last_sent_time: float = 0
def connect(self, url: str = "wss://api.tickdb.ai/v1/market/stream"):
"""
Establish WebSocket connection with automatic reconnection on failure.
"""
self._running = True
def on_open(ws: WebSocketApp):
print(f"[TickDB] Connection opened: {url}")
if self.on_connect:
self.on_connect()
# Send subscription message
ws.send(json.dumps({
"cmd": "subscribe",
"params": {
"symbols": ["NVDA.US", "AAPL.US"],
"channels": ["trades", "depth"]
}
}))
def on_message(ws: WebSocketApp, raw_message: str):
try:
message = json.loads(raw_message)
# Handle pong responses
if message.get("cmd") == "pong":
self._last_pong_time = time.time()
return
# Handle rate limit errors
if message.get("code") == 3001:
retry_after = float(
message.get("headers", {}).get("Retry-After", 5)
)
print(f"[TickDB] Rate limited. Waiting {retry_after}s...")
time.sleep(retry_after)
return
# Dispatch user message handler
if self.on_message:
self.on_message(message)
except json.JSONDecodeError as e:
print(f"[TickDB] Failed to decode message: {e}")
def on_error(ws: WebSocketApp, error):
error_msg = f"[TickDB] WebSocket error: {error}"
print(error_msg)
if self.on_error:
self.on_error(error_msg)
def on_close(ws: WebSocketApp, close_status_code: int, close_msg: str):
print(
f"[TickDB] Connection closed "
f"(code={close_status_code}, msg={close_msg})"
)
self._schedule_reconnect()
def on_pong(ws: WebSocketApp, raw_data: bytes):
self._last_pong_time = time.time()
self._ws = WebSocketApp(
f"{url}?api_key={self.api_key}",
on_open=on_open,
on_message=on_message,
on_error=on_error,
on_close=on_close,
on_pong=on_pong,
)
self._thread = threading.Thread(
target=self._ws.run_forever,
daemon=True,
name="TickDB-WebSocket-Thread"
)
self._thread.start()
# Start heartbeat thread
self._start_heartbeat()
def _start_heartbeat(self):
"""Send ping every 30 seconds to keep connection alive."""
def heartbeat_loop():
while self._running and self._ws:
time.sleep(30)
if self._running and self._ws:
try:
self._ws.send(json.dumps({"cmd": "ping"}))
self._last_sent_time = time.time()
except (WebSocketException, WebSocketConnectionClosedException):
break
threading.Thread(
target=heartbeat_loop,
daemon=True,
name="TickDB-Heartbeat-Thread"
).start()
def _schedule_reconnect(self):
"""Reconnect with exponential backoff and jitter."""
if not self._running:
return
# Calculate delay with exponential backoff and jitter
delay = self._reconnect_delay
jitter = random.uniform(0, delay * self._jitter_factor)
reconnect_time = delay + jitter
print(
f"[TickDB] Scheduling reconnect in {reconnect_time:.2f}s "
f"(attempt {self._reconnect_delay:.1f}s base)"
)
time.sleep(reconnect_time)
# Exponential backoff: double delay on each failure, capped at max
self._reconnect_delay = min(
self._reconnect_delay * 2,
self._max_reconnect_delay
)
# Reset delay on successful connection (call connect again)
self._reconnect_delay = 1.0
self.connect()
def send(self, message: dict):
"""Send a message through the WebSocket."""
if self._ws:
self._ws.send(json.dumps(message))
self._last_sent_time = time.time()
def close(self):
"""Gracefully close the WebSocket connection."""
print("[TickDB] Shutting down client...")
self._running = False
if self._ws:
self._ws.close()
if self._thread:
self._thread.join(timeout=5)
# Example usage
if __name__ == "__main__":
def handle_price_update(data: dict):
"""Process incoming real-time data."""
print(f"[TickDB] Received: {json.dumps(data)[:200]}")
client = TickDBWebSocketClient(on_message=handle_price_update)
try:
client.connect()
# Keep main thread alive
while True:
time.sleep(1)
except KeyboardInterrupt:
print("\n[TickDB] Interrupted by user")
finally:
client.close()
Critical engineering notes in this implementation:
- Exponential backoff with jitter: After a disconnection, the client doubles its wait time (1s → 2s → 4s → ... up to 60s). Jitter adds randomness to prevent "thundering herd" reconnection storms when many clients reconnect simultaneously after a server outage.
- Heartbeat mechanism: The client sends
{"cmd": "ping"}every 30 seconds and expects{"cmd": "pong"}responses. If pongs stop arriving, the connection is dead and should be rebuilt. - Thread safety: The WebSocket runs on a daemon thread, preventing the connection from blocking the main application.
- Rate-limit handling: When the server returns
code: 3001, the client reads theRetry-Afterheader and pauses before continuing, rather than hammering the server. - Graceful shutdown: The
close()method sets a flag to prevent reconnection attempts, then cleanly terminates threads.
Architectural Decision Framework
When designing a system that consumes TickDB data, apply this decision tree:
Is the data time-sensitive (affects execution within seconds)?
├── YES → Does the strategy require sub-second reaction?
│ ├── YES → Consider WebSocket + local order book reconstruction
│ │ ⚠️ Hardware and network proximity matter at this tier
│ └── NO → WebSocket with processing pipeline
└── NO → Is the data bounded (specific date range)?
├── YES → REST with pagination
└── NO → Periodic REST polling (e.g., daily position snapshots)
For most quantitative trading systems:
| Data type | Recommended protocol | Rationale |
|---|---|---|
| Historical backtesting | REST | Complete datasets, idempotent, cacheable |
| Live strategy signals | WebSocket | Low latency, push-based efficiency |
| Dashboard snapshots | REST (/kline/latest) |
Bounded, refresh-on-demand |
| Real-time P&L | WebSocket | Continuous updates, low staleness tolerance |
| End-of-day reconciliation | REST | Point-in-time snapshots, no streaming needed |
Anti-Patterns to Avoid
Polling Real-Time Data via REST
# ❌ Anti-pattern: Polling for "live" data
while True:
response = requests.get(
f"{BASE_URL}/market/kline/latest",
params={"symbol": "NVDA.US"},
headers={"X-API-Key": API_KEY},
timeout=(3.05, 10)
)
data = response.json()
process(data["data"])
time.sleep(0.5) # 500ms polling — burns rate limits during volatility
This approach generates unnecessary load, incurs latency from polling interval, and risks rate limiting precisely when you need data most.
WebSocket for Batch Processing
# ❌ Anti-pattern: Streaming historical data over WebSocket
ws.send({"cmd": "subscribe", "params": {"symbols": ["NVDA.US"]}})
# Then buffering thousands of messages client-side for batch analysis
This creates stateful, hard-to-debug streaming code where a simple REST call would return the same data as a complete, ordered JSON array.
Summary: The Right Tool for Each Job
REST and WebSocket are not competing standards—they are complementary protocols optimized for different communication patterns.
REST excels when you need a complete, bounded dataset and can tolerate request-response latency. Its stateless, cacheable, idempotent semantics align perfectly with historical data retrieval for backtesting and analysis.
WebSocket excels when data arrives continuously and latency matters. Its persistent connection model eliminates polling overhead and delivers updates the instant they occur.
The architecture of your trading system should reflect this duality. Use REST for your historical data pipeline and backtesting infrastructure. Use WebSocket for your live signal generation and real-time monitoring components. TickDB's dual-interface design is not an accident—it is a deliberate response to the genuine differences in how market data is consumed across the quantitative workflow.
Next Steps
If you're building a backtesting system, start with the TickDB REST API. The /v1/market/kline endpoint provides 10+ years of cleaned, aligned US equity OHLCV data with simple pagination. Sign up at tickdb.ai to get a free API key—no credit card required.
If you're monitoring real-time market conditions, integrate the WebSocket interface. The pattern shown above handles reconnection, rate limiting, and heartbeat automatically, leaving you free to focus on your signal logic.
If you need both, architect a hybrid system: REST for historical context and WebSocket for live updates. Keep a local order book state updated via WebSocket, and refresh it from REST on reconnect to handle gaps gracefully.
If you're using AI coding assistants, search for the tickdb-market-data SKILL on ClawHub to integrate TickDB data access directly into your development workflow.
This article does not constitute investment advice. Markets involve risk; past performance does not guarantee future results. API design patterns and code examples are provided for educational purposes.