- Make sure they have stable servers and connections (Exchanges in Asia, accessed from, for example, Europe, have notoriously unstable connections). Otherwise you'll have pot holes all over your data, perhaps even rendering it useless.
- if you absolutely require the data, consider renting a server closer to the exchange server center's physical location.
- Make sure they have meaningful volume. If there's only 4 coins traded, it will likely not give you a good picture of the entirety of the coin's market.
- Test their customer support first. If you happen to require their assistance some time later during crawling, a great support staff can make all the difference. Besides, it gives you a small indication of the exchange infrastructure's quality.
If you're fine with, say, 1-Minute snapshots of data (i.e. tickers, order_books, trades etc), requesting data via a Representational State Transfer (REST) API. This allows you to send a http
request to a specified URL, and receive a response containing the data requested (usually in JSON
-format).
In addition to this (if you're thinking 'Well, hey! Why not fetch data every second then?'), market data is often cached. A snapshot of an API endpoint is stored for a set interval, before actually being updated on the server side. Hence, you might send a request every second, but nonetheless receive identical data for the cache interval.
In principle, you open a connection to the WS
API, and subscribe to the endpoints you want data on. Typically, this is separated by endpoint and pair, but make sure to consult the API
documentation - some exchanges do not employ a channel subscription
model, and data comes flooding right in without further configuring required.
The Financial Information eXchange
(FIX
) Protocol is a standard started in 1992, and now commonly used by institutions and brokers in the financial markets. It is by far the least available API
at exchanges, with even fewer actually sending market data. It is most commonly used to place or cancel orders (for example at GDAX). However, some exchanges offer real-time market data via FIX.