Skip to main content
minor formatting change.
Source Link
deepbrook
  • 1.7k
  • 1
  • 16
  • 23

Feel free to add, correct or update any of this content in the comments below! Thanks.

Feel free to add, correct or update any of this content in the comments below! Thanks.

Updated text for clarity, replaced GDAX link with ConbasePro link
Source Link
deepbrook
  • 1.7k
  • 1
  • 16
  • 23
  • Make sure they have stable servers and connections (Exchanges in Asia, accessed from, for example, Europe, have notoriously unstable connections). Otherwise you'll have pot holes all over your data, perhaps even rendering it useless.
    • if you absolutely require the data, consider renting a server closer to the exchange server center's physical location.
  • Make sure they have meaningful volume. If there's only 4 coins tradedlittle volume changing owners over the course of a set time frame (usually, it willthe 24h volume metric is always available), the exchange is likely not give you a very good picture ofchoice. It skew the entiretypicture of the coin's market.
  • Test their customer support first. If you happen to require their assistance some time later during crawling, a great support staff can make all the difference. Besides, it gives you a small indication of the exchange infrastructure's quality.

If you're fine with, say, 1-Minute snapshots of data (i.e. tickers, order_books, trades etc), requesting data via a Representational State Transfer (REST) API is sufficient. This allows you to send a http request to a specified URL, and receive a response containing the data requested (usually in JSON-format).

In addition to this (if you're thinking 'Well, hey! Why not fetch data every second then?'), market data is often cached. A snapshot of an API endpoint is stored for a set interval, before actually being updated on the server side. Hence, you might send a request every second, but nonetheless receive identical data foruntil the server cache intervalis refreshed.

In principle, you open a connection to the WS API, and subscribe to the endpoints you want data on. Typically, this is separated by endpoint and pair, but make sure to consult the API documentation - some exchanges do not employ a channel subscription model, and data comes flooding right in without further configuring requiredconfiguration.

The Financial Information eXchange (FIX) Protocol is a standard started in 1992, and now commonly used by institutions and brokers in the financial markets. It is by far the least available API at exchanges, with even fewer actually sending market data. It is most commonly used to place or cancel orders (for example at GDAXCoinbasePro). However, some exchanges offer real-time market data via FIX.

  • Make sure they have stable servers and connections (Exchanges in Asia, accessed from, for example, Europe, have notoriously unstable connections). Otherwise you'll have pot holes all over your data, perhaps even rendering it useless.
    • if you absolutely require the data, consider renting a server closer to the exchange server center's physical location.
  • Make sure they have meaningful volume. If there's only 4 coins traded, it will likely not give you a good picture of the entirety of the coin's market.
  • Test their customer support first. If you happen to require their assistance some time later during crawling, a great support staff can make all the difference. Besides, it gives you a small indication of the exchange infrastructure's quality.

If you're fine with, say, 1-Minute snapshots of data (i.e. tickers, order_books, trades etc), requesting data via a Representational State Transfer (REST) API. This allows you to send a http request to a specified URL, and receive a response containing the data requested (usually in JSON-format).

In addition to this (if you're thinking 'Well, hey! Why not fetch data every second then?'), market data is often cached. A snapshot of an API endpoint is stored for a set interval, before actually being updated on the server side. Hence, you might send a request every second, but nonetheless receive identical data for the cache interval.

In principle, you open a connection to the WS API, and subscribe to the endpoints you want data on. Typically, this is separated by endpoint and pair, but make sure to consult the API documentation - some exchanges do not employ a channel subscription model, and data comes flooding right in without further configuring required.

The Financial Information eXchange (FIX) Protocol is a standard started in 1992, and now commonly used by institutions and brokers in the financial markets. It is by far the least available API at exchanges, with even fewer actually sending market data. It is most commonly used to place or cancel orders (for example at GDAX). However, some exchanges offer real-time market data via FIX.

  • Make sure they have stable servers and connections (Exchanges in Asia, accessed from, for example, Europe, have notoriously unstable connections). Otherwise you'll have pot holes all over your data, perhaps even rendering it useless.
    • if you absolutely require the data, consider renting a server closer to the exchange server center's physical location.
  • Make sure they have meaningful volume. If there's only little volume changing owners over the course of a set time frame (usually, the 24h volume metric is always available), the exchange is likely not a very good choice. It skew the picture of the market.
  • Test their customer support first. If you happen to require their assistance some time later during crawling, a great support staff can make all the difference. Besides, it gives you a small indication of the exchange infrastructure's quality.

If you're fine with, say, 1-Minute snapshots of data (i.e. tickers, order_books, trades etc), requesting data via a Representational State Transfer (REST) API is sufficient. This allows you to send a http request to a specified URL, and receive a response containing the data requested (usually in JSON-format).

In addition to this (if you're thinking 'Well, hey! Why not fetch data every second then?'), market data is often cached. A snapshot of an API endpoint is stored for a set interval, before actually being updated on the server side. Hence, you might send a request every second, but nonetheless receive identical data until the server cache is refreshed.

In principle, you open a connection to the WS API, and subscribe to the endpoints you want data on. Typically, this is separated by endpoint and pair, but make sure to consult the API documentation - some exchanges do not employ a channel subscription model, and data comes flooding right in without further configuration.

The Financial Information eXchange (FIX) Protocol is a standard started in 1992, and now commonly used by institutions and brokers in the financial markets. It is by far the least available API at exchanges, with even fewer actually sending market data. It is most commonly used to place or cancel orders (for example at CoinbasePro). However, some exchanges offer real-time market data via FIX.

Added crypto archive link in data sources.
Source Link
deepbrook
  • 1.7k
  • 1
  • 16
  • 23

Feel free to add, correct or update any of this content in the comments below! Thanks.

Feel free to add, correct or update any of this content in the comments below! Thanks.

added 75 characters in body
Source Link
deepbrook
  • 1.7k
  • 1
  • 16
  • 23
Loading
added 68 characters in body
Source Link
deepbrook
  • 1.7k
  • 1
  • 16
  • 23
Loading
added more links
Source Link
deepbrook
  • 1.7k
  • 1
  • 16
  • 23
Loading
Source Link
deepbrook
  • 1.7k
  • 1
  • 16
  • 23
Loading