1

I am designing an exchange application where users can deposit funds into their account by sending BTC to a unique address.

What is the best way to monitor 100,000+ addresses for incoming transactions that meet a certain number of minimum confirmations?

One approach would be to listen to the rawtx event using ZMQ in order to discover new transactions, then write a program to periodically query the number of confirmations for each transaction using JSON-RPC. However, this method seems slow since it would involve repeatedly hammering the node with RPC calls. I do not wish to use external APIs like https://blockchain.info.

What would be the most efficient way to accomplish this?

1 Answer 1

1

Actually, there is no need to periodically query the node for the number of confirmations. The number of confirmations is implicitly given by the block height where the transaction was confirmed and the current block height.

The following assumes you store your 100,000+ addresses in a database with reasonably fast lookups, that it can also store additional data, and that you index columns that you use in queries.

Only confirmed transactions

The problem is quite simple if you only need to track transactions once they get their first confirmation. Listen for the rawblock ZMQ event, go through all transactions, and whenever a transaction sends to an address you're tracking, add this transaction to the database along with the block height where it was confirmed. Then, if you want perform some action after e.g. 3 confirmations, when a new block arrives, just look up transactions confirmed 3 blocks earlier. Depending on what it is you do, for extra robustness you might want to keep track of which confirmed transactions were already successfully acted upon.

Unconfirmed transactions

If you also want to show unconfirmed transactions, it gets more complicated as transactions might get replaced or dropped from your node's mempool. See this question on how to potentially deal with this. Whatever approach you use, the rest is the same as above.

Catching up

If your application has to restart or crashes, you might miss blocks. To guard against this, you can keep track of the last processed block height, and on startup query the node for the current block height. If you find out you're behind, just query the missing blocks (using the getblock RPC) and process them as usual.

Reorg safety

There are a few extra steps in order to make your application work correctly in the event of a chain reorganization. An approach I've found to work well is to store a few* recent block headers** (just the hash and height is enough), and when a new block arrives, follow its previous block hash (using the getblockheader RPC) until your database has the same block hash at that height as your node, then rollback any transactions made after that height. If no reorg has happened, the latest common header will just be one block behind the new chaintip, so you won't need to rollback anything.

* If you're not worried about a few tens of MB's of storage, you can just as well store all block headers.
** Storing block headers is also a convenient way of keeping track of what was the last block you processed.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.