3

Let's say I want to build a database of transactions that is continuously being updated from a locally running Bitcoin Core node using JSON-RPC and ZeroMQ. My concern is how to make it robust against chain reorganizations and other events where transactions might get dropped.

If I restrict myself only to confirmed transactions, it seems simple enough to just listen for new blocks, and in the event of a reorganization, rollback all transactions confirmed at a specific height and reapply the new chaintip.

However, it gets more complicated if I also want to include unconfirmed transactions. Then for every new transaction (whether in the mempool or in a block) I need to check if it conflicts with one or more unconfirmed transactions and remove those. Bitcoin Core already does this, but other than publishing the new transaction in a rawtx ZeroMQ event it gives no indication that an unconfirmed transaction was dropped.

So, is there a simple way to maintain a mirror of Bitcoin Core's mempool?

Some ideas:

  • Call the getrawmempool RPC every 30 seconds or so to get the current state of the mempool. This has the advantage of being very simple, but it's inefficient and it means the mirror will lag behind.
  • Maintain an index of all outputs that are spent by transactions currently in the mempool, and check every new transaction against it. However, I'm not sure this covers every case where a transaction needs to be dropped, and it certainly doesn't cover transactions dropped due to -maxmempool and -mempoolexpiry.

1 Answer 1

3

There are two methods to do this: ZMQ and Tracepoints.

The zmqpubsequence can be used to notifications for all mempool events. See the PR and test for more information. It was designed specifically for the mempool mirroring use case.

The other method is tracepoints, but this requires applying a currently draft PR. Tracepoints work similarly to ZMQ in that your program will use a tracepoints interface and wait for notifications.

3
  • Isn't R printed as a notification when a transaction is removed from mempool if using sequenece? github.com/bitcoin/bitcoin/blob/…
    – user133407
    Jul 28, 2022 at 23:06
  • @1440000bytes Good point, forgot that zmqpubsequence was added. I've updated the answer.
    – Ava Chow
    Jul 29, 2022 at 19:35
  • I you want a project example which keeps a continuous updated copy of a mempool you can check github.com/mempoolexplorer/mempoolexplorer
    – dev7ba
    Feb 16, 2023 at 11:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.