How do I maintain a continuously updated copy of a local mempool?

Question

Let's say I want to build a database of transactions that is continuously being updated from a locally running Bitcoin Core node using JSON-RPC and ZeroMQ. My concern is how to make it robust against chain reorganizations and other events where transactions might get dropped.

If I restrict myself only to confirmed transactions, it seems simple enough to just listen for new blocks, and in the event of a reorganization, rollback all transactions confirmed at a specific height and reapply the new chaintip.

However, it gets more complicated if I also want to include unconfirmed transactions. Then for every new transaction (whether in the mempool or in a block) I need to check if it conflicts with one or more unconfirmed transactions and remove those. Bitcoin Core already does this, but other than publishing the new transaction in a rawtx ZeroMQ event it gives no indication that an unconfirmed transaction was dropped.

So, is there a simple way to maintain a mirror of Bitcoin Core's mempool?

Some ideas:

• Call the getrawmempool RPC every 30 seconds or so to get the current state of the mempool. This has the advantage of being very simple, but it's inefficient and it means the mirror will lag behind.
• Maintain an index of all outputs that are spent by transactions currently in the mempool, and check every new transaction against it. However, I'm not sure this covers every case where a transaction needs to be dropped, and it certainly doesn't cover transactions dropped due to -maxmempool and -mempoolexpiry.

Answer 1

There are two methods to do this: ZMQ and Tracepoints.

The zmqpubsequence can be used to notifications for all mempool events. See the PR and test for more information. It was designed specifically for the mempool mirroring use case.

The other method is tracepoints, but this requires applying a currently draft PR. Tracepoints work similarly to ZMQ in that your program will use a tracepoints interface and wait for notifications.