parse bitcoin addresses from transactions received through the P2P protocol using python

Question

I'm trying to extract public keys and transaction amounts from tx commands using python. Everyone on this stack exchange has been extremely helpful, but I think it's time to try and leverage a library to do this heavy lifting for me. Does anyone know any libraries where I can simply pass in the tx data without any parsing, so including info like magic number, and output details of the transaction?

Answer 1

I’m assuming that you are actually just interested in getting information on every transaction your node sees, rather than the actual p2p traffic.

If so, here is how I would approach your problem:

1. Run a Bitcoin Core full node. If you only care about unconfirmed transactions, consider making it a pruned node. If you want to collect data on all transactions, make it a non-pruning node and enable the transaction index with txindex=1.
2. Install PyZMQ
3. Enable ZeroMQ.
4. Listen to the hashtx ZMQ notifications
   hashtx notifies about all transactions, either when they are added to mempool or when a new block arrives, so you may get multiple notifications for the same txid.
5. For each hashtx notification you receive, check if the transaction is seen for the first time, then call bitcoin-cli getrawtransaction <txid>. In the response, read the value and address fields for each element of the vout vector. Optionally read n to get the output’s index. Note that some outputs don’t have addresses, e.g. P2PK and nulldata (OP_RETURN) outputs.
6. Optionally: Store the resulting outputs in a database using their "outpoint" (txid:n) as the key.

Note that some txs you see in your mempool may get replaced by a later transaction. If you want to make sure you only keep transactions that end up being part of the best chain, you may want to track transactions in blocks rather than transactions in the mempool, and be sure to handle blockchain reorganizations.

If that’s too slow for your purposes, you can get a whole block explorer of your own by running an instance of mempool.space or Esplora. Alternatively, you can probably pay for an API key to get this (public!) data from a number of different block explorers.

Note that you might need to smooth out some corners on this approach if you want use it, I have not tested this. There might also be easier more efficient ways, I never tried to solve this problem myself.