1

I recently started running bitcoind on a local Debian host, and I am still learning how to use the information in the local copy of the bitcoin blockchain.

I would like to extract all the information for one or more public bitcoin addresses currently available in the full node locally 1. EDIT 2: I want to do this both from the bash command line and programmatically. (IOW: no GUI-based tools, please.)

I recently learned that, contrary to my initial expectation, bitcoin core is not suitable for performing this sort of query.

Therefore, I went back to Google, but I've come up empty 3. Hence I'm posting my question here.

Specifically, I would like to know

  1. what software (preferably in the form of one or more Debian packages) would I need to install on the local full node that would enable me to extract all the information for one or more public bitcoin addresses currently available in the full node locally;
  2. how exactly would I use this software to perform such queries.

1 I want to do this without reference to any specific wallet. Having to add this requirement feels extremely weird to me (like having to add the requirement that answers to my question should work seven days a week). This is a clear sign to me that there is still at least one pretty gigantic gap and/or confusion in my understanding of bitcoin, because I don't understand why a wallet should be even relevant to the kind of query I want to do. I would have thought that a wallet would be, at most, an optional piece of information to modify the query, not an essential input for it. But the fact is that I have not been able to find any way to interrogate the full node's database for specific addresses that does not require specifying a wallet. This is just mind-boggling to me.

2 I apologize for not mentioning this critical requirement in my original post. That was a major oversight on my part!

3 I am beginning to realize that the queries I want to perform on a full node's database (and without reference to a wallet) is something practically no one cares to do, and therefore it is hard to find tools for doing them. Again, this strikes me as a bit weird, and feeds my growing suspicion that I am totally missing the point about bitcoin, blockchains, full nodes, etc.

Improve this question

2 Answers 2

Reset to default
1

Why this is hard

Bitcoin Core is chiefly a tool to synchronize with the current state of the network and to enforce the consensus rules with a self-owned piece of hardware. Beside being on the same page regarding the rules, this requires the node only to keep track of the ledger, the Unspent Transaction Output (UTXO) set. Bitcoin transactions need to be processed only once to that end.
The blockchain being now over 330 GiB, some people are actually choosing to run pruned nodes (that discard old blockchain data after processing it to calculate the UTXO set). Keeping additional indexes around with dozens of GiB in size is a special interest most users don't seem to care for.
In fact, Bitcoin Core does not feature a full address index out of the box. Originally maybe, because address reuse is considered a bad practice, later because other projects offer this, leaving Bitcoin Core developers feeling that they could focus on other things.

How you do it anyway

Bitcoin Core has an option to build a full transaction index -txindex (turning it on will require you to start over with synchronization and take about 30 GiB more space). This index is needed if you want to look up arbitrary transactions from the past. Other projects leverage this functionality and build additional indexing capability around it to offer Lightning Network support or Bitcoin explorers. I would recommend that you look into the Mempool Open Source Project or the Esplora Block Explorer. Either of these will allow you to run your own instance of a block explorer with APIs to look up blockchain data on basis of addresses. Note that the Esplora index will take about 1 TiB and apparently takes quite some time to build (H/T @Anonymous).

Improve this answer
9
  • Thank you! FWIW, I did have txindex=1 in my bitcoin.conf at the time I did the IBD, so I guess I should be good to go? (BTW, your answer explains something that had been puzzling me: my first attempt at IBD failed for lack of space, even though 350GB had been available when I started running bitcoind. Now I realize that the txindex=1 in my bitcoin.conf came with a much bigger space requirement than that.
    – kjo
    Mar 27, 2021 at 15:29
  • 1
    Yeah, I think the transaction index adds another 200 GiB or so. If you had it on already, you'd be good to go indeed.
    – Murch ♦
    Mar 27, 2021 at 15:52
  • After looking a bit into mempool and esplora, which appear to be GUI/browser-based tools, I realized that I should have spelled out that I want to run queries from the command line and/or programmatically. I am sorry about this omission! (I keep forgetting that those like me, who consider graphical interfaces basically useless for any real work, make up a minuscule minority.) I have now edited my post to make this requirement clear. (I fear, however, that, given that what I want to do appears to be difficult, this additional requirement may be a deal-breaker...)
    – kjo
    Mar 27, 2021 at 15:58
  • Oh, I just found github.com/Blockstream/esplora/blob/master/API.md, which you alluded to in your original reply, but I missed in my initial look-over. (I still have not been able to find much documentation for mempool, though.)
    – kjo
    Mar 27, 2021 at 16:08
  • OK, I guess this is it for mempool: mempool.space/api
    – kjo
    Mar 27, 2021 at 16:15
0

I have written a program that allows you to search for any address regardless if it belongs to your wallet or not, in Python3.

Here is the github link:

https://github.com/ORP967/Bitcoin_Core_RPC_par_address

Let me know what you think and if you have any improvements.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.