How do I get old blocks while running a pruned node?

Question

Hello I'm trying to build a NodeJS app that fetches Bitcoin blocks, potentially old ones, and I'm running into issues. I've downloaded Bitcoin Core but I don't have enough space to run a full node and so I opted to run a pruned one. However, when I go to query an old block it tells me that the block isn't available in pruned mode. So my question is, why doesn't Bitcoin Core query the block on the fly from another node if it doesn't find it on disk? It clearly has the ability to do this, else how would it sync the blocks in the first place? If this is not possible with Bitcoin Core, could I build my own JS app that mimics the network calls used in the IBD to get blocks on demand, instead of saving them to disk? Also, I can't use a 3rd party block explorer API because they don't have all the features I want and they are less reliable.

Answer 1

This is a great question and something I've wondered myself. Since you mentioned JS, I implemented this on a branch of bcoin: https://github.com/pinheadmz/bcoin/commit/ec1f5c95bb76f86aa034ec5a288fc09dd15d9597

$ bcoin --prune --daemon

$ bcoin-cli rpc getblockchaininfo
{
  ... 
  "pruneheight": 10823
}

$ bcoin-cli block 1000
{
  "hash": "00000000c937983704a73af28acdec37b049d214adbda81d7e2a3dd146f6ed09",
  "height": 1000,
  "depth": 10112,
  ...

This is untested code and should not be used in production, but it does demonstrate a mechanism by which a pruned node (or maybe even SPV node or any application really) can use arbitrary getdata calls to retrieve full blocks from full nodes on the Bitcoin network.

See details about getdata in the protocol section of the wiki: https://en.bitcoin.it/wiki/Protocol_documentation#getdata

All you really need is a block's hash and a p2p connection to a full node.

In my implementation on bcoin, the headers are required to be downloaded which prevent the user from requesting a non-existent or invalid block. This implementation is dangerous because it breaks the model of the pruned node and does not attempt to validate the block when it is special-requested. It does not record the actual CoinView of the downloaded block and leaves the DB in a bad state that could cause issues if a big enough re-org happened that disconnected the block.

Disclaimers aside, what you're asking is definitely possible. You just need to be careful how you handle the data you retrieve from the network.

Also worth noting that requesting arbitrary blocks from peer is an important component of Neutrino light clients:

https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki