Background Information

I am building an application that requires the ability to query any bitcoin full node (from a python script using raw tcp sockets) in order to read the OP_Return value listed in the following transaction (https://live.blockcypher.com/btc-testnet/tx/2599dbe540a583ede3512fef9a0f26be718c039ffd4d04d85ff3b339f40e73b1/)

What I have done to date

I have successfully managed to establish the version + verack, however I was faced with two issues.

  1. When I send a getdata for the necessary transaction Idit does not return the result. I have identified this is because my initial assumptions that getdata returns the necessary data was incorrect and instead it will only return transaction data for transactions still in mempool.
  2. This lead me to my main issue, I was hoping to replicate the initial block download on my python script, starting from the block containing the transaction I'm interested in. However when I do getblocks/getheaders no matter what I seem to put in the hash filter I get a dump of 500 (or 2000) blocks/headers. Below is a hex dump of the requests I am making (just the bitcoin protocol frame):

    • getblocks 0b110907676574626c6f636b00000000450000002a0af9950100000001000000000000592589e55cda6e8a093998e8356ea770d4aaeb7c0f5439b147d7000000000000017a09017d52db538d7a9ddcc48311866d7e5fdbbbec7d0faad5

    • getheaders 0b110907676574686561646572730000450000002a0af9950100000001000000000000592589e55cda6e8a093998e8356ea770d4aaeb7c0f5439b147d7000000000000017a09017d52db538d7a9ddcc48311866d7e5fdbbbec7d0faad5

I am looking for the following help:

  1. How can I query for transaction data without using bitcoin RPC if I can't use the getdata message?
  2. How do I get my block or headers response to only contain 1 inv item instead of 500/5000

EDIT (4/5/20) The implied requirement which wasn't made clear here was that we needed to do so in a decentralised way using the blockchain RPC api or equivalent.

  • You want to download 250GB of blocks from a remote peer in order to find a single output? – Anonymous Dec 14 '19 at 17:45
  • No I don't want to do that, but there doesn't seem like there are too many alternatives. I'm trying to pretend like I am a bitcoin full node trying to sync (but attempting to sync from the necessary block that I need, this way I would only be syncing 1 block worth of data) I just don't think I'm using the API properly – Dimitree Dec 14 '19 at 22:38
  • 1
    If your best choice is to do something absurd, maybe reconsider your strategy. The p2p network isn’t designed for arbitrary data access like you’re attempting. – Anonymous Dec 14 '19 at 22:41
  • I've gone through a few alternatives but the main goal here is to be able to look up the data in a decentralised censorship resistant manner. So I can't rely on looking at blockchain explorer apis. Also if I use bitcoin RPC I would need to be connecting to a specific bitcoin full node which could get taken down. – Dimitree Dec 15 '19 at 0:11
  • 2
    If you want something censorship resistant, your only choice is running your own node, and indexing the blocks in its data yourself, and then querying that. – Pieter Wuille Dec 16 '19 at 0:50

I don't see any endpoint to get the whole transaction in Blockcypher API documentation, so I used blockstream.info API:

            "scriptpubkey_asm":"OP_0 OP_PUSHBYTES_20 4ecdb2867150ff16c7d6e3258adca9ed8bac3ac1",
         "scriptpubkey_asm":"OP_RETURN OP_PUSHBYTES_14 3132372e302e302e313a38313831",
         "scriptpubkey_asm":"OP_HASH160 OP_PUSHBYTES_20 171f697fe358f7d7238b6128930bb1fa7363b44d OP_EQUAL",
         "scriptpubkey_asm":"OP_0 OP_PUSHBYTES_20 af8abd63a00c40ad75e47870b1b0a1ddb2ba7e66",

You can see here that the first output has the data is 3132372e302e302e313a38313831, which decoding to ASCII characters seems like a local URL:

hex = '3132372e302e302e313a38313831'
decoded = binascii.unhexlify(hex).decode('ascii')



To get the transaction that interests you without using any external API you will need to run a full bitcoin node with the txindex=1 option. If this transaction is tracked by your wallet, a pruned node would suffice. Then you can get the information about the transaction with this command:

>bitcoin-cli getrawtransaction 2599dbe540a583ede3512fef9a0f26be718c039ffd4d04d85ff3b339f40e73b1 true

You should get something very similar to the answer from blockstream's API I pasted above

| improve this answer | |
  • sorry if it wasn't clear, but I needed to do this in a decentralised way without relying on stored data from an API server (for example if they just respond with the incorrect data). I'm trying to read this data straight off the blockchain and can't find any way to do this – Dimitree Dec 16 '19 at 0:14
  • Check the edit in the final part of my answer, it explains how to do that without an external API – Pedro Dec 16 '19 at 0:25

The bitcoin p2p protocol does not directly permit transaction lookups this way. Maybe some bloom filtering would minimize the footprint, but it's still relatively complex to implement.

Electrum servers are maybe the next best thing - they do support the lookups you want, but they're less decentralized, and if your application is doing naughty things they could collude and manipulate only your results.

How do I get my block or headers response to only contain 1 inv item instead of 500/5000

I don't think you can limit the headers results (you have to know all the headers to do so) But once you have the header chain, you can send a block locator with start:HASH(n) end:HASH(n+1) and you can download a single block.

Re bloom filtering, if you find nodes with the BLOOM_SERVICE bit you can tell them your filter, and the block data only contains your txs and the false positives allowed by your filter.

| improve this answer | |

It turns out the main issue with the above code was that the the endianness in python was defaulting to my hex payload being mixed up. I simply needed to correct the ordering of the payload. The correct code can be viewed here: https://github.com/dummytree/blockchain-botnet-poc/blob/master/MalwareManager.py#L284-L299

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.