What is the most optimal way to parse a raw bitcoin block with python?

Question

Let's say one is connected to a zmq socket and listening for rawblock. How you'd parse the block header is as follows:

block_header = raw[:80]
version = block_header[:4]
prev_merkle_root = block_header[4:36]
merkle_root = block_header[36:68]
timestamp = block_header[68:72]
bits = block_header[72:76]
nonce = blockheader[76:80]

But, what is the best way to parse all the transactions in the block? Converting a rawtx to a tx object is easy, plenty of libraries do it. But what is the best way to parse a variable length block of txs?

Answer 1

We will compare modern libraries with segwit and bech32 support.

First will be pybtc from bitaps.com, second will be btcpy from chainside.net

Our test code:

import time
import pybtc

f = open('raw_block.txt')
fc = f.readline()
qt = time.time()
block_hex = fc[:-1]
bt = pybtc.Block(block_hex, format="decoded")
print("pybtc library parsed block in decoded format", time.time() - qt )

qt = time.time()
bt = pybtc.Block(block_hex, format="raw")
print("pybtc library parsed block in raw format", time.time() - qt )

from btcpy.structs.block import Block as btcpy_block

qt = time.time()
block = btcpy_block.unhexlify(block_hex)
print("btcpy library parsed block", time.time() - qt )

Raw block that used you can download here

Test results:

pybtc library parsed block in decoded format 0.3380300998687744
pybtc library parsed block in raw format 0.0953378677368164
btcpy library parsed block 0.6518878936767578

You can see that pybtc block parse performance is better. Decoded mode means that all bytes converted to hex and addresses to base58 or bech32. Raw mode faster, but all objects in raw bytes, addresses stored as hashes.

Answer 2

Check this library pybtc

Class to decode raw block https://github.com/bitaps-com/pybtc/blob/master/pybtc/block.py#L5