Is there any format change in block #498557

Question

I wrote some python code to parse the block data, all the result are fine, until block01095.dat, block #498557.

The code to deal with Transaction is:

class Tx: 
  def __init__(self, blockchain):
    start_pos = blockchain.tell()
    self.version = uint4(blockchain)
    check_pos = blockchain.tell()
    # For segwit
    # https://en.bitcoin.it/wiki/Transaction
    dummy1 = uint1(blockchain)
    dummy2 = uint1(blockchain)
    if dummy1 != 0 or dummy2 != 1:
      blockchain.seek(check_pos)
    self.inCount = varint(blockchain)
    self.inputs = []
    self.seq = 1
    for i in range(0, self.inCount):
      self.inputs.append(TxInput(blockchain, i))
    self.outCount = varint(blockchain)
    self.outputs = []
    if self.outCount > 0:
      for i in range(0, self.outCount):
        self.outputs.append(TxOutput(blockchain, i))
    # For segwit
    if dummy1 == 0 and dummy2 == 1:
      for i in range(0, self.inCount): 
        num_op = uint1(blockchain)
        for n in range(0, num_op):
          op_code=uint1(blockchain)
          _ = hashStr(blockchain.read(op_code))
    self.lockTime = uint4(blockchain)
    cur_pos = blockchain.tell()
    blockchain.seek(start_pos)
    self.raw_bytes = blockchain.read(cur_pos - start_pos)
    self.tx_hash = hash_tx(self.raw_bytes)
    print(self.tx_hash)

Full code is in: https://github.com/keviny/bitcoin-block-parser/blob/master/block.py

The last transaction in #498556 is https://www.blockchain.com/btc/tx/0baa42c9852b4edd9437e8c3fd38bd0dd89d977fe80ac7648961cf8ff33a2dbf which is ok to parse.

But in #498557 I can never create the right tx hash anymore. The first tx hash I get from block #498557 is https://www.blockchain.com/btc/tx/2da60422e1de3e8e64742bb16db5d1e3b2a9813f754cff416e549d570b2ad0e8 which is not a valid tx hash in blockchain.info.

Actually, the first three hash I got are: 2da60422e1de3e8e64742bb16db5d1e3b2a9813f754cff416e549d570b2ad0e8 e0a7e07ccb9b126bd20b5b134d41ec5f8b70bdb5408a263619ca3b8b3a90512d 6c695c567d14bce39a8e58359a2a5d26dc2bcbd9a83f043b5dbc1800c56f84c8 None of above can be used in blockchain.info.

I suspect there is some format change from this block.

Does Anyone here know any background about this issue?

Answer 1

From what I can tell, by testing your script, is that it is failing on Segwit transactions. This would make sense, if you haven't accounted for it, because it changes what is used to produce the tx hash. This was introduced with BIP141. Currently, not all transactions are required to be in this format, so you will have to account for both. The txid still uses the entire transaction structure, this is why you might notice bitcoin-core rpc calls will return a different txid than the tx hash (wtxid) for segwit transactions.

Definition of txid remains unchanged: the double SHA256 of the traditional serialization format:

[nVersion][txins][txouts][nLockTime]

A new wtxid is defined: the double SHA256 of the new serialization with witness data:

[nVersion][marker][flag][txins][txouts][witness][nLockTime]

see BIP141

What you'll need to do is look for the segwit flag 0x01 in the serialized transaction. It's an optional 2-byte array right after the first version bytes, see https://en.bitcoin.it/wiki/Transaction