How do I calculate the witness commitment hash for a given block?

Question

From my understanding, every SegWit enabled block has an added output script which is used as a place to store the "Witness commitment" specified in BIP 141.

One issue I've stumbled upon was calculating the commitment hash value only given the raw block/transaction data.

For example, I've chosen Block #542213 as it only has 4 total transactions (including the coinbase generation)

Coinbase witness commitment script:

OP_RETURN OP_PUSHDATA(36) [aa21a9ed4a657fcaa2149342376247e2e283a55a6b92dcc35d4d89e4ac7b74488cb63be2]

BIP141 states that the pushed data can be broken down to the following:

0xaa21a9ed - Commitment header (seemingly random)
0x4a657fcaa2149342376247e2e283a55a6b92dcc35d4d89e4ac7b74488cb63be2 - SHA256(SHA256(witness root hash|witness reserved value))

The documentation mentions that the witness root hash can be derived similarly to the merkle root in a block header, where each wtxid acts as the txid would in a traditional header merkle root.

To test this, I'm using this PHP script kindly provided on a different thread. This generates the merkle root used in the block header perfectly (that is not what I am trying to do, but what I have done to verify the code actually works).

Our example block has 4 transactions. Only two of which have witness data (coinbase and this tx). The commitment structure states that the wtxid of the coinbase transaction is always:

0x0000000000000000000000000000000000000000000000000000000000000000

and every other wtxid act as a leaf.

My first question:

• How are transactions that do not have witness data effected in the witness hash? Are they entirely omitted?

Assuming that the non-witness transactions are just omitted entirely from the merkle tree, I simply add the coinbase and the one transaction with witness data's wtxid into the array like so:

$txids = array(
    '0000000000000000000000000000000000000000000000000000000000000000',
    '6c6e3849acf1b570db352dc08f7776e99c344a56fbb2f019e1865d1b6e044889',
);

The result being: 3a84e2f2f1bcd42141b58f2acd9497296c9f2d710cc2164669ef6d7e3870dfe9

My second question:

• Where does the (concatenated?) witness reserved value come from?

Here it states:

The witness reserved value currently has no consensus meaning, but in the future allows new commitment values for future softforks

But it is still unclear to me what this value actually is before the hash is computed.

Please feel free to correct any inconsistencies or problems with my train of thought so I can improve my understanding for this part of segregated witness.

Answer 1

How are transactions that do not have witness data effected in the witness hash? Are they entirely omitted?

It will still include transactions that are not witness, it will just compute the normal hash to use as a leaf in the tree. See BlockWitnessMerkleRoot() in src/consensus/merkle.cpp and ComputeWitnessHash() in src/primitives/transaction.cpp:

uint256 CTransaction::ComputeWitnessHash() const
{
    if (!HasWitness()) {
        return hash;
    }
    return SerializeHash(*this, SER_GETHASH, 0);
}

Also, you are forgetting to reverse endianess, and hash the witness reserved value 0000000000000000000000000000000000000000000000000000000000000000 (from the last 32 bytes of the coinbase input witness). So step by step:

$ reverse_endian 66beaceb4be99da1e9824448231ab4fd37bacaee912381e779b37cf0e1dadad7
d7dadae1f07cb379e7812391eecaba37fdb41a23484482e9a19de94bebacbe66

$ reverse_endian aecb37e25954e15489e25548eb663ffdfd8a1362cac757ad62e9614453d2a577
77a5d2534461e962ad57c7ca62138afdfd3f66eb4855e28954e15459e237cbae

$ reverse_endian 5b211bc589cbdf5ad86cab1e2fe91f01c8ab934d21536b35864d30a3ff778456
568477ffa3304d86356b53214d93abc8011fe92f1eab6cd85adfcb89c51b215b

$ hash256 0000000000000000000000000000000000000000000000000000000000000000d7dadae1f07cb379e7812391eecaba37fdb41a23484482e9a19de94bebacbe66
d822d7717f284b89ea6a97874af567e9887a26a9f7db449ce02377d8fae4643f

$ hash256 77a5d2534461e962ad57c7ca62138afdfd3f66eb4855e28954e15459e237cbae568477ffa3304d86356b53214d93abc8011fe92f1eab6cd85adfcb89c51b215b
c182a158773ea7fc7a252fa725fc726de2a171088b40528ca00de0569fe53afe

$ hash256 d822d7717f284b89ea6a97874af567e9887a26a9f7db449ce02377d8fae4643fc182a158773ea7fc7a252fa725fc726de2a171088b40528ca00de0569fe53afe
30d002641e0eb81b11c3d14f4b120e80b4c06bcb3d34d90a5a606ca85e90c841

$ hash256 30d002641e0eb81b11c3d14f4b120e80b4c06bcb3d34d90a5a606ca85e90c8410000000000000000000000000000000000000000000000000000000000000000
4a657fcaa2149342376247e2e283a55a6b92dcc35d4d89e4ac7b74488cb63be2

Which matches the op_return data in the second coinbase output: RETURN PUSHDATA(36)[<aa21a9ed> <4a657fcaa2149342376247e2e283a55a6b92dcc35d4d89e4ac7b74488cb63be2>]

Where does the (concatenated?) witness reserved value come from?

I alluded to this before, but it is in the witness field of the coinbase input:

the coinbase's input's witness must consist of a single 32-byte array for the witness reserved value - BIP 141