Mining Block header bit reversing

Question

While trying to understand the mining process and checking in python..... i am puzzled with which info from/to pool are in Little Endian, which one needs reversing in building the block header + hashing and which one not. ??

For example ... Let's start with connecting to solo.ckpool.org response to mining.authorize gives me ..... job_id,prevhash,coinb1,coinb2,merkle_branch,version,nbits,ntime,clean_jobs =

{'id': None, 'method': 'mining.notify', 'params': ['59bc8dfc0000029f',
        '0eba48f47bc0ab4bb35b230849868bf1d79aeb19006eed460000000000000000',
        '01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff35039a6807000404d5bc5904fc873d140c',
        '0a636b706f6f6c112f736f6c6f2e636b706f6f6c2e6f72672fffffffff03c4626c4e000000001976a914fc6123f4bfd3a840b4387ab90e9801e98fb17cf888acb1caca00000000001976a914f4cbe6c6bb3a8535c963169c22963d3a20e7686988ac0000000000000000266a24aa21a9edab3b87f3963e55a23f3cea5e4435bf6637328fd5e329bb8fdac7d31c3a01d26200000000',
        ['c251d18568d2cdb07c376d5f3304a2161e3d09d289d1a7e53737199171d73ae6',
         '4f9bcac3ec00fc8238db04b7758a1b673bdf38b93e48159678b60b51517dd563',
         '2aadad2c9c8e6df5c71264524b090cdf89830f12949527f7872c243bc1dc552c',
         '8c5edf39c0d85c9853308eef3ce6f9b71c8d6584ca4120d411bc75987bb2f254',
         '48751c2ae0ddc05f5d504a3d6039e7d5afb3d5f2a6aaac93bd5822e5f8efc328',
         '3e9b5c6b986c2ddc09bf8604eb2bd2a3b49b46f11042c282331a206dadb4296e',
         'c65f502b11b325abc0fac847e8fc5cd300169160e7244b29dd5acc3d0e4b8605',
         'afe7d2287020c4291f609a772adbeafc952a8de0ced38a7346acb6c7103bec6d',
         '0caa545ae63de39d47baea100ea15a18061235d7e1a372fbca88edf9ca5ac797',
         'c5bfff15b4dd64e7faac98f1bba118fb11990e4ff146cb45c7f4af6b93e2ea03',
         'a8805201e8152c1f5835c9aa024752fb34aeb7fa55dd3af0ae6109918f328dbb'],
        '20000000',
        '1801310b',
        '59bcd504',
        True]}

Based on the pool response....I can build the coinbase just by

coinbase = coinb1 + extranonce1 + extranonce2 + coinb2 
coinbase_hash_bin = hashlib.sha256(hashlib.sha256(binascii.unhexlify(coinbase)).digest()).digest()

and same to include in merkle tree with hashing to get merkle_root. I am not sure if i need to reverse the bits here ?

But to build a block header ... should i do this..

header = version + prevhash + merkle_root + ntime + nbits + nonce

or this..

header = binascii.hexlify(binascii.unhexlify(version)[::-1]).decode('utf-8') + \
prevhash + \
merkle_root + \
binascii.hexlify(binascii.unhexlify(ntime)[::-1]).decode('utf-8') + \
binascii.hexlify(binascii.unhexlify(nbits)[::-1]).decode('utf-8') + \
binascii.hexlify(binascii.unhexlify(nonce)[::-1]).decode('utf-8')

blockheader:000000200eba48f47bc0ab4bb35b230849868bf1d79aeb19006eed460000000000000000fb68998fe1dd4cc22fa1225c2592c27df5331d40d9b33ffce99b26c0aead8ef404d5bc590b3101181e3eef8c

Can someone please post an example of a recent BLOCK HEADER ... i think i m doing something wrong !

I am comparing this hash

hash = hashlib.sha256(hashlib.sha256(binascii.unhexlify(header)).digest()).digest()[::-1]

with the target "000000000001310b000000000000000000000000000000000000000000000000"

Answer 1

Let's take a look at this block, because it only has one transaction (the coinbase): 000000000000000000eb2d0ed97a7b2cff7f1408417dca83908004beb6fd9b95

Let's grab the raw hex data:

0000002053f7ebc735f54ac8d4ebcc1eeb3d3bdea89603bdd27431000000000000000000ff964ec70ea5a5356a04fdb044ca6c59ffda2dd6be02c63d9ea58fb6d46e01991a22bb590b310118f6df26f801010000000001010000000000000000000000000000000000000000000000000000000000000000ffffffff3103b46707244d696e656420627920416e74506f6f6c6b2f4542312f4144362f4e59412f332059bb221ab8080000615c0300ffffffff02807c814a000000001976a914660371326d3a2e064c278b20107a65dad847e8a988ac0000000000000000266a24aa21a9ede2f61c3f71d1defd3fa999dfa36953755c690689799962b48bebd836974e8cf90120000000000000000000000000000000000000000000000000000000000000000000000000

There are 6 things which make up the block header:

1. First we have the version (4 bytes), 00000020, which is reversed bytes of 0x20000000.
2. Then we have the previous block hash (32 bytes), also in reversed byte order, 53f7ebc735f54ac8d4ebcc1eeb3d3bdea89603bdd27431000000000000000000 which is this block.
3. Next we have the merkle root of the transactions (32 bytes), in reverse byte order, ff964ec70ea5a5356a04fdb044ca6c59ffda2dd6be02c63d9ea58fb6d46e0199. Because there is only one transaction, the coinbase transaction, the merkle root is just equal to that transactions txid.
4. Then we have the time stamp (4 bytes), in reverse byte order, 1a22bb59 which is 0x59bb221a in hex or 1505436186 in decimal.
5. Then we have the reversed bytes of the 'bits' field (4 bytes), 0b310118 which is 0x1801310b
6. And then the nonce (4 bytes), f6df26f8 which is 0xf826dff6

Then after the header, we have the byte 01 which means there is one transaction, which follows. That one transaction is just the encoded coinbase transaction, as you can see here.

Of course, only the block header needs to be hashed, not the transactions too.

Answer 2

MeshCollider's answer is awesome. I want to add a note. Why do the previous block header hash and merkle root hash need to reverse bytes? And why other fields doesn't need it?

Actually, if you want any transaction's hash, you need to double-sha256 hash. So, the point is: if some data need sha256 digest, and digest stores in the computer is little-endian, some case need to transform to big int in big-endian, so reverse the bytes. And you get the double-sha256 hash in bitcoin. In some case, the big-endian is necessary. e.g. big-int(block-header-hash) < Mining Target?

If you need transaction hash, the result is big-endian too.

For example, a raw transaction hex is:

0100000001c997a5e56e104102fa209c6a852dd90660a20b2d9c352423edce25857fcd3704000000004847304402204e45e16932b8af514961a1d3a1a25fdf3f4f7732e9d624c6c61548ab5fb8cd410220181522ec8eca07de4860a4acdd12909d831cc56cbbac4622082221a8768d1d0901ffffffff0200ca9a3b00000000434104ae1a62fe09c5f51b13905f07f06b99a2f7159b2225f374cd378d71302fa28414e7aab37397f554a7df5f142c21c1b7303b8a0626f1baded5c72a704f7e6cd84cac00286bee0000000043410411db93e1dcdb8a016b49840f8c53bc1eb68a382e97b1482ecad7b148a6909a5cb2e0eaddfb84ccf9744464f82e160bfa9b8b64f9d4c03f999b8643f656b412a3ac00000000

then, you need transaction hash:

>>> import hashlib
>>> d = '0100000001c997a5e56e104102fa209c6a852dd90660a20b2d9c352423edce25857fcd3704000000004847304402204e45e16932b8af514961a1d3a1a25fdf3f4f7732e9d624c6c61548ab5fb8cd410220181522ec8eca07de4860a4acdd12909d831cc56cbbac4622082221a8768d1d0901ffffffff0200ca9a3b00000000434104ae1a62fe09c5f51b13905f07f06b99a2f7159b2225f374cd378d71302fa28414e7aab37397f554a7df5f142c21c1b7303b8a0626f1baded5c72a704f7e6cd84cac00286bee0000000043410411db93e1dcdb8a016b49840f8c53bc1eb68a382e97b1482ecad7b148a6909a5cb2e0eaddfb84ccf9744464f82e160bfa9b8b64f9d4c03f999b8643f656b412a3ac00000000'
>>> hashlib.sha256(bytes.fromhex(d)).hexdigest()
'240cf324ec3cf59609733e2a45e1408673306be8dcd4caf3067aa9355a0269e3'
>>> hashlib.sha256(bytes.fromhex('240cf324ec3cf59609733e2a45e1408673306be8dcd4caf3067aa9355a0269e3')).hexdigest()
'169e1e83e930853391bc6f35f605c6754cfead57cf8387639d3b4096c54f18f4'

169e1e83e930853391bc6f35f605c6754cfead57cf8387639d3b4096c54f18f4 is little-endian hash, in the bitcoin, all the double-sha256 hash is big-endian.

In fact, this transaction's identifier is big-endian hash: f4184fc596403b9d638783cf57adfe4c75c605f6356fbc91338530e9831e9e16.

When you need to find some info about this transaction, you need input the big-endian hash: https://www.blockchain.com/btc/tx/f4184fc596403b9d638783cf57adfe4c75c605f6356fbc91338530e9831e9e16

and little-endian hash you can find nothing.

Did you get it?

• previous block header hash need to reverse bytes, because bitcoin need big-endian.
• current block header hash need to reverse bytes, because bitcoin need big-endian.
• transaction hash need to reverse bytes, because bitcoin need big-endian.
• merkle root hash need to reverse bytes, because bitcoin need big-endian.

OMG! why? I found bitcoin core developer didn't know the reason.

Due to historical accident, the tx and block hashes that bitcoin core uses are byte-reversed. I'm not entirely sure why. May be something like using openssl bignum to store hashes or something like that, then printing them as a number.

ref: http://learnmeabitcoin.com/glossary/txid