How do you perform double-SHA-256 encoding?

Question

The Bitcoin Protocol-specification gives an example of double-SHA-256 encoding.

hello
2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824 (first round of sha-256)
9595c9df90075148eb06860365df33584b75bff782a510c6cd4883a419833d50 (second round of sha-256)

I've tried various SHA256 calculators and the first encoding matches no problem, but the second always resolves to

d7914fe546b684688bb95f4f888a92dfc680603a75f23eb823658031fff766d9

I've also tried UPPERCASE and swapping the byte endianness.

Answer 1

You're hashing the hexadecimal representation of the first hash. You need to hash the actual hash -- the binary data that the hex represents.

Try this:

$ echo -n hello |openssl dgst -sha256 -binary |openssl dgst -sha256

Answer 2

You want to work with the digests, not the hex strings.

Here's some Ruby:

require 'digest'
d = Digest::SHA2.new 256
d2 = Digest::SHA2.new 256
d << 'hello'
d.to_s
d2 << d.digest
d2.to_s

This will be the output from irb:

1.9.3p194 :001 > require 'digest'
 => true 
1.9.3p194 :003 >   d = Digest::SHA2.new 256
 => #<Digest::SHA2:256 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855> 
1.9.3p194 :004 > d2 = Digest::SHA2.new 256
 => #<Digest::SHA2:256 e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855> 
1.9.3p194 :005 > d << 'hello'
 => #<Digest::SHA2:256 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824> 
1.9.3p194 :006 > d.to_s
 => "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824" 
1.9.3p194 :007 > d2 << d.digest
 => #<Digest::SHA2:256 9595c9df90075148eb06860365df33584b75bff782a510c6cd4883a419833d50> 
1.9.3p194 :008 > d2.to_s
 => "9595c9df90075148eb06860365df33584b75bff782a510c6cd4883a419833d50"

Here's the same thing in Python:

import hashlib
d = hashlib.sha256(b"hello")
d2 = hashlib.sha256()
d.hexdigest()
d2.update(d.digest())
d2.hexdigest()

And the output from within a Python shell:

>>> d = hashlib.sha256(b"hello")
>>> d2 = hashlib.sha256()
>>> d.hexdigest()
'2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824'
>>> d2.update(d.digest())
>>> d2.hexdigest()
'9595c9df90075148eb06860365df33584b75bff782a510c6cd4883a419833d50'

Answer 3

For the second round of sha256, you need to hash the raw binary output from the first round, not the textual version.

A sha256 hash is 256 bits, or 32 bytes. Thus for the second round you should be hashing a piece of data that's 32 bytes. When hashing a hexadecimal string as the literal input for the second round, your data is 64 bytes.

Try a hashing tool that can interpret hexadecimal input. For example see here, copy/paste the hash to the input field and check the 'Hex' checkbox.

Answer 4

For easy reference, lets name the various inputs and outputs from your question like so:

pre-image 1:                     hello
                                   |
                                   v
hash operation 1:                SHA256
                                   |
                                   v
hash output 1 (aka pre-image 2): 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824
                                   |
                                   v
hash operation 2:                SHA256
                                   |
                                   v
hash output 2:                   9595c9df90075148eb06860365df33584b75bff782a510c6cd4883a419833d50

Pre-image 1 is binary (ASCII) data. We know this because the range of hexadecimal characters is 0-9 and a-f, and there are characters beyond f in pre-image 1. So hash operation 1 must be performed using a binary input.

By definition, a SHA256 hash operation always produces a 256 bit integer. We typically represent this in hexadecimal format, however it could just as correctly be written in decimal format and nothing would change:

hex:     2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824
decimal: 20329878786436204988385760252021328656300425018755239228739303522659023427620

(Decimal conversion using Wolfram Alpha)

Since we know that pre-image 2 is a hexadecimal number, then hash operation 2 must be performed upon a number, rather than upon binary data.

In Javascript, using the Stanford Javascript Crypto Library, the two hashes would be done like so:

var preImage1 = 'hello';
var hashOutput1 = sjcl.hash.sha256.hash(preImage1);
console.log('hash output 1: ' + sjcl.codec.hex.fromBits(hashOutput1));
var preImage2 = hashOutput1;
var hashOutput2 = sjcl.hash.sha256.hash(preImage2);
console.log('hash output 2: ' + sjcl.codec.hex.fromBits(hashOutput2));

The SJCL is clever enough to figure out the pre-image formats and perform the hash correctly in each case.

You can also do these hash operations live on my blog, here: https://analysis.null.place/how-do-the-bitcoin-mining-algorithms-work/#form10

Note how the hash output changes when you click the pre-image is hexadecimal checkbox (this only works when hexadecimal characters are present in the input).

Answer 5

Hope this helps.....

Had to make slight modifications to the ParseHex() provided by Andrew Chow....

Headers needed are:

#include <string>
#include <cstring>
#include <assert.h> // assert() is used

Sha256 implementation in use is the one provided by Zedwood !!

First is the ParseHex function provided by Andrew Chow

std::string ParseHex(std::string& s)
{
    assert(s.size() % 2 == 0);
    static const std::size_t symbol_count = 256;
    static const unsigned char hex_to_bin[symbol_count] = 
    {
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0x00 - 0x07
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0x08 - 0x0F
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0x10 - 0x17
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0x18 - 0x1F
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0x20 - 0x27
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0x28 - 0x2F
        0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, // 0x30 - 0x37
        0x08, 0x09, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0x38 - 0x3F
        0x00, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F, 0x00, // 0x40 - 0x47
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0x48 - 0x4F
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0x50 - 0x57
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0x58 - 0x5F
        0x00, 0x0A, 0x0B, 0x0C, 0x0D, 0x0E, 0x0F, 0x00, // 0x60 - 0x67
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0x68 - 0x6F
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0x70 - 0x77
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0x78 - 0x7F
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0x80 - 0x87
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0x88 - 0x8F
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0x90 - 0x97
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0x98 - 0x9F
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0xA0 - 0xA7
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0xA8 - 0xAF
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0xB0 - 0xB7
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0xB8 - 0xBF
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0xC0 - 0xC7
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0xC8 - 0xCF
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0xD0 - 0xD7
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0xD8 - 0xDF
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0xE0 - 0xE7
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0xE8 - 0xEF
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 0xF0 - 0xF7
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00  // 0xF8 - 0xFF
    };

    std::string out;
    auto itr = s.begin();
    while (itr != s.end())
    {
       unsigned char b = static_cast<unsigned char>(hex_to_bin[*(itr++)] << 4);
       b |= static_cast<unsigned char>(hex_to_bin[*(itr++)]     );
       out.push_back(b);
    }
    return out;
}

Then next is the modified sha256 function call.

std::string sha256(std::string input)
{
    string kk;
    int size = input.length();
    
    unsigned char header_data[size], hash[32];// header_data is "160" in size because midstate optimization is not implemented
    char buf[2*32+1];
    buf[2*32] = 0;

    input = ParseHex(input); // ParseHex() is UNDEFINED !!!
    init(); // Initiialize
    update((unsigned char*)input.c_str(), input.length()); // Update
    final(hash); // Finalize

    // Uncomment code below if you want to directly double hash input data using a single "sha256(input)" function call
    // init();
    // update(hash, 32);
    // final(hash);

    for (int i = 0; i < 32; i++) 
        sprintf(buf+i*2, "%02x", hash[i]); // feed data from hash into buf and format data

    return std::string(buf); // Return buf as string instead of char
}

• THESE FUNCTIONS NEED TO BE PASTED INSIDE THE SHA256 HEADER FILE
• YOU'D HAVE TO REVERSE ENDIANESS OF HASH RESULT.
• ALSO, THE "assert()" THROWS AN ERROR FOR UNEVEN LENGTHED INPUTS.. SO "abc" WON'T WORK.

Hope this helps you if you ever need this....

Cheers!!

Answer 6

Late to the party, but here is a Node.js implementation using the built-in crypto module:

const crypto = require('crypto');
/**
 *
 * @param {Buffer} data
 * @returns {Buffer}
 */
function doubleSHA256(data) {
  return crypto.createHash('sha256').update(crypto.createHash('sha256').update(data).digest()).digest();
}

Answer 7

It looks like you have coded your algo to always assume that your input msg always contains data in ASCII display format. That would explain why your first algo produced the correct hash table. For run2, you can have your input msg data in any format you like, as long as you have some way of telling your algo what to expect. I am going to assume that when you wrote your algo, that you coded it to always expect ASCII display format in the input msg file. So, if you didn't switch your first output block table to ASCII display format, then that would explain why you didn't get the expected final block table from your second run. Grab a list of "The IBM PC Character Set" and you'll see what I mean. eg if you're providing an 'a' and it's really a hexadecimal 'a' as opposed to a ASCII display format 'a', then you've GOT to let your algo know what you are doing - it can't second guess that you've switched formats without telling it. So similarly, you may have put the hexadecimal hash or even the binary hash table, from run1 output into run2 input, which is fine, BUT you have let your algo know that it should read it as what ever format you picked, to wack into the run2 input file. There are 2 ways to fix this IMO. (1) convert your output hash block table from run1, to ASCII display chars and wack them into the run2 input file. I wouldn't do this because you get some wierd and wonderful ASCII display chars that may do wierd things. (2) Re-code your algo, so that you can tell it what data will be coming at it, from the run2 input file. That's a more flexible way to do things, and it opens up a lot more options in terms of using any base you like, even your own custom ones, but YMMV. A lot of water has passed under the bridge since your problem was published, but this may help others.