Stratum protocol - problem with implementation in python

Question

For a couple of days I'm trying to write simple script in python to communicate with stratum pool server. I'm using simplified code of NightMiner (https://github.com/ricmoo/nightminer/) and some testnet I've foud ( stratum+tcp://pool.bitcoincloud.net:4008), but I can't get the proper result. Maybe someone more experienced could help me. Below I show each step of my script (step, no python source code).

1. connect to the server, authorize, get the job data

After connecting to the server I send this requests:

b'{"id": 1, "method": "mining.subscribe", "params": []}\n'
b'{"params": ["testuser", "anything"], "id": 2, "method": "mining.authorize"}\n'

and get the following response:

{"id":1,"result":[[["mining.set_difficulty","deadbeefcafebabea200000000000000"],["mining.notify","deadbeefcafebabea200000000000000"]],"40000004",4],"error":null}
{"id":null,"method":"mining.set_difficulty","params":[0.01]}
{"id":null,"method":"mining.notify","params":["4ca","4128bf630f57387b00e25b419d1bb77e667e4036a7d8fee80000015600000000","01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff2503a77614048c8a145e08","122f626974636f696e636c6f75642e6e65742f0000000002062c9c04000000001976a91423e020eacd64acfe093150331d44fdbcc0c7ce0688acc2eb0b00000000001976a91400bf6d61c2a34df5a9ea338fcad188c31bb4a52388ac00000000",[],"20000000","1a02b098","5e148a8c",true]}

2. get the coinbase and merkle root

With this data and extranounce='00000000' I can build the coinbase (in byte form, but I will show as hex in this example):

coinbase=coinb1+extranounce1+extranounce2+coinb2
coinbase=b'01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff2503a77614048c8a145e084000000400000000122f626974636f696e636c6f75642e6e65742f0000000002062c9c04000000001976a91423e020eacd64acfe093150331d44fdbcc0c7ce0688acc2eb0b00000000001976a91400bf6d61c2a34df5a9ea338fcad188c31bb4a52388ac00000000'

Next thing to do is double hash the coinbase with SHA256 algo, which gives as the following bytes:

b'9b9c9ab0b1c92844e4fed3f895f9443fefcda5ae02cc5a8ad4444f9303081a23'

There is no merkle branches (merkle branches array is empty), so hash above is our merkle root.

3. building the header

With merkle root and data received in step 1, I can start building a header:

header= version + prevhash + merkle root + ntime + nbits + nonce

I use the values as follows:

version - received '20000000', after little-endian conversion it is '00000020'
prevhash - received '4128bf630f57387b00e25b419d1bb77e667e4036a7d8fee80000015600000000', after conversion it is '63bf28417b38570f415be2007eb71b9d36407e66e8fed8a75601000000000000' (reverse bytes order every 4 bytes)
merkle root - get '9b9c9ab0b1c92844e4fed3f895f9443fefcda5ae02cc5a8ad4444f9303081a23' in step 2, stays that way
ntime - received '5e148a8c', after conversion it is '8c8a145e'
nbits - received '1a02b098', after conversion it is '98b0021a'

So the header is

b'0000002063bf28417b38570f415be2007eb71b9d36407e66e8fed8a756010000000000009b9c9ab0b1c92844e4fed3f895f9443fefcda5ae02cc5a8ad4444f9303081a238c8a145e98b0021a' + nonce

4. search nonce

the difficulty is 0.01, so my target is

00000063ff9c0000000000000000000000000000000000000000000000000000

after some calculations, I find this 4 bytes to use it as a nonce:

b'd0cf1040'

so my complete header is now

b'0000002063bf28417b38570f415be2007eb71b9d36407e66e8fed8a756010000000000009b9c9ab0b1c92844e4fed3f895f9443fefcda5ae02cc5a8ad4444f9303081a238c8a145e98b0021ad0cf1040'

after double-hashing it with SHA256 algo and reverse bytes order I get this hash:

000000032f201567fc4d02fa468bf2dc6a5c21ae70098c124e99ee7d2520ca26

which is below my target, so 'd0cf1040' should be the correct nonce.

5. submit nounce

last thing to do is submitting the results, so I send this request to the server:

b'{"id":2, "method":"mining.submit","params":["testuser","4ca","00000000","5e148a8c","d0cf1040"]}\n'

and this is response I get:

{"id":2,"result":false,"error":null}

I have no idea why I get result=false. I can only suppose there is a mistake in step 3 (building header) and this mistake is related to byte-order of some values, but I can't find this mistake (I have tried many variants of byte-order, but never get the response result=true).

I will be much obliged if anyone can check this calculations and give me some tips.

Answer 1

There is (at least) one mistake:

There is no merkle branches (merkle branches array is empty), so hash above is our merkle root

The merkle array for that specific job is not empty. It contains the following branches:

["12ae2555f80b865bac89fa3aeb4df41b730422670fb80d1a0536276b481fab53","e1f7983af4815cb2116f16c6af916f0a139ad5a7588fd029572c6e403f9fed5d","6d5861e558d057993fdae3007ff5b49807d6f43b218e1ef85ed81223747278f1","847e7a24328302a2a8dbdad177d44d5a20916a0ba36edf2dc667c01a09da298c"]

Then the proper way of calculating merkle root would be required.

Answer 2

Ok, I get it done.

There were two kind of errors I have found (not to mention the byte-order during submitting result - thank you @Charlie for that tip):

1. the result (false) I got as the last response from the pool server was not response to 'mining.submit' method

In my example communication log (question) I paste (among others) those requests:

b'{"params": ["testuser", "anything"], "id": 2, "method": "mining.authorize"}\n'

and

b'{"id":2, "method":"mining.submit","params":["testuser","4ca","00000000","5e148a8c","d0cf1040"]}\n'

You can see I have used the same "id" value (2) for both of this requests. So when I got the response:

{"id":2,"result":false,"error":null}

I have assumed this is "false" response to my 'mining.submit' request. But later I use individual id for each request and find out that this "false" result is response to 'mining.authorize' method. So my first problem was the authorization problem, not the nonce calculation/submitting.

I don't have any idea why I get response to 'mining.authorize' request after some other request, but this is some improvement ;)

2. wrong data submitting in 'mining.authorize' method

In my example communication log (question) I paste this request:

b'{"params": ["testuser", "anything"], "id": 2, "method": "mining.authorize"}\n'

But according to https://pool.bitcoincloud.net/ I should use as username ' your address (testnet address on test pools)'. So I changed this request to:

b'{"params": ["2N16oE62ZjAPup985dFBQYAuy5zpDraH7Hk", "anything"], "id": 2, "method": "mining.authorize"}\n'

Now it works, after submitting the result ('mining.submit' method) i got this response:

b'{"id":3,"result":true,"error":null}\n'

The only thing I need to figure out is why I get the response to 'mining.authorize' request as a response for some other request (the next one after 'mining.authorize' request). Any help would be appreciated.