4

I'm looking for specifics of Step15-17 from Redeeming a raw Tx Step By Step, which is essentially the step where the concatenated raw Tx structure is double sha256 hashed, and then signed with an ECDSA library. I've also referred to other raw transactions guides; I'm looking specifically how to sign a raw hex Tx structure without a GUI/website/etc

I've constructed a testnet Tx using Bitcoincore v0.10 (a full node) and attempted the same with Python and cannot correctly sign the raw Tx structure using Python ecdsa.

I'll show the Bitcoincore commands first:

importprivkey 93FxXUeMJp93YQAtGeW5cE23gFN4sJbBr1RBmerLFVUDuqQqKL5
mu858WTEPiWWpAJRTMxC4ka6DJqiaCZiSB
createrawtransaction '[{"txid" : "72b764383b99fb3d112ac8b474a5d7c4242b75dbfee2d4e9cf9a6703d90f805a", "vout" : 1}]' '{"n1hjyVvYQPQtejJcANd5ZJM5rmxHCCgWL7" : 0.990}'
signrawtransaction 01000000015a800fd903679acfe9d4e2fedb752b24c4d7a574b4c82a113dfb993b3864b7720000000000ffffffff01c09ee605000000001976a914dd6cce9f255a8cc17bda8ba0373df8e861cb866e88ac00000000

Final bitcoincore hex Tx (using signrawtransaction): 01000000015a800fd903679acfe9d4e2fedb752b24c4d7a574b4c82a113dfb993b3864b772010000008a47304402204d78d2e6c0f801573e4960fb8e51ad939380d119d25f97d15efdedf815b05f02022066bd2ab0b401e32e7ce67ea45f8224097eeafbef2335d563776e5efe6632732d01410479b22a5127d176a49d506c86f791031f94a389227ef46a8ddb725a88c944c37e3f753de6ee0b441a0237801f140810e111a1fd8276a2a5d0ee07224a1b551cc1ffffffff01c09ee605000000001976a914dd6cce9f255a8cc17bda8ba0373df8e861cb866e88ac00000000

Now the Python 2.7.9 code follows (FYI, using iPython 2.0 so there's boilerplate missing in the following code) :

# please excuse the hexlify/unhexlify, I work better in strings than bytes
import __future__
from pybitcointools import sha256
import hashlib, ecdsa
from ecdsa import SigningKey, SECP256k1
addr, pubkey = 'mu858WTEPiWWpAJRTMxC4ka6DJqiaCZiSB', '0479b22a5127d176a49d506c86f791031f94a389227ef46a8ddb725a88c944c37e3f753de6ee0b441a0237801f140810e111a1fd8276a2a5d0ee07224a1b551cc1'
rawtx = '01000000015a800fd903679acfe9d4e2fedb752b24c4d7a574b4c82a113dfb993b3864b7720100000000ffffffff01c09ee605000000001976a914dd6cce9f255a8cc17bda8ba0373df8e861cb866e88ac00000000' # TX structure with 00 as scriptSig, from bitcoincore createrawtransaction
# replacing the '00' scriptSig value
unsigned.append(rawtx[:82])  
unsigned.append('19'+'76a914953de657be4b305f606d9a9fbd35b070a682475788ac') # scriptSig = scriptPubKey input
unsigned.append(rawtx[84:]) 
unsigned.append('01000000')  # appending sighash_all
unsigned = ''.join(unsigned)

unsigned = '01000000015a800fd903679acfe9d4e2fedb752b24c4d7a574b4c82a113dfb993b3864b772010000001976a914953de657be4b305f606d9a9fbd35b070a682475788acffffffff01c09ee605000000001976a914dd6cce9f255a8cc17bda8ba0373df8e861cb866e88ac0000000001000000'

# See https://github.com/warner/python-ecdsa/blob/master/README.md
sig1 = sk.sign(sha256(sha256(unsigned)), hashfunc=hashlib.sha256, sigencode=ecdsa.util.sigencode_der) # note this references 2 different sha256 functions
sig2 = [hex(len(sig1 + '\x01'))[2:], hexlify(sig1), hexlify('\x01'), hex(len(unhexlify(pubkey)))[2:], pubkey]
sig2
>>> ['48',  '304502200e98d54ad642488121fa1fd4d055ff8f5b40773a21ecb42b5ef44ed9fd3b103c022100a2a56ffc3f27021a4be5d1d7a3016eff08fe7f550a4a894cca845ceab54dc53b', '01', '41', '0479b22a5127d176a49d506c86f791031f94a389227ef46a8ddb725a88c944c37e3f753de6ee0b441a0237801f140810e111a1fd8276a2a5d0ee07224a1b551cc1']
sig2 = len(''.join(map(unhexlify, sig2)))
hex(len(''.join(map(unhexlify, sig2))))[2:] 
>>> '8b'  # push 139 bytes
sig3 
>>> 8b48304502200e98d54ad642488121fa1fd4d055ff8f5b40773a21ecb42b5ef44ed9fd3b103c022100a2a56ffc3f27021a4be5d1d7a3016eff08fe7f550a4a894cca845ceab54dc53b01410479b22a5127d176a49d506c86f791031f94a389227ef46a8ddb725a88c944c37e3f753de6ee0b441a0237801f140810e111a1fd8276a2a5d0ee07224a1b551cc1 
signed = unsigned[:82] + sig3 + unsigned[84:]  # inserting the signature into the 00 the core software uses in place of ScriptSig

Final (invalid) Python Tx:

01000000015a800fd903679acfe9d4e2fedb752b24c4d7a574b4c82a113dfb993b3864b772010000008b48304502200e98d54ad642488121fa1fd4d055ff8f5b40773a21ecb42b5ef44ed9fd3b103c022100a2a56ffc3f27021a4be5d1d7a3016eff08fe7f550a4a894cca845ceab54dc53b01410479b22a5127d176a49d506c86f791031f94a389227ef46a8ddb725a88c944c37e3f753de6ee0b441a0237801f140810e111a1fd8276a2a5d0ee07224a1b551cc1ffffffff01c09ee605000000001976a914dd6cce9f255a8cc17bda8ba0373df8e861cb866e88ac00000000

Sending the Python generated Tx does not work. If I sendrawtransaction <PYTHON HEX> in Bitcoincore it returns an invalid signature error (Specifically, 16: mandatory-script-verify-flag-failed (Script evaluated without error but finished with a false/empty top stack element) (code -26)).

(TL;DR) QUESTION: Where am I going wrong in signing the raw transaction structure manually?

NB: I am aware pybitcointools and related libraries can sign Txs, but I am looking for specifics, as even these two very thorough Bitcoin SE resources (LINK1, LINK2) brush past the details. I'd prefer an answer for Python ECDSA or PyCrypto, however OpenSSL I suppose is the next best answer.

  • 1
    I've debugged this for the last two hours or so but haven't been able to find any errors. Sorry, I can't find anything wrong, and every time I try to submit a transaction it has error code -25. I've even verified that the hash being signed is the same one that the daemon signs through print lines in my compiled daemon. You can see my debugging file here, in case it helps you. – morsecoder Mar 15 '15 at 19:06
  • I see now that that transaction output has already been spent. Does this mean you successfully found your error? If so, you should post it! – morsecoder Mar 15 '15 at 20:37
  • @StephenM347 Yes! I'll add the code. The problem was the code for ECDSA – Wizard Of Ozzie Mar 16 '15 at 1:32
  • @StephenM347 See the answer below but the issue seemed to arise from using the public key in signing and a disparity between string/bytes/DER input – Wizard Of Ozzie Mar 16 '15 at 7:46
6

OK, I figured out how to sign the raw Tx using Python ecdsa. I'll step through it:

Recall:

createrawtransaction '[{"txid" : "72b764383b99fb3d112ac8b474a5d7c4242b75dbfee2d4e9cf9a6703d90f805a", "vout" : 1}]' '{"n2kx7k6JuA5Wy27fawaeiPX7dq8mbRDPAv" : 0.99}' = 01000000015a800fd903679acfe9d4e2fedb752b24c4d7a574b4c82a113dfb993b3864b7720100000000ffffffff01c09ee605000000001976a914e900510876cb689f1db6fa982376c301362b740c88ac00000000

And then adding in the scriptPubKey for scriptSig's 00:

unsigned = '01000000015a800fd903679acfe9d4e2fedb752b24c4d7a574b4c82a113dfb993b3864b772010000001976a914953de657be4b305f606d9a9fbd35b070a682475788acffffffff01c09ee605000000001976a914dd6cce9f255a8cc17bda8ba0373df8e861cb866e88ac0000000001000000'

Note unsigned has the 01000000 appended because the SIGNHASH is 01 (which will also be appended to the DER signature in a minute)

So, we double SHA256 bytes and return txhash as bytes: txhash = hashlib.sha256(hashlib.sha256(unsigned.decode('hex')).digest()).digest()

We also need the privkey for mu858WTEPiWWpAJRTMxC4ka6DJqiaCZiSB (in secret exponent form, ie hex): privkey = 'dc57c6d067376c36bbed632c9d00f03767867f337d5a86b5b0308a60004f08ee'.zfill(64) privkey = 'dc57c6d067376c36bbed632c9d00f03767867f337d5a86b5b0308a60004f08ee'

Now, we sign using this corrected code:

signingkey = ecdsa.SigningKey.from_string(privkey.decode('hex'), curve=ecdsa.SECP256k1)
SIG = signingkey.sign_digest(txhash, sigencode=ecdsa.util.sigencode_der) +'01'.decode('hex')

This is working code. From here on in, we're simply making the signature:

ScriptSig = <varint of total sig length> <SIG from code, including appended 01 SIGNHASH> <length of pubkey (0x21 or 0x41)> <pubkey>

where pubkey = '0479b22a5127d176a49d506c86f791031f94a389227ef46a8ddb725a88c944c37e3f753de6ee0b441a0237801f140810e111a1fd8276a2a5d0ee07224a1b551cc1

Giving us a valid signed Tx without using bitcoincore:

01000000015a800fd903679acfe9d4e2fedb752b24c4d7a574b4c82a113dfb993b3864b772010000008a47304402204d78d2e6c0f801573e4960fb8e51ad939380d119d25f97d15efdedf815b05f02022066bd2ab0b401e32e7ce67ea45f8224097eeafbef2335d563776e5efe6632732d01410479b22a5127d176a49d506c86f791031f94a389227ef46a8ddb725a88c944c37e3f753de6ee0b441a0237801f140810e111a1fd8276a2a5d0ee07224a1b551cc1ffffffff01c09ee605000000001976a914dd6cce9f255a8cc17bda8ba0373df8e861cb866e88ac00000000

  • Thank you very much!!! Took a bit of figuring out but your tips helped a lot. – ethought Mar 23 '15 at 18:23
  • That ecdsa signing code often returns 'Non-canonical signature: High S Value' that are now non-standard. Any ideas of how can this be solved? – Nathan Parker Dec 12 '15 at 8:34
  • 2
    @NathanParker SIG is <r><s> concatenated together. We need to check s < N/2 where N is the curve order, 115792089237316195423570985008687907852837564279074904382605163141518161494337. If s>N/2, then s = N-s – Wizard Of Ozzie Dec 13 '15 at 8:56
  • 3
    @NathanParker to use Python-ECDSA, replace sigencode=ecdsa.util.sigencode_der with sigencode=ecdsa.util.sigencode_der_canonize – Wizard Of Ozzie Dec 13 '15 at 9:43
  • I would have never guessed it was so incredibly easy. Thanks! – Nathan Parker Dec 14 '15 at 10:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.