Why is my raw transaction non-canonical, getting error -25, and being rejected?

Question

I'm trying to port TX generation logic to a different (and unpopular) language. I've been beating my head against this way too long and should have walked away but, well, you know, closure, and stubbornness.

Here's my code so far.

Blockchain.info, Electrum, and Bitcoin Core all decode my attempt at a TX into something reasonable looking, such as:

{
"txid" : "49c210ae472c5b5e39447e1f6d9bc020cd0f0075cc03d919afb0857a964e1f41",
"version" : 1,
"locktime" : 0,
"vin" : [
{
"txid" : "b097384c42a3be2730db3e3720a1806c76172b6b62b2b5ee007c2c6fd295cadf",
"vout" : 1,
"scriptSig" : {
"asm" :     "30460221009f478737296e39bbcff2ef7c6f013acc25bea75941acd8573bcea83ca910018b022100e9bff518297fec344d6e7e42cabc5ec793c4be507af68210bdde803a8a2958ed0104d8f39341451e2e66a00ef010c815f1284bc5e3a187476aab319b2d127b7e219c9b9e68bff311c63474242a9baab34f7ddec05de2c45bd140a74a64621ccb42cb",
"hex" :   "4930460221009f478737296e39bbcff2ef7c6f013acc25bea75941acd8573bcea83ca910018b022100e9bff518297fec344d6e7e42cabc5ec793c4be507af68210bdde803a8a2958ed014104d8f39341451e2e66a00ef010c815f1284bc5e3a187476aab319b2d127b7e219c9b9e68bff311c63474242a9baab34f7ddec05de2c45bd140a74a64621ccb42cb"
},
"sequence" : 4294967295
}
],
"vout" : [
{
"value" : 0.00439999,
"n" : 0,
"scriptPubKey" : {
"asm" : "OP_DUP OP_HASH160 7847eb9e366653aeb8857d541236fe4fd90c57e7 OP_EQUALVERIFY OP_CHECKSIG",
"hex" : "76a9147847eb9e366653aeb8857d541236fe4fd90c57e788ac",
"reqSigs" : 1,
"type" : "pubkeyhash",
"addresses" : [
"1BxzF8rgXtuiPuSN8azdMJgryzQSWt4Uoj"
]
}
}
]
}

Trying to sendrawtransaction in Bitcoin Core, I get "error -25". Reading through other StackExchange posts and Googling, the possible causes are listed as:

"Reading the source code, this error is returned when AcceptToMemoryPool fails, but not when it fails because the transaction is invalid. Does debug.log emit anything when this happens?"

"You get that obscure RPC error when your tx is using outputs that bitcoind never heard of." Nothing appears in ~/.bitcoin/debug.log.

Trying to send it with blockchain.info's slash pushtx service (oh, come on, is there seriously not a white list for sites I can link to?), I get "Script resulted in a non-true stack: []", which implies that I made a mistake in my signing logic somewhere.

I've walked through these:

• http://procbits.com/2013/08/27/generating-a-bitcoin-address-with-javascript

• http://www.righto.com/2014/02/bitcoins-hard-way-using-raw-bitcoin.html

• Redeeming a raw transaction step by step example required

And others, including various Ruby, Python, Java, and JavaScript implementations, but I need more than 10 reputation to post more than 2 links. I cannot for the life of me figure out where I'm deviating.

I know parts of those are out of date. The private key I'm starting with (from Electrum) is not flagged as compressed (33 bytes after DecodeBase58Check with the last being 0x01), so the publicKey in the transaction I'm trying to draw from shouldn't be compressed either.

The TX I'm trying to spend from (b097384c42a3be2730db3e3720a1806c76172b6b62b2b5ee007c2c6fd295cadf, second output, aka 1) with this TX has an output script which is Pay to Public Key Hash (d9495c762aed3dba15eec648beb55a8a43b8d1bd) according to blockchain info. The full scriptPubKey of the prevout is:

OP_DUP OP_HASH160 d9495c762aed3dba15eec648beb55a8a43b8d1bd OP_EQUALVERIFY OP_CHECKSIG

My private key, decoded from WIF, and run through ecdsa::pub_from_priv(), ecdsa::pub_encode(), matches that value.

It seems like getting the right stuff hashed before the hash is signed into the signature would be the hard part. I'm appending 0x01 to the signature after it comes out of ecdsa::Sign(). Before the TX is signed, it gets 01000000 appended to it. scriptSig (the Script in the input for the TX I'm building) is "OP_DUP, OP_HASH160, PUSHDATA, Bitcoin address (public key hash), OP_EQUALVERIFY, OP_CHECKSIG" before it is signed then it becomes a varstr of the sig and the pubKey, and that decodes to reasonable looking bytecode:

scriptSig:

0: OP_PUSHDATA 0x304602210094c538663c149f40929bb787d6174104a694181d063943a745e558b17d09e276022100b1812105ea6d7a8206c9019303a6459a9a2b2524b364debe69405b5d8b90c6c301
74: OP_PUSHDATA 0x04d8f39341451e2e66a00ef010c815f1284bc5e3a187476aab319b2d127b7e219c9b9e68bff311c63474242a9baab34f7ddec05de2c45bd140a74a64621ccb42cb

scriptPubKey:

0: OP_DUP
1: OP_HASH160
2: OP_PUSHDATA 0x7847eb9e366653aeb8857d541236fe4fd90c57e7
23: OP_EQUALVERIFY
24: OP_CHECKSIG

Using the old scriptPubKey as the scriptSig is approximated and assumes a simple pay-to-signature rather than actually pulling the exact scriptPubKey out of the TX being drawn on, but in this simple case, they do seem to match.

Here's the raw, (incorrectly?) signed TX:

0100000001dfca95d26f2c7c00eeb5b2626b2b17766c80a120373edb3027bea3424c3897b0010000008b48304502202647239b48610693967a24c7c976f0df903891113c56f50b6e3368c3f73eefb0022100e372ab8bf76ab35cba1ce6c1890fb6b27bb4d6d54f080180b46a6ecc2ae09248014104d8f39341451e2e66a00ef010c815f1284bc5e3a187476aab319b2d127b7e219c9b9e68bff311c63474242a9baab34f7ddec05de2c45bd140a74a64621ccb42cbffffffff01c0b60600000000001976a9147847eb9e366653aeb8857d541236fe4fd90c57e788ac00000000

Here's the code:

https://gist.github.com/scrottie/15f2fca963d164306dcb

PrivateKey upon request (there's $1 in there).

If anyone is able to dump that and figure out where I went wrong, I'd be ever so grateful.

Answer 1

%&@!

Fixed it with this change:

$privateKey = $privateKey->as_hex();  $privateKey =~ s{^0x}{} or die;
warn sprintf "doing: python sig.py  '%s' '%s'\n", $privateKey, to_hex($s256);
open my $python, '-|', 'python', 'sig.py', $privateKey, to_hex($s256) or die $!;
my $sig = readline $python;
warn "python says for sig: $sig\n";
$sig =~ s{[^0-9a-fA-F]}{}g;
$sig = from_hex($sig);

And adding this file as sig.py:

import ecdsa
import ecdsa.der
import ecdsa.util

import sys

privateKey = sys.argv[1]
s256 = sys.argv[2]

# print("privateKey len: ");  print(len(privateKey.decode('hex')))
# print(privateKey)

# print("s256 len: ");  print(len(s256.decode('hex')))
# print(s256)

sk = ecdsa.SigningKey.from_string(privateKey.decode('hex'), curve=ecdsa.SECP256k1)
sig = sk.sign_digest(s256.decode('hex'), sigencode=ecdsa.util.sigencode_der) + '\01' # 01 is hashtype
print sig.encode('hex')

I'm new here and I'm apparently trolling myself. I couldn't get the modules on CPAN that interface with openssl to work so I had one option for EC library, something written in pure Perl. That apparently has problems, even though it's unit tests pass. Or I was using it wrong.

That resulted in output that made https://blockchain.info/pushtx say "Transaction Submitted". And then coins moved.

Thanks to everyone who stopped to give this an eyeball. I owe you a beer. I'll try to play it forward with very, very good comments and docs.

Answer 2

Sorry, I do not use perl language and do not have perl environment installed to debug your code

Have a look to How to redeem a basic Tx?

I do not see step #13 where you have to append 01000000 = SIGHASH_ALL to the data you sign in your makeRawTransaction method