Redeeming a raw transaction step by step example required

Question

I am looking, specifically, for a step by step example transaction of sending Bitcoin which uses an actual transaction (How To Redeem A Basic Txn, from a few years back, does a great job of outlining most steps of sending a raw txn but does not use a real TxID).

I have played around with both the Python pybitcointools library, the SX library and JSON output from the Bitcoin Core client (Bitcoin-QT, Bitcoin-cli, Bitcoin daemon) but have yet to find an actual step by step guide using raw transaction output (single input / single output Bitcoin txn, ie not multisig).

Hopefully the bounty will bring a step by step example transaction (with private keys for the sending address) will bring an answer showing how it's done and, specifically:

• how the ScriptPubKey element fits into the raw txn
• how the ScriptSig element fits into the raw txn
• how the txn is signed (with DER encoding).

EDIT: I think the best resource is Ken Shirriff's Bitcoins the hard way: Using the raw Bitcoin protocol, but again, there's no single source online that answers my question without skimming past areas like scriptPubKey, signing etc.

When I activate the bounty, if you can answer this refer to this Tx as it can serve as the actual example (ie I'll provide ~$1 in BTC and private keys for 1From/1MBngSqZbMydscpzSoehjP8kznMaHAzh9y if interested)

EDIT 2: The RoyalFork Blog: Deconstructing Txns provides an unbelievably good reference for interactive Txn creation

Answer 1

Step-by-step description:

We start creating a new transaction which we hash and sign.

1. Add four-byte version field: 01000000
2. One-byte varint specifying the number of inputs: 01
3. 32-byte hash of the transaction from which we want to redeem an output (reverse order): be66e10da854e7aea9338c1f91cd489768d1d6d7189f586d7a3613f2a24d5396
4. Four-byte field denoting the output index we want to redeem from the transaction with the above hash (counting from zero): 00000000
5. Now comes the scriptSig. For the purpose of signing the transaction, this is temporarily filled with the scriptPubKey of the output we want to redeem. First we write a one-byte varint which denotes the length of the scriptSig (0x19 = 25 bytes): 19
6. Then we write the actual scriptSig (which is the scriptPubKey of the output we want to redeem): 76 a9 14 dd6cce9f255a8cc17bda8ba0373df8e861cb866e 88 ac (look to the bottom line line on https://blockchain.info/tx/96534da2f213367a6d589f18d7d6d1689748cd911f8c33a9aee754a80de166be?show_adv=true )
7. Then we write a four-byte field denoting the sequence. This is currently always set to 0xffffffff: ffffffff
8. Next comes a one-byte varint containing the number of outputs in our new transaction. We will set this to 1 in this example: 01
9. We then write an 8-byte field (64 bit integer, little-endian) containing the amount we want to redeem from the specified output. I will set this to the total amount available in the output minus a fee of 0.0001 BTC (128307 - 10000): 23ce010000000000
10. Then we start writing our transaction's output. We start with a one-byte varint denoting the length of the output script (0x19 or 25 bytes): 19
11. Then the actual output script: 76 a9 14 a2fd2e039a86dbcf0e1a664729e09e8007f89510 88 ac ( this is transferring funds back to address 1FromKBPAS8MWsk1Yv1Yiu8rJbjfVioBHc )
12. Then we write the four-byte "lock time" field: 00000000
13. And at last, we write a four-byte "hash code type" (1 in our case): 01000000

OK, the result is

01000000
01
be66e10da854e7aea9338c1f91cd489768d1d6d7189f586d7a3613f2a24d5396
00000000
19 76 a9 14 dd6cce9f255a8cc17bda8ba0373df8e861cb866e 88 ac
ffffffff
01
23ce010000000000
19 76 a9 14 a2fd2e039a86dbcf0e1a664729e09e8007f89510 88 ac
00000000
01000000

14. Now we double-SHA256 hash this entire structure, which yields the hash 1cde0239b55717cca8003104abc2ec2673d4f6fabea0b74351940e382e88486f

15. Now we should create ECDSA signature... 1MBngSqZbMydscpzSoehjP8kznMaHAzh9y is a brainwallet of "mrbubbymrbubbymrbubby!", which just happens to encode an address starting in 'MB' (making linking the 2 quite easy; see @WizardOfAussie comment below for phrase origins). Private key in WIF: 5HvofFG7K1e2aeWESm5pbCzRHtCSiZNbfLYXBvxyA57DhKHV4U3

In hex, the private key is 0ecd20654c2e2be708495853e8da35c664247040c00bd10b9b13e5e86e6a808d. There is a sign (key, digest) method in every crypto lib. It will return an array of bytes. This array will be no more than 72 bytes, and start with hex code 30. Lets imagine that the signature is 3046022100cf4d7571dd47a4d47f5cb767d54d6702530a3555726b27b6ac56117f5e7808fe0221008cbb42233bb04d7f28a715cf7c938e238afde90207e9d103dd9018e12cb7180e To this signature we append the one-byte hash code type: 01. The public key for 1MBngSqZbMydscpzSoehjP8kznMaHAzh9y is: 042daa93315eebbe2cb9b5c3505df4c6fb6caca8b756786098567550d4820c09db988fe9997d049d687292f815ccd6e7fb5c1b1a91137999818d17c73d0f80aef9

16. We construct the final scriptSig by concatenating: <One-byte script OPCODE containing the length of the DER-encoded signature plus the one-byte hash code type>|< The actual DER-encoded signature plus the one-byte hash code type>|< One-byte script OPCODE containing the length of the public key>|<The actual public key>

scriptSig will be

49 3046022100cf4d7571dd47a4d47f5cb767d54d6702530a3555726b27b6ac56117f5e7808fe0221008cbb42233bb04d7f28a715cf7c938e238afde90207e9d103dd9018e12cb7180e 01
41 042daa93315eebbe2cb9b5c3505df4c6fb6caca8b756786098567550d4820c09db988fe9997d049d687292f815ccd6e7fb5c1b1a91137999818d17c73d0f80aef9

first line is 'push signature concatenated with 01', second line is 'push pubkey'. The length of scriptSig is 140 bytes (0x8c in hex)

17. We then replace the one-byte, varint length-field from step 5 with the length of the data from step 16. The length is 140 bytes, or 0x8C bytes: 8c

18. And we replace the actual scriptSig with the data structure constructed in step 16.

19. We finish off by removing the four-byte hash code type we added in step 13, and we end up with the following stream of bytes, which is the final transaction:

    01000000 01 be66e10da854e7aea9338c1f91cd489768d1d6d7189f586d7a3613f2a24d5396 00000000 8c 49 3046022100cf4d7571dd47a4d47f5cb767d54d6702530a3555726b27b6ac56117f5e7808fe0221008cbb42233bb04d7f28a715cf7c938e238afde90207e9d103dd9018e12cb7180e 01 41 042daa93315eebbe2cb9b5c3505df4c6fb6caca8b756786098567550d4820c09db988fe9997d049d687292f815ccd6e7fb5c1b1a91137999818d17c73d0f80aef9 ffffffff 01 23ce010000000000 19 76 a9 14 a2fd2e039a86dbcf0e1a664729e09e8007f89510 88 ac 00000000

Answer 2

This is an excellent answer by amaclin and Wizard of Ozzie. I'd just like to add some more detail for the Wizard of Ozzie's transaction at blockchain.info.

1. The correct input to the signature process for this transaction is

    01000000
    01
    be66e10da854e7aea9338c1f91cd489768d1d6d7189f586d7a3613f2a24d5396
    00000000
    19 76 a9 14 dd6cce9f255a8cc17bda8ba0373df8e861cb866e 88 ac
    ffffffff
    01
    23ce010000000000
    19 76 a9 14 2bc89c2702e0e618db7d59eb5ce2f0f147b40754 88 ac
    00000000
    01000000
   

2. The double-SHA256 hash of this is computed by bx as

    d304448dff517bcf677cd36f3491e9ef2ccfdf40fb63af5782d9b768640af130
   

but this is written in "reverse" order for whatever peculiar historical reason (I've heard it was to be compatible with M*soft CRAPi). So when using this with the sign(key, digest) in most sensible crypto toolkits you may need to reverse the hash value to the "proper" order for SHA256(SHA256(m))

30f10a6468b7d98257af63fb40dfcf2cefe991346fd37c67cf7b51ff8d4404d3

3. The above input data will indeed validate against the signature given in the actual blockchain transaction

     3045022100da43201760bda697222002f56266bf65023fef2094519e13077f777baed553b102205ce35d05eabda58cd50a67977a65706347cc25ef43153e309ff210a134722e9e
   

using the given public key

042daa93315eebbe2cb9b5c3505df4c6fb6caca8b756786098567550d4820c09db988fe9997d049d687292f815ccd6e7fb5c1b1a91137999818d17c73d0f80aef9

4. To generate a reproducible signature over the same data, the following signature has been made by the deterministic method of RFC6979.

    30450220587ce0cf0252e2db3a7c3c91b355aa8f3385e128227cd8727c5f7777877ad772022100edc508b7c14891ed15ab38c687019d7ebaf5c12908cf21a83e8ae57e8c47e95c
   

using the matching private key provided by Wizard of Ozzie

0ecd20654c2e2be708495853e8da35c664247040c00bd10b9b13e5e86e6a808d

This signature also validates over the given data using the public key and should be reproducible.