35

Years ago I designed a .NET module which facilitates the transmission of BTC to my customers. It creates a binary representation of the desired transaction based on the material presented here and here:

The binary representation is then converted to hex and pushed out through various free services like BlockExplorer, BlockCypher, etc.

The system has been running flawlessly for years. Until yesterday. A customer requested that 0.0284377 BTC to be sent to 38MRMGjMBMp4k7vZhKLHhcM9Pm8AMLy18v. He never received it. Sure enough, he was right. It was never sent.

I looked through all of my logs and saw that my software did in fact submit a transaction with the following inputs and outputs:

  • input: 13P38hMYJXFdxDJJn8TtPUJZFXmcpf2J99
  • output #1: 38MRMGjMBMp4k7vZhKLHhcM9Pm8AMLy18v (my customer)
  • output #2: 1Ny3CV3rAsNMWpLfpxhXW3Fh71YmMEXXU7 (my change address)

Everything looked fine on my side but sure enough, when I looked on block explorer, what I saw shocked me. Consider transaction 9a138b14dcc8ae740073c06933ae04e3b08fe6be6ada0dc175e6484250dfe269 and look at the inputs and outputs:

As you can see, my input is correct. My change address XU7 is also correct. But now look at my customer's address. It says 17fQRjEudTVgexE8aDfhGyzDFEqSnnJJCA (which I'll refer to as JCA) instead of the expected 38MRMGjMBMp4k7vZhKLHhcM9Pm8AMLy18v (which I'll refer to as 18v). What the heck? The JCA address is known neither to me nor to my customer. It's a completely unknown address.

Clearly the problem is in my code somewhere so I dug deeper to find out what made this particular transaction unique. I identified that his desired address (18v) begins with a '3' whereas every other outbound transaction I've completed throughout the history of my app has targeted addresses beginning with '1'.

My stopgap measure was to force users to only specify addresses that begin with a 1. But that's a temporary solution. I need to find out what I'm doing wrong.

I turned to Google. Research indicated that addresses beginning with '3' are typically SegWit addresses where as those beginning with '1' are traditional, old-school addresses. I remember all the talk about SegWit a few months back but I figured it didn't affect me and my legacy transactions would still clear properly. Obviously that was a bad judgement call on my part so now I have to figure out what I did incorrectly and where the bogus JCA address came from.

That's where I'm stuck. I think my problem might have to do with uncompressed vs. compressed public keys based upon discussions here and here:

Here's what I do know: When it comes time to create the outputs for my transaction, my code does the following (explanation follows):

Func<String,byte[]> makeOutScript=(btcAddrHex)=>{
    byte[] addrBytes=BTCUtils.Base58Decode(btcAddrHex);
    byte[] pubKeyBytes=addrBytes.Take(addrBytes.Length-4).Skip(1).ToArray();

    using(MemoryStream ms=new MemoryStream()){
        using(BinaryWriter bw=new BinaryWriter(ms)){
            bw.Write((byte)0x76);   //op_dup
            bw.Write((byte)0xa9);   //op_hash160
            bw.Write((byte)20);     //size of public key
            bw.Write(pubKeyBytes);  //public key
            bw.Write((byte)0x88);   //op_equalverify
            bw.Write((byte)0xac);   //op_checksig

            bw.Flush();
            return ms.ToArray();
        }
    }
};

What I'm doing here is: Converting the output address to bytes by Base58 decoding it. I strip off the last four bytes, which are the checksum, and the first byte, which is always 0x00 (apparently a network indicator of some sort). That leaves me with the raw public key. Actually, looking at my internal notes, it's not actually the public key but rather 0x04 prepended to the public key and then passed through SHA256 and then RIPEMD160. But I'll refer to this blob as the public key. From there the byte sequence is 0x76, 0xa9, 20 (the size of the public key blob), the public key blob itself, 0x88, and 0xac.

I don't pretend to understand what all those single-byte entries mean but the blog posts above said to include them so that's what I did, and until now it's worked just fine.

The question is: How on earth did the 18v address get transformed into the JCA address when it was submitted to the network? Is my "size of public key" value (20) incorrect, perhaps? At a gut level, it does seem odd that a compressed (18v) and uncompressed (JCA) public key would both have a constant size of 20. But maybe I'm on the wrong track entirely and compression has nothing to do with it.

I don't have to tell you how awful it would be if this transaction were 100 BTC instead of 0.0284377. I lucked out that my bug was triggered on a low value transaction. But I really want to solve this. Can you point me down the right path?

  • Joined just to plus one a superb question! – bishop Dec 2 '17 at 19:02
  • 1
    same here @bishop ! heh! – Fattie Dec 4 '17 at 20:26
  • Don't mean to bandwagon here, but I joined to upvote. Glad you got your answer, and the thread in the answer below is just as interesting! – Atticus Dec 9 '17 at 1:46
44
+50

Addresses that start with 3... are P2SH addresses, and they've been around for almost 6 years now. Historically they saw limited use - used by multisig wallets like Copay and GreenAddress. The compatibility mode of SegWit however also uses P2SH, making their use more common recently.

The main issue is that your code for converting an address to a scriptPubKey does not look at the version byte. After Base58 decoding the address, you should get <1-byte version> <20-byte hash> <4-byte checksum> structure. If that version is 0, which it is for 1... addressses, that hash is a public key hash, and you correctly construct the corresponding scriptPubKey.

However, if that version number is 5, it is a P2SH address, which was specified in BIP13, which uses a corresponding scriptPubKey specified in BIP16.

While SegWit initially just used the old P2SH format, A new address format is being introduced for SegWit as well (which gives slightly better performance, flexibility, and error detection capabilities), described in BIP173. (disclaimer: I'm an author of BIP173). These addresses will start with bc1....

I advise you to stop your service, and at the very least implement verification that the version number is expected, together with a test set you can find in many implementations. If you feel like it, you can also implement BIP13 and BIP173 addresses, both of which will likely become more prevalent soon.

  • 4
    Whoa, this is crazy interesting. I don't know how I've spent all these years never dealing with a Bitcoin address starting with something other than '1'. Even outside of my app, all of my Electrum addresses start with '1' too. Ok, so you're right, when I unpack the 18v address it does in fact have a leading byte set to 5. So you're saying: (1) check that version byte! and (2) if it's a 5, I need to change the structure of that script? What exactly is "BIP"? Is that the reference doc? Why so many people claim that leading '3' is a "segwit" address? So much bad info out there :( – Festus Martingale Dec 1 '17 at 18:53
  • 21
    No, the first thing you need to do is check the checksum and the version number. Thinking that the version is always zero is the cause of this problem. Even if you believe it will always be zero, your software should fail if it isn't. – Pieter Wuille Dec 1 '17 at 19:07
  • 7
    BIPs, or Bitcoin Improvement Proposals is a process for publishing ideas around changes to Bitcoin's rules or their use: github.com/bitcoin/bips – Pieter Wuille Dec 1 '17 at 19:10
  • 2
    SegWit needed a way to be compatible with old software (so that old wallets could send to SegWit wallets), so it can be used either in compatibility mode (which uses P2SH 3... addresses) or native mode (which uses BIP173 bc1... addresses). Before SegWit, the only software that would hand out P2SH addresses is multisig wallets like GreenAddress or copay. – Pieter Wuille Dec 1 '17 at 19:12
  • 3
    You'll find that in BIP16. – Pieter Wuille Dec 1 '17 at 19:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.