3

Using "Sign Message" one can prove ownership of an address.
This happens by signing a human-readable string using the private key belonging to the address.
The owner then provides the address, message and signature, and anyone can verify that they indeed hold the private key of the address.
The ECDSA signature is normally provided in 65-byte format (Recovery byte + 32-byte R + 32-byte S) and base64 encoded.

However, there is also another type of signature, which is used for "Sign Transaction".
For each input of a transaction, there is some data to be signed. This data is binary data, normally provided in hex format.
The corresponding signature is provided in 70-, 71-, or 72-byte DER format.

In both cases, the message (message string in the first case, and binary per-input data in the second case) is transformed into a byte array, and the private key is used to generate an ECDSA signature for that message byte array.
The output format is different, but the 65-byte format can easily be converted into DER.

Consequently, I thought that I could just use "Sign Message" (with a minor modification, so that it takes a hex string instead of an ASCII/UTF-8 string) to sign a transaction's hex-encoded binary per-input data.
I did that, converted the signature to DER format and tried to send the transaction, but it didn't work, because the signature was wrong.

So my question is, what is the difference between how "Sign Message" and "Sign Transaction" generate signatures, and why aren't they compatible?

Is the difference that for "Sign Message", first a digest is created, which is then signed, while for "Sign Transaction" (in the meaning written above), the input is already a digest, so it shouldn't be hashed again?

Thank you!

2 Answers 2

5

Is the difference that for "Sign Message", first a digest is created, which is then signed, while for "Sign Transaction" (in the meaning written above), the input is already a digest, so it shouldn't be hashed again?

No. It has nothing to do with the hashing. Furthermore, the input is not already a digest, it must be hashed before signing. Both the message and the input data are hashed twice with SHA256.

The difference between message signing and transaction input signing is that message signing prepends the string Bitcoin Signed Message:\n (with \n being a newline character, not literally \n) to the message before it is hashed.

Additionally, the 65 byte signature produced by message signing has a byte at the beginning that is not part of the cryptographic signature itself. Instead it is a recovery id so that the correct public key can be recovered from the signature during signed message verification. In transaction signatures, there is a different byte appended to the end of the signature which is the sighash type, usually it is the byte 0x01.

3
  • Thank you, @andrew You write that in the 65-byte signature, there's a recovery byte at the end. The recovery byte is at the beginning (as mentioned in my question above). Also, In what TX signatures is there an additional byte at the end? In DER format, there are other DER-specific bytes, but not related to signhash. I tried to keep my question generic, but maybe I should have mentioned that the TX input I want to sign is BlockCypher's toSign. I think that's already a digest, so it just needs to be signed.
    – remv
    Oct 6, 2019 at 11:35
  • My bad, the recovery byte is indeed at the beginning. If you properly DER decode a signature that you put in a transaction (or a signature you get from a transaction), you will find that there is an extra byte at the end. That byte corresponds to the sighash type.
    – Andrew Chow
    Oct 6, 2019 at 17:22
  • It should be noted that the prepended "magic string" includes a prefixed size encoder byte, so that it reads "\x18Bitcoin Signed Message:\n" (This is different from the recovery byte in the signature)
    – JustKevin
    Dec 21, 2021 at 20:00
0

@andrew provided helpful information that a major difference between "Sign Message" and "Sign Transaction" is that "Sign Message" internally prepends a custom string to the textual message being signed.
Also, his answer addresses the case when you do all steps of the TX creation and signing.

I tried to keep my initial question generic, but now I see that I should have mentioned that the per-input TX data I want to sign is BlockCypher's toSign array.
In their documentation it's not immediately clear, but with some investigation, I've found out that the provided toSign hex strings are already hashed twice using SHA256, so they indeed just need to be signed.
Probably there are also other systems that provide such digests for signing.

So, in summary, if you are using an external system for creating TXs, which provides you data to sign with your private key, that data might already be hashed. One good indicator for such a digest is that it is 32 bytes long.
There are multiple reasons why "Sign Message" will not work out-of-the-box for signing such TX data:

  • The TX data is binary data (usually in hex format), and "Sign Message" expects ASCII/UTF-8 strings.
  • "Sign Message" internally adds a custom prefix to the message being signed.
  • The TX data is already hashed, so shouldn't be hashed again.
  • The output of "Sign Message" is a 65-byte ECDSA signature, and it's not in DER format (can easily be converted).

However, the actual signing is the same for both "Sign Message" and "Sign Transaction".

Hopefully this will save some research time for others with a similar use-case.

1
  • I had issues like 'Error reading signatures and pubkeys: Invalid signature, wrong length' with signing tosign data provided by Blockcypher by using bitcore Message class. As it turned out I should have used crypto.ECDSA.sign() function instead. This Q&A was very helpful for me. Sep 3 at 12:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.