What are the DER signature and SEC format

Question

I often see those 2 elements in bitcoin documentations and tutorials but I have not yet find a proper definition on the internet for DER signature and SEC format.

Answer 1

DER

The Distinguished Encoding Rules (DER) format is used to encode ECDSA signatures in Bitcoin. An ECDSA signature is generated using a private key and a hash of the signed message. It consists of two 32-byte numbers (r,s). As described by Pieter here the DER signature format has the following components:

1. 0x30 byte: header byte to indicate compound structure
2. one byte to encode the length of the following data
3. 0x02: header byte indicating an integer
4. one byte to encode the length of the following r value
5. the r value as a big-endian integer
6. 0x02: header byte indicating an integer
7. one byte to encode the length of the following s value
8. the s value as a big-endian integer

Note that the r and s value must be prepended with 0x00 if their first byte is greater than 0x7F. This causes variable signature lengths in the case that the r value is in the upper half of the range (referred to as "high r"). Signatures with high s values are non-standard and usually don't appear in the wild. Also note that in rare cases r or s can be shorter than 32 bytes which is legal and leads to shorter signatures. Note that in bitcoin transactions a byte is added at the end of a DER signature denoting the SigHash type used.

SEC

The Standards of Efficient Cryptography (SEC) encoding is used to serialize ECDSA public keys. Public keys in Bitcoin are ECDSA points consisting of two coordinates (x,y). x and y may be smaller than 32 bytes in which case they must be padded with zeros to 32 bytes (H/T Coding Enthusiast). Bitcoin uses two formats, uncompressed and compressed:

Uncompressed:

1. 0x04 byte: header byte to indicate ECDSA point
2. the x coordinate as a 32-byte big-endian integer
3. the y coordinate as a 32-byte big-endian integer

Compressed:

As the coordinates (x,y) must satisfy the secp256k1 curve equation y² = x³ + 7, the two possible values for y can be calculated from the x value. Thus, we can express a public key just as the x coordinate in combination with an indicator which of the two y values to use.

1. 0x02/0x03 byte: header byte to indicate compressed ECDSA point, 0x02 for even y, 0x03 for odd y
2. the x coordinate as a 32-byte big-endian integer

---

The above is more comprehensively explained e.g. in Jimmy Song's Programming Bitcoin: Ch4. Serialization.

Answer 2

There are different formats used to encode public keys and signatures into binary (octet-streams). They are defined in Standards for Efficient Cryptography 1 (SEC).

A public key is a point on an elliptic curve, consisting of an x and y coordinate. There needs to be a standard way for serializing these parts and deserializing them later. The standard defines a uncompressed and a compressed format for this serialization. Roughly, the uncompressed format uses the prefix byte 0x04 followed by the serialized x coordinate, followed by the serialized y coordinate. The lengths of x and y are determined by the curve. For bitcoin, the curve secp256k1 is always used, so the coordinates are 32 bytes. The resulting uncompressed public key is 65 bytes total.

The compressed version prefixes the octet-stream with 0x02 if the y coordinate is even and 0x03 if the y coordinate is odd. The x coordinate is serialized after it, resulting in a 33 byte public key. This information is sufficient because the y coordinate can be recovered if x is known and the sign is known. (Technically, we don't need the sign either).

Signatures consist of an r value (the x coordinate of a point on the curve), and an s value, which is an integer (32-bytes for secp256k1). The signature can be encoded in just 64 bytes in this compact form.

The paper previously linked also gives descriptions of these formats in ASN.1 (Abstract Syntax Notation 1). This is a textual interface description language which has been used to describe protocols for interoperability for a long time. Another standard, DER, or Distinguished Encoding Rules, describes a process for serializing data into octet-streams according to the ASN.1 specification for the data type.

The ASN.1 for signatures is (simplified):

ECDSA-Sig-Value ::= SEQUENCE {
    r INTEGER,
    s INTEGER
}

According to the DER rules: a SEQUENCE is of type 0x30 and is followed by a length (1 byte for length < 128), then the data for the sequence itself. INTEGER has type type 0x02 followed by its length, followed by the integer in binary. These integers are also signed, which means that if their value is >= 2^31 an additional zero valued byte must be prefixed to them. The resulting DER encoding for a Secp256k1 signature looks something like (assuming 32-byte r and s):

0x30 0x44 0x02 0x20 <r[32]> 0x02 0x20 <s[32]>

As you can see, this is pretty wasteful compared to the concise 64-byte serialization of (r,s). The uncompressed version of public keys is also very wasteful because every byte in a transaction is valuable. The Segwit upgrade to bitcoin introduced some restrictions on which formats can be used, and additionally, the upcoming Taproot upgrade will require 32-byte (x-only) public keys, which are not part of the SEC standard.