Why did bech32 define a new base32 encoding scheme rather than use an existing one?

Question

There are multiple existing base32 encodings (RFC 3548/4648, z-base-32, Crockford's etc). Why did bech32 define a new one rather than use an existing one?

Answer 1

There are multiple aspects to the design of bech32. There is the choice of character set (i.e. which characters are included in the 32 options and which characters are excluded because they visually look like other characters e.g. 1 because it looks like l), the mapping of specific bits to this character set, the type of checksum to use and the final address structure.

BIP 173 explains the choice of character set.

The character set is chosen to minimize ambiguity according to this visual similarity data, and the ordering is chosen to minimize the number of pairs of similar characters (according to the same data) that differ in more than 1 bit. As the checksum is chosen to maximize detection capabilities for low numbers of bit errors, this choice improves its performance under some error models.

The mapping of bits to bech32 characters is also included in the BIP.

e.g. 11 maps to t (3 on x axis and +8 on y axis) and 30 maps to 7 (6 on x axis and +24 on y axis)

Note that although 1 is included in bech32 addresses as a separator (see here) the character 1 is not included in the permitted character set of bech32. Nothing maps to 1 in this mapping.

Pieter Wuille explained the rationale for this bit to character mapping at SF Bitcoin Devs in March 2017:

The reason for this is we were able to select our code to optimize for low bit error rates. Wouldn’t it be great if we could choose the character set in such a way that 1 bit errors are more likely than non 1 bit errors. This character set is the result of another year of CPU time to optimize for this. We found a bunch of information on tables for similarity between various characters. What you can see on this slide, the z and the 2 are considered similar in some fonts or writing. As you can see they are 8 apart. One is 2 and the other is 10 so they are 1 bit apart. And r and t are 1 bit apart. And y and v are 1 bit apart. And x and k are 1 bit apart. And e and a are 1 bit apart. And s and 5 are 1 bit apart. And 4 and h are 1 bit apart. There are way more similar errors overlaid in this data that you can look for. It’s pretty cool. We made a character set that optimizes for 1 bit errors. As a result our code is distance 6 for 1 bit errors. If you just look at these 1 bit errors we guarantee 5 errors. If you only make errors like this you can detect 5.