Compress transaction hex string

Question

I realized transaction hex can be very long and you need it for broadcasting a tx using blockstream.info API:

https://blockstream.info/testnet/api/tx

https://github.com/Blockstream/esplora/blob/master/API.md

If the user wants to send this hex using text message which allows 160 characters in one message, what should be the best way to solve this problem? I tried researching about way to compress it, send to a number which forwards to the web server running a PHP code, the hex string is decompressed and sent to blockstream.info for broadcasting tx. Base64 encoding after gzcompress() couldn't reduce the number of characters to less than 160.

Example:

Hex:

02000000000101e939fb23e9991ebbc75fd08c736da32ca12d98a4ff1b8e970e97f5661927ee410100000000fdffffff02b0a90a000000000016001421e2f997b3bd36e273eaca365da8515a389444ae40420f0000000000160014829e2dbcf6b7f31bc93633971f71f6f6b9b5f89e0247304402200f8e3e573be749caf1964a85707bf540de2e7b367ae46c23bd4f21932ff82346022062dc3007072cd5a19b45e479525f4829bc48be4fd3c21b5a9ae34bcf9a3a3ccf0121020f88c7db36cbb492e80d3062fc19db55bed82687498f8cfe6d0cf47adf6687aa49f31b00

Compressed and Base64 encoded:

eNpdkNmJBEAIRFPyarsNxzP/ENZZGBZW9Eew6pVA8C0EbGObIG4zw47Ie6bg5WUtZ0pHKnsuMxiv7cLOHFU0ut2yCl+xqfktoAA3cPiz0R0hbBqzGxzF2nS5PZ31lL+Dx/mZiHgLCMH8v35kTRU5GncYI42V2S7Otu7W4syzBpLLIAK0Mec197kcfcXSB01lzS7cmCNQTb04etdUk5ZLhtCYZh6x6EdDqZIB9oSyjqOFnJZrh858oCLlRcsUJ2EcN2+WxTRn58wBJITN8/altV4ZIUb9oHi1J9EqzomuR/qW8s3LaS3Ikes1ult3sU9mgB8sW3Vy

Answer 1

Compressing a more-or-less random number is futile. Your best option is simply to use a more compact encoding. Base64 is better than Hex but there exists may other encodings that prform better

Wikipedia lists many and ranks them in order of efficiency

Encoding            Data type                    Efficiency
yEnc                Arbitrary, mostly non-text   ~98%
Ascii85             Arbitrary                     80%
Base85 (RFC 1924)   Arbitrary                     80%
Base64              Arbitrary                     75%
...

You can obviously do better if you use 8-bits (e.g. Unicode printable code points that have single byte encodings in UTF-8) but since SMS uses a 7-bit character set (so far as I know) you won't do much better

Answer 2

If the user wants to send this hex using text message which allows 160 characters in one message, what should be the best way to solve this problem?

Turning 222 bytes to 160 characters is going to be very challenging so our focus can not be only on the encoding but it should be directed at the 222 byte itself. Since it is not exactly random bytes and bitcoin transaction structure is not exactly the most efficient structure there are opportunities to compress these bytes (or more accurately to get rid of some "useless" bytes).
Let's first break it down and see which bytes we can get rid of:

1) 02000000
2) 0001
3) 01
4) e939fb23e9991ebbc75fd08c736da32ca12d98a4ff1b8e970e97f5661927ee41
5) 01000000
6) 00
7) fdffffff
8) 02
9) b0a90a0000000000
10) 16001421e2f997b3bd36e273eaca365da8515a389444ae
11) 40420f0000000000
12) 160014829e2dbcf6b7f31bc93633971f71f6f6b9b5f89e
13) 02
14) 47304402200f8e3e573be749caf1964a85707bf540de2e7b367ae46c23bd4f21932ff82346022062dc3007072cd5a19b45e479525f4829bc48be4fd3c21b5a9ae34bcf9a3a3ccf01
15) 21020f88c7db36cbb492e80d3062fc19db55bed82687498f8cfe6d0cf47adf6687aa
16) 49f31b00

1. Version
   Could turn into 1 byte encoded as a CompactInt -> 02
2. Witness flag
   Can be skipped -> (it works by modifying #6)
3. Input count
   No change -> 01
4. Input transaction hash
   No change -> e939fb23e9991ebbc75fd08c736da32ca12d98a4ff1b8e970e97f5661927ee41
5. Input index
   Another CompactInt -> 01
6. Signature script
   As a rule all scripts could start with a flag 1=P2PKH, 2=P2WPKH, 3=P2SH,... 255=notdefined, and when they are not defined the exact script is placed here with the size and everything else. Like every other part the receiver has to construct the script correctly (set witness flag #2, move these to witness items #14 & #15, add lengths for each push pushes, DER encode signature)
   Witness becomes: <1-byte-flag><32-byte-r><32-byte-s><33-byte-pubkey>
   <02><0f8e3e573be749caf1964a85707bf540de2e7b367ae46c23bd4f21932ff82346><62dc3007072cd5a19b45e479525f4829bc48be4fd3c21b5a9ae34bcf9a3a3ccf><01><020f88c7db36cbb492e80d3062fc19db55bed82687498f8cfe6d0cf47adf6687aa>
   If pubkey was uncompressed the first byte is changed to 0x82=0b10000010 (most significant bit set) to indicate uncompressed pubkey should be constructed.
7. Sequence
   can be encoded as a "StackInt" (a number that is pushed to stack which can be negative)
   1=OP_1, 2=OP_2 -1=OP_NegativeOne -2=0x82, 321321=0x4e29e70400
   Since 0xfdffffff is UInt32.Max-2 -> 82
8. Output count
   No change -> 02
9. Amount
   As a CompactInt -> feb0a90a00
10. Pubkey script
    Similar to signature script we could use flags for standard scripts, eg. 0x03 could be P2WPKH scripts -> 0321e2f997b3bd36e273eaca365da8515a389444ae
11. Amount
    As a CompactInt -> fe40420f00
12. Pubkey script
    Same as before 03829e2dbcf6b7f31bc93633971f71f6f6b9b5f89e
13. Witness item count
    Removed ->
14. Witness
    Removed (it is already placed in #6) ->
15. Witness
    Removed (it is already placed in #6) ->
16. Locktime
    No change -> 49f31b00

Now the 222 bytes is compressed into 192 bytes (13.5%) which can then be encoded with a more efficient encoding to get the best possible result.
This could also be compressed more if you are sending this transaction to the actual receiver of the funds by skipping their output(s). For instance when you were supposed to pay 698800 satoshi to bc1qy830n9anh5mwyul2egm9m2z3tgufg39wk4g0eu, the receiver already knows this and you don't have to tell them again. That means 26 bytes in #11 and #12 can be skipped reducing the size to 166 bytes (25.2%). But it has to be agreed upon that the receiving output is the first one for example.
This could be compressed even more by coming up with more agreements:

• Tx version should always be 2 -> 165 bytes
• SigHashType should always be SigHashAll -> 164 bytes (remove from each input #6 here)
• Each input sequence should be set to predefined int -> 163 bytes (skip #7)
• Tx should always have 2 outputs -> 162 bytes (skip #8)
• Locktime should always be 0 -> 158 bytes (skip #16) (28.8%)