1

I have a problem with reading tx out from the Bitcoin Core LevelDB chainstate storage. I know that the structure of the current version of the chainstate is key => value where the key is a prefix (0x43) | tx_id | coin_n (https://github.com/bitcoin/bitcoin/blob/0d509bab45d292caeaf34600e57b5928757c6005/src/txdb.cpp#L39) and the value is obfuscated by the obfuscation key through the Coin XOR obfuscation_key value of the Coin.

Ref: https://github.com/bitcoin/bitcoin/blob/0d509bab45d292caeaf34600e57b5928757c6005/src/txdb.cpp#L68

So, i have successfully read the prefix, obfuscation key and the txId with the output_n

key:
    prefix: [1]byte

(COutPoint)
    txID: [32]byte
    outN: VARINT

Ref: https://github.com/bitcoin/bitcoin/blob/0d509bab45d292caeaf34600e57b5928757c6005/src/primitives/transaction.h#L28

The Coin structure (https://github.com/bitcoin/bitcoin/blob/0d509bab45d292caeaf34600e57b5928757c6005/src/coins.h#L24)

code: VARINT((coinbase ? 1 : 0) | (height << 1))
txOut:  CTxOut (via TxOutCompression)

Now i stucked on the reading the code. In some values, i have a valid code with true block height and coinbase boolean. But in some values i have invalid data where the block height is not the same with the transaction's block height.

How you can reproduce this to help me?

My input (LevelDB view):

Key                                                                     | Value
0e006f62667573636174655f6b6579                                          | 08f05564aa8317fe24
42                                                                      | bde0dff15a88e0eb3d71c37fd986c4cdfd8916bd0b8d0b13343afef66829284d
4300c4626bf6dc028730d3a1f3951d398a71a164d8a898965d11bb8a699091144c00    | 8f6778aa979ddbbc669c187cbf11ad2e30ca53025f3f3b961e
43047ae5966a25bdc32ab4a258c57e28ffa763fc273c08b38fe85839e99e71402500    | ef6778aa979ddbbc669c187cbf11ad2e30ca53025f3f3b961e
430e3afb85bb66cc8a5edfd96f4aad1e65e4e6cf38f1dcc584c525d0b25b89599c00    | 706e56b68303740168c3add6552bf877fa95fb9d2bcbd6e142bb

When i am reading the first values and deobfuscate them, i am getting true block height and coinbase. But when i am reading the last value, i am getting this:

~ 706e56b68303740168c3add6552bf877fa95fb9d2bcbd6e142bb XOR f05564aa8317fe24
~ 803b321c00148a259896c97cd63c06530ac09f37a8dc28c5b2ee

So, when i am trying to read the uvarint from this, i am getting an invalid code with invalid block height:

package main

import (
    "bytes"
    "encoding/binary"
    "encoding/hex"
    "log"
)

func main() {
    utxo, _ := hex.DecodeString("803b321c00148a259896c97cd63c06530ac09f37a8dc28c5b2ee")
    utxoReader := bytes.NewReader(utxo)

    code, _ := binary.ReadUvarint(utxoReader)
    log.Println("code", code)
    
    blockHeight := code >> 1
    log.Println("blockHeight", blockHeight)
    
    isCoinbase := code|1 == 1
    log.Println("isCoinbase", isCoinbase)
}

Result

2024/04/03 15:24:06 code 7552
2024/04/03 15:24:06 blockHeight 3776
2024/04/03 15:24:06 isCoinbase false

Expected result: blockHeight must be 93, coinbase false.

Other information. My deobfuscate function code:

func deobfuscateValue(obfuscationKey, value []byte) []byte {
    deobfuscatedValue := make([]byte, len(value))

    copy(deobfuscatedValue, value)

    for i, c := range value {
        deobfuscatedValue[i] = c ^ obfuscationKey[i%len(obfuscationKey)]
    }

    return deobfuscatedValue
}

I am running the private regtest network on the Bitcoin Core v0.21 (i have tried on the 24 version and have the same results). I am use bitcoin-testnet-box docker image to test my program. Also, i have tested on the mainnet, but have the same problem too. My blockchain now have only 105 blocks.

Thank you for helping. I think I missed something simple and just don't know something about the chainstate or C++ realization or Golang :)

1
  • It seems like i am using binary.ReadUvarint which is not right function to read the code value
    – Novak
    Apr 3 at 14:29

1 Answer 1

3

I got it :_)

I used binary.ReadUvarint which not the same as ReadVarint in the Bitcoin Core (https://github.com/bitcoin/bitcoin/blob/0d509bab45d292caeaf34600e57b5928757c6005/src/serialize.h#L465)

So far I have not been able to explain the reason for this behavior, but i found the right code on the btcd (https://github.com/btcsuite/btcd/blob/ae5533602c46f4ea14a4da725c19cb3db7d5eb88/blockchain/compress.go#L93) and adapted it to my use case.

func DeserializeVLQ(buf io.ByteReader) (uint64, int, error) {
    var n uint64
    var size int

    for {
        val, err := buf.ReadByte()
        if err != nil {
            if err == io.EOF {
                break // End of stream is a natural break
            }
            return 0, 0, err // Return any error that occurs
        }

        size++
        n = (n << 7) | uint64(val&0x7F)
        if val&0x80 != 0x80 {
            break
        }
        n++
    }

    return n, size, nil
}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.