I know that Bitcoin Core uses LevelDB since 0.8 version. However, I couldn't find detailed explanation about how Bitcoin stores and retrieves from LevelDB. E.g. If B transaction uses an output from previous transaction A as input, how does Bitcoin lookup this transaction and see if it's spent? After transaction B is spent, how does this transaction get updated?
1 Answer
bitcoind
uses leveldb
to store its utxo database, which is also referred to as chainstate. This can be found in the Bitcoin data directory under chainstate/
.
UTXOs are identified by their txid
(using their little-endian representation) led by a C
. For versions before 0.15
, the entries are prefixed with c
instead.
Example
Transaction 246c5a81b6ad0dfc0dbc0b2ff5bde65ee1913f75a47d409b8ff8074a27ec1000
is identified in the chainstate by C0010ec274a07f88f9b407da4753f91e15ee6bdf52f0bbc0dfc0dadb6815a6c24
or what is the same 430010ec274a07f88f9b407da4753f91e15ee6bdf52f0bbc0dfc0dadb6815a6c24
(since the hex representation of the uppercase ASCII character 'C' is 43).
Each entry in the leveldb contains two values, the identifier
(already explained and used as key in the map) and the value
.
The value
of each entry (the UTXO
itself) is encoded using the following structure (check this comment from the source code for more detail):
- Serialized format
- VARINT(nVersion)
- VARINT(nCode)
- unspentness bitvector, for vout
[2]
and further; least significant byte first- the non-spent CTxOuts (via CTxOutCompressor)
- VARINT(nHeight) * The nCode value consists of:
- bit 1: IsCoinBase()
- bit 2: vout
[0]
is not spent- bit 4: vout
[1]
is not spent- The higher bits encode N, the number of non-zero bytes in the following bitvector.
- In case both bit 2 and bit 4 are unset, they encode N-1, as there must be at least one non-spent output).
Notice that, since the database used to trigger anti-virus software(2, 3), an obfuscation key was introduced so the data can be masked. The key is a 64-bit value identified by 0e00obfuscation_key
that should be XORed with each data value from the database. Given the entries in the database may be longer than the key, the key is concatenated with itself until it reaches the expected length to de-obfuscate a value.
Example
Let's use the following obfuscation key and entry in the database:
// Obfuscation key
o_k = "27c78118b7316105"
// Entry
{
"key": "43000002f414665fb03389dd19776732bf90883bcb399d23323747596e98dd1801",
"value": "26c326d7353661dc7005d274976f458691f24f0f05d141335f4ad5927e41"
}
As you can see, value
does not follow the above-introduced format, since it is obfuscated. value
is 60 characters long, so we need to extend our obfuscation key to match its size:
27c78118b731610527c78118b731610527c78118b731610527c78118b731
Now, if we perform the XOR between the value and the key, we obtain:
> 26c326d7353661dc7005d274976f458691f24f0f05d141335f4ad5927e41 XOR 27c78118b731610527c78118b731610527c78118b731610527c78118b731
0104a7cf820700d957c2536c205e2483b635ce17b2e02036788d548ac970
That matches the previously stated format, and it's indeed the UTXO
identified by 43000002f414665fb03389dd19776732bf90883bcb399d23323747596e98dd1801
-
Thanks. Finally I figured out that Bitcoin uses LevelDB to save the transaction index, which uses the transaction hash as key and file position as value. The actual binary transaction data are stored in the block files.– YangruiMar 17, 2017 at 21:46
-
1@Yangrui That used to be how Bitcoin Core worked up to version 0.7.x. As of 0.8.0, there is no transaction index at all anymore, and only a database with a copy of the unspent transaction outputs. This means no disk lookup for the actual transaction is needed anymore, and we're able to support pruning (deleting the actual block data from disk after validation). Mar 27, 2017 at 18:58
-
1Thanks Pieter. So if a transaction is spent, is it just removed from the UTXO db right away?– YangruiMar 28, 2017 at 20:03
-
For future travelers like me, the key for chainstate is not an ascii
0e00obfuscation_key
, but a hex\x0e\x00obfuscate_key
(as of v0.15.0.1)– num1Apr 10, 2018 at 1:09 -