Why did Core move from BDB to LevelDB? Why don't they use SQLite or move to Redis now? Is there a technical reason for this choice?

  • The bdb->leveldb change was made to increase speed while validating blocks and during initial block download. Also, doesn't redis require that you load your entire dataset into memory? Pretty painful for a 60GB blockchain. – Nick ODell Oct 12 '16 at 23:43
  • but redis uses LZF light data compressor , this won't help to reduce the data volume in memory? and i thought leveldb was chosen because it supports high caching data. – Etherkimist Oct 13 '16 at 8:22
  • 2
    LevelDB also supports compression. We explicitly disable it in Bitcoin Core because it does not help (almost all the data in the database consists of uncompressible cryptographic material anyway: hashes, keys, signatures). – Pieter Wuille Oct 13 '16 at 10:59
  • @Nick We wouldn't store the entire blockchain in the database anyway. – Pieter Wuille Oct 13 '16 at 11:05
  • 1
    Not even the blocks. Those are stored on disk, but not in a database. The only data of significance in the database is the UTXO set. – Pieter Wuille Oct 13 '16 at 11:14

Redis and LevelDB solve very different problems. We tried using SQLite and its performance was abysmal.

Bitcoin Core needs a database to store the set of unspent transaction outputs (UTXOs). This means we need fast simple reads, and fast batches of random updates.

We don't need a server/client architecture, as we can't have multiple applications accessing the database at once anyway: inconsistencies in the database would lead to forking risks (every node in the network needs to make exactly the same judgement about what is valid and invalid).

| improve this answer | |
  • 3
    Has LMDB been considered? – stefanwouldgo Nov 9 '16 at 0:49
  • @Pieter Wuille Do you consider that moving from BDB to LevelDB was a correct decision? Or LevelDB is causing major problems that didn't happened with BDB? (ex: file corruption, bugs ...) – Mark Messa Feb 16 '17 at 4:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.