3

The primary reason Bitcoin's C++ reference implementation uses Berkeley DB 4.8 seems to be backwards compatibility. It doesn't appear that BDB4.8 (or BDB at all for that matter) has any superior advantage over current-day alternatives like Redis or RocksDB, at least based on the limited research I've done (source). To the best of my knowledge, most alternative full-node implementations steer away from wallet support (source, source).

Performance benchmarks seem to indicate that there is no performance benefit to sticking to Berkeley DB (source) yet I cannot seem to see any widespread adoption of anything else except for recent introduction of SQLite support (source) for Bitcoin Core despite the growing pains of BDB4.8. I can't imagine any other reason as to why BDB wasn't dumped altogether if it wasn't for backwards support.

Is there any reason why alternatives like RocksDB (which supports at rest encryption though it isn't well documented), ArangoDB or Aerospike (all that seem to have existed at least five years ago) were not considered? Is there any security benefit to sticking with BDB? Any redeeming properties of BDB that make it a definitive choice over what is available currently?

Should a new implementation stick to BDB or use another library for wallet support and if so, what should it be?

Improve this question
5
  • 1
    There are a couple of questions here that at least partly answer your question (not the historical decision part though) At least if your needs are similar to the Core wallet today you would probably choose to use sqlite for the wallet rather than BDB bitcoin.stackexchange.com/questions/51435/… bitcoin.stackexchange.com/questions/99620/…
    – Michael Folkson
    Commented Feb 12, 2021 at 16:32
  • I'm familiar with both answers, the first one explains the transition away from BDB for blockchain storage and simply reinforces my view that BDB was stuck solely for wallet compatibility. I agree with the rationale of the second answer but why'd it take so long? BDB was a known pain for a long time. I guess in a nutshell I want to know if RocksDB/ArangoDB be used for both blockchain and wallet storage because moving away from BDB seemed long due.
    – Kittywhiskers Van Gogh
    Commented Feb 12, 2021 at 16:43
  • 2
    Backwards compatibility is a massive priority for Core and it is generally very conservative in its technology choices. I don't think it would consider a new (database) technology until it had been proven over many years on other projects. It is providing a consensus engine and infrastructure and doesn't have the same tolerance as startups or bigger corporations for cycling through different options and experimentation if it is going to endanger backwards compatibility or require many developer hours to revamp shortly afterwards.
    – Michael Folkson
    Commented Feb 12, 2021 at 16:57
  • That's what it seemed like, I know the maintainers went so far as to create a PPA just to have libdb4.8-dev as a package on Debian-based systems and can't imagine any other reason as to why they'd continue with BDB. I was wondering if there were technical advantages of BDB that I might've looked over or if there were non-support related reasons as to why BDB was stuck to for so long. Thanks for your help! <3
    – Kittywhiskers Van Gogh
    Commented Feb 12, 2021 at 17:02
  • No problem. If that answered your question I'll add it as an answer.
    – Michael Folkson
    Commented Feb 12, 2021 at 17:08

2 Answers 2

Reset to default
2

Backwards compatibility is a massive priority for Core and it is generally very conservative in its technology choices. I don't think it would consider a new (database) technology until it had been proven over many years on other projects. It is providing a consensus engine and infrastructure and doesn't have the same tolerance as startups or bigger corporations for cycling through different options and experimentation if it is going to endanger backwards compatibility or require many developer hours to revamp shortly afterwards.

Improve this answer
1

I can't imagine any other reason as to why BDB wasn't dumped altogether if it wasn't for backwards support.

There simply is none. The wallet file doesn't need to be a database, it could be a flat text file and that would operate just as well. BDB will remain for compatibility for the foreseeable future, and change is risky, so it remains.

Improve this answer
1
  • 2
    Perhaps worth pointing out that BDB is only used for wallet files, which tend to have extremely long lifetimes (just search this site for people trying to get their funds out of wallet.dat files that are a decade old). For other purposes LevelDB is used, and incompatible changes to those databases have been made several times, for performance or other reasons. It's only for the wallet file that it really doesn't matter what format is used (it's loaded entirely into memory, at least for now), and compatibility is paramount.
    – Pieter Wuille
    Commented Feb 12, 2021 at 18:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.