Skip to main content
17 votes
Accepted

Migration from Berkeley DB to LevelDB

As someone who was involved in doing that migration at the time, I believe it was the right decision. LevelDB is far from perfect, but I wouldn't know what else to use. In particular: BDB is much ...
Pieter Wuille's user avatar
14 votes
Accepted

Why is Bitcoin Core using LevelDB instead of Redis or SQLite?

Redis and LevelDB solve very different problems. We tried using SQLite and its performance was abysmal. Bitcoin Core needs a database to store the set of unspent transaction outputs (UTXOs). This ...
Pieter Wuille's user avatar
10 votes
Accepted

Why is the Bitcoin Core wallet database moving from Berkeley DB to SQLite?

The introduction of descriptor wallets presents an opportunity to introduce a new database backend as descriptor wallets are backwards incompatible. The following is taken from Andrew Chow's blog post ...
Michael Folkson's user avatar
9 votes

Why are blk*.dat files ~134200000 bytes?

The constant is called MAX_BLOCKFILE_SIZE and is set in src/validation.h. It is currently set to 0x8000000 which is 128 MiB (134,217,728 bytes). You can see where it is checked in FindBlockPos() ...
Nate Eldredge's user avatar
7 votes
Accepted

What database engine does Bitcoin and other top altcoins use?

It depends on the implementation. The network itself does not have a database engine. Bitcoin Core and software derived from it (most altcoin software is derived from Core) uses LevelDB to index the ...
Ava Chow's user avatar
  • 71k
6 votes
Accepted

UXTO DB structure

Each entry in the new 0.15 format is defined as outpoint:coin, and has the following structure: Outpoint is formed by: key | tx_hash | index. Where the key corresponds to b'C', or 43 in hex. The ...
sr_gi's user avatar
  • 3,220
6 votes
Accepted

How long is a Bitcoin?

A bitcoin is not a piece of data, and therefore does not have a length. A bitcoin, the unit of currency, is just like any other unit of measurement. It wouldn't make sense to ask "how much data is a ...
Jestin's user avatar
  • 8,812
6 votes
Accepted

API for large number of queries to different addresses every few minutes

You can use a self-hosted (but low-burden!) instance of Esplora. Blockstream is also exposing an instance for free (at the moment) at blockstream.info. Disclaimer: if you take part of Bitcoin as an ...
Antoine Poinsot's user avatar
6 votes
Accepted

Efficiency of storing/querying transactions in a relational database

Google BigQuery already maintains a public dataset for Bitcoin, with tables optimized for querying transactions and blocks. You can find the dataset here: https://console.cloud.google.com/marketplace/...
josie's user avatar
  • 681
6 votes

What data does the Bitcoin Core wallet search for during a blockchain rescan?

Does the wallet database store a table of all the actual output scripts (i.e. witness programs) derived from its keypool? Yes, exactly. The descriptors (and for legacy wallet, legacy data) is ...
Pieter Wuille's user avatar
5 votes

Is a private blockchain better in any sense than a database?

Researcher Arvind Narayanan stated his opinion on this matter in his post “Private blockchain” is just a confusing name for a shared database: It is true that adding signatures and hash pointers ...
dubek's user avatar
  • 151
5 votes
Accepted

Bitcoind Node crashes after using -dbcache option

You are allocating far too much dbcache. If you allocate 4000 MB to the dbcache, you will use up all of your RAM and it will crash. Your operating system needs RAM, Bitcoin Core itself needs RAM ...
Ava Chow's user avatar
  • 71k
5 votes
Accepted

What data does the Bitcoin Core wallet search for during a blockchain rescan?

Does the wallet database store a table of all the actual output scripts (i.e. witness programs) derived from its keypool? Kind of. For descriptor wallets, the set of all scriptPubKeys (up to the ...
Ava Chow's user avatar
  • 71k
4 votes

Where can I learn how to build a blockchain from scratch?

There was a recent presentation by Portia Burton going over some of the basic steps in creating your own custom blockchain based on Ethereum, however it is in Node not Python as you are requesting. ...
m1xolyd1an's user avatar
  • 5,666
4 votes

Why was Bitcoin Core in need of a fork of LevelDB?

Sometimes LevelDB has bugs and those bugs sometimes are not fixed in a timely manner in the upstream LevelDB project. These bugs can be problematic for Bitcoin Core so a fork of LevelDB was created to ...
Ava Chow's user avatar
  • 71k
4 votes
Accepted

How to save the bitcoin core (bitcoin-qt) database for later?

Ok, I think I got it. Here is a short guide based on this bitcoin wiki article. Notes: This does not concern backing up your keys. If you lost your wallet, or your keys, and you made the backup ...
talz's user avatar
  • 171
4 votes
Accepted

How to store private key in MySQL database?

Storing private keys in an online server is not best practices (even if you quickly transfer to an offline wallet). Generally, you should not store private keys on an online device Instead, if you ...
chytrik's user avatar
  • 18.3k
4 votes

What database tables structure do bitcoin uses?

There simply is none. There's a key value store in LevelDB for the UTXO, and for miscellaneous things like the locations of block files on disk. Relational databses are far too slow for the task, and ...
Claris's user avatar
  • 15.4k
4 votes

Getting "Unable to find DB driver for sqlite3" when trying to start c-lightning node

Most likely this is due to a mismatch between the sqlite3 version that c-lightining was compiled with and the version your OS currently has installed. With the recently release v0.10 2, c-lightining ...
cdecker's user avatar
  • 9,508
4 votes
Accepted

explore sqlite wallet

The data stored is binary data, which is why it comes out garbled. It is not text. So you will have to convert the data to something human readable. One such method is to convert it to hex. One such ...
Ava Chow's user avatar
  • 71k
4 votes
Accepted

How can one examine the contents of the wallet.dat file using sqlite3?

Why does wallet.dat need to be locked permanently if bitcoind is running? bitcoind does not retrieve data from disk when it is running. It only retrieves data on loading, and will occasionally write ...
Ava Chow's user avatar
  • 71k
3 votes
Accepted

Why was Bitcoin Core in need of a fork of LevelDB?

There are two reasons for having the fork in the first place: Upstream LevelDB does not support Windows, and the branches that are available elsewhere don't support MinGW (which Bitcoin Core uses for ...
Pieter Wuille's user avatar
3 votes

To create a blockchain using MySQL database

I believe you are over complicating your life. Having a BlockChain (a Linked Ledger) in SQL can be prototyped in the following manner Create a table, such as: CREATE TABLE IF NOT EXISTS ...
Greg Vitetzakis's user avatar
3 votes
Accepted

Why does Bitcoin store keys in a database?

Historically bdb was used for the UTXO as well, so you could justify using it for the wallet as well for consistency. It makes absolutely no sense today, other than that it will always need to be ...
Claris's user avatar
  • 15.4k
3 votes
Accepted

What is in the Bitcoin Core LevelDB dbcache? Is it full records or metadata?

The Bitcoin Core documentation describes dbcache as the cache for the UTXO database. Wikipedia describes a cache as a: software component that stores data so that future requests for that data can be ...
Michael Folkson's user avatar
3 votes

Are the block headers kept in memory?

I think you are confusing what it means to store things "in memory". As you say, indeed, the block index DB and the chainstate DB are both levelDB databases stored on disk. However, while ...
meshcollider's user avatar
  • 11.9k
3 votes

How to fix “Error opening block database”?

May you have corrupted files due to shutdown. I’ve suggest you to take a look at the debug.log file for further information. First try the -reindex option if you continue to have ...
Pegasus's user avatar
  • 121
3 votes
Accepted

What is involved in the "database compaction" of a LND node?

The compaction is only relevant for bbolt database files, which are very common with lnd. In this case, the channel database is stored as a bbolt file named channel.db. Note that you can also use ...
C-Otto's user avatar
  • 452
3 votes
Accepted

Why isn't the backed up sqlite3 exactly same as the lightningd.sqlite3?

The replication using the --wallet=sqlite3:///file1.db:/file2.db works by applying the SQL statements against both databases in sequence. The main replica gets is executed first, followed by the ...
cdecker's user avatar
  • 9,508
3 votes

What is the data format layout for txindex LevelDB values?

You are confusing the VarInt and CompactSize encoding. CompactSize encoding (also confusingly referred to as "varint" outside Bitcoin Core) is used in the Bitcoin protocol. VarInt encoding ...
Antoine Poinsot's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible