4

In particular, when a new block arrives:

  1. Is the block written to disk in an atomic fashion, so a new block either exists, or it doesn't? (reader can never see a partially written state)

  2. Are the summary databases (eg chainstate, block-index) written to, and flushed to disk in an atomic fashion along with the new block, such that the blockchain filestore will never be out-of-sync with the levelDB filestore(s)?

  3. More generally, can the bitcoin datastores be said to exhibit ACID properties?

  4. Are there any plans to change the datastore behavior in this regard?

(I believe the answers to 1-3 are: no, no, and no. But I'd like to hear more details, and also rationale for the status quo).

1 Answer 1

2

Yes, yes, yes(*), and no.

The blockchain-related on-disk data Bitcoin Core keeps generally fall into these three categories:

  • The block data (flat blocks/blk?????.dat files with concatenated blocks).
  • The block index (LevelDB database blocks/index/*).
  • The UTXO set (LevelDB database chainstate/*).

In this list, the later entries contain references to the previous ones. The chainstate contains the hash of the block that is currently considered "active" (i.e., the UTXO corresponds to it). This hash must exist in the block index at all times. Further, the block index stores in which block file and position within that file each block is stored.

To maintain consistency:

  • Whenever block index writes to disk happen, the currently open block files are flushed. It is possible that bitcoind crashes during the block file write, which may leave a partially written block on disk, but since this block will not be written to the block index, after restart, it will be as if this block wasn't received at all; it will be redownloaded, and written to disk again (possibly in the same file byte range that the earlier attempt was written to as well).
  • Block index writes are done using a LevelDB batch(*), which either makes it entirely to disk, or not at all.
  • Whenever chainstate writes to disk happen, the block index is flushed.
  • When the chainstate is written to disk, and there is a large number of modified entries, the write is split up into multiple LevelDB batches, to reduce peak memory usage during writing. Before this happens, a "flush in progress" entry is written to the database, which is removed after the flush completes. If bitcoind crashes in the middle, the chainstate will be in an inconsistent state, but on restart the "in progress" record will be detected, and the relevant range of blocks will be processed and applied again to the UTXO set.

(*) There have been reports of LevelDB itself causing corruption of its on-disk files if interrupted during a write, on some platforms.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.