What exactly is happening in src/validation.cpp and what is saved in mempool.dat, when is this data required, what happens if there is an issue saving or loading data from the file?
The current mempool of a node is kept in RAM during node runtime. As node operators, miners, and users we want transactions that are in our mempool and our own priority for these transactions to be persisted across node restarts. Imagine you are a mining-pool operator and you lose the contents of your mempool. After a restart, you'd be at a big disadvantage compared to other mining-pool operators who can choose from a broader set of transactions to include in their proposed block.
The mempool.dat file is a snapshot of your mempool at a given time. It's read with LoadMempool() on node startup and transactions are added to the node's mempool. On node shutdown or via the RPC savemempool the DumpMempool() function is triggered. This writes a current snapshot of the nodes mempool into the mempool.dat file.
Warning: The file format is internal to Bitcoin Core and can change without deprecation nor prior notice. The following describes the contents of a version 1 mempool.dat file (see the constant MEMPOOL_DUMP_VERSION).
The mempool.dat file contains a header that consists of a uint64 version number and the number of transactions contained in the file stored in a uint64 as well. This is followed by a vector of mempool entries. An entry consists of the raw transaction, the time it entered the nodes mempool as int64, and a fee-delta as uint64 used for manual transaction prioritization for miners. The vector of entries is followed by a serialized map of txid -> amount which stores miner transaction prioritization information as well.
On start-up, the Bitcoin Core node tries to import the transactions in LoadMempool(). Some transactions are ignored during import:
- transactions that expired in the meantime (transactions default to expire after two weeks unless the node configuration is changed).
- transactions that already confirmed in the meantime
- transactions that are already in the mempool (e.g. a peer already send it to us before we could import it)
Usually, something like this can be found in the debug.log file:
Imported mempool transactions from disk: 463 succeeded, 10 failed, 0 expired, 0 already there..
If the file is damaged or does not exist no transactions are loaded to the mempool. If the file can't be written during a shutdown, then the node simply can't load any transactions during its next start-up. This is only a real issue for mining-pool operators, but not normal users.
I've held a presentation on theLoadMempool() and DumpMempool() functions at the 2019 Chaincode Sumer Residency. The slides can be found in Google Slides or as PDF.
