What are the use cases where very old rev*.dat files are needed?

Question

There have been multiple questions on this but none, IMO, really get to the issue:

Given:

1. The rev*.dat files hold "undo" data for rolling back the UTXO state, block by block,
2. and they're 1::1 with block files (i.e., for a given NNNNN the files blkNNNNN.dat and revNNNNN.dat hold information for the same blocks),
3. and they're written and chunked in the order in which blocks are received by the node (from the network)
4. and the UTXO set is computed (can only be computed) by considering blocks in block-chain order,
5. and in any block the coinbase transaction is unspendable for 101 confirmations (by consensus) which is so that reorgs of up to 100 blocks don't cause serious problems with coins suddenly disappearing, and 100 blocks piled on top is apparently long enough to outwait any reorganization ...

Then: Are very old rev*.dat files ever used? Say, those that belong to blocks buried more than 100 blocks deep from the top of the chain? If they are ever used what is the use case?

(and, w.r.t. the answer here of the first question in the following list, what is the database corruption that occurs - presumably to the UTXO state - if the rev files are all deleted and why is it necessary to regenerate them?)

(bonus question: given a txindex: is there anything in the rev*.dat file that can't be regenerated simply by looking at the blocks in the corresponding blk*.dat file and using the txindex to find previous transactions?

Other questions here approaching this question but not dupes:

• "Are old rev*.dat file safely deletable?" - closest to being on point, the accepted answer explains (without really explaining) that if you delete them it "effectively corrupts the databases" and also that they'll be recreated if they're all deleted. But doesn't say why they need to be kept indefinitely, which is what I'm asking.
• "What exactly are the rev00000.dat files?" - directly answers what they're for but does not answer why they need to be kept forever
• "File format -- rev*.dat" - specifically about the format of the contents of the rev files
• "What's raw undo data for?" - sadly, the only answer doesn't even answer the question asked but is in fact a cut/paste of the answer to the file format question #57978 directly above in this list - yet, it got the OP's checkmark, go figure...
• "Undo files format change and reorg process question for v0.19" - extremely detailed answer about the format of the contents of the rev files
• The tag "undofiles" - 3 questions (before this one) - 2 are listed above and the other is someone's one-off operational problem (blank blk and rev files).
• This answer is very good as to when rev*.dat files are used (blockchain reorgs) but doesn't address why they're kept forever.

Answer 1

2. and they're 1::1 with block files (i.e., for a given NNNNN the files blkNNNNN.dat and revNNNNN.dat hold information for the same blocks),
3. and they're written and chunked in the order in which blocks are received by the node (from the network)

This assumption is incorrect. The rev*.dat files are actually written in height order. This is because they are written at a different time. The undo data for a block is only written after it has been connected to the chain tip, so the undo data ends up being in the order that blocks are connected. So for blocks which are downloaded but never connected, undo data does not exist for them (although this case is unlikely as blocks aren't downloaded unless they are supposed to be connected).

Then: Are very old rev*.dat files ever used? Say, those that belong to blocks buried more than 100 blocks deep from the top of the chain? If they are ever used what is the use case?

When it comes to the operation of the node, no, old undo data is not actually being used. However it does have some uses outside of node operation, particularly in examining the blockchain via RPC, and the creation of the block filter and the coin stats indices.

The RPC uses undo data in some places because it contains the UTXOs that were spend by that block. This allows getblock to calculate the transaction fees paid for each transaction in the block. However this is not a hard requirement, and if the undo data is not found, then it simply doesn't calculate the fees.

Another RPC that uses the undo data is getblockstats which uses the undo data to also calculate the fees for the block as well as the change in size of the UTXO set that the block causes. If the undo does not exist, then this RPC would fail.

For both the block filter and coin stats indices, the undo data is used to build them because it provides a snapshot of the UTXO set changes made. This allows the index to be populated without having to track the UTXO set while it is being built, so it reduces the memory usage and increases performance.

For node operation, undo data is only needed if the block were to be disconnected from the tip. This only occurs when reorgs happen. However Bitcoin is built with the possibility that there could be an extremely large work reorg that could reorg out things that are considered ancient history.

However, old undo data is indeed not used in operation of the node, and there could be a mode added which deletes them. On my node, this totals to 48 GB of data, so the space savings are not nearly as significant as normal pruning.

(and, w.r.t. the answer here of the first question in the following list, what is the database corruption that occurs - presumably to the UTXO state - if the rev files are all deleted and why is it necessary to regenerate them?)

They need to be regenerated because of how the block index works. Since it also contains the location of the undo data, if that data were to be missing, the block index will find that to be an error and thus require a reindex. If an option were added to allow the deletion of rev*.dat files, then that would obviously have to be changed.

(bonus question: given a txindex: is there anything in the rev*.dat file that can't be regenerated simply by looking at the blocks in the corresponding blk*.dat file and using the txindex to find previous transactions?

Yes, but that would be pretty slow.