4

As lightning network channels route payments, channel updates require the 2 channel peers (Alice & Bob) to do a round of signing and trade commitment transactions for the most up-to-date channel balance.

Is this the primary reason that the channel.db file may grow so dramatically?

For each update, Alice must keep a full record of all revocation keys from all past commitment transactions in case Bob broadcasts an invalid past channel state. If these keys are deleted, Alice would not be able to create & broadcast the correct penalty transaction to punish Bob.

Database compaction is typically run on startup of the node, but what exactly is involved in that compaction? Are the revocation keys being indexed / compressed in some lossless/recoverable fashion?

Do other implementations than LND have the same process?

1 Answer 1

4

The compaction is only relevant for bbolt database files, which are very common with lnd. In this case, the channel database is stored as a bbolt file named channel.db. Note that you can also use PostgreSQL with lnd to persist this information, in which case you don't need compaction (PostgreSQL has a concept named "vacuum" which can run in the background).

During operation of lnd changes (especially deletions) to the bbolt database cause gaps in the database file, which cannot easily be filled with new data. As such, after a while, the database file consumes lots of disk space, even though a fraction of it is just empty space inside the file. Deduplication/compression features of your filesystem might help to reduce the actual storage footprint.

When you trigger the compaction, a new database file is created and filled with the data from the original file by copying everything over. This way the actual information is identical, but gaps in the original file are skipped and do not consume space in the resulting file. This is somewhat similar to file system defragmentation from the Windows 9x (and before) era.

This compaction does not change (or interpret) the data in any way, i.e. no data is compressed or deleted. Every information persisted in the original database is copied over to the new database, bit by bit. The only difference is that the internal database structure changes, which you do not see from the outside (aside from the intended benefit of reducing the storage footprint of the file).

As compaction is bbolt specific, I don't think other implementations make use of this. In general all databases should already have some means of freeing up space of deleted data, and I guess that all node implementations delete most unnecessary information after a while.

2
  • How does the compacting algorithm work? On my node, during compacting a 60G channel.db, a file temp-dont-use.db is created. Its size grows up to 20G and then drops to about 0-1G. Then the process continues several times. If it was just defragmentation, a single linear scan should have been sufficient. Commented Aug 30 at 14:12
  • 1
    That's part of my answer already.
    – C-Otto
    Commented Aug 30 at 16:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.