0

In this case (which has actually happened to me), if I sync all blocks, they fill up the whole file system table of the volume.

What is the best way to solve it? Using one directory that is split over multiple volumes? How can you achieve that? Using symlinks or junctions?

There is also a limit of files per directory. This means that the entire copy of the blockchain database should be split over a double amount of directories each time it reaches this threshold (like dynamic hashing).

I tried to search for solutions that address these issues, but I simply couldn't find anything, like nobody has ever thought about these situations.

  • Which node software stores every block in a separate file? Bitcoin Core doesn't do that - it groups them into 128MB chunks, which should make this a non-issue. – Nate Eldredge Jan 14 '18 at 15:34
  • @Nate Cardano node. – Moon Jan 24 '18 at 11:27
1

This is an issue of inodes, a feature mainly in Unix filesystems (like ext4) which is a fixed-size table storing metadata of all files on the partition. When the partition is created, a fixed number of inodes is created.

On my 500GB drive using the default ext4 from Ubuntu, I have 27 million inodes (this can be checked using df -i if you're on Linux), many times more than the number of Bitcoin blocks in the blockchain. Thus I don't run into this issue, and I imagine most people don't - it's possibly something to do with the specific filesystem/setup you're using.

ext4 specifically doesn't have a cap on the number of files in a directory, so splitting the files into multiple directories would be unlikely to help. FAT32 however has a limit of 65,534 files per directory, the solution there is not to use an outdated filesystem like FAT32.

Generally there are two ways to approach this sort of issue:

  1. Increase the resource (make more inodes)

This can be done by specifying a larger number of inodes when creating the partition, which is relatively easy to do (example) but requires foresight that you're going to be running a node on the partition. Alternatively, make another partition with the required number of inodes.

  1. Use the resource more efficiently (use fewer inodes).

Perhaps the simplest way to achieve this is to just stick multiple blocks in a file - I'm not sure how this would work from an architectural point of view but it's something that would have to be implemented in the client.

  • Perhaps the simplest way to achieve this is to just stick multiple blocks in a file - which is exactly what Bitcoin Core does. It just concatenates blocks together until the file size reaches about 128 MB, then starts a new one. There is a leveldb index to keep track of which block is at which offset in which file. – Nate Eldredge Jan 14 '18 at 15:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.