Considering how large the total blockchain is relative to average download speeds, why were the blocks never data compressed before being sent?
Why could this not be implemented on a new alt?
1 Answer
Because the data is already serialized in a pretty compact way and large parts of that data are hashes which are effectively random numbers as far as compressability is concerned. (Random data is not compressable.)
Also when you search for for example a transaction id in an index, it will point you to the exact location in the blockchain. Having to read a whole bunch of data around that area and decompress it before being able to access the transaction that you wanted, adds disk I/O, CPU overhead and memory consumption. That severely limits the dictionary size any compression algorithm could use, and therefore the compression ratio.
Give it a try by zipping the whole blockchain file, but remember that any usable live compression would have a lower ratio than what you would find.
-
Thank you Jannes! I definitely agree that they should not be stored compressed, and thank you especially for showing me the futility of compressing hashes, but considering that the vast bulk of the data in a block is the transaction array, do you still think that that section can't be compressed? Thank you so much in advance!– user5107Commented Dec 27, 2013 at 5:21
-
Nevermind. Transactions are sent as signed hashes, not raw data. You're right, there probably isn't too much benefit in compressing the json keys. Thank you!– user5107Commented Dec 29, 2013 at 3:16