What is the purpose of full nodes storing merkle roots? Why don't they save the transactions only? The hashes could be calculated only when a lightnode asks for them and the merkle roots could be saved only be the lightnodes. Am I missing something here?
4 Answers
I suspect you are confusing merkle roots and merkle trees.
A merkle tree is the binary tree of hashes. The leaves are individual transactions, which are hashed, and then the hashes hashed together until only one hash remains. That remaining hash is the merkle root. A merkle branch is a particular path down the merkle tree from the merkle root to a transaction.
Full nodes store the merkle root because it is part of the block header. The block header contains the merkle root so that the block hash itself will also commit to the transactions contained within the block. While the merkle root itself isn't generally used elsewhere, as part of the block header, it frequently needs to be sent out to other nodes which are downloading blocks. It is expensive to have to recompute the merkle root every time, so instead we just keep it so that blocks can be sent out quickly.
Furthermore, by storing the complete block header, when the transactions themselves are discarded in pruning, the block headers are kept and so that the header chain is still tracked. Without the transactions, it would not be possible to verify this header chain again if the merkle root were not being stored.
Full nodes do not store the entire merkle tree or any merkle branches. These are unnecessary for normal operation. When a SPV client asks for a merkle branch, it is computed on the fly.
answered Aug 21, 2019 at 18:09
-
I suspect you didn't undestand the question. I don't confuse trees and roots. Regarding the final panagraph of your answer, I thought that a full node stored the whole array of transactions and the merkle root. Why does it store the root? Why is it not calculated on the fly like branches?– evannemoCommented Aug 21, 2019 at 19:03
-
1That's because the root is part of the block header, and block headers are frequently needed (far more often than the transactions). They could be computed on the fly, but it would slow things down tremendously, while saving just around 18 MB (at time of speaking). Futhermore, this isn't possible at all for pruned nodes (which still need the headers of old blocks, despite not having their transactions anymore). Commented Aug 21, 2019 at 19:13
-
@PieterWuille If you read the question, I talk about FULL nodes not needing to store the merkle root, not prune nodes. I know that block headers contain the merkle root,what I am asking is WHY the need it.– evannemoCommented Aug 21, 2019 at 20:48
-
1Pruned nodes can be full. There is no requirement to have all blocks and transactions in order to validate future blocks. The term full nodes refers to the fact that they validate fully; not whether they keep all historical data around. Commented Aug 21, 2019 at 20:49
-
1The Merkle root needs to be part of the block header (whether it's stored by nodes or not) for security; otherwise blocks wouldn't commit to their transaction contents. It doesn't technically need to be stored by nodes as it could be computed on the fly, but as I explained, this would massively slow down all operations involving block headers, and only save 18 MB for non-pruning full nodes (out of 250 GB). It's simply not a worthwhile optimization. Commented Aug 22, 2019 at 2:15
Full nodes do not maintain the blockchain only for servicing lightweight nodes. The primary reason is for its own security so as to ensure that no blocks that it has received over the network have been tampered with and they all meet the consensus requirements.
In order to verify that the blocks are linked going back to the genesis block, the full node will have to calculate the block header. Block header of a given block contains the previous block header hash and merkle root amongst other fields. The merkle root acts like an identifier to all the transactions included in the block as changing even a slightest component of the transaction can completely change the merkle root. So a full node verifies the transactions, and calculates the merkle root from those transaction hashes. Then this merkle root is used amongst other block header fields like previous block header hash to calculate the current block header hash. When verifying the next block in the blockchain, the node will ensure that the previous block header hash field in the block matches the one it had calculated for the last block. This allows the full node to ensure that all the blocks are linked in a chain without any tampering.
Moreover, the full nodes sync with one another headers first. In simple words, when a node boots up (at start of even after a pause) it would sync with the rest of the network syncing the headers of the blockchain first. And merkle root is part of the block header.
answered Aug 21, 2019 at 17:49
-
All right. But this still doesn't answer why the merkle root must be saved and not calculated on the fly during the verification/validation process.– evannemoCommented Aug 21, 2019 at 17:54
why the merkle root must be saved and not calculated on the fly during the verification/validation process
The merkle root is a part of the block header. When mining, it is the block header that miners must construct and then hash, in an attempt to find a valid block. So when mining, a miner must commmit to the transactions that will be included in their blocks. By this mechanism, after a valid proof-of-work has been found, a miner cannot alter which transactions are included in the block.
In other words, the merkle root ties the transactions in the block to the proof-of-work, thus securing the history and ordering of transactions in a way that is relatively simple and computationally non-intensive.
-
This doesn't answer my question. Why does the block header have to store that? Why isn't it calculated on the fly by the transaction array on the full block? The value would be the same either stored or unstored.– evannemoCommented Aug 21, 2019 at 21:00
-
I do think this answers the question: Without storing it in the header, there is no cryptographic commitment tying the transactions to the proof of work. The header is what the miners hash to find a valid proof of work, so it include the merkle root. Without the merkle root, the miners could just find a valid PoW and then append any block of transactions onto it. So even though the root can be computed on the fly, that doesn't solve the issue of needing a cryptographic commitment for transactions. That is the whole point of blocks: to commit to an ordering and history of transactions.– chytrikCommented Aug 21, 2019 at 21:08
-
The hash of previous block header could calculate the merkle root of the transactions of the previous block from the transactions of the previous block. The merkle root can still be included in the cryptographic function, without being stored on the header. Am I wrong?– evannemoCommented Aug 21, 2019 at 21:10
-
No, if the merkle root is not included in the header, then the previous block's hash would not provide any sort of commitment. The hash (proof of work) is calculated from the block header alone, there is no way to "add the transactions to it later".– chytrikCommented Aug 21, 2019 at 21:14
-
1Without the merkle root being explicitly included in the header, the transactions in that block will not be committed to. So there would be no way to verify that a certain set of transactions is the set that is included in a certain block. This commitment must happen before a valid PoW is found, what you are suggesting is to somehow add/calculate it afterwards. The whole point of the merkle root is to make this commitment in advance.– chytrikCommented Aug 21, 2019 at 22:25
I’ll add a second answer, that is an analogy instead of a technical explanation. As analogies go, it isn’t perfect, but I think it works well enough.
Consider a situation in which you must fill in a lottery card, by picking 5 numbers. This lottery card is then stamped by the lottery authority, so that you cannot change your numbers later to claim a false win. Obviously, it is critically important that you commit to the numbers before the lottery draw takes place. If you could change the numbers later, then the lottery would be broken.
This is similar to bitcoin mining: selecting numbers is like selecting transactions, and then the ‘stamp of approval’ is like the merkle root being committed to the header that hashes to a valid proof of work. Finding a valid proof of work is the mechanism by which an ordering of transactions is selected.
So, without the merkle root being included in the header, the proof of work would not be related to the transactions at all. It would be like miners were just submitting blank lottery cards, that they could change the numbers on later. Since the point of a blockchain is to commit to an ordering of transactions, it is important to commit to the transaction ordering prior to the valid block being found. There is no way to do this afterwards, in a cryptographically secure manner.
-
The proof of work can calculate the merkle root from the transactions and then it can include it in the hashing function. The algorithm remains the same. The merkle root is just not stored, but calculated every time on the fly. Am I wrong?– evannemoCommented Aug 21, 2019 at 22:58
-
1Doing that would break several functions, without any real gain. The extra bandwidth and storage required to include the merkle root in the header is very small, compared to the benefit of being able to do things like header-first validation, light-wallets, etc. Note that nodes do recompute the merkle root for each block anyways, as one check to ensure a block is valid. But including the merkle root in the header provides many additional benefits (rather than using it in the PoW computation, but not explicitly storing it, as you suggested).– chytrikCommented Aug 21, 2019 at 23:39