I understand how transactions are grouped into blocks. However, I don't understand why this is necessary in the first place. If it's a mining thing, I don't see why the miners can't just update the transactions with some metadata and leave it at that. Then the blockchain would simply be a cache on top of the transactions. I don't see the real need for a block to exist if there are the transactions, other than it being a sort of cache.
The purpose of the blockchain is to prevent the double-spending of coins by ensuring that there can be at most one valid transaction which spends some known coins. The only way to ensure this is that validators must be aware of all valid transactions.
While it is possible that someone may attempt to double-spend some coins, only one of the transactions may make it into the chain of valid blocks - which serve as an immutable record of which transactions were accepted as the correct ones. They are immutable because even the change of a single bit in a transaction would alter its transaction id, which in turn would alter the block's merkle root, and in turn, would require that all subsequent blocks have their block hash recomputed, because they all refer to a hash of the previous blocks header, which contains its merkle root.
The way all validators agree on which chain is the correct chain is by always taking the chain with the most proof-of-work done on it. This is necessary to keep the ledger immutable, because it means that an attacker attempting to modify a transaction, and thus, needing to recompute the merkle root and the hashes of subsequent blocks, would be constantly playing catch-up with the longest chain, and if they have less hashing power than the other honest miners who are working on extending the longest chain, they will not possibly be able to catch up and generate the chain with the most proof-of-work.
I understand how transactions are grouped into blocks. However, I don't understand why this is necessary in the first place.
The goal of mining would be to make transactions immutable, and to reward honest ("cooperating") miners that don't reorganize the chain. When is it improbable for the chain to be reorganized? When none of the miners own no more than 50% of the network hash rate, and when the largest pools are expected not to form a cartel. Thus, a high number of mining pools is good for Bitcoin. To achieve that, small or medium scaled miners with enough hashrate must have no or little disadvantage. To put another way, while large scale miners have their own ISPs, relay networks, and maybe transatlantic cables (exaggeration) to deal with latency, latency is a huge problem for small-medium miners. If we wanted to eliminate small-medium miners (for no reason), there are two things we can change:
- Remove the block size limit. (affects the amount of data transferred at once)
- Decrease the average 10 minute inter-block interval. (affects the frequency of transferring data)
If every transaction was mined separately, small-medium miners would have a large rate of orphaning, and they would therefore go out of business.
But, IOTA and NANO treat each transaction as a separate block, which is something I don't know very well.