Why does Bitcoin apply a difficulty function to block hashes and then use the output of this function to add up and determine the "longest" chain?
Why not just e.g. add together all block hashes (as bigints) and consider the longest chain to be the one with the biggest or smallest sum?
Is there some mathematical reason for this or is it just a performance optimization or a way of avoiding excessive dependence on big integer arithmetic?
Background info:
The block hash must be less than a certain value (as defined by the difficulty function) in order for the block to be valid.
Generally, as time has progressed, network difficulty has increased. So as the blockheight has increased, the cutoff value for a valid blockhash has decreased.
—————
Why not just e.g. add together all block hashes (as bigints) and consider the longest chain to be the one with the biggest or smallest sum?
I think it may be easiest to explain why this sort of implementation is broken by giving an example. Let’s consider a system following the ‘smallest sum for a given blockheight’ rule:
—————
For simplicity, let’s imagine a pow function that has an output value range of 1-100. Let’s say the network difficulty is currently set so that any output of 10 or less is valid.
Now, miner A finds a block at height X, with blockhash 10. This is valid, and so it is added to the chain.
Miner B sees this, but continues to mine blockheight X, because if they can find a block with hash 9 or less, then their new chain will replace the previously accepted one. By forcing a reorganization this way, they may be able to successfully commit a double spend attack, etc.
—————
This example shows how a miner could game the algorithm to rewrite the blockchain. The same is true for a ‘largest sum for a given blockheight’ rule: if a miner finds a small blockhash (eg. Blockhash = 5 in the example above), another miner could profit by continuing to mine on the older block, forcing a reorganization if they find a block with a larger, but still valid, blockhash.
For the bitcoin network to function properly, the game theory needs to incentivize miners to always mine at the chain tip. Using a difficulty function instead of the explicit blockhash value helps ‘level the playing field’, so to speak. This is because any valid blockhash is no better or worse than any other valid blockhash.
The "apparent" work is 2x higher than actual work due to luck, but the reason it's good bitcoin doesn't use the apparent work is because it has bad stability properties due to variance, and bad game theory interactions.
Imagine Bitcoin were worked the way you suggested-- Say, for example, you mine a block and by chance it has 10x the 'work' expected. It would be best for you to not announce that block but instead continue to try to extend it in private. If the publicly disclosed work starts getting close to your lucky block, then you announce it. This is a pretty bad incentive.
The extreme of this is that you start trying to mine at block 1, and hope by luck to end up with enough "apparent work" to wipe out the whole chain.
So under this model the probability of successfully reorging off a static chunk through pure luck doesn't go down with more blocks.
