I'm curious to hear an in-depth explanation of the hard fork and how it was caused; I've heard/read a great deal but I'm lacking a cohesive explanation.
What is SPV mining, and how did it (inadvertently) cause the fork after BIP66 was activated?
SPV mining is the term commonly used for 'less-than-full-node-validation' mining. It usually means that miners skip the verification of the block and the transactions within, and immediately start mining a new block referencing the just-solved block header. However, since they don't know what is in the last block, they have to mine without any transactions (except for the coinbase transaction), to be sure that they don't mine a block with transactions that conflict with transactions in the previous block.
After BIP 66 became enforced, about 5% of the network was still solving version 2 blocks (BIP 66 enforces blocks use version number >= 3). One of the miners in this 5% solved a block with version 2, and if everyone had been doing full validation, then their block would have been ignored and replaced by a version 3 block. That is what was supposed to happen.
But, unfortunately, a few pools (F2Pool was the biggest one, I think), started mining a new block that referenced the invalid block header without verifying the transactions in it or that the new header used a version number of 3 (as just became enforced). Miners will sometimes directly connect their full nodes or monitor each other's work APIs to see when another pool has solved a block, so that they can start working on a new block as quickly as possible. In this case, a few pools heard that BTC nuggets solved a block by listening to their Stratum API, and started mining on it. They didn't even have the block header, just the hash of the block header.
https://en.bitcoin.it/wiki/July_2015_Forks#Invalid_Block_Hashes shows the invalid blocks:
- 0000000000000000009cc829aa25b40b2cd4eb83dd498c12ad0d26d90c439d99 mined by BTC Nuggets (98 non-coinbase transactions)
- 0000000000000000155f2519d35cd5d2869900bcc5093594b27763a0315390b4 mined by F2Pool (0 non-coinbase transactions)
- 00000000000000000cb7a20ee4e199e347ad7369936abae53a1518efa531ec61 mined by F2Pool (0 non-coinbase transactions)
- 00000000000000000966d65e0fd87d1d5a8f154a2c955816c28e2006e381aa18 mined by AntPool (0 non-coinbase transactions)
- 00000000000000001301bfd6f566a421c7eeba103d09b312032ca065cb185de7 mined by F2Pool (0 non-coinbase transactions)
- 000000000000000013fe26675faa8f7dccd55ce5485bb6d0373fa66345901436 mined by F2Pool (0 non-coinbase transactions)
One of the Core Developers, Gregory Maxwell, also posted an explanation here, which you may find useful.
1It is my understanding that when they started mining, they didn't even have the previous block header (just its hash). See my comment on Reddit here. (Source) Because the block was invalid, they probably never actually received the block or its header through the regular P2P relay network. Jul 6, 2015 at 20:05
@DavidA.Harding, wow, that's even worse. I thought they had received the header over P2P relay. I will update my answer now, then. Jul 6, 2015 at 20:11
That's correct, they are mining on other pools stratum responses, not Bitcoin P2P headers.– ClarisJul 6, 2015 at 20:29