I'm trying to write a parser for blk*.dat files from scratch. Right now, I can read blocks, extract the header fields and calculate the correct block hash based on the header fields.
I just tried to run the parser through all blk-files and it failed after successfully reading the first ~300 blk-files. When inspecting the failing file with a hex-editor, I found two "magic byte fields", only separated by 4 other bytes.
Since the mandatory block-header is supposed to be 80 bytes in length, I'm now quite confused.
I found three probably related questions:
- blk file error when reading - did something change in the format?
- How can you tell if you're at the end of an incomplete blk*.dat file?
- Zeros in blk00*.dat files
My guess, based on these other questions, is the following:
- This is an 'incomplete block'
- This happens when a block can only be partially downloaded, or written to disk.
- Deleting the affected blk files and re-downloading parts of the blockchain doesn't make sense, since this can be expected to happen again.
- The parser should be able to skip these blocks.
- The overall blk file is not missing actual data and is not corrupt.
- The 'incomplete block' will be downloaded and written again further down in the blk file (or to an other blk-file).
Which of my assumptions are correct? .. which are not? Am I missing something?