I wrote a bitcoin core parser, for my personal project and because it helped me to start understanding the bitcoin protocol as a data structure.

This project mainly parsers every blk file and creates the corresponding json with the deserialized information.

I launched the alpha version of the project and actually parserizes the information and converts it into a json file, my application worked fine until the blk file number 976, where I get an exception from the blockchain library and precisely in the ReadCompactSize operation, the exception is of the type

unknown file: Failure
C++ exception with description "ReadCompactSize(): size too large: iostream error" thrown in the test body

I used the bitcoin core libraries for example serialize.h

I don't understand why this error only comes up to me after blk00976 blochi and for all the subsequent ones I tried also with the file blk00977 blk01124

I have some posts I have consulted and others where I asked about this problem, now I was opening an issue on github for bitcoins but there it is written that for general questions you have to refer here.

The questions I have read but do not solve the problem How to handle incomplete blocks when reading blk*.dat files?

My question https://bitcointalk.org/index.php?topic=5134784.msg50723554#msg50723554

My source code https://github.com/vincenzopalazzo/SpyCblock

Two words about the source code

I am a java programmer dealing with the bitcoin protocol and I decided to carry out the C ++ project to study the language too, so to those who linked the code I apologize if it looks like an offense because it is badly written but I am studying to refact and improve the performance of the parser but also the quality of the source code.

  • 1
    Does your code support segwit? (BIP144)? – Pieter Wuille Apr 23 at 19:33
  • The BIP144 is this right? Click me I don't think my code supports it, I didn't know it existed, so is this the reason for the error? Can you give me more information about this? why was a new type of serialization introduced? This type of serialization is valid only for transactions (input and output) or even raw transactions and blocks ?? – vincenzopalazzo Apr 23 at 19:50
  • 1
    It was part of the segregated witness upgrade, which was a major upgrade to Bitcoin's technology. It affects a lot (P2P protocol, signatures, transactions, ...). You can read more about the reasons for it and design here: bitcoincore.org/en/2016/01/26/segwit-benefits and bitcoincore.org/en/segwit_wallet_dev More about the history here: bitcoinmagazine.com/articles/… Currently around half of the transactions on the network use segwit features. – Pieter Wuille Apr 23 at 19:53
  • 1
    Segwit is designed so it uses an otherwise invalid encoding, so it can be detected. It isn't so much a mix of formats; the new format is an extension of the old format; but the blk files indeed contain blocks with witnesses and without. – Pieter Wuille Apr 23 at 20:04
  • 1
    Read the references, and BIP144 specifically. – Pieter Wuille Apr 23 at 21:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.