8

I want to process the Bitcoin network's transactions. As I understand it, the Bitcoin-Qt client downloads and stores all that information locally.

Since version 0.8 LevelDB has been used to access this data. I found two databases, in blocks/index and in chainstate. However, I cannot make sense of the content. The first entry looks like this:

B = ??????{?:???????`?5g??

Am I on the right track and where can I find a specification of the data?

Improve this question
2
  • The data is stored in an application-specific format optimized for compact storage, and wasn't really intended to be easily parsed by other applications. You can access the data in the chainstate through the gettxout RPC call, and the data in the block index through getblock. I'll try to find time for a write-up of the exact format.
    – Pieter Wuille
    Jul 29, 2013 at 7:17
  • @PieterWuille Your comment would be a great start for an answer.
    – Murch ♦
    Oct 3, 2014 at 10:47

3 Answers 3

Reset to default
6

The best way to write a parser of your own for the blockchain is to find the source code of one that already exists and from that deduce the precise underlying data structure and how to parse and interpret it. e.g. here in C++ and here in C# - and there are many others.

The data in the blockchain is stored in a custom binary format that is a little tricky to untangle without some help; this link seems as good as any to get started on the overall structure.

There are many"gotcha's" to watch out for. For example:

  • 'endianess' i.e. having to reverse binary digits before interpretation and
  • some of the interesting values associated with a transaction have to be calculated e.g. for addresses
  • 'big numbers' where the numbers used internally are so huge you may need custom code/libraries to deal with them

Comparing your results to those that are freely available via the blockchain and blockexplorer as well as the bitcoind.exe API would be a must. Otherwise it is easy to pull out seemingly valid transactions that are just nonsense. I speak from experience.

While writing your own parser for the blockchain is a worthwhile learning exercise you would be much better off IMHO using the bitcoin API and ignoring the underlying blockchain database for any 'real' code.

Improve this answer
2

The best way to access transaction database for analysis is probably through the blockchain.info API. You can also use the Bitcoin Block Explorer; in fact, this is what a study of the Bitcoin network did.

Improve this answer
1

The easiest way is to use my blockchain parser. This script can parse all the data from blk*****.dat files into the human readable text view.

blockchain parser

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.