0

I would like to do some research of the Bitcoin blockchain. Because i would like to do massive amounts of processing and lookups, I need a fast way to search the blockchain.

Http requests to insight.io just won't cut it...

I know of ABE but it seems no longer maintained and I don't know if it is up to par with the current implementation of the blockchain.

The environment I'm programming in is python.

Any ideas?

Improve this question
0

Depends on what you feel comfortable with.

You can go low level and query the lebeldb directly. The block index (information for every block) is in $DATADIR/blocks/index and the chainstate (information about the currently best known chain) in $DATADIR/chainstate.

An easier way is to have a fully synced bitcoind running locally and query over jsonrpc. From there you slice and dice the resulting JSON to fit your needs.

I haven't done leveldb querying, but I'd expect there to be a significant amount of work involved since it's just a key value store.

Improve this answer
  • The idea is to try and track coins over long periods of time with an innovative approach. So even bitcoind won't really cut it I think. – Dennis Decoene Apr 7 '19 at 14:05
  • Neither of those LevelDB databases maintained by bitcoind actually contain the information you want. Maybe some more modern indexing software like electrs is more appropriate. – Pieter Wuille Apr 7 '19 at 21:07
  • A Bitcoin equivalent of this XRP project? – user58807 May 7 '19 at 15:40
0

I held a research period on bitcoin (thanks to this project I became fond of Bitcoin) with the release of the alpha software at dicembe, I had to extract information for the construction of an information graph.

I believe in my opinion that there are really many ways to index information but I believe that the fastest ones are to use the same technique that uses bitcoins.

  • Parser blk files
  • Deserialize the information in a format of your choice and index it in db
  • query the db and make a read in the appropriate file.

There are two very difficult things - the construction of this indexing system - if you want to access id wallet you need to build a script decompiler or use the bitcoin node

ps: this is my experience, there can be other ways to build this information, which I don't know (mine is just a three-year thesis, my path has just begun)

This is a good parser for blk, one of the few compatible with Segregated Witness.

The abe project still works but his work times seem unacceptable with the current bitcoin dimension

If you have a big pc, this project is very fast (at least they say so)

Improve this answer
0

The fastest (simpliest) way is to use my blockchain parser which has just 240 lines of code and able to extract all information from blkXXXXX.dat files to the simple text view. It parse whole the raw bitcoin blockchain database that is stored in those files.

Then you can easily index the data as you need and then make any data manipulations for your academic research. For example, you can build UTXO set by yourself, or make some ECDSA signatures or keys research and so on.

Also, you can learn how the data is stored in blockchain if you check how my parser works with the sequences of bytes.

I'am sure it's the fastest, simpliest and most usefull way to learn the blockchain functionality.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.