I have seen several posts on this before, and have tried all the suggested solutions, but I can't get any of them to work.
I am interested in analysing transactions with the OP_RETURN opcode for a project, and need to dump the entire Bitcoin in a database to start working on that. I have already installed Bitcoin Core and downloaded all data until today.
There is a list of tools available to do this, and I have had problems with all of them:
- WebBTC: The files don't seem to be available
- BitcoinDataBaseGenerator: This seems like the most promising solution, as it is easy to set up on Windows, and a MS SQL Server database is easy to use. However, after file blk005**.dat it crashed, saying that there are blocks with an unknown version. I also can't try to run it again because the queries to delete data from that file won't finish executing and timeout after ~2 hours. Looking at the issues at GitHub, people seem to be having problems with that too, as the repo is not maintained any more.
- Blockparser + SQL: This only keeps the data in memory.
- BitcoinABE: I have not tried this one yet. I read many issues that it is very slow though, and there are generally many open issues. Might work though, I will try later.
- NBitcoin.Indexer: Made for Azure, I want to have a local database though.
- Blockchain2Graph: This also seems promising, and it is under active development. It uses neo4j, which is fine. At the moment I have no preference for either SQL or graph databases. I set up a Debian Virtual Machine and followed all the steps, which did work. However, at the end I can't get the application to actually show up in the browser. From my understanding it uses RPC calls and not the blk*.dat files directly.
Then I found one more on Google:
- Bitcoin-to-neo4j: This takes the block files directly and puts them in a neo4j database. The problem with this is that the database is apparently six times the size of the actual Bitcoin data. I don't have 1.7 Terabytes of free storage space available. It also mentions that it takes several weeks to process it all. I simply don't have the time to wait for that.
So, my question is if there are any other tools currently available to easily write the Bitcoin blk*.dat files to a local database. Maybe I did something wrong, with the above mentioned solutions, but none of them seem to be an ideal solution. How easy would it be to write a custom application that only extracts the data I'm interested in, and are there code templates for that?