Project on modelling Bitcoin blockchain data

Question

I am doing a thesis project on modelling Bitcoin blockchain data, but atm I am having trouble getting data from the Bitcoin blockchain into a format that is easy to deal with in e.g. Python.

I have all data from Bitcoin Core, do you know a way to aggregate the data, into a format that would give an idea about users balance or something like this?

If you have ever done a similar project, I am very interested in what you have done in the preliminary phases! Thanks.

Answer 1

As to how to convert anonymous transactions to "users" (at most a statistical attempt, as Bitcoin is in principle anonymous), which could lead to your requested "users balance" info, there are papers available that describe the process. These for example could serve as a start point (with good descriptions and images):

• "An Analysis of Anonymity in the Bitcoin System" by Fergal Reid and Martin Harrigan
• "Analyzing the Bitcoin Network: The First Four Years" - Matthias Lischke and Benjamin Fabian, Institute of Information Systems, Humboldt-Universität zu Berlin, Spandauer Str. 1, 10178 Berlin, Germany (CC BY 4.0)

Basically, citing the latter:

... a “user” is a grouping of public keys that were used as inputs into a single transaction (user owns the private key to each address) ...

As for the troubles using Python directly to treat raw data, you can use, perhaps as template or to get ideas, this post which uses Python to convert binary block header data to a CSV:

#!/usr/bin/env python
# convert binary file http://headers.electrum.org/blockchain_headers 
# to CSV ASCII

import binascii

STRUCT_OF_BLOCK = [ 4, 32, 32, 4, 4, 4 ] # blockchain_headers does not contain always "0x00" txn_count
BLOCK_SIZE = sum(STRUCT_OF_BLOCK)

FILE_OUT= open('blockchain_headers.csv','w')

FILE_OUT.write( "version,prev_block,merkle_root,timestamp,bits,nonce,txn_count\n" )

with open('blockchain_headers','rb') as FILE:
    block = FILE.read(BLOCK_SIZE)
    while block != b'':
        position = 0
        for i in STRUCT_OF_BLOCK:
            FILE_OUT.write( bytearray(binascii.hexlify( block[position:(position+i)][::-1] )).decode('ascii') + ',')
            position += i
            if position >= BLOCK_SIZE:
                FILE_OUT.write("00\n") # blockchain_headers does not contain always "0x00" txn_count
        block = FILE.read(BLOCK_SIZE)