Extracting information from utxo for offline processing

Question

I would like to extract as much information about the utxo set (all unspent transactions) up to a given block height, including value and scriptpubkey.

This information is meant to be inserted into a database and processed offline for some statistical analysis. I have been looking at multiple alternatives including bitcoind, bitcoin-ruby, libbitcoin and others, but cant figure out how to do this. Perhaps my programming skills arent that hot.

Hoping someone can help me out. Thanks!

Answer 1

Blocks and transactions have very simple structure. It is very easy to parse them in any language. This is a piece of my code (it definetely works). You may write something similar

#include <QTimer>
#include <QFile>

#include "BlockChain.h"
#include "Util.h"
#include "Chainer.h"
#include "Target.h"

BlockChain::BlockChain ( QObject* parent ) : QFile ( parent ), blkFile ( 0 )
{
  QTimer::singleShot ( 0, this, SLOT ( start ( ) ) );
}
//--------------------------------------------------------------
void BlockChain::start ( )
{
  setFileName ( blkFileName ( blkFile++ ) );
  if ( !open ( QIODevice::ReadOnly ) )
  {
    _trace ( QString ( "cant open [%1]" ).arg ( fileName ( ) ) );
    getParent ( ).block ( QByteArray ( ), 1 ); // end-of-blockchain signal
    deleteLater ( );
  }
  else
  {
    _trace ( QString ( "processing [%1]" ).arg ( fileName ( ) ) );
    QTimer::singleShot ( 0, this, SLOT ( next ( ) ) );
  }
}
//--------------------------------------------------------------
void BlockChain::next ( )
{
  bool lock ( true );
  if ( pos ( ) < size ( ) )
  {
    quint32 magic;
    quint32 size ( read ( (char*)&magic, 4 ) );
    xassert ( ( ( magic == MAGIC_ID ) || !magic ) && ( size == 4 ) );
    if ( magic )
    {
      read ( (char*)&size, 4 );
      xassert ( size > HEADER_SIZE && size <= MAX_BLOCK_SIZE );
      getParent ( ).block ( read ( size ), 0 );          // block signal
      QTimer::singleShot ( 0, this, SLOT ( next ( ) ) );
      return;
    }
    else
      lock = false;
  }
  close ( );
  getParent ( ).doneFile ( lock, blkFile - 1 );       // notification
  QTimer::singleShot ( 0, this, SLOT ( start ( ) ) ); // next blk file
}
//--------------------------------------------------------------
const QString BlockChain::blkFileName ( const int i )
{
  return
    ( i < 10 ) ? QString ( DATA_ROOT "\\blk0000%1.dat" ).arg ( i ) :
    ( i < 100 ) ? QString ( DATA_ROOT "\\blk000%1.dat" ).arg ( i ) :
    ( i < 1000 ) ? QString ( DATA_ROOT "\\blk00%1.dat" ).arg ( i ) :
    ( i < 10000 ) ? QString ( DATA_ROOT "\\blk0%1.dat" ).arg ( i ) :
    QString ( DATA_ROOT "\\blk%1.dat" ).arg ( i );
}