I need to generate a SQL database (preferably SQLite3) of every transaction that has occurred in the blockchain. Is there an efficient way to do this using open-source software? I'm attempting to do it myself using a Haskell script that makes system calls to bitcoind but I'm having trouble making it sufficiently fast.
I created a fork of blockparser with a 'csvdump' command that will generate a details dump of the blockchain in CSV format. There are also some support scripts in that repository for creation, population and periodic update of a PostgreSQL database.
Be aware that the resultant database is pretty large; you might want to consider PostgreSQL over SQLite for this one.
You can always use BlockchainSQL Server for SQL querying the Bitcoin blockchain.
The web version is free, the server is commercial.
Disclaimer: I am the developer.
Possibly checkout ABE - Alternative Block Parser. It uses python and a SQL-based database but you can hook up other databases based on the driver.
To transfer to a SQL Server database you can try BitcoinDatabaseGenerator.
You can download it from https://github.com/ladimolnar/BitcoinDatabaseGenerator/releases
Sources and wiki: https://github.com/ladimolnar/BitcoinDatabaseGenerator
The transfer is very fast. At the time of writing this, the transfer of 266 blockchain files totaling 33 GB took less than one hour and 30 minutes on my development machine. Of course, the time will vary depending on hardware and configuration.