Build an online wallet service with Bitcoind, can it scale? If not, what can be done?

Question

I would like to build an online wallet service. I thinks it will be easy to implement all the functionalities if I go with Bitcoind. Here is a possible workflow:

1. New user with email [email protected] can obtain a new account (in Bitcoind) with new address by calling getnewaddress [email protected]

2. We can then start polling listsinceblock xxx to get all new transactions. If we receive a new incoming transaction, store txid in database, then iteratively call gettransaction txid on all unconfirmed transactions. When the number of confirmations are bigger than a certain threshold, credit the account in database, then call move [email protected] safe, just to make the next step easier.

3. When safe account exceeds say 50BTC, send them to offline wallet.

4. When user wants to withdraw, refill safe if necessary, send the coins from safe, and update user's balance in the database.

However, in all the related questions I've read, most people say Bitcoind won't scale. Even the Bitcoin wiki says so. Now, if we are only to use the JSONRPC to interface the network, I can imagine the following workflow:

1. We generate say 1M addresses in database. When user signs up or require new address, we assign one address to them.

2. We enable blocknotify to get new block hash blkhash.

3. Call getblock blkhash to get all transaction hashes in the new block.

4. Call getrawtransaction txid to see whether we received an incoming transaction to our addresses. If so, keep the txid and keep polling gettransaction txid to update the confirmation number.

5. Call signrawtransaction, sendrawtransaction to move funds to offline wallet.

My questions are,

• Do you think my proposals are reasonable?

• What are the nuances in the operations above? For example,

  • by moving all funds to one account and then send money, does that count as one transaction or one transaction for each account? Are transaction fees the same?
  • when doing getrawtransaction, in the vout array, each object represents a destination, but each of them has an addresses array. There are cases where it contains multiple addresses. Does it mean it's a "multisig" transaction? Or is it related to reqSigs field? How do we handle such cases?

I've read following questions, but didn't find my answer:

Does the original Bitcoin client scale to contain millions of addresses?

Bitcoin Client API-RPC: Efficient way to list transactions from a list of addresses

Please help.. Thank you very much!

Answer 1

I hate to be this guy but if you're struggling with this chances are that you'll struggle to secure the private keys. Those private keys are the keys to your empire in a way you don't even want to think about.

Think very seriously about outsourcing this to a company with insurance for it. Or get insurance yourself. The outsourcing also takes the performance hassle away from you.

If you don't, please don't forget to make (really) cold wallets. Just place that cash elsewhere.

Perhaps more directly, there's several bitcoin daemons out there. You could make your own, too, with something like bitcoinJ. I met the guys behind btcd ("BitcoinD in Go") https://github.com/conformal/btcd at Bitcoin2014, their lead coder made a good impression.

I'm afraid there's no shelved answer for this question yet. Perhaps the best advice is "Do it, once you run into trouble you're successful already. Time enough to solve it later.".

Answer 2

There were a few more suggestions on an answer to this related question: Howto scale thousands of bitcoin accounts properly with bitcoind deamon?

Someone has already created a wrapper for the bitcoin daemon in Java: https://github.com/btc4j/btc4j-daemon - it might be easier to use that instead of the dameon directly and use a database to do what would be inefficient on the blockchain, like handling user accounts.

Ultimately, it all depends on what you're trying to accomplish. It sounds pretty clear from what you've already read and what I've read that the standard bitcoin daemon doesn't scale. Thus, your best bet would be to use this or another bitcoin daemon wrapper to create your wallet project.

If you want to interface directly with the bitcoin daemon (or as close to directly as possible with a wrapper), try using the Java wrapper and use a database for the tasks that the daemon doesn't handle efficiently. However, if you are wanting to design your online wallet service with security in mind, I would encourage you to use the Armory daemon: https://bitcointalk.org/index.php?topic=92496.0. From what I've read, it sounds like this daemon is not quite as full featured as the regular bitcoin daemon, but I haven't tried it out myself, so it's quite possible that it is sufficient to your needs. Still, it has a pretty big advantage in the realm of security that you can create a watching-only wallet on your web server for the customers' wallet.