0

I can start bitcoin core loading with 200 wallets. But when I increase it to 300 wallets, it fails. Below is the debug.log when I tried to load 300 wallets.

2018-12-17T01:36:27Z init message: Loading wallet...
2018-12-17T01:36:27Z [wallet-201.dat] nFileVersion = 170001
2018-12-17T01:36:27Z [wallet-201.dat] Keys: 3 plaintext, 0 encrypted, 3 w/ metadata, 3 total. Unknown wallet records: 1
2018-12-17T01:36:27Z [wallet-201.dat] Wallet completed loading in               1ms
2018-12-17T01:36:27Z [wallet-201.dat] setKeyPool.size() = 2
2018-12-17T01:36:27Z [wallet-201.dat] mapWallet.size() = 0
2018-12-17T01:36:27Z [wallet-201.dat] mapAddressBook.size() = 0
2018-12-17T01:36:27Z init message: Loading wallet...
2018-12-17T01:36:27Z [wallet-202.dat] nFileVersion = 170001
2018-12-17T01:36:27Z [wallet-202.dat] Keys: 3 plaintext, 0 encrypted, 3 w/ metadata, 3 total. Unknown wallet records: 1
2018-12-17T01:36:39Z [wallet-202.dat] Wallet completed loading in           12012ms
2018-12-17T01:36:39Z init message: Rescanning...
2018-12-17T01:36:39Z [wallet-202.dat] Rescanning last 1447907 blocks (from block 0)...
2018-12-17T01:36:39Z [wallet-202.dat] Rescan started from block 1447885...
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=14930383)
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=14955195)
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=15731604)
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=15152698)
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=14994412)
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=15000180)
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=15237899)
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=15073409)
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=15076691)
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=15162221)
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=15175705)
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=15238839)
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=15734430)
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=15746232)
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=15755504)
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=15759768)
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=15780715)
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=15795130)
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=15814178)
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=15841983)
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=15844548)
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=15870120)
2018-12-17T01:36:39Z Unable to open file /root/.bitcoin/testnet3/blocks/blk00155.dat
2018-12-17T01:36:39Z ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=155, nPos=15897975)
2018-12-17T01:36:39Z [wallet-202.dat] Rescan completed in               1ms
2018-12-17T01:36:39Z [wallet-202.dat] setKeyPool.size() = 2
2018-12-17T01:36:39Z [wallet-202.dat] mapWallet.size() = 0
2018-12-17T01:36:39Z [wallet-202.dat] mapAddressBook.size() = 0
2018-12-17T01:36:39Z init message: Loading wallet...
2018-12-17T01:36:51Z [wallet-203.dat] Releasing wallet
2018-12-17T01:36:51Z BerkeleyEnvironment::Flush: Flush(false)
2018-12-17T01:36:51Z BerkeleyEnvironment::Flush: Flush(false) took               0ms
2018-12-17T01:36:51Z 

************************
EXCEPTION: St13runtime_error       
BerkeleyBatch: Error 24, can't open database wallet.dat       
bitcoin in AppInit()
Improve this question

1 Answer 1

Reset to default
2

You are going to run into issues when opening many wallets at the same time due to memory constraints. When a wallet is opened, some data from the wallet is stored in memory. If you have too many wallets opened, then you may run out of memory to store all of that extra data. This will impact other parts of the process too and could result in corrupt databases and corrupted wallets.

Furthermore, opening wallets means that a new file is being opened. Modern operating systems have limits on the number of open files a process can have. In your case, the error number 24 means that there are too many file opened. You will need to lookup how to increase the open file limit for your OS.

I do not recommend that you continue to try to have so many wallets open at once. Bitcoin Core now has dynamic loading and unloading of wallets, so you can load a wallet when you need it and unload it when you are done. This will let you have fewer wallets open simultaneously and you will not run into such issues.

Improve this answer
5
  • Thanks for the suggestion. The reason I load so many wallets at the same time is because I wanted to use bitcoin walletNotify. I've tested that walletNotify won't work if I didn't load the wallet. I guess I need to import all addresses from each wallets into 1 wallet as watch-only addresses. Is there a limit on running a wallet with too many watch-only addresses?
    – Charles
    Commented Dec 17, 2018 at 3:19
  • You will run into memory issues eventually, but you won't run into the too many open files problem.
    – Ava Chow ♦
    Commented Dec 17, 2018 at 3:23
  • I don't see what in the log indicates memory? What am I missing? --other things that could also contribute is that bdb has many default limits that might be hit, or just running into the file descriptors ulimit.
    – G. Maxwell
    Commented Dec 17, 2018 at 9:37
  • @G. Maxwell He hit the file descriptor limit, but I am mentioning in my answer out of memory because he's going to hit that too eventually if he raises the file descriptor limit.
    – Ava Chow ♦
    Commented Dec 17, 2018 at 16:12
  • It's not unreasonable for a host to have hundreds of gigabytes of ram these days, ... I'd understand you saying that he'd run out of memory if he said he wanted to load tens of millions of wallets...
    – G. Maxwell
    Commented Jan 19, 2019 at 23:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.