Bitcoin Core Multi-wallet limit to 333 wallets

Question

Bitcoin Core 0.15.1 has Multi wallets support:

In the bitcoin.conf we can add multiple entries of:

wallet=wallet-1.dat
wallet=wallet-2.dat
wallet=wallet-3.dat
wallet=wallet-4.dat
...
wallet=wallet-350.dat

When Bitcoin Core starts, if the wallet file does not exists it will create it. which is great for our needs.

However, it always crashes on the 333 wallet. with db.log error:

Lock table is out of available locker entries

And debug.log:

2018-02-13 14:47:05 init message: Loading wallet...
2018-02-13 14:47:05 nFileVersion = 160000
2018-02-13 14:47:05 Keys: 21 plaintext, 0 encrypted, 21 w/ metadata, 21 total
2018-02-13 14:47:05  wallet                   36ms
2018-02-13 14:47:05 setKeyPool.size() = 20
2018-02-13 14:47:05 mapWallet.size() = 0
2018-02-13 14:47:05 mapAddressBook.size() = 0
2018-02-13 14:47:05 init message: Loading wallet...
2018-02-13 14:47:05 

************************
EXCEPTION: St13runtime_error       
CDB: Error 12, can't open database wallet-333.dat       
D:\BitcoinCore\bitcoin-0.16.0\bin\bitcoin-qt.exe in Runaway exception     

I also tried to minimize the -keypool=20 so that the wallets size will be much smaller in size, and the process of loading them is much faster.

I was suggested that this error is because each wallet.dat file is opened in the same BDB environment. The entire environment has a limited number of database locks, and it seems like 333 wallets hits that number of database locks.

The solution (which did not work) is to create a file named DB_CONFIG in the data dir, and put the following lines in that file:

set_lk_max_locks 400000
set_lk_max_objects 400000

By default it is 40000 and now 400000 (ten times more locks available)

But this gave the same results.

I tried also:

set_lg_dir database

With no luck.

I verified this on two different Windows 10 machines with 0.15.1 and 0.16.0. Same result. It always crashes when loading the 333 wallet.

Anyone have a solution to this?

I have put the list of wallets if anyone care to reproduce: https://pastebin.com/8wHD1tL3

Answer 1

Ok, I found a solution. Adding

set_lk_max_lockers 400000

Did the trick.

https://docs.oracle.com/cd/E17276_01/html/api_reference/C/set_lk_max_lockers_parameter.html

I tested and was able to auto-create and load 932 wallets, until I hit an out of memory error:

Logging region out of memory; you may need to increase its size
DB_TXN->abort: log undo failed for LSN: 227 171878: Not enough space
PANIC: Not enough space
PANIC: DB_RUNRECOVERY: Fatal error, run database recovery
PANIC: fatal region error detected; run recovery

Setting:

set_lg_regionmax 1048576

Resolves the above Logging region out of memory; you may need to increase its size issue.