0

I opened my laptop which always has bitcoind running and my log is going ballistic with messages like:

2020-11-05T12:41:57Z [FullyNoded-d703055f46595edcf8163916bd906f9563afc0903fbe74c2882a93c59359ca00] keypool index 38212 removed

Why? What is it doing? Removing keys I imported? I never asked it to do that :(

RPC commands are also unresponsive as it seems to be overwhelmed.

Bitcoin Core 0.20.1

Improve this question
3
  • Are these occurring in order (e.g. 38211 removed, then 38212 removed)? Did you retrieve a lot of addresses but only receive Bitcoin at one that came after a lot of addresses?
    – Ava Chow ♦
    Commented Nov 5, 2020 at 16:48
  • yes and yes. Im watching 99,999 addresses and in order to generate payment addresses I randomly generate an address from range 0...99999 for a customer to pay. Does this log output mean it is no longer going to watch for those addresses?
    – Fontaine
    Commented Nov 6, 2020 at 1:06
  • fwiw the keypool is not relevant for my use case, all that matters is that bitcoind is watching those addresses and that I can use deriveaddresses.
    – Fontaine
    Commented Nov 6, 2020 at 1:29

2 Answers 2

Reset to default
3

The keypool is ordered, not just a set of keys being watched. This allows for addresses to be given out in derivation order, as well as maintaining the gap limit. When an address is used, all keys that come before it will also be marked as used. This is done to ensure that already used addresses are not accidentally reused.

So when you give out and use a random address in the range, you end up having all of the previous keys in the range be marked as used.

Removing keys from the keypool is purely for giving addresses out to the user and to be able to generate further addresses. It does not effect the watching behavior; keys removed from the keypool will still be watched.

The reason that you see a performance hit is because removing from the keypool deletes a record from the database in addition to writing several scripst to the database. Furthermore, if the keypool size drops below the configured keypool size due to these removals, the keypool will be refilled which will involve additional database writes. These all acquire a lock on the wallet which prevents other threads from accessing the wallet, and writing all of this data (especially tens of thousands of keys) can take a long time.

To avoid this problem, I highly suggest giving out addresses in order, not randomly. Otherwise you may not want to add the keys to the keypool although that means that you will lose the keypool refilling to keep the range full.

Improve this answer
2
  • Makes a lot of sense, thanks for the answer. I should definitely not be adding them to the keypool for this specific use case.
    – Fontaine
    Commented Nov 7, 2020 at 2:46
  • This is so helpful thanks for this answer
    – Shovas
    Commented Mar 1 at 21:41
0

Not sure but I think it's marking keys as used and doing the topup as described in this PR: https://github.com/bitcoin/bitcoin/pull/11022

Logs about "keypool index %d removed" : https://github.com/bitcoin/bitcoin/pull/11044 which I see now in src/wallet/scriptpubkeyman.cpp https://github.com/bitcoin/bitcoin/blob/9bb078351b09862a263e936bb612dca82305a6fe/src/wallet/scriptpubkeyman.cpp#L1421

Improve this answer
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.