2

I have a weird issue that I thought was fixed a few years back with Bitcoin wallet to reduce fees. I guess not, or I am missing something...

My wallet received a transaction for the amount of 0.54530200 BTC, yet when I went and sent 0.51986042 BTC out using bitcoin-cli sendtomany to 16 outputs, the wallet used 856 inputs to send the amount that was available and cleared with at least 400+ confirmations in the first address.

The above TX cost me 0.03600705 BTC to send when it should have cost me a fraction of the amount had it used the address that contained the closest amount to the amount it's sending out...

Bitcoint-cli version 0.19.1

Any ideas what I could be doing wrong or what I could be missing?

6
  • Does the resulting transaction only have recipient outputs and no change outputs?
    – Andrew Chow
    Dec 8, 2020 at 0:37
  • It has 1 change output and 15 recipient outputs. Dec 8, 2020 at 0:58
  • Created an issue in GitHub repository for Bitcoin Core: github.com/bitcoin/bitcoin/issues/20598
    – user103136
    Dec 8, 2020 at 9:24
  • Hi Duetschpire, I am wondering how many unspents this wallet roughly had when you were building the transaction.
    – Murch
    Dec 9, 2020 at 19:47
  • Nevermind, I'm looking at the issue on Github now.
    – Murch
    Dec 9, 2020 at 19:54

2 Answers 2

1

The coin selection algorithm present in Bitcoin Core 0.19 tries to either find a solution where there is no change output, or a solution where the change is as close to 0.01 BTC as possible without going under that amount, ignoring fees.

There are two major components in 0.19's coin selection that we need to consider when analyzing this issue. The first is how it determined fees, and the second how it actually chooses the inputs. (There is a third part with finding a changeless solution, but because your transaction has change, this does not matter).

The fees were determined iteratively. It would choose inputs, calculate the fees for a transaction with those inputs, and if it did not happen to have enough value, it would increase the amount to select by that fee value and try selecting again.

For the actual selection part it would first pick out all of the inputs whose value is less than the target, and find the lowest larger input - the lowest valued input whose value is larger than the target. If the lowest larger input does not exactly match the target, it will then do a stochastic approximation.

This approximation will iterate all of the inputs whose value is less than the target and randomly decide whether to include or exclude the input. It will do this loop 1000 times. Of the input sets that this finds, it will choose the one whose total value is the minimum that is still greater than the target (similar to the lowest larger input).

If this approximation does not result in an exact match, and the sum of the inputs whose value is less than the target is greater than the target plus a minimum change of 0.01 BTC, then the approximation will be run again with the target increased by that 0.01 minimum change value.

With this approximation algorithm, an important effect is that if you have a lot of small inputs and just a few outliers that are much higher valued, it is very likely that the approximation will choose a lot of the small inputs to meet a higher value rather than the large inputs with a few small ones. This is due to the randomness used when deciding whether to include or exclude an output.

With that in mind, what happened with this transaction? While I can't say for sure without knowing the values of all of the inputs in the wallet at the time, I can make an educated guess. However we do know that the target value is 0.51977991.

The first thing to notice is that the large input is ~0.025 greater than the target, and with a wallet with lots of small inputs, it's likely that a set of smaller inputs can be chosen that is between the target and that single large input. So in the first iteration, what happened is that the approximation found a set of inputs where the value is between the target and the large input. This would be a large input set and thus require a high fee.

In the next iteration the target would be increased by that fee, and because the approximation is limited to the inputs smaller than the new target, it would still be picking small inputs. So it would end up choosing even more smaller inputs to make up for the fee that is needed.

With a few hundred more inputs of similar size to the ones that ended up being picked, it could be that the first iteration chose ~800 inputs. This would have pushed the target to be larger than your large single input, but your large single input would have just ended up not being picked amongst the multitude of smaller inputs.

1
  • Note that this iterative behavior of choosing the fee has since been replaced by using the effective value of inputs which directly accounts for the cost of each input when it is picked.
    – Murch
    Jun 3, 2022 at 13:32
0

tx-screenshot

I am not sure why coin selection algorithm didn't work or if you manually added some inputs or could have manually added inputs to avoid this. However, you could have still avoided paying more fees.

The transaction had virtual size of 78213 VByte You had used fee rate: 46 sat/vByte

Since this transaction involved lot of inputs and bigger size, you could have used a lower fee rate when mempool had less than 5 MB of transactions.

Assuming you had used 5 sat/vByte fee rate for this transaction, the total fees paid for the transaction would have been 0.0039 BTC which is 90% less than the fees you paid.

Looking at the mempool history, the transaction would have confirmed yesterday:

mempool-history

UTXO consolidation is normally done when mempool is clear or you have lots of time so that 1-5 sat/vByte can be used for such transactions.

Consolidation of 4 Million UTXOs at Xapo: https://bitcoinops.org/en/xapo-utxo-consolidation/

7
  • 1
    Thanks for the detailed explanation, the answer to your questions is: 1- This is scripted into the site, we send to 100s of addresses a day, we never manually select inputs. 2- We calculate the sat/byte on a 20h confirmation window depending on the mempool size. These tx are our customers' and waiting days for a tx to confirm is not ideal. Dec 8, 2020 at 5:01
  • Do you have a test website where we can try to reproduce this issue using testnet bitcoin? Maybe debug.log file might also have some useful information. I was assuming this is a personal transaction, in future I am not sure how will people manage fees if nodes do not allow fee rate less than 1 sat/vByte. Some businesses have started integrating layer 2 solutions like LN and Liquid for quick confirmation and less fees. Also this: i.imgur.com/aD2CVo4.png
    – user103136
    Dec 8, 2020 at 5:31
  • 1
    While this is not something that happens often, it's also not uncommon. The site does not receive large transactions, but when it does this happens. I have a feeling that coin control favours more inputs with less combined "change" over less inputs with more combined change. In this case the change address (3BF5VhQgnabWMsB8QhVrBoWD6is2XMKuQs) received 0.00008051 BTC, where the input with the closest full amount amount would have resulted in (0.02544158 - tx fee) in change. Dec 8, 2020 at 5:48
  • I will experiment and try few things on my local machine using bitcoin core v 0.20.0 (testnet) today. Will share the details here and also create an issue in GitHub repository if required: github.com/bitcoin/bitcoin/issues
    – user103136
    Dec 8, 2020 at 5:54
  • @Duetschpire I was able to reproduce the issue and think we can improve this. Let me document everything and see what can be changes in coin selection algorithm to fix this issue.
    – user103136
    Dec 8, 2020 at 8:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.