1

There are different estimations used in wallets, explorers and other Bitcoin projects. For example: estimatesmartfee in Bitcoin Core

This is the fee rate distribution right now according to https://btc.bitaps.com

mempool-stats

Are different estimations misleading and affect the way fees are used in Bitcoin transactions? Will it be better if we just share mempool stats and user can decide the fee rate accordingly?

If I compare this with BTCUSD orderbook on any exchange, are we trying to estimate at what price buy order will get filled in certain time? Does that make sense?

orderbook

In Bitcoin, block space is the supply, fee bidding to get the transaction included in a block is demand, miners are sellers who will fill the limit orders however they can include any transactions (not necessary it will be prioritized based on fee rate) so what exactly are we trying to estimate and why?

Market buy orders that fill the sell orders on exchanges can be compared to a transaction using fee rate 310 sat/vByte when highest bid in mempool is 300 sat/vByte. Although things are not instant in Bitcoin and there is a time gap between broadcasting with a certain fee rate and transaction being included in block. There are few other differences like if I try to buy 0.01 BTC right now on Bitfinex at 70,000 it will be executed at 53,450 and in Bitcoin there is no such thing, you can decide any fee rate and even if I use 1000 sat/vByte right now, same will be used and accepted by miners unless there is something that avoids such errors in wallet being used but overall picture looks similar and all this makes estimation even more difficult.

I consider it misleading because lot of users think a transaction with fee rate 1-5 sat/vByte will be included in 1 week or maybe a transaction with X sat/VByte will be included in Y time which is not true. User decide the fee rate and can do bidding, transaction will be included based on demand, supply and miners.

Will it be better if the wallets used this approach?

  1. Show mempool stats
  2. Leave the fee rate for user to decide
  3. RBF every transaction and follow different algorithms for automated bidding

A simple algorithm for automated bidding can be:

automated-fee-bidding

4
  • This seems to me like a bunch of different questions which ends up reading as an essay for the question followed by another essay in the answer. The title of the question "What are we trying to predict in fee estimation and why?" has an obvious answer which is attempting to predict how much it will cost to get in the next block, next 6 blocks etc. If you want this to be useful to others and perhaps get some input from others I'd recommend splitting the questions up into separate posts and try to be more concise and targeted. Jul 27 at 13:43
  • For example what has an algorithm for automated bidding got to do with what fee estimators are trying to predict and why? It is a jumble. Jul 27 at 13:47
  • I couldn't find a better way to explain everything in the question. It could be shorter but then it may look more confusing and closed as "need more details / clarity". Algorithm for automated bidding is related to fee estimates and alternative because it explains further the kind of solutions someone asking this question prefers over estimates.
    – Prayank
    Jul 27 at 14:28
  • I would include that in a separate question, something like "Other than wallet fee estimators what could be designed to ensure users end up paying an accurate fee for their desired confirmation time?" It is a different question to the purpose of fee estimators. Jul 27 at 15:27
1

What are we trying to predict in fee estimation and why?

Different Bitcoin projects try to predict fee rates for improving UX because its not easier for every user to understand all the technical things involved, look at different charts and decide a fee rate for the transaction.

Are different estimations misleading and affect the way fees are used in Bitcoin transactions? Will it be better if we just share mempool stats and user can decide the fee rate accordingly?

Some fee estimations maybe misleading but not everything. And they are only estimates which does not guarantee the transaction to be confirmed. Mempool stats can be shared in the wallets and most of the wallets already provide option to enter fee rate manually.

If I compare this with BTCUSD orderbook on any exchange, are we trying to estimate at what price buy order will get filled in certain time? Does that make sense?

There are few differences between BTCUSD market and Bitcoin fee market which are shared below.

Fee rates used for bitcoin transactions increase during the bull market and stay low in bear market.

fee-per-mb

BTCUSD follows cycles which are mentioned as bull and bear above.

btc-usd-cycles

The difference in two charts above is BTCUSD making new highs in every bull market and considered as a store of value in long term following logarithmic regression, whereas fee rates chart looks like sine wave and reset to 1 sat/vByte often.

bitcoin-price-chart

mempool-chart

Fee rates used in bitcoin transaction are lower during weekends because of less demand for block space however there can be exceptions and its not necessary that fee rates will always stay low during weekends. The amount of decrease in fee rates during weekends and fee market post weekend also varies. An example with weekends highlighted in last one month:

weekend-fee-market

Other things that affect the fee rates used are technical:

  1. Segwit usage: https://transactionfee.info/charts/payments-spending-segwit/
  2. Batching: https://bitcointechtalk.com/saving-up-to-80-on-bitcoin-transaction-fees-by-batching-payments-4147ab7009fb
  3. UTXO Consolidation: https://bitcoinops.org/en/xapo-utxo-consolidation/
  4. Fee estimation used in different wallets: https://blog.bitgo.com/the-challenges-of-bitcoin-transaction-fee-estimation-e47a64a61c72
  5. Coin selection algorithms used in different wallets: https://iohk.io/en/blog/posts/2018/07/03/self-organisation-in-coin-selection/
  6. Layer 2 usage: https://bitcoinkpis.com/layer2
  7. Exchanges: https://b10c.me/mempool-observations/2-bitmex-broadcast-13-utc/

Taproot will be added to this list once its activated and being used by different Bitcoin projects.

Will it be better if the wallets used this approach?

  1. Show mempool stats
  2. Leave the fee rate for user to decide
  3. RBF every transaction and follow different algorithms for automated bidding A simple algorithm for automated bidding can be: https://i.stack.imgur.com/1SlPv.png

RBF algorithms for automated fee bidding can be tried and users should also manually prefer to use RBF for most of the transactions.

ZmnSCPxj shared few exceptions on bitcoin dev mailing list which should be handled while creating such algorithms:

Again, a race condition can occur --- while the wallet is feebumping a new transaction that includes the new output, a random miner can find a new block that includes the old transaction. Further, of course it is convenient to be able to spend money even while it is unconfirmed.

But the sender of the unconfirmed input might be using the same software as this wallet as well, meaning that the actual transaction output might change as the original spender keeps fee-bumping it over time.

https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-May/018886.html

Jeremy Rubin shared few other ideas in response to my question related to fee estimates and RBF:

In the long term, this can be particularly useful since you can use a separate fee-only wallet to arrange the bumps if your main wallet keys are offline, and you do not disturb any of the on-chain dependants.

I'd like to share with you a draft proposal for a mechanism to replace CPFP and RBF for increasing fees on transactions in the mempool that should be more robust against attacks. A reference implementation demonstrating these rules is available here for those who prefer to not read specs. https://gist.github.com/JeremyRubin/92a9fc4c6531817f66c2934282e71fdf

https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-May/018882.html

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.