What are we trying to predict in fee estimation and why?
Different Bitcoin projects try to predict fee rates for improving UX because its not easier for every user to understand all the technical things involved, look at different charts and decide a fee rate for the transaction.
Are different estimations misleading and affect the way fees are used in Bitcoin transactions? Will it be better if we just share mempool stats and user can decide the fee rate accordingly?
Some fee estimations maybe misleading but not everything. And they are only estimates which does not guarantee the transaction to be confirmed. Mempool stats can be shared in the wallets and most of the wallets already provide option to enter fee rate manually.
If I compare this with BTCUSD orderbook on any exchange, are we trying to estimate at what price buy order will get filled in certain time? Does that make sense?
There are few differences between BTCUSD market and Bitcoin fee market which are shared below.
Fee rates used for bitcoin transactions increase during the bull market and stay low in bear market.
BTCUSD follows cycles which are mentioned as bull and bear above.
The difference in two charts above is BTCUSD making new highs in every bull market and considered as a store of value in long term following logarithmic regression, whereas fee rates chart looks like sine wave and reset to 1 sat/vByte often.
Fee rates used in bitcoin transaction are lower during weekends because of less demand for block space however there can be exceptions and its not necessary that fee rates will always stay low during weekends. The amount of decrease in fee rates during weekends and fee market post weekend also varies. An example with weekends highlighted in last one month:
Other things that affect the fee rates used are technical:
- Segwit usage: https://transactionfee.info/charts/payments-spending-segwit/
- Batching: https://bitcointechtalk.com/saving-up-to-80-on-bitcoin-transaction-fees-by-batching-payments-4147ab7009fb
- UTXO Consolidation: https://bitcoinops.org/en/xapo-utxo-consolidation/
- Fee estimation used in different wallets: https://blog.bitgo.com/the-challenges-of-bitcoin-transaction-fee-estimation-e47a64a61c72
- Coin selection algorithms used in different wallets: https://iohk.io/en/blog/posts/2018/07/03/self-organisation-in-coin-selection/
- Layer 2 usage: https://bitcoinkpis.com/layer2
- Exchanges: https://b10c.me/mempool-observations/2-bitmex-broadcast-13-utc/
Taproot will be added to this list once its activated and being used by different Bitcoin projects.
Will it be better if the wallets used this approach?
- Show mempool stats
- Leave the fee rate for user to decide
- RBF every transaction and follow different algorithms for automated bidding
A simple algorithm for automated bidding can be: https://i.stack.imgur.com/1SlPv.png
RBF algorithms for automated fee bidding can be tried and users should also manually prefer to use RBF for most of the transactions.
ZmnSCPxj shared few exceptions on bitcoin dev mailing list which should be handled while creating such algorithms:
Again, a race condition can occur --- while the wallet is feebumping a new transaction that includes the new output, a random miner can find a new block that includes the old transaction.
Further, of course it is convenient to be able to spend money even while it is unconfirmed.
But the sender of the unconfirmed input might be using the same software as this wallet as well, meaning that the actual transaction output might change as the original spender keeps fee-bumping it over time.
Jeremy Rubin shared few other ideas in response to my question related to fee estimates and RBF:
In the long term, this can be particularly useful since you can use a
separate fee-only wallet to arrange the bumps if your main wallet keys are
offline, and you do not disturb any of the on-chain dependants.
I'd like to share with you a draft proposal for a mechanism to replace
CPFP and RBF for
increasing fees on transactions in the mempool that should be more
robust against attacks. A reference implementation demonstrating these rules is available
for those who
prefer to not read specs.