Why does Bitcoin Core support a transaction index but not an address index?

Question

Bitcoin Core has the CLI option -txindex to maintain a full transaction index for RPC calls. That sounds useful for some use cases, like for a blockchain explorer. It seems perfectly logical then that there should also be an option to enable an address index, which would allow looking up address balances and transaction histories, but there isn't one. Why is that? And why exactly was -txindex included in the first place?

Answer 1

The transaction index exists because it is leftover from how Bitcoin Core used to work for validating new blocks and transactions. Prior to 0.8.0, Bitcoin Core maintained the transaction index, and when validating transactions, would look up outputs by going into the transaction index, finding the transaction being spent, then finding its output. The transaction index was integral in validating transaction.

0.8.0 introduced a feature called ultraprune which gave us the validation model that we have today. What ultraprune did was that it got rid of the reliance on the transaction index and instead UTXOs were stored in a separate database. New transactions would be validated by looking up the UTXO in this other database, and when the UTXO is spent, it is removed from the database. This was introduced because it is much faster and it allowed for the transaction index to not be built and thus "prune" the database.

But instead of deleting all of the transaction index code, it was decided to put that feature behind the -txindex option. This is because there were already users relying on the ability to look up arbitrary transactions, so it would not be good if Bitcoin Core just removed this feature out from under them. This is how we have the transaction index today - it wasn't a feature that was implemented because people wanted it, but rather it is a leftover from how Bitcoin originally worked.

(Side note: I think that if Bitcoin Core had used ultraprune originally and not needed a transaction index in the past, there would not be a transaction index feature.)

---

An address index doesn't have the benefit of existing previously, so it would have to be something that is completely new. And there have been several attempts to implement an address index, by my count, 4 attempts. None of them have been merged.

The main objection is really one of the scope of the Bitcoin Core project. Is it really in the scope of Bitcoin Core to provide a blockchain explorer to every user? Is that really necessary, and if so, is it worth it to maintain that feature? Because once the feature is added, it will basically have to be maintained in perpetuity.

To understand why many contributors have not been favorable towards adding an address index, we need to understand what the use case is.

As you mention, it is allow looking up address balances, but that can already be done without an address index. It's completely unnecessary to build an index of all addresses and their transaction histories to calculate their balance; a balance calculation can be done with the UTXO set itself (which every node must maintain anyways). In fact, Bitcoin Core already implements this via the scantxoutset RPC. So this use case doesn't actually need an address index.

As you mention, it would allow looking up the transaction history for an address. But so far no one has really came up with a compelling reason for why this is useful. Sure it's cool to be able to look at an addresses transaction history, but is it so cool that it's necessary to implement it into Bitcoin Core and now burden the Core developers to maintain it in perpetuity?

For many of the use cases of address indexes I have seen proposed, it seems like each one need slightly different data, or data formatted in slightly different ways. For example, Electrum servers require an address index. Currently they have their own (multiple) software that builds the address index and speaks the Electrum protocol. But because of some quirks of the Electrum protocol, everything is provided as hashes (not addresses). So in order for an address index in Bitcoin Core to be useful to Electrum, it would also have to index by script hashes. Otherwise, Electrum will still have to write their own software to build their own address indexes.

For building these kind of services on top of an address index, it seems like it is more reasonable for people making those services to just build an address index themselves. That way they can store the data in exactly the way that they need it rather than hoping that the developers of an upstream project will store the data in a way that they can use it.

So it really comes down to the fact that no one has come up with a compelling reason for Bitcoin Core to implement an address index. For all of the things that need an address index, they can build their own address index and tailor it to their needs.