BIP 157 and BIP 158 are both concerned with Compact Block filters. What is the distinction between them, and which of them does Bitcoin Core currently support?
Add a comment
|
1 Answer
BIP157/158 Overview
This answer is a slight modification on the description used in Bitcoin Optech Newsletter #43. Full credit and thanks to Dave Harding!
BIP158 introduces Compact block filters, which are are based on an efficient method for encoding a list of equally-sized items. In the case of the "basic" block filters described in the BIP, this is a list of all the spendable output scriptPubKeys in the current block plus all the scriptPubKeys for the outputs spent by this block's inputs (what developers call previous outputs (prevouts)). Each of the scriptPubKeys is hashed to give each item the same size and then these items are sorted into a list that has duplicated elements removed. This list is then encoded using the Golomb-Rice Coded Sets (GCS) algorithm also described in BIP158, losslessly reducing the size of the list. This specific basic filter provides enough information for anyone who knows a Bitcoin address to find any block containing a transaction either paying that address (output scriptPubKey) or spending funds previously received to that address (prevout scriptPubKey). The search may produce false-postive matches (so blocks which don't contain transactions for that address will be included in the results), but will never result in false-negatives (so blocks that do contain transactions for that address will never be omitted from the results).
A separate BIP, BIP157, describes how these compact block filters can be served over the network using the Bitcoin P2P protocol. BIP157 is designed to work with BIP158 "basic" filters but it can also be extended to support additional filters that encode lists of other items. One particularly noteworthy part of BIP157 is that it introduces the concept of filter headers where the header for each filter commits to a hash of the previous block's filter header plus a hash of the current filter. This creates a chain of filters similar to Bitcoin's chain of blocks and is designed to make it easy to compare filters from multiple peers: each peer can send just the filter header (32 bytes) and, if there are any headers that don't match, the client can request earlier and earlier headers in the chain until the divergence point is found. Generating a filter header on demand for a particular block would require hashing all previous filters, so servers that provide BIP157 filter headers will generally need to store them in advance.
Bitcoin Core Implementation
Bitcoin Core can generate and store
block filters and filter headers, and users can access them using the getblockfilter
RPC (implemented with PR#12254 & PR#14121).
Bitcoin Core can serve block filters over the P2P network, but not by default; it can be enabled with -peerblockfilters=1. BIP157 was implemented with a series of PRs, as shown in PR#18876.
Future Related Developments
PR#23549(v25.0) and PR#25957(v25.0)
are different efforts to achieve faster wallet rescans by using the node's already constructed compact block filter index (-blockfilterindex=1).
