5

BIP 157 and BIP 158 are both concerned with Compact Block filters. What is the distinction between them, and which of them does Bitcoin Core currently support?

4

This answer is a slight modification on the description used in Bitcoin Optech Newsletter #43. Full credit and thanks to Dave Harding!

BIP158 introduces Compact block filters, which are are based on an efficient method for encoding a list of equally-sized items. In the case of the "basic" block filters described in the BIP, this is a list of all the spendable output scriptPubKeys in the current block plus all the scriptPubKeys for the outputs spent by this block's inputs (what developers call previous outputs (prevouts)). Each of the scriptPubKeys is hashed to give each item the same size and then these items are sorted into a list that has duplicated elements removed. This list is then encoded using the Golomb-Rice Coded Sets (GCS) algorithm also described in BIP158, losslessly reducing the size of the list. This specific basic filter provides enough information for anyone who knows a Bitcoin address to find any block containing a transaction either paying that address (output scriptPubKey) or spending funds previously received to that address (prevout scriptPubKey). The search may produce false-postive matches (so blocks which don't contain transactions for that address will be included in the results), but will never result in false-negatives (so blocks that do contain transactions for that address will never be omitted from the results).

A separate BIP, BIP157, describes how these compact block filters can be served over the network using the Bitcoin P2P protocol. BIP157 is designed to work with BIP158 "basic" filters but it can also be extended to support additional filters that encode lists of other items. One particularly noteworthy part of BIP157 is that it introduces the concept of filter headers where the header for each filter commits to a hash of the previous block's filter header plus a hash of the current filter. This creates a chain of filters similar to Bitcoin's chain of blocks and is designed to make it easy to compare filters from multiple peers: each peer can send just the filter header (32 bytes) and, if there are any headers that don't match, the client can request earlier and earlier headers in the chain until the divergence point is found. Generating a filter header on demand for a particular block would require hashing all previous filters, so servers that are provide BIP157 filter headers will generally need to store them in advance.

At the time of writing (April 2019), Bitcoin Core can generate and store block filters and filter headers, and users can access them using the getblockfilter RPC. Bitcoin Core does not currently provide block filters or filters headers over the P2P network.

Here is an example getblockfilter return object:

$ bitcoin-cli getblockfilter $( bitcoin-cli getblockhash 170 )
{
"filter": "0357e49590040c79b0",
"header": "349eaecc8bb7793c9f3c28e78df6675ef904515e9a310e4532785aeb45526090"
}

We selected block 170 because its filter is the first to contain more than one element (it holds 3 elements) and because the latest block as of this writing (block 572,879) has a filter that contains 8,599 elements---far too much for us to print elegantly.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.