Is there a way to know how good a miner is choosing its transanctions to be mined from a fee maximization point of view?

Question

Well, this is the way I've tried:

When a mined block arrives to my node, I compare it against a transaction selection algorithm using my mempool. (I know my mempool it's different than the miner's one, and that's the point so keep reading ;-) )

The transaction selection algorithm sorts transactions in the usual greedy way: using the regular Ancestor Set Based (ASB) algorithm defined here, and considering transaction dependencies and CPFP, so basically is the same as invoking getblockTemplate with the state of my mempool just before the block arrives. (I also compare the incoming block by using the last getBlockTemplate result of bitcoind but as it's only calculated each 30s the results are not accurate).

An example of a comparison is shown here, (my webpage, in development). I divide the transactions in sets whether are ignored by the miner (from my node point of view), are in common with me, Ignored by me, Relayed to me, or not relayed to me.

And, of course, I also measure the fees I'd collected against the miner's block (having into account the incoming block coin base transaction size for better accuracy). You can check the results of the last blocks here, and an aggregate for the miners here.

This could give me an idea about how good a miner is selecting its transactions and/or has good network connectivity (in average, and against my mempool), but as I have no way of knowing the state of the miners mempool before they mine a block (not when I receive it), this effort seems inaccurate.

I make the assumption that a received block and my block template can be compared. But as the block propagation time is not, by any means, negligible, the comparison is skewed giving the idea that my algorithm/network situation is better. The reality is that I am receiving transactions while the block is being propagated to me, and thus, I have normally a bigger mempool than the miner when the block was mined. A bigger mempool gives my algorithm the opportunity to search for better transactions which had no opportunity to be mined by miners.

So, Is the accuracy of this methodology worthy? I've assumed its not. But as block propagation time is getting smaller (see this) And the results for some blocks/miners seems really different (see this block for example) I have my doubts.

Thanks in advance.

Answer 1

As you describe, you cannot distinguish the reason for the divergence between the block you received and the block you get from getblocktemplate.

It may be helpful to isolate the reasons:

• If you are trying to compare algorithms, you should try building a block with getblocktemplate (gbt) and with your own algorithm from the same mempool.
• If you're trying to compare network propagation, you may want to run multiple nodes, and assume that the miner built with gbt given everything that they had seen at this point. You may even want to keep track of when a transaction was seen, to reason about whether it is fathomable that it hasn't been seen by the miner.
  I think it's reasonable to assume that miners are building the best possible block according to gbt, although odd ordering of transactions may indicate some manual prioritization of transactions. However, you will have at least two sources of error: the latency from block propagation, and the delay between the mining pool operator giving out a template and the executing miner discovering the block and announcing it. However, these two delays may be too high for the block content to be an accurate representation of the mining pool's mempool content.

From what I understand, other research projects looking into transaction propagation measure it by keeping track of when nodes announce txids/block ids to them. It wouldn't be trivial to identify the nodes operated by miners to measure their network quality, though, if they are even part of the open network and not moated by border nodes.