How do block explorers determine propagation through nodes/P2P protocol?

Question

On several Bitcoin blockchain explorers, I have seen unconfirmed transactions given a dedicated "propagation" value that tells a user how much of the Bitcoin network has become aware of this transaction.

For example, BlockCypher provides a "confidence" value for unconfirmed transactions:

This has always lead me to wonder how exactly they can calculate this? Do they personally have a large distributed network of nodes that are constantly kept track of? Surely this would be cost ineffective and less decentralized.

Their documentation states:

By monitoring transaction propagation, the number of nodes that have received it, and how quickly they received it, we can calculate its probability to be the “winning” transaction at any given point in time, if a double-spend was attempted.

I have had quite a difficult time finding documentation on what exactly nodes can request from each-other directly. Can a node simply ask its peers if the transaction exists in their mempool with a getData request?

Answer 1

Blockchain explorers run nodes which have connections to other nodes. They may have hundreds of connections. They likely also run multiple nodes for both redundancy (in case one node fails, they are still online) and to ensure that they are receiving transactions from as many sources as possible. So they will be connected to hundreds, maybe thousands of nodes.

With these connections, they can listen to what all of those nodes are broadcasting. In the current P2P relay protocol, a node announces to its peers that it has a transaction (the inv message). It then waits for a request for it (getdata) before sending the transaction (tx). So someone listening to many nodes would be able to know which nodes have a transaction based on whether they have announced the transaction. This is likely how Blockcypher is computing the "number of memory pools with tx".

However you should be wary of such measurements. They aren't very accurate because they are not connected to all nodes on the network. Furthermore, the confidence they associate with it is largely meaningless. A transaction's likelihood of going into a block depends on many other things, and the number of nodes that have seen it does not effect that number very much.

Answer 2

They don't, essentially, the numbers given for this sort of thing are generally meaningless.

Many of these services attempt to connect to a large portion of the listening network and gain insight into the chance of a transaction confirming, however this is not a measure of safety for transactions. A transaction having been announced by many nodes does not mean that it will not be double spent by a miner, as has happened many times in the past. For BlockCypher in particular, the confidence number does not use a Bitcoin Core node for validation, so will show very high confidence for transactions which are invalid, but have a seemingly high fee.

In the case linked with betcoin dice and ghash.io, they would have shown absolute confidence of confirmation, however the mining pool was simply including alternate transactions that did not spend to the service every time their bet was a negative. For a betting site with a 1% house edge, they only need to be able to do this more than 1% of the time for the attack to be financially profitable for them. This happened for a period of months with nobody noticing, no users stopping using the pool, or any other punitive measure. "Transaction confidence" or other tools only work in a non-adversarial environment, which Bitcoin is not.

Can a node simply ask its peers if the transaction exists in their mempool with a getData request?

You can to some degree ask a node that you are connected to if it has a transaction in its mempool. Nodes implement logic called transaction trickling which intentionally batches and delays relay of transactions in an attempt to prevent privacy adverse behavior such as this. Note that most nodes in the network do not have listening sockets open, so you can not make outbound connections to them. Merchants and miners in particular will not have incoming connections to prevent against trivial denial of service attacks against them.