How a web-service like bitnode.earn can know each node's blockchain height?

Question

How a web-service like bitnode.earn can know each node's blockchain height?

We can get some information of each node in bitcoin network in this web-site: https://bitnodes.earn.com/nodes/

And, it is updated regularly(in approximately every 5 minutes).

My question is from here. How(the method) the web-service can get all the information from all the nodes in the network? They ask those information to all nodes in every 5 minutes? Doesn't it take quite long time to get the responses from all the nodes(about 10K nodes in the network)? Also, doesn't it required to be in peer-relationship with all of them to request those information?

I'm waiting any answer from you guys. Thank you in advance for your nice answer!

Answer 1

Sites like bitnodes run their own lite nodes which are crawlers. These nodes connect to other nodes and only receive data from those nodes. When a node receives a new block, it will send it to the nodes it is connected to. If one of these is a bitnodes crawler, it can record that it received a block at height X from the node. In this way it can track the current height of a node.

Furthermore, nodes send their current height when they connect to another node. The crawlers, if they happen to disconnect from a node, can quickly retrieve the latest information about the node once they reconnect by simply waiting for the version message which must be sent in order for the connection to continue (it is part of the handshake at the start of every connection). The version also contains other information such as protocol version number and user agent string.

Other metrics such as up and latency come from the fact that the crawlers have open and active connections to those nodes. The fact that they have a connection means it is up. They can use the ping message to measure latency.