While attempting to fetch a particular block my node just gets disconnected by peers?

Question

A system at an office has its testnet node stuck at a particular block.

The logs report "requesting block" over and over again for the same block and after each one reports "socket recv error" and that peer disconnects. Then it connects a new peer and the same thing happens. There are no log entries about INVALID blocks.

As a result the system hasn't had a new block in two days, but all the explorers report that the chain has continued to move advance.

The current block this node is on is a member of the best chain reported by the explorers, but thousands behind.

This node is running on Ubuntu in the US. The office network has a sonicwall firewall.

Answer 1

Usually when a node is stuck it is due to local corruption causing the node to incorrectly believe a block is invalid. But in those cases the logs will reflect the invalid blocks. Instead we don't see any invalidity here but the node is continually trying to get the block and getting disconnected.

This situation can be caused by overly aggressive anti-virus which is falsely detecting data in a block as malicious and cutting the connection. Sometimes this kind of filtering is applied to the general public by nation-state level filtering but it also shows up on office networks and in some home setups (e.g. in windows AV software).

For sonicwall you can see the disconnections in the firewalls logs and disable the filtering entirely by going into the firewall settings under sonicwall->manage->security services->gateway-antivirus and unchecking "Enable Gateway Anti-Virus", or by whitelisting the false positive match for the particular host in question.

(Thanks to Ken on IRC who had this actual issue, and for telling me how he disabled it on the sonicwall)