How nodes agree to consensus rules? Is it possible for someone to change the consensus rules for his own software and then remain in the previous network? If not, then why?
2 Answers
If you alter the consensus rules in the software running your node, there are two options:
You apply rules less restrict than before, accepting some transactions that were previously invalid: every block you mine will be displaced by blocks from longer chains mined by the miners that didn't change anything and have more hashpower than you. Because they'll never accept your blocks but you're still accepting theirs.
You apply rules more restrict than before, rejecting some transactions that were previously valid: you are on your own, and will have to mine every single block. Depending on your current hashpower, it could take weeks until the next one. Unless you also change the rules concerning minimum difficulty. But the blocks you mine still will be accepted only by your node.
Is it possible for someone to change the consensus rules for his own software and then remain in the previous network?
It is theoretically possible to do this and still remain on the previous network especially if you are imposing an additional restriction (an effective soft fork) rather than a relaxation (an effective hard fork). However I would strongly recommend against doing so on mainnet if any funds with real world monetary value are at stake. The reason for this is even with soft forks you can be forked off the network following a different blockchain if you reject a block due to a transaction in this block not meeting your additional restriction. An example of this was demonstrated prior to the Taproot soft fork when there was one user running a full node who claimed to have been forked off the network for applying the Taproot rules too early when 0xB10C spent an anyone-can-spend Pay-to-Taproot output prior to activation.
When it comes to consensus rules the punishment for diverging from the rules being enforced by the rest of the network is significant. You risk being forked off the network, following a different blockchain and verifying transactions and blocks that aren't representing the actual Bitcoin blockchain (effectively a different currency or asset). So by all means experiment on testnet, signet with different consensus rules but when verifying mainnet transactions and blocks do not do this.