First of all let's get a few misconceptions out of the way:
I was guessing that miners need to validate manually. Like, go to the Blockchain.com and check the last block and unconfirmed transactions and check by themselves, manually. So, I was imagining that it takes 3 or 4 minutes for them.
No, miners run full node software, just like you can run yourself, and they query this full node for block candidates to mine. If mining involved going to a centralized website, we could get rid of mining entirely, and just make blockchain.com decide what the next block is. The whole point is to never trust any other party, but be able to perform validation yourself. blockchain.com is a third-party website, operated by a private business, and has in no way a special position in the Bitcoin protocol or its operations. Using the website for anything but debugging purposes means trusting their interpretation of what's going on in the chain. Anyone can run such a website (and in fact, many competitors exist, like mempool.space, blockstream.info (disclaimer: my former employer), blockchair.com, blockcypher.com, ...).
So what this also means: deciding what the next block candidate is going to be isn't done by the mining hardware, but by software. That means one does not interfere with the other - they happen in parallel. The hardware is always mining on something, and the software is always verifying new transactions/blocks as they come in. From time to time, the software will give an updated block candidate to the hardware, based on its own best understanding of what blocks are transactions are available.
Now, how much time is spent on validating blocks and transactions? A significant amount of time, but it's important to point out that most of it isn't actually on the critical path for constructing new blocks. The process, in theory, goes approximately like this:
- New transactions are continuously received by the miner's node. These transactions are validated on the fly, and stored in the mempool (a place for unconfirmed transactions; every full node has its own independent one). Typical transactions take less than a millisecond to verify, but larger/more complex ones can exceed this. It also heavily depends on the hardware used, of course.
- At every point in time, the mempool is kept consistent with the best block the node knows about. That means that creating a new block template is very fast: just take the top transactions from the mempool (typically, for economical reasons, the ones with the highest fee per byte), as many as fit in a block (1 million bytes before BIP141; 4 million weight since). These transactions are already validated, so they don't need to be validated again at block candidate production time.
- From time to time a new block will be received. Typically, all, or almost all, transactions in that block will already be in the mempool when it is received. The P2P protocol is even optimized for this (see BIP152 compact blocks), avoiding the need to even repeat those transactions when transmitting the block. Because of this caching, block acceptance is very fast, but occasionally it may be slow if lots of unknown transactions were included in the block, for whatever reason. In this case, it make take 100s of milliseconds, or even multiple seconds on slower hardware, to validate a block.
- When a new block has been received and validated, the hardware must be informed as quickly as possible about a new candidate on top of this. Every millisecond before this is done means wasting the hardware's resources: it's working on a candidate on top of the previous block, which is unlikely to be accepted by the network (nodes typically only accept the earlier one of two competing blocks, unless the later one is extended first). In order to speed this up, some miners will in fact make their hardware switch over to working on an empty block on top of the newly received block, as soon as it arrives, and before waiting for it being validated. This is risky: as they don't know whether the received block is valid yet, this may mean switching over to working on a candidate that builds on top of an invalid block, which would be a waste of time itself. Around the time of BIP66's activation, this caused an entire chain of invalid blocks to be built, because miners started building on top of a block produced by an unupgraded other miner, without knowing it was invalid.
Most of this the same for every node. Non-miners obviously don't spend time on building candidate blocks, but apart from that, the effort spent on validating blocks and transactions is the same for everyone running a fully-validating node, miners and non-miners alike.