I have a Bitcoin client running on my PC and I turn it off. When I turn it on again how does it find the last common block in the blockchain so that it knows which blocks it should obtain?
Your node will just ask its peers for blocks. It actually does the same thing it does for the initial sync. It starts with what it currently considers to be the chain tip (in initial sync, that's the genesis block). From there, it connects to other nodes and asks those nodes for more block headers starting with the block at its own chain tip. Typically, that block will be part of the current best blockchain, so the other nodes will just give further headers and then blocks from your node's tip.
If there has been a fork (e.g. the tip is a stale block) and the other nodes happen to know your node's current tip, then those other nodes will find the common ancestor between that block and their current tip. Once they find that common ancestor, they send the rest of the blockchain from that common ancestor. The important thing here is that the other nodes find the common ancestor, not your node.
If the other nodes don't know your node's current tip, your node will also send other, older block hashes in its headers request. So the other nodes should be able to find a common ancestor given those other headers too.