Why do individual nodes run "full nodes" that keep track of the entire blockchain whereas mechants can get away with "lite node" that implements "simplified payment verification" needing to examine only the last few blocks?
individual you mean
users I would argue the contrary as many wallet are SPV based or have a centralised backend running a full node (form of lightness as any client does not run a fullnode).
Only few desktop bitcoin wallet such as bitcoin core or armory run as fullnode. Fullnode are necessary for miner but it's still good for anyone to run it as it does apply
rules on new blocks/transactions whereas SPV's node doesn't which result with an healthier network. SPV's node needs full node to exist but worth to keep it mind that an SPV has to
trust its peers as it does not verify/apply rules.
Quoting Mastering Bitcoin:
An SPV node can definitely prove that a transaction exists but cannot verify that a transaction, such as a double-spend of the same UTXO, doesn’t exist because it doesn’t have a record of all transactions.
A full blockchain node verifies a transaction by checking the chain of thousands of blocks below it and checks that the UTXO is not spent, whereas an SPV node checks how deep the block is buried by a hand‐ ful of blocks above it.
A typical a merchant accepting bitcoin would rather be 100% that there is no attempt of double-spending which is not possible with an SPV's node.