All the nodes in the Bitcoin Network have agreed on a certain set of rules. These rules are coded into the client and everything that changes the state of the ledger is checked against those rules.
Hardforks happen when a change in those rules (for example the reward halving rule or the difficulty adjustment rule) causes the new clients to make different decisions about the validity of transactions and blocks. This basically means that over time the state of the ledger on older clients would diverge from the state at new clients, creating two incompatible networks, as transactions would be valid in one but not in the other.
As transactions and blocks are chained together, even a single disagreement about the validity of a transaction or block can cause large parts of the state to become inconsistent over time, it is therefore crucial that the network as a whole agrees on the rules.
The developers keep a wishlist of changes for hardforks, because hardforks should be a rare event, breaking changes be bundled into on hardfork. The changes decided in hardforks have a trigger height after which they are applied, which is usually several months in the future in order to let developers of alternative clients and the users update their nodes before the switch, minimizing the number of incompatible nodes left behind.