For example I have an address (A) with 20 BTC and send 9 BTC to address (B). Bitcoin will create a "change" portion of the transaction where the remaining 11 BTC goes to address (C).
Why not send the change back to A?
Instead of:
Initial balance: A: 20 BTC B: 0 BTC C: 0 BTC Transaction Pair: A -> B 9 BTC A -> C 11 BTC End result A: 0 BTC B: 9 BTC C: 11 BTC
Why not?
Initial balance: A: 20 BTC B: 0 BTC Transaction Pair: A -> B 9 BTC A -> A 11 BTC End result A: 11 BTC B: 9 BTC
Two reasons:
So whoever sent to your "A" address can't claim to have sent you 31 bitcoins ("See! Look on block explorer, there are two transactions sending to "A", one for 20 bitcoins and one for 11-- send me back the extra 11!")
Using a new change address makes it more difficult for other people to track of how many bitcoins you have or where you're spending them.
While the other answers are slightly true, there's another reason. Addresses which have been spent are inherently less secure than unspent addresses. This is because, when spending on an address, you reveal the public key to the address. This means that in order to steal those funds, you only need to find the private key, whereas normally you'd need to break the hash and find the private key.
Both of these tasks are extremely difficult, so for now it's not much of a concern if you send to spent addresses. The problem arises with the theoretical quantum computer. ECDSA, the technology which secures your private key, is extremely easy for quantum computers to reverse. The hash isn't. Therefore, when quantum computers become popular used addresses will become exponentially less secure.
More info at https://bitcoin.stackexchange.com/a/7134/11666
Also: https://bitcointalk.org/index.php?topic=153302.0
In the BitcoinTalk post, the concern brought up about hackers cracking transactions between broadcast and inclusion in blocks is reasonable. Fortunately, honest miners include the first transaction first regardless of fee, so that attack is foiled. It is also prevented by the fact that it would still take quite some time to crack the public key.
The idea of the change addresses is an attempt to make transactions anonymous. Unfortunately, bitcoin transactions are not anonymous, even with the use of change addresses:
When you have a wallet filled with change addresses, transactions will be forced to use some of those change addresses to fill transactions you want to make. The fact that the receiver is the same, and that the time of transaction is the same means that an outside observer will have a high confidence that those addresses belong to the same person. If all of those addresses only received bitcoins from a single address, it will be very clear they are change addresses for that address.
What the above means is that your addresses become deanonymized when they are used. This makes your privacy basically worthless because anyone can look back a couple years and see everyone you paid or received money from.
This is not necessarily a bad thing, as the tracability of transactions will make it harder for governments to justify fighting the adoption of bitcoins.
While you might gain a tiny bit of privacy for a short period of time using change addresses, there is a HUGE downside: it makes backing up your money intractable. If you want to back up an address, you can do that easily. Just encrypt your wallet and copy it to various devices (your phone, external HD, USB drive, paper wallet, etc). If change addresses are used tho, you must copy to all your backups (as painful as printing out a new paper wallet) every time you make a transactions. That is intractable.
That intractability I think is a HUGE problem for bitcoins. People need a secure way of being "their own bank" and change addresses make bitcoins broken for most people.
UPDATE: Deterministic wallets somewhat solve this problem by allowing you to determine all of the wallet's potential addresses from a single seed value. This allows you to backup once, and restore at any later time by using the seed to generate and check addresses until you're sure you haven't used any beyond the ones you checked. Armory, Electrum, and CarbonWallet all use deterministic wallets.
