TL;DR - There's not enough coin for everyone on earth to have 1, or even 0.01 -- the value will eventually adjust once early adopters sell their massive stores of coin, thereby spreading the coin around.
Aside from what others have mentioned, here's the problem as I see it:
There's still quite a few early adopters who are selling at below what the market value should be. They mined early, and instead of buying 10,000 BTC pizzas, they sat on the coin until it was worth selling.
Now that it's hit some great numbers (in the greater scheme of things), they're selling steadily -- not enough to kill the price, but enough to keep it from climbing with the network difficulty.
Current miners who know the current cost of mining (which is pretty damned significant in terms of computer costs, power costs, and especially time) are a lot less likely to dump their coin at current-day prices. They, like the earliest adopters, would rather see bitcoin at higher marks to let their coins go for fiat. This is where I fall in as well.
Sure, I can sell my few coin at today's prices, but if I make a profit (not sure if I do after with power costs), the profit would barely be worth the time. I'd rather wait for more digits -- hitting $1,000/btc again would be nice to see, and I'd probably start spending a bit of coin here and there, but until we see $5,000+, I'm not going to sell any coin.
Given that $4,500 worth of hardware, tons of power and heat, and endless RMAs only gets me 1-2% of a bitcoin per day right now (WAY less than a few months back), I feel that if I sold all of my coin, it would take me a very long time to see a full coin again.
Remember that we've got over 7 billion people on this planet currently. By the time all bitcoin is mined, probably closer to 8 billion. With only 21 million coins to go around, if everyone had an equal share, they'd have what, 200k Satoshi? (0.002 bitcoin)
By sitting on a few bitcoin, you're hundreds of times better off than what the average person can possibly achieve in terms of bitcoin savings. Also, remember that the past few times the value jumped, it did so by 10-12x -- wait for a few more of those. :)