Suppose organization X has 51% of the hash power for a period of 1 week. In this week, what exactly can and can't X do?
Actually, it's very easy to do damage to the network once you have 51%; just build your own chain faster than the network, and broadcast it whenever you like. If you send some of your coins to a new address in your own chain, all the transactions issued in the live network by spending those same coins will be reversed at the moment the longer chain is broadcast.
Right from the bitcoin wiki (probably proof-read by many pairs of eyes) :
An attacker that controls more than 50% of the network's computing power can, for the time that he is in control, exclude and modify the ordering of transactions. This allows him to:
- Reverse transactions that he sends while he's in control
- Prevent some or all transactions from gaining any confirmations
- Prevent some or all other generators from getting any generations
The attacker can't:
- Reverse other people's transactions
- Prevent transactions from being sent at all (they'll show as 0/unconfirmed)
- Change the number of coins generated per block
- Create coins out of thin air
- Send coins that never belonged to him
It's much more difficult to change historical blocks, and it becomes exponentially more difficult the further back you go. As above, changing historical blocks only allows you to exclude and change the ordering of transactions. It's impossible to change blocks created before the last checkpoint.
Since this attack doesn't permit all that much power over the network, it is expected that no one will attempt it. A profit-seeking person will always gain more by just following the rules, and even someone trying to destroy the system will probably find other attacks more attractive. However, if this attack is successfully executed, it will be difficult or impossible to "untangle" the mess created — any changes the attacker makes might become permanent.
In theory, this attacker owns enough computing power that they could execute a "double spend" attack. They could spend coins in one place, allow the coins to enter the block chain as normal until the required confirmations are met, then fire up their 51% of the miners to craft a fraudulent fork of the block chain in which those coins were never spent, allowing them to re-spend the coins. This could theoretically be repeated for as long as the attacker maintained control of 51% or more of the hashrate.
Realistically, 51% is only the point at which this becomes possible not the point at which it becomes likely or easy. An attacker would probably need something like 65% to actually execute such an attack.
And then there is the denial of service possibility of suddenly withdrawing from the service, taking the necessary computing resources away to continue to solve blocks every ten minutes until the difficulty is adjusted down again (which could take a long time if there is only a block every day for example).
Of course, for that one would need much more than 51% of hash power.
Anyone who owns 51% of the network will have made a massive investment in hardware and systems to organize and construct a machine capable of executing such an attack. If their motive is profit, then the short term gain associated with forking the block chain to enable 'double spend' will net them a negligible benefit; it's difficult to imagine they would pursue this strategy on the basis that the resulting instability will ultimately de-value the very coins they seek to 'spend twice'. If their motive is to destroy Bitcoin, period, that is another matter altogether. That kind of techno-vandalism could only reasonably be motivated by someone with destruction and disruption serving as their primary motivation.
Instead, I posit it's much more likely that such a massive and powerful compute resource (Bitcoin supercomputer) will be used to power the vast bulk of the network within the bounds of its intended use, profiting long term from generation rewards and transaction fees, as the network grows and prospers over time.
The answers so far focus on the algorithm itself, I have a few social economic thoughts to add.
Let's assume Bitcoin is massively popular and indeed becomes THE global go-to currency, at this point this and similar questions become (very) relevant.
What happens in maturing industries is that through commoditization and mergers smaller and smaller numbers of players remain. Through scale advantages this small number of players will be able to provide services at lower cost and squeeze out smaller players. I see little reason the industry of Bitcoin transaction processing will be exempt from this general rule.
Next, we cannot foresee every aspect of the future, even though the Bitcoin designers did a terrific job there will be situations that will call for changes to the system. For example there might be a call from the people to stop child porn networks, to stop capital shelters for the rich, to stop overly profitable and powerful corporations,... etcetera, you name it. Whether justified or not, the people will demand for changes, not necessarily a villain government individual, the people.
Since there is only a small number of players it is actually possible to regulate the industry. For example the regulation could be that only payments with a traceable account number will be processed, or only payments with attached fees that include a portion for tax.
I would think the government could even demand changes to the core of the algorithm. Preventing, for example, "non-certified" players to enter, thereby further establishing the power of the existing payment processors.
The newly elected monopolists will then, in the final phase of capitalism self-destruction slowly but steadily raise their processing prices, eventually driving customers away and causing the Bitcoin to never reach the deflationary status many proponents and early investors claim it will have.
And let's just hope it ends this way, a forking scenario from this could be that the Bitcoin reaches "too big to fail" status, and the people demand further regulation (of processing fees, mining speed caps, etc). We will all keep paying a premium on the existence of the currency, just for the sake of stability and the fear for disruption of the status quo. Just like with today's currencies.
I'm not trying to be skeptical, I'm actually very hopeful the crypto currencies are going to help with globalization and advance humanity. As a deflationary currency to "easily" save for your (early) retirement I am not so sure. As a transaction system probably in some way.
Maybe we don't actually need a "currency" maybe all we need is a transaction. Maybe there can be a super layer on top of multiple competing crypto currencies that quickly and automatically switches your money back and forth between the best suitable mix of currencies and investment funds. After all what you really care about is how your salary is exchanged into goods and future promises.
I suspect he might be able to mine more than 51% of the blocks. See Can someone with 51% computing power earn more than he deserves?