There are two factors that define the profitability of mining, and they tend to change somewhat independently of one another or at least lag one another quite significantly.
The first factor is difficulty. Blocks should be generated every 10 minutes on average, regardless of the amount of processing power available to the network, so difficulty adjusts how hard it is to find the solution to a block. This factor is based solely on the total hashrate of the bitcoin network.
The second factor is price. Assuming we're speaking of profitability in terms of US Dollars or other fiat currencies, we have to account for our earnings in the same currency that was used to pay for our costs. To my knowledge there is not (yet) a power company that will allow you to pay your electric bill in bitcoins.
From personal experience I can say that my dedicated rigs consume about $4 worth of electricity to produce 1 BTC and if I'm lucky enough to be average my hardware will have a long enough MTBF (mean time between failures) that its cost amortized over the life of the machine will perhaps bring that up to $4.50 or $5 per coin. At today's rate of $8.69 per BTC, it is still profitable though just barely.
CPU mining cannot produce at a high enough efficiency to be worth the electricity cost, and the same can likely be said for less efficient GPUs. Of course some folks don't pay per kilowatt-hour for electricity usage and for these folks ANY kind of mining is still profitable.
There are also a few folks on the forums selling FPGA boards which, though quite expensive up front, consume only 10-15% of the electricity of a comparable GPU. A single Xilinx Spartan6 board, for example, produces 100 MH/s at a little under 7 watts, compared to a Radeon HD 5830 which produces around 200 MH/s (without overclocking) at about 108 watts (54 watts per 100 MH/s).
