Bitcoin mining is essentially just the repeated calculation of sha256 checksums. Given that sha256 is a fairly simple algorithm that uses only basic integer calculations absolutely any processor core should be capable of churning out sha256 checksums very quickly and any added complexity beyond the basic instructions required for sha256 is wasted. To that end the ideal candidate for mining hardware should have the least complex cores necessary for sha256 but the largest possible number of cores.
Your CPU has probably between 2 and 8 cores, each of them insanely complex. Your GPU has comparatively simple cores, but probably has over a thousand of them. Still, even GPU cores are more complex than they need to be for sha256, which is where FPGAs and ASICs come in.
An ASIC is a custom chip, designed to do only one thing - in this case it's designed to perform two rounds of sha256. It has only the necessary components for its one and only purpose. An FPGA is a device commonly used to prototype ASICs, it's basically a chip whose components can be rearranged to mimic another kind of chip. Because they retain this multi-purpose capability FPGAs are significantly less efficient than ASICs.
The only (temporary) benefit we saw from FPGAs is the cost curve. FPGAs are pre-manufactured consumer devices, you can buy just one of them and they usually don't cost a huge amount of money. An ASIC is something you have manufactured, and the minimum order size is usually absurd, requiring a huge up-front investment to get your chips made.