Peter Todd has an in-depth answer to this question here:
https://petertodd.org/assets/commitments/52ccc4802bd563076cbd25ec4c1ba88152098cb6aa356ba644c9e79a24182da5.txt
What attack does the difficulty drop rate limiter prevent?
The Bitcoin protocol limits the rate at which the difficulty can decrease to no
less than 1/4th of the prior period's difficulty per adjustment period.
Specifically, in the difficulty calculation the variable containing the actual
time it took to generate 2016 blocks is capped at 4x the expected time.
// Limit adjustment step
int64_t nActualTimespan = pindexLast->GetBlockTime() - nFirstBlockTime;
if (nActualTimespan > params.nPowTargetTimespan*4)
nActualTimespan = params.nPowTargetTimespan*4;
So why did Satoshi do this? What attack does this rate limiter prevent?
Suppose I want to trick you into accepting a chain that I've mined with less
work than the valid chain with the most work in existence. If I can
successfully sybil attack you, I can prevent you from learning about the
most-work chain.
Let's suppose I start the sybil attack when you know of the existance of block
n=(k*2016 - 1), which happens to be one block prior to the end of the difficulty
adjustment period, and has timestamp t.
I now mine block n+1 with the largest timestamp you'll accept (your local clock
plus 2hrs). Without the rate limiter, the amount that the difficulty would drop
is limited only by the difference between time t and now - that may be very
large if your node has been off for awhile, you're in initial sync, or I manage
to get you to reset your chainstate (e.g. by social engineering, or with a
chainstate corruption exploit). For example, if I manage to roll you back a
year, the difficulty would drop to 1/26th of the last block you know about.
With the difficulty dropped, I can then mine blocks quite cheaply to make you
think the transaction you're accepting has a significant number of
confirmations.
By limiting the rate at which difficulty can drop we make this attack
significantly more expensive: my cost-per-block is now only 1/4th of what it
should be, a significant increase compared to the situation without the drop
rate limit.
Synchrony/Latency
Of course, all consensus systems have a network synchrony assumption of some
kind - if the participants can't communicate in a sufficiently timely manner
it's impossible to come to consensus; the infinite latency case being an
obvious example. But even if the system isn't functioning properly, we still
want to limit what an attacker can do: in combination with PoW, the difficulty
drop limit ensures that even if latency is infinite, attacks against
participants are still expensive, turning a potentially disastrous
double-spend attack into a much less harmful DoS attack.
PoW vs PoS
Finally, note how important PoW was here: PoW guarantees that this attack will
be expensive, by virtue of the fact that the attacker has no way of avoiding
the need to spend funds to destroy energy - or in the case of stolen hash
power, give up the opportunity to mine blocks extending the most-work chain.
Meanwhile in a PoS system this attack can be done for free: even in a "slasher"
style PoS system, since we've sybil attacked the target we can prevent the
slasher mechanism from working. The proof of our fraud will never reach the
outside world, and thus has no impact on the value of our staked coins.