3

Could the mining difficulty adjust in real time, not only when blocks are found?

(NOTE: Not asking about retargetting after every block but continuously as a function of time.)

I'm worried about how Bitcoin (and many other cryptos) change the difficulty. The basic idea is that the difficulty is adjusted every N blocks (N=2016 for Bitcoin and N=1 for Monero). That is, a difficulty can change only when a block is found. The problem is that if hashrate suddenly drops significantly, say 99%, then it would take 100x more time than normally to reach new lower difficulty. The drop could happen if the price of the coin drops and thus mining isn't anymore profitable. Network slows down and the price goes even more down and thus even more miners stop mining, and thus the coin might die. This seems like a very unstable system to me. I don't see much incentive for anyone to mine bitcoin if the price drops dramatically. The only incentive for some people to mine would be that they have lots of coins and they don't want the system to die. Not sure if this is enough.

To avoid this problem, could the difficulty adjust in real time, not only when blocks are found? That is, the difficulty requirement for a valid block would depend on the time it was found. The difficulty would decrease as a function of time, thus eventually a block will be found very easily. For instance, if 2 hours has passed since the last block, the difficulty could have already dropped to 1% of what it has been for the previous blocks.

The difficulty requirement of block N at time T would be a decreasing function of T and depends on the timestamps and difficulties of past blocks somehow. For instance, the difficulty could be a exponentially decaying function of time since the 10th last block and the mean difficulty of those 10 blocks. The difficulty of a mined block must be at least as great as given by this function in order for it to be valid. So the timestamp of a block becomes an important part of the block data.

Would this idea make sense? Is it implemented in any coin? Or is there some reason why it is not a problem that the difficulty adjusts when a block is found?

  • 1
    Possible duplicate of Why not retarget on every block? – alcio Dec 13 '17 at 10:29
  • Not exactly, although somewhat similar. I'm not asking about retargetting after every block but as a function of time. – Jaakko Luttinen Dec 13 '17 at 10:37
  • Not the same, the distinction is important. Bitcoin and most other coins know of the passing of time because blocks are produced and broadcast on the netowrk.. Other than affixing time-stamps to data, they completely ignore real-time. They only act on these time-stamps when a block is received. The question here is about adjusting difficulty in the complete absence of block production. The triggering event is an amount of elapsed real time, not the arrival of a block. – dbkeys Dec 13 '17 at 22:31
0

It's a security property that the difficulty is not retargeted quickly, not a technical or design limitation. It prevents a case where you can isolate a node, or a portion of the network, wait a brief period, and then mine at a vastly lower difficulty in order to defraud those users. If the mining rate drops, the fees rise, and more miners are incentivized to join the network- it's a self balancing system.

  • Ok. However, my suggestion about retargetting in real time isn't directly related to how quickly the retargetting should happen. It could take 1 week to drop the difficulty to half or whatever speed is wanted. The point is that one doesn't need to find blocks in order to retarget. It would be nice to know that the difficulty will eventually drop to such a level that the system will operate again after a major hashrate collapse. – Jaakko Luttinen Dec 13 '17 at 8:41
  • Clarifying myself a bit: Although the retargetting would happen in real time, it doesn't mean that difficulty needs to change quickly. Difficulty can change slowly but it changes as a function of time, not (only) as a function of found blocks. So I don't see how this would be anymore unstable than the current retargetting solution. – Jaakko Luttinen Dec 13 '17 at 10:58
  • Let's think of a case where the network is separted into two large groups. It's a far fetched scenario, but interesting. Suppose a fiber optic link breaks to China and all of Asia is isolated from the rest of the world. Each group of miners is then faced with too-high difficulty. If it is much too high, blocks won't be produced, and the network does not process transactions and is unusable. There ought to be the option for "local difficulty reduction" and a later merging of the forked chains, clearly with safeguards against double spends, etc. – dbkeys Dec 13 '17 at 15:32
0

I believe the answer is an emphatic YES, but just after a block has been found I would not immediately start applying the function-of-time difficulty reduction. I believe that would not be desirable, because mining is a random process. One should allow for the natural variability of the process before judging it to be "out of balance" and needing extra-ordinary adjusment. (Think of mining as tossing a dice with 2^256 faces, but only a subset of those faces are winners for a given toss.) It obeys these laws of probability and statistics:

  • The number of blocks in an interval follows the Poisson distribution
  • The time between consecutive blocks follows the Exponential distribution Probability of a block being found in time less than or equal to "t" is: P(X≤t) = 1-e^(-λt) where λ = 1/β , and β = target interval time

Applying the exponential probability function in the case of bitcoin, the likelihood of a block being found in 10 minutes or less is:

1-e^(-(1/600 seconds) * 600 seconds) = 1-e^(-1) = 63.2120558829%

The likelihood of a block being found in 60 minutes or less is: 99.7521247823% .... and If we go out to 15,000 seconds (about 4.17 hours), the likelihood of a block having been found is 99.9999999986

Another way to state this is that, in 99.9999999986% of all cases, a block SHOULD HAVE BEEN FOUND by 4.17 hours; it is therefore a very sure bet, if the block has not been found, that there is, as you notice could happen, a serious mismatch between the true network hash rate and the difficulty factor of mining.

An arbitrary threshold time can be chose, beyond the which, if a block has not been found, the mining difficulty factor is to be lowered. Choosing this threshold is equivalent to asking 'How sure do you want to be that the system is unbalanced ... before "messing" with it ? '

Only if we're fairly certain that the system is really unbalanced (meaning difficulty is not matched to the current real network hashrate) should we start executing a protocol or algorithm, of 'extraordinary' difficulty reduction. It is at this point that I would consider the idea of an exponentially decaying-with-time difficulty function.

All this is predicated on accurate, shared time consensus. Establishing what the correct time is among a number of machines on the internet is not a trivial problem, but fortunately, has been solved since the mid 80's by the Network Time Protocol (NTP), and since the 1990's complemented by the inexpensive availability of very precise and globally availabe GPS satellite time signals.

(NTP was developed by Dr. Dave Mills at the University of Delaware and a new, streamlined and security-hardened version of NTPSec is by Eric S. Raymond and his group). The correct time is not a matter of opinion, but of astronomical fact. Building a strong consensus of the best expression of current real time is NTP's job, and it does it quite well.

So, assuming that honest nodes, the mining "true chimers", have and use the correct real time, they will all know when a block is overdue, and according to the coin's protocol, can then agree that it is "too long overdue": current wait for the next block has exceeded the pre-established trigger wait threshold. Extraordinary measures must be taken. The question now becomes how much to reduce the difficulty factor.

The difficulty factor must be set to match network hashrate, BUT since no blocks are being produced, curent network hashrate can not be calculated : what must follow is a guessing stage, where a first difficulty reduction is made, then the network waits a reasonable time for a block to be produced. But, if again, no blocks are produced, then another difficulty reduction has to be made, since clearly the initial reduction was insufficient.

This protocol for difficulty reduction, (let's call it extraordinary difficulty reduction mode or 'EDRed mode'), may have many variations. I would suggest the first reduction be maybe 1/2 to 2/3, and that he waiting period for the next block after 1st reduction be a shorter wait than the first trigger threshold. Eventually, after 1, 2 or maybe even 3 or 4 reductions, blocks will start to be produced again, because the network's mining difficulty factor will be matched to the real network hash rate. It is even likely that the difficulty will be a bit too "easy" since substantial, "guessing" reductions were done - but the regular difficulty adjusting algos (Kimoto's Gravity Well or Dark Gravity Wave for example) used in alt-coins are quite good at quickly raising difficulty to more precisely match hashrate.

Once blocks are coming out again (that is, 2 or 3 blocks have been produced without another EDRed mode reduction being triggered), then the usual KGW or DGW algos can be allowed to govern difficulty again and EDRed mode is canceled.

(Bitcoin may have to adopt a more sophisticated difficulty-adjusting algo for its regular operation to implement a real-time-aware difficulty reduction protocol like this. I believe the recent Bitcoin Cash fork did adopt a more responsive algo than the simple adjustment every 2016 blocks of Bitcoin Core)

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.