How is the "N" in PPLNS chosen? 2xDiff appears to make no sense - OR - Why are Scrypt share difficulties off by a factor of 65536?

Question

while my example in the following question is using the altcoin "Bitconnect", I believe it is actually a general question which fits here - if not, let me know.

I am struggling to understand PPLNS (I need to implement it in a pool). I know in general how the system works, but what is a mystery to me is where the N comes from.

I read on many places on the Internet that N = 2 * difficulty is a good value, and that this would mean that on average a miner will earn rewards from two blocks for the same shares.

However, it simply does not add up for me.

Practical example: BitConnect pool. Let's say BitConnect's difficulty is currently ~900,000.

An AntMiner L3+ submits a share at difficulty ~150,000 (which counts as a share value of 150,000 then) every ~15 seconds, so in 3 minutes, that one AntMiner has already filled the whole "N" of 900,000x2=1,800,000. That would, in reverse, mean that I would expect to mine a BitConnect block every 90 seconds on average, with one AntMiner, which is of course not true.

100 AntMiners running on our pool gave a share value of 230,000,000 in the last 5 minutes, which would have been more than 100 times the whole "N" already!

So something can't be right here, and I don't understand what. Should I not multiply the submitted share with the work difficulty? But then a small miner would get the same reward as a large one which makes no sense. Or is the whole Internet wrong about the 2*diff for "N", and that it's based on the assumption that it takes on average a value of shares equal to the difficulty to find a block? Doesn't sound likely.

On average, it takes 2^32 hashes to find a valid share at difficulty 1, right? And on average, it takes X shares to find a share at difficulty X, hence why you increase the difficulty at the pool for your miner if the miner is powerful (it will then less often submit a share), right...? But the last point starts to make less sense already, together with the fact that you need a share at difficulty Y to find a block if the currently coin block difficutly is Y... And the pool software counts a share with difficulty X the same as X shares with difficulty 1. At the end I'm left confused how it all works together and how I can arrive at a value for "N" which actually results in a miner getting on average two rewards for the same share...!

Please help me understand what is wrong.

EDIT: I found that all calculations differ from reality by a factor of 2^16. Then I found this: https://bitcoin.stackexchange.com/a/11816/62320 This would suggest that for Scrypt, pools use a scaled "share difficulty" value by exactly that factor, but the reported block difficulty would not be scaled. This would explain everything but it's the only place I found this on the Internet (and it would mean that I need different calculations depending on the algorithm I'm handling) - can anyone confirm this?

Answer 1

An AntMiner L3+ submits a share at difficulty ~150,000 (which counts as a share value of 150,000 then) every ~15 seconds, so in 3 minutes, that one AntMiner has already filled the whole "N" of 900,000x2=1,800,000. That would, in reverse, mean that I would expect to mine a BitConnect block every 90 seconds on average, with one AntMiner, which is of course not true.

I don't think that is how shares are counted. According to this Bitcointalk post,

When a share is submitted, assign to it a score of 1/D, where D is the difficulty at the time the share is submitted.

So in your scenario, when your AntMiner L3+ submits a share, that share is assigned a score of 1/900000, so its "share value" is 2, not 150000.

Answer 2

I found the reason for this confusion myself now:

It was very confusing and took a while to understand, especially because this information is not easily obtainable when researching on the Internet (without already knowing the answer in order to know the right keywords - forum threads didn't help either).

It seems like there are algorithms, Scrypt is one of them, which use a scaled value for share difficulty.

Let me explain:

For Bitcoin (which is SHA-256), the following applies:

• A hash is a "try" to find a certain solution
• On average, you need 2^32 (~4 billion) tries to find a valid "share" (originally this was the only type of share in existence, which has difficulty 1, but by now, all pools use shares with difficulties higher than 1, see below)
• If the coin difficulty were 1, this would be a valid block
• With a coin difficulty higher than 1, let's say N, on average only 1 out of N such "valid shares" (with difficulty 1) get you a valid block
• Recap: for coin difficulty N, you need on average N*2^32 hashes to find a block
• Therefore, a formula to know how many seconds on average it takes to find a block with a certain hashing power is "average time = difficulty * 2^32 / hashrate"
• When mining pools are concerned, pools accept only shares with difficulty >N, and 1 share with difficulty N is counted the same way as N shares with difficulty 1 (or N/2 shares with difficulty 2*N, etc.) - that's the pool difficulty
• This also means that with an N=2*coindiff in PPLNS, like most pools do, on average, the last "N" shares in the "PPLNS" cover 2 blocks

OK, so I tried to apply this info to Scrypt, because when researching things like Litecoin difficulty, the formulae I found were the same, e.g. "average time = difficulty * 2^32 / hashrate", and that on average you need 2^32 hashes to find a difficulty-one share, etc.

However it didn't make sense, everything was far off from what I saw on our mining pools.

In fact, it turned out that it was off by a factor suspiciously close to 2^16.

Further research and testing finally gave us the answer that for Scrypt, by convention, pools and miners scale the share difficulty value by a factor of 2^16.

But only the pools and the miner software (e.g. cgminer)! the network difficulty is still displayed without scaling!

Example:

Let's say the network difficulty is 900,000 for BCC (Bitconnect Coin)

In the original calculation, I would assume that it takes on average 21 hours at a hashrate of 50 GH/s to find a block (900k * 2^32 / 50G = ~77k seconds = ~21 hours), which is about right when looking at our pool.

However, I would then also assume that it takes a total value of shares of around 900,000 (exactly the network difficulty), however that's not true, because one AntMiner L3+ alone already submits a share with diff of ~150,000 every ~15 seconds, so I would assume that it finds a block every 90 seconds, which is of course not true (it's closer to 90 days than to 90 seconds).

And here is the issue: for Scrypt, share difficulty is measured with a value mulitplied by 65,536!

So, in fact, the "difficulty 150,000" which the pool uses with the AntMiners is actually 150,000/65,536 = 2.29

And, this means that in order to get to that coin difficulty value of 900,000, the pool needs to display a total value of shares of 900,000*65,536 = 58,982,400,000!

Then, everything suddenly makes sense again.

There are also other algorithms which use such a scaling factor. In fact, I found that the UNOMP pool software already has a file which also contains these values: https://github.com/UNOMP/node-merged-pool/blob/master/lib/algoProperties.js (look for multiplier). For SHA-256 it's 1, for Scrypt it's 2^16.

According to pieces of information scattered throughout the Internet, this was done because Scrypt is a more difficult algorithm per se, which resulted in pools accepting shares with difficulties smaller than 1, and to avoid issues with the fact that difficulties and share values could suddenly be non-integral, the whole number was scaled so that the unit "difficulty 1" now actually meant 2^(-16).

So, rule of thumb:

• Whenever you see a difficulty value in context of a miner or a pool for Scrypt, you need to divide it by 65,536 if you want to compare it with other numbers like the overall coin difficulty.
• For coin/block difficulty values (even in Scrypt), and in general for SHA-256 or X11, don't divide it. For other algorithms, look it up (for example, Blake uses 2^8).