This happens with all pools. Some pools measure over a long time and get a fairly accurate hashrate, but it is very slow to react to changes (stop mining, add hashpower, etc.). Other pools measure over a short time and get a more inaccurate hashrate, but one that reflects changes faster.
The estimated hashrate isn't affected much by rejected work, unless you have an abnormal amount of those. If you have a lot of hardware errors (miscalculations) from your devices, those can also affect it some. These two issues can give a persistently lower hashrate than you are actually hashing with. But the source of the swings up and down is primarily the randomness with finding proofs of work. Some nonce ranges you scan will contain 3 and some won't have any at all.
Mining with a higher worker difficulty will increase the variance with finding proofs of work. If you set a worker difficulty that is very high for your hashrate then the website's estimate of your hashrate will likely be all over the place.
For a more accurate estimation of your hashrate look at your average hashrate per shift or other longer term, rather than a live hashrate estimate.
So there is really no specific answer to how accurate an estimated hashrate should be. Different pools use different methods, and some even have different hashrates in the same pool (measured over short and long periods of time).