TLDR: You can find some OLD 2019 DATA comparing their accuracy for one-hour estimations here: https://bitpost.co/evaluations
I asked myself this question about two years ago and wasn't able to find an answer. I then set out to build an app that measured how accurate fee estimations from various sources were. Below are the design choices and some OUTDATED results.
The first challenge I found was that fee estimations come under many different formats, some recommend in time horizon, others in a block horizon. I made the simplification of converting all time horizon estimations to block horizon estimations. This is questionable because some estimators might be more accurate because they take into account that certain epochs are faster than others (hash rate changes). However, this resolves ambiguous situations where a block comes around the estimation horizon: suppose you make a fee estimation for 20min from now, and the second block arrives at 20min 1 seconds, do you include that block or not?
Another important consideration is "What does it mean to estimate a fee of, say, 8 sat/B for a one-hour horizon?". Does it mean that in 80% of the cases that will be enough? Or 50% of the cases? Bitcoin core estimator has for example two modes of estimation "ECONOMICAL" and "CONSERVATIVE". So, the first time in measuring the accuracy of a fee estimator should be to assign a statistical meaning to the fee estimation. I call this, finding the implicit probability of a fee estimator: the probability for which, the fee estimator achieves its highest accuracy.
Below I paste from https://bitpost.co/evaluations/help an outline of the procedure I used:
First, the application 'observes' a type of recommendation and assigns a probabilistic meaning to each recommendation type. For example, bitcoin core's 'economical' fee recommendations could mean 40% probability of being a transaction with the recommended feerate (sat/byte) be mined while bitcoin core's 'conservative' fee recommendations could mean 70% probability.
The application gathers a recommendation and hundreds of transactions that arrive at the bitcoin mempool during this period (new pending transactions).
According to the target of the recommendation (2,3,4,etc. blocks), it waits until that target is reached.
The application, by analyzing what transactions were mined and which ones weren't, derives a range of correct recommendations. If the recommendation given is out of this range, the error of the recommendation is given by subtracting from it the correct recommendation.
In https://bitpost.co/evaluations I show some of the results that I got in 2019. One evaluation means corresponds to one observation that is considered valid for one-minute. The measured fee estimators were:
