I think this is a fair summary. I would hesitate to phase out unit tests, rather supplement them with property based tests. In a library I maintain called bitcoin-s we use them in tandem.
Concepts
There are two primary concepts in property based testing systems.
The first one is a Generator. The point of a generator is to generate values you want to run through your tests. Think of a generator as specify the boundaries of what random values you want to be generated for your test.
The second concept is the test itself. In english, property based tests say that
"Given a generator X, i want to generate random values from X and run them through my test case. My assertion should always hold true for the value generated from my generator".
An example
I'm going to pull an example from this PR i've had open for a long time now. If you want to see some more extensive tests I recommend taking a lookg at it.
The "property" we are going to assert is that
"If we generate two random private keys in bitcoin, they should NOT be the same"
If this property fails, we have an indicatin that something is probably wrong with our pseudo random number generator.
Here is what that test case looks like in C++ with bitcoin core:
/** Check CKey uniqueness */
RC_BOOST_PROP(key_uniqueness, (const CKey& key1, const CKey& key2))
{
RC_ASSERT(!(key1 == key2));
}
That test case can be run N number of times (by default 10 in the rapidcheck library.
The next logical question would be "where do key1 and key2 come from"? The answer: They come from the generator! The beauty of property based testing is leverage the framework to do the generation of values for you.
Here is what the generator for CKey looks like:
/** Generator for a new CKey */
template <>
struct Arbitrary<CKey> {
static Gen<CKey> arbitrary()
{
return rc::gen::map<int>([](int x) {
CKey key;
key.MakeNewKey(true);
return key;
});
};
};
This gives rapidcheck the ability to generate arbitrary valid CKeys to pass in as a parameter to the properties that we are asserting holds true.
You can see more complex generators and assertions related to Script in that PR too if you want to see non trivial generators/assertions.
Summary
It's really useful to have multiple runs of a test case with newly generated values every time. Typically in software testing, you write a unit test asserting that your code works as expected for one or two values. This is a really limited approach to testing as you want your code to work for a range of values. With generators, you can get the computer to generate a range of valid values that your property should assert true on.
I think this is particularly useful for consensus systems like bitcoin where we need to reason about a wide range of values being fed as input into the consensus system. Property based testing is a great tool to use to offload some of that cognitive load when reasoning about corner cases.
