Which fuzzer should I use to fuzz test Bitcoin Core?

Question

In the Bitcoin Core fuzzing docs there are setup instructions for three different fuzzers: libFuzzer, american fuzzy lop (afl-fuzz) and Honggfuzz. Presumably I should try them in this order. How do they compare? Should I use one particular fuzzer in one scenario and a different fuzzer in another scenario? When I engage in fuzz testing should I run all of them one after each other?

Answer 1

This is a work in progress answer with some content currently taken from the Google fuzzing docs.

libFuzzer

libFuzzer is a tool which allows you to write fuzz tests in a similar style to unit tests. For developers setting up fuzzing for the first time, the Google fuzzing team recommends it for its simplicity.

[It] might be a good choice for testing libraries that have relatively small inputs, each input takes < 10ms to run, and the library code is not expected to crash on invalid inputs. Examples: regular expression matchers, text or binary format parsers, compression, network, crypto.

See http://llvm.org/docs/LibFuzzer.html#q-so-what-exactly-this-fuzzer-is-good-for for more information on how libFuzzer compares to other fuzzers.

AFL

AFL is the tool that did the most to drive the early success of coverage-guided fuzzing. The Google fuzzing team does not recommend AFL for new projects since it is not as well-maintained as Honggfuzz or libFuzzer. That said, because of its widespread it is common to encounter existing projects which rely on it for fuzzing.

Compared to other instrumented fuzzers, afl-fuzz is designed to be practical: it has modest performance overhead, uses a variety of highly effective fuzzing strategies and effort minimization tricks, requires essentially no configuration, and seamlessly handles complex, real-world use cases - say, common image parsing or file compression libraries.

See https://lcamtuf.coredump.cx/afl/ for more information on how afl-fuzz compares to other fuzzers.

Honggfuzz

Honggfuzz is another fuzzing tool with a rich set of features available to it. The Google fuzzing team recommends it for security researchers looking for more customization options in their fuzzers, or developers on complex projects where working within libFuzzer's unit-test-like style is difficult. It allows users to specify how input files should be fed to a target program (e.g. on the command line, on stdin) or allows more complicated customizations such as feeding input to a network server directly.