0

In the Bitcoin Core fuzzing docs there are setup instructions for three different fuzzers: libFuzzer, american fuzzy lop (afl-fuzz) and Honggfuzz. Presumably I should try them in this order. How do they compare? Should I use one particular fuzzer in one scenario and a different fuzzer in another scenario? When I engage in fuzz testing should I run all of them one after each other?

0

This is a work in progress answer with some content currently taken from the Google fuzzing docs.

libFuzzer

libFuzzer is a tool which allows you to write fuzz tests in a similar style to unit tests. For developers setting up fuzzing for the first time, the Google fuzzing team recommends it for its simplicity.

[It] might be a good choice for testing libraries that have relatively small inputs, each input takes < 10ms to run, and the library code is not expected to crash on invalid inputs. Examples: regular expression matchers, text or binary format parsers, compression, network, crypto.

See http://llvm.org/docs/LibFuzzer.html#q-so-what-exactly-this-fuzzer-is-good-for for more information on how libFuzzer compares to other fuzzers.

AFL

AFL is the tool that did the most to drive the early success of coverage-guided fuzzing. The Google fuzzing team does not recommend AFL for new projects since it is not as well-maintained as Honggfuzz or libFuzzer. That said, because of its widespread it is common to encounter existing projects which rely on it for fuzzing.

Compared to other instrumented fuzzers, afl-fuzz is designed to be practical: it has modest performance overhead, uses a variety of highly effective fuzzing strategies and effort minimization tricks, requires essentially no configuration, and seamlessly handles complex, real-world use cases - say, common image parsing or file compression libraries.

See https://lcamtuf.coredump.cx/afl/ for more information on how afl-fuzz compares to other fuzzers.

Honggfuzz

Honggfuzz is another fuzzing tool with a rich set of features available to it. The Google fuzzing team recommends it for security researchers looking for more customization options in their fuzzers, or developers on complex projects where working within libFuzzer's unit-test-like style is difficult. It allows users to specify how input files should be fed to a target program (e.g. on the command line, on stdin) or allows more complicated customizations such as feeding input to a network server directly.

6
  • alf is very much not applicable to bitcoin core. – Anonymous Nov 11 '20 at 14:29
  • Why do you say that? It is in the Bitcoin Core fuzzing docs – Michael Folkson Nov 11 '20 at 15:24
  • It’s not particularly useful for evaluating the sort of things in Bitcoin that actually matter. The consensus rules tests are partially derived from fuzzing but it isn’t ever going to find interesting bugs due to the amount of state bitcoin core has. That’s just opinion of course. – Anonymous Nov 11 '20 at 15:35
  • Interesting, thanks! – Michael Folkson Nov 11 '20 at 15:56
  • 2
    @Anonymous "It isn't going to find interesting bugs": that's untrue. AFL was used to find consensus bugs in other implementation of Bitcoin: github.com/jonasnick/bitcoinconsensus_testcases . Of course it's not a "bitcoin-core bug" but still pretty interesting bug-prone edgecases. – darosior Dec 11 '20 at 14:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.