llvm_mode whitelist (partial instrumentation) support added
MODIFY
docs/ChangeLog
docs/ChangeLog
@@ -17,6 +17,9 @@ sending a mail to <afl-users+subscribe@googlegroups.com>.
Version ++2.52d (tbd):
-----------------------------
+ - added whitelist support for llvm_mode via AFL_LLVM_WHITELIST to allow
+ only to instrument what is actually interesting. Gives more speed and less
+ map pollution (originally by choller@mozilla)
- added Python Module mutator support, python2.7-dev is autodetected.
see docs/python_mutators.txt (originally by choller@mozilla)
- added AFL_CAL_FAST for slow applications and AFL_DEBUG_CHILD_OUTPUT for debugging

MODIFY
docs/README
docs/README
@@ -21,8 +21,8 @@ american fuzzy lop plus plus
https://github.com/andreafioraldi/afl and got the community patches applied
to it.
- C. Hoellers Python mutator module support was added too
- (https://github.com/choeller/afl)
+ C. Hoellers afl-fuzz Python mutator module and llvm_mode whitelist support
+ was added too (https://github.com/choeller/afl)
So all in all this is the best-of AFL that is currently out there :-)

MODIFY
docs/env_variables.txt
docs/env_variables.txt
@@ -71,14 +71,8 @@ tools make fairly broad use of environmental variables:
- Setting AFL_CAL_FAST will speed up the initial calibration, if the
application is very slow
- - Setting AFL_DEBUG_CHILD_OUTPUT will not suppress the child output.
- Not pretty but good for debugging purposes.
-
- - For AFL_PYTHON_MODULE and AFL_PYTHON_ONLY - they require to be compiled
- with -DUSE_PYTHON. Please see docs/python_mutators.txt
-
-2) Settings for afl-clang-fast
-------------------------------
+2) Settings for afl-clang-fast / afl-clang-fast++
+-------------------------------------------------
The native LLVM instrumentation helper accepts a subset of the settings
discussed in section #1, with the exception of:
@@ -88,6 +82,30 @@ discussed in section #1, with the exception of:
- TMPDIR and AFL_KEEP_ASSEMBLY, since no temporary assembly files are
created.
+Then there are a few specific features that are only available in llvm_mode:
+
+ LAF-INTEL
+ =========
+ This great feature will split compares to series of single byte comparisons
+ to allow afl-fuzz to find otherwise rather impossible paths.
+
+ - Setting LAF_SPLIT_SWITCHES will split switch()es
+
+ - Setting LAF_TRANSFORM_COMPARES will split string compare functions
+
+ - Setting LAF_SPLIT_COMPARES will split > 8 bit CMP instructions
+
+ See llvm_mode/README.laf-intel for more information.
+
+ WHITELIST
+ =========
+ This feature allows selectively instrumentation of the source
+
+ - Setting AFL_LLVM_WHITELIST with a filename will only instrument those
+ files that match these names.
+
+ See llvm_mode/README.whitelist for more information.
+
Note that AFL_INST_RATIO will behave a bit differently than for afl-gcc,
because functions are *not* instrumented unconditionally - so low values
will have a more striking effect. For this tool, 0 is not a valid choice.
@@ -141,8 +159,8 @@ checks or alter some of the more exotic semantics of the tool:
- AFL_TMPDIR is used to write the .cur_input file to if exists, and in
the normal output directory otherwise. You would use this to point to
- a ramdisk/tmpfs. This increases the speed by a very minimal value but
- also reduces the stress on SSDs.
+ a ramdisk/tmpfs. This increases the speed by a small value but also
+ reduces the stress on SSDs.
- When developing custom instrumentation on top of afl-fuzz, you can use
AFL_SKIP_BIN_CHECK to inhibit the checks for non-instrumented binaries
@@ -159,6 +177,11 @@ checks or alter some of the more exotic semantics of the tool:
mutated files - say, to fix up checksums. See experimental/post_library/
for more.
+ - For AFL_PYTHON_MODULE and AFL_PYTHON_ONLY - they require to be compiled
+ with -DUSE_PYTHON. Please see docs/python_mutators.txt
+ This feature allows to configure custom mutators which can be very helpful
+ in e.g. fuzzing XML or other highly flexible structured input.
+
- AFL_FAST_CAL keeps the calibration stage about 2.5x faster (albeit less
precise), which can help when starting a session against a slow target.
@@ -183,6 +206,9 @@ checks or alter some of the more exotic semantics of the tool:
processing the first queue entry; and AFL_BENCH_UNTIL_CRASH causes it to
exit soon after the first crash is found.
+ - Setting AFL_DEBUG_CHILD_OUTPUT will not suppress the child output.
+ Not pretty but good for debugging purposes.
+
4) Settings for afl-qemu-trace
------------------------------

ADD
llvm_mode/README.whitelist
+========================================
+Using afl++ with partial instrumentation
+========================================
+
+ This file describes how you can selectively instrument only the source files
+ that are interesting to you using the LLVM instrumentation provided by
+ afl++
+
+ Originally developed by Christian Holler (:decoder) <choller@mozilla.com>.
+
+
+1) Description and purpose
+--------------------------
+
+When building and testing complex programs where only a part of the program is
+the fuzzing target, it often helps to only instrument the necessary parts of
+the program, leaving the rest uninstrumented. This helps to focus the fuzzer
+on the important parts of the program, avoiding undesired noise and
+disturbance by uninteresting code being exercised.
+
+For this purpose, I have added a "partial instrumentation" support to the LLVM
+mode of AFLFuzz that allows you to specify on a source file level which files
+should be compiled with or without instrumentation.
+
+
+2) Building the LLVM module
+---------------------------
+
+The new code is part of the existing afl++ LLVM module in the llvm_mode/
+subdirectory. There is nothing specifically to do :)
+
+
+3) How to use the partial instrumentation mode
+----------------------------------------------
+
+In order to build with partial instrumentation, you need to build with
+afl-clang-fast and afl-clang-fast++ respectively. The only required change is
+that you need to set the environment variable AFL_LLVM_WHITELIST when calling
+the compiler.
+
+The environment variable must point to a file containing all the filenames
+that should be instrumented. For matching, the filename that is being compiled
+must end in the filename contained in this whitelist (to avoid breaking the
+matching when absolute paths are used during compilation).
+
+For example if your source tree looks like this:
+
+project/
+project/feature_a/a1.cpp
+project/feature_a/a2.cpp
+project/feature_b/b1.cpp
+project/feature_b/b2.cpp
+
+And you only want to test feature_a, then create a whitelist file containing:
+
+feature_a/a1.cpp
+feature_a/a2.cpp
+
+However if the whitelist file contains this, it works as well:
+
+a1.cpp
+a2.cpp
+
+but it might lead to files being unwantedly instrumented if the same filename
+exists somewhere else in the project.
+
+The created whitelist file is then set to AFL_INST_WHITELIST when you compile
+your program. For each file that didn't match the whitelist, the compiler will
+issue a warning at the end stating that no blocks were instrumented. If you
+didn't intend to instrument that file, then you can safely ignore that warning.
+
+For old LLVM versions this feature might require to be compiled with debug
+information (-g), however at least from llvm version 6.0 onwards this is not
+required anymore (and might hurt performance and crash detection, so better not
+use -g)

MODIFY
llvm_mode/afl-llvm-pass.so.cc
llvm_mode/afl-llvm-pass.so.cc
#include <stdlib.h>
#include <unistd.h>
+#include <list>
+#include <string>
+#include <fstream>
+
+#include "llvm/IR/DebugInfo.h"
#include "llvm/IR/BasicBlock.h"
#include "llvm/IR/IRBuilder.h"
#include "llvm/IR/LegacyPassManager.h"
@@ -48,7 +53,21 @@ namespace {
public:
static char ID;
- AFLCoverage() : ModulePass(ID) { }
+ AFLCoverage() : ModulePass(ID) {
+ char* instWhiteListFilename = getenv("AFL_LLVM_WHITELIST");
+ if (instWhiteListFilename) {
+ std::string line;
+ std::ifstream fileStream;
+ fileStream.open(instWhiteListFilename);
+ if (!fileStream)
+ report_fatal_error("Unable to open AFL_LLVM_WHITELIST");
+ getline(fileStream, line);
+ while (fileStream) {
+ myWhitelist.push_back(line);
+ getline(fileStream, line);
+ }
+ }
+ }
bool runOnModule(Module &M) override;
@@ -56,6 +75,10 @@ namespace {
// return "American Fuzzy Lop Instrumentation";
// }
+ protected:
+
+ std::list<std::string> myWhitelist;
+
};
}
@@ -116,6 +139,51 @@ bool AFLCoverage::runOnModule(Module &M) {
BasicBlock::iterator IP = BB.getFirstInsertionPt();
IRBuilder<> IRB(&(*IP));
+ if (!myWhitelist.empty()) {
+ bool instrumentBlock = false;
+
+ /* Get the current location using debug information.
+ * For now, just instrument the block if we are not able
+ * to determine our location. */
+ DebugLoc Loc = IP->getDebugLoc();
+ if ( Loc ) {
+ DILocation *cDILoc = dyn_cast<DILocation>(Loc.getAsMDNode());
+
+ unsigned int instLine = cDILoc->getLine();
+ StringRef instFilename = cDILoc->getFilename();
+
+ if (instFilename.str().empty()) {
+ /* If the original location is empty, try using the inlined location */
+ DILocation *oDILoc = cDILoc->getInlinedAt();
+ if (oDILoc) {
+ instFilename = oDILoc->getFilename();
+ instLine = oDILoc->getLine();
+ }
+ }
+
+ /* Continue only if we know where we actually are */
+ if (!instFilename.str().empty()) {
+ for (std::list<std::string>::iterator it = myWhitelist.begin(); it != myWhitelist.end(); ++it) {
+ /* We don't check for filename equality here because
+ * filenames might actually be full paths. Instead we
+ * check that the actual filename ends in the filename
+ * specified in the list. */
+ if (instFilename.str().length() >= it->length()) {
+ if (instFilename.str().compare(instFilename.str().length() - it->length(), it->length(), *it) == 0) {
+ instrumentBlock = true;
+ break;
+ }
+ }
+ }
+ }
+ }
+
+ /* Either we couldn't figure out our location or the location is
+ * not whitelisted, so we skip instrumentation. */
+ if (!instrumentBlock) continue;
+ }
+
+
if (AFL_R(100) >= inst_ratio) continue;
/* Make up cur_loc */

llvm_mode whitelist (partial instrumentation) support added