"fixed" symbol multiply defined problems with LTO
MODIFY
llvm_mode/README.lto.md
llvm_mode/README.lto.md
@@ -91,6 +91,15 @@ AR=llvm-ar RANLIB=llvm-ranlib CC=afl-clang-lto CXX=afl-clang-lto++ ./configure -
```
and on some target you have to to AR=/RANLIB= even for make as the configure script does not save it ...
+### "linking globals named '...': symbol multiply defined" error
+
+The target program is using multiple global variables or functions with the
+same name. This is a common error when compiling a project with LTO, and
+the fix is `-Wl,--allow-multiple-definition` - however llvm-link which we
+need to link all llvm IR LTO files does not support this - yet (hopefully).
+Hence if you see this error either you have to remove the duplicate global
+variable (think `#ifdef` ...) or you are out of luck. :-(
+
### clang is hardcoded to /bin/ld
Some clang packages have 'ld' hardcoded to /bin/ld. This is an issue as this
@@ -148,3 +157,30 @@ Known issues:
* unrar-nonfree-5.6.6
* exiv 0.27
* jpeg-6b
+
+## History
+
+This was originally envisioned by hexcoder- in Summer 2019, however we saw no
+way to create a pass that is run at link time - although there is a option
+for this in the PassManager: EP_FullLinkTimeOptimizationLast
+("Fun" info - nobody knows what this is doing. And the developer who
+implemented this didn't respond to emails.)
+
+In December came then the idea to implement this as a pass that is run via
+the llvm "opt" program, which is performed via an own linker that afterwards
+calls the real linker.
+This was first implemented in January and work ... kinda.
+The LTO time instrumentation worked, however the "how" the basic blocks were
+instrumented was a problem, as reducing duplicates turned out to be very,
+very difficult with a program that has so many paths and therefore so many
+dependencies. At lot of stratgies were implemented - and failed.
+And then sat solvers were tried, but with over 10.000 variables that turned
+out to be a dead-end too.
+The final idea to solve this came from domenukk who proposed to insert a block
+into an edge and then just use incremental counters ... and this worked!
+After some trials and errors to implement this vanhauser-thc found out that
+there is actually an llvm function for this: SplitEdge() :-)
+Still more problems came up though as this only works without bugs from
+llvm 9 onwards, and with high optimization the link optimization ruins
+the instrumented control flow graph.
+As long as there are no larger changes in llvm this all should work well now ...

MODIFY
llvm_mode/afl-ld.c
llvm_mode/afl-ld.c
@@ -215,9 +215,12 @@ static void edit_params(int argc, char** argv) {
link_params[link_param_cnt++] = linked_file;
opt_params[0] = alloc_printf("%s/%s", LLVM_BINDIR, "opt");
- if (getenv("AFL_DONT_OPTIMIZE") == NULL)
+ if (getenv("AFL_DONT_OPTIMIZE") == NULL) {
+
opt_params[opt_param_cnt++] = "-O3";
- else
+ opt_params[opt_param_cnt++] = "--polly";
+
+ } else
opt_params[opt_param_cnt++] = "-O0";
// opt_params[opt_param_cnt++] = "-S"; // only when debugging
opt_params[opt_param_cnt++] = linked_file; // input: .ll file
@@ -599,7 +602,16 @@ int main(int argc, char** argv) {
if (pid < 0) PFATAL("fork() failed");
if (waitpid(pid, &status, 0) <= 0) PFATAL("waitpid() failed");
- if (WEXITSTATUS(status) != 0) exit(WEXITSTATUS(status));
+ if (WEXITSTATUS(status) != 0) {
+
+ SAYF(bSTOP RESET_G1 CURSOR_SHOW cRST cLRD \
+ "\n[-] PROGRAM ABORT : " cRST);
+ SAYF( "llvm-link failed, if this is because of a \"linking globals\n"
+ " named '...': symbol multiply defined\" error then there is nothing we can do -\n"
+ "llvm-link is missing an important feature :-(\n\n");
+ exit(WEXITSTATUS(status));
+
+ }
/* then we perform an optimization on the collected objects files */
if (!be_quiet)

"fixed" symbol multiply defined problems with LTO