I am trying to scan all possible techniques to disrupt the usage of a debugger on a Unix platform (ie POSIX and a bit more).I am thinking about techniques such as the PTRACE test at the beginning of a program (or at various points of its execution), the insertion of fake breakpoints (eg int3/0xcc x86 opcode) or time checks. But, also global strategies defined on the program to slow down the analysis of the program.For now, all the techniques I found on Internet were easily worked around as soon as the anti-debug technique has been understood. So, I wonder if there are stronger ones.
Anti-debug techniques on Unix platforms?
assembly;obfuscation;anti debugging
Here's some I've seen or heard about:Stripping the section headers. A simple and fully legal action that stops GDB dead in its tracks. Does not work against some other debuggers (e.g. IDA). Can be done using the sstrip tool.Using syscall function or direct syscalls instructions instead of calling specific functions like ptrace(). Can be defeated by setting the breakpoint on syscall function or just stepping through the file, but may be non-obvious if you don't know about it.Performing anti-debugging actions before main(), e.g. in constructors of global objects or using __attribute__((constructor)). Since GDB usually sets initial breakpoint in main(), it takes care of the default situation. Workaround is simple: put breakpoint on the actual file's entrypoint (info file in GDB).Sending itself debugging-related signals like SIGTRAP. (Note that this can be ignored with handle SIGTRAP nostop in GDB.)Forking and tracing itself with ptrace.Fake breakpoints insertion: Inserting int3/0xcc will force the debugger to stop on these bytes as they will be treated as software breakpoints. If they are numerous it might slow down considerably the analysis.Breakpoint detection: I saw this technique in this paper, you can attach a function that will be triggered when a breakpoint is encoutered. This paper covers also some other tricks.