Median of 3 Quicksort in C
Your swap_mem will get called O(n log n) times. This makes it worth taking a closer look at for optimization.

One thing you can do is avoid repeated calls to malloc and free and instead preallocate the tmp buffer in the root function call and pass it around or use alloca to allocate the buffer on the stack.

void swap_mem(void * ptr1, void * ptr2, size_t size){
    /*
     * Swaps the values at the addresses pointed at by ptr1 and ptr2
     */
    void * tmp = alloca(size); //will get freed automatically on function exit.
    memcpy(tmp, ptr1, size);
    memcpy(ptr1, ptr2, size);
    memcpy(ptr2, tmp, size);
}