Beating memcmp in C++
Since we're using the C++ wrappers around standard C functions (which is good - I like it!), we need namespace qualification of std::size_t and std::memcmp).  Although your implementation evidently takes advantage of the license to also include the unqualified names, it's not portable to rely on that.

Instead of repeating the type when casting a1 to s1 and a2 to s2, we can just use auto (and let's be clear about the cast - prefer reinterpret_cast to a catch-all C-style cast).

I managed to simplify memcmp_fixed_ by using a recursive template.  For me (with g++ -03), this gave roughly the same execution speed (I guess that loop unrolling makes the resultant binary very similar).  I got rid of the separate void* and unsigned char* overloads - that's just a compile-time overhead and makes no different to runtime speed, which gives 9 lines, compared to your 14 (counting physical lines that contain alphanumerics).

template&lt;std::size_t SIZE&gt;
int memcmp_fixed_(const void *a1, const void *a2)
{
    auto const s1 = reinterpret_cast&lt;const unsigned char*&gt;(a1);
    auto const s2 = reinterpret_cast&lt;const unsigned char*&gt;(a2);
    auto const diff = *s1 - *s2;
    return diff ? diff : memcmp_fixed_&lt;SIZE-1&gt;(s1+1, s2+1);
}

template&lt;&gt;
int memcmp_fixed_&lt;0&gt;(const void*, const void*)
{
    return 0;
}


I slightly simplified the test program to make it a single-character change to build the baseline test (and to send error messages where they belong), and massively reducing the repeated code:

#if 1
#define test_memcmp fast_memcmp
#else
#define test_memcmp std::memcmp
#endif

int main(int argc, char **argv)
{
    if (argc != 3) {
        std::fprintf(stderr, "Usage:\n\t%s [string1] [string2]\n", argv[0]);
        return 1;
    }

    const char *s1 = argv[1];
    const char *s2 = argv[2];
    auto const size  = std::min(strlen(s1), strlen(s2));

    volatile int x = 0;
    for (volatile std::size_t i = 0;  i &lt; MAX;  ++i)
        x += test_memcmp(s1, s2, size);

    std::printf("%d %d\n", test_memcmp(s1, s2, size), x);
}