Selecting function overload based on a type match with return value of passed callable
Higher level overview:
Personally I think the architecture has limited use cases. It was quite hard to implement, and even then doesn't match the specification.
From what I understood, this is the problem definition:

Given a container with element type T, and a callable which has return type U, create a new container with element type U and populate it by calling callable on each element of the first container. If T and U are the same, take the container by copy and transform it, otherwise use back inserter to populate it.

Although it is already good, in realm of template metaprogramming this is insufficient.
Implementation overview:
The solution places superficial limits (as per the problem definition). There are two problems in C++ template metaprogramming: assuming too much and assuming too little. Both of those happened in this case. Most of the time it is hard to tell if too little is assumed or too much, so they are mostly interchangeable.
The most obvious superficial assumptions are made here:
Container&lt;A, std::allocator&lt;A&gt;&gt;

Very few hand-rolled containers will have allocator type, or even any types at all. Remapping from one type to another is quite hard, and is the most significant problem in this case.
Somewhere the other stuff except the element type is left out:
if constexpr(std::is_same&lt;ContainerOfB, std::vector&lt;B&gt;&gt;::value) {
   result.reserve(xs.size());
}

A vector which has non-standard allocator will probably benefit from reserving too.
I don't think the code itself is unreadable. After some time on codereview and around a year of template metaprogramming this code is still readable. Although I would note that the comments in the center of the template parameter declaration got me really confused. I would put that above or below parameter declaration part.
Another solution:
I gave it a shot. I believe I covered more cases, at the expense of code size, although I think readability increased a bit:
namespace detail
{
    template &lt;typename Container&gt;
    using value_type = typename Container::value_type;

    template &lt;typename Callable, typename Container&gt;
    using invoke_result = std::result_of_t&lt;Callable(value_type&lt;Container&gt;)&gt;;
}

template &lt;typename Container, typename Callable&gt;
struct mutable_in_place :
        public std::bool_constant&lt;std::is_same_v&lt;detail::value_type&lt;Container&gt;,
                                  detail::invoke_result&lt;Callable, Container&gt;&gt;&gt;
{};

namespace detail
{
    template &lt;template &lt;typename ...&gt; typename Container, typename T, typename ... TArgs, typename Callable&gt;
    auto realfmap(Container&lt;T, TArgs...&gt; container, Callable&amp;&amp; callable, std::true_type)
    {
        std::transform(container.begin(), container.end(),
                       container.begin(), std::forward&lt;Callable&gt;(callable));
        return container;
    }

    template &lt;template &lt;typename ...&gt; typename Container, typename T, typename ... TArgs, typename Callable&gt;
    auto realfmap(Container&lt;T, TArgs...&gt;&amp;&amp; container, Callable&amp;&amp; callable, std::false_type)
    {
        using parameter_type = decltype(*container.begin());
        using invoke_result = std::result_of_t&lt;Callable(parameter_type)&gt;;
        Container&lt;invoke_result&gt; mapped_container;

        if constexpr(std::is_same_v&lt;decltype(mapped_container), std::vector&lt;invoke_result&gt;&gt;)
        {
            mapped_container.reserve(container.size());
        }

        std::transform(container.begin(), container.end(), std::back_inserter(mapped_container),
                       std::forward&lt;Callable&gt;(callable));
        return mapped_container;
    }
}

template &lt;typename Container, typename Callable&gt;
auto realfmap(Callable&amp;&amp; callable, Container&amp;&amp; container)
{
    return detail::realfmap(std::forward&lt;Container&gt;(container),
                            std::forward&lt;Callable&gt;(callable),
                            mutable_in_place&lt;std::decay_t&lt;Container&gt;, Callable&gt;{});
}

Although it grabs the other template arguments, it cannot use them. For example, consider the case when container=std::vector&lt;int&gt; and callable=std::string(int). The allocator which should get instantiated for the returning vector is std::allocator&lt;std::string&gt;, but the grabbed one is std::allocator&lt;int&gt;, so static assert will fire or some other library error notification facility, since std::vector&lt;T&gt;::value_type and std::vector&lt;T&gt;::allocator::value_type must match.
The reasoning behind not putting mutable_in_place is to expose this to the user. Some people could do template metaprogramming on top of this, and sometimes people might run into template bugs, and exposing the variable template should aid them.
About using std::result_of_t&lt;&gt;, the reason is that it is much easier to convert the type trait compared to hand rolled version with decltype() and stuff. It might even be possible to write a python script or something similar to convert throughout the code base, since the syntax is trivial.
Don't get me wrong, but plain transform has clear advantage here, both semantics and implementation wise.
Change the problem:
It seems like solving the current problem will have a very big engineering cost, thus I propose the following problem definition:

An StdContainer: type which exactly matches declaration template &lt;typename ElemType, typename Allocator = std::allocator&lt;ElemType&gt;&gt; class container except identifier names (type parameters and class name itself) and is allocator-aware. Given an StdContainer and Callable, create and populate another StdContainer with return type of the Callable as element type. Creating and populating depends on the equality of element type of the original container and return type of the callable. If they match, a copy is taken and std::transform() is called. Otherwise, the original container is taken by reference, new one is default constructed and it is populated though calling std::transform() with std::back_inserter() as output iterator.

Your solution will solve the new problem and won't have any bugs. Also, for me it would be sufficient to pass into code base, with some cosmetic tweaking.