Threaded generalised transform I'm writing an application which works with huge amounts of sequential data, and often found the need to use std::transform. I see two potential improvements to std::transform:


Allow for variable number parameters.
Take advantage of the linear separability of the data by multithreading.


Can anyone suggest any design/performance improvements on my implementation?

threaded_transform.h

#include &lt;vector&gt;
#include &lt;thread&gt;

template&lt;typename InputIterator, typename OutputIterator,
         typename Function, typename... Params&gt;
OutputIterator
trans(InputIterator first, InputIterator last, OutputIterator result,
          Function f, Params... params)
{
    for (; first != last; ++first, ++result)
        *result = f(*first, params...);
    return result;
}

template&lt;typename InputIterator, typename OutputIterator,
         typename Function, typename... Params&gt;
OutputIterator
threaded_transform(unsigned num_threads, InputIterator first,
                   InputIterator last, OutputIterator result,
                   Function f, Params... params)
{
    std::size_t num_values = last - first;
    std::size_t num_values_per_threads = num_values / num_threads;

    std::vector&lt;std::thread&gt; threads;
    threads.reserve(num_threads);

    for (unsigned i = 1; i &lt;= num_threads; ++i) {
        if (i == num_threads) {
            // The last thread processes the remaining values.
            threads.push_back(std::thread(trans&lt;InputIterator, OutputIterator, Function, Params...&gt;,
                                          first, last, result, f, params...));
        } else {
            threads.push_back(std::thread(trans&lt;InputIterator, OutputIterator, Function, Params...&gt;,
                                          first, first + num_values_per_threads, result, f, params...));
        }
        first  += num_values_per_threads;
        result += num_values_per_threads;
    }

    for (auto&amp; thread : threads)
        thread.join();

    return result;
}


main.cpp

#include &lt;vector&gt;
#include "threaded_transform.h"

int main()
{
    auto sum = [] (int a, int b) { return a + b; };

    std::vector&lt;int&gt; values = {1,2,3,4,5,6,7,8,9,10};
    std::vector&lt;int&gt; results;
    results.resize(10);

    threaded_transform(4, values.cbegin(), values.cend(), results.begin(), sum, 10);

    for (auto result : results) {
        std::cout &lt;&lt; result &lt;&lt; std::endl;
    }
}
You probably want to forward your parameters:

   *result = f(*first, params...);


Try:

   *result = f(*first, std::forward&lt;Params&gt;(params)...);


To go along with forwardign you probably want two versions of trans() on that takes values by reference/value one that takes r-value references:

// Normal parameters.
trans(InputIterator first, InputIterator last, OutputIterator result,
      Function f, Params const&amp;... params)

// R-Value parameters.
trans(InputIterator first, InputIterator last, OutputIterator result,
      Function f, Params&amp;&amp;... params)


Going to main transform function.

I am not sure you in realty want to pass a function and arguments. That's the whole point of the lambda. So you can wrap the function call and its parameters into a function.

threaded_transform(4, values.cbegin(), values.cend(), results.begin(), sum, 10);

// Or would you prefer:

threaded_transform(4, values.cbegin(), values.cend(), results.begin(),
    [](int other){ return add(other, 10);}
);

//Or even
threaded_transform(4, values.cbegin(), values.cend(), results.begin(),
    [](int other){ return 10 + other;}
);


If you do this you should write details about your iterator requirements.

The requirements for std::transform()

template&lt;class InputIterator1, class InputIterator2, class OutputIterator, class BinaryFunction&gt;
OutputIterator transform(InputIterator1 first1, InputIterator1 last1,
                         InputIterator2 first2, OutputIterator result,
                         BinaryFunction binary_op);


Where:
   InputIterator must be a model of Input Iterator.
   OutputIterator must be a model of Output Iterator. 


In your threaded implementation you have a more stringent requirement for the output iterator. I believe it needs to be random access iterator.

    OutputIterator must be a model of Random Access Iterator. 


See: http://www.sgi.com/tech/stl/RandomAccessIterator.html