Implementation of FastICA in multithreading approach I am to implement a parallel version of FastICA, so I implemented the serial version of FastICA. 

      

Now this is the architectural diagram of how it is going to parallelize. I just coded a basic thread model of this. But I need to know what I did was correct or wrong regarding inter threads synchronization.

#include &lt;condition_variable&gt;
#include &lt;mutex&gt;
#include &lt;vector&gt;
#include &lt;iostream&gt;
#include &lt;thread&gt;


int numberOfThreads=10;
std::vector&lt;int&gt; checkedEvent(numberOfThreads);
std::vector&lt;double&gt; data(numberOfThreads);
std::mutex lock;
int IsPrepare=0;

std::condition_variable data_prepare;

class MainProcess
{
private:
    int rank;
    double input;
public:
    MainProcess(int rank_, double dataPr){
        rank=rank_;
        input=dataPr;
    }
    MainProcess(){

    }
    void fastICA_processing()
    {
        while(true){
            std::unique_lock&lt;std::mutex&gt; lk(lock);
            data_prepare.wait(lk,[this]{return checkedEvent[rank]!=1;});
            data[rank]=input;
            checkedEvent[rank]=1;
            IsPrepare=IsPrepare+1;
            std::cerr &lt;&lt;IsPrepare&lt;&lt;" thread is processing"&lt;&lt;std::endl;
            lk.unlock();
        }

    }

    void controller()
    {
        while(true)
        {

            while(IsPrepare==numberOfThreads)
            {
                std::lock_guard&lt;std::mutex&gt; lk(lock);
                std::cerr&lt;&lt;"Now controller is checking fastICA is converged or not "&lt;&lt;std::endl;
                std::fill(checkedEvent.begin(), checkedEvent.end(), 0);
                IsPrepare=0;
                data_prepare.notify_all();
            }
        }

    }
    };

int main()
{
    std::vector&lt;std::thread&gt; thredArray;
    MainProcess p3(0,0);
    std::thread process_thread(&amp;MainProcess::controller,&amp;p3);
    for(int i=0;i&lt;numberOfThreads;i++){
        MainProcess* p1= new MainProcess(i,i);
        thredArray.push_back(std::thread (&amp;MainProcess::fastICA_processing,p1));
    }
    for(auto &amp;t:thredArray)
    {
        t.join();
    }
    process_thread.join();

    return 0;
}


Note: Here "data" stands for input data. First fastICA_processing threads do some mathematical calculations and then the controller thread gets all that data and processes it. This will happen again and again. I didn't add threads detaching part here.
The code is a mess, but I can tell you that it will spin forever right here:

std::thread process_thread(&amp;MainProcess::controller,&amp;p3);


This starts controller running immediately.

void controller()
{
    while(true)
    {
        while(IsPrepare==numberOfThreads)
        {
            std::lock_guard&lt;std::mutex&gt; lk(lock);
            std::cerr&lt;&lt;"Now controller is checking fastICA is converged or not "&lt;&lt;std::endl;
            std::fill(checkedEvent.begin(), checkedEvent.end(), 0);
            IsPrepare=0;
            data_prepare.notify_all();
        }
    }
}


IsPrepare is 0, and numberOfThreads is 10, and there's no synchronization to worry about here (no locks, no atomics), so we can definitely say that 0 == 10 is invariably false. Thus controller reduces to

void controller()
{
    while(true)
    {
        while(false) {}
    }
}


Okay, so that thread won't be doing anything anytime soon.
And with that code out of the way, the rest is pretty much irrelevant; it looks to me as if it will end up blocking forever, since there's nobody left to notify the condition variable.

Your problem might be suited to a TBB flowgraph; consider using TBB instead of hand-rolling your own stuff.