title: How does MPI allreduce work
body: <p>I don't have much experience with MPI and I try to understand how allreduce work. Below is a simple example inspired by this <a href="http://ipython.org/ipython-doc/dev/parallel/parallel_mpi.html" rel="nofollow">IPython tutorial</a>. 2 MPI engines are launched on a local computer from the IPython notebook dasboard here:</p>
  
  <pre><code>In [1]: import numpy as np
          from IPython.parallel import Client
  
  In [2]: c = Client(profile='mpi')
  
  In [3]: view = c[:]
  
  In [4]: view.scatter('a', np.arange(4.))
  Out[4]: &lt;AsyncResult: scatter&gt;
  
  In [5]: %%px
          from mpi4py import MPI
          import numpy as np
  
          print MPI.COMM_WORLD.allreduce(np.sum(a), op=MPI.SUM)
  [stdout:0] 1.0
  [stdout:1] 5.0
  </code></pre>
  
  <p>I would have expected each engine to print "6.0", like in the IPython tutorial. Here, it is as if the reduction operation was not performed. It's probably very simple, but I don't quite see what I am doing wrong?</p>
  
  <p>I use:</p>
  
  <ul>
  <li>Ubuntu 12.04</li>
  <li>Python 2.7.3 32-bit</li>
  <li>IPython 1.1.0</li>
  <li>mpi4py 1.2.2</li>
  <li>mpich2</li>
  </ul>