title: MPI allreduce with barrier
body: <p>There are few information about the internal structure of MPI allreduce operation and its relation to MPI barrier, and this few information is not consistent. So, I am trying to clarify the following two questions:</p>
  
  <ol>
  <li>Should I use barrier with allreduce (my MPI version is 3.1.3)?</li>
  <li>Are synchronization and blocking same things in the context of parallel programming, if not how they relate to allreduce and barrier operations?</li>
  </ol>