title: mpi_waitall in mpich2 with null values in array_of_requests
body: <p>I get the following error with MPICH-2.1.5 and PGI compiler;</p>
  
  <pre><code>Fatal error in PMPI_Waitall: Invalid MPI_Request, error stack:
  PMPI_Waitall(311): MPI_Waitall(count=4, req_array=0x2ca0ae0, status_array=0x2c8d220) failed
  PMPI_Waitall(288): The supplied request in array element 0 was invalid (kind=0)
  </code></pre>
  
  <p>in the following example Fortran code for a stencil based algorithm,</p>
  
  <pre><code>Subroutine data_exchange
  
  ! data declaration
  
  integer request(2*neighbor),status(MPI_STATUS_SIZE,2*neighbor)
  integer n(neighbor),iflag(neighbor)
  integer itag(neighbor),neigh(neighbor)
  
  ! Data initialization
  request = 0; n = 0; iflag = 0;
  
  ! Create data buffers to send and recv
  ! Define values of n,iflag,itag,neigh based on boundary values
  
  ! Isend/Irecv look like this
  ir=0
  do i=1,neighbor
     if(iflag(i).eq.1) then
        ir=ir+1
        call MPI_Isend(buf_send(i),n(i),MPI_REAL,neigh(i),itag(i),MPI_COMM_WORLD,request(ir),ierr)
        ir=ir+1
        call MPI_Irecv(buf_recv(i),nsize,MPI_REAL,neigh(i),MPI_ANY_TAG,MPI_COMM_WORLD,request(ir),ierr)
     endif
  enddo
  
  ! Calculations
  
  call MPI_Waitall(2*neighbor,request,status,ierr)
  
  end subroutine
  </code></pre>
  
  <p>The error occurs when the <code>array_of_request</code> in mpi_waitall gets a null value (<code>request(i)=0</code>). The null value in <code>array_of_request</code> comes up when the conditional <code>iflag(i)=1</code> is not satisfied. The straight forward solution is to comment out the conditional but then that would introduce overheads of sending and receiving messages of 0 sizes which is not feasible for large scale systems (1000s of cores).</p>
  
  <p>As per the <a href="http://www.mpi-forum.org/docs/mpi-11-html/node49.html" rel="nofollow">MPI-forum link</a>, the <code>array_of_requests</code> list may contain null or inactive handles.</p>
  
  <p>I have tried following, </p>
  
  <ol>
  <li>not initializing <code>array_of_requests</code>, </li>
  <li>resizing <code>array_of_request</code> to match the <code>MPI_isend + MPI_irecv</code> count, </li>
  <li>assigning dummy values to <code>array_of_request</code></li>
  <li>I also tested the very same code with MPICH-1 as wells as OpenMPI 1.4 and the code works without any issue.</li>
  </ol>
  
  <p>Any insights would be really appreciated!</p>