title: Understanding Block and Block-Cyclic Matrix Distributions
body: <p>In working with parallel decompositions of matrices, I'm familiar with a block distribution, where we have (say) 4 processes, each with their own subregion of the matrix:</p>
  
  <p><img src="https://i.stack.imgur.com/N2utM.png" alt="Block Matrix Decomposition"></p>
  
  <p>So for instance here we have the number of processes in a row (<code>procrows</code>) equal to 2, the number of processes in a column (<code>proccols</code>) also equal to two, and the sub-matrices <code>A_local</code> would have size <code>N/2 x M/2</code> if the original matrix size is <code>N x M</code>.</p>
  
  <p>I am reading this <a href="https://andyspiros.wordpress.com/2011/07/08/an-example-of-blacs-with-c/" rel="noreferrer">example</a> which uses a "block-cyclic" distribution, and in this part:</p>
  
  <pre><code>/* Begin Cblas context */
  /* We assume that we have 4 processes and place them in a 2-by-2 grid */
  int ctxt, myid, myrow, mycol, numproc;
  int procrows = 2, proccols = 2;
  Cblacs_pinfo(&amp;myid, &amp;numproc);
  Cblacs_get(0, 0, &amp;ctxt);
  Cblacs_gridinit(&amp;ctxt, "Row-major", procrows, proccols);
  </code></pre>
  
  <p>they have <code>procrows</code> and <code>proccols</code> are hardcoded, fine, but for a matrix that's read in, there's a header:</p>
  
  <blockquote>
    <p>Nb and Mb will be the number of rows and columns of the blocks [of the matrix]</p>
  </blockquote>
  
  <p>I don't understand this; aren't <code>Nb</code> and <code>Mb</code> completely determined by N, M, procrows, and proccols?</p>
  
  <hr>
  
  <p><em>EDIT</em></p>
  
  <p>From running the example I can see that the submatrix on process 0 has all the elements of the left up corner of the matrix, just like in my picture above, something that comes in contradiction with Jonathan's answer. However, it works fine with ScaLAPACK's Cholesky.</p>