title: Running openmp on cluster
body: <p>I have to run an openmp program on a cluster with different configuration (such as different number of nodes).
  But the problem I am facing is that whenever I am trying to run the program with say 2 nodes then the same piece of program runs 2 times instead of running in parallel.</p>
  
  <p>My program - </p>
  
  <pre><code>gettimeofday(&amp;t0, NULL);
  for (k=0; k&lt;size; k++) {
      #pragma omp parallel for shared(A)
      for (i=k+1; i&lt;size; i++) {
          //parallel code
      }
      #pragma omp barrier
      for (i=k+1; i&lt;size; i++) {
          #pragma omp parallel for
          //parallel code
      }
  }
  
  gettimeofday(&amp;t1, NULL);
  printf("Did %u calls in %.2g seconds\n", i, t1.tv_sec - t0.tv_sec + 1E-6 * (t1.tv_usec - t0.tv_usec));
  </code></pre>
  
  <p>It is an LU decomposition program.
  When I am running it on 2 node then I am getting output something like this -<br> 
  Did 1000 calls in 5.2 seconds<br>
  Did 1000 calls in 5.3 seconds<br>
  Did 2000 calls in 41 seconds<br>
  Did 2000 calls in 41 seconds<br></p>
  
  <p>As you see each the program is run two times for each value (1000,2000,3000...) instead of running in parallel.
  It is my homework program but I am stuck at this point.</p>
  
  <p>I am using SLURM script to run this program on my college computing cluster. This is the standard script provided by the professor.</p>
  
  <pre><code>#!/bin/sh
  ##SBATCH --partition=general-compute
  #SBATCH --time=60:00:00
  #SBATCH --nodes=1
  #SBATCH --ntasks-per-node=2
  ##SBATCH --mem=24000
  # Memory per node specification is in MB. It is optional. 
  # The default limit is 3GB per core.
  #SBATCH --job-name="lu_openmpnew2nodes"
  #SBATCH --output=luopenmpnew1node2task.out
  #SBATCH --mail-user=***@***.edu
  #SBATCH --mail-type=ALL
  ##SBATCH --requeue
  #Specifies that the job will be requeued after a node failure.
  #The default is that the job will not be requeued.
  
  
  echo "SLURM_JOBID="$SLURM_JOBID
  echo "SLURM_JOB_NODELIST"=$SLURM_JOB_NODELIST
  echo "SLURM_NNODES"=$SLURM_NNODES
  echo "SLURMTMPDIR="$SLURMTMPDIR
  
  cd $SLURM_SUBMIT_DIR
  echo "working directory = "$SLURM_SUBMIT_DIR
  
  module list
  ulimit -s unlimited
  #
  
  echo "Launch luopenmp with srun"
  export I_MPI_PMI_LIBRARY=/usr/lib64/libpmi.so
  for i in {1000..20000..1000}
  do
  srun ./openmpNew "$i"
  done
  
  #
  echo "All Done!"
  </code></pre>