title: Why can't I run multiple instances of the same python script simulataniously in SLURM
body: <p>I have been struggling trying to get multiple instances of a python script to run on SLURM.  In my login node I have installed python3.6 and I have a python script "my_script.py" which takes a text file as input to read in run parameters.  I can run this script on the login node using </p>
  
  <pre><code>python3.6 my_script.py input1.txt
  </code></pre>
  
  <p>Furthermore, I can submit a script submit.sh to run the job:</p>
  
  <pre><code>#!/bin/bash
  #
  #SBATCH --job-name=hostname_sleep_sample
  #SBATCH --output=output1.txt
  #SBATCH --cpus-per-task=1
  #
  #SBATCH --mem=2G
  
  python3.6 my_script.py input1.txt
  </code></pre>
  
  <p>This runs fine and executes as expected.  However, if I submit the following script:</p>
  
  <pre><code>#!/bin/bash
  #
  #SBATCH --job-name=hostname_sleep_sample
  #SBATCH --output=output2.txt
  #SBATCH --cpus-per-task=1
  #
  #SBATCH --mem=2G
  
  python3.6 my_script.py input2.txt
  </code></pre>
  
  <p>while the first is running I get the following error message in output2.txt:</p>
  
  <pre><code>/var/spool/slurmd/job00130/slurm_script: line 9: python3.6: command not 
  found
  </code></pre>
  
  <p>I found that I have this same issue when I try to submit a job as an array. For example, when I submit the following with sbatch:</p>
  
  <pre><code>!/bin/bash
  #
  #SBATCH --job-name=hostname_sleep_sample 
  #SBATCH --output=out_%j.txt
  #SBATCH --array=1-10
  #SBATCH --cpus-per-task=1
  #
  #SBATCH --mem=2G
  echo PWD $PWD
  cd $SLURM_SUBMIT_DIR
  python3.6 my_script.py input_$SLURM_ARRAY_TASK_ID.txt
  ~  
  </code></pre>
  
  <p>I find that only out_1.txt shows that the job ran.  All of the output files for tasks 2-10 show the same error message:</p>
  
  <pre><code>/var/spool/slurmd/job00130/slurm_script: line 9: python3.6: command not 
  </code></pre>
  
  <p>I am running all of these scripts in an HPC cluster that I created using the Compute Engine API in the Google Cloud Platform.  I used the following tutorial to set up the SLURM cluster:</p>
  
  <p><a href="https://codelabs.developers.google.com/codelabs/hpc-slurm-on-gcp/#0" rel="nofollow noreferrer">https://codelabs.developers.google.com/codelabs/hpc-slurm-on-gcp/#0</a></p>
  
  <p>Why is SLURM unable to run multiple python3.6 jobs at the same time and how can I get my array submission to work?  I have spent days going through SLURM FAQs and other stack questions but I have not found out a way to resolve this issue or a suitable explanation of whats causing the issue in the first place.</p>
  
  <p>Thank you                                                      </p>