title: How to get original location of script used for SLURM job?
body: <p>I'm starting the SLURM job with script and script must work depending on it's location which is obtained inside of script itself with <code>SCRIPT_LOCATION=$(realpath $0)</code>. But SLURM copies script to <code>slurmd</code> folder and starts job from there and it screws up further actions.</p>
  
  <p>Are there any option to get location of script used for slurm job before it has been moved/copied?</p>
  
  <p>Script is located in network shared folder <code>/storage/software_folder/software_name/scripts/this_script.sh</code> and it must to:</p>
  
  <ol>
  <li>get it's own location</li>
  <li>return the <code>software_name</code> folder</li>
  <li>copy the <code>software_name</code> folder to a local folder <code>/node_folder</code> on node</li>
  <li>run another script from copied folder <code>/node_folder/software_name/scripts/launch.sh</code></li>
  </ol>
  
  <p>My script is</p>
  
  <pre><code>#!/bin/bash
  #SBATCH --nodes=1
  #SBATCH --partition=my_partition_name
  
  # getting location of software_name 
  SHARED_PATH=$(dirname $(dirname $(realpath $0)))
  # separating the software_name from path
  SOFTWARE_NAME=$(basename $SHARED_PATH)
  # target location to copy project
  LOCAL_SOFTWARE_FOLDER='/node_folder'
  # corrected path for target
  LOCAL_PATH=$LOCAL_SOFTWARE_FOLDER/$SOFTWARE_NAME
  
  # Copying software folder from network storage to local
  cp -r $SHARED_PATH $LOCAL_SOFTWARE_FOLDER
  # running the script
  sh $LOCAL_PATH/scripts/launch.sh
  </code></pre>
  
  <p>It runs perfectly, when I run it on the node itself (without using SLURM) via: <code>sh /storage/software/scripts/this_script.sh</code>.</p>
  
  <p>In case of running it with SLURM as 
  <code>sbatch /storage/software/scripts/this_script.sh</code> it is assigned to one of nodes, but:</p>
  
  <ul>
  <li>before run it is copied to <code>/var/spool/slurmd/job_number/slurm_script</code> and it screws everything up since <code>$(dirname $(dirname $(realpath $0)))</code> returns <code>/var/spool/slurmd</code></li>
  </ul>
  
  <p>Is it possible to get original location (<code>/storage/software_folder/software_name/</code>) inside of script when it is started with SLURM?</p>
  
  <p>P.S. All machines are running Fedora 30 (x64)</p>
  
  <p><strong>UPDATE 1</strong></p>
  
  <p>There was a suggestion to run as <code>sbatch -D /storage/software_folder/software_name ./scripts/this_script.sh</code> and use the <code>SHARED_PATH="${SLURM_SUBMIT_DIR}"</code> inside of script itself.
  But it raise the error <code>sbatch: error: Unable to open file ./scripts/this_script.sh</code>.</p>
  
  <p>Also, I tried to use absolute paths:
  <code>sbatch -D /storage/software_folder/software_name /storage/software_folder/software_name/scripts/this_script.sh</code>. It tries to run, but:</p>
  
  <ul>
  <li>in such case it uses specified folder for creating output file only</li>
  <li>software still doesn't want to run</li>
  <li>attempt to use <code>echo "${SLURM_SUBMIT_DIR}"</code> inside of script prints <code>/home/username_who_started_script</code> instead of <code>/storage/software_folder/software_name</code></li>
  </ul>
  
  <p>Any other suggestions?</p>
  
  <p><strong>UPDATE 2:</strong>
  Also tried to use <code>#SBATCH --chdir=/storage/software_folder/software_name</code> inside of script, but in such case <code>echo "${SLURM_SUBMIT_DIR}"</code> returns <code>/home/username_who_started_script</code>or  <code>/</code> (if run as root)</p>
  
  <p><strong>UPDATE 3</strong></p>
  
  <p>Approach with <code>${SLURM_SUBMIT_DIR}</code> worked only if task is ran as:</p>
  
  <pre><code>cd /storage/software_folder/software_name
  sbatch ./scripts/this_script.sh
  </code></pre>
  
  <p>But it doesn't seem to be a proper solution. Are there any other ways?</p>
  
  <p><strong>SOLUTION</strong></p>
  
  <pre><code>#!/bin/bash
  #SBATCH --nodes=1
  #SBATCH --partition=my_partition_name
  
  # check if script is started via SLURM or bash
  # if with SLURM: there variable '$SLURM_JOB_ID' will exist
  # `if [ -n $SLURM_JOB_ID ]` checks if $SLURM_JOB_ID is not an empty string
  if [ -n $SLURM_JOB_ID ];  then
      # check the original location through scontrol and $SLURM_JOB_ID
      SCRIPT_PATH=$(scontrol show job $SLURM_JOBID | awk -F= '/Command=/{print $2}')
  else
      # otherwise: started with bash. Get the real location.
      SCRIPT_PATH=$(realpath $0)
  fi
  
  # getting location of software_name 
  SHARED_PATH=$(dirname $(dirname $(SCRIPT_PATH)))
  # separating the software_name from path
  SOFTWARE_NAME=$(basename $SHARED_PATH)
  # target location to copy project
  LOCAL_SOFTWARE_FOLDER='/node_folder'
  # corrected path for target
  LOCAL_PATH=$LOCAL_SOFTWARE_FOLDER/$SOFTWARE_NAME
  
  # Copying software folder from network storage to local
  cp -r $SHARED_PATH $LOCAL_SOFTWARE_FOLDER
  # running the script
  sh $LOCAL_PATH/scripts/launch.sh
  </code></pre>