title: Multi-threaded BASH programming - generalized method?
body: <p>Ok, I was running <a href="http://www.povray.org/" rel="nofollow noreferrer">POV-Ray</a> on all the demos, but POV's still single-threaded and wouldn't utilize more than one core.  So, I started thinking about a solution in BASH.</p>
  
  <p>I wrote a general function that takes a list of commands and runs them in the designated number of sub-shells.  This actually works but I don't like the way it handles accessing the next command in a <strike>thread-safe</strike> <i>multi-process</i> way:</p>
  
  <ul>
  <li>It takes, as an argument, a file with commands (1 per line), </li>
  <li>To get the "next" command, each process ("thread") will:
  
  <ul>
  <li>Waits until it can create a lock file, with: <strong>ln $CMDFILE $LOCKFILE</strong></li>
  <li>Read the command from the file,</li>
  <li>Modifies $CMDFILE by removing the first line,</li>
  <li>Removes the $LOCKFILE.</li>
  </ul></li>
  </ul>
  
  <p><strong>Is there a cleaner way to do this?</strong>  I couldn't get the sub-shells to read a single line from a FIFO correctly.
  <hr/>
  Incidentally, <strong>the point of this is to enhance what I can do on a BASH command line</strong>, and not to find non-bash solutions. I tend to perform a lot of complicated tasks from the command line and want another tool in the toolbox.</p>
  
  <p>Meanwhile, here's the function that handles getting the next line from the file.  As you can see, it modifies an on-disk file each time it reads/removes a line.  That's what seems hackish, but I'm not coming up with anything better, since FIFO's didn't work w/o <em>setvbuf()</em> in bash.</p>
  
  <pre><code>#
  # Get/remove the first line from FILE, using LOCK as a semaphore (with
  # short sleep for collisions).  Returns the text on standard output,
  # returns zero on success, non-zero when file is empty.
  #
  parallel__nextLine() 
  {
    local line rest file=$1 lock=$2
  
    # Wait for lock...
    until ln "${file}" "${lock}" 2&gt;/dev/null
    do sleep 1
       [ -s "${file}" ] || return $?
    done
  
    # Open, read one "line" save "rest" back to the file:
    exec 3&lt;"$file"
    read line &lt;&amp;3 ; rest=$(cat&lt;&amp;3)
    exec 3&lt;&amp;-
  
    # After last line, make sure file is empty:
    ( [ -z "$rest" ] || echo "$rest" ) &gt; "${file}"
  
    # Remove lock and 'return' the line read:
    rm -f "${lock}"
    [ -n "$line" ] &amp;&amp; echo "$line"
  }
  </code></pre>