title: Synchronized Output With Bash's Process Substitution
body: <p>I have to multiply call an inflexible external tool that takes as arguments some input data and an output file to which it will write the processed data, for example:</p>
  
  <pre><code>some_prog() {  echo "modified_$1" &gt; "$2"; }
  </code></pre>
  
  <p>For varying input, I want to call some_prog, filter the output and write the output of all calls into the same file "out_file". Additionally, I want to add a header line to the output file before each call of some_prog. Given the following dummy filter:</p>
  
  <pre><code>slow_filter() { 
      read input; sleep "0.000$(($RANDOM % 10))"; echo "filtered_$input"
  }
  </code></pre>
  
  <p>I wrote the following code:</p>
  
  <pre><code>rm -f out_file
  for input in test_input{1..8}; do
      echo "#Header_for_$input"  &gt;&gt; "out_file"
      some_prog $input &gt;( slow_filter  &gt;&gt; "out_file" )
  done
  </code></pre>
  
  <p>However, this will produce an out_file like this:</p>
  
  <pre><code>#Header_for_test_input1
  #Header_for_test_input2
  #Header_for_test_input3
  #Header_for_test_input4
  #Header_for_test_input5
  #Header_for_test_input6
  #Header_for_test_input7
  #Header_for_test_input8
  filtered_modified_test_input4
  filtered_modified_test_input1
  filtered_modified_test_input2
  filtered_modified_test_input5
  filtered_modified_test_input6
  filtered_modified_test_input3
  filtered_modified_test_input8
  filtered_modified_test_input7
  </code></pre>
  
  <p>The output I expected was:</p>
  
  <pre><code>#Header_for_test_input1
  filtered_modified_test_input1
  #Header_for_test_input2
  filtered_modified_test_input2
  #Header_for_test_input3
  filtered_modified_test_input3
  #Header_for_test_input4
  filtered_modified_test_input4
  #Header_for_test_input5
  filtered_modified_test_input5
  #Header_for_test_input6
  filtered_modified_test_input6
  #Header_for_test_input7
  filtered_modified_test_input7
  #Header_for_test_input8
  filtered_modified_test_input8
  </code></pre>
  
  <p>I realized that the >( ) process substitution forks the shell. Is there a way to synchronize the output of the subshells? Or is there another elegant solution to this problem? I want to avoid the obvious approach of writing to different files in each iteration because, in my code, the for loop has a few 100,000 iterations. </p>