title: Python and Kaldi: Broken Pipe Error
body: <p>The setup is the following: I am working with Kaldi tools. They are called in bash scripts of a certain form. Those bash scripts are called by a wrapper program written in python that submits the tasks to a Sun Grid Engine. </p>
  
  <p>The command I want to execute is the following </p>
  
  <pre><code>feat-to-dim 'ark:copy-feats scp:train.scp ark:- |' -
  </code></pre>
  
  <p>Executing this in the command line, yields the correct result '40' together with the warning </p>
  
  <pre><code>WARNING (feat-to-dim[5.0.23-f7b2f]:Close():kaldi-io.cc:501) Pipe copy-feats scp:train.scp ark:- | had nonzero return status 13
  </code></pre>
  
  <p>However, if I wrap this in the following way:</p>
  
  <pre><code>python -c "import os; os.system(\"feat-to-dim 'ark:copy-feats scp:train.scp ark:- |' -\")"
  </code></pre>
  
  <p>the program <code>copy-feats</code> fails with the error:</p>
  
  <pre><code>ERROR (copy-feats[5.0.23-f7b2f]:Write():kaldi-matrix.cc:1240) Failed to write matrix to stream
  </code></pre>
  
  <p>After various stack traces and following errors, the following warning is printed additionally: </p>
  
  <pre><code>WARNING (feat-to-dim[5.0.23-f7b2f]:Close():kaldi-io.cc:501) Pipe copy-feats scp:train.scp ark:- | had nonzero return status 134
  </code></pre>
  
  <p>This is what I've found out: <code>feat-to-dim</code> closes the pipe in advance whereas <code>copy-feats</code> tries to continue writing output. Since it is not possible, <code>copy-feats</code> is terminated. <code>13</code> probably indicates the broken pipe error. </p>
  
  <p>For Python, this is a serious issue why it turns it into an error and terminates. But in this case, it's not Python which yields this error, but <code>copy-feats</code>. Therefore, stuff like <a href="https://stackoverflow.com/questions/14207708/ioerror-errno-32-broken-pipe-python/30091579#30091579">try/catch</a> in python or <a href="https://stackoverflow.com/questions/33020759/piping-to-head-results-in-broken-pipe-in-shell-script-called-from-python">trapping the pipe</a> seem not to have any success in this case. </p>
  
  <p>Moreover, the following lines work perfectly fine without any warnings or errors:</p>
  
  <pre><code>python -c "import os; os.system(\"copy-feats scp:train.scp ark:-\")" &gt; cp
  python -c "import os; os.system(\"feat-to-dim ark:cp -\")"
  </code></pre>
  
  <p>and the following line yields the simple error message <code>cat: write error: Broken pipe</code> and exit status 256:</p>
  
  <pre><code>python -c "import os; os.system(\"feat-to-dim ark:'cat cp |' -\")"
  </code></pre>
  
  <p>Do you have any further suggestions here?</p>