title: Why does subprocess keep running after communicate() is finished?
body: <p>I have an older python 2.7.5 script which suddenly makes problems on Red Hat Enterprise Linux Server release 7.6 (Maipo). After all I see, it runs fine on Red Hat Enterprise Linux Server release 7.4 (Maipo).
  The script basically implements something like </p>
  
  <p><code>cat /proc/cpuinfo | grep -m 1 -i 'cpu MHz'</code></p>
  
  <p>by creating two subrocesses and piping the output of the first into the second (see code example below). On the newer OS version, the cat processes stay open until the script terminates.</p>
  
  <p>It seems, that the pipe to grep somehow holds the cat-process open and I can't find any documentation on how to explicitely close it.</p>
  
  <p>The issue can be reproduced by pasting this code into the python CLI and then checking the ps process list for a static process 'cat /proc/cpuinfo'.
  The code is breaking down what's originally happening inside a loop, so please don't argue about its style. ;-)</p>
  
  <pre><code>import shlex
  from subprocess import *
  cmd1 = "cat /proc/cpuinfo"
  cmd2 = "grep -m 1 -i 'cpu MHz'"
  args1 = shlex.split(cmd1) # split into args
  args2 = shlex.split(cmd2) # split into args
  # first process uses default stdin
  ps1 = Popen(args1, stdout=PIPE)
  # then use the output of the previous process as stdin
  ps2 = Popen(args2, stdin=ps1.stdout, stdout=PIPE)
  out, err = ps2.communicate()
  print(out)
  </code></pre>
  
  <p>Afterwards check the process list in a second session(!) with:</p>
  
  <p><code>ps -eF |grep -v grep|grep /proc/cpuinfo</code></p>
  
  <p>On RHEL7.4 I find no open process in the process list, whereas on RHEL 7.6 after some attempts it looks like this:</p>
  
  <pre><code>[reinski@myhost ~]$ ps -eF |grep -v grep|grep /proc/cpuinfo
  reinski    2422  89459  0 26993   356 142 18:46 pts/3    00:00:00 cat /proc/cpuinfo
  reinski    2597 139605  0 26993   352  31 18:39 pts/3    00:00:00 cat /proc/cpuinfo
  reinski    7809 139605  0 26993   352  86 18:03 pts/3    00:00:00 cat /proc/cpuinfo
  </code></pre>
  
  <p>These processes will only dissappear when I close the python CLI, in which case I get errors like this (I left the formatting messed up as it was):</p>
  
  <pre><code>cat: write error: Broken pipe
  cat: write errorcat: write error: Broken pipe
  : Broken pipe
  </code></pre>
  
  <p>Why is cat obviously still wanting to write to the pipe, even though it should have already output the whole /proc/cpuinfo and should have terminated itself? </p>
  
  <p>Or more important: How can I prevent this from happening?</p>
  
  <p>Thanks for any help!</p>
  
  <h2>Example 2:</h2>
  
  <p>Given the suggestion from VPfB it turned out, that my example was a little unlucky, since the expected result can be achieved by a single grep command.
  So here is a modified example to show the problem with piping in another way:</p>
  
  <pre><code>import shlex
  from subprocess import *
  cmd1 = "grep -m 1 -i 'cpu MHz' /proc/cpuinfo"
  cmd2 = "awk '{print $4}'"
  args1 = shlex.split(cmd1) # split into args
  args2 = shlex.split(cmd2) # split into args
  # first process uses default stdin
  ps1 = Popen(args1, stdout=PIPE)
  # then use the output of the previous process as stdin
  ps2 = Popen(args2, stdin=ps1.stdout, stdout=PIPE)
  out, err = ps2.communicate()
  print(out)
  </code></pre>
  
  <p>This time, the result is a single zombie process for the grep process (169731 is the pid of the python session):</p>
  
  <pre><code>[reinski@myhost ~]$ ps -eF|grep 169731
  reinski  169731 189499  0 37847  6024 198 17:51 pts/2    00:00:00 python
  reinski  193999 169731  0     0     0 142 17:53 pts/2    00:00:00 [grep] &lt;defunct&gt;
  </code></pre>
  
  <p>So, is this just another symptom of the same problem or am I doing something completely wrong here?</p>