title: Python3 calling Python2 multiprocessing in Linux acts differently than in Windows
body: <p>I have a simple sample code running in <code>Python3.8</code>, that opens a <code>subprocess</code> that executes in <code>Python2.7</code> (utilizing multiprocessing).</p>
  <p>In <code>Windows 10</code>, the behavior of my code is what my intention is.  Where the Python2 pool runs and prints to <code>stdout</code> accordingly.  And the <code>main.py</code> reads the stdout almost instantenously, as the pool writes on it.</p>
  <p>Unfortunately, I am seeing different results with Linux (<code>Ubuntu 20.04.1 LTS</code>).  It seems, in Linux, I wont get anything back until the whole pool is completed.</p>
  <p>How can I make the code work the same in Linux as well?</p>
  <p>Please see simple sample code below and output I am getting.</p>
  <p>Main.py</p>
  <pre><code>import subprocess
  import datetime
  import tempfile
  import os
  
  def get_time():
      return datetime.datetime.now()
  
  class ProcReader():
      def __init__(self, python_file, temp=None, wait=False):
          self.proc = subprocess.Popen(['python2', python_file], stdout=subprocess.PIPE)
  
      def __iter__(self):
          return self
  
      def __next__(self):
          while True:
              line = self.proc.stdout.readline()
              if not line:
                  raise StopIteration
              return line
  
  if __name__ == &quot;__main__&quot;:
      r1 = ProcReader(&quot;p2.py&quot;)
  
      for l1 in r1:
          print(&quot;Main reading at: {} for {}&quot;.format(get_time(), l1))
  </code></pre>
  <p>p2.py</p>
  <pre><code>import time
  import multiprocessing as mp
  from multiprocessing import freeze_support
  import datetime
  
  def get_time():
      return datetime.datetime.now()
  
  def f1(name):
      for x in range(2):
          time.sleep(1)
          print(&quot;{} Job#: {} from f1&quot;.format(get_time(), name))
  
  def f2(name):
      for x in range(2):
          time.sleep(2)
          print(&quot;{} Job#: {} from f2&quot;.format(get_time(), name))
  
  if __name__ == '__main__':
      freeze_support()
  
      pool = mp.Pool(2)
      tasks = [&quot;1&quot;, &quot;2&quot;, &quot;3&quot;, &quot;4&quot;, &quot;5&quot;, &quot;6&quot;, &quot;7&quot;]
      for i, task in enumerate(tasks):
          if i%2:
              pool.apply_async(f2, args=(task,))
          else:
              pool.apply_async(f1, args=(task,))
  
      pool.close()
      pool.join()
  </code></pre>
  <p>Output for Windows:</p>
  <pre><code>Main reading at: 2020-09-24 15:28:19.044626 for b'2020-09-24 15:28:19.044000 Job#: 1 from f1\n'
  Main reading at: 2020-09-24 15:28:20.045454 for b'2020-09-24 15:28:20.045000 Job#: 1 from f1\n'
  Main reading at: 2020-09-24 15:28:20.046711 for b'2020-09-24 15:28:20.046000 Job#: 2 from f2\n'
  Main reading at: 2020-09-24 15:28:21.045510 for b'2020-09-24 15:28:21.045000 Job#: 3 from f1\n'
  Main reading at: 2020-09-24 15:28:22.046334 for b'2020-09-24 15:28:22.046000 Job#: 3 from f1\n'
  Main reading at: 2020-09-24 15:28:22.047368 for b'2020-09-24 15:28:22.047000 Job#: 2 from f2\n'
  Main reading at: 2020-09-24 15:28:23.047519 for b'2020-09-24 15:28:23.047000 Job#: 5 from f1\n'
  Main reading at: 2020-09-24 15:28:24.046356 for b'2020-09-24 15:28:24.046000 Job#: 4 from f2\n'
  Main reading at: 2020-09-24 15:28:24.048356 for b'2020-09-24 15:28:24.048000 Job#: 5 from f1\n'
  Main reading at: 2020-09-24 15:28:26.047307 for b'2020-09-24 15:28:26.047000 Job#: 4 from f2\n'
  Main reading at: 2020-09-24 15:28:26.049168 for b'2020-09-24 15:28:26.049000 Job#: 6 from f2\n'
  Main reading at: 2020-09-24 15:28:27.047897 for b'2020-09-24 15:28:27.047000 Job#: 7 from f1\n'
  Main reading at: 2020-09-24 15:28:28.048337 for b'2020-09-24 15:28:28.048000 Job#: 7 from f1\n'
  Main reading at: 2020-09-24 15:28:28.049367 for b'2020-09-24 15:28:28.049000 Job#: 6 from f2\n'
  </code></pre>
  <p>Output for Linux:</p>
  <pre><code>Main reading at: 2020-09-24 19:28:45.972346 for b'2020-09-24 19:28:36.932473 Job#: 1 from f1\n'
  Main reading at: 2020-09-24 19:28:45.972559 for b'2020-09-24 19:28:37.933594 Job#: 1 from f1\n'
  Main reading at: 2020-09-24 19:28:45.972585 for b'2020-09-24 19:28:38.935255 Job#: 3 from f1\n'
  Main reading at: 2020-09-24 19:28:45.972597 for b'2020-09-24 19:28:39.936297 Job#: 3 from f1\n'
  Main reading at: 2020-09-24 19:28:45.972685 for b'2020-09-24 19:28:40.937666 Job#: 5 from f1\n'
  Main reading at: 2020-09-24 19:28:45.972711 for b'2020-09-24 19:28:41.938629 Job#: 5 from f1\n'
  Main reading at: 2020-09-24 19:28:45.972724 for b'2020-09-24 19:28:43.941109 Job#: 6 from f2\n'
  Main reading at: 2020-09-24 19:28:45.972735 for b'2020-09-24 19:28:45.943310 Job#: 6 from f2\n'
  Main reading at: 2020-09-24 19:28:45.973115 for b'2020-09-24 19:28:37.933317 Job#: 2 from f2\n'
  Main reading at: 2020-09-24 19:28:45.973139 for b'2020-09-24 19:28:39.935938 Job#: 2 from f2\n'
  Main reading at: 2020-09-24 19:28:45.973149 for b'2020-09-24 19:28:41.938587 Job#: 4 from f2\n'
  Main reading at: 2020-09-24 19:28:45.973157 for b'2020-09-24 19:28:43.941109 Job#: 4 from f2\n'
  Main reading at: 2020-09-24 19:28:45.973165 for b'2020-09-24 19:28:44.942306 Job#: 7 from f1\n'
  Main reading at: 2020-09-24 19:28:45.973173 for b'2020-09-24 19:28:45.943503 Job#: 7 from f1\n'
  </code></pre>
  <p>Please disregard the time since the clocks are different, but as you can see, in Windows <code>main.py</code> gets it as soon as it was written in python2 pool, but for <code>linux</code> everything in <code>main.py</code> is written only when all jobs were completed.  I am not too concerned about the order of the jobs being completed, I really just want <code>main.py</code> to get the <code>stdout</code> as soon as it is written in Python2 pool.</p>