title: Very slow communication between Python and C++ program with subprocess
body: <p>I am trying to speed up a Python program by outsourcing some repetitive calculations to a C++ program by using subprocess Python module.</p>
  
  <p>To illustrate my problem, I took a simple C++ code that return the double of the input. It takes 16 seconds for a million of integers, which seems very slow.</p>
  
  <p>Here is the C++ program (double.exe) :</p>
  
  <pre><code>#include &lt;iostream&gt;
  
  using namespace std;
  
  int main()
  {
      int a;
      bool goon = true;
      while (goon)
      {
          cin &gt;&gt; a;
          cout &lt;&lt; 2 * a &lt;&lt; endl;
          if (a == 0)
              goon= false;
      }
  }
  </code></pre>
  
  <p>And here the Python 3 code :</p>
  
  <pre><code>from time import time
  from subprocess import PIPE,Popen
  
  cmd = ["double"]
  process = Popen(cmd, stdin=PIPE,stdout=PIPE, bufsize=32,universal_newlines=True, shell=True)
  
  t0 = time()
  for i in range(1,int(1e6)):
      print(i, file=process.stdin, flush=True)
      output = int(process.stdout.readline())
  dt = time() - t0
  print("Time to communicate : %fs" % dt)
  print(0,file=process.stdin,flush=True) # close 'double' program
  </code></pre>
  
  <blockquote>
    <p>Time to communicate : 16.029137s</p>
  </blockquote>
  
  <p>To me the reason why it's so slow can only be the communication between the Python process and the C++ program through the pipes, but I haven't find how to accelerate it. Any solution to speed up this communication, with subprocess or an other library ?</p>
  
  <p>I'm using Python 3.5.2 on Windows.</p>