This is turning out not to be as simple as I thought it should be. What I am trying to do is chain a few commands using `Popen` by setting the `stdin` of the commands to the `stdout` of the previous command (or `PIPE` for the first `stdin`). Additionally I set the `stderr`s of the commands to `open`-ed files.

What I am seeing is that basically this doesn't work at all until I `close` the `stdin` of the first command. I am expecting the processes to communicate down the pipe chain when I write to the first command's `stdin` but nothing gets written to the second command until I close the first command's `stdin`. Similarly, none of the `stderr` files are actually written to until I close the first command's `stdin`.

Consider the following examples:

`./cat.py`

```
#!/usr/bin/python3

import sys, datetime

def log(msg, file):
   file.write(str(datetime.datetime.now()) + ': ' + msg + '\n')

log('starting now', sys.stdout)
log('rofl', sys.stderr)

for input_line in iter(sys.stdin.readline, ''):
   input_line = input_line.rstrip('\n')

   log(input_line, sys.stdout)
   log('lol ' + input_line, sys.stderr)

log('ending now', sys.stdout)

```

and

`./basic.py`

```
#!/usr/bin/python3

import sys, subprocess

subproc = []

length = 4
for i in range(0, length):
   if i == 0:
      subproc.append(subprocess.Popen('./cat.py', bufsize=0, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=open(str(i) + '.stderr', 'w')))
   elif i == length - 1:
      subproc.append(subprocess.Popen('./cat.py', bufsize=0, stdin=subproc[-1].stdout, stdout=open(str(i) + '.stdout', 'w'), stderr=open(str(i) + '.stderr', 'w')))
   else:
      subproc.append(subprocess.Popen('./cat.py', bufsize=0, stdin=subproc[-1].stdout, stdout=subprocess.PIPE, stderr=open(str(i) + '.stderr', 'w')))

for input_line in iter(sys.stdin.readline, ''):
   input_line = input_line.rstrip('\n')

   subproc[0].stdin.write((input_line + '\n').encode())

```

This will start a 4-process `Popen` chain and write the `stderr`s to `0.stderr`, `1.stderr`, etc. and write the final `stdout` to `3.stdout`. Output lines are timestamped so you can easily verify when a specific subprocess received input. Consider these outputs from a basic test I did waiting a few seconds between inputs.

`0.stderr`

```
2018-03-07 16:18:48.316847: rofl
2018-03-07 16:18:51.815376: lol hello 1
2018-03-07 16:18:55.561168: lol hello 2
2018-03-07 16:18:59.204273: lol hello 3

```

`1.stderr`

```
2018-03-07 16:18:48.320991: rofl
2018-03-07 16:19:01.888496: lol 2018-03-07 16:18:48.316825: starting now
2018-03-07 16:19:01.888508: lol 2018-03-07 16:18:51.815345: hello 1
2018-03-07 16:19:01.888515: lol 2018-03-07 16:18:55.561136: hello 2
2018-03-07 16:19:01.888523: lol 2018-03-07 16:18:59.204241: hello 3
2018-03-07 16:19:01.888531: lol 2018-03-07 16:19:01.886950: ending now

```

`3.stdout`

```
2018-03-07 16:18:48.323203: starting now
2018-03-07 16:19:01.899180: 2018-03-07 16:18:48.323766: starting now
2018-03-07 16:19:01.899208: 2018-03-07 16:19:01.895778: 2018-03-07 16:18:48.320973: starting now
2018-03-07 16:19:01.899216: 2018-03-07 16:19:01.895804: 2018-03-07 16:19:01.888476: 2018-03-07 16:18:48.316825: starting now
2018-03-07 16:19:01.899224: 2018-03-07 16:19:01.895813: 2018-03-07 16:19:01.888504: 2018-03-07 16:18:51.815345: hello 1
2018-03-07 16:19:01.899233: 2018-03-07 16:19:01.895820: 2018-03-07 16:19:01.888512: 2018-03-07 16:18:55.561136: hello 2
2018-03-07 16:19:01.899240: 2018-03-07 16:19:01.895829: 2018-03-07 16:19:01.888519: 2018-03-07 16:18:59.204241: hello 3
2018-03-07 16:19:01.899247: 2018-03-07 16:19:01.895836: 2018-03-07 16:19:01.888527: 2018-03-07 16:19:01.886950: ending now
2018-03-07 16:19:01.899254: 2018-03-07 16:19:01.895844: 2018-03-07 16:19:01.892480: ending now
2018-03-07 16:19:01.899262: 2018-03-07 16:19:01.895860: ending now
2018-03-07 16:19:01.899277: ending now

```

As you can see, the first subprocess gets my `write`s in real-time but then the chain stalls until I close the first subprocess's `stdin`. This is despite me using `bufsize=0` for all `Popen` calls. (Without using `0` then not even the first subprocess received any of my `write`s until after closing `stdin`.)

Is what I'm trying to do possible? What is the correct way to do this?

`communicate` will not work in my case, because that function closes `stdin`. I need to keep `stdin` open the whole while to process additional incoming data. I need to keep the first `stdin` open to receive additional input while the output from the command is sent down the pipe chain, handled by the next command and sent further down the pipe chain.

Any suggestions?

I am using Python 3.4.3.

Thank you,