Python Run a daemon sub-process & read stdout
Question: I need to run a program and gather its output to stdout. This program (socat)
needs to run in the background for the duration of the python script. Socat
sits in dameon mode once it's run, but first it outputs some lines to stdout
that I need for the rest of my script.

Command: `socat -d -d PTY: PTY:`

Output:

    
    
    2011/03/23 21:12:35 socat[7476] N PTY is /dev/pts/1
    2011/03/23 21:12:35 socat[7476] N PTY is /dev/pts/2
    2011/03/23 21:12:35 socat[7476] N starting data transfer loop with FDs [3,3] and [5,5]
    

...

I basically want to run that at the start of my program and leave it running
till script termination, but I need to read the two /dev/pts/X names into
python.

Can anyone tell me how to do this?

I came up with this which just hangs, I guess because it's blocking for the
child process to terminate.

    
    
    #!/usr/bin/python
    from subprocess import Popen, PIPE, STDOUT
    
    cmd = 'socat -d -d PTY: PTY: &'
    
    p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True)
    output = p.stdout.read()
    
    # Process the output 
    print(output)
    

Thanks for any help

EDIT: Seems it may write to stderr, but the script still just hanges with and
without the & even reading from stderr.


Answer: 
    #!/usr/bin/python
    from subprocess import Popen, PIPE, STDOUT
    import pty
    import os
    
    cmd = 'socat -d -d PTY: PTY:'
    
    master, slave = pty.openpty()
    
    p = Popen(cmd, shell=True, stdin=PIPE, stdout=slave, stderr=slave, close_fds=True)
    stdout = os.fdopen(master)
    print stdout.readline()
    print stdout.readline()
    

There are two problems with your version. Firstly, you call read without
argument which means it will attempt to read everything. But since socat
doesn't terminate, it never decides that it has read everything. By using
readline, python only reads until it finds a newline. From my understanding of
your problem that is what you need.

The second problem is that the C standard library will buffer outputs on
pipes. We solve that by creating a pty with the openpty() function and passing
it to both stdout and stderr of the subprocess. We use fdopen to make that
file descriptor into a regular python object and we get rid of the buffering.

I don't know what you are doing with the socat, but I wonder whether it could
replaced by using the pty module. You are copying one pty to another, and
openpty is creating a pair of ptys. Perhaps you can use those directly?

