How to write to named pipe with the threading module in Python?
Question: I am trying to write a program which communicates to an external program
through named pipes. The python script does not know when the external program
opens up the named pipe/file for reading, so the python script need to open
the pipe in blocking mode, see
[open](http://pubs.opengroup.org/onlinepubs/007908799/xsh/open.html). If the
python script opens in non-blocking mode and the external program has not
opened for reading, the open statement will result in an error.

So to open in blocking mode, the python script could open the named pipe in a
separate thread, and I have tried the threading module. In the below example I
just read from the named pipe in the main thread, but it produces the same
error:

    
    
    import threading
    import os
    
    pipe_name = 'pipe_test'
    
    
    class WriterNamedPipe(threading.Thread):
    
        def __init__(self, filepath, input):
            '''
            Write: generate that will output each line of input
            '''
            # Inherit
            threading.Thread.__init__(self, verbose = True)
            self.daemon = False
            self.filepath = filepath
            self.input = input
            self.start()
    
        def run(self):
            # Open blockingly
            with open(self.filepath, 'w') as f:
                f.write(self.input)
    
    if not os.path.exists(pipe_name):
        os.mkfifo(pipe_name)
    
    WriterNamedPipe(pipe_name, '1\n' * 100)
    
    with open(pipe_name, 'r') as f:
        print f.read()
    

This would results in a hang/freeze:

    
    
    MainThread: <WriterNamedPipe(Thread-1, initial)>.start(): starting thread
    Thread-1: <WriterNamedPipe(Thread-1, started 1078922160)>.__bootstrap(): thread started
    Thread-1: <WriterNamedPipe(Thread-1, started 1078922160)>.__bootstrap(): normal return
    
    Compilation hangup
    

However, a similar example from [here](http://www.python-course.eu/pipes.php)
works, but with `os.fork`:

    
    
    import os, time, sys
    pipe_name = 'pipe_test'
    
    def child( ):
        pipeout = os.open(pipe_name, os.O_WRONLY)
        counter = 0
        while True:
            time.sleep(1)
            os.write(pipeout, 'Number %03d\n' % counter)
            counter = (counter+1) % 5
    
    def parent( ):
        pipein = open(pipe_name, 'r')
        while True:
            line = pipein.readline()[:-1]
            print 'Parent %d got "%s" at %s' % (os.getpid(), line, time.time( ))
    
    if not os.path.exists(pipe_name):
        os.mkfifo(pipe_name)  
    pid = os.fork()    
    if pid != 0:
        parent()
    else:       
        child()
    

Why is the example with the threading module hanging?


Answer: This does probably not work due to the
[GIL](http://www.dabeaz.com/python/UnderstandingGIL.pdf). The open statement
in the thread blocks the whole program. This could be avoided by using
[multiprocessing](https://docs.python.org/2/library/multiprocessing.html)
module instead.

