How to write to a file using non blocking IO?
Question: I want to write to a file using a non-blocking method in Python. On some
googling, I found that the language supports `fcntl` in order to do so, but
the method to implement the same is not very clear to me.

This is the code snippet (I don't know where I am going wrong):

    
    
    import os, fcntl
    nf = fcntl.fcntl(0,fcntl.F_UNCLK)
    fcntl.fcntl(0,fcntl.F_SETFL , nf | os.O_NONBLOCK )
    nf = open ("test.txt", 'a') 
    nf.write ( " sample text \n")
    

Is this the correct way to perform a non-blocking IO operation on a file? I
doubt it. Also, could you suggest any other modules in Python which allow me
to do so?


Answer: This is how you turn non-blocking mode on for a file in UNIX:

    
    
    fd = os.open("filename", os.O_CREAT | os.O_WRONLY | os.O_NONBLOCK)
    os.write(fd, "data")
    os.close(fd)
    

On UNIX, however, [turning on non-blocking mode has no visible effect for
regular files](http://www.remlab.net/op/nonblock.shtml)! Even though the file
is in non-blocking mode, the `os.write` call won't return immediately, it will
sleep until the write is complete. To prove it to yourself experimentally, try
this:

    
    
    import os
    import datetime
    
    data = "\n".join("testing\n" * 10 for x in xrange(10000000))
    print("Size of data is %d bytes" % len(data))
    
    print("open at %s" % str(datetime.datetime.now()))
    fd = os.open("filename", os.O_CREAT | os.O_WRONLY | os.O_NONBLOCK)
    print("write at %s" % str(datetime.datetime.now()))
    os.write(fd, data)
    print("close at %s" % str(datetime.datetime.now()))
    os.close(fd)
    print("end at %s" % str(datetime.datetime.now()))
    

You'll notice that the `os.write` call does take several seconds. Even though
the call is non-blocking (technically, it's not blocking, it's sleeping), the
call is _not_ asynchronous.

* * *

AFAIK, there is no way to write to a file asynchronously on Linux or on
Windows. You can simulate it, however, using threads. Twisted has a method
named `deferToThread` for this purpose. Here's how you use it:

    
    
    from twisted.internet import threads, reactor
    
    data = "\n".join("testing\n" * 10 for x in xrange(10000000))
    print("Size of data is %d bytes" % len(data))
    
    def blocking_write():
        print("Starting blocking_write")
        f = open("testing", "w")
        f.write(data)
        f.close()
        print("End of blocking_write")
    
    def test_callback():
        print("Running test_callback, just for kicks")
    
    d = threads.deferToThread(blocking_code)
    reactor.callWhenRunning(cc)
    reactor.run()
    

