Which Python async library would be best suited for my code? Asyncore? Twisted?
Question: I have a program I'm working on that will be reading from two 'network
sources' simultaneously. I wanted to try out an asynchronous approach rather
than use threading. This has lead me to wonder which library to use...

I've come up with some simple example code that kind of demonstrates what my
program will be doing:

    
    
    import sniffer
    
    def first():
        for station in sniffer.sniff_wifi():
            log(station.mac())
    
    def second():
        for station in sniffer.sniff_ethernet():
            log(station.mac())
    
    first()
    second()
    

The two `sniffer` methods look somewhat like this:

    
    
    def sniff_wifi(self):
    
        while True:
            yield mac_address
    

The `while True` loop obviously makes them blocking.

I want to use `asyncore` for this as it is part of the standard library. No
3rd party dependencies are a bonus. However, that doesn't mean I won't use it
if you recommend I do...

Can I achieve what I'm trying to do with asyncore? If so, could you show me
how to convert my example code to 'asyncore code'? Do you know of any good
asyncore tutorials?


Answer: Twisted is better in pretty much every possible way. It's more portable, more
featureful, simpler, more scalable, better maintained, better documented, and
it can make a delicious omelette. Asyncore is, for all intents and purposes,
obsolete.

It's hard to demonstrate all the ways in which Twisted is superior in a short
answer (how could I demonstrate a
[http](http://twistedmatrix.com/documents/10.2.0/api/twisted.web.html)/[dns](http://twistedmatrix.com/documents/10.2.0/api/twisted.names.html)/[ssh](http://twistedmatrix.com/documents/10.2.0/api/twisted.conch.html)/[smtp/pop/imap](http://twistedmatrix.com/documents/10.2.0/api/twisted.mail.html)/[irc/xmpp](http://twistedmatrix.com/documents/10.2.0/api/twisted.words.html)/[process-
spawning](http://twistedmatrix.com/documents/10.2.0/api/twisted.internet.interfaces.IReactorProcess.html)/[multi-
threading](http://twistedmatrix.com/documents/10.2.0/api/twisted.internet.interfaces.IReactorThreads.html)
server in a short example?), so instead I'll focus on one of the most common
misconceptions that people seem to have about Twisted: that it's somehow more
complex or harder to use than asyncore.

Let's start with an asyncore example. In order to avoid a biased presentation,
I'll use an example from someone else who still likes asyncore a bit. Here's a
simple asyncore example [taken from Richard Jones'
weblog](http://www.mechanicalcat.net/richard/log/Python/A_simple_asyncore__echo_server__example)
(with comments elided for brevity).

First, here's the server:

    
    
    import asyncore, socket
    
    class Server(asyncore.dispatcher):
        def __init__(self, host, port):
            asyncore.dispatcher.__init__(self)
            self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
            self.bind(('', port))
            self.listen(1)
    
        def handle_accept(self):
            socket, address = self.accept()
            print 'Connection by', address
            EchoHandler(socket)
    
    class EchoHandler(asyncore.dispatcher_with_send):
        def handle_read(self):
            self.out_buffer = self.recv(1024)
            if not self.out_buffer:
                self.close()
    
    s = Server('', 5007)
    asyncore.loop()
    

and here's the client:

    
    
    import asyncore, socket
    
    class Client(asyncore.dispatcher_with_send):
        def __init__(self, host, port, message):
            asyncore.dispatcher.__init__(self)
            self.create_socket(socket.AF_INET, socket.SOCK_STREAM)
            self.connect((host, port))
            self.out_buffer = message
    
        def handle_close(self):
            self.close()
    
        def handle_read(self):
            print 'Received', self.recv(1024)
            self.close()
    
    c = Client('', 5007, 'Hello, world')
    asyncore.loop()
    

There are a few obscure cases that this code doesn't handle correctly, but
explaining them is boring and complicated, and the code has already made this
answer long enough.

Now, here's some code that does basically the same thing, with Twisted. First,
the server:

    
    
    from twisted.internet import reactor, protocol as p
    
    class Echo(p.Protocol):
        def dataReceived(self, data):
            self.transport.write(data)
    
    class EchoFactory(p.Factory):
        def buildProtocol(self, addr):
            print 'Connection by', addr
            return Echo()
    
    reactor.listenTCP(5007, EchoFactory())
    reactor.run()
    

And now, the client:

    
    
    from twisted.internet import reactor, protocol as p
    
    class EchoClient(p.Protocol):
        def connectionMade(self):
            self.transport.write(self.factory.data)
    
        def dataReceived(self, data):
            print 'Received:', data
            self.transport.loseConnection()
    
    class EchoClientFactory(p.ClientFactory):
        protocol = EchoClient
        def __init__(self, data):
            self.data = data
    
    reactor.connectTCP('localhost', 5007, EchoClientFactory('Hello, world'))
    reactor.run()
    

There are a couple of things that I'd like to draw your attention to. First of
all, the Twisted example is 25% shorter, even for something this trivial. 40
lines for asyncore, only 30 for Twisted. As your protocol grows more complex,
this difference will get bigger and bigger, as you need to write more and more
support code for asyncore that would have been provided for you by Twisted.

Second, Twisted provides a _complete abstraction_. With the asyncore example,
you have to use the `socket` module to do the actual networking; asyncore
provides only multiplexing. This is a problem if you need [portable behavior
on platforms such as Windows](http://itamarst.org/writings/win32sockets.html).
It also means that asyncore completely lacks facilities for doing asynchronous
sub-process communication on other platforms; you can't stuff arbitrary file
descriptors into a `select()` call on Windows.

Third, the Twisted example is _transport neutral_. None of `Echo` and
`EchoFactory` and `EchoClient` and `EchoClientFactory` is in any way specific
to TCP. You can make these classes into a library that can be connected via
SSH, or SSL, or a UNIX socket, or a pipe, only by changing the one
`connectTCP`/`listenTCP` call at the bottom. This is important, as supporting
something like TLS directly in your protocol logic is very tricky. For
example, a 'write' in TLS will trigger a 'read' at the lower level. So, you
need to separate these concerns out to get them right.

Finally, specific to your use-case, if you're dealing with MAC addresses and
ethernet frames directly, Twisted contains [Twisted
Pair](http://twistedmatrix.com/documents/10.2.0/api/twisted.pair.html), a low-
level library for dealing with IP and ethernet-level networking. This isn't
the most actively maintained part of Twisted; the code is quite old. But, it
should work, and if it doesn't we will take any bugs in it seriously and
(eventually) see that they get fixed. As far as I'm aware, there's no
comparable library for asyncore, and it certainly doesn't contain any such
code itself.

