title: A clean, lightweight alternative to Python's twisted?
body: <p>A (long) while ago I wrote a web-spider that I multithreaded to enable concurrent requests to occur at the same time.  That was in my Python youth, in the days before I knew about the <a href="http://www.dabeaz.com/python/GIL.pdf" rel="noreferrer">GIL</a> and the associated woes it creates for multithreaded code (IE, most of the time stuff just ends up serialized!)...</p>
  
  <p>I'd like to rework this code to make it more robust and perform better.  There are basically two ways I could do this: I could use the new <a href="http://docs.python.org/library/multiprocessing.html" rel="noreferrer">multiprocessing module</a> in 2.6+ or I could go for a reactor / event-based model of some sort.  I would rather do the later since it's far simpler and less error-prone.</p>
  
  <p>So the question relates to what framework would be best suited to my needs.  The following is a list of the options I know about so far:</p>
  
  <ul>
  <li><a href="http://twistedmatrix.com/trac/" rel="noreferrer">Twisted</a>: The granddaddy of Python reactor frameworks: seems complex and a bit bloated however.  Steep learning curve for a small task.</li>
  <li><a href="http://eventlet.net/" rel="noreferrer">Eventlet</a>: From the guys at <a href="http://lindenlab.com/" rel="noreferrer">lindenlab</a>.  Greenlet based framework that's geared towards these kinds of tasks. I had a look at the code though and it's not too pretty: non-pep8 compliant, scattered with prints (why do people do this in a framework!?), API seems a little inconsistent.</li>
  <li><a href="http://code.google.com/p/pyev/" rel="noreferrer">PyEv</a>: Immature, doesn't seem to be anyone using it right now though it is based on libevent so it's got a solid backend.</li>
  <li><a href="http://docs.python.org/library/asyncore.html" rel="noreferrer">asyncore</a>: From the stdlib: über low-level, seems like a lot of legwork involved just to get something off the ground.</li>
  <li><a href="http://www.tornadoweb.org/" rel="noreferrer">tornado</a>: Though this is a server oriented product designed to server dynamic websites it does feature an <a href="http://github.com/facebook/tornado/blob/master/tornado/httpclient.py" rel="noreferrer">async HTTP client</a> and a simple <a href="http://github.com/facebook/tornado/blob/master/tornado/ioloop.py" rel="noreferrer">ioloop</a>.  Looks like it could get the job done but not what it was intended for. 
  [edit: doesn't run on Windows unfortunately, which counts it out for me - its a requirement for me to support this lame platform]</li>
  </ul>
  
  <p>Is there anything I have missed at all?  Surely there must be a library out there that fits the sweet-spot of a simplified async networking library!</p>
  
  <p>[edit: big thanks to <a href="https://stackoverflow.com/users/177663/intgr">intgr</a> for his pointer to <a href="http://code.google.com/p/cogen/" rel="noreferrer">this page</a>.  If you scroll to the bottom you will see there is a really nice list of projects that aim to tackle this task in one way or another.  It seems actually that things have indeed moved on since the inception of Twisted: people now seem to favour a <a href="http://en.wikipedia.org/wiki/Coroutine" rel="noreferrer">co-routine</a> based solution rather than a traditional reactor / callback oriented one.  The benefits of this approach are clearer more direct code: I've certainly found in the past, especially when working with <a href="http://www.boost.org/doc/libs/1_41_0/doc/html/boost_asio.html" rel="noreferrer">boost.asio</a> in C++ that callback based code can lead to designs that can be hard-to-follow and are relatively obscure to the untrained eye.  Using co-routines allows you to write code that looks a little more synchronous at least.  I guess now my task is to work out which one of these many libraries I like the look of and give it a go!  Glad I asked now...]</p>
  
  <p>[edit: perhaps of interest to anyone who followed or stumbled on this this question or cares about this topic in any sense: I found a really great writeup of the current state of <a href="http://nichol.as/asynchronous-servers-in-python" rel="noreferrer">the available tools</a> for this job]</p>