title: Python asyncio buffer and process data
body: <p>I'm having trouble with some CPU-intensive tasks within an asyncio event loop. The troubles I have arise when dealing with maintaining a buffer of incoming data and building packets from it. I've tried using executors to do the CPU bound stuff, but then had trouble maintaining the ordering of the buffer when packets were removed from it.</p>
  
  <p>I'm looking for a best-practice method to achieve the following functionality without having the CPU-bound tasks executed within the event loop.</p>
  
  <pre><code>import asyncio
  import struct
  
  class Reader(asyncio.Protocol):
      def __init__(self):
          self.extra = bytearray()
  
      def data_received(self, data):
          self.extra.extend(data)
          packet = get_packet(bytes(self.extra))
          if packet:
              del self.extra[:len(packet)]
              if verify_hash(packet):  # CPU intensive
                  asyncio.async(distribute(packet))  # Some asyncio fan-out callback
  
  
  def get_packet(data):  # CPU intensive
      if len(data) &gt; HEADER_SIZE:
          payload_size, = struct.unpack_from(HEADER_FORMAT, data)
          if len(data) &gt;= HEADER_SIZE + payload_size:
              return data[:HEADER_SIZE + payload_size]
      return None
  
  loop = asyncio.get_event_loop()
  loop.run_until_complete(loop.create_server(Reader, '0.0.0.0', 8000))
  loop.run_forever()
  </code></pre>