title: Simple non-network concurrency with Twisted
body: <p>I have a problem with using Twisted for simple concurrency in python. The problem is - I don't know how to do it and all online resources are about Twisted networking abilities. So I am turning to SO-gurus for some guidance.</p>
  
  <p>Python 2.5 is used.</p>
  
  <p>Simplified version of my problem runs as follows:</p>
  
  <ol>
  <li>A bunch of scientific data</li>
  <li>A function that munches on the data and creates output</li>
  <li>??? &lt; here enters concurrency, it takes chunks of data from <strong>1</strong> and feeds it to <strong>2</strong></li>
  <li>Output from <strong>3</strong> is joined and stored</li>
  </ol>
  
  <p>My guess is that Twisted <code>reactor</code> can do the number three job. But how?</p>
  
  <p>Thanks a lot for any help and suggestions.</p>
  
  <p>upd1:</p>
  
  <p>Simple example code. No idea how reactor deals with processes, so I have given it imaginary functions:</p>
  
  <pre><code>datum = 'abcdefg'
  
  def dataServer(data):
      for char in data:
          yield chara
  
  def dataWorker(chara):
      return ord(chara)
  
  r = reactor()
  NUMBER_OF_PROCESSES_AV = 4
  serv = dataserver(datum)
  id = 0
  result = array(len(datum))
  
  while r.working():
      if NUMBER_OF_PROCESSES_AV &gt; 0:
          r.addTask(dataWorker(serv.next(), id)
          NUMBER_OF_PROCESSES_AV -= 1
          id += 1
      for pr, id in r.finishedProcesses():
          result[id] = pr
  </code></pre>