python asynchronization
Question: Imagine a text file of 20 mb. I am reading char by char and extracting the
useful information. I have actually 2 main functions, one is reading the file
and the second is extracting the info. Something like this:

    
    
    def reader(path):
        f = open(path, 'r')
        source = f.read()
        f.close()
    
        while True:
            # here is where I read char by char and call the function extractor
    
    def extractor(s):
        # here I extract the useful information
    

Now, my goal is to continue to read while extractor is working. So basicaly,
my question is what is the appropriate way to accomplish my goal?


Answer: You can use producer/consumer threads. The threads can be synchronized using a
`Queue.Queue`.

**EDIT** : an example of a producer/consumer system:

    
    
    from threading import Thread
    from Queue import Queue
    
    
    def produce(queue, n_items):
        for d in range(n_items):
            queue.put(d)
            print "put {0} in queue".format(d)
    
    def consume(queue, n_items):
        d = 0
        while d != n_items -1: # You need some sort of stop condition
            d = queue.get()
            print "got {0} from queue".format(d)
    
    def start_producer_and_consumer(wait):
        q = Queue()
        consumer_thread = Thread(target = consume, args = (q, 10))
        producer_thread = Thread(target = produce, args = (q, 10))
        producer_thread.start()
        consumer_thread.start()
        if wait:
            producer_thread.join()
            consumer_thread.join()
    
    if __name__ == '__main__':
        start_producer_and_consumer(True)
    

As you will see if you execute this, everything will be consumed in the
correct order.

