Multiprocessing with python to find Max value
Question: I'm working with Python 2.7.5 and OpenCV. I have a test image and I want to
find it's most similar image in an array of images. I have written a function
using OpenCV that will give me the total number of similarity points. The more
similar points I have the more similar the images are. Unfortunately this is a
rather time consuming function so I would like to parallelize my code to make
it faster.

    
    
    #img is the image that I am trying to find the most number of similar pointswith
    maxSimilarPts = 0;
    
    #testImages is a list of testImages
    for testImage in testImages:
        #getNumSimilarPts returns the number of similar points between two images
        similarPts = getNumSimilarPts(img, testImage) 
    
        if similarPts > maxSimilarPts:
            maxSimilarPts = similarPts
    

How can I do this in parallel with python? Any help would be greatly
appreciated.


Answer: The following is a (untested) parallel version of the original code. It runs 5
workers in parallel. Each one takes an image from the input queue, calculates
the similary, then puts the value and image onto an output queue. When all the
workers are done, there are no more images, then the parent process prints the
(similarity, imageID) of the most similar image.

    
    
    # adapted from Raymond Hettinger
    # http://stackoverflow.com/questions/11920490/how-do-i-run-os-walk-in-parallel-in-python/23779787#23779787
    
    from multiprocessing.pool import Pool
    from multiprocessing import JoinableQueue as Queue
    import os, sys
    
    
    def parallel_worker():
        while True:
            testImage = imageq.get()
            similarPts = getNumSimilarPts(img, testImage) 
            similarq.put( [similarPts, testImage] )
            imageq.task_done()
    
    similarq = Queue()
    imageq = Queue()
    for testImage in testImages:
        imageq.put(testImage)
    
    pool = Pool(5)
    for i in range(5):
        pool.apply_async(parallel_worker)
    
    imageq.join()
    print 'Done'
    
    print max(similarq)
    

