Python multicore without communication
Question: I want to call a methode getRecommendations which just pickle recommendations
to a specific user to a file. I used a code from a book which works. But I saw
that only one core works and I want that all my cores do the work, because
this would be much faster.

Here is the method.

    
    
    def getRecommendations(prefs,person,similarity=sim_pearson):
        print "working on recommendation"
        totals={}
        simSums={}
        for other in prefs:
        # don't compare me to myself
            if other==person: continue
            sim=similarity(prefs,person,other)
            # ignore scores of zero or lower
            if sim<=0: continue
            for item in prefs[other]:
                # only score movies I haven't seen yet
                if item not in prefs[person] or prefs[person][item]==0:
                    # Similarity * Score
                    totals.setdefault(item,0)
                    totals[item]+=prefs[other][item]*sim
                    # Sum of similarities
                    simSums.setdefault(item,0)
                    simSums[item]+=sim
        # Create the normalized list
        rankings=[(total/simSums[item],item) for item,total in totals.items( )]
        # Return the sorted list
        rankings.sort( )
        rankings.reverse( )
        ranking_output = open("data/rankings/"+str(int(person))+".ranking.recommendations","wb")
        pickle.dump(rankings,ranking_output)
        return rankings
    

It is called via

    
    
    for i in customerID: 
            print "working on ", int(i)
            #Make this working with multiple CPU's
            getRecommendations(pickle.load(open("data/critics.recommendations", "r")), int(i))
    

as you can see i try to make a recommendation to every customer. Which will be
used later.

So how can i multiprocess this method? I don't get it by reading a few
examples or even the
[documentation](http://docs.python.org/2/library/multiprocessing.html)


Answer: You want something (roughly, untested) like:

    
    
    from multiprocessing import Pool
    NUMBER_OF_PROCS = 5 # some number... not necessarily the number of cores due to I/O
    
    pool = Pool(NUMBER_OF_PROCS)
    
    for i in customerID:
        pool.apply_async(getRecommendations, [i])
    
    pool.close()
    pool.join()
    

(this is assuming you only pass 'i' into getRecommendations, since the
pickle.load should only be done once)

