Matplotlib: simultaneous plotting in multiple threads
Question: I am trying to do some plotting in parallel to finish large batch jobs
quicker. To this end, I start a thread for each plot I plan on making.

I had hoped that each thread would finish its plotting and close itself (as I
understand it, Python closes threads when they get through all the statements
in run()). Below is some code that shows this behavior.

If the line that creates a figure is commented out, it runs as expected.
Another plausibly helpful tidbit is that it also runs as expected when you
only spawn one thread.

    
    
    import matplotlib.pyplot as plt
    import time
    import Queue
    import threading
    
    def TapHistplots():
        ##  for item in ['str1']:
    # # it behaves as expected if the line above is used instead of the one below
        for item in ['str1','str2']:
            otheritem = 1
            TapHistQueue.put((item, otheritem))
            makeTapHist().start()
    
    class makeTapHist(threading.Thread):
        def run(self):
            item, otheritem = TapHistQueue.get()
            fig = FigureQueue.get()
            FigureQueue.put(fig+1)
            print item+':'+str(fig)+'\n',
            time.sleep(1.3)
            plt.figure(fig) # comment out this line and it behaves as expected
            plt.close(fig)
    
    TapHistQueue = Queue.Queue(0)
    FigureQueue = Queue.Queue(0)
    def main():
        start = time.time()
        """Code in here runs only when this module is run directly"""
        FigureQueue.put(1)
        TapHistplots()
        while threading.activeCount()>1:
            time.sleep(1)
            print 'waiting on %d threads\n' % (threading.activeCount()-1),
        print '%ds elapsed' % (time.time()-start)
    
    if __name__ == '__main__':
        main()
    

Any help is duly appreciated.


Answer: Why not just use multiprocessing? As far as I can tell from your description,
threading won't help you much, anyway...

Matplotlib already threads so that you can display and interact with multiple
figures at once. If you want to speed up batch processing on a multicore
machine, you're going to need multiprocessing regardless.

As a basic example (**Warning: This will create 20 small .png files in
whatever directory you run it in!**)

    
    
    import multiprocessing
    import matplotlib.pyplot as plt
    import numpy as np
    
    def main():
        pool = multiprocessing.Pool()
        num_figs = 20
        input = zip(np.random.randint(10,1000,num_figs), 
                    range(num_figs))
        pool.map(plot, input)
    
    def plot(args):
        num, i = args
        fig = plt.figure()
        data = np.random.randn(num).cumsum()
        plt.plot(data)
        plt.title('Plot of a %i-element brownian noise sequence' % num)
        fig.savefig('temp_fig_%02i.png' % i)
    
    main()
    

