Python: Using multiprocessing when the function called is dependent on a file
<p>I have a very old fortran file, which is too complex for me to convert into python. So I have to compile the file and run it via python.</p>

<p>For the fortran file to work it requires 3 input values on 3 lines from the <code>mobcal.run</code> file. They are as follows:</p>

<pre><code>line 1 - Name of file to run
line 2 - Name of output file
line 3 - random seed number
</code></pre>

<p>I change the input values per worker in the <code>run()</code> function. 
When I run my script (see below), only 2 output file were created, but all 32 processers were running which i found out via the <code>top</code> command</p>

<p>I'm guessing the issue is here, is that there was not enough time to change the <code>mobcal.run</code> file for each worker.</p>

<p>The only solution I have come up with so far is to put a <code>time.sleep(random.randint(1,100))</code> at the beginning of the <code>run()</code> function. But I dont find this solution very elegant and may not always work as two workers may have the same <code>random.randint</code>, is there a more pythonic way to solve this?  </p>

<pre><code>def run(mfj_file):
        import shutil
        import random
        import subprocess
        #shutil.copy('./mfj_files/%s' % mfj_file, './')
        print 'Calculating cross sections for: %s' % mfj_file[:-4]
        with open('mobcal.run', 'w') as outf:
                outf.write(mfj_file+'\n'+mfj_file[:-4]+'.out\n'+str(random.randint(5000000,6000000)))

        ccs = subprocess.Popen(['./a.out'])
        ccs.wait()

        shutil.move('./'+mfj_file[:-4]+'.out', './results/%s.out' % mfj_file[:-4])



def mobcal_multi_cpu():
        from multiprocessing import Pool
        import os
        import shutil
        mfj_list = os.listdir('./mfj_files/')

        for f in mfj_list:
                shutil.copy('./mfj_files/'+f, './')

        if __name__ == '__main__':
                pool = Pool(processes=32)              
                pool.map(run,mfj_list)       


mobcal_multi_cpu()
</code></pre>
