title: Numpy savez exceptions
body: <p>I recently found this statement:</p>
  
  <blockquote>
    <p>np.savez is not multi-process safe, because it always uses
    gettempdir() +  key + ".npy". So, if you're running the same script
    over different data  sets, you will roach your own data when one
    process removes the  /tmp/arr_0.npy of another process.</p>
    
    <p>As luck would have it, I'm in that exact situation, so I have an
    example  error:</p>
  
  <pre><code>Traceback (most recent call last):    File
  "/home/pedro/prospectus/prelim/bbvs/bin/skldthresh", line 99, in &lt;module&gt; main()    
  File "/home/pedro/prospectus/prelim/bbvs/bin/skldthresh", line 90, in  main
   np.savez("%smask-t%0.2f" % (outfile,threshold), result)    
  File "/usr/lib/python2.5/site-packages/numpy/lib/io.py", line 230, in savez
   zip.write(filename, arcname=fname)    
   File "/usr/lib/python2.5/zipfile.py", line 541, in write
    st = os.stat(filename)  OSError: [Errno 2] No such file or directory: '/tmp/arr_0.npy'
  </code></pre>
    
    <p>Fortunately, replacing <code>tempfile.gettempdir()</code> with <code>tempfile.mkdtemp()</code>
    AND  remembering to clean up at the end with <code>os.rmdir(direc)</code> is all it
    takes to  fix it as far as I can tell.</p>
  </blockquote>
  
  <p>What does this exactly mean and how can I personally avoid this? Does numpy store the data written to an <code>.npz</code> file in temporary files, which are potentially named like other temporary <code>.npz</code> files in other scripts?!</p>
  
  <p>I have some scientific experiments which run over several hours and save their results via <code>np.savez</code>. The destination path for the results saved with <code>savez</code> is different, so every experiment has its own result path. However, they the scripts themselves are located in the same directory.</p>
  
  <p>Interestingly the scripts sometimes run several hours before this error occurs:</p>
  
  <pre><code>&gt; Traceback (most recent call last):   File
  &gt; "/work/var/slurmd/state.node348.d/job20832/slurm_script", line 53, in
  &gt; &lt;module&gt;
  &gt;     E = Exp6_06()   File
  &gt; "/work/experiments/s06/cs06_ex06.py", line
  &gt; 150, in __init__
  &gt;     self.__start()   File "/work/experiments/s06/cs06_ex06.py", line
  &gt; 374, in __start
  &gt;     File "/home/fx092/.local/lib/python2.7/site-packages/numpy/lib/npyio.py",
  &gt; line 600, in savez_compressed
  &gt;     _savez(file, args, kwds, True)   File "/home/fx092/.local/lib/python2.7/site-packages/numpy/lib/npyio.py",
  &gt; line 630, in _savez
  &gt;     fd, tmpfile = tempfile.mkstemp(suffix='-numpy.npy')   File "/sw/env/openmpi/2Q4/lib/python2.7/tempfile.py",
  &gt; line 314, in mkstemp
  &gt;     return _mkstemp_inner(dir, prefix, suffix, flags)   File "/sw/env/openmpi/2Q4/lib/python2.7/tempfile.py",
  &gt; line 244, in _mkstemp_inner
  &gt;     fd = _os.open(file, flags, 0600) OSError: [Errno 2] No such file or directory:
  &gt; '/work/tmp/node001.admin.2016-03-06-152506.fx092.27432/tmp5UulEz-numpy.npy'
  </code></pre>
  
  <p><strong>Used numpy version</strong>: 1.10.4</p>