title: Python multiprocess/multithreading use for concurrent file copy operation
body: <p>I am writing python code to copy bunch of big files/folders from one location to other location on desktop (no network, everything is local). I am using shutil module for that. </p>
  
  <p>But the problem is it takes more time so I want to speed up this copy process. I tried using threading and multiprocessing modules. But to my surprise both are taking more time than sequential code.</p>
  
  <p>One more observation is - the time required increases with increase in number of processes for same number of folders. What I mean is suppose I have following directory structure</p>
  
  <pre><code>                       /a/a1, /a/a2, /b/b1 and /b/b2
  </code></pre>
  
  <p>If I create 2 processes to copy folders a and b the time taken is suppose 2 minutes.
  Now if I create 4 processes to copy folders a/a1, a/a2, b/b1 and b/b2 it takes around 4 minutes. I tried this only with multiprocessing not sure about threading.</p>
  
  <p>I am not sure what is going. Would any one had similar problem? Are there any best practices which you can share to use multiprocessing/threading in python?</p>
  
  <p>Thanks
  Abhijit</p>