title: Parallelization of os.walk()
body: <p>I have a large directory structure hosted on a network storage device that I'd like to traverse using <code>os.walk</code>. The system is rather slow, but I think the process could be done quicker if I could query multiple directories at the same time (all with the same common root). I don't care what order the outputs come in, so long as by the end I have parsed everything.</p>
  
  <p>I'm thinking of re-implementing <code>os.walk</code> to pass new directories into a pool of threaded workers. I'd much rather use someone else's code if it already exists (why reinvent the wheel?), but haven't come across any.</p>
  
  <p>Surely this is a common task? Has anyone come across something like this before? Maybe I'm missing something and it wouldn't result in a speedup.</p>
  
  <p>If I don't get an answer in a week or so, no doubt I'll post my attempt here.</p>