title: Hadoop MapReduce provide nested directories as job input
body: <p>I'm working on a job that processes a nested directory structure, containing files on multiple levels:</p>
  
  <pre><code>one/
  ├── three/
  │   └── four/
  │       ├── baz.txt
  │       ├── bleh.txt
  │       └── foo.txt
  └── two/
      ├── bar.txt
      └── gaa.txt
  </code></pre>
  
  <p>When I add <code>one/</code> as an input path, no files are processed, since none are immediately available at the root level. </p>
  
  <p>I read about <code>job.addInputPathRecursively(..)</code>, but this seems to have been deprecated in the more recent releases (I'm using hadoop 1.0.2). I've written some code to walk the folders and add each dir with <code>job.addInputPath(dir)</code>, which worked until the job crashed when trying to process a directory as an input file for some reason, e.g. - trying to <code>fs.open(split.getPath())</code>, when <code>split.getPath()</code> is a directory (This happens inside <code>LineRecordReader.java</code>). </p>
  
  <p>I'm trying to convince myself there has to be a simpler way to provide a job with a nested directory structure. Any ideas?</p>
  
  <p><strong>EDIT</strong> - apparently there's an <a href="https://issues.apache.org/jira/browse/MAPREDUCE-3193">open bug</a> on this.</p>