title: Spark Context Textfile: load multiple files
body: <p>I need to process multiple files scattered across various directories. I would like to load all these up in a single RDD and then perform map/reduce on it. I see that SparkContext is able to load multiple files from a single directory using wildcards. I am not sure how to load up files from multiple folders.</p>
  
  <p>The following code snippet fails:</p>
  
  <pre><code>for fileEntry in files:
      fileName = basePath + "/" + fileEntry
      lines = sc.textFile(fileName)
      if retval == None:
          retval = lines
      else:
          retval = sc.union(retval, lines)
  </code></pre>
  
  <p>This fails on the third loop with the following error message:</p>
  
  <pre><code>retval = sc.union(retval, lines)
  TypeError: union() takes exactly 2 arguments (3 given)
  </code></pre>
  
  <p>Which is bizarre given I am providing only 2 arguments. Any pointers appreciated.</p>