title: Read multiple files in a folder with Scala for a Spark job
body: <p>I have a folder which has several part files from an earlier job The folder name is "tera-output", and it has the files part-00000, part-00001, part-00002 etc..till part-00049. I am trying to write a scala program to now read each file in the folder. The code is given below:</p>
  
  <pre><code>import org.apache.spark.SparkContext
  import org.apache.spark.SparkContext._
  import org.apache.spark.SparkConf
  import org.apache.spark.storage.StorageLevel
  
  object SimpleApp {
    def main(args: Array[String]) {
      val conf = new SparkConf().setAppName("WordCount")
      val sc = new SparkContext(conf)
      val file = sc.textFile("s3n://test/tera-output")
      val splits = file.map(word =&gt; word)
      println(splits.count())
    }
  }
  </code></pre>
  
  <p>My problem is I don't know how to run a loop to read each file in the "tera-output" folder. Each file will be read through map(), and the data appended to "splits". I looked through some of the documentation and posts here and could not really find a function to do that. </p>
  
  <p>Could someone please help with this? Thank you in advance!!</p>