title: list files recursively and filter by extension
body: <p>I am trying to recursively go through a given directory and all it's subdirectories, and list all the files that end with .json.  Thanks to <a href="https://stackoverflow.com/a/52446246/2751573">this answer</a>, I've got the recursive bit working.  Now I'm trying to figure out the filtering part.</p>
  
  <p>Here's my current attempt:</p>
  
  <pre><code>    import org.apache.hadoop.fs.{FileSystem,Path, PathFilter}
      import org.apache.hadoop.fs.Path;
  
      val fs = FileSystem.get(spark.sparkContext.hadoopConfiguration)
  
      def listJsonFiles(hdfsPath: String): List[String] = {
        fs
          .listStatus(new Path(hdfsPath)).filter(_.getPath.getName.endsWith(".json"))
          //.listStatus(new Path(hdfsPath))
          .flatMap { obj =&gt;
            if (obj.isFile)
              List(hdfsPath + "/" + obj.getPath.getName)
            // If it's a dir and we're in a recursive option:
            else
              listJsonFiles(hdfsPath + "/" + obj.getPath.getName)
          }
          .toList
      }
      val files = listJsonFiles("/path/to/some/stuff")
  </code></pre>
  
  <p>That returns an empty list. If I use the listStatus line without the filter, it returns a list of all the files in the path I pass in, including all the *.json files. So I know they exist, I can just can't get the filter working.</p>