title: HDFS IO error (hadoop 2.8) with Flume
body: <p>I am getting the below error when I try to get a streaming data into hadoop through Flume.</p>
  
  <p>I have created link in flume/lib that point to the <code>.jar</code> files in hadoop/share/hadoop/</p>
  
  <p>I double checked the URL and I think they are all correct. Thought of posting to get some more eyes and some feedback.</p>
  
  <pre><code>      2017-07-20 10:53:18,959 (SinkRunner-PollingRunner-DefaultSinkProcessor) [WARN -org.apache.flume.sink.hdfs.HDFSEventSink.process HDFSEventSink.java:455)] HDFS IO error
        java.io.IOException: No FileSystem for scheme: hdfs
          at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2798)
          at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2809)
          at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:100)
          at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2848)
          at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2830)
          at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:389)
          at org.apache.hadoop.fs.Path.getFileSystem(Path.java:356)
          at org.apache.flume.sink.hdfs.BucketWriter$1.call(BucketWriter.java:243)
          at org.apache.flume.sink.hdfs.BucketWriter$1.call(BucketWriter.java:235)
          at org.apache.flume.sink.hdfs.BucketWriter$9$1.run(BucketWriter.java:679)
          at org.apache.flume.auth.SimpleAuthenticator.execute(SimpleAuthenticator.java:50)
          at org.apache.flume.sink.hdfs.BucketWriter$9.call(BucketWriter.java:676)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
          at java.lang.Thread.run(Thread.java:748)
  </code></pre>
  
  <p>Here is the Flume Sink Config</p>
  
  <pre><code>agent1.sinks.PurePathSink.type = hdfs
  agent1.sinks.PurePathSink.hdfs.path = hdfs://127.0.0.1:9000/User/bts/pp 
  agent1.sinks.PurePathSink.hdfs.fileType = DataStream
  agent1.sinks.PurePathSink.hdfs.filePrefix = export
  agent1.sinks.PurePathSink.hdfs.fileSuffix = .txt
  agent1.sinks.PurePathSink.hdfs.rollInterval = 120
  agent1.sinks.PurePathSink.hdfs.rollSize = 131072
  </code></pre>
  
  <p>core-site.xml - Hadoop 2.8</p>
  
  <pre><code>&lt;configuration&gt;
  
      &lt;property&gt;
          &lt;name&gt;hadoop.tmp.dir&lt;/name&gt;
          &lt;value&gt;/home1/tmp&lt;/value&gt;
          &lt;description&gt;A base for other temporary directories&lt;/description&gt;
      &lt;/property&gt;
      &lt;property&gt;
          &lt;name&gt;fs.default.name&lt;/name&gt;
          &lt;value&gt;hdfs://127.0.0.1:9000&lt;/value&gt;
      &lt;/property&gt;
  
      &lt;property&gt;
          &lt;name&gt;fs.file.impl&lt;/name&gt;
          &lt;value&gt;org.apache.hadoop.fs.LocalFileSystem&lt;/value&gt;
          &lt;description&gt;The FileSystem for file: uris.&lt;/description&gt;
      &lt;/property&gt;
  
      &lt;property&gt;
          &lt;name&gt;fs.hdfs.impl&lt;/name&gt;
          &lt;value&gt;org.apache.hadoop.hdfs.DistributedFileSystem&lt;/value&gt;
          &lt;description&gt;The FileSystem for hdfs: uris.&lt;/description&gt;
      &lt;/property&gt;
  </code></pre>