title: Replace HDFS form local disk to s3 getting error (org.apache.hadoop.service.AbstractService)
body: <p>We are trying to setup Cloudera 5.5 where HDFS will be working on s3 only for that we have already configured the necessory properties in Core-site.xml </p>
  
  <pre><code>&lt;property&gt;
      &lt;name&gt;fs.s3a.access.key&lt;/name&gt;
      &lt;value&gt;################&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
      &lt;name&gt;fs.s3a.secret.key&lt;/name&gt;
      &lt;value&gt;###############&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
      &lt;name&gt;fs.default.name&lt;/name&gt;
      &lt;value&gt;s3a://bucket_Name&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
      &lt;name&gt;fs.defaultFS&lt;/name&gt;
      &lt;value&gt;s3a://bucket_Name&lt;/value&gt;
  &lt;/property&gt;
  </code></pre>
  
  <p>After setting it up we were able to browse the files for s3 bucket from command</p>
  
  <pre><code>hadoop fs -ls /
  </code></pre>
  
  <p>And it shows the files available on s3 only.</p>
  
  <p>But when we start the yarn services JobHistory server fails to start with below error and on launching pig jobs we are getting same error</p>
  
  <pre><code>PriviledgedActionException as:mapred (auth:SIMPLE) cause:org.apache.hadoop.fs.UnsupportedFileSystemException: No AbstractFileSystem for scheme: s3a
  ERROR   org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils   
  Unable to create default file context [s3a://kyvosps]
  org.apache.hadoop.fs.UnsupportedFileSystemException: No AbstractFileSystem for scheme: s3a
      at org.apache.hadoop.fs.AbstractFileSystem.createFileSystem(AbstractFileSystem.java:154)
      at org.apache.hadoop.fs.AbstractFileSystem.get(AbstractFileSystem.java:242)
      at org.apache.hadoop.fs.FileContext$2.run(FileContext.java:337)
      at org.apache.hadoop.fs.FileContext$2.run(FileContext.java:334)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:415)
  </code></pre>
  
  <p>On serching on Internet we found that we need to set following properties as well in core-site.xml </p>
  
  <pre><code>&lt;property&gt;
    &lt;name&gt;fs.s3a.impl&lt;/name&gt;
    &lt;value&gt;org.apache.hadoop.fs.s3a.S3AFileSystem&lt;/value&gt;
    &lt;description&gt;The implementation class of the S3A Filesystem&lt;/description&gt;
  &lt;/property&gt;
  &lt;property&gt;
      &lt;name&gt;fs.AbstractFileSystem.s3a.impl&lt;/name&gt;
      &lt;value&gt;org.apache.hadoop.fs.s3a.S3AFileSystem&lt;/value&gt;
      &lt;description&gt;The FileSystem for  S3A Filesystem&lt;/description&gt;
  &lt;/property&gt;
  </code></pre>
  
  <p>After setting the above properties we are getting following error</p>
  
  <pre><code>org.apache.hadoop.service.AbstractService   
  Service org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager failed in state INITED; cause: java.lang.RuntimeException: java.lang.NoSuchMethodException: org.apache.hadoop.fs.s3a.S3AFileSystem.&lt;init&gt;(java.net.URI, org.apache.hadoop.conf.Configuration)
  java.lang.RuntimeException: java.lang.NoSuchMethodException: org.apache.hadoop.fs.s3a.S3AFileSystem.&lt;init&gt;(java.net.URI, org.apache.hadoop.conf.Configuration)
      at org.apache.hadoop.fs.AbstractFileSystem.newInstance(AbstractFileSystem.java:131)
      at org.apache.hadoop.fs.AbstractFileSystem.createFileSystem(AbstractFileSystem.java:157)
      at org.apache.hadoop.fs.AbstractFileSystem.get(AbstractFileSystem.java:242)
      at org.apache.hadoop.fs.FileContext$2.run(FileContext.java:337)
      at org.apache.hadoop.fs.FileContext$2.run(FileContext.java:334)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:415)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
      at org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:334)
      at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:451)
      at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:473)
      at org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils.getDefaultFileContext(JobHistoryUtils.java:247)
  </code></pre>
  
  <p>The jars needed for this is in place but still getting the error any help will be great. Thanks in advance</p>
  
  <p><strong>Update</strong></p>
  
  <p>I tried to remove the property <strong>fs.AbstractFileSystem.s3a.impl</strong> but it give me the same first exception the one i was getting previously which is </p>
  
  <pre><code>org.apache.hadoop.security.UserGroupInformation 
  PriviledgedActionException as:mapred (auth:SIMPLE) cause:org.apache.hadoop.fs.UnsupportedFileSystemException: No AbstractFileSystem for scheme: s3a
  ERROR   org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils   
  Unable to create default file context [s3a://bucket_name]
  org.apache.hadoop.fs.UnsupportedFileSystemException: No AbstractFileSystem for scheme: s3a
      at org.apache.hadoop.fs.AbstractFileSystem.createFileSystem(AbstractFileSystem.java:154)
      at org.apache.hadoop.fs.AbstractFileSystem.get(AbstractFileSystem.java:242)
      at org.apache.hadoop.fs.FileContext$2.run(FileContext.java:337)
      at org.apache.hadoop.fs.FileContext$2.run(FileContext.java:334)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:415)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
      at org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:334)
      at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:451)
      at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:473)
  </code></pre>