title: How can I access S3/S3n from a local Hadoop 2.6 installation?
body: <p>I am trying to reproduce an Amazon EMR cluster on my local machine. For that purpose, I have installed the <a href="http://ftp.cixug.es/apache/hadoop/common/hadoop-2.6.0/" rel="noreferrer">latest stable version of Hadoop as of now - 2.6.0</a>.
  Now I would like to access an S3 bucket, as I do inside the EMR cluster.</p>
  
  <p>I have added the aws credentials in core-site.xml:</p>
  
  <pre><code>&lt;property&gt;
    &lt;name&gt;fs.s3.awsAccessKeyId&lt;/name&gt;
    &lt;value&gt;some id&lt;/value&gt;
  &lt;/property&gt;
  
  &lt;property&gt;
    &lt;name&gt;fs.s3n.awsAccessKeyId&lt;/name&gt;
    &lt;value&gt;some id&lt;/value&gt;
  &lt;/property&gt;
  
  &lt;property&gt;
    &lt;name&gt;fs.s3.awsSecretAccessKey&lt;/name&gt;
    &lt;value&gt;some key&lt;/value&gt;
  &lt;/property&gt;
  
  &lt;property&gt;
    &lt;name&gt;fs.s3n.awsSecretAccessKey&lt;/name&gt;
    &lt;value&gt;some key&lt;/value&gt;
  &lt;/property&gt;
  </code></pre>
  
  <p><em>Note: Since there are some slashes on the key, I have escaped them with %2F</em></p>
  
  <p>If I try to list the contents of the bucket:</p>
  
  <pre><code>hadoop fs -ls s3://some-url/bucket/
  </code></pre>
  
  <p>I get this error:</p>
  
  <p><em>ls: No FileSystem for scheme: s3</em></p>
  
  <p>I edited core-site.xml again, and added information related to the fs:</p>
  
  <pre><code>&lt;property&gt;
    &lt;name&gt;fs.s3.impl&lt;/name&gt;
    &lt;value&gt;org.apache.hadoop.fs.s3.S3FileSystem&lt;/value&gt;
  &lt;/property&gt;
  
  &lt;property&gt;
    &lt;name&gt;fs.s3n.impl&lt;/name&gt;
    &lt;value&gt;org.apache.hadoop.fs.s3native.NativeS3FileSystem&lt;/value&gt;
  &lt;/property&gt;
  </code></pre>
  
  <p>This time I get a different error:</p>
  
  <pre><code>-ls: Fatal internal error
  java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.s3.S3FileSystem not found
          at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2074)
          at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2578)
          at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
  </code></pre>
  
  <p>Somehow I suspect the Yarn distribution does not have the necessary jars to be able to read S3, but I have no idea where to get those. Any pointers in this direction would be greatly appreciated.</p>