title: Setting Spark Classpath on Amazon EMR
body: <p>I am trying to run some simple jobs on EMR (AMI 3.6) with Hadoop 2.4 and Spark 1.3.1.  I have installed Spark manually without a bootstrap script.  Currently I am trying to read and process data from S3 but it seems like I am missing an endless number of jars on my classpath.  </p>
  
  <p>Running commands on spark-shell.  Starting shell using:</p>
  
  <pre><code>spark-shell --jars jar1.jar,jar2.jar...
  </code></pre>
  
  <p>Commands run on the shell:</p>
  
  <pre><code>val lines = sc.textFile("s3://folder/file.gz")
  lines.collect()
  </code></pre>
  
  <p>The errors always look something like: "Class xyz not found".  After I find the needed jar and add it to the classpath, I will get this error again but with a different class name in the error message. </p>
  
  <p>Is there a set of jars that are needed for working with (compressed and uncompressed) S3 files?</p>