title: NoClassDefFoundError when using avro in spark-shell
body: <p>I keep getting </p>
  
  <blockquote>
    <p>java.lang.NoClassDefFoundError: org/apache/avro/mapred/AvroWrapper</p>
  </blockquote>
  
  <p>when calling <code>show()</code> on a DataFrame object. I'm attempting to do this through the shell (spark-shell --master yarn). I can see that the shell recognizes the schema when creating the DataFrame object, but if I execute any actions on the data it will always throw the <code>NoClassDefFoundError</code> when trying to instantiate the <code>AvroWrapper</code>. I've tried adding <code>avro-mapred-1.8.0.jar</code> in my <code>$HDFS_USER/lib</code> directory on the cluster and even included it using the --jar option when launching the shell. Neither of these options worked. Any advice would be greatly appreciated. Below is example code:</p>
  
  <pre><code>scala&gt; import org.apache.spark.sql._
  scala&gt; import com.databricks.spark.avro._
  scala&gt; val sqc = new SQLContext(sc)
  scala&gt; val df = sqc.read.avro("my_avro_file") // recognizes the schema and creates the DataFrame object
  scala&gt; df.show // this is where I get NoClassDefFoundError
  </code></pre>