title: Apache Spark -- using spark-submit throws a NoSuchMethodError
body: <p>To submit a Spark application to a cluster, their documentation notes:</p>
  
  <blockquote>
    <p>To do this, create an assembly jar (or “uber” jar) containing your code and its dependencies. Both sbt and Maven have assembly plugins. When creating assembly jars, list Spark and Hadoop as provided dependencies; these need not be bundled since they are provided by the cluster manager at runtime. -- <a href="http://spark.apache.org/docs/latest/submitting-applications.html" rel="noreferrer">http://spark.apache.org/docs/latest/submitting-applications.html</a></p>
  </blockquote>
  
  <p>So, I added the Apache Maven Shade Plugin to my <code>pom.xml</code> file. (version 3.0.0) <br/>
  And I turned my Spark dependency's scope into <code>provided</code>. (version 2.1.0) <br/></p>
  
  <p>(I also added the Apache Maven Assembly Plugin to ensure I was wrapping all of my dependencies in the jar when I run <code>mvn clean package</code>. I'm unsure if it's truly necessary.)</p>
  
  <hr>
  
  <p>Thus is how <code>spark-submit</code> fails. It throws a <code>NoSuchMethodError</code> for a dependency I have (note that the code works from a local instance when compiling inside IntelliJ, assuming that <code>provided</code> is removed).</p>
  
  <pre><code>Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Stopwatch.createStarted()Lcom/google/common/base/Stopwatch;
  </code></pre>
  
  <p>The line of code that throws the error is irrelevant--it's simply the first line in my main method that creates a <code>Stopwatch</code>, part of the Google Guava utilities. (version 21.0)</p>
  
  <p>Other solutions online suggest that it has to do with version conflicts of Guava, but I haven't had any luck yet with those suggestions. Any help would be appreciated, thank you.</p>