title: How to spark-submit with main class in jar?
body: <p>There are a ton of questions about <code>ClassNotFoundException</code> but I haven't seen any (yet) that fit this specific case. I am attempting to run the following command:</p>
  
  <p><code>spark-submit --master local[*] --class com.stronghold.HelloWorld scala-ts.jar</code></p>
  
  <p>It throws the following exception:</p>
  
  <pre><code>\u@\h:\w$ spark_submit --class com.stronghold.HelloWorld scala-ts.jar                                                                                                                                                                                                                                                                               ⬡ 9.8.0 [±master ●●●] 
  2018-05-06 19:52:33 WARN  Utils:66 - Your hostname, asusTax resolves to a loopback address: 127.0.1.1; using 192.168.1.184 instead (on interface p1p1)                               
  2018-05-06 19:52:33 WARN  Utils:66 - Set SPARK_LOCAL_IP if you need to bind to another address                                                                                       
  2018-05-06 19:52:33 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable                                
  java.lang.ClassNotFoundException: com.stronghold.HelloWorld                               
          at java.net.URLClassLoader.findClass(URLClassLoader.java:381)                     
          at java.lang.ClassLoader.loadClass(ClassLoader.java:424)                          
          at java.lang.ClassLoader.loadClass(ClassLoader.java:357)                          
          at java.lang.Class.forName0(Native Method)                                        
          at java.lang.Class.forName(Class.java:348)                                        
          at org.apache.spark.util.Utils$.classForName(Utils.scala:235)                     
          at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:836)                                                                  
          at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:197)        
          at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:227)             
          at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:136)               
          at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)                    
  2018-05-06 19:52:34 INFO  ShutdownHookManager:54 - Shutdown hook called                   
  2018-05-06 19:52:34 INFO  ShutdownHookManager:54 - Deleting directory /tmp/spark-e8a77988-d30c-4e96-81fe-bcaf5d565c75
  </code></pre>
  
  <p>However, the jar clearly contains this class:</p>
  
  <pre><code>1     " zip.vim version v28                                                                                                                                                                                                                                                                                                                                               
      1 " Browsing zipfile /home/[USER]/projects/scala_ts/out/artifacts/TimeSeriesFilter_jar/scala-ts.jar
      2 " Select a file with cursor and press ENTER
      3  
      4 META-INF/MANIFEST.MF
      5 com/
      6 com/stronghold/
      7 com/stronghold/HelloWorld$.class
      8 com/stronghold/TimeSeriesFilter$.class
      9 com/stronghold/DataSource.class
     10 com/stronghold/TimeSeriesFilter.class
     11 com/stronghold/HelloWorld.class
     12 com/stronghold/scratch.sc
     13 com/stronghold/HelloWorld$delayedInit$body.class
  </code></pre>
  
  <p>Typically, the hang up here is on file structure, but I am pretty sure that's correct here:  </p>
  
  <pre><code>../
  scala_ts/
  | .git/
  | .idea/
  | out/
  | | artifacts/
  | | | TimeSeriesFilter_jar/
  | | | | scala-ts.jar
  | src/
  | | main/
  | | | scala/
  | | | | com/
  | | | | | stronghold/
  | | | | | | DataSource.scala
  | | | | | | HelloWorld.scala
  | | | | | | TimeSeriesFilter.scala
  | | | | | | scratch.sc
  | | test/
  | | | scala/
  | | | | com/
  | | | | | stronghold/
  | | | | | | AppTest.scala
  | | | | | | MySpec.scala                                                                                                                                                                                                                                                                                                                                                  
  | target/
  | README.md
  | pom.xml
  </code></pre>
  
  <p>I have run other jobs with the same structure at work (so, a different environment).  I am now trying to gain some more facility with a home project, but this seems to be an early hang up.  </p>
  
  <p>In a nutshell, am I just missing something glaringly obvious?</p>
  
  <p><strong>APPENDIX</strong></p>
  
  <p>For those that are interested, here is my pom:</p>
  
  <pre><code>&lt;project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"&gt;
    &lt;modelVersion&gt;4.0.0&lt;/modelVersion&gt;
    &lt;groupId&gt;com.stronghold&lt;/groupId&gt;
    &lt;artifactId&gt;scala-ts&lt;/artifactId&gt;
    &lt;version&gt;1.0-SNAPSHOT&lt;/version&gt;
    &lt;inceptionYear&gt;2008&lt;/inceptionYear&gt;
    &lt;properties&gt;
      &lt;scala.version&gt;2.11.8&lt;/scala.version&gt;
    &lt;/properties&gt;
  
    &lt;repositories&gt;
      &lt;repository&gt;
        &lt;id&gt;scala-tools.org&lt;/id&gt;
        &lt;name&gt;Scala-Tools Maven2 Repository&lt;/name&gt;
        &lt;url&gt;http://scala-tools.org/repo-releases&lt;/url&gt;
      &lt;/repository&gt;
    &lt;/repositories&gt;
  
    &lt;pluginRepositories&gt;
      &lt;pluginRepository&gt;
        &lt;id&gt;scala-tools.org&lt;/id&gt;
        &lt;name&gt;Scala-Tools Maven2 Repository&lt;/name&gt;
        &lt;url&gt;http://scala-tools.org/repo-releases&lt;/url&gt;
      &lt;/pluginRepository&gt;
    &lt;/pluginRepositories&gt;
  
    &lt;dependencies&gt;
      &lt;dependency&gt;
        &lt;groupId&gt;org.scala-lang&lt;/groupId&gt;
        &lt;artifactId&gt;scala-library&lt;/artifactId&gt;
        &lt;version&gt;2.11.8&lt;/version&gt;
      &lt;/dependency&gt;
      &lt;dependency&gt;
        &lt;groupId&gt;junit&lt;/groupId&gt;
        &lt;artifactId&gt;junit&lt;/artifactId&gt;
        &lt;version&gt;4.9&lt;/version&gt;
        &lt;scope&gt;test&lt;/scope&gt;
      &lt;/dependency&gt;
      &lt;dependency&gt;
        &lt;groupId&gt;org.scala-tools.testing&lt;/groupId&gt;
        &lt;artifactId&gt;specs_2.10&lt;/artifactId&gt;
        &lt;version&gt;1.6.9&lt;/version&gt;
        &lt;scope&gt;test&lt;/scope&gt;
      &lt;/dependency&gt;
      &lt;dependency&gt;
        &lt;groupId&gt;org.apache.spark&lt;/groupId&gt;
        &lt;artifactId&gt;spark-core_2.11&lt;/artifactId&gt;
        &lt;version&gt;2.2.0&lt;/version&gt;
      &lt;/dependency&gt;
      &lt;dependency&gt;
        &lt;groupId&gt;org.apache.spark&lt;/groupId&gt;
        &lt;artifactId&gt;spark-sql_2.11&lt;/artifactId&gt;
        &lt;version&gt;2.2.0&lt;/version&gt;
      &lt;/dependency&gt;
      &lt;dependency&gt;
        &lt;groupId&gt;org.apache.spark&lt;/groupId&gt;
        &lt;artifactId&gt;spark-catalyst_2.11&lt;/artifactId&gt;
        &lt;version&gt;2.2.0&lt;/version&gt;
      &lt;/dependency&gt;
      &lt;dependency&gt;
        &lt;groupId&gt;org.apache.hadoop&lt;/groupId&gt;
        &lt;artifactId&gt;hadoop-common&lt;/artifactId&gt;
        &lt;version&gt;2.7.3&lt;/version&gt;
      &lt;/dependency&gt;
    &lt;/dependencies&gt;
  
    &lt;build&gt;
      &lt;sourceDirectory&gt;src/main/scala&lt;/sourceDirectory&gt;
      &lt;testSourceDirectory&gt;src/test/scala&lt;/testSourceDirectory&gt;
      &lt;plugins&gt;
        &lt;plugin&gt;
          &lt;groupId&gt;org.scala-tools&lt;/groupId&gt;
          &lt;artifactId&gt;maven-scala-plugin&lt;/artifactId&gt;
          &lt;executions&gt;
            &lt;execution&gt;
              &lt;goals&gt;
                &lt;goal&gt;compile&lt;/goal&gt;
                &lt;goal&gt;testCompile&lt;/goal&gt;
              &lt;/goals&gt;
            &lt;/execution&gt;
          &lt;/executions&gt;
          &lt;configuration&gt;
            &lt;scalaVersion&gt;${scala.version}&lt;/scalaVersion&gt;
            &lt;args&gt;
              &lt;arg&gt;-target:jvm-1.5&lt;/arg&gt;
            &lt;/args&gt;
          &lt;/configuration&gt;
        &lt;/plugin&gt;
        &lt;plugin&gt;
          &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
          &lt;artifactId&gt;maven-eclipse-plugin&lt;/artifactId&gt;
          &lt;configuration&gt;
            &lt;downloadSources&gt;true&lt;/downloadSources&gt;
            &lt;buildcommands&gt;
              &lt;buildcommand&gt;ch.epfl.lamp.sdt.core.scalabuilder&lt;/buildcommand&gt;
            &lt;/buildcommands&gt;
            &lt;additionalProjectnatures&gt;
              &lt;projectnature&gt;ch.epfl.lamp.sdt.core.scalanature&lt;/projectnature&gt;
            &lt;/additionalProjectnatures&gt;
            &lt;classpathContainers&gt;
              &lt;classpathContainer&gt;org.eclipse.jdt.launching.JRE_CONTAINER&lt;/classpathContainer&gt;
              &lt;classpathContainer&gt;ch.epfl.lamp.sdt.launching.SCALA_CONTAINER&lt;/classpathContainer&gt;
            &lt;/classpathContainers&gt;
          &lt;/configuration&gt;
        &lt;/plugin&gt;
      &lt;/plugins&gt;
    &lt;/build&gt;
    &lt;reporting&gt;
      &lt;plugins&gt;
        &lt;plugin&gt;
          &lt;groupId&gt;org.scala-tools&lt;/groupId&gt;
          &lt;artifactId&gt;maven-scala-plugin&lt;/artifactId&gt;
          &lt;configuration&gt;
            &lt;scalaVersion&gt;${scala.version}&lt;/scalaVersion&gt;
          &lt;/configuration&gt;
        &lt;/plugin&gt;
      &lt;/plugins&gt;
    &lt;/reporting&gt;
  &lt;/project&gt;
  </code></pre>
  
  <p><strong>UPDATE</strong></p>
  
  <p>Apologies for the lack of clarity.  I ran the command from within the same directory as the <code>.jar</code> (<code>/home/[USER]/projects/scala_ts/out/artifacts/TimeSeriesFilter_jar/</code>). That said, just to be clear, specifying the full path does not change the outcome.</p>
  
  <p>It should also be noted that I can run HelloWorld from within Intellij, and it uses the same class reference (<code>com.stronghold.HelloWorld</code>).</p>