title: Hadoop NoClassDefFoundError when adding external Jar
body: <p>I am trying to run a MapReduce job on Hadoop with an external Jar. I have added the Jar to the HDFS using <code>copyFromLocal</code> to <code>/user/hduser/lib/</code>. </p>
  
  <p>In my main method, I add the Jar to the DistributedCache. However, when I run the MapReduce program I receive a NoClassDefFoundError in the Mapper class. I have tried many of the solutions posted on SO from others who've had a similar error however I have not resolved the issue. Any guidance is appreciated.</p>
  
  <p>From Main Method:</p>
  
  <pre><code>Configuration conf = new Configuration();
  String jarToAdd1 = "/user/hduser/lib/joda-time-2.9.1-no-tzdb.jar";
  String jarToAdd2 = "/user/hduser/lib/joda-time-2.9.1.jar";
  addJarToDistributedCache(jarToAdd1, conf);
  addJarToDistributedCache(jarToAdd2, conf);
  .........
  </code></pre>
  
  <p>Adding to Distributed Cache:</p>
  
  <pre><code>private static void addJarToDistributedCache(String jarToAdd, Configuration conf) throws IOException {
  
          Path hdfsJar = new Path(jarToAdd);
          DistributedCache.addFileToClassPath(hdfsJar,conf);
      }
  </code></pre>
  
  <p>Mapper Where Error Occurs:</p>
  
  <pre><code>public static class Map1 extends Mapper&lt;LongWritable, Text, IntWritable, UserData&gt; {
  
  Map&lt;IntWritable, UserData&gt; userLog = new HashMap&lt;IntWritable, UserData&gt;();
  
  public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
  
      String line = value.toString();                             
      StringTokenizer tokenizer = new StringTokenizer(line);      
      DateTimeFormatter formatter = DateTimeFormat.forPattern("yyyyddmm HH:mm:ss");    // *********ERROR HAPPENS HERE **********
  </code></pre>
  
  <p>StackTrace:</p>
  
  <pre><code>16/01/30 20:45:15 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  16/01/30 20:45:16 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
  16/01/30 20:45:16 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
  16/01/30 20:45:16 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
  16/01/30 20:45:16 INFO input.FileInputFormat: Total input paths to process : 1
  16/01/30 20:45:16 INFO input.FileInputFormat: Total input paths to process : 1
  16/01/30 20:45:16 INFO mapreduce.JobSubmitter: number of splits:3
  16/01/30 20:45:17 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1787244132_0001
  16/01/30 20:45:17 INFO mapred.LocalDistributedCacheManager: Creating symlink: /app/hadoop/tmp/mapred/local/1454204717451/joda-time-2.9.1-no-tzdb.jar &lt;- /usr/local/hadoop/sbin/joda-time-2.9.1-no-tzdb.jar
  16/01/30 20:45:17 INFO mapred.LocalDistributedCacheManager: Localized hdfs://localhost:54310/user/hduser/lib/joda-time-2.9.1-no-tzdb.jar as file:/app/hadoop/tmp/mapred/local/1454204717451/joda-time-2.9.1-no-tzdb.jar
  16/01/30 20:45:17 INFO mapred.LocalDistributedCacheManager: Creating symlink: /app/hadoop/tmp/mapred/local/1454204717452/joda-time-2.9.1.jar &lt;- /usr/local/hadoop/sbin/joda-time-2.9.1.jar
  16/01/30 20:45:17 INFO mapred.LocalDistributedCacheManager: Localized hdfs://localhost:54310/user/hduser/lib/joda-time-2.9.1.jar as file:/app/hadoop/tmp/mapred/local/1454204717452/joda-time-2.9.1.jar
  16/01/30 20:45:17 INFO mapred.LocalDistributedCacheManager: file:/app/hadoop/tmp/mapred/local/1454204717451/joda-time-2.9.1-no-tzdb.jar
  16/01/30 20:45:17 INFO mapred.LocalDistributedCacheManager: file:/app/hadoop/tmp/mapred/local/1454204717452/joda-time-2.9.1.jar
  16/01/30 20:45:17 INFO mapreduce.Job: The url to track the job: http://localhost:8080/
  16/01/30 20:45:17 INFO mapreduce.Job: Running job: job_local1787244132_0001
  16/01/30 20:45:17 INFO mapred.LocalJobRunner: OutputCommitter set in config null
  16/01/30 20:45:18 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
  16/01/30 20:45:18 INFO mapred.LocalJobRunner: Waiting for map tasks
  16/01/30 20:45:18 INFO mapred.LocalJobRunner: Starting task: attempt_local1787244132_0001_m_000000_0
  16/01/30 20:45:18 INFO mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
  16/01/30 20:45:18 INFO mapred.MapTask: Processing split: hdfs://localhost:54310/user/hduser/input/sentimentFeedback7.csv:0+143748596
  16/01/30 20:45:18 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
  16/01/30 20:45:18 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100
  16/01/30 20:45:18 INFO mapred.MapTask: soft limit at 83886080
  16/01/30 20:45:18 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600
  16/01/30 20:45:18 INFO mapred.MapTask: kvstart = 26214396; length = 6553600
  16/01/30 20:45:18 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
  16/01/30 20:45:18 INFO mapred.MapTask: Starting flush of map output
  16/01/30 20:45:18 INFO mapred.LocalJobRunner: Starting task: attempt_local1787244132_0001_m_000001_0
  16/01/30 20:45:18 INFO mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
  16/01/30 20:45:18 INFO mapred.MapTask: Processing split: hdfs://localhost:54310/user/hduser/input/allergyConsumption7.csv:0+134217728
  16/01/30 20:45:18 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
  16/01/30 20:45:18 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100
  16/01/30 20:45:18 INFO mapred.MapTask: soft limit at 83886080
  16/01/30 20:45:18 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600
  16/01/30 20:45:18 INFO mapred.MapTask: kvstart = 26214396; length = 6553600
  16/01/30 20:45:18 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
  16/01/30 20:45:18 INFO mapred.MapTask: Starting flush of map output
  16/01/30 20:45:18 INFO mapred.LocalJobRunner: Starting task: attempt_local1787244132_0001_m_000002_0
  16/01/30 20:45:18 INFO mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
  16/01/30 20:45:18 INFO mapred.MapTask: Processing split: hdfs://localhost:54310/user/hduser/input/allergyConsumption7.csv:134217728+105486421
  16/01/30 20:45:18 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
  16/01/30 20:45:18 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100
  16/01/30 20:45:18 INFO mapred.MapTask: soft limit at 83886080
  16/01/30 20:45:18 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600
  16/01/30 20:45:18 INFO mapred.MapTask: kvstart = 26214396; length = 6553600
  16/01/30 20:45:18 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
  16/01/30 20:45:18 INFO mapred.MapTask: Starting flush of map output
  16/01/30 20:45:18 INFO mapred.LocalJobRunner: map task executor complete.
  16/01/30 20:45:18 WARN mapred.LocalJobRunner: job_local1787244132_0001
  java.lang.Exception: java.lang.NoClassDefFoundError: org/joda/time/format/DateTimeFormat
      at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
      at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
  Caused by: java.lang.NoClassDefFoundError: org/joda/time/format/DateTimeFormat
      at org.peach.fooddiary.FoodDiaryMR$Map1.map(FoodDiaryMR.java:45)
      at org.peach.fooddiary.FoodDiaryMR$Map1.map(FoodDiaryMR.java:1)
      at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
      at org.apache.hadoop.mapreduce.lib.input.DelegatingMapper.run(DelegatingMapper.java:55)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
      at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745)
  Caused by: java.lang.ClassNotFoundException: org.joda.time.format.DateTimeFormat
      at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      ... 12 more
  16/01/30 20:45:18 INFO mapreduce.Job: Job job_local1787244132_0001 running in uber mode : false
  16/01/30 20:45:18 INFO mapreduce.Job:  map 0% reduce 0%
  16/01/30 20:45:18 INFO mapreduce.Job: Job job_local1787244132_0001 failed with state FAILED due to: NA
  16/01/30 20:45:19 INFO mapreduce.Job: Counters: 0
  </code></pre>