title: Why does Spark job fail on Mesos with "hadoop: not found"?
body: <p>I use Spark 1.6.1, Hadoop 2.6.4 and Mesos 0.28 on Debian 8.</p>
  
  <p>While trying to submit a job via <code>spark-submit</code> to a Mesos cluster a slave fails with the following in stderr log:</p>
  
  <pre><code>I0427 22:35:39.626055 48258 fetcher.cpp:424] Fetcher Info: {"cache_directory":"\/tmp\/mesos\/fetch\/slaves\/ad642fcf-9951-42ad-8f86-cc4f5a5cb408-S0\/hduser","items":[{"action":"BYP$
  I0427 22:35:39.628031 48258 fetcher.cpp:379] Fetching URI 'hdfs://xxxxxxxxx:54310/sources/spark/SimpleEventCounter.jar'
  I0427 22:35:39.628057 48258 fetcher.cpp:250] Fetching directly into the sandbox directory
  I0427 22:35:39.628078 48258 fetcher.cpp:187] Fetching URI 'hdfs://xxxxxxx:54310/sources/spark/SimpleEventCounter.jar'
  E0427 22:35:39.629243 48258 shell.hpp:93] Command 'hadoop version 2&gt;&amp;1' failed; this is the output:
  sh: 1: hadoop: not found
  Failed to fetch 'hdfs://xxxxxxx:54310/sources/spark/SimpleEventCounter.jar': Failed to create HDFS client: Failed to execute 'hadoop version 2&gt;&amp;1'; the command was e$
  Failed to synchronize with slave (it's probably exited)
  </code></pre>
  
  <ol>
  <li>My Jar file contains hadoop 2.6 binaries</li>
  <li>The path to spark executor/binary is via an <code>hdfs://</code> link </li>
  </ol>
  
  <p>My jobs don't appear in the framework tab, but they do appear in the driver with the status 'queued' and they just sit there till I shut down the <code>spark-mesos-dispatcher.sh</code> service.</p>