title: Running Apache Hadoop 2.1.0 on Windows
body: <p>I am new to Hadoop and have run into problems trying to run it on my Windows 7 machine. Particularly I am interested in running Hadoop 2.1.0 as its <a href="http://hadoop.apache.org/releases.html#25+August%2C+2013%3A+Release+2.1.0-beta+available">release notes</a> mention that running on Windows is supported. I know that I can try to run 1.x versions on Windows with Cygwin or even use prepared VM by for example Cloudera, but these options are in some reasons less convenient for me.</p>
  
  <p>Having examined a tarball from <a href="http://apache-mirror.rbc.ru/pub/apache/hadoop/common/hadoop-2.1.0-beta/">http://apache-mirror.rbc.ru/pub/apache/hadoop/common/hadoop-2.1.0-beta/</a> I found that there really are some *.cmd scripts that can be run without Cygwin. Everything worked fine when I formated HDFS partition but when I tried to run hdfs namenode daemon I faced two errors: first, non fatal, was that winutils.exe could not be found (it really wasn't present in the tarball downloaded). I found the sources of this component in the Apache Hadoop sources tree and compiled it with Microsoft SDK and MSbuild. Thanks to detailed error message it was clear where to put the executable to satisfy Hadoop. But the second error which is fatal doesn't contain enough information for me to solve:</p>
  
  <pre><code>13/09/05 10:20:09 FATAL namenode.NameNode: Exception in namenode join
  java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z
      at org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Native Method)
      at org.apache.hadoop.io.nativeio.NativeIO$Windows.access(NativeIO.java:423)
      at org.apache.hadoop.fs.FileUtil.canWrite(FileUtil.java:952)
      at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.analyzeStorage(Storage.java:451)
      at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs(FSImage.java:282)
      at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:200)
  ...
  13/09/05 10:20:09 INFO util.ExitUtil: Exiting with status 1
  </code></pre>
  
  <p>Looks like something else should be compiled. I'm going to try to build Hadoop from the source with Maven but isn't there a simpler way? Isn't there some option-I-know-not-of that can disable native code and make that tarball usable on Windows?</p>
  
  <p>Thank you.</p>
  
  <p>UPDATED. Yes, indeed. "Homebrew" package contained some extra files, most importantly winutils.exe and hadoop.dll. With this files namenode and datanode started successfully. I think the question can be closed. I didn't delete it in case someone face the same difficulty.</p>
  
  <p>UPDATED 2. To build the "homebrew" package I did the following:</p>
  
  <ol>
  <li>Got sources, and unpacked them.</li>
  <li>Read carefully BUILDING.txt.</li>
  <li>Installed dependencies:<br>
  3a) Windows SDK 7.1<br>
  3b) Maven (I used 3.0.5)
  3c) JDK (I used 1.7.25)<br>
  3d) ProtocolBuffer (I used 2.5.0 - <a href="http://protobuf.googlecode.com/files/protoc-2.5.0-win32.zip">http://protobuf.googlecode.com/files/protoc-2.5.0-win32.zip</a>). It is enough just to put compiler (protoc.exe) into some of the PATH folders.<br>
  3e) A set of UNIX command line tools (I installed Cygwin)  </li>
  <li>Started command line of Windows SDK. Start | All programs | Microsoft Windows SDK v7.1 | ... Command Prompt (I modified this shortcut, adding option /release in the command line to build release versions of native code). All the next steps are made from inside SDK command line window)</li>
  <li><p>Set up the environment:</p>
  
  <p>set JAVA_HOME={path_to_JDK_root}  </p></li>
  </ol>
  
  <p>It seems that JAVA_HOME <em>MUST NOT</em> contain space!  </p>
  
  <pre><code>set PATH={path_to_maven_bin};%PATH%  
  set Platform=x64  
  set PATH={path_to_cygwin_bin};%PATH%  
  set PATH={path_to_protoc.exe};%PATH%  
  </code></pre>
  
  <ol>
  <li>Changed dir to sources root folder (BUILDING.txt warns that there are some limitations on the path length so sources root should have short name - I used D:\hds)</li>
  <li><p>Ran building process:</p>
  
  <p>mvn package -Pdist -DskipTests  </p></li>
  </ol>
  
  <p>You can try without 'skipTests' but on my machine some tests failed and building was terminated. It may be connected to sybolic link issues mentioned in BUILDING .txt.
  8. Picked the result in hadoop-dist\target\hadoop-2.1.0-beta (windows executables and dlls are in 'bin' folder) </p>