title: Recover Hadoop NameNode Failure
body: <p>Scenario 1:</p>
  
  <p>The HDFS fsimage and editlog is written into multiple places including a NFS mount.</p>
  
  <p>A) NameNode Daemon Crash :
  Solution:
  Just restart the Namenode process </p>
  
  <p>B) Host is Down where the Name Node is running.</p>
  
  <p>Solution:</p>
  
  <ol>
  <li>Start the namenode in a different host with a empty dfs.name.dir</li>
  <li>Point  the dfs.name.dir to the NFS mount where we have copy of the meta data. 
  OR</li>
  <li>Use --importCheckpoint option while starting namenode after pointing fs.checkpoint.dir to checkpoint directory from Secondary NameNode</li>
  <li>Change  the fs.default.name to the backup host name URI  and restart the cluster with all the slave IP's in slaves file. </li>
  </ol>
  
  <p>Note - We may miss the edit that might have happened after the last checkpoint.</p>
  
  <p>Scenario 2: </p>
  
  <p>The HDFS fsimage  is written into a single directory.</p>
  
  <p>A ) NameNode Daemon Crash:
          Solution : Unknown </p>
  
  <p>B ) Host is down where the Name Node is running.</p>
  
  <p>Solution:</p>
  
  <ol>
  <li>Create a blank directory  pointing to dfs.name.dir to directory in (1)</li>
  <li>Start the Namenode with -importCheckpoint after pointing fs.checkpoint.dir to checkpoint directory from Secondary NameNode</li>
  <li>Change  the fs.default.name to the backup host name URI  and restart the cluster with all the slave IP's in slaves file. </li>
  </ol>
  
  <p>This way we would miss again the files edited after last checkpoint.</p>
  
  <p>Please let me know if this is how we can manually recover the cluster. </p>