title: Unhealthy node on the cluster
body: <p>What are all reasons due to which a node on cluster goes in unhealthy state?</p>
  
  <p>Based on my limited understanding it generally happens when the HDFS utilization on the given node goes beyond a threshold value. This threshold value is defined with max-disk-utilization-per-disk-percentage property. </p>
  
  <p>I have observed at times when a memory intensive spark job is triggered on spark-sql or using pyspark nodes go to unhealthy state. Upon further looking I did ssh on the node that was in unhealthy state and discovered that actually dfs utilization was less that 75% and the value that was set for the above mentioned property was 99 on my cluster.</p>
  
  <p>So I presume there is some other fact that I am missing which basically causes this behavior.</p>
  
  <p>Thanks in advance for your help.</p>
  
  <p>Manish Mehra</p>