title: Spark on AWS EMR cluster Master node 100% CPU utilization
body: <p>I am using Apache-Spark on AWS-EMR Cluster (1 Master 4 Slaves - r4.xlarge instances). </p>
  
  <p>My jobs are compiled and packaged as a jar in my Master node. Spark-submit is done from the master node referring to the local jar path in which the jobs are compiled. With Spark History Server, I find that the jobs are split across the data/task nodes for computation.</p>
  
  <p>But after a certain point in time, the master node's CPU utilization peaks to 100% while the data node's CPU utilization are at 25-40%.
  Even when no computation is done, my Master node's CPU utilization is around 40%.</p>
  
  <p>In this link, <a href="https://forums.aws.amazon.com/message.jspa?messageID=687581" rel="nofollow noreferrer">Spark History Server</a> is discussed as a root cause. But clearing the application logs dint work for me. </p>
  
  <ol>
  <li>Does placing the jar in master node cause the issue? </li>
  <li>Should it be placed Distributed Cache?</li>
  </ol>