title: Apache Spark: cores vs. executors
body: <p>Questions related to cores vs executors are asked number of times on SO.</p>
  
  <p><a href="https://stackoverflow.com/questions/24622108/apache-spark-the-number-of-cores-vs-the-number-of-executors">Apache Spark: The number of cores vs. the number of executors</a></p>
  
  <p>As each case is different, I'm asking similar question again.</p>
  
  <p>I'm running a cpu intensive application with same number of cores with different executors. Below are the observations.</p>
  
  <p><strong>Resource Manager :</strong> Kubernetes</p>
  
  <p><strong>Case 1:</strong> Executors - 6, Number of cores for each executor -2, Executor Memory - 3g, Amount of Data processing ~ 10GB, Partitions -36, Job Duration : <strong>75 mins</strong></p>
  
  <p><strong>Case 2:</strong> Executors - 4, Number of cores for each executor -3, Executor Memory - 3g, Amount of Data processing ~ 10GB, Partitions -36, Job Duration : <strong>101 mins</strong></p>
  
  <p>As per the above link anything less than 5 cores per executor is good for IO operations.</p>
  
  <p>In both my cases cores are same(12), however both jobs took different times. Any thoughts?</p>
  
  <p><strong>Updated</strong></p>
  
  <p><strong>Case 3:</strong> Executors - 12, Number of cores for each executor -1, Executor Memory - 3g, Amount of Data processing ~ 10GB, Partitions -36, Job Duration : <strong>81 mins</strong></p>