title: Why the near constant execution time when increasing workers of Spark standalone
body: <p>I'm told the recommended number of workers to set is one per core when using Spark. I am not getting a degradation in performance though when the number of workers is well above the cores on my computer. Why could this be?</p>
  
  <pre class="lang-py prettyprint-override"><code>from time import time
  from pyspark import SparkContext
  
  for j in range(1, 10):
      sc = SparkContext(master="local[%d]" % (j))
      t0 = time()
      for i in range(10):
          sc.parallelize([1, 2] * 1000000).reduce(lambda x, y: x + y)
      print("%2d executors, time=%4.3f" % (j, time() - t0))
      sc.stop()
  
  # 1 executors time=6.112
  # 2 executors time=5.202
  # 3 executors time=4.695
  # 4 executors time=5.090
  # 5 executors time=5.262
  # 6 executors time=5.156
  # 7 executors time=5.274
  # 8 executors time=5.376
  # 9 executors time=5.124
  </code></pre>
  
  <p>Hardware specs:</p>
  
  <ul>
  <li>MacBook Pro</li>
  <li>Intel Core i5, 2.3Ghz</li>
  <li>Total number of cores 2</li>
  <li>L2 Cache 256 KB</li>
  <li>L3 Cache 4 MB</li>
  <li>Memory 16 GB</li>
  </ul>