title: Total size of serialized results of 16 tasks (1048.5 MB) is bigger than spark.driver.maxResultSize (1024.0 MB)
body: <p>I get the following error when I add <code>--conf spark.driver.maxResultSize=2050</code> to my <code>spark-submit</code> command.</p>
  
  <pre><code>17/12/27 18:33:19 ERROR TransportResponseHandler: Still have 1 requests outstanding when connection from /XXX.XX.XXX.XX:36245 is closed
  17/12/27 18:33:19 WARN Executor: Issue communicating with driver in heartbeater
  org.apache.spark.SparkException: Exception thrown in awaitResult:
          at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:205)
          at org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:75)
          at org.apache.spark.rpc.RpcEndpointRef.askSync(RpcEndpointRef.scala:92)
          at org.apache.spark.executor.Executor.org$apache$spark$executor$Executor$$reportHeartBeat(Executor.scala:726)
          at org.apache.spark.executor.Executor$$anon$2$$anonfun$run$1.apply$mcV$sp(Executor.scala:755)
          at org.apache.spark.executor.Executor$$anon$2$$anonfun$run$1.apply(Executor.scala:755)
          at org.apache.spark.executor.Executor$$anon$2$$anonfun$run$1.apply(Executor.scala:755)
          at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1954)
          at org.apache.spark.executor.Executor$$anon$2.run(Executor.scala:755)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)
  Caused by: java.io.IOException: Connection from /XXX.XX.XXX.XX:36245 closed
          at org.apache.spark.network.client.TransportResponseHandler.channelInactive(TransportResponseHandler.java:146)
  </code></pre>
  
  <p>The reason of adding this configuration was the error:</p>
  
  <pre><code>py4j.protocol.Py4JJavaError: An error occurred while calling o171.collectToPython.
  : org.apache.spark.SparkException: Job aborted due to stage failure: Total size of serialized results of 16 tasks (1048.5 MB) is bigger than spark.driver.maxResultSize (1024.0 MB)
  </code></pre>
  
  <p>Therefore, I increased <code>maxResultSize</code> to 2.5 Gb, but the Spark job fails anyway (the error shown above).
  How to solve this issue?</p>