title: Spark sql "Futures timed out after 300 seconds" when filtering
body: <p>I get an exception when doing what seems to be simple spark sql filtering job:</p>
  
  <pre><code>    someOtherDF
        .filter(/*somecondition*/)
        .select($"eventId")
        .createOrReplaceTempView("myTempTable")
  
      records
        .filter(s"eventId NOT IN (SELECT eventId FROM myTempTable)")
  </code></pre>
  
  <p>Any idea how I can solve this ?</p>
  
  <p>Note: </p>
  
  <ul>
  <li>someOtherDF contains between ~1M and 5M rows after filtering and eventId are guids.</li>
  <li>records contains between 40M and 50M rows.</li>
  </ul>
  
  <p>Error:</p>
  
  <pre><code>Stacktrace:
  
  org.apache.spark.SparkException: Exception thrown in awaitResult:
          at org.apache.spark.util.ThreadUtils$.awaitResultInForkJoinSafely(ThreadUtils.scala:215)
          at org.apache.spark.sql.execution.exchange.BroadcastExchangeExec.doExecuteBroadcast(BroadcastExchangeExec.scala:131)
          at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeBroadcast$1.apply(SparkPlan.scala:124)
          at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeBroadcast$1.apply(SparkPlan.scala:124)
          at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:135)
          at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
          at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:132)
          at org.apache.spark.sql.execution.SparkPlan.executeBroadcast(SparkPlan.scala:123)
          at org.apache.spark.sql.execution.joins.BroadcastNestedLoopJoinExec.doExecute(BroadcastNestedLoopJoinExec.scala:343)
          at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:114)
          at ...
  Caused by: java.util.concurrent.TimeoutException: Futures timed out after [300 seconds]
      at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
      at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:223)
      at org.apache.spark.util.ThreadUtils$.awaitResultInForkJoinSafely(ThreadUtils.scala:212)
      ... 84 more
  </code></pre>