title: Apache Spark: broadcast join behaviour: filtering of joined tables and temp tables
body: <p>I need to join 2 tables in spark.
  But instead of joining 2 tables completely, I first filter out a part of second table:</p>
  <pre><code>spark.sql(&quot;select * from a join b on a.key=b.key where b.value='xxx' &quot;)
  </code></pre>
  <p>I want to use broadcast join in this case.</p>
  <p>Spark has a parameter which defines max table size for broadcast join: <em><strong><code>spark.sql.autoBroadcastJoinThreshold</code></strong></em>:</p>
  <blockquote>
  <p>Configures the maximum size in bytes for a table that will be
  broadcast to all worker nodes when performing a join. By setting this
  value to -1 broadcasting can be disabled. Note that currently
  statistics are only supported for Hive Metastore tables where the
  command ANALYZE TABLE  COMPUTE STATISTICS noscan has been
  run. <a href="http://spark.apache.org/docs/2.4.0/sql-performance-tuning.html" rel="nofollow noreferrer">http://spark.apache.org/docs/2.4.0/sql-performance-tuning.html</a></p>
  </blockquote>
  <p>I have following questions about this setup:</p>
  <ol>
  <li>which table size spark will compare with autoBroadcastJoinThreshold's value: FULL size, or size AFTER applying where clause?</li>
  <li>I am assuming that spark will apply <em>where</em> clause BEFORE broadcasting, correct?</li>
  <li>the doc says I need to run Hive's Analyze Table command beforehand. How it will work in a case when I am using temp view as a table? As far as I understand I cannot run Analyze Table command against spark's temp view created via dataFrame.createorReplaceTempView(&quot;b&quot;). Can I broadcast temp view contents?</li>
  </ol>