title: How to handle missing columns when processing flexible-schema json in scala spark?
body: <p>Using spark streaming to read from kafka  messages where V = Json event.</p>
  
  <p>The Json's schema isn't enforced, so you might fetch the following 2 events:</p>
  
  <pre><code>{"foo": "01/01/1973","bar": "d,e"}
  {"foo": "01/01/1974"}
  </code></pre>
  
  <p>using <code>read.json</code> and <code>df.registerTempTable("tempTable")</code>
  How do I create a <code>query = "select foo, bar from tempTable"</code> that plugs a NULL in case the field doesn't exist?</p>
  
  <p>*Using spark 1.6, but would be also happy to hear about spark2</p>