title: How to use SparkSession and StreamingContext together?
body: <p>I'm trying to stream CSV files from a folder on my local machine (OSX). I'm using SparkSession and StreamingContext together like so:</p>
  
  <pre><code>val sc: SparkContext = createSparkContext(sparkContextName)
  val sparkSess = SparkSession.builder().config(sc.getConf).getOrCreate()
  val ssc = new StreamingContext(sparkSess.sparkContext, Seconds(time))
  
  val csvSchema = new StructType().add("field_name",StringType)
  val inputDF = sparkSess.readStream.format("org.apache.spark.csv").schema(csvSchema).csv("file:///Users/userName/Documents/Notes/MoreNotes/tmpFolder/")
  </code></pre>
  
  <p>If I run <code>ssc.start()</code> after this, I get this error:</p>
  
  <pre><code>java.lang.IllegalArgumentException: requirement failed: No output operations registered, so nothing to execute
  </code></pre>
  
  <p>Instead if I try to start the <code>SparkSession</code> like this:</p>
  
  <pre><code>inputDF.writeStream.format("console").start()
  </code></pre>
  
  <p>I get:</p>
  
  <pre><code>java.lang.IllegalStateException: Cannot call methods on a stopped SparkContext.
  </code></pre>
  
  <p>Clearly I'm not understanding how <code>SparkSession</code> and <code>StreamingContext</code> should work together. If I get rid of <code>SparkSession</code>, <code>StreamingContext</code> only has <code>textFileStream</code> on which I need to impose a CSV schema. Would appreciate any clarifications on how to get this working.</p>