title: Why memory sink writes nothing in append mode?
body: <p>I used Spark's structured streaming to stream messages from Kafka. The data was then aggregated, and wrote to a memory sink with append mode. However, when I tried to query the memory, it returned nothing. Below are the code:</p>
  
  <pre><code>result  =    model
              .withColumn("timeStamp", col("startTimeStamp").cast("timestamp"))
              .withWatermark("timeStamp", "5 minutes")
              .groupBy(window(col("timeStamp"), "5 minutes").alias("window"))
              .agg(
                  count("*").alias("total")
              );
  
      // writing to memory
      StreamingQuery query = result.writeStream()
                  .outputMode(OutputMode.Append())
                  .queryName("datatable")
                  .format("memory")
                  .start();
  
      // query data in memory
      new Timer().scheduleAtFixedRate(new TimerTask() {
          @Override
          public void run() {
              sparkSession.sql("SELECT * FROM datatable").show();
          }
      }, 10000, 10000);
  </code></pre>
  
  <p>The result is always:</p>
  
  <pre><code>|window|total|
  +------+-----+
  +------+-----+
  </code></pre>
  
  <p>If I used <code>outputMode = complete</code>, then I could get the aggregated data. But that's not my choice as the requirement is to use append mode.</p>
  
  <p>Is there any problem with the code?
  Thanks,</p>