title: What is the real difference between Append mode and Update mode in Spark Streaming?
body: <p>What is the real difference between Append mode and Update mode in Spark Streaming?</p>
  
  <p>According to the documentation:</p>
  
  <blockquote>
    <p>Append mode (default) - This is the default mode, where only the new
    rows added to the Result Table since the last trigger will be
    outputted to the sink. This is supported for only those queries where
    rows added to the Result Table is never going to change. Hence, this
    mode guarantees that each row will be output only once (assuming
    fault-tolerant sink). For example, queries with only select, where,
    map, flatMap, filter, join, etc. will support Append mode.</p>
  </blockquote>
  
  <p>and </p>
  
  <blockquote>
    <p>Update mode - (Available since Spark 2.1.1) Only the rows in the
    Result Table that were updated since the last trigger will be
    outputted to the sink. More information to be added in future
    releases.</p>
  </blockquote>
  
  <p><strong>My confusion with append mode</strong>: It says where <strong>"only"</strong> the new rows added to the Result Table since the last trigger will be outputted to the sink. so, for example, say we have three rows</p>
  
  <p><code>r1, r2, r3</code> arrived at <code>t1, t2, t3</code> where <code>t1&lt;t2&lt;t3</code> </p>
  
  <p>Now say at t4 the row r2 got overwritten and if so, we will never see that in the sink while we are operating in append mode? Isn't it like losing a write?</p>
  
  <p><strong>My confusion with update mode</strong>: It says <strong>"only"</strong> the rows in the result table that were updated since the last trigger will be outputted to the sink. Does it imply that rows should already exist and only if the existing rows are updated it will output to sink? what happens if there are no existing rows and a new row comes in while we are in this update mode?  </p>