title: KafkaStreams application reprocessing from start after broker failures
body: <p>I have a 3-broker Docker Kafka AWS cluster, and a 3-container KafkaStreams application processing from an "events" topic with replication=2.</p>
  
  <p>I had the failure written by other user in <a href="https://stackoverflow.com/questions/46644764/kafka-cached-zkversion-not-equal-to-that-in-zookeeper-broker-not-recovering">this question</a> which later brought me to make <a href="https://stackoverflow.com/questions/50723435/kafka-streams-consumer-groups-concurrency-consuming-from-one-partition-only">this question</a>. After stopping the the KafkaStreams application, I restarted failed brokers, one on a Friday (which took like an hour) and the other one was restarted next Monday. </p>
  
  <p>When brokers joined the cluster apparently OK(showing the "failed, I restarted application.
  <code>__customer_offsets.sh</code> had replication=3, so they should be safe. </p>
  
  <p>(For reference, cluster situation is better described in <a href="https://stackoverflow.com/questions/50723435/kafka-streams-consumer-groups-concurrency-consuming-from-one-partition-only">this question</a> ). </p>
  
  <p>My application does some dispatching so the sum of output topics offsets should equal the sum of original events). Whatever does not come there, it goes to an <code>errors</code> topic in a catch-all-like fashion, so they always match.</p>
  
  <p>However, when restarting, the application began processing from the beginning, having almost doubled the number of processed messages(when the failures happened, there was some unprocessed lag). </p>
  
  <p>If the <code>__consumer_offsets</code> were not lost (and they weren't apparently - they just showed underreplicated but living in the broker that was fine during all the process), why would the application start processing from zero? Under which circumstances is this expected - or may this happen? </p>