title: Avoid re-consumption with Apache Kafka MirrorMaker 2 in an active-active configuration?
body: <p>I am running Apache MirrorMaker 2.7 on multiple active Kafka (2.6) clusters (named <code>prod1</code>, <code>prod2</code>). So <code>topic</code> on <code>prod1</code> is replicated by MirrorMaker2 as <code>prod1.topic</code> on the <code>prod2</code> cluster.</p>
  <p>I have a Kafka Consumer service running on both <code>prod1</code> and <code>prod2</code> using the same kafka consumer group-id.</p>
  <p>I have <code>emit.checkpoints.interval.seconds=1</code> in my MirrorMaker2 config, resulting in offsets being translated every second from
  <code>topic</code> to <code>prod1.topic</code>.</p>
  <p>The problem is, everytime my producer produces to <code>topic</code> on <code>prod1</code>, the data is replicated over to <code>prod1.topic</code> on <code>prod2</code>, and consumed by my consumer on <code>prod2</code> BEFORE the latest-commit offset from my <code>prod1</code> consumer is translated over to <code>prod2</code> by MirrorMaker2.</p>
  <p>This does not happen if I start my <code>prod2</code> consumer a few seconds after data is produced to <code>prod1</code>, as latest-commit offsets would have arrived on <code>prod2</code> by then, however, both my consumers on <code>prod1/2</code> need to be running live as I am using <code>prod1/2</code> as an active-active deployment configuration.</p>
  <p>So how could I ensure any data produced is consumed only once, by either of the <code>prod1</code>/ <code>prod2</code> consumers?</p>