title: Kafka consumer groups and partitions
body: <p>I'm having trouble to grasp the relationship behind partitions and customer groups.</p>
  
  <p>The ideas by themselves are pretty clears, each message that's pushed to a topic gets replicated to all of it's partitions, right?
  That way, if two different clients connect to two different partitions of the same topic then they should consume and commit the same messages without interrupting each other.</p>
  
  <p>Consumer groups, as i understand, are abstractions to the idea of partitions and they essentially promise the same thing, two different clients that connect to two different consumer groups of the same topic should consume and commit the same messages without interrupting each other.</p>
  
  <p>So as i see it, it should follow that two clients that connect to the same consumer group would consume messages from the same partition and two clients that connect to two different consumer groups would consume from two different partitions (given that there are at least two partitions for that topic) because otherwise the idea of consumer groups doesn't comply with the idea of partitions.</p>
  
  <p>However, when i run a simple consumer client in C#</p>
  
  <pre><code>string group = Console.ReadLine();
  
              var config = new Dictionary&lt;string, object&gt;()
              {
                  { "group.id", group },
                  { "bootstrap.servers", "10.0.0.3:9092" },
                  { "enable.auto.commit", true },
                  { "auto.commit.interval.ms", 1000 }
              };
  
              using (var consumer = new Consumer&lt;Null, string&gt;(config, null, new StringDeserializer(Encoding.UTF8)))
              {
                  consumer.Subscribe(new List&lt;string&gt;() { { "myFirstTopic" } });
                  while (true)
                  {
                      Message&lt;Null, string&gt; msg;
                      if (!consumer.Consume(out msg, TimeSpan.FromMilliseconds(100)))
                      {
                          continue;
                      }
  
                      Console.WriteLine($"Topic: {msg.Topic} Partition: {msg.Partition} Offset: {msg.Offset} {msg.Value}");
                  }
              }
  </code></pre>
  
  <p>I get this result:</p>
  
  <p><a href="https://i.stack.imgur.com/CGad6.png" rel="noreferrer"><img src="https://i.stack.imgur.com/CGad6.png" alt="enter image description here"></a></p>
  
  <p>The same consumer group consumes from 2 different partitions.
  When i run two clients that consume from different consumer groups (a and b) i get this:</p>
  
  <p><a href="https://i.stack.imgur.com/QA7Fc.png" rel="noreferrer"><img src="https://i.stack.imgur.com/QA7Fc.png" alt="enter image description here"></a></p>
  
  <p>Two different consumer groups consume from similar partitions.</p>
  
  <p>I don't understand how it happens, doesn't it mean that the idea of consumer groups and the idea of partitions contradict one another?</p>
  
  <p>If the same message appears in two different consumer groups under the same partition, doesn't it mean that the same message was inserted twice to the same partition?</p>
  
  <p>Please help me understand.</p>