title: MSK Not Deleting Old Messages
body: <p>I have three MSK clusters; dev, nonprod &amp; prod. They all have the following cluster configuration - there is no topic level configuration.</p>
  <pre><code>auto.create.topics.enable=false
  default.replication.factor=3
  min.insync.replicas=2
  num.io.threads=8
  num.network.threads=5
  num.partitions=1
  num.replica.fetchers=2
  log.retention.hours=100
  replica.lag.time.max.ms=30000
  socket.receive.buffer.bytes=102400
  socket.request.max.bytes=104857600
  socket.send.buffer.bytes=102400
  unclean.leader.election.enable=true
  zookeeper.session.timeout.ms=18000
  </code></pre>
  <p>Dev and Nonprod are clearing down messages older than 100 hours as defined in the <code>log.retention.hours=100</code> setting.</p>
  <p>We have a lot more traffic coming through our production cluster and old messages are not being removed. <strong>We have hundreds of thousands of messages older than 400 hours still on the cluster.</strong> I have thought about adding further config settings such as</p>
  <pre><code>segment.bytes
  segment.ms
  </code></pre>
  <p>To roll the segments quicker as maybe a segment hasn't rolled yet and can't be marked for deletion - however this same config is working nicely in the other clusters albeit not receiving as much traffic.</p>