title: How to fix kafka.common.errors.TimeoutException: Expiring 1 record(s) xxx ms has passed since batch creation plus linger time
body: <p>I am using kafka_2.11-2.1.1
  and Producer using spring 2.1.0.RELEASE.</p>
  
  <p>I am using spring while I am sending the messages to Kafka topic my producer generates a lot of <code>TimeoutExceptions</code></p>
  
  <pre><code>org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for COMPANY_INBOUND--19: 229 ms has passed since batch creation plus linger time
  </code></pre>
  
  <p>I am using below kafka producer settings</p>
  
  <pre><code>acks: 1
  retries: 1
  batchSize: 100
  lingerMs: 5
  bufferMemory: 33554432
  requestTimeoutMs: 60
  </code></pre>
  
  <p>I tried many combinations (specially <code>batchSize</code> &amp; <code>lingerMs</code>)  but nothing worked. Any help please what should be the setting for above scenario.</p>
  
  <p>Tried again with below configs ...but no luck same error</p>
  
  <pre><code>acks = 1
      batch.size = 15
      buffer.memory = 33554432
      client.id = 
      compression.type = none
      connections.max.idle.ms = 540000
      enable.idempotence = false
      interceptor.classes = []
      key.serializer = class org.apache.kafka.common.serialization.StringSerializer
      linger.ms = 0
      max.block.ms = 60000
      max.in.flight.requests.per.connection = 5
      max.request.size = 1048576
      metadata.max.age.ms = 300000
      metric.reporters = []
      metrics.num.samples = 2
      metrics.recording.level = INFO
      metrics.sample.window.ms = 30000
      partitioner.class = class com.spgmi.ca.prescore.partition.CompanyInfoPartitioner
      receive.buffer.bytes = 32768
      reconnect.backoff.max.ms = 1000
      reconnect.backoff.ms = 50
      request.timeout.ms = 120
      retries = 1
  </code></pre>
  
  <p><strong>Second Time Run :</strong></p>
  
  <p>I treid different combinations nothing worked.
  Hence i thought it would be problem with network , SSL etc.
  So I installed and run the Kafka on the same machine where producer is running i.e. in my local computer.</p>
  
  <p>I tried to run the producer again pointing to the local Kafka topics.
  But no luck same issue.</p>
  
  <p>Below are configuration params used.</p>
  
  <pre><code>2019-07-02 05:55:36.663  INFO 9224 --- [lt-dispatcher-2] o.a.k.clients.producer.ProducerConfig    : ProducerConfig values: 
      acks = 1
      batch.size = 0
      bootstrap.servers = [localhost:9092]
      request.timeout.ms = 60
      retries = 1
      buffer.memory = 33554432
      linger.ms = 0
      client.id = 
      compression.type = none
      connections.max.idle.ms = 540000
      enable.idempotence = false
      interceptor.classes = []
      max.block.ms = 60000
      max.in.flight.requests.per.connection = 5
      max.request.size = 1048576
      metadata.max.age.ms = 300000
      metric.reporters = []
      metrics.num.samples = 2
      metrics.recording.level = INFO
      metrics.sample.window.ms = 30000
      receive.buffer.bytes = 32768
      reconnect.backoff.max.ms = 1000
      reconnect.backoff.ms = 50
      retry.backoff.ms = 100
      sasl.client.callback.handler.class = null
      sasl.jaas.config = null
      sasl.kerberos.kinit.cmd = /usr/bin/kinit
      sasl.kerberos.min.time.before.relogin = 60000
      sasl.kerberos.service.name = null
      sasl.kerberos.ticket.renew.jitter = 0.05
      sasl.kerberos.ticket.renew.window.factor = 0.8
      sasl.login.callback.handler.class = null
      sasl.login.class = null
      sasl.login.refresh.buffer.seconds = 300
      sasl.login.refresh.min.period.seconds = 60
      sasl.login.refresh.window.factor = 0.8
      sasl.login.refresh.window.jitter = 0.05
      sasl.mechanism = GSSAPI
      security.protocol = PLAINTEXT
      send.buffer.bytes = 131072
      ssl.cipher.suites = null
      ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
      ssl.endpoint.identification.algorithm = https
      ssl.key.password = null
      ssl.keymanager.algorithm = SunX509
      ssl.keystore.location = null
      ssl.keystore.password = null
      ssl.keystore.type = JKS
      ssl.protocol = TLS
      ssl.provider = null
      ssl.secure.random.implementation = null
      ssl.trustmanager.algorithm = PKIX
      ssl.truststore.location = null
      ssl.truststore.password = null
      ssl.truststore.type = JKS
      transaction.timeout.ms = 60000
      transactional.id = null
  </code></pre>
  
  <p><strong>Facing same error :</strong>
  org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for inbound_topic--1: 69 ms has passed since batch creation plus linger time</p>
  
  <p>Also tried
  batch.size 5 , 10 &amp; 0
  linger_ms  0 , 5 , 10 etc.
  request_time_out  0 , 45, 60, 120 , 300 etc.</p>
  
  <p>Nothing working ...same error.</p>
  
  <p>What else should I try , what could be the solution ?</p>
  
  <p><strong>How to avoid negative key generation</strong> </p>
  
  <p>Yeah I set up local set up and print the log with partition info which shows as below</p>
  
  <p>2019-07-03 02:48:28.822  INFO 7092 --- [lt-dispatcher-2] c.s.c.p.p.CompanyInfoPartitioner         : Topic : inbound_topic  Key = 597736248- Entropy Cayman Solar Ltd.-null-null-null <strong>Partition = -1</strong>
  2019-07-03 02:48:28.931 ERROR 7092 --- [ad | producer-1] o.s.k.support.LoggingProducerListener    : Exception thrown when sending a message with key='597736248- Entropy Cayman Solar Ltd.-null-null-null' and payload='com.spgmi.ca.prescore.model.Company@8b12343' to topic inbound_topic  :</p>
  
  <p>org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for  inbound_topic  --1: 104 ms has passed since batch creation plus linger time</p>
  
  <p>My topics inbound_topic has two partitions as you see below
  C:\Software\kafka\kafka_2.11-2.1.1\bin\windows>kafka-topics.bat  --describe --zookeeper localhost:2181 --topic inbound_topic
  Topic:inbound_topic     PartitionCount:2        ReplicationFactor:1     Configs:
          Topic: inbound_topic    Partition: 0    Leader: 0       Replicas: 0     Isr: 0
          Topic: inbound_topic    Partition: 1    Leader: 0       Replicas: 0     Isr: 0</p>
  
  <p><em>But my producer seems to trying to send to Partition = -1.</em></p>
  
  <p>My partition logic is as below </p>
  
  <pre><code>int p = (((String)key).hashCode() * Integer.MAX_VALUE) % numPartitions;
          logger.info("Topic : "+ topic + "\t Key = " + (String)key + " Partition = " + p );
  </code></pre>
  
  <p>On key i am doing hashCode(). What need to be corrected here to avoid  this negative number partition number ? i.e. Partition = -1 </p>
  
  <p>What should be my partition key logic like ?</p>
  
  <p>anyhelp highly appriciated.</p>