title: Kafka connect raised an error: ERROR Error converting event to JSON
body: <p>This is the <code>docker-compose.yml</code> (This the code is the snippet of the compose file (just <code>kafka</code>, <code>kafka</code>, <code>connect</code> and <code>redis</code>) from this <a href="https://github.com/udacity/nd029-c2-apache-spark-and-spark-streaming-starter/blob/master/docker-compose.yaml" rel="nofollow noreferrer">repo</a>):</p>
  <pre><code>
  version: '3.7'
  
  services:
    redis:
      image: redis:6.0.6
      ports:
        - &quot;6379:6379&quot;
  
    zookeeper:
      image: confluentinc/cp-zookeeper:5.5.1
      ports:
        - &quot;2181:2181&quot;
      environment:
        ZOOKEEPER_CLIENT_PORT: &quot;2181&quot;
  
    kafka:
      image: confluentinc/cp-kafka:5.5.1
      ports:
        - &quot;9092:9092&quot;
      environment:
        KAFKA_BROKER_ID: 0
        KAFKA_ZOOKEEPER_CONNECT: &quot;zookeeper:2181&quot;
        KAFKA_ADVERTISED_LISTENERS: &quot;INTERNAL://kafka:19092,EXTERNAL://${DOCKER_HOST_IP:-127.0.0.1}:9092&quot;
        KAFKA_INTER_BROKER_LISTENER_NAME: &quot;INTERNAL&quot;
        KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: &quot;INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT&quot;
        KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: &quot;1&quot;
      depends_on:
        - &quot;zookeeper&quot;
   
    connect:
      image: gcr.io/simulation-images/kafka-connect-redis-source
      ports:
      - &quot;8083:8083&quot;
      - &quot;5005:5005&quot;
      environment:
        CONNECT_BOOTSTRAP_SERVERS: &quot;PLAINTEXT://kafka:19092&quot;
        CONNECT_GROUP_ID: &quot;connect&quot;
        CONNECT_REST_ADVERTISED_HOST_NAME: &quot;connect&quot;
        CONNECT_PLUGIN_PATH: &quot;/usr/share/java&quot;
        CONNECT_INTERNAL_KEY_CONVERTER: &quot;org.apache.kafka.connect.json.JsonConverter&quot;
        CONNECT_INTERNAL_VALUE_CONVERTER: &quot;org.apache.kafka.connect.json.JsonConverter&quot;
        CONNECT_KEY_CONVERTER: &quot;org.apache.kafka.connect.converters.ByteArrayConverter&quot;
        CONNECT_VALUE_CONVERTER: &quot;org.apache.kafka.connect.storage.StringConverter&quot;
        CONNECT_CONFIG_STORAGE_TOPIC: &quot;connect-config&quot;
        CONNECT_OFFSET_STORAGE_TOPIC: &quot;connect-offset&quot;
        CONNECT_STATUS_STORAGE_TOPIC: &quot;connect-status&quot;
        CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: &quot;1&quot;
        CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: &quot;1&quot;
        CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: &quot;1&quot;
        CONNECT_DEBUG: &quot;y&quot;
        DEBUG_SUSPEND_FLAG: &quot;y&quot;
        CLASSPATH: &quot;/usr/share/java/kafka-connect-redis-source/*&quot;
  
      depends_on:
      - &quot;kafka&quot;
      - &quot;redis&quot;
  
  </code></pre>
  <ul>
  <li><code>docker ps</code>:</li>
  </ul>
  <pre><code>=&gt; docker ps
  CONTAINER ID   IMAGE                                                 COMMAND                  CREATED         STATUS                            PORTS                                                                                            NAMES
  c9c0c942d956   gcr.io/simulation-images/kafka-connect-redis-source   &quot;/bin/sh -c 'connect…&quot;   4 minutes ago   Up 4 minutes (health: starting)   0.0.0.0:5005-&gt;5005/tcp, :::5005-&gt;5005/tcp, 0.0.0.0:8083-&gt;8083/tcp, :::8083-&gt;8083/tcp, 9092/tcp   connect_1
  dc94eb723e93   confluentinc/cp-kafka:5.5.1                           &quot;/etc/confluent/dock…&quot;   4 minutes ago   Up 4 minutes                      0.0.0.0:9092-&gt;9092/tcp, :::9092-&gt;9092/tcp                                                        kafka_1
  c030c3be44e3   confluentinc/cp-zookeeper:5.5.1                       &quot;/etc/confluent/dock…&quot;   4 minutes ago   Up 4 minutes                      2888/tcp, 0.0.0.0:2181-&gt;2181/tcp, :::2181-&gt;2181/tcp, 3888/tcp                                    zookeeper_1
  b7f2807280b1   redis:6.0.6                                           &quot;docker-entrypoint.s…&quot;   4 minutes ago   Up 4 minutes                      0.0.0.0:6379-&gt;6379/tcp, :::6379-&gt;6379/tcp                                                        redis_1
  </code></pre>
  <ul>
  <li><code>connect</code> service keep raising errors (It kept restarting):</li>
  </ul>
  <pre><code>connect_1    | [2021-12-07 22:42:29,455] ERROR Error converting event to JSON (org.apache.kafka.connect.redis.RedisSourceTask:102)
  connect_1    | com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class com.moilioncircle.redis.replicator.event.PreRdbSyncEvent and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS)
  connect_1    |  at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
  connect_1    |  at com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
  connect_1    |  at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:404)
  connect_1    |  at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.failForEmpty(UnknownSerializer.java:71)
  connect_1    |  at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.serialize(UnknownSerializer.java:33)
  connect_1    |  at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
  connect_1    |  at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
  connect_1    |  at com.fasterxml.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:4094)
  connect_1    |  at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:3404)
  connect_1    |  at org.apache.kafka.connect.redis.RedisSourceTask.getSourceRecord(RedisSourceTask.java:99)
  connect_1    |  at org.apache.kafka.connect.redis.RedisSourceTask.poll(RedisSourceTask.java:77)
  connect_1    |  at org.apache.kafka.connect.runtime.WorkerSourceTask.poll(WorkerSourceTask.java:272)
  connect_1    |  at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:239)
  connect_1    |  at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:184)
  connect_1    |  at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:234)
  connect_1    |  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
  connect_1    |  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
  connect_1    |  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  connect_1    |  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  connect_1    |  at java.lang.Thread.run(Thread.java:748)
  connect_1    | [2021-12-07 22:42:29,530] WARN [Producer clientId=connector-producer-redis-config-0] Error while fetching metadata with correlation id 3 : {redis-server=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient:1077)
  connect_1    | [2021-12-07 22:42:29,666] ERROR Error converting event to JSON (org.apache.kafka.connect.redis.RedisSourceTask:102)
  connect_1    | com.fasterxml.jackson.databind.exc.InvalidDefinitionException: No serializer found for class com.moilioncircle.redis.replicator.event.PreCommandSyncEvent and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationFeature.FAIL_ON_EMPTY_BEANS)
  connect_1    |  at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
  connect_1    |  at com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1191)
  connect_1    |  at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:404)
  connect_1    |  at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.failForEmpty(UnknownSerializer.java:71)
  connect_1    |  at com.fasterxml.jackson.databind.ser.impl.UnknownSerializer.serialize(UnknownSerializer.java:33)
  connect_1    |  at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
  connect_1    |  at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
  connect_1    |  at com.fasterxml.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:4094)
  connect_1    |  at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:3404)
  connect_1    |  at org.apache.kafka.connect.redis.RedisSourceTask.getSourceRecord(RedisSourceTask.java:99)
  connect_1    |  at org.apache.kafka.connect.redis.RedisSourceTask.poll(RedisSourceTask.java:77)
  connect_1    |  at org.apache.kafka.connect.runtime.WorkerSourceTask.poll(WorkerSourceTask.java:272)
  connect_1    |  at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:239)
  connect_1    |  at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:184)
  connect_1    |  at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:234)
  connect_1    |  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
  connect_1    |  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
  connect_1    |  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  connect_1    |  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  connect_1    |  at java.lang.Thread.run(Thread.java:748)
  </code></pre>
  <p>I know that the <code>connect</code> service is a sort of custom image so could be difficult to debug but I'd like to know whether these errors are related to Kafka environment variables(e.g. <code>KAFKA_ADVERTISED_LISTENERS</code>) or it is a code level (e.g. java) problem.</p>