Chapter 29. Managing rolling updates
Chapter 29. Managing rolling updates Use annotations to manually trigger a rolling update of Kafka and other operands through the Cluster Operator. Initiate rolling updates of Kafka, Kafka Connect, MirrorMaker 2, and ZooKeeper clusters. Manually performing a rolling update on a specific pod or set of pods is usually only required in exceptional circumstances. However, rather than deleting the pods directly, if you perform the rolling update through the Cluster Operator you ensure the following: The manual deletion of the pod does not conflict with simultaneous Cluster Operator operations, such as deleting other pods in parallel. The Cluster Operator logic handles the Kafka configuration specifications, such as the number of in-sync replicas. 29.1. Performing a rolling update using a pod management annotation This procedure describes how to trigger a rolling update of Kafka, Kafka Connect, MirrorMaker 2, or ZooKeeper clusters. To trigger the update, you add an annotation to the StrimziPodSet that manages the pods running on the cluster. Prerequisites To perform a manual rolling update, you need a running Cluster Operator. The cluster for the component you are updating, whether it's Kafka, Kafka Connect, MirrorMaker 2, or ZooKeeper, must also be running. Procedure Find the name of the resource that controls the pods you want to manually update. For example, if your Kafka cluster is named my-cluster , the corresponding names are my-cluster-kafka and my-cluster-zookeeper . For a Kafka Connect cluster named my-connect-cluster , the corresponding name is my-connect-cluster-connect . And for a MirrorMaker 2 cluster named my-mm2-cluster , the corresponding name is my-mm2-cluster-mirrormaker2 . Use oc annotate to annotate the appropriate resource in OpenShift. Annotating a StrimziPodSet oc annotate strimzipodset <cluster_name>-kafka strimzi.io/manual-rolling-update="true" oc annotate strimzipodset <cluster_name>-zookeeper strimzi.io/manual-rolling-update="true" oc annotate strimzipodset <cluster_name>-connect strimzi.io/manual-rolling-update="true" oc annotate strimzipodset <cluster_name>-mirrormaker2 strimzi.io/manual-rolling-update="true" Wait for the reconciliation to occur (every two minutes by default). A rolling update of all pods within the annotated resource is triggered, as long as the annotation was detected by the reconciliation process. When the rolling update of all the pods is complete, the annotation is automatically removed from the resource. 29.2. Performing a rolling update using a pod annotation This procedure describes how to manually trigger a rolling update of existing Kafka, Kafka Connect, MirrorMaker 2, or ZooKeeper clusters using an OpenShift Pod annotation. When multiple pods are annotated, consecutive rolling updates are performed within the same reconciliation run. Prerequisites To perform a manual rolling update, you need a running Cluster Operator. The cluster for the component you are updating, whether it's Kafka, Kafka Connect, MirrorMaker 2, or ZooKeeper, must also be running. You can perform a rolling update on a Kafka cluster regardless of the topic replication factor used. But for Kafka to stay operational during the update, you'll need the following: A highly available Kafka cluster deployment running with nodes that you wish to update. Topics replicated for high availability. Topic configuration specifies a replication factor of at least 3 and a minimum number of in-sync replicas to 1 less than the replication factor. Kafka topic replicated for high availability apiVersion: kafka.strimzi.io/v1beta2 kind: KafkaTopic metadata: name: my-topic labels: strimzi.io/cluster: my-cluster spec: partitions: 1 replicas: 3 config: # ... min.insync.replicas: 2 # ... Procedure Find the name of the Pod you want to manually update. Pod naming conventions are as follows: <cluster_name>-kafka-<index_number> for a Kafka cluster <cluster_name>-zookeeper-<index_number> for a ZooKeeper cluster <cluster_name>-connect-<index_number> for a Kafka Connect cluster <cluster_name>-mirrormaker2-<index_number> for a MirrorMaker 2 cluster The <index_number> assigned to a pod starts at zero and ends at the total number of replicas minus one. Use oc annotate to annotate the Pod resource in OpenShift: oc annotate pod <cluster_name>-kafka-<index_number> strimzi.io/manual-rolling-update="true" oc annotate pod <cluster_name>-zookeeper-<index_number> strimzi.io/manual-rolling-update="true" oc annotate pod <cluster_name>-connect-<index_number> strimzi.io/manual-rolling-update="true" oc annotate pod <cluster_name>-mirrormaker2-<index_number> strimzi.io/manual-rolling-update="true" Wait for the reconciliation to occur (every two minutes by default). A rolling update of the annotated Pod is triggered, as long as the annotation was detected by the reconciliation process. When the rolling update of a pod is complete, the annotation is automatically removed from the Pod . Note Unless the ContinueReconciliationOnManualRollingUpdateFailure feature gate is disabled, reconciliation continues even if the manual rolling update of the cluster fails. This allows the Cluster Operator to recover from certain rectifiable situations that can be addressed later in the reconciliation. For example, it can recreate a missing Persistent Volume Claim (PVC) or Persistent Volume (PV) that caused the update to fail.
annotate strimzipodset <cluster_name>-kafka strimzi.io/manual-rolling-update="true" annotate strimzipodset <cluster_name>-zookeeper strimzi.io/manual-rolling-update="true" annotate strimzipodset <cluster_name>-connect strimzi.io/manual-rolling-update="true" annotate strimzipodset <cluster_name>-mirrormaker2 strimzi.io/manual-rolling-update="true" apiVersion: kafka.strimzi.io/v1beta2 kind: KafkaTopic metadata: name: my-topic labels: strimzi.io/cluster: my-cluster spec: partitions: 1 replicas: 3 config: # min.insync.replicas: 2 # annotate pod <cluster_name>-kafka-<index_number> strimzi.io/manual-rolling-update="true" annotate pod <cluster_name>-zookeeper-<index_number> strimzi.io/manual-rolling-update="true" annotate pod <cluster_name>-connect-<index_number> strimzi.io/manual-rolling-update="true" annotate pod <cluster_name>-mirrormaker2-<index_number> strimzi.io/manual-rolling-update="true"