Chapter 4. Technology Previews
Chapter 4. Technology Previews Important Technology Preview features are not supported with Red Hat production service-level agreements (SLAs) and might not be functionally complete; therefore, Red Hat does not recommend implementing any Technology Preview features in production environments. This Technology Preview feature provides early access to upcoming product innovations, enabling you to test functionality and provide feedback during the development process. For more information about support scope, see Technology Preview Features Support Scope . 4.1. Kafka Static Quota plugin configuration Use the Kafka Static Quota plugin to set throughput and storage limits on brokers in your Kafka cluster. You enable the plugin and set limits by configuring the Kafka resource. You can set a byte-rate threshold and storage quotas to put limits on the clients interacting with your brokers. Example Kafka Static Quota plugin configuration apiVersion: kafka.strimzi.io/v1beta2 kind: Kafka metadata: name: my-cluster spec: kafka: # ... config: client.quota.callback.class: io.strimzi.kafka.quotas.StaticQuotaCallback client.quota.callback.static.produce: 1000000 client.quota.callback.static.fetch: 1000000 client.quota.callback.static.storage.soft: 400000000000 client.quota.callback.static.storage.hard: 500000000000 client.quota.callback.static.storage.check-interval: 5 See Setting limits on brokers using the Kafka Static Quota plugin 4.2. Cruise Control for cluster rebalancing Note Cruise Control remains in Technology Preview, with some new enhancements . You can deploy Cruise Control and use it to rebalance your Kafka cluster using optimization goals - defined constraints on CPU, disk, network load, and more. In a balanced Kafka cluster, the workload is more evenly distributed across the broker pods. Cruise Control is configured and deployed as part of a Kafka resource. You can use the default optimization goals or modify them to suit your requirements. Example YAML configuration files for Cruise Control are provided in examples/cruise-control/ . When Cruise Control is deployed, you can create KafkaRebalance custom resources to: Generate optimization proposals from multiple optimization goals Rebalance a Kafka cluster based on an optimization proposal Other Cruise Control features are not currently supported, including anomaly detection, notifications, write-your-own goals, and changing the topic replication factor. See Cruise Control for cluster rebalancing 4.2.1. Enhancements to the Technology Preview Refresh optimization proposals You can now reuse existing KafkaRebalance resources with a status of Ready , which indicates that cluster rebalancing completed successfully. You can reuse the optimization goals defined in the KafkaRebalance resource, or change the goals. To refresh an optimization proposal: Check the status of the KafkaRebalance resource: oc describe kafkarebalance REBALANCE-NAME Apply the strimzi.io/rebalance=refresh annotation: oc annotate kafkarebalance REBALANCE-NAME strimzi.io/rebalance=refresh Cruise Control refreshes the optimization proposal to reflect the latest state of your Kafka cluster. See Approving an optimization proposal View the broker load in optimization proposals Optimization proposals now consist of the broker load in addition to the summary status. The broker load is returned inside a ConfigMap and shows metrics for the load on each Kafka broker, including the CPU utilization, disk usage, network output rate, and more. The metrics are presented in three categories: before Current value before the optimization proposal is applied after Expected value after the optimization proposal is applied difference Difference between the after value and the before value The broker load ConfigMap has the same name as the KafkaRebalance resource. The metrics are encoded as a JSON string. To view them in a human readable format, use jq or a similiar JSON parser. For example: oc get configmap MY-REBALANCE -o json | jq '.["data"]["brokerLoad.json"]|fromjson|.' See Optimization proposals overview
apiVersion: kafka.strimzi.io/v1beta2 kind: Kafka metadata: name: my-cluster spec: kafka: # config: client.quota.callback.class: io.strimzi.kafka.quotas.StaticQuotaCallback client.quota.callback.static.produce: 1000000 client.quota.callback.static.fetch: 1000000 client.quota.callback.static.storage.soft: 400000000000 client.quota.callback.static.storage.hard: 500000000000 client.quota.callback.static.storage.check-interval: 5 describe kafkarebalance REBALANCE-NAME annotate kafkarebalance REBALANCE-NAME strimzi.io/rebalance=refresh get configmap MY-REBALANCE -o json | jq '.["data"]["brokerLoad.json"]|fromjson|.'