Chapter 8. Developer Previews
Chapter 8. Developer Previews Developer preview features included with Streams for Apache Kafka 2.9. As a Kafka cluster administrator, you can toggle a subset of features on and off using feature gates in the Cluster Operator deployment configuration. The feature gates available as developer previews are at an alpha level of maturity and disabled by default. Important Developer Preview features are not supported with Red Hat production service-level agreements (SLAs) and might not be functionally complete; therefore, Red Hat does not recommend implementing any Developer Preview features in production environments. This Developer Preview feature provides early access to upcoming product innovations, enabling you to test functionality and provide feedback during the development process. For more information about the support scope, see Developer Preview Support Scope . 8.1. Tiered storage for Kafka brokers Streams for Apache Kafka now supports tiered storage for Kafka brokers as a developer preview, allowing you to introduce custom remote storage solutions as well as local storage. Due to its current limitations , it is not recommended for production environments. Remote storage configuration is specified using kafka.tieredStorage properties in the Kafka resource. You specify a custom remote storage manager to manage the tiered storage. Example custom tiered storage configuration kafka: tieredStorage: type: custom remoteStorageManager: className: com.example.kafka.tiered.storage.s3.S3RemoteStorageManager classPath: /opt/kafka/plugins/tiered-storage-s3/* config: # remote storage manager configuration 1 storage.bucket.name: my-bucket config: ... rlmm.config.remote.log.metadata.topic.replication.factor: 1 2 1 Configure the custom remote storage manager with the necessary settings. The keys are automatically prefixed with rsm.config and appended to the Kafka broker configuration. 2 Streams for Apache Kafka uses the TopicBasedRemoteLogMetadataManager for Remote Log Metadata Management (RLMM). Add RLMM configuration using an rlmm.config. prefix. Note If you want to use custom tiered storage, you must first add the tiered storage plugin to the Streams for Apache Kafka image by building a custom container image. See Tiered storage (early access) .
kafka: tieredStorage: type: custom remoteStorageManager: className: com.example.kafka.tiered.storage.s3.S3RemoteStorageManager classPath: /opt/kafka/plugins/tiered-storage-s3/* config: # remote storage manager configuration 1 storage.bucket.name: my-bucket config: rlmm.config.remote.log.metadata.topic.replication.factor: 1 2