Chapter 13. GenericKafkaListenerConfiguration schema reference
Chapter 13. GenericKafkaListenerConfiguration schema reference Used in: GenericKafkaListener Full list of GenericKafkaListenerConfiguration schema properties Configures Kafka listeners. 13.1. Providing your own listener certificates The brokerCertChainAndKey property is for listeners that have TLS encryption enabled only. Use this property to provide your own Kafka listener certificates. Example loadbalancer listener configuration to provide certificates listeners: #... - name: external3 port: 9094 type: loadbalancer tls: true configuration: brokerCertChainAndKey: secretName: my-secret certificate: my-listener-certificate.crt key: my-listener-key.key # ... When the certificate or key in the brokerCertChainAndKey secret is updated, the operator automatically detects it in the reconciliation and triggers a rolling update of the Kafka brokers to reload the certificate. 13.2. Avoiding hops to other nodes The externalTrafficPolicy property is used with loadbalancer and nodeport listeners. When exposing Kafka outside of OpenShift, you can choose Local or Cluster . Local avoids hops to other nodes and preserves the client IP, whereas Cluster does neither. The default is Cluster . Example loadbalancer listener configuration avoiding hops listeners: #... - name: external3 port: 9094 type: loadbalancer tls: true configuration: externalTrafficPolicy: Local # ... 13.3. Providing CIDR source ranges for a loadbalancer The loadBalancerSourceRanges property is for loadbalancer listeners only. When exposing Kafka outside of OpenShift, use CIDR (Classless Inter-Domain Routing) source ranges in addition to labels and annotations to customize how a service is created. Example loadbalancer listener configuration to provide source ranges listeners: #... - name: external3 port: 9094 type: loadbalancer tls: true configuration: loadBalancerSourceRanges: - 10.0.0.0/8 - 88.208.76.87/32 # ... 13.4. Specifying a preferred node port address type The preferredNodePortAddressType property is for nodeport listeners only. Use this property in your listener configuration to specify the first address type checked as the node address. This property is useful, for example, if your deployment does not have DNS support or you only want to expose a broker internally through an internal DNS or IP address. If an address of this type is found, it is used. If the preferred address type is not found, Streams for Apache Kafka proceeds through the types in the standard order of priority: ExternalDNS ExternalIP Hostname InternalDNS InternalIP Example nodeport listener using a preferred node port address type listeners: #... - name: external4 port: 9094 type: nodeport tls: false configuration: preferredNodePortAddressType: InternalDNS # ... 13.5. Using fully-qualified DNS names The useServiceDnsDomain property is for internal and cluster-ip listeners. It defines whether the fully-qualified DNS names that include the cluster service suffix (usually .cluster.local ) are used. Set to false (default) to generate advertised addresses without the service suffix; for example, my-cluster-kafka-0.my-cluster-kafka-brokers.myproject.svc . Set to true to generate advertised addresses with the service suffix; for example, my-cluster-kafka-0.my-cluster-kafka-brokers.myproject.svc.cluster.local . Example internal listener using the service DNS domain listeners: #... - name: plain port: 9092 type: internal tls: false configuration: useServiceDnsDomain: true # ... 13.6. Specifying the hostname To specify the hostname used for the bootstrap resource or brokers, use the host property. The host property is for route and ingress listeners only. A host property value is mandatory for ingress listener configuration, as the Ingress controller does not assign any hostnames automatically. Make sure that the hostname resolves to the Ingress endpoints. Streams for Apache Kafka will not perform any validation to ensure that the requested hosts are available and properly routed to the Ingress endpoints. Example ingress listener with host configuration listeners: #... - name: external2 port: 9094 type: ingress tls: true configuration: bootstrap: host: bootstrap.myingress.com brokers: - broker: 0 host: broker-0.myingress.com - broker: 1 host: broker-1.myingress.com - broker: 2 host: broker-2.myingress.com # ... By default, route listener hosts are automatically assigned by OpenShift. However, you can override the assigned route hosts by specifying hosts. Streams for Apache Kafka does not perform any validation to ensure that the requested hosts are available. You must ensure that they are free and can be used. Example route listener with host configuration # ... listeners: #... - name: external1 port: 9094 type: route tls: true configuration: bootstrap: host: bootstrap.myrouter.com brokers: - broker: 0 host: broker-0.myrouter.com - broker: 1 host: broker-1.myrouter.com - broker: 2 host: broker-2.myrouter.com # ... Instead of specifying the host property for every broker, you can also use a hostTemplate to generate them automatically. The hostTemplate supports the following variables: The {nodeId} variable is replaced with the ID of the Kafka node to which the template is applied. The {nodePodName} variable is replaced with the OpenShift pod name for the Kafka node where the template is applied. The hostTemplate property applies only to per-broker values. The bootstrap host property must always be specified. Example ingress listener with hostTemplate configuration #... spec: kafka: #... listeners: #... - name: external2 port: 9095 type: ingress tls: true authentication: type: tls configuration: hostTemplate: broker-{nodeId}.myingress.com bootstrap: host: bootstrap.myingress.com #... 13.7. Overriding assigned node ports By default, the port numbers used for the bootstrap and broker services are automatically assigned by OpenShift. You can override the assigned node ports for nodeport listeners by specifying the desired port numbers. Streams for Apache Kafka does not perform any validation on the requested ports. You must ensure that they are free and available for use. Example nodeport listener configuration with overrides for node ports # ... listeners: #... - name: external4 port: 9094 type: nodeport tls: true configuration: bootstrap: nodePort: 32100 brokers: - broker: 0 nodePort: 32000 - broker: 1 nodePort: 32001 - broker: 2 nodePort: 32002 # ... 13.8. Requesting a specific loadbalancer IP address Use the loadBalancerIP property to request a specific IP address when creating a loadbalancer. This property is useful when you need to use a loadbalancer with a specific IP address. The loadBalancerIP property is ignored if the cloud provider does not support this feature. Example loadbalancer listener with specific IP addresses # ... listeners: #... - name: external3 port: 9094 type: loadbalancer tls: true configuration: bootstrap: loadBalancerIP: 172.29.3.10 brokers: - broker: 0 loadBalancerIP: 172.29.3.1 - broker: 1 loadBalancerIP: 172.29.3.2 - broker: 2 loadBalancerIP: 172.29.3.3 # ... 13.9. Adding listener annotations to OpenShift resources Use the annotations property to add annotations to OpenShift resources related to the listeners. These annotations can be used, for example, to instrument DNS tooling such as External DNS , which automatically assigns DNS names to the loadbalancer services. Example loadbalancer listener using annotations # ... listeners: #... - name: external3 port: 9094 type: loadbalancer tls: true configuration: bootstrap: annotations: external-dns.alpha.kubernetes.io/hostname: kafka-bootstrap.mydomain.com. external-dns.alpha.kubernetes.io/ttl: "60" brokers: - broker: 0 annotations: external-dns.alpha.kubernetes.io/hostname: kafka-broker-0.mydomain.com. external-dns.alpha.kubernetes.io/ttl: "60" - broker: 1 annotations: external-dns.alpha.kubernetes.io/hostname: kafka-broker-1.mydomain.com. external-dns.alpha.kubernetes.io/ttl: "60" - broker: 2 annotations: external-dns.alpha.kubernetes.io/hostname: kafka-broker-2.mydomain.com. external-dns.alpha.kubernetes.io/ttl: "60" # ... 13.10. GenericKafkaListenerConfiguration schema properties Property Property type Description brokerCertChainAndKey CertAndKeySecretSource Reference to the Secret which holds the certificate and private key pair which will be used for this listener. The certificate can optionally contain the whole chain. This field can be used only with listeners with enabled TLS encryption. class string Configures a specific class for Ingress and LoadBalancer that defines which controller is used. If not specified, the default controller is used. For an ingress listener, the operator uses this property to set the ingressClassName property in the Ingress resources. For a loadbalancer listener, the operator uses this property to set the loadBalancerClass property in the Service resources. For ingress and loadbalancer listeners only. externalTrafficPolicy string (one of [Local, Cluster]) Specifies whether the service routes external traffic to cluster-wide or node-local endpoints: Cluster may cause a second hop to another node and obscures the client source IP. Local avoids a second hop for LoadBalancer and Nodeport type services and preserves the client source IP (when supported by the infrastructure). If unspecified, OpenShift uses Cluster as the default. For loadbalancer or nodeport listeners only. loadBalancerSourceRanges string array A list of CIDR ranges (for example 10.0.0.0/8 or 130.211.204.1/32 ) from which clients can connect to loadbalancer listeners. If supported by the platform, traffic through the loadbalancer is restricted to the specified CIDR ranges. This field is applicable only for loadbalancer type services and is ignored if the cloud provider does not support the feature. For loadbalancer listeners only. bootstrap GenericKafkaListenerConfigurationBootstrap Bootstrap configuration. brokers GenericKafkaListenerConfigurationBroker array Per-broker configurations. ipFamilyPolicy string (one of [RequireDualStack, SingleStack, PreferDualStack]) Specifies the IP Family Policy used by the service. Available options are SingleStack , PreferDualStack and RequireDualStack : SingleStack is for a single IP family. PreferDualStack is for two IP families on dual-stack configured clusters or a single IP family on single-stack clusters. RequireDualStack fails unless there are two IP families on dual-stack configured clusters. If unspecified, OpenShift will choose the default value based on the service type. ipFamilies string (one or more of [IPv6, IPv4]) array Specifies the IP Families used by the service. Available options are IPv4 and IPv6 . If unspecified, OpenShift will choose the default value based on the ipFamilyPolicy setting. createBootstrapService boolean Whether to create the bootstrap service or not. The bootstrap service is created by default (if not specified differently). This field can be used with the loadbalancer listener. finalizers string array A list of finalizers configured for the LoadBalancer type services created for this listener. If supported by the platform, the finalizer service.kubernetes.io/load-balancer-cleanup to make sure that the external load balancer is deleted together with the service.For more information, see https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/#garbage-collecting-load-balancers . For loadbalancer listeners only. useServiceDnsDomain boolean Configures whether the OpenShift service DNS domain should be included in the generated addresses. If set to false , the generated addresses do not contain the service DNS domain suffix. For example, my-cluster-kafka-0.my-cluster-kafka-brokers.myproject.svc . If set to true , the generated addresses contain the service DNS domain suffix. For example, my-cluster-kafka-0.my-cluster-kafka-brokers.myproject.svc.cluster.local . The default is .cluster.local , but this is customizable using the environment variable KUBERNETES_SERVICE_DNS_DOMAIN . For internal and cluster-ip listeners only. maxConnections integer The maximum number of connections we allow for this listener in the broker at any time. New connections are blocked if the limit is reached. maxConnectionCreationRate integer The maximum connection creation rate we allow in this listener at any time. New connections will be throttled if the limit is reached. preferredNodePortAddressType string (one of [ExternalDNS, ExternalIP, Hostname, InternalIP, InternalDNS]) Defines which address type should be used as the node address. Available types are: ExternalDNS , ExternalIP , InternalDNS , InternalIP and Hostname . By default, the addresses are used in the following order (the first one found is used): ExternalDNS ExternalIP InternalDNS InternalIP Hostname This property is used to select the preferred address type, which is checked first. If no address is found for this address type, the other types are checked in the default order.For nodeport listeners only. publishNotReadyAddresses boolean Configures whether the service endpoints are considered "ready" even if the Pods themselves are not. Defaults to false . This field can not be used with internal listeners. hostTemplate string Configures the template for generating the hostnames of the individual brokers. Valid placeholders that you can use in the template are {nodeId} and {nodePodName} . advertisedHostTemplate string Configures the template for generating the advertised hostnames of the individual brokers. Valid placeholders that you can use in the template are {nodeId} and {nodePodName} . allocateLoadBalancerNodePorts boolean Configures whether to allocate NodePort automatically for the Service with type LoadBalancer . This is a one to one with the spec.allocateLoadBalancerNodePorts configuration in the Service type For loadbalancer listeners only.
listeners: # - name: external3 port: 9094 type: loadbalancer tls: true configuration: brokerCertChainAndKey: secretName: my-secret certificate: my-listener-certificate.crt key: my-listener-key.key listeners: # - name: external3 port: 9094 type: loadbalancer tls: true configuration: externalTrafficPolicy: Local listeners: # - name: external3 port: 9094 type: loadbalancer tls: true configuration: loadBalancerSourceRanges: - 10.0.0.0/8 - 88.208.76.87/32 listeners: # - name: external4 port: 9094 type: nodeport tls: false configuration: preferredNodePortAddressType: InternalDNS listeners: # - name: plain port: 9092 type: internal tls: false configuration: useServiceDnsDomain: true listeners: # - name: external2 port: 9094 type: ingress tls: true configuration: bootstrap: host: bootstrap.myingress.com brokers: - broker: 0 host: broker-0.myingress.com - broker: 1 host: broker-1.myingress.com - broker: 2 host: broker-2.myingress.com listeners: # - name: external1 port: 9094 type: route tls: true configuration: bootstrap: host: bootstrap.myrouter.com brokers: - broker: 0 host: broker-0.myrouter.com - broker: 1 host: broker-1.myrouter.com - broker: 2 host: broker-2.myrouter.com # spec: kafka: # listeners: # - name: external2 port: 9095 type: ingress tls: true authentication: type: tls configuration: hostTemplate: broker-{nodeId}.myingress.com bootstrap: host: bootstrap.myingress.com # listeners: # - name: external4 port: 9094 type: nodeport tls: true configuration: bootstrap: nodePort: 32100 brokers: - broker: 0 nodePort: 32000 - broker: 1 nodePort: 32001 - broker: 2 nodePort: 32002 listeners: # - name: external3 port: 9094 type: loadbalancer tls: true configuration: bootstrap: loadBalancerIP: 172.29.3.10 brokers: - broker: 0 loadBalancerIP: 172.29.3.1 - broker: 1 loadBalancerIP: 172.29.3.2 - broker: 2 loadBalancerIP: 172.29.3.3 listeners: # - name: external3 port: 9094 type: loadbalancer tls: true configuration: bootstrap: annotations: external-dns.alpha.kubernetes.io/hostname: kafka-bootstrap.mydomain.com. external-dns.alpha.kubernetes.io/ttl: "60" brokers: - broker: 0 annotations: external-dns.alpha.kubernetes.io/hostname: kafka-broker-0.mydomain.com. external-dns.alpha.kubernetes.io/ttl: "60" - broker: 1 annotations: external-dns.alpha.kubernetes.io/hostname: kafka-broker-1.mydomain.com. external-dns.alpha.kubernetes.io/ttl: "60" - broker: 2 annotations: external-dns.alpha.kubernetes.io/hostname: kafka-broker-2.mydomain.com. external-dns.alpha.kubernetes.io/ttl: "60"