Chapter 3. Configuring external listeners
Chapter 3. Configuring external listeners Use an external listener to expose your AMQ Streams Kafka cluster to a client outside an OpenShift environment. Specify the connection type to expose Kafka in the external listener configuration. nodeport uses NodePort type Services loadbalancer uses Loadbalancer type Services ingress uses Kubernetes Ingress and the NGINX Ingress Controller for Kubernetes route uses OpenShift Routes and the HAProxy router For more information on listener configuration, see GenericKafkaListener schema reference . Note route is only supported on OpenShift Additional resources Accessing Apache Kafka in Strimzi 3.1. Accessing Kafka using node ports This procedure describes how to access a AMQ Streams Kafka cluster from an external client using node ports. To connect to a broker, you need a hostname and port number for the Kafka bootstrap address , as well as the certificate used for authentication. Prerequisites An OpenShift cluster A running Cluster Operator Procedure Configure a Kafka resource with an external listener set to the nodeport type. For example: apiVersion: kafka.strimzi.io/v1beta2 kind: Kafka spec: kafka: # ... listeners: - name: external port: 9094 type: nodeport tls: true authentication: type: tls # ... # ... zookeeper: # ... Create or update the resource. oc apply -f KAFKA-CONFIG-FILE NodePort type services are created for each Kafka broker, as well as an external bootstrap service . The bootstrap service routes external traffic to the Kafka brokers. Node addresses used for connection are propagated to the status of the Kafka custom resource. The cluster CA certificate to verify the identity of the kafka brokers is also created with the same name as the Kafka resource. Retrieve the bootstrap address you can use to access the Kafka cluster from the status of the Kafka resource. oc get kafka KAFKA-CLUSTER-NAME -o=jsonpath='{.status.listeners[?(@.type=="external")].bootstrapServers}{"\n"}' If TLS encryption is enabled, extract the public certificate of the broker certification authority. oc get secret KAFKA-CLUSTER-NAME -cluster-ca-cert -o jsonpath='{.data.ca\.crt}' | base64 -d > ca.crt Use the extracted certificate in your Kafka client to configure TLS connection. If you enabled any authentication, you will also need to configure SASL or TLS authentication. 3.2. Accessing Kafka using loadbalancers This procedure describes how to access a AMQ Streams Kafka cluster from an external client using loadbalancers. To connect to a broker, you need the address of the bootstrap loadbalancer , as well as the certificate used for TLS encryption. Prerequisites An OpenShift cluster A running Cluster Operator Procedure Configure a Kafka resource with an external listener set to the loadbalancer type. For example: apiVersion: kafka.strimzi.io/v1beta2 kind: Kafka spec: kafka: # ... listeners: - name: external port: 9094 type: loadbalancer tls: true # ... # ... zookeeper: # ... Create or update the resource. oc apply -f KAFKA-CONFIG-FILE loadbalancer type services and loadbalancers are created for each Kafka broker, as well as an external bootstrap service . The bootstrap service routes external traffic to all Kafka brokers. DNS names and IP addresses used for connection are propagated to the status of each service. The cluster CA certificate to verify the identity of the kafka brokers is also created with the same name as the Kafka resource. Retrieve the address of the bootstrap service you can use to access the Kafka cluster from the status of the Kafka resource. oc get kafka KAFKA-CLUSTER-NAME -o=jsonpath='{.status.listeners[?(@.type=="external")].bootstrapServers}{"\n"}' If TLS encryption is enabled, extract the public certificate of the broker certification authority. oc get secret KAFKA-CLUSTER-NAME -cluster-ca-cert -o jsonpath='{.data.ca\.crt}' | base64 -d > ca.crt Use the extracted certificate in your Kafka client to configure TLS connection. If you enabled any authentication, you will also need to configure SASL or TLS authentication. 3.3. Accessing Kafka using ingress This procedure shows how to access a AMQ Streams Kafka cluster from an external client outside of OpenShift using Nginx Ingress. To connect to a broker, you need a hostname (advertised address) for the Ingress bootstrap address , as well as the certificate used for authentication. For access using Ingress, the port is always 443. TLS passthrough Kafka uses a binary protocol over TCP, but the NGINX Ingress Controller for Kubernetes is designed to work with the HTTP protocol. To be able to pass the Kafka connections through the Ingress, AMQ Streams uses the TLS passthrough feature of the NGINX Ingress Controller for Kubernetes . Ensure TLS passthrough is enabled in your NGINX Ingress Controller for Kubernetes deployment. Because it is using the TLS passthrough functionality, TLS encryption cannot be disabled when exposing Kafka using Ingress . For more information about enabling TLS passthrough, see TLS passthrough documentation . Prerequisites OpenShift cluster Deployed NGINX Ingress Controller for Kubernetes with TLS passthrough enabled A running Cluster Operator Procedure Configure a Kafka resource with an external listener set to the ingress type. Specify the Ingress hosts for the bootstrap service and Kafka brokers. For example: apiVersion: kafka.strimzi.io/v1beta2 kind: Kafka spec: kafka: # ... listeners: - name: external port: 9094 type: ingress tls: true authentication: type: tls configuration: 1 bootstrap: host: bootstrap.myingress.com brokers: - broker: 0 host: broker-0.myingress.com - broker: 1 host: broker-1.myingress.com - broker: 2 host: broker-2.myingress.com # ... zookeeper: # ... 1 Ingress hosts for the bootstrap service and Kafka brokers. Create or update the resource. oc apply -f KAFKA-CONFIG-FILE ClusterIP type services are created for each Kafka broker, as well as an additional bootstrap service . These services are used by the Ingress controller to route traffic to the Kafka brokers. An Ingress resource is also created for each service to expose them using the Ingress controller. The Ingress hosts are propagated to the status of each service. The cluster CA certificate to verify the identity of the kafka brokers is also created with the same name as the Kafka resource. Use the address for the bootstrap host you specified in the configuration and port 443 ( BOOTSTRAP-HOST:443 ) in your Kafka client as the bootstrap address to connect to the Kafka cluster. Extract the public certificate of the broker certificate authority. oc get secret KAFKA-CLUSTER-NAME -cluster-ca-cert -o jsonpath='{.data.ca\.crt}' | base64 -d > ca.crt Use the extracted certificate in your Kafka client to configure the TLS connection. If you enabled any authentication, you will also need to configure SASL or TLS authentication. 3.4. Accessing Kafka using OpenShift routes This procedure describes how to access a AMQ Streams Kafka cluster from an external client outside of OpenShift using routes. To connect to a broker, you need a hostname for the route bootstrap address , as well as the certificate used for TLS encryption. For access using routes, the port is always 443. Prerequisites An OpenShift cluster A running Cluster Operator Procedure Configure a Kafka resource with an external listener set to the route type. For example: apiVersion: kafka.strimzi.io/v1beta2 kind: Kafka metadata: labels: app: my-cluster name: my-cluster namespace: myproject spec: kafka: # ... listeners: - name: listener1 port: 9094 type: route tls: true # ... # ... zookeeper: # ... Warning An OpenShift Route address comprises the name of the Kafka cluster, the name of the listener, and the name of the namespace it is created in. For example, my-cluster-kafka-listener1-bootstrap-myproject ( CLUSTER-NAME -kafka- LISTENER-NAME -bootstrap- NAMESPACE ). Be careful that the whole length of the address does not exceed a maximum limit of 63 characters. Create or update the resource. oc apply -f KAFKA-CONFIG-FILE ClusterIP type services are created for each Kafka broker, as well as an external bootstrap service . The services route the traffic from the OpenShift Routes to the Kafka brokers. An OpenShift Route resource is also created for each service to expose them using the HAProxy load balancer. DNS addresses used for connection are propagated to the status of each service. The cluster CA certificate to verify the identity of the kafka brokers is also created with the same name as the Kafka resource. Retrieve the address of the bootstrap service you can use to access the Kafka cluster from the status of the Kafka resource. oc get kafka KAFKA-CLUSTER-NAME -o=jsonpath='{.status.listeners[?(@.type=="external")].bootstrapServers}{"\n"}' Extract the public certificate of the broker certification authority. oc get secret KAFKA-CLUSTER-NAME -cluster-ca-cert -o jsonpath='{.data.ca\.crt}' | base64 -d > ca.crt Use the extracted certificate in your Kafka client to configure TLS connection. If you enabled any authentication, you will also need to configure SASL or TLS authentication.
apiVersion: kafka.strimzi.io/v1beta2 kind: Kafka spec: kafka: # listeners: - name: external port: 9094 type: nodeport tls: true authentication: type: tls # # zookeeper: # apply -f KAFKA-CONFIG-FILE get kafka KAFKA-CLUSTER-NAME -o=jsonpath='{.status.listeners[?(@.type=="external")].bootstrapServers}{"\n"}' get secret KAFKA-CLUSTER-NAME -cluster-ca-cert -o jsonpath='{.data.ca\.crt}' | base64 -d > ca.crt apiVersion: kafka.strimzi.io/v1beta2 kind: Kafka spec: kafka: # listeners: - name: external port: 9094 type: loadbalancer tls: true # # zookeeper: # apply -f KAFKA-CONFIG-FILE get kafka KAFKA-CLUSTER-NAME -o=jsonpath='{.status.listeners[?(@.type=="external")].bootstrapServers}{"\n"}' get secret KAFKA-CLUSTER-NAME -cluster-ca-cert -o jsonpath='{.data.ca\.crt}' | base64 -d > ca.crt apiVersion: kafka.strimzi.io/v1beta2 kind: Kafka spec: kafka: # listeners: - name: external port: 9094 type: ingress tls: true authentication: type: tls configuration: 1 bootstrap: host: bootstrap.myingress.com brokers: - broker: 0 host: broker-0.myingress.com - broker: 1 host: broker-1.myingress.com - broker: 2 host: broker-2.myingress.com # zookeeper: # apply -f KAFKA-CONFIG-FILE get secret KAFKA-CLUSTER-NAME -cluster-ca-cert -o jsonpath='{.data.ca\.crt}' | base64 -d > ca.crt apiVersion: kafka.strimzi.io/v1beta2 kind: Kafka metadata: labels: app: my-cluster name: my-cluster namespace: myproject spec: kafka: # listeners: - name: listener1 port: 9094 type: route tls: true # # zookeeper: # apply -f KAFKA-CONFIG-FILE get kafka KAFKA-CLUSTER-NAME -o=jsonpath='{.status.listeners[?(@.type=="external")].bootstrapServers}{"\n"}' get secret KAFKA-CLUSTER-NAME -cluster-ca-cert -o jsonpath='{.data.ca\.crt}' | base64 -d > ca.crt