Chapter 19. AWS Load Balancer Operator
Chapter 19. AWS Load Balancer Operator 19.1. AWS Load Balancer Operator in OpenShift Container Platform The AWS Load Balancer (ALB) Operator deploys and manages an instance of the aws-load-balancer-controller . You can install the ALB Operator from the OperatorHub by using OpenShift Container Platform web console or CLI. 19.1.1. AWS Load Balancer Operator considerations Review the following limitations before installing and using the AWS Load Balancer Operator. The IP traffic mode only works on AWS Elastic Kubernetes Service (EKS). The AWS Load Balancer Operator disables the IP traffic mode for the AWS Load Balancer Controller. As a result of disabling the IP traffic mode, the AWS Load Balancer Controller cannot use the pod readiness gate. The AWS Load Balancer Operator adds command-line flags such as --disable-ingress-class-annotation and --disable-ingress-group-name-annotation to the AWS Load Balancer Controller. Therefore, the AWS Load Balancer Operator does not allow using the kubernetes.io/ingress.class and alb.ingress.kubernetes.io/group.name annotations in the Ingress resource. 19.1.2. AWS Load Balancer Operator The AWS Load Balancer Operator can tag the public subnets if the kubernetes.io/role/elb tag is missing. Also, the AWS Load Balancer Operator detects the following from the underlying AWS cloud: The ID of the virtual private cloud (VPC) on which the cluster hosting the Operator is deployed in. Public and private subnets of the discovered VPC. Prerequisites You must have the AWS credentials secret. The credentials are used to provide subnet tagging and VPC discovery. Procedure You can deploy the AWS Load Balancer Operator on demand from the OperatorHub, by creating a Subscription object: USD oc -n aws-load-balancer-operator get sub aws-load-balancer-operator --template='{{.status.installplan.name}}{{"\n"}}' Example output install-zlfbt Check the status of an install plan. The status of an install plan must be Complete : USD oc -n aws-load-balancer-operator get ip <install_plan_name> --template='{{.status.phase}}{{"\n"}}' Example output Complete Use the oc get command to view the Deployment status: USD oc get -n aws-load-balancer-operator deployment/aws-load-balancer-operator-controller-manager Example output NAME READY UP-TO-DATE AVAILABLE AGE aws-load-balancer-operator-controller-manager 1/1 1 1 23h 19.1.3. AWS Load Balancer Operator logs Use the oc logs command to view the AWS Load Balancer Operator logs. Procedure View the logs of the AWS Load Balancer Operator: USD oc logs -n aws-load-balancer-operator deployment/aws-load-balancer-operator-controller-manager -c manager 19.2. Understanding AWS Load Balancer Operator The AWS Load Balancer (ALB) Operator deploys and manages an instance of the aws-load-balancer-controller resource. You can install the AWS Load Balancer Operator from the OperatorHub by using OpenShift Container Platform web console or CLI. 19.2.1. Installing the AWS Load Balancer Operator You can install the AWS Load Balancer Operator from the OperatorHub by using the OpenShift Container Platform web console. Prerequisites You have logged in to the OpenShift Container Platform web console as a user with cluster-admin permissions. Your cluster is configured with AWS as the platform type and cloud provider. Procedure Navigate to Operators OperatorHub in the OpenShift Container Platform web console. Select the AWS Load Balancer Operator . You can use the Filter by keyword text box or use the filter list to search for the AWS Load Balancer Operator from the list of Operators. Select the aws-load-balancer-operator namespace. Follow the instructions to prepare the Operator for installation. On the AWS Load Balancer Operator page, click Install . On the Install Operator page, select the following options: Update the channel as stable-v0.1 . Installation mode as A specific namespace on the cluster . Installed Namespace as aws-load-balancer-operator . If the aws-load-balancer-operator namespace does not exist, it gets created during the Operator installation. Select Update approval as Automatic or Manual . By default, the Update approval is set to Automatic . If you select automatic updates, the Operator Lifecycle Manager (OLM) automatically upgrades the running instance of your Operator without any intervention. If you select manual updates, the OLM creates an update request. As a cluster administrator, you must then manually approve that update request to update the Operator updated to the new version. Click Install . Verification Verify that the AWS Load Balancer Operator shows the Status as Succeeded on the Installed Operators dashboard. 19.3. Creating an instance of AWS Load Balancer Controller After installing the Operator, you can create an instance of the AWS Load Balancer Controller. 19.3.1. Creating an instance of the AWS Load Balancer Controller using AWS Load Balancer Operator You can install only a single instance of the aws-load-balancer-controller in a cluster. You can create the AWS Load Balancer Controller by using CLI. The AWS Load Balancer(ALB) Operator reconciles only the resource with the name cluster . Prerequisites You have created the echoserver namespace. You have access to the OpenShift CLI ( oc ). Procedure Create an aws-load-balancer-controller resource YAML file, for example, sample-aws-lb.yaml , as follows: apiVersion: networking.olm.openshift.io/v1alpha1 kind: AWSLoadBalancerController 1 metadata: name: cluster 2 spec: subnetTagging: Auto 3 additionalResourceTags: 4 example.org/cost-center: 5113232 example.org/security-scope: staging ingressClass: alb 5 config: replicas: 2 6 enabledAddons: 7 - AWSWAFv2 8 1 Defines the aws-load-balancer-controller resource. 2 Defines the AWS Load Balancer Controller instance name. This instance name gets added as a suffix to all related resources. 3 Valid options are Auto and Manual . When the value is set to Auto , the Operator attempts to determine the subnets that belong to the cluster and tags them appropriately. The Operator cannot determine the role correctly if the internal subnet tags are not present on internal subnet. If you installed your cluster on user-provided infrastructure, you can manually tag the subnets with the appropriate role tags and set the subnet tagging policy to Manual . 4 Defines the tags used by the controller when it provisions AWS resources. 5 The default value for this field is alb . The Operator provisions an IngressClass resource with the same name if it does not exist. 6 Specifies the number of replicas of the controller. 7 Specifies add-ons for AWS load balancers, which get specified through annotations. 8 Enables the alb.ingress.kubernetes.io/wafv2-acl-arn annotation. Create a aws-load-balancer-controller resource by running the following command: USD oc create -f sample-aws-lb.yaml After the AWS Load Balancer Controller is running, create a deployment resource: apiVersion: apps/v1 kind: Deployment 1 metadata: name: <echoserver> 2 namespace: echoserver spec: selector: matchLabels: app: echoserver replicas: 3 3 template: metadata: labels: app: echoserver spec: containers: - image: openshift/origin-node args: - TCP4-LISTEN:8080,reuseaddr,fork - EXEC:'/bin/bash -c \"printf \\\"HTTP/1.0 200 OK\r\n\r\n\\\"; sed -e \\\"/^\r/q\\\"\"' imagePullPolicy: Always name: echoserver ports: - containerPort: 8080 1 Defines the deployment resource. 2 Specifies the deployment name. 3 Specifies the number of replicas of the deployment. Create a service resource: apiVersion: v1 kind: Service 1 metadata: name: <echoserver> 2 namespace: echoserver spec: ports: - port: 80 targetPort: 8080 protocol: TCP type: NodePort selector: app: echoserver 1 Defines the service resource. 2 Specifies the name of the service. Deploy an ALB-backed Ingress resource: apiVersion: networking.k8s.io/v1 kind: Ingress 1 metadata: name: <echoserver> 2 namespace: echoserver annotations: alb.ingress.kubernetes.io/scheme: internet-facing alb.ingress.kubernetes.io/target-type: instance spec: ingressClassName: alb rules: - http: paths: - path: / pathType: Exact backend: service: name: <echoserver> 3 port: number: 80 1 Defines the ingress resource. 2 Specifies the name of the ingress resource. 3 Specifies the name of the service resource. Verification Verify the status of the Ingress resource to show the host of the provisioned AWS Load Balancer (ALB) by running the following command: USD HOST=USD(kubectl get ingress -n echoserver echoserver -o json | jq -r '.status.loadBalancer.ingress[0].hostname') Verify the status of the provisioned AWS Load Balancer (ALB) host by running the following command: USD curl USDHOST 19.4. Creating multiple ingresses You can route the traffic to different services that are part of a single domain through a single AWS Load Balancer (ALB). Each Ingress resource provides different endpoints of the domain. 19.4.1. Creating multiple ingresses through a single AWS Load Balancer You can route the traffic to multiple Ingresses through a single AWS Load Balancer (ALB) by using the CLI. Prerequisites You have an access to the OpenShift CLI ( oc ). Procedure Create an IngressClassParams resource YAML file, for example, sample-single-lb-params.yaml , as follows: apiVersion: elbv2.k8s.aws/v1beta1 1 kind: IngressClassParams metadata: name: single-lb-params 2 spec: group: name: single-lb 3 1 Defines the API group and version of the IngressClassParams resource. 2 Specifies the name of the IngressClassParams resource. 3 Specifies the name of the IngressGroup . All Ingresses of this class belong to this IngressGroup . Create an IngressClassParams resource by running the following command: USD oc create -f sample-single-lb-params.yaml Create an IngressClass resource YAML file, for example, sample-single-lb-class.yaml , as follows: apiVersion: networking.k8s.io/v1 1 kind: IngressClass metadata: name: single-lb 2 spec: controller: ingress.k8s.aws/alb 3 parameters: apiGroup: elbv2.k8s.aws 4 kind: IngressClassParams 5 name: single-lb-params 6 1 Defines the API group and version of the IngressClass resource. 2 Specifies the name of the IngressClass . 3 Defines the controller name. ingress.k8s.aws/alb denotes that all Ingresses of this class should be managed by the aws-load-balancer-controller . 4 Defines the API group of the IngressClassParams resource. 5 Defines the resource type of the IngressClassParams resource. 6 Defines the name of the IngressClassParams resource. Create an IngressClass resource by running the following command: USD oc create -f sample-single-lb-class.yaml Create an AWSLoadBalancerController resource YAML file, for example, sample-single-lb.yaml , as follows: apiVersion: networking.olm.openshift.io/v1 kind: AWSLoadBalancerController metadata: name: cluster spec: subnetTagging: Auto ingressClass: single-lb 1 1 Defines the name of the IngressClass resource. Create an AWSLoadBalancerController resource by running the following command: USD oc create -f sample-single-lb.yaml Create an Ingress resource YAML file, for example, sample-multiple-ingress.yaml , as follows: apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: example-1 1 annotations: alb.ingress.kubernetes.io/scheme: internet-facing 2 alb.ingress.kubernetes.io/group.order: "1" 3 alb.ingress.kubernetes.io/target-type: instance 4 spec: ingressClassName: single-lb 5 rules: - host: example.com 6 http: paths: - path: /blog 7 pathType: Prefix backend: service: name: example-1 8 port: number: 80 9 --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: example-2 annotations: alb.ingress.kubernetes.io/scheme: internet-facing alb.ingress.kubernetes.io/group.order: "2" alb.ingress.kubernetes.io/target-type: instance spec: ingressClassName: single-lb rules: - host: example.com http: paths: - path: /store pathType: Prefix backend: service: name: example-2 port: number: 80 --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: example-3 annotations: alb.ingress.kubernetes.io/scheme: internet-facing alb.ingress.kubernetes.io/group.order: "3" alb.ingress.kubernetes.io/target-type: instance spec: ingressClassName: single-lb rules: - host: example.com http: paths: - path: / pathType: Prefix backend: service: name: example-3 port: number: 80 1 Specifies the name of an ingress. 2 Indicates the load balancer to provision in the public subnet and makes it accessible over the internet. 3 Specifies the order in which the rules from the Ingresses are matched when the request is received at the load balancer. 4 Indicates the load balancer will target OpenShift nodes to reach the service. 5 Specifies the Ingress Class that belongs to this ingress. 6 Defines the name of a domain used for request routing. 7 Defines the path that must route to the service. 8 Defines the name of the service that serves the endpoint configured in the ingress. 9 Defines the port on the service that serves the endpoint. Create the Ingress resources by running the following command: USD oc create -f sample-multiple-ingress.yaml 19.5. Adding TLS termination You can add TLS termination on the AWS Load Balancer. 19.5.1. Adding TLS termination on the AWS Load Balancer You can route the traffic for the domain to pods of a service and add TLS termination on the AWS Load Balancer. Prerequisites You have an access to the OpenShift CLI ( oc ). Procedure Install the Operator and create an instance of the aws-load-balancer-controller resource: apiVersion: networking.k8s.io/v1 kind: AWSLoadBalancerController group: networking.olm.openshift.io/v1alpha1 1 metadata: name: cluster spec: subnetTagging: Auto ingressClass: tls-termination 2 1 2 Defines the name of an ingressClass resource reconciled by the AWS Load Balancer Controller. This ingressClass resource gets created if it is not present. You can add additional ingressClass values. The controller reconciles the ingressClass values if the spec.controller is set to ingress.k8s.aws/alb . Create an Ingress resource: apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: <example> 1 annotations: alb.ingress.kubernetes.io/scheme: internet-facing 2 alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-west-2:xxxxx 3 spec: ingressClassName: tls-termination 4 rules: - host: <example.com> 5 http: paths: - path: / pathType: Exact backend: service: name: <example-service> 6 port: number: 80 1 Specifies the name of an ingress. 2 The controller provisions the load balancer for this Ingress resource in a public subnet so that the load balancer is reachable over the internet. 3 The Amazon Resource Name of the certificate that you attach to the load balancer. 4 Defines the ingress class name. 5 Defines the domain for traffic routing. 6 Defines the service for traffic routing.
oc -n aws-load-balancer-operator get sub aws-load-balancer-operator --template='{{.status.installplan.name}}{{"\n"}}' install-zlfbt oc -n aws-load-balancer-operator get ip <install_plan_name> --template='{{.status.phase}}{{"\n"}}' Complete oc get -n aws-load-balancer-operator deployment/aws-load-balancer-operator-controller-manager NAME READY UP-TO-DATE AVAILABLE AGE aws-load-balancer-operator-controller-manager 1/1 1 1 23h oc logs -n aws-load-balancer-operator deployment/aws-load-balancer-operator-controller-manager -c manager apiVersion: networking.olm.openshift.io/v1alpha1 kind: AWSLoadBalancerController 1 metadata: name: cluster 2 spec: subnetTagging: Auto 3 additionalResourceTags: 4 example.org/cost-center: 5113232 example.org/security-scope: staging ingressClass: alb 5 config: replicas: 2 6 enabledAddons: 7 - AWSWAFv2 8 oc create -f sample-aws-lb.yaml apiVersion: apps/v1 kind: Deployment 1 metadata: name: <echoserver> 2 namespace: echoserver spec: selector: matchLabels: app: echoserver replicas: 3 3 template: metadata: labels: app: echoserver spec: containers: - image: openshift/origin-node args: - TCP4-LISTEN:8080,reuseaddr,fork - EXEC:'/bin/bash -c \"printf \\\"HTTP/1.0 200 OK\r\n\r\n\\\"; sed -e \\\"/^\r/q\\\"\"' imagePullPolicy: Always name: echoserver ports: - containerPort: 8080 apiVersion: v1 kind: Service 1 metadata: name: <echoserver> 2 namespace: echoserver spec: ports: - port: 80 targetPort: 8080 protocol: TCP type: NodePort selector: app: echoserver apiVersion: networking.k8s.io/v1 kind: Ingress 1 metadata: name: <echoserver> 2 namespace: echoserver annotations: alb.ingress.kubernetes.io/scheme: internet-facing alb.ingress.kubernetes.io/target-type: instance spec: ingressClassName: alb rules: - http: paths: - path: / pathType: Exact backend: service: name: <echoserver> 3 port: number: 80 HOST=USD(kubectl get ingress -n echoserver echoserver -o json | jq -r '.status.loadBalancer.ingress[0].hostname') curl USDHOST apiVersion: elbv2.k8s.aws/v1beta1 1 kind: IngressClassParams metadata: name: single-lb-params 2 spec: group: name: single-lb 3 oc create -f sample-single-lb-params.yaml apiVersion: networking.k8s.io/v1 1 kind: IngressClass metadata: name: single-lb 2 spec: controller: ingress.k8s.aws/alb 3 parameters: apiGroup: elbv2.k8s.aws 4 kind: IngressClassParams 5 name: single-lb-params 6 oc create -f sample-single-lb-class.yaml apiVersion: networking.olm.openshift.io/v1 kind: AWSLoadBalancerController metadata: name: cluster spec: subnetTagging: Auto ingressClass: single-lb 1 oc create -f sample-single-lb.yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: example-1 1 annotations: alb.ingress.kubernetes.io/scheme: internet-facing 2 alb.ingress.kubernetes.io/group.order: "1" 3 alb.ingress.kubernetes.io/target-type: instance 4 spec: ingressClassName: single-lb 5 rules: - host: example.com 6 http: paths: - path: /blog 7 pathType: Prefix backend: service: name: example-1 8 port: number: 80 9 --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: example-2 annotations: alb.ingress.kubernetes.io/scheme: internet-facing alb.ingress.kubernetes.io/group.order: "2" alb.ingress.kubernetes.io/target-type: instance spec: ingressClassName: single-lb rules: - host: example.com http: paths: - path: /store pathType: Prefix backend: service: name: example-2 port: number: 80 --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: example-3 annotations: alb.ingress.kubernetes.io/scheme: internet-facing alb.ingress.kubernetes.io/group.order: "3" alb.ingress.kubernetes.io/target-type: instance spec: ingressClassName: single-lb rules: - host: example.com http: paths: - path: / pathType: Prefix backend: service: name: example-3 port: number: 80 oc create -f sample-multiple-ingress.yaml apiVersion: networking.k8s.io/v1 kind: AWSLoadBalancerController group: networking.olm.openshift.io/v1alpha1 1 metadata: name: cluster spec: subnetTagging: Auto ingressClass: tls-termination 2 apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: <example> 1 annotations: alb.ingress.kubernetes.io/scheme: internet-facing 2 alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-west-2:xxxxx 3 spec: ingressClassName: tls-termination 4 rules: - host: <example.com> 5 http: paths: - path: / pathType: Exact backend: service: name: <example-service> 6 port: number: 80