Chapter 23. AWS Load Balancer Operator
Chapter 23. AWS Load Balancer Operator 23.1. AWS Load Balancer Operator release notes The AWS Load Balancer (ALB) Operator deploys and manages an instance of the AWSLoadBalancerController resource. Important The AWS Load Balancer (ALB) Operator is only supported on the x86_64 architecture. These release notes track the development of the AWS Load Balancer Operator in OpenShift Container Platform. For an overview of the AWS Load Balancer Operator, see AWS Load Balancer Operator in OpenShift Container Platform . Note AWS Load Balancer Operator currently does not support AWS GovCloud. 23.1.1. AWS Load Balancer Operator 1.1.1 The following advisory is available for the AWS Load Balancer Operator version 1.1.1: RHEA-2024:0555 Release of AWS Load Balancer Operator 1.1.z on OperatorHub 23.1.2. AWS Load Balancer Operator 1.1.0 The AWS Load Balancer Operator version 1.1.0 supports the AWS Load Balancer Controller version 2.4.4. The following advisory is available for the AWS Load Balancer Operator version 1.1.0: RHEA-2023:6218 Release of AWS Load Balancer Operator on OperatorHub Enhancement Advisory Update 23.1.2.1. Notable changes This release uses the Kubernetes API version 0.27.2. 23.1.2.2. New features The AWS Load Balancer Operator now supports a standardized Security Token Service (STS) flow by using the Cloud Credential Operator. 23.1.2.3. Bug fixes A FIPS-compliant cluster must use TLS version 1.2. Previously, webhooks for the AWS Load Balancer Controller only accepted TLS 1.3 as the minimum version, resulting in an error such as the following on a FIPS-compliant cluster: remote error: tls: protocol version not supported Now, the AWS Load Balancer Controller accepts TLS 1.2 as the minimum TLS version, resolving this issue. ( OCPBUGS-14846 ) 23.1.3. AWS Load Balancer Operator 1.0.1 The following advisory is available for the AWS Load Balancer Operator version 1.0.1: Release of AWS Load Balancer Operator 1.0.1 on OperatorHub 23.1.4. AWS Load Balancer Operator 1.0.0 The AWS Load Balancer Operator is now generally available with this release. The AWS Load Balancer Operator version 1.0.0 supports the AWS Load Balancer Controller version 2.4.4. The following advisory is available for the AWS Load Balancer Operator version 1.0.0: RHEA-2023:1954 Release of AWS Load Balancer Operator on OperatorHub Enhancement Advisory Update Important The AWS Load Balancer (ALB) Operator version 1.x.x cannot upgrade automatically from the Technology Preview version 0.x.x. To upgrade from an earlier version, you must uninstall the ALB operands and delete the aws-load-balancer-operator namespace. 23.1.4.1. Notable changes This release uses the new v1 API version. 23.1.4.2. Bug fixes Previously, the controller provisioned by the AWS Load Balancer Operator did not properly use the configuration for the cluster-wide proxy. These settings are now applied appropriately to the controller. ( OCPBUGS-4052 , OCPBUGS-5295 ) 23.1.5. Earlier versions The two earliest versions of the AWS Load Balancer Operator are available as a Technology Preview. These versions should not be used in a production cluster. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope . The following advisory is available for the AWS Load Balancer Operator version 0.2.0: RHEA-2022:9084 Release of AWS Load Balancer Operator on OperatorHub Enhancement Advisory Update The following advisory is available for the AWS Load Balancer Operator version 0.0.1: RHEA-2022:5780 Release of AWS Load Balancer Operator on OperatorHub Enhancement Advisory Update 23.2. AWS Load Balancer Operator in OpenShift Container Platform The AWS Load Balancer Operator deploys and manages the AWS Load Balancer Controller. You can install the AWS Load Balancer Operator from OperatorHub by using OpenShift Container Platform web console or CLI. 23.2.1. AWS Load Balancer Operator considerations Review the following limitations before installing and using the AWS Load Balancer Operator: The IP traffic mode only works on AWS Elastic Kubernetes Service (EKS). The AWS Load Balancer Operator disables the IP traffic mode for the AWS Load Balancer Controller. As a result of disabling the IP traffic mode, the AWS Load Balancer Controller cannot use the pod readiness gate. The AWS Load Balancer Operator adds command-line flags such as --disable-ingress-class-annotation and --disable-ingress-group-name-annotation to the AWS Load Balancer Controller. Therefore, the AWS Load Balancer Operator does not allow using the kubernetes.io/ingress.class and alb.ingress.kubernetes.io/group.name annotations in the Ingress resource. You have configured the AWS Load Balancer Operator so that the SVC type is NodePort (not LoadBalancer or ClusterIP ). 23.2.2. AWS Load Balancer Operator The AWS Load Balancer Operator can tag the public subnets if the kubernetes.io/role/elb tag is missing. Also, the AWS Load Balancer Operator detects the following information from the underlying AWS cloud: The ID of the virtual private cloud (VPC) on which the cluster hosting the Operator is deployed in. Public and private subnets of the discovered VPC. The AWS Load Balancer Operator supports the Kubernetes service resource of type LoadBalancer by using Network Load Balancer (NLB) with the instance target type only. Procedure You can deploy the AWS Load Balancer Operator on demand from OperatorHub, by creating a Subscription object by running the following command: USD oc -n aws-load-balancer-operator get sub aws-load-balancer-operator --template='{{.status.installplan.name}}{{"\n"}}' Example output install-zlfbt Check if the status of an install plan is Complete by running the following command: USD oc -n aws-load-balancer-operator get ip <install_plan_name> --template='{{.status.phase}}{{"\n"}}' Example output Complete View the status of the aws-load-balancer-operator-controller-manager deployment by running the following command: USD oc get -n aws-load-balancer-operator deployment/aws-load-balancer-operator-controller-manager Example output NAME READY UP-TO-DATE AVAILABLE AGE aws-load-balancer-operator-controller-manager 1/1 1 1 23h 23.2.3. Using the AWS Load Balancer Operator in an AWS VPC cluster extended into an Outpost You can configure the AWS Load Balancer Operator to provision an AWS Application Load Balancer in an AWS VPC cluster extended into an Outpost. AWS Outposts does not support AWS Network Load Balancers. As a result, the AWS Load Balancer Operator cannot provision Network Load Balancers in an Outpost. You can create an AWS Application Load Balancer either in the cloud subnet or in the Outpost subnet. An Application Load Balancer in the cloud can attach to cloud-based compute nodes and an Application Load Balancer in the Outpost can attach to edge compute nodes. You must annotate Ingress resources with the Outpost subnet or the VPC subnet, but not both. Prerequisites You have extended an AWS VPC cluster into an Outpost. You have installed the OpenShift CLI ( oc ). You have installed the AWS Load Balancer Operator and created the AWS Load Balancer Controller. Procedure Configure the Ingress resource to use a specified subnet: Example Ingress resource configuration apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: <application_name> annotations: alb.ingress.kubernetes.io/subnets: <subnet_id> 1 spec: ingressClassName: alb rules: - http: paths: - path: / pathType: Exact backend: service: name: <application_name> port: number: 80 1 Specifies the subnet to use. To use the Application Load Balancer in an Outpost, specify the Outpost subnet ID. To use the Application Load Balancer in the cloud, you must specify at least two subnets in different availability zones. 23.2.4. AWS Load Balancer Operator logs You can view the AWS Load Balancer Operator logs by using the oc logs command. Procedure View the logs of the AWS Load Balancer Operator by running the following command: USD oc logs -n aws-load-balancer-operator deployment/aws-load-balancer-operator-controller-manager -c manager 23.3. Installing the AWS Load Balancer Operator The AWS Load Balancer Operator deploys and manages the AWS Load Balancer Controller. You can install the AWS Load Balancer Operator from the OperatorHub by using OpenShift Container Platform web console or CLI. 23.3.1. Installing the AWS Load Balancer Operator by using the web console You can install the AWS Load Balancer Operator by using the web console. Prerequisites You have logged in to the OpenShift Container Platform web console as a user with cluster-admin permissions. Your cluster is configured with AWS as the platform type and cloud provider. If you are using a security token service (STS) or user-provisioned infrastructure, follow the related preparation steps. For example, if you are using AWS Security Token Service, see "Preparing for the AWS Load Balancer Operator on a cluster using the AWS Security Token Service (STS)". Procedure Navigate to Operators OperatorHub in the OpenShift Container Platform web console. Select the AWS Load Balancer Operator . You can use the Filter by keyword text box or use the filter list to search for the AWS Load Balancer Operator from the list of Operators. Select the aws-load-balancer-operator namespace. On the Install Operator page, select the following options: Update the channel as stable-v1 . Installation mode as All namespaces on the cluster (default) . Installed Namespace as aws-load-balancer-operator . If the aws-load-balancer-operator namespace does not exist, it gets created during the Operator installation. Select Update approval as Automatic or Manual . By default, the Update approval is set to Automatic . If you select automatic updates, the Operator Lifecycle Manager (OLM) automatically upgrades the running instance of your Operator without any intervention. If you select manual updates, the OLM creates an update request. As a cluster administrator, you must then manually approve that update request to update the Operator updated to the new version. Click Install . Verification Verify that the AWS Load Balancer Operator shows the Status as Succeeded on the Installed Operators dashboard. 23.3.2. Installing the AWS Load Balancer Operator by using the CLI You can install the AWS Load Balancer Operator by using the CLI. Prerequisites You are logged in to the OpenShift Container Platform web console as a user with cluster-admin permissions. Your cluster is configured with AWS as the platform type and cloud provider. You are logged into the OpenShift CLI ( oc ). Procedure Create a Namespace object: Create a YAML file that defines the Namespace object: Example namespace.yaml file apiVersion: v1 kind: Namespace metadata: name: aws-load-balancer-operator Create the Namespace object by running the following command: USD oc apply -f namespace.yaml Create an OperatorGroup object: Create a YAML file that defines the OperatorGroup object: Example operatorgroup.yaml file apiVersion: operators.coreos.com/v1 kind: OperatorGroup metadata: name: aws-lb-operatorgroup namespace: aws-load-balancer-operator spec: upgradeStrategy: Default Create the OperatorGroup object by running the following command: USD oc apply -f operatorgroup.yaml Create a Subscription object: Create a YAML file that defines the Subscription object: Example subscription.yaml file apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: aws-load-balancer-operator namespace: aws-load-balancer-operator spec: channel: stable-v1 installPlanApproval: Automatic name: aws-load-balancer-operator source: redhat-operators sourceNamespace: openshift-marketplace Create the Subscription object by running the following command: USD oc apply -f subscription.yaml Verification Get the name of the install plan from the subscription: USD oc -n aws-load-balancer-operator \ get subscription aws-load-balancer-operator \ --template='{{.status.installplan.name}}{{"\n"}}' Check the status of the install plan: USD oc -n aws-load-balancer-operator \ get ip <install_plan_name> \ --template='{{.status.phase}}{{"\n"}}' The output must be Complete . 23.4. Preparing for the AWS Load Balancer Operator on a cluster using the AWS Security Token Service You can install the AWS Load Balancer Operator on a cluster that uses STS. Follow these steps to prepare your cluster before installing the Operator. The AWS Load Balancer Operator relies on the CredentialsRequest object to bootstrap the Operator and the AWS Load Balancer Controller. The AWS Load Balancer Operator waits until the required secrets are created and available. 23.4.1. Creating an IAM role for the AWS Load Balancer Operator An additional AWS Identity and Access Management (IAM) role is required to successfully install the AWS Load Balancer Operator on a cluster that uses STS. The IAM role is required to interact with subnets and Virtual Private Clouds (VPCs). The AWS Load Balancer Operator generates the CredentialsRequest object with the IAM role to bootstrap itself. You can create the IAM role by using the following options: Using the Cloud Credential Operator utility ( ccoctl ) and a predefined CredentialsRequest object. Using the AWS CLI and predefined AWS manifests. Use the AWS CLI if your environment does not support the ccoctl command. 23.4.1.1. Creating an AWS IAM role by using the Cloud Credential Operator utility You can use the Cloud Credential Operator utility ( ccoctl ) to create an AWS IAM role for the AWS Load Balancer Operator. An AWS IAM role is used to interact with subnets and Virtual Private Clouds (VPCs). Prerequisites You must extract and prepare the ccoctl binary. Procedure Download the CredentialsRequest custom resource (CR) and store it in a directory by running the following command: USD curl --create-dirs -o <credrequests-dir>/operator.yaml https://raw.githubusercontent.com/openshift/aws-load-balancer-operator/main/hack/operator-credentials-request.yaml Use the ccoctl utility to create an AWS IAM role by running the following command: USD ccoctl aws create-iam-roles \ --name <name> \ --region=<aws_region> \ --credentials-requests-dir=<credrequests-dir> \ --identity-provider-arn <oidc-arn> Example output 2023/09/12 11:38:57 Role arn:aws:iam::777777777777:role/<name>-aws-load-balancer-operator-aws-load-balancer-operator created 1 2023/09/12 11:38:57 Saved credentials configuration to: /home/user/<credrequests-dir>/manifests/aws-load-balancer-operator-aws-load-balancer-operator-credentials.yaml 2023/09/12 11:38:58 Updated Role policy for Role <name>-aws-load-balancer-operator-aws-load-balancer-operator created 1 Note the Amazon Resource Name (ARN) of an AWS IAM role. Note The length of an AWS IAM role name must be less than or equal to 12 characters. 23.4.1.2. Creating an AWS IAM role by using the AWS CLI You can use the AWS Command Line Interface to create an IAM role for the AWS Load Balancer Operator. The IAM role is used to interact with subnets and Virtual Private Clouds (VPCs). Prerequisites You must have access to the AWS Command Line Interface ( aws ). Procedure Generate a trust policy file by using your identity provider by running the following command: USD cat <<EOF > albo-operator-trust-policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::777777777777:oidc-provider/<oidc-provider-id>" 1 }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "<oidc-provider-id>:sub": "system:serviceaccount:aws-load-balancer-operator:aws-load-balancer-operator-controller-manager" 2 } } } ] } EOF 1 Specifies the Amazon Resource Name (ARN) of the identity provider. 2 Specifies the service account for the AWS Load Balancer Operator. Create the IAM role with the generated trust policy by running the following command: USD aws iam create-role --role-name albo-operator --assume-role-policy-document file://albo-operator-trust-policy.json Example output ROLE arn:aws:iam::777777777777:role/albo-operator 2023-08-02T12:13:22Z 1 ASSUMEROLEPOLICYDOCUMENT 2012-10-17 STATEMENT sts:AssumeRoleWithWebIdentity Allow STRINGEQUALS system:serviceaccount:aws-load-balancer-operator:aws-load-balancer-controller-manager PRINCIPAL arn:aws:iam:777777777777:oidc-provider/<oidc-provider-id> 1 Note the ARN of the created IAM role. Download the permission policy for the AWS Load Balancer Operator by running the following command: USD curl -o albo-operator-permission-policy.json https://raw.githubusercontent.com/openshift/aws-load-balancer-operator/main/hack/operator-permission-policy.json Attach the permission policy for the AWS Load Balancer Controller to the IAM role by running the following command: USD aws iam put-role-policy --role-name albo-operator --policy-name perms-policy-albo-operator --policy-document file://albo-operator-permission-policy.json 23.4.2. Configuring the ARN role for the AWS Load Balancer Operator You can configure the Amazon Resource Name (ARN) role for the AWS Load Balancer Operator as an environment variable. You can configure the ARN role by using the CLI. Prerequisites You have installed the OpenShift CLI ( oc ). Procedure Create the aws-load-balancer-operator project by running the following command: USD oc new-project aws-load-balancer-operator Create the OperatorGroup object by running the following command: USD cat <<EOF | oc apply -f - apiVersion: operators.coreos.com/v1 kind: OperatorGroup metadata: name: aws-load-balancer-operator namespace: aws-load-balancer-operator spec: targetNamespaces: [] EOF Create the Subscription object by running the following command: USD cat <<EOF | oc apply -f - apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: aws-load-balancer-operator namespace: aws-load-balancer-operator spec: channel: stable-v1 name: aws-load-balancer-operator source: redhat-operators sourceNamespace: openshift-marketplace config: env: - name: ROLEARN value: "<role-arn>" 1 EOF 1 Specifies the ARN role to be used in the CredentialsRequest to provision the AWS credentials for the AWS Load Balancer Operator. Note The AWS Load Balancer Operator waits until the secret is created before moving to the Available status. 23.4.3. Creating an IAM role for the AWS Load Balancer Controller The CredentialsRequest object for the AWS Load Balancer Controller must be set with a manually provisioned IAM role. You can create the IAM role by using the following options: Using the Cloud Credential Operator utility ( ccoctl ) and a predefined CredentialsRequest object. Using the AWS CLI and predefined AWS manifests. Use the AWS CLI if your environment does not support the ccoctl command. 23.4.3.1. Creating an AWS IAM role for the controller by using the Cloud Credential Operator utility You can use the Cloud Credential Operator utility ( ccoctl ) to create an AWS IAM role for the AWS Load Balancer Controller. An AWS IAM role is used to interact with subnets and Virtual Private Clouds (VPCs). Prerequisites You must extract and prepare the ccoctl binary. Procedure Download the CredentialsRequest custom resource (CR) and store it in a directory by running the following command: USD curl --create-dirs -o <credrequests-dir>/controller.yaml https://raw.githubusercontent.com/openshift/aws-load-balancer-operator/main/hack/controller/controller-credentials-request.yaml Use the ccoctl utility to create an AWS IAM role by running the following command: USD ccoctl aws create-iam-roles \ --name <name> \ --region=<aws_region> \ --credentials-requests-dir=<credrequests-dir> \ --identity-provider-arn <oidc-arn> Example output 2023/09/12 11:38:57 Role arn:aws:iam::777777777777:role/<name>-aws-load-balancer-operator-aws-load-balancer-controller created 1 2023/09/12 11:38:57 Saved credentials configuration to: /home/user/<credrequests-dir>/manifests/aws-load-balancer-operator-aws-load-balancer-controller-credentials.yaml 2023/09/12 11:38:58 Updated Role policy for Role <name>-aws-load-balancer-operator-aws-load-balancer-controller created 1 Note the Amazon Resource Name (ARN) of an AWS IAM role. Note The length of an AWS IAM role name must be less than or equal to 12 characters. 23.4.3.2. Creating an AWS IAM role for the controller by using the AWS CLI You can use the AWS command line interface to create an AWS IAM role for the AWS Load Balancer Controller. An AWS IAM role is used to interact with subnets and Virtual Private Clouds (VPCs). Prerequisites You must have access to the AWS command line interface ( aws ). Procedure Generate a trust policy file using your identity provider by running the following command: USD cat <<EOF > albo-controller-trust-policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::777777777777:oidc-provider/<oidc-provider-id>" 1 }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "<oidc-provider-id>:sub": "system:serviceaccount:aws-load-balancer-operator:aws-load-balancer-controller-cluster" 2 } } } ] } EOF 1 Specifies the Amazon Resource Name (ARN) of the identity provider. 2 Specifies the service account for the AWS Load Balancer Controller. Create an AWS IAM role with the generated trust policy by running the following command: USD aws iam create-role --role-name albo-controller --assume-role-policy-document file://albo-controller-trust-policy.json Example output ROLE arn:aws:iam::777777777777:role/albo-controller 2023-08-02T12:13:22Z 1 ASSUMEROLEPOLICYDOCUMENT 2012-10-17 STATEMENT sts:AssumeRoleWithWebIdentity Allow STRINGEQUALS system:serviceaccount:aws-load-balancer-operator:aws-load-balancer-controller-cluster PRINCIPAL arn:aws:iam:777777777777:oidc-provider/<oidc-provider-id> 1 Note the ARN of an AWS IAM role. Download the permission policy for the AWS Load Balancer Controller by running the following command: USD curl -o albo-controller-permission-policy.json https://raw.githubusercontent.com/openshift/aws-load-balancer-operator/main/assets/iam-policy.json Attach the permission policy for the AWS Load Balancer Controller to an AWS IAM role by running the following command: USD aws iam put-role-policy --role-name albo-controller --policy-name perms-policy-albo-controller --policy-document file://albo-controller-permission-policy.json Create a YAML file that defines the AWSLoadBalancerController object: Example sample-aws-lb-manual-creds.yaml file: apiVersion: networking.olm.openshift.io/v1 kind: AWSLoadBalancerController 1 metadata: name: cluster 2 spec: credentialsRequestConfig: stsIAMRoleARN: <role-arn> 3 1 Defines the AWSLoadBalancerController object. 2 Defines the AWS Load Balancer Controller name. All related resources use this instance name as a suffix. 3 Specifies the ARN role. The CredentialsRequest object uses this ARN role to provision the AWS credentials. 23.4.4. Additional resources Configuring the Cloud Credential Operator utility 23.5. Creating an instance of the AWS Load Balancer Controller After installing the AWS Load Balancer Operator, you can create the AWS Load Balancer Controller. 23.5.1. Creating the AWS Load Balancer Controller You can install only a single instance of the AWSLoadBalancerController object in a cluster. You can create the AWS Load Balancer Controller by using CLI. The AWS Load Balancer Operator reconciles only the cluster named resource. Prerequisites You have created the echoserver namespace. You have access to the OpenShift CLI ( oc ). Procedure Create a YAML file that defines the AWSLoadBalancerController object: Example sample-aws-lb.yaml file apiVersion: networking.olm.openshift.io/v1 kind: AWSLoadBalancerController 1 metadata: name: cluster 2 spec: subnetTagging: Auto 3 additionalResourceTags: 4 - key: example.org/security-scope value: staging ingressClass: alb 5 config: replicas: 2 6 enabledAddons: 7 - AWSWAFv2 8 1 Defines the AWSLoadBalancerController object. 2 Defines the AWS Load Balancer Controller name. This instance name gets added as a suffix to all related resources. 3 Configures the subnet tagging method for the AWS Load Balancer Controller. The following values are valid: Auto : The AWS Load Balancer Operator determines the subnets that belong to the cluster and tags them appropriately. The Operator cannot determine the role correctly if the internal subnet tags are not present on internal subnet. Manual : You manually tag the subnets that belong to the cluster with the appropriate role tags. Use this option if you installed your cluster on user-provided infrastructure. 4 Defines the tags used by the AWS Load Balancer Controller when it provisions AWS resources. 5 Defines the ingress class name. The default value is alb . 6 Specifies the number of replicas of the AWS Load Balancer Controller. 7 Specifies annotations as an add-on for the AWS Load Balancer Controller. 8 Enables the alb.ingress.kubernetes.io/wafv2-acl-arn annotation. Create the AWSLoadBalancerController object by running the following command: USD oc create -f sample-aws-lb.yaml Create a YAML file that defines the Deployment resource: Example sample-aws-lb.yaml file apiVersion: apps/v1 kind: Deployment 1 metadata: name: <echoserver> 2 namespace: echoserver spec: selector: matchLabels: app: echoserver replicas: 3 3 template: metadata: labels: app: echoserver spec: containers: - image: openshift/origin-node command: - "/bin/socat" args: - TCP4-LISTEN:8080,reuseaddr,fork - EXEC:'/bin/bash -c \"printf \\\"HTTP/1.0 200 OK\r\n\r\n\\\"; sed -e \\\"/^\r/q\\\"\"' imagePullPolicy: Always name: echoserver ports: - containerPort: 8080 1 Defines the deployment resource. 2 Specifies the deployment name. 3 Specifies the number of replicas of the deployment. Create a YAML file that defines the Service resource: Example service-albo.yaml file: apiVersion: v1 kind: Service 1 metadata: name: <echoserver> 2 namespace: echoserver spec: ports: - port: 80 targetPort: 8080 protocol: TCP type: NodePort selector: app: echoserver 1 Defines the service resource. 2 Specifies the service name. Create a YAML file that defines the Ingress resource: Example ingress-albo.yaml file: apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: <name> 1 namespace: echoserver annotations: alb.ingress.kubernetes.io/scheme: internet-facing alb.ingress.kubernetes.io/target-type: instance spec: ingressClassName: alb rules: - http: paths: - path: / pathType: Exact backend: service: name: <echoserver> 2 port: number: 80 1 Specify a name for the Ingress resource. 2 Specifies the service name. Verification Save the status of the Ingress resource in the HOST variable by running the following command: USD HOST=USD(oc get ingress -n echoserver echoserver --template='{{(index .status.loadBalancer.ingress 0).hostname}}') Verify the status of the Ingress resource by running the following command: USD curl USDHOST 23.6. Serving multiple ingress resources through a single AWS Load Balancer You can route the traffic to different services that are part of a single domain through a single AWS Load Balancer. Each Ingress resource provides different endpoints of the domain. 23.6.1. Creating multiple ingress resources through a single AWS Load Balancer You can route the traffic to multiple ingress resources through a single AWS Load Balancer by using the CLI. Prerequisites You have an access to the OpenShift CLI ( oc ). Procedure Create an IngressClassParams resource YAML file, for example, sample-single-lb-params.yaml , as follows: apiVersion: elbv2.k8s.aws/v1beta1 1 kind: IngressClassParams metadata: name: single-lb-params 2 spec: group: name: single-lb 3 1 Defines the API group and version of the IngressClassParams resource. 2 Specifies the IngressClassParams resource name. 3 Specifies the IngressGroup resource name. All of the Ingress resources of this class belong to this IngressGroup . Create the IngressClassParams resource by running the following command: USD oc create -f sample-single-lb-params.yaml Create the IngressClass resource YAML file, for example, sample-single-lb-class.yaml , as follows: apiVersion: networking.k8s.io/v1 1 kind: IngressClass metadata: name: single-lb 2 spec: controller: ingress.k8s.aws/alb 3 parameters: apiGroup: elbv2.k8s.aws 4 kind: IngressClassParams 5 name: single-lb-params 6 1 Defines the API group and version of the IngressClass resource. 2 Specifies the ingress class name. 3 Defines the controller name. The ingress.k8s.aws/alb value denotes that all ingress resources of this class should be managed by the AWS Load Balancer Controller. 4 Defines the API group of the IngressClassParams resource. 5 Defines the resource type of the IngressClassParams resource. 6 Defines the IngressClassParams resource name. Create the IngressClass resource by running the following command: USD oc create -f sample-single-lb-class.yaml Create the AWSLoadBalancerController resource YAML file, for example, sample-single-lb.yaml , as follows: apiVersion: networking.olm.openshift.io/v1 kind: AWSLoadBalancerController metadata: name: cluster spec: subnetTagging: Auto ingressClass: single-lb 1 1 Defines the name of the IngressClass resource. Create the AWSLoadBalancerController resource by running the following command: USD oc create -f sample-single-lb.yaml Create the Ingress resource YAML file, for example, sample-multiple-ingress.yaml , as follows: apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: example-1 1 annotations: alb.ingress.kubernetes.io/scheme: internet-facing 2 alb.ingress.kubernetes.io/group.order: "1" 3 alb.ingress.kubernetes.io/target-type: instance 4 spec: ingressClassName: single-lb 5 rules: - host: example.com 6 http: paths: - path: /blog 7 pathType: Prefix backend: service: name: example-1 8 port: number: 80 9 --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: example-2 annotations: alb.ingress.kubernetes.io/scheme: internet-facing alb.ingress.kubernetes.io/group.order: "2" alb.ingress.kubernetes.io/target-type: instance spec: ingressClassName: single-lb rules: - host: example.com http: paths: - path: /store pathType: Prefix backend: service: name: example-2 port: number: 80 --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: example-3 annotations: alb.ingress.kubernetes.io/scheme: internet-facing alb.ingress.kubernetes.io/group.order: "3" alb.ingress.kubernetes.io/target-type: instance spec: ingressClassName: single-lb rules: - host: example.com http: paths: - path: / pathType: Prefix backend: service: name: example-3 port: number: 80 1 Specifies the ingress name. 2 Indicates the load balancer to provision in the public subnet to access the internet. 3 Specifies the order in which the rules from the multiple ingress resources are matched when the request is received at the load balancer. 4 Indicates that the load balancer will target OpenShift Container Platform nodes to reach the service. 5 Specifies the ingress class that belongs to this ingress. 6 Defines a domain name used for request routing. 7 Defines the path that must route to the service. 8 Defines the service name that serves the endpoint configured in the Ingress resource. 9 Defines the port on the service that serves the endpoint. Create the Ingress resource by running the following command: USD oc create -f sample-multiple-ingress.yaml 23.7. Adding TLS termination You can add TLS termination on the AWS Load Balancer. 23.7.1. Adding TLS termination on the AWS Load Balancer You can route the traffic for the domain to pods of a service and add TLS termination on the AWS Load Balancer. Prerequisites You have an access to the OpenShift CLI ( oc ). Procedure Create a YAML file that defines the AWSLoadBalancerController resource: Example add-tls-termination-albc.yaml file apiVersion: networking.olm.openshift.io/v1 kind: AWSLoadBalancerController metadata: name: cluster spec: subnetTagging: Auto ingressClass: tls-termination 1 1 Defines the ingress class name. If the ingress class is not present in your cluster the AWS Load Balancer Controller creates one. The AWS Load Balancer Controller reconciles the additional ingress class values if spec.controller is set to ingress.k8s.aws/alb . Create a YAML file that defines the Ingress resource: Example add-tls-termination-ingress.yaml file apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: <example> 1 annotations: alb.ingress.kubernetes.io/scheme: internet-facing 2 alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-west-2:xxxxx 3 spec: ingressClassName: tls-termination 4 rules: - host: <example.com> 5 http: paths: - path: / pathType: Exact backend: service: name: <example-service> 6 port: number: 80 1 Specifies the ingress name. 2 The controller provisions the load balancer for ingress in a public subnet to access the load balancer over the internet. 3 The Amazon Resource Name (ARN) of the certificate that you attach to the load balancer. 4 Defines the ingress class name. 5 Defines the domain for traffic routing. 6 Defines the service for traffic routing. 23.8. Configuring cluster-wide proxy You can configure the cluster-wide proxy in the AWS Load Balancer Operator. After configuring the cluster-wide proxy, Operator Lifecycle Manager (OLM) automatically updates all the deployments of the Operators with the environment variables such as HTTP_PROXY , HTTPS_PROXY , and NO_PROXY . These variables are populated to the managed controller by the AWS Load Balancer Operator. 23.8.1. Trusting the certificate authority of the cluster-wide proxy Create the config map to contain the certificate authority (CA) bundle in the aws-load-balancer-operator namespace by running the following command: USD oc -n aws-load-balancer-operator create configmap trusted-ca To inject the trusted CA bundle into the config map, add the config.openshift.io/inject-trusted-cabundle=true label to the config map by running the following command: USD oc -n aws-load-balancer-operator label cm trusted-ca config.openshift.io/inject-trusted-cabundle=true Update the AWS Load Balancer Operator subscription to access the config map in the AWS Load Balancer Operator deployment by running the following command: USD oc -n aws-load-balancer-operator patch subscription aws-load-balancer-operator --type='merge' -p '{"spec":{"config":{"env":[{"name":"TRUSTED_CA_CONFIGMAP_NAME","value":"trusted-ca"}],"volumes":[{"name":"trusted-ca","configMap":{"name":"trusted-ca"}}],"volumeMounts":[{"name":"trusted-ca","mountPath":"/etc/pki/tls/certs/albo-tls-ca-bundle.crt","subPath":"ca-bundle.crt"}]}}}' After the AWS Load Balancer Operator is deployed, verify that the CA bundle is added to the aws-load-balancer-operator-controller-manager deployment by running the following command: USD oc -n aws-load-balancer-operator exec deploy/aws-load-balancer-operator-controller-manager -c manager -- bash -c "ls -l /etc/pki/tls/certs/albo-tls-ca-bundle.crt; printenv TRUSTED_CA_CONFIGMAP_NAME" Example output -rw-r--r--. 1 root 1000690000 5875 Jan 11 12:25 /etc/pki/tls/certs/albo-tls-ca-bundle.crt trusted-ca Optional: Restart deployment of the AWS Load Balancer Operator every time the config map changes by running the following command: USD oc -n aws-load-balancer-operator rollout restart deployment/aws-load-balancer-operator-controller-manager 23.8.2. Additional resources Certificate injection using Operators
remote error: tls: protocol version not supported oc -n aws-load-balancer-operator get sub aws-load-balancer-operator --template='{{.status.installplan.name}}{{"\n"}}' install-zlfbt oc -n aws-load-balancer-operator get ip <install_plan_name> --template='{{.status.phase}}{{"\n"}}' Complete oc get -n aws-load-balancer-operator deployment/aws-load-balancer-operator-controller-manager NAME READY UP-TO-DATE AVAILABLE AGE aws-load-balancer-operator-controller-manager 1/1 1 1 23h apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: <application_name> annotations: alb.ingress.kubernetes.io/subnets: <subnet_id> 1 spec: ingressClassName: alb rules: - http: paths: - path: / pathType: Exact backend: service: name: <application_name> port: number: 80 oc logs -n aws-load-balancer-operator deployment/aws-load-balancer-operator-controller-manager -c manager apiVersion: v1 kind: Namespace metadata: name: aws-load-balancer-operator oc apply -f namespace.yaml apiVersion: operators.coreos.com/v1 kind: OperatorGroup metadata: name: aws-lb-operatorgroup namespace: aws-load-balancer-operator spec: upgradeStrategy: Default oc apply -f operatorgroup.yaml apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: aws-load-balancer-operator namespace: aws-load-balancer-operator spec: channel: stable-v1 installPlanApproval: Automatic name: aws-load-balancer-operator source: redhat-operators sourceNamespace: openshift-marketplace oc apply -f subscription.yaml oc -n aws-load-balancer-operator get subscription aws-load-balancer-operator --template='{{.status.installplan.name}}{{"\n"}}' oc -n aws-load-balancer-operator get ip <install_plan_name> --template='{{.status.phase}}{{"\n"}}' curl --create-dirs -o <credrequests-dir>/operator.yaml https://raw.githubusercontent.com/openshift/aws-load-balancer-operator/main/hack/operator-credentials-request.yaml ccoctl aws create-iam-roles --name <name> --region=<aws_region> --credentials-requests-dir=<credrequests-dir> --identity-provider-arn <oidc-arn> 2023/09/12 11:38:57 Role arn:aws:iam::777777777777:role/<name>-aws-load-balancer-operator-aws-load-balancer-operator created 1 2023/09/12 11:38:57 Saved credentials configuration to: /home/user/<credrequests-dir>/manifests/aws-load-balancer-operator-aws-load-balancer-operator-credentials.yaml 2023/09/12 11:38:58 Updated Role policy for Role <name>-aws-load-balancer-operator-aws-load-balancer-operator created cat <<EOF > albo-operator-trust-policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::777777777777:oidc-provider/<oidc-provider-id>" 1 }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "<oidc-provider-id>:sub": "system:serviceaccount:aws-load-balancer-operator:aws-load-balancer-operator-controller-manager" 2 } } } ] } EOF aws iam create-role --role-name albo-operator --assume-role-policy-document file://albo-operator-trust-policy.json ROLE arn:aws:iam::777777777777:role/albo-operator 2023-08-02T12:13:22Z 1 ASSUMEROLEPOLICYDOCUMENT 2012-10-17 STATEMENT sts:AssumeRoleWithWebIdentity Allow STRINGEQUALS system:serviceaccount:aws-load-balancer-operator:aws-load-balancer-controller-manager PRINCIPAL arn:aws:iam:777777777777:oidc-provider/<oidc-provider-id> curl -o albo-operator-permission-policy.json https://raw.githubusercontent.com/openshift/aws-load-balancer-operator/main/hack/operator-permission-policy.json aws iam put-role-policy --role-name albo-operator --policy-name perms-policy-albo-operator --policy-document file://albo-operator-permission-policy.json oc new-project aws-load-balancer-operator cat <<EOF | oc apply -f - apiVersion: operators.coreos.com/v1 kind: OperatorGroup metadata: name: aws-load-balancer-operator namespace: aws-load-balancer-operator spec: targetNamespaces: [] EOF cat <<EOF | oc apply -f - apiVersion: operators.coreos.com/v1alpha1 kind: Subscription metadata: name: aws-load-balancer-operator namespace: aws-load-balancer-operator spec: channel: stable-v1 name: aws-load-balancer-operator source: redhat-operators sourceNamespace: openshift-marketplace config: env: - name: ROLEARN value: "<role-arn>" 1 EOF curl --create-dirs -o <credrequests-dir>/controller.yaml https://raw.githubusercontent.com/openshift/aws-load-balancer-operator/main/hack/controller/controller-credentials-request.yaml ccoctl aws create-iam-roles --name <name> --region=<aws_region> --credentials-requests-dir=<credrequests-dir> --identity-provider-arn <oidc-arn> 2023/09/12 11:38:57 Role arn:aws:iam::777777777777:role/<name>-aws-load-balancer-operator-aws-load-balancer-controller created 1 2023/09/12 11:38:57 Saved credentials configuration to: /home/user/<credrequests-dir>/manifests/aws-load-balancer-operator-aws-load-balancer-controller-credentials.yaml 2023/09/12 11:38:58 Updated Role policy for Role <name>-aws-load-balancer-operator-aws-load-balancer-controller created cat <<EOF > albo-controller-trust-policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::777777777777:oidc-provider/<oidc-provider-id>" 1 }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "<oidc-provider-id>:sub": "system:serviceaccount:aws-load-balancer-operator:aws-load-balancer-controller-cluster" 2 } } } ] } EOF aws iam create-role --role-name albo-controller --assume-role-policy-document file://albo-controller-trust-policy.json ROLE arn:aws:iam::777777777777:role/albo-controller 2023-08-02T12:13:22Z 1 ASSUMEROLEPOLICYDOCUMENT 2012-10-17 STATEMENT sts:AssumeRoleWithWebIdentity Allow STRINGEQUALS system:serviceaccount:aws-load-balancer-operator:aws-load-balancer-controller-cluster PRINCIPAL arn:aws:iam:777777777777:oidc-provider/<oidc-provider-id> curl -o albo-controller-permission-policy.json https://raw.githubusercontent.com/openshift/aws-load-balancer-operator/main/assets/iam-policy.json aws iam put-role-policy --role-name albo-controller --policy-name perms-policy-albo-controller --policy-document file://albo-controller-permission-policy.json apiVersion: networking.olm.openshift.io/v1 kind: AWSLoadBalancerController 1 metadata: name: cluster 2 spec: credentialsRequestConfig: stsIAMRoleARN: <role-arn> 3 apiVersion: networking.olm.openshift.io/v1 kind: AWSLoadBalancerController 1 metadata: name: cluster 2 spec: subnetTagging: Auto 3 additionalResourceTags: 4 - key: example.org/security-scope value: staging ingressClass: alb 5 config: replicas: 2 6 enabledAddons: 7 - AWSWAFv2 8 oc create -f sample-aws-lb.yaml apiVersion: apps/v1 kind: Deployment 1 metadata: name: <echoserver> 2 namespace: echoserver spec: selector: matchLabels: app: echoserver replicas: 3 3 template: metadata: labels: app: echoserver spec: containers: - image: openshift/origin-node command: - "/bin/socat" args: - TCP4-LISTEN:8080,reuseaddr,fork - EXEC:'/bin/bash -c \"printf \\\"HTTP/1.0 200 OK\r\n\r\n\\\"; sed -e \\\"/^\r/q\\\"\"' imagePullPolicy: Always name: echoserver ports: - containerPort: 8080 apiVersion: v1 kind: Service 1 metadata: name: <echoserver> 2 namespace: echoserver spec: ports: - port: 80 targetPort: 8080 protocol: TCP type: NodePort selector: app: echoserver apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: <name> 1 namespace: echoserver annotations: alb.ingress.kubernetes.io/scheme: internet-facing alb.ingress.kubernetes.io/target-type: instance spec: ingressClassName: alb rules: - http: paths: - path: / pathType: Exact backend: service: name: <echoserver> 2 port: number: 80 HOST=USD(oc get ingress -n echoserver echoserver --template='{{(index .status.loadBalancer.ingress 0).hostname}}') curl USDHOST apiVersion: elbv2.k8s.aws/v1beta1 1 kind: IngressClassParams metadata: name: single-lb-params 2 spec: group: name: single-lb 3 oc create -f sample-single-lb-params.yaml apiVersion: networking.k8s.io/v1 1 kind: IngressClass metadata: name: single-lb 2 spec: controller: ingress.k8s.aws/alb 3 parameters: apiGroup: elbv2.k8s.aws 4 kind: IngressClassParams 5 name: single-lb-params 6 oc create -f sample-single-lb-class.yaml apiVersion: networking.olm.openshift.io/v1 kind: AWSLoadBalancerController metadata: name: cluster spec: subnetTagging: Auto ingressClass: single-lb 1 oc create -f sample-single-lb.yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: example-1 1 annotations: alb.ingress.kubernetes.io/scheme: internet-facing 2 alb.ingress.kubernetes.io/group.order: "1" 3 alb.ingress.kubernetes.io/target-type: instance 4 spec: ingressClassName: single-lb 5 rules: - host: example.com 6 http: paths: - path: /blog 7 pathType: Prefix backend: service: name: example-1 8 port: number: 80 9 --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: example-2 annotations: alb.ingress.kubernetes.io/scheme: internet-facing alb.ingress.kubernetes.io/group.order: "2" alb.ingress.kubernetes.io/target-type: instance spec: ingressClassName: single-lb rules: - host: example.com http: paths: - path: /store pathType: Prefix backend: service: name: example-2 port: number: 80 --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: example-3 annotations: alb.ingress.kubernetes.io/scheme: internet-facing alb.ingress.kubernetes.io/group.order: "3" alb.ingress.kubernetes.io/target-type: instance spec: ingressClassName: single-lb rules: - host: example.com http: paths: - path: / pathType: Prefix backend: service: name: example-3 port: number: 80 oc create -f sample-multiple-ingress.yaml apiVersion: networking.olm.openshift.io/v1 kind: AWSLoadBalancerController metadata: name: cluster spec: subnetTagging: Auto ingressClass: tls-termination 1 apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: <example> 1 annotations: alb.ingress.kubernetes.io/scheme: internet-facing 2 alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-west-2:xxxxx 3 spec: ingressClassName: tls-termination 4 rules: - host: <example.com> 5 http: paths: - path: / pathType: Exact backend: service: name: <example-service> 6 port: number: 80 oc -n aws-load-balancer-operator create configmap trusted-ca oc -n aws-load-balancer-operator label cm trusted-ca config.openshift.io/inject-trusted-cabundle=true oc -n aws-load-balancer-operator patch subscription aws-load-balancer-operator --type='merge' -p '{"spec":{"config":{"env":[{"name":"TRUSTED_CA_CONFIGMAP_NAME","value":"trusted-ca"}],"volumes":[{"name":"trusted-ca","configMap":{"name":"trusted-ca"}}],"volumeMounts":[{"name":"trusted-ca","mountPath":"/etc/pki/tls/certs/albo-tls-ca-bundle.crt","subPath":"ca-bundle.crt"}]}}}' oc -n aws-load-balancer-operator exec deploy/aws-load-balancer-operator-controller-manager -c manager -- bash -c "ls -l /etc/pki/tls/certs/albo-tls-ca-bundle.crt; printenv TRUSTED_CA_CONFIGMAP_NAME" -rw-r--r--. 1 root 1000690000 5875 Jan 11 12:25 /etc/pki/tls/certs/albo-tls-ca-bundle.crt trusted-ca oc -n aws-load-balancer-operator rollout restart deployment/aws-load-balancer-operator-controller-manager