title: kubernetes ingress-controller `400 Bad request - plain HTTP request sent to HTTPS port`
body: <p>I have a AWS NLB ingress-controller and an ingress rule which routes traffic between an API and an SPA. The ingress-controller works perfectly on HTTP, but on HTTPS I'm getting a <code>400 Bad request - plain HTTP request sent to HTTPS port</code></p>
  <p>If I understand it correctly, after TLS has been terminated the request is being redirected via an Https port rather than HTTP, but I'm struggling to find where:</p>
  <p>ingress controller.yaml</p>
  <pre><code>
  apiVersion: v1
  kind: Namespace
  metadata:
    name: ingress-nginx
    labels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
  
  ---
  # Source: ingress-nginx/templates/controller-serviceaccount.yaml
  apiVersion: v1
  kind: ServiceAccount
  metadata:
    labels:
      helm.sh/chart: ingress-nginx-2.0.3
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/version: 0.32.0
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/component: controller
    name: ingress-nginx
    namespace: ingress-nginx
  ---
  # Source: ingress-nginx/templates/controller-configmap.yaml
  apiVersion: v1
  kind: ConfigMap
  metadata:
    labels:
      helm.sh/chart: ingress-nginx-2.0.3
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/version: 0.32.0
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/component: controller
    name: ingress-nginx-controller
    namespace: ingress-nginx
  data:
  ---
  # Source: ingress-nginx/templates/clusterrole.yaml
  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRole
  metadata:
    labels:
      helm.sh/chart: ingress-nginx-2.0.3
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/version: 0.32.0
      app.kubernetes.io/managed-by: Helm
    name: ingress-nginx
    namespace: ingress-nginx
  rules:
    - apiGroups:
        - ''
      resources:
        - configmaps
        - endpoints
        - nodes
        - pods
        - secrets
      verbs:
        - list
        - watch
    - apiGroups:
        - ''
      resources:
        - nodes
      verbs:
        - get
    - apiGroups:
        - ''
      resources:
        - services
      verbs:
        - get
        - list
        - update
        - watch
    - apiGroups:
        - extensions
        - networking.k8s.io   # k8s 1.14+
      resources:
        - ingresses
      verbs:
        - get
        - list
        - watch
    - apiGroups:
        - ''
      resources:
        - events
      verbs:
        - create
        - patch
    - apiGroups:
        - extensions
        - networking.k8s.io   # k8s 1.14+
      resources:
        - ingresses/status
      verbs:
        - update
    - apiGroups:
        - networking.k8s.io   # k8s 1.14+
      resources:
        - ingressclasses
      verbs:
        - get
        - list
        - watch
  ---
  # Source: ingress-nginx/templates/clusterrolebinding.yaml
  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRoleBinding
  metadata:
    labels:
      helm.sh/chart: ingress-nginx-2.0.3
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/version: 0.32.0
      app.kubernetes.io/managed-by: Helm
    name: ingress-nginx
    namespace: ingress-nginx
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: ingress-nginx
  subjects:
    - kind: ServiceAccount
      name: ingress-nginx
      namespace: ingress-nginx
  ---
  # Source: ingress-nginx/templates/controller-role.yaml
  apiVersion: rbac.authorization.k8s.io/v1
  kind: Role
  metadata:
    labels:
      helm.sh/chart: ingress-nginx-2.0.3
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/version: 0.32.0
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/component: controller
    name: ingress-nginx
    namespace: ingress-nginx
  rules:
    - apiGroups:
        - ''
      resources:
        - namespaces
      verbs:
        - get
    - apiGroups:
        - ''
      resources:
        - configmaps
        - pods
        - secrets
        - endpoints
      verbs:
        - get
        - list
        - watch
    - apiGroups:
        - ''
      resources:
        - services
      verbs:
        - get
        - list
        - update
        - watch
    - apiGroups:
        - extensions
        - networking.k8s.io   # k8s 1.14+
      resources:
        - ingresses
      verbs:
        - get
        - list
        - watch
    - apiGroups:
        - extensions
        - networking.k8s.io   # k8s 1.14+
      resources:
        - ingresses/status
      verbs:
        - update
    - apiGroups:
        - networking.k8s.io   # k8s 1.14+
      resources:
        - ingressclasses
      verbs:
        - get
        - list
        - watch
    - apiGroups:
        - ''
      resources:
        - configmaps
      resourceNames:
        - ingress-controller-leader-nginx
      verbs:
        - get
        - update
    - apiGroups:
        - ''
      resources:
        - configmaps
      verbs:
        - create
    - apiGroups:
        - ''
      resources:
        - endpoints
      verbs:
        - create
        - get
        - update
    - apiGroups:
        - ''
      resources:
        - events
      verbs:
        - create
        - patch
  ---
  # Source: ingress-nginx/templates/controller-rolebinding.yaml
  apiVersion: rbac.authorization.k8s.io/v1
  kind: RoleBinding
  metadata:
    labels:
      helm.sh/chart: ingress-nginx-2.0.3
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/version: 0.32.0
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/component: controller
    name: ingress-nginx
    namespace: ingress-nginx
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: Role
    name: ingress-nginx
  subjects:
    - kind: ServiceAccount
      name: ingress-nginx
      namespace: ingress-nginx
  ---
  # Source: ingress-nginx/templates/controller-service-webhook.yaml
  apiVersion: v1
  kind: Service
  metadata:
    labels:
      helm.sh/chart: ingress-nginx-2.0.3
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/version: 0.32.0
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/component: controller
    name: ingress-nginx-controller-admission
    namespace: ingress-nginx
  spec:
    type: ClusterIP
    ports:
      - name: https-webhook
        port: 443
        targetPort: webhook
    selector:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/component: controller
  ---
  # Source: ingress-nginx/templates/controller-service.yaml
  apiVersion: v1
  kind: Service
  metadata:
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
      service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: '60'
      service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: 'true'
      service.beta.kubernetes.io/aws-load-balancer-type: nlb
      service.beta.kubernetes.io/aws-load-balancer-ssl-ports: &quot;https&quot;
      service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:us-east-2:XXX:certificate/XXXXX
  
    labels:
      helm.sh/chart: ingress-nginx-2.0.3
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/version: 0.32.0
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/component: controller
    name: ingress-nginx-controller
    namespace: ingress-nginx
  spec:
    type: LoadBalancer
    externalTrafficPolicy: Local
    ports:
      - name: http
        port: 80
        protocol: TCP
        targetPort: http
      - name: https
        port: 443
        protocol: TCP
        targetPort: https
    selector:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/component: controller
  ---
  # Source: ingress-nginx/templates/controller-deployment.yaml
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    labels:
      helm.sh/chart: ingress-nginx-2.0.3
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/version: 0.32.0
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/component: controller
    name: ingress-nginx-controller
    namespace: ingress-nginx
  spec:
    selector:
      matchLabels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/component: controller
    revisionHistoryLimit: 10
    minReadySeconds: 0
    template:
      metadata:
        labels:
          app.kubernetes.io/name: ingress-nginx
          app.kubernetes.io/instance: ingress-nginx
          app.kubernetes.io/component: controller
      spec:
        dnsPolicy: ClusterFirst
        containers:
          - name: controller
            image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.32.0
            imagePullPolicy: IfNotPresent
            lifecycle:
              preStop:
                exec:
                  command:
                    - /wait-shutdown
            args:
              - /nginx-ingress-controller
              - --publish-service=ingress-nginx/ingress-nginx-controller
              - --election-id=ingress-controller-leader
              - --ingress-class=nginx
              - --configmap=ingress-nginx/ingress-nginx-controller
              - --validating-webhook=:8443
              - --validating-webhook-certificate=/usr/local/certificates/cert
              - --validating-webhook-key=/usr/local/certificates/key
            securityContext:
              capabilities:
                drop:
                  - ALL
                add:
                  - NET_BIND_SERVICE
              runAsUser: 101
              allowPrivilegeEscalation: true
            env:
              - name: POD_NAME
                valueFrom:
                  fieldRef:
                    fieldPath: metadata.name
              - name: POD_NAMESPACE
                valueFrom:
                  fieldRef:
                    fieldPath: metadata.namespace
            livenessProbe:
              httpGet:
                path: /healthz
                port: 10254
                scheme: HTTP
              initialDelaySeconds: 10
              periodSeconds: 10
              timeoutSeconds: 1
              successThreshold: 1
              failureThreshold: 3
            readinessProbe:
              httpGet:
                path: /healthz
                port: 10254
                scheme: HTTP
              initialDelaySeconds: 10
              periodSeconds: 10
              timeoutSeconds: 1
              successThreshold: 1
              failureThreshold: 3
            ports:
              - name: http
                containerPort: 80
                protocol: TCP
              - name: https
                containerPort: 443
                protocol: TCP
              - name: webhook
                containerPort: 8443
                protocol: TCP
            volumeMounts:
              - name: webhook-cert
                mountPath: /usr/local/certificates/
                readOnly: true
            resources:
              requests:
                cpu: 100m
                memory: 90Mi
        serviceAccountName: ingress-nginx
        terminationGracePeriodSeconds: 300
        volumes:
          - name: webhook-cert
            secret:
              secretName: ingress-nginx-admission
  ---
  # Source: ingress-nginx/templates/admission-webhooks/validating-webhook.yaml
  apiVersion: admissionregistration.k8s.io/v1
  kind: ValidatingWebhookConfiguration
  metadata:
    labels:
      helm.sh/chart: ingress-nginx-2.0.3
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/version: 0.32.0
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/component: admission-webhook
    name: ingress-nginx-admission
    namespace: ingress-nginx
  webhooks:
    - name: validate.nginx.ingress.kubernetes.io
      rules:
        - apiGroups:
            - extensions
            - networking.k8s.io
          apiVersions:
            - v1beta1
          operations:
            - CREATE
            - UPDATE
          resources:
            - ingresses
      failurePolicy: Fail
      clientConfig:
        service:
          namespace: ingress-nginx
          name: ingress-nginx-controller-admission
          path: /extensions/v1beta1/ingresses
      sideEffects: None
      admissionReviewVersions: [&quot;v1&quot;, &quot;v1beta1&quot;]
  
  ---
  # Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrole.yaml
  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRole
  metadata:
    name: ingress-nginx-admission
    annotations:
      helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
      helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
    labels:
      helm.sh/chart: ingress-nginx-2.0.3
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/version: 0.32.0
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/component: admission-webhook
    namespace: ingress-nginx
  rules:
    - apiGroups:
        - admissionregistration.k8s.io
      resources:
        - validatingwebhookconfigurations
      verbs:
        - get
        - update
  ---
  # Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrolebinding.yaml
  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRoleBinding
  metadata:
    name: ingress-nginx-admission
    annotations:
      helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
      helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
    labels:
      helm.sh/chart: ingress-nginx-2.0.3
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/version: 0.32.0
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/component: admission-webhook
    namespace: ingress-nginx
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: ingress-nginx-admission
  subjects:
    - kind: ServiceAccount
      name: ingress-nginx-admission
      namespace: ingress-nginx
  ---
  # Source: ingress-nginx/templates/admission-webhooks/job-patch/job-createSecret.yaml
  apiVersion: batch/v1
  kind: Job
  metadata:
    name: ingress-nginx-admission-create
    annotations:
      helm.sh/hook: pre-install,pre-upgrade
      helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
    labels:
      helm.sh/chart: ingress-nginx-2.0.3
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/version: 0.32.0
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/component: admission-webhook
    namespace: ingress-nginx
  spec:
    template:
      metadata:
        name: ingress-nginx-admission-create
        labels:
          helm.sh/chart: ingress-nginx-2.0.3
          app.kubernetes.io/name: ingress-nginx
          app.kubernetes.io/instance: ingress-nginx
          app.kubernetes.io/version: 0.32.0
          app.kubernetes.io/managed-by: Helm
          app.kubernetes.io/component: admission-webhook
      spec:
        containers:
          - name: create
            image: jettech/kube-webhook-certgen:v1.2.0
            imagePullPolicy: IfNotPresent
            args:
              - create
              - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.ingress-nginx.svc
              - --namespace=ingress-nginx
              - --secret-name=ingress-nginx-admission
        restartPolicy: OnFailure
        serviceAccountName: ingress-nginx-admission
        securityContext:
          runAsNonRoot: true
          runAsUser: 2000
  ---
  # Source: ingress-nginx/templates/admission-webhooks/job-patch/job-patchWebhook.yaml
  apiVersion: batch/v1
  kind: Job
  metadata:
    name: ingress-nginx-admission-patch
    annotations:
      helm.sh/hook: post-install,post-upgrade
      helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
    labels:
      helm.sh/chart: ingress-nginx-2.0.3
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/version: 0.32.0
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/component: admission-webhook
    namespace: ingress-nginx
  spec:
    template:
      metadata:
        name: ingress-nginx-admission-patch
        labels:
          helm.sh/chart: ingress-nginx-2.0.3
          app.kubernetes.io/name: ingress-nginx
          app.kubernetes.io/instance: ingress-nginx
          app.kubernetes.io/version: 0.32.0
          app.kubernetes.io/managed-by: Helm
          app.kubernetes.io/component: admission-webhook
      spec:
        containers:
          - name: patch
            image: jettech/kube-webhook-certgen:v1.2.0
            imagePullPolicy:
            args:
              - patch
              - --webhook-name=ingress-nginx-admission
              - --namespace=ingress-nginx
              - --patch-mutating=false
              - --secret-name=ingress-nginx-admission
              - --patch-failure-policy=Fail
        restartPolicy: OnFailure
        serviceAccountName: ingress-nginx-admission
        securityContext:
          runAsNonRoot: true
          runAsUser: 2000
  ---
  # Source: ingress-nginx/templates/admission-webhooks/job-patch/role.yaml
  apiVersion: rbac.authorization.k8s.io/v1
  kind: Role
  metadata:
    name: ingress-nginx-admission
    annotations:
      helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
      helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
    labels:
      helm.sh/chart: ingress-nginx-2.0.3
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/version: 0.32.0
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/component: admission-webhook
    namespace: ingress-nginx
  rules:
    - apiGroups:
        - ''
      resources:
        - secrets
      verbs:
        - get
        - create
  ---
  # Source: ingress-nginx/templates/admission-webhooks/job-patch/rolebinding.yaml
  apiVersion: rbac.authorization.k8s.io/v1
  kind: RoleBinding
  metadata:
    name: ingress-nginx-admission
    annotations:
      helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
      helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
    labels:
      helm.sh/chart: ingress-nginx-2.0.3
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/version: 0.32.0
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/component: admission-webhook
    namespace: ingress-nginx
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: Role
    name: ingress-nginx-admission
  subjects:
    - kind: ServiceAccount
      name: ingress-nginx-admission
      namespace: ingress-nginx
  ---
  # Source: ingress-nginx/templates/admission-webhooks/job-patch/serviceaccount.yaml
  apiVersion: v1
  kind: ServiceAccount
  metadata:
    name: ingress-nginx-admission
    annotations:
      helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
      helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
    labels:
      helm.sh/chart: ingress-nginx-2.0.3
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/version: 0.32.0
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/component: admission-webhook
    namespace: ingress-nginx
  </code></pre>
  <p>ingress-rules.yaml</p>
  <pre><code>apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    name: ingress-rules
    namespace: ingress-nginx
    annotations:
      nginx.ingress.kubernetes.io/rewrite-target: &quot;/$1&quot;
  spec:
  #  tls:
  #    - hosts:
  #      - mysite.com
  #      secretName: secret-name
    rules:
  #    - host: mysite.com
      -  http:
          paths:
            - path: /(.*)
              pathType: Prefix
              backend:
                service:
                  name: frontend
                  port:
                    number: 80
            - path: /(api/v0(?:/|$).*)
              pathType: Prefix
              backend:
                service:
                  name: backend
                  port:
                    number: 80
  </code></pre>
  <p>frontend-svc.yaml</p>
  <pre><code>apiVersion: v1
  kind: Service
  metadata:
    name: frontend
    namespace: ingress-nginx
  #  annotations:
  #    service.beta.kubernetes.io/aws-load-balancer-type: 'nlb'
  spec:
    selector:
      app: my-performance
      tier: frontend
    ports:
      - protocol: TCP
        port: 80
        targetPort: 80
    type: LoadBalancer
  </code></pre>
  <p>backend-svc.yaml</p>
  <pre><code>apiVersion: v1
  kind: Service
  metadata:
    name: backend
    namespace: ingress-nginx
  #  annotations:
  #    service.beta.kubernetes.io/aws-load-balancer-type: 'nlb'
  spec:
    selector:
      app: my-performance
      tier: backend
    ports:
      - protocol: TCP
        name: &quot;http&quot;
        port: 80
        targetPort: 8080
    type: LoadBalancer
  </code></pre>
  <p>I do have deployments behind these, but since the services themselves are working fine independently and in conjunction on http, I've ruled them out as the problem here.</p>
  <p>Any advice is greatly appreciated!</p>