title: ingress-nginx not working when using ingressClassName instead of kubernetes.io/ingress.class in annotations
body: <p>I have a baremetal cluster deployed using Kubespray with kubernetes 1.22.2, MetalLB, and ingress-nginx enabled. I am getting <code>404 Not found</code> when trying to access any service deployed via helm when setting <code>ingressClassName: nginx</code>. However, everything works fine if I don't use <code>ingressClassName: nginx</code> but <code>kubernetes.io/ingress.class: nginx</code> instead in the helm chart values.yaml. How can I get it to work using <code>ingressClassName</code>?</p>
  <p>These are my kubespray settings for <code>inventory/mycluster/group_vars/k8s_cluster/addons.yml</code></p>
  <pre><code># Nginx ingress controller deployment
  ingress_nginx_enabled: true
  ingress_nginx_host_network: false
  ingress_publish_status_address: &quot;&quot;
  ingress_nginx_nodeselector:
    kubernetes.io/os: &quot;linux&quot;
  ingress_nginx_tolerations:
    - key: &quot;node-role.kubernetes.io/master&quot;
      operator: &quot;Equal&quot;
      value: &quot;&quot;
      effect: &quot;NoSchedule&quot;
    - key: &quot;node-role.kubernetes.io/control-plane&quot;
      operator: &quot;Equal&quot;
      value: &quot;&quot;
      effect: &quot;NoSchedule&quot;
  ingress_nginx_namespace: &quot;ingress-nginx&quot;
  ingress_nginx_insecure_port: 80
  ingress_nginx_secure_port: 443
  ingress_nginx_configmap:
    map-hash-bucket-size: &quot;128&quot;
    ssl-protocols: &quot;TLSv1.2 TLSv1.3&quot;
  ingress_nginx_configmap_tcp_services:
    9000: &quot;default/example-go:8080&quot;
  ingress_nginx_configmap_udp_services:
    53: &quot;kube-system/coredns:53&quot;
  ingress_nginx_extra_args:
    - --default-ssl-certificate=default/mywildcard-tls
  ingress_nginx_class: &quot;nginx&quot;
  </code></pre>
  <p>grafana helm values.yaml</p>
  <pre><code>ingress:
    enabled: true
    # For Kubernetes &gt;= 1.18 you should specify the ingress-controller via the field ingressClassName
    # See https://kubernetes.io/blog/2020/04/02/improvements-to-the-ingress-api-in-kubernetes-1.18/#specifying-the-class-of-an-ingress
    ingressClassName: nginx
    # Values can be templated
    annotations:
    #  kubernetes.io/ingress.class: nginx
    #  kubernetes.io/tls-acme: &quot;true&quot;
    labels: {}
    path: /
  
    # pathType is only for k8s &gt;= 1.1=
    pathType: Prefix
  
    hosts:
      - grafana.mycluster.org
    tls:
     - secretName: mywildcard-tls
       hosts:
         - grafana.mycluster.org
  </code></pre>
  <p><code>kubectl describe pod grafana-679bbfd94-p2dd7</code></p>
  <pre><code>...
  Events:
    Type     Reason     Age                From               Message
    ----     ------     ----               ----               -------
    Normal   Scheduled  25m                default-scheduler  Successfully assigned default/grafana-679bbfd94-p2dd7 to node1
    Normal   Pulled     25m                kubelet            Container image &quot;grafana/grafana:8.2.2&quot; already present on machine
    Normal   Created    25m                kubelet            Created container grafana
    Normal   Started    25m                kubelet            Started container grafana
    Warning  Unhealthy  24m (x3 over 25m)  kubelet            Readiness probe failed: Get &quot;http://10.233.90.33:3000/api/health&quot;: dial tcp 10.233.90.33:3000: connect: connection refused
  </code></pre>
  <p><code>kubectl get svc</code></p>
  <pre><code>NAME         TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
  grafana      LoadBalancer   10.233.14.90   10.10.30.52   80:30285/TCP   55m
  kubernetes   ClusterIP      10.233.0.1     &lt;none&gt;        443/TCP        9d
  </code></pre>
  <p><code>kubectl get ing</code> (no node address assigned)</p>
  <pre><code>NAME      CLASS   HOSTS                    ADDRESS   PORTS     AGE
  grafana   nginx   grafana.mycluster.org             80, 443   25m
  </code></pre>
  <p><code>kubectl describe ing grafana</code> (no node address assigned)</p>
  <pre><code>Name:             grafana
  Namespace:        default
  Address:
  Default backend:  default-http-backend:80 (&lt;error: endpoints &quot;default-http-backend&quot; not found&gt;)
  TLS:
    mywildcard-tls terminates grafana.mycluster.org
  Rules:
    Host                    Path  Backends
    ----                    ----  --------
    grafana.mycluster.org
                            /   grafana:80 (10.233.90.33:3000)
  Annotations:              meta.helm.sh/release-name: grafana
                            meta.helm.sh/release-namespace: default
  Events:                   &lt;none&gt;
  </code></pre>
  <p><code>kubectl get all --all-namespaces</code></p>
  <pre><code>NAMESPACE        NAME                                                              READY   STATUS    RESTARTS   AGE
  default          pod/grafana-b988b9b6-pxccw                                        1/1     Running   0          2m53s
  default          pod/nfs-client-nfs-subdir-external-provisioner-68f44cd9f4-wjlpv   1/1     Running   0          17h
  ingress-nginx    pod/ingress-nginx-controller-6m2vt                                1/1     Running   0          17h
  ingress-nginx    pod/ingress-nginx-controller-xkgxl                                1/1     Running   0          17h
  kube-system      pod/calico-kube-controllers-684bcfdc59-kmsst                      1/1     Running   0          17h
  kube-system      pod/calico-node-dhlnt                                             1/1     Running   0          17h
  kube-system      pod/calico-node-r8ktz                                             1/1     Running   0          17h
  kube-system      pod/coredns-8474476ff8-9sbwh                                      1/1     Running   0          17h
  kube-system      pod/coredns-8474476ff8-fdgcb                                      1/1     Running   0          17h
  kube-system      pod/dns-autoscaler-5ffdc7f89d-vskvq                               1/1     Running   0          17h
  kube-system      pod/kube-apiserver-node1                                          1/1     Running   0          17h
  kube-system      pod/kube-controller-manager-node1                                 1/1     Running   1          17h
  kube-system      pod/kube-proxy-hbjz6                                              1/1     Running   0          16h
  kube-system      pod/kube-proxy-lfqzt                                              1/1     Running   0          16h
  kube-system      pod/kube-scheduler-node1                                          1/1     Running   1          17h
  kube-system      pod/kubernetes-dashboard-548847967d-qqngw                         1/1     Running   0          17h
  kube-system      pod/kubernetes-metrics-scraper-6d49f96c97-2h7hc                   1/1     Running   0          17h
  kube-system      pod/nginx-proxy-node2                                             1/1     Running   0          17h
  kube-system      pod/nodelocaldns-64cqs                                            1/1     Running   0          17h
  kube-system      pod/nodelocaldns-t5vv6                                            1/1     Running   0          17h
  kube-system      pod/registry-proxy-kljvw                                          1/1     Running   0          17h
  kube-system      pod/registry-proxy-nz4qk                                          1/1     Running   0          17h
  kube-system      pod/registry-xzh9d                                                1/1     Running   0          17h
  metallb-system   pod/controller-77c44876d-c92lb                                    1/1     Running   0          17h
  metallb-system   pod/speaker-fkjqp                                                 1/1     Running   0          17h
  metallb-system   pod/speaker-pqjgt                                                 1/1     Running   0          17h
  
  NAMESPACE     NAME                                TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                  AGE
  default       service/grafana                     LoadBalancer   10.233.1.104    10.10.30.52   80:31116/TCP             2m53s
  default       service/kubernetes                  ClusterIP      10.233.0.1      &lt;none&gt;        443/TCP                  17h
  kube-system   service/coredns                     ClusterIP      10.233.0.3      &lt;none&gt;        53/UDP,53/TCP,9153/TCP   17h
  kube-system   service/dashboard-metrics-scraper   ClusterIP      10.233.35.124   &lt;none&gt;        8000/TCP                 17h
  kube-system   service/kubernetes-dashboard        ClusterIP      10.233.32.133   &lt;none&gt;        443/TCP                  17h
  kube-system   service/registry                    ClusterIP      10.233.30.221   &lt;none&gt;        5000/TCP                 17h
  
  NAMESPACE        NAME                                      DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
  ingress-nginx    daemonset.apps/ingress-nginx-controller   2         2         2       2            2           kubernetes.io/os=linux   17h
  kube-system      daemonset.apps/calico-node                2         2         2       2            2           kubernetes.io/os=linux   17h
  kube-system      daemonset.apps/kube-proxy                 2         2         2       2            2           kubernetes.io/os=linux   17h
  kube-system      daemonset.apps/nodelocaldns               2         2         2       2            2           kubernetes.io/os=linux   17h
  kube-system      daemonset.apps/registry-proxy             2         2         2       2            2           &lt;none&gt;                   17h
  metallb-system   daemonset.apps/speaker                    2         2         2       2            2           kubernetes.io/os=linux   17h
  
  NAMESPACE        NAME                                                         READY   UP-TO-DATE   AVAILABLE   AGE
  default          deployment.apps/grafana                                      1/1     1            1           2m53s
  default          deployment.apps/nfs-client-nfs-subdir-external-provisioner   1/1     1            1           17h
  kube-system      deployment.apps/calico-kube-controllers                      1/1     1            1           17h
  kube-system      deployment.apps/coredns                                      2/2     2            2           17h
  kube-system      deployment.apps/dns-autoscaler                               1/1     1            1           17h
  kube-system      deployment.apps/kubernetes-dashboard                         1/1     1            1           17h
  kube-system      deployment.apps/kubernetes-metrics-scraper                   1/1     1            1           17h
  metallb-system   deployment.apps/controller                                   1/1     1            1           17h
  
  NAMESPACE        NAME                                                                    DESIRED   CURRENT   READY   AGE
  default          replicaset.apps/grafana-b988b9b6                                        1         1         1       2m53s
  default          replicaset.apps/nfs-client-nfs-subdir-external-provisioner-68f44cd9f4   1         1         1       17h
  kube-system      replicaset.apps/calico-kube-controllers-684bcfdc59                      1         1         1       17h
  kube-system      replicaset.apps/coredns-8474476ff8                                      2         2         2       17h
  kube-system      replicaset.apps/dns-autoscaler-5ffdc7f89d                               1         1         1       17h
  kube-system      replicaset.apps/kubernetes-dashboard-548847967d                         1         1         1       17h
  kube-system      replicaset.apps/kubernetes-metrics-scraper-6d49f96c97                   1         1         1       17h
  kube-system      replicaset.apps/registry                                                1         1         1       17h
  metallb-system   replicaset.apps/controller-77c44876d                                    1         1         1       17h
  </code></pre>
  <p><code>kubectl get ing grafana -o yaml</code></p>
  <pre><code>apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    annotations:
      kubernetes.io/tls-acme: &quot;true&quot;
      meta.helm.sh/release-name: grafana
      meta.helm.sh/release-namespace: default
      nginx.ingress.kubernetes.io/force-ssl-redirect: &quot;true&quot;
      nginx.ingress.kubernetes.io/ssl-redirect: &quot;true&quot;
    creationTimestamp: &quot;2021-11-11T07:16:12Z&quot;
    generation: 1
    labels:
      app.kubernetes.io/instance: grafana
      app.kubernetes.io/managed-by: Helm
      app.kubernetes.io/name: grafana
      app.kubernetes.io/version: 8.2.2
      helm.sh/chart: grafana-6.17.5
    name: grafana
    namespace: default
    resourceVersion: &quot;3137&quot;
    uid: 6c34d3bd-9ab6-42fe-ac1b-7620a9566f62
  spec:
    ingressClassName: nginx
    rules:
    - host: grafana.mycluster.org
      http:
        paths:
        - backend:
            service:
              name: ssl-redirect
              port:
                name: use-annotation
          path: /*
          pathType: Prefix
        - backend:
            service:
              name: grafana
              port:
                number: 80
          path: /
          pathType: Prefix
  status:
    loadBalancer: {}
  </code></pre>