title: Helm chart stuck in PodInitializing state indefinitely
body: <p>I'm running a <a href="https://microk8s.io/" rel="nofollow noreferrer">microk8s</a> cluster on an Ubuntu server at home, and I have it connected to a local NAS server for persistent storage. I've been using it as my personal proving grounds for learning Kubernetes, but I seem to encounter problem after problem at just about every step of the way.</p>
  
  <p>I've got the <a href="https://github.com/helm/charts/tree/master/stable/nfs-client-provisioner" rel="nofollow noreferrer">NFS Client Provisioner</a> Helm chart installed which I've confirmed works - it will dynamically provision PVCs on my NAS server. I later was able to successfully install the <a href="https://github.com/helm/charts/tree/master/stable/postgresql" rel="nofollow noreferrer">Postgres</a> Helm chart, or so I thought. After creating it I was able to connect to it using a SQL client, and I was feeling good.</p>
  
  <p>Until a couple of days later, I noticed the pod was showing 0/1 containers ready. Although interestingly, the nfs-client-provisioner pod was still showing 1/1. Long story short: I've deleted/purged the Postgres Helm chart, and attempted to reinstall it, but now it no longer works. In fact, nothing new that I try to deploy works. Everything looks as though it's going to work, but then just hangs on either Init or ContainerCreating forever.</p>
  
  <p>With Postgres in particular, the command I've been running is this:</p>
  
  <pre><code>helm install --name postgres stable/postgresql -f postgres.yaml
  </code></pre>
  
  <p>And my <code>postgres.yaml</code> file looks like this:</p>
  
  <pre><code>persistence:
      storageClass: nfs-client
      accessMode: ReadWriteMany
      size: 2Gi
  </code></pre>
  
  <p>But if I do a <code>kubectl get pods</code> I see still see this:</p>
  
  <pre><code>NAME                    READY  STATUS    RESTARTS  AGE
  nfs-client-provisioner  1/1    Running   1         11d
  postgres-postgresql-0   0/1    Init:0/1  0         3h51m
  </code></pre>
  
  <p>If I do a <code>kubectl describe pod postgres-postgresql-0</code>, this is the output:</p>
  
  <pre><code>Name:               postgres-postgresql-0
  Namespace:          default
  Priority:           0
  PriorityClassName:  &lt;none&gt;
  Node:               stjohn/192.168.1.217
  Start Time:         Thu, 28 Mar 2019 12:51:02 -0500
  Labels:             app=postgresql
                  chart=postgresql-3.11.7
                  controller-revision-hash=postgres-postgresql-5bfb9cc56d
                  heritage=Tiller
                  release=postgres
                  role=master
                  statefulset.kubernetes.io/pod-name=postgres-postgresql-0
  Annotations:        &lt;none&gt;
  Status:             Pending
  IP:                 
  Controlled By:      StatefulSet/postgres-postgresql
  Init Containers:
    init-chmod-data:
      Container ID:  
      Image:         docker.io/bitnami/minideb:latest
      Image ID:      
      Port:          &lt;none&gt;
      Host Port:     &lt;none&gt;
      Command:
        sh
        -c
        chown -R 1001:1001 /bitnami
        if [ -d /bitnami/postgresql/data ]; then
      chmod  0700 /bitnami/postgresql/data;
        fi
  
      State:          Waiting
        Reason:       PodInitializing
      Ready:          False
      Restart Count:  0
      Requests:
        cpu:        250m
        memory:     256Mi
      Environment:  &lt;none&gt;
      Mounts:
        /bitnami/postgresql from data (rw)
        /var/run/secrets/kubernetes.io/serviceaccount from default-token-h4gph (ro)
  Containers:
    postgres-postgresql:
      Container ID:   
      Image:          docker.io/bitnami/postgresql:10.7.0
      Image ID:       
      Port:           5432/TCP
      Host Port:      0/TCP
      State:          Waiting
        Reason:       PodInitializing
      Ready:          False
      Restart Count:  0
      Requests:
        cpu:      250m
        memory:   256Mi
      Liveness:   exec [sh -c exec pg_isready -U "postgres" -h localhost] delay=30s timeout=5s period=10s #success=1 #failure=6
      Readiness:  exec [sh -c exec pg_isready -U "postgres" -h localhost] delay=5s timeout=5s period=10s #success=1 #failure=6
      Environment:
        PGDATA:             /bitnami/postgresql
        POSTGRES_USER:      postgres
        POSTGRES_PASSWORD:  &lt;set to the key 'postgresql-password' in secret 'postgres-postgresql'&gt;  Optional: false
      Mounts:
        /bitnami/postgresql from data (rw)
        /var/run/secrets/kubernetes.io/serviceaccount from default-token-h4gph (ro)
  Conditions:
    Type              Status
    Initialized       False 
    Ready             False 
    ContainersReady   False 
    PodScheduled      True 
  Volumes:
    data:
      Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
      ClaimName:  data-postgres-postgresql-0
      ReadOnly:   false
    default-token-h4gph:
      Type:        Secret (a volume populated by a Secret)
      SecretName:  default-token-h4gph
      Optional:    false
  QoS Class:       Burstable
  Node-Selectors:  &lt;none&gt;
  Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
               node.kubernetes.io/unreachable:NoExecute for 300s
  Events:          &lt;none&gt;
  </code></pre>
  
  <p>And if I do a <code>kubectl get pod postgres-postgresql-0 -o yaml</code>, this is the output:</p>
  
  <pre><code>apiVersion: v1
  kind: Pod
  metadata:
    creationTimestamp: "2019-03-28T17:51:02Z"
    generateName: postgres-postgresql-
    labels:
      app: postgresql
      chart: postgresql-3.11.7
      controller-revision-hash: postgres-postgresql-5bfb9cc56d
      heritage: Tiller
      release: postgres
      role: master
      statefulset.kubernetes.io/pod-name: postgres-postgresql-0
    name: postgres-postgresql-0
    namespace: default
    ownerReferences:
    - apiVersion: apps/v1
      blockOwnerDeletion: true
      controller: true
      kind: StatefulSet
      name: postgres-postgresql
      uid: 0d3ef673-5182-11e9-bf14-b8975a0ca30c
    resourceVersion: "1953329"
    selfLink: /api/v1/namespaces/default/pods/postgres-postgresql-0
    uid: 0d4dfb56-5182-11e9-bf14-b8975a0ca30c
  spec:
    containers:
    - env:
      - name: PGDATA
        value: /bitnami/postgresql
      - name: POSTGRES_USER
        value: postgres
      - name: POSTGRES_PASSWORD
        valueFrom:
      secretKeyRef:
        key: postgresql-password
        name: postgres-postgresql
      image: docker.io/bitnami/postgresql:10.7.0
      imagePullPolicy: Always
      livenessProbe:
        exec:
      command:
      - sh
      - -c
      - exec pg_isready -U "postgres" -h localhost
        failureThreshold: 6
        initialDelaySeconds: 30
        periodSeconds: 10
        successThreshold: 1
        timeoutSeconds: 5
      name: postgres-postgresql
      ports:
      - containerPort: 5432
        name: postgresql
        protocol: TCP
      readinessProbe:
        exec:
      command:
      - sh
      - -c
      - exec pg_isready -U "postgres" -h localhost
        failureThreshold: 6
        initialDelaySeconds: 5
        periodSeconds: 10
        successThreshold: 1
        timeoutSeconds: 5
      resources:
        requests:
      cpu: 250m
      memory: 256Mi
      securityContext:
        procMount: Default
        runAsUser: 1001
      terminationMessagePath: /dev/termination-log
      terminationMessagePolicy: File
      volumeMounts:
      - mountPath: /bitnami/postgresql
        name: data
      - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
        name: default-token-h4gph
        readOnly: true
    dnsPolicy: ClusterFirst
    enableServiceLinks: true
    hostname: postgres-postgresql-0
    initContainers:
    - command:
      - sh
      - -c
      - |
        chown -R 1001:1001 /bitnami
        if [ -d /bitnami/postgresql/data ]; then
      chmod  0700 /bitnami/postgresql/data;
        fi
      image: docker.io/bitnami/minideb:latest
      imagePullPolicy: Always
      name: init-chmod-data
      resources:
        requests:
      cpu: 250m
      memory: 256Mi
      securityContext:
        procMount: Default
        runAsUser: 0
      terminationMessagePath: /dev/termination-log
      terminationMessagePolicy: File
      volumeMounts:
      - mountPath: /bitnami/postgresql
        name: data
      - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
        name: default-token-h4gph
        readOnly: true
    nodeName: stjohn
    priority: 0
    restartPolicy: Always
    schedulerName: default-scheduler
    securityContext:
      fsGroup: 1001
    serviceAccount: default
    serviceAccountName: default
    subdomain: postgres-postgresql-headless
    terminationGracePeriodSeconds: 30
    tolerations:
    - effect: NoExecute
      key: node.kubernetes.io/not-ready
      operator: Exists
      tolerationSeconds: 300
    - effect: NoExecute
      key: node.kubernetes.io/unreachable
      operator: Exists
      tolerationSeconds: 300
    volumes:
    - name: data
      persistentVolumeClaim:
        claimName: data-postgres-postgresql-0
    - name: default-token-h4gph
      secret:
        defaultMode: 420
        secretName: default-token-h4gph
  status:
    conditions:
    - lastProbeTime: null
      lastTransitionTime: "2019-03-28T17:51:02Z"
      message: 'containers with incomplete status: [init-chmod-data]'
      reason: ContainersNotInitialized
      status: "False"
      type: Initialized
    - lastProbeTime: null
      lastTransitionTime: "2019-03-28T17:51:02Z"
      message: 'containers with unready status: [postgres-postgresql]'
      reason: ContainersNotReady
      status: "False"
      type: Ready
    - lastProbeTime: null
      lastTransitionTime: "2019-03-28T17:51:02Z"
      message: 'containers with unready status: [postgres-postgresql]'
      reason: ContainersNotReady
      status: "False"
      type: ContainersReady
    - lastProbeTime: null
      lastTransitionTime: "2019-03-28T17:51:02Z"
      status: "True"
      type: PodScheduled
    containerStatuses:
    - image: docker.io/bitnami/postgresql:10.7.0
      imageID: ""
      lastState: {}
      name: postgres-postgresql
      ready: false
      restartCount: 0
      state:
        waiting:
      reason: PodInitializing
    hostIP: 192.168.1.217
    initContainerStatuses:
    - image: docker.io/bitnami/minideb:latest
      imageID: ""
      lastState: {}
      name: init-chmod-data
      ready: false
      restartCount: 0
      state:
        waiting:
      reason: PodInitializing
    phase: Pending
    qosClass: Burstable
    startTime: "2019-03-28T17:51:02Z"
  </code></pre>
  
  <p>I don't see anything obvious in these to be able to pinpoint what might be going on. And I've already rebooted the server just to see if that might help. Any thoughts? Why won't my containers start?</p>