title: kubernetes services stuck in not ready
body: <p>I have a running container showing running but not ready and I can't for the life of my figure out what's wrong. I am on google container engine running kubectl server 1.64 and kubectl client 1.62</p>
  
  <p>My healthcheck route is indicating that the healthcheck is being requested, albeit directly on the pod ip and not the service ip. I say this because when within a different pod, I am unable to <code>curl</code> the service that is not ready. </p>
  
  <p>Nothing seems to be wrong when I look at my yaml files which I've included below. I've also included some commands I've run and their outputs.</p>
  
  <p>Within the container, I can confirm secrets exist because I can <code>echo $MYSQL_USER</code> so I know they are being set.</p>
  
  <p>Finally, my docker file is extremely simply and runs the server on port 7000 so not sure that could be it. What am I doing wrong? How can I go about debugging this?</p>
  
  <p>from a different pod</p>
  
  <p><strong>wget -qO- blahblah</strong> </p>
  
  <pre><code>wget: can't connect to remote host (10.55.252.109): Operation timed out
  </code></pre>
  
  <p><strong>nslookup blahblah</strong>:</p>
  
  <pre><code>nslookup: can't resolve '(null)': Name does not resolve
  
  Name:      blahblah-service
  Address 1: 10.55.252.109 blahblah-service.staging.svc.cluster.local
  </code></pre>
  
  <p>from within the gke vm</p>
  
  <p><strong>curl 10.55.252.109</strong></p>
  
  <pre><code>Failed to connect to 10.55.252.109 port 80: Connection refused
  </code></pre>
  
  <p>from my terminal:</p>
  
  <p><strong>kubectl get svc blahblah</strong></p>
  
  <pre><code>NAME                       CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
  blahblah-service   10.55.252.109   &lt;nodes&gt;       80:31003/TCP   41m
  </code></pre>
  
  <p><strong>kubectl get endpoints blahblah-service</strong></p>
  
  <pre><code>NAME                       ENDPOINTS   AGE
  blahblah-service               29m
  </code></pre>
  
  <p><strong>kubectl get pods -l name=blahblah</strong></p>
  
  <pre><code>NAME                                           READY     STATUS    RESTARTS   AGE
  blahblah-3521139244-6tkj5   0/1       Running   0          29m
  </code></pre>
  
  <p><strong>kubectl describe svc blahblah-service</strong></p>
  
  <pre><code>Name:           blahblah-service
  Namespace:          staging
  Labels:         &lt;none&gt;
  Annotations:        &lt;none&gt;
  Selector:           name=blahblah
  Type:           NodePort
  IP:             10.55.252.109
  Port:           http    80/TCP
  NodePort:           http    31003/TCP
  Endpoints:
  Session Affinity:       None
  Events:         &lt;none&gt;
  </code></pre>
  
  <p>service &amp; deployment yamls</p>
  
  <pre><code>kind: Service
  apiVersion: v1
  metadata:
    name: blahblah-service
  spec:
    selector:
      name: blahblah
    type: NodePort
    ports:
      - port: 80
        targetPort: "http"
        name: http
  
  apiVersion: extensions/v1beta1
  kind: Deployment
  metadata:
    name: blahblah-deployment
  spec:
    replicas: 1
    minReadySeconds: 30
    template:
      metadata:
        labels:
          name: blahblah
      spec:
        containers:
        - name: blahblah
          image: eu.gcr.io/company_name/blahblah-service
          ports:
            - containerPort: 7000
              name: http
          readinessProbe:
            httpGet:
              path: /_internal_/ok
              port: http
          env:
            - name: NODE_ENV
              valueFrom:
                configMapKeyRef:
                  name: env-variables
                  key: ENV_NAME
            - name: MYSQL_USER
              valueFrom:
                secretKeyRef:
                  name: blahblah-mysql-secrets
                  key: MYSQL_USER
            - name: MYSQL_PASS
              valueFrom:
                secretKeyRef:
                  name: blahblah-mysql-secrets
                  key: MYSQL_PASS
  </code></pre>
  
  <p>dockerfile</p>
  
  <pre><code>FROM node:7.5.0-alpine
  
  RUN apk update &amp;&amp; apk upgrade \
      &amp;&amp; apk add --no-cache --update-cache git ca-certificates openssl\
      &amp;&amp; update-ca-certificates
  
  RUN cd /tmp &amp;&amp; wget https://yarnpkg.com/latest.tar.gz &amp;&amp; tar zxf latest.tar.gz
  
  COPY package.json /workspace/package.json
  WORKDIR /workspace
  COPY yarn.lock /workspace/yarn.lock
  RUN /tmp/dist/bin/yarn
  
  COPY . /workspace
  
  ARG NODE_ENV=production
  ENV NODE_ENV ${NODE_ENV}
  
  ENV PORT 7000
  
  EXPOSE $PORT
  
  ENTRYPOINT exec node_modules/pm2/bin/pm2-docker start pm2.json --only blahblah-service-$NODE_ENV
  </code></pre>
  
  <p><strong>EDIT</strong></p>
  
  <p>I also noticed that it only shows not ready when another service is run at the same time. I don't see any similarities between the two services so I'm not sure how running both would conflict? Both services use a configmap and have their own secrets but I don't see why that would cause conflicts. </p>