title: Error: forwarding ports: error upgrading connection: error dialing backend: - Azure Kubernetes Service
body: <p>We have upgraded our Kubernates Service cluster on Azure to latest version 1.12.4. After that we suddenly  recognize that pods and nodes cannot communicate between anymore by private ip :</p>
  
  <pre><code>kubectl get pods -o wide -n kube-system -l component=kube-proxy
  NAME               READY     STATUS    RESTARTS   AGE       IP           NODE
  kube-proxy-bfhbw   1/1       Running   2          16h       10.0.4.4     aks-agentpool-16086733-1
  kube-proxy-d7fj9   1/1       Running   2          16h       10.0.4.35    aks-agentpool-16086733-0
  kube-proxy-j24th   1/1       Running   2          16h       10.0.4.97    aks-agentpool-16086733-3
  kube-proxy-x7ffx   1/1       Running   2          16h       10.0.4.128   aks-agentpool-16086733-4
  </code></pre>
  
  <p>As you see the node aks-agentpool-16086733-0 has private IP 10.0.4.35 . When we try to check logs on pods which are on this node we got such error:</p>
  
  <blockquote>
    <p>Get
    <a href="https://aks-agentpool-16086733-0:10250/containerLogs/emw-sit/nginx-sit-deploy-864b7d7588-bw966/nginx-sit?tailLines=5000&amp;timestamps=true" rel="nofollow noreferrer">https://aks-agentpool-16086733-0:10250/containerLogs/emw-sit/nginx-sit-deploy-864b7d7588-bw966/nginx-sit?tailLines=5000&amp;timestamps=true</a>: dial tcp 10.0.4.35:10250: i/o timeout</p>
  </blockquote>
  
  <p>We got the Tiller ( Helm) on this node as well, and if try to connect to tiller we got such error from Client PC:</p>
  
  <blockquote>
    <p>shmits-imac:~ andris.shmits01$ helm version Client:
    &amp;version.Version{SemVer:"v2.12.3",
    GitCommit:"eecf22f77df5f65c823aacd2dbd30ae6c65f186e",
    GitTreeState:"clean"} Error: forwarding ports: error upgrading
    connection: error dialing backend: dial tcp 10.0.4.35:10250: i/o
    timeout</p>
  </blockquote>
  
  <p>Does anybody have any idea why the pods and nodes lost connectivity by private IP ? </p>