<p dir="auto">From the master, whenever I try to open a shell into a pod running on a remote node, kubectl would completely hang.<br>
Below  a verbose trace.</p>
<p dir="auto">Can someone point me to where else I can look?<br>
I know kubectl --&gt; master --&gt; proxy --&gt; pod/container.<br>
Where would this error come from.<br>
This has always worked prior to upgrading to kubernetes 1.1.7 and docker 1.10.<br>
I'm going to downgrade one at a time to see which upgrade broke it.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="kubectl exec --v=9 kibana-ndeps  bash -it
I0209 03:16:35.628245       1 debugging.go:102] curl -k -v -XGET  -H &quot;User-Agent: kubectl/v1.1.7 (linux/amd64) kubernetes/e4e6878&quot; http://minux:8080/api
I0209 03:16:35.639050       1 debugging.go:121] GET http://minux:8080/api 200 OK in 10 milliseconds
I0209 03:16:35.639064       1 debugging.go:127] Response Headers:
I0209 03:16:35.639069       1 debugging.go:130]     Content-Type: application/json
I0209 03:16:35.639074       1 debugging.go:130]     Date: Tue, 09 Feb 2016 03:16:35 GMT
I0209 03:16:35.639079       1 debugging.go:130]     Content-Length: 57
I0209 03:16:35.639093       1 request.go:746] Response Body: {
  &quot;kind&quot;: &quot;APIVersions&quot;,
  &quot;versions&quot;: [
    &quot;v1&quot;
  ]
}
I0209 03:16:35.639218       1 debugging.go:102] curl -k -v -XGET  -H &quot;User-Agent: kubectl/v1.1.7 (linux/amd64) kubernetes/e4e6878&quot; http://minux:8080/api/v1/namespaces/default/pods/kibana-ndeps
I0209 03:16:35.640083       1 debugging.go:121] GET http://minux:8080/api/v1/namespaces/default/pods/kibana-ndeps 200 OK in 0 milliseconds
I0209 03:16:35.640096       1 debugging.go:127] Response Headers:
I0209 03:16:35.640101       1 debugging.go:130]     Content-Type: application/json
I0209 03:16:35.640106       1 debugging.go:130]     Date: Tue, 09 Feb 2016 03:16:35 GMT
I0209 03:16:35.640130       1 request.go:746] Response Body: {&quot;kind&quot;:&quot;Pod&quot;,&quot;apiVersion&quot;:&quot;v1&quot;,&quot;metadata&quot;:{&quot;name&quot;:&quot;kibana-ndeps&quot;,&quot;generateName&quot;:&quot;kibana-&quot;,&quot;namespace&quot;:&quot;default&quot;,&quot;selfLink&quot;:&quot;/api/v1/namespaces/default/pods/kibana-ndeps&quot;,&quot;uid&quot;:&quot;a4649aad-ced8-11e5-80b6-7824afc0a170&quot;,&quot;resourceVersion&quot;:&quot;13142485&quot;,&quot;creationTimestamp&quot;:&quot;2016-02-09T02:55:57Z&quot;,&quot;labels&quot;:{&quot;name&quot;:&quot;kibana&quot;},&quot;annotations&quot;:{&quot;kubernetes.io/created-by&quot;:&quot;{\&quot;kind\&quot;:\&quot;SerializedReference\&quot;,\&quot;apiVersion\&quot;:\&quot;v1\&quot;,\&quot;reference\&quot;:{\&quot;kind\&quot;:\&quot;ReplicationController\&quot;,\&quot;namespace\&quot;:\&quot;default\&quot;,\&quot;name\&quot;:\&quot;kibana\&quot;,\&quot;uid\&quot;:\&quot;4a4de2fe-c888-11e5-b7f7-7824afc0a170\&quot;,\&quot;apiVersion\&quot;:\&quot;v1\&quot;,\&quot;resourceVersion\&quot;:\&quot;12885522\&quot;}}\n&quot;}},&quot;spec&quot;:{&quot;volumes&quot;:[{&quot;name&quot;:&quot;log&quot;,&quot;emptyDir&quot;:{}}],&quot;containers&quot;:[{&quot;name&quot;:&quot;kibana&quot;,&quot;image&quot;:&quot;minux:5000/kibana:latest&quot;,&quot;env&quot;:[{&quot;name&quot;:&quot;ELASTICSEARCH_HOST&quot;,&quot;value&quot;:&quot;elasticsearch&quot;}],&quot;resources&quot;:{},&quot;volumeMounts&quot;:[{&quot;name&quot;:&quot;log&quot;,&quot;mountPath&quot;:&quot;/var/log&quot;}],&quot;readinessProbe&quot;:{&quot;httpGet&quot;:{&quot;path&quot;:&quot;/&quot;,&quot;port&quot;:5601,&quot;scheme&quot;:&quot;HTTP&quot;},&quot;initialDelaySeconds&quot;:30,&quot;timeoutSeconds&quot;:1},&quot;terminationMessagePath&quot;:&quot;/dev/termination-log&quot;,&quot;imagePullPolicy&quot;:&quot;Always&quot;},{&quot;name&quot;:&quot;fluentd-kafka&quot;,&quot;image&quot;:&quot;minux:5000/fluentd-kafka:latest&quot;,&quot;env&quot;:[{&quot;name&quot;:&quot;ZOOKEEPER&quot;,&quot;value&quot;:&quot;zookeeper:2181&quot;},{&quot;name&quot;:&quot;LABEL&quot;,&quot;value&quot;:&quot;KIBANA&quot;}],&quot;resources&quot;:{},&quot;volumeMounts&quot;:[{&quot;name&quot;:&quot;log&quot;,&quot;mountPath&quot;:&quot;/var/log&quot;}],&quot;terminationMessagePath&quot;:&quot;/dev/termination-log&quot;,&quot;imagePullPolicy&quot;:&quot;Always&quot;}],&quot;restartPolicy&quot;:&quot;Always&quot;,&quot;terminationGracePeriodSeconds&quot;:30,&quot;dnsPolicy&quot;:&quot;ClusterFirst&quot;,&quot;nodeSelector&quot;:{&quot;zone&quot;:&quot;green&quot;},&quot;nodeName&quot;:&quot;192.168.1.176&quot;},&quot;status&quot;:{&quot;phase&quot;:&quot;Running&quot;,&quot;conditions&quot;:[{&quot;type&quot;:&quot;Ready&quot;,&quot;status&quot;:&quot;True&quot;,&quot;lastProbeTime&quot;:null,&quot;lastTransitionTime&quot;:null}],&quot;hostIP&quot;:&quot;192.168.1.176&quot;,&quot;podIP&quot;:&quot;10.244.176.4&quot;,&quot;startTime&quot;:&quot;2016-02-09T02:55:59Z&quot;,&quot;containerStatuses&quot;:[{&quot;name&quot;:&quot;fluentd-kafka&quot;,&quot;state&quot;:{&quot;running&quot;:{&quot;startedAt&quot;:&quot;2016-02-09T02:58:38Z&quot;}},&quot;lastState&quot;:{},&quot;ready&quot;:true,&quot;restartCount&quot;:0,&quot;image&quot;:&quot;minux:5000/fluentd-kafka:latest&quot;,&quot;imageID&quot;:&quot;docker://sha256:0948b0085872577fa50fc415feda363178bba8361834adae79952aee7431492f&quot;,&quot;containerID&quot;:&quot;docker://e1f509c8fe3f6004c81b26f42c2cfd0c389afef459ac67797ed60f162e30d960&quot;},{&quot;name&quot;:&quot;kibana&quot;,&quot;state&quot;:{&quot;running&quot;:{&quot;startedAt&quot;:&quot;2016-02-09T02:58:38Z&quot;}},&quot;lastState&quot;:{},&quot;ready&quot;:true,&quot;restartCount&quot;:0,&quot;image&quot;:&quot;minux:5000/kibana:latest&quot;,&quot;imageID&quot;:&quot;docker://sha256:8b09cbc206d53a799211a4d96ff5e06d19f216be258a8a4dec7204d5911bc18d&quot;,&quot;containerID&quot;:&quot;docker://d7c6bc51d4f70074c89ca21fdc65d075acb322f66fd52c2b7bf5a5cce479c823&quot;}]}}
I0209 03:16:35.640649       1 exec.go:179] defaulting container name to kibana
I0209 03:16:35.640722       1 debugging.go:102] curl -k -v -XPOST  -H &quot;X-Stream-Protocol-Version: v2.channel.k8s.io&quot; -H &quot;X-Stream-Protocol-Version: channel.k8s.io&quot; http://minux:8080/api/v1/namespaces/default/pods/kibana-ndeps/exec?command=bash&amp;container=kibana&amp;container=kibana&amp;stderr=true&amp;stdin=true&amp;stdout=true&amp;tty=true
I0209 03:16:35.668922       1 debugging.go:121] POST http://minux:8080/api/v1/namespaces/default/pods/kibana-ndeps/exec?command=bash&amp;container=kibana&amp;container=kibana&amp;stderr=true&amp;stdin=true&amp;stdout=true&amp;tty=true 101 Switching Protocols in 28 milliseconds
I0209 03:16:35.668936       1 debugging.go:127] Response Headers:
I0209 03:16:35.668941       1 debugging.go:130]     Connection: Upgrade
I0209 03:16:35.668947       1 debugging.go:130]     Upgrade: SPDY/3.1
I0209 03:16:35.668955       1 debugging.go:130]     X-Stream-Protocol-Version: v2.channel.k8s.io
I0209 03:16:35.668962       1 debugging.go:130]     Date: Tue, 09 Feb 2016 03:16:37 GMT
Content-Type specified (plain/text) must be 'application/json'"><pre class="notranslate"><code class="notranslate">kubectl exec --v=9 kibana-ndeps  bash -it
I0209 03:16:35.628245       1 debugging.go:102] curl -k -v -XGET  -H "User-Agent: kubectl/v1.1.7 (linux/amd64) kubernetes/e4e6878" http://minux:8080/api
I0209 03:16:35.639050       1 debugging.go:121] GET http://minux:8080/api 200 OK in 10 milliseconds
I0209 03:16:35.639064       1 debugging.go:127] Response Headers:
I0209 03:16:35.639069       1 debugging.go:130]     Content-Type: application/json
I0209 03:16:35.639074       1 debugging.go:130]     Date: Tue, 09 Feb 2016 03:16:35 GMT
I0209 03:16:35.639079       1 debugging.go:130]     Content-Length: 57
I0209 03:16:35.639093       1 request.go:746] Response Body: {
  "kind": "APIVersions",
  "versions": [
    "v1"
  ]
}
I0209 03:16:35.639218       1 debugging.go:102] curl -k -v -XGET  -H "User-Agent: kubectl/v1.1.7 (linux/amd64) kubernetes/e4e6878" http://minux:8080/api/v1/namespaces/default/pods/kibana-ndeps
I0209 03:16:35.640083       1 debugging.go:121] GET http://minux:8080/api/v1/namespaces/default/pods/kibana-ndeps 200 OK in 0 milliseconds
I0209 03:16:35.640096       1 debugging.go:127] Response Headers:
I0209 03:16:35.640101       1 debugging.go:130]     Content-Type: application/json
I0209 03:16:35.640106       1 debugging.go:130]     Date: Tue, 09 Feb 2016 03:16:35 GMT
I0209 03:16:35.640130       1 request.go:746] Response Body: {"kind":"Pod","apiVersion":"v1","metadata":{"name":"kibana-ndeps","generateName":"kibana-","namespace":"default","selfLink":"/api/v1/namespaces/default/pods/kibana-ndeps","uid":"a4649aad-ced8-11e5-80b6-7824afc0a170","resourceVersion":"13142485","creationTimestamp":"2016-02-09T02:55:57Z","labels":{"name":"kibana"},"annotations":{"kubernetes.io/created-by":"{\"kind\":\"SerializedReference\",\"apiVersion\":\"v1\",\"reference\":{\"kind\":\"ReplicationController\",\"namespace\":\"default\",\"name\":\"kibana\",\"uid\":\"4a4de2fe-c888-11e5-b7f7-7824afc0a170\",\"apiVersion\":\"v1\",\"resourceVersion\":\"12885522\"}}\n"}},"spec":{"volumes":[{"name":"log","emptyDir":{}}],"containers":[{"name":"kibana","image":"minux:5000/kibana:latest","env":[{"name":"ELASTICSEARCH_HOST","value":"elasticsearch"}],"resources":{},"volumeMounts":[{"name":"log","mountPath":"/var/log"}],"readinessProbe":{"httpGet":{"path":"/","port":5601,"scheme":"HTTP"},"initialDelaySeconds":30,"timeoutSeconds":1},"terminationMessagePath":"/dev/termination-log","imagePullPolicy":"Always"},{"name":"fluentd-kafka","image":"minux:5000/fluentd-kafka:latest","env":[{"name":"ZOOKEEPER","value":"zookeeper:2181"},{"name":"LABEL","value":"KIBANA"}],"resources":{},"volumeMounts":[{"name":"log","mountPath":"/var/log"}],"terminationMessagePath":"/dev/termination-log","imagePullPolicy":"Always"}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","nodeSelector":{"zone":"green"},"nodeName":"192.168.1.176"},"status":{"phase":"Running","conditions":[{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":null}],"hostIP":"192.168.1.176","podIP":"10.244.176.4","startTime":"2016-02-09T02:55:59Z","containerStatuses":[{"name":"fluentd-kafka","state":{"running":{"startedAt":"2016-02-09T02:58:38Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"minux:5000/fluentd-kafka:latest","imageID":"docker://sha256:0948b0085872577fa50fc415feda363178bba8361834adae79952aee7431492f","containerID":"docker://e1f509c8fe3f6004c81b26f42c2cfd0c389afef459ac67797ed60f162e30d960"},{"name":"kibana","state":{"running":{"startedAt":"2016-02-09T02:58:38Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"minux:5000/kibana:latest","imageID":"docker://sha256:8b09cbc206d53a799211a4d96ff5e06d19f216be258a8a4dec7204d5911bc18d","containerID":"docker://d7c6bc51d4f70074c89ca21fdc65d075acb322f66fd52c2b7bf5a5cce479c823"}]}}
I0209 03:16:35.640649       1 exec.go:179] defaulting container name to kibana
I0209 03:16:35.640722       1 debugging.go:102] curl -k -v -XPOST  -H "X-Stream-Protocol-Version: v2.channel.k8s.io" -H "X-Stream-Protocol-Version: channel.k8s.io" http://minux:8080/api/v1/namespaces/default/pods/kibana-ndeps/exec?command=bash&amp;container=kibana&amp;container=kibana&amp;stderr=true&amp;stdin=true&amp;stdout=true&amp;tty=true
I0209 03:16:35.668922       1 debugging.go:121] POST http://minux:8080/api/v1/namespaces/default/pods/kibana-ndeps/exec?command=bash&amp;container=kibana&amp;container=kibana&amp;stderr=true&amp;stdin=true&amp;stdout=true&amp;tty=true 101 Switching Protocols in 28 milliseconds
I0209 03:16:35.668936       1 debugging.go:127] Response Headers:
I0209 03:16:35.668941       1 debugging.go:130]     Connection: Upgrade
I0209 03:16:35.668947       1 debugging.go:130]     Upgrade: SPDY/3.1
I0209 03:16:35.668955       1 debugging.go:130]     X-Stream-Protocol-Version: v2.channel.k8s.io
I0209 03:16:35.668962       1 debugging.go:130]     Date: Tue, 09 Feb 2016 03:16:37 GMT
Content-Type specified (plain/text) must be 'application/json'
</code></pre></div>
<p dir="auto">I have a setup using .AttachToContainer. After upgrading to Docker 1.10-rc2 I'm getting an error <code class="notranslate">Content-Type specified (plain/text) must be 'application/json'</code>.</p>
<p dir="auto">Why am I getting this?</p>
<p dir="auto">Any help would be great!</p>
<p dir="auto">/beetree</p>