<p dir="auto">The following tests are failing on clusters where the master is at HEAD and the nodes are at 1.0 (in kubernetes-upgrade-gke-step3-e2e-old):</p>
<ul dir="auto">
<li>Kubernetes e2e suite.DNS should provide DNS for services</li>
<li>Kubernetes e2e suite.DNS should provide DNS for the cluster</li>
</ul>
<p dir="auto">They are both failing with:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Unexpected number of pods (2) matches the label selector k8s-app=kube-dns,kubernetes.io/cluster-service=true"><pre class="notranslate"><code class="notranslate">Unexpected number of pods (2) matches the label selector k8s-app=kube-dns,kubernetes.io/cluster-service=true
</code></pre></div>
<p dir="auto">These tests are <strong>not</strong> failing once the nodes have been upgraded to HEAD to match version with the master.</p>
<p dir="auto">See runs 33 - 39 of the kubernetes-upgrade-gke Jenkins job.</p>
<p dir="auto">It seems that HEAD masters can't delete pods on 1.0 nodes.</p>
<p dir="auto">After spinning up a (GKE) cluster at 1.0.6 and upgrading the master to <code class="notranslate">1.2.0-alpha.1.455+4b7329ad8c52f6-dirty</code>, I can't delete pods:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="$ kubectl get pods
NAME                     READY     STATUS    RESTARTS   AGE
$ kubectl create -f ./docs/user-guide/walkthrough/replication-controller.yaml 
replicationcontrollers/nginx-controller
$ kubectl get pods
NAME                     READY     STATUS    RESTARTS   AGE
nginx-controller-l4rhx   0/1       Running   0          8s
nginx-controller-tfwpp   0/1       Running   0          8s
$ kubectl delete -f ./docs/user-guide/walkthrough/replication-controller.yaml                                                                                        
replicationcontrollers/nginx-controller
$ kubectl get pods
NAME                     READY     STATUS    RESTARTS   AGE
nginx-controller-l4rhx   1/1       Running   0          6m
nginx-controller-tfwpp   1/1       Running   0          6m"><pre class="notranslate"><code class="notranslate">$ kubectl get pods
NAME                     READY     STATUS    RESTARTS   AGE
$ kubectl create -f ./docs/user-guide/walkthrough/replication-controller.yaml 
replicationcontrollers/nginx-controller
$ kubectl get pods
NAME                     READY     STATUS    RESTARTS   AGE
nginx-controller-l4rhx   0/1       Running   0          8s
nginx-controller-tfwpp   0/1       Running   0          8s
$ kubectl delete -f ./docs/user-guide/walkthrough/replication-controller.yaml                                                                                        
replicationcontrollers/nginx-controller
$ kubectl get pods
NAME                     READY     STATUS    RESTARTS   AGE
nginx-controller-l4rhx   1/1       Running   0          6m
nginx-controller-tfwpp   1/1       Running   0          6m
</code></pre></div>
<p dir="auto">I uncovered this because there are several e2e tests failing on version-skewed clusters, and they seem to be all having problems deleting pods.</p>
<p dir="auto">This is likely related to:</p>
<ul dir="auto">
<li>Kubectl e2e tests leave resources running after stop during upgrade <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="109892961" data-permission-text="Title is private" data-url="https://github.com/kubernetes/kubernetes/issues/15112" data-hovercard-type="issue" data-hovercard-url="/kubernetes/kubernetes/issues/15112/hovercard" href="https://github.com/kubernetes/kubernetes/issues/15112">#15112</a></li>
<li>DNS e2es are failing on version-skewed clusters, where master is at 1.1 and nodes are at 1.0 <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="109899067" data-permission-text="Title is private" data-url="https://github.com/kubernetes/kubernetes/issues/15120" data-hovercard-type="issue" data-hovercard-url="/kubernetes/kubernetes/issues/15120/hovercard" href="https://github.com/kubernetes/kubernetes/issues/15120">#15120</a></li>
</ul>