<p dir="auto">See Jenkins job <strong>kubernetes-upgrade-1.0-1.1-gke-step3-e2e-old</strong>'s last few runs.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="07:52:26   Validates MaxPods limit number of pods that are allowed to run. [It]
07:52:26   /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/max_pods.go:158
07:52:26 
07:52:26   Expected
07:52:26       &lt;[]api.Event | len:0, cap:0&gt;: []
07:52:26   not to be empty
07:52:26 
07:52:26   /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/max_pods.go:156"><pre class="notranslate"><code class="notranslate">07:52:26   Validates MaxPods limit number of pods that are allowed to run. [It]
07:52:26   /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/max_pods.go:158
07:52:26 
07:52:26   Expected
07:52:26       &lt;[]api.Event | len:0, cap:0&gt;: []
07:52:26   not to be empty
07:52:26 
07:52:26   /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/max_pods.go:156
</code></pre></div>
<p dir="auto">The relevant line is <a href="https://github.com/kubernetes/kubernetes/blob/release-1.0/test/e2e/max_pods.go#L156">max_pods.go:156</a>.  Note that this is running e2e tests off of the <code class="notranslate">release-1.0</code> branch.</p>
<p dir="auto">It's possible that this is also caused by <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="109904540" data-permission-text="Title is private" data-url="https://github.com/kubernetes/kubernetes/issues/15123" data-hovercard-type="issue" data-hovercard-url="/kubernetes/kubernetes/issues/15123/hovercard" href="https://github.com/kubernetes/kubernetes/issues/15123">#15123</a>, but it's not clear to me.</p>
<p dir="auto">This is a problem we're going to run into as we're running 1.0 e2e tests against 1.1 clusters, (which we do because we need to make sure that 1.1 clusters still operate the way 1.0 clusters did).  If a test fails, but the failure is due to a bad test rather than a problem in 1.1, what do we do?</p>
<p dir="auto">For example, Services/Nodeport e2es are failing on upgrade due to change in error message.  They fail when running 1.0 e2es against a HEAD master (in jobs kubernetes-upgrade-gke-step3-e2e-old and kubernetes-upgrade-gke-step5-e2e-old):</p>
<ul dir="auto">
<li>Kubernetes e2e suite.Services should check NodePort out-of-range</li>
</ul>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Expected
    &lt;string&gt;: Service &quot;nodeport-range-test&quot; is invalid: spec.ports[0].nodePort: invalid value '53127', Details: provided port is not in the valid range
to equal
    &lt;string&gt;: Service &quot;nodeport-range-test&quot; is invalid: spec.ports[0].nodePort: invalid value '53127': provided port is not in the valid range"><pre class="notranslate"><code class="notranslate">Expected
    &lt;string&gt;: Service "nodeport-range-test" is invalid: spec.ports[0].nodePort: invalid value '53127', Details: provided port is not in the valid range
to equal
    &lt;string&gt;: Service "nodeport-range-test" is invalid: spec.ports[0].nodePort: invalid value '53127': provided port is not in the valid range
</code></pre></div>
<ul dir="auto">
<li>Kubernetes e2e suite.Services should prevent NodePort collisions</li>
</ul>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Expected
    &lt;string&gt;: Service &quot;nodeport-collision2&quot; is invalid: spec.ports[0].nodePort: invalid value '31641', Details: provided port is already allocated
to equal
    &lt;string&gt;: Service &quot;nodeport-collision2&quot; is invalid: spec.ports[0].nodePort: invalid value '31641': provided port is already allocated"><pre class="notranslate"><code class="notranslate">Expected
    &lt;string&gt;: Service "nodeport-collision2" is invalid: spec.ports[0].nodePort: invalid value '31641', Details: provided port is already allocated
to equal
    &lt;string&gt;: Service "nodeport-collision2" is invalid: spec.ports[0].nodePort: invalid value '31641': provided port is already allocated
</code></pre></div>
<p dir="auto"><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ixdy/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ixdy">@ixdy</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/quinton-hoole/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/quinton-hoole">@quinton-hoole</a> Any ideas about how to fix this?  This definitely isn't a regression, but it's probably not a good idea to just disable these tests.  My best thought is to cherry-pick the 1.1 test changes into 1.0, and somehow pull these tests HEAD of the 1.0 branch.  That's a lot of mucking around though, and I'm not sure it's worth it.</p>
<p dir="auto">For now, I think we should punt on these specific tests until we have a better idea of how widespread this kind of version-skew problem is going to be.</p>