<p dir="auto">It seems that if a pod doesn't fit on any node, we are trying to schedule it without any backoff.</p>
<p dir="auto">As an example see those logs from our big cluster:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="I0516 11:17:25.112607       5 event.go:216] Event(api.ObjectReference{Kind:&quot;Pod&quot;, Namespace:&quot;kube-system&quot;, Name:&quot;heapster-v1.1.0.beta1-3233973386-krr39&quot;, UID:&quot;75476330-1b54-11e6-a7f5-42010af00002&quot;, APIVersion:&quot;v1&quot;, ResourceVersion:&quot;229057&quot;, FieldPath:&quot;&quot;}): type: 'Warning' reason: 'FailedScheduling' pod (heapster-v1.1.0.beta1-3233973386-krr39) failed to fit in any node
...
I0516 11:17:25.213493       5 event.go:216] Event(api.ObjectReference{Kind:&quot;Pod&quot;, Namespace:&quot;kube-system&quot;, Name:&quot;heapster-v1.1.0.beta1-3233973386-krr39&quot;, UID:&quot;75476330-1b54-11e6-a7f5-42010af00002&quot;, APIVersion:&quot;v1&quot;, ResourceVersion:&quot;229058&quot;, FieldPath:&quot;&quot;}): type: 'Warning' reason: 'FailedScheduling' pod (heapster-v1.1.0.beta1-3233973386-krr39) failed to fit in any node
...
I0516 11:17:25.290314       5 event.go:216] Event(api.ObjectReference{Kind:&quot;Pod&quot;, Namespace:&quot;kube-system&quot;, Name:&quot;heapster-v1.1.0.beta1-3233973386-krr39&quot;, UID:&quot;75476330-1b54-11e6-a7f5-42010af00002&quot;, APIVersion:&quot;v1&quot;, ResourceVersion:&quot;229162&quot;, FieldPath:&quot;&quot;}): type: 'Warning' reason: 'FailedScheduling' pod (heapster-v1.1.0.beta1-3233973386-krr39) failed to fit in any node
...
I0516 11:17:25.351509       5 event.go:216] Event(api.ObjectReference{Kind:&quot;Pod&quot;, Namespace:&quot;kube-system&quot;, Name:&quot;heapster-v1.1.0.beta1-3233973386-krr39&quot;, UID:&quot;75476330-1b54-11e6-a7f5-42010af00002&quot;, APIVersion:&quot;v1&quot;, ResourceVersion:&quot;229171&quot;, FieldPath:&quot;&quot;}): type: 'Warning' reason: 'FailedScheduling' pod (heapster-v1.1.0.beta1-3233973386-krr39) failed to fit in any node"><pre class="notranslate"><code class="notranslate">I0516 11:17:25.112607       5 event.go:216] Event(api.ObjectReference{Kind:"Pod", Namespace:"kube-system", Name:"heapster-v1.1.0.beta1-3233973386-krr39", UID:"75476330-1b54-11e6-a7f5-42010af00002", APIVersion:"v1", ResourceVersion:"229057", FieldPath:""}): type: 'Warning' reason: 'FailedScheduling' pod (heapster-v1.1.0.beta1-3233973386-krr39) failed to fit in any node
...
I0516 11:17:25.213493       5 event.go:216] Event(api.ObjectReference{Kind:"Pod", Namespace:"kube-system", Name:"heapster-v1.1.0.beta1-3233973386-krr39", UID:"75476330-1b54-11e6-a7f5-42010af00002", APIVersion:"v1", ResourceVersion:"229058", FieldPath:""}): type: 'Warning' reason: 'FailedScheduling' pod (heapster-v1.1.0.beta1-3233973386-krr39) failed to fit in any node
...
I0516 11:17:25.290314       5 event.go:216] Event(api.ObjectReference{Kind:"Pod", Namespace:"kube-system", Name:"heapster-v1.1.0.beta1-3233973386-krr39", UID:"75476330-1b54-11e6-a7f5-42010af00002", APIVersion:"v1", ResourceVersion:"229162", FieldPath:""}): type: 'Warning' reason: 'FailedScheduling' pod (heapster-v1.1.0.beta1-3233973386-krr39) failed to fit in any node
...
I0516 11:17:25.351509       5 event.go:216] Event(api.ObjectReference{Kind:"Pod", Namespace:"kube-system", Name:"heapster-v1.1.0.beta1-3233973386-krr39", UID:"75476330-1b54-11e6-a7f5-42010af00002", APIVersion:"v1", ResourceVersion:"229171", FieldPath:""}): type: 'Warning' reason: 'FailedScheduling' pod (heapster-v1.1.0.beta1-3233973386-krr39) failed to fit in any node
</code></pre></div>
<p dir="auto">So it seems we are trying to schedule as often as possible, which is definitely a bug.</p>
<p dir="auto"><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/davidopp/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/davidopp">@davidopp</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gmarek/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gmarek">@gmarek</a> @kubernetes/sig-scheduling</p>
<p dir="auto"><strong>Is this a request for help?</strong> (If yes, you should use our troubleshooting guide and community support channels, see <a href="http://kubernetes.io/docs/troubleshooting/" rel="nofollow">http://kubernetes.io/docs/troubleshooting/</a>.): No</p>
<p dir="auto"><strong>What keywords did you search in Kubernetes issues before filing this one?</strong> (If you have found any duplicates, you should instead reply there.): namespace terminating 1.4.6</p>
<hr>
<p dir="auto"><strong>Is this a BUG REPORT or FEATURE REQUEST?</strong> (choose one): Bug report</p>
<p dir="auto"><strong>Kubernetes version</strong> (use <code class="notranslate">kubectl version</code>):</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Client Version: version.Info{Major:&quot;1&quot;, Minor:&quot;3&quot;, GitVersion:&quot;v1.3.4&quot;, GitCommit:&quot;dd6b458ef8dbf24aff55795baa68f83383c9b3a9&quot;, GitTreeState:&quot;clean&quot;, BuildDate:&quot;2016-08-01T16:45:16Z&quot;, GoVersion:&quot;go1.6.2&quot;, Compiler:&quot;gc&quot;, Platform:&quot;linux/amd64&quot;}
Server Version: version.Info{Major:&quot;1&quot;, Minor:&quot;4&quot;, GitVersion:&quot;v1.4.6&quot;, GitCommit:&quot;e569a27d02001e343cb68086bc06d47804f62af6&quot;, GitTreeState:&quot;clean&quot;, BuildDate:&quot;2016-11-12T05:16:27Z&quot;, GoVersion:&quot;go1.6.3&quot;, Compiler:&quot;gc&quot;, Platform:&quot;linux/amd64&quot;}"><pre class="notranslate"><code class="notranslate">Client Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.4", GitCommit:"dd6b458ef8dbf24aff55795baa68f83383c9b3a9", GitTreeState:"clean", BuildDate:"2016-08-01T16:45:16Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.6", GitCommit:"e569a27d02001e343cb68086bc06d47804f62af6", GitTreeState:"clean", BuildDate:"2016-11-12T05:16:27Z", GoVersion:"go1.6.3", Compiler:"gc", Platform:"linux/amd64"}
</code></pre></div>
<p dir="auto"><strong>Environment</strong>:</p>
<ul dir="auto">
<li><strong>Cloud provider or hardware configuration</strong>: GKE</li>
<li><strong>OS</strong> (e.g. from /etc/os-release): Ubuntu 16.04</li>
<li><strong>Kernel</strong> (e.g. <code class="notranslate">uname -a</code>): 4.4.0-47-generic</li>
<li><strong>Install tools</strong>: N.A</li>
<li><strong>Others</strong>: N.A</li>
</ul>
<p dir="auto"><strong>What happened</strong>:<br>
After upgrading our cluster from 1.4.0 to 1.4.6, when deleting a namespace it gets stuck in Terminating.</p>
<p dir="auto"><strong>What you expected to happen</strong>:<br>
Have the namespace deleted.</p>
<p dir="auto"><strong>How to reproduce it</strong> (as minimally and precisely as possible):<br>
Upgrade from 1.4.0 to 1.4.6 and delete a namespace.</p>
<p dir="auto"><strong>Anything else do we need to know</strong>:<br>
I found <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="125105444" data-permission-text="Title is private" data-url="https://github.com/kubernetes/kubernetes/issues/19317" data-hovercard-type="issue" data-hovercard-url="/kubernetes/kubernetes/issues/19317/hovercard" href="https://github.com/kubernetes/kubernetes/issues/19317">#19317</a> and <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="190945865" data-permission-text="Title is private" data-url="https://github.com/kubernetes/kubernetes/issues/37278" data-hovercard-type="issue" data-hovercard-url="/kubernetes/kubernetes/issues/37278/hovercard" href="https://github.com/kubernetes/kubernetes/issues/37278">#37278</a>, but one is specific to thirdpartyresources and the other one to openshift. This issue happens on all namespaces with no special finalizer or thirdpartyresource.</p>