<p dir="auto">Or it silently notices, and we should add clarity.</p>
<p dir="auto"><a href="http://kubekins.dls.corp.google.com:8081/job/kubernetes-pull-build-test-e2e-gce/26411/#" rel="nofollow">http://kubekins.dls.corp.google.com:8081/job/kubernetes-pull-build-test-e2e-gce/26411/#</a><br>
<a href="https://pantheon.corp.google.com/storage/browser/kubernetes-jenkins/pr-logs/pull/20464/kubernetes-pull-build-test-e2e-gce/26411/" rel="nofollow">https://pantheon.corp.google.com/storage/browser/kubernetes-jenkins/pr-logs/pull/20464/kubernetes-pull-build-test-e2e-gce/26411/</a></p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="18:04:42 Feb  3 01:54:26.948: INFO: Waiting for service mutability-service-test in namespace e2e-tests-services-h1iyv to have no LoadBalancer ingress points (2.018815ms)
18:04:42 Feb  3 01:54:31.952: INFO: Waiting for service mutability-service-test in namespace e2e-tests-services-h1iyv to have no LoadBalancer ingress points (5.005524636s)
18:04:42 Feb  3 01:54:36.956: INFO: Waiting for service mutability-service-test in namespace e2e-tests-services-h1iyv to have no LoadBalancer ingress points (10.009565421s)
18:04:42 Feb  3 01:54:41.960: INFO: Waiting for service mutability-service-test in namespace e2e-tests-services-h1iyv to have no LoadBalancer ingress points (15.013511228s)
...
¢ Failure [827.605 seconds]
Services
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:896
  should be able to change the type and nodeport settings of a service [It]
  /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:610

  Expected error:
      &lt;*errors.errorString | 0xc8206a1f70&gt;: {
          s: &quot;service mutability-service-test in namespace e2e-tests-services-h1iyv still has LoadBalancer ingress points after 600.00 seconds&quot;,
      }
      service mutability-service-test in namespace e2e-tests-services-h1iyv still has LoadBalancer ingress points after 600.00 seconds
  not to have occurred"><pre class="notranslate"><code class="notranslate">18:04:42 Feb  3 01:54:26.948: INFO: Waiting for service mutability-service-test in namespace e2e-tests-services-h1iyv to have no LoadBalancer ingress points (2.018815ms)
18:04:42 Feb  3 01:54:31.952: INFO: Waiting for service mutability-service-test in namespace e2e-tests-services-h1iyv to have no LoadBalancer ingress points (5.005524636s)
18:04:42 Feb  3 01:54:36.956: INFO: Waiting for service mutability-service-test in namespace e2e-tests-services-h1iyv to have no LoadBalancer ingress points (10.009565421s)
18:04:42 Feb  3 01:54:41.960: INFO: Waiting for service mutability-service-test in namespace e2e-tests-services-h1iyv to have no LoadBalancer ingress points (15.013511228s)
...
¢ Failure [827.605 seconds]
Services
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:896
  should be able to change the type and nodeport settings of a service [It]
  /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:610

  Expected error:
      &lt;*errors.errorString | 0xc8206a1f70&gt;: {
          s: "service mutability-service-test in namespace e2e-tests-services-h1iyv still has LoadBalancer ingress points after 600.00 seconds",
      }
      service mutability-service-test in namespace e2e-tests-services-h1iyv still has LoadBalancer ingress points after 600.00 seconds
  not to have occurred
</code></pre></div>
<p dir="auto">Brief scanning of controller manager logs shows:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="I0203 01:50:56.732467       6 servicecontroller.go:222] Got new Added delta for service: &amp;{TypeMeta:{Kind: APIVersion:} ObjectMeta:{Name:mutability-service-test GenerateName: Namespace:e2e-tests-services-h1iyv SelfLink:/api/v1/namespaces/e2e-tests-services-h1iyv/services/mutability-service-test UID:911cc0de-ca18-11e5-9af4-42010af00002 ResourceVersion:9107 Generation:0 CreationTimestamp:2016-02-03 01:50:56 +0000 UTC DeletionTimestamp:&lt;nil&gt; DeletionGracePeriodSeconds:&lt;nil&gt; Labels:map[] Annotations:map[]} Spec:{Type:ClusterIP Ports:[{Name: Protocol:TCP Port:80 TargetPort:{Type:0 IntVal:80 StrVal:} NodePort:0}] Selector:map[testid:mutability-service-test-911c15d6-ca18-11e5-b988-42010af0000d] ClusterIP:10.0.205.218 ExternalIPs:[] LoadBalancerIP: SessionAffinity:None} Status:{LoadBalancer:{Ingress:[]}}}"><pre class="notranslate"><code class="notranslate">I0203 01:50:56.732467       6 servicecontroller.go:222] Got new Added delta for service: &amp;{TypeMeta:{Kind: APIVersion:} ObjectMeta:{Name:mutability-service-test GenerateName: Namespace:e2e-tests-services-h1iyv SelfLink:/api/v1/namespaces/e2e-tests-services-h1iyv/services/mutability-service-test UID:911cc0de-ca18-11e5-9af4-42010af00002 ResourceVersion:9107 Generation:0 CreationTimestamp:2016-02-03 01:50:56 +0000 UTC DeletionTimestamp:&lt;nil&gt; DeletionGracePeriodSeconds:&lt;nil&gt; Labels:map[] Annotations:map[]} Spec:{Type:ClusterIP Ports:[{Name: Protocol:TCP Port:80 TargetPort:{Type:0 IntVal:80 StrVal:} NodePort:0}] Selector:map[testid:mutability-service-test-911c15d6-ca18-11e5-b988-42010af0000d] ClusterIP:10.0.205.218 ExternalIPs:[] LoadBalancerIP: SessionAffinity:None} Status:{LoadBalancer:{Ingress:[]}}}
</code></pre></div>
<p dir="auto">Which matches up to when it was created.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="I0203 01:53:53.114032       6 servicecontroller.go:222] Got new Updated delta for service: &amp;{TypeMeta:{Kind: APIVersion:} ObjectMeta:{Name:mutability-service-test GenerateName: Namespace:e2e-tests-services-h1iyv SelfLink:/api/v1/namespaces/e2e-tests-services-h1iyv/services/mutability-service-test UID:911cc0de-ca18-11e5-9af4-42010af00002 ResourceVersion:9205 Generation:0 CreationTimestamp:2016-02-03 01:50:56 +0000 UTC DeletionTimestamp:&lt;nil&gt; DeletionGracePeriodSeconds:&lt;nil&gt; Labels:map[] Annotations:map[]} Spec:{Type:LoadBalancer Ports:[{Name: Protocol:TCP Port:19482 TargetPort:{Type:0 IntVal:80 StrVal:} NodePort:31183}] Selector:map[testid:mutability-service-test-911c15d6-ca18-11e5-b988-42010af0000d] ClusterIP:10.0.205.218 ExternalIPs:[] LoadBalancerIP: SessionAffinity:None} Status:{LoadBalancer:{Ingress:[{IP:104.197.200.228 Hostname:}]}}}"><pre class="notranslate"><code class="notranslate">I0203 01:53:53.114032       6 servicecontroller.go:222] Got new Updated delta for service: &amp;{TypeMeta:{Kind: APIVersion:} ObjectMeta:{Name:mutability-service-test GenerateName: Namespace:e2e-tests-services-h1iyv SelfLink:/api/v1/namespaces/e2e-tests-services-h1iyv/services/mutability-service-test UID:911cc0de-ca18-11e5-9af4-42010af00002 ResourceVersion:9205 Generation:0 CreationTimestamp:2016-02-03 01:50:56 +0000 UTC DeletionTimestamp:&lt;nil&gt; DeletionGracePeriodSeconds:&lt;nil&gt; Labels:map[] Annotations:map[]} Spec:{Type:LoadBalancer Ports:[{Name: Protocol:TCP Port:19482 TargetPort:{Type:0 IntVal:80 StrVal:} NodePort:31183}] Selector:map[testid:mutability-service-test-911c15d6-ca18-11e5-b988-42010af0000d] ClusterIP:10.0.205.218 ExternalIPs:[] LoadBalancerIP: SessionAffinity:None} Status:{LoadBalancer:{Ingress:[{IP:104.197.200.228 Hostname:}]}}}
</code></pre></div>
<p dir="auto">Which matches when the loadbalancer came up.</p>
<p dir="auto">but when I'd expect to see the update for flipping out of type=lb, i instead see:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="I0203 01:54:44.819605       6 servicecontroller.go:277] LB already exists and doesn't need update for service e2e-tests-services-h1iyv/mutability-service-test
I0203 01:54:44.819625       6 servicecontroller.go:277] LB already exists and doesn't need update for service e2e-tests-services-h1iyv/mutability-service-test"><pre class="notranslate"><code class="notranslate">I0203 01:54:44.819605       6 servicecontroller.go:277] LB already exists and doesn't need update for service e2e-tests-services-h1iyv/mutability-service-test
I0203 01:54:44.819625       6 servicecontroller.go:277] LB already exists and doesn't need update for service e2e-tests-services-h1iyv/mutability-service-test
</code></pre></div>
<p dir="auto">After running couple e2e tests on a GCE cluster created from HEAD endpoint controller stops working (no entries in controller log on master) and no endpoints are created for a service. I have this cluster still up and running - let me know if you need more data.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="kubectl get services
NAME         CLUSTER_IP   EXTERNAL_IP   PORT(S)   SELECTOR          AGE
kubernetes   10.0.0.1     &lt;none&gt;        443/TCP   &lt;none&gt;            1h
rc-default   10.0.5.65    &lt;none&gt;        80/TCP    name=rc-default   8m"><pre class="notranslate"><code class="notranslate">kubectl get services
NAME         CLUSTER_IP   EXTERNAL_IP   PORT(S)   SELECTOR          AGE
kubernetes   10.0.0.1     &lt;none&gt;        443/TCP   &lt;none&gt;            1h
rc-default   10.0.5.65    &lt;none&gt;        80/TCP    name=rc-default   8m
</code></pre></div>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="kubectl get endpoints --all-namespaces
NAMESPACE     NAME                    ENDPOINTS                         AGE
default       kubernetes              10.240.0.2:443                    59m
kube-system   default-http-backend    10.245.0.5:8080                   59m
kube-system   elasticsearch-logging   10.245.0.6:9200,10.245.1.5:9200   59m
kube-system   heapster                10.245.1.3:8082                   59m
kube-system   kibana-logging          10.245.0.3:5601                   59m
kube-system   kube-dns                10.245.0.7:53,10.245.0.7:53       59m
kube-system   kube-ui                 10.245.0.4:8080                   59m
kube-system   monitoring-grafana      10.245.1.4:3000                   59m
kube-system   monitoring-influxdb     10.245.1.4:8083,10.245.1.4:8086   59m"><pre class="notranslate"><code class="notranslate">kubectl get endpoints --all-namespaces
NAMESPACE     NAME                    ENDPOINTS                         AGE
default       kubernetes              10.240.0.2:443                    59m
kube-system   default-http-backend    10.245.0.5:8080                   59m
kube-system   elasticsearch-logging   10.245.0.6:9200,10.245.1.5:9200   59m
kube-system   heapster                10.245.1.3:8082                   59m
kube-system   kibana-logging          10.245.0.3:5601                   59m
kube-system   kube-dns                10.245.0.7:53,10.245.0.7:53       59m
kube-system   kube-ui                 10.245.0.4:8080                   59m
kube-system   monitoring-grafana      10.245.1.4:3000                   59m
kube-system   monitoring-influxdb     10.245.1.4:8083,10.245.1.4:8086   59m
</code></pre></div>
<p dir="auto">On some healthy cluster, with the same service definition</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="kubectl get endpoints --all-namespaces
NAMESPACE     NAME                   ENDPOINTS                         AGE
default       kubernetes             10.240.0.2:443                    1d
default       rc-default             &lt;none&gt;                            29s
[...]"><pre class="notranslate"><code class="notranslate">kubectl get endpoints --all-namespaces
NAMESPACE     NAME                   ENDPOINTS                         AGE
default       kubernetes             10.240.0.2:443                    1d
default       rc-default             &lt;none&gt;                            29s
[...]
</code></pre></div>