<p dir="auto">Just deleted 91 since gke-ci was completely hosed on quota. We hit the limit on <a href="http://kubekins.dls.corp.google.com/job/kubernetes-e2e-gke-ci/10377/" rel="nofollow">http://kubekins.dls.corp.google.com/job/kubernetes-e2e-gke-ci/10377/</a> so it was some set of suites before that.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="19:06:05 client [kubectl]. To install, run
19:06:05   $ gcloud components install kubectl
19:06:05 
19:06:06 ERROR: (gcloud.container.clusters.create) ResponseError: code=403, message=Insufficient project quota to satisfy request for resource: &quot;ROUTES&quot;. The request requires '3.0' and is short '2.0'. The project has a quota of '100.0' with '1.0' available. 
19:06:06 2016/01/17 19:06:06 e2e.go:309: Error running up: exit status 1
19:06:06 2016/01/17 19:06:06 e2e.go:305: Step 'up' finished in 2.107868944s
19:06:06 
2016/01/17 19:06:06 e2e.go:141: Error starting e2e cluster. Aborting.
19:06:06 exit status 1"><pre class="notranslate"><code class="notranslate">19:06:05 client [kubectl]. To install, run
19:06:05   $ gcloud components install kubectl
19:06:05 
19:06:06 ERROR: (gcloud.container.clusters.create) ResponseError: code=403, message=Insufficient project quota to satisfy request for resource: "ROUTES". The request requires '3.0' and is short '2.0'. The project has a quota of '100.0' with '1.0' available. 
19:06:06 2016/01/17 19:06:06 e2e.go:309: Error running up: exit status 1
19:06:06 2016/01/17 19:06:06 e2e.go:305: Step 'up' finished in 2.107868944s
19:06:06 
2016/01/17 19:06:06 e2e.go:141: Error starting e2e cluster. Aborting.
19:06:06 exit status 1
</code></pre></div>
<p dir="auto">@kubernetes/goog-testing</p>
<p dir="auto">Our <code class="notranslate">kubernetes-upgrade-gce</code> Jenkins job has been broken for quite a while now, and it looks like it's due to target pools and/or forwarding rules leaking and exceeding quota:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Sep  2 00:48:45.094: INFO: event for mutability-service-test: {service-controller } creating loadbalancer: creating loadbalancer
Sep  2 00:48:45.094: INFO: event for mutability-service-test: {service-controller } creating loadbalancer failed: error creating load balancer (will retry): failed to create external load balancer for service e2e-tests-services-mrlvq/mutability-service-test: googleapi: Error 403: Quota 'TARGET_POOLS' exceeded.  Limit: 50.0"><pre class="notranslate"><code class="notranslate">Sep  2 00:48:45.094: INFO: event for mutability-service-test: {service-controller } creating loadbalancer: creating loadbalancer
Sep  2 00:48:45.094: INFO: event for mutability-service-test: {service-controller } creating loadbalancer failed: error creating load balancer (will retry): failed to create external load balancer for service e2e-tests-services-mrlvq/mutability-service-test: googleapi: Error 403: Quota 'TARGET_POOLS' exceeded.  Limit: 50.0
</code></pre></div>
<p dir="auto">I've cleared out the old rules, and I'll try to diagnose what's leaking them and why.</p>