<p dir="auto">Repro:</p>
<ol dir="auto">
<li>Start up a kube cluster.</li>
<li>Make some API calls that result in 404 (e.g. <a href="http://apiserver/aaa" rel="nofollow">http://apiserver/aaa</a> or <a href="http://apiserver/api/v1/pods/aaa" rel="nofollow">http://apiserver/api/v1/pods/aaa</a>) or 503 (kill etcd and make some API calls).</li>
<li>Visit <a href="http://apiserver/metrics" rel="nofollow">http://apiserver/metrics</a>.</li>
</ol>
<p dir="auto">Expected behavior: The metrics that have a "code" tag (e.g. apiserver_request_count and rest_client_request_status_codes) should have lines referring to code="404" or code="503".<br>
Actual behavior: No such lines exist. Only code="200" lines exist.</p>
<p dir="auto">Happening at HEAD (<a class="commit-link" data-hovercard-type="commit" data-hovercard-url="https://github.com/kubernetes/kubernetes/commit/730a6a1d555a0c4c2d9ab52de5f9e21dd065fb0b/hovercard" href="https://github.com/kubernetes/kubernetes/commit/730a6a1d555a0c4c2d9ab52de5f9e21dd065fb0b"><tt>730a6a1</tt></a>). This is currently blocking us from monitoring and alerting on 5xx errors.</p>
<p dir="auto">When looking into those metrics, the code is always 200.</p>
<p dir="auto">I explicitly send a bad request:<br>
wojtekt@wojtekt-work:~/go/src/k8s.io/kubernetes/pkg/labels$ curl localhost:8080/api/v1/pods?timeoutSeconds=xxx<br>
{<br>
"kind": "Status",<br>
"apiVersion": "v1",<br>
"metadata": {},<br>
"status": "Failure",<br>
"message": "strconv.ParseInt: parsing "xxx": invalid syntax",<br>
"reason": "BadRequest",<br>
"code": 400<br>
}</p>
<p dir="auto">and in metrics it was reported as code "200".</p>
<p dir="auto">cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/lavalamp/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/lavalamp">@lavalamp</a> @kubernetes/goog-csi</p>