Chapter 5. AlertingRule [monitoring.openshift.io/v1]
Chapter 5. AlertingRule [monitoring.openshift.io/v1] Description AlertingRule represents a set of user-defined Prometheus rule groups containing alerting rules. This resource is the supported method for cluster admins to create alerts based on metrics recorded by the platform monitoring stack in OpenShift, i.e. the Prometheus instance deployed to the openshift-monitoring namespace. You might use this to create custom alerting rules not shipped with OpenShift based on metrics from components such as the node_exporter, which provides machine-level metrics such as CPU usage, or kube-state-metrics, which provides metrics on Kubernetes usage. The API is mostly compatible with the upstream PrometheusRule type from the prometheus-operator. The primary difference being that recording rules are not allowed here - only alerting rules. For each AlertingRule resource created, a corresponding PrometheusRule will be created in the openshift-monitoring namespace. OpenShift requires admins to use the AlertingRule resource rather than the upstream type in order to allow better OpenShift specific defaulting and validation, while not modifying the upstream APIs directly. You can find upstream API documentation for PrometheusRule resources here: https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/api.md Compatibility level 1: Stable within a major release for a minimum of 12 months or 3 minor releases (whichever is longer). Type object Required spec 5.1. Specification Property Type Description apiVersion string APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources kind string Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds metadata ObjectMeta Standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata spec object spec describes the desired state of this AlertingRule object. status object status describes the current state of this AlertOverrides object. 5.1.1. .spec Description spec describes the desired state of this AlertingRule object. Type object Required groups Property Type Description groups array groups is a list of grouped alerting rules. Rule groups are the unit at which Prometheus parallelizes rule processing. All rules in a single group share a configured evaluation interval. All rules in the group will be processed together on this interval, sequentially, and all rules will be processed. It's common to group related alerting rules into a single AlertingRule resources, and within that resource, closely related alerts, or simply alerts with the same interval, into individual groups. You are also free to create AlertingRule resources with only a single rule group, but be aware that this can have a performance impact on Prometheus if the group is extremely large or has very complex query expressions to evaluate. Spreading very complex rules across multiple groups to allow them to be processed in parallel is also a common use-case. groups[] object RuleGroup is a list of sequentially evaluated alerting rules. 5.1.2. .spec.groups Description groups is a list of grouped alerting rules. Rule groups are the unit at which Prometheus parallelizes rule processing. All rules in a single group share a configured evaluation interval. All rules in the group will be processed together on this interval, sequentially, and all rules will be processed. It's common to group related alerting rules into a single AlertingRule resources, and within that resource, closely related alerts, or simply alerts with the same interval, into individual groups. You are also free to create AlertingRule resources with only a single rule group, but be aware that this can have a performance impact on Prometheus if the group is extremely large or has very complex query expressions to evaluate. Spreading very complex rules across multiple groups to allow them to be processed in parallel is also a common use-case. Type array 5.1.3. .spec.groups[] Description RuleGroup is a list of sequentially evaluated alerting rules. Type object Required name rules Property Type Description interval string interval is how often rules in the group are evaluated. If not specified, it defaults to the global.evaluation_interval configured in Prometheus, which itself defaults to 30 seconds. You can check if this value has been modified from the default on your cluster by inspecting the platform Prometheus configuration: The relevant field in that resource is: spec.evaluationInterval name string name is the name of the group. rules array rules is a list of sequentially evaluated alerting rules. Prometheus may process rule groups in parallel, but rules within a single group are always processed sequentially, and all rules are processed. rules[] object Rule describes an alerting rule. See Prometheus documentation: - https://www.prometheus.io/docs/prometheus/latest/configuration/alerting_rules 5.1.4. .spec.groups[].rules Description rules is a list of sequentially evaluated alerting rules. Prometheus may process rule groups in parallel, but rules within a single group are always processed sequentially, and all rules are processed. Type array 5.1.5. .spec.groups[].rules[] Description Rule describes an alerting rule. See Prometheus documentation: - https://www.prometheus.io/docs/prometheus/latest/configuration/alerting_rules Type object Required alert expr Property Type Description alert string alert is the name of the alert. Must be a valid label value, i.e. may contain any Unicode character. annotations object (string) annotations to add to each alert. These are values that can be used to store longer additional information that you won't query on, such as alert descriptions or runbook links. expr integer-or-string expr is the PromQL expression to evaluate. Every evaluation cycle this is evaluated at the current time, and all resultant time series become pending or firing alerts. This is most often a string representing a PromQL expression, e.g.: mapi_current_pending_csr > mapi_max_pending_csr In rare cases this could be a simple integer, e.g. a simple "1" if the intent is to create an alert that is always firing. This is sometimes used to create an always-firing "Watchdog" alert in order to ensure the alerting pipeline is functional. for string for is the time period after which alerts are considered firing after first returning results. Alerts which have not yet fired for long enough are considered pending. labels object (string) labels to add or overwrite for each alert. The results of the PromQL expression for the alert will result in an existing set of labels for the alert, after evaluating the expression, for any label specified here with the same name as a label in that set, the label here wins and overwrites the value. These should typically be short identifying values that may be useful to query against. A common example is the alert severity, where one sets severity: warning under the labels key: 5.1.6. .status Description status describes the current state of this AlertOverrides object. Type object Property Type Description observedGeneration integer observedGeneration is the last generation change you've dealt with. prometheusRule object prometheusRule is the generated PrometheusRule for this AlertingRule. Each AlertingRule instance results in a generated PrometheusRule object in the same namespace, which is always the openshift-monitoring namespace. 5.1.7. .status.prometheusRule Description prometheusRule is the generated PrometheusRule for this AlertingRule. Each AlertingRule instance results in a generated PrometheusRule object in the same namespace, which is always the openshift-monitoring namespace. Type object Required name Property Type Description name string name of the referenced PrometheusRule. 5.2. API endpoints The following API endpoints are available: /apis/monitoring.openshift.io/v1/alertingrules GET : list objects of kind AlertingRule /apis/monitoring.openshift.io/v1/namespaces/{namespace}/alertingrules DELETE : delete collection of AlertingRule GET : list objects of kind AlertingRule POST : create an AlertingRule /apis/monitoring.openshift.io/v1/namespaces/{namespace}/alertingrules/{name} DELETE : delete an AlertingRule GET : read the specified AlertingRule PATCH : partially update the specified AlertingRule PUT : replace the specified AlertingRule /apis/monitoring.openshift.io/v1/namespaces/{namespace}/alertingrules/{name}/status GET : read status of the specified AlertingRule PATCH : partially update status of the specified AlertingRule PUT : replace status of the specified AlertingRule 5.2.1. /apis/monitoring.openshift.io/v1/alertingrules HTTP method GET Description list objects of kind AlertingRule Table 5.1. HTTP responses HTTP code Reponse body 200 - OK AlertingRuleList schema 401 - Unauthorized Empty 5.2.2. /apis/monitoring.openshift.io/v1/namespaces/{namespace}/alertingrules HTTP method DELETE Description delete collection of AlertingRule Table 5.2. HTTP responses HTTP code Reponse body 200 - OK Status schema 401 - Unauthorized Empty HTTP method GET Description list objects of kind AlertingRule Table 5.3. HTTP responses HTTP code Reponse body 200 - OK AlertingRuleList schema 401 - Unauthorized Empty HTTP method POST Description create an AlertingRule Table 5.4. Query parameters Parameter Type Description dryRun string When present, indicates that modifications should not be persisted. An invalid or unrecognized dryRun directive will result in an error response and no further processing of the request. Valid values are: - All: all dry run stages will be processed fieldValidation string fieldValidation instructs the server on how to handle objects in the request (POST/PUT/PATCH) containing unknown or duplicate fields. Valid values are: - Ignore: This will ignore any unknown fields that are silently dropped from the object, and will ignore all but the last duplicate field that the decoder encounters. This is the default behavior prior to v1.23. - Warn: This will send a warning via the standard warning response header for each unknown field that is dropped from the object, and for each duplicate field that is encountered. The request will still succeed if there are no other errors, and will only persist the last of any duplicate fields. This is the default in v1.23+ - Strict: This will fail the request with a BadRequest error if any unknown fields would be dropped from the object, or if any duplicate fields are present. The error returned from the server will contain all unknown and duplicate fields encountered. Table 5.5. Body parameters Parameter Type Description body AlertingRule schema Table 5.6. HTTP responses HTTP code Reponse body 200 - OK AlertingRule schema 201 - Created AlertingRule schema 202 - Accepted AlertingRule schema 401 - Unauthorized Empty 5.2.3. /apis/monitoring.openshift.io/v1/namespaces/{namespace}/alertingrules/{name} Table 5.7. Global path parameters Parameter Type Description name string name of the AlertingRule HTTP method DELETE Description delete an AlertingRule Table 5.8. Query parameters Parameter Type Description dryRun string When present, indicates that modifications should not be persisted. An invalid or unrecognized dryRun directive will result in an error response and no further processing of the request. Valid values are: - All: all dry run stages will be processed Table 5.9. HTTP responses HTTP code Reponse body 200 - OK Status schema 202 - Accepted Status schema 401 - Unauthorized Empty HTTP method GET Description read the specified AlertingRule Table 5.10. HTTP responses HTTP code Reponse body 200 - OK AlertingRule schema 401 - Unauthorized Empty HTTP method PATCH Description partially update the specified AlertingRule Table 5.11. Query parameters Parameter Type Description dryRun string When present, indicates that modifications should not be persisted. An invalid or unrecognized dryRun directive will result in an error response and no further processing of the request. Valid values are: - All: all dry run stages will be processed fieldValidation string fieldValidation instructs the server on how to handle objects in the request (POST/PUT/PATCH) containing unknown or duplicate fields. Valid values are: - Ignore: This will ignore any unknown fields that are silently dropped from the object, and will ignore all but the last duplicate field that the decoder encounters. This is the default behavior prior to v1.23. - Warn: This will send a warning via the standard warning response header for each unknown field that is dropped from the object, and for each duplicate field that is encountered. The request will still succeed if there are no other errors, and will only persist the last of any duplicate fields. This is the default in v1.23+ - Strict: This will fail the request with a BadRequest error if any unknown fields would be dropped from the object, or if any duplicate fields are present. The error returned from the server will contain all unknown and duplicate fields encountered. Table 5.12. HTTP responses HTTP code Reponse body 200 - OK AlertingRule schema 401 - Unauthorized Empty HTTP method PUT Description replace the specified AlertingRule Table 5.13. Query parameters Parameter Type Description dryRun string When present, indicates that modifications should not be persisted. An invalid or unrecognized dryRun directive will result in an error response and no further processing of the request. Valid values are: - All: all dry run stages will be processed fieldValidation string fieldValidation instructs the server on how to handle objects in the request (POST/PUT/PATCH) containing unknown or duplicate fields. Valid values are: - Ignore: This will ignore any unknown fields that are silently dropped from the object, and will ignore all but the last duplicate field that the decoder encounters. This is the default behavior prior to v1.23. - Warn: This will send a warning via the standard warning response header for each unknown field that is dropped from the object, and for each duplicate field that is encountered. The request will still succeed if there are no other errors, and will only persist the last of any duplicate fields. This is the default in v1.23+ - Strict: This will fail the request with a BadRequest error if any unknown fields would be dropped from the object, or if any duplicate fields are present. The error returned from the server will contain all unknown and duplicate fields encountered. Table 5.14. Body parameters Parameter Type Description body AlertingRule schema Table 5.15. HTTP responses HTTP code Reponse body 200 - OK AlertingRule schema 201 - Created AlertingRule schema 401 - Unauthorized Empty 5.2.4. /apis/monitoring.openshift.io/v1/namespaces/{namespace}/alertingrules/{name}/status Table 5.16. Global path parameters Parameter Type Description name string name of the AlertingRule HTTP method GET Description read status of the specified AlertingRule Table 5.17. HTTP responses HTTP code Reponse body 200 - OK AlertingRule schema 401 - Unauthorized Empty HTTP method PATCH Description partially update status of the specified AlertingRule Table 5.18. Query parameters Parameter Type Description dryRun string When present, indicates that modifications should not be persisted. An invalid or unrecognized dryRun directive will result in an error response and no further processing of the request. Valid values are: - All: all dry run stages will be processed fieldValidation string fieldValidation instructs the server on how to handle objects in the request (POST/PUT/PATCH) containing unknown or duplicate fields. Valid values are: - Ignore: This will ignore any unknown fields that are silently dropped from the object, and will ignore all but the last duplicate field that the decoder encounters. This is the default behavior prior to v1.23. - Warn: This will send a warning via the standard warning response header for each unknown field that is dropped from the object, and for each duplicate field that is encountered. The request will still succeed if there are no other errors, and will only persist the last of any duplicate fields. This is the default in v1.23+ - Strict: This will fail the request with a BadRequest error if any unknown fields would be dropped from the object, or if any duplicate fields are present. The error returned from the server will contain all unknown and duplicate fields encountered. Table 5.19. HTTP responses HTTP code Reponse body 200 - OK AlertingRule schema 401 - Unauthorized Empty HTTP method PUT Description replace status of the specified AlertingRule Table 5.20. Query parameters Parameter Type Description dryRun string When present, indicates that modifications should not be persisted. An invalid or unrecognized dryRun directive will result in an error response and no further processing of the request. Valid values are: - All: all dry run stages will be processed fieldValidation string fieldValidation instructs the server on how to handle objects in the request (POST/PUT/PATCH) containing unknown or duplicate fields. Valid values are: - Ignore: This will ignore any unknown fields that are silently dropped from the object, and will ignore all but the last duplicate field that the decoder encounters. This is the default behavior prior to v1.23. - Warn: This will send a warning via the standard warning response header for each unknown field that is dropped from the object, and for each duplicate field that is encountered. The request will still succeed if there are no other errors, and will only persist the last of any duplicate fields. This is the default in v1.23+ - Strict: This will fail the request with a BadRequest error if any unknown fields would be dropped from the object, or if any duplicate fields are present. The error returned from the server will contain all unknown and duplicate fields encountered. Table 5.21. Body parameters Parameter Type Description body AlertingRule schema Table 5.22. HTTP responses HTTP code Reponse body 200 - OK AlertingRule schema 201 - Created AlertingRule schema 401 - Unauthorized Empty