anchor.diff:          description: |
             The number of file descriptors currently used by the Gorouter job.
   
  -          **Use**: Indicates an impending issue with the Gorouter. Without proper mitigation, it is possible for an unresponsive app to eventually exhaust available Gorouter file descriptors and cause route starvation for other apps running on PAS. Under heavy load, this unmitigated situation can also result in the Gorouter losing its connection to NATS and all routes being pruned.
  -
  -          While a drop in `gorouter.total_routes` or an increase in `gorouter.ms_since_last_registry_update` helps to surface that the issue may already be occurring, alerting on `gorouter.file_descriptors` indicates that such an issue is impending.
  -
  -          The Gorouter limits the number of file descriptors to 100,000 per job. Once the limit is met, the Gorouter is unable to establish any new connections.
  -
  -          To reduce the risk of DDoS attacks, we recommend doing one or both of the following:
  -
  -          * Within PAS, set **Max Connections Per Backend** to define how many requests can be routed to any particular app instance. This prevents a single app from using all Gorouter connections. The value specified should be determined by the operator based on the use cases for that foundation.
  +          **Use**: Indicates an impending issue with the
  +            Gorouter. Without proper mitigation, it is possible for an
  +            unresponsive app to eventually exhaust available Gorouter
  +            file descriptors and cause route starvation for other apps
  +            running on PAS. Under heavy load, this unmitigated
  +            situation can also result in the Gorouter losing its
  +            connection to NATS and all routes being pruned.
  +
  +          While a drop in `gorouter.total_routes` or an increase in
  +          `gorouter.ms_since_last_registry_update` helps to surface
  +          that the issue may already be occurring, alerting on
  +          `gorouter.file_descriptors` indicates that such an issue is
  +          impending.
  +
  +          The Gorouter limits the number of file descriptors to
  +          100,000 per job. Once the limit is met, the Gorouter is
  +          unable to establish any new connections.
  +
  +          To reduce the risk of DDoS attacks, we recommend doing one
  +          or both of the following:
  +
  +          * Within PAS, set **Max Connections Per Backend** to define
  +          how many requests can be routed to any particular app
  +          instance. This prevents a single app from using all Gorouter
  +          connections. The value specified should be determined by the
  +          operator based on the use cases for that foundation.
  +
             * Add rate limiting at the load balancer level.
   
             **Origin**: Firehose
  
positive.code:         recommendedMeasurement: |
            Average over the last 5 minutes of the derived per second calculation
          recommendedResponse: |
            For optimizing the Gorouter, consider the requests-per-second derived metric in the context of router latency
            and Gorouter VM CPU utilization. From performance and load testing of the Gorouter, it's observed that
            at approximately 2500 requests per second, latency can begin to increase.
  
            To increase throughput and maintain low latency, scale the Gorouters either horizontally or vertically and
            watch that the system.cpu.user metric for the Gorouter stays in the suggested range of 60-70% CPU Utilization.
  
      - name: latency_by_router
        promql: quantile_over_time(0.95, latency{source_id="gorouter",deployment="$deployment"}[1m])
  
negative.code:         description: |
            The lifetime number of requests completed by the Gorouter VM, emitted per Gorouter instance
  
            **Use**: The aggregation of these values across all
            Gorouters provide insight into the overall traffic flow of a
            deployment. Unusually high spikes, if not known to be
            associated with an expected increase in demand, could
            indicate a DDoS risk. For performance and capacity
            management, consider this metric a measure of router
            throughput per job, converting it to requests-per-second, by
            looking at the delta value of `gorouter.total_requests` and
            deriving back to 1s, or `(gorouter.total_requests.delta)/5`,
            per Gorouter instance.  This helps you see trends in the
            throughput rate that indicate a need to scale the Gorouter
            instances. Use the trends you observe to tune the threshold
            alerts for this metric.
  
            **Origin**: Firehose
            **Type**: Counter (Integer)
  