title: How to gracefully avoid divide by zero in Prometheus
body: <p>There are times when you need to divide one metric by another metric.</p>
  
  <p>For example, I'd like to calculate a mean latency like that:</p>
  
  <pre><code>rate({__name__="hystrix_command_latency_total_seconds_sum"}[60s])
  /
  rate({__name__="hystrix_command_latency_total_seconds_count"}[60s])
  </code></pre>
  
  <p>If there is no activity during the specified time period, the <code>rate()</code> in the divider becomes <code>0</code> and the result of division becomes <code>NaN</code>.
  If I do some aggregation over the results (<code>avg()</code> or <code>sum()</code> or whatever), the whole aggregation result becomes <code>NaN</code>.</p>
  
  <p>So I add a check for zero in divider:</p>
  
  <pre><code>rate({__name__="hystrix_command_latency_total_seconds_sum"}[60s])
  /
  (rate({__name__="hystrix_command_latency_total_seconds_count"}[60s]) &gt; 0)
  </code></pre>
  
  <p>This removes <code>NaN</code>s from the result vector. And also tears the line on the graph to shreds.</p>
  
  <p>Let's mark periods of inactivity with <code>0</code> value to make the graph continuous again:</p>
  
  <pre><code>rate({__name__="hystrix_command_latency_total_seconds_sum"}[60s])
  /
  (rate({__name__="hystrix_command_latency_total_seconds_count"}[60s]) &gt; 0)
  or
  rate({__name__="hystrix_command_latency_total_seconds_count"}[60s]) &gt; bool 0
  </code></pre>
  
  <p>This effectively replaces <code>NaN</code>s with <code>0</code>, graph is continuous, aggregations work OK.</p>
  
  <p>But resulting query is slightly cumbersome, especially when you need to do more label filtering and do some aggregations over results. Something like that:</p>
  
  <pre><code>avg(
      1000 * increase({__name__=~".*_hystrix_command_latency_total_seconds_sum", command_group=~"$commandGroup", command_name=~"$commandName", job=~"$service", instance=~"$instance"}[60s])
      /
      (increase({__name__=~".*_hystrix_command_latency_total_seconds_count", command_group=~"$commandGroup", command_name=~"$commandName", job=~"$service", instance=~"$instance"}[60s]) &gt; 0)
      or
      increase({__name__=~".*_hystrix_command_latency_total_seconds_count", command_group=~"$commandGroup", command_name=~"$commandName", job=~"$service", instance=~"$instance"}[60s]) &gt; bool 0
  ) by (command_group, command_name)
  </code></pre>
  
  <p><strong>Long story short:</strong> Are there any simpler ways to deal with zeros in divider? Or any common practices?</p>