title: When are accumulators truly reliable?
body: <p>I want to use an accumulator to gather some stats about the data I'm manipulating on a Spark job. Ideally, I would do that while the job computes the required transformations, but since Spark would re-compute tasks on different cases the accumulators would not reflect true metrics. Here is how the documentation describes this:</p>
  
  <blockquote>
    <p>For accumulator updates performed inside actions only, Spark
    guarantees that each task’s update to the accumulator will only be
    applied once, i.e. restarted tasks will not update the value. In
    transformations, users should be aware of that each task’s update may
    be applied more than once if tasks or job stages are re-executed.</p>
  </blockquote>
  
  <p>This is confusing since most <strong><em>actions</em></strong> do not allow running custom code (where accumulators can be used), they mostly take the results from previous transformations (lazily). The documentation also shows this:</p>
  
  <pre><code>val acc = sc.accumulator(0)
  data.map(x =&gt; acc += x; f(x))
  // Here, acc is still 0 because no actions have cause the `map` to be computed.
  </code></pre>
  
  <p>But if we add <code>data.count()</code> at the end, would this be guaranteed to be correct (have no duplicates) or not? Clearly <code>acc</code> is not used "inside actions only", as map is a transformation. So it should not be guaranteed.</p>
  
  <p>On the other hand, discussion on related Jira tickets talk about "result tasks" rather than "actions". For instance <a href="https://github.com/apache/spark/pull/2524">here</a> and <a href="https://issues.apache.org/jira/browse/SPARK-732">here</a>. This seems to indicate that the result would indeed be guaranteed to be correct, since we are using <code>acc</code> immediately before and action and thus should be computed as a single stage.</p>
  
  <p>I'm guessing that this concept of a "result task" has to do with the type of operations involved, being the last one that includes an action, like in this example, which shows how several operations are divided into stages (in magenta, image taken from <a href="http://blog.cloudera.com/blog/2015/03/how-to-tune-your-apache-spark-jobs-part-1/">here</a>):</p>
  
  <p><img src="https://i.stack.imgur.com/d3OOf.png" alt="A job dividing several operations into multiple purple stages"></p>
  
  <p>So hypothetically, a <code>count()</code> action at the end of that chain would be part of the same final stage, and I would be guaranteed that accumulators used on the last map will no include any duplicates?</p>
  
  <p>Clarification around this issue would be great! Thanks.</p>