title: How can I get consistent program behavior when using floats?
body: <p>I am writing a simulation program that proceeds in discrete steps. The simulation consists of many nodes, each of which has a floating-point value associated with it that is re-calculated on every step. The result can be positive, negative or zero. </p>
  
  <p>In the case where the result is zero or less something happens. So far this seems straightforward - I can just do something like this for each node:</p>
  
  <pre><code>if (value &lt;= 0.0f) something_happens();
  </code></pre>
  
  <p>A problem has arisen, however, after some recent changes I made to the program in which I re-arranged the order in which certain calculations are done. In a perfect world the values would still come out the same after this re-arrangement, but because of the imprecision of floating point representation they come out very slightly different. Since the calculations for each step depend on the results of the previous step, these slight variations in the results can accumulate into larger variations as the simulation proceeds. </p>
  
  <p>Here's a simple example program that demonstrates the phenomena I'm describing:</p>
  
  <pre><code>float f1 = 0.000001f, f2 = 0.000002f;
  f1 += 0.000004f; // This part happens first here
  f1 += (f2 * 0.000003f);
  printf("%.16f\n", f1);
  
  f1 = 0.000001f, f2 = 0.000002f;
  f1 += (f2 * 0.000003f);
  f1 += 0.000004f; // This time this happens second
  printf("%.16f\n", f1);
  </code></pre>
  
  <p>The output of this program is</p>
  
  <pre><code>0.0000050000057854
  0.0000050000062402
  </code></pre>
  
  <p>even though addition is commutative so both results should be the same. Note: I understand perfectly well why this is happening - that's not the issue. The problem is that these variations can mean that sometimes a value that used to come out negative on step N, triggering something_happens(), now may come out negative a step or two earlier or later, which can lead to very different overall simulation results because something_happens() has a large effect.  </p>
  
  <p>What I want to know is whether there is a good way to decide when something_happens() should be triggered that is not going to be affected by the tiny variations in calculation results that result from re-ordering operations so that the behavior of newer versions of my program will be consistent with the older versions.</p>
  
  <p>The only solution I've so far been able to think of is to use some value epsilon like this:</p>
  
  <pre><code>if (value &lt; epsilon) something_happens();
  </code></pre>
  
  <p>but because the tiny variations in the results accumulate over time I need to make epsilon quite large (relatively speaking) to ensure that the variations don't result in something_happens() being triggered on a different step. Is there a better way?</p>
  
  <p>I've read <a href="http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm" rel="noreferrer">this excellent article</a> on floating point comparison, but I don't see how any of the comparison methods described could help me in this situation. </p>
  
  <p>Note: Using integer values instead is not an option. </p>
  
  <hr>
  
  <p><strong>Edit</strong> the possibility of using doubles instead of floats has been raised. This wouldn't solve my problem since the variations would still be there, they'd just be of a smaller magnitude. </p>