When does the order of multiplication matter in python?
Question: I wrote a program using dynamic programming and it took me quite a long time
to find that something is wrong with the different versions of my code. It is
as follows:

    
    
    #the old version
    if probs[i][k]*probs[k+1][j]*prob > tmp_prob:
        tmp_prob = prob*probs[i][k]*probs[k+1][j]
    
    #the new version
    res = probs[i][k]*probs[k+1][j]*prob
    if res > tmp_prob:
        tmp_prob = res
    

I thought the results should be the same but, in fact, they are not. And

    
    
    if probs[i][k]*probs[k+1][j]*prob > tmp_prob:
        tmp_prob = probs[i][k]*probs[k+1][j]*prob
    

and the result is the same as the new version. So I know the problem is that
`probs[i][k]*probs[k+1][j]*prob` is not always equal to
`prob*probs[i][k]*probs[k+1][j]`.

But when are they not equal? I think it is when there may be overflow, i.e.,
when `inf` occurs. But since `probs[i][k]`, `probs[k+1][j]`, `prob` are all
probabilities so they are all less than 1, so I don't think this is the case.
Are there any other possibilities?


Answer: With floating point values, you may see slight differences by changing the
order of the multiplications, although I would normally expect the difference
to be very small (relative to the magnitude of the value), unless an
intermediate result overflows or underflows, but I would expect those cases to
be more obvious.

Here's a simple example:

    
    
    >>> import math
    >>> t = math.sqrt(3.0)
    >>> 3*t*t
    9.0
    >>> t*t*3
    8.999999999999998
    >>> 3*t*t - t*t*3
    1.7763568394002505e-15
    >>>
    

Mathematically, both products should be `9.0` and their difference should be
`0.0`, but due to floating point roundoff, that is not the case. The actual
results may differ from one platform to another, but this is what I get on my
computer, and it illustrates one of the difficulties with floating point
arithmetic.

