How does Color Mix (Difference) node work?
In Cycles a color is just a set of 3 numbers representing the red, blue, and green content of the color.  To perform a mathematical operation on a color Cycles splits the color up into its RGB channels, performs the operation on each channel individually, and then combines them back into a Color datatype again.

The Mix RGB node, when set to difference, returns the absolute value of the first color minus the second color.  Mathematically: Color = |Color1 - Color2|.

The Fac value controls how much this influences the output.  So if Fac is 0 the output is always Color1, if Fac is 1 the output is always the exact result of the mix operation chosen (in your case, Difference).  It basically just mixes Color1 back into the result of the operation.  Mathematically: Color = |Color1 - Color2| * Fac + Color1 * (1-Fac).