Producing masks from mostly-static-background subtraction when given the background
You can try working off a different representation of the image, other than color. You can diff the image gradient for example. It is very unlikely that the gradient is the same on the face as the background. A gradient based descriptor such as HOG or SIFT captured at a suitable scale over the image has worked well for me in the past. You can even combine different representations of the image, such as RGB value as well as HOG. IF the image is different in either mark as a change. You will still get the odd few pixels that don't come up as a change but you can smooth over the result with a smoothing kernel with a similar scale tot he size of the object you want to detect.