1. A computer-implemented method for generating a background model of a scene depicted in a sequence of video frames captured by a video camera, the method comprising: receiving a video frame, wherein the video frame includes one or more appearance values for each of a plurality of pixels; classifying each pixel as depicting either foreground or background by comparing the one or more appearance values of each pixel to a background model of the scene; and performing one or more context based evaluations on one or more of the pixels classified as depicting foreground, wherein the context based evaluations selectively reclassifies one or more of the pixels as depicting foreground or background based on the classification of other pixels in the video frame as depicting either foreground or background.