1. A computer-implemented method for segmenting an image, the method comprising using a processor to perform the steps of: receiving at least two stereo-based images at a computer, each image containing a plurality of pixels; defining an energy function that is based at least in part on a likelihood model for stereo matching and a likelihood model for foreground and background color and that accounts for a stereo disparity, a stereo matching, and a color of at least a portion of the plurality of pixels; optimizing the energy function; based on the optimizing, determining a segmentation state variable value for each of the at least a portion of the plurality of pixels, each segmentation state variable value indicating one member of a segmentation group, the segmentation group comprising a foreground pixel indicator and a background pixel indicator.