1. A method for processing an image, comprising: receiving digitized data representing the image, the digitized data comprising a plurality of pixels; segmenting the pixels segments at a plurality of scale levels; determining feature vectors of the segments at the plurality of the scale levels, the feature vectors comprising one or more measures of visual perception of the segments; determining one or more similarities, each similarity determined by comparing two or more of the feature vectors; determining for each of a first subset of the segments, which is acquired through a bottom-up segmentation, a first measure of probability that the segment is a member of an object class; determining probability factors based on the determined first measures of the probability and similarity factors based on the determined similarities; and performing factor graph analysis to determine a second measure of probability for each of a second subset of the segments, which is acquired through a top-down segmentation, based on the probability factors and the similarity factors; wherein each of the second subset of segments are variable nodes in the factor graph analysis and the factor nodes between the variable nodes at the different scale levels are computed from the similarity factors and the probability factors, wherein each of the factor nodes represents a function product of a bottom-up prior probability term computed from the first subset of the segments, and a top-down observation likelihood term computed from the second subset of segments.