1. A computer-implemented method for detecting an object of interest in an image, the method comprising: obtaining candidate patches that may contain the object, the candidate patches comprising image patches from the image; and for each candidate patch from a set of candidate patches selected from the candidate patches: computing a local difference pattern feature; using the local difference pattern feature to associate the candidate patch with a category from a set of categories, each category having a different background profile and a classifier associated with the category; and applying the classifier of the category to which the candidate patch is associated to classify whether the object is in the candidate patch.