1. A computer-implemented method comprising: accessing data encoding a facial image that includes a face; generating a hierarchical graphical model of the face, the hierarchical graphical model including a node from a level that approximates the face within a contour; applying, to the node, a bank of filters that each correspond to a respective scale and a respective direction; determining a similarity measure that consolidates filter responses from the bank of filters by analyzing the filter responses from the bank of filters; based on the similarity measure, generating, for the node, a vector representation that represents the facial image at the node; and storing the vector representation.