1. A method, performed by a computer having a memory and a processor, for interpreting a user interface, the method comprising: capturing at least a portion of pixel data of a source window associated with the user interface; with a processor, identifying areas of the captured pixel data that at least substantially match features associated with at least one widget prototype of a prototype library comprising a plurality of widget prototypes, wherein each feature comprises a plurality of pixels; and with a processor, identifying a user interface widget within the captured pixel data at least in part by determining whether a spatial relationship between at least two of the identified areas is consistent with at least one widget prototype of the prototype library, wherein a first widget prototype of the prototype library includes, a plurality of features, wherein the plurality of features is representative of a first type of user interface widget, and at least one constraint on a spatial relationship between at least two features of the first widget prototype, and wherein the identifying the user interface widget within the captured pixel data is based at least in part on: matching pixels of a plurality of features of the first widget prototype to pixels of the captured pixel data to identify a plurality of features within the captured pixel data corresponding to the user interface widget within the captured pixel data, selecting a first feature of the identified plurality of features within the captured pixel data, selecting a second feature of the identified plurality of features within the captured pixel data, and determining whether the first feature of the identified plurality of features within the captured pixel data and the second feature of the identified plurality of features within the captured pixel data satisfy the at least one constraint of the first widget prototype.