15. One or more computer storage media having computer-executable instructions, the computer storage media being a hardware storage media, the instructions, when executed, perform steps, comprising: receiving a query image; determining a visual word vector for the query image; determining a visual pattern vector for the query image; determining similarity with each image of a set of database images, including, for each image of the set of database images, computing a first similarity score by evaluating the visual word vector for the query image with a visual word vector for each database image and computing a second similarity score by evaluating the visual pattern vector for the query image with a visual pattern vector for each database image; and using the first and second similarity scores according to a usage model by ranking the set of database images based on one of the first and second similarity scores, and re-ranking a selected set of highest ranked database images in the set of database images based on another of the first and second similarity score.
15. One or more computer storage media having computer-executable instructions, the computer storage media being a hardware storage media, the instructions, when executed, perform steps, comprising: receiving a query image; determining a visual word vector for the query image; determining a visual pattern vector for the query image; determining similarity with each image of a set of database images, including, for each image of the set of database images, computing a first similarity score by evaluating the visual word vector for the query image with a visual word vector for each database image and computing a second similarity score by evaluating the visual pattern vector for the query image with a visual pattern vector for each database image; and using the first and second similarity scores according to a usage model by ranking the set of database images based on one of the first and second similarity scores, and re-ranking a selected set of highest ranked database images in the set of database images based on another of the first and second similarity score. 16. The one or more computer storage media of claim 15 wherein using the first and second similarity scores according to the usage model comprises ranking database images based on the first and second similarity scores according to a pre-ranking usage model, according to a re-ranking usage model, according to a similarity merging usage model, or according to a vector merging usage model.