What is a good descriptor for similar objects? I am developing an image search engine. The engine is meant to retrieve wrist watches based on the input of the user. I am using SIFT descriptors to index the elements in the database and applying Euclidean distance to get the most similar watches. I feel like this type of descriptor is not the best since watches have a similar structure and shape. Right now, the average difference between the best and worst matches is not big enough (15%)

I've been thinking of adding colour to the descriptor, but I'd like to hear other suggestions.
How to develop a program that can sort images by similarity is similar to the problem of how to develop a program that can sort words by how similar they look.

For example:
"theory" is more similar to "teoryyy" than to "abc".

What determines similarity of two words or images are these factors:


how many parts are common to both images
how many parts are new in either image
how many parts are missing in either image
how many parts are zoomed in/out or stretched.
how many parts are displaced


There may be some other rules. By combining these rules you can explain how similarity recognition works. This is at the core of intelligence.