What is a good descriptor for similar objects?
How to develop a program that can sort images by similarity is similar to the problem of how to develop a program that can sort words by how similar they look.

For example:
"theory" is more similar to "teoryyy" than to "abc".

What determines similarity of two words or images are these factors:


how many parts are common to both images
how many parts are new in either image
how many parts are missing in either image
how many parts are zoomed in/out or stretched.
how many parts are displaced


There may be some other rules. By combining these rules you can explain how similarity recognition works. This is at the core of intelligence.