How to find chemical similarity between two compounds using Tanimoto method
Apparently, it is more generally known as the "Jaccard Index":


  In "A Computer Program for Classifying Plants", published in October 1960,[5] a method of classification based on a similarity ratio, and a derived distance function, is given. It seems that this is the most authoritative source for the meaning of the terms "Tanimoto similarity" and "Tanimoto Distance". The similarity ratio is equivalent to Jaccard similarity, but the distance function is not the same as Jaccard distance.


In terms of chemistry, this calculation is carried out on a fingerprint of the molecule. That wikipedia page has a nice reference to this paper, although it is not open access. There are no doubt many resources out there about fingerprints/Tanimoto, even open source implementations...

This also looks interesting: "Why is Tanimoto index an appropriate choice for fingerprint-based similarity calculations?".