How to cluster histograms or density distributions?
There are quite a few ways to measure the difference between two distributions. Take a look at this overview article on wikipedia.

One very common way that is used often in Machine and Deep Learning is the Kullback-Leibler (KL) Divergence. It is most commonly used in minimising the cross-entropy between the distribution of your training data and the expected (generalised) distribution of the problem you are analysing. In general, a value close to 0 indicates that two distributions are expected to show similar behaviour, while a large value indicates the distributions behave very differently - so knowing the first distribution doesn't help you know anything about the second.

A more general class of measures of dissimilarity between distributions is the so called $f$-divergence, "an average, weighted by the function $f$ of the odds ratio given by [probability measures] $P$ and $Q$". The Wikipedia article contains a formal definition and a table with functions $f$ that give some popular dissimilarities/distances, including the KL-divergence.


  The following table lists many of the common divergences between probability distributions and the $f$ function to which they correspond (cf. Liese &amp; Vajda (2006))




KL-Divergence is combined with the entropy itself, to define the cross-entropy. Take a look here, under the Information Theory View, for a bit more info.