title: Interpreting KL Divergence Value
body: <p>So I am using the scipy library's rel_entr module to calculate KL divergence.
  I have read through articles where it is mentioned that a KL divergence output of 0 indicates that the two distributions are essentially the same. Also, I have read that KL divergence is unbounded. So, KL value is between 0 and infinity.</p>
  <p>So, given this knowledge, something I have not come across in my research is whether there are 'thresholds' relating to the KL value. What I am saying is, at what value(given by the KL divergence), do I look at the value and conclude that 'P and Q are vastly different'? At what value, do I look at the value and conclude that 'P and Q are vastly similar'?</p>
  <p>We know that a value of 0 would mean that p = q but at what value(say 0.5, 0.75 etc) do we conclude that p and q share a majority of 'elements'(using elements here to avoid confusion with values) but differ in some 'elements'?</p>
  <p>Any thoughts?</p>
  <p>Thank you!</p>