(minimum compressed size), while the relative entropy of a target set of data, given a source set of data, is the data required to reconstruct the target given the source (minimum size of a patch). See also Akaike information criterion Bayesian information criterion Bregman divergence Cross-entropy Deviance information criterion Entropic value at risk Entropy power inequality Hellinger distance Information gain in decision trees Information gain ratio Information theory and measure theory Jensen–Shannon divergence Quantum relative entropy Solomon Kullback and Richard Leibler References . Republished by Dover Publications in 1968; reprinted in 1978: . External links Information Theoretical Estimators Toolbox Ruby gem for calculating Kullback–Leibler divergence Jon Shlens' tutorial on Kullback–Leibler divergence and likelihood theory Matlab code for calculating Kullback–Leibler divergence for discrete distributions Sergio Verdú, Relative Entropy, NIPS 2009. One-hour video lecture. A modern summary of info-theoretic divergence measures Entropy and information F-divergences Information geometry Thermodynamics