A short characterization of relative entropy

arXiv:1712.04903v1 [cs.IT] 13 Dec 2017

Tom Leinster∗

Abstract
We prove characterization theorems for relative entropy (also known as
Kullback–Leibler divergence), q-logarithmic entropy (also known as Tsallis entropy), and q-logarithmic relative entropy. All three have been characterized axiomatically before, but we show that earlier proofs can be
simplified considerably, at the same time relaxing some of the hypotheses.

1

Introduction

The Shannon entropy of a finite probability distribution p = (p1 , . . . , pn ),
X

H(p) =

i : pi >0

pi log

1
,
pi

is such an important quantity that many authors have sought short lists of
properties that determine H uniquely. Many such characterization theorems
have been found, beginning with one in Shannon’s seminal paper of 1948 ([17],
Theorem 2). For instance, Faddeev [6] proved that up to a constant factor,
H is uniquely characterized by symmetry, continuity, and a certain recursivity
property.
Accompanying Shannon entropy is the concept of relative entropy, defined
as follows. Given probability distributions p and r on n elements, the entropy
of p relative to r is
X
pi
∈ [0, ∞].
pi log
H(p k r) =
ri
i : p >0
i

Relative entropy goes by a multitude of names: Kullback–Leibler divergence,
directed divergence, discrimination information, relative information, information gain, and so on. In information theory, it measures the wastage when a
language whose n letters have frequencies p = (p1 , . . . , pn ) is encoded using
a system optimized for a different language with frequencies r, instead of the
system optimized for the original language. There are other interpretations in
other fields, as the plethora of names suggests.
Axiomatic characterizations of relative entropy have also been sought and
found. One such theorem is implicit in work of Kannappan and Ng [12]. It
states that up to a constant factor, relative entropy is uniquely determined
∗ School of Mathematics, University of Edinburgh, UK; Tom.Leinster@ed.ac.uk. Key words:
relative entropy, Kullback–Leibler divergence, q-logarithm, q-logarithmic entropy, Tsallis entropy, Tsallis relative entropy

1

by measurability in each of p and r separately, invariance under permutations
of {1, . . . , n}, the vanishing property H(p k p) = 0, and a certain recursivity
equation. (Remark 2.7 gives further details.) Their proof was a tour de force of
functional equations, involving the solution of the functional equation




x
y
= h(y) + (1 − y)k
(1)
f (x) + (1 − x)g
1−x
1−y
in four unknown functions, as well as the four-variable functional equation




u
x
v
y
F (x, y) + (1 − x)F
= F (u, v) + (1 − u)F
.
,
,
1−x 1−y
1−u 1−v

We give a much simpler proof, at the same time weakening the measurability
hypothesis. Our proof involves neither of these equations. Instead, it borrows heavily from a categorical characterization of relative entropy by Baez and
Fritz [1]. This is the first main result, Theorem 2.1.
Shannon entropy is just one member (albeit a special one) of a one-parameter
family of entropies (Sq )q∈R , first investigated by Havrda and Charvát [10] and
often misattributed to Tsallis (Remark 3.2(ii)). These entropies Sq , and the
accompanying relative entropies, are defined as follows.
For q ∈ R, the q-logarithm is the function lnq : (0, ∞) → R given by
Z x
t−q dt.
lnq (x) =
1

The q-logarithmic entropy and q-logarithmic relative entropy are defined
by
X
1
pi lnq ,
Sq (p) =
p
i
i : pi >0
X
ri
Sq (p k r) = −
pi lnq ,
p
i
i : p >0
i

for probability distributions p and r on n elements. When q = 1, these reduce
to the ordinary Shannon entropy and relative entropy.
There are several existing theorems characterizing the q-logarithmic entropy
for a given q 6= 1. Up until now, the simplest appears to have been the 1970
result of Daróczy [5]. We simplify further, weakening the hypotheses and shortening the proof to just a few lines (Theorem 3.1).
Finally, we use a similar and equally short argument to characterize the
q-logarithmic relative entropies Sq (− k −) (Theorem 4.1).
It is remarkable that when q 6= 1, the characterizations of q-logarithmic
entropy and q-logarithmic relative entropy need no regularity conditions whatsoever (not even measurability), in contrast to the theorems for q = 1.
The remaining three sections of this paper establish our three theorems in
turn, characterizing first relative entropy, then q-logarithmic entropy, then qlogarithmic relative entropy.

2

Relative entropy

For n ≥ 1, write

n
o
X
∆n = p = (p1 , . . . , pn ) ∈ Rn : pi ≥ 0,
pi = 1
2

for the set of probability distributions on {1, . . . , n}, and write

An = (p, r) ∈ ∆n × ∆n : pi = 0 whenever ri = 0 .

Evidently, (p, r) ∈ An if and only if the relative entropy
H(p k r) =

X

pi log

i : pi >0

pi
ri

is finite. (Viewing p and r as measures on {1, . . . , n}, we have (p, r) ∈ An just
when p is absolutely continuous with respect to r.)
We will characterize the sequence of functions

H(− k −) : An → R n≥1
uniquely up to a constant factor. It is easy to check that this sequence has the
following four properties, as does any scalar multiple cH(− k −) (for c ∈ R).

Measurability in the second argument For each n ≥ 1 and p ∈ ∆n , the
function
{r ∈ ∆n : (p, r) ∈ An } →
R
r
7→ H(p k r)
is Lebesgue measurable.
Symmetry For each n ≥ 1, (p, r) ∈ An and permutation σ of {1, . . . , n},
H(p k r) = H(pσ k rσ),

(2)

where pσ = (pσ(1) , . . . , pσ(n) ).
Vanishing H(p k p) = 0 for all n ≥ 1 and p ∈ ∆n .
Chain rule To state this, we need some notation. Given n, k1 , . . . , kn ≥ 1 and
w ∈ ∆n , p1 ∈ ∆k1 , . . . , pn ∈ ∆kn , and writing pi = (pi1 , . . . , piki ), define
w ◦ (p1 , . . . , pn ) = (w1 p11 , . . . , w1 p1k1 , . . . , wn pn1 , . . . , wn pnkn ) ∈ ∆k1 +···+kn .
The chain rule for relative entropy is that
n
X

e i ) (3)
e ◦ (e
e n ) = H(w k w)
e +
wi H(pi k p
H w ◦ (p1 , . . . , pn ) w
p1 , . . . , p
i=1

e ∈ An and (pi , p
e i ) ∈ Aki . (Under these hypotheses, the
whenever (w, w)
pair of distributions on the left-hand side belongs to Ak1 +···+kn .)

Theorem 2.1 Let I(− k −) : An → R n≥1 be a sequence of functions. The
following are equivalent:

i. I(− k −) satisfies the four properties above: measurability in the second
argument, symmetry, vanishing, and the chain rule;
ii. I(− k −) = cH(− k −) for some c ∈ R.

3

We have just noted that (ii) implies (i). We now embark on the proof of the
converse. For the rest of this section, let I(− k −) : An → R n≥1 be a sequence
of functions satisfying the four conditions. Define a function L : (0, 1] → R by

L(α) = I (1, 0) (α, 1 − α) .

The idea is that if I(− k −) = H(− k −) then L = − log. We will show that in
any case, L is a scalar multiple of log.
Lemma 2.2 Let (p, r) ∈ An with pk+1 = · · · = pn = 0, where 1 ≤ k ≤ n. Then
r1 + · · · + rk > 0 and
I(p k r) = L(r1 + · · · + rk ) + I(p′ k r′ ),
where
p′ = (p1 , . . . , pk ),

r′ =

(r1 , . . . , rk )
.
r1 + · · · + rk

Proof The case k = n reduces to the statement that L(1) = 0, which follows
from the vanishing property. Suppose, then, that k < n.
Since p is a probability distribution with pi = 0 for all i > k, there is some
i ≤ k such that pi > 0, and then ri > 0 as (p, r) ∈ An . Hence r1 + · · · + rk > 0.
Let r′′ ∈ ∆n−k be the normalization of (rk+1 , . . . , rn ) if rk+1 + · · · + rn > 0, or
choose r′′ arbitrarily in ∆n−k otherwise (which is possible since k < n). Then

I(p k r) = I (1, 0) ◦ (p′ , r′′ ) (r1 + · · · + rk , rk+1 + · · · + rn ) ◦ (r′ , r′′ ) .
The result now follows from the chain rule.



Lemma 2.3 L(αβ) = L(α) + L(β) for all α, β ∈ (0, 1].
Proof We evaluate the real number

x := I (1, 0, 0) (αβ, α(1 − β), 1 − α)

in two ways. By Lemma 2.2 with k = 1 and the vanishing property,

x = L(αβ) + I (1) (1) = L(αβ),

where (1) is the unique element of ∆1 . But also, by Lemma 2.2 with k = 2,

x = L(α) + I (1, 0) (β, 1 − β) = L(α) + L(β).
Comparing the two expressions for x gives the result.



Lemma 2.4 There is some c ∈ R such that L(α) = −c log α for all α ∈ (0, 1].
Proof Define f : [0, ∞) → R by f (t) = L(e−t ). By Lemma 2.3, f (t + u) =
f (t) + f (u) for all t, u ∈ [0, ∞). Also, f is measurable, since L is. It is wellknown [2] that these conditions force f (t) = ct for some constant c, giving
L(α) = −c log α.

Our next lemma is an adaptation of the most ingenious part of Baez and
Fritz’s argument ([1], Lemma 4.2).
4

Lemma 2.5 Let (p, r) ∈ An with pi > 0 for all i ∈ {1, . . . , n}. Then I(p k r) =
cH(p k r).
Proof The hypotheses imply that ri > 0 for all i. We can therefore choose
some α ∈ (0, 1] such that ri − αpi ≥ 0 for all i. We will compute the number

x := I (p1 , . . . , pn , 0, . . . , 0) (αp1 , . . . , αpn , r1 − αp1 , . . . , rn − αpn )
| {z }
n

in two ways. First, by Lemma 2.2, Lemma 2.4, and the vanishing property,
x = L(α) + I(p k p) = −c log α.
Second, by symmetry, the chain rule, and Lemma 2.4,


x = I (p1 , 0, . . . , pn , 0) (αp1 , r1 − αp1 , . . . , pn , rn − αpn )




= I p ◦ (1, 0), . . . , (1, 0) r ◦ α pr11 , 1 − α pr11 , . . . , α prnn , 1 − α prnn
= I(p k r) +

n
X
i=1

pi L α prii



= I(p k r) − c log α − cH(p k r).
Comparing the two expressions for x gives the result.



We have now proved that I(p k r) = cH(p k r) when p has full support. It
only remains to prove it for arbitrary p.
Proof of Theorem 2.1 Let (p, r) ∈ An . By symmetry, we can assume that
p1 , . . . , pk > 0 and pk+1 = · · · = pn = 0 for some k ∈ {1, . . . , n}. Writing
R = r1 + · · · + rk , we have R > 0 since (p, r) ∈ An , and t

I(p k r) = L(R) + I (p1 , . . . , pk ) R1 (r1 , . . . , rk )
by Lemma 2.2. Hence by Lemmas 2.4 and 2.5,

I(p k r) = −c log R + cH (p1 , . . . , pk )


.

1
R (r1 , . . . , rk )

But by the same argument applied to cH in place of I (or by direct calculation),
we also have

cH(p k r) = −c log R + cH (p1 , . . . , pk ) R1 (r1 , . . . , rk ) .
The result follows.



Remarks 2.6
i. The vanishing P
axiom cannot be dropped from Theorem 2.1. Indeed, the quantity i : pi >0 pi log r1i satisfies the other three
axioms but not vanishing.
ii. In the literature on information functions, the chain rule is often replaced
by one of two superficially simpler rules. The first is the special case
k1 = 2, k2 = · · · = kn = 1, which is

H (pw1 , (1 − p)w1 , w2 , . . . , wn ) (e
pw
e1 , (1 − pe)w
e1 , w
e2 , . . . , w
en )


e + w1 H (p, 1 − p) (e
=H w w
p, 1 − pe) (4)
5

e ∈ An , ((p, 1 − p), (e
((w, w)
p, 1 − pe)) ∈ A2 ). This is known as recursivity.
The second is the special case n = 2 of the chain rule, which is


we
ep ⊕ (1 − w)e
er

e ) + (1 − w)H(r k e
= H (w, 1 − w) (w,
e 1 − w)
e + wH(p k p
r), (5)

H wp ⊕ (1 − w)r

where

wp ⊕ (1 − w)r = (wp1 , . . . , wpk , (1 − w)r1 , . . . , (1 − w)rℓ )
e ) ∈ Ak , (r, e
and ((w, 1 − w), (w,
e 1 − w))
e ∈ A2 , (p, p
r) ∈ Aℓ . However,
straightforward inductions (similar to those in Feinstein [7], p. 5–6) show
that in the presence of the symmetry axiom, either one of the special
cases (4) or (5) is equivalent to the full chain rule (3). Which to use is,
therefore, simply a matter of taste.
Remark 2.7 Here we compare Theorem 2.1 with some earlier characterizations
of relative entropy. One of the first such theorems was that of Hobson [11], who
used stronger hypotheses for the same conclusion. In common with Theorem 2.1,
he assumed symmetry, vanishing, and the chain rule (in the equivalent form (5)).
But he also assumed continuity in both variables (instead of measurability in
one) and a monotonicity hypothesis unlike anything in Theorem 2.1.
In 1973, Kannappan and Ng [12] proved a result very close to Theorem 2.1.
They did not state that result in [12], but the closing remarks in another paper
by the same authors [13] and the approach of a contemporaneous paper by
Kannappan and Rathie [14] strongly suggest the intent. The result was stated
explicitly by Csiszár ([4], Section 2.1), who attributed it to Kannappan and Ng.
There are some slight differences of hypotheses between Kannappan and
Ng’s theorem and Theorem 2.1. They assumed measurability in both variables,
whereas we only assumed measurability in the second. (In fact, all we used
was that I((1, 0) k −) is measurable.) On the other hand, they only needed the
vanishing condition for (1/2, 1/2), whereas we needed it for all p. They used
the chain rule in the equivalent form (4). And as indicated in the Introduction,
the proofs are entirely different.

3

q-logarithmic entropy

Let q ∈ R. The definition of q-logarithm in the Introduction gives, explicitly,
lnq (x) =

1
(x1−q − 1)
1−q

for x ∈ (0, ∞) and q 6= 1, while ln1 is the natural logarithm log. Hence,
explicitly, the q-logarithmic entropy is given by
!
X q
1
p −1
Sq (p) =
1 − q i : p >0 i
i

for p ∈ ∆n and q 6= 1, while S1 is the Shannon entropy H. We have lnq (x) →
log(x) as q → 1, hence also Sq (p) → H(p) as q → 1.
6

Fix q ∈ R. The q-logarithmic entropy satisfies a chain rule
X

wiq Sq (pi )
Sq w ◦ (p1 , . . . , pn ) = Sq (w) +

(6)

i : wi >0

(w ∈ ∆n , p1 ∈ ∆k1 , . . . , pn ∈ ∆kn ), as is easily checked. In particular, this
holds when p1 = · · · = pn = p, say. For w ∈ ∆n and p ∈ ∆k , write
w ⊗ p = w ◦ (p, . . . , p)
= (w1 p1 , . . . , w1 pk , . . . , wn p1 , . . . , wn pk ) ∈ ∆nk .
In this case, the q-chain rule (6) gives a q-multiplicativity property:
!
X
q
wi Sq (p)
Sq (w ⊗ p) = Sq (w) +

(7)

i : wi >0

(n, k ≥ 1, w ∈ ∆n , p ∈ ∆k ).
Note also that Sq is symmetric in its arguments:
Sq (p) = Sq (pσ)

(8)

for all p ∈ ∆n and permutations σ of {1, . . . , n}.
The left-hand side of equation (7) is symmetric in w and p, but the righthand side is not obviously so. This is the key to our second theorem.
Theorem 3.1 Let 1 6= q ∈ R and let (I : ∆n → R)n≥1 be a sequence of functions. The following are equivalent:
i. I has the q-multiplicativity property (7) and the symmetry property (8)
(both with I in place of Sq );
ii. I = cSq for some c ∈ R.
Proof By the observations just made, (ii) implies (i). Now assume (i). By
symmetry, I(w ⊗ p) = I(p ⊗ w), so
!
!
X q
X
q
pi I(w),
wi I(p) = I(p) +
I(w) +
i : pi >0

i : wi >0

or equivalently
X

wiq

!

− 1 I(p) =

i : wi >0

X

i : pi >0

pqi

!

− 1 I(w),

for all w ∈ ∆n and p ∈ ∆k . Take w = (1/2, 1/2): then for all p ∈ ∆k ,
!
X q

1−q
pi − 1 I(1/2, 1/2).
2
− 1 I(p) =
i : pi >0

Since q 6= 1, we can define c =

1−q
21−q −1

· I(1/2, 1/2), and then I = cSq .

7



Remarks 3.2
i. The q-logarithms were used in Hardy, Littlewood and
Pólya’s classic book on inequalities, first published in 1934 ([9], proof
of Theorem 84). They have been an explicit object of study since at least
a 1964 paper of Box and Cox in statistics ([3], Section 3). The name
‘q-logarithm’ appears to have been introduced by Umarov, Tsallis and
Steinberg in 2008 [20], working in statistical mechanics.
ii. The q-logarithmic entropies have been discovered and rediscovered repeatedly. To my knowledge, they first appeared in a 1967 paper on information
and classification by Havrda and Charvát [10], who used a form adapted
to base 2 logarithms. They were rediscovered in 1970 by Daróczy [5]. The
base e version Sq seems to have first appeared in a 1982 article of Patil and
Taillie ([16], Section 3.2), where it was studied as an index of biodiversity.
In physics, meanwhile, the q-logarithmic entropies appeared in a 1971 article of Lindhard and Nielsen [15] (according to Csiszar [4], Section 2.4), and
in a 1978 survey by Wehrl ([21], p. 247). Finally, they were rediscovered
again in a 1988 paper on statistical physics by Tsallis [19].
Despite the twenty years of active life that the q-logarithmic entropies had
already enjoyed, it is after Tsallis that they are most commonly named.
The term ‘q-logarithmic entropy’ is new, but has the benefits of being
descriptive and of not perpetuating a misattribution.
iii. As in Remark 2.6(ii) or Feinstein [7] (p. 5–6), a simple inductive argument
shows that the q-chain rule of equation (6) follows from the special case

(9)
Sq pw1 , (1 − p)w1 , w2 , . . . , wn = Sq (w) + w1q Sq (p, 1 − p)
(p ∈ [0, 1], n ≥ 1, w ∈ ∆n ).

iv. A characterization of the q-logarithmic entropies similar to Theorem 3.1
was published by Daróczy in 1970 [5]. He assumed the full q-chain rule
for I(w ◦ (p1 , . . . , pn )) (in the equivalent form (9)), rather than just the
special case of I(w ⊗ p) that we used. However, where we assumed that
I : ∆n → R is symmetric for all n ≥ 2, Daróczy only assumed it for n = 3.
The two proofs are very different; the main step in Daróczy’s was the
solution of the functional equation (1) in the case f = g = h = k.
Other characterizations of Sq have been proved, using stronger hypotheses
than Theorem 3.1 to obtain the same conclusion (such as the theorem in
Section 2 of [18], and Theorem V.2 of [8]).

4

q-logarithmic relative entropy

For q 6= 1, the q-logarithmic relative entropy Sq : An → R, defined in the Introduction, is given explicitly by
!
X q 1−q
1
p r
−1 ,
Sq (p k r) =
q − 1 i : p >0 i i
i

for (p, r) ∈ An and q 6= 1. In the case q = 1, it reduces to the ordinary
relative entropy H(p k r). As in that case, restricting the arguments to lie in
An guarantees that Sq (− k −) takes only finite values.
8

Our third and final theorem is a characterization of q-logarithmic relative
entropy, very similar to the characterization of q-logarithmic entropy itself.
We begin by noting two properties of q-logarithmic relative entropy. First,
there is an easily-checked chain rule:
X

ei)
e ◦ (e
e n ) = Sq (w k w)
e +
wiq w
ei1−q Sq (pi k p
p1 , . . . , p
Sq w ◦ (p1 , . . . , pn ) w
i : wi >0

e ∈ An , (pi , p
e i ) ∈ Aki ). This specializes to a q-multiplicativity formula
((w, w)
!
X
q 1−q
e)
e ⊗p
e ) = Sq (w k w)
e +
Sq (p k p
(10)
wi w
ei
Sq (w ⊗ p k w
i : wi >0

e ∈ An , (p, p
e ) ∈ Ak ). Second, q-logarithmic relative entropy has the
((w, w)
same symmetry property as ordinary relative entropy:
Sq (p k r) = Sq (pσ k rσ)

(11)

for all n ≥ 1, (p, r) ∈ An , and permutations σ of {1, . . . , n}.

Theorem 4.1 Let 1 6= q ∈ R and let I(− k −) : An → R n≥1 be a sequence of
functions. The following are equivalent:
i. I(− k −) has the q-multiplicativity property (10) and the symmetry property (11) (both with I in place of Sq );
ii. I(− k −) = cSq (− k −) for some c ∈ R.
Proof It is trivial that (ii) implies (i). Now assume (i). By symmetry,
e ⊗p
e ) = I(p ⊗ w k p
e ⊗ w)
e
I(w ⊗ p k w

e ∈ An , and (p, p
e ) ∈ Ak . So by q-multiplicativity,
for all n, k ≥ 1, (w, w)
!
!
X q 1−q
X
q 1−q
e
e ) = I(p k p
e) +
e +
I(w k w),
pi pei
I(p k p
wi w
ei
I(w k w)
i : pi >0

i : wi >0

or equivalently,
X

i : wi >0

wiq w
ei1−q

!

e ). =
− 1 I(p k p

e = (1/2, 1/2): then
Take w = (1, 0) and w
(2

q−1

e) =
− 1)I(p k p

X

i : pi >0

X

i : pi >0

pqi pe1−q
i

pqi pe1−q
i

!

e
− 1 I(w k w).

!


− 1 I (1, 0) (1/2, 1/2)

e ) ∈ Ak . But q 6= 1, so we can define
for all (p, p
c=


1−q
· I (1, 0) (1/2, 1/2) ,
−1

2q−1

and then I(− k −) = cSq (− k −).



Remarks 4.2 Other characterization theorems for q-logarithmic relative entropy have been proved. For example, Furuichi ([8], Section IV) obtained the
same conclusion, but also assumed continuity and essentially the full chain rule
(that is, an equivalent special case, as in Remarks 2.6(ii) and 3.2(iii)).
9

Acknowledgements I thank John Baez and Tobias Fritz for their comments.
This work was partially supported by a BBSRC FLIP award (BB/P004210/1).

References
[1] J. Baez and T. Fritz. A Bayesian characterization of relative entropy. Theory and
Applications of Categories, 29:421–456, 2014.
[2] S. Banach. Sur l’équation fonctionnelle f (x + y) = f (x) + f (y). Fundamenta
Mathematicae, 1:123–124, 1920.
[3] G. E. P. Box and D. R. Cox. An analysis of transformations. Journal of the Royal
Statistical Society. Series B (Methodological), 26:211–252, 1964.
[4] I. Csiszár. Axiomatic characterizations of information measures. Entropy, 10:261–
273, 2008.
[5] Z. Daróczy. Generalized information functions. Information and Control, 16:36–
51, 1970.
[6] D. K. Faddeev. On the concept of entropy of a finite probabilistic scheme (in
Russian). Uspekhi Matematicheskikh Nauk, 11:227–231, 1956.
[7] A. Feinstein. Foundations of Information Theory. McGraw–Hill, New York, 1958.
[8] S. Furuichi. Uniqueness theorems for Tsallis entropy and Tsallis relative entropy.
IEEE Transactions on Information Theory, 51:3638–3645, 2005.
[9] G. Hardy, J. E. Littlewood, and G. Pólya. Inequalities. Cambridge University
Press, Cambridge, 2nd edition, 1952.
[10] J. Havrda and F. Charvát. Quantification method of classification processes:
concept of structural α-entropy. Kybernetika, 3:30–35, 1967.
[11] A. Hobson. A new theorem of information theory. Journal of Statistical Physics,
1(3):383–391, 1969.
[12] P. Kannappan and C. T. Ng. Measurable solutions of functional equations related
to information theory. Proceedings of the American Mathematical Society, 38:303–
310, 1973.
[13] P. Kannappan and C. T. Ng. On functional equations connected with directed
divergence, inaccuracy and generalized directed divergence. Pacific Journal of
Mathematics, 54(1):157–167, 1974.
[14] P. Kannappan and P. N. Rathie. On a characterization of directed divergence.
Information and Control, 22:163–171, 1973.
[15] J. Lindhard and V. Nielsen. Studies in statistical dynamics. Matematisk-Fysiske
Meddelelser: Kongelige Danske Videnskabernes Selskab, 38:1–42, 1971.
[16] G. P. Patil and C. Taillie. Diversity as a concept and its measurement. Journal
of the American Statistical Association, 77(379):548–561, 1982.
[17] C. E. Shannon. A mathematical theory of communication. Bell System Technical
Journal, 27:379–423, 1948.
[18] H. Suyari. On the most concise set of axioms and the uniqueness theorem for
Tsallis entropy. Journal of Physics A: Mathematical and General, 35:10731–10738,
2002.
[19] C. Tsallis. Possible generalization of Boltzmann–Gibbs statistics. Journal of
Statistical Physics, 52:479–487, 1988.
[20] S. Umarov, C. Tsallis, and S. Steinberg. On a q-central limit theorem consistent
with nonextensive statistical mechanics. Milan Journal of Mathematics, 76:307–
328, 2008.
[21] A. Wehrl. General properties of entropy. Reviews of Modern Physics, 50(2):221–
260, 1978.

10

