arXiv:1712.05311v1 [math.ST] 14 Dec 2017

Electronic Journal of Statistics
Vol. 0 (0000)
ISSN: 1935-7524
DOI: 10.1214/154957804100000000

Prior Distributions for the
Bradley-Terry Model of Paired
Comparisons
John T. Whelan
School of Mathematical Sciences and
Center for Computational Relativity and Gravitation
Rochester Institute of Technology
Rochester, NY 14623
Max-Planck-Institut für Gravitationsphysik
(Albert-Einstein-Institut)
D-30167 Hannover, Germany
e-mail: jtwsma@rit.edu
Abstract: The Bradley-Terry model assigns probabilities for the outcome
of paired comparison experiments based on strength parameters associated
with the objects being compared. We consider different proposed choices of
prior parameter distributions for Bayesian inference of the strength parameters based on the paired comparison results. We evaluate them according to
four desiderata motivated by the use of inferred Bradley-Terry parameters
to rate teams on the basis of outcomes of a set of games: invariance under
interchange of teams, invariance under interchange of winning and losing,
normalizability and invariance under elimination of teams. We consider
various proposals which fail to satisfy one or more of these desiderata, and
illustrate two proposals which satisfy them. Both are one-parameter independent distributions for the logarithms of the team strengths: 1) Gaussian
and 2) Type III generalized logistic.
MSC 2010 subject classifications: 62F15.
Keywords and phrases: Bayesian inference, paired comparisons, BradleyTerry model.

1. Background
1.1. Paired Comparison Experiments and the Bradley-Terry Model
A paired comparison experiment is a set of binary comparisons between pairs
out of a set of t objects. The Bradley-Terry model (Zermelo, 1929; Bradley and
Terry, 1952) assigns to each object i (i = 1, . . . t) a strength parameter πi , and
defines
πi
θij =
(1.1)
πi + πj
as the probability that object i will be preferred in any given comparison with
object j. Note that θji = 1 − θij . If nij the number of comparisons between i
and j, the probability of any particular set of outcomes D which includes object
0

John T. Whelan/Priors for Bradley-Terry

1

i being chosen over j wij times is
p(D|{θij }) = p(D|{πi }) =

t
t
Y
Y
i=1 j=i+1

w

θijij (1 − θij )nij −wij

(1.2)

The model has been used in a number of contexts, ranging from taste tests
between different foods to games between chess players. In the context of the
present paper, we are interested in sporting competitions, so we will henceforth
refer to the objects as “teams” and the comparisons as “games”. wij is thus
the number of games won by team i against team j, and nij = wij + wji is the
number of games between them.
1.2. Bayesian Approach
A typical problem is to make inferences about the strengths {πi }, or equivalently
the log-strengths {λi }, given the results D. Under a Bayesian approach, in terms
of the vector π of team strengths {πi |i = 1, . . . , t}, the posterior probability
distribution will be, up to a π-independent normalization constant,
fΠ|D (π|D, I) ∝ p(D|{πi })fΠ (π|I)

(1.3)

We are concerned with choices of the prior distribution fΠ (π|I), with a given
choice represented symbolically by I.
It is useful to define λi = ln πi and note that
ln

θij
= λi − λj =: γij
1 − θij

(1.4)

and
θij = (1 + e−γij )−1 ,

(1 − θij ) = (1 + eγij )−1

(1.5)

Since the parameters are continuous, the probability density functions transform
as follows:
fΛi (λi ) = eλi fΠi (eλi )
(1.6)
fΠi (πi ) =

1
fΛ (ln πi )
πi i

(1.7)

and likewise
fΓij (γij ) = (1 + e−γij )−1 (1 + eγij )−1 fΘij ([1 + e−γij ]−1 )

(1.8)

−1
−1
fΘij (θij ) = θij
(1 − θij )−1 fΓij (− ln[θij
− 1])

(1.9)

Note that the t strengths {πi } are only relevant in their use to determine
the probabilities {θij } (of which t − 1 are independent), so we consider two
probability distributions fΠ (π|I1 ) and fΠ (π|I2 ) equivalent if they produce the
same marginalized distribution for the {θij }.

John T. Whelan/Priors for Bradley-Terry

2

Definition 1.1. Let γ represent a t − 1-dimensional vector of linearly independent combinations of the log-odds-ratios {γij } from which all t(t − 1) can be
constructed according to
t−1
X
γij =
Cij,α γα
(1.10)
α=1

Possible choices are
γ12 , γ23 , . . . , γ(t−1),t

(1.11)

γ1t , γ2t , . . . , γ(t−1),t

(1.12)

or
or
1
1
1
√ γ12 , √ (γ13 + γ23 ), . . . , p
[γ12 + γ23 + . . . − (t − 1)γ(t−1),t ] (1.13)
2
6
t(t − 1)
The advantage of working with the {γij } is that we need not specify which
basis we are using for γ because the Jacobian determinants for transformations
between different bases are constant.
Definition 1.2. Two probability distributions are equivalent, fΠ (π|I1 ) ∼
= fΠ (π|I2 )
(or fΛ (λ|I1 ) ∼
= fΛ (λ|I2 )) if and only if fΓ (γ|I1 ) = fΓ (γ|I2 ).
Lemma 1.1. A sufficient condition for fΠ (π|I1 ) ∼
= fΠ (π|I2 ) is that there exists
a scalar function C(π) such that the transformation
π 0 = πC(π)

(1.14)

converts the probability density fΠ (π|I1 ) into fΠ (π|I2 ), i.e.,
fΠ0 (π 0 |I1 ) =

fΠ (π|I1 )
n 0 o = fΠ (π 0 |I2 )
∂π
det ∂πji

(1.15)

Proof. The transformation leaves θij unchanged
0
θij
=

πi0

πi0
πi C(π)
πi
=
=
= θij
0
+ πj
πi C(π) + πj C(π)
πi + πj

(1.16)

0
and therefore γij
= γij and the transformation γ → γ 0 leaves fΓ (γ|I) unchanged, and

fΓ (γ|I2 ) = fΓ (γ 0 |I2 ) = fΓ0 (γ 0 |I1 ) = fΓ (γ|I1 )

(1.17)

3

John T. Whelan/Priors for Bradley-Terry

1.3. Motivation and Desiderata
The primary interest motivating this work is design of rating systems to evaluate teams based on the outcome of games between them. To that end, prior
information which distinguishes between the teams is inappropriate, as it would
be considered “unfair” to build such information into the rating system. We are
interested in rating systems which obey as many as possibly of the following
desiderata.
Desideratum 1.1. Invariance under interchange of teams. A transformation
π → π 0 which, for some i and j, obeys πi0 = πj , πj0 = πi , πk0 = πk for all other k,
should transform fΠ (π|I1 ) into an equivalent distribution fΠ (π|I2 ) ∼
= fΠ (π|I1 ).
Desideratum 1.2. Invariance under interchange of winning and losing. The
transformation ∀i : πi → πi0 = π1i , which corresponds to λ0 = −λ, ∀i, j :
0
θij
= 1 − θij , and γ 0 = −γ, should transform fΠ (π|I1 ) into an equivalent
distribution fΠ (π|I2 ) ∼
= fΠ (π|I1 ). A distribution obeying this desideratum will
satisfy fΓ (γ|I1 ) = fΓ (−γ|I1 ).
Desideratum 1.3. Normalizability.
fΓ (γ|I) should be a proper prior, which
R∞
R∞
can be normalized to −∞ · · · −∞ dt−1 γ fΓ (γ|I) = 1.
Desideratum 1.4. Invariant under elimination of teams. This desideratum
assumes that a given principle can be used to generate prior distributions for
any number of teams. Let t > 2, and define π to be the vector of t strengths,
and π 0 to be the (t − 1)-element vector with πi0 = πi for i = 0, . . . , t − 1. Suppose the principle generates priors fΠ0 (π 0 |It−1 ) when there are t − 1 teams and
fΠ (π|It ) = fΠ0 ,Πt (π 0 , πt |It ) when there are t. The prior associated with It−1
should be equivalent to that associated with It , marginalized over πt , i.e.
Z ∞
Z ∞
0
0
∼
0
0
fΠ (π |It−1 ) =
dπt fΠ ,Πt (π , πt |It ) =
dπt fΠ (π|It )
(1.18)
0

0

1.4. Comparison via Prior Predictive Distribution
A convenient way to quantify the effects of a prior, and thus to compare different
priors, is to construct the prior predictive distribution
Z ∞
Z ∞
Z ∞
Z ∞
t
p(D|n, I) =
···
d π p(D|π, n) fΠ (π|I) =
···
dt−1 γ p(D|γ, n) fΓ (γ|I)
0

0

−∞

−∞

(1.19)
where the second equality holds because the t − 1 log-rating differences in γ
determine the sampling distribution p(D|π, n).
Lemma 1.2. For any n, p(D|n, I) = p(D|n, I 0 ) is a necessary condition for
fΠ (π|I) ∼
= fΠ (π|I 0 ).
Proof. Assume fΠ (π|I) ∼
= fΠ (π|I 0 ). Then, by definition fΓ (γ|I) = fΓ (γ|I 0 ). By
(1.19), p(D|n, I) can be constructed from fΓ (γ|I) and p(D|γ, n), and therefore
fΓ (γ|I) = fΓ (γ|I 0 ) implies p(D|n, I) = p(D|n, I 0 ).

John T. Whelan/Priors for Bradley-Terry

4

We can use the prior predictive distribution to check the desiderata.
Lemma 1.3. Given a n where nk` = n for all k 6= `, desideratum 1.1 implies
p(D|n, I) = p(D0 |n, I) where D and D0 differ only by the interchange of a pair
0
0
0
of teams i and j, i.e., wij
= wji , wik
= wjk , and wjk
= wik , for all k 6∈ {i, j}.

0
0
Proof. Defining n0k` = wk`
+ w`k
we see that for all k 6= `, n0k` = n = nk` , i.e.,
0
0
n = n. If we define π → π as in the statement of desideratum 1.1 (πi0 = πj ,
πj0 = πi , πk0 = πk for all k 6= i, j), we can see p(D0 |γ 0 , n, I) = p(D0 |π 0 , n, I) =
0
p(D0 |π 0 , n0 , I) = p(D|π, n, I) = p(D|γ, n, I) where as usual γij
= ln(πi0 /πj0 ). If
0
desideratum 1.1 holds, we have fΓ0 (γ |I) = fΓ (γ|I) and so
Z ∞
Z ∞
p(D0 |n, I) =
···
dt−1 γ 0 p(D0 |γ 0 , n) fΓ (γ 0 |I)
−∞
−∞
Z ∞
Z ∞
=
···
dt−1 γ 0 p(D|γ, n) fΓ (γ|I)
(1.20)
−∞
−∞
Z ∞
Z ∞
=
···
dt−1 γ p(D|γ, n) fΓ (γ|I) = p(D|n, I)
−∞

−∞

because the change of variables γ → γ 0 has unit Jacobian determinant and
leaves the range of the integration variables unchanged.
Lemma 1.4. For any n, desideratum 1.2 implies p(D|n, I) = p(D0 |n, I) where
0
wij
= nij − wij .

0
0
0
Proof. Since wij
= nij − wij = wji , n0ij = wij
+ wji
= wji + wij = nij . The rest
of the proof proceeds as with lemma 1.3, but with the appropriate definitions
of D → D0 and π → π 0 .

Lemma 1.5. For any n, desideratum 1.3 implies p(D|n, I) > 0 if D is a set of
results consistent with n.
Proof. Since p(D|θ, I) > 0 for all θ with 0 < θij < 1, and p(D|γ, I) = p(D|θ, I),
we
I) > 0 for all γ with −∞ < γij < ∞. Since fΓ (γ|I) ≥ 0 and
R ∞ haveRp(D|γ,
∞
t−1
·
·
·
d
γ fΓ (γ|I) = 1, we must have
−∞
−∞
Z

∞

p(D|n, I) =
−∞

···

Z

∞

−∞

dt−1 γ p(D|γ, n) fΓ (γ|I) ≥ 0

(1.21)

Lemma 1.6. Given t teams and a n with nit = 0, desideratum 1.4 implies
p(D|n, It ) = p(D|n, It−1 ) if D is a set of results consistent with n.
Proof. If nit = 0, πt is irrelevant to the sampling distribution, and p(D|π, n) =
p(D|π 0 , n) where π 0 is the (t−1)-element vector with πi0 = πi for i = 0, . . . , t−1,

John T. Whelan/Priors for Bradley-Terry

as in the statement of desideratum 1.4. Thus
Z ∞
Z ∞
dt π p(D|π, n) fΠ (π|It )
···
p(D|n, It ) =
0
0
Z ∞
Z ∞
Z ∞
=
···
dt−1 π 0 p(D|π 0 , n)
dπt fΠ (π|It )
0
0
0
Z ∞
Z ∞
=
···
dt−1 π 0 p(D|π 0 , n)fΠ0 (π 0 |It )
0

5

(1.22)

0

Desideratum 1.4 says that fΠ0 (π 0 |It ) ∼
= fΠ0 (π 0 |It−1 ), and lemma 1.2 states that
this implies p(D|n, It ) = p(D|n, It−1 ).
2. Choice of Prior Distribution
2.1. General Considerations for Special Cases
2.1.1. Two Teams
1
When t = 2, there is only one independent probability θ12 = π1π+π
, so any
2
distribution fΠ (π1 , π2 ) reduces to a function fΘ12 (θ12 ) via marginalization
Z ∞
fΓ12 (γ12 ) =
dλ2 fΛ (γ12 + λ2 , λ2 )
(2.1)

−∞

where the transformation (1.6) means
fΛ (λ1 , λ2 ) = eλ1 +λ2 fΠ (eλ1 , eλ2 )

(2.2)

−1
−1
fΘ12 (θ12 ) = θ12
(1 − θ12 )−1 fΓ12 (− ln[θ12
− 1])

(2.3)

and (1.9) means

For the case of two teams, desiderata 1.1 and 1.2 are equivalent, as both transformations reduce to θ12 → 1 − θ12 , or equivalently γ12 → −γ12 . They will be
satisfied if and only if fΓ12 (γ12 ) is an even function, or equivalently if fΘ12 (θ12 ) =
fΘ12 (1 − θ12 ). Desideratum 1.3 will be satisfied if and only if
1

Z
0

or equivalently
Z

dθ12 fΘ12 (θ12 ) < ∞

(2.4)

dγ12 fΓ12 (γ12 ) < ∞

(2.5)

∞

−∞

Suppose fΘ12 (θ12 ) belongs to the family of beta distributions (which is the
conjugate prior family for the likelihood (1.2)),
α−1
fΘ12 (θ12 ) ∝ θ12
(1 − θ12 )β−1

(2.6)

John T. Whelan/Priors for Bradley-Terry

6

then fΓ12 (γ12 ) is a generalized logistic distribution of Type IV (Prentice, 1976;
Nassar and Elmasry, 2012)
fΓ12 (γ12 ) ∝ (1 + e−γ12 )−α (1 + eγ12 )−β

(2.7)

Included in this family are
1. The Haldane prior (Haldane, 1932; Jeffreys, 1961) α = β = 0, which is
uniform γ12 . This improper prior corresponds to “total ignorance”.
2. The Jeffreys prior (Jeffreys, 1961, 1946) α = β = 12 .
3. The Bayes-Laplace prior α = β = 1, which is uniform in θ12 . This is also
the maximum entropy prior, if we assume a measure uniform in θ12 .
For the beta family, desiderata 1.1 and 1.2 will be satisfied if α = β. Desideratum
1.3 will be satisfied if α, β > 0.
With t = 2, the prior predictive probability for a set of results which include
w12 wins for team 1 and w21 = n12 − w12 wins for team 2 will be
Z 1
w12
p(D|n12 ) =
dθ12 θ12
(1 − θ12 )w21 fΘ12 (θ12 )
(2.8)
0

If fΘ12 (θ12 ) is in the Beta family (2.6), it will be
p(D|n12 , Iα,β ) =

Γ(α + β) Γ(α + w12 )Γ(β + w21 )
Γ(α)Γ(β)
Γ(α + β + n12 )

(2.9)

In particular, for the Bayes-Laplace prior,
p(D|n12 , I1,1 ) =

Γ(1 + w12 )Γ(1 + w21 )
=
Γ(2 + n12 )



−1
n12
(n12 + 1)
w12

(2.10)

For the Jeffreys prior,
p(D|n12 , I1/2,1/2 ) =

Γ( 21 + w12 )Γ( 12 + w21 )
(2w12 − 1)!!(2w21 − 1)!!
=
(2.11)
πΓ(1 + n12 )
2n12 n12 !

Viewing the Haldane prior as a limiting case, p(D|n12 , I1/2,1/2 ) = 0 unless w12 =
n12 or w21 = n12 . The prior predictive probabilities for w12 = n12 and w21 = n12
depend on the order in which the limits α → 0 and β → 0 are taken.
2.1.2. Three Teams
In the case t = 3, the {θij } are related by θji = 1 − θij as well as
−1
−1
−1
θ13
− 1 = (θ12
− 1)(θ23
− 1)

(2.12)

Although each θij is confined to the finite range [0, 1], the surface defined by
(2.12) is curved, which makes it difficult to display the two-dimensional probability distribution fΘ (θ) while preserving its intuitive interpretation. On the
other hand, in terms of the {γij } the constraints are γji = −γij and
γ13 = γ12 + γ23

(2.13)

John T. Whelan/Priors for Bradley-Terry

7

An especially convenient set of coördinates for displaying probability distributions fΓ (γ) is
1
y = γ23 ,
(2.14)
x = √ (γ12 + γ13 ),
3
which can be inverted to give
√
3
1
γ12 =
x − y,
2
2

γ23 = y,

γ13 =

√

3
1
x+ y
2
2

(2.15)

These two-dimensional coördinates on the space of log-odds-ratios γ are also
two of the three coördinates on the space of strengths λ, according to
r
2
1
y = λ2 − λ3 ,
z=
(λ1 + λ2 + λ3 ) (2.16)
x = √ (2λ1 − λ2 − λ3 ),
3
3
In these coördinates, to go from a distribution fΛ (λ) to fΓ (γ) one simply converts fΛ (λ) into fXY Z (x, y, z) (which involves a constant Jacobian determinant)
and then marginalizes over z. An example of such a plot is shown in Fig. 1.
2.2. Evaluation of Prior Distributions
We now consider several families of prior distributions which have been proposed, and evaluate them according to the desiderata in Sec. 1.3.
2.2.1. Haldane Prior
Perhaps the simplest prior that can be chosen is the improper prior
fΛ (λ|I0 ) = constant ,

(2.17)

uniform in all of the log-strengths, which is the generalization of the Haldane
prior considered in Sec. 2.1.1. Then the marginalized prior probability distribution for the log-odds-ratios is fΓ (γ|I0 ) = constant. This prior obviously satisfies
desiderata 1.1 and 1.2, as well as desideratum 1.4. Of course, since the prior
fΓ (γ|I0 ) is improper, it violates desideratum 1.3. The mode of the posterior
fΛ|D (λ|D, I0 ) will be the maximum likelihood solution, and the posterior will
be normalizable under the conditions given by Ford (1957) for the existence of
the maximum likelihood solution.
Note that while the improper prior fΠ (π|I00 ) = constant produces a different
prior on the individual strengths, since
!
t
X
0
fΛ (λ|I0 ) ∝ exp
λi
(2.18)
i=1

the two are equivalent, fΛ (λ|I00 ) ∼
= fΛ (λ|I0 ), because fΛ (λ|I00 ) depends only on
the sum of the log-strengths.

8

John T. Whelan/Priors for Bradley-Terry

γ23 = 6

6

γ1

0

6

γ 13

−2

γ1

−6

−4

2

=

=

λ2 − λ3

2

6

2

=

=

γ 13

−6

4

−6

γ23 = −6
−6

−4

0
2
−2
√
(2λ1 − λ2 − λ3 )/ 3

4

6

Fig 1. Contour plot of the prior probability distribution fΓ (γ|IS3 ) arising from the Maximum
entropy prescription of Sec. 2.2.2 when t = 3. The coördinates for the plot are the x and y
defined in (2.14), which determine all of the {γij } and thus the predicted probabilities. The
orthogonal direction z =
of the model.

p2
3

(λ1 + λ2 + λ3 ) defined in (2.16) is irrelevant to the predictions

2.2.2. Maximum Entropy
The next prior of interest is one which maximizes the Shannon entropy. For the
case t = 2, we defined the entropy as
Z 1
S2 = −
dθ12 fΘ12 (θ12 ) ln fΘ12 (θ12 )
(2.19)
0

which made the maximum entropy prior fΘ12 (θ12 |IS2 ) = const. In order for the
entropy of a continuous distribution f (x) to be invariant, it needs to be defined
with a measure µ(x) which transforms as a density under reparametrization.
Thus we can write
Z 1
Z 1
fΓ (γ12 )
fΘ (θ12 )
=−
dγ12 fΓ12 (γ12 ) ln 12
S2 = −
dθ12 fΘ12 (θ12 ) ln 12
µ
(θ
)
µ
Θ12 12
Γ12 (γ12 )
0
0
(2.20)

9

John T. Whelan/Priors for Bradley-Terry

where we have assumed that µΘ12 (θ12 ) = constant and thus, using (1.8) to
transform the measure,
µΓ12 (γ12 ) ∝ (1 + e−γ12 )−1 (1 + eγ12 )−1

(2.21)

If we maximize the entropy of a continuous distribution with normalization as
the only constraint, the probability density is proportional to the measure, so
fΓ12 (γ12 |IS2 ) ∝ (1 + e−γ12 )−1 (1 + eγ12 )−1

(2.22)

which is indeed of the form (2.7) with α = β = 1.
For general t, we could define by analogy a measure uniform in the {θij },
µΘ (θ) = constant, and then minimize the entropy
St = −

1

Z
0

···

1

Z

dt(t−1)/2 θ fΘ (θ) ln fΘ (θ)

(2.23)

0

subject to the constraints that the probability density vanishes unless the arguments satisfy
−1
−1
−1
θij
= (θik
− 1)(θkj
− 1)

i = 1, . . . , t; j = i + 1, . . . , t; k = i + 1, . . . , j − 1
(2.24)
It is equivalent, and more straightforward, to confine the distribution to the
constraint surface by writing it in terms of the t − 1 unique γα parameters:
Z ∞
Z ∞
fΓ (γ)
0
St = −
···
dt−1 γ fΓ (γ) ln
(2.25)
µ
Γ (γ)
−∞
−∞
where the measure is
µΓ (γ) ∝

t
t
Y
Y

(1 + e−γij )−1 (1 + eγij )−1

(2.26)

i=1 j=i+1

As before, the maximum entropy distribution is fΓ (γ|ISt ) ∝ µΓ (γ), or
fΓ (γ|ISt ) ∝

t
t
Y
Y
i=1 j=i+1

"
1 + exp −

t−1
X

!#−1 
Cij,α γα



1 + exp 

α=1

t−1
X

−1
Cij,β γβ 

β=1

(2.27)
We can see from the form of (2.26) that desiderata 1.1 and 1.2 are satisfied.
It is also easy to see that fΓ (γ|ISt ) is exponentially suppressed as any linear
combination of the {γα } goes to infinity, and therefore desideratum 1.3. For
example, as γα → ∞,


t
t
t
t
Y
Y
X
X
fΓ (γ|ISt ) →
e−|Cij,α |γα = exp −γα
|Cij,α |
(2.28)
i=1 j=i+1

i=1 j=i+1

10

John T. Whelan/Priors for Bradley-Terry

However, we can see that desideratum 1.4 is not satisfied by considering the
case t = 3 and showing that the marginal distribution
fΓ23 (γ23 |IS3 ) 6= fΓ23 (γ23 |IS2 )

(2.29)

Explicitly. using the coördinates (2.14), in which y = γ23 ,
√

√

fΓ (x, y|IS3 ) ∝ (1 + e

3
2 x

× (1 + e

e−y/2 )−1 (1 + e−

√
3
2 x

3
2 x

ey/2 )−1 (1 + e−y )−1 (1 + ey )−1

√

ey/2 )−1 (1 + e−

3
2 x

e−y/2 )−1 ,
(2.30)

which is plotted in Fig. 1. The marginalization integral can be done by partial
fractions to give
Z ∞
eγ23 [2(1 − eγ23 ) + γ23 (1 + eγ23 )]
fΓ (x, γ23 |IS3 ) dx ∝ γ23
fΓ23 (γ23 |IS3 ) =
(e − 1)3 (1 + e−γ23 )(1 + eγ23 )
−∞
(2.31)
which is manifestly different from
fΓ23 (γ23 |IS2 ) ∝

1
(1 + e−γ23 )(1 + eγ23 )

(2.32)

by more than just a normalization constant.
2.2.3. Jeffreys Prior
The Jeffreys prior construction (Jeffreys, 1946) can be carried out using the
likelihood (1.2), to produce a prior
p
(2.33)
fΛ (λ|IJ ) ∝ I(λ)
where I(λ) is the Fisher information associated with the likelihood. Since the
likelihood is written in terms of the {θij }, or equivalently in terms of the {γij },
it is simpler to generate
p
fΓ (γ|IJ ) ∝ I(γ)
(2.34)
directly. If we write the t − 1 independent elements of γ as
γij =

t−1
X

Cij,α γα

(1.10)

α=1

we can write


 2

∂ `(γ; D)
I(γ) = −E det
∂γα ∂γβ

(2.35)

where
`(γ; D) = ln pD|γ (D|γ) =

t
t
X
X

(wij γij − nij ln[1 + eγij ])

i=1 j=i+1

(2.36)

John T. Whelan/Priors for Bradley-Terry

11

The linear form of (1.10) allows us to determine the Fisher information matrix
for the {γα } from that for the {γij } as
t
t
t
t
∂ 2 `(γ; D) X X X X
∂ 2 `(γ; D)
=
Cij,α Ci0 j 0 ,β
∂γα ∂γβ
∂γij ∂γi0 j 0
0
0
i=1 j=i+1 0

(2.37)

i =1 j =i +1

Since the log-likelihood is relatively simple written in terms of the {γij }, we can
write
∂`(γ; D)
= wij − nij eγij (1 + eγij )−1 = wij − nij (1 + e−γij )−1
∂γij

(2.38)

and
∂ 2 `(γ; D)
= −δii0 δjj 0 nij e−γij (1+e−γij )−2 = −δii0 δjj 0 nij (1+e−γij )−1 (1+eγij )−1
∂γij ∂γi0 j 0
(2.39)
so
−

t
t
∂ 2 `(γ; D) X X
=
Cij,α Cij,β nij (1 + e−γij )−1 (1 + eγij )−1
∂γα ∂γβ
i=1 j=i+1

(2.40)

We can see by inspection of (2.40) that the Jeffreys prior always satisfies desideratum 1.2. We can verify that in the case t = 2, for which there is only one
independent γα , the Jeffreys prior becomes
fΓ (γ12 |IJ2 ) ∝ (1 + e−γij )−1/2 (1 + eγij )−1/2
which is of the form (2.7) with α = β =
If we write
−

1
2

(2.41)

as before.

t
t
∂ 2 `(γ; D) X X
Cij,α Cij,β nij (eγij /2 + e−γij /2 )−2
=
∂γα ∂γβ
i=1 j=i+1

(2.42)

Note that if we make the specific choice γα = γα,α+1 , we have
γij =

j−1
X

γα

(2.43)

α=i

which makes

(
Cij,α =

1
0

i≤α≤j−1
otherwise

(2.44)

and then the Fisher information matrix (2.40) is
−

∂ 2 `(γ; D)
=
∂γα ∂γβ

X
i,j
i≤α≤j−1
i≤β≤j−1

nij (eγij /2 + e−γij /2 )−2

(2.45)

John T. Whelan/Priors for Bradley-Terry

12

For t > 2, the Fisher information matrix (2.40) depends on the number of
games nij to be played between each pair of teams. However, in order to satisfy
desideratum 1.1, we need to have the same nij for each pair of teams, in which
case this nij becomes a constant which can be absorbed into the normalization,
and the prescription becomes unique.
By explicitly examining t = 3, we will show that the Jeffreys prior with all
{nij } equal fails to satisfy desideratum 1.4.1 In the case t = 3, the vector of
independent log-odds-ratios γ is two-dimensional, and the elements of the Fisher
information matrix are
∂ 2 `(γ; D)
= M12 + M13
∂γ1 ∂γ1
∂ 2 `(γ; D)
−
= M13
∂γ1 ∂γ2
∂ 2 `(γ; D)
= M13 + M23
−
∂γ2 ∂γ2

(2.46b)

Mij := nij (eγij /2 + e−γij /2 )−2 =: nij mij

(2.47)

−

(2.46a)

(2.46c)

where
The Fisher information is the determinant of this matrix
2
I(γ) = (M12 +M13 )(M13 +M23 )−M13
= M12 M13 +M12 M23 +M13 M23 (2.48)

so the Jeffreys prior is
p
√
fΓ (γ|IJ3 ) ∝ M12 M13 + M12 M23 + M13 M23 ∝ m12 m13 + m12 m23 + m13 m23
(2.49)
We can show that the Jeffreys prior fails to satisfy desideratum 1.4 by using the
posterior predictive distribution and Lemma 1.6. Suppose n12 = 2 and ni3 = 0.
Then (2.11) implies
(
0.125 w12 = 1
p(D|n, IJ2 ) =
(2.50)
0.375 w12 = 0 or 2
We can evaluate p(D|n, IJ3 ) numerically, and find
(
0.108 w12 = 1
p(D|n, IJ3 ) ≈
0.392 w12 = 0 or 2

(2.51)

showing explicitly that p(D|n, IJ3 ) 6= p(D|n, IJ2 ) and therefore desideratum 1.4
is violated.
1 It can be seen to satisfy 1.3 by noting that as a linear combination of the {γ } goes
α
to infinity, at most one of the {mij } can remain finite; the other two will be exponentially
suppressed, and thus each term in the square root in (2.49) will go to zero exponentially.

John T. Whelan/Priors for Bradley-Terry

13

2.2.4. Dirichlet Distribution
Chen and Smith (1984) discuss Bayesian estimators for the Bradley-Terry model
starting with a Dirichlet distribution
P

!
!
t
t
t
Γ
Y
X
i=1 αi
αi −1
πi
δ 1−
fΠ (π|IDt ) = Qt
πi
(2.52)
i=1 Γ(αi )
i=1
i=1
where δ(x) is the Dirac delta function. In particular, they note that the marginal
distribution for any of the {θij } is a beta distribution with parameters α = αi
and β = αj :
Γ(αi + αj ) αi −1
(1 − θij )αj −1
(2.53)
θ
fΘij (θij |IDt ) =
Γ(αi )Γ(αj ) ij
The Dirichlet prior satisfies desideratum 1.3 as long as all of the {αi } are positive; it also satisfies desideratum 1.1 if all of the parameters {αi } are equal
to
same value α. Although the delta function enforcing the constraint
Pta the
αi
π
i=1 i = 1 means that the different {πi } are not independently distributed
under IDt , we can see that desideratum 1.4 is satisfied by defining a change of
variables
πi
πi0 =
i = 1, . . . , t − 1
(2.54)
1 − πt
under which the probability density (2.52) becomes
P

!
!
t
t−1
t−1
Pt−1
Γ
αi
X
Y αi −1
i=1
0
0
0
fΠ0 ,Πt (π , πt |IDt ) = Qt
πi
δ 1−
πi (1−πt ) i=1 αi −1 πt αt −1
i=1 Γ(αi )
i=1
i=1
(2.55)
which, when we marginalize over πt , gives
P

!
!
t−1
t−1
t−1
Γ
α
Y αi −1
X
i
i=1
0
0
0
πi
δ 1−
πi
(2.56)
fΠ0 (π |IDt ) = Qt−1
i=1 Γ(αi )
i=1
i=1
which is a Dirichlet distribution with the same parameters {α1 , . . . , αt−1 }.
However, desideratum 1.2 is not satisfied, which we can see explicitly by
considering t = 3 and assuming α1 = α2 = α3 ≡ α, so that
fΠ (π1 , π2 , π3 |ID3 ) ∝ (π1 π2 π3 )α−1 δ(π1 + π2 + π3 − 1)

(2.57)

or equivalently

fΛ (λ1 , λ2 , λ3 |ID3 ) ∝ eα(λ1 +λ2 +λ3 ) δ eλ1 + eλ2 + eλ3 − 1

(2.58)

If we write
1/3 
1/3 
1/3 !
π1 π1
π2 π2
π3 π3
π1 + π2 + π3 = (π1 π2 π3 )
+
+
π2 π3
π1 π3
π1 π2
 γ12 +γ13

−γ12 +γ23
−γ13 −γ23
λ1 +λ2 +λ3
3
3
3
= eλ1 + eλ2 + eλ3 = e
e 3
+e
+e
(2.59)
1/3



14

John T. Whelan/Priors for Bradley-Terry

γ23 = 6

γ1

0

6

γ 13

−2

γ1

−6

−4

2

=

=

λ2 − λ3

2

6

2

=

=

γ 13

4

−6

6

−6

γ23 = −6
−6

−4

0
2
−2
√
(2λ1 − λ2 − λ3 )/ 3

4

6

Fig 2. Contour plot of the prior probability distribution fΓ (γ|ID3 ) arising from the Dirichlet
prior described in Sec. 2.2.4 when α = 1 and t = 3. We see that the distribution is not
invariant under the inversion γ → −γ, and therefore desideratum 1.2 is not satisfied.

we can see
i

h γ12 +γ13

−γ12 +γ23
−γ13 −γ23
3
3
δ eλ1 + eλ2 + eλ3 − 1 ∝ δ λ1 + λ2 + λ3 + 3 ln e 3
+e
+e
(2.60)
and so marginalizing over the combination λ1 +λ2 +λ3 leaves a prior distribution
 γ12 +γ13
−3α
−γ12 +γ23
−γ13 −γ23
3
3
fΓ (γ|ID3 ) ∝ e 3
+e
+e

(2.61)

We see that, for non-zero α, fΓ (γ|ID3 ) 6= fΓ (−γ|ID3 ). This is illustrated explicitly for α = 21 in Fig. 2. Another way of expressing this asymmetry if we’d started
with an “anti-Dirichlet” prior, i.e., requiring {1/πi } to be Dirichlet distributed,
we’d have got the distribution
 −γ12 −γ13
−3α
γ12 −γ23
γ13 +γ23
3
fΓ (γ|ID0 3 ) ∝ e
+e 3
+e 3

(2.62)

15

John T. Whelan/Priors for Bradley-Terry

2.2.5. Conjugate Prior Families
Davidson and Solomon (1973) construct a conjugate prior family of the form

! t
!
t
t
t
Y
Y
Y
X
0
0
vi
−n

fΠ (π|IC ) ∝
πi
(πi + πj ) ij  δ 1 −
πi
(2.63)
i=1

i=1 j=i+1

i=1

where n0ii = 0. In order to satisfy desideratum 1.1 (interchange of teams), we
require that vi0 = v 0 and n0ij = n0 for all i and j 6= i, so the prior becomes
fΠ (π|IC ) ∝

t
Y

!v0 
πi

t
t
Y
Y



i=1

−n0
δ 1−

(πi + πj )

i=1 j=i+1

t
X

!
πi

.

(2.64)

i=1

Note that Davidson and Solomon (1973) motivate vi0 and n0ij as coming from a
Pt
0
0
0
0
0
matrix wij
(with wii
= 0) via vi0 = i=1 wij
and n0ij = wij
+ wji
, which means
Pt Pt
P
t
0
0
that in particular i=1 j=1 nij = 2 i=1 vi , which in the case of single n0
and v 0 parameters would require v 0 = (t − 1)n0 /2. We will not impose that
condition at this stage, however.
To convert fΠ (π|IC ) into fΛ (λ|IC ), we note that
t
X

t
Y

πi =

i=1

!1/t
πk

k=1

1/t
t
Y
π
i

= exp
π
j
i=1
j=1

t
X



t

1X
λk
t
k=1

!


t
X
1
exp 
γij 
t j=1
i=1

t
X



(2.65)
and since, for t > 0,


δ(u)
δ 1 − eu/t = 1 u/t = tδ(u) ,
te



!
t
t
t
t
X
X
X
X
1
γij  .
δ 1−
πi = t δ 
λk + t ln
exp 
t
i=1
i=1
j=1

(2.66)

(2.67)

k=1

Similarly, we can write
t
t
Y
Y

(πi + πj ) = Qt

1

k=1

i=1 j=i+1

2πk

v
u t t
uY Y
t
(πi + πj )
i=1 j=1

v

u
!t  t t 

t
u Y
Y
Y
1
π
u
i 
t
= Qt
πk 
1+
π
2t k=1 πk
j
i=1 j=1
k=1

=

1
exp
2t



1/2
! t t
X
t
Y
Y
t
−1
λk 
[1 + eγij ]
2
i=1 j=1
k=1

(2.68)

16

John T. Whelan/Priors for Bradley-Terry

which makes the prior (recalling (1.6))
−n0 /2
! t t


 X
t
YY
t
fΛ (λ|IC ) ∝ exp v 0 + 1 − n0
−1
λk 
[1 + eγij ]
2
i=1 j=1
k=1



t
t
t
X
X
1 X 
γij
×δ
λk + t ln
exp 
t j=1
i=1
k=1

(2.69)
P

Pt
t
When we marginalize over k=1 λk , the Dirac delta function sets exp
k=1 λk
−t
P
 P
t
t
1
and the prior becomes
to
j=1 γij
i=1 exp t
−n0 /2
−t(v0 +1−n0 [t/2−1]) 


t
t
t
t Y
X
X
Y
1

fΓ (γ|IC ) ∝ 
exp 
[1 + eγij ]
γij 
t
i=1
j=1
i=1 j=1
(2.70)
The quantity in large square brackets is in general not symmetric under the
transformation γ → −γ, while the remainder of the expression is. This means, to
satisfy desideratum 1.2 (win-loss inversion), we should have v 0 = (t−2)n0 /2−1.
Note that this is not the same as the condition v 0 = (t−1)n0 /2 implied by Davidson and Solomon (1973)’s conditions on vi0 and n0ij . The precise form of their
restriction, however, comes from the fact they wrote down a “natural” conjugate prior family for fΠ (π|IC ); if they had started with fΛ (λ|IC ), they would
have ended up with, in the present notation, v 0 + 1 = (t − 1)n0 /2, which would
also
Pt not satisfy desideratum 1.2. However, if we hadn’t imposed the constraint
not invariant under πi → 1/πi ) in the first place,
i=1 πi = 1 (which is clearly
Pt
the marginalization over k=1 λk , would have rendered v 0 irrelevant and left
us with

−n0 /2
t Y
t
Y
fΓ (γ|IC ) ∝ 
[1 + eγij ]
(2.71)
i=1 j=1

in any event. We therefore take (2.71) as the form of the conjugate prior
fΓ (γ|IC ), and note that if n0 = 2, this reduces to the maximum entropy distribution (2.27) [see also (2.26)] considered in 2.2.2.
We can show that (2.71) violates desideratum 1.4 for any n0 ∈ (0, ∞) by
considering the prior predictive distribution and invoking Lemma 1.6. First,
note that for t = 2, (2.71) becomes
−n0 /2

fΓ (γ|IC2 ) ∝ ([1 + eγ12 ] [1 + eγ21 ])

(2.72)

which is just the beta/generalized logistic prior (2.7) with α = β = n0 /2.
Therefore, if we define D to be a set of results for which w12 = 1 = w21 , and let

17

John T. Whelan/Priors for Bradley-Terry

Prior predictive probability for w12 = w21
0.25

p(D|n, IC )

0.20
0.15

t=2
t=3
difference

0.10
0.05
0.00
0

5

10

15
n0

20

25

30

Fig 3. Prior predictive probability p(D|n, IC ) for the conjugate prior family of Davidson and
Solomon (1973), for a sequence of results in which w12 = w21 = 1 and n12 = 2. In order
to satisfy desiderata 1.1 and 1.2, we require n0ij = n0 and v 0 = (t − 2)n0 /2 − 1, which
produces the prior (2.71). For any finite positive n0 , the prior predictive probability from the
three-team prior is larger than that of the two-team prior, indicating that desideratum 1.4 is
not satisfied. (For n0 = 0 both versions reduce to the Haldane prior and p(D|n, IC ) = 0. As
n0 → ∞, they become delta functions at γ = 0 and p(D|n, IC ) → 0.25.)

n12 = 2 and ni3 = 0, (2.9) implies
0

p(D|n, IC2 ) =

0

Γ(n0 )Γ( n2 + 1)Γ( n2 + 1)
0
0
Γ( n2 )Γ( n2 )Γ(n0

+ 2)

=

1
4(1 +

1
n0 )

(2.73)

We evaluate p(D|n, IC3 ) numerically for a range of n0 values, plotted in Fig. 3,
and find that for any 0 < n0 < ∞, p(D|n, IC3 ) > p(D|n, IC2 )
2.2.6. Multivariate Gaussian Distribution
Leonard (1977) proposed a multivariate Gaussian prior on the {λi } of the form


t X
t
X
1
(λi − µi )[σ −2 ]ij (λj − µj )
(2.74)
fΛ (λ|IG ) ∝ exp −
2 i=1 j=1
where {[σ −2 ]ij } are the elements of the inverse of a positive semi-definite covariance matrix with elements {[σ 2 ]ij }.
Desideratum 1.3 will be satisfied if the covariance matrix {[σ 2 ]ij } is positive
definite, so that the prior is normalizable.
Desideratum 1.1 requires that all of the {µi } have the same value µ, all of
the variances {[σ 2 ]ii } have the same value σ 2 , and all of the cross-covariances

John T. Whelan/Priors for Bradley-Terry

18

{[σ 2 ]ij |i 6= j} have the same value ρσ 2 . In order for the matrix {[σ 2 ]ij } to be
1
positive definite we must have σ 2 > 0 and − t−1
< ρ < 1.
These conditions guarantee that desideratum 1.2 is satisfied.
Since the distribution fΓ (γ|IG ) is unchanged by the transformation λi →
λi − µ, we can assume without loss of generality that µ = 0. We are thus left
with ρ and σ 2 as the adjustable parameters
of the distribution. However, if we
Pt
make the transformation λi → λi +a j=1 λj , which leaves fΓ (γ|IG ) unchanged,
fΛ (λ|IG ) becomes a multivariate Gaussian with covariance matrix
σ 2 [ρ + δij (1 − ρ) + a(2 + at)(t + 1 − ρ)]
(2.75)
q
q




1−ρ
1−ρ
or a = − 1t 1 − tρ+1−ρ
the covariance matrix
if a = − 1t 1 + tρ+1−ρ

becomes diagonal, with a variance equal to (1 − ρ)σ 2 . Either value for a is
guaranteed to be real by the conditions on ρ which ensure a positive definite
correlation matrix. Thus fΓ (γ|IG ) is equivalent to a product of independent
Gaussian distributions for each λi . For simplicity, we refer to the variance of
each of these distributions as σ 2 rather than (1 − ρ)σ 2 .
Since the prior fΓ (γ|IG ) is equivalent to independent distributions on the
{λi }, it is invariant under elimination of teams, and satisfies desideratum 1.4.
2.2.7. Separable Priors

Thus far, the only prior considered to satisfy all four of our desiderata is the
Gaussian prior which is equivalent to
!
t
X
1
λ2i
fΛ (λ|IG ) =
(2.76)
exp −
2σ 2
(2πσ 2 )t/2
i=1
This is an example of a separable prior of the form
fΛ (λ|I) =

t
Y
i=1

fΛi (λi |I)

(2.77)

A prior of this form, which assigns prior pdfs to the strengths (or log-strengths)
of the teams is guaranteed by its construction to satisfy desideratum 1.4 (invariance under team-elimination). It will also satisfy desideratum 1.1 (interchange) if the distributions for the different {λi } are identical (fΛi (λi |I) =
fΛ (λi |I)), desideratum 1.2 (win-loss interchange) if the distribution fΛ (λi |I) is
even (fΛ (−λi |I) = fΛ (λi |I)) and desideratum 1.3 (normalizable) if fΛ (−λ|I) is
a proper prior.
2.2.8. Beta-Separable Priors
One family of separable priors can be constructed by defining
πi
ζi =
= (1 + e−λi )−1
1 + πi

(2.78)

John T. Whelan/Priors for Bradley-Terry

19

and assuming that ζi obeys a Beta distribution. Explicitly,
fZi (ζi |IB ) ∝ ζiαi −1 (1 − ζi )βi −1

(2.79)

Since ζi = logistic(λi ), just as θij = logistic(γij ), the prior on λi is a generalized
logistic distribution of Type IV (Prentice, 1976; Nassar and Elmasry, 2012):
fΛi (λi |IB ) ∝ (1 + e−λi )−αi (1 + eλi )−βi

(2.80)

To enforce desideratum 1.2, we require αi = βi , and for desideratum 1.1, we
require that all αi and βi parameters are the same, αi = βi = η, which makes
the prior a generalized logistic distribution of Type III.
fΛi (λi |IB ) ∝ (1 + e−λi )−η (1 + eλi )−η

(2.81)

An appealing feature of Beta-separable prior is that its functional form is similar
to the likelihood
p(D|λ) =

t
t
Y
Y

(1 + e−γij )−wij (1 + eγij )nij −wij

(2.82)

i=1 j=i+1

In particular, the posterior is proportional to the likelihood function which would
arise by adding to the actual game results a set of “fictitious games” corresponding to η wins and η losses for each actual team against a fictitious team assumed
to have a strength of 1. So any method for obtaining maximum likelihood estimates such as that of Ford (1957) could be adapted to obtaining maximum
a priori estimates with this prior. This method, with η = 12 , has been used by
Buttler (1993) to ensure regularity of the estimates of Bradley-Terry strengths.
Another “obvious” choice is η = 1, which is equivalent to a uniform prior on ζi .
(The Haldane prior is η = 0.)
3. Conclusions
We have considered various families of prior distributions for team strengths in
the Bradley-Terry model. Motivated by the application of a Bayesian BradleyTerry model to rate teams based only on their game results, we have evaluated
these priors according to the desiderata of invariance under interchange of teams
(1.1), interchange of winning and losing (1.2) and elimination of irrelevant teams
from the model (1.4), as well as normalizability (1.3). A Haldane-like prior of
complete ignorance is not normalizable (violation of 1.3), although it satisfies
the other desiderata. A prior based on maximum entropy arguments, as well
as one from the conjugate family of Davidson and Solomon (1973) which is
required to obey the other desiderata, will depend on the number of teams for
which it was constructed (violation of 1.4). The same is true for the Jeffreys
prior. A Dirichlet prior on the team strengths (Chen and Smith, 1984) can be
made to satisfy the other desiderata, but will not be invariant under interchange
of the definitions of winning and losing (violation of 1.2). Distributions can

John T. Whelan/Priors for Bradley-Terry

20

be constructed which satisfy the desiderata by imposing independent priors
on the strengths of all of the teams. In particular, a multivariate Gaussian in
the log-strengths (Leonard, 1977) which satisfies the desiderata is equivalent
to identical independent Gaussian priors on each of the log-strengths. Another
simple family of separable prior distributions imposes independent generalized
logistic distributions on the log-strengths. In each of these last two cases, a single
parameter remains. Phelan and Whelan (2017) consider the relationship between
the two, and propose a hierarchical method to estimate these parameters rather
than assuming values for them.
Acknowledgments
The author wishes to thank Kenneth Butler and Gabriel Phelan for useful discussions.
References
Bradley, R. A. and Terry, M. E. (1952). Rank Analysis of Incomplete Block
Designs: The Method Of Paired Comparisons. Biometrika 39 324-345.
Buttler,
K.
(1993).
Ken’s
Ratings
for
American
College
Hockey.
HOCKEY-L
mailing
list
post,
http://lists.maine.edu/cgi/wa?A2=Hockey-L;68c89935.9307.
Chen, C. and Smith, T. M. (1984). A Bayes-type estimator for the BradleyTerry model for paired comparison. Journal of Statistical Planning and Inference 10 9-14.
Davidson, R. R. and Solomon, D. L. (1973). A Bayesian approach to paired
comparison experimentation. Biometrika 60 477-487.
Ford, J. L. R. (1957). Solution of a Ranking Problem from Binary Comparisons. The American Mathematical Monthly 64 pp. 28-33.
Haldane, J. B. S. (1932). A note on inverse probability. Mathematical Proceedings of the Cambridge Philosophical Society 28 55–61.
Jeffreys, H. (1946). An Invariant Form for the Prior Probability in Estimation Problems. Proceedings of the Royal Society of London A: Mathematical,
Physical and Engineering Sciences 186 453–461.
Jeffreys, H. (1961). Theory of probability, Third ed. Clarendon Press, Oxford.
Leonard, T. (1977). An Alternative Bayesian Approach to the Bradley-Terry
Model for Paired Comparisons. Biometrics 33 pp. 121-132.
Nassar, M. M. and Elmasry, A. (2012). A study of generalized logistic distributions. Journal of the Egyptian Mathematical Society 20 126-133.
Phelan, G. C. and Whelan, J. T. (2017). Hierarchical Bayesian BradleyTerry for Applications in Major League Baseball. Mathematics for Applications. accepted for publication.
Prentice, R. L. (1976). A Generalization of the Probit and Logit Methods for
Dose Response Curves. Biometrics 32 761-768.

John T. Whelan/Priors for Bradley-Terry

21

Zermelo, E. (1929). Die Berechnung der Turnier-Ergebnisse als ein Maximumproblem der Wahrscheinlichkeitsrechnung. Mathematische Zeitschrift 29
436-460.

