Palindromic Bernoulli distributions
Giovanni M. Marchetti
Dipartimento di Statistica, Informatica, Applicazioni “G. Parenti”, Florence, Italy
e-mail: giovanni.marchetti@disia.unifi.it

and

arXiv:1510.09072v2 [stat.ME] 5 May 2016

Nanny Wermuth
Mathematical Statistics, Chalmers University of Technology, Gothenburg, Sweden
and Medical Psychology and Medical Sociology, Gutenberg University Mainz, Germany
e-mail: wermuth@chalmers.se
Abstract: We introduce and study a subclass of joint Bernoulli distributions which
has the palindromic property. For such distributions the vector of joint probabilities
is unchanged when the order of the elements is reversed. We prove for binary variables that the palindromic property is equivalent to zero constraints on all odd-order
interaction parameters, be it in parameterizations which are log-linear, linear or multivariate logistic. In particular, we derive the one-to-one parametric transformations
for these three types of model specifications and give simple closed forms of maximum
likelihood estimates. Several special cases are discussed and a case study is described.
AMS 2000 subject classifications: Primary 62E10; secondary 62H17, 62H20..
Keywords and phrases: Central symmetry; Linear in probability models; Loglinear models; Multivariate logistic models; Median-dichotomization; Orthant probabilities; Odd-order interactions.

Contents
1
2

3

4

5
6
7

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . .
Characterization in terms of interaction parameters . . . .
2.1 Notation . . . . . . . . . . . . . . . . . . . . . . . . .
2.2 Log-linear parameters . . . . . . . . . . . . . . . . .
2.3 Linear interactions or moment parameters . . . . . .
2.4 Multivariate logistic parameters . . . . . . . . . . . .
2.5 Properties of the palindromic Bernoulli distributions
Independences and dependences . . . . . . . . . . . . . . .
3.1 Induced dependences and effect reversal . . . . . . .
3.2 Combination of independences . . . . . . . . . . . .
Some special cases . . . . . . . . . . . . . . . . . . . . . .
4.1 Median dichotomization . . . . . . . . . . . . . . . .
4.2 Maximum likelihood estimation . . . . . . . . . . . .
Palindromic Ising models . . . . . . . . . . . . . . . . . .
A case study . . . . . . . . . . . . . . . . . . . . . . . . .
Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . .
1

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

2
3
3
4
4
5
6
9
10
11
12
12
14
16
17
19

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

Appendix 1. Proofs . . . . . .
Appendix 2. The data for the
Acknowledgement . . . . . .
References . . . . . . . . . . .

. . . . . .
case study
. . . . . .
. . . . . .

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

2

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

21
22
22
23

1. Introduction
A sequence of characters, such as QR*-TS, becomes a palindromic sequence when the
order of the characters is reversed and appended, here to give QR*-TSST-*RQ. The notion
is used in somewhat modified forms, among others, in musicology, biology and linguistics.
An example of a palindromic sentence which respects the spacings between words is ‘step
on no pets’.
Here, we adapt the term to Bernoulli distributions. For a single binary variable, the
distribution is palindromic if it is uniform, that is if both levels occur with probability
1/2. For a Bernoulli distribution of d binary variables A1 , . . . , Ad , having a probability
mass function p(a) with a in the set of all binary d-vectors, the distribution is palindromic
if p(a) = p(∼ a) for all a, where ∼ a is the complement of a; for instance, ∼ a = (0, 1, 0)
for a = (1, 0, 1).
With α, β, γ, δ denoting probabilities, bivariate and trivariate palindromic Bernoulli
distributions can be written, as in the following tables:
A1 A2 : 0

1

sum

0

α

β

1/2

1

β

α

1/2

sum

1/2 1/2

1

A3 : 0

0

1

1

A1 A2 : 0

1

0

1

sum

0

α

γ

δ

β

1/2

1

β

δ

γ

α

1/2

sum

α+β γ+δ γ+δ α+β

1

Continuous distributions may also be palindromic. This concept extends the discussion
of the diverse forms of multivariate symmetry by Serfling (2006) since it operationalizes
his notion of central symmetry in an attractive way. Let the above binary vector a define
for d mean-centred continuous variables their orthant probabilities. Such a distribution is
palindromic if the probabilities of the d(a)-orthant and the d(∼ a)-orthant coincide for all
a. Examples are for instance mean-centred Gaussian and spherical distributions.
For general discrete variables, the palindromic property differs from complete symmetry
defined by Bhapkar & Darroch (1990). Complete symmetric tables satisfy the condition
p(a) = p(σ(a)), for any a and for any permutation σ of the indices. Edwards (2000, app.
C) median-dichotomized joint Gaussian distributions and proved that the resulting binary
probabilities give a non-hierarchical log-linear model in which all odd-order interactions
vanish, that is all terms involving an odd number of factors are zero.
In this paper, we study properties of palindromic Bernoulli distributions in general. In
particular, we prove that the vanishing of all odd-order log-linear interactions is not only
a necessary but also a sufficient condition. We show the same characterization for models

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

3

3
90
2
85
Physics

Y

1
0

80
75

-1
70
-2
65
-3
-3

-2

-1

0

1

2

3

65

70

75

X

80

85

90

Algebra

Fig 1. Contour levels of a centrally symmetric bivariate density (left) and a median dichotomized sample
(right). The data are from the case study of section 6.

linear-in-probabilities, Streitberg (1990), and for the multivariate logistic parametrization,
Glonek & McCullagh (1995), and explain why palindromic Bernoulli distributions with
Markov structure are in the regular exponential family.
2. Characterization in terms of interaction parameters
In this section we introduce three types of parameterization and we show that the palindromic Bernoulli distributions can be characterized by the vanishing of all odd-order interactions, no matter whether these are log-linear, linear or multivariate logistic terms.
2.1. Notation
Let A = (A1 , . . . , Ad ) be a random vector with a multivariate Bernoulli distribution. Thus,
A takes values a = (a1 , . . . , ad ) in the set I = {0, 1}d with probabilities
p(a) = Pr(A1 = a1 , . . . , Ap = ap ),

P

a∈I

p(a) = 1.

For simplicity, we assume p(a) > 0 for all a. The probability distribution of A is determined
by the 2d × 1 vector π containing all the probabilities p(a) and belonging to the (2d − 1)dimensional simplex. We list vectors a in a lexicographic order such that the first index
in a runs fastest, then the second changes and the last index runs slowest. Cells of a
corresponding contingency table are in vector b ∈ I.
Given a subset M ⊆ V of the variables, the marginal distribution of the variables Av ,
for v ∈ M has itself a joint Bernoulli distribution, in the same lexicographic order:
pM (aM ) = Pr(Av = av , for all v ∈ M ).
We use three well-studied parameterizations for joint Bernoulli distributions, that is the
log-linear, the linear and the multivariate logistic parameterizations and show how and
why they differ even for palindromic Bernoulli distributions.

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

4

In general, a parameterization of A is a smooth one-to-one transformation, mapping π
into a 2p × 1 vector θ = G(π), say, whose entries θb , are called interaction parameters. To
index interactions, it is useful to have a one-to-one mapping between the cells in b and
subsets of V = {1, . . . , p}. For p = 3:
Lexicographic order
cells in b :

000

100

010

110

001

101

011

111

subset of V :

∅

1

2

12

3

13

23

123

θb :

θ∅

θ1

θ2

θ12

θ3

θ13

θ23

θ123

P
The cardinality of the set b, denoted by |b| = v bv , gives the number of ones in vector b.
Depending on |b| being odd or even, an interaction parameter θb is said to be of odd or
even order. For instance, the even-order θ13 is a two-factor interaction of A1 and A3 .
2.2. Log-linear parameters
Log-linear parameters are contrasts of log probabilities, that is linear combinations of log
p(a), with weights adding to zero. The vector of the log-linear parameters is
λ = Hd−1 log π
where


Hd =
|



1

1

1

−1

(2.1)


⊗ ··· ⊗
{z



1

1

1

−1

.
}

d

is a 2d ×2d symmetric design matrix whose generic entry is hab = (−1)a·b for (a, b) ∈ I ×I.
Its inverse, the contrast matrix, is Hd−1 = 2−d Hd . The special form of Hd , chosen here, uses
so-called effect coding; see for instance Wermuth & Cox (1992). The individual interactions
can be written as
P
(2.2)
λb = 2−d a∈IV (−1)a·b log p(a),
where a · b = a1 b1 + · · · + ap bp is the inner product of the two binary vectors a and b; see
Haberman (1973, p. 619). In equation (2.2), the symbol b is interpreted for λb as a subset
of V and in the expression (−1)a·b as binary vector.
The inverse mappings from λ to π may be explicitly computed as
π = exp(Hd λ),

p(a) = exp

a·b
b∈I (−1) λb

P



.

(2.3)

Bernoulli distributions with positive cell probabilities belong to the so-called regular
exponential family with the vector λ containing the canonical parameters.
2.3. Linear interactions or moment parameters
In contrast to log-linear models, the linear-in-probability models, discussed for instance by
Cox & Wermuth (1992, app. 2), and their interactions are based on moments. The vector

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

5

ξ = Hd π is a moment parameter vector and the mapping between ξ and λ is one-to-one
and differentiable; see Barndorff-Nielsen (1978, p. 121).
The moment vector ξ is proportional to the expected value of the sufficient statistics
for λ. With y denoting the vector of the frequencies and by the symmetry of Hd of equation (2.1), this vector of sufficient statistics is Hd y. The elements of ξ, called also linear
interactions, are
ξb =

a·b p(a)
a∈I (−1)

(2.4)

p(a) = 2−d

(2.5)

P

and gives as inverse transformations
π = 2−d Hd ξ,

P

a·b
b∈I (−1) ξb .

For instance, with d = 2 the two-factor linear interaction is ξ11 = p00 − p01 − p10 + p11 .
For the transformed random variables Dv = (−1)Av , which take value 1 if Av = 0 and
−1 if Av = 1, the individual interactions are

Q
bv .
(2.6)
D
ξb = E
v
v∈V
Because the element (−1)a·b in equation (2.4) may be written with dv = (−1)av as
(−1)a·b = (−1)a1 b1 × · · · × (−1)ap bp =

Q

bv
v∈V (dv ) ,

equation (2.4) gives the expected value of this product with respect to p(a).
Equation (2.6) implies that each moment parameter, ξb , is a marginal parameter, defined
in the marginal distribution of the random vector (Av )v∈b , while the log-linear parameter
λb is defined in the joint distribution. Therefore there is, in general, no simple relation
between the log-linear parameter λM
b , say, in the marginal distribution pM (aM ) and λb ,
the log-linear parameter in the joint distribution, but there are exceptions, see Example 4.2
and Section 5. By contrast, the moment vector ξbM defined in pM (aM ) coincides with ξb .
According to an important result for regular exponential families by Barndorff-Nielsen
(1978, pp. 121–122), for an arbitrary partition of the parameter vectors λ and ξ in two
sub-vectors such that λ = (λA , λB ) and ξ = (ξA , ξB ), the distribution π is uniquely parameterized by the mixed vector (λA , ξB ) or by (ξA , λB ) and there is a diffeomorphism between
this mixed parameterization and the log-linear parameter λ or the moment parameter ξ.
2.4. Multivariate logistic parameters
The multivariate logistic parametrization, introduced by Glonek & McCullagh (1995), is
defined by the highest order log-linear parameters, considered here under effect coding,
in each possible marginal distribution of A. The parameters are given by the vector η =
(ηb )b⊆V where
ηb = λbb ,
b ⊆ V.
(2.7)
Kauermann (1997) showed that the mapping T : λ 7→ η from the log-linear to the
multivariate logistic parameters is a diffeomorphism by proving that T is a composition

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

6

of smooth transformations between the canonical, the moment and the mixed parameters.
More details are given below in subsection 2.5.
p
Let Λ = R2 −1 be the parameter space for the log-linear parameters λ. Then the parameter space E = T (Λ) for η is the image of λ under the transformation T . Explicit
forms for the inverse function T −1 : E → Λ are known for p = 1 or p = 2 and in special
cases, such as in Example 4.2. An algorithm provided by Qaqish & Ivanova (2006) detects
simultaneously whether the vector η is compatible with a proper probability vector π.
2.5. Properties of the palindromic Bernoulli distributions
We now study several properties of palindromic Bernoulli distributions and start by proving that these distributions are closed under marginalization.
Proposition 2.1. If p(a) is a palindromic Bernoulli distribution then, for any subset M
of the variables, the marginal distribution pM (aM ) is palindromic.
P
Proof. Define the partition a = (aN , aM ) and let pM (aM ) = aN ∈{0,1}|N | p(aN , aM ). Then
if the distribution is palindromic, p(aN , aM ) = p(∼ aN , ∼ aM ) and
pM (aM ) =

P

∼aN ∈{0,1}|N |

p(∼ aN , ∼ aM ) = pM (∼ aM ).

Next, we characterise the distribution by zero constraints on interactions.
Proposition 2.2. A Bernoulli distribution is palindromic if and only if, with θb = ξb or
θb = λb , all odd-order linear or log-linear interactions vanish, that is if and only if
θb = 0,

for all b ⊆ V with |b| odd.

Proof. 1) (If A is palindromic then all odd-order ξb = 0.) Any linear interaction can be
written as
P
P
ξb = a∈I1 (−1)a·b p(a) + a∈I1 (−1)(∼a)·b p(∼ a),
(2.8)
where I1 denotes the subset of cells having a one as first element. Thus I1 contains half of
the cells. If the distribution is palindromic, p(∼ a) = p(a) and (−1)(∼a)·b) = (−1)|b| (−1)a·b .
Thus,
P
P
ξb = a∈I1 (−1)a·b p(a) + (−1)|b| a∈I1 (−1)a·b p(a).
(2.9)
When |b| is odd then (−1)|b| = −1 and ξb = 0; see also Edwards (2000, App. C).
2) (If all odd-order ξb = 0, then p(∼ a) = p(a).) If all odd-order interactions vanish,
then
1P
(2.10)
p(a) = d b∈Ieven (−1)a·b ξb ,
2
where Ieven is the subset of the cells b such that |b| is even. Thus,
p(∼ a) =

1P
1P
(∼a)·b
(−1)
ξ
=
(−1)|b| (−1)a·b ξb = p(a),
b
b∈I
even
2p
2p b∈Ieven

because |b| is even. So the distribution is palindromic.

(2.11)

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

7

3) The same arguments apply for the log-linear parameterization. The distribution is
palindromic if and only if log p(a) = log p(∼ a) for all a. Therefore, using equation (2.2),
and the previous lines of reasoning, λb = 0 whenever |b| is odd. Conversely, if all odd-order
log-linear parameters λb vanish, then from
log p(a) =

P

a·b
b∈I (−1) λb

we get log p(a) = log p(∼ a) and the distribution is palindromic.
By equation (2.6) and Proposition 2.2, the joint distribution of A is palindromic if
and only if all the odd-order moments of D = (−1)A are zero. Also, as the palindromic
property is characterized by linear constraints on the canonical pararameters λ, we have
the following result.
Corollary 2.3. Palindromic Bernoulli distributions are a regular exponential family.
We show next a similar characterization for the multivariate logistic parametrization.
Proposition 2.4. A Bernoulli distribution is palindromic if and only if all odd-order
multivariate logistic parameters vanish, that is if and only if
ηb = 0,

for all b ⊆ V with |b| odd.

The analogous result for the larger class of complete hierarchical marginal log-linear
parameterizations, Bergsma & Rudas (2002), will be discussed elsewhere.
The proof uses a transformation T : λ 7→ η introduced by Kauermann (1997, p. 265).
The composition of smooth one-to-one transformations TM gives T , for each nonempty
subset M ⊆ V . The functions TM operate on parameter transformations between the
canonical and the moment parametrizations, as follows. If M = V ,
TM (λP(V )\V ) , λV ) = (ξP(V )\V ) , λV ).
If M ⊂ V, |M | =
6 1:
TM (. . . , ξP(V )\V ) , ξM , . . . ) = (. . . , ξP(M )\M ) , ηM , . . . )
and the remaining parameters, which are not listed, are left unchanged. Finally, if |M | = 1,
TM (ξM , . . . ) = (ηM , . . . ).
For instance, to clarify, to get η = T (λ) for three variables we define
T (λ) = T1 ◦ T2 ◦ T3 ◦ T12 ◦ T13 ◦ T23 ◦ T123 (η)
and Table 1 gives the details of the required transformations TM .
Proof. Let odd = {b ∈ {0, 1}|V | : |b| odd} be the subset of all odd-order interactions.
Then, below we show that ηodd = 0 if and only if λodd = 0.

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

8

Table 1
The sequence of transformations required to obtain the multivariate logistic parameter η from the
log-linear parameter λ.

Transformation
T123 (λ)
T23 (θ(1) )
T13 (θ(2) )
T12 (θ(3) )
T3 (θ(4) )
T2 (θ(5) )
T1 (θ(6) )

ξ1
ξ1
ξ1
ξ1
ξ1
ξ1
η1

ξ2
ξ2
ξ2
ξ2
ξ2
η2
η2

Parameters
ξ12 ξ3 ξ13
ξ12 ξ3 ξ13
ξ12 ξ3 η13
η12 ξ3 η13
η12 η3 η13
η12 η3 η13
η12 η3 η13

ξ23
η23
η23
η23
η23
η23
η23

η123
η123
η123
η123
η123
η123
η123

Intermediate result
θ(1)
θ(2)
θ(3)
θ(4)
θ(5)
θ(6)
η

From Proposition 2.2 we know that a binary distribution is palindromic if and only
if λodd = 0. If π is palindromic then all the marginal distributions pb (ab ), b ⊆ V are
palindromic and thus in each of them, such that |b| is odd, λbb = ηb = 0. Thus, ηodd = 0.
Let even = P(V ) \ odd. Then
T (λeven , λodd = 0) = (ηeven , ηodd = 0).

(2.12)

Conversely, if ηodd = 0, let ηeven be arbitrarily chosen such as (ηeven , ηodd ) = 0) ∈ E0 ⊂ E
(the parameter space of the ηs). As E0 is connected we can directly use equation (2.12)
and the smoothness of the inverse transformation T −1 to get
T −1 (ηeven , ηodd = 0) = (λeven , λodd = 0),
and thus the distribution π is palindromic.
Table 2 illustrates the different parameters with a 23 table.
Table 2
Illustration of the the different parameters with a 23 table; constant terms omitted.

cells b :
80 π :
subsets of V :
ξ:
λ:
η:

000
15
∅
−
−
−

100
9
1
0
0
0

010
1
2
0
0
0

110
15
12
1/2
log(5)/2
log(3)/2

001
15
3
0
0
0

101
1
13
−1/5
−1/5
− log(3)/2

011
9
23
1/5
1/5
log(3)/2

111
15
123
0
0
0

Next, we state a result connected with binary probability distributions generated by
a linear triangular system, as studied in Wermuth, Marchetti & Cox (2009). Their joint
probabilities may be defined by the recursive factorization
Q
Pr(A1 = a1 , . . . , Ad = ad ) = Pr(A1 = a1 ) ds=2 Pr(As = as | A1 = a1 , . . . , As−1 = as−1 )
with uniform margins. With βsj denoting linear regression coefficients,
Pr(As = as | A1 = a1 , . . . , As−1 = as−1 ) = 12 (1 +

Pd

as +aj ).
1=s−1 βsj (−1)

(2.13)

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

9

The conditional expected values of As given variables A[s−1] = (A1 , . . . , As−1 ) are linear
regressions with only main effects and no constant term. For these distributions, all the
even-order linear interactions are known functions of the marginal correlations. Here, we
prove in Appendix 1 the following result and get back to such systems later.
Proposition 2.5. If a binary probability distribution is generated with a linear triangular
system, then it is palindromic.
3. Independences and dependences
Palindromic Bernoulli distributions share some but not all of the properties of joint Gaussian distributions. We elaborate here on properties of independences, of undirected dependences, also called associations, and of directed dependences, also called effects. Conditional independence of variables A, B given variable O, say, is written as A⊥
⊥ B|O, while
the complement of it, called conditional dependence of A, B given O, is written as A t B|O;
see Wermuth & Sadeghi (2012).
Starting with properties of general Bernoulli distributions, several measures of dependence are equivalent with respect to independences and the sign of a dependence; see Xie,
Ma & Geng (2008, thm. 1). The same happens for Gaussian distributions. In addition, for
bivariate palindromic Bernoulli distributions, many measures of dependence are even in
one-to-one correspondence; see e.g. Wermuth & Marchetti (2014). For the three variable
table in Section 1, conditional measures of main interest for (A1 , A2 ) at level 0 of A3 are
(αδ)/(βγ), the odds-ratio,
{δ/(γ + δ)} − {β/(α + β)}, the chance difference for success,
{δ/(γ + δ)}/{β/(α + β)}, the relative chance for success and
1

ρ12|k=0 = (αδ − βγ){(α + β)(γ + δ)(α + γ)(β + δ)}− 2 , the conditional correlation.
Thus, the chance difference for success and the conditional correlation ρ12|k=0 are multiples
of the cross-product difference, αδ − βγ, and coincide if and only if the probabilities for
success are identical for A1 and A2 . Moreover, the odds-ratio is equal to the relative chance
if and only if A1 ⊥
⊥ A2 |A3 , that is if and only if αδ = βγ. A dependence is positive if ρ12|k=0
(or the chance difference) is positive or the odds-ratio (or the relative chance) is > 1.
It can be verified that ρ12|k=0 = ρ12|k=1 , and given the linear form of the conditional
expectation E(A1 , A2 | A3 = k) = ρ12|k it follows from Baba Shibata & Sibuya (2004,
thm. 1) that the constant conditional correlation ρ12|k equals the partial correlation
ρ12.3 = (ρ12 − ρ13 ρ23 )/c

2
2
where c = {(1 − r13
)(1 − r23
)}1/2 .

Special one-to-one relations among the mentioned different measure of dependence are
given in the following Proposition, proved in Appendix 1.
Proposition 3.1. In a trivariate palindromic distribution:
(i) A1 ⊥
⊥ A2 | A3 ⇐⇒ λ12 = 0 ⇐⇒ ρ12|3 = 0 ⇐⇒ ρ12.3 = 0,

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

10

(ii) A1 t A2 | A2 > 0 ⇐⇒ λ12 > 0 ⇐⇒ ρi12|3 > 0 ⇐⇒ ρ12.3 > 0,
(iii) A1 t A2 | A3 < 0 ⇐⇒ λ12 < 0 ⇐⇒ ρ12|3 < 0 ⇐⇒ ρ12.3 < 0.
However, even if the chance difference is identical at all level combinations of the remaining variables, the relative chance for success may vary widely. This shows for the
palindromic distribution in Example 3.1, where the relative chance at level combination
(0, 0) of A3 , A4 is more than 10 times higher than at (1, 1). This may become more extreme
with equality just in sign.


Example 3.1

levels of A1 , A2 , A3 , A4 :
9200 π :

0000
4095

1000 0100 1100 0010 1010 0110 1110
91
91
47
91
47
47
91

The explanation is the presence of a four-factor log-linear interaction in the 24 table.
3.1. Induced dependences and effect reversal
Next, we give three examples of binary palindromic tables, which illustrate what have been
called the weak and the strong versions of the Yule-Simpson paradox. In Examples 3.2 and
3.3, an independence gets destroyed by changing the conditioning sets and in Example 3.4
the sign of a dependence gets reversed after marginalizing. The three examples illustrate
in addition, that in all palindromic Bernoulli distribution not only conditional parameters
are relevant but also the marginal parameters and, in particular, also simple correlations,
due to the one-to-one relation between an odds-ratio and the correlation in their bivariate
distributions. In the following examples the symbol λ0 indicates the log-linear interactions
in effect-coding obtained from the counts. They are identical to previous λ parameters
except for the constant term, where λ∅ = λ0∅ − log n.


Example 3.2
levels ijk of A, B, O : 000
A⊥
⊥ B|O 
100 π : 32
&AtB
log-lin. interaction λ0 : 2.08



100 010 110 001 101 011 111
8
8
2
2
8
8 32 
0
0
0
0 0.69 0.69
0



100
60
0

010
40
0

110 001 101
10 10
40
−0.25
0 0.45

011 111
60 90 
0.65
0



100
50
0

010
40
0

110
10
−0.17

011 111
50 100 
0.63
0

Example 3.3
levels ijk of A, B, O : 000
A t B|O 
400 π : 90
& A⊥
⊥B
log-lin. interaction λ0 : 3.65
levels ijk of A, B, O : 000
Example 3.4

A t B|O pos.
400 π 100
& A t B neg.
log-lin. interaction λ0 : 3.63

001 101
10
40
0 0.52





To understand the examples the following matrices with marginal and partial correlations
are given for the three examples. The variables are (A, B, O) in this order. The matrices
show correlations, ρst , forp(s, t) = (1, 2), (1, 3), (2, 3) in the lower triangle and partial
correlations, ρst.v = −ρst / ρss ρtt , in the upper triangle; v denotes the remaining variable
and ρst is a concentration, that is an element in the inverse covariance matrix.
1

0

0.51

1

−0.18

0.35

1

−0.13

0.41

0.36

1

0.51

0

1

0.52

0.10

1

0.50

0.60

0.60

1

0.30

0.50

1

0.40

0.50

1

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

11

Note that in Example 3.1, we have ρ12 = ρ13 ρ23 and in Example 3.2, ρ12.3 = −ρ13.2 ρ23.1 .
Though these situations may be surprising when one sees them for the first time, they
have simple explanations. The strong version of the Simpson’s paradox in Example 3.4
results for pair (A, B), say, when there are substantial dependences A t B|O and A t O|B,
and a strong dependence B t O; see (Wermuth, 1987, sec. 6).
The weak versions are due to a dependence-inducing property. This property is shared
by joint Gaussian distributions; see e.g. Wermuth & Cox (1998, lem. 2.1) and is known
as singleton transitivity when it is used for graphs representing a large class of graphical
Markov models; see Wermuth (2015, eqs. (10), (11)). The property has been studied for
binary variables by Simpson (1951, sec. 11) and Birch (1963), discussion of Equation 5.1,
in the form
(A⊥
⊥ B | O and A⊥
⊥ B) implies (A⊥
⊥ O or B ⊥
⊥ O).
(3.1)
Equivalently, equation (3.1) is formulated as dependence-inducing with
(A t O and B t O) implies (A⊥
⊥ B|O or A⊥
⊥ B) but not both,
and it applies to triples of variables also when a common conditioning set is added to each
statement. For variables A, O, B this shows as in Fig. 2 in graphs which are Vs :

Fig 2. Pairs of Vs for binary A, B, O with non-vanishing dependences associated to each edge present;
left: A ⊥
⊥ B|O and A t B is represented by a source V and by its Markov-equivalent, concentration-graph
V; right: A ⊥
⊥ B and A t B|O is represented by a sink Vand by its Markov-equivalent, covariance-graph V.

In our examples for (A, B, O), the log-linear interaction vector λ tells for Example 3.2
that the concentration graph is a V since λ12 = 0; for Examples 3.3 and 3.4 that it is a
complete graph since all two-factor terms are nonzero. On the other hand, the correlation
matrices tell for Example 3.3 that the covariance graph is a V since ρ12 = 0.
The two weak versions of the Yule-Simpson paradox show that a distribution may have
more independences than those displayed in its graphical representation. This has also
been called its lack of faithfulness to the graph; see Spirtes Glymour & Scheines (2000).
3.2. Combination of independences
For three binary variables, the combination of independences was first studied by Birch
(1963, sec. 5): (A⊥
⊥ B|O and A⊥
⊥ O|B) =⇒ A⊥
⊥ BO, but (A⊥
⊥ B and A⊥
⊥ O) =⇒
A⊥
⊥ BO holds only when the three-factor interaction is lacking.
More generally, if independences combine both downwards and upwards in a distribution as explained below, then the complete independence, a⊥
⊥ bc, has several equivalent
decompositions. Let a, b, c be disjoint subsets of {1, 2, . . . , d} for the random variables
X1 , X2 , . . . Xd , where each of a, b, c contains at least one element, then
a⊥
⊥ bc ⇐⇒ (a⊥
⊥ b|c and b⊥
⊥ c) ⇐⇒ (a⊥
⊥ b and a⊥
⊥ c) ⇐⇒ (a⊥
⊥ b|c and a⊥
⊥ c|b). (3.2)

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

12

In all probability distribution, the first equivalence holds but the third and fourth statement are only implied by a⊥
⊥ bc. If the third statement implies a⊥
⊥ bc, the independences
combine upwards and if the fourth statement implies a⊥
⊥ bc, they combine downward.
Equation (3.2) is a known property of joint Gaussian distributions; see Lněnička &
Matúš (2007, def. 1), and Marchetti & Wermuth (2009, app. 2), and of traceable regressions;
see Wermuth (2012, cor. 1). In the context of graphical Markov models, the upward and
downward combination of independences are called the composition and the intersection
property, respectively. Both are also properties of all currently known probabilistic graphs;
see Sadeghi & Lauritzen (2014).
A general sufficient condition for the downward combination in discrete distributions
are strictly positive probabilities, which are assumed in this paper. In contrast to Gaussian distributions, in palindromic Bernoulli distributions independences need not combine
upwards. An extreme form of this is in Example 3.5, where the correlation matrix is the
identity matrix, even though the four variables are dependent.


Example 3.5

levels of A1 , A2 , A3 , A4 :
880 π :

0000
100

1000 0100 1100 0010 1010 0110 1110
10
10 100
10 100 100
10

Here, the independences do not combine upwards since, for instance, both A1 A2 ⊥
⊥ A3 and
A1 A2 ⊥
⊥ A4 are satisfied, but A1 A2 t A3 A4 holds instead of A1 A2 ⊥
⊥ A3 A4 . The reason is
the reciprocal behaviour of the conditional odds-ratios which implies that the only nonzero
log-linear interaction is the four-factor term and that the conditional correlations vary with
the levels of the third variable and can therefore not coincide with a partial correlation.
Similarly, the equality of conditional correlations may get destroyed with special covariance structures even when there is essentially no log-linear four-factor interaction. In
Example 3.6, this happens with a funnel graph which generalises the sink V to more than
two uncoupled nodes pointing to a common response; see Lupparelli, Marchetti & Bergsma
(2009) for estimation in such covariance graph models.


Example 3.6

levels of A1 , A2 , A3 , A4 :
888 π :

0000 1000 0100 1100 0010 1010 0110 1110
87
24 102
9
60
51
87
24

The importance of this type of palindromic structure is that it models studies using a 2k
factorial design with equal allocation of the study subjects to all level combinations and
special sampling so as to get equal chances of success and of failure for a binary response.
4. Some special cases
We discuss now palindromic distributions arising when a Gaussian distribution is median dichotomized. Further, some details concerning the specification and estimation of
undirected graphical models are given.
4.1. Median dichotomization
Let (X1 , X2 ) have a joint distribution function F12 with marginal distributions functions F1
and F2 . Let further U1 = F1 (X1 ) and U2 = F2 (X2 ) be the probability integral transforms

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

13

ej be the medians of Xj , j = 1, 2.
of X1 and X2 , so that U1 and U2 are uniform. Also let X
Consider now the median dichotomized variables,
A1 = I[U1 > 21 ],

A2 = I[U2 > 12 ]

(4.1)

where I[·] is the indicator function. Then, the joint distribution of A1 and A2 is a bivariate
palindromic Bernoulli distribution, as given in Section 1, with α = P (U1 > 12 , U2 > 12 ).
The variables D1 = (−1)A1 and D2 = (−1)A2 , taking values 1, −1, have mean zero
and unit variance, so that ξ12 = E(D1 D2 ), the correlation coefficient between D1 and D2 ,
becomes the cross-sum difference of the joint probabilities
ξ12 = 2α − 2β = 4α − 1.

(4.2)

Thus the correlation between two binary variables, which is a multiple of the cross-product
difference, coincides in a bivariate palindromic Bernoulli distribution with the cross-sum
difference. This was not noted, when ξ12 was proposed as a measure of dependence between
any two random variables X1 and X2 by Blomqvist (1950):


e1 )(X2 − X
e2 ) > 0 − Pr (X1 − X
e1 )(X2 − X
e2 ) < 0
4α − 1 = Pr (X1 − X
= 2Pr(U1 ≤ 21 , U2 ≤ 12 ) + 2Pr(U1 > 21 , U2 > 12 ) − 1.
Remark 4.1. The probability α may be interpreted as the copula C( 12 , 12 ) of the random
vector (X1 , X2 ), where the function C(u, v) = Pr(U1 ≤ u, U2 ≤ v)), 0 ≤ u ≤ 1, 0 ≤ v ≤ 1.
With the linear interaction expansion of equation (2.5), the distribution of D1 , D2 is
P (D1 = i, D2 = j) = 14 (1 + ξ12 ij),

i, j = 1, −1.

(4.3)

After median-dichotomizing d > 2 continuous variables, the resulting binary variables
Av , v = 1, . . . , d are still marginally uniform, but their joint distribution is palindromic
ev has the same distribution as
only for centrally symmetric variables, that is when Xv − X
ev ) for each Xv , v = 1, . . . d.
−(Xv − X
With d = 3, the joint distribution of the median-dichotomized variables is palindromic
with parameters α, β, γ and δ, as given in section 1. Their marginal correlations are
ξ12 = 4α + 4δ − 1,

ξ13 = 4α + 4γ − 1,

ξ23 = 4α + 4β − 1

and the joint probability distribution is, with i, j, k = 1, −1.
P (D1 = i, D2 = j, D3 = k) = 81 (1 + ξ12 ij + ξ13 ik + ξ23 jk).
Example 4.2. The following example gives the orthant probabilities of a trivariate, meancentred Gaussian distribution having equal correlations: −1/2 < ρ < 1. The joint probability vector of the median-dichotomized variables is:
8π = (1 + 3ξ,

1 − ξ,

1 − ξ,

1 − ξ,

1 − ξ,

1 − ξ,

1 − ξ,

1 + 3ξ)

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

14

and the explicit transformations between the three types of parameters result with
ξ=

2
arcsin ρ,
π

λ=

1
1 + 3ξ
log
,
4
1−ξ

η = atanh ξ.

(4.4)

The arcsin transformation is due to Sheppard (1898) and the obtained distribution is a
concentric ring model ; see Wermuth, Marchetti & Zwiernik (2014).
Proposition 4.3. If X has a d-variate Gaussian distribution with mean zero and correlation matrix R = [ρst ] for s, t = 1, . . . , d and A is the binary random vector obtained
by median dichotomizing X, with linear interaction parameters ξb , then R can be reconstructed from the correlation matrix RA = [ξst ] between the binary variables A by
ρst = sin{(π/2)ξst },

s, t = 1, . . . , d.

The proof results by inverting the arcsin transformation of the quadrant probability
ξst = 4 Pr(Xs ≤ 0, Xt ≤ 0) − 1 = 2π −1 arcsin ρst .
As a consequence one may reconstruct the original correlations from the palindromic
Bernoulli distribution derived via the orthant probabilities.
4.2. Maximum likelihood estimation
For a palindromic Bernoulli distribution, given a random sample of size n, one has as
counts, that is as observed cell frequencies: n(a), a ∈ I. The likelihood is
Q

a∈I p(a)

n(a)

=

Q

n(a) p(∼
a∈I0 p(a)

a)n(∼a) =

Q

n(a)+n(∼a)
a∈I0 p(a)

(4.5)

where I0 is the set of half of the cells a such that a1 = 0. The sufficient statistics are
thus the set of the 2d−1 frequencies n(a) + n(∼ a), obtained by summing each cell and
its complement image. The maximum likelihood estimate of a cell probability (or of a cell
count) is the average of the two proportions (or of counts):
pb(a) = {n(a) + n(∼ a)}/(2n),

n
b(a) = {n(a) + n(∼ a)}/2.

This produces a symmetrized vector of counts.
For palindromic Bernoulli distributions, Wilks’ likelihood ratio test statistic is


P
2 n(a)
w = 2 a∈I n(a) log
.
n(a) + n(∼ a)

(4.6)

(4.7)

It has an asymptotic χ2 distribution with 2d−1 degrees of freedom; see Edwards (2000,
app. C). The maximum likelihood estimates of the linear interaction parameters are

0
if |b| odd,
ξbb = P

(−1)a·b n(a)/n. if |b| even.
a∈I

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

15

Thus, the estimated ξbb , for |b| even, matches the observed moment statistic and for
|b| odd is zero. For |b| = 2, the estimated marginal correlation, ξb12 coincides with the
correlation coefficient in the fitted table pb(a), hence is a cross-sum difference of the counts
ξb12 = (n00 + n11 ) − (n01 + n10 ).

(4.8)

Since the log-linear and the multivariate logistic parameters are in a one-to-one relation
to the linear interactions, the maximum likelihood estimates of their parameter vectors,
result by the same transformations that hold for the parameters; see Fisher (1922). For
the special transformations that apply here, see equations (2.3), (2.5).
In the following, we speak of maximum likelihood estimates simply as ‘estimates’. Estimates may simplify further, when the distribution satisfies independence constraints in
such a way that they lead to a graphical Markov model; see, for an overview of these
models, e.g., Darroch, Lauritzen & Speed (1980), Haberman (1973), Wermuth (2015).
Example 4.4 (A Markov chain). Let A1 , A2 and A3 be three binary random variables
where A1 and A3 are conditionally independent given A2 , so that the probabilities satisfy
pijk = p−1
+j+ pij+ p+jk for i, j, k = ±1 .
2
3, has a missing edge for
Its undirected graph, called a concentration graph, 1
nodes 1 and 3, representing A1 and A2 , and it is a simplest type of a graphical Markov
model, a Markov chain in 3 variables; see also Example 3.1.
The log-linear parameters are constrained by λ13 = λ123 = 0 for the conditional independence of pair (1, 3). If, in addition, the distribution is palindromic, the odd-order
parameters are zero so that also λ1 = λ2 = λ3 = 0. In general Bernoulli distributions, the
minimal sufficient statistics are the observed counts corresponding to the cliques of the
graph, i.e., the maximal complete subsets of the nodes, here just the node pairs (1,2) and
(2,3). However for a palindromic Bernoulli distribution, the minimal sufficient statistics
are the estimated counts n
bij+ and n
b+jk for margins (1, 2) and (2, 3), defined as in equation
(4.6) from the symmetrized table, so that
peijk = n−2 n
bij+ n
b+jk for i, j, k = ±1.
where peijk is the estimate of pijk under the Markov chain model.
This example illustrates how independence constraints, conditionally given all remaining
variables, simply add to the linear constraints on canonical parameters of a palindromic
Bernoulli distribution. Moreover, when the model is decomposable, since its concentration
graph is chordal, see e.g. Darroch, Lauritzen & Speed (1980), it can be generated by a
linear triangular system; see end of Section 2.
As far as the maximum likelihood estimation of palindromic graphical models is concerned, the hierarchical constraints of conditional independence and the non-hierarchical
constraints of central symmetry are well compatible with one another. Thus one can fit
a given graphical model to the symmetrized counts or equivalently symmetrize the fitted
counts under the model.

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

16

5. Palindromic Ising models
We now introduce palindromic Ising models, especially when combined with conditional
independence constraints. Ising models are joint Bernoulli distributions without any higher
than two-factor log-linear interactions. An Ising model is palindromic if it has also uniform
margins. As mentioned before, this leads for the (−1, 1) coding to binary variables which
have zero means, unit variances and covariances coinciding with Pearson’s correlations.
General Ising models have, for instance, been studied as lattice systems, Besag (1974),
and as binary quadratic-exponential distributions, Cox & Wermuth (1994b).
The concentration graph of an Ising model in d variables has d nodes and at most
one undirected edge coupling a node pair. The edge (i, j) in this concentration graph is
missing if the two-factor log-linear interaction of pair (i, j) vanishes. Each missing edge
(i, j) means i⊥
⊥ j|{1, . . . , d} \ {i, j}. Simpler independence statements such as i⊥
⊥ j|C, for
C ⊂ {1, . . . , d} \ {i, j} result if every path between i and j has a node in C; see for instance
Darroch, Lauritzen & Speed (1980). Recall that nodes and edges of the cliques of the graph
form its maximal complete subgraphs, that is those node subsets without any missing edge
which become incomplete when one more node is added.
Proposition 5.1. If the concentration graph of a palindromic Bernoulli distribution has
largest clique size three, then it is a palindromic Ising model.
Proof. An unconstrained palindromic Bernoulli distribution has a complete concentration
graph in nodes {1, . . . , d}. Each removed edge introduces an independence and reduces
the size of a generating clique. If the largest clique-size is three, then each of the 23
generating probabilities for the joint palindromic distribution has no 3-factor log-linear
interaction.
Proposition 5.2. Palindromic Ising models are closed under marginalizing for d ≤ 4.
Proof. In an Ising model, all higher than 2-factor log-linear interactions are zero and
all trivariate and bivariate marginal distributions are Ising models because palindromic
distributions are closed under marginalizing.
For palindromic Ising models of more than four variables, cliques of even order may get
induced by marginalizing together with corresponding even-order log-linear interactions.
This happens, for instance, if the concentration graph is a star graph with edges is, for
i = 1, . . . , 4, and marginalizing is over the common source s = 5. Notice however that,
in any case, the bivariate and trivariate marginal probabilities remain palindromic Ising
models.
Proposition 5.3. For a decomposable palindromic Ising model with largest clique size
three, the maximum likelihood estimates are obtained in closed form from the marginal
correlations in the symmetrized 2 × 2 tables within its 2-node and 3-node cliques.
Proof. If a model is decomposable, then an ordering of the cliques Ct , t = 1, . . . , T can be
found, such that the joint probability factorizes and estimation simplifies; see Sundberg

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

17

(1975). This ordering satisfies the running intersection property meaning that the sets
S1 = C1 ∩ (∪t>1 Ct ), S2 = C2 ∩ (∪t>2 Ct ), . . . , ST −1 = CT −1 ∩ CT , called separators, are
all complete, that is all nodes are coupled by an edge. Then, the joint probability p(a)
factorizes into the product of the marginal distributions over cliques Ct divided by the
product of the the marginal distributions over the separators St .
As cliques and separators have largest size ≤ 3, and the associated marginal distributions
are palindromic Ising models, they are fitted in closed form directly from the marginal 2×2
tables of the symmetrized counts by using for instance the marginal correlations of such
tables, that is the cross-sum differences.
Example 5.1 below gives three non-decomposable palindromic Ising models having a
so-called chordless four-cycle. They have maximal clique size two and edges for (1, 3),
(1, 4), (2, 3), (2, 4). They differ in that all nonzero log-linear interactions are positive in
the first and negative in the second case. In the third one, there is a chordless cycle
in the concentration graph as well as in the covariance graph, that is not only the two
independences of the concentration graph hold, 1⊥
⊥ 2|34 and 3⊥
⊥ 4|12, but, in addition,
1⊥
⊥ 2 and 3⊥
⊥ 4.



Example 5.1 


levels of A1 , A2 , A3 , A4
336 π
336 π
336 π

:
:
:
:

0000 1000 0100 1100 0010 1010 0110 1110
75
15
15
3
15
15
15
15 

3
15
15
75
15
15
15
15 
35
35
7
7
7
35
7
35

In these chordless cycles, the missing edges in the concentration graph show also as zeros
p
in the matrix of partial correlations given all remaining variables that is in −ρij / ρii ρjj .
This points to the possible extension of a result by Loh & Wainwright (2013) to include
chordless cycles for palindromic Ising models.
Proposition 5.4. There is no effect reversal in a palindromic Ising model if all its nonzero
log-linear interactions are positive.
This result is a direct consequence of Proposition 3.4 (ii) in Fallat et al. (2016) for
totally positive palindromic Ising models. We conjecture that the same holds, when all
nonzero log-linear interactions in a palindromic Ising model are negative, such as in our
second case of Example 5.1.
6. A case study
The following case study illustrates some of the obtained results. For a sample of grades
obtained at the University of Florence, we aim at predicting grades in Physics in terms
of given grades in Algebra, Analysis and Geometry. The passing grades range in each
subject from 18 to 31. We use sums of grades over exams in three successive years and
have data for n = 78 students who reached in each of the subjects a sum of at least 60
points. Instructors expect positive correlations for each pair of these grades and no sign
reversal for the correlations at fixed level combinations of the other variables. The data
are in Appendix 2.

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

18

The four summed grades are closely bell-shaped, each of their scatter plots shows a
nearly elliptic form as well as the plots of residual pairs obtained after linear least squares
regression of each grade on the other three. There is also no evidence for nonlinear relations in the probability plots of Cox & Wermuth (1994a). Thus, there is substantive and
empirical support for assuming a joint Gaussian distribution.
After replacing for pairs (1,4) and (2,4) the observed correlations by rb14 = r13 r34 ,
rb24 = r23 r34 , in Table 3 we have the estimate of the correlation matrix, which has zeros for
pairs (1,4) and (2,4) in its inverse, in its concentration matrix; see e.g. Wermuth, Marchetti
& Cox (2009), equation (2.8).
Table 3
For four fields and 78 students, observed marginal correlations, rij (below the diagonal), concentrations
on the diagonal and partial correlations, rij.kl (above the diagonal).
1:=Analysis
2:=Algebra
3:=Geometry
4:=Physics

Analysis Algebra Geometry Physics
2.64
0.27
0.34
0.17
0.72
3.03
0.51
0.04
0.76
0.80
4.07
0.38
0.62
0.60
0.71
2.09

Wilks’ likelihood-ratio test statistic on 2 degrees of freedom shows with a value of
w = 2.8 a good fit to the model with generating sets {{1, 2, 3}, {3, 4}}. This implies
conditional independence of the grade in Physics from those in Analysis and Algebra
given the grade in Geometry. This follows directly, for instance, with the corresponding
concentration graph, on the left of Fig. 3. It has the cliques {1, 2, 3} and {3, 4} for which
node 3 separates node 4 from nodes 1,2 since to reach nodes 1,2 from node 4, one has to
pass via node 3.
Similarly, after replacing the marginal correlations for pairs (1, 2), (1, 3) and (2, 3) by
their average rb = 0.76, we have for the submatrix of (1,2,3) the conventional estimate of
an equicorrelation matrix; see Olkin and Pratt (1958, Section 3). This is here well-fitting
since w = 3.4 on 2 degrees of freedom. The grade in Physics, correlates with this sum score
as 0.706, even slightly less than with the grade in Geometry alone, where r34 = 0.709. This
is plausible in view of the well-fitting Markov structure.
1

2

1
3

4

L

2
3
Fig 3. Left: the well-fitting concentration graph for the Florence grades; right: a possible generating graph
for grades 1, 2, 3.

A possible generating graph for the Gaussian equicorrelation matrix is the star graph
displayed on the right of Fig. 3. In it mathematical ability is represented by the unobserved
inner node, L, and the three grades are the outer nodes of the graph, shown as responses
to L by arrows starting at L and pointing to the uncoupled nodes 1,2,3; each arrow has

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

19

assigned to it the same positive correlation ρ. After marginalizing over L, each outer pair is
correlated like ρ2 . We shall see next how well these results are reflected in the dichotomized
data.
After median-dichotomizing the grades with jittering, we generate precisely uniform binary variables, the marginal distributions of which differ only little from those obtained by
simple median-dichotomizing. One obtains the estimate of a palindromic contingency table
in closed form using equation (4.6) and as we shall see, the same well-fitting concentration
graph as on the left of Fig. 3.
The observed contingency table is given next, together with, in the additional rows
and in the following order, the estimates of palindromic counts, and of the counts after
imposing, in addition, conditional independences for the model of Fig. 3 on the left, and
the estimates of the corresponding log-linear interactions.
Table 4
Cells ijkl, levels of interactions, counts nijkl , estimates of palindromic counts, of the palindromic
concentration graph counts and of the log-linear interactions under the latter model

0000 1000 0100 1100 0010 1010 0110 1110 0001 1001 0101 1101 0011 1011 0110 1111
∅
1
2
12
3
13
23 123
4
14
24 124
34 134 234 1234
22
3
3
0
1
0
1
9
6
2
2
1
3
2
1
22
22
2 2.5 1.5
1
1 1.5 7.5 7.5 1.5 1.0 1.0 1.5 2.5 2.0 22.0
21.2 2.5 2.5 1.8 0.7 1.0 1.0 8.3 8.3 1.0 1.0 0.7 1.8 2.5 2.5 21.2
0.90
0
0 0.45
0 0.62 0.62
0
0
0
0
0 0.47
0
0
0

The palindromic concentration graph model fits well, with w = 10.3 on 11 degrees
of freedom. This decomposes into w = 9.1 on 8 degrees of freedom for the saturated
palindromic model and w = 1.2 on 3 degrees of freedom for the additional independence
constraints.
Values of the studentized log-linear interactions are 2.5, 3.5, 3.5 and 3.7 for λ12 , λ13 , λ23
and λ34 , respectively. Thus, the same independences as for the underlying joint Gaussian
distribution fit also the median-dichotomized data and further simplifications are not compatible given the sizes of the remaining studentized interactions. The partial correlations
implied by the well-fitting palindromic Markov structure has also zeros for pairs (1, 4).
The sum score of the median-dichotomized grades 1,2,3 leads as in the underlying
Gaussian distribution not to an improved prediction of grades in Physics. To our starting
question, we get two summarizing answers. Given a grade below the median in Geometry,
one predicts that 72% of these students will have a grade below the median in Physics
and, similarly, given a grade above the median in Geometry, one predicts that 72% will
have a grade above the median in Physics.
7. Discussion
We say that centrally-symmetric Bernoulli distributions are palindromic since their probabilities, at the fixed level of one of the variables repeat in reverse order for the second

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

20

level of this variable and thereby mimic palindromic sequences of characters as introduced
in linguistics.
A palindromic Bernoulli distribution is characterized by the vanishing of all odd-order
log-linear interactions. Hence, such zero constraints lead to a non-hierarchical, log-linear
model which give centrally symmetric probabilities. Until now, it was only known that
in centrally-symmetric Bernoulli distributions, all odd-order log-linear parameters vanish;
see Edwards (2000, app. C). With these linear constraints, distributions result which are
in the regular exponential family.
Palindromic Bernoulli distributions may also be parameterized with all odd-order interactions vanishing in a linear-in-probability model and in a multivariate-logistic model. The
parameters in the three types of model are in one-to-one relations; see Section 2. These
relations are now available in closed form for the linear and the log-linear formulations,
while in general, iterative procedures are needed when the multivariate logistic formulation
is involved. In any case, equivalent parameterizations assure that the maximum-likelihood
estimates of the parameters are in the same one-to-one relation; see Fisher (1922).
It is remarkable that a palindromic Bernoulli distribution can be expressed precisely as a
log-linear and as a linear model, since log-linear parameters use the notion of multiplicative
interactions and the linear-in-probability models are based instead on the notion of additive
interactions as discussed, for instance by Darroch & Speed (1983).
The log-linear parameterization shows that positive palindromic Bernoulli distributions
are in the regular exponential family with and without additional independence constraints
in its concentration graph. A palindromic Ising model may have only log-linear two-factor
interactions as non-vanishing canonical parameters, while in their linear-model formulations higher-order interactions may be present. The palindromic property is preserved
under marginalizing over any subset of the variables; see Proposition 2.1, even though one
may no longer have an Ising model after marginalizing over some of the variables.
Another property is important for applications. In palindromic Bernoulli distributions,
many other measures of dependence of a variable pair are one-to-one functions of the
odds-ratio; in particular the relative risk, used mainly in epidemiology, and the risk difference, employed almost exclusively in the literature on causal modelling. Only if a measure
of dependence is a function of the odds-ratio, it varies independently of its margins; see
Edwards (1963) and only then, measures of bivariate dependence become directly comparable under different sampling schemes, for instance when the overall count is fixed as in a
cross-sectional study or one of the margins is fixed as in a prospective study or the other
margin is fixed as in a retrospective study.
We expect that with a direct extension of the palindromic property to discrete variables
of more levels, similar attractive properties can be obtained as for the palindromic Bernoulli
distribution.

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

21

Appendix 1. Proofs
Proof of Proposition 2.5. The proof is by induction. We know that A1 has a palindromic
distribution. For s = 2, . . . , d we assume that the random vector A[s−1] = (A1 , . . . , As−1 )
has a palindromic distribution, and then we show that the distribution of A[s] = (A1 , . . . , As )
is palindromic. Let Ieven denote the subset of {0, 1}s with even order and split it in two
parts
I0 = {a ∈ Ieven : as = 0}, I1 = {a ∈ Ieven : as = 1}.
We then start from the identity
Pr(A[s] = a[s] ) = Pr(A[s−1] = a[s−1] )Pr(As = as | A[s−1] = a[s−1] )
and after substituting equations (2.5) and (2.13) and taking into account that by assumption A[s−1] has a palindromic distribution and thus ξb = 0 for all b ∈ I1 , we have
Pr(A[s] = a) = 2−s

a·b
b∈I0 ξb (−1)

P

· {1 +

Ps−1

a·es,j }
j=1 βsj (−1)

where es,j is a binary vector of dimension s with ones exactly in positions s and j. After
multiplying and collecting terms we get with
ξb =

P

v∈I0 : v4{s,j}=b ξv βs,j ,

Pr(A[s] = a) = 2−s

P

a·b +
b∈I0 ξb (−1)

for b ∈ I1 ,

(7.1)


a·b ,
ξ
(−1)
b∈I1 b

P

where 4 denotes the symmetric difference of sets. Therefore A[s] has a linear parameterization with exclusively even order interactions and hence is palindromic. Thus, by induction,
the distribution of A[d] = A is palindromic. From the recursive equation (7.1), each linear
interaction is a linear function of the regression parameters βs,j .
Proof of Prop. 3.1. It is known that in a strictly positive 23 table A1 ⊥
⊥ A2 | A3 ⇐⇒
(λ12 = 0 and λ123 = 0). As in a trivariate palindromic table the three-factor interaction
is always zero the single condition λ12 = 0 is necessary and sufficient. This is in turn
equivalent to a single condition on the partial correlation ρ12.3 = 0.
With no constraints other than those of palindromic distributions, there is a smooth oneto-one transformation (λ12 , λ13 , λ23 ) ←→ (ρ12 , ρ13 , ρ23 ) where the marginal correlations are
the free parameters of ξ.
In addition, there is a one-to-one smooth transformation between the simple correlations
and the partial correlations (ρ12 , ρ13 , ρ23 ) ←→ (ρ12.3 , ρ13.2 , ρ23.1 ), since the conditional and
partial correlations coincide. Also, the function g : R → [−1, 1] : λ12 7→ r12.3 is strictly
monotone increasing for any values of λ13 and λ23 and has a single zero in the origin, so
that the partial correlation r12.3 has the same sign as the log odds-ratio λ12 .
In more detail, the three variable table of Section 1, supplemented by both margins for
the two conditional tables of A1 , A2 can be written in terms of simple correlations as

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

A3 :
0
A1 A2 :
0
0
α
1
β
sum (1 + ρ23 )/4

0
1
γ
δ
(1 − ρ23 )/4

1
sum
0
(1 + ρ13 )/4
δ
(1 − ρ13 )/4
γ
1/2
(1 − ρ23 )/4

1
1
β
α
(1 + ρ23 )/4

22

sum
(1 − ρ13 )/4
(1 + ρ13 )/4
1/2

The probabilities in each of the 22 tables, expressed with margins and ρ12|3 give e.g.
8α = (ρ12 − ρ13 ρ23 ) + {(1 + ρ13 )(1 + ρ23 )},

8δ = (ρ12 − ρ13 ρ23 ) + {(1 − ρ13 )(1 − ρ23 )},

since the product of all four margins is in both tables (1 − ρ213 )(1 − ρ223 )/162 and
(ρ12 − ρ13 ρ23 )/16 = { 12 − (β + γ + δ)}δ − βγ = αδ − βγ .
Thus, ρ12|3 = ρ12.3 and αδ − βγ = 0 if and only if λ12 = 0. The above 23 table shows also
directly that under the independence constraint 1⊥
⊥ 2|3, we have
α
e = {1 + ρ13 )(1 + ρ23 )}/8,

δe = {(1 − ρ13 )(1 − ρ23 )}/8

βe = {1 − ρ13 )(1 + ρ23 )}/8,

γ
e = {1 + ρ13 )(1 − ρ23 )}/8

e induced in the marginal table of A1 , A2 for (1, 1),
so that e.g. α
eδe and the probability α
e + δ,
are
α
eδe = (1 − ρ213 )(1 − ρ223 )/82 ,
α
e + δe = (1 + ρ13 ρ23 )/4.
Appendix 2. The data for the case study
The columns of the following table contain sums of grades of three exams in four subjects
for n = 78 mathematics students at the University of Florence.
Table 5
Summed grades over 3 exams in the order: Analysis, Algebra, Geometry and Physics.

78
76
82
85

78
75
81
77

74
71
74
80

80
77
71
80

88
81
85
79

77
79
74
80

79
77
81
75

85
90
83
82

82
79
73
71

82
72
73
71

74
62
71
72

89
90
86
87

85
75
84
82

77
83
84
69

93
92
93
90

85
88
82
75

79
80
78
75

85
88
90
82

74
68
70
70

69
80
79
78

78
75
71
72

88
88
89
77

67
70
68
69

92
89
91
93

85
88
91
87

69
75
62
68

79
78
71
79

92
92
92
90

76
78
84
86

88
87
88
78

73
68
64
69

91
85
83
75

76
78
82
82

71
79
69
71

65
68
71
63

74
76
75
72

80
89
80
78

71
74
71
74

78
81
85
81

77
74
69
67

70
68
67
66

83
89
88
72

89
81
83
82

72
76
75
75

82
81
83
79

77
74
82
76

91
92
93
92

92
92
92
87

75
69
72
75

90
79
90
79

90
82
89
78

93
93
93
89

92
93
93
89

87
83
87
77

81
69
74
81

82
70
67
79

76
75
80
84

86
71
69
72

92
80
87
80

87
70
77
81

79
70
69
70

91
77
92
79

88
88
83
77

90
92
91
72

90
85
82
88

92
92
91
81

89
84
86
86

83
83
83
81

77
82
80
78

69
74
83
76

89
83
83
77

92
92
90
79

86
74
78
73

76
71
71
69

68
62
65
69

79
68
74
72

76
66
83
80

88
89
91
85

Acknowledgement
We thank G. Bulgarelli, Univ. of Florence, for the data; G. Ottaviani and the referees for
their helpful comments. We used Matlab R , 2015, and for exact median-dichotomizing,
‘jitter.m’ by R. Cotton, who adapted the R version by W. Stahel and M. Maechler.

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

23

References
Besag, J. (1974). Spatial interaction and the statistical analysis of lattice systems. J. R.
Stat. Soc. Ser. B. Stat. Methodol., 36, 192–236.
Baba, K., Shibata, R. & Sibuya M. (2004). Partial correlation and conditional correlation as measures of conditional independence, Aust. N. Z. J. Stat., 46(4), 657–664.
Barndorff-Nielsen, O. E. (1978). Information and Exponential Families in Statistical
Theory. New York: Wiley and Sons.
Bergsma, W. & Rudas, T.(2002). Marginal models for categorical data. Ann. Statist.,
30(1). 140–159.
Bhapkar, V. P. & Darroch, J. N. (1990). Marginal symmetry and quasi symmetry of
general order. J. Multivariate Anal., 34, 173–184.
Birch, M. W. (1963). Maximum likelihood in three-way contingency tables. J. R. Stat.
Soc. Ser. B. Stat. Methodol., 25, 220–233.
Blomqvist, N. (1950). On a measure of dependence between two random variables. Ann.
Math. Statist. 21, 593–600.
Cox, D. R. & Wermuth, N. (1992). Response models for mixed binary and quantitative
variables. Biometrika 79, 441–461.
Cox, D. R. & Wermuth, N. (1994). Tests of linearity, multivariate normality and
adequacy of linear scores. J. R. Stat. Soc. Ser. C. Appl. Stat., 43, 347–355.
Cox, D. R. & Wermuth, N. (1994). A note on the quadratic exponential binary distribution. Biometrika, 81, 403–408.
Darroch, J. N., Lauritzen S. L. & Speed, T. P (1980). Markov fields and log-linear
interaction models for contingency tables. Ann. Statist., 8(3), 522-539.
Darroch, J. N., & Speed, T. P (1983). Additive and multiplicative models and interactions. Ann. Statist. (11), 724–738.
Edwards, A. W. F. (1963). The measure of association in a 2 × 2 table. J. Roy. Statist.
Soc. Ser. A, 126, 109–114.
Edwards, D. (2000). Introduction to graphical modelling. New York: Springer.
Fallat, S., Lauritzen, S., Sadeghi K., Uhler, C., Wermuth, N. & Zwiernik
P. (2016). Total positivity in Markov structures. To appear in Ann. Statist., arXiv:
1510.01290v2, 1–35.
Fisher, R. A. (1922). On the mathematical foundations of theoretical statistics. Philos.
Trans. Roy. Soc. London Ser. A, 222, 309–368.
Glonek, G. F. V. & McCullagh, P. (1995). Multivariate Logistic Models. J. R. Stat.
Soc. Ser. B. Stat. Methodol., 57, 533–546.
Haberman, S. J. (1973). Log-linear models for frequency data: sufficient statistics and
likelihood equations. Ann. Statist., 1, 617–632.
Lněnička, R. & Matúš, F. (2007). On Gaussian conditional independence structures
Kybernetika, 43(3), 327–342.
Loh, P. & Wainwright, M. J. (2013). Structure estimation for discrete graphical
models: Generalized covariance matrices and their inverses. Ann. Statist., 41, 3022–

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

24

3049.
Lupparelli M., Marchetti, G. M. & Bergsma, W. P. (2009). Parameterization and
fitting of discrete bi-directed graph models. Scand. J. Stat., 36, 559-–576.
Kauermann, G. (1997). A note on multivariate logistic models for contingency tables.
Aust. J. Stat., 39, 261–276.
Marchetti, G. M. & Wermuth, N. (2009) Matrix representations and independencies
in directed acyclic graphs. Ann. Statist., 37(2), 961–978.
Olkin, I. and Pratt, J.W. (1958). Unbiased estimation of certain correlation coefficients. Ann. Math. Statist., 29, 201–211.
Qaqish, B. F. & Ivanova, A. (2006). Multivariate logistic models. Biometrika, 93,
1011–1017.
Sadeghi, K. & Lauritzen, S.L. (2014) Markov properties for mixed graphs. Bernoulli,
20(2), 676–696.
San Martin, E. & Mouchart, M. & Rolin, J. (2005). Ignorable common information,
null sets and Basu’s first theorem. Sankhyā, 67, 674–698.
Serfling, R. J. (2006). Multivariate symmetry and asymmetry. In: Kotz, S., Balakrishnan, N, Read,C. B. & Vidakovic, B.(eds.) Encyc. Statist. Sciences, 2nd ed. 8, 5538–5345.
Sheppard, W. F. (1898). On the geometrical treatment of the ‘normal curve’ of statistics,
with special reference to correlation and to the theory of error. Proc. Roy. Soc. London,
62, 171–173.
Simpson, E. H. (1951). The Interpretation of interaction in contingency tables. J. R.
Stat. Soc. Ser. B. Stat. Methodol. 13(2), 238–241.
Spirtes, P., Glymour, C. N. & Scheines, R. (2000). Causation, prediction, and search
MIT press, Cambridge, Massachusetts.
Streitberg, B. (1990). Lancaster interactions revisited. Ann. Statist. 18, 1878–1885.
Sundberg, R. (1975). Some results about decomposable (or Markov-type) models for
multidimensional contingency tables: distribution of marginals and partitioning of tests.
Scand. J. Stat., 2, 71–79.
Wermuth, N. (1987). Parametric collapsibility and the lack of moderating effects in
contingency tables with a dichotomous response variable. J. R. Stat. Soc. Ser. B. Stat.
Methodol, 49, 353–364.
Wermuth, N. (1989). Moderating effects in multivariate normal distributions. Methodika,
III, 74–93.
Wermuth, N. (2012). Traceable regressions. Int. Statist. Review, 80, 415–438.
Wermuth, N. (2015). Graphical Markov models, unifying results and their interpretation.
In: (Balakrishnan, N. et al., eds.) Wiley Statsref: Statistics Reference Online; also on
arXiv:1505.02456.
Wermuth, N. & Cox, D. R. (1992). On the relation between interactions obtained with
alternative codings of discrete variables. Methodika VI, 76–85.
Wermuth, N. & Cox, D.R. (1998). Statistical dependence and independence In: P. Armitage, P. and Colton, T. (eds.), Encyclopedia of Biostatistics, New York: Wiley,

G. M. Marchetti and N. Wermuth/Palindromic Bernoulli distributions

25

4260–4264.
Wermuth, N. & Marchetti, G. M. (2014). Star graphs induce tetrad correlation for
Gaussian as well as for binary distributions. Electron. J. Stat. 8, 253–273.
Wermuth, N., Marchetti, G. M. & Cox, D. R. (2009). Triangular systems for
symmetric binary variables. Electron. J. Stat., 3, 932–955.
Wermuth, N., Marchetti, G. M. & Zwiernik, P. (2014). Binary distributions of
concentric rings. J. Multivariate Anal., 130, 252–260; also on arXiv:1311.5655.
Wermuth, N. & Sadeghi, K. (2012). Sequences of regressions and their independences.
Invited paper with discussion TEST, 21, 215–252; also on arXiv:1103.2523.
Xie, X., Ma, Z. & Geng, Z. (2008). Some association measure and their collapsibility.
Statist. Sinica 18, 1165–1183.

