Bayesian hypothesis tests with diffuse priors:
Can we have our cake and eat it too?1
by John T. Ormerod(1,2) , Michael Stewart(1)
Weichang Yu(1) and Sarah E. Romanes(1)
(1)

School of Mathematics and Statistics, University of Sydney, Sydney 2006, Australia
(2)

ARC Centre of Excellence for Mathematical & Statistical Frontiers

arXiv:1710.09146v1 [] 25 Oct 2017

25th of October 2017
Abstract
We introduce a new class of priors for Bayesian hypothesis testing, which we
name “cake priors”. These priors circumvent Bartlett’s paradox (also called the
Jeffreys-Lindley paradox); the problem associated with the use of diffuse priors
leading to nonsensical statistical inferences. Cake priors allow the use of diffuse
priors (having one’s cake) while achieving theoretically justified inferences (eating
it too). We demonstrate this methodology for Bayesian hypotheses tests for scenarios under which the one and two sample t-tests, and linear models are typically
derived. The resulting Bayesian test statistic takes the form of a penalized likelihood ratio test statistic. By considering the sampling distribution under the null
and alternative hypotheses we show for independent identically distributed regular
parametric models that Bayesian hypothesis tests using cake priors are Chernoffconsistent, i.e., achieve zero type I and II errors asymptotically. Lindley’s paradox
is also discussed. We argue that a true Lindley’s paradox will only occur with small
probability for large sample sizes.

Keywords: Jeffreys-Lindley-Bartlett paradoxes; improper priors; likelihood ratio tests;
Chernoff-consistent; cake priors; linear models; asymptotic properties of hypothesis tests.

1

The authors gratefully acknowledge funding from the Australian Research Council – DECRA Fellowship DE130101670

1

1

Introduction

Determining appropriate parameter prior distributions is of paramount importance in
Bayesian hypothesis testing. Bayesian hypothesis testing often centres around the concept of a Bayes factor, which was initially developed by Jeffreys (1935, 1961), and later
popularized by Kass and Raftery (1995). The Bayes factor is simply the odds of the
marginal likelihoods between two hypotheses and is analogous to the likelihood ratio
statistic in classical statistics, where instead of maximizing the likelihoods with respect
to the model parameters, the model parameters are marginalized out. In classical statistical theory testing a simple point null hypothesis against a composite alternative is
routine. However, such hypothesis tests can pose severe difficulties in the Bayesian inferential paradigm where the Bayes factors may exhibit undesirable properties unless
parameter prior distributions are chosen with exquisite care. This paper offers a solution
to this difficulty.
Prior distributions can be chosen in an informative or uninformative fashion. Employing informative priors (either based on data from previously conducted experiments,
or eliciting priors from subject matter experts) can be impractical, particularly when
the number of parameters in the model is large. Furthermore, informative priors can be
criticised on grounds that such priors are inherently subjective or may not let the data
from the current experiment speak for itself. However, using alternative priors can also
lead to problems.
One such problem occurs when using overly diffuse or flat improper priors. In the
former case, as priors become more diffuse the hypothesis corresponding to the smaller
model becomes increasingly favoured regardless of the evidence provided by the data.
This problem occurs due to the normalizing constants of the priors dominating the expression for the Bayes factor, and is sometimes referred to as Bartlett’s paradox (e.g,
Liang et al., 2008) or the Jeffreys-Lindley paradox (e.g, Robert, 1993, 2014), named after
the pioneering work of Jeffreys (1935, 1961), Lindley (1957), and Bartlett (1957), whose
authors identified this and other related problems associated with Bayes factors. Discussions of this paradox and the related Lindley’s paradox can be found in Aitkin (1991),
Bernardo (1999), Sprenger (2013), Spanos (2013), and Robert (2014).
An extension of Bartlett’s paradox occurs in the limit where diffuse priors become
2

flat to the point of being improper. The use of flat improper priors gives rise to arbitrary
constants in the numerator and denominator of the Bayes factor (see DeGroot, 1973).
Such arbitrary constants are problematic since, without suitable modification, they could
be chosen by the analyst to suit any preconceived conclusions preferred, and as such,
are not suitable for scientific purposes. Techniques for selecting the arbitrary constants
in Bayes factors in an acceptable way when employing flat improper priors have been
developed in several papers. Bernardo (1980) proposes to derive a reference prior for the
null hypothesis by maximizing a measure of missing information. Spiegelhalter and Smith
(1982), and Pettit (1992) use an imaginary data device leading to the arbitrary constants
cancelling with other terms in the Bayes factor. A further approach to the problem of
using diffuse priors was proposed by Robert (1993) who advocated for reweighing the
prior odds to balance against parameter priors as prior hyperparameters become diffuse.
O’Hagan (1995) considers the problem of using flat improper priors in the calculation
of the Bayes factor by splitting the data into a training and testing set. The training set
is used to construct an informative prior to be used to calculate the Bayes factor using
the remaining portion of the data. These ideas have been refined in O’Hagan (1997),
Berger and Pericchi (1996), and Berger and Pericchi (2001). A computational drawback
of some of these approaches is that the same model is fit multiple times. For models
where Bayesian inferential procedures are considered too slow for fitting a single model
these approaches to Bayesian testing become infeasible from a practical viewpoint.
Other Bayesian hypothesis testing approaches abandon the Bayes factor altogether by
constructing hypothesis testing criterion which only enter the criterion through parameter
posterior distributions themselves. These include information criteria type approaches
such as the Bayesian information criterion (BIC) and the deviance information criterion
(DIC). The BIC or Schwarz’s criterion uses a Laplace approximation where the prior term
is assumed to be asymptotically negligible as the sample size grows (Schwarz, 1978). The
DIC involves a linear combination of the log-likelihood evaluated at a suitably chosen
Bayesian point estimator and the posterior expectation of the log-likelihood (Spiegelhalter
et al., 2002). Under such a construction the DIC is not dominated by the prior as prior
hyperparameters diverge. Similarly, posterior Bayes factors proposed by Aitkin (1991)
are based on the posterior expectation of the likelihood function, rather than the joint
likelihood (comprising of the model likelihood and prior). Since this only involves the
3

prior in the calculation of the posterior distribution, the prior does not dominate posterior
Bayes factors. Berger and Pericchi (1996) criticized this approach because it employs a
double use of the data that is not consistent with typical Bayesian logic.
An interesting alternative approach to Bayesian hypothesis testing is that suggested
in Section 6.3 of Gelman et al. (2013) who discuss examining the posterior distribution
of carefully chosen test statistics such that large values of a given test statistic provides
evidence against the null hypotheses. This idea is explored more formally in Gelman
et al. (1996) and give rise to the concept of posterior predictive p-values, the probability
that a test statistic of posterior predictive values is greater than the observed value of
the test statistic.
Bayes factors in the context of linear model selection (Zellner and Siow, 1980; Mitchell
and Beauchamp, 1988; George and McCulloch, 1993; Fernández et al., 2001; Liang et al.,
2008; Maruyama and George, 2011; Bayarri et al., 2012) and generalized linear model
selection (Chen and Ibrahim, 2003; Hansen and Yu, 2001; Wang and George, 2007; Chen
et al., 2008; Gupta and Ibrahim, 2009; Bové and Held, 2011; Hanson et al., 2014; Li and
Clyde, 2015) have received an enormous amount of attention. While we defer discussion
of the types of priors used in these contexts to Section 4.3 we will draw special attention
to Liang et al. (2008). Liang et al. (2008) considers several prior structures in the context
of linear models. They employ Zellner’s g-prior (Zellner and Siow, 1980; Zellner, 1986)
for the regression coefficients where g is a prior hyperparameter. They consider several
choices for choosing g including setting g to various constants, selecting g using a local
and global empirical Bayes procedure, and via placing a hyperprior on g. Their results
suggest that in order for the resulting Bayes factors to be well behaved (including model
selection consistent) a hyperprior needs to be placed on g.
In this paper we will construct a new class of priors which was inspired by the priors used in the context of linear and generalized linear models. This class of priors is
constructed in such a way as to mimic Jeffreys priors (which have the desirable property
that they are invariant under parameter transformations Jeffreys, 1946) in the limit as a
prior hyperparameter g diverges. In order to circumvent a Bartlett’s like paradox from
occurring, the rate at which g diverges is different in the null and alternative hypothesis
in such a way that results in the cancellation of problematic terms in both the numerator
and denominator of the Bayes factor.
4

Bayes factors using cake priors have several desirable properties. In the examples we
consider the Bayes factor can be expressed as a difference in BIC values, i.e., a penalized
version of the likelihood ratio test (LRT) statistic. Using properties of the LRT statistic
we show that Bayesian hypothesis tests are Chernoff-consistent in the sense of (Shao,
2003, Section 2.13), i.e., they achieve asymptotically zero type I and type II errors as
the sample size diverges. In contrast classical hypothesis testing procedures are usually
chosen to have a fixed type I error and are consequently not Chernoff-consistent. In this
respect our Bayesian hypothesis tests are superior to classical procedures whose type I
error is held fixed. Due to the above properties we call the priors we develop “cake priors”
since they allow the use of diffuse priors (having ones cake) while being able to perform
sensible statistical inferences (eating it too). We will also discuss Lindley’s paradox in
the context of cake priors and argue that generally Lindley’s paradox will only occur with
vanishingly small probability for large samples.
In Section 2 we reintroduce Bayes factors, including the interpretation of Bayes factors.
In Section 3 we discuss more specifically the problems associated with Bayes factors,
including both Lindley’s and Bartlett’s paradoxes. In Section 4 we describe cake priors
and illustrate their use in the context of one sample tests for equal means (with unknown
variance), two sample tests for equal means (assuming unequal variances), linear models,
and one sample tests for equal means (with known variance). In Section 5 we derive some
asymptotic theory for our proposed of Bayesian hypothesis tests. In Section 6 discuss the
relationship between cake priors and improper priors and discuss how arbitrary constants
can be introduced into the Bayes factor. In Section 7 we take a closer look at the
interpretation of Bayes factors in light of our findings. In Section 8 we conclude.

2

Bayes factors

Bayes factors are a key concept in Bayesian hypothesis testing introduced by Jeffreys
(1935, 1961), although a similar concept was also developed independently by Good
(1952). Suppose that we have observed the data vector x = (x1 , . . . , xn )T which are
observed samples from P = { pi ( · ) : i = 1, . . . , n } and we have two hypotheses H0 and
H1 representing two models Pj = { pij ( · |θ j , Hj ) : i = 1, . . . , n }, j = 0, 1, describing two

5

potential distributions from which x was drawn, i.e.,
H0 : P ∈ P0

versus

H1 : P ∈ P1 .

(1)

The models could potentially have distinct parameters from two distinct models and the
models need not be nested. Let p(θ j |Hj ) be the prior distribution under hypothesis Hj
for j = 0, 1. The Bayes factor is then defined as
R
p(x|θ 0 , H0 )p(θ 0 |H0 )dθ 0
p(x|H0 )
=R
,
BF01 =
p(x|H1 )
p(x|θ 1 , H1 )p(θ 1 |H1 )dθ 1
where integrals are replaced with combinatorial sums for discrete random variables. The
posterior odds of H0 to H1 is defined by PO01 = BF01 × p(H0 )/p(H1 ), where the factor
p(H0 )/p(H1 ) is the prior odds. Assuming p(H0 ) = p(H1 ) = 1/2, the Bayes factors have
the interpretation that when BF01 is above 1 the hypothesis H0 is favoured and when
BF01 is below 1 the hypothesis H1 is favoured. However, if the prior odds is not equal to
one then the posterior odds should be the focus for inference.
A Bayesian hypothesis test function T (x) ∈ {0, 1} is based on T (x) = I(λBayes > 0)
where λBayes = −2 ln BF01 (which we will call the Bayesian test statistic, analogous to
the LRT statistic), and

 1 implies H is preferred;
1
T (x) =
 0 implies H0 is preferred.

and

As indicated in the above equation, the interpretation of results based on Bayesian
hypothesis tests is different from the interpretation of frequentist tests. Frequentist hypothesis testing which asks whether the data could have plausibly been drawn from the
null model (based on a chosen test statistic), without reference to an alternative model
(although LRT statistics are, for example, derived with reference to a specific alternative
model). Furthermore, in the Bayesian paradigm preference towards a particular hypothesis is stated, rather than rejection of the null. Note that preference should not be confused
with endorsement. One can have a preference between two poorly fitting models without
stating that either model fits the data well.
Kass and Raftery (1995) offer an interpretation of λBayes , and BF10 = 1/BF01 in Table
1 in terms of strength of evidence against the null hypothesis. In Section 7 we will take
a closer look at the interpretation of Bayes factors in light of the analysis in the current
6

paper. For the examples we consider, using the cake priors described later, the quantity
b0 ) − `1 (θ
b1 )] (the LRT
λBayes will turn out to be a penalized version of λLRT = −2[`0 (θ
bj ’s are the MLEs
statistic for the hypotheses in (1) where `j (θ j ) = ln p(x|θ j , Hj ) and the θ
under Hj ) given by
λBayes = λLRT − ν ln(n)

λBayes = λLRT − ν ln(n) + O(n−1 )

or

(2)

depending on the example, where ν is the difference in the number of parameters in H0
and H1 . Intuitively one might expect λBayes and λLRT to be related since the focus of
both approaches are based on the ratio of likelihoods, albeit different likelihoods.
λBayes

BF10

Strength of evidence

0 to 2

1 to 3

not worth more than a bare mention

2 to 6

3 to 20

positive

6 to 10 20 to 150
> 10

strong

> 150

very strong

Table 1: Table of interpretation of Bayes factors offered by Kass and Raftery (1995).

3

Paradoxes in Bayesian hypothesis testing

Problems with Bayesian hypothesis testing based on Bayes factors, for particular combinations of hypotheses and priors, have been identified as early as 1935 by Jeffreys (1935),
and later by Lindley (1957), and Bartlett (1957). As we will see for particular hypothesis
tests, when parameter priors are not chosen with care, the conclusions based on Bayes
factors will not be sensible. To give some context for the ensuing discussion we will
now consider the hypothesis testing problem introduced by Lindley (1957) in order to
illustrate potential problems.
Lindley’s example: Consider the hypothesis test where the sample is modelled via
xi |µ ∼ N (µ, σ 2 ), 1 ≤ i ≤ n independently, where µ and σ 2 are the mean and variance
parameters respectively. Here µ is an unknown value to be estimated and σ 2 is a fixed
known constant. Suppose that we wish to perform the hypothesis test
H0 : µ = µ0

versus
7

H1 : µ 6= µ0 ,

(3)

where µ0 is a known constant. Under H0 the values of all model parameters are fixed (so
that under H0 the model has zero unknown parameters), i.e., H0 is a simple point null
hypothesis. Suppose that for H1 we employ the prior µ|H1 ∼ N (µ0 , τ 2 ) where the prior
variance τ 2 is a known constant. The Bayes factor with the stated prior on µ is
r


nτ 2
nz(x)2
,
(4)
BF01 = 1 + 2 exp −
σ
2(n + σ 2 /τ 2 )
√
where z(x) = n(x − µ0 )/σ is the standard z-test statistic (see Bernardo, 1999). The
p-value for this test is P(χ21 > z(x)2 ).

If we were to choose µ|H1 as above then Lindley (1957) identified the following problem.
• Problem I: For any fixed p-value as n → ∞ we have BF01 → ∞.
Suppose that the observed value of z(x) is large so that, for any reasonably chosen level
α, the typical frequentist approach would reject the null hypothesis. For this value of z(x)
a Bayesian procedure based on the above Bayes factor would prefer the null hypothesis
for a sufficiently large n, drawing a contradiction between the two inferential paradigms.
Problem I is associated with Lindley’s paradox, also referred to as the Lindley-Bartlett
and the Jeffreys-Lindey paradox). For discussion of this see Smith and Spiegelhalter
(1980); Aitkin (1991); Bernardo (1999); Sprenger (2013); Robert (2014).
We now consider a second example posed by Sprenger (2013).
Sprenger’s example: Jahn et al. (1987) used electronic and quantum-mechanical random event generators with visual feedback; the subject with alleged psychokinetic ability
tries to “influence” the generator. The number of “successes” was s = 52, 263, 470 and
the number of trials was n = 104, 490, 000. Assuming independence of each trial we have
xi |ρ ∼ Bernoulli(ρ), 1 ≤ i ≤ n with p ∈ [0, 1]. If we test
H0 : ρ = 0.5

versus

H1 : ρ 6= 0.5,

(5)

a rejection of H0 leads to evidence that the subject has psychokinetic ability. Using the
data a classical hypothesis testing approach leads to a p-value approximately equal to
0.0003, leading to a rejection of the null hypothesis for the α = 0.05 cut-off. A 95%

8

confidence interval for ρ is (0.50008, 0.50027). A standard Bayesian hypothesis test uses
the prior ρ ∼ Beta(1/2, 1/2) (the Jeffreys prior) leads to:
λBayes = 2 ln Beta(1/2 + s, 1/2 + n − s) − 2 ln(π) + 2n ln(2),
where s =

Pn

i=1

(6)

xi . For the Bayesian test λBayes ≈ −5.86, which implies the null model

is preferred and an apparent contradiction between inferential paradigms.

3.1

Resolving Lindley’s paradox

We will now resolve Lindley’s paradox in both of the above examples.
Resolving Lindley’s example: We argue that Problem I for Lindley’s example only
occurs because it is assumed that the p-value is held fixed, and that a true Lindley’s
paradox only occurs with vanishingly small probability as n → ∞. The p-value cannot
be held fixed as n → ∞ as its behaviour depends on the data generation process. Let
X = (X1 , . . . , Xn )T be a random sample. Consider the value of λBayes for Lindley’s
example as a function of this random sample, i.e., where


nτ 2
nz(X)2
− ln 1 + 2 .
λBayes (X) =
n + σ 2 /τ 2
σ

(7)

The first term on the right-hand side of (7) depends on the data generating process for
X, whereas the second term is O(ln(n)). Consider the two cases:
iid

1. If Xi ∼ N (µ0 , σ 2 ), i.e., the data is generated from H0 . Then z(X)2 ∼ χ21 = Op (1)
and the O(ln(n)) term dominates. Hence, as n → ∞ we have λBayes (X) → −∞
implying P(T (X) = 0) → 1, i.e., the null hypothesis is preferred.
iid

2. If Xi ∼ N (µ1 , σ 2 ) for some µ1 with µ1 6= µ0 , i.e., the data is generated from H1 .
2
2
02
Then z(X)2 ∼ χ02
1 (n(µ1 − µ0 ) /σ ) where χν (λ) is the non-central chi-squared

distribution with degrees of freedom ν and non-centrality parameter λ. Then
z(X)2 = Op (n) dominating O(ln(n)) term. Then as n → ∞ we have λBayes (X) → ∞
implying P(T (X) = 1) → 1, i.e., the alternative hypothesis is preferred.
Note that 1. implies that a test based on the above Bayes factor has vanishing type I error
as n → ∞ and 2. implies that the Bayesian test is consistent in the sense of (Lehmann,

9

2004, Section 3.3). Combining 1. and 2. implies that the test is Chernoff consistent (see
Section 5 for a formal definition).
Resolving Sprenger’s example: Using properties of the beta function, the gamma
function, and Stirling’s approximation leads to approximating (6) by
λBayes (x) = λLRT (x) − ln(n) − ln(π/2) + O(n−1 ),
where λLRT is the LRT statistic corresponding to the hypotheses (5). Again, Lindley’s
paradox occurs here if we consider λLRT (x) (or equivalently the p-value) to be fixed. If
λLRT is held fixed and n diverges then the null hypothesis will be preferred in the limit.
Let X = (X1 , . . . , Xn )T be a random sample. We will later show (see Section 5) that

 O (1) if H is true; and
p
0
λLRT (X) =
 Op (n) if H1 is true.
Hence, as n → ∞ we have λBayes (X) → −∞ if H0 is true, and λBayes (X) → ∞ if H1
is true so that the test based on λBayes is Chernoff consistent. Figure 1 illustrates the
empirical probabilities for rejecting the null hypothesis (for the frequentist test at the 5%
level) or preferring the alternative hypothesis (for the Bayesian test) based on simulating
106 datasets with the true value of ρ in the set {0.5, 0.5001, 0.5002, 0.5003} for n on a
grid form n = 106.5 to n = 109 . The vertical line in Figure 1 indicates the actual value
of n in Sprenger’s example and the dashed grey line illustrates the estimated empirical
probability that the two tests disagree. In Figure 1 we see that when H0 is false, as
the sample size increases, both tests reject the null as n or ρ grows. When ρ = 0.5 the
frequentist test accepts the null model at the 5% level, while the Bayesian test prefers
the null model with very low probability. Furthermore, at the actual value of n in the
experiment the disagreement between frequentist and Bayesian tests could have occurred
by chance with relatively high probability. However, for much larger large n such a
disagreement will only occur with low probability when H1 is true.



We now note for Lindley’s example that the test based on λBayes is an LRT test with
an extremely small level α given by α = P[χ21 > {1 + σ 2 /(nτ 2 )} ln(1 + nτ 2 /σ 2 )] (when τ 2
is large) while for Sprenger’s example that the test based on λBayes is in also an LRT test
with asymptotic level α = P[χ21 > ln(n) + ln(π/2)]. Hence, we note, as was also argued by
10

1.0
0.8
0.2
0.0

0.00

7.5

8.0

8.5

9.0

6.5

7.0

7.5

8.0

log10(n)

log10(n)

ρ = 0.5002

ρ = 0.5003

8.5

9.0

8.5

9.0

0.8
0.6

Power

0.0

0.2

0.4
0.0

0.2

0.4

0.6

0.8

1.0

7.0

1.0

6.5

Power

0.6

0.10

Power

0.15

LRT
Bayes
Disagree Prob.

0.05

type I error

ρ = 0.5001

0.4

0.20

ρ = 0.5

6.5

7.0

7.5

8.0

8.5

9.0

6.5

log10(n)

7.0

7.5

8.0

log10(n)

Figure 1: Empirical probabilities of rejecting the null hypothesis/preferring the alternative hypothesis for the simulation described in Section 3 comparing the LRT and Bayesian
tests when ρtrue ∈ [0.5, 0.5004] and n = 104, 490, 000.
Naaman (2016) and noted by Lindley (1957), that Lindley’s paradox can also be resolved
by letting the level of the test α in the frequentist test tend to zero as n → ∞. We discuss
Lindley’s paradox more generally in Section 5.2.
The above expression for the level of the test for Lindley’s example draws attention
to a second problem for Lindley’s example which does not occur in Sprenger’s example.
• Problem II: For any given p-value as τ 2 → ∞ we have BF01 → ∞.
This problem occurs because as the prior variance increases the level of the test decreases.
Problem II is referred to as Bartlett’s paradox in Liang et al. (2008) and the JeffreysLindley paradox in Robert (2014). Bartlett’s paradox is paradoxical since as τ 2 becomes
large the prior on µ becomes increasingly vague regarding the location of µ. However,
in the attempt to be vague about the location of µ the prior becomes “informative” in
favouring H0 as the preferred hypothesis, again, regardless of the evidence provided by
the data. Unlike Lindley’s paradox, we believe that Bartlett’s paradox is a real problem
in practice since the use of diffuse priors can sometimes lead to testing procedures with
extremely small power. Our proposed cake priors described in the next section circumvent
this problem.
11

4

Cake priors

Consider the general hypotheses (1). Let d0 and d1 be the dimensions of θ 0 and θ 1
respectively. For the time being we will assume that 0 < d0 ≤ d1 (later we will consider 0 ≤ d0 ≤ d1 ). Define the observed information and Fisher information matrices
as J(θ) = −∇2θ `(θ) and I(θ) = Exi |θ [−∇2θ ln p(xi |θ)] respectively. Define the mean
e
observed information matrix as J(θ)
= n−1 J(θ) and the mean expected observed infore
mation matrix as I(θ)
= n−1 Ex|θ [J(θ)]. We will denote the Fisher information matrix
under the null and alternative hypotheses as I0 (θ 0 ) and I1 (θ 1 ) with similar use of sube We define a Jeffreys prior as any
scripts to denote similar quantities such as J, Ie and J.
density for θ such that p(θ) ∝ |I(θ)|1/2 .
We construct cake priors using the following ingredients:
1. Define the priors
h
d
p(θ j |Hj ; gj ) = exp − 2j ln(2πgj ) + 21 ln |Pj (θ j )| −

1
θ T Pj (θ j )θ j
2gj j

i

,

(8)

where Pj (θ j ) is a prior precision matrix (assumed to be full rank). For all of the
examples considered in this paper we will use Pj (θ j ) = Iej (θ j ).
2. Set gj = h1/dj where h is a common hyperparameter.
3. Calculate the Bayes Factor as
Z
 . Z

1/d0
1/d1
BF01 (h) =
p(x|θ 0 , H0 )p(θ 0 |H0 ; h )dθ 0
p(x|θ 1 , H1 )p(θ 1 |H1 ; h )dθ 1 .
4. Optional: Let h → ∞ if flat priors are desired.
When Pj (θ j ) ∝ Ij (θ j ), j = 0, 1, (8) leads to a Bayes Factor, in the limit as gj → ∞, that
would have been obtained if a Jeffrey’s prior is used. When Pj (θ j ) ∝ Ij (θ j ), j = 0, 1,
(8) are Jeffreys priors in the limit as gj → ∞. Letting gj → ∞ would be problematic
if not for 2. which leads to certain terms involving h cancelling in the Bayes factor.
As h → ∞, the priors on θ j are made diffuse, but at a rate that depends on the dj ’s.
We keep 4. optional due to the fact that particular Bayesian procedures may require
proper priors. For particular examples in the coming subsections the above steps will
raise complications. These include: (A) Model parameters may not be defined on the
12

whole real line, e.g., variances. Priors of the form (8) are not appropriate for such model
parameters; (B) The priors p(θ j |gj , Hj ) may not be proper densities; and (C) If d0 = 0
using g = h1/dj is problematic. The examples in the subsections below will illustrate how
each of these complications can be handled. (A) & (B) will be dealt with in sections 4.1
and 4.2. Complication (C) will be dealt with in Section 4.4.
We will now give some intuition for how cake priors avoid Bartlett’s paradox via the
following heuristic argument. Let Pj (θ j ) ≡ Pj , i.e., the prior precision matrices are conR
 R

stant, then letting BF01 =
p(x|θ 0 , H0 )p(θ 0 |H0 ; g0 )dθ 0 / p(x|θ 1 , H1 )p(θ 1 |H1 ; g1 )dθ 1 ,
(which depends on g0 and g1 rather than h). Then the Bayesian test statistic is
λBayes
Z


n
o
T
P1 θ 1
dθ
exp `1 (θ 1 ) − θ1 2g
1
1


= 2 ln  Z
n
o
 + d0 ln(2πg0 ) − d1 ln(2πg1 ) + ln(|P1 |/|P0 |)
T
P0 θ 0
exp `0 (θ 0 ) − θ0 2g
dθ
0
0

R
p(x|θ 1 , H1 )dθ 1
+ d0 ln(2πg0 ) − d1 ln(2πg1 ) + ln(|P1 |/|P0 |) + O(g0−1 + g1−1 )
= 2 ln R
p(x|θ 0 , H0 )dθ 0
where the second line is obtained using a Taylor series argument in g0 and g1 . Ignoring
the dependency of O(gj−1 ) terms on the θ j ’s, using Laplace’s method on the numerator
bj )
e j (θ
and denominator of the first term in the second line above, and setting Pj = J
bj are the MLEs for the θ j ’s), leads to
(where θ
λBayes = λLRT − ν ln(n) + d0 ln(g0 ) − d1 ln(g1 ) + O(g0−1 + g1−1 + n−1 ).

(9)

The O(n−1 ) error follows from the relative error of the Laplace’s method applied to
the numerator and denominator (Tierney et al., 1989; Kass et al., 1990). Suppose that
g0 = g1 = g. Then using the asymptotic χ2ν distribution λLRT the level of the test using
(9) is α = P[χ2ν ≥ ν ln(ng)]. So that again we see that the power of the test goes to 0
as g → ∞. Here also we see that setting g0 to be a large constant (making the prior for
θ 0 diffuse) leads the test to preferring H0 while making g1 large leads to preferring H1 .
Hence, the relative rates that g0 and g1 diverge must be considered.
Setting gj = h1/dj means that d0 ln(g0 ) = d1 ln(g1 ) and leads to λBayes = λLRT −
ν ln(n) + O(h−1/d0 + h−1/d1 + n−1 ). For sufficiently large h and n we have λBayes ≈
λLRT − ν ln(n). The level of the test becomes approximately α = P[χ2ν ≥ ν ln(n)]. We
recognise the above arguments are informal in nature and will shortly illustrate cake
priors in concrete examples.
13

Lastly, well us briefly discuss the choice of Pj . Setting Pj = I leads to λBayes =
b0 )|−ln |J
b1 )|+O(h−1/d0 +h−1/d1 +n−1 ). This would be undesirable
e0 (θ
e1 (θ
λLRT −ν ln(n)+ln |J
because of the additional computational burden of the log-determinant terms (which can
be considerable in some contexts), and because if λLRT ≈ ν ln(n) we would prefer the
bj )|, i.e., larger standard errors. For this reason we would like
e j (θ
model with larger ln |J
bj ) so that at least approximate cancellation occurs.
e j (θ
Pj ≈ J

4.1

One sample test for equal means (with unknown variance)
iid

Consider the hypothesis test (3) where xi |µ, σ 2 ∼ N (µ, σ 2 ), 1 ≤ i ≤ n, where µ and σ 2
are the mean and variance parameters respectively. Suppose now that both µ and σ 2
are unknown parameters to be estimated (unlike the example in Section 2 where σ 2 was
assumed to be known).
The mean expected information matrices for the null and alternative hypothesis respectively are

Ie0 (σ 2 ) = 1/(2σ 4 )

and

Ie1 (µ, σ 2 ) = 

σ −2
0



0
4

1/(2σ )

,

which coincide with the Fisher information matrices in this case. Hence, Jeffreys priors
for µ and σ 2 are p(µ) ∝ 1 and p(σ 2 ) ∝ (σ 2 )−1 I(σ 2 > 0) respectively.
We cannot directly use the methodology outlined in Section 4 as σ 2 > 0. To handle this complication we use the transformation σ 2 = exp(s). Under this transformation the mean expected information matrices become: Ie0 (s) = 1/2 and Ie1 (µ, s) =
diag(exp(−s), 1/2). Using the steps for Section 4 under this transformation we have
s|H0 ∼ N (0, 2g0 ), i.e., p(s|H0 ) = [2π(2g0 )]−1/2 exp[−s2 /(4g0 )]. Transforming back to the
σ 2 parametrisation gives σ 2 |H0 ∼ LN (0, 2g0 ), with density which is a Jeffreys prior for
σ 2 in the limit as g0 → ∞.
Note that for H1 the upper left entry of Ie1 (µ, s) depends on s. This implies a conditional dependence of µ on s which leads to µ|s, H1 ∼ N (0, g1 exp(s)) and s|H1 ∼
N (0, 2g1 ). Transforming back from the parametrisation in s to the parametrisation using
σ 2 gives µ|σ 2 , H1 ∼ N (0, g1 σ 2 ) and σ 2 |H1 ∼ LN (0, 2g1 ). Note again that these are both
Jeffreys priors in the limit as g1 → ∞.

14

The marginal distributions of x given H0 and H1 are
Z ∞
h
i
1
nb
σ02
ln(4πg0 )
(ln σ 2 )2
2
n+2
√ n exp − 2 ln(σ ) − 2σ2 − 2 − 4g0 dσ 2 ,
p(x|H0 ) =
and
2π
0
Z ∞

i
h
1
nb
σg2
ln(4πg12 )
(ln σ 2 )2
2
1
1
√ n exp − n+2
ln(σ
)
−
−
−
−
ln
n
+
dσ 2 ,
p(x|H1 ) =
2
2
2σ
2
4g1
2
g1
2π
0
where σ
b02 = n−1 kx − µ0 1k2 , and σ
bg2 = n−1 [kxk2 − (nx)2 /(n + g −1 )]. Suppose that we were
to use g = g0 = g1 and let g → ∞. Then we would see a manifestation of Bartlett’s
paradox where the null hypothesis is favoured since BF01 → ∞ as g → ∞. If we instead
use g0 = h and g1 = h1/2 then the Bayes factor simplifies to


Z ∞

nb
σ02 (ln σ 2 )2
2
n
exp − 2 + 1 ln(σ ) − 2 −
dσ 2
2σ
4h
BF01 (h) = Z ∞

 0

nb
σh21/2
(ln σ 2 )2 1
2
−1/2
n
) dσ 2
−
− 2 ln(n + h
exp − 2 + 1 ln(σ ) −
2
1/2
2σ
4h
0
which can be evaluated using univariate quadrature or other methods for any fixed h > 0.
Since both the above integrands are monotonic as a function of h with a well defined
limit as h → ∞ we can apply the monotonic convergence theorem and take the limit h →
∞ inside both integrals. After simplifications including σ
bh2 → σ
b12 where σ
b12 = n−1 kx−x1k2
is the MLE for σ 2 under the alternative hypothesis we obtain λBayes = λLRT −ln(n), where
σ12 ) is the LRT statistic corresponding to the hypothesis (3).
λLRT = n ln(b
σ02 ) − n ln(b
As h → ∞ the parameter posterior distributions are given by
σ 2 |x, H0 ∼ IG

n n 2
, σ
b
2 2 0



b12 ),
µ|x, H1 ∼ tn (x, n−1 σ

,

and σ 2 |x, H1 ∼ IG

n n 2
, σ
b
2 2 1



.

As a computational short-cut if the integrand is a monotonic function of h with a well
defined limit as h → ∞ we will write
Z ∞
h
h⇒∞
p(x|H0 ) =
exp −
Z0 ∞
h
h⇒∞
p(x|H1 ) =
exp −
0

n
2


+ 1 ln(σ 2 ) −

nb
σ02
2σ 2

n
2


+ 1 ln(σ 2 ) −

nb
σ12
2σ 2

i

dσ 2 ,

and
i
− 12 ln(n) dσ 2 ,

h⇒∞

where the notation = is used to denote “equality in the limit as h → ∞ after terms
related to h cancel in the numerator and denominator in the Bayes factor, or terms related
to h vanish as h diverges in the Bayes factor.” The above expressions can be more easily
simplified using standard results to reach the same expression for λBayes as above.
We conduct the following short simulation study to illustrate the differences between
the LRT and the Bayesian test for this problem. Letting µ0 = 0 we simulate a single set
15

of data from xi ∼ N (µtrue , 1), 1 ≤ i ≤ n. After simulating 106 such datasets for all values
of µtrue in the set {0, 0.05, 0.25, 0.5} and a grid of n from n = 15 to n = 1000 we plot
in Figure 2 the empirical probabilities of rejecting the null hypothesis (for the LRT test)
using α = 0.05 or preferring the alternative hypothesis (for the Bayesian test).
From Figure 2 we see empirically that the type I error of the Bayesian test is tending
to 0 as n grows when H0 is true, whereas the LRT test has, by design, a type I error
of 0.05. When H1 is true and ln(n) < χ21,α the Bayesian test is more powerful than the
LRT test, and when H1 is true and ln(n) > χ21,α the LRT test is more powerful than the
Bayesian test. When µtrue ∈ {0.25, 0.5} both tests appear to have power tending to 1 as

2.5

0.30
0.20
0.10

2.0

2.5

0.8
0.6
0.4
0.2

LRT
Bayes

0.0

0.4
0.2

2.0

log10(n)

2.5

1.0

µ = 0.5

Prob. of rejecting null/preferring alternative

µ = 0.25

0.6

0.8

1.5

log10(n)

LRT
Bayes

1.5

LRT
Bayes

0.00

0.00

2.0

µ = 0.05

log10(n)

1.0

1.5

Prob. of rejecting null/preferring alternative

LRT
Bayes

0.10

0.20

0.30

µ=0

0.0

Prob. of rejecting null/preferring alternative

Prob. of rejecting null/preferring alternative

n grows. Lastly, for the case µtrue = 0.5 when ln(n) > χ21,α both have very similar power.

1.5

2.0

2.5

log10(n)

Figure 2: Empirical probabilities of rejecting the null hypothesis/preferring the alternative hypothesis for the simulation described in Section 4.1 comparing the LRT and
Bayesian tests when µtrue ∈ {0, 0.05, 0.25, 0.5}.

16

4.2

Two sample test for equal means

Suppose we have data x = (x1 , . . . , xn )T . We want to test whether the first n0 samples
x0 = (x1 , . . . , xn0 )T from class 0 come from the same normal population as the second n1
samples x1 = (xn0 +1 , . . . , xn )T from class 1 with n0 + n1 = n. We wish to test
H0 : xi |µ, σ 2 ∼ N (µ, σ 2 ), 1 ≤ i ≤ n,
versus

 xi |µ0 , σ02 ∼ N (µ0 , σ02 ), 1 ≤ i ≤ n0 ,
H1 :

xi |µ1 , σ12 ∼ N (µ1 , σ12 ), n0 + 1 ≤ i ≤ n,

(10)

where µ, σ 2 , µ0 , σ02 , µ1 and σ12 are the means and variances under the one and two group
hypotheses respectively. Here θ 0 = (µ, σ 2 )T with d0 = 2 and θ 1 = (µ0 , µ1 , σ02 , σ12 )T
with d1 = 4. Using similar arguments as in Section 4.1 with P0 (θ 0 ) = Ie0 (θ 0 ) =
diag[σ −2 , 1/(2σ 4 )] and P1 (θ 1 ) = Ie1 (θ 1 ) = diag[n0 /(nσ02 ), n1 /(nσ12 ), n0 /(2nσ04 ), n1 /(2nσ14 )]
leads to the priors
µ|σ 2 , H0 ∼ N (0, g0 σ 2 ),

σ 2 |H0 ∼ LN (0, 2g0 ),
µ1 |σ12 , H1 ∼ N (0, g1 (n/n1 )σ12 ),

µ0 |σ12 , H1 ∼ N (0, g1 (n/n0 )σ02 ),
σ02 |H1 ∼ LN (0, 2(n/n0 )g1 ),

and σ12 |H1 ∼ LN (0, 2(n/n1 )g1 ).

Setting gj = h1/dj , j = 0, 1 results in
p(x|H0 )
=
h⇒∞

=

=

∞



nb
σ 2 1/2
1
ln(4πh)
(ln σ 2 )2
2
1
h
√ n exp − n+2
ln(σ
)
−
−
−
−
ln
n+
2
2σ 2
2
4h
2
2π
0
Z ∞
i
h
1
ln(4π)
2
n+2
nb
σ2
1
√ n exp − 2 ln(σ ) − 2σ2 − 2 − 2 ln(n) dσ 2
2π
0
h
i
 1
n
1
b
exp ln p(x|θ 0 ) + ξ 2 − 2 ln(2) − 2 ln(n) ,
Z

√1
h



dσ 2

b0 ) = −(n/2) ln (2πb
where ξ(x) = ln Γ(x) + x − x ln(x) − (1/2) ln(2π) and ln p(x|θ
σ 2 ) − n/2
b0 . Similarly, p(x0 |H1 , h) is
is the log-likelihood of the null model evaluated at its MLE θ
Z
p(x0 |H1 ) =
0

∞
√ 1 n0
2π

h

exp −

n0
2

+1



ln(σ02 )

−

1
2σ02

n
kx0 k2 −

− 12 ln(n + h−1/4 ) − 12 ln(2π(2n/n0 )h1/2 ) −

(n0 x0 )2
n0 +(n0 /n)h−1/4

(ln σ02 )2
4(n/n0 )h1/4

o

i
dσ02 .

By construction the ln(h1/2 ) term cancels in the numerator and denominator of the Bayes
factor. The integrand is a monotonic function of h (apart from the ln(h1/2 ) term which
17

cancels) and has a well defined limit as h → ∞. Taking h → ∞ the above expression



for p(x0 |H1 , h) simplifies to p(x0 |H1 ) = exp `(b
µ0 , σ
b02 ) + ξ n20 − 12 ln(2n2 /n0 ) , where
`(b
µ0 , σ
b02 ) = −(n0 /2) ln(2πb
σ02 ) − n0 /2. Combining with a similarly obtained expression for
p(x1 |H1 ) we obtain
λBayes = λLRT − 3 ln(n) − 2ξ(n/2) + 2ξ(n0 /2) + 2ξ(n1 /2) + ln(n0 n1 /2),
b2 = n−1 kx − x1k2 , σ
b02 =
σ12 ), the estimators σ
where λLRT = n ln(b
σ 2 ) − n0 ln(b
σ02 ) − n1 ln(b
2
2
n−1
b12 = n−1
0 kx0 − x0 1k , and σ
1 kx1 − x1 1k are the MLEs for the variance parameters.

Stirling’s asymptotic expansion of ln Γ(z) for large z is ln Γ(z) = z ln(z)−z −(1/2) ln(z)+
(1/2) ln(2π)+O(z −1 ) (see for example Equation 6.1.37 of Abramowitz and Stegun, 1972).
Hence, ξ(x) = − 21 ln(x) + O(x−1 ). Using this λBayes simplifies to
−1
λBayes = λLRT − 2 ln(n) + O(n−1
0 + n1 ).

Note that the coefficient of ln(n) is d1 − d0 = 2 which is the corresponding degrees of
freedom of the corresponding LRT.
The parameter posteriors are given by:

µ|x, H0 ∼ tn (x, n−1 σ
b2 ), σ02 |x, H0 ∼ IG n2 , n2 σ
b2 , µ0 |x, H1 ∼ tn0 (x0 , n−1
b02 )
0 σ


b02 , µ1 |x, H1 ∼ tn1 (x1 , n−1
b12 ), and σ12 |x, H1 ∼ IG n21 , n21 σ
b12 .
σ02 |x, H1 ∼ IG n20 , n20 σ
1 σ
It is important to note that all of the constant terms have cancelled from the asymptotic approximation for λBayes . This has been achieved by incorporating the (n/n0 ) and
(n/n1 ) factor in the priors for µ0 and µ1 , and the (2n/n0 ) and (2n/n1 ) factors in the
priors for σ02 and σ12 . Without these factors, cancellation of O(1) and larger terms in the
expression λBayes would not occur.
We conducted a short numerical experiment to compare our Bayesian test with the
LRT. We simulated 106 datasets where n0 = n1 = 50 with the true parameter values
(i) µ0 = 0, σ0 = σ1 = 1 and µ1 ∈ {0, 0.25, 0.5, 1}; or
(ii) µ0 = 0, µ1 = 0, σ0 = 1 and σ1 ∈ {0, 1.25, 1.5, 2.5}.
The empirical probabilities of rejecting the null (in the LRT case) or preferring the alternative (in the Bayesian test) are illustrated in Figure 3. Note that under H0 the type
18

I error approaches zero as n → ∞ for the Bayesian test, and under H1 the type II error
approaches zero as n → ∞ for both the Bayesian and LRT tests. When H0 is true the
LRT has a fixed 5% type I error.

4.3

Linear models

We will now consider hypothesis testing for linear models. Consider the base model
y|α, β, σ 2 ∼ N (α1 + Xβ, σ 2 I),
where y is a response vector of length n, β is a coefficient vector of length p, σ 2 is a
positive scalar, X is a full-rank n by p matrix of covariates, and I is the identity matrix
of appropriate dimension. In order to simplify some calculations we will transform y
and X so that y and the columns of X are standardized, i.e., y = 0, kyk2 = yT y = n,
XTj 1 = 0, and kXj k2 = n where Xj is the jth column of X. Let γ be a binary vector
of length p, and let Xγ be the submatrix X comprised from the columns of X whose
corresponding elements of γ are non-zero. Consider the hypothesis test
H0 : γ = γ 0

versus

H1 : γ = γ 1 ,

(11)

where γ 0 and γ 1 denote the models under the null and alternative hypotheses respectively
with 0 ≤ |γ 0 | ≤ |γ 1 |.
To simplify exposition for this example we will only use cake priors for α and β γ .
Since σ 2 is a common parameter across all parameters we can use the typical improper
(Jeffreys) priors for σ 2 given by p(σ 2 ) ∝ (σ 2 )−1 I(σ 2 > 0). This choice has been formally
justified in Berger et al. (1998). Cake priors can be used for all parameters for this
example, but the working out is lengthy and unnecessarily obfuscates the exposition.
e β γ ) = diag(σ −2 , σ −2 XT Xγ /n) for a particular model γ leads to
Using P(α, β γ ) = I(α,
γ

α|σ 2 , g ∼ N (0, gσ 2 ),
Further, we use p(β −γ ) =

Q

j : γj =0


and β γ |σ 2 , g ∼ N 0, gσ 2

−1
1 T
X X
n γ γ



.

(12)

δ(βj ; 0) where δ(x; a) is the Dirac delta function with

location a. The prior on β γ is simply the Zellner g-prior (Zellner, 1986) where the prior
covariance is scaled by a factor of n. The prior on β γ combined with the prior on β −γ is
a spike and slab prior for β.
19

3.0

3.5

4.0

3.5

4.0

3.0

3.5

4.0

2.0

1.0
0.8
0.6
0.4
0.2
0.0
1.0
0.8

2.5

4.0

3.0

3.5

4.0

1.0

4.0

0.8
0.6
0.4
0.2

1.5

2.0

2.5

0.4

0.6

0.8

1.0

σ1 = 2.5

3.5

3.5

0.0
1.0

σ1 = 1.5

3.0

3.0

LRT
Bayes

log10(n)

2.5

4.0

0.6

2.0

log10(n)

log10(n)

3.5

0.4
0.2
0.0

1.5

0.2

LRT
Bayes

0.0

0.8

Prob. of rejecting null/preferring alternative

0.15
0.10
0.05

1.5

1.0

σ1 = 1.25

2.5

3.0

LRT
Bayes

σ1 = 1

LRT
Bayes

1.0

2.5

log10(n)

0.00

2.0

Prob. of rejecting null/preferring alternative

3.0

Prob. of rejecting null/preferring alternative

0.8
0.6
0.4
0.2

2.5

1.0

1.5

2.0

log10(n)

LRT
Bayes

1.0

1.5

µ1 = 1

0.0

2.0

1.0

µ1 = 0.5

0.20

1.5

LRT
Bayes

log10(n)

LRT
Bayes

1.0

Prob. of rejecting null/preferring alternative

0.20
0.15
0.10
0.05
0.00

2.5

µ1 = 0.25

log10(n)

0.6

Prob. of rejecting null/preferring alternative

2.0

1.0

1.5

0.4

Prob. of rejecting null/preferring alternative

1.0

0.2

Prob. of rejecting null/preferring alternative

LRT
Bayes

0.0

Prob. of rejecting null/preferring alternative

µ1 = 0

1.0

1.5

2.0

2.5

3.0

3.5

4.0

log10(n)

Figure 3: The empirical probabilities of rejecting the null (in the LRT case) or preferring
the alternative (in the Bayesian test) when simulating two normal populations with n0 =
n1 = 50, and (i) µ0 = 0, σ0 = σ1 = 1 and µ1 ∈ {0, 0.25, 0.5, 1} (left four panels); or (ii)
µ0 = 0, µ1 = 0, σ0 = 1 and σ1 ∈ {0, 1.25, 1.5, 2.5} (right four panels).

20

Marginalizing over α, β and σ 2 for a particular model γ we obtain after simplification
h
Γ(n/2)
p(y|γ, g) =
exp −
(nπ)n/2

1+|γ|
2

ln(g) −

1+|γ|
2

−1

ln(n + g ) −

n
2



ln 1 −

g
R2
1+g γ

i
,

where Rγ2 is the usual R-squared statistic for model γ. This is equivalent to the g =
h1/dh of Section 4 to using the hyperpriors p(g|γ j ) = δ(g; h1/(1+|γ j |) ), j = 0, 1. After
marginalizing over g the Bayes factor as a function of h simplifies to
h




2
2
h1/(1+|γ 0 |)
h1/(1+|γ 1 |)
BF01 (h) = exp − n2 ln 1 − 1+h
+ n2 ln 1 − 1+h
1/(1+|γ 0 |) Rγ 0
1/(1+|γ 1 |) Rγ 1
0|
− 1+|γ
2

−1/(1+|γ 0 |)

ln n + h



+

1+|γ 1 |
2

−1/(1+|γ 1 |)

ln n + h

i

.

Taking h → ∞ we use the fact that 1 − Rγ2 = σ
bγ2 (where σ
bγ2 is the MLE for σ 2 under the
model γ) to obtain
h
i h
i
σγ2 1 ) − |γ 1 | ln(n) − −n ln(b
σγ2 0 ) − |γ 0 | ln(n)
λBayes = −n ln(b
= BICγ 0 − BICγ 1 = λLRT − ν ln(n),
bγ, σ
where BICγ = n ln(2πb
σγ2 ) − n + |γ| ln(n) = −2 ln p(y|b
αγ , β
bγ2 ) + |γ| ln(n), and, α
bγ
b γ are the MLEs for α and β under model γ, λLRT = n ln(b
and β
σγ2 0 ) − n ln(b
σγ2 1 ) is the
LRT statistic corresponding to the hypotheses (11) and ν = |γ 1 | − |γ 0 |. Hence, for these
models and prior structures the Bayesian test statistic is simply the difference between
two BIC values.
Note that as h → ∞ the parameter posteriors become
α|y, γ ∼ tn (0, σ
bγ2 /n),

2
T
b ,σ
β γ |y, γ ∼ tn (β
γ bγ Xγ Xγ

−1

),

and σ 2 |y, γ ∼ IG

n n 2
, σ
b
2 2 γ



,

b γ and σ
where β
bγ2 are the MLEs corresponding to model γ.
We will not provide any numerical examples due to the close relationship between our
Bayes factors and the BIC, and the fact that almost every paper ever written on model
selection for linear models uses the BIC in its comparisons. We direct the interested
reader to any of the papers in the discussion below all of which make comparisons with
the BIC as a model selection criteria.
There are four main differences between the priors used here and the priors that
have been used in the literature for linear models. The first such difference is the choice
of prior on α which the typical prior is to use the Jeffreys prior p(α) ∝ 1 which was
21

advocated in Berger et al. (1998). If we were to use this prior and were only to use cake
priors for β γ then p(g0 |γ 0 ; h) = δ(g0 ; h1/|γ 0 | ) instead of p(g0 |γ 0 ; h) = δ(g0 ; h1/(1+|γ 0 |) ).
The consequence of this would be that the null model (where γ = 0) would become
problematic to calculate.
The second difference is in the choice of prior for β. Most Bayesian approaches to
model selection for linear models use the Zellner g-prior where
β γ |σ 2 , g ∼ N (0, gσ 2 (XTγ Xγ )−1 ),

(13)

instead of the prior for β γ in (12). This difference is subtle.
Bayarri et al. (2012) advocate the priors for β should remain proper and not degenerate to a point mass. If we were to treat Xγ as random then under mild conditions
XTγ Xγ /n → E(XTγ Xγ ) almost surely suggesting that our prior for β γ does not degenerate
to a point mass. Lastly (12) and (13) simplify marginal likelihoods since terms involving
determinants cancel, and have the added advantage that they do not depend on the unit
of measurements of the covariates.
The third difference is the choice of prior on g. As stated in the introduction, Liang
et al. (2008) argues for a hyperprior to be assigned to g. Liang et al. (2008) considers
the hyper g-prior; the hyper g/n-prior; and the Zellner-Siow prior (equivalent to a particular inverse-gamma on g) (Zellner and Siow, 1980). Maruyama and George (2011)
use a different prior to (12) or (13) and a beta-prime prior with specially chosen prior
hyperparameter values. All of these choices, apart from Maruyama and George (2011),
either no closed form expression for the marginal likelihood exists, or such an expression
is in terms of a Gauss hypergeometric function which is numerically difficult to evaluate
(Pearson et al., 2017) so that approximation is required.
Model selection consistency is another desirable criteria of Bayarri et al. (2012). The
authors corresponding g/n, Zellner-Siow, beta-prior and robust priors are model selection
consistent for all possible models. Our prior specification results in a null based Bayes
factor is a simple function of the BIC and so achieves model selection consistency for iid
data (under some additional mild assumptions, Yang, 2005). See also Section 5.2.

22

4.4

Handling zero parameters in the null model

Let us now return to Lindley’s example posed by Lindley (1957) described in Section 3.
In order to apply the methodology of Section 4 the null model needs to have a non-zero
number of parameters. We provide the following novel artificial construct to handle this
case in order to augment the problems so that both hypotheses have a non-zero number
of parameters.
1. Introduce a second sample of hypothetical data, say z.
2. Modify the null and alternative hypotheses by adding a clause that the hypothetical
data has the same distribution under the null and alternative hypotheses.
3. Apply the methodology of Section 4 to the augmented problem.
In order to illustrate this approach suppose we have a second sample of hypothetical data
z = (z1 , . . . , zn )T and consider the augmented hypotheses
H0 : x1 , . . . , xn ∼ N (µ0 , σ 2 )

z1 , . . . , zn |e
µ ∼ N (e
µ, σ 2 )

and

H1 : x1 , . . . , xn |µ ∼ N (µ, σ 2 )

versus

z1 , . . . , zn |e
µ ∼ N (e
µ, σ 2 ),

and

where µ0 and σ 2 have known fixed values, and µ
e is an artificial mean parameter corresponding to the sample z. This is a modification of the original hypotheses (3) has
the same logical implication as the hypotheses (3) for the observed sample x since the
hypothetical data has the same hypothetical models under the null and alternative hypotheses. For the augmented problem we have θ 0 = µ
e with d0 = 1, and θ 1 = (µ, µ
e)T
and d1 = 2 so that we have avoided the problem of dividing by zero. The cake priors
become µ
e|H0 ∼ N (0, g0 σ 2 ), µ|H1 ∼ N (0, g1 σ 2 ) and µ
e|H1 ∼ N (0, g1 σ 2 ). For g0 and g1 we
use g0 = h and g1 = h1/2 . Then
ln p(x|H0 ) = − n2 ln(2πσ 2 ) −

kx−µ0 1k2
,
2σ 2

ln p(z|H0 ) = − n2 ln(2πσ 2 ) −

1
2σ 2

h
kzk2 −

(nz)2
n+h−1

ln p(x|H1 ) = − n2 ln(2πσ 2 ) −

1
2σ 2

h
kxk2 −

(nx)2
n+h−1/2

i


− 21 ln(h1/2 ) − 12 ln n +

√1
h



, and

ln p(z|H1 ) = − n2 ln(2πσ 2 ) −

1
2σ 2

h
kzk2 −

(nz)2
n+h−1/2

i


− 21 ln(h1/2 ) − 12 ln n +

√1
h



.

23

i

− 21 ln(h) − 12 ln(n + h−1 ),

The Bayes factor in the limit as h → ∞ is


p(x|H0 )p(z|H0 )
= λLRT − ln(n),
λBayes = lim −2 ln
h→∞
p(x|H1 )p(z|H1 )
where λLRT = σ −2 (kx − µ0 1k2 − kx − µ
b1k2 ) is the likelihood ratio test statistic corresponding to the hypothesis (3). We conducted a small simulation study identical to the
simulation study in Section 4.1 with the exception that σ 2 was treated as known. The
resulting figure and interpretation was nearly identical to that in Section 4.1 (not shown).

5

Theory

In all of the examples in Section 4 the quantity λBayes can be placed into the form (2).
We will now consider the asymptotic properties of hypothesis tests based on this form.
Shao (2003) developed theory regarding the asymptotic properties of hypothesis tests.
We will adopt his notation and definitions here. Let X = (X1 , ..., Xn )T be a random
sample from P = { pi ( · ) : i = 1, . . . , n }. The type I and type II errors are defined by
αT (P) = P( T (X) = 1) when P ∈ P0 and 1 − αT (P) = P( T (X) = 0) when P ∈ P1
respectively. Fix the level of significance α such that supP∈P0 {αT (P)} ≤ α. We will now
suppose that Tn (X) ≡ T (X) and consider scenarios where n diverges. In our ensuing
discussion we use the following definitions.
Definitions from 2.13 of Shao (2003):
(i) If lim sup {αTn (P)} ≤ α then α is an asymptotic significance level of Tn .
n→∞ P∈P0

(ii) If lim sup {αTn (P)} exists, then it is called the limiting size of Tn .
n→∞ P∈P0

(iii) The sequence of tests Tn is called consistent if and only if the type II error probability
converges to 0, i.e., limn→∞ [1 − αTn (P)] = 0, for any P ∈ P1 .
(iv) The sequence of tests Tn is called Chernoff-consistent if and only if Tn is consistent
and the type I error probability converges to 0, i.e., limn→∞ {αTn (P)} = 0, for any
P ∈ P0 . Furthermore, Tn is called strongly Chernoff-consistent if and only if Tn is
consistent and the limiting size of Tn is 0.
We note that any reasonable test which is consistent where the level α is controllable,
can be made Chernoff-consistent by letting αn ≡ α → 0 as n → ∞.
24

Wilks Theorem (Wilks, 1938) tells us that, assuming the data was generated under the
null distribution, under appropriate regularity conditions (including that the hypotheses
are nested) that λLRT converges to χ2ν in distribution so that λLRT = Op (1). A detailed
exposition on the characterization the asymptotic distribution of the LRT statistic under
quite general conditions, including when H0 and/or H1 is misspecified, and whether the
hypotheses are nested or non-nested, can be found in Vuong (1989). Below we summarize
the most pertinent results.

5.1

Asymptotic properties of the likelihood ratio test statistic

Suppose X1 , X2 , . . . , Xn are independent random variables from { p0i ( · ) : i = 1, . . . , n }
and that we have a parametric model P = { pi ( · |θ) : i = 1, . . . , n, θ ∈ Θ } (which
may or may not include the true distribution(s) { p0i ( · ) }). Define the log-likelihood as
P
`(θ) = ni=1 ln pi (Xi |θ), the MLE and “pseudo-true” value of θ as
b = arg max { `(θ) }
θ
θ

 

θ ∗ = arg max E n−1 `(θ) ,

and

θ

respectively (assuming both are well-defined). Assume also that E [n−1 `(θ ∗ )] → C,
for some finite C > 0. Under the “pseudo-true” value θ ∗ the resultant distribution
is { pi ( · |θ ∗ ) } which is the “best” distribution in the sense that it results in the smallest
Kullback-Leibler (KL) divergence with respect to the true distribution over all distributions for the model. We assume that these are unique.
If P is suitably regular, with Θ a nice subset of d-dimensional Euclidean space, then
certain derivatives exist and various statements can be made: the Euclidean norm of
b − θ ∗ is Op (n−1/2 ), and writing ∇`(θ) and ∇2 `(θ) for the first and second order partial
θ
b to get
derivatives we may expand `(θ ∗ ) about θ
h
i
b T ∇2 `(θ)
e (θ ∗ − θ)
b
b + (θ ∗ − θ)
b T ∇`(θ)
b + 1 (θ ∗ − θ)
`(θ ∗ ) = `(θ)
2
h
iT h
ih
i
b − 1 n1/2 (θ
b − θ ∗ ) E − 1 ∇2 `(θ ∗ ) n1/2 (θ
b − θ ∗ ) + op (1),
= `(θ)
2

n

b ≡ 0, for some θ
e between θ ∗ and θ;
b also the quadratic form is Op (1). Thus,
since ∇`(θ)
we may decompose the maximised log-likelihood into the following terms:
b = E [`(θ ∗ )] + [`(θ ∗ ) − E {`(θ ∗ )}] + Op (1).
`(θ)

25

The first term on the right hand side is asymptotic to nC; the second term is a random
sum of n terms with expectation zero, and so under further regularity conditions is
Op (n1/2 ). We refer to these three terms as the “Op (n)”, “Op (n1/2 )” and “Op (1)” terms
respectively (from left to right). Precise regularity conditions guaranteeing all of this nice
behaviour may be found in Vuong (1989); see also Chapter 5 of van der Vaart (1998).
They are all easily satisfied in all of our examples.
Finally, we note that the first term can be written as:
#
" n
#
" n
X
X  pi (Xi |θ) 
E [`(θ)] = E
ln
+E
ln p0i (Xi )
p0i (Xi )
i=1
i=1
" n
#
X
=
b − KL( p0i || pi ) + E
ln p0i (Xi )
i=1

where the first term corresponds to the negative KL-divergence between p0i and pi , and
the second term is related to the entropy of p0i . We see from the above equation that
maximizing E [`(θ)] with respect to θ is equivalent to minimizing the KL-divergence
Q
Q
between ni=1 p0i ( · ) and ni=1 pi ( · |θ).

5.2

Comparing models

Suppose now that we have two competing models Pj = { pij ( · |θ j , Hj ) : i = 1, . . . , n, θ j ∈
b0 and θ
b1 , and
Θ }, j = 0, 1, with corresponding log-likelihoods `0 ( · ) and `1 ( · ), MLEs θ
bj ). Suppose also
pseudo-true values θ ∗0 and θ ∗1 . For convenience we will write `bj ≡ `j (θ


that for j = 0, 1, we have E n−1 `j (θ ∗j ) → `∗j .
There are two main cases:
1. If `∗1 > `∗0 then the model under H1 has a smaller KL-divergence from the true
distribution than the model under H0 , and we have immediately
P
n−1 λLRT (X) = n−1 (`b1 − `b0 ) → `∗1 − `∗0 > 0 ,

that is the LRT statistic λLRT (X) = 2(`b1 − `b0 ) is of order n in probability.
2. If `∗1 = `∗0 then immediately we see that the two “Op (n)” terms in the difference
between the log-likelihoods would, at least asymptotically, cancel out and that the
“Op (n1/2 )” terms would “dominate”. However, in many practical examples the
26

“Op (n1/2 )” terms also cancel out, in which case
λLRT (X) = 2(`b1 − `b0 ) = Op (1) .
(see in particular Theorem 3.3 of Vuong, 1989). This occurs when the “best” member of each model yields the same distribution, that is when { pi0 ( · |θ ∗0 , H0 ) } =
{ pi1 ( · |θ ∗1 , H1 ) }. The parametrisations may be completely different, but nonetheless the distributions corresponding to the pseudo-true parameter values are identical. This occurs when the two models have some overlap, i.e., P0 ∩ P1 is non-empty
as a subset of all possible sets of distributions { pi ( · ) }. In such a case, the models
may or may not be nested, and may or may not be correctly specified; however the
“best” distribution in both is the same (and is part of the overlap).
We also have the following consequences.
• Lindley’s paradox: If H1 is correct (and H0 is not) then the above theory implies λLRT (X) is Op (n) and a test of the form (2) is consistent. Since Lindley’s
paradox occur with asymptotic probability p(χ2ν,α < λLRT < ν ln n), it occurs with
vanishingly small probability as n → ∞.
• Chernoff-consistency: If H0 is true then λLRT (X) is Op (1) and P(T (X) = 1) → 0
as n → ∞. If H1 is true then λLRT (X) is Op (n) and P(T (X) = 0) → 0 as n → ∞.
Hence, a Bayesian test of the form (2) is Chernoff consistent.
• Model selection consistency: Now suppose that we have M competing hy

potheses Hj , j = 1, . . . , M . For each model we have E n−1 `j (θ ∗j ) → `∗j for some
constants `∗j . We will call a hypothesis Hj correct if j ∈ C where
n
o
∗
∗
C = j : `j = max `k .
k=1,...,M

The hypotheses Hj such that j ∈ C correspond to correct models in the sense that
all such models are closest in terms of their KL-divergence to the data generating
distribution. Using (2) to compare models not in C with models in C leads to
λLRT (X) is Op (n) and the test preferring the model in C. Comparing any two
models in C will asymptotically prefer the model with the smallest size.
When cake priors become arbitrarily diffuse to the point of becoming improper and a
further problem occurs. We now discuss such problems.
27

6

Arbitrary constants

We now return to the issue of improper priors in the context of Bayes factors discussed in the introduction. When using improper priors, consider the conditional density
R
p(x|θ i , Hj ) where p(θ|Hj ) ∝ fi (θ j ) = Dj fj (θ j ) for some Dj > 0 such that fj (θ j )dθ j
does not exist for j = 0, 1. Then
B01

R
p(x|θ, H0 )f0 (θ 0 )dθ 0
D0
×R
.
=
D1
p(x|θ, H1 )f1 (θ 1 )dθ 1

(14)

This Bayes factor is problematic since it depends on two arbitrary constants D0 and D1 .
• Problem III: When using improper priors either the null or the alternative model
can be made to be preferred by artificially changing D0 or D1 to suit the a priori
preferred conclusion.
In the limit as as h → ∞ cake priors become improper. Suppose that instead of using
1/dj

gj = hj

we use gj = (Dj h)1/dj , where cj > 0 are arbitrary constants. This implies
λBayes = λLRT − ν ln(n) + ∆ + op (1)

where ∆ = ln(D1 /D0 ) is a controllable constant determined by how the gj parameters
diverge. Thus, we have not avoided the problem of arbitrary constants in our test.
Based on the theory developed in Section 5, if ∆ = O(1) then the corresponding test
will have all of the properties discussed in Section (5.2) where the level of the test requires
adjustment. For small sample sizes the value of ∆ trades-off the probabilities of type I
and type II errors. We do not believe that the model nor the data can make the choice of
∆ value on behalf of the analyst. Furthermore, any criteria used to select ∆ is making the
choice in trade-off of relative probabilities of type I and type II errors whether implicitly
or explicitly. Implicitly cake priors are choosing ∆ = 0.
This choice might be preferred for the following reasons:
1. This choice leads to a parsimonious expression for λBayes (all O(1) terms cancel);
2. When the number of parameters in the null and alternative hypotheses are equal
(ν = 0) using ∆ = 0 leads to preferring the model with the larger likelihood. If
∆ 6= 0 this is always the case. If the analyst desired to explicitly favour ether model
when ν = 0 then the prior odds should be altered to achieve this; and
28

3. Choosing different values of D0 and D1 would be inconsistent with similar choices
made in the literature. For example, in the linear models example one might also
use p(α2 |Hj ) = Dj or p(σ 2 |Hj ) = Dj (σ 2 )−1 I(σ 2 > 0). No papers in the model
selection literature, to our knowledge, chose different constants Dj for each model
under consideration. This suggests choosing D0 = D1 is reasonable.
Lastly, we note that if we select ∆ = ν ln(n) − χ2ν,α , where χ2ν,α denotes the upper
quantile function of the chi-squared distribution with degrees of freedom parameter ν,
that Bayesian tests can be made to mimic the frequentists LRT when the type I error is
controlled to have level α.

7

Interpretation

For Bayesian test using cake priors and the LRT test functions are (approximately)
TBayes (x) = I(λLRT (x) > ν ln(n))

and

TLRT (x) = I(λLRT (x) > χ2ν,α )

(15)

respectively. This also allows us to obtain a one-to-one mapping between p-values and
values of λBayes (x). It can be shown
ν + 2 ln(1/α) − 5/2 ≤ χ2ν,α ≤ ν + 2 ln(1/α) + 2

p

ν ln(1/α),

(16)

where the lower bound only holds for α ≤ 0.17 and ν ≥ 2 (Laurent and Massart, 2000;
Inglot, 2010). For our Bayesian tests the cut-off value, ν ln(n) grows with n implying
that the level of the test decays with n. A consequence is that our Bayesian tests offer
some protection against a sequential analysis (where samples are collected and statistical
significance checked sequentially). Suppose that α = (n/e)ν/2 then (16) becomes ν ln(n)−
p
5/2 ≤ χ2ν,α ≤ ν ln(n)+ 2ν 2 ln(n) so that χ2ν,α  ν ln(n), and the frequentist and Bayesian
testing procedures are roughly equivalent.
Given that (15) we can directly compare that the Bayes factors have taken the interpretation offered by Kass and Raftery (1995) in Table 1 appears to have the short-coming
of not taking into account the degrees of freedom ν of the test, nor the sample size n.
Consider Table 2 which directly compares p-values with their corresponding λBayes for
given n and ν. If one were to use a LRT instead where λBayes (x) takes the threshold values in Table 1, i.e., λBayes (x) ∈ {0, 2, 6, 10}, ν ranges from 1 to 5, and n ∈ {50, 102 , 103 }.
29

Note that every p-value is smaller than 5% suggesting that Bayesian tests are typically
more conservative at preferring the alternative than classical tests reject the null at the
usual 5% cut-off. Further, going from λBayes (x) = 2 to λBayes (x) = 6 and from λBayes = 6
to λBayes (x) = 10 roughly translates to a 5 to 10 fold reduction in the corresponding
p-value. Thus, the anticipated potential short-coming of Table 1 not depending on ν or
n does not pan out, at least for the values of n and ν considered in Table 2. We believe
Table 2 is a reasonable interpretation of strength of evidence for Bayesian tests.
p-values

λBayes

ν

λBayes

n = 50

n = 102

n = 103

n = 50

n = 102

n = 103

1

0

4.79E-02

3.18E-02

8.58E-04

0.05

−0.1

−0.8

−3.1

1

2

1.50E-02

1.02E-02

2.84E-03

0.01

2.7

2.0

−0.3

1

6

1.64E-03

1.13E-03

3.27E-04

0.001

6.9

6.2

3.9

1

10

1.92E-04

1.33E-04

3.92E-05

0.0001

11.2

10.5

8.2

2

0

2.00E-02

1.00E-02

1.00E-03

0.05

−1.8

−3.2

−7.8

2

2

7.36E-03

3.68E-03

3.68E-04

0.01

1.4

0.0

−4.6

2

6

9.96E-04

4.98E-04

4.98E-05

0.001

6.0

4.6

0.0

2

10

1.35E-04

6.74E-05

6.74E-06

0.0001

10.6

9.2

4.6

3

0

8.34E-02

3.16E-03

1.20E-04

0.05

−3.9

−6.0

−12.9

3

2

3.29E-03

1.24E-03

4.61E-05

0.01

−0.4

−2.5

−9.4

3

6

4.99E-04

1.85E-04

6.73E-06

0.001

4.5

2.5

−4.5

3

10

7.40E-05

2.73E-05

9.72E-07

0.0001

9.4

7.3

0.4

4

0

3.53E-03

1.02E-03

1.48E-05

0.05

−6.2

−8.9

−18.1

4

2

1.45E-03

4.12E-04

5.82E-06

0.01

−2.4

−5.1

−14.4

4

6

2.35E-04

6.58E-05

8.87E-07

0.001

2.8

0.0

−9.2

4

10

3.73E-05

1.02E-05

1.34E-07

0.0001

7.9

5.1

−4.1

p-value

Table 2: The third to sixth columns are LRT based p-value for the λBayes ∈ {0, 2, 6, 10}
specified in the second column for different values of n and ν. The eighth to 11th colums
are values of λBayes for the p-value specified in the seventh column for different values of
n and ν.

30

8

Conclusion

We have introduced a new class of priors we call cake priors having a number of desirable
properties. Cake priors can be made arbitrarily diffuse without the Bayes factor favouring
the null or alternative hypotheses as the prior becomes increasingly diffuse. In the limit,
at least for the examples we consider here, Bayes factors take the form of penalized
likelihood ratio statistics, one of the most thoroughly understood quantities in Statistics.
Due to their close link with Jeffreys priors, cake priors are parametrization invariant. The
resulting Bayesian test avoids the need to specify a p-value cut-off and are asymptotically
Chernoff-consistent. With a slight modification, an arbitrary but controllable constant
can be used to bridge the gap between Bayesian tests and likelihood ratio tests. Unlike
approaches that split the dataset up into parts or use imaginary data, cake priors are
transparent, uncomplicated, and easily implementable. Finally, Bayesian tests using cake
priors providing some protection against sequential testing being more conservative as the
sample size grows. We believe all of the above properties should make cake priors the
default choice when performing Bayesian hypothesis tests for hypothesis consisting of a
simple point null against a composite alternative for parametric models with iid data.

References
Abramowitz, M., Stegun, I. A., 1972. Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. New York: Dover Publications.
Aitkin, M., 1991. Posterior Bayes factors (with discussion). Journal of the Royal Statistical Society, Series B 53 (1), 111–142.
Bartlett, M. S., 1957. A comment on D.V. Lindley’s statistical paradox. Biometrika 44 (3),
533–534.
Bayarri, M. J., Berger, J. O., Forte, A., Garcı́a-Donato, G., 2012. Criteria for Bayesian
model choice with application to variable selection. The Annals of Statistics 40 (3),
1550–1577.
Berger, J. O., Pericchi, L. R., 1996. The intrinsic Bayes factor for model selection and
prediction. Journal of the American Statistical Association 91 (433), 109–122.
31

Berger, J. O., Pericchi, L. R., 2001. Objective Bayesian methods for model selection:
Introduction and comparison. IMS Lecture Notes - Monograph Series 38, 135–207.
Berger, J. O., Pericchi, L. R., Varshavsky, J. A., 1998. Bayes factors and marginal distributions in invariant situations. Sankhyā: The Indian Journal of Statistics, Series A
60 (3), 307–321.
Bernardo, J. M., 1980. A Bayesian analysis of classical hypothesis testing. Trabajos de
Estadistica Y de Investigacion Operativa 31 (1), 605–647.
Bernardo, J. M., 1999. Nested hypothesis testing: The Bayesian reference criterion.
Bayesian Statistics, 101–130.
Bové, D. S., Held, L., 2011. Hyper-g priors for generalized linear models. Bayesian Analysis 6 (3), 387–410.
Chen, M. H., Huang, L., Ibrahim, J. G., Kim, S., 2008. Bayesian variable selection and
computation for generalized linear models with conjugate priors. Bayesian Analysis
3 (3), 585–614.
Chen, M. H., Ibrahim, J. G., 2003. Conjugate priors for generalized linear models. Statistica Sinica 13 (2), 461–476.
DeGroot, M. H., 1973. Doing what comes naturally: Interpreting a tail area as a poterior
probability or as a likelihood ratio. Journal of the American Statistical Association 68,
966–969.
Fernández, C., Ley, E., Steel, M. F. J., 2001. Benchmark priors for Bayesian model
averaging. Journal of Econometrics 100 (2), 381 – 427.
Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., Rubin, D. B.,
2013. Bayesian Data Analysis, 3rd Edition. Chapman & Hall/CRC Texts in Statistical
Science. Taylor & Francis.
Gelman, A., Meng, X.-L., Stern, H., 1996. Posterior predictive assessment of model fitness
via realized discrepancies. Statistica Sinica 6 (4), 733–760.

32

George, E. I., McCulloch, R. E., 1993. Variable selection via Gibbs sampling. Journal of
the American Statistical Association 88 (423), 881–889.
Good, I. J., 1952. Rational decisions. Journal of the Royal Statistical Society, Series B
14 (1), 107–114.
Gupta, M., Ibrahim, J. G., 2009. An information matrix prior for Bayesian analysis in
generalized linear models with high dimensional data. Statistica Sinica 19 (4), 1641–
1663.
Hansen, M. H., Yu, B., 2001. Model selection and the principle of minimum description
length. Journal of the American Statistical Association 96 (454), 746–774.
Hanson, T. E., Branscum, A. J., Johnson, W. O., 2014. Informative g-priors for logistic
regression. Bayesian Analysis 9 (3), 597–612.
Inglot, T., 2010. Inequalities for quantiles of the chi-square distribution. Probability and
Mathematical Statistics 30 (2), 339–351.
Jahn, R. G., Dunne, B. J., Nelson, R. D., 1987. Engineering anomalies research. Journal
of Scientific Exploration 1 (1), 583–639.
Jeffreys, H., 1935. Some tests of significance, treated by the theory of probability. Mathematical Proceedings of the Cambridge Philosophical Society 31 (2), 203–222.
Jeffreys, H., 1946. An invariant form for the prior probability in estimation problems.
Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering
Sciences 186 (1007), 453–461.
Jeffreys, H., 1961. Theory of Probability. Vol. 2. Oxford University Press.
Kass, R. E., Raftery, A., 1995. Bayes Factors. Journal of the American Statistical Association 91 (6), 773–795.
Kass, R. E., Tierney, L., Kadane, J. B., 1990. The validity of posterior expansions based
on Laplace’s method. In: Geisser, S., Hodges, J. S., S.J. Press, S. J., Zellner, A. (Eds.),
Essays in Honor of George Bernard. North Holland, pp. 473–488.

33

Laurent, B., Massart, P., 10 2000. Adaptive estimation of a quadratic functional by model
selection. The Annals of Statistics 28 (5), 1302–1338.
Lehmann, E., 2004. Elements of Large-Sample Theory. Springer Texts in Statistics.
Springer New York.
Li, Y., Clyde, M. A., 2015. Mixture of g-priors in generalized linear models., arXiv
preprint arXiv:1503.06913.
Liang, F., Paulo, R., Molina, G., Clyde, M. A., Berger, J. O., 2008. Mixtures of gpriors for Bayesian variable selection. Journal of the American Statistical Association
103 (481), 410–423.
Lindley, D. V., 1957. A statistical paradox. Biometrika 44, 187–192.
Maruyama, Y., George, E. I., 2011. Fully Bayes factors with a generalized g-prior. The
Annals of Statistics 39 (5), 2740–2765.
Mitchell, T. J., Beauchamp, J. J., 1988. Bayesian variable selection in linear regression.
Journal of the American Statistical Association 83, 1023–1032.
Naaman, M., 2016. Almost sure hypothesis testing and the resolution of the JeffreysLindey paradox. Electronic Journal of Statistics 19, 1526–1550.
O’Hagan, A., 1995. Fractional Bayes factors for model comparison. Journal of the Royal
Statistical Society, Series B 57 (1), 99–138.
O’Hagan, A., 1997. Properties of intrinsic and fractional Bayes factors. Test 6, 101–118.
Pearson, J. W., Olver, S., Porter, M. A., 2017. Numerical methods for the computation of
the confluent and Gauss hypergeometric functions. Numerical Algorithms 74, 821–866.
Pettit, L. I., 1992. Bayes factors for outlier models using the device of imaginary observations. Journal of the American Statistical Association 87, 541–545.
Robert, C. P., 1993. A note on Jeffreys-Lindley paradox. Statistica Sinica 3, 601–608.
Robert, C. P., 2014. On the Jeffreys-Lindley paradox. Philosophy of Science 81 (2), 216–
232.
34

Schwarz, G., 1978. Estimating the dimension of a model. The Annals of Statistics 6 (2),
461–464.
Shao, J., 2003. Mathematical Statistics. Springer Texts in Statistics. Springer.
Smith, A. F. M., Spiegelhalter, D. J., 1980. Bayes factors and choice criteria for linear
models. Journal of the Royal Statistical Society, Series B 42 (2), 213–220.
Spanos, A., 2013. Who should be afraid of the Jeffreys-Lindley paradox? Philosophy of
Science 80 (1), 73–93.
Spiegelhalter, D. J., Best, N. G., Carlin, B. P., van der Linde, A., 2002. Bayesian measures
of model complexity and fit. Journal of the Royal Statistical Society, Series B 64 (4),
583–639.
Spiegelhalter, D. J., Smith, A. F. M., 1982. Bayes factors for linear and log-linear models
with vague prior information. Journal of the Royal Statistical Society, Series B 44 (3),
377–387.
Sprenger, J., 2013. Testing a precise null hypothesis: The case of Lindley’s paradox.
Philosophy of Science 80 (5), 733–744.
Tierney, L., Kass, R. E., Kadane, J. B., 1989. Fully exponential Laplace approximations to
expectations and variances of nonpositive functions. Journal of the American Statistical
Association 84 (407), 710–716.
van der Vaart, A. W., 1998. Asymptotic statistics, volume 3 of Cambridge Series in
Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge.
Vuong, Q. H., 1989. Likelihood ratio tests for model selection and nonnested hypotheses.
Econometrica 57 (2), 307–333.
Wang, X., George, E. I., 2007. Adaptive Bayesian criteria in variable selection for generalized linear models. Statistics Sinica 17, 667–690.
Wilks, S. S., 1938. The large-sample distribution of the likelihood ratio for testing composite hypotheses. The Annals of Mathematical Statistics 9 (1), 60–62.

35

Yang, Y., 2005. Can the strengths of AIC and BIC be shared? A conflict between model
indentification and regression estimation. Biometrika 92 (4), 937.
Zellner, A., 1986. On assessing prior distributions and Bayesian regression analysis with
g-prior distributions. Bayesian Inference and Decision Techniques: Essays in Honor of
Bruno de Finetti, 233–243.
Zellner, A., Siow, A., 1980. Posterior odds ratios for selected regression hypotheses. Trabajos de Estadistica Y de Investigacion Operativa 31 (1), 585–603.

36

