arXiv:1311.1067v2 [] 30 Mar 2016

Bernoulli 22(3), 2016, 1709–1728
DOI: 10.3150/15-BEJ708

Approximation of improper priors
CHRISTELE BIOCHE* and PIERRE DRUILHET**
1

Laboratoire de Mathématiques, UMR CNRS 6620, Clermont Université, Université Blaise
Pascal, 63177 Aubière Cedex, France. E-mail: * Christele.Bioche@math.univ-bpclermont.fr,
**
Pierre.Druilhet@univ-bpclermont.fr
We propose a convergence mode for positive Radon measures which allows a sequence of probability measures to have an improper limiting measure. We define a sequence of vague priors
as a sequence of probability measures that converges to an improper prior. We consider some
cases where vague priors have necessarily large variances and other cases where they have not.
We study the consequences of the convergence of prior distributions on the posterior analysis.
Then we give some constructions of vague priors that approximate the Haar measures or the
Jeffreys priors. We also revisit the Jeffreys–Lindley paradox.
Keywords: approximation of improper priors; conjugate priors; convergence of prior;
Jeffreys–Lindley paradox; non-informative priors; the Jeffreys prior; vague priors

1. Introduction
Improper priors such as flat priors (Laplace [21]), Jeffreys priors (Jeffreys [17]), reference
priors (Berger et al. [4]) or the Haar measures (Eaton [13]) are often used in Bayesian
analysis when no prior information is available. The posterior distribution is obtained
by applying the formal Bayes rule. There are several approaches to justify the use of
improper priors in statistics. Taraldsen and Lindqvist [26] explain how the theory of
conditional probability spaces developed by Rényi [23] is related to a theory for statistics
that includes improper priors. Their article is based on a generalization of Kolmogorov’s
theory to the σ-finite measures. They show in particular by examples that this theory
is different from the alternative theory of improper priors provided by Hartigan [14].
For many authors, the inference based on an improper prior Π is legitimated as limit of
inferences based on proper priors Πn . However, there are several ways to define this limit.
For example, Jeffreys [18], Stone [25], Bernardo and Smith ([6], Proposition 5.11), Jaynes
[16] consider the convergence, for any given observation x, of the posterior distributions
Πn (·|x) to Π(·|x) for some convergence mode such as total variation. Stone [24] consider a
convergence mode involving both the posterior distribution and the marginal distribution.
All these convergence modes are related to the statistical model through the likelihood.
Moreover, there is no standard convergence mode such that a sequence Πn of proper

This is an electronic reprint of the original article published by the ISI/BS in Bernoulli,
2016, Vol. 22, No. 3, 1709–1728. This reprint differs from the original in pagination and
typographic detail.
1350-7265

c

2016 ISI/BS

2

C. Bioche and P. Druilhet

priors may converge to an improper prior Π independently on the statistical model.
Consider, for example, a sequence of normal distributions N (0, n) with zero mean and
variance equal to n; it is often admitted that this sequence converges to the Laplace prior
since for many statistical models the Bayes estimate related to N (0, n) converges to the
Bayes estimate for the Laplace prior. A question then arises: does the limiting behaviour
of a sequence of proper priors depend on the statistical model? Is there any intrinsic
convergence mode?
The aim of this paper is to define a convergence mode on the set of prior distributions
without reference to any statistical model. In Section 2, we define this convergence mode.
We show that a sequence of vague priors is related to at most one improper prior. We also
show that any improper distribution can be approximated by proper distributions and
reciprocally. In Section 3, we give some conditions on the likelihood to derive convergence
of posterior distributions and Bayesian estimators from the convergence of prior distributions. In Section 4, we give some examples of construction of sequences of probability
measures which converge to improper priors such as the Haar measure or the Jeffreys
prior. In Section 6, we give a special interest in the convergence of Beta distributions. In
Section 7, we revisit the Jeffreys–Lindley paradox in the light of our convergence mode.

2. Definition, properties and examples of q-vague
convergence
Let X be a random variable and assume that X|θ ∼ Pθ , θ ∈ Θ. We assume that Θ is in
R, Rp with p > 1, or a countable set. In the Bayesian paradigm, a prior distribution Π is
given on Θ. In this article, we always assume that a prior Π is a positive Radon measure,
that is, a positive measure which is finite on compact sets. So, a prior may be proper or
improper. We denote by π the density function with respect to the Lebesgue measure in
the continuous case and the counting measure in the discrete case, or more generally to
some σ-finite measure. If Π is a probability measure, we can use the Bayes formula to
write the posterior density:
f (x|θ)π(θ)
,
f (x|θ)π(θ) dθ
Θ

π(θ|x) = R

(1)

where f (x|θ) is the likelihood function.
R
If Π is an improper measure but Θ f (x|θ)π(θ) dθ < +∞, we can formally apply the
Bayes formula to get a posterior distribution which will be proper. Now, if we replace Π
by αΠ, for α > 0, we obtain the same posterior distribution. So, in this case, the posterior
distribution is proper and
R independent of changes in the scaling of the prior. If Π is an
improper measure with Θ f (x|θ)π(θ) dθ = +∞, we cannot apply the Bayes formula. But
in this article, we allow posterior distribution to be improper, and in this case we will
define it by π(θ|x) = f (x|θ)π(θ) up to within a scalar factor.
We denote by CK (Θ) the space of real-valued continuous functions on Θ with compact
+
support and by CK
(Θ) the positive functions in CK (Θ). When there is no ambiguity on

Approximation of improper priors

3

+
the space, they will be simply denoted by CK or CK
. We also introduce the notation
Cb (Θ) for the space of bounded continuous functions on Θ, and C0 (Θ) for the space of
continuous functions g such that for all ε > 0, there exists
a compact K ⊂ Θ such that
R
h
dΠ
where h is a measurable
for all θ ∈ K c , g(θ) < ε. We use the notation
Π(h)
=
Θ
R
real-valued function, and |Π| = Π(1) = Θ dΠ, the total mass of Π.
We recall the two classic kinds of convergence of measures (Bauer [2]). A sequence
of probability measures {Πn }n converges narrowly (also said weakly) to a probability
measure Π if, for every function φ in Cb (Θ), {Πn (φ)}n converges to Π(φ). A sequence of
positive Radon measures {Πn }n converges vaguely to a positive Radon measure Π if, for
every function φ in CK (Θ), {Πn (φ)}n converges to Π(φ). We also recall a characterization
of vague convergence for a sequence of probability measures which will be useful later in
the article.

Lemma 2.1 (Billingsley [8], page 393). If {Πn }n is a sequence of probability measures and Π is a probability measure, then {Πn }n converges vaguely to Π iff for all
g ∈ C0 (Θ), {Πn (g)}n converges to Π(g).

2.1. Convergence of prior distribution sequences
In this section, we define a new convergence mode for sequences of positive Radon measures. The aim is to propose a formalization of an usual practice which consists of approximate an improper prior with a sequence of proper priors.
Definition 2.2. A sequence of positive Radon measures {Πn }n is said to converge qvaguely to a positive Radon measure Π if there exists a sequence of positive real numbers
{an }n such that {an Πn }n converges vaguely to Π.
Let us justify this definition. In equation (1), if we replace Π by αΠ, for α > 0, we
obtain the same posterior distribution, which means that the prior distribution is defined
up to within a scalar factor. So, it is natural to define the equivalence relation ∼ on the
space of positive Radon measures by
Π ∼ Π′

⇐⇒

∃α > 0

such that Π = αΠ′ .

(2)

Then it is natural to define the quotient space of positive Radon measures by the equivae
e=
> 0, Π
lence relation ∼. We denote by Π the equivalence class of Π, that is, Π = {Π/∃α
αΠ}. The q-vague convergence corresponds to the standard quotient topology on this
quotient space.
Remark 2.3. One referee pointed out that similar quotient spaces for σ-finite measures
were considered by Taraldsen and Lindqvist [27] to define conditional measures.
Proposition 2.4. Let {Πn }n and Π be positive Radon measures. The sequence {Πn }n
converges q-vaguely to Π iff {Πn }n converges to Π for the quotient topology.

4

C. Bioche and P. Druilhet

Proof. • Direct part: Assume that limn→∞ Πn = Π. The space of positive Radon measures is a metrisable space so it admits a countable neighbourhood base. Thus, there
exists a decreasing sequence of open sets
T {Oi }i∈N in the space of positive Radon measures such that for all i ∈ N, Π ∈ Oi and i∈N Oi = {Π}. So, for all i ∈ N, Π ∈ Oi . For any
Oi , there exists Ni such that for all n > Ni , Πn ∈ Oi . Without lost of generality, we can
choose Ni such that Ni > Ni−1 . For all n such that Ni ≤ n < Ni+1 , Πn ∈ C(Oi ) where
C(Oi ) = {λx with λ > 0 and x ∈ Oi }, that is, for all
T n such that Ni ≤ n < Ni+1 , there
exists an > 0 such that an Πn ∈ Oi . Moreover, since i∈N Oi = {Π}, limn→∞ an Πn = Π.
• Converse part: Assume that {an Πn }n converges to Π. Since the canonical mapping
φ defined by
φ : R → R/ ∼,

(3)

Π 7→ Π,

where R is the space of positive Radon measures, is continuous, {φ(an Πn )} = {Πn }
converges to φ(Π) = Π.

The following proposition shows that a sequence of prior measures cannot converge
q-vaguely to more than one limit up to within a scalar factor.
Theorem 2.5. Let {Πn }n be a sequence of priors such that {Πn }n converges q-vaguely
to both Πa and Πb , then necessarily there exists α > 0 such that Πa = αΠb .
Proof. This is a direct consequence of Proposition A.1 that states that R is a Hausdorff
space. However, we give here a direct proof that does not involve abstract topological
concept.
Assume that {Πn }n converges q-vaguely to both Πa and Πb . From Definition 2.2,
there exist two sequences of positive scalars {an }n and {bn }n such that {an Πn }n , respectively, {bn Πn }n , converges vaguely to Πa , respectively, Πb . We have to prove that
Πb = αΠa for some positive scalar α. Since Πa 6= 0 and Πb 6= 0, there exist ha and hb
+
in CK
such that Πa (ha ) > 0 and Πb (hb ) > 0. Put h0 = ha + hb ; we have Πa (h0 ) > 0
and Πb (h0 ) > 0. Moreover, limn→∞ an Πn (h0 ) = Πa (h0 ) and limn→∞ bn Πn (h0 ) = Πb (h0 ).
So, there exists N such that for n ≥ N , an Πn (h0 ) > 0 and bn Πn (h0 ) > 0. For any h
(h)
(h)
(h)
(h)
in CK and n > N , limn→∞ ΠΠnn(h
= limn→∞ aannΠΠnn(h
= ΠΠaa(h
and limn→∞ ΠΠnn(h
=
0)
0)
0)
0)
bn Πn (h)
Πb (h)
bn Πn (h0 ) = Πb (h0 ) .
Πa (h0 )
Πb (h0 ) Πb . The result

limn→∞

By uniqueness of the limit in R,

Πa =

follows.

Πa (h)
Πa (h0 )

=

Πb (h)
Πb (h0 ) .

Therefore,


Theorem 2.6 motivates to include the improper priors in the theory since it shows
these are obtained naturally from limits of proper priors. This can be compared with a
completion of a metric space.
Theorem 2.6. Any improper measure may be approximated by a sequence of probability
measures and conversely, any proper measure may be approximated by a sequence of
improper measures.

Approximation of improper priors

5

Proof. • Consider anSimproper measure Π and {Kn }n an increasing sequence of compacts such that Θ = n Kn . Then Πn = Π1Kn is a proper measure so, |Π1n | Πn is a
probability measure. Moreover, {Πn }n converges vaguely to Π, so { |Π1n | Πn } converges
q-vaguely to Π.
• Let Π be a probability measure. Consider the sequence Πn = Π + αn Π′ where Π′ is
an improper measure and {αn }n is a decreasing sequence which converges to 0. Then,
for all n ∈ N, Πn is an improper measure and {Πn }n converges q-vaguely to Π.

In many statistical models, there are several parameterizations of interest. We show
that the q-vague convergence is invariant by change of parameterization. Consider a new
e n = Πn ◦ h−1
parameterization η = h(θ) where h is a homeomorphism. We denote by Π
e = Π ◦ h−1 the prior distribution on η derived from the prior distribution on θ.
and Π
The following proposition establishes a link between q-vague convergence of {Πn }n and
e n }n .
{Π
Proposition 2.7. Let {Πn }n be a sequence of priors which converges q-vaguely to Π.
e n }n
Let h be a homeomorphism and consider the parameterization η = h(θ). Then {Π
e
converges q-vaguely to Π.

R
R
e n (η) and
Proof. From the change of variables formula, g(h(θ)) dΠn (θ) = g(η) dΠ
R
R
e
g(h(θ)) dΠ(θ) = g(η) dΠ(η). Moreover, if {Πn }n converges q-vaguely to Π, from Definition 2.2 there exists {an }n such that {an Πn }n Rconverges vaguely toR Π. Note that for all
g ∈ CK , g ◦ h ∈ CK . So, for all g ∈ CK , limn→∞ an g(h(θ)) dΠn (θ) = g(h(θ)) dΠ(θ), that
R
R
e n (η) = g(η) dΠ(η).
e
e n }n converges q-vaguely to Π.
e 
Thus, {Π
is, limn→∞ an g(η) dΠ

2.2. Convergence when approximants are probabilities
In this section, the sequence of approximants {Πn }n is assumed to be a sequence of
probability measures. Then we can establish some links between q-vague and narrow
convergence.
Indeed, if {Πn }n is a sequence of probabilities and Θ is a compact set, q-vague convergence is equivalent to narrow convergence.
More generally, we give a necessary and sufficient condition for the narrow convergence
of a sequence of probabilities which converges q-vaguely to a probability. We recall that
a sequence of bounded measures {Πn }n is said to be tight if, for each ε > 0, there exists
a compact set K such that, for all n, Πn (K c ) < ε.
Proposition 2.8. Let {Πn }n and Π be probability measures such that {Πn }n converges
q-vaguely to Π. Then {Πn }n converges narrowly to Π iff {Πn }n is tight.
Proof. Direct part: {Πn }n converges narrowly to Π a probability measure so {Πn }n is
tight.

6

C. Bioche and P. Druilhet

Converse part: Let us show that if {Πnk }k is any subsequence of {Πn }n which converges
narrowly then {Πnk }k converges to Π. From Billingsley ([8], Theorem 25.10), there exists
a subsequence {Πnk }k of {Πn }n which converges narrowly to some probability measure,
e Since {Πn }k is a sequence of probabilities which converges narrowly to Π,
e from
say Π.
k
e
Definition 2.2, {Πnk }k converges q-vaguely to Π. So, from Theorem 2.5, there exists
e but Π and Π
e are probabilities. So Π = Π.
e The result follows
α > 0 such that Π = αΠ,

from Billingsley ([8], Corollary of Theorem 25.10, page 346).
Now, we also assume that the limiting measure Π is an improper measure. Then we
can give a result about the sequence {an }n which will be useful thereafter.
Lemma 2.9. Let {Πn }n be a sequence of probability measures and {an }n a sequence
of positive scalars such that {an Πn }n converges vaguely to Π. If Π is improper, then
necessarily limn→∞ an = +∞.
Proof. We assume that {an Πn }n converges vaguely to Π so, we have Π(Θ) ≤
lim inf n an Πn (Θ) (see Bauer [2], Theorem 30.3). But for all n ∈ N, Πn (Θ) = 1 so
Π(Θ) ≤ lim inf an . Moreover, Π(Θ) = +∞ so lim inf n an = +∞. The result follows.

Lemma 2.10 (Lang [20], page 38). Let E be R, Rp with p > 1 or a countable set,
◦
S
for all compact K0 ⊂ ( n>0 Kn ) = E, there exists a function h ∈ CK (E) such that 1K0 ≤
h ≤ 1.
When a sequence of proper priors is used to approximate an improper prior, the mass
tends to concentrate outside any compact set.
Proposition 2.11. Let {Πn }n be a sequence of probability measures which converges
q-vaguely to an improper prior Π. Then, for any compact K in Θ, limn→∞ Πn (K) = 0,
and consequently, limn→∞ Πn (K c ) = 1.
Proof. From Definition 2.2, there exists {an }n such that limn→∞ an Πn (h) = Π(h)
for any h in CK . From Lemma 2.9, limn→∞ an = +∞ whereas Π(h) < +∞, so
limn→∞ Πn (h) = 0. Let K0 be a compact set in Θ. From Lemma 2.10, there exists a
function h ∈ CK such that 1K0 ≤ h. So Πn (K0 ) ≤ Πn (h) and limn→∞ Πn (K0 ) = 0. Since
Πn (K0 ) + Πn (K0c ) = 1 for all n ∈ N, thus limn→∞ Πn (K0c ) = 1.

Many authors consider that few knowledge on the parameter is represented by priors
with large variance. Here, we establish some links between the q-vague convergence of
priors and the convergence of the sequence of corresponding variances.
Proposition 2.12. Let {Πn }n be a sequence of probabilities on R such that EΠn (θ) is a
constant. If {Πn }n converges q-vaguely to an improper prior Π whose support is R, then
limn→∞ VarΠn (θ) = +∞.

Approximation of improper priors

7

2
Proof. Since EΠn (θ) is constant, lim
Πn (θ) = +∞ iff limn→∞ EΠn (θ ) = +∞.
R n→∞ Var
2
2
2
2
For any r > 0, we have EΠn (θ ) ≥ [−r,r]c θ dΠn (θ) so EΠn (θ ) ≥ r Πn ([−r, r]c ). From
Proposition 2.11, limn→∞ Πn ([−r, r]c ) = 1 and then limn→∞ EΠn (θ2 ) ≥ r2 . Since this

holds for any r > 0, limn→∞ EΠn (θ2 ) = +∞.

Corollary 2.13. Let {Πn }n be a sequence of probabilities with constant mean which
approximate the Lebesgue measure λR . Then, necessarily, limn→∞ VarΠn (θ) = +∞.
However, we will see in the examples in Section 5.4.1 that when we do not assume the
expectation to be constant; the variance does not necessarily diverge.

2.3. Characterization of q-vague convergence
In this section, we establish several sufficient conditions for the q-vague convergence of
{Πn }n to Π through their probability density function (p.d.f.). When Θ is continuous,
then πn and π are the standard p.d.f. with respect to the Lebesgue measure. When Θ
is discrete, then π(θ0 ) = Π(θ = θ0 ), that is, π is the p.d.f. with respect to the counting
measure.
When Θ = {θi }i∈I is a discrete set with I ⊂ N, we give an easy-to-check characterization of the q-vague convergence.
Proposition 2.14. Let {Πn }n and Π be priors on Θ = {θi }i∈I , I ⊂ N. The sequence
{Πn }n converges q-vaguely to Π iff there exists a sequence of positive real numbers {an }n
such that for all i ∈ I, limn→∞ an πn (θi ) = π(θi ).
Proof. It is a direct consequence of Definition 2.2 applied to the discreet case.



Now, we consider the continuous case.
Proposition 2.15. Let {Πn }n and Π be continuous priors on Θ in R or Rp with p > 1.
Assume that:
(1) there exists a sequence of positive real numbers {an }n such that the sequence
{an πn }n converges pointwise to π,
(2) there exists a continuous function g : Θ → R+ and N ∈ N such that for all n > N
and θ ∈ Θ, an πn (θ) < g(θ).
Then {Πn }n converges q-vaguely to Π.
Proof. Let h be in CK (Θ). Then, an h(θ)πn (θ) ≤ khkg1K (θ) where khk = maxθ∈Θ h(θ).
Since khkg1
integrable, by dominated convergence theorem,
K (θ) is Lebesgue
R
R

limn→∞ an πn (θ)h(θ) dθ = π(θ)h(θ) dθ.
The following result will be useful to establish a result in Section 4.2.

8

C. Bioche and P. Druilhet

Proposition 2.16. Let {Πn }n and Π be priors. Assume that:
(1) there exists a sequence of positive real numbers {an }n such that the sequence
{an πn }n converges pointwise to π,
(2′ ) for any compact set K, there exists a scalar M and some N ∈ N such that for
n > N , supθ∈K an πn (θ) < M .
Then {Πn }n converges q-vaguely to Π.
Proof. The proof is similar to the proof of Proposition 2.15 with an πn (θ)h(θ) ≤
M supθ∈K |h(θ)|1K (θ).

Remark 2.17. Proposition 2.15 and Proposition 2.16 hold if π(θ) is the p.d.f. with
respect to any positive Radon measure.

3. Convergence of posterior distributions and
estimators
Consider the model X|θ ∼ Pθ , θ ∈ Θ. We denote by f (x|θ) theR likelihood. The priors Πn
on Θ represent our prior knowledge. We always assume that Θ f (x|θ) dΠ(θ) > 0.
For a measure
Π and a measurable function g, we define the measure gΠ by gΠ(f ) =
R
Π(gf ) = f (θ)g(θ) dΠ(θ) for any f whenever the integrals are defined; gΠ is also denoted
g dΠ or Π ◦ g −1 by some authors.
In this paper, we define the posterior on θ, Π(·|x), by π(θ|x) ∝ f (x|θ)π(θ). Thus, the
posterior Π(·|x) may be proper or improper. There are three possible cases. First, if we use
a proper prior, by applying the Bayes formula, we obtain aR posterior which is a probability measure. If the prior is an improper measure such that Θ f (x|θ)π(θ) dθ < +∞, we can
formally apply the Bayes rule, which provides a posterior probability
measure by renorR
malization. At last, if the prior is an improper measure such that Θ f (x|θ)π(θ) dθ = +∞,
the posterior is an improper measure defined by π(θ|x) = f (x|θ)π(θ) up to within a scalar
factor.
In this section, we study the consequences of the q-vague convergence of {Πn }n on the
posterior analysis. In the general case where the posteriors may be proper or improper,
we give a result about the q-vague convergence of posteriors {Πn (·|x)}n to Π(·|x). When
posteriors are probability measures, we can establish results about the narrow convergence instead of the q-vague convergence.
Proposition 3.1. Let {Πn }n be a sequence of priors which converges q-vaguely to Π.
Assume that, θ 7−→ f (x|θ) is a non-zero continuous function on Θ. Then {Πn (·|x)}n
converges q-vaguely to Π(·|x).
Moreover, if {Πn (·|x)}n is a tight sequence of probabilities and Π(·|x) is a probability,
then {Πn (·|x)}n converges narrowly to Π(·|x).

Approximation of improper priors

9

Proof. Assume that {Πn }n converges q-vaguely to Π. From Definition 2.2, there exists a
sequence of positive scalars {an }n such that {an Πn }n converges vaguely to Π. So, for any
h ∈ CK , limn→∞ an Πn (h) = Π(h). Since f (x|·) is a continuous function, f (x|·)h ∈ CK and
limn→∞ an Πn (f (x|·)h) = Π(f (x|·)h). But Πn (f (x|·)h) = f (x|·)Πn (h) and Π(f (x|·)h) =
f (x|·)Π(h). So, {an f (x|·)Πn } converges vaguely to f (x|·)Π, or equivalently {f (x|·)Πn }n
converges q-vaguely to f (x|·)Π.
If {Πn (·|x)}n is a tight sequence of probabilities and Π(·|x) is a probability, the second

result follows from Proposition 2.8.
Remark 3.2. If Θ is discrete, then f (x|θ) is necessary continuous for the discrete topology.
The following results are based on Proposition 3.1 with easier-to-check assumptions.
Corollary 3.3. Let {Πn }n and Π be priors. Assume that:

(1) there exists a sequence of positive real numbers {an }n such that the sequence
{an πn }n converges pointwise to π,
(2) {an πn (θ)}n is non-decreasing for all θ ∈ Θ,
(3) θ 7−→ f (x|θ) is continuous and positive,
(4) all the posteriors Πn (·|x) and Π(·|x) are proper.
Then, {Πn (·|x)}n converges narrowly to Π(·|x).
Proof. The sequence {an f πn }n is a non-decreasing
sequence of non-negative
functions.
R
R
a
f
(x|θ)π
(θ)
dθ
=
lim
a
By monotone
convergence
theorem,
lim
n
n
n→∞ n f (x|θ) ×
n→∞
R
πn (θ) dθ = f (x|θ)π(θ) dθ. So, {an Πn (f )}n converges to Π(f ) > 0. So there exists
N such that for all n > N , an ΠnS
(f ) ≥ 21 Π(f ). Consider {Km }m an increasing sec
quence of compact sets such that Km = Θ. The sequence {Km
}m decreases to ∅
c ) = 0. Thus, for all ε > 0, there exists M such that, for all m ≥ M ,
so limm→∞ Π(f 1Km

c ) ≤ ε. So, for all n > N ,
Π(f 1Km

2ε
Π(f ) .

c
f Πn (KM
)
Πn (f )

=

c
f an Πn (KM
)
an Πn (f )

≤

2an Πn (f 1K c )

The second inequality comes from assumption (3). Thus,
result follows from Proposition 2.8.

2Π(f 1K c )

≤
Π(f )
f Πn
{ Πn (f ) }n is
M

M

Π(f )

≤

tight. The


Corollary 3.4. Let {Πn }n and Π be priors. Assume that:
(1) there exists a sequence of positive real numbers {an }n such that the sequence
{an πn }n converges pointwise to π,
(2) there exists a continuous function g : Θ → R+ such that f g is Lebesgue integrable
and for all n ∈ N and θ ∈ Θ, an πn (θ) < g(θ),
(3) θ 7−→ f (x|θ) is continuous and positive,
(4) all the posteriors Πn (·|x) and Π(·|x) are proper.
Then {Πn (·|x)}n converges narrowly to Π(·|x).
Proof. From Proposition 2.15, assumptions (1) and (2) imply that {Πn }n converges
q-vaguely to Π. From assumption (2), for all n, an f (x|θ)πn (θ) ≤ f (x|θ)g(θ). Since f g is

10

C. Bioche and P. Druilhet

R
convergence theorem, limn→∞ an f (x|θ)πn (θ) dθ =
RLebesgue integrable, by dominated
R
limn→∞ an f (x|θ)πn (θ) dθ = f (x|θ)π(θ) dθ. Thus, {an Πn (f )}n converges to Π(f ) > 0
so there exists N such that for all n > N , an S
Πn (f ) ≥ 21 Π(f ). Consider {Km }m∈N an
c
increasing sequence of compact sets such that Km = Θ. The sequence {Km
}m∈N dec ) = 0. Thus, for all ε > 0, there exists M such that for all
creases to ∅ so limm→∞ λ(f g1Km
c ) ≤ ε. So, for all n > N ,
m ≥ M , λ(f g1Km

c
f an Πn (KM
)
an Πn (f )

≤

2an Πn (f 1K c )
M

Π(f )

≤

2λ(f g1K c )
M

Π(f )

≤

2ε
Π(f ) .

Thus, {Πn (·|x)}n is a tight sequence of probabilities. The result follows from Proposi
tion 3.1.
The following result will be useful to explain the Jeffreys–Lindley paradox (see Section 7).
Corollary 3.5. Consider a sequence of probabilities {Πn }n which converges vaguely to
the proper measure Π. Assume that:
(1) θ 7−→ f (x|θ) is continuous and non-negative,
(2) f (x|·) ∈ C0 (Θ).
Then {Πn (·|x)}n converges narrowly to Π(·|x).

Proof. Since the Πn and Π are proper measures and f (·|θ) is a p.d.f., Πn (·|x) and Π(·|x)
are probabilities. We assume that {Πn }n converges vaguely, and so q-vaguely, to Π and
that f satisfies (1). So, from Proposition 3.1, {Πn (·|x)}n converges q-vaguely to Π(·|x).
From Lemma 2.1, {Πn (f )}n converges to Π(f ). So, there exists N such that for n > N ,
)
exists a compact K
Πn (f ) > Π(f
2 . Moreover, from assumption (2), for all ε > 0, there
f Πn (K c )
(f 1K c )
2ε
c
such that for all θ ∈ K , f (θ|x) ≤ ε. Thus, for all n > N , Πn (f ) ≤ 2ΠnΠ(f
≤ Π(f
)
).
Thus, { ΠfnΠ(fn ) }n is tight. The result follows from Proposition 3.1.



Now, we establish some links between the q-vague convergence of {Πn }n and the
convergence of the Bayes estimates EΠn (θ|x).
Proposition 3.6. Let {Πn }n be a sequence of priors which converges q-vaguely to Π.
Assume that:
(1) θ 7−→ f (x|θ) is a non-zero continuous function on Θ,
(2) the family {Πn (·|x)}n is a family of probabilities uniformly integrable (see Billingsley [7], page 32).
Then limn→∞ EΠn (θ|x) = EΠ (θ|x).
Proof. From Proposition 3.1, {Πn (θ|x)}n converges q-vaguely to Π(θ|x). For all n,
Πn (·|x) and Π(·|x) are probability measures and {Πn (·|x)}n uniformly integrable implies that {Πn (·|x)}n is tight. So, from Proposition 3.1, {Πn (θ|x)}n converges narrowly
to Π(θ|x). The result follows from Billingsley ([7], Theorem 5.4).


Approximation of improper priors

11

We give an other version of Proposition 3.6 with a more restrictive but easier-to-check
condition than uniform integrability.
Corollary 3.7. Let {Πn }n be a sequence of priors which converges q-vaguely to
Π. Assume that θ 7−→ f (x|θ) is a non-zero continuous function on Θ, and that
{Πn (·|x)}n is a family of probabilities such that {VarΠn (θ|x)}n is bounded above. Then
limn→∞ EΠn (θ|x) = EΠ (θ|x).
Proof. This is a consequence of Billingsley ([7], page 32) and Proposition 3.6.



4. Some constructions of sequences of vague priors
In this section, we give some constructions of sequences of probability measures that
approximate a given improper prior such as the Haar measures or the Jeffreys prior. We
have shown in the proof of Proposition 2.6 that any improper prior may be approximated
by truncation. Here, we give other constructions for the Haar measure or the Jeffreys
prior.

4.1. Location and scale models
The parameter θ is said to be a location parameter if there exists a p.d.f. g such that
f (x|θ) = g(x − θ). For instance, it is the case when X|θ ∼ N (θ, σ 2 ) with known σ 2 . The
underlying group is (R, +) and the Haar measure λR is improper.
Proposition 4.1. Let Π be a continuous probability measure on R. Assume that the p.d.f.
π(θ) of Π with respect to the Lebesgue measure λR is bounded above by a continuous
and increasing function and is continuous at θ = 0 with π(0) > 0. We define Πn by
πn (θ) = n1 π( nθ ). Then, {Πn }n>0 converges q-vaguely to λR .
Proof. Put πn (θ) = n1 π( nθ ). Put an = n, then limn→∞ an πn (θ) = limn→∞ π( nθ ) = π(0) >
0 since π is continuous at 0. Moreover, π is bounded above by a continuous and increasing
function, so there exists g such that, for all θ ∈ R and for all n > 0, π( nθ ) ≤ g( nθ ) ≤ g(θ).
The result follows from Proposition 2.15.

We note that Hartigan [15] used a dual approach. He reduced the influence of the prior
by letting the conditional variance σ 2 reducing to 0. He arrived at similar conclusions.
He assumed that Π is locally uniform at 0, but it is equivalent to assuming that Π is
continuous and positive at 0. We replace his condition “π tail-bounded” by the condition
“π bounded”.
Remark 4.2. Proposition 4.1 holds with the assumption “π bounded” instead of “π
bounded above by a continuous and increasing function”.

12

C. Bioche and P. Druilhet

We now study the scale model. The strictly positive parameter σ is said to be a scale
parameter if f (x|σ) = σ1 g( σx ) where g is a p.d.f. If σ is a scale parameter, log(σ) is a
location parameter for log(X). Here, the concerned group is (R+ \ {0}, ×) and the Haar
measure σ1 λR+ \{0} is improper. The following proposition is the equivalent of Proposition 4.1 for the Haar measure on (R+ \ {0}, ×).
Corollary 4.3. Let Π be a continuous probability measure on R+ \ {0}. Assume that
the p.d.f. π(σ) of Π with respect to the Lebesgue measure λR+ \{0} is bounded above by a
continuous and increasing function and is continuous at σ = 1 with π(1) > 0. We define
Πn by πn (σ) = n1 σ 1/n−1 π(σ 1/n ). Then {Πn }n>0 converges q-vaguely to σ1 λR+ \{0} .
e(θ) = eθ π(eθ ) which is bounded above by
Proof. Put θ = log(σ). From Proposition 2.7, π
θ
θ
the continuous and increasing function e g(e ). The result follows from Proposition 4.1. 

4.2. Jeffreys conjugate priors (JCPs)
The Jeffreys prior is one of the most popular prior when no information is available, but
in many cases, is improper. Consider that the distribution X|θ belongs to an exponential
family, i.e., f (x|θ) = exp{θ · t(x) − φ(θ)}h(x), for some functions t(x), h(x) and φ(θ),
and θ ∈ Θ, where Θ is an open set in Rp , p ≥ 1, such that f (x|θ) is a well-defined p.d.f.
We assume that φ(θ) and Iθ (θ) are continuous. These conditions are satisfied if t(X) is
not concentrated on an hyperplane almost surely (see Barndorff-Nielsen [1]). Druilhet
and Pommeret [12] proposed a class of conjugate priors that aims to approximate the
Jeffreys prior and that is invariant with respect to smooth reparameterization. The notion
of approximation was defined only from an intuitive point of view. We can now give a
more rigorous approach by using the q-vague convergence.
Denote by π J (θ) = |Iθ (θ)|1/2 the p.d.f. of the Jeffreys prior with respect to the Lebesgue
measure, where θ is the natural parameter of the exponential family and Iθ (θ) is the
determinant of the Fisher information matrix. The JCPs are defined through their p.d.f.
with respect to the Lebesgue measure by
J
πα,β
(θ) ∝ exp{α.θ − βφ(θ)}|Iθ (θ)|

1/2

,

and for a smooth reparameterization θ → η by
J
πα,β
(η) ∝ exp{α.θ(η) − βφ(θ(η))}|Iη (η)|

1/2

.

Proposition 4.4. Let {(αn , βn )}n be a sequence of real numbers that converges to (0, 0).
Then, for the natural parameter θ or for any smooth reparameterization η, {ΠJαn ,βn }n
converges q-vaguely to ΠJ .
Proof. Choose {an }n such that an παJ n ,βn (θ) = exp{αn θ − βn φ(θ)}|Iθ (θ)|1/2 , which
converges pointwise to |Iθ (θ)|1/2 . Put γn = (αn , βn ) and ψ(θ) = (θ, −φ(θ)). We have

Approximation of improper priors

13

γn · ψ(θ) = αn θ − βn φ(θ). By Cauchy–Schwarz inequality, γn · ψ(θ) ≤ kγn kkψ(θ)k. Since
γn converges to (0, 0), there exists N such that, for n > N , kγn k < 1. Let K be a compact
set in Θ, by continuity of ψ(θ), since φ(θ) is continuous, and by continuity of Iθ (θ), there
exist M1 and M2 such that, for all θ ∈ K, kψ(θ)k < M1 and |Iθ (θ)|1/2 < M2 . Therefore,

an παJ n ,βn (θ) ≤ M2 exp{M1 }. The result follows from Proposition 2.16.
Even if we have the convergence to the Jeffreys prior, we have no guaranty that
ΠJαn ,βn is a proper prior and there is no general result to characterize this property
such as in Diaconis and Ylvisaker [11] for usual conjugate priors. For example, con2
λ
1/2
sider inverse Gaussian models with likelihood f (x; µ, λ) = ( 2πx
)1{x>0}
exp( −λ(x−µ)
3)
2µ2 x
where µ > 0 denotes the mean parameter and λ > 0 stands for the shape parameJ
ter. Considering the parameterization (ψ = µ1 , λ), the JCPs are given by πα,β
(ψ, λ) ∝
2

J
(ψ, λ)
e−(λ/2)(α1 ψ −2βψ+α2 ) ψ −1/2 λ(β−1)/2 . Druilhet and Pommeret [12] showed that πα,β
√
1
is proper iff α1 > 0, α2 > 0 and − 2 ≤ β < α1 α2 . So, we may consider the sequences
1
α1,n = α2,n = n1 and βn = 2n
. By Proposition 4.4, ΠJαn ,βn (ψ, λ) is therefore a sequence of
proper priors that converges q-vaguely to the Jeffreys prior ΠJ .
g
Remark 4.5. For any continuous function g on Θ, we can define πα,β
(θ) ∝ exp{α.θ −
g
βφ(θ)}g(θ) and π (θ) = g(θ). Similarly to Proposition 4.4, it can be shown that {Πgαn ,βn }
converges q-vaguely to Πg .

5. Some examples
In this section, we consider some usual distributions and we look at the q-vague limiting
measure.

5.1. Approximation of flat prior from Uniform distributions
5.1.1. The discrete case
Consider Θ = N, and Πn = U({0, 1, . . . , n}) the Uniform distribution on {0, . . . , n}. Then
{Πn }n converges q-vaguely to the counting measure.
1
Indeed, πn (θ) = n+1
1{0,1,...,n} (θ). Put an = n + 1, then, for θ ∈ N, limn→∞ an πn (θ) =
limn→∞ 1{0,1,...,n} (θ) = 1. The result follows from Proposition 2.14.
5.1.2. The continuous case
Let Θ = R, and Πn = U([−n, n]) the Uniform distribution on [−n, n]. Then {Πn }n converges q-vaguely to the Lebesgue measure λR .
It corresponds to a location model so the result follows from Proposition 4.1 with
Π = U([−1, 1]).

14

C. Bioche and P. Druilhet

5.2. Poisson distribution
Here is an example where a family of proper priors does not converge q-vaguely. Let Θ = N
θ
and Πn be the Poisson distribution with πn (θ) = exp(−n) nθ! . Assume that there exists
Π such that {Πn }n converges q-vaguely to Π. Then, from Proposition 2.14, there exists
a sequence {an }n such that for all θ ∈ Θ, limn→∞ an πn (θ) = π(θ). Consider θ0 ∈ Θ such
that π(θ0 ) > 0. There exists N such that, for all n > N , πn (θ0 ) > 0. Consider θ > θ0 ,
(θ)
π(θ)
(θ)
= θθ!0 ! nθ−θ0 and limn→∞ ππnn(θ
= π(θ
< +∞. On the other side,
for all n > N , ππnn(θ
0)
0)
0)
limn→∞ θθ!0 ! nθ−θ0 = +∞. This is a contradiction. So, there is no prior Π such that {Πn }n
converges q-vaguely to Π.

5.3. Normal distribution
Let Θ = R and Πn = N (0, n) the normal distribution with zero mean and variance equal
to n. Then {Πn }n converges q-vaguely to the Lebesgue measure on R.
√
2
1
e−θ /2n and π(θ) = 1. Put an = 2πn, n > 0. Then {an πn }n>0
Indeed, πn (θ) = √2πn
converges pointwise to 1. Moreover, for all n and all θ, an πn (θ) < 2. The result follows
from Proposition 2.15.
Remark 5.1. From Theorem 2.5, {N (0, n)}n>0 cannot converge to another limiting
measure than the Lebesgue measure (up to within a scalar factor).
More generally, it can be shown that the limiting measure is the same for {N (µn , n)}n
where {µn }n is a constant or a bounded sequence. So, we consider now the case where
limn→∞ µn = +∞ by taking µn = n.
Proposition 5.2. We have three cases for the convergence of N (n, σn2 ):
1. If limn→∞
2. If limn→∞
3. If limn→∞

n
2
σn
n
2
σn
n
2
σn

= +∞, then {N (n, σn2 )}n does not converge q-vaguely.
= c with 0 < c < ∞, then {N (n, σn2 )}n converges q-vaguely to ecθ dθ.
= 0, then {N (n, σn2 )}n converges q-vaguely to λR .

Proof. For all n > 0, we denote by Πn = N (n, σn2 ), and by πn the p.d.f. with respect to
2
1
exp(− (θ−n)
the Lebesgue measure, πn (θ) = √2πσ
2σ2 ).
n

θ2
exp(− 2σ
2
n

θn
2 )
σn

n

2

2

+
and π
e(θ) = en /(2σn ) π(θ). So {Πn }n converges q1. Put π
en (θ) =
e n }n converges q-vaguely. Assume that there exists Π
e such that {Π
e n }n
vaguely iff {Π
e
e
converges q-vaguely to Π. Then there exists a sequence {an }n such that {an Πn }n cone Since Π
e 6= 0, there exists an interval [A1 , A2 ] such that −∞ < A1 <
verges vaguely to Π.

e
A2 < +∞ and 0 < Π([A
1 , A2 ]) < +∞. Consider [B1 , B2 ] such that A2 < B1 < B2 < +∞.
θ2
There exists N such that for n > N , θ 7−→ − 2n
+ σθn2 is non-decreasing. For a such n,
n

Approximation of improper priors

15

e n ([B1 , B2 ]) ≥ (B2 − B1 ) exp(− B12 +
Π
2σ
n

A2 n
2 ).
σn

B1 n
2 )
σn

e n ([A1 , A2 ]) ≤ (A2 − A1 ) exp(− A22 +
and Π
2σ
n

e n ([B1 ,B2 ])
(B 2 −A2 )
n(B1 −A2 )
Π
B2 −B1
1 −A2 )
So Π
− 12σ2 2 ≥ n(B2σ
.
2
2
e n ([A1 ,A2 ]) ≥ A2 −A1 exp(C(n)) with C(n) =
σn
n
n
e n ([B1 ,B2 ])
e
e n ([B1 ,B2 ])
Π
Π([B
Π
1 ,B2 ])
< +∞. So, {Πn }n
limn→∞ Π
e n ([A1 ,A2 ]) = +∞ but limn→∞ Π
e n ([A1 ,A2 ]) = Π([A
e
1 ,A2 ])

Thus,
does not converge q-vaguely.
1
n2
θ2
2. Put an = √2πσ
exp(− 2σ
2 ). Then limn→∞ an πn (θ) = limn→∞ exp(− 2σ 2 +
n

n

cθ

e . Since

limn→∞ σn2
n

n

= c, there exists N such that for all n > N ,
2

n
2
σn

θn
2 )
σn

=

∈ [c − ε, c + ε]. So,

θ
θn
for all n > N , exp(− 2σ
2 + σ 2 ) ≤ exp((c + ε)θ) which is continuous. The result follows
n
n
from Proposition 2.15.
3. This is the same reasoning as point 2. with limn→∞ an πn (θ) = 1 and an πn (θ) ≤ 1 + ε
for all n > N and N large enough.


Example 5.3. Assume that X|θ ∼ N (θ, σ 2 ), σ 2 known, and put the prior Πn = N (0, n)
σ2 n
on θ. Then Πn (θ|x) = N ( σ2nx
+n , σ2 +n ). From Section 5.3, the two first hypotheses are
satisfied and {N (0, n)}n converges q-vaguely to the Lebesgue measure λR so here, Π = λR .
Moreover, θ 7−→ f (x|θ) is continuous and positive on Θ and Π(·|x) = N (x, σ 2 ) is proper.
σ2 n
2
So, from Theorem 3.3, {N ( σ2nx
+n , σ2 +n )}n converges narrowly to N (x, σ ).
Example 5.4. To continue Example 5.3, VarΠn (θ|x) =

σ2 n
σ2 +n

is bounded above by

2

σ and the other hypothesis of Proposition 3.7 have already been verified in Example 5.3. So, from Proposition 3.7, limn→∞ EΠn (θ|x) = EΠ (θ). Indeed, limn→∞ EΠn (θ) =
limn→∞ σ2nx
+n = x = EΠ (θ).

5.4. Gamma distribution
5.4.1. Approximation of Π = 1θ 1θ>0 dθ
Let Θ = R+ and Πn = γ(αn , βn ) the Gamma distributions with limn→∞ (αn , βn ) = (0, 0).
βn αn αn −1 −βn θ
n)
αn −1 −βn θ
θ
e
. Put an = Γ(α
e
and
We have πn (θ) = Γ(α
βn αn . Then an πn (θ) = θ
n)
1
1
{an πn (θ)}n converges to θ . Put g(θ) = θ 1]0,1] (θ) + 1]1,+∞[ (θ). The sequence {αn }n goes
to 0 so there exists N such that for all n > N , αn < 1. So, for n > N and for θ > 0,
an πn (θ) ≤ θαn −1 ≤ g(θ). Since g is a continuous function on R∗+ , from Proposition 2.15,
{Πn }n converges q-vaguely to 1θ dθ.
Recall that for θ ∼ γ(a, b), E(θ) = ab and Var(θ) = ba2 . We can see below that the same
convergence may be obtained with different convergences of the mean and variance.
• For Πn = γ( n1 , n1 ), EΠn (θ) = 1 for all n and limn→∞ VarΠn (θ) = limn→∞ n = +∞.
• For Πn = γ( n1 , √1n ), limn→∞ EΠn (θ) = limn→∞ √1n = 0 and limn→∞ VarΠn (θ) = 1 for
all n.
1
• For Πn = γ( n1 , n1/3
), limn→∞ EΠn (θ) = limn→∞ n−2/3 = 0 and limn→∞ VarΠn (θ) =
limn→∞ n−1/3 = 0.

16

C. Bioche and P. Druilhet

• For Πn = γ( n1 , n12 ), limn→∞ EΠn (θ) = limn→∞ n = +∞ and limn→∞ VarΠn (θ) =
limn→∞ n3 = +∞.
1
• For Πn = γ( n1 , n2/3
), limn→∞ EΠn (θ) = n−1/2 = 0 and limn→∞ VarΠn (θ) =
limn→∞ n1/3 = +∞.
More generally, if lim inf n EΠn (θ) > 0 then limn→∞ VarΠn (θ) = +∞, since VarΠn (θ) =
with limn→∞ βn = 0.

EΠn (θ)
βn

5.4.2. Approximation of Π = 1θ e−θ 1θ>0 dθ
Let us show that {γ(αn , 1)} converges q-vaguely to θ1 e−θ 1θ>0 dθ when {αn } goes to 0. Put
Πn = {γ(αn , 1)}. Then πn (θ) = Γ(α1 n ) θαn −1 e−θ 1θ>0 is the p.d.f. of Πn . Put an = Γ(αn ),
then an πn (θ) = θαn −1 e−θ 1θ>0 converges to π(θ) = 1θ e−θ 1θ>0 . Moreover, since {αn }n goes
to 0, there exists N such that for n > N , αn < 1. Put g(θ) = 1θ 1]0,1] (θ) + 1]1,+∞[ (θ). So,
for n > N and θ > 0, an πn (θ) ≤ θαn −1 ≤ g(θ). The function g is continuous so from
Proposition 2.15, {γ(αn , 1)}n converges q-vaguely to 1θ e−θ 1θ>0 dθ. Since limn→∞ αn = 0,
we necessarily have limn→∞ EΠn (θ) = 0 and limn→∞ VarΠn (θ) = 0.

6. Convergence of Beta distributions
We now consider a more complex example which often appears in literature; see, for
example, Tuyl et al. [28]. Let X represents the number of successes in N Bernoulli trials,
and θ be the probability of a success in a single trial. Since the Beta distribution and the
Binomial distribution form a conjugate pair, a common prior distribution on θ is β(α, α)
which have mean and median equal to 21 . Three “plausible” non-informative priors were
listed by Berger ([3], page 89): the Bayes–Laplace prior β(1, 1), the Jeffreys prior β( 12 , 12 )
1
and the improper Haldane prior, wrote down β(0, 0), whose density is πH (θ) = θ(1−θ)
with respect to the Lebesgue measure on ]0, 1[. If we want β(α, α) with large variance,
necessarily α must be close to 0. Thus, we choose β( n1 , n1 ). The density of Πn = β( n1 , n1 )
1
θ1/n−1 (1 − θ)1/n−1 .
with respect to the Lebesgue measure on ]0; 1[ is πn (θ) = B(1/n,1/n)
As mentioned, for example, by Bernardo [5] or Lane and Sudderth [19], there are two
possible limiting distributions for β( n1 , n1 ) when n goes to +∞. The first one is 21 (δ0 + δ1 )
which is the limiting measure given by the standard probability theory. The second one is
the Haldane prior ΠH which is deduced from the posterior distributions and estimators
(Lehmann and Casella [22]). We show that it depends on the space where θ lives. Choosing
]0, 1[ or [0, 1] does not matter for β( n1 , n1 ) but it matters for the limiting distributions.
We may note that the Haldane prior is a Radon measure on ]0, 1[ but not on [0, 1] and
that 21 (δ0 + δ1 ) is not defined on ]0, 1[.

6.1. Convergence on ]0, 1[
In this section, we study the convergences on ]0, 1[ of {β( n1 , n1 )}n>0 , of the sequence of
posteriors and of the sequence of estimators.

Approximation of improper priors

17

Put an = B( n1 , n1 ), then an πn (θ) = θ1/n−1 (1 − θ)1/n−1 converges to πH (θ) = [θ(1 − θ)]−1
and for any θ and n, an πn (θ) < 5. Therefore, from Theorem 2.15, {β( n1 , n1 )}n>0 converges
q-vaguely to ΠH .
Consider the sequence of posteriors. The sequence of priors {Πn }n converges q-vaguely
to ΠH and θ 7−→ f (x|θ) is continuous on Θ. Then, from Lemma 3.1:
• if x = 0, {Πn (θ|x)}n converges q-vaguely to the improper measures with p.d.f. π(θ) =
(1 − θ)N −1 θ−1 ,
• if x = N , {Πn (θ|x)}n converges q-vaguely to the improper measures with p.d.f.
π(θ) = θN −1 (1 − θ)−1 ,
• if 0 < x < N , {Πn (θ|x)}n converges q-vaguely to ΠH (θ|x) = β(x, N − x).
For 0 < x < N , β(x, N − x) is proper and θ 7−→ f (x|θ) is continuous and positive. So,
from Theorem 3.3, {Πn (θ|x)}n>0 converges narrowly to ΠH (θ|x) = β(x, N − x).
x
1+nx
which tend to N
. So:
Consider now the Bayes estimators EΠn (θ|x) = 2+nN
• If x = 0, limn→∞ EΠn (θ|x = 0) = 0 whereas EΠH (θ|x = 0) = N1 .
• If x = N , limn→∞ EΠn (θ|x = N ) = 1 whereas EΠH (θ|x = N ) = +∞.
x
• If 0 < x < N , limn→∞ EΠn (θ|x) = N
= EΠH (θ|x).
RFor x = 0 and x = N , ΠH (·|x) is an improper measure. In this case, EΠH (θ|x) =
Θ θ dΠH (θ|x).

6.2. Convergence on [0, 1]
In this section, we study the convergences on [0, 1] of {β( n1 , n1 )}n>0 , of the sequence of
posteriors and of the sequence of estimators.
For all n and for 0 < t < 1, Πn ([0, t[) + Πn ([t, 1 − t]) + Πn (]1 − t, 1]) = 1. But on ]0, 1[,
{β( n1 , n1 )}n>0 converges q-vaguely to the improper measure ΠH , so limn→∞ Πn ([t, 1 −
t]) = 0. Moreover, for all n, Πn ([0, t[) = Πn (]1 − t, 1]). Thus, for all 0 < t < 1,
limn→∞ Πn ([0, t[) = 21 . From Billingsley ([8], page 192), {β( n1 , n1 )}n>0 converges narrowly
to 12 (δ0 + δ1 ) = Π{0,1} . By Theorem 2.5, {β( n1 , n1 )}n>0 cannot converge to an other limit
such as the Haldane measure, which is not a Radon measure on [0, 1].
The limit of the posterior distributions can be deduced from the limit of the prior
distributions only for x = 0 and x = N .
• If x = 0, {Πn (θ|x = 0)} converges narrowly to Π{0,1} (θ|x = 0) = δ0 .
• If x = N , {Πn (θ|x = N )} converges narrowly to Π{0,1} (θ|x = N ) = δ1 .
• If 0 < x < N , {Πn (θ|x)} converges narrowly to β(x, N − x) whereas Π{0,1} (θ|x) does
not exist.
Similarly, the limit of the estimators can be deduced from the limit of the prior distributions only for x = 0 and x = N .
• If x = 0, limn→∞ EΠn (θ|x = 0) = 0 = EΠ{0,1} (θ|x = 0).
• If x = N , limn→∞ EΠn (θ|x = N ) = 1 = EΠ{0,1} (θ|x = N ).
x
whereas EΠ{0,1} (θ|x) does not exist.
• If 0 < x < N , limn→∞ EΠn (θ|x) = N

18

C. Bioche and P. Druilhet

7. The Jeffreys–Lindley paradox
Consider the standard Gaussian model X|θ ∼ N (θ, 1) and the point null hypothesis
H0 : θ = 0 tested against H1 : θ 6= 0. If we use the prior π(θ) = 12 1θ=0 + 12 1θ6=0 with
respect to the measure δ0 + λR , it corresponds to the mass 21 on H0 and the Laplace
√
2
prior on H1√
. The posterior probability of H0 is Π(θ = 0|x) = [1 + 2πex /2 ]−1 so Π(θ =
0|x) ≤ [1 + 2π]−1 ≈ 0.285 whatever the data are. An alternative is to use a sequence of
2
2
1
e−θ /(2n ) . With these
proper priors {Πn }n whose p.d.f. are πn (θ) = 12 1θ=0 + 21 1θ6=0 √2πn
q
1
(n2 x2 )/(2(1+n2 )) −1
] which converges to 1. This
priors, we have πn (θ = 0|x) = [1 + 1+n
2e
√
2
limit differs from the “non-informative” answer [1 + 2πex /2 ]−1 and is considered as a
paradox. In the light of the concept of q-vague convergence, this result is not paradoxal
since, as shown in Proposition 7.1, the sequence of priors { 12 δ0 + 21 N (0, n2 )}n converges
vaguely to 12 δ0 , and, the limiting posterior distribution corresponds to the posterior of
the limit of the prior distributions. The following proposition generalizes this example.
e n }n be a
Proposition 7.1. Consider a partition: Θ = Θ0 ∪ Θ1 where Θ0 = {θ0 }. Let {Π
e and
sequence of probabilities on Θ which converges q-vaguely to the improper measure Π
e
e
e
such that Πn (θ0 ) = Π(θ0 ) = 0. Put Πn = ρδθ0 + (1 − ρ)Πn where 0 < ρ < 1, then {Πn }n
converges vaguely to ρδθ0 .
Moreover, assume that θ 7−→ f (x|θ) is continuous and belongs to C0 . Then {Πn (·|x)}
converges narrowly to Π(·|x).
e n }n converges vaguely
Proof. From Definition 2.2, there exists {an }n such that {an Π
e n (g). But
e For g ∈ CK , Πn (g) = ρg(θ0 ) + (1 − ρ)Π
e n (g) = ρg(θ0 ) + 1−ρ an Π
to Π.
an
1−ρ
e n (g) = Π(g)
e
e n (g) = 0 since, from Lemma 2.9,
< ∞. So, limn→∞ an an Π
limn→∞ an Π
limn→∞ an = +∞. Thus, limn→∞ Πn (g) = ρg(θ0 ). The first result follows.
The second part is a direct consequence of Theorem 3.5.

In the Proposition 7.1, it is assumed that θ 7−→ f (x|θ) ∈ C0 (Θ). Now, we consider the
case where the limit of the likelihood f (x|θ) when θ is outside of any compact is not 0
but f (x|θ0 ). In that case, the limit of the posterior probabilities is the same as the limit
of the prior probabilities, as stated in the following proposition.
Proposition 7.2. Consider the same notations and assumptions of Proposition 7.1.
Moreover, assume that θ 7−→ f (x|θ) is continuous and such that for all ε > 0, there exists
a compact K such that for all θ ∈ K c , |f (x|θ) − f (x|θ0 )| ≤ ε. Then limn→∞ Πn (θ =
θ0 |x) = Π(θ = θ0 ) and limn→∞ Πn (θ 6= θ0 |x) = Π(θ 6= θ0 ).
Proof. By Bayes formula: Πn (θ = θ0 |x) =

ρf (x|θ0 )
R
e n (θ) .
ρf (x|θ0 )+(1−ρ) Θ f (x|θ) dΠ
c

But, for all

ε > 0, there exists a compact K such that, for all θ ∈ K , |f (x|θ) − f (x|θ0 )| ≤ ε. So
R
R
R
e n (θ), where:
e n (θ) + c f (x|θ) dΠ
e n (θ) =
f (x|θ) dΠ
f (x|θ) dΠ
K
K
Θ

Approximation of improper priors

19

R
e n (θ) ≤ (f (x|θ0 ) + ε)Π
e n (K c ). From Proposie n (K c ) ≤
f (x|θ) dΠ
• (f (x|θ0 ) − ε)Π
Kc
R
e n (θ) = f (x|θ0 ).
e n (K c ) = 1. So, limn→∞ c f (x|θ) dΠ
tion 2.11, limn→∞ Π
K
• There exists g ∈ CK (Θ) such that 0 ≤ g ≤ 1 and g1K = 1. For a such g,
Z
Z
1
e n (θ) = 0
e
g(θ)f (x|θ) dΠ
f (x|θ) dΠn (θ) ≤ lim
lim
an
n→∞ an
n→∞ K
Θ

R
R
e
e n (θ) = g(θ)f (x|θ) dΠ(θ)
< +∞ and limn→∞ an =
since limn→∞ an Θ g(θ)f (x|θ) dΠ
Θ
+∞ from Lemma 2.9.

Thus, limn→∞ Πn (θ = θ0 |x) =

ρf (x|θ0 )
ρf (x|θ0 )+(1−ρ)f (x|θ0 )

= ρ = Π(θ = θ0 ).



To illustrate this result in a more general case, we consider an example proposed by
Dauxois et al. [10]. They consider a model choice between P(m) the Poisson distribution,
B(N, m) the Binomial distribution and N B(N, m) the Negative Binomial distribution.
These models belong to the general framework of Natural Exponential Families (NEFs)
and are determined by their variance function V (m) = am2 + m where m is the mean
parameter. Thus, a null value for a relates to the Poisson NEF, a negative one to the
Binomial NEF and a positive one to the Negative Binomial NEF. The prior distribution
chosen on the parameter a is ΠK defined by

1


,
if a = 0,


3


1
1
ΠK (a) =
,
if ∈ {1, . . . , K},

3K
a



1

 1 ,
if − ∈ {n0 , . . . , n0 + K − 1},
3K
a

where K is an hyperparameter. Note that ΠK (a = 0) = ΠK (a > 0) = ΠK (a < 0) = 31 .
Dauxois et al. [10] showed that the sequence of posterior distributions does not converge
to δ0 as in the previous case but ΠK (a = 0|X = x), ΠK (a > 0|X = x) and ΠK (a < 0|X =
x) converge to the prior probabilities ΠK (a = 0), ΠK (a > 0) and ΠK (a < 0) whatever
the data are when K → +∞.

Appendix: Properties of the quotient space
Proposition A.1. R is a Hausdorff space.
Proof. This proof is based on two results of Bourbaki [9].
Step 1. R is a topological space and Γ = {σα : Π 7−→ αΠ, α ∈ R∗+ } is a homeomorphism
group of R. We consider the equivalence relation: Π ∼ Π′ iff there exists α > 0 such that
Π = αΠ′ , that is, there exists σα ∈ Γ such that Π = σα (Π′ ). So, from Bourbaki ([9],
Section I.31), ∼ is open.

20

C. Bioche and P. Druilhet

Step 2. Let us show that G = {(Π, αΠ), (Π, αΠ) ∈ R × R} which is the graph of ∼
is closed. Let {(Πn , αn Πn )}n>0 be a sequence in G such that limn→∞ (Πn , αn Πn ) =
(Π0 , Π′0 ). The aim is to show that (Π0 , Π′0 ) ∈ G, that is, (Π0 , Π′0 ) takes the form
(Π0 , α0 Π0 ) where α0 Π0 6= 0. Since Π0 6= 0, there exists f0 ∈ CK such that Π0 (f0 ) > 0.
Moreover, limn→∞ Πn (f0 ) = Π0 (f0 ) so there exists N such that for all n ≥ N , Πn (f0 ) > 0.
Π′ (f )
n Πn (f0 )
For all n ≥ N , limn→∞ αn = limn→∞ αΠ
= Π00 (f00 ) = α0 . Thus, for all f ∈ CK ,
n (f0 )
limn→∞ αn Πn (f ) = α0 Π0 (f ) and limn→∞ αn Πn (f ) = Π′0 (f ). Since R is a Hausdorff
space, α0 Π0 (f ) = Π′0 (f ). So, the graph of ∼, G, is closed. The result follows from Bourbaki ([9], Section I.55).


Acknowledgment
The authors are grateful to Professors C.P. Robert, J. Rousseau and S. Dachian for
helpful discussions. We acknowledge the comments from reviewers which resulted in an
improved paper.

References
[1] Barndorff-Nielsen, O. (1978). Information and Exponential Families in Statistical Theory. Chichester: Wiley. MR0489333
[2] Bauer, H. (2001). Measure and Integration Theory. De Gruyter Studies in Mathematics
26. Berlin: de Gruyter. MR1897176
[3] Berger, J.O. (1985). Statistical Decision Theory and Bayesian Analysis, 2nd ed. Springer
Series in Statistics. New York: Springer. MR0804611
[4] Berger, J.O., Bernardo, J.M. and Sun, D. (2009). The formal definition of reference
priors. Ann. Statist. 37 905–938. MR2502655
[5] Bernardo, J.-M. (1979). Reference posterior distributions for Bayesian inference. J. Roy.
Statist. Soc. Ser. B 41 113–147. MR0547240
[6] Bernardo, J.-M. and Smith, A. F. M. (1994). Bayesian Theory. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. Chichester:
Wiley. MR1309682
[7] Billingsley, P. (1968). Convergence of Probability Measures. New York: Wiley.
MR0233396
[8] Billingsley, P. (1986). Probability and Measure, 2nd ed. Wiley Series in Probability and
Mathematical Statistics: Probability and Mathematical Statistics. New York: Wiley.
MR0830424
[9] Bourbaki, N. (1971). Elements of Mathematics. General Topology. Part 1. Paris: Hermann.
[10] Dauxois, J.-Y., Druilhet, P. and Pommeret, D. (2006). A Bayesian choice between
Poisson, binomial and negative binomial models. TEST 15 423–432. MR2273734
[11] Diaconis, P. and Ylvisaker, D. (1979). Conjugate priors for exponential families. Ann.
Statist. 7 269–281. MR0520238
[12] Druilhet, P. and Pommeret, D. (2012). Invariant conjugate analysis for exponential
families. Bayesian Anal. 7 903–916. MR3000019

Approximation of improper priors

21

[13] Eaton, M.L. (1989). Group Invariance Applications in Statistics. NSF-CBMS Regional
Conference Series in Probability and Statistics 1. Hayward, CA: IMS. MR1089423
[14] Hartigan, J.A. (1983). Bayes Theory. Springer Series in Statistics. New York: Springer.
MR0715782
[15] Hartigan, J.A. (1996). Locally uniform prior distributions. Ann. Statist. 24 160–173.
MR1389885
[16] Jaynes, E.T. (2003). Probability Theory. Cambridge: Cambridge Univ. Press. MR1992316
[17] Jeffreys, H. (1946). An invariant form for the prior probability in estimation problems.
Proc. Roy. Soc. London. Ser. A. 186 453–461. MR0017504
[18] Jeffreys, H. (1961). Theory of Probability, 3rd ed. Oxford: Clarendon Press. MR0187257
[19] Lane, D.A. and Sudderth, W.D. (1983). Coherent and continuous inference. Ann. Statist.
11 114–120. MR0684869
[20] Lang, S. (1977). Analyse Reelle. Paris: InterEditions.
[21] Laplace, P.-S. (1995). Théorie Analytique des Probabilités. Vol. II. Paris: Éditions Jacques
Gabay. Livre II: Théorie générale des probabilités. [Book II: General probability theory], Suppléments. [Supplements], Reprint of the 1820 third edition (Book II) and of
the 1816, 1818, 1820 and 1825 originals (Supplements). MR1400403
[22] Lehmann, E.L. and Casella, G. (1998). Theory of Point Estimation, 2nd ed. Springer
Texts in Statistics. New York: Springer. MR1639875
[23] Rényi, A. (1970). Foundations of Probability. San Francisco, CA: Holden-Day. MR0264723
[24] Stone, M. (1963). The posterior t distribution. Ann. Math. Statist. 34 568–573.
MR0149586
[25] Stone, M. (1970). Necessary and sufficient condition for convergence in probability to
invariant posterior distributions. Ann. Math. Statist. 41 1349–1353. MR0266359
[26] Taraldsen, G. and Lindqvist, B.H. (2010). Improper priors are not improper. Amer.
Statist. 64 154–158. MR2757006
[27] Taraldsen, G. and Lindqvist, B.H. (2015). Improper priors and conditional probability
spaces. Comm. Statist. Theory Methods. To appear.
[28] Tuyl, F., Gerlach, R. and Mengersen, K. (2009). Posterior predictive arguments in
favor of the Bayes–Laplace prior as the consensus prior for binomial and multinomial
parameters. Bayesian Anal. 4 151–158. MR2486242
Received July 2014 and revised January 2015

