arXiv:1609.07942v1 [] 26 Sep 2016

On the bracketing entropy condition and
generalized empirical measures
Davit Varron
Université de Bourgogne-Franche-Comté, UMR CNRS 6623
Abstract: We prove a Donsker and a Glivenko–Cantelli theorem for sequences
of random discrete measures generalizing empirical measures. Those two results
hold under standard conditions upon bracketing numbers of the indexing class of
functions. As a byproduct, we derive a posterior consistency and a Bernstein–von
Mises theorem for the Dirichlet process prior, under the topology of total variation,
when the observation space is countable. We also obtain new information about
the Durst–Dudley–Borisov theorem.

1

Introduction

In this article we shall adopt the generic notation (for r ∈ [1, ∞])
n
o
ℓr := p ∈ RN , || p ||r < ∞ , where
X
|| p ||rr :=
| pi |r , for 1 ≤ r < ∞, and where
i∈N

|| p ||∞ := sup | pi | , writing p = (pi )i∈N .
i∈N



Let X, AX be a measurable space. We shall use the notation
Z
f dQ
Q(f ) :=

(1)

X

for a given signed measure Q in AX with ﬁnite total variation, and for each f ∈
1
L1 (Q). Any X-valued sequence y = (yi )i∈N
 combined with an element p ∈ ℓ
deﬁnes a signed discrete measure on X, AX - with ﬁnite total variation - through
the following formula:
X
Py,p :=
pi δyi .
(2)
i∈N

1

1
Now substitute y by a A⊗N
X measurable sequence Y = (Yi )i∈N , and p by a ℓ -valued
Borel random variable β = (βi )i∈N (both of them on a probability space (Ω, A, P)).
Then the composition map PY,β deﬁnes random signed measure in the following
sense: for any speciﬁed bounded Borel function f , the map

PY,β (f ) :

(3)

ω → PYn (ω),β n (ω) (f )


is Borel from Ω, A to R. In the sequel we shall continue to adopt the same
convention (1) for P (f ) when P is a random or non random measure, and we shall
extend it - when meaningful - to functions f that are not necessarily bounded.
In [13], Varron started the investigation on how well known results in empirical
processes theory (see, e.g., [2, 12] for monographs on the subject) could be carried
over sequences of random signed measures of the form PYn ,βn where, for each n, the
sequence (Yi,n )i∈N is independent and identically distributed given β n . He showed
that the uniform entropy numbers and the Koltchinskii–Pollard uniform entropy
integral - two crucial notions in empirical processes theory - both adapt very well
to that wider class of random measures, which not only encompasses the empirical
measure, but also discrete nonparametric Bayesian priors. The latter notion of
uniform entropy integral
can be brieﬂy deﬁned as follows for a class F of real Borel

functions on X, AX :
J(δ, F) :=

Z

δ

0

s

log



sup
Q probab.

N ǫ || F ||Q,2 , F, || · ||Q,2


dǫ, δ ∈ (0, ∞].

Here || · ||Q,2 stands for the L2 (Q) norm, N (ǫ, F, || · ||Q,2 ) denotes the minimal
number of || · ||Q,2 balls with radius ǫ needed to cover F, and F stands for the
minimal measurable envelope of the class F - see, e.g., [12, p. 85]. Note that F
can be simply taken as
n
o
F (y) := sup | f (y) |, f ∈ F , y ∈ X,

when F is countable or pointwise measurable - see §2.2 below. When J(∞, F) is
ﬁnite, Varron proved a Donsker theorem under natural asymptotic conditions upon
(β n , Yn )n≥1 . Those two asymptotic theorems (see [13, Theorems 1 and 2]) involve
processes of the form



X
Gn (f ) :=
βi,n f (Yi,n ) − E f (Yi,n ) | βi,n , f ∈ F,
(4)
i∈N

indexed by a class F of real Borel functions. A rigorous deﬁnition of Gn (·) is not
immediate and is therefore voluntarily postponed to §2.2.

2

While the uniform entropy has been celebrated as a very useful condition to prove
that a class F is Donsker or Glivenko–Cantelli, another condition turned out to
be very fruitful as well: bracketing entropy. The bracket Jf − , f + K between two
Borel functions f − and f + is deﬁned as the set of Borel functions f fulﬁlling
f − ≺ f ≺ f + , the symbol ≺ standing for the everywhere pointwise comparison
between real functions on X. Denoting by N[] (ǫ, F, || · ||Q,2 ) the minimal number
of brackets with || · ||Q,2 diameter less than ǫ needed to cover F, the Q bracketing
entropy of F is deﬁned as
Z δq
(5)
J[] (δ, F, Q) :=
log N[] (ǫ, F, || · ||Q,2 )dǫ, δ ∈ (0, ∞].
0

A naturally arising question is then: does bracketing entropy adapt with the same
efficiency to sequences of random measures such as in (3)? The answer provided in
the present article is: yes, but to a lesser extent. More restrictions upon the weights
are needed. First the βi,n have to be non negative, since the idea of bracketing relies
on the comparison principle
f − ≺ f ≺ f + ⇒ Q(f − ) ≤ Q(f ) ≤ Q(f + ),
when Q is a non negative measure. Second, when looking for a Donsker theorem,
|| β n ||∞ has to tend to zero fast enough to counterbalance a the possible growth
of || β n ||1 . The amount of compensation is directly linked to the moments of
F (Y1,n ), n ∈ N∗ .
Those two conditions were not required under the assumption that J(∞, F) is
ﬁnite (see [13, Theorems 1 and 2]). This diﬀerence can be explained by the fact
that the use of the Koltchinskii–Pollard entropy is intimately linked to that of
symmetrization, namely the study of
X
G0n (f ) :=
ǫi βi,n f (Yi,n ), f ∈ F,
i∈N

where the ǫi are symmetric Bernoulli (or Rademacher) random variables, independent of (Yn , β n ). By subgaussianity of Rademacher processes, the G0n (·) inherit
several properties of inﬁnite dimensional Gaussian analysis. In particular, Hilbert
spaces take a predominant role. This explains why the results in [13] hold under
conditions upon || β n ||2 and || β n ||4 . On the other hand, bracketing methods do
not rely on subgaussianity, but on a form of Bernstein’s inequality. The latter is
a tradeoﬀ between subgaussian and subexponential tails for sums of independent
random variables that are uniformly bounded. This roughly explains why || β n ||∞
- and its conjugate norm || β n ||1 - needs to be controlled. Such a diﬀerence of
extent between bracketing and uniform entropy was not visible on the empirical

3

process for the following simple reason: when taking βi,n ≡ n−1/2 for i ≤ n and
−1/2 . This equality makes the
βi,n ≡ 0 otherwise, one has || β n ||∞ ≡|| β n ||−1
1 = n
counterbalance between those two norms hardly visible in the proof of the bracketing Donsker theorem.
Various interesting classes admit a ﬁnite bracketing entropy - see, e.g., [12, Chapter
2.7]. In addition, several examples of posterior distributions in (discrete) Bayesian
nonparametrics have the form PYn ,βn , or at least exhibit a predominant term that
can be expressed as such - see [13, Section 3]. Hence our main results present an
interesting range of applications, which we will here illustrate through two examples. The ﬁrst one takes place in the framework of frequentist asymptotic analysis
of nonparametric Bayesian priors: for a countable observation space, we prove a
posterior consistency and a Bernstein–von Mises theorem for the Dirichlet process
prior, under the topology of total variation (see §4.1). Along the proof, we also
revisit the Durst–Dudley–Borisov theorem and we obtain additional information
about this phenomenon. Our second example of application is a Donsker theorem
- under a bracketing condition - for a speciﬁc form of local empirical measures (see
§4.2). The remainder of this article is organized as follows: in §2 we give a careful
description of the mathematical framework. Then our two main results are stated
in §3. Applications follow in §4. The proofs of those results are then written in §5.
Finally, the Appendix is dedicated to a minor proof.

2

The mathematical framework

In order to properly state our main results, we ﬁrst need to carefully deﬁne their
underlying probabilistic framework. This section may be skipped at ﬁrst reading.

2.1

The underlying probability space

Empirical processes carry over some lacks of measurability that are usually tackled
by using outer expectations - see, e.g., [12, Chapter 1.2]. In order to make use of
Fubini’s theorem - which is in general untrue for outer expectations - mathematical
rigor imposes to deﬁne the underlying probability space (Ω, A, P) as a suitable
product space. First, for ﬁxed n ≥ 1, consider a Markov transition kernel from
ℓ1 to X, i.e., a family {Pn,p , p ∈ ℓ1 } of probability measures
 for which the maps

p → Pn,p (A), A ∈ AX are measurable from ℓ1 , Bor(ℓ1 ) to [0, 1], Bor([0, 1]) .
Also consider a probability measure Qn on ℓ1 , Bor(ℓ1 ) and deﬁne:
Ω̃ := ℓ1 × XN , endowed with its product σ-algebra

Ã := Bor(ℓ1 ) ⊗ A⊗N
X , with probability law deﬁned through the generic formula:

4

Pn



(p, y) ∈ Ω̃, p ∈ A, ∀j ∈ {1, . . . , k}, yi ∈ Bj
∗

∗

Then deﬁne Ω := Ω̃N , A := ÃN , P :=
as coordinate maps on Ω :



N

n≥1 Pn

:=

Z

k
Y

Pn,p (Bj )dQn (p).

p∈A j=1

on A and deﬁne the Yn and β n

β n (p1 , y1 , p2 , y2 , . . .) := pn , and Yn (p1 , y1 , p2 , y2 , . . .) := yn .
Note that, for ﬁxed n and p ∈ ℓ1 , P⊗N
n,p is the law of Yn given β n = p. We shall
denote by Pn the law of Y1,n .
To simplify the notations we now adopt the following convention: each time a map h
is deﬁned on a probability space, the symbol E∗ (h) will denote the outer expectation
with respect to that probability space. We shall adopt the same convention for
outer probabilities P∗ .

2.2

Definition of Gn

From now on, and throughout all this article, we shall make the assumption that
Pn (F ) < ∞ for all n ≥ 1. We also assume that F is pointwise measurable with
countable separant F0 in the following sense: for any f ∈ F, there exists (fm )m≥1 ∈
∗
F0N such that fm (y) → f (y) for each y ∈ X. Such a very standard assumption will
be useful to tackle annoying measurability issues.
Because the symbol Σi∈N in (4) is ambiguous, we need to give a rigorous deﬁnition
of the processes that will be involved in this article. Our deﬁnition diﬀers from
that used in [13] for two reasons. The ﬁrst (minor) one is to cover the case where
the || β n ||1 are not deterministically equal to 1. The second one is for technical
purposes: in our proofs, we shall truncate the f ∈ F from above using thresholds
that depend upon the weights. First note that, for any bounded function f and
any Borel map T from ℓ1 to R+ , the map
X 

Φf,T : (y, p) →
pi f 1{F ≤T (p)} (yi ) − Pn,p f 1{F ≤T (p)} (yi )
i∈N

is properly deﬁned (through the limits in R of partial sums) and Borel from XN × ℓ1
to R. One can hence deﬁne a random variable GTn (f ) by composition GTn (f ) :=
Φf,T ◦(Yn , β n ). We will say that a map ψ from F to R is F0 -separable whenever we
have || ψ ||F =|| ψ ||F0 . We shall also denote by B(F, F0 ) the space of all bounded
F0 -separable functions and by A||·||F is the σ-algebra spanned by the || · ||F -balls.
Lemma 2.1 For any choice of T as above, the map

ΦF ,T : (y, p) → f → Φf,T (y, p)

is measurable from XN × ℓ1 to (B(F, F0 ), A||·||F ).

5

Proof : Fix T . Let us ﬁrst prove that ΦF ,T takes its values in B(F, F0 ). Fix
y ∈ X⊗N and p ∈ ℓ1 . Since, for all k ∈ N:
X h
X
i
sup
pi f 1{F ≤T (p)} (yi ) − Pn,p f 1{F ≤T (p)}
≤ 2T (p)
| pi |,
f ∈F

i≥k+1

i≥k+1


and since B(F, F0 ), || · ||F
is a Banach space, it is suﬃcient to prove
 that
each trajectory f → f 1{F ≤T (p)} (yi ), i ∈ N, and f → Pn,p f 1{F ≤T (p)} is F0 separable. To see this, take f ∈ F and consider (fm )m≥1 ∈ F0N such that fm →
f0 pointwise. Thus Pn,p (fm 1{F ≤T (p)} ) → Pn,p f 1{F ≤T (p)} by the dominated
convergence theorem. Now since ΦF ,T takes its values in B(F, F0 ), the following
equality holds for any (y, p) ∈ XN × ℓ1 :
sup | Φf,T (y, p) |= sup | Φf,T (y, p) |,

f ∈F

(6)

f ∈F0

and then the measurability of each Φf,T , f ∈ F0 ensures that of ΦTF with respect
to A||·||F . 


Now denote by E˜F ,F0 the space of all measurable maps from Ω, A to B(F, F0 ), A||·||F .
The preceding lemma gives the opportunity to deﬁne the processes Gn on any class
of functions

FM := f 1{F ≤M } , f ∈ F
since (confounding M with a constant function on ℓ1 ) the composition map
GM
n := ΦF ,M ◦ (Yn , β n )
belongs to ẼF ,F0 . Denote by EF ,F0 the quotient space of ẼF ,F0 with respect to the
equivalence class
G ∼ G′ ⇔|| G − G′ ||F = 0, P-a.s.,
and endow EF ,F0 with the compatible distance


d(G, G′ ) := E arctan || G − G′ ||F ,

(7)

which is that of || · ||F -convergence in probability. The following lemma deﬁnes
Gn as a suitable limit of the GM
n when M → ∞.
Lemma 2.2 For fixed n ≥ 1, the sequence (GM
n )M ≥1 is Cauchy in the complete
metric space EF ,F0 , d . It hence converges to a limit which we take as the definition of Gn . Moreover, for any sequence (Tk ) of Borel thresholding maps fulfilling
Tk (β n ) →P ∞ as k → ∞, we have d(GTnk , Gn ) → 0 as k → ∞.

6

′

Proof : For integers M, M ′ we have, writing f M,M := f 1{M <F ≤M ′}


M′
d GM
,
G
n
n
!

=E arctan sup Φf M,M ′ (Yn , β n )
f ∈F

X

≤E arctan

i∈N



| βi,n | F

M,M ′



M,M ′
(Yi,n ) β n
(Yi,n ) + E F

!!

.

(8)

Using Fatou’s lemma for conditional expectations and the concavity of arctan on
R+ we have, almost surely:
!

X


′
′
| βi,n | F M,M (Yi,n ) + E F M,M (Yi,n ) β n
βn
E arctan
i∈N

 X


′
≤ arctan 2
| βi,n | E F M,M (Yi,n ) | β n
i∈N



M,M ′
(Y1,n ) | β n
= arctan 2 || β n ||1 E F



≤ arctan 2 || β n ||1 E F 1{F >M } (Y1,n ) | β n ,


(9)
(10)

where (9) comes from the fact that the law Yn given β n = p is P⊗N
n,p . It hence
suﬃces to prove that the right hand side (RHS) of (10) tends to 0 in probability
as M → ∞. This is true since Pn (F ) < ∞. Now to prove the last statement of
Lemma 2.2, formally replace M by Tk (p) in the preceding calculus and let M ′ → ∞
to obtain, using Fatou’s lemma for conditional expectations:
!




Tk
,
(11)
d Gn , Gn ≤ E arctan 2 || β n ||1 E F 1{F >Tk (β n )} (Y1,n ) | β n
which tends to 0 as k → ∞, by assumption upon (Tk (β n ))k≥1 and since Pn (F ) <
∞. 

2.3

Definition of the limit processes

One of our results is a Donsker theorem (see Theorem 2 below). The limit processes
are mixtures of F-indexed Brownian bridges, for which a rigorous deﬁnition is not
immediate due to the non separability of ℓ∞ (F). We shall use the deﬁnition of
[13]. However, since the possibility to condition upon the weights was not made

7

perfectly clear in [13], we feel the need to give more precisions in its reminder. First
ﬁx n ≥ 1. For p ≥ 1, f := (f1 , . . . , fp ) ∈ F p and p ∈ ℓ1 , write µfn,p for the centered
Gaussian distribution on Rp with covariance matrix
i

h
Σfn,p := Pn,p (fj − Pn,p (fj ))(fj ′ − Pn,p (fj ′ ))
.
′
2
(j,j )∈{1,...,p}

Now consider Ω′ := RF , endowed with its product Borel σ-algebra A′ , i.e the
σ-algebra spanned by the π-system
o
n
A′0 := CAf1 ,...,Afk , k ≥ 1, f ∈ F k , Afj Borel for each j ∈ {1, . . . , k} , with
n
o
CAf1 ,...,Afk := ψ ∈ Ω′ , ∀j ∈ {1, . . . , k}, ψ(fj ) ∈ Afj .

For ﬁxed p ∈ ℓ1 , Kolmogorov’s extension theorem (see, e.g., [7, p. 115, Theorem

6.16]) ensures the existence of a unique probability measure P′n,p on Ω′ , A′ which
 f
is compatible with the system of marginals µn,p , f ∈ F p , p ≥ 1 , namely


P′n,p CAf1 ,...,Afk = µfn,p Af1 × . . . × Afk ,

for all elements of A′0 . Now since {Pn,p , p ∈ ℓ1 } deﬁnes a transition kernel, then so
does the family {µfn,p , p ∈ ℓ1 } for any speciﬁed f : it is a simple consequence of the
fact that p → Σfn,p is Borel. Consequently the map p → P′n,p (C) is measurable for
any speciﬁed C ∈ A′0 . That measurability property is then immediately extended
to all C ∈ A by Dynkin’s π-λ theorem - see, e.g., [7, p. 2, Theorem 1.1]. As

a consequence, the family {P′n,p , p ∈ ℓ1 } deﬁnes a transition kernel on Ω′ , A′ .
Finally, we take Ω′′ := ℓ1 × Ω′ , endowed with Bor(ℓ1 ) ⊗ A′ , and with probability
P′′n deﬁned by
Z
P′n (A × C) :=

p∈A

P′n,p (C)dQn (p).

as the completion of Bor(ℓ1 ) ⊗ A′ with respect
to P′′n and we
We then take

deﬁne Wn as the coordinate map (p, ψ) → ψ on Ω′′ , A′′n , P′′n .
A′′n

Lemma 2.3 Let n ≥ 1 be an integer. Assume that, P-almost surely:


J[] ∞, F, || · ||Pn,β n ,2 < ∞.

Then Wn is P′′n -almost surely bounded.


Proof: Write β ′n as the canonical map (p, ψ) → p on Ω′′ , A′′n , P′′n and note that
β ′n and β n are equal in law. For any ﬁnite subclass {f1 , . . . , fp } ⊂ F we have,

8

by conditioning on β ′n and then using Dudley’s chaining theorem (see, e.g., [12, p.
101, Corollary 2.2.8])


Z +∞ q

E max | Wn (fj ) | β ′n ≤C0
log N ǫ, F, || · ||Pn,β′ ,2 dǫ
n
j≤p
0
Z +∞ q

log N[] ǫ, F, || · ||Pn,β ′ ,2 dǫ
≤C0
n
0


=C0 J[] ∞, F, Pn,β ′n , P′′n -almost surely,
where C0 is a universal constant. It follows by Fatou’s lemma for conditional
expectations that




′
E sup | Wn (f ) | β n ≤ C0 J[] ∞, F, Pn,β ′n , P′′n -almost surely.
f ∈F0

Now using the same arguments as those used to obtain [13, p. 2314, assertion (49)],
one can show that


P′′n ω ∈ Ω′′ , Wn (ω) is not F0 separable = 0,

which concludes the proof.

3

Results

Before stating our two
 main results, let us brieﬂy mention that the maps p →
N[] ǫ, F, || · ||Pn,p ,r and p → J[] δ, F, Pn,p are properly measurable for ﬁxed ǫ
and δ. This is proved in §6.

3.1

A Glivenko–Cantelli theorem

Our ﬁrst result is a Glivenko–Cantelli theorem. Recall that Pn is the law of Y1,n .
We shall denote by ℓ1,+ := ℓ1 ∩[0, ∞[N the set of non negative summable sequences.
Theorem 1 Assume that
lim



lim Pn F 1{F ≥M } = 0,

M →∞ n→∞

and that, for any ǫ > 0:



N[] ǫ, F, || · ||Pn,β n ,1

is bounded in probability.
n≥1

9

(12)

(13)

Also assume that β n ∈ ℓ1,+ is almost surely for all n, and that

|| β n ||1 n≥1 is bounded in probability.

(14)

Then, under the condition || β n ||2 →P 0 we have || Gn ||F →P 0.
Remark: It is important to compare the assumptions of Theorem 1 to those
of Dudley’s bracketing Glivenko–Cantelli theorem [12, p. 122, Theorem 2.4.1] for
Emp
the empirical measure PYn ,βEmp , where βi,n
:≡ n−1 for i ≤ n and is identically
n
null otherwise, and where Yn is constant in n - hence with constant law Pn = P0 .
A few easy arguments then show that, in this special case, those two theorems
exactly coincide: ﬁrst, for β n = β Emp
the convergence in probability is equivalent
n
to an almost sure convergence by Pollard’s reverse martingale argument (see, e.g.,
[12, p. 124, Lemma 2.4.5]). Second, (12)+(13) is here equivalent to the ﬁniteness
of N[] ǫ, F, || · ||P0 ,1 for each ǫ > 0.

3.2

A Donsker theorem

For a sequence Zn of maps from Ω to R we shall write
n
o
P∗
lim Zn := inf M ∈ R, lim P∗ (Zn ≥ M ) = 0 ,
n→∞

n→∞

P

with the convention inf ∅ = +∞, and we shall simply write limn→∞ Zn when the
maps Zn are measurable. Our second result is a Donsker theorem.
Theorem 2 Assume that
|| β n ||2 →P 1,

(15)

|| β n ||∞ →P 0,

(16)

|| β n ||1 × || β n ||p−1
is bounded in probability,
∞

(17)

lim lim J[] (δ, F, Pn,β n ) = 0,

lim lim Pn F p 1{F >M } = 0.

(18)

and that, for some p ∈ [2, ∞[
P

δ→0 n→∞

M →∞ n→∞

(19)

Also assume that there exists a semimetric ρ that makes F totally bounded, and
fulfilling


P∗
(20)
lim lim
sup
Pn,βn (f1 − f2 )2 = 0.
δ→0 n→∞

(f1 −f2 )∈F 2 ,
ρ(f1 ,f2 )<δ

10

Then



dBL Gn , Wn := sup

B∈BL1





→ 0,
E∗ B Gn − E∗ B Wn

(21)


that are
where BL1 is the set of all 1-Lipschitz functions on ℓ∞ (F), || · ||F
bounded by 1.
Moreover, if F is uniformly bounded, then (21) holds without assuming (17) nor
(19).
Remarks: We chose to state Theorem 2 under the most general assumptions that
our methodology can aﬀord. In order to give more substance to those conditions,
it seems convenient to discuss on the place of Theorem 2 in the existing literature
on Donsker theorems for empirical processes.
1. When β n is the vector of rescaled empirical weights (βi,n ≡ n−1/2 for i ≤ n
and βi,n ≡ 0 otherwise), and when Pn = P0 is constant in n, then PYn ,βn is
a sequence of empirical processes. Noting that - for p = 2 - the β n obviously
satisfy conditions (15), (16) and (17) one can immediately conclude that - in
this setup - Theorem 2 exactly coincides with Ossiander’s bracketing Donsker
theorem [8, Theorem 3.1]. Andersen et al. [1] did also prove a Donsker
theorem under more general conditions, where the ﬁniteness of J[] (∞, F, P0 )
is relaxed a to more abstract assumption, involving majorizing measures on
|| · ||P0 ,2 balls and "weak || · ||P0 ,2 " brackets. This possible extension of
Theorem 2 is beyond the scope of the present article and may deserve future
investigations.
2. Let us now relax the assumption that Pn is constant in n. In that case
the Gn fall into the framework of triangular arrays of empirical processes
with varying baseline measures, which were studied by Sheehy and Wellner
[10, Section 3]. These authors did prove a Donsker result for Gn indexed
by classes fulﬁlling J(∞, F) < ∞, under the envelope condition (19), and
assuming that Pn converges to a limit P0 in the following sense - see their
Corollary 3.1:
n


sup max | Pn (f1 − f2 )2 − P0 (f1 − f2 )2 |, | Pn (f1 ) − P0 (f1 ) |,
(f1 ,f2 )∈F 2

| Pn (f12 ) − P0 (f12 ) |

o

→ 0.

(22)

It is then clear that our Theorem 2 puts forward an analogue of their result, replacing their assumption J(∞, F) < ∞ by the bracketing condition
(18). To see this, just note that (20) is satisﬁed under (22), by choosing
ρ(f1 , f2 ) :=|| f1 − f2 ||P0 ,2 .

11

3. Let us now discuss on assumption (18), which might be the most cumbersome
to verify for applications. If J[] (∞, F, P0 ) < ∞, a simple way to check (18)
- by direct comparison of bracketing numbers - is to prove that

Pn,βn (f1 − f2 )2
 is bounded in probability.
sup
(23)
2
(f1 ,f2 )∈F 2 P0 (f1 − f2 )

Such a suﬃcient condition is quite restrictive and seems far from necessary,
but its veriﬁcation is sometimes very simple to perform. This is for example
the case for our application to the local empirical process at ﬁxed point - see
§4.2.

4. We conclude this series of remarks by pointing out that, whereas involving
random weights, Theorem 2 has almost no connections with Donsker theorems for bootstrap empirical measures. For more details see [13, Remark
2.2].

4
4.1

Applications
Posterior analysis of the Dirichlet process prior under the discrete total variation

Assume (in this subsection only) that X is inﬁnite countable. The class F of all
indicator functions of subsets of X:
n
o
F := 1C , C ⊂ X
(24)

is rich enough to deﬁne the discrete total variation between two measures on X,
since
|| Q − Q′ ||F = sup | Q(C) − Q′ (C) |=:|| Q − Q′ ||T ot.var. .
C⊂X

Clearly, F is too large to satisfy J(∞, F) < ∞. It was however shown in the
celebrated Durst–Dudley–Borisov theorem that F may have a ﬁnite bracketing
entropy J[] (∞, F, Q) under a simple necessary and suﬃcient criterion upon Q,
namely
Pp
Q({y}) < ∞.
(DDB(Q)) :
y∈X

Theorem 3 (Durst–Dudley–Borisov, 1981) For the class F defined in (24)
we have

J[] (∞, F, Q) < ∞ ⇔ DDB(Q) .

12

We shall combine Theorem 1 with a reﬁnement of Theorem 3 - see Lemma 5.5 to prove both a posterior consistency and a Bernstein–von Mises theorem for the
Dirichlet process prior, under the discrete total variation. To properly state it, we
need to introduce some more notations. From now on we shall denote by DP (α, M )
a Dirichlet process with mean probability measure α on X and concentration parameter M > 0. A possible representation of DP (α, M ) is that of Sethuraman
[9]:
DP (α, M ) =law P rY,β ,
(25)
where Y
α⊗N , and βi := Vi Πj≤i−1 (1 − Vj ) with (Vi )i∈N
Beta(1, M )⊗N being
independent of Y. Now consider the nonparametric Bayesian model where the prior
P r has distribution DP (α, M ) and where the sample (X1 , . . . , Xn ) has conditional
law P⊗n given P r = P. In this model it is well known (see [4]) that a natural
expression of the posterior distribution of P r given (X1 , . . . , Xn ) = (x1 , . . . , xn ) is
P ostn (x1 , . . . , xn ) := DP (α(x1 ,...,xn ) , M + n), where
n

α(x1 ,...,xn ) :=θn α + (1 − θn )P(x1 ,...,xn) , with θn :=

M
1X
δxi .
, and P(x1 ,...,xn ) :=
M +n
n
i=1

We shall take advantage of that explicit representation to prove the following two
results.
Corollary 1 (Posterior consistency) Take F as in (24). Let P0 be a proba∗
bility measure on countable X, and let (Xn )n≥1
PN
0 . Then for almost every
sequence (xn )n≥1 we have
P ostn (x1 , . . . , xn ) − P(x1 ,...,xn)

T ot.V ar.

→L 0.

Corollary 2 (Bernstein–von Mises) Assume in addition (DDB(P0 )) and (DDB(α)).
Then for almost every sequence (xn )n≥1 we have


√
n P ostn (x1 , . . . , xn ) − P(x1 ,...,xn) →L GP0 , in ℓ∞ (F).
As a consequence, for almost every sequence (xn )n≥1 we have
√
n P ostn (x1 , . . . , xn ) − P(x1 ,...,xn )
The corresponding proofs are written in §5.4.

13

T ot.V ar.

→L || GP0 ||F .

4.2

A Donsker theorem for local empirical measures under a bracketing condition

Assume in this subsection that X = Rd . The local empirical process indexed by
functions, introduced by Einmahl and Mason [3] has been intensively investigated
during the last decades, due to its connections with several smoothing nonparametric methods. One of its particular forms can be written as follows:
"
#
 
n



1 X  −1
Tn,hn (f ) := p
f hn Zi − z − E f h−1
,
n Zi − z
nhdn i=1

where (hn )n≥1 is a deterministic non negative sequence tending to 0, and where
(Zn )n≥1 is an i.i.d. sequence. Implicit in the results of Einmahl and Mason [3,
Theorem 1.1] is the following Donsker theorem.
Theorem 4 (from Einmahl and Mason, 1997) Let (hn )n≥1 be a non random
sequence of non negative numbers such that hn → 0 and nhdn → ∞. Assume that
J(∞, F) < ∞. Assume that the support S of F is bounded, and that Z1 admits a
version of Lebesgue density f on a neighborhood of z that is continuous at z and
such that f (z) > 0. Also assume that, taking P0 as the uniform distribution on S,
we have P0 (F 2 ) < ∞. Then we have the following weak convergence
1
p

λ(S)f (z)

Tn,hn (·) →L WP0 (·), in ℓ∞ (F),

where WP0 (·) denotes the L2 (P0 )-isonormal Gaussian process indexed by F (or
P0 -Brownian motion).
Their proof heavily relies on a representation of their own [3, Proposition 3.1]:
Tn,hn (f ) :=law p

1
nhdn

×

X
n

bi,n

i=1



f (Yi,n ) − Pn (f ) + Rn (f ),

as processes indexed by F, where:

• The (bi,n )i≤n are i.i.d Bernoulli with parameter an := P h−1
n (Y1 − z) ∈ S ;
• The (Yi,n )i≤n are i.i.d with law


Pn := P Y1 ∈ · | h−1
n (Y1 − z) ∈ S ,

with (bi,n )i≤n ⊥
⊥ (Yi,n )i≤n ;

14

(26)

• The term
Rn (f ) :=

n
P

i=1

(bi,n − an )
p
Pn (f )
nhdn

(27)

plays the asymptotic role of a correcting drift between the Brownian bridge
GP0 and the Brownian motion WP0 .
The following corollary of Theorem 2 is a Donsker theorem for Tn,hn under the
condition J[] (∞, F, P0 ) < ∞. Its proof is written in §5.5.
Corollary 3 Theorem 4 still holds if assumption J(∞, F) < ∞ is replaced by
J[] (∞, F, P0 ) < ∞.

5
5.1

Proofs
Proof of Theorem 1

The proof is divided in two lemmas.
Lemma 5.1 Take M > 0. Under the assumptions of Theorem 1 we have ||
GM
n ||F →P 0 as n → ∞.
Proof : Fix M > 0, ǫ > 0, and choose - by (13) and (14) - an integer N for which,
P(β n ∈ Sn ) > 1 − ǫ for all n ≥ 1, with
n
o

Sn := p ∈ ℓ1,+ , || p ||1 ≤ N and N[] ǫ, F, || · ||Pn,p ,1 ≤ N .


Now ﬁx n ≥ 1, and p ∈ Sn and denote by Br(n, p) = (fj− , fj+ ), j = 1, . . . , N
a covering bracket of F with maxj≤N || fj+ − fj− ||Pn,p ,1 ≤ ǫ/N . Using the same
comparison argument as in [12, p. 122] we have, for (f − , f + ) ∈ Br(n, p), f ∈
Jf − , f + K and y ∈ XN (recall (2)):

h
i
−
−
− N Pn,p (f + − f − )
Py,p f 1{F ≤M } − Pn,p f 1{F ≤M }


≤Py,p f 1{F ≤M } − Pn,p f 1{F ≤M }

h
i
+ N Pn,p (f + − f − ),
(28)
≤ Py,p f + 1{F ≤M } − Pn,p f + 1{F ≤M }
from where, writing Bn,p for the set of functions f that are a side of a bracket
(hence ♯Bn,p ≤ 2N , where "♯" stands for "cardinal"):


sup Py,p f 1{F ≤M } − Pn,p f 1{F ≤M }
f ∈F

15



≤ max Py,p f 1{F ≤M } − Pn,p f 1{F ≤M }
+ ǫ.
f ∈Bn,p

Note that the condition p ∈ ℓ1,+ is crucial to obtain (28). Now formally replacing
y by an i.i.d sequence (Yi )i∈N having distribution Pn,p we obtain, for p ∈ Sn :
!
X

E sup
≤ ∆n (p) + ǫ, where
(29)
pi f (Yi ) − Pn,p (f )
f ∈F

i∈N

∆2n (p)

2

:=E

X

f ∈Bn,p

≤E


X 
pi f (Yi ) − Pn,p (f )

 X

f ∈Bn,p

i∈N

!
 2
X 
pi f (Yi ) − Pn,p (f )
i∈N

≤(2N )2 max E
f ∈Bn,p

≤(2N )2 max

f ∈Bn,p

!

X
i∈N
2


X 
pi f (Yi ) − Pn,p (f )
i∈N

2

!



p2i Var f (Y1 )

(30)

≤(2N M || p ||2 ) .

Combining (29) and (30) yields, almost surely


M
E || Gn ||F β n 1Sn (β n ) ≤ 2N M || β n ||2 +ǫ.
This
/
 concludes the proof, since || β n ||2 →P 0 by assumption and since P β n ∈
Sn < ǫ. 
With Lemma 5.1 at hand, the proof of Theorem 1 will be concluded as follows:
Lemma 5.2 We have
lim lim d(GM
n , Gn ) = 0.

M →∞ n→∞

Proof : In view of (14) it is suﬃcient to show that
 



∀ǫ > 0, lim lim P E F 1{F >M } Y1,n β n > ǫ ≤ ǫ.
M →∞ n→∞

This is immediate by (12) combined with Markov’s inequality. 

16

5.2

Proof of Theorem 2

By (15) we can assume without loss of generality that || β n ||2 ≡ 1 for all n. First
note that (18) immediately implies



∀δ > 0, N[] δ, F, || · ||Pn,β n ,2
is bounded in probability.
(31)
n≥1

The proof of Theorem 2 follows the same directions as in that of Theorem 2 in [13].
The only crucial point that changes is that of proving the following asymptotic
equicontinuity condition
lim lim

P∗

δ→0 n→∞

sup
(f1 ,f2 )∈F 2 , ρ(f1 ,f2 )<δ

| Gn (f1 ) − Gn (f2 ) |= 0,

(32)

which would be the only missing ingredient to complete the proof of Theorem 2.
Proving (32) will be achieved by conditioning upon β n and using the following
chaining argument. It is an extension of usual chaining arguments for the bracketing entropy [11, p. 286, Lemma 19.34] to unbalanced empirical measures. Due to
the fact that inﬁnitely many weights are involved, only uniformly bounded classes
of functions are treated here for simplicity. This will be largely suﬃcient for our
purposes.
Lemma 5.3 Let p ∈ ℓ1,+ such that || p ||2 = 1 let Q be a probability measure and
let G be a uniformly bounded pointwise measurable class of functions with countable
separant G0 . Let δ ∈ (0, ∞] be such that
sup || g ||Q,2 ≤ δ and

(33)

g∈G

sup
g∈G, y∈X

| g(y) |≤|| p ||−1
∞ a(δ, Q), where

(34)

q

a(δ, Q) := δ/ log N[] δ, G, || · ||Q,2 .

Then for any i.i.d sequence (Yi )i∈N with distribution Q, we have


X 

E sup
≤ C1 J[] δ, G, Q ,
pi g(Yi ) − Q(g)
g∈G

i∈N

where C1 is a universal constant.
Proof : We shall use the notations
∆2 (G, Q) := sup Q(g2 ), and Γ(G) :=
g∈G

sup
g∈G, y∈X

17

| g(y) | .

(35)

Given a ﬁnite class of functions G̃ and given m ≥ 1 we have, by combining Lemmas
2.2.9 and 2.2.10 in [12, p. 102]:
!
m


X
pi g(Yi ) − Q(g)
E max
g∈G̃

i=0

"v
#
um
uX
≤24 t
p2i ∆2 (G̃, Q) log(1 + ♯G̃) + max | pi | Γ(G̃) log(1 + ♯G̃) ,
i≤m

i=0

"

#
q
≤24 || p ||2 ∆(G̃, Q) log(1 + ♯G̃)+ || p ||∞ Γ(G̃) log(1 + ♯G̃) ,

(36)

where the possible choice of factor 24 was actually shown in [11, p. 285, Lemma
19.33]. Since (36) does not depend upon m we then have, as soon as Γ(G̃) < ∞
(and recalling that || p ||2 = 1):
!

X 
E max
pi g(Yi ) − Q(g)
g∈G̃

i∈N

m


X
pi g(Yi ) − Q(g)
= lim E max
m→∞

g∈G̃

i=0

!



q
≤24 ∆(G̃, Q) log(1 + ♯G̃)+ || p ||∞ Γ(G̃) log(1 + ♯G̃) ,

(37)
(38)

where (37) is an application of the dominated convergence theorem, since all the
involved random variables are bounded by 2Γ(G̃). Now with (38) at hand, the
remainder of the proof is as follows: a careful look at all the arguments of the
proof of [11, p. 286, Lemma 19.34] - noting that their truncating argument is not
needed here - shows that the latter are still true with the systematic formal change
√
of n by || p ||−1
∞. 
We can now start our proof of (32). First ﬁx ǫ > 0. Using (18) and (20) there exist
δ1 , δ2 > 0 and n0 such that for all n ≥ n0 we have 1 − ǫ ≤ P∗ β n ∈ S′n , where S′n
is the set of all p ∈ ℓ1,+ satisfying the following conditions:
δ

√
1
2 2C1 J[]
(39)
, F, Pn,p ≤ ǫ
2
sup || f ||Pn,p ,2 < δ1 , where
f ∈Fδ2

o
n
Fδ2 := f1 − f2 , (f1 , f2 ) ∈ F 2 , ρ(f1 , f2 ) < δ2 ,

and where C1 denotes the universal constant in (35). Now ﬁx p, write
T (p) := || p ||−1
∞ a(δ1 , Pn,p )1{||p||∞ >0} , and deﬁne

18

o
n
Fp,δ1 := (f1 − f2 )1{F ≤T (p)} , (f1 , f2 ) ∈ F 2 , || f1 − f2 ||Pn,p ,2 < δ1 .

Next, apply Lemma 5.3 for ﬁxed p ∈ S′n to obtain (noticing that Fp,δ1 satisﬁes (33)
and (34) for the choice of Q := Pn,p and δ := δ1 )
!


X 

≤C1 J[] δ1 , Fp,δ1 , Pn,p
E
sup
pi f (Yi ) − Pn,p f
f ∈Fp,δ1

i∈N


δ
√
1
≤2 2C1 J[]
, F, Pn,p ,
2

(40)

where the Yi are i.i.d with law Pn,p and where (40) is a consequence of (39) and
standard comparisons of entropy numbers. Now since the latter inequality is valid
for all p ∈ S′n we have




T
E || Gn ||Fβn ,δ1 β n 1S′n β n ≤ ǫ1S′n β n , almost surely.
(41)

Note that the measurability || GTn ||Fβn ,δ1 is not immediate at all, but can be proved
using the same arguments as in [13, proof of Proposition 4.2]. In view of (41), and
since Fδ2 ⊂ Fp,δ1 for p ∈ S′n , the proof of (32) will be completed if we prove the
following lemma.
Lemma 5.4 We have d(GTn , Gn ) → 0 as n → ∞.
Proof : From (11) we have (noting that || βn ||2 ≡ 1 implies || βn ||∞ > 0 a.s.)

d GTn , Gn
!



(42)
≤E arctan 2 || β n ||1 E F 1{F >T (βn )} (Y1,n ) | β n


=E arctan 2 || β n ||1 E F 1{F >T (βn )} (Y1,n ) | β n


!

!

|| β n ||1  p
≤E arctan 2
E F 1{F >T (βn )} (Y1,n ) | β n
T (β n )p−1
!



|| β n ||1 × || β n ||p−1
∞
=E arctan 2
.
E F p 1{F >T (βn )} (Y1,n ) | β n
a(δ1 , Pn,β n )p−1


p−1
is
Since, by (17) and (31), the sequence || β n ||1 × || β n ||∞ /a(δ1 , Pn,βn )
bounded in probability, it only remains to prove that


En := E F p 1{F >T (βn )} (Y1,n ) | β n →P 0.
(43)

19

To prove this, ﬁx ǫ > 0 and choose M large enough so that


E F p 1{F >M } (Y1,n ) ≤ ǫ2 ,

for all n ≥ 1, which is possible by (19). Next apply Markov’s
inequality to En on

the set {T (β n ) > M } and then note that P T (β n ) ≤ M → 0 by (16) and (31). To
conclude the proof, let us now consider the isolated case where (17) and (19) are
removed from the set of assumptions of Theorem 2, but F is uniformly bounded,
i.e., F ≤ M for some constant M > 0. Then a look at (42) immediately yields the
claim, noticing that T (β n ) →P ∞. 

5.3

Proof of Corollary 1

With (25) in mind, let us deﬁne
n
A := (xn )n≥1 ,

α(x1 ,...,xn ) − P0

F

o
→0 .

(44)

The class of indicators of subsets of a countable set is universally Glivenko–Cantelli
- see, e.g., [2, p. 217, Remark 6.4.3]. Therefore, since θn → 0, the triangle inequality
∗
entails PN
0 (A) = 1. Now take an arbitrary sequence (xn )n≥1 ∈ A. We shall apply
Theorem 1 to the sequence P ostn (x1 , . . . , xn ). In this setup we have Pn,p = Pn =
α(x1 ,...,xn ) for all p ∈ ℓ1 , and
βi,n := Vi,n

i−1
Y

(1 − Vj,n ), i ∈ N, n ≥ 1,

j=0

with (Vi,n )i∈N
Beta(1, M + n)⊗N . To prove (13) let us ﬁrst remark that if
−
+
Jf , f K is a bracket between two indicator functions fulﬁlling || f + −f − ||P(x1 ,...,xn ) ,1 ≤
ǫ then || f + − f − ||α(x1 ,...,xn ) ,1 ≤ θn + (1 − θn)ǫ. Moreover, since the pointwise supremum/inﬁmum of a set of indicator functions is itself an indicator function, any
covering of F by brackets can be converted into another covering with the same
number of brackets, each of one between two indicator functions. Hence, since
θn → 0, we conclude that it is suﬃcient to prove that N[] (ǫ, F, P(x1 ,...,xn) ) is a
bounded sequence for ﬁxed ǫ > 0. This is done as follows: let us ﬁrst choose a
ﬁnite set C0 ⊂ X such that P0 (C0 ) > 1 − ǫ. Then by deﬁnition of A one has
P(x1 ,...,xn ) (C0 ) > 1 − ǫ for all large enough n.

(45)
∀C ⊂ X, C ∩ C0 ⊂ C ⊂ C ∩ C0 ∪ C0c .

Hence the the ﬁnite collection
n

J1C , 1C∪C0c K, C ⊂ C0

20

o

deﬁnes a covering of 2♯C0 brackets having || · ||P(x1 ,...,xn ) ,1 diameters less than ǫ.
This proves that N[] (ǫ, F, P(x1 ,...,xn) ) ≤ 2♯C0 for all large n, and hence proves (13).
Now conditions (12) and (14) are immediate since F is uniformly bounded and
|| β n ||1 ≡ 1 - see, e.g. [6,
 p. 112]. Finally, standard calculus on beta distributions
shows that E || β n ||22 ∼ n−1 , from where one can apply Theorem 1 and conclude
the proof.

5.4

Proof of Corollary 2

We shall now assume without loss of generality that the support of P0 is inﬁnite.

5.4.1

Two preliminary results

p
Theorem 3 states that the ﬁniteness of Σy∈X P0 ({y}) is equivalent to that of
J[] (∞, F, P0 ). Our next lemma goes one step further: it shows that it is possible
to control the magnitude of J[] (δ, F, P0 ), for small δ > 0, by "tail" sums of the
p
P0 ({y}).

Lemma 5.5 Define, for k ∈ N :
n
jP0 (k) := min J ∈ N,

X

y∈X : P0

({y})≤16−J

o
P0 ({y}) ≤ 4−k .

Then, for all p ≥ 1 we have, for a universal constant C2
v
sX p
X
u

−(p−1)
J[] 2
P0 ({y}) × u
, F, P0 ≤ C2
t
y∈X

y: P0

−j
(p)+1
({y})≤16 P0

p

(46)

P0 ({y}).

Moreover if the support of P0 is infinite we have jP0 (p) → ∞ as p → ∞.
Proof : The very last statement is obvious. We shall now write j(·) instead of
jP0 (·) for concision. The proof consists in enriching the arguments of Dudley [2, p.
245-246] with additional analytical precisions. We shall hence borrow his notations.
First, for j ∈ N write
o
n
Aj := y ∈ X, 16−j−1 < P0 ({y}) ≤ 16−j , and rj := ♯Aj .
Now deﬁne the following maps on N
m(·) : k →

j(k)
X
j=0

j(k)

rj = ♯

[

Aj ,

j=0

21

n
k(·) : J → min p ≥ 1, 4−p <

X

y: P0 ({y})≤16−J

n

o
P0 ({y}) ,

o
κ(·) : k → min κ ∈ N, j(κ) = j(k) .

For consistency of notations in the following calculus, we shall also deﬁne k(−1) :=
0. Note that, writing K for the range of κ(·), the map j(·) is one to one on K.
Fix k ≥ 1. Similarly as in Dudley [2, p. 245-246] we see that, for ﬁxed k ≥ 1, one
can use the same arguments as for (45), with the formal replacement of ǫ by 4−k
and C0 by
j(k)
[
Ck :=
Aj ,
j=0

which satisﬁes P0 (Ck ) ≥ 1 − 4−k by (46). This implies
∀k ≥ 1, N[] 2−k , F, || · ||P0 ,2



≤ 2m(k) .

(47)

Now, for any p ≥ 1, by monotonicity of the involved functions:


 Xq
−(p−1)
J[] 2
, F, P0 ≤
log N[] 2−k , F, || · ||P0 ,2 2−(k−1) − 2−k
k≥p

≤

p

log(2)

X
k≥p

p

m(k)
by (47).
2k

Next, ﬁx p ≥ 1 and write
p
j(k)
X m(k) X X
√
rj 2−k
≤
2k
k≥p

k≥p j=0

j(k) s
Xp
X
XX
p
P0 ({y}) 2j−k , since rj ≤
4j+1 P0 ({y})
2
≤
k≥p j=0

y∈Aj

y∈Aj

XsX p
X
P0 ({y})
2j−k
=2
j≥0

y∈Aj

k: k≥p,
j(k)≥j

vX
sX X p
u
≤2
P0 ({y}) × u
u
t j≥0
j≥0 y∈Aj

vX
sX p
u
=2
P0 ({y}) × u
u
t j≥0
y∈X

X

k: k≥p,
j(k)≥j

k≥p,
j(k)≥j

22

X

2
2j−k , using Cauchy–Schwartz

2
2j−k .

Now we have
X X
j≥0

≤4
=4

2

2j−k

k: k≥p,
j(k)≥j

X
j≥0

4j−k(j−1) ∧ 4j−p , since j(k) ≥ j implies k ≥ k(j − 1)

X

X

k≥0 j: j(k−1)≤j−1<j(k)

=4

X

X

4j−k ∧ 4j−p , since k(j) = k for j(k − 1) ≤ j < j(k)
X

4j−p + 4

k≤p j: j(k−1)≤j−1<j(k)

X

4j−k

k≥p+1 j: j(k−1)≤j−1<j(k)

h
i
X
≤8 4j(p)−p +
4j(k)−k
k≥p+1

=8

X

j(k)−k

4

k≥p

≤8

X

=8

X

k≥p

4j(k)−k+κ(k) ×
j(k)−k+κ(k)

4

k≥p

≤8

X X
j≥0

y∈Aj

×

ℓ : P0

X

P0 ({y}), by (46) and since j(k) = j(κ(k))

({y})≤16−j(k)+1

X

X

P0 ({y})

j≥j(k)−1 ℓ ∈Aj


P0 ({y})

X

k: k≥p, j(k)−1≤j


4j(k)−k+κ(k) .

Now notice that, when j(p) > j + 1 the set of indices {k ≥ p, j(k) − 1 ≤ j} is
empty, from where


X
X X
4j(k)−k+κ(k)
P0 ({y})
j≥0

≤
≤

y∈Aj

X

j≥j(p)−1

X

j≥j(p)−1

≤4

X

y∈Aj

X

j≥j(p)−1

≤32

X

k: k≥p, j(k)−1≤j

X

y∈Aj


P0 ({y})


P0 ({y})

X

y∈Aj

X

j≥j(p)−1 y∈Aj

X

4j(k)+κ(k)−k

k: j(k)−1≤j
′

X

4j(k )+k

′

′

X

k ′ ∈K, j(k ′ )≤j+1

X

k: κ(k)=k ′

k ′ ∈K, j(k ′ )≤j+1


P0 ({y})







4j(k ) , since κ(k) = k′ implies k ≥ k′

P0 ({y})4j , since κ(·) is one to one on K

23

4−k

≤32

X

Xp

j≥j(p)−1 y∈Aj

=32
y: P0

P0 ({y}), since y ∈ Aj implies P0 ({y})4j ≤

X

({y})≤16−j(p)+1

p

p

P0 ({y})

P0 ({y}).

This concludes the proof.
Our second preliminary result is as follows.
Lemma 5.6 Write

Iǫ := y ∈ X, P0 ({y}) ≤ ǫ , ǫ ∈ Q+ .

∗

Then for P0⊗N -almost any sequence (xn )n≥1 we have:
Xp
Xq
∀ǫ ∈ Q+ , lim
P(x1 ,...,xn ) ({y}) =
P0 ({y}).
n→∞

(48)

y∈Iǫ

y∈Iǫ

Proof: Since the class F is P0 -Donsker and admits a square integrable envelope
(F ≡ 1), the conditional multiplier Donsker theorem applies for a suitable i.i.d.
standard normal sequence (ξn )n≥1 - see, e.g., [12, p. 183, Theorem 2.9.7]. Hence
∗
for PN
0 -almost every sequence (xn )n≥1 we have - recalling that WP0 stands for the
L2 (P0 )-isonormal Gaussian process indexed by F:




W(x1 ,...,xn ) (f )
→L WP0 (f )
, where
(49)
f ∈F

f ∈F

n

1 X
ξi f (xi ), f ∈ F.
W(x1 ,...,xn ) (f ) := √
n
i=1

Here the weak convergence holds in the sense of Hoﬀman-Jőrgensen holds taking the
underlying probability space as the canonical product space for (ξn )n≥1 . Moreover
the involved processes are Gaussian, hence weak convergence implies convergence of
ﬁrst moments of absolute suprema. As a consequence, for such a sequence (xn )n≥1
fulﬁlling (49) we have, for all ǫ ∈ Q+
!
!
n
n


1 X
1 X
E 2 sup √
ξi 1A (xi ) − √
ξi 1Iǫ (xi ) → E 2 sup WP0 1A − WP0 1Iǫ
.
n
n
A⊂Iǫ
A⊂Iǫ
i=1

i=1

Finally, by the standard equality
sup

X

A⊂Iǫ y∈A

g(y) =


X
1 X
g(y) ,
| g(y) | +
2
y∈Iǫ

24

y∈Iǫ

we have (with g(y) := n−1/2

n
P

i=1

ξi 1{y} (xi ))

n
n
1 X
1 X
ξi 1A (xi ) − √
ξi 1Iǫ (xi )
E 2 sup √
n
n
A⊂Iǫ
i=1
i=1
!
n
X 1 X
√
=E
ξi 1{y} (xi )
n
i=1
y∈Iǫ
Xq
P(x1 ,...,xn ) ({y}),
=

!

y∈Iǫ

and similarly (with now g(y) := WP0 (1{y} ))


E 2 sup WP0 1A
A⊂Iǫ



− WP0 1Iǫ





=

Xp

P0 ({y})

y∈Iǫ

which concludes the proof.

5.4.2

Use of Theorem 2

Recall that A was deﬁned in (44) and has probability one. Let us consider the set


q
X
X p
+
α(x1 ,...,xn ) ({y}) ≤
P0 ({y}) .
B := (xn )n≥1 , ∀ǫ ∈ Q , lim
n→∞

y∈I2ǫ

y: α(x1 ,...,xn ) ({y})≤ǫ

We have, for any n ≥ 1 and ǫ ∈ Q+ (using
q
X
α(x1 ,...,xn ) ({y})

√
√
√
a + b ≤ a + b)

y: α(x1 ,...,xn ) ({y})≤ǫ

p
≤ θn

X

y: α(x1 ,...,xn ) ({y})≤ǫ

p

α({y}) +

p

1 − θn

X

y: α(x1 ,...,xn ) ({y})≤ǫ

q

P(x1 ,...,xn) ({y}).

Now if (xn )n≥1 belongs to A and since F induces the total variation distance we
have, for all n large enough


y ∈ X, α(x1 ,...,xn) ({y}) ≤ ǫ ⊂ y ∈ X, P0 ({y}) ≤ 2ǫ = I2ǫ .
∗

We hence conclude that PN
0 (A ∩ B) = 1 - recalling that θn → 0 and (DDB(α))
holds. Let us now consider a sequence (xn )n≥1 ∈ A ∩ B. Similarly as in §5.3,
we shall prove Corollary 2 by verifying all the assumptions of Theorem 2, for the

25

choice of Pn,p = Pn := P(x1 ,...,xn) . Because the class F is uniformly bounded by
1, the conditions upon
βi,n :=

√

nVi,n

i−1
Y

(1 − Vj,n ), i ∈ N,

j=0

that we need to check are (15) and (16), or equivalently
|| β n ||2 →P 1, and || β n ||4 →P 0.
These are respectively proved by direct computations of expectations and variances.
It now remains to verify (18) and (20). By deﬁnition of A and since θn → 0, the
sequence Pn obviously fulﬁlls (22) and therefore satisﬁes (20). Now in view of
Lemma 5.5, assertion (18) will be proved if we show that
X
p
lim lim
Pn ({y}) = 0.
(50)
p→∞ n→∞

y: α(x1 ,...,xn ) ({y})≤16−jPn (p)+1

Lemma 5.7 Take (xn )n≥1 ∈ A. For any p ≥ 1 we have jPn (p) ≥ jP0 (p) for all
large enough n.
Proof : Fix p ≥ 1. By deﬁnition of jP0 we have
X
P0 ({y}) > 4−p .
−jP (p)+1
0

y: P0 ({y})≤16

Now since (xn )n≥1 ∈ A, we have, for all n large enough:
X
Pn ({y}) > 4−p ,
−jP (p)+1
0

y: P0 ({y})≤16

whence jP0 (p) − 1 ≤ jPn (p) − 1 by deﬁnition of jPn . 
Now applying Lemma 5.7 we have, writing ǫ(p) := 16−jP0 (p)+1 :
X
p
Pn ({y})
lim lim
p→∞ n→∞

≤ lim lim

p→∞ n→∞

≤ lim lim

p→∞ n→∞

y: α(x1 ,...,xn ) ({y})≤16−jPn (p)+1

X

y: α(x1 ,...,xn ) ({y})≤ǫ(p)

X p

y∈I2ǫ(p)

p

Pn ({y})

P0 ({y}), since (xn )n≥1 ∈ B

26

=0,
by (DDB(P0 )) together with lim jP0 (p) → ∞. This proves (50) and we can now
apply Theorem 2 to obtain
!

√ 
dBL
n P ostn (x1 , . . . , xn ) − α(x1 ,...,xn ) , Gα(x1 ,...,xn ) → 0.
But since (xn )n≥1 ∈ A the sequence Pn := α(x1 ,...,xn) satisﬁes (22) from where (see
[13, Remark 2.2]):
Gα(x1 ,...,xn ) →L GP0 , in ℓ∞ (F),
which concludes the proof of Corollary 2. 

5.5

Proof of Corollary 3

Recall that P0 denotes here the uniform distribution on S and that Pn has been
deﬁned in (26). Let V be a neighborhood of z on which Z1 admits the density f .
Since S is bounded and hn → 0 we have z + hn S ⊂ V for n large enough. We may
assume without loss of generality that this is the case for all n ≥ 1.
Lemma 5.8 We have (taking here the convention 0/0 = 0)
!
n

X
bi,n 
s
f (Yi,n ) − Pn (f )
→L GP0 ,
n
P
i=1
f
∈F
2
bi,n
i=1

where GP0 denotes the P0 Brownian bridge.
Proof : Write
βi,n := s

bi,n
n
P

i=1

, for i = 1, . . . , n.

b2i,n

Since f is continuous at z we have

an ∼ λ(S)f (z)hdn , from where nan → ∞ and an → 0.

(51)

This property ensures that the sequence β n satisﬁes (15), (16) and (17) of Theorem
2 - taking p := 2 and recalling that bi,n ≡ b2i,n . In order to verify (18) and (31)
we will now prove (23), noting here that Pn,p := Pn for all p ∈ ℓ1,+ . The usual

27

change of variable u = h−1
n (v − z) in the next integrals gives, for an arbitrary non
negative function g with support included in S
Z


1
Pn (g) =
g h−1
n (v − z) f (v)dv
an z+hn S
Z
hd
g(u)f (z + hn u)du
= n
an S
Z
hdn
≤ sup f (z + hn u)
g(u)du
an S
u∈S
= sup f (z + hn u)
u∈S

hdn
λ(S)P0 (g).
an

This proves (23) by applying that inequality to elements of the form (f1 − f2 )2 ,
(f1 , f2 ) ∈ F 2 and recalling (51) together with the continuity of f at z. This also
proves (19), taking g := F 2 1{F >M } . Let us now verify (20) by proving (22). Using
a calculus similar as above we have, for an arbitrary function g ≺ (2F )2 ∨ (2F )
f (z)λ(S) Pn (g) − P0 (g)
Z
Z
f (z)λ(S)hdn
g(u)f (z)du
g(u)f (z + hn u)du −
=
an
S
S
Z
Z
f (z)λ(S)hdn
≤
g(u)f (z + hn u)du − f (z) g(u)du
×
an
S
S
Z
d
f (z)λ(S)hn
| g(u) | du
− 1 × f (z)
+
an
S
Z
f (z)λ(S)hdn
≤
× sup | f (z + hn v) − f (z) | × (2F )2 ∨ (2F )du
an
v∈S
S
Z
f (z)λ(S)hdn
+
− 1 × f (z) (2F )2 ∨ (2F )du,
an
S

(52)

which tends to zero independently of g ≺ (2F )2 ∨ (2F ). This proves (22) and
concludes the proof of Lemma 5.8.
Let us now continue the proof of Corollary 3. First, note that we have
n
X
i=1

b2i,n ∼ λ(S)f (z)nhdn in probability, from where

p

1
f (z)λ(S)nhdn

n
X
i=1

!


βi,n f (Yi,n ) − E f (Yi,n )

28

f ∈F

(53)
→L GP0 ,

(54)

and hence that sequence of processes is asymptotically tight (see, e.g., [12, p. 20,
Deﬁnition 1.3.7]). Now elementary probability calculus shows that
n
P

i=1

p

(bi,n − an )

f (z)λ(S)nhdn

(55)

→L Z,

where Z is standard normal. Moreover, since Pn satisﬁes (22) and since f →
P0 (f ) is continuous with respect to || · ||P0 ,2 , which makes F totally bounded,
the (deterministic) sequence Pn (·) is relatively compact in ℓ∞ (F). This, combined
with (55), implies that the sequence Rn (·) - deﬁned in (27) - is asymptotically tight,
and hence so is Tn,hn (·) by summation. It will hence be proved to converge to WP0
if we prove ﬁnite marginal convergences. This is done by elementary analysis of
characteristic functions, using the change of variable u = h−1
n (v−z) in the integrals.
We omit details. 

6

Appendix: a minor proof

In this section we prove the measurability properties claimed in §3.


Lemma 6.1 For fixed r ≥ 1 and n ≥ 1, the map (ǫ, p) → N[] ǫ, F, || · ||Pn,p ,r is

Borel from ]0, ∞[×ℓ1 to R+ . As a consequence, the maps

p → J[] δ, F, Pn,p , δ > 0,

are Borel.

Proof : Fix r ≥ 1 and n ≥ 1. Any bracket is closed 
for the the pointwise topology,
i.e., the topology spanned by the evaluation maps {f → f (y)}, y ∈ X . Hence
so is any ﬁnite union of brackets that covers F0 . Since F is included in the closure
of F0 for the pointwise topology, we deduce that




(56)
∀(ǫ, p) ∈]0, ∞[×ℓ1 , N[] ǫ, F, || · ||Pn,p ,r = N[] ǫ, F0 , || · ||Pn,p ,r .
Now the proof of Lemma 6.1 boils down to proving the measurability of


H : (ǫ, p) → N[] ǫ, F0 , || · ||Pn,p ,r .
This is done by noting that, for any K ∈ N, the set

K
o
n
[
K
Jfj− , fj+ K
BK := (fj− , fj+ )j=1,...,K ∈ (F02 ) , F0 ⊂
j=1

29

is countable, and that
H(ǫ, p) > K ⇔ ∀(fj− , fj+ )j=1,...,K ∈ BK , ∃j ∈ {1, . . . , K}, || fj+ − fj− ||Pn,p ,r > ǫ,
which yields the claimed result, since for ﬁxed Borel non negative g, the map
p →|| g ||Pn,p ,r is Borel (recall that {Pn,p , p ∈ ℓ1 } is regular).

References
[1] N. T. Andersen, E. Giné, M. Ossiander, and J. Zinn. The central limit theorem and the law of iterated logarithm for empirical processes under local
conditions. Probab. Theory Related Fields, 77(2):271–305, 1988.
[2] R.M. Dudley. Uniform Central Limit Theorems. Cambridge Univ. Press, 1999.
[3] U. Einmahl and D.M. Mason. Gaussian approximation of local empirical
processes indexed by functions. Probab. Theory Related Fields, 107(3):283–
311, 1997.
[4] T. S. Ferguson. A Bayesian analysis of some nonparametric problems. Ann.
statist., 1(2):209–230, 1973.
[5] X. Fernique. Sur la convergence étroite des mesures Gaussiennes. Z. Wahrsch.
Verw. Gebiete, 68:331–336, 1985.
[6] N.L. Hjort, C. Holmes, P. Müller, and S.G. Walker. Bayesian nonparametrics,
volume 28. Cambridge University Press, 2010.
[7] Olav Kallenberg. Foundations of modern probability. Springer Science & Business Media, 2006.
[8] M. Ossiander. A central limit theorem under metric entropy with l2 bracketing.
Ann. Probab., 15(3):897–919, 1987.
[9] J. Sethuraman. A constructive deﬁnition of Dirichlet measures. Statist. Sinica,
4:639–650, 1994.
[10] A. Sheehy and Jon A Wellner. Uniform Donsker classes of functions. Ann.
Probab., 20(4):1983–2030, 1992.
[11] A.W. Van der Vaart. Asymptotic Statistics. Cambridge University Press, 1998.
[12] A.W. Van der Vaart and J.A. Wellner. Weak Convergence and Empirical
Processes. Springer, New-York, 1996.

30

[13] D. Varron. Donsker and Glivenko-Cantelli theorems for a class of processes
generalizing the empirical process. Electron. J. Stat., 8:2301–2325, 2014.

31

