arXiv:1704.07873v2 [math.ST] 13 Sep 2017

Weak Convergence of Stationary Empirical Processes
Dragan Radulović
Department of Mathematics, Florida Atlantic University

Marten Wegkamp
Department of Mathematics & Department of Statistical Science, Cornell University

September 14, 2017

Abstract
We offer an umbrella type result which extends weak convergence of classical
empirical processes on the line to that of more general processes indexed by
functions of bounded variation. This extension is not contingent on the type of
dependence of the underlying sequence of random variables. As a consequence we
establish weak convergence for stationary empirical processes indexed by general
classes of functions under α-mixing conditions.
Running title: Weak convergence of stationary empirical processes
MSC2000 Subject classification: Primary 60F17; secondary 60G99.
Keywords and phrases: Integration by parts, bounded variation, empirical
processes, stationary distributions, weak convergence.

1

1

Introduction

We consider the empirical process
n

1 X
Z̄n (g) = √
{g(Xi ) − Eg(Xi )} , g ∈ G,
n i=1

(1)

indexed by some class G of functions g : R → R. It is an obvious generalization of the

classical process

n

1 X
Gn (t) = √
1(−∞,t] (Xi ) − F (t) ,
(2)
n i=1

in which case the indexing class is G = g(x) = 1(−∞,t] (x) : t ∈ R . If the underlying
sequence {Xi }i≥1 is i.i.d., then the limiting behavior (as n → ∞) of the empirical

processes {Z̄n (g), g ∈ G} is well understood. The same is true for the bootstrap coun∗
∗
terpart {Z̄∗n (g), g ∈ G} based on an i.i.d. bootstrap sample {X1,n
, . . . , Xm
}, see,
n ,n

for instance, Van der Vaart and Wellner (1996) and Dudley (1999). The theory of
weak convergence of empirical processes based on independent sequences has yielded a
wealth of statistical applications and, in particular, it was instrumental for establishing the weak convergence of numerous novel statistics. Often the limiting distributions
of these statistics do not allow for closed form solutions, in which case the bootstrap
version of the process is utilized. As is standard practice nowadays, see the thorough
exposition in the first chapter of Van der Vaart and Wellner (1996), weak convergence
in this paper should be understood in the sense of Hoffmann-Jørgensen and expectations should be interpreted as outer expectations.
For empirical processes based on stationary sequences {Xi }i≥1 the situation is rather

different. The classical process {Gn (t), t ∈ R} has been treated by numerous authors,

who established weak convergence under sharp mixing assumptions, see, for instance,

Rio (2000). However, nothing similar exists for more general processes. The only
work that we could find in the literature, Andrews and Pollard (1994), treats more
general indexing classes, but imposes rather restrictive assumptions on the decay of
α-mixing coefficients. This discrepancy between the conditions needed for Z̄n (g) and
2

those for Gn (t), is due to fact that the typical approach for proving the uniform limiting
theorems heavily relies on the estimation of entropy numbers, which in turn require
good exponential maximal inequalities. Only β-mixing, via decoupling, allows for such
an estimate. This is the reason why one can find in the literature the treatment of
Z̄n (g) only for β-mixing sequences, see Arcones and Yu (1994) and Doukhan, Massart
and Rio (1995).
The situation for bootstrap of stationary empirical processes is even worse. Although
introduced more than twenty years ago, we only found three papers that study the
bootstrap for stationary empirical processes indexed by general classes G, see Künsch

(1989), Liu and Singh (1992) and Sengupta, Volgushev and Shao (2016). All these
results operate under β-mixing assumptions. Moreover, only in the case of VC-classes

do we have the sharp conditions, see Radulović (1996). Bracketing classes were considered by Bühlmann (1995), but this was established under restrictive assumptions
on both β-mixing coefficients and bracketing numbers. It is worth mentioning that
the covariance function of the limiting Gaussian process is unknown, and consequently
in most actual applications of these results, we heavily rely on the bootstrap version
of the process for which adequate results are sorely lacking. In short, the most general α-mixing sequences have never been considered for stationary bootstrap processes
{Z̄∗n (g), g ∈ G}, while for the non-bootstrap version {Z̄n (g), g ∈ G} we have only one

example in the literature.

In what follows we prove two general results that allow us to extend weak convergence
of {Gn (t), t ∈ R} to that of {Z̄n (g), g ∈ G}, where G is a class of functions of uni-

formly bounded total variation (called BVT in this paper). We would like to point out

that although there are examples of Donsker classes of infinite variation (for instance,
f : [0, 1] → [0, 1] with |f (x) − f (y)| ≤ |x − y|α, 1/2 < α < 1), such cases are rather the

exception than the norm. The majority of examples of bounded Donsker classes that
are given in the literature are subsets of the class BVT .
This enlargement from {Gn (t), t ∈ R} to {Z̄n (g), g ∈ G} is not contingent on the dependence structure of underlying sequence {Xi }i≥1 (or {Xi∗ }i≥1 ) and the only requirement
3

is that Gn (t) converges weakly to a Gaussian process. This allows us to derive weak
convergence of {Z̄n (g), g ∈ G} and {Z̄∗n (g), g ∈ G} for α-mixing sequences. The same

extension applies for the short memory causal processes considered in Doukhan and

Surgailis (1998), as well as processes treated in Dehling, Durieu and Volny (2009).
The technique we employ is a simple application of the integration-by-parts formula,
followed by the continuous mapping theorem. Arguably this approach may have been
known before, although we could not find any communication of it in the literature,
and certainly not among the research related to stationary empirical processes. A
follow-up paper (Radulović, Wegkamp and Zhao (2017)) extends the results obtained
in this work to classes indexed by multivariate functions of bounded variation, with an
emphasis on empirical copula processes. An important technical difference with the
follow-up paper is that here we allow for general stationary distribution functions of
the underlying {Xi }i≥1 . The proof for general, non-continuous processes, is not trivial

caused by technical complications related to the interplay between the atoms of the

limiting process Z(t) and discontinuities of g ∈ G.
The paper is organized as follows. Section 2 contains the statement of the main result
(Theorem 1) and related discussions, while Section 3 presents the bootstrap version
(Theorem 6) of this result. The proofs of these two main results are collected in Section
4. For completeness, the well-known integration by parts formula can be found in the
appendix.

2

Main results

For the total variation norm of a function g : R → R, we use the notation
X
||g||T V = sup
|g(xi+1 ) − g(xi )|.
Π

xi ,xi+1 ∈Π

The supremum is taken over all countable partitions Π = {x1 < x2 < . . .} of R. We

set, for T > 0,

BVT := {g : R → R : kgkT V ≤ T, kgk∞ ≤ T }.
4

We let BVT′ ⊂ BVT be the class of all functions g in BVT that are right-continuous.
Finally, we let {Zn (t), t ∈ R} be an arbitrary stochastic process such that
A1: lim|t|→∞ Zn (t) = 0;
A2: The sample paths of Zn are right-continuous and of bounded variation.
Clearly, both requirements A1 and A2 are met for the canonical empirical process
{Gn (t), t ∈ R}. In this paper, we study the limit distribution (as n → ∞) of the
sequence of processes



Zn (g) :=

Z

g(x) dZn (x), g ∈ G



for some class G ⊆ BVT′ , for some finite T .
Although the motivation as well as the most notable applications of our results are
related to the canonical case Zn (x) = Gn (x), the actual proof carries over for any
process Zn as long as assumptions A1 and A2 are satisfied.
Our main result is the following theorem.
Theorem 1 Assume that {Zn (t), t ∈ R} converges weakly, as n → ∞, to a Gaussian

process {Z(t), t ∈ R}, that has uniformly continuous sample paths with respect to the

distance d(s, t) = |F0 (s) − F0 (t)| for some distribution function F0 .

Then, for any T < ∞ and G ⊆ BVT′ , {Z̄n (g), g ∈ G} converges weakly, as n → ∞, to
a Gaussian process in ℓ∞ (G), that has uniformly L1 (F0 )-continuous sample paths.

Remark. Usually, an envelope condition sup |g(x)| ≤ G(x) and G ∈ L2 (F0 ) is im-

posed. Since such a condition, coupled with kgkT V ≤ T , implies that the functions g

are uniformly bounded, we assume kgk∞ ≤ T in our definition of BVT .

Theorem 1 allows us to derive weak convergence of Z̄n (g) via Zn (t), regardless of the
structure of the latter process. For instance, taking Zn as the standard empirical
processes Gn based on stationary sequences {Xi }i≥1 , we obtain the following corollary
as an immediate consequence of Theorem 1.
5

Corollary 2 Let {Xk }k≥1 be a stationary sequence of random variables with distribu-

tion F and α-mixing coefficients αn satisfying αn = O(n−r ), for some r > 1 and all
n ≥ 1. Then, for any G ⊆ BVT′ ,

R
- { g dGn , g ∈ G} converges weakly to a Gaussian process with uniformly L1 (F )continuous sample paths in ℓ∞ (G).

- For continuous F , G can be enlarged to BVT .
Proof. It is well known, see Theorem 7.2, page 96 in Rio (2000), that αn = O(n−r )
with r > 1, implies that the standard empirical process Gn (t) converges weakly to a
Brownian bridge process with uniformly continuous paths with respect to the distance
d(s, t) = |F (s) − F (t)|, for the stationary distribution F of Xk . The result for G ⊆

BVT′ now follows trivially from Theorem 1. If F is continuous, then the limiting
Brownian bridge has uniformly continuous sample paths with respect to the Lebesgue
measure on R. This, combined with Lemma 3 below, implies Corollary 2.
We used the following lemma.
Lemma 3 We have, for Z̄n (g) based on the canonical process Zn (x) = Gn (x),
sup

inf

′
g∈BVT h∈BVT

Z̄n (g) − Z̄n (h) ≤ T sup |Gn (x) − Gn (x− )|.
x

Proof. Let g be an arbitrary function in BVT . We denote its countable many discontinuities by ai . Let g be the right-continuous version of g, that is, g(x) = g(x) for all
x 6= ai and g(ai ) = g(a+
i ) for all i. Then
Z
Z
X
g dGn − g dGn ≤
|g(ai ) − g(ai )||Gn (ai ) − Gn (a−
i )|
i

≤ kgkT V sup |Gn (x) − Gn (x− )|
x

and the conclusion follows.
We would like to point out that α-mixing is the least restrictive form of available mixing
assumptions in the literature. To the best of our knowledge, there are actually very few
6

R
results that treat processes { g dGn , g ∈ G} indexed by functions, and they all require

very stringent conditions on the entropy numbers of G and on the rate of decay for αk .

See, for instance, Andrews and Pollard (1994). This is due to fact that α-mixing does
not allow for sharp exponential inequalities for partial sums. Consequently, the only
known cases for which we have sharp conditions are under more restrictive, β-mixing
dependence. Indeed, β-mixing allows for decoupling and it does yield exponential inequalities not unlike the i.i.d. case. The current state-of-the art results, Arcones and
Yu (1994), Doukhan, Massart and Rio (1995), applied to bounded sequences, require
P
n βn < ∞.
While it is correct that α-mixing is the weakest of the known mixing concepts, a referee
pointed out that functionals of mixing processes constitutes a much weaker concept
of short range dependence. For instance, Billingsley (1968, Theorem 22.2) establishes
the empirical process CLT for certain functionals of φ-mixing processes. The same referee made us aware that Theorem 1 also applies to the empirical process of long-range
dependent data, established by Dehling and Taqqu (1989) for subordinate Gaussian
processes, and by Ho and Hsing (1996) for linear processes. We provide an example
to demonstrate that Theorem 1 goes beyond dependence defined via mixing conditions. It applies to short memory causal linear sequences {Xi }i≥1 that are defined by
P∞
Xi =
j=0 aj ξi−j based on i.i.d. random variables ξi and constants ai . While the
Xi form a stationary sequence, they do not necessarily satisfy any mixing condition.

Weak convergence of the empirical processes {Gn (t), t ∈ R} was established under
P
sharp conditions in Doukhan and Surgailis (1998) on the weights aj ( j≥0 |aj |γ < ∞
for some 0 < γ ≤ 1), characteristic function of ξ0 (|E[exp(iuξ0 )]| ≤ C(1 + |u|)−p for all

u ∈ R, some C < ∞ and 2/3 < p ≤ 1), and a moment condition on ξ0 (E[|ξ0 |4γ ] < ∞).
To the best of our knowledge, there are no extensions to the more general processes

{Z̄n (g), g ∈ G}. Theorem 1 and the Doukhan and Surgailis (1998) result combined

imply the following:

Corollary 4 Let {Xi =

P

j≥0

aj ξi−j }i≥1 be such that conditions of Doukhan and Sur-

gailis (1998, pp 87–88) are satisfied and let F be the stationary distribution of Xi .
7

R
Then, for any G ⊆ BVT , { g dGn , g ∈ G} converges weakly to a Gaussian process

with uniformly L1 (F )-continuous sample paths in ℓ∞ (G).

Proof. Doukhan and Surgailis (1998) proved that {Gn (t), t ∈ R} converges weakly to a
Gaussian process in the Skorohod space. Since F is continuous under their assumptions,
see Doukhan and Surgailis (1998, pp 88), the sample paths of limiting process of Gn
are uniformly continuous with respect to the distance d(s, t) = |F (s) − F (t)| based on

the stationary distribution F . The proof now follows trivially from Theorem 1 and
Lemma 3.

The recent papers by Dehling, Durieu and Volny (2009) and Dehling, Durieu and
Tusche (2014) offer yet another, clever way to prove the weak limit of the standard
empirical processes Gn based on stationary sequences that are not necessarily mixing.
Their technique uses finite dimensional convergence coupled with a bound on the higher
moments of partial sums, which in turn controls the dependence structure. Dehling,
Durieu and Volny (2009) establishes the weak convergence of Gn , while Dehling, Durieu
and Tusche (2014) extends this idea to Z̄n indexed by more general classes of functions.
However, these authors impose cumbersome entropy conditions and only manage to
marginally extend the classes. For example, they prove weak convergence of the process
R
ft dGn , indexed by functions ft (x) which constitute an one-dimensional monotone
class (under the restrictive requirement that s ≤ t ⇒ fs ≤ ft ). Theorem 1 applied in

their setting, yields a more general result.

Corollary 5 Let G ⊆ BVT and let F be the stationary distribution of the sequence

{Xi }i≥1 . Under assumptions (i) and (ii) in Section 1 of Dehling, Durieu and Volny
R
(2009), { g dGn , g ∈ G} converges weakly to a Gaussian process with uniformly

L1 (F )-continuous sample paths in ℓ∞ (G).

Proof. The underlying distribution function F of Xk in Dehling, Durieu and Volny
(2009) is continuous, see their display (3) at page 3702. Moreover, these authors
establish weak convergence of Gn to a Gaussian process that has uniformly continuous
sample paths with respect to the distance d(s, t) = |F (s) − F (t)|. The proof follows

trivially from Theorem 1 and Lemma 3.

8

3

Bootstrap

The weak limit of Z̄n based on Gn in Theorem 1 is a Gaussian process Z̄ with complicated covariance structure
E[Z̄(f )Z̄(g)] = Cov(f (X0 ), g(X0)) +

∞
X

Cov(f (X0 ), g(Xk ))

k=1

+

∞
X

Cov(f (Xk ), g(X0))

k=1

for f, g ∈ G ⊂ BVT . A closed form solution is seldom available, so that actual applications of weak limit results of Z̄n are hard to implement. This situation calls for the
bootstrap principle. Given the sample of first n observations {X1 , . . . , Xn }, we let G∗n

be the bootstrap empirical process
G∗n (t) =

√

mn

!
mn
n
X
1
1 X
1(−∞,t] (Xi∗ ) −
1(−∞,t] (Xi ) , t ∈ R,
mn i=1
n i=1

∗
∗
based on a bootstrap sample {X1,n
, . . . , Xm
}. We stress that no additional assumpn ,n

∗
tion on the structure of the variables Xi,n
is required for our purposes. Analogous to
R
R
Z̄n (g) = g dGn , we define Z̄∗n (g) := g dG∗n for any g ∈ BVT′ with T < ∞. We recall

that the bounded Lipschitz distance

dBL (Z∗n , Z) = sup |E∗ [h(Z∗n )] − E[h(Z)]|
h∈BL1

between two processes Z∗n and Z metrizes weak convergence. The symbol E∗ stands
∗
∗
for the expectation over the randomness of the bootstrap sample X1,n
, . . . , Xm
, conn ,n

ditionally given the original sample X1 , . . . , Xn , while BL1 denotes the space of Lipschitz functionals h : ℓ∞ (G) → R with |h(x)| ≤ 1 and |h(x) − h(y)| ≤ kx − yk∞ for
all x, y ∈ G ⊂ BVT . As is customary in the literature, we speak of weak convergence

in probability if the random variable dBL (Z∗n , Z) converges to zero in probability; if it
converges to zero almost surely, we speak of weak convergence almost surely.
Theorem 6 Let G ⊆ BVT′ . Assume that, conditionally on X1 , . . . , Xn , {G∗n (t), t ∈ R}

converges weakly to a Gaussian process, in probability, that has unifomly continuous
9

sample paths with respect to the distance d(s, t) = |F (s) − F (t)| based on the stationary

distribution F of Xi . The following three statements hold true:
R
1.
g dG∗n , g ∈ G converges weakly to a Gaussian process with uniformly L1 (F )continuous sample paths in ℓ∞ (G).

2. If the weak convergence of {G∗n (t), t ∈ R} holds almost surely, then the conclusion
R
that
g dG∗n , g ∈ G converges weakly in ℓ∞ (G) holds almost surely as well.
3. If {Gn (t), t ∈ R} and {G∗n (t), t ∈ R} converge to the same limit, then so do
R
R
g dGn , g ∈ G and
g dG∗n , g ∈ G .

The literature offers numerous bootstrapping techniques for stationary data, such as

moving block bootstrap, stationary bootstrap, sieved bootstrap, Markov chain bootstrap, to name a few, but their validity is proved for specific cases/statistics only.
Due to complications with entropy calculations for dependent triangular arrays, almost all results treat the standard empirical processes G∗n with few notable exceptions.
The moving block bootstrap was justified for VC-type classes, but only under rather
restrictive β-mixing conditions on Xi (Radulović, 1996). Bracketing classes were considered by Bühlmann (1995), but his conditions are even more restrictive. In contrast,
the process {Gn (t), t ∈ R} is rather easy to bootstrap. This coupled with Theorem 6

offers the following result.

Corollary 7 Let {Xj }j≥1 be a stationary sequence of random variables with continuP
ous stationary distribution function F and α-mixing coefficients satisfying k≥n αk ≤
Cn−γ , for some 0 < γ < 1/3 and C < ∞. Let G∗n be the bootstrapped standard empiri-

cal process based on the moving block bootstrap, with block sizes bn , specified in Peligrad
R
(1998, page 882). Then, for G ⊂ BVT , { g dG∗n , g ∈ G} converges weakly to Gaussian
process, almost surely, with uniformly L1 (F )-continuous sample paths.

Proof. Theorem 2.3 of Peligrad (1998) establishes the convergence of G∗n to a Gaussian
process with uniformly continuous sample paths with respect to the distance d(s, t) =
|F (s) − F (t)| based on the stationary distribution F of Xi . Invoke Theorem 6 and
Lemma 3 to conclude the proof.

10

Just as for weak convergence of the empirical process based on stationary sequences,
there are numerous results that consider bootstrap for stationary, non-mixing sequences. For example, El Ktaibi, Ivanoff and Weber (2014) study short memory causal
linear sequences, and prove weak convergence of G∗n under conditions (on the growth
of the weights aj , the characteristic function of ξ0 and moments of ξ0 ) akin to the ones
required for its non-bootstrap counterpart Gn (Doukhan and Surgailis, 1998). Again,
Theorem 6 easily extends their result.
Corollary 8 Let {Xi =

P

j≥0

aj ξi−j }i≥1 be a sequence of random variables with sta-

tionary distribution F such that conditions of El Ktaibi, Ivanoff and Weber (2014) are
R
satisfied. Then, for any G ⊂ BVT , { g dG∗n , g ∈ G} converges weakly to a Gaussian
limit with uniformly L1 (F )-continuous sample paths, almost surely.

Proof. El Ktaibi, Ivanoff and Weber (2014) establish weak convergence of G∗n in the
Skorohod space, for continuous F, which in turn implies that the limiting process has
uniformly continuous sample paths with respect to distance d(s, t) = |F (s) − F (t)|.

Again, invoke Theorem 6 and Lemma 3 to conclude the proof.

4

Proofs for Theorem 1 and Theorem 6

We first give a short proof of Theorem 1 if the limit Z(t) of Zn (t) has continuous sample
paths with respect to the Lebesgue measure. Let dBL be the bounded Lipschitz metric
that metrizes weak convergence, see, e.g., Van der Vaart and Wellner (1996, page 73)
R
for the definition. Set Z̄n (g) = g dZn for any g ∈ BVT′ . Assumptions A1 and A2

imply that the Lebesgue-Stieltjes integrals

e n (g) = −
Z

Z

Zn dg

are well defined. Next, by the integration by parts formula in Lemma A, we have
Z̄n (g) = Z̃n (g) + Rn (g)
with
Rn (g) ≤ T sup |Zn (x) − Zn (x− )| → 0,
x

11

in probability, as n → ∞, since Z has uniformly continuous sample paths. For fixed g,
R
Z̃n (g) converges weakly to Z̃(g) := − Z dg by the continuous mapping theorem and

weak convergence of Zn . The continuous mapping theorem also guarantees that the
R
limit Z̃ is tight in ℓ∞ (G) as the map X 7→ Γg (X) = − X dg, g ∈ G, is continuous. By
the triangle inequality,

dBL (Z̄n , Z̃) ≤ dBL (Z̄n , Z̃n ) + dBL (Z̃n , Z̃)
=

sup |EH(Z̄n ) − EH(Z̃n )| + sup |EH(Z̃n ) − EH(Z̃)|

H∈BL1

H∈BL1

−

≤ T E[sup |Zn (x) − Zn (x )|] + T sup |EH ′ (Zn ) − EH ′ (Z)|
x

H ′ ∈BL1

= T E[sup |Zn (x) − Zn (x− )|] + T dBL (Zn , Z)
x

R
The second inequality follows since the map Γf (X) := X df is linear and Lipschitz
R
with Lipschitz constant |df | ≤ T and the suprema are taken over all Lipschitz

functionals H : ℓ∞ (G) → R with kHk∞ ≤ 1 and |H(X) − H(Y )| ≤ kX − Y k∞ and
H ′ : ℓ∞ (R) → R with kH ′ k∞ ≤ 1 and |H ′(X) − H ′ (Y )| ≤ kX − Y k∞ , respectively.

Together with the tightness of the limit Z̃, this implies that the empirical process Z̄n (g)

indexed by g ∈ G ⊂ BVT′ converges weakly.
The above proof for continuous limit processes Z (that is, processes Z with uniformly
continuous sample paths) is rather simple. Nevertheless, we could not find an actual
publication of this trick. We would like to stress that the extension of this proof to
general, non-continuous processes, is not trivial. Technical complications related to the
interplay between the atoms of the limiting process Z(t) and discontinuities of g ∈ G,

require some care.

Lemma 9 Assume that Zn (t) converges weakly to a Gaussian process Z(t), as n → ∞.
R
Then, for any right continuous function g : R → R of bounded variation Zn (g)= g dZn

is well defined, and converges to a normal distribution on R.

Proof. Let g be an arbitrary right-continuous function of bounded variation. First, we
R
R
notice that g dZn and Zn dg are indeed well defined as Lebesgue-Stieltjes integrals.
12

Recall that g can have only countably many discontinuities which we will denote ai .
By the integration by parts formula, Lemma A in the appendix, we have
Z
g(x) dZn (x) = T1 (Zn ) + T2 (Zn )

with operators T1 , T2 : ℓ∞ (R) → R
Z
T1 (Zn ) := − Zn (x) dg(x)
Z Z
X
−
T2 (Zn ) :=
1x=y dg(x) dZn (y) =
(g(ai ) − g(a−
i ))(Zn (ai ) − Zn (ai )).
i

Since g has finite variation, it is bounded with

g(ai ) −

g(a−
i ).

P

i

|αi | < ∞ for the jumps αi :=

Hence, the operator T2 , given by
X
αi (Zn (ai ) − Zn (a−
T2 (Zn ) =
i ))
i

is linear and continuous. We conclude the proof by observing that the continuity of
the operators T1 and T2 and the weak convergence of Zn (t) to a Gaussian process Z
R
ensures that these sequences Yn := g dZn = T1 (Zn ) + T2 (Zn ) converge weakly to a

normal distribution via the continuous mapping theorem.

For any distribution function F0 and any β > 0, we define
Ψβ,F0 (Zn ) :=
ðβ,F0 (Zn ) :=

sup
|F0 (s)−F0 (t)|≤β

|Zn (s) − Zn (t)| ,

sup
F0 (s)−F0 (s− )>β

Zn (s) − Zn (s− ) ,

with the convention that the supremum taken over the empty set is equal to zero.
Clearly, ðβ,F0 (Zn ) = 0 for all β > 0 if F0 is continuous. In general, for arbitrary F0 , the
quantity ðβ,F0 (Zn ) is bounded in probability, for all β > 0, by the continuous mapping
theorem, as long as Zn converges weakly. The following lemma plays an instrumental
role and it could be perhaps of independent interest.
Lemma 10 (Decoupling Lemma) For any distribution function F0 on R, any rightcontinuous function g : R → R and any β > 0, we have

|Z̄n (g)| ≤ 2β −1||g||L1(F0 ) + 6||g||T V Ψ2β,F0 (Zn ) + β −1 kgkL1 (F0 ) ðβ,F0 (Zn ).
R
Here kgkL1 (F0 ) = |g| dF0 and Zn satisfies assumptions A1 and A2.
13

Proof. Without loss of generality we can assume that ||g||L1(F0 ) + ||g||T V < ∞. Since

F0 is a distribution function we can construct, for any 0 < β < 1, a finite grid −∞ =
s0 < s1 < ... < sM < ∞ such that

F0 (sj ) − F0 (sj−1 ) ≥ β, F0 (sM ) < 1 − 2β
and
F0 (s−
j ) − F0 (sj−1 ) ≤ 2β,
leaving the possible jumps F0 (sj ) − F0 (s−
j ) unspecified. Based on this grid we approx-

imate Zn (t) by

e n (t) :=
Z

M
X

Zn (sj−1)1[sj−i, sj ) (t)

i=1

e n (±∞) = 0. We observe that by construction
and we set Z
e n (x)| ≤
sup |Zn (x) − Z
x

≤

sup

max

1≤j≤M x∈[s
j−1 ,sj )

sup
|F0 (x)−F0 (y)|≤2β

|Zn (x) − Zn (sj−1 )| + sup |Zn (x)|
x≥sM

|Zn (x) − Zn (y)| = Ψβ,F0 (Zn )

(3)

e n inherits the bounded variation properties of Zn ,
Since the process Z
Z
Z
Z
e
e n )(s).
g(s) dZn (s) = g(s) dZn (s) + g(s) d(Zn − Z

is well defined for any right-continuous function g of bounded variation. Using the
integration by parts formula (Lemma A in the appendix) we obtain for the last term
on the right
Z
Z
Z Z
e
e
e n )(y).
g(s) d(Zn − Zn )(s) = − (Zn − Zn )(s) dg(s) +
1x=y dg(x) d(Zn − Z

Since g is of bounded variation, it has countably many discontinuities ai . Using (3),
we obtain
Z Z
e n )(y)
1x=y dg(x) d(Zn − Z

≤

X
i

−
e
e
|g(ai ) − g(a−
i )| (Zn − Zn )(ai ) − (Zn − Zn )(ai )

≤ 2 kgkT V Ψ2β,F0 (Zn ).
14

Consequently,

Z

e n )(s) ≤ 3 kgk Ψ2β,F (Zn )
g(s) d(Zn − Z
0
TV

Next we deal with the finite dimensional approximation
Z
M
X
e n (s) =
g(s)dZ
g(sj )(Zn (sj ) − Zn (sj−1)).
j=1

Clearly,
Z
M
M
X
X
e n (s) ≤
g(sj )(Zn (sj ) − Zn (s−
g(sj )(Zn (s−
)
−
Z
(s
))
+
g(s)dZ
n j−1
j ))
j

(4)

j=1

j=1

For the first term in (4) we introduce the step function
∗

g (t) =

M
X

1(sj−1 ,sj ] (t)

j=1

inf

sj−1 <s≤sj

|g(s)|.

designed to approximate |g(t)|. Clearly 0 ≤ g ∗(t) ≤ |g(t)| and
M
X
j=1

∗

g (sj ) ≤ β

−1

j=1

= β −1
≤ β

−1

Hence, by (3) we have
M
X
j=1

g(sj )(Zn (s−
j ) − Zn (sj−1 ))

M
X

Z

Z

g ∗ (sj )(F0 (sj ) − F0 (sj−1 ))

g ∗ dF0
|g| dF0 = β −1 ||g||L1(F0 )

≤ Ψ2β,F0 (Zn )

"M
X
j=1

(|g(sj )| − g ∗ (sj )) +

≤ Ψ2β,F0 (Zn )(kgkT V + β

−1

||g||L1(F0 ) )

M
X
j=1

g ∗ (sj ))

#
(5)

For the second term in (4), we have
M
X
j=1

≤

g(sj ) Zn (sj )) − Zn (s−
j )
X

j: F0 (sj )−F0 (s−
j )≤β



g(sj ) Zn (sj )) − Zn (s−
j )




+

X

j: F0 (sj )−F0 (s−
j )>β

≤ Ψ2β,F0 (Zn ) 2||g||T V + β −1 ||g||L1(F0 ) + ðβ,F0 (Zn )β −1 kgkL1(F0 ) ,
15

g(sj ) Zn (sj )) − Zn (s−
j )



(6)

using for the last inequality
M
X
j=1

|g(sj )| ≤ 2kgkT V + β −1 kgkL1(F0 )

and
M
X
j=1

|g(sj )|1{F0(sj ) −

F0 (s−
j )

> β} ≤ β

−1

≤ β

−1

M
X
j=1

|g(sj )||(F0(sj ) − F0 (s−
j )|

kgkL1 (F0 ) .

Lemma 10 now follows by combining the estimates (3) and (4) and (5).
An immediate corollary is the following result.
Corollary 11 For any distribution function F0 and for all T < ∞, δ > 0 and p ≥ 1,
we have

sup
g

Z

√
√
g dZn ≤ (2 δ + 6T )Ψ2√δ,F0 (Zn ) + ð√δ,F0 (Zn ) δ,

where the sup is taken over all right-continuous functions g with kgkT V ≤ T and
1/p
R p
kgkLp (F0 ) =
|g| dF0
≤ δ.
√
Proof. The proof follows trivially from Lemma 10 by taking β = δ and observing
that kgkL1 (F0 ) ≤ kgkLp (F0 ) for p ≥ 1.

Proof of Theorem 1.
First we recall, see, for instance, Chapter 1.5 in Van der Vaart and Wellner (1996),
that {Z̄n (g), g ∈ G} converges weakly to a tight limit in ℓ∞ (G), provided
(a) the marginals (Z̄n (g1 ), . . . , Z̄n (gk )) converge weakly for every finite subset g1 , . . . , gk ∈
G, and

(b) there exists a semi-metric ρ on G such that (G, ρ) is totally bounded and Z̄n (g)
is ρ-stochastically equicontinuous, that is,
(
lim lim sup P
δ↓0

n→∞

sup |Z̄n (f ) − Z̄n (g)| > ε

ρ(f,g)≤δ

for all ε > 0.
16

)

=0

The finite dimensional convergence (a) follows trivially from Lemma 9, linearity of the
process Zn (g) and the Cramèr-Wold device. As for the stochastic equicontinuity (b) of
Z n (g), it is sufficient to show that, for every ε > 0,
)
(
Z
h(t) dZn (t) > ε = 0,
lim lim sup P
sup
δ→0

n→∞

khkL

1 (F0 )

≤δ

where the supremum is taken over all differences h = g − g ′ with g, g ′ ∈ G and
R
khkL1 (F0 ) = |h|dF0 ≤ δ. Since h is also right-continuous and khkT V ≤ 2T , Corol-

lary 11 implies that
Z
√
√
sup
h(t) dZn (t) ≤ (2 δ + 12T )Ψ2√δ,F0 (Zn ) + ð√δ,F0 (Zn ) δ.
khkL

1 (F0 )

≤δ

Let ft (x) = 1{x ≤ t}, so that Zn (t) = Zn (ft ) and
d(s, t) = |F0 (s) − F0 (t)|
and observe that
sup |Zn (s) − Zn (t)| = Ψδ,F0 (Zn ).

d(s,t)≤δ

The weak convergence of Zn (t), or equivalently, Zn (ft ), to a continuous (with respect
to d(., .)) process Z(t) implies that
P

Ψ2√δ,F0 (Zn ) → 0 as δ → 0 and n → ∞.
Moreover, the weak convergence of Zn (t) implies that ð√δ,F0 (Zn ) is bounded in proba√
bility, so ð√δ,F0 (Zn ) δ → 0 as δ → 0 and n → ∞.
Summarizing, Z̄n (g), converges for each g ∈ G to a Gaussian limit and Z̄n is uniformly

L1 (F0 )-equicontinuous, in probability. Moreover, (G, L1 (F0 )) is totally bounded (for
any distribution F0 ). This well-known fact can be found in Example 2.6.21 in Van der
Vaart and Wellner (1996, page 149). It implies the weak convergence of Z̄n (g) to a
process with uniformly L1 (F0 )−continuous sample paths (see Theorems 1.5.4 and 1.5.7
in Van der Vaart and Wellner 1996).

17

Proof of Theorem 6.
Here we only prove the “in probability” case. The almost sure statement follows after
straightforward changes in the proof of Theorem 1. A simple modification of Lemma
9 yields

Z

g(t) dG∗n (t) = T1 (G∗n ) + T2 (G∗n )

for the same operators T1 and T2 as defined in Lemma 9. Since G∗n converges to a
Gaussian limit the finite dimensional convergence follows by repeating the computation
presented in the proof of Lemma 9 after replacing Gn (t) with G∗n (t). As for stochastic
equicontinuity of Z∗n (g) we find, analogous to Corollary 11, that for δ > 0
Z
√
√
g dG∗n ≤ (2 δ + 12T )Ψ2√δ,F0 (G∗n ) + ð√δ,F0 (G∗n ) δ.
sup
kgkLp (F0 ) ≤δ, kgkT V ≤T

Now the weak convergence of Z∗n follows from the convergence of G∗n , as in the proof of
Theorem 1, conditionally given the sample X1 , . . . , Xn . Moreover, if Gn (t) and G∗n (t)
converge to the same Gaussian process, then Lemma 9 coupled with the Cramèr-Wold
device implies that the finite dimensional distribution of the limiting process of Zn (g)
and Z∗n (g) are the same. This concludes the proof.

A

Appendix

For completeness, we state the following classical result and give a simple elementary
proof which was communicated to us by David Pollard. Theorem 18.4 in Billingsley
(1986) states the result for functions on a bounded interval, yet its proof can be easily
extended to the entire real line.
Lemma A. Let f and g be right-continuous functions of bounded variation and define
measures µ and ν as µ(−∞, x] = f (x) − f (−∞) and ν(−∞, y] = g(y) − g(−∞). Then
Z
Z
Z Z
f (x) dg(x) + g(x) df (x) = (f g)(∞) − (f g)(−∞) +
1x=y dµ(x) dν(y).
18

Moreover, if either f (±∞) = 0 or g(±∞) = 0, then
Z
Z
Z Z
f (x) dg(x) + g(x) df (x) =
1x=y dµ dν.
Proof. Set H(x, y) = 1{x ≤ y} and observe that by the very definition of Lebesgue

integral

f (y) =

Z

H(x, y) dµ(x) + f (−∞)

g(x) =

Z

H(y, x) dν(y) + g(−∞).

and

Hence

Z

Z

f (y) dν(y) + g(x) dµ(x)


Z Z
Z Z
=
H(x, y) dµ(x) dν(y) +
H(y, x) dν(y) dµ(x)
+f (−∞)(g(∞) − g(−∞)) + g(−∞)(f (∞) − f (−∞))

Next we apply Fubini


Z Z
Z Z
H(x, y) dµ(x) dν(y) +
H(y, x) dν(y) dµ(x)
Z Z
Z Z
=
(H(x, y) + H(y, x)) dµ(x) dν(y) =
(1x≤y + 1y≤x ) dµ(x) dν(y)
Z Z
Z Z
=
dµ(x) dν(y) +
1x=y dµ(x) dν(y)
to prove the lemma.

Acknowledgements.
We would like to thank both referees for their careful reading and many constructive
remarks.
The research of Wegkamp was supported in part by NSF grants DMS-1310119 and
DMS-1712709.

19

References
[1] D.W.K. Andrews and D.P. Pollard (1994). An Introduction to Functional Central Limit Theorems for Dependent Stochastic Processes. International Statistical
Review, 62, 119–132.
[2] M.A. Arcones, and B. Yu (1994). Central limit theorems for empirical and Uprocesses of stationary mixing sequences. Journal of Theoretical Probability, 7,
47–71.
[3] P. Billingsley (1968). Convergence of probability measures. J. Wiley & Sons, New
York.
[4] P. Billingsley (1986). Probability and Measure, 2nd edition. J. Wiley & Sons, New
York.
[5] P. Bühlmann (1995). The blockwise bootstrap for general empirical processes of
stationary sequences. Stochastic Processes and their Applications, 58, 247–265.
[6] R. M. Dudley, (1999), Uniform Central Limit Theorems. Cambridge University
Press.
[7] H. Dehling, O. Durieu and D. Volny (2009). New techniques for empirical processes
of dependent data. Stochastic Processes and their Applications, 119, 3699–3718.
[8] H. Dehling, O. Durieu and M. Tusche (2014). Approximating class approach for
empirical processes of dependent sequences indexed by functions. Bernoulli, 20,
1372–1403.
[9] H. Dehling and M.S. Taqqu (1989). The empirical process of some long-range
dependent sequences with an application to U-statistics. Annals of Statistics, 17,
1767–1783.
[10] P. Doukhan, and D. Surgailis (1998). Functional central limit theorem for the
empirical process of short memory linear processes. Comptes Rendus de l’Académie
des Sciences, Mathématique, 326, 87–92.
20

[11] P. Doukhan, P. Massart, and E. Rio (1995). Invariance principles for absolutely
regular empirical processes. Annales de l’Institute Henri Poincaré Probababilités
et Statististiques, 31, 393–427.
[12] F. El Ktaibi, B.G. Ivanoff, N.C. Weber (2014). Bootstrapping the empirical distribution of a linear process. Statistics & Probability Letters, 93, 134–142.
[13] T.H. Hildebrandt (1963). Introduction to the Theory of Integration. Academic
Press, New York, London.
[14] H.C. Ho and T. Hsing (1996). On the asymptotic expansion of the empirical process of long-memory moving averages. Annals of Statistics, 24, 992–1024.
[15] R.Y. Liu and K. Singh (1992), Moving blocks jackknife and bootstrap capture
weak dependence, in: R. LePage and L. Billard, eds., Exploring the Limits of
Bootstrap, Wiley, New York, pp. 225-248.
[16] H.R. Künsch (1989), The jackknife and the bootstrap for general stationary observations, Annals of Statistics, 17, 1217-1241.
[17] M. Peligrad (1998). On the blockwise bootstrap for empirical processes for stationary sequences. Annals of Probability, 26, 887–901.
[18] D. Radulović (1996). The bootstrap for empirical processes based on stationary
observations. Stochastic Processes and Their Applications, 65, 259–279.
[19] D. Radulović, M.H. Wegkamp and Y. Zhao (2017). Weak convergence of empirical
copula processes indexed by functions. Bernoulli, 23, 3346-3384.
[20] E. Rio (1998). Processus empiriques absolument réguliers et entropie universelle.
Probability Theory and Related Fields, 111, 585–608.
[21] E. Rio (2000). Théorie asymptotique des processus aléatoires faiblement dependants, Collection Mathématiques and Applications, 31. Springer.

21

[22] S. Sengupta, S. Volgushev and X. Shao (2016). A Subsampled Double Bootstrap
for Massive Data, Journal of the American Statistical Association, 111, 1222–
1232.
[23] G.R. Shorack and J.A. Wellner (2009). Empirical Processes with Applications to
Statistics, 2nd Edition, SIAM.
[24] A.W. van der Vaart and J. Wellner (2000). Weak convergence and empirical processes. Springer Series in Statistics. Springer.
[25] D. Viennet (1997). Inequalities for absolutely regular sequences: application to
density estimation. Probability Theory and Related Fields, 107, 467–492.

22

