Strong approximations for the p-fold integrated empirical process
with applications to statistical tests
Sergio Alvarez-Andrade∗, Salim Bouzebda† and Aimé Lachal‡

arXiv:1611.04513v2 [] 3 Nov 2017

∗†

‡

Sorbonne Universités, Université de Technologie de Compiègne
Laboratoire de Mathématiques Appliquées de Compiègne
Université de Lyon, Institut National des Sciences Appliquées de Lyon
Institut Camille Jordan

Abstract
The main purpose of this paper is to investigate the strong approximation of the p-fold integrated
empirical process, p being a fixed positive integer. More precisely, we obtain the exact rate of the
approximations by a sequence of weighted Brownian bridges and a weighted Kiefer process. Our
arguments are based in part on the Komlós et al. (1975)’s results. We obtain an exponential bound
for the tail probability of the weighted approximation to the p-fold integrated empirical process.
Applications include the two-sample testing procedures together with the change-point problems.
We also consider the strong approximation of integrated empirical processes when the parameters
are estimated. We study the behavior of the self-intersection local time of the partial sum process
representation of integrated empirical processes. Finally, simulation results are provided to illustrate
the finite sample performance of the proposed statistical tests based on the integrated empirical
processes.
Key words: Integrated empirical process; Brownian bridge; Kiefer process; Rates of convergence;
Local time; Two-sample problem; Hypothesis testing; Goodness-of-fit; Change-point.
AMS Classifications: primary: 62G30; 62G20; 60F17; secondary: 62F03; 62F12; 60F15.

1

Introduction

Let F = {F (t), t ∈ R} be a continuous distribution function [d.f.] and denote by Q = {Q(u), u ∈ [0, 1]}
the usual quantile function (generalized inverse) pertaining to F defined as
Q(u) := inf{t ∈ R : F (t) ≥ u} for u ∈ (0, 1),
Q(0) := lim Q(u) and Q(1) := lim Q(u).
u↓0

u↑1

The function Q is strictly increasing and we have F (Q(u)) = u for any u ∈ [0, 1]. We denote the
sets {0, 1, 2, . . .} and {1, 2, . . .} respectively by N and N∗ . Consider now a sequence of independent,
identically distributed [i.i.d.] random variables [r.v.’s] {Ui : i ∈ N∗ } uniformly distributed on [0, 1] and,
for each i ∈ N∗ , set Xi := Q(Ui ). The sequence {Xi : i ∈ N∗ } consists of i.i.d. r.v.’s with d.f. F :
F (t) = P{X1 ≤ t} for t ∈ R (cf., e.g., Shorack and Wellner (1986), p. 3 and the references therein).
Moreover, we conversely have Ui = F (Xi ).
∗

e-mail: sergio.alvarez@utc.fr
e-mail: salim.bouzebda@utc.fr
‡
e-mail: aime.lachal@insa-lyon.fr
†

1

For each n ∈ N∗ , let Fn and Un be the empirical d.f.’s based upon the respective samples X1 , . . . , Xn
and U1 , . . . , Un defined by
n

Fn (t) :=
Un (u) :=

1
1X
#{i ∈ {1, . . . , n} : Xi ≤ t} =
1{Xi ≤t}
n
n
1
1
#{i ∈ {1, . . . , n} : Ui ≤ u} =
n
n

i=1
n
X

for t ∈ R,

1{Ui ≤u} for u ∈ [0, 1],

i=1

where # denotes cardinality. For each n ∈ N∗ , we introduce the empirical process αn and the uniform
empirical process βn defined by
√
αn (t) := n (Fn (t) − F (t)) for t ∈ R,
(1.1)
√
βn (u) := n (Un (u) − u) for u ∈ [0, 1].
(1.2)
We have of course the usual relations between the empirical process and uniform empirical process:
αn (t) = βn (F (t)) for t ∈ R, n ∈ N∗ ,

(1.3)
∗

βn (u) = αn (Q(u)) for u ∈ [0, 1], n ∈ N .

(1.4)

In this paper, we consider integrated empirical d.f.’s based upon the samples X1 , . . . , Xn and
U1 , . . . , Un together with the corresponding integrated empirical processes in the following sense.
Definition 1.1 We define the families of integrated d.f.’s and integrated empirical d.f.’s associated with
the d.f. F , for any p ∈ N, any n ∈ N∗ and any t ∈ R, as
F (0) (t) := F (t),
F (1) (t) :=

Z

F(0)
n (t) := Fn (t),
Z t
F(1)
(t)
:=
Fn (s) dFn (s),
n

t

F (s) dF (s),
−∞

−∞

and for p ≥ 2,
F

(p)

Z

t

Z

(t) :=
Z

t

−∞

sp−1

F (sp ) dF (sp ),

−∞

Z

−∞

s1

Z

sp−1

dFn (s2 )· · ·

dFn (s1 )

:=

Z
dF (s2 )· · ·

dF (s1 )
−∞

F(p)
n (t)

s1

Fn (sp ) dFn (sp ),
−∞

−∞

together with the corresponding family of integrated empirical processes as

√ 
(p)
(t) .
αn(p) (t) := n F(p)
n (t) − F

(1.5)

(p)

Notice that F (p) (resp. Fn ) is a kind of p-fold integral with respect to the measure dF (resp. dFn ).
(p)
(p)
Hence, we will call F (p) (resp. Fn , αn ) throughout the paper p-fold integrated d.f. (resp. p-fold
integrated empirical d.f., p-fold integrated empirical process). Finally, we define exactly in the same
(p)
manner the p-fold integrated uniform empirical d.f. Un and the p-fold integrated uniform empirical
(p)
process βn .
(p)
Below, we provide explicit expressions for F (p) and Fn , the proof of which are postponed to Section 9.
Proposition 1.2 For each p ∈ N, we explicitly have, with probability 1,


1
F (t)p+1
nFn (t) + p
(p)
(p)
F (t) =
, Fn (t) = p+1
for t ∈ R, n ∈ N∗ .
(p + 1)!
n
p+1
2

(1.6)

0.0

0.0

0.2

0.2

0.4

0.4

Fn

(p)

F(p)(t)

(t)

0.6

0.6

0.8

0.8

1.0

1.0

n = 100

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

t

0.6

0.8

1.0

t

0.0
−1.5

−1.0

−0.5

αn

(p)

(0.8)

0.5

1.0

1.5

Figure 1: On the left: the theoretical functions F (p) , p = 0, 1, 2, 3; on the right: the empirical counter(1)
(2)
(3)
parts for the uniform distribution. (“–”: Fn , F ), (“–”: Fn , F (1) ), (“–”: Fn , F (2) ), (“–”: Fn , F (3) )

0

2000

4000

6000

8000

10000

n

Figure 2: The uniform empirical processes for p = 0, 1, 2, 3 as functions of the sample size. (“–”: αn , ),
(1)
(2)
(3)
(“–”: αn ,), (“–”: αn ,), (“–”: αn )

3

The particular case where p = 1 has often been considered in the literature. Henze and Nikitin (2000,
2002) introduced and deeply investigated the goodness-of-fit testing procedures based on the integrated
empirical process. Indeed, the asymptotic properties of their procedures, Kolmogorov-Smirnov, Cramérvon Mises and Watson-type statistics, can be derived from the limiting behavior of the integrated
empirical process. Henze and Nikitin (2003) considered a two-sample testing procedure and focused on
the approximate local Bahadur efficiencies of their statistical tests. It is noteworthy to point out that
tests based on some integrated empirical processes turn out to be more efficient for certain distributions,
we may refer at this point to Henze and Nikitin (2003, 2002, 2000). In Henze and Nikitin (2003), it
is shown that the statistics based on the integrated empirical processes perform better the classical
ones, under asymmetric alternatives. For instance, Kolmogorov-Smirnov test based on these integrated
empiricals has maximal Bahadur efficiency if the underlying distribution is skew-Laplace. We mention
also that in the paper Henze and Nikitin (2000) where a new approach to goodness-of-fit testing was
proposed by using integrated empirical processes. In this paper, the authors have established that the
integrated Kolmogorov- Smirnov test is locally Bahadur optimal for the logistic distribution and the
statistic
Z 

√
(p)
n
F(p)
(t)
−
F
(t)
dF0 (t), for p = 1,
n
0
R

turns out to be locally Bahadur optimal for the “root-logistic distribution”. In the paper by Henze and
Nikitin (2002), one of the new tests is locally Bahadur optimal for the hyperbolic cosine distribution, we
may refer to Theorem 6.1 therein for exact formulation. In Lachal (2001), another version of the p-fold
integrated empirical process (p ∈ N∗ ) was introduced. For the extension to the multivariate framework,
we may refer to Jing and Wang (2006) and Jing and Yang (2007) where some projected integrated
empirical processes for testing the equality of two multivariate distributions are considered. Inspired
by the work of Henze and Nikitin (2003), Bouzebda and El Faouzi (2012) developed multivariate twosample testing procedures based on the integrated empirical copula process that are extended to the
K-sample problem in Bouzebda et al. (2011). Emphasis is placed on the explanation of the strong
approximation methodology. The asymptotic behavior of weighted multivariate Cramér-von Mises-type
statistics under contiguous alternatives was characterized by Bouzebda and Zari (2013). For more recent
references, we refer to Durio and Nikitin (2016), Bouzebda (2016) and Alvarez-Andrade et al. (2017).
The main purpose of this paper is to investigate the strong approximation of the p-fold integrated
empirical process. Next we use the obtained results for studying the asymptotic properties of statistical
tests based on this process. We point out that strong approximations are quite useful and have received
considerable attention in probability theory. Indeed, many well-known and important probability theorems can be considered as consequences of results about strong approximation of sequences of sums by
corresponding Gaussian sequences.
We will first obtain an upper bound in probability for the distance between the p-fold integrated
empirical process and a sequence of appropriate Brownian bridges (see Theorem 2.3). This is the key
point of our study. From this, we will deduce a strong approximation of the p-fold integrated empirical process by this sequence of transformed Brownian bridges (see Corollary 2.5). As an application,
we will derive the rates of convergence for the distribution of smooth functionals of each p-fold integrated empirical process (see Corollary 2.4). Moreover, we will deduce strong approximations for
the Kolmogorov-Smirnov and Cramér-von Mises-type statistics associated with the p-fold integrated
empirical processes (see Corollary 2.8).
Second, we will obtain a strong approximation of the p-fold integrated empirical process by a transformed Kiefer processes (see Theorem 2.6). This latter is of particular interest; indeed, for instance, any
kind of law of the iterated logarithm which holds for the partial sums of Gaussian processes may then be
transferred to the p-fold integrated empirical processes (see Corollary 2.7). We may refer to DasGupta
(2008) (Chapter 12), Csörgő and Horváth (1993) (Chapter 3), Csörgő and Révész (1981) (Chapters
4-5) and Shorack and Wellner (1986) (Chapter 12) for expositions, details and references about this
4

problem.
Third, we will apply our theoretical results to some statistical applications. More precisely, we will
consider the famous two-sample and change-point problems for which we will develop procedures based
on statistics involving p-fold integrated empirical distribution functions.
We refer to Csörgő and Hall (1984), Csörgő (2007) and Mason and Zhou (2012) for a survey of
possible applications of the strong approximation and many references. There is a huge literature on
the strong approximations and their applications. It is not the purpose of this paper to survey this
extensive literature.
The layout of the article is as follows. In Section 2, we first present some strong approximation results
for the p-fold integrated empirical process; our main tools are the results of Komlós et al. (1975). In
section 3, we consider weighted approximations. Sections 4 and 5 are devoted to statistical applications,
namely the two-sample and change-point problems respectively. In Section 6, we deal with the strong
approximation of the p-fold integrated empirical process when parameters are estimated. Section 7 is
concerned with the behavior of the self-intersection local time of the partial sum process representation
of the p-fold integrated empirical process. In section 8, simulation results are performed in order to
illustrate the finite sample performances of the proposed statistical tests for testing uniformity based
on the integrated empirical processes. Finally, in the Appendix, we suggest some possible extensions of
our work for further investigations.
To prevent from interrupting the flow of the presentation, all mathematical developments are postponed to Section 9.

2
2.1

Strong approximation
Some processes

First, we introduce some definitions and notations. Let W = {W(s) : s ≥ 0} and B = {B(u) : u ∈ [0, 1]}
be the standard Wiener process and Brownian bridge, that is, the centered Gaussian processes with
continuous sample paths, and covariance functions
E(W(s)W(t)) = s ∧ t for s, t ≥ 0
and
E(B(u)B(v)) = u ∧ v − uv

for u, v ∈ [0, 1].

A Kiefer process K = {K(s, u) : s ≥ 0, u ∈ [0, 1]} is a two-parameters centered Gaussian process, with
continuous sample paths, and covariance function
E(K(s, u)K(t, v)) = (s ∧ t) (u ∧ v − uv) for s, t ≥ 0 and u, v ∈ [0, 1].
It satisfies the following distributional identities:
L

{K(s, u) : u ∈ [0, 1]} =
and

L

{K(s, u) : s ≥ 0} =

√

s B(u) : u ∈ [0, 1]

np
o
u(1 − u) W(s) : s ≥ 0

L

for s ≥ 0

for u ∈ [0, 1],

where = stands for the equality in distribution. The interested reader may refer to Csörgő and Révész
(1981) for details on the Gaussian processes mentioned above.

5

2.2

Brownian approximation

It is well-known that the empirical uniform process {βn : n ∈ N∗ } converges to B in D[0, 1] (the space
of all right-continuous real-valued functions defined on [0, 1] which have left-hand limits, equipped with
the Skorohod topology; see, for details, Billingsley (1968)). The rate of convergence of this process to B
is an important task in statistics as well as in probability that has been investigated by several authors.
We can and will assume without loss of generality that all r.v.’s and processes introduced so far and
later on in this paper can be defined on the same probability space (cf. Appendix 2 in Csörgő and
Horváth (1993)).
In Csörgő et al. (1986), it was stated the following Brownian bridge approximation for {βn : n ∈ N∗ }
(Formula (2.2)), along with a description of its proof with few details, which has been subsequently
refined by Mason and van Zwet (1987) (Formula (2.1)).
Theorem A On a suitable probability space, we may define the uniform empirical process {βn : n ∈ N∗ },
in combination with a sequence of Brownian bridges {Bn : n ∈ N∗ }, such that, for any d, n ∈ N∗ satisfying
d ≤ n and any positive number x,


1
P
sup |βn (u) − Bn (u)| ≥ √ (c1 log d + x) ≤ c2 exp(−c3 x)
(2.1)
n
u∈[0,d/n]
where c1 , c2 and c3 are suitable absolute constants. The same inequality holds when replacing the interval
[0, d/n] by [1 − d/n, 1]. In particular, for d = n,


1
P sup |βn (u) − Bn (u)| ≥ √ (c1 log n + x) ≤ c2 exp(−c3 x).
(2.2)
n
u∈[0,1]
In (2.2), suitable explicit values for c1 , c2 , c3 were exhibited by Bretagnolle and Massart (1989), Theorem 1: c1 = 12, c2 = 2, c3 = 1/6. In his manuscript, Major (2000) details the original proof of (2.2).
Chatterjee (2012) provided a new alternative approach for proving the famous KMT theorem.
Remark 2.1 In the sequel, the precise meaning of “suitable probability space” is that an independent
sequence of Wiener processes, which is independent of the originally given sequence of i.i.d. r.v.’s,
can be constructed on the assumed probability space. This is a technical requirement which allows the
construction of the Gaussian processes displayed in our theorems, and which is not restrictive since one
can expand the probability space to make it rich enough (see, e.g., Appendix 2 in Csörgő and Horváth
(1993), de Acosta (1982), Csörgő and Révész (1981) and Lemma A1 in Berkes and Philipp (1979)).
Throughout this paper, it will be assumed that the underlying probability spaces are suitable in this sense.
In the following theorem, we state the key point to access the strong Brownian approximation of
 (p)
the p-fold integrated uniform empirical process βn : n ∈ N∗ . The following Theorem can be seen
as a version of Csörgő et al. (1986) for iterated processes.
Theorem 2.2 Fix p ∈ N∗ . On a suitable probability space, we may define the p-fold integrated uniform
 (p)
empirical process βn : n ∈ N∗ , in combination with a sequence of Brownian bridges {Bn : n ∈ N∗ },
such that, for any d, n ∈ N∗ satisfying d ≤ n and large enough x,


p+1


X
1
(p)
(p)
P
sup βn (u) − Bn (u) ≥ √ (c1 log d + x) ≤ Bp
exp −Cp x2/k n1−2/k
(2.3)
n
u∈[0,d/n]
k=2

where Bp and Cp are positive constants depending on p, c1 is the constant arising in (2.2) and, for each
(p)
n ∈ N∗ , Bn is the process defined by
B(p)
n (u) :=

1 p
u Bn (u)
p!
6

for

u ∈ [0, 1].

The same inequality holds when replacing the interval [0, d/n] by [1 − d/n, 1].
In particular, making d = n in (2.3), we obtain the key estimate for the p-fold integrated empirical
 (p)
process αn : n ∈ N∗ below.
Theorem 2.3 Fix p ∈ N∗ . On a suitable probability space, we may define the p-fold integrated empirical
(p)
process {αn : n ∈ N∗ }, in combination with a sequence of Brownian bridges {Bn : n ∈ N∗ }, such that,
for large enough x and all n ∈ N∗ ,


p+1


X
1
(p)
(p)
P sup αn (t) − Bn (F (t)) ≥ √ (c1 log n + x) ≤ Bp
exp −Cp x2/k n1−2/k .
(2.4)
n
t∈R
k=2

One of the immediate consequences of Theorem 2.3 is an upper bound for the convergence of distributions
(p)
of smooth functionals of αn . Indeed, applying (2.4) with x = c log n for a suitable constant c yields
the result below. Notice that the following corollary is the analogous of the Corollary of Komlós et al.
(1975) page 113. Let D(A) be the space of right-continuous real-valued functions defined on A which
have left-hand limits, equipped with the Skorohod topology; refer to Billingsley (1968) for further details
on this problem.
Corollary 2.4 Fix p ∈ N∗ . Let B be a Brownian bridge and B(p) the process defined by
B(p) (u) :=

1 p
u B(u)
p!

for

u ∈ [0, 1].

Let Φ(·) be a functional defined on the space D(R), satisfying a Lipschitz condition
|Φ(v) − Φ(w)| ≤ L sup |v(t) − w(t)|.
t∈R


Assume further that the distribution of the r.v. Φ B(p) (F (·)) has a bounded density. Then, as n → ∞,


n
o
n
o


log n
(p)
(p)
sup P Φ αn (·) ≤ x − P Φ B (F (·)) ≤ x = O √
.
(2.5)
n
x∈R
For more comments on this kind of results, we may refer to Csörgő et al. (2000), Corollary 1.1 and
p. 2459.
By applying (2.4) to x = c0 log n for a suitable constant c0 and appealing to Borel-Cantelli lemma,
 (p)
one can obtain the following almost sure approximation of the process αn : n ∈ N∗ based on a
sequence of Brownian bridges.
Corollary 2.5 The following bound holds, with probability 1, as n → ∞:


log n
(p)
(p)
sup αn (t) − Bn (F (t)) = O √
.
(2.6)
n
t∈R
 (p)
The next result yields an almost sure approximation for αn : n ∈ N∗ based on a Kiefer process.
Theorem 2.6 On a suitable probability space, we may define the p-fold integrated empirical process
 (p)
αn : n ∈ N∗ , in combination with a Kiefer process {K(s, u) : s ≥ 0, u ∈ [0, 1]}, such that, with
probability 1, as n → ∞,
√ (p)

max sup k αk (t) − K(p) (k, F (t)) = O (log n)2
1≤k≤n t∈R

where K(p) is the process defined by
K(p) (s, u) :=

1 p
u K(s, u)
p!
7

for

s ≥ 0, u ∈ [0, 1].

Let us mention that the “extracted” Kiefer process {K(n, u) : n ∈ N∗ , u ∈ [0, 1]} may be viewed as the
partial sums process of a sequence of independent Brownian bridges {Bi : i ∈ N∗ }:
K(n, u) =

n
X

Bi (u) for n ∈ N∗ , u ∈ [0, 1].

i=1

From Theorem 2.6, we deduce the following law of iterated logarithm (“a.s.” stands for “almost
surely”).
Corollary 2.7 We have the following law of iterated logarithm for the p-fold integrated empirical process:
(p)

lim sup
n→∞

supt∈R αn (t)
(p + 1/2)p+1/2
√
=
p! (p + 1)p+1
log log n

a.s.

(2.7)

As a direct application of (2.6) and (2.7) to the problem of goodness-of-fit, for testing the null
hypothesis
H0 : F = F0 ,
we can use the following statistics: the p-fold integrated Kolmogorov-Smirnov statistic

√  (p)
(p)
S(p)
:=
sup
n
F
(t)
−
F
(t)
n
n
0
t∈R

as well as the p-fold integrated Cramér-von Mises statistic
Z 
2
(p)
(p)
(p)
Tn := n
Fn (t) − F0 (t) dF0 (t).
R

Corollary 2.8 Under H0 , with probability 1, as n → ∞, we have


log n
(p)
√
,
S(p)
−
sup
B
(F
(t))
=
O
0
n
n
n
t∈R
!
r
Z


log
log
n
2
T(p)
B(p)
log n .
n −
n (F0 (t)) dF0 (t) = O
n
R

(2.8)

(2.9)

We finish this part by pointing out the possibility of considering the statistics, for r ≥ 1,
Z
1/r
r
√
(p)
(p)
ωn,p,r = n
Fn (t) − F0 (t) dF0 (t)
.
R

It is clear, however, that we have the following convergence in distribution as n → ∞, under H0 :
Z
1/r
Z
1/r
r
r
√
(p)
(p)
(p)
n
Fn (t) − F0 (t) dF0 (t)
−→
B (F0 (t)) dF0 (t)
.
R

R

Denoting by X(1) ≤ X(2) ≤ · · · ≤ X(n) the order statistics of X1 , X2 , . . . , Xn , and putting X(0) =
−∞, X(n+1) = +∞, straightforward manipulations of integrals yield the alternative representation, for
p = 0 and r ≥ 1,
(n+1 

√
X
n
i−1
i−1 r
ωn,0,r =
F
(X
)
−
F
(X
)
−
0
0
(i)
(i)
n
n
(r + 1)1/r i=1



1/r
i−1
i−1 r
− F0 (X(i−1) ) −
F0 (X(i−1) ) −
.
n
n
In a future research, it would be of interest to deeply investigate such statistics.
8

3

Weighted approximations

1/2−ν
(p)
(p)
In this part, we consider the weighted difference βn (u) − Bn (u) / u(1 − u)
for a power ν ∈
[0, 1/2). Since this quantity is not defined at 0 and 1, we will work with the supremum on an interval
of the form [d/n, 1 − d/n] for d, n ∈ N∗ such that d ≤ n/2, n ≥ 2. The weighted approximations for the
classical uniform empirical process were deeply studied in the papers by Csörgő et al. (1986), Haeusler
and Mason (1999), Mason (1991), Mason (2001) and Mason and van Zwet (1987). For more details on
the subject we may refer to Csörgő and Horváth (1993).
We state here the analogue of Theorem 1.2 of Mason (2001).
Theorem 3.1 Fix p ∈ N∗ . On a suitable probability space, we may define the p-fold uniform integrated
 (p)
empirical process βn : n ∈ N∗ , in combination with a sequence of Brownian bridges {Bn : n ∈ N∗ }
and a positive constant Cp0 , such that, for every ν ∈ [0, 1/2), there exist a positive constant Bp,ν for
which we have, for any d, n ∈ N∗ satisfying d ≤ n/2 and large enough x,

p+1
(p)
(p)


X
nν βn (u) − Bn (u)
0 2/k 2ν/k 1−(1+2ν)/k
P
sup
≥
x
≤
B
exp
−C
x
d
n
.
p,ν
1/2−ν
p
u∈[d/n,1−d/n]
u(1 − u)
k=2


(3.1)

This theorem may be proved by appealing to Theorem 2.2. Applying (3.1) with x = c00 (log n)/n1/2−ν
for a suitable constant c00 yields the result below.
Corollary 3.2 On the same probability space of Theorem 3.1, we have, for any ν ∈ [0, 1/2) and any
d ∈ N∗ , with probability 1, as n → ∞,
sup
u∈[d/n,1−d/n]



(p)
(p)
βn (u) − Bn (u)
log n
ν
= O 1/2−ν .
n
u(1 − u)

By using Theorem 3.1 we have the following proposition.
Corollary 3.3 On the same probability space of Theorem 3.1, for all 0 ≤ ν < 1/2 there exists a constant
γ > 0 such that
!)
(
(p)
(p)
nν βn (u) − Bn (u)
sup
< ∞.
sup E exp γ
1/2−ν
n≥2
u∈[1/n,1−1/n]
u(1 − u)

4

The two-sample problem

For each m, n ∈ N∗ , let X1 , . . . , Xm and Y1 , . . . , Yn be independent random samples from continuous
(p)
(p)
d.f.’s F and G, respectively, and let Fm and Gn denote their p-fold integrated empirical d.f.’s. Tests
for the null hypothesis
H00 : F = G,
can be based on the p-fold integrated two-sample empirical process defined, for each m, n ∈ N∗ , by
r

mn  (p)
(p)
ξ m,n (t) :=
Fm (t) − G(p)
(t)
for t ∈ R.
n
m+n
Actually, as in Bouzebda and El Faouzi (2012), we will more generally consider the following modified
(p)
p-fold integrated two-sample empirical process (which includes the process ξ m,n ). Fix a positive integer q
which will serve as a power. We define, for each m, n ∈ N∗ ,
r
mn h (p) q  (p) q i
(p,q)
Fm (t) − Gn (t)
for t ∈ R.
ξ m,n (t) :=
m+n
9

Set also, for any m, n ∈ N∗ ,


log m log n
ϕ(m, n) := max √ , √
m
n

r
and φ(m, n) := max

log log m
log m,
m

r

!
log log n
log n .
n

Reasonable statistics for testing H00 would be the modified p-fold integrated Kolmogorov-Smirnov
statistic
(p,q)
Sm,n
:= sup ξ (p,q)
m,n (t)
t∈R

and the modified p-fold integrated Cramér-von Mises statistic
Z
2
ξ (p,q)
T(p,q)
:=
m,n (t) dF (t).
m,n
R

The following results are consequences of Corollary 2.5.
 (p,q)
Corollary 4.1 On a suitable probability
possible
to define ξ m,n : m, n ∈ N∗ , jointly with
 1 space, it is

two sequences of Brownian bridges Bm : m ∈ N∗ and B2n : n ∈ N∗ , such that, under H00 , with
probability 1, as min(m, n) → ∞,
(p,q)
sup ξ (p,q)
m,n (t) − Bm,n (t) = O(ϕ(m, n)),
t∈R

(p,q)

where, for each m, n ∈ N∗ , Bm,n is the Gaussian process defined by
r
r

n
m
(p + 1)q
pq+q−1
1
2
(p,q)
F (t)
B (F (t)) −
B (F (t))
Bm,n (t) :=
(p + 1)!q
m+n m
m+n n

for

t ∈ R.

Corollary 4.2 Under H00 , with probability 1, as min(m, n) → ∞, we have
(p,q)
S(p,q)
m,n − sup Bm,n (t) = O(ϕ(m, n)),
t∈R

T(p,q)
m,n −

Z

(p,q)
Bm,n
(t)2 dF (t) = O(φ(m, n)).

R

Remark 4.3 The family of statistics indexed by q may be used to maximize the power of the statistical
test for a specific alternative hypothesis as argued in Ahmad and Dorea (2001) in the case p = 1.
Now, we fix a positive integer K and we describe the more general K-sample
problem. For each

k ∈ {1, . . . , K}, we consider a setting made of independent observations Xik : i ∈ {1, . . . , nk } of a
real-valued r.v. X k . The d.f.’s of Xik , i ∈ {1, . . . , nk }, are denoted by F k and they are assumed to be
continuous. We would like to test, F0 being a fixed continuous d.f., the null hypothesis
H0K : F 1 = F 2 = · · · = F K = F0 .
P
For any K-tuple of positive integers n = (n1 , . . . , nK ), set |n| = K
k=1 nk and let
(Z1 , . . . , Z|n| ) := X11 , . . . , Xn11 , X12 , . . . , Xn22 , . . . , X1K , . . . , XnKK
(p)



be the pooled sample of total size |n|, DK,n be the p-fold integrated empirical d.f.

based upon

k,(p)
Fnk

Z1 , . . . , Z|n| , and, for each k ∈ {1, . . . , K},
be the p-fold integrated empirical d.f. based upon
X1k , . . . , Xnkk . Of course, we have the following identity:
K

(p)
DK,n

1 X
nk Fk,(p)
=
nk .
|n|
k=1

10

(4.1)

Next, we define the p-fold integrated K-sample empirical process in the following way: for any
K-tuple n = (n1 , . . . , nK ) ∈ (N∗ )K ,
(p)
ξ K,n (t)

:=

K
X


2
(p)
nk Fk,(p)
nk (t) − DK,n (t)

for t ∈ R.

k=1

Obvious candidates for testing Hypothesis H0K are the p-fold integrated K-sample Kolmogorov-Smirnov
statistic
(p)
(p)
SK,n := sup ξ K,n (t)
t∈R

and the p-fold integrated K-sample Cramér-von Mises functional (the usual square being included in
(p)
the definition of ξ K,n )
Z
(p)
(p)
ξ K,n (t) dF0 (t).
TK,n :=
R

Set

(s
φK (n) := max

1≤k≤K

)
log log nk
log nk .
nk

As a consequence of Corollary 2.5 and by using similar arguments to those used in Bouzebda et al.
(2011), we obtain the following results.
 (p)
Theorem 4.4 On a suitable probability space, it is possible to define ξ K,n : n ∈ (N∗ )K , jointly

with K sequences of Brownian bridges Bkm : m ∈ N∗ , k ∈ {1, . . . , K}, such that, under H0K , with
probability 1, for n = (n1 , . . . , nK ) such that min1≤k≤K nk → ∞,

(p)
(p)
sup ξ K,n (t) − BK,n (t) = O φK (n) ,
t∈R

(p)

where, for each n = (n1 , . . . , nK ) ∈ (N∗ )K , BK,n is the process defined by

!2 
K
K r
2p
X
X
2

nk k
F0 (t) 
(p)
Bknk F0 (t) −
Bnk F0 (t) 
BK,n (t) :=
2
p!
|n|
k=1

for

t ∈ R.

k=1

In the particular case K = 2 (i.e. the two-sample problem), the corresponding settings are related to
the previous ones according as

2

2
(p)
(p)
(p)
(p,1)
ξ 2,(n1 ,n2 ) (t) = ξ (p)
(t)
,
φ
((n
,
n
))
=
φ(n
,
n
),
S
=
S
, T2,(n1 ,n2 ) = T(p,1)
2
1 2
1 2
n1 ,n2
n1 ,n2
n1 ,n2 .
2,(n1 ,n2 )
(p)

Notice that BK,n (t) ≥ 0 for any t ∈ R and n ∈ (N∗ )K as it is easily seen with the aid of the Cauchy

(p)
(p,1) 2
Schwarz inequality. For K = 2, this process writes B2,(n1 ,n2 ) = Bn1 ,n2 , this is the square of a Gaussian
process.
(p)
The next result, which is an immediate consequence of the previous theorem (observe that SK,n
(p)

(p)

and TK,n are bounded linear functionals of the process ξ K,n ), gives the limit null distributions of the
statistics under consideration.
Corollary 4.5 Under H0K , with probability 1, for n = (n1 , . . . , nK ) such that min1≤k≤K nk → ∞, we
have

(p)
(p)
SK,n − sup BK,n (t) = O φK (n) ,
t∈R

(p)

Z

TK,n −


(p)
BK,n (t) dF0 (t) = O φK (n) .

R

11

5

The change-point problem

Here and elsewhere, btc denotes the largest integer not exceeding t. In many practical applications, we
assume the structural stability of statistical models and this fundamental assumption needs to be tested
before it can be applied. This is called the analysis of structural breaks, or change-points, which has
led to the development of a variety of theoretical and practical results. For good sources of references
to research literature in this area along with statistical applications, the reader may consult Brodsky
and Darkhovsky (1993), Csörgő and Horváth (1997) and Chen and Gupta (2000). For recent references
on the subject we may refer, among many others, to Bouzebda (2012), Aue and Horváth (2013), Chan
et al. (2013), Horváth and Rice (2014), Alvarez-Andrade and Bouzebda (2014) and Bouzebda (2014).
In this section, we deal with testing changes in d.f.’s for a sequence of independent real-valued r.v.’s
X1 , . . . , Xn . The corresponding null hypothesis that we want to test is
H000 : X1 , . . . , Xn have d.f. F.
As frequently done, the behavior of the derived tests will be investigated under the alternative hypothesis
of a single change-point
H100 : ∃ k ∗ ∈ {1, . . . , n − 1} such that X1 , . . . , Xk∗ have d.f. F and Xk∗ +1 , . . . , Xn have d.f. G.
The d.f.’s F and G are assumed to be continuous. The critical integer k ∗ can be written as bnsc for
a certain s ∈ (0, 1). Then, testing the null hypothesis H000 can be based on functionals of the following
process: set, for each n ∈ N∗ ,

bnsc(n − bnsc)  (p)−
(p)+
α
en(p) (s, t) :=
F
(t)
−
F
(t)
for s ∈ (0, 1), t ∈ R,
(5.1)
bnsc
n−bnsc
n3/2
(p)−

where Fk

(p)+

is the p-fold integrated empirical d.f. based upon the k first observations while Fn−k is that
(p)−

(p)+

based upon the (n − k) last ones. In (5.1) we extend the definition of Fk
and Fk
to the case where
(p)−
(p)+
(p)
k = 0 by setting F0
= F0
= 0, so that α
en (s, t) = 0 if s ∈ (0, 1/n).
We can define the r.v.’s X1 , . . . , Xbnsc and Xbnsc+1 , . . . , Xn on a probability space on which we
can simultaneously construct two Kiefer processes {K1 (s, u) : s ∈ R, u ∈ [0, 1]} and {K2 (s, u) : s ∈
R, u ∈ [0, 1]} such that the “restricted” processes {K1 (s, u) : s ∈ [1, n/2], u ∈ [0, 1]} and {K2 (s, u) : s ∈
 (p)
[n/2, n], u ∈ [0, 1]} are independent. It turns out that a natural approximation of α
e : n ∈ N∗ is
no
o n
(p)
given by the sequence of Gaussian processes Kn (s, F (t)) : s ∈ [0, 1], t ∈ R, n ∈ N∗ defined by
o

K(p)
n (s, u) :=

1 p o
u Kn (s, u) for s, u ∈ [0, 1], n ∈ N∗ ,
p!

(5.2)

o
o
where, for each n ∈ N∗ , Kn := Kn (s, u) : s, u ∈ [0, 1] is the Gaussian process defined by
 

 1
 √1 K2 (bnsc, u) − s(K1 (bn/2c, u) + K2 (bn/2c, u))
for
s
∈
0, 2 , u ∈ [0, 1],
o
n
Kn (s, u) :=




 √1 − K1 (bn(1 − s)c, u) + (1 − s)(K1 (bn/2c, u) + K2 (bn/2c, u)) for s ∈ 1 , 1 , u ∈ [0, 1].
2
n

More precisely, we have the following result.
 (p)
Theorem 5.1 On a suitable probability space, it is possible to define α
en : n ∈ N∗ , jointly with a
no
o
(p)
sequence of Gaussian processes Kn : n ∈ N∗ as above, such that, under H000 , with probability 1, as
n → ∞,


o
(log n)2
(p)
(p)
√
sup sup α
en (s, t) − Kn (s, F (t)) = O
.
n
s∈(0,1) t∈R
12

According to Csörgő et al. (1997), a way to test change-point is to use the following statistics:
en(p) (s, t) .
σn(p) := sup sup α

(5.3)

s∈(0,1) t∈R

The corollary below is a consequence of Theorem 5.1 which can be proved by following exactly the
same lines of Alvarez-Andrade et al. (2017).
Corollary 5.2 If H000 holds true, then we have the convergence in distribution, as n → ∞,
L

σn(p) −→

o

sup

K(p) (s, u) ,

s,u∈[0,1]
o
o
where K(p) = K(p) (s, u) : s, u ∈ [0, 1] is a Gaussian process with mean zero and covariance

o

o
1
E K(p) (s, u) K(p) (s0 , u0 ) = 2 up u0p (u ∧ u0 − uu0 )(s ∧ s0 − ss0 ).
p!
o

o

o

One has K(p) (s, u) = p!1 up K(s, u) where K is a tied-down Kiefer process. We refer to Csörgő and
Horváth (1997) for more details on the process K(p) in the case where p = 1.
Actually, according to Csörgő et al. (1997), the most appropriate way to test change-point is to use
the following weighted statistic:
(p)
α
en (s, t)
(p)
σn,w := sup sup
(5.4)
s∈(0,1) t∈R w (bnsc/n)
where w is a positive function defined on (0, 1), increasing in a neighborhood of zero and decreasing in
a neighborhood of one satisfying the condition


Z 1
εw2 (s)
ds
I(w, ε) :=
exp −
<∞
s(1 − s) s(1 − s)
0
for some constant ε > 0. For a history and further applications of I(w, ε), we refer to Csörgő and
Horváth (1993), Chapter 4. From Szyszkowicz (1992), an example of such function w is given by

w(t) :=

1
t(1 − t) log log
t(1 − t)

1/2
for t ∈ (0, 1).

By using similar techniques to those which are developed in Csörgő and Horváth (1997), one may show
that
o
(p) (s, u)
K
(p) L
.
σn,w −→ sup
w(s)
s,u∈[0,1]
For more details, we refer to Alvarez-Andrade and Bouzebda (2014).
Remark 5.3 As in Szyszkowicz (1994), we mention that the statistic given by (5.3) should be more
powerful for detecting changes that occur in the middle, i.e., near n/2, where k/n(1 − k/n) reaches its
maximum, than for the ones occurring near the end points. The advantage of using the weighted statistic
defined in (5.4) is the detection of changes that occur near the end points, while retaining the sensitivity
to possible changes in the middle as well.
We hope that the results presented in Sections 4 and 5 will be the prototypes of other various
applications.
13

6

Strong approximation of the integrated empirical process when parameters are estimated

In this section, we are interested in the strong approximation of the integrated empirical process when
parameters are estimated. Our approach is in the same spirit of Burke et al. (1979). Let us introduce,
(p)
for each n ∈ N∗ , the p-fold integrated estimated empirical process α
bn :

√ 
bn
α
b(p) (t) := n F(p) (t) − F (p) t, θ
for t ∈ R,
(6.1)
n

n


bn : n ∈ N∗ is a sequence of estimators of a parameter θ from a family of d.f.’s {F (t, θ) :
where θ
t ∈ R, θ ∈ Θ} (Θ being a subset of Rd and d a fixed positive integer) related to a sequence of i.i.d.
r.v.’s {Xi : i ∈ N∗ }. Let us mention that a general study of the weak convergence of the estimated
empirical process was carried out by Durbin (1973). For a more recent reference, we may refer to Genz
and Haeusler (2006) where the authors investigated the empirical processes with estimated parameters
under auxiliary information and provided some results regarding the bootstrap in order to evaluate the
limiting laws.
Let us introduce some notations.
(6.1) The transpose of a vector V of Rd will be denoted by V > .
(6.2) The norm k · k on Rd is defined by
k(y1 , . . . , yd )k := max |yi |.
1≤i≤d

d
(6.3) For a function (t, θ) 7→ g(t, θ) where θ = (θ1 , . . . , θd ) ∈ R
in
 , ∇θ g(t, θ 0 ) denotes the vector
d
R of partial derivatives (∂g/∂θ1 )(t, θ), . . . , (∂g/∂θd )(t, θ)) evaluated at θ = θ0 , and ∇2θ g(t, θ)
denotes the d × d matrix of second order partial derivatives (∂ 2 g/∂θi ∂θj )(t, θ)) 1≤i,j≤d .
R
(6.4) For a vector-valued function x 7→ V (x) = (v1 (x), . . . , vd (x)) defined on R, V denotes the vector
Z

Z
v1 (x1 ) dx1 , . . . , vd (xd ) dxd .
R

R

Next, we write out the set of all conditions (those of Burke et al. (1979)) which we will use in the
sequel.
bn admits the following form: for each n ∈ N∗ ,
(i) The estimator θ
n

X
√ 
bn − θ 0 = √1
n θ
l(Xi , θ 0 ) + εn ,
n
i=1

where θ 0 is the theoretical true value of θ, l(·, θ 0 ) is a measurable d-dimensional vector-valued
function, and εn converges to zero as n → ∞ in a manner to be specified later on. Notice that
n

√
1 X
√
l(Xi , θ 0 ) = n
n
i=1

Z

∞

l(s, θ 0 ) dFn (s).
−∞

(ii) The mean value of l(Xi , θ 0 ) vanishes:
E(l(Xi , θ 0 )) = 0.
14


(iii) The matrix M (θ 0 ) := E l(Xi , θ 0 )> l(Xi , θ 0 ) is a finite nonnegative definite d × d matrix.
(iv) The vector-valued function (t, θ) 7→ ∇θ F (t, θ) is uniformly continuous in t ∈ R and θ ∈ V, where
V is the closure of a given neighborhood of θ 0 .
(v) Each component of the vector-valued function t 7→ l(t, θ 0 ) is of bounded variation in t on each
finite interval of R.
(vi) The vector-valued function t 7→ ∇θ F (t, θ 0 ) is uniformly bounded in t ∈ R, and the vector-valued
function (t, θ) 7→ ∇2θ F (t, θ) is uniformly bounded in t ∈ R and θ ∈ V.
(vii) Set
`(s, θ 0 ) := l F −1 (s, θ 0 ), θ 0



for s ∈ (0, 1)

where
F −1 (s, θ 0 ) = inf{t ∈ R : F (t, θ 0 ) ≥ s}.
The limiting relations below hold:
lim

s&0

p
s log log(1/s) k`(s, θ 0 )k = 0

and
lim

s%1

p
(1 − s) log log[1/(1 − s)] k`(s, θ 0 )k = 0,

(viii) Set
∂`
(s, θ 0 ) for s ∈ (0, 1).
∂s
The partial derivative `0s (s, θ 0 ) exist for every s ∈ (0, 1) and the bounds below hold: there is a
positive constant C such that

s `0s (s, θ 0 ) ≤ C for all s ∈ 0, 12
`0s (s, θ 0 ) :=

and
(1 − s) `0s (s, θ 0 ) ≤ C

for all s ∈

1
2, 1



.

Now, we state an analogous result to Theorem 3.1 of Burke et al. (1979). For each n ∈ N∗ , let
{Gn (t) : t ∈ R} be the process defined by

Z


1
>
Gn (t) := √
K(n, F (t, θ 0 )) −
l(s, θ 0 ) ds K(n, F (s, θ 0 )) ∇θ F (t, θ 0 )
n
R

1 
= √ K(n, F (t, θ 0 )) − W(n)∇θ F (t, θ 0 )>
for t ∈ R,
n
where we set

Z
l(s, θ 0 ) ds K(τ, F (s, θ 0 )) for τ ≥ 0.

W(τ ) :=
R

The process {W(τ ) : τ ≥ 0} is a d-dimensional Brownian motion with a covariance matrix of rank that
(p)
of M (θ 0 ). The estimated empirical process given by α
bn (t) defined by (6.1) will be approximated by
 (p)
the sequence of processes Gn : n ∈ N∗ defined by
G(p)
n (t) :=

1
F (t, θ 0 )p Gn (t) for t ∈ R,
p!

(6.2)

as described in the next theorem. Set
ε(p)
bn(p) (t) − G(p)
n := sup α
n (t) .
t∈R

15

(6.3)


b n : n ∈ N∗
Theorem 6.1 Suppose that the sequence of estimators θ
(iii). Then, as n → ∞,
(p)

satisfies conditions (i), (ii) and

P

P

(a) εn −→ 0 if Conditions (iv), (v) hold and εn −→ 0;
a.s.

(p) a.s.

(b) εn −→ 0 if Conditions (vi)–(viii) hold and εn −→ 0;
(p)

(c) εn = O(max(h(n), n− )) for some  > 0 if Conditions (vi)–(viii) hold and εn = O(h(n)) for
some function h satisfying h(n) > 0 and h(n) → 0.
(p)

The limiting Gaussian process Gn of Theorem 6.1 depends crucially on F and also on the true
theoretical value θ 0 . In general, Theorem 6.1 cannot be used to test the composite hypothesis :
F ∈ {F (t, θ) : t ∈ R, θ ∈ Θ}.
In order to circumvent this problem, Burke et al. (1979) proposed an approximate solution, they introduce another process:


 
bn − W(n)∇θ F t, θ
bn > .
b n (t) := √1 K n, F t, θ
G
n
Under some regularity conditions, Burke et al. (1979) show that (see Theorem 3.2 therein), as n → ∞,
P
b n (t) − Gn (t) −→
sup G
0.
t∈R

b (p)
Setting G
n (t) :=

1
p!

bn
F t, θ

p

b n (t), one can show that, as n → ∞,
G
P
b (p) (t) − G(p) (t) −→
sup G
0.
n
n

(6.4)

t∈R

Consequently, we have, as n → ∞,
P
b (p) (t) −→
sup α
bn(p) (t) − G
0.
n
t∈R

7

Local time of the integrated empirical process

In this section, we are mainly concerned with the behavior of the local time of the p-fold integrated
empirical process. This behavior can be characterized by using a representation that expresses the
integrated empirical process in terms of a partial sums process, see (7.1) below. Let us recall the
definition of the process βn given in (1.2) and let us introduce the modified p-fold integrated uniform
(p)
empirical process βen defined, for each n ∈ N∗ , by
Z u
Z v1
Z vp−1
(p)
e
βn (u) :=
dv1
dvp−1 · · ·
βn (vp ) dvp
0
0
Z0 u
1
(u − v)p−1 βn (v) dv for u ∈ [0, 1].
=
(p − 1)! 0
(p)

(p)

In this part, we fo cus on the particular r.v. An := βen (1). It is easily seen that the representation
below holds:
!
n
(p)
X
√
1
1
1
Sn
p
(p)
(1 − Ui ) −
= √
An = n
n
p!
(p + 1)!
n
i=1

16

 (p)
where Sn : n ∈ N∗ is the following partial sums process where the summands are i.i.d. r.v.’s with
mean zero:

n 
1
1 X
p
(p)
(1 − Ui ) −
.
(7.1)
Sn :=
p!
p+1
i=1

This is a random walk with continuously distributed jumps. In the particular case where p = 1, we
retrieve the representation provided by Henze and Nikitin (2002) p. 185, namely

n 
(1)
X
Sn
1
(1)
(1)
An = √
with Sn :=
− Ui .
2
n
i=1

Notice that we are dealing with a sum of strongly non-lattice r.v.’s as, i.e., in p. 210 of Bass and
Khoshnevisan (1993a). Indeed, we easily check that the characteristic function χ(p) of the (1 − Ui )p −
1/(p + 1)’s, namely
Z 1
Z
exp(−iz/(p + 1)) z
dv
p
(p)
exp(iz(u − 1/(p + 1))) du =
χ (z) :=
exp(iv) 1−1/p
1/p
pz
v
0
0
satisfies the conditions
∀z ∈ R∗ , χ(p) (z) < 1 and

lim sup χ(p) (z) < 1.
|z|→∞

Next, we fix a neighborhood I of 0, e.g., I = [−1/2, 1/2], and we define the local time
λ(p) (x, n) :=

n
X

1I Si(p) − x



for x ∈ R, n ∈ N∗ .

(7.2)

i=1

 (p)
The local time λ(p) (x, n) represents the number of visits of the random walk Sn : n ∈ N∗ in the
neighborhood x + I of x up to discrete time n. Our aim is to obtain the rate of the approximation of
the self-intersection local time
Z
X


(p)
Ln (t) :=
1I Si(p) − x 1I Sj(p) − x dx
1≤i<j≤bntc R
(p)

by the integrated local time of some standard Wiener process. The quantity Ln (t) enumerates in a
certain manner the couples (i, j) of distinct and ordered indices up to time bntc such that Si − Sj is less
than the diameter of I.
To this aim, we recall that, if {W(t) : t ≥ 0} is the standard Wiener process with W(0) = 0, then
its local time process {l(x, t) : t ≥ 0, x ∈ R} is defined as
Z t
1
l(x, t) := lim
1{x−ε<W(s)<x+ε} ds for x ∈ R, t ≥ 0.
(7.3)
ε&0 2ε 0
Following exactly the same lines of Alvarez-Andrade et al. (2017), we can prove the two following
results.
Theorem 7.1 We have, with probability 1, as n → ∞,
Z


1 3/2
(p)
sup Ln (t) − n
ln (x, t)2 dx = O n5/4 (log n)1/2 (log log n)1/4 ,
2
R
t∈[0,1]
where ln is the normalized local time

√
1
ln (x, t) := √ l n x, bntc .
n
17

Corollary 7.2 We have, with probability 1, for any t ∈ (0, 1], there exist two positive constants κ1 and
κ2 such that, almost surely, for large enough n,
p
bntc3/2
3/2
κ1 √
≤ L(p)
log log n.
n (t) ≤ κ2 bntc
log log n
In particular, for any t ∈ (0, 1], almost surely, as n → ∞,
L(p)
n (t) =

8

1
bntc3/2+o(1) .
2

Simulation results for testing the uniformity

In this section, series of experiments are conducted in order to examine the performance of the proposed
statistical tests. More precisely, we have undertaken numerical illustrations regarding the power of these
statistical tests in finite sample situations. The computing program codes are implemented in R. In this
section, we considered four uniformity tests for different values of p = 0, 1, 2, 3, by making use of the
p-fold integrated Kolmogorov-Smirnov statistic

√  (p)
(p)
S(p)
n Fn (t) − F0 (t) .
n := sup
t∈R

(p)

Recall that for p = 0, the statistic Sn is the classical Kolmogorov Smirnov statistic. The simulations
involve random samples of size n = 10, 20, 40, 100; they are drawn from the underlying distributions
below, for each p ∈ {0, 1, 2, 3} and based on 10000 replications. In power comparison, we considered
the following distribution functions F (·) as alternatives.
Ak : F (x) = 1 − (1 − x)k for 0 ≤ x ≤ 1, k = 1.5, 2;
(
2k−1 xk
for 0 ≤ x < 0.5,
Bk : F (x) =
k = 1.5, 2, 3;
k−1
1 − 2 (1 − x) for 0.5 ≤ x ≤ 1,
(
0.5 − 2k−1 (0.5 − x)k for 0 ≤ x < 0.5,
Ck : F (x) =
k = 1.5, 2, 3.
0.5 + 2k−1 (0.5 − x)k for 0.5 ≤ x ≤ 1,
These alternatives were used by Stephens (1974) in his study of power comparisons of several tests
for uniformity. According to Stephens, alternative A gives points closer to zero than expected under
the hypothesis of uniformity. Alternative B gives points near 0.5 and alternative C gives two points
close to 0 and 1. Also, these alternatives were used by Dudewicz and van der Meulen (1981) and
Alizadeh Noughabi (2017) in their study of power comparisons of some uniformity tests. The densities
of the alternatives Ak , Bk , and Ck are depicted in the Figure 4 below.

18

50
40
30
(p)

0

10

20

Sn

0

2000

4000

6000

8000

10000

n

(p)

(0)

(1)

Figure 3: The statistics Sn p = 0, 1, 2, 3 as functions of the sample size. (“–”: Sn ), (“–”: Sn ), (“–”:
(2)
(3)
Sn ), (“–”: Sn ).

19

0.0
0.5

0.5

0.4

vT
0.8
0.0
2.0

2.5

0.4
0.8

0.8

vT

20
3.0

2.0

1.5

3.0

0.0

0.0

0.0

0.0

0.5

0.5

0.5

0.5

0.0

0.0

0.4

0.4
0.8

vT

Figure 4: Densities of Ak , Bk , and Ck families
1.0

Bf(vT, K)

Bf(vT, K)

1.0

Af(vT, K)

Af(vT, K)

1.0

1.0

1.5

1.5

2.0

1.5

2.0

1.5

A(2)

1.5

1.5

2.5

0.4

Cf(vT, K)

1.0

2.0

0.0

1.0

Cf(vT, K)

Cf(vT, K)

1.5

0.8

1.0

0.5

1.0

0.4

0.0

0.5
0.0

0.0

0.0

0.0

Bf(vT, K)

A(1.5)
B(1.5)
B(2)

0.8
0.0

0.0

0.4

vT
vT
vT
vT

B(3)
C(1.5)
C(2)
C(3)

0.4

vT

0.8

0.8

n = 10
Table 1: Powers (in percentages) of goodness of fit tests for the Uniform distribution on (0,1) for n = 10
and α = 0.01
n Samples Level
1.
10
10000
0.01
Alternative
A1.5
A2
B1.5
B2
B3
C1.5
C2
C3

(0)

Sn
4
15
0
0
0
3
6
15

(1)

Sn
7
24
1
3
7
1
1
4

(2)

Sn
7
21
1
3
9
0
1
5

(3)

Sn
0
0
0
0
0
3
8
19

Table 2: Powers (in percentages) of goodness of fit tests for the Uniform distribution on (0,1) for n = 10
and α = 0.05
n Samples Level
2.
10
10000
0.05
Alternative
A1.5
A2
B1.5
B2
B3
C1.5
C2
C3

(0)

Sn
15
38
3
4
8
11
19
36

(1)

Sn
25
54
9
17
36
5
8
18

(2)

Sn
22
48
11
20
44
5
9
21

(3)

Sn
0
0
0
0
0
13
22
38

Table 3: Powers (in percentages) of goodness of fit tests for the Uniform distribution on (0,1) for n = 10
and α = 0.10
n Samples Level
3.
10
10000
0.10
Alternative
A1.5
A2
B1.5
B2
B3
C1.5
C2
C3

(0)

Sn
25
51
8
12
24
19
30
53

21

(1)

Sn
37
68
18
31
58
11
16
29

(2)

Sn
32
61
19
32
61
11
16
32

(3)

Sn
0
0
3
1
0
23
32
50

n = 20
Table 4: Powers (in percentages) of goodness of fit tests for the Uniform distribution on (0,1) for n = 20
and α = 0.01
n Samples Level
1.
20
10000
0.01
Alternative
A1.5
A2
B1.5
B2
B3
C1.5
C2
C3

(0)

Sn
9
39
0
1
6
4
10
31

(1)

Sn
15
52
3
8
33
1
3
14

(2)

Sn
14
49
4
13
44
1
3
16

(3)

Sn
0
0
0
0
0
6
15
40

Table 5: Powers (in percentages) of goodness of fit tests for the Uniform distribution on (0,1) for n = 20
and α = 0.05
n Samples Level
2.
20
10000
0.05
Alternative
A1.5
A2
B1.5
B2
B3
C1.5
C2
C3

(0)

Sn
28
70
6
13
42
16
31
67

(1)

Sn
42
83
15
34
74
8
17
42

(2)

Sn
37
77
16
38
78
7
17
42

(3)

Sn
0
0
0
0
0
18
36
63

Table 6: Powers (in percentages) of goodness of fit tests for the Uniform distribution on (0,1) for n = 20
and α = 0.10
n Samples Level
3.
20
10000
0.10
Alternative
A1.5
A2
B1.5
B2
B3
C1.5
C2
C3

(0)

Sn
41
81
12
27
67
25
45
82

22

(1)

Sn
54
89
24
48
86
14
27
55

(2)

Sn
50
86
27
53
89
14
29
56

(3)

Sn
0
0
1
0
0
30
50
75

n = 40
Table 7: Powers (in percentages) of goodness of fit tests for the Uniform distribution on (0,1) for n = 40
and α = 0.01
n Samples Level
1.
40
10000
0.01
Alternative
A1.5
A2
B1.5
B2
B3
C1.5
C2
C3

(0)

Sn
25
82
1
7
49
7
24
70

(1)

Sn
37
90
7
29
83
3
13
46

(2)

Sn
35
87
10
37
89
3
13
47

(3)

Sn
0
10
0
0
6
8
27
68

Table 8: Powers (in percentages) of goodness of fit tests for the Uniform distribution on (0,1) for n = 40
and α = 0.05
n Samples Level
2.
40
10000
0.05
Alternative
A1.5
A2
B1.5
B2
B3
C1.5
C2
C3

(0)

Sn
51
95
11
38
92
24
56
96

(1)

Sn
65
98
24
62
97
13
36
77

(2)

Sn
63
97
30
70
98
14
37
75

(3)

Sn
3
28
0
3
26
27
55
87

Table 9: Powers (in percentages) of goodness of fit tests for the Uniform distribution on (0,1) for n = 40
and α = 0.10
n Samples Level
3.
40
10000
0.10
Alternative
A1.5
A2
B1.5
B2
B3
C1.5
C2
C3

(0)

Sn
65
98
23
60
98
35
74
99

23

(1)

Sn
77
99
37
75
99
21
50
89

(2)

Sn
74
98
42
80
99
21
49
84

(3)

Sn
5
40
1
6
43
40
69
92

n = 100
Table 10: Powers (in percentages) of goodness of fit tests for the Uniform distribution on (0,1) for
n = 100 and α = 0.01
n
Samples Level
1.
100
10000
0.01
Alternative
A1.5
A2
B1.5
B2
B3
C1.5
C2
C3

(0)

Sn
70
99
7
56
99
17
71
99

(1)

Sn
83
100
25
83
100
13
52
97

(2)

Sn
81
100
33
89
100
12
50
95

(3)

Sn
35
97
6
52
99
24
68
98

Table 11: Powers (in percentages) of goodness of fit tests for the Uniform distribution on (0,1) for
n = 100 and α = 0.05
n
Samples Level
2.
100
10000
0.05
Alternative
A1.5
A2
B1.5
B2
B3
C1.5
C2
C3

(0)

Sn
91
100
34
94
100
48
96
100

(1)

Sn
96
100
54
97
100
34
81
99

(2)

Sn
94
100
61
98
100
31
75
99

(3)

Sn
58
99
17
76
99
48
86
99

Table 12: Powers (in percentages) of goodness of fit tests for the Uniform distribution on (0,1) for
n = 100 and α = 0.10
n
Samples Level
3.
100
10000
0.10
Alternative
A1.5
A2
B1.5
B2
B3
C1.5
C2
C3

(0)

Sn
95
100
53
98
100
66
99
100

24

(1)

Sn
98
100
66
98
100
46
90
100

(2)

Sn
97
100
72
99
100
44
85
99

(3)

Sn
70
99
27
86
100
62
93
99

We computed the power values of the proposed tests using Monte Carlo simulation and next these
values are compared with each other. As it is expected, the power values of tests strongly depend on
(1)
the type of alternatives. Based on our simulation study, against alternative Ak , the proposed tests Sn
(2)
and Sn have a good power while against alternative Bk the tests have the highest power compared
(3)
with the classical Kolmogorov-Smirnov. The test based on Sn has a good power for alternative Ck .
In order to extract methodological recommendations for the use of the proposed statistics in this work,
it would be interesting to conduct extensive Monte Carlo experiments to compare our procedures with
other alternatives presented in the literature, but this would go far beyond the scope of the present
paper.

9

Mathematical developments

This section is devoted to the proofs of our results. The previously displayed notations continue to be
used in the sequel.

9.1

Some bounds for the empirical process, Brownian bridge and the Kiefer process

Let us immediately point out an obvious fact which will be used several times thereafter:
0 ≤ F (t) ≤ 1 and 0 ≤ Fn (t) ≤ 1 for t ∈ R, n ∈ N∗
which obviously entails that, for any p ∈ N,
∗
0 ≤ F (p) (t) ≤ 1 and 0 ≤ F(p)
n (t) ≤ 1 for t ∈ R, n ∈ N .

(9.1)

Similarly, we have, for any p ∈ N,
∗
0 ≤ U(p)
n (u) ≤ 1 for u ∈ [0, 1], n ∈ N .

(9.2)

We also mention some bounds that we will use further. By appealing to Chung’s law of the iterated
logarithm for the empirical process, see Chung (1949), which stipulates that
lim sup
n→∞

supt∈R |αn (t)|
1
√
=√
log log n
2

a.s.,

we see that, with probability 1, as n → ∞,
sup |αn (t)| = O

p

log log n .

(9.3)

t∈R

Moreover, by Komlós et al. (1975), on a suitable probability space, we can define the uniform empirical
process {βn : n ∈ N∗ }, in combination with a sequence of Brownian bridges {Bn : n ∈ N∗ } together with
a Kiefer process {K(s, u) : s ≥ 0, u ∈ [0, 1]}, such that, with probability 1, as n → ∞,


log n
sup |βn (u) − Bn (u)| = O √
(9.4)
n
u∈[0,1]
and

√
max sup

1≤k≤n u∈[0,1]

k βk (u) − K(k, u) = O (log n)2



from which we extract,with probability 1, as n → ∞,


1
(log n)2
√
sup βn (u) − √ K(n, u) = O
.
n
n
u∈[0,1]
25

(9.5)

As a result, by putting (9.3) into (9.4) and (9.5), one derives the following bounds: with probability 1,
as n → ∞,
p

p

sup |Bn (u)| = O log log n
and
sup |K(n, u)| = O n log log n .
(9.6)
u∈[0,1]

u∈[0,1]

Notice that the second bound in (9.6) comes also from the law of the iterated logarithm for the Kiefer
process; see Csörgő and Révész (1981), p. 81.

9.2

Proof of Proposition 1.2

We begin by making an observation: because of the hypothesis that the df F is continuous, the sampled
variables X1 , X2 , . . . , Xn are almost surely all different. Then, we can define with probability 1 the
order statistics
X1,n < X2,n < · · · < Xn,n

associated with X1 , X2 , . . . , Xn . Notice that the event Xi,n ≤ t is equal to {nFn (t) ≤ i}. Hence, we
can write that, for any function f , with probability 1,
nFn (t)
n
1X
1 X
f (s) dFn (s) =
f (Xi )1{Xi ≤t} =
f (Xi,n ) .
n
n
−∞

Z

t

i=1

(9.7)

i=1

Before proving (1.6), we first show by induction that
F(p)
n (t) =

1
np+1


# (i1 , . . . , ip+1 ) ∈ Np+1 : 1 ≤ i1 ≤ · · · ≤ ip+1 ≤ nFn (t) .

(9.8)

Of course, (9.8) holds for p = 0. Pick now a positive integer p and suppose that
F(p−1)
(t) =
n

1
# {(i1 , . . . , ip ) ∈ Np : 1 ≤ i1 ≤ · · · ≤ ip ≤ nFn (t)}.
np
(p)

(0)

By Definition B.1, we see that the family of functions Fn can be recursively defined by Fn = Fn and,
for any p ∈ N∗ and any t ∈ R, by
Z t
F(p−1)
(s) dFn (s).
F(p)
(t)
=
n
n
−∞

Therefore, by (9.7) and remarking that Fn (Xi,n ) = i/n, a.s.,
F(p)
n (t) =

nFn (t)
n
1 X (p−1)
1 X (p−1)
Fn
(Xi )1{Xi ≤t} =
Fn
(Xi,n )
n
n
i=1

=

=
=

1
np+1
1
np+1
1
np+1

i=1

nFn (t)

X

# {(i1 , . . . , ip ) ∈ Np : 1 ≤ i1 ≤ · · · ≤ ip ≤ nFn (Xi,n )}

i=1
nFn (t)

X

# {(i1 , . . . , ip ) ∈ Np : 1 ≤ i1 ≤ · · · ≤ ip ≤ i}

i=1


# (i1 , . . . , ip , ip+1 ) ∈ Np+1 : 1 ≤ i1 ≤ · · · ≤ ip ≤ ip+1 ≤ nFn (t) .

Hence, (9.8) is valid for any p ∈ N.
26

Now, we observe that the cardinality in (9.8) is nothing but the number of combinations with
repetitions of p+1 integers lying between 1 and nFn (t), which coincides with the number of combinations
(p)
without repetition of p + 1 integers lying between 1 and nFn (t) + p. This is the result concerning Fn
(p)
announced in (1.6). Finally, the formula concerning F
can be easily obtained by induction too. The
proof of Proposition 1.2 is finished.

(p)
In the proposition below, we provide a representation of Fn by means of Fn .
(p)

Proposition 9.1 The integrated empirical d.f. Fn can be expressed by means of Fn as follows: with
probability 1,
p
Fn (t)p+1 X (p) Fn (t)k
F(p)
(t)
=
+
ak
for t ∈ R, n ∈ N∗ ,
(9.9)
n
(p + 1)!
np−k+1
k=1

where the coefficients

(p)
ak ,

1 ≤ k ≤ p, are positive integers.

Proof. By expanding the combination in (1.6), we get that, a.s.,
F(p)
n (t) =

p
p
Y
Y
1
Fn (t)
(nF
(t)
+
i)
=
(nFn (t) + i) for t ∈ R, n ∈ N∗ .
n
(p + 1)! np+1
(p + 1)! np
i=0

i=1

Qp

Since i=1 (x + i) is a polynomial of degree p with coefficients in N, (9.9) immediately follows.
(p)
In the proposition below, we rely αn to αn .
(p)

Proposition 9.2 The p-fold integrated empirical process αn
according to, with probability 1,
αn(p) (t)



is related to the empirical process αn

p
p+1
k
p+1−k
X
X
1
(p) Fn (t)
(p) F (t)
k
p
= F (t) αn (t) +
α
(t)
+
a
bk
n
k
p!
n(k−1)/2
np−k+1/2
k=1
k=2
(p)

for

t ∈ R, n ∈ N∗ , (9.10)
(p)

where the coefficients ak , 1 ≤ k ≤ p, are those of Proposition 9.1 and the bk , 2 ≤ k ≤ p + 1, are
positive real numbers less than 1. Similarly,
βn(p) (u) =

p
p+1
k
p+1−k
X
X
1 p
(p) Un (u)
(p) u
k
β
(u)
+
a
u βn (u) +
bk
n
k
p!
n(k−1)/2
np−k+1/2
k=1
k=2

for

u ∈ [0, 1], n ∈ N∗ .

(9.11)

Proof. By Definition B.1 and Formulae (1.6) and (9.9), we write that, a.s., for t ∈ R, n ∈ N∗ ,
p

αn(p) (t) =

 X (p) Fn (t)k
√
1
n Fn (t)p+1 − F (t)p+1 +
ak
.
p−k+1/2
(p + 1)!
n
k=1

Applying the elementary identity below obtained by writing a = (a − b) + b and using the binomial
theorem

p+1 
X
p + 1 p+1−k
ap+1 − bp+1 = (p + 1)bp (a − b) +
b
(a − b)k
k
k=2

to a = Fn (t) and b = F (t) yields, a.s., for t ∈ R, n ∈ N∗ ,
1 √
n F (t)p (Fn (t) − F (t))
αn(p) (t) =
p!

p+1 
√ X p+1
1
+
n
F (t)p+1−k (Fn (t) − F (t))k
(p + 1)!
k
k=2

+

p
X
k=1

(p)

ak

Fn (t)k
.
np−k+1/2

(9.12)

√
(p)
(p)
Substituting Fn (t) − F (t) = αn (t)/ n into (9.12) gives (9.10) by setting bk :=
27

p+1
k



/(p + 1)!.



9.3

Proof of Theorem 2.2

P
(p)
Set Ap := pk=1 ak > 0 and I = [0, d/n] or [1 − d/n, 1]. Making use of (9.11) together with (9.2) and
(p)
the fact that 0 ≤ bk ≤ 1, it is clear that, a.s., for any d, n ∈ N∗ such that d ≤ n,
p+1

Ap
1 X 1
sup βn(p) (u) − B(p)
sup |βn (u)|k + √ .
n (u) ≤ sup βn (u) − Bn (u) + √
k/2−1
n
n
n
u∈I
u∈I
u∈[0,1]
k=2

Therefore,


1
(p)
(p)
P sup βn (u) − Bn (u) ≥ √ (c1 log d + x)
n
u∈I


p+1
1
1 X 1
k
sup |βn (u)| ≥ √ (c1 log d + x − Ap ) .
≤ P sup |βn (u) − Bn (u)| + √
n
n
nk/2−1 u∈[0,1]
u∈I
k=2
Now, using the elementary inequality
( r
)
r
r
X
X
X
ξk ≥
ak ≤
P{ξk ≥ ak },
P
k=1

k=1

k=1

which is valid for any positive integer r, any r.v.’s ξ1 , . . . , ξr and any real numbers a1 , . . . , ar , we obtain


1
√
(c
log
d
+
x)
P sup βn(p) (u) − B(p)
(u)
≥
1
n
n
u∈I


x
1
− Ap )
≤ P sup |βn (u) − Bn (u)| ≥ √ (c1 log d +
p+1
n
u∈I

p+1 
X
xnk/2−1
k
P sup |βn (u)| ≥
+
.
(9.13)
p+1
u∈[0,1]
k=2

On the other hand, the inequality of Dvoretzky et al. (1956) stipulates that there exists a positive
constant c4 such that, for any x > 0 and any n ∈ N∗ ,


x
P sup |Fn (t) − F (t)| ≥ √
≤ c4 exp(−2x2 ).
(9.14)
n
t∈R
Actually (9.14) simply reads, by means of βn , for any x > 0 and any n ∈ N∗ , as


P sup |βn (u)| ≥ x ≤ c4 exp(−2x2 ).
u∈[0,1]

Then,


p+1
p+1 
k/2−1
X
X
xn
≤ c4
exp−2
P sup |βn (u)|k ≥
p+1
u∈[0,1]
k=2

k=2

x nk/2−1

!2/k 


p+1

≤ c4 exp(−[2/(p + 1)] x) +

p+1
X
k=3


!
2
2/k 1−2/k
exp −
x n
. (9.15)
p+1

Now, by putting (2.1) and (9.15) into (9.13), we immediately complete the proof of Theorem 2.2 with
Bp := c2 exp(c3 Ap ) + c4

and Cp := min(c3 , 2)/(p + 1).

28

9.4

Proof of Corollary 2.4

The functional Φ being Lipschitz, there exists a positive constant L such that, for any functions v, w,
|Φ(v) − Φ(w)| ≤ L sup |v(t) − w(t)|.

(9.16)

t∈R

Let us choose for v, w the processes
Vn := αn(p) (·) and Wn := B(p)
n (F (·)).
Applying the elementary inequality

|P(A) − P(B)| ≤ P(A\B) + P(B\A) = P (A\B) ∪ (B\A)
to the events A = {Φ(Vn ) ≤ x} and B = {Φ(Wn ) ≤ x} provides, for any x ∈ R and any n ∈ N∗ ,
P{Φ(Vn ) ≤ x} − P{Φ(Wn ) ≤ x} ≤ P{Φ(Vn ) ≤ x ≤ Φ(Wn ) or Φ(Wn ) ≤ x ≤ Φ(Vn )}.
Now, applying to the elementary fact that [a ≤ x ≤ b or b ≤ x ≤ a implies |b − x| ≤ |b − a|] to the
numbers a = Φ(Vn ) and b = Φ(Wn )
P{Φ(Vn ) ≤ x ≤ Φ(Wn ) or Φ(Wn ) ≤ x ≤ Φ(Vn )} ≤ P{|Φ(Wn ) − x| ≤ |Φ(Wn ) − Φ(Vn )|},
from which, due to (9.16), we deduce that


P{Φ(Vn ) ≤ x} − P{Φ(Wn ) ≤ x} ≤ P |Φ(Wn ) − x| ≤ L sup |Vn (t) − Wn (t)| .

(9.17)

t∈R

On the other hand, by choosing x = c log n for a large enough constant c in (2.4) and putting n := (c+c1 )
√
log n/ n, we obtain the estimate below valid for large enough n:




Bp
log n
P sup |Vn (t) − Wn (t)| ≥ n ≤
=o √
.
(9.18)
n
n
t∈R
Now, by (9.17), we write
P{Φ(Vn ) ≤ x} − P{Φ(Wn ) ≤ x}


≤ P sup |Vn (t) − Wn (t)| < n , |Φ(Wn ) − x| ≤ L sup |Vn (t) − Wn (t)|
t∈R
t∈R


+ P sup |Vn (t) − Wn (t)| ≥ n , |Φ(Wn ) − x| ≤ L sup |Vn (t) − Wn (t)|
t∈R
t∈R


≤ P{|Φ(Wn ) − x| ≤ Ln } + P sup |Vn (t) − Wn (t)| ≥ n .

(9.19)

t∈R

Noticing that the distribution of Bn does not depend on n, which entails the equality
P{|Φ(Wn ) − x| ≤ Ln } = P{|Φ(W ) − x| ≤ Ln }
where
W := F (·)p B(F (·))/p!,
and recalling the assumption that the r.v. Φ(W ) admits a density function bounded by M say, we get
that, for any x ∈ R and any n ∈ N∗ ,
P{|Φ(Wn ) − x| ≤ Ln } ≤ 2LM n .

(9.20)

Finally, putting (9.18) and (9.20) into (9.19) leads to (2.5), which completes the proof of Corollary 2.4.
An alternative proof of a similar result may be found in Shorack and Wellner (1986) pp. 502–503. 
29

9.5

Proof of Corollary 2.5

Set, for any n ∈ N∗ ,


πn := P sup
t∈R

αn(p) (t)

−

B(p)
n (F (t))


log n
≥ c6 √
.
n

Applying (2.4) to x = c0 log n for a sufficiently large constant c0 yields, for large enough n,
πn ≤ Bp

p+1
X



exp −Cp c02/k (log n)2/k n1−3/k

k=2



1
=O 2
n


,

P
where c6 is a positive constant. Hence the series ( n≥1 πn ) is convergent and by appealing to BorelCantelli lemma, we get that



log n
(p)
(p)
P lim sup sup αn (t) − Bn (F (t)) ≥ c6 √
= 0,
n
n∈N∗
t∈R


which clearly implies Corollary 2.5.

9.6

Proof of Theorem 2.6

In view of (9.10), we write that, a.s., for any s ∈ (0, 1), any t ∈ R and any k ∈ N∗ ,
√

(p)

k αk (t) − K(p) (k, F (t)) =

h√
i
1
k αk (t) − K(k, F (t))
F (t)p
p!
p+1
p
p+1−i
i
X
X
(p) F (t)
(p) Fk (t)
i
+
bi
α
(t)
+
.
a
k
i
k (i−1)/2
k p−i+1/2
i=2
i=1
(p)

Then, by using (9.1) together with the fact that 0 ≤ bi
√
max sup

1≤k≤n t∈R

(9.21)

≤ 1, we deduce that, a.s., for any n ∈ N∗ ,

(p)

k αk (t) − K(p) (k, F (t))
≤



p+1
X
√
1
1
i
max sup k αk (t) − K(k, F (t)) +
max
sup
|α
(t)|
k
1≤k≤n k (i−1)/2 t∈R
p! 1≤k≤n t∈R
i=2

+

p
X
i=1

(p)
ai
max
1≤k≤n k p−i+1/2

≤ max sup
1≤k≤n t∈R

√

k αk (t) − K(k, F (t)) +

p+1 
X
i=2

i
max sup |αk (t)| + Ap .

1≤k≤n t∈R

Finally, by putting (9.5) and (9.3) into (9.22), we completes the proof of Theorem 2.6.

9.7

Proof of Corollary 2.7

From Theorem 2.6, we deduce that, with probability 1, as n → ∞,


1
(log n)2
(p)
(p)
√
sup αn (t) = √ sup K (n, F (t)) + O
.
n t∈R
n
t∈R
30

(9.22)


Therefore, a.s.,
(p)

lim sup
n→∞

supt∈R αn (t)
supt∈R K(p) (n, F (t))
√
√
= lim sup
log log n
n log log n
n→∞
supu∈[0,1] |up K(n, u)|
√
= lim sup
p! n log log n
n→∞
√
p
2
=
sup
Var(up K(1, u)).
p! u∈[0,1]

(9.23)

In the last equality, we have used Strassen’s law of the iterated logarithm for Gaussian processes;
see for instance Oodaira (1973), or Corollary 1.15.1 of Csörgő and Révész (1981). Observing that
Var(up K(1, u)) = u2p+1 (1 − u) and that
sup u2p+1 (1 − u) =
u∈[0,1]

(2p + 1)2p+1
,
(2p + 2)2p+2


(9.23) readily implies (2.7) which proves Corollary 2.7.

9.8

Proof of Corollary 2.8

We work under Hypothesis H0 . Let us introduce the p-fold integrated empirical process related to the
d.f. F0 :

√ 
(p)
(p)
for t ∈ R, n ∈ N∗ .
α0,n (t) := n F(p)
n (t) − F0 (t)
By the triangular inequality, we plainly have
(p)

(p)
(p)
S(p)
n − sup Bn (F0 (t)) ≤ sup α0,n (t) − Bn (F0 (t))
t∈R

t∈R

from which together with (2.6) we deduce (2.8).
Similarly,
Z
Z
 (p)
2
(p)
Tn −
Bn (F0 (t)) dF0 (t) ≤
R


2
(p)
α0,n (t)2 − Bn(p) (F0 (t)) dF0 (t)

R
(p)

≤ sup α0,n (t) − B(p)
n (F0 (t))
t∈R


(p)
(p)
× sup α0,n (t) + sup Bn (F0 (t)) .
t∈R

(9.24)

t∈R

In the last inequality above appears the supremum
sup B(p)
n (F0 (t)) ≤ sup Bn (u) .
t∈R

u∈[0,1]

Then, by putting (2.6), (2.7) and (9.6) into (9.24), we immediately deduce (2.9). The proof of Corollary 2.8 is finished.


31

9.9

Proof of Theorem 3.1

We imitate the proof of Theorem 2.1 in Mason (2001). Let us introduce
(p)

∆(p)
n,ν (d)

:=

(p)

βn (u) − Bn (u)
ν
,
u(1 − u)

sup
u∈[d/n,1−d/n]
(p)

∆1,(p)
n,ν (d)

:=

sup
u∈[d/n,1]

(p)

βn (u) − Bn (u)
,
uν
(p)

∆2,(p)
n,ν (d)

:=

sup
u∈[0,1−d/n]

(p)

βn (u) − Bn (u)
.
(1 − u)ν

We clearly have
(p)

1,(p)
∆n,ν
(d)

= max

(p)

sup
u∈[d/n,1/2]

(p)

≤ 2ν max

sup
u∈[d/n,1/2]

which entails that

(p)

(p)

βn (u) − Bn (u)
βn (u) − Bn (u)
ν
ν
,
sup
u(1 − u)
u(1 − u)
u∈[1/2,1−d/n]
(p)

(p)

(p)

!

βn (u) − Bn (u)
βn (u) − Bn (u)
,
sup
ν
u
(1 − u)ν
u∈[1/2,1−d/n]



ν
1,(p)
2,(p)
∆(p)
(d)
≤
2
max
∆
(d),
∆
(d)
.
n,ν
n,ν
n,ν

!

(9.25)
(p)1

(p)2

Hence, it suffices to derive inequalities of the form (3.1) for the auxiliary quantities ∆n,ν (d) and ∆n,ν (d).
First, notice that if we have for each i ∈ {d, . . . , n − 1},
ν
βn(p) (u) − B(p)
n (u) < x(i/n) ,

sup
u∈[0,(i+1)/n]

then
(p)

∆1,(p)
n,ν (d) =
≤

sup

max

i=d,...,n−1

max

u∈[i/n,(i+1)/n]

 
n ν

i=d,...,n−1

i

(p)

βn (u) − Bn (u)
uν

!

βn(p) (u) − B(p)
n (u)

sup


< x.

u∈[0,(i+1)/n]

Hence, we derive the inequality
n−1
X
 1,(p)
P ∆n,ν
(d) ≥ x ≤
δi,n ,

(9.26)

i=d

where
δi,n


:= P

sup

βn(p) (u)

−

B(p)
n (u)

ν

≥ x(i/n)


.

u∈[0,(i+1)/n]

Quite similarly,
n−d−1
X
 2,(p)
P ∆n,ν (d) ≥ x ≤
δi,n .
i=0

Set now (under the assumption ν > 0)


log(i + 1)
aν := max 1, c1 sup
,
iν
i∈N∗
32

(9.27)

where the constant c1 is that of (2.1). Of course, aν ≥ 1 and, for any i ∈ N∗ ,
aν iν ≥ c1 log(i + 1).
Then, by writing
x = (x − aν nν−1/2 ) + aν nν−1/2
in δi,n above and appealing to (2.3), we have, for i ≥ d, any n ∈ N∗ and large enough x (recall the
assumption ν < 1/2),


1 
ν
1/2−ν
√
δi,n ≤ P
sup
βn(p) (u) − B(p)
(u)
≥
c
log(i
+
1)
+
i
(xn
−
a
)
1
ν
n
n
u∈[0,(i+1)/n]
≤ Bp

p+1
X



exp −Cp (xn1/2−ν − aν )2/k i2ν/k n1−2/k .

(9.28)

k=2

In order to split the quantity (xn1/2−ν − aν )2/k , we use the elementary inequality

(a + b)µ ≥ 2µ−1 aµ + bµ ,
which is valid for any a, b ≥ 0 and any µ ∈ (0, 1]. If, in addition, a and b satisfy a ≥ 4b, which entails
that
a − 2b ≥ a/2,
then

 1

(a − b)µ ≥ 2µ−1 (a − 2b)µ + bµ ≥ aµ + bµ .
2

Thanks to this last inequality, we can write, for any i ≥ d, any k ≤ p + 1 and any x ≥ 4aν nν−1/2 , that


exp −Cp (xn1/2−ν − aν )2/k i2ν/k n1−2/k




2ν/k 1−2/k
exp −Cp00 x2/k d2ν/k n1−(2ν+1)/k
≤ exp −Cp00 a2/k
n
ν i




00 2ν/(p+1)
00 2/k 2ν/k 1−(2ν+1)/k
≤ exp −Cp i
exp −Cp x d
n
,
(9.29)
where we set Cp00 = Cp /2. Consequently, by putting (9.29) into (9.28), and next this latter into (9.26)
and (9.27), we get
p+1


P ∆1,(p)
n,ν (d) ≥ x

and P



∆2,(p)
n,ν (d)



X
1
≥ x ≤ Bp,ν
exp −Cp00 x2/k d2ν/k n1−(2ν+1)/k ,
2

(9.30)

k=2

with the constant
Bp,ν := 2Bp

∞
X



exp −Cp00 i2ν/(p+1) < +∞.

i=0

Finally, by (9.25),
n
n

xo
xo
1,(p)
2,(p)
P ∆(p)
(d)
≥
x
≤
P
∆
(d)
≥
+
P
∆
(d)
≥
,
n,ν
n,ν
n,ν
2
2
and by putting (9.30) into (9.31), we obtain (3.1) with Cp0 = Cp00 /22/(p+1) .

33

(9.31)


9.10

Proof of Corollary 3.2

Applying (3.1) to x = c00 log n for a sufficiently large constant c00 yields, for large enough n,


p+1


X
(p)
00 log n
P ∆n,ν (d) ≥ c 1/2−ν ≤ Bp,ν
exp −Cp0 c002/k d2ν/k (log n)2/k n1−2/k
n
k=2
 
1
=O 2 .
n


We conclude with the aid of Borel-Cantelli lemma as in Subsection 9.5.

9.11

Proof of Corollary 4.1
1,(p)

2,(p)

For each m, n ∈ N∗ , let αm and αn
denote the empirical processes respectively associated with
√
(p)
1,(p)
(p)
the samples X1 , . . . , Xm and Y1 , . . . , Yn . By replacing Fm (t) by αm (t)/ m + F (p) (t) and Gn (t) by
√
2,(p)
αn (t)/ n + G(p) (t), using the binomial theorem and recalling that, under H00 , F = G, we write
"
!q
!q #
r
1,(p)
2,(p)
mn
α
(t)
α
(t)
m
n
√
√
+ F (p) (t) −
+ F (p) (t)
ξ (p,q)
m,n (t) =
m+n
m
n

!k
!k 
r
q  
2,(p)
1,(p)

α
(t)
mn X q
α
(t)
q−k 
n
m

√
√
=
F (p) (t)
−
m+n
k
m
n
k=1
r
r

q−1
n
m
1,(p)
= q F (p) (t)
αm
(t) −
αn2,(p) (t) + ∆m,n (t)
m+n
m+n
where
r
∆m,n (t) =


!k
q  
1,(p)

mn X q
q−k  αm (t)
(p)
√
F (t)
−
m+n
k
m
k=2

2,(p)

αn (t)
√
n

!k 
.

By (2.7) and (9.1), it is easily seen that, with probability 1, as m, n → ∞,
!
!
(log log m)q/2
(log log n)q/2
√
√
sup |∆m,n (t)| = O
+O
.
m
n
t∈R

(9.32)


On the
hand, by Corollary 2.5, we can construct two sequences of Brownian bridges B1m : m ∈ N∗
 other
and B2n : n ∈ N∗ such that, with probability 1, as m, n → ∞,


1
log m
1,(p)
p 1
sup αm (t) − F (t) Bm (F (t)) = O √
,
p!
m
t∈R
(9.33)


1
log
n
sup αn2,(p) (t) − F (t)p B2n (F (t)) = O √
.
p!
n
t∈R
(p,q)

Setting Bm,n as in Corollary 4.1, we have
ξ (p,q)
m,n (t)

−

B(p,q)
m,n (t)

r


q
n
1
p 1
(p+1)(q−1)
1,(p)
=
F (t)
αm (t) − F (t) Bm (F (t))
(p + 1)!q−1
m+n
p!
r


m
1
−
αn2,(p) (t) − F (t)p B2n (F (t)) + ∆m,n (t).
(9.34)
m+n
p!

By putting (9.32) and (9.33) into (9.34), we deduce the result announced in Corollary 4.1.
34



9.12

Proof of Theorem 4.4

Let us introduce, for each k ∈ {1, . . . , K}, the p-fold integrated empirical process associated with the
d.f. F k

√ 
αnk,(p) (t) := n Fnk,(p) (t) − F k,(p) (t)
for t ∈ R, n ∈ N∗ .
By recalling (4.1) and making use of the most well-known variance formula
K
X

2

nk (xk − x̄) =

k=1

K
X

nk (xk − x0 )2 − |n| (x̄ − x0 )2 ,

k=1

where we have denoted
|n| =

K
X

K

nk and x̄ =

1 X
nk xk ,
|n|

for n = (n1 , . . . , nK ),

k=1

k=1
(p)

we rewrite ξ K,n (t) under Hypothesis H0K as
(p)

ξ K,n (t) =

K
X



(p)

nk Fk,(p)
nk (t) − F0 (t)

1
−
|n|

2

k=1

=

K
X

αnk,(p)
(t)2
k

−

K r
X
nk

|n|

k=1

k=1

K
X

nk




(p)
Fk,(p)
(t)
−
F
(t)
nk
0

!2

k=1
!2

αnk,(p)
(t)
k

.

(p)

Next, setting BK,n as in Theorem 4.4, we have
(p)

(p)

ξ K,n (t) − BK,n (t) = ∆1n (t) − ∆2n (t),
where we put, for any t ∈ R and any n = (n1 , . . . , nK ) ∈ N∗ ,
K 
X


2
F0 (t)2p k
=
−
Bnk F0 (t)
,
p!2
k=1
!2
!2
K r
K r
X

nk k,(p)
F0 (t)p X nk k
2
α
(t) −
B F0 (t) .
∆n (t) =
|n| nk
p!
|n| nk
∆1n (t)

αnk,(p)
(t)2
k

k=1

k=1

By setting, for any k ∈ {1, . . . , K}, any t ∈ R and any n = (n1 , . . . , nK ) ∈ N∗ ,
δk,n (t) = αnk,(p)
(t) −
k


F0 (t)p k
Bnk F0 (t) ,
p!

k,n (t) = αnk,(p)
(t) +
k


F0 (t)p k
Bnk F0 (t)
p!

and

and writing ∆1n (t) and ∆2n (t) as
∆1n (t) =

K
X

δk,n (t) k,n (t),

k=1

and
∆2n (t)

=

K r
X
nk
k=1

|n|

δk,n (t)

K r
X
nk
k=1

35

|n|

k,n (t),

(9.35)

we derive the following inequalities:
sup |∆1n (t)|
t∈R

≤

K 
X



sup |δk,n (t)| sup |k,n (t)| ,

k=1

t∈R

t∈R

(9.36)
sup ∆2n (t) ≤
t∈R

K
X

!
sup |δk,n (t)|

k=1 t∈R

K
X

!
sup |k,n (t)| .

k=1 t∈R

By (2.5), (2.7) and (9.6), we get the bounds, a.s., for each k ∈ {1, . . . , K}, as nk → ∞,


p

log nk
and sup |k,n (t)| = O log log nk .
sup |δk,n (t)| = O √
nk
t∈R
t∈R

(9.37)

Finally, by putting (9.37) into (9.36), and next into (9.35), we finish the proof of Theorem 4.4.

9.13



Proof of Theorem 5.1

In the computations below, the superscript “−” in the quantities F, F(p) , α and β refers to the k first
observations while the superscript “+” refers to the (n − k) last ones. By (1.5) and (5.1), we write that,
for n ∈ N∗ , s ∈ (0, 1) and t ∈ R,
 
i
bnsc(n − bnsc) h (p)−
(p)+
(p)
(p)
F
(t)
−
F
(t)
−
F
(t)
−
F
(t)
bnsc
n−bnsc
n3/2


(p)−
(p)+
αn−bnsc (t)
bnsc(n − bnsc)  αbnsc (t)
.
p
p
−
=
n3/2
bnsc
n − bnsc

α
en(p) (s, t) =

Using (9.10), we derive, a.s., for any n ∈ N∗ , any s ∈ (0, 1) and any t ∈ R, the form
α
en(p) (s, t) = In (s, t) − IIn (s, t) + IIIn (s, t) + IVn (s, t)

(9.38)

where
−
bnsc(n − bnsc) αbnsc (t)
p
F (t)p ,
In (s, t) =
p! n3/2
bnsc
+
bnsc(n − bnsc) αn−bnsc (t)
p
IIn (s, t) =
F (t)p ,
p! n3/2
n − bnsc
p+1
bnsc(n − bnsc) X (p)
IIIn (s, t) =
bk F (t)p+1−k
n3/2
k=2
p
bnsc(n − bnsc) X (p)
IVn (s, t) =
ak
n3/2
k=1

k
F−
bnsc (t)

bnscp−k+1

−
αbnsc
(t)k

bnsck/2

−

+
αn−bnsc
(t)k

(n − bnsc)k/2
!
k
F+
n−bnsc (t)
−
.
(n − bnsc)p−k+1

!
,

Concerning IIIn , we have the estimate below:

!k
!k 
−
+
p+1
X
α
(t)
α
(t)
bnsc(n − bnsc)
bnsc
n−bnsc

 p
+ p
|IIIn (s, t)| ≤
3/2
n
bnsc
n
−
bnsc
k=2
!k
!k
−
+
p−1
p−1
X
X
αbnsc
(t)
αn−bnsc
(t)
1 −
1 +
2
2
p
p
≤ √ αbnsc (t)
+ √ αn−bnsc (t)
n
n
bnsc
n − bnsc
k=0
k=0
36

(9.39)

√
We learn from (9.1) that |αn (t)/ n| = |Fn (t) − F (t)| ≤ 1 for any t ∈ R and any n ∈ N∗ and, of course,
−
+
similar inequalities hold for αbnsc
and αn−bnsc
. We deduce that both sums displayed in (9.39) are not
greater than p and by (9.3), with probability 1, as n → ∞, uniformly in s and t,



p  −
log log n
+
√
|IIIn (s, t)| ≤ √
.
(9.40)
αbnsc (t)2 + αn−bnsc
(t)2 = O
n
n
Concerning IVn , we have the estimate below:
|IVn (s, t)| ≤ A0p

−
+
p
p
k
k
n − bnsc X Fbnsc (t)
bnsc X Fn−bnsc (t)
+ 3/2
(n − bnsc)p−k
n3/2 k=1 bnscp−k
n
k=1

!
(9.41)

(p)

where we set A0p := max1≤i≤p ai > 0. Because of (9.1) and the convention that F−
bnsc = 0 if s ∈ (0, 1/n),
we see that both sums displayed in (9.41) are not greater than p and, as n → ∞, uniformly in s and t,


1
√
IVn (s, t) = O
.
(9.42)
n
As a byproduct, we get from (9.38), (9.40) and (9.42) that, with probability 1, as n → ∞, uniformly in
s and t,


log log n
(p)
√
α
en (s, t) = In (s, t) − IIn (s, t) + O
.
(9.43)
n
Next, it is convenient to introduce, for n ∈ N∗ and s, u ∈ (0, 1),
γn (u) =

√

n βn (u) =

n
X

1{Ui ≤u} − u ,


i=1
−
γbnsc
(u) =

bnsc
X
p
−
bnsc βbnsc
(u) =

1{Ui ≤u} − u ,


i=1

p
+
+
γn−bnsc
(u) = n − bnsc βn−bnsc
(u) =

n
X

1{Ui ≤u} − u ,


i=bnsc+1

I0n (s, u) =
II0n (s, u)

−
βbnsc
(u)

bnsc(n − bnsc)
n − bnsc p −
p
up =
u γbnsc (u),
3/2
p! n
p! n3/2
bnsc

+
bnsc p +
bnsc(n − bnsc) βn−bnsc (u) p
p
=
u =
u γn−bnsc (u),
3/2
p! n
p! n3/2
n − bnsc

and
δn (s, u) = I0n (s, u) − II0n (s, u).
Then, by (1.3), we plainly have the following equalities:
I0n (s, F (t)) = In (s, t),
II0n (s, F (t)) = IIn (s, t),
−
+
γn (u) = γbnsc
(u) + γn−bnsc
(u),

and we rewrite (9.43), with probability 1, as n → ∞, uniformly in s and t, as


log log n
(p)
√
α
en (s, t) = δn (s, F (t)) + O
.
n
37

(9.44)

Now, let us rewrite δn (s, u) as


up
bnsc
−
√
γbnsc
(u) −
γn (u)
n
p! n


p
n − bnsc
u
+
= √
γn (u) − γn−bnsc (u) .
n
p! n

δn (s, u) =

(9.45)

We know from Komlós et al. (1975) and Csörgő and Horváth (1997) that, with probability 1, as n → ∞,

−
sup γbnsc
(u) − K2 (bnsc, u) = O (log n)2 ,

sup

s∈[0,1/2] u∈[0,1]

(9.46)
sup

sup

s∈[1/2,1] u∈[0,1]

+
γn−bnsc
(u)

2



− K1 (bnsc, u) = O (log n) .

In particular, we have, with probability 1, as n → ∞,

−
(u) − K2 (bn/2c, u) = O (log n)2 ,
sup γbn/2c

(9.47)


+
(u) − K1 (bn/2c, u) = O (log n)2 .
sup γn−bn/2c

(9.48)

u∈[0,1]

u∈[0,1]

As a byproduct, by adding (9.47) and (9.48), we readily infer that, with probability 1, as n → ∞,


(9.49)
sup γn (u) − K1 (bn/2c, u) + K2 (bn/2c, u) = O (log n)2 .
u∈[0,1]
o

(p)

Recall the definition of the process Kn given by (5.2). From (9.45), (9.46), and (9.49), we deduce that,
with probability 1, as n → ∞,


o
(log n)2
(p)
√
sup δn (s, u) − Kn (s, u) = O
.
(9.50)
n
s,u∈[0,1]
Finally, we conclude by using the triangle inequality
o

sup sup α
en(p) (s, t) − Kn(p) (s, F (t)) ≤ sup sup α
en(p) (s, t) − δn (s, F (t))

s∈[0,1] t∈R

s∈[0,1] t∈R

o

+ sup

δn (s, u) − K(p)
n (s, u)

(9.51)

s,u∈[0,1]

and next by putting (9.44) and (9.50) into (9.51). The proof of Theorem 5.1 is completed.

9.14



Proof of Theorem 6.1
(p)

Recall the definition of α
bn (t) given by (6.1) and write it as follows: by using (9.10) mutatis mutandis,
a.s., for any t ∈ R and any n ∈ N∗ ,
p+1

α
bn(p) (t)

p

X (p)
X (p) Fn (t)k


1
bn (t)k
bn p α
bn p+1−k α
= F t, θ
bn (t) +
bk F t, θ
+
a
.
p!
n(k−1)/2 k=1 k np−k+1/2
k=2

38

(9.52)


Substituting α
bn (t) = α
bn (t) − Gn (t) + Gn (t) into (9.52) and using the binomial theorem yield, a.s., for
any t ∈ R and any n ∈ N∗ ,
α
bn(p) (t) − G(p)
n (t) =

p

k
X

 1 

1
(p) Fn (t)
bn p α
bn p − F (t, θ 0 )p Gn (t) +
F t, θ
bn (t) − Gn (t) +
F t, θ
ak
p!
p!
np−k+1/2
k=1

p+1
k  
b p+1−k X
X
i
k
(p) F t, θ n
+
bk
Gn (t)k−i α
bn (t) − Gn (t) .
(9.53)
(k−1)/2
i
n
i=0

k=2

By (9.1) and by appealing to the elementary identity ap − bp = (a − b)
(9.53) the following inequality: a.s., for any n ∈ N∗ ,

Pp−1
i=0

ai bp−i−1 , we extract from


Ap
bn − F (t, θ 0 ) sup |Gn (t)| + √
bn (t) − Gn (t) + sup F t, θ
bn(p) (t) − G(p)
sup α
n (t) ≤ sup α
n
t∈R
t∈R
t∈R
t∈R
! p+1
!
p+1
X
A00p X
k
+√
sup |Gn (t)|k
sup α
bn (t) − Gn (t)
n
t∈R
t∈R
k=0

where we set A00p := max0≤i,k≤p

k
i



k=0

(p)

> 0. Recall the notation (6.3) of εn and set
η n := sup |b
αn (t) − Gn (t)|.
t∈R

We have thus obtained the inequality

b
ε(p)
n ≤ η n + sup F t, θ n − F (t, θ 0 ) sup |Gn (t)|
t∈R
t∈R
! p
!
p
00
X
Ap X k
Ap
k
+√
ηn
sup |Gn (t)| + √ .
n
n
t∈R
k=0

(9.54)

k=0

We know from Theorem 3.1 of Burke et al. (1979) that η n satisfies the same limiting results than those
(p)
displayed in Theorem 6.1 for εn .

bn − F (t, θ 0 ) as n → ∞.
Next, we need to derive some bounds for supt∈R |Gn (t)| and supt∈R F t, θ
First, by using (9.6) and noticing that the same bound holds true for W(n), and by Condition (iv) and
the definition of Gn (t), we see that, with probability 1, as n → ∞,

p
(9.55)
sup |Gn (t)| = O log log n .
t∈R
∗
On the other hand, using
the
 one-term Taylor expansion of F (·, θ) with respect to θ 0 , there exists θ n

bn such that
lying in the segment θ 0 , θ



bn − F (t, θ 0 ) = θ
bn − θ 0 ∇θ F (t, θ ∗ )> .
(9.56)
F t, θ
n



√ b
1/4 θ
bn − θ 0 tends to
In case (a) of Theorem 6.1, n θ
n − θ 0 is asymptotically normal and then n

bn −F (t, θ 0 ) sup
zero as n → ∞ in probability. Therefore, by (9.55) and (9.56), supt∈R F t, θ
t∈R |Gn (t)|
also tends to zero as n → ∞, in probability. Putting this into (9.54), we easily complete the proof of
Theorem 6.1 in this case. In cases (b) and (c) of Theorem 6.1, referring to Burke et al. (1979) p. 779,
bn − θ 0 : with probability 1, as n → ∞,
we have the following bound for θ
!
r
log
log
n
bn − θ 0 = O
θ
.
n
39

By putting this into (9.56) and next in (9.54) with the aid of (9.55), we complete the proof of Theorem
6.1 in these two cases.
b (p)
Finally, concerning G
n (t), we have

 
bn p G
b (p) (t) − G(p) (t) = 1 F t, θ
b n (t) − Gn (t)
G
n
n
p!


1 
bn p − F (t, θ 0 )p Gn (t),
+
F t, θ
p!
from which we deduce
(p)
b
b (p)
sup G
n (t) − Gn (t) ≤ sup Gn (t) − Gn (t)
t∈R

t∈R


bn − F (t, θ 0 ) sup |Gn (t)|.
+ sup F t, θ
t∈R

t∈R



Using the same bounds than previously, we immediately derive (6.4).

A

Appendix : other integrated empirical distribution functions

To end up this article, let us point out that a similar analysis may be carried out with other integrated
empirical d.f.s and integrated processes. For instance, we present below two other families of integrated
empirical d.f.’s. The underlying d.f. F is still assumed to be continuous.
Definition A.1 We define the families of integrated d.f.’s and integrated empirical d.f.’s, for any p ∈ N,
any n ∈ N∗ and any t ∈ R, as
Z t
Z t
e(p) (t) :=
Fe(p) (t) :=
F (s)p dF (s), F
Fn (s)p dFn (s)
n
−∞

and
F̆

(p)

Z

−∞

t

F (t) − F (s)

(t) :=

p

dF (s),

−∞

F̆(p)
n (t)

Z

t

Fn (t) − Fn (s)

:=

p

dFn (s)

−∞

together with the corresponding family of integrated empirical processes as

√  (p)
e (t) − Fe(p) (t) ,
α
en(p) (t) := n F
n


√
(p)
ᾰn(p) (t) := n F̆(p)
(t)
−
F̆
(t)
.
n
We have, from (9.7), a.s., for any p ∈ N, any n ∈ N∗ and any t ∈ R,
nFn (t)
1 X
(p)
e
Fn (t) =
Fn (Xi,n )p
n

F̆(p)
n (t)

and

i=1

nFn (t)
p
1 X
=
Fn (t) − Fn (Xi,n ) .
n
i=1

Since Fn (Xi,n ) = i/n, we obtain the following closed forms.
Proposition A.2 For each p ∈ N, we explicitly have, with probability 1,
F (t)p+1
Fe(p) (t) = F̆ (p) (t) =
p+1

t ∈ R,

for

and
e(p) (t) =
F
n

1
np+1

nFn (t)

X
i=1

p

i ,

F̆(p)
n (t)

=

1
np+1
40

nFn (t)−1

X
i=0

ip

for

t ∈ R, n ∈ N∗ .

(A.1)

Observe the relation, a.s. valid, for all p ∈ N∗ , any n ∈ N∗ and any t ∈ R,
1
e(p)
F̆(p)
Fn (t)p .
n (t) = Fn (t) −
n
e(p)
Proposition A.3 The empirical d.f. F
n can be expressed by means of Fn as follows: with probability 1,
p

p+1
X (p) Fn (t)k
e(p) (t) = Fn (t)
F
+
e
ak
n
p+1
np−k+1

for

t ∈ R, n ∈ N∗ ,

(A.2)

k=1

(p)

where the coefficients e
ak , 1 ≤ k ≤ p, are rational numbers.
Proof. Appealing to Bernoulli’s formula
n
X
i=1


p−1 
1 p
1 X p+1
1
p+1
Bp+1−k nk
n
+ n +
i =
k
p+1
2
p+1
p

k=1

where the Bk ’s are the Bernoulli numbers (see, e.g., http://en.wikipedia.org/wiki/Bernoulli_number),
(A.1) immediately yields (A.2) with the coefficients


p+1
(p)
e
ak :=
Bp+1−k /(p + 1) for k ∈ {1, . . . , p − 1},
k
(p)

and e
ap := 1/2.
(p)
Below, we state the expression of α
en by means of αn analogous to (9.10).



(p)

Proposition A.4 The integrated empirical process α
en is related to the empirical process αn according
to, with probability 1,
α
en(p) (t)

p+1
X

p
p+1−k
k
X
(p) F (t)
(p) Fn (t)
k
e
= F (t) αn (t) +
bk
α
(t)
+
e
a
n
k
n(k−1)/2
np−k+1/2
k=2
k=1
p

for

t ∈ R, n ∈ N∗ ,

(p)
(p)
where the coefficients e
ak , 1 ≤ k ≤ p, are those of Proposition A.3 and the ebk , 2 ≤ k ≤ p + 1, are
positive real numbers less than p!.

(p)
(p)
The coefficients bk are given by bk := p+1
k /(p + 1).

More generally, we could define a broader family indexed by polynomials of two variables.
Definition A.5 We define the family of integrated d.f.’s and integrated empirical d.f.’s, for any polynomials P of two variables, any n ∈ N∗ and any t ∈ R, as
Z t
F (P) (t) :=
P(F (s), F (t)) dF (s),
−∞
t

F(P)
n (t) :=

Z

P(Fn (s), Fn (t)) Fn (s),
−∞

together with the corresponding family of integrated empirical processes as

√
P
αnP (t) := n FP
n (t) − F (t) .
41

Below, we state the last result of the paper which is a representation of αnP by means of αn analogous
to (9.10). This is the key point for deriving bounds similar to those obtained throughout the paper.
Proposition A.6 The empirical process αnP can be express as follows: with probability 1,
αnP (t) = Q(t) αn (t) + Rn (t)

for

t ∈ R, n ∈ N∗ ,

where Q is the polynomial function of F defined by
Z F (t)
∂P
Q(t) = P(F (t), F (t)) +
(x, F (t)) dx
∂y
0

for

(A.3)

t ∈ R,

and Rn satisfies the inequality
d

X αn (t)k
A
|Rn (t)| ≤ √ + B
n
n(k−1)/2

for

t ∈ R, n ∈ N∗ ,

k=2

A, B, d being three constants depending on P.
P P
Proof. Set P(x, y) = pi=0 qj=0 aij xi y j for some integers p, q and some coefficients aij . Then
F (P) (t) =

p X
q
X
aij
F (t)i+j+1
i+1
i=0 j=0

and, a.s., for any n ∈

N∗

and any t ∈ R,

F(P)
n (t)

=

p X
q
X

e(i) (t)
aij Fn (t)j F
n

i=0 j=0

=

p X
q
p X
q X
i
j+k
X
X
aij
(i) Fn (t)
aij e
ak
Fn (t)i+j+1 +
.
i+1
ni−k+1
i=0 j=0

Consequently, a.s., for any n ∈
αnP (t)

i=0 j=0 k=1

N∗

and any t ∈ R,

p
q

√ X X aij
Fn (t)i+j+1 − F (t)i+j+1
= n
i+1
i=0 j=0

+

p X
q X
i
X

(i)

aij e
ak

i=0 j=0 k=1

Fn (t)j+k
ni−k+1/2

which, by using the same method than (9.10), we rewrite as (A.3) with
Q(t) =

=

p X
q
X
i+j+1
i=0 j=0
p X
q
X

i+1

aij F (t)i+j

i+j

aij F (t)

i=0 j=0

Rn (t) =

p X
q X
i
X
i=0 j=0 k=1

+

p X
q
X
i=0 j=0

(i)

aij e
ak

j
aij F (t)i+j ,
i+1

Fn (t)j+k
ni−k+1/2

p X
q i+j+1
X
X i + j + 1 aij αn (t)k
+
F (t)i+j+1−k .
k
i + 1 n(k−1)/2
i=0 j=0 k=2



We easily conclude by using (9.1).
42

B

Some possible extensions

We will work under the following notation borrowed from Zhang (1997b). Let X1 , . . . , Xn be a sample
of independent and identically distributed random variables of a random variable X with unknown
distribution function F (·). For the unknown distribution function F (·) underlying the random sample
X1 , . . . , Xn we assume that we have the following auxiliary information: there exist r (r ≥ 1) functionally
independent functions η1 (·), . . . , ηr (·) such that, by putting η(·) = (η1 (·), . . . , ηr (·))> ,
E(η(X)) = 0.

(B.1)

By r linearly independent functions η1 (·), . . . , ηr (·) we mean that any ηi (·) can not be expressed as a
linear combination of η1 (·), . . . , ηi−1 (·), ηi+1 (·), . . . , ηr (·) for i = 1, . . . , r. Let p = (p1 , . . . , pn ) denote a
multinomial distribution on the points X1 , . . . , Xn and put
L(p) =

n
Y

pi .

i=1

Under the assumption (B.1), the profile empirical likelihood function L is defined by
L = max L(p) = max
p

p

n
Y

pi

i=1

where the maximum is taken on the n-uples p = (p1 , . . . , pn ) subject to the constraints
n
X

n
X

pi = 1,

i=1

pi η(Xi ) = 0 and pi > 0 for i = 1, . . . , n.

i=1

If 0 is inside the convex hull of the points η(X1 ), . . . , η(Xn ), then L exists uniquely. A little calculus of
variations shows that
n
Y
L = max
pbi ,
p

where

1
pbi =
n



i=1



1
1 + λ> η(Xi )

for 1 ≤ i ≤ n,

λ = (λ1 , . . . , λr )> being the solution of
n
X

n

pbi η(Xi ) =

i=1

Now, let
Fbn (z) =

1X
1
η(Xi ) = 0.
>
n
1 + λ η(Xi )
i=1

n
X

pbi 1{Xi ≤z} =

i=1

1 X 1{Xi ≤z}
.
n
1 + λ> η(Xi )
n

i=1

Then Fbn (·) can be viewed as an alternative estimator of F (·) satisfying (B.1). On the other hand, in
the absence of the knowledge of (B.1), the profile empirical likelihood L attains its maximum at the
distribution p = (1/n, . . . , 1/n), and hence Fbn = Fn , the standard empirical distribution function. Let
us assume that Σ = E(η(X)η > (X)) is a positive definite matrix and
Z ∞
2
E(kη(X)kr ) =
kη(x)k2r dF (x) < ∞,
−∞

43

where k · kr is the Euclidean norm in Rr . Let us denote by {D(x) : x ∈ R} a centered Gaussian process
with sample continuous paths satisfying
E(D(x)D(y)) = F (x ∧ y) − F (x)F (y)


− E η > (X)1{X≤x} Σ−1 E η(X)1{X≤y} for x, y ∈ R.
Zhang (1997a) proved the following functional central limit theorem for the empirical process pertaining
to Fbn (·):
√
n(Fbn − F ) ⇒ D in D(R).
Above, “⇒ D in D(R)” denotes convergence in distribution in D(R).
Definition B.1 We define the families of integrated empirical d.f.’s, under auxiliary information (B.1),
associated with the d.f. F , for any p ∈ N, any n ∈ N∗ and any t ∈ R, as
b(0) (t) := Fbn (t),
F
n
b(1) (t) :=
F
n

Z

t

Fbn (s) dFbn (s),
−∞

and, for p ≥ 2,
b(p) (t) :=
F
n

Z

t

Z

s1

−∞

Z

sp−1

dFbn (s2 )· · ·

dFbn (s1 )
−∞

Fbn (sp ) dFbn (sp ),
−∞

together with the corresponding family of integrated empirical processes as

√  (p)
b (t) − F (p) (t) .
n
F
α(p)
(t)
:=
n
n
• In many interesting applications, we may have some partial information about the distribution
of the population, although we do not know exactly the underlying distribution function of the
sample issued from the population; for further details and motivation about such problem we
refer to Owen (1990, 1991, 2001) and Zhang (2000). One of possible extension is to consider the
(p)
problem of the strong and weak convergences of the processes αn (·) by making effective use of
auxiliary information (B.1).
• According to Cheng (1995), Cheng and Parzen (1997), some studies have shown that a smoothed
estimator may be preferable to the sample estimator. First, smoothing reduces the random variation in the data, resulting in a more efficient estimator. Second, smoothing gives a smooth curve
for the quantile function that better displays the interesting features of the population distribution. Motivated by all these facts, it will be of interest to consider the smoothed versions of the
processes considered in the present paper and to study their asymptotic properties.

References
Alizadeh Noughabi, H. (2017). Entropy-based tests of uniformity: A Monte Carlo power comparison.
Comm. Statist. Simulation Comput., 46(2), 1266–1279.
Ahmad, I. A. and Dorea, C. C. Y. (2001). A note on goodness-of-fit statistics with asymptotically
normal distributions. J. Nonparametr. Statist., 13(4), 485–500.
Alvarez-Andrade, S. and Bouzebda, S. (2014). Some nonparametric tests for change-point detection
based on the P-P and Q-Q plot processes. Sequential Anal., 33(3), 360–399.
44

Alvarez-Andrade, S., Bouzebda, S. and Lachal, A. (2017). Some asymptotic results for the integrated
empirical process with applications to statistical tests. Comm. Statist. Theory Methods, 46(7), 3365–
3392.
Aue, A. and Horváth, L. (2013). Structural breaks in time series. J. Time Ser. Anal., 34(1), 1–16.
Bass, R. F. and Khoshnevisan, D. (1993a). Rates of convergence to Brownian local time. Stoch. Process.
Appl., 47(2), 197–213.
Bass, R. F. and Khoshnevisan, D. (1993b). Strong approximations to Brownian local time. In Seminar
on Stochastic Processes, 1992 (Seattle, WA, 1992), Vol. 33 of Progr. Probab., pages 43–65. Birkhäuser
Boston, Boston, MA.
Berkes, I. and Philipp, W. (1979). Approximation theorems for independent and weakly dependent
random vectors. Ann. Probab., 7(1), 29–54.
Billingsley, P. (1968). Convergence of probability measures. John Wiley & Sons, Inc., New York-LondonSydney.
Bouzebda, S. (2012). On the strong approximation of bootstrapped empirical copula processes with
applications. Math. Methods Statist., 21(3), 153–188.
Bouzebda, S. (2014). Asymptotic properties of pseudo maximum likelihood estimators and test in
semi-parametric copula models with multiple change points. Math. Methods Statist., 23(1), 38–65.
Bouzebda, S. (2016). Some applications of the strong approximation of the integrated empirical copula
processes. Math. Methods Statist., 25(4), 281–303.
Bouzebda, S. and El Faouzi, N.-E. (2012). New two-sample tests based on the integrated empirical
copula processes. Statistics, 46(3), 313–324.
Bouzebda, S., Keziou, A. and Zari, T. (2011). K-sample problem using strong approximations of
empirical copula processes. Math. Methods Statist., 20(1), 14–29.
Bouzebda, S. and Zari, T. (2013). Asymptotic behavior of weighted multivariate Cramér-von Mises-type
statistics under contiguous alternatives. Math. Methods Statist., 22(3), 226–252.
Bretagnolle, J. and Massart, P. (1989). Hungarian constructions from the nonasymptotic viewpoint.
Ann. Probab., 17(1), 239–256.
Brodsky, B. E. and Darkhovsky, B. S. (1993). Nonparametric methods in change-point problems, Vol.
243 of Mathematics and its Applications. Kluwer Academic Publishers Group, Dordrecht.
Burke, M. D., Csörgő, M., Csörgő, S. and Révész, P. (1979). Approximations of the empirical process
when parameters are estimated. Ann. Probab., 7(5), 790–810.
Chan, J., Horváth, L. and Hušková, M. (2013). Darling-Erdős limit results for change-point detection
in panel data. J. Statist. Plann. Inference, 143(5), 955–970.
Chatterjee, S. (2012). A new approach to strong embeddings. Probab. Theory Related Fields, 152(1-2),
231–264.
Chen, J. and Gupta, A. K. (2000). Parametric statistical change point analysis. Birkhäuser Boston,
Inc., Boston, MA.

45

Cheng, C. (1995). Uniform consistency of generalized kernel estimators of quantile density. Ann. Statist.,
23(6), 2285–2291.
Cheng, C. and Parzen, E. (1997). Unified estimators of smooth quantile and quantile density functions.
J. Statist. Plann. Inference, 59(2), 291–307.
Chung, K.-L. (1949). An estimate concerning the Kolmogoroff limit distribution. Trans. Amer. Math.
Soc., 67, 36–50.
Csörgő, M. (2007). A glimpse of the KMT (1975) approximation of empirical processes by Brownian
bridges via quantiles. Acta Sci. Math. (Szeged), 73(1-2), 349–366.
Csörgő, M. and Horváth, L. (1993). Weighted approximations in probability and statistics. Wiley Series
in Probability and Mathematical Statistics: Probability and Mathematical Statistics. John Wiley &
Sons, Ltd., Chichester.
Csörgő, M. and Horváth, L. (1997). Limit theorems in change-point analysis. Wiley Series in Probability
and Statistics. John Wiley & Sons, Ltd., Chichester.
Csörgő, M., Horváth, L. and Kokoszka, P. (2000). Approximation for bootstrapped empirical processes.
Proc. Amer. Math. Soc., 128(8), 2457–2464.
Csörgő, M., Horváth, L. and Szyszkowicz, B. (1997). Integral tests for suprema of Kiefer processes with
application. Statist. Decisions, 15(4), 365–377.
Csörgő, M., Csörgő, S., Horváth, L. and Mason, D. M. (1986). Weighted Empirical and Quantile
Processes Ann. Probab. 14(1), 31–85.
Csörgő, M. and Révész, P. (1981). Strong approximations in probability and statistics. Probability and
Mathematical Statistics. Academic Press, Inc. [Harcourt Brace Jovanovich Publishers], New YorkLondon.
Csörgő, S. (1981). Strong approximation of empirical Kac processes. Ann. Inst. Statist. Math., 33,
417–423.
Csörgő, S. and Hall, P. (1984). The Komlós-Major-Tusnády approximations and their applications.
Austral. J. Statist., 26(2), 189–218.
DasGupta, A. (2008). Asymptotic theory of statistics and probability. Springer Texts in Statistics.
Springer, New York.
De Acosta, A. (1982). Invariance principles in probability for triangular arrays of B-valued random
vectors and some applications. Ann. Probab., 10(2), 346–373.
Dudewicz, E. J. and van der Meulen, E. C. (1981). Entropy-based tests of uniformity. J. Amer. Statist.
Assoc., 76(376), 967–974.
Durbin, J. (1973). Weak convergence of the sample distribution function when parameters are estimated.
Ann. Statist., 1, 279–290.
Durio, A. and Nikitin, Y. Y. (2016). Local efficiency of integrated goodness-of-fit tests under skew
alternatives. Statist. Probab. Lett., 117, 136–143.
Dvoretzky, A., Kiefer, J. and Wolfowitz, J. (1956). Asymptotic minimax character of the sample
distribution function and of the classical multinomial estimator. Ann. Math. Statist., 27, 642–669.
46

Genz, M. and Haeusler, E. (2006). Empirical processes with estimated parameters under auxiliary
information. J. Comput. Appl. Math., 186(1), 191–216.
Haeusler, E. and Mason, D. M. (1999). Weighted approximations to continuous time martingales with
applications. Scand. J. Statist., 26(2), 281–295.
Henze, N. and Nikitin, Y. Y. (2000). A new approach to goodness-of-fit testing based on the integrated
empirical process. J. Nonparametr. Statist., 12(3), 391–416.
Henze, N. and Nikitin, Y. Y. (2002). Watson-type goodness-of-fit tests based on the integrated empirical
process. Math. Methods Statist., 11(2), 183–202.
Henze, N. and Nikitin, Y. Y. (2003). Two-sample tests based on the integrated empirical process. Comm.
Statist. Theory Methods, 32(9), 1767–1788.
Horváth, L. and Rice, G. (2014). Extensions of some classical methods in change point analysis. TEST,
23(2), 219–255.
Komlós, J., Major, P. and Tusnády, G. (1975). An approximation of partial sums of independent RV’s
and the sample DF. I. Z. Wahrscheinlichkeitstheorie und Verw. Gebiete, 32, 111–131.
Komlós, J., Major, P. and Tusnády, G. (1976). An approximation of partial sums of independent RV’s
and the sample DF. II. Z. Wahrscheinlichkeitstheorie und Verw. Gebiete, 34(1), 33–58.
Jing, P. and Wang, J. (2006). Testing the equality of multivariate distributions using the bootstrap and
integrated empirical processes. Comm. Statist. Theory Methods, 35(4-6), 661–670.
Jing, P. and Yang, Y. (2007). Testing the equality of multivariate distributions using the integrated
empirical processes. Math. Appl. (Wuhan), 20(3), 614–620.
Lachal, A. (2001). Study of some new integrated statistics: computation of Bahadur efficiency, relation
with non-standard boundary value problems. Math. Methods Statist., 10(1), 73–104.
Major, P. (2000). The approximation of the normalized empirical distribution function by a brownian
bridge. Notes available from http: // www. renyi. hu/ ~major/ probability/ empir. html .
Mason, D. M. (1991). A note on weighted approximations to the uniform empirical and quantile processes. In Sums, trimmed sums and extremes, volume 23 of Progr. Probab., pages 269–283. Birkhäuser
Boston, Boston, MA.
Mason, D. M. (2001). An exponential inequality for a weighted approximation to the uniform empirical
process with applications. In State of the art in probability and statistics (Leiden, 1999), volume 36
of IMS Lecture Notes Monogr. Ser., pages 477–498. Inst. Math. Statist., Beachwood, OH.
Mason, D. M. and Zhou, H. H. (2012). Quantile coupling inequalities and their applications. Probab.
Surv., 9, 439–479.
Mason, D. M. and van Zwet, W. R. (1987). A refinement of the KMT inequality for the uniform
empirical process. Ann. Probab., 15(3), 871–884.
Owen, A. (1990). Empirical likelihood ratio confidence regions. Ann. Statist., 18(1), 90–120.
Owen, A. (1991). Empirical likelihood for linear models. Ann. Statist., 19(4), 1725–1747.
Owen, A. (2001). Empirical Likelihood. Chapman and Hall/CRC, London.

47

Oodaira, H. (1973). The Law of the Iterated Logarithm for Gaussian Processes. Ann. Probab. 1(6),
954–967.
Shorack, G. R. and Wellner, J. A. (1986). Empirical processes with applications to statistics. Wiley
Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. John
Wiley & Sons, Inc., New York.
Stephens, M. A. (1974). Edf statistics for goodness of fit and some comparisons. J. Amer. Statist.
Assoc., 69(347), 730–737.
Szyszkowicz, B. (1992). Asymptotic distribution of weighted pontograms under contiguous alternatives.
Math. Proc. Camb. Phil. Soc., 112, 431–447.
Szyszkowicz, B. (1994). Weak convergence of weighted empirical type processes under contiguous and
changepoint alternatives. Stoch. Proc. Appl., 55, 281–313.
Zhang, B. (1997a). Estimating a distribution function in the presence of auxiliary information. Metrika,
46(3), 221–244.
Zhang, B. (1997b). Quantile processes in the presence of auxiliary information. Ann. Inst. Statist.
Math., 49(1), 35–55.
Zhang, B. (2000). Estimating the treatment effect in the two-sample problem with auxiliary information.
J. Nonparametr. Statist., 12(3), 377–389.

48

