A Large Deviation Inequality for β-mixing Time Series and its
Applications to the Functional Kernel Regression Model∗

arXiv:1701.05380v3 [math.ST] 4 Jul 2017

Johannes T. N. Krebs†
July 6, 2017

Abstract
We give a new large deviation inequality for sums of random variables of the form Zk = f (Xk , Xt ) for k, t ∈

N,

t fixed, where the underlying process X is β-mixing. The inequality can be used to derive concentration inequalities.
We demonstrate its usefulness in the functional kernel regression model of Ferraty et al. (2007) where we study the
consistency of dynamic forecasts.
Keywords: Asymptotic inference; Asymptotic inequalities; β-mixing; Bernstein inequality; Nonparametric statistics;
Time Series
MSC 2010: Primary: 62G20; 62M10; 37A25; Secondary: 62G09

1 Introduction
In this article we study asymptotic bounds for the probability

P n

−1

n
X

f (Xk , Xt ) − E [ f (X0 , x) ]

k=1

x=Xt

≥ε

!

where t ∈ Z

(1.1)

for a real-valued function f and for a stationary stochastic process {Xk : k ∈ Z} which takes values in a state space
(S, S). If the function f is bounded by B > 1, we obtain for a certain constant c an exponential rate of convergence for
(1.1) which is in O (exp(−cεn/(B log n log log n) )). So modulo a logarithmic factor which comes from the dependence
in the data, the rate corresponds to the rates of classical large deviations inequalities for independent random variables.
Large deviation inequalities are a major tool for the asymptotic analysis in probability theory and statistics. One of
the first inequalities of this type was published by Bernstein (1927) who considers the case P(|Sn | > ε), where
Pn
Sn =
k=1 Xk for bounded real-valued random variables X1 , . . . , Xn which are i.i.d. and have expectation zero.
There are various versions and generalizations of Bernstein’s inequality, e.g., Hoeffding (1963). In particular, deviation
inequalities for dependent data such as stochastic processes are nowadays important: Bernstein inequalities for time
series are developed in Carbon (1983), Collomb (1984), Bryc and Dembo (1996) and Merlevède et al. (2009). Arcones
(1995) develops Bernstein-type inequalities for U -statistics. Valenzuela-Domı́nguez et al. (2017) give a further generalization to strong mixing random fields {Xs : s ∈ ZN } which are defined on the regular lattice ZN for some lattice
dimension N ∈ N+ . A similar version for independent multivariate random variables is given by Ahmad and Amezziane
(2013). Krebs (2017) gives a Bernstein inequality for strong mixing random fields which are defined on exponentially
growing graphs. The corresponding definitions of dependence and their interaction properties can be found in Doukhan
∗ This research was supported by the Fraunhofer ITWM, 67663 Kaiserslautern, Germany, which is part of the Fraunhofer Gesellschaft zur Förderung
der angewandten Forschung e.V.
† Department of Mathematics, University of Kaiserslautern, 67653 Kaiserslautern, Germany, email: krebs@mathematik.uni-kl.de

1

(1994) and in Bradley (2005). Bernstein inequalities often find their applications when deriving large deviation results or
(uniform) consistency statements in nonparametric regression and density estimation, compare for instance Györfi et al.
(2002).
Consider again Equation (1.1) where X is a strong mixing process and t ∈ Z is a fixed integer. In the special case that
the aggregating function f (x, y) = g(x)h(y) is multiplicative, the statement follows from the large deviation inequalities for strong mixing processes given e.g. in Merlevède et al. (2009). However, for a general aggregating function
which does not allow a similar decomposition, we face the problem that the process given by Zk = f (Xk , Xt ) is not
necessarily strong mixing any more.
Thus, we need further assumptions on X which ensure that the contribution of a single summand f (Xk , Xt ) is asymptotically negligible in order to derive the necessary bounds. We show that for β-mixing processes there exist such bounds
under certain conditions which are quite likely to be fulfilled in practice.
In this manuscript we also give a natural motivation for the new large deviation inequality for (1.1) and we demonstrate
its power in a non-trivial application to the functional kernel regression model. Here the regression operator, which maps
from an infinite-dimensional space, e.g., a Hilbert space, to the real numbers, is estimated with a compactly supported
kernel function. The literature on this topic is extensive and we refer the reader to Ferraty et al. (2007), Delsol (2009)
and Ferraty et al. (2010) for a general introduction.
This paper is organized as follows: we give the definitions in Section 2. In Section 3 we present the new exponential
inequalities for β-mixing time series. In Section 4 we apply the exponential inequalities in the nonparametric functional
regression model of Ferraty et al. (2007) and investigate dynamic forecasts in this model.

2 Definitions and Notation
In this section we give the mathematical definitions and notation which we shall use to derive the results. We first
introduce the α- and β-mixing coefficients. The first coefficient is due to Rosenblatt (1956), the latter was introduced by
Kolmogorov and Rozanov (1960):
Definition 2.1 (α- and β-mixing coefficient). Let (Ω, A, P) be a probability space. Given two sub-σ-algebras F and G
of A, the α-mixing coefficient is defined by
α(F, G) := sup {|P(A ∩ B) − P(A)P(B)| : A ∈ F, B ∈ G} .
It is well-known that α(F, G) ≤ 1/4. The β-mixing coefficient is defined by




X
X
1
β(F, G) := sup
P(Ui ∩ Vj ) − P(Ui )P(Vj ) : (Ui )i∈I , (Vj )j∈J are finite partitions of Ω .


2
i∈I j∈J

Then 2α(F, G) ≤ β(F, G) ≤ 1, compare to Bradley (2005). If X and Y are two random variables on (Ω, A, P), then
we denote by α(X, Y ) the mixing coefficient α(σ(X), σ(Y )) and by β(X, Y ) the mixing coefficient β(σ(X), σ(Y )).
Furthermore, if (Xk : k ∈ Z) is a stochastic process, then we write for n ∈ N


α(n) := sup α σ(Xs : s ≤ k), σ(Xs : s ≥ k + n) and β(n) := sup β σ(Xs : s ≤ k), σ(Xs : s ≥ k + n) .
k∈

N

k∈

N

The stochastic process X is said to be strong mixing (or α-mixing) if α(n) → 0 (n → ∞). Furthermore, X is β-mixing
if β(n) → 0 (n → ∞). The β-mixing coefficient of two random variables X, Y on (Ω, A, P) is related to the total
variation norm as (cf. Doukhan (1994))
β(X, Y ) = kPX⊗Y − PX ⊗ PY kT V = sup |PX⊗Y (A) − PX ⊗ PY (A)|.

(2.1)

A∈A

Moreover, we denote for a set A ∈ A by 1{A} the indicator function on A. If not stated otherwise, we agree on the
convention to abbreviate constants by C in the proofs. In the following we derive inequalities of the Bernstein-type for
2

β-mixing stochastic processes.

3 Exponential inequalities for β-mixing processes
This section contains the main results. The following proposition is the first result of this article: we give inequalities
for β-mixing processes which are similar to Davydov’s inequality (cf. Davydov (1968)).
Proposition 3.1 (β-mixing and integration w.r.t. the joint distribution). Let (Ω, A, P) be a probability space and let
(S, S) and (T, T) be measurable, topological spaces. Let (X, Y ) : Ω → S × T be S ⊗ T-measurable such that the
joint distribution of X and Y is absolutely continuous w.r.t. their product measure on S ⊗ T with an essentially bounded
Radon-Nikodým derivative g, i.e.,

P(X,Y ) ≪ PX ⊗ PY such that g :=

dP(X,Y )
satisfies kgk∞,PX ⊗PY < ∞.
d(PX ⊗ PY )

Let p, q ∈ [1, ∞] be Hölder conjugate, i.e., p−1 + q −1 = 1. Let h : S × T → R be measurable and p-integrable w.r.t.
the product measure, i.e., khkp,PX ⊗PY = E [ E [ |h(X, y)|p ] |y=Y ]1/p < ∞. Then
Z

S×T

h dP(X,Y ) −

Z

h d(PX ⊗ PY ) ≤ 21/q (1 + kgk∞,PX ⊗PY )1/p khkp,PX ⊗PY β(X, Y )1/q .

S×T

(3.1)

If p = ∞ and q = 1 and if khk∞ = sup(x,y)∈S×T |h(x, y)| < ∞, this statement is true without the assumption on the
absolute continuity of the distributions:
Z
Z
h d(PX ⊗ PY ) ≤ 2 khk∞ β(X, Y ).
(3.2)
h dP(X,Y ) −
S×T

S×T

Proof of Proposition 3.1. We begin with the first statement. Therefore we shall use (2.1) and the following fact: since
the joint distribution is absolutely continuous, the total variation (and the β-mixing coefficient) can be written as
Z
1
(3.3)
|g − 1| d(PX ⊗ PY ).
β(X, Y ) = sup |PX⊗Y (A) − PX ⊗ PY (A)| =
2 S×T
A∈A
Indeed, this well-known equality follows if one considers the sets {g < 1}, {g > 1} ∈ A. Thus, Equation (3.3) and the
Hölder inequality enable us to write
Z
≤

S×T

Z

≤2

h dP(X,Y ) −

|h| d(PX
p

S×T
1/q

Z

S×T

h d(PX ⊗ PY ) ≤

Z

S×T

|h| |g − 1| d(PX ⊗ PY )

 Z
1/p
1/q
1
(q−1)/q
1/q
⊗ PY )
|g − 1| d(PX ⊗ PY )
2
kg − 1k∞,PX ⊗PY
2 S×T

(1 + kgk∞,PX ⊗PY )1/p khkp,PX ⊗PY β(X, Y )1/q .

We come to the second statement. Let p = ∞ and q = 1. Let h be bounded as sup(x,y)∈S×T |h(x, y)| < ∞ and assume
w.l.o.g. that h is non negative. Such a function h can be approximated uniformly on S × T by a sequence of simple
functions hn which converge from below to h, i.e., 0 ≤ sup(x,y)∈S×T h(x, y) − hn (x, y) ≤ 2−n . Hence, it suffices to
PN
consider simple functions: let h = i=1 ai 1{Ai } for N ∈ N and real numbers ai ∈ R+ and pairwise disjoint sets
Ai ∈ S ⊗ T. Set I := {i : 1 ≤ i ≤ N, PX ⊗ PY (Ai ) ≤ P(X,Y ) (Ai )} and J := {1, . . . , N } \ I. Hence, I (respectively
J) is the index sets where the probability of the joint distribution exceeds (respectively is less than) the probability of
the product measure. Then, the second statement follows:
Z

S×T

h dP(X,Y ) −

Z

S×T

h d(PX ⊗ PY )

3

≤ khk∞
= khk∞

(
(

X

P(X,Y ) (Ai ) − PX ⊗ PY (Ai ) +

i∈I

X

)

PX ⊗ PY (Ai ) − P(X,Y ) (Ai )

i∈J

P(X,Y )

[

Ai

i∈I

!

− PX ⊗ PY

[

i∈I

Ai

!

+ PX ⊗ PY

[

i∈J

Ai

!

− P(X,Y )

[

i∈J

Ai

!)

≤ 2 khk∞ β(X, Y ).

We present the main results of this article: we derive the exponential inequalities which prove the convergence of the
P
sum n−1 nk=1 f (Xk , Xt ) − E [ f (X0 , x) ] x=X for some t ∈ Z fixed. We assume for the rest of the article that the
t
β-mixing coefficients (and the α-mixing coefficients) of X are exponentially decreasing, i.e., there are κ0 , κ1 ∈ R+
such that β(n) ≤ κ0 exp(−κ1 n). Dedecker and Prieur (2004) give an example of such processes: consider a stationary
Markov chain Xn = ψ(Xn−1 ) + εn where the innovations are integrable and i.i.d. and ψ is Lipschitz-continuous with a
Lipschitz constant strictly smaller than 1. Then the chain is geometrically β-mixing if the distribution of the innovations
has an absolutely continuous component which is strictly positive in a neighborhood of zero. For more details, we refer
to Dedecker and Prieur (2004) and the references therein.
Moreover, we assume w.l.o.g. for the next results that the probability space contains an additional random variable
X0′ which has the same marginals as the stationary process X but is independent of X. We follow the ideas of
Merlevède et al. (2009) who investigate sums of α-mixing processes. Therefore, we define an extension of the discrete process X = {Xk : k ∈ Z} to the real interval: define for a real number t the random variable Xt by X⌊t⌋
which makes the process {Xt : t ∈ R} right continuous. In the same way, we extend the mixing coefficients to the real
numbers, i.e., α(t) = α(⌊t⌋) and β(t) = β(⌊t⌋) for t ∈ R. It follows from this definition of the continuous time process
R n+1
Pn
that k=1 f (Xk , Xt ) = 1
f (Xs , Xt ) ds. We present the main result of this section.
Theorem 3.2. Let {Xk : k ∈ Z} be a stationary β-mixing process whose marginals Xk take values in a state space
(S, S). The β-mixing coefficients satisfy β(n) ≤ κ0 exp(−κ1 n) for some κ0 , κ1 ∈ R+ . Let f : S × S → [−B, B],
B ∈ R+ , be a bounded and measurable function which fulfills E [ f (Xs , x) ] = 0 for PX0 -almost all x ∈ S. Let
κ1
1
A ≥ 14 ∨ (2κ1 ). Then there is a constant C which only depends on κ0 and κ1 such that for all 0 < γB ≤ 1∧κ
2 ∧ 4 log A
"

E exp(γ

Z

f (Xk , Xt ))

(0,A]

#





γBA
κ1 A
+ exp Cγ 2 B 2 A log A +
.
≤ 3κ0 exp −
4 log A
log A

Proof of Theorem 3.2. Let t ∈ R be fixed. In the first step we divide the interval (0, A] into three pairwise disjoint
intervals IL , I˜ and IR . Therefore, let δ := 1/ log A ∈ (0, 1), then I˜ is given by
I˜ := (−δA/2 + t, δA/2 + t] ∩ (0, A].
IL is then given as the left residual part of I, namely, IL = (−∞, −δ/2A + t] ∩ (0, A]. In the same way, IR is the right
residual part, IR = (δ/2A + t, ∞] ∩ (0, A]. Note that both IL and IR have a minimal length of (1 − δ)A/2 and that their
measure sums up to at most A. Furthermore, for all A ≥ 14, IR and IL have a length of at least 4. In order to estimate
the Laplace transform, we bound the integral over the interval I˜ by its maximal value and then apply Proposition 3.1 and
Proposition A.1:
"
!#
Z

E exp γ

A

f (Xs , Xt ) ds

0



 Z
f (Xs , Xt ) ds
exp{γBδA}
≤ E exp γ
IL ∪IR
( 

 Z
 Z
≤ E exp γ
f (Xs , Xt ) ds − exp γ

IL ∪IR

IL ∪IR

4

f (Xs , X0′ ) ds



≤

+E



(



2β



 Z
exp γ

IL ∪IR

δA
2

+ EE





f (Xs , X0′ ) ds

)

exp{γBδA}

exp(γB(1 − δ)A) + α(δA) exp(γB(1 − δ)A)

 Z
exp γ

f (Xs , x) ds
IL



E



 Z
exp γ

f (Xs , x) ds

IR



κ1 δA
+ γBA
≤ 3κ0 exp −
2

 

 Z
 Z
+ E  E exp γ
f (Xs , x) ds
E exp γ
IL

IR

f (Xs , x) ds





x=X0′

x=X0′

)


exp{γBδA}



(3.4)

 exp{γBδA}.

Consider the inner expectations in (3.4): since the random variable f (Xs , x) is centered for PX0 -almost all x ∈ S, we
can apply Lemma 10 from Merlevède et al. (2009) and obtain if (1 − δ)A/2 > 4 ∨ (2κ1 ) that
 


 Z
 Z

E exp γ
E exp γ
f (Xs , x) ds
f (Xs , x) ds
≤ exp Cγ 2 B 2 (|IL | log |IL | + |IR | log |IR |)
IL

IR


≤ exp Cγ 2 B 2 A log A ,

(3.5)

where 0 ≤ γB ≤ (κ1 ∧ 1)/2 and C is a constant which only depends on the bound of the mixing coefficients κ0 , κ1 .
Combining (3.4) with (3.5) yields
"
!#




Z A
γBA
κ1 A
.
+ γBA + exp Cγ 2 B 2 A log A +
f (Xs , Xt ) ds
≤ 3κ0 exp −
E exp γ
2 log A
log A
0

Since the bound B is the only property of the function f which is determining the bound on the Laplace transform given
in Theorem 3.2, we can easily extend the statement to a sequence of functions (fn : n : N) which are all uniformly
bounded: fn : S × S → [−B, B] for n ∈ N. Therefore, we give the corollary
Corollary 3.3. Let the stochastic process X be as in Theorem 3.2. Let (fn : n : N) be a sequence of functions such that
each element fn : S × S → [−B, B] is as in Theorem 3.2 and satisfies in particular E [ fn (X0 , x) ] = 0 for PX0 -almost
all x ∈ S. Then for all n ∈ N there are constants a1 , a2 which only depend on the parameter κ0 , κ1 such that
!


n
X
εn
−1
.
fn (Xk , Xt ) ≥ ε ≤ a1 exp −a2
P n
B log n log log n
k=1

Proof of Corollary 3.3. If we apply Theorem 3.2 to the interval (0, n] and to the process X which is formally shifted one
integer to the left, we find with the choice γ = (κ1 ∧ 1)/(4B log n log log n) and Markov’s inequality that for n ≥ 14
!


n
X
(κ1 ∧ 1)ε n
−1
fn (Xk , Xt ) ≥ ε ≤ exp −
P n
4B log n log log n
k=1
(



)
γBn
κ1 n
2 2
· 3κ0 exp −
+ exp Cγ B n log n +
.
4 log n
log n

Note that the last term in the curly brackets grows at a rate of n/ log n(log log n)2 . This finishes the proof.

We consider the special case where fn (X, Y ) ≡ X in the context of Corollary 3.3. We obtain in this case for a β
Pn
mixing process X that P n−1 k=1 Xk ≥ ε ≤ a1 exp {−a2 ε n/(log n log log n)}. This corresponds to the rate
5

given by Merlevède et al. (2009) who investigate concentration inequalities of centered, real-valued and strongly mixing
processes X where the α-mixing coefficients decay at an exponential rate. This means that our new inequality from
Corollary 3.3 attains the same rate in this special case.
We can derive an extension of Theorem 3.2 to a sequence of unbounded functions fn : S × S → R.
Theorem 3.4. Let the stationary process X be as in Theorem 3.2. Furthermore, let (fn : n ∈ N) be a sequence of
functions fn : S × S → R which fulfills E [ fn (X0 , x) ] = 0 for PX0 -almost all x ∈ S. Let the joint distribution of
(X0 , Xk ) be absolutely continuous w.r.t. the product measure such that the corresponding Radon-Nikodým derivatives
are essentially bounded uniformly over k ∈ N, i.e.,
dP(X0 ,Xk )
< ∞.
d (PX0 ⊗ PXk )

sup
k∈

N

Then, there are constants a1 , a2 ∈ R+ which do not depend on n ∈ N and t ∈ Z such that for all ε > 0
!
n
X
−1
fn (Xk , Xt ) ≥ ε
P n
k=1

≤ inf

B>1

(


a1 ε−1 exp −a2

+ a1 (nε)−1 E

εn
B log n log log n

1/(pr)
[ fn (X0 , X0′ )pr ]





+ 4ε−1 (k − 1)−1 B −(k−1) E fn (X0 , X0′ )k

B −k/(pu) E



1/(pu)
fn (X0 , X0′ )k

(3.6)

)

where p > 1, r−1 + u−1 = 1, r, u > 1 and k > 1.
Proof of Theorem 3.4. If the right hand side of (3.6) is infinite, there is nothing to prove. So we can assume that the
parameters k, p, r and u are such that the all moments on the right hand side exist. Since E [ fn (X0 , x) ] = 0 for PX0 almost all x ∈ S, we use for B > 1 the fundamental decomposition fn (x, y) = fn,+ (x, y) + fn,0 (x, y) + fn,− (x, y)
where
fn,+ (x, y) := fn (x, y) − min(fn (x, y), B) ≥ 0,

fn,− (x, y) := fn (x, y) − max(fn (x, y), −B) ≤ 0
and fn,0 (x, y) := max(min(fn (x, y), B), −B).

Then,

P n

−1

n
X

fn (Xk , Xt ) > 3ε

k=1

!

≤P n

−1

+P n
+P n

n
X

fn,+ (Xk , Xt ) − E [ fn,+ (Xk , x) ] |x=Xt > ε

!

k=1
n
X
−1

fn,0 (Xk , Xt ) − E [ fn,0 (Xk , x) ] |x=Xt > ε

−1

fn,− (Xk , Xt ) − E [ fn,− (Xk , x) ] |x=Xt > ε .

k=1
n
X

!

(3.7)

!

k=1

The asymptotic behavior of the second probability in (3.7) is given in Corollary 3.3. The first and the third probability
can be bounded with Markov’s inequality and Proposition 3.1, we only consider fn,+ here:

P n−1

n
X

fn,+ (Xk , Xt ) − E [ fn,+ (X0 , x) ] |x=Xt > ε

k=1
n
X
−1

≤ (nε)

k=1

+ε

−1

E |fn,+ (Xk , Xt ) − E [ fn,+ (X0 , x) ] |x=Xt | − fn,+ (Xk , X0′ ) − E [ fn,+ (X0 , x) ] |x=X0′


E fn,+ (X0 , X0′ ) − E [ fn,+ (X0 , x) ] |x=X0′


!

6





≤ C(nε)−1

n
X

β(|t − k|)1/q E

k=1

+ε

−1



fn,+ (X0 , X0′ ) − E [ fn,+ (X0 , x) ] |x=X0′

E fn,+ (X0 , X0′ ) − E [ fn,+ (X0 , x) ] |x=X0′


≤ C(nε)−1 E [ |fn,+ (X0 , X0′ )|p ]

1/p



p 1/p

+ 2ε−1 E [ |fn,+ (X0 , X0′ )| ] ,

(3.8)

where p, q > 1 such that p−1 + q −1 = 1 as in Equation (3.1) and where we use in the last inequality that the β-mixing
coefficients are summable. We bound the two expectations given in (3.8) further. For the first expectation we obtain for
k>1

Z ∞
Z ∞



k
−k
′
t dt E fn (X0 , y)
≤E
E [ fn,+ (X0 , X0 ) ] = E
P(fn (X0 , y) > t) dt
y=X0′
y=X0′
B
B


−1 −(k−1)
′ k
≤ (k − 1) B
E fn (X0 , X0 ) .

The second expectation can be bounded with Hölder’s inequality for r−1 + u−1 = 1, r, u > 1 and Markov’s inequality:

E [ |fn,+ (X0 , X0′ )|p ] = E [ |fn(X0 , X0′ ) − B|pr 1{fn (X0 , X0′ ) ≥ B} ]1/r P(fn (X0 , X0′ ) ≥ B)1/u
≤ E [ fn (X0 , X0′ )pr ]

1/r

for k > 1. This proves the claim.

1/u

,
B −k/u E fn (X0 , X0′ )k

4 An application in the nonparametric functional regression model
In this section, we embed the developed inequalities in the nonparametric functional kernel regression model of Ferraty et al.
(2007), Delsol (2009) and Ferraty et al. (2010). We cannot discuss all details of this model here due to its complexity and
assume that the reader has some prior knowledge on this subject. Nevertheless, we describe all necessary assumptions:
let (H, h·, ·i) be a separable Hilbert space over R where the norm k · k is induced by the inner product h·, ·i. In typical
applications the Hilbert space is given as a function space L2 ([0, 1], B([0, 1]), dx) or more general L2 (K, B(K), ν) for
a bounded and convex set K ∈ Rd and a finite measure ν. Let {(Xk , Yk ) : k ∈ Z} ⊆ H × R be a stationary process
which is β-mixing with exponentially decaying coefficients. The process fulfills the regression equation
Yk = ψ(Xk ) + εk ,

k ∈ Z.

(4.1)

The regression function ψ : H → R is Lipschitz-continuous, i.e., |ψ(x)−ψ(y)| ≤ L kx − yk for all x, y ∈ H. The error


terms fulfill E [ εk |Xk ] = 0 and E ε2k |Xk < ∞. Given an observed sample (X1 , Y1 ), . . . , (Xn , Yn ) the estimator of
ψ is
Pn
k=1 Yk K(kXk − xk /h)
ψ̂(x) = P
,
(4.2)
n
k=1 K(kXk − xk /h)

where h > 0 is the bandwidth and K is a kernel function. The kernel function is supported in [0, 1] and zero otherwise.
It admits a continuous derivative K ′ on [0, 1) such that K ′ ≤ 0 and K(1) > 0 as in Ferraty et al. (2007). Moreover, we
define the quantities
ĝh (x) := (nFx (h))−1

n
X

Yk K(kXk − xk /h)

and

k=1

fˆh (x) := (nFx (h))−1

n
X

K(kXk − xk /h),

k=1

where Fx (h) = P(kX0 − xk ≤ h). Note that we do not normalize fˆh and ĝh by a division with the bandwidth instead
we multiply the inverse of the small ball probability Fx (h). Here we assume that Fx (h) > 0 for h > 0 and that
Fx (0) = 0 for all x ∈ H. We choose the bandwidth h as a function of n such that for h → 0 as n → ∞ the summability

7

condition
∞
X

n=1



n−2 E FX0 (h)−2 (log n)2 (log log n)2 < ∞

(4.3)



is fulfilled where we assume that E FX0 (h)−2 is finite for each h > 0. E.g. we can choose h → 0 such that



n−2 E FX0 (h)−2 = o n−(1+δ) for some δ ∈ (0, 1). Note that for the pointwise convergence ψ̂(x) → ψ(x)
Ferraty et al. (2007) require n−1 Fx (h)−1 → 0.
Assume that there is a small ball probability function τ : [0, 1] → [0, 1] which the uniform limit of the fractions
Fx (hs)/Fx (h), i.e.,
lim sup

h↓0 x∈H

Fx (hs)
− τ (s) = 0.
Fx (h)

(4.4)

R1
Further, set M = K(1) − 0 K ′ (s)τ (s) ds which is assumed to be positive.
We shortly discuss the issue of the convergence of Fx (hs)/Fx (h) for h ↓ 0 and s ∈ (0, 1), a more detailed introduction
to this topic offer Ferraty et al. (2007) and the references therein. If the underlying Hilbert space is a function space,
then one has in many applications that for a point x in the Hilbert space P(kX0 − xk < h) ∼ C(x)P(kX0 k < h)
for h ↓ 0. Hence the small ball problem at the point x shifts to the origin and in the limit the quotient Fx (hs)/Fx (h)
becomes independent of the point x. This motivates assumption (4.4).
We study the dynamic forecast ψ̂(Xt ) for an observation Xt , this means, we give sufficient conditions that the difference
ψ̂(Xt ) − ψ(Xt ) converges to zero a.s. We can use Theorem 3.2 and Theorem 3.4 for this problem and do not need to
consider the difference ψ̂ − ψ pointwise for x ∈ H. We have the following theorem
Theorem 4.1. Let (X, ε) = {(Xk , εk ) : k ∈ Z} ⊆ H × R be a stochastic process as in Theorem 3.4. Let {εk : k ∈ Z}
be a sequence of innovations such that the regression model (4.1) is fulfilled. Let (4.3) be satisfied for h → 0 as n → ∞.
Then for a component Xt of the process (t ∈ Z) it is true that ψ̂(Xt ) → ψ(Xt ) a.s as n → ∞.
Proof of Theorem 4.1. We show that under the conditions we have both fˆh (Xt ) → M a.s. and ĝh (Xt ) → ψ(Xt )M
h
i
a.s. We begin with fˆh (Xt ). We have that fˆh (Xt ) − E fˆh (x)
→ 0 a.s. Indeed, let ε > 0 be arbitrary but
x=Xt

fixed, we apply Equation (3.6) from Theorem 3.4 with the following parameters
B=

(log n)2

n
,
(log log n)2

p = 3/2,

k = 3 and u = r = 2.

Note that fn (X0 , X0′ ) corresponds to the function K (kX0 − X0′ k /h) /FX0′ (h) in this case. An application of the
theorem of Fubini-Tonelli yields
"
3 #


K (kX0 − X0′ k /h)
′ k
E fn(X0 , X0 ) = E
FX0′ (h)



Z 1
−2


FX ′ (hs)
· K(1)3 −
(K(s)3 )′ 0
= E FX0′ (h)
ds
= O E FX0′ (h)−2
FX0′ (h)
0

by the uniform convergence of the small probability from (4.4). Hence, we obtain for some constants a1 , a2 ∈ R+ that


h
i
ˆ
ˆ
≥ ε ≤ a1 ε−1 exp (−a2 log n · log log n)
P fh (Xt ) − E fh (x)
x=Xt
(4.5)


−1
2
2 −2
−2
′
+ a1 ε (log n) (log log n) n E FX0 (h)
.

Consequently, the probabilities from the left hand side of (4.5) are summable
h over ni ∈
ˆ
of the choice of the bandwidth from (4.3). Thus, the claim that fh (Xt ) − E fˆh (x)

N+ for ε > 0 fixed because

x=Xt

8

→ 0 a.s. follows with an

application of the first Borel-Cantelli Lemma. An application of the theorem of Fubini-Tonelli yields


i
h
K(kX0 − xk /h)
ˆ
−M
sup E fh (x) − M = sup E
Fx (h)
x∈H
x∈H
Z 1
Fx (hs)
− τ (s) ds → 0.
≤ sup
K ′ (s)
Fx (h)
x∈H 0
The last inequality follows similarly as in Ferraty et al. (2007) and from the requirement (4.4). This proves that
fˆh (Xt ) → M > 0 a.s. Consider next ĝh (Xt ). Once more, we have ĝh (Xt ) − E [ ĝh (x) ]

x=Xt

→ 0 a.s. using

the requirement (4.3). Furthermore, we obtain for a point x ∈ H with the assumption that the regression function ψ is
Lipschitz continuous and that the conditional expectation of the innovations is zero




K(kX0 − xk /h)
K(kX0 − xk /h)
|E [ ĝh (x) ] − M ψ(x)| ≤ E (ψ(X0 ) − ψ(x))
−M
+ |ψ(x)| E
Fx (h)
Fx (h)


K(kX0 − xk /h)
−M .
≤ LhM + (|ψ(x)| + o(1)) E
Fx (h)

E [ ĝh (x) ]

− M ψ(Xt ) → 0 a.s. and proves the second statement ĝh (Xt ) → ψ(Xt )M a.s.
All in all, we have ψ̂(Xt ) − ψ(Xt ) = ĝh (Xt )/fˆh (Xt ) − ψ(Xt ) → 0 a.s. by the continuous mapping theorem.

This ensures that

A

x=Xt

Appendix

The following statement can be seen as a multivariate generalization of Davydov (1968) in a special case:
Proposition A.1 (Ibragimov (1962)). Let Z1 , . . . , Zn be real-valued non-negative random variables each a.s. bounded.
Denote by α := supk∈{1,...,n} α (σ(Zi : i ≤ k), σ(Zi : i > k)). Then

E

"

n
Y

i=1

Zi

#

−

n
Y

E [ Zi ] ≤ (n − 1) α

i=1

n
Y

kZi k∞ .

i=1

Acknowledgments
The author is very grateful to a referee for thoughtful suggestions and comments which significantly improved and clarified the manuscript.

References
I. A. Ahmad and M. Amezziane. Probability inequalities for bounded random vectors. Statistics & Probability Letters,
83(4):1136–1142, 2013.
M. A. Arcones. A Bernstein-type inequality for U-statistics and U-processes. Statistics & Probability Letters, 22(3):
239–247, 1995.
S. Bernstein. Sur l’extension du théorème limite du calcul des probabilités aux sommes de quantités dépendantes.
Mathematische Annalen, 97(1):1–59, 1927.
R. C. Bradley. Basic properties of strong mixing conditions. A survey and some open questions. Probability surveys, 2
(2):107–144, 2005.

9

W. Bryc and A. Dembo. Large deviations and strong mixing. In Annales de l’IHP Probabilités et statistiques, volume 32,
pages 549–569, 1996.
M. Carbon. Inégalité de Bernstein pour les processus fortement mélangeants non nécessairement stationnaires. C.R.
Acad. Sc. Paris I, 297:303–306, 1983.
G. Collomb. Propriétés de convergence presque complète du prédicteur à noyau. Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete, 66(3):441–460, 1984.
Y. A. Davydov. Convergence of distributions generated by stationary stochastic processes. Theory of Probability & Its
Applications, 13(4):691–696, 1968.
J. Dedecker and C. Prieur. Coupling for τ -dependent sequences and applications. Journal of Theoretical Probability, 17
(4):861–885, 2004.
L. Delsol. Advances on asymptotic normality in non-parametric functional time series analysis. Statistics, 43(1):13–33,
2009.
P. Doukhan. Mixing, volume 85 of Lecture Notes in Statistics. Springer-Verlag, New York, 1994.
F. Ferraty, A. Mas, and P. Vieu. Nonparametric regression on functional data: inference and practical aspects. Australian
& New Zealand Journal of Statistics, 49(3):267–286, 2007.
F. Ferraty, I. v. Keilegom, and P. Vieu. On the validity of the bootstrap in non-parametric functional regression. Scandinavian Journal of Statistics, 37(2):286–306, 2010.
L. Györfi, M. Kohler, A. Krżyzak, and H. Walk. A distribution-free theory of nonparametric regression. Springer Berlin,
New York, Heidelberg, 2002.
W. Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American statistical
association, 58(301):13–30, 1963.
I. A. Ibragimov. Some limit theorems for stationary processes. Theory of Probability & Its Applications, 7(4):349–382,
1962.
A. N. Kolmogorov and Y. A. Rozanov. On strong mixing conditions for stationary Gaussian processes. Theory of
Probability & Its Applications, 5(2):204–208, 1960.
J. T. N. Krebs. A Bernstein inequality for exponentially growing graphs. arXiv:1701.04188 [math, stat], Preprint,
submitted, 2017.
F. Merlevède, M. Peligrad, and E. Rio. Bernstein inequality and moderate deviations under strong mixing conditions,
volume Volume 5 of Collections, pages 273–292. Institute of Mathematical Statistics, Beachwood, Ohio, USA, 2009.
M. Rosenblatt. A central limit theorem and a strong mixing condition. Proceedings of the National Academy of Sciences,
42(1):43–47, 1956.
E. Valenzuela-Domı́nguez, J. T. N. Krebs, and J. E. Franke. A Bernstein inequality for spatial lattice processes. arXiv
preprint arXiv:1702.02023, 2017.

10

