Kernel entropy estimation for linear processes
Hailin Sanga , Yongli Sangb and Fangjun Xuc

arXiv:1712.00196v1 [math.ST] 1 Dec 2017

a

Department of Mathematics, University of Mississippi, University, MS 38677, USA
sang@olemiss.edu

b

Department of Mathematics, University of Louisiana at Lafayette, Lafayette, LA 70504, USA
yongli.sang@louisiana.edu

c

School of Statistics, East China Normal University, Shanghai 200241, China and NYU-ECNU
Institute of Mathematical Sciences at NYU Shanghai, Shanghai 200062, China
fangjunxu@gmail.com; fjxu@finance.ecnu.edu.cn
Abstract
Let {Xn : n ∈ N} be a linear process with bounded
probability density function f (x). We
R
study the estimation of the quadratic functional R f 2 (x) dx. With a Fourier transform on the
kernel function and the projection method, it is shown that, under certain mild conditions, the
estimator


X
2
Xi − Xj
K
n(n − 1)hn
hn
1≤i<j≤n

has similar asymptotical properties as the i.i.d. case studied in Giné and Nickl (2008) if the linear
process {Xn : n ∈ N} has the defined short range dependence. We also provide an application
to L22 divergence and the extension to multivariate linear processes. The simulation study for
linear processes with Gaussian and α-stable innovations confirms our theoretical results. As
an illustration, we estimate the L22 divergences among the density functions of average annual
river flows for four rivers and obtain promising results.
Keywords: linear process, kernel entropy estimation, quadratic functional, projection operator
2010 Mathematics Subject Classification: Primary 60F05; Secondary 62M10, 60G10,
62G05.

1

Introduction

Let f (x) be the probability density function ofRa sequence of identically distributed observations
2
{Xn }∞
n=1 . The quadratic functional Q(f ) R= R f (x) dx plays an important role in the study
2
of quadratic
R Rényi entropy R(f ) = − ln( f (x) dx), Rényi (1970), and the Shannon entropy
S(f ) = − f (x) ln f (x) dx, Shannon (1948). Entropy is widely applied in the fields of information
theory, statistical classification, pattern recognition and so on since it is a measure of uncertainty
in a probability distribution. In the literature, different estimators for the quadratic functional and
entropies with independent data have been well studied. For example, the nearest-neighbor estimator [Leonenko, Pronzato and Savani (2008); Penrose and Yukich (2013)], the kernel estimator
[Hall and Marron (1987); Bickel and Ritov (1988); Giné and Nickl (2008)], the orthogonal projection estimator [Laurent (1996, 1997)] and the U -statistics estimator (a special kernel estimator)
[Leonenko and Seleznjev (2010); Källberg, Leonenko and Seleznjev (2012)] under the independence
assumption. It is a challenging problem to study the estimation of the quadratic functional and the
corresponding entropies for dependent case. In Källberg, Leonenko and Seleznjev (2014) the authors successfully extended the U -statistics method to m-dependence sequence. They showed the
rate optimality and asymptotic normality of the U -statistics estimator for multivariate sequence.
1

Ahmad (1979) obtained the strong consistency of the quadratic functional by orthogonal series
method for stationary time series with strong mixing condition. Nevertheless, to the best of our
knowledge, general results for quadratic functional estimations of regular time series data under
mild conditions are still unknown.
R
In this paper, we study the quadratic functional R f 2 (x) dx for the following linear process
Xn =

∞
X

ai εn−i ,

(1)

i=0

where the innovations εi are independent and identically distributed (i.i.d.) real-valued random
∞
P
ai εn−i convariables in some probability space (Ω, F, P) and ai are real coefficients such that
i=0

verges in distribution. We would apply the kernel method which was first introduced by Rosenblatt
(1956) and Parzen (1962). This method was proved to be successful in estimating the probability density functions and their derivatives, the regression functions and so on in the independent
setting; see the books [Devroye and Györfi (1985); Silverman (1986); Nadaraya (1989); Wand and
Jones (1995); Schimek (2000); Scott (2015)] and references therein. Kernel method was also proved
to be successful in estimating the density functions and regression functions for time series data
[see Tran (1992); Honda (2000); Wu and Mielniczuk (2002); Wu, Huang and Huang (2010)].
In the classical kernel estimation of the quadratic functional Q(f ) with i.i.d. observations
n
{X
R i }2i=1 , Giné and Nickl (2008) applied a convolution method to obtain the bias E Tn (hn ) −
R f (x) dx of the kernel estimator


X
Xi − Xj
2
Tn (hn ) =
.
(2)
K
n(n − 1)hn
hn
1≤i<j≤n

They then used the Hoeffding’s decomposition for U -statistics to study the stochastic part Tn (hn )−
E Tn (hn ). In this way, they showed the rate optimality and efficiency of the kernel estimator for
the quadratic functional. Krishnamurthy et. al. (2015) applied a similar method to study the
L22 divergence between two distributions.
R Nevertheless, this method does not work well in the
estimation of the quadratic functional R f 2 (x)dx for linear process Xn given in (1) due to the
dependence structure.
Instead we utilize the Fourier transform and projection methods to derive the asymptotic
properties of the kernel estimator for the quadratic functional on time series data. With the
help of the Fourier transform, one can easily separate the random part Xi − Xj and the kernel
function K in the estimator Tn (hn ) given in (2) and also obtain the Hoeffding’s decomposition for
Tn (hn ) − E Tn (hn ) when {Xn } is a linear process; see (33). In fact, the Hoeffding’s decomposition
works for general stationary processes. Then we use projection and some kind of chain rule to
determine which terms contribute to the limit and which terms are negligible. Our method is
different from the martingale approximation method applied in Wu and Mielniczuk (2002) when
studying the kernel density estimation for linear processes. Moreover, this method allows us to get
rid of the 2nd moment assumption on the innovations of linear processes, that is, E |ε1 |2 < ∞. This
allows innovations to take heavy tail distributions, which are often followed by data in finance,
see Mittnik, Rachev and Paolella (1998). We only need E |eιλε1 − φε (λ)|2 ≤ cγ,2 (|λ|2γ ∧ 1) or
E |eιλε1 − φε (λ)|4 ≤ cγ,4 (|λ|4γ ∧ 1) for some γ ∈ (0, 1], where φε (λ) is the characteristic function
of the innovations. The range γ ∈ (0, 1] here is optimal when ε1 has non-degenerate distribution.
In this paper, the innovations have non-degenerate distribution because of the existence of the
probability density function f (x) of the linear process. See Lemma 7.4. On the other hand,
our method gives a clear picture on the relationship between the linear process {Xn } and the
efficiency of the estimator Tn (hn ). In addition, this methodology would have applications in kernel
2

estimations. For example, we can use it to extend results in Krishnamurthy et. al. (2015) to
short memory linear processes, and simplify existing proofs and assumptions in kernel density
estimations. Furthermore, we could suggest a more general definition for short memory and long
memory linear processes (see Definition 2.1 below) and extend our results to multivariate linear
processes.
Throughout this paper, if not mentioned otherwise, the letter c, with or without a subscript,
denotes a generic positive finite constant √
whose exact value is independent of n and may change
from line to line. We use ι to denote −1. For a complex number z, we use z and |z| to
denote its conjugate and modulus, respectively. The notation k · k2 means [E | · |2 ]1/2 . For two
functions a(x) and b(x) of real numbers, a(x)∼b(x) means a(x)/b(x)R → 1 as x → ∞. For any
integrable function g(x), its Fourier transform is defined as gb(u) = R eιxu g(x) dx. Let φ(λ) be
∞
P
ai εn−i , φn (λ) the empirical characteristic
the characteristic function of linear process Xn =
i=0

function from observations {Xi }ni=1 and φε (λ) the characteristic function of the innovations. That
n
P
is, φ(λ) = E [eιλXi ], φn (λ) = n1
eιλXi and φε (λ) = E [eιλε1 ]. For each i ∈ N, define H(Xi ) :=
i=1

H(Xi )(λ) := eιλXi − φ(λ), λ ∈ R.
The paper has the following structure. The main results are given in Section 2. We then give
a few examples in Section 3, an application of the results to L22 divergence in Section 4, simulation
study and real data analysis in Section 5, and extensions to multivariate linear processes in Section
6. Section 7 is devoted to the proofs of Theorems 2.1 and 2.2, based on the Fourier transform.

2

Main results

If {εi : i ∈ Z} is a sequence of i.i.d. random variables in Lp (R) for some p > 0, E εi = 0 when p ≥ 1,
∞
P
and {ai }∞
|ai |2∧p < ∞, then the linear process Xn
i=0 is a sequence of real coefficients such that
i=0

given in (1) exists and is well-defined. For p = 2, the process has short memory (short range
∞
P
dependence) if
|ai | < ∞ and long memory (long range dependence) otherwise. As for linear
i=0

processes with symmetric α-stable innovations (0 < α ≤ 2), i.e., the law of innovations having
characteristic function E [eιλε1 ] = exp(−cα |λ|α ) for some positive constant cα only depending on
∞
∞
P
P
α, {Xn } has short memory if
|ai |α/2 < ∞ and it has long memory if
|ai |α/2 = ∞ but
∞
P

i=0

i=0

|ai |α < ∞, respectively. See, e.g., Hsing (1999).

i=0

Let f (x) be the probability density function of the linear process Xn =

∞
P

ai εn−i , n ∈ N defined

i=0 R

in (1). We study the estimation of the quadratic functional of f (x), that is, R f 2 (x)dx, when the
∞
P
linear process Xn =
ai εn−i has short memory in the sense of the following definition.
i=0

Definition 2.1 A linear process Xn =

∞
P

ai εn−i defined in (1) has short memory if

i=0
∞ q
X
Var (eιλai ε1 ) < ∞
i=1

3

for all λ close enough to 0, and long memory if
∞
∞ q
X
X
Var (eιλai ε1 ) = ∞, but
Var (eιλai ε1 ) < ∞
i=1

i=1

for all λ close enough to 0.
In fact this definition generalizes the definitions of short memory or long memory linear processes
stated at the beginning of this section. Note that Var (eιλε1 ) can also be written as E |eιλε1 −φε (λ)|2
or 1 − |φε (λ)|2 . By the first part of Lemma 7.3, this definition contains the original one for linear
processes with finite second moment innovations. Using the characteristic function of α-stable
distribution, E |eιλε1 − φε (λ)|2 ≤ c α2 ,2 (|λ|α ∧ 1). Hence it also contains the one introduced in Hsing
(1999) for symmetric α-stable innovations. See the details in Example 3.2.
Moreover, Definition 2.1 can be used to define short and long memory linear processes with
general infinite variance innovations. Therefore, it can be applied to define all short and long
memory linear processes with finite variance innovations or infinite variance innovations. The
advantage of Definition 2.1 is that we only need to know the behavior of Var (eιλai ε1 ) = 1 −
|φε (ai λ)|2 without any moment assumption. This definition relates the coefficients {ai } with the
distribution of the innovations which is fully represented by the characteristic function. By contrast,
the traditional definition only relates the coefficients {ai } with the moment information of the
innovations. The definition in Hsing (1999) is only for linear processes with symmetric α-stable
innovations and it only relates the coefficients {ai } with the parameter α of the α-stable innovations.
Throughout the paper, we use Definition 2.1 to classify linear processes with short or long memory.
R
To estimate the quadratic functional R f 2 (x)dx, we shall apply the kernel method


X
Xi − Xj
2
Tn (hn ) =
K
,
n(n − 1)hn
hn
1≤i<j≤n
R
R
where the kernel K is a symmetric and bounded function with R K(u) du = 1 and R u2 |K(u)| du <
∞. The bandwidth sequence hn satisfies 0 < hn → 0 as n → ∞. The following are the main results
of this paper.
Theorem 2.1 Assume that

∞
P

|ai |γ < ∞,

i=0

2γ
2
R |λ| |φε (λ)| dλ

R

< ∞ and


E |eιλε1 − φε (λ)|4 ≤ cγ,4 |λ|4γ ∧ 1

(3)

for some γ ∈ (0, 1]. We further assume that f is bounded. Then there exist positive constants c1
and c2 such that


Z
1
2γ
2
+ hn ,
(4)
E Tn (hn ) −
f (x) dx ≤ c1
n
R
n

 1

ηn,γ
1 X 2
h2γ
n
E Tn (hn ) − E Tn (hn ) −
Yi ≤ c2 2 + 2 +
n
n hn
n
n


(5)

i=1

and, if additionally nhn → ∞ as n → ∞,
i
√ h
L
n Tn (hn ) − E Tn (hn ) −→ N (0, 4σ 2 ),
where Yi = 2 f (Xi ) −

R

n P
∞

P
2 (x) dx , η
f
=
|ai |γ and σ 2 = lim n−1 Var (Sn ) with
n,γ
R
n→∞

`=0 i=`


Z
n 
X
2
Sn =
f (Xi ) −
f (x) dx .
R

i=1

4

(6)

Theorem 2.2 Assume that

∞
P
i=0

|ai |γ < ∞,

2γ
2
R |λ| |φε (λ)| dλ

R

< ∞ and


E |eιλε1 − φε (λ)|2 ≤ cγ,2 |λ|2γ ∧ 1
(7)
R
b
for some γ ∈ (0, 1]. We further assume that f is bounded and R |K(λ)|
dλ < ∞. Then there exist
positive constants c3 and c4 such that


Z
1
2
2γ
f (x) dx ≤ c3
E Tn (hn ) −
+ hn ,
(8)
n
R
n

 1
hγn 
1X
+√
Yi ≤ c4
E Tn (hn ) − E Tn (hn ) −
n
nhn
n

(9)

i=1

and, if additionally

√

nhn → ∞ as n → ∞,
i
√ h
L
n Tn (hn ) − E Tn (hn ) −→ N (0, 4σ 2 )

(10)

for some σ 2 ∈ (0, ∞).
Remark 2.1 By Definition 2.1, Theorems 2.1 and 2.2 are results for short memory linear processes. On the other hand, we work on the well-defined linear process (1) in Theorem 2.1, Theorem
∞
P
2.2 and the rest of the paper. If we assume ε1 ∈ Lγ (R), together with condition
|ai |γ < ∞ in the
i=0

theorems, then the linear process (1) is well-defined. However, the moment condition ε1 ∈ Lγ (R)
is not needed in the proofs of our results.
Remark 2.2 When {Xn } is an i.i.d. sequence, the kernel estimator (2) was studied in Giné
and Nickl R(2008) with assumptions that the density function f (x) is
and the smoothness
R bounded
1
2γ
2
2
condition R |λ| |φ(λ)|R dλ < ∞ of f (x) for γ restricted to (0, 2 ]. If R u |K(u)| du < ∞ is replaced
by a weaker condition R |u|β |K(u)| du < ∞ for some 0 < β ≤ 2, then the upper bounds in (4) and
(8) above should be replaced by a constant multiple of n1 + hβ∧2γ
.
n
Remark 2.3 The assumption (3) in Theorem 2.1 is required to obtain (5) and (6). To just get the
inequality (4), we only need to assume (7). Using Lemma 7.3, we could easily see that E |ε1 |4γ < ∞
and E |ε1 |2γ < ∞ are sufficient for these two assumptions, respectively. However, these two moment conditions are not necessary. For example, when the innovation ε1 takes symmetric α-stable
distributions with α ∈ (0, 2), E |eιλε1 − φε (λ)|4 ≤ c α4 ,4 (|λ|α ∧ 1), E |eιλε1 − φε (λ)|2 ≤ c α2 ,2 (|λ|α ∧ 1),
but E |ε1 |α = ∞. On the other hand, assumption (3) is stronger than assumption (7). This explains
why the results in Theorem 2.1 are better than the results in Theorem 2.2 if both assumptions (3)
and (7) hold for a same γ in (0, 1].
Remark 2.4 If the underlying linear
process {Xn } is i.i.d. or m-dependent, then the assumption

E |eιλε1 − φε (λ)|4 ≤ cγ,4 |λ|4γ ∧ 1 for some γ ∈ (0, 1] in Theorem 2.1 can be removed by making
some minor modifications of its proof, and ηn,γ can be replaced by 0. Furthermore, for the i.i.d.
case, the right hand side of (4) can be replaced by c1 h2γ
n .
R
Remark 2.5 In Theorems 2.1 and 2.2, if we further have R |λ|4γ |φε (λ)|2 dλ < ∞, then, using
2β for all β ∈ [0, 1], the inequality (5) can be improved to
b
b
the fact |K(λh
n ) − K(0)| ≤ cβ |λhn |
n

1X
E Tn (hn ) − E Tn (hn ) −
Yi
n
i=1

5

2

≤ c2

 1

ηn,γ
h4γ
n
+
+
n2 hn
n2
n

while (9) to
n

E Tn (hn ) − E Tn (hn ) −

 1

h2γ
1X
n
+√ .
Yi ≤ c4
n
nhn
n
i=1

As consequences of Theorem 2.1 and Theorem 2.2, we can easily obtain the following results.
Corollary 2.1 Under the assumptions in Theorem 2.1. Let hn have order n
then
Z
− 4γ
f 2 (x) dx = OP (n 4γ+1 );
Tn (hn ) −

2
− 4γ+1

. If 0 < γ ≤ 1/4,

(11)

R

if 1/4 < γ ≤ 1, then
Z
i
√ h
L
n Tn (hn ) −
f 2 (x) dx −→ N (0, 4σ 2 ),

(12)

R

and, if we additional requireR K(x) ≥ 0 for all x, the estimator − ln( n1 + Tn (hn )) of the quadratic
Rényi entropy R(f ) = − ln( R f 2 (x) dx) satisfies


i
√ h
1
4σ 2
L
,
n − ln( + Tn (hn )) − R(f ) −→ N 0, R 2
n
( R f (x) dx)2
where σ 2 = lim n−1 Var (Sn ) with Sn =
n→∞

n
P

f (Xi ) −

i=1

R

Rf

2 (x) dx



(13)

.

Corollary 2.2 Under the assumptions in Theorem 2.2. Let hn have order n
then
Z
− 2γ
Tn (hn ) −
f 2 (x) dx = OP (n 2γ+1 );

1
− 2γ+1

. If 0 < γ ≤ 1/2,

(14)

R

if 1/2 < γ ≤ 1, then
Z
i
√ h
L
f 2 (x) dx −→ N (0, 4σ 2 ),
n Tn (hn ) −

(15)

R

and, if we additional requireR K(x) ≥ 0 for all x, the estimator − ln( n1 + Tn (hn )) of the quadratic
Rényi entropy R(f ) = − ln( R f 2 (x) dx) satisfies


i
√ h
1
4σ 2
L
n − ln( + Tn (hn )) − R(f ) −→ N 0, R 2
n
( R f (x) dx)2
for some σ 2 ∈ (0, ∞).

3

Examples

In this section, we shall give a few examples of linear processes to illustrate our results.

6

(16)

Example 3.1 (Linear process with Gaussian white noise innovations) Here the distribuλ2
tion
Gaussian with mean 0 and variance σ 2 .  Note that E [eιλε1 ] = exp(− 2σ
So
2 ).

R 2 of ε1 is
2 dλ < ∞, E |eιλε1 − φ (λ)|4 ≤ c
4 ∧ 1 and E |eιλε1 − φ (λ)|2 ≤ c
2∧1 .
λ
|φ
(λ)|
λ
λ
ε
ε
1,4
ε
1,2
R
Therefore, Theorems 2.1, 2.2 and Corollaries 2.1 and 2.2 hold for all short memory linear processes with Gaussian white noise innovations.
Example 3.2 (Linear process with symmetric α-stable innovations) Here the distribution of ε1 is the symmetric α-stableR (SαS) with 0 < α < 2. Note that E [eιλε1 ] = exp(−cα |λ|α )
for some positive constant cα . So R λ2 |φε (λ)|2 dλ < ∞, E |eιλε1 − φε (λ)|4 ≤ c α4 ,4 (|λ|α ∧ 1) and
E |eιλε1 − φε (λ)|2 ≤ c α2 ,2 (|λ|α ∧ 1). Therefore, Theorems 2.1 and 2.2 hold for all short memory
linear processes having symmetric α-stable innovations (0 < α < 2) with γ = α4 and γ = α2 , respectively. The central limit theorem (12) in Corollary 2.1 and the central limit theorem (15) in
Corollary 2.2 hold if 1 < α < 2. We should apply Theorem 2.2 and Corollary 2.2 instead of Theorem 2.1 and Corollary 2.1 P
in order to study all short-memory processes with α-stable innovations
and coefficients satisfying i |ai |α/2 < ∞. See Step 2 of Section 5 for the detailed discussion.
Example 3.3 (Causal ARM A(p, q) process) An ARM A(p, q) model has the form
φ(B)Xt = θ(B)εt ,
where εt are i.i.d random variables with E εt = 0 and E ε2t = σε2 , φ and θ are two polynomials with
no common zeros,
φ(z) = 1 − φ1 z − φ2 z 2 − · · · − φp z p , φp 6= 0,
θ(z) = 1 + θ1 z + θ2 z 2 + · · · − θq z q , θq 6= 0,
and B is the back shift operator with B k Xt = Xt−k . An ARM A(p, q) process is causal if it can be
∞
∞
P
P
written as a one-sided linear process Xt =
ai εt−i with
|ai | < ∞. It is causal if and only if
i=0

i=0

φ(z) 6= 0 for |z| ≤ 1. In this case, the coefficients of the linear process can be determined by solving
a(z) =

∞
X

ai z i =

i=0

φ(z)
,
θ(z)

where |z| ≤ 1. It is easy to see that the coefficients ai decay exponentially fast to zero. So
∞ P
∞
P
|ai |γ < ∞ for any γ > 0. This means that we can choose γ as small as possible, which will
`=0 i=`

enlarge the choices of the innovations ε. Moreover, the upper bound in (5) of Theorem 2.1 can be
2γ
improved to a constant multiple of n21hn + hnn .

4

An application to L22 divergence

Let f (x) and g(x) be the probability densities for independent linear processes Xn =
and Yn =

∞
P
j=0

∞
P

ai εn−i

i=0

bj ξn−j defined as in (1), respectively. Then the L22 divergence between f and g is

defined as
Z
D(f, g) :=
R

(f (x) − g(x))2 dx =

Z

f 2 (x) dx +

R

Z
R

7

g 2 (x) dx − 2

Z
f (x)g(x) dx.
R

The estimator for D(f, g) is
b g) = Tn (f, hn ) + Tn (g, hn ) − 2Tn (f, g, hn ),
D(f,
where


Xi − Xj
K
,
hn
1≤i<j≤n


X
Yi − Yj
2
K
Tn (g, hn ) =
n(n − 1)hn
hn

2
Tn (f, hn ) =
n(n − 1)hn



X

1≤i<j≤n

and

1
Tn (f, g, hn ) = 2
n hn



X

K

1≤i,j≤n

Xi − Yj
hn


.

Krishnamurthy et. al. (2015) studied the estimation of L22 divergence for multivariate independent
2n
i.i.d. samples {Xi }2n
i=1 ∼ f (x) and {Yi }i=1 ∼ g(x). They used data splitting in the estimation
which is necessary to obtain the asymptotic normality [see Gretton et. al. (2012); Krishnamurthy
et. al. (2015)]. However, in our case, the data splitting is not needed.
Using similar arguments as in the proof of Theorems 2.1 and 2.2, one can obtain the same
results for Tn (f, g, hn ). Together with Theorems 2.1 and 2.2, we then have the following results.
Theorem 4.1 Assume that the independent linear processes {Xn } and {Yn } satisfy the same
conditions as in Theorem 2.1. Then there exist positive constants c1 and c2 such that


1
2γ
b
E D(f, g) − D(f, g) ≤ c1
+ hn ,
(17)
n
n

X
b g) − E D(f,
b g) − 1
E D(f,
Wi
n

2

i=1


 1
ηn,γ
h2γ
n
≤ c2 2 + 2 +
n hn
n
n

and, if additionally nhn → ∞ as n → ∞,
i
√ h
L
b g) − E D(f,
b g) −→
n D(f,
N (0, 4σ 2 ),

(18)

(19)

where
Z
Z
Z


f 2 (x) dx + g(Yi ) −
g 2 (x) dx + g(Xi ) + f (Yi ) − 2 f (x)g(x) dx ,
Wi = 2 f (Xi ) −
R

ηn,γ =

n P
∞
P
`=0 i=`

|ai |γ ∨

n P
∞
P

R

|bi |γ , and σ 2 = lim n−1 Var (Sn ) with Sn =
n→∞

`=0 i=`

Let hn have order n

R

2
− 4γ+1

1
2

n
P

Wi .

i=1

b g).
. Then similar results as in Corollary 2.1 can be obtained for D(f,

Theorem 4.2 With the same notations as in Theorem 4.1, assume that the independent linear
processes {Xn } and {Yn } satisfy the same conditions as in Theorem 2.2. Then there exists a
positive constant c such that
n
 1
X
hγn 
b g) − E D(f,
b g) − 1
E D(f,
Wi ≤ c
+√
n
nhn
n

(20)

i=1

and (17), (19) hold for some
Let hn have order n

1
− 2γ+1

σ2

∈ (0, ∞).

b g).
. Then similar results as in Corollary 2.2 can be obtained for D(f,

8

5

Simulation study and real data analysis

In this section we first conduct a simulation study to verify the central limit theorems in Section
2 for the kernel entropy estimator of linear processes with different coefficient sequences and innovations with stable or Gaussian laws. For this purpose, we shall draw normal quantile plots and
histograms, perform hypothesis tests and provide confidence intervals on the means. In the second
part of this section, we apply the kernel method to study the divergences between each pair of the
annual average flow distributions of four rivers on the earth.
The simulation study has two steps.
R
Step 1: Calculate or approximate the true value of the quadratic functional R f 2 (x)dx.
By the inverse Fourier transform, the probability density function f (x) of the linear process in
(1) and its characteristic function φ(t) = E[eιtX1 ] have the following relationship:
Z
1
f (x) =
φ(t)e−ιtx dt.
(21)
2π R
Also, we have
φ(t) = E[eιtXn ] =

∞
Y

E[eιtai εn−i ].

i=0
Γ(i+d)
Now let ai = Γ(d)Γ(i+1)
, i ≥ 0, for any d < 21 , d 6= 0, −1, −2, · · · . Applying Stirling’s formula,
√
Γ(x) ∼ 2πe−x+1 (x − 1)x−1/2 as x → ∞, ai ∼ id−1 /Γ(d) as i → ∞. In particular, under the
condition −1 < d < 21 , d 6= 0, the FARIMA(0, d, 0) processes Xn = (1 − B)−d εn , where B is the
Γ(i+d)
backshift operator, are causal and they have the form of linear process with ai = Γ(d)Γ(i+1)
. See
Bondon and Palma (2007) for the extension of causality of FARIMA(p, d, q) processes to the range
of −1 < d < 1/2. We shall perform simulation study for linear processes with Gaussian or Cauchy
innovations and some selected values of d.
t2
Case 1. Suppose the innovations {εi } are i.i.d. N (0, 1) random variables, then φε (t) = e− 2
t2

P∞

2

and hence φ(t) = e− 2 i=0 ai . By the Gauss’s theorem [Gauss (1866)] for hypergeometric series,
P∞ 2 Γ(1−2d)
1
i=0 ai = Γ2 (1−d) for any d < 2 , d 6= 0, −1, −2, · · · . See also Bailey (1935) or its direct calculation
in Sang and Sang (2017). By the formula (21) and the application of MATLAB,
√
Z
2 − πx2
1 ∞ − t22 ΓΓ(1−2d)
2 (1−d)
f (x) =
e
cos(tx)dt =
e 8
π 0
4
if we take d = −0.5. As a result,
Z

f 2 (x)dx = 0.2500 if d = −0.5.

R

In this way, we also calculate or approximate the value of
in Table 1.
Table 1: The approximate value of
d
R

2
R f (x)dx

−0.1
0.2801

−0.3
0.2678

R

Rf

2 (x)dx

−0.5
0.2500

R

Rf

2 (x)dx

for other d values as listed

for linear processes with Gaussian innovations

−0.7
0.2300

−0.9
0.2095

−1.5
0.1531

−2.5
0.0856

−3.5
0.0462

Case 2. If the innovations {εi } P
have i.i.d. symmetric α-stable distribution with 0 < α < 2,
∞
α
α
α
then φε (t) = e−|t| and φ(t) = e−|t| i=0 |ai | . In particular, if α = 1, we have the standard Cauchy
9

P∞

−|t| i=0 |ai |
distribution with φε (t) = e−|t| and
P∞φ(t) = e P100,000 . In this case, with the help of the software
|ai | and then calculate f (x) and the value of
we approximate i=0 |ai | by i=0
RMathematica,
2 (x)dx. The result is in Table 2.
f
R

Table 2: The approximate value of
d
R

2
R f (x)dx

R

Rf

−0.1
0.0934

2 (x)dx

−0.3
0.0806

for linear processes with Cauchy innovations
−0.5
0.0796

−0.7
0.0796

−0.9
0.0796

Step 2: For each case, linear processes with Gaussian or Cauchy innovations, and each memory
parameter d, we produce m = 1, 000 linear processes with n = 4, 096 observations {Xi }ni=1 in each
of them by applying the MATLAB code in Faÿ et. al. (2009) with some modification. For each
generated linear
R 2 process, we apply the estimator Tn (hn ) as in (2) to estimate the true quadratic
functional R f (x)dx. The estimation is realized in MATLAB. Here we choose the normal kernel,
x2

that is, K(x) = √12π e− 2 , in the estimator Tn (hn ). The other common kernel functions are also
applicable since the main theorems and corollaries only assume some basic conditions on the kernel
functions.
Γ(i+d)
1
For the chosen ai = Γ(d)Γ(i+1)
since
, i ≥ 0, and for all results in Section 2, we require γ > 1−d
P∞
γ
d−1
/Γ(d).
i=0 |ai | < ∞ and ai ∼ i
Case 1. Suppose the innovations εi , i ∈ Z, are i.i.d. N (0, 1) random variables. Then E [eιλε1 ] =

R 2
λ2
2
ιλε1 − φ (λ)|4 ≤ c
4
ιλε1 − φ (λ)|2 ≤
exp(− 2σ
ε
1,4 λ ∧ 1 and E |e
ε
2 ). So
R λ |φε (λ)| dλ < ∞, E |e
c1,2 λ2 ∧ 1 . In this case, we should apply Theorem 2.1 and Corollary 2.1 instead of Theorem
2.2 and Corollary 2.2. The convergence rate (11) in Corollary 2.1 is faster than the convergence
rate (14) in Corollary 2.2. The central limit theorem (12) in Corollary 2.1 holds for 1/4 < γ ≤ 1
while the central limit theorem (15) in Corollary 2.2 holds only for 1/2 < γ ≤ 1. See also Remark
1
2.3. To have the central limit theorem (12), we require 1−d
∨ 41 < γ ≤ 1 and the bandwidth hn
−

2

to have the order of n 4γ+1 . In this Gaussian case, we recommend to take γ = 1 and hn to have
2
the order of n− 5 to make (12) hold for all d < 0. To apply kernel method in estimation, one
should select an optimal bandwidth based on some criteria, for example, to minimize the mean
squared error.
R For the i.i.d. case, Giné and Nickl (2008) construct a kernel-based rate adaptive
in the first step. It is interesting
estimator of R f 2 (x)dx by selecting the bandwidth hn adaptively
R 2
to investigate the bandwidth adaptive kernel estimator of R f (x)dx for linear processes from both
theoretical and application viewpoints. However, it seems that the study in this direction is very
difficult. We leave it as an open question for future study. For the current simulation study, we
take hn = n−2/5 in the Gaussian case.
R
Case 2. In the α-stable case, R λ2 |φε (λ)|2 dλ < ∞, E |eιλε1 − φε (λ)|4 ≤ c α4 ,4 (|λ|α ∧ 1) and
E |eιλε1 − φε (λ)|2 ≤ c α2 ,2 (|λ|α ∧ 1). Therefore, Theorem 2.1 (Corollary 2.1) and Theorem 2.2
(Corollary 2.2) hold for all short memory linear processes having symmetric
α-stable innovations
P
α/4 < ∞ in Theorem
(0 < α < 2) with γ = α4 and γ = α2 , respectively. Hence we require ∞
|a
|
i=0 i
P
α/2 < ∞ in Theorem 2.2 and Corollary 2.2. For
2.1 and Corollary 2.1 and we only require ∞
|a
|
i
i=0
this reason, we should apply Theorem 2.2 and Corollary 2.2 instead of Theorem 2.1 and Corollary
2.1. Since γ = α2 , Theorem 2.2 holds for all α ∈ (0, 2) while in Corollary 2.2, we have (14) for
α ∈ (0, 1] and the central limit theorem (15) only for α ∈ (1, 2). Hence if 1 < α < 2, we should take
1
hn to have the order of n− α+1 to confirm the central limit theorem (15). If 0 < α ≤ 1, we should
√
not expect to have (15). But in this case we can take hn with nhn → ∞ to confirm the central
limit theorem (10) in Theorem 2.2. In the simulation study, we take hn = n−2/5 . In Theorem 2.2
and Corollary 2.2, the coefficients ai should have parameter d with d < 1 − γ1 = 1 − α2 .
10

Figure 1 and Figure 2 show
the histograms with kernel density fits and normal Q-Q plots of rn =
i
R 2
√ h
n Tn (hn ) − R f (x) dx from m = 1, 000 simulated linear processes with Gaussian innovations
and d = −0.1, −0.9, −3.5. It is clear that the realizations of rn are distributed as some normal
distribution in each case. We also have performed simulation study for d = −0.3, −0.5, −0.7, −1.5
and −2.5 and obtained similar plots and results.
We then perform hypothesis test H0 : µ = 0; H1 : µ 6= 0 for the mean of the asymptotic
distribution of rn and provide the 95% confidence interval for µ for each value of d. The result is
listed in Table 3. It is clear that rn is asymptotically centered in each case. Over all, the simulation
study here confirms the analysis for the Gaussian case based on the main results in Section 2.

2

Histogram: Gaussian case, d=-0.1

2.5

Histogram: Gaussian case, d=-0.9

8

1.8

Histogram: Gaussian case, d=-3.5

7

1.6

2
6

1.4
1.2

5

1.5

1

4

0.8

1

3

0.6
2
0.4

0.5
1

0.2
0
-1

-0.5

0

0.5

1

0
-1

1.5

-0.5

0

0.5

0
-0.3

1

-0.2

-0.1

0

0.1

0.2

0.3

Figure 1: The histograms with kernel density fits of rn from m = 1, 000 simulated processes with
Gaussian innovations and d = −0.1, −0.9, −3.5.

QQ Plot: Gaussian case, d=-0.1

0.8

Quantiles of Input Sample

Quantiles of Input Sample

0.2

0.6

0.6
0.4
0.2
0
-0.2
-0.4

-2

0

2

Standard Normal Quantiles

4

0.2
0
-0.2
-0.4

-0.8
-4

QQ Plot: Gaussian case, d=-3.5

0.15

0.4

-0.6

-0.6
-0.8
-4

QQ Plot: Gaussian case, d=-0.9

Quantiles of Input Sample

1
0.8

0.1
0.05
0
-0.05
-0.1
-0.15

-2

0

2

Standard Normal Quantiles

4

-0.2
-4

-2

0

2

4

Standard Normal Quantiles

Figure 2: The normal Q-Q plots of rn from m = 1, 000 simulated processes with Gaussian innovations and d = −0.1, −0.9, −3.5.

11

Table 3: The p-values on testing H0 : µ = 0; H1 : µ 6= 0 for the asymptotic distribution of rn and
the 95% confidence intervals for µ.
d
p-value
CI
d
p-value
CI

−0.1
0.3996
(-0.0191,0.0076)
−0.9
0.6705
(-0.0148, 0.0095)

−0.3
0.7810
(-0.0159,0.0120)
−1.5
0.8297
(-0.0064,0.0080)

−0.5
0.5058
(-0.0086,0.0174)
−2.5
0.2055
(-0.0021, 0.0098)

−0.7
0.6676
(-0.0099, 0.0154)
−3.5
0.2211
(-0.0058, 0.0013)

Figureh 3 and Figure 4 show
i the histograms with kernel density fits and normal Q-Q plots of
R 2
√
rn = n Tn (hn ) − R f (x) dx from m = 1, 000 simulated processes with Cauchy innovations and
d = −0.1, −0.5, −0.9. It is clear that the realizations of rn do not follow normal distribution in
the case d = −0.1 or d = −0.5. We also have performed simulation study for d = −0.3, −0.7 and
obtained similar plots and results. For d = −0.9, the histogram and normal Q-Q plot indicate that
the distribution of rn is very close to some normal distribution with mean 0. This is because the
central limit theorem (15) in Corollary 2.2 holds for α = 1 +  with  an arbitrary small positive
number, d < 1 − α2 , which is close to the case α = 1, d = −0.9 here. In the case 0 < α ≤ 1, we
have the central limit theorem (10) although (15) does not hold any more.

0.6

Histogram: Cauchy case, d=-0.1

1.2

0.5

1

0.4

0.8

0.3

0.6

0.2

0.4

0.1

0.2

Histogram: Cauchy case, d=-0.5

2.5

Histogram: Cauchy case, d=-0.9

2

1.5

1

0
-8

-6

-4

-2

0

2

4

0
-4

0.5

-3

-2

-1

0

1

2

0
-1.5

-1

-0.5

0

0.5

1

Figure 3: The histograms with kernel density fits of rn from m = 1, 000 simulated processes with
Cauchy innovations and d = −0.1, −0.5, −0.9.

12

QQ Plot: Cauchy case, d=-0.1

4

1.5

QQ Plot: Cauchy case, d=-0.5

0.6

1

2
1
0
-1
-2
-3
-4

0.5
0
-0.5
-1
-1.5
-2

-6
-4

-2

0

2

4

-3
-4

Standard Normal Quantiles

0.2
0
-0.2
-0.4
-0.6
-0.8

-2.5

-5

QQ Plot: Cauchy case, d=-0.9

0.4

Quantiles of Input Sample

Quantiles of Input Sample

Quantiles of Input Sample

3

-2

0

2

4

-1
-4

-2

Standard Normal Quantiles

0

2

4

Standard Normal Quantiles

Figure 4: The normal Q-Q plots of rn from m = 1, 000 simulated processes with Cauchy innovations
and d = −0.1, −0.5, −0.9.
Figure 5 and Figure 6 show the histograms with kernel density fits and normal Q-Q plots
√
of nTn (hn ) from m = 1, 000 simulated processes with stable innovations (α = 0.5) and d =
√
−1.8, −2.5, −3.5. It is clear that the values of nTn (hn ) do not follow normal distribution in the
case d = −1.8 while they do follow normal distribution in the case d = −3.5. For d = −2.5, the
√
histogram and normal Q-Q plot indicate that the distribution of nTn (hn ) is not Gaussian but
√
is close to some normal distribution. This confirms the analysis that nTn (hn ) follows a normal
distribution when d < 1 − α2 even if 0 < α ≤ 1.

Histogram: alpha=0.5, d=-1.8

6

12

Histogram: alpha=0.5, d=-2.5

5

10

25

4

8

20

3

6

15

2

4

10

1

2

5

0

0
0.1

0

0.2

0.4

0.6

0.8

1

0.2

0.3

0.4

0.5

Histogram: alpha=0.5, d=-3.5

30

0.6

0
0.14

0.16

0.18

0.2

0.22

0.24

0.26

0.28

√
Figure 5: The histograms with kernel density fits of nTn (hn ) from m = 1, 000 simulated processes
with stable innovations (α = 0.5) and d = −1.8, −2.5, −3.5.

13

QQ Plot: alpha=0.5, d=-1.8

0.6

Quantiles of Input Sample

Quantiles of Input Sample

0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
-4

QQ Plot: alpha=0.5, d=-2.5

0.28

QQ Plot: alpha=0.5, d=-3.5

0.55

Quantiles of Input Sample

1
0.9

0.5
0.45
0.4
0.35
0.3
0.25

0.26
0.24
0.22
0.2
0.18

0.2
-2

0

2

Standard Normal Quantiles

4

0.15
-4

-2

0

2

Standard Normal Quantiles

4

0.16
-4

-2

0

2

4

Standard Normal Quantiles

√
Figure 6: The normal Q-Q plots of nTn (hn ) from m = 1, 000 simulated processes with stable
innovations (α = 0.5) and d = −1.8, −2.5, −3.5.
b g) to study
For the purpose of illustration, we apply the L22 kernel divergence estimator D(f,
the difference of annual average river flows among four rivers on the earth. The data sets consist of
annual average river flows (1,000 cubic meters per second) from Danube River at Orshava, Romania,
Gota River near Sjotop-Vannersburg, Sweden, Mississippi River near St. Louis, Missouri, USA and
Rhine River near Basle, Switzerland for 96 years between 1861 and 1956. The river flow data of
these rivers are well studied in ecology and statistics, e.g., Hipel and McLeod (1994). The original
data sets cover different time periods. We choose part of each data set from year 1861 to year 1956
in order to compare the flows in the same time period. The Kwiatkowski, Phillips, Schmidt, and
Shin (KPSS) test, the time series plots in Figure 7 and the autocorrelation plots in Figure 8 show
the stationarity and short memory property of the river flow level of each river.
We apply the estimator to calculate the divergence between the flow distributions of every two
rivers. As in the simulation study, we take the standard normal density function as the kernel
function. The right part of Figure 7 gives the kernel density estimations of river flows for each
river. Since they approximately follow some normal distributions, we let the bandwidth to be
hn = 96−0.4 ≈ 0.161. All the divergences are presented in Table 4. According to the density
curves in Figure 7, it is not a surprise that the biggest L22 divergence is between the Gota River
and the Rhine River. The density functions of these two rivers do not have much overlap although
they are very close in location. On the other hand, the density functions of Gota River and
Rhine River, in particular the density function of the Gota River, have much bigger values in
their supports than those of the other two rivers. The squared difference (f (x) − g(x))2 , not the
relative location between the supports of f (x) and g(x), makes contribution to the value of the
L22 divergence D(f, g). Mississippi River and Danube River have very similar density functions in
terms of supports and values. Mississippi River has the smallest maximum density and it has a
little bit overlap with Gota River or Rhine River. There is almost no overlap between the Danube
River and Gota River or Rhine River. This explains the order of the divergences in Table 4.

14

Danube River

Yearly flow

8

3.5

4

1870

1880

1890

1900

1910

1920

1930

1940

1950

1960
3

Gota River

0.8

Yearly flow

Danube
Gota
Mississippi
Rhine

6

2
1860

0.6
2.5
0.4
0.2
1860

1870

1880

1890

Yearly flow

1900

1910

1920

1930

1940

1950

1960

2

Mississippi River

10

1.5

5

0
1860

1870

1880

1890

1900

1910

1920

1930

1940

1950

1

1960

Rhine River

1.4

Yearly flow

Probability density curves of four river flows

4

1.2

0.5

1
0.8
0.6
1860

1870

1880

1890

1900

1910

1920

1930

1940

1950

0
-2

1960

Time (years)

0

2

4

6

8

10

12

Average annual river flow (1,000 cubic meters per second)

Figure 7: The time series plots and probability density functions of average annual river flow.

Sample ACF: Danube River

0.8
0.6
0.4
0.2
0
-0.2
-0.4

0

5

10

15

Sample ACF: Gota River

1

Sample Autocorrelation

Sample Autocorrelation

1

0.8
0.6
0.4
0.2
0
-0.2
-0.4

20

0

5

Lag
Sample ACF: Mississippi River

0.8
0.6
0.4
0.2
0
-0.2
-0.4

0

5

10

15

20

15

Sample ACF: Rhine River

1

Sample Autocorrelation

Sample Autocorrelation

1

10

Lag

0.8
0.6
0.4
0.2
0
-0.2
-0.4

20

Lag

0

5

10

15

20

Lag

Figure 8: The sample autocorrelation functions of average annual river flow.
15

Table 4: River flow divergences
The rivers
Gota River, Rhine River
Gota River, Danube River
Gota River, Mississippi River
Rhine River, Danube River
Rhine River, Mississippi River
Danube River, Mississippi River

6

Divergence
2.7797
2.1421
2.0725
1.6129
1.5433
0.0348

Extensions to multivariate linear processes

√
In this section, we denote norm functions in various linear spaces by | · |. For x ∈ Rd , |x| := x xT
where xT is the transposition of x. For a d × d deterministic real matrix a, |a| := max |a xT |.
x∈Rd ,|x|=1

In the sequel, we give extensions to the following multivariate linear process
Xn =

∞
X

ai εn−i ,

(22)

i=0

where the innovations εi are i.i.d. d × 1 random vectors in some probability space (Ω, F, P) and ai
∞
P
are d × d deterministic real matrices such that
ai εn−i converges in distribution. It is easy to
i=0

see that the multivariate linear process Xn in (22) exists and is well defined when εi are i.i.d. d × 1
random vectors in Lp (Rd ) for some p > 0 and ai are d × d deterministic real matrices satisfying
∞
P
|ai |2∧p < ∞. Let f be the probability density function of Xn . To estimate the quadratic
i=0

functional of f , we would use the estimator
Tn (hn ) =

1
n(n − 1) det(hn )

X


K h−1
n (Xi − Xj ) ,

1≤i6=j≤n

R
R
where the kernel K is a symmetric and bounded function with Rd K(u) du = 1 and Rd |u|2 |K(u)| du <
∞. The bandwidth sequence hn are diagonal matrices diag[hn1 , hn2 , · · · , hnd ] satisfying 0 < hni →
0 for all i = 1, 2, · · · , d.
We require the following assumptions on ai and εi .
Assumption
R 1 det(a0 ) 6= 0, det(am ) 6= 0 for some m ≥ 1 and ai = d × d zero matrix for all
i > m, and Rd |λ|2γ |φε (λ)|2 dλ < ∞ for some γ ∈ (0, 1].
Assumption 2 det(a0 ) 6= 0, det(ap ) 6= 0 and det(aq ) 6= 0 with p = min{i ∈ N : det(ai ) 6= 0} and
∞

P
q = min{i ∈ N : det(ai ) 6= 0 and i > p} < ∞,
|ai |γ < ∞, E |eιλε1 − φε (λ)|4 ≤ cγ,4 |λ|4γ ∧ 1
i=0
R
and Rd |λ|2γ |φε (λ)|2 dλ < ∞ for some γ ∈ (0, 1], ∃ two indices ir s.t. det(air +r − air ) 6= 0 for each
r ∈ {1, 2, · · · , q}.
Assumption 3 det(a0 ) 6= 0, det(ap ) 6= 0 with p = min{i ∈ N : det(ai ) 6= 0} < ∞, ∃ indice ir such
∞

P
that det(air − air +r ) 6= 0 for each r ∈ {1, · · · , p},
|ai |γ < ∞, E |eιλε1 − φε (λ)|2 ≤ cγ,2 |λ|2γ ∧ 1
i=0
R
and Rd |λ|2γ |φε (λ)|2 dλ < ∞ for some γ ∈ (0, 1].
16

Using similar arguments as in the proof of Theorems 2.1 and 2.2 with some proper modifications,
we can get the following results.
Theorem 6.1 Under the assumption 1 or 2, we further assume that f is bounded. Then there
exist positive constants c1 and c2 such that


Z
1
2
2γ
f (x) dx ≤ c1
+ det(hn )
,
(23)
E Tn (hn ) −
n
Rd
n

E Tn (hn ) − E Tn (hn ) −

1X
Yi
n

2

≤ c2



i=1

ηn,γ
1
det(hn )2γ 
+
+
n2 det(hn )
n2
n

(24)

and, if additionally n det(hn ) → ∞ as n → ∞,
i
√ h
L
n Tn (hn ) − E Tn (hn ) −→ N (0, 4σ 2 ),
where Yi = 2 f (Xi ) −

R

Rd

(25)

n P
∞

P
f 2 (x) dx , ηn,γ =
|ai |γ and σ 2 = lim n−1 Var (Sn ) with
n→∞

`=0 i=`

Sn =

n 
X

Z
f (Xi ) −

2



f (x) dx .
Rd

i=1

Theorem 6.2 Under the assumption 3, we further assume that f is bounded and
∞. Then there exist positive constants c3 and c4 such that


Z
1
2
2γ
+ det(hn )
,
E Tn (hn ) −
f (x) dx ≤ c3
n
Rd

R

Rd

b
|K(λ)|
dλ <

(26)

n


1X
1
det(hn )γ 
E Tn (hn ) − E Tn (hn ) −
Yi ≤ c4
+ √
n
n det(hn )
n

(27)

i=1

and, if additionally

√

n det(hn ) → ∞ as n → ∞,
i
√ h
L
n Tn (hn ) − E Tn (hn ) −→ N (0, 4σ 2 )

(28)

for some σ 2 ∈ (0, ∞).

Remark
6.1 In the i.i.d case, that is, det(a0 ) 6= 0 and ai = d × d zero matrix for all i ≥ 1, and
R
2γ |φ (λ)|2 dλ < ∞ for some γ ∈ (0, 1], then results in Theorem 6.1 still hold with the right
|λ|
ε
Rd
hand side of (23) replaced by a constant multiple of det(hn )2γ and ηn,γ in (24) by 0, respectively.

17

7

Proofs

In this section, the proofs of our main results, Theorems 2.1 and 2.2, are given. First we provide
two useful lemmas based on the projection method.
For each i ∈ Z, let Fi be the σ-field generated by {εk : k ≤ i}. Given an integrable complexvalued random variable Z, we define the following projection operator Pi as
Pi Z = E [Z|Fi ] − E [Z|Fi−1 ]

(29)

for each i ∈ Z. It is easy to see that
E [Pi Z Pj W ] = 0

(30)

for any two integrable complex-valued random variables Z and W if i 6= j.
The estimation in the following lemma is useful in deriving the order of the bias of the kernel
estimator (2).
Lemma 7.1 Suppose

∞
P


|ai |γ < ∞ and E |eιλε1 − φε (λ)|2 ≤ cγ,2 |λ|2γ ∧ 1 for some γ ∈ (0, 1].

i=0

Then there exists a positive constant c such that
i
h
X
E H(Xi )(λ)H(Xj )(λ)
1≤i6=j≤n
p Y
∞
h
i
X
≤ c n |φε (λa0 )| |λ|2γ |φε (λa0 )| + |λ|γ |φε (λap )| +
|φε (λ(a` − a`+r ))| ,
r=1 `=1

where p = min{i ∈ N : ai 6= 0} < ∞. Moreover, the right hand side of the above inequality is equal
to 0 if ai = 0 for all i ∈ N.
Proof. It suffices to show the case when ai 6= 0 for some i ∈ N. Recall the definition of projection
operator Pk in (29). Applying telescoping, (30) and the triangle inequality, we have
X

h
i
E H(Xi )(λ)H(Xj )(λ) ≤ 2

X

i
X

i
h
E Pk H(Xi )(λ)Pk H(Xj )(λ)

1≤i<j≤n k=−∞

1≤i6=j≤n

=2

X

i
X

i
h
E P0 H(Xi−k )(λ)P0 H(Xj−k )(λ) .

1≤i<j≤n k=−∞

Note that
∞
j−k−1
i−k−1
i
h ιP
i
Y
λ(a`+i−k −a`+j−k )ε−` ih Y
E P0 H(Xi−k )(λ)P0 H(Xj−k )(λ) = E e `=1
|φε (λa` )|2
φε (−λa` )

h

`=0

`=i−k

h

i
× E (eιλai−k ε0 − φε (λai−k ))(e−ιλaj−k ε0 − φε (−λaj−k )) .
Pn−1 Pn
Pi
Next, we decompose the triple sum i=1
j=i+1
k=−∞ into the sum with k < i, the sum with
k = i and i + 2 ≤ j ≤ n and the sum with k = i and j = i + 1. By Cauchy-Schwartz inequality, we

18

have
h
i
E H(Xi )(λ)H(Xj )(λ)

X
1≤i6=j≤n

≤2

n−1
X

n
X

i−1
X

|φε (λa0 )|2

p

1 − |φε (λai−k )|2

q
1 − |φε (λaj−k )|2

i=1 j=i+1 k=−∞

+2

n−1
X

j−i−1
Y

n
X

|φε (λa` )|

i=1 j=i+2 `=0

+2

n−1
X

∞
Y

q
p
|φε (λ(a` − a`+j−i ))| 1 − |φε (λa0 )|2 1 − |φε (λaj−i )|2

`=1

|φε (λa0 )|

i=1

∞
Y


|φε (λ(a` − a`+1 ))| .

`=1

Note that the second term on the right hand side of the above inequality is less than
2

n−1
X

n
X

|φε (λa0 )||φε (λap )|

q
1 − |φε (λaj−i )|2

i=1 j=i+1+p

+2

i+p
n−1
X X

|φε (λa0 )|

i=1 j=i+2

∞
Y

q
|φε (λ(a` − a`+j−i ))| 1 − |φε (λaj−i )|2 .

`=1

Therefore, using the conditions of the lemma,
X

n−1
n
h
i
X X
2γ
E H(Xi )(λ)H(Xj )(λ) ≤ c1 |λ|

i−1
X

|φε (λa0 )|2 |ai−k |γ |aj−k |γ

i=1 j=i+1 k=−∞

1≤i6=j≤n

γ

+ c1 |λ|

n−1
X

n
X

|φε (λa0 )||φε (λap )||aj−i |γ

i=1 j=i+1+p

+ c1

n−1
X

|φε (λa0 )|

p Y
∞
X

|φε (λ(a` − a`+r ))|.

r=1 `=1

i=1

This gives the desired result.
Remark 7.1 Note that in Lemma 7.1, if the linear process {Xn } is m-dependent, then am 6=
p Q
∞
P
0, m ≥ p and
|φε (λ(a` − a`+r ))| ≤ p|φε (λam )|. If lim ai = 0, a0 6= 0 and ai 6= 0 for
i→∞

r=1 `=1

infinite many i ∈ N, then for each r ∈ {1, 2, · · · , p}, qr = min{i ∈ N : ai − ai+r 6= 0} < ∞ and
p Q
p
∞
P
P
|φε (λ(a` − a`+r ))| ≤
|φε (λ(aqr − aqr +r ))|.
r=1 `=1

r=1

The next lemma is needed in the proof of Theorem 2.2.

Lemma 7.2 For any m ∈ N and γ ∈ (0, 1], if E |eιλε1 − φε (λ)|2 ≤ cγ,2 |λ|2γ ∧ 1 , then there exists
a positive constant c such that
h

E φn (λ) − φ(λ)

2

i

∞
i2
X
Y
 γ m−1
ch
γ
≤
m+2
|ai | |λ|
|φε (λaj )| .
n
i=m

j=0

Proof. Obviously, E H(X1 ) = 0, E |H(X1 )|2 < ∞ and
n(φn (λ) − φ(λ)) =

n h
X

n
i X
eιλXi − E eιλXi =
H(Xi ).

i=1

i=1

19

Note that
P0 H(Xi ) = E [H(Xi )|F0 ] − E [H(Xi )|F−1 ]
for each i ∈ N. Then,
P0 H(Xi ) = E [eιλXi |F0 ] − E [eιλXi |F−1 ]
ι

=e

∞

i−1

λaj εi−j

j=i

ι

=e

∞
P

∞
P

i

P
P
h ι P λaj εi−j i
ι
λaj εi−j h ι
λaj εi−j i
E e j=0
− e j=i+1
E e j=0
i−1

λaj εi−j

j=i+1

h ι P λaj εi−j ih
i
E e j=0
eιλai ε0 − E eιλai ε0

:= I ∗ II ∗ III,
ι

where I = e

∞
P

i−1

λaj εi−j

j=i+1

h ι P λaj εi−j i i−1
Q
, II = E e j=0
=
φε (λaj ) and
j=0

III = eιλai ε0 − E [eιλai ε0 ] = eιλai ε0 − φε (λai ).
Since I and III are independent,
i
h
kP0 H(Xi )k22 = E P0 H(Xi )P0 H(Xi )
= [II ∗ II]E [I ∗ Ī]E [III ∗ III]
=

i−1
Y

|φε (λaj )|2 [1 − |φε (λai )|2 ]

j=0

for i ≥ 1 and kP0 H(X0 )k22 = 1 − |φε (λa0 )|2 . Consequently,
∞
X

kP0 H(Xi )k2 =

∞ Y
i−1
X
p
p
1 − |φε (λa0 )|2 +
|φε (λaj )| 1 − |φε (λai )|2

i=0

i=1 j=0

≤m+

∞ Y
i−1
X

p
|φε (λaj )| 1 − |φε (λai )|2

i=m j=0

≤m+

m−1
Y

∞ p
X
|φε (λaj )|
1 − |φε (λai )|2
i=m

j=0

≤ m + cγ



∞
X

γ

|ai |



|λ|

i=m

γ

m−1
Y

|φε (λaj )|.

j=0

This completes the proof.
Now we give the proof of Theorem 2.1.
Proof of Theorem 2.1. The proof will be done in several steps.
Step 1. We give the estimation for
Z
E [Tn (hn )] −
R

20

f 2 (x) dx .

By Fourier inverse transform, Tn (hn ) can be written as
X Z
1
−ιλ(Xi −Xj )
b
K(λh
dλ.
Tn (hn ) =
n )e
πn(n − 1)
R

(31)

1≤i<j≤n

Together with the Plancherel theorem, we have
Z
E [Tn (hn )] −
f 2 (x) dx
R
h
i
X Z
1
ιλXi
b
=
K(λh
− φ(λ))(e−ιλXj − φ(−λ)) dλ
n ) E (e
2πn(n − 1)
1≤i6=j≤n R
Z 

1
2
b
b
K(λh
+
n ) − K(0) |φ(λ)| dλ.
2π R

(32)

2β for any β ∈ [0, 1],
b
b
Using Lemma 7.1 and the inequality |K(λh
n ) − K(0)| ≤ cβ |λhn |
Z
Z
Z
c1
2
2γ
2
2γ
E [Tn (hn )] −
f (x) dx ≤
(1 + |λ| )|φε (λ)| dλ + c1 hn
|λ|2γ |φ(λ)|2 dλ
n R
R
R


1
2γ
≤ c2
+ hn .
n

In the case that {Xi }ni=1 are i.i.d. random variables, the term (32) equals zero and hence E [Tn (hn )]−
R 2
2γ
R f (x) dx ≤ c2 hn . This is consistent with the result using convolution method as in Giné and
Nickl (2008).
Step 2. We give the decomposition for
Tn (hn ) − E Tn (hn ).
Using (31), we can obtain the following decomposition
Tn (hn ) − E Tn (hn ) = 2Nn + Dn − E [Dn ],

(33)

where
n

1 X
Nn =
2πn

Z


ιλXi
b
− φ(λ) φ(−λ) dλ
K(λh
n) e

R

i=1

and
1
Dn =
2πn(n − 1)

X

Z



ιλXi
b
K(λh
− φ(λ) e−ιλXj − φ(−λ) dλ.
n) e

1≤i6=j≤n R

Step 3. Here we estimate E |Dn |2 .
We first assume that {Xn } is m-dependent, that is, a0 6= 0, am 6= 0 and ai = 0 for all i > m.
In this case,
X Z
2
 −ιλX

1
2
ιλXi
j
b
E |Dn | ≤ 2
E
K(λh
)
e
−
φ(λ)
e
−
φ(−λ)
dλ
n
n (n − 1)2
j−i>4m R
X Z
2


1
ιλXi
b
+ 2
E
K(λh
− φ(λ) e−ιλXj − φ(−λ) dλ .
(34)
n) e
2
n (n − 1)
R
0<j−i≤4m

21

For the first expectation on the right hand side of (34),
X Z
2


ιλXi
b
E
K(λh
− φ(λ) e−ιλXj − φ(−λ) dλ
n) e
j−i>4m R

Z
≤
R2

X

b 1 hn )||K(λ
b 2 hn )|
|K(λ



E (eιλ1 Xi1 − φ(λ1 ))(e−ιλ2 Xi2 − φ(−λ2 ))

|i1 −i2 |≤m

X

×


E (e−ιλ1 Xj1 − φ(−λ1 ))(eιλ2 Xj2 − φ(λ2 )) dλ1 dλ2


|j1 −j2 |≤m

Z
≤
R2

b 1 hn )||K(λ
b 2 hn )|
|K(λ

h

X


i2
|E eιλ1 Xi1 −ιλ2 Xi2 | + |φ(λ1 )φ(−λ2 )| dλ1 dλ2

|i1 −i2 |≤m

Z

h
i
b 1 hn )||K(λ
b 2 hn )| |φ(λ1 − λ2 )|2 + |φε (λ1 a0 )φε (λ2 am )|2 + |φ(λ1 )φ(λ2 )|2 dλ1 dλ2
|K(λ
2
Z
ZR
2
2
2
2
b 2 hn )|2 |φ(λ1 − λ2 )|2 dλ1 dλ2
b
|K(λ
|K(λ1 hn )| |φ(λ1 − λ2 )| dλ1 dλ2 + c4 n
≤ c4 n
2
2
R Z
ZR
2
2
2
+ c4 n
|φε (λ1 a0 )φε (λ2 am )| dλ1 dλ2 + c4 n
|φ(λ1 )φ(λ2 )|2 dλ1 dλ2
≤ c3 n2

R2

≤ c5

n2
hn

R2

,

(35)

where in the last inequality we used the Plancherel theorem.
Set Kn (x) = h1n K( hxn ). Then
X Z


ιλXi
b
E
− φ(λ) e−ιλXj − φ(−λ) dλ
K(λh
n) e

2

0<j−i≤4m R

≤ c6 n2
≤ c7 n

2

4m
X
k=1
4m
X

Z



ιλX1
b
K(λh
− φ(λ) e−ιλX1+k − φ(−λ) dλ
n) e

E
R

E Kn2 (X1

2

Z

2

ιλX1
b
K(λh
φ(−λ) dλ
n) e

− X1+k ) + c7 n E
R

k=1

+ c7 n2

2

4m
X
k=1

Z

Z

2

−ιλX1+k
b
K(λh
φ(λ) dλ + c7 n2
n) e

E
R

2

2
b
K(λh
n )|φ(λ)| dλ .

R

Hence
E

Z

X



ιλXi
b
K(λh
− φ(λ) e−ιλXj − φ(−λ) dλ
n) e

2

0<j−i≤4m R

≤ c8 n

2

Z

c2 (λ)||φε (λa0 )||φε (λam )| dλ + c8 n2
|K
n
R
Z
2
2
+ c8 n
|φ(λ)|2 dλ

Z

2
b
|K(λh
n )| dλ

Z

R

|φ(λ)|2 dλ

R

R

n2
≤ c9
,
hn

(36)

c2 (λ)| ≤
where in the last inequality we used the fact that |K
n
Combining (34), (35) and (36) gives
E |Dn |2 ≤

22

c10
.
n 2 hn

1
hn

R

RK

2 (x) dx

for all λ ∈ R.

In the sequel, we can assume that a0 6= 0 and ai 6= 0 for infinite many i ∈ N. Let p1 = min{i ∈
N : ai 6= 0} and p2 = min{i ∈ N : ai 6= 0, i > p1 }. Note that
kDn k2 ≤ I0 + I1 + · · · + Ip1 + Ip1 +1 ,

(37)

where
Z

1
Ip =
2πn(n − 1)

X

X

1≤i6=j≤n

k≤i, `≤j
|k−i|+|`−j|=p

b
K(λh
n)
R



Pk H(Xi )(λ)P` H(Xj )(−λ) dλ ,
2

for 0 ≤ p ≤ p1 and
Ip1 +1

Z

1
=
2πn(n − 1)

X

X

1≤i6=j≤n

k≤i, `≤j
|k−i|+|`−j|≥p1 +1

b
K(λh
n)
R

We first estimate I0 . Note that
Z
1
b
I0 =
K(λh
n)
2πn(n − 1) R

X





Pk H(Xi )(λ)P` H(Xj )(−λ) dλ .
2


Pi H(Xi )(λ)Pj H(Xj )(−λ) dλ ,
2

1≤i6=j≤n

and
I02

Z

c11
≤ 4
n

b
K(λh
n)
R

+

X 
j−i=1

Z

c11
n4


Pi H(Xi )(λ)Pj H(Xj )(−λ) dλ

b
K(λh
n)
R

X 

2
2


Pi H(Xi )(λ)Pj H(Xj )(−λ) dλ

j−i=2

2
2

Z

+ ··· +
+

c11
n4

X 

c11
b
K(λh
Pi H(Xi )(λ)Pj H(Xj )(−λ) dλ
n)
4
n
R
j−i=p2
Z
X 
2

b
K(λh
Pi H(Xi )(λ)Pj H(Xj )(−λ) dλ
n)
R

2
2

2

j−i≥p2 +1

:= I0,1 + I0,2 + · · · + I0,p2 + I0,p2 +1 .
Recall properties of the projection operator Pk defined in (29). It is easy to see that, for
1 ≤ q ≤ p2 ,
I0,q ≤
=

c12
n3

Z

c12
n3

Z

R2

R2

b 1 hn )||K(λ
b 2 hn )|
|K(λ

∞
Y

E [eι(λ2 −λ1 )(ak+q −ak )εj−k−q ] dλ1 dλ2

k=1

b 1 hn )||K(λ
b 2 hn )|
|K(λ

∞
Y

|φε ((λ2 − λ1 )(ak+q − ak ))| dλ1 dλ2 .

k=1

Since lim ai = 0 and ai 6= 0 for infinite many i ∈ N, there exist infinite many indices k such that
i→∞

ak+q 6= ak . Doing some simple calculations then gives
I0,q ≤

c13
,
n3 hn

23

for 1 ≤ q ≤ p2 . To estimate the last term in the upper bound of I02 , observe that
Z
c11
b 1 hn )K(−λ
b
I0,p2 +1 = 4
K(λ
2 hn )
n R2
X
E [Pi1 H(Xi1 )(λ1 )Pi2 H(Xi2 )(−λ2 )Pj H(Xj )(−λ1 )Pj H(Xj )(λ2 )] dλ1 dλ2
j−i1 ≥p2 +1, j−i2 ≥p2 +1

c11
= 4
n

Z

Z
c11
b
b
b 1 hn )K(λ
b 2 hn )I2 (λ1 , λ2 ) dλ1 dλ2
K(λ1 hn )K(−λ2 hn )I1 (λ1 , λ2 ) dλ1 dλ2 + 4
K(λ
n R2
R2
Z
c11
b 1 hn )K(−λ
b
+ 4
K(λ
2 hn )I3 (λ1 , λ2 ) dλ1 dλ2 ,
n R2

where
X

I1 (λ1 , λ2 ) =

E [Pi1 H(Xi1 )(λ1 )Pi2 H(Xi2 )(−λ2 )Pj H(Xj )(−λ1 )Pj H(Xj )(λ2 )] ,

j−i1 ≥p2 +1, j−i2 ≥p2 +1, i2 >i1

X

I2 (λ1 , λ2 ) =

E [Pi1 H(Xi1 )(λ1 )Pi2 H(Xi2 )(−λ2 )Pj H(Xj )(−λ1 )Pj H(Xj )(λ2 )] ,

j−i1 ≥p2 +1, j−i2 ≥p2 +1, i1 <i2

X

I3 (λ1 , λ2 ) =

E [Pi1 H(Xi1 )(λ1 )Pi2 H(Xi2 )(−λ2 )Pj H(Xj )(−λ1 )Pj H(Xj )(λ2 )] .

j−i1 ≥p2 +1, j−i2 ≥p2 +1, i2 =i1

Note that
X

|I1 (λ1 , λ2 )| ≤

|φε (ap1 (λ1 − λ2 ))| |φε (ap2 (λ1 − λ2 ))|

j−i1 ≥p2 +1, j−i2 ≥p2 +1, i2 >i1

i
h
× E (e−ιλ2 a0 εi2 − φε (−λ2 a0 ))(e−ι(λ1 −λ2 )aj−i2 εi2 − φε (−(λ1 − λ2 )aj−i2 )
X
|φε ((λ1 − λ2 )ap1 )| |φε ((λ1 − λ2 )ap2 )|

≤

j−i1 ≥p2 +1, j−i2 ≥p2 +1, i2 >i1

× e−ι(λ1 −λ2 )aj−i2 εi2 − φε (−(λ1 − λ2 )aj−i2 )
2
X
|φε ((λ1 − λ2 )ap1 )| |φε ((λ1 − λ2 )ap2 )| |λ1 − λ2 |γ |aj−i2 |γ

≤ c16
≤

j−i1 ≥p2 +1, j−i2 ≥p2 +1, i2 >i1
c17 n2 |λ1 − λ2 |γ |φε ((λ1 − λ2 )ap1 )| |φε ((λ1

− λ2 )ap2 )| .

Similarly,
|I2 (λ1 , λ2 )| ≤ c18 n2 |λ1 − λ2 |γ |φε ((λ1 − λ2 )ap1 )||φε ((λ1 − λ2 )ap2 )|.
Finally,
|I3 (λ1 , λ2 )| ≤
≤

X

E [Pi H(Xi )(λ1 )Pi H(Xi )(−λ2 )Pj H(Xj )(−λ1 )Pj H(Xj )(λ2 )]

j−i≥p2 +1
c19 n2 |φε ((λ1

− λ2 )ap1 )||φε ((λ1 − λ2 )ap2 )|.

So
I0,p2 +1

Z
c20
b 1 hn )||K(λ
b 2 hn )|(1 + |λ1 − λ2 |γ )|φε ((λ1 − λ2 )ap )||φε ((λ1 − λ2 )ap )| dλ1 dλ2
≤ 2
|K(λ
1
2
n R2
c21
≤ 2 .
n hn

24

Therefore,
I02 ≤ I0,1 + I0,2 + · · · + I0,p2 + I0,p2 +1 ≤

c22
.
n2 hn

(38)

Using similar arguments, for 1 ≤ p ≤ p1 , we can obtain
Ip2 ≤

c23
.
n 2 hn

(39)

We now estimate Ip1 +1 . Note that
1
(2π)2 n2 (n − 1)2

Ip21 +1 =

Z
R2

b 1 hn )K(−λ
b
K(λ
2 hn ) Jn (λ1 , λ2 ) dλ1 dλ2 ,

where
Jn (λ1 , λ2 ) =

i
h
E Pk1 H(Xi1 )(λ1 )Pk2 H(Xj1 )(−λ1 )Pk3 H(Xi2 )(−λ2 )Pk4 H(Xj2 )(λ2 ) .

X
i1 6=j1 , i2 6=j2
k1 ≤i1 , k2 ≤j1 , k3 ≤i2 , k4 ≤j2
|k1 −i1 |+|k2 −j1 |≥p1 +1
|k3 −i2 |+|k4 −j2 |≥p1 +1

Recall properties of the projection operator Pk defined in (29). By Cauchy-Schartz inequality and
the condition (3), we can obtain



|Jn (λ1 , λ2 )| ≤ c24 |λ1 |2γ |λ2 |2γ |φε (λ1 a0 )|2 + |φε (λ1 ap1 )|2 |φε (λ2 a0 )|2 + |φε (λ2 ap1 )|2
X
|ai1 −k1 |γ |aj1 −k2 |γ |ai2 −k3 |γ |aj2 −k4 |γ . (40)
×
i1 6=j1 , i2 6=j2
k1 ≤i1 , k2 ≤j1 , k3 ≤i2 k4 ≤j2
|k1 −i1 |+|k2 −j1 |≥p1 +1, |k3 −i2 |+|k4 −j2 |≥p1 +1
at least two k indices are identical,
identical k indices are greater than the others

There are only four possibilities for the k indices in the above summation: 1) all the k indices
are identical; 2) three k indices are identical and strictly larger than the last one; 3) two pairs of
different identical k indices; 4) one pair of identical k indices and strictly larger than the remaining
two different indices. For the first three possibilities, the summation in (40) is less than a constant
∞
P
multiple of n2 since
|ai |γ < ∞. For the last possibility, the summation in (40) is less than a
i=0

constant multiple of n2 ηn,γ where ηn,γ =

n P
∞
P

|ai |γ . Therefore,

`=0 i=`

Ip21 +1 ≤

c25 ηn,γ
n2


 2
2γ
2
2
b
|K(λh
)|
|λ|
|φ
(λa
)|
+
|φ
(λa
)|
dλ .
n
ε
0
ε
p1

Z
R

Putting inequalities (37), (38), (39) and (41) together gives


ηn,γ
1
2
+ 2 .
kDn k2 ≤ c26
n2 hn
n
Step 4. Here we estimate


E |Nn − N n |2 ,
where
Nn =

1
2π

Z


b
K(0)
φn (λ) − φ(λ) φ(−λ) dλ.

R

25

(41)

Note that




E |Nn − N n |2 = E

1
2π

Z



b
b
K(λh
n ) − K(0) φn (λ) − φ(λ) φ(−λ) dλ
R

 Z
2
 ιλX1

b
b
K(λhn ) − K(0) e
φ(−λ) dλ
E

1
(2π)2 n
R
i2
1 h
= E Kn ∗ f (X1 ) − f (X1 ) ,
n

≤

where Kn (x) = h1n K( hxn ).
Since f is bounded,
2



E |Nn − N n |



Z

2
c27
Kn ∗ f (x) − f (x) dx
≤
n R
Z
c27
2
2
b
b
=
|K(λh
n ) − K(0)| |φ(λ)| dλ
(2π)2 n R
Z
c27
2
2
b
b
|K(λh
≤
n ) − K(0)| |φε (a0 λ)| dλ
(2π)2 n R
c28 h2γ
n
≤
,
n

where in the first equality we used the Plancherel theorem.
It is easy to see that

Z
n 
1X
2
Nn =
f (Xi ) −
f (x) dx .
n
R
i=1

Combining all the above results gives
n

1X
E Tn (hn ) − E Tn (hn ) −
Yi
n

2

≤ c29

i=1

ηn,γ
h2γ
1
n
+
+
2
2
n hn
n
n

!
.

Step 5. We show the central limit theorem for
n

1 X
√
f (Xi ) −
n
i=1

Z


f 2 (x) dx .

R

It suffices to consider the case when the Xi are dependent. Note that
Z


P0 f (Xi ) −
f 2 (x) dx
R

= E [f (Xi )|F0 ] − E [f (Xi )|F−1 ]
i−1
∞
P
P

Z
i
λaj εi−j +ι
λaj εi−j h
ι
1
ιλai ε0
ιλai ε00
j=i+1
j=0
=
E
φ(λ) e
e
−e
dλ F0 ,
2π
R
where {ε0i : i ∈ Z} is an independent copy of {εi : i ∈ Z}.

26

2



Then

P0 f (Xi ) −

Z

2

f (x) dx
R

ι

Z


2

≤

φ(λ) e
ZR

≤

∞
P

λaj εi−j +ι

j=i+1

|φ(λ)||eιλai ε0 − e

i−1
P
j=0

ιλai ε00

λaj εi−j 

| dλ

R
Z

0 
eιλai ε0 − eιλai ε0 dλ

2

|φ(λ)||eιλai ε0 − φε (λai )| dλ
RZ
|λ|γ |φ(λ)| dλ
≤ 2|ai |γ
RZ
|λ|γ |φε (λ)|2 dλ.
≤ c30 |ai |γ
≤2

2

2

R

Now by Lemma 1 in Wu (2006), we can easily obtain
n

1 X
√
f (Xi ) −
n
i=1

Z


d
f 2 (x) dx −→ N (0, σ 2 )

R

for some σ 2 ∈ (0, ∞). Using similar arguments as in Step 3, we can show that
sup
n

where Sn =
Hence,

n
P

i=1
σ2 =

f (Xi ) −

R

Rf

2 (x) dx



1
E |Sn |4 < ∞,
n2

.

lim n−1 Var (Sn ). This completes the proof.

n→∞

Proof of Theorem 2.2. The proof of this theorem is similar to that of Theorem 2.1. We only
need to modify the Step 3 in the proof of Theorem 2.1 and consider the dependent case. Using
the same notations as in the proof of Theorem 2.1 and then Lemma 7.2,
Z


√
√
2
b
n E |Dn | ≤ n
|K(λh
dλ
n )|E |φn (λ) − φ(λ)|
R
Z
i
h
1
ιλX1
ιλX1 2
b
+√
|K(λh
)|E
|e
−
E
e
|
dλ
n
n R
Z
Z
c1
c1
b
≤√
|K(λ)|
dλ + √
|λ|2γ |φε (λa0 )|2 dλ.
nhn R
n R
This completes the proof.
At the end, we give two useful lemmas which are related to the sufficient and necessary conditions for the assumptions in Theorems 2.1 and 2.2.
Lemma 7.3 Let φε (λ) = E [eιλε1 ] for all λ ∈ R. For any γ ∈ [0, 1], if E |ε1 |2γ < ∞, then there
exists a positive constant cγ,2 such that
E |eιλε1 − φε (λ)|2 ≤ cγ,2 |λ|2γ .
Moreover, if E |ε1 |4γ < ∞, there exists a positive constant cγ,4 such that
E |eιλε1 − φε (λ)|4 ≤ cγ,4 |λ|4γ .
27

Proof. This first inequality follows from
E |eιλε1 − φε (λ)|2 = 1 − |φε (λ)|2 ≤ 1 − [E cos(λε1 )]2 ≤ c1 |λ|2γ .
For the second inequality,
E |eιλε1 − φε (λ)|4

2
= E 1 − eιλε1 φε (−λ) − e−ιλε1 φε (λ) + |φε (λ)|2
= 1 + φε (2λ)φ2ε (−λ) + φε (−2λ)φ2ε (λ) − 3|φε (λ)|4


= 1 + 2E cos(2λε1 ) E 2 cos(λε1 ) − E 2 sin(λε1 ) + 4E sin(2λε1 )E cos(λε1 )E sin(λε1 )

2
− 3 E 2 cos(λε1 ) + E 2 sin(λε1 )



λε1 2
= 1 + 2 1 − 2E sin2 (λε1 ) (1 − 2E sin2 (
)) − E 2 sin(λε1 )
2


λε
λε1
1
+ 8E sin(λε1 )(1 − 2 sin2 (
)) E sin(λε1 )[1 − 2E sin2 (
)]
2
2

2
λε1 2
− 3 (1 − 2E sin2 (
)) + E 2 sin(λε1 )
2
4γ
≤ c2 |λ| .
This completes the proof.
Lemma 7.4 Let φε (λ) = E [eιλε1 ] for all λ ∈ R. If ε1 has non-degenerate distribution, i.e., it is
not equal to a constant almost surely, then in the inequalities
E |eιλε1 − φε (λ)|2 ≤ cγ,2 (|λ|2γ ∧ 1)
and
E |eιλε1 − φε (λ)|4 ≤ cγ,4 (|λ|4γ ∧ 1),
the range γ ∈ (0, 1] is optimal.
Proof. By Cauchy-Schwarz inequality, we only need to show that the range γ ∈ (0, 1] is optimal
for the first inequality. Suppose E |eιλε1 −φε (λ)|2 = 1−|φε (λ)|2 ≤ c(|λ|2γ ∧1) for some γ > 1. Since
ε2 is an independent copy of ε1 , let φε1 −ε2 (λ) = E [eιλ(ε1 −ε2 ) ], then 1 − φε1 −ε2 (λ) = 1 − |φε (λ)|2 ≤
c(|λ|2γ ∧ 1) for some γ > 1.
Note that
E |eιλ(ε1 −ε2 ) − 1|2 = 1 − φε1 −ε2 (λ) + 1 − φε1 −ε2 (−λ) = 2(1 − φε1 −ε2 (λ)).
Therefore, for λ close enough to 0,

1
2
|φε1 −ε2 (x + λ) − φε1 −ε2 (x)| ≤ E |eιλε1 −ε2 − 1| ≤ E |eιλε1 −ε2 − 1|2
≤ c1 |λ|γ .
Since γ > 1, φ0ε1 −ε2 (x) = 0 for all x ∈ R. That is, |φε (x)|2 = φε1 −ε2 (x) = 1 for all x ∈ R or ε1
equals a constant almost surely. So the γ must be less than or equal to 1.

Acknowledgement
We would like to thank the two referees and the Associate Editor for their valuable comments.
We would also like to thank Professor Xia Chen for very helpful discussions. F. Xu is partially
supported by National Natural Science Foundation of China (Grant No.11401215), Shanghai Pujiang Program (14PJ1403300), 111 Project (B14019) and Natural Science Foundation of Shanghai
(16ZR1409700).

28

References
[1] Ahmad IA. 1979. Strong consistency of density estimation by orthogonal series methods for
dependent variables with applications. Ann. Inst. Statist. Math. 31: 279–288.
[2] Bailey WN. 1935. General Hypergeometric Series. Cambridge Tracts in Mathematics and
Mathematical Physics.
[3] Bickel JP and Ritov Y. 1988. Estimating integrated squared density derivatives: Sharp best
order of convergence estimates. Sankhyā Ser. A 50: 381–393.
[4] Bondon P and Palma W. 2007. A class of antipersistent processes. J. Time Series Anal. 28:
261-273.
[5] Devroye L and Györfi L. 1985. Nonparametric Density Estimation: The L1 View. New York:
Wiley.
[6] Faÿ G, Moulines E, Roueff F and Taqqu MS. 2009. Estimators of long-memory: Fourier versus
wavelets. J. Econometrics 151: 159-177.
[7] Gauss F. 1866. Disquisitiones generales circa sericm infinitam, Ges. Werke 3: 123-163 and
207-229.
[8] Giné E and Nickl R. 2008. A simple adaptive estimator of the integrated square of a density.
Bernoulli 14: 47–61.
[9] Gretton A, Borgwardt KM, Rasch MJ, Schölkopf B and Smola A. 2012. A kernel two-sample
test. J. Mach. Learn. Res. 13: 723–773.
[10] Hall P and Marron JS. 1987. Estimation of integrated squared density derivatives. Statist.
Probab. Lett. 6: 109–115.
[11] Hipel KW and McLeod AI. 1994. Time Series Modelling of Water Resources and Environmental Systems. Elsevier.
[12] Honda T. 2000. Nonparametric density estimation for a long-range dependent linear process.
Ann. Inst. Statist. Math. 52: 599–611.
[13] Hsing T. 1999. On the asymptotic distributions of partial sums of functionals of infinitevariance moving averages. Ann. Probab. 27: 1579–1599.
[14] Källberg D, Leonenko N and Seleznjev O. 2012. Statistical inference for Rényi entropy functionals. Lecture Notes in Comput. Sci. 7260: 36–51.
[15] Källberg D, Leonenko N and Seleznjev O. 2014. Statistical estimation of quadratic Rényi
entropy for a stationary m-dependent sequence. Journal of Nonparametric Statistics 26: 385–
411.
[16] Krishnamurthy A, Kandasamy K, Poczós B and Wasserman L. 2015. On estimating L22 divergence. Proceedings of the 18th International Conference on Artificial Intelligence and Statistics
(AISTATS), San Diego, CA, USA. JMLR: W&CP 38.
[17] Laurent B. 1996. Efficient estimation of integral functionals of a density. Ann. Statist. 24:
659–681.
[18] Laurent B. 1997. Estimation of integral functionals of a density and its derivatives. Bernoulli
3: 181–211.
29

[19] Leonenko N, Pronzato L and Savani V. 2008. A class of Rényi information estimators for
multidimensional densities. Ann. Statist. 36: 2153–2182. Corrections, Ann. Statist. 38 (2010),
3837–3838.
[20] Leonenko N and Seleznjev O. 2010. Statistical inference for the ε-entropy and the quadratic
Rényi entropy. J. Multivariate Anal. 101: 1981–1994.
[21] Mittnik S, Rachev ST and Paolella MS. 1998. Stable paretian modeling in finance: Some
empirical and theoretical aspects. In : Adler, R. J., Feldman, R.E., Taqqu, M.S., eds. A
Practical Guide to Heavy Tails. Birkhäuser: 79–110.
[22] Nadaraya EA. 1989. Nonparametric Estimation of Probability Densities and Regression
Curves. Kluwer Academic Pub.
[23] Parzen E. 1962. On estimation of a probability density and mode. Ann. Math. Stat. 31: 1065–
1079.
[24] Penrose M and Yukich JE. 2013. Limit theory for point processes in manifolds. Ann. Appl.
Probab. 23: 2161–2211.
[25] Rényi A. 1970. Probability Theory Amsterdam: North-Holland.
[26] Rosenblatt M. 1956. Remarks on some nonparametric estimates of a density function, Ann.
Math. Statist. 27: 832–837.
[27] Sang H and Sang Y. 2017. Memory properties of transformations of linear processes. Stat.
Inference Stoch. Process. 20: 79-103.
[28] Schimek MG. 2000. Smoothing and Regression : Approaches, Computation, and Application.
John Wiley & Sons.
[29] Scott DW. 2015. Multivariate Density Estimation: Theory, Practice, and Visualization 2.
John Wiley & Sons.
[30] Shannon CE. 1948. A mathematical theory of communication. Bell Syst. Tech. Jour. 27:
379–423, 623–656.
[31] Silverman BW. 1986. Density Estimation for Statistics and Data Analysis. London: Chapman
and Hall.
[32] Tran LT. 1992. Kernel density estimation for linear processes. Stochastic Process. Appl. 41:
281–296.
[33] Wand MP and Jones MC. 1995. Kernel Smoothing. London: Chapman and Hall.
[34] Wu WB. 2006. Unit root testing for functionals of linear processes. Econometric Theory 22:
1–14.
[35] Wu WB, Huang Y and Huang Y. 2010. Kernel estimation for time series: An asymptotic
theory. Stochastic Process. Appl. 120: 2412–2431.
[36] Wu WB and Mielniczuk J. 2002. Kernel density estimation for linear processes. Ann. Statist.
30: 1441–1459.
[37] Zhou Z. 2014. Inference of weighted V-statistics for non stationary time series and its applications. Ann. Statist. 42: 87–114.

30

