arXiv:1710.03111v1 [math.ST] 6 Oct 2017

Improved robust model selection methods for
the Lévy nonparametric regression in
continuous time ∗
Pchelintsev E.A., †
Pchelintsev V.A.,
Pergamenshchikov S.M.§

‡

Abstract
In this paper we develop the James - Stein improved estimation
method for a some nonparametric periodic function observed with the
Lévy noises in continuous time. An adaptive model selection procedure based on the improved weighted least square estimates is constructed. The improvement effect for the nonparametric models is
obtained. It turns out that in the nonasymptotic studies the accuracy
improvement for nonparametric problems is more significantly than
for the parametric one. Moreover, sharp oracle inequalities for the
robust risks have been shown and the efficiency property for the improved model selection procedure has been established in the adaptive
setting.

Key words: Improved non-asymptotic estimation, James - Stein procedures, Robust quadratic risk, Non-parametric regression, Lévy process,
Model selection, Sharp oracle inequality, Asymptotic efficiency.
∗

This work was done under the Ministry of Education and Science of the Russian
Federation in the framework of the research project no. 2.3208.2017/4.6, by RFBR Grant
16-01-00121 and by the partial financial support of the RSF grant number 14-49-00079
(National Research University ”MPEI” 14 Krasnokazarmennaya, 111250 Moscow, Russia).
†
Department of Information Technologies and Business Analytics, Tomsk State University, Lenin str. 36, 634050 Tomsk, Russia, e-mail: evgen-pch@yandex.ru
‡
Department of Higher Mathematics and Mathematical Physics, Tomsk Polytechnic
University, Lenin str. 30, 634050 Tomsk, Russia, e-mail: vpchelintsev@vtomske.ru
§
Laboratoire de Mathématiques Raphael Salem, Avenue de l’Université, BP. 12, Université de Rouen, F76801, Saint Etienne du Rouvray, Cedex France and International
Laboratory of Statistics of Stochastic Processes and Quantitative Finance of National
Research Tomsk State University, e-mail: Serge.Pergamenchtchikov@univ-rouen.fr

1

AMS (2010) Subject Classification : primary 62G08; secondary
62G05

2

1

Introduction

In this paper we consider the following nonparametric estimation problem
for the regression model in continuous time
d yt = S(t)d t + dξt ,

0 ≤ t ≤ n,

(1.1)

where S(·) is an unknown 1 - periodic function, (ξt )0≤t≤n is an unobserved
noise. The problem is to estimate the function S on the observations (yt )0≤t≤n .
Note that, if (ξt )0≤t≤n is a brownian motion, then we obtain the well-known
”signal+white noise” model which is very popular in statistical radio-physics
(see, for example, [11, 31, 21, 22]). In this paper we assume that in addition
to the intrinsic noise in the radio-electronic system, approximated usually
by the gaussian white or color noise, the useful signal S is distorted by the
impulse flow described by the Lévy process, i.e. we assume that the noise
process (ξt )0≤t≤n is defined as
ξt = ̺1 wt + ̺2 zt

and zt = x ∗ (µ − µ
e)t ,

(1.2)

where ̺1 and ̺2 are some unknown constants, (wt )t≥ 0 is a standard brownian motion, µ(ds dx) is a jump measure with deterministic compensator
µ
e(ds dx) = dsΠ(dx), Π(·) is a Lévy measure, i.e. some positive measure on
R∗ = R \ {0}, (see, for example, [12, 4] for details) such that

Π(x2 ) = 1 and Π(x6 ) < ∞ .
(1.3)
R
Here we use the notation Π(|x|m ) = R |z|m Π(dz). Note that the Lévy
∗
measure Π(R∗ ) could be equal to +∞. In the sequel we will denote by Q
the distribution of the process (ξt )0≤t≤n in the Skorokhod space D[0, n] and
by Q∗n we denote all these distributions for which the parameters ̺1 and ̺2
satisfy the conditions
0 < ̺ ≤ ̺21

and σQ = ̺21 + ̺22 ≤ ς ∗ ,

(1.4)

where the bounds ̺ and ς ∗ are functions of n, i.e. ̺ = ̺ and ς ∗ = ςn∗ such
n

that for any δ̌ > 0
lim inf nδ̌ ̺ > 0 and
n→∞

n

lim n−δ̌ ςn∗ = 0 .

n→∞

(1.5)

The cause of the appearance of a pulse stream in the radio-electronic systems
can be, for example, either external unintended (atmospheric) or intentional
impulse noise and the errors in the demodulation and the channel decoding
3

for the binary information symbols. Note that, for the first time the impulse
noises for the detection signal problems have been introduced on the basis of
the compound Poisson processes was introduced by Kassam in [14]. Later,
such processes was used in [30, 20, 18, 19] for the parametric and nonparametric signal estimation problems. However, the compound Poisson process can describe only the large impulses influence of fixed single frequency.
Taking into account that, for example, in the telecommunication systems,
the impulses are without limitations on frequencies one needs to extend the
framework of the observation model by making use the Lévy processes (1.2).
In this paper we consider the estimation problem in the adaptive setting, i.e.
when the regularity of S is unknown. Moreover, we also assume that the
distribution Q of the noise process (ξt )0≤t≤n on the Skorokhod space D[0, n]
is unknown. We know only that this distribution belongs to the distribution
family Q∗n defined in (1.4)–(1.5). By these reasons we use the robust estimation approach developed for nonparametric problems in [7, 18, 19]. To this
end we define the robust risk as
R∗n (Sbn , S) = sup RQ (Sbn , S) ,

(1.6)

Q∈Q∗n

where Sbn is an estimation, i.e. any function of (yt )0≤t≤n , RQ (·, ·) is the usual
quadratic risk defined as
Z 1
2
2
b
b
RQ (Sn , S) := EQ,S kSn − Sk and kSk =
S 2 (t)dt .
(1.7)
0

In this paper we use the model selection methods. The interest to such statistical procedures is explained by the fact that they provide adaptive solutions
for the nonparametric estimation through oracle inequalities which give the
non-asymptotic upper bound for the quadratic risk including the minimal
risk over chosen the estimators family. It should be noted that for the first
time the model selection methods were proposed by Akaike [1] and Mallows
[24] for parametric models. Then, these methods had been developed for
the nonparametric estimation and the oracle inequalities for the quadratic
risks was obtained by Barron, Birgé and Massart [2], Massart [28], by Fourdrinier and Pergamenshchikov [6] for the regression models in discrete time
and Konev and Pergamenshchikov [17] in continuous time. Unfortunately,
the oracle inequalities obtained in these papers can not provide the efficient
estimation in the adaptive setting, since the upper bounds in these inequalities have some fixed coefficients in the main terms which are more than
one. To obtain the efficiency property for estimation procedures one has to
obtain the sharp oracle inequalities, i.e. the inequalities in which the factor at the principal term on the right-hand side of the inequality is close to
4

unity. To obtain such inequalities for the general non gaussian observations
one needs to use the method proposed by Konev and Pergamenshchikov in
[15, 16, 18, 19] for semimartingale models in continuous time based on the
model selection tool developed by Galtchouk and Pergamenshchikov in [8, 9]
for the heteroscedastic non Gaussian regression models in discrete time.
The goal of this paper is to develop a new sharp model selection method
for estimating the unknown signal S using the improved estimation approach.
Usually, the model selection procedures are based on the least squares estimates. But in this paper we propose to use the improved least squares estimates which enable to improve considerably the non asymptotic estimation
accuracy. At the first time such idea was proposed in [6] for the regression in
discrete time and in [17] for the Gaussian regression model in continuous time.
Our goal is to develop these methods for the non-Gaussion regression models
in continuous time and to obtain the sharp oracle inequalities. It should be
noted that generally for the conditionally gaussian regression models we can
not use the well-known improve estimators proposed in [13, 5] for gaussian or
symmetric spheric observations. To apply the improve estimation methods to
the non-Gaussian regression models in continuous time one needs to modify
the well-known James - Stein procedures and their modifications in the way
proposed in [30, 20]. So, using these estimators we construct the improved
model selection procedure and we show a sharp non asymptotic oracle inequality for the robust risks (1.6). Then to study the efficiency property for
the proposed estimation procedure we need to obtain a lower bound for the
quadratic risks. Usually, to do this one uses the van Trees inequality which
did not exist for the Lévy processes. In this paper we obtain the corresponding van Trees inequality for the Lévy regression model and then by using
this we derive the needed asymptotic sharp lower bound for the normalized
risks, i.e. we find the Pinsker constant for the model (1.1). As to the upper bound, similarly to [16], we use the obtained sharp oracle inequality for
the weight least square estimators containing the efficient Pinsker procedure.
Therefore, through the oracle inequality we estimate from above the risk for
the constructed model selection procedure by the risk of the efficient Pinsker
procedure up to some coefficient which goes to one. As a result we show the
asymptotic efficiency in the adaptive setting.
The rest of the paper is organized as follows. In Section 2 we construct
the improved least squares estimates and study the improvement effect for
the Lévy model (1.1). In Section 3 we construct the improved model selection
procedure and show the sharp oracle inequalities. The asymptotic efficiency
is studed in Section 4. In Section 5 we study some properties of the stochastic
integrals with respect to the Lévy processes. In Section 6 we prove the van
Trees inequality for the model (1.1). In Section 7 we prove all main results
5

and in Appendix we give some technical results.

2

Improved estimation

Let (φj )j≥ 1 be an orthonormal basis in L2 [0, 1]. We extend these functions
by the periodic way on R, i.e. φj (t)=φj (t + 1) for any t ∈ R.
B1 ) Assume that the basis functions are uniformly bounded, i.e. for some
constant φ∗ > 0, which may be depend on n,
sup |φj (t)| ≤ φ∗ < ∞ .

sup

(2.1)

0≤j≤n 0≤t≤1

For example, we can take the trigonometric basis defined as Tr1 ≡ 1 and for
j≥2

√  cos(2π[j/2]x) for even j ;
Trj (x) = 2
 sin(2π[j/2]x) for odd j ,

where [x] denotes integer part of x.

For estimating the unknown function S in (1.1) we consider it’s Fourier
expansion
∞
X
S(t) =
θj φj (t).
j=1

The corresponding Fourier coefficients
Z
θj = (S, φj ) =

1

S(t) φj (t) dt

0

can be estimated as

1
θbj,n =
n

Z

n

φj (t) dyt .

0

In view of (1.1), one obtains
1
θbj,n = θj + √ ξj,n ,
n

ξj,n

1
= √ In (φj ) ,
n

In (f ) =

Z

n

f (t) dξt .

(2.2)

0

As in [16] we define a class of weighted least squares estimates for S(t) as
Sbλ =

n
X
j=1

λ(j)θbj,n φj ,
6

(2.3)

where the weights λ = (λ(j))1≤j≤n ∈ Rn belong to some finite set Λ from
[0, 1]n for which we set
ν = card(Λ) and |Λ|∗ = max
λ∈Λ

n
X

λ(j) ,

j=1

where card(Λ) is the number of the vectors λ in Λ. In the sequel we assume
that all vectors from Λ satisfies the following condition.
B2 ) Assume that for any vector λ ∈ Λ there exists some fixed integer
d = d(λ) such that their first d components equal to one, i.e. λ(j) = 1 for
1 ≤ j ≤ d for any λ ∈ Λ.
Remark 2.1. Note that the weight coefficients satisfying the condition B2 )
was introduced in [29] to construct the efficient estimation for the nonparametric regression model in discrete time.
Now we need the σ - field generated by the jumps of the process (1.2),
i.e. we set Gn = σ{zt , 0 ≤ t ≤ n}.
Proposition 2.1. For any n ≥ 1 the random vector ξed,n = (ξj,n)1≤j≤d is the
Gn - conditionally Gaussian in Rd with zero mean and the covariance matrix

Gn = E ξi,n ξj,n |Gn )
1≤i,j≤d

such that

inf (tr Gn − λmax (Gn )) ≥ (d − 1)̺ ,

Q∈Q∗n

where λmax (A) is the maximal eigenvalue of the matrix A.
Now, for the first d Fourier coefficients in (2.2) we use the improved
estimation method proposed for parametric models in [30]. To this end we
P
set θen = (θbj,n )1≤j≤d . In the sequel we will use the norm |x|2d = dj=1 x2j for
any vector x = (xj )1≤j≤d from Rd . Now we define the shrinkage estimators
as
c
∗
(2.4)
θj,n
= (1 − g(j)) θbj,n and g(j) = n 1{1≤j≤d} ,
e
|θn |d

where

(d − 1)̺
 .
cn = 
p
rn∗ + dς ∗ /n n
7

The positive parameter rn∗ is such that
lim

n→∞

rn∗

= ∞ and

lim

n→∞

rn∗
nδ̌

= 0

(2.5)

for any δ̌ > 0.
Now we introduce a class of shrinkage weighted least squares estimates
for S as
n
X
∗
∗
Sλ =
λ(j)θj,n
φj .
(2.6)
j=1

We denote the difference of quadratic risks of the estimates (2.3) and
(2.6) as
∆Q (S) := RQ (Sλ∗ , S) − RQ (Sbλ , S) .
We obtain the following result.

Theorem 2.2. Let the observed process (yt )0≤t≤n describes by the equations
(1.1)–(1.2). Then for any n ≥ 1
sup
Q∈Q∗n

sup ∆Q (S) ≤ −c2n .

(2.7)

∗
kSk≤rn

Remark 2.2. The inequality (2.7) means that non asymptotically, i.e. for
any n ≥ 1, the estimate (2.6) outperforms in mean square accuracy the
estimate (2.3). As we will see later in the efficient weight coefficients d ≈ nδ̌
as n → ∞ for some δ̌ > 0. Therefore, in view of the definition of the constant
cN in (2.4) and the conditions (1.5) and (2.5) ncn → ∞ as n → ∞. This
means that improvement is considerably may better than for the parametric
regression when the parameter dimension d is fixed [30].

3

Model selection

This Section gives the construction of a model selection procedure for estimating a function S in (1.1) on the basis of improved weighted least squares
estimates and states the sharp oracle inequality for the robust risk of proposed procedure.
The model selection procedure for the unknown function S in (1.1) will be
constructed on the basis of a family of estimates (Sλ∗ )λ∈Λ . The performance
of any estimate Sλ∗ will be measured by the empirical squared error
Errn (λ) = kSλ∗ − Sk2 .
8

In order to obtain a good estimate, we have to write a rule to choose a weight
vector λ ∈ Λ in (2.6). It is obvious, that the best way is to minimise the
empirical squared error with respect to λ. Making use the estimate definition
(2.6) and the Fourier transformation of S implies
Errn (λ) =

n
X
j=1

∗ 2
λ2 (j)(θj,n
) −2

n
X
j=1

∗
λ(j)θj,n
θj + kSk2 .

(3.1)

Since the Fourier coefficients (θj )j≥1 are unknown, the weight coefficients
(λj )j≥1 can not be found by minimizing this quantity. To circumvent this
∗
difficulty one needs to replace the terms θj,n
θj by their estimators θej,n . We
set
σ
b
∗ b
θej,n = θj,n
θj,n − n ,
n
2
where σ
bn is the estimate for the limiting variance of σQ = EQ ξj,n
which we
choose in the following form
σ
bn =

n
X

√
j=[ n]+1

b
t2j,n

1
and b
tj,n =
n

Z

0

n

Trj (t)dyt .

(3.2)

For this change in the empirical squared error, one has to pay some penalty.
Thus, one comes to the cost function of the form
Jn (λ) =

n
X

λ

2

j=1

∗ 2
(j)(θj,n
)

−2

n
X
j=1

λ(j) θej,n + δ Pbn (λ) ,

where δ is some positive constant, Pbn (λ) is the penalty term defined as
σ
b |λ|2
Pbn (λ) = n n .
n

(3.3)

(3.4)

We define the improved model selection procedure as
S ∗ = Sλ∗∗

and λ∗ = argminλ∈Λ Jn (λ) .

(3.5)

It will be noted that λ∗ exists because Λ is a finite set. If the minimizing
sequence in (3.5) λ∗ is not unique, one can take any minimizer. Now, to
write the oracle inequality we set
ΨQ,n = 19 (2 + φ4∗ Π(x4 ))(1 + c∗n )σQ ν + 18c∗n ,
9

(3.6)

where c∗n = nc2n . It is useful to note that in view of the first condition in
(1.5) and the properties (2.5) the constant c∗n is not large as n → ∞, i.e. for
any ǫ > 0
c∗
lim nǫ = 0 .
(3.7)
n→∞ n
First we study the non asymptotic properties for the procedure (3.5).
Theorem 3.1. For any n ≥ 1 and 0 < δ < 1/3, the risk (1.7) of estimate
(3.5) for S satisfies the oracle inequality
RQ (Sλ∗∗ , S) ≤
+

ΨQ,n
1 + 3δ
min RQ (Sλ∗ , S) +
1 − 3δ λ∈Λ
nδ

14|Λ|∗EQ |b
σn − σQ |
.
n

In the case, when the value of σQ is known, one can take σ
bn = σQ and
σQ |λ|2n
,
Pn (λ) =
n

(3.8)

(3.9)

then we can rewrite the oracle inequality (3.8) in the following form
RQ (Sλ∗∗ , S) ≤

ΨQ,n
1 + 3δ
min RQ (Sλ∗ , S) +
.
1 − 3δ λ∈Λ
nδ

Also we study the accuracy properties for the estimator (3.2).
Proposition 3.2. Let in the model (1.1) the function S(·) is continuously
differentiable. Then, for any n ≥ 2,
EQ |b
σn − σQ | ≤

2
6kṠk2 + 4σQ + (2 + φ4∗ Π(x4 ))σQ
√
.
n

To obtain the oracle inequality for the robust risk we impose the following
additional conditions.
C∗1 ) Assume that for the upper bound for the basic function defined (2.1)
lim

n→∞

φ∗
=0
nǫ

for any ǫ > 0.

10

C∗2 ) Assume that the set Λ is such that for any ǫ > 0
νn
= 0 and
n→∞ nǫ
lim

lim

n→∞

|Λ|∗
= 0.
n1/2+ǫ

We note that Theorem 3.1 and Proposition 3.2 directly imply the following inequality.
Theorem 3.3. If the conditions C∗1 ) – C∗2 ) hold for the distribution Q of
the process ξ in (1.1), then for any n ≥ 2 and 0 < δ < 1/3, the robust risk
(1.6) of estimate (3.5) for continuously differentiable function S satisfies the
oracle inequality
R∗n (Sλ∗∗ , S) ≤

B ∗ (1 + kṠk2 )
1 + 3δ
min R∗n (Sλ∗ , S) + n
,
1 − 3δ λ∈Λ
nδ

where the term Bn∗ is independent of S and for any ǫ > 0
lim

n→∞

Bn∗
= 0.
nǫ

Now we specify the weight coefficients (λ(j))j≥1 in the way proposed in [8]
for a heteroscedastic regression model in discrete time. Consider a numerical
grid of the form
An = {1, . . . , k ∗ } × {r1 , . . . , rm } ,

where ri = iε and m = [1/ε2 ]. Both parameters k ∗ ≥ 1 and 0 < ε ≤ 1 are
assumed to be functions of n, i.e. k ∗ = k ∗ (n) and ε = ε(n), such that for any
δ>0

k ∗ (n)

 limn→∞ k ∗ (n) = +∞ , limn→∞
= 0,
ln n


limn→∞ ε(n) = 0 and limn→∞ nδ ε(n) = +∞ .
One can take, for example,
ε(n) =

1
ln(n + 1)

and k ∗ (n) =

p
ln(n + 1) .

For each α = (β, r) ∈ An we introduce the weight sequence λα = (λα (j))j≥1
as

λα (j) = 1{1≤j≤d} + 1 − (j/ωα )β 1{d<j≤ωα }
(3.10)
where d = d(α) = [ωα / ln(n + 1)],
ωα = τβ r vn

1/(2β+1)

,

τβ =

(β + 1)(2β + 1)
π 2β β
11

and vn =

n
.
ς∗

We set
Λ = {λα , α ∈ An } .

(3.11)

It will be noted that in this case νn = k ∗ m.

Remark 3.1. It will be observed that the specific form of weights (3.10) was
proposed by Pinsker [31] for the filtration problem with known smoothness
of the regression function observed with an additive gaussian white noise in
continuous time. Nussbaum [29] used such weights for the gaussian regression
estimation problem in discrete time.

4

Asymptotic efficiency

In order to study the asymptotic efficiency we define the following functional
Sobolev ball
k
X
k
Wk,r = {f ∈ Cp [0, 1] :
kf (j) k2 ≤ r} ,
j=0

where r > 0 and k ≥ 1 are some unknown parameters, Ckp [0, 1] is the space
of k times differentiable 1 - periodic R → R functions such that for any
0≤i≤k−1
f (i) (0) = f (i) (1) .

In order to formulate our asymptotic results we define the Pinsker constant
which gives the lower bound for normalized asymptotic risks

2k/(2k+1)
k
1/(2k+1)
l∗ (r) = ((1 + 2k)r)
.
(4.1)
π(k + 1)
It is well known that for any S ∈ Wk,r the optimal rate of convergence is
n
(see, for example, [31, 29]). On the basis of the model selection
procedure we construct the adaptive procedure S∗ for which we obtain the
following asymptotic upper bound for the quadratic risk, i.e. we show that
the parameter (4.1) gives a lower bound for the asymptotic normalized risks.
To this end we denote by Σn the set of all estimators Sbn of S measurable
with respect to the process (1.1), i.e. measurable with respect to σ-field
σ{yt , 0 ≤ t ≤ n}.
−2k/(2k+1)

Theorem 4.1. The robust risk (1.6) admits the following asymptotic lower
bound
lim inf inf vn2k/(2k+1) sup R∗n (Sbn , S) ≥ l∗ (r) .
n→∞

b ∈Σ
S
n
n

S∈Wk,r

12

We show that this lower bound is sharp in the following sense.
Theorem 4.2. The quadratic risk (1.6) for the estimating procedure S ∗ has
the following asymptotic upper bound
lim sup vn2k/(2k+1) sup R∗n (S ∗ , S) ≤ l∗ (r) .
n→∞

S∈Wk,r

It is clear that Theorem 4.2 and Theorem 4.1 imply
Corollary 4.3. The model selection procedure S ∗ is asymptotically efficient,
i.e.
2k
lim (vn ) 2k+1 sup R∗n (S ∗ , S) = l∗ (r) .
(4.2)
n→∞

S∈Wk,r

Remark 4.1. Note that the equality (4.2) implies that the parameter (4.1)
is the Pinsker constant in this case (cf. [31]).
Remark 4.2. It should be noted that the equality (4.2) means that the robust
efficiency holds with the convergence rate
2k

(vn ) 2k+1 .
It is well known that for the simple risks the optimal (minimax) estimation
convergence rate for the functions from the set Wk,r is n2k/(2k+1) (see, for
example, [31, 29, 11]). So, if the distribution upper bound ς ∗ → 0 as n → ∞
we obtain the more rapid rate, and if ς ∗ → ∞ as n → ∞ we obtain the more
slow rate. In the case when ς ∗ is constant the robust rate is the same as the
classical non robust convergence rate.

5

Stochastic calculus for the Lévy processes

In this section we study the process (1.2). First we recall the Novikov inequalities, [27], also referred to as the Bichteler–Jacod inequalities, see [3, 26],
providing bounds of the moments of the supremum of purely discontinuous
local martingales for p ≥ 2 and for any n ≥ 1

p/2

E sup |Υ ∗ (µ − µ
e)t |p ≤ Cp∗ E |Υ|2 ∗ µ
en
+ E |Υ|p ∗ µ
en ,
(5.1)
0≤t≤n

where Cp∗ is some positive constant. Further for any two functions f and g
from L2 [0, t] with t > 0 we use the following notations
Z t
Z t
2
(f, g)t =
f (s)g(s)ds and kf kt =
f 2 (s)ds .
(5.2)
0

0

13

Proposition 5.1. For any nonrandom function f and g from L2 [0, t]
E It (f )It (g) = σQ (f, g)t ,

(5.3)

where the noise variance σQ is given in (1.4).
Proof. Taking into account the definition of It (f ) in (2.2) and (1.2) we
obtain through the Ito formula that
It (f ) It (g) = σQ (f, g)t + Mf,g
,
t
where
Mf,g
t

=

Z

t

Υf,g
s−

0

dξs +

̺22

Z

(5.4)

t

f (s) g(s) dms ,

0

Υf,g
= f (s)Is (g) + g(s)Is(f ) and mt = x2 ∗ (µ − µ̃)t . Using now the inequality
s
(5.1) with Υ = xf and p = 2 we obtain that for any f ∈ L2 [0, t]
E

Z

t

f (s)dzs

0

2

≤ C ∗ kf k2t < ∞ .

Therefore, taking the expectation in (5.4) we obtain (5.3). Hence Proposition
5.1.
Now we set
f (f ) = Mf,f .
Iet (f ) = It2 (f ) − E It2 (f ) and M
t
t

(5.5)

For any [0, n] → R function f we introduce the following uniform norm
kf k∗,n = sup |f (t)| .
0≤t≤n

Proposition 5.2. Let f and g be two borel [0, n] → R functions such that
kf k∗,n ≤ φ∗ and kgk∗,n ≤ φ∗ . Then for any 0 < t ≤ n

2
E Iet (f ) Iet (g) ≤ σQ
2(f, g)2t + φ4∗ Π(x4 )t .

(5.6)

Proof. Using (5.4) with f = g we can obtain that the process (Iet )t≥0 satisfies
the following stochastic equation
f (f ) ,
dIet (f ) = dM
t

Ie0 (f ) = 0 .

Note that from the definition of Mt (f, f ) in (5.4) we can represent Iet (f ) as
Iet (f ) = Ietc (f ) + Ietd (f ) ,
14

(5.7)

where Ietc (f ) = 2̺1

Rt

Ietd (f )

0

Is (f )f (s)dws and

= 2̺2

Z

Z

t

Is− (f )f (s)dzs +
0

̺22

t

f 2 (s) dms .

0

Moreover, by the Ito formula
Z t
Z t
Iet (f ) Iet (g) =
Ies− (f ) dIes (g) +
Ies− (g) dIes (f )
0

+ 4̺21

0

Z

t

0

f (s) g(s)Is(f )Is (g)ds + Jˇtf,g ,

P
where Jˇtf,g = 0<s≤t ∆Iesd (f ) ∆Iesd (g). Using the last in condition (1.3) and
the inequality (5.1) we can obtain that for any bounded measurable [0, n] →
R function h
Z

6

t

sup E

0≤t≤n

h(s) dzs

0

< ∞.

(5.8)

From this and the Hölder inequality we obtain that
2/3
1/3
sup E It4(g) It2(f ) ≤ sup E It6 (g)
E It6 (f )
< ∞.
0≤t≤n

0≤t≤n

Therefore, in view of Proposition (5.1)
Z t
2
E Iet (f ) Iet (g) = 4σQ ̺1
f (s) g(s)(f, g)sds + E Jˇtf,g = 2σQ ̺21 (f, g)2t + E Jˇtf,g .
0

From the definition of the discrete part of Ietd (f ) in (5.7) we can represent the
jumps term Jˇtf,g as
f,g
f,g
f,g
Jˇtf,g = 4̺22 Jˇ1,t
+ 2̺32 Jˇ2,t
+ ̺42 Jˇ3,t
,

f,g
where Jˇ1,t
=

P

I (f )Is− (g)f (s)g(s)
0<s≤t s−

f,g
Jˇ2,t
=

X

0<s≤t

(5.9)

(∆zs )2 ,


Is− (f )f (s)g 2(s) + Is− (g)(f 2 (s) (∆zs )3

P
f,g
and Jˇ3,t
= 0<s≤t f 2 (s)g 2(s) (∆zs )4 . In view of Proposition 5.1 and the
upper bound (5.8) and taking into account that Π(x2 ) = 1 we calculate
Z t
Z t
σQ
f,g
ˇ
(f, g)2t .
E J1,t =
E Is (f )Is (g)f (s)g(s)ds = σQ
(f, g)s f (s)g(s)ds =
2
0
0
15

Similarly, we obtain that
Z t
f,g
3
ˇ
E J2,t = Π(x )
f (s)g(s) (g(s)E Is(f ) + f (s)E Is (g)) ds = 0 .
0

f,g
= Π(x4 )
and EJˇ3,t

Rt
0

f 2 (s)g 2(s) ds. So,

E Jˇtf,g = 2̺22 σQ (f, g)2t + ̺42 Π(x4 ) (f 2 , g 2)t ,
and, therefore,
2
E Iet (f ) Iet (g) = 2σQ
(f, g)2t + ̺42 Π(x4 ) (f 2 , g 2)t .

2
Taking into account here that ̺42 ≤ σQ
and the conditions of the proposition
we obtain the upper bound (5.6). Hence Proposition 5.2.
Now for any y ∈ Rn we define the following function

I t (y) =

n
X
j=1

yj Iet (φj ) .

(5.10)

For this we show the following property.
Proposition 5.3. For any n ≥ 1
2

sup
y∈Rn , kyk≤1

2 2
n .
E I n (y) ≤ (2 + φ4∗ Π(x4 ))σQ

(5.11)

Proof. From Proposition 5.2 it follows that
2
E I n (y)

=

n
X

i,j=1

2
≤ 2σQ

yi yj E Ien (φi )Ien (φj )
n
X

i,j=1

|yi | |yj | φi , φj

2

n

2
+ nφ4∗ σQ
Π(x4 )

n
X
i=1

|yi |

!2

.

Taking into account here that the functions (φj )j≥1 are orthonormal, and the
2
Pn
fact that
|y | ≤ n, we obtain the bound (5.11).
i=1 i

16

6

The van Trees inequality for the Lévy processes.

In this section we consider the following continuous time parametric regression model (1.1) with the function S defined as
S(t, θ) =

d
X

θi ψi (t) ,

i=1

with the unknown parameters θ = (θ1 , . . . , θd )′ . Here we assume that the
functions (ψ)1≤j≤d are 1-periodic and orthogonal functions.
Let us denote by νξ the distribution of the process (ξt )0≤t≤n on the Skorokhod space D[0, n]. One can check directly that in this space for any
parameters θ ∈ Rd , the distribution Pθ of the process (1.1) is absolutely
continuous with respect to the νξ and the corresponding Radon-Nikodym
derivative, for any function x = (xt )0≤t≤n from D[0, n], is defined as
dPθ
f (x, θ) =
(x) = exp
dνξ
where
xct

1
=
̺1

Z


Z
xt −

n

0

t
0

Z

R

S(t, θ) c
dxt −
̺1

Z

n
0


S 2 (t, θ)
dt ,
2̺21

v (µx (ds , dv) − Π(dv)ds)

(6.1)



and for any measurable set Λ in R with 0 ∈
/Λ
X
µx ([0, t] × Λ) =
1{∆ξs ∈̺2 Λ} .
0≤s≤t

Let Φ be a prior density on Rd having the following form:
Φ(θ) = Φ(θ1 , . . . , θd ) =

d
Y

ϕj (θj ) ,

j=1

where ϕj is some continuously differentiable density in R. Moreover, let
g(θ) be a continuously differentiable Rd → R function such that, for each
1 ≤ j ≤ d,
Z
|gj′ (θ)| Φ(θ) dθ < ∞ ,
(6.2)
lim g(θ) ϕj (θj ) = 0 and
|θj |→∞

Rd

17

where gj′ (θ) = ∂g(θ)/∂θj . For any B(X ) × B(Rd )-measurable integrable
function H = H(x, θ) we denote
Z Z
e
EH =
H(x, θ) dPθ Φ(θ)dθ
Rd

=

Z

Rd

X

Z

H(x, θ) f (x, θ) Φ(θ)dνξ (x) dθ ,

X

where X = D[0, n].
Lemma 6.1. For any Fny -measurable square integrable function b
gn and for
any 1 ≤ j ≤ d, the following inequality holds

where
Λj =

Z

e g − g(θ))2 ≥
E(b
n

Rd

gj′ (θ) Φ(θ) dθ

Λ2j

nkψj k2 ̺−2
1 + Ij
and Ij =

,

Z ϕ̇2 (z)
j
R

ϕj (z)

dz .

Proof. First of all note that, the density (6.1) on the process ξ is bounded
with respect to θj ∈ R and for any 1 ≤ j ≤ d
lim sup f (ξ, θ) = 0

a.s.

|θj |→∞

Now, we set
e =Φ
e (x, θ) =
Φ
j
j

∂ (f (x, θ)Φ(θ))/∂θj
.
f (x, θ)Φ(θ)

Taking into account the condition (6.2) and integrating by parts yield

 Z
∂
e
e
(f (x, θ)Φ(θ)) dθ νξ (dx)
E (b
gn − g(θ))Φj =
(b
gn (x) − g(θ))
∂θj
X ×Rd
Z
 Y !
Z
=
gj′ (θ) f (x, θ)Φ(θ)dθj
dθi νξ (dx) = Λj .
X ×Rd−1

R

i6=j

Now by the Bouniakovskii-Cauchy-Schwarz inequality we obtain the following
lower bound for the quadratic risk
e g − g(θ))2 ≥
E(b
n
18

Λ2j
.
e 2
EΨ
j

To study the denominator in the left hand of this inequality note that in
view of the representation (6.1)
Z n
1 ∂ f (y, θ)
1
=
ψj (t) dwt .
f (y, θ) ∂θj
̺1 0
Therefore, for each θ ∈ Rd ,
Eθ
and
Eθ
Using equality



1 ∂ f (y, θ)
f (y, θ) ∂θj
e =
Φ
j

we get

Hence Lemma 6.1.

7
7.1

1 ∂ f (y, θ)
=0
f (y, θ) ∂θj
2

1
= 2
̺1

Z

0

n

ψj2 (t)dt =

n
kψk2 .
̺21

1 ∂ f (x, θ)
1 ∂ Φ(θ))
+
,
f (x, θ) ∂θj
Φ(θ) ∂θj
e 2 = n kψk2 + I .
EΨ
j
j
̺21

Proofs
Proof of Proposition 2.1

Proof. Using (1.2) we put for any square integrated functions f
Z t
Z t
(1)
(2)
It (f ) =
f (s)dws and It (f ) =
f (s)dzs .
0

0

From here and (2.2) we can see that the vector ξ˜ has the conditionally Gaussian distribution with respect to Gn with zero mean and its covariance matrix
Gn can be rewritten as
Gn = ̺21 Id + ̺22 Dn ,
where Id is the identity matrix and the (i, j)-th element of the matrix Dn is
(2)
(2)
defined as E(In (φi )In (φj )|Gn ). Using the celebrated inequality of Lidskii
and Wieland (see, for example, in [25], G.3.a., p.334) we obtain
trGn − λmax (Gn ) ≥ ̺21 (tr Id − λmax (Id )) a.s.
Now, using (1.4) we come to desire results.
19

7.2

Proof of Theorem 2.2

Consider the quadratic error of the estimate (2.6)
kSλ∗

2

− Sk =
=

n
X

∗
(λ(j)θj,n

j=1

(λ(j)θbj,n − θj )2 + c2n − 2cn

j=1

n
X

2

− θj ) =

= kSbλ − Sk2 + c2n − 2cn

d
X

∗
(λ(j)θj,n

j=1

d
X
j=1

d
X
j=1

2

− θj ) +

(λ(j)θbj,n − θj )2

j=d+1

(θbj,n − θj )

(θbj,n − θj )ιj (θen ) ,

n
X

θbj,n
kθe k

n d

where ιj (x) = xj /kxkd for x = (xj )1≤j≤d ∈ Rd . Therefore, we can represent
the risk for the improved estimate Sλ∗ as
RQ (Sλ∗ , S)

= RQ (Sbλ , S) + c2n − 2cn EQ,S

d
X
j=1

(θbj,n − θj ) Jj,n ,

where Jj,n = E(ιj (θen )(θbj,n − θj )|Gn ). Now, taking into account that the
vector θen = (θbj,n )1≤j≤d is the Gn conditionally gaussian vector in Rd with
mean θe = (θj )1≤j≤d and covariance matrix n−1 Gn , we obtain
Z
Jj,n =
ιj (x)(x − θj )p(x|Gn )dx .
Rd

Here p(x|Gn ) is the conditional distribution density of the vector θen , i.e.


(x − θ)′ G−1
(x − θ)
1
n
p
.
exp −
p(x|Gn ) =
2
(2π)d/2 det Gn

Recall, that the ′ denotes the transposition. Changing the variables by u =
G−1/2
(x − θ), one finds that
n
Jj,n



Z
d
kuk2d
1 X
g
ι̃ (u)ul exp −
du ,
=
(2π)d/2 l=1 j,l Rd j,n
2

where e
ιj,n (u) = ιj (Gn1/2 u + θ) and gij denotes the (i, j)-th element of G1/2
.
n
Furthermore, integrating by parts, the integral Jj,n can be rewritten as
Jj,n =



∂ιj
E gjl gkl
(u)|u=θe |Gn .
n
∂uk
k=1

d X
d
X
l=1

20

In view of the inequality z ′ Az ≤ λmax (A)kzk2 and Proposition 2.1 we obtain
that
!
e′ G θe
θ
trG
n
n
n
∆Q (S) = c2n − 2cn n−1 EQ,S
− n
kθe k
kθe k3
n d

≤ c2n − 2cn (d − 1)̺n−1 EQ,S

n

1
.
kθe k
n d

Moreover, using the Jensen inequality we can estimate the last expectation
from below as
EQ,S (kθen kd )−1 = EQ,S (kθe + n−1/2 ξen kd )−1 ≥ (kθkd + n−1/2 EQ,S kξen kd )−1 .

From Proposition 5.1 and the condition (1.4) we obtain
EQ,S kξen k2d ≤ ςn∗ d .

So, for kSk ≤ rn∗

q
−1

EQ,S kθen k−1 ≥ rn∗ + dςn∗ /n

and, therefore,

∆Q (S) ≤ c2n − 2cn
Hence Theorem 2.2.

7.3

(d − 1)̺
 = −c2n .
p
∗
∗
rn + dςn /n n

Proof of Theorem 3.1

Substituting (3.3) in (3.1) yields for any λ ∈ Λ
Errn (λ) = Jn (λ) + 2

n
X
j=1

Now we set



σ
bn
∗
∗ b
− θj,n θj
λ(j) θj,n θj,n −
n

+ kSk2 − δ Pbn (λ) .
L(λ) =

n
X
j=1

n

λ(j) ,

1 X
B1,n (λ) = √
λ(j)gj θbj,n ξj,n ,
n
j=1

21

(7.1)

n

1 X
λ(j)θj ξj,n
M(λ) = √
n

and B2,n (λ) =

j=1

n
X
j=1



2
2
λ(j) ξj,n − EQ ξj,n ,

e )1
where gj = (cn /|θ|
d
{1≤j≤d} . Taking into account the definition (3.3), we
can rewrite (7.1) as
Errn (λ) = Jn (λ) + 2
− 2B1,n (λ) + 2

p

σQ − σ
bn
L(λ) + 2 M(λ)
n

B2,n (λ)
Pn (λ) √
+ kSk2 − δ Pbn (λ) ,
σQ n

(7.2)

where λ = λ/|λ|n . Let λ0 = (λ0 (j))1≤n be a fixed sequence in Λ and λ∗ be
as in (3.5). Substituting λ0 and λ∗ in (7.2), we consider the difference
Errn (λ∗ ) − Errn (λ0 ) ≤ 2
+2

σQ − σ
bn
L(x) + 2M(x) − 2B1,n (x)
n

p

p
B2,n (λ∗ )
B2,n (λ0 )
− 2 Pn (λ0 ) √
Pn (λ∗ ) √
σQ n
σQ n

− δ Pbn (λ∗ ) + δ Pbn (λ0 ) ,

where x = λ∗ − λ0 . Note that |L(x)| ≤ 2|Λ|∗. Applying the elementary
inequality
2|ab| ≤ εa2 + ε−1 b2

with any ε > 0 , one comes

∗
2
p
B2,n
B2,n
(λ)
B2,n (λ)
≤ εPn (λ) +
,
2 Pn (λ) √
≤ εPn (λ) +
σQ n
εσQ n
εσQ n

∗
2
where B2,n
= maxλ∈Λ B2,n
(λ). Note that from Proposition 5.3 we obtain
that
X
∗
2
2
EQ B2,n
≤
EQ B2,n
(λ) ≤ (2 + φ4∗ Π(x4 ))σQ
ν.
(7.3)
λ∈Λ

Now, taking into account that

n
X
j=1

2
gj2 θbj,n
= c2n ,

we obtain through the Cauchy–Schwarz inequality, that
1/2
1/2 |λ|
 2

|λ|
2
2
|B1,n (λ)| ≤ √ n cn λ (j) ξj,n
.
= √ n cn σQ + B2,n (λ )
n
n
22

So, for any ε > 0
2|B1,n (λ)| ≤ εPn (λ) +

c2n
∗
(σ + B2,n
).
εσQ Q

Using these bounds with ε = δ/2 one has
Errn (λ∗ ) ≤ Errn (λ0 ) + 4|Λ|∗
+

∗
4B2,n
|σQ − σ
bn |
+ 2M(x) +
n
nδσQ

4c2n
∗
(σ + B2,n
) + 2δ Pbn (λ0 ).
δσQ Q

(7.4)

Now we examine the second term in the right-hand side
2|M(x)| ≤ ε1 kSx k2 +
where

nM 2 (x)
Z = sup
2
x∈Λ1 kSx k
∗

Z∗
,
nε1

and Sx =

n
X

xj θj .

j=1

∗

It is easy to check that EQ Z ≤ σQ ν. Moreover, we have
2

kSµ k −

kSµ∗ k2

=

n
X

µ2j (θj2

n
X

x2j θj2 −

j=1

=

j=1

−

∗ 2
(θj,n
))

n
X
j=1

≤ −2M1 (x) + 2
where

≤ −2

n
X
j=1

∗
µ2j (θj,n
− θj )θj

∗
∗
(µj θj,n
− θj ) − λ0j (θj,n
− θj ) + xj θj )2

n
X
j=1

xj θj λ∗j gj (λ∗ )θbj,n − 2

n
X
j=1

n

1 X 2
xj θj ξj,n .
M1 (x) = √
n
j=1

Since for any ε2 > 0
2

n
X
j=1

xj θj λj gj (λ)θbj,n ≤ ε2 kSx k2 +
23

c2n
,
ε2

xj θj λ0j gj (λ0 )θbj,n ,

then
(1 − 2ε2 )kSx k2 ≤ −2M1 (x) + kSλ∗∗ − Sλ∗0 k2 +
≤ ε3 kSx k2 +

2c2n
ε2

2c2
Z1∗
+ kSλ∗∗ − Sλ∗0 k2 + n
nε3
ε2

or
kSλ∗∗ − Sλ∗0 k2
Z1∗
2c2n
kSx k ≤
+
+
,
nε3 (1 − 2ε2 − ε3 ) (1 − 2ε2 − ε3 ) ε2 (1 − 2ε2 − ε3 )
2

where ε3 > 0,
Z1∗

nM12 (x)
= sup
2
x∈Λ1 kSx k

and EQ Z1∗ ≤ σQ ν.

Setting ε2 = δ/4 and ε1 = ε3 = δ/2 in this inequality and using the
estimate
kSλ∗∗ − Sλ∗0 k2 ≤ 2(Errn (λ∗ ) + Errn (λ0 ))
we obtain
2|M(x)| ≤

δ(Errn (λ∗ ) + Errn (λ0 ))
4c2n
2Z ∗
Z1∗
+
+
+
.
n(1 − δ)
(1 − δ)
(1 − δ)
nδ

From here and (7.4), it follows that
Errn (λ∗ ) ≤
+
+

(1 − δ)|σQ − σ
bn |
1
Errn (λ0 ) + 4|Λ|∗
1 − 2δ
n(1 − 2δ)

∗
4(1 − δ)B2,n

nδ(1 − 2δ)σQ

+

4(1 − δ)c2n
(B ∗ + σQ )
δ(1 − 2δ)σQ 2,n

4c2n
Z1∗
2(1 − δ)Z ∗ 2δ(1 − δ) b
P (λ0 ).
+
+
+
1 − 2δ n(1 − 2δ) nδ(1 − 2δ)
1 − 2δ n

Taking the expectation and using the upper bound for Pbn (λ0 ) in Lemma A.1
with ε = δ yields
RQ (Sλ∗∗ , S) ≤

14|Λ|∗EQ |σQ − σ
bn |
1 + 3δ
1
RQ (Sλ∗0 , S) +
ΨQ,n +
.
1 − 3δ
nδ
n

Here we used also the estimates 1 + 2δ < 1 + 3δ, 1 − 2δ > 1 − 3δ and
1 − 2δ ≥ 1/3 > δ. Since this inequality holds for each λ0 ∈ Λ, this implies
Theorem 3.1.
24

7.4

Proof of Theorem 4.1

Firstly, note, that for any fixed Q ∈ Q∗n
sup R∗n (Sbn , S) ≥ sup RQ (Sbn , S) .

(7.5)

S∈Wk,r

S∈Wk,r

Now for any fixed 0 < ε < 1 we set


k + 1 1/(2k+1)
d = dn =
v
l∗ (rε )
k n

and rε = (1 − ε)r .

(7.6)

Next we approximate the unknown function by a trigonometric series with
d = dn terms, i.e. for any array z = (zj )1≤j≤dn , we set
Sz (x) =

dn
X

zj φj (x) .

j=1

To define the bayesian risk we choose a prior distribution on Rd as
κ = (κj )1≤j≤dn

and κj = sj ηj ,

where ηj are i.i.d. gaussian N (0, 1) random variables and the coefficients
s
 k
s∗j
dn
∗
and sj =
sj =
− 1.
vn
j
Furthermore, for any function f , we denote by p(f ) its projection in L2 [0, 1]
onto Wk,r , i.e.
p(f ) = PrWk,r (f ) .
Since Wk,r is a convex set, we obtain
kSb − Sk2 ≥ kb
p − Sk2

Therefore,

b S) ≥
sup R(S,

S∈Wk,r

Z

{z∈Rd

b .
b = p(S)
with p

: Sz ∈Wk,r }

ESz kb
p − Sz k2 µκ (dz) .

Using the distribution µκ we introduce the following Bayes risk
Z
b S ) µ (dz) .
e
b
RQ (S,
RQ (S) =
z
κ
Rd

25

Taking into account now that kb
pk2 ≤ r we obtain
b S) ≥ R
e (b
sup RQ (S,
Q p) − 2 R0,n

(7.7)

S∈Wk,r

with

R0,n =

Z

{z∈Rd

: Sz ∈W
/ k,r }

(r + kSz k2 ) µκ (dz) .

Therefore, in view of (7.5)
e (b
sup R∗n (Sbn , S) ≥ sup R
Q p) − 2 R0,n .

(7.8)

Q∈Q∗n

S∈Wk,r

In Lemma A.2 we studied the last term in this inequality. Now it is easy to
see that
dn
X
2
kb
p − Sz k ≥
(b
zj − zj )2 ,
j=1

R1

b (t) φj (t)dt. So, in view of Lemma 6.1 and reminding that
where zbj = 0 p
vn = n/ς ∗ n we obtain
e (b
sup R
Q p) ≥

Q∈Q∗n

sup

dn
X

−2
0<̺21 ≤σ∗ j=1 n̺1

1
+ vn (s∗j )−1


dn
dn 
s∗j
jk
1 X
1 X
1− k .
=
=
vn j=1 s∗j + 1
vn j=1
dn

Therefore, using now the definition (7.6), Lemma A.2 and the inequality (7.8)
we obtain that
1
2k
lim inf inf vn2k+1 sup R∗n (Sbn , S) ≥ (1 − ε) 2k+1 l∗ .

b
n→∞ S∈Π
n

S∈Wk,r

Taking here limit as ε → 0 we come to the Theorem 4.1.

7.5

Proof of Theorem 4.2

This theorem follows from Theorem 2.2 and Theorem 3.1 in [16].

26

8

Conclusion

In the conclusion we would like to emphasize that in this paper we develop
a new model selection method based on the improved least square estimator
versions. It turns out that the improvement effect in the nonparametric
estimation given in (2.7) is more important than in the parameter estimation
problems since the accuracy improvement is proportional to the parameter
dimension d which goes to infinity in the nonparametric setting. Recall
that, the improved estimation methods was usually used for the parametric
estimation problem only, where the parameter dimension d is always fixed
(see, for example, [5]). Therefore, the gain in the non asymptotic quadratic
accuracy from the application of the improved estimation methods is much
more significant in statistical treatment problems for nonparametric signals.
Acknowledgements.
The last author is partially supported by the Russian Federal Professor
program (project no. 1.472.2016/1.4, Ministry of Education and Science)
and by the project XterM - Feder, University of Rouen, France.

A
A.1

Appendix
Property of Penalty term

Lemma A.1. For any n ≥ 1, λ ∈ Λ and 0 < ε < 1

|Λ|∗ EQ |b
σn − σQ |
E Errn (λ)
c2n
Pbn (λ) ≤
+
+
.
1−ε
ε(1 − ε)
n

Proof. By the definition of Errn (λ) one has
Errn (λ) =

n
X
j=1

≥

n
X

∗
(λj θj,n
− θj )2 =

∗
λ2j (θj,n

j=1

2

− θj ) + 2

n
X

∗
λj (θj,n
− θj ) + (λj − 1)θj

j=1

n
X
j=1

2

∗
λj (λj − 1)θj (θj,n
− θj ).

Taking into account the condition B2 ) and the definition (2.4) we obtain that
the last term in tho sum can be replaced as
n
X
j=1

λj (λj −

∗
1)θj (θj,n

− θj ) =

n
X

27

j=1

λj (λj − 1)θj (θbj,n − θj ) ,

P
∗
i.e. E nj=1 λj (λj − 1)θj (θj,n
− θj ) = 0 and, therefore, taking into account
the definition (3.9) we obtain that

2
n
n
X
X
ξj,n
2
∗
2
2
b
E Errn (λ) ≥
λj E (θj,n − θj ) =
λj E √ − g(j)θj,n
n
j=1

j=1

n
n
X
1
1 X 2 2 b2
2
b
= Pn (λ) − 2 √ E
λj g (j)θj,n .
λj g(j)θj,n ξj,n ≥ (1 − ε) Pn (λ) − E
ε
n
j=1

j=1

Note again that the condition B2 ) and the definition (2.4) imply that
n
X
j=1

Thus,
Pn (λ) ≤

λ2j g(j)θbj,n = c2n .
c2n
E Errn (λ)
+
1−ε
ε(1 − ε)

and through (3.4) we obtain Lemma A.1.

A.2

Properties of the term R0,n .

Lemma A.2. For any m > 0 the term R0,n introduced in (7.7) satisfies the
following property
lim nm R0,n = 0 .
(A.1)
n→∞

Pdn

Proof. First, setting ζn = j=1 κ2j aj , we obtain that


dn
k
X

X

2
(l) 2
Sκ ∈
/ Wk,r =
κj
kφj k > r = {ζn > r} .


j=1

l=0

Moreover, note that one can check directly that

dn
1 X
s∗j aj = rε = (1 − ε)r .
lim E ζn = lim
n→∞
n→∞ vn
j=1

So, for sufficiently large n we obtain that
n
o

e
Sκ ∈
/ Wk,r ⊂ ζn > r1 ,
28

where r1 = rε/2,
dn
X
1
ζen = ζn − E ζn =
s∗j aj ηej
vn
j=1

and ηej = ηj2 − 1 .

Through the correlation inequality from [10] we can get that for any p ≥ 2
there exists some constant Cp > 0 for which

p/2
d
X
p
1
(s∗j )2 a2j  ≤ C vn− 4k+2 ,
E ζenp ≤ Cp p 
vn
j=1

i.e. the expectation E ζenp → 0 as n → ∞. Therefore, using the Chebychev
inequality we obtain that for any m > 1
Hence Lemma A.2.

nm P(ζen > r1 ) → 0 as n → ∞ .

References
[1] Akaike H. A new look at the statistical model identification. IEEE
Trans. on Automatic Control 19 (1974) 716–723.
[2] Barron A., Birgé L. and Massart P. (1999) Risk bounds for model
selection via penalization. Probab. Theory Relat. Fields 113, 301–415.
[3] Bichteler K., Jacod J. Calcul de Malliavin pour les diffusions avec sauts:
existence dune densité dans le cas unidimensionnel. Séminaire de probabilité, XVII, Lecture Notes in Math., 986, Springer, Berlin, 1983,
132–157.
[4] Cont R., Tankov P. Financial Modelling with Jump Processes. Chapman & Hall, 2004.
[5] Fourdrinier D., Strawderman W. E. (1996). A paradox concerning
shrinkage estimators: should a known scale parameter be replaced by
an estimated value in the shrinkage factor? Journal of Multivariate
Analysis, 59(2), 109 –140.
29

[6] Fourdrinier D., Pergamenshchikov S. (2007) Improved selection model
method for the regression with dependent noise. Annals of the Institute
of Statistical Mathematics, 59(3), 435–464.
[7] Galtchouk L.I., Pergamenshchikov S.M. (2006) Asymptotically efficient
estimates for non parametric regression models. Statistics and Probability Letters, 76 (8), 852–860.
[8] Galtchouk L.I., Pergamenshchikov S. M. (2009) Sharp non-asymptotic
oracle inequalities for nonparametric heteroscedastic regression models.
Journal of Nonparametric Statistics 21 (1), 1 - 16.
[9] Galtchouk L., Pergamenshchikov S. (2009) Adaptive asymptotically efficient estimation in heteroscedastic nonparametric regression.Journal
of Korean Statistical Society, 38(4), 305–322.
[10] Galtchouk L., Pergamenshchikov S. (2013) Uniform concentration inequality for ergodic diffusion processes observed at discrete times.
Stochastic Processes and their Applications, 123, 91109
[11] Ibragimov I. A., Khasminskii R. Z. Statistical Estimation: Asymptotic
Theory. Springer, New York, 1981.
[12] Jacod J., Shiryaev A.N. Limit theorems for stochastic processes. 2nd
edition, Springer, Berlin, 2002.
[13] James W., Stein C. (1961). Estimation with quadratic loss. In Proceedings of the Fourth Berkeley Symposium Mathematics, Statistics
and Probability, University of California Press, Berkeley, 1, 361380
[14] Kassam S.A. Signal detection in non-Gaussian noise.
Springer-Verlag Inc., IX, 1988.

New York:

[15] Konev V. V., Pergamenshchikov S. M. Nonparametric estimation in a
semimartingale regression model. Part 1. Oracle Inequalities. Journal
of Mathematics and Mechanics of Tomsk State University 3 (2009)
23–41.
[16] Konev V. V., Pergamenshchikov S. M. Nonparametric estimation in a
semimartingale regression model. Part 2. Robust asymptotic efficiency.
Journal of Mathematics and Mechanics of Tomsk State University 4
(2009) 31–45.

30

[17] Konev V. V., Pergamenshchikov S. M. General model selection estimation of a periodic regression with a Gaussian noise. Annals of the
Institute of Statistical Mathematics 62 (2010) 1083–1111.
[18] Konev V. V., Pergamenshchikov S. M. Efficient robust nonparametric estimation in a semimartingale regression model. Ann. Inst. Henri
Poincaré Probab. Stat., 48 (4), 2012, 1217–1244.
[19] Konev V. V., Pergamenshchikov S. M. Robust model selection for a
semimartingale continuous time regression from discrete data. Stochastic processes and their applications, 125, 2015, 294 – 326.
[20] Konev V., Pergamenshchikov S. and Pchelintsev E. (2014) Estimation
of a regression with the pulse type noise from discrete data. Theory
Probab. Appl., 58 (3), 442–457.
[21] Kutoyants Yu. A. Estimation of the signal parameter in a Gaussian
Noise. Problems of Information Transmission, 1977, vol. 13 (4), p. 29
36
[22] Kutoyants Yu. A. Parameter Estimation for Stochastic Processes.
Heldeman-Verlag, Berlin, 1984.
[23] Jacod J., Shiryaev A. N. Limit theorems for stochastic processes. Vol.1,
Springer, New York, 1987.
[24] Mallows C. Some comments on Cp .Technometrics 15 (1973) 661–675.
[25] Marshall A.W., Olkin I. Inequalities: Theory of Majorization and Its
Applications. Springer Series in Statistics, Springer, Academic.
[26] Marinelli C., Röckner M. (2014) On maximal inequalities for purely
discontinuous martingales in infinite dimensions. Séminaire de Probabilités, Lect. Notes Math., XLVI , 293–315.
[27] Novikov A.A. (1975) On discontinuous martingales. Theory Probab.
Appl., 20, 1, 11–26.
[28] Massart P. A non-asymptotic theory for model selection. European
Congress of Mathematics, Eur. Math. Soc., Zrich, 2005.
[29] Nussbaum M. (1985) Spline smoothing in regression models and asymptotic efficiency in L2 .- Ann. Statist. 13, 984–997.

31

[30] Pchelintsev E. (2013) Improved estimation in a non-Gaussian parametric regression. Stat. Inference Stoch. Process., 16 (1), 15 – 28.
[31] Pinsker M.S. (1981) Optimal filtration of square integrable signals in
gaussian white noise. Problems of Transimission information 17, 120–
133.

32

