arXiv:1710.10653v1 [math.ST] 29 Oct 2017

Robust adaptive efficient estimation for a
semi-Markov continuous time regression from
discrete data ∗
Vlad Stefan Barbu†, Slim Beltaief‡and Serguei Pergamenshchikov§

Abstract
In this article we consider the nonparametric robust estimation problem
for regression models in continuous time with semi-Markov noises observed
in discrete time moments. An adaptive model selection procedure is proposed. A sharp non-asymptotic oracle inequality for the robust risks is obtained. We obtain sufficient conditions on the frequency observations under
which the robust efficiency is shown. It turns out that for the semi-Markov
models the robust minimax convergence rate may be faster or slower than the
classical one.

MSC: primary 62G08, secondary 62G05
Keywords: Non-asymptotic estimation; Robust risk; Model selection; Sharp oracle
inequality; Asymptotic efficiency.

∗

This work was done under partial financial support of the grant of RSF number 14-49-00079
(National Research University ”MPEI” 14 Krasnokazarmennaya, 111250 Moscow, Russia) and by
RFBR Grant 16-01-00121 .
†
Laboratoire de Mathématiques Raphaël Salem, UMR 6085 CNRS-Université de Rouen Normandie, France, e-mail: barbu@univ-rouen.fr
‡
Laboratoire de Mathématiques Raphaël Salem, UMR 6085 CNRS-Université de Rouen Normandie, France, e-mail: slim.beltaief1@univ-rouen.fr
§
Laboratoire de Mathématiques Raphaël Salem, UMR 6085 CNRS-Université de Rouen Normandie, France and International Laboratory of Statistics of Stochastic Processes and Quantitative Finance of National Research Tomsk State University, Russia, e-mail: Serge.Pergamenshchikov@univrouen.fr

1

1

Introduction

In this paper we consider the semi-Markov regression model in continuous time
introduced in [1], i.e.
d yt = S(t)d t + d ξt ,

0 ≤ t ≤ n,

(1.1)

where S(·) is an unknown 1-periodic function defined on R with values on R,
(ξt )t≥0 is the unobserved noise process defined through a certain semi-Markov
process in Section 2.
Our problem in the present paper is to estimate the unknown function S in the
model (1.1) on the basis of observations
1
(ytj )0≤j≤np , tj = j∆, ∆ = ,
(1.2)
p
where the integer p ≥ 1 is the observation frequency. Firstly, this problem was
considered in the framework “signal+white noise” (see, for example, [6] or [22]).
Later, to introduce a dependence in the continuous time regression model in [11],
[9], [8] [14], the Ornstein-Uhlenbeck processes has been used to model the “color
noise”. Moreover, in order to introduce the dependence and the jumps in the regression model (1.1), the papers [15] and [16] use the non Gaussian OrnsteinUhlenbeck processes defined in [2]. The problem in all these papers is that the
introduced Ornstein-Uhlenbeck type of dependence decreases with a geometric
rate. So, asymptotically when the duration of observations goes to infinity, we obtain the same “signal+white noise” model very quick. To keep the dependence for
sufficiently large duration of observations, in [1] it was proposed the model (1.1)
with a semi-Markov component in the jumps of the noise process (ξt )t≥0 .
The main goal of this paper is to develop adaptive robust method from [1], that
was based on continuous observations, to the estimation problem based on discrete
observations given in (1.2). In this paper we use quadratic risk defined as
RQ (Sen , S) = EQ,S kSen − Sk2 ,

(1.3)

where Sen (·) is some estimate (i.e. any periodical function measurable with respect
R1
to the observations σ{yt0 , . . . ytn }), kf k2 = 0 f 2 (s)ds and EQ,S is the expectation with respect to the distribution PQ,S of the process (1.1) corresponding to the
unknown noise distribution Q in the Skorokhod space D[0, n]. We assume that this
distribution belongs to some distribution family Qn specified in Section 2.
To study the properties of the estimators uniformly over the noise distribution
(what is really needed in practice), we use the robust risk defined as
R∗n (Sen , S) = sup RQ (Sen , S) .
Q∈Qn

2

(1.4)

Thus the goal of this paper is to develop a robust efficient model selection
method based on the observations (1.2) for the model (1.1) with the semi-Markov
components in the jumps of the noise (ξt )t≥0 . We use the approach proposed
by Konev and Pergamenshchikov in [16] for continuous-time regression models
observed in the discrete time moments. Unfortunately, we cannot use directly this
method for semi-Markov regression models, since their tool essentially uses the
fact that the Ornstein-Uhlenbeck dependence decreases with geometrical rate and
obtain sufficiently quickly the “white noise” case. In the present paper, in order to
obtain the sharp non-asymptotic oracle inequalities, we use the renewal methods
from [1] developed for the model (1.1). As a consequence, we can obtain the
constructive sufficient conditions that provide the robust efficiency for proposed
model selection procedures.
The rest of the paper is organized as follows. In Section 2 we state the main
conditions under which we consider the model (1.1). In Section 3 we construct
the model selection procedure on the basis of weighted least squares estimates,
here we also specify the set of admissible weight sequences in the model selection
procedure. In Section 4 we state the main results in the form of oracle inequalities
for the quadratic risk and the robust risk. In Section 5 we study some properties
of the regression model (1.1). Section 6 is devoted to some numerical results. In
section A.2 we study some properties of the stochastic integral. Section 7 gives
the proofs of the oracle inequalities for the regression model (1.1) with the noises
introduced in Section 2. Some auxiliary are given in an Appendix.

2

Main conditions

First, we assume that the noise process (ξt )t≥ 0 in the model (1.1) is defined as
ξt = %1 Lt + %2 zt ,

(2.1)

where %1 and %2 are unknown coefficients, (Lt )t≥ 0 is a Levy process defined as
Lt = %̌ wt +

p
1 − %̌2 Ľt ,

Ľt = x ∗ (µ − µ
e)t ,

(2.2)

where, 0 ≤ %̌ ≤ 1 is some unknown constant, (wt )t≥ 0 is a standard Brownian
motion, µ(ds dx) is the jump measure with deterministic compensator µ
e(ds dx) =
dsΠ(dx), Π(·) is some positive measure on R (see, for example [10, 7] for details),
with
Π(x2 ) = 1 and Π(x8 ) < ∞ .
(2.3)

3

R
Here we use the usual notations for Π(|x|m ) = R |z|m Π(dz). Note that Π(|x|)
may be equal to +∞. In this paper we assume that the “dependent part” in the
noise (2.1) is modeled by the semi- Markov process (zt )t≥ 0 defined as
zt =

Nt
X

Yi ,

(2.4)

i=1

where (Yi )i≥ 1 is an i.i.d. sequence of random variables with
E Yi = 0, E Yi2 = 1 and E Yi4 < ∞ .

(2.5)

Here Nt is a general counting process (see, for example, [18]) defined as
Nt =

∞
X

1{Tk ≤t}

and

Tk =

k
X

τl ,

(2.6)

l=1

k=1

with (τl )l≥ 1 an i.i.d. sequence of positive integrated random variables with the
distribution η and mean τ̌ = E τ1 > 0. We assume that the processes (Nt )t≥0 and
(Yi )i≥ 1 are independent between them and are also independent of (Lt )t≥0 . Note
that the process (zt )t≥ 0 is a special case of a semi-Markov process (see, e.g., [3]
and [4]).
Remark 2.1. It should be noted that, if τj is an Exponential random variable, i.e.
g is the Exponential density, then (Nt )t≥0 is a Poisson process and, in this case,
(ξt )t≥0 is a Lévy process for which this model is studied in [12], [13] and [15].
But, in the general case when the process (2.4) is not a Lévy process, this process
has a memory and cannot be treated in the framework of semi-martingales with
independent increments. One needs to develop a new tool based on the renewal
theory arguments.
Let us denote by ρ the density of the renewal measure η̌ defined as
η̌ =

∞
X

η (l) ,

(2.7)

l=1

where η (l) is the lth convolution power of the measure η. As to the parameters in
(2.1), we assume that
σQ = %21 + %22 /τ̌ ≤ ς ∗ ,
(2.8)
where the unknown bound ς ∗ is a function of n, i.e. ς ∗ = ς ∗ (n), such that for any
ˇ > 0
ς ∗ (n)
lim nˇ ς ∗ (n) = +∞ and lim
= 0.
(2.9)
n→∞
n→∞ nˇ
4

We denote by Qn the family of all distributions of the process (2.1) in D[0, n]
satisfying the properties (2.8) – (2.9).
Remark 2.2. As we will see later, the parameter σQ is the limit of the Fourier
transform of the noise process (2.1). Such a limit is called variance proxy (see
[15]).
We assume that the distribution η has a density g that satisfies the following
conditions.
H1 ) Assume that, for any x ∈ R, there exist the finite limits
g(x−) = lim g(z)

and

z→x−

g(x+) = lim g(z)
z→x+

and, for any K > 0, there exists δ = δ(K) > 0 for which
Z δ
|g(x + t) + g(x − t) − g(x+) − g(x−)|
sup
dt < ∞.
t
|x|≤K 0

(2.10)

H2 ) For any γ > 0,
sup z γ |2g(z) − g(z−) − g(z+)| < ∞ .
z≥0

H3 ) There exists β > 0 such that

R
R

eβx g(x) dx < ∞.

Remark 2.3. It should be noted that Condition H3 ) means that there exists an
exponential moment for the random variable (τj )j≥1 , i.e. these random variables
are not too large. This is a natural constraint since these random variables define
the intervals between jumps, i.e. the jump frequency. So, to study the influence
of the jumps in the model (1.1) one needs to consider the noise process (2.1) with
“small” interval between jumps or large jump frequency.
For the next condition we need the Fourier transform for any function f : R →
R from L1 (R) defined by
Z
1
b
f (θ) =
eiθx f (x) dx .
(2.11)
2π R
H4 ) There exists t∗ > 0 such that the function gb(θ − it) belongs to L1 (R) for
any 0 ≤ t ≤ t∗ .
5

It is clear that Conditions H1 )–H4 ) hold true for any continuously differentiable function g having an exponential moment, for example, for the Γ density.
It should be noted that in view of Proposition 5.2 from [1], Conditions H1 )–
H4 ) imply
Z +∞
|Υ(x)| dx < ∞ ,
(2.12)
kΥk1 =
0

where Υ(x) = ρ(x) − 1/τ̌ .

3

Model selection

In this section we construct a model selection procedure for estimating the unknown function S given in(1.1) starting from the discrete-time observations (1.2)
and we establish the oracle inequality for the associated risk. To this end, note that
for any function f : [0, n] → R from L2 [0, n], the integral
Z n
f (s)dξs
(3.1)
In (f ) =
0

is well defined, with EQ In (f ) = 0. Moreover, as it is shown in Lemma A.2 under
the conditions H1 )–H4 ),
Z n
2
EQ In (f ) ≤ κQ
fs2 d s ,
(3.2)
0

where κQ = %21 + %22 |ρ|∗ and |ρ|∗ = supt≥0 |ρ(t)| < ∞.
In this paper we will use the trigonometric basis (φj )j≥ 1 in L2 [0, 1] defined as
φ1 = 1 ,

φj (x) =

√

2Trj (2π[j/2]x) ,

j ≥ 2,

(3.3)

where the function Trj (x) = cos(x) for even j and Trj (x) = sin(x) for odd j, [x]
denotes the integer part of x. By making use of this basis we consider the discrete
Fourier transformation of S
S(t) =

p
X

t ∈ {t1 , ..., tp },

θj,p φj (t),

(3.4)

j=1

where the Fourier coefficients are defined by
p

θj,p

1X
= (S, φj )p =
S(ti )φj (ti ).
p
i=1

6

(3.5)

In the sequel the corresponding norm will be denoted by kxk2p = (x, x)p . These
Fourier coefficients θj,p can be estimated by
1
θbj,p =
n

Z

n

Ψj,p (t)d yt ,

and

Ψj,p (t) =

0

np
X

φj (tl )1{tl−1 <t≤tl } .

(3.6)

l=1

Let us note that the system of the functions (Ψj,p )1≤j≤p is orthonormal in
L2 [0, 1] because
Z 1
Ψj,p (t)Ψi,p (t)d t = (φj , φi )p = 1{i=j} .
0

In the sequel we need the Fourier coefficients of the function S with respect to the
new basis (Ψj,p )1≤j≤p . These coefficients can be written as
Z 1
θj,p =
S(t)Ψi,p (t)d t = θj,p + hj,p ,
(3.7)
0

where
hj,p (S) =

p Z
X
l=1

tl

φj (tl )(S(t) − S(tl ))d t .

tl−1

From (1.1) it follows directly that these Fourier coefficients satisfy the equation
1
θbj,p = θj,p + √ ξj,p ,
n

1
where ξj,p = √ In (Ψj,p ) .
n

(3.8)

For any 0 ≤ t ≤ 1 we estimate the function S by the weighted least squares
estimator
n
X
b
Sλ (t) =
λ(j)θbj,p Ψj,p (t) ,
(3.9)
j=1

where the weight vector λ = (λ(1), ....., λ(n)) belongs to some finite set Λ from
[0, 1]n , θbj,n was defined in (3.6) and φj in (3.3). Now let us consider
ν = #(Λ) and

|Λ|∗ = max L(λ) ,
λ∈Λ

(3.10)

P
where #(Λ) is the cardinal number of Λ and L(λ) = nj=1 λ(j). In the sequel we
assume that |Λ|∗ ≥ 1 and λ(j) = 0 for j ≥ p.
In order to find a proper weight sequence λ in the set Λ, one needs to specify
a cost function. When choosing an appropriate cost function, one can use the
following argument. Let as consider the empirical squared error
Err(λ) = kSbλ − Sk2 ,
7

(3.11)

which in our case is equal to
Err(λ) =

n
X

2

2
λ (j)θbj,p
−2

j=1

n
X

λ(j)θbj,p θj,p + kSk2 .

(3.12)

j=1

Since the Fourier coefficients (θj )j≥ 1 are unknown, the weight coefficients (λ(j))1≤j≤p
cannot be determined by minimizing this quality. To circumvent this difficulty, one
needs to replace the terms θbj,p θj,p by their estimators θej,p . Let us set
σ
b
2
θej,p = θbj,p
− n.
n

(3.13)

Here σ
bn is an estimate for the proxy variance σQ defined in (2.8). For, example,
we can take it as
p̌

n X b2
σ
bn =
θj,p
p̌

and p̌ = min(p, n),

(3.14)

j=l

√
bn = 0 f or l > p. For this change in the empirical
where l = [ n] and we set σ
squared error, one has to pay some penalty. Thus we obtain the cost function of the
form
n
n
X
X
2
2
b
λ(j)θej,n + δ Pn (λ),
(3.15)
Jn (λ) =
λ (j)θj,n − 2
j=1

j=1

where δ > 0 is some threshold which will be specified later and the penalty term is
Pn (λ) =

σ
bn |λ|2
.
n

(3.16)

Minimizing the cost function, that is
λ̂ = argminλ∈Λ Jn (λ),

(3.17)

and substituting the obtained weight coefficients λ̂ in (3.9), lead to the model selection procedure
Sb∗ = Sbλ̂ .
(3.18)
We recall that the set Λ is finite, so λ̂ exists. In the case when λ̂ is not unique we
take one of them.

8

4
4.1

Main results
Oracle inequalities

First we define the following constant which will be used to describe the rest term
in the oracle inequalities. We set
√

n
1
gn,p = 1 + |Λ|∗
+√
.
(4.1)
p̌
p̌
Firstly, we obtain the non asymptotic oracle inequality for the model selection
procedure (3.18).
Theorem 4.1. Assume that Conditions H1 )–H4 ) hold true. Then, there exists some
constant l∗ > 0 such that, for any noise distribution Q, the weight vector set Λ, for
any periodic function S for any n ≥ 1, p ≥ 3 and 0 < δ ≤ 1/6, the procedure
(3.18) satisfies the following oracle inequality
1 + 3δ
RQ (Sb∗ , S) ≤
min R (Sbλ , S)
1 − 3δ λ∈Λ Q

ν
+ l∗
σQ + |Λ|∗ EQ |b
σn − σQ | .
δn

(4.2)

Corollary 4.2. Assume that Conditions H1 )–H4 ) hold true and that the proxy
variance σQ is known. Then there exists some constant l∗ > 0 such that for any
noise distribution Q, the weight vectors set Λ, for any periodic function S for any
n ≥ 1, p ≥ 3 and 0 < δ ≤ 1/6, the procedure (3.18) with σ
bn = σQ , satisfies the
following oracle inequality
σQ ν
1 + 3δ
RQ (Sb∗ , S) ≤
min RQ (Sbλ , S) + l∗
.
1 − 3δ λ∈Λ
δn

(4.3)

Now we study the model selection procedure (3.18) using the proxy estimate (3.14).
Theorem 4.3. Assume that the function S is continuously differentiable and that
Conditions H1 )–H4 ) hold true. Then there exists some constant l∗ > 0 such that
for any noise distribution Q, the weight vectors set Λ, for any periodic function S
for any n ≥ 1, p ≥ 3 and 0 < δ ≤ 1/6, the procedure (3.18) satisfies the following
oracle inequality
RQ (Sb∗ , S) ≤

1 + 3δ
min R (Sbλ , S)
1 − 3δ λ∈Λ Q


ν
+ l∗ (1 + σQ )3 1 + kṠk2 gn,p .
δn
9

(4.4)

Let us study the robust risks (1.4) for the procedure (3.18). In this case this family
consists of all distributions on the Skorokhod space D[0, n] of the process (2.1)
with the parameters satisfying Conditions (2.8)–(2.9).
In order to obtain the efficiency property, we specify the weight coefficients
(λ(j))1≤j≤n in the procedure (3.18). Consider, for some fixed 0 < ε < 1, a
numerical grid of the form
A = {1, . . . , k ∗ } × {ε, . . . , mε},

(4.5)

where m = [1/ε2 ]. We assume that both parameters k ∗ ≥ 1 and ε are functions of
n, i.e. k ∗ = k ∗ (n) and ε = ε(n), such that

k ∗ (n)


= 0,
 limn→∞ k ∗ (n) = +∞ , limn→∞
ln n
(4.6)



limn→∞ ε(n) = 0 and limn→∞ nδ̌ ε(n) = +∞
for any δ̌ > 0. One can take, for example, for n ≥ 2
ε(n) =

1
ln n

and

k ∗ (n) = k0∗ +

√
ln n ,

(4.7)

where k0∗ ≥ 0 is some fixed constant. For each α = (β, l) ∈ A, we introduce the
weight sequence
λα = (λα (j))1≤j≤p
with the elements


λα (j) = 1{1≤j<j∗ } + 1 − (j/ωα )β 1{j∗ ≤j≤ωα } ,

(4.8)

where j∗ = 1 + [ln υn ], ωα = (dβ lυn )1/(2β+1) ,
dβ =

(β + 1)(2β + 1)
π 2β β

and υn = n/ς ∗ .

We remind that the threshold ς ∗ is introduced in the definition of the distribution
family Qn in (2.8). Now we define the set Λ as
Λ = {λα , α ∈ A} .

(4.9)

These weight coefficients are used in [15, 16] for continuous time regression
models to show the asymptotic efficiency. Note also that in this case the cardinal
of the set Λ is
ν = k∗ m .
(4.10)
10

Moreover, taking into account that dβ < 1 for β ≥ 1 we obtain for the set (4.9)
|Λ|∗ ≤ 1 + sup ωα ≤ 1 + (υn /ε)1/3 .

(4.11)

α∈A

Therefore, the last condition in (4.6) yields
|Λ|∗
= 0 for any ˇ > 0 .

n→∞ n1/3+ˇ
lim

Our goal is to bound asymptotically the term (4.1) by any power of n. To this
end, we assume the following condition on the frequency of the observations.
H5 ) Assume that there exists δ̌ > 0 such that for any n ≥ 3
p ≥ n5/6 .

(4.12)

Now, Theorem 4.3 implies the following oracle inequality.
Theorem 4.4. Assume that the unknown function S is continuously differentiable.
Moreover, assume that Conditions H1 )–H5 ) hold true. Then, for the robust risks
defined in (1.4) through the distribution family (2.8)–(2.9), the procedure (3.18)
with the coefficients (4.8) for any n ≥ 1 and 0 < δ < 1/6 satisfies the following
oracle inequality
U∗ (S)
1 + 3δ
R∗ (Sb∗ , S) ≤
min R∗ (Sbλ , S) + n
,
1 − 3δ λ∈Λ
nδ

(4.13)

where the sequence U∗n (S) > 0 is such that under condition (4.6) for any r > 0
and δ̌ > 0,
U∗n (S)
lim sup
= 0.
(4.14)
n→∞ kṠk≤r
nδ̌

4.2

Robust asymptotic efficiency

Now we study the asymptotically efficiency properties for the procedure (3.18),
(4.8) with respect to the robust risks (1.4) defined by the distribution family (2.8)
– (2.9). To this end, we assume that the unknown function S in the model (1.1)
belongs to the Sobolev ball
k
Wrk = {f ∈ Cper
[0, 1] ,

k
X
j=0

11

kf (j) k2 ≤ r} ,

(4.15)

k [0, 1] is the set of k times continwhere r > 0 , k ≥ 1 are some parameters, Cper

uously differentiable functions f : [0, 1] → R such that f (i) (0) = f (i) (1) for all
0 ≤ i ≤ k. The function class Wrk can be written as an ellipsoid in l2 , i.e.
Wrk

= {f ∈

k
Cper
[0, 1]

:

∞
X

aj θj2 ≤ r}

(4.16)

j=1

P
where aj = ki=0 (2π[j/2])2i .
Similarly to [15, 16] we will show here that the asymptotic sharp lower bound
for the robust risk (1.4) is given by
2k/(2k+1)

k
1/(2k+1)
∗
.
(4.17)
rk = l(r) = ((2k + 1)r)
(k + 1)π
Note that this is the well-known Pinsker constant obtained for the nonadaptive
filtration problem in “signal + small white noise” model (see, for example, [22]).
Let Πn be the set of all estimators Sbn measurable with respect to the sigmaalgebra σ{yt , 0 ≤ t ≤ n} generated by the process (1.1).
Theorem 4.5. Under Conditions (2.8) and (2.9)
sup R∗n (Sbn , S) ≥ rk∗ ,

lim inf υn2k/(2k+1) inf
n→∞

(4.18)

b ∈Π S∈W k
S
n
n
r

where υn = n/ς ∗ .
Note that, if the parameters r and k are known, i.e. for the non-adaptive estimation
case, in order to obtain the efficient estimation for the “signal+white noise” model,
Pinsker proposed in [22] to use the estimate Sbλ0 defined in (3.9) with the weights
(4.8) in which
λ0 = λα0 and α0 = (k, l0 ) ,
(4.19)
where l0 = [r/ε]ε. For the model (1.1) – (2.1) we show the same result.
Proposition 4.6. The estimator Sbλ satisfies the following asymptotic upper bound
0

lim υ 2k/(2k+1)
n→∞ n

sup R∗n (Sbλ0 , S) ≤ rk∗ .

S∈Wrk

For the adaptive estimation we user the model selection procedure (3.18) with
the parameter δ defined as a function of n satisfying
lim δn = 0 and

n−→∞

lim nδ̌ δn = 0

n−→∞

for any δ̌ > 0. For example, we can take δn = (6 + ln n)−1 .

12

(4.20)

Theorem 4.7. Assume that Conditions H1 )–H5 ) hold true. Then the robust risk
defined in (1.4) through the distribution family (2.8)–(2.9) for the procedure (3.18)
with the coefficients (4.8) and the parameter δ = δn satisfying (4.20) has the following asymptotic upper bound
lim sup υn2k/(2k+1) sup R∗n (Sb∗ , S) ≤ rk∗ .

(4.21)

S∈Wrk

n→∞

Theorem 4.5 and Theorem 4.7 imply the following result.
Corollary 4.8. Under the conditions of Theorem 4.7,
lim υn2k/(2k+1)

n→∞

sup R∗n (Sbn , S) = rk∗ .

inf

(4.22)

b ∈Π S∈W k
S
n
n
r

Remark 4.1. It is well known that the optimal (minimax) risk convergence rate for
the Sobolev ball Wrk is n2k/(2k+1) (see, for example, [22], [21]). We see here that
the efficient robust rate is υn2k/(2k+1) , i.e. if the distribution upper bound ς ∗ → 0
as n → ∞ we obtain a faster rate with respect to n2k/(2k+1) , and if ς ∗ → ∞ as
n → ∞ we obtain a slower rate. In the case when ς ∗ is constant the robuste rate
is the same as the classical non robuste convergence rate.

5

Properties of the regression model (1.1)

In order to prove the oracle inequalities we need to study the conditions introduced
in [15] for the general semi-martingale model (1.1). To this end, we set for any
x ∈ Rn the functions
B1,Q (x) =

n
X

xj



2
EQ ξj,p
− σQ



and

B2,Q (x) =

j=1

n
X

xj ξej,p ,

(5.1)

j=1

2 − E ξ2 .
where σQ is defined in (2.8) and ξej,p = ξj,p
Q j,p

Proposition 5.1. Assume that Conditions H1 )–H4 ) hold true. Then
L1,Q = sup

sup

p≥3 x∈[−1,1]n

B1,Q (x) < 2τ̌ kΥk1 σQ .

(5.2)

Proof. Firstly, we set
InL (f ) =

Z

n

f (t)dLt

and

0

Inz (f ) =

Z

f (t)dzt .
0

13

n

(5.3)

In view of (2.4) the last integral can be represented as
Inz (f ) =

∞
X

f (Tl )Yl 1{Tl ≤n} .

(5.4)

l=1

Therefore,

%
%
ξj,n = √1 InL (Ψj,p ) + √2 Inz (Ψj,p )
n
n

and
2
Eξj,n
=

%21
n

∞

n

Z

Ψ2j,p (t)d t +

0

%22 X 2
E
Ψj,p (Tl )1{Tl ≤n} .
n

(5.5)

l=1

Using Proposition 5.2 from [1] we get
Z n
∞
X
Ψ2j,p (Tl )1{Tl ≤n} =
Ψ2j,p (x) ρ(x)dx
E
0

l=1

1
=
τ̌

Z

n

Ψ2j,p (x)dx

Z
+

0

n

Ψ2j,p (x)Υ(x)dx ,

0

where ρ is the renewal density introduced in (2.7). Then we obtain,
Z
%22 n 2
2
Ψj,p (x)Υ(x)dx
Eξj,n = σQ +
n 0
and

Z
sup
j≥1

n

Ψ2j,p (x)Υ(x)dx ≤ 2kΥk1 ,

(5.6)

0

where σQ = %21 + %22 /τ̌ . This directly implies the desired result.
2
To study the function B2,Q (x), we have to analyze the correlation properties
for the following stochastic integrals
Ien (f ) = In2 (f ) − EIn2 (f ) .

(5.7)

To do this we set
č1 = 1 + Π(x4 ) + kΥk21 + |ρ|∗

and

č2 = 12(1 + τ̌ )2 (1 + č1 ) .

(5.8)

Now we investigate the behavior of the integrals defined in (5.7) as functions of f .
Proposition 5.2. For any left continuous functions f, g : (0, ∞) −→ R such that
kf k∗ ≤ 1, kgk∗ ≤ 1, we have

2
(5.9)
|EIen (f )Ien (g)| ≤ 12σQ
(1 + τ̌ )2 (f, g)2n + nč1 .
14

Using these properties we can obtain the following bound.
Proposition 5.3. Assume that Conditions H1 )–H4 ) hold true. Then, for all n ≥ 1,
2
2
L2,Q = sup sup E B2,Q
(x) ≤ č2 σQ
,

(5.10)

p≥3 |x|≤1

where |x|2 =

Pn
j=1

x2j .

Proof. Note that

2
n
n
n
X
1 XX
e


E
xj ξj,p
|xj | |xl ||EIen (Ψj,p )Ien (Ψl,p )| .
≤ 2
n
j=2

j=1 l=1

Using here Proposition 5.2 and taking into account that
Z n
(Ψj,p , Ψl,p )n =
Ψj,p (t)Ψl,p (t)dt = n1{j=l} ,
0

we obtain the bound (5.10). Hence we obtain the desired result.
Now we can study the estimate (3.18).

2

Proposition 5.4. Assume that Conditions H1 ) and H4 ) hold true for the model
(1.1) and that S(·) is continuously differentiable. Then, for any n ≥ 2 and p ≥ 3,
√

1
n
+√
EQ,S |b
σn − σQ | ≤ č3
(1 + kṠk2 )(1 + σQ )2 ,
(5.11)
p̌
p̌
√
where č3 = 6 (14 + 2|ρ|∗ + 3 1 + č1 ) (1 + τ̌ ).
Remark 5.1. Propositions 5.1 and 5.3 are used to obtain the oracle inequalities
given in Section 4 (see, for example, [15]).

6

Simulation

In this section we report the results of a Monte Carlo experiment to assess the
performance of the proposed model selection procedure (3.18). In (1.1) we chose
a 1-periodic function which, for 0 ≤ t ≤ 1, is defined as
(
|t − 21 | if 41 ≤ t ≤ 34 ,
S(t) =
(6.1)
1
elsewhere.
4

15

n

R

R∗

20

0.0398

0.211

100

0.0091

0.0483

200

0.0067

0.0355

1000

0.0022

0.0116

Table 1: Empirical risks
We simulate the model
dyt = S(t)dt + dξt ,
where ξt = 0.5dwt + 0.5dzt . Here zt is the semi-Markov process defined in
(2.4) with a Gaussian N (0, 1) sequence (Yj )j≥1 and (τk )k≥1 used in (2.6) taken as
τk ∼ χ23 .
We use the
p model selection procedure (3.18) with the weights (4.8) in which
k ∗ = 100 + ( ln(n)), ti = i/ ln(n), m = [ln2 (n)] and δ = (3 + ln(n))−2 . We
define the empirical risk as
p

2
1X 
R=
Ê Ŝn (tj ) − S(tj ) ,
p

(6.2)

j=1

where the observation frequency p = 100001 and the expectations was taken as an
average over N = 10000 replications, i.e.


Ê Ŝn (.) − S(.)

2

N
2
1 X l
=
Ŝn (·) − S(·) .
N
l=1

We set the relative quadratic risk as
p

R∗ = R/kSk2p

and

kSk2p =

1X 2
S (tj ) .
p

(6.3)

j=0

In our case kSk2p = 0.1883601. The table below gives the values for the sample
risks (6.2) and (6.3) for different numbers of observations n.
16

Figures 1–3 show the behavior of the regression function and its estimates by
the model selection procedure (3.18) depending on the values of observation periods n. The black full line is the regression function (6.1) and the red dotted line is
the associated estimator.
Remark 6.1. From numerical simulations of the procedure (3.18) with various observations numbers n we may conclude that the quality of the proposed procedure
is good for practical needs, i.e. for reasonable (non large) number of observations. We can also add that the quality of the estimation improves as the number of
observations increases.

17

0.5
0.0
-0.5
-1.0
0.0

0.2

0.4

0.6

0.8

1.0

-1.0

-0.5

0.0

0.5

Figure 1: Estimator of S for n = 20

0.0

0.2

0.4

0.6

0.8

1.0

Figure 2: Estimator of S for n = 100

18

0.5
0.0
-0.5
-1.0
0.0

0.2

0.4

0.6

0.8

1.0

-1.0

-0.5

0.0

0.5

Figure 3: Estimator of S for n = 200

0.0

0.2

0.4

0.6

0.8

1.0

Figure 4: Estimator of S for n = 1000

19

7

Proofs

7.1

Proof of Theorem 4.1

Using the cost function given in (3.15), we can rewrite the empirical squared error
in (3.12) as follows
Err(λ) = Jn (λ) + 2

n
X

λ(j)θ̌j,p + kSk2 − ρP̂n (λ),

(7.1)

j=1

where
σQ − σ
bn
1
1
1
,
θ̌j,p = θej,p − θj,p θbj,p = √ θj,p ξj,p + ξej,p + ςj,n +
n
n
n
n
2 − σ and ξe = ξ 2 − E ξ 2 . Setting
with ςj,p = EQ ξj,p
Q j,p
Q
j,p
j,p
n

1 X
M (λ) = √
λ(j)θj ξj,p
n
j=1

and

Pn0

σQ |λ|2
=
,
n

(7.2)

we can rewrite (7.1) as
σQ − σ
bn
2
L(λ) + 2M (λ) + B1,Q (λ)
n
n
p
B2,Q (e(λ))
+ 2 Pn0 (λ) √
+ kSk2 − ρPn (λ),
σQ n

Err(λ) = Jn (λ) + 2

(7.3)

where e(λ) = λ/|λ| and the function L(·) was defined in (3.10). Let λ0 =
b be defined as in (3.17). Substituting
(λ0 (j))1≤j≤ p be a fixed sequence in Λ and λ
b in Equation (7.3), we obtain
λ0 and λ
σQ − σ
bn
2
L($) + B1,Q ($) + 2M ($)
n
n
q
q
B (b
e)
B2,Q (e0 )
b √2,Q
+ 2 Pn0 (λ)
− 2 Pn0 (λ0 ) √
σQ n
σQ n

b − Err(λ0 ) =J(λ)
b − J(λ0 ) + 2
Err(λ)

b + δPn (λ0 ),
− δPn (λ)
b − λ , eb = e(λ)
b and e0 = e(λ0 ). Note that, by (3.10),
where $ = λ
0
|L($)| ≤ L(λ̂) + L(λ) ≤ 2|Λ|∗ .
20

(7.4)

The inequality
2|ab| ≤ δa2 + δ −1 b2

(7.5)

implies that, for any λ ∈ Λ,
2 (e(λ))
p
B2,Q
|B2,Q (e(λ))|
2 Pn0 (λ) √
≤ δPn0 (λ) +
.
σQ n
δσQ n

Taking into account that 0 < δ < 1, we get
∗
2L1,Q 2B2,Q
+
Err(λ̂) ≤ Err(λ0 ) + 2M ($) +
n
δσQ n

+

1
b 2 + |λ0 |2 ) + 2δPn (λ0 ) ,
|b
σ − σQ |(|λ|
n n

∗
2 ((e(λ)). Moreover, noting that in view of (3.10)
where B2,Q
= supλ∈Λ B2,Q
2
supλ∈Λ |λ| ≤ |Λ|∗ , we can rewrite the previous bound as

b ≤Err(λ0 ) + 2M ($) +
Err(λ)
+

∗
2L1,Q 2B2,Q
+
n
δσQ n

4|Λ|∗
|b
σ − σQ | + 2δPn (λ0 ).
n

(7.6)

To estimate the second term in the right side of this inequality we set
Sx =

n
X

x = (x(j))1≤j≤n ∈ Rn .

x(j)θj,p φj ,

j=1

Thanks to (3.2) we estimate the term M (x) for any x ∈ Rn as
n

EQ M 2 (x) ≤ κQ

1
1X 2
2
x (j)θj,p = κQ kSx k2 .
n
n

(7.7)

j=1

To estimate this function for a random vector x ∈ Rn , we set
Z ∗ = sup
xεΛ1

nM 2 (x)
,
kSx k2

Λ1 = Λ − λ0 .

So, through the Inequality (7.5), we get
2|M (x)| ≤ δkSx k2 +
21

Z∗
.
nδ

(7.8)

It is clear that the last term here can be estimated as
EQ Z ∗ ≤

X nEQ M 2 (x)
X
κQ = κQ ν ,
≤
2
kSx k

x∈Λ1

(7.9)

x∈Λ1

where ν = card(Λ). Moreover, note that, for any x ∈ Λ1 ,
kSx k2 − kSbx k2 =

n
X

2

x2 (j)(θj,p − θbj2 ) ≤ −2M1 (x),

(7.10)

j=1

P
2
where M1 (x) = n−1/2 nj=1 x2 (j)θj,p ξj,n . Taking into account now that, for any
x ∈ Λ1 , the components |x(j)| ≤ 1, we can estimate this term as in (7.7), i.e.
EQ M12 (x) ≤ κQ

kSx k2
.
n

Similarly to the previous reasoning we set
Z1∗ = sup

xεΛ1

nM12 (x)
kSx k2

and we get
EQ Z1∗ ≤ κQ ν .

(7.11)

Using the same type of arguments as in (7.8), we can derive
2|M1 (x)| ≤ δkSx k2 +

Z1∗
.
nδ

(7.12)

From here and (7.10), we get
kSx k2 ≤

Z1∗
kSbx k2
+
1−δ
nδ(1 − δ)

for any 0 < δ < 1. Using this bound in (7.8) yields
2M (x) ≤

δkSbx k2
Z ∗ + Z1∗
+
.
1−δ
nδ(1 − δ)

b + Err(λ0 )), we obtain
Taking into account that kSb$ k2 ≤ 2 (Err(λ)
2M ($) ≤

b + Err(λ0 ))
2δ(Err(λ)
Z ∗ + Z1∗
+
.
1−δ
nδ(1 − δ)
22

(7.13)

Using this bound in (7.6) we obtain
∗
2B2,Q
2L1,Q
1+δ
Z ∗ + Z1∗
b
Err(λ) ≤
Err(λ0 ) +
+
+
1 − 3δ
nδ(1 − 3δ) n(1 − 3δ) δ(1 − 3δ)σQ n

+

(4|Λ|∗ + 2)
2δ
|b
σ − σQ | +
P 0 (λ0 ).
n(1 − 3δ)
(1 − 3δ) n

Moreover, for 0 < δ < 1/6 we can rewrite this inequality as
b ≤
Err(λ)

∗
1+δ
2(Z ∗ + Z1∗ ) 4L1,Q 4B2,Q
Err(λ0 ) +
+
+
1 − 3δ
nδ
n
δσQ n

+

(8|Λ|∗ + 2)
2δ
|b
σ − σQ | +
P 0 (λ0 ).
n
(1 − 3δ) n

Now, in view of the condition Proposition 5.3, we estimate the expectation of the
∗ in (7.6) as
term B2,Q
∗
EQ B2,Q
≤

X

2
EQ B2,Q
(e(λ)) ≤ νL2,Q .

λ∈Λ

Now, taking into account that |Λ|∗ ≥ 1, we get
RQ (Sb∗ , S) ≤

4κQ ν
4L1,Q 4νL2,Q
1+δ
RQ (Sbλ0 , S) +
+
+
1 − 3δ
nδ
n
δσQ n
+

2δ
10|Λ|∗
EQ |b
σ − σQ | +
P 0 (λ0 ).
n
(1 − 3δ) n

By using the upper bound for Pn (λ0 ) in Lemma A.1, we obtain that
4κQ ν
4L1,Q 4νL2,Q
1 + 3δ
RQ (Sb∗ , S) ≤
RQ (Sbλ0 , S) +
+
+
1 − 3δ
nδ
n
δσQ n
+

2δL1,Q
10|Λ|∗
EQ |b
σ − σQ | +
.
n
(1 − 3δ)n

Taking into account here that 1 − 3δ ≥ 1/2 for 0 < δ < 1/3 and that κQ ≤
(1 + τ̌ |ρ|∗ )σQ and using the bounds (5.2) and (5.10) we obtain the inequality (4.2).
Hence Theorem 4.1 .
2

23

7.2

Proof of Proposition 5.2

By Ito’s formula one gets
dIt2 (f ) = 2It− (f )dIt (f ) + %21 %̌2 f 2 (t)d t +

X

f 2 (s)(∆ξsd )2 ,

(7.14)

0≤s≤t

p
where ξtd = %3 Ľt + %2 zt and %3 = %1 1 − %̌2 . Taking into account that the
processes (Ľt )t≥0 and (zt )t≥0 are independent and the time of jumps Tk defined in
(2.6) has a density, we have ∆zs ∆Ľs = 0 a.s. for any s ≥ 0. Therefore, we can
rewrite the differential (7.14) as
dIt2 (f ) =2It− (f )dIt (f ) + %21 %̌2 f 2 (t)d t
X
X
+ %23 d
f 2 (s)(∆Ľs )2 + %22 d
f 2 (s)(∆zs )2 .
0≤s≤t

(7.15)

0≤s≤t

Therefore, using Lemma A.3 we obtain
√
EIt2 (f ) = %21 kf k2t + %22 kf ρk2t ,
Rt
P
η (j) and
where kf k2t = 0 f 2 (t)dt, ρ is the density of the renewal measure ∞
j=1
with η the distribution of τ1 . Therefore,
dIet (f ) = 2It− (f )f (t)dξt + f 2 (t)dm
et ,

m
e t = %23 m̌t + %22 mt ,

(7.16)

Rt
P
P
where m̌t = 0≤s≤t (∆Ľs )2 − t and mt = 0≤s≤t (∆zs )2 − 0 ρ(s)ds. By the
Ito formula we get
Z n
e
e
EIn (f )In (g) =E
Iet− (f )dIet (g)
0

n

Z
+E
0

h
i
e ), I(g)
e
Iet− (g)dIet (f ) + E I(f
.

(7.17)

n

First, note that the process (m̌t )t≥0 is a martingale and, using Lemma A.5, we get
Z n
Z n
Z n
2
2
2
2
e
e
e
E
It− (f )dIt (g) = ρ2 E
It− (f )g (t)dmt = ρ2 E
It−
(f )g 2 (t)dmt .
0

0

0

The last integral can be represented as
Z n
2
E
It−
(f )g 2 (t)dmt = J1 − J2 ,
0

24

where
J1 = E

X

IT2

k

2
− (f )g (Tk )1{Tk ≤n}

Z
J2 =

and

k≥1

n

0

E It2 (f )g 2 (t)ρ(t)dt .

By Lemma A.4 we get
J1 = E


E IT2

X

k


(f
)|G
g 2 (Tk )1{Tk ≤n} = %21 J1,1 + %22 J1,2 ,
−

k≥1

where
J1,1 = E

X

kf k2T g 2 (Tk )1{Tk ≤n}

and J1,2 = E

k

k≥1

k−1
XX

f 2 (Tl ) g 2 (Tk )1{Tk ≤n} .

k≥1 l=1

We obtain directly that
Z

n

kf k2t g 2 (t)ρ(t)dt

J1,1 =
0

and
J1,2 = E

X
l≥1

X

2

f (Tl )

n

Z

2

2

Z

f (x)

g (Tk )1{Tk ≤n} =

0

k≥l+1

From Lemma A.3 we obtain that
Z
Z n
2
2 2
2
kf kt g (t)ρ(t)dt + %2
J2 = %1

n

n−x



2

g (x + t)ρ(t)dt ρ(x)dx.
0

√
kf ρk2t g 2 (t)ρ(t)dt .

0

0

Therefore,
Z n
Z
2
2
2
E
It− (f )g (t)dmt = %2
0

n

Z

2

f (x)

0

n


g (t)(ρ(t − x) − ρ(t))dt ρ(x)dx .
2

x

Taking into account that ρ(t − x) − ρ(t) = Υ(t − x) − Υ(t) we can estimate the
last integral as
Z
n

|E
0

2
It−
(f )g 2 (t)dmt | ≤ 2%22 nkΥk1 .

From this and by the symmetry arguments we obtain that
Z n
Z n
|E
Iet− (f )dIet (g)| + |E
Iet− (g)dIet (f )| ≤ 4%42 nkΥk1 .
0

0

25

(7.18)

Note now that
h

E
i
D
e ), I(g)
e
I(f
= Iec (f ), Iec (g) + Dn (f, g) ,

(7.19)

n

n

where
Dn (f, g) =

X

∆Ietd (f )∆Ietd (g) .

0≤t≤n

It should be noted that the continuous and the discrete parts of the processes (7.16)
can be represented as
Z t
Z t
Z t
d
d
c
e
e
f 2 (s)dm
es .
Is− (f )f (s)dξs +
Is (f )f (s)dws and It (g) = 2
It (f ) = 2%1 %̌
0

0

0

So, in view of Lemma 6.1 from [1],
Z n
c
c
2 2
e
e
E < I (f ),I (g) >n = 4ρ1 %̌
E(It (f )It (g))f (t)g(t)dt
0
Z n
Z n
4 2
2 2 2
= 4ρ1 %̌
(f, g)t f (t)g(t)dt + 4ρ1 ρ2 %̌
(f, gρ)t f (t)g(t)dt
0
0
Z n
(f, gΥ)t f (t)g(t)dt ,
(7.20)
= 4ρ21 %̌2 σQ (f, g)2n + 4ρ21 ρ22 %̌2
0

Rt

with (f, g)t = 0 f (s)g(s)ds. Taking into account that kf k∗ ≤ 1 and kgk∗ ≤ 1,
we can estimate the last integral as
Z n
(f, gΥ)t f (t)g(t)dt ≤ nkΥk1 .
0

Therefore,
D
E
E Iec (f ), Iec (g)

n


2
(f, g)2n + nτ̌ kΥk1 .
≤ 4σQ

(7.21)

To study the last term in (7.19) note that


X 
Dn (f, g) =
2It− (f )f (t)∆ξtd + f 2 (t)∆m
e t 2It− (g)g(t)∆ξtd + g 2 (t)∆m
et .
0≤t≤n

Taking into account that for any t > 0
∆ξtd ∆m
e t = %33 (∆Ľt )3 + %32 (∆zt )3 ,

26

we obtain that
X

E

It− (f )f (t)g 2 (t)∆ξtd ∆m
e t = %33 Π(x3 ) + %32 E Y13



Z

0≤t≤n

n

E It (f )f (t)g 2 (t)dt = 0 .

0

So, using the symmetry arguments, we find that
EDn (f, g) = 4E D1,n (f, g) + E D2,n (f, g) ,

(7.22)

where
D1,n (f, g) =

X

It− (f )It− (g)f (t)g(t)(∆ξtd )2

and D2,n (f, g) =

0≤t≤n

X

f 2 (t) g 2 (t)(∆m
e t )2 .

0≤t≤n

Note that
e (f, g) ,
D1,n (f, g) = %23 Ď1,n (f, g) + %22 D
1,n
where
X

Ď1,n (f, g) =

It− (f )It− (g)f (t)g(t)(∆Ľt )2

0≤t≤n

and
X

e (f, g) =
D
1,n

It− (f )It− (g)f (t)g(t)(∆zt )2 .

0≤t≤n

Now, similarly to (7.20) and taking into account that Π(x2 ) = 1, we get
Z n
Z n
E Ď1,n (f, g) =
f (t)g(t)E It (f )It (g) dt = %21
f (t)g(t) (f, g)t dt
0

+ %22

0

Z

n

f (t)g(t) (f, gρ)t dt
0

=σQ (f, g)2n + %22

Z

n

f (t)g(t) (f, gΥ)t dt .
0

So,

|E Ď1,n (f, g)| ≤ σQ (f, g)2n + nτ̌ kΥk1 .
Moreover, taking into account that EY12 = 1 we get
e (f, g) = E
ED
1,n

X

ITk − (f )ITk − (g)f (Tk )g(Tk ) 1{Tk ≤n} .

k≥1

27

(7.23)

So, in view of Lemma A.4

X 
e (f, g) = E
ED
E
I
(f
)I
(g)|G
f (Tk )g(Tk ) 1{Tk ≤n}
1,n
Tk −
Tk −
k≥1

= %21 E
= %21

Z

X
0
(f , g)Tk f (Tk )g(Tk ) 1{Tk ≤n} + %22 E D1,n (f, g)
k≥1
n

0

(f, g)t f (t)g(t)ρ(t)dt + %22 E D1,n (f, g) ,

0

where
0

D1,n (f, g) =

k−1
XX

f (Tl ) g(Tl )f (Tk )g(Tk ) 1{Tk ≤n} .

k≥1 l=1

Noting now that
Z n
Z n
1
2
(f, g)t f (t)g(t)ρ(t)dt =
(f, g)n +
(f, g)t f (t)g(t)Υ(t)dt ,
2τ̌
0
0
we obtain

n

Z

(f, g)t f (t)g(t)ρ(t)dt| ≤

|
0

1
(f, g)2n + nkΥk1 .
2τ̌

0

Furthermore, the expectation of D1,n (f, g) can be represented as
0

X

E D1,n (f, g) = E

X

f (Tl ) g(Tl )

l≥1
n

Z
=

n−x

Z
f (x)g(x)

0

=

f (Tk )g(Tk ) 1{Tk ≤n}

k≥l+1


f (x + t)g(x + t)ρ(t)dt ρ(x)dx

0

1
00
(f, g)2n + D1,n (f, g) ,
2τ̌

where the last term in this equality can be represented as
Z n−x

Z n
00
D1,n (f, g) =
f (x)g(x)
f (x + t)g(x + t)Υ(t)dt ρ(x)dx
0

+

0

1
τ̌

Z

n

Z

n−x

f (x)g(x)
0


f (x + t)g(x + t)Υ(t)dt Υ(x)dx .

0

This implies
1
00
|D1,n (f, g)| ≤ n(1 + )(1 + kΥk21 ) .
τ̌
28

Therefore,

e (f, g)| ≤ σ (f, g)2 + n(1 + τ̌ )kΥk2 .
|E D
1,n
Q
n
1

(7.24)

Finally we obtain that

2
|E D1,n (f, g)| ≤ σQ
(1 + τ̌ )2 (f, g)2n + nkΥk21 .
As to the last term in (7.22) we can calculate directly
Z
Z n
2
2
4
4
4
f (t) g (t)dt + %2
E D2,n (f, g) = %3 Π(x )

(7.25)

n

f 2 (t) g 2 (t)ρ(t)dt ,

0

0

i.e.

2
E D2,n (f, g) ≤ nσQ
Π(x4 ) + |ρ|∗ (1 + τ̌ )2 .
From here we obtain that

2
|E Dn (f, g)| ≤ σQ
(1 + τ̌ )2 4(f, g)2n + nč1 ,

(7.26)

where č1 is given in (5.8). From this and (7.21) we find

2
2
2
e ), I(g)]
e
E[I(f
n ≤ 8σQ (1 + τ̌ ) (f, g)n + nč1 .

2

This bound and (7.18) implies (5.9). Hence Lemma 5.2.

7.3

(7.27)

Proof of Proposition 5.4

It is clear that the Inequality (5.11) holds true for l > p̌. Let now l 6 p̌. Setting
0
xj = 1{[√n]6j6p̌} and subtituting (3.8) in (3.14) yields,
σ
bn =

p̌
p̌
nX
2n
1 X 2
0
M (x ) +
(θj,p )2 +
ξj,p ,
p̌
p̌
p̌
j=l

(7.28)

j=l

0

00

where M (x ) is defined in (7.2). Furthermore, putting xj = p̌−1/2 1{l6j6p̌} , one
can write the last term on the right hand side of (7.28) as
p̌
(p̌ − l + 1)σQ
1 X 2
1
1
00
0
ξj,p = √ B2,Q (x ) + B1,Q (x ) +
,
p̌
p̌
p̌
p̌
j=l

where the functions B1,Q and B2,Q are given in (5.1). Using Proposition 5.1,
Proposition 5.3 and Lemma A.7 , we come to the following upper bound
L1,Q
16kṠk2 n 2n
0
EQ |b
σn − σQ | ≤
+
EQ |M(x )| +
+
lp
p
p
29

p
L2,Q σQ (l − 1)
+
.
√
p
p

In the same way as in (7.7), we obtain

1/2
p
X
κ
4(κQ kṠk2 )1/2
0
2
Q


EQ |M (x )| ≤
θj,p
.
≤
n
l
j=l

Taking into account that κQ ≤ (1 + τ̌ |ρ|∗ )σQ and using the bounds (5.2) and
(5.10) we obtain the inequality (5.11). Hence Proposition 5.4 holds true.
2

7.4

Proof of Theorem 4.3

This prof directly follows from Theorem 4.1 and Proposition 5.4.

7.5

2

Proof of Theorem 4.5

First, we denote by Q0 the distribution of the noise (2.1) and (2.2) with the parameter %1 = ς ∗ , %̌ = 1 and %2 = 0, i.e., the distribution for the “signal + white noise”
model. So, we can estimate as below the robust risk
R∗n (Sen , S) ≥ RQ0 (Sen , S) .
Now, Theorem 6.1 from [13] yields the lower bound (4.18). Hence this finishes the
proof.
2

7.6

Proof of Proposition 4.6

First, we note that in view of (3.9) one can represent the quadratic risk for the
empiric norm k · kp as
EQ kSbλ0 − Sk2p =

p̌
1 X 2
2
λ0 (j) EQ ξj,p
+ Θp ,
n
j=1

2

Pp

where Θp =
j=1 θj,p − λ0 (j) θ j,p . We put here λ0 (j) = 0 for j > n if
p > n. The first term can be estimated by the bound (5.2) as
sup EQ

Q∈Qn

where
get

L∗1,n

p̌
X

2
λ20 (j) ξj,p
≤ ς∗

j=1

n
X

λ20 (j) + L1,Q .

j=1

= supQ∈Q L1,Q . Therefore, taking into account that υn = n/σ ∗ , we
n

sup EQ kSbλ0 − Sk2p ≤

Q∈Qn

n
L∗1,n
1 X 2
λ0 (j) +
+ Θp .
υn
n
j=1

30

Note that

n
X

1

lim

n→∞

1/(2k+1)

υn

λ20 (j) =

j=1

2(τk r)1/(2k+1) k 2
.
(k + 1)(2k + 1)

(7.29)

Furthermore, by the Inequality (7.5) for any 0 < εe < 1 we get
−1

Θp ≤ (1 + εe) Θp + (1 + εe )

p
X

h2j,p ,

(7.30)

j=1

where Θp =
this term as

Pp

2 2
j=1 (1 − λ0 (j)) θj,p .

Θp =

[ω0 ]
X

(1 −

In view of Definition (4.8), we can represent

2
λ0 (j))2 θj,p

p
X

+

j=ι0

2
:= Θ1,p + Θ2,p ,
θj,p

j=[ω0 ]+1

where ι0 = j∗ (α0 ), ω0 = ωα0 = (τk l0 υn )1/(2k+1) and l0 = [r/ε] ε. Applying
Lemma A.9 yields
Θ1,p ≤ (1 + εe)

[ω0 ]
X

(1 − λ0 (j))2 θj2 + 4π 2 r(1 + εe−1 ) ω03 p−2 .

j=l

Similarly, through Lemma A.8 we have
X
Θ2,p ≤ (1 + εe)

θj2 + (1 + εe−1 ) r p−2 .

j≥[ω0 ]+1

Hence,

Θp ≤ (1 + εe) Θ∗ι + (1 + εe−1 ) 4π 2 rω03 + r p−2 ,
0

where Θ∗l =

P
j≥l

(1 − λ0 (j))2 θj2 . Moreover, note that
sup max h2j,p ≤ kṠk2 p−2 ≤ r p−2 .

S∈Wr1 1≤j≤p

Moreover, Wrk ⊆ Wr2 for any k ≥ 2. From here and Lemma A.10 we get
sup

p
X

S∈Wrk j=1



h2j,p ≤ r p−1 1{k=1} + 3p−2 1{k≥2} .

31

Moreover, in view of Condition H5 )


lim υn2k/(2k+1) p−1 1{k=1} + ω03 p−2 = 0 .
n→∞

So,
lim sup υn2k/(2k+1) sup Θp ≤ lim sup υn2k/(2k+1) sup Θ∗ι .
S∈Wrk

n→∞

S∈Wrk

n→∞

0

To estimate the term Θ∗ι we set
0

Un = υn2k/(2k+1) sup (1 − λ0 (j))2 /aj ,
j≥ι0

where the sequence (aj )j≥1 is defined in (4.16). This leads to the inequality
X
aj θj2 ≤ Un r .
sup υn2k/(2k+1) Θ∗ι ≤ Un
0

S∈Wr1

j≥1

Taking into account that limn→∞ t0 = r, we get
lim sup Un ≤ π −2k (τk r)−2k/(2k+1) ,
n→∞

where the coefficient τk is given in (4.8). This implies immediately that
lim sup υn2k/(2k+1) sup Θp ≤
n→∞

S∈Wrk

r1/(2k+1)
.
π 2k (τk )2k/(2k+1)

(7.31)

Moreover, note that
Rk∗ =

r1/(2k+1)
2(τk r)1/(2k+1) k 2
+ 2k
.
(k + 1)(2k + 1)
π (τk )2k/(2k+1)

So, applying (7.29) and (7.31), yields
lim υ 2k/(2k+1) sup sup
n→∞ n
S∈Wrk Q∈Qn

EQ kSbλ0 − Sk2p ≤ Rk∗ .

(7.32)

Furthermore, Lemma A.6 yields that for any εe > 0
sup R∗n (Sbλ0 , S) ≤ (1 + εe) sup sup EQ kSbλ0 − Sk2p + (1 + εe−1 )r p−2 .

S∈Wrk

S∈Wrk Q∈Qn

So, in view of Condition H5 ), we derive the desired inequality
lim υn2k/(2k+1) sup R∗n (Sbλ0 , S) ≤ Rk∗ .

n→∞

S∈Wrk

32

2

Hence we obtain Proposition 4.6.

Acknowledgments. The last author is partially supported by the RSF grant 1449-00079 (National Research University “MPEI” 14 Krasnokazarmennaya, 111250
Moscow, Russia), the RSF grant 17-11-01049 (National Research Tomsk State
University), by RFBR Grant 16-01-00121, by the Ministry of Education and Science of the Russian Federation in the framework of the research project No 2.3208.2017/PCH
by the Russian Federal Professor program (Project No 1.472.2016/FPM, Ministry
of Education and Science of the Russian Federation) and by the project Systèmes
compleXes, intelligence TERritoriale et Mobilité XterM–Feder.

8
A.1

Appendix
Property of the penalty term

Lemma A.1. For any n ≥ 1 and λ ∈ Λ,
Pn0 (λ) ≤ RQ (Sbλ , S) +

L1,Q
,
n

where the coefficient Pn0 (λ) is defined in (7.2) and the L1,Q is defined in(5.2).
Proof. By the definition of Err(λ) in (3.11) one has
2
p̌ 
X
λ(j)
Err(λ) ≥
(λ(j) − 1)θj,p +
ξ
.
n j,p
j=1

In view of Proposition 5.1 we obtain that
RQ (Sbλ , S) = EQ Err(λ) ≥

n
L1,Q
1X 2
2
λ (j)EQ ξj,n
≥ Pn0 (λ) −
.
n
n
j=1

Hence we otain Lemma A.1.

A.2

Properties of stochastic integrals (3.1)

In this section we give some results of stochastic calculus for the process (ξt )t≥ 0
given in (2.1), needed all along this paper. As the process ξt is the combination of
a Lévy process and a semi-Markov process, these results are not standard and need
to be provided.
33

Lemma A.2. Assume that Conditions H1 )–H4 ) hold true. Then, for any n ≥ 1
and for any non random function f from L2 [0, n], the stochastic integral (3.1) exists
and satisfies the properties (3.2) with the coefficient κQ given in (3.2).
Lemma A.3. Let f and g be any non-random functions from L2 [0, n] and (It (f ))t≥ 0
be the process defined in (3.1). Then, for any 0 ≤ t ≤ n,
E It (f )It (g) = %21 (f, g)t + %22 (f, gρ)t ,
where (f, g)t =

Rt
0

(A.1)

f (s) g(s)ds and ρ is the density defined in (2.7).

Lemma A.4. Let f and g be bounded functions defined on [0, ∞) × R. Then, for
any k ≥ 1,




E ITk− (f ) ITk− (g) | G =

%21 (f

, g)Tk +

%22

k−1
X

f (Tl ) g(Tl ),

l=1

where G is the σ-field generated by the sequence (Tl )l≥1 , i.e., G = σ{Tl , l ≥ 1}.
Lemma A.5. Assume that Conditions H1 )–H4 ) hold true. Then, for any measurable bounded non-random functions f and g, one has
Z n
2
It−
(f )It− (g)g(t)dξt = 0.
E
0

Lemmas A.2 – A.5 are proved in [1].

A.3

Properties of the Fourier coefficients

Lemma A.6. Let f be an absolutely continuous function, f : [0, 1] → R, with
kf˙k < ∞ and g be a simple function, g : [0, 1] → R of the form g(t) =
P
p
j=1 cj χ(tj−1 ,tj ] (t), where cj are some constants. Then for any ε > 0, the function ∆ = f − g satisfies the following inequalities
k∆k2 ≤ (1+ εe)k∆k2p +(1+ εe−1 )

kf˙k2
,
p2

k∆k2p ≤ (1+ εe)k∆k2 +(1+ εe−1 )

kf˙k2
.
p2

Lemma A.7. Let the function S(t) in (1.1) be absolutly continuous and have an
absolutely integrable derivative. Then the coefficients (θj,p )16j6p defined in (3.7)
satisfy the inequalities
√
|θ1,p | 6 kSk1 and
max j|θj,p | 6 2 2kṠk1 .
(A.2)
26j6p

34

Lemma A.8. For any p ≥ 2, 1 ≤ N ≤ p and r > 0, the coefficients (θj,p )1≤j≤p
of functions S from the class Wr1 satisfy, for any εe > 0, the following inequality
p
X

2
θj,p
≤ (1 + εe)

X

θj2 + (1 + εe−1 ) r p−2 .

(A.3)

j≥N

j=N

Lemma A.9. For any p ≥ 2 and r > 0, the coefficients (θj,p )1≤j≤p of functions S
from the class Wr1 satisfy the following inequality

√
max sup |θj,p − θj | − 2π r j p−1 ≤ 0 .
(A.4)
1≤j≤p S∈W 1
r

Lemma A.10. For any p ≥ 2 and r > 0 the correction coefficients hj,p


1≤j≤p

for the functions S from the class Wr2 satisfy the following inequality
sup
S∈Wr2

p
X

h2j,p ≤ 3r p−2 .

j=1

Lemmas A.6 – A.10 are proven in [16].

35

(A.5)

References
[1] V. S. Barbu, S. Beltaif and S. M. Pergamenchtchikov. Robust adaptive
efficient estimation for semi-Markov nonparametric regression models. Preprint, 25 March (2017), https://arxiv.org/pdf/1604.04516.pdf submitted
for publication in Statistical inference for stochastic processes.
[2] O. E. Barndorff-Nielsen and N. Shephard. Non-Gaussian OrnsteinUhlenbeck-based models and some of their uses in financial mathematics.
J. Royal Stat. Soc., B 63, 167–241, 2001.
[3] V. S. Barbu and N. Limnios. Semi-Markov Chains and Hidden Semi-Markov
Models toward Applications - Their use in Reliability and DNA Analysis.
Lecture Notes in Statistics, 191, Springer, New York, 2008.
[4] N. Limnios and G. Oprisan. Semi-Markov Processes and Reliability.
Birkhäuser, Boston, 2001.
[5] C. M. Goldie. Implicit renewal theory and tails of solutions of random equations. The Annals of Applied Probability, 1 (1), 126–166, 1991.
[6] I. A. Ibragimov and R. Z. Khasminskii. Statistical Estimation: Asymptotic
Theory. Springer, BerlinNew York, 1981.
[7] R. Cont, P. Tankov. Financial Modelling with Jump Processes. Chapman &
Hall, 2004.
[8] R. Höpfner and Yu. A. Kutoyants. Estimating discontinuous periodic signals
in a time inhomogeneous diffusion. Statistical Inference for Stochastic Processes, 13 (3), 193–230, 2010.
[9] R. Höpfner and Yu. A. Kutoyants. On LAN for parametrized continuous periodic signals in a time inhomogeneous diffusion. Statist. Decisions, 27 (4),
309–326, 2009.
[10] J. Jacod, A. N. Shiryaev. Limit theorems for stochastic processes. Springer,
Berlin, 2nd edition, 2002.
[11] V. V. Konev and S. M. Pergamenshchikov. Sequential estimation of the parameters in a trigonometric regression model with the gaussian coloured
noise. Statistical Inference for Stochastic Processes, 6, 215–235, 2003.
[12] V. V. Konev and S. M. Pergamenshchikov. Nonparametric estimation in a
semimartingale regression model. Part 1. Oracle Inequalities. Vestnik Tomskogo Universiteta, Mathematics and Mechanics, 3, 23–41, 2009.
36

[13] V. V. Konev and S. M. Pergamenshchikov. Nonparametric estimation in a
semimartingale regression model. Part 2. Robust asymptotic efficiency. Vestnik Tomskogo Universiteta, Mathematics and Mechanics, 4, 31–45, 2009.
[14] V. V. Konev and S. M. Pergamenshchikov. General model selection estimation of a periodic regression with a Gaussian noise. Annals of the Institute of
Statistical Mathematics, 62, 1083–1111, 2010.
[15] V. V. Konev and S.M. Pergamenshchikov. Efficient robust nonparametric estimation in a semimartingale regression model. Ann. Inst. Henri Poincaré
Probab. Stat., 48 (4), 1217–1244, 2012.
[16] V.V. Konev and S.M. Pergamenshchikov. Robust model selection for a semimartingale continuous time regression from discrete data. Stochastic processes and their applications, 125, 294–326, 2015.
[17] D. Lamberton and B. Lapeyre. Introduction to stochastic calculus applied to
finance. Chapman & Hall, London, 1996.
[18] T. Mikosch. Non-Life Insurance Mathematics. An Introduction with Stochastic Processes. Springer, 2004.
[19] J. Jacod and A. N. Shiryaev. Limit theorems for stochastic processes. Vol.1,
Springer, New York, 1987.
[20] C. Mallows. Some comments on Cp . Technometrics, 15, 661–675, 1973.
[21] M. Nussbaum. Spline smoothing in regression models and asymptotic efficiency in L2 . Ann. Statist., 13, 984–997, 1985.
[22] M. S. Pinsker. Optimal filtration of square integrable signals in gaussian
white noise. Problems of Transimission information, 17, 120–133, 1981.

37

