Adaptation for nonparametric estimators of locally stationary processes
Rainer Dahlhaus Stefan Richter



Rainer Dahlhaus and Stefan Richter
Adaptation for locally stationary processes





1]Rainer Dahlhaus
 
[1]Institut für Angewandte Mathematik, 
    Heidelberg University, Im Neuenheimer Feld 205, Heidelberg, Germany
  
  

    
1]Stefan Richter
 
2019




Two adaptive bandwidth selection methods for nonparametric estimators in locally stationary processes are proposed. We investigate a cross validation approach and a method based on contrast minimization and derive asymptotic properties of both methods.
The results are applicable for different statistics under a broad setting of locally stationarity including nonlinear processes. At the same time we deepen the general framework for local stationarity based on stationary approximations. For example a general Bernstein inequality is derived for such processes. A simulation study performed on the covariance function and more complicated functionals shows that both adaptation methods work  well.























§ INTRODUCTION


In this paper we develop data adaptive bandwidth selection rules for nonstationary processes under a novel paradigm of local stationarity recently introduced in <cit.>. Time series sampled at high frequency or just long time series exhibit more frequently nonstationarity instead of stationarity, and correspondingly the use of models with time varying parameters or of locally stationary processes in general has increased a lot during recent years. A prominent example from financial econometrics is the use of GARCH-models to model conditional heteroscedasticity: While in the beginning ordinary GARCH-models have been regarded as sufficient to model conditional heteroscedasticity of the volatility, insight has grown that for example modeling of the daily pattern can be improved by a time varying GARCH-model (cf.
<cit.>, <cit.>,<cit.>). Analogously, time varying models for the trading intensity could be used such as locally stationary Hawkes models (<cit.>). Also motivated by financial returns Koo and Linton <cit.> have studied locally stationary diffusion processes with a time varying drift and a volatility coefficient varying over time and space.

The intention and novelty of this paper is twofold. On the one hand we want to establish methods for adaptive bandwidth selection for nonparametric estimators, on the other hand we want to deepen  the general framework for local stationarity based on stationary approximations. The latter goes hand in hand with the former since several results (such as Bernstein inequalities) are of high value beyond the topic of adaptation. The development of such a general framework / hyper-model for locally stationary processes is important, since such a hyper-model can for example serve as a general assumption for nonparametric estimation, as a framework to prove general technical results such as strong laws of large numbers or the Bernstein inequality of this paper, as a framework to judge parametric models under model-misspecification, or as a setting for model selection strategies. General frameworks for locally stationary processes that have been used before, are time varying linear processes as in <cit.>,
and time-varying Bernoulli shifts in combination with the functional dependence measure (<cit.>,<cit.>). Furthermore, processes with evolutionary spectra in the setting of Priestley (<cit.>, <cit.>)  may also be regarded as a hyper-model - although the setting does not allow for asymptotic considerations in a strict mathematical sense since it is not an infill asymptotics approach.

In <cit.> we have introduced a different framework which formalizes the original idea behind local stationarity - namely that at each point in time the observed nonstationary process can be approximated by a stationary process. Such a property was proved in the context of time varying ARCH-processes in <cit.> and investigated further in the context of random coefficient models in <cit.>. The use of such approximations as a general model was recommended by <cit.>, who investigated
nonparametric regression for locally stationary time series, and by <cit.>, who investigated semiparametric estimation for locally stationary models.

Within this new framework, the focus of this paper is on deriving methods for adaptive bandwidth selection of general nonparametric estimators. These include for example estimators of the time varying covariance function, the autocorrelation function, the time varying characteristic function or general moment estimators.

Different to nonparametric regression, there exist only very few theoretical results about adaptivity for locally stationary processes. We mention <cit.> who discussed adaptive covariance estimation for a general class of locally stationary processes. Other results are constructed for specific models and are partly dependent on further tuning parameters: <cit.> discussed online-adaptive forecasting of tvAR processes and <cit.>, <cit.> proposed methods for sequential and minimax-optimal bandwidth selection for tvAR processes of order 1. In <cit.> adaptive estimation was developed for 
time varying parameter curves by means of local M-estimators (i.e. in a locally parametric setting), while in this paper the task is nonparametrical inference also locally. Technically, the difference is that we do no longer assume that the observed time series comes from a specific model like tvGARCH or tvAR (and use this knowledge to build the estimator). Instead we are interested in general properties of the time series like the mean, covariances, correlations or characteristic functions.

 
In Section <ref> we introduce the framework of local stationarity based on stationary approximations and derivative processes. We give a short overview of the basic results from <cit.> and extend these results in that we prove an invariance property of the results also for nonlinear transformations based on infinitely many lags. In Section <ref> we prove asymptotic optimality of a global bandwidth selection approach based on cross validation  with respect to a mean squared error type distance measure. We  discuss its behavior in practice via simulations. In Section <ref> we investigate a local bandwidth selection procedure using a contrast minimization approach in the spirit of <cit.>. We prove that the resulting nonparametric estimator attains the optimal rate for the mean squared error up to a log-factor. We compare the obtained method with a global optimal selection routine and show the superiority of our method in selected examples. The section contains also a Bernstein inequality which is of interest beyond the present paper. Section <ref> contains some conclusions. The Appendix in Section <ref> contains several technical results including the proofs of the main theorems and a more general result for the setting in Section <ref>.



§ THE MODEL AND MAIN RESULTS





 §.§ The Model


We assume that we observe n realizations of a process X_t,n at time points t = 1,...,n. 
The process is considered to be locally stationary in the following sense (cf. <cit.>):

    
	Let q ≥ 1. There exists some D > 0 and for each u∈[0,1], there exists a (strictly) stationary process (X̃_t(u))_t∈ such that for all t = 1,...,n and u,u'∈[0,1], sup_u∈[0,1]X̃_0(u)_q ≤ D, sup_t,nX_t,n_q ≤ D and
	
    X_t,n - X̃_t(t/n)_q ≤ D n^-1,     X̃_0(u) - X̃_0(u')_q ≤ D |u-u'|.

	Here, we use Z_q := [|Z|^q]^1/q for random variables Z.

The conditions mean that X_t,n can be approximated locally, for |u-t/n|  ≪ 1, by a stationary process X̃_t(u). The continuity condition stated on u↦X̃_t(u) implies that the stationary approximations vary smoothly over time. This motivates the interpretation of locally stationary processes as processes which change there (approximate) stationary properties smoothly over time. The main properties of X_t,n therefore are encoded in the stationary approximations and it is therefore of interest to analyze terms of the form g(X̃_t(u),X̃_t-1(u),...) which are a natural approximation of g(Y_t,n,Y_t-1,n,...).


More detailed, define Y_t,n := (X_t,n,X_t-1,n,...,X_1,n,0,0,...) and Ỹ_t(u) := (X̃_s(u):s ≤ t). Our goal is to estimate functionals of the form

    u ↦ G(u) :=  g(Ỹ_t(u)),

where g:^→^d is some measurable function. Important examples are:

	
  * Time-varying covariances, c(u,k) := X̃_0(u) X̃_k(u) with g(x_0,x_1,...,x_k) := x_0 x_k,
	
  * Time-varying characteristic functions,  ϕ(t,u) :=  e^i t X̃_0(u), with  g_t(x) = e^itx.
	
  * Time-varying distribution functions, F(y,u) := _{X̃_t(u) ≤ y}, with g_y(x) := _{x≤ y}.
	
 
A standard estimator is given by a localized moment estimator,

    Ĝ_h(u) := 1/n∑_t=1^nK_h(t/n-u)· g(Y_t,n),


where h∈ (0,∞) is some bandwidth and K is a kernel function coming from the class  defined below.


	A function K is in the set  if K is nonnegative, Lipschitz continuous, has support [-1/2,1/2] and ∫ K(x) x = 1. We set |K|_∞ := sup_x∈[-1/2,1/2]|K(x)|.


In the following we present a general theory how to obtain asymptotic results for such estimators with a focus on adaptation, i.e. on choosing the bandwidth h. We therefore assume that g belongs to the class (M,χ,C) below which is a Lipschitz-type condition with polynomially growing constants (ie. the Lipschitz condition is relaxed for larger x and y). For some sequence of non-negative real-valued numbers χ = (χ_i)_i∈ and some sequence of complex-valued numbers x = (x_i)_i∈ define |x|_χ := ∑_i∈χ_i |x_i|.


	We say that g:^→ belongs to the class (M,χ,C) if there exists some M∈, some constant C > 0,ε > 0 and some sequence of nonnegative real numbers χ = (χ)_i∈ with χ_i = O(i^-2-ε) such that
	
    sup_x≠y|g(x) - g(y)|/|x-y|_χ· (1 + |x|_χ^M-1 + |y|_χ^M-1)≤ C.

	A function g:^→^d (d∈) is in (M,χ,C) if each component belongs to (M,χ,C).


By Hoelder's inequality it is easy to see that the following 'invariance principle' of local stationarity holds:

	If (X_t,n)_t=1,...,n is locally stationary in the sense of Assumption <ref> (with  some q > 0) and g ∈(M,χ,C), then the same holds for g(Y_t,n) (with q' = q/M).


Based on this result we can use Theorem 2.7 in <cit.> to obtain

 If X_t,n is locally stationary in the sense of Assumption  <ref> with q ≥ M and g ∈(M,χ,C), then

    Ĝ_h(u) → G(u)

in probability and in L^1 provided that nh→∞, h→ 0.


To provide a more detailed expansion of the bias Ĝ_h(u), we need the following additional assumption on the stationary approximation sequence:


	For each t∈, the process u ↦X̃_t(u) is twice continuously differentiable with
	
    sup_u∈[0,1]|∂_u X̃_t(u)|_M ≤ D,     sup_u∈[0,1]|∂_u^2 X̃_t(u)|_M ≤ D

	Additionally, g ∈(M,χ,C) is twice continuously differentiable such that for all i,j, ∂_x_i g ∈(max{M-1,1},χ,C χ_i) and ∂_x_ix_j^2 g ∈(max{M-2,1},χ,C χ_i χ_j).


The following theorem is a combination of Lemma 3.3  from the supplementary material of <cit.> and Lemmas <ref>, <ref> from the Appendix.

 Suppose that Assumption <ref> and Assumption <ref> are fulfilled for q = 2M. Let K∈.

Define σ_K^2 = ∫ K(x)^2  x, μ_K := ∫ K(x) x^2  x and the long-run variance of X̃_t(u) (see Theorem <ref> below),

    Σ(u) = ∑_k∈(g(Ỹ_0(u)),g(Ỹ_k(u))).

Then u ↦ G(u) is twice continuously differentiable and uniformly in u ∈ [-h/2,h/2], it holds that

    
  (i) 
    |Ĝ_h(u) - G(u) - h^2/2μ_K ·∂_u^2 G(u)|_2 = O((nh)^-1) + o(h^2),

    
  (ii) 
    |Ĝ_h(u) - G(u)|_2^2 = σ_K^2/nh·(Σ(u)) + h^4/4μ_K^2 · |∂_u^2 G(u)|_2^2 +  o((nh)^-1 + h^4).




From the decomposition (ii) in Theorem <ref> it can be easily seen that h ↦ |Ĝ_h(u) - G(u)|_2^2 is minimized by

    h_opt(u) = (4 σ_K^2 (Σ(u))/μ_K^2|∂_u^2 G(u)|_2^2)^1/5· n^-1/5

if |∂_u^2 G(u)|_2^2 > 0. Accordingly, we obtain that h ↦∫_0^1|Ĝ_h(u) - G(u)|_2^2  w(u)  u is minimized by

    h_opt = (4 σ_K^2 ∫_0^1(Σ(u)) w(u)  u/μ_K^2∫_0^1|∂_u^2 G(u)|_2^2 w(u)  u)^1/5· n^-1/5

where w:[0,1] → [0,∞) is some weight function taking care of boundary issues.



 §.§ Asymptotic normality


To prove central limit theorems, we additionally have to assume mixing conditions. In the setting of Assumption <ref> it is enough to state these assumptions pointwise on the stationary approximations X̃_t(u). An elegant way to formulate mixing assumptions was the introduction of the functional dependence measure of <cit.>. Suppose that ζ_t, t∈ is a sequence of i.i.d. random variables, and put _t := (ζ_t,ζ_t-1,...), t ≥ 0. Let ζ_t^*, t∈ be an independent copy of ζ_t, t∈, and put _t^* := (ζ_t,ζ_t-1,...,ζ_1,ζ_0^*,ζ_-1,ζ_-2,...), t ≥ 0.


    Assume that for each u∈[0,1], X̃_t(u) = H(_t,u) with some measurable function H. Suppose that
    
    δ_q^X̃(u)(k) := X̃_t(u) - X̃_t^*(u)_q,    k ≥ 0,

    fulfills ∑_k=0^∞sup_u∈[0,1]δ_q^X̃(u)(k) < ∞.


The following theorem is due to Theorem 2.10 in <cit.> and Theorem <ref>:

 Suppose that Assumption <ref>, Assumption <ref> and Assumption <ref> are fulfilled for q = 2M. Additionally, assume that sup_u∈[0,1]|X̃_t(u)|_2M < ∞. Then, as n→∞, h→ 0 and nh^5 = O(1),

    √(nh)[Ĝ_h(u) - G(u) - h^2/2μ_K·∂_u^2 G(u)]  N(0, σ_K^2·Σ(u)).





§ GLOBAL ADAPTIVE BANDWIDTH SELECTION: CROSS VALIDATION



In the following we discuss a global bandwidth selection method for Ĝ_h(u). The goal is to find an estimator which minimizes

    d_M(h) := ∫_0^1|G(u) - Ĝ_h(u)|_2^2   w(u)  u,

where w:[0,1] → [0,∞) is some weight function of bounded variation which takes care of the boundary effects. Typically, w(·) = _[γ,1-γ](·) with some γ>0.  
In practice this means to find an estimator ĥ which is close to h_opt from hopt_global.


An important advantage of cross validation over other methods is that it allows bandwidth selection without introducing sensitive tuning parameters. Typically, cross validation works well for global bandwidth selection but gets instable due to its high variance if one wants to use it locally. We therefore only present a theory for global selection. The next chapter discusses local bandwidth selection which aims to find an estimator for h_opt(u) for each u∈[0,1] separately.






The method presented here is based on the following idea: G(u) =  g(Ỹ_0(u)) is a minimizer of the functional

    a ↦[ (g(Ỹ_0(u)) - a)^2].

Therefore we expect G to be a minimizer of the empirical version

    H(G) := 1/n∑_t=1^n| g(Y_t,n) - G(t/n)|_2^2  w(t/n).

 It turns out that the simple plug-in approach to minimize h ↦ H(Ĝ_h) does not work. The reason being that in this case, H(Ĝ_h) is no unbiased estimator of ∫_0^1|g(Ỹ_t(u)) - Ĝ_h(u)|_2^2  w(u)  u.
 
To obtain an unbiased estimator, we have to eliminate the dependencies which occur between the observation g(Y_t,n) and Ĝ_h(t/n) which leads to a natural change of the estimator Ĝ_h:
Define for α∈ (0,1), ε > 0,

    K^(n)(x) := 
    		K(x),     |x| ≥ n^-α,
    
    		0,     |x| ≤ (1-ε) n^-α,
    linear,     (1-ε) n^-α < |x| < n^-α,

as a Lipschitz-continuous approximation of K(x)_{|x| ≥ n^-α}, and

    Ĝ_h^-(u) := (1/n∑_t=1^nK_h^(n)(t/n-u))^-1·1/n∑_t=1^nK_h^(n)(t/n-u) · g(Y_t,n).

Note that Ĝ_h^-(t/n) and g(Y_t,n) are now (approximately) uncorrelated if the sequence g(Y_t,n), t = 1,...,n fulfills appropriate dependence conditions.

We now define ĥ accordingly to our original idea as

    ĥ := _h∈ H_n H(Ĝ_h^-).

The final estimator of G is then given by Ĝ_ĥ.

To judge the quality of Ĝ_ĥ, we use the mean-squared error distance d_M(h).










The following theorem states that ĥ, chosen by the cross validation procedure def_crossval, is asymptotically optimal in the sense that ĥ (i.e. the estimator Ĝ_ĥ associated to ĥ) attains the minimal distance to G with respect to d_A over all possible bandwidths h∈ H_n.



Suppose that Assumption <ref> holds for all q > 0 with some constants D_q > 0. Suppose that
sup_u∈[0,1]δ^X̃_0(u)_q(k) = O(k^-κ) with some κ > 3, and g ∈(M,χ,C) with some χ = (χ_i)_i∈ with χ_i = O(i^-κ).

Let K ∈. For arbitrary small η > 0, let H_n = [n^-1+α+η,n^min{2α-1,0}-η]. Assume that the support of w is [γ,1-γ] with some γ > 0.
Then almost surely,

    lim_n→∞d_M(ĥ)/inf_h∈ H_nd_M(h) = 1.




	
  * Choice of α: From a theoretical point of view, it is possible to choose α∈ (0,1) very near to 1. In practice however it may lead to more stable choices to take α smaller, for instance α≈1/2. This is discussed in more detail in the simulations below.
	
  * Choice of ε: While it is necessary for theoretical proofs to choose ε>0 (so that K^(n) is still Lipschitz continuous), in practice there seems to be no drawback when using ε=0.
	
  * Choice of H_n: In practice it is not necessary to bound H_n from above. However, to obtain meaningful bandwidth selections it is necessary to implement the lower bound given in the conditions of Theorem <ref> or directly restrict the search to local minima.



By using the representation formel_mse integrated over u ∈ [0,1], we directly obtain

    Under the assumptions of Theorem <ref>, almost surely
    
    ĥ/h_opt→ 1,

    where h_opt is defined in hopt_global.



Many common estimators are given as compositions of moment estimators, a prominent example being the autocorrelation function of lag 1,

    γ(u) := c(u,1)/c(u,0),      c(u,k) = X̃_0(u) X̃_k(u).

A natural way to estimate γ(u) is to estimate c(u,0), c(u,1) by applying the above cross validation method and afterwards calculating γ̂(u) from the corresponding estimators ĉ(u,0), ĉ(u,1). It is obvious that this may not lead to a good result if the nature of c(u,0) and c(u,1) is very different from γ(u) itself. Here, we give a brief sketch how to generalize our method to this case.

Generally speaking, we are interested in estimating F(G(u)) where  F:^d →^d̃ is some given function. The obvious generalization of def_crossval by defining H̃(G) = 1/n∑_t=1^n| F(g(Y_t,n)) - F(G(t/n))|_2^2 w(t/n) is not feasible due to exploding values of F(g(Y_t,n)). Instead, we approximate F(g(Y_t,n)) - F(G(t/n)) by a Taylor expansion of order one and define

    H̃(G)    :=   1/n∑_t=1^n|∂ F(G(t/n))· (g(Y_t,n) - G(t/n))|_2^2 w(t/n)
       =   1/n|g(Y_t,n) - G(t/n)|_F(G(t/n))^t F(G(t/n))^2 w(t/n),

where |x|_A^2 := x^t A x for some vector x and matrix A. We then define, as before,

    ĥ^comp := _h∈ H_nH̃(Ĝ_h^-).

From cv_modified it can be seen that instead of finding a minimizer of ∫_0^1|Ĝ_h(u) - G(u)|_2^2 w(u)  u, we now minimize a weighted combination Ĝ_h(u) - G(u) (weighted by the matrix F(G(u))^t F(G(u))) which introduces the specific nature of the function F. In view of techniques from <cit.>, we conjecture that a similar result as in Theorem <ref> holds for ĥ^comp and a modified distance measure d_M^comp(h):= ∫_0^1|F(G(u)) - F(Ĝ_h(u))|_2^2 w(u)  u.



 Suppose that instead of G(u) =  g(Ỹ_t(u)), we are interested in estimating G_θ(u) :=  g_θ(Ỹ_t(u)) uniformly in Θ, where g_θ:^→^d and θ∈Θ⊂^d̃ with ∫_Θ 1 θ > 0 (which excludes non-unique parametrizations). A prominent example being the characteristic function of X̃_0(u),

    ϕ(u,θ) :=  e^i θX̃_0(u),

cf. <cit.> (i being the imaginary unit). The cross validation procedure from def_crossval can be easily modified to cover such cases by using

    H̅(G) := ∫_ΘH(G_θ) θ,     ĥ^fun := _h∈ H_nH̅(Ĝ_θ,h^-).

Note that here, Ĝ_θ,h^- naturally depends on θ through g_θ. It is straightforward to show that the following modification of Theorem <ref> holds if the conditions therein are fulfilled uniformly in θ: Almost surely,

    lim_n→∞d_M^fun(h)/inf_h∈ H_nd_M^fun(h)=1,

where d_M^fun(h) := ∫_Θ∫_0^1|G(u) - Ĝ_h(u)|_2^2 w(u)  u θ.






 §.§ Simulations


Since our estimators are model-free, we only discuss the behavior of the method if the underlying time series model is a tvAR(1) process given by

    X_t,n = a(t/n)· X_t-1,n + ζ_t,   t = 1,...,n,

where ζ_t are i.i.d. N(0,1) and a(u) = sin(2π u). We have performed similar simulations with tvARCH- and tvMA models leading to similar results. We discuss the following three quantities:

    
  * c(u,1) = X̃_0(u) X̃_1(u),
    
  * γ(u) = c(u,1)/c(u,0) (cf. Remark <ref>),
    
  * ϕ(u,θ) = cos(θ u) (cf. Remark <ref>, we restrict to the real part for simplicity)

by using the introduced cross validation methods. In all simulations, we use w(·) = _[0.05,0.95](·) and the Epanechnikov kernel K(x) = 3/2(1-(2x)^2)_[-1/2,1/2](x). We use a time series length of n = 500 of model model_simulations and the set of bandwidths H_n = {k/50, k=1,...,50}.



  §.§.§ Estimation of c(u,1)

We simulate N = 2000 replications and use   n^-α = 0.12. 
We choose ĥ to be the largest local minimum of h ↦ H(Ĝ_h^-).

In Figure <ref> (left) we have plotted a histogram of the bandwidths chosen by our algorithm, together  with the deterministic bandwidth h_opt from hopt_global which minimizes d_M(h) (and is not available in practice). We observe that ĥ concentrates around h_opt with Gaussian shape.
To judge the performance of our procedure, we compare the achieved distances d_M(h) for h ∈{ĥ, h_opt, h^*}, where h^* := _h∈ H_nd_M(h) is the bandwidth which minimizes d_M(h) on the current realization. In Figure <ref> (right), we have visualized the results with a boxplot. We can see that ĥ performs quite well, the values of d_M(ĥ) are comparable to those of d_M(h^*).




More satisfying results could be obtained by choosing α dependent on the realization. For a fixed realization, it may occur due to nonasymptotic behavior that a single fixed α does lead to cross validation functional H(Ĝ_h^-) (cf. def_crossval) which has no local minima. In practice it is therefore recommended to apply the procedure to different α covering the whole interval (0,1) and search for a suitable minimizer to obtain more stable bandwidth choices. In Figure <ref> we have shown the curves h ↦ H(Ĝ_h^-) obtained for different α such that n^-α is ranging from 0.01 to 0.35 in steps of 0.01. From the definition of H(G) or H̃(G) it is obvious that one should look for a local minimum of H(Ĝ_h^-) with respect to h and not for a global minimum (the latter would result in very small values of h such as h=0.02 and therefore to an overfitting of the observations g(Y_t,n)). It can be seen in Figure <ref> that for some α no local minimum is available. The whole collection of functions, however, gives a well-founded indication which h to choose, namely an h close to 
0.08 which corresponds to the smallest local minimum of H(Ĝ_h^-). Smaller α with a reasonable shape of H(Ĝ_h^-) are preferred since then more observations in the direct neighbourhood of u are used to estimate Ĝ_h^-(u). This has lead to the choice n^-α = 0.3 and the resulting local minimum ĥ=0.08 (from a practical point of view, one may also consider to choose the second local minimum h ≈ 0.3 where the situation seems to be more stable. This leads to similar results as above).








  §.§.§ Estimation of γ(u)

Writing G(u) = (c(u,0),c(u,1)) and F(x,y) = y/x, we have γ(u) = c(u,1)/c(u,0) = F(G(u)) and

    H̃(G) = 1/n∑_t=1^nX_t-1,n^2/c(t/n,0)^2(X_t,n - c(t/n,1)/c(t/n,0)X_t-1,n)^2 w(t/n).

We perform N = 2000 replications with n^-α = 0.08. As before, we compare ĥ^comp chosen by cross validation with h_opt^comp which minimizes the integrated mean-quared error d_M^comp(h) and (h^*)^comp = _h∈ H_nd_M^comp(h).
In Figure <ref> (left) we plotted the bandwidths ĥ^comp and h_opt^comp as a vertical red line, in Figure <ref> (right) one can see the boxplots corresponding to d_M^comp(h) for h∈{ĥ^comp, h_opt^comp, (h^*)^comp}. As before, ĥ^comp concentrates around h_opt^comp and the distances d_M^comp(ĥ^comp), d_M^comp((h^*)^comp) are of comparable size.





  §.§.§ Estimation of ϕ(u,θ)

Writing G_θ(u) =  g_θ(X̃_t(u)) with g_θ(x) = cos(θ x), we have

    H̅(G) = ∫_Θ1/n∑_t=1^n|cos(θ X_t,n) - Ĝ_θ,h(u)|_2^2 w(t/n) θ

To perform the simulations, we choose Θ = [-10,10]. Again, we use N = 2000 replications with n^-α = 0.10. We compare ĥ^fun chosen by cross validation with h_opt^fun which minimizes the integrated mean-quared error d_M^fun(h) and (h^*)^fun = _h∈ H_nd_M^fun(h).
In Figure <ref> (left) we plotted the bandwidths ĥ^fun and h_opt^fun as a vertical red line, in Figure <ref> (right) one can see the boxplots corresponding to d_M^fun(h) for h∈{ĥ^fun, h_opt^fun, (h^*)^fun}. Again, the results are satisfying in our opinion.






§ LOCAL MODEL SELECTION: A CONTRAST MINIMIZATION APPROACH
 


The approach presented in the following allows to choose h locally for each u∈[0,1] in the estimator Ĝ_h(u). This enables the procedure to take into account local smoothness changes of the function G(u). The algorithm is based on a contrast minimization approach which was introduced by <cit.> for a different model. In the following we use a slightly modified estimator for G(u), namely

    Ĝ_h^∘(u) := (1/n∑_t=1^nK_h(t/n-u))^-1·Ĝ_h(u)

which corrects for the deviation of the kernel Riemannian sum from its integral ∫ K(x)  x = 1.

The approach is based on a comparison of Ĝ_h(u) - Ĝ_h'(u) with the theoretical variance of Ĝ_h(u). This leads to a procedure which does not need to estimate the bias of Ĝ_h(u) but only its variance and still finds a approximate minimizer of the corresponding mean squared error. As seen in Theorem <ref>, the asymptotic variance of Ĝ_h(u) is connected to the so-called long-run variance of g(Ỹ_t(u)),

    Σ(u) = ∑_k∈(g(Ỹ_0(u)), g(Ỹ_k(u)).

Let Σ̂_n(u) be an estimator of Σ(u). In practice, the requirements on the quality of Σ̂_n(u) are not too strong, however we need that it has the same order of magnitude as Σ(u). Large deviations from the true Σ(u) may lead to instable bandwidth choices. For theoretical derivations, we suppose:

[Assumptions on Σ̂_n(u)]
There exists some constant c_Σ > 0 such that for each u∈[0,1],

	
  * Σ̂_n(u) is consistent such that for every ε > 0, (|Σ̂_n(u) - Σ(u)|_∞ > ε) = O(n^-2).
	
  * |Σ̂_n(u)|_∞ _2 ≤ c_Σ.


A possible choice for Σ̂_n(u) is given by

    Σ̂_n(u) := ∑_k=-r_n^r_nĉ^g_η(u,k),     ĉ_η^g(u,k) := 1/n∑_t=1^nK_η(t/n-u)· g(Y_t,n)g(Y_t+k,n)'

with some 0 ≤ r_n ≤ n, r_n →∞ and η = η_n → 0. We are now able to define the local bandwidth selection procedure as follows:
	Let λ(h) := max{1,√(log(1/h))} and
	
    v̂^2(h,u) := σ_K^2/nh(Σ̂_n(u)).

	Let H_n ⊂ (0,1] be a geometrically decaying grid of bandwidths given by
	
    H_n = {a^-k:k∈_0}∩ [h, 1],

	where h = h_n > 0 is some lower bound specified below.
	For some C^# > 0, define
	
    ĥ(u) :∈sup{h∈ H_n:|Ĝ_h^∘(u) - Ĝ_h'^∘(u)|_2 ≤ C^#·v̂(h',u) λ(h')  for all  h' < h, h'∈ H_n}.



	
	
  * v̂^2(h,u) is approximately the variance of Ĝ_h^∘(u) - Ĝ_h'^∘(u). It is multiplied by a log factor λ(h') to account for local random deviations. As described in <cit.>, the bandwidth lokale_wahl_bandbreite can be seen as the largest bandwidth where Ĝ_h^∘(u) does not deviate significantly from G(u).
	
  * To prove theoretical results, H_n should not allow for too small bandwidths.
	
  * The condition C^#≥ 64 is needed for the theoretical results. However, for many applications C^# should be chosen smaller to avoid too conservative (i.e. too large) choices of ĥ(u).
	


















We now provide the assumptions on the process X̃_0(u) under which the theoretical statements for the local bandwidth selection procedure ĥ(u) holds.

[Moment / Dependence assumptions]
	Let α≥ 0. Let Assumption <ref> hold for all q ≥ 1. Define N_α(q) := Γ(α q + 2). Assume that for all u∈ [0,1],
	
		
  (i) sup_u∈[0,1]X̃_0(u)_q ≤ D · N_α(q),
		
  (ii) sup_u∈[0,1]δ_q^X̃_0(u)(k) ≤ D · N_α(q) ·ρ^k.
	



	
		
  * Assumption <ref>(i) basically asks for a quantification of the growth of the moments X̃_0(u)_q^q in q. It can be easily seen that the condition sup_u∈[0,1]X̃_0(u)_q ≤ D · N_α(q) follows if X̃_0(u) has a Lebesgue density ∼exp(-x^1/α).
		
  * Assumption <ref>(ii) additionally asks the process to have geometrically decaying dependence measure δ_q^X̃_0(u)(k).
	


The assumptions stated above are mainly used to prove a Bernstein inequality which is a key ingredient to prove the consistency of contrast minimization methods. The geometric dependence decay is used to apply a Bernstein inequality from <cit.>, while the moment conditions are necessary to allow for a large set of bandwidths H_n by establishing a simple exponential inequality. The Bernstein inequality is formulated in terms of the process

    G̃_h(u) := 1/n∑_t=1^nK_h(t/n-u)· g(Ỹ_t(t/n))

which is a natural approximation of Ĝ_h(u) itself. The deviation Ĝ_h(u) - G̃_h(u) can be controlled by using the property X_t,n - X̃_t(t/n)_q ≤ Dn^-1 from Assumption <ref>.


	Fix u∈ [0,1]. Assume that g∈(M,χ,C) and Assumption <ref> holds. Then there exist some constants c̃_1,c_4,c_5,c_H > 0 only dependent on M,χ,C,D,ρ,|K|_∞,α such that for all j = 1,...,d, h∈ H_n:
	
    ((nh)|G̃_h(u)_j - G̃_h(u)_j| > γ) ≤ 2exp(-γ^2/32 (nh)^2 v_j^2(h,u)+ c_4 a_n^1/3γ^5/3) + c_5 n^-1/γ,

	with a_n := c̃_1(8log(n))^α M, v_j^2(h,u) := σ_K^2/nhΣ(u)_jj and h = c_H·log(n)^5+2α M in def_bandwidth_grid.




The following theorem states that | Ĝ_ĥ(u)(u) - G(u)|_2^2 is bounded by an universal constant times |Ĝ_h_opt(u)(u) - G(u)|_2^2 up to an additional log-factor (cf. hopt_lokal for h_opt(u)).


	Fix u∈ (0,1). Suppose that Assumptions <ref> and <ref> are fulfilled. 
	
	Suppose further that g ∈(M,χ,C) such that for all j ≥ 0, |χ_j| ≤ C ρ^j and G is twice continuously differentiable around u with  |∂_u^2 G(u)|_2 ≠ 0.
	
	Then there exists some universal constant c = c(a) > 0 and some constant c_H > 0 only dependent on M,χ,C,D,ρ,|K|_∞,α such that for h = c_H·log(n)^5+2α M, C^#≥ 64 and n large enough,
	
    |Ĝ_ĥ(u)^∘(u) - G(u)|_2^2 ≤ c ·(σ_K^2 (Σ(u)))^4/5·(μ_K^2 |∂_u^2 G(u)|_2^2)^1/5·(log(n)/n)^4/5.




    The parameter C^# is a tuning parameter of the procedure. However, from a theoretical point of view it does not depend on unknown quantities like G(u) but is a universal constant and therefore can be chosen based on training data before applying the procedure to real data. In practice, we obtained good results with C^#∈ [0.5,1.0].




 §.§ Simulations


We discuss the behavior of ĥ(u) if the underlying time series model is a tvAR(1) process given by model_simulations with ζ_t  i.i.d. N(0,1) and

    
  * a(u) = 0.6 sin(2π u),
    
  * a(u) = 0.5 · (1 - 2_[0.5,1](u))

In both situations we will estimate

    c(u,1) = X̃_0(u) X̃_1(u)

by using N = 2000 replications of the time series with length n = 2000. In the simulations, we use the Epanechnikov kernel K(x) = 3/2(1-(2x)^2)_[-1/2,1/2](x) and the set of bandwidths H_n with a = 0.9 and h = 0.9^29≈ 0.047 with C^# = 1.0 (for (a)) and C^# = 0.8 (for (b)). We estimate the long run variance Σ(u) by Σ̂_n(u) given in longrun_covariance_estimator with a simple ad-hoc choice of η = 0.35 and r_n = 18.

A typical phenomenon arises if h is chosen too small, i.e. H_n allows for too small bandwidths: Then, ĥ(u) tends to select the smallest possible bandwidth abruptly at random locations u ∈ [0,1]. The theoretical reason being that in such cases h does not fulfill h≥ c_H log(n)^5+2α M/n which is needed to discuss large deviations of Ĝ_h(u) in the Bernstein inequality. In practice, one may apply the selection procedure ĥ(u) for all u∈[0,1] and for different h, choosing the smallest h where no abrupt outliers occur.

In Figure <ref> (left) we depicted the typical behavior of ĥ(u) (red) for the two scenarios (a),(b) for a single realization. Moreover, ĥ(u) itself is drawn with a green line. The rough shape of G_ĥ(u)(u) is due to the  stepwise form of ĥ(u) ∈ H_n.
Generally speaking it can be seen that ĥ(u) gets smaller if a too large bandwidth would introduce an unnecessary bias (see the peaks in (a) or the step in (b)). ĥ(u) gets large if the variance of the procedure can be significantly reduced. In (b), this is the case on the boundaries where a nearly constant function has to be estimated and in the exact middle at u = 0.5 where the target value 0 is estimated best by taking the mean over all observations. In (a) this is the case at points u where G(u) has nearly linear shape (turning points). A typical drawback of the local selection procedure ĥ(u) is that it may be more sensitive to strong local dependencies. In (b) one can see that around u ≈ 0.8, ĥ(u) is much smaller than the bandwidths chosen around it which is due to a strong peak of the process which is due to a violation of the asymptotic statements. 
In Figure <ref> (right) the corresponding covariance estimator Σ̂_n(u) is drawn against the true long run variance Σ(u). It can be seen that the method works quite satisfying even if Σ(u) is not estimated well; however it is important that the order of magnitudes coincide.
Comparing G_ĥ(u)(u) (red) and G_h^*(u) (blue), where h^* = _h∈ H_nd_M(h), it can be seen that in case (b), the local bandwidth selection may outperform the global bandwidth selection because of the significantly changing smoothness properties of G(u).



In the following we choose the weight function w(u) = _[0.05,0.95](u) and compare the integrated distance d_M^local(h(u)) := ∫_0^1 |Ĝ_h(u)(u) - G(u)|_2^2 w(u)  u for the local selector h(u) = ĥ(u) and the global optimal selector h(u) = h^* which is not known in practice. The results are summarized in Figure <ref> via boxplots (right) and 0.05- and 0.95-quantile curves (left). It should be noted that even comparable results of d_M^local(ĥ(u)) with d_M^local(h^*) are worth to be emphasized since ĥ(u) suffers from the typical higher variance of a data-driven selection procedure which is not the case for h^*.
In case (a), we see that the quantile curves of both selection procedures are comparable with a little bit higher variance around the peaks . The global optimal bandwidth clearly outperforms ĥ(u) since G has nearly smoothness properties over the whole time line.

In the case (b), it can be seen from the quantile curves that for u near 0 and 1, the local selector outperforms the global optimal selector since it is able to choose larger bandwidths there, reducing the variance of Ĝ_ĥ(u)(u). For u around 0.5, Ĝ_ĥ(u)(u) manages to mimic the step of the true function G(u) better than the global selector h_opt(u).





§ CONCLUSION


In this paper, we have developed two methods for bandwidth selection for nonparametric moment estimators of locally stationary processes of some curve G(u). We have derived theoretical results for their optimality with respect to mean-squared-error type distance measures and found with simulations that they work quite well in practice. The first method is  based on a cross validation approach and allows for global bandwidth selection. A critical issue is to deal with the dependency of the observed time series which is controlled by some  quantity α. We have given theoretical and heuristical motivations how to choose this value to obtain a stable procedure. It should be emphasized that this choice is rather  straightforward and data-dependent and therefore should not be considered as a tuning parameter. In a series of remarks, we have shown how to generalize our method to more general settings, for instance if G(u) is a composition of moment functions.

The second method is for local bandwidth selection, i.e. for each time point a different bandwidth is chosen. This allows for taking into account local smoothness properties of the unknown curve. The method needs an estimator Σ̂_n(u) of the asymptotic long-run variance and is dependent on a parameter C^#. We have seen that the quality of Σ̂_n(u) does not influence the bandwidth selection procedure very much, while the choice of C^# however is crucial to obtain meaningful results. In our theoretical results we have shown that C^# is some universal constant which does not need to depend on the unknown quantities to be estimated. However, the conditions stated in theory may lead to too conservative estimates (i.e. the variance is reduced too strongly by introducing a large bias). In our simulations we have given ad hoc choices of C^# which work fairly well, but in more specific applications it may be necessary to adjust C^# further. We have seen in simulations that the local bandwidth selection procedure may outperform global bandwidth selection procedures. Finally, it should be noted that the presented local bandwidth selection procedure reduces the problem of choosing several  tuning parameters separately (all the local bandwidths h(u), u∈[0,1]) to a proper choice of C^#.

In practice, we propose to first use the cross validation method for a first guess ĥ of the global optimal bandwidth. Afterwards one may  apply the local selection procedure, using ĥ (or the estimator of G(u) based on ĥ, respectively) to calibrate C^#.

One may try to improve the theory for the presented methods, allowing for more general structures of G(u) and its estimators or investigating G(u) with moments of two sided functions. These questions are left to further research.




§ APPENDIX



During the proofs, c' > 0 is a generic constant which may depend on

    
  * M,χ,C (from g ∈(M,χ,C)), 
    
  * D,α,ρ (from Assumption <ref> and Assumption <ref>),
    
  * K ∈ (the kernel).

Its value may change from line to line. We define

    G̃_h(u) := 1/n∑_t=1^nK_h(t/n-u)· g(Ỹ_t(t/n))

and G̃_h^∘(u) := (1/n∑_t=1^nK_h(t/n-u))^-1G̃_h(u).

 (i) By Lemma <ref>(ii), we have in each component that













    sup_u∈[0,1]Ĝ_h(u) - 1/n∑_t=1^nK_h(t/n-u)· g(Ỹ_t(t/n))_1 ≤ c' (nh)^-1.

Since u ↦X̃_t(u) is twice differentiable,  the same holds for each component of u ↦ g(Ỹ_t(u)) by using the chain rule. We conclude that

    g(Ỹ_t(u'))    =    g(Ỹ_t(u)) + (u' - u)·∂_u g(Ỹ_t(u)) + 1/2(u'-u)^2 ·∂_u^2 g(Ỹ_t(u))
             + ∫_u^u'(s-u) ·{∂_u^2 g(Ỹ_t(s)) - ∂_u^2 g(Ỹ_t(u))} s,

and

    [1/n∑_t=1^nK_h(t/n-u)· g(Ỹ_t(t/n))] - G(u)
       =   (1/n∑_t=1^nK_h(t/n-u)· g(Ỹ_t(u)) - G(u))
               + 1/n∑_t=1^nK_h(t/n-u)· (t/n-u)·∂_u g(Ỹ_t(u))
               + 1/2n∑_t=1^nK_h(t/n-u)· (t/n-u)^2·∂_u^2 g(Ỹ_t(u))
               + 1/n∑_t=1^nK_h(t/n-u)∫_u^t/n(s-u)[∂_u^2 g(Ỹ_t(s)) - ∂_u^2 g(Ỹ_t(u))]  s.

Due to Lipschitz continuity and symmetry of K, the first two terms in theorem_bias_eq1 are ≤const. (nh)^-1. The third term in theorem_bias_eq1 is h^2/2μ_K ·∂_u^2 G(u) + O((nh)^-1).

Due to Assumption <ref>, it can be seen with Hoelder's inequality that 
    sup_u∈[0,1]|∂_u^2 g(Ỹ_t(u))| _1 < ∞.

The dominated convergence theorem yields that the last term in theorem_bias_eq1 is o(h^2), giving the final result.

(ii) By Lemma <ref>(ii), we have in each component that

    |Ĝ_h(u) - G(u)_2 - G̃_h(u) - G(u)_2| ≤Ĝ_h(u) - G̃_h(u)_2 ≤ c' (nh)^-1.

We furthermore have

    |G̃_h(u) - G(u)|_2^2 = |G̃_h(u) - G̃_h(u)|_2^2 + |G̃_h(u) - G(u)|_2^2.

Similar as in (i), we obtain

    G̃_h(u) - G(u) = h^2/2μ_K ∂_u^2 G(u) + O((nh)^-1) + o(h^2).

By Lemma <ref>, we have

    |G̃_h(u) - G̃_h(u)|_2^2
        = ∑_j=1^d(G̃_h(u)_j)
       =   1/nhσ_K^2 ∑_j=1^d∑_k∈(g_j(Ỹ_0(u)), g_j(Ỹ_k(u))) + O((nh)^-2).

Inserting theorem_bias_eq4 and theorem_bias_eq3 into theorem_bias_eq2, we obtain the result.





 §.§ Proof of Theorem <ref>




	We use the proof techniques from <cit.>. Choosing ℓ(x,y,θ) = 1/2(g(x,y) - θ)^2, we find that θ↦ L(u,θ) := ℓ(X̃_0(u),Ỹ_-1(u),θ) is uniquely minimized by G(u) =  g(Ỹ_0(u)).

	Assumptions 3.1, 3.3, 3.4 and 3.7 easily follow from the assumptions stated at the beginning (weight function w..., set of bandwidths H_n, G(u) twice continuously diff, conditions on g, twice cont. ...). It can be easily seen that Assumption 3.7(3) therein is only needed in the version we ask for on u ↦∂_u^2 g(Ỹ_0(u)).

	The only condition which is not fulfilled is Assumption 3.3(4) therein, which asks
	
    ∇_θℓ(Ỹ_0(u),θ)|_θ = G(u) = G(u) - g(Ỹ_0(u))

	to be an uncorrelated sequence. We now follow the main steps of the proof of Theorem 3.6 in <cit.> and emphasize the steps where the condition of uncorrelatedness has to be circumvented.

	By (18) in <cit.>, we have uniformly in h∈ H_n,
	
    d_M(h)    :=   ∫_0^1|Ĝ_h(u) - G(u)|_2^2 w(u)  u
       =   μ_K/nh + h^4/4d_K^2 ∫_0^1|∂_u^2 G(u)|_2^2 w(u)  u + o((nh)^-1) + o(h^4).

	Using the same arguments as in <cit.>, equation (26) therein, we have
	
    sup_h∈ H_n|d_A(h) - d_M(h)/d_M(h)| → 0   a.s.

	Define
	
    d_A,-(h) := 1/n∑_t=1^n|Ĝ_h^-(t/n) - G(t/n)|_2^2 w(t/n).

	Then we have 
	
    |d_A(h) - d_A,-(h)|    ≤   1/n∑_t=1^n|Ĝ_h(t/n) - Ĝ_h^-(t/n)|_2^2 w(t/n)
                 + |2/n∑_t=1^n⟨ G(t/n) - Ĝ_h(t/n), Ĝ_h(t/n) - Ĝ_h^-(t/n)⟩ w(t/n)|
       ≤    W_n,h + 2 W_n,h^1/2d_A(h)^1/2,

	where with k_n,h(t) := 1/n∑_s=1^nK_h^(n)((s-t)/n),
	
    W_n,h = 1/n^3∑_t=1^n(∑_s=1^n|g(Y_s,n)|_1 {K_h-  K_h^(n)/k_n,h(t)}((s-t)/n))^2w(t/n)^2.

	Note that W_n,h is Lipschitz-continuous in the sense that
	
    |W_n,h - W_n,h'| ≤ C(n)· |h-h'|

	with some polynomial C(n) in n. This allows us to use chaining arguments to prove uniform convergences in h. In the following we use the decomposition
	
    W_n,h≤ 4(W_n,h,1 + W_n,h,2 + W_n,h,3),

	where with c(s_1,s_2) := ∑_t=1^n{K_h-  K_h^(n)/k_n,h(s_1)}((s_1-t)/n)·{K_h-  K_h^(n)/k_n,h(s_2)}((s_2-t)/n) w(t/n)^2 and _0(Z) = Z -  Z,
	
    W_n,h,1   :=   1/n^3∑_t=1^n(∑_s=1^n |g(Ỹ_s(s/n))|_1 {K_h-  K_h^(n)/k_n,h(t)}((s-t)/n))^2 w(t/n)^2,
    
    		W_n,h,2   :=   1/n^3∑_s_1,s_2=1^n_0 |g(Ỹ_s_1(s_1/n))|_1 ·_0 |g(Ỹ_s_2(s_2/n))|_1 · c(s_1,s_2) ,
    
    		W_n,h,3   :=   1/n^3∑_t=1^n(∑_s=1^n|g(Y_s,n) - g(Ỹ_s(s/n))|_1 {K_h-  K_h^(n)/k_n,h(t)}((s-t)/n))^2 w(t/n)^2.

	Since for all q ≥ 1, sup_h∈ H_n(nh)W_n,h,3_q ≤ O(h), it is easy to see by a chaining argument that sup_h ∈ H_n(nh)|W_n,h,3| → 0 a.s.

	Note that
	
    k_n,h(t) - 1 = O((nh)^-1 + n^-α),

	and for t ∈{1,...,n} with w(t/n) ≠ 0,
	
    ∑_s=1^n|K_h((s-t)/n) - K_h^(n)((s-t)/n/k_n,h(t)| = O(n· (n^-α + (nh)^-1)).

	By Lemma <ref>, we have sup_s,n|g(Ỹ_s(s/n))|_1 =O(1), thus with some constant c' > 0,
	
    |W_n,h,1| ≤ (c')^2 |K|_∞^2 |w|_∞^2 (n^-2α + (nh)^-2).

	Therefore we obtain sup_h∈ H_n(nh)|W_n,h,2| → 0 if (nh)· n^-2α→ 0.

	Using Lemma 8.1(ii) from the Supplementary material of <cit.>, we have with some constants c',c” > 0,
	
    W_n,h,2   ≤    c'|w|_∞^2/n^3sup_k∈∑_1≤ t,t+k≤ n|c(t,t+k)| ≤ c”|w|_∞^2|K|_∞^2/n(nh)^2· n· (n^-α + (nh)^-1)(nh)
       =    c”|w|_∞^2|K|_∞^2 · (n^-α + (nh)^-1)(nh)^-1,

	which shows that sup_h∈ H_n(nh)| W_n,h,2| → 0.

	Finally, by the same Lemma we obtain for all q > 0 with some constant c' = c'(q) > 0, c” = c”(q) > 0:
	
    W_n,h,2 -  W_n,h,2_q    ≤    c' 1/n^3(∑_s_1,s_2=1^nc(s_1,s_2)^2)^1/2
       ≤    c”|K|_∞^2 (n^-α + (nh)^-1)(nh)/n(nh)^2· ( (n^-α + (nh)^-1)(nh)· n)^1/2
       ≤     c” |K|_∞^2 (n^-α + (nh)^-1)^3/2h^1/2/nh,

	showing with a chaining argument that sup_h∈ H_n(nh)|W_n,h,2 -  W_n,h,2| → 0 a.s.
	In total, we have seen that
	
    sup_h∈ H_n(nh)|W_n,h| → 0   a.s.,

	and thus with cvproof3,
	
    sup_h∈ H_n|d_A(h) - d_A,-(h)|/d_A(h)→ 0    a.s.

	We now analyze the difference
	
    2[H(Ĝ_h^-) - H(G)] - d_A,-(h) =  -2/n∑_t=1^n⟨ g(Y_t,n) - G(t/n), Ĝ_h^-(t/n)-G(t/n)⟩ =: S_n,h

	Following the proof of Lemma 3.16 in <cit.>, we show
	
    sup_h∈ H_n|2[H(Ĝ_h^-) - H(G)]-d_A,-(h)/d_M(h)| → 0   a.s.

	Define G̅_h^-(u) := 1/n∑_t=1^n K^(n)_h(t/n-u)· g(Ỹ_t(u)). Following the proof of Lemma 2.5 in <cit.> ((41) and the discussion of R_0,n, R_1,n, R_2,n, R_3,n therein), we obtain that
	
    S̃_n,h :=  -2/n∑_t=1^n⟨ g(Ỹ_t(t/n)) - G(t/n), G̅_h^-(t/n)-G̅_h^-(t/n)⟩

	fulfills sup_h∈ H_n|S_n,h - S̃_n,h/d_M(h)| → 0 a.s. To use the argument for R_3,n therein to obtain cvproof1, we have to verify that
	
    sup_h∈ H_nS̃_n,h/d_M(h)→ 0.

	We now cannot argue with the uncorrelatedness of ∇_θℓ(Ỹ_0(u),θ)|_θ = G(u) = G(u) - g(Ỹ_0(u)). Instead we use a direct calculation of S̃_n,h: It holds that sup_u∈[0,1]δ_2^g(Ỹ(u))(k) ≤ c' k^-κ with some c' > 0 (cf. also lemma_emp_eq1), thus
	
    |S̃_n,h|    ≤   2/n(nh)∑_s,t=1^n|K^(n)((s-t)/(nh)|· |(g(Ỹ_t(t/n)), g(Ỹ_s(t/n)))|
       ≤   2/n(nh)∑_s,t=1^n|K^(n)((s-t)/(nh)|·sup_u∈[0,1]δ_2^g(X̃(u))(|s-t|)
       ≤   4c'|K|_∞/n(nh)∑_t=1^n∑_k=⌊ (1-ε)n^-α(nh)⌋^∞ k^-κ
       =    O((n^-α(nh))^-κ+1/nh)

	which shows cvproof2 if n^-α(nh) →∞.
	
	Using cvproof1, cvproof4 and cvproof5, the result now follows along the same lines as the proof of Theorem 3.6 in <cit.>.




 §.§ Proof of Theorem <ref>



	Define Δ_h(u) := sup_h' ≤ h|G_h'(u) - G(u)|_2. Since h'∈ H_n, for n large enough it holds that
	
    1/n∑_t=1^nK_h'(t/n-u) ≥1/2.

	Furthermore, with some constant c' > 0 only depending on |K|_∞, |G|_∞ and the corresponding Lipschitz constants of K,G:
	
    |1/n∑_t=1^nK_h'(t/n-u){G(t/n) - G(u)} - ∫_0^1K_h'(v-u){G(v) - G(u)} v|_∞≤ c' n^-1.

	Since G is twice continuously differentiable,
	
    ∫_0^1K_h'(v-u){G(v) - G(u)} v = h^2/2μ_K·∂_u^2 G(u) + o(h^2).

	We obtain
	
    Δ_h(u)    =   sup_h'≤ h(1/n∑_t=1^nK_h'(t/n-u))^-1·|1/n∑_t=1^nK_h'(t/n-u){G(t/n) - G(u)}|_2
       ≤    2·h^2/2μ_K· |∂_u^2 G(u)|_2 + o(h^2),

	and thus for h small enough, Δ_h(u) ≤ 2h^2 μ_K· |∂_u^2 G(u)|_2. Define
	
    h_opt,n(u)    :=   _h > 0{log(n)/nhσ_K^2 ·(Σ(u)) + h^4/4μ_K^2· |∂_u^2 G(u)|_2^2}
       =   (4σ_K^2 ·(Σ(u))/μ_K^2· |∂_u^2 G(u)|_2^2)^1/5·(log(n)/n)^1/5.

	By Proposition <ref>, we have
	
    |Ĝ_ĥ(u)^∘(u) - G(u)|_2^2    ≤    c · v^2(h̅,u) λ(h̅)^2 + c∑_h∈ H_nv^2(h_0(u),u) λ(h_0(u))^2_{h_0(u) = a· h}
              + c'·log(n)^2 n^-1.

	For the second summand in upperbound_thm2_eq2 we find, for n large enough, the upper bound
	
    ∑_h∈ H_nv^2(h_0(u),u) λ(h_0(u))^2_{h_0(u) = a· h}
       ≤   ∑_h∈ H_nmin{v^2(ah,u) λ(ah)^2,1/D_1^2Δ_h(u)^2}
       ≤   1/D_1^2∑_h∈ H_n, h ≤ h_opt,n(u)Δ_h(u)^2 + ∑_h∈ H_n, h > h_opt,n(u)v^2(ah,u)λ(ah)^2
       ≤   4/D_1^2(1-a^4)h_opt,n(u)^4μ_K^2|∂_u^2 G(u)|_2^2 + D_2/a(1-a)σ_K^2 (Σ(u))log(n)/n h_opt,n(u),

	which together with upperbound_thm2_eq2 gives the result.




	Suppose that Assumption <ref> holds for all q > 0, and that Assumption <ref> and Assumption <ref> hold. Define G_h(u) := Ĝ_h^∘(u), and
	
    h_0(u) := sup{h∈ H_n:|G_h'(u) - G(u)|_2 ≤C^#/8 v(h,u) λ(h)   for all  h'∈ H_n, h' ≤ h}.

	Suppose that
	
    H_n = {a^-k:k∈_0}∩ [c_H·log(n)^5+2α M, 1]

	with some constant c_H > 0 large enough. Then there exists some universal constant c = c(a) > 0 and some constant c' > 0 (depending on M,χ,C,ρ,D,α,a,K) such that for n large enough,
	
    |Ĝ_ĥ(u)^∘(u) - G(u)|_2^2 ≤ c · v^2(h_0(u),u) λ(h_0(u))^2 + c'·log(n)^2 n^-1.




	We follow the proof strategy of <cit.>. During the proof, we use c' > 0 for a  constant only dependent on M,C,D,α,ρ,K.
	Put
	
    v^2(h,u)    :=   1/nh∫ K(x)^2  x ·tr(Σ(u)),
    
    		v^2(h,h',u)    :=   1/n∫{K_h(x) - K_h'(x)}^2  x ·tr(Σ(u)).

	Define
	
    S(u) := {ĥ(u) ≥ h_0(u)}.

	Then
	
    |Ĝ_ĥ(u)^∘(u) - G(u)|_2^2 = |Ĝ_ĥ(u)^∘(u) - G(u)|_2^2 _S(u) + |Ĝ_ĥ(u)^∘(u) - G(u)|_2^2 _S(u)^c.

	Discussion of the first summand in upperbound_eq1: It holds that
	
    |Ĝ_ĥ(u)^∘(u) - G(u)|_2^2 _S(u)   ≤    2|Ĝ_ĥ(u)^∘(u) - Ĝ_h_0(u)(u)|_2^2 _S(u) + 2 |Ĝ_h_0(u)^∘(u) - G_h_0(u)(u)|_2^2
              + 2|G_h_0(u)(u) - G(u)|_2^2.

	By Lemma <ref> and Lemma <ref> and since 1/n∑_t=1^nK_h(t/n-u) ≥1/2 for n large enough, we have
	
    |Ĝ_h_0(u)^∘(u) - G_h_0(u)(u)|_2^2    ≤    8 |Ĝ_h_0(u)(u) - G̃_h_0(u)(u)|_2^2 + 8|G̃_h_0(u)(u) - G_h_0(u)(u)|_2^2
       ≤    8v^2(h_0(u)) + c · n^-1.

	By definition of ĥ(u) and monotonicity of λ(·),
	
    |Ĝ_ĥ(u)^∘(u) - Ĝ_h_0(u)^∘(u)|_2^2 _S(u)≤ (C^#)^2[v̂^2(h_0(u),u)]λ(h_0(u))^2.

	We now discuss [v̂^2(h_0(u),u)]. It holds that
	
    [v̂^2(h_0(u),u)_{(Σ̂_n(u)) > 2 (Σ(u))}]    ≤   v̂^2(h_0(u),u)_2 ·(|(Σ̂_n(u)) - (Σ(u))| > (Σ(u)))^1/2
       ≤    v^2(h_0(u),u)

	for n large enough due to Assumption <ref>. We therefore have
	
    v̂^2(h_0(u),u) ≤ 2v^2(h_0(u),u) + [v̂^2(h_0(u),u)_{(Σ̂_n(u)) > 2 (Σ(u))}] ≤ 3 v^2(h_0(u),u).

	Using  upperbound_eq3 and upperbound_eq4, we obtain
	
    |Ĝ_ĥ(u)^∘(u) - Ĝ_h_0(u)^∘(u)|_2^2_S(u)≤ 3(C^#)^2v^2(h_0(u),u)λ(h_0(u))^2.

	By definition of h_0(u), we have
	
    |G_h_0(u)(u) - G(u)|_2 ≤C^#/8v(h_0(u),u)λ(h_0(u)).

	Inserting upperbound_eq2_5, upperbound_eq5 and upperbound_eq6 into upperbound_eq2, we obtain
	
    |Ĝ_ĥ(u)^∘(u) - G(u)|_2^2 _S(u)≤[8 + (C^#/8)^2 + 3(C^#)^2] v^2(h_0(u),u)λ(h_0(u))^2 + c'· n^-1.

	Discussion of the second summand in upperbound_eq1: Let H_n(h) := {h' ∈ H_n: h' < h}. By definition of h_0(u) and by monotonicity of v(·), λ(·), we obtain for h' ≤ h ≤ h_0(u):
	
    |G_h'(u) - G(u)|_2 ≤C^#/8v(h_0(u),u)λ(h_0(u)) ≤C^#/8v(h,u)λ(h).

	Decompose
	
    S(u)^c = ⋃_h∈ H_n(a· h_0(u))⋃_h'∈ H_n(h)E(h,h',u),     E(h,h',u) := {|Ĝ_h^∘(u) - Ĝ_h'^∘(u)|_2 > C^#v̂(h',u)λ(h')}.

	Let
	
    A_1 := {tr(Σ̂_n(u)) ≥1/2tr(Σ(u))},

	and define
	
    N(h,h',u)    :=    (Ĝ_h^∘(u) - G_h(u)) - (Ĝ_h'^∘(u) - G_h'(u)),
    Ñ(h,h',u)    :=    (G̃_h^∘(u) - G_h(u) - (G̃_h'^∘(u) - G_h'(u)),
    
    		N(h,u)    :=   Ĝ_h^∘(u) - G_h(u),
    Ñ(h,u)    :=   G̃_h^∘(u) - G_h(u).

	We have
	
    E(h,h',u) ∩ A_1    ⊂   {|Ĝ_h^∘(u) - Ĝ_h'^∘(u)|_2 > C^#/2v(h',u)λ(h')}
       ⊂   {2C^#/8v(h',u)λ(h') + |N(h,h',u)|_2 > C^#/2v(h',u)λ(h')}
       ⊂   {|N(h,h',u)|_2 > C^#(1/2 - 2/8)v(h',u)λ(h')} =: E_0(h,h',u).

	We conclude with the Cauchy-Schwarz inequality, upperbound_eq10 and Assumption <ref>:
	
    |Ĝ_ĥ(u)^∘(u) - G(u)|_2^2 _S(u)^c
       ≤   ∑_h∈ H_n(a h_0(u))∑_h' ∈ H_n(h)|Ĝ_h^∘(u) - G(u)|_2^2 _E(h,h',u)
       ≤   ∑_h∈ H_n(a h_0(u))∑_h' ∈ H_n(h)|Ĝ_h^∘(u) - G(u)|_2^2 _E_0(h,h',u) + c' log(n)^2 n^-1
       ≤   ∑_h∈ H_n(a h_0(u))∑_h' ∈ H_n(h)[ (|N(h,u)|_2 + C^#/8v(h,u)λ(h_0(u)))^2] _E_0(h,h',u) + c' log(n)^2 n^-1.

	With Lemma <ref>, we can replace |N(h,u)|_2 with |Ñ(h,u)|_2 with  error O(log(n)^2 n^-1) due to
	
    [ (|N(h,u)|_2 - |Ñ(h,u)|_2)^2] ≤ | N(h,u) - Ñ(h,u)_2 |_2^2 ≤ c'n^-2.

	Similarly, the set A(h,h') := {| |N(h,h',u)|_2 - |Ñ(h,h',u)|_2| ≤C^#/2(1/2-2/8)v(h_0(u),u)λ(h_0(u))} has the property
	
    (A(h,h')) ≤c' n^-4/(v(h_0(u),u)λ(h_0(u)))^4 =O(n^-2),

	allowing to replace |N(h,h',u)|_2 by |Ñ(h,h',u)|_2 in E_0(h,h',u) with replacement error O(log(n)^2 n^-1). Together with v(h,h',u) ≤ v(h',u), we have shown that
	
    Ẽ_0(h,h',u) := {|Ñ(h,h',u)|_2 > C^#/2(1/2 - 2/8)v(h,h',u)λ(h')},

	fulfills
	
    |Ĝ_ĥ(u)(u) - G(u)|_2^2 _S(u)^c   ≤   ∑_h∈ H_n(a h_0(u))∑_h' ∈ H_n(h)[|Ñ(h,u)|_2 + C^#/8v(h,u)λ(h_0(u))]^2 _Ẽ_0(h,h',u)
                               + c' log(n)^2 n^-1.

	
	
	Put x := C^#/8λ(h_0(u)) and D^# := C^#/2(1/2-2/8). We now discuss the summands in upperbound_eq12. It holds that
	
    [x + v(h,u)^-1|Ñ(h,u)|_2]^2 _{|Ñ(h,h',u)|_2 > D^#v(h,h',u)λ(h')}
       =    2∫_0^∞z·(v(h,u)^-1|Ñ(h,u)|_2 > z-x,|Ñ(h,h',u)|_2 > D^#v(h,h',u)λ(h'))  z
       ≤    2 ∫_0^x + √(2^7log(n))z·(|Ñ(h,h',u)|_2 > D^#v(h,h',u)λ(h'))  z
               + 2 ∫_x+√(2^7log(n))^∞z·(v(h)^-1|Ñ(h,u)|_2 > z-x)  z
       =    (x+√(2^7log(n)))^2 (|Ñ(h,h',u)|_2 > D^#v(h,h',u)λ(h'))
               + 2∫_√(2^7log(n))^∞(x+y)(|Ñ(h,u)|_2 > v(h,u) y)  y.

	Discussion of the first summand in upperbound_eq13. Put v_j^2(h,u) := 1/nh∫ K(x)^2  x ·Σ(u)_jj and v_j^2(h,h',u) := 1/n∫{K_h(x) - K_h'(x)}^2  x ·Σ(u)_jj. Then, with Lemma <ref>,
	
    (|Ñ(h,h',u)|_2 > D^#v(h,h',u)λ(h'))
       ≤   (∑_j=1^dÑ(h,h',u)_j^2 > (D^#)^2∑_j=1^dv_j^2(h,h',u)λ(h')^2)
       ≤   ∑_j=1^d((nh')|Ñ(h,h',u)_j|  > D^#(nh')v_j(h,h',u)λ(h'))
       ≤    d·sup_j=1,...,d{2exp(-(D^#)^2λ(h')^2/32 + c_4(D^#a_n/(nh')v_j(h,h',u))^1/3λ(h')^5/3(D^#)^5/3) + c_5n^-1/D^#(nh')v_j(h,h',u)λ(h')}.

By Lemma <ref>(ii), c_4(a_n/(nh')v_j(h,h',u))^1/3λ(h')^5/3(D^#)^5/3≤ 32 is fulfilled if

    (32/c_4)^3λ(h')^-5· (nh')v_j(h,h',u)    ≥   (32/c_4)^3 1/D_2^5/2·[c_6 (a-1)^2 inf_j=1,...,d(Σ(u)_jj)]^1/2· (nh'/log(n)^5)^1/2
       ≥    2a_n = 2c̃_1 · 8^1/τ_2log(n)^1/τ_2,

i.e. if h' ≥ c' ·log(n)^5+2/τ_2· n^-1 for c' > 0 large enough (which is fulfilled due to h'∈ H_n).
We obtain that for h∈ H_n, h' ∈ H_n(h) with D^# = 8:

    (|Ñ(h,h',u)|_2 > v(h,h',u)λ(h')) ≤ 2d·h'/h̅ +  c_5/D^#c_6^1/2 |a-1|inf_j=1,...,d(Σ(u)_jj)^1/2· n^-1.

	Discussion of the second summand in upperbound_eq13. We have by Lemma <ref> that
	
    (|Ñ(h,u)|_2 > v(h,u)y)
       ≤   ∑_j=1^d((nh)|Ñ(h,u)_j| > (nh)v_j(h,u)y)
       ≤    d sup_j=1,...,d{2exp(-y^2/32 + c_4 (a_n/(nh)v_j(h,u))^1/3 y^5/3) + c_5 n^-1/(nh)v_j(h,u)y}.

	We have with c_n := a_n/(nh)v_j(h,u):
	
    exp(-y^2/32 + c_4 (a_n/(nh)v_j(h,u))^1/3 y^5/3) ≤exp(-y^2/64),     y ≤ (32/c_4)^3/5c_n^-1/5 =: d_n,
    exp(-1/2c_4(y/c_n)^1/3),     y > d_n

	Thus
	
    ∫_√(2^7log(n))^∞(x+y)(|Ñ(h,u)|_2 > v(h,u) y)  y
       ≤    4dsup_j[∫_√(2^7log(n))^d_n(x+y)exp(-y^2/2^6)  y + ∫_d_n^∞(x+y) exp(-1/2c_4(y/c_n)^1/3)  y
                       + c_5/σ_K·inf_j=1,...,d(Σ(u)_jj)^1/2 n^-1].

	We now discuss the first two summands in upperbound_eq15. We have
	
    ∫_√(2^7log(n))^d_n(x+y)exp(-y^2/2^6)  y ≤ [x+d_n] n^-2,

	and, with some constant c̃_4 > 0 only depending on c_4,
	
    ∫_d_n^∞(x+y)exp(-1/2c_4(y/c_n)^1/3)  y
       =    c_n ∫_d_n/c_n^∞(x+c_n z)exp(-1/2c_4z^1/3)  z
       ≤   c̃_4 · c_n·exp(-1/2c_4(d_n/c_n)^1/3)·[x·((d_n/c_n)^2/3+1)) + c_n((d_n/c_n)^5/3 + 1)].

	Here,
	
    1/2c_4(d_n/c_n)^1/3 = 1/2(16/c_4^6)^1/5c_n^-2/5 =  1/2[16∫ K(x)^2  x ·Σ(u)_jj/c_4^6·nh/a_n^2]^1/5.

	We conclude that upperbound_eq16 is ≥ 2log(n) if h ≥ c' ·log(n)^5+2/τ_2· n^-1 for c' > 0 large enough (which is fulfilled due to h∈ H_n). Clearly, upperbound_eq16 is ≤ O(n^1/5). Summarizing these results into upperbound_eq15, we obtain for all h ∈ H_n that
	
    ∫_√(2^7log(n))^∞(x+y)(|Ñ(h,u)|_2 > v(h,u) y)  y ≤ c' n^-1

	with some constant c' > 0.
	
	Inserting upperbound_eq20 and upperbound_eq21 into upperbound_eq13 and upperbound_eq12, we obtain with ∑_h'∈ H_n(h)h' ≤h/1-a:
	
    |Ĝ_ĥ(u)(u) - G(u)|_2^2 _S(u)^c
       ≤   2d/h̅·∑_h∈ H_n(a h_0(u))∑_h'∈ H_n(h)v^2(h,u)· h' + c'log(n)^2 n^-1
       ≤   ∫ K^2(x)  x ·(Σ(u))/1-a·2d/h̅·∑_h∈ H_n(a h_0(u)) n^-1 + c'log(n)^2 n^-1
       ≤    c' ·log(n)^2 n^-1.

	By upperbound_eq24 and upperbound_eq23, the result follows.



Assume that g ∈(M,χ,C). Suppose that Assumption <ref> holds. Put τ_2 = (α M)^-1. Then there exist constants c̃_1,c̃_2 > 0 only depending on M,χ,C,D such that for q ≥ 2:
	
    g(Ỹ_t(u))_q    ≤   c̃_1 N_α(qM)^M,
    exp[1/2(|g(Ỹ_t(u))|/c̃_1)^τ_2]    ≤   c̃_2,


 (i) It holds that
	
    g(Ỹ_t(u)) - g(0)_q    ≤    C ∑_j=1^∞χ_jX̃_t-j+1(u)_qM·(1 + |χ|_1^M-1X̃_t(u)_qM^M-1)
       ≤    C |χ|_1 D N_α(qM) ·( 1+ |χ|_1^M-1D^M-1 N_α(qM)^M-1).

	Since |g(0)| ≤ C, we obtain g(Ỹ_t(u))_q ≤c̃_1 N_α(qM)^M with some c̃_1 only depending on M,χ,C,D.
	
	(ii) Define λ = (2 c̃_1^τ_2)^-1. By a series expansion of exp, we have
	
    exp(λ |g(Ỹ_t(u))|^τ_2) = ∑_q=0^∞λ^q g(Ỹ_t(u))_τ_2 q^τ_2 q/q!.

	If τ_2 q ≥ 2, we have
	
    g(Ỹ_t(u)) _τ_2 q^τ_2 q≤c̃_1^τ_2 q·Γ(α q τ_2 M+2) = c̃_1^τ_2 qΓ(q + 2).

	This shows ∑_τ_2 q ≥ 2λ^q  g(Ỹ_t(u)) _τ_2 q^τ_2 q/q!≤∑_τ_2 q ≥ 0^∞(λc̃_1^τ_2)^q·Γ(q+2)/Γ(q+1) = ∑_τ_2 q ≥ 2q+1/2^q≤ 4.

	In the case τ_2 q < 2, we have
	
    g(Ỹ_t(u)) _τ_2 q^τ_2 q≤ g(Ỹ_t(u)) _2^τ_2 q≤c̃_1^τ_2 qΓ(2α M+2)^τ_2q/2≤c̃_1^τ_2 qΓ(2α M + 2).

	This shows ∑_τ_2 q < 2λ^q  g(Ỹ_t(u)) _τ_2 q^τ_2 q/q!≤Γ(2α M+2)∑_q=0^∞2^-q/q! = exp(2^-1)Γ(2α M + 2). The result is obtained with c̃_2 := 4 + exp(2^-1)Γ(2α M + 2).



	Assume that ϕ:[0,1] → is some measurable function, and g∈(M,χ,C). Suppose that Assumption <ref> holds. Define
	
    F_n(ϕ,g) := 1/n∑_t=1^nϕ(t/n)·{g(Ỹ_t(t/n)) -  g(Ỹ_t(t/n))}.

	Put τ = τ(α,M) := (1/2+α M)^-1. Then there exist constants c_1,c_2 > 0 only depending on M,χ,C,D such that
	
		
  (i) 
    F_n(ϕ,g)_q ≤ c_1(q-1)^1/2n^-1/2(1/n∑_t=1^nϕ(t/n)^2)^1/2· N_α(qM)^M,

		
  (ii) 
    (|F_n(ϕ,g)| > γ) ≤ c_2exp[-1/4e(√(n)·γ/c_1 (1/n∑_t=1^nϕ(t/n)^2)^1/2)^τ].

	

 
	 (i) Let δ(k) := Dρ^k. By Hoelder's inequality, we have with some constant c̃ only dependent on M,χ,C,D:

    g(Ỹ_t(u)) - g(Ỹ_t^*(u))_q
       ≤    C ∑_j=1^tχ_j X̃_t-j+1(u) - X̃_t-j+1^*(u)_qM·(1 + 2 |χ|_1^M-1D^M-1· N_α(qM)^M-1)
       ≤   c̃∑_j=1^tχ_j δ(t-j+1) · N_α(qM)^M.

Let ξ(t) := ∑_j=1^tχ_j·δ(t-j+1). Obviously, ∑_t=1^∞ξ(t) = ∑_j=1^∞χ_j∑_t=j^∞δ(t-j+1) < |χ|_1 |δ(·)|_1. We have shown that the dependence measure fulfills δ^g(Ỹ(u))_q(k) ≤c̃·ξ(k)· N_α(qM)^M and is absolutely summable.

By Theorem 2.1 from <cit.> for q > 2 (and for q = 2 directly by calculating the variance of the following term), we have

    F_n(ϕ,g) _q    ≤   1/n∑_t=1^nϕ(t/n){g(Ỹ_t(t/n)) -  g(Ỹ_t(t/n))}_q
       ≤   1/n∑_k=0^∞∑_t=1^nϕ(t/n) P_t-kg(Ỹ_t(t/n))_q
       ≤   1/n∑_k=0^∞(q-1)^1/2|(∑_t=1^nϕ(t/n)^2 P_t-kg(Ỹ_t(t/n))_q^2)^1/2|_2
       ≤    (q-1)^1/2 n^-1/2(1/n∑_t=1^nϕ(t/n)^2)^1/2·c̃∑_k=0^∞ξ(k)· N_α(qM)^M.

(ii) Define Z_n := c̃n^-1/2(1/n∑_t=1^nϕ(t/n)^2)^1/2·∑_k=0^∞ξ(k). By Stirling's formula, we have for all x ≥ 1:

    √(2π)x^x-1/2e^-x≤Γ(x) ≤ e^1/12·√(2π)x^x-1/2e^-x.

By Markov's inequality, we have for γ,λ > 0:

    (|F_n(ϕ,g)| ≥γ) ≤ e^-λγ^τ[e^λ |F_n(ϕ,g)|^τ] = e^-λγ^τ∑_q=0^∞λ^q |F_n(ϕ,g)|_2_τ q^τ q/q!.

In the case τ q ≥ 2, we have 

    λ^q F_n(ϕ,g)_τ q^τ q/q!≤λ^q/Γ(q+1)(τ q)^τ q/2D(u)^τ q·Γ(α M τ q + 2).

Note that α M τ≤ 1 and τ(α M + 1/2) = 1, thus

    q^τ q/2Γ(α M τ q + 2)/Γ(q+1)   ≤    (q+2)^τ q/2·(α M τ q + 2)^α M τ q + 3/2 e^-(α M τ q+2) e^1/12/ (q+1)^q+1/2 e^-(q+1)
       =    e^1/12(q+2)·(q+2/q+1)^q+1/2 e^-1 e^q(1-α M τ)
       ≤    e^1/12 (q+2) e^q.

Define λ := (4e)^-1 Z_n^-τ. Since τ≤ 2, it holds that τ^τ/2≤ 2. Thus

    ∑_q ≥ 2/τλ^q F_n(ϕ,g)_τ q^τ q/q!≤ e^1/12·∑_q ≥ 2/τ(q+2)(λ· 2e Z_n^τ)^q ≤ e^1/12∑_q ≥ 2/τq+2/2^q≤ 4e^1/12.

In the case τ q < 2, we have

    λ^q F_n(ϕ,g)_τ q^τ q/q!   ≤   λ^q |E_n,b(g,u)|_2_2^τ q/q!≤λ^q/q!Z_n^τ q·Γ(2α M + 2)^τ q/2
       ≤   (4e)^-q/q!·Γ(2α M+2),

thus ∑_q < 2/τλ^q F_n(ϕ,g)_τ q^τ q/q!≤exp((4e)^-1)Γ(2α M + 2). So the result is obtained with c_2 := 4e^1/2 + exp((4e)^-1)Γ(2α M + 2) and c_1 = c̃∑_k=0^∞ξ(k) = c̃ |χ|_1 |δ(·)|_1.



	Assume that g∈(M,χ,C) and that Assumption <ref> holds. Let ϕ:[0,1] → be a measurable function.
	Define
	
    W_n := ∑_t=1^nϕ(t/n){g(Ỹ_t(t/n)) -  g(Ỹ_t(t/n))}.

	Assume that s_n := #{t∈{1,...,n}: ϕ(t/n) ≠ 0} fulfills
	
    (W_n) ≥const.(M,χ,C,D,ρ,ϕ)· s_n.

	Then there exist some constants c_4,c_5 > 0 only dependent on M,χ,C,D,ρ,|ϕ|_∞ such that:
	
    (|∑_t=1^nϕ(t/n) {g(Ỹ_t(t/n)) -  g(Ỹ_t(t/n))}| > γ) ≤ 2exp(-γ^2/16 (W_n) + c_4 a_n^1/3γ^5/3) + c_5 n^-1/γ.

		
	with a_n := c̃_1(8log(n))^1/τ_2 (c̃_1, τ_2 from Lemma <ref>).
	
		
		
	

 (i) Step 1: Truncation. Define W_n^∘ := ∑_t=1^nZ_t^∘, where Z_t^∘ = Z_t^∘(t/n), Ψ(x) = x_{|x| ≤ a_n} + a_n _{|x| > a_n},

    Z_t^∘(u) := ϕ(u)· [Ψ(g(Ỹ_t(u))) - Ψ(g(Ỹ_t(u)))].

We have

    W_n - W_n^∘_q    ≤    2∑_t=1^n|ϕ(t/n)|·g(Ỹ_t(t/n))_{|g(Ỹ_t(t/n))| > a_n}_q
       ≤    2|ϕ|_∞n·sup_u∈[0,1]g(Ỹ_0(u))_2q·sup_u∈[0,1](|g(Ỹ_0(u))|>a_n)^1/2

By Lemma <ref>, we have

    (|g(Ỹ_0(u))|>a_n) ≤c̃_2·exp(-1/2(a_n/c̃_1)^τ_2) ≤c̃_2 n^-4.

Inserting this into proof_lemma_bernstein_eq0 and using Lemma <ref> to bound sup_u∈[0,1]g(Ỹ_0(u))_2q≤c̃_1 N_α(2qM)^M, we obtain.

    W_n - W_n^∘_q ≤ 2|ϕ|_∞c̃_1 N_α(2qM)^M√(c̃_2)· n^-1 =: c̃'(q)· n^-1

With proof_lemma_bernstein_eq03 and Markov's inequality, we obtain

    (|W_n| > γ)    ≤   (|W_n - W_n^∘| > γ/2) + (|W_n^∘| > γ/2)
       ≤   c̃'(1) ·n^-1/γ + (|W_n^∘| > γ/2).

Step 2: Applying a Bernstein inequality from <cit.>. Note that by Assumption <ref> and lemma_emp_eq1, we have with some constant c̃ only dependent on M,χ,C,D:

    δ_q^g(Ỹ(u))(t) ≤c̃·(∑_j=1^tχ_j·ρ^t-j+1)N_α(qM)^M ≤c̃· tρ^t+1 N_α(qM)^M,

i.e.

    δ_q^g(Ỹ(u))(t) ≤c̃'·ρ̃^t· N_α(qM)^M

with c̃' > 0, ρ̃∈ (0,1) only depending on M,χ,C,D,ρ.

For s_1,...,s_u,t_1,...,t_v∈, it holds that

    |(Z_s_1^∘… Z_s_u^∘,Z_t_1^∘… Z_t_v^∘)|
       ≤   ∑_k=0^∞|P_s_u-k(Z_s_1^∘… Z_s_u^∘)· P_s_u-k(Z_t_1^∘… Z_t_v^∘)|
       ≤   ∑_k=0^∞P_s_u-k(Z_s_1^∘… Z_s_u^∘)_2·P_s_u-k(Z_t_1^∘… Z_t_v^∘)_2.

We have

    P_s_u-k(Z_s_1^∘… Z_s_u^∘)_2 ≤ 2Z_s_1^∘… Z_s_u^∘_2 ≤ 2(2|ϕ|_∞a_n)^u.

By Lemma <ref>, L := |ϕ|_∞sup_u∈[0,1]g(Ỹ_0(u))_2 ≤ |ϕ|_∞·c̃_1 N_α(2M)^M. We obtain with proof_lemma_bernstein_eq1 and Lipschitz continuity of Ψ that

    P_s_u-k(Z_t_1^∘… Z_t_v^∘)_2    ≤   Z_t_1^∘… Z_t_v^∘ - (Z_t_1^∘)^*… (Z_t_v^∘)^*_2
       ≤    L|ϕ|_∞(2|ϕ|_∞a_n)^u-2∑_k=1^vδ_2^g(Ỹ(u))(t_1 - s_u + k)
       ≤   c̃'· L|ϕ|_∞ (2|ϕ|_∞a_n)^u-2· v ·ρ̃^t_1-s_u.

Furthermore,

    ∑_s=0^∞(s+1)^k ρ̃^k≤ (1/1-ρ̃)^k+1,    |Z_t|^k ≤ (2|K|_∞a_n)^k.

Using Theorem 1 in <cit.> (with μ=0, ν=1 therein) yields 

    (W_n^∘ > γ/2) ≤(∑_t=1^nZ_t^∘≥γ/2) ≤exp(-(γ/2)^2/2/A_n + B_n^1/3(γ/2)^5/3),

where A_n :=(W_n^∘) and with s_n := #{t∈{1,...,n}:ϕ(t/n) ≠0},

    B_n = 2(√(c̃ L |ϕ|_∞)∨ (2|ϕ|_∞a_n))·1/1-ρ̃·((2^5 s_n c̃' L |ϕ|_∞/1-ρ̃/A_n) ∨ 1)

(we use s_n instead of n which is possible due to a change in the upper bound in their equation (43)).
Here, we have with proof_lemma_bernstein_eq03 and h ≤ 1 that

    |(W_n^∘) - (W_n)| ≤W_n^∘ - W_n_2 ≤c̃'(2)n^-1.

By assumption, we conclude that B_n ≤const.(M,χ,C,D,ρ)· a_n for n large enough.




As a direct corollary of Lemma <ref>, we obtain Theorem <ref> by using the following arguments:


	We apply Lemma <ref> with ϕ(v) := K((v-u)/h). Here, s_n = #{t∈{1,...,n}: ϕ(t/n) ≠ 0}≤ n· h. By Lemma <ref>, we have W_n = (nh)·G̃_h(u)_j and 
	
    |(G̃_h(u)_j) - v_j^2(h,u)| ≤ c· ((nh)^-2 + n^-1log(n)^2),

	showing that 1/2(nh)^2 v_j^2(h,u) ≤(W_n) ≤ 2(nh)^2 v_j^2(h,u) is fulfilled for n large enough since h∈ H_n. Therefore, 
	lemma_bernstein_condition1 is fulfilled for n large enough.
	
	
	



Furthermore, we obtain a Bernstein inequality for the difference G̃_h(u) - G̃_h'(u) for two different bandwidths h' ≤ h:


	Fix u∈ [0,1] and a ∈ (0,1). Assume that g∈(M,χ,C) and Assumption <ref> holds. Then there exist some constants c_4,c_5 > 0 only dependent on M,χ,C,D,ρ,|K|_∞,a such that for h' ≤ a· h,
	
    ((nh')|(G̃_h(u)_j - G̃_h'(u)_j) - (G̃_h(u)_j - G̃_h'(u)_j)| > γ)
       ≤    2exp(-γ^2/32 (nh')^2v_j^2(h,h',u) + c_4 a_n^1/3γ^5/3) + c_5 n^-1/γ.

		
	with a_n := c̃_1(8log(n))^1/τ_2 (c̃_1, τ_2 from Lemma <ref>).
	
		
		
	


	We apply Lemma <ref> with ϕ(v) := h'(K_h(v-u) - K_h'(v-u)). Here #{t∈{1,...,n}: ϕ(t/n) ≠ 0}≤ 2n h. By Lemma <ref>, we have W_n = (nh')·G̃_h(u)_j and 
	
    |(G̃_h(u)_j - G̃_h'(u)_j) - v_j^2(h,h',u)| ≤ c· ((nh')^-2 + n^-1log(n)^2),

	showing that
	
    1/2(nh')^2 v_j^2(h,h',u) ≤(W_n) ≤ 2(nh')^2 v_j^2(h,h',u)

	is fulfilled for n large enough since h∈ H_n. Therefore, 
	lemma_bernstein_condition1 is fulfilled for n large enough.
	
	
	






	Let q ≥ 1. Then with some constant c' = c'(u) > 0,
	
	
  (i) Ĝ_h(u) - G̃_h(u)_q≤ c' n^-1.
	
  (ii) uniformly in u∈[0,1], Ĝ_h(u) - G̃_h(u)_q≤ c' (nh)^-1.
	

 (i) By Hoelder's inequality,

    g(Y_t,n) - g(Ỹ_t(t/n))_q
       ≤    C(1+2DN_α(qM)^M-1)·[∑_i=1^t-1χ_i X_t-i,n - X̃_t-i((t-i)/n)_qM + ∑_i=t^∞χ_i X̃_t-i(t/n)_qM]
       ≤    CD(1+2DN_α(qM)^M-1)·[n^-1∑_i∈χ_i + ∑_i=t^∞χ_i]

Put c” = CD(1+2DN_α(qM)^M-1). Then
	
    Ĝ_h(u) - G̃_h(u)_q≤ |K|_∞c”n^-1∑_i∈χ_i + c”/nh∑_t=1^nK(t/n-u/h)·∑_i=t^∞χ_i.

	If h ≤ u, then summation is only done over t/n≥ u-h/2≥u/2, thus t ≥u/2· n. Since ∑_i=t^∞χ_i ≤ c t^-1≤2c/u·1/n, we obtain the result.

	If h ≥ u, then |1/nh∑_t=1^nK(t/n-u/h)·∑_i=t^∞χ_i| ≤|K|_∞/nh∑_t=1^∞∑_i=t^∞χ_i and nh ≥ nu, we obtain  the result.
	
	(ii) is immediate from lemma_statapprox_eq1 and
	
    Ĝ_h(u) - G̃_h(u)_q≤|K|_∞/nh∑_t=1^ng(Y_t,n) - g(Ỹ_t(t/n))_q.





 Assume that

    σ_p^2(u) := ∑_k∈(p(Ỹ_0(u)), p(Ỹ_k(u)))

fulfills σ_p,min^2 := inf_u∈[0,1]σ^2_p(u) > 0.
For P̃_h(u) = 1/n∑_t=1^nK_h(t/n-u)· p(Ỹ_t(t/n)) with p ∈(M,χ,C), it holds that (i)

    (P̃_h(u)) = 1/nh∫ K(x)^2  x ·σ_p^2(u) + R_n,h,

and for h' ≤ a· h (with fixed a∈(0,1)), there exist constants c',c_6 > 0 such that (ii)

    (P̃_h(u) - P̃_h'(u))    ≥    c_6 (a-1)^2σ_p,min^2· (nh')^-1,
    (P̃_h(u) - P̃_h'(u))    =   1/n∫{K_h(x) - K_h'(x)}^2  x ·σ_p^2(u) + R_n,h,h',

where |R_n,h| ≤ c'· ((nh)^-2 + n^-1log(n)^2), |R_n,h,h'| ≤ c'· ((nh')^-2 + n^-1log(n)^2).


	(i) Put P̅_h(u) := 1/n∑_t=1^nK_h(t/n-u)· p(Ỹ_t(u)). With some constant c̃ only dependent on M,χ,C,D, we have by proof_lemma_bernstein_eq1,
	
    P_t-k{p(Ỹ_t(t/n)) - p(Ỹ_t(u))}_2    ≤    2p(Ỹ_t(t/n)) - p(Ỹ_t(u))_2 ≤c̃· |t/n-u|,
    P_t-k{p(Ỹ_t(t/n)) - p(Ỹ_t(u))}_2    ≤    2sup_u∈[0,1]δ_2^p(Ỹ(u))(k) ≤c̃·ρ̃^k

	 and thus
	
    (P̃_h(u) - P̃_h(u)) - (P̅_h(u) - P̅_h(u))_2
       ≤   1/nh∑_k=0^∞∑_t=1^nK((t/n-u)/h)· P_t-k{p(Ỹ_t(t/n)) - p(Ỹ_t(u))}_2
       =   1/nh∑_k=0^∞(∑_t=1^n K((t/n-u)/h)^2·P_t-k{p(Ỹ_t(t/n)) - p(Ỹ_t(u))}_2^2)^1/2
       ≤   c̃· |K|_∞(nh)^-1/2∑_k=0^∞min{ρ̃^k,h}.

	Since h ≥ n^-1, we have with some constant c̅ only dependent on ρ̃ that
	
    ∑_k=0^∞min{ρ̃^k,h}≤∑_k=0^⌊log(h)/log(ρ̃)⌋h + ∑_k=⌈log(h)/log(ρ̃)⌉^∞ρ̃^k ≤c̅· hlog(n),

	thus
	
    (P̃_h(u) - P̃_h(u)) - (P̅_h(u) - P̅_h(u))_2 ≤c̃c̅ |K|_∞n^-1/2h^1/2log(n).

	Abbreviate p_t := p(Ỹ_t(u)). Then

    P̅_h(u)-P̅_h(u)_2^2
       =   1/n^2∑_k=0^∞∑_t=1^nK_h(t/n-u)· P_t-kp_t_2^2
       =   1/n^2∑_k,l=0^∞[∑_t=1^nK_h(t/n-u)· P_t-kp_t·∑_s=1^nK_h(s/n-u)· P_s-lp_s]
       =   1/(nh)^2∑_k,l=0^∞[P_0p_k· P_0p_l]
                  ×∑_t:1≤ t ≤ n, 1 ≤ t-k+l≤ nK((t/n-u)/h)K(((t-k+l)/n-u)/h).

K(((t-k+l)/n-u)/h) can be replaced by K((t/n-u)/h) due to Lipschitz-continuity (Lipschitz constant L_K) of K with replacement error

    ≤   L_K/(nh)^3∑_k,l=0^∞(k+l)· |[P_0p_k· P_0p_l]|∑_t=1^n|K((t/n-u)/h)|
       ≤   |K|_∞L_K/(nh)^2∑_k,l=0^∞ (k+l)·sup_u∈[0,1]δ_2^p(Ỹ(u))(k) ·sup_u∈[0,1]δ_2^p(Ỹ(u))(l) ≤C̃· |K|_∞L_K(nh)^-2

due to proof_lemma_bernstein_eq1 with some C̃ only depending on M,χ,C,D,ρ. After the replacement, lemma_variance_calc_eq1 reads

    1/(nh)^2∑_t=1^nK((t/n-u)/h)^2·∑_k,l=0^∞[P_0p_k· P_0g_l]
       =   1/(nh)^2∑_t=1^nK((t/n-u)/h)^2·σ_p^2(u).

Since K is Lipschitz continuous, this can be replaced by 1/nh∫ K(x)^2  x ·σ_p^2(u) with replacement error L_K |K|_∞(nh)^-2σ_p^2(u).

(ii) First note that we have with Q := h'/h∈ (0,a]:

    1/n∫{K_h(x) - K_h'(x)}^2  x = 1/nh'∫{ Q K(Q y) - K(y) }^2  y

Let f(Q) := 1/(Q-1)^2∫{Q K(Qy) - K(y)}^2  y. Then

    lim_Q → 1f(Q)    =   lim_Q→ 1∫{Qy K(Qy) - K(y)/Qy-y + K(y)}^2  y
       =   ∫{y K'(y) + K(y)}^2  y > 0

by assumption, and

    lim_Q→0f(Q) = ∫ K(y)^2  y > 0.

Since Q ↦ f(Q) is a continuous function, we conclude that f_min := inf_Q∈[0,1]f(Q) > 0, and thus

    α_n,h,h'^2 := 1/n∫{K_h(x) - K_h'(x)}^2  x ·σ_p^2(u)

fulfills

    nh'/(Q-1)^2α_n,h,h'^2 ≥σ_p,min^2· f_min > 0.

We conclude that

    α_n,h,h'^2 ≥ (a-1)^2σ_p,min^2 f_min· (nh')^-1.

The rest of the proof is the same as in (i).




Define y_t := (t/n-u)(h')^-1. Then with some intermediate values ξ_t ∈ [Qy_t,y_t], ỹ_t ∈ [t-1/n-u/h',t/n-u/h'] and ξ̃_t ∈ [Qỹ_t,ỹ_t]:

    |1/n∑_t=1^n{K_h(t/n-u) - K_h'(t/n-u)}^2 - ∫{K_h(x) - K_h'(x)}^2  x|
       ≤   (Q-1)^2/nh'∑_t=1^n|{Q y_t K'(ξ_t) + K(y_t)}^2 - {Q ỹ_t K'(ξ̃_t) + K(ỹ_t)}^2|
       ≤   (2|K'|_∞+|K|_∞)(Q-1)^2/nh'∑_t=1^n(Q|y_t K'(ξ_t) - ỹ_tK'(ξ̃_t)| + |K(y_t)- K(ỹ_t)|).

It holds that

    ∑_t=1^nQ|y_t - ỹ_t|· |K'(ξ_t)| ≤Qnh'/nh'≤ Q,    ∑_t=1^n|K(y_t) - K(ỹ_t)| ≤|K'|_∞nh'/nh'≤ |K'|_∞,

and, since K' is Lipschitz continuous with Lipschitz constant L_K',

    ∑_t=1^nQ|ỹ_t|· |K'(ξ_t) - K'(ξ̃_t)| ≤ L_K'∑_t=1^n|ξ_t - ξ̃_t| ≤ L_K'(|(Q-1)y_t| + |(Q-1)ỹ_t| + (nh')^-1),

which shows that lemma_variance_calc_eq100 is bounded by

    2(|K'|_∞ + |K|_∞)(Q-1)^2/nh'(








We have

    |1/n∑_t=1^n{K_h(t/n-u) - K_h'(t/n-u)}^2 - ∫{K_h(x-u) - K_h'(x-u)}^2  x|
          |∫_0^1{K_h(⌈ xn ⌉/n-u) - K_h'(⌈ xn ⌉/n-u)}^2 - {K_h(x-u) - K_h'(x-u)}^2  x|
    
           =   (Q-1)^2/h'|∫_0^1/h'{Q/Q-1∫_Q(⌈ ynh'⌉/nh' - u/h')^⌈ ynh'⌉/nh'-u/h'K'(z)  z + K(⌈ ynh'⌉/nh'-u/h')}^2
                        - {Q/Q-1∫_Q(y-u/h')^y-u/h'K'(z) z + K(y-u/h')}^2 y|

We have

    |Q/Q-1∫_Q(y-u/h')^y-u/h'K'(z) z + K(y-u/h')|
       ≤    |K'|_∞|Q(y-u/h')|_{|Q(y-u/h')| ≤1/2 + |K|_∞≤1/2|K'|_∞ + |K|_∞,

and

    Q/Q-1∫_0^1/h'|∫_Q(⌈ ynh'⌉/nh' - u/h')^⌈ ynh'⌉/nh'-u/h'K'(z)  z - ∫_Q(y-u/h')^y-u/h'K'(z) z|  y + ∫_0^1/h'|K(⌈ ynh'⌉/nh'-u/h') - K(y-u/h')|  y,

where

    ∫_-1/2-1/nh'+u/h'^1/2+1/nh'+u/h'|K(⌈ ynh'⌉/nh'-u/h') - K(y-u/h')|  y ≤ (1+2/nh')|K'|_∞/nh'

and

    Q/Q-1∫_0^1/h'|∫_Q(⌈ ynh'⌉/nh' - u/h')^⌈ ynh'⌉/nh'-u/h'K'(z)  z - ∫_Q(y-u/h')^y-u/h'K'(z) z|  y
       ≤    Q∫_-1/2Q-1/nh'+u/h'^1/2Q+1/nh'+u/h'|⌈ ynh'⌉/nh' - y|·|∫_0^1K'(Q(⌈ ynh'⌉/nh' - u/h') + (1-Q)(⌈ ynh'⌉/nh'-u/h') · w)  z|  y
                   + ∫_-1/2Q-1/nh'+u/h'^1/2Q+1/nh'+u/h' Q|y|∫_0^1|K'(Q(⌈ ynh'⌉/nh' - u/h') + (1-Q)(⌈ ynh'⌉/nh'-u/h') · w)
                                      - K'(Q(y - u/h') + (1-Q)(y-u/h') · w) w|  y






It holds that

    ∑_t=1^n|K(y_t) - K(ỹ_t)s| ≤|K'|_∞nh'/nh'≤ |K'|_∞.

Furthermore, since K' is Lipschitz continuous with Lipschitz constant L_K',

    1/Q-1|∫_Qy_t^y_tK'(z)  z - ∫_Qỹ_t^ỹ_tK'(z)  z|
       ≤    |K'|_∞|y_t-ỹ_t|  + |ỹ_t|·∫_0^1|K'(Qy_t + (1-Q)y_t w) - K'(Qỹ_t + (1-Q)ỹ_t w)|  w
       ≤    |K'|_∞|y_t-ỹ_t| + L_K'|ỹ_t| [ Q|y_t - ỹ_t| + (1-Q)|y_t - ỹ_t|].

Summing only over the support of K' (which consists of nh elements in the worst case), this yields

    ∑_t=1^nQ/Q-1|∫_Qy_t^y_tK'(z)  z - ∫_Qỹ_t^ỹ_tK'(z)  z| ≤ |K'|_∞ + L_K'[1 + 1-Q/Q].

Inserting lemma_variance_calc_eq101 and lemma_variance_calc_eq102 into lemma_variance_calc_eq100, we obtain that with some constant c > 0,

    1/α_n,h,h'^2|1/n^2∑_t=1^n{K_h(t/n-u) - K_h'(t/n-u)}^2 - 1/n∫{K_h(x) - K_h'(x)}^2  x|
       ≤    c(n^-1 + 1-Q/nQ).

Note that (nQ)^-1 = h/nh'→ 0 uniformly in h,h'∈ H_n.






Similar as in lemma_variance_calc_eq104, h' ≤ h implies

    {(P̃_h(u) - P̃_h(u)) - (P̃_h'(u) - P̃_h'(u))}
               - {(P̅_h(u) - P̅_h(u)) - (P̅_h'(u) - P̅_h'(u))}_2
       ≤   1/n∑_k=0^∞(∑_t=1^n{ K_h(t/n-u) - K_h'(t/n-u)}^2·P_t-k{p(Ỹ_t(t/n)) - p(Ỹ_t(u))}_2^2)^1/2
       ≤   c̃/n^1/2∑_k=0^∞min{ρ̃^k, h}·(1/n∑_t=1^n{ K_h(t/n-u) - K_h'(t/n-u)}^2)^1/2
       ≤   c̃c̅ h log(n)/n^1/2(1/n∑_t=1^n{ K_h(t/n-u) - K_h'(t/n-u)}^2)^1/2
    
    	   ≤   c̃c̅ h log(n)/n^1/2· 2|K|_∞.
















In the same way as in lemma_variance_calc_eq1,

    (P̅_h(u) - P̅_h'(u)) - (P̅_h(u) - P̅_h'(u))_2^2
       =   1/n^2∑_k,l=0^∞[P_0 p_k· P_0 p_l]·∑_t=1^n{K_h(t/n-u) - K_h'(t/n-u)}
                                ×{K_h(t-k+l/n-u) - K_h'(t-k+l/n-u)}

With the abbreviations z_t := (t/n-u)(h')^-1, z̃_t := (t-k+l/n-u)(h')^-1 (suppressing the dependence on k,l in z̃_t), we conclude that

    |(P̅_h(u) - P̅_h'(u)) - (P̅_h(u) - P̅_h'(u))_2^2 - 1/n^2∑_t=1^n{K_h(t/n-u) - K_h'(t/n-u)}^2 ·σ_p^2(u)|
       ≤   (Q-1)^2/(nh')^2∑_k,l=0^∞|[P_0 p_k· P_0 p_l]|·∑_t=1^n|Q/Q-1∫_Qz_t^z_tK'(z) z + K(z_t)|
                                      ×{|Q/Q-1|·|∫_Qz̃_t^z̃_tK'(z) z - ∫_Qz_t^z_tK'(z)| + |K(z̃_t) - K(z_t)|}
       ≤   (Q-1)^2 c̃^2 (2|K'|_∞ + |K|_∞)/(nh')^2∑_k,l=0^∞ρ̃^k ρ̃^l
                        ×∑_t=1^n{|Q/Q-1|·|∫_Qz̃_t^z̃_tK'(z) z - ∫_Qz_t^z_tK'(z)| + |K(z̃_t) - K(z_t)|}.

Es gilt

    |Q/Q-1|·|∫_Qz̃_t^z̃_tK'(z) z - ∫_Qz_t^z_tK'(z)|
       ≤    Q|z̃_t - z_t| + Q |z_t|∫_0^1|K'(Q z̃_t + (1-Q) z̃_t w) - K'(Q z_t + (1-Q) z_t w)|  w
       ≤     Q|z̃_t - z_t| + Q |z_t|(Q|z̃_t - z_t| + (1-Q)|z̃_t - z_t|).


K_h(t-k+l/n-u) can be replaced by K_h(t/n-u) with replacement error

    ≤   L_K/n(nh)^2∑_k,l=0^∞(k+l)· |[P_0p_k· P_0p_l]|∑_t=1^n{|K_h(t/n-u)| + |K_h'(t/n-u)|}
       ≤   C̃· |K|_∞L_K/(nh)^2,

and similarly K_h(t-k+l/n-u) can be replaced by K_h(t/n-u) with replacement error C̃ |K|_∞L_K/(nh')^2. After the replacement, lemma_variance_calc_eq3 reads

    1/n^2∑_t=1^n{K_h(t/n-u) - K_h'(t/n-u)}^2·∑_k∈(p(Ỹ_0(u)), p(Ỹ_k(u))).

This can be replaced by 1/n∫{K_h(x) - K_h'(x)}^2  x∑_k∈(p(Ỹ_0(u)), p(Ỹ_k(u))) with replacement error 2L_K |K|_∞((nh)^-2+(nh')^-2)∑_k∈(p(Ỹ_0(u)), p(Ỹ_k(u))) due to Lipschitz continuity of K.























plain

















