ON THE OPTIMAL RECONSTRUCTION OF PARTIALLY
OBSERVED FUNCTIONAL DATA

arXiv:1710.10099v1 [] 27 Oct 2017

By Alois Kneip and Dominik Liebl
University of Bonn
We propose a new reconstruction operator that aims to recover
the missing parts of a function given the observed parts. This new
operator belongs to a new, very large class of functional operators
which includes the classical regression operators as a special case. We
show the optimality of our reconstruction operator and demonstrate
that the usually considered regression operators generally cannot be
optimal reconstruction operators. Our estimation theory allows for
autocorrelated functional data and considers the practically relevant
situation in which each of the n functions is observed at m discretization points. We derive rates of consistency for our nonparametric estimation procedures using a double asymptotic (n → ∞, m → ∞).
For data situations, as in our real data application where m is considerably smaller than n, we show that our functional principal components based estimator can provide better rates of convergence than
any conventional nonparametric smoothing method.

1. Introduction. Our work is motivated by a data set from energy economics which is shown in Figure 1. The data consist of partially observed
price functions. Practitioners use these functions, for instance, to do comparative statics, i.e., a ceteris-paribus analysis of price effects with respect
to changes in electricity demand (cf. Weigt, 2009; Hirth, 2013). The possibilities of such an analysis, however, are limited by the extent to which we
can observe the price functions. This motivates the goal of our work, which
is to develop a reconstruction procedure that allows us to recover the total
functions from their partial observations.
Let X1 , . . . , Xn be an identically distributed, possibly weakly dependent
sample of continuous random functions, where each function Xi is an element
of the separable Hilbert space L2 ([a, b]) with [a, b] ⊂ R and E(||Xi ||42 ) < ∞,
Rb
where ||Xi ||22 = a (Xi (x))2 dx.
We denote the observed and missing parts of Xi by XiOi and XiMi , where
XiOi (u) := Xi (u)
XiMi (u) := Xi (u)

for u ∈ Oi ⊆ [a, b]
for u ∈ Mi = [a, b] \ Oi ,

AMS 2000 subject classifications: Primary62M20, 62H25, 62G05, 62G08
Keywords and phrases: functional data analysis, functional principal components, incomplete functions

1

2

40

60

80

Price Function of June 13, 2012
All Price Functions

20

Electr. Prices (EUR/MWh)

100

A. KNEIP & D. LIEBL

0

O
30000

40000

50000

60000

70000

80000

Electr. Demand (MW)

Fig 1. Partially observed electricity price functions XiO (u) with u ∈ Oi ⊆ [a, b].

and where Oi = [Ai , Bi ] ⊆ [a, b] is a random subinterval, independent from
Xi , and with Bi − Ai > 0 almost surely. In what follows we use “O” and
“M ” to denote a given realization of Oi and Mi . In addition, we use the
following shorthand notation for conditioning on Oi and Mi :
XiO (u) := XiOi (u) |(Oi = O)
XiM (u) := XiMi (u)|(Mi = M );
typical realizations of XiO and O are shown in Figure 1. In order to denote
the inner product and norm of L2 (O), we use generic notations h., .i2 and
||.||2 ; their dependency on O will be made obvious by writing, for instance,
O O
O 2
O O
O O
2
O O
hx
R , y i2 and ||x ||2 = hx , x i2 for all x , y ∈ L (O), where hx , y i2 =
O x(u)y(u)du.
For a start, we consider centered random functions, i.e., E(Xi (u)) = µ(u)
with µ(u) = 0 for all u ∈ [a, b]. Our object of interest is the following linear
reconstruction problem:
(1)

XiM = L(XiO ) + Zi ,

u ∈ M,

which aims to reconstruct the unobserved missing parts XiM ∈ L2 (M ) given
the partial observation XiO ∈ L2 (O). Our objective is to identify the optimal
linear reconstruction operator L : L2 (O) → L2 (M ) which minimizes the

2 
mean squared error loss E XiM (u) − [L(XiO )](u)
at any u ∈ M .
The case of partially observed functional data was first considered in
Goldberg, Ritov and Mandelbaum (2014) and Kraus (2015). The work of

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

3

Gromenko et al. (2017) is also related as it proposes an inferential framework for incomplete spatially and temporally correlated functional data.
Goldberg, Ritov and Mandelbaum (2014) consider the case of finite dimensional functional data and their results have well-known counterparts in
multivariate statistics. Kraus (2015) starts by deriving his “optimal” reconstruction operator as a solution to the Frécht-type normal equation, where he
assumes the existence of a bounded solution. The theoretical results in our
paper imply that this assumption generally holds only under the very restrictive case of linear regression operators, i.e., Hilbert-Schmidt operators. For
showing consistency of his empirical reconstruction operator, Kraus (2015)
restricts his work to this case of Hilbert-Schmidt operators. We demonstrate,
however, that a Hilbert-Schmidt operator generally cannot be the optimal
reconstruction operator.
In order to see the latter, we need some conceptional work. HilbertSchmidt operators on L2 spaces correspond to linear regression operators,
Z
(2)
L(XiO ) =
β(u, v)XiO (v)dv, with β ∈ L2 (M × O).
O

However, such a regression operator generally does not provide the optimal
solution of the reconstruction problem in (1). For instance, let us consider
the “last observed”(=“first missing”) points, namely, the boundary points1
ϑ ∈ ∂M of M . For any optimal reconstruction operator L, it must hold
that the “first reconstructed” value, [L(XiO )](ϑ), connects with the “last
observed” value, XiO (ϑ), i.e., that
XiO (ϑ) = [L(XiO )](ϑ)

for all

ϑ ∈ ∂M.

There is no hope, though, of finding
a slope function β(ϑ, .) ∈ L2 (O) that
R
O
fulfills the equation Xi (ϑ) = O β(ϑ, v)XiO (v)dv, except in a degenerated
manner if β(ϑ, .) equals the Dirac-δ function δϑ (v) = 1(v=ϑ) . Dirac-δ functions in L2 (O) are, however, equivalent to the zero function z(v) ≡ 0, since
||δϑ − z||2 = 0, which makes it impossible to identify the optimal reconstruction operator L within the class of linear regression operators defined by
(2).
Best possible linear reconstruction operators depend, of course, on the
structure of the random function Xi , and possible candidates have only to
be well-defined for any function in the support of Xi . We therefore consider the class of all linear operators L with V([L(XiO )](u)) < ∞ and thus
1

The boundary ∂M of a subset M is defined as ∂M := M ∩ O, where M and O denote
the closures of the subsets M and O.

4

A. KNEIP & D. LIEBL

P(|[L(XiO )](u)| < ∞) = 1 for any u ∈ M . This class of reconstruction operators is much larger than the class of regression operators and contains the
latter as a special case. A theoretical characterization is given in Section 2.
We then show that the optimal linear reconstruction operator, minimizing
E[(XiM (u) − [L(XiO )](u))2 ] for all u ∈ M , is given by
[L(XiO )](u) =
(3)

=

∞
O
M
O
O
X
hφO
k , Xi i2 E[Xi (u)hφk , Xi i2 ]
λO
k
k=1
∞
X
hφO , X O i2 hφO , γu i2
k

k=1

i

k

λO
k

,

u ∈ M,

O
where (φO
and nonzero eigenvalk , λk )k≥1 denote the pairs of eigenfunctions
R
ues of the covariance operator ΓO (x)(u) = γ O (u, v)x(v)dv with x ∈ L2 (O).
Here γ O (u, v) = Cov(XiO (u), XiO (v)) denotes the covariance function of XiO ,
and γu (v) = γ(u, v) the covariance function γ(u, v) = Cov(XiM (u), XiO (v)).
The general structure of L in (3) coincides with the structure of the operators considered in the literature on functional linear regression, which, however, additionally postulates that L has an (restrictive) integral-representation
as in (2); see, for instance, Cardot, Mas and Sarda (2007), Cai and Hall
(2006), Hall and Horowitz (2007) in the context of functional linear regression, or Kraus (2015) in a setup similar to ours. There is, though, no reason
to expect that the optimal reconstruction operator L satisfies (2).
To see the point, consider again the reconstruction at a boundary point
O O (ϑ), since for boundO
ϑ ∈ ∂M , where hφO
k , γϑ i2 simplifies to hφk , γϑ i2 = λk φkP
O
O O
O
O
O
ary points ϑ we have γϑ = γϑ and γϑ (.) = γ (ϑ, .) = ∞
k=1 λk φk (ϑ)φk (.).
Plugging this simplification into (3) and using
Karhunen-Loéve decomP the O
O
O
O
hφ
position of XiO implies that [L(XiO )](ϑ) = ∞
k=1 k , Xi i2 φk (ϑ) = Xi (ϑ).
That is, as required for optimal reconstructions, our reconstruction operator
L connects the “last observed” value XiO (ϑ) with the “first reconstructed”
value [L(XiO )](ϑ). As we argue above, though, this property generally cannot be achieved by a regression operator as defined in (2); therefore, we
abstain from imposing such a restrictive assumption.
So far we have considered only boundary points ϑ, though we will show
in Section 2.3 that in most practically important situations there is also no
reason to expect that regression operators are the optimal reconstruction
operators for any non-boundary point u ∈ M either.
The problem of estimating L(XiO ) from real data is considered in Section
3. Motivated by our real data application, our estimation theory allows for
an autocorrelated time series of functional data and considers the practically relevant case where the functions XiO are only observed at m many

5

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

discretization points (Yi1 , Ui1 ), . . . , (Yim , Uim ) with Yij = XiO (Uij ) + εij ,
i = 1, . . . , n, and j = 1, . . . , m.
We basically follow the standard approach to estimate L(XiO ) through
approximating the infinite series (3) by a truncated sequence relying only
on the K largest eigenvalues of the covariance operator. But note that our
data structure implies that we are faced with two simultaneous estimation
problems. One is efficient estimation of [L(XiO )](u) for u ∈ M , the other
one is a best possible estimation of the function Xi (u) for u ∈ O from the
observations (Yi1 , Ui1 ), . . . , (Yim , Uim ). We consider two different estimation
strategies; both allow us to accomplish these two estimation problems.
The first consists in using a classical functional principal components
based approximation of Xi on O, which is simply given by extending the
operator L in (3) by extending γu (v) = Cov(XiM (u), XiO (u)) to γu (v) =
Cov(Xi (u), Xi (u)). This way the empirical counterpart of the truncated sum
[LK (XiO )](u) =

K
X
hφO , X O i2 hφO , γu i2
k

k=1

i

k

λO
k

,

for u ∈ O ∪ M

will simultaneously provide estimates of the true function XiO (u) on the
observed interval O and of the optimal reconstruction [L(XiO )](u) on the
unobserved interval M .
The second consists in estimating the true function XiO (u) on the observed interval O directly from the observations (Yi1 , Ui1 ), . . . , (Yim , Uim )
using, for instance, a local linear smoother and to estimate [L(XiO )](u) for
u ∈ M through approximating the infinite series (3) by its truncated version. But a simple truncation would result in a jump at a boundary point
ϑu , with ϑu denoting the closest boundary point to the considered u ∈ M ,
i.e., ϑu = Ai if |Ai −u| < |Bi −u| and ϑu = Bi otherwise. We know, however,
O O
that for any u ≈ ϑu we must have hφO
k , γu i2 ≈ λk φk (ϑ) for all k ≥ 1, since
O O
hφO
k , γϑ i2 = λk φk (ϑ) for all boundary points ϑu ∈ ∂M . Therefore, we explicitly incorporate boundary points and estimate L(XiO ) by the empirical
counterpart of the truncated sum
[L∗K (XiO )](u) = XiO (ϑ) +

K
X
k=1

O
hφO
k , Xi i2




hφO
O
k , γu i2
−
φ
(ϑ)
,
k
λO
k

u ∈ M.

The above truncation does not lead to an artificial jump at a boundary
O
O
point ϑu , since (hφO
k , γu i2 /λk − φk (ϑu )) → 0 continuously as u → ϑu for all
k = 1, . . . , K.
For estimating the mean and covariance functions—the basic ingredients
of our reconstruction operator—we suggest using Local Linear Kernel (LLK)

6

A. KNEIP & D. LIEBL

estimators. These LLK estimators are commonly used in the context of
sparse functional data (see, e.g., Yao, Müller and Wang, 2005a), though,
we do not consider the case of sparse functional data. In the context of
partially observed functional data, it is advisable to use LLK estimators,
since these will guarantee smooth estimation results, which is not the case
when using the empirical moment estimators for partially observed functions
as proposed in Kraus (2015).
We derive consistency as well as uniform rates of convergence under a
double asymptotic (n → ∞, m → ∞ and K ≡ Km,n → ∞) which allows us
to investigate all data scenarios from almost sparse to dense functional data.
This leads to different convergence rates depending on the relative order of
m and n. For data situations, as in our real data application where m is
considerably smaller than n and the sample curves are very smooth, we show
that our functional principal components based estimator achieves almost
parametric convergence rates and can provide better rates of convergence
than any conventional nonparametric smoothing method.
Our development focuses on the regular situation where (with probability
tending to 1) there exist functions that are observed over the total interval
[a, b]. Only then is it possible to consistently estimate the covariance function
γ(u, v) for all possible pairs (u, v). In our application this is not completely
fulfilled, and there is no information on γ(u, v) for very large values |u − v|.
Consequently, for some intervals O and M the optimal reconstruction operator cannot be identified. This situation corresponds to the case of so-called
fragmentary observations, as considered by Delaigle and Hall (2013), Delaigle and Hall (2016), and Descary and Panaretos (2017). To solve this
problem we suggest an iterative reconstruction algorithm. Optimal reconstruction operators are determined for a number of smaller subintervals, and
a final operator for a larger interval is obtained by successively plugging in
the reconstructions computed for the subintervals. We also provide some
inequality bounding the accumulating reconstruction error.
The rest of this paper is structured as follows: Section 2 introduces our
reconstruction operator and contains the optimality result. Section 3 comprises our estimation procedure. The asymptotic results are presented in
Section 4. Section 5 describes the iterative reconstruction algorithm. Section 6 contains the simulation study and Section 7 the real data application.
All proofs can be found in the online supplement supporting this article.
2. Optimal reconstruction of partially observed functions. Let
our basic setup be as described in Section 1. Any (centered) random function

7

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

XiO then adopts the well-known Karhunen-Loéve (KL) representation
XiO (u)

(4)

∞
X

=

O O
ξik
φk (u),

u ∈ O,

k=1
O = hX O , φO i , where E(ξ O ) = 0 and E(ξ O ξ O ) = λO
with the pc-scores ξik
i
k 2
ik
ik il
k
O > · · · > 0.
for all k = l and zero else and λO
>
λ
1
2
By the classical eigen-equations we have that

φO
k (u) =

(5)

O
hφO
k , γu i2
,
λO
k

u ∈ O,

where γuO (v) = γ O (u, v) = E(XiO (u)XiO (v)). Equation (5) can obviously be
generalized for all u ∈ M which leads to the following “extrapolated” kth
basis function:
φ̃O
k (u) =

(6)

hφO
k , γu i2
,
λO
k

u ∈ M,

where γu (v) = E(XiM (u)XiO (v)). Equation (6) leads to the definition of our
reconstruction operator Lu as a generalized version of the KL representation
in (4):
[L(XiO )](u) =

(7)

∞
X

O O
ξik
φ̃k (u),

u ∈ M.

k=1

Remark. Note that the KL representation provides the very basis of a majority of the works in functional data analysis (cf. Ramsay and Silverman,
2005; Horváth and Kokoszka, 2012). Functional Principal Component Analysis (FPCA) relies on approximating Xi by its first K principal components.
This is justified by the best basis property, i.e., for any K ≥ 1
!
∞
K
X
X
O
O O
λO
ξik
φk (u)k22
k = E kXi (u) −
k=K+1

(8)

k=1

=

min

v1 ,...,vK ∈L2 (O)

E

min

ai1 ,...,aik ∈R

kXiO (u) −

K
X

!
aik vk (u)k22

.

k=1

Remark. For later use it is important to note that the definitions of φ̃O
k (u)
and [L(XiO )](u) in (6) and (7) can be extended for all u ∈ O ∪ M by setting
O
γu = E(Xi (u)Xi (v)). Then by construction φ̃O
k (u) = φk (u) for all u ∈ O
O
O
and, therefore, [L(Xi )](u) = Xi (u) for all u ∈ O.

8

A. KNEIP & D. LIEBL

2.1. A theoretical framework for reconstruction operators. Before we consider the optimality properties of L, we need to define a sensible class of
operators against which to compare our reconstruction operator. We cannot
simply choose the usual class of regression operators, since L does generally
not belong to this class, as pointed out in Section 1. Therefore, we introduce
the following (very large) class of “reconstruction operators”:
Definition 2.1 (Reconstruction operators). Let the (centered) random
function XiO have a KL representation as in (4). We call every linear operator L : L2 (O) → L2 (M ) a “reconstruction operator with respect to XiO ” if
V([L(XiO )](u)) < ∞ for all u ∈ M .
It is important to note that this definition of “reconstruction operators”
is specific to the considered process Xi . This should not be surprising, since
a best possible linear reconstitution will of course depend on the structure
of the relevant random function Xi . The following theorem provides a useful
representation of this class of linear operators:
Theorem 2.1 (Representation of reconstruction operators). Let L :
→ L2 (M ) be a “reconstruction operator with respect to XiO ” according to Definition 2.1. Then there exists a unique (deterministic) parameter
function αu ∈ H such that almost surely
L2 (O)

[L(XiO )](u) = hαu , XiO iH ,

u ∈ M,


where H :=P f ∈ L2 (O) : ||f ||2H < ∞ is a Hilbert space with inner product
∞
O
O
O
2
hf, giH :=
p k=1 hf, φk i2 hg, φk i2 /λk for all f, g ∈ L (O) and induced norm
||f ||H = hf, f iH .
The space H is the Reproducing P
Kernel Hilbert Space (RKHS) that takes
∞
O O
O
the covariance kernel γ O (u, v) =
k=1 λk φk (u)φk (v) as its reproducing
kernel. By construction, we obtain that the variance of [L(XiO )](u) equals
the H-norm of the parameter function αu , i.e., V([L(XiO )](u)) = ||αu ||H .
Let us consider two examples of possible reconstruction operators:
Example 1 - Point of impact: Consider [L(XiO )](u) = XiO (τ ), i.e., a
model with one “impact
τ ∈ O for all missing points u ∈ M . With
P∞ point”
O
O
O O
O
γτ (v) := γ(τ, v) = k=1 λk φk (τ )φO
k (v) we have λk φk (τ ) = hγτ , φk i2 , and

9

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

hence
[L(XiO )](u) = XiO (τ ) =

∞
X

O O
ξik
φk (τ ) =

k=1

(9)

=

∞
X
hX O , φO i2 hγτ , φO i2
i

k

k

λO
k

k=1

∞
X
hX O , φO i2 λO φO (τ )
i

k

k

k

λO
k

k=1

=

= hγτ , XiO iH ,

P
P∞
O O
O
2
where γτ (·) := ∞
k=1 λk φk (τ )φk (·) ∈ H with ||γτ ||H =
k=1
P∞ O O 2
k=1 λk φk (τ ) = V(Xi (τ )) < ∞.

2 O
2
(λO
k ) φk (τ )
λO
k

=

Example 2 - Regression operator: Let L be a regression operator (see
(2)). Then there exists a βu ∈ L2 (O) such that [L(XiO )](u) = hβu , XiO i2 .
Since eigenfunctions can
to an orthonormal basis of L2 (O), we
P∞be completed
O
2 < ∞ for β
necessarily have that k=1 βu,k
u,k := hβu , φk i2 . Then
[L(XiO )](u)

=

hβu , XiO i2

=

∞
X

O
βu,k
ξik

=

k=1

(10)

=

∞
X
hX O , φO ihαu , φO i
i

k=1

k

λO
k

k

∞
X
hX O , φO iλO βu,k
i

k=1

k

k

λO
k

= hαu , XiO iH ,

2
P∞ λO
P∞ O
k βu,k
O (·) ∈ H with ||α||2 = β 2
β
φ
where αu (·) :=
λ
=
u,k
O
k=1 λ
j=1 k
H
k
k
P∞ O 2
O
O
k=1 λk βu,k < ∞. Also note that for any k we have hαu , φk i2 = λk βu,k .
O i constitutes a regression
This means that for αu ∈ H the operator hαu , XiP
H
∞
2
O
hαu , φO
operator if and only
if
in
addition
to
kα
k
=
u
H
k=1
k i2 /λk < ∞ we
P∞
O
2
O
2
also have that
k=1 hαu , φk i2 /(λk ) < ∞ (the latter is not satisfied in
Example 1).

These examples show that Definition 2.1 leads to a very large class of
linear operators which contains the usually considered class of regression
operators as a special case. Of course, the class of reconstruction operators
as defined by Definition 2.1 also contains much more complex operators than
those illustrated in the examples.
Using Theorem 2.1, our reconstruction problem in (3) of finding a “best
linear” reconstruction operator minimizing the squared error loss can now
be restated in a theoretically precise manner: Find the linear operator L :
L2 (O) → L2 (M ) which for all u ∈ M minimizes
h
2 i
E XiM (u) − [L(XiO )](u)

10

A. KNEIP & D. LIEBL

with respect to all reconstruction operators L satisfying [L(XiO )](u) = hαu , XiO iH
for some αu ∈ H. In the next subsection we show that the solution is given
by the operator L defined in (7) which can now be rewritten in the form
(11)

[L(XiO )](u) = hγu , XiO iH ,

u ∈ M,

where γu (v) = γ(u, v) for v ∈ O and u ∈ M . In particular, Theorem 2.2
below shows that V([L(XiO )](u)) = ||γu ||H < ∞ for any u ∈ M , i.e., that L
is indeed a reconstruction operator according to Definition 2.1.
Remark. In the context of reconstructing functions, problems with the use
of regression operators are clearly visible. But the above arguments remain
valid for standard functional linear regression, where for some real-valued
(centered) response variable Yi with V(Yi ) < ∞ one aims to determine
the best linear functional L̃ : L2 (O) → R according to the model Yi =
L̃(XiO ) + εi . Straightforward generalization of Theorems 2.2 and 2.3 below
then shows that the optimal functional L̃(XiO ) is given by
L̃(XiO ) = hσ, XiO iH ,
where σ(u) := E(Yi XiO (u)) for u ∈ O. Following the arguments of Example
2) it is immediately seen that it constitutes a restrictive, additional condition, to assume that L̃(XiO ) can be rewritten in the form [L(XiO )](u) =
hβ, XiO i2 for some βu ∈ L2 (O).
2.2. Theoretical properties. Result (a) of the following theorem assures
that L is a reconstruction operator according to Definition 2.1, and result
(b) assures unbiasedness.
Theorem 2.2. Let the (centered) random function XiO have a KL representation as in (4).
(a) [L(XiO )](u) in (7) has a continuous and finite variance function, i.e.,
V([L(XiO )](u)) < ∞ for all u ∈ M .
(b) [L(XiO )](u) is unbiased in the sense that E([L(XiO )](u)) = 0 for all
u ∈ M.
The following theorem describes the fundamental properties of the reconstruction error
Zi := XiM − L(XiO ),

Zi ∈ L2 (M ),

and contains the optimality result for our reconstruction operator L. Result
(a) shows that the reconstruction error Zi is orthogonal to XiO . This result

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

11

serves as an auxiliary result for result (b) which shows that L(XiO ) is the
optimal linear reconstruction of XiM . Finally, result (c) allows us to identify
cases where XiM can be reconstructed without any reconstruction error.
Theorem 2.3 (Optimal linear reconstruction).
that:
(a) For every v ∈ O and u ∈ M ,

(12)
E XiO (v)Zi (u) = 0
(13)

Under our setup it holds

and
2

V(Zi (u)) = E (Zi (u))



= γ(u, u) −

∞
X

2
O
λO
k φ̃k (u) .

k=1

(b) For any linear operator L : L2 (O) → L2 (M ) that is a reconstruction
operator with respect to XiO , according to Definition 2.1,

2 
E XiM (u) − [L(XiO )](u)
≥ V(Zi (u)), for all u ∈ M.
(c) Assume that the underlying process is Gaussian, and let Xi and Xj ,
i 6= j, be two independent copies of the random variable Xi . Then for
all u ∈ M the variance of the reconstruction error can be written as

2
1  
(14)
V(Zi (u)) = E E Xi,1 (u) − Xi,2 (u) XiO = XjO
2
where XiO = XjO means equality on O, i.e., Xi (v) = Xj (v) for all
v ∈ O.
Whether or not a sensible reconstruction of partially observed functions
is possible, of course, depends on the character of the underlying process.
For very rough and unstructured processes no satisfactory results can be
expected. An example is the standard Brownian motion on [0, 1] which is
a pure random process with independent increments. If Brownian motions
Xi are only observed on an interval O := [0, ϑ], it is well known that the
“best” (i.e., the only unbiased) prediction of Xi (u) for u ∈ M := (ϑ, 1] is the
last observed value XiO (ϑ). This result is consistent with our definition of
an “optimal” operator L: The covariance function of the Brownian motion
is given by γu (v) = min(u, v), and hence for all v ∈ [0, ϑ] and u ≥ ϑ one
obtains γu (v) = min(u, v) = min(ϑ, v) = γϑ (v) = v. Therefore, by (11)
and (9) we have [L(XiO )](u) = hγu , XiO iH = hγϑ , XiO iH = XiO (ϑ) for all
u ∈ [ϑ, 1]. This obviously does not constitute a regression operator. In this
paper we focus on processes that lead to smooth, regularly shaped sample
curves which may allow for a sensible reconstruction.

12

A. KNEIP & D. LIEBL

Result (c) of Theorem 2.3 may be useful to identify cases that allow
for a perfect reconstruction. By (14) there is no reconstruction error, i.e.,
V(Zi (u)) = 0 for u ∈ M if the event XiO = XjO implies that also XiM = XjM .
This might be fulfilled for very simply structured processes. It
Pis necessarily
satisfied for finite dimensional random functions XiK (u) = K
k=1 ξik φk (u),
λK+1 = λK+2 = · · · = 0, as long as the basis functions φ1 , . . . , φK are linear
independent over O.
2.3. A deeper look at the structure of L. Remember that the definition
of L can be extended to an operator L : L2 (O) → L2 (O ∪ M ). For elements
u ∈ O of the observed part O the best “reconstruction” of Xi (u) is obviously the observed value Xi (u) itself, and indeed for any u ∈ O (11) yields
[L(XiO )](u) = hγu , XiO iH = Xi (u). Equation (7) then holds with
φ̃O
k (u) :=

hφO
k , γu i2
= φO
k (u),
λO
k

u ∈ O.

Since γu (v) = γ(u, v) = E(Xi (u)Xi (v)) is a continuous function on O ∪ M
it follows that the resulting “reconstructed” function [L(XiO )] is continuous
on O ∪ M . In particular, [L(XiO )] is continuous at any boundary point ϑu ∈
∂M , and
lim

[L(XiO )](u) = Xi (ϑu ), as well as

lim

O
φ̃O
k (u) = φk (ϑu ), k = 1, 2, . . .

u∈M,u→ϑu
u∈M,u→ϑu

Equation (7) together with our definition of Zi imply that the complete
function Xi on O ∪ M can be represented in the form
(15)
Xi (v) =

∞
X
k=1

O O
ξik
φk (v), v

∈ O, and Xi (u) =

∞
X

O O
ξik
φ̃k (u) + Zi (u), u ∈ M.

k=1

This sheds
light on result (14). We will have Zi (u) ≈ 0 and
P someOadditional
O (u) if on the segment M the process is essentially driven
Xi (u) ≈ ∞
ξ
φ̃
k=1 ik k
O as those determining its structure on O.
by the same random components ξik
Additional random components Zi (u), not present on O, and uncorrelated
O , then have to be of minor importance. If the observed interval is suffiwith ξik
ciently long, then this may be approximately true for processes
with smooth,
P
O O
similarly shaped trajectories. Note that even if Xi (u) = ∞
ξ
k=1 ik φ̃k (u), the
eigenfunctions of XiM will usually not coincide with φ̃O
k since there is no
reason to expect that these functions be mutually orthogonal.

120

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

●

●

Data Pairs of June 13, 2012
Pooled Data pairs

● ●

●
● ●
●
●
●
●
●●
●
●●●
● ●●
●
●
●
●
●
●
●●
●
●●
● ●●●
●
●
●●●●●●● ●
●
●
●
● ●●●●
●●
●●●
●●
●
●● ●
●
● ●●●
●●
●
●
●
●●●● ●●
●
● ●
●
●
●●●
●●
●●
●
● ●
●
●●●
●
● ●
●
●●
●
● ● ●●
●
●●
● ●●●
●●
●
●
●● ●●
●● ●
●●
●
●
●●
●
●
●●●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
● ●●
●
●●●
●●
●
●
●●
●●
●
●
●●●
●●
●● ●
●
●
●●
●●
●
●
●
●●●●
●
●
●
●
●● ●
●
●●
●
●
●
●●
●
●
●
●●●
●
●●●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●
●
●●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●●● ● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●●
●● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ● ●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●● ●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
● ●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●
●
●
●
●●
●
●
●●●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●●●●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●●●● ●
●
●
●
●
●
● ●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●● ●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
● ● ●● ●
●
●
●
●●
●
●
●●●●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●●
●●
●
●●● ●
●●
●
●
●●
●●●
●
●
●
●
●
●
● ●
●
●
●
●
●●
●●
●
●
●●
●●●
●●
● ●● ●● ●●
●
●●
●
●●
●●
●
●
●
●
●●
●●●
●●●
●
●
●
●●
●
●
●●
●●
●
●
●●
●
●●
●●
●●
●●
●●●●
●
●
●
●
●
●
●
●
●
●
● ●
●●●● ● ●●
●●● ●
●
●●● ●
●●
●●
●●
●●
●● ●
●
●●●● ● ●● ● ●
● ●●
●●●●
●●
● ●●
●●●
●●
●●
●
●

80
60
40
0

●

●

●●

20

Electr. Prices (EUR/MWh)

100

●

13

O

●
●

30000

● ●

40000

50000

60000

70000

80000

Electr. Demand (MW)

Fig 2. Scatter plot of the observed data pairs (Yij , Uij ).

3. Estimation. We typically do not observe a functional trajectory directly, but only its discretization with or without measurement errors. For
instance, Figure 1 shows the pre-smoothed functions; however, the actual
raw data is shown in Figure 2. Let XO
i := ((Yi1 , Ui1 ), . . . , (Yim , Uim )) denote
the observable data pairs of a function XiO , where
(16)

Yij = XiO (Uij ) + εij ,

i ∈ {1, . . . , n},

j ∈ {1, . . . , m},

and Uij ∈ Oi = [Ai , Bi ]. We consider the case where Ui1 , . . . , Uim are iid
random variables with strictly positive density over the random subinterval
[Ai , Bi ], which in practice can be approximated by Ai ≈ min1≤j≤m (Uij ) and
Bi ≈ max1≤j≤m (Uij ). Let the error term εij be a real iid random variable
with mean zero and finite variance V(εij ) = σ 2 , with 0 < σ 2 < ∞. Motivated by our real data application we will concentrate on the case that n
is considerably larger than m, which also holds in many other important
applications.
So far, we have considered centered random functions XiO . Henceforth,
we consider non-centered functions and will make the empirical centering
explicit in all estimators. As already outlined in Section 1, we propose to
estimate [L(XiO )](u) by the empirical counterpart of the truncated sum
(17)

[LK (XiO )](u) = µ(u) +

K
X
k=1

O O
ξik
φ̃k (u) = µ(u) +

K
X
k=1

O
ξik

hφO
k , γu i2
,
λO
k

14

A. KNEIP & D. LIEBL

O and φ̃O (u) are replaced by suitable
where the unknown true values of ξik
k
estimates defined below.
Note, however, that our data structure (16) implies that we are faced
with two simultaneous estimation problems. One is efficient estimation of
[L(XiO )](u) for u ∈ M , the other one is a best possible estimation of the
underlying function Xi (u) for u ∈ O. There are two possible strategies which
can be employed.
The first is motivated by the best basis property (8) and simply consists
in using an FPCA-approximation of Xi on O. Recall that [L(XiO )](u) can
be extended to an operator on O ∪ M . For u ∈ O we then obtain φ̃O
k (u) =
O
O
O
O
hφk , γu i2 /λk = φk (u), and thus [L(Xi )](u) = Xi (u). That is, estimates
O
[LbK (XO
i )](u) of [LK (Xi )](u) for u ∈ O ∪ M will simultaneously provide
estimates of the true function Xi (u) on the observed interval O and of the
optimal reconstruction [L(XiO )](u) on the unobserved interval M .
The second approach is to rely on nonparametric curve estimation, e.g.,
local linear smoothers, to approximate XiO on O, while (17) is only used
for reconstructing the unobserved part M . We then, however, run into the
boundary problem already mentioned in the introduction. Let ϑu be the
boundary point closest to the considered u ∈ M , i.e., ϑu = Ai if |Ai − u| <
|Bi − u| and ϑu = Bi else. Usually nonparametric estimates of XiO and
reconstruction estimates based on (17) will not coincide for u = Ai or u = Bi .
A correction, leading to continuous function estimates on O ∪ M may then
be based on the identity [L(XiO )](u) = XiO (ϑu ) + [L(XiO )](u) − [L(XiO )](ϑu )
and its truncated version

[L∗K (XiO )](u) = XiO (ϑu ) + [LK (XiO )](u) − [LK (XiO )](u)
(18)

= µ(u) − µ(ϑu )

K
X

O O
ξik
(φ̃k (u) − φ̃O
k (ϑu ),

for u ∈ M

k=1

In this paper we propose to use the following empirical counterparts of
[LK (XiO )](u) or [L∗K (XiO )](u):
(19)

[LbK (XO
i )](u) := µ̂(u; hµ ) +

K
X

O ˆO
ξˆik
φ̃k (u)

for u ∈ O ∪ M,

k=1

hφ̂O
ˆ
k , γ̂u i2
,
where φ̃O
(u)
:=
k
λ̂O
k

k = 1, . . . , K,

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

15

O
O
bO
b
b
[Lb∗K (XO
i )](u) : = Xi (ϑ; hX ) + [LK (Xi )](u) − [LK (Xi )](ϑ)

(20)

biO (ϑu ; hX ) + µ̂(u; hµ ) − µ̂(ϑu ; hµ ) +
=X

K
X



ˆ
O
O
ξˆik
φ̃O
k (u) − φ̂k (ϑu ) .

k=1

b O (u; hX ) is defined by X
b O (u; hX ) =
Here, for u ∈ O the LLK estimator X
i
i
β̂0 , where
(21)

(β̂0 , β̂1 ) = arg min
β0 ,β1

m
X

[Yij − β0 − β1 (Uij − u)]2 KhX (Uij − u)

j=1

for Kh (.) = κ(./h)/h. The kernel function κ is assumed to be a univariate
symmetric pdf with compact support supp(κ) = [−1, 1] such as, e.g., the
Epanechnikov kernel
kernel constants are
R 2(see Assumption A5).RThe usual
2
given by ν2 (κ) := v κ(v)dv, and R(κ) := κ(v) dv.
The LLK mean estimator µ̂(u; hµ ) is defined by µ̂(u; hµ ) = β̂0 , where
(22)

(β̂0 , β̂1 ) = arg min
β0 ,β1

n X
m
X

[Yij − β0 − β1 (Uij − u)]2 Khµ (Uij − u).

i=1 j=1

The estimator γ̂u (v) = γ̂(u, v) is defined as follows. Due to the symmetry of the covariance function, γ(u, v) = γ(v, u), it suffices to estimate
only the upper-diagonal part of γ(u, v) with u ≤ v. For all upper-diagonal
points (u, v) ∈ [a, b]2 with u ≤ v, the LLK estimator γ̂(u, v; hγ ) is defined as
γ̂(u, v; hγ ) = β̂0 , where
(23)
(β̂0 , β̂1 , β̂2 ) =arg min

n
X
X

bijl − β0 − β1 (Uij − u) − β2 (Uil − v)]2
[C

β0 ,β1 ,β2 i=1 1≤j,l≤m
Uij <Ujl

× Khγ (Uij − u)Khγ (Uil − v),
bijl defined as
with raw-covariance points C
(24)

bijl := (Yij − µ̂(Uij ))(Yil − µ̂(Uil )),
C

for Uij < Uil .

For estimating the lower-diagonal of γ(u, v) with u ≥ v set γ̂(u, v; hγ ) =
γ̂(v, u; hγ ).
Like Yao, Müller and Wang (2005a), we do not include the diagonal rawbijj for which Uij = Uij as these would introduce an estimation
covariances C

16

A. KNEIP & D. LIEBL

bias through taking squares of the error term εij contained in Yij . Our
LLK estimator γ̂(u, v) differs from that in Yao, Müller and Wang (2005a),
since it avoids smoothing across the diagonal. This allows us to estimate
also covariance functions that are not differentiable at the diagonal such as,
for instance, the covariance function of a Brownian motion or many other
stochastic processes.
Remark. Using only the upper-diagonal raw-covariance points does not
mean that we are using less information, since the upper- and lower-diagonal
raw-covariance points are only the mirrored points to each other. Yao, Müller
and Wang (2005a) assume a more restrictive situation where all second order
derivatives at the diagonal γ(u, u) are continuous. Our approach is consistent
there as well, without any loss in terms of convergence rates.
O
Estimates of the eigenvalues λO
k and the eigenfunctions φk are defined by
the corresponding solutions of the empirical eigen-equations
Z
O O
(25)
γ̂(u, v; hγ )φ̂O
u ∈ O.
k (v) dv = λ̂k φ̂k (u),
O
O is defined by the following integral
Finally, the empirical pc-score ξˆik
O
approximation of ξik :

(26)

O
=
ξˆik

m
X

φ̂O
k (Ui(j) )(Yi(j) − µ̂(Ui(j) ; hµ ))(Ui(j) − Ui,(j−1) ),

j=2

where (Yi(j) , Ui(j) ) are ordered data pairs for which the ordering is determined through the order sample Ui(1) ≤ · · · ≤ Ui(m)
In our theoretical analysis we consider K ≡ Knm → ∞ as the sample size
nm → ∞. In practice, the truncation parameter K can be chosen by one of
the usual procedures such as, for instance, Cross Validation or the Fraction
of Variance Explained (FVE) criterion.
Alternatively, one can use an adapted version of the AIC-type criterion in
Yao, Müller and Wang (2005a). For this let (Yi(1) , Ui(1) ), . . . , (Yi(m) , Ui(m) )
denote the ordered data as in (26) and define the following quantities by
partitioning the data into pseudo-missing (P M ) and pseudo-observed (P O)
halves:
>
YiP M = Yi(1) , . . . , Yi(bm/2c)
>
XPi O = (Yi(bm/2c+1) , Ui(bm/2c+1) ), . . . , (Yi(m) , Ui(m )

b P M = [LbK (XP O )](Ui(bm/2c+1) ), . . . , [LbK (XP O )](Ui(m) ) >
Y
iK
i
i

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

17

b K + K with
The information criterion is then defined as AICK = −L
(27)

bK =
L

n n
X
i=1

bm/2c
bm/2c
log(2π) −
log σ̂ 2
2
2

o
1
b P M > YP M − Y
b PM ,
− 2 YiP M − Y
iK
i
iK
2σ̂
−

P
where σ̂ 2 = n−1 ni=1 σ̂i2 and σ̂i2 is a nonparametric variance estimator
such
P
as, for instance, that of Rice (1984), i.e., σ̂i2 = (2(m − 1))−1 m−1
(Y
i(j) −
j=1
2
Yi(j−1) ) .
4. Asymptotic results. Our theoretical analysis analyzes the reconstruction of an arbitrary sample function Xi satisfying O ⊂ Oi .
Our asymptotic results on the convergence of our nonparametric estimators are developed under the following assumptions which are generally
close to those in Yao, Müller and Wang (2005b) and Hall, Müller and Wang
(2006). We additionally allow for weakly dependent time series of random
functions (Xi )i , and we consider a different asymptotic setup excluding the
case of sparse functional data. Only second-order kernels are employed.
A1 (Stochastic) For some dmin > 0 the conditional random variables
Ui1 |Oi ,. . . ,Uim |Oi are iid with pdf fU |Oi (u) ≥ dmin for all u ∈ Oi = [Ai , Bi ]
and zero else. For the marginal pdf fU it is assumed that fU (u) > 0 for
all u = [a, b] and zero else. The time series (Ai )i=1,...,n , (Bi )i=1,...,n , and
(Xi )i=1,...,n is a strictly stationary ergodic functional time series with finite fourth moments, i.e., E(||Xi ||42 ) < ∞ and autocovariance functions
with geometric decay. I.e., there are constants CA , CB , C, Ċ, ιA , ιB , ι, ι̇ with
0 < CA , CB , C, Ċ < ∞ and 0 < ιA , ιB , ι, ι̇ < 1, such that | Cov(Ai , Bi+h )| ≤
CA ιhA , | Cov(Bi , Bi+h )| ≤ CB ιhB , sup(u,v)∈[a,b]2 |γh (u, v)| ≤ Cιh , and
sup(u1 ,v1 ,u2 ,v2 )∈[a,b]4 |γ̇h ((u1 , v1 ), (u2 , v2 ))| ≤ Ċ ι̇h for all h ≥ 0, where
γh (u, v) := Cov(Xi+h (u), Xi (v)) and
γ̇h ((u1 , v1 ), (u2 , v2 )) := Cov(Xi+h (u1 )Xi+h (v1 ), Xi (u2 )Xi (v2 )).
The error term εij is assumed to be independent from all other random variables. The random variables Uij and Oi are assumed to be independent from
(Xi )i=1,...,n , which leads to the so-called “missing completely at random” assumption. The event Oi × Oi = [a, b]2 has a strictly positive probability and
Bi > Ai almost surely.
A2 (Asymptotic scenario) nm → ∞, where n → ∞ and m = mn  nθ with
0 < θ < ∞. Here, an  bn is used to denote that limn→∞ (an /bn ) = c for
some constant 0 < c < ∞. It is assumed that mn ≥ 2 for all n.

18

A. KNEIP & D. LIEBL

A3 (Smoothness) For µ̂: All second order derivatives of µ(u) on [a, b], fU (u)
on [a, b], γ(u, v) on [a, b]2 , and of fY U (y, u) on R × [a, b] are uniformly continuous and bounded, where fY U is the joint pdf of (Yij , Uij ), and where
supp(fU ) = [a, b]. For γ̂: All second order derivatives of γ(u, v) on {(u, v) ∈
[a, b]2 : u 6= v}, fU U (u, v) on [a, b]2 , γ̇((u1 , v1 ), (u2 , v2 )) on [a, b]4 , and of
fCU U (c, u, v) on R × [a, b]2 are uniformly continuous and bounded, where
fCU U is the joint pdf of (Cijl , Uij , Uil ), and where supp(fU U ) = [a, b]2 . Fi |f 0 (u)| 
nally, all fU |Oi (u) are a.s. continuously differentiable, and E f U |Oi(u)2 < ∞.
U |Oi

A4 (Bandwidths) For estimating XiO : hX ≡ hm,X → 0 and (m hX )−1/2 →
∞ as m → ∞. For estimating µ: hµ ≡ hnm,µ → 0 and (nm hµ )−1/2 → ∞
as nm → ∞. For estimating γ: hγ ≡ hnM,γ → 0 and (nM hγ )−1/2 → ∞ as
nM → ∞, where M = (m2 − m)/2.
A5 (Kernel function) κ is a second-order kernel with compact support
supp(κ) = [−1, 1].
We follow Hall, Müller and Wang (2006) and consider a deterministic
sample size m; however, in case of a random m our results apply conditionally on m. In the case of different values mi for each i = 1, . . . , n, our results
can be adapted using the “OBS” and “SUBJ” weighting schemes proposed
by Zhang and Wang (2016).
While (A1) - (A5) suffice to determine rates of convergence of mean and
covariance estimators, it is well-known from the literature that rates of convergence of estimated eigenfunctions will depend on the rate of decay characterizing the convergence of λO
k to zero as k → ∞.
We want to note that for a subinterval O ⊂ [a, b] the decay of eigenvalues
O
λO
1 , λ2 , . . . will usually be faster than the rate of decay of the eigenvalC
2
2
ues λC
1 , λ2 , . . . of the complete covariance operator defined on [a, b] ⊃ O .
C
C
This is easily seen. Let γ1 , γ2 , . . . denote the corresponding eigenfunctions
C|O
C|O
on [a, b], and define γk ∈ L2 (O) by γk (u) = γk (u) for u ∈ O and k =
C|O
1, 2, . . . . For the special case vk = γk , k = 1, . . . , K, inequality (8) then imR C|O 2
P∞
P∞
C
plies that for all K ≥ 1 we have k=K+1 λO
(u) du ≤
k=K+1 λk O γk
k ≤
R
R
P∞
C|O
b
C
(u)2 du ≤ a γkC (u)2 du = 1 for all k = 1, 2, . . . .
k=K+1 λk , since O γk
To complete our asymptotic setup, we consider reconstruction of arbitrary
sample functions Xi observed over an interval Oi = [Ai , Bi ] with length
Bi − Ai ≥ `min , where 0 < `min < b − a is an (arbitrary) constant. We then
impose the following additional assumptions.
A6 (Eigenvalues) For any subinterval O = [A, B] ⊂ [a, b] with B − A ≥ `min
O
the ordered eigenvalues λO
1 > λ2 > · · · > 0 have all multiplicity one. Fur-

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

19

thermore, there exist some aO > 1 and some 0 < cO < ∞, possibly deO
−aO −1 with 0 < c < ∞, and
pending on O, such that λO
O
k − λk+1 ≥ cO k
−aO ) as well as 1/λO = O(k aO ) as k → ∞.
λO
=
O(k
k
k
A7 (Eigenfunctions) For any subinterval O = [A, B] ⊂ [a, b] with B − A ≥
`min there exists a constant 0 < DO < ∞ such that supu∈[a,b] supk≥1 |φ̃O
k (u)| ≤
O
O
DO (recall that φ̃k (U ) = φk (u) for u ∈ O).
Assumption (A6) requires a polynomial decay of the sequence of eigenvalues. It cannot be tested, but it corresponds to the usual assumption characterizing a majority of work concerning eigenanalysis of functional data,
although some authors also consider exponential decays. There exist various types of functional data, but this paper focuses on applications where
the true sample functions are very smooth and all possess similar functional
structure. This is quite frequent in practice, and in applied papers it is then
often found that few functional principal components suffice to approximate
sample functions with high accuracy. In view of the best basis property (8)
one may then tend to assume that (A6) holds for some very large aO  1. Indeed, for increasing k eigenfunctions φO
k will become less and less “smooth”
since the number of sign changes will necessarily tend to infinity. If observed
trajectories are very smooth, then the influence of such high-frequency comO
ponents must be very small, indicating a very small eigenvalue λO
k = E(ξk )
for large k. This is of substantial interest, since the theorems below show
that rates of convergence of our final estimators are better the larger aO .
Assumption (A7) imposes a (typical) regularity condition on the strucO
O
ture of the eigenfunctions φO
k (u), since φ̃k (u) = φk (u) for u ∈ O. For
O
u ∈ M = [a, b] \ O condition |φ̃k (u)| ≤ DO is much weaker than the standard assumption
operator which would go along with the
P∞ of Oa regression
2
requirement
k=1 φ̃k (u) < ∞. But, for u ∈ M , theory only ensures that
P∞
O (u))2 < ∞ (see Theorem 2.3 (a)) and (A7) is restrictive in so
λ
(
φ̃
k=1 k k
far as it excludes the possible case that for u ∈ M we have |φ̃O
k (u)| → ∞
as k → ∞. We are not sure whether the latter excluded case constitutes a
realistic scenario in practical applications, since by (15) it would correspond
to the fairly odd situation that for large k the high-frequency components
O possess much larger influence on M than on O. Nevertheless, we want to
ξik
emphasize that the arguments used in the proof of our theorems may easily
be generalized to prove consistency of our estimators even in this excluded
case; however, rates of convergence deteriorate and asymptotic expressions
become much more involved.

20

A. KNEIP & D. LIEBL

Theorem 4.1 (Preliminary consistency results).
Under Assumptions (A1)-(A5) we have that:
(a) supu∈[a,b] |µ̂(u; hµ ) − µ(u)| = Op (rµ )
b O (u; hµ , hX ) − X O (u)| = Op (rX )
(ã) Conditional on XiO : supu∈O |X
i
i
(b) sup(u,v)∈[a,b]2 |γ̂(u, v; hγ ) − γ(u, v)| = Op (rµ + rγ ), where
p
√
rµ ≡ rµ (hµ , n, m) := h2µ + 1/ nm hµ + 1/ n
√
rX ≡ rX (hX , m) := h2X + 1/ q
m hX
√
2
rγ ≡ rγ (hγ , n, M ) := hγ + 1/ nM h2γ + 1/ n,
and where (ã) is effectively only a corollary of result (a).
If additionally Assumption (A6) and (A7) hold, we obtain for every subinterval O = [A, B] ⊂ [a, b] with B − A ≥ `min :
O
(c) supk≥1 |λ̂O
k − λk | = Op (rµ + rγ ) for all k ≥ 1
O
(d) sup1≤k≤K δkO kĉk φ̂O
k − φk k2 = Op ((rµ + rγ ))
O
O
O
O
where ĉk := sgn(hφ̂O
k , φk i2 ) and δk := minj6=k {λj − λk }.

Related results can be found in Yao, Müller and Wang (2005a), Li and
Hsing (2010), and Zhang and Wang (2016). Our proof of results (a)-(b)
follows that of Yao, Müller and Wang (2005a), but is more restrictive as we
allow only for compact second order kernels. Results (c) and (d) follow from
standard arguments as used in Bosq (2000).
Theorem 4.2 (Consistency results for LbK (XO
i )). Consider an arbitrary
i ∈ 1, . . . , n and assume that O := Oi = [Ai , Bi ] satisfies Bi − Ai ≥ `min .
For some 0 < C < ∞ let K̄mn = C · (min{n1/2 , (nM )1/3 })1/(aO +3/2) .
The following results hold under Assumptions A1-A7, for 1 ≤ K ≤ K̄mn ,
hX  m−1/5 , hµ  (nm)−1/5 and hγ  (nM )−1/6 , as n → ∞ and m → ∞
with m  nθ , 0 < θ < ∞. For any u ∈ [a, b]:

O
[LbK (XO
i )](u) = [LK (Xi )](u) + Op

K

1
K aO /2+3/2
+
m1/2 min{n1/2 , (nM )1/3 }

!!

(28)

[L(XiO )](u) − [LK (XiO )](u) = O 

∞
X
k=K+1

!1/2 
λO
k



 = Op K −(aO −1)/2

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

21

Furthermore, for all u ∈ M := [a, b] \ O
∗
O
[Lb∗K (XO
i )](u) = [LK (Xi )](u) + Op

m−2/5 + K

1
K aO /2+3/2
+
m1/2 min{n1/2 , (nM )1/3 }

(29)

[L(XiO )](u) − [L∗K (XiO )](u) = O 

∞
X

!1/2 
λO
k



 = Op K −(aO −1)/2

k=K+1

The theorem tell us that for any u ∈ [a, b] the estimator [LbK (XO
i )](u)
achieves the same rate of convergence. But recall that for u ∈ O = Oi we
have [L(XiO )](u) = Xi (u), and thus [LbK (XO
i )](u) can be seen as a nonparametric estimator of Xi . In contrast, for u ∈ M we have [L(XiO )](u) =
Xi (u) + Zi (u), and therefore the distance between Xi (u) and [LbK (XO
i )](u)
will additionally depend on the reconstruction error Zi (u).
Note that by the above result the rates of convergence depend on m
and n, and the optimal K depends on these quantities in a complex way.
However, the situation simplifies if m is considerably smaller than n such
that m = mn  nθ for θ ≤ 1/2. The following corollary then is a direct
consequence of (28).
Corollary 4.1. Under the conditions of Theorem 4.2 additionally assume that θ ≤ 1/2. With K ≡ Km  m1/(aO +2) we obtain for all u ∈ [a, b]


a −1
− 2(aO +2)
O
O
b
O
|[L(Xi )](u)) − [LK (Xi )](u)| = Op m
(30)
Result (30) implies that if the true sample curves are very smooth in the
sense that (A6) holds for a very large aO  1, then the rate of convergence of
the individual curve estimators [LbK (XO
i )](u) is very close to the parametric
rate m−1/2 .
Recall that that the main difference between LbK and Lb∗K consists in the
way of estimating Xi on the observed interval Oi . Lb∗K is based on local linear
smoothing, and the associated estimation error of order m−2/5 appears in
result (29). When relying solely on [LbK (XO
i )](u)|, we approximate Xi (u) for
u ∈ Oi by an FPCA-approximation, as used in hundreds of papers for different purposes. We can infer from (30) that at least under restrictive conditions
(m considerably smaller than n; very large aO  1) FPCA-approximations
of Xi (u), u ∈ Oi , may provide better rates of convergence than any conventional smoothing method. We believe that this is an interesting result in

!!

22

A. KNEIP & D. LIEBL

its own right, which to our knowledge has not yet been established in the
literature.
5. Iterative reconstruction algorithm. So far we have focused on
the regular situation where the covariance function γ(u, v) is estimable for
all points (u, v) ∈ [a, b]2 . Under this situation we can reconstruct the entire
missing parts of the functions, such that the reconstructed functions X̃i with
(
[L(XiO )](u) if u ∈ M
X̃i (u) =
(31)
XiO (u)
if u ∈ O
are identifiable for all u ∈ [a, b].
In our application, however, we face the more restrictive situation where
the mean function µ(u) can still be estimated for all u ∈ [a, b], but where
there is no information on γ(u, v) for large values |u − v|; see Figure 6. This
makes it impossible to reconstruct the entire missing part of a function, such
that X̃i (u) cannot be identified for all u ∈ [a, b].
In order to reconstruct functions X̃i that cover the total interval [a, b],
or a very large part of it, we propose successively plugging in the optimal
reconstructions computed for subintervals. In the following we describe our
iterative reconstruction algorithm:
Algorithm 5.1 (Iterative reconstruction algorithm).
1st Step Denote the originally observed interval O as O1 and compute
(
[L(XiO1 )](u) if u ∈ M1
X̃i,1 (u) =
XiO1 (u)
if u ∈ O1
rth Step (r ≥ 2) Choose a new “observed” interval Or ⊂ Or−1 ∪ Mr−1 and
use X̃iOr (u) := X̃i,r−1 (u) with u ∈ Or as the new “observed” fragment.
Compute
(
[L(X̃iOr )](u) if u ∈ Mr
X̃i,r (u) =
X̃iOr (u)
if u ∈ Or .
Join the reconstructed fragments X̃i,1 , . . . , X̃i,r to form the new “observed” fragment
S X̃i,r−1 on Or−1 ∪ Mr−1 and repeat the rth step.
Stopping Stop if rl=1 Ol ∪ Ml = [a, b] or if r = rmax .
This algorithm has to be applied to every fragment XiO . An exemplary 1st
Step of the reconstruction algorithm is shown in Figure 3. The subinterval

23

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

O1 ∪ M1 is determined by the original interval O1 and the extend to which γ
can be estimated (see right panel). The function X̃i,1 shown in the left panel
still lacks the upper fragment for values u ∈ [77362 (MW), 82282(MW)] such
that a second step of the reconstruction algorithm is necessary.

70000

● ●●

●

40000

50000

30000

20

O1
30000

O1

40000

●

● ● ●

60000

Electr. Demand (MW)

100
80
60

●
●
●

● ●●
●●

O 1∪ M 1

50000

120

O 1∪ M 1

40

Electr. Prices (EUR/MWh)

140

160

80000

1st Run of the Reconstruction Algorithm

60000

Electr. Demand (MW)

70000

80000

30000

40000

50000

60000

70000

80000

Electr. Demand (MW)

Fig 3. Explanatory plots for the first run of the reconstruction algorithm.

This 2nd Step is shown in Figure 4. There the new interval O2 ⊆ O1 ∪ M1
is chosen such that the still missing upper fragment becomes reconstructible.
The new large interval O2 ∪ M2 contains the missing upper fragments, such
that we can stop the algorithm.
The choice of the subset Or in the rth step is crucial. On the one hand,
Or should be chosen as large as possible to contain as much information
as possible. On the other hand, Or must be chosen such that Mr contains
a still missing fragment which is—in tendency—met by smaller intervals
Or . That is, any efficient implementation of the algorithm and the choice of
rmax depends on the extend to which γ can be estimated. A simple practical
implementation is described in our application in Section 7.2.
In each iteration of the reconstruction algorithm we accumulate reconstruction errors. The following proposition provides a theoretical description
of this accumulation of reconstruction errors:
Proposition 5.1 (Accumulating reconstruction error). For simplicity,
let E(Xi (u)) = 0 for all u ∈ [a, b] and consider the second step of the reconstruction algorithm. Let XiM2 (u) denote a missing value that we aim to
reconstruct by [L(X̃iO2 )](u) using X̃iO2 which is taken from the reconstruc-

24

A. KNEIP & D. LIEBL

O2

40000

50000

60000
50000

O 2∪ M 2

30000

20

O 1∪ M 1
30000

O2

40000

60

80

100

Electr. Demand (MW)

120

70000

140

O 2∪ M 2

40

Electr. Prices (EUR/MWh)

80000

160

2nd Run of the Reconstruction Algorithm

60000

70000

80000

30000

40000

Electr. Demand (MW)

50000

60000

70000

80000

Electr. Demand (MW)

Fig 4. Explanatory plots for the second run of the reconstruction algorithm.

tion of the 1st Step. The mean squared reconstruction error can then be
approximated as following:


2 
2 
M2
O2
M2
O2
E Xi (u) − [L(X̃i )](u)
≤E
Xi (u) − [L(Xi )](u)

2 
M2
O1
+E
Xi (u) − [L(Xi )](u)
,
where L(XiO1 ) and L(XiO2 ) are the hypothetical reconstruction operators if
γ were fully observed over [a, b]2 , and XiO2 were observable.
That is, the mean squared reconstruction error in the second run of the
iterative algorithm is bounded from above by the two hypothetical mean
squared reconstruction errors of [L(XiO1 )](u) and [L(XiO2 )](u).
6. Simulation study. For our simulation study we generate n iid normal and n iid exponential random functions Xi (u) = µ(u) +√ξi1 φ1 (u) +
ξi2 φ2 (u), where µ(u) =
√ u + sin(u), φ1 (u) = − cos((πu)/(b − a))/ 5, φ2 (u) =
− cos(2(πu)/(b−a))/ 5, a = 1, and b = 10. For the normal case we let ξik ∼
N (0, λk ) with λ1 = 4 and λ2 = 3. For the exponential case we let ξik be a
centered exponential random variable with rate λk and centered by λ−1
k . Furiid

iid

thermore, Ui1 , . . . , Uim ∼ Unif[Ai , Bi ], where Ai ∼ Unif[a, a + (b − a) · 0.25]
iid

and Bi ∼ Unif[b−(b−a)·0.25, b]. Finally, the observations Yij are generated

25

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

according to Yij = Xt (Uij ) + εij with εij ∼ N (0, 0.2). The following data
dimensions are investigated: n ∈ {100, 200} and m ∈ {15, 25, 50, 75}. For
reconstructing the missing parts XiMi (u) with u ∈ Mi = [a, b] \ [Ai , Bi ], we
use the FPCA-based estimator LbK in (19). The estimate of the truncation
parameter K̂ := arg min AICK is selected by minimizing the AIC criterion
in (27). The bandwidths hX , hµ and hγ are determined using generalized
cross validation. In the jth (j = 1, . . . , B) of in total B = 100 simulation
runs we compute the Mean Absolute Reconstruction Error (MARE) as
MAREj = n−1

n
X
i=1

max XiMi (u) − [LbK̂ (XO
i )](u) ,

u∈Mi

j = 1, . . . , B

P
and finally report the averages MARE = B −1 B
j=1 MAREj . The implementation is done in R (R Core Team, 2017) and the R-codes are available
from the authors.
Exponential Case
●

n=100
n=200

0.9

n=100
n=200
●

●

●
●

●

0.6

●

0.6

●

●

0.7

●

0.8

●

MARE

0.8
0.7

MARE

0.9

Normal Case

m=15

m=25

●

●

m=50

m=75

●
●

m=15

m=25

m=50

●

m=75

Fig 5. Mean absolute reconstruction errors for the normal and exponential data generating
processes.

Figure 5 shows the MARE for the two different data generating processes
and the different data dimensions. The simulation results demonstrate that
the improvements in the MARE decline for larger values of m. This observation is in line with our asymptotic results. If m is small relatively to
b O ) (see
n, increasing m will significantly reduce the estimation error in D(X
i
Corollary 4.1). For relatively large values of m, however, the estimation error
b O ) does not depend anymore on m (see Theorem 4.2).
in D(X
i
Table 1 shows the fractions of correctly estimated K. The AIC criterion
correctly selects two eigen-components in at least 95% of the Monte Carlo
repetitions within each simulation scenario, except for the cases with m =
15, where there are at leat 78% of correct selections.

26

A. KNEIP & D. LIEBL
Table 1
Fractions of correctly estimated K
m = 15

m = 25

m = 50

m = 75

n = 100

Normal Case
Exponential Case

0.90
0.78

0.99
0.95

1.00
0.98

1.00
0.99

n = 200

Normal Case
Exponential Case

0.95
0.92

1.00
0.99

1.00
1.00

1.00
1.00

7. Application. The data for our analysis come from three different
sources. Hourly spot prices of the German electricity market are provided
by the European Energy Power Exchange (EPEX) (www.epexspot.com),
hourly values of Germany’s gross electricity demand and electricity exchanges with other countries are provided by the European Network of
Transmission System Operators for Electricity (www.entsoe.eu), and German wind and solar power infeed data are provided by the transparency
platform of the European energy exchange (www.eex-transparency.com).
For academic, i.e., non-commercial, purposes the data can be accessed free
of charge from these sources. The data dimensions are given by m = 24
hours and n = 241 working days between March 15, 2012 and March 14,
2013. Very few (0.4%) of the data pairs (Yij , Uij ) with prices Yij > 120
EUR/MWh and Uij > 82000 MW are considered as outliers and reset to
Yij = 120.
The German electricity market, like many other electricity markets, provides purchase guarantees for Renewable Energy Sources (RES). Therefore,
the relevant variable for pricing at the energy exchange is electricity demand minus electricity infeeds from RES (Nicolosi, 2010). Correspondingly,
the hourly values of electricity demand Uij actually denote residual electricity demand, i.e., electricity demand minus infeeds from RES. That is, Uij :=
Elect.Demandij − RESij , where RESij = Wind.Infeedij + Solar.Infeedij .
The effect of further RES such as biomass is still negligible for the German
electricity market.
7.1. Checking for perfect reconstructions. According to result (c) of Theorem 2.3 the reconstruction error can be zero if the structure of the random
functions is simple enough. In the following, we asses empirically whether
the functions can be reconstructed perfectly. The idea is to artificially partition the observed data XO
i into an observed and a pseudo missing half.
This way we can check whether the reconstructed part equals the artificially missing part, i.e., whether the functions are perfectly reconstructible.
In practice, however, we have to take into account the estimation errors,

27

80000

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

●
●

30000

40000

50000

60000

Electr. Demand (MW)

70000

80000

20

18

60000

70000

16

50000

●
●
● ●
●●
●●
●
● ●●
●● ●●●●
●●
●●
●●
●●
●
●
●
●
●●
●●●●●●● ●
●
●
●
● ●●●●
●●
●●●
●●
●
● ●
●
● ●●●
●●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●●
●●●
●●●● ● ● ●●●
●●
● ●
●
●
●● ●
●● ●●
● ●●● ●
●●●●
●● ●●
●
●
●●
●
●
●●●
●
●
●●● ●
●●
●
●
●●●●
●●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●●●●
●
●
●
●
●
●●
●●
●
●
●●
●
●
●
●●
●●●●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●
●●●●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
● ●
●●
●
●●
●
●
●●●●●●
●
●●
●
●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●
●
●
●
●●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●●
●
●
●●● ● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ●●
●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
● ●●●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ●●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
● ●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
● ● ●●
●●
●
●
●
●
●
●
●●
●
●
●
●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●●●
●
●
●●
●●
●
●
●
●
●
●
●
●
●
●●●● ●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●● ●
●
●●
●●●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●●
●●●
●
●●
●
●
●●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
● ● ●● ●
●
●
●
●●
●
●
●●●●
●
●
●●
●
●
●
●●
●
●
●
●
●●
●●
●
●
●●
●●
●
●●● ●
●●
●
●●
●
●●●
●
●
●
●
●●
●
● ●
●
●
●
●
●●
●●
●
●
●● ●● ●● ●●
●●●●
●●
●
●●
●
●●
●●
●
●
●●
●●
●●
●●●●
●●●
●
●
●●
●
●
●●
●
●
●●
●
●●
●●
●●
●●●●
●● ●●
●●
●
●
●
● ●●
●●
● ● ●
●●●●
●●●
●
●●● ●
●●● ●
●
●
●
●
●
●
●
●
●● ●
● ●● ●●●●●
●
●
●●●●
●●
● ●●
●●●
●●●
●●
●

14

12

40000

●

●
● ●
●
●

30000

●

●

Electr. Demand (MW)

80
60
40
0

20

Electr. Prices (EUR/MWh)

100

Estimated Meanfunction

10

●

● ●

16

30000

14

16
40000

50000

12

60000

10
70000

80000

Electr. Demand (MW)

Fig 6. Left Panel: Estimated mean function plus a scatter plot of the data pairs
(Yij , Uij ). Right Panel: Contour plot of the estimated covariance function. The white
regions reflect the outer off-diagonal parts which are infeasible to estimate.

which imply that the empirical reconstruction error Zbi (u) 6= 0 even though
the theoretical reconstruction error Zi (u) = 0 for all u ∈ M .
We take the lower halves of the data XO
i as the observed parts and use
them to reconstruct their pseudo missing upper halves and vice-versa which
leads to, in total, 2n checks. To account for the pre-smoothing estimation
error we use bootstrap (1 − αBonf ) confidence intervals with Bonferroni adjusted αBonf = 0.05/(2n) in order to account for the 2n = 482 checks.
Only if a reconstructed curve is within this confidence interval is the reconstruction considered “perfect”. The function-wise pre-smoothing is done
using the local linear kernel estimator, where the bandwidth is selected by
cross-validation. For reconstructing the pseudo missing halves we use the
FPCA-based estimator in (19). The left panel of Figure 7 demonstrates this
approach for the case where we get a “perfect” reconstruction of the pseudo
missing upper half given the information from the observed lower half of the
data XO
i . Using this approach, 96.5% of the 2n reconstructions are considered perfect, which is roughly in line with the chosen significance level of
α = 5% from which we expect 5% of the reconstructions to be falsely classified as “imperfect”. This suggests that we can expect good to very good
reconstructions using our reconstruction operator, although we emphasize
that our reconstructions are optimal among all linear reconstructions—no
matter whether the functions are perfectly reconstructible or not.

28

150

Pre−Smoothed Function
Reconstructed Function
Bonf. Adjusted 95% CI

100
0

50

60

80

Electr. Prices (EUR/MWh)

All Reconstructed Functions
Partially Observed Functions

40

Electr. Prices (EUR/MWh)

100

A. KNEIP & D. LIEBL

20

Pseudo Missing Half

45000

55000

65000

Electr. Demand (MW)

75000

30000

40000

50000

60000

70000

80000

Electr. Demand (MW)

Fig 7. Left Panel Checking whether the reconstructed function is within the Bonferroni
adjusted 95% confidence interval of the pre-smoothed function. Right Panel Recovered
functions (gray) and the original, partially observed (pre-smoothed) functions (black).

7.2. Empirical results. The estimated mean and covariance functions are
shown in Figure 6. The outer off-diagonal parts of the covariance function
γ cannot be estimated, since these parts of the domain are not covered by
data pairs (Uij , Uil ), j 6= l. In order to reconstruct the entire missing parts
XiM , we use the FPCA-based estimator in (19) and our iterative reconstruction algorithm 5.1, where our implementation is as follows. We use three
iterations for each partially observed price function. In the first step, we
use the information with respect to the original observations XO
i in order to
reconstruct the missing parts as far as possible. In the second step, we use
ˆ and try to reconthe upper half of the empirically reconstructed curve X̃
i,1
struct possibly further missing upper fragments. In the final step we use the
ˆ and try to reconstruct possibly further missing lower fraglower half of X̃
i,1
ments. This simple approach allows us to recover 91% of the price functions
over the total support. The reconstructed functions are shown in the right
panel of Figure 7. Our preceding analysis suggests that the reconstructed
functions are reliable estimates of the true price functions. Note that the
price functions with negative electricity prices are perfectly plausible as the
EPEX allows for negative prices (see, for instance, Nicolosi, 2010). Electricity producers are willing to sell electricity at negative prices (i.e., to pay
for taking the electricity) if shutting off and restarting their power plants is
more expensive than selling their electricity at negative prices.

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

29

References.
Bosq, D. (2000). Linear Processes in Function Spaces: Theory and Applications 149.
Springer Verlag.
Cai, T. T. and Hall, P. (2006). Prediction in functional linear regression. The Annals
of Statistics 34 2159–2179.
Cardot, H., Mas, A. and Sarda, P. (2007). CLT in functional linear regression models.
Probability Theory and Related Fields 138 325–361.
Delaigle, A. and Hall, P. (2013). Classification using censored functional data. Journal
of the American Statistical Association 108 1269–1283.
Delaigle, A. and Hall, P. (2016). Approximating fragmented functional data by segments of Markov chains. Biometrika 00 1–21.
Descary, M.-H. and Panaretos, V. M. (2017). Recovering covariance from functional
fragments. arXiv preprint arXiv:1708.02491.
Goldberg, Y., Ritov, Y. and Mandelbaum, A. (2014). Predicting the continuation
of a function with applications to call center data. Journal of Statistical Planning and
Inference 147 53–65.
Gromenko, O., Kokoszka, P., Sojka, J. et al. (2017). Evaluation of the cooling trend
in the ionosphere using functional regression with incomplete curves. The Annals of
Applied Statistics 11 898–918.
Hall, P. and Horowitz, J. L. (2007). Methodology and convergence rates for functional
linear regression. The Annals of Statistics 35 70–91.
Hall, P., Müller, H. G. and Wang, J. L. (2006). Properties of principal component
methods for functional and longitudinal data analysis. The Annals of Statistics 34
1493–1517.
Hirth, L. (2013). The market value of variable renewables: The effect of solar wind power
variability on their relative price. Energy Economics 38 218–236.
Horváth, L. and Kokoszka, P. (2012). Inference for Functional Data with Applications
200. Springer.
Kraus, D. (2015). Components and completion of partially observed functional data.
Journal of the Royal Statistical Society: Series B (Statistical Methodology) 77 777–801.
Li, Y. and Hsing, T. (2010). Uniform convergence rates for nonparametric regression and
principal component analysis in functional/longitudinal data. The Annals of Statistics
38 3321–3351.
Nicolosi, M. (2010). Wind power integration and power system flexibility – An empirical
analysis of extreme events in Germany under the new negative price regime. Energy
Policy 38 7257–7268.
R Core Team, (2017). R: A Language and Environment for Statistical Computing. R
Foundation for Statistical Computing.
Ramsay, J. O. and Silverman, B. W. (2005). Functional Data Analysis, 2. ed. Springer
Series in Statistics. Springer.
Rice, J. (1984). Bandwidth Choice for Nonparametric Regression. The Annals of Statistics
12 1215–1230.
Ruppert, D. and Wand, M. P. (1994). Multivariate locally weighted least squares regression. The Annals of statistics 1346–1370.
Weigt, H. (2009). Germany’s wind energy: The potential for fossil capacity replacement
and cost saving. Applied Energy 86 1857–1863.
Yao, F., Müller, H. G. and Wang, J. L. (2005a). Functional data analysis for sparse
longitudinal data. Journal of the American Statistical Association 100 577–590.
Yao, F., Müller, H.-G. and Wang, J.-L. (2005b). Functional linear regression analysis

30

A. KNEIP & D. LIEBL

for longitudinal data. The Annals of Statistics 33 2873–2903.
Zhang, X. and Wang, J.-L. (2016). From sparse to dense functional data and beyond.
The Annals of Statistics 44 2281–2321.
Alois Kneip and Dominik Liebl
Statistische Abteilung
University of Bonn
Adenauerallee 24-26
53113 Bonn, Germany
E-mail: akneip@uni-bonn.de
E-mail: dliebl@uni-bonn.de

Supplementary paper for:
On the Optimal Reconstruction of Partially
Observed Functional Data
by Alois Kneip and Dominik Liebl
CONTENT
In the following we give the proofs of our theoretical results. The main
steps in our proofs of Theorems 4.1 and 4.2 are as in Yao, Müller and
Wang (2005a). Though, by contrast to Yao, Müller and Wang (2005a), we
allow for a time series context (see Assumption A1), impose more restrictive
assumptions on the kernel function (see Assumption A5), and consider a
different asymptotic setup (see Assumption A2).
APPENDIX A: PROOFS
Proof of Theorem 2.1: For every linear operator L : L2 (O) → L2 (M )
that is a reconstruction operator with respect to XiO according to Def. 2.1,
we have that
(32)

V([L(XiO )](u)) =

∞
X

2
O
λO
k [L(φk )](u)

for every u ∈ M.

k=1

Existence: Writing [L(XiO )](u) as [L(XiO )](u) = hαu , XiO iH for some αu ∈ H
and computing again the variance of [L(XiO )](u) yields that
(33)

V([L(XiO )](u))

=

∞
X
k=1

λO
k



2
hαu , φO
k i2
λO
k

2
.

O 2
O
Since (32) and (33) must be equal, we have that [L(φO
k )](u) = hαu , φk i2 /λk
for all k ≥ 1, which establishes that there exits a αu ∈ H for every reconstruction [L(XiO )](u).
Uniqueness: Assume that there is an alternative α̃u ∈ H such that [L(φO
k )](u) =
O
2
O
O
2
O
hα̃u , φk i2 /λk for all k ≥ 1. Then hαu − α̃u , φk i2 /λk = 0 for all k ≥ 1
O
or equivalently hαu , φO
k i2 − hα̃u , φk i2 = 0 for all k ≥ 1 which shows that
α̃u − αu = 0.

Proof of Theorem 2.2, part (a): First, note that continuity of γ(u, v)
implies continuity of V([L(XiO )](u)). Second, note that for any K and every
I

II

A. KNEIP & D. LIEBL

u ∈ M , we have
(34)

0≤E



XiM (u) −

K
X

O O
ξik
φ̃k (u)

2

!
= γ(u, u) −

k=1

K
X

O
2
λO
k φ̃k (u) .

k=1

P
PK
O O
O O
2
But this implies that V(Lu,K (XiO )) = V( K
k=1 ξik φ̃k (u)) =
k=1 λk φ̃k (u)
O
converges to a fixed limit 0 ≤ V([L(Xi )](u)) < ∞ as K → ∞ for all u ∈ M .
Part (b): Follows directly from observing that E([L(XiO )](u)) = 0 for all
u ∈ M.
Proof of Theorem 2.3, part (a): For all v ∈ O and u ∈ M we have that


E XiO (v)Zi (u) = E XiO (v) XiM (u) − [L(XiO )](u) =
!
∞
∞


X
X
M
O O
O O
=
ξik φ̃k (u)
=E
ξik φk (v) Xi (u) −
k=1

=

∞
X

k=1



O
O
(u)
.
X
(u))
−
λ
φ̃
(v)
E(ξ
φO
k k
ik i
k

k=1
O O
O M
From the definition φ̃O
k (u) in (6) we get that E(ξik Xi (u)) = λk φ̃k (u),
which leads to E(XiO (v)Zi (u)) = 0 for all u ∈ M . This proves (12), while
(13) directly follows from the definition of Zi (u).

Part (b): By Theorem 2.1 there exists a unique bu ∈ H such that
[`(XiO )](u) = hbu , XiO iH .
By (12) and the orthogonality property of the least squares projection we
thus obtain

2 
E Xi (u) − [`(XiO )](u)
=

2 
= E [L(XiO )](u) + Zi (u) − hbu , XiO iH
=


2
+ E(Zi (u)2 )+
= E [L(XiO )](u) − hbu , XiO iH

+ 2 E([L(XiO )](u) Zi (u)) − E hbu , XiO iH Zi (u) =

2 
= E [L(XiO )](u) − hbu , XiO iH
+ E(Zi2 (u)) ≥ E(Zi2 (u)).
Part (c): Observe that V(Zi (u)−Zj (u)) = V(Zi (u))+V(Zj (u))−2 Cov(Zi (u), Zj (u)) =
2 V(Zi (u)) for all u ∈ M and i 6= j. Rearranging and using that E(Zi (u)) =
E(Zj (u)) = 0 for all u ∈ M and all i, j ∈ {1, . . . , n} yields V(Zi (u)) =

III

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS
1
2

E((Zi (u) − Zj (u))2 ). From result (a) we know that Zi (u) and XiO (v) are
orthogonal and therefore
uncorrelated for

 all u ∈ M and all v ∈ O, that
is, E XiO (v)Zi (u) = Cov XiO (v), Zi (u) = 0. Under the assumption of an
independent Gaussian process, we have then independence between Zi (u)
and XiO (v), such that
V(Zi (u)) =



1
E E (Zi (u) − Zj (u))2 |XiO = XjO ,
2

where XiO = XjO means that XiO (u) = XiO (u) for all u ∈ O. For the
two random functions Zi (u) and Zj (u) we can write Zi (u) = XiM (u) −
[L(XiO )](u) and Zj (u) = XjM (u) − [L(XjO )](u). It follows from the definition
O
O
of φ̃O
k (u) in (6) that [L(Xi )](u) = [L(Xj )](u) for all u ∈ M , if and only if
O
O
Xi = Xj . Therefore,
V(Zi (u)) =



1
E E (XiM (u) − XjM (u))2 |XiO = XjO ,
2

for all u ∈ M .
Proof of Theorem 4.1 For proofing the results in Theorem 4.1 we make
use of the following two lemmas:
Lemma A.1.
(35)

Define



Uij − u
1 X
Ψq,nm (u; hµ ) =
κ
ψq (Uij − u, Yij ) ,
nmhµ
hµ
ij

where

q

 (Uij − u)
ψq (Uij − u, Yij ) =
Yij


(Uij − u) Yij

for q ∈ {0, 1, 2}
for q = 3
for q = 4.

Then, under Assumptions A1-A5,
τq,nm = sup |Ψq,nm (u; hµ ) − mq (u)| = Op
u∈[a,b]

1
1
+√
h2µ + p
n
nm hµ

!
,

where m0 (u) = fU (u), m1 (u) = 0, m2 (u) = fU (u)ν2 (κ), m3 (u) = µ(u)fU (u) =
E(Yij |Uij = u)fU (u), and m4 (u) = 0.

IV

A. KNEIP & D. LIEBL

Lemma A.2.

Define

(36)
Θq,nM (u, v; hγ ) =

X  Uij − u   Uil − v 
1
κ
κ
ϑq (Uij − u, Uil − u, Cijl ) ,
nM hγ
hγ
hγ
i,j6=l

where

q
q

for q ∈ {0, 1, 2}
 (Uij − u) (Uil − v)
ϑq (Uij − u, Uil − v, Cijl ) =
Cijl
for q = 3


(Uij − u) (Uil − v) Cijl for q = 4.
Then, under Assumptions A1-A5,

%q,nM =

sup
(u,v)∈[a,b]2

|Θq,nM (u, v; hγ ) − ηq (u, v)| = Op h2γ + q


1

1
+ √ ,
n
nM h2γ

where η0 (u, v) = fU U (u, v), η1 (u, v) = 0, η2 (u, v) = fU U (u, v)(ν2 (κ))2 ,
η3 (u, v) = γ(u, v)fU U (u, v) = E(Cijl |(Uij , Uil ) = (u, v))fU U (u, v), and
m4 (u, v) = 0.

Proof of Lemma A.1: Remember that E(|τq,nm |) = O(ratenm ) implies
that τq,nm = Op (ratenm ), therefore, we focus in the following on E(|τq,nm |),
where E(|τq,nm |) = E(τq,nm ). Adding a zero and applying the triangle inequality yields that E(τq,nm ) =
E( sup |Ψq,nm (u; hµ ) − mq (u)|) ≤ sup | E(Ψq,nm (u; hµ )) − mp (u)|+
u∈[a,b]

u∈[a,b]

+ E( sup |Ψq,nm (u; hµ ) − E(Ψq,nm (u; hµ ))|).

(37)

u∈[a,b]

Let us first focus on the second summand in (37). The next steps will make
use of the Fourier transformation of the kernel function κ (see, e.g., Tsybakov, 2008, Ch. 1.3):
Z
Z 1
ft
κ (x) := F[κ](x) =
κ(z) exp(−izx)dz =
κ(z) exp(−izx)dz
R

√

−1

with i = −1. By Assumption A5, κ(.) has a compact support [−1, 1]. The
inverse transform gives then
Z
Z
1
1
κft (x) exp (ixs) dx =
κft (x) exp (ixs) dx 1(|s|<1) .
κ (s) =
2π R
2π R

V

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

Furthermore, we can use that (see Tsybakov, 2008, Ch. 1.3, (1.34)) F[κ(./hµ )/hµ ](x) =
F[κ](xhµ ) = κft (xhµ ) which yields
Z
1
F[κ(./hµ )/hµ ](x) exp (ixs) dx 1(|s|<hµ )
κ (s/hµ ) /hµ =
2π R
Z
1
=
(38)
κft (xhµ ) exp (ixs) dx 1(|s|<hµ ) .
2π R
Plugging (38) into (35) yields Ψq,nm (u; hµ ) =


Uij − u 1
1 X
ψq (Uij − u, Yij )
=
κ
nm
hµ
hµ
ij
Z

1 X 1
=
κft (xhµ ) exp ix(Uij − u) dx 1(|Uij −u|<hµ ) ψq (Uij − u, Yij )
nm
2π R
ij


Z
X


1
1

=
exp ixUij ψq (Uij − u, Yij ) 1(|Uij −u|<hµ )  exp ixu κft (xhµ )dx.
2π R nm
ij

Using that | exp(ixu)| ≤ 1 leads to
1
E
E( sup |Ψq,nm (u; hµ ) − E(Ψq,nm (u; hµ ))|) ≤
2π
u∈[a,b]

!

Z

ft

ω̃q,nm (u, x) · κ (xhµ )dx

sup
u∈[a,b]

,

R

where
ω̃q,nm (u, x) =


1 X
exp ixUij ψq (Uij − u, Yij ) 1(|Uij −u|<hµ ) −
nm
ij



E exp ixUij ψq (Uij − u, Yij ) 1(|Uij −u|<hµ ) .

Using further that κft is
by Assumption
 symmetric, since κ is symmetric

A5, and that exp ixUij = cos xUij + i sin xUij leads to
1
E
2π

!

Z

ω̃q,nm (u, x) · κ (xhµ )dx

sup
u∈[a,b]

ft

R

1
E
=
2π

!

Z

ωq,nm (u, x) · κ (xhµ )dx

sup
u∈[a,b]

R

where
ωq,nm (u, x) =
(39)

ft


1 Xh
cos xUij ψq (Uij − u, Yij ) 1(|Uij −u|<hµ ) −
nm
ij

i

E cos xUij ψq (Uij − u, Yij ) 1(|Uij −u|<hµ ) ,

,

VI

A. KNEIP & D. LIEBL

such that
E( sup |Ψq,nm (u; hµ ) − E(Ψq,nm (u; hµ ))|)
u∈[a,b]

1
≤
2π

!

Z
sup ωq,nm (u, x)

E
R

v 
Z u
u
1
u 
tE
≤
2π R
v
Z u
u
1
tE
=
2π R

(40)

· κft (xhµ ) dx

u∈[a,b]

!2 
 · κft (xhµ ) dx

sup ωq,nm (u, x)
u∈[a,b]

!
sup

ωq,nm (u, x)

2

· κft (xhµ ) dx.

u∈[a,b]

In order to simplify the notation we will denote

Wijq (x, u) = cos xUij ψq (Uij − u, Yij ) ,


such that E supu∈[a,b] (ωq,nm (u, x))2 =

E

i2
1 Xh q
q
W
(x,
u)
1
−
E(W
(x,
u)
1
)
+
(|Uij −u|<hµ )
(|Uij −u|<hµ )
ij
ij
2
u∈[a,b] (nm)
ij
X  q
1
(Wij (x, u)1(|Uij −u|<hµ ) − E(Wijq (x, u)1(|Uij −u|<hµ ) ))·
(nm)2
(i,j)6=(r,l)

.
·(Wrlq (x, u)1(|Url −u|<hµ ) − E(Wrlq (x, u)1(|Url −u|<hµ ) ))
sup



As u takes only values within the compact interval [a, b], there exist constants C1 and C2 such that, uniformly for all u ∈ [a, b], P(|Uij − u| < hµ ) ≤
C1 hµ < ∞, for all i, j, and P(|Uij − u| < hµ AND |Url − u| < hµ ) ≤ C2 h2µ <
∞, for all (i, j) 6= (r, l). Together
 with the triangle inequality, this yields
2
that E supu∈[a,b] (ωq,nm (u, x)) ≤
C1 hµ X
E
(nm)2 ij
C2 h2µ
(nm)2

!

2
sup Wijq (x, u) − E(Wijq (x, u))

+

u∈[a,b]

X
(i,j)6=(r,l)

E

!
 q

q
q
q
sup (Wij (x, u) − E(Wij (x, u)))(Wrl (x, u) − E(Wrl (x, u)))

u∈[a,b]

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

VII

From our moment assumptions (Assumption A1) and the fact that [a, b]
is compact, we can conclude that there must exist a constant C3 such that,
point-wise for every x ∈ R,

2 
(41)
E
sup Wijq (x, u) − E(Wijq (x, u))
≤ C3 < ∞
u∈[a,b]

for all i and j.
“Within function” dependencies: By the same reasoning there must exist
a constant C4 such that, point-wise for every x ∈ R,


E sup Wijq (x, u) − E(Wijq (x, u)) · sup Wilq (x, u) − E(Wilq (x, u))
u∈[a,b]

u∈[a,b]

(42)
≤ C4 < ∞
for all j 6= l and all i.
“Between function” dependencies: Our weak dependency assumption (Assumption A1) and the fact that [a, b] is compact yields that point-wise for
every x ∈ R


E sup Wijq (x, u) − E(Wijq (x, u)) · sup Wrlq (x, u) − E(Wrlq (x, u))
u∈[a,b]

u∈[a,b]

(43)
|i−r|

≤ c1 ι1

for all j, l and |i − r| ≥ 1, where 0 < c1 <
 ∞ and 0 < ι1 < 1.

Eq.s (41), (42), and (43) yield that E supu∈[a,b] (ωq,nm (u, x))2 ≤
C2 h2µ X
C2 h2µ X
C1 hµ X
|i−r|
C
+
C
+
c1 ι1
4
3
2
2
2
(nm)
(nm)
(nm)
ij
i,j6=l
i6=r,jl
!
!
2
2
hµ (m − 1) hµ
h2µ
hµ
hµ
+
+
=O
+
,
=O
nm
nm
n
nm
n

≤

such that
(44)

v
u
u
tE

!
sup (ωq,nm (u, x))2
u∈[a,b]

r
=O

hµ
hµ
+√
nm
n

!
.

VIII

A. KNEIP & D. LIEBL

Plugging (44) into (40) and integration by substitution leads to
E( sup |Ψq,nm (u; hµ ) − E(Ψq,nm (u; hµ ))|) ≤
u∈[a,b]

(45)
v
Z u
u
1
tE
2π R

!
2

sup (ωq,nm (u, x))

· κft (xhµ ) dx = O

u∈[a,b]

1
1
p
+√
n
nm hµ

!
.

Let us now focus on the first summand in (37). From standard arguments
in nonparametric statistics (see, e.g., Ruppert and Wand (1994)) we know
that
E(Ψq,nm (u; hµ )) − mq (u) = O(h2µ )
for each u ∈ [a, b] and for all q ∈ {0, . . . , 4}. Under our smoothness Assumption A3, the “O(h2µ )” term becomes uniformly valid for all u ∈ [a, b] and all
q ∈ {0, 1, 2, 4}, since all of the involved functions have uniformly bounded
second order derivatives. We can conclude with respect to the first summand
in (37) that
(46)

sup | E(Ψq,nm (u; hµ )) − mp (u)| = O(h2µ )

for all

q ∈ {0, . . . , 4}.

u∈[a,b]

Finally, plugging our results (45) and (46) into (37) leads to
(47) τq,nm = sup |Ψq,nm (u; hµ ) − mq (u)| = Op
u∈[a,b]

1
1
h2µ + p
+√
n
nm hµ

!

for all q ∈ {0, . . . , 4}.
Proof of Lemma A.2: Analogously to that of Lemma A.1.
Proof of Theorem 4.1, part (a): Let us rewrite the estimator µ̂ using
matrix notation as in Ruppert and Wand (1994), i.e.,
(48)


−1
>
µ̂(u; hµ ) = e>
[1,
U
]
W
[1,
U
]
[1, Uu ]> Wµ,u Y,
u
µ,u
u
1

where e1 = (1, 0)> , [1, Uu ] is a nm × 2 dimensional data matrix with typical
rows (1, Uij − u), the nm × nm dimensional diagonal weighting matrix Wµ,u
−1
holds the kernel weights Kµ,h (Uij − u) = h−1
µ κ(hµ (Uij − u)). The objects

IX

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

Uu and Wµ,u are filled in correspondence with the nm dimensional vector
Y = (Y11 , Y12 , . . . , Yn,m−1 , Yn,m )> .
This way we can decompose the estimator µ̂(u; hµ ) as
−1
µ̂(u; hµ ) = e>
1 L1,nm,u L2,nm,u ,

(49)
with 2 × 2 matrix

L1,nm,u = (nm)−1 [1, Uu ]> Wµ,u [1, Uu ]



P
Uij −u
1
κ
nmhµ ij
hµ
=
P
Uij −u
1
(Uij − u)
ij κ
nmhµ
hµ







P
Uij −u
1
ij κ
nmhµ
hµ  (Uij − u) 

,
P
Uij −u
1
(Uij − u)2
ij κ
nmhµ
hµ

and 2 × 1 vector

L2,nm,u = (nm)−1 [1, Uu ]> Wµ,u Y = 




P
Uij −u
Y
ij
ij κ

hµ
.
P
Uij −u
(U
−
u)
Y
ij
ij
ij κ
hµ

1
nmhµ
1
nmhµ

Using the notation and the results from Lemma A.1 we have that
!
Ψ0,nm (u; hµ ) Ψ1,nm (u; hµ )
L1,nm,u =
Ψ1,nm (u; hµ ) Ψ2,nm (u; hµ )
(50)
=

fU (u)
0
0
fU (u)ν2 (κ)

!

!
Ψ3,nm (u; hµ )
=
Ψ4,nm (u; hµ )

!
µ(u)fU (u)
+ OpUnif
0

+ OpUnif

1
1
h2µ + p
+√
n
nm hµ

!
and

(51)
L2,nm,u =

1
1
h2µ + p
+√
n
nm hµ

!

where we write Ψq,nm (u; hµ ) − mq (u) = OpUnif (rate) in order to denote that
supu∈[a,b] |Ψq,nm (u; hµ )−mq (u)| = Op (rate). Taking the inverse of (50) gives
(52)
L−1
nm,u

=

!
1/fU (u)
0
+ OpUnif
0
1/(fU (u)ν2 (κ))

h2µ

1
1
+p
+√
n
nm hµ

Plugging (52) and (51) into (49) leads to

sup |µ̂(u; hµ ) − µ(u)| = Op
u∈[a,b]

h2µ

1
1
+p
+√
m
nm hµ

!
.

!
.

,

X

A. KNEIP & D. LIEBL

Proof of Theorem 4.1, part (ã): Observe that
biO (u; hµ , hX ) − XiO (u)| ≤
sup |X

u∈O

sup |X̂ic,O (u; hX ) − (XiO (u) − µ(u))| + sup |µ̂(u; hµ ) − µ(u)|.

u∈O

u∈O

From Theorem 4.1,ppart (a), we have that supu∈O |µ̂(u; hµ ) − µ(u)| = Op (rµ )
√
with rµ = h2µ + 1/ nm hµ + 1/ m. From a simplified version of the proof
of Theorem 4.1, part (a), with n = 1, it follows that


1
sup |X̂ic,O (u; hX ) − (XiO (u) − µ(u))| = Op h2X + √
mhX
u∈O
Proof of Theorem 4.1, part (b):
Let us rewrite the estimator γ̂ using matrix notation as in Ruppert and
Wand (1994), i.e., γ̂(u, v; hγ ) =
−1
>
(53)
[1, Uu , Uv ]> Wγ,u,v Ĉ,
= e>
1 [1, Uu , Uv ] Wγ,u,v [1, Uu , Uv ]
where e1 = (1, 0, 0)> , [1, Uu , Uv ] is a nM × 3 dimensional data matrix with
typical rows (1, Uij −u, Uil −v), the nM ×nM dimensional diagonal weighting
matrix Wγ,u,v holds the bivariate kernel weights Kγ,h (Uij −u, Uil −v). For the
bivariate kernel weights Kγ,h (z1 , z2 ) = h−2
γ κγ (z1 , z2 ) we use a multiplicative
kernel function κγ (z1 , z2 ) = κ(z1 )κ(z2 ) with κ as defined above. The usual
kernel constants are then ν2 (κγ ) := (ν2 (κ))2 and R(κγ ) := R(κ)2 . The rows
of the matrices [1, Uu , Uv ] and Wγ,u,v are filled in correspondence with the
nM elements of the vector of raw-covariances Ĉ = (. . . , Ĉijl , . . . )> .
Let us initially consider the infeasible estimator γ̂C that is based on the
infeasible “clean” raw-covariances Cijl = (Yij − µ(Uij ))(Yil − µ(Uil )) instead
of the estimator γ̂ in (53) that is based on the “dirty” raw-covariances Ĉijl =
(Yij − µ̂(Uij ))(Yil − µ̂(Uil )). Equivalently to the estimator µ̂ above, we can
write the estimator γ̂C as
−1
γ̂C (u, v; hγ ) = e>
1 S̃1,nM,(u,v) S̃2,nM,(u,v) ,

(54)
with
−1
S̃1,nM,(u,v)

=

(55)

=

!−1
Θ0,nM (u, v; hγ ) Θ1,nM (u, v; hγ )
Θ1,nM (u, v; hγ ) Θ2,nM (u, v; hγ )


!
1
1
1/fU U (u, v)
0
+ OpUnif h2γ + q
+√ 
n
0
1/fU U (u, v)(ν2 (κ))2
nM h2γ

XI

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

and S̃2,nM,(u,v) =
(56)
=

Θ3,nM (u, v; hγ )
Θ4,nM (u, v; hγ )

!
=



!
1
1
γ(u, v)fU U (u, v)
+ OpUnif h2γ + q
+ √ ,
n
2
0
nM hγ

where we use the notation and the results from Lemma A.2, and where
we write Θq,nM (u, v; hγ ) − ηq (u, v) = OpUnif (rate) in order to denote that
sup(u,v)∈[a,b]2 |Θq,nM (u, v; hγ ) − ηq (u, v)| = Op (rate).
Plugging (55) and (56) into (54) leads to


1
1
sup |γ̂C (u, v; hµ ) − γ(u, v)| = Op h2γ + q
(57)
+ √ .
n
2
(u,v)∈[a,b]
nM h2
γ

It remains to consider the additional estimation error, which comes from
using the “dirty” response variables Ĉijl instead of “clean” dependent variables Cijl . Observe that we can expand Ĉijl as following:
Ĉijl = Cijl + (Yij − µ(Uij ))(µ(Uil ) − µ̂(Uil ))
+ (Yil − µ(Uil ))(µ(Uij ) − µ̂(Uij ))
+ (µ(Uij ) − µ̂(Uij ))(µ(Uil ) − µ̂(Uil )).
Using our finite moment assumptions on Yij (Assumption A1) and our result
in Theorem 4.1, part (a), we have that
!
1
1
+√
Ĉijl = Cijl + Op (1)Op h2µ + p
n
nm hµ
!
1
1
+ Op (1)Op h2µ + p
+√
n
nm hµ
!!2
1
1
+ Op h2µ + p
+√
= Cijl + Op
n
nm hµ

h2µ

1
1
+√
+p
n
nm hµ

!
,

uniformly for all j 6= l ∈ {1, . . . , m} and i ∈ {1, . . . , n}. Therefore


1
1
1
sup |γ̂(u, v; hµ ) − γ(u, v)| = Op h2γ + h2µ + q
+q
+ √ .
n
2
(u,v)∈[a,b]
nM h2γ
nm h2µ

Proof of Theorem 4.1, parts (c) and (d): Part (c) follows directly from
O
inequality supk≥1 |λ̂O
k − λk | ≤ ||γ̂ − γ||2 ; see inequality (4.43) in Bosq (2000).
Part (d) follows directly from Lemma 4.3 in Bosq (2000).

XII

A. KNEIP & D. LIEBL

In the following let O := Oi = [Ai , Bi ] for some i ∈ 1, . . . , n. By assumption of Theorem 4.2 we have Bi − Ai ≥ `min , and recall that by Assumption
(A1) the structure of a function Xi , to be observed on Oi , does not depend
on the specific interval Oi .
For the proof of Theorem 4.2 we need some additional lemmas. Generally
note that under the assumed choice of bandwidths we have rµ + rγ  rmn
for rmn = min{n1/21,(nM )1/3 } , since for all n and m  nθ sufficiently large we
have (mn)2/5 ≥ min{n1/2 , (nM )1/3 }.
hφ̂O
ˆ
k ,γ̂u i2
Recall that φ̃O
and φ̃O
O
k (u) =
k (u) =
λ̂k

hφO
k ,γu i2
λO
k

for u ∈ O ∪ M , where

ˆ
O
in the particular case of u ∈ O we additionally have φ̃O
k (u) = φ̂k (u) and
aO +3/2
O
φ̃O
rmn = O(1) and that by (A6) we
k (u) = φk (u). Also recall that K̄mn
O
−a
−1
O
O
have δk = O(k
) as well as 1/δk = O(k aO +1 ).
Lemma A.3. Under the assumptions of Theorem 4.2 we have for all
1 ≤ k ≤ K ≤ K̄mn


ˆ
O
1/2
sup sup δkO |φ̃O
(u)
−
φ̃
(u)|
=
O
K
r
(58)
p
mn ,
k
k
u∈O∪M 1≤k≤K

Proof of Lemma A.3: Using results (b) and (c) of Theorem 4.1 we obtain
hφ̂O
ˆ
k , γu i2
(59) φ̃O
+ R1,k (u),
k (u) =
λ̂O
k

sup

sup λO
k R1,k (u) = Op (rmn ) .

u∈O∪M 1≤k≤K

But by the established properties
(12) in Theorem 2.3) of our
P∞ (inOparticular
O (u)φO (v) for all u ∈ O ∪ M and
λ
φ̃
operator we have γ(u, v) =
j
j=1 j j
v ∈ O. Hence
(60)

hφ̂O
k , γu i2
λ̂O
k

=

∞
1 X

λ̂O
k j=1

O
O
O
λO
j φ̃j (u)hφ̂k , φj i2

Now note that for all j ≥ 1
Z
O
O
O
λO
h
φ̂
,
φ
i
=
γ(u, v)φ̂O
j
j 2
k
k (u)φj (v)dudv
2
O
Z
O O
O
O
(61)
= λ̂k hφ̂k , φj i2 +
(γ(u, v) − γ̂(u, v))φ̂O
k (u)φj (v)dudv
O2

R
Let vγ̂,γ,k (u) := O (γ(u, v) − γ̂(u, v))φ̂O
k (u)du. By the orthonormality of the
O , . . . of eigenfunctions, the Cauchy-Schwarz inequality, and
system φO
,
φ
1
2

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

XIII

Theorem 4.1 we have
∞ Z
X

sup

= sup

∞
X

1≤k≤K j=1

(62)

≤ sup

(γ(u, v) −

O2

1≤k≤K j=1

O
γ̂(u, v))φ̂O
k (u)φj (v)dudv

2

2
2
hvγ̂,γ,k , φO
j i2 ≤ sup hvγ̂,γ,k , vγ̂,γ,k i2
1≤k≤K


2
sup |γ(u, v) − γ̂(u, v)|2 = Op rmn
.

1≤k≤K u,v∈O

And since by Assumption (A7) supj supu∈O∪M |φ̃O
j (u)| ≤ DO < ∞ the
Cauchy-Schwarz inequality yields
(63)
sup

K
X

sup

u∈O∪M 1≤k≤K j=1

|φ̃O
j (u)

Z
O2



O
1/2
(u)φ
(v)dudv|
=
O
K
r
(γ̂(u, v) − γ(u, v))φ̂O
p
mn ,
j
k

O
O
By (c) of Theorem 4.1, (61), (63), hφO
k − φk , φj i2 = 0 for j 6= k, and
O
O
hφO
k − φk , φk i2 = 1 relation (60) can thus be rewritten in the form

hφ̂O
k , γu i2
λ̂O
k
(64)

= φ̃O
k (u) +
+

1
λO
k

K
X

j=1
∞
X

O
O
φ̃O
j (u)hφ̂k − φk , φj i2

O
O
O
O
λO
j φ̃j (u)hφ̂k − φk , φj i2 + R2,k (u),

j=K+1

where

sup



1/2
rmn ,
sup λO
k R2,k (u) = Op K

u∈O∪M 1≤k≤K

Result (d) of Theorem 4.1 additionally implies
(65)
sup (δkO )2

1≤k≤K

∞
X


O
O 2
O 2 O
O
O
O 2
2
hφ̂O
k − φk , φj i2 ≤ (δk ) hφ̂k − φk , φ̂k − φk i2 = Op rmn ,

j=1

and, similar to (63), the Cauchy-Schwarz inequality leads to
(66)

sup

sup δkO |

u∈O∪M 1≤k≤K

K
X
j=1



O
O
1/2
φ̃O
(u)h
φ̂
−
φ
,
φ
i
|
=
O
K
r
j 2
p
mn .
j
k
k

XIV

A. KNEIP & D. LIEBL

By our assumptions on the sequence of eigenvalues we have for all u ∈ O ∪M
∞
X

2 O
2
2
(λO
j ) φ̃j (u) ≤ DO

j=K+1

= O(

∞
X

2
(λO
j )

j=K+1
∞
X

2
j −2aO ) = O(K −2aO +1 ) = O(K(λO
K) )

j=K+1

When combining this result with (65), a further application of the CauchySchwarz inequality yields
sup |δkO

sup

u∈O∪M 1≤k≤K

∞
1 X O O
O
O
λj φ̃j (u)hφ̂O
k − φk , φj i2 |
λO
k j=K+1

(67)
≤ δkO
Since

δkO
λO
k

∞
∞


X
X
1
2
O 2 1/2
O
O
O 2 1/2
1/2
(D
(λ
)
)
(
h
φ̂
−
φ
,
φ
i
)
=
O
K
r
p
mn .
O
j
j 2
k
k
λO
k
K+1
j=k+1

→ 0 as k → ∞, the desired result is an immediate consequence of

(59) - (67).
A technical difficulty in the proof of Theorem 4.2 consists in the fact that
φ̂O
k and the observations (Yij , Uij ) corresponding to the selected i ∈ 1, . . . , n.
But let γ̂−i (t, s) ≡ γ̂−i (t, s, hγ ) denote the estimate of the covariance matrix
when eliminating the m observations {(Yij , Uij )}j=1,...,m from the sample,
O
and let λ̂O
k,−i and φ̂k,−i , k = 1, 2, . . . , denote eigenvalues and eigenfunctions
of the corresponding covariance operator. Although in our time series context there may still exist dependencies between Xi and φ̂O
k,−i , our assumpO
tions imply that then all φ̂k,−i are independent of the particular samples
{ij }j=1,...,m and {Uij }j=1,...,m . The following Lemma now provides bounds
hφ̂O
hφ̂O
ˆ
ˆO
k,−i ,γ̂−i;u i2
k ,γ̂u i2
for the differences between φ̃O
(u)
=
and
φ̃
(u)
=
,
O
O
k
k,−i
λ̂k

λ̂k,−i

where γ̂−i;u (v) := γ̂−i (u, v; hγ ).
Lemma A.4. Under the assumptions of Theorem 4.2 we have for all
1 ≤ k ≤ K ≤ K̄mn


a) sup(u,v)∈[a,b]2 γ̂(u, v; hγ ) − γ̂−i (u, v; hγ ) = Op n1/21r
.
mn




1
1
Ok = O
O −λ̂O | = O
b) supk≤K δkO kφ̂O
−
φ̂
,
sup
|
λ̂
p n1/2 r
p n1/2 r
k≤K
k,−i
k 2
k,−i
mn
mn
 a +3/2
 k
O
1
K
O
c) supu∈O sup1≤k≤K δk |φ̂k,−i (u) − φ̂k (u)| = Op n1/2 r
= Op ( n1/2 )
mn

XV

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

Proof of Lemma A.4: Based on the definitions and techniques introduced
in the proof of Theorem 4.1 and Lemma A.2 it is immediately seen that
uniform rates of convergence of γ̂(u, v; hγ ) − γ̂−i (u, v; hγ ) can be derived
by considering the following difference: For any (u, v) ∈ O2 the estimator
σ̂(u, v)
sup
(u,v)∈[a,b]2

sup
(u,v)∈[a,b]2

sup
(u,v)∈[a,b]2

Θq,nM (u, v; hγ ) − Θ−i
q,nM (u, v; hγ ) ≤
 


Uij − u
Uil − v
1 X
κ
ϑq (Uij − u, Ui∗ l − u, Ci∗ jl ) +
κ
nM h2γ
hγ
hγ
j6=l

 

X
Ukj − u
1
Ukl − v
1
κ
κ
ϑq (Ukj − u, Ukl − u, Ckjl ) .
n(n − 1) M h2γ
hγ
hγ
k6=i,j6=l

Using similar arguments as in the proof of Lemma A.1, leads to


1
q 1
sup
Θq,nM (u, v; hγ ) − Θ−i
+√ 
q,nM (u, v; hγ ) = Op
n2
(u,v)∈[a,b]2
n2 M h2γ
which implies


sup
(u,v)∈[a,b]2


1
γ̂(u, v; hγ ) − γ̂−i (u, v; hγ ) = Op  q
+ √ ,
2
2
n2
n M hγ
1

and assertion a) of the Lemma is an immediate consequence.
The inequalities used to prove (c) and (d) of Theorem 4.1 now lead to
O
−1/2 r
O
O
O
supk≥1 |λ̂O
mn and sup1≤k≤K δ̂k,−i kφ̂k,−i − φ̂k k2 =
k,−i − λ̂k | = Op n

O
O
Op n−1/2 rmn , where δ̂k,−i
:= minj6=k {λ̂O
j − λ̂k }. By (c) of Theorem 4.1,
a +3/2

O
our assumptions on λj , and k ≤ K ≤ K̄mn with K̄mn
rmn = O(1) we
−1
O
O
additionally have δ̂k,−i (δk,−i ) = 1 + op (1). This proves assertion b) of the
Lemma.
O
hφ̂O
O (u) = hφ̂k,−i ,γ̂−i;u i2
k ,γ̂u i2
Furthermore, we have φ̂O
(u)
=
as
well
as
φ̂
O
O
k
k,−i

λ̂k

λ̂k,−i

for u ∈ O. The difference can be rewritten in the form
O
φ̂O
k (u) − φ̂k,−i (u) =

(68)

O
hφ̂O
k − φ̂k,−i , γu i2

λ̂O
k
+

+

O
hφ̂O
k − φ̂k,−i , γ̂u − γu i2

hφ̂O
k,−i , γ̂u − γ̂−i;u i2
λ̂O
k

λ̂O
k
+

O
O
(λ̂O
k,−i − λ̂k )hφ̂k,−i , γ̂−i;u i2
O
λ̂O
k λ̂k,−i

.

XVI

A. KNEIP & D. LIEBL

When analyzing the terms in (68) first note that by assertions a) and b) of
the lemma, and by (c) of Theorem 4.1
hφ̂O
k,−i , γ̂u − γ̂−i;u i2

sup sup

λ̂O
k,−i

u∈O 1≤k≤K


= Op

1 rmn
1/2
λO
K n




= Op

K aO rmn
n1/2



(69)
O
hφ̂O
k − φ̂k,−i , γ̂u − γu i2

sup sup

λ̂O
k

u∈O 1≤k≤K


= Op

2
K 2aO +1 rmn
n1/2




= op

K aO rmn
n1/2



O
The differences between the eigenfunctions φ̂O
k,−i (u) and φ̂k (u) reflect the
elimination of one single curve, and it is immediately clear that the convergence results of Theorem 4.1 and all arguments of Lemma A.3 remain
valid when considering estimated covariances γ̂−i (u, v), eigenvalues λ̂O
k,−i ,
O
and eigenfunctions φ̂k,−i (u) of the reduced sample. It thus follows from

Lemma A.3 and our assumption on K ≤ K̄mn that

hφ̂O
k,−i ,γ̂−i;u i2
λ̂O
k,−i

is asymp-

totically uniformly bounded over all u ∈ O ∪ M and k ≤ K. Hence,
(70)
sup

sup

O
O
(λ̂O
k,−i − λ̂k )hφ̂k,−i , γ̂−i;u i2
O
λ̂O
k λ̂k,−i

u∈O∪M 1≤k≤K


= Op

1 rmn
1/2
λO
K n




= Op

K aO rmn
n1/2

The first term on the right side of (68) can now be analyzed by generalizing
the arguments of Lemma A.3. Similar to (60) we obtain
O
hφ̂O
k − φ̂k,−i , γu i2

λ̂O
k

=

∞
1 X

λ̂O
k

O
O
O
O
λO
j φ̃j (u)hφ̂k − φ̂k,−i , φj i2 ,

j=1

while (61) becomes
O
O
O
O
O
O
O
O
λO
j hφ̂k − φ̂k,−i , φj i2 = (λ̂k − λ̂k,−i )hφ̂k − φ̂k,−i , φj i2
Z
O
O
+
(γ̂(u, v) − γ(u, v))(φ̂O
k (u) − φ̂k,−i (u))φj (v)dudv
2
O
Z
O
+
(γ̂(u, v) − γ̂−i (u, v))φ̂O
k,−i (u)φj (v)dudv
O2

Using result b), a straightforward generalization of the arguments given by



XVII

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

(62) and (63) then leads to
O
hφ̂O
k − φ̂k,−i , γu i2

λ̂O
k

=

K
O
X
(λ̂O
k − λ̂k,−i )

λ̂O
k
+

1
λO
k

where

O
O
φ̃O
j (u)hφ̂k − φ̂k,−i , φj i2

j=1

∞
X

(−i)

O
O
O
O
λO
j φ̃j (u)hφ̂k − φ̂k,−i , φj i2 + R1,k (u),

j=K+1

sup

sup

u∈O∪M 1≤k≤K

(−i)
R1,k (u)


= Op

K aO rmn
n1/2


,

and proceeding similar to (65) - (67) we can conclude that
(71)

sup

sup

u∈O∪M 1≤k≤K

O
hφ̂O
k − φ̂k,−i , γu i2

λ̂O
k

= Op

K aO +3/2 rmn
n1/2

!

Assertion c) of the lemma is now an immediate consequence of (68) - (71).
Proof of Theorem 4.2.
We have to consider the asymptotic behavior of
[LbK (XO
i )](u) = µ̂(u; hµ ) +

K
X

O ˆO
ξˆik
φ̃k (u),

u ∈ O ∪ M.

k=1

Rates of convergence of µ̂(u; hµ ) are given by Theorem 4.1 (a), while Lemma
ˆ
A.3 provides rates of convergence for φ̃O
k (u). We therefore additionally have
to consider convergence of the PC scores
O
ξˆik
=

m
X

φ̂O
k (Ui(j) )(Yi(j) − µ̂(Ui(j) ; hµ ))(Ui(j) − Ui,(j−1) ),

j=2

where Ui(j) , j = 1, . . . , m is the order sample of observation points. By our
assumption on hµ result (a) of Theorem 4.1 directly implies that
O
ξˆik
=

m
X

φ̂O
k (Ui(j) )(Yi(j) − µ(Ui(j) ))(Ui(j) − Ui,(j−1) ) + R1,k,i ,

j=2

(72)

where



sup |R1,k,i | = Op (mn)−2/5
1≤k≤K

A technical difficulty of subsequent analysis consists in the fact that φ̂O
k and
{(Yij , Uij )}j=1,...,m are correlated. As defined above, we thus eliminate the m

XVIII

A. KNEIP & D. LIEBL

observations representing curve Xi and consider the eigenfunction φ̂O
k,−i (u)
of the reduced sample. We can then infer from result c) of Lemma A.4 that
R2,k,i := |

m
X

O
(φ̂O
k,−i (Ui(j) ) − φ̂k (Ui(j) ))(Yi(j) − µ(Ui(j) ))(Ui(j) − Ui,(j−1) )|

j=2

(73)

satisfies

sup |R2,k,i | = Op (n−1/2 )
1≤k≤K

Recall that Yij = Xi (Uij )+ij and that φ̂O
k,−i is independent of ij , Uij . Since
k ≤ K ≤ K̄mn , result c) of Lemma A.4 also implies that φ̂O
k,−i , k ≤ K, are
asymptotically uniformly bounded over all u ∈ O ∪ M and k ≤ K. By our
assumptions on the error term we can now immediately infer that
m
X

φ̂O
k,−i (Ui(j) )(Yi(j) − µ(Ui(j) )(Ui(j) − Ui,(j−1) )

j=2

=

m
X

φ̂O
k,−i (Ui(j) )(Xi (Ui(j) ) − µ(Ui(j) ))(Ui(j) − Ui,(j−1) ) + R3,k,i ,

j=2


1
with E R23,k,i | γ̂−i ≤ Dk,i and
m

(74)

sup |Dk,i | = Op (1)
1≤k≤K

Let Xi∗ := Xi − µ, and let FU |O denote the distribution function of Uij . It
is then well-known that the random variables Vij := FU |O (Uij ) are U (0, 1)1
distributed, and E(Vi(j) ) − Vi(j−1) )) = m+1
while V(Vi(j) ) − Vi(j−1) )) =
m
. By our assumptions the density fU |O (O ≡ Oi ) of Uij a Taylor
(m+1)2 (m+2)
expansion now yields
−1
1
FU−1
+ R∗4,k,i with E(R∗4,k,i ) =
|O (Vi(j) ) − FU |O (Vi(j−1) ) = mf
(F −1 (V
))
U |O

Op

(1/m2 ).
m
X

U |O

i(j)

This implies

∗
φ̂O
k,−i (Ui(j) )Xi (Ui(j) )(Ui(j) − Ui,(j−1) )

j=2

=

m
X

−1
φ̂O
(Vi(j) ))Xi∗ (F −1 (Vi(j) ))(F −1 (Vi(j) ) − F −1 (Vi(j−1) ))
k,−i (F

j=2

=

m
X
j=2

∗
φ̂O
k,−i (Ui(j) ))Xi (Ui(j) )

1
+ R∗∗
4,k,i
mf (Ui(j) )



∗ 1 for some D ∗ < ∞ satisfying sup
∗
with E R∗∗
|
γ̂
≤ Dik
−i
1≤k≤K |Dk,i | =
4,k,i
ik
m

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

Op (1). Obviously,

m
X
∗
E
φ̂O
k,−i (Ui(j) ))Xi (Ui(j) )
j=2

XIX


Z
1
m
−
1
O
∗
| φ̂
, Xi  =
φ̂O
k,−i (u)Xi (u)du,
mf (Ui(j) ) k,−i
m
O

and independent of k the conditional variance of this random variable can
be bounded by 1/m. We therefore arrive at
m
X

∗
φ̂O
k,−i (Ui(j) ))Xi (Ui(j) )(Uij

Z
− Ui,j−1 ) =

∗
φ̂O
k,−i (u)Xi (u)du + R4,k,i ,

O

j=2

(75)

∗∗ 1
, and
with E R24,k,i | γ̂−i ≤ Dik
m

∗∗
sup |Dk,i
| = Op (1).

1≤k≤K

Additionally note that Lemma A.3 together with K aO +3/2 rmn = O(1)
ˆ
implies that supu∈O∪M sup1≤k≤K φ̃O
k (u) = O(1), while (a) of Theorem 4.1
yields supu∈O∪M |µ(u) − µ̂(u)| = Op ((mn)−2/5 ) We can therefore infer from
(72) - (75) that for u ∈ O ∪ M
(76)
[LbK (XO
i )](u) = µ(u) +



K Z
X
K
K
ˆO
∗
O
( φ̂k,−i (u)Xi (u)du)φ̃k (u) + Op
+
.
m1/2 n1/2
k=1 O

P
O O
Since [LK (XiO )](u) = µ(u) + K
k=1 ξk φ̃j (u) the next step is to consider
R
∗
O
O −
the errors ξik
O φ̂k,−i (u)Xi (u)du for k = 1, . . . , K.
O
The differences between the eigenfunctions φ̂O
k,−i (u) and φ̂k (u) reflect the
elimination of one single curve, and it is immediately clear that the convergence results of Theorem 4.1 and all arguments of Lemma A.3 remain
valid when considering estimated covariances γ̂−i (u, v), eigenvalues λ̂O
k,−i ,
O
and eigenfunctions φ̂k,−i (u) of the reduced sample. Furthermore, note that
O
O
O
O 2
− 12 kφ̂O
since kφ̂O
k,−i k2 = kφj k2 = 1 implies hφ̂k,−i , φk i2 = 1P
k,−i − φk k2 , and
∞
O φO (u).
recall the Karhunen-Loève decomposition Xi∗ (u) = j=1 ξik
k

XX

A. KNEIP & D. LIEBL

We can thus infer from (61) in Lemma A.3 that
Z

∗
φ̂O
k,−i Xi (u)(u) =

O

∞
X

O O
O O
O
ξij
hφ̂k,−i , φO
j i2 = ξik hφ̂k,−i , φk i2 +

j=1

O
= ξik
+

k−1
X

O
ξij

j=1

λ̂O
k,−i
λO
j

X

O O
ξij
hφ̂k,−i , φO
j i2

j6=k

O
O
hφ̂O
k,−i − φk , φj i2 +

∞
X

O O
O
ξik
hφ̂k,−i − φO
k , φj i2

j=k+1

(77)
Z
k−1
O
X
ξik
O 1
O
O 2
+
ξik O
(γ̂−i (u, v) − γ(u, v))φ̂O
kφ̂O
k (u)φj (v)dudv −
k,−i − φk k2 .
2
2
λ
O
j
j=1
By our assumptions on the sequence of eigenvalues we have
(78)
Qi,k :=

k−1
X
j=1







k−1
k−1
k−1
O 2
X
X E((ξ O )2 )
X 1
(ξik
)
ik
 = Op 
 = Op 
j aO  = Op (k aO +1 )
= Op 
O )2
O
2
(λO
(λ
λ
)
j
j
j=1
j=1
j=1 j

Using the Cauchy-Schwary inequality, (78), inequality (65) in Lemma A.3,
and (d) of Theorem 4.1 lead to
|

k−1
X
j=1

(79)

O
ξij

λ̂O
k,−i

hφ̂O
k,−i
λO
j

1/2

−

O
φO
k , φj i2 |

where R5,k,i ≥ 0 with

≤

λO
k Qi,k
δkO

R5,k,i ,

sup R5,k,i = Op (rmn )
1≤k≤K

for all 1 ≤ k ≤ K. Hence,
1/2
!1/2  K
K
k−1
K
O
O 2
X
X
X
X
λ̂
(λk ) Qi,k
ˆ
ˆ
O k,−i
2
2
O
O

|φ̃O
ξij
hφ̂O
R5,k,i
φ̃O
k (u)||
k,−i − φk , φj i2 | ≤
j (u)
O
O )2
λ
(δ
j
k
j=1
k=1
k=1
j=k


(80)
= Op K aO /2+5/2 rmn

P
P∞
O 2
O
−aO +1 ), similar
Since E(Q∗i,k ) := E( ∞
j=k+1 (ξij ) ) =
j=k+1 λj = O(k
arguments based on inequalities (65) and (62) in Lemma A.3 yield

PK ˆO
P∞
O O
O
O
aO /2+5/2 r
mn as well as
k=1 |φ̃k (u)||
j=k+1 ξij hφ̂k,−i − φk , φj i2 | = Op K

Pk−1 O 1 R
PK ˆO
O
O
aO /2+3/2 r
mn ,
j=1 ξij λO O2 (γ̂−i (u, v)−γ(u, v))φ̂k (u)φj (v)dudv| = Op K
k=1 |φ̃k (u)||
j
O|

P
|ξik
ˆO
O
O 2
3aO /2+3 r 2
while by (d) of Theorem 4.1 K
mn .
k=1 |φ̃k (u)| 2 kφ̂k,−i −φk k2 = Op K

XXI

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

Together with (76) and K aO +3/2 rmn = O(1) we therefore arrive at
[LbK (XO
i )](u) = µ(u) +

K
X


 
1
aO /2+3/2
O ˆO
+
K
r
ξik
φ̃k (u) + Op K
mn
m1/2
k=1

(81)
=

LK (XiO )

+

K
X

O ˆO
ξik
(φ̃k (u)

−

φ̃O
k (u))

k=1


 
1
aO /2+3/2
+K
rmn
+ Op K
m1/2

P
O 2 ˆO
O
2
2 2
By Lemma A.3 we have K
k=1 (δk ) (φ̃k (u) − φ̃k (u)) = O(K rmn ), and the
Cauchy-Schwarz inequality implies

!1/2 
!1/2 K
K
K
O )2 )
X
X
X
E((ξ
ˆ
2
O
O ˆO
ik


(φ̃k (u) − φ̃O
ξik
(δkO )2 (φ̃O
k (u)) = Op
k (u) − φ̃k (u))
O )2
(δ
k
k=1
k=1
k=1


!
1/2
K


X

1/2
2
 = Op K aO /2+5/2 rmn
= Op 
k aO +2
K 2 rmn
k=1

and therefore
(82)

[LbK (XO
i )](u)

=

LK (XiO )](u)

 

1
aO /2+3/2
+ Op K
+K
rmn
m1/2

We finally have to consider the truncation error. Recall that it is assumed
that there is a constant DO < ∞ such that supu∈O∪M supk≥1 |φ̃O
k (u)| ≤ DO
P∞ O O
O
Since [LK (Xi )](u) = µ(u) + k=1 ξik φ̃k (u) we have
E

[LK (XiO )](u)

− µ(u) −

K
X
k=1

(83)

∞
X

!
2
O O
ξik
φ̃k (u)

=

2
λk φ̃O
k (u)

k=K+1
∞
X

=O

≤

2
DO

∞
X

λO
k

k=K+1

!
k −aO

= O K −aO +1

k=K+1

Result (28) now follows from (82) and (83). When additionally noting that
standard arguments imply that the local linear estimator of Xi with bandb O (ϑu ; hX ) − Xi (ϑu )| = O m−2/5 , result (29)
width hX  m−1/5 satisfies |X
i
follows from (28) and definition of Lb∗K (XO
i ).



XXII

A. KNEIP & D. LIEBL

Proof of Proposition 5.1:
For u ∈ O2 the optimal linear reconstruction of XiO2 (u), given XiO1 , is
[L(XiO1 )](u), such that XiO2 (u) = [L(XiO1 )](u) + ZiO2 (u), with X̃iO2 (u) =
[L(XiO1 )](u). From result (a) of Theorem 2.3, we know that ZiO2 (u) and
XiO1 (v) are uncorrelated for all u ∈ O2 and all v ∈ O1 . Consequently, by
linearity of L, also
E([L(XiO1 )](u) [L(ZiO2 )](u)) = 0

for all u ∈ O2 ,

where in [L(ZiO2 )](u) we are using that L is also well defined as a linear
mapping from L2 (O2 ) to O2 ; see remark to (7).
Therefore,

2 
M2
O1
O2
0 ≤ E Xi (u) − [L(Xi )](u) − [L(Zi )](u)
=


2 


2 
M2
O1
M2
O2
O2
=E
Xi (u) − [L(Xi )](u)
− 2 E Xi (u)[L(Zi )](u) + E [L(Zi )](u)

2 
M2
O1
⇒ − E Xi (u) − [L(Xi )](u)
≤



2 
M2
O2
O2
≤ −2 E Xi (u)[L(Zi )](u) + E
[L(Zi )](u)
.
But then,

2 
M2
O2
E Xi (u) − [L(Zi )](u)
=



2


2 
M2
M2
O2
O2
= E Xi (u)
− 2 E Xi (u)[L(Zi )](u) + E
[L(Zi )](u)


2 
2 
M2
M2
O1
≥ E Xi (u)
−E
Xi (u) − [L(Xi )](u)
On the other hand, we have that E([L(X̃iO2 )](u)[L(ZiO2 )](u)) = 0 for all
u ∈ M2 , which follows by the same reasoning as used above, since X̃iO2 (u) =
[L(XiO1 )](u), with u ∈ O2 , is just another linear transformation of XiO1 and
XiO1 (v) is known to be uncorrelated with ZiO2 (u) for all u ∈ O2 and v ∈ O1

RECONSTRUCTING PARTIALLY OBSERVED FUNCTIONS

XXIII

by result (a) of Theorem 2.3. Therefore, using also the latter inequality,


2 
2 
M2
O2
M2
O2
O2
E
= E Xi (u) − [L(X̃i )](u) − [L(Zi )](u)
=
Xi (u) − [L(Xi )](u)




2 
2 
M2
O2
M2
O2
O2
E
− 2 E Xi (u) [L(X̃i )](u) + E [L(X̃i )](u)
Xi (u) − [L(Zi )](u)


2 
2 
≥E
− E XiM2 (u) − [L(XiO1 )](u)
XiM2 (u)



2 
M2
O2
O2
− 2 E Xi (u) [L(X̃i )](u) + E [L(X̃i )](u)
=


2 
2 
=E
XiM2 (u) − [L(X̃iO2 )](u)
− E XiM2 (u) − [L(XiO1 )](u)


XiM2 (u)

2 

[L(X̃iO2 )](u)

⇒E
−
≤


2 
2 
M2
O1
M2
O2
≤E
Xi (u) − [L(Xi )](u)
+ E Xi (u) − [L(Xi )](u)
REFERENCES
Tsybakov, A. (2008). Introduction to Nonparametric Estimation, 1. ed. Springer Series
in Statistics. Springer.
Alois Kneip and Dominik Liebl
Statistische Abteilung
University of Bonn
Adenauerallee 24-26
53113 Bonn, Germany
E-mail: akneip@uni-bonn.de
E-mail: dliebl@uni-bonn.de

