A SPLINE-ASSISTED SEMIPARAMETRIC APPROACH TO
NONPARAMETRIC MEASUREMENT ERROR MODELS

arXiv:1804.00793v1 [math.ST] 3 Apr 2018

By Fei Jiang∗ and Yanyuan Ma†
University of Hong Kong∗ and Penn State University†
Nonparametric estimation of the probability density function of
a random variable measured with error is considered to be a difficult
problem, in the sense that depending on the measurement error property, the estimation rate can be as slow as the logarithm of the sample
size. Likewise, nonparametric estimation of the regression function
with errors in the covariate suffers the same possibly slow rate. The
traditional methods for both problems are based on deconvolution,
where the slow convergence rate is caused by the quick convergence
to zero of the Fourier transform of the measurement error density,
which, unfortunately, appears in the denominators during the construction of these methods. Using a completely different approach
of spline-assisted semiparametric methods, we are able to construct
nonparametric estimators of both density functions and regression
mean functions that achieve the same nonparametric convergence
rate as in the error free case. Other than requiring the error-prone
variable distribution to be compactly supported, our assumptions
are not stronger than in the classical deconvolution literatures. The
performance of these methods are demonstrated through some simulations and a data example.

1. Introduction. Density estimation is a familiar problem in the nonparametric estimation literature. Generally, we observe independent and
identically distributed (iid) variables X1 , . . . , Xn from a distribution with
probability density function (pdf) fX (x) and nonparametric estimators such
as kernel methods are available in the literature to estimate fX (x). Even
when the Xi ’s are not directly observed, nonparametric estimation of fX (x)
can still be carried out based on their surrogates. Specifically, assume that
instead of observing Xi , we observe Wi ≡ Xi + Ui , where Ui is a mean
zero random error independent of Xi and follows a distribution with pdf
fU (u). This problem was studied extensively in the literature and by large
the main stream approach is deconvolution (Carroll and Hall, 1988; Liu and
Taylor, 1989; Stefanski and Carroll, 1990; Zhang, 1990; Fan, 1991). However,
as established in these works, the deconvolution based estimator of fX (x)
may converge very slowly. For example, when the error distribution fU (u)
MSC 2010 subject classifications: Primary 60K35, 60K35; secondary 60K35
Keywords and phrases: errors in covariates, measurement error, semiparametrics, spline

1

2

is normal or other distributions that are “super smooth”, the deconvolution
estimator only converges to fX (x) at the rate of {log(n)}−k where k is a
positive constant. When the error distribution fU (u) is Laplace or other
“ordinary smooth” type, the convergence rate is n−k , where k is a positive
constant smaller than 0.2. Here “super smooth” and “ordinary smooth” are
characteristics that play an important role in determining the convergence
properties of the deconvolution estimators. Because these characteristics are
irrelevant in our approach, we skip the precise description and refer to Fan
(1991) for details.
Given that when Xi ’s are observed, nonparametric density estimation
usually performs better than these results, it prompts us to ask whether we
could further improve the estimation performance for the error-in-variable
case. However, the convergence rates described above are shown to be “optimal” in the literature as long as deconvolution is used. Thus, if better
convergence is to be achieved, the estimation has to go beyond the deconvolution framework. In other words, novel approaches totally different from
the existing literature will be required.
A related problem to nonparametric density estimation is nonparametric
regression. Likewise, when iid observations (Xi , Yi ), i = 1, . . . , n, are available, many nonparametric estimators such as kernel and spline based methods have been proposed to estimate the regression mean of Y conditional
on X. Here the assumption is that Yi = m(Xi ) + i , where for simplicity,
we assume i is independent of Xi and has mean zero with density f ().
When Xi is unavailable and instead only Wi described above is available,
we encounter the problem of nonparametric regression with measurement
error. The traditional way to estimate m(x) is also deconvolution (Fan and
Truong, 1993), resulting in the same possibly slow rate of convergence. Because the same reason that results in the slow convergence rate in the density
estimation is also responsible for the nonparametric regression estimation,
methods that do not rely on deconvolution have to be employed in order to
achieve improvement.
Our breakthrough came when it struck us that if the functions that we
want to estimate, fX (x) and m(x), had been parametric, we would have
much simpler problems. If these problems can be solved without using deconvolution, then we can approximate fX (x) and m(x) with B-spline, and
operationally we would be dealing with parametric models and deconvolution can be avoided too. Thus, we study both problems in this work with the
help of a spline representation. Using spline representation in measurement
error models is not entirely new, although it was mostly used in the Bayesian
framework (Berry, Carroll and Ruppert, 2002; Staudenmayer, Ruppert and

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR

3

Buonaccorsi, 2008). While the density estimator is relatively easy to devise,
regression estimator turns out to be challenging and requires further semiparametric treatment. A second challenge in both problems is in establishing
the convergence rates of the resulting estimators. The common obstacle in
both estimators is the fact that it is a latent function that needs to be approximated with a spline representation, which requires unusual treatment
different from the typical handling of the spline approximation. In addition,
for the regression problem, we encounter further difficulties because not only
the estimator, but also the estimating equation that generates the estimator, do not have closed forms. Much effort is spent in clearing these hurdles, where we resort to extensive use of bilinear operators (Conway, 1990),
which are very different from typical regression spline asymptotic analysis
(Ma et al., 2015). The detailed proofs are respectively in the Appendix at
the end of the paper and in an online supplementary document.
The success of the B-spline-assisted procedures in achieving the better
rates of convergence in these problems is encouraging. Despite of the satisfactory result, it is not immediately clear what is the intuitive reason behind
the success. To rationalize the convergence rate of the B-spline based procedures, we can consider the parametric pdf and regression problems with
measurement error. In these simpler problems, root-n convergence of the
parameter estimation is indeed achievable. Thus, bridging a nonparametric problem to a parametric problem via B-splines, it is not unreasonable
to expect a classical nonparametric convergence rate for measurement error
problems as well. A critical assumption in such B-spline treatment is that the
possible range of the unobserved covariates are bounded, hence we can confine to a compact set in estimating the functions of interest. This restricted
the impact from the tails of the measurement error distribution, which is exactly what brings down the convergence rate of the deconvolution methods.
More explicitly, in the B-spline approximation, the compact set on which
we perform the estimation is built into the procedure at the very beginning
and we benefit from that throughout the procedure. However, in the deconvolution procedures, the Fourier and inverse Fourier transformation steps
do not take advantage of the compact set knowledge, and it automatically
estimates these functions on the whole interval, which is much harder to do.
In the following, we devise the estimation procedures for both the probability density function and the regression mean function in Section 2, and
summarize the theoretical properties of our estimators in Section 3. We
provide simulation studies to demonstrate the rate properties of the new
estimators in Section 4, and illustrate the methods in a data example in
Section 5. The paper is concluded with a discussion in Section 6.

4

2. B-spline-assisted estimation procedures.
2.1. Probability density function estimation. To set the notation, we use
fX (x) to denote a generic pdf function of the random variable X, and use
fX0 (x) to denote the true pdf that generates the data. We approximate
fX0 (x) on its support using B-splines (Masri and Redner, 2005). For simplicity, let the support be contained in [0, 1]. To ensure that the density
function is nonnegative and integrates to 1, we let the approximation be
fX (x, θ) ≡ R 1
0

exp{Br (x)T θ}
exp{Br (x)T θ}dx

,

where Br (x) is a vector of B-spline basis functions, and θ is the B-spline
coefficient vector to be estimated. Then
R1
exp{Br (x)T θ}fU (w − x)dx
fW (w, θ) ≡ 0 R 1
T
0 exp{Br (x) θ}dx

is an approximation to the pdf of W ≡ X + U , a surrogate of X. We then
perform a simple maximum likelihood estimator (MLE) to obtain θ, i.e. we
maximize
n
X

=

i=1
n
X
i=1

logfW (Wi , θ)
log

Z

0

1

T

exp{Br (x) θ}fU (Wi − x)dx − nlog

Z

1

exp{Br (x)T θ}dx

0

b and then reconstruct fX (x, θ)
b and use it as
with respect to θ to obtain θ,
b
b
the estimator for fX0 (x), i.e. fX0 (x) = fX (x, θ).
While the estimation procedure for fX0 (x) is extremely simple, it is not as
straightforward to establish the large sample properties of the estimator. In
Section 3, we will show that fbX0 (x) converges to fX0 (x) at a nonparametric
rate under mild conditions.
2.2. Regression function estimation. Unlike in the density estimation
case, the estimation procedure in the regression model is much more complex as we now describe. The key observation is that as soon as the nonparametric function m(x) is approximated with the B-spline representation, the
regression function itself without measurement error is a purely parametric
model (Wang and Yang, 2009), hence the idea behind Tsiatis and Ma (2004)
can be adapted. Specifically, we treat the B-spline coefficients as parameters of interest, treat the unspecified distribution of X, fX (x), as nuisance

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR

5

parameter, and cast the problem as a semiparametric estimation problem.
We can then construct the efficient score function, which relies on fX0 (x).
A key observation of Tsiatis and Ma (2004) is that by replacing fX0 (x) with
an arbitrary working model, the consistency of the estimation can still be
retained. We now describe the estimation procedure in detail.
∗ (x) be a working pdf of X. Note that f ∗ (x) may not be the
First, let fX
X
∗ (x) is possibly misspecified. Following the practice
same as fX0 (x), that is, fX
of density estimation in Section 2.1, assume that the true density function
fX0 (x) has compact support contained in [0, 1]. Therefore we only need to
consider m(x) on [0, 1]. We approximate m(x) using spline representation
Br (x)T β. Define
Z 1
∂
∗
log
f {Y − Br (x)T β}fU (W − x)fX
(x)dµ(x)
∂β
0
R1 0
∗ (x)B (x)dµ(x)
f {Y − Br (x)T β}fU (W − x)fX
r
.
= − 0 R1
∗
T
0 f {Y − Br (x) β}fU (W − x)fX (x)dµ(x)

S∗β (W, Y, β) =

As the notation suggests, S∗β (W, Y, β) is the score function with respect to
∗ (x)
β calculated from the joint pdf of (W, Y ) under the working model fX
∗ (x),
and the spline approximation. Due to the possible misspecification of fX
∗
the mean of Sβ (W, Y, β) is not necessarily zero even if the mean function
P
is exactly Br (x)T β. Therefore simply solving ni=1 S∗β (Wi , Yi , β) = 0 may
generate an inconsistent estimator. The idea behind our estimator is to find
a function a(x, β) so that
(1)

E{S∗β (W, Y, β) | X}
"R 1
#
T β}f (W − x)f ∗ (x)dµ(x)
a(x,
β)f
{Y
−
B
(x)

r
U
X
= E 0 R1
|X ,
T β}f (W − x)f ∗ (x)dµ(x)
f
{Y
−
B
(x)
r
U
X
0 

and then solve for β using the estimating equation
(2)

n
X
 ∗
Sβ (Wi , Yi , β)
i=1

−

R1
0

#
∗ (x)dµ(x)
a(x, β)f {Yi − Br (x)T β}fU (Wi − x)fX
= 0.
R1
T β}f (W − x)f ∗ (x)dµ(x)
f
{Y
−
B
(x)

i
r
i
U
X
0

This will guarantee a consistent estimator of β if the mean function is indeed
Br (x)T β, because our construction ensures the left hand side of (2) has mean
zero. Note that the right hand side of (1) is the conditional expectation of

6

a(X, β) calculated under the B-spline approximation and the posited model
∗ (x), hence we alternatively write it as E ∗ {a(X, β) | Y , W , β}.
fX
i
i
To solve for
a(x,
β),
we
discretize
the
integral
equation
(1).
In particular,
PL
∗
let fX (x) = j=1 cj I(x = xj ), where xj ’s are the points selected on [0, 1],
P
and cj ≥ 0, L
j=1 cj = 1. Then
S∗β (W, Y, β)

= −

PL

0
T
j=1 Br (xj )f {Y − Br (xj ) β}fU (W − xj )cj
.
PL
T
j=1 f {Y − Br (xj ) β}fU (W − xj )cj

Next, to write out the right hand side of (1) upon discretization, let A(β)
be an L × L matrix with its (i, j) entry
Z
f {y − Br (xj )T β}fU (w − xj )cj
Aij (β) =
PL
T
j=1 f {y − Br (xj ) β}fU (w − xj )cj
×f {y − Br (xi )T β}fU (w − xi )dµ(y)dµ(w).

Let ai = a(xi , β), a = (a1 , . . . , aL ). Further, define Hj (β) = {H1j (β), . . . , HLj (β)},
where
Z
f 0 {y − Br (xj )T β}fU (w − xj )cj
Hij (β) = − PL 
T
j=1 cj f {y − Br (xj ) β}fU (w − xj )
×f {y − Br (xi )T β}fU (w − xi )dµ(y)dµ(w),

and let b(β) be a p × L matrix, with its ith column
bi (β) =

L
X

Hij (β)Br (xj ).

j=1

P
Then b(β) = L
j=1 Br (xj )Hj (β).
With
these
notations,
the integral equation (1) is equivalently written
PL
PL
as
j=1 Aij aj =
j=1 Hij (β)Br (xj ) for i = 1, . . . , L, or more concisely,
PL
T
aA (β) = j=1 Br (xj )Hj (β). Therefore
a(β) =

L
X

Br (xk )Hk (β){A−1 (β)}T ,

k=1

hence
aj (β) =

L
X
k=1

Br (xk )Hk (β){A−1 (β)}T ej ,

7

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR

where ej is a length L vector with the jth component 1 and all others zero.
Thus, we have obtained a(X, β) on the discrete set x1 , . . . xL and can form
E ∗ {a(X, β) | Y, W, β}
PL
T
j=1 aj (β)f {Y − Br (xj ) β}fU (W − xj )cj
=
PL
T
j=1 f {Y − Br (xj ) β}fU (W − xj )cj
P
L
−1
T
T
X
Hk (β) L
j=1 {A (β)} ej f {Y − Br (xj ) β}fU (W − xj )cj
=
Br (xk )
PL
T
j=1 f {Y − Br (xj ) β}fU (W − xj )cj
k=1
=

L
X

Br (xk )Pk (W, Y, β),

k=1

where
Pk (W, Y, β) =

Hk (β)

PL

−1
T
T
j=1 {A (β)} ej f {Y − Br (xj ) β}fU (W
PL
T
j=1 f {Y − Br (xj ) β}fU (W − xj )cj

− xj )cj

.

We then obtain the estimator for β by solving the estimating equation (2)
with the corresponding a(x, β) plugged in. Note that in all the functions
that are explicitly written to depend on β, the dependence is always through
Br (·)T β.
b converges to m(x) at a nonparametric rate and we
We show that Br (x)T β
derive its estimation variance in Section 3 under mild conditions. As pointed
∗ (x),
out by an anonymous referee, instead of adopting a working model fX
we could in fact estimate fX0 (x) using the method developed in Section
∗ (x) to carry out the
2.1 to obtain fbX (x), and then use fbX (x) instead of fX
subsequent operations. We believe that this is the optimal thing to do and
will generate the most efficient estimator. However, rigorously establishing
the theoretical property of such practice is challenging so we leave this option
to future research.
3. Asymptotic results.
3.1. Results of probability density function estimation. We assume the
following regularity conditions.
(C1) The true density function fX0 (x) has compact support contained in
[0, 1], is bounded on its support and satisfies fX0 (x) ∈ C q ([0, 1]),
q ≥ 1. fU (u) is bounded above. The conditional pdf of X given W ,
fX|W (x|w), is bounded.
(C2) The spline order r ≥ q.

8

(C3) We define the knots t−r+1 = · · · = t0 = 0 < t1 < · · · < tN < 1 =
tN +1 = · · · = tN +r , where N is the number of interior knots and [0, 1]
is divided into N + 1 subintervals. Let dθ = N + r. N satisfies N → ∞,
N −1 n(logn)−1 → ∞, and N n−1/(2q) → ∞ when n → ∞.
(C4) Let hp be the distance between the (p + 1)th and pth interior knots
and let hb = max hp , hs = min hp . There exists a constant
r≤p≤N +r

r≤p≤N +r

chb , 0 < chb < ∞, such that
hb /hs < chb .
Therefore, hb = O(N −1 ), hs = O(N −1 ).
(C5) θ 0 is a spline coefficient such that supx∈[0,1] |Br (x)T θ 0 −log{fX0 (x)}| =
Op (hqb ). The existence of such θ 0 has been shown in De Boor (1978).
(C6) The expectation
"R 1
#!
R1
T
T
0 exp{Br (x) θ}fU (Wi − x)Br (x)dx
0 exp{Br (x) θ}Br (x)dx
E
−
R1
R1
T θ}f (W − x)dx
T
exp{B
(x)
r
i
U
0
0 exp{Br (x) θ}dx
is a smooth function of θ and has unique root for θ in the neighborhood
of θ 0 .

Condition (C1) contains some basic boundedness and smoothness conditions on various densities and is quite standard. The only requirement
that appears nonstandard is that fX0 (x) has compact support. This however is practically relevant in many situations as the range of the values
a random variable can be may very well be bounded. Condition (C2) is a
standard requirement to ensure that splines of sufficiently high order are
utilized. Condition (C3) requires suitable amount of spline basis to be used
according to the sample size, and Condition (C4) makes sure that the spline
knots are distributed sufficiently evenly. In summary, Conditions (C2), (C3)
and (C4) are standard requirements and together with Condition (C1), they
ensure Condition (C5). We list Condition (C5) instead of stating it in the
proof for convenience. Finally, Condition (C6) ensures that we are not in
the degenerate case where the expression in (C6) is constantly zero. Under
these conditions, we obtain the following properties of fbX0 (x).
b maxiProposition 1. Assume Conditions (C1) – (C6) to hold. Let θ
mize
n
X
logfW (Wi , θ)
i=1

=

n
X
i=1

log

Z

0

1

T

exp{Br (x) θ}fU (Wi − x)dx − nlog

Z

0

1

exp{Br (x)T θ}dx,

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR

9

b − θ 0 = op (1) element-wise.
then θ

b maxiProposition 2. Assume Conditions (C1) – (C6) to hold. Let θ
mize
n
X
logfW (Wi , θ)
i=1

=

n
X

log

Z

0

i=1

1

T

exp{Br (x) θ}fU (Wi − x)dx − nlog

Z

1

exp{Br (x)T θ}dx.

0

b − θ 0 k2 = Op {(nhb )−1/2 }. Further, define
Then kθ
(
)
n Z 1
X
f
(x)f
(W
−
x)
i
X0
U
R00 = n−1
− fX0 (x) Br (x)dx.
R1
i=1 0
0 fX0 (x)fU (Wi − x)dx
Then

=

b − θ0
θ
"R 1

fX0 (x)fU (Wi − x)Br (x)Br (x)T dx
R1
0 fX0 (x)fU (Wi − x)dx
nR
o⊗2
1
Z 1
f
(x)f
(W
−
x)B
(x)dx
i
r
X0
U
0
−
fX0 (x)Br (x)Br (x)T dx
−
nR
o2
1
0
0 fX0 (x)fU (Wi − x)dx
Z
⊗2 #!−1
0

E

1

+

fX0 (x)Br (x)dx

0

R00 {1 + op (1)}.

Propositions 1 and 2 establishes the consistency and convergence rate
b to θ 0 defined in Condition (C5). The proof of Propositions
properties of θ
1 and 2 are in Supplement S.3 and S.4 respectively. We then utilize these
properties to analyze the bias, variance and convergence rate of fbX0 (x) in
Theorem 1, with its proof in Appendix A.1.

b be defined
Theorem 1. Assume Conditions (C1) – (C6) to hold. Let θ
in Proposition 2 and
b
exp{Br (x)T θ}
fbX0 (x) = R 1
.
Tb
0 exp{Br (x) θ}dx

Then supx∈[0,1] |log{fbX0 (x)} − log{fX0 (x)}| = Op {(nhb )−1/2 }. Further
q−1/2

bias{fbX0 (x)} ≡ E{fbX0 (x) − fX0 (x)} = Op (hb

),

10

and

h
i
nhb fbX0 (x) − fX0 (x) − bias{fbX0 (x)}
"R 1
p
fX0 (x)fU (Wi − x)Br (x)Br (x)T dx
∂
exp{Br (x)T θ 0 }
=
E 0
nhb T R 1
R1
∂θ 0 exp{Br (x)T θ 0 }dx
0 fX0 (x)fU (Wi − x)dx
nR
o⊗2
1
Z 1
0 fX0 (x)fU (Wi − x)Br (x)dx
−
−
fX0 (x)Br (x)Br (x)T dx
nR
o2
1
0
0 fX0 (x)fU (Wi − x)dx
Z
⊗2 #!−1
p

1

+

fX0 (x)Br (x)dx

0

×n−1

n Z
X
i=1

0

1

(

R1
0

fX0 (x)fU (Wi − x)

fX0 (x)fU (Wi − x)dx

)

− fX0 (x) Br (x)dx + op (1).

Theorem 1 shows that the B-spline MLE density estimator is root-nhb
consistent to fX0 (x), which is a standard nonparametric density estimation rate when no measurement error occurs, and is improved substantially
compared to the nonparametric deconvolution method.
3.2. Results of regression mean function estimation. To facilitate the description of the regularity conditions and the asymptotic results, we introduce some notations. Define
P
−1
T
T
H(x, β) L
j=1 {A (β)} ej f {Y − Br (xj ) β}fU (W − xj )cj
P (x, W, Y, β) =
,
R1
∗
T
0 f {Y − Br (x) β}fU (W − x)fX (x)dµ(x)
H(x, β) = {H1 (x, β), . . . , HL (x, β)},
Z
∗ (x)
f0 {y − Br (x)T β}fU (w − x)fX
Hi (x, β) = − R 1
∗
T
0 f {y − Br (x) β}fU (w − xj )fX (x)dµ(x)
×f {y − Br (xi )T β}fU (w − xi )dµ(y)dµ(w),

and write
∗

E {a(X, β) | Y, W, β} =
S∗β (W, Y, β)

=

Z

1

Br (x)P (x, W, Y, β)dµ(x),

0

Z

0

1

∗ (x)
Br (x)f0 {Y − Br (x)T β}fU (W − x)fX
dµ(x).
R1
∗ (x)
− 0 f {Y − Br (x)T β}fU (W − x)fX

We further define S∗β (Wi , Yi , m), E ∗ {a(X, m) | Yi , Wi , m}, P (x, W, Y, m),
Pk (W, Y, m) to be the resulting quantities when we replace all the appearance of Br (·)T β in S∗β (Wi , Yi , β), E ∗ {a(X, β) | Yi , Wi , β}, P (x, W, Y, β), and

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR 11

Pk (W, Y, m) by m(·) respectively. Here a(X, m) is a function that satisfies
E[S∗β (Wi , Yi , m) − E ∗ {a(X, m)|Yi , Wi , m}|X, m] = 0,
where the last m is used to emphasize that the calculation of the outside
expectation depends on m. Note that these definitions do not conflict with
the notations used before. In fact, some generalize the previous notations.
We further define Sm (Y, W, m) to be a linear operator on Lp whose value at
s ∈ Lp is
Sm (Y, W, m)(s)
#
Z 1"
∗ (x)dµ(x)
f0 {Y − m(x)}fU (W − x)fX
=
−R 1
− P (x, Y, W, m) s(x)dx.
∗ (x)dµ(x)
f
{Y
−
m(x)}f
(W
−
x)f
0

U
X
0

The derivative of Sm (Y, W, m) is a bilinear operator on Lp × Lq with 1/p +
1/q = 1, 1 ≤ p, q ≤ ∞, whose value at s ∈ Lp , v ∈ Lq is
∂Sm (Y, W, m)
∂Sm (Y, W, m + tv)(s)
(s, v) =
∂m
∂t

.
t=0

Further, we define a bilinear operator
2
Sm
(Y, W, m)(s, v) = Sm (Y, W, m)(s)Sm (Y, W, m)(v).

From the above definition, we can see that

n
Also,

Pn
−1

E{Sm (Y, W, m)}(s) = 0,

i=1 Sm (Yi , Wi , m)(s)

− E{Sm (Yi , Wi , m)}(s) = op (1)kskp .

n
o
P
i ,Wi ,m)
i ,Wi ,m)
n−1 ni=1 ∂Sm (Y∂m
(s, v) − E ∂Sm (Y∂m
(s, v) = op (1)kskp kvkq ,
P
2 (Y , W , m)(s, v) − E{S 2 (Y , W , m)}(s, v) = o (1)ksk kvk ,
n−1 ni=1 Sm
i
i
i
p
p
q
m i

2 (Y , W , m)}(s, v) = E{S (Y, W, m)(s)S (Y, W, m)(v)} and
where E{Sm
i
i
m
m
E{∂Sm (Yi , Wi , m)/∂m}(s, v) = E[{∂Sm (Y, W, m)/∂m}(s, v)].
We now list the regularity conditions.

(D1) The true density functions fX0 (x), f () are bounded on their supports.
In addition, the support of fX0 (x) is compact.
(D2) E {∂Sm (Y, W, m)/∂m} is a bounded bilinear operator on L2 ×L2 , L1 ×
2 (Y , W , m)} is a bounded bilinear operator
L∞ , and L∞ × L1 . E{Sm
i
i
on L2 × L2 .

12

(D3) m(x) is bounded on [0, 1] and it satisfies m(x) ∈ C q ([0, 1]), q ≥ 1.
(D4) β 0 is a dβ dimensional spline coefficient such that supx∈[0,1] |Br (x)T β 0 −
m(x)| = Op (hqb ). The existence of such β 0 has been shown in De Boor
(1978).
(D5) The expectation
E[S∗β (Wi , Yi , β) − E ∗ {a(X, β)|Yi , Wi , β}]
has unique root for β in the neighborhood of β 0 . Its derivative with
respect to β is a smooth function of β in the neighborhood of β 0 , with
its singular values bounded and bounded away from zero. Denote the
unique zero β ∗ .
Conditions (D1) and (D2) contain bounded requirement on pdfs and operators and are not stringent. Condition (D1) further requires the compact
support of the distribution of X. This is similar to the density estimation
case and is crucial. Conditions (D3) and (D4) are regarding the mean function m(x) and its spline approximation which are quite standard. Condition
(D5) is a unique root requirement similar to (C6) and is used to exclude the
pathologic case where the estimating equation is constantly zero.
In the following, we establish the consistency of the parameter estimation
in Proposition 3 and then further analyze its convergence rate in Proposition
4. The results in these propositions are subsequently used to further establish
the asymptotic properties of the estimator of the regression mean function
m(x) in Theorem 2. The proofs of both the propositions and the theorem
are in Supplement S.5, S.6 and Appendix A.2 respectively.
Proposition 3.
b satisfy
Let β
n
X
i=1

Assume Conditions (C2) – (C4), (D1) – (D5) to hold.

b − E ∗ {a(X, β)
b | Yi , Wi , β}
b = 0.
S∗β (Wi , Yi , β)

b − β = op (1) element-wise.
Then β
0
Proposition 4.
b satisfy
Let β
n
X
i=1

Assume Conditions (C2) – (C4), (D3) – (D5) to hold.

b − E ∗ {a(X, β)
b | Yi , Wi , β}
b = 0.
S∗β (Wi , Yi , β)

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR 13

b − β k2 = Op {(nh)−1/2 }. Further,
Then kβ
0
b −β =
β
0

(

E

∂[S∗β (Wi , Yi , β) − E ∗ {a(X, β) | Yi , Wi , β}]
∂β T

Br (·)T β=m(·)

!)−1

×T00 {1 + op (1)},

where
T00 = n

−1

n
X
i=1

S∗β (Wi , Yi , m) − E ∗ {a(X, m) | Yi , Wi , m}.

Theorem 2. Assume Conditions (C2) – (C4), (D1) – (D5) to hold. Let
b Then sup
b
− m(x)| = Op {(nhb )−1/2 }. Specifim(x)
b
= Br (x)T β.
x∈[0,1] |m(x)
q−1/2

cally, bias{m(x)}
b
= E{m(x)
b
− m(x)} = Op (hb
) and
p
nhb [m(x)
b
− m(x) − bias{m(x)}]
b
p
T
=
nh Br (x)
 b(
!)−1
∂[S∗β (Wi , Yi , β) − E ∗ {a(X, β) | Yi , Wi , β}]
× − E
∂β T
Br (·)T β=m(·)
#
n
X
−1
×n
S∗β (Wi , Yi , m) − E ∗ {a(X, m) | Yi , Wi , m} + op (1).
i=1

∗ (x), m(x)
Theorem 2 shows that even with misspecified fX
b
converges to
m(x) with root-nhb rate, which is the standard nonparametric rate of the
regression mean function without measurement errors, and is considerably
faster than deconvolution estimators.

4. Simulation studies.
4.1. Performance of B-spline-assisted MLE. We conduct two simulation
studies to illustrate the finite sample performance of the proposed pdf and
regression mean estimators. To evaluate the B-spline MLE method for estimating the density functions, we generated data sets of sample sizes from
n = 500 to n = 2000, where X is generated from a beta distribution with
both shape parameters equal to 4. We then generated the measurement error
U from three models:
I(a): a normal distribution with mean 0, variance 0.25, denoted by
N (0, 0.25),

14

√
2, denoted by
I(b): a Laplace
distribution
with
mean
0
and
scale
0.5/
√
Lap(0, 0.5/ 2),
I(c): a uniform distribution on [−0.25, 0.25], denoted by Unif(−0.25, 0.25).
We used cubic splines to approximate the density functions, with number
of knots equal to the smallest integer larger than 1.7n1/5 . In the left panel
of Figure 1, we plotted
√ the averaged root-(nhb ) maximum absolute error
(MAE), calculated as nhb supx |fbX0 (x) − fX0 (x)|, versus the sample sizes
n. Following Theorem 1, the root-(nhb ) MAE has a constant order. This
translates to the curves in the plots that stabilize in a small range, which is
what we observe, especially when sample size grows to larger than 700.
We also compared the B-spline MLE method with the widely used deconvolution method for density estimation. In the upper row of Figure 2, we
plotted the average values of supx |fbX0 (x) − fX0 (x)| based on 200 simulations for both methods at different sample sizes. We adopted the two-stage
plug-in bandwidth selection method proposed in Delaigle and Gijbels (2002)
in implementing the deconvolution method. Much to our surprise, results in
the upper row of Figure 2 indicate that the B-spline MLE method outperforms the deconvolution method with rather significant gain in this case.
We suspect that this is because the measurement errors are quite large here
which caused difficulties for both methods but especially for the deconvolution method.
To further examine the performance of individual estimated pdf curves
from both methods, we also plotted the estimated density curves for sample
sizes 500, 1000 and 2000. Because the deconvolution method performs poorly
when the measurement errors are large (see upper row of Figure 2), we
reduced the error variances, and generated the three error distributions from
models
√
II(a): N (0, 0.0025), II(b): Lap(0, 0.05/ 2), II(c): Unif(−0.125, 0.125).
Under the reduced error variability, we performed the estimation and plotted the resulting density estimates and their 90% confidence bands. Figure
3 contains the results of the B-spline MLE and the deconvolution estimator
using the two-stage plug-in bandwidth (Delaigle and Gijbels, 2002), at sample size 500. The results show that although not as dramatic as the large
error case, the B-spline MLE is still closer to the true pdf with narrower
confidence band, hence is more precise than the deconvolution method. We
also provide the similar comparison results at sample sizes 1000 and 2000
in Figures S.1 and S.2 in the supplement. For a quantitative comparison,
we also computed the MAE between the true pdf curve and the estimated
pdf curve in the upper part of Table 1. These results show that the B-spline

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR 15

MLE performs consistently better than deconvolution.
4.2. Performance of B-spline-assisted semiparametric estimator. We next
evaluated the finite sample performance of the B-spline semiparametric
mean regression method described in Section 2.2. We used sample sizes n
from 500 to 2000 and generated X from a beta distribution with both shape
parameters equal to 2. The true regression mean function is m(x) = sin(2πx)
and we generated the regression model errors  from a normal distribution
with mean zero variance 0.25. We generated the measurement errors U from
the three different distributions described in model I (a)–(c). Cubic splines
were used to approximate the mean function, and the number of interior
knots were proportional to n1/5 . In the right √
panel of Figure 1, we plotted
the averaged root-(nhb ) MAE calculated via nhb supx |m(x)
b
− m(x)| as a
function of the sample size n. Similar to the density estimation experiment,
the curves stabilize as sample size increases, and is largely flattened after
n = 1000, indicating that supx |m(x)
b
− m(x)| has order (nhb )−1/2 .
We further compared the B-spline semiparametric method with the deconvolution method in the nonparametric mean regression model. In the lower
b
− m(x)| over 200 simurow of Figure 2, we plotted the average supx |m(x)
lations for both methods. Again, in this case, with moderate to significant
amount of noise, the B-spline semiparametric method greatly outperforms
the deconvolution method with smaller average error.
Similar to the pdf investigation, we reduce the measurement error variability and generated the errors from model II (a)–(c) to investigate the
mean function curve fitting. We plotted the mean function estimates and
the 90% confidence bands for the B-spline semiparametric and deconvolution methods in Figure 4 and Figures S.3 and S.4 in the supplement for
sample sizes 500, 1000 and 2000 respectively. These results indicate that the
B-spline semiparametric estimator indeed outperforms the deconvolution
method. Their performance difference in terms of averaged MAE between
the estimated and true mean curves are further provided in the lower half
of Table 1.
5. Data Example. Heavy fine particulate matter (PM2.5) air pollution has become a serious problem in China in recent years and its possible
effect on respiratory diseases has been a concern in public health. Starting from 2012, Beijing Environmental Protection Bureau (BEPB) has been
recording the daily PM2.5 levels in Beijing. Based on these data, Xu et al.
(2016) studied the effect of PM2.5 on asthma in year 2013. Specifically, they
explored the PM2.5 effect on the number of daily asthma emergency room
visits (ERV) in ten hospitals in Beijing, but found no significant effect. In

16

fact, the mean number of daily asthma ERVs even shows a decreasing trend
along the increase of measured PM2.5. This contradicts with the general
conclusion that PM2.5 has short term adverse effect on asthma (Fan et al.,
2016).
A potential reason of this inconsistency is the errors in the PM2.5 measurements which were not taken into account in the above analysis. In fact,
there are many debates on the accuracy of the PM2.5 reports, especially
in the early years such as 2013. For example, we compared the daily average PM2.5 reports in 2013 from 17 ambient air quality monitoring stations
and those reports from the “Mission China Beijing” website (Mission-China,
2016) maintained by the U.S. department of state, and show the two estimated pdfs of PM2.5 in Figure 5. It is clear that the estimated pdfs of PM2.5
from the two sources are very different, where the PM2.5 concentrations obtained from the BEPB yields a pdf estimate with the mode to the left of
that obtained from the “Mission China Beijing”, indicating a generally less
severe air pollution problem. This motivates us to consider the measurement
error issue in studying the effect of PM2.5 on the daily asthma ERV.
We restrict our analysis of PM2.5 in the range from 0 to 400. The largest
recorded PM2.5 value is 328. The data set we analyzed contains 337 observations. In the data available to us, the ith observation contains the number
of daily asthma ERVs, which we treat as response Yi , the average PM2.5
level over 17 ambient air quality monitoring stations from BEPB, which we
denote as Wi , and the PM2.5 level from “Mission China Beijing”, which
we write as W0i . To carry out the analysis, we let the true PM2.5 value
be Xi , and let Wki , Uki be the observed PM2.5 value and its corresponding
measurement error at the kth monitoring station, k = 1, . . . , 17.
We assume
P17
Uki ’s are independent
of each other and of Xi . Then Wi = k=1 Wki /17.
P
Writing U i = 17
U
k=1 ki /17, we have Wi = Xi + U i . Because our preliminary analysis result in Figure 5 suggests a possible discrepancy between
the measurements in Wi and in W0i , we allow a potential bias term b and
model W0i = b + Xi + U0i , where U0i is the measurement error of W0i .
We assume all the Uki , k = 0, . . . , 17 to have
with
P the same distribution
P
mean zero, and we estimate b by bb = n−1 ( ni=1 W0i − ni=1 Wi ), which
yields the value bb = 132. We further estimated the variance of U i based on
var(U ) = {var(W0i − Wi )}/18. This yields var(U
c ) = 0.008. Further, because
U i is the average of 17 iid Uki ’s, it is sensible to assume that U i has a normal
distribution.
Based on the above preliminary analysis, we proceed to estimate the pdf of
PM2.5, i.e. Xi , and the mean regression function of asthma ERVs conditional
on PM2.5, i.e. E(Yi | Xi ), using the B-spline-assisted MLE/semiparametric

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR 17

methods in Sections 2.1 and 2.2. We further implemented 100 bootstraps to
estimate the asymptotic variances of the resulting estimators. We also compared the B-spline MLE and the B-spline semiparametric regression estimator with the deconvolution density and regression estimators. In implementing the B-spline approximation, we used two and three equally spaced knots
respectively, and in implementing the deconvolution methods, we used bandwidth 0.05. The number of knots is chosen based on the simulation studies in
Section 4. In terms of the bandwidth selection, because the crossvalidation
(Stefanski and Carroll, 1990) and the plug-in (Delaigle and Gijbels, 2002)
bandwidth selection algorithms tend to select small bandwidths that induce
large numerical errors, we used the smallest bandwidth that produced stable results in our analysis. The upper panel in Figure 6 shows the estimated
pdfs based on the B-spline MLE and the deconvolution method. Compared
with the kernel estimator in the same plots which ignores the measurement
errors, the B-spline MLE shows more difference than the deconvolution estimator. This is sensible given that the noise-to-signal ratio is quite large at
var(Ūi )/var(Xi ) = 0.2, hence measurement error is likely not ignorable. The
lower panel in Figure 6 provides the estimated mean of Yi as a function of
Xi . The B-spline semiparametric estimator shows a fluctuating pattern in
the range from 0 to 200, although the pattern is not significant. In the range
of PM2.5 concentration larger than 200, it shows clearly an increasing trend,
which agrees with the conclusion in Fan et al. (2016) that the exposure to
high PM2.5 has adverse effect on asthma onset. In contrast, the relation
from the deconvolution estimator is similar to that of the local linear regression estimator ignoring the measurement errors, and it is unable to detect
the increasing trend of the asthma ERVs as the PM2.5 level increases.
6. Discussion. We have developed a B-spline-assisted MLE for nonparametric pdf estimation and a B-spline-assisted semiparametric estimator
for nonparametric regression mean function, under the situation that the
covariates are measured with errors. The performance of both procedures
are superior to the widely used deconvolution methods, in terms of both
their theoretical convergence rate and their numerical performance.
A key difference between our procedures and the deconvolution procedures
is that we restrict our interest in estimating the functions on a compact set.
Given that practically most time, the possible range of a random variable
is indeed finite, hence the relevant information is indeed only functions in a
finite range, we are very curious if deconvolution methods can achieve the
same convergence property in such case, with possibly some modifications
on the existing procedures. To this end, Hall and Qiu (2005) provides some

18

relevant results, while we leave this general question as an open problem for
researchers who specialize in deconvolution methods.
Another curious issue is regarding kernel treatment to nonparametric
functions. In many nonparametric estimation problems, kernel methods and
B-spline methods generate comparable results. However, our procedures critically rely on the B-spline approximation to convert the unknown functions to “parametric” ones, hence we cannot readily replace the B-spline
method with kernel method. Whether or not kernel methods can be applied
in nonparametric density estimation and nonparametric regression estimation problems with measurement errors to achieve the same convergence rate
as B-spline based methods remains a challenging and interesting question.
The density and regression function estimation problems studied in this
work is the most basic ones. In practice, various complications may occur.
For example, the error distribution fU (u) may not be known and need to be
estimated parametrically or nonparametrically based on repeated measurements or other instruments. If one estimates fU (·) parametrically first then
insert it into our procedure, it will not have first order effect. However, if
fU (·) is estimated nonparametrically, it will in general affect both the bias
and variance of the resulting estimation of both the density and regression
functions. See Delaigle and Hall (2016) for deconvolution based estimation
incorporating unknown error distribution. The story may be even more interesting when U and X are correlated hence heteroscedastic measurement
errors occur. These are all very interesting problems and are worth careful
investigation.
Appendix.
A.1. Proof of Theorem 1. Note that
sup |log{fbX (x)} − log{fX0 (x)}|

x∈[0,1]

=

Tb

sup |Br (x) θ − log

x∈[0,1]

=

Z

0

1

b
exp{Br (x)T θ}dx
− log{fX0 (x)}|

b − Br (x)T θ 0 + Br (x)T θ 0 − log{fX0 (x)}
sup |Br (x)T θ

x∈[0,1]

−log
−log

Z

1

0

Z

0

1

b
exp{Br (x)T θ}dx
+ log

exp{Br (x)T θ 0 }dx|

Z

0

1

exp{Br (x)T θ 0 }dx

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR 19

≤

b − θ 0 k2 } + sup |Br (x)T θ 0 − log{fX0 (x)}|
sup {kBr (x)k2 kθ

x∈[0,1]

+|log

x∈[0,1]

Z

1

exp{Br (x) θ}dx − log

0

+|log

Z

1

0

=

Z

1

0

exp{Br (x)T θ 0 }dx|

exp{Br (x)T θ 0 }dx|

b − θ 0 k2 } + sup |Br (x)T θ 0 − log{fX0 (x)}|
sup {kBr (x)k2 kθ

x∈[0,1]

x∈[0,1]

Z

Z 1
1
∂
T ?
b
log
exp{B
(x)
θ
}dx(
θ
−
θ
)|
+
|log
exp{Br (x)T θ 0 }dx|
r
0
T
∂θ
0
0
T
b
sup {kBr (x)k2 kθ − θ 0 k2 } + sup |Br (x) θ 0 − log{fX0 (x)}|

+|
≤

Tb

x∈[0,1]

Z

x∈[0,1]

1

∂
b − θ 0 )k2 + |log
+k T log
exp{Br (x)T θ ? }dxk2 k(θ
∂θ
0
= Op {(nhb )−1/2 + hqb },

Z

0

1

exp{Br (x)T θ 0 }dx|

b and θ 0 . The last equality holds
where θ ? is the point on the line connecting θ
R1
by Condition (C5), Proposition 2, and the fact that log 0 exp{Br (x)T θ 0 }dx
is a smooth function of Br (x)T θ 0 , which implies
Z 1
|log
exp{Br (x)T θ 0 }dx|
0
Z 1
Z 1
exp{Br (x)T θ 0 }dx − log
exp[log{fX0 (x)}]dx| = Op (hqb )
= |log
0

0

following Condition (C5). Further for any x,
"
#
T θ}
b
exp{B
(x)
r
bias{fbX (x)} = E R 1
− fX0 (x)
Tb
0 exp{Br (x) θ}dx
#
"
# "
b
exp{Br (x)T θ}
exp{Br (x)T θ 0 }
− R1
= E R1
Tb
exp{Br (x)T θ 0 }dx
0 exp{Br (x) θ}dx
"
# "0
#
exp{Br (x)T θ 0 }
exp[log{fX0 (x)}]
+ R1
− R1
T
0 exp{Br (x) θ 0 }dx
0 exp[log{fX0 (x)}]dx
"
#
!
∂
exp{Br (x)T θ 3 }
b − θ 0 ) + Op (hq ),
(A.1)
= E
(θ
R
b
∂θ T 01 exp{Br (x)T θ 3 }dx

b The third equality holds by
where θ 3 is on the line connecting θ 0 and θ.
Condition (C5). Consider the expectation in (A.1), recall that from (S.8),
b − θ 0 = −R1 (θ ∗ )−1 (R00 + R01 ),
(A.2)
θ

20

where R00 , R01 , R1 (θ) are defined in Proposition 2 and (S.10) and (S.9)
respectively. Further
!
#
"
∂
exp{Br (x)T θ 3 }
∗ −1
R1 (θ ) R00
E
R
∂θ T 01 exp{Br (x)T θ 3 }dx
"
#
∂
exp{Br (x)T θ 3 }
= E
R1 (θ ∗ )−1
T R1
T
∂θ
0 exp{Br (x) θ 3 }dx
)
!
Z 1(
fX0 (t)fU (Wi − t)
×
− fX0 (t) Br (t)dt
R1
0
0 fX0 (x)fU (Wi − x)dx
#
"
Z 1Z
∂
exp{Br (x)T θ 3 }
R1 (θ ∗ )−1
=
R
∂θ T 01 exp{Br (x)T θ 3 }dx
0
)
(
fX0 (t)fU (Wi − t)
× R1
− fX0 (t) fW 0 (Wi )Br (t)dWi dt
0 fX0 (x)fU (Wi − x)dx
#
"
Z 1Z
∂
exp{Br (x)T θ 3 }
=
R1 (θ ∗ )−1
T R1
T
∂θ
0
0 exp{Br (x) θ 3 }dx
!
fX0 (t)fU (Wi − t)
×R 1
fW 0 (Wi )Br (t) dWi dt
f
(x)f
(W
−
x)dx
i
X0
U
0
"
#
Z 1Z
∂
exp{Br (x)T θ 3 }
−
R1 (θ ∗ )−1 fX0 (t)fW 0 (Wi )Br (t)dWi dt
R
∂θ T 01 exp{Br (x)T θ 3 }dx
0
"
#
Z 1Z
exp{Br (x)T θ 3 }
∂
R1 (θ ∗ )−1 fX0 (t)fU (Wi − t)Br (t)dWi dt
=
R
∂θ T 01 exp{Br (x)T θ 3 }dx
0
"
#
Z 1Z
∂
exp{Br (x)T θ 3 }
−
R1 (θ ∗ )−1 fX0 (t)fU (Wi − t)Br (t)dWi dt
R
∂θ T 01 exp{Br (x)T θ 3 }dx
0
= 0.
(A.3)
In addition, we have
E
(A.4) ≤ E

∂
∂θ T
∂
∂θ T

"

R1

"

0

exp{Br (x)T θ 3 }

exp{Br (x)T θ 3 }dx

R1
0

exp{Br (x)T θ 3 }

#

exp{Br (x)T θ 3 }dx

R1 (θ ∗ )−1 R01

#

2

!

kR1 (θ ∗ )−1 k2 kR01 k2

!

.

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR 21

First note that,
#
"
exp{Br (x)T θ 3 }
∂
(A.5)
R1
T
∂θ
0 exp{Br (x) θ 3 }dx
(d
θ
X
exp{Br (x)T θ 3 }Brk (x)
=
R1
T
k=1
0 exp{Br (x) θ 3 }dx

2

!2 1/2
R1
exp{Br (x)T θ 3 } 0 exp{Br (x)T θ 3 }Brk (x)dx 
−
R1

[ 0 exp{Br (x)T θ 3 }dx]2

#2 1/2
"
dθ
T
X
exp{Br (x) θ 3 }Brk (x) 
≤ 
R1
T
k=1
0 exp{Br (x) θ 3 }dx

!2 1/2
Z 1
dθ

X
T
T
exp{Br (x) θ 3 } exp{Br (t) θ 3 }
Brk (t)dt
+
R1


[ exp{Br (t)T θ 3 }dt]2
0
k=1

"

≤  R1
+

exp{Br (x)T θ 3 }

0


Z


0

exp{Br (x)T θ 3 }dx

1

0

"

#2

dθ
X
k=1

(x)T θ

1/2

2
Brk
(x)

(t)T θ

exp{Br
3 } exp{Br
R1
[ 0 exp{Br (t)T θ 3 }dt]2
#2

3}

!2

1/2

dθ
X
k=1

1/2

2
Brk (t)dt


dθ
X
exp{Br (x)T θ 3 }
2
=  R1
(x)
Brk
T θ }dx
exp{B
(x)
r
3
k=1
0
1/2

!
dθ

Z 1 exp{B (x)T θ } exp{B (t)T θ } 2 X
r
3
r
3
∗
2
+
dt
Brk (t )
R1

 0
[ exp{Br (t)T θ 3 }dt]2
k=1

0

= Op (1),

where t∗ is the point in (0, 1). In the above derivation, the first inequality
holds by the trianglar inequality, the second inequality holds by CauchySchwarz inequality, the second equality holds by the mean value theorem,
the last equality holds because
kBr (x)k22

=

dθ
X
k=1

2
Brk
(x)

≤ sup Brk (x)
k

dθ
X
k=1

Brk (x) = sup Brk (x) = Op (1).
k

22

Combining (A.4), (A.5), (S.12), (S.18), we have
!
#
"
exp{Br (x)T θ 3 }
∂
R1 (θ ∗ )−1 R01
E
R
∂θ T 01 exp{Br (x)T θ 3 }dx
(A.6)

1/2

q−1/2

q+1
= Op (h−1
b dθ hb ) = Op (hb

).

Combining (A.6), (A.2) and (A.3), we obtain
#
!
"
∂
exp{Br (x)T θ 3 }
b − θ 0 ) = Op (hq−1/2 ).
E
(θ
R
b
∂θ T 01 exp{Br (x)T θ 3 }dx
q−1/2

Plug this result to (A.1), we have bias{fbX (x)} = Op (hb
).
Hence,
h
i
p
nhb fbX (x) − fX0 (x) − bias{fbX (x)}
#
"
b
p
exp{Br (x)T θ}
− fX0 (x) − bias{fbX (x)}
=
nhb R 1
T θ}dx
b
exp{B
(x)
r
" 0
b
p
exp{Br (x)T θ 0 }
exp{Br (x)T θ}
=
− R1
nhb R 1
Tb
T
0 exp{Br (x) θ}dx
0 exp{Br (x) θ 0 }dx
#
exp{Br (x)T θ 0 }
+R 1
− fX0 (x) − bias{fbX (x)}
T θ }dx
exp{B
(x)
r
0
0
#
"
b
p
exp{Br (x)T θ 0 }
exp{Br (x)T θ}
− R1
+ Op (hqb n1/2 )
=
nhb R 1
T
T
b
0 exp{Br (x) θ}dx
0 exp{Br (x) θ 0 }dx
#
"R 1
"
p
fX0 (x)fU (Wi − x)Br (x)Br (x)T dx
exp{Br (x)T θ 0 }
∂
=
nhb T R 1
E 0
R1
T
∂θ
0 exp{Br (x) θ 0 }dx
0 fX0 (x)fU (Wi − x)dx
nR
o⊗2
1
Z 1
0 fX0 (x)fU (Wi − x)Br (x)dx
−
−
fX0 (x)Br (x)Br (x)T dx
nR
o2
1
0
0 fX0 (x)fU (Wi − x)dx
(
Z 1
⊗2 #!−1
n Z 1
X
fX0 (x)fU (Wi − x)
+
fX0 (x)Br (x)dx
n−1
R1
0
i=1 0
0 fX0 (x)fU (Wi − x)dx
−fX0 (x)} Br (x)dx + op (1).

The last equality hold by Proposition 2 and Condition (C3). This proves the
results.

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR 23

A.2. Proof of Theorem 2.
sup |m(x)
b
− m(x)|

x∈[0,1]

=

b − Br (x)T β + Br (x)T β − m(x)|
sup |Br (x)T β
0
0

x∈[0,1]

≤

b − Br (x)T β | + sup |Br (x)T β − m(x)|
sup |Br (x)T β
0
0

x∈[0,1]

x∈[0,1]

−1/2

= Op {(nhb )

+

hqb },

by Proposition 2 and Condition (C5).
bias{m(x)}
b

(A.7)

= E{m(x)
b
− m(x)}
b − Br (x)T β } + {Br (x)T β − m(x)}
= E{Br (x)T β
0
0
b − Br (x)T β } + Op (hq ).
= E{Br (x)T β
0

b

The last equality holds by Condition (C5). By (S.21) and Proposition 2, we
can see that
b − Br (x)T β }|
|E{Br (x)T β
0

∗

∂[Sβ (Wi , Yi , β) − E ∗ {a(X, β) | Yi , Wi , β}]
= E Br (x)T E

∂β T

× (T00 + T01 ){1 + op (1)}}


∂[S∗β (Wi , Yi , β) − E ∗ {a(X, β) | Yi , Wi , β}]
T
= E Br (x) E

∂β T
×T01 {1 + op (1)}}


∂[S∗β (Wi , Yi , β) − E ∗ {a(X, β) | Yi , Wi , β}]
≤ E
E

∂β T

Br (·)T β=m(·)

!−1

Br (·)T β=m(·)

!−1

Br (·)T β=m(·)

!−1

T

×kBr (x) k2 kT01 k2 {1 + op (1)}
q+1/2

= Op (h−1
b hb

q−1/2

) = Op (hb

).

The second equality holds because E(T00 ) = 0. The fourth line holds by the
Pdβ
Pdβ
fact that k=1
{uk Brk (x)}2 = M k=1
Brk (x) = M for some finite constant

2

24

M and hence kBr (x)T k2 = O(1),
E

∂[S∗β (Wi , Yi , β) − E ∗ {a(X, β) | Yi , Wi , β}]
∂β

T

Br (·)T β=m(·)

1/2

q+1/2

by (S.28), and kT01 k2 ≤ dβ kT01 k∞ = Op (hb
result to (A.7), we have
q−1/2

Hence
p
=

p

=

p

bias{m(x)}
b
= Op (hb

!−1

2

= Op (h−1
b )

) by (S.25). Plug this

).

nhb [m(x)
b
− m(x) − bias{m(x)}]
b

nh Br (x)T
 b(
!)−1
∂[S∗β (Wi , Yi , β) − E ∗ {a(X, β) | Yi , Wi , β}]
× − E
∂β T
Br (·)T β=m(·)
#
n
p
1X ∗
Sβ (Wi , Yi , m) − E ∗ {a(X, m) | Yi , Wi , m} − nhb bias{m(x)}
b
×
n
i=1
p
nh Br (x)T
=
 b(
!)−1
∂[S∗β (Wi , Yi , β) − E ∗ {a(X, β) | Yi , Wi , β}]
× − E
∂β T
Br (·)T β=m(·)
#
n
X
∗
−1
∗
×n
Sβ (Wi , Yi , m) − E {a(X, m) | Yi , Wi , m} + Op (n1/2 hqb )
i=1

nh Br (x)T
 b(
!)−1
∂[S∗β (Wi , Yi , β) − E ∗ {a(X, β) | Yi , Wi , β}]
× − E
∂β T
Br (·)T β=m(·)
#
n
X
−1
×n
S∗β (Wi , Yi , m) − E ∗ {a(X, m) | Yi , Wi , m} + op (1),
i=1

q−1/2

The second equality holds by the fact that bias{m(x)}
b
= Op (hb
last equality holds by Condition (C3). This proves the result.

). The

Acknowledgment. We thank Yang Li for providing the daily asthma
emergency room visits data and the referees for helpful comments.

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR 25

normal error
laplace error
uniform error

7
6
5
4

Root nh maximum absolute error

3

10

2

5

Root nh maximum absolute error

15

8

normal error
laplace error
uniform error

500

1000

1500

2000

500

1000

1500

Sample size

2000

size

Fig 1. Performance of B-spline MLE pdf estimation (upper) and B-spline semiparametric
mean estimation (lower). Results based on 200 simulations.

1.0

0.40
0.35
0.25

0.30

Maximum absolute error

2.5
2.0
1.5

Maximum absolute error

1.0
0.8
0.6

0.20

0.4

Maximum absolute error

0.45

0.50

Fig 2. Comparison of pdf (upper) and mean (lower) estimators based on the B-spline MLE
or B-spline semiparametric estimator (solid) and the deconvolution (dashed) method, when
measurement errors are norm (left), Laplace (middle) and uniform (right) respectively.
Average maximum absolute error supx |fbX (x) − fX (x)| (upper) or supx |m
b X (x) − mX (x)|
(lower) is computed based on 200 simulations at sample sizes from 500 to 2000.

1500

2000

500

1000

1500

2000

500

1500

2000

0.6

Maximum absolute error

0.5
0.4

0.2

0.2

0.3

0.3

Maximum absolute error

2000

0.7

0.7
0.6

1.0
0.8
0.6
0.4

1500
Sample size

0.1

0.2

Maximum absolute error

1000

Sample size

0.8

Sample size

0.5

1000

0.4

500

500

1000

1500
Sample size

2000

500

1000

1500
Sample size

2000

500

1000
Sample size

26

4
3
f

2
1
0

0

1

f

2

3

4

Fig 3. B-spline MLE density estimation (left) and deconvolution estimation (right) from
200 simulations: The solid lines represent the true functions and the dash lines represent
the estimated functions and their 90% confidence bands. The first row to third row are the
results for model II (a)–(c) respectively. Sample size 500.

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.8

1.0

0.6

0.8

1.0

0.6

0.8

1.0

4
3
f

0

1

2

3
f

2
1
0
0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

3
f

1
0

0

1

2

2

3

4

x

4

x

f

0.6
x

4

x

0.0

0.2

0.4

0.6
x

0.8

1.0

0.0

0.2

0.4
x

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR 27

3
m
−2

−3

−2

−1

−1

0

0

m

1

1

2

2

3

Fig 4. B-spline semiparametric regression estimation (left) and deconvolution estimation
(right) from 200 simulations: The solid lines represent the true functions and the dash
lines represent the estimated functions and their 90% confidence bands. The first row to
third row are the results for model II (a)–(c), respectively. Sample size 500.

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

0.6

0.8

1.0

0.6

0.8

1.0

m
−2

−3

−2

−1

−1

0

0

m

1

1

2

2

3

x

3

x

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

m
0

0

−1

−1

−2

−2
−3

m

1

1

2

2

3

x

3

x

0.0

0.2

0.4

0.6
x

0.8

1.0

0.0

0.2

0.4
x

28

0.004
0.000

0.002

Density

0.006

Fig 5. The estimated pdf of PM2.5 without considering measurement error, based on data
from Beijing Environmental Protection Bureau (solid line) and “Mission China” website
(dash-line).

0

100

200

300

400

500

Observed PM2.5

Table 1
Performance comparison between B-spline MLE/semiparametric estimator and
deconvolution. Mean of the maximum absolute differences are reported. Results based on
average over 200 simulations.

model II(a)
model II(b)
model II(c)

model II(a)
model II(b)
model II(c)

pdf estimation: E{supx |fbX0 (x) − fX0 (x)|}
B-spline MLE
Deconvolution
n = 500 n = 1000 n = 2000 n = 500 n = 1000 n = 2000
0.204
0.160
0.109
0.238
0.212
0.160
0.213
0.153
0.110
0.230
0.197
0.158
0.222
0.184
0.124
0.315
0.242
0.230
mean estimation: E{supx |m(x)
b
− m(x)|}
B-spline semiparametric
Deconvolution
n = 500 n = 1000 n = 2000 n = 500 n = 1000 n = 2000
0.370
0.263
0.175
0.908
0.796
0.762
0.425
0.264
0.163
0.857
0.828
0.779
0.414
0.291
0.219
0.880
0.832
0.801

References.
Berry, S. M., Carroll, R. J. and Ruppert, D. (2002). Bayesian smoothing and regression splines for measurement error problems. Journal of the American Statistical
Association 97 160–169.
Carroll, R. J. and Hall, P. (1988). Optimal rates of convergence for deconvolving a
density. Journal of the American Statistical Association 83 1184-1186.
Conway, J. B. (1990). A Course in Functional Analysis. Springer-Verlag, New York.
De Boor, C. (1978). A practical guide to splines 27. Springer-Verlag, New York.
Delaigle, A. and Gijbels, I. (2002). Estimation of integrated squared density derivatives
from a contaminated sample. Journal of the Royal Statistical Society: Series B 64 869–
886.

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR 29

Density
0

50

100

150

200

250

0.000 0.002 0.004 0.006 0.008 0.010 0.012 0.014

Density

0.000 0.002 0.004 0.006 0.008 0.010 0.012 0.014

Fig 6. The B-spline MLE (upper left) and deconvolution (upper right) pdf estimators
and their 95% confidence bands of PM2.5. The B-spline semiparametric (bottom left) and
deconvolution (bottom right) regression estimators and their 95% confidence bands. The
red line is the naive estimator ignoring measurement errors.

300

0

50

100

200

250

300

250

300

8
7
4

5

6

Mean asthma ERV

7
6
5
4

Mean asthma ERV

150

True PM2.5

8

True PM2.5

0

50

100

150

200

True PM 2.5

250

300

0

50

100

150

200

True PM 2.5

30
Delaigle, A. and Hall, P. (2016). Methodology for non-parametric deconvolution when
the error distribution is unknown. Journal of the Royal Statistical Society: Series B 78
231–252.
Fan, J. (1991). On the Optimal Rates of Convergence for Nonparametric Deconvolution
Problems. Annals of Statistics 19 1257-1272.
Fan, J. and Truong, Y. K. (1993). Nonparametric Regression with Errors in Variables.
Annals of Statistics 21 1900-1925.
Fan, J., Li, S., Fan, C., Bai, Z. and Yang, K. (2016). The impact of PM2.5 on asthma
emergency department visits: a systematic review and meta-analysis. Environmental
Science and Pollution Research 23 843–850.
Hall, P. and Qiu, P. (2005). Discrete-transform approach to deconvolution problems.
Biometrika 92 135–148.
Liu, M. C. and Taylor, R. L. (1989). A consistent nonparametric density estimator for
the deconvolution problem. Canadian Journal of Statistics 17 427-438.
Ma, S., Carroll, R. J., Liang, H. and Xu, S. (2015). Estimation and Inference in Generalized Additive Coefficient Models for Nonlinear Interactions with High-dimensional
Covariates. Annals of Statistics 43 2102–2131.
Masri, R. and Redner, R. A. (2005). Convergence rates for uniform B-spline density
estimators on bounded and semi-infinite domains. Nonparametric Statistics 17 555–
582.
Mission-China (2016). Accessed:09-30. http://www.stateair.net/web/historical/1/1.html.
Staudenmayer, J., Ruppert, D. and Buonaccorsi, J. P. (2008). Density Estimation in
the Presence of Heteroscedastic Measurement Error. Journal of the American Statistical
Association 103 726–736.
Stefanski, L. A. and Carroll, R. J. (1990). Deconvoluting kernel density estimators.
Statistics 21 169-184.
Tsiatis, A. A. and Ma, Y. (2004). Locally Efficient Semiparametric Estimators for Functional Measurement Error Models. Biometrika 91 835-848.
Wang, L. and Yang, L. (2009). Spline estimation of single-index models. Statistica Sinica
765–783.
Xu, Q., Li, X., Wang, S., Wang, C., Huang, F., Gao, Q., Wu, L., Tao, L., Guo, J.,
Wang, W. et al. (2016). Fine particulate air pollution and hospital emergency room
visits for respiratory disease in urban areas in Beijing, China, in 2013. PloS one 11
e0153099.
Zhang, C. H. (1990). Fourier methods for estimating mixing densities and distributions.
Annals of Statistics 18 806-830.
Department of Statistics
University of Hongkong
Pok Fu Lam, Hongkong
E-mail: feijiang@hku.hk

Department of Statistics
Penn State University
University Park, PA 16802
E-mail: yzm63@psu.edu

SUPPLEMENTARY DOCUMENT FOR “A SPLINE-ASSISTED
SEMIPARAMETRIC APPROACH TO NONPARAMETRIC
MEASUREMENT ERROR MODELS”
By Fei Jiang∗ and Yanyuan Ma†
University of Hong Kong∗ and Penn State University†

S.1. Definitions. Let H be the separable Hilbert space of square integrable
R1
functions on [0, 1], and let < φ1 , φ2 >≡ 0 φ1 (x)φ2 (x)dx denote the inner product
of the functions φ1 , φ2 in H. Let B ⊂ H be the space spanned by the B-spline bases.
Then the L2 norm k · k2 is the norm induced by the inner product. We denote the
functional sup norm as k · k∞ , and the Lp norm as k · kp .
In addition, for any random function g(x) with the ith observation gi (x), we
define Γ(g) to be a second moment based linear operator such that
Z 1
Γ(g)φ(x) ≡
E{gi (x)gi (s)}φ(s)ds
0

= E{< gi , φ > gi (x)}

while its empirical version can be written as
Γn (g)φ(x) ≡ n

−1

n
X

< gi , φ > gi (x).

i=1

Using these definitions, we can write
Z 1
Z
< Γ(g)Brk , Brl > = E
Brk (x)gi (x)dx
< Γn (g)Brk , Brl > = n

−1

0
n Z 1
X
i=1

0



1

0

Brk (x)gi (x)dx

Brl (x)gi (x)dx ,
Z

1

Brl (x)gi (x)dx.

0

b
Define C(g) as a dθ × dθ matrix with its (k, l) element < Γ(g)Brk , Brl >, and C(g)

as a dθ × dθ matrix matrix with its (k, l) element < Γn (g)Brk , Brl >.
S.2. Lemmas and their Proofs.

1

2

Lemma 1.

There is a constant Dr > 0 such that for each spline

and for each 1 ≤ p ≤ ∞
0

Dr kc kp ≤ k

dθ
X
k=1

Pdθ

k=1 ck Brk ,

ck Brk kp ≤ kc0 kp ,

where c0 = {ck {(tk − tk−r )/r}1/p , k = 1, . . . , dθ }T .
Proof: This is a direct consequence of Theorem 5.4.2 on page 145 in DeVore and
Lorentz (1993).
Lemma 2.

Let Γ(g) be a strictly positive definite operator on B. Let u be a

dθ -dimensional vector with kuk2 = 1. There exist positive constants D1 , D2 , D3 , D4
such that
D1 hb ≤ uT C(g)u ≤ D2 hb ,
−1 −1
T
−1
D2−1 h−1
b ≤ u C(g) u ≤ D1 hb ,

and
b
D3 hb ≤ uT C(g)u
≤ D4 hb ,

−1 −1
Tb
−1
D4−1 h−1
b ≤ u C(g) u ≤ D3 hb

in probability.

Proof: First note that by the Cauchy-Schwartz inequality, Lemma 1 and Condition
(A6), we have
E

"


Z

≤ E

uT

0

0

=

Z

2 #
Br (x)gi (x)dx

(d
θ
1 X

Z 1 (X
dθ
0

1

uk Brk (x)

k=1

)2

uk Brk (x)

k=1

≤ ku0 k22 O(1)
= O(hb ),

)2

dx

dxE

Z

1

0

Z

0

1



gi (x)2 dx
2

gi (x) dx



NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR

3

where u0 = {uk {(tk − tk−r )/r}1/2 , k = 1, . . . , dθ }T , whose L2 norm is of order
1/2

O(hb ). Thus we have uT C(β)u ≤ D2 hb for some positive constant D2 < ∞.

As shown in (28) in Cardot, Ferraty and Sarda (2003), since the eigenvalues of
the covariance operator Γ(g) are strictly positive definite on B, there is a positive
constant D such that
< Γ(g)φ, φ >≥ Dkφk2 , for φ ∈ B.
Note that BT
r u ∈ B, so we have
T
T
2
uT C(g)u =< Γ(g)BT
r u, Br u >≥ DkBr uk2 ≥ D1 kuk hb = D1 hb

for a positive constant D1 by Lemma 1 and Condition (A4). Therefore, D1 hb ≤

−1 −1
T −1
uT C(g)u ≤ D2 hb . And so D2−1 h−1
b ≤ u C (g)u ≤ D1 hb . Further, with Theo-

rem 1.19 in Chatelin (1983), we have
b
kC(g)
− C(g)k2 ≤ sup

dθ
X

1≤l≤dθ k=1

kΓn − Γk2 | < Brk , Brl > |

As shown in Cardot, Ferraty and Sarda (2003), Lemma 5.3 in Cardot, Ferraty and
Sarda (1999) implies kΓn − Γk2 = op (n(hb −1)/2 ). Further, by the property of BPθ
spline basis, we have when |k − l| > r, Brk Brl = 0. Therefore, sup1≤l≤dθ dk=1
|<
Brk , Brl > | = O(hb ), which implies
(S.1)

b
kC(g)
− C(g)k2 ≤ op (hb n(hb −1)/2 ).

Now because hb < 1, combine with the result that D1 hb ≤ uT C(g)u ≤ D2 hb , by the
triangular inequality we obtain

b
b
uT C(g)u
= uT C(g)u + uT {C(g)
− C(g)}u

b
≤ D2 hb + kuk2 k{C(g)
− C(g)}uk2
b
≤ D2 hb + k{C(g)
− C(g)}k2 kuk2
= D2 hb + op (hb )

and
b
b
uT C(g)u
= uT C(g)u + uT {C(g)
− C(g)}u

4

b
≥ D1 hb − kuk2 k{C(g)
− C(g)}uk2
b
≥ D1 hb − k{C(g)
− C(g)}k2 kuk2
= D1 hb + op (hb ).

b
Thus, D3 hb ≤ uT C(g)u
≤ D4 hb in probability for some positive constant D3 , D4 <

b −1 u ≤ D−1 h−1 in probability. This proves the result.
∞. And so D4−1 hb ≤ uT C(g)
3
b
Lemma 3.

Assume Ci (·) is a continuous random function of x ∈ [0, 1]. At each

x, |E{Ci (x)}| < ∞. kCi (·)k2 < ∞ a.s.. Then
|n

−1

n Z
X

1

Brk (x)Ci (t)dx| = Op (hb )

0

i=1

if E{Ci (x)} =
6 0 and
|n

n Z
X

−1

i=1

1

0

Brk (x)Ci (x)dx| = Op {

p
hb n−1 log(n)}

if E{Ci (x)} = 0.
Proof: By the Bernstein’s inequality in Bosq (1998), we have
n Z
X

1/n

i=1

"

= Op 

n
X
i=1

0

1

Brk (x)Ci (x)dx − E


E 1/n

Z

Z

1

Brk (x)Ci (x)dx

0

1

Brk (x)Ci (x)dx

0

p
= Op { hb n−1 log(n)}.

2

logn

#1/2 





The last equality holds from Lemma 1 by choosing gi = Ci . Now if E{Ci (x)} =
6 0,
then because Brk is positive in the interval (tk−r , tk ), and is 0 otherwise (page 88 in
De Boor (1978))
Z
E

0

1

Brk (x)Ci (x)dx



=

Z

1

Brk (x)E{Ci (x)}dx
Z 1
= |E{Ci (ξ)}
Brk (x)dx|
0
Z 1
= |E{Ci (ξ)}|
Brk (x)dx
0

0

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR

5

≤ D9 (tk − tk−r )
(S.2)

= Op (hb ).

where ξ is a point in the interval [0, 1], D9 is a finite constant. The second equality holds by the assumption that Ci (·) is continuous function in x and the mean
value theorem. The inequality holds because the support of Brk is the interval
(tk−r , tk ) and |E{Ci (x)}| < ∞ for any x ∈ [0, 1]. Therefore, by Condition (A4)

that N −1 n(logn)−1 → ∞ we have
kn

−1

n Z
X
i=1

1

Brk (x)Ci (x)dxk2 = Op (hb )

0

for E{Ci (x)} =
6 0 and
kn

−1

n Z
X
i=1

1

Brk (x)Ci (x)dxk2 =

0

p

hb n−1 log(n)

for E{Ci (x)} = 0. This proves the results.
S.3. Proof of Proposition 1. First note that for any h(x), we have
(S.3)

R1

exp[log{fX0 (x)}]fU (Wi − x)h(x)dx
R1
0 exp[log{fX0 (x)}]fU (Wi − x)dx
!
R1
exp[log{f
(x)}]h(x)dx
X0
− 0R 1
0 exp[log{fX0 (x)}]dx
)
Z 1
Z (R 1
f
(x)f
(y
−
x)h(x)dx
i
X0
U
0
−
fX0 (x)h(x)dx
=
R1
0
0 fX0 (x)fU (yi − x)dx
Z 1
×
fX0 (x)fU (wi − x)dxdµ(yi )
0
Z 1Z
Z 1
=
fX0 (x)fU (wi − x)h(x)dµ(wi )dx −
fX0 (x)h(x)dx
E

0

0

(S.4)

0

= 0,

hence
"R 1

exp[log{fX0 (x)}]fU (Wi − x)Brk (x)dx
R1
0 exp[log{fX0 (x)}]fU (Wi − x)dx
#
R1
0 exp[log{fX0 (x)}]Brk (x)dx
−
=0
R1
exp[log{f
(x)}]dx
X0
0

E

0

6

for each Brk (x), k = 1, . . . , dθ . Therefore, by Condition (C5), we have
"R 1
exp{Br (x)T θ 0 }fU (Wi − x)Brk (x)dx
E 0 R1
T
0 exp{Br (x) θ 0 }fU (Wi − x)dx
#
R1
T
0 exp{Br (x) θ 0 }Brk (x)dx
= Op (hqb ),
−
R1
T
0 exp{Br (x) θ 0 }dx
which suggests that

Further, since

n
X

0

n
X

"R 1

b U (Wi − x)Brk (x)dx
exp{Br (x)T θ}f
R1
Tb
i=1
0 exp{Br (x) θ}fU (Wi − x)dx
#
R1
T θ}B
b rk (x)dx
exp{B
(x)
r
− 0 R1
= 0,
b
exp{Br (x)T θ}dx

n−1
(S.5)

"R 1

exp{Br (x)T θ 0 }fU (Wi − x)Brk (x)dx
R1
T
i=1
0 exp{Br (x) θ 0 }fU (Wi − x)dx
#
R1
T θ }B (x)dx
exp{B
(x)
r
0
rk
− 0 R1
= op (1).
T
0 exp{Br (x) θ 0 }dx

n

−1

0

0

we have

(S.6)

n
X

"R 1

exp{Br (x)T θ 0 }fU (Wi − x)Br (x)dx
R1
T
i=1
0 exp{Br (x) θ 0 }fU (Wi − x)dx
#
R1
T θ }B (x)dx
exp{B
(x)
r
0
r
− 0 R1
T
0 exp{Br (x) θ 0 }dx
"
R1
n
Tb
X
−1
0 exp{Br (x) θ}fU (Wi − x)Br (x)dx
−n
R1
Tb
i=1
0 exp{Br (x) θ}fU (Wi − x)dx
#
R1
Tb
0 exp{Br (x) θ}Br (x)dx
= op (1)
− R1
b
exp{Br (x)T θ}dx
n

−1

0

0

element-wise. By Condition (C6), as a function of θ,
"R 1
#!
R1
T
T
0 exp{Br (x) θ}fU (Wi − x)Br (x)dx
0 exp{Br (x) θ}Br (x)dx
E
−
R1
R1
T θ}f (W − x)dx
T
exp{B
(x)
r
i
U
0
0 exp{Br (x) θ}dx

has its derivative with respect to θ being a nonsingular matrix in the neighborhood
of its zero. Thus,
"R 1
#
R1
n
T
T
X
−1
0 exp{Br (x) θ}Brk (x)dx
0 exp{Br (x) θ}fU (Wi − x)Br (x)dx
n
−
,
R1
R1
T θ}f (W − x)dx
T θ}dx
exp{B
(x)
exp{B
(x)
r
i
r
U
i=1
0
0

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR

7

also has nonsingular derivative with respect to θ and hence is an invertible function
of θ in the neighborhood. Therefore, from (S.5) and (S.6), by the continuous mapping
b − θ 0 = op (1) element-wise. This proves the results.
theorem, we have θ

S.4. Proof of Proposition 2. Proof: First note by using (S.3) at h(x) =

δ(x − x0 ) and then set x0 = x, we have
(S.7)

E

R1
0

= 0.

exp[log{fX0 (x)}]fU (Wi − x)

exp[log{fX0 (x)}]fU (Wi − x)dx

− R1
0

exp[log{fX0 (x)}]
exp[log{fX0 (x)}]dx

b is the maximum likelihood estimator, we have
Since θ
"
Z 1
n
X
∂
−1
b U (Wi − x)dx
n
log
exp{Br (x)T θ}f
0 =
∂θ
0
i=1

Z 1
Tb
−log
exp{Br (x) θ}dx
0

(S.8)

where for any θ,

b − θ 0 ) + op {kR1 (θ 0 )(θ
b − θ 0 )k2 },
= R0 + R1 (θ 0 )(θ

R1 (θ) =
(S.9)

"
Z 1
n
X
∂2
−1
n
log
exp{Br (x)T θ}fU (Wi − x)dx
∂θ∂θ T
0
i=1

Z 1
exp{Br (x)T θ}dx ,
−log
0

b and
and θ ∗ is on the line connecting θ 0 and θ,
n R1
T
X
−1
0 exp{Br (x) θ 0 }fU (Wi − x)Br (x)dx
R0 = n
R1
T
i=1
0 exp{Br (x) θ 0 }fU (Wi − x)dx
R1
exp{Br (x)T θ 0 }Br (x)dx
− 0 R1
T
0 exp{Br (x) θ 0 }dx
= R00 + R01 ,
where
R00 ≡ n

−1

n Z
X

1

R1

exp[log{fX0 (x)}]fU (Wi − x)

exp[log{fX0 (x)}]fU (Wi − x)dx
!
exp[log{fX0 (x)}]
−R 1
Br (x)dx
exp[log{f
(x)}]dx
X0
0
i=1

0

0

!

8

= n

−1

n Z
X
i=1

1

0

(

and
(S.10)

R01 ≡ n−1

n Z
X

R1

fX0 (x)fU (Wi − x)

0

1

fX0 (x)fU (Wi − x)dx

"

R1

)

− fX0 (x) Br (x)dx

exp{Br (x)T θ 0 }fU (Wi − x)

exp{Br (x)T θ 0 }fU (Wi − x)dx
#
exp{Br (x)T θ 0 }
Br (x)dx
−R 1
T θ }dx
exp{B
(x)
r
0
0
n Z 1
X
exp[log{fX0 (x)}]fU (Wi − x)
−1
−n
R1
i=1 0
0 exp[log{fX0 (x)}]fU (Wi − x)dx
!
exp[log{fX0 (x)}]
−R 1
Br (x)dx.
0 exp[log{fX0 (x)}]dx

Now

0

i=1

kR00 k∞ = sup n

−1

0

n Z
X

1

R1

exp[log{fX0 (x)}]fU (Wi − x)

exp[log{fX0 (x)}]fU (Wi − x)dx
!
exp[log{fX0 (x)}]
−R 1
Brk (x)dx
0 exp[log{fX0 (x)}]dx
p
= Op { hb n−1 log(n)}.
k

(S.11)

0

i=1

0

The last equality holds by Lemma 3 in the supplementary material with
Ci (x) =

R1
0

fX0 (x)fU (Wi − x)

fX0 (x)fU (Wi − x)dx

− R1
0

fX0 (x)
fX0 (x)dx

,

which satisfies E{Ci (x)} = 0 by (S.7), and kCi (x)k2 < ∞ by Condition (C1).
Further,
(S.12)

kR01 k∞
"

= sup

n−1

n Z
X

1

("

R1

exp{Br (x)T θ 0 }fU (Wi − x)

exp{Br (x)T θ 0 }fU (Wi − x)dx
#
exp[log{fX0 (x)}]fU (Wi − x)
exp{Br (x)T θ 0 }
−R 1
− R1
T
exp{fX0 (x)}fU (Wi − x)dx
0 exp{Br (x) θ 0 }dx
!) 0
#
exp[log{fX0 (x)}]
−R 1
Brk (x)dx
exp[log{f
(x)}]dx
X0
0
"
Z 1 ("
n
X
exp{Br (x)T θ 0 }fU (Wi − x)
q
−q
−1
= hb sup n
hb
R1
T
k
0
i=1
0 exp{Br (x) θ 0 }fU (Wi − x)dx
k

i=1

0

0

9

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR

−R 1
0

exp{Br (x)T θ 0 }

exp{Br (x)T θ 0 }dx

#

exp[log{fX0 (x)}]

−R 1

exp[log{fX0 (x)}]dx

0

−

!)

R1
0

exp[log{fX0 (x)}]fU (Wi − x)

exp[log{fX0 (x)}]fU (Wi − x)dx
#

Brk (x)dx

= Op (hq+1
b ).
The last equality holds by using Lemma 3 in the supplementary material. with

=

Ci (x)
("
h−q
b
−

R1
0

R1
0

exp{Br (x)T θ 0 }fU (Wi − x)

exp{Br (x)T θ 0 }fU (Wi − x)dx

exp[log{fX0 (x)}]fU (Wi − x)

− R1
0

#

exp{Br (x)T θ 0 }dx
!)
exp[log{fX0 (x)}]
0

−R 1

exp[log{fX0 (x)}]fU (Wi − x)dx

exp{Br (x)T θ 0 }

exp[log{fX0 (x)}]dx

,

which satisfies kCi (x)k2 < ∞. Combining (S.11) and (S.12) we have R0 = R00 {1 +
op (1)} by Conditions (C3) and (C4). Further,
E(RT
00 R00 )
(

)2 d Z
2
θ
1
X
fX0 (ξ)fU (Wi − ξ)
≤ n−1 E  sup R 1
− fX0 (ξ)
Brk (x)dx 
ξ
f
(x)f
(W
−
x)dx
0
i
U
l=k
0 X0
= Op (n−1 hb ).

where ξ is a point in (0, 1). The last equality holds because
fX0 (ξ)fU (Wi − ξ)

sup R 1
ξ

fX0 (x)fU (Wi − x)dx

0

− fX0 (ξ)

is bounded by Condition (C1). Further,
dθ Z
X
k=1

0

1

Brk (x)dx

2

=

dθ
X
k=1

Dk (tk − tk−r )2

= sup |tk − tk−r |D
k

= Op (hb ),
for some finite constant Dk and D. The second equality holds because the support
of Brk is the interval (tk−r , tk ). The last equality holds because Dk is bounded above
since Brk (x) < 1 for any x. Therefore,
(S.13)

1/2

kR0 k2 = Op (n−1/2 hb ).

10

To analyze R1 (θ 0 ), we first have that for any u such that kuk2 = 1, from Lemma
1 in the supplementary material,
R1
T
T
T 0 Br (x)Br (x) exp{Br (x) θ 0 }fU (Wi − x)dx
u
u
R1
T θ }f (W − x)dx
exp{B
(x)
r
0
i
U
0
2

1/2

T

=

u exp{Br (x)T θ 0 /2}fU (Wi − x)

Br (x) R 1
[ 0 exp{Br (x)T θ 0 }fU (Wi − x)dx]1/2

1/2
u exp{Br (x)T θ 0 /2}fU (Wi − x)
= Op hb R 1
[ 0 exp{Br (x)T θ 0 }fU (Wi − x)dx]1/2

2

2
2

= Op (hb ),



and
R1
T
T
T 0 Br (x)Br (x) exp{Br (x) θ 0 }dx
u
u
R1
T
0 exp{Br (x) θ 0 }dx

=

T

u exp{Br (x)T θ 0 /2}

Br (x) R 1
[ 0 exp{Br (x)T θ 0 }dx]1/2 2


2
T
u exp{Br (x) θ 0 /2}

= O hb R 1
[ 0 exp{Br (x)T θ 0 }dx]1/2 2
= O(hb ).

Similarly
R
T θ }f (W − x)dx 1 BT (x) exp{B (x)T θ }dx
B
(x)
exp{B
(x)
0
r
r
r
0
i
U
r
u
uT 0
R01
R1
T
T
0 exp{Br (x) θ 0 }fU (Wi − x)dx 0 exp{Br (x) θ 0 }dx
"R 1
#2
T
T
0 u Br (x) exp{Br (x) θ 0 }fU (Wi − x)dx
≤
R1
T
0 exp{Br (x) θ 0 }fU (Wi − x)dx
"R 1
#
T (x)u exp{B (x)T θ }dx 2
B
r
0
+ 0 Rr1
,
T
0 exp{Br (x) θ 0 }dx
R1

while

"R 1
0

T
BT
r (x)u exp{Br (x) θ 0 }fU (Wi − x)dx
R1
T
0 exp{Br (x) θ 0 }fU (Wi − x)dx

#2

2

T
BT
r (x)u exp{Br (x) θ 0 }fU (Wi − x)
≤
R1
T
0 exp{Br (x) θ 0 }fU (Wi − x)dx
1
"
u exp{Br (x)T θ 0 }fU (Wi − x)
≤ O p hb R 1
T
0 exp{Br (x) θ 0 }dxfU (Wi − x)

≤

Op (h2b ),

2

1

#2

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR

and

"R 1
0

T
BT
r (x)u exp{Br (x) θ 0 }dx
R1
T
0 exp{Br (x) θ 0 }dx

#2

2

T
BT
r (x)u exp{Br (x) θ 0 }
≤
R1
T
0 exp{Br (x) θ 0 }dx
1
"
T
u exp{Br (x) θ 0 }
≤ O hb R 1
T
0 exp{Br (x) θ 0 }dx

1

#2

≤ O(h2b ).
Combining these results, we have

∂ 2 fW (w, θ 0 )/∂θ 0 ∂θ T
0
u
fW (w, θ 0 )
"R 1
T
T
T
0 Br (x)Br (x) exp{Br (x) θ 0 }fU (w − x)dx
= u
R1
T
0 exp{Br (x) θ 0 }fU (w − x)dx
R1 T
R1
T
T
0 Br (x) exp{Br (x) θ 0 }fU (w − x)dx 0 Br (x) exp{Br (x) θ 0 }dx
−
R1
R1
T
T
0 exp{Br (x) θ 0 }fU (w − x)dx 0 exp{Br (x) θ 0 }dx
R
R1
1 T
T
T
0 Br (x) exp{Br (x) θ 0 }dx 0 Br (x) exp{Br (x) θ 0 }fU (w − x)dx
−
R1
R1
T
T
0 exp{Br (x) θ 0 }dx 0 exp{Br (x) θ 0 }fU (w − x)dx
R1
T
Br (x)BT
r (x) exp{Br (x) θ 0 }dx
− 0
R1
T
0 exp{Br (x) θ 0 }dx
#
R
R1
T θ }dx 1 BT (x) exp{B (x)T θ }dx
B
(x)
exp{B
(x)
r
0
r
r
0
r
0
+2 0
u
R1
{ 0 exp{Br (x)T θ 0 }dx}2
= Op (hb ),
uT

and
E

"
(

T∂

u

2 f (W , θ )/∂θ ∂θ T
i 0
0
W
0

"R 1

fW (Wi , θ 0 )

u

2 #

T
Br (x)BT
r (x) exp{Br (x) θ 0 }fU (Wi − x)dx
R1
T
0 exp{Br (x) θ 0 }fU (Wi − x)dx
R1
R
T θ }f (W − x)dx 1 BT (x) exp{B (x)T θ }dx
B
(x)
exp{B
(x)
r
r
0
i
r
0
U
r
− 0
R1
R01
T
T
0 exp{Br (x) θ 0 }fU (Wi − x)dx 0 exp{Br (x) θ 0 }dx
R1
R
T θ }dx 1 BT (x) exp{B (x)T θ }f (W − x)dx
B
(x)
exp{B
(x)
r
r
0
r
0 U
i
r
− 0
R1
R01
T
T
0 exp{Br (x) θ 0 }dx 0 exp{Br (x) θ 0 }fU (Wi − x)dx
R1
T
Br (x)BT
r (x) exp{Br (x) θ 0 }dx
− 0
R1
T
0 exp{Br (x) θ 0 }dx
# !2 
R1
R1 T

T
T
Br (x) exp{Br (x) θ 0 }dx 0 Br (x) exp{Br (x) θ 0 }dx
+2 0
u
R1

{ exp{Br (x)T θ 0 }dx}2

= E

u

T

0

0

11

12

= Op (h2b ).
Now using Bernstein’s inequality, for  = hb /log(n),


pr uT R1 (θ 0 )u − uT E{R1 (θ 0 )}u > 


−n2 2 /2
≤ exp
2
nE(uT [∂ 2 fW (Wi , θ 0 )/∂θ 0 ∂θ T
0 /fW (Wi , θ 0 ) − E{R1 (θ 0 )}]u) + nhb /3


−n2 /2
≤ exp
2
E[{uT ∂ 2 fW (Wi , θ 0 )/∂θ 0 ∂θ T
0 /fW (Wi , θ 0 )u} ] + hb /3


−n2 /2
→0
≤ exp
2
hb + hb /3
when n → ∞. Thus,
|uT R1 (θ 0 )u − uT E{R1 (θ 0 )}u| = op (hb ).

(S.14)
Further,
(S.15)

E{R1 (θ 0 )}
"
 2

 #
∂ fW (Wi , θ 0 )/∂θ 0 ∂θ T
∂logfW (Wi , θ 0 ) ⊗2
0
= E
−E
fW (Wi , θ 0 )
∂θ 0
"

 #
Z  2
∂ fW (w, θ 0 )/∂θ 0 ∂θ T
∂logfW (Wi , θ 0 ) ⊗2
0
=
fW 0 (w)dw − E
fW (w, θ 0 )
∂θ 0
"
#

∂logfW (Wi , θ 0 ) ⊗2
q−1
,
= Op (hb ) − E
∂θ 0

where fW 0 (w) is the true density of W . The last equality holds by Condition (C5)
so that
Z 


∂ 2 fW (w, θ 0 )/∂θ 0 ∂θ T
0
fW 0 (w)dw
fW (w, θ 0 )

Z  2
∂ fW (w, θ 0 )/∂θ 0 ∂θ T
0
fW (w, θ 0 )dw
−
fW (w, θ 0 )
R
and the fact that fW (w, θ)dw = 1 so
Z 

∂ 2 fW (w, θ 0 )/∂θ 0 ∂θ T
0
fW (w, θ 0 )
R
∂ 2 fW (w, θ 0 )dw
=
∂θ 0 ∂θ T
0
= 0.



2

= Op (hq−1
b )

fW (w, θ 0 )dw

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR

Now

13

"

 #
∂logfW (Wi , θ) ⊗2
E
∂θ
"R 1
T
0 exp{Br (x) θ}fU (Wi − x)Br (x)dx
= E
R1
T
0 exp{Br (x) θ}fU (Wi − x)dx
#⊗2 
R1
T
exp{Br (x) θ}Br (x)dx
.
− 0 R1
T θ}dx
exp{B
(x)
r
0

(S.16)

First,
" R
#⊗2 
R1
1
T
T
exp{Br (x) θ}Br (x)dx
exp{Br (x) θ}fU (Wi − x)Br (x)dx
 6= 0
− 0 R1
E 0 R1
T
T θ}dx
exp{B
(x)
θ}f
(W
−
x)dx
exp{B
(x)
r
i
r
U
0
0
by Condition (C6). Also, it is easy to see that,
" R
# 
R1
1
T θ}f (W − x)B (x)dx
T θ}B (x)dx ⊗2
exp{B
(x)
exp{B
(x)
r
i
r
r
r
U

E 0 R1
− 0 R1
T θ}f (W − x)dx
T
exp{B
(x)
r
i
U
0
0 exp{Br (x) θ}dx
is C(g) defined in Section S.1 in the supplementary material with
gi (x) = R 1
0

exp{Br (x)T θ}fU (Wi − x)

exp{Br (x)T θ}fU (Wi − x)dx

− R1
0

exp{Br (x)T θ}
exp{Br (x)T θ}dx

.

First note that Γ(g) is a strictly positive definite operator on B. We prove this
by contradiction. Note that Γ(g) is obviously positive definite. Assume Γ(g) is not
strictlyy positive definite, then it has a zero eigenvalue when operating on B, hence
the matix

"R 1

exp{Br (x)T θ}fU (Wi − x)Br (x)dx
R1
T
0 exp{Br (x) θ}fU (Wi − x)dx
#⊗2 
R1
T
exp{Br (x) θ}Br (x)dx

− 0 R1
T θ}dx
exp{B
(x)
r
0

< Γ(g)Br , Br > = E

0

has a zero eigenvalue. By (S.15) and (S.16), this implies E{R1 (θ 0 )} is not be invertible, which contradicts with Condition (C6) that the score function has unique
solution around the truth. Hence by using Lemma 2 in the supplementary material,
for u such that kuk2 = 1, we have that there exist constants C > c > 0, such that
"R 1
T
T
0 exp{Br (x) θ}fU (Wi − x)Br (x)dx
(S.17)
chb ≤ u E
R1
T
0 exp{Br (x) θ}fU (Wi − x)dx

14

−

R1
0

exp{Br (x)T θ}Br (x)dx
R1
T
0 exp{Br (x) θ}dx

#⊗2 

 u ≤ Chb .

Combining the results in (S.14), (S.15), (S.16) and (S.17), there exist constants
0 < c1 < C1 , such that
−C1 hb ≤ uT R1 (θ 0 )u ≤ −c1 hb
in probability, and in turn
−1
−1 −1
T
−1
−c−1
1 hb ≤ u R1 (θ 0 ) u ≤ −C1 hb

(S.18)

b 0 k2 = Op (kR1 (θ 0 )−1 R0 k2 ) ≤ Op {kR1 (θ 0 )−1 k2 kR0 k2 } =
in probability. Therefore, red kθ−θ

b 0 )| ≤
Op {(nhb )−1/2 }. Therefore, for any vector a with kak2 = Op (1) we have |aT (θ−θ

b − θ 0 )k2 = Op {(nhb )−1/2 }.
kak2 k(θ

Further, expanding the second derivative, we can write
R1 (θ 0 )
R1

exp[log{fX0 (x)}]fU (Wi − x)Br (x)Br (x)T dx
R1
0 exp[log{fX0 (x)}]fU (Wi − x)dx
hR
i⊗2
1
exp[log{f
(x)}]f
(W
−
x)B
(x)dx
i
r
X0
U
0
−
hR
i2
1
exp[log{f
(x)}]f
(W
−
x)dx
i
X0
U
0
R1
exp[log{fX0 (x)}]Br (x)Br (x)T dx
− 0
R1
0 exp[log{fX0 (x)}]dx
hR
i⊗2 
1
exp[log{f
(x)}]B
(x)dx
r
X0
0

+
 {1 + op (1)}
hR
i2
1
exp[log{f
(x)}]dx
X0
0
"R 1
fX0 (x)fU (Wi − x)Br (x)Br (x)T dx
= E 0
R1
0 fX0 (x)fU (Wi − x)dx
nR
o⊗2
1
f
(x)f
(W
−
x)B
(x)dx
i
r
X0
U
0
−
nR
o2
1
f
(x)f
(W
−
x)dx
i
X0
U
0
Z 1
⊗2 #
Z 1
T
{1 + op (1)}.
−
fX0 (x)Br (x)Br (x) dx +
fX0 (x)Br (x)dx
0

= E

0

0

Hence we can write
b − θ0
θ

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR

=

15

"R 1

fX0 (x)fU (Wi − x)Br (x)Br (x)T dx
R1
0 fX0 (x)fU (Wi − x)dx
nR
o⊗2
1
f
(x)f
(W
−
x)B
(x)dx
i
r
U
0 X0
−
nR
o2
1
f
(x)f
(W
−
x)dx
i
U
0 X0
Z 1
−
fX0 (x)Br (x)Br (x)T dx
0
Z
⊗2 #!−1
E

0

1

+

fX0 (x)Br (x)dx

0

R00 {1 + op (1)}.

This proves the results.
S.5. Proof of Proposition 3. First note that by the definition of a(X, m),
we have
E[S∗β (Wi , Yi , m) − E ∗ {a(X, m) | Yi , Wi , m} | X, m] = 0.
This leads to
(S.19)

E[S∗β (Wi , Yi , β 0 ) − E ∗ {a(X, β 0 ) | Yi , Wi , β 0 }] = op (1)

element-wise by Condition (D4). By Condition (D5), E[S∗β (Wi , Yi , β)−E ∗ {a(X, β) |

Yi , Wi , β}] is invertible near β ∗ and its inverse function is a one-to-one function with
bounded first derivative in the neighborhood of zero. Therefore kβ ∗ − β 0 k2 = op (1).
On the other hand, since
(S.20)

n−1

n
X
i=1

we obtain

b − E ∗ {a(X, β)
b | Yi , Wi , β}
b = 0,
S∗β (Wi , Yi , β)

b − E ∗ {a(X, β)
b | Yi , Wi , β}]
b = op (1).
E[S∗β (Wi , Yi , β)

b − β ∗ k2 = op (1).
Using the same argument regarding β 0 in (S.19), we obtain kβ

b − β k2 = op (1).
Thus, kβ
0

S.6. Proof of Proposition 4. By the Taylor expansion, we have
0 =

n
X
i=1

b − E ∗ {a(X, β)
b | Yi , Wi , β}
b
S∗β (Wi , Yi , β)

16

b − β ),
= T0 + T1 (β ∗ )(β
0

(S.21)

b , and for any β
where β ∗ is a point on the line connecting β 0 and β

n
X
∂[S∗β (Wi , Yi , β) − E ∗ {a(X, β) | Yi , Wi , β}]

T1 (β) = n−1

∂β T

i=1

,

and
T0 = n−1

n
X
i=1

S∗β (Wi , Yi , β 0 ) − E ∗ {a(X, β 0 ) | Yi , Wi , β 0 }

= T00 + T01 ,
where
T00 = n−1

n
X

S∗β (Wi , Yi , m) − E ∗ {a(X, m) | Yi , Wi , m},

i=1

and
T01 = n−1
"

n
X
i=1

− n

−1

S∗β (Wi , Yi , β 0 ) − E ∗ {a(X, β 0 ) | Yi , Wi , β 0 }
n
X

S∗β (Wi , Yi , m)

i=1

∗

#

− E {a(X, m) | Yi , Wi , m} .

By the definition of S∗β (Wi , Yi , m), we can write
kT00 k∞ = sup n−1
k

n Z
X
i=1

1

0

"

−R 1
0

∗ (x)
f0 {Yi − m(x)}fU (Wi − x)fX

∗ (x)dµ(x)
f {Yi − m(x)}fU (Wi − x)fX


−P (x, Wi , Yi , m) Brk (x)dµ(x) .

(S.22)
Now define
"

C1i (x) = − R 1
0

∗ (x)
f0 {Yi − m(x)}fU (Wi − x)fX

∗ (x)dµ(x)
f {Yi − m(x)}fU (Wi − x)fX

#

− P (x, Wi , Yi , m) ,

by the Bernstein’s inequality in Bosq (1998), we have
1/n

n Z
X
i=1

"

= Op 

0

n
X
i=1

1

Brk (x)C1i (x)dx − E


E 1/n

Z

0

Z

1

Brk (x)C1i (x)dx

0

1

Brk (x)C1i (x)dx

2

#1/2 

logn



NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR

= Op {

(S.23)

17

p
hb n−1 log(n)}.

The last equality holds from Lemma 2 in the supplementary material, by choosing
gi (x) = C1i (x) and setting u with uk = 1 and ul = 0, for l 6= k. Now
E[S∗β (Wi , Yi , m) − E ∗ {a(X, m) | Yi , Wi , m}]
#
!
Z 1"
∗ (x)
f0 {Y − m(x)}fU (W − x)fX
− P (x, W, Y, m) Br (x)dx
= E
−R 1
∗ (x)dx
f
{Y
−
m(x)}f
(W
−
x)f
0

U
X
0
= 0,
nR
1

which further implies E

0

o
Brk (x)C1i (x)dx = 0 for all k. Plug it into (S.23), and

combine with (S.22), we obtain

p
kT00 k∞ = Op { hb n−1 log(n)}.

(S.24)
Further,
kT01 k∞

(S.25)

= sup n−1
k

n Z
X

1

0

i=1

"

−R 1
0

"

∗ (x)
f0 {Yi − Br (x)T β 0 }fU (Wi − x)fX

∗ (x)dµ(x)
f {Yi − Br (x)T β 0 }fU (Wi − x)fX

−P (x, Wi , Yi , β 0 )] − − R 1
0

∗ (x)
f0 {Yi − m(x)}fU (Wi − x)fX

∗ (x)dµ(x)
f {Yi − m(x)}fU (Wi − x)fX

−P (x, Wi , Yi , m)]) Brk (x)dµ(x)

= sup n

−1

k

n
X
∂Sm (Yi , Wi , m)
i=1

= Op (1) sup
k

=

∂m

Op (hq+1
b ),

Z

0

1

Brk (x)dx

(Brk , BT
r β − m) {1 + op (1)}

(

)

sup |Br (x)T β − m(x)| {1 + op (1)}

x∈[0,1]

The second equality holds by the Taylor expansion with respect to Br (·)T β. The
third equality holds by Condition (D2). The last equality holds by Condition (D4)
and the fact that the support of Brk is the interval (tk−r , tk ). Combining (S.24) and
(S.25) we have T0 = T00 {1 + op (1)} by Conditions (C3) and (C4). Further,
E(TT
00 T00 )

18


dβ Z 1 "
 X
∗ (x)
f 0 {Yi − m(x)}fU (Wi − x)fX
−R 1 
= n−1 E 
∗

l=1 0
0 f {Yi − m(x)}fU (Wi − x)fX (x)dµ(x)
o
−P (x, Wi , Yi , m)] Brl (x)dx)2
≤ n
=

−1

G1

dβ Z
X

l=1
−1
Op (n hb ),

1

Brl (x)dx

0

2

for some constant G1 . The second equality holds by Condition (D2) and the fact
that,
dβ Z
X

X
l=1

Brl (x)dx

0

l=1
dβ

=

1

2

D1l (tl − tl−r )2

≤ sup |tl − tl−r |D1
l

= Op (hb ),
for some finite constant D1l and D1 . The second equality holds because the support
of Brl is the interval (tl−r , tl ). The last equality holds because D1l are bounded
above since Brl (x) < 1 for any x. Therefore,
1/2

kT0 k2 = Op (n−1/2 hb ).

(S.26)

Further for arbitrary vector u with kuk2 = 1, we can write T1 (β ∗ ) as


dβ
dβ
n
∗)
X
X
X
∂S
(Y
,
W
,
m
m i
i

|uT T1 (β ∗ )u| = n−1
uk Brk ,
uk Brk 
∂m∗
i=1

k=1

k=1

dβ

= {c + op (1)}k

X
k=1

uk Brk k22

for a positive constant c, where m∗ (·) = Br (·)T β ∗ . The last equality holds by ConP
dition (D2) and the fact that n−1 ni=1 ∂Sm (Yi , Wi , m∗ )/∂m∗ is a non-singular operator on B. To show the non-singularity, we prove by contradiction. Recall that
Sm (Y, W, m∗ )(s)

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR

=

Z

0

and

1

"

∗ (x)dµ(x)
f0 {Y − m∗ (x)}fU (W − x)fX

−R 1
0

∗ (x)dµ(x)
f {Y − m∗ (x)}fU (W − x)fX

∗

19

#

− P (x, Y, W, m ) s(x)dx.

S∗β (Wi , Yi , β ∗ ) − E ∗ {a(X, β ∗ )|Yi , Wi , β ∗ }
"
#
Z 1
∗ (x)dµ(x)
f0 {Y − m∗ (x)}fU (W − x)fX
∗
Br (x) − R 1
=
− P (x, Y, W, m ) dx.
∗ (x)}f (W − x)f ∗ (x)dµ(x)
f
{Y
−
m
0

U
X
0

Suppose E{∂Sm (Yi , Wi , m∗ )/∂m∗ } is singular on B, then
)
(
∂S∗β (Wi , Yi , β ∗ ) − E ∗ {a(X, β ∗ )|Yi , Wi , β ∗ }
E
∂β ∗ T
#
(Z
"
1
∗ (x)dµ(x)
f0 {Y − m∗ (x)}fU (W − x)fX
∂
− P (x, Y, W, m∗ )
−R 1
= E
Br (x)
∗ (x)}f (W − x)f ∗ (x)dµ(x)
∂m∗ (x)
f
{Y
−
m
0

U
X
0
Br (x)T dx

is a singular matrix. However by Condition (D5) we know that the score function
E[S∗β (Wi , Yi , β) − E ∗ {a(X, β)|Yi , Wi , β}]
has a unique solution around β 0 , and hence
(
)
∂S∗β (Wi , Yi , β ∗ ) − E ∗ {a(X, β ∗ )|Yi , Wi , β ∗ }
E
∂β ∗ T
cannot be singular. Therefore, E{∂Sm (Yi , Wi , m∗ )/∂m} is indeed a non-singular operator on B.. Further,
P
n−1 ni=1 {∂Sm (Yi , Wi , m∗ )/∂m∗ } is a consistent estimator of E{∂Sm (Yi , Wi , m∗ )/∂m∗ },

and hence
P
n−1 ni=1 {∂Sm (Yi , Wi , m∗ )/∂m∗ } is also a non-singular operator on B for sufficiently large n. Further by using Lemme 1 in the supplementary material, we have
Pdβ
1/2
1/2
g3 hb = Dr ku0 k2 ≤ k k=1
uk Brk k2 ≤ ku0 k2 = G3 hb , where u0 = {uk {(tk −

tk−r )/r}1/2 , k = 1, . . . , dβ }T and g3 , G3 are finite constants. These together with
Lemma 2 imply
(S.27)

c9 hb ≤ uT T1 (β ∗ )u ≤ C9 hb ,

and
(S.28)

−1 −1
T
∗ −1
C9−1 h−1
b ≤ u T1 (β ) u ≤ c9 hb ,

20

in probability. Combining with (S.21), (S.26), we have
b − β k2 ≤ kT1 (β ∗ )−1 k2 kT0 k2 = Op {(nhb )−1/2 }.
kβ
0

By the consistency of Br (x)T β ∗ to m(x), we have
T1 (β ∗ )
= E

∂[S∗β (Wi , Yi , β) − E ∗ {a(X, β) | Yi , Wi , β}]
∂β T

Br (·)T β=m(·)

!

{1 + op (1)}.

Therefore, we can write
b −β
β
0

= −E

∂[S∗β (Wi , Yi , β) − E ∗ {a(X, β) | Yi , Wi , β}]
∂β T

Br (·)T β=m(·)

!−1

×T00 {1 + op (1)}.
This proves the results.
References.
Bosq, D. (1998). Nonparametric statistics for stochastic processes : estimation and prediction D.
Bosq. Lecture notes in statistics: 110. Springer, New York.
Cardot, H., Ferraty, F. and Sarda, P. (1999). Functional linear model. Statistics & Probability
Letters 45 11–22.
Cardot, H., Ferraty, F. and Sarda, P. (2003). Spline estimators for the functional linear model.
Statistica Sinica 13 571–592.
Chatelin, F. (1983). Spectral Approximation of Linear Operators. SIAM, New York.
De Boor, C. (1978). A practical guide to splines 27. Springer-Verlag, New York.
DeVore, R. A. and Lorentz, G. G. (1993). Constructive approximation 303. Springer Science
& Business Media, New York.
Department of Statistics

Department of Statistics

University of Hongkong

Penn State University

Pok Fu Lam, Hongkong

University Park, PA 16802

E-mail: feijiang@hku.hk

E-mail: yzm63@psu.edu

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR

21

4
3
f

2
1
0

0

1

f

2

3

4

Fig S.1: B-spline MLE density estimation (left) and deconvolution estimation (right)
from 200 simulations: The solid lines represent the true functions and the dash lines
represent the estimated functions and their 90% confidence bands. The first row to
third row are the results for model II (a)–(c) respectively. Sample size 1000.

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.8

1.0

0.6

0.8

1.0

0.6

0.8

1.0

4
3
f

0

1

2

3
f

2
1
0
0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

3
f

1
0

0

1

2

2

3

4

x

4

x

f

0.6
x

4

x

0.0

0.2

0.4

0.6
x

0.8

1.0

0.0

0.2

0.4
x

22

4
3
f

2
1
0

0

1

f

2

3

4

Fig S.2: B-spline MLE density estimation (left) and deconvolution estimation (right)
from 200 simulations: The solid lines represent the true functions and the dash lines
represent the estimated functions and their 90% confidence bands. The first row to
third row are the results for model II (a)–(c) respectively. Sample size 2000.

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.8

1.0

0.6

0.8

1.0

0.6

0.8

1.0

4
3
f

0

1

2

3
f

2
1
0
0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

3
f

1
0

0

1

2

2

3

4

x

4

x

f

0.6
x

4

x

0.0

0.2

0.4

0.6
x

0.8

1.0

0.0

0.2

0.4
x

NEW NONPARAMETRIC MEASUREMENT ERROR MODEL ESTIMATOR

23

3
m
−2

−3

−2

−1

−1

0

0

m

1

1

2

2

3

Fig S.3: B-spline semiparametric regression estimation (left) and deconvolution estimation (right) from 200 simulations: The solid lines represent the true functions and
the dash lines represent the estimated functions and their 90% confidence bands.
The first row to third row are the results for model II (a)–(c), respectively. Sample
size 1000.

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

0.6

0.8

1.0

0.6

0.8

1.0

m
−2

−3

−2

−1

−1

0

0

m

1

1

2

2

3

x

3

x

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

m
0

0

−1

−1

−2

−2
−3

m

1

1

2

2

3

x

3

x

0.0

0.2

0.4

0.6
x

0.8

1.0

0.0

0.2

0.4
x

24

3
m
−2

−3

−2

−1

−1

0

0

m

1

1

2

2

3

Fig S.4: B-spline semiparametric regression estimation (left) and deconvolution estimation (right) from 200 simulations: The solid lines represent the true functions and
the dash lines represent the estimated functions and their 90% confidence bands.
The first row to third row are the results for model II (a)–(c), respectively. Sample
size 2000.

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

0.6

0.8

1.0

0.6

0.8

1.0

m
−2

−3

−2

−1

−1

0

0

m

1

1

2

2

3

x

3

x

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

m
0

0

−1

−1

−2

−2
−3

m

1

1

2

2

3

x

3

x

0.0

0.2

0.4

0.6
x

0.8

1.0

0.0

0.2

0.4
x

