Hypotheses tests in boundary regression
models
Holger Drees, Natalie Neumeyer and Leonie Selk∗

arXiv:1408.3979v2 [stat.ME] 11 Oct 2016

University of Hamburg, Department of Mathematics
Bundesstrasse 55, 20146 Hamburg, Germany
October 12, 2016
Abstract
Consider a nonparametric regression model with one-sided errors and regression
function in a general Hölder class. We estimate the regression function via minimization of the local integral of a polynomial approximation. We show uniform rates of
convergence for the simple regression estimator as well as for a smooth version. These
rates carry over to mean regression models with a symmetric and bounded error distribution. In such a setting, one obtains faster rates for irregular error distributions
concentrating sufficient mass near the endpoints than for the usual regular distribu√
tions. The results are applied to prove asymptotic n-equivalence of a residual-based
(sequential) empirical distribution function to the (sequential) empirical distribution
function of unobserved errors in the case of irregular error distributions. This result is
remarkably different from corresponding results in mean regression with regular errors.
It can readily be applied to develop goodness-of-fit tests for the error distribution. We
present some examples and investigate the small sample performance in a simulation
study. We further discuss asymptotically distribution-free hypotheses tests for independence of the error distribution from the points of measurement and for monotonicity
of the boundary function as well.

AMS 2010 Classification: Primary 62G08; Secondary 62G10, 62G30, 62G32
Keywords and Phrases: goodness-of-fit testing, irregular error distribution, one-sided
errors, residual empirical distribution function, uniform rates of convergence

1

Introduction

We consider boundary regression models of the form
Yi = g(xi ) + εi ,
1

i = 1, . . . , n,

with negative errors εi whose survival function 1−F (y) behaves like a multiple of |y|α for some
α > 0 near the origin. Such models naturally arise in image analysis, analysis of auctions
and records, or in extreme value analysis with covariates. For such a boundary regression
model with multivariate random covariates and twice differentiable regression function, Hall
and Van Keilegom (2009) establish a minimax rate for estimation of g(x) (for fixed x) under
quadratic loss and determine pointwise asymptotic distributions of an estimator which is
defined as a solution of a linear optimization problem (cf. Remark 2.6). Relatedly, Müller
and Wefelmeyer (2010) consider a mean regression model with (unknown) symmetric support
of the error distribution and Hölder continuous regression function. They discuss pointwise
MSE rates for estimators of the regression function that are defined as the average of local
maxima and local minima. Meister and Reiß (2013) consider a regression model with known
bounded support of the errors. They show asymptotic equivalence in the strong LeCam sense
to a continuous-time Poisson point process model when the error density has a jump at the
endpoint of its support. For a regression model with error distribution that is one-sided and
regularly varying at 0 with index α > 0, Jirak et al. (2014) suggest an estimator for the
boundary regression function which adapts simultaneously to the unknown smoothness of
the regression function and to the unknown extreme value index α. Reiß and Selk (2016+)
construct efficient and unbiased estimators of linear functionals of the regression function in
the case of exponentially distributed errors as well as in the limiting Poisson point process
experiment by Meister and Reiß (2013).
Closely related to regression estimation in models with one-sided errors is the estimation
of a boundary function g based on a sample from (X, Y ) with support {(x, y) ∈ [0, 1]×[0, ∞] |
y ≤ g(x)}. For such models, Härdle et al. (1995) and Hall et al. (1998) proved minimax
rates both for g(x) and for the L1 -distance between g and its estimator. Moreover, they
showed that an approach using local polynomial approximations of g yields this optimal
rate. Explicit estimators in terms of higher order moments were proposed and analyzed
by Girard and Jacob (2008) and Girard et al. (2013). Daouia et al. (2016) consider spline
estimation of a support frontier curve and obtain uniform rates of convergence.
The aim of the paper is to develop tests for model assumptions in boundary regression
models. In particular we will suggest asymptotically distribution-free tests for
• parametric classes of error distributions (goodness-of-fit)
• independence of the error distribution from the points of measurement
• monotonicity of the boundary function.
The test statistics are based on (sequential) empirical processes of residuals. To investigate
these, we need uniform rates of convergence for the regression estimator, which are of interest
on its own. To our knowledge, uniform rates so far have only been shown by Daouia et al.
(2016) who do not obtain optimal rates. Our results can also be applied to mean regression
2

models with bounded symmetric error distribution. For regression functions g in a Hölder
class of order β, we obtain the rate ((log n)/n)β/(αβ+1) . Thus, for tail index α ∈ (0, 2) of the
error distribution, the rate is faster than the typical rate one has in mean regression models
with regular errors. For pointwise and Lp -rates of convergence, it has been known in the
literature that faster rates are possible for nonparametric regression estimation in models
with irregular error distribution, see e.g. Gijbels and Peng (2000), Hall and Van Keilegom
(2009), or Müller and Wefelmeyer (2010).
The uniform rate of convergence for the regression estimator enables us to derive asymptotic expansions for residual-based empirical distribution functions and to prove weak convergence of the residual-based (sequential) empirical distribution function. We state conditions
under which the influence of the regression estimation is negligible such that the same results
are obtained as in the case of observable errors. We apply the results to derive goodnessof-fit tests for parametric classes of error distributions. Asymptotic properties of residual
empirical distribution functions in mean regression models were investigated by Akritas and
Van Keilegom (2001), among others. As the regression estimation strongly influences the
asymptotic behavior of the empirical distribution function in these regular models, asymptotic distributions of goodness-of-fit test statistics are involved, and typically bootstrap is
applied to obtain critical values, see Neumeyer et al. (2006). In contrast, in the present
situation with an irregular error distribution, standard critical values can be used.
In nonparametric frontier models, Wilson (2003) discusses several possible tests for assumptions of independence, for instance independence between input levels and output inefficiency. Those assumptions are needed to prove validity of bootstrap procedures and are
thus crucial in applications, but they may be violated; see Simar and Wilson (1998). Wilson (2003) points out the analogy to tests for independence between errors and covariates
in regression models, but no asymptotic distributions are derived. Tests for independence
in nonparametric mean and quantile regression models that are similar to the test we will
consider are suggested by Einmahl and Van Keilegom (2008) and Birke et al. (2016+).
There is an extensive literature on regression with one-sided error distributions and similar models (in particular production frontier models) which assume monotonicity of the
boundary function, see Gijbels et al. (1999), the literature cited therein and the monotone
nonparametric maximum likelihood estimator in Reiß and Selk (2016+). Monotonicity of
a production frontier function in each component is given under the strong disposability
assumption, but may often not be fulfilled; see e.g. Färe and Grosskopf (1983). We are
not aware of hypothesis tests for monotonicity or other shape constraints in the context of
boundary regression, but would like to mention Gijbels’ (2005) review on testing for monotonicity in mean regression. Tests similar in spirit to the one we are suggesting here were
considered by Birke and Neumeyer (2013) and Birke et al. (2016+) for mean and quantile
regression models, respectively.
The remainder of the article is organized as follows. In Section 2 the regression model
3

under consideration is presented and model assumptions are formulated. The regression
estimator is defined and uniform rates of convergence are given. A smooth modification of
the estimator is considered and uniform rates of convergence for this estimator as well as its
derivative are shown. In Section 3 residual based empirical distribution functions based on
both regression estimators are investigated. Conditions are stated under which the influence
√
of regression estimation is asymptotically n-negligible. Furthermore, an expansion of the
residual empirical distribution function is shown that is valid under more general conditions.
In Section 4 goodness-of-fit tests for the error distribution are discussed in general and in
some detailed examples. We investigate the finite sample performance of the tests in a small
simulation study. We further discuss hypotheses tests for independence of the error distribution from the design points as well as a test for monotonicity of the boundary function.
All proofs are given in the appendix.

2

The regression function: uniform rates of convergence

We consider a regression model with fixed equidistant design and one-sided errors,
Yi = g( ni ) + εi ,

i = 1, . . . , n,

(2.1)

under the following assumptions:
(F1) The errors ε1 , . . . , εn are independent and identically distributed and supported on
(−∞, 0]. The error distribution function fulfills
F (y) = 1 − c|y|α + r(y),

y < 0,

for some α > 0, with r(y) = o(|y|α ) for y % 0.
(G1) The regression function g belongs to some Hölder class of order β ∈ (0, ∞), i. e. g is
bβc-times differentiable on [0, 1] and the bβc-th derivative satisfies
cg := sup
t,x∈[0,1]
t6=x

|g (bβc) (t) − g (bβc) (x)|
< ∞.
|t − x|β−bβc

In Figure 1 some scatter plots of data according to model (2.1) are shown for different
tail indices α of the error distribution.
Remark 2.1 Strictly speaking, we consider a triangular scheme in (2.1), and the errors εi
depend on n too, as the ith regression point i/n varies with n. For notational simplicity, we
suppress the second index, because the distribution of the errors does not depend on n. 
4

0.0

0.2

0.4

0.6
x

0.8

1.0

0.2

0.4

0.6

0.8

1.0

0.2
0.0
g (x)

−0.6

−0.4

g (x)

−0.4
−0.6
0.0

0.0

x

−0.2

0.0

0.2

α=3

−0.2

0.0
−0.6

−0.4

g (x)

−0.2

0.0
−0.2
−0.6

−0.4

g (x)

α=2

0.2

α=1

0.2

α = 0.5

0.2

0.4

0.6

0.8

1.0

0.0

x

0.2

0.4

0.6

0.8

1.0

x

Figure 1: Scatter plots of ( ni , Yi ), i = 1, . . . , n, and the true regression function g(x) =
−3(x − 0.4)3 . The error distribution is Weibull F (y) = exp(−(|y|/θ)α )I(−∞,0) (y) + I[0,∞) (y)
with scale θ = 0.3 and shape parameter α.
We consider an estimator that locally approximates the regression function by a polynomial while lying above the data points. More specifically, for x ∈ [0, 1], let
ĝn (x) := ĝ(x) := p(x)
where p is a polynomial of order dβe − 1 and minimizes the local integral
Z x+hn
p(t) dt

(2.2)

x−hn

under the constraints p( nj ) ≥ Yj for all j ∈ {1, . . . , n} such that | nj − x| ≤ hn . For the
asymptotic analysis of this estimator, we need the following assumption:
(H1) Let (hn )n∈N be a sequence of positive bandwidths that satisfies limn→∞ hn = 0 and
limn→∞ nhn / log n = ∞.
We obtain the following uniform rates of convergence.
Theorem 2.2 In model (2.1), under the assumptions (F1), (G1), and (H1), we have
 | log h | 1/α 
n
sup |ĝ(x) − g(x)| = O(hβn ) + OP
.
nh
n
x∈[hn ,1−hn ]
Note that the deterministic part O(hβn ) arises from approximating the regression function
by a polynomial, whereas the random part originates from the observational error. Balancing
1
the two sources of error by setting hn  ((log n)/n) αβ+1 gives
β
 log n  αβ+1

sup |ĝ(x) − g(x)| = OP
.
(2.3)
n
x∈[hn ,1−hn ]
(Here an  bn means that 0 < lim inf n→∞ |an /bn | ≤ lim supn→∞ |an /bn | < ∞.)
This result is of particular interest in the case of irregular error distributions, i. e. α ∈
β
(0, 2), when the rate improves upon the typical optimal rate OP (((log n)/n) 2β+1 ) for estimating mean regression functions in models with regular errors.
5

Remark 2.3 Jirak et al. (2014) consider a similar boundary regression estimator while
P
replacing the integral in (2.2) by its Riemann approximation ni=1 p( ni )I{| ni − x| ≤ hn }. In
particular, they use the Lepski method to construct a data-driven bandwidth that satisfies
1
hn  ((log n)/n) αβ+1 in probability. For this modified estimator, we obtain the same uniform
rate of convergence as in Theorem 2.2 by replacing Proposition A.1 in the proof of Theorem
2.2 by Theorem 3.1 in Jirak et al. (2014). 
Remark 2.4 For Hölder continuous regression functions with exponent β ∈ (0, 1] the estimator reduces to a local maximum, i. e. ĝ(x) = max{Yi | i = 1, . . . , n s. t. | ni − x| ≤ hn }.
In this case we obtain the rate of convergence as given in Theorem 2.2 uniformly over the
whole unit interval. 
Remark 2.5 Müller and Wefelmeyer (2010) consider a mean regression model Yi = m(Xi )+
ηi , i = 1, . . . , n, with symmetric error distribution supported on [−a, a] (with a unknown);
see the left panel of Figure 2. The error distribution function fulfills F (a − y) ∼ 1 − y α for
y & 0. The local empirical midrange of responses, i. e.

1
min Yi + max Yi
m̂(x) =
i∈{1,...,n}
i∈{1,...,n}
2 |X
|X −x|≤h
−x|≤h
n

i

n

i

is shown to have pointwise rate of convergence O(hβn ) + OP ((nhn )−1/α ) to m(x) if m is
Hölder continuous with exponent β ∈ (0, 1]. Theorem 2.2 enables us to extend Müller’s
and Wefelmeyer’s (2010) results in two ways (in a model with fixed design Xi = ni ): we
consider more general Hölder classes with general index β > 0, and we obtain uniform rates
ˆ
of convergence. To this end, we use the mean regression estimator m̂ = (ĝ − g̃)/2
with ĝ as
i
before and g̃ˆ defined analogously, but based on ( n , −Yi ), i = 1, . . . , n; see the right panel of
Figure 2. The rates obtained for supx∈[hn ,1−hn ] |m̂(x) − m(x)| are the same as in Theorem
2.2. 

1.5
1.0
y

0.5
0.0
−1.0

−0.5

0.0
−0.5
−1.0

y

0.5

1.0

1.5

2.0

mean regression: boundary curves

2.0

mean regression with symmetric compactly supported error

0.0

0.2

0.4

0.6

0.8

1.0

0.0

x

0.2

0.4

0.6
x

Figure 2: Example for data as in Remark 2.5.

6

0.8

1.0

Remark 2.6 For β ∈ (1, 2], Hall and Van Keilegom (2009) consider the following local
linear boundary regression estimator:
n
o

2
i
i
ǧ(x) = inf α0 (α0 , α1 ) ∈ R : Yi ≤ α0 + α1 n − x ∀i ∈ {1, . . . , n} s. t. n − x ≤ hn .
(2.4)
R x+hh
Because of x−hn (α0 + α1 (t − x)) dt = 2α0 hn this estimator coincides with ĝ for β ∈ (1, 2].
However, in the case β > 2 replacing the linear function in (2.4) by a polynomial of order
dβe − 1 renders the estimator ǧ useless. One obtains ǧ(x) = −∞ for x 6∈ { nj | j = 1, . . . , n}
while ǧ( nj ) = Yj , j = 1, . . . , n. This was already observed by Jirak et al. (2014). 
Note that typically the estimator ĝ is not continuous. One might prefer to consider
a smooth estimator by convoluting ĝ with a kernel. Such a modified estimator will also
be advantageous when deriving an expansion for the residual based empirical distribution
function in the next section. Therefore we define


Z 1−hn
x−z
1
dz
(2.5)
ĝ(z) K
g̃(x) =
bn
bn
hn
and formulate some additional assumptions.
R
(K1) K is a continuous kernel with support [−1, 1] and order bβc + 1, i.e. K(u) du = 1,
R r
u K(u) du = 0 ∀r = 1, . . . , bβc. Furthermore, K is differentiable with Lipschitzcontinuous derivative K 0 on (−1, 1).
(B1) The sequence (bn )n∈N of positive bandwidths satisfies limn→∞ bn = 0.
(

 log n 1/α

o b1+2δ
if δ ≤
n
β
(1+2δ)∨(3−(β−1)(1/δ−1))
(B2.δ) hn +
= o bn
=

3−(β−1)(1/δ−1)
nhn
o bn
if δ >

β−1
2
β−1
.
2

Here we assume that the parameter δ, which quantifies the minimal required smoothness of
the estimator of g 0 , lies in (0, 1 ∧ (β − 1)). For example, if β < 3 and the optimal bandwidth
hn  ((log n)/n)1/(αβ+1) is chosen, then (B2.δ) is fulfilled with δ = (β − 1)/2 for any bn that
satisfies hn = o(bn ).
The estimator g̃ is differentiable and we obtain the following uniform rates of convergence
for g̃ and its derivative g̃ 0 .
Theorem 2.7 If the model assumptions (2.1), (F1), (G1) with β > 1, (H1), (K1), and
(B1) hold, then for In = [hn + bn , 1 − hn − bn ]
 | log h |  α1 
n
β
β
(i) sup |g̃(x) − g(x)| = O(bn ) + O(hn ) + OP
nhn
x∈In
 | log h |  α1 
n
(ii) sup |g̃ (x) − g (x)| =
+O
+ OP
.
nhn
x∈In
If hβn + (log n/(nhn ))1/α = o(bn ), then supx∈In |g̃ 0 (x) − g 0 (x)| = oP (1); in particular this
holds if (B2.δ) is fulfilled for some δ ∈ (0, 1 ∧ (β − 1)).
0

0

O(bβ−1
n )

β
b−1
n hn



7



b−1
n

(iii) For all δ ∈ (0, 1 ∧ (β − 1)), under the additional assumption (B2.δ),
|g̃ 0 (x) − g 0 (x) − g̃ 0 (y) + g 0 (y)|
= oP (1).
|x − y|δ
x,y∈In ,x6=y
sup

3
3.1

The error distribution
Estimation

In this section we consider estimators of the error distribution in model (2.1). For the
asymptotic analysis we need the following additional assumption.
(F2) The cdf F of the errors is Hölder continuous of order α ∧ 1.
We define residuals ε̂i = Yi − ĝ( ni ), and a resulting modified sequential empirical distribution function by
bnsc
1 X
F̂n (y, s) =
I{ε̂i ≤ y}I{hn <
mn i=1

i
n

≤ 1 − hn },

where mn = ]{i ∈ {1, . . . , n} | hn < ni ≤ 1 − hn } = n − bnhn c − dnhn e. We consider the
sequential process, because it will be useful for testing hypotheses in section 4. With slight
abuse of notation, let F̂n (y) = F̂n (y, 1) denote the corresponding estimator for F (y).
We first treat a simple case where the influence of the regression estimation on the residual
empirical process is negligible. To this end, let Fn denote the standard empirical distribution
function of the unobservable errors ε1 , . . . , εn . Furthermore, define s̄n = bn(s ∧ (1 − hn ))c −

bn(s ∧ hn )c /mn and interpret s̄n /bnsc as 0 for s = 0. Note that s̄n = 1 if s = 1 and sn → s
as n → ∞, for each fixed s.
Theorem 3.1 Assume that the conditions (F1), (G1), and (F2) are fulfilled with β > 1.
Furthermore, assume β1 < α < 2 − β1 and hn  ((log n)/n)1/(αβ+1) . Then we have
sup

|F̂n (y, s) − s̄n Fbnsc (y)| = oP (n−1/2 ).

y∈R,s∈[0,1]

√
Thus the process { n(F̂n (y, s) − s̄n F (y)) | s ∈ [0, 1], y ∈ R} converges weakly to a Kiefer
process KF , a centered Gaussian process with covariance function ((s1 , y1 ), (s2 , y2 )) 7→ (s1 ∧
s2 )(F (y1 ∧ y2 ) − F (y1 )F (y2 )).
Remark 3.2 The assertion of Theorem 3.1 holds true under the following weaker conditions
on the (possibly random) bandwidth:

hn = oP n−1/(2(α∧1)β) ,

n(α∨1)/2−1 log n = oP (hn ).
8

(3.1)

In particular, one may use the adaptive bandwidth proposed by Jirak et al. (2014).
Condition (3.1) can be fulfilled if and only if β1 < α < 2 − β1 , which in turn can be satisfied
for all α ∈ (0, 2), provided the regression function g is sufficiently smooth. It ensures that
one can choose a rate an of larger order than the uniform bound on the estimation error
established in Theorem 2.2 such that
−1/2
|F (y + an ) − F (y)| = O(aα∧1
). 
n ) = o(n

Remark 3.3 Theorem 3.1 implies that for α ∈ (1/β, 2 − 1/β) the estimation of the regression function has no impact on the estimation of the irregular error distribution. This is
remarkably different from corresponding results on the estimation of the error distribution
in mean regression models with regular error distributions. Here the empirical distribution
√
function of residuals, say F̌n , is not asymptotically n-equivalent to the empirical distribu√
tion function of true errors. The process n(F̌n − F ) converges to a Gaussian process whose
covariance structure depends on the error distribution in a complicated way; cf. Theorem 2
in Akritas and Van Keilegom (2001). In the simple case of a mean regression model with
equidistant design and an error distribution F with bounded density f one has
√

n

f (y) X
n(F̌n (y) − Fn (y)) = √
εi + oP (1)
n i=1

uniformly with respect to y ∈ R when the regression function is estimated by a local polynomial estimator, under appropriate bandwidth conditions (see Proposition 3 in Neumeyer
and Van Keilegom (2009)). 
In order to obtain asymptotic results for estimators of the error distribution for α ≥ 2− β1 ,
a finer analysis is needed. In what follows, we will use the smooth regression estimator
g̃ defined in (2.5). Let F̃n denote the empirical distribution function based on residuals
ε̃j = Yj − g̃( nj ), i. e.
n
1 X
F̃n (y) =
I{ε̃j ≤ y}I{ nj ∈ In }
mn j=1
where In = [hn + bn , 1 − hn − bn ] and mn = ]{j ∈ {1, . . . , n} | hn + bn ≤ nj ≤ 1 − hn − bn } =
n − 2dn(hn + bn )e + 1. Then the following asymptotic expansion is valid.
Theorem 3.4 If the conditions (F1), (F2), (G1) with β > 1, (H1), (K1), (B1), and
(B2.δ) for some δ ∈ (1/α − 1, 1 ∧ (β − 1)) are fulfilled, then
F̃n (y) =

n
n
 1 


1X
1 X
I{εj ≤ y}+
F y + (g̃ − g)( nj ) − F (y) I{ nj ∈ In }+oP √
(3.2)
n j=1
mn j=1
n

uniformly for all y ∈ R.
9

Remark 3.5 One can choose bandwidths hn and bn such that the conditions (H1), (B1)
and (B2.δ) are fulfilled for some δ ∈ (1/α − 1, 1 ∧ (β − 1)) if this interval is not empty, which
in turn is equivalent to α > 1/(β ∧ 2). Thus the expansion given in Theorem 3.4 is also valid
for regular error distributions.
If one assumes (B2.δ) for some δ ∈ (0, 1 ∧ (β − 1)), but drops the condition δ > 1/α − 1 and,
in addition, replaces (F2) with the assumption that F is Lipschitz continuous on (−∞, κ]
for some κ < 0, then expansion (3.2) still holds uniformly on (−∞, κ̃] for all κ̃ < κ. In
particular, this holds if F has a bounded density on (−∞, κ]. 
Next we examine under which conditions the additional term in (3.2) depending on the
estimation error is asymptotically negligible. We focus on those arguments y which are
bounded away from 0, because in this setting weaker conditions on α and β are needed.
Moreover, for the analysis of the tail behavior of the error distribution at 0, tail empirical
processes are better suited and will be considered in future work.
Note that the estimator ĝ tends to underestimate the true function because it is defined
via a polynomial which is minimal under the constraint that it lies above all observations
(i/n, Yi ), which in turn all lie below the true boundary function. As this systematic underestimation does not vanish from (local or global) averaging, we first have to introduce a bias
correction.
Let Eg≡0 denote the expectation if the true regression function is identical 0. For the
remaining part of this section, we assume that Eg≡0 (ĝ(1/2)) is known or that it can be estimated sufficiently accurately. For example, if the empirical process of residuals shall be used
to test a simple null hypothesis, then one may calculate or simulate this expectation under
the given null distribution. We define a bias corrected version of the smoothed estimator by
g̃n∗ (x) := g̃(x) − Eg≡0 (ĝ(1/2)),
for x ∈ In . The following lemma ensures that the above results for g̃ carry over to this
variant if the following condition on the lower tail of F holds:
(F3) There exists τ > 0 such that F (−t) = o(t−τ ) as t → ∞.
Lemma 3.6 If model (2.1) holds with g identical 0 and the conditions (F1), (F3), (G1),
and (H1) are fulfilled, then for all x ∈ [hn , 1 − hn ]
Eg≡0 (|ĝn (x)|) = Eg≡0 (|ĝn (1/2)|) = O

 log n 1/α 
.
nhn

We need some additional conditions on the rates at which the bandwidths hn and bn tend
to 0:

(H2) hn = o n−1/(2β) ∧ n−1/(αβ+1) , nα/4−1 log n = o(hn )
10



  nh 2/α

n
−1/(2β)
−2β −1
−1
(B3) bn = o n
∧ hn n
∧
n
log n
In particular, these assumptions ensure that the bias terms of order hβn + bβn are of smaller
order than n−1/2 and (nhn )−1/α and hence asymptotically negligible, and that quadratic
terms in the estimation error are uniformly negligible, that is, supx∈In |g̃n∗ (x) − g(x)|2 =
oP (n−1/2 ).
Theorem 3.7 Suppose the model assumptions (2.1) with α ∈ (0, 2), β > 1, (F1), (F3),
(G1), (H1), (H2), (K1), (B1), (B2.δ) for some δ > 0, and (B3) hold and F has a
bounded density on (−∞, κ] for some κ < 0. Then
sup
y∈(−∞,κ]

n


1 X
F y + (g̃n∗ − g)( nj ) − F (y) I{ nj ∈ In } = oP (n−1/2 ).
mn j=1

Remark 3.8 The conditions on hn and bn used in Theorem 3.7 can be fulfilled if and only
if α < 2β − 1. In particular, this theorem is applicable if β ≥ 3/2 and the error distribution
is irregular, i.e., α < 2. A possible choice of bandwidths is
 1

β
2β − 1 
,
∧
. 
hn  n−1/(2β) ∧ n−1/(αβ+1) / log n, bn  n−λ for some λ ∈
2β αβ + 1
2αβ
We obtain asymptotic equivalence of the empirical process of residuals (restricted to
(−∞, κ]) to the empirical process of the errors. To formulate the result, let F̃n∗ be defined
analogously to F̃n , but with g̃ replaced by g̃ ∗ .
Corollary 3.9 Under the assumptions of Theorems 3.4 and 3.7, we have supy∈(−∞,κ] |F̃n∗ (y)−
√
Fn (y)| = oP (n−1/2 ). Thus the process ( n(F̃n∗ (y) − F (y)))y∈(−∞,κ] converges weakly to a
centered Gaussian process with covariance function (y1 , y2 ) 7→ F (y1 ∧ y2 ) − F (y1 )F (y2 ),
y1 , y2 ∈ (−∞, κ].
Note that for the Corollary one needs the condition 1/(β ∧ 2) < α < (2β − 1) ∧ 2.

4
4.1

Hypotheses testing
Goodness-of-fit testing

Let F = {Fϑ | ϑ ∈ Θ} denote a continuously parametrized class of error distributions such
that for each ϑ ∈ Θ, Fϑ (y) = 1 − cϑ |y|αϑ + rϑ (y) with rϑ (y) = o(|y|αϑ ) for y % 0. Our aim is
to test the null hypothesis H0 : F ∈ F. We assume that αϑ ∈ (1/β, 2 − 1/β) for all ϑ ∈ Θ,
such that Theorem 3.1 can be applied under H0 . Let ϑ̂ denote an estimator for ϑ based
on residuals ε̂i = Yi − ĝ( ni ), i = 1, . . . , n. The goodness-of-fit test is based on the empirical
process
√
Sn (y) = n(F̂n (y) − Fϑ̂ (y)), y ∈ R,
11

where, as before, F̂n (y) = F̂n (y, 1). Under any fixed alternative that fulfills (F1) for some α,
ĝ still uniformly consistently estimates g, and thus F̂n is a consistent estimator of the error
distribution F . If ϑ̂ converges to some ϑ∗ ∈ Θ under the alternative, too, then a consistent
hypothesis test is obtained by rejecting H0 for large values of, e. g., a Kolmogorov-Smirnov
test statistic supy∈R |Sn (y)|. Note that under H0 it follows from Theorem 3.1 that
Sn (y) =

√

n(Fn (y) − Fϑ (y)) −

√

n(Fϑ̂ (y) − Fϑ (y)) + oP (1),

where ϑ denotes the true parameter. We consider two examples.
Example 4.1 Consider the mean regression model Yi = m( ni ) + ηi , i = 1, . . . , n, with
symmetric error cdf F and β > 1, and define m̂ with some bandwidth hn  ((log n)/n)1/(αβ+1)
as in Remark 2.5. We want to test the null hypothesis H0 : F ∈ F = {Fϑ | ϑ ∈ Θ} for
some Θ ⊂ (0, ∞), where Fϑ denotes the distribution function of the uniform distribution on
[−ϑ, ϑ] (with αϑ = 1 for all ϑ > 0). Define residuals η̂i = Yi − m̂( ni ), i = 1, . . . , n, and let


max
|η̂i |.
ϑ̂n = max
max
η̂i , − min
η̂i =
nhn ≤i≤n−nhn

nhn ≤i≤n−nhn

nhn ≤i≤n−nhn

Then |ϑ̂n − ϑ| is bounded by | maxnhn ≤i≤n−nhn |ηi | − ϑ| + supx∈[hn ,1−hn ] |m̂(x) − m(x)| =
oP (n−1/2 ). Since Fϑ (y) = y+ϑ
I
(y) + I(ϑ,∞) (y), one may conclude supy∈R |Fϑ̂n (y) −
2ϑ [−ϑ,ϑ]
−1/2
Fϑ (y)| = oP (n
). Thus the process Sn converges weakly to a Brownian bridge B composed
with F . The Kolmogorov-Smirnov test statistic supy∈R |Sn (y)| converges in distribution to
supt∈[0,1] |B(t)|. Thus although our testing problem requires the estimation of a nonparametric function and we have a composite null hypothesis, the same asymptotic distribution
arises as in the Kolmogorov-Smirnov test for the simple hypothesis H0 : F = F0 based on
an iid sample with distribution F . 
Example 4.2 Again assume that the Hölder coefficient β is greater than 1. Consider the
α
null hypothesis H0 : F ∈ F = {Fϑ | ϑ ∈ (0, ∞)}, where Fϑ (y) = e−(−ϑy) I(−∞,0) (y)+I[0,∞) (y)
denotes a Weibull distribution with some fixed shape parameter α ∈ (1/β, 2 − 1/β) and
unknown scale parameter ϑ. Note that Fϑ satisfies (F1) with c = ϑ.
 P
− α1
n
j
1
α
Define the moment estimator ϑ̂n = mn j=1 (−ε̂j ) I{hn < n ≤ 1 − hn }
which is
α
−α
α
motivated by Eϑ [(−ε1 ) ] = ϑ . A Taylor expansion of x 7→ x at x = −εj yields
ϑ̂αn

n
 1 X
j
− ϑ = −(ϑ̂n ϑ)
((−εj )α − ϑ−α )I{hn < ≤ 1 − hn }
mn j=1
n
α

α

n
 j

α X
j 
j
α−1
+
(−ξj )
ĝ( ) − g( ) I{hn < ≤ 1 − hn }
mn j=1
n
n
n
n
1 X
j
= −ϑ
((−εj )α − ϑ−α )I{hn < ≤ 1 − hn } + oPϑ (n−1/2 )
mn j=1
n
2α

12

= OPϑ (n−1/2 )
for some ξj between ε̂j and εj , where in the last steps we have applied Theorem 2.2, the law
of large numbers and a central limit theorem.
For all z, z̃ ∈ R one has |e−z − e−z̃ − (z − z̃)e−z | = e−z |ez−z̃ − 1 − (z − z̃)| ≤ e−z∧z̃ (z − z̃)2 .
Thus
2
2
α
α
Fϑ̂n (y) − Fϑ (y) − e−(−ϑy) ϑ̂αn − ϑα | ≤ e−(−(ϑ̂n ∧ϑ)y) (ϑ̂αn − ϑα )(−y)α = OPϑ (n−1 )
uniformly for all y ∈ (−∞, 0]. Now analogously to the proof of Theorem 19.23 in van der
Vaart (2000) we can conclude weak convergence of
Sn (y) =

√

n(Fn (y) − F (y)) − e

√ X
n
n
1
j
((−εj )α − α )I{hn < ≤ 1 − hn }
ϑ (−y)
mn j=1
ϑ
n

−(−ϑy)α 2α

α

+ oPϑ (1),
y ∈ R, to a Gaussian process with covariance function (y1 , y2 ) 7→ Fϑ (y1 ∧y2 )−Fϑ (y1 )Fϑ (y2 )−
α α
α
e−(−ϑ) (y1 +y2 ) (y1 y2 )α ϑ2α , where the covariance function follows by simple calculations and
α
the fact that Eϑ [I{ε1 ≤ y}((−ε1 )α − ϑ−α )] = (−y)α e−(−ϑy) .
For the special case of a test for exponentially distributed errors (α = 1), the asymptotic
R
quantiles for the Cramér-von-Mises test statistic Sn (y)2 dFϑ̂n (y) are tabled in Stephens
(1976). 
Simulations
To study the finite sample performance of our goodness-of-fit test, we investigate its behaviour on simulated data according to Examples 4.1 and 4.2 for samples of size 50, 100, 200
and 500. In both settings the regression function is given by g(x) = 0.5 sin(2πx) + 4x. We
1
use the local linear estimator (corresponding to β = 2) with bandwidth n− 3 , which is up
to a log term of optimal rate for α = 1 and β = 2. The hypothesis tests are based on the
R
adjusted Cramér-von-Mises test statistic mnn Sn (y)2 dFϑ̂n (y) and have nominal size 5%. The
results reported below are based on 200 Monte Carlo simulations for each model.
In the situation of Example 4.1, the errors are drawn according to the density fε (y) =
0.5(ζ + 1)(1 − |y|)ζ I[−1,1] (y) for different values of ζ ∈ (−1, 0] . Note that the null hypothesis
H0 : ∃ϑ : εi ∼ U [−ϑ, ϑ] holds if and only if ζ = 0. Figure 3 shows the empirical power of
the Cramér-von-Mises type test. The actual size is close to the nominal level for all sample
sizes and the power function is monotone both in ζ and the sample size n. For parameter
values ζ ∈ [−0.2,0), one needs rather large sample sizes to detect the alternative, as the error
distribution is too similar to the uniform distribution.
In the setting of Example 4.2 we simulate Weibull(ϑ, α) distributed errors for ϑ = 1
and different values of α > 0. We test the null hypothesis H0 : ∃ϑ : −εi ∼ Exp(ϑ) of
13

1.0

exponentiality, which is only fulfilled for α = 1. In Figure 4 the empirical power function of
our test is displayed for different sample sizes. Again the actual size is close to the 5% and
the power increases with α departing from one as well as with increasing n.
To examine the influence of the bandwidth choice, in addition we have simulated the
1
same models with hn = c · n− 3 for different values of c ranging from c = 0.2 to c = 1.2. The
results for the test of uniformity in Example 4.1 are similar to those displayed in Figure 3
for all these bandwidths. In the situation of Example 4.2 we obtain similar power functions
as reported above for c between 0.8 and 1.2, whereas for smaller bandwidths the actual size
of the test exceeds its nominal value substantially.

●

●

●

0.6
0.4

Rejection Probability

0.8

●

n=50
n=100
n=200
n=500

0.2

●

level

●
●

0.0

●

−0.8

−0.6

−0.4

−0.2

0.0

ζ

Figure 3: Monte-Carlo simulations for Example 4.1

4.2

Test for independence

In model (2.1) we assume that the distributions of the errors εi (i = 1, . . . , n) do not depend
on the point of measurement xi = i/n. We can test this assumption by comparing the sequential empirical distribution function F̂n (y, s) for the residuals with the estimator s̄n F̂n (y),
which should behave similarly if the errors are iid. The following corollary to Theorem 3.1
describes the asymptotic behavior of the Kolmogorov-Smirnov type test statistic
Tn =

sup

√
n|F̂n (y, s) − s̄n F̂n (y)|

s∈[0,1],y∈R

14

1.0

●

●

●

●

●

●

●

●

●

●

●

0.6

●

0.4

●

0.2

Rejection Probability

0.8

●

●

●

level

●

n=50
n=100
n=200
n=500

0.0

●

0.5

1.0

1.5

2.0

2.5

3.0

α

Figure 4: Monte-Carlo simulations for Example 4.2

under the null hypothesis of iid errors.
Corollary 4.3 Assume model (2.1) with (F1), (F2), (G1), and β1 < α < 2 − β1 . Choose a
bandwidth hn  ((log n)/n)1/(αβ+1) .
Then Tn converges in distribution to sups∈[0,1],z∈[0,1] |G(s, z)| where G is a completely
tucked Brownian sheet, i. e. a centered Gaussian process with covariance function
((s1 , z1 ), (s2 , z2 )) 7→ (s1 ∧ s2 − s1 s2 )(z1 ∧ z2 − z1 z2 ).
The proof is given in the appendix. Note that under the assumptions of the corollary the
limit of the test statistic Tn is distribution free. The asymptotic quantiles tabled by Picard
(1985) can be used to determine the critical value for a given asymptotic size of the test.

4.3

Test for monotone boundary functions

We consider model (2.1) and aim at testing the null hypothesis
H0 :

g is increasing,

which is a common assumption in boundary models. Let g̃ denote the smooth local polynomial estimator for g defined in (2.5). Such an unconstrained estimator can be modified
15

to obtain an increasing estimator g̃I . To this end, for any function h : [0, 1] → R define the
increasing rearrangement on [a, b] ⊂ [0, 1] as the function Γ(h) : [a, b] → R with
Z b
o
n
I{h(t) ≤ z} dt ≥ x .
Γ(h)(x) = inf z ∈ R a +
a

Denote by Γn the operator Γ with [a, b] = In . We define the increasing rearrangement of
g̃ as g̃I = Γn (g̃), so that g̃I = g̃ if g̃ is nondecreasing (see Anevski and Fougères, 2007,
and Chernozhukov et al., 2009). We now consider residuals obtained from the monotone
estimator: ε̂I,i = Yi − g̃I ( ni ), i = 1, . . . , n. Under the null hypothesis, these residuals should
be approximately iid, whereas under the alternative they show a varying behavior for ni in
different subintervals of [0, 1]. For illustration see Figure 5 where we have generated a data
set (upper panel) with true non-monotone boundary curve g (dashed curve). The solid curve
is the increasing rearrangement gI . The lower left panel shows the errors εi , i = 1, . . . , n,
with iid-behaviour. The lower right panel shows εI,i = Yi − gI ( ni ), i = 1, . . . , n, with a clear
non-iid pattern.

●
●

●

●

●

●
● ●

●

●
●
●

●

●

●
●

●

●

●

●

●

● ● ●

●
●

●
●

● ●●

●

●

●

●
●

●

●
●

●●

●

●

●

●
●

●●

●

●

● ●

●

●

●

●

●

●

●

●
●
●

●

●

●

●

●

●

●

●
●
●

●

●

●

●

●
●

●

●

●

●

●

●

●

●

●

●
●

●

●

●

●
●

●

●

●
●

●
●

●

●

●

●

●

●

●

●

●
●

●

●

●

●
●

●
●

●

●
●

●

●
●

●

●

●

●

●●

●●

●

●●
●

●
●

●

●
●

●
●

●

●

●

● ●

●
●●

●
●

●

●
●

●

●●

●
●

●

●

●

●

●

●

●
●

s

●

●

●

●

●

●

●

●

0
−1
−2
−3
−4

−4

−3

−2

−1

0

●

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

Figure 5: The upper panel shows data points and the true boundary function (dashed curve)
as well as the increasing rearrangement (solid curve). The lower left panel shows the errors.
The lower right panel shows residuals built with respect to the increasing rearrangement.
16

Similarly as in Subsection 4.2, we compare the sequential empirical distribution function
bnsc
1 X
I{ε̃I,j ≤ y}I{ nj ∈ In }
F̃I,n (y, s) =
mn j=1

based on the increasing estimator g̃I with the product estimator s̄n F̃n (y, 1), where again
In := [hn + bn , 1 − hn − bn ] and mn := n − 2dn(hn + bn )e + 1. Let
√
G̃n (s, y) = n(F̃I,n (y, s) − s̄n F̃n (y)), s ∈ [0, 1], y ∈ R.
To derive its limit distribution under the null hypothesis, we need an additional assumption:
(I1) Let inf x∈[0,1] g 0 (x) > 0.
Theorem 4.4 Assume model (2.1) with (F1), (F2), (G1), (K1), (I1), β > 1 and
α < 2 − β1 . If hn  ((log n)/n)1/(αβ+1) and bn  ((log n)/n)1/(αβ+1) , then
sup

|F̃I,n (y, s) − s̄n Fbnsc (y)| = oP (n−1/2 ).

1
β

<

(4.1)

y∈R,s∈[0,1]

Thus the Kolmogorov-Smirnov test statistic sups∈[0,1],y∈R |G̃n (s, y)|, converges in distribution
to sups∈[0,1],z∈[0,1] |G(s, z)| where G is the completely tucked Brownian sheet (see Corollary
4.3).
The conditions on the bandwidths can be substantially relaxed; cf. Remark 3.2.
Remark 4.5 A test that rejects H0 for large values of the Kolmogorov-Smirnov test statistic
Tn = sups∈[0,1],y∈R |G̃n (s, y)| is consistent. To see this note that by Theorem 1 of Anevski and
Fougères (2007), supx∈In |g̃I (x) − gI (X)| ≤ supx∈In |g̃(x) − g(x)| = oP (1) with gI denoting
the increasing rearrangement of g. Thus n−1/2 Tn converges to
Z s
T = sup
Fε (y + (gI − g)(x)) dx − sFε (y) .
s∈[0,1],y∈R

0

Since T > 0 under the alternative hypothesis g 6= gI , the test statistic Tn converges to
infinity. 

A
A.1

Appendix: Proofs
Auxiliary results

Proposition A.1 Assume that model (2.1) holds and that the regression function g fulfills
condition (G1) for some β ∈ (0, β ∗ ] and some cg ∈ [0, c∗ ]. Then there exist constants
Lβ ∗ ,c∗ , Lβ ∗ > 0 and a natural number jβ ∗ (depending only on the respective subscripts) such
that

|ĝ(x) − g(x)| ≤ Lβ ∗ ,c∗ hβn + Lβ ∗ max
min
|εi | .
1≤j≤2jβ ∗

17

i:−1+(j−1)/jβ ∗ ≤|i/n−x|/hn ≤−1+j/jβ ∗

This proposition can be verified by an obvious modification of the proof of Theorem 3.1
by Jirak et al. (2014).
Lemma A.2 Under assumptions (F1) and (H1) for any fixed set I1 , . . . , Im of disjoint
non-degenerate subintervals of [−1, 1] we have
sup

max

x∈[hn ,1−hn ] 1≤j≤m

min
i∈{1,...,n}
(i/n−x)/hn ∈Ij

|εi | = OP

 | log h | 1/α 
n
.
nhn

1/α
Proof. Let rn := | log hn |/(nhn )
. Obviously it suffices to prove that for all nondegenerate subintervals I ⊂ [−1, 1] there exists a constant L such that
n
o
lim P
sup
min |εi | > Lrn = 0.
n→∞

x∈[hn ,1−hn ]

i∈{1,...,n}
(i/n−x)/hn ∈I

Denote by d = sup I − inf I > 0 the diameter of I and let dn := dnhn de − 1 and ln := bn/dn c.
Then for all x > 0
n
o
n
o
P
sup
min |εi | > x ≤ P
max
min
|εi | > x
x∈[hn ,1−hn ]

j∈{1,...,n−dn } i∈{j,...,j+dn }

i∈{1,...,n}
(i/n−x)/hn ∈I

≤ P

n

o
n
o
max Mn,l > x + P
max Mn,l > x

l∈{0,...,ln }
l even

l∈{0,...,ln }
l odd

with
Mn,l :=

max

min

j∈{ldn +1,...,(l+1)dn } i∈{j,...,j+dn }

|εi |.

Since the random variables Mn,l for l even are iid, we have
n
o
bln /2c+1
P
max Mn,l > x = 1 − 1 − P {Mn,0 > x}
,
l∈{0,...,ln }
l even

and an analogous equation holds for the maxima over the odd numbered block maxima Mn,l .
Let G be the cdf of |εi |. If Mn,0 exceeds x, then there is a smallest index j ∈ {1, . . . , dn }
for which mini∈{j,...,j+dn } |εi | > x. Hence
P {Mn,0 > x} = P

n

min

i∈{1,...,1+dn }

dn
o X
n
|εi | > x +
P |εj−1 | ≤ x,
j=2

min

i∈{j,...,j+dn }

o
|εi | > x

= (1 − G(x))dn +1 + (dn − 1)G(x)(1 − G(x))dn +1
≤ (1 + dn G(x))(1 − G(x))dn .
To sum up, we have shown that
P

n

sup

min

x∈[hn ,1−hn ]

i∈{1,...,n}
(i/n−x)/hn ∈I

|εi | > Lrn

o



bln /2c+1 
dn
≤ 2 1 − 1 − (1 + dn G(Lrn ))(1 − G(Lrn ))
.
18

It remains to be shown that the right hand side tends to 0 for sufficiently large L which is
true if and only if
(1 + dn G(Lrn ))(1 − G(Lrn ))dn = o(1/ln ).
This is an immediate consequence of 1/ln ∼ dhn and
| log hn |
(1 + o(1))
nhn


| log hn |
=⇒ (1 − G(Lrn ))dn = exp − nhn dcLα
(1 + o(1))
nh
 n

=⇒ (1 + dn G(Lrn ))(1 − G(Lrn ))dn = O | log hn | exp − cdLα | log hn |(1 + o(1)) = o(hn )

G(Lrn ) = cLα

2

if cdLα > 1.

A.2

Proof of Theorem 2.2

The assertion directly follows from Proposition A.1 and Lemma A.2.

A.3

2

Proof of Theorem 2.7

(i) Using Theorem 2.2, a Taylor expansion of g of order bβc and assumption (K1), one can
show by direct calculations that for some τu ∈ (0, 1)
1−hn




x−z
dz
sup |g̃(x) − g(x)| ≤ sup
bn
x∈In
x∈In
hn


Z 1−hn
1
x−z
(g(z) − g(x)) K
+ sup
dz
bn
bn
x∈In
hn
Z 1
(g(x − ubn ) − g(x))K (u) du
≤
sup |ĝ(z) − g(z)|O(1) + sup
Z

1
(ĝ(z) − g(z)) K
bn

x∈In

z∈[hn ,1−hn ]

−1

 | log h |  α1 
n
β
≤ O(hn ) + OP
nh
Z 1n
1
ubβc (g (bβc) (x − τu ubn ) − g (bβc) (x))K(u)du .
+bbβc
n sup
x∈In bβc! −1
Now the Hölder property of g combined by (K1) yields the desired result.
(ii) Since g is bounded on [hn , 1 − hn ] and supx∈[hn ,1−hn ] |ĝ(x) − g(x)| = oP (1), ĝ is
eventually bounded on [hn , 1 − hn ] too. Note that the partial derivative of ĝ(z)b−1
n K((x −
z)/bn ) with respect to x is continuous and bounded (for fixed n). Thus we can exchange
integration and differentiation and obtain
0

Z

0

1−hn

sup |g̃ (x) − g (x)| = sup
x∈In

x∈In

ĝ(z)
hn

19

1 0 x − z 
K
dz − g 0 (x) .
b2n
bn

Integration by parts yields




Z 1−hn
Z 1−hn
1
1 0 x−z
x−z
0
g (z) K
g(z) 2 K
dz =
dz
bn
bn
bn
bn
hn
hn
since K(−1) = K(1) = 0. Therefore


Z 1−hn
1 0 x−z
0
0
(ĝ(z) − g(z)) 2 K
dz
sup |g̃ (x) − g (x)| ≤ sup
bn
bn
x∈In
x∈In
hn


Z 1−hn
1
x−z
0
0
(g (z) − g (x)) K
+ sup
dz
bn
bn
x∈In
hn
Z 1
−1
(g 0 (x − ubn ) − g 0 (x))K (u) du .
≤
sup |ĝ(z) − g(z)|O(bn ) + sup
x∈In

z∈[hn ,1−hn ]

−1

Similarly as in the proof of (i), assertion (ii) follows by Theorem 2.2, a Taylor expansion of
g 0 of order bβc − 1 and the assumptions (K1) and (G1).
(iii) We distinguish the cases |x − y| > an and |x − y| ≤ an for some suitable sequence
(an )n∈N with limn→∞ an = 0 specified later. In the first case, we obtain
|g̃ 0 (x) − g 0 (x) − g̃ 0 (y) + g 0 (y)|
|x − y|δ

sup
x,y∈In ,|x−y|>an

≤ 2 sup |g̃ 0 (x) − g 0 (x)|a−δ
n
x∈In

O(bβ−1
n )+

=

O(hβn ) + OP



| log hn |
nhn

 α1 ! ! !
b−1
a−δ
n
n .

(A.1)

In the second case, we use a decomposition like in the proof of (ii):
sup
x,y∈In ,0<|x−y|≤an

≤

sup

|g̃ 0 (x) − g 0 (x) − g̃ 0 (y) + g 0 (y)|
|x − y|δ
 

 
R 1−hn
1
0 x−z
0 y−z
(ĝ(z)
−
g(z))
K
−
K
dz
2
b
bn
bn
hn
n

|x − y|δ
R 1−hn

x,y∈In ,0<|x−y|≤an
0

+

sup
x,y∈In
0<|x−y|≤an

0

|g (x) − g (y)|
+
|x − y|δ

hn

sup
x,y∈In
0<|x−y|≤an

 

 
y−z
g 0 (z) b1n K x−z
−
K
dz
bn
bn
|x − y|δ

.

By Lipschitz continuity of K 0 and Theorem 2.2, the first term on the right hand side is of
the order
!
 | log h |  α1  1
n
O(hβn ) + OP
O(an1−δ ).
(A.2)
nhn
b3n
For β ≥ 2, the second term is of the order an1−δ as g 0 is Lipschitz continuous, while for
β ∈ (1, 2) assumption (G1) yields the rate anβ−1−δ . In both cases, condition (B2.δ) ensures
20

that the second term converges to 0.
The last term on the right hand side can be rewritten as
R1 0
(g (x − hn u) − g 0 (y − hn u))K(u) du
−1
sup
|x − y|δ
x,y∈In
0<|x−y|≤an

and is thus of the same order as the second term by assumption (G1).
To conclude the proof, one needs to find a sequence an = o(1) such that (A.1) and (A.2)
tend to 0 in probability, i.e.
 b3 
ϑn
β−1
δ
1−δ
bn +
= o(an ) and an = o n
bn
ϑn
with ϑn := hβn + (| log hn |/(nhn ))1/α . Obviously, such a sequence an exists if and only if
bβ−1
+
n

δ 
 b3  1−δ
ϑn
n
,
=o
bn
ϑn

2

which in turn is equivalent to condition (B2.δ).

A.4

Proof of Theorem 3.1

The assumptions about α ensure that β/(αβ + 1) > 1/(2(α ∧ 1)), and so in view of (2.3)
the uniform estimation error of ĝ is stochastically of smaller order than n−1/(2(α∧1)) . Hence
there exists a sequence
1
an = o(n− 2(α∧1) )
(A.3)
such that
P




|ĝ(x) − g(x)| ≤ an −−−→ 1.

sup

n→∞

x∈[hn ,1−hn ]

Let F̄n (y, s) :=

1
mn

Pbnsc
j=1

I{εj ≤ y}I{hn <

j
n

≤ 1 − hn }. Since

bnsc
1 X
F̂n (y, s) =
I{εj ≤ y + (ĝ − g)( nj )}I{hn <
mn j=1

j
n

≤ 1 − hn }

we may conclude
√
√
n(F̄n (y − an , s) − s̄n Fbnsc (y)) ≤ n(F̂n (y, s) − s̄n Fbnsc (y))
√
≤ n(F̄n (y + an , s) − s̄n Fbnsc (y))
for all y ∈ R and s ∈ [0, 1] with probability converging to 1.
We take a closer look at the bounds. The sequential empirical process
−1/2

En (y, s) = n

bnsc
X

(I{εj ≤ y} − F (y)),

j=1

21

y ∈ R, s ∈ [0, 1],

(A.4)

converges weakly to a Kiefer process; see e.g. Theorem 2.12.1 in van der Vaart and Wellner
(1996). Now, n ∼ mn , the asymptotic equicontinuity of the process En , the Hölder continuity
(F2) and (A.3) imply

√
n F̄n (y ± an , s) − s̄n Fbnsc (y)

n 
=
En (y ± an , s ∧ (1 − hn )) − En (y, s ∧ (1 − hn )) − En (y ± an , s ∧ hn ) + En (y, s ∧ hn )
mn
√
√
+ ns̄n (F (y ± an ) − F (y)) + n(F̄n (y, s) − s̄n Fbnsc (y))
√
= oP (1) + n(F̄n (y, s) − s̄n Fbnsc (y))
uniformly for all y ∈ R, s ∈ [0, 1].
It remains to be shown that
√
n(F̄n (y, s) − s̄n Fbnsc (y))
√ X
√
bnsc
bnsc
n
ns̄n X
j
=
(I{εj ≤ y} − F (y))I{hn < ≤ 1 − hn } −
(I{εj ≤ y} − F (y))
mn j=1
n
bnsc j=1
 n


=
− 1 En (y, s ∧ (1 − hn )) − En (y, s ∧ hn )
mn

 ns̄
n
−
− 1 En (y, s)
bnsc

+ En (y, s ∧ (1 − hn )) − En (y, s ∧ hn ) − En (y, s)
(A.5)
tends to 0 in probability uniformly for all y ∈ R, s ∈ [0, 1].
The first term vanishes asymptotically, because En is uniformly stochastically bounded
and n ∼ mn .
Next note that s̄n = 0 for s < hn , while for s ≥ hn
ns̄n
bn(s ∧ (1 − hn )c − bnhn c
O(nhn )
−1=
−1=
,
−1
bnsc
(1 − 2hn + O(n ))bnsc
(1 − 2hn + O(n−1 ))bnsc

(A.6)

which is uniformly bounded for all s ∈ [hn , 1] and tends to 0 uniformly with respect to s ∈
1/2
[hn , 1]. Moreover, En is uniformly stochastically bounded and sup0≤s≤h1/2
|En (y, s)| =
n ,y∈R
oP (1), because En is asymptotically equicontinuous with En (y, 0) = 0. Hence, the second
term in (A.5) converges to 0 in probability, too. Likewise, the convergence of the last term
to 0 follows from the asymptotic equicontinuity of En , which concludes the proof.
2

A.5

Proof of Theorem 3.4 and of Remark 3.5

For any interval I ⊂ R and constant k > 0, define the following class of differentiable
functions:


n
|d0 (x) − d0 (y)| o
1+δ
0
Ck (I) = d : I → R max sup |d(x)|, sup |d (x)|, sup
≤k .
|x − y|δ
x∈I
x∈I
x,y∈I,x6=y
22

1+δ
Then Theorem 2.7 yields P ((g̃ − g) ∈ C1/2
(In )) → 1 as n → ∞. Hence there exist random functions dn : [0, 1] → R such that dn (x) = (g̃ − g)(x) for all x ∈ In and

P dn ∈ C11+δ ([0, 1]) → 1 for n → ∞. (For instance, one may extrapolate g̃ − g linearly on
[0, hn ] and on [1 − hn , 1].)
On the space F := R × C11+δ ([0, 1]) we define the semimetric
o
n
∗
∗
∗ ∗
|F (y + γ(x)) − F (y + γ(x))| , sup |d(x)−d (x)| .
ρ((y, d), (y , d )) = max sup
sup
x∈[0,1] γ∈C 1+δ ([0,1])

x∈[0,1]

1

For ϕ = (y, d) ∈ F let
√
n
1
Znj (ϕ) :=
I{εj ≤ y + d( nj )}I{ nj ∈ In } − √ I{εj ≤ y}
mn
n
and

n
X
Gn (ϕ) :=
(Znj (ϕ) − E[Znj (ϕ)]).
j=1

Note that
Gn (y, dn )
√ X
n
n
I{εj ≤ y − g(j/n) + g̃(j/n)}I{j/n ∈ In }
=
mn j=1
√ X
n
n
√
n
1 X
I{εj ≤ y} + nF (y)
−
F (y + (g̃ − g)(j/n))I{j/n ∈ In } − √
mn j=1
n j=1
=

n
n



√ 
1X
1 X
n F̃n (y) −
I{εj ≤ y} −
F y + (g̃ − g)( nj ) − F (y) I{ nj ∈ In } .
n j=1
mn j=1

We will apply Theorem 2.11.9 of van der Vaart and Wellner (1996) to show that the process
(Gn (ϕ))ϕ∈F converges to a (Gaussian) limiting process. In particular, Gn is asymptotically equicontinuous, which readily yields the assertion, because supy∈R ρ((y, dn ), (y, 0)) =
supx∈[0,1] |dn (x)| = oP (1) and the variance of
n

1 X n
Gn (y, 0) = √
− 1 I{εj ≤ y}I{j/n ∈ In } − I{εj ≤ y}I{j/n 6∈ In }
n j=1 mn

tends to 0, implying that Gn (y, 0) = oP (1) uniformly in y. Thus Gn (y, dn ) = oP (1) uniformly
in y and the assertion holds.
One may proceed as in the proof of Lemma 3 in Neumeyer and Van Keilegom (2009) (see
the online supporting information to that article) to prove that the conditions of Theorem
2.11.9 of van der Vaart and Wellner (1996) are fulfilled. The proof of the first two displayed
formulas of this theorem are analogous. The only difference is that Neumeyer and Van
23

Keilegom (2009) assume a bounded error density while we use Hölder continuity of F , see
assumption (F2). Next we show that the bracketing entropy condition (i.e., the last displayed
condition in Theorem 2.11.9 of van der Vaart and Wellner, 1996) is fulfilled and that (F, ρ)
is totally bounded.
To this end, let dLm ≤ dUm , m = 1, . . . , M , be brackets for C11+δ ([0, 1]) of length η 2/(α∧1)
w.r.t. the supremum norm. According to van der Vaart and Wellner (1996), Theorem 2.7.1

and Corollary 2.7.2, M = O exp(κη −2/((1+δ)(α∧1)) ) brackets are needed. For each m define
P
L
FmL (y) := n−1 nj=1 F (y + dLm (j/n)) and choose ym,k
, k = 1, . . . , K = O(η −2 ) such that
L
L
L
L
:= −∞ and ym,K+1
:= ∞.
) < η 2 for all k ∈ {1, . . . , K + 1} with ym,0
) − FmL (ym,k−1
FmL (ym,k
U
U
L
L
U
U
Define Fm and ym,k analogously, ỹm,k := ym,k and denote by ỹm,k the smallest ym,l larger
L
than or equal to ym,k+1
. Then F is covered by
L
U
Fmk := {(y, d) ∈ F | ỹm,k
≤ y ≤ ỹm,k
, dLm ≤ d ≤ dUm },

m = 1, . . . , M, k = 1, . . . , K.

Check that by condition (F2)
sup |FmU (y) − FmL (y)| ≤ sup n−1
y∈R

y∈R

n
X

|F (y + dUm (j/n)) − F (y + dLm (j/n))|

j=1

≤ LF sup |dUm (x) − dLm (x)|α∧1 ≤ LF η 2

(A.7)

x∈R

with LF denoting the Hölder constant of F . Thus
n
i2
1X h
E
sup
I{εj ≤ y + d(j/n)} − I{εj ≤ y ∗ + d∗ (j/n)}
n j=1
(y,d),(y ∗ ,d∗ )∈Fmk
n

2
1X 
U
L
E I{εj ≤ ỹm,k
≤
+ dUm (j/n)} − I{εj ≤ ỹm,k
+ dLm (j/n)}
n j=1
U
L
≤ FmU (ỹm,k
) − FmL (ỹm,k
)
U
L
L
L
L
L
≤ |FmU (ỹm,k
) − FmU (ỹm,k+1
)| + |FmU (ỹm,k+1
) − FmL (ỹm,k+1
)| + |FmL (ỹm,k+1
) − FmL (ỹm,k
)|

≤ (2 + LF )η 2
L
U
where the last step follows from (A.7) and the definitions of ỹm,k
and ỹm,k
. Hence we obtain
n
for the squared diameter of Fmk w.r.t. L2
n
h
X
E
j=1

sup

i2
|Znj (y, d) − Znj (y , d )|
∗

∗

(y,d),(y ∗ ,d∗ )∈Fmk

n
i2
n X h
∗
∗
E
sup
I{εj ≤ y + d(j/n)} − I{εj ≤ y + d (j/n)} I{j/n ∈ In }
≤ 2 2
mn j=1
(y,d),(y ∗ ,d∗ )∈Fmk
n
i2
2X h
+
E
sup
I{εj ≤ y} − I{εj ≤ y ∗ }
n j=1
(y,d),(y ∗ ,d∗ )∈Fmk

24

≤ 3(2 + LF )η 2
for sufficiently large n. This shows that the bracketing number satisfies log N[ ] (η, F, Ln2 ) =

O(log M + log K) = O η −2/((1+δ)(α∧1)) , and the last displayed condition of Theorem 2.11.9
of van der Vaart and Wellner (1996) follows from δ > 1/α − 1.
It remains to show that (F, ρ) is totally bounded, i.e. that, for all η ∈ (0, 1), the space
F can be covered by finitely many sets with ρ-diameter less than 5η. To this end, choose
dLm and dUm as above. For each m ∈ {1, . . . , M } and j ∈ {0, . . . , J := dη −1 e}, let sj :=
jη 1/(α∧1) ∧ 1 and Fjm (y) := P (ε1 ≤ y + dLm (sj )), and choose an increasing sequence yjm,k ,
k = 1, . . . , K := bη −1 c, such that Fjm (yjm,k ) − Fjm (yjm,k−1 ) < η for all k ∈ {1, . . . , K + 1}
with yjm,0 := −∞ and yjm,K+1 := ∞. Denote by ȳl , 1 ≤ l ≤ L, all points yjm,k , j ∈
{0, . . . , J}, m ∈ {1, . . . , M }, k ∈ {1, . . . , K}, in increasing order. We show that all sets
Fml := {(y, d) | ȳl−1 ≤ y ≤ ȳl , dLm ≤ d ≤ dUm } have ρ-diameter less than 5η. Check that, for
all 1 ≤ l ≤ L, one has
sup

|F (ȳl + γ(x)) − F (ȳl−1 + γ(x))|

sup

x∈[0,1] γ∈C 1+δ ([0,1])
1

≤ max

sup

max

h
|F (ȳl + γ(x)) − F (ȳl + γ(sj ))|

sup

1≤j≤J sj−1 ≤x≤sj 1≤m≤M dL ≤γ≤dU
m
m

+|F (ȳl + γ(sj )) − F (ȳl + dLm (sj ))| + |F (ȳl + dLm (sj )) − F (ȳl−1 + dLm (sj ))|
i
+|F (ȳl−1 + dLm (sj )) − F (ȳl−1 + γ(sj ))| + |F (ȳl−1 + γ(sj )) − F (ȳl−1 + γ(x))|
h
i
< max (sj − sj−1 )α∧1 + η 2 + η + η 2 + (sj − sj−1 )α∧1
1≤j≤J

≤ 5η.
Therefore, for all (y, d), (y ∗ , d∗ ) ∈ Fml
ρ((y, d), (y ∗ , d∗ ))
n
≤ max sup

sup

x∈[0,1] γ∈C 1+δ ([0,1])

|F (ȳl + γ(x)) − F (ȳl−1 + γ(x))|, sup dUm (x) − dLm (x)

o

x∈[0,1]

1

≤ max{5η, η 2/(α∧1) } = 5η,
which concludes the proof of Theorem 3.4.
If we drop the assumption δ > 1/α − 1 but require F to be Lipschitz continuous, then
we use brackets for C11+δ ([0, 1]) of length η 2 (instead of η 2/(α∧1) ) and replace (A.7) with
sup |FmU (y)
y∈R

−

FmL (y)|

−1

≤ sup n
y∈R

n
X

|F (y + dUm (j/n)) − F (y + dLm (j/n))|

j=1

≤ LF sup |dUm (x) − dLm (x)| ≤ LF η 2
x∈R


with LF denoting the Lipschitz constant of F to prove log N[ ] (η, F, Ln2 ) = O η −2/(1+δ) ,
which again yields the third condition of Theorem 2.11.9 of van der Vaart and Wellner
25

(1996). Likewise, in the last part of the proof, one defines sj := jη ∧ 1 and replaces (A.8)


with max1≤j≤J (sj − sj−1 ) + η 2 + η + η 2 + (sj − sj−1 ) ≤ 5η.
2
In the remaining proofs to Section 3, we use the index n for the estimators to emphasis
the dependence on the sample size and to distinguish between estimators and polynomials
corresponding to a given sample on the one hand and corresponding objects in a limiting
setting on the other hand.

A.6

Proof of Lemma 3.6

Proposition A.1 and the proof of Lemma A.2 show that there exist constants d, d˜ > 0
˜
depending only on β and cg such that E(ĝn (x)) ≤ dE(M
n,0 ) and P {Mn,0 > t} ≤ 1 +

dnhn
dnhn (1 − F (−t)) (F (−t))
for all t > 0.
1/α
Let an := a(log n/(nhn ))
for a suitable constant a > 0 and fix some t0 > 0 such that
α
(1 − F (−t))/(ct ) ∈ (1/2, 2) for all t ∈ (0, t0 ]. Then
E(Mn,0 )
Z ∞
P {Mn,0 > t} dt
=
0
Z t0
Z

dnhn
≤ an +
1 + dnhn (1 − F (−t)) (F (−t))
dt + (1 + dnhn )

∞

((F (−t))dnhn dt.

t0

an

Now, for sufficiently large n,
Z t0
an

≤
≤
≤
≤
=


1 + dnhn (1 − F (−t)) (F (−t))dnhn dt
Z t0

c dnhn
dt
1 + 2cdnhn tα 1 − tα
2
an
Z t0
 c

(1 + 2cd)nhn
tα exp − dnhn tα dt
2
an
Z tα0
 c

t0
exp − dnhn u du
(1 + 2cd)nhn
α aαn
2


2(1 + 2cd)nhn t0
c
exp − daα log n
αcdnhn
2
−ξ
o(n )

for all ξ > 0 if a is chosen sufficiently large. Hence the assertion follows from (H1) and (F3)
which imply
Z ∞
Z ∞
dnhn
dnhn
+
t−dτ nhn dt = o(n−ξ )
(F (−t))
dt ≤ nhn ((F (−t0 ))
nhn

t0

2

for all ξ > 0.
26

A.7

Proof of Theorem 3.7

As the density f is bounded and Lipschitz continuous, one has
Z (g̃n∗ −g)(j/n)
∗
∗
f (y + t) − f (y) dt
F y + (g̃n − g)(j/n)) − F (y) − f (y)(g̃n − g)(j/n) =
0

= O (g̃n∗ − g)2 (j/n)
uniformly for y ≤ y0 and j/n ∈ In . Hence the remainder term can be approximated by a
sum of estimation errors as follows:
n
n

1 X
f (y) X ∗
∗
F y + (g̃n − g)(j/n)) − F (y) I{j/n ∈ In } −
(g̃ − g)(j/n)I{j/n ∈ In }
mn j=1
mn j=1 n

n
 log n 2/α 

1 X ∗
2β
2
2β
= O
(g̃n − g) (j/n)I{j/n ∈ In } = OP hn + bn +
= oP (n−1/2 )
mn j=1
nhn

where for the last conclusions we have used Theorem 2.2, Lemma 3.6 and the assumptions
(H2) and (B3). Thus the assertion follows if we show that
n
1 X ∗
(g̃ − g)(j/n)I{j/n ∈ In } = oP (n−1/2 ).
mn j=1 n

To this end, note that g̃n∗ (x) and g̃n∗ (y) are independent for |x − y| > 2(hn + bn ). For
simplicity, we assume that 2n(hn + bn ) =: kn is a natural number. If we split the whole sum
into blocks with kn consecutive summands, then all blocks with odd numbers are independent
and all blocks with even numbers are independent. It suffices to show that
1
mn

bn/(2kn )c

X

∆n,2`−1 = oP (n−1/2 )

`=1

1
mn

bn/(2kn )c

X

∆n,2` = oP (n−1/2 )

`=1

P(l+2)kn −1 ∗
where ∆n,l = j=(l+1)k
(g̃n − g)(j/n), 1 ≤ ` ≤ bn/kn c. We only consider the second sum,
n
because the first convergence obviously follows by the same arguments.
It suffices to verify

(A.8)
E ∆2n,2` = o(kn )



E ∆n,2` = o n−1/2 kn = o n1/2 (hn + bn )
(A.9)
uniformly for all 1 ≤ ` ≤ bn/(2kn )c, since then
E

 bn/(2k
Xn )c
`=1

∆n,2`

2

bn/(2kn )c

=

X

V ar(∆n,2` ) +

`=1

 bn/(2k
Xn )c
`=1

27

E(∆n,2` )

2

= o(n),

which implies the assertion.
To prove (A.8), note that according to Lemma 3.6, Proposition A.1 and the proofs of
Lemma A.2 and of Theorem 2.7(i), there exist constants c1 , c2 , c3 > 0 (depending only on β,
cg and the kernel K) such that

 log n 1/α

+ max(M1∗ , M2∗ )
sup |g̃n∗ (x) − g(x)| ≤ c1 hβn + bβn +
nhn
x∈In
where M1∗ , M2∗ are independent random variables such that P {Mi∗ > t} ≤ 1 − (1 − P {Mn,0 >
t})c2 (hn +bn )/hn with

P {Mn,0 > t} ≤ 1 + c3 nhn (1 − F (−t)) (F (−t))c3 nhn .

1/2
Because kn hβn + bβn + (log n/(nhn ))1/α = o(kn ) by (H2) and (B3), it suffices to show that
Z ∞

∗ 2
E (Mi ) =
P {Mi∗ > t1/2 } dt = o(1/kn ).
(A.10)
0

Fix some t0 ∈ (0, (2c)−2/α ) such that (1 − F (−t))/(ctα ) ∈ (1/2, 2) for all t ∈ (0, t0 ]. In
what follows, d denotes a generic constant (depending only on β, cg , c and K) which may
vary from line to line. Applying the inequalities exp(−2ρu) ≤ (1 − u)ρ ≤ exp(−ρu), which
holds for all ρ > 0 and u ∈ (0, 1/2), we obtain for (nhn / log n)−2/α < t ≤ t0 and sufficiently
large n

c (hn +bn )/hn
P {Mi∗ > t1/2 } ≤ 1 − 1 − (1 + c3 nhn 2ctα/2 )(1 − ctα/2 /2)c3 nhn 2


≤ 1 − 1 − 3c3 cnhn tα/2 exp − c3 cnhn tα/2 /2


≤ 1 − exp − dn(hn + bn )tα/2 exp − c3 cnhn tα/2 /2

≤ dn(hn + bn )tα/2 exp − c3 cnhn tα/2 /2 .
Therefore, for sufficiently large a > 0,
Z

t20

P {Mi∗ > t1/2 } dt

0

≤ a

 nh −2/α
n

log n

Z

t0


tα/2−1 exp − c3 cnhn tα/2 /2 dt

+ dt0 n(hn + bn )
a(nhn / log n)−2/α


≤ o(1/(n(hn + bn ))) + dt0 n(hn + bn ) exp − c3 caα/2 log n/2
= o(1/(n(hn + bn )))

(A.11)

where in the last but one step we apply the conditions (B3) and (H2). Now, assertion
(A.10) (and hence (A.8)) follows from
Z ∞
Z ∞
h
ic2 (hn +bn )/hn
∗
1/2
1/2 c3 nhn
1 − 1 − c3 nhn (F (−t ))
dt
P {Mi > t } dt ≤
t20

t20

28

Z

∞



1/2

1 − exp − dn(hn + bn )(F (−t
Z

c3 nhn
≤ dn(hn + bn ) nhn (F (−t0 ))
+
≤

c3 nhn

))



dt

t20

∞


t−τ c3 nhn /2 dt

nhn
−ξ

= o(n )
for all ξ > 0 and sufficiently large n, where we have used (H2) and (F3).
To establish (A.9), first note that for a kernel K of order d + 1 with d := bβc
E(g̃n (x) − g(x)) = E

Z

1



ĝn (x + bn u) −

−1

Z

d
X
g (j) (x)
j=0

j!


(bn u)j K(u) du

1

=

E(ĝn (x + bn u) − g(x + bn u))K(u) du + O(bβn )

−1

uniformly for all x ∈ [hn + bn , 1 − hn − bn ]. In view of (K1), (H2) and (B3), it thus suffices
to show that
E(ĝn (x) − g(x)) − Eg≡0 (ĝn (1/2)) = E(ĝn (x) − g(x)) − Eg≡0 (ĝn (x)) = o(n−1/2 ) (A.12)
uniformly for Lebesgue almost all x ∈ [hn , 1 − hn ]. Note that the distribution of ĝn (x) does
not depend on x if g equals 0.
Recall that ĝn (x) = p̃n (0) where p̃n is a polynomial on [−1, 1] of degree d that solves the
linear optimization problem
Z 1
p̃n (t) dt → min!
−1

under the constraints
p̃n

 i/n − x 
hn

≥ Yi ,

∀ i ∈ [n(x − hn ), n(x + hn )].

Define polynomials
d
X
1 (k)
qx (t) :=
g (x)(hn t)k ,
k!
k=0

pn (t) := (nhn )1/α (p̃n (t) − qx (t)),

t ∈ [−1, 1].

Then qx ((u − x)/hn ) is the Taylor expansion of order d of g(u) at x and the estimation error
can be written as
ĝn (x) − g(x) = (nhn )−1/α pn (0).
(A.13)
Note that pn is a polynomial of degree d that solves the linear optimization problem
Z

1

pn (t) dt → min!
−1

29

subject to
pn

 i/n − x 
hn

≥ (nhn )1/α ε̄i ,

∀ i ∈ [n(x − hn ), n(x + hn )],

(A.14)

with
ε̄i := εi + g(i/n) − qx

 i/n − x 
hn

.

We now use point process techniques to analyze the asymptotic behavior of this linear
program.
Denote by
X
Nn :=
δ
((i/n − x)/hn , (nhn )1/α ε̄i )
i∈[n(x−hn ),n(x+hn )]

a point process of standardized error random variables. Then the constraints (A.14) can be
reformulated as Nn (Apn ) = 0 where Af := {(t, u) ∈ [−1, 1] × R | u > f (t)} denotes the open
epigraph of a function f .
Since by (H2) |ε̄i − εi | = g(in ) − qx ((i/n − x)/hn )) = O(hβn ) = o((nhn )−1/α ) uniformly
for all i ∈ [n(x − hn ), n(x + hn )], one has

E Nn ([−1, 1] × (−1, ∞))) ∼ 2nhn P ε̄1 > −(nhn )−1/α → 2c.
Therefore, Nn converges weakly to a Poisson process N on [−1, 1]×R with intensity measure
2cU[−1,1] ⊗να where να has Lebesgue density x 7→ α|x|α−1 I(−∞, 0) (see, e.g., Resnick (2007),
Theorem 6.3). By Skorohod’s representation theorem, we may assume that the convergence
holds a.s.
Next we analyze the corresponding linear program in the limiting model to minimize
R1
p(t) dt over polynomials of degree d subject to N (Ap ) = 0. In what follows we use a
−1
P
representation of the Poisson process as N = ∞
i=1 δ(Ti ,Zi ) where Ti are independent random
variables which are uniformly distributed on [−1, 1].
First we prove by contradiction that the optimal solution is almost surely unique. Suppose
that there exist more than one solution. From the theory of linear programs it is known
that then there exists a solution p such that J := {j ∈ N | p(Tj ) = Zj } has at most d
elements. Because p is bounded and N has a.s. finitely many points in any bounded set,
η := inf{|p(Ti ) − Zi | | i ∈ N \ J} > 0 a.s. Since p is an optimal solution, all polynomials ∆
R1
of degree d such that ∆(Tj ) = 0, j ∈ J, and k∆k∞ < η must satisfy −1 ∆(t) dt = 0, because
both p + ∆ and p − ∆ satisfy the constraints N (Ap±∆ ) = 0. In particular, for all polynomials
Q
q of degree d − |J|, ∆(t) = τ i∈J (t − Ti )q(t) is of that type if τ > 0 is sufficiently small.
Q
P|J|−1
Write i∈J (t − Ti ) in the form t|J| + l=0 al tl . Then necessarily
Z

1

Y
(t − Ti )tj dt =

−1 i∈J

|J|−1
X
2
2al
I{|J| + j even} +
I{l + j even} = 0,
|J| + j + 1
l
+
j
+
1
l=0

30

for all j ∈ {0, . . . d − |J|}. This implies that (Ti )i∈J lies on a manifold M|J|,d of dimension
|J| − (d − |J| + 1) = 2|J| − d − 1 which only depends on |J| and d. However, by Proposition
A.1, kpk∞ ≤ Kd Zmax where
Zmax := max min{|Zi | | Ti ∈ [−1 + (j − 1)/jd , −1 + j/jd ]}.
1≤i≤jd

The above conclusion contradicts P {Zmax > K} → 0 as K → ∞, since

P ∃J ⊂ N : |J| ≤ d, (Tj )j∈J ∈ M|J|,d , max |Zj | ≤ Kd K = 0
j∈J

for all K > 0 (i.e., the fact that among finitely many values Ti a.s. there does not exist a
subset which lies on a given manifold of lower dimension).
Therefore the solution p must be a.s. unique which in turn implies that it is a basic
feasible solution, i.e., |J| ≥ d + 1. On the other hand, because the intensity measure of N is
absolutely continuous, |J| ≤ d + 1 a.s. and thus |J| = d + 1. Because of Nn → N a.s., one
has Nn ([−1, 1] × [−Kd Zmax , ∞)) = N ([−1, 1] × [−Kd Zmax , ∞)) =: M for sufficiently large n.
Moreover, one can find a numeration of the points (Tn,i , Zn,i ), 1 ≤ i ≤ M , of Nn and (Ti , Zi ),
1 ≤ i ≤ M , of N in [−1, 1] × [−Kd Zmax , ∞) such that (Tn,i , Zn,i ) → (Ti , Zi ).
R1
Next we prove that the solution to the linear program to minimize −1 pn (t) dt subject
to Nn (Apn ) = 0 is eventually unique with pn → p a.s. Since any optimal solution can be
written as a convex combination of basic feasible solutions, w.l.o.g. we may assume that
Jn := {1 ≤ i ≤ M | pn (Tn,i ) = Zn,i } has at least d + 1 elements. The polynomial pn is
uniquely determined by this set Jn . Suppose that along a subsequence n0 the set Jn0 is
constant, but not equal to J. Then p0n converges uniformly to the polynomial p̄ of degree
d that is uniquely determined by the conditions p̄(Ti ) = Zi for all i ∈ Jn0 . In particular,
p̄ is different from the unique optimal polynomial p for the limit Poisson process, but it
R1
R1
satisfies the constraints N (Ap ) = 0. Thus −1 p̄(t) dt > −1 p(t) dt. On the other hand, for
all η > 0 the polynomial p + η eventually satisfies the constraints Nn (Ap+η ) = 0 and thus
R1
R1
p(t)
+
η
dt
≥
p̄ (t) dt, which leads to a contradiction.
−1
−1 n
Hence, Jn = J for all sufficiently large n and the optimal solution pn for Nn is unique and
it converges uniformly to the optimal solution p for the Poisson process N . Moreover, using
the relation (pn (Tn,j ))j∈J = (Zn,j )j∈J (which is a system of linear equation in the coefficients
of pn ), pn (0) can be calculated as wnt (Zn,j )j∈J for some vector wn which converges to a limit
vector w (corresponding to the analogous relation for p).
Exactly the same arguments apply if we replace ε̄i with εi , which corresponds to the
case that g is identical 0. Since the points (T̃n,i , Z̃n,i ) of the pertaining point process equal

Tn,i , Zn,i − (nhn )1/α (g(i/n) − qx ((i/n) − x)/hn ) and thus |Z̃n,i − Zn,i | ≤ cg (nhn )1/α hβn , the
difference of the resulting values for optimal polynomial at 0 is bounded by a multiple of
(nhn )1/α hβn . In view of (A.13) and (H2), we may conclude that the difference between the
estimation errors can be bounded by a multiple of hβn = o(n−1/2 ), which finally yields (A.12)
and thus the assertion.
2
31

A.8

Proof of Corollary 4.3

bnsc
√ (Fbnsc (y)−Fn (y)) = En (y, s)−
En (y, 1) with En defined in (A.4). A similar
Note that bnsc
n
n
reasoning as in the proof of Theorem 3.1 (see (A.6)) shows that
 ns̄
 bnsc
n
sup
− 1 √ (Fbnsc (y) − Fn (y)) = oP (1).
bnsc
n
y∈R,s∈[0,1]

Hence, by Theorem 3.1, uniformly for all y ∈ R, s ∈ [0, 1],
√
n(F̂n (y, s) − s̄n F̂n (y))
√
√
√
= n(F̂n (y, s) − s̄n Fbnsc (y)) − s̄n n((F̂n (y) − Fn (y))) + s̄n n(Fbnsc (y) − Fn (y))
=
=

ns̄n bnsc
√ (Fbnsc (y) − Fn (y)) + oP (1)
bnsc n
En (y, 1) + oP (1)
En (y, s) − bnsc
n

= En (y, s) − sEn (y, 1) + oP (1)
which converges weakly to KF (y, s)−sKF (y, 1) for the Kiefer process KF defined in Theorem
3.1. Check that this Gaussian process has the same law as G(s, F (y)), because they have
the same covariance function. Thus the Kolmogorov-Smirnov statistic Tn converges weakly
to sups∈[0,1],y∈R |G(s, F (y)| = sups∈[0,1],z∈[0,1] |G(s, z)|, where the last equality holds by the
continuity of F .
2

A.9

Proof of Theorem 4.4

Note that under the given assumptions, the statements of Theorem 2.7 (i) and (ii) are valid
with rate oP (1). Let Ωn := {inf x∈In g̃ 0 (x) > 0}. From assumption (I1) and Theorem 2.7 (ii)
it follows that P (Ωn ) → 1 for n → ∞. But on Ωn the estimators g̃I and g̃ are identical,
and thus F̃I,bnsc = F̃bnsc . Now (4.1) can be concluded as in the proof of Theorem 3.1,
because Theorem 2.7 (i) yields supx∈In |g̃(x) − g(x)| = oP (n−1/(2(α∧1)) ). The convergence of
the Kolmogorov-Smirnov test statistic then follows exactly as in the proof of Corollary 4.3.
2

Acknowledgement
Financial support by the DFG (Research Unit FOR 1735 Structural Inference in Statistics:
Adaptation and Effciency) is gratefully acknowledged.

References
Akritas, M. and Van Keilegom, I. (2001). Nonparametric estimation of the residual distribution. Scand. J. Statist. 28, 549–567.
32

Anevski, D. and Fougères, A.-L. (2007). Limit properties of the monotone rearrangement
for density and regression function estimation. arXiv:0710.4617v1
Birke, M. and Neumeyer, N. (2013). Testing Monotonicity of Regression Functions - An
Empirical Process Approach. Scand. J. Statist. 40, 438–454.
Birke, M., Neumeyer, N. and Volgushev, S. (2016+). The independence process in conditional quantile location-scale models and an application to testing for monotonicity.
Statistica Sinica, to appear.
Chernozhukov, V., Fernández-Val, I. and Galichon, A. (2009). Improving point and interval
estimators of monotone functions by rearrangement. Biometrika 96, 559–575.
Daouia, A., Noh, H. and Park, B. U. (2016). Data envelope fitting with constrained polynomial splines. J. R. Stat. Soc. B. 78, 3–30.
Einmahl, J. H. J. and Van Keilegom, I. (2008). Specification tests in nonparametric regression. Journal of Econometrics 143, 88–102.
Färe, R. and Grosskopf, S. (1983). Measuring output efficiency. European Journal of
Operational Research 13, 173–179.
Gijbels, I. (2005). Monotone regression. In: N. Balakrishnan, S. Kotz, C.B. Read and B.
Vadakovic (eds), The Encyclopedia of Statistical Sciences, 2nd edition. Hoboken, NJ:
Wiley.
Gijbels, I., Mammen, E., Park, B. and Simar, L. (2000). On estimation of monotone and
concave frontier functions. J. Amer. Statist. Assoc. 94, 220–228.
Gijbels, I. and Peng, L. (2000). Estimation of a support curve via order statistics. Extremes
3, 251–277.
Girard, S. and Jacob, P. (2008). Frontier estimation via kernel regression on high powertransformed data. J. Multivariate Anal. 99, 403–420.
Girard, S., Guillou, A. and Stupfler, G. (2013). Frontier estimation with kernel regression
on high order moments. J. Multivariate Anal. 116, 172–189.
Hall, P., Park, B.U. and Stern, S.E. (1998). On polynomial estimators of frontiers and
boundaries. J. Multivariate Anal. 66, 71–98.
Hall, P. and Van Keilegom, I. (2009). Nonparametric “regression” when errors are positioned at end-points. Bernoulli 15, 614–633.
33

Härdle, W., Park, B.U. and Tsybakov, A.B. (1995). Estimation of non-sharp support
boundaries. J. Multivariate Anal. 55, 205–218.
Jirak, M., Meister, A. and Reiß, M. (2014). Adaptive estimation in nonparametric regression with one-sided errors. Ann. Statist. 42, 1970–2002.
Meister, A. and Reiß, M. (2013). Asymptotic equivalence for nonparametric regression with
non-regular errors. Probab. Th. Rel. Fields 155, 201–229.
Müller, U.U. and Wefelmeyer W. (2010). Estimation in Nonparametric Regression with
Non-Regular Errors. Comm. Statist. Theory Methods 39, 1619–1629.
Neumeyer, N. and Van Keilegom, I. (2009). Change-Point Tests for the Error Distribution
in Nonparametric Regression. Scand. J. Statist. 36, 518–541.
online supporting information available at
http://onlinelibrary.wiley.com/doi/10.1111/j.1467-9469.2009.00639.x/suppinfo
Picard, D. (1985). Testing and estimating change-points in time series. Adv. Appl. Probab.
17, 841–867.
Reiß, M. and Selk, L. (2016+). Efficient nonparametric functional estimation for one-sided
regression. Bernoulli, to appear.
Resnick, S.I. (2007). Heavy-Tail Phenomena. Springer.
Simar, L. and Wilson, P.W. (1998). Sensitivity analysis of efficiency scores: how to bootstrap in nonparametric frontier models. Management Science 44, 49–61.
Stephens, M.A. (1976). Asymptotic Results for Goodness-of-Fit Statistics with Unknown
Parameters. Ann. Statist. 4, 357–369.
van der Vaart, A.W. (2000). Asymptotic Statistics. Cambridge University Press.
van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. Springer, New York.
Wilson, P.W. (2003). Testing independence in models of productive efficiency. Journal of
Productivity Analysis 20, 361–390.

34

