Uniformly Valid Inference
in Nonparametric Predictive Regression

arXiv:1509.05017v3 [] 29 Dec 2017

James A. Duffy∗
December 2017

Abstract
A significant problem in predictive regression concerns the invalidity of conventional
OLS-based tests, when the regressor is highly persistent. Recent work has suggested that, in contrast, nonparametric regression-based inferences are free of this problem. However, existing results are insufficient to support the conclusion that standard nonparametric testing procedures have the correct asymptotic size, in the sense
of controlling null rejection probabilities uniformly in the parameters describing the
persistence of the regressor. We provide a proof of precisely such a result, thereby establishing the posited validity of these methods. In the course of doing so, we develop
new results concerning the asymptotics of kernel density estimators, when these are
applied to autoregressive processes exhibiting moderate deviations from a unit root.
This leads to a unified asymptotic theory for these estimators, encompassing a class of
processes that includes both stationary and integrated processes, and arrays formed
from such processes.

∗
First version: September 2015. Corpus Christi College and Economics Department, University of
Oxford; email: james.duffy@economics.ox.ac.uk. I thank V. Berenguer-Rico, S. Mavroeidis, B. Nielsen,
and participants at seminars at Cambridge, Princeton, Vienna and UCL for comments on an earlier version
of this paper. The manuscript was prepared with LYX 2.2.3 and JabRef 2.7b.

i

Contents
1 Introduction

1

2 Nonparametric predictive regression

3

2.1

Data generating process . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3

2.2

Nonparametric estimation and inference . . . . . . . . . . . . . . . . . . . .

4

2.3

Uniform validity of (pointwise) inferences . . . . . . . . . . . . . . . . . . .

5

3 Density estimation: a unified limit theory

8

3.1

Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

8

3.2

Finite-dimensional convergence . . . . . . . . . . . . . . . . . . . . . . . . . 10

3.3

Weak convergence of the density estimator process . . . . . . . . . . . . . . 13

4 Conclusion

13

5 References

13

A Proof of uniform validity of inferences

15

B Proof of finite-dimensional convergence

20

B.1 Proof of Proposition 3.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
B.2 Proof of Theorem 3.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
C Proofs of auxiliary results for mild integration

22

C.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
C.2 Proof of Proposition B.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
C.3 Proof of Proposition B.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
D Proof of weak convergence (Theorem 3.2)

28

S Supplementary material [for online publication]

S1

S.1 Verification of Remark 3.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . S1
S.2 Proofs of auxiliary lemmas from Appendix A . . . . . . . . . . . . . . . . . S2
S.3 Proofs of auxiliary lemmas from Appendix C . . . . . . . . . . . . . . . . . S5
S.4 Proofs of auxiliary lemmas from Appendix D . . . . . . . . . . . . . . . . . S9
S.5 Proofs of Lemmas S.1–S.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . S11
S.6 Index of notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . S17

ii

j. a. duffy

1

Introduction

Inference in a predictive regression – as distinct from cross-sectional regression – faces two
distinctive challenges. The first arises when the regressor is strongly serially dependent. As
is now well known, in this case the limiting distribution of the OLS t statistic is non-pivotal,
being not only non-Gaussian but also depending on the unknown degree of persistence
of the regressor. This renders conventional inferential procedures invalid. The second
difficulty concerns the possibility of a relationship between series with strikingly different
dependence properties. For example, in the context of testing for stock return predictability
in finance, it is common to confront a series exhibiting martingale-difference-like behaviour,
such as excess returns, with a candidate predictor that appears to be integrated (or nearly
so), such as the dividend–price ratio. But parametric linear models, though widely used to
test for such predictability, imply that both the regressor and the dependent variable should
manifest similar degrees of persistence – unless, perhaps, they are very weakly related.
The first of these problems has been the subject of a substantial literature, which has
sought to either: develop procedures capable of handling the non-standard limiting distribution of the OLS estimator; or to propose novel estimators that remain asymptotically
normal, regardless of the persistence of the regressor. (See, amongst others: Cavanagh,
Elliott, and Stock, 1995; Campbell and Yogo, 2006; Jansson and Moreira, 2006; Magdalinos and Phillips, 2009; Phillips and Lee, 2013; and Elliott, Müller, and Watson, 2015.)
However, since this work has all been carried out in a parametric linear regression setting, it does little to address the second of the two problems noted above. The successful
resolution of that problem may plausibly lie with nonlinear regression models, since the
application of nonlinear transformations to dependent processes has been shown to produce new series with radically different memory properties (Marmer, 2008). The absence
of any theoretical priors as to the functional form of these possible nonlinearities leads us
naturally to consider nonparametric methods.
Some significant steps in this direction were taken in a recent paper by Kasparis, Andreou, and Phillips (2015, hereafter KAP), who studied the behaviour of kernel regression
estimators – and associated t-statistic-based tests of non-predictability – within a certain
class of strongly dependent regressor processes. Building on earlier work on local time
density estimation by Wang and Phillips (2009a,b), the authors showed that, despite the
assumed strong dependence of the regressor, nonparametric t statistics have standard normal limits, exactly as they do when the regressors are weakly dependent. Their result
holds out the prospect that nonparametric methods may be able to simultaneously resolve
both of the problems identified above: for not only do they allow us to estimate models
relating series with differing degrees of persistence, but they also yield estimates whose
limiting distributions are apparently unaffected by the persistence of the regressor.
One would thus like to be able to conclude that standard nonparametric tests retain
their validity, in a predictive regression, regardless of the extent of the serial correlation
affecting the regressor. Formally, what needs to be shown is that the asymptotic null rejec-

1

uniformly valid inference in nonparmetric regression

tion probabilities of these tests can be controlled uniformly in the parameters describing
the persistence of the regressor – which in this paper will be summarised by an autoregressive coefficient ρ. But while KAP’s results – together with existing results for stationary
(weakly dependent) regressors – are highly suggestive that such control is possible, they
are insufficient to sustain any such claim. What we crucially require – and what is missing
from the existing literature – are results concerning the asymptotics of kernel regression
estimators when regressors are stationary but exhibit ‘moderate deviations from a unit
root’ (Phillips and Magdalinos, 2007); we shall term such processes mildly integrated.
The uniformity sought in the present paper requires that we consider triangular arrays
of regressor processes, which will be parametrised in terms of ρ = ρn . Stationary processes
are identified as those for which ρn → ρ < 1, whereas local-to-unity processes (the class
considered by KAP) have ρn = 1 + O(n−1 ). Mildly integrated processes lie on the bridge
between these two classes, with ρn → 1 but n(1 − ρn ) → ∞. They therefore inherit some of
the properties of both stationary and local-to-unity processes, but are distinct from both,
and their treatment requires the development of some genuinely novel limit theory.
The first contribution of the paper is to show that nonparametric t statistics remain
asymptotically normal when regressors are mildly integrated. This result – in conjunction
with previous work – is sufficient to permit the conclusion that t-statistic-based tests and
confidence intervals have the correct asymptotic size, in the sense that the relevant null
rejection (or coverage) probabilities are controlled uniformly in the degree of persistence
of the regressor (Section 2). In view of this, nonparametric inference may be conducted
entirely without concern for the (unknown) degree of serial dependence in the regressor.
Underpinning this finding are some new results concerning the asymptotics of kernel
density estimators under mild integration, which constitute the other major contribution
of this paper (Section 3), and fill a significant gap left by previous work on the behaviour of
kernel density estimators when applied to either stationary and integrated processes (see,
e.g., Wu and Mielniczuk, 2002; Wang and Phillips, 2009a,b). The proofs of these results
rely on a combination of arguments appropriate to the stationary and local-to-unity cases.
The dependence of mildly integrated processes is sufficiently weak, such that kernel density
estimators converge not to the local time of some limiting process, but to a (non-random)
standard normal probability density. In this respect, mildly integrated processes are more
akin to stationary processes, except for the noted Gaussianity of the limiting density. On
the other hand, they also share the diminished recurrence and slower rates of convergence
characteristic of local-to-unity processes.
In combination with previous work, the results of this paper thus yield a unified theory for the behaviour of kernel density estimators under all possible values – and drifting
sequences – of the autoregressive parameter ρ. The theoretical results in the paper will undoubtedly prove useful for the analysis of other inferential problems, beyond the predictive
regression setting studied here.
Proofs of the main results appear in Appendices A–D. Proofs of technical results that
are either conceptually straightforward, or closely related to those that have already ap2

j. a. duffy

peared in the literature, are given in the Supplement, which also provides a complete index
of notation.
Notation. All limits are taken as n → ∞ unless otherwise stated. For sequences {an }, {bn }:
an ≍ bn denotes limn→∞ an /bn = c ∈ R\{0}, and an ∼ bn denotes limn→∞ an /bn = 1. For
positive sequences: an . bn denotes lim supn→∞ an /bn < ∞ – equivalently, an = O(bn ).
For random sequences {xn }, {yn }: xn .p yn denotes xn = Op (yn ). BL denotes the class of
bounded and Lipschitz functions on R, and Lp the class of Lebesgue p-integrable functions
on R.
and

denotes weak convergence in the sense of van der Vaart and Wellner (1996),
fdd

the convergence of finite-dimensional distributions. For x ≥ 0, ⌊x⌋ denotes the

greatest integer less than or equal to x.

2

Nonparametric predictive regression

2.1

Data generating process

Our setting is the nonlinear predictive regression model studied by KAP. The data generating process (DGP) is
(2.1)

yt = m(xt−1 ) + ut
where m and (xt , ut ) are as per
Assumption

DGP.

DGP1

m ∈ M := {m0 : R → R | supx∈R |m′0 (x)| ≤ M } for some M < ∞.

DGP2

iλε0
{εt }∞
t=−∞ is a scalar i.i.d. sequence; ε0 has characteristic function ψε (λ) := Ee

satisfying ψε ∈ L1 and a Lebesgue density fε that is Lipschitz continuous and
everywhere nonzero; Eε0 = 0 and Eε20 = 1.
DGP3

∞
{xt }∞
t=0 and {vt }t=1 are generated according to

xt := ρxt−1 + vt

vt :=

∞
X

φk εt−k ,

(2.2)

P∞

< ∞; and

k=0

DGP4

with x0 = 0; ρ ∈ R := [−1 + δ, 1] for some δ > 0; φ0 6= 0;
P
φ := ∞
k=0 φk 6= 0.

k=0 |φk |

{ut }∞
t=1 is a martingale difference sequence with respect to Gt := σ({xs , us }s≤t ),
with E[u2t | Gt−1 ] = σu2 a.s. and supt E[u4t | Gt−1 ] < ∞ a.s.

Remark 2.1. The assumption that fε is Lipschitz is used only in the stationary region,
i.e. when ρ < 1. While this requirement could likely be dispensed with, we have retained
it here so as to facilitate the direct application of results from Wu, Huang, and Huang
(2010). Lipschitzness of fε is implied, for example, if λψε (λ) ∈ L1 . Strict positivity of
fε is also assumed merely for convenience, to ensure that the stationary solution to (2.2)
3

uniformly valid inference in nonparmetric regression

has a density that is strictly positive at every x ∈ R, thereby avoiding the possibility of
(inadvertently) attempting to estimate m(x) at points of zero density. (Aside from ensuring
such points are avoided, this assumption is not needed for Proposition 2.1 below.)
Remark 2.2.

DGP3

is cognate with Assumptions 2.3 and 2.4 in KAP, with the key difference

that we do not restrict {xt } to the local-to-unity region, in which ρ = 1 + nc for some fixed
c ∈ R. We instead allow ρ to range over the entirety of R = [−1 + δ, 1]. Our main results
also easily extend to sequences of parameter spaces of the form R = Rn := [−1 + δ, 1 + nc ],
for some fixed c ∈ (0, ∞).
Owing to the initialisation x0 = 0, the regressor process is nonstationary, regardless of
the value of ρ. However, when ρ < 1 (2.1) admits a stationary solution, which corresponds
to the weak limit of xn as n → ∞. The assumption of a fixed initialisation is made only
for convenience; our results below would still hold provided x0 is stochastically bounded
(and adapted to G0 ).
P∞
k=0 |φk | < ∞ implies that {vt } is a short-memory process, and so excludes the long-

memory and anti-persistent cases that are also considered in KAP. It is likely that our
results could also be extended to cover these, but we have refrained from considering these
here in order to keep this paper to a manageable length.
Remark 2.3.

DGP4

implies that the regressor xt−1 is exogenous, so that m is identified from

m(x) = E[yt | xt−1 = x]. If the model (2.1) were reformulated with xt in place of xt−1 ,
then estimation of m would remain possible when ρ = 1 (and, indeed, if ρ = ρn → 1),
despite the potential endogeneity of the regressor (see Wang and Phillips, 2009b). On the
other hand, if ρ < 1 any putative estimate of m would suffer from the usual endogeneity
biases (even asymptotically).

2.2

Nonparametric estimation and inference

An estimate of m in (2.1), for each x ∈ R, is provided by the local level (Nadaraya-Watson)
regression estimator,

Pn
t=1 Kh (xt − x)yt+1
m̂n (x; h) := P
,
n
t=1 Kh (xt − x)

(2.3)

where K : R → R is a smooth probability density, h > 0 denotes the bandwidth, and
Kh (x) := h−1 K(h−1 u). We shall suppose h = hn , for {hn } a shrinking bandwidth sequence
as per
Assumption

SM

(smoothing).

SM1

K ∈ BL is positive and compactly supported, with

SM2

hn > 0 for all n, hn = o(1) and n1/2 hn → ∞.

R

RK

= 1.

Remark 2.4. The persistence of xt , as encapsulated in ρ, is intimately connected with the
recurrence of xt , which we may quantify in terms of the rate at which the local signal
P
Sn := nt=1 Kh (xt − x) diverges, for each fixed x ∈ R. As is well known, when h is fixed, in
4

j. a. duffy

the stationary region (ρn → ρ < 1) Sn grows at rate n (probabilistically); whereas when in
the local-to-unity region (ρn = 1 + O(n−1 )), this rate is reduced to n1/2 . Mildly integrated
processes are strictly intermediate between these cases, corresponding to a growth rate of
n(1 − ρ2n )1/2 for Sn .
Insofar as ρ is unknown, the maximum rate at which hn may shrink to zero, while still
permitting the growth of Sn – and hence, the consistency of m̂n – will thus be determined
by the region in which that growth is slowest, i.e. the local-to-unity region. This accounts
for the requirement that n1/2 hn → ∞ in

SM2.

This could be relaxed if hn were chosen so

as to adapt to the (unknown) recurrence of {xt }, but a consideration of such procedures
is beyond the scope of this paper.
For a fixed point x ∈ R in the domain of the regressor, a test of
H0 : m(x) = θ

H1 : m(x) 6= θ

against

(2.4)

may be based on the nonparametric t-statistic
t̂n (x; θ, h) = sn (x; h)−1 [m̂n (x; h) − θ],

(2.5)

where
s2n (x; h)

R
σ̂u2 (x) R K 2
:= Pn
h t=1 Kh (xt − x)

σ̂u2 (x)

:=

Pn

2
n (xt − x)[yt+1 − m̂n (x)]
t=1 Kh
P
. (2.6)
n
t=1 Khn (xt − x)

Critical values for the test are provided by the quantiles of a standard normal distribution

(as will be justified by Proposition 2.1 below). Test inversion leads to the familiar equaltailed confidence interval for m(x),
Cn (x; h) := {θ ∈ R | |t̂n (x; θ, h)| ≤ z1−α/2 }

(2.7)

= [m̂n (x; h) − z1−α/2 sn (x; h), m̂n (x; h) + z1−α/2 sn (x; h)],
where zτ denotes the τ th quantile of the standard normal distribution. Cn (x; h) is a
‘pointwise’ confidence interval, in the sense that it concerns the value of m at a single fixed
x ∈ R, rather than over a continuum of such points.

2.3

Uniform validity of (pointwise) inferences

The DGP is completely described by (m, ρ, γ), where γ := (ψε , {φk }, σu2 , {Fut }) and Fut
denotes the conditional distribution ut | Ft−1 ; let Γ denote the set of possible values for
γ. In this paper, the regression function m ∈ M is the parameter of interest, whereas
(ρ, γ) ∈ R × Γ are merely nuisance parameters.
In the context of testing the hypothesis in (2.4) above, the subset of the parameter
space consistent with H0 is given by
H := {m ∈ M | m(x) = θ} × R × Γ,
5

uniformly valid inference in nonparmetric regression

whence the size of a test of H0 depends on its maximum rejection probability over all
points in H . In keeping with the literature on the parametric (linear) predictive regression
problem, in which ρ ∈ R is a particularly troublesome nuisance parameter – owing to the
discontinuity in the limiting distribution of the OLS estimator at ρ = 1 – we shall only
seek to control the rejection probability of tests of H0 on the smaller set
H ∗ := {m ∈ M | m(x) = θ} × R × {γ}.
(In other words, our asymptotics will hold γ fixed as n → ∞.)
Our focus on H ∗ , rather than H , may be justified by the complications posed, even
in the present setting, by controlling the (asymptotic) rejection probability of a test of
H0 , uniformly in the persistence parameter ρ ∈ R. The proof that standard nonparametric
testing procedures indeed achieve such size control requires some genuinely new limit theory
for density developments, as is developed in Section 3 below. On the other hand, the
passage from H

∗

to H would merely call for relatively straightforward array extensions

of existing results, along with those given in this paper.
It is known from previous work – e.g. from Lemma 2 in KAP – that
t̂n (x; hn )

(2.8)

N [0, 1]

for every fixed (m, ρ) ∈ M × R. However, while this result is highly suggestive, it is
insufficient to show that a test based on the t statistic (with normal critical values) has
asymptotic size α, in the sense that
lim sup

Pm,ρ {|t̂n (x; hn )| ≥ z1−α/2 } = α,

sup

n→∞ (m,ρ)∈H

(2.9)

∗

where Pm,ρ is indexed by the values of m and ρ generating the data.
The proof of (2.9) requires that (2.8) hold not merely for fixed (m, ρ), but uniformly
in these parameters – which is equivalent to (2.8) holding along all drifting sequences
{(mn , ρn )} ⊂ M × R. once this uniformity has been established, it follows immediately
that the t-test is also asymptotically similar, in the sense that
lim inf

inf

n→∞ (m,ρ)∈H

∗

Pm,ρ {|t̂n (x; hn )| ≥ z1−α/2 } = α

holds in addition to (2.9). It also follows that the confidence set Cn (x; hn ) is asymptotically
similar, i.e.
lim inf

inf

n→∞ (m,ρ)∈M ×R

CPn (x; m, ρ) = lim sup

sup

n→∞ (m,ρ)∈M ×R

CPn (x; m, ρ) = 1 − α

where CPn (x; m, ρ) := Pm,ρ {m(x) ∈ Cn (x; hn )}, denotes the (finite-sample) coverage probability of Cn (x; hn ).
Our main result on the uniform validity of (pointwise) tests and confidence sets may
6

j. a. duffy

now be stated. Let X = {x1 , . . . xk } denote a fixed, finite subset of R; for a map a : R → R,
let [a(x)]x∈X denote the vector (a(x1 ), . . . , a(xk ))′ .
Proposition 2.1. Suppose

DGP

and

SM

hold, and that additionally hn = o(n−1/3 ). Then

for every finite X ⊂ R,
[t̂n (x; mn (x), hn )]x∈X

(2.10)

N [0, I#X ]

along every {mn } ⊂ M and {ρn } ⊂ R. Consequently, for each x ∈ R, the nonparametric
t test of (2.4) and the associated confidence interval Cn (x; hn ) are asymptotically similar.
Remark 2.5. Establishing the required uniformity of (2.8) with respect to m ∈ M poses
no particular difficulty: due to the linearity of the local level estimator, m affects only the
bias of m̂n , and the uniform negligibility of this term follows from standard arguments.
On the other hand, handling the nuisance parameter ρ requires more care. Essentially, the
problem reduces to one of proving that
1/2 Pn
hn
Khn (xt − x)ut+1
υn (x) := P t=1

R
n
2 1/2
σu
t=1 Khn (xt − x) K

N [0, 1]

(2.11)

along a sufficiently large class of drifting sequences {ρn } ⊂ R. By adapting an argument
from Andrews and Cheng (2012), it is shown that it is sufficient to prove that (2.11) holds
for the following classes of sequences {ρn } with limits in R:
• stationary (with parameter ρ): {ρn } ∈ RρST if ρn → ρ ∈ [−1 + δ, 1);
• mildly integrated: {ρn } ∈ RMI if ρn → 1 but n(ρn − 1) → −∞; and
• local to unity (with parameter c): {ρn } ∈ RcLU if ρn → 1 and n(ρn − 1) → c ∈ R.
We further restrict {ρn } ∈ RρST so that ρn ∈ [−1 + δ, 1) for all n, and {ρn } ∈ RMI so
that ρn ∈ (0, 1) for all n: this slightly simplifies some of the subsequent arguments. Let
S
S
RST := ρ∈[−1+δ,1) RρST and RLU := c∈R RcLU ; it will be useful to group together the
three classes of sequences considered above as

R := RST ∪ RMI ∪ RLU .

(2.12)

In all cases, the numerator of (2.11) is a martingale, and so is amenable to the application of existing martingale central limit theory. The principal difficulty is thus to
P
show that the conditional variance σu2 nt=1 Kh2n (xt − x), upon standardisation, converges

weakly to an a.s. nonzero limit. Results of this kind are available in the literature for
{ρn } ∈ RST ∪ RLU , but the proof of this convergence for {ρn } ∈ RMI necessitates the
theoretical work undertaken in Section 3 below.
Remark 2.6. hn = o(n−1/3 ) is required to undersmooth the bias. If DGP1 were strengthened
such that the second derivatives of m ∈ M were assumed to be uniformly bounded, then
7

uniformly valid inference in nonparmetric regression
it would be possible to relax this requirement to hn = o(n−1/6 ): see e.g. Wang and Phillips
(2009b, Rem. C; 2011). Under the null of non-predictability considered below, m is a
constant function: in this case the bias of m̂n vanishes, and the preceding condition on hn
may be relaxed to hn = o(1).
KAP are particularly concerned with testing the null that xt−1 cannot predict yt , which
may be formally expressed as
H0 : m(x) = θ, ∀x ∈ R.
The authors base their tests of H0 on a vector of t-statistics, [t̂n (x; θ, hn )]x∈X , for some
fixed X ⊂ R. The resulting tests are perhaps more correctly regarded as tests of
H0′ : m(x) = θ, ∀x ∈ X
rather than of H0 , insofar as they only have power against alternatives to H0′ .
Although θ is unknown, it is consistently estimable at rate n−1/2 under H0 , uniformly
P
over ρ ∈ R, by θ̂n := n1 n+1
t=2 yt . Accordingly, [t̂n (x; θ̂n , hn )]x∈X may be shown to inherit

the limiting distribution of [t̂n (x; θ, hn )]x∈X . KAP consider the following test statistics,
F̂n,sum :=

X

t̂2n (x; θ̂n , hn )

F̂n,max := max t̂2n (x; θ̂n , hn )

Fsum

x∈X

x∈X

Fmax , (2.13)

where, by Proposition 2.1, the stated convergence holds along all {ρn } ⊂ R (recall m(x) = θ
for all x ∈ R under H0 ), for Fsum having a χ2 [#X ] distribution, and Fmax the same
distribution as the maximum of #X independent χ2 [1] variates. For i ∈ {sum, max}, let
cτ,i denote the τ quantile of Fi , so that an α-level test of H0 based on F̂n,i rejects if and
only if F̂n,i ≥ c1−α,i . Largely as a consequence of Proposition 2.1, we have
Proposition 2.2. Suppose

DGP

and

SM

hold. Then for i ∈ {sum, max} and every finite

X ⊂ R, a test of H0 based on F̂n,i is asymptotically similar, i.e.
lim sup

sup

n→∞ (θ,ρ)∈R×R

Pθ,ρ {F̂n,i ≥ c1−α,i } = lim inf

inf

n→∞ (θ,ρ)∈R×R

Pθ,ρ {F̂n,i ≥ c1−α,i } = α.

Proofs of Propositions 2.1 and 2.2 appear in Appendix A.

3
3.1

Density estimation: a unified limit theory
Preliminaries

The preceding section is underpinned by some new results concerning the limiting behaP
viour of nt=1 Khn (xt − x) – which becomes a density estimator if suitably normalised –
when {xt } is mildly integrated, i.e. along those drifting parameter sequences {ρn } ∈ RMI
that exhibit moderate deviations from a unit root.
8

j. a. duffy

The proofs of these results in turn rely on the following extension of Theorem 2.1 in
Wang and Phillips (2009a, hereafter WP). We first restate their assumptions, some of
which will also be needed here. Let {x̃n,t }nt=1 be a triangular array, {F̃n,t }nt=1 a collection
of σ-fields such that each x̃n,t is F̃n,t -measurable, f : R → R, and define
Ωn (η) := {(s, t) | ηn ≤ s ≤ (1 − η)n, s + ηn ≤ t ≤ n}
for η ∈ (0, 1).
Assumption

WP.

WP1

f ∈ L1 ∩ L2 .

WP2

There exists a stochastic process X(r) on [0, 1] having continuous local time LX (r, a)
such that x̃n,⌊nr⌋

WP3

X(r) in ℓ∞ ([0, 1]).

There exists an n0 ∈ N such that for all 0 ≤ s < t ≤ n and n ≥ n0 , there exist
constants {dn,s,t } such that
(a) for some m0 > 0 and C > 0, inf (s,t)∈Ωn (η) ≥ η m0 /C as n → ∞, and
P
i. limη→0 limn→∞ n1 nt=(1−η)n d−1
n,0,t = 0,
Ps+ηn −1
1
dn,s,t = 0,
ii. limη→0 limn→∞ n max0≤s≤(1−η)n t=s+1
P
n
iii. lim supn→∞ n1 max0≤s≤n−1 t=s+1 d−1
n,s,t < ∞;

(b) conditional on F̃n,s , (x̃n,t − x̃n,s )/dn,s,t has a density hn,s,t(x) which is uniformly bounded (in n, s and t) by a constant K < ∞, and
lim lim

sup |hn,s,t (u) − hn,s,t (0)| = 0.

sup

δ→0 n→∞ (s,t)∈Ω (δ1/2m0 ) |u|≤δ
n

Note that WP have n0 = 1 in their statement of

WP3,

(3.1)

but it is clearly sufficient for

their result that this condition hold only for n sufficiently large. Our extension of WP’s
Theorem 2.1, stated as Proposition 3.1 below, consists of replacing
Assumption
WP2′

WP

WP2

with

(continued).

There exists a stochastic process µ̃ : [0, 1] × R → R+ , which is continuous a.s. with
R
R µ̃(r, x) dx < ∞ for all r ∈ [0, 1], such that for every g ∈ BL,
⌊nr⌋
1X
g(x̃n,t − a)
n

fdd

Z

R

t=1

over (r, a) ∈ [0, 1] × R.

9

g(x − a)µ̃(r, x) dx,

(3.2)

uniformly valid inference in nonparmetric regression

Proposition 3.1. Suppose

WP1, WP2′

and

WP3

hold. Then if c̃n → ∞ and c̃n /n → 0

⌊nr⌋
c̃n X
f [c̃n (x̃n,t − a)]
n

fdd

µ̃(r, a)

t=1

Z

(3.3)

f
R

over (r, a) ∈ [0, 1] × R.
Remark 3.1. While

WP2

is certainly sufficient for

WP2′

with µ̃ = LX , it is unnecessarily

restrictive. Indeed, it is evident from Jeganathan (2004) that (3.3) may obtain even if
the convergence in

WP2

holds only in the sense of the finite-dimensional convergence. The

proof of his Lemma 8 further implies that

WP2′

holds whenever x̃n,⌊nr⌋

fdd

X(r) and

{x̃n,⌊nr⌋ } satisfies the following weak asymptotic ‘equicontinuity in probability’ condition:
that for every ǫ > 0,
lim lim sup

sup

δ→0 n→∞ |r1 −r2 |≤δ

P{|x̃n,⌊nr1 ⌋ − x̃n,⌊nr2 ⌋ | > ǫ} = 0.

(3.4)

This is considerably weaker than asymptotic equicontinuity (i.e. tightness), which would
require control over sup|r1 −r2 |≤δ |x̃n,⌊nr1 ⌋ − x̃n,⌊nr2 ⌋ |. However, as discussed in more detail
in Remark 3.5 below, when {x̃n,t } is derived from a mildly integrated process, even such
an apparently weak requirement as (3.4) fails to hold: though the finite-dimensional limit
of x̃n,[nr] exists, it is not separable. For these processes,

WP2′

must therefore be verified by

other means.
Remark 3.2. (3.2) extends straightforwardly, via a suitable choice of approximating BL
functions, to x 7→ 1{x ≤ a}, and thereby entails the convergence of the empirical distribution function of {x̃n,t } to its population counterpart, i.e.
n

Fn (a) :=

1X
1{x̃n,t ≤ a}
n

fdd

t=1

where F is itself a distribution function if

R

Z

µ̃(1, x) dx =: F (a),

(3.5)

{x≤a}

R µ̃(1, x) dx

= 1, as is generally the case. Insofar

as (3.5) holds, F may be identified as the ‘spatial distribution’ associated to the finitedimensional limit X of x̃n,⌊nr⌋ . We might accordingly refer to x 7→ µ̃(1, x) as the ‘spatial
density’ associated to X. Some such unifying term is needed here, because depending on the
process generating x̃n,⌊nr⌋ , µ̃(1, x) may correspond to either the (non-random) probability
density of X(1), or the local time density of r 7→ X(r), but not both.

3.2

Finite-dimensional convergence

Proposition 3.1 is broad enough to cover the entire class of regressor processes contemplated
in

DGP,

even when ρ = ρn varies with n. Indeed, it is the manner in which ρn approaches

unity (if at all) that determines the density µ̃ appearing in (3.2). In accordance with the

10

j. a. duffy

division of the sequences {ρn } ∈ R given in Section 2.5 above, define


rν (a)

 ρ
µ(r, a; {ρn }) := rϕ(a)



Lc (r, a)

if {ρn } ∈ RρST
(3.6)

if {ρn } ∈ RMI
if {ρn } ∈ RcLU

where νρ is the density corresponding to the stationary solution to (2.2), normalised to have
unit variance; ϕ is the standard Gaussian density; and Lc (r, a) is the local time density (at
time r ∈ [0, 1] and point a ∈ R) associated to the normalised Ornstein–Uhlenbeck process,
Z

Jc (r) :=

1

2(1−s)c

e

ds

0

−1/2 Z

r

e(r−s)c dW (s),

(3.7)

0

for W a standard Brownian motion on [0, 1].
Our main result on the finite-dimensional convergence of density estimators, when
applied to a series {xt } satisfying

may be stated as follows. Let {hn } denote a
deterministic, nonzero bandwidth sequence, define dn := var(xn )1/2 , and recall fh (x) :=
DGP,

h−1 f (h−1 x). The proof appears in Appendix B.
Theorem 3.1. Suppose

DGP

holds with ρ = ρn for some {ρn } ∈ R, and f ∈ L1 ∩ L2 .

Then if hn = o(dn ) and nd−1
n hn → ∞,
⌊nr⌋
dn X
fhn (xt − dn a)
µn (r, a; f, hn ) :=
n

fdd

µ(r, a; {ρn })

t=1

Z

f,

(3.8)

R

over (r, a) ∈ [0, 1] × R.
Remark 3.3. dn → ∞ whenever {ρn } ∈ RMI ∪ RLU (see Remark 3.6 below) and so the
arguments given in the proof of Theorem 3.1 also imply that, in this case,
⌊nr⌋
dn X
fhn (xt − x)
n
t=1

µ(r, 0; {ρn })

Z

f
R

jointly with (3.8), for each x ∈ R.
Remark 3.4. The stationary ({ρn } ∈ RST ) and local-to-unity ({ρn } ∈ RLU ) cases are
covered by the results of Wu and Mielniczuk (2002), Wang and Phillips (2009b) and Wu
et al. (2010). The proof under mild integration ({ρn } ∈ RMI ) is new to the literature,
and the arguments employed are a combination of those appropriate to the stationary and
local-to-unity cases.
As in stationary case, one might envisage a ‘direct’ proof of (3.8), by proving the
asymptotic negligibility of
n
dn X
[fhn (xt ) − Efhn (xt )],
µn − Eµn =
n
t=1

11

uniformly valid inference in nonparmetric regression

and then demonstrating the convergence of Eµn to the r.h.s. of (3.8) (here we have taken
a = 0 and r = 1 for simplicity). However, the lesser recurrence of mildly integrated
processes, as reflected in the reduced standardisation nd−1
n , significantly complicates the
problem. Straightforward calculations show that the bound given in (13) in Wu et al.
(2010) would here imply only that
|µn − Eµn | .p (nhn )−1/2 dn + n−1/2 d3n .

(3.9)

Since dn ≍ (1−ρ2n )−1 under mild integration (see Remark 3.6 below), requiring negligibility
of the r.h.s. would thus exclude those {ρn } ∈ RMI for which 1 − ρn = O(n−1/3 ).
The failure of the bound in (3.9) to be useful over the whole of the mildly integrated
region necessitates the different proof strategy employed here, which is to use a kind of law
of large numbers to establish (3.2) for the scale-normalised array
x̃n,t := var(xn )−1/2 xt = d−1
n xt ,

(3.10)

(see Proposition B.1 in Appendix B), which establishes that {x̃n,t } satisfies satisfies
Since

WP1

and

WP3,

WP2′ .

it is then possible to invoke Proposition 3.1.

Remark 3.5. The tripartite classification in (3.6) is reflected in the different possible finitedimensional limits X(r; {ρn }) of the standardised regressor process Xn (r) := d−1
n x⌊nr⌋ .
Under both stationarity and mild integration, the relatively weak dependence between
Xn (r1 ) and Xn (r2 ) vanishes in the limit, and so X has the property that X(r1 ) and
X(r2 ) are independent for every r1 6= r2 . This explains why even such an apparently mild
equicontinuity requirement as (3.4) is unavailing for the purposes of proving Theorem 3.1.
Under mild integration, dn → ∞ and an invariance principle operates to ensure that
the marginals of X(r) are standard normal; whereas in the stationary case, dn is bounded
and the marginals have density νρ , which depends on the distribution of {εt }. The limiting
process X under mild integration thus corresponds to a continuous-time, standard normal
white noise process, which we denote by G. (A rigorous basis for these assertions is provided
by Proposition B.1(ii) in Appendix B, and the proof thereof.)
The strong dependence between Xn (r1 ) and Xn (r2 ) that is a characteristic of localto-unity processes ensures that, in this case, Xn converges weakly to the diffusion Jc (see
(3.7) above). As c → −∞, the finite-dimensional distributions of Jc converge to those of
G, and in this sense there is continuity, in the limit, at the boundary demarcating mildly
integrated and local-to-unity processes.
Remark 3.6. When {ρn } ∈ RST ∪ RMI , it may be shown that d2n ∼ nωn2 (ρn )φ2 , where
ωn2 (ρ)

:=

Z

1

2 −1)

e(1−r)n(ρ

dr =

0

1
2
[1 − e−n(1−ρ ) ].
2
n(1 − ρ )

In particular, d2n ∼ φ2 (1 − ρ2n )−1 if {ρn } ∈ RMI , and d2n ∼ nφ2
(See Section S.1 in the Supplement for further details.)
12

R1
0

(3.11)

e2(1−s)c ds if {ρn } ∈ RcLU .

j. a. duffy

3.3

Weak convergence of the density estimator process

By adapting some of the arguments from Duffy (2016), and fixing r = 1, it is possible
to strengthen the conclusion of Theorem 3.1 to weak convergence in ℓucc (R), the space of
bounded real-valued functions on R, equipped with the topology of uniform convergence on
compacta. Results of this kind are an essential ingredient to proofs of uniform convergence
rates for kernel density estimators, along the lines of Duffy (2016, 2017). We also allow
the bandwidth to be data-dependent, as well as to depend on the location a ∈ R, as per
Assumption

H.

hn = o(dn ) and

hn :
−1
hn =

R → R+ is continuous, with hn (a) ∈ [hn , hn ] for all a ∈ R, where
−2
n).
o(nd−1
n log

Theorem 3.2. Suppose H and DGP hold, the latter with ρ = ρn for some {ρn } ∈ R. Then
R
for any f ∈ BL with R |f (x)x| dx < ∞,
n
dn X
µn (a; f, hn ) :=
fhn(a) (xt − dn a)
n

µ(1, a; {ρn })

t=1

Z

f =: µ(a; {ρn })
R

Z

f
R

in ℓucc (R).
Remark 3.7. The stated convergence entails that the distribution of the process a 7→
µn (a; f, hn ), viewed as a random element of ℓucc (R), converges to that of a 7→ µ(a; {ρn }).
Remark 3.8. The result may be extended to a broader class of functions than BL, such as
is allowed for by Theorem 3.1 in Duffy (2016), by means of a similar bracketing argument
as is given in the proof of that result.
The proof of Theorem 3.2 appears in Appendix D.

4

Conclusion

This paper has established the validity of conventional nonparametric inferential procedures in a predictive regression, where the degree of persistence of the regressor is unknown
(and possibly very high). This opens the way for the systematic application of nonparametric methods in this setting, where they also enjoy the considerable advantage of being able
to easily relate series with radically different memory properties. Our work on this problem
has necessitated the development of some new limit theory for kernel density estimators, in
the presence of mildly integrated processes. These new results fill an important gap in the
existing literature, and have allowed us to provide a unified treatment of these estimators
in an autoregressive setting.

5

References

Abadir, K. M., W. Distaso, L. Giraitis, and H. L. Koul (2014): “Asymptotic
normality for weighted sums of linear processes,” Econometric Theory, 30, 252–84.
13

uniformly valid inference in nonparmetric regression

Andrews, D. W. K. and X. Cheng (2012): “Estimation and inference with weak,
semi-strong, and strong identification,” Econometrica, 80, 2153–2211.
Campbell, J. Y. and M. Yogo (2006): “Efficient tests of stock return predictability,”
Journal of Financial Economics, 81, 27–60.
Cavanagh, C. L., G. Elliott, and J. H. Stock (1995): “Inference in models with
nearly integrated regressors,” Econometric Theory, 11, 1131–1147.
Duffy, J. A. (2016): “A uniform law for convergence to the local times of linear fractional
stable motions,” Annals of Applied Probability, 25, 45–72.
——— (2017): “Uniform convergence rates over maximal domains in structural nonparametric cointegrating regression,” Econometric Theory, 33, 1387–1417.
Elliott, G., U. K. Müller, and M. W. Watson (2015): “Nearly optimal tests when
a nuisance parameter is present under the null hypothesis,” Econometrica, 83, 771–811.
Feller, W. (1971): An Introduction to Probability Theory and its Applications, vol. II,
New York, USA: Wiley.
Hall, P. and C. C. Heyde (1980): Martingale Limit Theory and Its Application, New
York (USA): Academic Press.
Jansson, M. and M. J. Moreira (2006): “Optimal inference in regression models with
nearly integrated regressors,” Econometrica, 74, 681–714.
Jeganathan, P. (2004): “Convergence of functionals of sums of r.v.s to local times of
fractional stable motions,” Annals of Probability, 32, 1771–95.
Kasparis, I., E. Andreou, and P. C. B. Phillips (2015): “Nonparametric predictive
regression,” Journal of Econometrics, 185, 468–94.
Magdalinos, T. and P. C. B. Phillips (2009): “Econometric inference in the vicinity
of unity,” CoFie Working Paper 7, Singapore Management University.
Marmer, V. (2008): “Nonlinearity, nonstationarity, and spurious forecasts,” Journal of
Econometrics, 142, 1–27.
Phillips, P. C. B. and J. H. Lee (2013): “Predictive regression under various degrees of
persistence and robust long-horizon regression,” Journal of Econometrics, 177, 250–264.
Phillips, P. C. B. and T. Magdalinos (2007): “Limit theory for moderate deviations
from a unit root,” Journal of Econometrics, 136, 115–130.
Ray, D. (1963): “Sojourn times of diffusion processes,” Illinois Journal of Mathematics,
7, 615–30.
van der Vaart, A. W. and J. A. Wellner (1996): Weak Convergence and Empirical
Processes: with applications to statistics, New York (USA): Springer.
Wang, Q. (2014): “Martingale limit theorems revisited and non-linear cointegrating regression,” Econometric Theory, 30, 509–35.
Wang, Q. and P. C. B. Phillips (2009a): “Asymptotic theory for local time density
estimation and nonparametric cointegrating regression,” Econometric Theory, 25, 710–
38.
14

j. a. duffy

——— (2009b): “Structural nonparametric cointegrating regression,” Econometrica, 77,
1901–1948.
——— (2011): “Asymptotic theory for zero energy functionals with nonparametric regression applications,” Econometric Theory, 27, 235–259.
Wu, W. B., Y. Huang, and Y. Huang (2010): “Kernel estimation for time series: an
asymptotic theory,” Stochastic Processes and their Applications, 120, 2412–2431.
Wu, W. B. and J. Mielniczuk (2002): “Kernel density estimation for linear processes,”
Annals of Statistics, 30, 1441–59.

A

Proof of uniform validity of inferences

Throughout the Appendices (excepting Section B.1) and the Supplement, Assumptions DGP
and

SM

are always maintained, even when not explicitly referenced.

R
Notation. For p ∈ (1, ∞) and a function f : R → R, define kf kp := ( |f (x)|p dx)1/p and
kf k∞ := supx∈R |f (x)|; for a random variable X, kXkp := (E|X|p )1/p , and kXk∞ denotes
the essential supremum of X. C, C1 , etc., denote generic constants which may take on
different values even at different places in the same proof. In keeping with the discussion
of the nuisance parameters γ ∈ Γ in Section 2.3, any dependence of these constants on γ
is generally ignored throughout.
We shall need the following auxiliary results, the proofs of which appear in Section S.2
of the Supplement. Recall from (2.12) that R := RST ∪ RMI ∪ RLU . Corresponding to
{ρn } ∈ R, define


ν (σ −1 x) if {ρn } ∈ RρST

 ρ ρ
τ (x) := τ (x, {ρn }) := ϕ(0)
if {ρn } ∈ RMI



Lc (1, 0)
if {ρn } ∈ RcLU ,

where σρ2 denotes the variance of the distribution having density νρ . Let
en := en ({ρn }) := nd−1
n
where dn := var(xn )1/2 as was defined in (3.10): {en } is the normalising sequence that
P
turns nt=1 Khn (xt − x) into a density estimator (see Theorem 3.1). We note here that, in
view of Remark 3.6,

n1/2 . en ({ρn }) . n

for all {ρn } ∈ R.
The first lemma is a direct consequence of Theorem 3.1, and is the principal implication
of that theorem needed for the proofs of Propositions 2.1 and 2.2.

15

uniformly valid inference in nonparmetric regression

Lemma A.1. Suppose {ρn } ∈ R. Then for i ∈ {1, 2}


n
1 X 1 i xt − x
K
en
hn
hn

τ (x)

t=1

Z

K i,

where τ (x) > 0 a.s.
Lemma A.2. Suppose {ρn } ∈ R. Then there exists a C < ∞ such that
n
1 X
E|f (xt )| ≤ Ckf k1 .
en t=1

Lemma A.3. For i ∈ {1, 2},
n
1 X
Khn (xt − x)[mn (xt ) − mn (x)]i = Op (hin ).
en
t=1

Lemma A.4. For every x ∈ R, σ̂u2 (x) = σu2 + op (1).
Proof of Proposition 2.1. The proof proceeds as follows:
(i) Suppose {ρn } ∈ R := RST ∪ RMI ∪ RLU , and show that (2.10) holds in this case.
(ii) Deduce from (i) that (2.10) holds for all {ρn } ⊂ R.
Then letting (mn , ρn ) ∈ H ∗ be chosen such that
Pmn ,ρn {|t̂n (x; hn )| ≥ z1−α/2 } ≥

Pm,ρ {|t̂n (x; hn )| ≥ z1−α/2 } − n−1

sup
(m,ρ)∈H

∗

it follows from (ii) that
α = P{|N [0, 1]| ≥ z1−α/2 } = lim Pmn ,ρn {|t̂n (x; hn )| ≥ z1−α/2 }
n→∞

= lim

Pm,ρ {|t̂n (x; hn )| ≥ z1−α/2 },

sup

n→∞ (m,ρ)∈H

∗

whence the t test has asymptotic size α. Asymptotic similarity of the t test, and of the
associated confidence interval, follow by similar arguments.
(i) Let x ∈ R, {mn } ∈ M , and {ρn } ∈ R. In view of Lemma A.4, straightforward
calculations yield
t̂n (x; mn (x), hn ) = [υn (x) + bn (x)](1 + op (1))
where

1/2 Pn
hn
Khn (xt − x)ut+1
υn (x) = P t=1
R

n
2 1/2
σu
t=1 Khn (xt − x) K

16

j. a. duffy

is as defined in (2.11), and
1/2

bn (x) :=

hn

By Lemma A.3,

Pn

t=1 Khn (xt − x)[mn (xt ) − mn (x)]
1/2
R
P
σu R K 2 nt=1 Khn (xt − x)

=:

bn,1 (x)
.
bn,2 (x)

bn,1 (x) = Op (h3/2
n en ),

(A.1)

(A.2)

and by Lemma A.1,


e−1/2
bn,2 (x)
n

η(x) := σu τ (x)

Z

K
R

2

1/2

(A.3)

,

which is strictly positive a.s.; here σρ2 denotes the variance of the stationary solution to
3/2 1/2

(2.2). Together (A.1)–(A.3) yield |bn (x)| .p hn en , which is o(1) since hn = o(n−1/3 ) by
assumption, whence
t̂n (x; mn (x), hn ) = υn (x)[1 + op (1)].
The joint limiting distribution of [t̂n (x; mn (x), hn )]x∈X can thus be obtained via an
application of an appropriate martingale CLT. Consider
Mn (x) :=
Under

DGP4,



hn
en

1/2 X
n

Khn (xt − x)ut+1 .

(A.4)

t=1

Mn is a martingale with conditional variance


n
σu2 X 2 xt − x
K
hMn (x)i =
en hn
hn

η 2 (x),

(A.5)

t=1

by Lemma A.1. Furthermore, the (standardised) summands in (A.4) satisfy a conditional
Lyapunov condition, since under


hn
en

DGP4

2 X
n
[Khn (xt − x)]4 · E[|ut+1 |4 | Gt ] .p
t=1

by Lemma A.2, and since n1/2 hn → ∞ under

SM2.

1
= o(1),
en hn

When {ρn } ∈ RρST ∪ RMI , the r.h.s. of

(A.5) is non-random, and so the asymptotic normality of (A.4) follows from Theorem 3.2
in Hall and Heyde (1980). When {ρn } ∈ RLU , we note further that


hn
nen

1/2 X
n

Khn (xt − x)|Et εt+1 ut+1 |

t=1

≤ σu



hn
nen

1/2 X
n

3/2

Khn (xt − x) .p

t=1

hn en
n

!1/2

.

 e 1/2
n

n

= o(1)

by the Cauchy-Schwarz inequality and Lemma A.2. Thus, an appeal to Theorem 2.1 of

17

uniformly valid inference in nonparmetric regression

Wang (2014), together with the preceding, ensures that for all {ρn } ∈ R,
Mn (x)

ξη(x),

(A.6)

where ξ =d N [0, 1] is independent of η; the preceding holds jointly with (A.3).
Finally, regarding the joint limiting distribution of the vector [Mn (x)]x∈X , we note
that, for any x 6= x′ and αx , αx′ ∈ R,
hαx Mn (x) + αx′ Mn (x′ )i
n
2
σu2 hn X
=
αx Khn (xt − x) + αx′ Khn (xt − x′ )
en
t=1




n 
n
2
′
X
σu
σu2 X
2
2 xt − x
2 2 xt − x
=
gn (xt )
+ αx′ K
+ 2αx αx′
α K
en hn t=1 x
hn
hn
en t=1

where gn (u) := hn · Khn (u − x) · Khn (u − x′ ). By Lemma A.2,

 

Z
n
1 X
u−x
1
u − x′
|gn (xt )| .p kgn k1 =
K
K
du
en
hn R
hn
hn
t=1


Z
x − x′
K(u)K u +
=
du
hn
R
= o(1)
and so by the arguments that led to (A.6) above, and the Cramï¿œr–Wold device,
[Mn (x)]x∈X

[ξ(x)η(x)]x∈X ,

where [ξ(x)]x∈X =d N [0, I#X ], independent of [η(x)]x∈X . Since this occurs jointly with
the convergence in (A.3), we may conclude that (2.10) indeed holds for all {ρn } ∈ R.
The argument here largely follows the proof of Lemma 2.1 in Andrews and
Cheng (2012). Let Tn := [t̂n (x; mn (x), hn )]x∈X , T∞ =d N [0, I#X ], and f be an arbitrary
(ii).

BL function. It follows from part (i) of the proof that
Emn ,ρn f (Tn ) → Ef (T∞ )

(A.7)

for every {mn } ⊂ M and {ρn } ∈ R, where Emn ,ρn is indexed by the true parameters mn
and ρn . We need to show that the preceding holds for every {ρn } ⊂ R. To that end, let
{ρn } ⊂ R be given. It suffices to show that for every subsequence {pn } of {n}, there exists
a further subsequence {wn } of {pn } such that
Emwn ,ρwn f (Twn ) → Ef (T∞ ).
Let {pn } be an arbitrary subsequence of {n}, and cn := n(ρn −1). By a compactification
18

j. a. duffy

of R, {(ρpn , cpn )} has an accumulation point (ρ, c) ∈ R × [−∞, 0]. Now let {wn } be a
subsequence of {pn }, chosen as follows. If
(i) ρ < 1: choose {wn } such that ρwn → ρ and ρwn < 1, for all n ∈ N;
(ii) ρ = 1 and either:
(a) c ∈ (−∞, 0]: choose {wn } such that cwn → c; or
(b) c = −∞: choose {wn } such that (ρwn , cwn ) → (1, −∞).
Note that in case (ii)(b),
wn−1 cwn = ρn − 1 → 0

(A.8)

as n → ∞.
Corresponding to these three cases, construct a new sequence {ρ∗n } as follows.
(i) ρ∗n = ρwk for wk ≤ n < wk+1 : then ρ∗n → ρ < 1, whence {ρ∗n } ∈ RρST .
(ii) ρ∗n = 1 + n−1 cwk for wk ≤ n < wk+1 . Then by construction,
c∗n := n(ρ∗n − 1) = cwk

for wk ≤ n ≤ wk+1 ,

and hence in case:
(a) limn→∞ c∗n = limk→∞ c∗wk = c ∈(−∞, 0], so {ρ∗n } ∈ RcLU ;
(b) limn→∞ c∗n = −∞, and for wk ≤ n ≤ wk+1 ,
|ρ∗n − 1| = n−1 |c∗n | = n−1 |cwk | ≤ wk−1 |cwk | → 0
as k → ∞, by (A.8). Thus ρ∗n → 1 and {ρ∗n } ∈ RMI .
It follows that {ρ∗n } ∈ R in all cases, and thus (A.7) holds for {ρ∗n } by the first part of the
proof. Since by construction ρ∗wn = ρwn for all n ∈ N, we finally have
Ef (T∞ ) = lim Emn ,ρ∗n f (Tn ) = lim Emwn ,ρ∗wn f (Twn ) = lim Emwn ,ρwn f (Twn ).
n→∞

n→∞

n→∞

Proof of Proposition 2.2. Let {ρn } ⊂ R. Under H0 , E(θ̂n − θ)2 = n−1 σu2 , whence
t̂n (x; θ̂n , hn ) − t̂n (x; θ, hn ) =
=

1/2 Pn
t=1 Khn (xt − x) · (θ̂n − θ)
−
1/2
R
P
σ̂u R K 2 nt=1 Khn (xt − x)


en hn 1/2
Op

hn

n

= op (1)
Therefore Proposition 2.1 gives
[t̂n (x; θ̂n , hn )]x∈X = [t̂n (x; θ, hn )]x∈X + op (1)
19

N [0, I#X ],

(A.9)

uniformly valid inference in nonparmetric regression

(Note that since m = θ under the null, the estimator m̂n has no bias, and so only hn = o(1)
is needed to prove (A.9).) Hence, by the continuous mapping theorem,
lim sup Pθ,ρn {F̂n,i ≥ c1−α,i } = lim inf Pθ,ρn {F̂n,i ≥ c1−α,i } = α
n→∞

n→∞

for i ∈ {sum, max}.

B
B.1

Proof of finite-dimensional convergence
Proof of Proposition 3.1

Similarly to the proof of Theorem 2.1 in Wang and Phillips (2009a), define

Ln (r, a) :=

⌊nr⌋
c̃n X
f [c̃n (x̃k,n − a)]
n
k=1

Ln,ǫ (r, a) :=

⌊nr⌋ Z

c̃n X
n
k=1

f [c̃n (x̃k,n − a + zǫ)]ϕ(z) dz,

R

and set ϕǫ (x) := ǫ−1 ϕ(ǫ−1 x). It follows from Lemma 7 in Jeganathan (2004) that, for each
ǫ > 0 fixed, there is a non-random δn = o(1) such that
Z
⌊nr⌋
1X
ϕǫ (x̃k,n − a) f ≤ δn → 0.
Ln,ǫ (r, a) −
n
R
k=1

Furthermore, the arguments used by Wang and Phillips (2009a) to prove that
lim lim E|Ln (r, a) − Ln,ǫ (r, a)| = 0,

ǫ→0 n→∞

for each a ∈ R, which corresponds to (5.1) in that paper, require only their Assumptions 2.1
and 2.3, both of which are maintained here (as WP1 and WP3 respectively). Finally, by WP2′ ,
⌊nr⌋
1X
ϕǫ (x̃k,n − a)
n
k=1

fdd

Z

ϕǫ (x − a)µ̃(r, x) dx
R

=

Z

ϕ(x)µ̃(r, ǫx + a) dx = µ̃(r, a) + op (1)
R

over (r, a) ∈ [0, 1] × R as n → ∞ and then ǫ → 0, since µ̃ is continuous a.s.

B.2

Proof of Theorem 3.1

{ρn } ∈ RLU . Proposition 7.1 in Wang and Phillips (2009b), together with the arguments used to prove their Proposition 7.2, establish that {x̃n,t } satisfies
(Technically, the authors only consider sequences of the form ρn = 1 +

20

c
n

WP2

and

WP3.

for fixed c ∈ R,

j. a. duffy

but their arguments clearly carry over to the slightly more general situation in which
ρn = 1 +

cn
n

for cn → c ∈ R, as permitted by RLU .) Thus, in this case, the result follows

by Proposition 3.1.
{ρn } ∈ RMI . In this case, we shall need the following two results, the proofs of which
are given in Appendix C. Recall the definition of x̃n,t given in (3.10) above.
Proposition B.1. Suppose g ∈ BL and {ρn } ∈ RMI . Then
P⌊nr⌋
P⌊nr⌋
(i) n1 t=1 g(x̃n,t ) = n1 t=1 Eg(x̃n,t ) + op (1); and
R
P⌊nr⌋
(ii) n1 t=1 Eg(x̃n,t ) → r R g(x)ϕ(x) dx.

Proposition B.2. Suppose {ρn } ∈ RMI . Then x̃n,t satisfies

WP3

with F̃n,t := σ({εs }s≤t ).

It follows immediately from Proposition B.1 that for every g ∈ BL,
Z
⌊nr⌋
⌊nr⌋
1X
1X
p
g(x̃n,t − a) =
Eg(x̃n,t − a) + op (1) → r g(x − a)ϕ(x) dx
n t=1
n t=1
R
for each (r, a) ∈ [0, 1] × R. Thus
{x̃n,t } satisfies

WP3,

WP2′

holds with µ̃(r, a) = rϕ(a). By Proposition B.2,

whence the result follows by Proposition 3.1.

{ρn } ∈ RST . Since dn . 1 in this case, it follows from Theorem 1 in Wu et al. (2010),
with minor modifications, that
⌊nr⌋
dn X
µn (r, a; f, hn ) =
Efhn (xt − dn a) + op (1).
n
t=1

Let pρ,t and ψρ,t respectively denote the Lebesgue density and characteristic function of
xt , and pρ and ψρ those of the stationary solution to (2.2), for ρ < 1.
P
Define vt := ∞
k=0 φk ǫt−k for t ≤ 0, and let tn ∈ N with tn ≤ n and tn → ∞. Since

ρn → ρ < 1 is bounded away from unity, we have
xtn =

tX
n −1

ρsn vtn −s =d

s=0

tX
n −1

p

ρsn v−s →

∞
X

ρs v−s

(B.1)

s=0

s=0

where the r.h.s. has the same distribution as the stationary solution to (2.2). Deduce
ψρn ,tn (λ) → ψρ (λ) for each λ ∈ R, whence
kpρn ,tn − pρ k∞ ≤

Z

{|λ|≤A}

|ψρn ,tn (λ) − ψρ (λ)| dλ +

Z

{|λ|>A}

[|ψρn ,tn (λ)| + |ψρ (λ)|] dλ

→ 0,
as n → ∞ and then A → ∞, where we have used |ψρn ,tn (λ)| ∨ |ψρn (λ)| ≤ |ψε (φ0 λ)| to
control the integral over {|λ| > A}.
21

uniformly valid inference in nonparmetric regression

Since the convergence in (B.1) also holds in mean square, taking tn = n yields dn =
var(xn )1/2 → σρ , the standard deviation of the stationary solution to (2.2). Thus
Efhn (xtn − dn a) =

Z

R

f (x)pρn ,tn (dn a + hn x) dx
Z
Z
f (x)pρ (dn a + hn x) dx + o(1) → pρ (σρ a) f.
=
R

R

Noting that νρ (a) = σρ pρ (σρ a), we thus have
dn
n

⌊nr⌋

X

Efhn (xt − dn a) → (r − δ)νρ (a)

t=⌊nδ⌋+1

Z

f → rνρ (a)
R

Z

f
R

as n → ∞ and then δ → 0, while
⌊nδ⌋
dn X
Efhn (xt − dn a)
n
t=1

≤ δ · dn max |Efhn (xt − dn a)| ≤ Cδ max kpρn ,t k∞ kf k1 → 0
1≤t≤⌊nδ⌋

1≤t≤⌊nδ⌋

as n → ∞ and then δ → 0, since
Z
Z
kpρn ,t k∞ ≤ |ψρn ,t (λ)| dλ ≤ |ψε (φ0 λ)| dλ < ∞.
R

R

C

Proofs of auxiliary results for mild integration

C.1

Preliminaries

Under

DGP,

we may write xt =

P∞

k=0 at,k εt−k ,

where
k∧(t−1)

at,k := at,k (ρ) :=

X

ρl φk−l .

(C.1)

l=0

Observe that this quantity does not depend on t for 0 ≤ k ≤ t − 1, and we will accordingly
write ak := at,k in this case.
We shall make frequent use, throughout the following, of the decomposition
xt =

∞
X
k=0

at,k εt−k =

∞
X

k=t−s+1

at,k εt−k +

t−s
X

ak εt−k =: x−∞,s−1,t + xs,t,t ,

(C.2)

k=0

s−1
- and Fst -measurable
for s ∈ {1, . . . , t}: x−∞,s−1,t and xs,t,t are independent, being F−∞

22

j. a. duffy
respectively, for Fst := σ({εr }tr=s ). For r ∈ {s + 1, . . . , t − 1}, xs,t,t further decomposes as
xs,t,t =

t
X

at−k εk =

r
X

at−k εk +

at−k εk = xs,r,t + xr+1,t,t ,

(C.3)

k=r+1

k=s

k=s

t
X

t -measurable.
where xs,r,t and xr+1,t,t are respectively Fsr - and Fr+1

The following elementary results are collected as lemmas for ease of reference: proofs
appear in Section S.3 of the Supplement. (The proof of Lemma C.2, while somewhat
lengthy, involves only tedious algebra is not conceptually difficult.) Recall that dn =
P
var(xn ) and φ = ∞
k=0 φk .
Lemma C.1. Suppose {ρn } ∈ RMI . Then
(i) ρnǫ
n → 0 for any ǫ > 0;
(ii) (1 − ρ2n ) ∼ 2(1 − ρn );
Lemma C.2. Suppose {ρn } ∈ RMI and ǫ > 0. Then
(i) d2n ∼ φ2 (1 − ρ2n )−1 ; and
(ii) for any sequence {tn } with nǫ ≤ tn ≤ n,
var(xtn ) ∼ var(x1,tn ,tn ) ∼ d2n .

C.2

Proof of Proposition B.1

We first state and prove the following auxiliary lemma, which is the key ingredient in
the proof of the first part of Proposition B.1. For a function g ∈ BL, let kgkLip :=
supx6=y |g(x) − g(y)|/|x − y|.
Lemma C.3. For any g ∈ BL,
n
∞
n
X
X
X
E
[g(xt ) − Eg(xt )] ≤ kgkLip
a2t,k
t=1

k=0

t=1

!1/2

≤ kgkLip n

1/2

P∞

k=0 |φk |
,
1 − |ρ|

(C.4)

where the second inequality holds if |ρ| < 1.
t ]. We decompose
Proof. Let Et [·] := E[· | F−∞
∞
X
g(xt ) − Eg(xt ) =
[Et−k g(xt ) − E(t−1)−k g(xt )]
k=0

where the sum on the r.h.s. converges a.s., since Et−k g(xt ) → Eg(xt ) a.s. as k → ∞, by
the reverse martingale convergence theorem. Therefore we may write
∞ X
∞
n
n
X
X
X
[Et−k g(xt ) − E(t−1)−k g(xt )] =:
[g(xt ) − Eg(xt )] =
Mn,k .
t=1

k=0 t=1

k=0

23

(C.5)

uniformly valid inference in nonparmetric regression

Clearly, by the orthogonality of martingale differences,
2
EMn,k
=

n
X

E[Et−k g(xt ) − E(t−1)−k g(xt )]2 .

(C.6)

t=1

Now by the decomposition (C.2),
xt =

k−1
X

at,s εt−s + at,k εt−k +

s=0

=d

k−1
X

∞
X

at,s εt−s

s=k+1

at,s εt−s + at,k ε∗

+

s=0

∞
X

at,s εt−s =: x∗t

s=k+1

t−k
. Thus
where ε∗ =d ε0 is defined to be independent of {εt }, and hence also of F−∞

E(t−1)−k g(xt ) = Et−k g(x∗t ), whence
|Et−k g(xt ) − E(t−1)−k g(xt )| = |Et−k [g(xt ) − g(x∗t )]| ≤ kgkLip |at,k | · Et−k |εt−k − ε∗ |.
Hence, by (C.6) and Jensen’s inequality, and recalling that σε2 = 1,
2
≤ 2kgk2Lip
EMn,k

n
X

a2t,k ,

t=1

which together with (C.5) yields the first inequality in (C.4).
For the second inequality, we note from (C.1) that
max |at,k | ≤

1≤t≤n

n−1
X

|ρ|l |φk−l |,

l=0

with the convention that φ−l := 0 for l < 0. Hence if |ρ| < 1,
∞
n
X
X
k=0

t=1

a2t,k

!1/2

≤n

1/2

∞
X
k=0

max |at,k | ≤ n

1/2

1≤t≤n

n−1
X

l

|ρ|

l=0

∞
X

|φk−l | ≤ n

k=0

1/2

P∞

k=0 |φk |
.
1 − |ρ|

Proof of Proposition B.1(i). We take r = 1 for simplicity; the proof for fixed r ∈ [0, 1) is
analogous. When ρ ∈ (0, 1), applying Lemma C.3 to the unstandardised process {xt } gives
the bound

P∞
n
X
1/2
k=0 |φk |
E
[g(xt ) − Eg(xt )] ≤ kgkLip n
.
1−ρ
t=1

It follows that replacing xt by the rescaled process x̃n,t = d−1
n xt in (C.7) gives
n

E

1X
1
n1/2
[g(x̃n,t ) − Eg(x̃n,t )] . ·
n t=1
n dn (1 − ρn )

24

(C.7)

j. a. duffy

≍

1
n1/2

·

(1 − ρ2n )1/2
1
= o(1),
≍
1 − ρn
[n(1 − ρn )]1/2

where we have used Lemmas C.1–C.2.
Proof of Proposition B.1(ii). Let ǫ > 0. It is proved below that along every sequence
{tn } ⊂ [nǫ, n],
x̃n,tn
whence Eg(x̃n,tn ) →
boundedness of g,

R

R g(x)ϕ(x) dx,

(C.8)

N [0, 1],

since g is bounded. Then by the preceding and the


Z
Z
⌊nr⌋
1X
Eg(x̃n,t ) − gϕ ≤ ǫkgk∞ + sup Eg(x̃n,t ) − gϕ → ǫkgk∞ .
n
t∈[nǫ,n]
t=1

Since ǫ was arbitrary, the result follows.
It remains to prove (C.8). It follows from Lemma C.2 that var(x̃n,tn ) → 1. Moreover,
P
we may write x̃n,tn = nk=−∞ δn,k εk , where
δn,k =

( −1
dn atn ,k
0

if k ≤ tn ,

otherwise;

and
−1
max|δn,k | ≤ d−1
n max|atn ,k | ≤ dn
k≤n

k≤tn

∞
X

|φi | = o(1).

i=0

(C.8) therefore follows from Lemma 2.1(i) in Abadir, Distaso, Giraitis, and Koul (2014).

C.3

Proof of Proposition B.2

We shall need the following results, proofs of which appear in Section S.3 of the Supplement.
For {ρn } ∈ RMI ∪ RLU , define kn := kn ({ρn }) to be the largest integer for which
kn ({ρn }) ≤ 1 ∨

(
[(1 − ρn )−1 ∧ n]/2 if {ρn } ∈ RMI

if {ρn } ∈ RLU ,

n/2

(C.9)

for each n sufficiently large; observe (by Remark 3.6) that kn ≍ d2n in both cases. Recall
the definition of ak = ak (ρn ) given in (C.1) above.
Lemma C.4. Suppose {ρn } ∈ RMI ∪ RLU . There exist k0 , n0 ∈ N with k0 even, such that
(i) ρkn , ρ−k
n ∈ [C1 , C2 ] for some C1 , C2 ∈ (0, ∞) for all n ≥ n0 and 0 ≤ k ≤ 2kn ; and
(ii) for some a, a ∈ (0, ∞), |a0 | ≥ a and for all n ≥ n0 ,
a≤

min

k0 /2≤k≤2kn

|ak | ≤ max |ak | ≤ a.
0≤k≤n

25

(C.10)

uniformly valid inference in nonparmetric regression
Lemma C.5. Let {ϑk }k∈N have σϑ2 :=
exists a bounded function

G(A, σ 2 , ψǫ ),

E iλ

{|λ|≥A}

∞
X

ϑk εk

k=1

!

2
k=1 ϑk

> 0, and {εt }t∈Z be as in

DGP2.

There

not otherwise depending on {ϑk }, such that σ 2 7→

G(A; σ 2 , ψǫ ) is decreasing in σ,
Z

P∞

dλ ≤ G(A; σϑ2 , ψǫ ) ≤ Cσϑ−1 ,

∀A ≥ 0

(C.11)

for some C < ∞ depending only on kψǫ k1 , and limA→∞ G(A; σϑ2 , ψǫ ) = 0.
Lemma C.6. Let {ρn } ∈ RMI and η ∈ (0, 1]. Then
1
n

Z

ηn
1

1
du = η + o(1).
(1 − ρun )1/2

Proof of Proposition B.2. We shall take
dn,s,t := (1 − ρ2(t−s)
)1/2 .
n
Since ρn → 1, we assume throughout that n is sufficiently large that ρn ≥ ( 12 , 1).
We first consider part (a) of
1
n

n
X

d−1
n,0,t

t=(1−η)n

1
=
n

WP3.

For (a)(i), we have

n
X

1
ηn
1
≤ ·
→η
2t
1/2
2(1−η)n
n
(1 − ρn )
(1 − ρn
)1/2
t=(1−η)n

by Lemma C.1. For (a)(ii), we note that
s+ηn
ηn
ηn
X
1
1X
1
1
1X
=
d−1
max
≤
1/2
k )1/2
n 0≤s≤(1−η)n t=s+1 n,s,t n
n
(1 − ρ2k
)
(1
−
ρ
n
n
k=1
k=1


Z ηn
1
1
1
+
du → η,
≤
n (1 − ρn )1/2
(1 − ρun )1/2
1

where the final convergence follows by Lemma C.6. Finally, for (a)(iii), essentially the
preceding with η = 1 yields
n
n
X
1X
1
1
d−1
=
→ 1.
max
n,s,t
1/2
n 0≤s≤n−1 t=s+1
n
(1 − ρ2k
n )
k=1

Thus part (a) of

WP3

is satisfied.

We next turn to part (b) of

WP3.

By the Fourier inversion formula and Lemma C.5,

uniform boundedness of {hn,s,t } will follow if the variance of (x̃n,t − x̃n,s )/dn,s,t , conditional
on F̃n,s := σ({εr }r≤s ), is bounded away from zero. The decomposition (C.2) yields
xt =

∞
X

k=t−s

at,k εt−k +

t−s−1
X

ak εt−k =: x−∞,s,t + xs+1,t,t .

k=0

26

j. a. duffy

Since xs+1,t,t =d x1,t−s,t−s is independent of xs and x−∞,s,t, and dn,s,t = dn,0,t−s , taking
r := t − s we have




var(x1,r,r )
xs+1,t,t
1 − ρ2n
x̃n,t − x̃n,s
| F̃n,s = var
var(x1,r,r ) =: Cφ gn,r
= 2
≥
C
var
φ
dn,s,t
dn,s,t dn
1 − ρ2r
dn,0,r d2n
n
by Lemma C.2, for some Cφ > 0 and all n sufficiently large.
We thus need to show that inf n≥n0 inf 1≤r≤n gn,r > 0 for some n0 ∈ N. To that end, we
note that for k0 as in Lemma C.4 and kn as in (C.9),


1
if 1 ≤ r ≤ k0

r

X
var(x1,r,r ) =
a2k ≥ a2 · r/2
if k0 + 1 ≤ r ≤ kn


k=0

kn /2 if kn + 1 ≤ r ≤ n

(C.12)

for n sufficiently large. We also note the inequality
1 − x2
1
1
= Pr
≥ ,
2r
2l
1−x
r
l=0 x

∀r ∈ N, x ∈ (0, 1).

Considering each of the three cases in (C.12) in turn, we have:
(i) 1 ≤ r ≤ k0 : then
gn,r ≥

1
1 2
1 − ρ2n
· a2 ≥ a2 ≥
a ;
2r
1 − ρn
r
2k0

gn,r ≥

1 − ρ2n r 2 1 2 1 2
· ·a ≥ a ≥ a ;
1 − ρ2r
2
2
2
n

(ii) k0 + 1 ≤ r ≤ kn : then

(iii) kn + 1 ≤ r ≤ n: then for some C ∈ (0, ∞),
gn,r ≥

C
C
1 − ρ2n kn 2
·
·a ≥
a2 ≥ a2 ,
2r
2r
1 − ρn
2
(1 − ρn )
2

where the second inequality follows from kn ≍ (1 − ρn )−1 ≍ (1 − ρ2n )−1 , and the third
inequality from Lemma C.4.
Thus inf 1≤r≤n gn,r is bounded away from zero for n sufficiently large, whence {hn,s,t } is
uniformly bounded.
Finally, in view of the definition of Ωn (η), (3.1) only concerns s and t for which (1 −
n 1/2
δ)n ≥ t−s = r = rn ≥ nδ for some δ ∈ (0, 1). For such rn , we have dn,0,rn = (1−ρ2r
→
n )
1 by Lemma C.1, and so arguments given in the proof of Proposition B.1(ii) yield

zn :=

x1,rn ,rn
= (1 + op (1)) · d−1
n x1,rn ,rn
dn · dn,0,rn

N [0, 1].

Letting ψzn denote the characteristic function of zn , arguments given in the proof of Corol27

uniformly valid inference in nonparmetric regression

lary 2.2 in Wang and Phillips (2009a) then imply that (3.1) holds if the sequence {ψzn } is
uniformly integrable. But this is immediate from Lemma C.5 and the fact that var(zn ) → 1,
which itself follows from Lemma C.2.

D

Proof of weak convergence (Theorem 3.2)

{ρn } ∈ RMI ∪ RLU .

In this case, the proof of Theorem 3.2 closely follows the proof

of Theorem 3.1(i) in Duffy (2016), as outlined in Section 4 of that paper. The key step is
the proof of Proposition D.1 below, which here plays the role of Propositions 4.1 and 4.2
in that paper.
To state the two auxiliary lemmas leading to Proposition D.1, we must first introduce
a martingale decomposition similar to that developed in Section 7.1 of Duffy (2016). To
t ], and consider
this end, let Et [·] := E[· | F−∞
f (xt ) =

t∧k
Xn

[Et−k+1 f (xt ) − Et−k f (xt )] + E[t−kn ]+ f (xt ),

k=1

where [a]+ := a ∨ 0: note that unlike Duffy (2016, Sec. 7.1), the decomposition here is
truncated at [t − kn ]+ rather than at 0, where kn is as in (C.9) above. Defining
ξk,t f := Et f (xt+k ) − Et−1 f (xt+k )

(D.1)

we have
Sn f :=

n
X
t=1

f (xt ) =

n
X

E[t−kn ]+ f (xt ) +

t=1

= Nn f +

kX
n −1

n
X

[Et−k f (xt ) − Et−k−1 f (xt )]

k=0 t=k+1

kX
n −1 n−k
X

[Et f (xt+k ) − Et−1 f (xt+k )] = Nn f +

k=0 t=1

kX
n −1

Mn,k f

(D.2)

k=0

P
Pn−k
t }n−k forms a
where Nn f := nt=1 E[t−kn ]+ f (xt ) and Mn,k f := t=1
ξk,tf . {ξk,t f, F−∞
t=1
martingale difference sequence for each k by construction, and so control over Mn,k f , for
each k, will be deduced from control over
Un,k f := [Mn,k f ] =

n−k
X

2
ξk,t
f

Vn,k f := hMn,k f i =

n−k
X

2
f.
Et−1 ξk,t

(D.3)

t=1

t=1

To state our bounds on the foregoing, we need a few definitions. First, define the norm
kf k[β] := inf{c ∈ R+ | |fˆ(λ)| ≤ c|λ|β , ∀λ ∈ R},
for β ∈ (0, 1], where fˆ(λ) :=

R

(D.4)

eiλx f (x) dx denotes the Fourier transform of f . (See

28

j. a. duffy

Section 4.2 and Lemma 9.1 in Duffy (2016) for more details on kf k[β] .) Define
BI[β] := {f ∈ BI | kf k[β] < ∞}
where BI denotes the class of bounded and integrable functions,
ςn (β, f ) := kf k∞ + en d−β
n (kf k1 + kf k[β] )
and

kf k2∞ + en kf k22
2
i
h
σn,k (β, f ) :=
en k−(3+2β)/2 kf k2 + e−γ1 k kf k2
1
[β]

if k ∈ {0, . . . , k0 − 1},
if k ∈ {k0 , . . . , kn − 1},

where k0 is as in Lemma C.4 above. For F ⊆ BI[β] , let

−β
δn (β, F ) := kF k∞ + e1/2
n kF k2 + en dn (kF k1 + kF k[β] ),

where kF k := supf ∈F kf k, and let
β := β({ρn }) := sup{β ∈ (0, 1] | e−1/2
dβn = o(1)}.
n
Since dn . n1/2 . en , β({ρn }) ≥

1
2

for all {ρn } ∈ RMI ∪ RLU . Finally, as in Duffy (2016,

Sec. 4.2) let τ1 and τ3/2 denote the Orlicz norms respectively associated to the convex and
increasing functions
τ1 (x) := ex − 1

τ3/2 (x) :=


x(e − 1)
x2/3

e

if x ∈ [0, 1],

− 1 if x ∈ (1, ∞).

(D.5)

Proofs of the following two lemmas appear in Section S.4 of the Supplement.
Lemma D.1. Suppose {ρn } ∈ RMI ∪ RLU and β ∈ (0, β). Then there exists a C < ∞
such that
kNn f k∞ ≤ Cςn (β, f )

(D.6)

2
kUn,k f kτ1 ∨ kVn,k f kτ1 ≤ Cσn,k
(β, f )

(D.7)

and

for all n ≥ n0 , 0 ≤ k ≤ kn − 1 and f ∈ BI.
Lemma D.2. Suppose {ρn } ∈ RMI ∪ RLU and β ∈ (0, β). Then there exists a C < ∞
such that
sup ςn (β, f ) +

f ∈G

kX
n −1

sup σn,k (β, f ) ≤ Cδn (β, G )

k=0 f ∈G

for all G ⊂ BI[β] .
With the preceding lemmas taking the places of Lemmas 7.3 and 7.5 in Duffy (2016),
29

uniformly valid inference in nonparmetric regression

the next result follows from almost exactly the same arguments as are used to prove
Propositions 4.1 and 4.2 in that paper. The very minor modifications that are required
merely reflect the slight differences between ςn , σn,k and δn as they appear above, and the
corresponding quantities in Duffy (2016); the reader is accordingly referred to that paper
for the details of the proof. Let κ(x) := (1 − |x|)1{x ∈ [−1, 1]} denote the triangular kernel
function, and µn (a; f ) := µn (1, a; f, 1).
Proposition D.1. Suppose {ρn } ∈ RMI ∪ RLU and 0 < β < β({ρn }). Then
(i) there exists a C < ∞ such that
sup kµn (a1 ; κ) − µn (a2 ; κ)kτ2/3 ≤ C|a1 − a2 |β ;

a1 ,a2 ∈R

(ii) if Fn ⊂ BI[β] with #Fn . nC , then maxf ∈Fn |Sn f | .p δn (β, Fn ) log n; and so if
kFn k1 . 1, kFn k[β] = o(dβn ) and kFn k∞ = o(en log−2 n), then
e−1
n max |Sn f | = o(1).
f ∈Fn

The proof of Theorem 3.2 now proceeds almost exactly along the lines of the proof
of Theorem 3.1(i) in Duffy (2016), with Proposition D.1 here playing the role of Propositions 4.1 and 4.2 there. Let M < ∞; the desired convergence in ℓucc (R) will follow from
convergence in ℓ∞ ([−M, M ]), the space of bounded functions on [−M, M ], equipped with
the topology of uniform convergence. As per the argument in Section 6 of Duffy (2016), it
follows immediately from part (i) of Proposition D.1 that µn (a; κ) is tight in ℓ∞ ([−M, M ]),
whence µn (a; κ)

µ(a) in ℓ∞ ([−M, M ]). Further, for any f as in the statement of The-

orem 3.2,
sup

µn (a; f, hn (a)) − µ(a; κ)

a∈[−M,M ]

≤

Z

f
R

sup
(a,h)∈[−M,M ]×[hn ,hn ]

µn (a; f, h) − µ(a; κ)

Z

f = op (1)
R

where the inequality holds under H, while the equality follows from part (ii) of Proposition D.1, together with (6.2)–(6.3) in Duffy (2016) and the subsequent arguments there.
Thus µn (a; f, hn (a))
µ(a) in ℓ∞ ([−M, M ]).
{ρn } ∈ RST . In this case, the result follows from essentially the same arguments as are
used to prove Theorem 2 in Wu et al. (2010).

30

j. a. duffy

S

Supplementary material [for online publication]

S.1

Verification of Remark 3.6

The claim to be proved is that, when {ρn } ∈ RMI ∪ RLU , d2n := var(xn ) ∼ nωn2 (ρn )φ2 , for
ωn as in (3.11). When {ρn } ∈ RMI , this follows from Lemma C.2. We therefore suppose
R1
that {ρn } ∈ RLU . In this case, cn := n(ρn − 1) → c ∈ R and ωn2 (ρn ) → 0 e2(1−s)c ds.

Define

gk (ρ) :=

k−1
X

ρ2l =

l=0

1 − ρ2k
1 − ρ2

(S.1)

for k ∈ {1, . . . , n} and ρ ≥ 0, with the final equality holding by continuity when ρ = 1.
Taking s = 1 and t = n in the decomposition (C.2), we have
var(x2n ) =

n−1
X

a2k (ρn ) +

k=0

where
2
ς1,n
(ρn ) =

n−1
X

∞
X

2
2
a2n,k (ρn ) =: ς1,n
(ρn ) + ς2,n
(ρn )

k=n

n−1
X n−1
X

φ2i gn−i (ρn ) + 2

φi φj gn−j (ρn ).

(S.2)

i=0 j=i+1

i=0

If we can show that
(i)

1
n gn−i (ρn )

→

R1
0

e2(1−s)c ds as n → ∞, for each fixed i ≥ 0; and

(ii) max1≤k≤n n1 |gk (ρn )| is uniformly bounded;
then in view of

P∞

i=0 |φi |

< ∞ and (S.2), it will follow immediately that
2
n−1 ς1,n
(ρn )

2

→φ

Z

1

e2(1−s)c ds

0

2 (ρ ) ∼ nω 2 (ρ )φ2 .
as n → ∞, whence ς1,n
n
n n

For (i), we first suppose that cn → c 6= 0. Then
2(n−i)
Z 1
−1
1 + cnn
1
e2c − 1
gn−i (ρn ) =
→
=
e2(1−s)c ds.
n
n(ρ2n − 1)
2c
0
To handle the case where cn → 0, we note first that y x = 1 + x + o(x) as (y, x) → (e, 0).
In particular



2cn
cn  n−i
cn 2(n−i)
cn
−1 = 1+
− 1 = 2cn (1 + o(1)),
1+
n
n

from which it follows that
2cn (1 + o(1))
1
gn−i (ρn ) =
→1=
n
2cn (1 + o(1))

S1

Z

0

1

e2(1−s)c ds

.
c=0

uniformly valid inference in nonparmetric regression

For (ii), we note from (S.1) that |gk (ρ)| ≤ k − 1 whenever ρ ≤ 1, while if ρ > 1, |gk (ρ)| is
maximised by taking k = n, and so boundedness follows from (i) with i = 0.
2 (ρ ) = o(n). Taking n sufficiently large, ρ > 0 and so
It remains to show that ς2,n
n
n
P∞
k
n
−n
n
−n
c
−c
ρn ∈ [ρn , ρn ] for any 0 ≤ k ≤ n. Since (ρn , ρn ) → (e , e ) and i=0 |φi | < ∞,
2
ς2,n
(ρn ) =

∞
n−1
X
X

k=n

ρln φk−l

l=0

!2

!2
∞
n−1
∞ n−1
X
X
X
X
.
|φk−l |
.
|φk−l |.
k=n

l=0

k=n l=0

Finally,
2n
X

∞ n−1
X
X
|φk−l | =

S.2

+

! n−1
∞
∞
n X
X
X
X
|φk | = o(n).
|φl | + n
|φk−l | ≤

k=2n+1

k=n

k=n l=0

∞
X

k=n

k=0 l=k

l=0

Proofs of auxiliary lemmas from Appendix A

Proof of Lemma A.1. As noted in the text, the stated convergence follows immediately
from Theorem 3.1: see also Remark 3.3. Regarding the strict positivity of τ (x): when
{ρn } ∈ RLU , this follows from Ray’s (1963) theorem; when {ρn } ∈ RMI this is immediate
from ϕ being the standard normal density; and when {ρn } ∈ RST , this follows from the
density fε of εt having been assumed strictly positive (see

DGP2).

Proof of Lemma A.2. Suppose {ρn } ∈ RST . Then Fourier inversion and the decomposition
xt = φ0 εt + x−∞,t−1,t gives, for any positive-valued f ∈ L1 ,
Ef (xt ) ≤ C

Z

|fˆ(λ)||ψε (−φ0 λ)| dλ ≤ Cφ−1
0 kψε k1 kf k1

with φ0 6= 0 by assumption. This, together with en ≍ n, yields the result in this case.
Now suppose {ρn } ∈ RMI ∪ RLU . In this case, we note


kn
n
n
X
X
X
1
kf k1 
kf k1
1
≤ C1
Ef (xt ) ≤ C
k0 +
k−1/2 + (n − kn )kn−1/2 ,
1/2
en t=1
en
en
var(x
)
t
t=k
t=k
0

0

where the first inequality follows from Lemma C.5; the second follows from Lemma C.4
(for k0 and kn as in the statement of that result) via the lower bound (C.12) (which also
holds if {ρn } ∈ RMI ). The bracketed term on the r.h.s. has the same order as
1/2
−1/2
e−1
) = e−1
n (kn + nkn
n dn + 1 . 1
−1/2

since, in particular, nkn

≍ nd−1
n = en (see (C.9) and the following text).

S2

j. a. duffy

Proof of Lemma A.3. Under
n
X

DGP1,

a mean-value expansion gives
i

Khn (xt − x)|mn (xt ) − mn (x)| ≤

hin

·

km′n k∞



n
X
xt − x
1
xt − x
K
·
hn
hn
hn

i

t=1

t=1

.p hin en

with the final bound following by Lemma A.2.
Proof of Lemma A.4. We first show that m̂n (x) = mn (x) + op (1). To that end, decompose
m̂n (x) − mn (x) =

1
en

where:
An,1 :=

An,1 + An,2
Pn
t=1 Khn (xt − x)

n
1 X
Khn (xt − x)[mn (xt ) − mn (x)] .p hn
en
t=1

by Lemma A.3; and
An,2

n
1 X
Khn (xt − x)ut+1 = op (1)
en

:=

t=1

where the claimed negligibility follows since An,2 is a martingale with variance
EA2n,2



n
1 X
2 xt − x
= 2 2
EK
u2t+1
en hn
hn
t=1



n
n
σ 2 X 1 X 2 xt − x
1
1
E
K
·
= o(1)
.p
=
en hn en
hn
hn
en hn
t=1

by Lemma A.2 and n1/2 hn → ∞ (see

SM2).

t=1

Since by Lemma A.1

n
1 X
Khn (xt − x)
en

τ (x)

t=1

which is a.s. positive, we have m̂n (x) = mn (x) + op (1) as claimed.
The remainder of the proof follows similar lines to the proof of Theorem 3.2 in Wang
and Phillips (2009b). Recalling
σ̂u2 (x)

=

1
en

we decompose the numerator as

Pn

2
t=1 Khn (xt − x)[yt+1 − m̂n (x)]
P
n
1
t=1 Khn (xt − x)
en

n
1 X
Khn (xt − x)[yt+1 − m̂n (x)]2
en t=1

S3

uniformly valid inference in nonparmetric regression
n
n
1 X
2 X
Khn (xt − x)u2t+1 +
Khn (xt − x)[mn (xt ) − m̂n (x)]ut+1
en
en

=

t=1

t=1

n
1 X
Khn (xt − x)[mn (xt ) − m̂n (x)]2
+
en
t=1

=: Bn,1 + 2Bn,2 + Bn,3 .

Letting ζt := u2t − σu2 , we claim that
Bn,1 =

n
n
σu2 X
1 X
Khn (xt − x) +
Khn (xt − x)ζt+1
en t=1
en t=1

(S.3)

σu2 τ (x).

The convergence of the first r.h.s. term in (S.3) follows from Lemma A.1. Regarding the
second r.h.s. term, we note that since ζt+1 := u2t+1 − σu2 is a martingale difference under
DGP4,

this term is a martingale with conditional variance


n
1
1 X 1 2 xt − x
1
2
·
K
= o(1)
E[ζt+1
| Gt ] . p
en hn en t=1 hn
hn
en hn

2
| Gt ] < ∞ a.s. (under
by Lemma A.2 and supt E[ζt+1

DGP4).

It follows by Corollary 3.1 of

Hall and Heyde (1980) that, indeed,
n
1 X
p
Khn (xt − x)ζt+1 → 0.
en t=1

Next, we have
Bn,3

n

1 X
≤C
Khn (xt − x) [mn (xt ) − mn (x)]2 + [m̂n (x) − mn (x)]2
en
t=1

=

Op (h2n ) + op (1)

= op (1)
by Lemmas A.2–A.3, and m̂n (x) = mn (x) + op (1) (as was proved above). Finally
Bn,2 ≤ (Bn,1 )1/2 (Bn,3 )1/2 ,
by the Cauchy-Schwarz inequality; whence by Lemma A.1 and the preceding,
σ̂u2 (x) =

Bn,1 + Bn,2 + Bn,3
1 Pn
t=1 Khn (xt − x)
en

S4

σu2 τ (x)
= σu2 .
τ (x)

j. a. duffy

S.3

Proofs of auxiliary lemmas from Appendix C

Proof of Lemma C.1. Letting cn := n(ρn − 1) → −∞, we note that for every M < ∞, we
may take n sufficiently large such that cn < −M , whence
ρnǫ
n




M nǫ
cn nǫ
≤ 1−
= 1+
→ e−M ǫ → 0
n
n

as n → ∞ and then M → ∞. Thus (i) holds. (ii) follows from
1 − ρ2n
= 1 + ρn → 2.
1 − ρn
Proof of Lemma C.2. Taking s = 1 in (C.2), we have
xt =

t−1
X

ak εt−k +

k=0

∞
X

at,k εt−k = x1,t,t + x−∞,0,t

k=t

2
2 := var(x
where x1,t,t and x−∞,0,t are independent, with variances ς1,t
1,t,t ) and ς2,t :=

var(x−∞,0,t ) respectively. Let {tn } ⊆ [nǫ, n] be as in the statement of part (ii) of the
lemma. We shall prove below that
2
2
) → φ2 ,
(1 − ρ2n ) var(xtn ) = (1 − ρ2n )(ς1,t
+ ς2,t
n
n

from which both parts of the lemma immediately follow.
Some tedious algebra (verified immediately below this proof) yields
2
ς1,t
n

=

tX
n −1

k
X

k=0

ρnk−l φl

l=0

!2

=

tX
n −1
i=0

φ2i

tnX
−i−1

ρ2k
n

+2

tX
n −1
n −1 tX

φi φj

i=0 j=i+1

k=0

tnX
−j−1

ρn2k+(j−i) (S.4)

k=0

whence, since ρn ∈ (0, 1),
2
=
(1 − ρ2n )ς1,t
n

tX
n −1

φ2i (1 − ρn2(tn −i) ) + 2

φi φj (1 − ρn2(tn −j)+(j−i) )

i=0 j=i+1

i=0

2(t −i)

tX
n −1
n −1 tX

2(⌊nǫ⌋−i)

Since ρn n
≤ ρn
→ 0 as n → ∞ for each fixed i ∈ N by Lemma C.1(i), and
P∞
i=0 |φi | < ∞, it follows that
2
→
(1 − ρ2n )ς1,t
n

∞
X

φ2i + 2

∞
∞ X
X

φi φj = φ2 .

i=0 j=i+1

i=0

2 , we note that since |ρ | ≤ 1 and C :=
Regarding ς2,t
n
φ
n

2
ς2,t
n

=

∞
X

k=tn

tX
n −1
l=0

l

ρ φk−l

!2

≤ Cφ

∞ tX
n −1
X

k=tn l=0

S5

P∞

ρln |φk−l |

i=0 |φi |

≤ Cφ

<∞

tX
n −1
l=0

ρln φ̃tn −l ,

uniformly valid inference in nonparmetric regression

where φ̃j :=
tX
n −1
l=0

P∞

i=j |φi |.



Further,

⌊tn /2⌋−1

X

ρln φ̃tn −l = 

tX
n −1

+

l=0

l=⌊tn /2⌋



ρln φ̃tn −l

/2⌋−1
 ⌊tnX

≤ φ̃⌊tn /2⌋−1 + Cφ ρn⌊tn /2⌋
ρln = o[(1 − ρ2n )−1 ],
l=0

⌊t /2⌋

since φ̃⌊tn /2⌋ → 0 and ρn n

→ 0 (by Lemma C.1(i)), and

⌊tn /2⌋

X

ρln ≤ (1 − ρn )−1 ≍ (1 − ρ2n )−1

l=0

2
= o[(1 − ρ2n )−1 ].
by Lemma C.1(ii), whence ς2,t
n

Verification of (S.4). Dropping the n subscript from tn and ρn for simplicity, and setting
m := t − 1, we have

k
m
X
X
k=0

ρ

k−l

φl

l=0

!2

=

k
k X
m X
X

ρ2k−i−j φi φj

k=0 i=0 j=0

=
=

=

m
m X
X

φi φj

m
X

ρ2k−i−j

i=0 j=0
k=i∨j
m
m
X X
φ2i
ρ2(k−i) +
i=0
k=i

m
X

φ2i

i=0

m−i
X

ρ2k + 2

2

m X
m
X

m
X

φi φj

i=0 j=i+1
m
X

φi φj

i=0 j=i+1

k=0

m
X

ρ2(k−j)+(j−i)

k=j

m−j
X

ρ2k+(j−i) .

k=0

Proof of Lemma C.4. When {ρn } ∈ RLU , the result follows essentially from arguments
given in Wang and Phillips (2009b): see their (7.14), in particular. We therefore turn to
the case {ρn } ∈ RMI . Then ρn ∈ (0, 1), and the upper bound in (C.10) follows trivially
P
from |ak (ρn )| ≤ ∞
i=0 |φi |. Further, for any 0 ≤ k ≤ 2kn ,
−2kn
n
.
≤ ρkn ≤ ρ−k
ρ2k
n
n ≤ ρn

−1

Noting that ρ(1−ρ)

−2kn ) →
n
→ e−1 as ρ → 1, and 2kn ∼ (1− ρn )−1 , it follows that (ρ2k
n , ρn

(e−1 , e) as n → ∞. Thus there exists an n0 ∈ N and C1 , C2 ∈ (0, ∞) such that ρkn , ρ−k
n ∈
[C1 , C2 ] for all n ≥ n0 and 0 ≤ k ≤ 2kn .
P
Now ak (ρn ) = ρkn kl=0 ρ−l
n φl , and for any m ≤ k ≤ 2kn ,
k
X
l=0

ρ−l
n φl =

m
X
l=0

φl −

m
X

(1 − ρ−l
n )φl +

l=0

S6

k
X

l=m+1

ρ−l
n φl .

j. a. duffy
Therefore, since |ρkn | ≤ 1,
ak (ρn ) −

ρkn

m
X

m
X

|1

m0
X

φl ≥

φl ≤

l=0

− ρ−l
n ||φl | +

l=0

k
X

|φl |

l=m+1

Let m0 be chosen such that both
ρkn

m0
X

φl ≥ C1

l=0

for all n ≥ n0 , and
such that

P∞

l=m0 +1 |φl |

|ak (ρn )| ≥ ρkn

l=0

C1
|φ| =: 3a
2

≤ a. Since ρ−l
n → 1 for each l, there exists an n1 ≥ n0

m0
X

φl −

m0
k
X
X
|φl | ≥ a
|1 − ρ−l
||φ
|
−
l
n
l=m0 +1

l=0

l=0

for all n ≥ n1 . Taking k0 := 2m0 and re-designating n1 as n0 gives the claimed lower
bound in (C.10).
Finally, since a0 = φ0 is nonzero by

DGP3,

replacing a by a ∧ |φ0 | yields a lower bound

that also applies to |a0 |.
Proof of Lemma C.5. Since ψε ∈ L1 , ε0 has a bounded continuous density. Thus by the
Riemann-Lebesgue lemma (Feller, 1971, Lem. XV.3.3) lim sup|λ|→∞ |ψε (λ)| = 0. Further, ψε ∈ L1 cannot be periodic, and so |ψε (λ)| < 1 for all λ 6= 0 (Feller, 1971,
Lem. XV.1.4); since ψε is necessarily continuous (Feller, 1971, Lem. XV.1.1), it follows
that sup|λ|≥1 |ψǫ (λ)| ≥ e−γ0 for some γ0 ∈ (0, ∞). By the moments theorem for characteristic functions (Feller, 1971, Lem. XV.4.2), we have ψε (λ) = 1 − 12 λ2 (1 + o(1)) as λ → 0.
2
Thus there exists a γ1 ∈ (0, ∞) such that |ψǫ (λ)| ≤ e−γ1 λ . Taking γ := γ0 ∧ γ1 thus gives
|ψǫ (λ)| ≤
Let ψϑ (λ) := E exp(iλ

P∞

k=1 ϑk εk )

( −γλ2
e
e−γ

=

if |λ| ∈ [0, 1],
if |λ| ≥ 1.

Q∞

k=1 ψǫ (ϑk λ);

(S.5)

we want to control the integral

of (the modulus of) this function over [A, ∞). Without loss of generality, assume the
coefficients {ϑk } are ordered such that |ϑi | ≥ |ϑi+1 |. Since
∞
X
3σ 2

ϑ

k=1

the set

π

∞

·k

−2

σ2
1X 2
= ϑ =
ϑk ,
2
2
k=1



3σϑ2 −2
2
·k
K := k ∈ N | ϑk ≥
π

must be nonempty; let k∗ denote the smallest element of K.

S7

uniformly valid inference in nonparmetric regression

We will bound the integral of |ψϑ | separately over each of the two r.h.s. sets in
−1
[A, ∞) = [A, A ∨ ϑ−1
k ∗ ] ∪ [A ∨ ϑk ∗ , ∞).

We first have
Z

{|λ|∈[A,A∨ϑ−1
]}
k∗

|ψϑ (λ)| dλ ≤

Z

Y

{|λ|∈[A,A∨ϑ−1
]} k∈K
k∗

≤(2)
≤(3)

|ψǫ (ϑk λ)| dλ
!

Z

exp −γλ2

Z

exp(−γλ2 σϑ2 /2) dλ

{|λ|≥A}

{|λ|≥A}

X

ϑ2k

k∈K

dλ

where ≤(2) follows from (S.5) and
|λ| ∈ [A, A ∨ ϑ−1
k ∗ ] =⇒ |ϑk ∗ λ| ≤ 1 =⇒ |ϑk λ| ≤ 1,

∀k ≥ k∗ ;

while ≤(3) follows from
X

k∈K

ϑ2k = σϑ2 −

X

ϑ2 ≥ σϑ2 −

k ∈K
/

3σϑ2 X −2 σϑ2
·
k ≥
.
π
2
k ∈K
/

Next, we have
∗

Z

,∞)}
{|λ|∈[A∨ϑ−1
k∗

|ψϑ (λ)| dλ ≤

Z

k
Y

,∞)} k=1
{|λ|∈[A∨ϑ−1
k∗
−γ(k ∗ −1)

≤(2) e

−γ(k ∗ −1)

≤e

Z

Z

ψǫ (ϑk λ) dλ

,∞)}
{|λ|∈[A∨ϑ−1
k∗

|ψǫ (ϑk∗ λ)| dλ

|ψǫ (ϑk∗ λ)| dλ

{|λ|≥A}

= e−γ(k

∗ −1)

ϑ−1
k∗

Z

|ψǫ (λ)| dλ

{|λ|≥ϑk∗ A}

−1 −γ(k ∗ −1) ∗
k
≤(5) c−1
0 σϑ e

Z

|ψǫ (λ)| dλ,

{|λ|≥c0 σϑ A/k ∗ }

for c0 := (3/π)1/2 , where ≤(2) holds trivially if k∗ = 1, and otherwise follows from
∗
|λ| ∈ [A ∨ ϑ−1
k ∗ , ∞) =⇒ |ϑk λ| ≥ 1 =⇒ |ϑk λ| ≥ 1,

while ≤(5) follows from ϑ2k∗ ≥ (3σϑ2 /π) · (k∗ )−2 .
Finally, define
G(A; σ 2 , ψǫ )

S8

∀k ≤ k∗ ;

j. a. duffy

:=

Z

{|λ|≥A}

−1
exp(−γλ2 σ 2 /2) dλ + c−1
sup e−γ(k−1) k
0 σ
k≥1

Z

|ψǫ (λ)| dλ,

{|λ|≥c0 σA/k}

which clearly satisfies the first inequality in (C.11), and is decreasing in σ 2 ; the second
inequality in (C.11) follows by evaluating G(0; σ 2 , ψǫ ), and noting supk≥1 e−γ(k−1) k < ∞.
It thus remains to show that G(A; σ 2 , ψǫ ) → 0 as A → ∞. To that end, let ǫ > 0 and note
that there exists a k′ such that
−γ(k ′ −1) ′

e

k

Since
−γ(k−1)

e

k

Z

Z

|ψǫ (λ)| dλ < ǫ.
R

|ψǫ (λ)| dλ → 0
{|λ|≥c0 σA/k}

as A → ∞, for each fixed k ∈ {1, . . . , k ′ }, the claim follows.
Proof of Lemma C.6. Making the change of variables u = ρx , we have
Z

a
1

1
1
dx =
x
1/2
− log ρ
(1 − ρ )

Z

ρ
ρa

iρ
1 h
1
−1
1/2
du
=
.
−2
tanh
{(1
−
u)
}
− log ρ
ρa
(1 − u)1/2 u

for ρ ∈ (0, 1), where tanh−1 (x) :=

1
2

log{(1 + x)/(1 − x)} is inverse hyperbolic tangent

function. Now set ρ = ρn for {ρn } ∈ RMI and a = nη: and note that ρn → 1, whereas
ρηn
n → 0 by Lemma C.1. Then
1
n

Z

ηn
1

n
o
1
1
1
−1
ηn 1/2
dx
=
2
tanh
[(1
−
ρ
)
]
+
o(1)
·
n
n − log ρn
(1 − ρxn )1/2
1/2 ]
1 log[1 − (1 − ρηn
n )
∼ ·
.
n
log ρn

Next, note that by two applications of L’Hï¿œpital’s rule
1
(1 − x)−1/2 /[1 − (1 − x)1/2 ]
log[1 − (1 − x)1/2 ]
= lim 2
x→0
x→0
log x
1/x
x
1
1
1
= lim
= 1,
= lim 1
1/2
2 x→0 1 − (1 − x)
2 x→0 2 (1 − x)−1/2

lim

whence

1/2 ]
1 log(ρηn
1 log[1 − (1 − ρηn
n )
n )
·
∼ ·
=η
n
log ρn
n log ρn

and the result follows.

S.4

Proofs of auxiliary lemmas from Appendix D

The proof of Lemma D.1 requires the following two results, which here play the role of
Lemmas 7.4 and 9.3 in Duffy (2016); the proofs appear in Section S.5. Recall the definition
of kn = kn ({ρn }) given in (C.9); throughout the following, n0 and k0 are as in Lemma C.4.

S9

uniformly valid inference in nonparmetric regression

Lemma S.1. Suppose f ∈ BI and β ∈ (0, β). There exists a C < ∞ such that
2
f k∞
kξk,t

+

n−k−t
X

2
2
(β, f )
f k∞ ≤ Cσn,k
kEt ξk,t+s

(S.6)

s=1

when k ∈ {0, . . . , k0 − 1}, and
(n−k−t)∧kn
2
f k∞
kξk,t

X

+

2
2
(β, f )
f k∞ ≤ Cn−1 kn σn,k
kEt ξk,t+s

(S.7)

s=1

when k ∈ {k0 , . . . , kn − 1}, for all n ≥ n0 , 1 ≤ t ≤ n − k and f ∈ BI[β] .
Lemma S.2. Suppose f ∈ BI. There exists a C < ∞ such that
(i) for every t ≥ 0 and k0 ≤ k ≤ n − t,
Et |f (xt+k )| ≤ C(k ∧ kn )−1/2 kf k1 ;
(ii) if in addition f ∈ BI[β] for some β ∈ (0, 1], then for every t ≥ 0 and k0 ≤ k ≤ n − t,
i
h
|Et f (xt+k )| ≤ C (k ∧ kn )−(1+β)/2 kf k[β] + e−γ1 (k∧kn ) kf k1 .
Proof of Lemma D.1. By Lemma S.2(ii) and kn . n,


kX
0 −1

|Nn f | ≤ 

+

t=1

kn
X

+

t=k0

n
X



|E[t−kn ] f (xt )|
+

t=kn +1

−γ1 kn

. kf k∞ + ne

kf k1 + nkn−(1+β)/2 kf k[β]

−(1+β)/2

whence (D.6), noting that ne−γ1 kn . nkn

−(1+β)

≍ ndn

= en d−β
n . Regarding (D.7),

in view of Lemma 7.2 in Duffy (2016) it suffices to show
2
kUn,k f kp ∨ kVn,k f kp ≤ C(p!)1/p σn,k
(β, f )

(S.8)

for every p ∈ N. To prove (S.8), consider decomposing Vn,k into L blocks, as per
Vn,k f =

nl
L
X
X

2
f
Et−1 ξk,t

l=1 t=nl−1

=:

L
X

Vn,k,l f

l=1

for endpoints 0 ≤ n0 ≤ · · · ≤ nL ≤ n − k. For the lth block, we have
p

E|Vn,k,l f | ≤ p! ·

nl
X

t1 =nl−1 +1

···

nl
X

nl
X

tp−1 =tp−2 tp =tp−1

i
h
2
2
2
f
)
.
E Et1 −1 (ξk,t
f
)
·
·
·
E
(ξ
f
)
·
E
(ξ
t
−1
t
−1
p
k,tp−1
k,tp
p−1
1
S10

j. a. duffy

By the law of iterated expectations, and separately treating the cases where tp = tp−1 and
tp > tp−1 , we obtain the bound
E|Vn,k,l f |p ≤ p! ·

nl
X

nl
X

···

tp−1 =tp−2

t1 =nl−1 +1



h
i
2
2
E Et1 −1 (ξk,t
f
)
·
·
·
E
(ξ
f
)
tp−1 −1 k,tp−1
1
nl −tp−1

2
f k∞ +
· kξk,t
p−1

X
s=1



2
k ; (S.9)
kEtp−1 −1 ξk,t
p−1 +s ∞

2
2
k , suitable bounds for the final term (in
k ≤ kEtp−1 ξk,t
Since kEtp−1 −1 ξk,t
p−1 +s ∞
p−1 +s ∞

parentheses) are provided by Lemma S.1. In particular, when k ∈ {0, . . . , k0 − 1}, we may
take n0 = 0 and n1 = n − k, so that (S.6) immediately yields
2p
E|Vn,k f |p ≤ p! · C p σn,k
(β, f ).

When k ∈ {k0 , . . . , kn − 1}, we set nl := kn l ∧ (n − k), with L = Ln chosen to be the
smallest integer such that kn Ln ≥ n − k. Then applying (S.7) to (S.9) gives
2p
E|Vn,k,l f |p ≤ p! · C p (n−1 kn )p σn,k
(β, f )

for l ∈ {1, . . . , Ln }, and
kVn,k f kp ≤

Ln
X
2
kVn,k,l f kp ≤ C(p!)1/p σn,k
(β, f )
l=1

since Ln . nkn−1 . Thus Vn,k f satisfies (S.8); an analogous argument establishes that this
is also true of Un,k f .
Proof of Lemma D.2. This follows exactly as per the proof of Lemma 7.5 in Duffy (2016):
we need only to note that, in the present case,
e1/2
n

kn
X

k

−(3+2β)/4

.

en1/2 kn1/4

k=0

1/4

since kn

S.5

1/2

≍ dn

kn
X

k−1−β/2 . en1/2 d1/2−β
. en d−β
n
n ,

k=0

1/2

. en .

Proofs of Lemmas S.1–S.2

We shall need the following results, whose proofs appear at the end of this section. We
first recall the following useful inequality, from Lemma 9.1(i) in Duffy (2016):
|fˆ(λ)| ≤ (|λ|β kf k[β] ) ∧ kf k1

S11

(S.10)

uniformly valid inference in nonparmetric regression
for every β ∈ (0, 1] and f ∈ BI[β] ; recall fˆ denotes the Fourier transform of f . Let
ϑ(z1 , z2 ) := E[e−iz1 ǫ0 − Ee−iz1 ǫ0 ][e−iz2 ǫ0 − Ee−iz2 ǫ0 ].
Lemma S.3. There exists a C < ∞ such that for every z1 , z2 ∈ R,
|ϑ(z1 , z2 )| ≤ C[|z1 |2 ∧ 1]1/2 [|z2 |2 ∧ 1]1/2 .
Lemma S.4. Suppose {ρn } ∈ RMI ∪ RLU , and let n0 and k0 be as in the statement of
Lemma C.4. There exists a γ ∈ (0, ∞) such that
sup

max

n≥n0 k0 /2≤k≤2kn

|ψε [ak (ρn )λ]| ≤

( −γλ2
e
−γ

e

if |λ| ≤ 1,
if |λ| ≥ 1.

Lemma S.5. There exists a γ1 > 0 and a C < ∞ such that, for every p ∈ [0, 5], z1 , z2 ∈
R+ , and k0 ≤ k ≤ 2kn ,
Z

(z1 |λ|p ∧ z2 )

R

Y

l∈K

i
h
|ψε [al (ρn )λ]| dλ ≤ C z1 k−(p+1)/2 + z2 e−γ1 k

uniformly over all K ⊆ {⌊k/2⌋, . . . , k} with #K ≥ k/4.
Corollary S.1. There exists a γ1 > 0 and a C < ∞ such that
(i) for every p ∈ [0, 5], z1 , z2 ∈ R+ , k0 ≤ k ≤ n and 1 ≤ t ≤ n − k,
Z

R

i
h
(z1 |λ|p ∧ z2 )|Ee−iλxt+1,t+k,t+k | dλ ≤ C z1 (k ∧ kn )−(p+1)/2 + z2 e−γ1 (k∧kn ) ;

(ii) for every k0 ≤ k ≤ kn , 1 ≤ t ≤ n − k and 2 ≤ s ≤ t,
Z

|Ee−iλxt−s+1,t−1,t+k | dλ ≤ C(s ∧ kn )−1/2 .

R

Proof of Lemma S.1. The argument is similar to that used to prove Lemma 7.4 in Duffy
2 fk
2
(2016). We first suppose that k ∈ {0, . . . , k0 − 1}. Trivially, kξk,t
∞ ≤ Ckf k∞ , while by

Jensen’s inequality and Lemma S.2(i),
2
|Et ξk,t+s
| ≤ CEt f 2 (xt+s+k ) ≤ C1

Hence, noting that
2
f k∞
kξk,t

+

Pk n

s=1 s

−1/2

1/2

. kn

(
kf k2∞

(s ∧ kn )

kf k22

if s ≥ k0 .

−1/2

−1/2

≍ nd−1
n = en ,

. nkn

n−k−t
X
s=1

if 1 ≤ s ≤ k0 − 1,
−1/2

and nkn

i
h


2
k∞ ≤ C kf k2∞ + nkn−1/2 kf k22 ≤ C1 kf k∞ + en kf k22 ,
kEt ξk,t+s

as required for (S.6).
S12

j. a. duffy

It remains to consider the case where k ∈ {k0 , . . . , kn − 1}. We shall obtain a bound
2 f – for t ∈ {2, . . . , n − k} and s ∈ {1, . . . , k ∧ (t − 1)} – which depends on k
for Et−s ξk,t
n
2
f in (S.7).
and s but not t, thus permitting us to deduce the required bound for Et ξk,t+s

As per (C.2) and (C.3) above, decompose
xt+k = x−∞,0,t+k + x1,t−1,t+k + ak εt + xt+1,t+k,t+k ,
so that by Fourier inversion
ξk,tf = Et f (xt+k ) − Et−1 f (xt+k )
Z
i
h
1
fˆ(λ)e−iλx−∞,0,t+k e−iλx1,t−1,t+k e−iλak εt − Ee−iλak εt Ee−iλxt+1,t+k,t+k dλ,
=
2π R
whence
2
f=
ξk,t

1
(2π)2

ZZ

fˆ(λ1 )fˆ(λ2 )e−i(λ1 +λ2 )x−∞,0,t+k e−i(λ1 +λ2 )x1,t−1,t+k
i
ih
h
· e−iλ1 ak εt − Ee−iλ1 ak εt e−iλ2 ak εt − Ee−iλ2 ak εt

R2

(S.11)

· Ee−iλ1 xt+1,t+k,t+k Ee−iλ2 xt+1,t+k,t+k dλ1 dλ2 .

Since 1 ≤ s ≤ t − 1, making the further decomposition
x1,t−1,t+k = x1,t−s,t+k + xt−s+1,t−1,t+k
and taking conditional expectations on both sides of (S.11) gives
2
f
Et−s ξk,t

1
=
(2π)2

ZZ

R2

fˆ(λ1 )fˆ(λ2 )e−i(λ1 +λ2 )x−∞,0,t+k e−i(λ1 +λ2 )x1,t−s,t+k
· Ee−i(λ1 +λ2 )xt−s+1,t−1,t+k · ϑ(λ1 ak , λ2 ak )
· Ee−iλ1 xt+1,t+k,t+k Ee−iλ2 xt+1,t+k,t+k dλ1 dλ2 ,

where ϑ(z1 , z2 ) := E[e−iz1 ǫ0 − Ee−iz1ǫ0 ][e−iz2 ǫ0 − Ee−iz2 ǫ0 ]. Thus by (S.10) and Lemmas C.4
and S.3, there exist C, C1 < ∞ such that
2
f
Et−s ξk,t

≤C

ZZ

R2

|fˆ(λ1 )fˆ(λ2 )|[|λ1 |2 ∧ 1]1/2 [|λ2 |2 ∧ 1]1/2
· |Ee−i(λ1 +λ2 )xt−s+1,t−1,t+k |
· |Ee−iλ1 xt+1,t+k,t+k ||Ee−iλ2 xt+1,t+k,t+k | dλ1 dλ2

≤ C1

Z

R

|fˆ(λ1 )|2 (|λ1 |2 ∧ 1)|Ee−iλ1 xt+1,t+k,t+k |
Z
· |Ee−i(λ1 +λ2 )xt−s+1,t−1,t+k | dλ2 dλ1

(S.12)

R

where we have used |ab| ≤ |a|2 + |b|2 , and appealed to symmetry (in λ1 and λ2 ) to obtain

S13

uniformly valid inference in nonparmetric regression

the final bound. Now by Corollary S.1(ii), and recalling that k ≤ kn and s ≤ kn ,
Z

−i(λ1 +λ2 )xt−s+1,t−1,t+k

|Ee

| dλ2 =

Z

|Ee−iλxt−s+1,t−1,t+k | dλ ≤ Cs−1/2 ,

(S.13)

R

R

while (S.10) and Corollary S.1(i) give
Z

|fˆ(λ)|2 (|λ|2 ∧ 1)|Ee−iλxt+1,t+k,t+k | dλ
R
Z
≤ [(|λ|2(1+β) kf k2[β] ) ∧ kf k21 ]|Ee−iλxt+1,t+k,t+k | dλ
R
i
h
≤ C k−(3+2β)/2 kf k2[β] + e−γ1 k kf k21 .

(S.14)

Together, (S.12)–(S.14) yield


2
f ≤ Cs−1/2 k−(3+2β)/2 kf k2[β] + e−γ1 k kf k21 ,
Et−s ξk,t

2
which does not depend on t, and thus applies also to kEt ξk,t+s
k∞ . Hence
(n−k−t)∧kn

X
s=1



2
kEt ξk,t+s
k∞ ≤ Ckn1/2 k−(3+2β)/2 kf k2[β] + e−γ1 k kf k21 .

(S.15)

2 f k . Returning to (S.11), we have by (S.10) and CorolWe come finally to kξk,t
∞

lary S.1(i) that
2
kξk,t
f k∞

≤C

Z

′
|fˆ(λ)||Ee−iλxt+1,t+k,t+k | dλ

R

≤ C1

Z

2

−iλx′t+1,t+k,t+k

β

[(|λ| kf k[β] ) ∧ kf k1 ]|Ee

R

2

≤ C2 k−(1+β)/2 kf k[β] + e−γ1 k kf k1

≤ C3 k−(1+β) kf k2[β] + e−γ1 k kf k21


≤ C4 kn1/2 k−(3+2β)/2 kf k2[β] + e−γ1 k kf k21 ,

| dλ

2

(S.16)

where the final bound follows because k ≤ kn . The result now follows from (S.15) and
(S.16), and the fact that
kn1/2 = (n−1 kn )nkn−1/2 ≍ (n−1 kn )en .
Proof of Lemma S.2. Exactly as in the proof of Lemma 9.3 in Duffy (2016), for f ∈ BI
Et |f (xt+k )| ≤ Ckf k1

Z

|Ee−iλxt+1,t+k,t+k | dλ,
R

S14

j. a. duffy

while for f ∈ BI[β] ,
|Et f (xt+k )| ≤ C

Z

[(kf k[β] |λ|β ) ∧ kf k1 ]|Ee−iλxt+1,t+k,t+k | dλ,

R

whereupon both parts follow by Corollary S.1(i).
Proof of Lemma S.3. Exactly as in the proof of Lemma 9.4 in Duffy (2016),
E|e−iλε0 − Ee−iλε0 |2 ≤ CE[|λε0 |2 ∧ 1] ≤ C1 |λ|2
since Eε20 < ∞. The result now follows by noting that the l.h.s. is also bounded by 4, and
applying the Cauchy-Schwarz inequality.
Proof of Lemma S.4. This follows from Lemma C.4 and the same arguments that led to
(S.5).
Proof of Lemma S.5. Let h(λ) := z1 |λ|p ∧ z2 and K := #K. By Hï¿œlder’s inequality,
Z

h(λ)

R

Y

|ψε [al (ρn )λ]| dλ ≤

l∈K

Y Z

l∈K

1/K
h(λ)|ψε [al (ρn )λ]| dλ
K

R

Z

≤ max h(λ)|ψε [al (ρn )λ]|K dλ
l∈K R
Z
h(λ) max |ψε [al (ρn )λ]|K dλ.
≤
k0 /2≤l≤2kn

R

Further, by Lemma S.4, the preceding is bounded by
z1

Z

2K

|λ|p e−γλ
R

i
h
dλ + z2 e−γK kψε k1 ≤ C z1 K −(p+1)/2 + z2 e−γK .

Since K ≥ k/4, the result follows.
Proof of Corollary S.1. Since
k−1
X

xt+1,t+k,t+k =

al εt+k−l

xt−s+1,t−1,t+k =

l=0

|Ee

al εt+k−l ,

l=k+1

we have
−iλxt+1,t+k,t+k

k+s−1
X

k∧k
n −1
Y

|≤

|ψε (al (ρn )λ)|

l=⌊(k∧kn )/2⌋+1

and so part (i) follows immediately from Lemma S.5. For part (ii), we note that
−iλxt−s+1,t−1,t+k

|Ee

|≤

k−1+s∧k
Y n
l=k+1

S15

|ψε (al (ρn )λ)|,

uniformly valid inference in nonparmetric regression

where k−1+s∧kn ≤ 2kn , since k ≤ kn . Thus when s ≥ k0 , the required bound also follows
from Lemma S.5. When s < k0 , the crude bound |Ee−iλxt−s+1,t−1,t+k | ≤ |ψε (ak+1 (ρn )λ)|
suffices, in view of ψε ∈ L1 and Lemma C.4.

S16

j. a. duffy

S.6

Index of notation

Greek and Roman symbols
Listed in (Roman) alphabetical order. Greek symbols are listed according to their English
names: thus ω, as ‘omega’, appears before θ, as ‘theta’.
at,k

coefficient sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(C.1)

ak

equals at,k for 0 ≤ k ≤ t − 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(C.1)

BI

bounded and integrable functions on R . . . . . . . . . . . . . . . . . . .

App. A

BI[β]

f ∈ BI with kf k[β] < ∞ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

App. D

BL

bounded and Lipschitz functions on R . . . . . . . . . . . . . . . . . . . .

Sec. 1

C, C1

generic constants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

App. A

Cn

confidence set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(2.7)

dn

equals var(xn )1/2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(3.10)

dn,s,t

standardising constants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

WP3

δn (β, F )

bound on function class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

App. D

εt

innovation sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

DGP2

en

norming sequence, equals nd−1
...........................
n
t
..........................
expectation conditional on F−∞

App. A

Et

App. C.2

η
fˆ

mixing variate in limiting variance . . . . . . . . . . . . . . . . . . . . . . .

(A.3)

Fourier transform of f . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

App. D

fǫ

Lebesgue density of εt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

DGP2

F̂n,i

non-predictability test statistic . . . . . . . . . . . . . . . . . . . . . . . . . . .

(2.13)

Fst

σ({εr }tr=s ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

App. C.1

Gt

σ({xs , us }s≤t ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

DGP4

(ψε , {φk }, σu2 , {Fut })

γ

nuisance parameters

.................

Sec. 2.3

Γ

parameter space for γ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Sec. 2.3

h, hn

bandwidth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(2.3)

hn , hn

upper and lower bounds on bandwidth . . . . . . . . . . . . . . . . . . .

H

probability density . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

WP3

subset of parameters consistent with null hypothesis . . . . . .

Sec. 2.3

k0

index to coefficient sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Lem. C.4

kn

real sequence related to ρn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(C.9)

hn,s,t
H,H

∗

h−1 K(h−1 u)

K, Kh

smoothing kernel, Kh (x) :=

.................

(2.3)

Jc

normalised OU process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(3.7)

ℓucc (R)

bounded functions with ucc topology . . . . . . . . . . . . . . . . . . . . .

Sec. 3.3

Lp

Lebesgue p-integrable functions on R . . . . . . . . . . . . . . . . . . . . .

Sec. 1

Lc

local time of Jc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(3.6)

S17

uniformly valid inference in nonparmetric regression

m

regression function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(2.1)

m̂n

local level estimate of m . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(2.3)

M

class of allowable regression functions . . . . . . . . . . . . . . . . . . . .

DGP1

Mn,k f

martingale components in decomposition of Sn f . . . . . . . . . .

(D.2)

µ

limiting spatial density under R . . . . . . . . . . . . . . . . . . . . . . . . . .

(3.6)

µ̃

generic limiting spatial density . . . . . . . . . . . . . . . . . . . . . . . . . . .

(3.2)

µn

spatial density estimate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(3.8)

Nn f

remainder from decomposition of Sn f . . . . . . . . . . . . . . . . . . . .

(D.2)

νρ

limiting stationary density . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(3.6)

ωn2

scaling factor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(3.11)

Ωn

collection of indices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Sec. 3.1

φ

sum of the φk ’s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

DGP3

φk

coefficients defining the linear process vt . . . . . . . . . . . . . . . . . .

(2.2)

ϕ

standard Gaussian density . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(3.6)

ψε

characteristic function of ε0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

DGP2

ρ

autoregressive parameter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(2.2)

R

parameter space for ρ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

DGP3

R

RST ∪ RMI ∪ RLU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(2.12)

RST

stationary sequences in R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Rem. 2.5

RMI

mildly integrated sequences in R . . . . . . . . . . . . . . . . . . . . . . . . .

Rem. 2.5

RLU

local-to-unity sequences in R . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Rem. 2.5

s2n

asymptotic variance estimator . . . . . . . . . . . . . . . . . . . . . . . . . . . .
P
summation operator, Sn f := nt=1 f (xt ) . . . . . . . . . . . . . . . . . .

(2.6)

stationary variance at ρ < 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

App. A

(conditional) variance of ut . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

DGP4

estimate of σu2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(2.6)

bound related to function f . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

App. D

bound related to function f . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

App. D

t̂n

t-statistic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(2.5)

τ (x)

limit of density estimator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

App. A

τ1 , τ 3/2

convex and increasing functions . . . . . . . . . . . . . . . . . . . . . . . . . .

(D.5)

θ

hypothesised value of m(x) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Sec. 2.3

θ̂n

sample mean of yt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Sec. 2.3

ut

regression disturbance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(2.1)

Un,k f

squared variation of Mn,k f . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(D.3)

υn

martingale component of t̂n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(2.11)

Sn
σρ2
σu2
σ̂u2
2 (β, f )
σn,k
ςn2 (β, f )

S18

(D.2)

j. a. duffy

vt

linear process built from {εt } . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(2.2)

Vn,k f

conditional variance of Mn,k f . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(D.3)

W

standard Brownian motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(3.7)

xt

regressor, partial sum of {vt } . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(2.2)

x̃n,t

standardised regressor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(3.10)

xs,r,t

component of xt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(C.3)

Xn

standardised regressor process . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Rem. 3.5

X

finite-dimensional limit of Xn . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Rem. 3.5

X

subset of R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Sec. 2.3

ξkt f

martingale difference components of Mnk f . . . . . . . . . . . . . . .

(D.1)

yt

dependent variable in regression . . . . . . . . . . . . . . . . . . . . . . . . . .

(2.1)

Symbols not connected to Greek or Roman letters
Ordered alphabetically by their description.
=d
p

→

both sides have the same distribution
converges in probability to
finite-dimensional convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Sec. 1

⌊·⌋

floor function (integer part) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Sec. 1

kf k[β]

Fourier norm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(D.4)

.p

l.h.s. bounded in probability by the r.h.s. . . . . . . . . . . . . . . . .

Sec. 1

fdd

(an .p bn if an = Op (bn ))
.

l.h.s of no greater order than the r.h.s. . . . . . . . . . . . . . . . . . . .

Sec. 1

(an . bn if an = O(bn ))
kf kLip
kf kp

App. C.2

Lipschitz norm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
R
Lp norm, ( |f |p )1/p , for function f . . . . . . . . . . . . . . . . . . . . . . .

App. A

Lp norm, (E|X|p )1/p , for random variable X . . . . . . . . . . . . . .

App. A

denotes supx∈R |f (x)| when p = ∞

kXkp

denotes essential supremum when p = ∞
∼

strong asymptotic equivalence . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Sec. 1

(an ∼ bn if limn→∞ an /bn = 1)
kF k

supremum of norm k·k over F : supf ∈F kf k . . . . . . . . . . . . . .

App. D

[a(x)]x∈X

vector (a(x1 ), . . . , a(xm ))′ , for {x1 , . . . xm } = X . . . . . . . . . .

Sec. 2.3

≍

weak asymptotic equivalence . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Sec. 1

(an ≍ bn if limn→∞ an /bn ∈ (−∞, ∞)\{0})
weak convergence (van der Vaart and Wellner, 1996) . . . . .

S19

Sec. 1

