arXiv:1707.02555v3 [] 26 Feb 2018

Asymptotic Theory for the Maximum of an Increasing
Sequence of Parametric Functions
Jonathan B. Hill ∗
University of North Carolina
February 27, 2018

Abstract
Hill and Motegi (2017) present a new general asymptotic theory for the maximum of a
random array {Xn (i) : 1 ≤ i ≤ L}n≥1 , where each Xn (i) is assumed to converge in probability
as n → ∞. The array dimension L is allowed to increase with the sample size n. Existing
extreme value theory arguments focus on observed data Xn (i), and require a well defined
limit law for max1≤i≤L |Xn (i)| by restricting dependence across i. The high dimensional central limit theory literature presumes approximability by a Gaussian law, and also restricts
attention to observed data. Hill and Motegi (2017) do not require max1≤i≤Ln |Xn (i)| to have
a well defined limit nor be approximable by a Gaussian random variable, and we do not make
any assumptions about dependence across i. We apply the theory to filtered data when the
variable of interest Xn (i, θ 0 ) is not observed, but its sample counterpart Xn (i, θ̂ n ) is observed
where θ̂ n estimates θ0 . The main results are illustrated by looking at unit root tests for a
high dimensional random variable, and a residuals white noise test.
Keywords : maximum of multivariate mean, high dimensionality, non-Gaussian approximation.
MSC2010 subject classifications : 62E17, 62F40, 62M10, 05D10.

1

Introduction

Consider an array of random variables on a probability measure space (Ω, F , P):
{Xn (i), Yn (i) : 1 ≤ i ≤ In }n≥1 ,
where {In }n≥1 is a sequence of positive integers, In → ∞ as n → ∞. Under the assumption
p

that Xn (i) → 0 as n → ∞ for each i, Hill and Motegi (2017) prove for some sequence of positive
∗

Department of Economics, University of North Carolina at Chapel Hill; jbhill@email.unc.edu.

1

integers {Ln } that satisfies Ln → ∞:
p

max |Xn (i)| → 0.

(1)

1≤i≤Ln

By operating on (1), for any two arrays of F -measurable random variables {Xn (i), Yn (i) : 1 ≤ i
p

≤ In }n≥1 , if Xn (i) − Yn (i) → 0 for each i Hill and Motegi (2017) then show that the difference
in maxima satisfy:
p

max |Xn (i)| − max |Yn (i)| → 0.

1≤i≤Ln

1≤i≤Ln

(2)

P
√ P
Examples are a sample mean Xn (i) = 1/n nt=1 xt (i) or standardized means Xn (i) = 1/ n nt=1 xt (i)
√ P
and Yn (i) = 1/ n nt=1 yt (i), where {xt (i), yt (i)}nt=1 are the sample paths of some processes on
(Ω, F , P). This has been studied extensively in the Gaussian approximation and high dimensional

Gaussian central limit theory literatures, discussed below. At the highest level of generality we
work with maxima of positive values in order to exploit convenient inequalities.
The focus of the present paper is to extend the key ideas of Hill and Motegi (2017) to filtered
residuals, and to apply the result to a unit root test and a white noise test. The authors do not
impose any restrictions on dependence in (Xn (i), Yn (i)) across coordinates i, nor do we require
(Xn (i), Yn (i)) to belong to a specific domain of attraction. For example, in the normalized mean
case we neither require either (Xn (i), Yn (i)) to be Gaussian nor converge in law to Gaussian random variables. The generality arises from a new result for convergence of high dimensional arrays
which does not rely on probabilistic properties, although naturally lends itself to probabilistic
applications.
The approach differs from standard weak convergence methods when applied to an array of
√ P
standardized means {Xn (i) : 1 ≤ i ≤ In }n≥1 , e.g. Xn (i) = 1/ n nt=1 xt (i). Weak convergence

for {Xn (i) : i ∈ N} in the broad sense of Hoffmann-Jorgensen (1984, 1991) is a potential option.
It is known that such weak convergence to a Gaussian limit, with a versin that has uniformly
bounded and uniformly continuous sample paths, is equivelant to pointwise convergence and
the existence of a pseudo metric d on N such that (N, d) is a totally bounded pseudo metric

2

space and a stochastic equicontinuity property based on d holds. See Dudley (1978, 1984) and
Pollard (1990, Chapters 9-10). If d is the Euclidean distance, for example, then (N, d) is not
totally bounded. Hill and Motegi (2017) take a different approach that completely sidesteps the
approach of Hoffmann-Jorgensen (1984, 1991) by first demonstrating pointwise convergence Xn (i)
d

→ X (i). They then work with a probability construction for the array {Xn (i) − X (i) : 1 ≤ i ≤
In }n≥1 and apply our general result for convergence of arrays to be able to show | max1≤i≤Ln |Xn (i)|
p

− max1≤i≤Ln |X (i)|| → 0.
The primary tools used here date in some form to seminal theory developed by Ramsey (1930)
and its implications for monotone subsequences and convergence, cf. the Erdős-Szekeres theorem
(Erdös and Szekeres, 1935). Hill and Motegi (2017) significantly augment a convergence result
for non-stochastic arrays presented in Boehme and Rosenfeld (1974, Lemma 1) in key ways. The
latter claim that if an array {Ak,n : 1 ≤ k ≤ In }n≥1 , where In → ∞ as n → ∞, lies in a first
countable topological space, and limk→∞ limn→∞ Ak,n = 0, then liml→∞ AL(nl ),nl = 0 for some
infinite subsequence {nl }∞
l=1 of positive integers, and some mapping L(nl ) → ∞. (Recall that
any metric space is a first countable topological space.) In order to use the result for maxima,
we extend {nl }∞
l=1 to N and therefore achieve limn→∞ AL(n),n = 0 in Lemma 2.1. We require
additional assumptions that lends itself to deriving (1) and (2). A practical application includes
R1
p
when Ak,n = 0 P (1 − e− max1≤i≤k |Xn (i)| > ǫ)dǫ, the foundation for showing max1≤i≤Ln |Xn (i)| →
0 in the proof of Theorem 2.2 below.

√ P
Chernozhukov, Chetverikov, and Kato (2013a, 2016) work with normalized sample means 1/ n nt=1 xt (i)

where {xt (i)}nt=1 are independent, zero mean and square integrable random variables. They by-

pass extreme value theoretic arguments and therefore do not need to restrict dependence across
i. This is accomplished by developing new tools for deriving Gaussian approximations based on
Slepian and Sudakov-Fernique methods. They prove the impressive result that for some (K, ζ) >
c

0, and Ln → ∞ with Ln = O(eo(n ) ) for some c > 0:
n

ALn,n ≡ sup P
c≥0

1 X
xt (i) ≤ c
max √
1≤i≤Ln
n t=1

!

n

−P

3

1 X
max √
yt (i) ≤ c
1≤i≤Ln
n t=1

!

≤ Kn−ζ

(3)

where yt (i) are zero mean normally distributed with covariance function E[yt (i)yt (j)] =
P
1/n nt=1 E[xt (i)xt (j)].

The literature on such Gaussian couplings has a substantial history, where generally Ln =

O(nc ) for some c. See, for example, Pollard (2002, Chapter 10) for a general review, see Yurinskii
(1977) for a seminal result, and see Dudley and Philipp (1983, Lemma 2.12) and Le Cam (1988).
See also Portnoy (1985, 1986), Götze (1991) and Chernozhukov, Chetverikov, and Kato (2013b,
Appendix L) for theory and references on high dimensional Gaussian central limit theory.
In the above literature, Gaussianicity is key, which therefore neglects non-standard asymptotics, including heavy tailed data or non-stationary data. It also does not apply in general when
working with filtered data: in this case an intermediate step is required linking two sequences in
which neither may be Gaussian. As such, it does not include cases where a filter leads to nonstandard asymptotics, including with some parameters are weakly or non-identified and when a
parameter boundary value occurs (e.g. Andrews, 1999, Andrews and Cheng, 2012).
Consider, for example, random functions xt (i, θ0 (i)) where θ0 (i) is not observed but estimable,
√ P
write Xn (i, θ0 (i)) ≡ 1/ n nt=1 xt (i, θ0 (i)), and let θ̂ n (i) be a plug-in estimator for θ0 (i). Write

Xn (i) ≡ Xn (i, θ̂n (i)). In order to gain inference on max1≤i≤Ln |Xn (i)| we first require an asymptotic
expansion Xn (i) = Yn (i) + op (1) for some process {Yn (i)} that ostensibly depends on θ0 (i) and
p

pre-asymptotic properties of θ̂n (i), leading to | max1≤i≤Ln |Xn (i)| − max1≤i≤Ln |Yn (i)|| → 0 as
in (2). Even if Yn (i) is asymptotically normal, it need not be Gaussian, hence the Gaussian
approximation literature does not apply. Of course, Yn (i) need not be asymptotically normal for
(2) to apply. In Section 4 we illustrate non-standard asymptotics by allowing for unit root nonstationarity, and we treat a white noise test for an unobserved regression error term to illustrate
the use of a filter.
The maximum of an increasing sequence of normalized sample covariances γ̂ n (i) ≡ 1/n

Pn

t=1

xt xt−i

has been studied at least since Berman (1964) and Hannan (1974). See Jirak (2011) and Xiao and Wu
(2014) for recent theory and references. In this literature xt is assumed observed, the exact
√
asymptotic distribution form of a suitably normalized n max1≤i≤Ln |γ̂ n (i) − E[xt xt−i ]| is sought,
√
and underlying assumptions ensure n(γ̂ n (i) − E[xt xt−i ]) converges in finite dimensional dis4

tributions to a Gaussian law Z(i). In turn, max1≤i≤Ln |Z(i)| must converge in law to a well
defined random variable, which requires asymptotic independence E[Z(i)Z(j)] → 0 as |i − j|
→ ∞. See, e.g., Leadbetter, Lindgren, and Rootzén (1983, Chapter 6), Hüsler (1986, 1993),
Homble and McCormick (1995) and Falk, Hüsler, and Reiss (2011, Chapter 9). We sidestep extreme value theoretic arguments, dependence across i is unrestricted, and residuals are allowed,
hence xt need not be observed in practice. See Section 4.
We cannot generally provide an upper bound on the divergence rate Ln → ∞, similar to ones
in Chernozhukov, Chetverikov, and Kato (2013a, 2016) in the Gaussian coupling literature, and
Xiao and Wu (2014) in the extreme value theory literature. This is an unavoidable cost for our
(i) basing probabilistic statements like (2) on a general array convergence result that itself does
not make use of probabilistic properties of (Gaussian) random variables; and (ii) allowing for
filtered data and therefore requiring asymptotic linkages between maxima that do not involve
Gaussian processes. Chernozhukov, Chetverikov, and Kato (2013a, 2015, 2016), however, appear
to have the sharpest result and most general bound on Ln → ∞ for Gaussian approximations (for
independent data).
We do not treat a bootstrap theory, for example for the sample mean maximum
√ P
max1≤i≤Ln |1/ n nt=1 xt (i)|, because by using our main results any existing bootstrap theory

will hold under its specified pointwise assumptions. See Hill and Motegi (2017).

The remaining sections are organized as follows. In Section 2 we present the main results (1)
and (2) of Hill and Motegi (2017). These are used in Section 3 for convergence of maxima when
filtered residuals are used. Examples are provided in Section 4, and concluding remarks are left
for Section 5.
In the following | · | and || · || are respectively the l1 - and l2 -matrix norms.

2

Main Results

All random variables are assumed to exist on a complete measure space, and probabilities where
applicable are outer probability measures. See, e.g., Dudley (1984, p. 101) and Pollard (1984,

5

Appendix C). See Dudley and Philipp (1983) for theory that sidesteps measurability challenges
specifically for normalized means.
The first result due to Hill and Motegi (2017) concerns convergence of arrays. It is based on,
and extends, a result from Boehme and Rosenfeld (1974, Lemma 1). They work in first countable
spaces. Because we require some structure on the space we work in, and because any metric space
is first countable, we simply work in (R, d) with metric d for ease of notation. All proofs are
placed in the appendix.
Lemma 2.1 Assume the array {Ak,n : 1 ≤ k ≤ In }n≥1 lies (R, d), where {In }n≥1 is a sequence
of positive integers, In → ∞ as n → ∞. Let limn→∞ Ak,n = 0 for each fixed k, and Ak,n ≤ Ak+1,n
for each n and all k. Then limn→∞ ALn,n = 0 for some sequence {Ln } of positive integers, Ln →
∞ and Ln ≤ In , that is not unique.
Remark 1 Boehme and Rosenfeld (1974) only require limk→∞ limn→∞ Ak,n = 0, and do not impose monotonicity, thus limn→∞ Ak,n = 0 ∀k and Ak,n ≤ Ak+1,n are stronger assumptions. The
first property, however, is key towards proving limn→∞ ALn,n = 0, rather than merely a subsequence liml→∞ ALnl ,nl = 0 as in Boehme and Rosenfeld (1974). Monotonicity is used to identify
Ln as a function only of n based on using a multiple subsequence argument. The maximum over a
subsequence of positive values satisfies monotonicity, and limn→∞ Ak,n = 0 ∀k holds when applied
to pointwise probability convergence problems discussed in the sequel.
Remark 2 Monotonicity implies {Ln } is not unique since limn→∞ AL̊n,n = 0 for any sequence
{L̊n } of positive integers with L̊n → ∞ and lim supn→∞ {L̊n /Ln } < 1.
The next result uses Lemma 2.1 as the basis for deriving (1) and (2).
Theorem 2.2 Let {Xn (i), Yn (i) : 1 ≤ i ≤ In }n≥1 be arrays of random variables, where {In }n≥1
is a sequence of positive integers, In → ∞ as n → ∞.
p

p

a. If Xn (i) → 0 for each fixed i, then max1≤i≤Ln |Xn (i)| → 0 for some sequence {Ln } of positive
integers with Ln → ∞, that is not unique.
6

p

b. If Xn (i) − Yn (i) → 0 for each fixed i then for some sequence {Ln } of positive integers with Ln
→ ∞, that is not unique:
p

max |Xn (i)| − max |Yn (i)| ≤ max |Xn (i) − Yn (i)| → 0.
1≤i≤Ln

1≤i≤Ln

1≤i≤Ln

Remark 3 A similar result exists under almost sure convergence, although a different argument
is required. See the supplemental material Hill (2017, Appendix B).
Remark 4 The method of proof for Lemma 2.1 shows the existence of such a sequence {Ln } and
therefore cannot provide an upper bound on the rate Ln → ∞. This seems unavoidable since
we are working with general array convergence rather than, for example, the specific attributes
of Gaussian probability tails. The payoff is that such generality ultimately permits non-Gaussian
couplings, a data filter, and arbitrary dependence across i, as we treat in turn below.

3

Asymptotics for Maxima Based on Filtered Data

We now work with a parametric array {Xn (i, θ 0 (i)) : 1 ≤ i ≤ L}n≥1 , where θ 0 (i) is an unknown
but estimable parameter in Rk that may depend on i. Let θ̂n (i) be an estimator of θ0 (i). We
assume Xn (i, θ0 (i)) is unobserved while Xn (i, θ̂ n (i)) is observed. Our leading example is a sample
serial correlation coefficient for time series regression model errors (see Section 4). Another is a
sample mean of an observed time series scaled by its conditional variance, e.g. GARCH residuals.
Our primary goal is to prove under fairly general conditions that for some stochastic process
{Zn (i)}i∈N , and some non-unique {Ln }, Ln → ∞:
p

max Xn (i, θ̂n (i)) − max |Zn (i)| → 0.
1≤i≤Ln

1≤i≤Ln

(4)

p

In view of Theorem 2.2, it suffices to prove Xn (i, θ̂n (i)) − Zn (i)) → 0 for each fixed i. Sufficient conditions follow. These are not the most general possible, but give a reasonably general
environment to work in.

7

Assumption 1
a. Xn (i, θ) is continuous and differentiable in θ ∈ Θ(i), where Θ(i) is a compact subset of Rk , k
∈ N.
b. θ 0 (i) ∈ Θ(i).
c. There exists a continuous non-stochastic function D(i, ·) : Θ(i) → Rk , and a compact neighborhood N0 (i) of θ0 (i) with positive Lebesgue measure such that
sup
θ∈N0 (i)

1 ∂
p
√
Xn (i, θ) − D(i, θ) → 0.
n ∂θ

d. There exists stochastic processes {Sn (i), Mn (i) : i ∈ N} such that Xn (i, θ0 (i)) = Sn (i)+ op (1)
√
and n(θ̂n (i) − θ0 (i)) = Mn (i) + op (1) for each i. Moreover, there exists non-degenerate stochasd

tic processes {S(i), M(i) : i ∈ N} such that (Sn (i), Mn (i)) → (S(i), M(i)) for each i.
√ p
Remark 5 Uniform convergence (c) holds when (∂/∂θ)Xn (i, θ)/ n → D(i, θ) for each θ ∈ N0 (i),
and a stochastic equicontinuity. condition holds. The latter holds, for example, when Xn (i, θ) is
twice continuously differentiable and the envelope
"

1
∂2
Xn (i, θ)
E √ sup
n θ∈N0 (i) ∂θ i ∂θ j

#

= O(1).

See, for example, Newey (1991) and Andrews (1992).
√ P
Remark 6 Joint convergence (d) holds, for example, when Xn (i, ·) = 1/ n nt=1 xt (i) where xt (i)
√
√ P
has a zero mean and is square integrable, n(θ̂n (i) − θ0 (i)) = 1/ n nt=1 mt (i) + op (1) for some
zero mean square integrable random variables mt (i), and {xt (i), mt (i)} satisfy suitable moment

and dependence properties. An example concerning a residuals white noise test is provided in
Section 4.
Remark 7 Conditions (b) and (d) allow for non-standard cases. One example is when θ0 (i) lies
on the boundary of Θ(i) (e.g. Andrews, 1999), and another is when a component of θ0 (i) is weakly
8

or non-identified (e.g. Andrews and Cheng, 2012). In each case
d

√

n(θ̂n (i) − θ0 (i)) = Mn (i) +

op (1) → M(i)) is non-Gaussian.
Theorem 3.1 Let Assumption 1 hold, and write D(i) ≡ D(i, θ0 ) where D(i, θ) is defined in
Assumption 1.c. For some sequence {Ln } of positive integers that is not unique, with Ln →
p

∞: | max1≤i≤Ln |Xn (i, θ̂n (i))| − max1≤i≤Ln |Sn (i) + D(i)′ Mn (i)|| → 0 and | max1≤i≤Ln |Sn (i) +
p

D(i)′ Mn (i)| − max1≤i≤Ln |S(i)+D(i)′ M(i)|| → 0, hence | max1≤i≤Ln |Xn (i, θ̂n (i))| − max1≤i≤Ln |S(i)
p

+ D(i)′ M(i)|| → 0.

4

Illustrations

We now consider maximum statistics that involve unit root test statistics and a residuals white
noise test statistic. Both require a non-Gaussian approximation theory, demonstrating the unique
applicability of Theorem 2.2.

4.1

Unit Root Tests

Consider unit root tests over a set of processes {yt (i) : i ∈ N}. Suppose yt (i) = φ0 (i)yt−1 (i)
+ ǫt (i), |φ0 (i)| ≤ 1, for i ∈ N. We assume {ǫt (i) : i ∈ N} lies in a probability measure space
(Ω, σ(∪t∈N Ft ), P), where {Ft }t∈N is a sequence of σ-felds. Assume ǫt (i) is Ft -measurable for
each i. Write Fst ≡ σ(∪tτ =s Fτ ). Zero mean ǫt (i) is stationary, and E|ǫt (i)|r < ∞ for some r >
2. Moreover, α-mixing coefficients αh ≡ supA∈F t−h ,B∈Ft∞ |P(A ∩ B) − P(A)P (B)| satisfy αh =
−∞

r/(r−2)

O(h

/ ln(h)). Thus, ǫt (i) and any measurable function of {ǫt (i) : 1 ≤ i ≤ k} has coefficients

αh . Let φ̂n (i) be the least squares estimator.
We want to test the hypothesis that all processes have a unit root H0 : φ0 (i) = 1 for all i ∈
N. The proposed test statistic is

Tn (k) ≡ n max φ̂n (i) − 1 .
1≤i≤k

9

√ P
Define σ 2 (i) ≡ limn→∞ E[(1/ n nt=1 ǫt (i))2 ] and σ 2ǫ (i) ≡ E[ǫ2t (i)], and define
1
T (i) ≡
W(i, 1)2 − σ 2ǫ (i)/σ 2 (i) /
2

Z

0

1

W(i, ν)2 dν

where {W(·, ν) : [0, 1]} are standard Wiener processes. T (i) is the well known limit law for the
d

least squares estimator when there is a unit root and ǫt (i) is possibly dependent: n(φ̂n (i) − 1) →
T (i) (Phillips, 1987, Theorem 3.1). If ǫt (i) is iid then σ 2ǫ (i)/σ 2 (i) = 1, cf. White (1958).
d

Apply the mapping theorem to yield n|φ̂n (i) − 1| → |T (i)| for each i under H0 . Therefore
n|φ̂n (i) − 1| − |T (i)| = op (1) for each i under H0 . Now invoke Theorem 2.2 to deduce |Tn (Ln ) −
p

max1≤i≤Ln |T (i)|| → 0 under H0 for some non-unique sequence of positive integers {Ln }, Ln →
∞. Conversely, if |φ0 (i∗ )| < 1 for some i∗ ∈ N, then such n(φ0 (i∗ ) − 1) → −∞ will eventually
dominate Tn (k) = max1≤i≤k |n(φ̂n (i) − φ0 (i)) − n(φ0 (i) − 1)| for any k ≥ i∗ . Therefore Tn (Ln )
p

→ ∞ for any sequence of positive integers {Ln }, Ln → ∞.
The limit law T (i) is not pivotal since it contains nuisance parameters σ 2ǫ (i) and σ 2 (i). Let
σ̂ 2n,ǫ (i) and σ̂ 2n (i) be consistent estimators of σ 2ǫ (i) and σ 2 (i) respectively (see Phillips, 1987, Section
4), and define
!

σ̂ 2n (i) − σ̂ 2n,ǫ (i)
P
T̃n (k) ≡ max n φ̂n (i) −
− 1 = max T̃n,i ,
2
1≤i≤k
1≤i≤k
(i)
1/n2 ni=2 yt−1
11
n2

where T̃n,i is implicitly defined. The term T̃n,i was proposed in Phillips (1987, Section 5) as an
adjustment that leads to a pivotal asymptotic law.
R1
Denote T̃ (i) ≡ (1/2)(W(i, 1)2 − 1)/ 0 W(i, ν)2 dν, the now classic limit law for the least

squares estimator when there is a unit root and iid error (White, 1958). Theorem 5.1 in Phillips

p
(1987) implies T̃n,i = T̃ (i) + op (1). Now use Theorem 2.2 to yield |T̃n (Ln ) − maxi∈I(Ln ) T˜ (i)| →

0 under H0 for some non-unique sequence of positive integers {Ln }, Ln → ∞.

10

4.2

Residuals White Noise Test

Consider a process {yt } modeled as an AR(p) for finite p ≥ 1,
y t = c0 +

p
X

φ0,i yt−1 + ǫt = θ′0 xt + ǫt

i=1

where θ0 ≡ [c0 , φ′0 ]′ , xt ≡ [1, yt−1 , ..., yt−p ]′ , and 1 −

Pp

i=1

φ0,i z i has roots outside the unit circle.

The AR model is assumed to be pseudo true in the sense that θ0 is the unique point in the
interior of compact Θ that satisfies E[ǫt xt ] = 0. Assume E[yt2 ] > 0 and E|yt |r < ∞ for some r
> 4. Define σ-fields Ft ≡ σ(yτ : τ ≤ t) and Fst ≡ σ(yτ : s ≤ τ ≤ t), and assume Ft−1 ⊂ Ft ∀t.
t−m
Assume yt is stationary α-mixing with coefficients αh ≡ supA⊂F−∞
,B⊂Ft∞ |P (A ∩ B) − P (A)P (B)|

= O(hr/(r−2) / ln(h)). Sufficient conditions for the strong mixing property in linear processes are
presented in Gorodetskii (1977) and Withers (1981), amongst others.
′

We want to test H0 : E[ǫt ǫt−h ] = 0 ∀h ≥ 1. Let θ̂n ≡ [ĉn , φ̂n ] be the least squares estimator of
θ0 , and define an error function, and the residual sample serial correlation at lag h ≥ 1:
√ 1/n
ǫt (θ) ≡ yt − θ′ xt and Xn (i, θ) ≡ n

Pn

t=1+h ǫt (θ)ǫt−h (θ)
P
.
1/n nt=1 ǫ2t (θ)

A valid test can be based on the maximum absolute correlation, max1≤h≤Ln |Xn (i, θ̂n )|, cf. Hill and Motegi
(2017). We only provide a proof linking max1≤h≤Ln |Xn (i, θ̂n )| to the maximum of a process that
depends on ǫt and properties of the plug-in estimator, based on an asymptotic expansion. See
Hill and Motegi (2017, Section 2, Theorem 2.5) for an asymptotically valid dependent wild bootstrap based on the expansion.
The following is based on arguments in Hill and Motegi (2017, Lemma 2.1).1 The proof relies
on a standard expansion, and Theorem 2.2. We therefore prove the claim in the supplemental
1

As in the present example, Hill and Motegi (2017) work with a residuals sample correlation in a general
parametric regression model setting. Since the form of Xn (i, θ) is therefore known, they do not need to verify
conditions like Assumption 1, and instead deliver an expansion using a more direct proof.

11

material Hill (2017). Define

D(h) = −E [ǫt xt−h ] − E [ǫt−h xt ] and zt (h) ≡
wt (λ) = λ1

1
−1
ǫ ǫ − D(h)′ (E [xt x′t ]) xt ǫt
2 t t−h
E [ǫt ]

1
−1
ǫ + λ2 (E [xt x′t ]) xt for arbitrary (λ1 , λ2 ) ∈ R, λ21 + λ22 = 1.
2 t−h
E [ǫt ]

Theorem 4.1
√ P
a. Let H0 hold and assume limn→∞ E(1/ n nt=1 wt (λ)ǫt )2 ] > 0. Moreover, assume for any

asymptotic draw {yt , ǫt }∞
t=1 that inf θ∈Θ |ǫt (θ)| ≥ ι a.s. for all t ∈ N/S where S is finite, and for
some non-random ι > 0. Assumption 1 applies, and therefore the conclusions of Theorem 3.1
hold. In particular, for some non-unique sequence of positive integers {Ln }, Ln → ∞:
max Xn (i, θ̂ n ) − max

1≤h≤Ln

1≤h≤Ln

n
1 X
p
√
zt (h) → 0.
n t=1+h

(5)

√ P
p
Moreover, max1≤h≤Ln ||1/ n nt=1+h zt (h)| − max1≤h≤Ln |Z(h)|| → 0 where {Z(h) : h ∈ N} is a
P
zero mean Gaussian process with covariance kernel E[Z(h)Z(h̃)] = limn→∞ 1/n s,t=1 E[zs (h)zt (h̃)]
and variance E[Z(h)2 ] < ∞.

p

b. If H0 is false then max1≤h≤Ln |Xn (i, θ̂n )| → ∞ for any sequence of positive integers {Ln }, Ln
→ ∞.
√ P
Remark 8 The bound limn→∞ E(1/ n nt=1 wt (λ)ǫt )2 ] > 0 ensures a non-degenerate limit the-

ory for a key joint process arising in a sample correlation first order expansion. The limit is
finite by the mixing property and E|wt (λ)ǫt |r/2 < ∞ where r > 4 by assumed Lr -boundedness

(Ibragimov, 1975, Theorem 1.7). The assumption inf θ∈Θ |ǫt (θ)| ≥ ι > 0 a.s. for all t ∈ N/S and
finite S expedites the expansion proof. This is mild since Ft−1 ⊂ Ft ∀t implies inf θ∈Θ |ǫt (θ)| > 0
a.s. ∀t (see the proof of Theorem 4.1).
Remark 9 The result significantly augments known results in the max-correlation literature by
permitting residuals, and without restricting dependence in the limit process {Z(h)}. Existing
12

extreme value theory works with observed data, and imposes conditions that ensure

√

n(γ̂ n (i) −

E[xt xt−i ]) converges in finite dimensional distributions to a Gaussian law Z(i), and max1≤i≤Ln |Z(i)|
converges in law to a well defined random variable (e.g. Xiao and Wu, 2014). The latter requires
asymptotic independence E[Z(i)Z(j)] → 0 as |i − j| → ∞, cf. Leadbetter, Lindgren, and Rootzén
(1983, Chapter 6) and Hüsler (1986, 1993). The high dimensional central limit theory literature
√ P
can tackle max1≤h≤Ln |1/ n nt=1+h zt (h)|, e.g. Chernozhukov, Chetverikov, and Kato (2013a),

but not the intermediate step (5) since zt (h) is generally not Gaussian.

5

Conclusion

We provide a general result for convergence of arrays that permits a new theory for the maximum of an increasing sequence of random variables. When linking the maximum of two random
variables Xn (i) and Yn (i), unlike the extreme value theory and high dimensional Gaussian central limit theory literatures, we do not require normality or even asymptotic normality of Yn (i).
This permits new results for maxima, covering heavy tailed data, non-stationary data, and filtered data where the filter may lead to non-standard asymptotics. Two illustrations are provided
covering unit root tests and a residual white noise test, both of which appear to be new. A
shortcoming of our general approach, based ultimately on Ramsey (1930) theory and its implications for array convergence, is that we cannot bound the allowed array dimension Ln as n → ∞.
This runs contrary to the max-correlation literature (e.g. Xiao and Wu, 2014), and the Gaussian
coupling literature, most recently punctuated by Chernozhukov, Chetverikov, and Kato (2013a,
2016), where the best known bounds on Ln are available.

A

Appendix: Proofs

Proof of Lemma 2.1.

We prove in Step 1 that liml→∞ AL(nl ),nl = 0 for some sequence of

positive integers {nl }∞
l=1 , nl < nl+1 ∀l, and some mapping L(nl ) ≤ L(nl+1 ), L(nl ) → ∞ and nl
→ ∞ as l → ∞. We use that result in Step 2 to prove the claim.
Step 1.

We now prove liml→∞ AL(nl ),nl = 0. By assumption {Ak,n : 1 ≤ k ≤ In }n≥1 lies in
13

(R, d), which is a first countable topological space, and limk→∞ limn→∞ Ak,n = 0. Therefore, by
Lemma 1 in Boehme and Rosenfeld (1974) there exists a sequence of positive integers {Li }∞
i=1 , Li
→ ∞ as i → ∞, and an integer mapping n(L) → ∞ as L → ∞ such that limi→∞ ALi ,n(Li) = 0.
The relation n(L) → ∞ as L → ∞ holds by construction of the array {Ak,n : 1 ≤ k ≤ In }n≥1
with In → ∞ as n → ∞.
We can always assume monotonicity: Li ≤ Li+1 ∀i. Simply note that limi→∞ ALi,n(Li ) =
0 implies liml→∞ ALil ,n(Lil ) = 0 for every infinite subsequence {il }l≥1 of {i}i≥1 . Since Li → ∞
as i → ∞, we can find a subsequence {i∗l }l≥1 such that i∗l ≤ i∗l+1 and Li∗l ≤ Li∗l+1 for each l.
This follows from the monotone subsequence theorem, which itself follows from Ramsey’s (1930)
theorem, cf. Erdös and Szekeres (1935) and Burkill and Mirsky (1973). Now define L∗l ≡ Li∗l ,
hence liml→∞ AL∗l ,n(L∗l ) = 0 where L∗l ≤ L∗l+1 and L∗l → ∞ as l → ∞.
∞
Now let {ni }∞
i=1 and {L(ni )}i=1 be any sequences satisfying ni = n(Li ) and L(ni ) = Li . Hence

by the above argument L(ni ) ≤ L(ni + 1), L(ni ) → ∞ and ni → ∞, such that limi→∞ AL(ni ),ni
= 0. Note that limi→∞ AL(ni),ni = 0 if and only if liml→∞ AL(nil ),nil = 0 for every subsequence
∞
{nil }∞
l=1 of {ni }i=1 . Since ni → ∞ as i → ∞, by the monotone subsequence theorem there exists

a strictly monotonically increasing subsequence {nil }∞
l=1 . Therefore, as required liml→∞ AL(nl ),nl
= 0 for some sequence of positive integers {nl }∞
l=1 , nl < nl+1 ∀l, and L(nl ) ≤ L(nl+1 ), L(nl ) →
∞ and nl → ∞ as l → ∞.
Step 2.

By assumption limn→∞ Ak,n = 0 ∀k. Therefore:
lim Ak,ns = 0 for every k and every infinite subsequence {ns }s≥1 .

s→∞

(A.1)

Now repeat the Step 1 argument for each {Ak,ns }s≥1 : there exists a strictly monotonically increasing subsequence of positive integers {nsl }l≥1 and some integer mapping Ls (nsl ) that may depend
on s, with nsl → ∞ and Ls (nsl ) → ∞ as l → ∞ ∀s, such that liml→∞ ALs (nsl ),nsl = 0 ∀s. As
above, we may take Ls (·) to be monotonic: Ls (ñ) ≤ Ls (ñ + 1) ∀ñ.
Since monotonic Ls (ñ) → ∞ as ñ → ∞ ∀s, there exists an integer mapping L(·) such that
L(n) → ∞ as n → ∞ and for each s, lim supn→∞ {L(n)/Ls (n)} < 1. By monotonicity Ak,n ≤
14

Ak+1,n this mapping satisfies
lim AL(nsl ),nsl ≤ lim ALs(nsl ),nsl = 0 ∀s.

(A.2)

l→∞

l→∞

Notice L(·) is not unique: for any L(·) that satisfies (A.2) there exists L̃(n) → ∞ such that
lim supn→∞ L̃(n)/L(n) < 1, hence by monotonicity liml→∞ AL̃(ns ),ns ≤ liml→∞ AL(nsl ),nsl = 0.
l

l

Now write Bn ≡ AL(n),n . By a direct implication of (A.1) and (A.2), for every subsequence
{Bns }s≥1 there exists a further subsequence {Bnsl }l≥1 that converges liml→∞ Bnsl = 0. Therefore
limn→∞ Bn = 0 (see Royden, 1988, p. 39). This proves limn→∞ ALn,n = 0 with Ln = L(n) as
required. QED.
Proof of Theorem 2.2.
p

p

By assumption each Xn (i) → 0 and therefore max1≤i≤k |Xn (i)| → 0. Define Ak,n
R∞
≡ 1 − exp{− max1≤i≤k |Xn (i)|)} and Pk,n ≡ 0 P (Ak,n > ǫ)dǫ. By construction Ak,n ∈ [0, 1] a.s.

Claim (a).

p

∀k. Lebesgue’s dominated convergence theorem, and Ak,n → 0, therefore yield for each k:
lim Pk,n = lim

n→∞

n→∞

Z

∞

P (Ak,n > ǫ) dǫ = lim

n→∞

0

Z

1

P (Ak,n > ǫ) dǫ =

0

Z

1

lim P (Ak,n > ǫ) dǫ = 0.

0 n→∞

Now apply Lemma 2.1 to Pk,n to deduce that there exists a positive integer sequence {Ln }
R1
that is not unique, Ln → ∞ and Ln = o(n), such that limn→∞ PLn ,n = limn→∞ 0 P (ALn,n >
R1
p
ǫ)dǫ = 0. Therefore, by construction E[ALn,n ] = 0 P (ALn,n > ǫ)dǫ → 0. Hence ALn ,n → 0 by
p

Markov’s inequality, which yields max1≤i≤Ln |Xn (i)| → 0 as claimed.

The sequence {Ln } is not unique for either of the following reasons: (i) the probability limit
is asymptotic hence we can always change Ln for finitely many n; and (ii) by monotonicity of
max1≤i≤k |Xn (i)| any other {L̊n } that satisfies L̊n → ∞ and lim supn→∞ {L̊n /Ln } < 1 satisfies
p

max1≤i≤L̊n |Xn (i)| ≤ max1≤i≤Ln |Xn (i)| → 0 as n → ∞.
Claim (b).

Apply the triangle inequality twice to yield both max1≤i≤k |Xn (i)| ≤ max1≤i≤k |Yn (i)|

+ max1≤i≤k |Xn (i) − Yn (i)| and max1≤i≤k |Yn (i)| ≤ max1≤i≤k |Xn (i)| + max1≤i≤k |Xn (i) − Yn (i)|,
15

hence
max |Xn (i)| − max |Yn (i)| ≤ max |Xn (i) − Yn (i)|

1≤i≤Ln

1≤i≤Ln

1≤i≤Ln

Now apply (a) to Xn (i) − Yn (i) to yield the desired result. QED.
By the mean value theorem, there exists θ ∗n (i), ||θ∗n (i) − θ 0 (i)|| ≤

Proof of Theorem 3.1.
||θ̂n (i) − θ0 (i)||, such that:


√ 
1 ∂
∗
Xn (i, θ̂n (i)) = Xn (i, θ0 (i)) + √
X
(i,
θ
(i))
n
θ̂
(i)
−
θ
(i)
.
n
n
0
n
n ∂θ ′
Under Assumption 1.d,
p

√

n(θ̂n (i) − θ 0 (i)) = Mn (i) + op (1) = Op (1), hence ||θ∗n (i) − θ 0 (i)|| ≤

||θ̂n (i) − θ0 (i)|| → 0. Therefore θ ∗n (i) lies in any compact neighborhood N0 (i) of θ0 (i) with positive
Lebesgue measure asymptotically with probability approaching one. Now use Assumption 1.c and
p

continuity of D(i, ·) to yield n−1/2 (∂/∂θ)Xn (i, θ∗n (i)) → D(i) ≡ D(i, θ0 ). Further, by Assumption
1.d Xn (i, θ0 (i)) = Sn (i)+ op (1). We may therefore write
Xn (i, θ̂n (i)) = Sn (i) + D(i)′ Mn (i) + op (1).

(A.3)
d

Apply Assumption 1,d and the continuous mapping theorem to deduce Sn (i) + D(i)Mn (i) →
S(i) + D(i)′ M(i). By the definition of convergence in distribution, we may write for each i:
Sn (i) + D(i)′ Mn (i) = S(i) + D(i)′ M(i) + op (1).

(A.4)

The claim now follows from (A.3) and (A.4), and two applications of Theorem 2.2. QED.

References
Andrews, D. W. K. (1992): “Generic Uniform Convergence,” Econometric Theory, 8, 241–257.
(1999): “Estimation When a Parameter is on a Boundary,” Econometrica, 67, 1341–1383.
Andrews, D. W. K., and X. Cheng (2012): “Estimation and Inference with Weak, SemiStrong and Strong Identification,” Econometrica, 80, 2153–2211.
Berman, S. M. (1964): “Limit theorems for the maximum term in stationary sequences,” Annals
of Mathematical Statistics, 35, 502–516.
16

Boehme, T. K., and M. Rosenfeld (1974): “An Example of Two Compact Hausdorff Frechet
Spaces Whose Product is not Frechet,” Journal of the Lodon Mathematical Society, 8, 339–344.
Burkill, H., and L. Mirsky (1973): “Monotonicity,” Journal of Multivariate Analysis and
Applications, 35, 502–516.
Chernozhukov, V., D. Chetverikov, and K. Kato (2013a): “Gaussian Approximations
and Multiplier Bootstrap for Maxima of Sums of High-Dimensional Random Vectors,” Annals
of Statistics, 41, 2786–2819.
(2013b): “Supplemental Material for ‘Gaussian Approximations and Multiplier Bootstrap for Maxima of Sums of High-Dimensional Random Vectors’,” Massachusetts Institute of
Technology.
(2015): “Comparison and Anti-Concentration Bounds for Maxima of Gaussian Random
Vectors,” Probability Theory and Related Fields, 162, 47–70.
(2016): “Central Limit Theorems and Bootstrap in High Dimensions,” Annals of Probability, forthcoming.
Dudley, R. M. (1978): “Central Limit Theorems for Empirical Measures,” Annals of Probability,
6, 899–929.
(1984): “A Course on Empirical Processes,” in Ecole Eté de Probabilités de St. Flour,
Lecture Notes in Math, vol. 1097, pp. 1–142. Springer, New York.
Dudley, R. M., and W. Philipp (1983): “Invariance Principles for Sums of Banach Space
Valued Random Elements and Empirical Processes,” Zeitschrift für Wahrscheinlichkeitstheorie
und Verwandte Gebiete, 62, 509–552.
Erdös, P., and G. Szekeres (1935): “A combinatorial problem in geometry,” Compositio
Mathematica, 2, 463470.
Falk, M., J. Hüsler, and R.-D. Reiss (2011): Law of Small Numbers: Extremes and Rare
Events. Birkhäuser, Berlin, 3 edn.
Gorodetskii, V. V. (1977): “On the Srong Mixing Property for Linear Processes,” Theory of
Probability and its Applications, 22, 411–413.
Götze, F. (1991): “On the Rate of Convergence in the Multivariate Central Limit Theorem,”
Annals of Probability, 19, 724–739.
Hannan, E. J. (1974): “The Uniform Convergence of Autocovariances,” Annals of Statistics, 2,
803–806.
Hill, J. B. (2017): “Supplemental Material for ‘Asymptotic Theory for the Maximum of an
Increasing Sequence of Parametric Functions’,” .
Hill, J. B., and K. Motegi (2017): “A Max-Correlation White Noise Test for Weakly Dependent Time Series,” Discussion paper, University of North Carolina - Chapel Hill.
Hoffmann-Jorgensen, J. (1984): “Convergence of Stochastic Processes on Polish Spaces,”
17

mimeo.
(1991): “Convergence of Stochastic Processes on Polish Spaces,” in Various Publications
Series 39. Aarhus Universitet, Aarhus, Denmark, mimeo.
Homble, P., and P. McCormick (1995): “Weak Limit Results for the Extremes of a Class of
Shot Noise Processes,” Journal of Applied Probability, 32, 707–726.
Hüsler, J. (1986): “Extreme Values of Non-Stationary Random Sequences,” Journal of Applied
Probability, 23, 937–950.
(1993): “Extremes: Limit Results for Univariate and Multivariate Nonstationary Sequences,” in Extreme Value Theory and Applications, ed. by J. Galambos, J. Lechner, and
E. Simiu, pp. 283–204. Prooceedings on the Conference on Extreme Value Theorem and Applications, Volume I, Gaithersburg, Maryland, Kluwer Academic Publishers.
Ibragimov, I. A. (1975): “A Note on the Central Limit Theorems for Dependent Random
Variables,” Theory of Probability and its Applications, 20, 135–141.
Jirak, M. (2011): “On the Maximum of Covariance Estimators,” Journal of Multivariate Analysis, 102, 1032–1046.
Le Cam, L. (1988): “On the Prohorov distance between the empirical process and the associated
Gaussian bridge,” Discussion paper, University of Califormia - Berkely, Dept. of Statistics, UCBerkeley.
Leadbetter, M. R., G. Lindgren, and H. Rootzén (1983): Extremes and Related Properties of Random Sequences and Processes, Springer Series in Statistics. Springer-Verlag, New
York.
Newey, W. K. (1991): “Uniform Convergence in Probability and Stochastic Equicontinuity,”
Econometrica, 59, 1161–1167.
Phillips, P. C. B. (1987): “Time Series with a Unit Root,” Econometrica, 55, 277–301.
Pollard, D. (1984): Convergence of Stochastic Processes. Springer, New York.
(1990): “Empirical Processes: Theory and Applications,” in NSF-CBMS Regional Conference Series in Probability and Statistics, vol. II. Institute of Mathematical Statistics and the
American Statistical Association.
(2002): A Users Guide to Measure Theoretic Probability. Cambridge Series in Statistical
and Probabilistic Mathematics 8. Cambridge Univ. Press, Cambridge.
Portnoy, S. (1985): “Asymptotic Behavior of M Estimators of p Regression Parameters when
p2 /n is Large; II.,” Annals of Statistics, 13, 1403–1417.
(1986): “On the Central Limit Theorem in Rp when p → ∞,” Probability Theory and
Related Fields, 73, 571–583.
Ramsey, F. P. (1930): “On a Problem of Formal Logic,” Proceedings of the London Mathematical
Society, 30, 264–286.
18

Royden, H. L. (1988): Real Analysis. Prentise Hall, 3rd edn.
White, J. S. (1958): “The Limiting Distribution of the Serial Correlation Coefficient in the
Explosive Case,” Annals of Mathematical Statistics, 29, 1188–1197.
Withers, C. S. (1981): “Conditions for Linear Processes to be Strong Mixing,” Zeitschrift für
Wahrscheinlichkeitstheorie und Verwandte Gebiete, 57, 477–480.
Xiao, H., and W. B. Wu (2014): “Portmanteau Test and Simultaneous Inference for Serial
Covariances,” Statistica Sinica, 24, 577–599.
Yurinskii, V. V. (1977): “On the Error of the Gaussian Approximation for Convolutions,”
Theory of Probability and its Applications, 22, 236–247.

19

