arXiv:1506.05830v2 [stat.AP] 12 Dec 2016

Asymptotic Theory for M-Estimates in Unstable AR(p)
Processes with Infinite Variance Innovations ∗
Maryam Sohrabi1 †and Mahmoud Zarepour2 ‡
1

School of Mathematics and Statistics, Carleton University, Ottawa, ON, Canada
Department of Mathematics and Statistics, University of Ottawa, ON, Canada

2

January 16, 2018

ABSTRACT
In this paper, we present the asymptotic distribution of M-estimators for parameters in unstable AR(p) processes. The innovations are assumed to be in the domain of attraction of a
symmetric stable law with index 0 < α ≤ 2. In particular, in the case of repeated unit roots or
conjugate complex unit roots, M-estimators have a higher asymptotic rate of convergence compared to the least square estimators and the asymptotic results can be written as Itô stochastic
integrals.
AMS classification: 62M10, 60G52, 62F40.
K EYWORDS : Autoregressive model, Unit root, Stable process, Non-stationary, Bootstrapping.

1

Introduction

Consider the autoregressive process of order p (AR(p))
φ(B)Xt = t ,

(1.1)

φ(z) = 1 − φ1 z − φ2 z 2 − · · · − φp z p .

(1.2)

where B is the backward operator and

∗
Research supported by a research grant from the Natural Sciences and Engineering Research Council of Canada
(NSERC)
†
E-mail addresses: Maryam.Sohrabi3@carleton.ca (M. Sohrabi)
‡
E-mail addresses: mahmoud.zarepour@uottawa.ca (M. Zarepour)

1

Sohrabi and Zarepour: Asymptotic Theory for M-Estimates in Unstable AR(p) Processes

2

The errors {t } in (1.1) form a sequence of independent and identically distributed (i.i.d.) random
variables in the domain of attraction of a symmetric stable law with index 0 < α ≤ 2. The model
(1.1) is referred to as non-stationary autoregressive time series if the characteristic polynomial φ(·)
has at least one root on the boundary of the unit circle.
It is well known that the unit root tests are particularly an important tool to classify if a time
series is stationary or non-stationary. When economic variables are non-stationary, estimates may
generate a spurious model unless they are cointegrated. A unit root test can be used for cointegration of two processes. The analysis of unit-root processes and cointegrated time series is likely to
be the one of the most important and controversial topics in econometrics in the last few decades.
The case where innovations are in the domain of attraction of the Gaussian distribution has received
considerable attention in cointegration literature; see for example Engle and Granger (1987) and
Park and Phillips (1988). However, many empirical studies show that heavy-tailed and asymmetrically distributed samples are frequently observed in economic and, especially, financial time series.
In these cases, the Gaussian models are not applicable. Paulauskas and Rachev (1998) develop the
asymptotic theory for econometric cointegration processes under the assumption of infinite variance
innovations with different tail indices.
The problem of conducting asymptotic inference for time series with unit roots has been a
challenging topic of interest for some time. In cases where errors (innovations) have finite variance,
Dickey and Fuller (1979), and Phillips and Perron (1988) provide the asymptotic theory for the
least squares (LS) estimators in an AR(1) process with one unit root. Chan and Wei (1988) study
the large sample theory for a non-stationary autoregressive AR(p) model when the innovations form
a sequence of martingale differences with respect to an increasing sequence of σ-fields {Fn }.
With infinite variance innovations, Chan and Tran (1989) consider the Dickey-Fuller test when
the errors are in the domain of attraction of a stable law. Phillips (1990) extends the results of
Chan and Tran (1989) to find the limit theory of the parameters in an AR(1) process with weakly
dependent errors in the domain of attraction of a stable law. Since both the Dickey-Fuller and
Phillips-Perron statistics are based on LS estimation, they do not take advantage of the heavy tails
of the innovations and can exhibit rather poor power performance. Thus, it is important to consider
estimation and inference procedures that are robust to departures from finite variance condition.
One way to achieve robustness is the use of the M-estimate method. With an appropriate choice
of a loss function, M-estimates have a number of desirable properties when the errors are heavy
tailed. Knight (1989) considers the asymptotic behavior of the LS estimates and M-estimates for
the random walk model. The results establish that self-normalized M-estimates are asymptotically
normal and their rate of convergence is higher than the LS estimates. Davis, Knight, and Liu (1992)
mention that M-estimates are more appropriate when the distribution of innovations are heavytailed. This follows from the fact that M-estimates give less weight to the outliers. Samarakoon and
Knight (2009) develop a class of unit root tests based on M-estimators in an AR process with a unit
root derived by infinite variance innovations.
Chan and Zhang (2012) obtain the limiting distribution for the LS estimates of the parameters
for unstable AR(p) processes, with i.i.d. innovations in the domain of attraction of a stable law. They
show that the limiting distribution of the LS estimate is a function of integrated stable processes.
However, for model (1.1) with unit roots when {t } is a sequence of random variables with infinite

Sohrabi and Zarepour: Asymptotic Theory for M-Estimates in Unstable AR(p) Processes

3

variance, a complete theory on a more efficient estimating technique is still missing in the literature.
In this paper, we consider an important class of unstable autoregressive time series models with
many practical implications. An example of these time series are seasonal models, where φ(z)
may have several real and complex conjugate roots on the unit circle. We derive the asymptotic
distribution of M-estimators for the parameters in an unstable AR(p) process, where the innovations
are in the domain of attraction of a stable law with index 0 < α ≤ 2. Our results show that,
similar to the previous cases, M-estimators have higher asymptotic rate of convergence than LS
estimators. This paper is organized as follows. Section 2 provides some necessary preliminary
concepts. In Section 3, the limiting distribution of M-estimates in an AR(p) model is presented and
Section 4 consists of our simulation study. Due to the complexity of the limiting distributions a brief
discussion and a bootstrap simulation scheme are presented in Section 5. We summarize our results
in Section 6 and finally, the proof of the main theorem is outlined in Appendix A.

2

Preliminaries

Consider the AR(p) model in (1.1) with characteristic polynomial in (1.2). The classical M-estimator
Φ̂ = (φ̂1 , φ̂2 , . . . , φ̂p )| of Φ = (φ1 , φ2 , . . . , φp )| minimizes
n
X

ρ(Xt − β1 Xt−1 − · · · − βp Xt−p ),

t=p+1

with respect to (β1 , . . . , βp ), where ρ is an almost everywhere differentiable convex function. This
guarantees the uniqueness of the solution. For more details see Davis et al. (1992). Usually, ρ(x)
grows at a slower rate than x2 , as |x| gets large. An example for ρ(·) is the Huber loss function
given by

ρH (x) = 12 x2 I (|x| ≤ c) + c |x| − 12 c2 I (|x| > c)
(2.1)
for a known constant c, where I(·) denotes the indicator function. Throughout this paper we impose
the following assumptions on the function ρ(·).
Assumption 1. (A1) Let ρ be a convex and twice differentiable function, and take ψ = ρ0 .
Assumption 2. (A2) E(ψ(1 )) = 0 and E(ψ 2 (1 )) < ∞.
Assumption 3. (A3) 0 < |E(ψ 0 (1 ))| < ∞ and ψ 0 (·) satisfies the Lipschitz- continuity condition;
i.e., there exists a real constant k > 0 such that for all x and y,
|ψ 0 (x) − ψ 0 (y)| ≤ k|x − y|.
Note that for the Assumptions A1-A3, sometimes ρ0 does not exist everywhere. In this case, although ρ0 is not differentiable at a countable number of points, the results will usually hold with
some additional complexity in the proofs. Moreover, we assume that the innovations {t } satisfy:

Sohrabi and Zarepour: Asymptotic Theory for M-Estimates in Unstable AR(p) Processes

4

Assumption 4. (A4) The innovations {t } are i.i.d. random variables in the domain of attraction
of a stable law with index 0 < α ≤ 2. Note that for 0 < α < 2, the innovations have regularly
varying tail probabilities as specified by
P (|1 | > x) = x−α L(x)
for some slowly varying function L at ∞ with α > 0 as x → ∞,
P (1 > x)
→p
P (|1 | > x)

,

P (1 ≤ −x)
→ q,
P (|1 | > x)

, 0 ≤ p ≤ 1, q = 1 − p.

Symmetry is a common assumption for innovations. For 0 < α < 1, symmetry is not required.
However, for 1 < α ≤ 2 we only need E(1 ) = 0. For sake of simplicity we assume symmetry on
innovations; i.e., p = q = 1/2; see Assumption A4 implies that:
Sn (t) =

a−1
n

[nt]
X

d

k → S(t) in

D[0, 1],

(2.2)

k=1
d

where → denotes convergence in distribution with respect to the Skorohod topology and [x] stands
for integer part of x. The Skorohod space of the càdlàg functions defined on [0, 1], equipped with
the Skorohod topology, is denoted by D = D[0, 1]. Here, {an } is a sequence of positive constants
such that
an = inf{x : P [|1 | > x] ≤ n−1 }.

(2.3)

Moreover, S(·) is a stable process and from Resnick and Greenwood (1979) (see also Knight
(1989)), the representation for S can be shown to be
( P
−1/α
∞
I (Uk ≤ t) if 0 < α < 2,
k=1 δk Γk
(2.4)
S (t) =
standard Brownian motion if α = 2.
Here and throughout this paper, {Uk } is a sequence of i.i.d. U [0, 1] random variables and {δk } is a
sequence of i.i.d. random variables such that P (δk = 1) = p, P (δk = −1) = q, and p+q = 1. Also,
Γ1 , Γ2 , . . . are the arrival times of a Poisson process with Lebesgue mean measure and independent
of {δk }. Note that {Uk , Γk , δk } are mutually independent and the series in (2.4) is convergent if
either 0 < α < 1 or p = q = 1/2. For more details, see LePage, Woodroofe, and Zinn (1981).
To derive the main result of this paper, we assume that conditions A1-A4 hold and we define
the following processes on the Skorohod space D[0, 1]:
Sn(1) (t)

=

a−1
n

[nt]
X



k

(−1) k , Tn (t) =

k=1


Rn (t) =

R1,n (t)
R2,n (t)

Wn (t) = n−1/2

[nt]
X
k=1



[nt]

= n−1/2

X
k=1

T1,n (t)
T2,n (t)


=

sin ((k − 1)θ)
cos ((k − 1)θ)

ψ(k ), Vn (t) = n−1/2

a−1
n


[nt] 
X
cos(kθ)
k ,
sin(kθ)

k=1


ψ(k ),

[nt]
X
 0

ψ (k ) − E (ψ 0 (k )) .
k=1

(2.5)

Sohrabi and Zarepour: Asymptotic Theory for M-Estimates in Unstable AR(p) Processes

5

Similar to Theorem 4 of Resnick and Greenwood (1979), we can show that
| d
|
Sn (·), Sn(1) (·), Tn (·), Wn (·), Rn (·), Vn (·) → S(·), S (1) (·), T(·), W (·), R(·), V (·) , (2.6)
where S(·) and S (1) (·) are stable processes and T(·) is a bivariate stable process which is defined in Lemma 3 (see (7.1)).
Also, W (·) and V (·) are standard Brownian-motion
with

Pn processes
2
2 (t)) = tE ψ 2 ( ) and E(V 2 (t)) = tVar (ψ 0 ( )). Finally, since both
sin
((k
−
1)θ)
E(WP
1
1
k=1
and nk=1 cos2 ((k − 1)θ) are Op (n) for large values of n, the Lindeberg Feller central limit theorem and the tightness of the partial sum process Rn (·) give




E ψ 02 (1 )
R1 (t)
W1 (t)
R(t) =
=
.
(2.7)
R2 (t)
W2 (t)
2
Here, W1 (·), and W2(·) are independent standard Brownian motion processes. For 0 < α <
|
2, S(·), S (1) (·), T(·) is independent of (W (·), R(·), V (·))| . The dependence structure can be
constructed by applying the continuous mapping theorem on (5.1) in Appendix 1. For more details
see Resnick and Greenwood (1979).

3

The limiting distribution for AR(p)

The limiting distribution for the M-estimates of the parameters for an infinite-variance random-walk
processes is obtained in Knight (1989). To generalize, we extend the results of Knight (1989) to the
AR(p) process when characteristic roots may have different multiplicities and lie on the unit circle.
To derive the asymptotic behavior of the M-estimates, consider the AR(p) model in (1.1) when the
errors satisfy Assumption A4. Define the process
n h
i
X
−1
ρ(t − q1 b−1
An (q1 , . . . , qp ) =
X
−
·
·
·
−
q
b
X
)
−
ρ(
)
,
(3.1)
t−1
p
t−p
t
n1
np
t=p+1

where q n = Bn (Φ̂ − Φ) and Bn = diag(bn1 , . . . , bnp ) is the matrix of appropriate normalizing
constants (see Davis et al. (1992)). Note that the diagonal entries of Bn vary according to different
roots with different multiplicities and they will be specified in Theorem 1. Thus, it is reasonable to
expect that the minimizer of the process An can be written as

|
(B|n )−1 φ̂1 − φ1 , φ̂2 − φ2 , . . . , φ̂p − φp = D−1
(3.2)
n Pn .
Here
Pn = Bn

n
X

Xt−1 ψ(t ),

t=p+1

n
X

Xt−2 ψ(t ), . . . ,

t=p+1

!|

n
X

Xt−p ψ(t )

t=p+1

(n)

and Dn = (di,j ) is a p × p matrix such that
 Pn
Pn
2
0 (n)

Dn = Bn 


t=p+1
Pn

t=p+1

..
.
Pn

t=p+1

Xt−1 ψ (ct

)

(n)

Xt−1 Xt−2 ψ 0 (ct

(n)

Xt−1 Xt−p ψ 0 (ct

)

Pt=p+1
n

)

..
.
Pn

t=p+1

t=p+1

(n)

Xt−1 Xt−2 ψ 0 (ct
(n)

2 ψ 0 (c
Xt−2
t

)

···
···
..
.

)

(n)

Xt−2 Xt−p ψ 0 (ct

)

···

Pn

Pt=p+1
n

t=p+1

..
.
Pn

t=p+1

(n)

)

(n)

)

Xt−1 Xt−p ψ 0 (ct

Xt−2 Xt−p ψ 0 (ct
(n)

2 ψ 0 (c
Xt−p
t


 |
 Bn ,


)

(3.3)

Sohrabi and Zarepour: Asymptotic Theory for M-Estimates in Unstable AR(p) Processes

(n)

6

(n)

−1
0
where |ct − t | ≤ |q1 b−1
n1 Xt−1 − · · · − qp bnp Xt−p |. Asymptotically, ψ (ct ) can be replaced
by ψ 0 (t ) in (3.3). To see the proof, we assume that bni = n1/2 an for all i = 1, . . . , p. Since
Pp
(n)
−1/2 a−1 X
|ψ 0 (t ) − ψ 0 (ct )| ≤ k
t−i , we have
n
i=1 qi n

n−1 a−2
n

n
X

0

Xt−i Xt−j |ψ (t ) − ψ

0

(n)
(ct )|

≤k

t=p+1

p
X

|qk |n−1/2 n−1 a−3
n

n
X

P

|Xt−i Xt−j Xt−k | → 0.

t=p+1

k=1

In Theorem 1, we show that n1/2 an is the minimum value for bni , i = 1, . . . , p. Thus, this result
holds for all other normalizing constants. For the other entries of Dn , results follow similarly.
Furthermore, in (7.6) (Appendix A) we prove that asymptotically each ψ 0 (t ) can be replaced by
E (ψ 0 (t )). Therefore, as n → ∞, the matrix Dn will be nonsingular with probability 1.
The limiting distribution of the parameters in (3.2) is obtained through several steps. First, since
time series with different characteristic roots are expected to behave differently, we can decompose
the characteristic polynomial defined in (1.2) into the following polynomial:
r

l
Y

s

φ(z) = (1 − z) (1 + z)

(1 − 2 cos(θk )z + z 2 )dk ϕ(z),

(3.4)

k=1

where
P polynomial ϕ(·) corresponds to q roots which are outside the unit circle and q + r + s +
2 lk=1 dk = p. Similar to Chan and Wei (1988), we transform {Xt } into various components based
on the location of their roots. Then we find the limiting behavior of each component individually.
Davis et al. (1992) consider the asymptotic behavior of M-estimate for the parameters in a stationary
AR(p) process. In this paper we take ϕ(z) = 1. Define ut = φ(B)(1 − B)−r Xt , vt = φ(B)(1 +
B)−s Xt , and wt (k) = φ(B)(1 − 2 cos(θk )B + B 2 )−dk Xt for k = 1, 2, . . . , l. Equivalently,
t = (1 − B)r ut = (1 + B)s vt = (1 − 2 cos(θk )B + B 2 )dk wt (k).
From Chan and Wei (1988), there exists a nonsingular p × p matrix Q such that
|

QXt = (u|t , vt| , wt| (1), . . . , wt| (l)) ,
where Xt = (Xt , . . . , Xt−p+1 )| , ut = (ut , . . . , ut−r+1 )| , vt = (vt , . . . , vt−s+1 )| , and wt (k) =
(wt (k), . . . , wt−2dk +1 (k))| for k = 1, 2, . . . , l. Moreover, let Gn = diag (Jn , Kn , Ln (1), . . . , Ln (l))
be a normalization matrix, where Jn and Kn are as specified in (7.4) and (7.11) of Appendix A.
Also, Ln (i) for 1 ≤ i ≤ l ≤ p are defined similar to Ln in (7.18) in Appendix A. Then we have
p

Gn QXt ∼ diag (Jn ut , Kn vt , Ln (1)wt (1), . . . , Ln (l)wt (l))
and




| | −1
(Q Gn ) (Φ̂ − Φ) ∼ 



p

−1 P
n
ut−1 ut−1 | ψ 0 (t )
t=r+1 ut−1 ψ(t )
P
−1 P
| −1
n
n
|
0
(Kn )
t=s+1 vt−1 vt−1 ψ (t )
t=s+1 vt−1 ψ(t )
P
−1 P
n
n
|
−1
|
0
(Ln (1) )
t=2d1 +1 wt−1 (1)wt−1 (1)ψ (t )
t=2d1 +1 wt−1 (1)ψ(t )
P

(Jn| )−1

n
t=r+1

.
..
(Ln (l)| )−1

P

n
t=2dl +1

−1 P
n
wt−1 (l)wt−1 | (l)ψ 0 (t )
t=2dl +1 wt−1 (l)ψ(t )





,



(3.5)

Sohrabi and Zarepour: Asymptotic Theory for M-Estimates in Unstable AR(p) Processes

7

p

where an ∼ bn means an = bn + op (1). From (3.5), we can find the weak limit behavior of
M-estimates for the AR(p) processes defined in (1.1). The result is presented in Theorem 1.
Theorem 1. Suppose {Xt } satisfies (1.1) and conditions A1-A4 hold. Then
d

−1
|
|
(Q| G|n ))−1 (Φ̂ − Φ) → (Γ−1 F)| , (Υ−1 H)| , (Λ−1
1 G1 ) , . . . , (Λl Gl )

|

,

(3.6)

where (Γ−1 F) and (Υ−1 H), respectively, are defined in (7.9) and (7.14) in Appendix A. Also,
−1
(Λ−1
i Gi ) for i = 1, . . . , l are given similar to Λ G in (7.21). Note that, these limiting distributions
are functional of multiple stochastic integrals of stable processes.
Proof. See Appendix A.
An illustration of the validity of the results in Theorem 1 is given by the following example for
an AR(2) process.
Example 1. Suppose {Xt } is an AR(2) process. Consider the following cases.
(i) When φ(z) = 1 − 2z cos θ + z 2 ,
n1/2 an





φ̂1 − 2 cos θ
φ̂2 + 1


d


→ Γ−1
1

2 sin θE1/2 (ψ 2 (ε1 ))z1
R
R
E(ψ 0 (ε1 ))( 01 T12 (t)d(t)+ 01 T22 (t)d(t))
1/2
2
2 sin θE
(ψ (ε1R))z2
R
E(ψ 0 (ε1 ))( 01 T12 (t)d(t)+ 01 T22 (t)d(t))


,

where
1

Z

Z

1

T1 (t)dR1 (t) −

z1 = cos θ
0


Z
T2 (t)dR2 (t) + sin θ

0

and
Z

1

Z

0

Z

1

0


T2 (t)dR1 (t)

0


T2 (t)dR2 (t), Γ1 =

1

T1 (t)dR2 (t) +

0

T1 (t)dR1 (t) −

z2 =

1

1
cos θ

cos θ
1


.

(ii) When φ(z) = 1 − 2z + z 2 ,


n1/2 an (φ̂1 − 2)
3/2
n an (φ̂1 − 2) + n3/2 an (φ̂2 + 1)




d


→ Γ−1
2

R
E1/2 (ψ 2 (1 )) 01 S(t) dW (t)
0
E(ψ
R (
R 1 ))
E1/2 (ψ 2 (1 )) 01 0t S(s) ds dW (t)
E(ψ 0 (1 ))

where
R1

Γ2 =

S 2 (t) dt
0
R1
Rt
S (t) 0 S (s) ds dt
0

R1

S (t)
0
R 1 R t
0

0

Rt

S (s) ds dt
0
2
S (s) ds dt

!
.


,

Sohrabi and Zarepour: Asymptotic Theory for M-Estimates in Unstable AR(p) Processes

8

(iii) When φ(z) = 1 + 2z + z 2 ,


n

3/2

3/2

an (φ̂1 + 2) − n an (φ̂2 + 1)
n1/2 an (φ̂2 + 1)




d


→ −Γ−1
3

R R
E1/2 (ψ 2 (ε1 )) 01 0t S (1) (s) ds dW (t)
E(ψR0 (ε1 ))
E1/2 (ψ 2 (ε1 )) 01 S (1) (t) dW (t)
E(ψ 0 (ε1 ))


,

where
!
2
R 1 (1)
R t (1)
(1)
S
(s)
ds
dt
S
(t)
S
(s)
ds
dt
0
0
0
0
.
R 1 (1)
R t (1)
R 1 (1) 2
S
(t)
S
(s)
ds
dt
S
(t)
dt
0
0
0
R 1 R t

Γ3 =

(iv) When φ(z) = 1 − z 2 ,


n1/2 an φ̂1 /2 + n1/2 an (φ̂2 − 1)/2
n1/2 an φ̂1 /2 − n1/2 an (φ̂2 − 1)/2




d

→ 

R
E1/2 (ψ 2 (ε1 )) 01 S(t) dW (t)
R1
E(ψ 0 (ε1 )) 0 S 2 (t) dt
R
E1/2 (ψ 2 (ε1 )) 01 S (1) (t) dW (t)
R1
−
2
E(ψ 0 (ε1 )) 0 (S (1) (t)) dt


,

where S(·), S (1) (·), T(·), W (·), and R(·) are defined in (2.6).
Remark 1. Example 1 shows a higher convergence rate of the M-estimates compared to the classical LS estimates in Chan and Zhang (2012). For instance, the rate of convergence of the M-estimates
in case (i) is n1/2 an which is significantly higher than that of LS estimate at the rate of n, specially
for small values of α. Moreover, the results in Example 1, cases (ii) and (iii), show different convergence rates for the estimators. Broadly speaking, the linear combination of φ̂1 and φ̂2 converges
faster than each of them individually; see Chan and Zhang (2012).

4

Simulation

To investigate the results given in Theorem 1, we carry out a simulation for model (1.1), when p = 2
and φ(B) = 1 − φ1 B − φ2 B 2 has complex conjugate unit roots. In other words, we illustrate the
asymptotic result of case (i) in Example 1 by simulating the following AR(2) process
Xt = 2 cos θXt−1 − Xt−2 + t ,

(4.1)

where the innovations are i.i.d. symmetric α-stable random variables. Figure 1 shows different
sample paths of Model (4.1) when α = 1.3 and n = 500. For the sake of brevity, we only present
the M-estimates for φ1 when θ = π/4. The simulations for the other unstable cases are similar.
In our simulation study, we consider the time series {Xt }nt=0 in model (4.1), for n = 10, 20,
30, 40, and 50 with α = 0.5, 1, 1.3, 1.7 and 2. The time series are generated 10,000 times for
each choice of n and α. The M-estimates of φ1 in AR(2) are calculated for each replicate. Here,
we use the Huber loss function, ρH (x) in (2.1), with c = 5 in all cases. The sample median and
the sample 90% inter-percentile range (IPR) (95th percentile - 5th percentile) for |φ̂1 − 2 cos θ| are

Sohrabi and Zarepour: Asymptotic Theory for M-Estimates in Unstable AR(p) Processes

9

Figure 1: Different sample paths for model (4.1) when α = 1.3 and n = 500.
tabulated in Table 1. Furthermore, the simulated results of the sample median and 90% IPR for
|φ̂1 − 2 cos θ| with the LS method are shown in Table 2. As seen in Tables 1 and 2, the M-estimates
are significantly closer to the actual values as n gets larger compared to the LS estimates, especially
for small values of α. Note that, when α = 2, the rate of convergence of both M-estimate and LS
estimate are the same. Moreover, the 90% IPR value for the M-estimates are significantly smaller
than those for the LS estimates. The tables confirm that M-estimates are more precise than LS
estimates.
Table 1: Median and 90% IPR (in parentheses) for |φ̂1 − 2 cos θ| in model (4.1) by the M-estimate
method using the Huber loss function
n
10
20
30
40
50

5

0.5
0.0258(0.4141)
0.0030(0.0815)
0.0009(0.0245)
0.0004(0.0107)
0.0002(0.0063)

1
0.1170(0.4097)
0.0342(0.2494)
0.0180(0.1367)
0.0115(0.0841)
0.0083(0.0584)

Index of stability α
1.3
0.1501(0.4048)
0.0552(0.2981)
0.0328(0.1797)
0.0216(0.1234)
0.0159(0.0916)

1.7
0.1839(0.4003)
0.0734(0.3456)
0.0444(0.2133)
0.0329(0.1628)
0.0263(0.1278)

2
0.1947(0.3961)
0.0852(0.3673)
0.0538(0.2323)
0.0391(0.1788)
0.0318(0.1396)

Bootstrapping

The asymptotic distributions for the proposed estimator of Φ = (φ1 , . . . , φp )| by Chan and Zhang
(2012) and this article are generally intractable. Due to the complexity of the limiting distributions,

Sohrabi and Zarepour: Asymptotic Theory for M-Estimates in Unstable AR(p) Processes

10

Table 2: Median and 90% IPR (in parentheses) for |φ̂1 − 2 cos θ| in model (4.1) by the LS estimate
method
n
10
20
30
40
50

0.5
0.0824(2.0654)
0.0297(0.3620)
0.0180(0.2038)
0.0133(0.1323)
0.0105(0.1070)

1
0.1439(0.9341)
0.0543(0.3295)
0.0351(0.2224)
0.0257(0.1567)
0.0202(0.1247)

Index of stability α
1.3
0.1633(0.7900)
0.0660(0.3382)
0.0429(0.2200)
0.0311(0.1622)
0.0243(0.1259)

1.7
0.1867(0.7794)
0.0765(0.3541)
0.0477(0.2223)
0.0362(0.1719)
0.0286(0.1351)

2
0.194(0.7595)
0.0852(0.3674)
0.0538(0.2323)
0.0391(0.1788)
0.0318(0.1397)

to make inferences based on φ̂1 , . . . , φ̂p , one may consider a resampling scheme. In this section,
we briefly suggest using the result of Theorem 1 of Moreno and Romo (2012) and Theorem 3.1 of
Zarepour and Knight (1999a) in model (1.1), when {t } is a sequence of i.i.d. random variables
and belongs to the domain of attraction of a stable law with index α. Define the point process
n
P
ξn =
ε(k/n,a−1
, where εx is a measure defined by εx (A) = I(x ∈ A) for any Borel set A
n k )
k=1
such that A ⊆ [0, 1] × R. From Resnick (1987), we have
n
X

∞
X

d

ε(i/n,a−1
→
n i )

i=1

εU ,δ Γ−1/α  ,

i=1

i

(5.1)

i i

i.i.d.

d

where Ui ∼ U [0, 1] is independent from {δi , Γi }, which are defined in (2.4). Here → indicates weak convergence with respect to vague topology,
P see Resnick (1987). Now, assume that
{∗1 , ∗2 , · · · , ∗n } is an i.i.d. sample from Fn (x) = n1 ni=1 I(i ≤ x). Then, the bootstrapped point
process
n
X

d

ε(i/n,a−1
∗ ([0, t] × ·) →
n  )

∞
X

i

i=1

i=1

P∗i (t)εU ,δ Γ−1/α  (·)
i

(5.2)

i i

in distribution. Here, {P∗i (t)} is a sequence of independent Poisson processes with Lebesgue
mean measure and {δi , Γi } are the same as before. Note that in this case, the regular bootstrap
asymptotically fails for the bootstrapped point process since the limit in (5.2) contains the extra
Poisson point process P∗i (t). Subsampling of size m when m/n → 0 as n → ∞ resolves the
asymptotic failure of the regular bootstrap; see Zarepour and Knight (1999b). By the same argument used for the stationary AR(p) processes with infinite variance in Davis and Wu (1997),
we can prove that the results of the limiting distributions are asymptotically valid when we use
the bootstrap scheme with m = o(n) resampling sample size. Given (X1 , . . . , Xn ), we find Mestimates of Φ in model (1.1) using the objective function in (2.1). The residuals are calculated
from et = Xt − φ̂1 Xt−1 − · · · − φ̂p Xt−p , t = p + 1, 2, . . . , n. Then, the bootstrap replicate
∗ ) are generated from X ∗ = φ̂ X ∗ + · · · + φ̂ X ∗ + e∗ , where e∗ , . . . , e∗ is a sam(X1∗ , . . . , Xm
1 t−1
p t−p
t
t
m
1
1 Pn
1 Pn
ple of size m from F̂n (·) = n−p i=p+1 I(ei − ē ≤ ·), where ē = n−p i=p+1 ei . The bootstrap

Sohrabi and Zarepour: Asymptotic Theory for M-Estimates in Unstable AR(p) Processes

11

replicate of Φ̂ is found by minimizing
A∗m (q1∗ , . . . , qp∗ ) =

m h
X

i
∗
∗ −1 ∗
∗
ρ(e∗t − q1∗ b−1
X
−
·
·
·
−
q
b
X
)
−
ρ(e
)
,
m1 t−1
p mp t−p
t

t=p+1




with the minimum given by q̂ ∗m = Bm Φ̂∗m − Φ̂ , where Bm = diag bm1 , . . . , bmp is the matrix
of appropriate normalizing constants. The following lemma helps to derive the limiting distribution
of bootstrap estimates.
Lemma 1. Let {e∗1 , . . . , e∗m } be an i.i.d. sample from F̂n and E∗ denotes the expectation under F̂n .
Also, under conditions A1-A4 and with the subsampling of size m where m → ∞ and m/n → 0,
we have
P[m·] ∗ d
∗ (·) = a−1
(i) Sm
m
i=1 ei → S(·) in probability, where S(·) is the stable process defined in
(2.4),
(ii) E∗ (ψ(e∗1 )) = 0,
P[m·]
d
(iii) m−1/2 σ̂ −1 i=1 ψ(e∗i ) → W (·) in probability, where W (·) is a standard Brownian motion
p
1 Pn
2
2
independent of S(·) and σ̂ 2 = E∗ (ψ 2 (e∗1 )) = n−p
i=p+1 ψ (ei ) → E(ψ (1 )),
0

p

0

(iv) E∗ (ψ (e∗1 )) → E(ψ (1 )).
We omit the proof of Lemma 1 since it follows from a similar argument used in Proposition 4
of Moreno and Romo (2012) and Arcones and Giné (1989). See also Sohrabi and Zarepour (2016).
From Lemma 1, we can conclude that
∗
∗(1)
∗
Sm
(·), Sm
(·), T∗m (·), Wm
(·), R∗m (·), Vm∗ (·)

|

d

→ S(·), S (1) (·), T(·), W (·), R(·), V (·)

|

n
o
∗ (·) is defined as in Lemma 1 and S ∗(1) (·), T∗ (·), W ∗ (·), R∗ (·), V ∗ (·)
in probability. Here Sm
m
m
m
m
m

is defined in (2.5) by replacing i by e∗i and n by m. Also, S(·), S (1) (·), T(·), W (·), R(·), V (·)
is as specified in (2.6). Similar to Theorem 3.1 of Davis and Wu (1997), along with applying the
continuous mapping theorem, and Lemma 1 we have

w
−1
|
| |
(Q| G|m )−1 (Φ̂∗ − Φ̂) → (Γ−1 F)| , (Υ−1 H)| , (Λ−1
1 G1 ) , . . . , (Λl Gl )
in probability, where (Γ−1 F), (Υ−1 H), and (Λ−1
i Gi ) for i = 1, . . . , l are specified as in Theorem
1. For a detailed proof of the asymptotic validity of the m out of n bootstrap in a special case see
Sohrabi and Zarepour (2016). Also see Zarepour and Knight (1999a, b), Moreno and Romo (2012),
Davis and Wu (1997), and Sohrabi (2016) for some similar techniques. An illustration of the validity of the bootstrap scheme is given by a simulation study in the following example.

Sohrabi and Zarepour: Asymptotic Theory for M-Estimates in Unstable AR(p) Processes

12

Example 2. We consider bootstrapping for the parameter φ1 in model (4.1) when θ = π/4. We
generate the time series {Xt }nt=0 in model (4.1), for the actual sample sizes n = 50, 100, and 200
with α = 1.3, and 1.7 (the cases with α > 1 are of practical interest). The behavior of the other
unstable cases are more or less similar. Then we implement the following algorithm for each choice
of n and α.
(i) We find M-estimates of the parameters φ1 and φ2 in model (4.1) using the Huber loss function
with c = 5. Then take
ˆt = Xt − φ̂1 Xt−1 − φ̂2 Xt−2 .
(ii) We draw a sample of size m from centered residuals denoted by ˆ∗1 , . . . , ˆ∗m , and find {Xt∗ }m
t=0
from (4.1). Then we estimate the parameter φ1 using the bootstrap observations by the same
minimization of the objective function used in step (i).
3,000∗
. To find a naive 95% confidence
(iii) We repeat step (ii) B = 3, 000 times to get φ̂1∗
1 , . . . , φ̂1
interval for φ1 , we obtain the 2.5th and 97.5th percentiles of 3,000 bootstrap estimates as the
lower and upper bound of our confidence interval.

In order to compute the coverage rate of the bootstrap confidence intervals, the original time series are generated 10,000 times for each choice of n and α. Then by applying (i)-(iii), the naive
95% bootstrap confidence interval for φ1 is calculated for each replicate. Moreover, to study how
the selection of the resampling size would affect our estimation, we perform the second step with
three different resampling sizes m = n/ ln(ln(n)), n0.9 , and n0.95 . Finally, the resulting coverage percentages of the naive 95% bootstrap confidence intervals for the parameter φ1 for different
values of m, n, and α are presented in Table 3. This table shows that the coverage percentages of
the naive 95% bootstrap confidence intervals for φ1 are very close to 95%. This illustrates that the
bootstrap scheme with m = o(n) resampling sample size is approximately valid when we have a
non-stationary time series with innovations in the domain of attraction of a stable law. The simulation procedure shows m = n0.95 performs consistently well in all our cases.
Table 3: Coverage for the naive 95% bootstrap confidence interval for φ1 in model (4.1)
α
n
m = n/ ln(ln(n))
m = n(0.9)
m = n(0.95)

6

50
96.1%
97.2%
95.7%

1.3
100
96.5%
97.4%
96.6%

200
97.4%
97.7%
96.3%

50
95.1%
96.6%
94.0%

1.7
100
96.5%
96.9%
94.8%

200
96.8%
97.4%
94.8%

Conclusion

In this paper, the asymptotic properties of the M-estimate have been thoroughly studied for the
unstable AR(p) processes when {t } is a sequence of random variables in the domain of attraction
of a stable law with index 0 < α ≤ 2. The behavior of these time series with several real and

Sohrabi and Zarepour: Asymptotic Theory for M-Estimates in Unstable AR(p) Processes

13

complex conjugate roots on the unit circle, is completely different from processes with a single unit
root. The robust M-estimate method has been used to drive the limiting distribution of estimates of
the parameters in the AR(p) model defined in (1.1). Although the M-estimates have a better rate of
convergence compared to LS estimates, the limiting distributions are not computationally tractable.
To remedy this difficulty, we suggest a valid m out of n bootstrap where m/n → 0 as n → ∞. Our
simulation study proves that the resampling bootstrap scheme is valid for our non-stationary time
series when errors are in the domain of attraction of a stable law.
Based on our analysis, when errors are heavy tailed, M-estimators are always superior to LS
estimators. This superiority holds for LAD but some extra conditions should be imposed on innovations. In this case, Knight (1989) and Davis et al. (1992) assume that the innovations need to have
0 median and a density with respect to Lebesgue measure. Since the computational complexity of
both M-estimates and LAD estimates are similar, we do not consider the LAD estimate to avoid
imposing extra conditions.

REFERENCES
[1] Arcones, M. A. and Giné, E. (1989). The bootstrap of the mean with arbitrary sample size.
Annales de l’Institut Henri Poincaré, 25, 457-481.
[2] Banjevic, D., Ishwaran, H. and Zarepour, M. (2002). A recursive method for functionals of
Poisson processes. Bernoulli, 8, 295-311.
[3] Chan, N.H. and Tran, L.T. (1989). On the first order autoregressive process with infinite variance. Econometric Theory, 5, 354-862.
[4] Chan, N.H. and Wei, C.Z. (1988). Limiting distribution of least square estimates of unstable
autoregressive processes. Annals of Statistics, 16, 367-401.
[5] Chan, N.H. and Zhang, R.M. (2012). Non-stationary autoregressive processes with infinite
variance. Journal of Time Series Analysis, 33, 916-934.
[6] Davis, R.A., Knight, K. and Liu, J. (1992). M-estimation for autoregressions with infinite
variance. Stochastic Processes and Their Applications, 40, 145-180.
[7] Davis, R.A., Wu, W. (1997). Bootstrapping M-estmaites in regression and autoregression with
infinite varianve. Statistica Sinica, 7, 1135-1154.
[8] Dickey, D.A. and Fuller, W. (1979). Distribution of the estimators for autoregressive time
series with a unit root. Journal of the American Statistical Association, 74, 427-431.
[9] Engle, R. F. and Granger, C. W. J. (1987). Co-integration and error correction: Representation,
estimation and testing. Econometrica, 55, 251-276.

Sohrabi and Zarepour: Asymptotic Theory for M-Estimates in Unstable AR(p) Processes

14

[10] Jeganathan, P. (1991). On the asymptotic behavior of least-squares estimators in AR time series
with unit roots near the unit circle. Econometric Theory, 7, 269-306.
[11] Knight, K. (1989). Limit theory for autoregressive-parameter estimates in an infinite-variance
random walk. Canadian Journal of Statistics, 17, 261-278.
[12] LePage, R., Woodroofe, M. and Zinn, J. (1981). Convergence to a stable distribution via order
statistics. Annals of Probability, 9, 624-632.
[13] Moreno, M. and Romo, J. (2012). Unit root bootstrap tests under infinite variance. Journal of
Time Series Analysis, 33, 32-47.
[14] Paulauskas, V, and Rachev, S.T. (1998). Cointegrated processes with infinite variance innovations. Annals of Applied Probability, 8, 775-792.
[15] Park, J. Y. and Phillips, P. C. B. (1988). Statistical inference in regressions with integrated
processes: part I. Econometric Theory, 4, 468-497.
[16] Phillips, P. C. B. (1990). Time series regression with a unit root and infinite-variance errors.
Econometric Theory, 6, 44-62.
[17] Phillips, P. C. B. and Perron, P. (1988). Testing for a unit root in time series regression.
Biometrika, 75, 335-346.
[18] Resnick, S. and Greenwood, P. (1979). A bivariate stable characterization and domain of attraction. Journal of Multivariate Analysis, 9, 206-221.
[19] Resnick, S.I. (1986). Regular Variation and Weak Convergence. Applied Probability Trust, 18,
66-138.
[20] Resnick, S.I. (1987). Extreme Values, Regular Variation and Point Processes. Springer-Verlag,
New York.
[21] Samarakoon, D.M. and Knight, K. (2009). A note on unit root tests with infinite variance noise.
Econometric Reviews, 28, 314-334.
[22] Sohrabi, M. (2016). On Robust Asymptotic Theory of Unstable AR(p) Processes with Infinite
Variance Innovations. Ph.D. thesis, Department of Mathematics and Statistics, University of
Ottawa, 120 pp.
[23] Sohrabi, M. and Zarepour, M. (2016). Bootstrapping M-estimates in Unstable AR(2) Processes
with Infinite Variance Innovations. Available at http://arxiv.org/abs/1603.02665.
[24] Zarepour, M. and Knight, K. (1999a). Bootstrapping unstable first order autoregressive process
with errors in the domain of attraction of stable law. Stochastic Models, 15, 11-27.
[25] Zarepour, M. and Knight, K. (1999b). Bootstrapping point processes with some applications.
Stochastic processes and their applications, 84, 81-90.

Sohrabi and Zarepour: Asymptotic Theory for M-Estimates in Unstable AR(p) Processes

7

15

Appendix A

To facilitate the proof of Theorem 1, we first present the following two lemmas. The proof of
Lemma 2 is similar to the proof given in Theorem 1 of Banjevic, Ishwaran and Zarepour (2002).
Lemma 2. Let (Z1,k , Z2,k )| be a sequence of symmetric i.i.d. random vectors on Z2 = {(z1 , z2 )| :
z12 + z22 = 1} with a probability distribution Q on the boundary of the unit circle, and

X ∞
X∞
−1/α
−1/α |
Z2,k Γk
Z1,k Γk
,
.
X=
k=1

k=1

Then the characteristic function for X is φ(s) = exp [−KE (|s1 Z1,1 + s2 Z2,1 |α )], where s =
(s1 , s2 ), and K is given by
 cos(πα/2)Γ(1 − α)
if 0 < α < 1,
if α = 1,
K = π(2 − α)/(2α)
cos(πα/2)Γ(3 − α)/(α2 − α)

if 1 ≤ α < 2.

where {Γk } is defined in (2.4) and is independent from {Zi,k }, i = 1, 2.
Lemma 3. Let Tn be defined by (2.5), then
d

Tn (·) → T(·),
where T(·) = (T1 (·), T2 (·))| is a bivariate stable process with index α.
Proof. Consider the point process convergence in (5.1). By applying the continuous mapping
theorem, for any θ ∈ (0, 2π), we have
n
X

d

ε(cos(θk/n),sin(θk/n),a−1
→
n k )

k=1

∞
X

εcos(θU

k=1

−1/α
k ),sin(θUk ),δk Γk

.

Similar to Resnick’s (1986) Proposition 3.4, it can be shown that
a−1
n

[nt]
X

d

(cos (θk/n) , sin (θk/n)) k →

k=1

∞
X

−1/α

(cos (θUk ) , sin (θUk )) δk Γk

I(Uk ≤ t).

(7.1)

k=1

Notice that we can easily find the finite dimensional distribution for the limiting process in (7.1).
For instance, for t = 1 and by applying Lemma 2 where (Z1,k , Z2,k ) = (cos(θUk )δk , sin(θUk )δk ),
and p = q = 1/2, we have

Z
φ(s) = exp −K

1


|s1 cos(θu) + s2 sin(θu)|α du .

0


Proof of Theorem 1 with Roots equal to 1. First, we consider the unit root model
(1 − B)r ut (·) = t .

Sohrabi and Zarepour: Asymptotic Theory for M-Estimates in Unstable AR(p) Processes

16

To avoid singular limiting distributions, similar to Chan and Wei (1988), we define
ut (j) = (1 − B)r−j ut for j = 1, 2, . . . , r,

(7.2)

P
P
or equivalently ut (1) = tj=1 j and ut (j) = tk=1 uk (j − 1) for j = 2, . . . , r. From definition
(2.2), we have
d
a−1
n u[nt] (1) = Sn (t) =: S1,n (t) → S(t) =: S1 (t).
P
Since ut (j) = tk=1 uk (j − 1) for j = 2, . . . , r, the continuous mapping theorem implies that
−(j−1)
a−1
u[nt] (j)
n n

Z
=

t

d

Sj−1,n (s)ds =: Sj,n (t) → Sj (t)

(7.3)

0

for j = 2, . . . , r. We also define
Jn = Nn−1 C,

(7.4)

where


1
1
..
.
1


C=

0
−1
..
.

(−1) r−1
1

0
0
..
.
(−1)2

r−1
2

···
···
..
.
···

0
0
..
.
(−1)r−1






r−1/2 a , n(r−1)−1/2 a , . . . , n1/2 a .
and the appropriate P
normalizing constant is N
=
diag
n
n
n
n
n
 |
n
| 0
th
First, notice that C
t=r+1 ut−1 ut−1 ψ (t ) C = Π, where the (i, j) entry of Π is
πi,j =

n
X

ut−1 (r − (i − 1)) ut−1 (r − (j − 1)) ψ 0 (t ) for i, j = 1, 2, . . . , r.

t=r+1

The joint behavior of Π can be studied through some tedious calculations. Therefore, for simplicity,
we only calculate the limiting behavior of each term of Π individually by using the following steps.
As discussed before, each ψ 0 (t ) can be replaced by E (ψ 0 (t )) when n → ∞. To see this, note that
n
X

ut−1 (i)ut−1 (j)ψ 0 (t ) =

t=r+1

=

n
X
t=r+1
n
X

ut−1 (i)ut−1 (j) [ψ 0 (t ) − E (ψ 0 (t )) + E (ψ 0 (t ))]
n
X

ut−1 (i)ut−1 (j) [ψ 0 (t ) − E (ψ 0 (t ))] + E (ψ 0 (1 ))

t=r+1

ut−1 (i)ut−1 (j)

t=r+1

(7.5)

for i, j = 1, 2, . . . , r. Notice that by (2.5) and (7.3), the first term on the right side of (7.5) is
Op (a2n n(i+j−3/2) ). Then we have
−(i+j−1)
a−2
n n

n
X
t=r+1

 p
ut−1 (i)ut−1 (j)[ψ 0 (t ) − E ψ 0 (t ) ] → 0.

(7.6)

Sohrabi and Zarepour: Asymptotic Theory for M-Estimates in Unstable AR(p) Processes

17

From (7.3), along with applying the continuous mapping theorem and Proposition 2 of Paulauskas
and Rachev (1998), we can find the limiting of each term of the matrix Π where
Z
n
X
 1
d
−2 −(i+j−1)
0
0
Si (t)Sj (t)dt for i, j = 1, 2, . . . , r.
an n
ut−1 (i)ut−1 (j)ψ (t ) → E ψ (1 )
0

t=r+1

Therefore,
Jn

n
X

!
|

d

0

ut−1 ut−1 ψ (t ) Jn| → Γ,

t=r+1

where Γ = (γi,j ) is a r × r random matrix such that
Z
 1
0
γi,j = E ψ (1 )
Sr−(i−1) (t)Sr−(j−1) (t)dt for i, j = 1, 2, . . . , r.

(7.7)

0

Moreover, note that
Jn

n
X

d

ut−1 ψ(t ) → F,

t=r+1

where
F=E

1/2

2

ψ (1 )



Z

1

Sr (t)dW (t),

Z

0

1

Sr−1 (t)dW (t), . . . ,

0

Z

1

|
S1 (t)dW (t) .

(7.8)

0

Now, summarize the results to obtain the following limiting distribution
!−1 n
n
X
X
d
(Jn| )−1
ut−1 ut−1 | ψ 0 (t )
ut−1 ψ(t ) → Γ−1 F,
t=r+1

(7.9)

t=r+1

where Γ = (γi,j )r×r and F, respectively, are defined in (7.7) and (7.8).



Proof of Theorem 1 with Roots equal to -1. Consider the following model
(1 + B)s vt = t .
The limiting distribution in this case is similar to the case in which the time series has unit root 1,
except that i is replaced by (−1)i i for i = 1, 2, . . . , t. Similar to Chan and Wei (1988), we define
vt (j) = (1 + B)s−j vt for j = 1, 2, . . . , s.
(7.10)
P
Notice that (7.10) implies that (−1)t vt (j + 1) = ti=1 (−1)i vt (j). By (2.5) and (2.6), we have
(1)

d

(1)

t
(1)
(1)
a−1
n (−1) v[nt] (1) = Sn (t) =: S1,n (t) → S (t) =: S1 (t),

which implies that
−(j−1)
a−1
(−1)t vt (j)
n n

Z
=
0

t

(1)

(1)

d

(1)

Sj−1,n (s)ds =: Sj,n (t) → Sj (t)

Sohrabi and Zarepour: Asymptotic Theory for M-Estimates in Unstable AR(p) Processes

18

for j = 2, . . . , s. Then, let
Kn = ℵ−1
n Ç,

(7.11)

where

Ç


=

1
1
..
.
1

0
1
..
.
s−1

0
0
..
.
s−1

1

2

···
···
..
.
···

0
0
..
.
1






and the appropriate normalizing constant is ℵn = diag ns−1/2 an , n(s−1)−1/2 an , . . . , n1/2 an .
Therefore, similar to the case with root 1, we have
!
n
X
d
| 0
Kn
vt−1 vt−1 ψ (t ) Kn| → Υ,
t=s+1

where Υ = (υi,j ) is a s × s random matrix whose entries are as follows:
Z
 1 (1)
(1)
0
υi,j = E ψ (1 )
Ss−(i−1) (t)Ss−(j−1) (t)dt for i, j = 1, 2, . . . , s.

(7.12)

0

We also have
Kn

n
X

d

vt−1 ψ(t ) → H,

t=s+1

where
H = −E

1/2

2


ψ (1 )

1

Z

S(1)
s (t)dW (t),

Z

0

0

1

(1)
Ss−1 (t)dW (t), . . . ,

Z

1

0

|

(1)
S1 (t)dW (t)

.

(7.13)

Finally, we get
n
X

−1
(Kn| )

!−1
|

0

vt−1 vt−1 ψ (t )

t=s+1

n
X

d

vt−1 ψ(t ) → Υ−1 H,

(7.14)

t=s+1

where Υ = (υi,j )s×s and H are defined in (7.12) and (7.13), respectively.



Proof of Theorem 1 with Complex Conjugate Unit Roots. Consider the model
(1 − 2 cos θB + B 2 )d wt = t .
Similar to Chan and Wei (1988), let yt (j) = (1 − 2 cos θB + B 2 )d−j wt for j = 1, 2, . . . , d and Yt =
(yt (1), yt−1 (1), . . . , yt−1 (d), yt−1 (d))| . Therefore, we have (1 − 2 cos θB + B 2 )yt (j + 1) = yt (j), and
yt (j + 1) =

t
1 X
sin (θ(t − k + 1)) yk (j) for j = 0, 1, . . . , d − 1.
sin θ
k=1

Sohrabi and Zarepour: Asymptotic Theory for M-Estimates in Unstable AR(p) Processes

19

We can find a 2d × 2d matrix D such that Dwt = Yt , where wt = (wt , . . . , wt−2d+1 ). For more details see
Chan and Wei (1988). By applying trigonometric identities, we have
sin(θ)yt (j) = an sin ((t + 1)θ) T1,t (j − 1) − an cos ((t + 1)θ) T2,t (j − 1),
(7.15)
Pt
Pt
−1
where T1,t (j) = a−1
n
k=1 cos(kθ)yk (j) and T2,t (j) = an
k=1 sin(kθ)yk (j). Note that by Lemma 3,
we have
| d

(T1,t (0), T2,t (0)) → T(·) = (T1 (t), T2 (t))| ,

(7.16)

where T(·) is defined in Lemma 3. Now, consider the following representation:
 Pn

t=2d+1

n
X

 .
wt−1 wt−1 | ψ 0 (t ))D| = 
D(
 ..
Pn
t=2d+1

t=2d+1

yt2 (1)ψ 0 (t )

···
..
.
···

yt−1 (d)yt (1)ψ 0 (t )

Pn

t=2d+1

..
.P
n

t=2d+1

yt (1)yt−1 (d)ψ 0 (t )
2 (d)ψ 0 ( )
yt−1
t



.


 |
Pn
| 0
To find the limiting distribution of D
t=2d+1 wt−1 wt−1 ψ (t ) D , by using Lemma 3.3.6. of Chan
and Wei (1988) and from Proposition 8 of Jeganathan (1991), we have
sup
0<j≤n

j
X

eikθ Ti,n

k=1

 
k
= op (n)
n

(7.17)

for i = 1, . . . , d. Moreover, by letting
Ln = Mn−1 D,

where Mn = diag n1/2 an I, . . . , n(d−1)/2 an I and I = diag (1, 1), we have
!
n
X
d
| 0
Ln
wt−1 wt−1 ψ (t ) L|n → Λ.

(7.18)

t=2d+1

Here Λ = (λi,j ) is a 2d × 2d random matrix, where λ2i−1,2j−1 = λ2i,2j and λ2i−1,2j = λ2i,2j−1 for
i, j = 1, 2, . . . , 2d. By using (7.16) and (7.17) along with the continuous mapping theorem, we can show that
λi,j is presented as follows:
Z 1

Z 1
E (ψ 0 (1 ))
λ2i−1,2j−1 = λ2i,2j =
×
T1,t (i − 1)T1,t (j − 1)d(t) +
T2,t (i − 1)T2,t (j − 1)d(t) ,
2
0
0
E (ψ 0 (1 ))
λ2i−1,2j = λ2i,2j−1 =
2

Z 1

Z 1
× cos θ
T1,t (i − 1)T1,t (j − 1)d(t) +
T2,t (i − 1)T2,t (j − 1)d(t)
Z

0
1

− sin θ

Z

0
1

T1,t (i − 1)T2,t (j − 1)d(t) −
0


T1,t (j − 1)T2,t (i − 1)d(t) .

0

(7.19)
Also, we have
Ln

n
X
t=2d+1

d

wt−1 ψ(t ) → G,

Sohrabi and Zarepour: Asymptotic Theory for M-Estimates in Unstable AR(p) Processes

20

where G = (G1 , G2 , . . . , G2d )| . Note that Gi for i = 1, . . . , 2d can be expressed as follows:
G2i−1 = E1/2 ψ 2 (1 )



1


Z
cos θ

1

Z
T1,t (j − 1)dR2 (t) +

+ sin θ
0

G2i = E

1/2

2



1


T2,t (j − 1)dR1 (t) ,

0

1

1

Z
T1,t (j − 1)dR1 (t) −

ψ (1 ) ×


T2,t (j − 1)dR2 (t)

0

0

Z
Z

1

Z
T1,t (j − 1)dR1 (t) −

0


T2,t (j − 1)dR2 (t) ,

(7.20)

0

where R = (R1 (·), R2 (·))| is defined in (2.7). Thus, we have the following result:
−1
(L|n )

n
X
t=2d+1

!−1
|

0

wt−1 wt−1 ψ (t )

n
X

d

wt−1 ψ(t ) → Λ−1 G,

(7.21)

t=2d+1

where Λ = (λi,j ) and G = (G1 , G2 , . . . , G2d )| are defined in (7.19) and (7.20), respectively.



Proof of Theorem 1 with cross product terms. Chan and Zhang (2012) show that the limiting distributions
of the cases involving the cross product terms converge to zero in probability. We skip the proof of this case,
as it is similar to their proof.


