Noname manuscript No.
(will be inserted by the editor)

On consistency of the likelihood moment estimators
for a linear process with regularly varying
innovations

arXiv:1507.03429v2 [math.ST] 25 May 2016

Lukas Martig · Jürg Hüsler

Received: 8 July 2015 / Accepted: date

Abstract In 1975 James Pickands III showed that the excesses over a high
threshold are approximatly Generalized Pareto distributed. Since then, a variety of estimators for the parameters of this cdf have been studied, but always
assuming the underlying data to be independent. In this paper we consider the
special case where the underlying data arises from a linear process with regularly varying (i.e. heavy-tailed) innovations. Using this setup, we then show
that the likelihood moment estimators introduced by Zhang (2007) are consistent estimators for the parameters of the Generalized Pareto distribution.
Keywords Generalized Pareto distribution · Linear processes · Heavy-tailed
data · Likelihood moment estimators · Consistency
Mathematics Subject Classification (2010) 60G50 · 60G70 · 62G32

1 Introduction
A random variable X has a heavy right or left tail if there exists a positive
parameter γ, called extremal index, such that as x → ∞:
P(X > x) ∼ x−1/γ L(x)

or

P(X < −x) ∼ x−1/γ L(x),

(1.1)

where L(x) is a slowly varying function satisfying L(yx) ∼ L(x) for any
y > 0. Here we use the common notation g(x) ∼ f (x) as x → ∞ for
limx→∞ g(x)/f (x) = 1. Since heavy-tail analysis is a special branch of extreme
L. Martig
Sidlerstrasse 5, 3012 Bern, Switzerland
Tel.: +41-31-6318811
Fax: +41-31-6313870
E-mail: lukasmartig@gmail.com
J. Hüsler
Sidlerstrasse 5, 3012 Bern, Switzerland

2

Lukas Martig, Jürg Hüsler

value theory, there are basically two main approaches for modeling extreme
events:
The first approach – the more traditional one introduced by Fisher & Tippet
(1928, [6]) – is based on the Generalized
Extreme Value distribution (GEVD)

Gγ (x) := exp −(1 + γx)−1/γ , 1 + γx > 0, and is used to model block extremes, i.e. maxima (or minima) observed within a pre-defined period. Actually, as γ is assumed to be positive in (1.1), the GEVD simply reduces to the
Fréchet distribution in this special setup.
The second and more recent approach – the one being discussed in this paper
– is to model exceedances over high thresholds. This kind of modeling was
introduced by Pickands (1975, [13]) and is based on the Generalized Pareto
−1/γ
, where 1 + γx/σ ∗ > 0
distribution (GPD) Hγ,σ∗ (x) := 1 − (1 + γx/σ ∗ )
∗
∗
and σ = σ (t) is a suitable scale function. Notice that as before, the GPD
reduces to the Pareto distribution in case γ > 0.
Classical studies on how to fit a GPD (see section 2 for references) usually cover
the simplest case when the underlying data are i.i.d. Though the assumption
of independence has its practical advantages, most collected data sets have
non-negligible dependence structures which need to be included in statistical
modeling. An easy but quite powerful way to model such a dependence is the
use of so-called autoregressive moving average (ARMA) time series models. In
detail, a stationary time series {Xn }n∈Z0 is called an ARMA(p,q)-process if it
has additive innovations {Zn }n∈Z0 satisfying the following recurrence formula:
Xn − φ1 Xn−1 − . . . − φp Xn−p = Zn + θ1 Zn−1 + . . . + θq Zn−q ,

(1.2)

with φ1 , . . . , φp ; θ1 , . . . , θq ∈ R and p, q ∈ N0 . From [3], Theorem 3.1.1, it then
follows that if φ(z) has no root on the unit circle and φ(z) and θ(z) have no
common zeroes, the ARMA process in (1.2) has the causal representation
Xn =

∞
X

cj Zn−j ,

(1.3)

j=0

where the coefficients {cj }j≥0 are determined by the relation
c(z) =

∞
X

cj z j = θ(z)/φ(z),

|z| ≤ 1.

j=0

[4] studied the asymptotic behavior of linear processes (also known as MA(∞)processes) given in (1.3) for the case where the innovation’s marginal cdf has
regularly varying tails in the sense of (1.1). Resnick & Stâricâ proved that
Hill’s estimator is a consistent estimator for γ (1995, [16]) and additionally
showed its asymptotic normality (1997, [17]). In this paper, the aforementioned
results shall be extended to the case where one wants to fit a GPD instead
of simply estimating the extremal index. In section 2 we are going to fit a
special pair of estimators introduced by Zhang (2007, [21]): the likelihood

Title Suppressed Due to Excessive Length

3

moment estimators (LMEs) and analogously to [16], we show in section 3 that
the LMEs are consistent estimators for the parameters of the Generalized
Pareto distribution. In section 4 we finally discuss further steps needed to
show asymptotic normality and why the classical results for tail empirical
processes by Resnick & Stâricâ are more promising than the widely used and
powerful tools established by Drees ([5]) when studying asymptotic behavior
of linear processes.

2 Fitting a Generalized Pareto distribution for a linear process
with regularly varying tails
Let γ > 0 and {cj }j≥0 ∈ R∞ and consider a linear process
Xn =

∞
X

cj Zn−j

(A.1)

j=0

whose iid innovations Zn have a marginal cumulative distribution function
(cdf) GZ having regularly varying tails with index −1/γ, i.e.
1 − GZ (z) ∼ π1 z −1/γ L(z)

and

GZ (−z) ∼ π2 z −1/γ L(z)

as z → ∞,
(A.2)
for π1 , π2 ≥ 0, π1 + π2 = 1 and a slowly varying function L(z) satisfying
L(yz) ∼ L(z) as z → ∞ for any y > 0. Clearly, (A.2) implies 1 − G|Z| (z) ∼
z −1/γ L(z), where G|Z| is the (marginal) distribution function of |Zn |.
In order to make some statements about the marginal distribution of Xn , we
need some restrictions on the sequence of coefficients {cj }j≥0 in (A.1). We
assume that there is at least one cj 6= 0 and that for some 0 < δ < 1/γ ∧ 1:
∞
X

|cj |δ < ∞.

(A.3)

j=0

Notice that (A.3) always holds for causal ARMA processes as in (1.2) ([4], p.
341). For simplicity and due to their frequent use, (A.1)-(A.3) will be called
the Basic Assumptions.
If we denote the marginal distribution function of |Xn | by F|X| , then, with
the help of Lemma 5.2 of [4], we directly conclude that the Basic Assumptions
imply
∞
1 − F|X| (t) X
=
|ck |1/γ := ||c||.
lim
t→∞ 1 − G|Z| (t)
k=0

Under mild restrictions on the coefficients cj the tail behavior of F|X| thus
coincides with that of G|Z| up to the constant ||c|| and consequently, the
marginal distribution of the time series |Xn | has also a regularly varying tail
with index −1/γ. This fact has an important consequence: By [8], Theorem

4

Lukas Martig, Jürg Hüsler

1.1.6, it directly follows that F|X| belongs to the domain of attraction of the

Generalized Extreme Value distribution Gγ (x) = exp −(1 + γx)−1/γ and
this simply means that there exists a suitable positive function a|X| (t) such
that for t → ∞:
b|X| (tx) − b|X| (t)
xγ − 1
→
,
(2.1)
a|X| (t)
γ
where b|X| (t) := (1/(1−F|X| ))← (t) := inf{y : 1/(1−F|X| (y)) ≥ t} is the 1−1/tquantile of F|X| . On the other hand, following [13], Theorem 7, there also exists

a positive function σ ∗ (t) such that for F|X|,t (x) := P |X| − t ≤ x |X| > t :
"


lim sup F|X|,t (x) − 1 − 1 +

t→∞ x>0

−1/γ #
γ
x
= 0.
σ ∗ (t)

(2.2)

As anticipated in section 1, the function Hγ,σ∗ (x) = 1 − (1 + γx/σ ∗ (t))−1/γ is
known as the Generalized Pareto distribution with scale function σ ∗ (t). Relation (2.2) thus tells us, that the random variable |X| − t |X| > t, the excess
above a high threshold t, has approximately a Generalized Pareto distribution.
In practice, t is replaced by the (k + 1)th largest observation |X|n,n−k for a
sufficiently large k << n. Consequently, the target will be to estimate γ and
σ ∗ (t) using the k excess-values |X|n,n − |X|n,n−k , . . . , |X|n,n−k+1 − |X|n,n−k .
Up to the present day a variety of estimators have been proposed for the
two parameters γ and σ ∗ = σ ∗ (t) such as the maximum likelihood estimators ([20]), the moment and probability weighted moment estimators ([9]), the
likelihood moment estimators ([21]) or the goodness-of-fit estimators ([10]). A
nice summary is also available in [8].
In this paper only the likelihood moment estimators (LMEs) will be studied.
The reason behind this choice is that the LMEs can be considered as a generalization of two aforementioned estimators: If we choose r = −γ, we’ll get
the usual maximum likelihood equations. By choosing r = −1, we just get
the goodness-of-fit equations of [10]. As the name suggests, the calculation of
the LMEs is based on a mixture between the maximum likelihood and the
moment estimation method. In detail, we seek to solve the following system
of equations for γ and σ ∗ :
k−1


γ
1X
log 1 + ∗ (|X|n,n−i − |X|n,n−k ) = γ
k i=0
σ
k−1
r/γ
γ
1 X
1 + ∗ (|X|n,n−i − |X|n,n−k )
= (1 − r)−1 ,
k i=0
σ

(2.3)

(2.4)

where r < 1.
Defining γ/σ ∗ := θ∗ and inserting the first equation into the second one, this

Title Suppressed Due to Excessive Length

5

task reduces to the one-dimensional problem of solving the following equation
for θ∗ :
!
k−1
1
r log(1 + θ∗ (|X|n,n−i − |X|n,n−k ))
1X
−
exp 1 Pk−1
= 0. (2.5)
∗
k i=0
(1
−
r)
i=0 log (1 + θ (|X|n,n−i − |X|n,n−k ))
k

According to [21], Theorem 2.1., equation (2.5) has exactly one solution θ̂LME
for r < 1/2 and r 6= 0 which can be easily computed using a Newton-Rhapson
algorithm. (Notice that a similar procedure is performed when calculating the
maximum likelihood estimators, but the obtained target function may have
several roots which are difficult to identify; see [7]). γ̂LME then is obtained by
inserting θ̂LME in (2.3) and from there, it’s a straightforward step to derive
σ̂LME .

3 Consistency
In this section, we show that the likelihood moment estimators (the solutions
of the system of equations (2.3) and (2.4)) are consistent estimators in case
that our Basic Assumptions hold and provided that r < 0.
We begin with some preliminaries: Using standard arguments and [8], Theorem
1.1.6, one can show that the connection between the two “auxilary” functions
a|X| and σ ∗ from (2.1) and (2.2) is as follows (cf. [11]):
σ(t) := σ ∗ (b|X| (t)) = a|X| (b←
|X| (b|X| (t))) ∼ a|X| (t).

(3.1)

Combining (2.1) and (3.1), we then simply get
b|X| (tx) − b|X| (t)
xγ − 1
→
,
σ(t)
γ
which finally yields ([8], Lemma 1.2.9)
b|X| (t)
1
→ ,
σ(t)
γ

(3.2)

both convergences as t → ∞. As we will see, the (new) scaling function σ(t)
plays a key role throughout this section.
Lemma 1 Let the Basic Assumptions hold. Also, let’s denote the space of
right-continuous functions on [0, ∞) by D[0, ∞). Then for any z > 0 we have
as n, k, n/k → ∞:

P
(a) for µX (z, ∞] := k1 ni=1 1 |Xi |/b|X| (n/k) ∈ (z, ∞] and µ(z, ∞] := z −1/γ :
P

µX (z, ∞] → µ(z, ∞],

(3.3)

6

Lukas Martig, Jürg Hüsler

(b) for the (⌈ks⌉ + 1)th largest absolute observation |X|n,n−⌈ks⌉ :
|X|n,n−⌈ks⌉ P −γ
→ s
in D[0, ∞),
b|X| (n/k)

(3.4)

where ⌈ks⌉ is the smallest integer greater than or equal to ks with s > 0.
Proof The proof of (b) is a direct consequence of (a) and can simply be looked
up in [15], p. 81-83. To prove (3.3), let’s rewrite our sequence Xt as
Xt =

∞
X

cj Zt−j =

∞
X

cj sign(Zt−j )|Zt−j | :=

c′j |Zt−j |.

j=0

j=0

j=0

∞
X

Pm
(m)
Furthermore, let m ∈ N {0} and define by Xt := j=0 c′j |Zt−j | the associated (and truncated) mth order moving-average MA(m). Notice that the c′j s
are random now, but as we will see below, one can get around this problem
by conditioning on the first m + 1 innovations {sign(Zt−j )}m
j=0 .
The proof of (3.3) is now set
up
in
two
parts:
First
we
are
going
to show that
o
n
Pn
(m)
(m)
1
for µX (z, ∞] := k i=1 1 |Xi |/b|X| (n/k) ∈ (z, ∞] , the random measure
of the truncated time series, we have
Pm
1/γ
P
(m)
j=0 |cj |
µX (z, ∞] → P∞
µ(z, ∞]
(3.5)
1/γ
j=0 |cj |
as n, k, n/k → ∞. By showing (3.5), the proof of (3.3) is straightforward then.
Denote the set of all possible (and disjoint) random sequences {sign(Zt−j )}j≥0
(∞)
by S∞ . A single element of this set is further denoted by Sl , l ≥ 1. Define
(m)

(m)

X̃t,l := Xt

(∞)

|Sl

=

m
X

c′′j,l |Zt−j |,

j=0

a MA(m) process with positive iid innovations |Zt−j | consisting of determin(∞)
istic constants c′′j,l that equal either to cj or −cj , according to Sl . Finally
let
∞
X
(m)
c′′j,l |Zt−j |.
X̃ = X̃t,l := lim X̃t,l =
m→∞

j=0

If we denote the distribution function of X̃ and −X̃ by FX̃ and F−X̃ , respectively, we deduce from [16], Proposition 3.2, that for any m and a fixed l we
have as n, k, n/k → ∞:
Pm  ′′ 1/γ +
(
)
(m)
n
X̃i,l
j=0 [cj,l ]
1X
P
1
∈ (z, ∞] → P
(3.6)

+ µ(z, ∞],
k i=1
bX̃ (n/k)
∞
′′ ]1/γ
[c
j=0
j,l

Title Suppressed Due to Excessive Length
n
1X
1
k i=1

(

(m)
−X̃i,l

b−X̃ (n/k)

∈ (z, ∞]

7

)

P

Pm 
j=0

[c′′j,l ]1/γ

j=0

[c′′j,l ]1/γ

→ P

∞

−

− µ(z, ∞],

(3.7)

where bX̃ (t) and b−X̃ (t) denote the 1 − 1/t-quantiles of FX̃ and F−X̃ , respectively, and x+ := max(x, 0), x− := − min(x, 0).
The goal is now to rescale the empirical processes in (3.6) and (3.7) in order
to obtain new processes that depend on b|X| (n/k). Using Lemma 5.2 of [4], it
follows that for n, k, n/k → ∞:
∞

+
X
1 − FX̃ (n/k)
[c′′j,l ]1/γ
→
,
1 − G|Z| (n/k)
j=0

∞ 
−
X
1 − F−X̃ (n/k)
[c′′j,l ]1/γ
,
→
1 − G|Z| (n/k)
j=0

(3.8)

(3.9)

∞

X
1 − F|X| (n/k)
|cj |1/γ .
→
1 − G|Z| (n/k)
j=0

(3.10)

Repeating twice the final steps of the proof of Proposition 3.2 of [16], (3.8)(3.10) imply




P∞  ′′ 1/γ + −γ
P∞  ′′ 1/γ − −γ
[c
]
[c
]
j,l
j,l
b|X| (n/k)
b|X| (n/k)


 j=0
 j=0
→  P∞
→  P∞
 ,
 ,
1/γ
1/γ
bX̃ (n/k)
b
(n/k)
|c
|
|c
|
−X̃
j=0 j
j=0 j
both convergences as n, k, n/k → ∞, and from (3.6) and (3.7), we finally get
Pm  ′′ 1/γ +
(
)
(m)
n
X
X̃i,l
j=0 [cj,l ]
1
P
P∞
µ(z, ∞],
1
∈ (z, ∞] →
1/γ
k i=1
b|X| (n/k)
j=0 |cj |
Pm  ′′ 1/γ −
(
)
(m)
n
X
−X̃i,l
j=0 [cj,l ]
1
P
P∞
µ(z, ∞].
1
∈ (z, ∞] →
1/γ
k i=1
b|X| (n/k)
j=0 |cj |

Hence we conclude that for any m and for arbitrary l as n, k, n/k → ∞:
)
(
(m)
n
|X̃i,l |
1X
(m)
µX̃,l (z, ∞] :=
∈ (z, ∞]
1
k i=1
b|X| (n/k)
Pm
1/γ
P
j=0 |cj |
µ(z, ∞].
(3.11)
→ P∞
1/γ
j=0 |cj |
(m)

The convergence in (3.11) thus shows that the limit of µX̃,l (z, ∞] as n, k, n/k →
∞ does not depend on l any more because we only need to consider the absolute value of the |cj |s.

8

Lukas Martig, Jürg Hüsler

Next consider the set Sm of random sequences {sign(Zt−j )}m
j=0 with m ∈
m+1
N {0}. Sm consists of 2
elements and we will denote a single element by
(m)
(m)
Sl∗ , 1 ≤ l∗ ≤ 2m+1 . Clearly, for every Sl∗ there exists at least one element
(m) (m)
(m)
(∞)
(m) (∞)
in S∞ , say Sl0 , such that Xt |Sl∗ = Xt |Sl0 = X̃t,l0 and it follows
from (3.11) that for any l∗ and any ε > 0:
!
Pm
1/γ
j=0 |cj |
(m)
(m)
µ(z, ∞] > ε Sl∗
P µX (z, ∞] − P∞
1/γ
j=0 |cj |
( (m) (m)
) Pm
!
n
1/γ
|Xi | Sl∗ |
1X
j=0 |cj |
1
∈ (z, ∞] − P∞
=P
µ(z, ∞] > ε
1/γ
k i=1
b|X| (n/k)
j=0 |cj |
( (m) (∞)
) Pm
!
n
1/γ
|Xi | Sl0 |
1X
j=0 |cj |
1
∈ (z, ∞] − P∞
µ(z, ∞] > ε
=P
1/γ
k i=1
b|X| (n/k)
j=0 |cj |
!
Pm
1/γ
j=0 |cj |
(m)
µ(z, ∞] > ε → 0,
= P µX̃,l (z, ∞] − P∞
1/γ
0
j=0 |cj |

n, k, n/k → ∞. By an application of the formula of total probability it then
follows that for any m ∈ N {0}:
!
Pm
1/γ
(m)
j=0 |cj |
µ(z, ∞] > ε
P µX (z, ∞] − P∞
1/γ
j=0 |cj |
=

m+1
2X

P

(m)
µX (z, ∞]

−

l∗ =1

≤ sup P
l∗

→ 0,

(m)
µX (z, ∞]

−

Pm

j=0
P∞
j=0

Pm

j=0
P∞
j=0

|cj |1/γ

|cj |1/γ

|cj |1/γ

|cj |1/γ

µ(z, ∞] > ε

µ(z, ∞] > ε

(m)
S l∗

(m)
S l∗

!



(m)
P S l∗

! 2m+1
X
l∗ =1



(m)
P S l∗

n, k, n/k → ∞. This proves (3.5) and finishes the first part of the proof.
To show that the convergence in (3.3) holds, it now suffices to check whether


(m)
lim lim sup P µX (z, ∞] − µX (z, ∞] > ε = 0.
m→∞ n,k,n/k→∞

([15], Theorem 3.5). Based on the proof of [16], Proposition 3.3, define the
events
(
)
(m)
|Xi |
|Xi |
Ai := 0 ≤
−
≤δ ,
b|X| (n/k) b|X| (n/k)
(
)
(m)
|Xi |
|Xi |
Bi := 0 ≤
−
≤δ ,
b|X| (n/k) b|X| (n/k)

Title Suppressed Due to Excessive Length

Ci :=

(

(m)

|Xi |
|Xi |
>δ
−
b|X| (n/k) b|X| (n/k)

9

)

for some 0 < δ < z. Then for m ∈ N {0} and ε > 0:


(m)
P µX (z, ∞] − µX (z, ∞] > ε
(

)
!


(m)
|Xi |
|Xi |
∈ (z, ∞] − 1
∈ (z, ∞] 1Ai > ε/3
≤ P
b|X| (n/k)
b|X| (n/k)
(
)
!


n
(m)
1X
|Xi |
|Xi |
+P
1
∈ (z, ∞] − 1
∈ (z, ∞] 1Bi > ε/3
k i=1
b|X| (n/k)
b|X| (n/k)
!
n
1X
1C > ε/3
+P
k i=1 i
" (
!
)

#
n
(m)
1X
|Xi |
|Xi |
≤ P
1
∈ (z, ∞] − 1
∈ (z, ∞] 1Ai > ε/3
k i=1
b|X| (n/k)
b|X| (n/k)
" 
!
(
)#

n
(m)
|Xi |
|Xi |
1X
1
∈ (z, ∞] − 1
∈ (z, ∞]
1Bi > ε/3
+P
k i=1
b|X| (n/k)
b|X| (n/k)
)!
(
(m)
|X1 |
|X1 |
3n
>δ
E 1
−
+
εk
b|X| (n/k) b|X| (n/k)
n
1X
1
k i=1

:= I1 + I2 + I3 .
Here Chebyshev’s inequality was used in the second last line. Now
" (
)
(
)#
!
n
(m)
(m)
1X
|Xi |
|Xi |
I1 ≤ P
1
∈ (z, ∞] − 1
∈ (z + δ, ∞]
> ε/3
k i=1
b|X| (n/k)
b|X| (n/k)
and since we know from (3.5) that
(
(
)
)
n
n
(m)
(m)
|Xi |
|Xi |
1X
1X
1
1
∈ (z, ∞] −
∈ (z + δ, ∞]
k i=1
b|X| (n/k)
k i=1
b|X| (n/k)
Pm
1/γ 

P
j=0 |cj |
z −1/γ − (z + δ)−1/γ
→ P∞
1/γ
j=0 |cj |

we conclude I1 → 0 as n, k, n/k → ∞ letting δ := δ(ε) ց 0.
Similarly
" (
)
(
)#
!
n
(m)
(m)
|Xi |
1X
|Xi |
1
I2 ≤ P
∈ (z − δ, ∞] − 1
∈ (z, ∞]
> ε/3 ,
k i=1
b|X| (n/k)
b|X| (n/k)

10

Lukas Martig, Jürg Hüsler

and by another application of (3.5) we conclude I2 → 0 as n, k, n/k → ∞
letting again δ := δ(ε) ց 0. Finally, for δ ′ > 0 and n, k, n/k large:
!
∞
 3n
X
3n 
(m)
P |X1 | − |X1 | > δ b|X| (n/k) ≤
P
cj Z1−j > δ b|X| (n/k)
|I3 | =
εk
εk
m+1


P∞
P∞
′
3P
3P
m+1 cj Z1−j > δ b|X| (n/k)
m+1 cj Z1−j > δ b|Z| (n/k)


≤
∼
ε
ε
P |Z1 | > b|Z| (n/k)
P |Z1 | > b|Z| (n/k)
∼

∞
3 X
|cj /δ ′ |1/γ .
ε m+1

as n, k, n/k → ∞, where we used the fact that ||a − b| − |a|| ≤ |b| for a, b ∈ R in
the first line, k/n ∼ P(|Z1 | > b|Z| (n/k)) (Theorem 3.6 of [15]) and b|X| (n/k) ∼
const. b|Z| (n/k) ([4], Lemma 5.2, and [14], Proposition 0.8(vi)) in line 2, and
again [4], Lemma 5.2, in line 3. Since the last sum converges to zero as m → ∞,
the proof of (3.3) is complete.
⊓
⊔
We next study the behavior of the tail empirical measure of |Xi | − |X|n,n−k ,
1 ≤ i ≤ n.
Lemma 2 Let the Basic Assumptions hold and let σ(t) be defined as in (3.1),
then, as n, k, n/k → ∞, we have for z ≥ 0:
n
1X
P
1 {|Xi | − |X|n,n−k > σ(n/k)z} → (zγ + 1)−1/γ ,
k i=1

(3.12)

Proof If z = 0, the proof is trivial and hence we lay our focus on the case
where z > 0. By (3.2) and (3.4), we have that
σ(n/k)
1
|X|n,n−k
P
→ γ+ ,
+
b|X| (n/k) z b|X| (n/k)
z
n, k, n/k → ∞, and together with (3.3), we get the following joint convergence
in probability:




1
|X|n,n−k
σ(n/k)
P
.
(3.13)
→ µ(z, ∞], γ +
+
µX (z, ∞],
bX (n/k) z b|X| (n/k)
z
([1], Theorem 4.4).
The final step then is to make use of the operator
T : R+ × R+ → R+ , T (µX (z, ∞], x) 7→ µX (zx, ∞].
Since T is continuous in (µ(z, ∞], x) (see [16], proof of Proposition 2.1), a
simple application of the continuous mapping theorem on (3.13) yields


σ(n/k)
|X|n,n−k
T µX (z, ∞],
+
bX (n/k) z b|X| (n/k)

Title Suppressed Due to Excessive Length

11



n
|Xi |
σ(n/k)z + |X|n,n−k
1X
>
1
=
k i=1
b|X| (n/k)
b|X| (n/k)
n
1X
1 {|Xi | − |X|n,n−k > σ(n/k)z}
k i=1


1
P
= µ(zγ + 1, ∞].
→ T µ(z, ∞], γ +
z

=

⊓
⊔
The next result gives the link between the tail empirical measure and a generalized form of the two tail array sums involved in the likelihood moment
equations.
Lemma 3 Assume that the Basic Assumptions hold and let σ(t) be defined
as in (3.1). Then, as n, k, n/k → ∞, we have for r < 0 and x, y > 0:


k−1
1X
γx
log 1 +
(|X|n,n−i − |X|n,n−k )
k i=0
σ(n/k)
Z ∞
dz
P
:= ψ1 (x),
→

1/γ
z
0
+
1
(z
+
1)
x

r/y
k−1
X
1
γx
1+
(|X|n,n−i − |X|n,n−k )
k i=0
σ(n/k)
Z ∞
dz
P r
→
+ 1 := ψ2 (x, y).
1/γ
z
y 0
1−r/y
+
1
(z
+
1)
x

(3.14)

(3.15)

Proof We are going to prove (3.15). (3.14) can be shown using the very same
steps.
By some straightforward steps and denoting X + := max(X, 0), it is easy to
see that
r/y
k−1 
γx
1X
1+
(|X|n,n−i − |X|n,n−k )
k i=0
σ(n/k)
r/y
n 
γx
1X
n−k
+
1+
=
(|Xi | − |X|n,n−k )
−
k i=1
σ(n/k)
k

=

=

+
Z
n
1 X r γx(|Xi |−|X|n,n−k ) /σ(n/k)
dz
+1
k i=1 y 0
(z + 1)1−r/y

r
y

Z

0

|

∞



n
σ(n/k)z
dz
1X
1 |Xi | − |X|n,n−k >
+1.
k i=1
γx
(z + 1)1−r/y
{z
}
:= χn,x,y

12

Lukas Martig, Jürg Hüsler

Next define χn,x,y (t) :=

Rt

1
0 k

Pn

i=1 1

n

γx
σ(n/k) (|Xi | − |X|n,n−k ) >
Rt
f (·) 7→ 0 f (s)/(s + 1)1−r/y

z

o

dz
(z+1)1−r/y

for t < ∞. Notice that the mapping
ds is continuous (we are integrating over a finite region) and it follows from (3.12) and the
continuous mapping theorem that for all t:
Z

P

χn,x,y (t) →

t
0

dz
z
x

+1

1/γ

(z + 1)1−r/y

.

The result thus follows again if we are able to show, that for any ε > 0:
lim

lim sup P(|χn,x,y − χn,x,y (t)| > ε) = 0.

t→∞ n,k,n/k→∞

Pick δ > 0, ε′ < 1 and 0 < ε′′ < 1/γ − r/y. Then there exists a pair (n0 , k0 ) =
(n0 (δ, ε′ , ε′′ ), k0 (δ, ε′ , ε′′ )) such that for all n > n0 and k > k0 :
P(|χn,x,y − χn,x,y (t)| > ε)
Z ∞



zσ(n/k)
|X|n,n−k
dz
= P
µX
+
,∞
>ε
γxb|X| (n/k) b|X| (n/k)
(z + 1)1−r/y
t

Z ∞
(3.2)
dz
>
ε
≤ P
µX (z(1 − ε′ )/x + (1 − δ), ∞]
(z + 1)1−r/y
t


|X|n,n−k
<1−δ
+P
b|X| (n/k)
Z ∞

(3.4)
dz
≤ P
µX ((1 − δ − ε′ )(z/x + 1), ∞]
>
ε
+ o(1)
(z + 1)1−r/y
t


Z ∞
(1 − δ − ε′ )
n
|X1 |
dz
−1
+ o(1)
≤ ε
P
>
(z + x)
k
b|X| (n/k)
x
(z + 1)1−r/y
t
≤ const.

Z

t

∞

(z +

dz
+ o(1)
· (z + 1)1−r/y

x)1/γ−ε′′
′′

∼ const. · t r/y−1/γ+ε + o(1),
where we used Chebyshev’s inequality in line 6 and Potter’s inequality ([2]) in
line 7. Hence for n, k, n/k large:
lim

lim sup P(|χn,x,y − χn,x,y (t)| > ε)

t→∞ n,k,n/k→∞

≤ lim

′′

lim sup const. · t r/y−1/γ+ε + o(1) = 0.

t→∞ n,k,n/k→∞

⊓
⊔

Title Suppressed Due to Excessive Length

13

Notice that for the special cases where x = 1 and y = γ, the integrals in
(3.14) and (3.15) are easy to calculate so that ψ1 (1) = γ and ψ2 (1, γ) =
ψ2 (1, ψ1 (1)) = 1/(1 − r). This will be essential in our main theorem which
follows next.
Theorem 1 Assume that the Basic Assumptions hold and let σ(t) be defined
as in (3.1). Then for r < 0, we have as n, k, n/k → ∞:
!
!
γ
γ̂LME
P
→
.
(3.16)
σ̂LME /σ(n/k)
1
P

Proof We are going to show that θ̂LME /θ(n/k) → 1 as n, k, n/k → ∞, where
θ̂LME is the (unique) solution of equation (2.5) and θ(n/k) := γ/σ(n/k). If this
Pk−1
P
convergence holds, then γ̂ = k1 i=0
log(1 + θ̂LME (|X|n,n−i − |X|n,n−k )) →
ψ1 (1) = γ by (3.14) and the continuous mapping theorem and hence we have
P
that σ̂LME /σ(n/k) = γ̂LME /γ · θ(n/k)/θ̂LME → 1 as n, k, n/k → ∞. The
joint convergence in (3.16) finally follows by [1], Theorem 4.4.
Define Yi := |X|n,n−i − |X|n,n−k , Y := {Y0 , Y1 , . . . , Yk−1 } and for x, y > 0:
Pk−1
Pk−1
ψ1 (x, Y ) := (1/k) i=0 log(1+xθ(n/k)Yi ) and ψ2 (x, y, Y ) := (1/k) i=0 (1+
xθ(n/k)Yi )r/y .
Now, from Result 3.3 and again Theorem 4.4 of [1], it easily follows as n, k, n/k
→ ∞ that
P
(ψ1 (x, Y ), ψ2 (x, y, Y )) → (ψ1 (x), ψ2 (x, y)).
Next consider the operator
T̃ : R+ × R+ → R+ , T̃ (ψ1 (x, Y ), ψ2 (x, y, Y )) 7→ ψ2 (x, ψ1 (x, Y ), Y ).
Since both ψ1 (x, Y ) and ψ2 (x, y, Y ) are continuous in x, y > 0, T̃ is also
continuous and hence by the continuous mapping theorem:
P

ψ2 (x, ψ1 (x, Y ), Y ) → ψ2 (x, ψ1 (x)),
which clearly yields
1
1
P
→ ψ2 (x, ψ1 (x)) −
:= ψ(x)
(1 − r)
(1 − r)
(3.17)
as n, k, n/k → ∞. Notice thereby that ψ(x, Y ) is a strictly decreasing function
in x ∈ (0, ∞) (cf. [21], proof of Theorem 2.1) and according to Lemma 4, its
limit function ψ(x) has the very same property.
ψ(x, Y ) := ψ2 (x, ψ1 (x, Y ), Y ) −

P

Recall from (3.2) and (3.4) that θ(n/k)Y0 = γY0 /σ(n/k) ∼ Y0 /b|X|(n/k) → ∞
as n, k, n/k → ∞. Hence there exists no solution θ̂LME of (2.5) such that
θ̂LME /θ(n/k) converges to a negative value in probability.
P

Let’s consequently assume first that θ̂LME /θ(n/k) → a ∈ (0, ∞). Obviously,

14

Lukas Martig, Jürg Hüsler

the mapping θ̂LME /θ(n/k) 7→ ψ(θ̂LME /θ(n/k), Y ) is continuous since ψ(x, Y )
is continuous in x ∈ (0, ∞) and from (3.17), it follows once more by [1],
Theorem 4.4, and the continuous mapping theorem that
P

ψ(θ̂LME /θ(n/k), Y ) − ψ(a) → 0,

(3.18)

n, k, n/k → ∞. But by definition of θ̂LME (see (2.5)), ψ(θ̂LME /θ(n/k), Y ) = 0
and thus the convergence in (3.18) holds if and only if a = 1 because only in
that case ψ(a) = 0 by the monotonicity of ψ.
P

Thus, the proof is complete if we are able to show that neither θ̂LME /θ(n/k) →
P

0 nor θ̂LME /θ(n/k) → ∞.
For the first case, use again the monotonicity of ψ(x, Y ) to see that for any
ε ∈ (0, 1):




lim
P θ̂LME /θ(n/k) ≥ ε =
lim
P θ̂LME /θ(n/k) ≥ ε
n,k,n/k→∞

=

lim

n,k,n/k→∞

n,k,n/k→∞



P ψ(θ̂LME /θ(n/k), Y ) ≤ ψ(ε, Y ) =

lim

n,k,n/k→∞

P (0 ≤ ψ(ε, Y ))

= 1,
P

because ψ(ε, Y ) → ψ(ε) > 0. Hence


P θ̂LME /θ(n/k) < ε → 0
as n, k, n/k → ∞.
Similarly for the second case:


lim
P θ̂LME /θ(n/k) > ε−1 =
n,k,n/k→∞

lim

n,k,n/k→∞


P 0 < ψ(ε−1 , Y ) = 0,

P

because ψ(ε−1 , Y ) → ψ(ε−1 ) < 0 by the monotonicity of ψ(x, Y ). This concludes the proof.
⊓
⊔
Lemma 4 ψ(x) defined in (3.17) is a strictly decreasing function in x ∈
(0, ∞).
Proof Let ψ(x, Y) be defined as in (3.17). Then the aim is to show that
ψ ′ (x, Y) := d ψ(x, Y)/dx converges in probability to a negative limit for all
x > 0 as n, k, n/k → ∞.
Recall that Yi = |X|n,n−i − |X|n,n−k and denote Zi,x := log(1 + x θ(n/k)Yi )
Pk−1
Zj,x (which in fact is equivalent to ψ1 (x, Y) in (3.14)) for
and Z̄x := k1 j=0
i = 0, 1, . . . , k − 1 and x > 0. Notice thereby that Zi,x > 0 for all i and x > 0
P

and furthermore Zi,x > Zj,x if i < j. Also, Z̄x → ψ1 (x) as n, k, n/k → ∞ by

Title Suppressed Due to Excessive Length

15

(3.14).
Further recall by (3.2) and (3.4) that for l ∈ {1, 2, 3, 4}:
θ(n/k)Y⌈k·l/5⌉

 −γ
l
∼ Y⌈k·l/5⌉ /b|X| (n/k) →
− 1,
5
P

and thus, by a simple application of the continuous mapping theorem,
Z⌈k·l/5⌉,x

!!
 −γ
l
−1
→ log 1 + x
5
P

(3.19)

as n, k, n/k → ∞.
Now, going along the steps of the proof of Theorem 2.1 in [21], ψ ′ (x, Y) may
be expressed in the following way:
ψ ′ (x, Y) = Z̄x−2 k −2

X

Zi,x Zj,x · (u(Zi,x ) − u(Zj,x )) · (v(Zi,x ) − v(Zj,x )),

0≤i<j≤k−1

where u(a) := r exp(ra/Z̄x )/x is strictly increasing and v(a) := (1 − exp(a))/a
is strictly decreasing in a. Hence ψ ′ (x, Y) < 0 for all x > 0. If we define the set
A∗i,j := {⌈k/5⌉ ≤ i ≤ ⌈2k/5⌉, ⌈3k/5⌉ ≤ j ≤ ⌈4k/5⌉}, then, by straightforward
steps:
ψ ′ (x, Y)
X
< k −2
Zi,x Zj,x /Z̄x2 · (u(Zi,x ) − u(Zj,x )) · (v(Zi,x ) − v(Zj,x ))
A∗
i,j

≤ k −2

X

A∗
i,j


max
Zi,x Zj,x /Z̄x2 · (u(Zi,x ) − u(Zj,x )) · (v(Zi,x ) − v(Zj,x ))
∗
Ai,j


∼ 25−1 max
Zi,x Zj,x /Z̄x2 · (u(Zi,x ) − u(Zj,x )) · (v(Zi,x ) − v(Zj,x ))
∗
Ai,j


{u(Zi,x ) − u(Zj,x )} max
{v(Zi,x ) − v(Zj,x )}
≤ 25−1 min
Zi,x Zj,x /Z̄x2 min
∗
∗
∗
Ai,j

Ai,j

Ai,j



= 25−1 Z⌈2k/5⌉,x Z⌈4k/5⌉,x /Z̄x2 u(Z⌈2k/5⌉,x ) − u(Z⌈3k/5⌉,x )

· v(Z⌈2k/5⌉,x ) − v(Z⌈3k/5⌉,x ) := 25−1 (I1 · I2 · I3 ).

Using the fact that both u(a), v(a) are continuous in a > 0, (3.14), (3.19) and
P
several applications of the continuous mapping theorem simply yield I1 →
P
P
P
const. > 0, I2 → const. > 0 and I3 → const. < 0 so that I1 · I2 · I3 → const. < 0
as n, k, n/k → ∞.
⊓
⊔

16

Lukas Martig, Jürg Hüsler

4 Concluding remarks and outlook
At first glance it may seem a bit unconventional to use the classical results of
Resnick & Stâricâ rather than a well-known set of powerful tools established
by Drees ([5]) when studying asymptotic behavior of linear processes. To our
misfortune, it is a very tough question whether Drees’ main Theorem 2.1.
can be applied to MA(∞)-processes as the author himself admits that one
of its conditions “is more complicated to check [...] for general linear time
series” (p. 635). Nevertheless, he was able to show that all conditions hold
for an AR(1)-process (section 3.2), a result which was recently extended to
arbitrary AR(p)-processes, p ∈ N {0}, by Kulik et al. ([12]). It thus would
be of great interest to extend these results once more to the whole class of
ARMA processes.
Of course, the next step will be to show asymptotic normality for γ̂LME and
σ̂LME . A very promising way to do so is to use and to extend the theory
of tail array sums established by [18], [19], which both served as basis for
Resnick & Stâricâ’s studies on asymptotic normality of Hill’s estimator for
autoregressive data ([17]). Also, a simulation of the finite sample behavior of
the LMEs would be very useful since the condition n, k, n/k → ∞ is only
fulfilled if one really collects a very large amount of data – a well-known
problem when using intermediate order statistics like |X|n,n−k .

Title Suppressed Due to Excessive Length

17

References
1. Billingsley, P.: Convergence of Probability Measures. Wiley, New York (1968)
2. Bingham, N., Goldie, C., Teugels, J.: Regular Variation, Encyclopedia of Mathematics
and its Applications. Cambridge University Press, Cambridge (1987)
3. Brockwell, P., Davis, R.: Time Series: Theory and Methods, 2nd edition. Springer, New
York (1991)
4. Datta, S., McCormick, W.P.: Inference for tail parameters of a linear process with heavy
tail innovations. Ann. Inst. Statist. Math. 50, 337-359 (1998)
5. Drees, H.: Extreme quantile estimation for dependent data, with applications to finance.
Bernoulli 9(1), 617-657 (2003)
6. Fisher, R., Tippet, L.: Limiting forms of the frequency distribution of the largest or
smallest member of a sample. Proc. Cambridge Philos. Soc. 24, 180-190 (1928)
7. Grimshaw, S.D.: Computing maximum likelihood estimates for the generalized Pareto
distribution. Technometrics 35, 185-191 (1993)
8. Haan, L. de, Ferreira, A.: Extreme Value Theory. Springer, New York (2006)
9. Hosking, J.R.M., Wallis, J.R.: Parameter and Quantile Estimation for the Generalized
Pareto Distribution. Technometrics 29, 339-349 (1987)
10. Hüsler, J., Li, D., Raschke, M.: Estimation for the Generalized Pareto Distribution
using Maximum Likelihood and Goodness-of-fit. Comm. Statist. Theory and Methods.
40, 2500-2510 (2011)
11. Hüsler, J., Li, D., Raschke, M.: Tail index estimator using maximum likelihood and
moment estimation. To appear in: Comm. Statist. Theory and Methods (2015)
12. Kulik, R., Soulier, Ph., Wintenberger, O.: The tail empirical process of
regularly varying functions of geometrically ergodic Markov chains. Preprint:
http://arxiv.org/abs/1511.04903 (2015)
13. Pickands, J.: Statistical Inference using Extreme Order Statistics. Ann. Statist. 3, 119131 (1975)
14. Resnick, S.: Extreme Values, Regular Variation and Point Processes. Springer, New
York (1987)
15. Resnick, S.: Heavy-Tail Phenomena. Probabilistic and Statistical Modeling. Springer,
New York (2007)
16. Resnick, S., Stâricâ, C.: Consistency of Hill’s estimator for dependent data. J. Appl.
Prob. 32, 139-167 (1995)
17. Resnick, S., Stâricâ, C.: Asymptotic behaviour of Hill’s Estimator for Autoregressive
Data. Comm. Statist. Stoch. Models 13(4), 703-721 (1997)
18. Rootzén, H., Leadbetter, M.R., Haan, L. de: Tail and quantile estimation for strongly
mixing stationary sequences. Technical Report 292, Center for Stochastic Processes, Department of Statistics, University of North Carolina, Chapel Hill, NC 27599-3260 (1990)
19. Rootzén, H., Leadbetter, M.R., Haan, L. de: On the distribution of tail array sums for
strongly mixing stationary sequences. Ann. Appl. Probab. 8(3), 868-885 (1998)
20. Smith, R.L.: Threshold Methods for Sample Extremes. In: Statistical Extremes and
Applications, J. Tiago de Oliveira (Ed.). D. Reidel, Dordrecht, 621-638 (1984)
21. Zhang, J.: Likelihood Moment Estimation for the Generalized Pareto Distribution. Aust.
N.Z. J. Stat. 49, 69-77 (2007)

