arXiv:1509.02473v2 [math.ST] 3 May 2017

A new non-parametric detector of univariate outliers for
distributions with unbounded support
Jean-Marc Bardet
and
Solohaja-Faniaha Dimby∗
S.A.M.M., Université de Paris 1 Panthéon-Sorbonne
90, rue de Tolbiac, 75634, Paris, France

Abstract
The purpose of this paper is to construct a new non-parametric detector of univariate outliers
and to study its asymptotic properties. This detector is based on a Hill’s type statistic. It satisfies
a unique asymptotic behavior for a large set of probability distributions with positive unbounded
support (for instance: for the absolute value of Gaussian, Gamma, Weibull, Student or regular
variations distributions). We have illustrated our results by numerical simulations which show the
accuracy of this detector with respect to other usual univariate outlier detectors (Tukey, MAD or
Local Outlier Factor detectors). The detection of outliers in a database providing the prices of used
cars is also proposed as an application to real-life database.

Keywords: Outlier detection; order statistics; Hill estimator; non-parametric test.

∗

The authors gratefully acknowledge the enterprise Autobiz

1

Introduction

Let (X1 , · · · , Xn ) be a sample of positive, independent, identically distributed random variables with
unbounded distribution. This article aims to provide a non-parametric outlier detector among the
”large” values of (X1 , · · · , Xn ).
Remark 1. If we wish to detect outliers among the ”small” values of (X1 , · · · , Xn ), it would be possible
to consider max(X1 , · · · , Xn )−Xi instead of Xi , for i = 1, · · · , n. Moreover, if Xi , i = 1, · · · , n, are not
positive random variables, as in the case of quantile regression residuals, we can consider |Xi | instead
of Xi .
There are numerous outlier detectors in such a framework. Generally, such detectors consist of
statistics directly applied to each observation, deciding if this observation can be considered or not
as an outlier (see for instance the books of Hawkins, 1980, Barnett and Lewis, 1994, Rousseeuw and
Leroy, 2005, or the article of Beckman and Cook, 1983). The most frequently used, especially in the
case of regression residuals, is the Student-type detector (see a more precise definition in Section 4).
However, it is a parametric detector that is theoretically defined for a Gaussian distribution. Another
well-known detector is the robust Tukey detector (see for example Rousseeuw and Leroy, 2005). Its
confidence factor is computed from quartiles of the Gaussian distribution, though it is frequently used
for non-Gaussian distributions. Finally, we can also cite the M ADe detector using a confidence factor
computed from the median of absolute value of Gaussian distribution (see also Rousseeuw and Leroy,
2005).
Hence all the most commonly used outlier detectors are based on Gaussian distribution and they are
not really accurate for heavier distributions (for regression residuals, we can also cite the Grubbs-Type
detectors introduced in Grubbs, 1969, extended in Tietjen and Moore, 1972). Such a drawback could
be avoided by considering a non-parametric outlier detector. However, in the literature there are few
non-parametric outlier detectors. We could cite the Local Outlier Factor (LOF) introduced in Breunig
et al. (2000), which is also valid for multivariate outliers. For any integer k, the LOF algorithm compares the density of each point to the density of its k-closest neighbors. Unfortunately a theoretical or
numerical procedure for choosing the number k of cells and its associated threshold still does not exist.
There are also other detectors essentially based on a classification methodology (for instance: Knorr et
al., 2000) or robust statistics (for instance: Hubert and Vendervieren, 2008, for univariate skewed data
and Hubert and Van der Veeken, 2008, for multivariate ones).
An interesting starting point for defining a non-parametric detector of outlying observations is provided
by the order statistics (X(1) , . . . , X(n) ) of (X1 , · · · , Xn ). Thus, Tse and Balasooriya (1991) introduced a
detector based on increments of order statistics, but only for the exponential distribution. Recently, a
procedure based on the Hill estimator was also developed for detecting influential data points in Paretotype distributions (see Hubert et al., 2012). The Hill estimator (see Hill, 1975) has been defined from


the following property: the family of r.v. j log(X(n−j+1) ) − log(X(n−j) ) 1≤j≤k(n) is asymptotically

(when min k(n) , n − k(n) −→ ∞) a sample of independent r.v. following exponential distributions
n→∞

(Rényi exponential representation) for distributions in the max-domain of attraction of Gγ where Gγ
is the cumulative distribution function of the extreme value distribution (see Beirlant et al., 2004).
Here we will use an extension of this property for detecting a finite number of outliers among the
sample (X1 , · · · , Xn ). Indeed, an intuitive idea for detecting them is the following: the presence of

outliers generates a jump in the family of r.v. X(n−j+1) /X(n−j) j , therefore we also see a jump in

the family of the r.v. log(X(n−j+1) ) − log(X(n−j) ) j . Thus an outlying data detector can be obtained
when the maximum of this family exceeds a threshold (see details in (2.8) or (3.3)). In the sequel, see
some assumptions on probability distributions for applying this new test of outlier presence. This also
provides an estimator of the number of outliers. It is relevant to say that this test is not only valid
for Pareto-type distribution (for instance Pareto, Student or Burr probability distributions), but more
generally to a class of regular variations distributions. It can also be applied to numerous probability distributions with an exponential decreasing probability distribution function (such as Gaussian,
Gamma or Weibull distributions). So our new outlier detector is a non-parametric estimator defined
from an explicit threshold, which does not require any tuning parameter and can be applied to a very
large family of probability distributions.
Numerous Monte-Carlo experiments carried out in the case of several probability distributions attest to
the accuracy of this new detector. It is compared to other famous outlier detectors or extended versions
of these detectors and the simulation results obtained by this new detector are convincing especially
since it ignores false outliers. Moreover, an application to real-life data (price, mileage and age of used
cars) is done, allowing the detection of two different kinds of outliers.

We have drafted our paper along following lines. Section 2 contains the definitions and several probabilistic results while Section 3 describes how to use them to build a new outlier detector. Section 4
is devoted to Monte-Carlo experiments, Section 5 presents the results of the numerical application on
used car variables and the proofs of this paper are to be found in Section 6.

2

Definition and first probabilistic results

For (X1 , · · · , Xn ) a sample of positive i.i.d.r.v. with unbounded distribution, define:
G(x) = P(X1 > x)

for x ∈ R.

(2.1)

It is clear that G is a decreasing function and G(x) → 0 when x → ∞. Hence, define also the pseudoinverse function of G by
G−1 (y) = sup{x ∈ R, G(x) ≥ y}

y ≥ 0.

(2.2)

G−1 is also a decreasing function. Moreover, if the support of the probability distribution of X1 is
unbounded then G−1 (x) → ∞ when x → 0.

Now, we consider both the following spaces of functions:
n

f2 (α)
1 
• A1 = f : [0, 1] → R, such as for any α > 0, f (αx) = f1 (x) 1 +
+O
when
log(x)
log2 (x)
o
x → 0 where f1 : [0, 1] → R satisfies limx→0 f1 (x) = ∞ and f2 is a C 1 ([0, ∞)) diffeomorphism .
n
• A2 = g : [0, 1] → R, there exist a > 0 and a function g1 : [0, 1] → R satisfying limx→0 g1 (x) = ∞,
o

1
and for all α > 0, g(αx) = α−a g1 (x) 1 + O log(x)
when x → 0 .
Example 2.1. We will show below that numerous famous ”smooth” probability distributions such as
absolute values of Gaussian, Gamma or Weibull distributions satisfy G−1 ∈ A1 . Moreover, numerous
heavy-tailed distributions such as Pareto, Student or Burr distributions have G−1 ∈ A2 .
Using the order statistics X(1) ≤ X(2) ≤ · · · ≤ X(n) , define the following ratios (τj ) by:
τj =

X(j+1)
X(j)

if X(j) > 0, and τj = 1 if not, for any j = 1, · · · , n − 1

τj0 = (τj − 1) log(n)

for any j = 1, · · · , n − 1

(2.3)
(2.4)

In the sequel, we are going to provide some probabilistic results on the maximum of these ratios.
Proposition 1. Assume G−1 ∈ A1 . Then, for any J ∈ N∗ , and with (Γi )i∈N∗ a sequence of r.v.
satisfying Γi = E1 + · · · + Ei for i ∈ N∗ where (Ei )i∈N∗ is a sequence of i.i.d.r.v. with exponential
distribution of parameter 1,
D

0
max {τn−j
} −→

j=1,··· ,J

max

n→∞ k=1,··· ,J



f2 (Γk+1 ) − f2 (Γk ) .

(2.5)

Here f2 corresponds to G−1 in the manner described in the definition of A1 .
Now, we consider a particular case of functions belonging to A1 . Let A01 be the following function space:

A01 = f ∈ A1 and there are C1 , C2 ∈ R satisfying f2 (α) = C1 + C2 log α for all α > 0 .
Example 2.2. Here there are some examples of classical probability distributions satisfying G−1 ∈ A01 :
• Exponential distribution E(λ): In this case, G−1 (x) = − λ1 log(x), and this implies G−1 ∈ A01
with f1 (x) = − λ1 log(x) and f2 (α) = log α (C1 = 0 and C2 = 1).
• Gamma distributions Γ(a) In this case, G(x) =

1
Γ(a)

R∞
x

ta−1 e−t dt for a ≥ 1 and we obtain,

using an asymptotic expansion of the incomplete gamma function (see Abramowitz and Stegun,
1964):
G−1 (x) =


1
− log x + (a − 1) log(− log x) + O(|(ln x)−1 |)
Γ(a)

x → 0.

As a consequence, we deduce G−1 ∈ A01 with
f1 (x) =


1
− log x + (a − 1) log(− log x)
Γ(a)

and

f2 (α) = log α (C1 = 0 and C2 = 1).

• Absolute value of standardized Gaussian distribution |N (0, 1)|: In this case, we can write
√
R∞
2
G(x) = √22π x e−t /2 dt = erfc(x/ 2), where erf c is the complementary Gauss error function.

But we know (see for instance Blair et al., 1976) that for x → 0, then erfc−1 (x) = √12 −
1/2
+ O(|(ln x)−1 |). As a consequence, for any α > 0,
log(πx2 ) − log(− log x)


log α
erfc−1 (α x) = erfc−1 (x) 1 +
+ O(|(ln x)−2 |)
2 log x

x → 0.

(2.6)

Consequently G−1 ∈ A01 with
f1 (x) =

p
−2 log x − log(− log x) − 2 log π

and

f2 (α) =

1
log α,
2

implying C1 = 0 and C2 = 12 .
k

• Weibull distributions: In this case, with a ≥ 0 and 0 < b ≤ 1, G(x) = e−(x/λ) with λ > 0 and
1/k
k ∈ N∗ , for x ≥ 0. Then it is obvious that G−1 (x) = λ − log x
and therefore G−1 ∈ A01 with
1/k
f1 (x) = λ − log x
and f2 (α) = k1 log α (implying C1 = 0 and C2 = 1/k).
When G−1 ∈ A01 , it is possible to specify the limit distribution of (2.5). Thus, we show the following
result:
Proposition 2. Assume that G−1 ∈ A01 . Then
J


Y

0
P max {τn−j } ≤ x −→
1 − e−jx/C2 .
j=1,··· ,J

(2.7)

n→∞ j=1

Such a result is interesting since it provides the asymptotic behavior of a vector of normalized and
centered ratios τi . Its asymptotic distribution is the distribution of a vector of independent exponentially
distributed r.v’s. However the parameters of these exponential distributions are different. Thus, if we
consider the statistic
0
Tb = max {τn−j
},
j=1,··· ,J

(2.8)

the computation of the cumulative distribution function of Tb requires consideration of the function

Q
y ∈ [0, ∞) 7→ R(y) = Jj=1 1 − e−jy . This function converges quickly to 1 when J increases. Hence
we numerically obtain that for J ≥ 3, R(3.042) ' 0.95. Then we deduce from (2.7) and (2.8) that for
J ≥ 3,

P Tb ≤ 3.042 × C2 ' 0.95.

This implies that for instance that for J ≥ 3 and n large enough,


• P Tb ≤ 3.042 ' 0.95


• P Tb ≤ 1.521 ' 0.95

when X follows a Gamma distribution
when |X| = |N (0, 1)|,

0
with the computation of C2 for each distribution. We remark that the ratio τn−1
is the main contributor

to the statistic Tb and it contains almost all the information. For giving equivalent weights to the other
ratios τk0 , k ≤ n − 1 and so as not to be troubled by the nuisance parameter C2 , it is necessary to modify
the statistic Tb. Then we consider:
Ten = max

j=1,··· ,J



j

0
τn−j

1
×
sJ

where

J
1 X 0
sJ =
j τn−j .
J

(2.9)

j=1

For (un )n and (vn )n two sequences of real numbers, denote un

∼

vn when un /vn −→ 1. The

n→∞

n→∞

following proposition can be established:
Proposition 3. Assume that G−1 ∈ A01 . Then, for a sequence (Jn )n satisfying Jn −→ ∞ and
n→∞

Jn / log n −→ 0,
n→∞


Pr Ten ≤ x

∼

1 − e−x

 Jn

for x > 0.

(2.10)

n→∞

In the case where G−1 ∈ A2 , similar results can also be established, we demonstrated below.
Example 2.3. Here there are some examples of classical distributions such as G−1 ∈ A2 :
• Pareto distribution P(α): In this case, with c > 0 and C > 0, G−1 (x) = C x−c for x → 0, and
this implies G−1 ∈ A2 with a = c.
• Burr distributions B(α): In this case, G(x) = (1 + xc )−k for c and k positive real numbers.
Thus G−1 (x) = (x−1/k − 1)1/c for x ∈ [0, 1], implying G−1 ∈ A2 with a = (ck)−1 .
• Absolute value of Student distribution |t(ν)| with ν degrees of freedom: In the case of a
Student distribution with ν degrees of freedom, the cumulative distribution function is Ft(ν) (x) =
1
2 (1

+ I(y, ν/2, 1/2)) with y = ν(ν + x2 )−1 and therefore G|t(ν)| (x) = I(y, ν/2, 1/2), where I is

the normalized beta incomplete function. Using the handbook of Abramowitz and Stegun (1964),
we have the following expansion G|t(ν)| (x) =

2ν ν/2−1
B(ν/2,1/2)

x−ν + O(x−ν+1 ) for x → 0, where B is the

usual Beta function. Therefore,
G−1
|t(ν)| (x) =

B(ν/2, 1/2) −1/ν
x
+ O(x−1/ν−1 )
2ν ν/2−1

Consequently G−1
|t(ν)| ∈ A2 with a = 1/ν.

x → ∞.

Remark 2. The case of standardized log-normal distribution is singular. Indeed, the probability dis
√x
tribution of X is the same than the one of exp(Z) where Z ∼ N (0, 1). Therefore, G(x) = 21 erfc log
2
√

implying G−1 (x) = exp 2 erfc−1 (2x) . Using the previous expansion (2.6), we obtain for any α > 0:
√


2 erfc−1 (2x α)
√

log α
= exp
2 erfc−1 ( 2x) 1 +
+ O(|(ln x)−2 |)
2 log x

G−1 (α x) = exp

= G−1 (x) 1 + O(|(ln x)−1/2 |).
Therefore, the standardized log-normal distribution is such that G−1 ∈
/ A1 ∪ A2 .
For probability distributions such as G−1 ∈ A2 we obtain the following classical result (see also Embrechts et al., 1997):
Proposition 4. Assume that G−1 ∈ A2 . Then,
J


Y

P max {log(τn−j )} ≤ x −→
1 − e−jx/a .
j=1,··· ,J

(2.11)

n→∞ j=1

Hence the case of G−1 ∈ A2 also provides interesting asymptotic properties on the ratios. In the
forthcoming section devoted to the construction of an outlier detector from previous results, we are
going to consider a test statistic which could be as well applied to distributions with functions G−1
belonging to A01 and A2 .

3

A new non-parametric outlier detector

We are going to consider the following test problem:

 H : there is no outlier in the sample
0
.
 H : there is at least one outlier in the sample
1

(3.1)

However we have to specify which kind of outlier and therefore which kind of contamination we consider.
Our guide for this is typically the case of oversized regression residuals. Thus we would like to detect
from (X1 , . . . , Xn ) when there is a ”gap” between numerous Xi coming from a common probability
distribution and one or several (but not a lot!) Xj which are larger than the other one and generated
from another distributions. As a consequence the previous test problem can be specified as follows:



H : (X1 , . . . , Xn ) are i.i.d. r.v. with G−1 ∈ A01 ∪ A2

 0
H1 : there exists K ∈ N∗ such as (X(1) , . . . , X(n−K) ) are i.i.d. r.v. with G−1 ∈ A01 ∪ A2 . (3.2)




and for i = 0, . . . , K − 1, the distribution of X
satisfy G−1
> G−1 .
(n−i)

X(n−i)

A consequence of this specification is the following: we would like to detect outliers which appear as
oversized data. Hence, under H1 we could expect that there exists a ”jump” between the smallest

outlier and the largest non contaminated data. As a consequence under H1 we could expect that the
ratio τ(n−J) is larger than it should be.
For doing such a job, we propose to consider the following outlier detector based on ratios and which
could be used as well when G−1 belongs to A01 or A2 . Hence, define:
b Jn = log 2
D
b Jn
L

max j log(τn−j )

j=1,··· ,Jn

where



b Jn = median j log(τn−j )
L
.
1≤j≤Jn

(3.3)

Then, we obtain the following theorem:
Theorem 3.1. Assume that G−1 ∈ A01 ∪ A2 . Then, for a sequence (Jn )n satisfying Jn −→ ∞ and
n→∞

Jn / log n −→ 0,
n→∞

b Jn ≤ x
Pr D



∼

1 − e−x

 Jn

.

(3.4)

n→∞

b Jn we prefer an estimation of the parameter of the exponential
Remark 3. In the definition of D
distribution with a robust estimator (median) instead of the usual efficient estimator (empirical mean),
since several outliers could corrupt this estimation.
Therefore, Theorem 3.1 can be applied for distributions with G−1 belonging to A01 or A2 , i.e. as well as
for Gaussian, Gamma or Pareto distributions. Hence, for a type I error α ∈ (0, 1), the outlier detector

b Jn can be computed, and with t = − log 1 − (1 − α)1/Jn ,
D
b Jn ≤ t then we conclude that there is no outlier in the sample.
• If D
b Jn > t then the largest index b
b Jn ≥ t indicates that we decide that
• If D
k0 such as b
k0 log(τn−bk0 )/L
the observed data (X(n−bk0 +1) , X(n−bk0 +2) , . . . , X(n) ) can be considered as outliers, implying that
there are b
k0 detected outliers.
As a consequence this outlier detector allows a decision of the test problem (3.1) and also the identification of the exact outliers.

4

Monte-Carlo experiments

We are going to compare the new outlier detector defined in (3.3) with usual univariate outlier detectors.
b Jn , we present the results of Monte-Carlo
After giving some practical details of the application of D
experiments under several probability distributions.
Practical procedures of outlier detections
b Jn is simple, and in practice just requires the specification of 2 parameters:
The definition of D

b J when there are K
Table 1: Numerical choice of Jn : average frequencies for potential outliers with D
outliers from the seven considered probability distributions, and for several values of K, sample size n
and parameter J. Here α = 0.007, 20000 independent replications are done and multiplicative outliers
(see below) are generated.
J

12

14

16

18

20

22

24

26

28

0.595

0.588

0.583

0.573

0.568

0.565

0.561

0.557

0.554

0.694

0.708

0.702

0.696

0.694

0.689

0.686

0.683

0.679

0.727

0.719

0.712

0.705

0.700

0.691

0.687

0.685

0.680

K = 10

0.796

0.818

0.823

0.822

0.817

0.813

0.811

0.810

0.804

n = 1000, K = 5

0.760

0.752

0.744

0.738

0.732

0.727

0.722

0.716

0.711

K = 10

0.822

0.848

0.857

0.858

0.856

0.853

0.850

0.847

0.844

n = 5000, K = 5

0.802

0.793

0.786

0.779

0.773

0.769

0.763

0.759

0.754

K = 10

0.846

0.878

0.891

0.894

0.896

0.893

0.890

0.888

0.887

n = 100, K = 5
K = 10
n = 500, K = 5

• The type I error α is the risk to detect outliers in the sample while there is no outlier. Hence, a
natural choice could be the ”canonical” α = 0.05. However, we chose to be strict concerning the
risk of false detection, i.e. we chose α = 0.007 (as it was chosen by Tukey himself for building
boxplots) which implies that we prefer not to detect ”small” outliers and hence we avoid to detect
a large number of outliers while there is no outlier.
• The number Jn of considered ratios. On the one hand, it is clear that the smaller Jn , the smaller
the detection threshold, therefore more sensitive is the detector to the presence of outliers. On
the other hand, the larger Jn , the more precise is the estimation of the parameter of asymptotic
b Jn is √n) and larger is the possible number
exponential distribution (the convergence rate of L
of detected outliers. We carried out numerical simulations using 20000 independent replications,
for several probability distributions (the seven distributions presented below) for several values
of the number of outliers K, sample size n and parameter J. Results are reported in Table 1. A
first conclusion: the larger n and K the larger test power. Another conclusion, but this is not a
surprise, is the fact that the ”optimal” choice of J depends on K and n.
As a consequence, for at least detecting K = 10 outliers, we use Jn = 1 + [4 ∗ log3/4 (n)] that is
an arbitrary choice satisfying Jn = o(log(n)) and fitting well the results of these simulations, i.e.
for n = 100, Jn = 13, for n = 1000, Jn = 18 and for n = 5000, Jn = 20.
b Jn to five common and well-known univariate outlier detectors
We have compared the new detector D
computed from the sample (X1 , · · · , Xn ).

1. The Student’s detector (see for instance Rousseeuw and Leroy, 2005): an observation from the
sample (X1 , · · · , Xn ) will be consider as an outlier when P(Xk > X n + ss × σ n ) where X n and
σ 2n are respectively the usual empirical mean and variance computed from (X1 , · · · , Xn ), and ss
is a threshold. This threshold is usually computed from the assumption that (X1 , · · · , Xn ) is a

Gaussian sample and therefore ss = qt(n−1) (1 − α/2) , where qt(n−1) (p) denotes the quantile of
the Student distribution with (n − 1) degrees of freedom for a probability p.
2. The Tukey’s detector (see Tukey, 1977) which provides the famous and usual boxplots: Xk is
considered to be an outlier from (X1 , · · · , Xn ) if Xk > Q3 + 1.5 × IQR, where IQR = Q3 − Q1,
with Q3 and Q1 the third and first empirical quartiles of (X1 , · · · , Xn ). Note that the confidence
factor 1.5 was chosen by Tukey such that the probability of Gaussian random variable to be
decided as an outlier is close 0.7% which is good trade-off.
3. An adjusted Tukey’s detector as it was introduced and studied in Hubert and Vandervieren (2008):
Xk is considered to be an outlier from (X1 , · · · , Xn ) if Xk > Q3 + 1.5Q3 + 1.5 e3 M C × IQR, where
M C is the medcouple, defined by M C = medianXi ≤Q2≤Xj h(Xi , Xj ), where Q2 is the sample
median and the kernel function h is given by h(Xi , Xj ) =

(Xj −Q2)−(Q2−Xi )
.
Xj −Xi

This new outlier

detector improves considerably the accuracy of the usual Tukey’s detector for skewed distributions.
4. The M ADe detector (see for instance Rousseeuw and Leroy, 2005): Xk is considered as an outlier
from (X1 , · · · , Xn ) if |Xk − Q2| > 3 ∗ 1.483 ∗ median(|X1 − Q2|, · · · , |Xn − Q2|). The coefficient
1.483 is obtained from the Gaussian case based on the relation SD ' 1.483 × median(|X1 −
Q2|, · · · , |Xn − Q2|), while the confidence factor c = 3 is selected for building a conservative test.
5. The Local Outlier Factor (LOF), which is a non-parametric detector (see for instance Breunig et
al., 2000). This procedure is based on this principle: an outlier can be distinguished when its
normalized density (see its definition in Breunig et al., 2000) is larger than 1 or than a threshold
larger than 1. However, the computation of this density requires to fix the parameter k of the
used k-distance and a procedure or a theory for choosing a priori k does not still exist. The
authors recommend k > 10 and k < 50 (generally). We chose to fix k = Jn , where Jn is used for
b J . Then, using the same kind of simulations than those reported in Table
the computation of D
1, we tried to optimize the choice of a threshold sLOF defined by: if LOF (Xi ) > sLOF then the
observation Xi is considered to be an outlier. We remark that it is not really possible to choose
a priori k and sLOF with respect to α. Table 2 provides the results of simulations from 10000
independent replications and the seven probability distributions. We have chosen to optimize a
sum of empirical type I (case K = 0) and II (case K = 5 and K = 10) errors. This leads one to
choose sLOF = 8 for n = 100 as well as for n = 1000.

Table 2: Numerical choice of the LOF threshold sLOF : Average frequencies of potential outliers with
LOF detector when there are K outliers from the seven considered probability distributions, and for
several values of K, sample size n and threshold sLOF . Note that 10000 independent replications are
generated and multiplicative outliers (see below) are generated.
sLOF

K

n = 100

n = 1000

2

4

6

8

10

12

K=0

0.903

0.447

0.285

0.222

0.174

0.150

K=5

1.000

1.000

1.000

0.996

0.961

0.863

K = 10

1.000

0.860

0.422

0.232

0.141

0.106

K=0

0.955

0.533

0.328

0.237

0.196

0.179

K=5

1.000

1.000

1.000

1.000

1.000

0.995

K = 10

1.000

1.000

0.981

0.888

0.744

0.581

Student, Tukey and M ADe detectors are more or less based on Gaussian computations. We would
not be surprised if those methods failed to detect outliers when the distribution of X is ”far” from the
Gaussian distribution (but these usual detections of outliers, for instance the Student detection obtained
on Studentized residuals from a least squares regression, are done even if the Gaussian distribution is
not attested). Moreover, the computations of these detectors’ thresholds are based on an individual
detection of outlier, i.e. a test deciding if a fixed observation Xi0 is an outlier or not. Hence, if we apply
them to each observation of the sample, the probability to detect an outlier increases with n. This is
not exactly the same as deciding whether if there are no outliers in a sample, which is the object of test
problem (3.1).
b Jn , it is appropriate to change the thresholds of these detectors
Then, to compare these detectors to D
following test problem (3.1) and its precision (3.2). Hence we have to define a threshold s > 0 of this
test from the relation P(∃k = 1, · · · , n, Xk > s) = α. Therefore, from the independence property
1 − α = P(X1 < s)n implying that s has to satisfy P(X1 > s) = (1 − (1 − α)1/n ) ' α/n when n is large
and α close to 0 (typically α = 0.01). In the sequel we are going to compute the confidence factors of
previous famous detectors to the particular case of absolute values of Gaussian variables. Hence, when
for Z a N (0, 1) random variable and if n is large, then P(|Z| > sG ) '

α
n

with sG = qN (0,1) (1 −

α
2n ).

Then, we define:
1. The Student detector 2: we consider that Xk from (X1 , · · · , Xn ) is an outlier when Xk > X n +
p
p
√
p
α
cS × σ n , with 2/ 2π + cS (π − 2)/π = sG implying cS = qN (0,1) (1 − 2n
) − 2/π π/(π − 2)
(here we assume that n is a large number inducing that the Student distribution with (n − 1)
degrees of freedom could be approximated by the standard Gaussian distribution).

2. The Tukey detector 2: we consider that Xk from (X1 , · · · , Xn ) is an outlier when Xk > Q3 + cT ×
IQR. In the case of the absolute value of a standard Gaussian variable, Q1 ' 0.32 and Q3 ' 1.15
implying cT = (sG − 1.15)/0.83 ' 1.20 × sG − 1.38.
3. The M ADe detector 2: we consider that Xk from (X1 , · · · , Xn ) is an outlier when Xk − Q2 >
cM × median(|X1 − Q2|, · · · , |Xn − Q2|). In the case of the absolute value of a standard Gaussian
variable, Q2 ' 0.67 and median(|X1 − Q2|, · · · , |Xn − Q2|) ' 0.40. This induces cM ' 2.50 ×
sG − 1.69.
First results of Monte-Carlo experiments for samples without outlier: the size of the test
We apply the different detectors in different frames and for several probability distributions which are:
• The absolute value of Gaussian distribution with expectation 0 and variance 1, denoted N (0, 1)
(case A01 );
• The exponential distribution with parameter 1, denoted E(1) (case A01 );
• The Gamma distribution with parameter 3, denoted Γ(3) (case A01 );
• The Weibull distribution with parameters (3, 4), denoted W (3, 4) (case A01 );
• The absolute value of a Student distribution with 2 degrees of freedom, denoted |t(2)| (case A2 );
• The standard log-normal distribution, denoted log −N (0, 1) (not case A01 or A2 );
• The absolute value of a Cauchy distribution, denoted |C| (case A2 ).
In the sequel, we will consider samples (X1 , · · · , Xn ) following these probability distributions, for n =
100 and n = 1000, and for several numbers of outliers.
We begin by generating independent replications of samples without outlier, which corresponds to be
under H0 . Then we apply the outlier detectors. The results are reported in Table 3.

A first conclusion from this simulations is the following: as we already said, the original Student, Tukey
e Jn because their empirical sizes came out
and M ADe detectors can not be compared to the detector D
larger: they are not constructed to answer to our test problem. Therefore we are going now to consider
only their global versions Student 2, Tukey 2 and M ADe 2. Moreover, as they have been constructed
from the Gaussian case, these second versions of detectors provide generally poor results in case of
non-Gaussian distributions especially for Student, log-normal and Cauchy distributions (where outliers
are always detected while there are no generated outliers).

Table 3: The size of the test: average frequencies (Av. Freq.) of potential outliers with the different
outlier detectors, for the different probability distributions, n = 100 and n = 1000, while there is no
generated outlier in samples. Here α = 0.007 and 20000 independent replications are generated.
N (0, 1)

E(1)

Γ(3)

W (3, 4)

|t(2)|

log −N (0, 1)

|C|

e Jn
Av. Freq. D

0.007

0.008

0.008

0.008

0.010

0.010

0.018

Av. Freq. LOF

0.001

0.029

0.013

0

0.643

0.259

0.970

Av. Freq. Student

0.861

0.993

0.921

0.308

1

1

1

Av. Freq. Tukey

0.804

0.991

0.918

0.327

1

1

1

Av. Freq. Adj. Tukey

0.213

0.344

0.408

0.353

0.844

0.710

0.981

Av. Freq. M ADe

0.751

0.995

0.879

0.163

1

1

1

Av. Freq. Student 2

0.002

0.110

0.017

0

0.655

0.515

0.936

Av. Freq. Tukey 2

0.022

0.486

0.119

0

0.951

0.937

1

Av. Freq. M ADe 2

0.023

0.624

0.112

0

0.971

0.975

1

N (0, 1)

E(1)

Γ(3)

W (3, 4)

|t(2)|

log −N (0, 1)

|C|

e Jn
Av. Freq. D

0.009

0.009

0.009

0.009

0.014

0.011

0.016

Av. Freq. LOF

0.005

0.023

0.019

0.001

0.843

0.281

0.998

Av. Freq. Student

1

1

1

0.986

1

1

1

Av. Freq. Tukey

1

1

1

0.939

1

1

1

0.492

0.899

0.950

0.796

1

1

1

1

1

1

0.664

1

1

1

Av. Freq. Student 2

0.006

0.552

0.112

0

1

0.998

1

Av. Freq. Tukey 2

0.008

0.943

0.259

0

1

1

1

Av. Freq. M ADe 2

0.008

0.991

0.246

0

1

1

1

n = 100

n = 1000

Av. Freq. Adj Tukey
Av. Freq. M ADe

Second results of Monte-Carlo experiments for samples with outliers
Now, we consider the cases where there is a few number of outliers in the samples (X1 , · · · , Xn ).
Denote K the number of outliers, and ` > 0 a real number which represents a positive real number.
We generated 3 kinds of contaminations:
• A shift contamination: (X(1) , · · · , X(n−K) , X(n−K+1) +`, · · · , X(n) +`) instead of (X(1) , · · · , X(n) ).
We chose ` = 10. By the way the cluster of outliers is necessarily separated from the cluster of
non-outliers.
• A multiplicative contamination: (X(1) , · · · , X(n−K) , 3X(n−K+1) , · · · , 3X(n) ) instead of (X(1) , · · · , X(n) ).
By the way the cluster of outliers is necessarily separated from the cluster of non-outliers.
• A point contamination: (X(1) , · · · , X(n−K) , `, · · · , `) instead of (X(1) , · · · , X(n) ). We chose ` =
1000. By the way, the cluster of outliers is generally separated to the cluster of non-outliers (but
not necessary, especially for Student or Cauchy probability distributions).
First, we consider the second versions of Student, Tukey and M ADe detectors. But we also consider
parametric versions of these detectors, which are denoted Student-para, Tukey-para and M ADe -para:
the thresholds and confidence factors are computed and used with the knowledge of the probability
distribution of the sample (these thresholds change following the considered probability distributions).
e Jn or LOF are non-parametric detectors. We chose these
Hence, they are parametric detectors while D
parametric versions because they allow to obtain the same size of all the detectors and then a comparison of the test powers is more significant.

First results are reported in Tables 4 (n = 100) and 5 (n = 1000) for K shifted outliers (K = 5
or K = 10). From Tables 4 and 5, it appears:
• Student 2, as well as Student-para detectors are not really good choices for the detection of outliers
because they are not robust statistics (the empirical variance is totally modified by the values of
outliers).
• Tukey and M ADe detectors provide more and less similar results. But even their parametric
versions are not able to detect outliers for skewed distributions (Student, log-normal and Cauchy
e Jn clearly provides better results, even if they are not very
distributions). With the same size, D
accurate (especially for the Cauchy distribution).
• LOF detector does not provide accurate results (especially when K = 10). It is certainly a more
interesting alternative in case of multivariate data.

Table 4: Average frequencies (Av. Freq.) of potential outliers with the different outlier detectors and
average number (Av. Numb.) of detected outliers when outliers are detected, for n = 100, with K
shifted outliers. Here α = 0.007 and 20000 independent replications are generated.
n = 100

K

N (0, 1)

E(1)

Γ(3)

W (3, 4)

|t(2)|

log −N (0, 1)

|C|

e Jn
Av. Freq. D

0

0.007

0.008

0.008

0.008

0.010

0.010

0.018

5

0.998

0.562

0.574

1

0.848

0.813

0.066

10

1

0.973

0.959

1

1

1

0.076

5

5.12

5.23

5.25

5.14

5.09

5.07

9.17

10

10.60

10.64

10.58

10.62

10.05

10.05

10.01

0

0.001

0.111

0.018

0

0.655

0.515

0.934

5

0

0.013

0.004

0

0.538

0.411

0.687

10

0

0

0

0

0.241

0.126

0.628

5

0

1

1

0

1.04

1.02

1.67

10

0

0

0

0

1.01

1

1.66

0

0.022

0.483

0.118

0

0.951

0.937

1

5

1

1

1

1

1

1

1

10

1

1

0.999

1

1

1

1

5

5.01

5.35

5.05

5

5.10

5.09

10.32

10

10

10.14

9.67

10

10

10

13.19

0

0.023

0.619

0.109

0

0.971

0.975

1

5

1

1

1

1

1

1

1

10

1

1

0.999

1

1

1

1

5

5.01

5.67

5.06

5

5.24

5.37

12.66

10

10

10.38

9.86

10

10

12.16

16.43

0

0.001

0.029

0.013

0

0.643

0.259

0.970

5

0.998

0.521

0.123

1

0.309

0.129

1

10

0.046

0.011

0

0.200

0.03

0.003

0.548

5

5

3.97

2.86

5

2.71

2.03

3.39

10

9.11

1.10

-

9.47

1.56

1.40

2.27

0

0.002

0

0.001

0.005

-

0

-

5

0

0

0

1

-

0

-

10

0

0

0

0

-

0

-

5

0

0

0

5

-

0

-

10

0

0

0

0

-

0

-

0

0.022

0.013

0.014

0.025

0.008

0.009

0.007

5

1

0.991

0.960

1

0.009

0.026

0.005

10

1

0.903

0.782

1

0.007

0.026

0.004

5

5.01

4.87

4.40

5.01

1

1.03

1.94

10

10

7.27

4.50

10

1

1.01

1.88

0

0.023

0.014

0.013

0.025

0.007

0.009

0.007

5

1

0.995

0.969

1

0.009

0.024

0.006

10

1

0.966

0.867

1

0.008

0.025

0.005

5

5.01

4.92

4.47

5.01

1

1.02

1.94

10

10

8.51

5.46

10

1

1.02

1.89

Av. Numb.

Av. Freq. Student 2

Av. Numb.

Av. Freq. Tukey 2

Av. Numb.

Av. Freq. M ADe 2

Av. Numb.

Av. Freq. LOF

Av. Numb.

Av. Freq. Student-para

Av. Numb.

Av. Freq. Tukey-para

Av. Numb.

Av. Freq. M ADe -para

Av. Numb.

Table 5: Average frequencies (Av. Freq.) of potential outliers with the different outlier detectors and
average number (Av. Numb.) of detected outliers when outliers are detected, for n = 1000, with K
shifted outliers. Here α = 0.007 and 10000 independent replications are generated.
n = 1000

K

N (0, 1)

E(1)

Γ(3)

W (3, 4)

|t(2)|

log −N (0, 1)

|C|

e Jn
Av. Freq. D

0

0.009

0.009

0.009

0.009

0.014

0.011

0.016

5

1

0.892

1

1

0.241

0.517

0.072

10

1

0.997

0.991

1

0.987

1

0.088

5

5.23

5.27

5.32

5.27

5.34

5.14

8.85

10

10.37

10.82

10.75

10.41

10.05

10.06

7.83

0

0.006

0.565

0.111

0

1

0.998

1

5

1

1

1

1

1

1

1

10

1

1

1

1

1

1

1

5

5

5.02

5.01

5

4.69

4.98

4.74

10

10

10

9.97

10

7.73

9.75

4.59

0

0.008

0.949

0.256

0

1

1

1

5

1

1

1

1

1

1

1

10

1

1

1

1

1

1

1

5

5.01

7.90

5.27

5

22.34

20.09

64.41

10

10.01

12.64

10.23

10.01

22.49

20.06

67.76

0

0.008

0.992

0.237

0

1

1

1

5

1

1

1

1

1

1

1

10

1

1

1

1

1

1

1

5

5.01

9.86

5.26

5

26.94

28.52

83.04

10

10.01

14.55

10.23

10.01

27.10

28.46

86.59

0

0.001

0.029

0.013

0

0.843

0.281

0.970

5

1

0.802

0.473

1

0.291

0.113

0.65

10

0.961

0.014

0.004

1

0.125

0.011

0.571

5

5.01

4.27

3.37

5

2.21

1.21

3.07

10

9.93

3.36

3.50

10

1.96

1.63

2.92

0

0.006

0.004

0.005

0.007

-

0

-

5

1

0.652

0.979

1

-

0

-

10

1

0.017

0.275

1

-

0

-

5

5

1.21

2.48

5.01

-

0

-

10

10

1

1.07

10

-

0

-

0

0.008

0.008

0.008

0.010

0.007

0.007

0.007

5

1

1

1

1

0.007

0.013

0.007

10

1

1

1

1

0.007

0.013

0.007

5

5.01

5.01

5.01

1.02

1.95

1

2.00

10

10.01

10.01

9.93

10.01

1

1

2

0

0.008

0.007

0.008

0.010

0.007

0.007

0.007

5

1

1

1

1

0.007

0.013

0.037

10

1

1

1

1

0.008

0.013

0.007

5

5.01

5.01

5.01

5

1.01

1.01

2

10

10.01

10.01

9.94

10.01

1

1

2

Av. Numb.

Av. Freq. Student 2

Av. Numb.

Av. Freq. Tukey 2

Av. Numb.

Av. Freq. M ADe 2

Av. Numb.

Av. Freq. LOF

Av. Numb.

Av. Freq. Student-para

Av. Numb.

Av. Freq. Tukey-para

Av. Numb.

Av. Freq. M ADe -para

Av. Numb.

Those conclusions are confirmed when both the other contaminations (multiplicative and point) are
used for generating outliers (see Table 6 and 7).
Conclusions of simulations
b Jn provide a very good trade-off for the detection of outliers for a very large
The log-ratio detector D
choice of probability distributions compared with usual outliers detectors or their fitted versions. The
second non-parametric outlier detector, LOF, is not really as accurate (especially for K = 10), its size
is not controlled and results did not depend on the choice of α (a theoretical study should be done for
writing the threshold as a function of α).

5

Application to real data

We apply the theoretical results to real datasets of detailed data on individual transactions in the
used car market. The purpose of the experiment was to detect as many outliers as possible. The
original dataset contains information about n = 6079 transactions on the car Peugeot 207 1.4 HDI 70
Trendy Berline including year and month which is the manufacture date, the price, and the number
of kilometres driven. We then define 3 variables: Age (the age of the car, in months), Price (in euros)
and Mileage (in km). We chose these cars because they were advertised often enough to permit us to
create a relatively homogeneous sample. Figure 1 depicts the relationship between the price and some
variables: Price with Mileage, Price with Age. Such data were collected by Autobiz society, and can
be used for forecasting the price of a car following its age and mileage. Hence it is crucial to construct
a model for the price from a reliable data set including the smallest number of outliers.
We now apply our test procedure to identify eventual outlying observations or atypical combination
between variables. After preliminary studies, we chose two significant characteristics for each car of
the sample. The first one is the number of kilometres per month. The second one is the residual
obtained, after an application of the exponential function, from a robust quantile regression between
the logarithm of the price as the dependent variable and Age and Mileage as exogenous variables (an
alternative procedure for detecting outliers in robust regression has been developed in Gnanadesikan
and Kettenring, 1972). The assumption of independence is plausible for both these variables’ residuals.
Figure 2 exhibits the boxplots of the distributions of those two variables.
b Jn is carried out on those two variables with Jn = 20 (given by the empirical
The outlier test D
choice obtained in Section 4 with n = 6079). As the sample size is large, we can accept to eliminate
data detected as outliers while there are not really outliers and we chose α = 0.05. The results are

Table 6: Average frequencies (Av. Freq.) of potential outliers with the different outlier detectors and
average number (Av. Numb.) of detected outliers when outliers are detected for n = 100 and n = 1000,
with K multiplicative outliers. Here α = 0.007 and 20000 independent replications are generated.
n = 100

K

N (0, 1)

E(1)

Γ(3)

W (3, 4)

|t(2)|

log −N (0, 1)

|C|

e Jn
Av. Freq. D

0

0.007

0.008

0.008

0.008

0.010

0.010

0.018

5

1

0.620

1

1

0.743

0.797

0.221

10

1

1

1

1

1

1

0.910

5

5.03

5.04

5.03

5.03

5.05

5.03

5.17

10

10.01

10.01

10.01

10.01

10

10

10

0

0.002

0

0.001

0.005

-

0

-

5

0.022

0

0.021

1

-

0

-

10

0.019

0

0.022

0.190

-

0

-

5

1

0

1

4.08

-

0

-

10

1

0

1

1.04

-

0

-

0

0.022

0.013

0.014

0.025

0.008

0.009

0.007

5

1

0.983

1

1

0.065

0.304

0.024

10

1

0.980

1

1

0.063

0.305

0.022

5

5

3.83

5

5

1.03

1.21

1

10

9.99

4.45

9.84

10

1.03

1.19

1.01

0

0.023

0.014

0.013

0.025

0.007

0.009

0.007

5

1

0.981

1

1

0.065

0.303

0.023

10

1

0.977

1

1

0.063

0.303

0.022

5

5

3.81

5

5

1.03

1.21

1

10

9.99

4.48

9.77

10

1.03

1.20

1.01

n = 1000

K

N (0, 1)

E(1)

Γ(3)

W (3, 4)

|t(2)|

log −N (0, 1)

|C|

e Jn
Av. Freq. D

0

0.009

0.009

0.009

0.009

0.014

0.011

0.016

5

1

1

1

1

0.933

0.997

0.200

10

1

1

1

1

1

1

0.947

5

5.06

5.06

5.06

5.06

5.08

5.06

5.34

10

10.02

10.03

10.03

10.03

10.03

10.03

10.02

0

0.006

0.004

0.005

0.007

-

-

-

5

1

0.996

1

1

-

0

-

10

1

0.664

1

1

-

0

-

5

5

2.46

5

5

-

0

-

10

10

1.25

5.10

10

-

0

-

0

0.008

0.008

0.008

0.010

0.007

0.007

0.007

5

1

1

1

1

0.061

0.441

0.021

10

1

1

1

1

0.061

0.443

0.021

5

5

5

5

5

1.02

1.32

1.01

10

10

9.97

10

10

1.03

1.34

1.01

0

0.008

0.007

0.008

0.010

0.007

0.007

0.007

5

1

1

1

1

0.061

0.441

0.021

10

1

1

1

1

0.060

0.450

0.022

5

5

5

5

5

1.03

1.32

1.01

10

10

9.97

10

10

1.03

1.34

1.01

Av. Numb.

Av. Freq. Student-para

Av. Numb.

Av. Freq. Tukey-para

Av. Numb.

Av. Freq. M ADe -para

Av. Numb.

Av. Numb.

Av. Freq. Student-para

Av. Numb.

Av. Freq. Tukey-para

Av. Numb.

Av. Freq. M ADe -para

Av. Numb.

Table 7:

Average frequencies (Av. Freq.) of potential outliers with the different outlier detectors

and average number (Av. Numb.) of detected outliers when outliers are detected, for n = 100 and
n = 1000, with K point outliers. Here α = 0.007 and 20000 independent replications are generated.
n = 100

K

N (0, 1)

E(1)

Γ(3)

W (3, 4)

|t(2)|

log −N (0, 1)

|C|

e Jn
Av. Freq. D

0

0.007

0.008

0.008

0.008

0.010

0.010

0.018

5

1

1

1

1

1

1

0.719

10

1

1

1

1

1

1

0.979

5

5.11

5.14

5.19

5.15

5.43

5.27

5.83

10

10.63

10.67

10.75

10.68

11.10

10.98

11.25

0

0.002

0

0.001

0.005

-

0

-

5

0

0

0

1

-

0

-

10

0

0

0

0

-

0

-

5

0

0

0

5

-

0

-

10

0

0

0

0

-

0

-

0

0.022

0.013

0.014

0.025

0.008

0.009

0.007

5

1

1

1

1

1

1

0.005

10

1

1

1

1

1

1

0.004

5

5.01

5

5.01

5.01

5

5

1.01

10

10

10

10

10

10

10

1

0

0.023

0.014

0.013

0.025

0.007

0.009

0.007

5

1

1

1

1

1

1

0.006

10

1

1

1

1

1

1

0.006

5

5.01

5.01

5.01

5.01

5.01

5.01

1.03

10

10

10

10

10

10

10

1

n = 1000

K

N (0, 1)

E(1)

Γ(3)

W (3, 4)

|t(2)|

log −N (0, 1)

|C|

e Jn
Av. Freq. D

0

0.009

0.009

0.009

0.009

0.014

0.011

0.016

5

1

1

1

1

1

1

0.246

10

1

1

1

1

1

1

0.709

5

5.23

5.27

5.32

5.26

5.69

5.28

7.76

10

11.03

11.11

11.22

11.13

11.09

11.35

12.73

0

0.006

0.004

0.005

0.007

-

0

-

5

1

1

1

1

-

0

-

10

1

1

1

1

-

0

-

5

5

5

5

5

-

0

-

10

10

10

10

10

-

0

-

0

0.008

0.008

0.008

0.010

0.007

0.007

0.007

5

1

1

1

1

1

1

0.007

10

1

1

1

1

1

1

0.007

5

5.01

5.01

5.01

5.01

5.01

5.01

1

10

10.01

10.01

10.01

10.01

10.01

10.01

1.01

0

0.008

0.007

0.008

0.010

0.007

0.007

0.007

5

1

1

1

1

1

1

0.007

10

1

1

1

1

1

1

0.008

5

5.01

5.01

5.01

5.01

5.01

5.01

1

10

10.01

10.01

10.01

10

10.01

10.01

1.01

Av. Numb.

Av. Freq. Student-para

Av. Numb.

Av. Freq. Tukey-para

Av. Numb.

Av. Freq. M ADe -para

Av. Numb.

Av. Numb.

Av. Freq. Student-para

Av. Numb.

Av. Freq. Tukey-para

Av. Numb.

Av. Freq. M ADe -para

Av. Numb.

Figure 1: Relationship between the dependent variables and the regressors: Price with Mileage (left),

●

20000

●
●
●

5000

10000

15000

●

●
●
●
●
●
●
●
●
●
●
●
●
●
●

0

0

1000

2000

3000

4000

Price with Age (right).

Figure 2: Boxplots of kilometres per month (left) and of absolute values of quantile regression residuals
(right).

Table 8:

b Jn applied to 3 samples: the number of kilometres per month (km/m),
The outlier test D

max(km/m) − km/m and the residuals obtained from a quantile regression of the log-prices onto the
age and the mileage.
Sample

Jn

b Jn
D

t

Outliers

km/m (Sup)

20

6.7232

5.96721

n=6

km/m (Inf)

20

5.1200

5.96721

n=0

Res

20

6.3322

5.96721

n=2

presented in Tables 8, 9 and 10. Note that, concerning the study of kilometres per month (km/m),
we directly applied the test to this variable for detecting eventual ”too” large values, but also to
max(km/m) − (km/m) for detecting eventual ”too” small values.
Conclusions of the applications
We first remark that we did not get the same outliers from the different analysis. It could be expected
because the test on residuals worked as a multivariate test and identify atypical association between
the three variables Age, Mileage and Price while the tests done on kilometres/month identifies outlying
values in a bivariate case i.e. a typical association between the two variables Age and Mileage. From
a practitioner’s point of view it may be advisable to apply the test for the two cases together one
by one to be sure to detect the largest number of outliers. A second remark concerns the ”type” of
the detected outliers. We can state that concerning kilometres/month, outliers are simply the largest
values (the test did not identify outliers for ”too” small values). But for the regression residuals, the
detected outliers clearly correspond to typing errors on the prices (the prices have been replaced by the
mileages!). Thus, two kinds of outliers have been detected.

Table 9: Detailed analysis of the detected outliers obtained from the sample of kilometers per month
(large values).
Detected Outliers

Price

Mileage

outlier(1)

9590

70249

outlier(2)

11690

outlier(3)

Age

Kilometers per Month

Predicted Price

16

4391

9909

61484

14

4392

10286

10490

61655

14

4404

10280

outlier(4)

9390

61891

14

4421

10272

outlier(5)

11500

39826

9

4425

11285

outlier(6)

11900

65411

15

4361

10111

Table 10: Detailed analysis of outliers detected from the residual’s sample.

6

Detected Outliers

Price

Mileage

Age

Predicted Price

Outlier(1)

34158

34158

28

10626

Outlier(2)

29000

29000

11

11600

Proofs

Proof of Proposition 1. We begin by using the classical following result (see for example Embrechts et
al. 1997):


 



D
X(n−J) , X(n−J+1) , · · · , X(n) = G−1 ΓJ+1 /Γn+1 , G−1 ΓJ /Γn+1 , · · · , G−1 Γ1 /Γn+1 ,

(6.1)

where (Γi )i∈N∗ is a sequence of random variables such as Γi = E1 + · · · + Ei for i ∈ N∗ and (Ei )j∈N∗ is
a sequence of i.i.d.r.v. with distribution E(1). Consequently, we have




  G−1 Γ /Γ
G−1 ΓJ−1 /Γn+1
G−1 Γ1 /Γn+1 
n+1
J
D
,
 ,··· ,
 .
τn−J , τn−J+1 , · · · , τn−1 =
G−1 ΓJ+1 /Γn+1
G−1 ΓJ /Γn+1
G−1 Γ2 /Γn+1
 1


a.s.
But for j ∈ N∗ , G−1 Γj /Γn+1 = G−1
×Γj . From the strong law of large numbers, Γn+1 −→ ∞,
Γn+1
n→∞
−1
therefore since G ∈ A1 , we almost surely obtain:

G−1 Γj /Γn+1 = f1

1 
Γn+1


× 1−


f2 (Γj )
1
+O
.
log(Γn+1 )
log2 (Γn+1 )

Using once again the strong law of large numbers, we have Γn+1 ∼ n −→ ∞ almost surely. Hence, we
n→∞

can write for all j = 1, · · · , J,

G−1 Γj /Γn+1

G−1 Γj+1 /Γn+1

1−

f2 (Γj )
log(Γn+1 )

+O

1
log2 (Γn+1 )



1−

f2 (Γj+1 )
log(Γn+1 )

+O

1
log2 (Γn+1 )



=

= 1+


f2 (Γj+1 ) − f2 (Γj )
1
+O
,
2
log(Γn+1 )
log (Γn+1 )

(6.2)

using a Taylor expansion. By considering now the family (τj0 )j defined by τj0 = (τj − 1) log(n) and the
limit of the previous expansion, we obtain
0
0
0
τn−J
, τn−J+1
, · · · , τn−1



D

−→




f2 (ΓJ+1 ) − f2 (ΓJ ) , f2 (ΓJ ) − f2 (ΓJ−1 ) , · · · , f2 (Γ2 ) − f2 (Γ1 ) . (6.3)

n→∞

The function (x1 , · · · , xJ ) ∈ RJ 7→ max(x1 , · · · , xJ ) ∈ R is a continuous function on RJ and therefore
we obtain (2.5).

Proof of Proposition 2. We use the asymptotic relation (2.5). Since G−1 ∈ A01 , for k = 1, · · · , J,



f2 (Γk+1 ) − f2 (Γk ) = C2 log Γk+1 /Γk = C2 log Γk+1 /ΓJ+1 − C2 log Γk /ΓJ+1 ,
But the random variable f2 (Γk+1 ) − f2 (Γk ) = f2 (Γk + Ek+1 ) − f2 (Γk ) is absolutely continuous with
respect to Lebesgue measure since Γk and Ek+1 are independent random variables. Using once again
the property (6.1), and since for an exponential distribution E(1), G−1 (x) = − log(x), then





 D  0
0
0
− log ΓJ /ΓJ+1 , − log ΓJ−1 /ΓJ+1 , · · · , − log Γ1 /ΓJ+1 = E(1)
, E(2)
, · · · , E(J)
.

0 ≤ E0 ≤ · · · ≤ E0
where (Ej0 )j is a sequence of i.i.d.r.v. following a E(1) distribution and E(1)
(2)
(J) is the

order statistic from (E10 , · · · , EJ0 ). Consequently,



f2 (ΓJ+1 ) − f2 (ΓJ ), f2 (ΓJ ) − f2 (ΓJ−1 ) , · · · , f2 (Γ2 ) − f2 (Γ1 )






= C2 − log ΓJ /ΓJ+1 , log ΓJ /ΓJ+1 − log ΓJ−1 /ΓJ+1 , · · · , log Γ2 /ΓJ+1 − log Γ1 /ΓJ+1


D
0
0
0
0
0
= C2 E(1)
, E(2)
− E(1)
, · · · , E(J)
− E(J−1)

With y = x/C2 and using (2.5), this implies
P

max

{τj0 } ≤ x

j=n−J,··· ,n−1



−→
n→∞

−→
n→∞

0
0
0
0
0
P E(1)
≤ y, E(2)
≤ y + E(1)
, · · · , E(J)
≤ y + E(J−1)




0
0
J! P E10 ≤ y, E10 ≤ E20 ≤ y + E10 , · · · , EJ−1
≤ EJ0 ≤ y + EJ−1
.

The explicit computation of this probability is possible. Indeed:

0
0
P E10 ≤ y, E10 ≤ E20 ≤ y + E10 , · · · , EJ−1
≤ EJ0 ≤ y + EJ−1
Z y+eJ−2 Z y+eJ−1
Z y Z y+e1 Z y+e2
=
···
e−e1 e−e2 e−e3 · · · e−eJ−1 e−eJ de1 de2 de3 · · · deJ−1 deJ
0

e1

= 1−e


−y

Z

e2
y Z y+e1

eJ−2
y+e2

eJ−1
y+eJ−2

Z

Z

···
0

e1

e2

e−e1 e−e2 e−e3 · · · e−2eJ−1 de1 de2 de3 · · · deJ−1

eJ−2

Z Z
Z
Z y+eJ−3

 y y+e1 y+e2
1
−y
−2y
1−e
···
e−e1 e−e2 e−e3 · · · e−3eJ−2 de1 de2 de3 · · · deJ−2
= 1−e
2
0
e1
e2
eJ−3
..
..
..
..
..
..
..
=
.
.
.
.
.
.
.
Z y Z y+e1



1
1 − e−y 1 − e−2y × · · · × 1 − e−(J−2)y
e−e1 e−(J−1)e2 de1 de2
=
(J − 2)!
Z0 e1


 y −Je1
1
−y
−2y
−(J−1)y
=
e
de1
1−e
1−e
× ··· × 1 − e
(J − 1)!
0



1
=
1 − e−y 1 − e−2y × · · · × 1 − e−Jy .
J!
Then, we obtain (2.7).

Proof of Proposition 3. Such a result can be obtained by modifications of Propositions 1 and 2. Indeed,
we begin by extending Proposition 1 in the case where Jn −→ ∞ and Jn / log n −→ 0. This is possible
n→∞

n→∞

D

since Γn+1 /n = 1 + n−1/2 εn with εn −→ N (0, 1) from usual Central Limit Theorem. Using the Deltan→∞
D

method, we also obtain log(Γn+1 /n) = n−1/2 ε0n with ε0n −→ N (0, 1). Hence, for any j = 1, · · · , Jn ,
n→∞

from (6.3),

 G−1 Γ /Γ

j
n+1
 −1
= log(n)
G−1 Γj+1 /Γn+1
1 
D
= f2 (Γj+1 ) − f2 (Γj ) + O
log(n)
1 
D
= C2 log(Γj+1 /Γjn ) + O
.
log(n)

0
0
Denote Fn the cumulative distribution function of τn−J
,
·
·
·
,
τ
, and Fen the one of f2 (ΓJn +1 ) −
n−1
n


f2 (ΓJn ), · · · , f2 (Γ2 ) − f2 (Γ1 ) = C2 log(ΓJn +1 /ΓJn ), · · · , log(Γ2 /Γ1 ) . Then, using the second equation
0
τn−j

D

of the proof of Proposition 1, for all (x1 , · · · , xJn ) ∈ (0, ∞)Jn ,
Fn (x1 , · · · , xJn ) = Fen (x1 + u1n , · · · , xJn + uJnn ),

1
with uin = O log(n)
. But it is clear that the probability measure of f2 (ΓJn +1 ) − f2 (ΓJn ), · · · , f2 (Γ2 ) −

f2 (Γ1 ) is absolutely continuous with respect to the Lebesgue measure on RJn . Thus, the partial

derivatives of the function Fen exist. Then from a Taylor-Lagrange expansion,
Fen (x1 + u1n , · · · , xJn + uJnn ) = Fen (x1 , · · · , xJn ) +

Jn
X

ujn ×

j=1
j
j=1 un

PJn

where (x01 , · · · , x0Jn ) ∈ (0, ∞)Jn . Hence, we obtain
Jn
C 0 log
n

for some positive real numbers C and

C 0.

Fn (x1 , · · · , xJn )

×

∂
Fn (x01 , · · · , x0Jn ),
∂xj

∂
0
∂xj Fn (x1 , · · ·

, x0Jn ) ≤ C

j
j=1 un

PJn

≤

Consequently, we have:
Fen (x1 , · · · , xJn ).

∼
n→∞

Now, we are going back to the proof of Proposition 2 by computing Fen (x1 , · · · , xJn ). This leads to
compute the following integral:
Z yJ −1 +eJ −2 Z
Z y1 Z y2 +e1 Z y3 +e2
n
n
···
0

e1

e2

eJn −2

yJn +eJn −1

e−e1 e−e2 e−e3 · · · e−eJn −1 e−eJn de1 de2 de3 · · · deJn −1 deJn ,

eJn −1

with yi = xi /C2 , and with the same iteration than in the proof of Proposition 2, we obtain
Fen (x1 , · · · , xJn ) =

Jn
Y


1 − e−jxJn −j+1 /C2 .

j=1

Then, by considering the vector ((n − j)τj0 )n−Jn ≤j≤n−1 and the continuity of the function max, we have
for all x ≥ 0
Pr

max

{(n − j)τj0 } ≤ x



j=n−Jn ,··· ,n−1

∼

1 − e−x/C2

 Jn

.

(6.4)

n→∞

To achieve the proof, we use the Slutsky’s Theorem. Indeed, we have sJn =

1
Jn

PJn

0
j=1 j τn−j

with

Jn → ∞. As a consequence,
s Jn

=

PJn
Jn

C2 X
j=1 j 
j log Γj+1 /Γj + O
Jn
Jn log(n)

(6.5)

Jn

C2 X
Jn 
j log Γj+1 /Γj + O
Jn
log(n)

(6.6)

Jn
1 X
Jn 
εj + O
,
Jn
log(n)

(6.7)

j=1

=

j=1

=

j=1

with (ε1 , · · · , εJn ) a family of i.i.d.r.v. with exponential distribution of parameter 1/C2 . Using the Law
of Large Numbers and the condition Jn / log(n) −→ 0, we deduce that
n→∞
P

sJn −→ C2 .
n→∞

Then the proof can be concluded using (6.4), (6.8) and Slutsky’s Theorem.

(6.8)

Proof of Proposition 4. We begin by considering the proof of Proposition 1. Hence, since G−1 ∈ A2
and always with Γn+1

∼

n almost surely, we obtain for k = 1, · · · , J,

n→∞

 

 G−1 Γ /Γ
 Γ−a g (1/Γ
n+1
1
n+1 ) 1 + O(1/ log(Γn+1 ))
k
k


log
= log
G−1 Γk+1 /Γn+1
Γ−a
k+1 g1 (1/Γn+1 ) 1 + O(1/ log(Γn+1 ))

= −a log Γk /Γk+1 + O(1/ log(n)).
Then, we directly use the proof of Proposition 2.

Proof of Theorem 3.1. First consider the case G−1 ∈ A01 . Using the proof of Proposition 1, we have
 
 G−1 Γ /Γ
f (Γ ) − f2 (Γj )
1 
j
n+1
 = 2 j+1
log
+O
.
2
−1
log(n)
G Γj+1 /Γn+1
log (n)
Consequently, using G−1 ∈ A01 and therefore the definition of f2 , we obtain:
log(τj ) =


C2
1 
.
log Γj+1 /Γj + O
2
log(n)
log (n)

To prove (3.4), it is sufficient to use again the proof of Proposition 3, to normalize the numerator and
b Jn , which converges in probability to log 2/C2
denominator with log n and therefore to consider log n × L
(indeed, the median of the sample converges to log 2/λ which is the median of the distribution of the
E(λ) distribution).
When G−1 ∈ A2 , we can use the same argument that the ones of the proof of Proposition 3 with C2
replaced by a (the reminder 1/ log n obtained from the definition of A2 allows the achievement the proof
when Jn is negligible compared to log n).
Acknowledgements. The authors are extremely grateful to the referees for their impressive and
scrupulous reports with many relevant corrections, suggestions and comments which helped to improve
the contents of the paper. We also would like to thank Mia Hubert for having kindly provided R routines.

References
Abramowitz, M. and Stegun, I.A. (1964). Handbook of Mathematical Functions with Formulas, Graphs,
and Mathematical Tables. New York: Dover Publications.
Barnett,V. and Lewis, T. (1994). Outliers in Statistical Data. Wiley Series in Probability & Statistics,
Wiley.

Beckman, R.J. and Cook, R.D. (1983). Outlier....s, Technometrics, 25, 119-149.
Beirlant, J., Goegebeur, Y., Teugels, J. and Segers, J. (2004). Statistics of Extremes: Theory and
Applications. Wiley Series in Probability & Statistics, Wiley.
Beirlant, J., Vynckiera P. and Teugels, J. (1996). Tail Index Estimation, Pareto Quantile Plots Regression Diagnostics, Journal of the American Statistical Association, 91, 1659-1667.
Blair, J.M., Edwards C.A. and Johnson J.H. (1976). Rational Chebyshev approximations for the inverse
of the error function, Math. Comp. 30, 827-830.
Breunig, M.M., Kriegel, H.-P., Ng, R.T. and Sander, J. (2000). LOF: Identifying Density-based Local
Outliers. Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data.
Embrechts, P., Kleppelberg, C. and Mikosch, T. (1997). Modelling Extreme Events for Insurance and
Finance. Springer.
Gnanadesikan, R. and Kettenring, J.R. (1972). Robust estimates, residuals, and outlier detection with
multiresponse data, Biometrics, 28, 81-124.
Grubbs, F.E. (1969). Procedures for Detecting Outlying Observations in Samples. Technometrics, 11,
1-21.
Hawkins, D.M. (1980). Identification of Outliers. Chapman and Hall
Hill, B. (1975). A simple general approach to inference about the tail of a distribution. The Annals of
Statistics, 3, 1163-1174.
Hubert, M., Dierckx, G. and Vanpaemel, D. (2013). Detecting influential data points for the Hill
estimator in Pareto-type distributions. Computational Statistics and Data Analysis, 65, 13-28.
Hubert, M. and Van der Veeken, S. (2008). Outlier detection for skewed data. Journal of Chemometrics,
22, 235-246.
Hubert, M. and Vandervieren, E. (2008). An adjusted boxplot for skewed distributions. Computational
Statistics and Data Analysis, 52, 5186-5201.
Knorr, E.M., Ng, R.T. and Tucakov, V. (2000). Distance-based outliers: algorithms and applications,
The VLDB Journal, 8, 237-253.
Rousseeuw, P.J. and Hubert, M. (2011). Robust statistics for outlier detection WIREs Data Min.
Knowl. Disc., 1, 73-79.

Rousseeuw, P.J. and Leroy, A.M. (2005). Robust Regression and Outlier Detection. Wiley Series in
Probability and Statistics, Wiley.
Tietjen, G.L. and Moore, R.H. (1972). Some Grubbs-Type Statistics for the Detection of Several
Outliers, Technometrics, 14, 583-597.
Tse, Y.K. and Balasooriya, U. (1991). Tests for Multiple Outliers in an Exponential Sample, The
Indian Journal of Statistics, Series B, 53, 56-63.
Tukey, J.W. (1977). Exploratory data analysis. Reading, MA, Addison-Wesley.

