Noname manuscript No.
(will be inserted by the editor)

Adaptive group LASSO selection in quantile models

arXiv:1601.08065v2 [] 13 Jul 2016

GABRIELA CIUPERCA

Received: date / Accepted: date

Abstract The paper considers a linear model with grouped explanatory variables. If the model
errors are not with zero mean and bounded variance or if model contains outliers, then the least
squares framework is not appropriate. Thus, the quantile regression is an interesting alternative.
In order to automatically select the relevant variable groups, we propose and study here the adaptive group LASSO quantile estimator. We establish the sparsity and asymptotic normality of the
proposed estimator in two cases: fixed number and divergent number of variable groups. Numerical
study by Monte Carlo simulations confirms the theoretical results and illustrates the performance
of the proposed estimator.
Keywords group selection · quantile model · adaptive LASSO · selection consistency · oracle
properties.
Mathematics Subject Classification (2010) 62J05 · 62J07
1 Introduction
Classically, for the regression model, the errors are assumed to be independent, of mean zero and
bounded variance. Then, the model is estimated by the least squares (LS) method, eventually with
a penalty of LASSO type when automatic detection of significant variables is performed. If the
assumptions on the first two moments of the model error are not satisfied, then the LS framework
breaks down. In this case, an alternative is to consider the quantile regression with a LASSO type
penalty. This is one of the interests of this paper. The quantile regression is robust and allows
relaxation of the two first moment conditions of the model error.
Often enough in practice, for example in the variance analysis case, are considered the regression
linear models with grouped variables. For models with grouped explanatory variables it is more
meaningful to identify relevant variable groups instead of individual variables. If the errors have
GABRIELA CIUPERCA
Université de Lyon, Université Lyon 1, CNRS, UMR 5208, Institut Camille Jordan, Bat. Braconnier, 43, blvd du 11
novembre 1918, F - 69622 Villeurbanne Cedex, France,
E-mail: Gabriela.Ciuperca@univ-lyon1.fr
tel: 33(0)4.26.23.45.57, fax: 33(0)4.72.43.16.87

2

GABRIELA CIUPERCA

Normal distribution, then for detecting the relevant variable groups, the F-statistic test is used. If
the errors are not Gaussian and if more the number of groups is large, then the F-statistic test is
inappropriate. From where, another interest of this paper: we consider the quantile process with
LASSO type penalty in order to automatically detect the irrelevant variable groups.
The automatic selection method of the grouped variables using the LASSO penalties was introduced by Yuan and Lin (2008) for gaussian errors, by proposing the LASSO group penalty for
the process of the error squares sum. Several recent papers have considered group selection using
LASSO type penalties. For fixed parameter space and mean zero, finite second moment i.i.d. model
errors, Nardi and Rinaldo (2008) established the model selection consistency and asymptotic normality of nonzero group LASSO estimator. The same estimator is studied by Nardi and Rinaldo
(2008) when number of covariates is larger, for particular case of normal errors. For gaussian errors,
Xu and Ghosh (2015) realize a Bayesian variable selection by penalization of the error squares sum
with Bayesian group LASSO. For this estimation method, the posterior median estimator satisfies
the sparsity property. The adaptive group LASSO estimators, when the number p of groups is fixed,
was studied by Wang and Leng (2008). For high-dimensional model, Wei and Huang (2010) studied
the selection and estimation properties of the adaptive group LASSO, but under assumption that
the errors are gaussian. Still for the error squares sum penalized with adaptive LASSO penalty,
Zhang and Xiang (2015) consider the case of the number of groups pn converges to infinity when
n → ∞, for i.i.d. errors ε such that IE[ε] = 0 and V ar[ε] < ∞. The consistency and asymptotic normality of the parameter estimator are established. A paper that doesn’t consider the LS penalized
process, but a process associated to a twice differentiable convex function, with LASSO penalty,
for the case p large and small n was considered by Wang et al. (2015). When the number of groups
can grow at a certain polynomial rate, the automatic selection property of variable groups for a
LS process with SCAD penalty has been proven in Guo et al. (2015). Automatic selection of the
relevant variable groups, when p converges to infinity, has also considered by Zou and Zhang (2009)
penalizing the LS process with adaptive elastic-net penalty. For a review of group selection methods
and several applications of these methods the reader can see Huang et al. (2012).
In this paper we consider the model selection problem and the estimation in a linear model with
p groups of explanatory variables. We propose and study the asymptotic properties of the adaptive
group LASSO quantile estimator in two cases: p fixed and p → ∞ as n → ∞. This estimator is the
minimizer of the quantile process penalized by an adaptive group LASSO penalty. The oracle properties, i.e. the automatic selection of significant variables groups and their asymptotic distribution,
are proved.
The remainder of the paper is organized as follows. In Section 2 we present the model and
introduce some notations used throughout in this paper. Oracle properties for the adaptive group
LASSO quantile estimator are proved for p fixed in Section 3 and for p → ∞ as n → ∞ in Section 4. Section 5 reports some simulation results which illustrate the method interest. We compare
the adaptive group LASSO quantile estimation performance with the adaptive group LASSO least
squares estimations, proposed by Zhang and Xiang (2015). All proofs are given in Section 6.
2 Model and notations
In this section, we present the statistical model and we also introduce some notations used throughout in the paper.
We begin by introducing some general notations. All vectors and matrices are denoted by bold symbols and all vectors are written as column vectors. For a vector v, we denote by vt its transposed

Adaptive group LASSO selection in quantile models
L

3

P

and by kvk its Euclidean norm. Notations −→ , −→ represent the convergence in distribution and
n→∞ n→∞

in probability, respectively, as n → ∞. For a positive definite matrix M, we denote by λmin (M)
and λmax (M) its the smallest and largest eigenvalues, respectively.
We will also use the following notations: if Vn and Un are random variable sequences, Vn = oIP (Un )
means that limn→∞ IP [|Un /Vn | > e] = 0 for any e > 0, Vn = OIP (Un ) means that there exists
a finite C > 0 such that IP [|Un /Vn | > C] < e for any n and e. If Vn and Un are deterministic
sequences, Vn = o(Un ) means that the sequence Vn /Un → 0 for n → ∞, Vn = O(Un ) means that
the sequence Vn /Un is bounded for sufficiently large n.
Throughout this paper, C will denote generic constant; not depending on size n which may take
different values in different formula or even in different parts of the same formula. The value of C
is not of interest. We will also use the notation 0k for the zero k-vector.
We consider the following linear model with p groups of explanatory variables:
Yi =

p
X

Xtij β j + εi = Xti β + εi ,

j=1

i = 1, · · · , n,

(1)

with Yi , εi random variables. For each group j = 1, · · · , p, the vector of the parameters is βj ≡
(βj1 , · · · , βjdj ) ∈ Rdj and the design for observation i is Xij , a column vector of size dj . The vector
with all coefficients is β ≡ (β 1 , · · · , βp ) and for observation i, the vector with all explanatory
0
0
variables is Xi = (Xi1 , · · · , Xip ). Denote by β0j = (βj1
, · · · , βjd
) the true value (unknown) of the
j
parameter β j . For observation i, we denote by Xij,k the kth variable of the jth group.
We emphasize that for the ith sample, we observe (Yi , Xi ), i = 1, · · · , n.
The relevant groups of explanatory variables correspond to the nonzero vectors. Without loss of
generality, on suppose that the first p0 (p0 ≤ p) groups of explanatory variables are relevant:
kβ0j k 6= 0,

for all j ≤ p0

and kβ0j k = 0,

for all j > p0 ,

P
where k.k is the Euclidean norm. Let r be the total number of explanatory variables, so r = pj=1 dj .
P
0
We denote by r0 = pj=1
dj . So, p0 is the number of nonzero true parameter vectors and r0 is the
total number of parameters in these nonzero true vectors.
The multi-factor ANOVA model is an example of this model.
We introduce now the quantile framework. For a fixed quantile index τ ∈ (0, 1), the check
function ρτ (.) : R → R+ is defined by ρτ (u) = u(τ − 11u<0 ).
The quantile estimator of β, is the minimizer of the quantile process associated to model (1):
e ≡ arg min
β
n
β∈Rr

n
X
i=1

ρτ (Yi − Xti β).

(2)

For the particular case τ = 1/2 we obtain the median regression and (2) becomes the least absolute
deviations estimator. A great advantage of the quantile framework is that, compared to classical
estimation methods that are sensitive to outliers, the quantile method provides more robust estimators. Moreover, the required assumptions to the error moments are relaxed.
e = (β
e ,β
e ,··· ,β
e ) has as dj -subvector β
e
The estimator β
n
n;1
n;2
n;p
n;j for each group j = 1, · · · , p.
The quantile estimation method doesn’t perform automatic variable selection. For finding the zero

4

GABRIELA CIUPERCA

vectors, i.e. the irrelevant groups of variables, hypothesis tests are required. However when model
(1) has a large group number p, it is useful to estimate simultaneously the parameter groups and
to eliminate the irrelevant groups without crossing every time by a hypothesis test. The adaptive
LASSO penalties have the advantage of automatic selection and of parameter estimation (see for
example Zhang and Xiang (2015), Wei and Huang (2010), Wang and Leng (2008)).
In order to introduce and study the adaptive LASSO estimator, we consider the following index
set
A ≡ {j; kβ0j k 6= 0} = {1, · · · , p0 }
and Ac ≡ {j; kβ0j k = 0} = {p0 + 1, · · · , p} its complementary set. The set A contains the index set
corresponding to groups with nonzero true parameters.
For β a r-vector of parameters, we denote by βA the r0 -subvector of β which contains βj , for
j = 1, · · · , p0 . Similarly, the (r − r0 )-vector β Ac contains βj for j = p0 + 1, · · · , p.
In practice, the set A is unknown. Then, we must find the set A and estimate the corresponding
parameters.
b ∗ , which minimizes the quantile
In Sections 3 and 4 we will introduce an estimator, denoted β
n
process penalized with an adaptive group LASSO penalty, for two cases: p fixed and p → ∞ as
n → ∞. We generalize the adaptive LASSO quantile estimator proposed by Ciuperca (2016) for
individual variable selection to the case of group selection. We call this estimator, adaptive group
LASSO quantile (ag LASSO Q ) estimator.
b ∗ satisfies the oracle properties if:
We say that β
n
√ b∗
0
(i) asymptotic normality: n(β
n − β )A converges in law to a centred Normal distribution.
b ∗ k 6= 0d }] = 1.
(ii) sparsity property: limn→∞ P[A = {j = 1, · · · , p; kβ
j
n;j
3 Fixed p case
In this section we propose and study the asymptotic properties of the ag LASSO Q estimator for
the parameter β of model (1) when the group number p is fixed.
We define the ag LASSO Q estimator by:
b ∗ ≡ arg min Q(β),
β
n
β∈Rr

where Q(β) is the penalized quantile process with the adaptive group LASSO penalty:
Q(β) ≡

n
X
i=1

ρτ (Yi −

p
X
j=1

Xtij β j ) + µn

p
X
j=1

ω
bn;j kβj k,

(3)

e k−γ , γ > 0. The estimator β
b ∗ is written as β
b ∗ = (β
b∗ , · · · , β
b ∗ ) and
with the weight ω
bn;j ≡ kβ
n;j
n
n
n;1
n;p
b ∗ is a subvector of size dj , for j = 1, · · · , p.
β
n;j
For a particular case of a quantile model with non-grouped variables, dj = 1 for all j = 1, · · · , p,
we obtain the adaptive LASSO quantile estimator proposed and studied by Ciuperca (2016).

Adaptive group LASSO selection in quantile models

5

b ∗ in the fixed p case, we give the required assumptions.
Before presenting the main results for β
n
The tuning parameter µn and the constant γ are such that, for n → ∞,
µn
µn → ∞, √ → 0, n(γ−1)/2 µn → ∞.
(4)
n
For the design (Xi )16i6n we consider theP
following assumption:
n
(A1) n−1 max1≤i≤n Xti Xi −→ 0 and n−1 i=1 Xi Xti −→ Υ, with Υ a r × r positive definite matrix.
n→∞
n→∞
For the errors εi we suppose that:
(A2) (εi )16i6n are independent, identically distributed, with F : B → [0, 1] the distribution function and a continuous positive density f in a neighborhood of 0. The τ th quantile of εi is zero:
τ = F (0). Moreover, for every e ∈ int(B), 1r ∈ Rr we have
n Z xti 1r
X
√
√
1
n[F (e + v/ n) − F (e)]dv = f (e)1tr Υ1r ,
(5)
lim n−1
n→∞
2
i=1 0

where 1r is the r-vector with all components 1. The set B is a real set, with 0 ∈ B.

Assumption (A1) is standard for LASSO methods and (A2) is classic for quantile regression
(see Ciuperca (2016), Koenker (2005), Zou and Yuan (2008), Wu and Liu (2009)). Assumption
(A1) requests that the design matrix has a reasonable good behaviour. For the tuning parameter
µn , the same conditions on (4) are required in Ciuperca (2016) for adaptive LASSO quantile model
but with ungrouped explanatory variables.
We make the remark that for ANOVA model, since in the analysis of variance there is a constraint for each level of a factor, we consider as constraint that the effect of this level is zero. Then
this zero level is not considered in the model in order that assumption (A1) is satisfied.
b ∗ , let us consider the index set
In order to study the asymptotic properties of the estimator β
n
of the groups selected by the adaptive group LASSO quantile method:
b∗c its complementary set.
and A
n

b ∗ k 6= 0}
Ab∗n ≡ {j ∈ {1, · · · , p}; kβ
n;j

The following Theorem shows that the ag LASSO Q estimators with the index in the set A
are asymptotically Gaussian. Then, the estimators of the nonzero parameter vectors have the same
asymptotic distribution they would have if the zero parameter vectors were known.
√ b∗
L
0
Theorem 1 Under assumptions (A1), (A2) and condition (4), we have n(β
n − β )A −→
n→∞

−1
N 0r0 , τ (1 − τ )f −2 (0)ΥA
, with P
ΥA the submatrix of Υ with the row and column indices in
p0
dj }.
{1, · · · , d1 , d1 + 1, · · · , d1 + d2 , · · · , j=1

We give now the Karush-Kuhn-Tucker(KKT) optimality conditions, needed to prove the sparsity
b∗ .
property for β
n
For all j ∈ Ab∗n , we have, with probability one, the following dj equalities
τ

n
X
i=1

Xij −

n
X
i=1

Xij 11Yi <Xt βb ∗
i

n

b∗
µn ω
bn;j β
n;j
=
.
∗
b k
kβ
n;j

(6)

6

GABRIELA CIUPERCA

For all j 6∈ Ab∗n , for all k = 1, · · · , dj we have, with probability one, the following inequality
τ

n
X
i=1

Xij,k −

n
X
i=1

Xij,k 11Yi <Xt βb ∗ ≤ µn ω
bn;j .
i

n

(7)

The following theorem shows the sparsity property of the ag LASSO Q estimator. This result
states that the adaptive group LASSO quantile estimators of the nonzero parameter vectors are
exactly nonzero with a probability converging to one when n diverges to infinity.
Theorem 2 Under the assumptions of Theorem 1 and under the condition nγ/2−1 λn → ∞, as
n → ∞, we have limn→∞ P[Ab∗n = A] = 1.

Theorem 1 and Theorem 2 establish the asymptotic normality and the sparsity of the ag LASSO Q
estimator, which means that this estimator still share the oracle properties in the case of fixed p.

Remark 1 For the weight ω
bn;j associated to the jth group, we considered the quantile estimator
norm to the power −γ. In view of the proofs of Theorem 1 and Theorem 2, these two theorems
−1/2
e
remain true also when β
, under
n;j is replaced by any estimator of β j , with convergence rate n
assumptions (A1), (A2).
4 The case of p depending on n
Consider now same model (1) with grouped variables, but with the number p of groups depending
on n: p = pn and pn → ∞ as n → ∞. More precisely, we consider pn = O(nc ), with the P
constant
c ∈ (0, 1). For readability, we keep the notation p instead of pn . Similarly, we have r = pj=1 dj ,
with r depending on n. Always for simplicity of notation, for the design Xi , for the parameter β,
ever if their dimension depends on n, we do not put subscript n.
f of β. Afterwards, we will
We will first find the convergence rate of the quantile estimator β
n
propose for β an adaptive group LASSO quantile estimator. Even though the number p diverges as
n → ∞, this estimator keeps the oracle properties.
Since the design size depends on n, we need reconsider the assumptions on Xi . Then, let us
consider the following assumptions for the errors (εi ), design (Xi ) and for the number p of groups:
(A3) (εi )1≤i≤n are i.i.d. Let F be the distribution function and f be the density function of (εi ). The
density function f is continuously, strictly positive in a neighbourhood of zero and has a bounded
first derivative in the neighbourhood of 0. The τ th quantile of εi is zero: τ = F (0). P
(A4) There
constants 0 < m0 ≤ M0 < ∞, such that m0 ≤ λmin (n−1 ni=1 Xi Xti ) ≤
Pnexist two
−1
t
λmax (n
i=1 Xi Xi ) ≤ M0 .
1/2
(A5) (p/n) max16i6n kXi k → 0, as n → ∞.
(A6) p is such that p = O(nc ), with 0 < c < 1.
Since p → ∞, condition (5) of assumption (A2) for the case p fixed is now replaced by f ′ bounded
in the neighborhood of 0. This assumption also been considered for always high-dimensional quantile model, with seamless L0 penalty by Ciuperca (2015). In Ciuperca (2015), assumptions (A4) and
(A5) are also required. Assumption (A6) was considered by Zhang and Xiang (2015) for an highdimensional linear model where the objective function is the error squares sum, penalized with

Adaptive group LASSO selection in quantile models

7

an adaptive group LASSO penalty. Assumptions (A4), (A5), (A6) are also required for an highdimensional linear model by Zou and Zhang (2009), which penalize the LS process with adaptive
elastic-net penalty. In respect to the case p fixed, assumptions (A4) and (A5) are the similar of (A1).
We will start by finding the convergence rate of quantile estimator (2) in the case p → ∞ as
n → ∞. For this, consider the quantile process:
Gn (β) ≡

n
X
i=1

ρτ (Yi − Xti β).

For the quantile estimator existence, we assume that the total number r of parameters is strictly
less than n.
e is of
We recall that in the case p fixed, the convergence rate of the quantile estimator β
n
−1/2
order n
(see for example Koenker (2005)). We will show that, the quantile estimator has the
convergence of order (p/n)1/2 , when the explanatory group variable number diverges with the
e depends only of p and
sample size. In view of the proof of Lemma 1, the convergence rate of β
n
not of total number r of parameters, thanks to assumption (A5). One needs the convergence rate
of the quantile estimator is necessary for studying the asymptotic behaviour of the penalty which
intervenes in adaptive group LASSO quantile process.
pp
e − β 0 k = OP
Lemma 1 Under assumptions (A3)-(A6), we have kβ
n
n .
Consider now the following adaptive group LASSO quantile (ag LASSO Q ) estimator:


p
X
∗
1
b ≡ arg min  Gn (β) + λn
ω
bn;j kβj k ,
β
n
n
β∈Rd
j=1

where λn is a tuning parameter (positive) and the weights of the LASSO penalty are ω
bn;j ≡
e k−γ , with γ > 0. The relation between the tuning parameter µn of relation (3) for the case
kβ
n;j
p fixed and λn for the case p depending on n is λn = µn /n. We prefer to consider these forms as
tuning parameter and as objective process, for having a similarity with the adaptive group LASSO
LS (ag LASSO LS ) case considered by Zhang and Xiang (2015).
b ∗ we need to impose an additional condition on
In order to study the asymptotic normality of β
n
the total number of nonzero parameters. More precisely, r0 it is assumed to be the same order as
p0 . This is for controlling the penalty, so that it is smaller than the quantile process.
Concerning the size of the nonzero parameter vectors, we take the following assumption:
(A7) r0 = O(p0 ).
For the smallest nonzero vector norm and on constant c of assumption (A6) we assume:
(A8) Let us denote h0 ≡ min16j6p0 kβ0j k. There exists a constant M > 0 such that M ≤ n−α h0
and α > (c − 1)/2.
These two assumptions were also found in the paper Zhang and Xiang (2015), for ag LASSO LS
method in high-dimensional linear model, but with a supplementary condition for r: r = O(p).
Here, we do not need this requirement, since assumption (A5) is imposed. On the other hand, in

8

GABRIELA CIUPERCA

Zhang and Xiang (2015), instead of assumption (A5) the condition n−1/2 max16i6n kXi,A k2 → 0,
as n → ∞, is required.
The following theorem gives the convergence rate of the ag LASSO Q estimator when p → ∞.
We obtain the same convergence rate that of quantile estimator when group number diverges. This
convergence rate is also obtained by Zhang and Xiang (2015) for the ag LASSO LS estimator, but
for errors (ε)16i6n with mean zero and bounded variance.
Theorem 3 Under assumptions (A3)-(A6), (A8) and the tuning parameter (λn )n∈N satisfying
pp
b ∗ − β 0 k = OP
λn n(1+c)/2−αγ → 0, as n → ∞, we have kβ
n
n .

The following theorem shows the oracle properties for ag LASSO Q estimator when the number
p of groups diverges. We denote by Xi,A a r0 -vector which contains the sub-vectors Xi,j , for j ∈
{1, · · · , p0 }.

Theorem 4 Suppose that assumptions (A3)-(A6), (A8) are satisfied and also that the tuning pa(1−c)(1+γ)/2
rameter
→ ∞, λn n(c+1)/2−αγ → 0, as n → ∞. Then:
h satisfies
i λn n
∗
(i) P Abn = A → 1, for n → ∞.

(ii) If moreover assumption (A7) holds, then, for any vector u of size r0 such that kuk = 1, with
Pn
∗
√
L
−1/2 t b
notation Υn,A ≡ n−1 i=1 Xi,A Xti,A , we have n(ut Υ−1
u (β n − β0 )A −→ N 0, τ (1 −
n,A u)
n→∞

τ )f −2 (0) .

For the tuning parameter λn , the same conditions are required in Zhang and Xiang (2015) such
that, the ag LASSO LS estimator in an high-dimensional linear model satisfies the oracle properties.
e k−γ , with β
e
Remark 2 As for the case p fixed, we considered the weight ω
bn;j = kβ
n;j
n;j the quantile
estimator of the dj -vector β j , for any j = 1, · · · , p. In view of the proof of Theorem 4, the oracle
1/2
e
properties for ag LASSO Q estimator remain true also when β
n;j is replaced by any (p/n)
estimator of β j , under assumptions (A3)-(A6).
Remark 3 If h0 , defined in assumption (A8), doesn’t depend on n, then α = 0. In this case, the
conditions required on (λn )n∈N in Theorem 3 imply γ > 2c/(1 − c), and then γ can take values in
the interval (0, ∞). The value of γ increase with that of c ∈ (0, 1). For example, if c = 1/2 then
γ > 2.

5 Simulations
In order to evaluate the performance of the proposed estimation method, Monte Carlo simulations are realized in this section. To assess this performance we compare the ag LASSO Q and
ag LASSO LS estimation methods.
The design Xi is generated in the same way as in paper Wei and Huang (2010): X = (X1 , · · · , Xp ),
with the group explanatory variables Xj = (X5(j−1)+1 , · · · , X5j ), for all j = 1, · · · , p. We first
generate r = 5p independent random variables R1 , · · · , Rr of standard normal distribution. We

Adaptive group LASSO selection in quantile models

9

also generate the variables Zj of multivariate normal distribution with mean zero and covariance
Cov(Zj1 , Zj2 ) = 0.9|j1 −j2 | . Finally, the variables X1 , · · · , Xr are generated as:
X5(j−1)+k =

Zj + R5(j−1)+k
√
,
2

1 ≤ j ≤ p,

1 ≤ k ≤ 5.

Two model errors are considered: Normal N (0, 32 ) and Cauchy C(0, 32 ). For the parameters we
take: β01 = (0.5, 1, 1.5, 1, 0.5), β02 = (1, 1, 1, 1, 1), β03 = (−1, 0, 1, 2, 1.5), β04 = (−1.5, 1, 0.5, 0.5, 0.5)
and all other parameters are zero vectors. The nonzero vectors were also considered in Example 2
of Wei and Huang (2010) for errors N (0, 32 ), p = 10, when the parameters were estimated by LS
method with adaptive group LASSO penalty.
The constant c of assumption (A6) is c = 0.43. Then, we will consider the following value couples for n and p: (30, 5), (60, 5), (60, 10), (100, 10), (200, 10), (400, 15), (1000, 25) and (1000, 100).
On
hand, p0 will always be equal to 4. The response variable Y is generated as: Yi =
Pp the tother
0
j=1 Xij β j + εi , for i = 1, · · · , n.
We will compare the obtained results by the adaptive group LASSO quantile method, proposed in
this paper, with those obtained by the adaptive group LASSO LS method, proposed by Wei and Huang
(2010), Zhang and Xiang (2015).
For simulations, we used the R language. After a scale transformation, we can use the group LASSO
methods instead of the adaptive LASSO group methods. Then, in order to calculate the adaptive
group LASSO LS estimations we have used the function grpreg of package grpreg, the tuning parameter being chosen on a value grid, using the AIC criterion. In order to calculate the adaptive
group LASSO quantile estimations, we have used the function groupQICD of package rqPen and
the tuning parameter varies on a value grid.
For each considered case, 1000 Monte Carlo replications was made.
In Table 1 we give how the two estimation methods identify the parameter vectors (zero or
nonzero), for the part that contains the four nonzero parameter vectors β0j , j = 1, · · · , 4, and
for the part with p − 4 zero vectors. We present the minimum, three quartiles (Q1, median, Q3),
the mean and the maximum of the number of nonzero vectors (j = 1, · · · , 4), respectively, zero
(j = 5, · · · , p), found by the two estimation methods.
For n large (equal to 100, 200, 400, 1000), we observe that for errors of N (0, 32 ) law, the two
estimation methods well identify the zero and nonzero parameter vectors. However, for Cauchy errors, the ag LASSO LS method poorly identifies nonzero vectors (the group of the four significant
variables). The zero vectors are very well identified by the two methods.
For n small (equal to 30 or 60), the two estimation methods well identify the four relevant variable groups, that errors are Normal or Cauchy (except for ag LASSO Q, in the case n = 60,
p = 5, ε ∼ C(0, 32 )). However, the (p − 4) irrelevant variable groups are not well identified by the
ag LASSO LS method.
Conclusion
For gaussian errors, the ag LASSO LS method identifies well the two (relevant and irrelevant)
variable groups for n large. For n small, the irrelevant variable groups are not well identified. For
Cauchy errors, this method, either does not identify the relevant variable groups or irrelevant variable groups, regardless of the value n. Then, for Cauchy errors, the ag LASSO LS estimations do
not have the sparsity property.

10

GABRIELA CIUPERCA

The ag LASSO Q method, for the two types of errors, identifies the two variable groups (significant and irrelevant), the precision increasing with n. Then, the ag LASSO Q estimations have the
sparsity property.
We conclude then that the simulations confirm the theoretical results for the ag LASSO Q estimators.
6 Proofs
In this section we provide the proofs of all results presented in Sections 3 and 4.
6.1 Proofs for results of Section 3
Proof of Theorem 1. The proof is similar to that of Theorem 4.1 of Zou and Yuan (2008).
√ b∗
√
0
b n , and in general n(β − β 0 ) ≡ u ≡ (u1 , · · · , up ), with uj =
We denote n(β
n − β ) ≡ u
(uj,1 , · · · , uj,dj ), for j = 1, · · · , p.

Since Yi = Xti β 0 + εi , then Yi − Xti β =

Xti u
√
n

+ εi . Let us consider the following random variables

Di ≡ (1 − τ )11εi <0 − τ 11εi ≥0 ,
n
1 X
Di ,
vn ≡ √
n i=1
√
n Z Xti u/ n
X
Bn (u) ≡
[11εi <t − 11εi <0 ]dt
i=1

and the random vector

(8)

0

n

1 X
Xi Di .
zn ≡ √
n i=1

Obviously, IE[Di ] = 0 et IE[zn ] = 0r . By the CLT, using assumptions (A1) and (A2), we have
L

zn −→ N (0r , τ (1 − τ )Υ),
n→∞

L

vn −→ N (0, τ (1 − τ )).
n→∞

(9)

b n is the minimizer of the following random process:
The vector u
Ln (u) ≡

p
n
X
X




Xt u 
uj
ρτ εi − √i − ρτ (εi ) + µn
ω
bn;j β0j + √ − kβ 0j k ,
n
n
i=1
j=1

which can be written under the following form:
Ln (u) = [ztn u + Bn (u)] + µn

p
X
j=1

ω
bn;j



√
 n
uj
β 0j + √
− kβ0j k √ .
n
n

(10)

We first study the last sum of the right hand side of (10).
For all j ≤ p0 (thus kβ0j k 6= 0) we have, since the quantile estimators are consistent:
P

ω
bn;j −→ kβ0j k−γ 6= 0
n→∞

(11)

30

5

60

5

60

10

100

10

200

10

400

15

1000

25
100

ε
C
N
C
N
C
N
C
N
C
N
C
N
C
N
C
N

min
LS
Q
0
0
4
3
0
1
4
3
0
0
4
3
0
3
4
3
0
3
4
3
0
3
4
3
0
4
4
4
0
3
4
3

Q1
LS
4
4
0
4
4
4
0
4
0
4
0
4
0
4
0
4

nonzero true parameter vectors
Q2
mean
Q3
Q
LS
Q
LS
Q
LS
3
4
3
3.5
3.3
4
4
4
4
4
3.8
4
3
1
4
1.3
3.6
4
4
4
4
4
3.9
4
3
4
4
3.4
3.5
4
4
4
4
4
3.9
4
4
0
4
0.9
3.9
1
4
4
4
4
3.9
4
3
0
4
0.5
3.7
1
4
4
4
4
3.9
4
3
0
4
0.4
3.6
11
4
4
4
4
3.99
4
4
0
4
0.5
4
1
4
4
4
4
4
4
3
0
4
0.3
3.5
0
4
4
4
4
3.98
4

Q
4
4
4
4
4
4
4
4
4
4
0
4
4
4
4
4

max
LS
Q
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
4
3
4
4
4

true
1
1
1
1
6
6
6
6
6
6
4
11
21
21
96
96

min
LS
Q
0
0
0
0
0
0
0
0
0
2
0
1
0
2
0
1
0
2
0
3
8
10
0
9
19
19
17
20
94
96
92
95

Q1
LS
0
0
0
0
0
0
5
0
6
0
11
9
21
20
96
95

zero true parameter vectors
Q2
mean
Q
LS
Q
LS
Q
0
0
1
0.1
0.7
0
0
1
0.01
0.5
1
1
1
0.7
0.8
0
0
1
0.1
0.7
5
0
5
0.8
5.2
4
0
4
0.06
4.3
4
6
5
5
5
4
0
5
2
4.4
6
6
6
5.7
5.7
5
4
6
3.1
5.5
11
11
11
10.8
10.9
11
10
11
9.3
10.8
21
21
21
20.8
20.7
21
21
21
20.3
20.9
96
96
96
95.8
96
96
96
96
95.4
96

Q3
LS
0
0
1
0
0
0
6
5
6
6
11
11
21
21
96
96

Q
1
1
1
1
6
5
6
5
6
6
11
11
21
21
96
96

max
LS
Q
1
1
1
1
1
1
1
1
6
6
6
6
6
6
6
6
6
6
6
6
11
11
11
11
21
21
21
21
96
96
96
96

Adaptive group LASSO selection in quantile models

p

11

Table 1 Model selection results by ag LASSO Q and ag LASSO LS methods for p0 = 4, c = 0.43, errors N (0, 32 )
or C(0, 32 ).

n

12

GABRIELA CIUPERCA

and by elementary calculus

utj β 0j
√  0
n kβ j + n−1/2 uj k − kβ 0j k −→
.
n→∞ kβ 0 k
j

(12)

Then, using condition µn n−1/2 → 0, when n → ∞, taking into account relations (11) and (12), by
Slutsky’s Lemma, we have:
µn

p0
X
j=1


 P
ω
bn;j kβ0j + n−1/2 uj k − kβ0j k −→ 0.

For j > p0 , we have β 0j = 0dj . Then:
assumption n

(γ−1)/2

(13)

n→∞


√  0
P
n kβj + n−1/2 uj k − kβ0j k = kuj k. Since ω
bn;j −→ ∞, by

µn → ∞, we have n

n→∞

−1/2

P

µn ω
bn;j −→ ∞. Thus
n→∞


√
 P
n
0, if uj = 0dj
µn ω
bn,j √ kβ0j + n−1/2 uj k − kβ0j k −→
∞, if kuj k =
6 0.
n→∞
n

(14)

Then, taking into account relations (13) and (14), we have the following result for the third term
of the right hand side of (10):
µn

p
X
j=1

with

ω
bn;j



√
p
 n P X
uj
W (β0j , u),
− kβ0j k √ −→
β 0j + √
n
n n→∞ j=1

(15)


0
 0, if β j 6= 0dj
W (β 0j , u) ≡ 0, if β 0j = 0dj and uj = 0dj

∞, if β 0j = 0dj and uj 6= 0dj .

On the other hand, by the two results of (9), we have for the first two terms of the right hand side
of (10), with z a random d-vector of law N (0d , τ (1 − τ )Υ), that
L

ztn u −→ zt u,
n→∞

P

Bn (u) −→

n→∞

1
f (0)ut Υu.
2

Taking into account these last two results and relation (15), then, Ln (u) of relation (10) has an
asymptotic distribution:
p
X
1
L
W (β 0j , u).
Ln (u) −→ zt u + f (0)ut Υu +
n→∞
2
j=1

b n = (b
b 2n ), where u
b 1n
Let us denote u =P
(u1 , u2 ) with u1 of size r0 , u2 of size r − r0 and u
u1n , u
p0
0
b n = arg minu∈Rr Ln (u), we obtain that
contains the first
j=1 dj = r elements of u. Since u
P

L

b 1n −→ N (0r0 , τ (1 − τ )f −2 (0)Υ−1
b 2n −→ 0r−r0 et u
u
A ).
n→∞

n→∞



Adaptive group LASSO selection in quantile models

13

√ b∗
L
0
Proof of Theorem 2. By Theorem 1, for all j ∈ A we have that n(β
n;j − β j ) −→ N (0dj , τ (1 −
−2
τ )f (0)ΥAj ) as n → ∞. The square matrix ΥAj of size dj × dj is the submatrix of Υ with the
row and column indices in {dj−1 + 1, dj−1 + 2, · · · , dj }, with d0 = 0. Since β0j 6= 0dj , then
lim P[A ⊆ Ab∗n ] = 1.

(16)

n→∞

To finish the proof we show that for all j 6∈ A we have P[j ∈ Ab∗n ] → 0 as n → ∞. Since j 6∈ A, then
β 0j = 0dj . Considering the Eucildean norm for equalities (6) we have with probability one, since we
suppose j ∈ Ab∗n , that:
µn ω
bn;j < 2k

n
X
i=1

Xij k ≤ 2

n
X
i=1

kXij k = 2

n
X
i=1




dj
X

k=1

1/2

2 
Xij,k

.

By the Cauchy-Schwarz inequality, we have that,


1/2 
1/2
dj
dj
n
n
X
X
X
X
1
1
2 
2 

Xij,k
Xij,k
≤
=
n
n
i=1
i=1
k=1

k=1

n

1X
kXij k2
n i=1

!1/2

.

Then, taking into account assumption (A1), there exists a bounded constant C1 > 0 such that
1
µn ω
bn;j < 2
n

n

1X
kXij k2
n i=1

!1/2

≤ C1 < ∞.

(17)

On the other hand, left-hand side of inequality (17), can be written:
µn ω
bn;j
µn
nγ/2
=
.
e kγ n
n
nγ/2 kβ
n;j

Since we have supposed that j ∈ Ab∗n and j 6∈ A, we have that for all ǫ > 0, there exists ηǫ > 0
e k−1 > ηǫ ] > 1 − ǫ. The last two relations, together with the supposition
such that P[n−1/2 kβ
n;j
γ/2−1
n
µn → ∞, imply, for all constant A > 0,
lim P

n→∞




µn ω
bn;j
> A = 1.
n

(18)

Then, relations (17) and (18) are in contradiction. Thus

lim P[j ∈ Ac ∩ Ab∗n ] = 0.

n→∞

The theorem follows from relations (16) and (19).

(19)


14

GABRIELA CIUPERCA

6.2 Proofs for results of Section 4
Proof of Lemma 1. We show that for all ǫ > 0, there exists a constant Bǫ > 0 (without loss of
generality, we consider Bǫ > 0, otherwise we take |Bǫ |) large enough such that for n large enough:
r 



p
0
0
P inf Gn β + Bǫ
u > Gn (β ) ≥ 1 − ǫ.
(20)
n
kuk=1
For this, we consider for some constant C > 0, the expectation of the difference:
r
r 
 
 
 X


n
p
p t
IE ρτ εi − C
IE Gn β 0 + C
u − Gn (β 0 ) =
Xi u − ρτ (εi )
n
n
i=1
=

n
X

IE

i=1

"Z

C

√p

#

t
n Xi u

110<εi <t dt =

0

n Z
X
i=1

0

C

√p

t
n Xi u

[F (t) − F (0)] dt.

(21)

p
By assumption (A6) we have p/n → 0. Moreover, by assumption (A5), we have that np Xti u = o(1),
for kuk = 1. Thus, by mean value theorem and since the density f has a bounded first derivative
in the neighbourhood of 0, relation (21) becomes:
!
n
n
pX t t
f (0) 2 p X t 2
(X u) + o
u (Xi Xi )u .
C
=
2
n i=1 i
n i=1
Then, taking into account assumption (A4),
r 
 

n
X
p
1
0
0
2 f (0) p 1
(Xt u)2 (1 + o(1)).
IE Gn β + C
u − Gn (β ) = C
n
n
2 n n i=1 i

(22)

Let be the following random variable
r
r


p t
p
Xi u − ρτ (εi ) − C
Di Xti u
Ri ≡ ρτ εi − C
n
n
and the following random vector
Wn ≡ C

r

n

pX
Di Xti ,
n i=1

with the random variable Di defined by (8). The vector Wn is the similar of the vector zn when p
was fixed. Then, the process Gn can be written:
r 
r 

 

n
X
p
p
[Ri − IE[Ri ]].
Gn β 0 + C
u − Gn (β 0 ) = IE Gn β 0 + C
u − Gn (β 0 ) + Wn u +
n
n
i=1
(23)
First all, remark that
n

kuk2 λmin



1
1X t
1
u Xi Xti u ≤ kuk2 λmax Xi Xti .
Xi Xti ≤
n
n i=1
n

(24)

Adaptive group LASSO selection in quantile models

15

Since the errors (εi )16i6n are independent, using also |Ri | < C
with assumption (A5), we obtain

pp

n

|Xti u| 11|εi |<C √ p |Xt u| , together
n

i

r
r
i
h
p
p
√
kXi k ≤ C
max kXi k = o(1),
IE 11|εi |<C p |Xt u| ≤ C
i
n
n
n 16i6n
which imply, since (εi )16i6n are i.i.d.,
"

n
X
[Ri − IE[Ri ]]
IE
i=1

#2

=

n
X
i=1

IE[Ri − IE[Ri ]]2 ≤

n
X

IE[R2i ]

i=1

and by assumptions (A3), (A5) together with relation (24), we have
i
pX t 2 h
≤C
|Xi u| IE 11|εi |<C √ p |Xt u| = o
i
n
n i=1
n

2

n

pX t
u Xi Xti u
n i=1

!

= o(p).

(25)

For the last relation we have used assumption (A4).P
Let be the following random variable Un ≡ p−1/2 ni=1 [Ri − IE[Ri ]]. Then, relation (25) implies
IE[Un2 ] = o(1). This, together with IE[Un ] = 0, imply, by the Bienaymé-Tchebychev inequality, that
P
P
Un −→ 0. Thus ni=1 [Ri − IE[Ri ]] = oP (p1/2 ). Then, relation (23) becomes
n→∞

r 
r 
 


p
p
0
0
0
u − Gn (β ) = IE Gn β + C
u − Gn (β 0 ) + Wn u + oP (p1/2 )
Gn β + C
n
n

which is equal to, using (22):
"
f (0)
= C2
p
2

n

1X t
u Xi Xti u
n i=1

!

n
X
Di Xti
√
+C p
n
i=1

√

(26)

! #

u (1 + oP (1)) + oP (p1/2 ).

Since P
IE[Di ] = 0, V ar[Di Xti u] = τ (1 − τ )ut Xi Xti u and kuk = 1, then, using (24), we have that
n
−1/2
t
n
i=1 Di Xi u converges in law to a centred Gaussian distribution. Taking into account assumptions (A4) and (A6), for B large enough, we obtain
!
r 

n
X
1
p
Gn β 0 + B
ut Xi Xti u (1 + oP (1)) > 0,
(27)
u − Gn (β 0 ) = B 2 f (0)p
n
n i=1
for n large enough. Thus, relation (20) follows taking into account assumptions (A3) and (A4). 
Proof of Theorem 3. We have the following inequality, with probability 1, for the quantile
e :
estimator β
n;j
e − β 0 k + min kβ
e k.
min kβ0j k ≤ max kβ
n;j
j
n;j
j∈A

j∈A

j∈A

e − β 0 k = OP ((p/n)1/2 )) = OP (n(c−1)/2 ). On the other
By Lemma 1, we have that maxj∈A kβ
n;j
j
hand, we denoted in assumption (A8), h0 = minj∈A kβ0j k. Then, we have with probability one,

16

GABRIELA CIUPERCA

e k and taking into account assumption (A8): M ≤
h0 n−α ≤ OP (n(c−1)/2−α ) + n−α minj∈A kβ
n;j
e k. Hence:
oP (1) + n−α minj∈A kβ
n;j



M nα
e
lim P min kβn;j k >
= 1.
n→∞
j∈A
2

(28)

We consider the r-vector u such that kuk = 1. Similarly to Theorem 2.1 of Zhang and Xiang (2015),
we have, for a constant B,
p
X
j=1

e k
kβ
n;j

−γ

r
r p0
r

 X


p0
p
p
pX e
0
0
0
0
−γ
e
kβj + B
kβn;j k
kβn;j k−γ kuj k.
uj k − kβj k ≥
uj k − kβj k ≥ −B
kβj + B
n
n
n
j=1
j=1

Since kuk = 1, taking into account relation (28), applying the Cauchy-Schwarz inequality, we obtain
−

r

p0
p X e
kβ k−γ kuj k ≥ −
λn
n j=1 n;j

r

p0
X

p
e k−2γ 1/2 kuk ≥ −
kβ
λn
n;j
n
j=1

r

and by assumption λn n(1+c)/2−αγ −→ 0, we obtain:
n→∞

= OP

p
n

r
√
√
λn p0
p
p λn p0
γ ≥ −
α γ
e k
n minj∈A kβ
n Mn
n;j
2

.

(29)

The Theorem is proved if we have the similar of inequality (20) for
r
r 
r 




p
X
p
p
p
1
1
0
0
0
0
0
λn ω
bn;j kβj +
Gn β + B
u − Gn (β )+
Buj k − kβj k ≡ Qn β + B
u −Qn (β 0 ).
n
n
n
n
n
j=1

We show that for all ǫ > 0 there exists Bǫ large enough, such that, for any n large enough
r 



p
0
0
u > Qn (β ) > 1 − ǫ.
P inf Qn β + Bǫ
n
kuk=1

(30)

By the definition of Qn , we have for all constant B > 0, that
r 
r 
r




p
X
p
p
p
1
1
Qn β 0 + B
λn ω
bn,j kβ0j +
u −Qn (β 0 ) = Gn β 0 + B
u − Gn (β 0 )+
Buj k − kβ0j k
n
n
n
n
n
j=1

and using relations (27) and (29)

p
> B f (0)
n
2

!
n
p
1X t
t
u Xi Xi u (1 + oP (1)) − BOP
.
n i=1
n

Relation (30) follows from the last relation, for n and B large enough and using assumption (A4). 
b ∗ belongs, with a probability converging
Proof of Theorem 4. (i) By
Theorem 3, we have
that β
n

p
to one, to the set: Vp (β 0 ) ≡ β; kβ − β 0 k ≤ B np , with B > 0 large enough as in relation (30).

Adaptive group LASSO selection in quantile models

17

For p > p0 , we show that for all β = (β A , β Ac ) ∈ Vp (β 0 ) such that kβA − β 0A k = O
all constant C ∈ (0, B), we have
Qn (β A , 0p−p0 ) =

pp
n

min √ Qn (βA , βAc ),

kβ Ac k≤C

and for
(31)

p
n

with a probability tending to one, as sample
 size n → ∞.
Let us consider the parameter set Wn ≡ β ∈ Vp (β 0 ); kβAc k > 0 .
b ∗ ∈ Wn ] → 0, as n → ∞. For this, we firstly consider two parameter vectors
We show that P[β
n
(1)
(1)
(1)
(1)
β = (β A , βAc ) ∈ Wn and β(1) = (β A , β Ac ) ∈ Vp (β 0 ), such that βA = β A and β Ac = 0d−d0 .
Let us take the difference of the objective random process for the two parameter vectors. We denote
this difference Dn (β, β (1) ):
Dn (β, β (1) ) ≡ Qn (β) − Qhn (β (1) )
i P
Pn
p
bn;j kβ j k.
= n−1 i=1 ρτ (Yi − Xti β) − ρτ (Yi − Xti β(1) ) + j=p0 +1 λn ω

(32)

From Knight (1998), we have the following identity, for any x, y ∈ R:
Z y
(11x≤t − 11x≤0 )dt.
ρτ (x − y) − ρτ (x) = y(11x≤0 − τ ) +
0

Using this relation for the first sum of (32), we obtain:
n
n
X

1
1 X
Xi [11Yi −Xti β(1) ≤0 − τ ]
ρτ (Yi − Xti β) − ρτ (Yi − Xti β (1) ) = (β − β (1) )t
n i=1
n
i=1
(1)
n Z t
1 X Xi (β−β )
+
[11Yi −Xti β(1) ≤t − 11Yi −Xti β(1) ≤0 ]dt ≡ T1n + T2n .
n i=1 0

(33)

For T1n , since the density f is bounded in a neighbourhood of 0, we have, by assumption (A5),
that:
n

IE[T1n ] = (β−β(1) )t

n

f (0) X
1X
Xi [F (Xti (β (1) −β0 ))−F (0)] = (β−β (1) )t
Xi Xti (β 0 −β(1) )(1+o(1)).
n i=1
n i=1

Pn
Then,
|IE[T1n ]| ≤ kβ − β (1) k · n−1 i=1 Xi Xti · kβ0 − β(1) kf (0)(1 + o(1)). Since P
the matrix
P
n
n
n−1 i=1 Xi Xti is Hermitian, we have, taking into account assumption (A4), that n−1 k i=1 Xi Xti k
Pn
(1)
0
(1)
−1
t
= λmax (n
k·kβ −β k. Therefore
i=1 Xi Xi ) ≤ M0 . Hence, we have |IE[T1n ]| ≤ M0 f (0)kβ−β
(1) 2
IE[T1n ] = O(kβ − β k ). By calculations analogous to IE[T1n ], using independence of (εi )16i6n ,
2
2
we have that IE[T1n
] = Cn−1 kβ − β(1) k3 → 0, for n → ∞. Since V ar[T1n ] ≤ IE[T1n
], using the
Bienaymé-Tchebychev inequality, we have:
e 2 (1 + oP (1)).
T1n = Ckβ − βk

(34)

Consider now T2n of relation (33), which can be written as:
T2n = n

−1

n Z
X
i=1

0

Xti (β−β(1) )

[11εi ≤t−Xt (β0 −β(1) ) − 11εi ≤−Xt (β0 −β(1) ) ]dt.
i

i

18

GABRIELA CIUPERCA

Taking into account that β ∈ Vp (β 0n ), together with assumptions (A3), (A5), we get IE[T2n ] =
Pn R Xt (β−β(1) )
Pn R Xt (β−β(1) )
n−1 i=1 0 i
[F (t−Xti (β 0 −β(1) ))−F (−Xti (β 0 −β(1) ))]dt = n−1 i=1 0 i
[tf (Xti (β (1)
0
−β )) + o(t)]dt.
By the proof of Lemma 1, assumptions (A3), (A5), we obtain that f (Xti (β (1) − β 0 )) P
is bounded by
a constant C > 0. Thus, as for T1n , using assumption (A4) and the fact that n−1 ni=1 kXi k2 −

P
P
P
tr n−1 ni=1 Xi Xti −→ 0, we have |IE[T2n ]| ≤ Cn−1 ni=1 kXi k2 kβ−β(1) k·kβ(1) −β0 k+o n−1 ni=1
n→∞

2
Xti (β − β(1) ) = Ckβ − β (1) k2 . We show similarly that IE[T2n
] = Cn−1 kβ − β(1) k3 . Then, by the
Bienaymé-Tchebychev inequality, we get:
T2n = Ckβ − β(1) k2 (1 + oP (1)).

(35)

Hence, by relations (34), (35), we obtain
T1n + T2n = Ckβ − β(1) k2 (1 + oP (1)).

(36)

This last relation together with relations (33), (36), give for relation (32):
Dn (β, β

(1)

) = Ckβ − β

(1) 2

k (1 + oP (1)) +

p
X

j=p0 +1

On the other hand, by Lemma 1, we have
ω
bn;j =

1
e kγ
kβ
n;j

=

λn ω
bn;j kβj k.


 
p −γ/2
= OP
n
− β 0j kγ
1

e
kβ
n;j

and moreover for all j ≥ p0 + 1, since β ∈ Wn ⊆ Vp (β 0 ) we have 0 < kβj k = O
p
X

j=p0 +1

Thus

Dn (β, β(1) )
kβ − β (1) k

λn ω
bn;j kβj k =

p
X

λn OP

j=p0 +1

≥ Ckβ − β (1) k(1 + oP (1)) +


 
p (1−γ)/2
.
n
p
X

λn OP

j=p0 +1

pp
n . Then

 

p −γ/2
n

.

  
1/2
We have that kβ−β(1) k = O np
. Since p/n = O(nc−1 ), under the assumption that λn n(1−c)(1+γ)/2 →
∞, as n → ∞, and p > p0 , we have
Dn (β, β (1) )
kβ − β(1) k



≥ λn OP nγ(1−c)/2 .

(37)

To finish the proof of relation (31), consider now other two parameter vectors: β 0 the true value
(1)
(1)
(1)
(1)
and β(1) a parameter such that β (1) ≡ (β A , βAc ), β A = βA , βAc = β 0Ac = 0d−d0 . We obtain as
for (36) that:
Dn (β 0 , β(1) ) = n−1

n h
i
X
ρτ (Yi − Xti β 0 ) − ρτ (Yi − Xti β (1) ) = Ck(β 0 − β (1) )A k2 (1 + oP (1)).
i=1

Adaptive group LASSO selection in quantile models

19

Then
Dn (β 0 , β(1) )
kβ 0 − β(1) k

0

= Ck(β − β

(1)

)A k(1 + oP (1)) = OP

  
p 1/2
n



= OP n(c−1)/2 .

(38)

Since λn n(1−c)(1+γ)/2 → ∞ as n → ∞, we have that (37) is much bigger than (38), for n large
enough. Then, relation (31) follows.
To finish the proof of (i), we will show that


b ∗ k > 0 = 1.
lim P min kβ
(39)
n;j
n→∞

j∈A

b ∗ −β0 k. By Theorem
b ∗ k ≥ minj∈A kβ0 k−maxj∈A kβ
With probability 1, we have that: minj∈A kβ
n;j
j
n;j
j
3 and assumption (A8), we have,


∗
M
−α
b
lim P n min kβn;j k ≥
= 1,
n→∞
j∈A
2

which implies



b ∗ k > 0 = 1,
lim P n−α min kβ
n;j

n→∞

j∈A

from which relation (39) follows. Relations (39) and (31) imply (i).
b ∗ can be written with a
(ii) Taking into account claim (i) and assumption (A7), the estimator β
n
p
∗
b = β 0 + p δ, with δ = (δ A , δ Ac ), δ Ac = 0r−r0 , kδ A k ≤ C. Then
probability converging to 1 as β
n
n
r
r 

 


n
1X
p
p
ρτ Yi − Xti (β 0 +
Qn β 0 +
δ − Qn (β 0 ) =
δ) − ρτ (εi ) + P,
(40)
n
n i=1
n


λn ω
bn;j kβ j k − kβ0j k .
p
b ∗ k − kβ0 k = C p δ t β0 , with a probability converging to 1.
For all j ∈ {1, · · · , p0 } we have kβ
n;j
j
n j j

with P ≡

Pp0

j=1

On the other hand, using assumption (A8) and Lemma 1 we have for the quantile estimators
r 




p
e k = kβ 0 k + OP
kβ
≥ h0 + OP n−(1−c)/2 = O(nα ) + OP n−(1−c)/2 = O(nα ).
n;j
j
n

Thus, we have ω
bn;j ≤ OP (n−αγ ), for all j ∈ A. Then, for the the second term on the right hand
side of (40), we have with a probability converging to one:
r
r
p0
p0
X
X
p
p t 0
0
0
−αγ
λn ω
bn;j kβj +
|P| =
λn
δ j k − kβj k ≤ Cn
δ j βj
n
n
j=1
j=1
r
r
p0
p X
p 0
0
−αγ
kδ j k · kβ j k ≤ C
λn
r λn n−αγ = Cr0 λn n(c−1)/2−αγ .
(41)
≤ Cn
n j=1
n
We study now the first term of the right hand side of (40), which can be written as:
r 



n 
1X
p
0
t
ρτ Yi − Xi β +
δ
− ρτ (εi )
n i=1
n

20

GABRIELA CIUPERCA

1
=
n

r

n

n

pX t
1X
Xi δ[11ε≤0 − τ ] +
n i=1
n i=1

Z √ np Xti δ
0

[11εi ≤t − 11εi ≤0 ]dt ≡ J1 + J2 .

Since IE[J1 ] = 0, using assumption (A5) and the Cauchy-Schwarz inequality, we have:
V ar[J1 ] ≤ IE[J12 ] =

n
n
X
X
p
p
1 p
t 2
kXi,A k2 kδ A k2 ≤ C 3 n max kXi,A k2 −→ 0.
(X
δ)
≤
τ
(1−τ
)
τ
(1−τ
)
i
2
3
n→∞
n n
n
n 1≤i≤n
i=1
i=1

Using assumption (A2), we have for the expectation of J2 :
Z √ np Xti δ
n
1
f (0) p t 1 X
Xi Xti δ(1 + o(1)).
IE[J2 ] =
δ
(tf (0) + o(t2 ))dt =
n 0
2 n n i=1

(42)

Using assumption (A4) and relation (24), we have that
f (0)pn−1 kδk22 λmin n−1

n
X
i=1

Taking into account the fact that

n
X


Xi Xti .
Xi Xti ≤ IE[J2 ] ≤ f (0)pn−1 kδk22 λmax n−1
i=1

kδk22

kδ A k22

≤ C, we have:
p
IE[J2 ] = Cf (0) .
n
 
p 3/2
→ 0, as n → ∞.
n
=

(43)


Similarly we obtain V ar[J2 ] = O n1

But IE[J2 ] = O nc−1 and by assumption (A7), together with relation (41), we have |P| ≤



|P|
= OP λn n(c+1)/2−αγ → 0 and thus
OP λn nc n(c−1)/2−αγ = OP λn n(3c−1)/2−αγ . Then IE[J
2]
IE[J2 ] ≫ |P|.
p
Hence, minimizing (40) amounts to minimizing J1 + J2 , with respect to np δ. Using relation (42),
we obtain:
pp n
r
n
X
p
f (0) p t
1X
0
t
n
δ))− ρτ (εi )] =
δ Υn,A δ A (1 + oP (1)).
[ρτ (Yi − Xi (β n +
Xti,A δ A [11εi <0 − τ ]+
n i=1
n
n i=1
2 n A
The minimizer of the right hand side of the last equation is:
r
n
X

p
1 1
Xi,A (11εi ≤0 − τ ) .
δA = −
Υ−1
n,A
n
n f (0)
i=1

(44)

For studying (44), let us consider the following independent random variable sequence, for i =
1, · · · , n,
Wi ≡ (f (0))−1 ut Υ−1
n,A XiA (11εi ≤0 − τ ),
Pn
0
with u a vector of dimension r , such that kuk2 = 1. We have that IE[Wi ] = 0 and i=1 V ar[Wi ] =
nτ (1 − τ )(f (0))−2 ut Υ−1
n,A u. Then, by CLT for independent random variable sequences (Wi )16i6n ,
we have
b − β0 )
√
ut (β
L
A
A
−→ N (0, 1).
(45)
nf (0) q
n→∞
−1
t
τ (1 − τ )(u Υn,A u)
p
b − β 0 = p δ A , together with relations
Claim (ii) results by taking into account of the fact that β
A
A
n
(44), (45).


Adaptive group LASSO selection in quantile models

21

References
Ciuperca, G., (2015). Model selection in high-dimensional quantile regression with seamless L0 penalty. Statistics
and Probability Letters, 107, 313-323.
Ciuperca, G., (2016). Adaptive LASSO model selection in a multiphase quantile regression. Statistics, DOI:
10.1080/02331888.2016.1151427.
Guo, X., Zhang, H., Wang, Y., Wu, J.L., (2015). Model selection and estimation in high dimensional regression
models with group SCAD. Statistics and Probability Letters, 103, 86-92.
Huang, J., Breheny, P., Ma, S., (2012). A selective review of group selection in high-dimensional models. Statistical
Science, 27, No. 4, 481-499.
Knight, K., (1998). Limiting distributions for L1 regression estimators under general conditions. The Annals of
Statistics, 26, 755-770.
Koenker, R., (2005). Quantile Regression. Cambridge University Press.
Nardi, Y., Rinaldo, A., (2008). On the asymptotic properties of the group lasso estimator for linear models. Electronic
Journal of Statistics, 2, 605-633.
Xu, X., Ghosh, M., (2015). Bayesian variable selection and estimation for group Lasso. Bayesian Analysis, 10(4),
909-936.
Yuan, M., Lin, Y., (2006). Model selection and estimation in regression with grouped variables. J. R. Statist. Soc.
B, 68(1), 49-67.
Zhang, C., Xiang, Y., (2015). On the oracle property of adaptive group LASSO in high-dimensional linear models.
Statistical Papers, 57(1), 249-265.
Zou, H., Yuan, M., (2008). Composite quantile regression and the oracle model selection theory. The Annals of
Statistics, 36(3), 1108-1126.
Zou, H., Zhang, H.H., (2009). On the adaptive elastic-net with a diverging number of parameters. The Annals of
Statistics, 37(4), 1733-1751.
Wang, H., Leng, C., (2008). A note on adaptive group lasso. Computational Statistics and Data Analysis, 52,
5277-5286.
Wang, L., You, Y., Lian, H., (2015). Convergence and sparsity of Lasso and group Lasso in high-dimensional
generalized linear models. Statistical Papers, 56, No. 3, 819-828.
Wei, F., Huang, J., (2010). Consistent group selection in high-dimensional linear model. Bernoulli, 16(4), 1369-1384.
Wu, Y., Liu, Y., (2009). Variable selection in quantile regression. Statistica Sinica, 19, 801-817.

