arXiv:1607.05536v1 [math.ST] 19 Jul 2016

Adaptive Fused LASSO in Grouped Quantile
Regression
GABRIELA CIUPERCA

∗

Université Claude Bernanrd Lyon 1, UMR 5208, Institut Camille Jordan, France
July 20, 2016

Abstract

This paper considers quantile model with grouped explanatory variables.
In order to have the sparsity of the parameter groups but also the sparsity between two successive groups of variables, we propose and study an
adaptive fused group LASSO quantile estimator. The number of variable
groups can be fixed or divergent. We find the convergence rate under classical assumptions and we show that the proposed estimator satisfies the
oracle properties.
Keywords:

group selection; quantile regression; adaptive fused LASSO; selection con-

sistency; oracle properties.
AMS 2010 subject classifications : Primary 62F35; secondary 62F12.

∗ Université Claude Bernanrd Lyon 1, Institut Camille Jordan, Bat. Braconnier, 43, blvd du 11 novembre 1918,
F - 69622 Villeurbanne Cedex, France, E-mail: Gabriela.Ciuperca@univ-lyon1.fr,

1

Adaptive Fused LASSO in Grouped Quantile Regression

1 Introduction
The idea of this paper comes from the ascertainment that in many practical applications, for
studying a process or a random variable in function of grouped explanatory variables, we want to
identify significant groups of variables but also to make a hierarchy between these groups. The
explanatory variables can be continuous or discrete. The most common example of linear model
with grouped variables is the multivariate variance analysis. But in many situations for theoretical
study of the linear models, classical assumptions are imposed on errors: zero mean and bounded
variance, which is not often the case in applications. Then, if these classical assumptions are not
satisfied or if the model has heavy-tailed errors, a very interesting approach is the quantile method.
Moreover, compared to classical estimation methods (least squares, least absolute deviations) which
give the model behaviour around of the mean or of the median, the quantile method offers a very
complex and global insight. This method allows to study how the explanatory variables influence
the response variable distribution. For a complete overview on quantile method, we refer the reader
to book of Koenker (2005). Then, in order to cover more possible cases of models, in this paper, we
will consider grouped quantile regression, which allows the relaxation of the classical conditions on
the two first moments of the model error. We also want to identify the relevant variable groups, an
automatic selection by a LASSO type method of the significant variable groups being more meaningful that an automatic selection of individual variables. The LASSO estimator, introduced by
Tibshirani (1996) for the least squares framework, doesn’t always satisfy the automatic selection,
and then a solution is the adaptive LASSO estimator, proposed initially by Zou (2006).
The LASSO methods have been the subject of active research in the last decade. We give here only
the references concerning the LASSO methods for models with grouped variables. Zhang and Xiang
(2015), Ciuperca (2016a) have considered the adaptive group LASSO in high-dimensional linear
model by penalizing the sum of squares, respectively quantile process. Earlier, Wei and Huang
(2010) had considered the adaptive group LASSO estimator but for gaussian model error. Wang et al.
(2015) study also the convergence and the sparsity of the (non adaptive) group LASSO estimator in
a high-dimensional generalized linear model. When the number of groups is fixed, Wang and Leng

2

Adaptive Fused LASSO in Grouped Quantile Regression

(2008) show the model consistency obtained by an adaptive group LASSO method. For a review,
but until 2012, of group selection methods and several applications of these methods the reader can
see Huang et al. (2012).
For a linear model, without grouped variables, in order to encourage sparsity of the parameters
but also the sparsity of their differences, to identify predictive variable clusters, Tibshirani et al.
(2005) introduced an additional penalty to the LASSO penalty, taking the L1 norm of the differences
between two successive parameters. They called the obtained estimator, fused LASSO estimator.
Applications of the fused LASSO method can be found in Jang et al. (2015) or in Li and Zhu
(2007). The fused LASSO idea was adopted by Jiang et al. (2013), Jiang et al. (2014), Zhao et al.
(2014) to quantile model. A very recent paper of Viallon et al. (2016), considers a generalized linear
models, estimated by adaptive fused LASSO method. Adaptive fused LASSO penalty is also used
by Sun et al. (2016) for estimation of the spatial and temporal quantile functions. These papers
have in common that the models have fixed number of explanatory variables.
The fused LASSO penalty for grouped variables, proposed and studied in this paper, will be
helpful to strengthen the sparsity between two successive groups of variables. To the knowledge of
the author, the adaptive fused LASSO method wasn’t considered for a linear model and further,
with the possibility that the number of groups converges to infinity when number of observations
diverges. Even for quantile linear models without grouped variables, there is no work in literature on the adaptive fused LASSO method. This is the originality of the present paper. Emphasize
that the proposed estimator and obtained results are valid for a wide spectrum of error distributions.

The paper is organized as follows. In Section 2, we present the quantile model with grouped
variables, we also introduce the adaptive fused estimator and we give general notations and assumptions. In Section 3, we study the convergence rate, oracle properties of the estimator when
the group number is fixed. A general convergence rate when the group number diverges is found in
Section 4. In the same section, we state that the oracle properties remain true. All proofs will be
postponed in Section 5.

3

Adaptive Fused LASSO in Grouped Quantile Regression

2 Model, notations and assumptions
In this section, we first introduce the quantile model with grouped variables. Afterwards, some
notations used throughout in the paper are given, followed by the introduction of the proposed
fused estimator. Finally, general assumptions on the model errors, design and on the group number
are given.
Let us consider the following linear model with p groups of variables

Yi =

p
X

Xtij βj + εi = Xti β + εi ,

j=1

i = 1, · · · , n.

(2.1)

The random variables in model (2.1) are: Yi the response variable and εi the model error. The
column vector Xi is the ith observation of the explanatory variables and its contains p groups of
variables. For each group j, with j = 1, · · · , p, the vector of the parameters is βj ≡ (βj1 , · · · , βjdj ) ∈
Rdj and the design Xij for observation i, is a vector of size dj ×1. The vector with the all coefficients
0
0
is β ≡ (β 1 , · · · , β p ) and β0j = (βj1
, · · · , βjd
) the true (unknown) value of the parameter βj , for
j

j = 1, · · · , p. For observation i, we denote by Xij,k the kth variable of the jth group. We will
assume that dj = d for any j = 1, · · · , p by taking d = maxj=1,··· ,p dj , filling the components of β j
between dj and d with 0 and the values of Xi also with 0.
For model (2.1) it is possible that there are insignificant variables groups. For this, we will consider
the index set of the significant groups:
A ≡ {j ∈ {1, · · · p}; kβ0j k 6= 0}
and obviously the index set of the insignificant groups Ac ≡ {j; kβ0j k = 0}. We denoted by |A|
the cardinal of the index set A. Obviously, in practical applications, the two sets A and Ac are
unknown.
On the other hand, we denote by |A| = p0 , r0 = dp0 and r = dp. The numbers p and d are known,
p0 is contrariwise unknown and then r0 also.
For a r-vector of parameters β, we denote βA the subvector of β, of dimension r0 × 1, which
4

Adaptive Fused LASSO in Grouped Quantile Regression

contains βj , for j ∈ A. The (r − r0 )-vector β Ac contains βj for j ∈ Ac .
We introduce now the quantile model. This method allows the non necessity of the classical
assumptions on errors: IE[εi ] = 0 and V ar(εi ) < ∞. Since these assumptions are not often satisfied
in practical applications, the quantile method can be used extensively in many different areas.
So, for a quantile index τ ∈ (0, 1), the check function ρτ (.) : R → R+ is defined by ρτ (u) =
u(τ − 11u<0 ). In this paper, the index τ is considered fixed.
Before defining the adapted fused LASSO estimator for the parameter β of (2.1), we give some
general notations. All throughout the paper, C denotes a positive generic constant not dependent
on n, which may take different values in different formula or even in different parts of the same
formula. The value of C is not of interest. All vectors and matrices are denoted by bold symbols
and all vectors are written as column vectors. For a vector v, we denote by vt its transposed and
L

P

by kvk its Euclidean norm. Notations −→ , −→ represent the convergence in distribution and
n→∞

n→∞

in probability, respectively, as n → ∞. For a positive definite matrix M, we denote by λmin (M)
and λmax (M) its the smallest and largest eigenvalues, respectively. When it is not specified, the
convergence is for n → ∞.
For model (2.1), n observations of (Yi , Xi )16i6n are available. In order to define the estimator
that will allow automatic selection of significant groups of variables, we must first consider the
quantile process:

Gn (β) ≡

n
X
i=1

ρτ (Yi − Xti β).

The quantile estimator for β is the minimizer of the quantile process:
e ≡ arg min Gn (β).
β
n

(2.2)

β∈Rr

5

Adaptive Fused LASSO in Grouped Quantile Regression

e = (β
e ,β
e ,··· ,β
e ), with
This estimator can be written taking into account each group β
n
n;1
n;2
n;p
e
e
β
n;j a vector of size d, for j = 1, · · · , p. We will use β n for constructing the two adaptive LASSO

penalties. Note that, the number r of the total variables needs to be smaller than the sample size
n. For model (2.1), we define the adaptive fused group LASSO quantile (afg LASSO Q ) estimator,
b , as the minimizer of the following process:
denoted by β
n
Qn (β) ≡ Gn (β) + µ(1)
n

p
X
j=1

(1)

ω
bn;j kβj k + µ(2)
n

p
X
j=2

(2)

ω
bn;j kβj − β j−1 k,

(2.3)

(2)

(1)

−γ
e −β
e
e k−γ , ω
and γ > 0 a fixed known parameter.
bn;j ≡ kβ
with the weights ω
bn;j ≡ kβ
n;j
n;j−1 k
n;j

b is written β
b = (β
b ,··· ,β
b ) and β
b
The estimator β
n
n
n;1
n;p
n;j is a vector of size d, for j = 1, · · · , p.
(1)

(2)

The tuning parameters µn , µn are assumed to converge to infinity as n → ∞. Additional condi(1)

(2)

tions on µn , µn , taking into account γ and the group number p, will be given later.

b , mainly the oracle
The purpose of this paper is to study the properties of the estimator β
n

properties: the significant groups of variables are estimated, with an optimal estimation rate,
by asymptotically gaussian estimators and the corresponding parameters to nonsignificant groups
are shrunk directly to 0 with a probability converging to one. In order to study the asymptotic
b , for some r-vector β ∈ Rr , we also consider the
properties of the afg LASSO Q estimator β
n

process:

Ln (β) ≡ Qn (β) − Qn (β 0 ).

(2.4)

Let us note that, for τ = 1/2, model (2.1) becomes median regression with grouped variables. The
b becomes in this case, adapted fused grouped LASSO median estimator.
estimator β
n
b will be studied under the following assumptions for errors,
The asymptotic properties for β
n

design and group number p:

(A1) (εi )1≤i≤n are i.i.d., with the distribution function F and density function f . The density

6

Adaptive Fused LASSO in Grouped Quantile Regression

function f is continuously, strictly positive in a neighborhood of zero and has a bounded first
derivative in the neighborhood of 0. The τ th quantile of εi is zero: τ = F (0).
(A2) There exist constants 0 < m0 ≤ M0 < ∞ such that
m0 ≤ λmin (n−1
1/2

(A3) (p/n)

n
X
i=1

Xi Xti ) ≤ λmax (n−1

n
X
i=1

Xi Xti ) ≤ M0 .

max16i6n kXi k → 0.

(A4) p is such that p = O(nc ), with 0 ≤ c < 1.
For the smallest nonzero vector norm and on constant c of assumption (A4) we assume:
(A5) Let us denote h0 ≡ min16j6p0 kβ 0j k. There exists a constant M > 0 such that M ≤ n−α h0
and α > (c − 1)/2.
Concerning the size of the nonzero parameter vectors, we take the following assumption:
(A6) r0 = O(p0 ).

Assumptions (A2), (A3) are standard for LASSO methods and (A1) is classic for quantile
regression (see Ciuperca (2016b), Koenker (2005), Zou and Yuan (2008), Wu and Liu (2009)).
Assumptions (A3), (A4) are also considered in Ciuperca (2016a), Zou and Zhang (2009) for highdimensional linear model, while (A5) and (A6) are required for adaptive group LASSO least square
estimator in Zhang and Xiang (2015) and in Ciuperca (2016a) for adaptive group LASSO quantile
estimator.
For the case p fixed, then c = 0, only assumptions (A1) - (A3) will be needed. For the case
p = pn → ∞ as n → ∞, assumptions (A4), (A5) and (A6) are also considered, with c ∈ (0, 1) in
assumption (A3).

In Sections 3 and 4, we will study the adaptive fused group LASSO quantile (afg LASSO Q )
b for two cases of the group number: p fixed and p → ∞ as n → ∞, respectively.
estimator β
n

7

Adaptive Fused LASSO in Grouped Quantile Regression

3 Case c = 0
In this section we will propose and study the asymptotic properties of the afg LASSO Q estimator of the parameter β for model (2.1), when the number of groups p is fixed.

b , only
Regarding assumptions, as specified above, in order to prove the oracle properties for β
n

(A1), (A2), (A3) will be needed, with a weaker condition in (A1) on error density f . So, the
condition that f has a bounded derivative in the neighbourhood of 0 with an weaker condition can
be replaced in assumption (A1) by: for every e ∈ int(B), 1r ∈ Rr , we have
lim n−1

n→∞

n Z
X
i=1

xti 1r

0

√
1
n[F (e + n−1/2 v) − F (e)]dv = f (e)1tr Υ1r .
2

(3.1)

The r-vector 1r contains as elements 1. The matrix Υ is defined by (3.2).
Note also that assumption (A3) becomes: n−1 max1≤i≤n Xti Xi −→ 0 and assumption (A2) implies
n→∞

that
n−1

n
X
i=1

Xi Xti −→ Υ,

(3.2)

n→∞

with Υ a positive definite matrix.

(1)

(2)

The tuning parameters µn , µn and the positive constant γ are such that, for n → ∞,
→ ∞,
µ(m)
n

n−1/2 µ(m)
→ 0,
n

n(γ−1)/2 µ(m)
→ ∞, for m = 1, 2.
n

(3.3)

For m = 1, we get the conditions imposed on the tuning parameter by Ciuperca (2016a) for
(1)

(2)

adaptive group LASSO quantile estimator (non fused). Conditions in (3.3) on µn and µn are
also found in Viallon et al. (2016), where an adaptive fused LASSO for generalized linear models is
considered. For the particular case γ = 1, for a quantile model without grouped variables (d = 1),
we obtain the conditions on the tuning parameters imposed by Jiang et al. (2014).

8

Adaptive Fused LASSO in Grouped Quantile Regression

The proofs of all results are given in Section 5, sub-section 5.1.
By the following lemma we show that, when the variables are grouped, the adapted fused group
LASSO quantile parameter estimator has the same convergence rate as by classical quantile method,
without grouping variables, without adapted fused LASSO penalty. This convergence rate will serve
as an essential tool for studying process Ln (β) when β belongs to a neighbourhood of β 0 of order
radius n−1/2 and for showing the asymptotic normality of the parameter estimators corresponding
to the significant groups.
Lemma 3.1 Under assumptions (A1), (A2), (A3) and conditions in (3.3) for the tuning parameb − β 0 k = OP (1).
ters, we have, n1/2 kβ
n

b , let us consider the index set of the
In order to study the oracle properties of the estimator β
n

groups selected by the following adaptive fused group LASSO quantile method:
b k 6= 0}
Abn ≡ {j ∈ {1, · · · , p}; kβ
n;j

and Abcn its complementary.

The following theorem shows a first oracle property, that the afg LASSO Q estimators with
indices in the set A are asymptotically Gaussians.
√ b
Theorem 3.1 Under assumptions (A1), (A2) and (A3) and conditions of (3.3), we have n(β
n−

L
−1
, with ΥA the submatrix of Υ with the row and column
β 0 )A −→ N 0r0 , τ (1 − τ )f −2 (0)ΥA
n→∞

indices in A.

Compared to the adaptive LASSO quantile method, if an additional penalty fused is considered,
we got the same variance matrix for the asymptotic gaussian law (see Ciuperca (2016a)).
In practical applications, the set A is unknown. In exchange, it can be estimated by Abn . Then, for

b to be interesting, it is necessary that these two sets coincide with probability
that the estimator β
n
9

Adaptive Fused LASSO in Grouped Quantile Regression

converging to 1, as n converges to infinity. By the following theorem, we show that the second
oracle property, i.e. the sparsity, is satisfied for the afg LASSO Q estimator.
Theorem 3.2 Under the same assumptions as in Theorem 3.1, we have, limn→∞ P[Abn = A] = 1.
The proof of Theorem 3.2, given in sub-section 5.1, is in two parts. The result limn→∞ P[A ⊆

Abn ] = 1 is an immediate consequence of Theorem 3.1. In order to prove limn→∞ P[Card(Ac ∩ Abn ) ≥

b and imposed
1] = 0, the proof is quite technical, taking into account already proven properties to β
n
conditions for the tuning parameters.

4 Case c > 0
In this section we consider same model (2.1) with grouped variables, but with the number p of
groups depending on n and divergent: p = pn and pn → ∞ as n → ∞. For readability we keep
notation p instead of pn . Similarly for r = pd. The main purpose is to show that the afg LASSO Q
estimator keeps the oracle properties even though the group number diverges. The proofs of all
results are given in Section 5, sub-section 5.2. A major difficulty that appears in the proofs is that
the size of vectors and of matrices converges to infinity when n tends to infinity.

In order to show the main result of this Section, we will first find the convergence rate of
c of β. Afterwards, we will show that this
adaptive fused group LASSO quantile estimator β
n
(1)

(2)

estimator satisfies the oracle properties. We recall that the two tuning parameters µn and µn
converge to infinity as n → ∞.
(m)

Lemma 4.1 Under assumptions (A1)-(A5) and the two tuning parameters (µn )n∈N satisfying

(m)
b − β 0 k = OP (pn−1 )1/2 .
µn n(c−1)/2−αγ → 0, as n → ∞, for m = 1, 2, we have kβ
n

We observe that for fixed p, we obtain the result of Lemma 3.1. The convergence rate as p → ∞

of the afg LASSO Q estimator is the same as that of Ciuperca (2016a) for adaptive group LASSO
quantile estimator. Then, the fused penalty doesn’t affect the estimator rate convergence. For the
(m)

particular case c = α = 0, the condition imposed on µn , for m = 1, 2, in Lemma 4.1, is the second
10

Adaptive Fused LASSO in Grouped Quantile Regression

condition of (3.3).

In order to prove the sparsity property, the assumptions used in Lemma 4.1 are sufficient.
Since p → ∞, we need in addition assumption (A6) for showing the asymptotic normality of the
afg Q LASSO estimators for the significant groups of variables. For the tuning parameters, we
consider a generalization for the third condition of (3.3).
Theorem 4.1 Suppose that assumptions (A1)-(A5) are satisfied and also that the tuning parame
(m) (c−1)/2−αγ
(m)
−c(1+γ)+γ−1 /2
ters satisfy µn n
→ 0, µn n
→ ∞, as n → ∞, for m = 1, 2. Then:
h
i
(i) P Abn = A → 1, as n → ∞.

(ii) If moreover assumption (A6) holds, for any vector u of size r0 such that kuk = 1, if we denote

P
√
L
−1/2 t b
Υn,A ≡ n−1 ni=1 Xi,A Xti,A , then, n(ut Υ−1
u (βn − β 0 )A −→ N 0, τ (1 − τ )f −2 (0) .
n,A u)
n→∞

We observe that, in respect to the case p fixed, now we first prove the sparsity property. For


b k > 0 = 1. For showing
showing P[A ⊆ Abn ] → 1, we prove that: limn→∞ P minj∈A kβ
n;j
limn→∞ P[Card(Ac ∩ Abn ) ≥ 1] = 0, we use the asymptotic properties of quantile process and

imposed conditions for the tuning parameters. In order to proof the asymptotic normality of
b )A , we mainly use the sparsity property and we prove that for the penalized process Ln (β),
(β
n

with β in a n−1/2 -neighbourhood of β 0 , the penalties are much smaller than the quantile process.
Finally, a CLT for the independent random variable sequences is applied.
Remark 4.1 Results of Lemma 4.1 and of Theorem 4.1 are new even for the particular case of
quantile model without grouped variables.
Remark 4.2 Algorithm and the related numerical part are a very difficult task, firstly since in the
process Gn (β) and in the two penalties of (2.3), the variables are grouped. On the other hand,
quantile process and penalties are continuous but not differentiable in respect to parameters β. The
author has not found any numerical work, even for the particular case d = 1, of ungrouped variables,
for a linear quantile model, with adaptive fused LASSO penalty. Consequently, for the method

11

Adaptive Fused LASSO in Grouped Quantile Regression

proposed in the present paper, another work should be conducted on numerical method, firstly for a
quantile model without grouped variables and afterwards for quantile model with grouped variables.

5 Proofs
In this section, the proofs of Lemmas and of Theorems presented is Sections 3 and 4 are presented.
b , we consider the
In order to study the asymptotic properties of the afg LASSO Q estimator β
n
following random variable

Di ≡ (1 − τ )11εi <0 − τ 11εi ≥0 .

(5.1)

Obviously, IE[Di ] = 0 and ρτ (εi ) = −εi Di .
5.1 Result proofs for c=0 case
We start be giving the proofs of results presented in Section 3.

Proof of Lemma 3.1. We show that for all ǫ > 0, there exists a constant Bǫ > 0 (without loss of
generality, we take Bǫ > 0, otherwise we take |Bǫ |) sufficiently large such that for n large enough:
P






inf Ln β0 + Bǫ n−1/2 u > 0 ≥ 1 − ǫ,

kuk=1

(5.2)

with u ∈ Rr , kuk = 1.


Let C1 > 0 be some constant. We will study the random process: Ln β0 + C1 n−1/2 u = Gn (β 0 +
 0
 (2) Pp
(1) Pp
(2)
−γ
e
C1 n−1/2 u)−Gn (β 0 )+µn
kβ j +n−1/2 C1 uj k−kβ0j k +µn
bn;j kβ0j −β0j−1 +
j=1 kβ n;j k
j=2 ω

C1 n−1/2 (uj − uj−1 )k −kβ0j − β 0j−1 k .

For each observation i, consider the random variable Ri ≡ ρτ (εi − C1 n−1/2 Xti u) − C1 n−1/2 Di Xti u,
Pn
with Di defined by (5.1). Consider also the following random vector Wn ≡ C1 n−1/2 i=1 Di Xti .

12

Adaptive Fused LASSO in Grouped Quantile Regression

Then the loss term of the random process Ln β 0 + C1 n−1/2 u can be written:

n
h
i
X
Gn (β 0 +C1 n−1/2 u)−Gn (β 0 ) = IE Gn (β 0 + C1 n−1/2 u) − Gn (β 0 ) +Wn u+ (Ri −IE[Ri ]). (5.3)
i=1

For the first term of the right-hand side of (5.3) we have:
 

 X
n Z C1 n−1/2 Xti u
u
IE Gn β 0 + C1 √
− Gn (β 0 ) =
[F (t) − F (0)]dt.
n
i=1 0
Since kuk = 1, by assumption (A3), we have, n−1/2 Xti u = o(1). Using assumption (A1) together
with relation (3.1), by the mean value theorem, we obtain:



n
f (0) 1 X t 2
u
0
0
− Gn (β ) = C12
IE Gn β + C1 √
(X u) (1 + o(1)).
2 n i=1 i
n


(5.4)

For the third term of the right-hand side of (5.3), since the errors εi are i.i.d., we have,

IE

"

n
X
i=1

#2

(Ri − IE[Ri ])

≤

n
X
i=1

≤

IE[R2i ] ≤

C12 n−1

n
X
i=1

n
X

IE

i=1



|Xti u|2 IE

C1 n−1/2 |Xti u|11|εi |<C1 n−1/2 |Xti u|

i
h
11|εi |<C1 n−1/2 |Xti u| .

2 

(5.5)

But, using assumption (A3),
h
i
IE 11|εi |<C1 n−1/2 |Xti u| ≤ Cn−1/2 kXi k ≤ Cn−1/2 max kXi k = o(1).

(5.6)

16i6n

Using assumption (A2), relations (5.5) and (5.6) imply: IE [
Bienaymé-Tchebychev inequality, we have
n
X
(Ri − IE[Ri ]) = oP (1).

Pn

i=1 (Ri

2

− IE[Ri ])] ≤ o(1). Then, by

(5.7)

i=1

For the second term of the right-hand side of (5.3) we have that random variable Wn u converges
in distribution to a centred Gaussian law. Then, taking also into account relations (5.4) and (5.7),
13

Adaptive Fused LASSO in Grouped Quantile Regression

we obtain that relation (5.3) becomes:

0

Gn (β + C1 n

−1/2

0

u) − Gn (β ) =

n

1X t 2
(X u)
2 n i=1 i

f (0)
C12

!

(1 + oP (1)).

(5.8)


Now we study the penalty terms for Ln β 0 + C1 n−1/2 u .
(1)

• For the penalty µn
the index j:

Pp

j=1



e k−γ kβ0 + n−1/2 C1 uj k − kβ0 k , two cases are considered for
kβ
n;j
j
j

e
– if j ∈ A, then, since the quantile estimator β
n;j is consistent, we have with proba

(1)
e k−γ kβ0 + n−1/2 C1 uj k − kβ0 k <
bility converging to 1 as n → ∞, that µn kβ
n;j
j
j
(1)

Cµn n−1/2 kuj k → 0, by conditions of (3.3).

−1/2
e
– if j ∈ Ac , then, taking into account that the convergence rate of β
, this
n;j to 0 is n




(1)
(1) e
penalty is OP µn kβn;j k−γ n−1/2 kuj k = OP µn n(γ−1)/2 kuj k , which converges to

∞ when kuj k 6= 0 by (3.3) and it is equal to 0 when kuj k = 0.

(2) Pp
(2)
• We will now study the penalty µn
bn;j kβ0j − β0j−1 + C1 n−1/2 (uj − uj−1 )k −kβ0j −
j=2 ω

β0j−1 k . We consider the two possible cases for the index j:
(2)

(2)

– if β0j = β 0j−1 , then we have, µn ω
bn;j kβ0j − β 0j−1 + C1 n−1/2 (uj − uj−1 )k − kβ0j − β0j−1 k

(2) (2) −1/2
kuj − uj−1 k > 0.
bn;j n
= OP µn ω
(2)



(2)

– if β0j 6= β 0j−1 , then, using conditions (3.3), we obtain, µn ω
bn;j kβ0j −β0j−1 +C1 n−1/2 (uj −


(2) (2)
uj−1 )k − kβ0j − β0j−1 k = OP µn ω
bn;j (uj − uj−1 )t n−1/2 (β 0j − β 0j−1 )kβ0j − β0j−1 k−1 =

(2)
OP n−1/2 µn (β 0j − β 0j−1 ) = oP (1).
(m)

Then, since in the following relation OP (µn n(γ−1)/2 ) > 0, for any m = 1, 2, and taking into

account relation (5.8) together with the study realised on the penalties, we have for n and Bǫ large

14

Adaptive Fused LASSO in Grouped Quantile Regression

enough that:

n


X
0
−1/2
2 f (0)
(γ−1)/2
−1/2
t
2
u = Bǫ
L n β + Bǫ n
) + OP (µ(1)
)
(X u) + Bǫ OP (µ(1)
n n
n n
2n i=1 i

−1/2
(γ−1)/2
+OP (µ(2)
) + OP (µ(2)
) .
n n
n n
Taking into account (3.3), we obtain relation (5.2) for n and Bǫ large enough.



Proof of Theorem 3.1. For u ∈ Rr , let us consider the random process: Ln (β 0 + n−1/2 u), with
the process Ln defined by relation (2.4) and u ∈ Rr .
√ b
0
0
−1/2
b n = n(β
u). In view of the
Let’s recall that u
n − β ) is the minimizer in u de Ln (β + n

b obtained by Lemma 3.1, we will consider u ≡ (u1 , · · · , up )
convergence rate of the estimator β
n

bounded. On the other hand, the process Ln (β 0 + n−1/2 u) can be written:

h
i √n
(1)
ω
bn;j kβ0j + n−1/2 uj k − kβ0j k √
n
j=1




√
p
X (2)
uj
uj−1
n
k − kβ0j − β0j−1 k √ ,
ω
bn;j kβ0j + √ − β 0j−1 + √
+µ(2)
n
n
n
n
j=2

Ln (β 0 + n−1/2 u) = [ztn u + Bn (u)] + µ(1)
n

p
X

(5.9)

with
n

1 X
zn ≡ √
Xi Di ,
n i=1

Bn (u) ≡

n Z
X
i=1

√
Xti u/ n

[11εi <t − 11εi <0 ]dt,

0

and the random variable Di defined by (5.1). Since IE[Di ] = 0, we have that IE[zn ] = 0r . For the
loss term (the first bracket of the right-hand side) of (5.9), by the CLT, using (A1), (A2) and (A3),
we have
L

ztn u −→ zt u,
n→∞

P

Bn (u) −→

n→∞

1
f (0)ut Υu,
2

with the random r-vector z ∼ N (0r , τ (1 − τ )Υ).
We now study the two penalties of the right-hand side of (5.9).

15

(5.10)

Adaptive Fused LASSO in Grouped Quantile Regression

For the first penalty term of Ln (β 0 + n−1/2 u) of (5.9), we have, using the conditions of relation
(3.3), that,

µn

p
X
j=1

ω
bn;j



β 0j

√
p
uj
n P X (1) 0
0
W (β j , u),
− kβj k √ −→
+√
n
n n→∞ j=1

(5.11)

with



0, if



W (1) (β 0j ; uj ) ≡
0, if




 ∞, if

β 0j 6= 0d
β 0j = 0d and uj = 0d
β 0j = 0d and uj 6= 0d .

For the second penalty term of Ln (β 0 + n−1/2 u) of (5.9), consider the following notations:

√
(2) (2) −1/2
P2,j .
bn;j n
P2,j ≡ n kβ0j − β 0j−1 + n−1/2 (uj − uj−1 )k − kβ0j − β 0j−1 k and S2,j ≡ µn ω

For β 0j , β0j−1 two cases are possibles.

−1/2
e −β
e
If β 0j = β0j−1 , then P2,j = kuj −uj−1 k ≤ C. On the other hand, since kβ
),
n,j
n,j−1 k = OP (n
(2)

(2)

(2)

P

−γ −1/2
e −β
e
n
−→ ∞. Then
using conditions (3.3), we have µn ω
bn;j n−1/2 = Cµn kβ
n,j
n,j−1 k
n→∞

P

S2,j −→

n→∞



 0,


 ∞,

if

uj = uj−1

if

uj 6= uj−1 .
P

If β 0j 6= β0j−1 , since limn→∞ P2,j = (uj −uj−1 )t (β 0j −β 0j−1 )kβ0j −β0j−1 k−1 , we have that, S2,j −→ 0.
n→∞

So, considering both cases, we can write
p
X
j=1

P

S2,j −→ W (2) (β 0j , β 0j−1 ; uj , uj−1 ),

(5.12)

n→∞

with



0, if



0
0
W (2) (β j , βj−1 ; uj , uj−1 ) ≡
0, if




 ∞, if

β0j 6= β 0j−1
β0j = β 0j−1 and uj = uj−1
β0j = β 0j−1 and uj 6= uj−1 .
16

Adaptive Fused LASSO in Grouped Quantile Regression

Thus, for process (5.9), taking into account of relations (5.10), (5.11) and (5.12), we obtain,
L

Ln (β 0 + n−1/2 u) −→ L(u),
n→∞

with, the limit random variable,
p
p
X
X
1
W (2) (β 0j , β 0j−1 ; uj , uj−1 ).
W (1) (β0j ; uj ) +
L(u) ≡ zt u + f (0)ut Υu +
2
j=2
j=1

b n = arg minu Ln (u). On the other hand, L(u) is bounded for any j such that β0j 6= 0d ,
But u

when β0j 6= β 0j−1 or when (β 0j = β 0j−1 , uj = uj−1 ). In these cases, the expression of L(u) is:
L(u) ≡ zt u + 2−1 f (0)ut Υu. Since the minimizer of L(u) is the gaussian vector f −1 (0)z, we deduct
b n is asymptotically Normal and the theorem follows.
that u
Proof of Theorem 3.2. By Theorem 3.1, for any j ∈ A we have that


√ b
n(β

n;j

L

− β0j ) −→

n→∞

N (0d , τ (1 − τ )f −2 (0)ΥAj ), with ΥAj a square matrix of size d × d, the submatrix of Υ. Since
β 0j 6= 0d , then j ∈ Abn . Thus
lim P[A ⊆ Abn ] = 1.

(5.13)

n→∞

To finish the proof, we show that, limn→∞ P[Card(Ac ∩ Abn ) ≥ 1] = 0. We assume without loss of

generality that Ac ∩ Abn = {j1 }. If this intersection contains more than one element, the calculations

are the same, except they are painful.

b which has the j1 th group such that Ac ∩ Abn = {j1 }, let us consider
In addition to the estimator β
n
b ) > Qn (β ∗ ) with
b , 0Ac ), we will show that Qn (β
a second estimator β∗ for β. Taking β∗ ≡ (β
n
n;A

a probability converging to 1.

17

Adaptive Fused LASSO in Grouped Quantile Regression

For this, we will study the following difference:
b )−Qn (β ∗ ) =
Qn (β
n

n 
X
i=1

 X

n 
(1) b
t ∗
−
ρτ (Yi −Xi β )−ρτ (εi ) +µ(1)
bn;j1 kβ
n;j1 k
n ω

b )−ρτ (εi )
ρτ (Yi −Xti β
n

i=1



(2)
(2) b
(2)
(2) b
b
b
b
b
k
β
k
.
k
β
k
−
ω
b
k
−
ω
b
k
β
−
β
−
β
k
+
ω
b
k
β
+µ(2)
ω
b
n;j1 +1
n;j1 −1
n;j1 +1
n;j1
n;j1 −1
n;j1
n
n;j1 +1
n;j1
n;j1 +1
n;j1
Pn



b )
Xti β
n


− ρτ (εi ) and

Similarly as in the proof of Theorem 3.1, we have that,
i=1 ρτ (Yi −


Pn
t ∗
i=1 ρτ (Yi − Xi β ) − ρτ (εi ) are bounded, with a probability converging to 1. Then, with a
probability converging to 1, we have that,



(2)
(1) b
(2)
b
b
b
b ) − Qn (β ∗ ) = C + µ(1) ω
ω
bn;j1 kβ
k
+
µ
k
β
Qn (β
b
n;j1 − β n;j1 −1 k − kβ n;j1 −1 k
n
n;j1
n
n
n;j1

(5.14)

(2)
b
b
b
+b
ωn;j1 +1 kβ
k
−
k
β
k
−
β
.
n;j1 +1
n;j1 +1
n;j1
P
P
(1)
e
Since j1 ∈ Ac , we have that β
bn;j1 −→ ∞. For the first term of the
n;j1 −→ 0 and then ω
n→∞

n→∞

(1)
(1) (1) b
γ/2
b
penalty of (5.14), by conditions (3.3), we have: µn ω
bn;j1 kβn;j1 k = µn n−1/2 kn1/2 β
n;j1 kn


P
(1)
−γ
e
= OP µn n(γ−1)/2 −→ ∞.
·kn1/2 β
n;j1 k
n→∞

(1)

(2)

We will study now the two penalties, in µn and in µn of (5.14). For β0j1 −1 , β 0j1 , β 0j1 +1 three
cases are possibles.
Case 1. β 0j1 6= β0j1 −1 and β 0j1 +1 6= β0j1 .

(2)

(2)

bn;j1 +1 converge in probability for n → ∞ to a
In this case, we have that, the weights ω
bn;j1 and ω

strictly positive bounded constant. On the other hand, we have the following obvious inequalities,
b
b
b
b
b
b
b
with probability 1: kβ
n;j1 − β n;j1 −1 k − kβ n;j1 −1 k ≥ −kβn;j1 k and kβ n;j1 +1 − β n;j1 k − kβ n;j1 +1 k ≥

b
−kβ
n;j1 k. Then, with probability 1, we have,

i
 h (1) (1)
(1) b
(2) (2)
(2) (2)
b
b
b
b
−
µ
ω
b
−
β
k
−
k
β
k
≥
µ
ω
b
k
β
k
+
µ
ω
b
k
β
µ(1)
ω
b
n
n;j1 −1
n;j1 −1
n
n;j1
n
n;j1
n
n;j1 kβ n;j1 k
n;j1
n;j1
n;j1

(1)

1/2 e
b
β n;j1 k−γ are bounded with probability converging to 1, since n(γ−1)/2 µn
and since kn1/2 β
n;j1 k, kn

18

Adaptive Fused LASSO in Grouped Quantile Regression

→ ∞, we have that the above relation is


(2) 
(1)
−1/2 (1) (1)
b
µn ω
bn;j1
bn;j1 kn1/2 β
= n−1/2 µ(1)
bn;j1 − µ(2)
n;j1 k = OP n
n ω
n ω
−γ
γ/2
e
kn1/2 β
= OP n−1/2 µ(1)
n;j1 k
n n



P

−→ ∞.

n→∞

For the following term of the penalty, we have:

(2)
(2) (2)
b
b
b
b
bn;j1 +1 kβ
µ(2)
bn;j1 +1 kβ
n;j1 k
n;j1 +1 − β n;j1 k − kβ n;j1 +1 k ≥ −µn ω
n ω


1/2 b
β n;j1 k = oP (1).
= OP − Cn−1/2 µ(2)
n kn

(2)

We have used the fact that ω
bn;j1 +1 converges in probability, for n → ∞, to a strictly positive
(2)

b
bounded constant, n−1/2 µn → 0 and kn1/2 β
n;j1 k is bounded, with probability converging to 1.
P

(1)

−γ
e
−→ ∞, we have with a probability
Thus, for relation (5.14), since µn n(γ−1)/2 kn1/2 β
n;j1 k
n→∞

converging to 1, that

−γ
e
b ) − Qn (β ∗ ) > C + µ(1) n(γ−1)/2 kn1/2 β
+ oP (1) > 0.
Qn (β
n;j1 k
n
n

(5.15)

Case 2. β 0j1 6= β0j1 −1 and β 0j1 +1 = β0j1 (or vice versa, the calculations are the same).
Since β0j1 = 0, then β0j1 +1 = 0 and β 0j1 −1 6= 0.

(2)
b
cn = {j1 }, then j1 + 1 6∈ A
cn and thus β
Since Ac ∩ A
bn;j1 converges
n;j1 +1 = 0. Therefore, the weight ω
(2)

P

for n → ∞ to a strictly positive bounded constant and ω
bn;j1 +1 −→ ∞. Then
n→∞



 (2)
(2)
(1) b
(2)
b
b
b
b
bn;j1 +1 kβ
bn;j1 kβ
µ(1)
bn;j1 kβ
n;j1 k
n;j1 − β n;j1 −1 k − kβ n;j1 −1 k + ω
n;j1 k + µn ω
n ω
≥

(1) b
µ(1)
bn;j1 kβ
n;j1 k
n ω

+

µ(2)
n



−

(2) b
ω
bn;j1 kβ
n;j1 k

+

(2)
b
ω
bn;j1 +1 kβ
n;j1 k



P
(1) b
(2) (2)
b
bn;j1 +1 kβ
≥ µ(1)
bn;j1 kβ
n;j1 k(1 + oP (1)) −→ ∞,
n;j1 k + µn ω
n ω
n→∞

19

Adaptive Fused LASSO in Grouped Quantile Regression

and thus, for all ǫ > 0,
h
i
b ) − Qn (β ∗ ) > ǫ = 1.
lim P Qn (β
n

(5.16)

n→∞

Case 3. β 0j1 = β0j1 −1 = β0j1 +1 .

b
b
b
In this case, we have β
n;j1 6= 0, β n;j1 −1 = β n;j1 +1 = 0. The penalty in this case is:


P
(2)
(2)
(1) b
(2)
b
+
ω
b
k
+
µ
ω
b
k
β
µ(1)
ω
b
n
n;j1
n
n;j1 +1 kβ n;j1 k −→ ∞,
n;j1
n;j1
n→∞

which means for relation (5.14), that for all ǫ > 0,
h
i
b ) − Qn (β ∗ ) > ǫ = 1.
lim P Qn (β
n

(5.17)

n→∞

Thus, in all three cases, taking into account (5.14), (5.15), (5.16), (5.17), we get:
h
i
b ) > Qn (β ∗ ) = 1,
lim P Qn (β
n

n→∞

b is not the minimizer of Qn . Therefore, limn→∞ P[j1 ∈ Ac ∩ Abn ] = 0 and the
which implies that, β
n
proof is finished.



5.2 Result proofs for c > 0 case
We present now the proofs of the results stated in Section 4. For some results, we will consider
obtained results in Ciuperca (2016a) for asymptotic behaviour of the non fused adaptive group
LASSO quantile estimator.

Proof of Lemma 4.1. Let be a positive constant C1 and a r-vector u such that kuk = 1. For the

20

Adaptive Fused LASSO in Grouped Quantile Regression

process Ln defined by (2.4), let us consider
r 
r 
r




p
X
p
p
p
0
0
0
(1) (1)
0
0
Ln β + C1
µn ω
bn;j kβj +
u = Gn β + C1
u − Gn (β ) +
C1 uj k − kβj k
n
n
n
j=1
+µ(2)
n

p
X
j=2



(2)
ω
bn;j

kβ0j

+

r

r



p
p
0
0
0
C1 uj − β j−1 +
C1 uj−1 k − kβj − βj−1 k ≡ T1 +T2 +T3 . (5.18)
n
n

(1)

For T1 , T2 , using µn n(c−1)/2−αγ → 0, proceeding as in the proof of Theorem 3 of Ciuperca (2016a),
we get:

T1 + T2 >

C12 f (0)p

!
n
1X t
t
u Xi Xi u (1 + oP (1)) − C1 OP (p) .
n i=1

(5.19)

For T3 , we have:
T3 ≥ µ(2)
n

≥ −C1

X

{j;β 0j 6=β 0j−1 }

r

p (2)
µ
n n

r
r




p
p
(2)
C1 uj − β0j−1 +
C1 uj−1 k − kβ0j − β0j−1 k
ω
bn;j kβ0j +
n
n
X

{j;β 0j 6=β 0j−1 }

(2)

ω
bn;j kuj − uj−1 k ≥ −C1 OP (p)

(5.20)

(2)

by similar reasoning to that T1 , using also condition µn n(c−1)/2−αγ → 0 and assumptions (A4),
(A5).
Taking into account (5.18), (5.19) and (5.20), we have for n and C1 large enough, that for all ǫ > 0:
r 


p
0
P inf Ln β + C1
u > 0 ≥ 1 − ǫ,
kuk=1
n


and the theorem follows.



Proof of Theorem 4.1. (i) We show, as for the case p fixed, that limn→∞ P[Card(Ac ∩ Abn ) ≥

1] = 0. Therefore we not give some calculation details. We assume without loss of generality that

21

Adaptive Fused LASSO in Grouped Quantile Regression

b , 0Ac ) of β and we will show that
Ac ∩ Abn = {j1 }. We consider the second estimator β ∗ ≡ (β
n;A

b ) > Qn (β ∗ ) with a probability converging to 1. We then study the following difference:
Qn (β
n
b ) − Qn (β∗ ) =
Qn (β
n

(1) b
+µ(1)
bn;j1 kβ
n;j1 k
n ω

n
X

i=1

n


 X
b ) − ρτ (εi ) −
ρτ (Yi − Xti β∗ ) − ρτ (εi )
ρτ (Yi − Xti β
n
i=1

 (2)
(2)
b
b
b
b
bn;j1 +1 kβ
ω
bn;j1 kβ
+
n;j1 +1 − β n;j1 k
n;j1 − β n;j1 −1 k + ω

(2)
(2) b
b
bn;j1 +1 kβ
−b
ωn;j1 kβ
n;j1 +1 k .
n;j1 −1 k − ω

(5.21)

µ(2)
n

Under assumptions (A1)-(A4), we have, as in the proof of Theorem 4 of Ciuperca (2016a), that:

 X
n 
n 
X

t ∗
b −β∗ k2 = OP (p) = OP (nc ).
b )−ρτ (εi ) −
ρ
(Y
−X
β
)−ρ
(ε
)
= OP nkβ
ρτ (Yi −Xti β
τ i
τ
i
n
i
n
i=1

i=1

(5.22)

For the first term of the penalty of (5.21), we have:
(1)

(1)
b
0 < µ(1)
bn;j1 kβ
n;j1 k = µn OP
n ω
c

Since j1 ∈ A and also

 p (1−γ)/2
n



(1)
µn n −c(1+γ)+γ−1 /2

(5.22).

(c−1)(1−γ)/2
= µ(1)
).
n OP (n

(5.23)

→ ∞, as n → ∞,we have that term (5.23) dominates

(2)

Now we study the penalty corresponding to µn , by considering the three possible cases for
β 0j1 −1 , β 0j1 , β0j1 +1 .
Case 1. β 0j1 6= β0j1 −1 and β 0j1 +1 6= β0j1 .

−γ 
(2)
0
0
−γ
−γ
e
e −β0 −β
e
e −β
= kβ
Then ω
bn;j1 = kβ
= OP (pn−1 )1/2 − β 0j1 −1
j1 −1 k
j1
j1 −1 +β j1 −1 −β j1 −1 k
j1
j1




(1)
bn;j1 = OP (pn−1 )−γ/2 = OP n(1−c)γ/2 .
= OP (n(c−1)/2 −nα )−γ = OP n−αγ . On the other hand, ω
(1)

(1)

(2)

(1)

b
bn;j1 kβ
bn;j1 , with a probability converging to 1. Therefore, µn ω
Since α > (c−1)/2, then, ω
bn;j1 ≫ ω
n;j1 k+

P
(2) (2)
(1) (1) b
b
b
b
µn ω
bn;j1 kβ
bn;j1 kβn;j1 k(1 + oP (1)) −→ ∞.
n;j1 − β n;j1 −1 k − kβ n;j1 −1 k ≥ µn ω
n→∞

(2) (2)
(2) b
−γ
b
e
e
e −β
e
On the other hand, µn ω
bn;j1 +1 kβ
, with kβ
n;j1 k = µnkβ n;j1 k · kβ j1 − β j1 +1 k
j1 +1 k =
j1


(1)
0
0
e
bn;j1 ≫
= OP n(c−1)/2 + nα = OP (nα ). Thus, ω
OP (pn−1 )1/2 + k − β
j1 +1 + β j1 +1 − β j1 +1 k

22

Adaptive Fused LASSO in Grouped Quantile Regression

(2)

ω
bn;j1 +1 , with a probability converging to 1.

(1) b
b )−Qn (β ∗ ) of relation (5.21) is dominated by µ(1)
In conclusion, the right-hand side of Qn (β
bn;j1 kβ
n ω
n;j1 k,
n

which converges in probability to ∞.

Case 2. β 0j1 6= β0j1 −1 and β 0j1 +1 = β0j1 = 0 (or vice versa).


(2)
−γ
e
e −β
= OP k(pn−1 )1/2 − β 0j1 −1 k−γ = OP (n−αγ ). Then,
In this case we have, ω
bn;j1 = kβ
j1 −1 k
j1
(1)

(2)

bn;j1 , with a probability converging to 1.
ω
bn;j1 ≫ ω



(2)
(2)
e
e −γ = OP (pn−1 )−γ/2 = OP n(1−c)γ/2 ≫ ω
bn;j1 , with
We have also that ω
bn;j1 +1 = kβ
j1 +1 − β j1 k
probability converging to 1.

Afterwards, relations are similar to those of the fixed p case and we deduce (5.16).
Case 3. β 0j1 = β0j1 −1 = β0j1 +1 , is similar to the fixed p case to derive relation (5.17).
b ) > Qn (β ∗ ) with a probability converging to one, which
Thus, in all three cases, we have: Qn (β
n

b is not the minimizer of Qn . Thus, limn→∞ P[j1 ∈ Ac ∩ Abn ] = 0.
implies β
n

To complete the demonstration of claim (i) we need to show that P[A ⊆ Abn ] → 1. For this, we

prove that:



b k > 0 = 1.
lim P min kβ
n;j

n→∞

j∈A

The proof is similar to that of Theorem 4 of Ciuperca (2016a).

b can be written, with a
(ii) Taking into account claim (i) and assumption (A6), the estimator β
n

b = β 0 +(pn−1 )1/2 δ, with δ = (δ A , δ Ac ), δ Ac = 0r−r0 , kδ A k ≤ C.
probability converging to 1, as, β
n
Then, we will consider the parameters of the form β = β 0 + (pn−1 )1/2 δ.

In order to prove claim (ii), let us consider the following penalized random process:
r
r 



n  
X
p
p
0
t
0
ρτ Yi − Xi (β +
δ =
δ) − ρτ (εi ) + P1 + P2 ,
Ln β +
n
n
i=1

23

(5.24)

Adaptive Fused LASSO in Grouped Quantile Regression

(1)

with the penalties, P1 ≡ µn

Pp

j=1



(1) 
(2) 
(2) Pp
ω
bn;j kβj k − kβ0j k and P2 ≡ µn
bn;j kβ j − β j−1 k − kβ0j − β 0j−1 k .
j=2 ω

For the first term of the right-hand side of (5.24), we have
n 
X
i=1

=n

r



p
0
t
ρτ Yi − Xi (β +
δ) − ρτ (εi )
n
1
n

r

n

n

pX t
1X
Xi δ[11ε≤0 − τ ] +
n i=1
n i=1

Z √ np Xti δ
0

[11εi ≤t − 11εi ≤0 ]dt

!

≡ n(J1 + J2 ).

As in the proof of Theorem 4 of Ciuperca (2016a), we obtain that IE[J2 ] = Cf (0)pn−1 , also that
the difference of the quantile processes is:
n 
X
i=1

r


 r X
n
p
p
f (0) t
0
t
ρτ Yi −Xi (β +
Xti,A δ A [11εi <0 −τ ]+
δ) −ρτ (εi ) =
pδ A Υn,A δ A (1+oP (1))
n
n i=1
2

(5.25)

and for the first penalty of (5.24), that,


(c−1)/2−αγ
|P1 | ≤ OP µ(1)
.
n pn

(5.26)

This last relation implies that |P1 | nIE[J2 ]
(1)

posed on µn .

−1



(1)
= OP µn n(c−1)/2−αγ = oP (1), by condition im-

Let us now consider penalty P2 of (5.24). Using assumptions (A4), (A5), we have,
|P2 | ≤

Cpµ(2)
n

max

j∈A\{2}

(2) 
ω
bn;j

 p 1/2
n


max
= OP nc n(c−1)/2 µ(2)
n

j∈A\{2}

(2)



(2) 
ω
bn;j



−αγ
.
= OP n(3c−1)/2 µ(2)
n n

Thus, by condition imposed to µn , we have,


|P2 |
(c−1)/2−αγ
= oP (1).
n
= OP µ(2)
n
nIE[J2 ]

(5.27)

24

Adaptive Fused LASSO in Grouped Quantile Regression

In conclusion, taking into account relations (5.24)-(5.27), we have that:
r  r n

p
pX t
f (0) t
0
Ln β +
δ =
pδ A Υn,A δ A (1 + oP (1)).
Xi,A δ A [11εi <0 − τ ] +
n
n i=1
2
The minimizer of the right-hand side of the last equation is:
r

n
X

1 1
p
Xi,A (11εi ≤0 − τ ) .
δA = −
Υ−1
n,A
n
n f (0)
i=1

b − β0 = (pn−1 )1/2 δ A and by applying
Claim (ii) follows by taking into account of the fact that β
A
A


the CLT for the following independent random variable sequence: (f (0))−1 ut Υ−1
X
(1
1
−
τ
)
εi ≤0
n,A iA
with u a r0 -vector such that kuk = 1.



References
Ciuperca, G., 2016a.

Adaptive group LASSO selection in quantile models.

arXiv preprint

arXiv:1601.08065.
Ciuperca, G., 2016b. Adaptive LASSO model selection in a multiphase quantile regression. Statistics, 50, No. 5, 1100-1131.
Huang, J., Breheny, P., Ma, S., 2012. A selective review of group selection in high-dimensional
models. Statistical Science, 27, No. 4, 481-499.
Jang, W., Lim, J., Lazar, N.A., Loh, J.M., Yu, D., 2015. Some properties of generalized fused lasso
and its applications to high dimensional data. Journal of the Korean Statistical Society, 44, No.
3, 352–365.
Jiang, L., Wang, H.J., Bondell, H.D., 2013. Interquantile shrinkage in regression models. Journal
of Computational and Graphical Statistics, 22, 970–986.
Jiang, L., Bondell, H.D., Wang, H.J., 2014. Interquantile shrinkage and variable selection in quantile
regression. Computational Statistics and Data Analysis, 69, 208–219.
25

1≤i≤n

,

Adaptive Fused LASSO in Grouped Quantile Regression

Koenker, R., 2005. Quantile Regression. Cambridge University Press.
Li, Y., Zhu, J., 2007. Analysis of array CGH data for cancer studies using fused quantile regression.
Bioinformatics, 23(18), 2470-2476.
Sun, Y., Wang, H.J., Fuentes, M., 2016. Fused adaptive lasso for spatial and temporal quantile
function estimation. Technometrics, 58, No. 1, 127-137.
Tibshirani, R., 1996. Regression shrinkage and selection via the LASSO. Journal of the Royal
Statistical Society: Series B, 58, 267-288.
Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., Knight, K., 2005. Sparsity and smoothness via
the fused lasso. Journal of the Royal Statistical Society: Series B, 67, 91-108.
Yuan, M., Lin, Y., 2006. Model selection and estimation in regression with grouped variables.
Journal of the Royal Statistical Society: Series B, 68(1), 49-67.
Zhang, C., Xiang, Y., 2015. On the oracle property of adaptive group LASSO in high-dimensional
linear models. Statistical Papers, 57(1), 249-265.
Zhao, W., Zhang, R., Liu, J., 2014. Regularization and model selection for quantile varying coefficient model with categorical effect modifiers. Computational Statistics and Data Analysis, 79,
44-62.
Zou, H., 2006. The adaptive Lasso and its oracle properties. Journal of the American Statistical
Association, 101, 1418-1428.
Zou, H., Yuan, M., 2008. Composite quantile regression and the oracle model selection theory. The
Annals of Statistics, 36(3), 1108-1126.
Zou, H., Zhang, H.H., 2009. On the adaptive elastic-net with a diverging number of parameters.
The Annals of Statistics, 37(4), 1733-1751.
Viallon, V., Lambert-Lacroix, S., Hoefling, H., Picard, F., 2016. On the robustness of the generalized
fused lasso to prior specifications. Statistics and Computing, 26, 285–301.
26

Adaptive Fused LASSO in Grouped Quantile Regression

Wang, H., Leng, C., 2008. A note on adaptive group lasso. Computational Statistics and Data
Analysis, 52, 5277-5286.
Wang, L., You, Y., Lian, H., 2015. Convergence and sparsity of Lasso and group Lasso in highdimensional generalized linear models. Statistical Papers, 56, No. 3, 819-828.
Wei, F., Huang, J., 2010. Consistent group selection in high-dimensional linear model. Bernoulli,
16(4), 1369-1384.
Wu, Y., Liu, Y., 2009. Variable selection in quantile regression. Statistica Sinica, 19, 801-817.

27

