Inference in partially identified models with many moment

arXiv:1604.02309v3 [math.ST] 23 May 2016

inequalities using Lasso∗
Federico A. Bugni

Mehmet Caner

Department of Economics

Department of Economics

Duke University

Ohio State University

Anders Bredahl Kock

Soumendra Lahiri

Department of Economics & Business

Department of Statistics

Aarhus University

North Carolina State University
May 24, 2016

Abstract
This paper considers inference in a partially identified moment (in)equality model with many moment
inequalities. We propose a novel two-step inference procedure that combines the methods proposed by
Chernozhukov et al. (2014c) (CCK14, hereafter) with a first-step moment inequality selection based on
the Lasso. Our method controls size uniformly, both in underlying parameter and data distribution. Also,
the power of our method compares favorably with that of the corresponding two-step method in CCK14
for large parts of the parameter space, both in theory and in simulations. Finally, our Lasso-based first
step is straightforward to implement.

Keywords and phrases: Many moment inequalities, self-normalizing sum, multiplier bootstrap, empirical
bootstrap, Lasso, inequality selection.
JEL classification: C13, C23, C26.

∗ We thank useful comments and suggestions from the participants in the 2015 World Congress in Montreal, the Second
International Workshop in Financial Econometrics, and the seminar at University of Maryland. Bugni acknowledges support
by National Institutes of Health under grant no. 40-4153-00-0-85-399. Bredahl Kock acknowledges support from CREATES Center for Research in Econometric Analysis of Time Series (DNRF78), funded by the Danish National Research Foundation.
Lahiri acknowledges support from National Science Foundation under grant no. DMS 130068.

1

1

Introduction

This paper contributes to the growing literature on inference in partially identified econometric models
defined by many unconditional moment (in)equalities, i.e., inequalities and equalities. Consider an economic
model with a parameter θ belonging to a parameter space Θ, whose main prediction is that the true value
of θ, denoted by θ0 , satisfies a collection of moment (in)equalities. This model is partially identified, i.e.,
the restrictions of the model do not necessarily restrict θ0 to a single value, but rather they constrain it
to belong to a certain set, called the identified set. The literature on partially identified models discusses
several examples of economic models that satisfy this structure, such as selection problems, missing data, or
multiplicity of equilibria (see, e.g., Manski (1995) and Tamer (2003)).
The first contributions in the literature of partially identified moment (in)equalities focus on the case in
which there is a fixed and finite number of moment (in)equalities, both unconditionally1 and conditionally2 .
In practice, however, there are many relevant econometric models that produce a large set of moment
conditions (even infinitely many). As several references in the literature point out (e.g. Menzel (2009, 2014)),
the associated inference problems cannot be properly addressed by an asymptotic framework with a fixed
number of moment (in)equalities.3 To address this issue, Chernozhukov et al. (2014c) (hereafter referred
to as CCK14) obtain inference results in a partially identified model with many moment (in)equalities.4
According to this asymptotic framework, the number of moment (in)equalities, denoted by p, is allowed to
be larger than the sample size n. In fact, the asymptotic framework allows p to be an increasing function
of n and even to grow at certain exponential rates. Furthermore, CCK14 allow their moment (in)equalities
to be “unstructured”, in the sense that they do not impose restrictions on the correlation structure of the
sample moment conditions.5 For these reasons, CCK14 represents a significant advancement relative to the
previous literature on inference in moment (in)equalities.
This paper builds on the inference method proposed in CCK14. Their goal is to test whether a collection
of p moment inequalities simultaneously holds or not. In order to implement their test they propose a test
1 These include Chernozhukov et al. (2007), Andrews et al. (2004), Imbens and Manski (2004), Galichon and Henry (2006,
2013), Beresteanu and Molinari (2008), Romano and Shaikh (2008), Rosen (2008), Andrews and Guggenberger (2009), Stoye
(2009), Andrews and Soares (2010), Bugni (2010, 2015), Canay (2010), Romano and Shaikh (2010), Andrews and Jia-Barwick
(2012), Bontemps et al. (2012), Bugni et al. (2012), Romano et al. (2014), and Pakes et al. (2015), among others.
2 These include Kim (2008), Ponomareva (2010), Armstrong (2014b,a), Chetverikov (2013), Andrews and Shi (2013), and
Chernozhukov et al. (2013c), among others.
3 As pointed out by Chernozhukov et al. (2014c), this is true even for conditional moment (in)equality models (which typically
produce an infinite number of unconditional moment (in)equalities). As they explain, the unconditional moment (in)equalities
generated by conditional moment (in)equality models inherit the structure from the conditional moment conditions, which
limits the underlying econometric model.
4 See also the related technical contributions in Chernozhukov et al. (2013b,a, 2014a,b).
5 This feature distinguishes their framework from a standard conditional moment (in)equality model. While conditional
moment conditions can generate an uncountable set of unconditional moment (in)equalities, their covariance structure is greatly
restricted by the conditioning structure.

2

statistic based on the maximum of p Studentized statistics and several methods to compute the critical
values. Their critical values may include a first stage inequality selection procedure with the objective of
detecting slack moment inequalities, thus increasing the statistical power. According to their simulation
results, including a first stage can result in significant power gains.
Our contribution is to propose a new inference method based on the combination of two ideas. On the
one hand, our test statistic and critical values are based on those proposed by CCK14. On the other hand,
we propose a new first stage selection procedure based on the Lasso. The Lasso was first proposed in the
seminal contribution by Tibshirani (1996) as a regularization technique in the linear regression model. Since
then, this method has found wide use as a dimension reduction technique in large dimensional models with
strong theoretical underpinnings.6 It is precisely these powerful shrinkage properties that serve as motivation
to consider the Lasso as a procedure to separate out and select binding moment inequalities from the nonbinding ones. Our Lasso first step inequality selection can be combined with any of the second step inference
procedures in CCK14: self-normalization, multiplier bootstrap, or empirical bootstrap.
The present paper considers using the Lasso to select moments in a partially identified moment
(in)equality model. In the context of point identified problems, there is an existing literature that proposes the Lasso to address estimation and moment selection in GMM settings. In particular, Caner (2009)
introduce Lasso type GMM-Bridge estimators to estimate structural parameters in a general model. The
problem of selection of moment in GMM is studied in Liao (2013) and Cheng and Liao (2015). In addition,
Caner and Zhang (2014) and Caner et al. (2016) find a method to estimate parameters in GMM with diverging number of moments/parameters, and selecting valid moments among many valid or invalid moments
respectively. In addition, Fan et al. (2015) consider the problem of inference in high dimensional models with
sparse alternatives. Finally, Caner and Fan (2015) propose a hybrid two-step estimation procedure based
on Generalized Empirical Likelihood, where instruments are chosen in a first-stage using an adaptive Lasso
procedure.
We obtain the following results for our two-step Lasso inference methods. First, we provide conditions
under which our methods are uniformly valid, both in the underlying parameter θ and the distribution of
the data. According to the literature in moment (in)equalities, obtaining uniformly valid asymptotic results
is important to guarantee that the asymptotic analysis provides an accurate approximation to finite sample

6 For

excellent reviews of this method see, e.g., Belloni and Chernozhukov (2011), Bühlmann and van de Geer (2011), Fan
et al. (2011), and Hastie et al. (2015).

3

results.7 Second, by virtue of results in CCK14, all of our proposed tests are asymptotically optimal in a
minimax sense. Third, we compare the power of our methods to the corresponding one in CCK14, both in
theory and in simulations. Since our two-step procedure and the corresponding one in CCK14 share the
second step, our power comparison is a comparison of the Lasso-based first-step vis-à-vis the ones in CCK14.
On the theory front, we obtain a region of underlying parameters under which the power of our method
dominates that of CCK14. We also conduct extensive simulations to explore the practical consequences of
our theoretical findings. Our simulations indicate that a Lasso-based first step is usually as powerful as the
one in CCK14, and can sometimes be more powerful. Fourth, we show that our Lasso-based first step is
straightforward to implement.
The remainder of the paper is organized as follows. Section 2 describes the inference problem and
introduces our assumptions. Section 3 introduces the Lasso as a method to distinguish binding moment
inequalities from non-binding ones and Section 4 considers inference methods that use the Lasso as a first
step. Section 5 compares the power properties of inference methods based on the Lasso with the ones in
the literature. Section 6 provides evidence of the finite sample performance using Monte Carlo simulations.
Section 7 concludes. Proofs of the main results and several intermediate results are reported in the appendix.
Throughout the paper, we use the following notation. For any set S, |S| denotes its cardinality, and for
Pd
any vector x ∈ Rd , ||x||1 ≡ i=1 |xi |.

2

Setup

For each θ ∈ Θ, let X(θ) : Ω → Rk be a k-dimensional random variable with distribution P (θ) and mean
µ(θ) ≡ EP (θ) [X(θ)] ∈ Rk . Let µj (θ) denote the jth component of µ(θ) so that µ(θ) = {µj (θ)}j≤k . The
main tenet of the econometric model is that the true parameter value θ0 satisfies the following collection of
p moment inequalities and v ≡ k − p moment equalities:

µj (θ0 ) ≤ 0 for j = 1, . . . , p,
µj (θ0 )

=

0 for j = p + 1, . . . , k.

(2.1)

As in CCK14, we are implicitly allowing the collection P of distributions of X(θ) and the number of moment
(in)equalities, k = p + v to depend on n. In particular, we are primarily interested in the case in which
7 In these models, the limiting distribution of the test statistic is discontinuous in the slackness of the moment inequalities,
while its finite sample distribution does not exhibit such discontinuities. In consequence, asymptotic results obtained for any
fixed distribution (i.e. pointwise asymptotics) can be grossly misleading, and possibly producing confidence sets that undercover
(even asymptotically). See Imbens and Manski (2004), Andrews and Guggenberger (2009), Andrews and Soares (2010), and
Andrews and Shi (2013) (Section 5.1).

4

p = pn → ∞ and v = vn → ∞ as n → ∞, but the subscripts will be omitted to keep the notation simple. In
particular, p and v can be much larger than the sample size and increase at rates made precise in Section
2.1. We allow the econometric model to be partially identified, i.e., the moment (in)equalities in Eq. (2.1)
do not necessarily restrict θ0 to a single value, but rather they constrain it to belong to the identified set,
denoted by ΘI (P ). By definition, the identified set is as follows:
(
ΘI (P ) ≡

(
θ∈Θ:

))

µj (θ) ≤ 0 for j = 1, . . . , p,

.

(2.2)

µj (θ) = 0 for j = p + 1, . . . , k.
Our goal is to test whether a particular parameter value θ ∈ Θ is a possible candidate for the true
parameter value θ0 ∈ ΘI (P ). In other words, we are interested in testing:

H0 : θ 0 = θ

vs.

H1 : θ0 6= θ.

(2.3)

By definition, the identified set is composed of all parameters that are observationally equivalent to the
true parameter value θ0 , i.e., every parameter value in ΘI (P ) is a candidate for θ0 . In this sense, θ = θ0
is observationally equivalent to θ ∈ ΘI (P ) and so the hypothesis test in Eq. (2.3) can be equivalently
reexpressed as:

H0 : θ ∈ ΘI (P )
(
i.e., H0 :

vs. H1 : θ 6∈ ΘI (P ),

µj (θ) ≤ 0 for all j = 1, . . . , p,

)
vs.

H1 : “not H0 ”.

(2.4)

µj (θ) = 0 for all j = p + 1, . . . , k.
In this paper, we propose a procedure to implement the hypothesis test in Eq. (2.3) (or, equivalently,
Eq. (2.4)) with a given significance level α ∈ (0, 1) based on a random sample of X(θ) ∼ P (θ), denoted by
X n (θ) ≡ {Xi (θ)}i≤n . The inference procedure will reject the null hypothesis whenever a certain test statistic
Tn (θ) exceeds a critical value cn (α, θ), i.e.,

φn (α, θ) ≡ 1[Tn (θ) > cn (α, θ)],

(2.5)

where 1[·] denotes the indicator function. By the duality between hypothesis tests and confidence sets, a
confidence set for θ0 can be constructed by collecting all parameter values for which the inference procedure

5

is not rejected, i.e.,
Cn (1 − α) ≡ {θ ∈ Θ : Tn (θ) ≤ cn (α, θ)}.

(2.6)

Our formal results will have the following structure. Let P denote a set of probability distributions. We will
show that for all P ∈ P and under H0 ,

P Tn (θ) > cn (α, θ)



≤ α + o(1).

(2.7)

Moreover, the convergence in Eq. (2.7) will be shown to occur uniformly over both P ∈ P and θ ∈ Θ. This
uniform size control result in Eq. (2.7) has important consequences regarding our inference problem. First,
this result immediately implies that the hypothesis test procedure in Eq. (2.5) uniformly controls asymptotic
size i.e., for all θ ∈ Θ and under H0 : θ0 = θ,

lim sup sup E[φn (α, θ)] ≤ α.
n→∞

(2.8)

P ∈P

Second, the result also implies that the confidence set in Eq. (2.6) is asymptotically uniformly valid, i.e.,

lim inf inf
n→∞

P ∈P

inf
θ∈ΘI (P )


P θ ∈ Cn (1 − α)

≥ 1 − α.

(2.9)

The rest of the section is organized as follows. Section 2.1 specifies the assumptions on the probability
space P that are required for our analysis. All the inference methods described in this paper share the test
statistic Tn (θ) and differ only in the critical value cn (α, θ). The common test statistic is introduced and
described in Section 2.2.

2.1

Assumptions

The collection of distributions P ≡ {P (θ) : θ ∈ Θ} are assumed to satisfy the following assumptions.
Assumption A.1.

For every θ ∈ Θ, let X n (θ) ≡ {Xi (θ)}i≤n be i.i.d. k-dimensional random vectors

distributed according to P (θ). Further, let EP (θ) [X1j (θ)] ≡ µj (θ) and V arP (θ) [X1j (θ)] ≡ σj2 (θ) > 0, where
Xij (θ) denotes the j component of Xi (θ).
Assumption A.2. For some δ ∈ (0, 1], maxj=1,...,k supθ∈Θ (EP (θ) [|X1j (θ)|2+δ ])1/(2+δ) ≡ Mn,2+δ < ∞ and
2+δ
Mn,2+δ
(ln(2k − p))(2+δ)/2 n−δ/2 → 0.

6

Assumption A.3.

For some c ∈ (0, 1), (n−(1−c)/2 ln(2k − p) + n−3/2 (ln(2k − p))2 )Bn2 → 0, where

supθ∈Θ (EP (θ) [maxj=1,...,k |Z1j (θ)|4 ])1/4 ≡ Bn < ∞ and Zij (θ) ≡ (Xij (θ) − µj (θ))/σj (θ).
3
2
Assumption A.4. For some c ∈ (0, 1/2) and C > 0, max{Mn,3
, Mn,4
, Bn }2 ln((2k − p)n)7/2 ≤ Cn1/2−c ,

where Mn,2+δ and Bn are as in Assumptions A.2-A.3.
We now briefly describe these assumptions. Assumption A.1 is standard in microeconometric applications.
Assumption A.2 has two parts. The first part requires that Xij (θ) has finite (2 + δ)-moments for all j =
1, . . . , k. The second part limits the rate of growth of Mn,2+δ and the number of moment (in)equalities.
Notice that Mn,2+δ is a function of the sample size because maxj=1,...,k supθ∈Θ (EP (θ) [|X1j (θ)|2+δ ])1/(2+δ) is
function of P and k = v + p, both of which could depend on n. Also, notice that 2k − p = 2v + p, i.e.,
the total number of moment inequalities p plus twice the number of moment equalities v, all of which could
depend on n. Assumption A.3 could be interpreted in a similar fashion as Assumption A.2, except that it
refers to the standardized random variable Zij (θ) ≡ (Xij (θ) − µj (θ))/σj (θ). Assumption A.4 is a technical
assumption that is used to control the size of the bootstrap test in CCK14.8

2.2

Test statistic

Throughout the paper, we consider the following test statistic:
√


Tn (θ) ≡ max

where, for j = 1, . . . , k, µ̂j (θ) ≡

1
n

Pn

i=1

max

j=1,...,p

nµ̂j (θ)
, max
σ̂j (θ) s=p+1,...,k

Xij (θ) and σ̂j2 (θ) ≡

1
n

Pn

i=1

√

n µ̂s (θ)
σ̂s (θ)


,

(2.10)

2
Xij (θ) − µ̂j (θ) . Note that Eq. (2.10)

is not properly defined if σ̂j2 (θ) = 0 for some j = 1, . . . , k and, in such cases, we use the convention that
C/0 ≡ ∞ × 1[C > 0] − ∞ × 1[C < 0].
The test statistic is identical to that in CCK14 with the exception that we allow for the presence of
moment equalities. By definition, large values of Tn (θ) are an indication that H0 : θ = θ0 is likely to be
violated, leading to the hypothesis test in Eq. (2.5). The remainder of the paper considers several procedures
to construct critical values that can be associated to this test statistic.

8 We point out that Assumptions A.1-A.4 are tailored for the construction of confidence sets in Eq. (2.6) in the sense that
all the relevant constants are defined uniformly in θ ∈ Θ. If we were only interested in the hypothesis testing problem for a
particular value of θ, then the previous assumptions could be replaced by their “pointwise” versions at the parameter value of
interest.

7

3

Lasso as a first step moment selection procedure

In order to propose a critical value for our test statistic Tn (θ), we need to approximate its distribution under
the null hypothesis. According to the econometric model in Eq. (2.1), the true parameter satisfies p moment
inequalities and v moment equalities. By definition, the moment equalities are always binding under the
null hypothesis. On the other hand, the moment inequalities may or may not be binding, and a successful
approximation of the asymptotic distribution depends on being able to distinguish between these two cases.
Incorporating this information into the hypothesis testing problem is one of the key issues in the literature
on inference in partially identified moment (in)equality models.
In their seminal contribution, CCK14 is the first paper in the literature to conduct inference in a partially identified model with many unstructured moment inequalities. Their paper proposes several procedures to select binding moment inequalities from non-binding based on three approximation methods:
self-normalization (SN), multiplier bootstrap (MB), and empirical bootstrap (EB). Our relative contribution
is to propose a novel approximation method based on the Lasso. By definition, the Lasso penalizes parameters values by their `1 -norm, with the ability of producing parameter estimates that are exactly equal to
zero. This powerful shrinkage property is precisely what motivates us to consider the Lasso as a first step
moment selection procedure in a model with many moment (in)equalities. As we will soon show, the Lasso
is an excellent method to detect binding moment inequalities from non-binding ones, and this information
can be successfully incorporated into an inference procedure for many moment (in)equalities.
For every θ ∈ Θ, let J(θ) denote the true set of binding moment inequalities, i.e., J(θ) ≡ {j =
1, . . . , p : µj (θ) ≥ 0}. Let µI (θ) ≡ {µj (θ)}pj=1 denote the moment vector for the moment inequalities
and let µ̂I (θ) ≡ {µ̂j (θ)}pj=1 denote its sample analogue. In order to detect binding moment inequalities, we
consider the weighted Lasso estimator of µI (θ), given by:

µ̂L (θ) ≡ arg min

0



µ̂I (θ) − t Ŵ (θ) µ̂I (θ) − t + λn Ŵ (θ)

1/2


t

,

(3.1)

1

t∈Rp

where λn is a positive penalization sequence that controls the amount of regularization and Ŵ (θ) is a
positive definite weighting matrix. To simplify the computation of the Lasso estimator, we impose Ŵ (θ) ≡
diag{1/σ̂j (θ)2 }pj=1 . As a consequence, Eq. (3.1) becomes:

µ̂L (θ) =

arg min

n

op
2
µ̂j (θ) − m + λn σ̂j (θ)|m|

m∈R

.

(3.2)

j=1

Notice that instead of using the Lasso in one p-dimensional model we instead use it in p one-dimensional

8

models. As we shall see later, µ̂L (θ) in Eq. (3.2) is closely linked to the soft-thresholded least squares
estimator, which implies that its computation is straightforward. The Lasso estimator µ̂L (θ) implies a
Lasso-based estimator of J(θ), given by:

JˆL (θ) ≡ {j = 1, . . . , p : µ̂j,L (θ)/σ̂j (θ) ≥ − λn }.

(3.3)

In order to implement this procedure, we need to choose the sequence λn , which determines the degree
of regularization imposed by the Lasso. A higher value of λn will produce a larger number of moment
inequalities considered to be binding, resulting in a lower rejection rate. In consequence, this is a critical
choice for our inference methodology. According to our theoretical results, a suitable choice of λn is given
by:

−1/2
2
λn = (4/3 + ε)n−1/2 Mn,2+δ
n−δ/(2+δ) − n−1

(3.4)

for any arbitrary ε > 0. Assumption A.2 implies that λn in Eq. (3.4) satisfies λn → 0. Notice that Eq. (3.4)
is infeasible as it depends on the unknown expression Mn,2+δ . In practice, one can replace this unknown
expression with its sample analogue:

2
M̂n,2+δ
= max

sup

j=1,...,k θ∈Θ



n−1

Xn
i=1

|Xij (θ)|2+δ

2/(2+δ)

.

In principle, a more rigorous choice of λn can be implemented via a modified BIC method designed for
divergent number of parameters as in Wang et al. (2009) or Caner et al. (2016).9
As explained earlier, our Lasso procedure is used as a first step in order to detect binding moment
inequalities from non-binding ones. The following result formally establishes that our Lasso procedure
includes all binding ones with a probability that approaches one, uniformly.
Lemma 3.1. Assume Assumptions A.1-A.3, and let λn be as in Eq. (3.4). Then,
 nδ/(2+δ)  
 M
2+δ 
n,2+δ
P [J(θ) ⊆ JˆL (θ)] ≥ 1 − 2p exp −
1
+
K
+
1
+ K̃n−c = 1 + o(1),
2
2Mn,2+δ
nδ/(2(2+δ))
where K, K̃ are universal constants and the convergence is uniform in all parameters θ ∈ Θ and distributions
P that satisfy the assumptions in the statement.
Thus far, our Lasso estimator of the binding constrains in Eq. (3.3) has been defined in terms of the
9 Nevertheless,

it is unclear whether the asymptotic properties of this method carry over to our partially identified moment
(in)equality model. We consider that a rigorous handling of these issues is beyond the scope of this paper.

9

solution of the p-dimensional minimization problem in Eq. (3.2). We conclude the subsection by providing
an equivalent closed form solution for this set.
Lemma 3.2. Eq. (3.3) can be equivalently reexpressed as follows:

JˆL (θ) = {j = 1, . . . , p : µ̂j (θ)/σ̂j (θ) ≥ −3λn /2}.

(3.5)

Lemma 3.2 is a very important computational aspect of our methodology. This result reveals that JˆL (θ)
can be computed by comparing standardized sample averages with a modified threshold of −3λn /2. In
other words, our Lasso-based first stage can be implemented without the need of solving the p-dimensional
minimization problem in Eq. (3.2).

4

Inference methods with Lasso first step

In the remainder of the paper we show how to conduct inference in our partially identified many moment
(in)equality model by combining the Lasso-based first step in Section 3 with a second step based on the
inference methods proposed by CCK14. In particular, Section 4.1 combines our Lasso-based first step with
their self-normalization approximation, while Section 4.2 combines it with their bootstrap approximations.

4.1

Self-normalization approximation

Before describing our self-normalization (SN) approximation with Lasso first stage, we first describe the
“plain vanilla” SN approximation without first stage moment selection. Our treatment extends the SN
method proposed by CCK14 to the presence of moment equalities.
As a preliminary step, we now define the SN approximation to the (1 − α)-quantile of Tn (θ) in a hypothetical moment (in)equality model composed of |J| moment inequalities and k − p moment equalities, given
by:
cSN
n (|J|, α) ≡





 0





if 2(k − p) + |J| = 0,

Φ−1 (1−α/(2(k−p)+|J|))
r 
2
1− Φ−1 (1−α/(2(k−p)+|J|)) /n

if 2(k − p) + |J| > 0.

(4.1)

Lemma A.4 in the appendix shows that cSN
n (|J|, α) provides asymptotic uniform size control in a hypothetical
moment (in)equality model with |J| moment inequalities and k−p moment equalities under Assumptions A.1A.2. The main difference between this result and CCK14 (Theorem 4.1) is that we allow for the presence of
moment equalities. Since our moment (in)equality model has |J| = p moment inequalities and k − p moment

10

equalities, we can define the regular (i.e. one-step) SN approximation method by using |J| = p in Eq. (4.1),
i.e.,
cSN,1S
(α) ≡ cSN
n
n (p, α) =

r

Φ−1 (1−α/(2k−p))
.

2
1− Φ−1 (1−α/(2k−p)) /n

The following result is a corollary of Lemma A.4.
Theorem 4.1 (One-step SN approximation). Assume Assumptions A.1-A.2, α ∈ (0, 0.5), and that H0
holds. Then,


 2+δ
2+δ
−δ/2
−1
P Tn (θ) > cSN,1S
(α)
≤
α
+
αKn
M
1
+
Φ
1
−
α/(2k
−
p)
= α + o(1),
n
n,2+δ
where K is a universal constant and the convergence is uniform in all parameters θ ∈ Θ and distributions P
that satisfy the assumptions in the statement.
By definition, this SN approximation considers all moment inequalities in the model as binding. A
more powerful test can be constructed by using the data to reveal which moment inequalities are slack. In
particular, CCK14 propose a two-step SN procedure which combines a first step moment inequality based
on SN methods and the second step SN critical value in Theorem 4.1. If we adapt their procedure to the
presence of moment equalities, this would be given by:

ˆ
cSN,2S (θ, α) ≡ cSN
n (|JSN (θ)|, α − 2βn )

(4.2)

with:

JˆSN (θ) ≡

o
n
√
j ∈ {1, . . . , p} : nµ̂j (θ)/σ̂j (θ) > −2cSN,1S (βn ) ,

where {βn }n≥1 is an arbitrary sequence of constants in (0, α/3). By extending arguments in CCK14 to
include moment equalities, one can show that inference based on the critical value cSN,2S (θ, α) in Eq. (4.2)
is asymptotically valid in a uniform sense.
In this paper, we propose an alternative SN procedure by using our Lasso-based first step. In particular,
we define the following two-step Lasso SN critical value:

ˆ
cSN,L
(θ, α) ≡ cSN
n
n (|JL (θ)|, α),

(4.3)

where JˆL (θ) is as in Eq. (3.5). The following result shows that an inference method based on our two-step
11

Lasso SN critical value is asymptotically valid in a uniform sense.
Theorem 4.2 (Two-step Lasso SN approximation). Assume Assumptions A.1-A.3, α ∈ (0, 0.5), and that
H0 holds, and let λn be as in Eq. (3.4). Then,

P Tn (θ) > cSN,L
(θ, α)
n
"
≤α+

#

2+δ
αKn−δ/2 Mn,2+δ
(1 + Φ−1 1 − α/(2k − p) )2+δ +

h

2+δ i
−2
4p exp −2−1 nδ/(2+δ) Mn,2+δ
1 + K n−δ/(2(2+δ)) Mn,2+δ + 1
+ 2K̃n−c

= α + o(1),

where K, K̃ are universal constants and the convergence is uniform in all parameters θ ∈ Θ and distributions
P that satisfy the assumptions in the statement.
We now compare our two-step SN Lasso method with the SN methods in CCK14. Since all inference
methods share the test statistic, the only difference lies in the critical values. While the one-step SN critical
values considers all p moment inequalities as binding, our two-step SN Lasso critical value considers only
|JˆL (θ)| moment inequalities as binding. Since |JˆL (θ)| ≤ p and cSN
n (α, |J|) is weakly increasing in |J| (see
Lemma A.3 in the appendix), then our two-step SN method results in a weakly larger rejection probability
(θ, α) and cnSN,2S (θ, α) is not straightforward
for all sample sizes. In contrast, the comparison between cSN,L
n
as these differ in two aspects. First, the set of binding constrains JˆSN (θ) according to SN differs from the set
of binding constrains JˆL (θ) according to the Lasso. Second, the quantile of the critical values are different:
the two-step SN method in Eq. (4.2) considers the α − 2βn quantile while the Lasso-based method considers
the usual α quantile. As a result of these differences, the comparison of these critical values is ambiguous
and so is the resulting power comparison. This topic will be discussed in further detail in Section 5.

4.2

Bootstrap methods

CCK14 also propose two bootstrap approximation methods: multiplier bootstrap (MB) and empirical bootstrap (EB). Relative to the SN approximation, bootstrap methods have the advantage of taking into account
√
the dependence between the coordinates of { nµ̂j (θ)/σ̂j (θ)}pj=1 involved in the definition of the test statistic
Tn (θ).
As in the previous subsection, we first define the bootstrap approximation to the (1−α)-quantile of Tn (θ)
in a hypothetical moment (in)equality model composed of moment inequalities indexed by the set J and the
B
k − p moment equalities. The corresponding MB and EB approximations are denoted by cM
n (θ, J, α) and

12

cEB
n (θ, J, α), respectively, and are computed as follows.
Algorithm 4.1. Multiplier bootstrap (MB)
1. Generate i.i.d. standard normal random variables {i }ni=1 , and independent of the data X n (θ).
2. Construct the multiplier bootstrap test statistic:
(
WnM B (θ, J) = max max

√1
n

Pn

i=1 i (Xij (θ)

− µ̂j (θ))

σ̂j (θ)

j∈J

,

max

√1 |
n

Pn

i=1 i (Xis (θ)

− µ̂s (θ))|

)
.

σ̂s (θ)

s=p+1,...,k

B
MB
3. Calculate cM
(θ, J) (given X n (θ)).
n (θ, J, α) as the conditional (1 − α)-quantile of Wn

Algorithm 4.2. Empirical bootstrap (EB)
1. Generate a bootstrap sample {Xi∗ (θ)}ni=1 from the data, i.e., an i.i.d. draw from the empirical distribution of X n (θ).
2. Construct the empirical bootstrap test statistic:
(
WnEB (θ, J)

= max max
j∈J

√1
n

Pn

∗
i=1 (Xij (θ)

− µ̂j (θ))

σ̂j (θ)

,

max

s=p+1,...,k

√1 |
n

Pn

∗
i=1 (Xis (θ)

− µ̂s (θ))|

σ̂s (θ)

)
.

EB
n
3. Calculate cEB
n (θ, J, α) as the conditional (1 − α)-quantile of Wn (θ, J) (given X (θ)).

All the results in the remainder of the section will apply to both versions of the bootstrap, and under
the same assumptions. For this reason, we can use cB
n (θ, J, α) to denote the bootstrap critical value where
B ∈ {M B, EB} represents either MB or EB. Lemma A.5 in the appendix shows that cB
n (θ, J, α) for B ∈
{M B, EB} provides asymptotic uniform size control in a hypothetical moment (in)equality model composed
of moment inequalities indexed by the set J and the k − p moment equalities under Assumptions A.1 and
A.4. As in Section 4.1, the main difference between this result and CCK14 (Theorem 4.3) is that we allow for
the presence of the moment equalities. Since our moment (in)equality model has |J| = p moment inequalities
and k − p moment equalities, we can define the regular (i.e. one-step) MB or EB approximation method by
using |J| = p in Algorithm 4.1 or 4.2, respectively, i.e.,

cB,1S
(θ, α) ≡ cB
n
n (θ, {1, . . . , p}, α),
where cB
n (θ, J, α) is as in Algorithm 4.1 if B = M B or Algorithm 4.2 if B = EB. The following result is a
corollary of Lemma A.5.
13

Theorem 4.3 (One-step bootstrap approximation). Assume Assumptions A.1, A.4, α ∈ (0, 0.5), and that
H0 holds. Then,

P Tn (θ) > cB,1S
(θ, α) ≤ α + C̃n−c̃ ,
n
where c̃, C̃ > 0 are positive constants that only depend on the constants c, C in Assumption A.4. Furthermore,
if µj (θ) = 0 for all j = 1, . . . , p, then

|P Tn (θ) > cB,1S
(θ, α) − α| ≤ C̃n−c̃ .
n

Finally, the proposed bounds are uniform in all parameters θ ∈ Θ and distributions P that satisfy the
assumptions in the statement.
As in the SN approximation method, the regular (one-step) bootstrap approximation considers all moment
inequalities in the model as binding. A more powerful bootstrap-based test can be constructed using the data
to reveal which moment inequalities are slack. However, unlike in the SN approximation method, Theorem
4.3 shows that the size of the test using the bootstrap critical values converges to α when all the moment
inequalities are binding. This difference comes from the fact that the bootstrap can better approximate the
correlation structure in the moment inequalities, which is not taken into account by the SN approximation.
As we will see in simulations, this translates into power gains in favor of the bootstrap.
CCK14 propose a two-step bootstrap procedure, combining a first step moment inequality based on the
bootstrap with the second step bootstrap critical value in Theorem 4.3.10 If we adapt their procedure to the
presence of moment equalities, this would be given by:

ˆ
cB,2S (θ, α) ≡ cB
n (θ, JB (θ), α − 2βn )

(4.4)

with:

JˆB (θ) ≡ {j ∈ {1, . . . , p} :

√

nµ̂j (θ)/σ̂j (θ) > −2cB,1S (α, βn )},

where {βn }n≥1 is an arbitrary sequence of constants in (0, α/2). Again, by extending arguments in CCK14
to the presence of moment equalities, one can show that an inference method based on the critical value
cB,2S (θ, α) in Eq. (4.4) is asymptotically valid in a uniform sense.
10 They also consider the so-called “hybrid” procedures in which the first step can be based on one approximation method
(e.g. SN approximation) and the second step could be based on another approximation method (e.g. bootstrap). While these
are not explicitly addressed in this section they are included in the Monte Carlo section.

14

This paper proposes an alternative bootstrap procedure by using our Lasso-based first step. For B ∈
{M B, EB}, define the following two-step Lasso bootstrap critical value:

B
ˆ
cB,L
n (θ, α) ≡ cn (θ, JL (θ), α),

(4.5)

where JˆL (θ) is as in Eq. (3.5), and cB
n (θ, J, α) is as in Algorithm 4.1 if B = M B or Algorithm 4.2 if B = EB.
The following result shows that an inference method based on our two-step Lasso bootstrap critical value is
asymptotically valid in a uniform sense.
Theorem 4.4 (Two-step Lasso bootstrap approximation). Assume Assumptions A.1, A.2, A.3, A.4, α ∈
(0, 0.5), and that H0 holds, and let λn be as in Eq. (3.4). Then, for B ∈ {M B, EB},
P Tn (θ) > cB,L
n (θ, α)




h
i
2
≤ α + C̃n−c̃ + Cn−c + 2K̃n−c + 4p exp 2−1 nδ/(2+δ) /Mn,2+δ
1 + K(Mn,2+δ /nδ/(2(2+δ) + 1)2+δ
= α + o(1),

where c̃, C̃ > 0 are positive constants that only depend on the constants c, C in Assumption A.4, K, K̃ are
universal constants, and the convergence is uniform in all parameters θ ∈ Θ and distributions P that satisfy
the assumptions in the statement. Furthermore, if µj (θ) = 0 for all 1 ≤ j ≤ p and

h
i
2
K̃n−c + 2p exp 2−1 nδ/(2+δ) /Mn,2+δ
1 + K(Mn,2+δ /nδ/(2(2+δ) + 1)2+δ ≤ C̃n−c̃ ,

(4.6)

then,

−c̃
|P Tn (θ) > cB,L
+ Cn−c = o(1),
n (θ, α) − α| ≤ 3C̃n
where all constants are as defined earlier and the convergence is uniform in all parameters θ ∈ Θ and
distributions P that satisfy the assumptions in the statement.
By repeating arguments at the end of Section 4.1, it follows that our two-step bootstrap method results in
a larger rejection probability than the one-step bootstrap method for all sample sizes.11 Also, the comparison
B,2S
between cB,L
(θ, α) is not straightforward as these differ in the same two aspects described
n (θ, α) and cn

Section 4.1. This comparison will be the topic of the next section.

11 To

establish this result, we now use Lemma A.6 instead of Lemma A.3.

15

5

Power comparison

CCK14 show that all of their inference methods satisfy uniform asymptotic size control under appropriate
assumptions. Theorems 4.2 and 4.4 show that our Lasso-based two-step inference methods also satisfy
uniform asymptotic size control under similar assumptions. Given these results, the natural next step is to
compare these inference methods in terms of criteria related to power.
One possible such criterion is minimax optimality, i.e., the ability that a test has of rejecting departures
from the null hypothesis at the fastest possible rate (without loosing uniform size control). CCK14 show
that all their proposed inference methods are asymptotically optimal in a minimax sense, even in the absence
of any inequality selection (i.e. defined as in Theorems 4.1 and 4.3 in the presence of moment equalities).
Since our Lasso-based inequality selection can only reduce the number of binding moment inequalities (thus
increasing rejection), we can also conclude that all of our two-step Lasso-based inference methods (SN, MB,
and EB) are also asymptotically optimal in a minimax sense. In other words, minimax optimality is a
desirable property that is satisfied by all tests under consideration and, thus, cannot be used as a criterion
to distinguish between them.
Thus, we proceed to compare our Lasso-based inference procedures with those proposed by CCK14 in
terms of rejection rates. Since all inference methods share the test statistic Tn (θ), the power comparison
depends exclusively on the critical values.

5.1

Comparison with one-step methods

As pointed out in previous sections, our Lasso-based two-step inference methods will always be more powerful
than the corresponding one-step analogue, i.e.,


P Tn (θ) > cSN,L
(θ, α) ≥
n

P Tn (θ) > cB,L
≥
n (θ, α)

P Tn (θ) > cSN,1S
(α)
n




P Tn (θ) > cB,1S
(θ, α) ∀B ∈ {M B, EB},
n

for all θ ∈ Θ and n ∈ N. This is a direct consequence of the fact that one-step critical values are based on
considering all moment inequalities as binding, while the Lasso-based first-step will restrict attention to the
subset of them that are sufficiently close to binding, i.e., JˆL (θ) ⊆ {1, . . . , p}.

16

5.2

Comparison with two-step methods

The comparison between our two-step Lasso procedures and the two-step methods in CCK14 is not straightforward for two reasons. First, the set of binding inequalities according to the Lasso might be different from
the other methods. Second, our Lasso-based methods considers the usual α quantile while the other two-step
methods consider the α − 2βn quantile for a sequence of positive constants {βn }n≥1 .
To simplify the discussion, we focus exclusively on the case where the moment (in)equality model is only
composed of inequalities, i.e., k = p, which is precisely the setup in CCK14. This is done for simplicity of
exposition, the introduction of moment equalities would not qualitatively change the conclusions that follow.
We begin by comparing the two-step SN method with the two-step Lasso SN method. For all θ ∈ Θ and
n ∈ N, our two-step Lasso SN method will have more power than the two-step SN method if and only if
(α). By inspecting the formulas in CCK14, this occurs if and only if:
(θ, α) ≤ cSN,2S
cSN,L
n
n
α
|JˆSN (θ)|,
α − 2βn

|JˆL (θ)| ≤

(5.1)

where, by definition, {βn }n≥1 satisfies βn ≤ α/3. We provide sufficient conditions for Eq. (5.1) in the
following result.
Theorem 5.1. For all θ ∈ Θ and n ∈ N,
JˆL (θ) ⊆ JˆSN (θ)

(5.2)

implies
P Tn (θ) > cSN,L
(θ, α)
n




≥ P Tn (θ) > cnSN,2S (α) .

(5.3)

In turn, Eq. (5.2) occurs under any of the following circumstances:

2
βn ≤ 0.1, Mn,2+δ
n2/(2+δ)

√
4 SN
cn (βn ) ≥ nλn , or,
3

 9 4
p
−2
√
≥ 2, and ln
≥ ( + ε)2 nδ/(2+δ) Mn,2+δ
,
8 3
2βn 2π

(5.4)
(5.5)

where ε > 0 is as in Eq. (3.4).
Theorem 5.1 provides two sufficient conditions under which our two-step Lasso SN method will have
greater or equal power than the two-step SN method in CCK14. The power difference is a direct consequence
of Eq. (5.2), i.e., our Lasso-based first step inequality selection procedure chooses a subset of the inequalities

17

in the SN-based first step. The first sufficient condition, Eq. (5.4), is sharper than the second one, Eq. (5.5),
but the second one is of lower level and, thus, easier to interpret and understand. Eq. (5.5) is composed
of three statements and only the third one could be considered restrictive. The first one, β ≤ 10%, is
non-restrictive as CCK14 require that βn ≤ α/3 and the significance level α is typically less than 30%. The
2
2
second, Mn,2+δ
n2/(2+δ) ≥ 2, is also non-restrictive since Mn,2+δ
is a non-decreasing sequence of positive

constants and n2/(2+δ) → ∞.
In principle, Theorem 5.1 allows for the possibility of the inequality in Eq. (5.3) being an equality.
However, in cases in which the Lasso-based first step selects a strict subset of the moment inequalities
chosen by the SN method (i.e. the inclusion in Eq. (5.2) is strict), the inequality in Eq. (5.3) can be strict.
In fact, the inequality in Eq. (5.3) can be strict even in cases in which the Lasso-based and SN-based first
step agree on the set of binding moment inequalities. The intuition for this is that our Lasso-based method
considers the usual α-quantile while the other two-step methods consider the (α − 2βn )-quantile for the
sequence of positive constants {βn }n≥1 . This slight difference always plays in favor of the Lasso-based first
step having more power.12
The relevance of Theorem 5.1 depends on the generality of the sufficient conditions in Eq. (5.4) and
(5.5). Figure 1 provides heat maps that indicate combinations of values of Mn,2+δ and p under which Eqs.
(5.4) and (5.5) are satisfied. The graphs clearly show these conditions are satisfied for a large portion of the
parameter space. In fact, the region in which Eq. (5.4) fails to hold is barely visible. In addition, the graph
also confirms that Eq. (5.4) applies more generally than Eq. (5.5).
Remark 5.1. Notice that the power comparison in Theorem 5.1 is a finite sample result. In other words,
under any of the sufficient conditions Theorem in 5.1, the rejection of the null hypothesis by an inference
method with SN-based first step implies the same outcome for the corresponding inference method with
Lasso-based first step. Expressed in terms of confidence sets, the confidence set with our Lasso first step will
be a subset of the corresponding confidence set with a SN first step.
To conclude the section, we now compare the power of the two-step bootstrap procedures.
Theorem 5.2. Assume Assumption A.4 and let B ∈ {M B, EB}.
Part 1: For all θ ∈ Θ and n ∈ N,
JˆL (θ) ⊆ JˆB (θ)

(5.6)

12 This is clearly shown in Designs 5-6 of our Monte Carlos. In these cases, both first-step methods to agree on the correct
set of binding moment inequalities (i.e. JˆL (θ) = JˆSN (θ)). Nevertheless, the slight difference in quantiles produces small but
positive power advantage in favor of methods that use the Lasso in a first stage.

18

Figure 1: Consider a moment inequality model with n = 400, βn = 0.1%, C = 2, M = Mn,2+δ ∈ [0, 10], and
k = p ∈ {1, . . . , 1000}. The left (right) panel shows in red the configurations (p, M ) that do not satisfy Eq.
(5.4) (Eq. (5.5), respectively).

implies

P (Tn (θ) > cB,2S
(α)) ≤ P (Tn (θ) > cB,L
n
n (θ, α)).

(5.7)

Part 2: Eq. (5.6) occurs with probability approaching one, i.e.,


P JˆL (θ) ⊆ JˆB (θ) ≥ 1 − Cn−c

(5.8)

2
under the following sufficient conditions: Mn,2+δ
n2/(2+δ) ≥ 2, βn ≥ Cn−c for some C, c > 0, and any one

of the following conditions:



4
−1
δ/(2(2+δ))
(
+
ε)n
M
≥ 3βn ,
or,
n,2+δ
23/2 3
p
p
3 4
−1
(1 − ρ(θ)) log(p)/2 − 2 log(1/[1 − 3βn ]) ≥ 3/2 ( + ε)nδ/(2(2+δ)) Mn,2+δ
,
3
2
1−Φ

3

where ρ(θ) ≡ maxj1 6=j2 corr[Xj1 (θ), Xj2 (θ)].

19

(5.9)
(5.10)

Part 3: Under any of the sufficient conditions in part 2,

P Tn (θ) > cB,2S
(α)
n




−c
≤ P Tn (θ) > cB,L
n (θ, α) + Cn

(5.11)

Theorem 5.2 provides sufficient conditions under which any power advantage of the two-step bootstrap
method in CCK14 relative to our two-step bootstrap Lasso vanishes as the sample size diverges to infinity.
Specifically, Eq. (5.11) indicates that, under any of the sufficient conditions, this power advantage does not
exceed C̃n−c̃ . As in the SN approximation, this relative power difference is a direct consequence of Eq.
(5.6), i.e., our Lasso-based first step inequality selection procedure chooses a subset of the inequalities in the
bootstrap-based first step.
The relevance of the result in Theorem 5.2 depends on the generality of the sufficient condition. This
2
condition has three parts. The first part, i.e., Mn,2+δ
n2/(2+δ) ≥ 2, was already argued to be non-restrictive
2
since Mn,2+δ
is a non-decreasing sequence of positive constants and n2/(2+δ) → ∞. The second part, i.e.,

βn ≥ Cn−c is also considered mild as {βn }n≥1 is a sequence of positive constants and Cn−c converges to
zero. The third part is Eq. (5.9) or (5.10) and we deem it to be the more restrictive condition of the three.
In the case of the latter, this condition can be understood as imposing an upper bound on the maximal
pairwise correlation within the moment inequalities of the model.

6

Monte Carlo simulations

We now use Monte Carlo simulations to investigate the finite sample properties of our tests and to compare
them to those proposed by CCK14. Our simulation setup follows closely the moment inequality model
considered in their Monte Carlo simulation section. For a hypothetical fixed parameter value θ ∈ Θ, we
generate data according to the following equation:

Xi (θ) = µ(θ) + A0 i

i = 1, . . . , n = 400,

where Σ(θ) = A0 A, i = (i,1 , . . . , i,p ), and p ∈ {200, 500, 1000}. We simulate {i }ni=1 to be i.i.d. with
E[i ] = 0p and V ar[i ] = Ip×p , and so {Xi (θ)}ni=1 are i.i.d. with E[Xi (θ)] = µ(θ) and V ar[Xi (θ)] = Σ(θ).
This model satisfies the moment (in)equality model in Eq. (2.1) if and only if µ(θ) ≤ 0p . In this context, we
are interested in implementing the hypothesis test in Eqs. (2.3) (or, equivalently, Eq. (2.4)) with a significance
level of α = 5%.
We simulate i = (i,1 , . . . , i,p ) to be i.i.d. according to two distributions: (i) i,j follows a t-distribution
20

with four degrees of freedom divided by

√

√
√ √
2, i.e., i,j ∼ t4 / 2 and (ii) i,j ∼ U (− 3, 3). Note that

both of these choices satisfy E[i ] = 0p and V ar[i ] = Ip×p . Since (i,1 , . . . , i,p ) are i.i.d., the correlation
structure across moment inequalities depends entirely on Σ(θ), for which we consider two possibilities: (i)
Σ(θ)[j,k] = 1[j = k] + ρ · 1[j 6= k] and (ii) a Toeplitz structure, i.e., Σ(θ)[j,k] = ρ|j−k| with ρ ∈ {0, 0.5, 0.9}.
We repeat all experiments 2, 000 times.
The description of the model is completed by specifying µ(θ), given in Table 1. We consider ten different
specifications of µ(θ) which, in combination with the rest of the parameters, results in fourteen simulation
designs. Our first eight simulation designs correspond exactly to those in CCK14, half of which satisfy the
null hypothesis and half of which do not. We complement these simulations with six designs that do not
satisfy the null hypothesis. The additional designs are constructed so that the moment inequalities that agree
with the null hypothesis are only slightly or moderately negative.13 As the slackness of these inequalities
decreases, it becomes harder for two-step inference methods to correctly classify the non-binding moment
conditions as such. As a consequence, these new designs will help us understand which two-step inference
procedures have better ability in detecting slack moment inequalities.
Design no.
1
2
3
4
5
6
7
8
9
10
11
12
13
14

{µj (θ) : j ∈ {1, . . . , p}}
−0.8 · 1[j > 0.1p]
−0.8 · 1[j > 0.1p]
0
0
0.05
0.05
−0.75 · 1[j > 0.1p] + 0.05 · 1[j ≤ 0.1p]
−0.75 · 1[j > 0.1p] + 0.05 · 1[j ≤ 0.1p]
−0.6 · 1[j > 0.1p] + 0.05 · 1[j ≤ 0.1p]
−0.5 · 1[j > 0.1p] + 0.05 · 1[j ≤ 0.1p]
−0.4 · 1[j > 0.1p] + 0.05 · 1[j ≤ 0.1p]
−0.3 · 1[j > 0.1p] + 0.05 · 1[j ≤ 0.1p]
−0.2 · 1[j > 0.1p] + 0.05 · 1[j ≤ 0.1p]
−0.1 · 1[j > 0.1p] + 0.05 · 1[j ≤ 0.1p]

Σ(θ)
Equicorrelated
Toeplitz
Equicorrelated
Toeplitz
Equicorrelated
Toeplitz
Equicorrelated
Toeplitz
Toeplitz
Toeplitz
Toeplitz
Toeplitz
Toeplitz
Toeplitz

Hypothesis
H0
H0
H0
H0
H1
H1
H1
H1
H1
H1
H1
H1
H1
H1

CCK14 Design no.
2
4
1
3
5
7
6
8
New
New
New
New
New
New

Table 1: Parameter choices in our simulations.
We implement all the inference methods described in Table 2. These include all of the procedures
described in previous sections some additional “hybrid” methods (i.e. MB-H and EB-H). The bootstrap
based methods are implemented with B = 1, 000 bootstrap replications. Finally, for our Lasso-based first

13 For

reasons of brevity, these additional designs only consider Σ(θ) with a Toeplitz structure. We carried out the same
designs with equicorrelated Σ(θ) and obtained qualitatively similar results. These are available from the authors, upon request.

21

Method
SN Lasso
MB Lasso
EB Lasso
SN-1S
SN-2S
MB-1S
MB-H
MB-2S
EB-1S
EB-H
EB-2S

No. of steps
Two
Two
Two
One
Two
One
Two
Two
One
Two
Two

First step
Lasso
Lasso
Lasso
None
SN
None
SN
MB
None
SN
EB

Second step
SN
MB
EB
SN
SN
MB
MB
MB
EB
EB
EB

Parameters
C ∈ {2, 4, 6} in Eq. (6.1)
C ∈ {2, 4, 6} in Eq. (6.1)
C ∈ {2, 4, 6} in Eq. (6.1)
None
None
None
βn ∈ {0.01%, 0.1%, 1%}
βn ∈ {0.01%, 0.1%, 1%}
None
βn ∈ {0.01%, 0.1%, 1%}
βn ∈ {0.01%, 0.1%, 1%}

Table 2: Inference methods.
step, we use:

2
n−1/3 − n−1
λn = C · n−1/2 M̂n,3

with C ∈ {2, 4, 6} and M̂n,3 ≡ maxj=1,...,p (n−1

Pn

i=1

−1

,

(6.1)

|Xij (θ)|3 )1/3 . This corresponds to the empirical ana-

logue of Eq. (3.4) when δ = 1 and ε ∈ {2/3, 8/3, 14/3}.
We shall begin by considering the simulation designs in CCK14 as reported in Tables 3-10. The first
four tables are concerned with the finite sample size control. The general finding is that all tests under
consideration are very rarely over-sized. The maximal size observed for our procedures is 7.15 (e.g. EB Lasso
in Designs 3-4, p = 1, 000, ρ = 0, and uniform errors) while the corresponding number for CCK14 is 7.25
(e.g. EB-1S in Designs 3-4, p = 1, 000, ρ = 0, and uniform errors). Some procedures, such as SN-1S, can be
heavily under-sized. Our simulations reveal that in order to achieve empirical rejection rates close to α = 5%
under the null hypothesis, one requires using a two-step inference procedure with a bootstrap-based second
step (either MB or EB).
Before turning to the individual setups for power comparison, let us remark that a first step based on
our Lasso procedure compares favorably with a first step based on SN. For example, SN-Lasso with C = 2
has more or equal power than SN-2S with βn = 0.1%. While the differences may often be small, this finding
is in line with the power comparison in Section 5.
Tables 7-10 contain the designs used by CCK14 to gauge the power of their tests. Tables 7 and 8 consider
the case where all moment inequalities are violated. Since none of the moment conditions are slack, there is
no room for power gains based on a first-step inequality selection procedure. In this sense, it is not surprising
that the first step choice makes no difference in these designs. For example, the power of SN-Lasso is identical

22

to the one of SN-1S while the power of SN-2S is also close to the one of SN-1S. However, the SN-2S has
lower power than SN-1S for some values of βn while the power of SN Lasso appears to be invariant to the
choice of C. The latter is in accordance with our previous findings. The bootstrap still improves power for
high values of ρ.
Next, we consider Tables 9 and 10. In this setting, 90% of the moment conditions have µj (θ) = −0.75
and our results seem to suggest that this value is relative far away from being binding. We deduce this from
the fact that all first-step selection methods agree on the set of binding moment conditions, producing very
similar power results. Table 17 shows the percentage of moment inequalities retained by each of the first-step
procedures in Design 8. When the error terms are t-distributed, all first-step procedures retain around 10%
of the inequalities which is also the fraction that are truly binding (and, in this case, violated). Thus, all
two-step inference procedures are reasonably powerful. When the error terms are uniformly distributed, all
first-step procedures have an equal tendency to aggressively remove slack inequalities. However, we have
seen from the size comparisons that this does not seem to result in oversized tests. Finally, we notice that
the power of our procedures hardly varies with the choice of C.
The overall message of the simulation results in Designs 1-8 is that our Lasso-based procedures are
comparable in terms of size and power to the ones proposed by CCK14.
Tables 11-16 present simulations results for Designs 9-14. These correspond to modifications of the setup
in Design 8 in which progressively decrease the degree of slackness of the non-binding moment inequalities
from −0.75 to values between −0.6 and −0.1.
Tables 11-12 shows results for Designs 9 and 10. As in the case of Design 8, the degree of slackness of
the non-binding moment inequalities is still large enough so that it can be correctly detected by all first
first-step selection methods.
As Table 13 shows, this pattern changes in Design 11. In this case, the MB Lasso with C = 2 has a
rejection rate that is at least 20 percentage points higher than the most powerful procedure in CCK14. For
example, with t-distributed errors, p = 1, 000, and ρ = 0, our MB Lasso with C = 2 has a rejection rate
of 71.40% whereas the MB-2S with βn = 0.01% has a rejection rate of 20.55%. Table 18 holds the key to
these power differences. Ideally, a powerful procedure should retain only the 10% of the moment inequalities
that are binding (in this case, violated). The Lasso-based selection indeed often retains close to 10% of the
inequalities for C ∈ {2, 4}. On the other hand, SN-based selection can sometimes retain more than 90% of
the inequalities (e.g. see t-distributed errors, p = 1, 000, and ρ = 0).
The power advantage in favor of the Lasso-based first step is also present in Design 12 as shown in Table
14. In this case, the MB Lasso with C = 2 has a rejection rate which is at least 15 percentage points higher

23

than the most powerful procedure in CCK14. For t-distributed errors, the MB Lasso always has a rejection
rate that is at least 20 percentage points higher than its competitors and sometimes more than 50 percentage
points (e.g. p = 1, 000 and ρ = 0). As in the previous design, this power gain mainly comes from the Lasso
being better at removing the slack moment conditions.
Table 15 shows the results for Design 13. For t-distributed errors, the MB Lasso with C = 2 has a higher
rejection rate than the most powerful procedure of CCK14 (which is often MB-1S) by at least 5 percentage
points. Sometimes the difference is larger than 45 percentage points (e.g. see p = 1, 000 and ρ = 0). For
uniformly distributed errors, there seems to be no significant difference between our procedures and the ones
in CCK14; all of them have relatively low power.
Design 14 is our last experiment and it is shown in Table 16. In this case, the degree of slackness of
the non-binding moment inequalities is so small that it cannot be detected by any of the first-step selection
methods. As a consequence, there are very little differences among the various inference procedures and all
of them exhibit relatively low power.
The overall message from Tables 11-16 is that our Lasso-based inference procedures can have higher
power than those in CCK14 when the slack moment inequalities are difficult to distinguish from zero.

7

Conclusions

This paper considers the problem of inference in a partially identified moment (in)equality model with
possibly many moment inequalities. Our contribution is to propose a novel two-step inference method based
on the combination of two ideas. On the one hand, our test statistic and critical values are based on those
proposed by CCK14. On the other hand, we propose a new first step selection procedure based on the Lasso.
Our two-step inference method can be used to conduct hypothesis tests and to construct confidence sets for
the true parameter value.
Our inference method has very desirable properties. First, under reasonable conditions, it is uniformly
valid, both in underlying parameter θ and distribution of the data. Second, by virtue of results in CCK14,
our test is asymptotically optimal in a minimax sense. Third, the power of our method compares favorably
with that of the corresponding two-step method in CCK14, both in theory and in simulations. On the
theory front, we provide sufficient conditions under which the power of our method dominates. These can
sometimes represent a significant part of the parameter space. Our simulations indicate that our inference
methods are usually as powerful as the corresponding ones in CCK14, and can sometimes be more powerful.
Fourth, our Lasso-based first step is straightforward to implement.

24

25

√ √
U (− 3, 3)

200

√
t4 / 2

1000

500

200

1000

500

p

1000

500

200

1000

Density

√ √
U (− 3, 3)

200

√
t4 / 2

500

p

Density

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

ρ

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

ρ

5.00
4.65
2.25
5.05
4.55
1.65
4.50
4.70
2.40

4.10
4.40
2.30
4.45
4.35
1.75
5.05
4.55
1.90

C=2

5.00
3.10
1.10
5.05
2.60
0.75
4.50
2.15
0.45

4.10
3.05
0.90
4.45
2.75
0.55
5.05
2.10
0.55

C=2

5.00
4.65
2.25
5.05
4.55
1.65
4.50
4.70
2.40

4.00
4.40
2.30
4.35
4.25
1.70
4.85
4.15
1.85

C=4

SN Lasso

5.00
3.10
1.10
5.05
2.60
0.75
4.50
2.15
0.45

4.00
3.05
0.90
4.35
2.75
0.55
4.85
2.10
0.55

C=4

SN Lasso

5.60
5.20
4.85
5.70
5.35
5.05
5.85
4.95
5.20

5.20
5.10
5.00
5.65
5.70
5.80
5.85
4.80
4.60

C=2

5.60
5.20
4.85
5.70
5.35
5.05
5.85
4.95
5.20

5.00
5.10
5.00
5.45
5.70
5.80
5.80
4.80
4.60

C=4

5.60
5.20
4.85
5.70
5.35
5.05
5.85
4.95
5.20

5.00
5.10
5.00
5.45
5.70
5.80
5.80
4.80
4.60

C=6

5.80
5.05
5.05
6.05
5.40
4.70
5.55
5.10
5.20

4.55
4.85
5.15
5.00
5.60
5.60
5.55
4.70
4.65

C=2

5.80
5.05
5.05
6.05
5.40
4.70
5.55
5.10
5.20

4.50
4.85
5.15
4.85
5.60
5.60
5.35
4.70
4.65

C=4

EB Lasso

5.80
5.05
5.05
6.05
5.40
4.70
5.55
5.10
5.20

4.50
4.85
5.15
4.85
5.60
5.60
5.35
4.70
4.65

C=6

0.45
0.40
0.25
0.80
0.40
0.05
0.55
0.40
0.00

0.40
0.25
0.05
0.50
0.35
0.15
0.45
0.25
0.05

SN-1S

4.95
3.10
1.10
5.05
2.55
0.75
4.50
2.15
0.45

3.95
3.05
0.90
4.30
2.70
0.55
4.65
2.10
0.55

0.01%

4.85
3.05
1.10
5.05
2.45
0.75
4.40
2.00
0.40

3.85
2.95
0.85
4.30
2.70
0.55
4.65
2.10
0.55

0.10%

SN-2S

3.20
2.00
0.65
3.10
1.60
0.45
3.00
1.50
0.30

2.30
1.85
0.45
2.85
1.60
0.40
3.00
1.40
0.35

1.00%

0.55
1.15
2.80
1.10
1.10
2.95
0.70
1.40
2.75

0.55
0.85
2.80
0.65
1.30
2.75
0.55
1.15
2.75

MB-1S

5.55
5.20
4.85
5.65
5.30
5.05
5.80
4.95
5.20

4.80
5.05
4.95
5.25
5.65
5.80
5.55
4.80
4.55

0.01%

5.45
4.90
4.70
5.45
5.15
4.75
5.45
4.85
4.80

4.50
4.80
4.80
5.20
5.55
5.60
5.45
4.65
4.40

0.10%

MB-H

3.65
3.15
3.00
3.70
3.25
3.05
3.65
2.75
2.70

2.50
2.80
3.10
3.40
3.50
3.10
3.60
2.95
2.70

1.00%

5.55
5.20
4.85
5.65
5.30
5.05
5.80
4.95
5.20

4.85
5.05
4.95
5.30
5.65
5.80
5.60
4.80
4.60

0.01%

5.45
4.90
4.70
5.45
5.15
4.75
5.45
4.85
4.80

4.50
4.80
4.80
5.20
5.55
5.60
5.40
4.65
4.40

0.10%

MB-2S

CCK14’s methods

3.65
3.15
3.00
3.70
3.25
3.05
3.65
2.75
2.70

2.50
2.85
3.10
3.40
3.50
3.10
3.60
2.95
2.70

1.00%

5.00
4.65
2.25
5.05
4.55
1.65
4.50
4.70
2.40

4.00
4.40
2.30
4.35
4.25
1.70
4.80
4.10
1.85

C=6

5.60
5.55
5.25
5.70
5.70
5.20
5.85
6.15
5.55

5.00
5.20
5.20
5.45
5.45
5.20
5.80
5.60
5.20

C=4

5.60
5.55
5.25
5.70
5.70
5.20
5.85
6.15
5.55

5.00
5.20
5.20
5.45
5.45
5.20
5.80
5.60
5.20

C=6

5.80
5.45
5.15
6.05
5.30
5.00
5.55
5.90
5.30

4.55
5.05
5.00
5.00
5.15
5.30
5.55
5.50
4.95

C=2

5.80
5.45
5.15
6.05
5.30
5.00
5.55
5.90
5.30

4.50
5.00
4.95
4.85
5.10
5.30
5.35
5.40
4.95

C=4

EB Lasso

5.80
5.45
5.15
6.05
5.30
5.00
5.55
5.90
5.30

4.50
5.00
4.95
4.85
5.10
5.30
5.35
5.35
4.95

C=6

0.45
0.55
0.55
0.80
0.55
0.15
0.55
0.80
0.20

0.40
0.35
0.35
0.50
0.40
0.10
0.45
0.45
0.20

SN-1S

4.95
4.65
2.25
5.05
4.50
1.65
4.50
4.70
2.40

3.95
4.40
2.30
4.30
4.20
1.65
4.65
4.00
1.85

0.01%

4.85
4.50
2.15
5.05
4.35
1.65
4.40
4.50
2.40

3.85
4.20
2.30
4.30
4.15
1.60
4.65
3.90
1.70

0.10%

SN-2S

3.20
3.00
1.50
3.10
2.80
1.00
3.00
2.60
1.60

2.30
2.70
1.55
2.85
2.70
1.15
3.00
2.35
1.15

1.00%

0.55
0.65
0.85
1.10
0.65
0.35
0.70
0.90
0.50

0.55
0.50
0.70
0.65
0.45
0.35
0.55
0.65
0.35

MB-1S

5.55
5.50
5.15
5.65
5.65
5.15
5.80
6.10
5.50

4.80
5.05
5.10
5.25
5.35
5.10
5.55
5.45
5.15

0.01%

5.45
5.30
4.90
5.45
5.50
4.90
5.45
5.90
5.35

4.50
5.10
4.85
5.20
5.25
4.95
5.45
5.25
4.90

0.10%

MB-H

3.65
3.80
3.40
3.70
3.50
2.60
3.65
3.35
3.50

2.50
3.30
3.35
3.40
3.25
2.70
3.60
3.35
2.90

1.00%

5.55
5.50
5.15
5.65
5.65
5.15
5.80
6.10
5.50

4.85
5.10
5.15
5.30
5.40
5.15
5.60
5.55
5.15

0.01%

5.45
5.30
4.90
5.45
5.50
4.90
5.45
5.90
5.35

4.50
5.05
4.85
5.20
5.25
4.95
5.40
5.30
4.90

0.10%

MB-2S

CCK14’s methods

3.65
3.80
3.40
3.70
3.50
2.60
3.65
3.35
3.50

2.50
3.30
3.35
3.40
3.25
2.70
3.60
3.35
2.90

1.00%

Table 4: Simulation results in Design 2: µj (θ) = −0.8 · 1[j > 0.1p], Σ(θ) Toeplitz

5.60
5.55
5.25
5.70
5.70
5.20
5.85
6.15
5.55

5.20
5.30
5.20
5.65
5.60
5.20
5.85
5.80
5.25

C=2

MB Lasso

Our methods

0.55
0.80
0.80
1.05
0.60
0.35
0.80
0.95
0.40

0.50
0.45
0.65
0.45
0.45
0.35
0.45
0.60
0.25

EB-1S

0.55
1.15
2.75
1.05
1.05
2.90
0.80
1.35
2.80

0.50
0.85
2.80
0.45
1.20
2.90
0.45
1.20
2.65

EB-1S

Table 3: Simulation results in Design 1: µj (θ) = −0.8 · 1[j > 0.1p], Σ(θ) equicorrelated

5.00
3.10
1.05
5.05
2.60
0.75
4.50
2.15
0.45

4.00
3.05
0.90
4.35
2.75
0.55
4.80
2.10
0.55

C=6

MB Lasso

Our methods

5.90
5.45
5.20
6.10
5.30
5.00
5.60
5.90
5.30

4.35
4.90
4.85
4.55
4.85
5.00
4.85
4.95
4.85

0.01%

5.90
5.15
5.05
6.10
5.45
4.70
5.60
5.15
5.20

4.35
4.85
5.15
4.55
5.45
5.60
4.85
4.50
4.60

0.01%

EB-H

5.65
5.20
5.15
5.90
5.25
5.00
5.40
5.70
5.25

4.40
4.90
4.80
4.55
4.70
5.15
4.90
5.00
4.75

0.10%

EB-H

5.65
5.00
4.95
5.90
5.15
4.50
5.40
5.00
5.05

4.40
4.60
4.85
4.55
5.30
5.40
4.90
4.50
4.40

0.10%

3.70
3.85
3.50
3.85
3.35
3.00
3.65
3.60
3.50

2.65
3.05
3.35
3.10
3.00
2.60
3.40
3.05
2.85

1.00%

3.70
3.10
3.00
3.85
3.15
3.00
3.65
2.95
2.75

2.65
3.10
2.95
3.10
3.25
3.15
3.40
2.90
2.65

1.00%

5.70
5.95
5.15
5.70
5.95
4.80
5.30
6.05
5.35

4.75
5.15
5.30
4.85
4.75
5.25
4.95
4.75
4.95

0.01%

5.70
5.15
4.85
5.70
5.20
4.85
5.30
5.25
5.15

4.75
4.75
5.10
4.85
5.60
5.65
4.95
4.60
4.70

0.01%

5.45
5.85
5.05
5.50
5.75
4.65
5.20
5.65
5.10

4.55
4.95
5.20
4.75
4.55
4.90
4.80
4.60
4.85

0.10%

EB-2S

5.45
5.00
4.75
5.50
5.10
4.75
5.20
4.95
4.95

4.55
4.65
4.95
4.75
5.35
5.40
4.80
4.45
4.60

0.10%

EB-2S

3.65
3.40
3.45
3.85
3.15
2.90
3.75
3.50
3.55

2.70
3.20
3.10
3.15
3.05
2.55
3.15
2.85
2.90

1.00%

3.65
3.15
3.05
3.85
3.00
3.00
3.75
2.75
3.05

2.70
2.85
3.10
3.15
3.35
3.00
3.15
2.60
2.85

1.00%

26

√ √
U (− 3, 3)

200

√
t4 / 2

1000

500

200

1000

500

p

1000

500

200

1000

Density

√ √
U (− 3, 3)

200

√
t4 / 2

500

p

Density

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

ρ

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

ρ

3.85
4.35
2.50
4.20
3.85
2.75
5.05
5.00
2.95

3.20
3.85
2.40
3.00
3.90
2.35
3.40
3.70
2.30

C=2

3.85
1.95
0.40
4.20
1.50
0.10
5.05
1.40
0.05

3.20
1.45
0.25
3.00
1.45
0.20
3.40
1.40
0.15

C=2

3.85
4.35
2.50
4.20
3.85
2.75
5.05
5.00
2.95

3.05
3.80
2.40
2.95
3.75
2.35
3.20
3.60
2.30

C=4

SN Lasso

3.85
1.95
0.40
4.20
1.50
0.10
5.05
1.40
0.05

3.05
1.45
0.25
2.95
1.45
0.20
3.20
1.40
0.15

C=4

SN Lasso

5.20
4.65
4.95
5.60
5.05
5.10
6.95
5.90
5.10

4.35
4.60
5.20
4.45
5.65
5.45
4.65
4.25
4.40

C=2

5.20
4.65
4.95
5.60
5.05
5.10
6.95
5.90
5.10

4.30
4.60
5.20
4.25
5.65
5.45
4.40
4.25
4.40

C=4

5.20
4.65
4.95
5.60
5.05
5.10
6.95
5.90
5.10

4.30
4.60
5.20
4.25
5.65
5.45
4.40
4.25
4.40

C=6

4.90
4.55
5.20
5.65
4.70
4.50
7.15
5.55
5.05

3.65
4.15
5.30
3.50
5.05
5.55
3.75
3.85
4.15

C=2

4.90
4.55
5.20
5.65
4.70
4.50
7.15
5.55
5.05

3.55
4.15
5.30
3.50
5.05
5.55
3.70
3.85
4.15

C=4

EB Lasso

4.90
4.55
5.20
5.65
4.70
4.50
7.15
5.55
5.05

3.55
4.15
5.30
3.50
5.05
5.55
3.70
3.85
4.15

C=6

3.85
1.95
0.40
4.20
1.50
0.10
5.05
1.40
0.05

3.05
1.45
0.25
2.95
1.45
0.20
3.20
1.40
0.15

SN-1S

3.85
1.95
0.40
4.20
1.50
0.05
5.00
1.40
0.05

3.05
1.45
0.25
2.95
1.45
0.20
3.15
1.40
0.15

0.01%

3.75
1.85
0.40
4.00
1.50
0.05
4.95
1.40
0.05

2.95
1.40
0.20
2.90
1.35
0.20
3.05
1.35
0.15

0.10%

SN-2S

2.00
1.45
0.30
2.50
1.10
0.05
3.05
1.05
0.05

1.55
1.00
0.15
1.55
0.85
0.15
1.80
0.90
0.05

1.00%

5.20
4.65
4.95
5.60
5.05
5.10
6.95
5.90
5.10

4.30
4.60
5.20
4.25
5.65
5.45
4.40
4.25
4.40

MB-1S

5.15
4.65
4.95
5.45
5.05
5.10
6.95
5.85
5.10

4.25
4.60
5.15
4.20
5.65
5.45
4.40
4.25
4.40

0.01%

4.90
4.65
4.85
5.20
4.95
4.85
6.85
5.65
4.95

4.10
4.40
4.90
3.95
5.65
5.25
4.25
4.10
4.15

0.10%

MB-H

2.85
2.80
2.85
3.35
2.80
2.85
4.30
3.70
3.30

2.15
2.60
3.10
2.20
3.45
3.50
2.75
2.90
2.50

1.00%

5.15
4.65
4.95
5.45
5.05
5.10
6.95
5.85
5.10

4.25
4.60
5.15
4.20
5.65
5.45
4.40
4.25
4.40

0.01%

4.90
4.65
4.85
5.20
4.95
4.85
6.85
5.65
4.95

4.10
4.40
4.90
3.95
5.65
5.25
4.25
4.10
4.15

0.10%

MB-2S

CCK14’s methods

2.85
2.80
2.85
3.35
2.80
2.85
4.30
3.70
3.30

2.15
2.60
3.10
2.20
3.45
3.50
2.75
2.90
2.50

1.00%

4.95
4.55
5.20
5.75
4.75
4.60
7.25
5.55
5.05

3.60
4.20
5.30
3.55
5.10
5.55
3.70
3.85
4.15

EB-1S

3.85
4.35
2.50
4.20
3.85
2.75
5.05
5.00
2.95

3.05
3.80
2.40
2.95
3.75
2.35
3.20
3.60
2.30

C=6

5.20
5.20
5.40
5.60
5.25
5.35
6.95
6.75
6.40

4.30
4.85
5.90
4.25
5.45
5.20
4.40
5.20
5.15

C=4

5.20
5.20
5.40
5.60
5.25
5.35
6.95
6.75
6.40

4.30
4.85
5.90
4.25
5.45
5.20
4.40
5.20
5.15

C=6

4.90
5.25
5.30
5.65
5.15
5.55
7.15
6.90
6.30

3.65
4.50
5.55
3.50
5.00
4.60
3.75
4.70
5.00

C=2

4.90
5.25
5.30
5.65
5.15
5.55
7.15
6.90
6.30

3.55
4.45
5.55
3.50
4.85
4.55
3.70
4.65
4.85

C=4

EB Lasso

4.90
5.25
5.30
5.65
5.15
5.55
7.15
6.90
6.30

3.55
4.45
5.55
3.50
4.85
4.55
3.70
4.65
4.85

C=6

3.85
4.35
2.50
4.20
3.85
2.75
5.05
5.00
2.95

3.05
3.80
2.40
2.95
3.75
2.35
3.20
3.60
2.30

SN-1S

3.85
4.35
2.50
4.20
3.85
2.75
5.00
5.00
2.95

3.05
3.80
2.40
2.95
3.75
2.35
3.15
3.60
2.30

0.01%

3.75
4.05
2.40
4.00
3.75
2.70
4.95
4.90
2.95

2.95
3.80
2.35
2.90
3.45
2.20
3.05
3.40
2.30

0.10%

SN-2S

2.00
2.95
1.95
2.50
2.55
1.50
3.05
3.05
1.85

1.55
2.45
1.75
1.55
1.95
1.45
1.80
1.95
1.55

1.00%

5.20
5.20
5.40
5.60
5.25
5.35
6.95
6.75
6.40

4.30
4.85
5.90
4.25
5.45
5.20
4.40
5.15
5.15

MB-1S

5.15
5.15
5.40
5.45
5.15
5.35
6.95
6.75
6.40

4.25
4.85
5.90
4.20
5.45
5.15
4.40
5.15
5.10

0.01%

4.90
5.05
5.15
5.20
5.05
5.25
6.85
6.70
6.20

4.10
4.55
5.80
3.95
5.30
5.00
4.25
4.95
5.05

0.10%

MB-H

2.85
3.35
3.85
3.35
3.60
3.30
4.30
4.55
3.95

2.15
3.05
3.60
2.20
3.05
2.85
2.75
3.15
3.20

1.00%

5.15
5.15
5.40
5.45
5.15
5.35
6.95
6.75
6.40

4.25
4.85
5.90
4.20
5.45
5.15
4.40
5.15
5.10

0.01%

4.90
5.05
5.15
5.20
5.05
5.25
6.85
6.70
6.20

4.10
4.55
5.80
3.95
5.30
5.00
4.25
4.95
5.05

0.10%

MB-2S

CCK14’s methods

2.85
3.35
3.85
3.35
3.60
3.30
4.30
4.55
3.95

2.15
3.05
3.60
2.20
3.05
2.85
2.75
3.15
3.20

1.00%

Table 6: Simulation results in Design 4: µj (θ) = 0 for all j = 1, . . . , p, Σ(θ) Toeplitz.

5.20
5.20
5.40
5.60
5.25
5.35
6.95
6.75
6.40

4.35
4.90
5.90
4.45
5.70
5.20
4.65
5.30
5.20

C=2

MB Lasso

Our methods

4.95
5.40
5.40
5.75
5.20
5.70
7.25
6.95
6.35

3.60
4.55
5.55
3.55
4.95
4.55
3.70
4.70
4.95

EB-1S

Table 5: Simulation results in Design 3: µj (θ) = 0 for all j = 1, . . . , p, Σ(θ) equicorrelated.

3.85
1.95
0.40
4.20
1.50
0.10
5.05
1.40
0.05

3.05
1.45
0.25
2.95
1.45
0.20
3.20
1.40
0.15

C=6

MB Lasso

Our methods

4.90
5.35
5.35
5.70
5.20
5.70
7.20
6.90
6.35

3.60
4.55
5.55
3.55
4.95
4.55
3.70
4.70
4.95

0.01%

4.90
4.55
5.20
5.70
4.70
4.55
7.20
5.55
5.05

3.60
4.20
5.30
3.55
5.10
5.55
3.70
3.85
4.15

0.01%

EB-H

4.70
5.10
5.25
5.55
5.05
5.50
7.15
6.75
6.25

3.50
4.20
5.45
3.50
4.55
4.45
3.60
4.55
4.65

0.10%

EB-H

4.70
4.45
5.05
5.55
4.65
4.35
7.15
5.45
4.85

3.50
4.05
4.80
3.50
5.00
5.35
3.60
3.75
4.00

0.10%

2.95
3.50
3.70
3.60
3.35
3.50
4.20
4.50
4.15

2.00
2.85
3.50
1.90
2.45
2.90
1.95
2.80
2.90

1.00%

2.95
2.75
2.90
3.60
2.45
2.80
4.20
3.75
3.40

2.00
2.55
3.00
1.90
3.15
2.95
1.95
2.55
2.50

1.00%

5.30
5.30
5.35
5.75
5.20
5.30
7.20
6.90
6.40

3.45
4.55
5.65
3.70
4.50
4.70
3.90
4.35
4.90

0.01%

5.30
4.70
4.85
5.75
5.00
4.60
7.20
5.85
5.60

3.45
4.35
5.15
3.70
5.10
5.35
3.90
4.20
4.35

0.01%

5.10
5.10
5.20
5.55
5.10
5.25
6.95
6.85
6.20

3.35
4.45
5.40
3.50
4.40
4.65
3.85
4.25
4.80

0.10%

EB-2S

5.10
4.45
4.65
5.55
4.75
4.50
6.95
5.65
5.30

3.35
4.15
4.95
3.50
5.00
5.25
3.85
4.05
4.10

0.10%

EB-2S

2.80
3.45
3.65
3.35
3.25
3.45
4.50
4.55
4.25

1.95
2.95
3.20
2.20
2.55
2.65
2.25
2.80
2.80

1.00%

2.80
2.75
2.90
3.35
2.65
2.65
4.50
3.90
3.45

1.95
2.45
2.95
2.20
3.15
3.00
2.25
2.75
2.60

1.00%

27

√ √
U (− 3, 3)

200

√
t4 / 2

1000

500

200

1000

500

p

1000

500

200

1000

Density

√ √
U (− 3, 3)

200

√
t4 / 2

500

p

Density

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

ρ

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

ρ

70.00
65.25
34.10
77.30
74.60
42.85
83.40
80.50
53.30

70.30
64.10
33.70
77.50
72.90
43.50
81.70
79.45
51.25

C=2

70.00
22.50
4.20
77.30
20.50
2.70
83.40
20.55
2.30

70.30
22.15
4.40
77.50
20.95
3.35
81.70
17.50
2.05

C=2

70.00
65.25
34.10
77.30
74.60
42.85
83.40
80.50
53.30

70.25
64.05
33.70
77.45
72.80
43.50
81.65
79.40
51.25

C=4

70.00
22.50
4.20
77.30
20.50
2.70
83.40
20.55
2.30

70.25
22.15
4.40
77.45
20.95
3.35
81.65
17.50
2.05

C=6

76.50
38.20
27.30
83.70
40.20
28.70
90.00
40.65
28.25

75.70
37.75
27.80
84.10
38.35
28.30
89.15
36.65
26.20

C=2

76.50
38.20
27.30
83.70
40.20
28.70
90.00
40.65
28.25

75.70
37.75
27.80
84.05
38.35
28.30
89.10
36.65
26.20

C=4

MB Lasso

76.50
38.20
27.30
83.70
40.20
28.70
90.00
40.65
28.25

75.70
37.75
27.80
84.05
38.35
28.30
89.10
36.65
26.20

C=6

Our methods

76.30
37.95
27.80
84.45
39.70
27.95
89.95
40.10
28.00

73.15
36.75
27.20
80.60
37.05
28.10
84.65
34.85
26.00

C=2

76.30
37.95
27.80
84.45
39.70
27.95
89.95
40.10
28.00

73.15
36.75
27.20
80.55
37.05
28.10
84.50
34.85
26.00

C=4

EB Lasso

76.30
37.95
27.80
84.45
39.70
27.95
89.95
40.10
28.00

73.15
36.75
27.20
80.55
37.05
28.10
84.50
34.85
26.00

C=6

70.00
22.50
4.20
77.30
20.50
2.70
83.40
20.55
2.30

70.25
22.15
4.40
77.45
20.95
3.35
81.65
17.50
2.05

SN-1S

69.80
22.50
4.20
77.25
20.45
2.70
83.35
20.55
2.30

70.10
22.10
4.40
77.40
20.90
3.35
81.55
17.45
2.05

0.01%

68.90
21.75
4.05
76.25
19.90
2.70
82.05
20.45
2.20

68.95
21.60
4.25
76.10
20.75
3.25
80.45
17.20
2.05

0.10%

SN-2S

55.90
17.30
2.90
61.90
16.00
2.15
68.50
16.60
1.55

53.85
16.55
2.80
62.00
15.70
2.10
66.30
13.90
1.65

1.00%

76.50
38.20
27.30
83.70
40.20
28.70
90.00
40.65
28.25

75.70
37.75
27.80
84.05
38.35
28.30
89.10
36.65
26.20

MB-1S

76.40
38.05
27.15
83.65
40.10
28.70
89.85
40.55
28.25

75.60
37.70
27.75
83.90
38.25
28.30
89.00
36.55
26.15

0.01%

75.80
37.40
26.65
82.80
39.45
28.05
89.20
39.70
28.00

74.55
37.25
27.00
83.30
37.85
27.45
88.05
35.70
25.70

0.10%

MB-H

62.10
29.00
20.25
71.25
29.95
20.65
79.20
30.85
20.80

61.55
28.50
20.20
70.90
30.05
21.00
77.75
28.30
19.05

1.00%

76.40
38.05
27.15
83.65
40.10
28.70
89.85
40.55
28.25

75.60
37.70
27.75
83.90
38.25
28.30
89.00
36.55
26.15

0.01%

75.80
37.40
26.65
82.80
39.45
28.05
89.20
39.70
28.00

74.55
37.25
27.00
83.30
37.85
27.45
88.05
35.70
25.70

0.10%

MB-2S

CCK14’s methods

62.10
29.00
20.25
71.25
29.95
20.65
79.20
30.85
20.80

61.55
28.50
20.20
70.90
30.05
21.00
77.75
28.30
19.05

1.00%

76.50
72.65
53.45
83.70
83.55
64.80
90.00
88.70
75.35

75.70
71.15
53.05
84.10
81.65
65.40
89.15
87.75
73.30

C=2

76.50
72.65
53.45
83.70
83.55
64.80
90.00
88.70
75.35

75.70
71.15
53.05
84.05
81.55
65.40
89.10
87.75
73.25

C=4

76.50
72.65
53.45
83.70
83.55
64.80
90.00
88.70
75.35

75.70
71.15
53.05
84.05
81.55
65.40
89.10
87.75
73.25

C=6

76.30
73.00
53.90
84.45
83.10
65.70
89.95
89.15
74.65

73.15
69.05
52.35
80.60
78.35
64.50
84.65
83.65
71.55

C=2

76.30
73.00
53.90
84.45
83.10
65.70
89.95
89.15
74.65

73.15
69.00
52.35
80.55
78.30
64.50
84.50
83.65
71.50

C=4

EB Lasso

76.30
73.00
53.90
84.45
83.10
65.70
89.95
89.15
74.65

73.15
69.00
52.35
80.55
78.30
64.50
84.50
83.65
71.50

C=6

70.00
65.25
34.10
77.30
74.60
42.85
83.40
80.50
53.30

70.25
64.05
33.70
77.45
72.75
43.50
81.65
79.40
51.25

SN-1S

69.80
65.15
34.10
77.25
74.55
42.80
83.35
80.35
53.20

70.10
64.00
33.65
77.40
72.75
43.45
81.55
79.35
51.25

0.01%

68.90
63.50
33.30
76.25
73.60
41.70
82.05
79.55
52.40

68.95
62.60
33.00
76.10
71.75
42.10
80.45
78.55
50.25

0.10%

SN-2S

55.90
51.85
25.70
61.90
59.85
32.90
68.50
66.55
42.30

53.85
49.40
25.15
62.00
58.20
31.60
66.30
65.15
39.00

1.00%

76.50
72.65
53.45
83.70
83.55
64.80
90.00
88.70
75.35

75.70
71.15
53.05
84.05
81.55
65.40
89.10
87.75
73.25

MB-1S

76.40
72.50
53.20
83.65
83.45
64.80
89.85
88.70
75.10

75.60
71.00
53.05
83.90
81.45
65.30
89.00
87.65
73.20

0.01%

75.80
71.55
52.00
82.80
83.10
64.15
89.20
87.90
74.25

74.55
70.10
52.35
83.30
80.55
64.15
88.05
86.50
72.50

0.10%

MB-H

62.10
59.20
41.85
71.25
69.75
53.10
79.20
77.00
62.50

61.55
57.85
41.35
70.90
67.40
52.10
77.75
76.15
59.35

1.00%

76.40
72.50
53.20
83.65
83.45
64.80
89.85
88.70
75.10

75.60
71.00
53.05
83.90
81.45
65.30
89.00
87.65
73.20

0.01%

75.80
71.55
52.00
82.80
83.10
64.15
89.20
87.90
74.25

74.55
70.10
52.35
83.30
80.55
64.15
88.05
86.50
72.50

0.10%

MB-2S

CCK14’s methods

62.10
59.20
41.85
71.25
69.75
53.10
79.20
77.00
62.50

61.55
57.85
41.35
70.90
67.40
52.10
77.75
76.15
59.35

1.00%

Table 8: Simulation results in Design 6: µj (θ) = 0.05 for all j = 1, . . . p, Σ(θ) Toeplitz.

70.00
65.25
34.10
77.30
74.60
42.85
83.40
80.50
53.30

70.25
64.05
33.70
77.45
72.80
43.50
81.65
79.40
51.25

C=6

MB Lasso

Our methods

Table 7: Simulation results in Design 5: µj (θ) = 0.05 for all j = 1, . . . p, Σ(θ) equicorrelated.

SN Lasso

70.00
22.50
4.20
77.30
20.50
2.70
83.40
20.55
2.30

70.25
22.15
4.40
77.45
20.95
3.35
81.65
17.50
2.05

C=4

SN Lasso

76.65
73.50
54.25
84.75
83.45
66.05
90.35
89.25
75.30

73.55
69.30
52.60
80.75
78.60
64.75
84.95
84.15
72.10

EB-1S

76.65
38.20
27.80
84.75
40.00
28.10
90.35
40.15
28.15

73.55
37.00
27.30
80.75
37.30
28.15
84.95
35.05
26.20

EB-1S

76.60
73.35
54.10
84.75
83.40
66.05
90.25
89.20
75.20

73.50
69.20
52.40
80.65
78.50
64.65
84.95
84.05
72.00

0.01%

76.60
38.10
27.75
84.75
39.75
28.05
90.25
40.10
28.10

73.50
36.80
27.25
80.65
37.25
28.15
84.95
34.95
26.15

0.01%

EB-H

75.60
72.20
53.15
84.00
82.55
65.25
89.85
88.30
74.05

72.50
68.35
51.40
79.85
77.60
63.70
84.05
83.40
71.05

0.10%

EB-H

75.60
37.25
27.30
84.00
39.20
27.40
89.85
39.50
27.65

72.50
36.10
26.55
79.85
36.50
27.30
84.05
34.40
25.15

0.10%

63.55
59.95
41.85
72.30
70.85
53.30
79.70
77.25
62.65

58.05
54.00
40.70
66.05
64.80
51.25
70.60
71.30
58.00

1.00%

63.55
29.25
20.15
72.30
30.05
20.55
79.70
31.65
20.45

58.05
28.15
20.05
66.05
28.45
20.55
70.60
26.55
18.45

1.00%

76.35
73.10
53.80
84.25
83.60
64.90
90.05
88.85
75.95

73.10
69.20
52.35
80.60
78.35
64.75
84.95
84.75
71.95

0.01%

76.35
38.15
27.65
84.25
39.55
28.00
90.05
39.80
28.55

73.10
37.00
27.00
80.60
36.95
28.15
84.95
35.30
25.70

0.01%

75.50
72.05
52.40
83.40
82.60
64.20
89.35
88.05
74.90

71.90
68.10
51.35
79.90
77.25
63.45
84.10
84.00
70.70

0.10%

EB-2S

75.50
37.35
27.20
83.40
38.65
27.50
89.35
39.40
27.75

71.90
36.25
26.35
79.90
36.70
27.60
84.10
34.45
24.90

0.10%

EB-2S

62.90
59.65
41.35
72.00
70.40
53.40
80.05
78.15
62.75

58.60
54.85
40.00
65.90
64.25
50.95
70.75
71.35
58.35

1.00%

62.90
29.25
19.95
72.00
30.15
20.70
80.05
31.45
20.40

58.60
27.95
20.20
65.90
28.90
20.20
70.75
26.75
18.15

1.00%

28

√ √
U (− 3, 3)

200

√
t4 / 2

1000

500

200

1000

500

p

1000

500

200

1000

Density

√ √
U (− 3, 3)

200

√
t4 / 2

500

p

Density

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

ρ

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

ρ

50.75
26.75
9.60
59.10
26.15
7.10
64.55
24.90
5.75

52.15
27.15
10.80
59.20
25.35
7.75
66.10
22.75
5.00

C=4

50.75
26.75
9.60
59.05
26.10
7.10
64.55
24.85
5.70

52.15
27.15
10.80
59.20
25.35
7.75
66.10
22.75
5.00

C=6

54.05
35.15
27.85
63.25
36.80
28.00
70.15
37.50
27.65

55.45
35.35
28.25
64.70
36.65
28.30
71.40
36.05
26.90

C=2

54.05
35.15
27.85
63.25
36.80
28.00
70.15
37.50
27.65

55.40
35.35
28.25
64.65
36.65
28.30
71.25
36.05
26.90

C=4

MB Lasso

53.90
35.10
27.85
63.25
36.80
28.00
70.10
37.50
27.65

55.40
35.35
28.25
64.65
36.65
28.30
71.25
36.05
26.90

C=6

Our methods

53.30
35.25
27.80
64.45
36.55
27.75
69.65
37.40
27.55

54.20
34.55
28.25
62.70
36.05
28.20
69.10
35.45
26.35

C=2

53.30
35.25
27.80
64.45
36.55
27.75
69.65
37.40
27.55

54.20
34.55
28.25
62.60
36.05
28.20
69.00
35.45
26.35

C=4

EB Lasso

53.30
35.25
27.80
64.35
36.55
27.75
69.60
37.40
27.55

54.20
34.55
28.25
62.60
36.05
28.20
68.95
35.45
26.35

C=6

11.85
7.10
2.30
14.00
7.25
1.80
15.75
6.75
1.15

11.80
7.10
2.20
14.15
7.70
1.85
16.15
6.25
1.40

SN-1S

50.70
26.70
9.60
59.05
26.00
7.05
64.45
24.75
5.75

51.15
26.90
10.75
58.05
25.05
7.75
64.00
22.50
5.00

0.01%

49.85
26.25
9.30
58.00
25.40
6.95
63.45
24.35
5.70

50.55
26.35
10.60
57.85
24.75
7.50
63.80
22.35
4.95

0.10%

SN-2S

38.80
20.90
6.95
45.80
19.60
5.00
48.40
19.20
3.65

39.50
20.90
7.85
45.20
19.75
6.05
50.90
18.00
3.60

1.00%

13.80
14.35
19.20
16.80
16.55
20.35
20.05
17.45
20.60

13.75
15.15
19.15
17.95
16.00
20.20
19.70
17.00
19.05

MB-1S

54.05
35.15
27.70
63.05
36.80
27.90
70.05
37.45
27.60

54.65
35.15
28.25
63.30
36.15
28.25
69.40
35.70
26.85

0.01%

52.75
34.65
27.25
62.25
36.05
27.25
68.95
36.70
26.95

54.10
34.10
27.75
62.90
35.65
27.75
69.35
34.95
26.15

0.10%

MB-H

41.80
27.25
20.40
50.50
29.40
21.15
54.15
29.40
20.95

42.85
27.20
20.85
50.50
28.35
21.10
56.25
27.60
19.40

1.00%

54.05
35.15
27.70
63.05
36.80
27.90
70.05
37.45
27.60

55.20
35.25
28.25
63.70
36.30
28.25
70.30
35.90
26.85

0.01%

52.75
34.65
27.25
62.25
36.05
27.25
68.95
36.70
26.95

54.15
34.10
27.75
62.95
35.60
27.75
69.45
35.00
26.15

0.10%

MB-2S

CCK14’s methods

41.80
27.25
20.40
50.50
29.40
21.15
54.15
29.40
20.95

42.85
27.20
20.85
50.50
28.35
21.10
56.25
27.60
19.40

1.00%

14.20
15.05
18.70
17.50
16.45
20.40
20.40
17.70
20.20

12.80
14.80
19.20
15.70
15.05
19.60
17.95
15.55
18.30

EB-1S

53.55
35.30
27.80
64.50
36.70
27.80
70.25
37.40
27.65

52.90
34.50
28.35
59.80
35.45
28.10
64.05
34.30
26.35

0.01%

50.75
44.80
19.50
59.10
52.75
25.70
64.55
58.65
30.65

52.15
43.55
18.85
59.20
53.45
25.75
66.10
59.35
30.75

C=4

50.75
44.70
19.45
59.05
52.70
25.70
64.55
58.65
30.65

52.15
43.55
18.85
59.20
53.45
25.75
66.10
59.35
30.75

C=6

54.05
49.75
34.65
63.25
58.00
40.65
70.15
65.75
46.55

55.45
48.40
33.95
64.70
59.40
41.60
71.40
65.75
46.85

C=2

54.05
49.75
34.65
63.25
58.00
40.65
70.15
65.75
46.55

55.40
48.40
33.95
64.65
59.30
41.60
71.25
65.65
46.85

C=4

MB Lasso

53.90
49.70
34.60
63.25
57.95
40.60
70.10
65.70
46.55

55.40
48.40
33.90
64.65
59.30
41.60
71.25
65.60
46.85

C=6

Our methods

53.30
49.65
34.20
64.45
57.50
39.90
69.65
65.30
45.85

54.20
47.25
34.15
62.70
57.55
41.25
69.10
64.10
46.25

C=2

53.30
49.65
34.20
64.45
57.50
39.90
69.65
65.30
45.85

54.20
47.20
34.15
62.60
57.45
41.25
69.00
64.05
46.25

C=4

EB Lasso

53.30
49.65
34.20
64.35
57.45
39.90
69.60
65.30
45.75

54.20
47.20
34.15
62.60
57.45
41.25
68.95
64.00
46.25

C=6

11.85
9.95
4.75
14.00
13.40
5.65
15.75
14.75
6.80

11.80
10.15
4.85
14.15
12.45
6.00
16.15
15.40
6.90

SN-1S

50.70
44.75
19.50
59.05
52.70
25.65
64.45
58.55
30.60

51.15
42.95
18.70
58.05
52.65
25.70
64.00
57.65
30.50

0.01%

49.85
43.80
19.25
58.00
51.60
25.00
63.45
57.45
29.95

50.55
42.15
18.50
57.85
51.95
25.25
63.80
57.45
30.00

0.10%

SN-2S

38.80
33.65
14.90
45.80
40.15
19.25
48.40
44.25
23.35

39.50
32.60
14.25
45.20
41.55
18.50
50.90
46.45
23.55

1.00%

13.80
11.95
8.85
16.80
16.40
11.05
20.05
18.55
13.40

13.75
12.50
8.45
17.95
15.55
11.20
19.70
19.80
13.60

MB-1S

54.05
49.70
34.50
63.05
57.85
40.60
70.05
65.65
46.35

54.65
48.10
33.50
63.30
58.05
41.00
69.40
64.40
46.25

0.01%

52.75
48.70
34.00
62.25
57.05
40.15
68.95
64.70
45.55

54.10
47.20
32.90
62.90
57.80
40.80
69.35
64.25
45.75

0.10%

MB-H

41.80
37.75
24.55
50.50
44.85
31.75
54.15
51.55
36.55

42.85
35.80
24.45
50.50
46.60
31.50
56.25
52.40
36.75

1.00%

54.05
49.70
34.50
63.05
57.85
40.60
70.05
65.65
46.35

55.20
48.10
33.65
63.70
58.65
41.25
70.30
65.10
46.70

0.01%

52.75
48.70
34.00
62.25
57.05
40.15
68.95
64.70
45.55

54.15
47.30
32.90
62.95
57.75
40.80
69.45
64.40
45.90

0.10%

MB-2S

CCK14’s methods

41.80
37.75
24.55
50.50
44.85
31.75
54.15
51.55
36.55

42.85
35.85
24.45
50.50
46.65
31.55
56.25
52.55
36.75

1.00%

14.20
12.25
9.15
17.50
16.65
11.55
20.40
19.20
13.05

12.80
11.50
8.60
15.70
14.60
10.75
17.95
18.00
13.50

EB-1S

Table 10: Simulation results in Design 8: µj (θ) = −0.75 · 1[j > 0.1p] + 0.05 · 1[j ≤ 0.1p], Σ(θ) Toeplitz.

50.75
44.80
19.50
59.10
52.75
25.70
64.55
58.65
30.65

52.15
43.60
18.85
59.40
53.45
25.75
66.20
59.60
30.80

C=2

SN Lasso

53.55
49.75
34.35
64.50
57.80
39.90
70.25
65.60
46.00

52.90
46.25
33.75
59.80
55.65
40.45
64.05
59.80
45.75

0.01%

Table 9: Simulation results in Design 7: µj (θ) = −0.75 · 1[j > 0.1p] + 0.05 · 1[j ≤ 0.1p], Σ(θ) equicorrelated.

50.75
26.75
9.60
59.10
26.15
7.10
64.55
24.90
5.75

52.15
27.15
10.80
59.40
25.35
7.75
66.20
22.75
5.00

C=2

SN Lasso

EB-H

52.65
49.15
33.75
63.00
56.80
39.30
69.00
64.05
45.05

52.70
46.00
33.10
60.35
55.40
40.15
65.20
60.55
45.40

0.10%

EB-H

52.65
34.50
27.30
63.00
35.90
27.15
69.00
36.90
26.85

52.70
34.10
27.60
60.35
34.95
27.65
65.20
34.15
25.65

0.10%

42.20
38.25
25.15
50.15
45.15
32.05
55.00
51.35
36.40

41.55
35.90
24.40
48.15
44.45
31.10
53.00
49.45
37.05

1.00%

42.20
26.70
20.10
50.15
28.80
20.75
55.00
29.60
20.00

41.55
26.65
20.70
48.15
27.50
20.70
53.00
26.50
19.40

1.00%

54.05
49.65
34.20
64.15
57.80
40.25
69.85
65.40
46.55

53.00
47.35
32.95
60.85
56.10
41.50
65.25
61.80
45.20

0.01%

54.05
34.95
27.95
64.15
36.65
28.25
69.85
37.10
27.60

53.00
34.85
28.10
60.85
35.80
28.15
65.25
34.65
26.25

0.01%

53.10
49.00
33.45
63.40
57.15
39.95
69.15
64.20
45.90

52.05
46.55
32.45
60.00
55.40
40.80
64.75
61.05
45.05

0.10%

EB-2S

53.10
34.40
27.40
63.40
35.65
27.40
69.15
36.25
27.00

52.05
34.30
27.80
60.00
35.30
27.65
64.75
34.05
25.80

0.10%

EB-2S

41.85
37.70
25.50
50.40
45.70
31.95
55.40
52.45
35.90

42.05
35.50
24.30
47.40
44.55
31.05
52.50
49.90
36.30

1.00%

41.85
27.25
20.10
50.40
28.55
20.75
55.40
29.80
20.35

42.05
26.45
20.55
47.40
27.55
21.05
52.50
26.55
19.30

1.00%

29

√ √
U (− 3, 3)

200

√
t4 / 2

1000

500

200

1000

500

p

1000

500

200

1000

Density

√ √
U (− 3, 3)

200

√
t4 / 2

500

p

Density

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

ρ

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

ρ

15.45
14.30
7.10
18.90
18.75
9.25
21.95
21.55
11.90

50.65
41.75
14.95
58.80
53.10
23.80
65.90
59.20
30.40

C=6

54.05
49.75
34.65
63.25
58.00
40.65
70.15
65.75
46.55

55.45
48.40
33.95
64.70
59.40
41.60
71.40
65.75
46.85

C=2

54.00
49.75
34.65
63.25
57.95
40.65
70.15
65.70
46.55

55.40
48.40
33.95
64.65
59.30
41.60
71.25
65.65
46.85

C=4

17.85
16.65
12.50
23.10
21.80
15.55
26.70
26.25
19.85

53.90
46.75
26.40
64.40
58.80
38.45
71.15
65.50
45.20

C=6

53.30
49.65
34.20
64.45
57.50
39.90
69.65
65.30
45.85

54.20
47.25
34.15
62.70
57.55
41.25
69.10
64.10
46.25

C=2

53.30
49.65
34.20
64.45
57.50
39.90
69.60
65.30
45.85

54.20
47.20
34.15
62.60
57.45
41.25
69.00
64.05
46.25

C=4

EB Lasso

18.05
16.95
12.55
23.55
22.25
15.50
27.55
26.80
19.60

52.45
44.80
25.95
61.90
56.70
37.50
68.30
63.45
44.45

C=6

11.85
9.95
4.75
14.00
13.40
5.65
15.75
14.75
6.80

11.80
10.15
4.85
14.15
12.45
6.00
16.15
15.40
6.90

SN-1S

45.35
39.05
17.35
46.40
40.65
19.15
43.60
39.90
20.85

40.50
33.55
16.15
40.65
38.15
18.05
41.80
38.00
20.30

0.01%

49.10
43.50
19.20
56.60
50.05
24.45
58.95
54.00
28.40

47.80
39.15
17.90
52.00
47.15
23.55
54.35
49.95
27.30

0.10%

SN-2S

38.80
33.65
14.90
45.75
40.15
19.25
48.25
44.05
23.25

38.85
32.30
14.20
44.35
40.35
18.40
48.25
44.70
23.30

1.00%

13.80
11.95
8.85
16.80
16.40
11.05
20.05
18.55
13.40

13.75
12.50
8.45
17.95
15.55
11.20
19.70
19.80
13.60

MB-1S

48.45
43.60
28.40
51.05
45.45
29.80
49.60
46.25
31.60

43.20
37.50
25.70
46.65
43.10
28.15
48.15
44.85
30.25

0.01%

52.30
48.15
33.40
60.85
55.90
38.65
65.40
60.80
41.60

50.90
44.50
31.75
56.65
52.80
36.40
59.90
57.20
39.95

0.10%

MB-H

48.25
44.15
30.30
58.25
53.10
36.85
64.00
58.85
41.25

46.55
40.85
29.35
51.10
48.45
35.20
54.10
51.15
39.35

1.00%

53.20
48.40
33.60
61.00
55.70
38.75
65.25
61.55
43.25

51.10
44.55
32.10
56.90
52.75
38.05
61.80
58.55
41.25

0.01%

52.45
48.35
33.65
61.30
55.90
38.95
66.40
62.15
43.55

51.15
44.75
32.10
57.85
53.50
38.45
62.70
59.35
41.90

0.10%

MB-2S

CCK14’s methods

41.80
37.75
24.55
50.50
44.85
31.75
54.00
51.55
36.40

42.60
35.55
24.45
49.60
46.05
31.45
54.90
51.40
36.35

1.00%

14.20
12.25
9.15
17.50
16.65
11.55
20.40
19.20
13.05

12.80
11.50
8.60
15.70
14.60
10.75
17.95
18.00
13.50

EB-1S

37.65
33.60
15.90
44.90
41.10
20.45
49.05
46.95
25.85

51.95
43.45
18.60
59.10
53.45
25.65
66.05
59.35
30.75

C=4

SN Lasso

11.85
10.15
4.80
14.00
13.40
5.80
15.75
14.80
6.90

42.45
32.85
8.10
56.85
49.35
15.75
64.85
57.65
23.75

C=6

54.05
49.75
34.65
63.25
58.00
40.65
70.15
65.75
46.55

55.45
48.40
33.95
64.70
59.40
41.60
71.40
65.75
46.85

C=2

41.10
37.50
25.50
49.70
46.10
32.15
55.40
54.00
38.40

55.25
48.30
32.80
64.60
59.30
41.50
71.20
65.55
46.80

C=4

MB Lasso

13.85
12.00
8.90
16.85
16.50
11.10
20.15
18.60
13.50

45.45
36.90
14.45
61.70
54.70
25.85
70.15
64.15
36.80

C=6

Our methods

53.30
49.65
34.20
64.45
57.50
39.90
69.65
65.30
45.85

54.20
47.25
34.15
62.70
57.55
41.25
69.10
64.10
46.25

C=2

41.00
38.00
25.60
50.30
46.85
31.85
55.70
53.70
38.15

53.90
47.10
32.80
62.50
57.25
40.75
68.85
63.85
45.95

C=4

EB Lasso

14.05
12.30
9.05
17.30
16.55
11.50
19.85
18.95
12.95

42.20
34.55
13.85
57.60
51.20
24.65
65.40
60.30
35.50

C=6

11.85
9.95
4.75
14.00
13.40
5.65
15.75
14.75
6.80

11.80
10.15
4.85
14.15
12.45
6.00
16.15
15.40
6.90

SN-1S

16.85
14.85
7.00
17.20
16.90
7.60
18.65
17.50
8.15

17.75
15.15
6.70
19.50
16.65
7.50
19.40
19.05
8.55

0.01%

29.40
25.25
11.40
26.60
23.55
11.75
25.60
23.70
11.95

27.65
23.95
10.60
27.05
24.90
11.55
25.85
25.45
12.50

0.10%

SN-2S

35.35
30.75
13.70
35.75
31.75
15.15
33.20
31.10
15.10

31.40
27.45
12.55
31.55
29.60
13.90
30.40
30.15
15.40

1.00%

13.80
11.95
8.85
16.80
16.40
11.05
20.05
18.55
13.40

13.75
12.50
8.45
17.95
15.55
11.20
19.70
19.80
13.60

MB-1S

19.30
17.40
12.00
21.50
20.05
12.90
23.30
21.50
15.00

20.80
18.40
11.55
22.95
20.80
13.25
23.80
23.60
15.40

0.01%

32.20
27.90
18.10
31.50
28.25
18.65
30.50
28.45
19.80

31.05
27.10
17.15
30.95
29.35
18.05
31.80
30.90
20.40

0.10%

MB-H

41.85
37.20
24.10
41.15
37.90
23.70
38.90
36.65
25.25

34.25
31.10
20.85
32.40
31.30
22.00
30.80
31.15
23.35

1.00%

33.45
29.80
20.85
33.85
30.85
22.05
33.25
31.80
23.00

32.50
28.30
19.40
33.40
31.60
20.80
34.80
32.60
22.95

0.01%

35.85
32.10
22.15
35.95
33.10
23.55
35.10
32.30
24.20

34.40
29.90
21.05
35.20
33.30
21.90
36.85
33.90
23.80

0.10%

MB-2S

CCK14’s methods

39.20
35.00
23.35
42.95
38.65
27.70
42.70
39.60
29.50

36.50
31.35
22.65
39.05
36.75
26.50
40.45
37.55
28.60

1.00%

14.20
12.25
9.15
17.50
16.65
11.55
20.40
19.20
13.05

12.80
11.50
8.60
15.70
14.60
10.75
17.95
18.00
13.50

EB-1S

Table 11: Simulation results in Design 9: µj (θ) = −0.6 · 1[j > 0.1p] + 0.05 · 1[j ≤ 0.1p], Σ(θ) Toeplitz.

50.75
44.80
19.50
59.10
52.75
25.70
64.55
58.65
30.65

52.15
43.55
18.85
59.20
53.45
25.75
66.10
59.35
30.75

C=4

MB Lasso

Our methods

Table 12: Simulation results in Design 10: µj (θ) = −0.5 · 1[j > 0.1p] + 0.05 · 1[j ≤ 0.1p], Σ(θ) Toeplitz.

50.75
44.80
19.50
59.10
52.75
25.70
64.55
58.65
30.65

52.15
43.60
18.85
59.40
53.45
25.75
66.20
59.60
30.80

C=2

50.75
44.80
19.50
59.10
52.75
25.70
64.55
58.65
30.65

52.15
43.60
18.85
59.40
53.45
25.75
66.20
59.60
30.80

C=2

SN Lasso

19.85
17.90
12.30
22.05
20.35
13.15
23.75
22.40
14.95

18.40
17.15
11.45
20.35
18.40
12.40
20.35
21.30
15.00

0.01%

48.60
43.80
28.70
51.65
46.00
30.15
50.50
46.70
31.85

40.50
35.95
24.95
40.30
38.85
27.30
40.25
38.65
28.65

0.01%

EB-H

32.55
29.35
18.05
32.30
28.55
18.70
31.15
29.50
19.65

28.50
24.95
16.40
27.40
26.15
17.15
26.10
26.50
19.30

0.10%

EB-H

52.50
48.70
33.20
61.90
55.90
37.95
65.80
60.60
41.30

48.05
42.80
31.25
51.45
48.80
35.40
52.70
49.75
38.40

0.10%

38.85
34.75
22.35
40.15
37.20
23.35
39.25
36.55
25.35

31.85
29.00
19.65
30.40
29.70
21.55
28.50
29.75
23.10

1.00%

42.20
38.25
25.15
50.15
45.10
32.05
54.95
51.25
36.15

39.90
35.20
24.15
44.55
42.05
30.20
48.05
45.55
35.50

1.00%

34.20
30.90
21.20
34.75
31.70
21.55
35.10
32.30
23.50

27.85
24.60
18.85
26.30
25.30
19.20
25.50
26.25
21.40

0.01%

53.40
48.65
33.20
62.10
55.95
37.95
65.45
61.45
42.50

47.05
42.55
30.45
48.80
47.20
36.60
49.15
49.00
39.50

0.01%

36.80
32.50
23.05
37.10
33.45
23.05
36.55
34.25
24.50

29.05
25.80
20.15
27.05
26.20
20.70
26.50
27.10
22.45

0.10%

EB-2S

52.90
48.75
33.10
62.35
56.30
39.05
66.85
62.05
43.30

48.15
43.00
31.10
50.20
48.45
36.95
51.45
50.45
40.15

0.10%

EB-2S

39.45
35.65
24.35
42.85
39.20
27.70
42.95
40.20
28.90

32.70
28.95
21.85
30.65
30.75
24.65
29.75
30.75
26.50

1.00%

41.85
37.70
25.50
50.40
45.70
31.95
55.35
52.45
35.80

40.45
34.65
24.15
44.65
42.50
30.45
47.35
46.05
35.10

1.00%

30

√ √
U (− 3, 3)

200

√
t4 / 2

1000

500

200

1000

500

p

1000

500

200

1000

Density

√ √
U (− 3, 3)

200

√
t4 / 2

500

p

Density

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

ρ

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

ρ

11.85
9.95
4.75
14.00
13.40
5.65
15.75
14.75
6.80

26.70
19.15
5.75
42.65
34.95
9.60
56.10
48.25
14.45

C=6

53.95
49.75
34.65
63.25
57.95
40.65
70.05
65.70
46.55

55.45
48.40
33.95
64.70
59.40
41.60
71.40
65.75
46.85

C=2

15.25
13.70
10.20
19.10
18.80
12.85
22.60
21.60
15.95

51.55
43.30
20.65
63.55
57.70
33.35
70.75
65.25
42.25

C=4

13.80
11.95
8.85
16.80
16.40
11.05
20.05
18.55
13.40

30.10
22.65
10.25
47.30
40.00
16.85
61.15
54.30
23.55

C=6

53.25
49.65
34.20
64.40
57.50
39.80
69.55
65.30
45.75

54.20
47.25
34.15
62.70
57.55
41.25
69.10
64.10
46.25

C=2

15.65
13.80
10.45
19.50
19.00
12.80
22.75
22.25
15.65

49.10
40.85
19.60
60.75
55.05
32.10
67.20
62.45
41.80

C=4

EB Lasso

14.00
12.20
9.05
17.30
16.40
11.45
19.85
18.95
12.90

27.05
21.00
10.20
42.05
36.40
15.90
55.30
49.50
22.95

C=6

11.85
9.95
4.75
14.00
13.40
5.65
15.75
14.75
6.80

11.80
10.15
4.85
14.15
12.45
6.00
16.15
15.40
6.90

SN-1S

11.85
10.20
4.80
14.00
13.40
5.75
15.70
14.75
6.80

12.10
10.40
4.85
14.35
12.50
6.00
16.20
15.45
6.95

0.01%

12.75
10.80
5.15
14.05
13.65
5.95
15.60
14.75
6.80

12.90
11.20
5.00
14.85
12.90
6.05
16.50
15.65
6.90

0.10%

SN-2S

13.25
11.15
5.25
12.60
12.15
5.15
12.80
12.40
5.80

13.15
11.45
5.00
13.85
11.75
5.40
13.40
13.80
6.15

1.00%

13.80
11.95
8.85
16.80
16.40
11.05
20.05
18.55
13.40

13.75
12.50
8.45
17.95
15.55
11.20
19.70
19.80
13.60

MB-1S

13.95
12.05
8.95
16.90
16.50
11.00
20.10
18.55
13.35

14.05
12.65
8.45
17.90
15.75
11.20
19.80
19.90
13.60

0.01%

14.95
12.95
9.35
17.05
16.60
11.10
20.05
18.70
13.30

15.20
13.55
8.90
18.70
16.10
11.30
19.85
20.20
13.55

0.10%

MB-H

16.20
14.35
10.15
17.60
16.95
11.05
19.50
18.50
12.50

14.65
13.30
9.20
16.40
14.90
10.70
17.20
17.10
12.20

1.00%

15.45
13.80
10.00
18.25
17.80
11.60
20.65
19.45
13.85

15.90
14.30
9.30
19.70
16.70
11.90
20.55
20.50
14.10

0.01%

15.55
13.65
10.05
17.90
17.60
11.60
20.35
19.25
13.85

16.30
14.35
9.30
19.55
16.55
11.90
20.35
20.65
14.05

0.10%

MB-2S

CCK14’s methods

16.00
14.35
11.10
16.45
16.50
11.30
18.10
16.80
12.40

16.50
14.70
10.65
17.80
15.75
11.30
18.60
18.35
12.70

1.00%

14.20
12.25
9.15
17.50
16.65
11.55
20.40
19.20
13.05

12.80
11.50
8.60
15.70
14.60
10.75
17.95
18.00
13.50

EB-1S

11.85
10.10
4.75
14.00
13.40
5.75
15.75
14.75
6.80

30.90
23.15
6.25
45.85
38.85
11.55
58.00
50.40
17.05

C=4

SN Lasso

11.85
9.95
4.75
14.00
13.40
5.65
15.75
14.75
6.80

15.75
12.85
5.10
23.75
20.15
7.00
36.15
28.95
9.10

C=6

40.30
36.85
24.90
48.15
44.50
30.85
53.25
52.50
37.00

55.15
48.05
31.75
64.60
59.35
40.80
71.30
65.65
46.20

C=2

13.80
11.95
8.85
16.80
16.45
11.10
20.10
18.55
13.45

33.95
26.60
11.05
51.40
44.25
18.95
63.10
56.85
26.95

C=4

MB Lasso

13.80
11.95
8.85
16.80
16.40
11.05
20.05
18.55
13.40

17.70
15.60
9.00
28.40
24.70
12.15
41.45
35.20
16.70

C=6

Our methods

39.95
37.45
25.05
48.95
45.35
30.90
53.65
51.55
36.60

53.50
46.75
31.65
62.35
57.05
39.60
68.70
63.75
45.40

C=2

14.00
12.20
9.05
17.30
16.50
11.45
19.85
18.95
12.90

31.50
24.70
11.10
45.70
40.20
17.60
58.00
51.75
26.15

C=4

EB Lasso

14.00
12.20
9.05
17.30
16.40
11.45
19.85
18.95
12.90

16.45
14.40
9.05
24.05
22.35
11.55
35.50
30.70
16.25

C=6

11.85
9.95
4.75
14.00
13.40
5.65
15.75
14.75
6.80

11.80
10.15
4.85
14.15
12.45
6.00
16.15
15.40
6.90

SN-1S

11.65
9.95
4.75
14.00
13.40
5.65
15.65
14.75
6.80

11.75
10.15
4.85
14.10
12.45
6.00
16.15
15.40
6.90

0.01%

11.50
9.75
4.65
13.55
13.15
5.40
15.25
14.40
6.70

11.70
9.80
4.70
14.00
12.05
5.90
15.65
15.20
6.75

0.10%

SN-2S

8.30
7.65
3.90
8.95
9.40
3.90
10.60
10.90
5.15

8.15
6.75
3.60
9.85
8.40
3.85
10.90
11.45
4.70

1.00%

13.80
11.95
8.85
16.80
16.40
11.05
20.05
18.55
13.40

13.75
12.50
8.45
17.95
15.55
11.20
19.70
19.80
13.60

MB-1S

13.80
11.90
8.75
16.80
16.35
11.00
20.00
18.50
13.30

13.65
12.50
8.45
17.75
15.55
11.20
19.65
19.75
13.60

0.01%

13.60
11.70
8.70
16.30
16.20
10.65
19.50
18.15
13.05

13.30
12.10
8.35
17.25
15.20
10.80
19.25
19.50
13.50

0.10%

MB-H

12.20
10.80
7.70
15.10
14.60
9.50
17.80
17.10
11.35

11.20
10.05
7.45
13.85
11.95
9.15
15.55
15.95
11.05

1.00%

13.85
11.90
8.75
16.85
16.40
11.00
20.00
18.50
13.30

13.70
12.55
8.45
17.80
15.55
11.20
19.65
19.75
13.60

0.01%

13.60
11.70
8.70
16.30
16.20
10.70
19.50
18.15
13.05

13.35
12.15
8.35
17.25
15.20
10.80
19.25
19.50
13.50

0.10%

MB-2S

CCK14’s methods

10.60
9.15
6.30
11.85
12.25
7.90
14.30
13.90
8.80

10.30
8.95
6.50
12.50
11.10
7.75
14.20
14.65
9.05

1.00%

14.20
12.25
9.15
17.50
16.65
11.55
20.40
19.20
13.05

12.80
11.50
8.60
15.70
14.60
10.75
17.95
18.00
13.50

EB-1S

Table 14: Simulation results in Design 12: µj (θ) = −0.3 · 1[j > 0.1p] + 0.05 · 1[j ≤ 0.1p], Σ(θ) Toeplitz.

37.30
32.90
15.75
43.85
39.85
19.85
46.55
45.00
24.35

51.60
42.85
18.25
58.90
53.30
25.30
66.00
59.55
30.60

C=2

13.40
11.40
5.75
15.60
15.55
7.30
18.10
17.50
8.65

48.25
38.80
12.35
58.35
51.90
20.60
65.60
58.70
28.50

C=4

MB Lasso

Our methods

Table 13: Simulation results in Design 11: µj (θ) = −0.4 · 1[j > 0.1p] + 0.05 · 1[j ≤ 0.1p], Σ(θ) Toeplitz.

50.70
44.75
19.45
59.10
52.75
25.70
64.55
58.65
30.65

52.10
43.60
18.85
59.40
53.45
25.75
66.20
59.60
30.80

C=2

SN Lasso

14.20
12.25
9.15
17.45
16.55
11.50
20.30
19.15
13.05

12.75
11.55
8.55
15.65
14.60
10.75
17.95
17.95
13.50

0.01%

14.30
12.35
9.35
17.45
16.70
11.55
20.35
19.20
13.10

13.10
11.90
8.60
16.05
14.60
10.75
18.05
18.05
13.55

0.01%

EB-H

13.70
11.85
8.95
17.00
16.25
11.20
19.75
18.75
12.70

12.45
11.05
8.20
15.30
14.25
10.45
17.50
17.50
13.10

0.10%

EB-H

15.10
13.30
9.70
17.90
17.15
11.55
20.50
19.30
12.80

13.95
12.60
9.00
16.70
14.90
10.85
18.05
18.10
13.50

0.10%

10.45
9.15
6.35
12.45
12.20
7.65
14.55
14.10
8.90

9.05
8.15
6.20
11.20
10.25
7.30
12.05
13.10
8.75

1.00%

15.40
13.35
9.30
15.70
15.05
9.75
17.50
16.45
10.80

13.70
12.30
8.15
14.40
12.95
9.30
14.40
15.15
10.15

1.00%

13.55
12.10
8.95
17.30
16.40
10.90
20.65
19.50
13.25

12.90
11.55
8.25
15.45
14.50
10.60
17.55
17.95
13.05

0.01%

15.30
13.90
10.50
19.15
17.85
11.65
21.50
20.05
13.70

14.50
13.20
9.30
16.75
15.05
11.40
18.00
18.50
13.60

0.01%

13.35
11.75
8.80
17.10
16.00
10.55
19.90
18.85
12.75

12.50
11.00
8.20
15.10
13.85
10.45
17.25
17.80
12.50

0.10%

EB-2S

15.40
14.20
10.40
18.95
17.85
11.75
21.00
19.70
13.60

14.35
13.10
9.25
16.55
15.05
11.40
18.05
18.50
13.35

0.10%

EB-2S

10.10
9.25
6.25
11.95
12.45
7.65
14.75
13.95
9.00

9.15
7.95
6.15
10.95
10.00
7.15
12.10
13.05
8.50

1.00%

16.15
14.55
11.35
17.05
16.65
11.30
18.30
17.05
11.95

14.10
12.90
9.80
14.70
13.25
10.25
14.45
15.50
11.40

1.00%

31

√ √
U (− 3, 3)

200

√
t4 / 2

1000

500

200

1000

500

p

1000

500

200

1000

Density

√ √
U (− 3, 3)

200

√
t4 / 2

500

p

Density

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

ρ

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

ρ

11.85
9.95
4.75
14.00
13.40
5.65
15.75
14.75
6.80

12.40
10.50
4.90
16.75
13.85
6.10
21.05
18.45
7.00

C=6

15.75
14.15
10.40
19.85
19.30
12.85
23.30
22.30
15.95

42.00
34.10
15.40
56.55
50.10
24.80
65.55
59.70
33.70

C=2

13.80
11.95
8.85
16.80
16.40
11.05
20.05
18.55
13.40

17.70
15.25
8.95
27.20
23.90
12.05
38.60
32.90
16.25

C=4

13.80
11.95
8.85
16.80
16.40
11.05
20.05
18.55
13.40

14.30
12.95
8.55
20.25
17.30
11.30
24.40
23.10
14.00

C=6

15.80
14.30
10.55
20.20
19.30
12.90
23.25
22.60
15.60

38.85
32.35
14.90
52.20
46.40
22.90
60.70
54.95
32.50

C=2

14.00
12.20
9.05
17.30
16.40
11.45
19.85
18.95
12.90

16.10
14.00
8.95
23.25
20.75
11.50
32.05
28.60
15.90

C=4

EB Lasso

14.00
12.20
9.05
17.30
16.40
11.45
19.85
18.95
12.90

13.25
11.95
8.60
17.70
15.75
10.80
21.75
20.50
13.85

C=6

11.85
9.95
4.75
14.00
13.40
5.65
15.75
14.75
6.80

11.80
10.15
4.85
14.15
12.45
6.00
16.15
15.40
6.90

SN-1S

11.65
9.95
4.75
14.00
13.40
5.65
15.65
14.75
6.80

11.75
10.15
4.85
14.10
12.45
6.00
16.15
15.40
6.90

0.01%

11.50
9.75
4.65
13.55
13.15
5.40
15.25
14.40
6.70

11.70
9.80
4.65
14.00
12.05
5.90
15.65
15.20
6.75

0.10%

SN-2S

8.20
7.55
3.80
8.95
9.40
3.75
10.60
10.90
5.15

7.80
6.45
3.45
9.75
8.20
3.85
10.90
11.35
4.70

1.00%

13.80
11.95
8.85
16.80
16.40
11.05
20.05
18.55
13.40

13.75
12.50
8.45
17.95
15.55
11.20
19.70
19.80
13.60

MB-1S

13.80
11.90
8.75
16.80
16.35
11.00
20.00
18.50
13.30

13.65
12.50
8.45
17.75
15.55
11.20
19.65
19.75
13.60

0.01%

13.60
11.70
8.70
16.30
16.20
10.65
19.50
18.15
13.05

13.30
12.05
8.35
17.25
15.15
10.80
19.25
19.50
13.50

0.10%

MB-H

10.35
9.05
6.00
11.80
12.15
7.90
14.15
13.85
8.65

9.75
8.80
6.15
12.25
10.90
7.65
14.15
14.50
9.05

1.00%

13.80
11.90
8.75
16.80
16.35
11.00
20.00
18.50
13.30

13.65
12.50
8.45
17.75
15.55
11.20
19.65
19.75
13.60

0.01%

13.60
11.70
8.70
16.30
16.20
10.65
19.50
18.15
13.05

13.30
12.05
8.35
17.25
15.15
10.80
19.25
19.50
13.50

0.10%

MB-2S

CCK14’s methods

10.35
9.05
6.00
11.80
12.15
7.90
14.15
13.85
8.65

9.75
8.80
6.15
12.25
10.90
7.65
14.15
14.50
9.05

1.00%

14.20
12.25
9.15
17.50
16.65
11.55
20.40
19.20
13.05

12.80
11.50
8.60
15.70
14.60
10.75
17.95
18.00
13.50

EB-1S

11.85
9.95
4.75
14.00
13.40
5.65
15.75
14.75
6.80

12.05
10.20
4.85
15.05
12.80
6.05
18.00
16.85
6.95

C=4

SN Lasso

11.85
9.95
4.75
14.00
13.40
5.65
15.75
14.75
6.80

11.85
10.15
4.85
14.30
12.55
6.00
16.60
15.80
6.90

C=6

13.85
11.95
8.85
16.85
16.45
11.10
20.15
18.60
13.45

17.80
15.10
8.95
26.10
22.60
12.65
33.45
29.35
16.35

C=2

13.80
11.95
8.85
16.80
16.40
11.05
20.05
18.55
13.40

13.95
12.55
8.50
18.80
15.90
11.25
21.90
21.40
13.70

C=4

MB Lasso

13.80
11.95
8.85
16.80
16.40
11.05
20.05
18.55
13.40

13.75
12.50
8.45
18.05
15.60
11.20
20.25
20.00
13.60

C=6

Our methods

14.00
12.30
9.05
17.30
16.50
11.45
19.85
18.95
12.90

16.15
14.00
8.95
22.20
19.75
11.90
27.05
25.70
15.65

C=2

14.00
12.20
9.05
17.30
16.40
11.45
19.85
18.95
12.90

12.95
11.60
8.50
16.25
14.85
10.70
19.45
19.00
13.60

C=4

EB Lasso

14.00
12.20
9.05
17.30
16.40
11.45
19.85
18.95
12.90

12.75
11.55
8.50
15.75
14.65
10.70
18.20
18.05
13.45

C=6

11.85
9.95
4.75
14.00
13.40
5.65
15.75
14.75
6.80

11.80
10.15
4.85
14.15
12.45
6.00
16.15
15.40
6.90

SN-1S

11.65
9.95
4.75
14.00
13.40
5.65
15.65
14.75
6.80

11.75
10.15
4.85
14.10
12.45
6.00
16.15
15.40
6.90

0.01%

11.50
9.75
4.65
13.55
13.15
5.40
15.25
14.40
6.70

11.70
9.80
4.65
14.00
12.05
5.90
15.65
15.20
6.75

0.10%

SN-2S

8.20
7.55
3.80
8.95
9.40
3.75
10.60
10.90
5.15

7.80
6.45
3.45
9.75
8.20
3.85
10.90
11.35
4.70

1.00%

13.80
11.95
8.85
16.80
16.40
11.05
20.05
18.55
13.40

13.75
12.50
8.45
17.95
15.55
11.20
19.70
19.80
13.60

MB-1S

13.80
11.90
8.75
16.80
16.35
11.00
20.00
18.50
13.30

13.65
12.50
8.45
17.75
15.55
11.20
19.65
19.75
13.60

0.01%

13.60
11.70
8.70
16.30
16.20
10.65
19.50
18.15
13.05

13.30
12.05
8.35
17.25
15.15
10.80
19.25
19.50
13.50

0.10%

MB-H

10.35
9.05
6.00
11.80
12.15
7.90
14.15
13.85
8.65

9.75
8.80
6.15
12.25
10.90
7.65
14.15
14.50
9.05

1.00%

13.80
11.90
8.75
16.80
16.35
11.00
20.00
18.50
13.30

13.65
12.50
8.45
17.75
15.55
11.20
19.65
19.75
13.60

0.01%

13.60
11.70
8.70
16.30
16.20
10.65
19.50
18.15
13.05

13.30
12.05
8.35
17.25
15.15
10.80
19.25
19.50
13.50

0.10%

MB-2S

CCK14’s methods

10.35
9.05
6.00
11.80
12.15
7.90
14.15
13.85
8.65

9.75
8.80
6.15
12.25
10.90
7.65
14.15
14.50
9.05

1.00%

14.20
12.25
9.15
17.50
16.65
11.55
20.40
19.20
13.05

12.80
11.50
8.60
15.70
14.60
10.75
17.95
18.00
13.50

EB-1S

Table 16: Simulation results in Design 14: µj (θ) = −0.1 · 1[j > 0.1p] + 0.05 · 1[j ≤ 0.1p], Σ(θ) Toeplitz.

11.85
10.15
4.80
14.00
13.40
5.75
15.75
14.75
6.80

15.50
12.80
5.10
22.00
18.45
7.30
27.30
24.55
9.10

C=2

11.85
9.95
4.75
14.00
13.40
5.65
15.75
14.75
6.80

15.50
12.80
5.10
22.85
19.25
7.05
32.25
27.00
9.00

C=4

MB Lasso

Our methods

Table 15: Simulation results in Design 13: µj (θ) = −0.2 · 1[j > 0.1p] + 0.05 · 1[j ≤ 0.1p], Σ(θ) Toeplitz.

13.70
11.80
5.85
16.05
15.90
7.40
18.60
17.80
8.65

39.05
30.85
9.45
51.75
44.25
15.40
59.95
53.50
22.25

C=2

SN Lasso

14.20
12.25
9.15
17.45
16.55
11.50
20.30
19.15
13.05

12.75
11.50
8.55
15.65
14.60
10.75
17.95
17.95
13.50

0.01%

14.20
12.25
9.15
17.45
16.55
11.50
20.30
19.15
13.05

12.75
11.50
8.55
15.65
14.60
10.75
17.95
17.95
13.50

0.01%

EB-H

13.70
11.85
8.95
17.00
16.25
11.15
19.75
18.75
12.70

12.45
11.05
8.20
15.30
14.25
10.45
17.50
17.45
13.10

0.10%

EB-H

13.70
11.85
8.95
17.00
16.25
11.15
19.75
18.75
12.70

12.45
11.05
8.20
15.30
14.25
10.45
17.50
17.45
13.10

0.10%

10.30
9.00
6.25
12.35
12.15
7.60
14.55
14.10
8.90

8.80
7.85
6.10
11.20
10.15
7.30
12.00
13.10
8.75

1.00%

10.30
9.00
6.25
12.35
12.15
7.60
14.55
14.10
8.90

8.80
7.85
6.10
11.20
10.15
7.30
12.00
13.10
8.75

1.00%

13.50
12.05
8.95
17.30
16.40
10.90
20.65
19.50
13.25

12.90
11.50
8.25
15.45
14.50
10.60
17.55
17.95
13.05

0.01%

13.50
12.05
8.95
17.30
16.40
10.90
20.65
19.50
13.25

12.90
11.50
8.25
15.45
14.50
10.60
17.55
17.95
13.05

0.01%

13.35
11.75
8.80
17.10
16.00
10.55
19.85
18.85
12.75

12.40
11.00
8.20
15.10
13.85
10.45
17.25
17.80
12.50

0.10%

EB-2S

13.35
11.75
8.80
17.10
16.00
10.55
19.85
18.85
12.75

12.40
11.00
8.20
15.10
13.85
10.45
17.25
17.80
12.50

0.10%

EB-2S

9.85
9.00
5.85
11.75
12.30
7.60
14.75
13.85
8.95

8.80
7.60
5.80
10.90
9.90
7.00
12.05
12.95
8.50

1.00%

9.85
9.00
5.85
11.75
12.30
7.60
14.75
13.85
8.95

8.80
7.60
5.80
10.90
9.90
7.00
12.05
12.95
8.50

1.00%

32

1000

500

200

1000

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

ρ

2.00
2.00
2.00
5.00
5.00
5.00
10.00
10.00
10.00

9.99
9.99
10.00
9.97
9.98
9.99
9.94
9.95
9.99
2.00
2.00
2.00
5.00
5.00
5.00
10.00
10.00
10.00

10.00
10.00
10.00
10.00
10.00
10.00
10.00
10.00
10.00

C=4

2.00
2.00
2.01
5.01
5.01
5.01
10.02
10.01
10.02

10.00
10.00
10.01
10.00
10.00
10.00
10.00
10.00
10.00

C=6

2.00
2.00
2.00
5.00
5.00
5.00
10.00
10.00
10.00

10.35
10.26
10.06
10.49
10.37
10.11
10.64
10.49
10.14

0.01%

2.00
2.00
2.00
5.00
5.00
5.00
10.00
10.00
10.00

10.15
10.11
10.02
10.21
10.17
10.05
10.27
10.20
10.05

0.10%

2.00
2.00
2.00
5.00
5.00
5.00
10.00
10.00
10.00

10.06
10.04
10.01
10.10
10.08
10.02
10.12
10.09
10.02

1.00%

2.00
2.00
2.00
5.00
5.00
5.00
10.00
10.00
10.00

10.16
10.12
10.02
10.23
10.18
10.04
10.29
10.22
10.05

0.01%

2.00
2.00
2.00
5.00
5.00
5.00
10.00
10.00
10.00

10.13
10.10
10.02
10.19
10.14
10.04
10.23
10.17
10.04

0.10%

2.00
2.00
2.00
5.00
5.00
5.00
10.00
10.00
10.00

10.05
10.04
10.01
10.09
10.07
10.02
10.10
10.08
10.02

1.00%

MB selection

C=2

CCK14’s methods
SN selection

Our methods
Lasso selection

2.00
2.00
2.00
5.00
5.00
5.00
10.00
10.00
10.00

10.25
10.17
10.03
10.37
10.27
10.06
10.49
10.35
10.07
2.00
2.00
2.00
5.00
5.00
5.00
10.00
10.00
10.00

10.17
10.11
10.02
10.25
10.18
10.04
10.32
10.23
10.05

0.10%

2.00
2.00
2.00
5.00
5.00
5.00
10.00
10.00
10.00

10.06
10.04
10.01
10.10
10.07
10.02
10.12
10.09
10.02

1.00%

EB Selection
0.01%

1000

500

200

1000

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

0
0.5
0.9
0
0.5
0.9
0
0.5
0.9

ρ

2.00
2.00
2.00
5.01
5.01
5.00
10.02
10.01
10.01

9.99
9.99
10.00
9.97
9.98
10.00
9.95
9.96
9.99
17.04
16.40
15.35
41.99
40.28
37.28
83.05
79.39
72.46

12.01
12.93
26.27
10.41
10.62
16.18
10.14
10.20
12.54

C=4

20.00
20.00
20.00
50.00
50.00
50.00
100.00
100.00
100.00

46.98
55.22
86.88
25.12
31.38
70.57
16.32
19.76
55.08

C=6

19.60
19.58
19.55
49.62
49.59
49.56
99.63
99.62
99.58

95.88
96.32
97.22
98.06
98.33
98.89
98.94
99.11
99.42

0.01%

17.27
17.22
17.10
46.61
46.55
46.40
96.22
96.12
95.92

81.21
82.13
84.36
89.18
89.97
91.81
93.19
93.79
95.12

0.10%

10.54
10.49
10.43
34.35
34.26
34.17
78.71
78.53
78.06

49.76
50.27
51.58
63.79
64.65
66.98
73.30
74.24
76.52

1.00%

16.73
16.71
15.83
45.25
45.21
43.90
94.10
94.00
91.86

78.87
79.80
78.14
86.63
87.31
86.58
90.72
91.45
91.05

0.01%

15.95
15.95
14.88
44.09
43.99
42.36
92.53
92.39
89.74

74.71
75.76
73.37
83.75
84.48
83.40
88.60
89.38
88.81

0.10%

9.38
9.32
7.57
30.97
30.85
26.88
72.32
71.98
64.80

44.79
45.03
37.67
57.92
58.51
52.79
67.23
67.98
63.52

1.00%

MB selection

C=2

CCK14’s methods
SN selection

Our methods
Lasso selection

16.66
16.63
15.68
45.20
45.21
43.84
93.88
93.82
91.92

82.96
82.90
78.91
90.44
90.38
88.21
94.05
94.05
92.20

15.88
15.84
14.75
43.98
43.96
42.26
92.26
92.14
89.76

79.09
78.99
74.25
87.96
87.87
85.11
92.39
92.41
90.07

0.10%

9.33
9.20
7.49
30.77
30.72
26.82
71.79
71.51
64.87

47.60
47.15
38.46
61.93
61.67
53.90
71.51
71.49
65.04

1.00%

EB Selection
0.01%

Table 18: Percentage of moment inequalities retained by first step selection procedures in Design 11: µj (θ) = −0.4 · 1[j > 0.1p] + 0.05 · 1[j ≤ 0.1p],
Σ(θ) Toeplitz.

√ √
U (− 3, 3)

200

√
t4 / 2

500

p

Density

Table 17: Percentage of moment inequalities retained by first step selection procedures in Design 8: µj (θ) = −0.75 · 1[j > 0.1p] + 0.05 · 1[j ≤ 0.1p],
Σ(θ) Toeplitz.

√ √
U (− 3, 3)

200

√
t4 / 2

500

p

Density

A

Appendix

Throughout the appendix, we omit the dependence of all expressions on θ. Furthermore, LHS and RHS abbreviate
“left hand side” and “right hand side”, respectively.

A.1

Auxiliary results

Lemma A.1. Assume Assumptions A.1-A.2. Then, for any γ s.t.

P



√

nγ/

p

−1
1 + γ 2 ∈ [0, nδ/(2(2+δ)) Mn,2+δ
],


p
p
√
√
2+δ
max |µ̂j − µj |/σ̂j > γ ≤ 2p(1 − Φ( nγ/ 1 + γ 2 ))[1 + Kn−δ/2 Mn,2+δ
(1 + nγ/ 1 + γ 2 )2+δ ],

j=1,...,p

(A.1)

where K is a universal constant.
Proof. For any i = 1, . . . , n and j = 1, . . . , p, let Zij ≡ (Xij − µj )/σj and Uj ≡

qP
√ Pn
n
2
n i=1 (Zij /n)/
i=1 (Zij /n).

We divide the rest of the proof into three steps.
q
√
Step 1. By definition, n(µ̂j − µj )/σ̂j = Uj / 1 − Uj2 /n and so
p
√
n|µ̂j − µj |/σ̂j = |Uj |/ 1 − |Uj |2 /n.

(A.2)

Since the RHS of Eq. (A.2) is increasing in |Uj |, it follows that:
n

max |µ̂j − µj |/σ̂j > γ

o

j=1,...,p

=

n

max |Uj |/

1≤j≤p

o
p
p
√ o n
√
1 − |Uj |2 /n > nγ ⊆ max |Uj | ≥ nγ/ 1 + γ 2 .
1≤j≤p

(A.3)

Step 2. For every j = 1, . . . , p, {Zij }n
i=1 is a sequence of independent random variables with E[Zij ] = 0,
Pn
Pn
2+δ
2
2
2
] = 1, and E[|Zij |2+δ ] ≤ Mn,2+δ
< ∞. If we let Snj =
E[Zij
i=1 Zij , and 0 < Dnj =
i=1 Zij , Vnj =
P
−1
2+δ 1/(2+δ)
]]
≤ Mn,2+δ < ∞, then CCK14 (Lemma A.1) implies that for all t ∈ [0, nδ/(2(2+δ)) Dnj
],
[n−1 n
i=1 E[|Zij |
P (Snj /Vnj ≥ t)
2+δ
− 1 ≤ Kn−δ/2 Dnj
(1 + t)2+δ ,
1 − Φ(t)

(A.4)

where K is a universal constant.
By using that Uj = Snj /Vnj , Dnj ≤ Mn,2+δ , and applying Eq. (A.4) to t =
p
√
−1
],
γ s.t. nγ/ 1 + γ 2 ∈ [0, nδ/(2(2+δ)) Mn,2+δ

p
√
nγ/ 1 + γ 2 , it follows that for any



p
p
p
p
2+δ

√
√
√
√
2+δ
1 − Φ( nγ/ 1 + γ 2 ) 1 + nγ/ 1 + γ 2
.
P Uj ≥ nγ/ 1 + γ 2 − (1 − Φ( nγ/ 1 + γ 2 )) ≤ Kn−δ/2 Dnj

Thus, for any γ s.t.
p
X

p
√
−1
nγ/ 1 + γ 2 ∈ [0, nδ/(2(2+δ)) Mn,2+δ
],



p
p
p


√
√
√
2+δ
P Uj ≥ nγ/ 1 + γ 2 ≤ p 1 − Φ( nγ/ 1 + γ 2 ) 1 + Kn−δ/2 Mn,2+δ
(1 + nγ/ 1 + γ 2 )2+δ .

j=1

33

(A.5)

By applying the same argument for −Zij instead of Zij , it follows that for any γ s.t.

p
√
nγ/ 1 + γ 2 ∈

−1
[0, nδ/(2(2+δ)) Mn,2+δ
],
p
X



p
p
p


√
√
√
2+δ
P −Uj ≥ nγ/ 1 + γ 2 ≤ p 1 − Φ( nγ/ 1 + γ 2 ) 1 + Kn−δ/2 Mn,2+δ
(1 + nγ/ 1 + γ 2 )2+δ .

(A.6)

j=1

Step 3. Consider the following argument.

P



max |µ̂j − µj |/σ̂j > γ

j=1,...,p



≤
≤

P



max |Uj | ≥

1≤j≤p

p
X

√

nγ/


p
1 + γ2



p
√
P |Uj | ≥ nγ/ 1 + γ 2

j=1

≤

p
X

p

 X
h
i
p
p
√
√
P Uj ≥ nγ/ 1 + γ 2 +
P −Uj ≥ nγ/ 1 + γ 2

j=1

≤

j=1

h

p
p
2+δ i
√
√
2+δ
1 + nγ/ 1 + γ 2
,
2p 1 − Φ( nγ/ 1 + γ 2 ) 1 + Kn−δ/2 Mn,2+δ

where the first inequality follows from Eq. (A.3), the second inequality follows from Bonferroni bound, and the fourth
inequality follows from Eqs. (A.5) and (A.6).
Lemma A.2. Assume Assumptions A.1-A.2 and let {γn }n≥1 ⊆ R satisfy γn ≥ γn∗ for all n sufficiently large, where
2+δ
2
2+δ
)−2/(2+δ) )−1/2 → 0.
)−1/(2+δ) (1 − (nMn,2+δ
γn∗ ≡ n−1/2 (Mn,2+δ
n−δ/(2+δ) − n−1 )−1/2 = (nMn,2+δ

(A.7)

Then,

P






2
max |µ̂j − µj |/σ̂j > γn ≤ 2p exp −2−1 nδ/(2+δ) /Mn,2+δ
1 + K(Mn,2+δ /nδ/(2(2+δ)) + 1)2+δ → 0.

j=1,...,p

(A.8)

2+δ
Proof. First, note that the convergence to zero in Eq. (A.7) follows from nMn,2+δ
→ ∞. Since γn ≥ γn∗ , Eq. (A.8)

holds if we show:

P






2
max |µ̂j − µj |/σ̂j > γn∗ ≤ 2p exp −2−1 nδ/(2+δ) /Mn,2+δ
1 + K(Mn,2+δ /nδ/(2(2+δ)) + 1)2+δ → 0.

j=1,...,p

As we show next, Eq. (A.9) follows from using Lemma A.1 with γ = γn∗ . This choice of γ implies

√

nγn∗ /

p

(A.9)

1 + (γn∗ )2 =

−1
nδ/(2(2+δ)) Mn,2+δ
making γ = γn∗ a valid choice in Lemma A.1. Then, Lemma A.1 with γ = γn∗ implies that:

P



max |µ̂j − µj |/σ̂j > γn∗

j=1,...,p



≤

−1
2p 1 − Φ(nδ/(2(2+δ)) Mn,2+δ
)

≤

2
2p exp −2−1 nδ/(2+δ) /Mn,2+δ

2

where we have used that 1 − Φ(t) ≤ e−t

/2



2+δ
−1
1 + Kn−δ/2 Mn,2+δ
(1 + nδ/(2(2+δ)) Mn,2+δ
)2+δ






1 + K(n−δ/(2(2+δ)) Mn,2+δ + 1)2+δ ,

. We now show that the RHS of the above display converges to zero

(2+δ)

by Assumption A.2. First, notice that Mn,2+δ (ln(2k − p))(2+δ)/2 n−δ/2 → 0. Next, (2k − p) > 1 implies that

34

(2+δ)

(2+δ)

Mn,2+δ n−δ/2 → 0 and, in turn, this implies that n−δ/(2(2+δ)) Mn,2+δ → 0. Furthermore, notice that Mn,2+δ (ln(2k −
(2+δ)

2
p))(2+δ)/2 n−δ/2 → 0, Mn,2+δ (ln(2k −p))(2+δ)/2 n−δ/2 > 0, and (2k −p) ≥ p implies that nδ/(2+δ) (Mn,2+δ
ln p)−1 → ∞.

This implies that:




2
2
p exp −2−1 nδ/(2+δ) /Mn,2+δ
] = exp ln p 1 − 2−1 [nδ/(2+δ) (Mn,2+δ
ln p)−1 ] → 0,

completing the proof.
Proof of Lemma 3.1. By definition, J ⊆ JI where JI is as defined in the proof of Theorem 4.2. Then, the result is a
corollary of Step 2 in the proof of Theorem 4.2.
Proof of Lemma 3.2. Fix j = 1, . . . , p arbitrarily. Bühlmann and van de Geer (2011, Eq. (2.5)) implies that the Lasso
estimator in Eq. (3.2) satisfies:

µ̂L,j = sign(µ̂j ) × max{|µ̂j | − σ̂j λn /2, 0}

∀j = 1, . . . , p.

(A.10)

To complete the proof, it suffices to show that:

{µ̂L,j ≥ −σ̂j λn } = {µ̂j ≥ −3σ̂j λn /2}.

(A.11)

We divide the verification into four cases. First, consider that σ̂j = 0. If so, −σ̂j λn = −3σ̂j λn /2 = 0 and µ̂L,j =
sign(µ̂j ) × max{|µ̂j |, 0} = µ̂j , and so Eq. (A.11) holds. Second, consider that σ̂j > 0 and µ̂j ≥ 0. If so, µ̂j ≥ 0 ≥
−3σ̂j λn /2 and so the RHS condition in Eq. (A.11) is satisfied. In addition, Eq. (A.10) implies that µ̂L,j ≥ 0 ≥ −σ̂j λn
and so the LHS of condition in Eq. (A.11) is also satisfied. Thus, Eq. (A.11) holds. Third, consider that σ̂j > 0
and µ̂j ∈ [−σ̂j λn /2, 0). If so, µ̂j ≥ −σ̂j λn /2 ≥ −3σ̂j λn /2 and so the RHS condition in Eq. (A.11) is satisfied. In
addition, Eq. (A.10) implies that µ̂L,j = 0 ≥ −σ̂j λn and so the LHS of condition in Eq. (A.11) is also satisfied.
Thus, Eq. (A.11) holds. Fourth and finally, consider that σ̂j > 0 and µ̂j < −σ̂j λn /2. Then, Eq. (A.10) implies that
µ̂L,j = µ̂j + σ̂j λn /2 and so Eq. (A.11) holds.

A.2

Results for the self-normalization approximation

Lemma A.3. For any π ∈ (0, 0.5], n ∈ N, and d ∈ {0, 1 . . . , 2k − p}, define the function:

CV (d) ≡




 0

if d = 0,

Φ−1 (1−π/d)


 √1−(Φ−1 (1−π/d))2 /n

if d > 0.

Then, CV : {0, 1 . . . , 2k − p} → R+ is weakly increasing for n sufficiently large.

35

Proof. First, we show that CV (d) ≤ CV (d + 1) for d = 0. To see this, use that π ≤ 0.5 such that Φ−1 (1 − π) ≥ 0,
implying that CV (1) ≥ 0 = CV (0).
Second, we show that CV (d) ≤ CV (d + 1) for any d > 0. To see this, notice that CV (d) and CV (d + 1) are both
the result of the composition g1 (g2 (·)) : {1 . . . , 2k − p} → R where:

g1 (y)

≡

p
√
y/ 1 − y 2 /n : [0, n) → R+

g2 (d)

≡

Φ−1 (1 − π/d) : {1 . . . , 2k − p} → R.

We first show that g1 (g2 (·)) is properly defined by verifying that the range of g2 is included in support of g1 . Notice
that g2 is an increasing function and so g2 (d) ∈ [g2 (1), g2 (2(k − p) + p)] = [Φ−1 (1 − π), Φ−1 (1 − π/(2k − p))]. For
the lower bound, π ≤ 0.5 implies that Φ−1 (1 − π) ≥ 0. For the upper bound, consider the following argument. On
√
the one hand, (1 − Φ( n)) ≤ exp(−n/2)/2 holds for all n large enough. On the other hand, Assumption A.2 implies
√
that exp(−n/2)/2 ≤ π/(2k − p). By combining these two, we conclude that Φ−1 (1 − π/(2k − p)) ≤ n for all n
large enough, as desired. From here, the monotonicity of CV (d) follows from the fact that g1 and g2 are both weakly
increasing functions and so CV (d) = g1 (g2 (d)) ≤ g1 (g2 (d + 1)) = CV (d + 1).
Lemma A.4. Assume Assumptions A.1-A.2, α ∈ (0, 0.5), and that H0 holds. For any non-stochastic set L ⊆
{1, . . . , p}, define:

Tn (L)
cSN
n (|L|, α)

≡
≡


√
max max nµ̂j /σ̂j ,
j∈L

√

√

max

s=p+1,...,k

Φ−1 (1−α/(2(k−p)+|L|))

1−(Φ−1 (1−α/(2(k−p)+|L|)))2 /n


n|µ̂s |/σ̂s

.

Then,

P Tn (L) > cSN
n (|L|, α) ≤ α + Rn ,
2+δ
where Rn ≡ αKn−δ/2 Mn,2+δ
(1 + Φ−1 (1 − α/(2k − p)))2+δ → 0 and K is a universal constant.

Proof. Under H0 ,

√
√
√
√
nµ̂j /σ̂j ≤ n(µ̂j − µj )/σ̂j for all j ∈ L and n|µ̂s |/σ̂s = n|µ̂s − µs |/σ̂s for s = p + 1, . . . , k.

From this, we deduce that:

Tn (L)

=
≤

n
o
√
√
max max nµ̂j /σ̂j , max
n|µ̂s |/σ̂s
j∈L
s=p+1,...,k
n
o
√
√
max max n(µ̂j − µj )/σ̂j , max
n|µ̂s − µs |/σ̂s = Tn∗ (L).
j∈L

s=p+1,...,k

For any i = 1, . . . , n and j = 1, . . . , k, let Zij ≡ (Xij − µj )/σj and Uj ≡

36

qP
√ Pn
n
2
n i=1 (Zij /n)/
i=1 (Zij /n). It then

follows that

q
√
n[µ̂j − µj ]/σ̂j = Uj / 1 − Uj2 /n and so,
√

n(µ̂j − µj )/σ̂j

=

√
n|µ̂j − µj |/σ̂j

=

p
1 − |Uj |2 /n
p
|Uj |/ 1 − |Uj |2 /n.
Uj /

Notice that the expressions on the RHS are increasing in Uj and |Uj |, respectively. Therefore, for any c ≥ 0,
{Tn∗ (L) > c}

=
=
=

o n
o
√
n(µ̂j − µj )/σ̂j > c ∪
max
n|µ̂s − µs |/σ̂s > c
j∈L
s=p+1,...,k
n
o n
o
p
p
2
max Uj / 1 − |Uj | /n > c ∪
max |Us |/ 1 − |Us |2 /n > c
j∈L
s=p+1,...,k
o n
o
n
p
p
2
max |Us | > c/ 1 + c2 /n .
max Uj > c/ 1 + c /n ∪

n

max

√

j∈L

s=p+1,...,k

From here, we conclude that for all c ≥ 0 such that: c/

P (Tn (L) > c)

p

−1
1 + c2 /n ∈ [0, nδ/(2(2+δ)) Mn,2+δ
],

≤

P (Tn∗ (L) > c)
n
o n
p
P
max Uj > c/ 1 + c2 /n ∪

≤

k



X 
X
p
p
P Uj > c/ 1 + c2 /n +
P |Us | > c/ 1 + c2 /n

≤

j∈L

s=p+1,...,k

k
k





X
X
X 
p
p
p
P Us > c/ 1 + c2 /n +
P −Ug > c/ 1 + c2 /n
P Uj > c/ 1 + c2 /n +
s=p+1

j∈L

≤

o
p
|Us | > c/ 1 + c2 /n

s=p+1

j∈L

≤

max



(2(k − p) + |L|) 1 − Φ(c/

p

1+

g=p+1

h

c2 /n)

−δ/2

1 + Kn

2+δ
(1
Mn,2+δ

+ c/

p

i
1 + c2 /n)2+δ ,

(A.12)

where the first inequality follows from Tn (L) ≤ Tn∗ (L), the third inequality is based on a Bonferroni bound, the last
√
inequality follows from Eqs. (A.4)-(A.5) in Lemma A.1 upon choosing γ = c/ n in that result.
We are interested in applying Eq. (A.12) with c = cSN
n (|L|, α) which satisfies:


p
2
(2(k − p) + |L|) 1 − Φ(cSN
1 + cSN
n (|L|, α)/
n (|L|, α) /n) = α.

(A.13)

Before doing this, we need to verify that this is a valid choice, i.e., we need to verify that, for all sufficiently large n,
cSN
n (|L|, α)/

p
δ/(2(2+δ))
−1
2
1 + cSN
Mn,2+δ
].
n (|L|, α) /n ∈ [0, n

(A.14)

p
SN
2
1 + cSN
On the one hand, note that cSN
n (α, |L|) /n ≥ 0. On the other hand,
n (|L|, α) ≥ 0 implies that cn (|L|, α)/
p
−1
2
note that, by definition, cSN
1 + cSN
(1 − α/(2(k − p) + |L|)) and so it suffices to show
n (|L|, α)/
n (|L|, α) /n = Φ
that Φ−1 (1 − α/(|L| + 2(k − p)))Mn,2+δ n−δ/(2(2+δ)) → 0. To show this, note that Φ−1 (1 − α/(2(k − p) + |L|)) ≤
p
p
2 ln((|L| + 2(k − p))/α) ≤ 2 ln((2k − p)/α), where the first inequality uses that 1 − Φ(t) ≤ exp(−t2 /2) for any
2
t > 0 and the second inequality follows from |L| ≤ p. These inequalities and ln((2k − p)/α)Mn,2+δ
n−δ/(2+δ) → 0 (by

37

Assumption A.2) complete the verification.
Therefore, by Eq. (A.12) with c = cSN
n (|L|, α) we conclude that:
−δ/2
2+δ
P (Tn > cSN
Mn,2+δ
(1 + Φ−1 (1 − α/(2(k − p) + |L|)))2+δ ≤ α + Rn ,
n (|L|, α)) ≤ α + αKn

where the first inequality uses Eq. (A.13) and the second inequality follows from the definition Rn and f (x) ≡
Φ−1 (1 − α/(2(k − p) + x)) being increasing and |L| ≤ p. To conclude the proof, it suffices to show that Rn → 0. To
this end, consider the following argument:

Rn

≡

2+δ
αKn−δ/2 Mn,2+δ
(1 + Φ−1 (1 − α/(2k − p)))2+δ

≤

2+δ
α21+δ Kn−δ/2 Mn,2+δ
(1 + |Φ−1 (1 − α/(2k − p))|2+δ )

≤

2+δ
(1 + 21/2 (ln((2k − p)/α))(2+δ)/2 )) = o(1),
α21+δ Kn−δ/2 Mn,2+δ

where the first inequality uses the convexity of f (x) = x2+δ and δ > 0 and Jensen’s Inequality to show (1 + a)2+δ ≤
21+δ (1 + a2+δ ) for any a > 0, the second inequality follows from 1 − Φ(t) ≤ exp(−t2 /2) for any t > 0 and so
p
2+δ
Φ−1 (1−α/(2k −p)) ≤ 2 ln((2k − p)/α), and the convergence to zero is based on n−δ/2 Mn,2+δ
(ln(2k −p))(2+δ)/2 → 0
2+δ
(by Assumption A.2) which for 2k − p > 1 implies that n−δ/2 Mn,2+δ
→ 0.

Proof of Theorem 4.1. This result follows from Lemma A.4 with L = {1, . . . , p}.
Proof of Theorem 4.2. This proof follows similar steps than CCK14 (Proof of Theorem 4.2). Let us define the
sequence of sets:
JI ≡ {j = 1, . . . , p : µj /σj ≥ −3λn /4}
We divide the proof into three steps.
Step 1. We show that µ̂j ≤ 0 for all j ∈ JIc with high probability, i.e., for any c ∈ (0, 1),



2
P ∪j∈JIc {µ̂j > 0} ≤ 2p exp −2−1 nδ/(2+δ) /Mn,2+δ
1 + K(Mn,2+δ /nδ/(2(2+δ)) + 1)2+δ + K̃n−c → 0,
where K and K̃ are universal constants.
First, we show that for any r ∈ (0, 1),



∪j∈JIc {µ̂j > 0} ∩

n

o n
o
sup |σ̂j /σj − 1| ≤ r/(1 + r) ⊆
sup |µ̂j − µj |/σ̂j > (1 − r)λn 3/4 .
j=1,...,p

j=1,...,p

To see this, suppose that there is an index j = 1, . . . , p s.t. µj /σj < −λn 3/4 and µ̂j > 0. Then, |µ̂j − µj |/σ̂j >
λn (3/4)(σj /σ̂j ). In turn, supj=1,...,p |1 − σ̂j /σj | ≤ r/(1 + r) implies that |1 − σj /σ̂j | ≤ r and so (σj /σ̂j )λn 3/4 ≥
(1 − r)λn 3/4. By combining these, we conclude that supj=1,...,p |µ̂j − µj |/σ̂j > (1 − r)λn (3/4).

38

Based on this, consider the following derivation for any r ∈ (0, 1),




 P (∪j∈J c {µ̂j > 0} ∩ sup

|σ̂
/σ
−
1|
≤
r/(1
+
r))+
j
j
j=1,...,p
I

P (∪j∈JIc {µ̂j > 0}) =



 P (∪j∈J c {µ̂j > 0} ∩ supj=1,...,p |σ̂j /σj − 1| > r/(1 + r)) 
I




≤ P
sup |µ̂j − µj |/σ̂j > (1 − r)λn 3/4 + P
sup |σ̂j /σj − 1| > r/(1 + r) .
j=1,...,p

(A.15)

j=1,...,p

By evaluating Eq. (A.15) with r = rn = (((n−(1−c)/2 ln p + n−3/2 (ln p)2 )Bn2 )−1 − 1)−1 → 0 (by Assumption A.3), we
deduce that:
2
P (∪j∈JIc {µ̂j > 0}) ≤ 2p exp(−2−1 nδ/(2+δ) /Mn,2+δ
)[1 + K(Mn,2+δ /nδ/(2(2+δ)) + 1)2+δ ] + K̃n−c ,

2
where the first term is a consequence of Lemma A.2, rn → 0, and (1 − rn )λn 3/4 ≥ n−1/2 (Mn,2+δ
n−δ/(2+δ) −

n−1 )−1/2 for all n sufficiently large, and the second term is a consequence of CCK14 (Lemma A.5) and rn /(1 + rn ) =
[n−(1−c)/2 ln p + n−3/2 (ln p)2 ]Bn2 → 0.
Step 2. We show that JI ⊆ JˆL with high probability, i.e.,
2
P (JI ⊆ JˆL ) ≥ 1 − 2p exp(−2−1 nδ/(2+δ) /Mn,2+δ
)[1 + K(Mn,2+δ /nδ/(2(2+δ)) + 1)2+δ ] + K̃n−c ,

where K, K̃ are uniform constants.
First, we show that for any r ∈ (0, 1),
n

{JI 6⊆ JˆL } ∩

n

oo
n
o
sup |σ̂j /σj − 1| ≤ r/(1 + r)
⊆
sup |µ̂j − µj |/σ̂j > λn (1 − r)3/4 .

j=1,...,p

j=1,...,p

To see this, consider the following argument. Suppose that j ∈ JI and j 6∈ JˆL , i.e., µj /σj ≥ −λn 3/4 and µ̂L,j /σ̂j <
−λn or, equivalently by Eq. (A.11), µ̂j /σ̂j < −λn 3/2. Then, |µj − µ̂j |/σ̂j > λn [ 23 − 34 (σj /σ̂j )]. In turn, supj=1,...,p |1−
σ̂j /σj | ≤ r/(1 + r) implies that |σj /σ̂j − 1| ≤ r and so λn [ 32 − 43 (σj /σ̂j )] ≥ λn (1 − r)3/4. By combining these, we
conclude that supj=1,...,p |µ̂j − µj |/σ̂j > λn (1 − r)3/4, as desired.
Based on this, consider the following derivation for any r ∈ (0, 1),

P (JI 6⊆ JˆL )

=

≤



 


 P {JI 6⊆ JˆL } ∩ {supj=1,...,p |σ̂j /σj − 1| ≤ r/(1 + r)}




 +P {JI 6⊆ JˆL } ∩ {supj=1,...,p |σ̂j /σj − 1| > r/(1 + r)} 





P
sup |µ̂j − µj |/σ̂j > λn (1 − r)3/4 + P
sup |σ̂j /σj − 1| > r/(1 + r) .
j=1,...,p

j=1,...,p

Notice that the expression on the RHS is exactly the RHS of Eq. (A.15). Consequently, by evaluating this equation
in r = rn and repeating arguments used in step 1, the desired result follows.

39

Step 3. We now complete the argument. Consider the following derivation:
n

o n
o

{Tn > cSN,L
(α)} ∩ {JI ⊆ JˆL } ∩ {∩j∈JIc {µ̂j ≤ 0}} ⊆ {Tn > cSN
n
n (|JI |, α)} ∩ {∩j∈JIc µ̂j ≤ 0 }


√
√
n
n|µ̂s | o
nµ̂j
, max
> cSN
(α,
|J
|)
,
⊆ max max
I
n
j∈JI
σ̂j s=p+1,...,k σ̂s

SN
ˆ
where we have used cSN,L
(α) = cSN
n
n (α, |JL |), Lemma A.3 (in that cn (α, d) is a non-negative increasing function of
√
d ∈ {0, 1 . . . , 2k − p}), and we take maxj∈JI nµ̂j /σ̂j = −∞ if JI = ∅. Thus,

P (Tn > cSN,L
(α)) =
n





 P ({Tn > cSN,L
ˆ
(α)} ∩ {{JI ⊆ JL } ∩ {∩j∈JIc {µ̂j ≤ 0}}})+ 
n



 P ({Tn > cSN,L
(α)} ∩ {{JI 6⊆ JˆL } ∪ {∪j∈JIc {µ̂j > 0}}}) 
n


√
√
n
n|µ̂s | o
nµ̂j
ˆ
, max
> cSN
≤ P max max
n (α, |JI |) + P (JI 6⊆ JL ) + P (∪j∈JIc {µ̂j > 0})
j∈JI
σ̂j s=p+1,...,k σ̂s




2+δ
−δ/2
−1
2+δ


αKn
Mn,2+δ (1 + Φ (1 − α/(2k − p))) +
≤α+


2
 4p exp(−2−1 nδ/(2+δ) /Mn,2+δ
)[1 + K(Mn,2+δ /nδ/(2(2+δ)) + 1)2+δ ] + 2K̃n−c 

≤ α + o(1),

(A.16)

where the third line uses Lemma A.4 and steps 1 and 2, and the convergence in the last line holds uniformly in the
manner required by the result.

A.3

Results for the bootstrap approximation

Lemma A.5. Assume Assumptions A.1, A.4, α ∈ (0, 0.5), and that H0 holds. For any non-stochastic set L ⊆
{1, . . . , p}, define:
n
√
Tn (L) ≡ max max nµ̂j /σ̂j ,
j∈L

max

s=p+1,...,k

√

o
n|µ̂s |/σ̂s ,

and let cB
n (L, α) with B ∈ {M B, EB} denote the conditional (1 − α)-quantile based on the bootstrap. Then,
−c̃
P (Tn (L) > cB
,
n (L, α)) ≤ α + C̃n

where c̃, C̃ > 0 are positive constants that only depend on the constants c, C in Assumption A.4. Furthermore, if
µj = 0 for all j ∈ L then:
−c̃
|P (Tn (L) > cB
.
n (L, α)) − α| ≤ C̃n

Finally, since c̃, C̃ depend only on the constants c, C in Assumption A.4, the proposed bounds are uniform in all
parameters θ ∈ Θ and distributions P that satisfy the assumptions in the statement.
Proof. In the absence of moment equalities equalities, this results follow from replacing {1, . . . , p} with L in CCK14
(proof of Theorem 4.3). As we show next, our proof can be completed by simply redefining the set of moment
inequalities by adding the moment equalities as two sets of inequalities with reversed sign.

40

Define A = A(L) ≡ L ∪ {p + 1, . . . , k} ∪ {k + 1, . . . , 2k − p} with |A| = |L| + 2(k − p) and for any i = 1, . . . , n,
define the following |A|-dimensional auxiliary data vector:

XiE ≡ {Xij }0j∈L , {Xis }0s=p+1,...,k , {−Xis }0s=p+1,...,k

0

.

Based on these definitions, we modify all expressions analogously, e.g.,
µE

=

{{µj }0j∈L , {µs }0s=p+1,...,k , {−µs }0s=p+1,...,k }0 ,

σE

=

{{σj }0j∈L , {σs }0s=p+1,...,k , {σs }0s=p+1,...,k }0 ,

and notice that H0 is equivalently re-written as µE ≤ 0|A| .
In the new notation, the test statistic is re-written as Tn (L) = maxj∈A

√ E E
nµ̂j /σ̂j , and the critical values can

re-written analogously. In particular, the MB and EB test statistics are respectively defined as follows:
WnM B (L)
WnEB (L)

n
√ X
E
E
i (Xij
− µ̂E
n
j )/σ̂j ,

=

max

=

n
√ X
∗,E
E
max n
(Xij
− µ̂E
j )/σ̂j .

j∈A

i=1

j∈A

i=1

Given this setup, the result follows immediately from CCK14 (Theorem 4.3).
Proof of Theorem 4.3. This result follows from Lemma A.5 with |L| = {1, . . . , p}.
Lemma A.6. For any α ∈ (0, 0.5), n ∈ N, B ∈ {M B, EB}, and L1 ⊆ L2 ⊆ {1, . . . , p},
B
cB
n (L1 , α) ≤ cn (L2 , α).


−c
Furthermore, under the above assumptions, P cB
, where c, C are universal constants.
n (L1 , α) ≥ 0 ≥ 1 − Cn
B
Proof. By definition, L1 ⊆ L2 implies that WnB (L1 ) ≤ WnB (L2 ) which, in turn, implies cB
n (L1 , α) ≤ cn (L2 , α).

We now turn to the second result. If the model has at least one moment equality, then WnB (L1 ) ≥ 0 and so
cB
n (α, L1 ) ≥ 0. If the model has no moment equalities, then we consider consider a different argument depending on
the type of bootstrap procedure being implemented.
√ P
First, consider MB. Conditionally on the sample, WnM B (L1 ) = maxj∈L (1/ n) n
i=1 i (Xij − µ̂j )/σ̂j is the maxB
imum of L1 zero mean Gaussian random variables. Thus, α ∈ (0, 0.5) implies that cM
(α, L1 ) ≥ 0.
n

Second, consider EM. Let c0 (L1 , α) denote the (1 − α)-quantile of maxj∈L1 Yj with {Yj }j∈L1 ∼ N (0, E[Z̃ Z̃ 0 ])
with Z̃ = {Zj }j∈L1 and Z as in Assumption A.3. At this point, we apply CCK14 (Eq. (66)) to our hypothetical
model with the moment inequalities indexed by L1 . Applied to this model, their Eq. (66) yields:

P cEB
≥ 1 − Cn−c ,
n (L1 , α) ≥ c0 (L1 , α + γn )

41

(A.17)

√
where γn ≡ ζn2 + νn + 8ζn1 log p ∈ (0, 2Cn−c ), for sequences {(ζn1 , ζn2 , νn )}n≥1 and universal positive constants
(c, C), all specified in CCK14. Since α < 0.5 and γn < 2Cn−c , it follows that for all n sufficiently large, α + γn < 0.5
and so c0 (α + γn , L1 ) > 0. The desired result follows from combining this with Eq. (A.17).
Proof of Theorem 4.4. This proof follows similar steps than CCK14 (Proof of Theorem 4.4). Let us define the
sequence of sets:
JI ≡ {j = 1, . . . , p : µj /σj ≥ −3λn /4}
We divide the proof into three steps. Steps 1-2 are exactly as in the proof of Theorem 4.2 so they are omitted.
Step 3. Defining Tn (JI ) as in Lemma A.5 and consider the following derivation:
n
o n
o n
o n
o

ˆ
ˆ
Tn > cB
∩ cB
n (JL , α) ∩ JI ⊆ JL ∩ ∩j∈JIc µ̂j ≤ 0
n (JI , α) ≥ 0
o n
o
n
o n

∩ cB
⊆ Tn > cB
n (α, JI ) ≥ 0
n (JI , α) ∩ ∩j∈JIc µ̂j ≤ 0
n
o
⊆ Tn (JI ) > cB
n (JI , α) ,

where the first inclusion follows from Lemma A.6, and the second inclusion follows from noticing that ∩j∈JIc {µ̂j ≤ 0}
B
and {Tn > cB
n (α, JI ) ≥ 0} implies that {Tn (JI ) > cn (α, JI )}. Thus,



ˆ
P Tn > cB,L
(α) = P Tn > cB
n
n (JL , α)
(
=

)

B
ˆ
ˆ
P {Tn > cB
n (JL , α)} ∩ {{JI ⊆ JL } ∩ {∩j∈JIc {µ̂j ≤ 0}} ∩ {cn (α, JI ) ≥ 0}} +

B
ˆ
ˆ
P {Tn > cB
n (JL , α)} ∩ {{JI 6⊆ JL } ∪ {∪j∈JIc {µ̂j > 0}} ∪ {cn (α, JI ) < 0}}

B
ˆ
≤ P (Tn (JI ) > cB
n (JI , α)) + P (JI 6⊆ JL ) + P (∪j∈JIc {µ̂j > 0}) + P (cn (α, JI ) < 0)
−2
)[1 + K(n−δ/(2(2+δ)) Mn,2+δ + 1)2+δ ] + 2K̃n−c
≤ α + Cn−c + C̃n−c̃ + 4p exp(−2−1 nδ/(2+δ) Mn,2+δ

≤ α + o(1),

(A.18)

where the convergence in the last line is uniform in the manner required by the result. The third line of Eq. (A.18)
uses Lemmas A.5 and A.6 as well as steps 1 and 2.
We next turn to the second part of the result. By the case under consideration, µ = 0p and so JI = {1, . . . , p}.
Thus, in this case, {JI ⊆ JˆL } = {JˆL = JI = {1, . . . , p}}. By this and step 2 of Theorem 4.2, it follows that:

2
P JˆL = JI = {1, . . . , p} ≥ 1 − 2p exp(−2−1 nδ/(2+δ) /Mn,2+δ
)[1 + K(Mn,2+δ /nδ/(2(2+δ)) + 1)2+δ ] + K̃n−c , (A.19)
where K, K̃ are uniform constants. In turn, notice that {JˆL = JI = {1, . . . , p}} implies that cB,1S
(α) = cB
n
n (JI , α) =

42

B,L
ˆ
cB
(α). Thus,
n (JL , α) = cn

P (Tn > cB,L
(α))
n

=

P ({Tn > cB,L
(α)} ∩ {JˆL = JI = {1, . . . , p}}) + P ({Tn > cB,L
(α)} ∩ {JˆL = JI = {1, . . . , p}}c )
n
n

≥

B,1S
P ({Tn > cn
(α)} ∩ {JˆL = JI = {1, . . . , p}})

≥

P (Tn > cB,1S
(α)) − P ({JˆL = JI = {1, . . . , p}}c )
n

≥

α − 2C̃n−c̃ ,

(A.20)

where the last inequality uses the second result in Theorem 4.3, Eq. (4.6), and Eq. (A.19). If we combine this with
Eq. (A.18), the result follows.

A.4

Results for power comparison

Proof of Theorem 5.1. The arguments in the main text show that Eq. (5.2) implies Eq. (5.3). To complete the proof,
it suffices to show that the two sufficient conditions imply Eq. (5.2). By definition and Lemma 3.2,
JˆSN

=

√
{j = 1, . . . , p : µ̂j /σ̂j ≥ −2cSN,1S
(βn )/ n},
n

JˆL

=

{j = 1, . . . , p : µ̂j,L /σ̂j ≥ −λn } = {j = 1, . . . , p : µ̂j /σ̂j ≥ −λn 3/2},

Condition 1. We show this by contradiction, i.e., suppose that Eq. (5.4) and JˆL 6⊆ JˆSN hold. By the latter,
√
√
c
, i.e., −2cSN,1S
(βn )/ n > µ̂j /σ̂j ≥ −λn 3/2, which implies that cSN,1S
∃j = 1, . . . , p s.t. j ∈ JˆL ∩ JˆSN
(βn )4/3 < nλn ,
n
n
contradicting Eq. (5.4).
SN,1S
Condition 2. By definition, cn
(βn )4/3 ≥

√
nλn is equivalent to

Φ−1 (1 − βn /p)

2

≥ nλ2n

9
.
16

(A.21)

The remainder of the proof shows that Eq. (A.21) holds under Eq. (5.5).
First, we establish a lower bound for the LHS of Eq. (A.21). For any x ≥ 1, consider the following inequalities:

1 − Φ(x) ≥

2
2
1
1
1 1 −x2 /2
1
√ e−x /2 ≥
√ e
≥ √ e−x ,
x + 1/x 2π
2x 2π
2 2π

where the first inequality holds for all x > 0 by Gordon (1941, Eq. (10)), the second inequality holds by x ≥ 1 and
so x > 1/x, and the third inequality holds by e−x
Φ

−1

2

/2

≤ 1/x for all x > 0. Note that for βn ≤ 10% and p ≥ 1,

(1 − βn /p) ≥ 1. Evaluating the previous display at x = Φ−1 (1 − βn /p) yields:
Φ−1 (1 − βn /p)

2

≥ ln

43




p
√
.
2 2πβn

(A.22)

Second, we establish an upper bound for the RHS of Eq. (A.21). By Eq. (3.4),
nλ2n = (4/3 + ε)2

n
−2
≤ 2(4/3 + ε)2 nδ/(2+δ) Mn,2+δ
,
2
n2/(2+δ) Mn,2+δ
−1

(A.23)

2
where the last inequality used that 1/(x − 1) ≤ 2/x for x ≥ 2 and that n2/(2+δ) Mn,2+δ
≥ 2. Thus,

9
18
9
−2
−2
nλ2n ≤
(4/3 + ε)2 nδ/(2+δ) Mn,2+δ
= (4/3 + ε)2 nδ/(2+δ) Mn,2+δ
.
16
16
8

(A.24)

To conclude the proof, notice that Eq. (A.21) follows directly from combining Eqs. (5.5), (A.22), and (A.24).
Proof of Theorem 5.2. This result has several parts.
Part 1: The same arguments used for SN method imply that Eq. (5.6) implies Eq. (5.7).
Part 2: By definition and Lemma 3.2,
JˆB

=

√
{j = 1, . . . , p : µ̂j /σ̂j ≥ −2cB,1S
(βn )/ n},
n

JˆL

=

{j = 1, . . . , p : µ̂j,L /σ̂j ≥ −λn } = {j = 1, . . . , p : µ̂j /σ̂j ≥ −λn 3/2},

√
c
s.t. −2cB
Suppose that JˆL ⊆ JˆB does not occur, i.e., ∃j ∈ JˆL ∩ JˆB
n (βn )/ n > µ̂j /σ̂j ≥ −λn 3/2. From this, we
conclude that:
√
ˆ
ˆ
{cB
n (βn )4/3 ≥ λn n} ⊆ {JL ⊆ JB }.
Let c0 (3βn ) denote the (1 − 3βn )-quantile of max1≤j≤p Yj with (Y1 , . . . , Yp ) ∼ N (0, E[ZZ 0 ]) with Z as in Assumption A.3. In the remainder of this step, we consider two strategies to establish the following result:
√
c0 (3βn )4/3 ≥ λn n.

(A.25)

Under Eq. (A.25), we can conclude that:
√
B
ˆ
ˆ
{cB
n (βn ) ≥ c0 (3βn )} ⊆ {cn (βn )4/3 ≥ λn n} ⊆ {JL ⊆ JB }.

From this and since c0 (·) is decreasing, we conclude that for any µn ≤ 3βn ,
P JˆL ⊆ JˆB



≥ P cB
n (βn ) ≥ c0 (3βn )




≥ P cB
n (βn ) ≥ c0 (µn ) .

(A.26)

To complete the proof, it suffices to provide a uniformly high lower bound for the RHS of Eq. (A.26). To this end,
√
we consider CCK14 (Eq. (66)) at the following values: α = βn , νn = Cn−c , and (ζn2 , ζn1 ) s.t. ζn2 +8ζn1 ln p ≤ Cn−c .
√
Under our assumptions, these choices yield µn ≡ βn + ζn2 + vn + 8ζn1 log p ≤ βn + 2Cn−c ≤ 3βn . By plugging these
on CCK14 (Eq. (66)), the RHS of Eq. (A.26) exceeds 1 − Cn−c , as desired.

44

To complete the proof the step, we now describe the two strategies that can be used to show Eq. (A.25). The
first strategy relies on Eq. (5.9) and the second strategy relies on Eq. (5.10).
Strategy 1. By definition,
c0 (3βn ) ≥ Φ−1 (1 − 3βn ),

(A.27)

By combining Eqs. (5.9), (A.23), and (A.27), it follows that:
c0 (3βn )4/3 ≥ Φ−1 (1 − 3βn )4/3 ≥

√
√
−1
2(4/3 + ε)nδ/(2(2+δ)) Mn,2+δ
≥ nλn .

Strategy 2. First, the Borell-Cirelson-Sudakov inequality (see, e.g., Boucheron et al. (2013, Theorem 5.8)), implies
that for x ≥ 0,
P




2
max Yj ≤ E[ max Yj ] − x ≤ e−x /2 ,

1≤j≤p

1≤j≤p

(A.28)

where we used that the diagonal E[ZZ 0 ] is a vector of ones. Equating the RHS of Eq. (A.28) to (1 − 3βn ) yields
p
x = 2 log(1/[1 − 3βn ]) such that:

c0 (3βn ) ≥ E[ max Yj ] −
1≤j≤p

p
2 log(1/[1 − 3βn ]).

(A.29)

We now provide a lower bound for the first term on the RHS of Eq. (A.29). Consider the following derivation:

E[ max Yj ] ≥ min
1≤j≤p

i6=j

p
p
E(Yi − Yj )2 log(p)/2 ≥
2(1 − ρ) log(p)/2,

(A.30)

where the first inequality follows from Sudakov’s minorization inequality (see, e.g., Boucheron et al. (2013, Theorem
13.4)) and the second inequality follows from E[ZZ 0 ] having a diagonal elements equal to one and the maximal
absolute correlation less that ρ. Eqs. (A.29)-(A.30) imply that:

c0 (3βn ) ≥

p
p
(1 − ρ) log(p)/2 − 2 log(1/[1 − 3βn ]).

By combining Eqs. (5.10), (A.23), and (A.31), it follows that:

c0 (3βn )4/3 ≥ 4/3(

p
p
√
√
−1
(1 − ρ) log(p)/2 − 2 log(1/[1 − 3βn ])) ≥ 2(4/3 + ε)nδ/(2(2+δ)) Mn,2+δ
≥ nλn .

Part 3: Consider the following argument.
P (Tn ≥ cB,2S
(α)) = P (Tn ≥ cB,2S
(α) ∩ JˆL ⊆ JˆB ) + P (Tn ≥ cB,2S
(α) ∩ JˆL 6⊆ JˆB )
n
n
n
≤ P (Tn ≥ cB,L
(α)) + P (JˆL 6⊆ JˆB )
n
≤ P (Tn ≥ cB,L
(α)) + Cn−c ,
n

45

(A.31)

where the first inequality uses part 1, and the second inequality uses that the sufficient conditions imply Eq. (5.8).

References
Andrews, D. W. K., S. Berry, and P. Jia-Barwick (2004): “Confidence Regions for Parameters in
Discrete Games with Multiple Equilibria with an Application to Discount Chain Store Location,” Mimeo:
Yale University and M.I.T.
Andrews, D. W. K. and P. Guggenberger (2009): “Validity of Subsampling and “Plug-in Asymptotic”
Inference for Parameters Defined by Moment Inequalities,” Econometric Theory, 25, 669–709.
Andrews, D. W. K. and P. Jia-Barwick (2012): “Inference for Parameters Defined by Moment Inequalities: A Recommended Moment Selection Procedure,” Econometrica, 80, 2805–2826.
Andrews, D. W. K. and X. Shi (2013): “Inference Based on Conditional Moment Inequalities,” Econometrica, 81, 609–666.
Andrews, D. W. K. and G. Soares (2010): “Inference for Parameters Defined by Moment Inequalities
Using Generalized Moment Selection,” Econometrica, 78, 119–157.
Armstrong, T. B. (2014a): “Asymptotically Exact Inference in Conditional Moment Inequality Models,”
Journal of Econometrics, 186, 51–65.
——— (2014b): “Weighted KS Statistics for Inference on Conditional Moment Inequalities,” Journal of
Econometrics, 181, 92–116.
Belloni, A. and V. Chernozhukov (2011): High dimensional sparse econometric models: An introduction, Springer.
Beresteanu, A. and F. Molinari (2008): “Asymptotic Properties for a Class of Partially Identified
Models,” Econometrica, 76, 763–814.
Bontemps, C., T. Magnac, and E. Maurin (2012): “Set Identified Linear Models,” Econometrica, 80,
1129–1155.
Boucheron, S., G. Lugosi, and P. Massart (2013): Concentration Inequalities: A Nonasymptotic
Theory of Independence, Oxford University Press.

46

Bugni, F. A. (2010): “Bootstrap Inference in Partially Identified Models Defined by Moment Inequalities:
Coverage of the Identified Set,” Econometrica, 78, 735–753.
——— (2015): “A comparison of inferential methods in partially identified models in terms of error in
coverage probability (Formerly circulated as “Bootstrap Inference in Partially Identified Models Defined by
Moment Inequalities: Coverage of the Elements of the Identified Set”),” Econometric Theory, FirstView,
1–56.
Bugni, F. A., I. A. Canay, and P. Guggenberger (2012): “Distortions of Asymptotic Confidence Size
in Locally Misspecified Moment Inequality Models,” Econometrica, 80, 1741–1768.
Bühlmann, P. and S. van de Geer (2011): Statistics for high-dimensional data: methods, theory and
applications, Springer Science & Business Media.
Canay, I. A. (2010): “E.L. Inference for Partially Identified Models: Large Deviations Optimality and
Bootstrap Validity,” Journal of Econometrics, 156, 408–425.
Caner, M. (2009): “Lasso Type GMM Estimator,” Econometric Theory, 25, 270–290.
Caner, M. and Q. M. Fan (2015): “Hybrid GEL Estimators: Instrument Selection with Adaptive Lasso,”
Journal of Econometrics, 187, 256–274.
Caner, M., X. Han, and Y. Lee (2016): “Adaptive elastic net GMM estimation with many Invalid
Moment Conditions: Simultaneous Model and Moment Selection,” Forthcoming: Journal of Business and
Economics Statistics.
Caner, M. and H. Zhang (2014): “Adaptive Elastic Net GMM with Diverging Number of Moments,”
Journal of Business and Economics Statistics, 32, 30–47.
Cheng, X. and Z. Liao (2015): “Select the Valid and Relevant Moment Conditions: A One-step Procedure
for GMM with Many Moments,” Journal of Econometrics, 186, 443–464.
Chernozhukov, V., D. Chetverikov, and K. Kato (2013a): “Comparison and anti-concentration
bounds for maxima of Gaussian random vectors,” Working paper. Forthcoming in Probability Theory
Related Fields.
——— (2013b): “Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional
random vectors,” Annals of Statistics, 41, 2786–2819.

47

——— (2014a): “Anti-concentration and honest, adaptive confidence bands,” Annals of Statistics, 42, 1787–
1818.
——— (2014b): “Central Limit Theorems and Bootstrap in High Dimensions,” Working paper.
——— (2014c): “Testing Many Moment Inequalities,” Working paper.
Chernozhukov, V., H. Hong, and E. Tamer (2007): “Estimation and Confidence Regions for Parameter
Sets in Econometric Models,” Econometrica, 75, 1243–1284.
Chernozhukov, V., S. Lee, and A. M. Rosen (2013c): “Intersection Bounds: Estimation and Inference,”
Econometrica, 81, 667–737.
Chetverikov, D. (2013): “Adaptive Test of Conditional Moment Inequalities,” Unpublished manuscript.
Fan, J., Y. Liao, and J. Yao (2015): “Power Enhancement in High-Dimensional Cross-Sectional Tests,”
Econometrica, 84, 1496–1541.
Fan, J., J. Lv, and L. Qi (2011): “Sparse high dimensional models in economics,” Annual review of
economics, 3, 291–317.
Galichon, A. and M. Henry (2006): “Inference in Incomplete Models,” Mimeo: Ecole Polytechnique,
Paris - Department of Economic Sciences and Pennsylvania State University.
——— (2013): “Dilation Bootstrap: A methodology for constructing confidence regions with partially identified models,” Journal of Econometrics, 177, 109–115.
Gordon, R. D. (1941): “Values of Mills’ ratio of area to bounding ordinate and of the normal probability
integral for large values of the argument,” The Annals of Mathematical Statistics, 12, 364–366.
Hastie, T., R. Tibshirani, and M. Wainwright (2015): Statistical Learning with Sparsity: The Lasso
and Generalizations, CRC Press.
Imbens, G. and C. F. Manski (2004): “Confidence Intervals for Partially Identified Parameters,” Econometrica, 72, 1845–1857.
Kim, K. (2008): “Set Estimation and Inference with Models Characterized by Conditional Moment Inequalities,” Mimeo: Michigan State University.
Liao, Z. (2013): “Adaptive GMM Shrinkage Estimation with Consistent Moment Selection,” Econometric
Theory, 29, 1–48.
48

Manski, C. F. (1995): Identification Problems in the Social Sciences, Harvard University Press.
Menzel, K. (2009): “Essays of set estimation and inference with moment inequalities,” Ph.D. Thesis
dissertation, MIT.
——— (2014): “Consistent estimation with many moment inequalities,” Journal of Econometrics, 182,
329350.
Pakes, A., J. Porter, K. Ho, and J. Ishii (2015): “Moment Inequalities and Their Application,”
Econometrica, 83, 315–334.
Ponomareva, M. (2010): “Inference in Models Defined by Conditional Moment Inequalities with Continuous Covariates,” Mimeo: Northern Illinois University.
Romano, J. P. and A. M. Shaikh (2008): “Inference for Identifiable Parameters in Partially Identified
Econometric Models,” Journal of Statistical Planning and Inference, 138, 2786–2807.
——— (2010): “Inference for the Identified Set in Partially Identified Econometric Models,” Econometrica,
78, 169–211.
Romano, J. P., A. M. Shaikh, and M. Wolf (2014): “A Practical Two-Step Method for Testing Moment
Inequalities,” Econometrica, 82, 1979–2002.
Rosen, A. M. (2008): “Confidence Sets for Partially Identified Parameters that Satisfy a Finite Number of
Moment Inequalities,” Journal of Econometrics, 146, 107–117.
Stoye, J. (2009): “More on Confidence Intervals for Partially Identified Parameters,” Econometrica, 77,
299–1315.
Tamer, E. (2003): “Incomplete Simultaneous Discrete Response Model with Multiple Equilibria,” Review
of Economic Studies, 70, 147–165.
Tibshirani, R. (1996): “Regression shrinkage and selection via the lasso,” Journal of the Royal Statistical
Society, Series B (Methodological), 58, 267–288.
Wang, H., B. Li, and C. Leng (2009): “Shrinkage tuning parameter selection with a diverging number
of parameters,” J. R. Statist. Soc. B, 71, 671–683.

49

