Confidence Intervals for
Projections of Partially Identified Parameters∗

arXiv:1601.00934v2 [math.ST] 1 Nov 2017

Hiroaki Kaido†

Francesca Molinari‡

Jörg Stoye§

October 25, 2017
Abstract
We propose a bootstrap-based calibrated projection procedure to build confidence intervals for single components and for smooth functions of a partially identified parameter
vector in moment (in)equality models. The method controls asymptotic coverage uniformly over a large class of data generating processes.
The extreme points of the calibrated projection confidence interval are obtained by
extremizing the value of the component (or function) of interest subject to a proper
relaxation of studentized sample analogs of the moment (in)equality conditions. The
degree of relaxation, or critical level, is calibrated so that the component (or function)
of θ, not θ itself, is uniformly asymptotically covered with prespecified probability. This
calibration is based on repeatedly checking feasibility of linear programming problems,
rendering it computationally attractive.
Nonetheless, the program defining an extreme point of the confidence interval is generally nonlinear and potentially intricate. We provide an algorithm, based on the response
surface method for global optimization, that approximates the solution rapidly and accurately. The algorithm is of independent interest for inference on optimal values of
stochastic nonlinear programs. We establish its convergence under conditions satisfied by
canonical examples in the moment (in)equalities literature.
Our assumptions and those used in the leading alternative approach (a profiling based
method) are not nested. An extensive Monte Carlo analysis confirms the accuracy of
the solution algorithm and the good statistical as well as computational performance of
calibrated projection, including in comparison to other methods.
Keywords: Partial identification; Inference on projections; Moment inequalities; Uniform inference.

∗

We are grateful to Elie Tamer and three anonymous reviewers for very useful suggestions that substantially improved the paper. We thank for their comments Ivan Canay and seminar and conference participants
at Bonn, BC/BU joint workshop, Brown, Cambridge, Chicago, Columbia, Cornell, CREST, Kobe, Maryland,
Michigan, Michigan State, NYU, Penn State, Royal Holloway, Syracuse, Toronto, UCLA, UCSD, UPenn, Vanderbilt, Vienna, Yale, Wisconsin, CEME, ES-NAWM 2015, Frontiers of Theoretical Econometrics Conference,
ES-World Congress 2015, ES-Asia Meeting 2016, KEA-KAEA International Conference, Verein für Socialpolitik Ausschuss für Ökonometrie, and ES-ESM 2017. We are grateful to Zhonghao Fu, Debi Mohapatra, Sida
Peng, Talal Rahim, Matthew Thirkettle, and Yi Zhang for excellent research assistance. A MATLAB package
implementing the method proposed in this paper, Kaido, Molinari, Stoye, and Thirkettle (2017), is available
at https://molinari.economics.cornell.edu/programs/KMSportable_V3.zip. We are especially grateful to
Matthew Thirkettle for his contributions to this package. Finally, we gratefully acknowledge financial support
through NSF grants SES-1230071 (Kaido), SES-0922330 (Molinari), and SES-1260980 (Stoye).
†
Department of Economics, Boston University, hkaido@bu.edu.
‡
Department of Economics, Cornell University, fm72@cornell.edu.
§
Departments of Economics, Cornell University and University of Bonn, stoye@cornell.edu.

1

Introduction

This paper provides theoretically and computationally attractive confidence intervals for projections and smooth functions of a parameter vector θ P Θ Ă Rd , d ă 8, that is partially
or point identified through a finite number of moment (in)equalities. The values of θ that
satisfy these (in)equalities constitute the identification region ΘI .
Until recently, the rich literature on inference in this class of models focused on confidence
sets for the entire vector θ, usually obtained by test inversion as
Cn pc1´α q ” tθ P Θ : Tn pθq ď c1´α pθqu ,

(1.1)

where Tn pθq is a test statistic that aggregates violations of the sample analog of the moment
(in)equalities, and c1´α pθq is a critical value that controls asymptotic coverage, often uniformly over a large class of data generating processes (DGPs). In point identified moment
equality models, this would be akin to building confidence ellipsoids for θ by inversion of the
F -test statistic proposed by Anderson and Rubin (1949).
However, applied researchers are frequently primarily interested in a specific component
(or function) of θ, e.g., the returns to education. Even if not, they may simply want to report
separate confidence intervals for components of a vector, as is standard practice in other
contexts. Thus, consider the projection p1 θ, where p is a known unit vector. To date, it has
been common to report as confidence interval for p1 θ the projection of Cn pc1´α q:
«
CInproj

“

ff
1

inf

p θ,

θPCn pc1´α q

sup

1

pθ ,

(1.2)

θPCn pc1´α q

where n denotes sample size; see for example Ciliberto and Tamer (2009), Grieco (2014) and
Dickstein and Morales (2016). Such projection is asymptotically valid, but typically yields
conservative and therefore needlessly large confidence intervals. The potential severity of this
?
effect is easily appreciated in a point identified example. Given a n-consistent estimator
θ̂n P Rd with limiting covariance matrix equal to the identity matrix, a 95% confidence
interval for θk is obtained as θ̂n,k ˘ 1.96, k “ 1, . . . , d. In contrast, if d “ 10, then projection
of a 95% Wald confidence ellipsoid yields θ̂n,k ˘ 4.28 with true coverage of essentially 1. We
refer to this problem as projection conservatism.
Our first contribution is to provide a bootstrap-based calibrated projection method that
largely anticipates and corrects for projection conservatism. For each candidate θ, ĉn pθq is
calibrated so that across bootstrap repetitions the projection of θ is covered with at least some
pre-specified probability. Computationally, this bootstrap is relatively attractive because we
linearize all constraints around θ, so that coverage of p1 θ corresponds to the projection of a

[1]

stochastic linear constraint set covering zero. We then propose the confidence interval
«
CIn ”

ff
p1 θ, sup p1 θ .

inf
θPCn pĉn q

(1.3)

θPCn pĉn q

We prove that CIn asymptotically covers p1 θ with probability at least 1 ´ α uniformly over
a large class of DGPs and that it is weakly shorter than (1.2) for each n.1 We also provide
simple conditions under which it is asymptotically strictly shorter.
Our second contribution is a general method to accurately and rapidly compute projectionbased confidence intervals. These can be our calibrated projection confidence intervals, but
they can also correspond to projection of many other methods for inference on either θ
or its identified set ΘI . Examples include Chernozhukov, Hong, and Tamer (2007), Andrews and Soares (2010), or (for conditional moment inequalities) Andrews and Shi (2013).
Projection-based inference extends well beyond its application in partial identification, hence
our computational method proves useful more broadly. For example, Freyberger and Reeves
(2017a,b, Section S.3) use it to construct uniform confidence bands for an unknown function
of interest under (nonparametric) shape restrictions.
We propose an algorithm that is based on the response surface method, thus it resembles
an expected improvement algorithm (see e.g. Jones, 2001; Jones, Schonlau, and Welch, 1998,
and references therein). Bull (2011) established convergence of the expected improvement
algorithm for unconstrained optimization problems where the objective is a “black box”
function. Building on his results, we show convergence of our algorithm for constrained
optimization problems in which the constraint functions are “black box” functions, assuming
that they are sufficiently smooth. We then verify this smoothness condition for canonical
examples in the moment (in)equalities literature. Our extensive Monte Carlo experiments
confirm that the algorithm is fast and accurate.2
Previous implementations of projection-based inference were based on approximating the
set Cn pc1´α q Ă Rd by searching for vectors θ P Θ such that Tn pθq ď c1´α pθq (using, e.g., gridsearch or simulated annealing with no cooling), and reporting the smallest and largest value
of p1 θ among parameter values that were “guessed and verified” to belong to Cn pc1´α q. This
becomes computationally cumbersome as d increases because it typically requires a number of
evaluation points that grows exponentially with d. In contrast, our method typically requires
a number of evaluation points that grows linearly with d.
The main alternative inference prodedure for projections was introduced in Romano and
Shaikh (2008) and significantly advanced in Bugni, Canay, and Shi (2017, BCS henceforth).
It is based on profiling out a test statistic. The classes of DGPs for which our procedure and
1

This comparison is based on projection of the confidence set of Andrews and Soares (2010) and holds the
choice of tuning parameters and criterion function in (1.2) and (1.3) constant across methods.
2
Freyberger and Reeves (2017b, Section S.3) similarly find our method to be accurate and to considerably
reduce computational time.

[2]

the profiling-based method of BCS (BCS-profiling henceforth) can be shown to be uniformly
valid are non-nested. We show that in well behaved cases, calibrated projection and BCSprofiling are asymptotically equivalent. We also provide conditions under which calibrated
projection has lower probability of false coverage, thereby establishing that the two methods’
power properties are non-ranked. Computationally, calibrated projection has the advantage
that the bootstrap iterates over linear as opposed to nonlinear programming problems. While
the “outer” optimization problems in (1.3) are potentially intricate, our algorithm is geared
toward them. Our Monte Carlo simulations suggest that these two factors give calibrated
projection a considerable computational edge over BCS-profiling, with an average speed gain
of about 78-times.
In an influential paper, Pakes, Porter, Ho, and Ishii (2011) also use linearization but,
subject to this approximation, directly bootstrap the sample projection.3 This is valid only
under stringent conditions, and we show that calibrated projection can be much simplified
under those conditions. Other related papers that explicitly consider inference on projections
include Andrews, Berry, and Jia (2004), Beresteanu and Molinari (2008), Bontemps, Magnac,
and Maurin (2012), Chen, Tamer, and Torgovitsky (2011), Kaido (2016), Kitagawa (2012),
Kline and Tamer (2015), and Wan (2013). However, some are Bayesian, as opposed to
our frequentist approach, and none of them establish uniform validity of confidence sets.
Chen, Christensen, and Tamer (2017) establish uniform validity of MCMC-based confidence
intervals for projections, but these are aimed at covering the entire set tp1 θ : θ P ΘI pP qu,
whereas we aim at covering the projection of θ. Finally, Gafarov, Meier, and Montiel-Olea
(2016) have used our insight in the context of set identified spatial VARs.
Structure of the paper. Section 2 sets up notation and describes our approach in
detail. Section 3 describes computational implementation of the method and establishes
convergence of our proposed algorithm. Section 4 lays out our assumptions and, under these
assumptions, establishes uniform validity of calibrated projection for inference on projections
and smooth functions of θ. It also shows that more stringent conditions allow for several
simplifications to the method, including that it can suffice to evaluate ĉn at only two values
of θ and that one can dispense with a tuning parameter. The section closes with a formal
comparison of calibrated projection and BCS-profiling. Section 5 reports the results of Monte
Carlo simulations. Section 6 draws conclusions. The proof of convergence of our algorithm
is in Appendix A. All other proofs, background material for our algorithm, and additional
results are in the Online Appendix.4
3

The published version, i.e. Pakes, Porter, Ho, and Ishii (2015), does not contain the inference part.
Section B provides convergence-related results and background material for our algorithm and describes
how to compute ĉn pθq. Section C verifies, for a number of canonical moment (in)equality models, the assumptions that we invoke to show validity of our inference procedure and for our algorithm. Section D contains
proofs of the Theorems in this paper’s Section 4. Section E collects Lemmas supporting the preceding proofs.
Section F provides further comparisons with the profiling method of Bugni, Canay, and Shi (2017), including
an example where calibrated projection has higher power in finite sample. Section G provides comparisons with
“uncalibrated” projection of the confidence region in Andrews and Soares (2010), including simple conditions
4

[3]

2

Detailed Explanation of the Method

Let Xi P X Ď RdX be a random vector with distribution P , let Θ Ď Rd denote the parameter
space, and let mj : X ˆ Θ Ñ R for j “ 1, . . . , J1 ` J2 denote measurable functions characterizing the model, known up to parameter vector θ P Θ. The true parameter value θ is
assumed to satisfy the moment inequality and equality restrictions
EP rmj pXi , θqs ď 0, j “ 1, ¨ ¨ ¨ , J1

(2.1)

EP rmj pXi , θqs “ 0, j “ J1 ` 1, ¨ ¨ ¨ , J1 ` J2 .

(2.2)

The identification region ΘI pP q is the set of parameter values in Θ satisfying (2.1)-(2.2). For
a random sample tXi , i “ 1, ¨ ¨ ¨ , nu of observations drawn from P , we write
m̄n,j pθq ” n´1
σ̂n,j ” pn

řn

i“1 mj pXi , θq,

řn
´1

j “ 1, ¨ ¨ ¨ , J1 ` J2

2
i“1 rmj pXi , θqs

´ rm̄n,j pθqs2 q1{2 , j “ 1, ¨ ¨ ¨ , J1 ` J2

(2.3)
(2.4)

for the sample moments and the analog estimators of the population moment functions’
standard deviations σP,j .5
The confidence interval in (1.3) then becomes CIn “ r´sp´p, Cn pĉn qq, spp, Cn pĉn qqs, where
spp, Cn pĉn qq ” sup p1 θ s.t.
θPΘ

? m̄n,j pθq
n
ď ĉn pθq, j “ 1, ¨ ¨ ¨ , J
σ̂n,j pθq

(2.5)

and similarly for p´pq. Here, we define J ” J1 ` 2J2 moments, where m̄n,J1 `J2 `k pθq “
´m̄J1 `k pθq for k “ 1, ¨ ¨ ¨ , J2 . That is, we split moment equality constraints into two opposing
inequality constraints and relax them separately.6
For a class of DGPs P that we specify below, define the asymptotic size of CIn by
lim inf inf

inf

nÑ8 P PP θPΘI pP q

P pp1 θ P CIn q.

(2.6)

Our goal is to calibrate ĉn so that (2.6) is at least equal to a prespecified level 1 ´ α ě 1{2
while anticipating projection conservatism. To build intuition, fix pθ, P q s.t. θ P ΘI pP q, P P
under which CIn is asymptotically strictly shorter than CInproj .
5
M
Under Assumption 4.3-(II), in equation (2.5) instead of σ̂n,j we use the estimator σ̂n,j
specified in (E.188)
in Lemma E.10 p.52 of the Online Appendix for j “ 1, . . . , 2R1 (with R1 ď J1 {2 defined in the assumption).
In equation (3.2) we use σ̂n,j for all j “ 1, . . . , J. To ease notation, we distinguish the two only where needed.
6
For a simple analogy, consider the point identified model defined by the single moment
equality
?
EP pm1 pXi , θqq “ EP pXi q ´ θ “ 0, where θ is a scalar. In this case, Cn pĉ?
n q “ X̄ ˘ ĉn σ̂n { n. The upper
endpoint of the confidence interval can be written as supθ tp1 θ s.t. ´ ĉn ď npX̄ ´ θq{σ̂n ď ĉn u, with p “ 1,
and similarly for the lower endpoint.

[4]

P. The projection of θ is covered when
´ sp´p, Cn pĉn qq ď p1 θ ď spp, Cn pĉn qq
#
+
inf ϑ p1 ϑ

#

+

supϑ p1 ϑ

?
?
ď p1 θ ď
nm̄n,j pϑq
nm̄n,j pϑq
s.t. ϑ P Θ,
ď
ĉ
pϑq,
@j
s.t.
ϑ
P
Θ,
n
σ̂n,j pϑq
σ̂n,j pϑq ď ĉn pϑq, @j
,
$
$
?
?
/
’
’
sup
p1 λ
p1 λ
inf
.
&
&
´ λP ¯npΘ´θq
´ λP ¯npΘ´θq
?
?
´
¯
´
¯
nm̄n,j θ` ?λn
nm̄n,j θ` ?λn
ď0ď
ô
¯ ď ĉn θ ` ?λ
¯ ď ĉn θ ` ?λ
´
´
/
’
’
,
@j
, @j
s.t.
%
%s.t.
λ
λ
n
n

ô

σ̂n,j θ` ?n

σ̂n,j θ` ?n

(2.7)
?
where the second equivalence follows from substituting ϑ “ θ ` λ{ n and taking λ to be the
?
choice parameter. (Intuitively, we localize around θ at rate 1{ n.)
We control asymptotic size by finding ĉn such that 0 asymptotically lies within the optimal
values of the NLPs in (2.7) with probability 1 ´ α. To reduce computational burden, we
approximate the event in equation (2.7) through linear expansion in λ of the constraint set.
?
?
?
To each constraint j, we add and subtract nEP rmj pXi , θ ` λ{ nqs{σ̂n,j pθ ` λ{ nq and
apply the mean value theorem to obtain
ˆ
˙
?
λ
nm̄n,j θ` ?
n
ˆ
˙
λ
?
σ̂n,j θ`
n

Here Gn,j p¨q ”

?

ˆ

!
´
“
Gn,j θ `

?λ
n

¯
` DP,j pθ̄qλ `

?

˙

) σP,j θ` ?λ
n
ˆ
˙.
nγ1,P,j pθq
λ

(2.8)

σ̂n,j θ` ?
n

npm̄n,j p¨q ´ EP rmj pXi , ¨qsq{σP,j p¨q is a normalized empirical process indexed

by θ P Θ, DP,j p¨q ” ∇θ tEP rmj pXi , ¨qs{σP,j p¨qu is the gradient of the normalized moment,
γ1,P,j p¨q ” EP pmj pXi , ¨qq{σP,j p¨q is the studentized population moment, and the mean value
?
θ̄ lies componentwise between θ and θ ` λ{ n.7
Calibration of ĉn requires careful analysis of the local behavior of the moment restrictions at each point in the identification region. This is because the extent of projection
conservatism depends on (i) the asymptotic behavior of the sample moments entering the
inequality restrictions, which can change discontinuously depending on whether they bind at
θ (γ1,P,j pθq “ 0) or not, and (ii) the local geometry of the identification region at θ, i.e. the
shape of the constraint set formed by the moment restrictions, and its relation to the level
set of the objective function p1 θ. Features (i) and (ii) can be quite different at different points
in ΘI pP q, making uniform inference for the projection challenging. In particular, (ii) does
not arise if one only considers inference for the entire parameter vector, and hence is a new
challenge requiring new methods.8 This is where this paper’s core theoretical innovation lies.
7

The mean value θ̄ changes with j but we omit the dependence to ease notation.
This is perhaps best expressed in the testing framework: Inference for projections entails a null hypothesis
specifying the value of a single component (or a function) of θ. The components not under test become
additional nuisance parameters, and dealing with them presents challenges that one does not face when
testing hypotheses that specify the value of the entire vector θ.
8

[5]

,
/
.
,
/
-

An important component of this innovation is to add to (2.7) the constraint that λ P
ρB d ,

where B d “ r´1, 1sd and ρ ą 0 a tuning parameter. This is slightly conservative but

regularizes the effect of the local geometry of ΘI pP q at θ on the inference problem. See
Section 4.3 for further discussion. We show that the probability of the event in (2.7), with
λ restricted to be in ρB d , is asymptotically approximated by the probability that 0 lies
between the optimal values of two programs that are linear in λ. The constraint sets of these
programs are characterized by (i) a Gaussian process GP,j pθq evaluated at θ (that we can
approximate through a simple nonparametric bootstrap), (ii) a gradient DP,j pθq (that we
can uniformly consistently estimate9 on compact sets), and (iii) the parameter γ1,P,j pθq that
measures the extent to which each moment inequality is binding (that we can conservatively
estimate using insights from Andrews and Soares (2010)). This suggests a computationally
attractive bootstrap procedure based on linear programs.

3

Computing Calibrated Projection Confidence Intervals

3.1

Computing the Critical Level

For a given θ P Θ, we calibrate ĉn pθq through a bootstrap procedure that iterates over linear
programs.10 Define
Λbn pθ, ρ, cq “ tλ P

?

where Gbn,j p¨q “ n´1{2

npΘ ´ θq X ρB d : Gbn,j pθq ` D̂n,j pθqλ ` ϕj pξˆn,j pθqq ď c, j “ 1, . . . , Ju,

řn

b
i“1 pmj pXi , ¨q´ m̄n,j p¨qq{σ̂n,j p¨q

(3.1)

is a bootstrap analog of GP,j ,11 D̂n,j p¨q

is a consistent estimator of DP,j p¨q, ρ ą 0 is a constant chosen by the researcher (see Section
4.3), B d “ r´1, 1sd , and ξˆn,j is defined by
ξˆn,j pθq ”

$
&κ´1 ?nm̄

j “ 1, . . . , J1

%0

j “ J1 ` 1, . . . , J,

n

n,j pθq{σ̂n,j pθq

(3.2)

where κn is a user-specified thresholding sequence such that κn Ñ 8, ϕ : RJr˘8s Ñ RJr˘8s is
one of the generalized moment selection (GMS) functions proposed by Andrews and Soares
(2010), and Rr˘8s “ R Y t˘8u. A common choice of ϕ is given component-wise by

ϕj pxq “

$
&0

if x ě ´1

(3.3)

%´8 if x ă ´1.
Restrictions on ϕ and the rate at which κn diverges are imposed in Assumption 4.2.
9

See Online Appendix C for proposal of such estimators in some canonical moment (in)equality examples.
If Θ is defined through smooth convex (in)equalities, these can be linearized too.
ř
11
Bugni, Canay, and Shi (2017) approximate the stochastic process GP,j using n´1{2 n
i“1 rpmj pXi , ¨q ´
n
m̄n,j p¨qq{σ̂n,j p¨qsχi with tχi „ N p0, 1qui“1 i.i.d. This approximation is equally valid in our approach, and can
be computationally faster as it avoids repeated evaluation of mj pXib , ¨q across bootstrap replications.
10

[6]

Remark 3.1: For concreteness, in (3.3) we write out the “hard thresholding” GMS function. As we establish below, our results apply to all but one of the GMS functions in Andrews
and Soares (2010).12
Heuristically, the random convex polyhedral set Λbn pθ, ρ, cq in (3.1) is a local (to θ) linearized bootstrap approximation to the random constraint set in (2.7). To see this, note
that the bootstrapped empirical process and the estimator of the gradient approximate the
first two terms in the constraint in (2.7) as linearized in (2.8). Next, for θ P ΘI pP q, the
?
GMS function conservatively approximates the local slackness parameter nγ1,P,j pθq. This
?
is needed because the scaling of nγ1,P,j pθq precludes consistent estimation. The problem
is resolved by shrinking estimated intercepts toward zero, thereby tightening constraints and
hence increasing ĉn pθq. As with other uses of GMS, the resulting conservative distortion
vanishes pointwise but not uniformly. Finally, restricting λ to the “ρ-box” ρB d has a strong
regularizing effect: It ensures uniform validity in challenging situations, including several that
are assumed away in most of the literature. We discuss this point in more detail in Section
4.3.
The critical level ĉn pθq to be used in (1.3) is the smallest value of c that makes the
bootstrap probability of the event
min

λPΛbn pθ,ρ,cq

p1 λ ď 0 ď

max

λPΛbn pθ,ρ,cq

p1 λ

(3.4)

at least 1 ´ α. Because Λbn pθ, ρ, cq is convex, we have
"
min

λPΛbn pθ,ρ,cq

p1 λ ď 0 ď

max

λPΛbn pθ,ρ,cq

*
!
)
p1 λ ðñ Λbn pθ, ρ, cq X tp1 λ “ 0u ‰ H ,

so that we can equivalently define
ĉn pθq ” inftc P R` : P ˚ pΛbn pθ, ρ, cq X tp1 λ “ 0u ‰ Hq ě 1 ´ αu,

(3.5)

where P ˚ denotes the law of the random set Λbn pθ, ρ, cq induced by the bootstrap sampling
process, i.e. by the distribution of pX1b , . . . , Xnb q, conditional on the data. Importantly, P ˚
can be assessed by repeatedly checking feasibility of a linear program.13 We describe in detail
in Online Appendix B.4 how we compute ĉn pθq through a root-finding algorithm.
?
These are ϕ1 ´ ϕ4 in Andrews and Soares (2010), all of which depend on κ´1
nm̄n,j pθq{σ̂n,j pθq. We do
n
not consider GMS function ϕ5 in Andrews and Soares (2010), which depends also on the covariance matrix of
the moment functions.
13
We implement a program in Rd for simplicity but, because p1 λ “ 0 defines a linear subspace, one could
reduce this to Rd´1 .
12

[7]

3.2

Computation of the Outer Maximization Problem

Projection based methods as in (1.2) and (1.3) have nonlinear constraints involving a critical
value which in general is an unknown function of θ. Moreover, in all methods, including ours
and Andrews and Soares (2010), the gradients of the critical values with respect to θ are
not available in closed form. When the dimension of the parameter vector is large, directly
solving optimization problems with such constraints can be expensive even if evaluating the
critical value at each θ is cheap.
To mitigate this issue, we provide an algorithm that is a contribution to the moment
(in)equalities literature in its own right and that can be helpful for implementing other
approaches.14 We apply it to constrained optimization problems of the following form:
p1 θ˚ ” sup p1 θ
θPΘ

s.t. gj pθq ď cpθq, j “ 1, ¨ ¨ ¨ , J,

(3.6)

where θ˚ is an optimal solution of the problem, gj , j “ 1, . . . , J are known functions, and c is a
?
function that requires a higher computational cost. In our context, gj pθq “ nm̄n,j pθq{σ̂n,j pθq
and, for calibrated projection, cpθq “ ĉn pθq. Conditional on the data tX1 , ¨ ¨ ¨ , Xn u, these
functions are considered deterministic. A key feature of the problem is that the function
cp¨q is relatively costly to evaluate.15 Our algorithm evaluates cp¨q on finitely many values
of θ. For other values, it imposes a probabilistic model that gets updated as specific values
are computed and that is used to determine the next evaluation point. Under reasonable
conditions, the resulting sequence of approximate optimal values converges to p1 θ˚ .
Specifically, after drawing an initial set of evaluation points that grows linearly with the
dimensionality of parameter space, the algorithm has three steps called E, A, and M below.
Initialization-step: Draw randomly (uniformly) over Θ a set pθp1q , ¨ ¨ ¨ , θpkq q of initial evaluation points. We suggest setting k “ 10d ` 1.
E-step: (Evaluation) Evaluate cpθp`q q for ` “ 1, ¨ ¨ ¨ , L, where L ě k. Set Υp`q “ cpθp`q q, ` “
1, ¨ ¨ ¨ , L. The current estimate p1 θ˚,L of the optimal value can be computed using
θ˚,L P argmaxθPC L p1 θ,

(3.7)

where C L ” tθp`q : ` P t1, ¨ ¨ ¨ , Lu, gj pθp`q q ď cpθp`q q, j “ 1, ¨ ¨ ¨ , Ju is the set of feasible
evaluation points.
14

This algorithm is based on the response surface method used in the optimization literature; see Jones
(2001), Jones, Schonlau, and Welch (1998), and references therein.
15
Here we assume that computing the sample moments is less expensive than computing the critical value.
When computation of moments is also very expensive, our proposed algorithm can be used to approximate
these too.

[8]

A-step: (Approximation) Approximate θ ÞÑ cpθq by a flexible auxiliary model. We use a
Gaussian-process regression model (or kriging), which for a mean-zero Gaussian process
p¨q indexed by θ and with constant variance ς 2 specifies
Υp`q “ µ ` pθp`q q, ` “ 1, ¨ ¨ ¨ , L
1

1

(3.8)

1

Corrppθq, pθ qq “ Kβ pθ ´ θ q, θ, θ P Θ,

(3.9)

Ś
where Kβ is a kernel with parameter vector β P dk“1 rβ k , β k s Ă Rd`` , e.g. Kβ pθ ´ θ1 q “
ř
expp´ dk“1 |θk ´θk1 |2 {βk q. The unknown parameters pµ, ς 2 q can be estimated by running
a GLS regression of Υ “ pΥp1q , ¨ ¨ ¨ , ΥpLq q1 on a constant with the given correlation
matrix. The unknown parameters β can be estimated by a (concentrated) MLE.
The (best linear) predictor of the critical value and its gradient at an arbitrary point
are then given by
cL pθq “ µ̂ ` rL pθq1 R´1
L pΥ ´ µ̂1q,

(3.10)

∇θ cL pθq “ µ̂ ` QL pθqR´1
L pΥ ´ µ̂1q,

(3.11)

where rL pθq is a vector whose `-th component is Corrppθq, pθp`q qq as given above
with estimated parameters, QL pθq “ ∇θ rL pθq1 , and RL is an L-by-L matrix whose
1

p`, `1 q entry is Corrppθp`q q, pθp` q qq with estimated parameters. This approximating
(or surrogate) model has the property that its predictor satisfies cL pθp`q q “ cpθp`q q, ` “
1, ¨ ¨ ¨ , L. Hence, it provides an analytical interpolation to the evaluated critical values
together with an analytical gradient.16 Further, the amount of uncertainty left in cpθq
(at an arbitrary point) is captured by the following variance:
ςˆ2 s2L pθq

´
2¯
p1 ´ 11 R´1
1 ´1
L rL pθqq
“ ςˆ 1 ´ rL pθq RL rL pθq `
.
11 R´1
L 1
2

(3.12)

M-step: (Maximization): With probability 1 ´ , maximize the expected improvement
function EIL to obtain the next evaluation point, with:
´
´ ḡpθq ´ c pθq ¯¯
L
,
θpL`1q ” arg max EIL pθq “ arg maxpp1 θ ´ p1 θ˚,L q` 1 ´ Φ
ς
ˆ
s
pθq
L
θPΘ
θPΘ

(3.13)

where ḡpθq “ maxj“1,¨¨¨ ,J gj pθq. This step can be implemented by standard nonlinear
optimization solvers, e.g. Matlab’s fmincon or KNITRO (see Appendix B.3 for details).
With probability , draw θpL`1q randomly from a uniform distribution over Θ.
Once the next evaluation point θpL`1q is determined, one adds it to the set of evaluation
16

See details in Jones, Schonlau, and Welch (1998). We use the DACE Matlab kriging toolbox (http:
//www2.imm.dtu.dk/projects/dace/) for this step in our Monte Carlo experiments.

[9]

points and iterates the E-A-M steps. This yields an increasing sequence of approximate
optimal values p1 θ˚,L , L “ k ` 1, k ` 2, ¨ ¨ ¨ . Once a convergence criterion is met, the value
p1 θ˚,L is reported as the end point of CIn . We discuss convergence criteria in Section 5.
Remark 3.2: The advantages of E-A-M are as follows. First, we control the number of
points at which we evaluate the critical value. Since the evaluation of the critical value is the
relatively expensive step, controlling the number of evaluations is important. One should also
note that the E-step with the initial k evaluation points can easily be parallelized. For any
additional E-step (i.e. L ą k), one needs to evaluate cp¨q only at a single point θpL`1q . The
M-step is crucial for reducing the number of additional evaluation points. To determine the
next evaluation point, one needs to take into account the trade-off between “exploitation”
(i.e. the benefit of drawing a point at which the optimal value is high) and “exploration” (i.e.
the benefit of drawing a point in a region in which the approximation error of c is currently
large). The expected improvement function in (3.13) quantifies this trade-off, and draws a
point only in an area where one can expect the largest improvement in the optimal value,
yielding substantial computational savings.17
Second, the proposed algorithm simplifies the M-step by providing constraints and their
gradients for program (3.13) in closed form. Availability of analytical gradients greatly aids
fast and stable numerical optimization. The price is the additional approximation step. In
the numerical exercises of Section 5, this price turns out to be low.

3.3

Convergence of the E-A-M Algorithm

We now provide formal conditions under which p1 θ˚,L converges to the true end point of
CIn as L Ñ 8.18 Our convergence result recognizes that the parameters of the Gaussian
process prior in (3.8) are estimated for each iteration of the A-step using the “observations”
tθ` , cpθ` quL
`“1 , and hence change with L. Because of this, a requirement for convergence
is that cpθq is a sufficiently smooth function of θ. We show that a high-level condition
guaranteeing this level of smoothness ensures a general convergence result for the E-A-M
algorithm. This is a novel contribution to the literature on response surface methods for
constrained optimization.
In the statement of Theorem 3.1 below, Hβ pΘq is the reproducing kernel Hilbert space
(RKHS) on Θ Ď Rd determined by the kernel used to define the correlation functional in (3.9).
The norm on this space is } ¨ }Hβ ; see Online Appendix B.2 for details. Also, the expectation
EQ is taken with respect to the law of pθp1q , ¨ ¨ ¨ , θpLq q determined by the Initialization-step
and the M-step, holding the sample fixed. See Appendix A for a precise definition of EQ and
a proof of the theorem.
17

It is also possible to draw multiple points in each iteration. See Schonlau, Welch, and Jones (1998).
We build on Bull (2011), who proves a convergence result for the algorithm proposed by Jones, Schonlau,
and Welch (1998) applied to an unconstrained optimization problem in which the objective function is unknown
outside the evaluation points.
18

[10]

Theorem 3.1: Suppose Θ Ă Rd is a compact hyperrectangle with nonempty interior and
that }p} “ 1. Let the evaluation points pθp1q , ¨ ¨ ¨ , θpLq q be drawn according to the Initialization
and the M steps. Let Kβ in (3.9) be a Matérn kernel with index ν P p0, 8q and ν R N. Let
c : Θ ÞÑ R satisfy }c}Hβ̄ ď R for some R ą 0, where β̄ “ pβ̄1 , ¨ ¨ ¨ , β̄d q1 . Then
“
‰
EQ p1 θ˚ ´ p1 θ˚,L`1 Ñ 0 as L Ñ 8.

(3.14)

Remark 3.3: The requirement that Θ is a compact hyperrectangle with nonempty interior can be replaced by a requirement that Θ belongs to the interior of a closed hyperrectangle
in Rd such that c satisfies the smoothness requirement in Theorem 3.1 on that rectangle.
In order to apply Theorem 3.1 to calibrated projection, we provide low level conditions
(Assumption B.1 in Online Appendix B.1.1) under which the map θ ÞÑ ĉn pθq uniformly
stochastically satisfies a Lipschitz-type condition. To get smoothness, we work with a mollified
version of ĉn , denoted ĉn,τn and provided in equation (B.1), with τn “ opn´1{2 q.19 Theorem
B.1 in the Online Appendix shows that ĉn and ĉn,τn can be made uniformly arbitrarily close
to each other and that ĉn,τn yields valid inference in the sense of equation (2.6). In practice,
one may therefore directly apply the E-A-M steps to ĉn .
Remark 3.4: The key condition imposed in Theorem B.1 is Assumption B.1. It requires
that the GMS function used is Lipschitz in its argument, and that the standardized moment
functions are Lipschitz in θ. In Online Appendix C.1 we establish that the latter condition
is satisfied by some canonical examples in the moment (in)equality literature, namely the
mean with missing data, linear regression and best linear prediction with interval data (and
discrete covariates), and entry games with multiple equilibria (and discrete covariates).20

4

Asymptotic Validity of Inference

4.1

Assumptions

We posit that P , the distribution of the observed data, belongs to a class of distributions
denoted by P. We write stochastic order relations that hold uniformly over P P P using the
notations oP and OP ; see Online Appendix D.1 for the formal definitions. Below, , ε, δ,
ω, σ, M , M̄ denote generic constants which may be different in different appearances but
cannot depend on P . Given a square matrix A, we write eigpAq for its smallest eigenvalue.
Assumption 4.1: (a) Θ Ă Rd is a compact hyperrectangle with nonempty interior.
(b) All distributions P P P satisfy the following:
19

For a discussion of mollification, see e.g. Rockafellar and Wets (2005, Example 7.19)
It can also be shown to hold in semi-parametric binary regression models with discrete or interval valued
covariates under the assumptions of Magnac and Maurin (2008).
20

[11]

(i) EP rmj pXi , θqs ď 0, j “ 1, . . . , J1 and EP rmj pXi , θqs “ 0, j “ J1 ` 1, ¨ ¨ ¨ , J1 ` J2 for
some θ P Θ;
(ii) tXi , i ě 1u are i.i.d.;
2 pθq P p0, 8q for j “ 1, ¨ ¨ ¨ , J for all θ P Θ;
(iii) σP,j

(iv) For some δ ą 0 and M P p0, 8q and for all j, EP rsupθPΘ |mj pXi , θq{σP,j pθq|2`δ s ď M .
Assumption 4.2: The function ϕj is continuous at all x ě 0 and ϕj p0q “ 0; κn Ñ 8
and κn “ opn1{2 q. If Assumption 4.3-(II) is imposed, κn “ opn1{4 q.
Assumption 4.1-(a) requires that Θ is a hyperrectangle, but can be replaced with the
assumption that θ is defined through a finite number of nonstochastic inequality constraints
smooth in θ and such that Θ is convex. Compactness is a standard assumption on Θ for
extremum estimation. We additionally require convexity as we use mean value expansions of
EP rmj pXi , θqs{σP,j pθq in θ; see (2.8). Assumption 4.1-(b) defines our moment (in)equalities
model. Assumption 4.2 constrains the GMS function and the rate at which its tuning parameter diverges. Both 4.1-(b) and 4.2 are based on Andrews and Soares (2010) and are standard
in the literature,21 although typically with κn “ opn1{2 q. The slower rate κn “ opn1{4 q is
?
satisfied for the popular choice, recommended by Andrews and Soares (2010), of κn “ ln n.
Next, and unlike some other papers in the literature, we impose restrictions on the correlation matrix of the moment functions. These conditions can be easily verified in practice
because they are implied when the correlation matrix of the moment equality functions and
the moment inequality functions specified below have a determinant larger than a predefined
constant for any θ P Θ.
Assumption 4.3: All distributions P P P satisfy one of the following two conditions for
some constants ω ą 0, σ ą 0,  ą 0, ε ą 0, M ă 8:
(I) Let J pP, θ; εq ” tj P t1, ¨ ¨ ¨ , J1 u : EP rmj pXi , θqs{σP,j pθq ě ´εu. Denote
`
˘1
m̃pXi , θq ” tmj pXi , θqujPJ pP,θ;εq , mJ1 `1 pXi , θq, ¨ ¨ ¨ , mJ1 `J2 pXi , θq ,
Ω̃P pθq ” CorrP pm̃pXi , θqq.
Then inf θPΘI pP q eigpΩ̃P pθqq ě ω.
(II) The functions mj pXi , θq are defined on Θ “ tθ P Rd : dpθ, Θq ď u. There exists
R1 P N, 1 ď R1 ď J1 {2, and measurable functions tj : X ˆ Θ Ñ r0, M s, j P R1 ”
t1, . . . , R1 u, such that for each j P R1 ,
mj`R1 pXi , θq “ ´mj pXi , θq ´ tj pXi , θq.
21

Continuity of ϕj for x ě 0 is restrictive only for GMS function ϕp2q in Andrews and Soares (2010).

[12]

(4.1)

: j pXi , θq P tmj pXi , θq, mj`R1 pXi , θqu,
For each j P R1 X J pP, θ; εq and any choice m
denoting Ω̃P pθq ” CorrP pm̃pXi , θqq, where
´
m̃pXi , θq ” tm
: j pXi , θqujPR1 XJ pP,θ;εq ,
¯1
tmj pXi , θqujPJ pP,θ;εqzt1,...,2R1 u , mJ1 `1 pXi , θq, ¨ ¨ ¨ , mJ1 `J2 pXi , θq ,
one has
inf

eigpΩ̃P pθqq ě ω.

(4.2)

σP,j pθq ą σ for j “ 1, . . . , R1 .

(4.3)

θPΘI pP q

Finally,
inf
θPΘI pP q

Assumption 4.3-(I) requires that the correlation matrix of the moment functions corresponding to close-to-binding moment conditions has eigenvalues uniformly bounded from
below. This assumption holds in many applications of interest, including: (i) instances when
the data is collected by intervals with minimum width;22 (ii) in treatment effect models
with (uniform) overlap; (iii) in static complete information entry games under weak solution
concepts, e.g. rationality of level 1, see Aradillas-Lopez and Tamer (2008).
We are aware of two examples in which Assumption 4.3-(I) may fail. One are missing
data scenarios, e.g. scalar mean, linear regression, and best linear prediction, with a vanishing
probability of missing data. The other example, which is extensively simulated in Section
5, is the Ciliberto and Tamer (2009) entry game model when the solution concept is pure
strategy Nash equilibrium. We show in Online Appendix C.2 that these examples satisfy
Assumption 4.3-(II).
Remark 4.1: Assumption 4.3-(II) weakens 4.3-(I) by allowing for (drifting to) perfect
correlation among moment inequalities that cannot cross. This assumption is often satisfied
in moment conditions that are separable in data and parameters, i.e. for each j “ 1, . . . , J,
EP rmj pXi , θqs “ EP rhj pXi qs ´ vj pθq,

(4.4)

for some measurable functions hj : X Ñ R and vj : Θ Ñ R. Models like the one in Ciliberto
and Tamer (2009) fall in this category, and we verify Assumption 4.3-(II) for them in Online
Appendix C.2. The argument can be generalized to other separable models.
22
Empirically relevant examples are that of: (a) the Occupational Employment Statistics (OES) program
at the Bureau of Labor Statistics, which collects wage data from employers as intervals of positive width, and
uses these data to construct estimates for wage and salary workers in 22 major occupational groups and 801
detailed occupations; and (b) when, due to concerns for privacy, data is reported as the number of individuals
who belong to each of a finite number of cells (for example, in public use tax data).

[13]

In Online Appendix C.2, we also verify Assumption 4.3-(II) for some models that are
not separable in the sense of equation (4.4), for example best linear prediction with interval
outcome data. The proof can be extended to cover (again non-separable) binary models with
discrete or interval valued covariates under the assumptions of Magnac and Maurin (2008).
In what follows, we refer to pairs of inequality constraints indexed by tj, j ` R1 u and satisfying (4.1) as “paired inequalities.” Their presence requires a modification of the bootstrap
procedure. This modification exclusively concerns the definition of Λbn pθ, ρ, cq in equation
(3.1). We explain it here for the case that the GMS function ϕj is the hard-thresholding one
in (3.3), and refer to Online Appendix E equations (E.12)-(E.13) for the general case. If
ϕj pξˆn,j pθqq “ 0 “ ϕj pξˆn,j`R1 pθqq,
we replace Gbn,j`R1 pθq with ´Gbn,j pθq and D̂n,j`R1 pθq with ´D̂n,j pθq, so that inequality
Gbn,j`R1 pθq ` D̂n,j`R1 pθqλ ď c is replaced with ´Gbn,j pθq ´ D̂n,j pθqλ ď c in equation (3.1). In
words, when hard threshold GMS indicates that both paired inequalities bind, we pick one of
them, treat it as an equality, and drop the other one. In the proof of Theorem 4.1, we show
that this tightens the stochastic program.23 The rest of the procedure is unchanged.
Instead of Assumption 4.3, BCS (Assumption 2) impose the following high-level condition:
(a) The limit distribution of their profiled test statistic is continuous at its 1 ´ α quantile
if this quantile is positive; (b) else, their test is asymptotically valid with a critical value of
zero. In Online Appendix D.2.3, we show that we can replace Assumption 4.3 with a weaker
high level condition (Assumption D.1-(I)) that resembles the BCS assumption but constrains
the limiting coverage probability. (We do not claim that the conditions are equivalent.) The
substantial amount of work required for us to show that Assumption 4.3 implies Assumption
D.1-(I) is suggestive of how difficult these high-level conditions can be to verify.24 Moreover, in
Online Appendix F.2 we provide a simple example that violates Assumption 4.3 and in which
all of calibrated projection, BCS-profiling, and the boosttrap procedure in Pakes, Porter, Ho,
and Ishii (2011) fail. The example leverages the fact that when binding constraints are
near-perfectly correlated, the projection may be estimated superconsistently, invalidating the
simple nonparametric bootstrap.25
Together with imposition of the ρ-box constraints, Assumption 4.3 allows us to dispense
with restrictions on the local geometry of the set ΘI pP q. Restrictions of this type, which
are akin to constraint qualification conditions, are imposed by BCS (Assumption A.3-(a)),
23

M
When paired inequalities are present, in equation (2.5) instead of σ̂n,j we use the estimator σ̂n,j
specified
in (E.188) in Lemma E.10 p.52 of the Online Appendix for σP,j , j “ 1, . . . , 2R1 (with R1 ď J1 {2 defined in
the assumption). In equation (3.2) we use σ̂n,j for all j “ 1, . . . , J. To ease notation, we do not distinguish
the two unless it is needed.
24
Assumption 4.3 is used exclusively to obtain the conclusions of Lemma E.6, E.7 and E.8, hence any
alternative assumption that delivers such results can be used.
25
The example we provide satisfies all assumptions explicitly stated in Pakes, Porter, Ho, and Ishii (2011),
illustrating an oversight in their Theorem 2.

[14]

Pakes, Porter, Ho, and Ishii (2011, Assumptions A.3-A.4), Chernozhukov, Hong, and Tamer
(2007, Condition C.2), and elsewhere. In practice, they can be hard to verify or pre-test for.
We study this matter in detail in Kaido, Molinari, and Stoye (2017).
We next lay out regularity conditions on the gradients of the moments.
Assumption 4.4: All distributions P P P satisfy the following conditions:
(i) For each j, there exist DP,j p¨q ” ∇θ tEP rmj pX, ¨qs{σP,j p¨qu and its estimator D̂n,j p¨q
such that supθPΘ }D̂n,j pθq ´ DP,j pθq} “ oP p1q.
(ii) There exist M, M̄ ă 8 such that for all θ, θ̃ P Θ maxj“1,¨¨¨ ,J }DP,j pθq ´ DP,j pθ̃q} ď
M }θ ´ θ̃} and maxj“1,¨¨¨ ,J supθPΘI pP q }DP,j pθq} ď M̄ .
Assumption 4.4 requires that each of the J normalized population moments is differentiable, that its derivative is Lipschitz continuous, and that this derivative can be consistently
estimated uniformly in θ and P .26 We require these conditions because we use a linear expansion of the population moments to obtain a first-order approximation to the nonlinear
programs defining CIn , and because our bootstrap procedure requires an estimator of DP .
A final set of assumptions is on the normalized empirical process. For this, define the
variance semimetric %P by
› “
` ´1
˘‰1{2 (J ››
›
´1
%P pθ, θ̃q ” › V arP σP,j
.
pθqmj pX, θq ´ σP,j
pθ̃qmj pX, θ̃q
j“1 ›

(4.5)

For each θ, θ̃ P Θ and P , let QP pθ, θ̃q denote a J-by-J matrix whose pj, kq-th element is the
covariance between mj pXi , θq{σP,j pθq and mk pXi , θ̃qq{σP,k pθ̃q.
Assumption 4.5: All distributions P P P satisfy the following conditions:
´1
(i) The class of functions tσP,j
pθqmj p¨, θq : X Ñ R, θ P Θu is measurable for each j “

1, ¨ ¨ ¨ , J.
(ii) The empirical process Gn with j-th component Gn,j is uniformly asymptotically %P equicontinuous. That is, for any  ą 0,
˜
lim lim sup sup P
δÓ0

nÑ8

P PP

¸
sup

}Gn pθq ´ Gn pθ̃q} ą 

“ 0.

(4.6)

sup }QP pθ1 , θ̃1 q ´ QP pθ2 , θ̃2 q} “ 0.

(4.7)

%P pθ,θ̃qăδ

(iii) QP satisfies
lim
δÓ0

26

sup

}pθ1 ,θ̃1 q´pθ2 ,θ̃2 q}ăδ P PP

The requirements are imposed on Θ . Under Assumption 4.3-(I) it suffices they hold on Θ.

[15]

Under this assumption, the class of normalized moment functions is uniformly Donsker
(Bugni, Canay, and Shi, 2015). We use this fact to show validity of our method.

4.2

Theoretical Results

First set of results: Uniform asymptotic validity in the general case.
The following theorem establishes the asymptotic validity of the proposed confidence
interval CIn ” r´sp´p, Cn pĉn qq, spp, Cn pĉn qqs, where spp, Cn pĉn qq was defined in equation (2.5)
and ĉn in (3.5).
Theorem 4.1: Suppose Assumptions 4.1, 4.2, 4.3, 4.4, and 4.5 hold. Let 0 ă α ă 1{2.
Then
lim inf inf

inf

nÑ8 P PP θPΘI pP q

P pp1 θ P CIn q ě 1 ´ α.

(4.8)

A simple corollary to Theorem 4.1, whose proof is omitted, is that we can provide joint
confidence regions for several projections, in particular confidence hyperrectangles for subvectors. Thus, let p1 , . . . , pk denote unit vectors in Rd , k ď d. Then:
Corollary 4.1: Suppose Assumptions 4.1, 4.2, 4.3, 4.4, and 4.5 hold. Let 0 ă α ă 1{2.
Then,
lim inf inf

inf

nÑ8 P PP θPΘI pP q

P pp`1 θ P CIn,` , ` “ 1, . . . , kq ě 1 ´ α,

(4.9)

”
ı
where CIn,` “ inf θPCn pĉkn q p`1 θ, supθPCn pĉkn q p`1 θ and ĉkn pθq ” inftc P R` : P ˚ pΛbn pθ, ρ, cq X
tXk`“1 tp`1 λ “ 0uu ‰ Hq ě 1 ´ αu.
The difference in this Corollary compared to Theorem 4.1 is that ĉkn is calibrated so that
(3.4) holds for all p1 , . . . , pk simultaneously.
In applications, a researcher might wish to obtain a confidence interval for a known nonlinear function f : Θ ÞÑ R. Examples include policy analysis and counterfactual estimation
in the presence of partial identification, or demand extrapolation subject to rationality constraints. It is possible to extend our results to uniformly continuously differentiable functions
f . Because the function f is known, the conditions on its gradient required below can be
easily verified in practice (especially if the first one is strengthened to hold over Θ).
Theorem 4.2: Let CInf be a confidence interval whose lower and upper points are obtained
solving
inf { sup f pθq s.t.

θPΘ

θPΘ

?
nm̄n,j pθq{σ̂n,j pθq ď ĉfn pθq, j “ 1, ¨ ¨ ¨ , J,

[16]

where ĉfn pθq ” inftc ě 0 : P ˚ pΛbn pθ, ρ, cq X t}∇θ f pθq}´1 ∇θ f pθqλ “ 0u ‰ Hq ě 1 ´ αu. Suppose
Assumptions 4.1, 4.2, 4.3, 4.4, and 4.5 hold. Suppose that there exist $ ą 0 and M ă 8
such that inf P PP inf θPΘI pP q }∇f pθq} ě $ and supθ,θ̄PΘ }∇f pθq ´ ∇f pθ̄q} ď M }θ ´ θ̄}, where
∇θ f pθq is the gradient of f pθq. Let 0 ă α ă 1{2. Then,
lim inf inf

inf

nÑ8 P PP θPΘI pP q

P pf pθq P CInf q ě 1 ´ α.

(4.10)

Second set of results: Simplifications for special cases.
We now consider more restrictive assumptions on the model, defining a subset of DGPs
Q Ă P; across theorems below, the set Q differs based on which assumptions are maintained.
If P P Q, a number of simplifications to the method, including dropping the ρ-box constraints,
are possible. Here we state the formal results and then we give a heuristic explanation of
the conditions needed for these simplifications. Online Appendix D.3.1 contains the exact
assumptions and Online Appendix D.3.2 the proofs. We remark that all of the additional
assumptions are implied by assumptions in Pakes, Porter, Ho, and Ishii (2011), hence under
their conditions Theorem 4.3 applies in its entirety.
Theorem 4.3: Suppose Assumptions 4.1, 4.2, 4.3, 4.4, and 4.5 hold. Let 0 ă α ă 1{2.
(I) If Assumption D.2-(1) holds for either p or ´p (or both), then setting
CIn “

“

inf
θPCn pĉn,´p q

p1 θ,

sup

‰
p1 θ ,

(4.11)

θPCn pĉn,p q

ĉn,q pθq “ inftc P R` : P ˚ pΛbn pθ, ρ, cq X tq 1 λ ě 0u ‰ Hq ě 1 ´ αu, q P tp, ´pu, (4.12)
we have
lim inf inf

inf

nÑ8 P PQ θPΘI pP q

P pp1 θ P CIn q ě 1 ´ α.

(4.13)

(II) If Assumptions D.2-(1) (for either p or ´p or both), D.3 and D.4 hold, then (4.13)
continues to be satisfied with CIn as defined in (4.11) and evaluated at ĉn,q pθq “ ĉn,q pθ̂q q
for q P t´p, pu and for all θ P Θ in (4.12), where θ̂q P arg maxθPΘ̂I q 1 θ and Θ̂I “ tθ P
Θ : m̄n,j pθq ď 0, j “ 1, . . . , Ju.
(III) If Assumptions D.2-(2) (for either p or ´p or both) and D.5 hold, then setting ρ “ `8
to obtain ĉn,q pθ̂q q in (4.12) and using these values for q P t´p, pu for each θ P Θ in
computing CIn as defined in (4.11), we have that (4.13) continues to be satisfied.
Remark 4.2: If Theorem 4.3-(II) applies and the standardized moment conditions in
(2.5) are linear in θ, then CIn can be computed by solving just two linear programs.

[17]

Assumption D.2-(1) in Theorem 4.3-(I) ensures that some point in tp1 θ, θ P ΘI pP qu is
covered with probability approaching 1. Hence, the inference problem is effectively onesided at the projection’s end points and degenerate in between. It then suffices to intersect
two one-sided p1 ´ αq-confidence intervals. Under Assumptions 4.1-4.5, Assumption D.2 is
implied both by a “degeneracy condition”in Chernozhukov, Hong, and Tamer (2007) and by
an assumption in Pakes, Porter, Ho, and Ishii (2011). A simple sufficient condition is that
there exists a parameter value at which all population constraints hold with slack.
Assumptions D.3 and D.4 in Theorem 4.3-(II) are logically independent “polynomial
minorant” conditions imposed in Chernozhukov, Hong, and Tamer (2007) and Bugni, Canay,
and Shi (2017). Jointly, they assure that the sample support set Hpp, Θ̂I q is an “inner
consistent” estimator of the population support set Hpp, ΘI q.27 That is, any accumulation
point of a selection from Hpp, Θ̂I q is in Hpp, ΘI q, but Hpp, Θ̂I q may be much smaller than
Hpp, ΘI q. Then for one-sided inference, it suffices to compute ĉn pθq exactly once, namely at
one arbitrary selection θ̂ P Hpp, Θ̂I q, and to set ĉn pθq “ ĉn pθ̂q for all θ. We again remark that
these conditions are implied by assumptions in Pakes, Porter, Ho, and Ishii (2011).
Assumptions D.2-(2) and D.5 in Theorem 4.3-(III) yield that the support set is a singleton
and the tangent cone at the support set is pointy (in a uniform sense). We show that, in this
case, the ρ-box constraints can be entirely dropped. This assumption is directly imposed by
Pakes, Porter, Ho, and Ishii (2011), but we weaken it by showing that it is only needed in a
local sense; hence, it suffices that the support set consists of distinct extreme points and all
corresponding tangent cones are pointy.
Result 3: A comparison with BCS-profiling. We finally compare calibrated projection
to BCS-profiling in well behaved cases. Suppose that Theorem 4.3 applies. Then CIn is the
intersection of two one-sided confidence intervals and we can set ρ “ `8. Hence, a scalar s
is in the one-sided (unbounded from below) confidence interval for p1 θ if

Tn pθq ”

?

min Tn pθq ď ĉn pθ̂p q,

(4.14)

n max m̄n,j pθq{σ̂n,j pθq.

(4.15)

p1 θ“s
j

While it was not originally constructed in this manner, this simplified confidence interval
is the lower contour set of a profiled test statistic.28 Indeed, up to an inconsequential squaring,
Tn is a special case of the statistic used in Bugni, Canay, and Shi (2017). This raises the
question of how the tests compare. In the especially regular case where all parts of Theorem
4.3 apply, and assuming that calibrated projection is implemented with the corresponding
simplifications, the answer is as follows:
27

For a given unit vector p and compact set A Ă Rd , the support set of A is Hpp, Aq ” arg maxaPA p1 a.
By contrast, the corresponding expression without Theorem 4.3-(II) is minp1 θ“s tTn pθq ´ ĉn pθqu ď 0, which
is not usefully interpreted as test inversion.
28

[18]

Theorem 4.4: Suppose Assumptions 4.1, 4.2, 4.3, 4.4, 4.5, D.2, D.3, D.4, D.5, and
D.6 hold. Let BCS-profiling be implemented with the criterion function in equation (4.15)
and GMS function ϕpxq “ mint0, xu.29 Let calibrated projection be implemented using the
simplifications from Theorem 4.3, including setting ρ “ `8. If both methods furthermore
use the same κn , they are uniformly asymptotically equivalent:
lim inf inf

inf

nÑ8 P PQ sPrminθPΘ p1 θ,maxθPΘ p1 θs

´
¯
P 1ts P CIn u “ 1ts P CInprof u Ñ 1,

where CInprof denotes the confidence interval resulting from the BCS-profiling method.
Thus there is strong agreement between methods in extremely well-behaved cases.30 We
also show in Online Appendix F.1 that, in a further specialization of the above setting, finite
sample power is higher with calibrated projection. This effect is due to a conservative distortion of order 1{κn in Bugni, Canay, and Shi (2017) and therefore vanishes asymptotically;
however, due to the slow rate at which κn diverges, it can be large in samples of considerable size. In sum, the approaches are not ranked in terms of power in empirically relevant
examples.

4.3

Role of the ρ-box Constraints and Heuristics for Choosing ρ

When we use the bootstrap to calibrate ĉn p¨q, we restrict the localization vector λ to lie in a
ρ-box; see equation (3.1). This restriction has a crucial regularization effect. Comparing (2.7)
and (3.4), it is apparent that we estimate coverage probabilities by replacing a nonlinear program with a linear one. It is intuitive that a Karush-Kuhn-Tucker condition (with uniformly
bounded Lagrange multipliers) is needed for this to work (uniformly), and also that the linearization in (2.8) should be uniformly valid. But direct imposition of a Karush-Kuhn-Tucker
condition would amount to a hard-to-verify constraint qualification. Rather than doing this,
we show that Assumption 4.3 and imposition of the ρ-box constraints jointly yield such constraint qualification conditions on the set Λbn pθ, ρ, cq (defined in (3.1)) with arbitrarily high
probability for n large enough, as well as uniform validity of the linearization. If one knows
(or assumes) a priori that the population (limit) counterpart of the constraint set in (2.7) is
contained in a ball with a radius bounded in probability (see Assumption D.1-(II) in Online
Appendix D.2.2), then ρ can be set equal to `8. The assumptions in Theorem 4.3-(III) are
sufficient for this condition to hold.31
In practice, the choice of ρ requires trading off how much conservative bias one is willing
to bear in well-behaved cases against how much finite-sample size distortion one is willing
29

The restriction on the GMS function is needed only because the “penalized resampling” approximation
in BCS employs a specific “slackness function” equal to ξˆn,j .
30
This is not true for Pakes, Porter, Ho, and Ishii (2011) because they do not studentize the moment
inequalities.
31
See Online Appendix D.1 for proofs of these statements.

[19]

to bear in ill-behaved cases.32 We propose a heuristic approach to calibrate ρ focusing
on conservative bias in the well behaved cases just considered, i.e. cases such as those
characterized in Assumptions D.2, D.3, D.4, D.5 and D.6, in which the ρ-box could be
dropped. In these cases, the optimal value of each of the two programs in equation (3.4) is
distributed asymptotically normal as a linear combination of d binding inequalities. When
in fact J1 ` J2 “ d, constraining λ P ρB d increases the coverage probability by at most
η “ 1 ´ r1 ´ 2Φp´ρqsd . The parameter ρ can therefore be calibrated to achieve a conservative
bias of at most η. When J1 ` J2 ą d, we propose to calibrate ρ using the benchmark
η “ 1 ´ r1 ´ 2Φp´ρqsdp

J1 `J2
d

q,

(4.16)

again achieving a target conservative bias (in well-behaved cases) of η. For a few numerical
examples, set η “ 0.01: then J1 ` J2 “ 10 and d “ 3 imply ρ “ 4.2, whereas J1 ` J2 “ 100
and d “ 10 imply ρ “ 8.4. In the Monte Carlo experiments of Section 5, we investigate
sensitivity of calibrated projection to the choice of ρ.

5

Monte Carlo Simulations

We evaluate the statistical and numerical performance of calibrated projection and EAM in
two sets of Monte Carlo experiments run on a server with two Intel Xeon X5680 processors
rated at 3.33GHz with 6 cores each and with a memory capacity of 24Gb rated at 1333MHz.33
Both simulate a two-player entry game. The first experiment compares calibrated projection and BCS-profiling in the Monte Carlo exercise of BCS, using their code.34 The other
experiments feature a considerably more involved entry model with and without correlated
unobservables. We were unable to numerically implement BCS-profiling for this model.35

5.1

The General Entry Game Model

We consider a two player entry game based on Ciliberto and Tamer (2009):

Y1 “ 0
Y1 “ 1

Y2 “ 0
0, 0
1
Z1 ζ1 ` u1 , 0

Y2 “ 1
0, Z21 ζ1 ` u2
Z11 pζ1 ` ∆1 q ` u1 , Z21 pζ2 ` ∆2 q ` u2

Here, Y` , Z` , and u` denote player `1 s binary action, observed characteristics, and unobserved
characteristics. The strategic interaction effects Z`1 ∆` ď 0 measure the impact of the opponent’s entry into the market. We let X ” pY1 , Y2 , Z11 , Z21 q1 . We generate Z “ pZ1 , Z2 q as
32

In Kaido, Molinari, and Stoye (2017) we provide examples of well-behaved and ill-behaved cases.
To run the more than 120 distinct simulations reported here, we employed multiple servers. We benched
the relative speed of each and report average computation time normalized to the server just described.
34
See http://qeconomics.org/ojs/index.php/qe/article/downloadSuppFile/431/1411.
35
For implementations of calibrated projection with real-world data, we refer the reader to Mohapatra and
Chatterjee (2015), where d “ 5, J1 “ 44, and J2 “ 0.
33

[20]

an i.i.d. random vector taking values in a finite set whose distribution pz “ P pZ “ zq is
known. We let u “ pu1 , u2 q be independent of Z and such that Corrpu1 , u2 q ” r P r0, 1s
and V arpu` q “ 1, ` “ 1, 2. We let θ ” pζ11 , ζ21 , ∆11 , ∆12 , rq1 . For a given set A Ă R2 , we define
Gr pAq ” P pu P Aq. We choose Gr so that the c.d.f. of u is continuous, differentiable, and
has a bounded p.d.f. The outcome Y “ pY1 , Y2 q results from pure strategy Nash equilibrium
play. For some value of Z and u, the model predicts monopoly outcomes Y “ p0, 1q and p1, 0q
as multiple equilibria. When this occurs, we select outcome p0, 1q by independent Bernoulli
trials with parameter µ P r0, 1s. This gives rise to the following restrictions:
Er1tY “ p0, 0qu1tZ “ zus ´ Gr pp´8, ´z11 ζ1 q ˆ p´8, ´z21 ζ2 qqpz “ 0

(5.1)

Er1tY “ p1, 1qu1tZ “ zus ´ Gr pr´z11 pζ1 ` ∆1 q, `8q ˆ r´z21 pζ2 ` ∆2 q, `8qqpz “ 0 (5.2)
Er1tY “ p0, 1qu1tZ “ zus ´ Gr pp´8, ´z11 pζ1 ` ∆1 qq ˆ r´z21 ζ2 , `8qqpz ď 0
(5.3)
”
´Er1tY “ p0, 1qu1tZ “ zus ` Gr pp´8, ´z11 pζ1 ` ∆1 qq ˆ r´z21 ζ2 , `8q
ı
´ Gr pr´z11 ζ1 , ´z11 pζ1 ` ∆1 qq ˆ r´z21 ζ2 , ´z21 pζ2 ` ∆2 qq pz ď 0. (5.4)
We show in Online Appendix C that this model satisfies Assumptions B.1 and 4.3-(II).36
Throughout, we analytically compute the moments’ gradients and studentize them using
sample analogs of their standard deviations.

5.2

Specific Implementations and Results

Set 1: A comparison with BCS-Profiling
BCS specialize this model as follows. First, u1 , u2 are independently uniformly distributed
on r0, 1s and the researcher knows r “ 0. Equality (5.1) disappears because p0, 0q is never
W
an equilibrium. Next, Z1 “ Z2 “ r1; tWk udk“0
s, where Wk are observed market type inW
s.37 The parameter
dicators, ∆` “ rδ` ; 0dW s for ` “ 1, 2, and ζ1 “ ζ2 “ ζ “ r0; tζ rks udk“0

vector is θ “ rδ1 ; δ2 ; ζs with parameter space Θ “ tθ P R2`dW : pδ1 , δ2 q P r0, 1s2 , ζk P
r0, mintδ1 , δ2 us, k “ 1, . . . , dW u. This leaves 4 moment equalities and 8 moment inequalities (so J “ 16); compare equation (5.1) in BCS. We set dW “ 3, P pWk “ 1q “ 1{4, k “
0, 1, 2, 3, θ “ r0.4; 0.6; 0.1; 0.2; 0.3s, and µ “ 0.6. The implied true bounds on parameters are
δ1 P r0.3872, 0.4239s, δ2 P r0.5834, 0.6084s, ζ r1s P r0.0996, 0.1006s, ζ r2s P r0.1994, 0.2010s, and
ζ r3s P r0.2992, 0.3014s.
The BCS-profiling confidence interval CInprof inverts a test of H0 : p1 θ “ s0 over a grid for
s0 . We do not in practice exhaust the grid but search inward from the extreme points of Θ in
directions ˘p. At each s0 that is visited, we compute (the square of) a profiled test statistic
36

The specialization in which we compare to BCS also fulfils their assumptions. The assumptions in Pakes,
Porter, Ho, and Ishii (2011) exclude any DGP that has moment equalities.
37
This allows for market-type homogeneous fixed effects but not for player-specific covariates nor for observed
heterogeneity in interaction effects.

[21]

minp1 θ“s0 Tn pθq; see equations (4.14)-(4.15) above. The corresponding critical value ĉprof
ps0 q
n
is a quantile of the minimum of two distinct bootstrap approximations, each of which solves
a nonlinear program for each bootstrap draw. Computational cost quickly increases with
grid resolution, bootstrap size, and the number of starting points used to solve the nonlinear
programs.
Calibrated projection computes ĉn pθq by solving a series of linear programs for each
bootstrap draw.38 It computes the extreme points of CIn by solving NLP (2.5) twice, a task
that is much accelerated by the E-A-M algorithm. Projection of Andrews and Soares (2010)
operates very similarly but computes its critical value ĉproj
n pθq through bootstrap simulation
without any optimization.
We align grid resolution in BCS-profiling with the E-A-M algorithm’s convergence threshold of 0.005.39 We run all methods with B “ 301 bootstrap draws, and calibrated and
“uncalibrated” (i.e., based on Andrews and Soares (2010)) projection also with B “ 1001.40
Some other choices differ: BCS-profiling is implemented with their own choice to multi-start
the nonlinear programs at 3 oracle starting points, i.e. using knowledge of the true DGP;
our implementation of both other methods multi-starts the nonlinear programs from 30 data
dependent random points (see Kaido, Molinari, Stoye, and Thirkettle (2017) for details).
Table 1 displays results for pδ1 , δ2 q and for 300 Monte Carlo repetitions of all three methods. All confidence intervals are conservative, reflecting the effect of GMS. As expected,
uncalibrated projection is most conservative, with coverage of essentially 1. Also, BCSprofiling is more conservative than calibrated projection. We suspect this relates to the
conservative effect highlighted in Online Appendix F.1. The most striking contrast is in computational effort, where uncalibrated projection is fastest but calibrated projection also beats
BCS-profiling by a factor of about 78. There are two effects at work here: First, because
the calibrated projection bootstrap iterates over linear programs, it is much faster than the
BCS-profiling one. Second, both uncalibrated projection and calibrated projection confidence
intervals were computed using the E-A-M algorithm. Indeed, the computation times reported
for uncalibrated projection indicate that, in contrast to received wisdom, this procedure is
computationally somewhat easy. This is due to the E-A-M algorithm and therefore part of
this paper’s contribution.
Table 2 extends the analysis to all components of θ and to 1000 Monte Carlo repetitions.
We were unable to compute this or any of the next tables for BCS-profiling.
Set 2: Heterogeneous interaction effects and potentially correlated errors
38
We implement this step using the high-speed solver CVXGEN, available from http://cvxgen.com and
described in Mattingley and Boyd (2012).
39
This is only one of several individually necessary stopping criteria. Others include that the current
optimum θ˚,L and the expected improvement maximizer θL`1 (see equation (3.13)) satisfy |p1 pθL`1 ´ θ˚,L q| ď
0.005. See Kaido, Molinari, Stoye, and Thirkettle (2017) for the full list of convergence requirements.
40
Based on some trial runs of BCS-profiling for δ1 , we estimate that running it with B “ 1001 throughout
would take 3.14-times longer than the computation times reported in Table 1. By comparison, calibrated
projection takes only 1.75-times longer when implemented with B “ 1001 instead of B “ 301.

[22]

In our second set of experiments, we let u “ pu1 , u2 q be bivariate Normal with (nondegenerate) correlation r, so all outcomes have positive probability. We let Z include a constant
and a player specific, binary covariate, so Z1 P tp1, ´1q, p1, 1qu and Z2 P tp1, ´1q, p1, 1qu. This
r2s

r2s

implies J1 “ J2 “ 8, hence J “ 24. The marginal distribution of pZ1 , Z2 q is multinomial
with weights p0.1, 0.2, 0.3, 0.4q on pp´1, ´1q, p´1, 1q, p1, ´1q, p1, 1qq.
In our Set 2-DGP1, we set ζ1 “ p.5, .25q1 , ∆1 “ p´1, ´1q1 , and r “ 0. Set 2-DGP2 differs
by setting ∆1 “ p´1, ´.75q1 . In both cases, pζ2 , ∆2 q “ pζ1 , ∆1 q and µ “ 0.5; we only report
results for pζ1 , ∆1 q. Although parameter values are similar, there is a qualitative difference:
r1s

In DGP1, parameters are point identified; in DGP2, they are not but the true bounds (ζ1 P
r2s

r1s

r2s

r0.405, 0.589s, ζ1 P r0.236, 0.266s, ∆1 P r´1.158, ´0.832s, ∆1 P r´0.790, ´0.716s) are not
wide compared to sampling uncertainty. We therefore expect all methods that use GMS to
be conservative in DGP2.41 In both Set 2-DGP1& DGP2 we use knowledge that r “ 0, so
that d “ 8. Our Set 2-DGP3 preserves the same payoff parameters values as in Set 2-DGP2
but sets r “ 0.5 and this parameter is also unknown, so that d “ 9.
Within Set 2-DGP2, we also experiment with the sensitivity of coverage probability and
length of CIn to the choice of ρ and κn . We consider choices of ρ that are (1) very large or
“liberal”, so that in well behaved cases the ρ-box constraints induce an amount η of overcoverage in CIn smaller than machine precision (see equation (4.16)); (2) “default”, so that
η “ 0.01; (3) small or “conservative”, so that η “ 0.025. For κn , we have experimented with
?
a “conservative” choice κn “ n1{7 , and a “liberal” choice κn “ ln ln n, while out “default”
?
is κn “ ln n.
Results are reported in Tables 3 through 7. An interesting feature of Table 3 is that
in this (point identified) DGP, calibrated projection is not conservative at all. This presumably reflects an absence of near-binding inequalities. Conservative bias is larger in the
partially identified Set 2-DGP2 in Table 4. For these two tables, we do note the increased
computational advantage of uncalibrated projection over calibrated projection. This advantage is bound to increase as DGP’s, and therefore the linear programs iterated over in the
bootstrap, become more complex. Table 5 shows that allowing for correlation of the errors
does not change the results much in terms of the confidence intervals’ length and coverage
probabilities. However, due to the repeated evaluation of the bivariate normal CDFs, both
calibrated and uncalibrated projection have higher computational time than the case with
r “ 0. Another feature to note is that both confidence intervals for r tend to be wide although
the projection of ΘI is short, which suggests that this component may be weakly identified.
Table 6 examines the effect of varying the tuning parameter ρ. Increasing ρ necessarily
(weakly) decreases length and also coverage of intervals, and this effect is evident in the
table but is arguably small. This is even more the case for the GMS tuning parameter
κn . Numerically, for n “ 4000, the values explored in the table are rather different at
41

We also note that this is a case where non-uniform methods may severely undercover in finite sample.

[23]

40001{7 « 3.27 and

a
lnplnp4000qq « 1.45, but the effect on inference is very limited, see

Table 7. Indeed, differences in coverage are so small that reported results are occasionally
slightly nonmonotonic, reflecting numerical and simulation noise.

6

Conclusions

This paper introduces a computationally attractive confidence interval for linear functions of
parameter vectors that are partially identified through finitely many moment (in)equalities.
The extreme points of our calibrated projection confidence interval are obtained by minimizing
and maximizing p1 θ subject to properly relaxed sample analogs of the moment conditions.
The relaxation amount, or critical level, is computed to insure uniform asymptotic coverage
of p1 θ rather than θ itself. Its calibration is computationally attractive because it is based on
repeatedly checking feasibility of (bootstrap) linear programming problems. Computation of
the extreme points of the confidence intervals is also computationally attractive thanks to an
application, novel to this paper, of the response surface method for global optimization that
is of independent interest in the partial identification literature. Indeed, a key contribution
of the paper is to establish convergence of this algorithm.
Our Monte Carlo analysis shows that, in the DGPs that we considered, calibrated projection is fast and accurate: Computation of the confidence intervals is orders of magnitude
faster than for the main alternative to our method, a profiling-based procedure due to Bugni,
Canay, and Shi (2017). The class of DGPs over which we can establish uniform validity of our
procedure is non-nested with corresponding class for the alternative method. Important cases
covered here but not elsewhere include linear functions of best linear predictor parameters
with interval valued outcomes and discrete covariates. The price to pay for this generality is
the use of one additional (non-drifting) tuning parameter. We provide conditions under which
this parameter can be eliminated and compare the power properties of calibrated projection
and BCS-profiling. The false coverage properties of the two methods are non-ranked but are
asymptotically the same in very well-behaved cases. We establish considerable finite sample
advantage in a specific case.
Similarly to confidence regions proposed in Andrews and Soares (2010), Bugni, Canay,
and Shi (2017), Stoye (2009), and elsewhere, our confidence interval can be empty, namely if
sample violations of moment inequalities exceed ĉn pθq at each θ. This event can be interpreted
as rejection of maintained assumptions. See Stoye (2009) and especially Andrews and Soares
(2010) for further discussion and Bugni, Canay, and Shi (2015) for a paper that focuses on
this interpretation and improves on ĉproj
for the purpose of specification testing. We leave a
n
detailed analysis of our implied specification test to future research.

[24]

A

Convergence of the E-A-M Algorithm

In this appendix, we provide details on the algorithm used to solve the outer maximization problem
as described in Section 3.2. Below, let pΩ, Fq be a measurable space and ω a generic element of Ω. Let
L P N and let pθp1q , ¨ ¨ ¨ , θpLq q be a measurable map on pΩ, Fq whose law is specified below. The value
of the function c in (3.6) is unknown ex ante. Once the evaluation points θp`q , ` “ 1, ¨ ¨ ¨ , L realize,
the corresponding values of c, i.e. Υp`q ” cpθp`q q, ` “ 1, ¨ ¨ ¨ , L, are known. We may therefore define
the information set
FL ” σpθp`q , Υp`q , ` “ 1, ¨ ¨ ¨ , Lq.

(A.1)

We note that θ˚,L ” argmaxθPC L p1 θ is measurable with respect to FL .
Our algorithm iteratively determines evaluation points based on the expected improvement (Jones,
Schonlau, and Welch, 1998). For this, we formally introduce a model that describes the uncertainty
associated with the values of c outside the current evaluation points. Specifically, the unknown function
c is modeled as a Gaussian process such that42
Ercpθqs “ µ, Covpcpθq, cpθ1 qq “ ς 2 Kβ pθ ´ θ1 q,

(A.2)

where β “ pβ1 , ¨ ¨ ¨ , βd q P Rd controls the length-scales of the process. Two values cpθq and cpθ1 q are
highly correlated when θk ´ θk1 is small relative to βk . Throughout, we assume β k ď βk ď β k for some
0 ă β k ă β k ă 8 for k “ 1, ¨ ¨ ¨ , d. We let β̄ “ pβ̄1 , ¨ ¨ ¨ , β̄d q1 P Rd . Specific suggestions on the forms
of Kβ are given in Appendix B.2.
For a given pµ, ς, βq, the posterior distribution of c given FL is then another Gaussian process
whose mean cL p¨q and variance ς 2 s2L p¨q are given as follows (Santner, Williams, and Notz, 2013, Section
4.1.3):
cL pθq “ µ ` rL pθq1 R´1
L pΥ ´ µ1q
´
2¯
p1 ´ 11 R´1
L rL pθqq
ς 2 s2L pθq “ ς 2 1 ´ rL pθq1 R´1
.
L rL pθq `
11 R´1
L 1

(A.3)
(A.4)

Given this, the expected improvement function can be written as
EIL pθq ” Erpp1 θ ´ p1 θ˚,L q` 1tḡpθq ď cpθqu|FL s
“ pp1 θ ´ p1 θ˚,L q` Ppcpθq ě max gj pθq|FL q
j“1,¨¨¨ ,J
ˆ
˙
cpθq
´
cL pθq
maxj“1,¨¨¨ ,J gj pθq ´ cL pθq
ě
“ pp1 θ ´ p1 θ˚,L q` P
ςsL pθq
ςsL pθq
´
´ ḡpθq ´ c pθq ¯¯
L
“ pp1 θ ´ p1 θ˚,L q` 1 ´ Φ
,
ςsL pθq

(A.5)

The evaluation points pθp1q , ¨ ¨ ¨ , θpLq q are then generated according to the following algorithm (M-step
in Section 3.2).
42
We use P and E to denote the probability and expectation for the prior and posterior distributions of c
to distinguish them from P and E used for the sampling uncertainty for Xi .

[25]

Algorithm A.1: Let k P N.
Step 1: Initial evaluation points θp1q , ¨ ¨ ¨ , θpkq are drawn randomly independent of c.
Step 2: For L ě k, with probability 1 ´ , let θpL`1q “ argmaxθPΘ EIL pθq. With probability , draw
θpL`1q uniformly at random from Θ.
Below, we use Q to denote the law of pθp1q , ¨ ¨ ¨ , θpLq q determined by the algorithm above. We also
note that θ˚,L`1 “ arg maxθPC L`1 p1 θ is a function of the evaluation points and therefore is a random
variable whose law is governed by Q.

A.1

Proof of Theorem 3.1

Proof. We adopt the method used in the proof of Theorem 5 in Bull (2011), who proves a convergence
result for an unconstrained optimization problem in which the objective function is unknown outside
the evaluation points.
Below, we let L ě 2k. Let 0 ă ν ă 8. Let 0 ă η ă  and AL P F be the event that at least
tηLu of the points θpk`1q , ¨ ¨ ¨ , θpLq are drawn independently from a uniform distribution on Θ. Let
BL P F be the event that one of the points θpL`1q , ¨ ¨ ¨ , θp2Lq is chosen by maximizing the expected
improvement. For each L, define the mesh norm:
hL ” sup min }θ ´ θp`q }.
θPΘ `“1,¨¨¨L

(A.6)

For a given M̄ ą 0, let CL P F be the event that hL ď M̄ pL{ ln Lq´1{d . We then let
DL ” AL X BL X CL .

(A.7)

On DL , the following results hold. First, let βL be the estimated parameter. Noting that there
are tηLu uniformly sampled points and arguing as in (A.24)-(A.25), it follows that
sup sL pθ; βL q ď M rL ,

(A.8)

θPΘ

for some constant M ą 0 by ω P CL , and rL is defined by
rL ” pL{ ln Lq´ν{d .

(A.9)

For later use, we note that, for any L ě 2,
rL´1 {rL “

` L ˘ν{d ` lnpL ´ 1q ˘ν{d
ď 2ν{d .
L´1
ln L

(A.10)

Second, by ω P BL , there is ` such that L ď ` ď 2L and θp`q is chosen by maximizing the expected
improvement. For θ P Θ and L P N, let IL pθq ” pp1 θ ´ p1 θ˚,L q` 1tḡpθq ď cpθqu. Recall that θ˚ is an

[26]

optimal solution to (3.6). Then,
p1q

p1 θ˚ ´ p1 θ˚,`´1 “ I`´1 pθ˚ q
`
` R ˘˘´1
ď EI`´1 pθ˚ q 1 ´ Φ
ς
p3q
`
` R ˘˘´1
p`q
ď EI`´1 pθ q 1 ´ Φ
ς
¯`
p4q ´
` R ˘˘´1
ď I`´1 pθp`q q ` M1 s`´1 pθp`q q expp´M2 s`´1 pθp`q q´2 q 1 ´ Φ
ς
¯`
p5q ´
`
˘˘
R
´1
´2
ď I`´1 pθp`q q ` M M1 r`´1 expp´M ´2 M2 r`´1
q 1´Φ
ς
¯`
p6q ´
` R ˘˘´1
ď I`´1 pθp`q q ` 2ν{d M M1 r` expp´p2ν{d M q´2 M2 r`´2 q 1 ´ Φ
ς
´
¯`
` R ˘˘´1
“ pp1 θp`q ´ p1 θ˚,`´1 q1tḡpθp`q q ď cpθp`q qu ` 2ν{d M M1 r` expp´p2ν{d M q´2 M2 r`´2 q 1 ´ Φ
ς
¯`
p7q ´
`
˘˘
R
´1
ď pp1 θ˚,` ´ p1 θ˚,`´1 q ` 2ν{d M M1 r` expp´p2ν{d M q´2 M2 r`´2 q 1 ´ Φ
ς
¯`
p8q ´
`
˘˘
R ´1
ď h` ` 2ν{d M M1 r` expp´p2ν{d M q´2 M2 r`´2 q 1 ´ Φ
,
(A.11)
ς
p2q

where (1) follows by construction, (2) follows from Lemma A.1 (ii), (3) follows from θp`q being the
maximizer of the expected improvement, (4) follows from Lemma A.1 (i), (5) follows from (A.8), (6)
follows from r`´1 ď 2ν{d r` for ` ě 2 by (A.10), (7) follows from θ˚,` “ argmaxθPC ` p1 θ, (8) follows from
p1 θ˚,` ´ p1 θ˚,`´1 being dominated by the mesh-norm. Therefore, by ω P CL , there exists a constant
M ą 0 such that
´
¯`
` R ˘˘´1
p1 θ˚ ´ p1 θ˚,`´1 ď M p`{ ln `q´1{d ` M r` expp´M r`´2 q 1 ´ Φ
.
ς

(A.12)

Since L ď ` ď 2L, p1 θ˚,L is non-decreasing in L, and rL is non-increasing in L, we have
´
¯`
` R ˘˘´1
´2
p1 θ˚ ´ p1 θ˚,2L ď M pL{ ln Lq´1{d ` M rL expp´M rL
q 1´Φ
ς
´2
“ Opp2L{ ln 2Lq´1{d q ` Opr2L expp´M r2L
qq, (A.13)

where the last equality follows from the existence of a positive constant C such that rL “ Cr2L and
redefining multiplying constants properly.
Now consider the case ω R DL . By (A.7),
c
QpDL
q ď QpAcL q ` QpBLc q ` QpCLc q.

(A.14)

Let Z` be a Bernoulli random variable such that Z` “ 1 if θp`q is randomly drawn from a uniform
distribution. Then, by the Chernoff bounds (see e.g. Boucheron, Lugosi, and Massart, 2013, p.48),
QpAcL q “ Qp

L
ÿ

Z` ă tηLuq ď expp´pL ´ k ` 1qp ´ ηq2 {2q.

`“k`1

[27]

(A.15)

Further, by the definition of BL ,
QpBLc q “ L ,

(A.16)

and finally by taking M̄ large upon defining the event CL and applying Lemma 4 in Bull (2011), one
has
QpCLc q “ OppL{ ln Lq´γ q,

(A.17)

for any γ ą 0. Combining (A.14)-(A.17), for any γ ą 0,
c
QpDL
q “ OppL{ ln Lq´γ q.

(A.18)

Finally, noting that p1 θ˚ ´ p1 θ˚,2L is bounded by some constant M ą 0 due to the boundedness of Θ,
we have
ż
“

1 ˚

1 ˚,2L

EQ p θ ´ p θ

‰

ż
1 ˚

1 ˚,2L

pθ ´pθ

“
DL

p1 θ˚ ´ p1 θ˚,2L dQ

dQ `
c
DL

´2
“ Opp2L{ ln 2Lq´1{d q ` Opr2L expp´M r2L
qq ` Opp2L{ ln 2Lq´γ q “ op1q, (A.19)

where the second equality follows from (A.13) and (A.18). This completes the proof.
The following lemma is an analog of Lemma 8 in Bull (2011), which links the expected improvement
to the actual improvement achieved by a new evaluation point θ.
Lemma A.1: Suppose Θ Ă Rd is bounded and p P Sd´1 . Suppose the evaluation points pθp1q , ¨ ¨ ¨ , θpLq q
are drawn by Algorithm A.1 and }c}Hβ̄ ď R for some R ą 0. For θ P Θ and L P N, let IL pθq ”
pp1 θ ´ p1 θ˚,L q` 1tḡpθq ď cpθqu. Then, (i) there exist constants Mj ą 0, j “ 1, 2 that only depend on
pς, Rq and an integer L̄ P N such that
EIL pθq ď IL pθq ` M1 sL pθq expp´M2 s´2
L pθqq

(A.20)

for all L ě L̄. Further, (ii) for any L P N and θ P Θ,
´
´ R ¯¯´1
IL pθq ď EIL pθq 1 ´ Φ
.
ς

(A.21)

Proof of Lemma A.1. (i) If sL pθq “ 0, then the posterior variance of cpθq is zero. Hence, EIL pθq “
IL pθq, and the claim of the lemma holds.
For sL pθq ą 0, we first show the upper bound. Let u ” pḡpθq ´ cL pθqq{sL pθq and t ” pḡpθq ´

[28]

cpθqq{sL pθq. By Lemma 6 in Bull (2011), we have |u ´ t| ď R. Since 1 ´ Φp¨q is decreasing, we have
´
´ u ¯¯
EIL pθq “ pp1 θ ´ p1 θ˚,L q` 1 ´ Φ
ς
´
´ t ´ R ¯¯
ď pp1 θ ´ p1 θ˚,L q` 1 ´ Φ
ς
´
´ t ´ R ¯¯
“ pp1 θ ´ p1 θ˚,L q` p1tḡpθq ď cpθqu ` 1tḡpθq ą cpθquq 1 ´ Φ
ς
´
´ t ´ R ¯¯
ď IL pθq ` pp1 θ ´ p1 θ˚,L q` 1tḡpθq ą cpθqu 1 ´ Φ
,
ς

(A.22)

where the last inequality used 1 ´ Φpxq ď 1 for any x P R. Note that one may write
´
´ t ´ R ¯¯
´
´ ḡpθq ´ cpθq ´ s pθqR ¯¯
L
1tḡpθq ą cpθqu 1 ´ Φ
“ 1tḡpθq ą cpθqu 1 ´ Φ
.
ς
ςsL pθq

(A.23)

Below we assume ḡpθq ą cpθq because otherwise, the expression above is 0, and the claim holds. To
be clear about the parameter value at which we evaluate sL , we will write sL pθ; βq. By the hypothesis
that }c}Hβ̄ ď R and Lemma 4 in Bull (2011), we have
}c}HβL ď S,

(A.24)

śd
where S “ R2 k“1 pβ k {β k q. Note that there are tηLu uniformly sampled points, and Kβ is associated
with index ν P p0, 8q, ν R N. By Corollary 6.4 in Narcowich, Ward, and Wendland (2003),
sup sL pθ; βq “ OpM pβqhνL q,

(A.25)

θPΘ

uniformly in β, where hL “ supθPΘ min`“1,¨¨¨L }θ ´ θp`q } and β ÞÑ M pβq is a continuous function (note
that the exponent ν in our notation matches matches pk ` νq{2 in theirs). Hence, sL pθq “ op1q. This,
together with ḡpθq ą cpθq, implies that there are a constant M and L̄ P N such that
0 ă M ă pḡpθq ´ cpθq ´ sL pθqRq{ς, @L ě L̄.

(A.26)

Therefore, again by the fact that 1 ´ Φp¨q is decreasing, one obtains
´
´ ḡpθq ´ cpθq ´ s pθqR ¯¯ ´
´ M ¯¯
L
1tḡpθq ą cpθqu 1 ´ Φ
ď 1´Φ
ςsL pθq
sL pθq
´
sL pθq
M ¯
ď
φ
,
M
sL pθq

(A.27)

where φ is the density of the standard normal distribution, and the last inequality follows from
1 ´ Φpxq ď φpxq{x, which is due to Gordon (1941). The claim on the upper bound then follows from
(A.22), pp1 θ ´ p1 θ˚,L q ď M for some M ą 0 due to Θ being bounded, and (A.27).

[29]

(ii) For the lower bound in (A.21), we have
´
´ t ` R ¯¯
EIL pθq ě pp1 θ ´ p1 θ˚,L q` 1 ´ Φ
ς
´
´ t ` R ¯¯
“ pp1 θ ´ p1 θ˚,L q` 1tḡpθq ď cpθqu 1 ´ Φ
ς
´
´ R ¯¯
ě IL pθq 1 ´ Φ
,
ς

(A.28)

where the last inequality follows from t “ pḡpθq ´ cpθqq{sL pθq ď 0 and the fact that 1 ´ Φp¨q is
decreasing.

[30]

Tables
Table 1: Results for Set 1 with n “ 4000, M Cs “ 300, B “ 301, ρ “ 5.04, κn “
1´α
δ1 “ 0.4
δ2 “ 0.6

0.95
0.90
0.85
0.95
0.90
0.85

CInprof
[0.330,0.495]
[0.340,0.485]
[0.345,0.475]
[0.515,0.655]
[0.525,0.647]
[0.530,0.640]

Median CI
CIn
[0.336,0.482]
[0.342,0.474]
[0.348,0.466]
[0.518,0.650]
[0.533,0.643]
[0.540,0.639]

CInproj
[0.290,0.557]
[0.298,0.543]
[0.303,0.536]
[0.461,0.682]
[0.473,0.675]
[0.481,0.670]

CInprof Coverage
Lower
Upper
0.997
0.990
0.990
0.980
0.970
0.970
0.987
0.993
0.977
0.973
0.967
0.957

CIn Coverage
Lower Upper
0.993
0.973
0.980
0.963
0.960
0.937
0.980
0.987
0.957
0.953
0.943
0.923

?

ln n.

CInproj Coverage
Lower
Upper
1
1
1
1
1
1
1
1
1
1
1
1

Average Time
CInprof
CIn
CInproj
1858.42 22.86
13.82
1873.23 22.26
15.81
1907.84 23.00
13.98
1753.54 23.84
19.10
1782.91 24.45
17.16
1809.65 23.38
17.33

Notes: (1) Projections of ΘI are: δ1 P r0.3872, 0.4239s, δ2 P r0.5834, 0.6084s, ζ1 P r0.0996, 0.1006s, ζ2 P r0.1994, 0.2010s, ζ3 P r0.2992, 0.3014s. (2) “Upper” coverage
is for maxθPΘI pP q p1 θ, and similarly for “Lower”. (3) “Average time” is computation time in seconds averaged over MC replications. (4) CInprof results from
BCS-profiling, CIn is calibrated projection, and CInproj is uncalibrated projection.

[31]

Table 2: Results for Set 1 with n “ 4000, M Cs “ 1000, B “ 1001, ρ “ 5.04, κn “
1´α
δ1 “ 0.4
δ2 “ 0.6
ζ r1s “ 0.1
ζ r2s “ 0.2
ζ r3s “ 0.3

0.95
0.90
0.85
0.95
0.90
0.85
0.95
0.90
0.85
0.95
0.90
0.85
0.95
0.90
0.85

Median CI
CIn
CInproj
[0.333,0.479] [0.288,0.555]
[0.342,0.470] [0.296,0.542]
[0.347,0.464] [0.302,0.534]
[0.526,0.653] [0.466,0.683]
[0.538,0.646] [0.478,0.677]
[0.545,0.642] [0.485,0.672]
[0.054,0.143] [0.020,0.179]
[0.060,0.137] [0.028,0.171]
[0.064,0.132] [0.033,0.166]
[0.156,0.245] [0.120,0.281]
[0.162,0.238] [0.128,0.273]
[0.166,0.235] [0.133,0.268]
[0.257,0.344] [0.222,0.379]
[0.262,0.337] [0.230,0.371]
[0.266,0.333] [0.235,0.366]

Notes: Same DGP and conventions as in Table 1.

CIn Coverage
Lower Upper
0.990
0.979
0.978
0.957
0.960
0.942
0.969
0.978
0.948
0.959
0.925
0.941
0.951
0.952
0.916
0.916
0.868
0.863
0.950
0.949
0.910
0.908
0.869
0.863
0.945
0.944
0.896
0.900
0.866
0.863

CInproj
Lower
1
1
1
1
1
1
1
0.998
0.998
1
0.999
0.995
1
0.998
0.995

Coverage
Upper
1
1
1
1
0.999
1
1
0.998
0.998
1
0.998
0.995
1
0.998
0.995

?

ln n.

Average Time
CIn
CInproj
42.35
15.79
41.13
11.60
39.91
15.36
41.40
24.30
41.39
32.78
38.49
31.55
35.57
20.80
38.42
28.07
38.63
28.77
35.99
18.07
33.29
23.13
33.76
17.33
39.92
31.27
43.37
29.17
43.60
26.99

Table 3: Results for Set 2-DGP1, Corrpu1 , u2 q “ 0, n “ 4000, M Cs “ 1000, ρ “ 6.02, κn “
1´α
r1s

ζ1 “ 0.50
r2s

ζ1 “ 0.25
r1s

∆1 “ ´1
r2s

∆1 “ ´1

0.95
0.90
0.85
0.95
0.90
0.85
0.95
0.90
0.85
0.95
0.90
0.85

Median CI
CIn
CInproj
[0.355,0.715]
[0.127,0.938]
[0.374,0.687]
[0.172,0.902]
[0.387,0.669]
[0.200,0.878]
[0.115,0.354]
[0.003,0.488]
[0.132,0.340]
[0.024,0.464]
[0.142,0.330]
[0.040,0.448]
[-1.321,-0.716] [-1.712,-0.296]
[-1.284,-0.755] [-1.647,-0.368]
[-1.259,-0.778] [-1.611,-0.416]
[-1.179,-0.791]
[-1.443,0.500]
[-1.153,-0.814] [-1.398,-0.544]
[-1.136,-0.832] [-1.370,-0.575]

Coverage
CIn
CInproj
0.948
1
0.902
0.999
0.856
0.996
0.954
0.998
0.904
0.996
0.848
0.996
0.946
1
0.895
0.999
0.849
0.997
0.950
1
0.891
0.999
0.853
0.999

?

ln n.

Average Time
CIn CInproj
82.34
23.56
84.33
21.61
87.33
22.31
103.58
32.63
106.20
26.52
110.10
32.01
88.21
22.11
94.38
22.65
92.77
27.52
96.97
27.31
98.69
25.13
102.16
25.11

Table notes: (1) ΘI is a singleton in this DGP. (2) B “ 1001 bootstrap draws. (3) “Average time” is computation time in
seconds averaged over MC replications. (4) CIn is calibrated projection and CInproj is uncalibrated projection.

Table 4: Results for Set 2-DGP2, Corrpu1 , u2 q “ 0, n “ 4000, M Cs “ 1000, ρ “ 6.02, κn “
1´α
r1s

ζ1 “ 0.50
r2s

ζ1 “ 0.25
r1s

∆1 “ ´1
r2s

∆1 “ ´0.75

0.95
0.90
0.85
0.95
0.90
0.85
0.95
0.90
0.85
0.95
0.90
0.85

Median CI
CIn
CInproj
[0.249,0.790]
[-0.007,1.004]
[0.271,0.765]
[0.038,0.969]
[0.287,0.750]
[0.067,0.948]
[0.112,0.376]
[0.009,0.523]
[0.128,0.359]
[0.025,0.498]
[0.138,0.348]
[0.038,0.489]
[-1.467,-0.497] [-1.869,-0.003]
[-1.432,-0.544] [-1.806,-0.091]
[-1.408,-0.571] [-1.766,-0.146]
[-0.979,-0.514] [-1.276,-0.237]
[-0.953,-0.539] [-1.226,-0.282]
[-0.936,-0.556] [-1.194,-0.312]
r1s

r2s

CIn Coverage
Lower Upper
0.954
0.971
0.918
0.941
0.883
0.919
0.969
0.963
0.938
0.927
0.909
0.891
0.960
0.967
0.932
0.939
0.901
0.902
0.973
0.969
0.941
0.940
0.916
0.917
r1s

CInproj Coverage
Lower
Upper
0.999
1
0.998
1
0.999
1
0.998
1
0.997
0.999
0.998
0.996
0.999
0.999
1
0.999
1
0.999
1
1
1
1
1
0.999

?

ln n.

Average Time
CIn
CInproj
85.76
50.10
91.47
50.51
91.39
61.10
94.09
36.46
93.26
52.80
95.68
61.25
82.54
27.25
89.97
28.63
91.72
28.38
97.75
32.09
95.86
27.34
104.52
31.15
r2s

Notes: (1) Projections of ΘI are: ζ1 P r0.405, 0.589s; ζ1 P r0.236, 0.266s; ∆1 P r´1.158, ´0.832s; ∆1 P r´0.790, ´0.716s.
(2) “Upper” coverage refers to coverage of maxtp1 θ : θ P ΘI pP qu, and similarly for “Lower”. (3) “Average time” is
computation time in seconds averaged over MC replications. (4) B “ 1001 bootstrap draws. (5) CIn is calibrated projection
and CInproj is uncalibrated projection.

[32]

Table 5: Results for Set 2-DGP3, Corrpu1 , u2 q “ 0.5, n “ 4000, M Cs “ 1000, ρ “ 6.02, κn “
1´α
r1s

ζ1 “ 0.50
r2s

ζ1 “ 0.25
r1s

∆1 “ ´1
r2s

∆1 “ ´0.75
r “ 0.5

0.95
0.90
0.85
0.95
0.90
0.85
0.95
0.90
0.85
0.95
0.90
0.85
0.95
0.90
0.85

Median CI
CIn
CInproj
[0.196,0.895]
[-0.043,1.053]
[0.224,0.864]
[-0.009,1.009]
[0.244,0.844]
[0.015,1.000]
[0.099,0.436]
[0.001,0.586]
[0.115,0.417]
[0.016,0.583]
[0.126,0.404]
[0.031,0.564]
[-1.664,-0.372] [-1.956,-0.000]
[-1.609,-0.441] [-1.929,-0.000]
[-1.568,-0.490] [-1.912,-0.000]
[-1.065,-0.504] [-1.312,-0.1938]
[-1.037,-0.525] [-1.286,-0.241]
[-1.021,-0.542] [-1.276,-0.266]
[0.000,0.830]
[0.000,0.925]
[0.000,0.802]
[0.000,0.925]
[0.042,0.784]
[0.000,0.925]
r1s

CIn Coverage
Lower Upper
0.978
0.978
0.958
0.966
0.945
0.945
0.974
0.969
0.951
0.950
0.939
0.941
0.957
0.962
0.939
0.930
0.909
0.916
0.956
0.955
0.940
0.947
0.918
0.928
0.968
0.968
0.935
0.935
0.897
0.897
r2s

CInproj
Lower
0.996
0.993
0.989
0.997
0.997
0.993
0.986
0.986
0.986
0.994
0.994
0.989
0.995
0.994
0.995

Coverage
Upper
0.995
0.984
0.972
0.996
0.997
0.994
0.993
0.996
0.994
0.995
0.997
0.998
0.995
0.995
0.995

r1s

?

ln n.

Average Time
CIn
CInproj
561.66 163.42
583.80 163.42
562.05
99.90
626.00 245.39
597.29 206.35
681.24 234.50
578.63 156.00
594.27 145.85
638.16 132.73
559.10 214.71
553.53 128.71
645.54 129.67
269.98
42.66
308.58
47.55
334.43
49.54
r2s

Notes: (1) Projections of ΘI are: ζ1 P r0.465, 0.533s; ζ1 P r0.240, 0.261s; ∆1 P r´1.069, ´0.927s; ∆1 P
r´0.782, ´0.720s; r P r0.4998, 0.5000s. (2) “Upper” coverage refers to coverage of maxtp1 θ : θ P ΘI pP qu, and
similarly for “Lower”. (3) “Average time” is computation time in seconds averaged over MC replications. (4)
B “ 1001 bootstrap draws. (5) CIn is calibrated projection and CInproj is uncalibrated projection.

[33]

Table 6: Results for Set 2-DGP2, Corrpu1 , u2 q “ 0, n “ 4000, M Cs “ 1000, varying ρ, κn “
1´α

r1s

ζ1 “ 0.50
r1s

∆1 “ ´1

0.95
0.90
0.85
0.95
0.90
0.85

Median CIn
ρ “ 5.87
ρ “ 10
[0.248,0.790]
[0.271,0.766]
[0.286,0.749]
[-1.471,-0.498]
[-1.434,-0.543]
[-1.410,-0.571]

[0.254,0.776]
[0.275,0.754]
[0.289,0.738]
[-1.454,-0.512]
[-1.418,-0.555]
[-1.394,-0.583]

CIn Coverage
ρ “ 5.87
ρ “ 10
Lower Upper Lower Upper
0.959
0.971
0.947
0.962
0.921
0.939
0.908
0.925
0.888
0.916
0.868
0.895
0.964
0.965
0.955
0.959
0.933
0.940
0.927
0.924
0.904
0.905
0.887
0.895

?

ln n.

Average Time
ρ “ 5.87 ρ “ 10
116.19
121.24
115.41
104.34
113.63
114.23

104.14
115.65
112.38
108.77
114.74
119.55

Notes: Same DGP, number of bootstrap draws and conventions as in Table 4. Results are for calibrated
projection CIn .

Table 7: Results for Set 2-DGP2, Corrpu1 , u2 q “ 0, n “ 4000, M Cs “ 1000, ρ “ 6.02, varying κn .
1´α

r1s

ζ1 “ 0.50
r1s

∆1 “ ´1

0.95
0.90
0.85
0.95
0.90
0.85

Median CIn ?
κn “ n1{7
κn “ ln ln n
[0.249,0.790]
[0.270,0.765]
[0.286,0.748]
[-1.469,-0.497]
[-1.432,-0.542]
[-1.408,-0.568]

[0.250,0.787]
[0.274,0.763]
[0.287,0.746]
[-1.464,-0.501]
[-1.426,-0.548]
[-1.402,-0.577]

CIn Coverage ?
κn “ n1{7
κn “ ln ln n
Lower Upper Lower Upper
0.955
0.972
0.955
0.970
0.922
0.943
0.914
0.936
0.891
0.916
0.870
0.901
0.966
0.968
0.956
0.959
0.935
0.938
0.926
0.923
0.909
0.908
0.889
0.892

Average Time
?
κn “ n1{7 κn “ ln ln n
85.11
89.12
89.82
80.33
85.12
86.95

89.65
94.49
92.15
81.70
88.07
89.34

Notes: Same DGP, number of bootstrap draws and conventions as in Table 4. Results are for calibrated
projection CIn .

[34]

References
Anderson, T. W., and H. Rubin (1949): “Estimation of the Parameters of a Single Equation in a
Complete System of Stochastic Equations,” The Annals of Mathematical Statistics, 20, 46–63.
Andrews, D. W. K., S. T. Berry, and P. Jia (2004): “Confidence Regions for Parameters in
Discrete Games with Multiple Equilibria, with an Application to Discount Chain Store Location,”
mimeo.
Andrews, D. W. K., and X. Shi (2013): “Inference Based on Conditional Moment Inequalities,”
Econometrica, 81, 609–666.
Andrews, D. W. K., and G. Soares (2010): “Inference for Parameters Defined by Moment
Inequalities Using Generalized Moment Selection,” Econometrica, 78, 119–157.
Aradillas-Lopez, A., and E. Tamer (2008): “The Identification Power of Equilibrium in Simple
Games,” Journal of Business & Economic Statistics, 26(3), 261–283.
Beresteanu, A., and F. Molinari (2008): “Asymptotic properties for a class of partially identified
models,” Econometrica, 76, 763–814.
Bontemps, C., T. Magnac, and E. Maurin (2012): “Set Identified Linear Models,” Econometrica,
80, 1129–1155.
Boucheron, S., G. Lugosi, and P. Massart (2013): Concentration inequalities: A nonasymptotic
theory of independence. Oxford university press.
Bugni, F. A., I. A. Canay, and X. Shi (2015): “Specification tests for partially identified models
defined by moment inequalities,” Journal of Econometrics, 185(1), 259–282.
(2017): “Inference for subvectors and other functions of partially identified parameters in
moment inequality models,” Quantitative Economics, 8(1), 1–38.
Bull, A. D. (2011): “Convergence rates of efficient global optimization algorithms,” Journal of
Machine Learning Research, 12(Oct), 2879–2904.
Chen, X., T. M. Christensen, and E. Tamer (2017): “MCMC Confidence Sets for Identified
Sets,” Discussion paper, https://arxiv.org/abs/1605.00499.
Chen, X., E. Tamer, and A. Torgovitsky (2011): “Sensitivity Analysis in Semiparametric
Likelihood Models,” Working paper.
Chernozhukov, V., H. Hong, and E. Tamer (2007): “Estimation and Confidence Regions for
Parameter Sets In Econometric Models,” Econometrica, 75, 1243–1284.
Ciliberto, F., and E. Tamer (2009): “Market Structure and Multiple Equilibria in Airline Markets,” Econometrica, 77, 1791–1828.
Dickstein, M., and E. Morales (2016): “What do Exporters Know?,” Discussion paper, mimeo.
Freyberger, J., and B. Reeves (2017a): “Inference Under Shape Restrictions,” mimeo.

[35]

(2017b): “Supplementary Appendix: Inference Under Shape Restrictions,” mimeo.
Gafarov, B., M. Meier, and J. L. Montiel-Olea (2016): “Projection Inference for Set-Identified
SVARs,” mimeo.
Gordon, R. (1941): “Values of Mills’ ratio of area to bounding ordinate and of the normal probability
integral for large values of the argument,” The Annals of Mathematical Statistics.
Grieco, P. L. E. (2014): “Discrete games with flexible information structures: an application to
local grocery markets,” The RAND Journal of Economics, 45(2), 303–340.
Jones, D. R. (2001): “A Taxonomy of Global Optimization Methods Based on Response Surfaces,”
Journal of Global Optimization, 21(4), 345–383.
Jones, D. R., M. Schonlau, and W. J. Welch (1998): “Efficient Global Optimization of Expensive Black-Box Functions,” Journal of Global Optimization, 13(4), 455–492.
Kaido, H. (2016): “A dual approach to inference for partially identified econometric models,” Journal
of Econometrics, 192(1), 269 – 290.
Kaido, H., F. Molinari, and J. Stoye (2017): “Constraint qualifications in projection inference,”
Work in progress.
Kaido, H., F. Molinari, J. Stoye, and M. Thirkettle (2017): “Calibrated Projection in MATLAB,” Discussion paper, available at https://molinari.economics.cornell.edu/docs/KMST_
Manual.pdf.
Kitagawa, T. (2012): “Inference and Decision for Set Identified Parameters Using Posterior Lower
Probabilities,” CeMMAP Working Paper.
Kline, B., and E. Tamer (2015): “Bayesian inference in a class of partially identified models,”
Quantitative Economics, forthcoming.
Magnac, T., and E. Maurin (2008): “Partial Identification in Monotone Binary Models: Discrete
Regressors and Interval Data,” Review of Economic Studies, 75, 835–864.
Mattingley, J., and S. Boyd (2012): “CVXGEN: a code generator for embedded convex optimization,” Optimization and Engineering, 13(1), 1–27.
Mohapatra, D., and C. Chatterjee (2015): “Price Control and Access to Drugs: The Case of
India’s Malaria Market,” Working Paper. Cornell University.
Narcowich, F., J. Ward, and H. Wendland (2003): “Refined error estimates for radial basis
function interpolation,” Constructive approximation.
Pakes, A., J. Porter, K. Ho, and J. Ishii (2011): “Moment Inequalities and Their Application,”
Discussion Paper, Harvard University.
(2015): “Moment Inequalities and Their Application,” Econometrica, 83, 315334.

[36]

Rockafellar, R. T., and R. J.-B. Wets (2005): Variational Analysis, Second Edition. SpringerVerlag, Berlin.
Romano, J. P., and A. M. Shaikh (2008): “Inference for Identifiable Parameters in Partially
Identified Econometric Models,” Journal of Statistical Planning and Inference, 138, 2786–2807.
Santner, T. J., B. J. Williams, and W. I. Notz (2013): The design and analysis of computer
experiments. Springer Science & Business Media.
Schonlau, M., W. J. Welch, and D. R. Jones (1998): “Global versus local search in constrained
optimization of computer models,” New Developments and Applications in Experimental Design,
Lecture Notes-Monograph Series, Vol. 34, 11–25.
Stoye, J. (2009): “More on Confidence Intervals for Partially Identified Parameters,” Econometrica,
77, 1299–1315.
Wan, Y. (2013): “An Integration-based Approach to Moment Inequality Models,” Working Paper.

[37]

Online Appendix:
Confidence Intervals for
Projections of Partially Identified Parameters
Contents
Appendix B Additional Convergence Results and Background Materials for the E-A-M algorithm
and for Computation of ĉn pθq
2
B.1 Theorem B.1: An Approximating Critical Level Sequence for the E-A-M Algorithm . . . . . . . . . .
2
B.2 The kernel of the Gaussian Process and its Associated Function Space . . . . . . . . . . . . . . . . .
6
B.3 A Reformulation of the M-step as a Nonlinear Program . . . . . . . . . . . . . . . . . . . . . . . . .
7
B.4 Root-Finding Algorithm Used to Compute ĉn pθq . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
Appendix C Verification of Assumptions for the Canonical Moment (In)equalities Examples
C.1 Verification of Assumption B.1 in Theorem B.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
C.2 Verification of Assumption 4.3-(II) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9
10
11

Appendix D Proof of Theorems 4.1, 4.2, 4.3 and 4.4
13
D.1 Notation and Structure of the Proof of Theorem 4.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
D.2 Proof of Theorems 4.1 and 4.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
D.3 Proof of Theorems 4.3 and 4.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Appendix E Auxiliary Lemmas
28
E.1 Lemmas Used to Prove Theorems 4.1 and 4.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
E.2 Lemmas Used to Prove Theorem B.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
E.3 Almost Sure Representation Lemma and Related Results . . . . . . . . . . . . . . . . . . . . . . . . 60
Appendix F Further Comparison of Calibrated Projection and BCS-Profiling
63
F.1 Finite Sample Comparison in a Specific Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
F.2 Example of Methods Failure When Assumption 4.3 Fails . . . . . . . . . . . . . . . . . . . . . . . . . 66
Appendix G Comparison with Projection of AS
67
proj
G.1 Necessary and Sufficient Condition for ĉn pθq ă ĉn pθq . . . . . . . . . . . . . . . . . . . . . . . . . . 70

Structure of the Appendix
Section B states and proofs Theorem B.1, which establishes convergence-related results for our E-A-M algorithm.
It also provides provides background material for the E-A-M algorithm, and details on the root-finding algorithm

[1]

that we use to compute ĉn pθq. Section C verifies some of our main assumptions for moment (in)equality models
that have received much attention in the literature. Section D summarizes the notation we use and the structure
of the proof of Theorem 4.1,43 and provides a proof of Theorems 4.1 (both under our main assumptions and under
a high level assumption replacing Assumption 4.3 and dropping the ρ-box constraints), 4.2, 4.3 and 4.4. Section
E contains the statements and proofs of the lemmas used to establish Theorems 4.1 and B.1, as well as a rigorous
derivation of the almost sure representation result for the bootstrap empirical process that we use in the proof
of Theorem 4.1. Section F provides further results comparing our calibrated projection method and the profiling
method proposed by Bugni, Canay, and Shi (2017, BCS-profiling henceforth), and gives an example of methods’
failure (including calibrated projection, BCS-profiling and the method in Pakes, Porter, Ho, and Ishii (2011)) when
some key assumptions are violated. Section G provides a formal comparison of our calibrated projection method
and projection of the confidence set of Andrews and Soares (2010, AS henceforth).
Throughout the Appendix we use the convention 8 ¨ 0 “ 0.

Appendix B

Additional Convergence Results and Background Materials for the E-A-M algorithm and for Computation of
ĉn pθq

B.1

B.1.1

Theorem B.1: An Approximating Critical Level Sequence for the E-A-M Algorithm
Assumption B.1: A Low Level Condition Yielding a Stochastic Lipschitz-Type Property for ĉn

In order to establish convergence of our E-A-M algorithm, we need ĉn to uniformly stochastically exhibit a Lipschitztype property so that its mollified counterpart (see equation (B.1)) is sufficiently smooth and yields valid inference.
Below we provide a low level condition under which we are able to establish the Lipschitz-type property. In Appendix
C.1 we verify the condition for the canonical examples in the moment (in)equality literature.
Assumption B.1: The model P for P satisfies:
(i) |σP,j pθq´1 mj px, θq ´ σP,j pθ1 q´1 mj px, θ1 q| ď M̄ pxq}θ ´ θ1 } with EP rM̄ pXq2 s ă M for all θ, θ1 P Θ, x P
X , j “ 1, ¨ ¨ ¨ , J, and there exists a function F such that |σP,j pθq´1 mj p¨, θq| ď F p¨q for all θ P Θ and
EP r|F pXqM̄ pXq|2 s ă M .
(ii) ϕj is Lipschitz continuous in x P R for all j “ 1, . . . , J.

B.1.2

Statement and Proof of Theorem B.1

For all τ ą 0 let ĉn,τ pθq be a mollified version of ĉn pθq, i.e.:
ż
ż
ĉn,τ pθq “
ĉn pθ ´ νqφτ pνqdν “
Rd

ĉn pθqφτ pθ ´ νqdν,

(B.1)

Rd

43
Section D.1 provides in Table D.0 a summary of the notation used throughout, and in Figure D.1 and Table D.1 a flow
diagram and heuristic explanation of how each lemma contributes to the proof of Theorem 4.1.

[2]

where the family of functions φτ is a mollifier as defined in Rockafellar and Wets (2005, Example 7.19). Choose it
ş
to be a family of bounded, measurable, smooth functions such that φτ pzq ě 0 @z P Rd , Rd φτ pzqdz “ 1 and with
Bτ “ tz : φτ pzq ą 0u “ tz : }z} ď τ u.
Theorem B.1: Suppose Assumptions 4.1, 4.2, 4.4, 4.5 and B.1 hold. Let τn be a positive sequence such that
τn “ n´ζ with ζ ą 1{2. Let tβn u be a positive sequence such that βn “ op1q and }D̂n ´ DP }8 “ OP pβn q. Let
?
εn “ κ´1
nτn _ βn . Then,
n
1.
˜

¸

lim sup sup P
nÑ8

P PP

sup

|ĉn pθq ´ ĉn pθ1 q| ą Cεn

“ 0;

(B.2)

}θ´θ 1 }ďτn

2. Let ĉn,τn be defined as in (B.1) with τn replacing τ . Then there exists C ą 0 such that
´
¯
lim inf inf P }ĉn ´ ĉn,τn }8 ď Cεn “ 1;
nÑ8 P PP

(B.3)

3. There exists R ą 0 such that }ĉn,τn }Hβ ď R.
4. Let Assumption 4.3 also hold. Let tPn , θn u be a sequence such that Pn P P and θn P ΘI pPn q for all n and
?
u
nγ1,Pn ,j pθn q Ñ π1j P Rr´8s , j “ 1, . . . , J, ΩPn Ñ Ω, and DPn pθn q Ñ D. Let
κ´1
n
λρ
ĉn,ρ,τ pθq ” inf ĉn,τ pθ ` ? q.
d
n
λPBn,ρ

(B.4)

For c ě 0, let Un pθn , cq be defined as in (D.25). Then,
lim inf Pn pUn pθn , ĉn,ρ,τn q ‰ Hq ě 1 ´ α.
nÑ8

(B.5)

Proof. We establish each part of the theorem separately.
Part 1. Throughout, let C ą 0 denote a positive constant, which may be different in different appearances.
Define the event
En ” x8 P X 8 : }D̂n ´ DP }8 ď Cβn ,

}Gn pθq ´ Gn pθ1 qq} ď pln nq2 τn ,

sup
}θ´θ 1 }ďτn

?
sup |ηn,j pθq| ď C{ n,
θPΘ

max

sup

j“1,¨¨¨ ,J }θ´θ 1 }ăτn

(
|ηn,j pθq ´ ηn,j pθ1 q| ď Cτn . (B.6)

Note that pln nq2 τn {p´τn ln τn q “ pln nq2 {ζ ln n “ ln n{ζ, and hence tends to 8. By Assumption B.1-(i) and arguing
as in the proof of Theorem 2 in Andrews (1994), condition (E.216) in Lemma E.11 is satisfied with v “ d. Also, by
Lemma E.13, (E.217) in Lemma E.11 holds with γ “ 1. This therefore ensures the conditions of Lemma E.11.
2
Similarly, by Assumption B.1-(i) m2j px, θq{σP,j
pθq satisfies
ˇ m2 px, θq m2 px, θq ˇ ˇ m px, θq m px, θ1 q ˇˇ m px, θq m px, θ1 q ˇ
ˇ j
ˇ ˇ j
ˇˇ j
ˇ
j
j
j
´ 2
`
´
ˇ 2
ˇďˇ
ˇˇ
ˇ
σP,j pθq
σP,j pθq
σP,j pθq
σP,j pθ1 q σP,j pθq
σP,j pθ1 q
ď 2F pxqM̄ pxq}θ ´ θ1 }.

(B.7)
(B.8)

Let F̄ pxq ” 2F pxqM̄ pxq. By Theorem 2.7.11 in van der Vaart and Wellner (2000),
Nrs p}F̄ }L2P , M2P , } ¨ }L2P q ď N p, Θ, } ¨ }q ď pdiampΘq{qd ,

[3]

(B.9)

where N p, Θ, } ¨ }q is the covering number of Θ. This ensures
ż8
b
sup ln Nrs p}F̄ }L2P , M2P , } ¨ }L2P qd ă 8.
0

(B.10)

P PP

Further, for any C ą 0
EP rF̄ 2 pXq1tF̄ pXq ą Cus ď EP rF̄ 2 pXqsP pF̄ pXq ą Cq
ď 4EP r|F pXqM pXq|2 s

}F̄ }L1P
C

ď

4M 2
,
C

(B.11)

which implies limCÑ8 supP PP EP rF̄ 2 pXq1tF̄ pXq ą Cus “ 0. By Theorems 2.8.4 and 2.8.2 in van der Vaart and
Wellner (2000), this implies that SP is Donsker and pre-Gaussian uniformly in P P P. This therefore ensures
the conditions of Lemma E.12 (i). Note also that Assumption B.1-(i) ensures the conditions of Lemma E.12 (ii).
Therefore, by Lemmas E.11-E.12 and Assumption 4.4, for any η ą 0, there exists C ą 0 such that inf P PP P pEn q ě
1 ´ η for all n sufficiently large.
Let θ, θ1 P Θ. For each j, we have
ˇ
ˇ
ˇ b
ˇ
ˇGn,j pθq ` ρD̂n,j pθqλ ` ϕj pξˆn,j pθqq ´ Gbn,j pθ1 q ´ ρD̂n,j pθ1 qλ ´ ϕj pξˆn,j pθ1 qqˇ
ď |Gbn,j pθq ´ Gbn,j pθ1 q| ` ρ}D̂n,j pθq ´ D̂n,j pθ1 q} sup }λ} ` |ϕj pξˆn,j pθqq ´ ϕj pξˆn,j pθ1 qq|. (B.12)
λPB d

8
b
Assume that the sample path tXi u8
i“1 is such that the event En holds. Conditional on tXi ui“1 and using Gn,j pθq ´
b
b
Gn,j pθq “ Gn,j pθqηn,j pθq,

|Gbn,j pθq ´ Gbn,j pθ1 q| ď |Gbn,j pθq ´ Gbn,j pθ1 q| ` 2 sup |Gbn,j pθq| sup |ηn,j pθq|
θPΘ

θPΘ

C
ď |Gbn,j pθq ´ Gbn,j pθ1 q| ` 2 sup |Gbn,j pθq| ? . (B.13)
n
θPΘ
Define the event Fn P C for the bootstrap weights by
Fn ” m n P Q :

sup
}θ´θ 1 }ďτn

(
}Gbn pθq ´ Gbn pθ1 q} ď pln nq2 τn , sup }Gbn pθq} ď C .

(B.14)

θPΘ

By Lemma E.11 (ii) and the asymptotic tightness of Gbn , for any η ą 0, there exists a C such that Pn˚ pFn q ě 1 ´ η
for all n sufficiently large. Suppose that the multinomial bootstrap weight Mn is such that Fn holds. Then, the
?
right hand side of (B.13) is bounded by pln nq2 τn ` C{ n for some C ą 0.
Next, by the triangle inequality and Assumption 4.4,
}D̂n,j pθq ´ D̂n,j pθ1 q} ď }D̂n,j pθq ´ DP,j pθq} ` }DP,j pθq ´ DP,j pθ1 q} ` }D̂n,j pθ1 q ´ DP,j pθ1 q}
ď Cβn ` Cτn . (B.15)
Finally, note that by the Lipschitzness of ϕj , |ϕj pξˆn,j pθqq ´ ϕj pξˆn,j pθ1 qq| ď C|ξˆn,j pθq ´ ξˆn,j pθ1 q| and
ξˆn,j pθq ´ ξˆn,j pθ1 q
”? ´ m̄ pθq
EP rmj pX, θqs ¯ ? ´ m̄n,j pθ1 q
EP rmj pX, θ1 qs ¯ı
n,j
1
n
p1
`
η
pθqq
´
´
n
p1
`
η
pθ
qq
´
“ κ´1
n,j
n,j
n
σP,j pθq
σP,j pθq
σP,j pθ1 q
σP,j pθ1 q
´
? EP rmj pX, θqs EP rmj pX, θ1 qs ¯
` κ´1
n
´
. (B.16)
n
σP,j pθq
σP,j pθ1 q

[4]

Hence,
1
|ξˆn,j pθq ´ ξˆn,j pθ1 q| ď κ´1
n |Gn,j pθq ´ Gn,j pθ q|
ˇ
?
? ˇˇ m̄n,j pθq
m̄n,j pθ1 q
1 ˇ
nˇ
ηn,j pθq ´
η
pθ
q
nDP,j pθ̄q}θ ´ θ1 }. (B.17)
` κ´1
ˇ ` κ´1
n,j
n
n
σP,j pθq
σP,j pθ1 q

By Lemma E.11, the right hand side of (B.17) can be further bounded by
? ˇˇ m̄n,j pθq m̄n,j pθ1 q ˇˇ
2
´1
´
κ´1
pln
nq
τ
`
κ
nˇ
ˇ|ηn,j pθq|
n
n
n
σP,j pθq
σP,j pθ1 q
? ˇˇ m̄n,j pθ1 q ˇˇ
?
` κ´1
nˇ
nτn
ˇ|ηn,j pθq ´ ηn,j pθ1 q| ` Cκ´1
n
n
1
σP,j pθ q
?
?
?
C
2
´1
ď κ´1
nτn ? ` Cκ´1
nτn ` Cκ´1
nτn , (B.18)
n pln nq τn ` κn
n
n
n
where the last inequality follows from Condition (i) and Lemma E.12 (ii).
Combining (B.12), (B.13), (B.15), and (B.16)-(B.18), we obtain
ˇ
ˇ
ˇ b
ˇ
ˇGn,j pθq ` D̂n,j pθqλ ` ϕj pξˆn,j pθqq ´ Gbn,j pθ1 q ´ D̂n,j pθ1 qλ ´ ϕj pξˆn,j pθ1 qqˇ ď Cεn .

(B.19)

˘
`
`
In particular, if 1 Λbn pθ, ρ, ĉn pθqq X tp1 λ “ 0u ‰ H “ 1, it also holds that 1 Λbn pθ1 , ρ, ĉn pθq ` Cεn q X tp1 λ “ 0u ‰
˘
H “ 1 because
Gbn,j pθ1 q ` D̂n,j pθ1 qλ ` ϕj pξˆn,j pθ1 qq ď Gbn,j pθq ` D̂n,j pθqλ ` ϕj pξˆn,j pθqq ` Cεn ď ĉn pθq ` Cεn ,
Recalling that Pn˚ pFn q ě 1 ´ η for all n sufficiently large, we then have
Pn˚

`

(˘
Λbn pθ1 , ρ, ĉn pθq ` Cεn q X tp1 λ “ 0u ‰ H
`
(
˘
ě Pn˚ Λbn pθ1 , ρ, ĉn pθq ` Cεn q X tp1 λ “ 0u ‰ H X Fn
`
(
˘
ě Pn˚ Λbn pθ, ρ, ĉn pθqq X tp1 λ “ 0u ‰ H X Fn ě 1 ´ α ´ η. (B.20)

Since η is arbitrary, we have
ĉn pθ1 q ď ĉn pθq ` Cεn .
Reversing the roles of θ and θ1 and noting that supP PP P pEn q Ñ 0 yields the first claim of the lemma.
Part 2. To obtain the result in equation (B.3), we use that for any θ, θ1 P Θ such that }θ ´ θ1 } ď τn ,
|ĉn pθq ´ ĉn pθ1 q| ď Cεn with probability approaching 1 uniformly in P P P by the result in Part 1. This implies
ˇż
ˇ ż
ˇ
ˇ
|ĉn pθ ´ νq ´ ĉn pθq| φτn pνqdν
|ĉn pθq ´ ĉn,τn pθq| “ ˇˇ
ĉn pθ ´ νqφτn pνqdν ´ ĉn pθqˇˇ ď
Rd
Rd
ż
ż
|ĉn pθ ´ νq ´ ĉn pθq| φτn pνqdν ď Cεn
“
φτn pνqdν ď Cεn .
Bτn

Bτn

Part 3. By the construction of the mollified version of the critical value, we have ĉn,τn P C 8 pΘq (Adams and
Fournier, 2003, Theorem 2.29). Therefore it has derivatives of all order. Using the multi-index notation, for any
s ą 0 and |α| ď s, the partial derivative ∇α ĉn,τn is bounded by some constant M ą 0 on the compact set Θ, and

[5]

hence
ż
|∇α ĉn,τn pθq|2 dυpθq ď M υpΘq ă 8,
Θ

where υ denote the Lebesgue measure on Rd . This ensures ∇α ĉn,τn P L2υ pΘq for all |α| ď s. Hence, ĉn,τn is in the
Sobolev-Hilbert space H s pΘo q for any s ą 0. Note that when a Matérn kernel with ν ă 8 is used and ĉn,τn is
continuous, Lemma 3 in Bull (2011) implies that the RKHS-norm } ¨ }Hβ̄ (in Hβ̄ pΘq) and the Sobolev-Hilbert norm
} ¨ }H ν`d{2 are equivalent. Hence, there is R ą 0 such that }ĉn,τn }Hβ ď C}ĉn,τn }H ν`d{2 ď R.
Part 4. By Part 2 and the definition of ĉn,ρ,τ in (B.4), it follows that
ĉn,ρ,τn pθn q ě ĉn,ρ pθn q ´ en
ě

cIn,ρ pθn q

(B.21)

´ en ,

for some en “ OP pεn q, where the second inequality follows from the construction of cIn,ρ in the proof of Lemma
P

E.1. Note that Lemma E.3 and the fact that εn “ oP p1q by Part 1 imply cIn,ρ pθn q ´ en Ñn c˚π˚ . Replicate equation
(E.22) with ĉn,ρ,τn replacing ĉn,ρ , and mimic the argument following (E.22) in the proof of Lemma E.1. Then, the
conclusion of the lemma follows.

B.2

The kernel of the Gaussian Process and its Associated Function Space

Following Bull (2011), we consider two commonly used classes of kernels. The first one is the Gaussian kernel,
which is given by
d
` ÿ
˘
Kβ pθ ´ θ1 q “ exp ´
|pθk ´ θk1 q{βk |2 , βk P rβ k , β k s, k “ 1, ¨ ¨ ¨ , d,

(B.22)

k“1

where 0 ă β k ă β k ă 8 for all k. The second one is the class of Matérn kernels defined by
Kβ pθ ´ θ1 q “

d
d
¯ν ´? ÿ
¯
21´ν ´? ÿ
2ν
|pθk ´ θk1 q{βk |2 kν
2ν
|pθk ´ θk1 q{βk |2 , ν P p0, 8q, ν R N,
Dpνq
k“1
k“1

where D is the gamma function, and kν is the modified Bessel function of the second kind.44 The index ν controls
the smoothness of Kβ . In particular, the Fourier transform K̂β pζq of the Matérn kernel is bounded from above
and below by the order of }ζ}´2ν´d as }ζ} Ñ 8, i.e. K̂β pζq “ Θp}ζ}´2ν´d q. Similarly, the Fourier transform of
the Gaussian kernel satisfies K̂β pζq “ Op}ζ}´2ν´d q for any ν ą 0. Below, we treat the Gaussian kernel as a kernel
associated with ν “ 8.
Each kernel is associated with a space of functions Hβ pRd q, called the reproducing kernel Hilbert space (RKHS).
Below, we give some background on this space and refer to Steinwart and Christmann (2008); van der Vaart and
van Zanten (2008) for further details. For D Ď Rd , let K : D ˆ D Ñ R be a symmetric and positive definite
function. K is said to be a reproducing kernel of a Hilbert space HpDq if Kp¨, θ1 q P HpDq for all θ1 P D, and
f pθq “ xf, Kp¨, θqyHpDq
holds for all f P HpDq and θ P D. The space HpDq is called a reproducing kernel Hilbert space (RKHS) over
D if for all θ P D, the point evaluation functional δθ : HpDq Ñ R defined by δθ pf q “ f pθq is continuous. When
44
The requirement ν R N is not essential for the convergence result. However, it simplifies some of the arguments as one
can exploit the 2ν-Hölder continuity of Kβ at the origin without a log factor (Bull, 2011, Assumption 4).

[6]

Kpθ, θ1 q “ Kβ pθ ´ θ1 q is used as the correlation functional of the Gaussian process, we denote the associated RKHS
by Hβ pDq. Using Fourier transforms, the norm on Hβ pDq can be written as
ż
ĝpζq
}f }Hβ ” inf
dζ,
(B.23)
g|D “f
K̂β pζq
where the infimum is taken over functions g : Rd Ñ R whose restrictions to D coincide with f , and we take 0{0 “ 0.
The RKHS has a connection to other well-known classes of functions. In particular, when D is a Lipschitz
domain, i.e. the boundary of D is locally the graph of a Lipschitz function (Tartar, 2007) and the kernel is
associated with ν P p0, 8q, Hβ pDq is equivalent to the Sobolev-Hilbert space H ν`d{2 pDo q, which is the space of
functions on Do such that
ż
ĝpζq
}f }2H ν`d{2 ” inf
dζ
(B.24)
g|Do “f
p1 ` }ζ}2 qν`d{2
is finite, where the infimum is taken over functions g : Rd Ñ R whose restrictions to Do coincide with f . Further,
if ν “ 8, Hβ pDq is continuously embedded in H s pDo q for all s ą 0 (Bull, 2011, Lemma 3).
Theorem 3.1 requires that c has a finite RKHS norm. This is to ensure that the approximation error made
by the best linear predictor cL of the Gaussian process regression is controlled uniformly (Narcowich, Ward, and
Wendland, 2003). When a Matérn kernel is used, it suffices to bound the norm in the Sobolev-Hilbert space H ν`d{2
to bound c’s RKHS norm. We do so in Theorem B.1 by introducing a mollified version of ĉn .

B.3

A Reformulation of the M-step as a Nonlinear Program

In (3.13), θpL`1q is defined as the maximizer of the following maximization problem
´
´ ḡpθq ´ c pθq ¯¯
L
˚
,
maxpp1 θ ´ p1 θL
q` 1 ´ Φ
θPΘ
ςˆsL pθq

(B.25)

where ḡpθq “ maxj“1,¨¨¨ ,J gj pθq. Since Φ is strictly increasing, one may rewrite the objective function as
´
´ g pθq ´ c pθq ¯¯
´
´ g pθq ´ c pθq ¯¯
j
L
j
L
˚
˚
pp1 θ ´ p1 θL
“ min pp1 θ ´ p1 θL
.
q` 1 ´ max Φ
q` 1 ´ Φ
j“1,¨¨¨ ,J
j“1,¨¨¨ ,J
ςˆsL pθq
ςˆsL pθq
Hence, θpL`1q is a solution to the maximin problem:
´
´ g pθq ´ c pθq ¯¯
j
L
˚
max min pp1 θ ´ p1 θL
q` 1 ´ Φ
,
θPΘ j“1,¨¨¨ ,J
ςˆsL pθq
which can be solved, for example, by Matlab’s fminimax function. It can also be rewritten as a nonlinear program:
max v
pθ,vqPΘˆR

´
´ g pθq ´ c pθq ¯¯
j
L
˚
s.t.pp1 θ ´ p1 θL
q` 1 ´ Φ
ě v, j “ 1, ¨ ¨ ¨ , J,
ςˆsL pθq
which can be solved by nonlinear optimization solvers, e.g. Matlab’s fmincon or KNITRO. We note that the objective
function and constraints together with their gradients are available in closed form.

B.4

Root-Finding Algorithm Used to Compute ĉn pθq

This section explains in detail how ĉn pθq in equation (3.5) is computed. For a given θ P Θ, P ˚ pΛbn pθ, ρ, cq X tp1 λ “
0u ‰ Hq increases in c (with Λbn pθ, ρ, cq defined in (3.1)), and so ĉn pθq can be quickly computed via a root-

[7]

finding algorithm, such as the Brent-Dekker Method (BDM), see Brent (1971) and Dekker (1969). To do so, define
řB
hα pcq “ B1 b“1 ψb pcq ´ p1 ´ αq where
ψb pcpθqq “ 1pΛbn pθ, ρ, cq X tp1 λ “ 0u ‰ Hq.
Let c̄pθq be an upper bound on ĉn pθq (for example, the asymptotic Bonferroni bound c̄pθq ” Φ´1 p1 ´ α{Jqq.
It remains to find ĉn pθq so that hα pĉn pθqq “ 0 if hα p0q ď 0. It is possible that hα p0q ą 0 in which case we
output ĉn pθq “ 0. Otherwise, we use BDM to find the unique root to hα pcq on r0, c̄pθqs where, by construction,
hα pc̄n pθqq ě 0. We propose the following algorithm:
Step 0 (Initialize)
(i) Set Tol equal to a chosen tolerance value;
(ii) Set cL “ 0 and cU “ c̄pθq (values of c that bracket the root ĉn pθq);
(iii) Set c´1 “ cL and c´2 “ rs to be undefined for now (proposed values of c from 1 and 2 iterations prior). Also
set c0 “ cL and c1 “ cU .
(iv) Compute ϕj pξˆn,j pθqq j “ 1, ¨ ¨ ¨ , J;
(v) Compute D̂P,n pθq;
(vi) Compute Gbn,j for b “ 1, ¨ ¨ ¨ , B, j “ 1, ¨ ¨ ¨ , J;
(vii) Compute ψb pcL q and ψb pcU q for b “ 1, ¨ ¨ ¨ , B;
(viii) Compute hα pcL q and hα pcU q.
Step 1 (Method Selection)
Use the BDM rule to select the updated value of c, say c2 . The value is updated using one of three methods:
Inverse Quadratic Interpolation, Secant, or Bisection. The selection rule is based on the values of ci , i “
´2, ´1, 0, 1 and the corresponding function values.
Step 2 (Update Value Function)
Update the value of hα pc2 q. We can exploit previous computation and monotonicity function ψb pc2 q to reduce
computational time:
1. If ψb pcL q “ ψb pcU q “ 0, then ψb pc2 q “ 0;
2. If ψb pcL q “ ψb pcU q “ 1, then ψb pc2 q “ 1.
Step 3 (Update)
(i) If hα pc2 q ě 0, then set cU “ c2 . Otherwise set cL “ c2 .
(ii) Set c´2 “ c´1 , c´1 “ c0 , c0 “ cL , and c1 “ cU .
(iii) Update corresponding function values hα p¨q.
Step 4 (Convergence)
(i) If hα pcU q ď Tol or if |cU ´ cL | ď Tol , then output ĉn pθq “ cU and exit. Note: hα pcU q ě 0, so this criterion
ensures that we have at least 1 ´ α coverage.
(ii) Otherwise, return to Step 1.

[8]

The computationally difficult part of the algorithm is computing ψb p¨q in Step 2. This is simplified for two reasons.
First, evaluation of ψb pcq entails determining whether a constraint set comprised of J ` 2d ´ 2 linear inequalities in
d ´ 1 variables is feasible. This can be accomplished efficiently employing commonly used software.45 Second, we
exploit monotonicity in ψb p¨q, reducing the number of linear programs needed to be solved.

Appendix C

Verification of Assumptions for the Canonical Moment
(In)equalities Examples

In this section we verify: (i) Assumption B.1 which is the crucial condition in Theorem B.1, and (ii) Assumption
4.3-(II), for the canonical examples in the moment (in)equalities literature:
1. Mean with interval data (of which missing data is a special case). Here we assume that W0 , W1 are
two observable random variables such that P pW0 ď W1 q “ 1. The identified set is defined as
ΘI pP q “ tθ P Θ Ă R : EP pW0 q ´ θ ď 0, θ ´ EP pW1 q ď 0u.

(C.1)

2. Linear regression with interval outcome data and discrete regressors. Here the modeling assumption
is that W “ Z 1 θ ` u, where Z “ rZ1 ; . . . ; Zd s is a d ˆ 1 random vector with Z1 “ 1. We assume that Z
has k points of support denoted z 1 , . . . , z k P Rd with maxr“1,...,k }z r } ă M ă 8. The researcher observes
tW0 , W1 , Zu with P pW0 ď W ď W1 |Z “ z r q “ 1, r “ 1, . . . , k. The identified set is
ΘI pP q “ tθ P Θ Ă Rd : EP pW0 |Z “ z r q ´ z r1 θ ď 0, z r1 θ ´ EP pW1 |Z “ z r q ď 0, r “ 1, . . . , ku.

(C.2)

3. Best linear prediction with interval outcome data and discrete regressors. Here the variables are
defined as for the linear regression case. Beresteanu and Molinari (2008) show that the identified set for the
parameters of a best linear predictor of W conditional on Z is given by the set ΘI pP q “ EP pZZ 1 q´1 EP pZWq,
where W “ rW0 , W1 s is a random closed set and, with some abuse of notation, EP pZWq denotes the Aumann
expectation of ZW.
Here we go beyond the results in Beresteanu and Molinari (2008) and derive a moment inequality representation for ΘI pP q when Z has a discrete distribution. We denote by ur the vector ur “ er1 pMP1 MP q´1 MP1 EP pZZ 1 q,
r “ 1, . . . , k, where er is the r-th basis vector in Rk and MP is a dˆK matrix with r-th column equal to P pZ “
z r qz r ; we let q r “ ur EP pZZ 1 q´1 . Observe that for any selection W̃ P W a.s. one has ur EP pZZ 1 q´1 EP pZ W̃ q “
er1 rEP pW̃ |Z “ z 1 q; . . . ; EP pW̃ |Z “ z k qs, so that the support function in direction ur is maximized/minimized
by setting EP pW̃ |Z “ z r q equal to EP pW1 |Z “ z r q and EP pW0 |Z “ z r q, respectively. Hence, the identified
set can be written in terms of moment inequalities as
ΘI pP q “ tθ P Θ Ă Rd : q r rEP pZpZ 1 θ ´ W0 ´ 1pq r Z ą 0qpW1 ´ W0 qqqs ď 0
´ q r rEP pZpZ 1 θ ´ W0 ´ 1pq r Z ă 0qpW1 ´ W0 qqqs ď 0, r “ 1, . . . , ku.

(C.3)

The set is expressed through evaluation of its support function, given in Bontemps, Magnac, and Maurin
(2012, Proposition 2), at directions ˘ur ; these are the directions orthogonal to the flat faces of ΘI pP q.
4. Complete information entry games with pure strategy Nash equilibrium as solution concept.
45

Examples of high-speed solves for linear programs include CVXGEN, availiable from http://www.cvxgen.com and
Gurobi, available from http://www.gurobi.com.

[9]

Here again we assume that the vector Z has k points of support with bounded norm, and the identified set is
ΘI pP q “ tθ P Θ Ă Rd : equations (5.1), (5.2), (5.3), (5.4) hold for all Z “ z r , r “ 1, . . . , ku.

(C.4)

In the first three examples we let X ” pW0 , W1 , Zq1 . In the last example we let X ” pY1 , Y2 , Zq1 . Throughout, we
propose to estimate EP pW` |Z “ z r q and EP pY1 “ s, Y2 “ t|Z “ z r q, ` “ 0, 1, ps, tq P t0, 1u ˆ t0, 1u and r “ 1, . . . , k,
using
řn
W 1pZi “ z r q
řn `,i
Ên pW` |Z “ z r q “ i“1
,
(C.5)
r
i“1 1pZi “ z q
řn
1pY1,i “ s, Y2,i “ t, Zi “ z r q
řn
Ên pY1 “ s, Y2 “ t|Z “ z r q “ i“1
,
(C.6)
r
i“1 1pZi “ z q
as it is done in, e.g., Ciliberto and Tamer (2009). We assume that for each of the four canonical examples under
consideration, Assumption 4.1 as well as one of the assumptions below hold.
Assumption C.1: The model P for P satisfies min`“0,1 minr“1,...,k V arP pW` |Z “ z r q ą σ ą 0 and
minr“1,...,k P pZ “ z r q ą $ ą 0.
Assumption C.2: The model P for P satisfies: (1) eigpMP1 MP q ą ς; (2) eigpEP pZZ 1 qq ą ς;
(3) eigpCorrP prvechpZZ 1 q; W0 sqq ą ς and eigpCorrP prvechpZZ 1 q; W1 sqq ą ς; for some ς ą 0, where vechpAq denotes
the half-vectorization of the matrix A.
Assumption C.3: The model P for P satisfies minr“1,...,k,ps,tqPt0,1uˆt0,1u P pY1 “ s, Y2 “ t, Z “ z r q ą $ ą 0.
These are simple to verify low level conditions. We note that Imbens and Manski (2004) and Stoye (2009)
directly assume the unconditional version of C.1, while Beresteanu and Molinari (2008) assume C.1 itself.

C.1

Verification of Assumption B.1 in Theorem B.1

We show that in each of the four examples
that DP can be estimated at rate n´1{2 .

mj px,θq
σP,j pθq ,

j “ 1, . . . , J is Lipschitz continuous in θ P Θ for all x P X and

1. Mean with interval data. Here σP,` pθq “ σP,` , and under Assumption C.1 it is uniformly bounded from
below. Then
ˇ
ˇ
ˇ mj px, θq mj px, θ1 q ˇ }pθ1 ´ θq}
ˇ
ˇ“
´
, ` “ 0, 1,
ˇ σP,j
σP,j ˇ
σP,j pθq
DP,` pθq “

p´1qp1´`q
, ` “ 0, 1.
σP,`

Assumption C.1 then guarantees that Assumption B.1 is satisfied.
2. Linear regression with interval outcome data and discrete regressors. Here again σP,`r pθq “ σP,`r ,
and under Assumptions C.1-C.2 it is uniformly bounded from below. We first consider the rescaled function
p´1qj pW` 1pZ“z r q{P pZ“z r q´z r1 θq
:
σP,`r
ˇ
ˇ
1
ˇ p´1qj pW` 1pZ “ z r q{P pZ “ z r q ´ z r1 θq p´1qj pW` 1pZ “ z r q{P pZ “ z r q ´ z r1 θ1 q ˇ
ˇ
ˇ “ }z r } }pθ ´ θq} , ` “ 0, 1,
´
ˇ
ˇ
σP,`r
σP,`r
σP,`r pθq

[10]

so that Assumption B.1 is satisfied for these rescaled functions by Assumptions C.1-C.2. Next, we observe
that
DP,j “

p´1qp1´jq z r1
, ` “ 0, 1, r “ 1, . . . , k,
σP,`r

and it can be estimated at rate n´1{2 by Lemma E.12. Theorem B.1 then holds observing that |P pZ “
řn
z r q{ i“1 1pZi “ z r q ´ 1| “ OP pn´1{2 q and treating this random element similarly to how we treat ηn,j p¨q in
the proof of Theorem B.1.
3. Best linear prediction with interval outcome data and discrete regressors. Here
mr pXi , θq “ q r rZi pZi1 θ ´ pW0,i ` 1pq r Zi ą 0qpW1,i ´ W0,i qqqs

(C.7)

hence is Lipschitz in θ with constant Zi Zi1 . Under Assumptions C.1-C.2, V arP pmr pXi , θqq is uniformly
pXi ,θq
bounded from below, and Lipschitz in θ with a constant that depends on Zi4 . Hence mσrP,r
pθq is Lipschitz in θ
with a constant that depends on powers of Z. Because Z has bounded support, Assumption B.1 is satisfied.
A simple argument yields that DP can be estimated at rate n´1{2 .
4. Complete information entry games with pure strategy Nash equilibrium as solution concept.
Here again σP,str pθq “ σP,str , and under Assumptions 4.1 and C.3 it is uniformly bounded from below. The
result then follows from a similar argument as the one used in Example 2 (Linear regression with interval
outcome data and discrete regressors), observing that the rescaled function of interest is now
1pY1 “ s, Y2 “ t|Z “ z r q{P pZ “ z r q ´ gstr pθq
, ps, tq P t0, 1u ˆ t0, 1u, r “ 1, . . . , k,
σP,str
and the gradient is
1
∇θ gstr pθq, ps, tq P t0, 1u ˆ t0, 1u, r “ 1, . . . , k,
σP,str
where gstr pθq are model-implied entry probabilities, and hence taking their values in r0, 1s. The entry models
typically posited assume that payoff shocks have smooth distributions (e.g., multivariate normal), yielding
that ∇θ gstr pθq is well defined and bounded.

C.2

Verification of Assumption 4.3-(II)

Here we verify Assumption 4.3-(II) for the canonical examples in the moment (in)equalities literature:
1. Mean with interval data. In the generalization of this example in Imbens and Manski (2004) and Stoye
(2009), equations (4.1)-(4.2) are satisfied by construction, equation (4.3) is directly assumed.
2. Linear regression with interval outcome data and discrete regressors. Equation (4.1) is satisfied by
construction. Given the estimator that we use for the population moment conditions, we verify equation (4.3)
?
for the variances of the limit distribution of the vector r npÊn pW` |Z “ z r q ´ EP pW` |Z “ z r qqs`Pt0,1u,r“1,...,k .
We then have that equation (4.3) follows from Assumption C.1. Concerning equation (4.3), this needs to be
verified for the correlation matrix of the limit distribution of a r ˆ 1 random vector that for each r “ 1, . . . , k
?
?
equals any choice in t npÊn pW0 |Z “ z r q ´ EP pW0 |Z “ z r qq, npÊn pW1 |Z “ z r q ´ EP pW1 |Z “ z r qqu, which
suffices for our results to hold. We then have that (4.2) holds because the correlation matrix is diagonal.

[11]

3. Best linear prediction with interval outcome data and discrete regressors. Equation (4.1) is again
satisfied by construction. Equation (4.2) holds under Assumptions C.1-C.2. Equation (4.3) is verified to hold
under Assumption C.1 in Beresteanu and Molinari (2008, p. 808).
4. Complete information entry games with pure strategy Nash equilibrium as solution concept.
In this case equations (5.3) and (5.4) are paired, but the corresponding moment functions differ by the model
implied probability of the region of multiplicity, hence equation (4.1) is satisfied by construction. Given
the estimator that we use for the population moment conditions, we verify equations (4.2) and (4.3) for the
?
variances and for the correlation matrix of the limit distribution of the vector npÊn pY1 “ s, Y2 “ t|Z “
z r q ´ EP pY1 “ s, Y2 “ t|Z “ z r qps,tqPt0,1uˆt0,1u,r“1,...,k q, which suffices for our results to hold. Equation
(4.2) holds provided that |CorrpYi1 p1 ´ Yi2 q, Yi1 Yi2 q| ă 1 ´  for some  ą 0 and Assumption C.3 holds.46
To see that equation (4.3) also holds, note that Assumption C.3 yields that P pYi1 “ 1, Yi2 “ 0, Zi “ z r q is
uniformly bounded away from 0 and 1, thereby implying that for each ps, tq P t0, 1u ˆ t0, 1u, r “ 1, . . . , k,
pP pY1 “ s, Y2 “ t|Z “ z r qp1 ´ P pY1 “ s, Y2 “ t|Z “ z r qqq{pP pZ “ z r qp1 ´ P pZ “ z r qqq is uniformly bounded
away from zero.

46
In more general instances with more than two players, it follows if the multinomial distribution of outcomes of the game
(reduced by one element) has a correlation matrix with eigenvalues uniformly bounded away from zero.

[12]

Appendix D
D.1

Proof of Theorems 4.1, 4.2, 4.3 and 4.4

Notation and Structure of the Proof of Theorem 4.1

For any sequence of random variables tXn u and a positive sequence an , we write Xn “ oP pan q if for any , η ą 0,
there is N P N such that supP PP P p|Xn {an | ą q ă η, @n ě N . We write Xn “ OP pan q if for any η ą 0, there is a
M P R` and N P N such that supP PP P p|Xn {an | ą M q ă η, @n ě N .
Table D.0: Important notation. Here pPn , θn q P tpP, θq : P P P, θ P ?
ΘI pP qu is a subsequence as defined in (D.3)-(D.4) below,
? d
1
d
d
d
d
θn P pθn ` ρ{ nB q X Θ, B “ tx P R : |xi | ď 1, i “ 1, . . . , du, Bn,ρ ” ρn pΘ ´ θn q X B d , Bdρ “ limnÑ8 Bn,ρ
, and λ P Rd .
Gn,j p¨q

“

Gbn,j p¨q

“

ηn,j p¨q

“

?
npm̄n,j p¨q´EP pmj pXi ,¨qqq
, j “ 1, . . . , J
σP,j p¨q
?
npm̄bn,j p¨q´m̄n,j p¨qq
, j “ 1, . . . , J
σ̂n,j p¨q
σP,j p¨q
´
1,
j
“
1,
...,J
σ̂n,j p¨q

´

DP,j p¨q

“

γ1,Pn ,j p¨q

“

π1,j

“

˚
π1,j

“

ξˆn,j p¨q

“

ϕ˚j pξq

“

un,j,θn pλq

“

Sample empirical process.
Bootstrap empirical process.
Estimation error in sample moments’ asymptotic standard deviation.

¯

EP pmj pXi ,¨qq
, j “ 1, . . . , J
∇θ
σP,j p¨q
EPn pmj pXi ,¨qq
, j “ 1, . . . , J
σPn ,j p¨q

Gradient of population moments w.r.t. θ, with estimator D̂n,j p¨q.

?
limnÑ8 κ´1
nγ1,Pn ,j pθn1 q
n
"
0, if π1,j “ 0,
´8, if π1,j ă 0.
" ´1 ?
κn nm̄n,j p¨q{σ̂n,j p¨q, j “ 1, . . . , J1
0,
j “ J1 ` 1, . . . , J
$
’
&ϕj pξq π1,j “ 0
´8
π1,j ă 0
’
%
0
j “ J1 ` 1, ¨ ¨ ¨ , J.
λρ
˚ up1 ` η pθ `
q ` ρDPn ,j pθ̄n qλ ` π1,j
tGn,j pθn ` ?
n,j n
n

?
Limit of rescaled population moments, constant @θn1 P pθn ` ρ{ nB d q X Θ
by Lemma E.5.

Studentized population moments.

“Oracle” GMS.
Rescaled studentized sample moments, set to 0 for equalities.

Infeasible GMS that is less conservative than ϕj .
λρ
?
qq
n

Mean value expansion of nonlinear constraints with sample empirical process
and “oracle” GMS, with θ̄n componentwise between θn and θn `

Un pθn , cq

“

wj pλq

“

d : p1 λ “ 0 X u
λ P Bn,ρ
n,j,θn pλq ď c, @j “ 1, . . . , J
˚
Zj ` ρDj λ ` π1,j

λ P Bdρ : p1 λ “ 0 X wj pλq ď c, @j “ 1, . . . , J

Wpcq

“

cπ˚

“

inftc P R` : PrpWpcq ‰ Hq ě 1 ´ αu.

b
vn,j,θ
1 pλq
n

“

Gbn,j pθn1 q ` ρD̂n,j pθn1 qλ ` ϕj pξˆn,j pθn1 qq

Vnb pθn1 , cq

“

I
vn,j,θ
1 pλq
n

“

(

Feasible set for linearized limit problem intersected with p1 λ “ 0.
Limit problem critical level.
Linearized constraints with bootstrap empirical process and sample GMS.
(

Gbn,j pθn1 q ` ρD̂n,j pθn1 qλ ` ϕ˚j pξˆn,j pθn1 qq

“

ĉn pθq

“

inftc P R` : P ˚ pVnb pθ, cq ‰ Hq ě 1 ´ αu

ĉn,ρ pθq

“

inf λPBn,ρ
ĉn pθ `
d

d : p1 λ “ 0 X v I
λ P Bn,ρ
1 pλq ď c, @j “ 1, . . . , J
n,j,θn

M pθq
σ̂n,j

“

µ̂n,j pθqσ̂n,j pθq ` p1 ´ µ̂n,j pθqqσ̂n,j`R1 pθq

λρ
?
q
n

Feasible set for nonlinear sample problem intersected with p1 λ “ 0.
Linearized constraints with a Gaussian shift and “oracle” GMS.

(

d : p1 λ “ 0 X v b
λ P Bn,ρ
1 pλq ď c, @j “ 1, . . . , J
n,j,θn

VnI pθn1 , cq

λρ
?
.
n

Feasible set for linearized bootstrap problem with sample GMS and p1 λ “ 0.
Linearized constraints with bootstrap empirical process and infeasible sample GMS.

(

Feasible set for linearized bootstrap problem with infeasible sample GMS and p1 λ “ 0.
Bootstrap critical level.
d neighborhood of θ.
Smallest value of the bootstrap critical level in a Bn,ρ

Weighted sum of the estimators of the standard deviations of paired inequalities

[13]

Figure D.1: Structure of Lemmas used in the proof of Theorem 4.1.
Theorem 4.1

Lemma E.1

Lemma E.2

Lemma E.10

Lemma E.3

Lemma E.4

Lemma E.5

Lemma E.6
Lemma E.9

Lemma E.8

Lemma E.7

Table D.1: Heuristics for the role of each Lemma in the proof of Theorem 4.1. Notes: (i) Uniformity in Theorem 4.1 is enforced
arguing along subsequences; (ii) When needed, random variables are realized on the same probability space as shown in Lemma
E.1 and Lemma E.17 (see Appendix E.3 for details); (iii) Here
? pPn , θn q P tpP, θq : P P P, θ P ΘI pP qu is a subsequence as defined in
(D.3)-(D.4) below; (iv) All results hold for any θn1 P pθn ` ρ{ nB d q X Θ.

Theorem 4.1

Pn pp1 θn P CIq ě Pn pUn pθn , ĉn,ρ pθn qq ‰ Hq .
Coverage is conservatively estimated by the probability that Un is nonempty.

Lemma E.1

lim inf Pn pUn pθn , ĉn,ρ pθn qq ‰ Hq ě 1 ´ α.

Lemma E.2

Pn pU pθn , cIn pθn qq ‰ H, Wpcπ˚ q “ Hq ` Pn pU pθn , cIn pθn qq “ H, Wpcπ˚ q ‰ Hq “ oP p1q.
Argued by comparing Un and its limit W (after coupling).

Lemma E.3

Pn˚ pVnI pθn1 , cq ‰ Hq ´ PrpWpcq ‰ Hq Ñ 0 and cIn pθn1 q Ñn cπ˚ if cπ˚ ą 0.
The bootstrap critical value that uses the less conservative GMS yileds a convergent critical value.

Lemma E.4

I
supλPB d | maxj pun,j,θn pλq ´ cIn pθn qq ´ maxj pwj pλq ´ cπ˚ q| “ oP p1q, and similarly for wj and vn,j,θ
1 .
n
The criterion functions entering Un and W converge to each other.

Lemma E.5

Local-to-binding constraints are selected by GMS uniformly over the ρ-box (intuition: ρn´1{2 “ oP pκ´1
n q),
? ´1 1
1 qs} “ o p1q.
and }ξˆn pθn1 q ´ κ´1
nσ
pθ
qE
rm
pX
,
θ
j
i
P
P
n
n
n
Pn ,j n

Lemma E.6

@η ą 0 Dδ ą 0, : PrptWpcq ‰ Hu X tW´δ pcq “ Huq ă η, and similarly for VnI .
It is unlikely that these sets are nonempty but become empty upon slightly tightening stochastic constraints.

Lemma E.7

Intersections of constraints whose gradients are almost linearly dependent are unlikely to realize inside W.
Hence, we can ignore irregularities that occur as linear dependence is approached.

Lemma E.8

If there are weakly more equality constraints than parameters, then c is uniformly bounded away from zero.
This simplifies some arguments.

Lemma E.9

If two paired inequalities are local to binding, then they are also asymptotically identical up to sign.
This justifies “merging” them.

Lemma E.10

ηn,j p¨q converges to zero uniformly in P and θ.

P

[14]

D.2

Proof of Theorems 4.1 and 4.2

D.2.1

Main Proofs

Proof of Theorem 4.1
Following Andrews and Guggenberger (2009), we index distributions by a vector of nuisance parameters relevant
for the asymptotic size. For this, let γP ” pγ1,P , γ2,P , γ3,P q, where γ1,P “ pγ1,P,1 , ¨ ¨ ¨ , γ1,P,J q with
´1
γ1,P,j pθq “ σP,j
pθqEP rmj pXi , θqs, j “ 1, ¨ ¨ ¨ , J,

(D.1)

γ2,P “ pspp, ΘI pP qq, vechpΩP pθqq, vecpDP pθqqq, and γ3,P “ P . We proceed in steps.
Step 1. Let tPn , θn u P tpP, θq : P P P, θ P ΘI pP qu be a sequence such that
lim inf inf

inf

nÑ8 P PP θPΘI pP q

P pp1 θ P CIn q “ lim inf Pn pp1 θn P CIn q,
nÑ8

(D.2)

with CIn “ r´sp´p, Cn pĉn qq, spp, Cn pĉn qqs. We then let tln u be a subsequence of tnu such that
lim inf Pn pp1 θn P CIn q “ lim Pln pp1 θln P CIln q.
nÑ8

nÑ8

(D.3)

Then there is a further subsequence tan u of tln u such that
?
´1
lim κ´1
an an σPan ,j pθan qEPan rmj pXi , θan qs “ π1,j P Rr´8s , j “ 1, . . . , J.

an Ñ8

(D.4)

To avoid multiple subscripts, with some abuse of notation we write pPn , θn q to refer to pPan , θan q throughout this
Appendix. We let
#
0
if π1,j “ 0,
˚
(D.5)
π1,j “
´8 if π1,j ă 0.
The projection of θn is covered when
´ sp´p, Cn pĉn qq ď p1 θn ď spp, Cn pĉn qq
+
#
+
#
1
sup
p
ϑ
inf p1 ϑ
?
?
ô
ď p1 θn ď
nm̄n,j pϑq
nm̄n,j pϑq
s.t. ϑ P Θ,
ď ĉn pϑq, @j
s.t. ϑ P Θ,
ď ĉn pϑq, @j
σ̂n,j pϑq
σ̂n,j pϑq
$
,
&
.
inf λ p1 λ
?
λρ
?
? q
ô
ď0
n
m̄
pθ
`
n,j
n
λρ
n
%s.t.λ P ρn pΘ ´ θn q,
ď ĉn pθn ` ?
q, @j λρ
n
σ̂n,j pθn ` ?
q
n
$
,
1
&
.
supλ p λ
?
λρ
?
?
ď
n
m̄
pθ
`
q
n,j
n
λρ
n
%s.t.λ P ρn pΘ ´ θn q,
ď ĉn pθn ` ?
q, @j λρ
n
?
σ̂n,j pθn `

(D.6)

q
n

$
’
&

,
inf
p1 λ
/
.
?λ
n
s.t.λ P ρ pΘ ´ θn q,
ô
ď0
’
/
?
%
λρ
λρ
λρ
λρ
?
?
?
nγ
pθ
`
tGn,j pθn ` ?
q
`
ρD
p
θ̄
qλ
`
qup1
`
η
pθ
`
qq
ď
ĉ
pθ
`
q,
@j
Pn ,j n
1,Pn ,j n
n,j n
n n
n
n
n
n
$
,
1
sup
pλ
’
/
&
.
?λ
n
s.t.λ P ρ pΘ ´ θn q,
ď
, (D.7)
’
/
?
%
λρ
λρ
λρ
?
?
?
tGn,j pθn ` n q ` ρDPn ,j pθ̄n qλ ` nγ1,Pn ,j pθn qup1 ` ηn,j pθn ` n qq ď ĉn pθn ` n q, @j
?
with ηn,j p¨q ” σP,j p¨q{σ̂n,j p¨q ´ 1 and where we localized ϑ in a n{ρ-neighborhood of Θ ´ θn and we took a mean

[15]

value expansion yielding @j
?
λρ
nm̄n,j pθn ` ?
q
n
σ̂n,j pθn `
d
Denote Bn,ρ
”

?

n
ρ pΘ

λρ
?
q
n

?
λρ
λρ
“ tGn,j pθn ` ? q ` ρDPn ,j pθ̄n qλ ` nγ1,Pn ,j pθn qup1 ` ηn,j pθn ` ? qq.
n
n

(D.8)

´ θn q X B d , with B d “ tx P Rd : |xi | ď 1, i “ 1, . . . , du. The event in (D.7) is implied by

$
’
&

,
inf λ p1 λ
/
.
d
s.t.λ P Bn,ρ ,
ð
ď0
/
’
?
%tG pθ ` ?
λρ
λρ
λρ
?
?
q
`
ρD
p
θ̄
qλ
`
nγ
pθ
qup1
`
η
pθ
`
qq
ď
ĉ
pθ
`
q,
@j
Pn ,j n
1,Pn ,j n
n,j n
n n
n,j n
n
n
n
$
1
supλ p λ
’
&
d
s.t.λ P Bn,ρ
,
ď
’
?
%tG pθ ` ?
λρ
λρ
λρ
nγ
pθ
qup1
` ηn,j pθn ` ?
q
`
ρD
p
θ̄
qλ
`
qq ď ĉn pθn ` ?
q, @j
1,Pn ,j n
n,j n
Pn ,j n
n
n
n

,
/
.
,
/
(D.9)

Step 2. This step is used only when Assumption 4.3-(II) is invoked. When this assumption is invoked, recall that
in equation (2.5) we use the estimator specified in Lemma E.10 equation (E.188) for σP,j , j “ 1, . . . , 2R1 (with
R1 ď J1 {2 defined in the statement of the assumption). In equation (3.1) we use the sample analog estimators of
σP,j for all j “ 1, . . . , J. To keep notation manageable, we explicitly denote the estimator used in (2.5) by σ̂jM only
in this step but in almost all other parts of this Appendix we use the generic notation σ̂j .
For each j “ 1, . . . , R1 such that
˚
˚
π1,j
“ π1,j`R
“ 0,
1

(D.10)

where π1˚ is defined in (D.5), let
$
&
1
λρ
µ̃j “
γ1,Pn ,j`R1 pθn qp1`ηn,j`R1 pθn ` ?
qq
n
%
λρ
λρ
qq`γ
pθ
qp1`η
γ1,Pn ,j`R1 pθn qp1`ηn,j`R1 pθn ` ?
n
n,j pθn ` ?n qq
1,Pn ,j
n
$
&
0
λρ
µ̃j`R1 “
γ1,Pn ,j pθn qp1`ηn,j pθn ` ?
qq
n
%
λρ
λρ
?
?
γ1,Pn ,j`R1 pθn qp1`ηn,j`R1 pθn `

n

qq`γ1,Pn ,j pθn qp1`ηn,j pθn `

n

if γ1,Pn ,j pθn q “ 0 “ γ1,Pn ,j`R1 pθn q,
otherwise,
if γ1,Pn ,j pθn q “ 0 “ γ1,Pn ,j`R1 pθn q,

qq

otherwise,

For each j “ 1, . . . , R1 , replace the constraint indexed by j, that is
?
λρ
nm̄n,j pθn ` ?
q
λρ
n
ď ĉn pθn ` ? q,
λρ
M
n
σ̂n,j pθn ` ?n q
with the following weighted sum of the paired inequalities
?
?
λρ
λρ
nm̄n,j pθn ` ?
q
nm̄j`R1 ,n pθn ` ?
q
λρ
n
n
µ̃j
´
µ̃
ď ĉn pθn ` ? q,
j`R
1
λρ
λρ
M
M
n
?
?
σ̂n,j pθn ` n q
σ̂n,j`R1 pθn ` n q
and for each j “ 1, . . . , R1 , replace the constraint indexed by j ` R1 , that is
?
λρ
nm̄j`R1 ,n pθn ` ?
q
λρ
n
ď ĉn pθn ` ? q,
λρ
M
n
?
σ̂n,j`R
pθ
`
q
n
n
1

[16]

(D.11)

(D.12)

(D.13)

(D.14)

(D.15)

with
?
´µ̃j

λρ
?
q
n
λρ
? q
n

nm̄n,j pθn `
M pθ `
σ̂n,j
n

?
` µ̃j`R1

λρ
?
q
n
λρ
? q
n

nm̄j`R1 ,n pθn `
M
σ̂n,j`R
pθn `
1

λρ
ď ĉn pθn ` ? q,
n

(D.16)

It then follows from Assumption 4.3-(II) that these replacements are conservative because
m̄j`R1 ,n pθn `
M
σ̂n,j`R
pθn `
1

λρ
?
q
n
λρ
? q
n

m̄n,j pθn `
ď´

M pθ `
σ̂n,j
n

λρ
?
q
n
,
λρ
? q
n

and therefore (D.14) implies (D.13) and (D.16) implies (D.15).
Step 3. Next, we make the following comparisons:
?
˚
˚
π1,j
“ 0 ñ π1,j
ě nγ1,Pn ,j pθn q,
?
˚
π1,j
“ ´8 ñ nγ1,Pn ,j pθn q Ñ ´8.

(D.17)
(D.18)

?

˚
˚
For any constraint j for which π1,j
“ 0, (D.17) yields that replacing nγ1,Pn ,j pθn q in (D.9) with π1,j
introduces a
conservative distortion. Under Assumption 4.3-(II), for any j such that (D.10) holds, the substitutions in (D.14)
?
?
λρ
λρ
qq ´ µ̃j`R1 nγ1,Pn ,j`R1 pθn qp1 ` ηn,j`R1 pθn ` ?
qq “ 0, and
and (D.16) yield µ̃j nγ1,Pn ,j pθn qp1 ` ηn,j pθn ` ?
n
n
˚
˚
therefore replacing this term with π1,j “ 0 “ π1,j`R1 is inconsequential.
?
˚
˚
For any j for which π1,j
“ ´8, (D.18) yields that for n large enough, nγ1,Pn ,j pθn q can be replaced with π1,j
.
d
To see this, note that by the Cauchy-Schwarz inequality, Assumption 4.4 (i)-(ii), and λ P Bn,ρ , it follows that
?
?
?
ρDPn ,j pθ̄n qλ ď ρ dp}DPn ,j pθ̄n q ´ DPn ,j pθn q} ` }DPn ,j pθn q}q ď ρ dpρM { n ` M̄ q,
(D.19)

where M̄ and M are as defined in Assumption 4.4-(i) and (ii) respectively, and we used that θ̄n lies component-wise
λρ
between θn and θn ` ?
. Using that Gn,j is asymptotically tight by Assumption 4.5, we have that for any τ ą 0,
n
there exists a T ą 0 and N1 P N such that for all n ě N1 ,
˜
¸
?
λρ
λρ
d
Pn
max
tGn,j pθn ` ? q ` ρDPn ,j pθ̄n qλ ` nγ1,Pn ,j pθn qup1 ` ηn,j pθn ` ? qq ď 0, @λ P Bn,ρ ą 1 ´ τ {2.
˚
n
n
j:π1,j
“´8
(D.20)
?

˚
To see this, note that πij
“ ´8 if and only if limnÑ8 κnn γ1Pn j pθn q “ π1j P r´8, 0q. Suppose first that π1j ą ´8.
ˇ
ˇ?
ˇ
ˇ
Then for all  ą 0 there exists N2 P N such that ˇ κnn γ1Pn j pθn q ´ π1j ˇ ď , for all n ě N2 . Choose  ą 0 such that

[17]

π1j `  ă 0. Let N “ maxtN1 , N2 u. Then we have
˜
¸
?
λρ
λρ
d
tGn,j pθn ` ? q ` ρDPn ,j pθ̄n qλ ` nγ1,Pn ,j pθn qup1 ` ηn,j pθn ` ? qq ď 0, @λ P Bn,ρ
Pn
max
˚
n
n
“´8
j:π1,j
˜
¸
?
?
λρ
λρ
tT ` ρpM̄ ` ρM { nq ` nγ1,Pn ,j pθn qup1 ` ηn,j pθn ` ? qq ď 0 X max
Gn,j pθn ` ? q ď T
ě Pn
max
˚
˚
n
n
“´8
“´8
j:π1,j
j:π1,j
¸
˜
?
λρ
λρ
Gn,j pθn ` ? q ď T
tT ` ρpM̄ ` ρM { nq ` κn pπ1j ` qup1 ` ηn,j pθn ` ? qq ď 0 X max
ě Pn
max
˚
˚
n
n
“´8
“´8
j:π1,j
j:π1,j
˜
¸
"
*
?
T
ρ
λρ
λρ
“ Pn
max
`
pM̄ ` ρM { nq ` pπ1j ` q p1 ` ηn,j pθn ` ? qq ď 0 X max
Gn,j pθn ` ? q ď T
˚
˚
κn
n
n
j:π1,j
“´8 κn
j:π1,j
“´8
˜
¸
λρ
Gn,j pθn ` ? q ď T ą 1 ´ τ {2, @n ě N.
“Pn
max
˚
n
j:π1,j “´8
If π1j “ ´8 the same argument applies a fortiori. We therefore have that for n ě N ,
,
$
˜’
inf λ p1 λ
/
.
&
d
s.t.λ P Bn,ρ ,
ď0
Pn
/
’
?
%tG pθ ` ?
λρ
λρ
λρ
?
?
q ` ρDPn ,j pθ̄n qλ ` nγ1,Pn ,j pθn qup1 ` ηn,j pθn ` n qq ď ĉn pθn ` n q, @j
n,j n
n
$
,
¸
supλ p1 λ
’
/
&
.
d
s.t.λ P Bn,ρ ,
ď
’
/
?
%tG pθ ` ?
λρ
λρ
λρ
q ` ρDPn ,j pθ̄n qλ ` nγ1,Pn ,j pθn qup1 ` ηn,j pθn ` ?
qq ď ĉn pθn ` ?
q, @j n,j n
n
n
n
(D.21)
$
˜’
&
ěPn

1

,
/
.

inf λ p λ
d
,
s.t.λ P Bn,ρ
ď0
’
/
%tG pθ ` ?
λρ
λρ
λρ
˚
?
?
q ` ρDPn ,j pθ̄n qλ ` π1,j up1 ` ηn,j pθn ` n qq ď ĉn pθn ` n q, @j
n,j n
n
$
,
¸
supλ p1 λ
’
/
&
.
d
s.t.λ P Bn,ρ ,
ď
´ τ {2. (D.22)
’
/
%tG pθ ` ?
λρ
λρ
λρ
˚
?
?
q ` ρDPn ,j pθ̄n qλ ` π1,j up1 ` ηn,j pθn ` n qq ď ĉn pθn ` n q, @j
n,j n
n

Since the choice of τ is arbitrary, the limit of the term in (D.21) is not smaller than the limit of the first term in
(D.22). Hence, we continue arguing for the event whose probability is evaluated in (D.22).
λρ
Finally, by definition ĉn p¨q ě 0 and therefore inf λPBn,ρ
ĉn pθn ` ?
q exists. Therefore, the event whose probability
d
n
is evaluated in (D.22) is implied by the event
$
,
inf λ p1 λ
’
/
&
.
d
s.t.λ P Bn,ρ
,
ď0
’
/
%tG pθ ` ?
λρ
λρ
λρ
˚
?
?
q ` ρDPn ,j pθ̄n qλ ` π1,j up1 ` ηn,j pθn ` n qq ď inf λPBn,ρ
ĉn pθn ` n q, @j
d
n,j n
n
$
,
supλ p1 λ
’
/
&
.
d
s.t.λ P Bn,ρ
,
ď
(D.23)
’
/
%tG pθ ` ?
λρ
λρ
λρ
˚
q ` ρDPn ,j pθ̄n qλ ` π1,j up1 ` ηn,j pθn ` ?n qq ď inf λPBn,ρ
ĉn pθn ` ?n q, @j
d
n,j n
n

[18]

For each λ P Rd , define
λρ
λρ
˚
un,j,θn pλq ” tGn,j pθn ` ? q ` ρDPn ,j pθ̄n qλ ` π1,j
up1 ` ηn,j pθn ` ? qq,
n
n

(D.24)

˚
˚
where under Assumption 4.3-(II) when π1,j
“ 0 and π1,j`R
“ 0 the substitutions of equation (D.13) with equation
1
(D.14) and of equation (D.15) with equation (D.16) have been performed. Let
(
d
Un pθn , cq ” λ P Bn,ρ
: p1 λ “ 0 X un,j,θn pλq ď c, @j “ 1, . . . , J ,
(D.25)

and define
λρ
ĉn,ρ ” inf ĉn pθ ` ? q.
d
n
λPBn,ρ

(D.26)

Then by (D.23) and the definition of Un , we obtain
Pn pp1 θn P CIn q ě Pn pUn pθn , ĉn,ρ q ‰ Hq .

(D.27)

By passing to a further subsequence, we may assume that
DPn pθn q Ñ D,

(D.28)

u

for some J ˆ d matrix D such that }D} ď M and ΩPn Ñ Ω for some correlation matrix Ω. By Lemma 2 in Andrews
d
λρ
q Ñ Z for a normal random
and Guggenberger (2009) and Assumption 4.5 (i), uniformly in λ P B d , Gn pθn ` ?
n
vector with the correlation matrix Ω. By Lemma E.1,
lim inf Pn pUn pθn , ĉn,ρ q ‰ Hq ě 1 ´ α.
nÑ8

(D.29)

The conclusion of the theorem then follows from (D.2), (D.3), (D.27), and (D.29).
Proof of Theorem 4.2
The argument of proof is the same as for Theorem 4.1, with the following modification. Take pPn , θn q as defined
following equation (D.4). Then f pθn q is covered when
#
+
#
+
inf f pϑq
sup
f
pϑq
?
?
ď f pθn q ď
nm̄n,j pϑq
nm̄n,j pϑq
s.t. ϑ P Θ,
ď ĉfn pϑq, @j
s.t. ϑ P Θ,
ď ĉfn pϑq, @j
σ̂n,j pϑq
σ̂n,j pϑq
$
,
.
& inf λ ∇f pθ̃n qλ
?
λρ
?
? q
ô
ď0
n
m̄
pθ
`
n,j
n
λρ
n
%s.t.λ P ρn pΘ ´ θn q,
ď ĉfn pθn ` ?
q, @j λρ
n
σ̂n,j pθn ` ?
q
n
$
,
& supλ ∇f pθ̃n qλ
.
?
λρ
?
?
ď
,
n
m̄
pθ
`
q
n,j
n
λρ
n
%s.t.λ P ρn pΘ ´ θn q,
ď ĉfn pθn ` ?
q, @j λρ
n
?
σ̂n,j pθn `

n

q

where we took a mean value expansion yielding
λρ
ρ
f pθn ` ? q “ f pθn q ` ? ∇f pθ̃n qλ,
n
n
for θ̃n a mean value that lies componentwise between θn and θn `

λρ
?
,
n

(D.30)

and we used that the sign of the last term

in (D.30) is the same as the sign of ∇f pθ̃n qλ. With the objective function in (D.30) so redefined, all expression in
the proof of Theorem 4.1 up to (D.24) continue to be valid. We can then redefine the set Un pθn , cq in (D.25) as

[19]

(
d
Un pθn , cq ” λ P Bn,ρ
: }∇f pθ̃n q}´1 ∇f pθ̃n qλ “ 0 X un,j,θn pλq ď c, @j “ 1, . . . , J .
Replace p1 with }∇f pθ̃n q}´1 ∇f pθ̃n q in all expressions involving the set Un pθn , ĉfn,ρ pθn qq, and replace p1 with }∇f pθn q1 }´1 ∇f pθn1 q
in all expressions for the sets VnI pθn1 , ĉfn pθn1 qq, and in all the almost sure representation counterparts of these sets.
Observe that we can select a convergent subsequence from t}∇f pθn q1 }´1 ∇f pθn1 qu that converges to some p in the
unit sphere, so that the form of Wpcπ˚ q in (E.17) is unchanged. This yields the result, noting that by the assumption
?
}∇f pθ̃n q ´ ∇f pθn1 q} “ OP pρ{ nq

D.2.2

A High Level Condition Replacing Assumption 4.3 and the ρ-Box Constraints

Next, we consider an assumption which is composed of two parts. The first part aims at informally mimicking
Assumption A.2 in Bugni, Canay, and Shi (2017) and replaces Assumption 4.3. The second part replaces the use
of the ρ-box constraints. Below, for a given set A Ă Rd , let }A}H “ supaPA }a} denote its Hausdorff norm.
Assumption D.1: Consider any sequence tPn , θn u P tpP, θq : P P P, θ P ΘI pP qu such that
?
κ´1
nγ1,Pn ,j pθn q Ñ π1j P Rr´8s , j “ 1, . . . , J,
n
u

ΩPn Ñ Ω,
DPn pθn q Ñ D.
˚
˚
Let π1j
“ 0 if π1j “ 0 and π1j
“ ´8 if π1j ă 0. Let Z be a Gaussian process with covariance kernel Ω. Let
˚
wj pλq ” Zj ` ρDj λ ` π1,j
.

(D.31)

(I) Let
(
Wpcq ” λ P Bdρ : p1 λ “ 0 X wj pλq ď c, @j “ 1, . . . , J ,
cπ˚ ” inftc P R` : PrpWpcq ‰ Hq ě 1 ´ αu.

(D.32)
(D.33)

Then:
(a) If cπ˚ ą 0, Pr pWpcq ‰ Hq is continuous and strictly increasing at c “ cπ˚ .
(b) If cπ˚ “ 0, lim inf nÑ8 Pn pUn pθn , 0q ‰ Hq ě 1 ´ α, where Un pθn , cq, c ě 0 is as in (D.25).
(II) Let
(
W̄pcq ” λ P Rd : p1 λ “ 0 X wj pλq ď c, @j “ 1, . . . , J ,
which differs from (D.32) by not constraining λ to Bdρ , and let c̄ ” Φ´1 p1 ´ α{Jq denote the asymptotic
Bonferroni critical value. Then for every η ą 0 there exists Mη ă 8 s.t. Prp}W̄pc̄q}H ą Mη q ď η.

D.2.3

Proof of Theorem 4.1 with High Level Assumption D.1-(I) Replacing Assumption
4.3, and Dropping the ρ-Box Constraints Under Assumption D.1-(II)

Lemma D.1: Suppose that Assumption 4.1, 4.2, 4.4 and 4.5 hold.

[20]

(I) Let also Assumption D.1-(I) hold. Let 0 ă α ă 1{2. Then,
lim inf inf

inf

nÑ8 P PP θPΘI pP q

P pp1 θ P CIn q ě 1 ´ α.

(II) Let also Assumption D.1-(II) and either Assumption 4.3 or D.1-(I) hold. Let ĉn “ inftc P R` : P ˚ ptΛbn pθ, `8, cqX
tp1 λ “ 0uu ‰ Hq ě 1 ´ αu, where Λbn is defined in equation (3.1) and CIn ” r´sp´p, Cn pĉn qq, spp, Cn pĉn qqs
with spq, Cn pĉn qq, q P tp, ´pu defined in equation (2.5). Then
lim inf inf

inf

nÑ8 P PP θPΘI pP q

P pp1 θ P CIn q ě 1 ´ α.

Proof. We establish each part of the Lemma separately.
Part (I). This part of the lemma replaces Assumptions 4.3 with Assumption D.1-(I). Hence we establish the
result by showing that all claims that were made under Assumption 4.3 remain valid under Assumption D.1-(I).
We proceed in steps.
Step 1. Revisiting the proof of Lemma E.6, equation (E.133).
Let J ˚ be as defined in (E.29). If J ˚ “ H we immediately have that Lemma E.6 continues to hold. Hence we
assume that J ˚ ‰ H. To keep the notation simple, below we argue as if all j “ 1, . . . , J belong to J ˚ .
Consider the case that cπ˚ ą 0. For some cπ˚ ą δ ą 0, let
(
Wpc ´ δq ” λ P Bdρ : p1 λ “ 0 X wj pλq ď c ´ δ, @j “ 1, . . . , J ,
(D.34)
where we emphasize that the set Wpc ´ δq is obtained by a δ-contraction of all constraints, including those indexed
by j “ J1 ` 1, . . . , J. By Assumption D.1-(I), for any η ą 0 there exists a δ such that
η ě |Pr pWpcπ˚ q ‰ Hq ´ Pr pWpcπ˚ ´ δq ‰ Hq| “ Pr ptWpcπ˚ q ‰ Hu X tWpcπ˚ ´ δq “ Huq ,
η ě |Pr pWpcπ˚ ` δq ‰ Hq ´ Pr pWpcπ˚ q ‰ Hq| “ Pr ptWpcπ˚ ` δq ‰ Hu X tWpcπ˚ q “ Huq .
The result follows.
Step 2. Revisiting the proof of Lemma E.2.
Case 1 of Lemma E.2 is unaltered. Case 2 of Lemma E.2 follows from the same argument as used in Case 1 of
Lemma E.2, because under Assumption D.1-(I) as shown in step 1 of this proof all inequalities are tightened. In
Case 3 of Lemma E.2 the result in (D.29) holds automatically by Assumption D.1-(I)-(ii). (As a remark, Lemmas
E.7-E.8 are no longer needed to establish Lemma E.2.)
Step 3. Revisiting the proof of Lemma E.3. Under Assumption D.1 we do not need to merge paired inequalities.
Hence, part (iii) of Lemma E.3 holds automatically because ϕ˚j pξq ď ϕj pξq for any j and ξ. We are left to establish
parts (i) and (ii) of Lemma E.3. These follow immediately, because Lemma E.6 remains valid as shown in step 1
and by Assumption D.1-(I), PrpWpcq ‰ Hq is strictly increasing at c “ cπ˚ if cπ˚ ą 0. (As a remark, Lemma E.9
is no longer needed to establish Lemma E.3.)
In summary, the desired result follows by applying Lemma E.1 in the proof of Theorem 4.1 as Lemmas E.2, E.3
and E.6 remain valid, Lemmas E.4, E.5, E.10 and the Lemmas in Appendix E.3 are unaffected, and Lemmas E.7,
E.8, E.9 are no longer needed.
Part (II). This is established by adapting the proof of Theorem 4.1 as follows:
In the main proof, we pass to an a.s. representation early on, so that W realizes jointly with other random
variables (we denote almost sure representations adding a superscript “˚ ” on the original variable). At the same

[21]

time, we entirely drop ρ. This means that algebraic expressions, e.g. in the main proof, simplify as if ρ “ 1, but
d
also removes any constraints along the lines of λ P Bn,ρ
in equation (D.9). Indeed, (D.9) is replaced by:
$
,
’
/
inf λ p1 λ
&
.
¨¨¨ ð
ď0
s.t.λ P W̄˚ pc̄q,
’
/
?
?
?
?
% ˚
tGn,j pθn ` λ{ nq ` DPn ,j pθ̄n qλ ` nγ1,Pn ,j pθn qup1 ` ηn,j pθn ` λ{ nqq ď ĉn pθn ` λ{ nq, @j
$
’
supλ p1 λ
&
ď
s.t.λ P W̄˚ pc̄q,
’
?
?
?
?
% ˚
tGn,j pθn ` λ{ nq ` DPn ,j pθ̄n qλ ` nγ1,Pn ,j pθn qup1 ` ηn,j pθn ` λ{ nqq ď ĉn pθn ` λ{ nq, @j

it

,
/
.
,
/
-

yielding a new definition of the set Un˚ as
(
Un˚ pθn , cq ” λ P W̄˚ pc̄q : p1 λ “ 0 X u˚n,j,θn pλq ď c, @j “ 1, . . . , J .
?
Subsequent uses of ρ in the main proof use that }λ} ď dρ “ OP p1q. For example, consider the argument following
equation (E.30) or the argument just preceding equation (D.29), and so on. All these continue to go through because
W̄˚ pc̄q “ Op1q by assumption.
Similar uses occur in Lemma E.1. The next major adaptation is that in (E.27) and (E.28): we again drop ρ
but nominally introduce the constraint that λ P W̄˚ pc̄q. However, for c ď c̄, this condition cannot constrain W˚ pcq,
and so we can as well drop it: The modified W˚ pcq equals W̄˚ pcq.
Next we argue that Lemma E.7 continues to hold, now claimed for W̄˚ . To verify that this is the case, replace
B d with W̄pc̄q throughout in Lemma E.7. This requires straightforward adaptation of algebra as W̄pc̄q is only
stochastically and not deterministically bounded.
Finally, in Lemma E.3 we remove the ρ-constraint from Vnb and VnI without replacement, and note that the
?
lemma is now claimed for θn1 P θ ` }W̄pc̄q}H { nB d . Recall that in the lemma the a.s. representation of a set A is
Ă̄ Now we compare
denoted by Ã, and with some abuse of notation let the a.s. representation of W̄ be denoted W.
Ă̄ To ensure that λ is uniformly stochastically bounded in expressions like (E.95), we verify that
Ṽnb and ṼnI with W.
the modified Ṽnb and ṼnI inherit the property in Assumption D.1-(II). To see this, fix any unit vector t K p and
Ă̄
notice that any t “ λ{}λ} for λ P Wpcq
or for λ P Ṽnb pθn1 , cq or for λ P ṼnI pθn1 , cq, 0 ă c ď c̄, satisfies this condition.
t1 λ “ Op1q for any c ď c̄. Since the value
By Assumption D.1-(II) and the Cauchy-Schwarz inequality, maxλPWpcq
Ă̄
of this program is necessarily attained by a basic solution whose associated gradients span t, it must be the case
that such solution is itself Op1q. Formally, let C be the index set characterizing the solution, ZC
i be the vector
j
C 1
of realizations Zi corresponding to j P C, and K pθn q the matrix that stacks the corresponding gradients; then
P
1
pK C pθn1 qq´1 pc̄1 ´ ZC
i q “ Op1q. By Lemma E.7 and the fact that D̂n pθn q Ñ D by Assumption 4.4, we then also
have that pK̂ C pθn1 qq´1 pc̄1 ´ Gbn,j q “ OP p1q, and so for c ď c̄, V b is bounded in this same direction. It follows that,
by similar reasoning to the preceding paragraph, the comparison between VnI pθn1 , cq and W̄pcq in Lemma E.3 goes
through.

[22]

D.3
D.3.1

Proof of Theorems 4.3 and 4.4
Assumptions in Pakes, Porter, Ho, and Ishii (2011), Chernozhukov, Hong, and
Tamer (2007), and Bugni, Canay, and Shi (2017) That Allow for Simplifications
of the Method

We analyze calibrated projection under assumptions that are more stringent than for Theorem 4.1. The reward
is considerable computational simplification and, in some cases, removal of a tuning parameter. The additional
assumptions have been used in the related literature. Their logical relation to each other and to explicit constraint
qualifications is further analyzed in Kaido, Molinari, and Stoye (2017). For our purposes in this paper, we just state
without proof that, given Assumptions 4.1, 4.2, 4.3, 4.4, and 4.5, all assumptions below, including the minorant
assumptions attributed to other papers, are implied by assumptions in Pakes, Porter, Ho, and Ishii (2011); hence,
all results reported below apply under the Pakes, Porter, Ho, and Ishii (2011) assumptions.47
For θ P BΘI pP q, denote by J pP, θq the set of inequalities j s.t. EP pmj pXi , θqq “ 0. Denote by N pP, θq the
1
t ď 0, j P J pP, θqu the corresponding dual cone. (These
positive span of pDP,j qjPJ pP,θq and by T pP, θq “ tt : DP,j
are the normal and tangent cones of ΘI pP q at θ.) For a given p P Rd : }p} “ 1, let spp, ΘI pP qq “ maxθPΘI pP q p1 θ
and Hpp, ΘI pP qq ” arg maxθPΘI pP q p1 θ.
Assumption D.2 (A weakening of Assumption 4(a) in Pakes, Porter, Ho, and Ishii (2011)): There is a class
of DGPs Q Ă P such that any P P Q satisfies the following conditions:
1. There exists a (universal) εD ą 0 s.t.
min

min

max

θPHpp,ΘI pP qq }t}“1

jPt1,...,Ju:
EP pmj pXi ,θq{σj pθqqą´εD

t1 DP,j pθq ă ´εD .

2. There exists a (universal) εD ą 0 s.t.
max

min

max

θPHpp,ΘI pP qq }t}“1

jPt1,...,Ju:
EP pmj pXi ,θq{σj pθqqą´εD

t1 DP,j pθq ă ´εD .

There are two layers to these assumptions. First, they say that from some support point (part (1)) or all support
points (part (2)), there are directions that point uniformly inside ΘI pP q in the sense of all moment inequalities
decreasing in value. The obvious counterexample would be an extremely pointy corner (a “spike”).
In addition, the assumptions apply to “tightened” tangent cones that use all inequalities which are almost
binding, where “almost” is operationalized with the small but positive constant εD . Together with smoothness of
moment conditions, this implies that, by moving a small (but boundedly nonzero) distance in the direction of steepest
descent from the support point, one can find a point θ at which maxj EP pmj pXi , θq{σj pθqq is boundedly negative.
This implies that the sample analog of ΘI pP q is nonempty with probability approaching 1 (the proof in Appendix
D.3.2 includes a formal version of this argument). In particular, it implies that a vestige of the “degeneracy”
assumption in Chernozhukov, Hong, and Tamer (2007) is imposed. Some invocations of the assumption strictly
speaking only use one of the two features (again, see Kaido, Molinari, and Stoye (2017) for details), but we do not
disentangle them here. Note, however, that the second implication renders the assumption implausible whenever
the sample analog of ΘI pP q is empty, an empirically frequent occurrence.
47

Our own assumptions meaningfully exceed those of Pakes, Porter, Ho, and Ishii (2011) only through Assumption 4.3.
The absence of such an assumption in Pakes, Porter, Ho, and Ishii (2011) is actually an oversight, and ours or a similar
assumption must be added for their Theorem 2 to hold.

[23]

Next, consider:
Assumption D.3 (Linear Minorant – Chernozhukov, Hong, and Tamer (2007) display (4.5)): There exist
universal constants C, δ ą 0 and a class of DGPs Q Ă P such that for each P P Q,
max EP pmj pXi , θq{σj pθqq ě C min tδ, d pθ, ΘI pP qqu .

j“1,...,J

Assumption D.4 (Linear Minorant Along Support Plane – Bugni, Canay, and Shi (2017) Assumption A3(a)):
There exist universal constants C, δ ą 0 and a class of DGPs Q Ă P such that for each P P Q and for each
q P tp, ´pu,
max EP pmj pXi , θq{σj pθqq ě C min tδ, d pθ, Hpq, ΘI pP qqqu
j“1,...,J

1

for all θ with q θ “ spq, ΘI pP qq.
These assumptions are lifted from the cited papers. In the original papers, they are polynomial minorant
conditions: The minima are raised to some power χ. However, for our setting and criterion function, the special
case χ “ 1 applies. Note also that Assumption D.4 is closely analogous to Assumption D.3 but imposes the
minorant condition on the “null restricted model” in which the parameter space is restricted to the true supporting
hyperplane of ΘI pP q. It is easy to see that the assumptions are logically independent.
A further strengthening of assumptions is:
Assumption D.5 (A Weakening of Assumption 3 in Pakes, Porter, Ho, and Ishii (2011)): There exists a
universal constant δ ą 0 and a class of DGPs Q Ă P such that for any P P Q and for each q P tp, ´pu and any
θ P Hpq, ΘI pP qq, T pθq Ď tt : q 1 t{ }t} ď ´δu.
Note the implication that T pP, θq is uniformly pointy. The assumption is weaker than in Pakes, Porter, Ho,
and Ishii (2011) because they also assume ΘI pP q Ď T pθq and separately (although it is also an implication) that
Hpp, ΘI pP qq is a singleton.
Our final assumption gives a further strengthening by requiring the support set in direction of projection to be
a singleton:
Assumption D.6 (Assumption 1 in Pakes, Porter, Ho, and Ishii (2011)): There is a class of DGPs Q Ă P such
that for any P P Q and q P tp, ´pu, Hpq, ΘI pP qq is a singleton. (Its sole element will be denoted θq˚ below.)

D.3.2

Proof of Theorem 4.3: Simplifications for Calibrated Projection

Part I
Let θp˚ attain the outer minimum in Assumption D.2-1, let t˚ attain the inner minimum given θp˚ , and consider
any η ď εD {2M , where εD is from Assumption D.2-1 and M is from Assumption 4.4(ii). Then a Mean Value
Theorem yields
EP pmj pXi , θp˚ ` ηt˚ qq
σP,j pθp˚ ` ηt˚ q

ùñ max
j

EP pmj pXi , θp˚ ` ηt˚ qq
σP,j pθp˚ ` ηt˚ q

[24]

“

EP pmj pXi , θp˚ qq
` ηDPj pθ̄qt˚
σP,j pθp˚ q

ď

0 ` ηpηM ´ εD q

ď

´ηεD {2.

(D.35)

This will be used later but also implies
EP pmj pXi , θp˚ ` t˚ εD {2M qq
ď ´ε2D {4M ă 0
j
σP,j pθp˚ ` t˚ εD {2M q
ˆ
˙
m̄j pXi , θp˚ ` t˚ εD {2M q
P max
ă
0
Ñ1
j
σ̂j pθp˚ ` t˚ εD {2M qq
max

ùñ

(D.36)

P pθp˚ ` t˚ εD {2M P CIn q Ñ 1
“
‰
uniformly in Q. Hence, noncoverage risk for any γ P ´sp´p, ΘI pP qq, p1 pθp˚ ` t˚ εD {2M q is entirely driven by the
“ 1 ˚
‰
possibility that CIn is too high, and conversely for γ P p pθp ` t˚ εD {2M, spp, ΘI pP qqq . As these noncoverage risks
are monotonic in γ, the simplification is justified.
ùñ

Part II
Note first that, as an immediate implication of D.36, the event that minθPΘ maxj |m̄n,j pθq{σ̂n,j pθq|` “ 0, hence
this value is attained on Θ̂I , occurs w.p.a. 1 uniformly in Q.
?
Next, we show that npspp, Θ̂I q ´ spp, ΘI pP qqq “ OQ p1q. Define
"
*
Cp´εq “ θ P Θ : max EP pmj pXi , θq{σP,j pθqq ď ´ε .
j“1,...,J

Note that in this notation, ΘI pP q “ Cp0q. By (D.36) and because Cp´εq is closed by assumptions on mj , we have
that Hpp, Cp´εqq is nonempty for ε P r0, ε2D {4M s. Next, consider any η ď εD {2M , then p1 pθp˚ ` ηt˚ q ě p1 θp˚ ´ η,
which together with (D.35) implies
spp, Cp´ηεD {2qq ´ spp, ΘI q ě ´η.
Set ε “ ηεD {2, then equivalently we find that for ε ď ε2D {4M , spp, Cp´εq ´ spp, ΘI qq ě ´2ε{εD . Next, we have that
uniformly over θ P YεPr0,ε2D {4M s Hpp, Cp´εqq,
!
ˇ
ˇ )
?
?
n max |m̄n,j pθq{σ̂n,j pθq|` “ max p1 ´ ηn,j pθqq ˇGn,j pθq ` nEP pmj pXi , θq{σP,j pθqqˇ`
j
j
ÿ
ˇ
ˇ
?
ď
p1 ´ ηn,j pθqq ˇGn,j pθq ` nEP pmj pXi , θq{σP,j pθqqˇ`
j

ď

ˇ
? ˇ
Jp1 ` oQ p1qq ˇOQ p1q ´ nεˇ` ,

?
so in analogy to CHT (Theorem 4.2, step 1 of proof) we find n|spp, Θ̂I q ´ spp, ΘI q|´ “ OQ p1q. On the other hand,
from Assumption D.3 we have that uniformly over θ P Θ,
!
ˇ
ˇ )
?
?
n max |m̄n,j pθq{σ̂n,j pθq|` “ max p1 ´ ηn,j pθqq ˇGn,j pθq ` nEP pmj pXi , θq{σP,j pθqqˇ`
j

j

ě

J
ˇ
ˇ
?
1 ÿ
p1 ´ ηn,j pθqq ˇGn,j pθq ` nEP pmj pXi , θq{σP,j pθqqˇ`
J j“1

ě

J
ˇ
ˇ
?
1 ÿ
p1 ` oQ p1qq ˇOQ p1q ` nC min tδ, dpθ, ΘI pP qquˇ` ,
J j“1

?
hence n|spp, Θ̂I q ´ spp, ΘI pP qq|` “ OQ p1q.
We next argue that dpθ̂p , Hpp, ΘI pP qqq “ OQ pn´1{2 q (the proof for dpθ̂´p , Hp´p, ΘI pP qqq is identical). To do
so, let k̂ ” spp, ΘI pP qq ´ spp, Θ̂I q and define θ̃ “ θ̂p ` k̂p, noting that p1 θ̃ “ spp, ΘI pP qq by construction and so

[25]

Assumption D.4 applies to θ̃. Let θ̄ P Hpp, ΘI pP qq be such that dpθ̃, Hpp, ΘI pP qqq “ }θ̄ ´ θ̃}, then
›
› ›
› ›
´
¯
´
¯
´
¯ ›
›
› ›
›
› ›
d θ̂p , Hpp, ΘI pP qq ď d θ̂p , θ̃ ` d θ̃, Hpp, ΘI pP qq ď ›θ̂p ´ θ̃› ` ›θ̃ ´ θ̄› “ ›θ̃ ´ θ̄› ` k̂.
We already have

?

?
nk̂ “ OQ p1q, so it suffices to show n|θ̃ ´ θ̄| “ OQ p1q. Using Assumption D.4, we have
#
+
!
)
EP pmj pXi , θ̃qq
C min δ, }θ̃ ´ θ̄} ď max
j“1,...,J
σP,j pθ̃q
+
#
EP pmj pXi , θ̂p qq
` k̂DPj pθ̌j qp
“ max
j“1,...,J
σP,j pθ̂p q
#
+
!
)
EP pmj pXi , θ̂p qq
` max k̂DPj pθ̌j qp .
ď max
j“1,...,J
j“1,...,J
σP,j pθ̂p q

Here, the equality step uses that θ̃ “ θ̂p ` k̂p and introduces θ̌j , which lies componentwise between θ̃ and θ̂p . In
the last line, the first term equals 0 w.p.a. 1 because θ̂p P Θ̂I , and the second term is bounded by k̂ M̄ , hence the
result. To justify Simplification 2, combine the above algebra with the following observations:
(i) For a sequence Pn P Q, coverage of p1 θ for some θ P Hpp, ΘI pPn qq implies coverage of spp, ΘI pPn qq. In
the proof of Theorem 4.1, starting with display D.7, it therefore suffices to show the claim for some, possibly
data dependent, sequence θn P Hpp, ΘI pPn qq, and then again (in case of two-sided testing) for a sequence θn P
Hp´p, ΘI pPn qq.
(ii) All proofs go through if coverage is evaluated at θn but DP,j and Gn,j are estimated at some θ̂n,p “
θn ` OQ pn´1{2 q. To give one example, Assumption 4.4 implies that }D̂n,j pθ̂n,p q ´ DPn ,j pθn q} “ oQ p1q.
Part III
This is established by showing that Assumption D.1-(II) is implied. Thus, let W be as in (D.32). Because the
marginals of Z are standard normal, for any η ą 0 we have the Bonferroni bounds
Pr pWpc̄q Ď Lη q ě 1 ´ η,
where
Lη

"
*
(
d
1
´1
“
λ P R : p λ “ 0 X max Φ pη{Jq ` DPj λ ď c̄
j
"
*
d
1
´1
“
λ P R : p λ “ 0 X max DPj λ ď c̄ ` Φ p1 ´ η{Jq .
j

It remains to bound }Lη }H “ max t}λ} : λ P Lη u. To do so, we show below that
a
(
1 ` δ̄ 2 ´ 1
1
εD ,
p λ “ 0 ñ max DPj λ{ }λ} ě a
j
1 ` δ̄ 2 ` 1
loooooomoooooon

(D.37)

“:a

where δ̄ is from Assumption D.5 and εD is from Assumption D.2. Solving (D.37) for }λ} and inspecting the definition
of Lη yields
c̄ ` Φ´1 p1 ´ η{Jq
max t}λ} : λ P Lη u ď
aεD
and therefore an Op1q upper bound on }Wpc̄q}. It remains to show (D.37). Suppose by contradiction that
maxj tDj λ{ }λ}u ă aεD . Let the unit vector t˚ achieve the minimum from Assumption D.2-2, then maxj tDj pλ{ }λ} ` dt˚ qu ă

[26]

0 and therefore t ” λ{ }λ} ` dt˚ P T . We compute
´
¯
λ
1
˚
λ
`
at
1
}λ}
λt
›“
›
“
›
›
}λ} }t}
}λ} › λ ` at˚ ›
}λ}

1 ˚

t
a
1 ` a λ}λ}
1´a
›
›ą
“
1{
1 ` δ̄ 2 ,
› λ
›
1
`
a
˚
`
at
› }λ}
›

where the inequality is strict because λ ‰ t˚ . We conclude that maxtPT
a
λ1 λ̂
the projection of λ onto T , then }λ}}
ą 1{ 1 ` δ̄ 2 .48
λ̂}

λ1 t
}λ}}t}

a
ą 1{ 1 ` δ̄ 2 . In particular, if λ̂ is

However, we also have p1 λ̂{}λ̂} ď ´δ̄ by Assumption D.5. It follows that p1 pλ ´ λ̂q{}λ̂} ě δ̄, hence }λ ´ λ̂}2 ě
δ̄ 2 }λ̂}2 by Cauchy-Schwarz
(recall p is a unit vector). But also }λ ´ λ̂}2 ` }λ̂}2 “ }λ}2 . Simple algebra a
then yields
a
λ1 λ̂
2
}λ̂}{}λ} ď 1{ 1 ` δ̄ . But }λ̂}{}λ} is also the cosine of the angle formed by λ and λ̂. Thus, }λ}}λ̂} ď 1{ 1 ` δ̄ 2 , a
contradiction.49

D.3.3

Proof of Theorem 4.4: Asymptotic Equivalence with BCS-Profiling in Well-Behaved
Cases

?
Recall that under this Theorem’s assumptions, Hpp, ΘI q is a singleton tθp˚ u whose element is n-consistently
estimated by a sample analog θ̂p . We restrict attention to s ě p1 pθp˚ ` t˚ εD {2M q, where terms are as in the proof of
Theorem 4.3-(I). The proof for s ă p1 pθp˚ ` t˚ εD {2M q is analogous. Similarly to earlier proofs, consider a sequence
?
pPn , sn q that asymptotically minimizes the probability from the Theorem. If npsn ´ spp, ΘI pPn qqq Ñ 8, then
minp1 θ“sn Tn pθq Ñ 8 by arguments in the proof of Theorem 4.3-(II), and the conclusion obtains because both
?
indicator functions vanish. Similarly, if npsn ´ spp, ΘI pPn qqq Ñ ´8, then both indicator functions equal 1 with
?
probability approaching 1 (indeed, recall the sample support function is n-consistent). It remains to analyze the
?
case where npsn ´ spp, ΘI pPn qqq “ OQ p1q.
Recalling that no ρ-box is used, ĉn pθ̂p q is the p1 ´ αq quantile of
!
)
ˇ
? ˇ
b
´1
ˇ
ˇ
Tnb “ min
max
G
p
θ̂
q
`
κ
`
D̂
p
θ̂
qλ
n
m̄
p
θ̂
q{σ̂
p
θ̂
q
p
p
n,j
n,j
p
n,j
p
n,j
n
´
p1 λ“0 j
!
)
ˇ
ˇ
?
p1q
b
´1
ˇmj pXi , θ̂p q{σP,j pθ̂p qˇ ` D̂n,j pθ̂p qλ ` oQ p1q
“ min
max
G
p
θ̂
q
`
κ
nE
p
P
n,j
n
´
p1 λ“0 j
!
)
ˇ
ˇ
?
p2q
b
˚
´1
ˇmj pXi , θp˚ q{σP,j pθp˚ qˇ ` Dn,j pθp˚ qλ ` oQ p1q,
G
pθ
q
`
κ
“ min
max
nE
P
n,j
p
n
´
1
p λ“0

j

Here, (1) uses Lemma E.5-(iii). Step (2) uses that by Theorem 4.3-(III), the values of λ solving the optimization
?
problems are OQ p1q; by 4.3-(II), npθ̂p ´ θp˚ q “ OQ p1q; and smoothness conditions as well as consistent estimation
of gradients. These jointly imply that |D̂n,j pθ̂p qλ ´ Dn,j pθp˚ qλ| “ oQ p1q uniformly over the relevant range of λ.
To compare BCS-profiling, let θ̂p,sn be the selection from arg minp1 θ“sn |m̄n,j pθq{σ̂n,j pθq|` that solves the prob?
lem in the definition of TnDR psn q below. Arguments very similar to Theorem 4.3-(II) imply that npθ̂p,sn ´ θp˚ q “
48

Verbally, if λ is near tangential to all constraints, it is near tangential to T . The counterexample to this would be a
“spike,” which is excluded by Assumption D.2-2.
49
Verbally, if p1 λ “ 0, then λ cannot be near tangential to T because of the “pointy cone” assumption D.5, yielding a
contradiction.

[27]

OQ p1q. We can use this, again Lemma E.5-(iii), and smoothness conditions to write
(
?
TnDR psn q “ min max Gbn,j pθq ` κ´1
n |m̄n,j pθq{σ̂n,j pθq|´
s.t. θ P arg 1min max |m̄n,j pθq{σ̂n,j pθq|`
n
j
θ
p θ“sn j
!
ˇ )
? ˇ
b
´1
“ max Gn,j pθ̂p,s q ` κn nˇm̄n,j pθ̂p,sn q{σ̂n,j pθ̂p,s qˇ
n

j

“
“

n

´

!
ˇ
ˇ )
?
ˇ
ˇ
max Gbn,j pθ̂p,sn q ` κ´1
nE
m
pX
,
θ̂
q{σ
p
θ̂
q
P
j
i p,sn
P,j p,sn ´ ` oQ p1q
n
j
)
!
ˇ
ˇ
?
nEP ˇmj pXi , θp˚ q{σP,j pθp˚ qˇ´ ` oQ p1q.
max Gbn,j pθp˚ q ` κ´1
n
j

Next,
TnP R psn q

“
p1q

“

min
1

(
?
max Gbn,j pθq ` κ´1
nm̄n,j pθq{σ̂n,j pθq
n

min
1

˘
?
nEP pmj pXi , θq{σP,j pθqq u ` oQ p1q
max Gbn,j pθq ` κ´1
n

θPΘ:p θ“sn

θPΘ:p θ“sn

p2q

“

p3q

“

p4q

“

p5q

“

min

j

j

θPΘ:p1 θ“spp,ΘI q

˘
?
max Gbn,j pθq ` κ´1
nEP pmj pXi , θq{σP,j pθqq u ` oQ p1q
n
j

!
`
˘)
?
b
˚
´1{2
´1
˚
´1{2
˚
´1{2
min
max
G
pθ
`
λκ
n
q
`
κ
nE
m
pX
,
θ
`
λκ
n
q{σ
pθ
`
λκ
n
q
` oQ p1q
n
P
j
i p
n
P,j p
n
n,j p
n
1

p λ“0

j

`
˘
(
?
nEP mj pXi , θp˚ q{σP,j pθp˚ q ` DP,j pθp˚ qλ ` oQ p1q
min max Gbn,j pθp˚ q ` κ´1
n
j
)
!
ˇ
ˇ
?
˚
˚
˚ ˇ
b
˚
´1
ˇ
`
D
pθ
qλ
` oQ p1q
nE
m
pX
,
θ
q{σ
pθ
q
min
max
G
pθ
q
`
κ
P,j
P
j
i
P,j
p
p
p ´
n,j p
n
1

p1 λ“0

p λ“0

j

Here, (1) uses Lemma E.5-(iii). The first crucial step is (2), which uses that the distance between the hyperplanes
tp1 θ “ sn u and tp1 θ “ spΘI , pqu is of order OQ pn´1{2 q, together with smoothness conditions. Step (3) reparameterizes θ “ θp˚ ` λκn n´1{2 . Crucially, BCS prove that the λ solving the problem is OQ p1q. This means the
problem can be uniformly linearized, justifying step (4). Step (4) also observes cancellation of factors multiplying
DP,j pθp˚ qλ. Step (5) uses that θp˚ P ΘI . Finally, Assumption 4.3 ensures that the true distribution of Tn , as well as
the above approximations, are of order OQ p1q. We conclude that TnP R psn q asymptotically agrees with, and TnDR psn q
asymptotically dominates, Tnb .

Appendix E
E.1

Auxiliary Lemmas

Lemmas Used to Prove Theorems 4.1 and 4.2

Throughout this Appendix, we let pPn , θn q P tpP, θq : P P P, θ P ΘI pP qu be a subsequence as defined in the proof
of Theorem 4.1. That is, along pPn , θn q, one has
?
κ´1
nγ1,Pn ,j pθn q Ñ π1j P Rr´8s , j “ 1, . . . , J,
(E.1)
n
u

ΩPn Ñ Ω,

(E.2)

DPn pθn q Ñ D.

(E.3)

?
Fix c ě 0. For each λ P Rd and θ P pθn ` ρ{ nB d q X Θ, let
˚
wj pλq ” Zj ` ρDj λ ` π1,j
,

[28]

(E.4)

˚
where π1,j
is defined in (D.5) and we used Lemma E.5. Under Assumption 4.3-(II) if
˚
˚
,
π1,j
“ 0 “ π1,j`R
1

(E.5)

we replace the constraints
Zj ` ρDj λ ď c,

(E.6)

Zj`R1 ` ρDj`R1 λ ď c,

(E.7)

with
µj pθqtZj ` ρDj λu ´ µj`R1 pθqtZj`R1 ` ρDj`R1 λu ď c,

(E.8)

´µj pθqtZj ` ρDj λu ` µj`R1 pθqtZj`R1 ` ρDj`R1 λu ď c,

(E.9)

where
#
µj pθq “
#
µj`R1 pθq “

1

if γ1,Pn ,j pθq “ 0 “ γ1,Pn ,j`R1 pθq,

γ1,Pn ,j`R1 pθq
γ1,Pn ,j`R1 pθq`γ1,Pn ,j pθq

otherwise,

0

if γ1,Pn ,j pθq “ 0 “ γ1,Pn ,j`R1 pθq,
otherwise,

γ1,Pn ,j pθq
γ1,Pn ,j`R1 pθq`γ1,Pn ,j pθq

(E.10)

(E.11)

When Assumption 4.3-(II) is invoked with hard-threshold GMS, replace constraints j and j `R1 in the definition
?
of Λbn pθn1 , ρ, cq, θn1 P pθn ` ρ{ nB d q X Θ in equation (3.1) as described on p.14 of the paper; when it is invoked with
a GMS function ϕ that is smooth in its argument, replace them, respectively, with
µ̂n,j pθn1 qtGbn,j pθn1 q ` D̂n,j pθn1 qλu ´ µ̂n,j`R1 pθn1 qtGbn,j`R1 pθn1 q ` D̂n,j`R1 pθn1 qλu ` ϕj pξˆn,j pθn1 qq ď c,
(E.12)
1
b
1
1
1
b
1
1
1
ˆ
´µ̂n,j pθn qtGn,j pθn q ` D̂n,j pθn qλu ` µ̂n,j`R1 pθn qtGn,j`R1 pθn q ` D̂n,j`R1 pθn qλu ` ϕj`R1 pξn,j`R1 pθn qq ď c, (E.13)
where
$
¨
&
µ̂n,j`R1 pθn1 q “ min max ˝0,
%

1
m̄n,j pθn
q
1 q
σ̂n,j pθn
1 q
1 q
m̄n,j`R1 pθn
m̄n,j pθn
1
1 q ` σ̂
σ̂n,j`R1 pθn
n,j pθn q

,
.
‚, 1 ,
˛

µ̂n,j pθn1 q “ 1 ´ µ̂n,j`R1 pθn1 q.

(E.14)
(E.15)

d
Let Bdρ “ limnÑ8 Bn,ρ
. Let the intersection of tλ P Bdρ : p1 λ “ 0u with the level set associated with the so
defined function wj pλq be
(
Wpcq ” λ P Bdρ : p1 λ “ 0 X wj pλq ď c, @j “ 1, . . . , J .
(E.16)

Due to the substitutions in equations (E.6)-(E.9), the paired inequalities (i.e., inequalities for which (E.5) holds
under Assumption 4.3-(II)) are now genuine equalities relaxed by c. With some abuse of notation, we index them
among the j “ J1 ` 1, . . . , J. With that convention, for given δ P R, define
Wδ pcq ” λ P Bdρ : p1 λ “ 0 X wj pλq ď c ` δ, @j “ 1, . . . , J1 ,
(
X wj pλq ď c, @j “ J1 ` 1, . . . , J .

[29]

(E.17)

Define the pJ ` 2d ` 2q ˆ d matrix
fi
1 `J2
rρDP,j pθqsJj“1
ffi
—
—r´ρDP,j´J2 pθqsJj“J1 `J2 `1 ffi
ffi
—
ffi
—
Id
ffi .
—
KP pθ, ρq ” —
ffi
´Id
ffi
—
ffi
—
1
fl
–
p
»

(E.18)

´p1
Given a square matrix A, we let eigpAq denote its smallest eigenvalue. In all Lemmas below, we assume α ă 1{2.

Lemma E.1: Let Assumptions 4.1, 4.2, 4.3, 4.4, and 4.5 hold. Let tPn , θn u be a sequence such that Pn P P and
?
u
θn P ΘI pPn q for all n and κ´1
nγ1,Pn ,j pθn q Ñ π1j P Rr´8s , j “ 1, . . . , J, ΩPn Ñ Ω, and DPn pθn q Ñ D. Then,
n
lim inf Pn pUn pθn , ĉn,ρ q ‰ Hq ě 1 ´ α.

(E.19)

nÑ8

Proof. We consider a subsequence along which lim inf nÑ8 Pn pUn pθn , ĉn,ρ ‰ Hq is achieved as a limit. For notational
simplicity, we use tnu for this subsequence below.
Below, we construct a sequence of critical values such that
ĉn pθn1 q ě cIn pθn1 q ` oPn p1q,

(E.20)

?
P
and cIn pθn1 q Ñn cπ˚ for any θn1 P pθn ` ρ{ nB d q X Θ. The construction is as follows. When cπ˚ “ 0, let cIn pθn1 q “ 0 for
?
P
all θn1 P pθn ` ρ{ nB d q X Θ, and hence cIn pθn1 q Ñn cπ˚ . If cπ˚ ą 0, let cIn pθn q ” inftc P R` : Pn˚ pVnI pθn , cqq ě 1 ´ αu,
where VnI is defined as in Lemma E.3. By Lemma E.3 (iii), this critical value sequence satisfies (E.20) with
?
P
probability approaching 1. Further, by Lemma E.3 (ii), cIn pθn1 q Ñn cπ˚ for any θn1 P pθn ` ρ{ nB d q X Θ.
For each θ P Θ, let
λρ
cIn,ρ pθq ” inf cIn pθ ` ? q.
d
n
λPBn,ρ

(E.21)

Since the oPn p1q term in (E.20) does not affect the argument below, we redefine cIn,ρ pθn q as cIn,ρ pθn q ` oPn p1q. By
(E.20) and simple addition and subtraction,
´
¯
´
¯
Pn Un pθn , ĉn,ρ pθn qq ‰ H ě Pn Un pθn , cIn,ρ pθn qq ‰ H
” ´
¯
´
¯ı
“ PrpWpcπ˚ q ‰ Hq ` Pn Un pθn , cIn,ρ pθn qq ‰ H ´ Pr Wpcπ˚ q ‰ H .
(E.22)
As previously argued, Gn pθn `

d
λρ
?
qÑ
n

p

Z. Moreover, by Lemma E.10, supθPΘ }ηn pθq} Ñ 0 uniformly in P, and by

p

Lemma E.3, cIn,ρ pθn q Ñ cπ˚ . Therefore, uniformly in λ P B d , the sequence tpGn pθn `
satisfies

λρ
?
q, ηn pθn
n

`

λρ
?
q, cIn,ρ pθn qqu
n

λρ
λρ
d
pGn pθn ` ? q, ηn pθn ` ? q, cIn,ρ pθn qq Ñ pZ, 0, cπ˚ q.
n
n
In what follows, using Lemma 1.10.4 in van der Vaart and Wellner (2000) we take pG˚n pθn `

(E.23)
λρ
?
q, ηn˚ , c˚n q
n

to be the

λρ
λρ
almost sure representation of pGn pθn ` ?
q, ηn pθn ` ?
q, cIn,ρ pθn qq defined on some probability space pΩ, F, Pq such
n
n

that pG˚n pθn `

a.s.
λρ
?
q, ηn˚ , c˚n q Ñ
n

d

pZ˚ , 0, cπ˚ q, where Z˚ “ Z.

[30]

For each λ P Rd , we define analogs to the quantities in (D.24) and (E.4) as
λρ
˚
˚
u˚n,j,θn pλq ” tG˚n,j pθn ` ? q ` ρDPn ,j pθ̄n qλ ` π1,j
up1 ` ηn,j
q,
n
˚
w˚j pλq ” Z˚j ` ρDj λ ` π1,j
.

(E.24)
(E.25)

?
?
?
nγ1,P,j pθn q´κ´1
nγ1,P,j pθn1 q “ op1q uniformly over θn1 P pθn `ρ{ nB d qXΘ
where we used that by Lemma E.5, κ´1
n
n
˚
and therefore π1,j
is constant over this neighborhood, and we applied a similar replacement as described in equations
˚
˚
(E.6)-(E.9) for the case that π1,j
“ 0 “ π1,j`R
. Similarly, we define analogs to the sets in (D.25) and (E.16) as
1
(
d
Un˚ pθn , c˚n q ” λ P Bn,ρ
: p1 λ “ 0 X u˚n,j,θn pλq ď c˚n , @j “ 1, . . . , J ,
(E.26)
(
˚
d
1
˚
W pcπ˚ q ” λ P Bρ : p λ “ 0 X wj pλq ď cπ˚ , @j “ 1, . . . , J .
(E.27)
It then follows that equation (E.22) can be rewritten as
´
¯
” ´
¯
´
¯ı
Pn Un pθn , ĉn,ρ pθn qq ‰ H ě PpW˚ pcπ˚ q ‰ Hq ` P Un˚ pθn , c˚n q ‰ H ´ P W˚ pcπ˚ q ‰ H .

(E.28)

By the definition of cπ˚ , we have PpW˚ pcπ˚ q ‰ Hq ě 1 ´ α. Therefore, we are left to show that the second term
on the right hand side of (E.28) tends to 0 as n Ñ 8.
Define
˚
J ˚ ” tj “ 1, ¨ ¨ ¨ , J : π1,j
“ 0u.
(E.29)
˚
Case 1. Suppose first that J ˚ “ H, which implies J2 “ 0 and π1,j
“ ´8 for all j. Then we have
d
Un˚ pθn , c˚n q “ tλ P Bn,ρ
: p1 λ “ 0u,

W˚ pcπ˚ q “ tλ P Bdρ : p1 λ “ 0u,

(E.30)

with probability 1, and hence
´
¯
P tUn˚ pθn , c˚n q ‰ Hu X tW˚ pcπ˚ q ‰ Hu “ 1.

(E.31)

ˇ ´
¯
´
¯ˇ
ˇ
ˇ
ˇP Un˚ pθn , c˚n q ‰ Hu ´ P W˚ pcπ˚ q ‰ Hu ˇ “ 0,

(E.32)

This in turn implies that

where we used |PpAq ´ PpBq| ď PpA∆Bq ď 1 ´ PpA X Bq for any pair of events A and B. Hence, the term in the
square brackets in (E.28) is 0.
Case 2. Now consider the case that J ˚ ‰ H. We show that the term in the square brackets in (E.28) converges
to 0. To that end, note that for any events A, B,
ˇ
ˇ ˇ
ˇ
ˇ
ˇ ˇ
ˇ
(E.33)
ˇPpA ‰ Hq ´ PpB ‰ Hqˇ ď ˇPptA “ Hu X tB ‰ Huq ` PptA ‰ Hu X tB “ Huqˇ
Hence, we aim to establish that for A “ Un˚ pθn , c˚n q, B “ W˚ pcπ˚ q, the right hand side of equation (E.33) converges
to zero. But this is guaranteed by Lemma E.2. Therefore, the conclusion of the lemma follows.
Lemma E.2: Let Assumptions 4.1, 4.2, 4.3, 4.4, and 4.5 hold. Let pPn , θn q have the almost sure representations
given in Lemma E.1, and let J ˚ be defined as in (E.29). Assume that J ˚ ‰ H. Then for any η ą 0, there exists
N P N such that
´
¯
P tUn˚ pθn , c˚n q ‰ Hu X tW˚ pcπ˚ q “ Hu ď η{2,
(E.34)
´
¯
P tUn˚ pθn , c˚n q “ Hu X tW˚ pcπ˚ q ‰ Hu ď η{2,
(E.35)

[31]

for all n ě N , where the sets in the above expressions are defined in equations (E.26) and (E.27).
˚
Proof. We begin by observing that for j R J ˚ , π1,j
“ ´8, and therefore the corresponding inequalities
ˆ
˙
λρ
˚
˚
G˚n,j pθn ` ? q ` ρDPn ,j pθ̄n qλ ` π1,j
p1 ` ηn,j
q ď c˚n ,
n
˚
Z˚j ` ρDj λ ` π1,j
ď cπ˚

are satisfied with probability approaching one by similar arguments as in (D.20). Hence, we can redefine the sets
of interest as
(
d
Un˚ pθn , c˚n q ” λ P Bn,ρ
(E.36)
: p1 λ “ 0 X u˚n,j,θn pλq ď c˚n , @j P J ˚ ,
(
˚
d
1
˚
˚
(E.37)
W pcπ˚ q ” λ P Bρ : p λ “ 0 X wj pλq ď cπ˚ , @j P J .
We first show (E.34). For this, we start by defining the events
*
"
ˇ
ˇ
An ” sup max˚ ˇpu˚n,j,θn pλq ´ c˚n q ´ pw˚j pλq ´ cπ˚ qˇ ě δ .

(E.38)

λPB d jPJ

By Lemma E.4, using the assumption that J ˚ ‰ H, for any η ą 0 there exists N P N such that
PpAn q ă η{2, @n ě N.

(E.39)

Define the sets of λs, Un˚,`δ and W˚,`δ by relaxing the constraints shaping Un˚ and W˚ by δ:
d
Un˚,`δ pθn , cq ” tλ P Bn,ρ
: p1 λ “ 0 X u˚n,j,θn pλq ď c ` δ, j P J ˚ u,

W

˚,`δ

pcq ” tλ P

Bdρ

1

:pλ“0X

w˚j pλq

˚

ď c ` δ, j P J u.

(E.40)
(E.41)

Compared to the set in equation (E.17), here we replace u˚n,j,θn pλq for un,j,θn pλq and w˚j pλq for wj pλq, we retain
only constraints in J ˚ , and we relax all such constraints by δ ą 0 instead of relaxing only those in t1, . . . , J1 u.
Next, define the event Ln ” tUn˚ pθn , c˚n q Ă W˚,`δ pcπ˚ qu and note that Acn Ď Ln .
We may then bound the left hand side of (E.34) as
´
¯
´
¯
P tUn˚ pθn , c˚n q ‰ Hu X tW˚ pcπ˚ q “ Hu ď P tUn˚ pθn , c˚n q ‰ Hu X tW˚,`δ pcπ˚ q “ Hu
´
¯
` P tW˚,`δ pcπ˚ q ‰ Hu X tW˚ pcπ˚ q “ Hu ,
(E.42)
where we used P pA X Bq ď P pA X Cq ` P pB X C c q for any events A, B, and C. The first term on the right hand
side of (E.42) can further be bounded as
´
¯
´
¯
P tUn˚ pθn , c˚n q ‰ Hu X tW˚,`δ pcπ˚ q “ Hu ď P tUn˚ pθn , c˚n q Ę W˚,`δ pcπ˚ qu
“ PpLcn q ď PpAn q ă η{2, @n ě N ,

(E.43)

where the penultimate inequality follows from Acn Ď Ln as argued above, and the last inequality follows from (E.39).
For the second term on the left hand side of (E.42), by Lemma E.6, there exists N 1 P N such that
´
¯
P tW˚,`δ pcπ˚ q ‰ Hu X tW˚ pcπ˚ q “ Hu ď η{2, @n ě N 1 .
(E.44)
Hence, (E.34) follows from (E.42), (E.43), and (E.44).
To establish (E.35), we distinguish three cases.

[32]

Case 1. Suppose first that J2 “ 0 (recalling that under Assumption 4.3-(II) this means that there is no j “ 1, . . . , R1
˚
˚
such that π1,j
“ 0 “ π1,j`R
), and hence one has only moment inequalities. In this case, by (E.36) and (E.37), one
1
may write
(
d
(E.45)
Un˚ pθn , cq ” λ P Bn,ρ
: p1 λ “ 0 X u˚n,j,θn pλq ď c, j P J ˚ ,
(
(E.46)
W˚,´δ pcq ” λ P Bdρ : p1 λ “ 0 X w˚j pλq ď c ´ δ, j P J ˚ ,
where W˚,´δ , δ ą 0, is obtained by tightening the inequality constraints shaping W˚ . Define the event
R2n ” tW˚,´δ pcπ˚ q Ă Un˚ pθn , c˚n qu,

(E.47)

and note that Acn Ď R2n . The result in equation (E.35) then follows by Lemma E.6 using again similar steps to
(E.42)-(E.44).
Case 2. Next suppose that J2 ě d. In this case, we define W˚,´δ to be the set obtained by tightening by δ the
inequality constraints as well as each of the two opposing inequalities obtained from the equality constraints. That
is,
W˚,´δ pcπ˚ q ” tλ P Bdρ : p1 λ “ 0 X w˚j pλq ď c ´ δ, j P J ˚ u,

(E.48)

that is, the same set as in (E.133) with w˚j pλq replacing wj pλq and defining the set using only inequalities in J ˚ .
Note that, by Lemma E.8, there exists N P N such that for all n ě N cIn pθq is bounded from below by some c ą 0
with probability approaching one uniformly in P P P and θ P ΘI pP q. This ensures cπ˚ is bounded from below
by c ą 0. This in turn allows us to construct a non-empty tightened constraint set with probability approaching
1. Namely, for δ ă c, W˚,´δ pcπ˚ q is nonempty with probability approaching 1 by Lemma E.6, and hence its
superset W˚ pcπ˚ q is also non-empty with probability approaching 1. However, note that Acn Ď R2n , where R2n is
in (E.47) now defined using the tightened constraint set W˚,´δ pcπ˚ q being defined as in (E.48), and therefore the
same argument as in the previous case applies.
Case 3. Finally, suppose that 1 ď J2 ă d. Recall that, with probability 1 (under P),
cπ˚ “ lim c˚n ,
nÑ8

(E.49)

and note that by construction cπ˚ ě 0. Consider first the case that cπ˚ ą 0. Then, by taking δ ă cπ˚ , the argument
in Case 2 applies.
Next consider the case that cπ˚ “ 0. Observe that
´
¯
´
¯
P tUn˚ pθn , c˚n q “ Hu X tW˚ pcπ˚ q ‰ Hu ď P tUn˚ pθn , c˚n q “ Hu X tW˚,´δ p0q ‰ Hu
´
¯
` P tW˚,´δ p0q “ Hu X tW˚ p0q ‰ Hu , (E.50)
with W˚,´δ p0q defined as in (E.17) with c “ 0 and with w˚j pλq replacing wj pλq. By Lemma E.6, for any η ą 0 there
exists δ ą 0 and N P N such that
´
¯
P tW˚,´δ p0q “ Hu X tW˚ p0q ‰ Hu ă η{3 @n ě N.
(E.51)
Therefore, the second term on the right hand side of (E.50) can be made arbitrarily small.

[33]

We now consider the first term on the right hand side of (E.50). Let g be a J ` 2d ` 2 vector with
$
´Zj , j P J ˚ ,
’
’
’
& 0,
j P t1, ¨ ¨ ¨ , JuzJ ˚ ,
gj “
’
1,
j “ J ` 1, . . . , J ` 2d,
’
’
%
0,
j “ J ` 2d ` 1, J ` 2d ` 2,

(E.52)

˚
where we used that π1,j
“ 0 for j P J ˚ and where the last assignment is without loss of generality because of the
considerations leading to the sets in (E.36)-(E.37).
For a given set C Ă t1, . . . , J ` 2d ` 2u, let the vector g C collect the entries of g C corresponding to indices in
C. Let
fi
»
1 `J2
rρDj sJj“1
ffi
—
—r´ρDj´J2 sJj“J1 `J2 `1 ffi
ffi
—
ffi
—
Id
ffi .
—
(E.53)
K”—
ffi
´Id
ffi
—
ffi
—
fl
–
p1

´p1
Let the matrix K C collect the rows of K corresponding to indices in C.
Let Cr collect all size d subsets C of t1, ..., J ` 2d ` 2u ordered lexicographically by their smallest, then second
smallest, etc. elements. Let the random variable C equal the first element of Cr s.t. det K C ‰ 0 and λC “
pK C q´1 g C P W˚,´δ p0q if such an element exists; else, let C “ tJ ` 1, ..., J ` du and λC “ 1d , where 1d denotes a d
vector with each entry equal to 1. Recall that W˚,´δ p0q is a (possibly empty) measurable random polyhedron in
a compact subset of Rd , see, e.g., Molchanov (2005, Definition 1.1.1). Thus, if W˚,´δ p0q ‰ H, then W˚,´δ p0q has
extreme points, each of which is characterized as the intersection of d (not necessarily unique) linearly independent
constraints interpreted as equalities. Therefore, W˚,´δ p0q ‰ H implies that λC P W˚,´δ p0q and therefore also that
C Ă J ˚ Y tJ ` 1, . . . , J ` 2d ` 2u. Note that the associated random vector λC is a measurable selection of a random
closed set that equals W˚,´δ p0q if W˚,´δ p0q ‰ H and equals Bdρ otherwise, see, e.g., Molchanov (2005, Definition
1.2.2).
Lemma E.7 establishes that for any η ą 0, there exist εη ą 0 and N s.t. n ě N implies
ˇ
ˇ
`
˘
P W˚,´δ p0q ‰ H, ˇdet K C ˇ ď εη ď η,
(E.54)
which in turn, given our definition of C, yields that there is M ą 0 and N such that
´ˇ
¯
` ˘´1 ˇ
ˇ ď M ě 1 ´ η, @n ě N.
P ˇ det K C
Let gn be a J ` 2d ` 2 vector with
$
˚
’
c˚n {p1 ` ηn,j
q ´ G˚n,j pθ `
’
’
’
&
?
0,
gn,j pθ ` λ{ nq ”
’
1,
’
’
’
% 0,

[34]

λρ
?
q
n

(E.55)

if j P J ˚ ,
if j P t1, ¨ ¨ ¨ , JuzJ ˚ ,
if j “ J ` 1, . . . , J ` 2d,
if j “ J ` 2d ` 1, J ` 2d ` 2,

(E.56)

˚
using again that π1,j
“ 0 for j P J ˚ . For each P P P, let

fi
1 `J2
rρDP,j pθqsJj“1
ffi
—
—r´ρDP,j´J2 pθqsJj“J1 `J2 `1 ffi
ffi
—
ffi
—
Id
ffi .
KP pθ, ρq ” —
ffi
—
´Id
ffi
—
ffi
—
fl
–
p1
»

(E.57)

´p1
For each n and λ P B d , define the mapping φn : B d Ñ Rdr˘8s by
`
˘´1 C
λρ
φn pλq ” KPCn pθ̄pθn , λq, ρq
gn pθn ` ? q,
n

(E.58)

where the notation θ̄pθn , λq emphasizes that θ̄ depends on θn and λ because it lies component-wise between θn and
λρ
. We show that φn is a contraction mapping and hence has a fixed point.
θn ` ?
n
For any λ, λ1 P B d write
›`
˘´1 C
`
˘´1 C
λρ
λ1 ρ ››
›
}φn pλq ´ φn pλ1 q} “ › KPCn pθ̄pθn , λq, ρq
gn pθn ` ? q ´ KPCn pθ̄pθn , λ1 q, ρq
gn pθn ` ? q›
n
n
›`
›
1
˘´1 ›› ›› C
λ
ρ
λρ
› C
›
ď › KPn pθ̄pθn , λq, ρq
› ›gn pθn ` ? q ´ gnC pθn ` ? q›
n
n
2
›`
› ›
˘´1 ` C
˘
λ1 ρ ››
´1
› C
›
›
` › KPn pθ̄pθn , λq, ρq
´ KPn pθ̄pθn , λ1 q, ρq
(E.59)
› ›gnC pθn ` ? q›,
n
2
where } ¨ }2 denotes the spectral norm (induced by the Euclidean norm).
By Assumption 4.5 (ii), for any η ą 0, k ą 0, there is N P N such that
›
˙
ˆ›
› C
λρ
λ1 ρ ››
1
C
›
P ›gn pθn ` ? q ´ gn pθn ` ? q› ď k}λ ´ λ }
n
n
`
˘
λρ
λ1 ρ
˚,C
?
?
“ P }G˚,C
pθ
`
q
´
G
pθ
`
q} ď k}λ ´ λ1 } ě 1 ´ η, @n ě N.
n
n
n
n
n
n

(E.60)

Moreover, by arguing as in equation (D.20), for any η there exist 0 ă L ă 8 and N P N such that @n ě N
›
›
˙
ˆ
› C
λ1 ρ ››
›
(E.61)
P sup ›gn pθn ` ? q› ď L ě 1 ´ η.
n
λ1 PB d
?
´1
For any invertible matrix K, }K ´1 }2 “ pmint α : α is an eigenvalue of KK 1 uq . Hence, by the proof of Lemma
E.7 and the definition of C, for any η ą 0, there exist 0 ă L ă 8 and N P N such that
`›` ˘´1 ›
˘
› ď L ě 1 ´ η, @n ě N,
P › KC
(E.62)
2
By Horn and Johnson (1985, ch. 5.8), for any invertible matrices K, K̃ such that }K̃ ´1 pK ´ K̃q}2 ă 1,
}K ´1 ´ K̃ ´1 }2 ď

}K̃ ´1 pK ´ K̃q}2
}K̃ ´1 }2 .
1 ´ }K̃ ´1 pK ´ K̃q}2

(E.63)

By the assumption that DPn pθn q Ñ D and Assumption 4.4, for any η ą 0, there exists N P N such that
sup }KPCn pθ̄pθn , λq, ρq ´ K C }2 ď η, @n ě N.

λPB d

[35]

(E.64)

By (E.63), the definition of the spectral norm, and the triangle inequality, for any η ą 0, there exist 0 ă L1 , L2 ă 8
and N P N such that
›`
˘´1 ›
˘
`
› ď 2L1
P sup › KPCn pθ̄pθn , λq, ρq
2
λPB d

`›` ˘´1 ›
˘
› ` sup }KPC pθ̄pθn , λq, ρq´1 ´ pK C q´1 }2 ď 2L1
ě P › KC
n
2
λPB d
˜
¸
` ˘´1 2
›` C ˘´1 ›
} KC
}2
L1
C
C
›
›
ď L2 , sup }KPn pθ̄pθn , λq, ρq ´ K }2 ď
ď L1 ,
ěP
K
` ˘´1 C
2
L2
λPB d
pKPn pθ̄pθn , λq, ρq ´ K C q}2
1 ´ } KC
ě 1 ´ 2η, @n ě N,

(E.65)

Again by applying (E.63), for any k ą 0, there exists N P N such that
`›`
˘´1 ` C
˘´1 ›
˘
› ď k}λ ´ λ1 }
P › KPCn pθ̄pθn , λqq
´ KPn pθ̄pθn , λ1 qq
2
´
¯
›`
˘´1 ›2
› M ρ}θ̄pθn , λq ´ θ̄pθn , λ1 q} ď k}λ ´ λ1 } ě 1 ´ η, @n ě N,
ě P sup › KPCn pθ̄pθn , λqq
2

(E.66)

λPB d

?
where the first inequality follows from }KPCn pθ̄pθn , λqq ´ KPCn pθ̄pθn , λ1 qq}2 ď M ρ}θ̄pθn , λq ´ θ̄pθn , λ1 q} ď M ρ2 { n}λ ´
λ1 } by Assumption 4.4 (ii), and the last inequality follows from (E.65).
By (E.59)-(E.61) and (E.65)-(E.66), it then follows that there exists β P r0, 1q such that for any η ą 0, there
exists N P N such that
`
˘
P |φn pλq ´ φn pλ1 q| ď β}λ ´ λ1 }, @λ, λ1 P B d ě 1 ´ η, @n ě N.
(E.67)
This implies that with probability approaching 1, each φn p¨q is a contraction, and therefore by the Contraction
Mapping Theorem it has a fixed point (e.g., Pata (2014, Theorem 1.3)). This in turn implies that for any η ą 0
there exists a N P N such that
`
˘
P Dλfn : λfn “ φn pλfn q ě 1 ´ η, @n ě N.
(E.68)
Next, define the mapping
` ˘´1 C
ψn pλq ” K C
g .

(E.69)

This map is constant in λ and hence is uniformly continuous and a contraction with Lipschitz constant equal to
zero. It therefore has λCn as its fixed point. Moreover, by (E.58) and (E.69) arguing as in (E.59), it follows that for
any λ P B d ,
›`
˘´1 ›› ›› C
λρ ››
›
}ψn pλq ´ φn pλq} ď › KPCn pθ̄pθn , λq, ρq
› ›g ´ gnC pθn ` ? q›
n
2
›` ˘
˘´1 ›› › C ›
› C ´1 ` C
`› K
´ KPn pθ̄pθn , λq, ρq
(E.70)
› ›g ›.
2

By (E.52) and (E.56)
›
λρ
λρ ››
› C
˚
q ` G˚n,j pθn ` ? q|
›g ´ gnC pθn ` ? q› ď max˚ | ´ Z˚j ´ c˚n {p1 ` ηn,j
n
n
jPJ
λρ
˚
ď max˚ |Z˚j ´ G˚n,j pθn ` ? q| ` max˚ |c˚n {p1 ` ηn,j
q|.
n
jPJ
jPJ

(E.71)

˚
˚
We note that when Assumption 4.3-(II) is used, for each j “ 1, . . . , R1 such that π1,j
“ 0 “ π1,j`R
we have that
1
|µ̃j ´ µj | “ oP p1q because supθPΘ |ηj pθq| “ oP p1q, where µ̃j and µj were defined in (D.11)-(D.12) and (E.10)-(E.11)

[36]

respectively. Moreover, G˚n,j pθn `

λρ a.s.
?
q Ñ
n

Z˚ and (E.49) implies c˚n Ñ 0 so that we have

λρ a.s.
sup }g C ´ gnC pθn ` ? q} Ñ 0.
n
d
λPB

(E.72)

Further, by (E.63), DPn Ñ D and, Assumption 4.4-(ii), for any η ą 0, there exists N P N such that
›` ˘
`
˘´1 ››
´1
›
´ KPCn pθ̄pθn , λq, ρq
sup › K C
› ď η, @n ě N.

(E.73)

In sum, by (E.61), (E.65), and (E.71)-(E.73), for any η, ν ą 0, there exists N ě N such that
ˆ
˙
P sup }ψn pλq ´ φn pλq} ă ν ě 1 ´ η, @n ě N.

(E.74)

2

λPB d

λPB d

Hence, for a specific choice of ν “ κp1 ´ βq, where β is defined in equation (E.67), we have that supλPB d }ψn pλq ´
φn pλq} ă κp1 ´ βq implies
}λCn ´ λfn } “ }ψn pλCn q ´ φn pλfn q}
ď }ψn pλCn q ´ φn pλCn q} ` }φn pλCn q ´ φn pλfn q}
ď κp1 ´ βq ` β}λCn ´ λfn }

(E.75)

Rearranging terms, we obtain }λCn ´ λfn } ď κ. Note that by Assumptions 4.4 (i) and 4.5 (i), for any δ ą 0, there
exists κδ ą 0 and N P N such that
´
¯
P
sup |u˚n,j,θn pλq ´ u˚n,j,θn pλ1 q| ă δ ě 1 ´ η, @n ě N.
(E.76)
}λ´λ1 }ďκδ

For λCn P W˚,´δ p0q, one has
w˚j pλCn q ` δ ď 0, j P t1, ¨ ¨ ¨ , J1 u X J ˚ .

(E.77)

Hence, by (E.39), (E.49), and (E.76)-(E.77), }λCn ´ λfn } ď κδ{4 , for each j P t1, ¨ ¨ ¨ , J1 u X J ˚ we have
u˚n,j,θn pλfn q ´ c˚n pθn q ď u˚n,j,θn pλCn q ´ c˚n pθn q ` δ{4 ď w˚j pλCn q ` δ{2 ď 0.
For j P tJ1 ` 1, ¨ ¨ ¨ , 2J2 u X J ˚ , the inequalities hold by construction given the definition of C.
In sum, for any η ą 0 there exists δ ą 0 and N P N such that for all n ě N we have
´
¯
´
¯
P tUn˚ pθn , c˚n q “ Hu X tW˚,´δ p0q ‰ Hu ď P Eλfn P Un˚ pθn , c˚n q, DλCn P W˚,´δ p0q
ˆ"
*c ˙
ďP
sup }ψn pλq ´ φn pλq} ă κδ p1 ´ βq X An
ď η{3,

(E.78)

(E.79)

λPB d

where Ac denotes the complement of the set A, and the last inequality follows from (E.39) and (E.74).
Lemma E.3: Suppose Assumptions 4.1, 4.2, 4.3, 4.4, and 4.5 hold. Let tPn , θn u P tpP, θq : P P P, θ P ΘI pP qu
be a sequence satisfying (E.1)-(E.3). For each j, let
I
vn,j,θ
pλq ” Gbn,j pθn q ` ρD̂n,j pθn qλ ` ϕ˚j pξˆn,j pθn qq,
n
˚
wj pλq ” Zj ` ρDj λ ` π1,j
,

[37]

(E.80)
(E.81)

where

ϕ˚j pξq

$
’
’
&ϕj pξq

π1,j “ 0

´8

π1,j ă 0

0

j “ J1 ` 1, ¨ ¨ ¨ , J.

“
’
’
%

(E.82)

For each c ě 0, define
d
I
VnI pθn , cq ” tλ P Bn,ρ
: p1 λ “ 0 X vn,j,θ
pλq ď c, j “ 1, ¨ ¨ ¨ , Ju,
n
(
Wpcq ” λ P Bdρ : p1 λ “ 0 X wj pλq ď c, @j “ 1, . . . , J .

(E.83)
(E.84)

We then let cIn pθn q ” inftc P R` : Pn˚ pVnI pθn , cq ‰ Hq ě 1 ´ αu and cπ˚ ” inftc P R` : PrpWpcq ‰ Hq ě 1 ´ αu.
?
Then, (i) for any c ą 0 and tθn1 u Ă Θ such that θn1 P pθn ` ρ{ nB d q X Θ for all n,
Pn˚ pVnI pθn1 , cq ‰ Hq ´ PrpWpcq ‰ Hq Ñ 0,

(E.85)

with probability approaching 1;
P
(ii) If cπ˚ ą 0, cIn pθn1 q Ñn cπ˚ ;
?
(iii) For any tθn1 u Ă Θ such that θn1 P pθn ` ρ{ nB d q X Θ for all n,
ĉn pθn1 q ě cIn pθn1 q ` oPn p1q.

(E.86)

?
Proof. Throughout, let c ą 0 and let tθn1 u Ă Θ be a sequence such that θn1 P pθn ` ρ{ nB d q X Θ for all n. By
p
1
Lemma E.15, in l8 pΘq uniformly in P conditional on tXi u8
i“1 , and by Assumption 4.4 }D̂n pθn q ´ DPn pθn q} Ñ 0.
P
Further, by Lemma E.5, ξˆn,j pθn1 q Ñn π1,j . Therefore,
d
pGbn pθn1 q, D̂n pθn1 q, ξˆn pθn1 qq|tXi u8
i“1 Ñ pZ, D, π1 q.

(E.87)

for almost all sample paths tXi u8
i“1 . By Lemma E.17, conditional on the sample path, there exists an almost sure
1
b
˜
representation pG̃n pθn q, D̃n , ξn q of pGbn pθn1 q, D̂n pθn1 q, ξˆn pθn1 qq defined on another probability space pΩ̃, F̃, P̃q such
d
that pG̃bn pθn1 q, D̃n , ξ˜n q “ pGbn pθn1 q, D̂n pθn1 q, ξˆn pθn1 qq conditional on the sample path. In particular, conditional on the
sample, pD̂n pθn1 q, ξˆn pθn1 qq are non-stochastic. Therefore, we set pD̃n , ξ˜n q “ pD̂n pθn1 q, ξˆn pθn1 qq, P̃ ´ a.s. The almost
a.s.
d
sure representation satisfies pG̃bn pθn1 q, D̃n , ξ˜n,j q Ñ pZ̃, D, π1 q for almost all sample paths, where Z̃ “ Z. The almost
sure representation pG̃bn , D̃n , ξ˜n q is defined for each sample path x8 “ txi u8
i“1 , but we suppress its dependence on
x8 for notational simplicity (see Appendix E.3 for details). Using this representation, define
I
b
1
˚ ˜
ṽn,j,θ
1 pλq ” G̃n,j pθn q ` ρD̃n λ ` ϕj pξn,j q,
n

(E.88)

˚
w̃j pλq ” Z̃j ` ρDj λ ` π1,j
,

(E.89)

and

d

where Z̃ “ Z, and G̃bn pθn1 q Ñ Z̃, P̃ ´ a.s. conditional on tXi u8
i“1 . With this construction, one may write
|Pn˚ pVnI pθn1 , cq ‰ Hq ´ PrpWpcq ‰ Hq| “ |P̃pṼnI pθn1 , cq ‰ Hq ´ P̃pW̃pcq ‰ Hq|
ď |P̃pṼnI pθn1 , cq “ H X W̃pcq ‰ Hq ` P̃pṼnI pθn1 , cq ‰ H X W̃pcq “ Hq|, (E.90)

[38]

where the inequality is due to (E.33). First, we bound the first term on the right hand side of (E.90). Note that
P̃pṼnI pθn1 , cq “ H X W̃pcq ‰ Hq ď P̃pṼnI,`δ pθn1 , cq “ H X W̃pcq ‰ Hq ` P̃pṼnI,`δ pθn1 , cq ‰ H X ṼnI pθn1 , cq “ Hq,
(E.91)
where ṼnI,`δ is defined as
!
)
d
I
˚
ṼnI,`δ ” λ P Bn,ρ
: p1 λ “ 0 X ṽn,j,θ
.
1 pλq ď c ` δ, j P J
n

(E.92)

!
)
I
An ” ω̃ P Ω̃ : sup max˚ |ṽn,j,θ
1 pλq ´ w̃j pλq| ě δ .
n

(E.93)

1
˚
E ” ttxi u8
|ϕ˚j pξˆn,j pθn1 qq ´ π1,j
| ă ηu.
i“1 : }D̂n pθn q ´ D} ă η, max
˚

(E.94)

Let
λPB d jPJ

Let
jPJ

Note that, Pn pEq ě 1 ´ η for all n sufficiently large by Assumption 4.4 and Lemma E.5. On E, we therefore have
˚
˚
}D̃n ´ D} ă η and maxjPJ ˚ |ξ˜n,j ´ π1,j
| ă η, P̃ ´ a.s. Below, we condition on tXi u8
i“1 P E. For any j P J ,
I
b
1
˚ ˜
˚
|ṽn,j,θ
1 pλq ´ w̃j pλq| ď |G̃n,j pθn q ´ Z̃j | ` ρ}D̃j,n ´ Dj }}λ} ` |ϕj pξn,j q ´ π1,j | ď p2 ` ρqη,
n

(E.95)

uniformly in λ P B d , where we used G̃bn Ñ Z̃, P̃ ´ a.s. Since η can be chosen arbitrarily small, this in turn implies
` ˘
P̃ An ă η{2,
I
I,`δ 1
pθn , cq,
for all n sufficiently large. Note also that supλPB d maxjPJ ˚ |ṽn,j,θ
1 pλq ´ w̃j pλq| ă δ implies W̃pcq Ď Ṽn
n
and hence Acn is a subset of
!
)
Ln ” ω̃ P Ω̃ : W̃pcq Ď ṼnI,`δ pθn1 , cq .
(E.96)

Using this,
P̃pṼnI,`δ pθn1 , cq “ H X W̃pcq ‰ Hq ď P̃pW̃pcq Ę ṼnI,`δ pθn1 , cqq “ P̃pLcn q ď P̃pAn q ă η{2,

(E.97)

for all n sufficiently large. Also, by Lemma E.6,
P̃pṼnI,`δ pθn1 , cq ‰ H X ṼnI pθn1 , cq “ Hq ă η{2,
for all n sufficiently large.
Combining (E.91), (E.93), (E.97), (E.98), and using Pn pEq ě 1 ´ η for all n, we have
ż
ż
P̃pṼnI pθn1 , cq “ H X W̃pcq ‰ HqdPn `
P̃pṼnI pθn1 , cq “ H X W̃pcq ‰ HqdPn ď ηp1 ´ ηq ` η ď 2η.
E

(E.98)

(E.99)

Ec

The second term of the right hand side of (E.90) can be bounded similarly. Therefore, |P ˚ pVnI pθn1 , cq ‰ Hq ´
PrpWpcq ‰ Hq| Ñ 0 with probability (under Pn ) approaching 1. This establishes the first claim.
(ii) By Part (i), for c ą 0, we have
Pn˚ pVnI pθn1 , cq ‰ Hq ´ PrpWpcq ‰ Hq Ñ 0.

[39]

(E.100)

Fix c ą 0, and set
$
’
& c ´ Zj ,
gj “
1,
’
%
0,

j “ 1, . . . , J,
j “ J ` 1, . . . , J ` 2d,
j “ J ` 2d ` 1, J ` 2d ` 2.

(E.101)

Mimic the argument following (E.137). Then, this yields
|Pr pWpcq ‰ Hq ´ Pr pWpc ´ δq ‰ Hq| “ Pr ptWpcq ‰ Hu X tWpc ´ δq “ Huq ď η,

(E.102)

|Pr pWpc ` δq ‰ Hq ´ Pr pWpcq ‰ Hq| “ Pr ptWpc ` δq ‰ Hu X tWpcq “ Huq ď η,

(E.103)

which therefore ensures that c ÞÑ PrpWpcq ‰ Hq is continuous at c ą 0.
Next, we show c ÞÑ Pr pWpcq ‰ Hq is strictly increasing at any c ą 0. For this, consider c ą 0 and c ´ δ ą 0 for
δ ą 0. Define the J vector e to have elements ej “ c ´ Zj , j “ 1, . . . , J. Suppose for simplicity that J ˚ contains
˚
the first J ˚ inequality constraints. Let er1:J s denote the subvector of e that only contains elements corresponding
˚
to j P J ˚ , define Dr1:J ,:s correspondingly, and write
fi
»
»
»
fi
fi
˚
˚
1J ˚
Dr1:J ,:s
er1:J s
ffi
—
—
—
ffi
ffi
— 0d ffi
—
— ρ ¨ 1d ffi
ffi
Id
ffi
—
—
—
ffi
ffi
ffi , g “ — ρ ¨ 1d ffi , τ “ — 0d ffi .
(E.104)
K“—
ffi
—
— ´Id
—
ffi
ffi
ffi
—
—
—
ffi
ffi
1
0
p
0
fl
–
–
–
fl
fl
0
0
´p1
By Farkas’ lemma (Rockafellar, 1970, Theorem 22.1) and arguing as in (E.142),
`
˘
Pr ptWpcq ‰ Hu X tWpc ´ δq “ Huq “ Pr tµ1 g ě 0, @µ P Mu X tµ1 pg ´ δτ q ă 0, Dµ P Mu ,

(E.105)

˚

where M “ tµ P RJ` `2d`2 : µ1 K “ 0u. By Minkowski-Weyl’s theorem (Rockafellar and Wets, 2005, Theorem 3.52),
there exists tν t P M, t “ 1, ¨ ¨ ¨ , T u, for which one may write
M “ tµ : µ “ b

T
ÿ

at ν t , b ą 0, at ě 0,

T
ÿ

at “ 1u.

(E.106)

µ1 g ě 0, @µ P M ô ν t1 g ě 0, @t P t1, ¨ ¨ ¨ , T u

(E.107)

t“1

t“1

This implies

µ1 pg ´ δτ q ă 0, Dµ P M ô ν t1 g ă δν t1 τ, Dt P t1, ¨ ¨ ¨ , T u.

(E.108)

Hence,
`
˘
`
˘
Pr tµ1 g ě 0, @µ P Mu X tµ1 pg ´ δτ q ă 0, Dµ P Mu “ Pr 0 ď ν s1 g, 0 ď ν t1 g ă δν t1 τ, @s, Dt

(E.109)

Note that by (E.104), for each s P t1, ¨ ¨ ¨ , T u,
ν s1 g “ ν s,r1:J

˚

s1

pc1J ˚ ´ ZJ ˚ q ` ρ

J ˚ÿ
`2d

ν s,rjs ,

(E.110)

j“J ˚ `1
˚

s1

ν τ“

J
ÿ

ν s,rjs .

j“1

[40]

(E.111)

For each s P t1, ¨ ¨ ¨ , T u, let
˚

hU
s

”c

J
ÿ

ν

s,rjs

j“1

`ρ

J ˚ÿ
`2d

ν s,rjs

(E.112)

j“J ˚ `1
˚

hL
s ” pc ´ δq

J
ÿ

ν s,rjs ,

(E.113)

j“1
˚

J `2d`2
U
s
. One may therefore rewrite the
where 0 ď hL
s ă hs for all s P t1, ¨ ¨ ¨ , T u due to 0 ă c ´ δ ă c and ν P R`
probability on the right hand side of (E.109) as
´
¯
`
˘
˚
L
t,r1:J ˚ s1
U
˚ ď h
Pr 0 ď ν s1 g, 0 ď ν t1 g ă δν t1 τ, @s, Dt “ Pr ν s,r1:J s1 ZJ ˚ ď hU
,
h
ă
ν
Z
@s,
Dt
ą 0, (E.114)
J
s
t
t

where the last inequality follows because ZJ ˚ ’s correlation matrix Ω has an eigenvalue bounded away from 0 by
Assumption 4.3. By (E.105), (E.109), and (E.114), c ÞÑ Pr pWpcq ‰ Hq is strictly increasing at any c ą 0.
Suppose that cπ˚ ą 0, then arguing as in Lemma 5.(i) of Andrews and Guggenberger (2010), we obtain
P
cIn pθn1 q Ñn cπ˚ .
(iii) Begin with observing that one can equivalently express ĉn (originally defined in (3.5)) as ĉn pθq “ inftc P
R` : Pn˚ pVnb pθ, cq ‰ Hq ě 1 ´ αu.
Suppose first that Assumption 4.3-(I) holds. In this case, there are no paired inequalities, and VnI differs from
Vnb only in terms of the function ϕ˚j in (E.82) used in place of the GMS function ϕj . In particular, ϕ˚j pξq ď ϕj pξq
for any j and ξ, and therefore ĉn pθn q ě cIn pθn q by construction.
Next, suppose Assumption 4.3-(II) holds and VnI pθn1 , cq is defined with hard threshold GMS as in equation (3.3),
i.e. with GMS function ϕ1 in AS. The only case that might create concern is one in which
π1,j P r´1, 0q and π1,j`R1 “ 0.

(E.115)

In this case, only the j ` R1 -th inequality binds in the limit, but with probability approaching 1, GMS selects both
of the pair. Therefore, we have
˚
˚
“ 0,
π1,j
“ ´8, and π1,j`R
1
ϕj pξˆn,j pθn1 qq “ 0, and ϕj`R1 pξˆn,j`R1 pθn1 qq “ 0,

(E.116)
(E.117)

so that in VnI pθn1 , cq, inequality j ` R1 , which is
Gbn,j`R1 pθn1 q ` ρD̂n,j`R1 pθn1 qλ ď c,

(E.118)

´Gbn,j pθn1 q ´ ρD̂n,j pθn1 qλ ď c,

(E.119)

is replaced with inequality

as explained in Section 4.1. In this case, ĉn pθn q ě cIn pθn q is not guaranteed in finite sample. However, let vnIP be as
in (E.80) but replacing j ` R1 -th component Gbn,j`R1 pθn q ` D̂n,j`R1 pθn qλ ` ϕ˚j`R1 pξˆn,j`R1 pθn qq with ´Gbn,j pθn q ´
D̂n,j pθn qλ ´ ϕ˚j pξˆn,j pθn qq. Define VnIP as in (E.83) but replacing vnI with vnIP . Define cIP
n pθn q ” inftc P R` :
? d
˚
IP
1
IP 1
1
P pVn pθn , cqq ě 1 ´ αu. By construction, ĉn pθn q ě cn pθn q for any θn P pθn ` ρ{ nB q X Θ. Therefore, it suffices
1
I
1 Pn
to show that cIP
n pθn q ´ cn pθn q Ñ 0. For this, note that Lemma E.9-(3) establishes
sup }Gbn,j`R1 pθn1 q ` ρD̂n,j`R1 pθn1 qλ ` Gbn,j pθn1 q ` ρD̂n,j pθn1 qλ} “ oP ˚ p1q,

d
λPBn,ρ

[41]

(E.120)

IP
for almost all sample paths tXi u8
i“1 . Therefore, replacing the j ` R1 -th inequality with the j-th inequality in Vn
is asymptotically negligible. Mimicking the arguments in Parts (i) and (ii) then yields
P

n
1
cIP
n pθn q Ñ cπ ˚ .

(E.121)

P

n
1
I
1
This therefore ensures cIP
n pθn q ´ cn pθn q Ñ 0.
If the set VnI pθn1 , cq is defined with a GMS function satisfying Assumption 4.2 and continuous in its argument,
we can mimic the above argument using the replacements in (E.12)-(E.13) with µ̂n,j`R1 as defined in (E.14) and
µ̂n,j pθn1 q as in (E.15). Then when both πj P p´8, 0s and πj`R1 P p´8, 0s we have:
›
›
∆pµ, µ̂q ” ›µ̂n,j pθn1 qtGbn,j pθn1 q ` ρD̂n,j pθn1 qλu ´ µ̂n,j`R1 pθn1 qtGbn,j`R1 pθn1 q ` ρD̂n,j`R1 pθn1 qλu
›
›
´µj pθn1 qtGbn,j pθn1 q ` ρD̂n,j pθn1 qλu ` µj`R1 pθn1 qtGbn,j`R1 pθn1 q ` ρD̂n,j`R1 pθn1 qλu› “ oP p1q,

?
where µj , µj`R1 are defined in equations (E.10)-(E.11) for θ P θn `pθn `ρ{ nB d qXΘ. Replacing µ̂n,j “ 1´ µ̂n,j`R1
and µj “ 1 ´ µj`R1 in the definition of ∆pµ, µ̂q, we have
ˇ
ˇ›
›
(E.122)
∆pµ, µ̂q ď ˇµ̂n,j`R1 pθn1 q ´ µj`R1 pθn1 qˇ›tGbn,j`R1 pθn1 q ` ρD̂n,j`R1 pθn1 qλu ` tGbn,j pθn1 q ` ρD̂n,j pθn1 qλu›.
d
If both πj P p´8, 0s, πj`R1 P p´8, 0s, the result follows by the fact that λ P Bn,ρ
and µ̂n,j , µ̂n,j`R1 , µj , µj`R1 are
bounded in r0, 1s, by Lemma E.9-(3)-(4), and by Assumption 4.4-(i). The rest of the argument follows similarly as
for the case of hard-threshold GMS.

Lemma E.4: Let Assumptions 4.1, 4.2, 4.4, and 4.5 hold. Let pPn , θn q be the sequence satisfying (E.1)-(E.3),
?
let J ˚ be defined as in (E.29), and assume that J ˚ ‰ H. Then, for any ε, η ą 0 and θn1 P pθn ` ρ{ nB d q X Θ,
2
2
there exists N 1 P N and N P N such that for all n ě maxtN 1 , N u,
¸
˜
ˇ
ˇ
ˇ
ˇ
(E.123)
P sup ˇˇ max pu˚n,j,θn pλq ´ c˚n q ´ max pw˚j pλq ´ cπ˚ qˇˇ ě ε ă η,
λPB d j“1,¨¨¨ ,J

˜
P̃

j“1,¨¨¨ ,J

¸
ˇ
ˇ
ˇ
ˇ
I
ˇ
ˇ
sup ˇ max w̃j pλq ´ max ṽn,j,θn1 pλq ˇ ě ε ă η, w.p.1,
j“1,¨¨¨ ,J
d j“1,¨¨¨ ,J

(E.124)

λPB

where the functions u˚n , w˚ , ṽn , w̃ are defined in equations (E.24),(E.25), (E.88), and (E.89).
˚
Proof. We first establish (E.123). By definition, π1,j
“ ´8 for all j R J ˚ and therefore

´
¯
P sup | max pu˚n,j,θn pλq ´ c˚n q ´ max pw˚j pλq ´ cπ˚ q| ě ε
j“1,¨¨¨ ,J
λPB d j“1,¨¨¨ ,J
´
¯
“ P sup | max˚ pu˚n,j,θn pλq ´ c˚n q ´ max˚ pw˚j pλq ´ cπ˚ q| ě ε . (E.125)
λPB d jPJ

jPJ

Hence, for the conclusion of the lemma, it suffices to show, for any ε ą 0,
´
¯
lim P sup | max˚ pu˚n,j,θn pλq ´ c˚n q ´ max˚ pw˚j pλq ´ cπ˚ q| ě ε “ 0.
nÑ8

λPB d jPJ

jPJ

˚
For each λ P Rd , define rn,j,θn pλq ” pu˚n,j,θn pλq ´ c˚n q ´ pw˚j pλq ´ cn q. Using the fact that π1,j
“ 0 for j P J ˚ ,

[42]

?

n
ρ pΘ

and the triangle and Cauchy-Schwarz inequalities, for any λ P B d X

´ θn q and j P J ˚ , we have

λρ
|rn,j,θn pλq| ď |G˚n,j pθn ` ? q ´ Z˚j | ` ρ}DPn ,j pθ̄n q ´ Dj }}λ}
n
λρ
˚
` |G˚n,j pθn ` ? q ` ρDPn ,j pθ̄n qλ|ηn,j
` |c˚n ´ cπ˚ |
n
λρ
˚
` oP p1q
“ |G˚n,j pθn ` ? q ´ Z˚j | ` op1q ` tOP p1q ` Op1quqηn,j
n
“ oP p1q

(E.126)
?

where the first equality follows from }λ} ď d, DPn pθ̄n q Ñ D due to DPn pθn q Ñ D, Assumption 4.4-(ii), and
?
?
θ̄n being a mean value between θn and θn ` λρ{ n. We also note that }Gn,j pθ ` λ{ nq} “ OP p1q, }DP,j pθq}
a.s.
being uniformly bounded for θ P ΘI pP q (Assumption 4.4-(i)), and c˚n Ñ cπ˚ . The last equality follows from
a.s.
λρ
G˚n,j pθn ` ?
q ´ Z˚j Ñ 0 and supθPΘ |ηn,j pθq| “ oP p1q by Lemma E.10.
n
˚
˚
We note that when paired inequalities are merged, for each j “ 1, . . . , R1 such that π1,j
“ 0 “ π1,j`R
we
1
have that |µ̃j ´ µj | “ oP p1q because supθPΘ |ηj pθq| “ oP p1q, where µ̃j and µj were defined in (D.11)-(D.12) and
(E.10)-(E.11) respectively.
By (E.126) and the fact that j P J ˚ , we have
sup | max˚ pu˚n,j,θn pλq ´ c˚n q ´ max˚ pw˚j pλq ´ cπ˚ q| ď sup max˚ |rn,j,θn pλq| “ oP p1q.

λPB d jPJ

(E.127)

λPB d jPJ

jPJ

The conclusion of the lemma then follows from (E.125) and (E.127).
The result in (E.124) follows from similar arguments.
Lemma E.5: Let Assumptions 4.1, 4.2, 4.4, and 4.5 hold. Given a sequence tQn , ϑn u P tpP, θq : P P P, θ P
?
ΘI pP qu such that limnÑ8 κ´1
nγ1,Qn ,j pϑn q exists for each j “ 1, . . . , J, let χj ptQn , ϑn uq be a function of the
n
sequence tQn , ϑn u defined as
#
?
nγ1,Qn ,j pϑn q “ 0,
0,
if limnÑ8 κ´1
n
(E.128)
χj ptQn , ϑn uq ”
?
´1
´8, if limnÑ8 κn nγ1,Qn ,j pϑn q ă 0.
?
?
Then for any θn1 P θn ` ?ρn B d for all n, one has: (i) κ´1
nγ1,Pn ,j pθn q´κ´1
nγ1,Pn ,j pθn1 q “ op1q; (ii) χptPn , θn uq “
n
n
?

˚
; and (iii) κ´1
χptPn , θn1 uq “ π1,j
n

1
nm̄n,j pθn
q
1 q
σ̂n,j pθn

?

´ κ´1
n

1
nEPn rmj pXi ,θn
qs
1 q
σPn ,j pθn

“ oP p1q.

Proof. For (i), the mean value theorem yields
ˇ?
ˇ
ˇ nE pm pX, θqq ?nE pm pX, θ1 qq ˇ
ˇ
ˇ
P
j
P
j
sup
sup ?
´
ˇ
ˇ
1q
ˇ
ˇ
κ
σ
pθq
κ
σ
pθ
1
d
n P,j
n P,j
P PP θPΘI pP q,θ Pθ`ρ{ nB
?
ď sup

sup

?
P PP θPΘI pP q,θ 1 Pθ`ρ{ nB d

n}DP,j pθ̃q}}θ1 ´ θ}
“ op1q, (E.129)
κn

where θ̃ represents a mean value that lies componentwise between θ and θ1 and where we used the fact that DP,j pθq
is Lipschitz continuous and supP PP supθPΘI pP q }DP,j pθq} ď M̄ . Result (ii) then follows immediately from (E.128).

[43]

For (iii), note that
?
?
ˇ
ˇ
1
ˇ ´1 nm̄n,j pθn1 q
´1 nEPn rmj pXi , θn qs ˇ
sup?
´
κ
ˇκn
ˇ
n
σ̂n,j pθn1 q
σPn ,j pθn1 q
1 Pθ `ρ{ nB d
θn
n
?
?
ˇ
ˇ
1
ˇ ´1 npm̄n,j pθn1 q ´ EPn rmj pXi , θn1 qsq
ˇ
1
´1 nEPn rmj pXi , θn qs
ď
sup?
p1
`
η
pθ
qq
`
κ
ηn,j pθn1 qˇ
ˇκn
n,j n
n
1
1
σ
pθ
q
σ
pθ
q
1
d
n,j n
Pn ,j n
θn Pθn `ρ{ nB
ˇ ?nE rm pX , θ1 qs
ˇ
ˇ
Pn
j
i n
1
1
1 ˇ
ď
sup?
|κ´1
G
pθ
qp1
`
η
pθ
qq|
`
η
pθ
q
ˇ
n n
n,j n
n,j n ˇ “ oP p1q, (E.130)
n
κn σPn ,j pθn1 q
1 Pθ `ρ{ nB d
θn
n
where the last equality follows from supθPΘ |Gn pθq| “ OP p1q due to asymptotic tightness of tGn u (uniformly in P )
by Lemma D.1 in Bugni, Canay, and Shi (2015), Theorem 3.6.1 and Lemma 1.3.8 in van der Vaart and Wellner
(2000), and supθPΘ |ηn,j pθq| “ oP p1q by Lemma E.10-(i).
?
Lemma E.6: Let Assumptions 4.1, 4.2, 4.3, 4.4, and 4.5 hold. For any θn1 P pθn ` ρ{ nB d q X Θ,
(i) For any η ą 0, there exist δ ą 0 such that
sup PrptWpcq ‰ Hu X tW´δ pcq “ Huq ă η.

(E.131)

cě0

Moreover, for any η ą 0, there exist δ ą 0 and N P N such that
sup Pn˚ ptVnI pθn1 , cq ‰ Hu X tVnI,´δ pθn1 , cq “ Huq ă η, @n ě N.

(E.132)

(
W´δ pcq ” λ P Bdρ : p1 λ “ 0 X wj pλq ď c ´ δ, @j “ 1, . . . , J ,

(E.133)

(
d
I
VnI,´δ pθn1 , cq ” λ P Bn,ρ
: p1 λ “ 0 X vn,j,θ
1 pλq ď c ´ δ, @j “ 1, . . . , J .
n

(E.134)

cě0

(ii) Fix c ą 0 and redefine

and

Then for any η ą 0, there exists δ ą 0 such that
sup PrptWpcq ‰ Hu X tW´δ pcq “ Huq ă η.

(E.135)

cěc

with W´δ pcq defined in (E.133). Moreover, for any η ą 0, there exist δ ą 0 and N P N such that
sup Pn˚ ptVnI pθn1 , cq ‰ Hu X tVnI,´δ pθn1 , cq “ Huq ă η, @n ě N,

(E.136)

cěc

with Vn´δ pθn1 , cq defined in (E.134).
Proof. We first show (E.131). If J ˚ “ H, with J ˚ as defined in (E.29), then the result is immediate. Assume then
that J ˚ ‰ H. Any inequality indexed by j R J ˚ is satisfied with probability approaching one by similar arguments
as in (D.20) (both with c and with c ´ δ). Hence, one could argue for sets Wpcq, W´δ pcq defined as in equations
(E.16) and (E.17) but with j P J ˚ . To keep the notation simple, below we argue as if all j “ 1, . . . , J belong to

[44]

J ˚ . Let c ě 0 be given. Let g be a J ` 2d ` 2 vector with entries
$
’
& c ´ Zj , j “ 1, . . . , J,
gj “
1,
j “ J ` 1, . . . , J ` 2d,
’
%
0,
j “ J ` 2d ` 1, J ` 2d ` 2,
˚
recalling that π1,j
“ 0 for j “ J1 ` 1, ¨ ¨ ¨ , J. Let τ be a pJ ` 2d ` 2q vector with entries
#
1, j “ 1, . . . , J1 ,
τj “
0, j “ J1 ` 1, . . . , J ` 2d ` 2.

(E.137)

(E.138)

Then we can express the sets of interest as

W

Wpcq “ tλ : Kλ ď gu,

(E.139)

´δ

(E.140)

pcq “ tλ : Kλ ď g ´ δτ u.

By Farkas’ Lemma, e.g. Rockafellar (1970, Theorem 22.1), a solution to the system of linear inequalities in (E.139)
exists if and only if for all µ P RJ`2d`2
such that µ1 K “ 0, one has µ1 g ě 0. Similarly, a solution to the system of
`
linear inequalities in (E.140) exists if and only if for all µ P RJ`2d`2 such that µ1 K “ 0, one has µ1 pg ´ δτ q ě 0.
Define
J`2d`2
M ” tµ P R`
: µ1 K “ 0u.

(E.141)

Then, one may write
PrptWpcq ‰ Hu X tW ´δ pθn1 , cq “ Huq
“ Prptµ1 g ě 0, @µ P Mu X tµ1 pg ´ δτ q ă 0, Dµ P Muq
“ Prptµ1 g ě 0, @µ P Mu X tµ1 g ă δµ1 τ, Dµ P Muq.

(E.142)

Note that the set M is a non-stochastic polyhedral cone which may change with n. By Minkowski-Weyl’s theorem
(see, e.g. Rockafellar and Wets (2005, Theorem 3.52)), for each n there exist tν t P M, t “ 1, ¨ ¨ ¨ , T u, with T ă 8
a constant that depends only on J and d, such that any µ P M can be represented as

µ“b

T
ÿ

at ν t ,

(E.143)

t“1

where b ą 0 and at ě 0, t “ 1, . . . , T,
of vector ν t , we have

řT
t“1

at “ 1. Hence, if µ P M satisfies µ1 g ă δµ1 τ , denoting ν t1 the transpose
T
ÿ

at ν t1 g ă δ

t“1

T
ÿ

at ν t1 τ.

(E.144)

t“1

However, due to at ě 0, @t and ν t P M, this means ν t1 g ă δν t1 τ for some t P t1, . . . , T u. Furthermore, since ν t P M,

[45]

we have 0 ď ν t1 g. Therefore,
`
˘
Pr tµ1 g ě 0, @µ P Mu X tµ1 g ă δµ1 τ, Dµ P Mu
T
`
˘ ÿ
`
˘
ď Pr 0 ď ν t1 g ă δν t1 τ, Dt P t1, ¨ ¨ ¨ , T u ď
Pr 0 ď ν t1 g ă δν t1 τ . (E.145)
t“1

Case 1. Consider first any t “ 1, . . . , T such that ν t assigns positive weight only to constraints in tJ ` 1, . . . , J `
2d ` 2u. Then
ν t1 g “

J`2d
ÿ

νjt ,

j“J`1

δν t1 τ “ δ

J`2d`2
ÿ

νjt τj “ 0,

j“J`1

where the last equality follows by (E.138). Therefore Pr p0 ď ν t1 g ă δν t1 τ q “ 0.
Case 2. Consider now any t “ 1, . . . , T such that ν t assigns positive weight also to constraints in t1, . . . , Ju. Recall
that indices j “ J1 ` 1, . . . , J1 ` 2J2 correspond to moment equalities, each of which is written as two moment
inequalities, therefore yielding a total of 2J2 inequalities with Dj`J2 “ ´Dj for j “ J1 ` 1, . . . , J1 ` J2 , and:
#
c ´ Zj
j “ J1 ` 1, . . . , J1 ` J2 ,
g“
(E.146)
c ` Zj´J2 j “ J1 ` J2 ` 1, . . . , J.
For each ν t , (E.146) implies
J1 ÿ
`2J2

νjt gj “ c

j“J1 `1

J1ÿ
`2J2

νjt `

j“J1 `1

J1ÿ
`J2

t
pνjt ´ νj`J
qZj .
2

(E.147)

j“J1 `1

For each j “ 1, ¨ ¨ ¨ , J1 ` J2 , define
ν̃jt ”

$
&ν t

j “ 1, ¨ ¨ ¨ , J1

j

%ν t ´ ν t
j

j`J2

.

(E.148)

j “ J1 ` 1, ¨ ¨ ¨ , J1 ` J2 .

t
t
q1 and have
We then let ν̃ t ” pν̃n,1
, ¨ ¨ ¨ , ν̃n,J
1 `J2

ν t1 g “

J1ÿ
`J2

ν̃jt Zj ` c

j“1

J
ÿ
j“1

νjt `

J`2d
ÿ

νjt .

(E.149)

j“J`1

t1

Case 2-a. Suppose ν̃ t ‰ 0. Then, by (E.149), ννt1 τg is a normal random variable with variance pν̃ t1 τ q´2 ν̃ 1t Ων̃ t . By
Assumption 4.3, there exists a constant ω ą 0 such that the smallest eigenvalue of Ω is bounded from below by ω
for all θn1 . Hence, letting } ¨ }p denote the p-norm in RJ`2d`2 , we have
ν̃ 1t Ων̃ t
ω}ν̃ t }22
ω
ě
ě
.
t1
2
pν̃ τ q
pJ ` 2d ` 2q2 }ν̃ t }22
pJ ` 2d ` 2q2

(E.150)

Therefore, the variance of the normal random variable in (E.145) is uniformly bounded away from 0, which in turn
t1
allows one to find δ ą 0 such that Prp0 ď ννt1 τg ă δq ď η{T .
Case 2-b. Next, consider the case ν̃ t “ 0. Because we are in the case that ν t assigns positive weight also to
t
constraints in t1, . . . , Ju, this must be because νjt “ 0 for all j “ 1, ¨ ¨ ¨ , J1 and νjt “ νj`J
for all j “ J1 `
2

[46]

řJ
řJ
1, ¨ ¨ ¨ , J1 ` J2 , while νjt ‰ 0 for some j “ J1 ` 1, ¨ ¨ ¨ , J1 ` J2 . Then we have j“1 νjt g ě 0, and j“1 νjt τj “ 0
because τj “ 0 for each j “ J1 ` 1, . . . , J. Hence, the argument for the case that ν t assigns positive weight only to
constraints in tJ ` 1, . . . , J ` 2d ` 2u applies and again Pr p0 ď ν t1 g ă δν t1 τ q “ 0. This establishes equation (E.131).
To see why equation (E.132) holds, observe that the bootstrap distribution is conditional on X1 , . . . , Xn . Therefore, the matrix K̂n , defined as the matrix in equation (E.57) but with D̂n replacing DP , can be treated as nonstochastic. This implies that the set M̂n , defined as the set in equation (E.141) but with K̂n replacing K, can be
treated as nonstochastic as well.
By an application of Lemma D.2.8 in Bugni, Canay, and Shi (2015) together with Lemma E.17 (through an
d
argument similar to that following equation (E.87)), Gbn Ñ GP in l8 pΘq uniformly in P conditional on tX1 , ¨ ¨ ¨ , Xn u,
P
and by Assumption 4.4 D̂n pθn1 q Ñn D, for almost all sample paths. Set
$
˚
1
b
1
’
& c ´ ϕj pξn,j pθn qq ´ Gn,j pθn q, j “ 1, . . . , J,
1
gPn ,j pθn q “
(E.151)
1,
j “ J ` 1, . . . , J ` 2d,
’
%
0,
j “ J ` 2d ` 1, J ` 2d ` 2,
d

and note that |ϕ˚j pξn,j pθn1 qq| ă η for all j P J ˚ , and Gbn,j pθn1 q|tXi u8
i“1 Ñ N p0, Ωq. Then one can mimic the argument
following (E.137) to conclude (E.132).
The results in (E.135)-(E.136) follow by similar arguments, with proper redefinition of τ in equation (E.138).
Lemma E.7: Let Assumptions 4.3 and 4.5 hold. Let pPn , θn q have the almost sure representations given in
Lemma E.1, let J ˚ be defined as in (E.29), and assume that J ˚ ‰ H. Let Cr collect all size d subsets C of
t1, ..., J ` 2d ` 2u ordered lexicographically by their smallest, then second smallest, etc. elements. Let the random
variable C equal the first element of Cr s.t. det K C ‰ 0 and λC “ pK C q´1 g C P W˚,´δ p0q if such an element exists;
else, let C “ tJ ` 1, ..., J ` du and λC “ 1d , where 1d denotes a d vector with each entry equal to 1, and K, g and
W˚,´δ are as defined in Lemma E.2. Then, for any η ą 0, there exist 0 ă εη ă 8 and N P N s.t. n ě N implies
ˇ
ˇ
˘
`
(E.152)
P W˚,´δ p0q ‰ H, ˇdet K C ˇ ď εη ď η.
Proof. We bound the probability in (E.152) as follows:
ˇ
ˇ
`
˘
P W˚,´δ p0q ‰ H, ˇdet K C ˇ ď εη
ď
ď
ď

´
¯
ˇ
ˇ
P DC P Cr : λC P B d , ˇdet K C ˇ ď εη
ÿ
`
˘
P λC P B d
r |det K C |ďεη
CPC:
ÿ
`
˘
P λC P B d ,

(E.153)
(E.154)
(E.155)

r C |ďε2{d
CPC:|α
η

where αC denote the smallest eigenvalue of K C K C1 . Here, the first inequality holds because W˚,´δ Ď B d and so
the event in the first probability implies the event in the next one; the second inequality is Boolean algebra; the
`
˘
last inequality follows because | det K C | ě |αC |d{2 . Noting that Cr has J`2d`2
elements, it suffices to show that
d
ˇ Cˇ
` C
˘
η
d
ˇα ˇ ď ε2{d
ď η ” `J`2d`2˘ .
η ùñ P λ P B
d

r Let q C denote the eigenvector associated with αC and recall that because K C K C1 is symmetric,
Thus, fix C P C.

[47]

› C›
›q › “ 1. Thus the claim is equivalent to:

2{d

Now, if |q C1 K C K C1 q C | ď εη

hence

|q C1 K C K C1 q C | ď ε2{d
ùñ PppK C q´1 g C P Bdρ q ď η.
η

(E.156)

and pK C q´1 g C P Bdρ , then the Cauchy-Schwarz inequality yields
ˇ C1 C ˇ ˇˇ C1 C ` C ˘´1 C ˇˇ ? 1{d
ˇq gP ˇ “ ˇq K K
g ˇ ă dεη ,
n

(E.157)

´
¯
?
PppK C q´1 g C P Bdρ q ď P |q C1 g C | ă dε1{d
.
η

(E.158)

If q C assigns non-zero weight only to non-stochastic constraints, the result follows immediately. If q C assigns
non-zero weight also to stochastic constraints, Assumptions 4.3 and 4.5 (iii) yield
eigpΩ̃q ě ω
ùñ V arP pq C1 g C q ě ω
¯
´ ?
? 1{d ¯
?
C1 C
dεη
ùñ P |q C1 g C | ă dε1{d
“ P ´ dε1{d
η
η ăq g ă
? 1{d
2 dεη
,
ă ?
2ωπ
´

(E.159)

where the result in (E.159) uses that the density of a normal r.v. is maximized at the expected value. The result
follows by choosing
ˆ ?
˙d
η 2ωπ
?
εη “
.
2 d

Lemma E.8: Let Assumptions 4.1, 4.2, 4.3, 4.4, and 4.5 hold. If J2 ě d, then Dc ą 0 s.t.
lim inf inf

inf

nÑ8 P PP θPΘI pP q

P pcIn pθq ě cq “ 1.

Proof. Fix any c ě 0 and restrict attention to constraints tJ1 ` 1, ..., J1 ` d, J1 ` J2 ` 1, ..., J1 ` J2 ` du, i.e.
the inequalities that jointly correspond to the first d equalities. We separately analyze the case when (i) the
corresponding estimated gradients tD̂n,j pθq : j “ J1 ` 1, ..., J1 ` du are linearly independent and (ii) they are
not. If tD̂n,j pθq : j “ J1 ` 1, ..., J1 ` du converge to linearly independent limits, then only the former case occurs
infinitely often; else, both may occur infinitely often, and we conduct the argument along two separate subsequences
if necessary.
For the remainder of this proof, because the sequence tθn u is fixed and plays no direct role in the proof, we
suppress dependence of D̂n,j pθq and Gbn,j pθq on θ. Also, if C is an index set picking certain constraints, then D̂nC is
the matrix collecting the corresponding estimated gradients, and similarly for Gb,C
n .
Suppose now case (i), then there exists an index set C̄ Ă tJ1 ` 1, ..., J1 ` d, J1 ` J2 ` 1, . . . , J1 ` J2 ` du picking
one direction of each constraint s.t. p is a positive linear combination of the rows of D̂PC̄ . (This choice ensures
that a Karush-Kuhn-Tucker condition holds, justifying the step from (E.160) to (E.161) below.) Then the coverage

[48]

probability P ˚ pVnI pθ, cq ‰ Hq is asymptotically bounded above by
´
!
)
¯
´
!
)
¯
P˚
sup
p1 λ : D̂n,j λ ď c ´ Gbn,j , j P J ˚ ě 0 ďP ˚ sup p1 λ : D̂n,j λ ď c ´ Gbn,j , j P C̄ ě 0
d
λPρBn,ρ

(E.160)

λPRd

´
¯
C̄
“P ˚ p1 pD̂nC̄ q´1 pc1d ´ Gb,
q
ě
0
n
ˆ 1 C̄ ´1
˙
C̄
p pD̂n q pc1d ´ Gb,
n q
˚
b
“P
ě0
C̄ q´1 p
p1 pD̂nC̄ q´1 ΩC
p
D̂
n
P
ˆ 1
˙
C̄
C̄
p adjpD̂n qpc1d ´ Gb,
n q
“P ˚ b
ě0
C̄
p1 padjpD̂nC̄ qΩC
P adjpD̂n qp
ˆ
˙
p1 adjpD̂nC̄ qc1d
b
“Φ
` oP p1q
C̄
p1 padjpD̂nC̄ qΩC
P adjpD̂n qp
ďΦpdω ´1{2 cq ` oP p1q.

(E.161)
(E.162)

(E.163)

(E.164)
(E.165)

Here, (E.160) removes constraints and hence enlarges the feasible set; (E.161) solves in closed form; (E.162) divides
through by a positive scalar; (E.163) eliminates the determinant of D̂nC̄ , using that rows of D̂nC̄ can always be
rearranged so that the determinant is positive; (E.164) follows by Assumption 4.5, using that the term multiplying
C̄
is OP p1q; and (E.165) uses that by Assumption 4.3, there exists a constant ω ą 0 that does not depend
Gb,
n
on θ such that the smallest eigenvalue of ΩP is bounded from below by ω. The result follows for any choice of
c P p0, Φ´1 p1 ´ αq ˆ ω 1{2 {dq.
In case (ii), there exists an index set C̄ Ă tJ1 ` 2, ..., J1 ` d, J1 ` J2 ` 2, ..., J1 ` J2 ` du collecting d ´ 1 or
fewer linearly independent constraints s.t. D̂n,J1 `1 is a positive linear combination of the rows of D̂PC̄ . (Note that
C̄ cannot contain J1 ` 1 or J1 ` J2 ` 1.) One can then write
´
!
)
¯
P˚
sup
p1 λ : D̂n,j λ ď c ´ Gbn,j , j P C̄ Y tJ1 ` J2 ` 1u ě 0
(E.166)
d
λPρBn,ρ

´
¯
ď P ˚ Dλ : D̂n,j λ ď c ´ Gbn,j , j P C̄ Y tJ1 ` J2 ` 1u
´
!
)
ď P˚
sup
D̂n,J1 `1 λ : D̂n,j λ ď c ´ Gbn,j , j P C̄ ě
d
λPρBn,ρ

inf

d
λPρBn,ρ

(E.167)
!
)¯
D̂n,J1 `1 λ : D̂n,J1 `J2 `1 λ ď c ´ Gbn,J1 `J2 `1
(E.168)
¯

´

C̄
b
“ P ˚ D̂n,J1 `1 D̂nC̄1 pD̂nC̄ D̂nC̄1 q´1 pc1d¯ ´ Gb,
n q ě ´c ` Gn,J1 `J2 `1 .

(E.169)

Here, the reasoning from (E.166) to (E.168) holds because we evaluate the probability of increasingly larger events;
in particular, if the event in (E.168) fails, then the constraint sets corresponding to the sup and inf can be separated
by a hyperplane with gradient D̂n,J1 `1 and so cannot intersect. The last step solves the optimization problems
in closed form, using (for the sup) that a Karush-Kuhn-Tucker condition again holds by construction and (for the
inf) that D̂n,J1 `J2 `1 “ ´D̂n,J1 `1 . Expression (E.169) resembles (E.162), and the argument can be concluded in
analogy to (E.163)-(E.165).
Lemma E.9: Let Assumptions 4.1, 4.2, 4.3-(II), 4.4, and 4.5 hold. Suppose that both π1,j and π1,j`R1 are
finite, with π1,j , j “ 1, . . . , J, defined in (D.4). Let pPn , θn q be the sequence satisfying the conditions of Lemma
?
E.3. Then for any θn1 P pθn ` ρ{ nB d q X Θ,
(1) σP2 n ,j pθn1 q{σP2 n ,j`R1 pθn1 q Ñ 1 for j “ 1, ¨ ¨ ¨ , R1 .

[49]

(2) CorrPn pmj pXi , θn1 q, mj`R1 pXi , θn1 qq Ñ ´1 for j “ 1, ¨ ¨ ¨ , R1 .
P˚

P

n
0 for almost all tXi u8
(3) |Gn,j pθn1 q ` Gn,j`R1 pθn1 q| Ñn 0, and |Gbn,j pθn1 q ` Gbn,j`R1 pθn1 q| Ñ
i“1 .

(4) ρ}DPn ,j`R1 pθn1 q ` DPn ,j pθn1 q} Ñ 0.
?

Proof. By Lemma E.5, for each j, limnÑ8 κ´1
n

1
nEPn rmj pXi ,θn
qs
1 q
σPn ,j pθn

“ π1,j , and hence the condition that π1,j , π1,j`R1
?

are finite is inherited by the limit of the corresponding sequences κ´1
n
We first establish Claims 1 and 2. We consider two cases.
Case 1.

1
nEPn rmj pXi ,θn
qs
1 q
σPn ,j pθn

?

and κ´1
n

1
nEPn rmj`J11 pXi ,θn
qs
.
1 q
σPn ,j`J11 pθn

κn
lim ? σPn ,j pθn1 q ą 0,
n

nÑ8

which implies that σPn ,j pθn1 q Ñ 8 at rate

?

(E.170)

n{κn or faster. Claim 1 then holds because

σP2 n ,j`R1 pθn1 q
σP2 n ,j pθn1 q ` V arPn ptj pXi , θn1 qq ` 2CovPn pmj pXi , θn1 q, tj pXi , θn1 qq
“
Ñ 1,
σP2 n ,j pθn1 q
σP2 n ,j pθn1 q

(E.171)

where the convergence follows because V arPn ptj pXi , θn1 qq is bounded due to Assumption 4.3-(II),
|CovPn pmj pXi , θn1 q, tj pXi , θn1 qq{σP2 n ,j pθn1 q| ď pV arPn ptj pXi , θn1 qqq1{2 {σPn ,j pθn1 q,
and the fact that σPn ,j pθn1 q Ñ 8. A similar argument yields Claim 2.
Case 2.
κn
lim ? σPn ,j pθn1 q “ 0.
n

nÑ8

(E.172)

In this case, π1,j being finite implies that EPn mj pXi , θn1 q Ñ 0. Again using the upper bound on tj pXi , θn1 q similarly
to (E.171), it also follows that
κn
lim ? σPn ,j`R1 pθn1 q “ 0,
nÑ8
n
and hence that EPn ptj pXi , θn1 qq Ñ 0. We then have, using Assumption 4.3-(II) again,
ż
V arPn ptj pXi , θn1 qq “ tj px, θn1 q2 dPn pxq ´ EPn rtj pXi , θn1 qs2
ż
ď M tj px, θn1 qdPn pxq ´ EPn rtj pXi , θn1 qs2 Ñ 0.

(E.173)

(E.174)

Hence,
σP2 n ,j`R1 pθn1 q
σP2 n ,j pθn1 q ` V arPn ptj pXi , θn1 qq ` 2CovPn pmj pXi , θn1 q, tj pXi , θn1 qq
“
σP2 n ,j pθn1 q
σP2 n ,j pθn1 q
ď

σP2 n ,j pθn1 q ` V arPn ptj pXi , θn1 qq 2pV arPn ptj pXi , θn1 qqq1{2
`
σP2 n ,j pθn1 q
σPn ,j pθn1 q

Ñ 1,

(E.175)

and the first claim follows.

[50]

To obtain claim 2, note that
CorrPn pmj pXi , θn1 q, mj`R1 pXi , θn1 qq “

´σP2 n ,j pθn1 q ´ CovPn pmj pXi , θn1 q, tj pXi , θn1 qq
σPn ,j pθn1 qσPn ,j`R1 pθn1 q
(E.176)

Ñ ´1,
where the result follows from (E.174) and (E.175).
To establish Claim 3, consider Gn below. Note that, for j “ 1, ¨ ¨ ¨ , R1 ,
fi
řn
1
1
ff »
«
i“1 pmj pXi ,θn q´EPn rmj pXi ,θn qsq
1
?1
1
Gn,j pθn q
σPn ,j pθn q ř
n
řn
fl .
1
1
n
1
1
“–
?1
i“1 pmj pXi ,θn q´EPn rmj pXi ,θn qsq` n
i“1 ptj pXi ,θn q´EPn rtj pXi ,θn qsq
1
Gn,j`R1 pθn1 q
?
´
1

(E.177)

σPn ,j`R1 pθn q

n

Under the conditions of Case 1 above, we immediately obtain
P

|Gn,j pθn1 q ` Gn,j`R1 pθn1 q| Ñn 0.

(E.178)

řn

Under the conditions in Case 2 above, ?1n i“1 ptj pXi , θn1 q ´ EPn rtj pXi , θn1 qs “ oP p1q due to the variance of this
term being equal to V arPn ptj pXi , θn1 qq Ñ 0 and Chebyshev’s inequality. Therefore, (E.178) obtains again. These
results imply that Zj ` Zj`R1 “ 0, a.s. By Lemma E.15, tGbn u converges in law to the same limit as tGn u for almost
all sample paths tXi u8
i“1 . This and (E.178) then imply the second half of Claim 3.
To establish Claim 4, finiteness of π1,j and π1,j`R1 implies that
ˆ
˙
ˆ
˙
mj pX, θn1 q mj`R1 pX, θn1 q
κn
?
`
“
O
.
(E.179)
EPn
P
σPn ,j pθn1 q
σPn ,j`R1 pθn1 q
n
Define the 1 ˆ d vector
qn ” DPn ,j`R1 pθn1 q ` DPn ,j pθn1 q.

(E.180)

Suppose by contradiction that
ρqn Ñ ς ‰ 0,
where }ς} might be infinite. Write
r̃n “

qn1
.
}qn }

(E.181)

Let
?
rn “ r̃n ρκ2n { n.

(E.182)

Using a mean value expansion (where θ̄n and θ̃n in the expressions below are two potentially different vectors that
lie component-wise between θn1 and θn1 ` rn ) we obtain
ˆ
˙
ˆ
˙ ´
¯
mj pX, θn1 ` rn q mj`R1 pX, θn1 ` rn q
mj pX, θn1 q mj`R1 pX, θn1 q
EPn
`
“
E
`
` DPn ,j pθ̄n q ` DPn ,j`R1 pθ̃n q rn
Pn
1
1
1
1
σPn ,j pθn ` rn q
σPn ,j`R1 pθn ` rn q
σPn ,j pθn q
σPn ,j`R1 pθn q
´
¯
`
˘
`
˘
κn
“OP p ? q ` DPn ,j pθn1 q ` DPn ,j`R1 pθn1 q rn ` DPn ,j pθ̄n q ´ DPn ,j pθn1 q rn ` DPn ,j`R1 pθ̃n q ´ DPn ,j`R1 pθn1 q rn
n
κn
ρκ2
ρ2 κ4n
“OP p ? q ` ? n ` OP p
q.
(E.183)
n
n
n
It then follows that there exists N P N such that for all n ě N , the right hand side in (E.183) is strictly greater

[51]

than zero.
Next, observe that
ˆ
˙
mj pX, θn1 ` rn q mj`R1 pX, θn1 ` rn q
EPn
`
σP ,j pθn1 ` rn q
σPn ,j`R1 pθn1 ` rn q
˙ ˆ
˙
ˆ n
σPn ,j`R1 pθn1 ` rn q
EPn pmj`R1 pX, θn1 ` rn qq
mj pX, θn1 ` rn q mj`R1 pX, θn1 ` rn q
`
´
´
1
“EPn
1
1
1
σPn ,j pθn ` rn q
σPn ,j pθn ` rn q
σPn ,j pθn ` rn q
σPn ,j`R1 pθn1 ` rn q
ˆ
˙
mj pX, θn1 ` rn q mj`R1 pX, θn1 ` rn q
ρκ2n
? q.
“EPn
`
´
o
p
(E.184)
P
σPn ,j pθn1 ` rn q
σPn ,j pθn1 ` rn q
n
Here, the last step is established as follows. First, using that σPn ,j pθn1 ` rn q is bounded away from zero for n large
enough by the continuity of σp¨q and Assumption 4.3-(II), we have
σPn ,j`R1 pθn1 ` rn q
σP ,j`R1 pθn1 q
´1“ n
´ 1 ` oP p1q “ oP p1q,
1
σPn ,j pθn ` rn q
σPn ,j pθn1 q

(E.185)

where we used Claim 1. Second, using Assumption 4.4, we have that
EPn pmj`R1 pX, θn1 ` rn qq
EPn pmj`R1 pX, θn1 qq
κn
ρκ2n
?
? q.
“
`
D
p
θ̃
qr
“
O
p
q
`
O
p
P
,j`R
n
n
P
P
n
1
σPn ,j`R1 pθn1 ` rn q
σPn ,j`R1 pθn1 q
n
n

(E.186)

ρκ2

The product of (E.185) and (E.186) is therefore oP p ?nn q and (E.184) follows.
To conclude the argument, note that for n large enough, mj`R1 pX, θn1 ` rn q ď ´mj pX, θn1 ` rn q a.s. because
?
for any θn P ΘI pPn q and θn1 P pθn ` ρ{ nB d q X Θ for n large enough, θn1 ` rn P Θ and Assumption 4.3-(II) applies.
Therefore, there exists N P N such that for all n ě N , the left hand side in (E.183) is strictly less than the right
hand side, yielding a contradiction.
Below, we let R1 “ t1, ¨ ¨ ¨ , R1 u and R2 “ tR1 ` 1, ¨ ¨ ¨ , 2R1 u.
Lemma E.10: Suppose Assumptions 4.1, 4.2, and 4.5 hold. For each θ P Θ, let ηn,j pθq “ σP,j pθq{σ̂n,j pθq ´ 1.
Then, (i) for each j “ 1, . . . , J1 ` J2
´
¯
inf P sup |ηn,j pθq| Ñ 0 “ 1.
(E.187)
P PP

θPΘ

(ii) For any j “ 1, . . . , R1 let
M
M
σ̂n,j
pθq “ σ̂n,j`R
pθq ” µ̂n,j pθqσ̂n,j pθq ` p1 ´ µ̂n,j pθqqσ̂n,j`R1 pθq.
1

(E.188)

?
nγ1,Pn ,j pθn q Ñ π1j P Rr´8s . Let J ˚ be
Let pPn , θn q be a sequence such that Pn P P, θn P Θ for all n, and κ´1
n
defined as in (E.29). Then, for any η ą 0, there exists N P N such that
ˇσ
ˇ
´
¯
ˇ Pn ,j pθn q
ˇ
Pn
max
´
1
ą
η
ăη
(E.189)
ˇ
ˇ
M pθ q
jPpR1 YR2 qXJ ˚ σ̂n,j
n
for all n ě N .
Proof. We first show that, for any  ą 0 and for any j “ 1, . . . , J1 ` J2 ,
ˇ σ̂ pθq
ˇ
´
¯
ˇ
ˇ n,j
inf P sup sup ˇ
´ 1ˇ ď  Ñ 1.
P PP
měn θPΘ σP,j pθq

[52]

(E.190)

For this, define the following sets:
Mj ” tmj p¨, θq{σP,j pθq : θ P Θ, P P Pu
2

Sj ” tpmj p¨, θq{σP,j pθqq : θ P Θ, P P Pu.

(E.191)
(E.192)

By Assumptions 4.1-(a), 4.1 (iv), 4.5 (i), (iii), and arguing as in the proof of Lemma D.2.2 (and D.2.1) in Bugni,
Canay, and Shi (2015), it follows that Sj and Mj are Glivenko-Cantelli (GC) classes uniformly in P P P (in the
sense of van der Vaart and Wellner, 2000, page 167).
Therefore, for any  ą 0,
ˇ n´1 řn m pX , θq2
´
¯
EP rmj pX, θq2 s ˇˇ
ˇ
j
i
i“1
inf P sup sup ˇ
(E.193)
´
ˇď Ñ1
2
2
P PP
σP,j pθq
σP,j pθq
měn θPΘ
ˇ m̄ pθq ´ E rm pX, θqs ˇ
¯
´
ˇ n,j
ˇ
P
j
(E.194)
inf P sup sup ˇ
ˇ ď  Ñ 1.
P PP
σP,j pθq
měn θPΘ
Note that, by Assumption 4.1 (iv), |EP rmj pX, θqs{σP,j pθq| ď M for some constant M ą 0 that does not depend on
P and px2 ´ y 2 q ď |x ` y||x ´ y| ď 2M |x ´ y| for all x, y P r´M, M s. By (E.194), for any  ą 0, it follows that
ˇ m̄ pθq2 ´ E rm pX, θqs2 ˇ
´
¯
ˇ n,j
ˇ
P
j
inf P sup sup ˇ
(E.195)
ˇ ď  Ñ 1.
2
P PP
σP,j pθq
měn θPΘ
?
By the uniform continuity of x ÞÑ x on R` , for any  ą 0, there is a constant η ą 0 such that
ˇ
ˇ σ̂ pθq
ˇ
ˇ σ̂ 2 pθq
ˇ
ˇ n,j
ˇ
ˇ n,j
´ 1ˇ ď η ñ ˇ
´ 1ˇ ď .
ˇ 2
σP,j pθq
σP,j pθq
2
pθq and the triangle inequality,
By the definition of σP,j
ˇ σ̂ 2 pθq
ˇ ˇ n´1 řn mpX , θq2 ´ Erm pX , θq2 s ˇ ˇ m̄ pθq2 ´ Erm pX , θqs2 ˇ
ˇ n,j
ˇ ˇ
ˇ ˇ n,j
ˇ
i
j
i
j
i
i“1
´ 1ˇ ď ˇ
ˇ 2
ˇ`ˇ
ˇ.
2 pθq
2 pθq
σP,j pθq
σP,j
σP,j

(E.196)

(E.197)

By (E.196)-(E.197), bounding each of the terms on the right hand side of (E.197) by η{2 implies |σ̂n,j pθq{σP,j pθq ´
1| ď . This, together with (E.193) and (E.195), ensures that, for any  ą 0, (E.190) holds.
Note that |σ̂n,j pθq{σP,j pθq ´ 1| ď  implies σ̂n,j pθq ą 0, and argue as in the proof of Lemma D.2.4 in Bugni,
Canay, and Shi (2015) to conclude that
ˇ σ pθq
ˇ
´
¯
ˇ P,j
ˇ
inf P sup sup ˇ
´ 1ˇ ď  Ñ 1.
(E.198)
P PP
měn θPΘ σ̂n,j pθq
Finally, recall that ηn,j pθq “ σP,j pθq{σ̂n,j pθq ´ 1 and note that for any  ą 0,
´
¯
1 “ lim inf P sup sup |ηn,j pθq| ď 
nÑ8 P PP
měn θPΘ
´ č
(¯
ď inf lim P
sup |ηn,j pθq| ď 
P PP nÑ8

měn

´
“ inf P

lim

nÑ8

P PP

č

θPΘ

tsup |ηn,j pθq| ď 

(¯

měn θPΘ

´

¯
“ inf P sup |ηn,j pθq| ď , for almost all n ,
P PP

(E.199)

θPΘ

where the second equality is due to the continuity of probability with respect to monotone sequences. Therefore,
the first conclusion of the lemma follows.

[53]

(ii) We first give the limit of µ̂n,j pθn q. Recall the definitions of µ̂n,j`R1 and µ̂n,j pθn q in (E.14)-(E.15).
Note that
?
?
ˇ
ˇ
1
ˇ ´1 nm̄n,j pθn1 q
´1 nEPn rmj pXi , θn qs ˇ
sup?
´
κ
ˇκn
ˇ
n
1
1
σ̂n,j pθn q
σPn ,j pθn q
1 Pθ `ρ{ nB d
θn
n
?
?
ˇ
ˇ
1
ˇ ´1 npm̄n,j pθn1 q ´ EPn rmj pXi , θn1 qsq
1
´1 nEPn rmj pXi , θn qs
1 ˇ
ď
sup?
p1
`
η
pθ
qq
`
κ
η
pθ
q
ˇκn
n,j
n,j
n
n
n ˇ
σn,j pθn1 q
σPn ,j pθn1 q
1 Pθ `ρ{ nB d
θn
n
ˇ ?nE rm pX , θ1 qs
ˇ
ˇ
Pn
j
i n
´1
1
1
1 ˇ
|κn Gn pθn qp1 ` ηn,j pθn qq| ` ˇ
η
pθ
ď
sup?
q
ˇ “ oP p1q, (E.200)
n,j
n
κn σPn ,j pθn1 q
1 Pθ `ρ{ nB d
θn
n
where the last equality follows from supθPΘ |Gn pθq| “ OP p1q due to asymptotic tightness of tGn u (uniformly in P )
by Lemma D.1 in Bugni, Canay, and Shi (2015), Theorem 3.6.1 and Lemma 1.3.8 in van der Vaart and Wellner
(2000), and supθPΘ |ηn,j pθq| “ oP p1q by part (i) of this Lemma. Hence,
!
)
π1,j
P
µ̂n,j pθn q Ñn 1 ´ min maxp0,
q, 1 ,
(E.201)
π1,j`R1 ` π1,j
unless π1,j`R1 ` π1,j “ 0 (this case is considered later). This implies that if π1,j P p´8, 0s and π1,j`R1 “ ´8, one
has
P

µ̂n,j pθn q Ñn 1.

(E.202)

Similarly, if π1,j “ ´8 and π1,j`R1 P p´8, 0s, one has
P

µ̂n,j`R1 pθn q Ñn 1.

(E.203)

Now, one may write
¯
¯ ´σ
´ σ̂ pθ q
¯
σPn ,j pθn q ´ σ̂n,j pθn q
σPn ,j pθn q
Pn ,j pθn q
n,j n
´
1
“
O
´
1
“
´
1
`
p1q
´
1
` oPn p1q,
P
n
M pθ q
M pθ q
M pθ q
σ̂n,j pθn q σ̂n,j
σ̂n,j pθn q
σ̂n,j
σ̂n,j
n
n
n

(E.204)

where the second equality follows from the first conclusion of the lemma. Hence, for the second conclusion of the
M
lemma, it suffices to show σ̂n,j pθn q{σ̂n,j
pθn q ´ 1 “ oP p1q. For this, we consider three cases.
˚
˚
˚
Suppose first j P R1 X J and j ` R1 R J ˚ . Then, π1,j
“ 0 and π1,j`R
“ ´8. Then,
1
M
σ̂n,j
pθn q “ µ̂n,j pθn qσ̂n,j pθn q ` p1 ´ µ̂n,j pθn qqσ̂n,j`R1 pθn q

(E.205)

“ p1 ` oPn p1qqσ̂n,j pθn q ` p1 ´ µ̂n,j pθn qqOPn pσ̂n,j pθn qq,

(E.206)

where the second equality follows from (E.202) and the fact that
´
¯1{2
2
y n pmj pXi , θq, tj pXi , θqq ` Vy
σ̂n,j`R1 pθn q “ σ̂n,j
pθn q ` 2Cov
arn ptj pXi , θqq
´
¯1{2
2
“ σ̂n,j
pθn q ` OPn pσ̂n,j pθn qq ` OPn p1q
“ OPn pσ̂n,j pθn qq, (E.207)
where the second equality follows from, V arPn ptj pXi , θqq being bounded by Assumption 4.3-(II) and
Vy
arn ptj pXi , θqq “ V arPn ptj pXi , θqq ` oPn p1q
y n pmj pXi , θq, tj pXi , θqq ď σ̂n,j pθn qVy
Cov
arn ptj pXi , θqq
where the last inequality is due to the Cauchy-Schwarz inequality.

[54]

(E.208)
1{2

,

(E.209)

Therefore,
M
σ̂n,j pθn q ´ σ̂n,j
pθn q
p1 ´ µ̂n,j pθn qqOPn pσ̂n,j pθn qq
σ̂n,j pθn q
´
1
“
“
“ oPn p1q,
M
M
p1 ` oPn p1qqσ̂n,j pθn q ` p1 ´ µ̂n,j pθn qqOPn pσ̂n,j pθn qq
σ̂n,j pθn q
σ̂n,j pθn q

(E.210)

´1
M
pθn q´
where we used σ̂n,j
pθn q “ OPn p1q by equation (4.3) and part (i) of the lemma. By (E.204) and (E.210), σPn ,j pθn q{σ̂n,j
˚
˚
1 “ oPn p1q. Using a similar argument, the same conclusion follows when j P R1 , j R J , but j ` R1 P R2 X J .
˚
˚
Now consider the case j P R1 X J ˚ and j ` R1 P R2 X J ˚ . Then, π1,j
“ 0 and π1,j`R
“ 0. In this case,
1
µ̂n,j pθn q P r0, 1s for all n and by Lemma E.9 (1),
ˇ σ
ˇ
ˇ
ˇ Pn ,j pθn q
(E.211)
´ 1ˇ “ oPn p1q, for j “ 1, ¨ ¨ ¨ , R1 ,
ˇ
σPn ,j`R1 pθn q

and therefore,
M
pθn q
σPn ,j pθn q ´ σ̂n,j
σPn ,j pθn q
´
1
“
M
M
σ̂n,j pθn q
σ̂n,j pθn q

“

rµ̂n,j pθn q ` p1 ´ µ̂n,j pθn qqsσPn ,j pθn q ´ rµ̂n,j pθn qσ̂n,j pθn q ` p1 ´ µ̂n,j pθn qqσ̂n,j`R1 pθn qs
M pθ q
σ̂n,j
n

“

µ̂n,j pθn qrσPn ,j pθn q ´ σ̂n,j pθn qs p1 ´ µ̂n,j pθn qqrσPn ,j`R1 pθn q ´ σ̂n,j`R1 pθn q ` oPn p1qs
`
, (E.212)
M pθ q
M pθ q
σ̂n,j
σ̂n,j
n
n

M
where the second equality follows from the definition of σ̂n,j
pθn q, and the third equality follows from (E.211) and
σPn ,j`R1 bounded away from 0 due to (4.3). Note that
¯
µ̂n,j pθn qrσPn ,j pθn q ´ σ̂n,j pθn qs
σ̂n,j pθn q ´ σPn ,j pθn q
“
µ̂
pθ
q
´
1
“ oPn p1q,
(E.213)
n,j
n
M pθ q
M pθ q
σ̂n,j pθn q
σ̂n,j
σ̂n,j
n
n

where the second equality follows from the first conclusion of the lemma. Similarly,
p1 ´ µ̂n,j pθn qqrσPn ,j`R1 pθn q ´ σ̂n,j`R1 pθn q ` oPn p1qs
M pθ q
σ̂n,j
n
“ p1 ´ µ̂n,j pθn qq

¯
σ̂n,j`R1 pθn q ´ σPn ,j`R1 pθn q
´
1
`
o
p1q
“ oPn p1q. (E.214)
Pn
M pθ q
σ̂n,j`R1 pθn q
σ̂n,j
n

M
By (E.212)-(E.214), it follows that σPn ,j pθn q{σ̂n,j
pθn q ´ 1 “ oPn p1q. Therefore, the second conclusion holds for all
subcases.

E.2

Lemmas Used to Prove Theorem B.1

Let tXib uni“1 denote a bootstrap sample drawn randomly from the empirical distribution. Define
n
˘
1 ÿ`
Gbn,j pθq ” ?
mj pXib , θq ´ m̄n pθq {σP,j pθq
n i“1
n
1 ÿ
“?
pMn,i ´ 1qmj pXi , θq{σP,j pθq,
n i“1

(E.215)

where tMn,i uni“1 denotes the multinomial weights on the original sample, and we let Pn˚ denote the conditional
distribution of tMn,i uni“1 given the sample path tXi u8
i“1 (see Appendix E.3 for details on the construction of the
bootstrapped empirical process).

[55]

Lemma E.11: (i) Let MP ” tf : X Ñ R : f p¨q “ σP,j pθq´1 mj p¨, θq, θ P Θ, j “ 1, ¨ ¨ ¨ , Ju and let F be its
envelope. Suppose that (i) there exist constants K, v ą 0 that do not depend on P such that
sup N p}F }L2Q , MP , L2Q q ď K´v , 0 ă  ă 1,

(E.216)

Q

where the supremum is taken over all discrete distributions; (ii) There exists a positive constant γ ą 0 such that
}pθ1 , θ̃1 q ´ pθ2 , θ̃2 q} ď δ

ñ

sup }QP pθ1 , θ̃1 q ´ QP pθ2 , θ̃2 q} ď M δ γ .

(E.217)

P PP

Let δn be a positive sequence tending to 0 and let n be a positive sequence such that n {|δnγ ln δn | Ñ 8 as n Ñ 8.
Then,
˜
¸
sup P

}Gn pθq ´ Gn pθ1 qq} ą n

sup

P PP

“ op1q.

(E.218)

}θ´θ 1 }ďδn

Further,
˜
lim

nÑ8

Pn˚

¸
sup

}θ´θ 1 }ďδn

}Gbn pθq

´

Gbn pθ1 qq}

ą

n |tXi u8
i“1

“ 0.

(E.219)

for almost all sample paths tXi u8
i“1 uniformly in P P P.
Proof. For the first conclusion of the lemma, it suffices to show that there is a sequence tn u such that, uniformly
in P :
˜
¸
P

max |Gn,j pθq ´ Gn,j pθ1 q| ą n

sup

}θ´θ 1 }ďδn j“1,¨¨¨ ,J

“ op1q.

(E.220)

For this purpose, we mostly mimic the argument required to show the stochastic equicontinuity of empirical processes
(see e.g. van der Vaart and Wellner, 2000, Ch.2.5). Before doing so, note that, arguing as in the proof of Lemma
D.1 (Part 1) in Bugni, Canay, and Shi (2015), one has
}θ ´ θ1 } ď δn ñ %P pθ, θ1 q ď δ̃n ,

(E.221)

where δ̃n “ Opδnγ q by assumption. Define
MP,δ̃n “ tσP,j pθq´1 mj p¨, θq ´ σP,j pθ1 q´1 mj p¨, θ1 q|θ, θ1 P Θ, %P pθ, θ̃q ă δ̃n , j “ 1, ¨ ¨ ¨ , Ju.

(E.222)

?
Define Zn pδ̃n q ” supf PMδ̃ | npPn ´ P qf |. Then, by (E.221), one has
n

˜
P

¸
sup

max |Gn,j pθq ´ Gn,j pθ1 qq| ą n q ď P pZn pδ̃n q ą n

}θ´θ 1 }ďδn j“1,¨¨¨ ,J

.

(E.223)

From here, we deal with the supremum of empirical processes though symmetrization and an application of a
maximal inequality. By Markov’s inequality and Lemma 2.3.1 (symmetrization lemma) in van der Vaart and
Wellner (2000), one has
ˇ
ˇff
«
n
ˇ 1 ÿ
ˇ
2
ˇ
ˇ
Wi f pXi qˇ ,
P pZn pδ̃n q ą n q ď EP ˆP W
sup ˇ ?
(E.224)
ˇ
n
f PMP,δ̃n ˇ n i“1
W
where tWi uni“1 are i.i.d. Rademacher random variables independent of tXi u8
i“1 whose law is denoted by P . Now,

[56]

fix the sample path tXi uni“1 , and let P̂n be the empirical distribution. By Hoeffding’s inequality, the stochastic
řn
řn
process f ÞÑ tn´1{2 i“1 Wi f pXi qu is sub-Gaussian for the L2P̂ seminorm }f }L2 “ pn´1 i“1 f pXi q2 q1{2 . By the
n
P̂n
maximal inequality (Corollary 2.2.8) and arguing as in the proof of Theorem 2.5.2 in in van der Vaart and Wellner
(2000), one then has
ˇff
ˇ
«
ż δ̃n b
n
ˇ
ˇ 1 ÿ
ˇ
ˇ
Wi f pXi qˇ ď K
EP W
sup ˇ ?
ln N p, MP,δ̃n , L2P̂ qd
n
ˇ
f PMδ̃n ˇ n i“1
0
ż δ̃n {}F } 2
b
L
Q
ďK
sup ln N p}F }L2Q , MP , L2Q qd
Q

0

ż δ̃n {}F }

L2
Q

ď K1

?

´v ln d,

(E.225)

0

?
for some K 1 ą 0, where the last inequality follows from (E.216). Note that ´ ln  ď ´ ln  for  ď δ̃n {}F }L2Q with
n sufficiently large. Hence,
ˇ
ˇff
«
ż δ̃n {}F } 2
n
ˇ 1 ÿ
ˇ
L
Q
ˇ
ˇ
sup ˇ ?
EP W
Wi f pXi qˇ ď K 1 v 1{2
p´ ln qd “ K 1 v 1{2 pδ̃n ´ δ̃n lnpδ̃n qq.
(E.226)
ˇ
f PMδ̃n ˇ n i“1
0
By (E.224) and taking expectations with respect to P in (E.226), it follows that
P pZn pδ̃n q ą n q ď 2K 1 v 1{2 pδ̃n ´ δ̃n lnpδ̃n qq{n “ Opδnγ {n q ` Op|δnγ lnpδn q|{n q “ op1q,

(E.227)

where the last equality follows from the rate condition on n . By (E.223) and (E.227), conclude that the first claim
of the lemma holds.
?
For the second claim, define Zn˚ pδ̃n q ” supf PMδ̃ | npP̂n˚ ´ P̂n qf |, where P̂n˚ is the empirical distribution of
n
tXib uni“1 . Then, by (E.221), one has
˜
¸
ˇ
ˇ
`
˘
ˇ
Pn˚
max |Gbn,j pθq ´ Gbn,j pθ1 q| ą n ˇtXi u8
sup
ď Pn˚ Zn˚ pδ̃n q ą n ˇtXi u8
(E.228)
i“1
i“1 .
}θ´θ 1 }ďδn j“1,¨¨¨ ,J

By Markov’s inequality and Lemma 2.3.1 (symmetrization lemma) in van der Vaart and Wellner (2000), one has
ˇ
«
ff
n
ˇ 1 ÿ
ˇˇ
ˇ
` ˚
˘
2
ˇ
˚
8
b ˇˇ
8
ˇ
Pn Zn pδ̃n q ą n tXi ui“1 ď EPn˚ ˆP W
Wi f pXi qˇˇtXi ui“1
sup ˇ ?
(E.229)
ˇ
n
n i“1
f PMP,δ̃n
ˇ
«
«
ff ˇ
ff
n
ˇ 1 ÿ
ˇˇ
ˇ
2
ˇ
ˇ
ˇ
ˇ
8
sup ˇ ?
Wi f pXib qˇˇtXib u, tXi u8
“ EPn˚ EP W
(E.230)
i“1 ˇtXi ui“1 ,
ˇ
ˇ
n
n i“1
f PMP,δ̃
n

n
where tWi uni“1 are i.i.d. Rademacher random variables independent of tXi u8
i“1 and tMn,i ui“1 . Argue as in (E.224)(E.227). Then, it follows that
γ
γ
Pn˚ pZn˚ pδ̃n q ą n |tXi u8
i“1 q “ Opδn {n q ` Op´δn lnpδn q{n q “ op1q,

for almost all sample paths. Hence, the second claim of the lemma follows.
Lemma E.12: Suppose Assumptions 4.1, 4.2, and 4.5 hold. Let SP ” tf : X Ñ R : f p¨q “ σP,j pθq´2 m2j p¨, θq, θ P
Θ, j “ 1, ¨ ¨ ¨ , Ju and let F be its envelope. (i) If SP is Donsker and pre-Gaussian uniformly in P P P, then
?
sup |ηn,j pθq|˚ “ OP p1{ nq;
(E.231)
θPΘ

[57]

(ii) If |σP,j pθq´1 mj px, θq ´ σP,j pθ1 q´1 mj px, θ1 q| ď M̄ pxq}θ ´ θ1 } with EP rM̄ pXq2 s ă M for all θ, θ1 P Θ, x P X ,
j “ 1, ¨ ¨ ¨ , J, and P P P, then, for any η ą 0, there exists a constant C ą 0 such that
´
¯
lim sup sup P
max
sup |ηn,j pθq ´ ηn,j pθ1 q| ą Cδ ă η.
(E.232)
nÑ8

P PP

j“1,¨¨¨ ,J }θ´θ 1 }ăδ

Proof. We show the claim by first showing that, for any δ ą 0, there exist M ą 0 and N P N such that
ˇ
ˇ σ̂ pθq
´
? ¯
ˇ
ˇ n,j
´ 1ˇ ď M { n ě 1 ´ δ, @n ě N.
inf P 8 sup ˇ
P PP
θPΘ σP,j pθq

(E.233)

By Assumptions 4.1 (iv), 4.5 and Theorem 2.8.2 in van der Vaart and Wellner (2000), MP is a Donsker class
uniformly in P P P. By hypothesis, SP is a Donsker class uniformly in P P P.
Therefore, by the continuous mapping theorem, for any  ą 0,
ˇ
ˇ ´?
ˇ n´1 řn m pX , θq2
¯
EP rmj pX, θq2 s ˇˇ
ˇ
ˇ
ˇ
j
i
i“1
(E.234)
n
sup
´
ˇP
ˇ
ˇ ď C1 ´ Prpsup |HP,j pθq| ď C1 qˇ ď 
2
2
σ
pθq
σ
pθq
θPΘ
θPΘ
P,j
P,j
ˇ ´?
ˇ m̄ pθq ´ E rm pX, θqs ˇ
ˇ
¯
ˇ
ˇ n,j
ˇ
ˇ
P
j
n sup ˇ
(E.235)
ˇP
ˇ ď C2 ´ Prpsup |GP,j pθq| ď C2 qˇ ď .
σ
pθq
P,j
θPΘ
θPΘ
for n sufficiently large uniformly in P P P, where HP,j and GP,j are tight Gaussian processes, and C1 and C2
are the continuity points of the distributions of supθPΘ |HP,j pθq| and supθPΘ |GP,j pθq| respectively. As in the proof
?
?
of Lemma
each term of the right hand side of (E.197) by C1 { n and C2 { n implies that
ˇ 2 E.10 (i),
ˇ bounding
?
ˇ σ̂ pθq
ˇ
supθPΘ ˇ σn,j
n for some constant C ą 0. Now choose C1 ą 0 and C2 ą 0 so that
2 pθq ´ 1ˇ ď C{
P,j

Prpsup |HP,j pθq| ď C1 q ą 1 ´ δ{3, and Prpsup |GP,j pθq| ď C2 q ą 1 ´ δ{3,
θPΘ

(E.236)

θPΘ

and set  ą 0 sufficiently small so that 1 ´ 2δ{3 ´ 2 ě 1 ´ δ. The existence of such continuity points C1 , C2 ą 0 is
due to Theorem 11.1 in Davydov, Lifshitz, and Smorodina (1995) applied to supθPΘ |HP,j pθq| and supθPΘ |GP,j pθq|
respectively. Then, for sufficiently large n,
ˇ n´1 řn m pX , θq2
´?
EP rmj pX, θq2 s ˇˇ
ˇ
j
i
i“1
1´δ ďP
n sup ˇ
´
ˇ ď C1 ,
2 pθq
2 pθq
σP,j
σP,j
θPΘ
ˇ m̄ pθq ´ E rm pX, θqs ˇ
¯
?
ˇ n,j
ˇ
P
j
n sup ˇ
ˇ ď C2
σP,j pθq
θPΘ
ˇ
ˇ σ̂ 2 pθq
¯
´
?
ˇ
ˇ n,j
ď P sup ˇ 2
´ 1ˇ ď C{ n ,
(E.237)
σ
pθq
θPΘ
P,j
uniformly in P P P.
Next, note that, for x ą 0 and 0 ă η ă 1, |x2 ´ 1| ď η implies |x ´ 1| ď 1 ´ p1 ´ ηq1{2 ď η, and hence by (E.237),
for sufficiently large n,
ˇ σ̂ pθq
ˇ
´
? ¯
ˇ n,j
ˇ
1 ´ δ ď P sup ˇ
´ 1ˇ ď C{ n ,
(E.238)
θPΘ σP,j pθq
uniformly in P P P. Finally, note again that |σ̂n,j pθq{σP,j pθq ´ 1| ď  implies σ̂n,j pθq ą 0, and by the local Lipshitz
continuity of x ÞÑ 1{x on a neighborhood around 1, there is a constant C 1 such that
´
? ¯
(E.239)
P sup |ηn,j pθq| ď C 1 { n ě 1 ´ δ,
θPΘ

uniformly in P P P for all n sufficiently large. This establishes the first claim of the lemma.

[58]

(ii) First, consider
˙2
n ˆ
2
ÿ
σ̂n,j
pθq
mpXi , θq
´1
´
“
n
2 pθq
σP,j
σP,j pθq
i“1

˜
n

´1

n
ÿ
mpXi , θq
σP,j pθq
i“1

¸2
.

(E.240)

We claim that this function is Lipschitz with probability approaching 1. To see this, note that, for any θ, θ1 P Θ,
ˇ
˜
˜
¸2
¸2 ˇ
n
n
ˇ
ÿ
mpXi , θ1 q ˇˇ
ˇ ´1 ÿ mpXi , θq
´1
´n
ˇn
ˇ
ˇ
ˇ
σP,j pθq
σP,j pθ1 q
i“1
i“1
ˇ
˜
¸˜
¸ˇ
n
ˇ
mpXi , θq mpXi , θ1 q ˇˇ
ˇ ´1 ÿ mpXi , θq mpXi , θ1 q
“ ˇn
`
´
ˇ
ˇ
σP,j pθq
σP,j pθ1 q
σP,j pθq
σP,j pθ1 q ˇ
i“1
n
ˇ mpX , θq ˇ
ÿ
ˇ
ˇ
i
ď n´1
2 sup ˇ
(E.241)
ˇM̄ pXi q}θ ´ θ1 }.
σ
pθq
P,j
i“1 θPΘ
ˇ
ˇ
řn
ˇ
i ,θq ˇ
Define Bn ” n´1 i“1 2 supθPΘ ˇ mpX
σP,j pθq ˇM̄ pXi q. By Markov and Cauchy-Schwarz inequalities,
„
P pBn ą Cq ď

ErBn s
ď
C

2EP

ˇ
ˇ 1{2
ı1{2
”
ˇ mpXi ,θq ˇ2
EP M̄ pXi q2
supθPΘ ˇ σP,j pθq ˇ
ď

C

2M
,
C

(E.242)

where the third inequality is due to Assumptions 4.1 (iv) and the assumption on M̄ . Hence, for any η ą 0, one may
find C ą 0 such that supP PP P pBn ą Cq ă η for all n.
Similarly, for any θ, θ1 P Θ,
ˇ˜
¸2 ˜
¸2 ˇ
n
n
ˇ
ÿ
mpXi , θ1 q ˇˇ
ˇ ´1 ÿ mpXi , θq
´1
´ n
ˇ
ˇ n
ˇ
ˇ
σP,j pθq
σP,j pθ1 q
i“1
i“1
ˇ
ˇˇ
ˇ
n
n
n
n
ˇ
1 ˇˇ
1 ˇ
ÿ
ÿ
ÿ
mpX
,
θ
q
mpX
,
θq
mpX
,
θ
q
ˇ ´1 ÿ mpXi , θq
ˇ
ˇ
ˇ
i
i
i
“ ˇn
` n´1
n´1
´ n´1
1 q ˇˇˇˇ
1 q ˇˇ
ˇ
σ
pθq
σ
pθ
σ
pθq
σ
pθ
P,j
P,j
P,j
P,j
i“1
i“1
i“1
i“1
ˇ
ˇ
n
n
ˇ
ˇ
ÿ
ˇ mpXi , θq ˇ ´1 ÿ
ď n´1
2 sup ˇ
M̄ pXi q}θ ´ θ1 }.
(E.243)
ˇn
ˇ
ˇ
σ
pθq
P,j
θPΘ
i“1
i“1
Define B̃n ” n´1

ˇ
ˇ
ˇ mpXi ,θq ˇ ´1 řn
2
sup
ˇ
θPΘ σP,j pθq ˇn
i“1
i“1 M̄ pXi q. By Markov, Cauchy-Schwarz, and Jensen’s inequalities,

řn

”´
2EP
ErB̃n s
P pB̃n ą Cq ď
ď
C

n´1

ř

ˇ
ˇ¯2 ı1{2 ”´
¯2 ı1{2
ř
ˇ
i ,θq ˇ
supθPΘ ˇ mpX
EP n´1 M̄ pXi q
σP,j pθq ˇ
C
ď

”
ˇ
ˇ ı1{2
i ,θq ˇ2
2EP supθPΘ ˇ mpX
EP rM̄ pXi q2 s1{2
σP,j pθq
C

ď

2M
, (E.244)
C

where the last inequality is due to Assumptions 4.1 (iv) and the assumption on M̄ . Hence, for any η ą 0, one may
find C ą 0 such that supP PP P pB̃n ą Cq ă η for all n.
Finally, let gpyq ” y ´1{2 ´ 1 and note that |gpyq ´ gpy 1 q| ď 21 supȳPp1´,1`q |ȳ|´3{2 |y ´ y 1 | on p1 ´ , 1 ` q. As
2
2
pθq converges to 1 in probability, and g is locally Lipschitz on a neighborhood of 1.
shown in (E.238), σ̂n,j
pθq{σP,j
Combining this with (E.240)-(E.244) yields the desired result.
Lemma E.13: Suppose Assumption 4.1 holds. Suppose further that |σP,j pθq´1 mj px, θq ´ σP,j pθ1 q´1 mj px, θ1 q| ď

[59]

M̄ pxq}θ ´ θ1 } with EP rM̄ pXq2 s ă M for all θ, θ1 P Θ, x P X , j “ 1, ¨ ¨ ¨ , J, and P P P.
Then,
sup }QP pθ1 , θ̃1 q ´ QP pθ2 , θ̃2 q} ď M }pθ1 , θ̃1 q ´ pθ2 , θ̃2 q},

(E.245)

P PP

for some M ą 0 and for all θ1 , θ̃1 , θ2 , θ̃2 P Θ.
Proof. Recall that
rQP pθ1 , θ̃1 qsj,k “ EP

” m pX , θ q m pX , θ̃ q ı
” m pX , θ q ı ” m pX , θ̃ q ı
j
i 1
j
i 1
k
i 1
k
i 1
´ EP
.
EP
σP,j pθ1 q σP,k pθ̃1 q
σP,j pθ1 q
σP,k pθ̃1 q

(E.246)

For any θ1 , θ̃1 , θ2 , θ̃2 P Θ,
ˇ ” m pX , θ q m pX , θ̃ q ı
” m pX , θ q m pX , θ̃ q ıˇ
ˇ
j
i 1
j
i 2
k
i 1
k
i 2 ˇ
´ EP
ˇEP
ˇ
σP,j pθ1 q σP,k pθ̃1 q
σP,j pθ2 q σP,k pθ̃2 q
ˇ ”´ m pX , θ q m pX , θ q ¯ m pX , θ̃ q ıˇ ˇ ” m pX , θ q ´ m pX , θ̃ q m pX , θ̃ q ¯ıˇ
ˇ
ˇ
ˇ
j
i 1
j
i 2
k
i 2 ˇ
k
i 1
k
i 2
j
i 1
ď ˇEP
´
´
ˇ ` ˇEP
ˇ
σP,j pθ1 q
σP,j pθ2 q
σP,j pθ1 q
σP,k pθ̃2 q
σP,k pθ̃1 q
σP,k pθ̃2 q
ˇ m pX , θq ˇ
ˇ m pX , θq ˇ
ı
”
ı
”
ˇ j i
ˇ k i
ˇ
ˇ
ď EP sup ˇ
ˇM̄ pXi q }θ1 ´ θ2 } ` EP sup ˇ
ˇM̄ pXi q }θ̃1 ´ θ̃2 }
σP,k pθq
σP,j pθq
θPΘ
θPΘ
ď M p}θ1 ´ θ2 } ` }θ̃1 ´ θ̃2 }q,

(E.247)

where the last inequality is due to the Cauchy-Schwarz inequality, Assumption 4.1 (iv), and the assumption on M̄ .
Similarly, for any θ1 , θ̃1 , θ2 , θ̃2 P Θ,
ˇ ” m pX , θ q ı ” m pX , θ̃ q ı
” m pX , θ q ı ” m pX , θ̃ q ıˇ
ˇ
j
i 1
j
i 2
k
i 1
k
i 2 ˇ
´ EP
EP
EP
ˇEP
ˇ
σP,j pθ1 q
σP,j pθ2 q
σP,k pθ̃1 q
σP,k pθ̃2 q
ˇ ” m pX , θ q m pX , θ q ıˇˇ ” m pX , θ̃ q ıˇ ˇ ” m pX , θ q ıˇˇ ” m pX , θ̃ q m pX , θ̃ q ıˇ
ˇ
ˇ
j
i 1
j
i 2 ˇˇ
k
i 2 ˇ
j
i 1 ˇˇ
k
i 1
k
i 2 ˇ
ď ˇEP
´
´
ˇˇEP
ˇ ` ˇEP
ˇˇEP
ˇ
σP,j pθ1 q
σP,j pθ2 q
σP,j pθ1 q
σP,k pθ̃2 q
σP,k pθ̃1 q
σP,k pθ̃2 q
ˇ m pX , θq ˇı
ˇ m pX , θq ˇı
”
”
ˇ
ˇ
ˇ k i
ˇ j i
ď EP sup ˇ
ˇ EP rM̄ pXi qs}θ1 ´ θ2 } ` EP sup ˇ
ˇ EP rM̄ pXi qs}θ̃1 ´ θ̃2 }
σP,k pθq
σP,j pθq
θPΘ
θPΘ
ď M p}θ1 ´ θ2 } ` }θ̃1 ´ θ̃2 }q,

(E.248)

where the last inequality is due to the Cauchy-Schwarz inequality, Assumption 4.1 (iv), and the assumption on M̄ .
The conclusion of the lemma then follows from (E.246)-(E.248).

E.3

Almost Sure Representation Lemma and Related Results

In this appendix, we provide details on the almost sure representation used in Lemmas E.3, E.4, E.6, and E.9. We
start with stating a uniform version of the bootstrap consistency in van der Vaart and Wellner (2000). For this, we
define the original sample X 8 “ pX1 , X2 , ¨ ¨ ¨ q and a n-dimensional multinomial vector Mn on a common probability
space pX 8 , A8 , P 8 q ˆ pZ, C, Qq. We then view X 8 as the coordinate projection on the first 8 coordinates of the
probability space above. Similarly, we view Mn as the coordinate projection on Z. Here, Mn follows a multinomial
distribution with parameter pn; 1{n, ¨ ¨ ¨ , 1{nq and is independent of X 8 . We then let EM r¨|X 8 “ x8 s denote the
conditional expectation of Mn given X 8 “ x8 . Throughout, we let `8 pΘ, RJ q denote uniformly bounded RJ -valued
functions on Θ. We simply write `8 pΘq when J “ 1.

[60]

Using the multinomial weight, we rewrite the empirical bootstrap process as
n
1 ÿ
Gbn,j p¨q “ gj pX 8 , Mn q ” ?
pMn,i ´ 1qmj pXi , ¨q{σ̂n,j p¨q, j “ 1, ¨ ¨ ¨ , J,
n i“1

(E.249)

where gj : X 8 ˆ Z Ñ `8 pΘq is a function that maps the sample path and the multinomial weight pX 8 , Mn q to the
empirical bootstrap process Gbn,j . We then let g : X 8 ˆ Z Ñ `8 pΘ, RJ q be defined by g “ pg1 , ¨ ¨ ¨ , gJ q1 . For any
function f : `8 pΘ, RJ q Ñ R, the conditional expectation of f pGbn q given the sample path X 8 is
ż
EM rf pGbn q|X 8 “ x8 s “ f ˝ gpx8 , mn qdQpmn q,
(E.250)
where, with a slight abuse of notation, we use Q for the induced law of Mn .
Let F be the function space tf p¨q “ pm1 p¨, θq{σP,1 pθq, ¨ ¨ ¨ , mJ p¨, θq{σP,J pθqq, θ P Θ, P P Pu. For each j, define
a bootstrapped empirical process standardized by σP,j as follows:
n
˘
1 ÿ`
mj pXib , θq ´ m̄n pθq {σP,j pθq
Gbn,j pθq ” ?
n i“1
n
1 ÿ
“?
pMn,i ´ 1qmj pXi , θq{σP,j pθq.
n i“1

(E.251)

The following result was shown in the proof of Lemma D.2.8 in Bugni, Canay, and Shi (2015), which is a uniform
version of (a part of) Theorem 3.6.2 in van der Vaart and Wellner (2000). For the definition of a uniform version
of Donskerness and pre-Gaussianity, we refer to van der Vaart and Wellner (2000) pages 168-169. Below, we let
P ˚ denote the outer probability of P and let T ˚ denote the minimal measurable majorant of any (not necessarily
measurable) random element T .
Lemma E.14: Let F be a class of measurable functions with finite envelope function. Suppose F is such that
(i) F is Donsker and pre-Gaussian uniformly in P P P; and (ii) supP PP P ˚ }f ´ P f }2F ă 8. Then,
as˚

sup |EM rhpGbn q|X 8 s ´ ErhpGP qs| Ñ 0,

(E.252)

hPBL1

uniformly in P P P.
The result above gives uniform consistency of the standardized bootstrap process Gbn . We now extend this to
the studentized bootstrap process Gbn .
Lemma E.15: Suppose Assumptions 4.1, 4.2, and 4.5 hold. Then,
as˚

sup |EM rhpGbn q|X 8 s ´ ErhpGP qs| Ñ 0,

(E.253)

hPBL1

uniformly in P P P.
Proof. By Assumptions 4.1 (iv) and 4.5, Assumptions A.1-A.4 in Bugni, Canay, and Shi (2015) hold, which in turn
implies that, by their Lemma D.1.2, F is Donsker and pre-Gaussian uniformly in P P P. Further, by Assumption
4.1 (iv) again, supP PP P ˚ }f ´ P f }F ă 8. Hence, by Lemma E.14,
´
¯
inf P 8 sup |EM rhpGbn q|X 8 s ´ ErhpGP qs|˚ Ñ 0 “ 1.
(E.254)
P PP

hPBL1

[61]

For later use, we define the following set of sample paths, which has probability 1 uniformly in P P P.
!
)
A ” x8 P X 8 : sup |EM rhpGbn q|X 8 “ x8 s ´ ErhpGP qs|˚ Ñ 0 .

(E.255)

hPBL1

Note that Gbn,j and Gbn,j are related to each other by the following relationship:
ˆ
˙
σP,j pθq
b
b
b
Gn,j pθq ´ Gn,j pθq “ Gn,j pθq
´ 1 “ Gbn,j pθqηn,j pθq, θ P Θ.
σ̂n,j pθq
By Assumptions 4.1, 4.2, and 4.5, Lemma E.10 applies. Hence,
´
¯
inf P 8 sup |ηn,j pθq|˚ Ñ 0 “ 1.
P PP

(E.256)

(E.257)

θPΘ

Define the following set of sample paths:
!
)
B ” x8 P X 8 : sup |ηn,j pθq|˚ Ñ 0, @j “ 1, ¨ ¨ ¨ , J .

(E.258)

θPΘ

For any x8 P A X B, it then follows that
ˇ
ˇ˚
sup ˇEM rhpGbn q|X 8 “ x8 s ´ ErhpGP qsˇ Ñ 0,

(E.259)

hPBL1

due to (E.254) and (E.256), h being Lipschitz, Gbn,j being bounded (given x8 ), and supθPΘ |ηn,j pθq|˚ Ñ 0 for all
j. Finally, note that inf P PP P 8 pA X Bq “ 1 due to (E.254), (E.257), and De Morgan’s law. This establishes the
conclusion of the lemma.
The following lemma shows that, for almost all sample path x8 , one can find an almost sure representation of
the bootstrapped empirical process that is convergent.
Lemma E.16: Suppose Assumptions 4.1, 4.2, and 4.5 hold. Then, for each x8 P X 8 , there exists a sequence
tG̃n,x8 P `pΘ, RJ q, n ě 1u and a random element G̃P,x8 P `pΘ, RJ q defined on some probability space pΩ̃, Ã, P̃q such
that
ż
ż
h ˝ gpx8 , mn qdQpmn q “ hpG̃n,x8 pω̃qqdP̃˚ pω̃q, @h P BL1
(E.260)
ż
ż
hpGP pωqqdP pωq “ hpG̃P,x8 pω̃qqdP̃˚ pω̃q, @h P BL1 ,
(E.261)
for all x8 P C for some set C Ă X 8 such that inf P PP P 8 pCq “ 1 and
´
(¯
P̃´as˚
inf P 8 x8 P X 8 : G̃n,x8 Ñ G̃P,x8
“ 1.

(E.262)

Proof. Define the following set of sample paths:
!
)
C ” x8 P X 8 : sup |EM rhpGbn,j q|X 8 “ x8 s ´ ErhpGP qs|˚ Ñ 0 .

(E.263)

P PP

hPBL1

By Lemma E.15, inf P PP P 8 pCq “ 1.
For each fixed sample path x8 P C, consider the bootstrap empirical process gpx8 , Mn q in (E.249). This is a
random element in `8 pΘ, RJ q with a law governed by Q. For each x8 P C, by Lemma E.15,
ˇż
ˇ˚
ˇ
ˇ
sup ˇˇ h ˝ gpx8 , mn qdQpmn q ´ ErhpGP qsˇˇ Ñ 0.
(E.264)
hPBL1

[62]

Hence, by Theorem 1.10.4 in van der Vaart and Wellner (2000), for each x8 P C, one may find an almost sure
representation G̃n,x8 of gpx8 , Mn q on some probability space pΩ̃, Ã, P̃q such that
ż
ż
h ˝ gpx8 , mn qdQpmn q “ hpG̃n,x8 pω̃qqdP̃˚ pω̃q, @h P BL1 .
(E.265)
In particular, the proof of Theorem 1.10.4 in van der Vaart and Wellner (2000) (see also Addendum 1.10.5) allows
us to take G̃n,x8 to be defined for each ω̃ P Ω̃ as
G̃n,x8 pω̃q “ gpx8 , Mn pφn pω̃qqq,

(E.266)

for some perfect map φn : Ω̃ Ñ Z (see the construction of φα in the middle of page 61 in VW). One may define
G̃n,x8 arbitrarily for any x8 R C. The almost sure representation G̃P,x8 of GP,j is defined similarly.
By Theorem 1.10.4 in van der Vaart and Wellner (2000), Eq. (E.259), and inf P PP P pCq “ 1, it follows that
´
(¯
P̃´as˚
“ 1.
(E.267)
inf P 8 x8 P X 8 : G̃n,x8 Ñ G̃P,x8
P PP

This establishes the claim of the lemma.
Lemma E.17: Suppose Assumptions 4.1, 4.2, and 4.5 hold. Let Wn ” pGbn , Yn q be a sequence in W ” `pΘ, RJ qˆ
R such that Yn “ g̃pX 8 , Mn q for some map g̃ : X 8 ˆ Z Ñ RdY and
˘
`
(E.268)
inf P 8 sup |EM rhpWn q|X 8 “ x8 s ´ ErhpW qs|˚ Ñ 0 “ 1,
dY

P PP

hPBL1

where W “ pG, Y q is a Borel measurable random element in W.
˚
˚
Then, for each x8 P X 8 , there exists a sequence tWn,x
8 P W, n ě 1u and a random element Wx8 P W defined
on some probability space pΩ̃, Ã, P̃q such that
ż
8
8
˚
˚
EM rhpWn q|X “ x s “ hpWn,x
(E.269)
8 pω̃qqdP̃ pω̃q, @h P BL1
ż
ErhpW qs “ hpWx˚8 pω̃qqdP̃˚ pω̃q, @h P BL1 ,
(E.270)
for all x8 P C for some set C Ă X 8 such that inf P PP P 8 pCq “ 1, and
´
(¯
P̃´as˚
˚
inf P 8 x8 P X 8 : Wn,x
Ñ W̃x˚8
“ 1.
8
P PP

(E.271)

Proof. Let C ” tx8 : suphPBL1 |EM rhpWn q|X 8 “ x8 s ´ ErhpW qs|˚ Ñ 0u. The rest of the proof is the same as the
one for Lemma E.16 and is therefore omitted.
Remark E.1: When called by the Lemmas in Appendix E, Lemma E.17 is applied, for example, with Yn “
pvecpD̂n pθn1 qq, ξˆn pθn1 qq and Y “ pvecpDq, π1 q.

Appendix F

Further Comparison of Calibrated Projection and BCSProfiling

We next show that finite sample power can be higher with calibrated projection than with BCS-profiling, and
that, due to the slow rate at which κn diverges, this effect can be large in samples of considerable size. Thus, the
approaches are not nested in terms of power in empirically relevant examples. We then provide an example where

[63]

all of calibrated projection, BCS-profiling and the method of Pakes, Porter, Ho, and Ishii (2011) fail in a specific
instance where Assumption 4.3 is not satisfied.

F.1

Finite Sample Comparison in a Specific Example

We next analyze a stylized example of one-sided testing when the support set in direction p is a singleton identified
as the intersection of d moment inequalities with regular geometry. In this example, calibrated projection has more
power (less false coverage) than BCS-profiling, and the numerical difference can be large. The example resembles
empirically important cases, namely polyhedral identified sets with large interior, e.g. linear regression with interval
outcome data; recall that by Theorem 4.3, the two-sided testing problem reduces to two one-sided ones in these
cases. At the same time, we emphasize that other examples will go the other way, especially as the present example
utilizes the simplifications from Theorem 4.3 and therefore has no ρ-box.
Let θ be partially identified by moment conditions
EP pz j1 θ ´ Xj q ď 0, j “ 1, . . . , d.
Note that to simplify the analysis, we assume exactly d conditions. Assume that tz 1 , . . . , z d u are linearly independent
and also that p is in their positive span, so that ΘI is bounded in direction p but not ´p. The confidence intervals
will be accordingly one-sided. Since gradients are known, all simplifications from Theorem 4.3 apply. We borrow
from algebra in the proof of Theorem 4.4 to observe that, with the simplifications in place, CIn and CInprof invert
tests that use the same test statistic but different bootstrap approximations to its distribution as follows:
(
TnDR “ max Gbn,j
j
$
,
’
/
’
/
?
&
j1 ˚
j1 .
´
X̄
z
θ̂
n
z
λ
j
p,s
n
b
PR
Tn psn q “ min
max Gn,j `
`
p1 λ“0 j ’
κn
σ̂n,j
σ̂n,j /
’
/
loooooooooomoooooooooon
%
ą0

Tnb

$
,
’
/
’
/
? j1 ˚
&
n z θ̂p ´ X̄j
z j1 λ .
b
“ min
max Gn,j `
`
ď mintTnDR , TnP R psn qu,
/
p1 λ“0 j ’
κ
σ̂
σ̂
n
n,j
n,j
’
/
loooooooomoooooooon
%
“0

where (as in Theorem 4.3) sn is the value of p1 θ being tested and θ̂p,sn minimizes the sample criterion subject to
p1 θ “ sn . The last inequality is strict unless the problem defining Tnb is solved by λ “ 0. The assessments of
intercept terms in TnP R psn q and Tnb use that by construction of the example, all sample constraints bind at θ̂p˚ and
˚
are violated at θ̂p,s
(else, the test statistic would be 0 and the critical value not computed). Equality thus requires
n
knife-edge realizations of Gbn,j , so its probability vanishes as Gbn,j approaches multivariate normality and is in fact
0 for typical empirical samples. We conclude that the calibrated projection CIn is deterministically a weak (and
essentially always a strict) subset of the BCS-profiling CInprof in this example.
We next provide a numerical comparison in a further stripped-down version of the example. Thus, consider
one-sided testing with moment conditions
´θ1 ` θ2 ´ EP pX1 q

ď 0

θ1 ` θ2 ´ EP pX2 q

ď 0

where the data are pX1 , X2 q „ N ppEP pX1 q, EP pX2 qq, I2 q and EP pX1 q “ EP pX2 q “ 0. All of these facts other than

[64]

EP pX1 q “ EP pX2 q “ 0, but including the gradients and variance matrix, are known. This enables closed form
arguments. Also, for a researcher knowing this, the natural bootstrap implementation is a parametric bootstrap:
` b b˘
X1 , X2 „ N ppX̄1 , X̄2 q, I2 q
?
ùñ
npX̄1b ´ X̄1 , X̄2b ´ X̄1 q “ pZ1 , Z2 q „ N p0, I2 q
which we will use, i.e. pZ1 , Z2 q will take the role of pGbn,1 , Gbn,2 q. Numerical computations refer to α “ 5%.
Let p “ p0, 1q. We construct one-sided confidence intervals for spp, ΘI pP qq. All intervals contain p´8, spp, Θ̂I qs,
X̄2
. Also noting that in this example spp, ΘI pP qq “ 0 and, for sn ą spp, Θ̂I q,
and simple algebra shows spp, Θ̂I q “ X̄1 `
2
"ˆ
˙*
´X̄1 ` X̄2 X̄1 ` X̄2
,
Hpp, Θ̂I q “
2
2
"
* "ˆ
˙*
´X̄1 ` X̄2
1
Θ̂I psn q ”
θ P Θ : p θ “ sn , Qn pθq ď
inf
Qn pθq “
, sn
θPΘ:p1 θ“sn
2
*
"
?
X̄1 ` X̄2
,0 ,
n max sn ´
Tn psn q “
2
?
where Qn pθq “ maxj“1,...,J1 p nm̄n,j pθq{σ̂n,j pθqq` , we compute
˘ ? `
˘ (
? ` b
TnDR “
min max
n X̄1 ´ X̄1 , n X̄2b ´ X̄2 , 0
θPΘ̂I psn q

˘ ? `
˘ (
? ` b
n X̄1 ´ X̄1 , n X̄2b ´ X̄2 , 0 „ maxtZ1 , Z2 , 0u
˘
˘ ? `
˘
˘ (
? ` b
? `
? `
min max
n X̄1 ´ X̄1 ` κn´1 n ´θ1 ` sn ´ X̄1 , n X̄2b ´ X̄2 ` κ´1
n θ1 ` sn ´ X̄2 , 0 .
n

“ max
TnP R psn q

“

θ1 PR

Unless its value is 0, the minimization problem defining TnP R psn q is solved by setting two terms equal:
˘ ? `
˘
˘
? `
? ` b
n X̄1 ´ µ̂1 ´ n X̄2b ´ X̄2 ` κ´1
n X̄2 ´ X̄1
n
,
θ1 “
?
2κ´1
n
n
leading to
TnP R psn q

“
“

#? `
˘ ? `
˘
ˆ
˙ +
?
n X̄1b ´ X̄1 ` n X̄2b ´ X̄2
X̄1 ` X̄2
´1
max
` κn n sn ´
,0
2
2
"
ˆ
˙ *
"
*
?
Z1 ` Z2
X̄1 ` X̄2
Z1 ` Z2
´1
max
` κ´1
n
s
´
,
0
“
max
`
κ
T
ps
q,
0
.
n
n n
n
n
2
2
2

Finally, very similar reasoning to the above gives
"
ˆ
˙
˘
? ` b
?
X̄1 ´ X̄2
X̄1 ` X̄2
b
´1
Tn “ min max
n X̄1 ´ X̄1 ` κn n min
`
´ X̄1 , 0 ´ λ,
λPR
2
2
ˆ
˙
*
˘
? ` b
?
´X̄1 ` X̄2
X̄1 ` X̄2
n X̄2 ´ X̄2 ` κ´1
n
min
`
´
X̄
,
0
`
λ,
0
2
n
2
2
˘
˘
(
? ` b
? ` b
“ min max
n X̄1 ´ X̄1 ´ λ, n X̄2 ´ X̄2 ` λ, 0
λPR
#? `
˘ ? `
˘ +
n X̄1b ´ X̄1 ` n X̄2b ´ X̄2
“ max
,0
2
*
"
Z1 ` Z2
“ max
,0 .
2
?
Thus calibrated projection yields a critical value of ĉn “ Φ´1 p1 ´ αq{ 2 « 1.16, whereas simple projection uses

[65]

Table F.1: Finite sample noncoverage rates in a specific example.

Power at γn´1{2 , γ “ ...
0
1
2
3
4

Type of cv

n

Value

ĉproj
n

any

1.95

.003

.089

.523

.930

.998

c̃prof
n
prof
c̃n
c̃prof
n
c̃prof
n
c̃prof
n
c̃prof
n
c̃prof
n
c̃prof
n
c̃prof
n
c̃prof
n
c̃prof
n

103
105
107
109
1011
1013
1015
1017
1019
1050
10100

1.63
1.52
1.47
1.43
1.40
1.38
1.37
1.36
1.35
1.28
1.24

.011
.016
.019
.022
.024
.025
.026
.027
.028
.036
.039

.188
.231
.254
.271
.284
.292
.299
.307
.313
.348
.366

.701
.751
.774
.790
.800
.807
.813
.819
.823
.847
.858

.974
.982
.985
.987
.988
.989
.989
.990
.990
.993
.994

1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000
1.000

ĉn

any

1.16

.050

.409

.882

.995

1.000

?
“ Φ´1 p 1 ´ αq « 1.95; both are independent of sn as well as n. BCS-profiling uses a critical value ĉprof
psn q
ĉproj
n
n
that increases in the test statistic (hence, conditional on the data, in sn ) because the statistic itself enters TnP R . To
psn q. BCS-profiling is equivalent to
facilitate a comparison, one can compute the fixed point at which Tn psn q “ ĉprof
n
comparing Tn psn q to that fixed point at all sn , and we will therefore equate it with use of this critical value, labeled
c̃prof
below. This critical value converges to ĉn at a rate of κ´1
n
n , illustrating asymptotic equivalence of inference
?
methods off the null in this case. However, for the popular choice of κn “ log n, convergence is so slow that it
should not be taken to describe behavior at realistic sample sizes. Table F.1 displays the numerical value of c̃prof
n
?
and the implied noncoverage probability (or power) at γ{ n for γ P t0, 1, 2, 3, 4u; note that γ “ 0 corresponds to the
true support function. By construction, c̃prof
interpolates between ĉproj
and ĉn in this example, but convergence
n
n
prof
to ĉn requires extreme sample sizes. For example, on the boundary edge of the true projection CI.95
has finite
sample coverage of .975, which is effectively halfway between projection and calibrated projection, for n “ 1013 .

F.2

Example of Methods Failure When Assumption 4.3 Fails

Consider one-sided testing with two inequality constraints in R2 . The constraints are
θ1 ` θ2

ď EP pX1 q

θ1 ´ θ2

ď EP pX2 q.

The projection of ΘI pP q in direction p “ p1, 0q is p´8, pEP pX1 q ` EP pX2 qq{2s, the support set is Hpp, ΘI q “
tppEP pX1 q`EP pX2 qq{2, pEP pX1 q´EP pX2 qq{2qu, and the support function takes value θ1˚ “ pEP pX1 q`EP pX2 qq{2.

[66]

The random variables pX1 , X2 q1 have a mixture distribution as follows:
ff¸
˜ «
$
«
ff ’
1
´1
& N 0,
with probability 1 ´ 1{n,
X1
„
´1 1
’
X2
%
δp1,1q (degenerate)
otherwise,
hence EP pX1 q “ EP pX2 q “ θ1˚ “ 1{n. Note in particular the implication that
#
0 with probability 1 ´ 1{n,
X1 ` X2
“
2
1 otherwise.
The natural estimator of θ1˚ is θ̂1˚ “ pX̄1 ` X̄2 q{2. It is distributed as Z{n, where Z is Binomial with parameters
p1{n, nq. For large n, the distribution of Z is well approximated as Poisson with parameter 1. In particular, with
probability approximately e´1 « 37%, every sample realization of pX1 `X2 q{2 equals zero. In this case, the following
happens: (i) The projection of the sample analog of the identified set is p´8, 0s, so that a strictly positive critical
value or level would be needed to cover the true projection. (ii) Because the empirical distribution of pX1 ` X2 q{2
is degenerate at zero, the distribution of pX̄1b ` X̄2b q{2 is as well. Hence, all of Pakes, Porter, Ho, and Ishii (2011),
Bugni, Canay, and Shi (2017), and calibrated projection (each with either parametric or nonparametric bootstrap)
compute critical values or relaxation levels of 0.
This bounds from above the true coverage of all of these methods at e´1 « 63%. Note that pm ă nq-subsampling
will encounter the same problem. Next we provide some discussion of the example.
Violation of Assumptions. The example violates our Assumption 4.3 because CovpX1 , X2 q Ñ 1. It also violates
Assumption 2 in Bugni, Canay, and Shi (2017): Their Assumption A2-(b) should apply, but the profiled test statistic
on the true null concentrates at 1{n. The example satisfies the assumptions explicitly stated in Pakes, Porter, Ho,
and Ishii (2011), illustrating an oversight in their Theorem 2. (We here refer to the inference part of their 2011
working paper. We identified corresponding oversights in the proof of their Proposition 6.)
The example satisfies the assumptions of Andrews and Soares (2010) and Andrews and Guggenberger (2009),
and both methods work here. The reason is that both focus on the distribution of the criterion function at a fixed
θ and are not affected by the irregularity of θ̂1˚ .
Relation to Mammen (1992). In this example, all of Bugni, Canay, and Shi (2017), Pakes, Porter, Ho, and Ishii
(2011), and our calibrated projection method reduce to one-sided nonparametric percentile bootstrap confidence
intervals for pEP pX1 q`EP pX2 qq{2 estimated by pX̄1 `X̄2 q{2. By Mammen (1992, Theorem 1), asymptotic normality
of an appropriately standardized estimator, i.e.
`
˘ d
Dtan u : an pX̄1 ` X̄2 q ´ pEP pX1 q ` EP pX2 qq Ñ N p0, 1q,
is necessary and sufficient for this interval to be valid. This fails (the true limit is recentered Poisson at rate an “ n),
so that validity of any of the aforementioned methods would contradict the Theorem.

Appendix G

Comparison with Projection of AS

In this Appendix we establish that for each n P N, CIn is a subset of a confidence interval obtained by projecting an
AS confidence set and denoted CInproj .50 Moreover, we derive simple conditions under which our confidence interval
50
Of course, AS designed their confidence set to uniformly cover each vector in ΘI with prespecified asymptotic probability,
a different inferential problem than the one considered here.

[67]

is a proper subset of the projection of AS’s confidence set. Below we let ĉproj
denote the critical value obtained applyn
?
?
ing AS with criterion function Qn pθq “ max tmaxj“1,...,J1 p nm̄n,j pθq{σ̂n,j pθqq` , maxj“J1 `1,¨¨¨ ,J1 `J2 | nm̄n,j pθq{σ̂n,j pθq|u
and with the same choice as for ĉn of GMS function ϕ and tuning parameter κn .
Theorem G.1: Suppose Assumptions 4.1, 4.2, 4.3, 4.4, and 4.5 hold. Let 0 ă α ă 1{2. Then for each n P N
CIn Ď r´sp´p, Cn pĉproj
qq, spp, Cn pĉproj
qqs,
n
n

(G.1)

where for given function c, Cn pcq is defined in (1.1)
Proof. For given θ, the event
max

j“1,...,J

!
)
Gbn,j pθq ` ϕj pξˆn,j pθqq ď c

(G.2)

implies the event
max

λPΛbn pθ,ρ,cq

p1 λ ě 0 ě

min

λPΛbn pθ,ρ,cq

p1 λ,

(G.3)

!
)
with Λbn defined in (3.1). This is so because if maxj“1,...,J Gbn,j pθq ` ϕj pξˆn,j pθqq ď c, λ “ 0 is feasible in both
optimization problems in (G.3), hence the event in (G.3) is implied. In turn this yields that for each n P N and
θ P Θ,
cproj
pθq ě ĉn pθq,
(G.4)
n
and therefore the result follows.
The result in Theorem G.1 is due to the following fact. Recall that AS’s confidence region calibrates its critical
value so that, at each θ, the following event occurs with probability at least 1 ´ α:
!
)
max Gbn,j pθq ` ϕj pξˆn,j pθqq ď c.
(G.5)
j“1,...,J

A natural question is, then, whether there are conditions under which CIn is strictly shorter than the projection
of AS’s confidence region. Heuristically, this is the case with probability approaching 1 when ĉn pθq is strictly less
than ĉproj
pθq at each θ that is relevant for projection. For this, restrict ϕp¨q to satisfy ϕj pxq ď 0 for all x, fix θ and
n
consider the pointwise limit of (G.5):
GP,j pθq ` ζP,j pθq ď c, j “ 1, ¨ ¨ ¨ , J,

(G.6)

?
where tGP,j pθq, j “ 1, ¨ ¨ ¨ , Ju follows a multivariate normal distribution, and ζP,j pθq ” p´8q1p nγ1,P,j pθq ă 0q is
the pointwise limit of ϕj pξˆn,j pθqq (with the convention that p´8q0 “ 0). Under mild regularity conditions, ĉproj
pθq
n
then converges in probability to a critical value c “ cproj pθq such that (G.6) holds with probability 1 ´ α. Similarly,
the limiting event that corresponds to our problem (3.4) is
Λpθ, ρ, cq X tp1 λ “ 0u ‰ H,

(G.7)

where the limiting feasibility set Λpθ, ρ, cq is given by
d
Λpθ, ρ, cq “ tλ P ρBn,ρ
: GP,j pθq ` DP,j pθqλ ` ζP,j pθq ď c, j “ 1, ¨ ¨ ¨ , Ju.

(G.8)

Note that if the gradient DP,j pθq is a scalar multiple of p, i.e. DP,j pθq{}DP,j pθq} P tp, ´pu, for all j such that
ζP,j pθq “ 0, the two problems are equivalent because (G.6) implies (G.7) (by arguing that λ “ 0 is in Λpθ, ρ, cq),
and for the converse implication, whenever (G.7) holds, there is λ such that GP,j pθq ` DP,j pθqλ ` ζP,j pθq ď c and

[68]

p1 λ “ 0. Since DP,j pθqλ “ 0 for all j such that ζP,j pθq “ 0, one has GP,j pθq ` ζP,j pθq ď c for all j.51 In this special
case, the limits of the two critical values coincide asymptotically, but any other case is characterized by projection
conservatism. Lemma G.1 below formalizes this insight. Specifically, for fixed θ, the limit of ĉn pθq is strictly less
than the limit of ĉproj
pθq if and only if there is a constraint that binds or is violated at θ and has a gradient that
n
is not a scalar multiple of p.52
The parameter values that are relevant for the lengths of the confidence intervals are the ones whose projections
are in a neighborhood of the projection of the identified set. Therefore, a leading case in which our confidence
interval is strictly shorter than the projection of AS asymptotically is that in which at any θ (in that neighborhood
of the projection of the identified set) at least one local-to-binding or violated constraint has a gradient that is not
parallel to p. We illustrate this case with an example based on Manski and Tamer (2002).
Example G.1 (Linear regression with an interval valued outcome): Consider a linear regression model:
ErY |Zs “ Z 1 θ,

(G.9)

where Y is an unobserved outcome variable, which takes values in the interval rYL , YU s with probability one, and
YL , YU are observed. The vector Z collects regressors taking values in a finite set SZ ” tz1 , ¨ ¨ ¨ , zK u, K P N. We
then obtain the following conditional moment inequalities:
EP rYL |Z “ zj s ď zj1 θ ď EP rYU |Z “ zj s, j “ 1, ¨ ¨ ¨ , K,
which can be converted into unconditional moment inequalities with J1 “ 2K and
$
&Y 1tZ “ z u{gpz q ´ z 1 θ,
j “ 1, ¨ ¨ ¨ , K
L
j
j
j
mj pX, θq “
%z 1 θ ´ YU 1tZ “ zj´K u{gpzj´K q j “ K ` 1, ¨ ¨ ¨ , 2K,
j´K

(G.10)

(G.11)

where g denotes the marginal distribution of Z, which is assumed known for simplicity. Consider making inference
for the value of the regression function evaluated at a counterfactual value z̃ R SZ . Then, the projection of interest
is z̃ 1 θ. Note that the identified set is a polyhedron whose gradients are given by DP,j pθq “ ´zj {σP,j , j “ 1, ¨ ¨ ¨ , K
and DP,j pθq “ zj´K {σj´K , j “ K ` 1, ¨ ¨ ¨ , 2K. This and z̃ R SZ imply that for any θ not in the interior of the
identified set, there exists a binding or violated constraint whose gradient is not a scalar multiple of p. Hence, for
pθq asymptotically. In this case, our confidence interval
all such θ, our critical value is strictly smaller than cproj
n
becomes strictly shorter than that of AS asymptotically. We also note that the same argument applies even if the
marginal distribution of Z is unknown. In such a setting, one needs to work with a sample constraint of the form
řn
řn
n´1 i“1 YL,i 1tZi “ zj u{n´1 i“1 1tZi “ zj u ´ zj θ (and similarly for the upper bound). This change only alters
the (co)variance of the Gaussian process in our limiting approximation but does not affect any other term.
We now provide a numerical illustration for a further simplified example. Assume that p “ pd´1{2 , ..., d´1{2 q P Rd
and that there are d binding moment inequalities whose gradients are known and correspond to rows of the identity
matrix. Assume furthermore that G is known to be exactly d-dimensional multivariate standard Normal. (Thus,
ΘI is the negative quadrant. Its unboundedness from below is strictly for simplicity.) Also, by Theorem 4.3, one
can set ρ “ `8 in this example.
51

The gradients of the non-binding moment inequalities do not matter here because GP,j pθq ` ζP,j pθq ď c holds due to
ζP,j pθq “ ´8 for such constraints.
52
The condition that all binding moment inequalities have gradient collinear with p is not as exotic as one might think. An
important case where it obtains is the “smooth maximum,” i.e. the support set is a point of differentiability of the boundary
of ΘI .

[69]

Table G.1: Conservatism from projection in a one-sided testing problem as a function of d

d
ĉn
ĉproj
n
1 ´ α˚

1
1.64
1.64
.95

2
1.16
1.95
.77

3
0.95
2.12
.57

4
0.82
2.23
.40

5
0.74
2.32
.27

6
0.67
2.39
.18

7
0.62
2.44
.11

8
0.58
2.49
.07

9
0.55
2.53
.04

10
0.52
2.57
.03

100
0.16
3.28
10´25

8
0
8
0

Under these simplifying assumptions (which can, of course, be thought of as asymptotic approximations), it is
easy to calculate in closed form that
ĉn

“

ĉproj
n

“

d´1{2 Φ´1 p1 ´ αq,
´
¯
Φ´1 p1 ´ αq1{d .

Furthermore, for any α ă 1{2, one can compute α˚ s.t. applying ĉn with target coverage p1 ´ αq yields the same
confidence interval as using ĉproj
with target coverage p1 ´ α˚ q.53 Some numerical values are provided in Table G.1
n
(with α “ 0.05).
To cover p1 θ in R10 with probability 95%, it suffices to project an AS-confidence region of size 3%. The example
is designed to make a point; our Monte Carlo analyses in Section 5 showcase less extreme cases. However, the core
defining feature of the example – namely, the identified set has a thick interior, and the support set is the intersection
of d moment inequalities – frequently occurs in practice, and all such examples will qualitatively resemble this one
as d grows large.

G.1

Necessary and Sufficient Condition for ĉn pθq ă ĉproj
n pθq

The following lemma establishes the effect of ρ on ĉn pθq. In doing so it establishes a necessary and sufficient
pθq, because the latter can be seen as the former calibrated with ρ set equal to zero. The
condition for ĉn pθq ă ĉproj
n
lemma requires ϕj pxq ď 0 for all x.54
Lemma G.1: Fix θ P Θ, P P P and a value ρ P R` . Suppose Assumptions 4.1, 4.2, 4.3, 4.4 and 4.5 hold and
also that ϕj pxq ď 0 for all x and j. Let 0 ă δ ă ρ. For n ě N , let ĉn pθq be calibrated using ρ in place of ρ, which
necessarily yields a larger value for ĉn pθq. With a modification of notation, explicitly highlight ĉn pθq’s dependence
on ρ through the notation ĉn pθ, ρq. Then
p

|ĉn pθ, ρq ´ ĉn pθ, ρ ´ δq| Ñ 0

(G.12)

if and only if DP,j pθq{}DP,j pθq} P tp, ´pu for all j P J ˚ pθq ” tj : EP rmj pXi , θqs ě 0u.
p

p

Remark G.1: For θ such that J ˚ pθq “ H, we have ĉn pθ, ρq Ñ 0 but also ĉproj
pθq Ñ 0. This is consistent with
n
Lemma G.1 because the condition on gradients vacuously holds in this case.
53

Equivalently, p1´α˚ q is the probability that Cn pĉproj
q contains t0u, the true support set in direction p which furthermore,
n
in this example, minimizes coverage within ΘI pP q. The closed-form expression is 1 ´ α˚ “ Φpd´1{2 Φ´1 p1 ´ αqqd . AS prove
validity of their method only for α ă 1{2, but this is not important for the point made here.
54
To keep the treatment general, we have not imposed this restriction throughout the paper. However, we only recommend
functions ϕj with this feature anyway: for any ϕj that can take strictly positive values, substituting mintϕj pxq, 0u attains
the same asymptotic size but generates CIs that are weakly shorter for all and strictly shorter for some sample realizations.

[70]

Proof. Recall that θ and P are fixed, i.e. we assume a pointwise perspective. Then
p

d
ĉn pθ, ρq Ñ inftc ě 0 : P ptλ P ρBn,ρ
: GP,j pθq ` DP,j pθqλ ď c, j P J ˚ pθqu X tp1 λ “ 0u ‰ Hq ě 1 ´ αu.

(G.13)

Here, we used convergence of Gbj pθq to GP,j pθq and of D̂j pθq to DP,j pθq, boundedness of gradients, and the fact that
$
&0
?
if j P J ˚ pθq
p
ϕj pκ´1
n
m̄
pX
,
θq{σ
pθqq
Ñ
(G.14)
j
i
P,j
n
%´8 otherwise,
where the first of those cases uses nonpositivity of ϕj . It therefore suffices to show that the right hand side of G.13
strictly decreases in ρ if and only if the conditions of the Lemma hold.
To simplify notation, henceforth omit dependence of GP,j pθq, DP pθq, and J ˚ pθq on P and θ. Define the J
vector e to have elements ej “ c ´ Gj , j “ 1, . . . , J. Suppose for simplicity that J ˚ contains the first J ˚ inequality
˚
constraints. Let er1:J s denote the subvector of e that only contains elements corresponding to j P J ˚ , define
˚
Dr1:J ,:s correspondingly, and write
»
fi
»
fi
»
fi
˚
˚
Dr1:J ,:s
er1:J s
0 ¨ 1J ˚
—
ffi
—
ffi
—
ffi
—
ffi
— ρ ¨ 1d ffi
— 1d ffi
Id
—
ffi
—
ffi
—
ffi
ffi , g “ — ρ ¨ 1d ffi , τ “ — 1d ffi .
K“—
— ´Id
ffi
—
ffi
—
ffi
—
ffi
—
ffi
—
ffi
p1
0
0
–
fl
–
fl
–
fl
1
´p
0
0
where Id denotes the d ˆ d identity matrix. By Farkas’ Lemma (Rockafellar, 1970, Theorem 22.1), the linear system
˚
Kλ ď g has a solution if and only if for all µ P RJ` `2d`2 ,
µ1 K “ 0 ñ µ1 g ě 0.

(G.15)
˚

To further simplify expressions, fix p “ r1 0 . . . 0s. Let M “ tµ P RJ`

`2d`2

: µ1 K “ 0u.

Step 1. This step shows that
d
P ptλ P ρBn,ρ
: GP,j ` DP,j λ ď c, j P J ˚ u X tp1 λ “ 0u ‰ Hq
d
ą P ptλ P pρ ´ δqρBn,ρ
: GP,j ` DP,j λ ď c, j P J ˚ u X tp1 λ “ 0u ‰ Hq

(G.16)

if and only if the condition on gradients holds. This is done by showing that
P ptµ1 g ě 0 @µ P Mu X tµ1 g ´ δτ ă 0 Dµ P Muq ą 0.
under that same condition. The event tµ1 g ě 0 @µ P Mu obtains if and only if
(
min
µ1 g : µ1 K “ 0 ě 0
˚
µPRJ
`

(G.17)

(G.18)

`2d`2

and analogously for µ1 pg ´ δτ q ě 0. The values of these programs are not affected by adding a constraint as follows:
#
+
min

˚ `2d`2
µPRJ
`

µ1 g : µ1 K “ 0, µ P arg

min

J ˚ `2d`2
µ̃PR`

pµ̃1 g : µ̃r1:J

˚

s

˚

“ µr1:J s , µ̃1 K “ 0q ,

(G.19)

That is, we can restrict attention to a concentrated out subset of vectors µ, where the last p2d ` 2q components of
any µ minimize the objective function among all vectors that agree with µ in the first J ˚ components. The inner

[71]

minimization problem in equation (G.19) can be written as
»
µ̃J ˚ `1 ´ µ̃J ˚ `d`1 ` µ̃J ˚ `2d`1 ´ µ̃J ˚ `2d`2
—
J ˚ÿ
`2d
— µ̃J ˚ `2 ´ µ̃J ˚ `d`2
— .
µ̃
s.t.
min
ρ
j
— .
2d`2
µ̃rJ ˚ `1:J ˚ `2d`2s PR`
– .
˚
j“J `1
µ̃J ˚ `d ´ µ̃J ˚ `2d

fi
ffi
ffi
ffi “ ´µr1:J ˚ s1 Dr1:J ˚ ,:s .
ffi
fl
(G.20)

Thus, the solution of the problem is uniquely pinned down as
fi
0
”
ı
ffi
—
˚
— ´ Dr1:J ,2:ds1 µr1:J ˚ s ^ 0 ¨ 1d´1 ffi
ffi
—
ffi
—
ffi
—
0
ffi ,
“—
˚
ffi
— Dr1:J ,2:ds1 µr1:J ˚ s _ 0 ¨ 1
d´1
ffi
—
”
ı
ffi
—
˚
˚
r1:J ,1s1 µr1:J s ^ 0
ffi
—
´
D
fl
–
˚
˚
1:J
,1
1
r1:J s
r
s
D
µ
_0
»

µrJ

˚

˚

`1:J ˚ `2d`2s

(G.21)

˚

where Dr1:J ,2:ds1 µr1:J s _ 0 ¨ 1d´1 indicates a component-wise comparison. Now we consider the following case
distinction:
˚
˚
˚
Case (i). If Dj {}Dj } P tp, ´pu for all j P J ˚ , then µr1:J s1 D “ pµr1:J s1 Dr1:J ,1s , 0, ..., 0q1 and therefore all
˚
˚
but the last two entries of µrJ `1:J `2d`2s equal zero. One can, therefore, restrict attention to vectors µ with
˚
˚
µrJ `1:J `2ds “ 0. But for these vectors, µ1 τ “ 0 and so the programs we compare necessarily have the same value.
The probability in equation (G.17) is therefore zero.
Case (ii). Suppose that at least one row of D, say its first row (though it can be one direction of an equality
constraint), is not collinear with p, so that }Dr1,2:ds } ‰ 0.
Let
»
fi
1
—
ffi
0 ¨ 1J ˚ ´1
—
ffi
—
ffi
—
ffi
0
— “
ffi
‰
—´ pDr1,2:ds1 q ^ 0 ¨ 1
ffi
—
d´1 ffi
$“—
(G.22)
ffi
—
ffi
0
—
ffi
— pDr1,2:ds1 q _ 0 ¨ 1
ffi
d´1 ffi
—
“ r1,1s
‰
—
ffi
–
fl
´ pD
q^0
r1,1s
pD
q_0
˚

˚

and note that $rJ `1:J `2ds ‰ 0, hence $1 τ ą 0.
As in the proof of Lemma E.6, the set M can be expressed as positive span of a finite, nonstochastic set of
˚
affinely independent vectors ν t P RJ` `2d`2 that are determined only up to multiplication by a positive scalar.
All of these vectors have the “concentrated out structure” in equation (G.21). But then $ must be one of them
˚
because it is the unique concentrated out vector with $r1:J s “ p1, 0, ..., 0q1 , and p1, 0, ..., 0q1 cannot be spanned by
nonnegative J ˚ -vectors other than positive multiples of itself.
We now establish positive probability of the event
ν t1 g

ě

0, all ν t

ν t1 pg ´ δτ q

ă

0, some ν t

[72]

by observing that if we define
»
—
—
—
—
ιk “ —
—
—
—
–

ˇ
řd ˇ
´ρ ¨ i“2 ˇDr1,is ˇ
k ¨ 1J ˚ ´1
ρ ¨ 1d
ρ ¨ 1d
0
0

fi
ffi
ffi
ffi
ffi
ffi ,
ffi
ffi
ffi
fl

(G.23)

then we have
0 “ $1 ιk “ min ν t1 ιk .
t

˚

Any other spanning vector ν t will not have $r2:J s “ 0 and so for any such vector, ν t1 ιk strictly increases in k. As
there are finitely many spanning vectors, all of them have strictly positive inner product with ιk if k is chosen large
enough.
A realization of g “ ιk would, therefore, yield
ν t1 g ě 0 @ν t P M, and $t1 pg ´ δτ q ă ´,

(G.24)

Γk “ tι : ι “ ιk ` {2b, }b} ď 1 and $1 b ą 0u.

(G.25)

ν t1 ι ě 0 @ν t P M, and $t1 pι ´ δτ q ă ´{2, @ι P Γk .

(G.26)

for some  ą 0. Let
Then

The probability in equation (G.17) is therefore strictly positive.
Step 2. Next, we argue that
d
P ptλ P ρBn,ρ
: Gj ` Dj λ ď c, j P J ˚ u X tp1 λ “ 0u ‰ Hq

(G.27)

strictly continuously increases in c. The rigorous argument is very similar to the use of Farkas’ Lemma in step 1
and in Lemma E.6. We leave it at an intuition: As c increases, the set of vectors g fulfilling the right hand side of
(G.15) strictly increases, hence the set of realizations of Gj that render the program feasible strictly increases, and
Gj has full support.
Step 3. Steps 1 and 2 imply that
d
inf tP ptλ P ρBn,ρ
: Gj ` Dj λ ď c, j P J ˚ u X tp1 λ “ 0u ‰ Hq ě 1 ´ αu

cě0

d
ą inf tP ptλ P pρ ´ δqρBn,ρ
: Gj ` Dj λ ď c, j P J ˚ u X tp1 λ “ 0u ‰ Hq ě 1 ´ αu
cě0

(G.28)

and hence the result.

References
Adams, R. A., and J. J. Fournier (2003): Sobolev spaces, vol. 140. Academic press.
Andrews, D. W. (1994): “Chapter 37 Empirical process methods in econometrics,” Handbook of Econometrics,
4, 2247 – 2294.

[73]

Andrews, D. W. K., and P. Guggenberger (2009): “Validity of Subsampling and ’Plug-In Asymptotic’
Inference for Parameters Defined by Moment Inequalities,” Econometric Theory, 25(3), 669–709.
(2010): “Asymptotic Size and a Problem With Subsampling and With the m Out Of n Bootstrap,”
Econometric Theory, 26, 426–468.
Andrews, D. W. K., and G. Soares (2010): “Inference for Parameters Defined by Moment Inequalities Using
Generalized Moment Selection,” Econometrica, 78, 119–157.
Beresteanu, A., and F. Molinari (2008): “Asymptotic properties for a class of partially identified models,”
Econometrica, 76, 763–814.
Bontemps, C., T. Magnac, and E. Maurin (2012): “Set Identified Linear Models,” Econometrica, 80, 1129–
1155.
Brent, R. P. (1971): “An algorithm with guaranteed convergence for finding a zero of a function,” The Computer
Journal, 14(4), 422–425.
Bugni, F. A., I. A. Canay, and X. Shi (2015): “Specification tests for partially identified models defined by
moment inequalities,” Journal of Econometrics, 185(1), 259–282.
(2017): “Inference for subvectors and other functions of partially identified parameters in moment inequality
models,” Quantitative Economics, 8(1), 1–38.
Bull, A. D. (2011): “Convergence rates of efficient global optimization algorithms,” Journal of Machine Learning
Research, 12(Oct), 2879–2904.
Chernozhukov, V., H. Hong, and E. Tamer (2007): “Estimation and Confidence Regions for Parameter Sets
In Econometric Models,” Econometrica, 75, 1243–1284.
Ciliberto, F., and E. Tamer (2009): “Market Structure and Multiple Equilibria in Airline Markets,” Econometrica, 77, 1791–1828.
Davydov, Y. A., M. Lifshitz, and N. Smorodina (1995): Local properties of distributions of stochastic functionals. American Mathematical Society.
Dekker, T. (1969): “Finding a zero by means of successive linear interpolation,” Constructive aspects of the
fundamental theorem of algebra, pp. 37–51.
Horn, R. A., and C. R. Johnson (1985): Matrix Analysis. Cambdridge University Press.
Imbens, G. W., and C. F. Manski (2004): “Confidence Intervals for Partially Identified Parameters,” Econometrica, 72, 1845–1857.
Kaido, H., F. Molinari, and J. Stoye (2017): “Constraint qualifications in projection inference,” Work in
progress.
Mammen, E. (1992): When Does Bootstrap Work? Asymptotic Results and Simulations. Spinger Verlag, New
York, NY.

[74]

Manski, C. F., and E. Tamer (2002): “Inference on Regressions with Interval Data on a Regressor or Outcome,”
Econometrica, 70(2), 519–546.
Molchanov, I. (2005): Theory of Random Sets. Springer, London.
Narcowich, F., J. Ward, and H. Wendland (2003): “Refined error estimates for radial basis function interpolation,” Constructive approximation.
Pakes, A., J. Porter, K. Ho, and J. Ishii (2011): “Moment Inequalities and Their Application,” Discussion
Paper, Harvard University.
Pata, V. (2014): “Fixed Point Theorems and Applications,” Mimeo.
Rockafellar, R. T. (1970): Convex Analysis. Princeton University Press, Princeton.
Rockafellar, R. T., and R. J.-B. Wets (2005): Variational Analysis, Second Edition. Springer-Verlag, Berlin.
Steinwart, I., and A. Christmann (2008): Support vector machines. Springer Science & Business Media.
Stoye, J. (2009): “More on Confidence Intervals for Partially Identified Parameters,” Econometrica, 77, 1299–1315.
Tartar, L. (2007): An introduction to Sobolev spaces and interpolation spaces, vol. 3. Springer Science & Business
Media.
van der Vaart, A., and J. Wellner (2000): Weak Convergence and Empirical Processes: With Applications
to Statistics. Springer-Verlag, Berlin.
van der Vaart, A. W., and J. H. van Zanten (2008): “Reproducing kernel Hilbert spaces of Gaussian priors,”
in Pushing the limits of contemporary statistics: contributions in honor of Jayanta K. Ghosh, pp. 200–222.
Institute of Mathematical Statistics.

[75]

