arXiv:1705.00998v2 [stat.ME] 13 Feb 2018

Minimal Approximately Balancing Weights:
Asymptotic Properties and Practical Considerations
Yixin Wang
Department of Statistics
Columbia University
yixin.wang@columbia.edu
José R. Zubizarreta
Department of Health Care Policy
Department of Statistics
Harvard University
zubizarreta@hcp.med.harvard.edu
February 14, 2018
Abstract
In observational studies, sample surveys, and regression settings, weighting methods are
widely used to adjust for or balance observed covariates. Conventionally, weights are estimated
by fitting a model for the propensity score and then inverting the predicted propensities. Recently, several other methods have been proposed that instead focus on directly balancing the
covariates while minimizing the dispersion of the weights. In this paper, we call this general
class of weights minimal approximately balancing weights (MABW) and study their properties by establishing a connection with shrinkage estimation of the propensity score. This connection allows us to characterize the asymptotic properties of MABW by building on the large
sample results from propensity score estimation. In particular, we show that, under standard
technical conditions, MABW are consistent estimates of the true inverse probability weights.
To the best of our knowledge, this functional consistency of balancing weights has not been
established in the literature. We also show that the resulting weighting estimator is consistent,
asymptotically normal, and semiparametrically efficient. For applications, we present a finite
sample oracle inequality that shows that the loss incurred by balancing too many functions of
the covariates is limited in MABW. This is because the number of active balancing constraints
is implicitly bounded in MABW. We also provide an algorithm for choosing the degree of

1

approximate balancing in MABW. This is a general problem in weighting and thus our algorithm is of independent interest. Finally, we conclude with numerical results that suggest
approximate balancing is preferable to exact balancing, especially when there is limited overlap in covariate distributions. We illustrate this phenomenon in three empirical studies and
show that the root mean squared error of the weighting estimator can be reduced by nearly a
half as opposed to exact balancing when the data exhibits poor overlap.

Keywords: Causal Inference; Missing Data; Observational Study; Sample Surveys; Weighting.

1 Introduction
1.1 Weighting methods for covariate adjustment
In observational studies of causal effects, in sample surveys and panel data with unit non-response,
and in regression settings with missing and/or mismeasured covariates, weighting methods are
commonly used to adjust for observed covariates (e.g., Rosenbaum 1987; Robins et al. 1994;
Hirano et al. 2003). The great popularity of weighting methods is explained in part by the fact
that they do not require modeling the response surface of the outcome (Rosenbaum, 1987). Moreover, unlike standard regression methods, they do not require the outcomes for their adjustments.
As such, they are considered to be part of the design stage as opposed to the analysis stage of the
study, which promotes the objectivity of the study and preserves the validity of its statistical tests
(Rubin, 2008).
Conventionally, the weights are estimated by modeling the propensities of receiving treatment
or exhibiting missingness, and then inverting the predicted propensities. Although ubiquitous, in
practice it is often difficult to properly adjust for or balance the observed covariates. To address
this problem, a number of methods have been proposed recently. These methods take a different
perspective: instead of modeling the propensities of treatment or missingness, they directly balance the covariates. Some methods in addition minimize a measure of dispersion of the weights.
We call this class of weights minimal approximately balancing weights (MABW). Examples of
these methods are: Hainmueller (2012), Imai and Ratkovic (2014), Zubizarreta (2015), Chan et al.
(2016), Li et al. (2016), Zhao (2016), and Zhao and Percival (2017). Related methods that predate these balancing approaches include Deville and Särndal (1992) and Hellerstein and Imbens
(1999). Two promising approaches that use similar weights together with outcome information for
effect estimation are Athey et al. (2016) and Hirshberg and Wager (2017).
With the exception of Zubizarreta (2015), Athey et al. (2016), and Hirshberg and Wager (2017),
all the aforementioned methods balance covariates exactly rather than approximately. This is a subtle but important difference, because weights that satisfy the exact balancing constraints might not
exist at all. MABW uses box balancing constraints instead of equality balancing constraints to
decrease the sample size required for a feasible weighting solution. This allows us to balance more
functions of the covariates for a given sample size and is especially valuable in high-dimensional
2

settings (Athey et al., 2016; Hirshberg and Wager, 2017). While this approach has been shown to
work well in practice (Zubizarreta, 2015; Athey et al., 2016; Hirshberg and Wager, 2017), to the
best of our knowledge, the formal properties of MABW have not been established.

1.2 Asymptotic properties and practical considerations of minimal approximately balancing weights
In this paper, we study the general class of minimal approximately balancing weights (MABW).
Unlike Athey et al. (2016) and Hirshberg and Wager (2017), we focus on weighting estimators that
do not use outcome information and thus are ‘design-based’ in the sense of Rubin (2008). First,
we establish a connection between MABW and shrinkage estimation of the propensity score. We
find that the dual optimization problem of the MABW takes a similar form to that of parameter
estimation in generalized linear models with ℓ1 regularization under a particular loss function.
This allows us to establish the large-sample properties of MABW by leveraging the large sample
results of weights based on propensity score estimation. In particular, we show that under standard
technical conditions MABW are consistent estimates of the true inverse probability weights in
both the ℓ2 and ℓ∞ norm. To the best of our knowledge, this functional consistency of balancing
weights has not been established in the literature even for exact balancing.
Second, we show that the resulting weighting estimator is consistent, asymptotically normal,
and semiparametrically efficient. This result is related to Chan et al. (2016), Fan et al. (2016), Zhao
(2016) and Zhao and Percival (2017) in that we establish asymptotic optimality but for approximately – as opposed to exactly – balancing weights. Our main assumptions rely on the uniform
approximability of the propensity score function and the outcome function by basis functions.
This uniform approximability depends on the smoothness of the two functions and the dimension
d of the covariates. It further requires the basis functions to constitute a complete bases. If both
functions belong to the Hölder class with a smoothness parameter s on the domain [0, 1]d , then
the uniform approximability holds for the spline basis and the wavelet basis with a uniform rate
of convergence s/ d (Newey, 1997; Chen, 2007). In this case, our results require a weaker condition over s than others in the literature: ours require s/ d > 1/2 on both the propensity score and
outcome functions, whereas those of Hirano et al. (2003) require s/ d > 7 on the propensity score
function and s/ d > 1 on the outcome function, Chan et al. (2016) require s/ d > 13 on the propensity score function and s/ d > 3/2 on the outcome function, and Fan et al. (2016) require s/ d > 3/4
on the propensity score function and s/ d > 1/2 on the outcome function. To establish these results
under weaker assumptions, we use Bernstein’s inequality as in Fan et al. (2016) and leverage the
particular structure of MABW.
Third, we address two practical problems in MABW: choosing the number of balancing basis
functions and selecting the degree of approximate balancing. We derive a finite-sample upper
bound for the potential loss incurred by balancing too many functions of the covariates. This result
shows that the loss due to balancing too many basis functions is hedged in MABW because the

3

number of active balancing constraints is implicitly bounded. More precisely, MABW mimic
exact balancing weights while capping the number of active balancing constraints. We further
provide a tuning algorithm for calibrating the degree of balancing in MABW. This is a general
problem in weighting and thus our algorithm can be of independent interest. Finally, we conclude
with three empirical studies that show MABW yield weighting estimators that have considerably
lower root mean squared error (RMSE) than their exact balancing counterparts when the treated
and the control groups exhibit poor overlap.

2 Minimal approximately balancing weights (MABW)
For simplicity of exposition, we focus on the problem of estimating a population mean from a
sample with incomplete outcome data under the assumption that the outcomes are missing at
random (Mealli and Rubin, 2015). Under the closely related assumption of strong ignorability
(Rosenbaum and Rubin, 1983), this problem is analogous to estimating an average treatment effect
in an observational study. See Kang and Schafer (2007) for an example connecting the problems
of causal inference and estimation with incomplete outcome data.
Define P as a target population, S as a random sample of P of size n, and R as a sample
of r respondents from S . Let Z i be the response indicator, so that Z i = 1 if unit i responds
and Z i = 0 otherwise, and write X i for a vector of observed covariates. Put Yi for the outcome
variable. We assume the propensity score π( x) = pr( Z = 1 | X = x) satisfies 0 < π( x) < 1. Our goal
is to estimate the population mean of the outcome ȲN under the assumption that respondents are
missing at random. This assumption states that missingness can be fully explained by the observed
P
covariates. To achieve this goal, we use the weighting estimator Ŷw = ni=1 w i Z i Yi , where the
weights w i adjust for or balance the observed covariates.
Conventionally, the weights are obtained by fitting a model for the probability of response and
then inverting the predicted probabilities. While this approach is widely used, it suffers from two
problems in practice: first, achieving covariate balance can be difficult; second, estimates can be
unstable due to the variability of the weights (see Zubizarreta 2015 for a discussion). To address
these problems, several weighting methods have been proposed recently which are encompassed
by the following optimization program:
minimize
w

subject to

n
X

Z i f (w i )

(1.1)

i=1

¯
¯
¯X
¯
n
n
1X
¯
¯
B k ( X i )¯ ≤ δk , k = 1, ..., K ,
¯ wi Z i Bk ( X i ) −
¯ i=1
¯
n i=1

(1)
(1.2)

where f is a convex function of the weights, and B k ( X i ), k = 1, ..., K , are regular functions of the
covariates. Typically, the functions B k are basis functions for E (Yi ) and are chosen as the moments
of the covariates (see Assumption 1.4 and Assumption 1.6 below). The constants δk ’s are scalars
4

that constrain the imbalances in the B k functions. They are summarized in the vector δK ×1 =
(δ , ..., δK ) ≥ 0. Setting δ1 = 0 and B1 ( x) = 1 for all x will reduce the first balancing constraint to
Pn1
i=1 w i = 1. This constraint, in addition to the positivity constraint, 0 ≤ w i , i = 1, ..., n, (which can
also be accommodated in (1.2)) will require the weights not to extrapolate.
We call the class of weights that solve the above optimization program minimal approximately
balancing weights (MABW). They are minimal because they often minimize a measure of dispersion and extremity of the weights. They are approximately balancing because they have the
flexibility to approximately (as opposed to exactly) balance covariates. This flexibility plays an
important role in practice by trading bias for variance.
Special cases of MABW are the entropy balancing weights (Hainmueller, 2012), with f ( x) =
x log x and δ = 0; the stable balancing weights (Zubizarreta, 2015), with f ( x) = ( x − 1/ r )2 and
δ ∈ R+
0 ; and the empirical balancing calibration weights (Chan et al., 2016), with f ( x) = D ( x, 1),
where D ( x, x0 ) is a distance measure for a fixed x0 ∈ R that is continuously differentiable in x0 ∈
R, non-negative and strictly convex in x and δ = 0. With the exception of the stable balancing
weights, these methods commonly perform exact balancing by restricting δ = 0 while assuming
the existence of a solution to the optimization problem.
The dynamics between the feasibility and the efficacy of covariate balancing constraints is
central to causal inference and estimation with incomplete outcome data. Making the covariate
balancing constraints too tight could make the weighting problem infeasible, but relaxing them
too much compromises the goal of removing the biases due to covariate imbalances. Studying
these dynamics, however, calls for an alternative formulation of problem (1). Equality covariate
balancing constraints are well-aligned with the method of moments, as the dual optimization problem is an unconstrained one. Properties of its solution have been studied in other contexts (e.g.,
Deville and Särndal 1992; Hellerstein and Imbens 1999; Chan et al. 2016). Inequality covariate
balancing constraints, by contrast, correspond to a constrained dual optimization problem, whose
solution can be much more difficult to characterize.
Theorem 1 addresses this difficulty. It writes this constrained dual optimization problem as
an unconstrained one by utilizing the special structure of MABW. This dual problem allows
us to establish a connection between MABW and shrinkage estimation of the propensity score.
In short, the dual problem is implicitly fitting a generalized linear model on the functions of the
covariates B k (·) to the inverse propensity score function under a particular loss function with ℓ1
regularization.
Theorem 1. The dual of problem (1) is equivalent to the unconstrained optimization problem
minimize
λ

n £
¤
1X
− Z j nρ (B( X j )⊤ λ) + B( X j )⊤ λ + |λ|⊤ δ
n j=1

(2)

where λK ×1 is the vector of dual variables associated with the K balancing constraints, B( X j ) =
(B1 ( X j ), ..., B K ( X j )) denotes the K functions of the covariates, with ρ ( t) = t/ n− t( h′)−1 ( t)+ h(( h′)−1 ( t))
5

and h( x) = f (1/ n − x). Moreover, the primal solution w∗j satisfies
w∗j = ρ ′ (B( X j )⊤ λ† ), j = 1, ..., n,

(3)

where λ† is the solution to the dual optimization problem.
We defer the proof to Appendix A. Equation (2) takes the same form as an ℓ1 shrinkage estimation problem. The inverse propensity score function is estimated as a generalized linear model
of the basis functions with ρ ′ as the link function in equation (3). The dual variable λ can be seen
as the coefficients of the basis functions in the propensity score regression model. Its estimation is
regularized by the ℓ1 norm of these coefficients λ. The loss function is
L(λ) = − Znρ (B( x)⊤ λ) + B( x)⊤ λ.

(4)

This loss function has its expectation minimized when λ satisfies { nπ( x)}−1 = ρ ′ (B( x)⊤ λ) = w∗ .
This is the key equation connecting MABW to the propensity score π( x).
Theorem 1 implies that if the propensity score depends heavily on a given covariate, then
problem (1) will try hard to balance this covariate by assigning to it a large dual variable, a.k.a.
shadow price λ. On the other hand, the ℓ1 regularizer decreases the dependence of the weights on
hard-to-balance covariates.

3 Asymptotic properties
Theorem 1 formalizes the connection between MABW and shrinkage estimation of the inverse
propensity score function. This allows us to establish the asymptotic properties of MABW by
building on the results from nonparametric propensity score estimation.
At a high level, we view the optimization problem of the MABW as the following estimation
problem: assume the true propensity score function has the form π( x) = { nρ ′ ( m∗ ( x))}−1 , where
m∗ (·) is an unknown smooth function; approximate m∗ ( x) by a linear combination of the K basis
functions B( x); calculate the coefficients in this linear combination by minimizing the loss function
in Equation (2).
Below we assume the following conditions hold and prove that MABW are consistent estimates of the inverse propensity score function 1/π( x).
Assumption 1. Assume the following conditions hold:
1. The minimizer λo = arg minλ∈Θ E − Znρ (B( X )⊤ λ) + B( X )⊤ λ is unique, where Θ is the
parameter space for λ.
©

ª

2. λo ∈ int(Θ), where Θ is a compact set.
6

3. There exist constants 0 < c0 < 1/2, such that c0 ≤ nρ ′ (v) ≤ 1 − c0 for any v = B( x)⊤ λ with
λ ∈ int(Θ). Also, there exist constants c 1 < c 2 < 0, such that c 1 ≤ nρ ′′ (v) ≤ c 2 < 0 in some
small neighborhood B of v∗ = B( x)⊤ λ† .
4. There exists some constant C such that supx∈X ||B( x)||2 ≤ CK 1/2 and E {B( X i )B( X i )⊤ } ≤ C.
5. K = o( n).
6. There exist r π > 1 and λ∗1 such that the true propensity score function satisfies supx∈X |m∗ ( x)−
B( x)⊤ λ∗1 | = O(K −r π ).

7. ||δ||2 = O p (K 1/2 (log K )/ n + K 1/2−r π ).
Assumptions 1.1 and 1.2 are standard regularity conditions for the consistency of minimum risk
estimators. Assumption 1.3 is what enables the consistency of λ† to translate into the consistency
of the weights. This is satisfied by common choices of f in problem (1), including the variance,
the mean absolute deviation, and the negative entropy of the weights. Assumption 1.4 is a standard
technical condition that restricts the magnitude of the basis functions (see also assumption 4.1.6 of
Fan et al. 2016 and assumption 2(ii) of Newey 1997). This condition is satisfied by many classes
of basis functions including the regression spline, trigonometric polynomial, and wavelet bases
(Newey, 1997; Horowitz et al., 2004; Chen, 2007; Belloni et al., 2015; Fan et al., 2016). Assumption 1.5 controls the growth rate of the number of basis functions relative to the number of units.
Assumption 1.6 is a uniform approximation condition on the inverse propensity score function. It
requires the basis B( x) to be a complete basis, or m∗ ( x) to be well specified via the linear model.
For splines and power series bases, this assumption will be satisfied with r π = s/ d , where s is
the number of continuous derivatives of m∗ (·) that exist and d is the dimension of x with a compact domain (Newey, 1997). Assumption 1.7 quantifies the extent to which the equality covariate
balancing constraints can be relaxed such that the consistency of the resulting weight estimates
is maintained. Under these assumptions, we can prove MABW are consistent with the inverse
propensity score function.
Theorem 2. Let λ† be the solution to problem (1) and w∗ ( x) = ρ ′ (B( x)⊤ λ† ). Under Assumption 1,
1. supx∈X | nw∗ ( x) − 1/π( x)| = O p (K (log K )/ n + K 1−r π )) = o p (1),
2. ||nw∗ ( X ) − 1/π( x)||P,2 = O p (K (log K )/ n + K 1−r π )) = o p (1).
The proof is deferred to Appendix B. It consists of two steps. First we show that λ† , the solution
to the dual problem, is close in the ℓ2 norm to λ∗1 . The consistency of the weights as an estimate
of the inverse propensity score function then follows by the Lipschitz property of the ρ ′ function
and the bounds on the norms of the basis functions in Assumption 1.
We now assume the following additional conditions hold and prove that the resulting weighting
estimator is consistent and semiparametric efficent for the mean outcome.
7

Assumption 2. Assume the following conditions hold:
1. E |Y − Y ( X )| < ∞.
2. E |ȲN |2 < ∞.
3. There exist r y > 1/2 and λ∗2 such that the outcome model Y ( x) = E (Y | X = x) satisfies
sup x∈X |Y ( x) − B( x)⊤ λ∗2 | = O(K −r y ).

4. Let m∗ (·) ∈ M and Y (·) ∈ H , where m∗ (·) = (ρ ′ )−1 (1/( nπ( x))) as is defined at the start of this
section, Y (·) is the potential outcome function. M and H are two sets of smooth functions
satisfying log n [] (ε, M , L 2 (P )) ≤ C (1/ε)1/k1 and log n [] (ε, H , L 2 (P )) ≤ C (1/ε)1/k2 , where C is
a positive constant and k 1 , k 2 > 1/2. n [] (ε, M , L 2(P )) denotes the covering number of M by
ε-brackets.
−1

5. n0.5(r π+r y −0.5) = o(K ).
Theorem 3. Suppose that the conditions in assumptions 1 and 2 hold. Then
d

n1/2 (Ŷw∗ − ȲN ) → N (0, Vo pt ),

where Vo pt = var(Y ( X i )) + E {var(Y | X i )/π( X i )} attains the semiparametric asymptotic variance
bound.
The proof uses empirical process techniques inspired by Fan et al. (2016) and is deferred to
Appendix B. The proof involves the standard decomposition of Ŷw∗ − ȲN into four components,
where three of them converge to zero in probability, and the other one is asymptotically normal and
semiparametrically efficient. Each of the first three components can be controlled by the bracketing
number of the function classes of the inverse propensity score function and the outcome function.
Assumption 2.2 provides this control. This assumption is satistified, for example, by the Hölder
class with smoothness parameter s defined on a bounded convex subset of Rd with s/ d > 1/2
(Van Der Vaart and Wellner, 1996; Fan et al., 2016). This assumption is weaker than the one of
related results in the literature: s/ d > 7 in Hirano et al. (2003), s/ d > 13 in Chan et al. (2016), and
s/ d > 3/4 in Fan et al. (2016).

4 Practical considerations
4.1 Bounding the loss due to over-balancing
From Theorem 1 we can see that, for each basis function B k we balance, we are implicitly assuming
there is a similar term in the inverse propensity score model. A concern that arises in practice is the
estimation loss resulting from specifying a very complex model, i.e., over-balancing or balancing
more basis functions than needed. The following oracle inequality relieves this concern.
8

Theorem 4. Let λ† be the solution to the dual of the MABW problem (problem (2)) and λ‡ be the
solution to the dual of the exact balancing weights problem with the number of binding constraints
capped by setting δ = 0 and restricting ||λ‡ ||0 ≤ C 0 for some C 0 > 0. Under standard technical
conditions,
E {L(λ† ) − L(λ∗1 )} ≤ 3E {L(λ‡) − L(λ∗1 )} + c 0 ||λ‡ ||0 ,

where λ∗1 is the oracle estimate as in Assumption 1.6, L(·) is the dual loss as in Equation (4), and
c 0 is a positive constant depending on the number of basis functions K .
See Appendix C for full technical details. This oracle inequality bounds the loss of overbalancing: it says that the excess risk due to over-balancing in MABW is of the same order as the
expected risk in exact balancing weights with a capped number of balancing constraints. In other
words, approximate balancing with MABW mimics the act of bounding the number of balancing
constraints with exact balancing weights. Hence, MABW do not suffer much from excessive
balancing when few constraints are active. This sparse behavior of the balancing constraints is
commonly seen in real data sets; for example, in the 2010 Chilean post earthquake survey data
(see Figure 1 of Zubizarreta 2015).

4.2 Choosing δ
In a similar way to the parameter accompanying the ℓ1 norm in lasso estimation, δ is a tuning
parameter the investigator needs to choose for the degree of approximate balancing. Choosing δ is
particularly hard because there is not a clear target to optimize toward. For choosing δ, we propose
the following algorithm.
Algorithm 1. Tuning δ for MABW
For each δ in a grid D of possible imbalances
(e.g., D = {0,1e-3, 1e-2, 1e-1, 1} ∗ standard deviation)
Compute {w i }ni=1 by solving problem (1)
For each k ∈ {1, ..., K }
Take a bootstrap sample K k over the original data set
Evaluate covariate balance C k on the subset K k ,
P
P
w i Z i B k ( X i ))/( i∈K k w i Z i ) − ni=1 B k ( X i )/ n||2 /sd(B k ( X ))
P
Compute the mean covariate balance, C S (δ) := K
k=1 C k / K
∗
Output δ = arg minδ∈D C S (δ)

C k := ||(

P

i∈K k

At a high level, we view covariate balance in bootstrapped samples as a proxy for how well
the target quantities are estimated. The intuition is that the true inverse propensity score weights
9

theoretically balance the population, as well as draws from this population. Therefore, if the
weights are well-calibrated and robust to sampling variation, they should share this same property.
To this end, we evaluate covariate balance on bootstrapped samples with the weights calculated
from the original data set. In the following section, we show that the δ selected with Algorithm
(1) often coincides with or neighbors the optimal δ that gives the smallest root mean squared error
(RMSE) in estimating the target quantities. minimal approximately balancing weights (MABW)

4.3 Empirical studies
We now illustrate the performance of MABW in three empirical studies. In these three studies
we consider varying levels of covariate balance: 1e-3, 1e-2, 1e-1 and 1 standard deviations (sd) of
the basis functions considered, and three measures of dispersion of the weights: the ℓ1 dispersion,
|w − w̄|; the ℓ2 dispersion, (w − w̄)2 ; and the negative entropy, w log w. We find that MABW admit
a solution in cases where exact balancing does not. It also achieves considerably lower RMSE
when the treated/control or the observed/unobserved population exhibit poor overlap.
We defer two of these studies to Appendix D: one on the Kang and Schafer (2007) example and
another on the LaLonde (1986) data set. Here in details we present a simulation study based on the
Right Heart Catheterization (RHC) data set of Connors et al. (1996). This data set is used to study
the effectiveness of RHC in the initial care of critically ill patients. The data set has 2998 patients,
each with 77 variables including observed covariates, a treatment indicator, and outcomes.
Based on the original data set, we simulate 10000 data sets as follows. We construct the
treatment indicator Z i as Z i = 1{ Z∗i >0} where Z ∗i = (α + β X i )/ c + Unif(−0.5, 0.5), with X i as the
observed covariates, and α and β obtained by fitting a logistic regression to the original treatment
indicator in the data set. We simulate two scenarios, one with good overlap ( c = 10) and another
with bad overlap ( c = 1). For both scenarios, we generate pairs of potential outcomes (Yi (0), Yi (1))
by fitting a regression model to the original treated outcomes and control outcomes respectively
and predicting on the whole population. We obtain the observed outcome by letting Yi = Z i Yi (1) +
(1 − Z i )Yi (0).
In both scenarios, we compare the RMSE of the estimated average treatment effect on the
treated (ATT) using both MABW and exact balancing weights. Exact balancing the 77 available
covariates does not admit a feasible solution, so for comparison purposes we restrict the analyses
to the 23 important covariates listed in Table 1 of Connors et al. (1996). The results are presented
in Table 1 below and Table 2 in Appendix D.
We note that MABW can markedly reduce RMSE in data sets with bad overlap. Approximate
balancing with δ chosen by Algorithm 1 systematically reduces the RMSE of the ATT estimates
by more than a half with respect to exact balancing. In data sets with good overlap, MABW with
small δ perform similarly to exact balancing weights. In general, MABW exhibit better empirical
performance in the RHC data set. We observe similar patterns in the Kang and Schafer (2007)
simulation study and in the LaLonde (1986) data set. Please see Appendix D for details.

10

δ sd

0
1e-3
1e-2
1e-1
1

Good Overlap
|w − w̄|

(w − w̄)2

(0.20, 0.39)
(0.20, 0.39)
(0.20, 0.39)
(0.20, 0.58)
(0.24, 2.13)

(0.09, 0.08)
(0.09, 0.08)
(0.09, 0.07)
(0.09, 0.26)
(0.24, 2.12)

δ sd

w log w

Bad Overlap
|w − w̄|

(w − w̄)2

(0.08, 0.11)
0 (0.32, 1.10) (0.14, 0.35)
(0.08, 0.11) 1e-3 (0.32, 1.10) (0.13, 0.34)
(0.09, 0.10) 1e-2 (0.29, 1.06) (0.11, 0.34)
(0.08, 0.27) 1e-1 (0.13, 1.05) (0.06, 0.29)
(0.22, 2.12)
1 (0.24, 3.07) (0.24, 3.07)

(a) Good overlap

w log w

(0.20, 0.48)
(0.19, 0.47)
(0.17, 0.42)
(0.10, 0.38)
(0.24, 3.07)

(b) Bad overlap

Table 1: (RMSE , C S ) for the ATT in the RHC study. The RMSE can be reduced by more than
a half by balancing covariates approximately as opposed to exactly when there is bad overlap.
The bolded numbers signal the optimal δ with the smallest RMSE and the δ resulting in the best
covariate balance over the bootstrapped data sets.

References
Athey, S., Imbens, G. W., and Wager, S. (2016). Approximate residual balancing: De-biased
inference of average treatment effects in high dimensions. arXiv preprint arXiv:1604.07125.
Belloni, A., Chernozhukov, V., Chetverikov, D., and Kato, K. (2015). Some new asymptotic theory
for least squares series: Pointwise and uniform results. Journal of Econometrics, 186(2):345–
366.
Chan, K. C. G., Yam, S. C. P., and Zhang, Z. (2016). Globally efficient nonparametric inference
of average treatment effects by empirical balancing calibration weighting. Journal of the Royal
Statistical Society: Series B, 78(3):673–700.
Chen, X. (2007). Large sample sieve estimation of semi-nonparametric models. Handbook of
econometrics, 6:5549–5632.
Connors, A. F., Speroff, T., Dawson, N. V., Thomas, C., Harrell, F. E., Wagner, D., Desbiens,
N., Goldman, L., Wu, A. W., Califf, R. M., et al. (1996). The effectiveness of right heart
catheterization in the initial care of critically iii patients. Jama, 276(11):889–897.
Deville, J.-C. and Särndal, C.-E. (1992). Calibration estimators in survey sampling. Journal of the
American statistical Association, 87(418):376–382.
Fan, J., Imai, K., Liu, H., Ning, Y., and Yang, X. (2016). Improving covariate balancing propensity
score: A doubly robust and efficient approach.

11

Hahn, J. (1998). On the role of the propensity score in efficient semiparametric estimation of
average treatment effects. Econometrica, pages 315–331.
Hainmueller, J. (2012). Entropy balancing for causal effects: a multivariate reweighting method to
produce balanced samples in observational studies. Political Analysis, 20(1):25–46.
Hellerstein, J. K. and Imbens, G. W. (1999). Imposing moment restrictions from auxiliary data by
weighting. Review of Economics and Statistics, 81(1):1–14.
Hirano, K., Imbens, G. W., and Ridder, G. (2003). Efficient estimation of average treatment effects
using the estimated propensity score. Econometrica, 71(4):1161–1189.
Hirshberg, D. A. and Wager, S. (2017). Balancing out regression error: Efficient treatment effect
estimation without smooth propensities. arXiv preprint arXiv:1712.00038.
Horowitz, J. L., Mammen, E., et al. (2004). Nonparametric estimation of an additive model with a
link function. The Annals of Statistics, 32(6):2412–2443.
Imai, K. and Ratkovic, M. (2014). Covariate balancing propensity score. Journal of the Royal
Statistical Society: Series B, 76(1):243–263.
Kang, J. D. Y. and Schafer, J. L. (2007). Demystifying double robustness: A comparison of
alternative strategies for estimating a population mean from incomplete data (with discussion).
Statistical Science, 22(4):523–539.
LaLonde, R. J. (1986). Evaluating the econometric evaluations of training programs with experimental data. The American Economic Review, pages 604–620.
Li, F., Morgan, K. L., and Zaslavsky, A. M. (2016). Balancing covariates via propensity score
weighting. Journal of the American Statistical Association, page forthcoming.
Mealli, F. and Rubin, D. B. (2015). Clarifying missing at random and related definitions, and
implications when coupled with exchangeability. Biometrika, 102(4):995–1000.
Newey, W. K. (1997). Convergence rates and asymptotic normality for series estimators. Journal
of econometrics, 79(1):147–168.
Robins, J. M., Rotnitzky, A., and Zhao, L. P. (1994). Estimation of regression coefficients when
some regressors are not always observed. Journal of the American Statistical Association,
89(427):846–866.
Rosenbaum, P. R. (1987). Model-based direct adjustment. Journal of the American Statistical
Association, 82(398):387–394.

12

Rosenbaum, P. R. and Rubin, D. B. (1983). The central role of the propensity score in observational
studies for causal effects. Biometrika, 70(1):41–55.
Rubin, D. B. (2008). For objective causal inference, design trumps analysis. The Annals of Applied
Statistics, 2(3):808–840.
Tropp, J. A. et al. (2015). An introduction to matrix concentration inequalities. Foundations and
Trends® in Machine Learning, 8(1-2):1–230.
Tseng, P. and Bertsekas, D. P. (1991). Relaxation methods for problems with strictly convex costs
and linear constraints. Mathematics of Operations Research, 16(3):462–481.
Van de Geer, S. A. (2008). High-dimensional generalized linear models and the lasso. The Annals
of Statistics, pages 614–645.
Van Der Vaart, A. W. and Wellner, J. A. (1996). Weak convergence. In Weak Convergence and
Empirical Processes, pages 16–28. Springer.
Zhao, Q. (2016). Covariate balancing propensity score by tailored loss functions. arXiv preprint
arXiv:1601.05890.
Zhao, Q. and Percival, D. (2017). Entropy balancing is doubly robust. Journal of Causal Inference,
5(1).
Zubizarreta, J. R. (2015). Stable weights that balance covariates for estimation with incomplete
outcome data. Journal of the American Statistical Association, 110(511):910–922.

13

Supplementary materials
A Proof for the unconstrained dual formulation
Proof of Theorem 1
Proof. We first present a vanilla form of the dual.
Lemma 1. The dual of the optimization problem (1) writes
l (λ)

minimize
λ

subject to λ ≥ 0
where

n
1X
{− Z j nρ (Q ⊤j λ) + Q ⊤j λ} + λ⊤ d,
n j=1

l (λ) =



A K ×n = 

B1 ( X 1 )

B1 ( X 2 )

..
.

...

..
.

..
.

B1 ( X n )

..
.

B K ( X 1 ) B K ( X 2 ) ... B K ( X n )
Q 2K ×n =

µ

and
d 2K ×1 =

¶

A K ×n
− A K ×n
µ

δK ×1
δK ×1

¶





,
K ×n

,
2K ×n

.

2K ×1

We prove this lemma towards
the ¶end of this section.
µ
We then write λ2K ×1 =
l (λ) =
=

λ+,K ×1
λ−,K ×1

. We have

2K ×1

n
1X
⊤
{− Z j nρ ( A ⊤j λ+ − A ⊤j λ− ) + ( A ⊤j λ+ − A ⊤j λ− )} + λ⊤
+ δ + λ− δ
n j=1
n
1X
⊤
{− Z j nρ ( A ⊤j (λ+ − λ− )) + A ⊤j (λ+ − λ− )} + (λ⊤
+ + λ− )δ.
n j=1

Suppose the optimizer is

†
λ2K ×1

=

Ã

†

λ+,K ×1

†
λ−,K ×1

!

2K ×1

. We claim that λ†+,k · λ†−,k = 0, k = 1, ..., K , where

the index k points to the kth entry of a vector.

14

We prove this claim by contradiction. Suppose the opposite. If λ†+,k > 0 and λ†−,k > 0 for some
k, then
†

†

†

†

†

λ††⊤ = (λ+ − (0, ..., 0, min(λ+,k , λ−,k ), 0, ..., 0), λ†− − (0, ..., 0, min(λ+,k , λ−,k ), 0, ..., 0))

has
l (λ†† ) = l (λ† ) − 2 min(λ†+,k , λ†−,k ) · δ < l (λ† )

by δ > 0 and min(λ†+,k , λ†−,k ) > 0. This contradicts the fact that λ† is the optimizer. Theorem 1 then

follows by rewriting λ+ − λ− as λ and deducing λ+ + λ− = |λ| from λ†+,k · λ†−,k = 0, k = 1, ..., K .

Proof of Lemma 1
Proof. Rewriting problem (1) in matrix notation,
minimize
w

n
X

Z i h( s i )

i=1

subject to Q 2K ×n s n×1 ≤ d 2K ×1
where
1
s n×1 = ( s i )n×1 = ( − Z i w i )n×1 ,
n


B1 ( X 1 ) B1 ( X 2 ) ... B1 ( X n )


..
..
..
..
,Q 2K ×n
A K ×n = 

.
.
.
.
B K ( X 1 ) B K ( X 2 ) ... B K ( X n ) K ×n
µ
¶
δK ×1
.
d 2K ×1 =
δK ×1 2K ×1

=

µ

A K ×n
− A K ×n

¶

,
2K ×n

Again as special cases, stable balancing weights have h( x) = ( n1 − 1r − x)2 and entropy balancing has
h( x) = ( n1 − x) log( n1 − x).
The problem is now in the form of Tseng & Bertsekas (1987) and Tseng and Bertsekas (1991).
The dual of this problem is
maximize g(λ)
λ

subject to λ ≥ 0,

15

where g(λ) = −

Pn

∗
⊤
j =1 h j (Q j λ)− < λ, d

>, and h∗j (·) is the convex conjugate of Z j h(·).

h∗j ( t) = sup{ ts j − Z j h( s j )}
sj

t
1
− Z j h ( − Z j w j )}
n
n
wj
t
1
= sup{− tZ j w j + − Z j h( − w j )}
n
n
wj
t
1
= − tZ j w∗j + − Z j h( − w∗j ),
n
n
= sup{− tZ j w j +

where w∗j satisfies the first order condition
− tZ j + Z j h′ (

1
− w∗j ) = 0,
n

1
− w∗j ) = t,
n
1
⇒w∗j = − ( h′)−1 ( t).
n

⇒ h′ (

Therefore,
t
1
+ tZ j ( h′ )−1 ( t) + − Z j h(( h′)−1 ( t)),
n
n
t
t
′ −1
= − Z j { − t( h ) ( t) + h(( h′)−1 ( t))} + .
n
n

h∗j ( t) = − tZ j

Denote ρ (·) as

ρ ( t) =

This gives

t
− t( h′)−1 ( t) + h(( h′)−1 ( t).
n
t
h∗j ( t) = − Z j ρ ( t) + .
n

Also we notice that
1
− ( h′ )−1 ( t) − t(( h′)−1 ( t))′ + h′ (( h′)−1 ( t)) · (( h′ )−1 ( t))′
n
1
= − ( h′ )−1 ( t) − t(( h′)−1 ( t))′ + t(( h′ )−1 ( t))′
n
1
= − ( h′ )−1 ( t).
n

ρ ′ ( t) =

16

This implies
w ∗ = ρ ′ ( t ).

The dual formulation thus becomes
minimize l (λ)
λ

subject to λ ≥ 0
where
l (λ) =

n
1X
{− Z j nρ (Q ⊤j λ) + Q ⊤j λ} + λ⊤ d.
n j=1

B Proof of the Asymptotic properties
Proof of Theorem 2
Proof. The proof utilizes the Bernstein’s inequality as is inspired from Fan et al. (2016).
We first prove the following lemma.
Lemma 2. There exists a global minimizer λ† such that
||λ† − λ∗1 ||2 = O p (K 1/2 / n1/2 + K −r π ).

Proof. Write A j = B( X j ) = (B1 ( X j ), ..., B K ( X j )). Recall that the optimization objective is
G (λ) :=

n
1X
[− Z j nρ ( A ⊤j λ) + A ⊤j λ] + |λ|⊤ δ,
n j=1

where G (·) is convex in λ by the concavity of ρ (·). To show that a minimizer ∆∗ of G (λ∗1 + ∆) exists
in C = {∆ ∈ RK : ||∆||2 ≤ C (K 1/2 (log K )/ n + K 1/2−r π } for some constant C , it suffices to show that
E { inf G (λ∗1 + ∆) − G (λ∗1 ) > 0} → 1, as n → ∞, (∗)
∆∈C

by the continuity of G (·).

17

To show (∗), we use mean value theorem: for some λ̃ between λ† and λ∗1 ,
G (λ∗1 + ∆) − G (λ∗1 )
n
n
X
1X
1
≥∆ ·
{− Z j nρ ′ ( A ⊤j λ∗1 ) A j + A j } + ∆⊤ · { − Z j ρ ′′ ( A ⊤j λ̃) A ⊤j A j } · ∆ − |∆|⊤ δ
n j=1
2
j =1
≥ − ||∆||2 · ||

(6)

n
1 X
− Z j nρ ′ ( A ⊤j λ∗1 ) A j + A j ||2
n j=1

n
X
1
+ ∆⊤ · { − Z j ρ ′′ ( A ⊤j λ̃) A ⊤j A j } · ∆ − ||∆||2 ||δ||2
2
j =1

≥ − ||∆||2 · ||

(5)

(7)

n
1 X
− Z j nρ ′ ( A ⊤j λ∗1 ) A j + A j ||2 − ||∆||2 ||δ||2
n j=1

(8)

The first inequality is due to the triangle inequality, |λ∗1 + ∆| − |λ∗1 | ≥ −|∆|. The second inequality is due to Cauchy-Shwarz inequality. The third inequality is due to the positivity of
1 ⊤ Pn
′′
⊤
⊤
2 ∆ · [ j =1 − Z j ρ ( A j λ̃) A j A j ] · ∆ by Assumption 1.3.
Notice that
||
≤||
≤||

n
1X
[− Z j nρ ′ ( A ⊤j λ∗1 ) A j + A j ]||2
n j=1

n
n
1X
1X
1
1
[− Z j A j + A j ]||2 + ||
− Z j ( − nρ ′ ( A ⊤j λ∗1 )) A j ||2
n j=1
πj
n j=1
πj
n
n
Zj
1X
1X
(1 − ) A j ||2 +
|| A j ||2 O(K −r π )
n j=1
πj
n j=1

The first inequality is due to the triangle inequality. The second unequality is due to Assumption 1.3 and 1.6.
We first use the Bernstein’s inequality to bound both terms.
Recall that the Bernstein’s inequality for random matrices in Tropp et al. (2015) says the following. Let { Z k } be a sequence of independent random matrices with dimensions d 1 × d 2 . Assume
that EZ k = 0 and || Z k ||2 ≤ R n almost surely. Define
σ2n = max{||

Then for all t ≥ 0,
pr (||

n
X

k=1

n
X

k=1

E ( Z k Z k⊤ )||2 , ||

n
X

k=1

E ( Z k⊤ Z k )||2 }.

Z k ||2 ≥ t) ≤ ( d 1 + d 2 ) exp(−

18

t2 /2
σ2n + R n t/3

).

For the first term || n1

Zj
j =1 (1 − π j ) A j ||2 ,

Pn

we notice that

Zj
Zj
1
1
E { (1 − ) A j } = E [E { (1 − ) A j | X j }] = 0.
n
πj
n
πj

(9)

The last equality is because E ( Z j ) = π j .
P
Then for || n1 nj=1 (1 − Z j /π j ) A j ||2 , we have

Zj
1
|| (1 − ) A j ||2
n
πj
Zj
1
≤ ||(1 −
)||2 || A j ||2
n
πj
1 1 −πj
)CK 1/2
≤ (
n πj
1
= ( nρ ′ ( A ⊤j λ∗1 ) − 1)CK 1/2
n
K 1/2
≤C ′
.
n

(10)
(11)
(12)
(13)
(14)

The first inequality is due to Cauchy-Shwarz inequality. The second inequality is due to Assumption 1.4 and E (1 − Z j /π j )2 = var(1 − Z j /π j ) = π j (1 − π j )/π2j = (1 − π j )/π j . The third equality is
due to π j = { nρ ′( A ⊤j λ∗1 )}−1 . The fourth inequality is due to Assumption 1.3.
Finally, for ||

Pn

k=1

Z

E ( n12 (1 − π jj )2 A j A ⊤j )||2 , we have
n
X

Zj 2
1
(1
−
) A j A ⊤j }||2
2
π
n
j
k=1
Zj
1
≤ sup(1 − )2 ||E ( A j A ⊤j )||2
n j
πj
||

≤

(15)

E{

(16)

C ′′
.
n

(17)
Z

The first inequality is taking the sup over (1 − π jj )2 . The second inequality is due to Assumption
1.3, 1.4, and π j = { nρ ′ ( A ⊤j λ∗1 )}−1 .
(9), (14), and (17), together with the Bernstein’s inequality, imply
pr(||

n
Zj
1X
t2 /2
).
(1 − ) A j ||2 ≥ t) ≤ (K + 1) exp(− ′′
1/2
C
n j=1
πj
+ C ′ K · t/3
n

19

n

(18)

The right side goes to zero as K → ∞ when

t2 /2

C ′′
n

1/2

+ C ′ Kn · t/3

≥ log K .

It suffices when t = O p (K 1/2 (log K )/ n).
Therefore, we have
n
Zj
1X
(1 − ) A j ||2 = O p (K 1/2 (log K )/ n).
n j=1
πj
P
Now we work on the second term n1 nj=1 || A j ||2 O(K −r π ).

||

n
1X
|| A j ||2 O(K −r π ) ≤ CK 1/2−r π
n j=1

This inequality is due to Assumption 1.4.
Combining (19), (20), and Assumption 1.7, we have
G (λ∗1 + ∆) − G (λ∗1 )
= − ||∆||2 · O p (

1
K 1/2 log K
+ K 1/2−r π )) + ||∆||22 ||δ||2
n
2

≥0
1/2

for ∆ = K nlog K + K 1/2−r π ) with large enough constant C > 0.
(∗) is thus proved.
Now we are ready to prove Theorem 2.
sup |nw∗ ( x) −
x∈X

1
|
π( x)

= sup |nρ ′ (B( x)⊤ λ† ) − nρ ′ ( m∗ ( x))|
x∈X

≤ sup |nρ ′ (B( x)⊤ λ† ) − nρ ′ (B( x)⊤ λ∗1 )| + sup |nρ ′ (B( x)⊤ λ∗1 ) − nρ ′ ( m∗ ( x))|
x∈X

⊤ †

=O(sup |B( x) λ
x∈X

x∈X
⊤ ∗
−r π
− B( x) λ1 |) + O(K
)

≤O(sup ||B( x)||2 ||λ† − λ∗1 ||2 ) + O(K −r π )
x∈X

log K
+ K −r π )) + O(K −r π )
n
K log K
+ K 1−r π ))
=O p (
n
= o p (1)

=O p ( K (

20

(19)

(20)

The first equality rewrites π( x) = { nρ ′(B( x)⊤ λ∗1 )}−1 . The second inequality is due to triangle
inequality. The third inequality is due to Assumptions 1.3 and 1.6. The fourth inequality is due to
Cauchy-Schwarz inequality. The fifth equality is due to Lemma 2 and Assumption 1.4. The sixth
equality is due to the first term dominates the second. The seventh equality is due to Assumptions
1.5 and 1.6.
Also,
1
||P,2
π( x)
1
=||nρ ′ (λ†⊤ B( X )) −
||P,2
π( x)
||nw∗ ( X ) −

1
− nρ ′ (λ∗⊤
1 B ( X ))||P,2
π( x)
.||(λ† − λ∗1 )⊤ B( X )||P,2 + sup |m∗ ( x) − λ∗⊤
1 B ( x )|

.||nρ ′ (λ†⊤ B( X )) − nρ ′ (λ∗⊤
1 B ( X ))||P,2 + ||
x∈X

log K
+ K −r π )) + O(K −r π )
n
K 1/2 log K
=O p (
+ K 1/2−r π )
n
= o p (1)
=O p (K 1/2 (

The first equality rewrites π( x) = { nρ ′(B( x)⊤ λ∗1 )}−1 . The second inequality is due to triangle
inequality. The third inequality is due to Assumption 1.3. The fourth inequality is due to Lemma 2,
Assumption 1.4 and Assumption 1.6. The fifth equality is due to the first term dominates the
second. The sixth equality is due to Assumption 1.5 and Assumption 1.6.

Proof of Theorem 3
Proof. The proof utilizes empirical processes techniques as is inspired from Fan et al. (2016).

21

We first decompose Ŷw∗ − ȲN into several residual terms.
Ŷw∗ − ȲN =
=
=

n
X

i=1
n
X

i=1
n
X

i=1

Z i w∗i Yi − ȲN
Z i w∗i (Yi − Y ( X i )) +
Z i w∗i (Yi − Y ( X i )) +

n
X

i=1
n
X

i=1

( Z i w∗i −
( Z i w∗i −

n
1X
1
)Y ( X i ) + (
Y ( X i ) − ȲN )
n
n i=1

1
)(Y ( X i ) − λ∗⊤
2 B ( X i ))
n

n
1X
1
B
(
X
)
+
(
Y ( X i ) − ȲN )
( Z i w∗i − )λ∗⊤
i
n 2
n i=1
i=1
n
1X
=
S i + R0 + R1 + R2 ,
n i=1

+

n
X

where
Zi
(Yi − Y ( X i )) + (Y ( X i ) − ȲN ),
πi
n
X
1
) Z i (Yi − Y ( X i )),
R 0 = (w∗i −
nπ i
i=1
n
X
1
R 1 = ( Z i w∗i − )(Y ( X i ) − λ∗⊤
2 B ( X i )),
n
i=1
n
X
1
B( X i )).
R 2 = ( Z i w∗i − )(λ∗⊤
n 2
i=1
Si =

Below we show R j = o p ( n−1/2 ), 0 ≤ j ≤ 2. The conclusion then follows from S i taking the same
form as the efficient score (Hahn, 1998). Ŷw∗ is thus asymptotically normal and semiparametrically
efficient.
P
We first study R 0 = ni=1 ( nw∗i − 1/π i ) Z i (Yi − Y ( X i ))/ n. Consider an empirical process Gn ( f 0 ) =
P
n1/2 [ ni=1 f 0 ( Z i , Yi , X i )/ n − E { f 0 ( Z, Y , X )}], where
¶
1
.
f 0 ( Z, Y , X ) = Z (Y − Y ( X )) nρ ( m( X )) −
π( x)
µ

′

By the missing at random assumption, we have that E f 0 ( Z, Y (1), X ) = 0.
By Theorem 2, we have
sup |ρ ′ (B( x)⊤ λ† ) −
x∈X

1
K log K
| = Op(
+ K 1−r π )) = o p (1).
nπ( x)
n

22

By Markov’s inequality and maximal inequality, we have
n1/2 R 0 ≤ sup Gn ( f 0 ) . E sup Gn ( f 0 ) . J[] (||F0 ||P,2 , F , L 2 (P )),
f 0 ∈F

f 0 ∈F

where the set of functions is F = { f 0 : ||m − m∗ ||∞ ≤ δ0 }, where || f ||∞ = supx∈X | f ( x)| and δ0 =
C (K (log K )/ n + K 1−r π )) for some constant C > 0.
The second inequality is due to Markov’s inequality. J[] (||F0 ||P,2 , F , L 2 (P ) is the bracketing
integral. F0 := δ0 |Y − Y ( X )| & | f 0 ( Z, Y , X )| is the envelop function. We also have ||F0 ||P,2 =
(EF02 )1/2 . δ0 by E |Y − Y ( X )| < ∞.
Next we bound J[] (||F0 ||P,2 , F , L 2 (P ) by n [] (ε, F , L 2 (P )):
J[] (||F0 ||P,2 , F , L 2 (P ) .

Zδ
0

{ n [] (ε, F , L 2 (P ))}1/2dε.

Define a new set of functions F0 = { f 0 : ||m − m∗ ||∞ ≤ C } for some constant C > 0, because
need a constant different than δ0 as δ0 → 0 can change. Then,
log n [] (ε, F , L 2 (P )) . log n [] (ε, F0 δ0 , L 2 (P ))
= log n [] (ε/δ0 , F0 , L 2 (P ))

. log n [] (ε/δ0 , M , L 2(P )
. (δ0 /ε)(1/k1 ) .

The first inequality is due to ρ ′ (·) bounded away from 0 and Lipschitz. The last inequality is due
to Assumption 2.2.
Therefore, we have
J[] (||F0 ||P,2 , F , L 2 (P ) .

Zδ
0

{log n [] (ε, F , L 2 (P ))}

1/2

dε .

Zδ
0

(δ0 /ε)(1/2k1 ) dε.

This goes to 0 as δ goes to 0 by 2 k 1 > 1 and the integral converges. Thus, this shows that
n R 0 = o p (1).
1/2

Next, we consider R 1 = ni=1 ( nZ i w∗i − 1)(Y ( X i ) − λ∗⊤
2 B ( X i ))/ n. Define the empirical pro1/2 Pn
cess Gn ( f 1 ) = n [ i=1 f 1 ( Z i , X i )/ n − E { f 1 ( Z, X )}], where f 1 ( Z, X ) = ( nZ ρ ′( m( x)) − 1)(Y ( X ) −
λ∗⊤
2 B ( X )).
−r y
Write ∆( X ) := Y ( X ) − λ∗⊤
.
2 B ( X ). By Assumption 2.3, we have ||∆||∞ . K
By Theorem 2, we have
P

|nρ ′ (λ†⊤ B( X )) −

1
K log K
||P,2 = O p (
+ K 1 − r π ).
π( x)
n

23

Therefore, we have
n1/2 R 1 = Gn ( f 1 ) + n1/2 E f 1 ( Z, X )

≤ sup Gn ( f 1 ) + n1/2 sup E f 1
f 1 ∈F1

f 1 ∈F1

where F1 = { f 1 : ||m − m∗ ||P,2 ≤ δ1 ,||∆||∞ ≤ δ2 },δ1 = C (K 1/2 (log K )/ n + K 1/2−r π ), δ2 = CK −r y for
some constant C > 0.
Again, by Markov’s inequality and the maximal inequality,
sup Gn ( f 1 ) . E sup Gn ( f 1 ) . J[] (||F1 ||P,2 , F , L 2 (P )),

f 1 ∈F1

f 1 ∈F1

where F1 := C δ2 for some constant C > 0 so that ||F1 ||P,2 . δ2 .
Similar to characterizing R 1 , we we bound J[] (||F1 ||P,2 , F1 , L 2 (P ) by n [] (ε, F1 , L 2 (P )):
J[] (||F1 ||P,2 , F1 , L 2 (P ) .

Zδ
0

{ n [] (ε, F1 , L 2 (P ))}1/2 dε.

Then, we bound n [] (ε, F1 , L 2 (P )).
log n [] (ε, F1 , L 2 (P )) . log n [] (ε/δ2 , F0 , L 2 (P ))

. log n [] (ε/δ2 ,G 10 , L 2 (P )) + log n [] (ε/δ2 ,G 20 , L 2 (P ))
. log n [] (ε/δ2 , M , L 2(P )) + log n [] (ε/δ2 , H , L 2(P ))
. (δ1 /ε)1/k1 + (δ2 /ε)1/k2 .

where
F0 = { f 1 : ||m − m∗ ||P,2 ≤ C, ||∆||P,2 ≤ 1},
G10 = { m ∈ M + m∗ : ||m||P,2 ≤ C },

G20 = {∆ ∈ H − λ∗⊤
2 B ( x) : ||∆||P,2 ≤ 1}.

The second inequality is due to ρ ′ is Lipschitz and bounded away from 0.
Therefore we have
J[] (||F1 ||P,2 , F1 , L 2 (P ) .

Zδ
0

(δ1 /ε)

(1/2 k 1 )

dε +

Zδ
0

(δ2 /ε)(1/2k2 ) d ε.

By 2 k 1 > 1, 2 k 2 > 1, we have J[] (|| f 1 ||P,2 , F , L 2 (P )) = o(1). This gives sup f 1 ∈F1 Gn ( f 1 ) =

o p (1).

24

Now we look at n1/2 sup f 1 ∈F1 E f 1 .
n1/2 sup E f 1 = n1/2
f 1 ∈F

= n1/2

sup
m∈G1 ,∆∈G2

sup
m∈G1 ,∆∈G2

E (π( X )( nρ ′( m( X )) − 1)∆( X ))
E (( nρ ′( m( x)) −

. n1/2 sup ||nρ ′ ( m( x)) −
.n

1/2

m∈G1

1
)π( x)∆( x))
π( x)

1
||P,2 sup ||∆( x)||P,2
π( x)
∆∈G2

δ1 δ2 = o p (1),

where G1 = { m ∈ M : ||m − m∗ ||P,2 ≤ δ1 }, G2 = {∆ ∈ H − λ∗⊤
2 B ( x) : ||∆||∞ ≤ δ2 }.
1/2
r π +r y −1/2
The last equality is due to assumption n . K
.
Therefore, we can conclude n1/2 R 1 = o p (1).
Pn
Pn
∗
∗
2
Lastly, R 2 = λ∗⊤
2 ( i=1 ( Z i w i − 1/ n)B ( X i )) = o p (1) by
i=1 ( Z i w i − 1/ n)B ( X i ) ≤ ||δ|| = o p (1)
due to the optimization condition.

C Theorem 4 explained
By establishing a connection to shrinkage estimation we can see that for each basis function that
we balance we are implicitly assuming a corresponding term in the inverse propensity score model.
A concern that may arise in practice is that we may run into estimation loss when we specify a very
complex model, i.e., when we balance more terms than needed.
Theorem 4 is an oracle inequality that bounds this loss and states that approximate balancing
— as opposed to exact balancing — mimics the act of upper bounding the number of effective
balancing constraints. Hence, MABW do not suffer much from excessive balancing when few
constraints are active. We also remark that this sparsity assumption on the balancing constraints is
commonly satisfied in real data sets. This is exemplified by the sparsity of the shadow prices in the
2010 Chilean post earthquake survey data; see Figure 1 of Zubizarreta (2015).
This oracle inequality we prove leverages an oracle inequality for lasso in the high dimensional
generalized linear model literature (Van de Geer, 2008). The original oracle inequality says the
lasso estimator (with ℓ1 penalty) under general Lipshitz losses behaves similarly to the estimator
with ℓ0 penalty, if the true generalized linear model is sparse.
Recall that the MABW compute
n
X

¸
·
1
⊤
⊤
+ |λ|⊤ δ.
− Z j ρ ( A j λ) + A j λ ·
λ := arg min G (λ) = arg min
n
j =1
†

This is a lasso estimator under the loss function
L w ( x, z) = − z · n(ρ ◦ (ρ ′ )−1 ◦ w)( x) + ((ρ ′ )−1 ◦ w)( x),

25

where the fit for w is ŵ( x) = ρ ′ (B( x)⊤ λ̂). This loss function is the same loss function as in (4) but
written as a function of w. Correspondingly, the empirical loss is
iX
=1
n

Lw(X i, Zi) =

n £
¤
1X
− Z i n · (ρ ◦ (ρ ′ )−1 ◦ w)( X i ) + ((ρ ′ )−1 ◦ w)( X i ) ,
n i=1

and the theoretical risk is
n
£
¤
1X
E − Z i n · (ρ ◦ (ρ ′ )−1 ◦ w)( X i ) + ((ρ ′ )−1 ◦ w)( X i )
n i=1
n £
¤
1X
−π( X i ) · n(ρ ◦ (ρ ′ )−1 ◦ w)( X i ) + ((ρ ′ )−1 ◦ w)( X i ) .
=
n i=1

EL w ( X , Z ) =

We define our target w0 as the minimizer of the theoretical risk
w0 ( x) := arg min EL w ( X , Z ) =

1
.
nπ( x)

The last equality is due to setting ∂EL w ( X , Z )/∂w = 0. This is the true inverse propensity score
function we use as weights. We are interested in studying the excess risk of estimators
E ( w ) : = E { L w ( X , Z ) − L w 0 ( X , Z )} .

For simplicity of notation, we write wλ ( x) = ρ ′ (B( x)⊤ λ), λ ∈ RK . Approximate balancing
weights thus perform the empirical risk minimization of
λ† := arg min{
λ

n
1X
L wλ ( X i , Z i ) + |λ|⊤ δ}.
n i=1

We look at the case of δ = δ+ (σ̂1 , σ̂2 , ..., σ̂K ), for some δ+ > 0, where σ̂k is the (sample) standard
error of B k ( X ), k = 1, ..., K . This aligns closely with our common way of setting δ; we specify
approximate balancing constraints in units of the standard error of each covariate.
We consider the following oracle estimator
λ∗ := arg min{EL wλ ( X , Z ) + ||λ||0 · C 0 },
λ

for some constant C 0 > 0. λ∗ can also be seen as the minimizer of PL wλ under the constraint that
||λ||0 ≤ C 1 for some C 1 > 0.
||λ||0 is the number of nonzero entries of λ. This is also the number of binding (or effective)
covariate balancing constraints in our optimization problem (1). In this sense, the oracle estimator
roughly performs the same covariate balancing exactly as its approximate counterpart λ† but has
its number of effective constraints being capped by some constant.
We are now ready to present the oracle inequality.
26

Assumption 3. The following conditions hold.
1. There exist constants 0 < c0 < 1/2, such that c0 ≤ nρ ′ (v) ≤ 1 − c0 for any v = B( x)⊤ λ with
λ ∈ int(Θ). Also, there exist constants c 1 < c 2 < 0, such that c 1 ≤ nρ ′′ (v) ≤ c 2 < 0 in some
small neighborhood B of v∗ = B( x)⊤ λ† .
2. ǫ0 < π( x) < 1 − ǫ0 , ∀ x, for some constant 0 < ǫ0 < 1,
3. M := max ||B k ( x)||∞ /σk < ∞, where σk is the (population) standard deviation of B k ( X ), k =
1, ..., K .

Commenting on the previous assumptions, Assumption 3.1 is similar to Assumption 1.3. Assumption 3.2 is similar to the overlap condition of propensity scores. Both of them ensure the
quadratic margin condition required by the lasso oracle inequality (the quadratic margin condition
says in the ℓ∞ neighborhood of w0 the excess risk E is bounded from below by a quadratic function). Assumption 3.3 is similar to Assumption 1.4. It ensures the existence of the constant λ̄ > 0
in the theorem.
We further assume the following technical conditions.
Assumption 4. Assume the following technical conditions hold.
1. There exists η > 0 such that n||wλ∗ − w0 ||∞ ≤ η and n||wλ̃−w0 ||∞ ≤ η, where
λ̃ =

arg min
P
λ∈Θ: k σ k |λ−λ∗ |≤9E (wλ∗ )+675λ¯2 ||λ∗ ||0

{E (wλ) − 15λ̄

X

k:λ∗ 6=0

σk |λ − λ∗ |},

2. {log(2K )}1/2 n−1/2 M ≤ 0.13,
3. a n := {2 log(2K )}1/2 Mn−1/2 + log(2K ) Mn−1/2 ,
4. For some t > 0 we are free to set,
λ̄ := 4a n (1 + t{2(1 + 8a n M )}1/2 + 8 t2 a n M /3) > 6.4{log(2K )}1/2 n−1/2 ,

5. s > 0 solves a n (1 + s{2(1 + 2a n M )}1/2 + 2 t2 a n M /3) = 9/5,
6. α = exp(− na2n s2 ) + 7 exp(−4 na n t2 ).
The technical asumptions are all inherited from Theorem 2.2 of Van de Geer (2008).
The first technical assumption is needed because our quadractic margin condition E ( nwλ ) ≥
cn||wλ − w0 ||2 only holds locally for wη within the η neighborhood of w0 , ||wλ − w0 ||∞ ≤ η/ n.
The estimator λ̃ balances excess risk with being different from the oracle estimator λ∗ in the ℓ1
neighborhood of λ∗ .
27

The second technical assumption is to ensure the applicability of Bousquet’s inequality to the
empirical process induced by Z conditional
on
p
p X . The constant 0.13 is rather arbitrary; it could be
replaced by any constant smaller that ( 6 − 2)/2 if other constants are adjusted accordingly.
The third technical assumption on a n is due to the usual rate of decay in probability for Gaussian linear model with orthogonal design, resulting from a symmetrization inequality and a contraction inequality.
The fourth technical assumption on λ̄ is setting a lower bound for the smoothing parameter. It
also come from the Bousquet’s inequality. t is a parameter to be set by users; we need to strike
the balance between small excess risk due to small t and large confidence in the upper bound for
excess risk due to large t.
The fifth technical condition on s is due to the contraction inequality for the additional randomness in standard error of covariates σ̂k relative to the true standard deviation σk .
The sixth technical condition on α defines “with high probability” as with probability 1 − α
where α decays exponentially in n.
With these assumptions, we have the following theorem.
Theorem 5. Under Assumption 3 and Assumption 4, with probability at least 1 − α, we have
E (wλ† ) ≤ 3E ( nwλ∗ ) + 225λ̄2||λ∗ ||0 ,

and
X
k

†

σk |λk − λ∗k | ≤

1575λ̄ ∗
21
||λ ||0 ,
λ̄E ( nwλ∗ ) +
4
4

where λ̄ > 0 is a constant that depends on K .
Theorem 4 in Section 4.1 is a consequence of Theorem 5 and Assumption 1.
Theorem 5 is a consequence of Theorem 2.2 in Van de Geer (2008) where the oracle properties
for lasso estimators are established under general convex loss. We only need to show that the
assumptions for Theorem 5.1 imply the assumptions of Theorem 2.2 in Van de Geer (2008) so that
their conclusion applies.
When there are few binding covariate balancing constraints, ||λ∗ ||0 will be small. The theorem
then says that the excess risk of MABW is of the same order of that of the oracle estimator. Therefore, MABW mimics the exact balancing weights under a capped number of effective constraints.
In other words, resorting to approximation in covariate balancing enjoys a similar effect of capping
the number of effective balancing constraints. Hence, MABW is immune to the loss of excessive
balancing.
An important practical question is how many covariates we should balance. Exact balancing
weights can only balance a few covariates, because otherwise the problem does not admit a solution. MABW relieve this problem: we can balance much more covariates with δ appropriately
set. This oracle inequality says that we do not need to worry about excessive balancing in terms of
the order of magnitude of the loss. We only need to find an equilibrium between balancing many
28

covariates loosely and balancing a few covariates strictly. This amounts to setting δ appropriately,
which we address in Section 4.
Below we prove Theorem 5.
Proof. We only need to show assumptions L, B, and C in Theorem 2.2 of Van de Geer (2008) so
that their oracle inequality applies to our case.
First we show assumption L: the loss function is convex and Lipschitz. Our loss function writes
L w ( x, z) = − z · (ρ ◦ (ρ ′ )−1 ◦ nw)( x) + ((ρ ′ )−1 ◦ w)( x). Fixing z, we have
∂L w ( x, z)
∂w

= {− z · nw( x) + 1}{−

ρ ′′

n (ρ ′ )2

( nw( x))}.

This is bounded due to assumptions 3.1 and 3.3, implying the Lipschitz property: derivatives of ρ
and bounded, z is bounded by [0,1] and nw is bounded due to nρ ′ is bounded. The convexity of
the loss is shown in Appendix B of Chan et al. (2016).
We then show assumption B: the quadratic marginal condition. We compute the second derivative of EL w :
∂2 EL w ( X , Z )
∂ w2

= − π( x) · (( nρ ′ )−1 )′ nw( x) + {−π( x) nw( x) + 1}{(( nρ ′)−1 )′′ nw( x)}
≥π( x)

ρ ′′

(ρ ′ )

(
2

1
) + |η| · {((ρ ′ )−1 )′′ nw( x)}.
π( x)

This is lower bounded by a positive constant when η > 0 is small enough. This is ensured again by
Assumption 3.1, in particular the concavity of ρ . The last step is due to a Taylor expansion around
nw( x) = 1/π( x) in its η-neighborhood.
P
Lastly we show assumption C: k∈K σk |λk − λ̃k | ≤ |K | · ||wλ − wλ̃ ||. This is again ensured by
Assumption 3.1, in particular the boundedness of the first and second derivative.
The theorem then follows from Theorem 2.2 of Van de Geer (2008) where H = cu2 /2 and
G = u2 /(2 c) for some constant c > 0 due to the quadratic margin condition.

D Details on Empirical studies
D.1 Average treatment effect (ATE) estimates for the RHC simulation
Table 2 reports the RMSE of the ATE and average covariate balance C S across simulated data
sets for the ATE for the RHC simulation. When the data exhibits poor overlap, MABW gives
solutions when the exact balancing counterpart does not. With good overlap, MABW performs
similarly to its exact balancing counterpart and outperforms it sometimes.

29

δ sd

0
1e-3
1e-2
1e-1
1

Good Overlap
|w − w̄|

(w − w̄)2

(0.19, 0.36)
(0.19, 0.36)
(0.19, 0.34)
(0.17, 0.73)
(0.21, 1.92)

(0.20, 0.13)
(0.20, 0.13)
(0.20, 0.13)
(0.18, 0.52)
(0.23, 2.15)

δ sd

w log w

Bad Overlap
|w − w̄|

(w − w̄)2

w log w

(0.19, 0.15)
0
(0.19, 0.15) 1e-3
(0.19, 0.14) 1e-2
(0.17, 0.50) 1e-1 (0.24, 1.02) (0.20, 0.71) (0.21, 0.80)
(0.23, 2.17)
1 (0.22, 2.80) (0.24, 3.10) (0.25, 3.12)

(a) Good overlap

(b) Bad overlap

Table 2: (RMSE, C S ) on the ATE on the RHC data set. MABW gives solutions when the exact
balancing counterpart does not. The bolded numbers signal the optimal δ with the smallest RMSE
and the δ resulting in the best covariate balance over the bootstrap samples.

D.2 The Kang and Schafer Example
We next compare MABW to their exact balancing counterpart in the Kang and Schafer (2007)
iid
simulated example. This example consists of four unobserved covariates U i ∼ N (0, I 4), i =
1, ..., n. They are used to generate four covariates X i that are observed by the investigator: X i1 =
exp(U i1 /2), X i2 = U i2 /(1 + exp(U i1 )) + 10, X i3 = (U i1U i3 + 0.6)3 , and X i4 = (U i2 +U i4 + 20)2. There
is an outcome variable Yi generated by Yi = 210 + 27.4U i1 + 13.72U i2 + 13.7U i3 + 13.7U i4 + ǫ i
iid
where ǫ i ∼ N (0, 1), and an incomplete outcome indicator Z i generated as a Bernoulli random
variable with parameter p i = exp(−U i1 − 2U i2 − 0.25U i3 − 0.1U i4). This incomplete outcome indicator denotes whether the outcome is observed ( Z i = 1) or not ( Z i = 0).
Using this data generation mechanism, the mean difference of the observed covariates between
the complete and incomplete outcome data is of (-0.4, -0.2, 0.1, -0.1) standard deviations. We consider this the “good overlap” case. We also consider another case where the generating mechanism
of p i is slightly different: p i = exp(−U i1 − 0.5U i2 − 0.25U i3 − 0.1U i4). This makes covariate balance slightly worse, resulting in slightly larger mean differences of (-0.3, -0.5, -0.1, -0.4) standard
deviations. We consider this the “bad overlap” case.
Tables 3 and 4 presents the RMSE of the weighting mean estimates and the average covariate
balance C S over the bootstrap samples in both the good overlap and the bad overlap cases across
10000 simulated data sets. As expected, the approximate balancing weights perform considerably
better in the bad overlap case. The improvement is not as marked as we documented in the RHC
data set because the good and bad overlap cases do not differ much: the mean difference goes
from (-0.4, -0.2, 0.1, -0.1) in the good overlap to (-0.3, -0.5, -0.1, -0.4) in the bad overlap case.
With this relatively small change in covariate balance, MABW immediately outperform the exact
balancing weights in the bad overlap cases. This gives us an understanding of when we should use
MABW. We also observe that MABW can outperform the exact balancing weights in the good
30

δ sd

|w − w̄|

f ( w)
(w − w̄)2

δ sd

w log w

|w − w̄|

f ( w)
(w − w̄)2

w log w

0 (4.34, 0.94) (4.54, 0.84) (4.49, 0.88)
0 (8.39, 1.65) (8.42, 1.46) (8.58, 1.58)
1e-3 (5.77, 1.12) (6.10, 0.93) (5.92, 0.99) 1e-3 (7.74, 1.61) (7.65, 1.44) (7.70, 1.53)
1e-2 (6.54, 1.10) (7.13, 0.92) (6.72, 0.98) 1e-2 (9.22, 1.70) (8.49, 1.46) (8.80, 1.54)
1e-1 (10.15, 1.78) (11.27, 1.38) (11.04, 1.46) 1e-1 (11.52, 1.69) (11.36, 1.52) (11.19, 1.57)
1 (32.38, 2.76) (32.38, 2.76) (32.33, 2.76)
1 (33.30, 5.76) (34.04, 5.82) (33.98, 5.81)
(a) Good overlap

(b) Bad overlap

Table 3: (RMSE, C S ) on the mean unobserved outcome in the Kang-Schafer example.
δ sd

0
1e-3
1e-2
1e-1
1

|w − w̄|

f ( w)
(w − w̄)2

(4.49, 1.49)
(4.45, 1.25)
(4.42, 1.04)
(4.44, 0.97)
(6.11, 1.32)

(4.32, 1.45)
(4.41, 1.19)
(4.41, 0.94)
(4.49, 0.84)
(6.11, 1.32)

δ sd

w log w

|w − w̄|

f ( w)
(w − w̄)2

(4.41, 1.41)
0 (5.99, 2.15) (5.26, 1.93)
(4.44, 1.20) 1e-3 (3.83, 2.01) (3.59, 1.88)
(4.39, 0.97) 1e-2 (3.85, 1.96) (3.92, 1.80)
(4.43, 0.92) 1e-1 (4.13, 1.97) (4.28, 1.85)
(4.03, 2.74)
1 (5.80, 3.03) (5.81, 3.02)

(a) Good overlap

w log w

(5.53, 2.08)
(3.64, 1.90)
(3.96, 1.82)
(4.21, 1.91)
(3.96, 5.90)

(b) Bad overlap

Table 4: (RMSE, C S ) on the mean outcome in the Kang-Schafer example.
overlap case.

D.3 The LaLonde Data
Lastly, we study the performance of MABW versus exact balancing weights in the LaLonde data
set (LaLonde, 1986). This data set has two components: an experimental part from a randomized
experiment evaluating a large scale job training program (the National Supported Work Demonstration, NSW) on 185 participants; and an observational part, where the experimental control
group from the randomized experiment is replaced by a control group of 15992 of nonparticipants
drawn from the Current Population Survey (CPS). The experimental part provides a benchmark for
the effect of the job training program to be recovered from observational part of the data set. This
benchmark is $1794 for the ATT with a 95% confidence interval of [551, 3038].
Table 5 presents the ATT estimates and their 95% confidence intervals using MABW and its
exact balancing counterpart. We use δ·sd for different levels of approximate balancing. MABW
together with our tuning algorithm produces much more efficient mean ATT estimates while re-

31

δ sd

|w − w̄|

f ( w)
(w − w̄)2

w log w

0
(712(2602), 6.27) (1668(1076), 0.86)
1e-3 (1094(2450), 5.74) (1627(1062), 0.73)
1e-2 (2039(2263), 5.67)
(1481(986), 0.55) (1382(1078), 0.68)
1e-1
(744(1257), 1.47)
(1387(886), 0.52)
(745(935), 0.75)
1 (-2684(846),12.66) (-1828(801), 13.26) (-2714(824), 14.08)
Table 5: (mean(sd), C S ) of the ATT effect estimates and 95% confidence intervals in the LaLonde
data set.
maining close to the experimental target $1794 (the dotted line). The 95% confidence intervals all
contain the experimental 95% confidence interval and they become more efficient as δ increases.
When δ grows to as large as 1 sd, the ATT estimates starts to shift away from the target. This
is intuitive as overly large δ would imply we are no longer balancing the covariates. In this regard, we conclude MABW produce more efficient ATT estimates while being faithful to the truth
(experimental target).

32

