text:
Matching As An Econometric Evaluation Estimator
- Brandon Lane
- 3 years ago
- Views:
1 Review of Economic Studies (1998) 65, ^ /98/ $ The Review of Economic Studies Limited Matching As An Econometric Evaluation Estimator JAMES J. HECKMAN University of Chicago HIDEHIKO ICHIMURA University of Pittsburgh and PETRA TODD University of Pennsylvania First version received October 1994, final version accepted October 1997 (Eds.) This paper develops the method of matching as an econometric evaluation estimator. A rigorous distribution theory for kernel-based matching is presented. The method of matching is extended to more general conditions than the ones assumed in the statistical literature on the topic. We focus on the method of propensity score matching and show that it is not necessarily better, in the sense of reducing the variance of the resulting estimator, to use the propensity score method even if propensity score is known. We extend the statistical literature on the propensity score by considering the case when it is estimated both parametrically and nonparametrically. We examine the benefits of separability and exclusion restrictions in improving the efficiency of the estimator. Our methods also apply to the econometric selection bias estimator. 1. INTRODUCTION Matching is a widely-used method of evaluation. It is based on the intuitively attractive idea of contrasting the outcomes of programme participants (denoted Yi) with the outcomes of "comparable" nonparticipants (denoted yo). Differences in the outcomes between the two groups are attributed to the programme. Let /o and /i denote the set of indices for nonparticipants and participants, respectively. The following framework describes conventional matching methods as well as the smoothed versions of these methods analysed in this paper. To estimate a treatment effect for each treated person ie/i, outcome F,, is compared to an average of the outcomes Yoj for matched persons jelo in the untreated sample. Matches are constructed on the basis of observed characteristics X in R"". Typically, when the observed characteristics of an untreated person are closer to those of the treated person ieli, using a specific distance measure, the untreated person gets a higher weight in constructing the match. The estimated gain for each person i in the treated sample is where lvno,n,(i,j) is usually a positive valued weight function, defined so that for each '^^1' Z/e/ ^No.NXiJ) = ^' and No and A^i are the number of individuals in /o and /i, respectively. The choice of a weighting function reflects the choice of a particular distance measure used in the matching method, and the weights are based on distances in the X 261 (1)
2 262 REVIEW OF ECONOMIC STUDIES space. For example, for each ieli the nearest-neighhour method selects one individual jelo as the match whose Xj is the "closest" value to Z,, in some metric. The kernel methods developed in this paper construct matches using all individuals in the comparison sample and downweighting "distant" observations. The widely-used evaluation parameter on which we focus in this paper is the mean effect of treatment on the treated for persons with characteristics X. where Z)= 1 denotes programme participation. Heckman (1997) and Heckman and Smith (1998) discuss conditions under which this parameter answers economically interesting questions. For a particular domain ^ for X, this parameter is estimated by Lei, ^^o-nmiyu-y^j^,^ W^o.NXi,J)Yoj], (2) where different values of Wffo.NXi) may be used to select different domains ^ or to account for heteroskedasticity in the treated sample. Different matching methods are based on different weighting functions {w^,,^xi)} and {fvi^.^nxij}} The method of matching is intuitively appealing and is often used hy applied statisticians, but not by economists. This is so for four reasons. First, it is difficult to determine if a particular comparison group is truly comparable to participants (i.e. would have experienced the same outcomes as participants had they participated in the programme). An ideal social experiment creates a valid comparison group. But matching on the measured characteristics available in a typical nonexperimental study is not guaranteed to produce such a comparison group. The published literature presents conditional independence assumptions under which the matched group is comparable, but these are far stronger than the mean-independence conditions typically invoked by economists. Moreover, the assumptions are inconsistent with many economic models of programme participation in which agents select into the programme on the basis of unmeasured components of outcomes unobserved by the econometrician. Even if conditional independence is achieved for one set of X variables, it is not guaranteed to be achieved for other sets of X variables including those that include the original variables as subsets. Second, if a valid comparison group can be found, the distribution theory for the matching estimator remains to be established for continuously distributed match variables Z.' Third, most of the current econometric literature is based on separability between observables and unobservables and on exclusion restrictions that isolate different variables that determine outcomes and programme participation. Separability permits the definition of parameters that do not depend on unobservables. Exclusion restrictions arise naturally in economic models, especially in dynamic models where the date of enrollment into the programme differs from the dates when consequences of the programme are measured. The available literature on matching in statistics does not present a framework that incorporates either type of a priori restriction. Fourth, matching is a data-hungry method. With a large number of conditioning variables, it is easy to have many cells without matches. This makes the method impractical or dependent on the use of arbitrary sorting schemes to select hierarchies of matching variables. (See, e.g. Westat (1980,1982,1984).) In an important paper, Rosenbaum and Rubin (1983) partially solve this problem. They establish that if matching on X is valid, so is matching solely on the probability of selection into the prograrrune PT(D=\\X)- 1. When the match variables are discrete, the matching estimator for each cell is a mean and consistency and asymptotic normality of the matching estimator are easily established.
3 HECKMAN ET AL. MATCHING AS AN ESTIMATOR 263 P{X). Thus a multidimensional matching problem can be recast as a one-dimensional problem and a practical solution to the curse of dimensionality for matching is possible.^ Several limitations hamper the practical application of their theoretical result. Their theorem assumes that the probability of selection is known and is not estimated. It is also based on strong conditional independence assumptions that are difficult to verify in any application and are unconventional in econometrics. They produce no distribution theory for their estimator. In this paper we first develop an econometric framework for matching that allows us to incorporate additive separability and exclusion restrictions. We then provide a sampling theory for matching from a nonparametric vantage point. Our distribution theory is derived under weaker conditions than the ones currently maintained in the statistical literature on matching. We show that the fundamental identification condition of the matching method for estimating (P-1) is whenever both sides of this expression are well defined. In order for both sides to be well defined simultaneously for all X it is usually assumed that O<P(X)<1 so that Supp ix\d= 1) = Supp {X\D = G). As Heckman, Ichimura, Smith, Todd (1998), Heckman, Ichimura, Smith and Todd (19966) and Heckman, Ichimura and Todd (1997) point out, this condition is not appropriate for important applications of the method. In order to meaningfully implement matching it is necessary to condition on the support common to both participant and comparison groups S, where S=Supp {X\D= 1) n Supp and to estimate the region of common support. Equality of the supports need not hold a priori although most formal discussions of matching assumes that it does. Heckman, Ichimura, Smith and Todd (1998) and Heckman, Ichimura and Todd (1997) report the empirical relevance of this point for evaluating job training programmes. Invoking assumptions that justify the application of nonparametric kernel regression methods to estimate programme outcome equations, maintaining weaker mean independence assumptions compared to the conditional independence assumptions used in the literature, and conditioning on S, we produce an asymptotic distribution theory for matching estimators when regressors are either continuous, discrete or both. This theory is general enough to make the Rosenbaum-Rubin theorem operational in the commonly-encountered case where P^X) is estimated either parametrically or nonparametrically. With a rigorous distribution theory in hand, we address a variety of important questions that arise in applying the method of matching: (1) We ask, if one knew the propensity score, P{X), would one want to use it instead of matching on XI (2) What are the effects on asymptotic bias and variance if we use an estimated value of P? We address this question both for the case of parametric and nonparametric PiX). Finally, we ask (3) what are the benefits, if any, of econometric separability and exclusion restrictions on the bias and variance of matching estimators? The structure of this paper is as follows. Section 2 states the evaluation problem and the parameters identified by the analysis of this paper. Section 3 discusses how matching solves the evaluation problem. We discuss the propensity score methodology of Rosenbaum and Rubin (1983). We emphasize the importance ofthe common support condition 2. They term P(X) the propensity score. For the relationship between propensity score methods and selection models, see Heckman and Robb (1986) or Heckman, Ichimura, Smith and Todd (1998).
4 264 REVIEW OF ECONOMIC STUDIES assumed in the literature and develop an approach that does not require it. Section 4 contrasts the assumptions used in matching with the separability assumptions and exclusion restrictions conventionally used in econometrics. A major goal of this paper is to unify the matching literature with the econometrics literature. Section 5 investigates a central issue in the use of propensity scores. Even if the propensity score is known, is it better, in terms of reducing the variance of the resulting matching estimator, to condition on X or P(X)? There is no unambiguous answer to this question. Section 6 presents a basic theorem that provides the distribution theory for kernel matching estimators based on estimated propensity scores. In Section 7, these results are then applied to investigate the three stated questions. Section 8 summarizes the paper. 2. THE EVALUATION PROBLEM AND THE PARAMETERS OF INTEREST Each person can be in one of two possible states, 0 and 1, with associated outcomes (Jo, Yt), corresponding to receiving no treatment or treatment respectively. For example, "treatment" may represent participation in the social programme, such as the job training programme evaluated in our companion paper where we apply the methods developed in this paper. (Heckman, Ichimura and Todd (1997).) Let D= 1 if a person is treated; > = 0 otherwise. The gain from treatment is A= 7,- yo. We do not know A for anyone because we observe only Y=DYi + {l-d)yo, i.e. either yo or y,. This fundamental missing data problem cannot be solved at the level of any individual. Therefore, the evaluation problem is typically reformulated at the population level. Focusing on mean impacts for persons with characteristics X, a commonly-used parameter of interest for evaluating the mean impact of participation in social programmes is (P-1). It is the average gross gain from participation in the programme for participants with characteristics X. If the full social cost per participant is subtracted from (P-1) and the no treatment outcome for all persons closely approximates the no programme outcome, then the net gain informs us of whether the programme raises total social output compared to the no programme state for the participants with characteristics X.^ The mean E(Yt\D=l,X) can be identified from data on programme participants. Assumptions must be invoked to identify the counterfactual mean E(Yo\D=l,X), the no-treatment outcome of programme participants. In the absence of data from an ideal social experiment, the outcome of self-selected nonparticipants E(Yo\D = O,X) is often used to approximate E(Yo\D=l,X). The selection bias that arises from making this approximation is Matching on X, or regression adjustment of yo using X, is based on the assumption that B(X) = O so conditioning on X eliminates the bias. Economists have exploited the idea of conditioning on observables using parametric or nonparametric regression analysis (Barnow, Cain and Goldberger (1980), Barros (1986), Heckman and Robb (1985, 1986)). Statisticians more often use matching methods, pairing treated persons with untreated persons of the same X characteristics (Cochrane and Rubin (1973)). The literature on programme evaluation gives two distinct responses to the problem of estimating (P-1) with continuous conditioning variables. The first borrows from the 3. See Heckman (1997) or Heckman and Smith (1998) for a precise statement of when this parameter answers an mteresting economic evaluation question.
5 HECKMAN ET AL. MATCHING AS AN ESTIMATOR 265 kernel regression literature. It uses a smoothing procedure that borrows strength from adjacent values of a particular value of A'=x and produces uniformly consistent estimators of (P-1) at all points of the support for the distributions oi X given D=\ or D = 0. (See Heckman, Ichimura, Smith and Todd (1996a) or Heckman, Ichimura and Todd (1997).) Parametric assumptions about E(Yo\D=l,X) play the same role as smoothing assumptions, and in addition allow analysts to extrapolate out of the sample for X. Unless the class of functions to which (P-1) may belong is restricted to be smaller than the finiteorder continuously-diiferentiable class of functions, the convergence rate of an estimator of (P-1) is governed by the number of continuous variables included in X (Stone (1982)). The second response to the problem of constructing counterfactuals abandons estimation of (P-1) at any point of A' and instead estimates an average of (P-1) over an interval of X values. Commonly-used intervals include Supp (X i)=l), or subintervals of the support corresponding to different groups of interest. The advantage of this approach is that the averaged parameter can be estimated with rate A'^"'''^, where N is sample size, regardless of the number of continuous variables in X when the underlying functions are smooth enough. Averaging the estimators over intervals ofx produces a consistent estimator of M(5) = E(y,-yo i)=l,a'e5'), (P-2) with a well-defined iv~'^^ distribution theory where S is a subset of Supp {X\D=l). There is considerable interest in estimating impacts for groups so (P-2) is the parameter of interest in conducting an evaluation. In practice both pointwise and setwise parameters may be of interest. Historically, economists have focused on estimating (P-1) and statisticians have focused on estimating (P-2), usually defined over broad intervals of X values, including Supp (Jir jd= 1). In this paper, we invoke conditions sufficiently strong to consistently estimate both (P-1) and (P-2). 3. HOW MATCHING SOLVES THE EVALUATION PROBLEM Using the notation of Dawid (1979) let denote the statistical independence of (yo, Yi) and D conditional on X. An equivalent formulation of this condition is This is a non-causality condition that excludes the dependence between potential outcomes and participation that is central to econometric models of self selection. (See Heckman and Honore (1990).) Rosenbaum and Rubin (1983), henceforth denoted RR, establish that, when (A-1) and 0<P{X)<\, (A-2) are satisfied, (yo, Yi)]LD\P{X), where P(X) = PT ( >= 1 IA'). Conditioning on PiX) balances the distribution of yo and Yi with respect to D. The requirement (A-2) guarantees that matches can be made for all values of AT. RR called condition (A-1) an "ignorability" condition for D, and they call (A-1) and (A-2) together a "strong ignorability" condition. When the strong ignorability condition holds, one can generate marginal distributions of the counterfactuals Foiyo\D=l,X) and F,(y, > = O,Ar),
6 266 REVIEW OF ECONOMIC STUDIES but one cannot estimate the joint distribution of (yo, Yi), Fiyo,yi \D, X), without making further assumptions about the structure of outcome and programme participation equations.'* If PiX) = 0 or i'(a') = 1 for some values of Af, then one cannot use matching conditional on those X values to estimate a treatment effect. Persons with such X characteristics either always receive treatment or never receive treatment, so matches from both D = 1 and D = 0 distributions cannot be performed. Ironically, missing data give rise to the problem of causal inference, but missing data, i.e. the unobservables producing variation in D conditional on X, are also required to solve the problem of causal inference. The model predicting programme participation should not be too good so that PiX) = 1 or 0 for any X. Randomness, as embodied in condition (A-2), guarantees that persons with the same characteristics can be observed in both states. This condition says that for any measurable set A, Pr ixea ) = 1) > 0 if and only if Pr ixea ) = 0) >0, so the comparison of conditional means is well defined.' A major finding in Heckman, Ichimura, Smith, Todd (1996a, b, 1998) is that in their sample these conditions are not satisfied, so matching is only justified over the subset Supp (Af Z)= 1) n Supp ix\d = O). Note that under assumption (A-1) so E(yo Z>=l,Are5) can be recovered from E(yo Z) = 0,Ar) by integrating overat using the distribution ofx given D=l, restricted to S. Note that, in principle, both E(yo Af, D = 0) and the distribution of A' given D= 1 can be recovered from random samples of participants and nonparticipants. It is important to recognize that unless the expectations are taken on the common support of 5, the second equality does not necessarily follow. While EiYo\D = O,X) is always measurable with respect to the distribution ofat given D = 0, (/i(ar Z) = O)), it may not be measurable with respect to the distribution ofat given Z)= 1, (//(Af ) = 1)). Invoking assumption (A-2) or conditioning on the common support S solves the problem because nix\d = O) and //(Af )=l), restricted to S, are mutually absolutely continuous with respect to each other. In general, assumption (A-2) may not be appropriate in many empirical applications. (See Heckman, Ichimura, and Todd (1997) or Heckman, Ichimura, Smith and Todd (1996a, b, 1998).) The sample counterpart to the population requirement that estimation should be over a common support arises when the set S is not known. In this case, we need to estimate S. Since the estimated set, S, and S inevitably differ, we need to make sure that asymptotically the points at which we evaluate the conditional mean estimator ofeiyo\d = 0,X) are in S. We use the "trimming" method developed in a companion paper (Heckman, Ichimura, Smith, Todd (1996a)) to deal with the problem of determining the points in S. Instead of imposing (A-2), we investigate regions S, where we can reasonably expect to learn about E(yi- Yo\D=l, S). Conditions (A-1) and (A-2) which are commonly invoked to justify matching, are stronger than what is required to recover E(yi - yo )= 1, Af) which is the parameter of 4. Heckman, Smith and Clements (1997) and Heckman and Smith (1998) analyse a variety of such assumptions. 5. Thus, the implications of 0<Pr (D= l\x)<\ is that conditional measures of A" given Z) = O and that given D=\ are absolutely continuous with respect to each other. These dominating measure conditions are standard in the matching literature.
7 HECKMAN ET AL. MATCHING AS AN ESTIMATOR 267 interest in this paper. We can get by with a weaker condition since our objective is construction of the counterfactual E(yo Af, D = 1) (A-3) which implies that Pr(yo</ X>=l,A') = Pr(yo</ D = 0,Af) for Af65. In this case, the distribution of yo given X for participants can be identified using data only on nonparticipants provided that AfeS. From these distributions, one can recover the required counterfactual mean E(yo D= 1,A') for AfeS. Note that condition (A-3) does not rule out the dependence of D and Y] or on A = yi - yo given X. * For identification of the mean treatment impact parameter (P-1), an even weaker mean independence condition sufiices E(yo id=l,af) = E(yo > = 0,Af) for AfeS. (A-1') Under this assumption, we can identify EiYo\D= 1, X) for XeS, the region of common support.' Mean independence conditions are routinely invoked in the econometrics literature.* Under conditions (A-1) and (A-2), conceptually different parameters such as the mean effect of treatment on the treated, the mean effect of treatment on the untreated, or the mean effect of randomly assigning persons to treatment, all conditional on X, are the same. Under assumptions (A-3) or (A-1'), they are distinct.' Under these weaker conditions, we demonstrate below that it is not necessary to make assumptions about specific functional forms of outcome equations or distributions of unobservables that have made the empirical selection bias literature so controversial. What is controversial about these conditions is the assumption that the conditioning variables available to the analyst are sufficiently rich to justify application of matching. To justify the assumption, analysts implicitly make conjectures about what information goes into the decision sets of agents, and how unobserved (by the econometrician) relevant information is related to observables. (A-1) rules out dependence of D on yo and Yx and so is inconsistent with the Roy model of self selection. See Heckman (1997) or Heckman and Smith (1998) for further discussion. 4. SEPARABILITY AND EXCLUSION RESTRICTIONS In many applications in economics, it is instructive to partition X into two not-necessarily mutually exclusive sets of variables, (T, Z), where the T variables determine outcomes and the Z variables determine programme participation o, (3a) g i ) u (3b) Pr(Z)=l Af) = Pr( )=l Z) = /'(Z). (4) 6. By symmetric reasoning, if we postulate the condition YxlD\X and (A-2), then Vr(D=\\Y\,X) = Pr {D= 1 \X), so selection could occur on Jo or A, and we can recover VT(Yi<t\D=Q,X). Since?r{Ya<t\D = 0,X) can be consistently estimated, we can recover E{Y\- Ya\D=Q,X). 7. We can further identify E(y,- yol^=0) if we assume E(r, ^=l.^)=e(ki /)=O,;f) for A" in S. 8. See, for example, Barnow, Cain and Goldberger (1980) or Heckman and Robb (1985, 1986). 9. See Heckman (1990, 1997) for a discussion of the three parameters. See also Heckman and Smith (1998).
8 268 REVIEW OF ECONOMIC STUDIES Thus in a panel data setting y, and yo may be outcomes measured in periods after programme participation decisions are made, so that Z and T may contain distinct variables, although they may have some variables in common. Different variables may determine participation and outcomes, as in the labour supply and wage model of Heckman (1974). Additively-separable models are widely used in econometric research. A major advantage of such models is that any bias arising from observing yo or y, by conditioning on D is confined to the "error term" provided that one also conditions on T, e.g. EiYo\D = \,X)=goiT) + EiUo\D=\,Z) and E(y, i)= 1,Af)=g,(r)-t-E(:/, >= 1, Z). Another major advantage of such models is that they permit an operational definition ofthe effect of a change in T holding U constant. Such effects are derived from the ^o and ^i functions. The Rosenbaum-Rubin Theorem (1983) does not inform us about how to exploit additive separability or exclusion restrictions. The evidence reported in Heckman, Ichimura, Smith and Todd (1996a), reveals that the no-training earnings of persons who chose to participate in a training programme, yo, can be represented in the following way where Z and T contain some distinct regressors. This representation reduces the dimension of the matching or nonparametric regression if the dimension of Z is two or larger. Currently-available matching methods do not provide a way to exploit such information about the additive separability of the model or to exploit the information that Z and T do not share all of their elements in common. This paper extends the insights of Rosenbaum and Rubin (1983) to the widely-used model of programme participation and outcomes given by equations (3a), (3b) and (4). Thus, instead of (A-1) or (A-3), we consider the case where UoiLD\X. (A-4a) Invoking the exclusion restrictions PiX) = PiZ) and using an argument analogous to Rosenbaum and Rubin (1983), we obtain so that E{ ) C/o,/'(Z)} =E{E(Z) C/o, Af) I = E{P(Z) t/ UolD\PiZ). (A-4b) Under condition (A-4a) it is not necessarily true that (A-1) or (A-3) are valid but it is obviously true that In order to identify the mean treatment effect on the treated, it is enough to assume that E(t/o Z)=l,P(Z)) = E(C/o i) = 0,P(Z)), (A-4b') instead of (A-4a) or (A-4b). Observe that (A-4a), (A-4b), and (A-4b') do not imply that E(f/o /'(Z)) = 0 or that E(f/i li'(z)) = 0. They only imply that the distributions ofthe unobservables are the same in populations of Z)=l and Z) = 0, once one conditions on PiZ). Yo and y, must be adjusted to eliminate the effects of T on outcomes. Only the residuals can be used to exploit the RR conditions. Thus P(Z) is not, in general, a valid instrumental variable.
9 HECKMAN ET AL. MATCHING AS AN ESTIMATOR 269 In order to place these results in the context of classical econometric selection models, consider the following index model setup >=1 = 0 otherwise. If Z and V are independent, then P(Z) = F^(v/(Z)), where Fv( ) is the distribution function of v. In this case identification condition (A-4b') implies or when F^ is strictly increasing, LL (* oo C >ii(z) = \ I Uo J-oo 'v(z) If, in addition, y/iz) is independent of (f/o, v), and E(f/o) = 0, condition (*) implies ( oo I'V Uo, v)dvduo=0, for any v(z), which in turn implies EiUo\v = s) = O for any 5 when v^(z) traces out the entire real line. Hence under these conditions our identification condition implies there is no selection on unobservables as defined by Heckman and Robb (1985, 1986). However, ilfiz) may not be statistically independent of (f/o, v). Thus under the conditions assumed in the conventional selection model, the identification condition (A-4b') may or may not imply selection on unobservables depending on whether y/iz) is independent of (Co, v) or not. 5. ESTIMATING THE MEAN EFFECT OF TREATMENT: SHOULD ONE USE THE PROPENSITY SCORE OR NOT? Under (A-l') with S=SuppiX\D = \) and random sampling across individuals, if one knew EiYo\D = 0,X=x), a consistent estimator of (P-2) is where /i is the set of / indices corresponding to observations for which D, = 1. If we assume E(yol^=l,i'(A')) = E(yo Z> = 0,P(^))forZ6Supp(/'(^) Z)=l), (A-l") which is an implication of (A-l), and E(rol^ = O, PiX)=p) is known, the estimator is consistent for E(A D= 1). We compare the efficiency of the two estimators, Ap and Ax. We show that neither is necessarily more efficient than the other. Neither estimator is feasible because both assume the conditional mean function and PiX) are known whereas in practice they need
10 270 REVIEW OF ECONOMIC STUDIES to be estimated. However, the analysis of this case is of interest because the basic intuition from the simple theorem established below continues to hold when the conditional mean function and P{X) are estimated. and Theorem 1. Assume: (i) (A-l') and (A-l") hotdfor 5=Supp {X\D= 1); (ii) {^li,-^1 }/6/, cire independent and identicatty distributed; (iii) 0<E(F?) E(y?)<oo. Then Kx and Ap are both consistent estimators of (P-2) with asymptotic distributions that are normat with mean 0 and asymptotic variances Vx and Vp, respectively, where and (r, /)= 1, ^) Z)= 1]-H Var [E(y, - ro /)= 1, A') Z)= 1], Z)= 1,/>(X)) >= 1]-^ Var [E(r, - ro /)= 1, P(Z)) Z)= 1]. The theorem directly follows from the central limit theorem for iid sampling with finite second moment and for the sake of brevity its proof is deleted. Observe that E[Var (F, )= 1,Z) D= l]^e[var (r, /)= 1, i>(z)) Z)= 1], because X is in general a better predictor than P{X) but Var [E(y, - Y^\D= \,X)\D= 1]^ Var [El(y, - Yo) \D^\, P{X)) \D= 1], because vector X provides a finer conditioning variable than P{X). In general, there are both costs and benefits of conditioning on a random vector X rather than P{X). Using this observation, we can construct examples both where F^g Vp and where K^-^ Vp. Consider first the special case where the treatment effect is constant, that is E{Yi-Yo\D=\,X) is constant. An iterated expectation argument implies that E(y - yo Z)= 1, P{X)) is also constant. Thus, the first inequality, F^-g Vp holds in this case. On the other hand, if Yx = m{p{x)) + U for some measurable function m{ ) and / and X are independent, then which is non-negative because vector X provides a finer conditioning variable than P{X). So in this case Vx^Vp. When the treatment effect is constant, as in the conventional econometric evaluation models, there is only an advantage to conditioning on X rather than on P{X) and there is no cost.'" When the outcome 7, depends on X only through P(X), there is no advantage to conditioning on X over conditioning on P(X). Thus far we have assumed that P(X) is known. In the next section, we investigate the more realistic situation where it is necessary to estimate both P{X) and the conditional 10. Heckman (1992), Heckman and Smith (1998) and Heckman, Smith and Clements (1997) discuss the central role ofthe homogeneous response assumption in conventional econometric models of program evaluation.
11 HECKMAN ET AL. MATCHING AS AN ESTIMATOR 271 means. In this more realistic case, the trade-off between the two terms in Vx and Vp persists." When we need to estimate P{X) or E(yo Z) = 0, X), the dimensionality of the ^ is a major drawback to the practical application of the matching method or to the use of conventional nonparametric regression. Both are data-hungry statistical procedures. For high dimensional X variables, neither method is feasible in samples of the size typically available to social scientists. Sample sizes per cell become small for matching methods with discrete A"s. Rates of convergence slow down in high-dimensional nonparametric methods. In a parametric regression setting, one may evade this problem by assuming functional forms for E(t/o A') (see e.g. Barnow, Cain and Goldberger (1980) and the discussion in Heckman and Robb (1985, 1986)), but this approach discards a major advantage of the matching method because it forces the investigator to make arbitrary assumptions about functional forms of estimating equations. Conditioning on the propensity score avoids the dimensionality problem by estimating the mean function conditional on a one-dimensional propensity score P{X). However, in practice one must estimate the propensity score. If it is estimated nonparametrically, we again encounter the curse of dimensionality. The asymptotic distribution theorem below shows that the bias and the asymptotic variance of the estimator of the propensity score affects the asymptotic distribution of the averaged matching estimator more the larger the effect of a change in the propensity score on the conditional means of outcomes. 6. ASYMPTOTIC DISTRIBUTION THEORY FOR KERNEL-BASED MATCHING ESTIMATORS We present an asymptotic theory for our estimator of treatment effect (P-2) using either identifying assumption (A-l') or (A-4b'). The proof justifies the use of estimated P values under general conditions about the distribution of X. We develop a general asymptotic distribution theory for kemel-regression-based and local-polynomial-regression-based matching estimators of (P-2). Let Tand Z be not necessarily mutually exclusive subvectors of ^, as before. When a function depends on a random variable, we use corresponding lower case letters to denote its argument, for example, g(,t,p) = E{Yo\D=\, T=t,P{Z)=p) and P(z) = Pr (D= 1 Z=z). Although not explicit in the notation, it is important to remember that g{t, p) refers to the conditional expectation conditional on Z)= 1 as well as T=t and P(Z)=p. We consider estimators of g(r, i'(z)) where P{z) must be estimated. Thus we consider an estimator g^t, P{z)), where P{z) is an estimator of Piz). The general class of estimators of (P-2) that we analyse are of the form ^ NT'L.,[Yu-giTi,PiZd)]IiXieS) ^ ^'' (6) where I{A) = l if A holds and =0 otherwise and S is an estimator of S, the region of overlapping support, where S = Supp {X\D=\}r\Supp {X\D = O}. ^ To establish the properties of matching estimators of the form K based on different estimators of P(z) and g{t, Piz)), we use a class of estimators which we call asymptotically 11. If we knew E( y, Z) = 1, P(X )=p)as well, the estimator would be more efficient than Ap. In practical applications, we do not know either E(Yi\D=\,P{x)=p) or E{Yo\D = 0, P(X)=p) so this point is only a theoretical curiosum and is not investigated further.
12 272 REVIEW OF ECONOMIC STUDIES linear estimators with trimming. We analyse their properties by proving a series of lemmas and corollaries leading up to Theorem 2. With regard to the estimators P(z) and git,p), we only assume that they can he written as an average of some function of the data plus residual terms with appropriate convergence properties that are specified below. We start by defining the class of asymptotically linear estimators with trimming. Definition L An estimator Oix) of Oix) is an asymptotically linear estimator with trimming IixeS) if and only if there is a function y/ ", defined over some subset of a finite-dimensional Euclidean space, and stochastic terms Six) and Rix) such that for sample size n: (i) [9ix)-9ix)]IixeS) = n-' Yl^^ ^ / ixi, Yr,x) + Six (ii) E{yf ixl,yr,x)\x=x} = O, (iii) p\im ^^n-'/^x"=, KX,) = b<oo; An estimator ^ of )3 is called asymptotically linear if holds.'^ Definition 1 is analogous to the conventional definition, hut extends it infiveways to accommodate nonparametric estimators. First, since the parameter Oix) that we estimate is a function evaluated at a point, we need a notation to indicate the point x at which we estimate it. Conditions (i)-(iv) are expressed in terms of functions of x. Second, for nonparametric estimation, asymptotic linearity only holds over the support of X the region where the density is hounded away from zero. To define the appropriate conditions for this restricted region^ we introduce a trimming function IixeS) that selects ohservations only if they lie in S and discards them otherwise. Third, nonparametric estimators depend on smoothing parameters and usually have bias functions that converge to zero for particular sequences of smoothing parameters. We introduce a subscript n to the ^-function and consider it to he an element of a class of functions ^n, instead of a fixed function, in order to accommodate smoothing parameters. For example, in the context of kernel estimators, if we consider a smoothing parameter of the form ai^x) hn, different choices of /» generate an entire class of functions H'n indexed hy a function c( ) for any given kernel." We refer to the function Vn as a score function. The stochastic tenn^^(j:) is the hias term arising from estimation. For parametric cases, it often happens that Z>(jc) = O. Fourth, we change the notion of the residual term heing "small" from Op(/i~'^^) to the weaker condition (iv). We will demonstrate that this weaker condition is satisfied hy some nonparametric estimators when the stronger condition 0;,(n~'^^) is not. Condition (iii) is required to restrict the behaviour of the hias term. The hias term has to be reduced to a rate o(«'^^) in order to properly centre expression (i) asymptotically. For the case ofrf-dimensionalnonparametric model with/?-times continuously differentiable functions. Stone (1982) proves that the optimal uniform rate of convergence of the nonparametric regression function with respect to mean square error is («/log n)'''^^^'''"^. His result implies that some undersmoothing, compared to this optimal rate, is required to achieve the desired rate of convergence in the hias term alone. Note that the higher the dimension of the estimand, the more adjustment in smoothing parameters to reduce bias is required. 12. See Newey and McFadden (1994) p See Ichimura (1996).
13 HECKMAN ET AL. MATCHING AS AN ESTIMATOR 273 This is the price that one must pay to safeguard against possible misspecifications of g(?, p) or Piz). It is straightforward to show that parametric estimators of a regression function are asymptotically linear under some mild regularity conditions. In the Appendix we establish that the local polynomial regression estimator of a regression function is also asymptotically linear. A typical estimator of a parametric regression function mix; P) takes the form w(x; P), where m is a known function and P is an asymptotically linear estimator, with ^ n''^^^). In this case, by a Taylor expansion., P)-mix, p)] = n-'/' Y.%, [dmix, p)/8pwix,, 7,) + [dmix, P)/dp-8mix, ' where P lies on a line segment between P and p. When E{ii/iXi, Yi)}=0 and E{\i/iXi, Yi){i/iXi, Y,)'}<oo, under iid sampling, for example, n~'^^ej=i ViXi, Yi) = Opil) and plim ^o,p = P so that p\imr,^^\dmix, P)/dp-dmix, P)/8p\=0j,il) if dmix, P)/dp is Holder continuous at p.^* Under these regularity conditions ^[mix, p)-mix, )3)] = «- /'X-=, [^'«(^' P)/8PMX,, y,) + o^(l). The bias term of the parametric estimator mix, P) is Six) = 0, under the conditions we have specified. The residual term satisfies the stronger condition that is maintained in the traditional definition of asymptotic linearity. (a) Asymptotic linearity of the kernel regression estimator We now establish that the more general kernel regression estimator for nonparametric functions is also asymptotically linear. Corollary 1 stated below is a consequence of a more general theorem proved in the Appendix for local polynomial regression models used in Heckman, Ichimura, Smith and Todd (1998) and Heckman, Ichimura and Todd (1997). We present a specialized result here to simplify notation and focus on main ideas. To establish this result we first need to invoke the following assumptions. Assumption 1. Sampling of {Xi, 7,} is i.i.d., X, takes values in R!' and F, in R, and Var(y,)<oo. When a function is />-times continuously differentiable and its p-th derivative satisfies Holder's condition, we call the function/7-smooth. Let mix) = E{Yi\Xi=x}. Assumption 2. mix) is ^-smooth, where p>d. 14. A function is Holder continuous at A'=xo with constant 0< a g 1 if q^x, 0) - (p{xo, fl) gc \\x-xo\\ for some C> 0 for all x and 0 in the domain of the function cp(-,-). Usually Holder continuity is defined for a function with no second argument B. We assume that usual Holder continuity holds uniformly over 0 whenever there is an additional argument.
14 274 REVIEW OF ECONOMIC STUDIES We also allow for stochastic bandwidths: Assumption 3. Bandwidth sequence a, satisfies p\im ^^a /h = ao>0 for some deterministic sequence {h } that satisfies nh^/logn-yao and nhf-*c<ao for some c^o. This assumption implies 2^>rf but a stronger condition is already imposed in Assumption 2.'^ Assumption 4. Kernel function ^( ) is symmetric, supported on a compact set, and is Lipschitz continuous. The assumption of compact support can be replaced by a stronger assumption on the distribution of Xi so that all relevant moments exist. Since we can always choose Ki ), but we are usually not free to pick the distribution of Z,, we invoke compactness. In this paper we consider trimming functions based on S and S that have the following structure. hqifxix) be the Lebesgue density oix,, S= {xer'';fxix)^qo}, and S= {xer''; fxix)^qo}, where supxes\fxix)-fxix)\ converges almost surely to zero, and/;i-(jc) ispsmooth. We also require that fxix) has a continuous Lebesgue density j ^ in a neighbourhood of qo vnthffiqo) > 0. We refer to these sets S and S as p-nice on S. The smoothness of fxix) simplifies the analysis and hence helps to establish the equicontinuity results we utilize. Assumption 5. Trimming is ^-nice on S. In order to control the bias of the kernel regression estimator, we need to make additional assumptions. Certain moments of the kernel function need to be 0, the underlying Lebesgue density of Xt,fxix), needs to be smooth, and the point at which the function is estimated needs to be an interior point ofthe support of Z,. It is demonstrated in the Appendix that these assumptions are not necessary for ^-th order local polynomial regression estimator. Assumption 6. Kernel function Ki ) has moments of order 1 through p - 1 that are equal to zero. Assumption!. />(:>(:) is ^-smooth. Assumption 8. A point at which m(-) is being estimated is an interior point of the support of A',. The following characterization of the bias is a consequence of Theorem 3 that is proved in the Appendix. CoroUary 1. Under Assumptions 1-8, if Kiu^,..., Ud)=kiui) kiuj) where ki ) is a one dimensional kernel, the kernel regression estimator rhoix) of mix) is asymptotically 15. Assumption 3 implies /> - 0 and log n A ^^ "'' - 0. These two imply 2p > d. Also notice that the assumption implies K-^O.
15 HECKMAN ET AL. MATCHING AS AN ESTIMATOR 275 linear with trimming., where, writing e,= Yi-B{Yi\Xi}, and Wn(.Xi, Yi x) = (naohi)-'eiki{xi-x)/iaohr.))r{xes)/fx(x), Six) = Our use of an independent product form for the kernel function simplifies the expression for the bias function. For a more general expression without this assumption see the Appendix. Corollary 1 differs from previous analyses in the way we characterize the residual term. (b) Extensions to the case of local polynomial regression In the Appendix, we consider the more general case in which the local polynomial regression estimator for g(t, p) is asymptotically linear with trimming with a uniformly cqiisistent derivative. The latter property is useful because, as the next lemma shows, if both P{z) and g{t, p) are asymptotically linear, and if dg^t, p)/dp is uniformly consistent, then g^t, P(z)) is also asymptotically linear under some additional conditions. We also verify in the Appendix that these additional conditions are satisfied for the local polynomial regression estimators. Let P,(z) be a function that is defined by a Taylor's expansion pf i(r, P(z)) in the neighbourhood of P(z), i.e. g{t, P(,z))=g(,t, P(z)) + dg(t, P,(z))/dp [P(z)-P{z)l Lemma 1. Suppose that: (i) Both P(z) andg(t,p) are asymptotically linear with trimming where (ii) dg(t,p)/dp and P(z) are uniformly consistent and converge to dg{t,p)/dp and P(z), respectively and dg(t,p)/dp is continuous; (iii) plim -.oo «"'/' i;-, S,(T,, P(Z,)) = b, and plim«^««-'/' i;_, 8g(Ti, PiZ,))/8p b,it,, P(Z,)) = b,^; ^ (iv) plim^.oo «"'^' Z;-, lsg(t>, PT.iZi))/Sp-8g(Ti, P(Z,))/8p] (V) plim^^^ n-'^' i;^, i ; 8 d i j P{Zd)/ then g(t, P{z)) is also asymptotically linear where [g(t, Piz))-g{t, PizW(xeS) andli ~'^^ Z" MX) = n-' S;=, IVnAYj, Tj, P{Zj); t, P(z)) + dgit, P(z))/dp yifnp^dj, Zj; z)] -I-b{x) + R{x),
16 276 REVIEW OF ECONOMIC STUDIES An important property of this expression which we exploit below is that the effect of the kernel function v^n^ca, 'Zj\'^) always enters multiplicatively with dg{t, P{z))/dp. Thus both the bias and variance of P{z) depend on the slope to g with respect to p. Condition (ii) excludes nearest-neighbour type matching estimators with a fixed number of neighbours. With conditions (ii)-(v), the proof of this lemma is just an application of Slutsky's theorem and hence the proof is omitted. In order to apply the theorem, however, we need to verify the conditions. We sketch the main arguments for the case of parametric estimator P(z) of P{z) here and present proofs and discussion of the nonparametric case in the Appendix. ^ Under the regularity conditions just presented, the bias function for a parametric P{z) is zero. Hence condition (iii) holds '\fg(,t,p) is asymptotically linear and its derivative is uniformly consistent for the true derivative. Condition (iv) also holds since \Rp{Z,) = Op {n '/^) and the derivative oig{t, p) is uniformly consistent. Condition (v) can be verified by exploiting the particular form of score function obtained earlier. Observing that W{D Z; Z) v'(z) WpiiDj, Zf), we obtain, P{Zi))/dp] g(ti, P{Zi))/dp]xifp,{Zi)n-''^ Y.%, VpiiDj, Zj), so condition (v) follows from an application of the central limit theorem and the uniform consistency of the derivative of g(t,p). For the case of nonparametric estimators, Vnp does not factor and the double summation does not factor as it does in the case of parametric estimation. For this more general case, we apply the equicontinuity results obtained by Ichimura (1995) for general U- statistics to verify the condition. We verify all the conditions for the local polynomial regression estimators in the Appendix. Since the derivative of g{t,p) needs to be defined we assume Assumption 9. K() is 1-smooth. Lemma 1 implies that the asymptotic distribution theory of A can be obtained for those estimators based on asymptotically linear estimators with trimming for the general nonparametric (in P and g) case. Once this result is established, it can be used with lemma 1 to analyze the properties of two stage estimators of the form g(t, P(z)). (c) Theorem 2: The asymptotic distribution of the matching estimator under general conditions Theorem 2 enables us to produce the asymptotic distribution theory of a variety of estimators A of the treatment effect under different identifying assumptions. It also produces the asymptotic distribution theory for matching estimators based on different estimators of g(t,p) and P(z). In Sections 3 and 4, we presented various matching estimators for the mean effect of treatment on the treated invoking different identifying assumptions. An altemative to matching is the conventional index-sufficient selection estimators that can be used to construct estimators of E (yo /)= 1, A^), as described in our companion paper Heckman, Ichimura and Todd (1997) and in Heckman, Ichimura, Smith and Todd (1996a, 1998). Our analysis is sufficiently general to cover the distribution theory for that case as well provided an exclusion restriction or a distributional assumption is invoked.
17 HECKMAN ET AL. MATCHING AS AN ESTIMATOR 277 Denote the conditional expectation or variance given that A' is in S by Es ( ) or Var5( ), respectively. Let the number of observations in sets h and h be No and N\, respectively, where N=No + N^ and that 0<limAr_oo Ni/No=0<co. Theorem 2. Under the following conditions: (i) {yo,, Xi },e/o and {Yu, A', },e/, are independent and within each group they are i.i.d. and Yoi for ielo and Yufor ielx each has a finite second moment; (ii) The estimator g{x) of g{x) = ^{Yo\Di=\,Xi=x] is asymptotically linear with trimming., where Mix)-g{x)]I{xeS} = No'!,,, ^fon^sx J'o,,Xi; x) and the score functions y/dnon,(yd,x; x) for d=0 and 1, the bias term bgix), and the trimming function satisfy: (ii-a) E {Vrf^vo^v,(y</MXi;X)\Di = d,x,d=\)}=ofor d=0 and 1, and Var {ii/cinonxydi,xi,x)} = oin) for each ielou/, ; (ii-b) pum^v.^co A^r'^' I,,,, kxi) = b; (ii-c) plim^,^<:o Var {E[y/oNoNXYo>,Xi;X)\ Yoi, Di = O,Xi,D=l] />= 1} = Ko<oo plim^v.^oo Var {E[y/,!,,^,iYu, Xi; X) \ Yu, Di= 1, Xi, D= 1] Z)= 1} = K, < co, and lim^v.^co E{[Yu-giXi)]IiXieS) lxd (ii-d) S and S are p-nice on S, where p>d, where d is the number of regressors in X and fix) is a kernel density estimator that uses a kernel function that satisfies Assumption 6. Then under (A-T) the asymptotic distribution of is normal with mean ib/pr ixes\d=l)) and asymptotic variance YAT, PiZ), D= + PriXeS\D=l)-^{V,+2- Proof. See the Appendix. Theorem 2 shows that the asymptotic variance consists of five components. The first two terms are the same as those previously presented in Theorem 1. The latter three terms are the contributions to variance that arise from estimating gix) = E{Yi\Di=l,Xi=x}. The third and the fourth terms arise from using observations for which D = 1 to estimate gix). If we use just observations for which D = 0 to estimate gix), as in the case of the simple matching estimator, then these two terms do not appear and we only acquire the
18 278 REVIEW OF ECONOMIC STUDIES fifth term.'* We consider the more general case with all five terms. If A^o is much larger than AT,, then the sampling variation contribution of the Z) = 0 observations is small as 6 is small. Condition (i) covers both random and choice-based sampling and enables us to avoid degeneracies and to apply a central limit theorem. Condition (ii) elaborates the asymptotic linearity condition for the estimator of gix). We assume/7-nice trimming. The additional condition on the trimming function is required to reduce the bias that arises in estimating the support. Note that there is no need for gix) to be smooth. A smoothness condition on gix) is used solely to establish asymptotic linearity of the estimator of gix). Also note that the sampling theory above is obtained under mean independence Strong ignorability conditions given by (A-l), (A-2) or (A-3), while conventional in the matching literature, are not needed but they obviously imply these equalities. Theorem 2 can be combined with the earlier results to obtain an asymptotic distribution theory for estimators that use git, Piz)). One only needs to replace functions iyoi>xi; x) and iifin^xyuyxr, x) and the bias term by those obtained in Lemma ANSWERS TO THE THREE QUESTIONS OF SECTION 1 AND MORE GENERAL QUESTIONS CONCERNING THE VALUE OF A PRIORI INFORMATION Armed with these results, we now investigate the three questions posed in the Section 1. il) Is it better to match on PiX) or X if you know Matching on X, Ax involves ^-dimensional nonparametric regression function estimation whereas matching on PiX), Kp only involves one dimensional nonparametric regression function estimation. Thus from the perspective of bias, matching on PiX) is better in the sense that it allows ^A^-consistent estimation of (P-2) for a wider class of models than is possible if matching is performed directly on X. This is because estimation of higherdimensional functions requires that the underlying functions be smoother for bias terms to converge to zero. If we specify parametric regression models, the distinction does not arise if the model is correctly specified. When we restrict consideration to models that permit >/F-consistent estimation either by matching on PiX) or on X, the asymptotic variance of Kp is not necessarily smaller than that of Kx. To see this, consider the case where we use a kernel regression for the D = 0 observations i.e. those with ielo. In this case the score function w (Y..., ^_ WONoN, ( -fol, ^i, X) where,= yo,-e{ I'o/IA',, A = 0} and we write/a-(x Z) = O) for the Lebesgue density of X, given Z), = 0. (We use analogous expressions to denote various Lebesgue densities.) Clearly V] and Covi are zero in this case. Using the score function we can calculate Ko when we match on X. Denoting this variance by Vox, 16. An earlier version ofthe paper assumed that only observations for which ) = 0 are used to estimate 8(x).
19 HECKMAN ET AL. MATCHING AS AN ESTIMATOR 279 M' J J Now observe that conditioning on Xi and Fo,, e, is given, so that we may write the last expression as Now I r^((^r:^)^^^mf^ 1 ij r UjxiX\D = O)JKiu)du^ J' I r/.((x-^)/a.)/(x.5) 1 d' j can be written in the following way, making the change of variable ix/- anow\d = 0) Taking limits as No-^oa, and using assumptions 3, 4 and 7, so we can take limits inside the integral since OATO^O and ^Kiw)dw/lKiu)du= 1. Thus, since we sample the Xi for which A = 0, L./A-(.^/IM "J Hence the asymptotic variance of AA- is, writing A=Pr {A'e5 Z) = 0}/Pr (.YeS' )= 1), Pr ixes\d = l)-'{yats [EsiY,- Yo\X, D= 1)\D= l] + Es[Vars(F, \X, Z)= Similarly for Ap, VOP is The first two terms for both variance expressions are the same as those that appear in Vx and Vp in Theorem 1. To see that one variance is not necessarily smaller than the other, consider the case where fxix\d=l)=fxix\d = O) and 9 = 1. Clearly in this case fpipix)\d= l)=fpipix)\d = O). Propensity score matching has smaller variance if and
20 280 REVIEW OF ECONOMIC STUDIES only if Es{EsiY,\PiX),D=l)EsiYo\PiX),D=l)\D=l} >Es{EsiYt\X,D=l)EsiYo\X,D=l)\D=\}. Since the inequality does not necessarily hold, the propensity score matching estimator in itself does not necessarily improve upon the variance ofthe regular matching estimator."'* (2) What are the effects on asymptotic bias and variance if we use an estimated value of F>. When Pix) is estimated nonparametrically, the smaller bias that arises from matching on the propensity score no longer holds true if estimation of Pix) is a ^/-dimensional nonparmetric estimation problem where rf> 1. In addition, estimation of Pix) increases the asymptotic variance. Lemma 1. informs us that the score, when we use estimated Piz) but no other conditioning variables, is for ieij,d=o,\, where \i/dnon,g are the scores for estimating gip) and IJ/NP is the score for estimating Piz). By assumption (ii-a) they are not correlated with dgipiz))/8p- WspiDj, Zj; z), and hence the variance ofthe sum ofthe scores is the sum ofthe variances of each score. So the variance increases by the variance contribution of the score SgiPiz))/dp y/npidj, Zjiz) when we use estimated, rather than known, Piz). Even with the additional term, however, matching on X does not necessarily dominate matching on PiX) because the additional term may be arbitrarily close to zero when dgipiz))/8p is close to zero. (3) What are the benefits, if any, of econometric separability and exclusion restrictions on the bias and variance of matching estimators? We first consider exclusion restrictions in the estimation of Pix). Again we derive the asymptotic variance formulae explicitly using a kernel regression estimator. Using Corollary 1, the score function for estimating Pix) is a%fxix)\kiu)du uj=dj-e{dj\xj}. Hence the variance contribution of estimation of Piz) without imposing exclusion restrictions is V2x= lim y&r{e[dgipiz))/dp N*co Kiu)du]\Dj,Xj,D=l]} idj\xj)[dgipizj))/dp]' -flixjld^ l)/f%ixj)][pt {XjeS}]-\ Analogously, we define the variance contribution of estimating P(z) imposing exclusion restrictions by Viz. Observe that when Z is a subset of the variables in X, and when there 17. FoTexamp\eEs(Y,\X,D\) Es{Yo\X,D=\)oTEs(Y,\X,D=\) = -Es(Yo\X,D=\)ainho\d. 18. In a later analysis, Hahn (1996) considers a special case of the models considered in this paper and shows in a model with no exclusion restrictions that when P is not known, the estimated propensity score estimator is efficient and that knowledge of P improves efficiency. '
Some Practical Guidance for the Implementation of Propensity Score Matching
DISCUSSION PAPER SERIES IZA DP No. 1588 Some Practical Guidance for the Implementation of Propensity Score Matching Marco Caliendo Sabine Kopeinig May 2005 Forschungsinstitut zur Zukunft der Arbeit InstituteMore information
Misunderstandings between experimentalists and observationalists about causal inference
J. R. Statist. Soc. A (2008) 171, Part 2, pp. 481 502 Misunderstandings between experimentalists and observationalists about causal inference Kosuke Imai, Princeton University, USA Gary King Harvard University,More information
THE PROBLEM OF finding localized energy solutions
600 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 45, NO. 3, MARCH 1997 Sparse Signal Reconstruction from Limited Data Using FOCUSS: A Re-weighted Minimum Norm Algorithm Irina F. Gorodnitsky, Member, IEEE,More information
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 52, NO. 4, APRIL 2006 1289. Compressed Sensing. David L. Donoho, Member, IEEE
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 52, NO. 4, APRIL 2006 1289 Compressed Sensing David L. Donoho, Member, IEEE Abstract Suppose is an unknown vector in (a digital image or signal); we plan toMore information
Sample Attrition Bias in Randomized Experiments: A Tale of Two Surveys
DISCUSSION PAPER SERIES IZA DP No. 4162 Sample Attrition Bias in Randomized Experiments: A Tale of Two Surveys Luc Behaghel Bruno Crépon Marc Gurgand Thomas Le Barbanchon May 2009 Forschungsinstitut zurMore information
Steering User Behavior with Badges
Steering User Behavior with Badges Ashton Anderson Daniel Huttenlocher Jon Kleinberg Jure Leskovec Stanford University Cornell University Cornell University Stanford University firstname.lastname@example.com {dph,More information
An Introduction to Regression Analysis
The Inaugural Coase Lecture An Introduction to Regression Analysis Alan O. Sykes * Regression analysis is a statistical tool for the investigation of relationships between variables. Usually, the investigatorMore information
BIS RESEARCH PAPER NO. 112 THE IMPACT OF UNIVERSITY DEGREES ON THE LIFECYCLE OF EARNINGS: SOME FURTHER ANALYSIS AUGUST 2013 1 THE IMPACT OF UNIVERSITY DEGREES ON THE LIFECYCLE OF EARNINGS: SOME FURTHERMore information
How to Use Expert Advice
NICOLÒ CESA-BIANCHI Università di Milano, Milan, Italy YOAV FREUND AT&T Labs, Florham Park, New Jersey DAVID HAUSSLER AND DAVID P. HELMBOLD University of California, Santa Cruz, Santa Cruz, CaliforniaMore information
Inequality and Growth: What Can the Data Say?
Inequality and Growth: What Can the Data Say? Abhijit V. Banerjee and Esther Duflo June 2003 Abstract This paper describes the correlations between inequality and the growth rates in crosscountry data.More information
An Introduction into the SVAR Methodology: Identification, Interpretation and Limitations of SVAR models
Kiel Institute of World Economics Duesternbrooker Weg 120 24105 Kiel (Germany) Kiel Working Paper No. 1072 An Introduction into the SVAR Methodology: Identification, Interpretation and Limitations of SVARMore information
All you need is LATE Primary Job Market Paper
All you need is LATE Primary Job Market Paper Clément de Chaisemartin November 2012 Abstract Instrumental variable (IV) is a standard tool to measure the effect of a treatment. However, it relies on aMore information
Controllability and Observability of Partial Differential Equations: Some results and open problems
Controllability and Observability of Partial Differential Equations: Some results and open problems Enrique ZUAZUA Departamento de Matemáticas Universidad Autónoma 2849 Madrid. Spain. email@example.com information
If You re So Smart, Why Aren t You Rich? Belief Selection in Complete and Incomplete Markets
If You re So Smart, Why Aren t You Rich? Belief Selection in Complete and Incomplete Markets Lawrence Blume and David Easley Department of Economics Cornell University July 2002 Today: June 24, 2004 TheMore information
The Capital Asset Pricing Model: Some Empirical Tests
The Capital Asset Pricing Model: Some Empirical Tests Fischer Black* Deceased Michael C. Jensen Harvard Business School firstname.lastname@example.com and Myron Scholes Stanford University - Graduate School of BusinessMore information
Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning
Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning Richard S. Sutton a, Doina Precup b, and Satinder Singh a a AT&T Labs Research, 180 Park Avenue, Florham Park,More information
Communication Theory of Secrecy Systems
Communication Theory of Secrecy Systems By C. E. SHANNON 1 INTRODUCTION AND SUMMARY The problems of cryptography and secrecy systems furnish an interesting application of communication theory 1. In thisMore information
Not Only What but also When: A Theory of Dynamic Voluntary Disclosure
Not Only What but also When: A Theory of Dynamic Voluntary Disclosure By ILAN GUTTMAN, ILAN KREMER, AND ANDRZEJ SKRZYPACZ We examine a dynamic model of voluntary disclosure of multiple pieces of privateMore information
Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods
SIAM REVIEW Vol. 45,No. 3,pp. 385 482 c 2003 Society for Industrial and Applied Mathematics Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods Tamara G. Kolda Robert MichaelMore information
Scalable Collaborative Filtering with Jointly Derived Neighborhood Interpolation Weights
Seventh IEEE International Conference on Data Mining Scalable Collaborative Filtering with Jointly Derived Neighborhood Interpolation Weights Robert M. Bell and Yehuda Koren AT&T Labs Research 180 ParkMore information
Journal of the Royal Statistical Society. Series B (Methodological), Vol. 39, No. 1. (1977), pp. 1-38.
Maximum Likelihood from Incomplete Data via the EM Algorithm A. P. Dempster; N. M. Laird; D. B. Rubin Journal of the Royal Statistical Society. Series B (Methodological), Vol. 39, No. 1. (1977), pp. 1-38.More information
I Want to, But I Also Need to : Start-Ups Resulting from Opportunity and Necessity
DISCUSSION PAPER SERIES IZA DP No. 4661 I Want to, But I Also Need to : Start-Ups Resulting from Opportunity and Necessity Marco Caliendo Alexander S. Kritikos December 2009 Forschungsinstitut zur ZukunftMore information
Indexing by Latent Semantic Analysis. Scott Deerwester Graduate Library School University of Chicago Chicago, IL 60637
Indexing by Latent Semantic Analysis Scott Deerwester Graduate Library School University of Chicago Chicago, IL 60637 Susan T. Dumais George W. Furnas Thomas K. Landauer Bell Communications Research 435More information
WHICH SCORING RULE MAXIMIZES CONDORCET EFFICIENCY? 1. Introduction
WHICH SCORING RULE MAXIMIZES CONDORCET EFFICIENCY? DAVIDE P. CERVONE, WILLIAM V. GEHRLEIN, AND WILLIAM S. ZWICKER Abstract. Consider an election in which each of the n voters casts a vote consisting ofMore information
EVALUATION OF GAUSSIAN PROCESSES AND OTHER METHODS FOR NON-LINEAR REGRESSION. Carl Edward Rasmussen
EVALUATION OF GAUSSIAN PROCESSES AND OTHER METHODS FOR NON-LINEAR REGRESSION Carl Edward Rasmussen A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy, GraduateMore information
Inference with Difference in Differences and Other Panel Data
Inference with Difference in Differences and Other Panel Data Stephen G. Donald University of Texas at Austin and Kevin Lang Boston University and NBER Abstract We examine inference in panel data whenMore information
Some Sharp Performance Bounds for Least Squares Regression with L 1 Regularization
Some Sharp Performance Bounds for Least Squares Regression with L 1 Regularization Tong Zhang Statistics Department Rutgers University, NJ firstname.lastname@example.com Abstract We derive sharp performance boundsMore information
Federal Reserve Bank of Minneapolis. Are Forecasting Models Usable for Policy Analysis? (p. 2) Gresham's Law or Gresham's Fallacy? (p.
Federal Reserve Bank of Minneapolis Are Forecasting Models Usable for Policy Analysis? (p. 2) Christopher A. Sims Gresham's Law or Gresham's Fallacy? (p.17) Arthur J. Rolnick Warren E. Weber 1985 ContentsMore information
The Mystery of the Vanishing Benefits: An Introduction to Impact Evaluation
the world bank economic review, vol. 15, no. 1 115 140 The Mystery of the Vanishing Benefits: An Introduction to Impact Evaluation Martin Ravallion This article provides an introduction to the conceptsMore information
EVIDENCE FROM THE U. S. HOUSE* DAVID S. LEE ENRICO MORETTI MATTHEW J. BUTLER There are two fundamentally different views of the role of elections in policy formation. In one view, voters can affect candidatesMore information


text:
ProductsFeatured BooksCards, Posters & More...Book PackagesBooks by Topic
Accelerate Powerful LeadershipAccelerate Team PerformanceAccelerate Inspired Sales & ServiceAccelerate Personal GrowthAccelerate Educational Leadership
Themed Sets ofInspirationalProducts
Nature of ExcellenceBook Package
Contact Info:
P.O. Box 764087
Dallas, TX 75376
Fax: 972.274.2884
Please contact email@example.com to set up a time to work with your webmaster to add The CornerStone Leadership products to your website. For questions about The CornerStone Leadership Affiliate program review our Affiliate FAQ page.

subset: commoncrawl


text:
- Back to Home »
- Classroom , Display , Teacher , Year One »
- Progress
I have been naughty and avoiding this but I have got some updates on the way soon. Been manic busy during the new term at school getting room sorted and observations etc. It is a stress but its part and parcel of what I do. My new class are a challenging bunch but the praise I have received so far has brought a smile to my face.
I do hope to post some pictures of my classroom displays and layout and pass on why I have done things as I think it would be very useful to other teachers and any feedback is more than welcome!
Talk Soon!


text:
Alfa Romeo 164 #2461182164 - Music/164 Illust/takumi torigoe 「天ノ弱(ama no jaku)」 has a double meaning in japanese 天邪鬼(ama no jaku)=a contrary person. Year 164 was a leap year starting on Saturday (link will display the full calendar) of the Julian calendar.At the time, it was known as the Year of the Consulship of Macrinus and Celsus (or, less frequently, year 917 Ab urbe condita).The denomination 164 for this year has been used since the early medieval period, when the Anno Domini calendar era became the prevalent method in Europe for. The Volvo 164 is a 4-door, 6-cylinder luxury sedan unveiled by Volvo at the Paris Motor Show early in October 1968 and first sold as a 1969 model. 46,008 164s were built before the car was superseded by the 264 in 1975. The 164 was Volvo's first venture into the luxury segment since the end of PV 60 production in 1950, and was the first six-cylinder Volvo since the PV800 last produced in 1958..
"Notwithstanding subparagraph (A) [amending this section], any amount paid or accrued in a taxable year beginning after December 31, 1976, to the Atomic Energy Commission or its successors for municipal-type services shall be allowed as a deduction under section 164 if such amount would have been deductible by reason of section 164(f) (as in effect for a taxable year ending on December 31. In the autumn of 1968 Volvo presented the Volvo 164. It was a development of the Volvo 144 but had a more prestige-oriented position. The principle difference in technical terms was that the Volvo 164 had a 3.0-liter, in-line, 6-cylinder engine under the hood.. ABOUT PRODUCER: 164 (read Ichirokuyon, イチロクヨン), also known as Hiroshi (ひろし), is an acknowledged "VOCAROCK" producer.His name of his personal circle, the 203soundworks, is made up of his producer name of 164 plus the pronunciation of Miku's name (39, the character "mi" (ミ) is read as three and the character "ku" (ク) as nine Japanese), creating 203..
Title 45 published on 07-Dec-2018 03:47. The following are ALL rules, proposed rules, and notices (chronologically) published in the Federal Register relating to 45 CFR Part 164 after this date.. 45 CFR 164 - Protection of Electronic Protected Health Information, Breach of Unsecured Protected Health Information and Individually Identifiable Health Information.. Gain access to pertinent information only relevant to the IBEW Local 164 membership. In this section, we will post sensitive information that you will need to be aware of..
HHS announces a final rule that implements a number of provisions of the HITECH Act to strengthen the privacy and security protections for health information established under HIPAA..
Kubota Service Manuals For L245dt Tractor
Alfa Romeo 164 - Wikipedia
Alfa Romeo 164 - Wikipedia
Kuhn Gmd 40 Parts Manual
1989 Alfa Romeo 164 brochure
1989 Alfa Romeo 164 brochure
Kubota Workshop Manual Head Bolt Torque V2203
Image #4 - VOLVO 164 Volvo 164. >
Image #4 - VOLVO 164 Volvo 164. >
Kubota Tractor Service Manual M7580
A Brief History & Look at the Grumman G-164 Ag-Cat - grumman-g-164-ag-cat-04
A Brief History & Look at the Grumman G-164 Ag-Cat - grumman-g-164-ag-cat-04


text:
More information
FINANCING - Complete the form below and we will contact you shortly.
Financing in Grande Prairie
If you're unsure if you can qualify for car or truck financing in Alberta or British Columbia, no need to worry when you come to a Revolution Auto Group dealership for your car loan needs!
Even if banks and lending institutions are turning you away, you'll find a positive response when you speak with a Revolution Auto Group financing and credit specialist. When you apply for credit approval, even if you have bad credit or no credit built up, we can find the right solution for your needs, and one that won't cost you an arm and a leg or have you on a budget you can't deal with!
Second Chance Credit to Rebuild Your Credit
No matter your credit history or if you are currently suffering through a divorce, bankruptcy or other situation, and no matter if you're self-employed or under-employed, we can get you approved! We work with our extensive network of lending partners to find you the best deal, so all you need to worry about is which vehicle to choose for your needs. Join the Revolution in car financing today!
Leasing Vs. Financing - How Do I Choose?
It can be difficult to decide between leasing and financing a car. When customers ask us whether they should lease or finance, there are a number of factors to consider. While leasing payments are generally lower than finance payments on a comparable vehicle, you will not own your leased vehicle unless you decide to purchase it at your lease-end.
When considering leasing a vehicle, remember that leased vehicles generally come with a yearly kilometer allowance. If you travel further than this distance you must purchase additional kilometers at the time you sign your lease agreement.
Most leases only require an initial payment of your first month's lease and a refundable security deposit, while a down payment may be required if you are financing a vehicle. At the end of your lease, you have the option of returning your vehicle and leasing a new one (great for people who want a new vehicle every few years), returning your leased vehicle and walking away, or purchasing your leased vehicle.


text:
Hi, I'm Tess. (If that's news to you, there's an FAQ.)
I work on Web standards for the WebKit team at Apple.
I live in SF with my wife Erin, our daughter Erin, and our dog Rusty.
You can find me on Twitter, Instagram, Facebook, and elsewhere.
Copyright © 1998–2018 Theresa O'Connor. CC BY-SA 3.0.


text:
Featuring Sean Black]
Cause we gonna do it like this nigga (for 9 6) My motherfuckin' team
representin' nigga Fuck y'all niggas talkin' about? Where A&D clique
youknowhatI'msayin'? We gonna do it Carlito's way
"Nigga Fuck that" Buckshot 'Black Moon'
Verse 1:
Flippin' it over like the Illmatic one this rap addict
Sean Black be a freestyle fanatic
Sippin' Nas and all that plus I be
One of America's most B
Lthe UNTED blunted forever
Styles change up like the weather
Rugged like a Norface my flow is smoother then your leather so
Open it up son it's too gettin' hot
Matter of fact take it off your back before you get shot
My crew is strapped down ready to go to war and you don't need it
(so just chill sit back) Spark up an L and get weeded while I prodeed
Internationally to make your brains bleed
(You ain't really got no flavor) plus your girl brill is keyed
Don't ask me why I'm always flippin' on these wack rappers
Cause they fake thug niggas wanna-be gat clappers
Talkin' about this and that but always on beats
Cause all that gangsta shit is dead nigga let's take it to the streets
Yo if you real keep it real and if you fake burn a roada (why?)
Cause the niggas I'm runnin' with will attack you from your throat up
"Whatever I see I attack"-Buckshot (x4)
"Get off my back nigga fuck dat"-Buckshot
Verse 2:
A-yo check one two for the murderous gun crew
Three four is for the raw lyrical I got in store
For them next niggas frontin' like they ill but chill
Actin' like they bunch of criminals cause they pack steal
But that's aiight niggas do what they gotta do to get the cream
(Step to the wrong team)
And all your cash thoughts will be a dream
N'awmean? Cause you insist of gettin' me vex
Fuck it my rhymes is maxed now who's next to show techs on the el
Your crew is weak and they rhymes sound infected
Tryin'to get a car track get your whole fam rejected
It's the black attack trying to get cash like John Gotti
Still gettin' my loot and lettin' my lyrics catch the body
So when nothin' change everything remains the same
The nigga that's known for puttin' SOP stickers all over the train
(What? Rewind that back)
And let me flip this track


text:
Wine Advocate Score
Submit a Critic ScoreClose
Bodegas Muga Tempranillo Blend Rioja Aro 2000
Winemaker's Notes:
Tasting Elegant red colour with violet hues which hint at a well preserved youthfulness. In the aroma there is a subtle mingling of fruit nuances of ripe red berries with hints of spices. This aroma is intense, clean and subtle. On the palate it has lots of flavour, with an elegant attack, evolving into a meaty mouth-feel. Its powerful, though modulated tannins carry the wine onwards into a long, promising finish. A complex, supple, enjoyable wine with excellent potency. Bottling After the ageing the wine is lightly clarified and it is bottled without filtering. The cork used for ARO is one of the most expensive in the world. The cork selection has been a huge effort, in order to guarantee an exceptional closing and the best keeping. Ageing The barrels have been made in the winery for more than thirty years. As result of the control we have over all the manufacturing process, we can guarantee the maximum quality. This involves the following tasks: - Origin control: Bodegas Muga buys the wood from the origin. - Curing Control: We have outdoor drying yards. - Manufacturing Control: The winemaker inspects all the barrels to guarantee the absence of bubbles or other defects. - Toasting Control: It is one of the most important issues. An adequate toasting of the oak, made by experts and tailored to wines potentiality will contribute in a decisive way to the final quality. The oak we are using for comes from Tronçais forest, being exceptionally fine and the staves have been selected one by one. It is cured for two years. The toasting is not very intense but for a long time, which improves the oak influence. Time of ageing in oak is 18 months.
Bodegas Muga is a celebrated winery in the Railway District in the town of Haro in Rioja, Spain. Moved to the present location in 1971 by the children of founder Isaac Muga Martinez, Muga Winery was been producing a variety of wines since its founding in 1932. Bodegas Muga currently is in control of one thousand acres of fertile wine growing land, and is one of the few wineries in the world that owns its own cooperage.
Food Pairings for Bodegas Muga Tempranillo Blend Rioja Aro
Photos+ Add an Image for this wine
Best Wine Deals
$22.9926% offFrancois Raquillet Mercurey Vieilles Vignes 2011
$29.9926% offWestport Rivers Brut Cuvee 2006


text: I'm looking for firmware for my WRTSL54GS that allows me to choose the WAN "From" (or source) IP from which to forward. For example, From WAN IP: 184.108.40.206 From Port: 23 To LAN IP: 192.168.0.5 To Port: 23 The example above would forward the Telnet port to a device on my LAN *but only if the connection is from my work IP*. In other words I am not opening up that port to the whole world and his mother. So far I have tried DD-WRT v23 SP2 and v2.00.5, HyperWRT + thibor15c (they looked like my best bets) and they both appear to just forward connections from any WAN IP. I'm sure it can be scripted and maybe that's what I'll end up doing but I'm surprised this (basic?) functionality isn't already there in the GUI. I have looked at Toxic's great firmware comparison guide for clues but it (understandably) doesn't cover port forwarding in this kind of detail. Can anyone suggest a firmware release that allows the specification of the source IP for port forwarding, please? Thanks Andrew


text:
- Contributor Biographies
Kristina Ackley (Oneida/Bad River Ojibwe) teaches Native American studies at The Evergreen State College. In addition to coediting a collection of Laura Cornelius Kellogg's writings with Cristina Stanciu, she is completing a manuscript on Oneida placemaking.
Chadwick Allen is professor of English and coordinator for the American Indian Studies program at The Ohio State University. Author of Blood Narrative: Indigenous Identity in American Indian and Maori Literary and Activist Texts (2002) and Trans-Indigenous: Methodologies for Global Native Literary Studies (2012), he is the current editor of Studies in American Indian Literatures.
Cathleen D. Cahill, associate professor of history at the University of New Mexico, is the author of Federal Fathers and Mothers: A Social History of the United States Indian Service, 1869–1933 (2011).
Marti L. Chaatsmith (Comanche), associate director of the Newark Earthworks Center at The Ohio State University at Newark, lives and works in the Ohio Valley near the extraordinary Newark Earthworks. Current collaborative projects include contemporary Native artists' responses to the earthworks; ancientohiotrail.org, funded by the National Endowment for the Humanities; facilitating Indian participation in the World Heritage Site nomination of Ohio's earthworks; and an interactive computer model of the lunar observatory at Octagon Earthworks funded by Battelle.
Philip J. Deloria is Carroll Smith-Rosenberg Collegiate Professor of History and American Studies and associate dean for undergraduate [End Page 357] education in the College of Literature, Science, and the Arts at the University of Michigan. He is the author of Playing Indian (1998) and Indians in Unexpected Places (2004), and coeditor, with Neal Salisbury, of A Companion to American Indian History (2004).
P. Jane Hafen (Taos Pueblo) is professor of English at the University of Nevada, Las Vegas. She is the editor of Dreams and Thunder: Stories, Poems and the Sun Dance Opera by Zitkala-Ša (2001) and Critical Insights: Louise Erdrich (2012), a coeditor for The Great Plains Reader (2003), and the author of articles and book chapters on American Indian literatures.
K. Tsianina Lomawaima (Muskogee/Creek) is professor of American Indian studies at the University of Arizona. A historian who has focused on federal boarding schools and federal Indian policy in the twentieth century, she is the author of They Called it Prairie Light: The Story of Chilocco Indian School (1994), Away from Home: American Indian Boarding School Experiences (2000, with Margaret Archuleta and Brenda Child), Uneven Ground: American Indian Sovereignty and Federal Law (2001, with David Wilkins), and "To Remain an Indian": Lessons for Democracy from a Century of Native American Education (2006, with Teresa McCarty).
Kyle T. Mays, of Saginaw Anishinaabek heritage, is a PhD candidate in history at the University of Illinois, Urbana–Champaign. His dissertation explores representations of American Indians, Native political culture, and the relationship between blackness and indigeneity in nineteenth- and twentieth-century Detroit.
Thomas C. Maroukis, chair of the Department of History at Capital University in Columbus, Ohio, is the author of Peyote and the Yankton Sioux: The Life and Times of Sam Necklace (2004) and The Peyote Road: Religious Freedom and the Native American Church (2010).
David Martínez (Gila River Pima) is associate professor of American Indian studies at Arizona State University. He is the author of Dakota Philosopher: Charles Eastman and American Indian Thought (2009) and editor of The American Indian Intellectual Tradition: An Anthology of Writings from 1772 to 1972 (2011). [End Page 358]
Julianne Newmark, an associate professor of English at New Mexico Tech, teaches courses in American and Native American literature, writing, and visual rhetoric, and serves as the editor of the ejournal Xchanges. Her current research focuses on early twentieth-century Native textual activism and on the impacts of specific US legislative actions on Indigenous writing.
Margaret Noodin / Giiwedinoodin (Anishinaabe heritage, waabzheshiinh doodem) received an mfa in writing and a PhD in English and linguistics from the University of Minnesota. She is director of the Comprehensive Studies Program and teaches American Indian literature at the University of Michigan. Her work focuses on the recovery and maintenance of Anishinaabe language and literature. Current research includes language proficiency and the study of Indigenous literary aesthetics. Visit www.ojibwe.net to hear and see more.
Beth H...


text:
Oh, dear. It seems I wasn't quite ready for regular posting yet. I'm sorry I've been so absent this week! But it's time again for Sharon's Independence Day's Challenge, and somehow I must post that, even when I've done very little. Let's hope it's the jumpstart I need to get back into the rhythm...
1. Plant something:
I went to my favorite annual plant sale this weekend in the pouring rain. Little plants, dug from local gardens, and so nearly guaranteed to thrive in mine. I love it! This year, I tried to show restraint, and focused on edible plants. I got licorice mint, lavender, "bloody sorrel," lemon balm, and horseradish, and planted them all as soon as the rain cleared.
I'm sad to report that the bushes I planted over the past two weeks aren't doing very well. I didn't care for them as well as I should (hard to water things so far from the house, especially with a broken hose nozzle...) and only about half of them are leafing out. Pooh.
2. Harvest something:
Eggs. Lettuce thinnings. Daily chive snacking in the garden for the kids.
6. Build community food systems:
Well, I supported local farmers by braving the farmer's market in the cold pouring rain. Does that count?
7. Eat the food:
Lots of rhubarb cuppa-cuppa-cuppa (Lucy can now say "rhubarb"--with a big smile; she loves it). Parsnip souffle. Not much else interesting.
Thursday, May 6, 2010
Independence Days: not very productive this week
Labels: challenges, gardening, independence days, local food
1 comment:
- muffinmoon said...
Lise, you do so much! I have been having days where I feel happy if I haven't cried uncontrolably for a while. It has comforted me that you and some of the other wonderful women I read about don't skip through every week. I love that you keep it real. I am blogging solely about diabetes this week but will blog next week about other things and it won't all be laughs, I can tell you!
I adored those photos of the plants. So ready for warmer weather here but it's chilly still.
- May 11, 2010 at 12:59 PM
Subscribe to: Post Comments (Atom)


text:
Becky Keays, sales representative with REMAX Jazz Inc., Brokerage. Oshawa Ontario, Canada.
- Luxury Pickering real estate -
Find MLS listings of luxury Pickering homes. Many executive Pickering estates are available, to book a showing please call our Keller Williams office today.
View pictures, price & descriptions of luxury real estate for sale in Pickering… get started today! Call a Keller Williams REALTOR and start looking at the best, with the best. Luxury property at your finger tips.
Dan Wood, Sales representative – 905-903-5442.
Last updated 03/15/2014.
RE/MAX Jazz Oshawa:
193 King Street East Oshawa, ON L1H 1C2 Office: (905) 728-1600 Fax: (905) 436-1745
Becky Keays & Associates.
*Sales Representative
Whitby homes for sale Oshawa homes for sale
Each REMAX office is independently owned and operated.


text:
Routes : North America : United States : Kentucky : East : Red River Gorge (Bouldering) : The Cove : Donkey Dick Traverse
Donkey Dick Traverse - V4Average Rating : 0.00 out of 5
Route sequence (left to right): 17
Route Summary | Ascent Notes (0)
Premier Sponsor:
Description:to the right and around the corner from More Than a Broken Ankle. Sit start on a large chunk of rock (note that this piece of rock feels loose and is might very well fall on you....ending your life) and head right to finish on Happy Days.
Submitted by: daggerx on 2008-10-05
Views: 717
Route ID: 11872


text:
WOW, summer is finally here. This week the kids are finishing up with school. Courtney had a field trip to Bladensburg Waterfront Park and Junior had a field trip to Universoul circus. Tomorrow Riley, Ryker and I will go with Riley's pre-k class on a D.C. Duck tour in an amphibious vehicle. It should be a lot of fun.
Ryan is currently TDY to Europe until next week. He will come home and leave back out 24 hours later. Then he will be out for another 10 days here in the U.S.
So our weather has been one for the books. As you know we are in D.C. and severe weather does not happen very often. We have had more than 3 days of severe weather this week alone. Yesterday was 4 different storms that spawned several tornadoes in our metro area. It reminded me of springtime in Texas.
As for me just doing the mom thing. I have a special party on friday night for a babysitter of mine who has been amazing over the summer. I have been just doing laundry and dishes and cooking for my babies. Hanging out with Ryker in the garden and watching the kids on the Slip and Slide. Also I took the kids to the pool too. All in all it has been a very busy yet fun week.


text:
Scuba Diving Kuwait
Come Dive with Us.
Let us take you diving
Diving with the Pros. is safe and enjoyable
We offer diving classes, diving trips in Kuwait and abroad.
All what you need is give us a call and tell us what you need and how you want it to be done.
Begginners Courses
For beginners we provides two diving courses, Scuba Diver Course and OpenWater Diver Course. These courses are the foundation of diving. Here you'll learn many safety tips and other skills to dive with confident after completing them.
Advanced Courses
We provides two more courses here, which is the Adventure in Diving Program and the Advanced OpenWater Course. If you want to enjoy and learn more then here is your next step. Adventures are really wide and we provides a varity of these adventures. All what you need to do is ask out experts on what you need and how we can provides your needs to you.
Rescue Diving
Caring of your partner underwater is a good habbit. So here is your chance to know how to take care of your partner and be safe at the same time. So here we provide Rescue Diver Course. One of the most excited diving courses and will let you learn how to take care of your partner and your self and dive safely.
There are many things to do underwater depends on what you like. So why not to know more about what you love to do underwater? So we provide a varity of Diving Specialties. (example: Underwater Photography, Wreck Diving, Deep Diving,...etc). Do what you love to know and learn more about it with our Pros.
Professionalism is the way to show how experience you are. To start your Pro. level is taking a Dive Master Course. You will enlarge your knowledge and skills and the way to deal with divers (underwater/on land).
First Aid & CPR
Want to help Friends, Family or people who needs that help. Join our First Aid & CPR Courses and then help whoever needs with confident. We provide our First And and CPR Courses through Emergency First Response. One of the largest First Aid & CPR Organizations in the World. Our Pros. are approved to get you from providers levels to Instructors.


text:
Edward Lampert, the hedge fund manager who controls Sears Holdings Corp., has signed a deal to buy a seven-bedroom home owned by Paul and Trudy Cejas on exclusive Indian Creek Island for $40 million, according to One Sotheby's International Realty Inc.
The 17,000-square-foot mansion sits on 2.7 acres and wasn't publicly listed, Bloomberg News reported. The sale likely will be completed in less than 30 days, Kevin Tomlinson, a Miami Beach vice president at One Sotheby's, told Bloomberg.
Located in Biscayne Bay off Miami Beach, Indian Creek is home to less than two dozen luxury homes and a private golf-and-country club. Residents include auto magnate Norman Braman.
The Indian Creek Island mansion, which was rebuilt in the "last couple of years," has a reflection pool and a so-called arrival court, he said. Steve Lipin, a spokesman for Lampert, didn't immediately return a telephone call seeking comment.
Lampert is ranked 117th on the Forbes 400 Richest Americans list, with a net worth of $3 billion as of September. He is the latest in a growing stream of investment managers who have moved to Miami in recent years. Cejas, a former ambassador to Belgium, founded CareFlorida Health Systems and now runs PLC Investments, a private asset management firm.


text:
After a mysterious accident turns beautiful, privileged but ultimately damaged Cady into a dazed, pill popping shadow of her former self, she returns to her family's island getaway in the company of her fellow Liars, cousins Johnny and Mirren and handsome family friend Gat, in attempt to understand the truth about what happened, what she has become and what the destructive quartet have done.
Let me just get my cursing out of the way. Holy shit! I loved this book!
I don't know if I would have picked up We Were Liars were it not for some stellar recommendations from the #ukyachat group on Twitter. Stories about poor little rich kids aren't usually my cup of tea, but I gave it a go anyway. And I'm so glad I did. We Were Liars is a hauntingly beautiful book, written from the point of view of sullen, spoiled teenager Cady Sinclair. After an accident she cannot recall, she spends much of her time dosed up on pills. As a result, what we end up with a sullen, unreliable narrator as she reunites at her childhood island getaway with her fellow Liars and partners in crime; cousins Johnny and Mirren and family friend Gat, who Cady may or may not be in love with. There are plenty of questions: What happened that fateful summer? Why do Cady's family refuse to speak of the accident? Why did none of her friends speak to her until now? There are no immediate answers. Instead what follows is a story of nostalgia for days gone by, young love/hopeless infatuation and the grim facade of real people behind seemingly privileged lives, one that on many levels is instantly relatable to anyone who has felt that yearning for care free days that you thought would go on forever. Reading this book in the park under the first falling autumn leaves (damn your British summertime!), it was impossible for me not to be moved, to reminisce about my own journey into adulthood, the people I left behind and those who left me behind. E Lockhart is a gifted author, her writing style beautifully poetic and almost lyrical in places, the perfect tone for the dreamy feeling of nostalgia that hangs through the book. My interest usually wanes at some points in slow-paced, reflective books with an introverted narrator, but I can honestly say I didn't want to put the book down once!
The novel is beautifully interwoven with fables, underscoring the constant feeling of unease and tension bubbling away as Cady slowly begins to recall more of what happened that fateful summer and the cracks in the Sinclair family begin to deepen and widen. Reality begins to bleed into nostalgia, akin to that scene in 500 Days of Summer where Tom's fantasy and reality about reuniting with Summer play out on split screen, and eventually, Lockhart reveals a sucker punch ending that puts everything that came before it in a whole new light. I won't spoil anything, but I thought it was absolute perfection. I pride myself on seeing twists coming from a mile away in books or movies, but this one completely blindsided me!
A wonderful contrast of nihilistic despair and rose-tinted reminiscence, combined with secrets and lies from a family falling to pieces behind the facade of privilege, We Were Liars is a truly gripping read, one I'm going to join in raving about!


text:
india # 2 … step wells
Step wells – or Baoris, is one of the things we were looking forward to seeing in India.
Step wells could be seeing as a flipped pyramid surrounded by stairs. They were build with the purpose of storing monsoon rain water.
The construction of step wells in India dates from at least 600 BC (though the first rock-cut step wells in India date from 200-400 BC) with its peak between the 11th to 16th century.
The step wells had social and religious activities significance, but after British domination, authorities forced the abandonment of the wells, for sanitary reasons.
Ever since then the step wells acquired the connotation of being hunted places.
We have been to Chand Baori in Abhaneri, a village in the Dausa district of Rajasthan, situated around 95 km from Jaipur, on the Jaipur-Agra road.
Chand Baori consists of 3,500 steps over 13 stories. It extends approximately 30 m. into the ground making it one of the deepest and largest stepwells in India.
The Rajasthan area is extremely arid, and the design and final structure of Chand Baori was intended to conserve as much water as possible.
At the bottom of the well, the air remains 5 to 6 degrees cooler than at the surface, and Chand Baori was used as a community gathering place for locals during periods of intense heat.
In Delhi we have been to Agrasen ki Baoli .
With 103 steps, this Baoli is among a few of its kind in Delhi. The visible parts of this historical step well consist of three levels. Each level is lined with arched niches on both sides.
The atmosphere is always kind of heavy when you are inside a Baori.
The air is quite humyd, and you keep going from extreme sunshine to cold shadows, where you may find a good amount of bats, not a pleasant surprise…
But though the odd atmosphere, this ´negative-spaces' is beyond interesting.
Visiting the Baoris was for sure magic, and if you are plannig to travel around India, dont miss out the step wells on your way!
talk to you soon,


text:
1. Money – I have forgotten to get money and gone scrambling the night before a show to find small bills for my show. That is a nightmare! Remember to get money from the bank before you have a show. I usually get about $100 in small bills. So far that has been plenty for my shows.
2. Credit Card Machine - If you take credit cards they are a must. If you are using an electronic processor, be sure you can get a signal at your show. I have been to some shows as a customer where I could not purchase what I wanted because they were unable to get a signal and had no backup.
3. Tents/Tables/Chairs – Tables and shelves, you need something to display your work on.
Chairs: Chairs are optional. Some people feel that sitting down in a booth gives the wrong impression. Others could care less one way or another. I do bring chairs with me since I sometimes create new items while at a show.
Tents: I love my tent. Most people who work in outdoor craft shows will tell you a tent is a must. It provides protection for you and your product from the elements. You can get even more protection if you invest in walls for your tent as well. Be careful with walls though, on hot summer days they will bake you by not allowing a breeze through.
4. Displays – Laying your items flat on a table will not bring in customers as well as displaying your products on shelves or displays will. Prop your items up with displays to catch the eye of your customers.
5. Packaging – Customers want to easily transport their purchases home. Make sure you have some bags and boxes available if your customers ask. You may also wish to consider gift wrap or gift boxes during the holiday seasons.
6. Signs and Business Cards – Always have plenty of business cards available for your customers. Also make sure all your signs are typed in large print to make it easy for your customer to read.
7. Promotional Items – Any promotional items you have that will advertise your business like pens, key chains, bookmarks, notepads, ect.
8. Products – No products, no show. Nuff said.
9. Snacks and Drinks – I have attended a few shows where there were not any food vendors, so I always pack a cooler with snacks and drinks. Be careful to only pack items that are mess free (like goldfish crackers), you don't want to wear a mustard stain on your outfit during a show. It will not look good to your customers.
10. Misc Items: Any items you may need to set up and keep up your booth. Items like
B.Velcro (a biggie),
C. String,
D. Weights to tie down your displays (I prefer fishing weights)
E. Pens
F. Tags (Price tags or promo tags)
G. Camera (Especially if you have updated your booth look lately)
H. Calculator
And any other items you may need to run your booth during the day.
Monday, August 31, 2009
4 comments:
Great list of items!ReplyDelete
These are great tips! I won't be doing Craft Shows until next year, but I'm definitely gonna keep this list on hand! Thanks!ReplyDelete
Great hints; I make a CRAFT SHOW LIST that I follow TO THE letter when packing for a show; very important, especially if I have had a break in time ( not doing shows back to back;) otherwise SOMETHING gets forgotten, and there is nothing worse than being MILES from home and missing something important ( like Money :) )ReplyDelete
Subscribe to: Post Comments (Atom)


text: **Did anyone else check out their Spotify "Wrapped 2017" stats? I listened to over 130,000 minutes of music this year! Donovan Woods was my top listened to artist of the year this year; unsurprisingly his song "Portland Maine" was also my most listened to song. Crazy to see all of this info in one place!


text:
Wednesday, 19 June 2013
3 comments:
Such a great post, and a wonderful reminder that so much of what we think of as "Christian" is actually the opposite of what Christian practice is or at least should be.ReplyDelete
- Anonymous21 June 2013 at 22:28
Is it really the case that the anathemas on the Sunday of Orthodoxy are not read because people have a problem with the notion of anathematising? Or is it because the list is too darn long for a Sunday morning after the Liturgy? Plus the fact hardly anyone knows who all these people are? We should also remember that there is more to the text than the anathemas - there are also "eternal memories" for the defenders of Orthodoxy. I think what most churches read is what is most important about the feast, though there is a lovely passage that is often omitted for reasons I do not understand. We do read a small selection of anathemas at my church. But the passages I find most inspiring are the positive affirmations of our faith, though I particularly like the condemnation of iconoclasts who affirm the incarnation in words but deny it in images as "fantasiasts".ReplyDelete