--- abstract: | For treatment effects - one of the core issues in modern econometric analysis - prediction and estimation are two sides of the same coin. As it turns out, machine learning methods are the tool for generalized prediction models. Combined with econometric theory, they allow us to estimate a personalized treatment effect - the conditional average treatment effect (CATE). In this tutorial, we give an overview of novel methods, explain them in detail, and apply them via Quantlets in real data applications. We study the effect that microcredit availability has on the amount of money borrowed and if 401(k) pension plan eligibility has an impact on net financial assets, as two empirical examples. The presented toolbox of methods contains meta-learners, like the DR, R-, T- and X-learner, and methods that are specially designed to estimate the CATE like causal BART and generalized random forest. In an additional simulation study we further compare the methods to observe patterns and similarities. **Keywords** Causal Inference, CATE, Machine Learning, Tutorial author: - | Daniel Jacob\ <kparker@example.org> bibliography: - 'references.bib' title: | CATE meets ML [^1]\ <span>The Conditional Average Treatment Effect and Machine Learning </span> --- Introduction ============ Estimation and prediction of treatment effects are important tasks for every economist and financial econometrician since treatment effects are often the basis for policy and business decisions. As an illustration, let us look at an idea of microcredits, dating back to Muhammad Yunus, a Nobel Price winner, who discovered in 1976 that very small loans could make a disproportional difference to a poor person. Microcredits work as shown in Figure \[fig:microcredit\]. They can increase investments since such credit is easy to get and pay back. Business activity is hence more flexible and could be improved. Increasing gains from a business could increase the household income and further allow for more savings which can be invested in, for example, education. ![The theory of microcredits. <span data-label="fig:microcredit"></span>](Figures/Microcredit_final.pdf){width="80.00000%"} This specific example was recently applied by @crepon2015estimating who studied the setting where certain villages in Morocco get access to microcredit (the treatment group) while others don't (the control group). As economists, one is interested in the effect that microcredit availability has on the amount of loans which could be an indicator of how demanded such microcredits are. Since we observe certain characteristics for each household, we can condition on such observed variables to see if there is heterogeneity in the effect from microcredit. Figure \[Fig:CATE\_emp\] shows an example of what we want to do. The goal is to find subgroups based on characteristics where we believe that the treatment effect is different. As an example, we can partition the households by age and compare young vs. older household members in terms of their effect on microcredit. In both subgroups, we need to make sure that we observe people that are treated and others that did not receive treatment. We can estimate the average treatment effect (ATE) for the young household members, for example, by taking the difference of their mean outcome given treatment status. We repeat this for the subgroup of older households. Recent methods to estimate the ATE using nonparametric methods on the whole sample include target maximum likelihood estimation (TMLE) [@van2010targeted] and double machine learning [@chernozhukov2018double]. If the data has many covariates (let us say it has high-dimensionality) and if we don't know which specific subgroup we should focus on, as is the case here, we can use methods that are presented in this tutorial. These methods estimate a treatment effect for each observation based on their covariates, the conditional (on covariates) average treatment effect (CATE). In a further step, we can then look at the heterogeneity and try to link characteristics that are drivers for different treatment effects. \[edge from parent/.append style=<span>-&gt;</span>,sibling distance=10em,level 2/.style= <span>level distance = 50pt, sibling distance=100pt</span>, level 1/.style= <span>level distance = 50pt, sibling distance=200pt</span>,level 3/.style= <span>level distance = 50pt, sibling distance=200pt</span>, every node/.style = <span>shape=rectangle, rounded corners, draw, align=center, top color=white, bottom color=blue!20</span>\]\] (root) <span>All households</span> child <span> node <span>young households</span> child <span> node <span>treatment group</span> </span> child <span> node <span>control group</span> child\[xshift=-1.5cm\] <span>node (cate1) <span>CATE = 2000</span></span> </span> </span> child <span> node <span>older households</span> child <span> node <span>treatment group</span></span> child <span> node <span>control group</span> child\[xshift=-1.5cm\] <span>node (cate2) <span>CATE = 900</span></span> </span> </span>; (root-1-1) – (cate1); (root-2-1) – (cate2); The high-dimensionality of a dataset does not necessarily mean that one has more covariates than observations by default. However, if we are unsure about the structural form, we could include interaction and quadratic terms, and soon the number of dimensions increases. For example, if we have 1000 observations and 30 covariates, then by only including quadratic interactions the amount of covariates increases to 495. Including up to cubic terms leads to a dimension of 5455. If we further assume that only a few covariates are dependent on the outcome and the treatment (often called the approximate sparsity assumption), the task transfers into a selection problem where standard parametric models are limited and we might want to use machine learning (ML) methods. The reason why this is the case is either that we have more covariates than observations or that the functional forms are complex and we don't know which interaction terms to include in a linear model. Machine learning is not easily defined. It contains many algorithms with the main focus of prediction (regression), classification, and grouping tasks like clustering. While clustering, as a form of dimensionality reduction, only uses covariates but not outcomes with labels, we call this branch unsupervised ML. The counterpart is called supervised ML. Supervised ML, in general, uses a set of covariates to predict an observed outcome. When talking about prediction we mean the following: Construct an estimator $\hat{\mu}(x)$ of $ \mathsf{E}[Y|X_i=x]$ using $Y$ and a set of covariates from some training set and predict the values of $Y$ from an independent test set. The goal is to minimize deviations between the true outcome and predicted outcomes from the test set. Note that this is in contrast to the term forecasting. The only assumption so far is that the observations are independent and that the joint distribution of $X$ and $Y$ in the training set is the same as that of the test set. To achieve the goodness of fit (for example minimize the average of the mean-squared error), in an independent test set many alternative models are estimated and the model that maximizes a criterion is selected. We will talk about cross-validation - a concept for model selection - later. The key is that the functional form is mostly determined as a function of the data. Regularization together with systematic model selection may be the main advantages of ML methods. When we talk about ML in this tutorial, we mean supervised machine learning models that are used to make predictions. For a detailed discussion about machine learning in economics see @mullainathan2017machine and @atheyimpact. How do we get from prediction to causal inference? A simple, pure prediction approach to get the CATE is to estimate two conditional mean functions, one for the treated observations and one for the non-treated (the control group). For each observation, we can predict the outcome under treatment and control by plugging each observation into both functions. Taking the difference between the two outcomes results in the CATE. Mapping the support of $X$ on $Y$ is a classic regression task for which machine learning methods are well suited to find generalizable predictive patterns. Since we are only interested in getting a good prediction of the conditional mean, we do not need to know the underlying structural form of this function which enables vanilla ML methods to be sufficient. We call such functions, where the parameters are not of immediate interest, a nuisance function. While the above example of estimating the CATE is quite simple and intuitive, we will see that there are more efficient or automated methods to estimate heterogeneous treatment effects. We will also see that while prediction models are easy to evaluate, causal parameters are not. This is mainly since the objective is different. In prediction, we can optimize a goodness of fit criterion since we observe the true outcome. The causal parameter, however, is never observed in any dataset. As in econometrics, we need to carefully design methods that aim to estimate such parameters of interest, apply statistical theory and expand the set of assumptions to interpret a parameter as causal. This tutorial is structured as follows. First, we provide an overview of the potential outcome framework and state the necessary assumptions to interpret our parameter of interest as a causal parameter. We then explain different methods that we consider, methods that are very flexible in the choice of the ML algorithm, and methods that are developed to estimate the CATE, mostly relying on tree-based algorithms. As in classical ML, we make use of sample splitting to limit overfitting and allowing for less restrictive assumptions on the nuisance functions. We cover explanations on why and how to do sample splitting and cross-validation. Next, we investigate two empirical datasets, the microcredit example, and the 401(k) pension plan survey. Last, we include a simulation study where we generate the true treatment effect. This allows us to directly compare all different methods in terms of accuracy. Whenever possible we provide and link to Quantlets [![image](qletlogo_tr.png){height="3mm"}](https://github.com/QuantLet/Meta_learner-for-Causal-ML) that are ready-to-use code snippets to implement the discussed methods (the Quantlets are all written in `R`). The files are not only a replication code for the empirical analysis and the simulation study but contain functions to implement novel methods that aim to estimate the CATE directly. During this tutorial, we will use the terms model, method, and algorithm interchangeably. Figure \[Fig:splitting-tree\] gives an idea of how a causal structure may look. In the first graph, only the treatment has an impact on the outcome while the second graph also includes covariates that might make the treatment effect dependent on some characteristics. The same is true for the third graph but now the covariates also influence the treatment probability. We say that such a setting is from an observational study since the researcher has no control of the treatment assignment. The first two settings can be seen as a randomized controlled trial (RCT) but only in the second and third can we hope to observe treatment heterogeneity and hence estimate the CATE. \[\] (0,0) node\[left\] (D) <span>treatment</span> – (2,0) node\[right\] (Y) <span>outcome</span>; (D) – coordinate (middle) (Y); (ATEe) <span>1. ATE\ (RCT)</span>; 0.4cm \[\] (0,0) node\[left\] (D) <span>treatment</span> – (2,0) node\[right\] (Y) <span>outcome</span>; (D) – coordinate (middle) (Y); (X) <span>covariates</span>; (X) – (Y); (ATE) <span>2. CATE\ (RCT)</span>; 0.4cm \[\] (0,0) node\[left\] (D) <span>treatment</span> – (2,0) node\[right\] (Y) <span>outcome</span>; (D) – coordinate (middle) (Y); (X) <span>covariates</span>; (X) – (D); (X) – (Y); (ATE) <span>3. CATE\ (observational)</span>; Methods ======= Let us start with an introduction of the potential outcome framework for which we use the following notations: Each observation has two potential outcomes, $Y^1$ and $Y^0$ of which we only observe one, namely the former if someone was treated or the latter if not. The binary treatment indicator is $D \in \{0;1\}$ and we denote observed covariates $X \in \mathbb{R}$. To interpret the estimated parameter as a causal relationship, the following assumptions are needed; see, for example, @rubin1980randomization: 1.0cm 1\. Conditional independence ( or conditional ignorability/exogeneity or conditional unconfoundedness): $$\begin{aligned} \left(Y_{i}^{1}, Y_{i}^{0}\right) {\mathrel{\text{\scalebox{1.07}{$\perp\mkern-10mu\perp$}}}}D_{i}|X_{i}.\end{aligned}$$ 2\. Stable Unit Treatment Value Assumption (SUTVA) (or counterfactual consistency): $$\begin{aligned} Y_i = Y_i^0 + D_i (Y_i^1-Y_i^0).\end{aligned}$$ 3\. Overlap Assumption (or common support or positivity): $$\begin{gathered} \forall x \in supp(X_i), \quad 0 < P(D_i=1|X_i=x) < 1, \nonumber \\ P(D_i=1|X_i=x) \overset{\mathrm{def}}{=} e(x).\label{equ:prop}\end{gathered}$$ 4\. Exogeneity of covariates: $$\begin{aligned} X_i^1 = X_i^0.\end{aligned}$$ Assumption 1 together with Assumption 4 is very natural since they state that the treatment assignment is independent of the two potential outcomes and that the covariates are not affected by the treatment. Assumption 2 ensures that there is no interference, no spillover effects, and no hidden variation between treated and non-treated observations. Assumption 3 states that no subpopulation defined by $X_i = x$ is entirely located in the treatment or control group, hence the treatment probability needs to be bounded away from zero and one. Equation (\[equ:prop\]) is called the propensity score. Now we define the conditional expectation of the outcome for the treatment or control group as $$\begin{aligned} \mu_d(x) = \mathsf{E}[Y_i | X_i = x, D_i = d] \quad with \quad D \in \{0,1\}.\end{aligned}$$ If we don't use any subscript, we refer to this function as the general conditional expectation. Our parameter of interest is the CATE ($\tau(x)$), which is formally defined as: $$\begin{aligned} \tau(x)=\mathsf{E}\left[Y_i^1-Y_i^0 \mid X_{i}=x\right]=\mu_{1}(x)-\mu_{0}(x).\end{aligned}$$ Equation \[equ:cate\_derivation\] shows how the two conditional mean functions can represent the two potential outcomes and hence, by taking the difference, lead to the CATE. $$\begin{aligned} \tau(x) &=\mu_{1}(x)-\mu_{0}(x) \nonumber \\ &=\mathsf{E}\left[Y_{i} \mid D_{i}=1, {X}_{i}= x\right]-\mathsf{E}\left[Y_{i} \mid D_{i}=0, {X}_{i}= x\right] \nonumber \\ &=\mathsf{E}\left[Y_{i}^{1} \mid D_{i}=1, {X}_{i}= x\right]-\mathsf{E}\left[Y_{i}^{0} \mid D_{i}=0, {X}_{i}= x\right]\nonumber \\ &=\mathsf{E}\left[Y_{i}^{1} \mid {X}_{i}= {x}\right]-\mathsf{E}\left[Y_{i}^{0} \mid {X}_{i}=x\right] \nonumber \\ &=\mathsf{E}\left[Y_{i}^{1}-Y_{i}^{0} \mid {X}_{i}= {x}\right] \label{equ:cate_derivation} \end{aligned}$$ This estimator is of special interest in many areas like medicine or policy actions since it tells us if there are differences in the treatment effect in the population and how big these differences are. It could be, for example, that the average treatment effect of a policy is $+2$, containing half of the people with a treatment effect of $+6$ and the other half of $-2$. Instead of treating everyone, we should only treat people that have a positive effect from the policy (if positive means better). If this is not possible, let us say due to legal or ethical reasons, the policy should not be kparker@example.org. The CATE will tell us the exact distribution of the effects and, at best, allows us to identify subgroups. To estimate the CATE, we are not primarily interested in the coefficient from regressing $X$ on $Y$, nor are we interested in the coefficients from the propensity score model. What we want instead is to have a good approximation of the function and hence good estimates from e.g. $\mu_{1}(x)$ and $\mu_{0}(x)$. This is why ML methods are well suited for the job. When reviewing recently proposed methods for the estimation of the CATE, we can categorize them into two groups. The first group contains methods that are built on off-the-shelf machine learning methods (such as the lasso, random forest (RF), Bayesian additive regression trees (BART), boosting methods, or neural networks). Since the base learners are not designed to estimate the CATE directly, the literature calls them meta-learners or generic ML algorithms. The second group of methods alters existing machine learning methods in a way that they can be used to estimate the CATE directly (examples are causal boosting by @powers2018methods, causal forest by @athey2019generalized or Bayesian regression tree models for causal inference by @hahn2020). See @kuenzel2019meta for a comparison between meta-learners like the S-, T-, and X-learner as well as the causal forest in a simulation study. @knaus2018machine compare meta-learners like inverse probability weighting (IPW) estimator, doubly-robust (DR), modified covariate method (MCM), R-learner, and different versions of the causal forest in an empirical Monte Carlo study while @nie2017quasi compare their R-learner with the S-, T-, X- and U-learner as well as causal boosting. Regarding the base learners (the ML methods), @kuenzel2019meta use a random forest and BART. @knaus2018machine use RF and lasso while @nie2017quasi use boosting and the lasso for the estimation of the nuisance functions and the treatment effects. In high dimension, the use of machine learning methods, such as boosting or random forests to estimate the propensity score, works quite well as @mccaffrey2004propensity and @wyss2014role show. The estimation of probabilities given a large set of covariates is nothing less than a prediction problem in where ML methods are superior. In Table \[tab:methods\_quantlet\] we list popular methods by category, including links to the Quantlets. The references refer to recent papers that use these methods and provide theoretical properties. <span>llll</span> Category & Method & Reference & Quantlet\ & DR-learner & @kennedy2020optimal &[![image](qletlogo_tr.png){height="5mm"}$_{DR}$](https://github.com/QuantLet/Meta_learner-for-Causal-ML/tree/main/DR-learner)\ & IPW-learner & @horvitz1952generalization &[![image](qletlogo_tr.png){height="5mm"}$_{IPW}$](https://github.com/QuantLet/Meta_learner-for-Causal-ML/tree/main/IPW-learner)\ & R-learner & @nie2017quasi &[![image](qletlogo_tr.png){height="5mm"}$_{R}$](https://github.com/QuantLet/Meta_learner-for-Causal-ML/tree/main/R-learner)\ &S-learner & @hill2011bayesian & [![image](qletlogo_tr.png){height="5mm"}$_{S}$](https://github.com/QuantLet/Meta_learner-for-Causal-ML/tree/main/S-learner)\ & T-learner & @hansotia2002incremental &[![image](qletlogo_tr.png){height="5mm"}$_{T}$](https://github.com/QuantLet/Meta_learner-for-Causal-ML/tree/main/T-learner)\ & X-learner & @kuenzel2019meta &[![image](qletlogo_tr.png){height="5mm"}$_{X}$](https://github.com/QuantLet/Meta_learner-for-Causal-ML/tree/main/X-learner)\ & Causal BART & @hahn2020 &[![image](qletlogo_tr.png){height="5mm"}$_{CBART}$](https://github.com/QuantLet/Meta_learner-for-Causal-ML/tree/main/Causal-BART)\ &Causal Boosting & @powers2018methods &[![image](qletlogo_tr.png){height="5mm"}$_{CB}$](https://github.com/QuantLet/Meta_learner-for-Causal-ML/tree/main/Causal-Boosting)\ &Causal Forest & @athey2019generalized &[![image](qletlogo_tr.png){height="5mm"}$_{GRF}$](https://github.com/QuantLet/Meta_learner-for-Causal-ML/tree/main/GRF)\ Meta-Learners ------------- In the following, we briefly describe the considered meta-learners. We follow the definition of meta-learners by @kuenzel2019meta and describe them as methods to estimate the CATE using ML methods that are built for regression or classification tasks only. The S- and T-learner, for example, can use any vanilla ML method to predict the conditional outcome. The prediction models can then be used to estimate the conditional average treatment effects. The second class of methods uses additional information from the propensity score. They contain the DR- and X-learner. Again, the conditional mean functions, as well as the propensity score, can be estimated using a broad range of ML methods. The aforementioned methods are also called transformed outcome methods. The idea is to generate a pseudo-outcome using the estimated nuisance functions in the first step. This can be seen as an approximation of the conditional average treatment effect. The pseudo-outcome, which we show in Table \[tab:learners\], is an unbiased estimate of the CATE given that the nuisance parameters are known (e.g. if we would know the true propensity score). In a second step, the pseudo-outcome is mapped on the covariates to get the final estimate and to make predictions on new observations. The reason for prediction is that the observed data after a treatment assignment includes the outcome, covariates, and the treatment assignment variable. If we want to classify new observations, we only observe the covariates. Hence we need a model that maps the covariates on the estimated treatment effect. The mapping also serves as a smoother since it could be the case that some pseudo-outcome values are quite extreme (e.g. if the propensity score estimate is very low or high). The last method that we examine in this category is the R-learner. However, it does not generate a pseudo-outcome in the classical sense as it needs algorithms that can modify the loss function. Still, many ML methods can be used which is why we include the R-learner into the category of meta-learners. Currently, R-packages are available for the R-, S-, T-, U-, and X-learner (`install_github(xnie/rlearner)`) and the M-, S-, T-, and X-learner (`install_github(soerenkuenzel/causalToolbox)`). Causal analysis via the potential outcome framework and causal graph theory for Python can be found in @dowhy. For heterogeneous treatment effect analysis via machine learning in Python see @econml. The list of methods above is not a complete list of methods in this subject. For example, we do not talk about methods for instrumental variables, multiple-treatment, difference-and-difference methods, or regression discontinuity designs. We also note that there are other methods to estimate treatment effects in cross-sectional settings. For example, one of the first methods developed to control for confounding bias is the inverse probability weighting (IPW) estimator by @horvitz1952generalization. In Algorithm \[pseudo:IPW\] we show how to implement this method. Some methods use neural networks to estimate heterogeneous treatment effects. See, for example, the recent work by @farrell2021deep who use a deep neural network for semiparametric inference and develop nonasymptotic high probability bounds. ### Single- (S-learner) and two-model learner (T-learner): Let us first start with a very simple and intuitive method, the T-learner. It is a two-step approach where the conditional mean functions $\mu_1(x) = \mathsf{E}[Y^1 |X_i=x]$ and $\mu_0(x) = \mathsf{E}[Y^0 |X_i=x]$ are estimated separately with any generic machine learning algorithm. The difference between the two functions results in the CATE as shown in Table \[tab:learners\] and as seen in equation \[equ:cate\_derivation\]. One problem with the T-learner is that it aims to minimize the mean squared error for each separate function rather than the mean squared error of the treatment effect. By splitting the sample into two groups there is only information on one group. This might be problematic if the two functions shrink different covariates which are important in both groups. This is especially the case in an RCT. See, for example, @kuenzel2019meta [@kennedy2020optimal] for settings when the T-learner is not the optimal choice. An alternative is to model only one function and include the treatment assignment into this function. This approach is called the S-learner. See for example, @hill2011bayesian and @foster2011subgroup for early examples of proposing the S-learner. Algorithm \[pseudo:s\] in the Appendix describes how to implement the S-learner while algorithm \[pseudo:t\] shows the implementation for the T-learner. \[ht!\] Split sample $Z$ into $K$ random subsets\ **combine** $\hat{\tau}(X_i) = \{\hat{\tau}_1,\hat{\tau}_k, \ldots,\hat{\tau}_K$}\ ### Doubly-Robust learner (DR-learner): A more efficient method than the T-learner can be the DR-learner. It builds on the T-learner and adds a version of the inverse probability weighting (IPW) scheme on the residuals of both regression functions {$Y^d - \hat{\mu}_d(x)$}. We can think of it as combining two different models and hence avoid drawbacks like the minimization goal from the T-learner and a potentially high variance from an IPW model when some propensity scores are small. The doubly-robust learner takes its name from a double robustness property which states that the estimator remains consistent if either the propensity score model or the conditional outcome model is correctly specified [@lunceford2004stratification]. The DR-learner creates a pseudo-outcome which is an unbiased estimator for the CATE: $$\begin{aligned} \hat{\psi}_{DR} = \hat{\mu}_1(x) - \hat{\mu}_0(x)+\dfrac{D\left\{Y-\hat{\mu}_{1}\left(x \right)\right\}}{\hat{e}\left(x\right)} -\dfrac{\left(1-D\right)\left\{Y-\hat{\mu}_{0}\left(x \right)\right\}}{\left(1-\hat{e}\left(x\right)\right)}. \label{equ:dr-full}\end{aligned}$$ Equation \[equ:dr-proof\] in the Appendix shows the double-robustness property by rewriting equation \[equ:dr-full\]. Whenever the propensity score or the conditional mean function is correctly specified the doubly-robust estimator converges to $\textsf{E}[Y^1] - \textsf{E}[Y^0]$. The first part in equation \[equ:dr-full\] can be seen as a regression adjustment parameter (this is the difference between the two conditional mean functions). The second part, which makes use of the propensity score, can be seen as an inverse probability weighting estimator applied to the residual from the conditional mean functions. The DR-learner is very flexible in the choice of the ML method. Estimating the nuisance parameters we can use any ML method. Since the pseudo-outcome is already an unbiased estimator for the CATE the loss-function in the final regression task is to minimize the mean squared error. This allows using the same range of ML methods as in the first step. Recently, this estimator has gained popularity to estimate the CATE, especially in high-dimensional settings. See, for example, the work by @fan2019estimation. Most recently, @kennedy2020optimal find that for estimating the CATE, the finite-sample error-bound from the DR-learner at most deviates from an oracle error rate by the product of the mean squared error of the propensity score and the conditional mean estimator. As can be seen from equation \[equ:dr-full\], extreme propensity score estimates can lead to large pseudo-outcome estimates. Hence it is necessary to look at the distribution of the propensity score and, if necessary, apply methods to make the overlap assumption more realistic. 0.5cm \[ht!\] \[pseudo:dr\] Split sample $Z$ into $K$ random subsets\ **combine** $\hat{\psi}_{DR} = \{\hat{\psi}_{DR,1},\hat{\psi}_{DR,k}, \ldots,\hat{\psi}_{DR,K}\}$\ *Cross-fitting:*\ **row bind** $\hat{\tau}(X_i) = \{\hat{\tau}_1,\hat{\tau}_2\}$\ ### R-learner: The R-learner makes use of the idea of orthogonalization to cancel out any selection bias that may arise in observational studies from observed covariates. Here, the residuals from the regression of $Y$ on $X$ are regressed on the residuals from the regression of $D$ on $X$ and weighted by the squared residuals, $\{D-\hat{e}(x)\}^2$. This is similar to the double machine learning approach from @chernozhukov2018double where their estimator of interest is the ATE. @nie2017quasi develop a general class of two-step algorithms for the estimation of the CATE. The R-learner, as from residualized and as an homage to Peter M. Robinson, makes explicit use of machine learning methods. Achieving Neyman orthogonality using a residuals-on-residuals (or debiasing) approach has a long history in econometrics (see the Frisch–Waugh–Lovell theorem from the 1930s for linear regression) and mainly builds on the work by @robinson1988root who replaces the linear parts by non-parametric kernel regression. The CATE from the R-learner is obtained by the following minimization task: $$\begin{aligned} \begin{split} \hat{\tau}(\cdot)=\operatorname{argmin}_{\tau}\left\{\frac{1}{n} \sum_{i=1}^{n}\left[\left\{Y_{i}-\hat{\mu}^{(-i)}\left(X_{i}\right)\right\}\right.\right. \\ \left.\left.-\left\{D_{i}-\hat{e}^{(-i)}\left(X_{i}\right)\right\} \tau\left(X_{i}\right)\right]^{2}+\Lambda_{n}\{\tau(\cdot)\}\right\}. \end{split}\end{aligned}$$ The superscript $(-i)$ indicates the sample splitting. The conditional mean functions are trained without the $i$-th observations and evaluated only for $i$. We will explain certain sample splitting procedures later. The term $\Lambda_{n}\{\tau(\cdot)\}$ can be interpreted as a regularizer on the complexity of the $\tau(\cdot)$ function. In practice, this regularization term could be explicitly given as in penalized regression or implicitly introduced, e.g., as provided by a carefully designed deep neural network. The main difference to the pseudo-outcome estimators (the DR- and X-learner) is that the R-learner needs to alter the loss-function of the ML method. Even if $\psi_R$ with weights equal to $1$ as an estimator for $\tau(x)$ it can suffer from high variance if the nuisance functions are not known. The variance is mainly caused by the propensity score since $\{D-\hat{e}(x)\}$ is in the denominator. This is where the weighting comes into play. Observations that have a high variance are weighted by the squared of $\{D-\hat{e}(x)\}$ and hence are less important. The weights for each observation directly influence the loss function (e.g. in boosting methods they manipulate the gradient). Therefore, applying the R-learner needs ML methods that have the option of altering the loss-function through weighting. The following methods have this option: lasso and ridge regression (`glmnet`), boosting (included in the DMatrix format) (`XGBoost`), neural network (`nnet`) and the random forest (`ranger`). Note that the ranger package seems to be the only implementation of weights for a random forest. The weights are applied on the whole training sample and observations with larger weights will be selected with higher probability in the bootstrap (or subsampled) samples for the trees. 0.5cm \[ht!\] \[pseudo:R\] Split sample $Z$ into $K$ random subsets\ **combine** $\hat{\psi}_{R} = \{\hat{\psi}_{R,1},\hat{\psi}_{R,k}, \ldots,\hat{\psi}_{R,K}\}$, $w_R = \{w_1,w_k,\ldots,w_K\}$\ *Cross-fitting:*\ **row bind** $\hat{\tau}(X_i) = \{\hat{\tau}_1,\hat{\tau}_2\}$\ ### X-learner: @kuenzel2019meta propose the X-learner which estimates a treatment effect separately for the control and the treatment group. This might be especially helpful in situations where the proportion of the two groups is highly imbalanced. The X-learner has several steps. The first step is identical to the T-learner, namely estimating the two conditional mean functions. In the second step, we predict the counterfactual outcome using the two functions. If a person is treated and hence the observed outcome is $Y^1$ we subtract the estimated counterfactual. If a person is in the control group we use the estimated counterfactual outcome and subtract the observed outcome ($Y^0$). This results in two imputed treatment effects: $$\begin{aligned} \hat{\psi}_{X}^1\overset{\mathrm{def}}{=}Y^{1}-\hat{\mu}_{0}\left(x^{1}\right) \quad \text{for} \quad D_i = 1,\\ \hat{\psi}_{X}^0\overset{\mathrm{def}}{=}\hat{\mu}_{1}\left(x^{0}\right)-Y^{0} \quad \text{for} \quad D_i = 0.\end{aligned}$$ These imputed effects are now used in a third step to regress them individually on the covariates to obtain $\hat{\tau}_0(x)$ (the CATE for the control group) and $\hat{\tau}_1(x)$ (the CATE for the treatment group). The final estimator combines the two estimators plus some weights, $g(x)$: $$\begin{aligned} \hat{\tau}(x)=g(x) \hat{\tau}_{0}(x)+\{1-g(x)\} \hat{\tau}_{1}(x).\end{aligned}$$ In a randomized controlled trial, the two estimates should not differ significantly. If there are confounding variables and if the support of the treatment variable given covariates differs among the treatment status we would expect the two estimates to be different. Hence, a natural weighting function for $g(x)$ could be the propensity score. We would use $1 -\hat{e}(x)$ for the treatment group and $\hat{e}(x)$ for the control group estimate, respectively. Algorithm \[pseudo:X\] describes the procedure in detail and takes sample-splitting into account. 0.5cm \[ht\] Split sample $Z$ into $K$ random subsets\ **combine** $\hat{\psi}_{X}^1 = \{\hat{\psi}_{X,1}^1,\hat{\psi}_{X,k}^1, \ldots,\hat{\psi}_{X,K}^1\}$, $\hat{\psi}_{X}^0 = \{\hat{\psi}_{X,1}^0,\hat{\psi}_{X,k}^0, \ldots,\hat{\psi}_{X,K}^0\}$, $\hat{e}\left(X_{i}\right)= \{\hat{e}_1\left(X_{i}\right),\hat{e}_k\left(X_{i}\right), \ldots,\hat{e}_K\left(X_{i}\right)\}$\ **regress** $\hat{\psi}_{X}^1 = \hat{t}^1(X_i) + W^1_i$, with $i \in Z$\ **regress** $\hat{\psi}_{X}^0 = \hat{t}^0(X_i) + W^0_i$, with $i \in Z$\ 1.0cm**estimate** $\hat{\tau}^1_k(X_i) = \hat{t}^1(X_i)$, with $i \in Z$\ 1.0cm**estimate** $\hat{\tau}^0_k(X_i) = \hat{t}^0(X_i)$, with $i \in Z$\ **average** $\hat{\tau}_k(X_i) = \hat{e}(X_i)\hat{\tau}^0_k + (1-\hat{e}(X_i))\hat{\tau}^1_k$\ ### Summary of meta-learners: We summarise the considered meta-learners in Table \[tab:learners\] where $\hat{\psi}$ states the pseudo-outcome or estimator for each of the learners. The last column counts the number of nuisance functions needed to estimate the pseudo-outcome or estimator. In brackets, we state the total number of models needed to get the final CATE estimate. Note that the X-learner is regressed only for the treated observations and again only for the observations in the control group. This is why we need two more additional models for the final estimate. The estimators from Table \[tab:learners\] can be represented as a weighted minimization problem which solves the following: $$\begin{aligned} \min _{\tau}\left\{N^{-1} \sum_{i=1}^{N} w_{i}\left\{\hat{\psi}_{i}-\tau\left(x\right)\right\}^{2}\right\}.\end{aligned}$$ ### The choice of ML algorithms for meta-learners: The accuracy of the CATE estimation depends on the accuracy of the nuisance functions and hence on the choice of the ML method. To minimize the dependence of the ML methods on our estimates, we do not assign specific machine learning methods for the estimation but consider a range of different popular methods. To choose which ML method to use for each nuisance function as well as for any additional functions, we use a stacking method. In such a setting, not only one ML method may be chosen but an ensemble of methods that are stacked together with different weights. We use the SuperLearner package as proposed by @polley2011super. It also enables us to choose different models for each nuisance function and setting. The package offers a general class of prediction methods to be considered by the ensemble. From the 42 different algorithms, we select gradient boosted trees (`xgboost`), neural network (`nnet`) and random forest (`ranger`) for our analysis. Note that the R-learner needs to include weights to minimize the R-loss in the algorithm, so we need to make sure that the ML methods we use have this possibility included. While many researchers include the lasso in their simulations or empirical analysis, we do not use this approach. The reason is that the lasso algorithm would ideally need to assume a parametric form. This means that if we believe that there are interaction and or polynomial effects from $X$ on $Y$, we would need to include such transformations. There are extensions like the adaptive lasso that expand the feature space by including such additional factors. The computation time does however increase the more features we include. These are the main reasons why we do not include the lasso in our analysis. We use 10-fold cross-validation to estimate the performance of all machine learning models. Cross-validation is a resampling procedure used to evaluate ML models on a finite data sample. Depending on the ML model, the data can be fit perfectly and hence produce a high variance (overfitting). This is, however, on the training sample and the model can behave poorly on unseen data. Hence, we have to validate our models. We could use a part of the data for validation. Since there is never enough data, removing a part of it poses a potential for underfitting (we might lose trends in the data or important patterns). What we require instead is a method that provides enough data for training the model and also leaves enough data for validation. K-fold cross-validation does exactly that. This approach involves randomly splitting the set of observations into $K$ groups, or folds, of approximately equal size. The model is fit on folds 2 to $K$ while the first fold is used as a validation set. It is also important that any preparation of the data before fitting the model occur on the training sample that is used for cross-validation within the loop rather than on the broader data sample. This also applies to any hyperparameter tuning, for example, the number of trees, the minimum observations within a node, learning rates, or shrinkage parameters. There is no formal rule for the choice of $K$ but usually, it is set to 5 or 10. These values have been shown empirically to yield test error rate estimates that suffer neither from excessively high bias nor from very high variance. The reason is the following: The larger $K$, the smaller the difference in size between the (original) training set and the resampling subset ($K-1$ folds). As this difference decreases, the bias of the technique becomes smaller. This means that the bias is smaller for $K = 10$ than for $K = 5$. A special case of cross-validation is the leave-one-out cross-validation (LOOCV). In this case, $K$ is set to the sample size and only one observation is the validation sample. In all procedures, the $I$ resampled estimates of performance are summarized (e.g. by the mean and the standard error). Since we apply multiple models to estimate the nuisance functions, we create a weighted average among all models. Using stacking, we can find the optimal combination of a collection of prediction algorithms or even different settings within one model. In other words, we build a linear model that uses the outcome variable of the validation set as the dependent variable and all different base learners as the input variables. For the random forest, we set the following tuning parameters: `n.trees=1000, min.node.size=10`. Modified ML Methods ------------------- We now describe some methods that modify existing ML methods to estimate the CATE directly. In contrast to meta-learners that are flexible in the choice of the ML algorithm, these methods use a specific ML method (mostly tree-based algorithms). Packages or code in R are available for the causal forest (`grf`), the causal Boosting (`https://github.com/saberpowers/causalLearning`) and the causal BART (`install _github(vdorie/bartCause)`. Since causal boosting is computationally expensive, we do not consider this method in our analysis. ### Causal Forest: The causal forest method, part of the generalized random forest (GRF) by @athey2019generalized builds on a random forest algorithm to find neighborhoods in the covariate space. These neighborhoods are built by recursive splitting the covariates into subgroups while the criterion to do so is based on heterogeneity in treatment effects. The idea is to find leaves where the treatment effect is constant but different from other leaves. If we know that $\tau(x)$ were constant over some neighbourhood $N(x)$, we could solve a partially linear model over $N(x)$ using the residual-on-residual approach (see e.g. @robinson1988root): First, we estimate $e(x)= \mathsf{E}[D_i|X_i =x]$ and second, $\mu(x)= \mathsf{E}[Y_i|X_i =x]$. We can use any non-parametric method like the lasso, random forests, boosting methods, neural networks and others. The final step is to estimate $\tau(x)$ over the neighbourhood $N(x)$: $$\begin{aligned} \hat{\tau}(x)=\frac{\sum_{\left\{i: X_{i} \in \mathcal{N}(x)\right\}}\left\{Y_{i}-\hat{\mu}\left(X_{i}\right)\right\}\left\{D_{i}-\hat{e}\left(X_{i}\right)\right\}}{\sum_{\left\{i: X_{i} \in \mathcal{N}(x)\right\}}\left\{D_{i}-\hat{e}\left(X_{i}\right)\right\}^{2}}. \label{equ:R}\end{aligned}$$ Note that this approach looks similar to the R-learner. @chernozhukov2018double showed that when using any of the aforementioned ML methods for the estimation of the nuisance functions and then use the residual-on-residual approach to estimate the average treatment effect the following regularity condition holds: Given that, $$\begin{aligned} \mathsf{E}\left[\left\{\mu\left(X_{i}\right)-\hat{\mu}\left(X_{i}\right)\right\}^{2}\right]^{\frac{1}{2}} \ll \frac{1}{n^{1 / 4}}, \quad \mathsf{E}\left[\left\{e\left(X_{i}\right)-\hat{e}\left(X_{i}\right)\right\}^{2}\right]^{\frac{1}{2}} \ll \frac{1}{n^{1 / 4}},\end{aligned}$$ we get a central limit theorem such that $\sqrt{n}(\hat{\tau}-\tau) \Rightarrow \mathcal{N}(0, V)$. The treatment effect in the above setting, however, has to be constant. We can assume that with heterogeneous treatment effects, there are subgroups such that the constant effect assumption holds. The question of how to find such accurate subgroups is exactly where the (causal) random forest comes into play. To create leaves that consist of observations with the same (average) treatment effect, the splitting criterion has to rely on maximizing the heterogeneity in treatment effects between leaves (similar to maximizing the variance between the leaves). Here we use again the method from equation (\[equ:R\]). In observational studies where self-selection into treatment is present, the first splits might not be a good representation of the treatment effect rather than differences due to confounding variables. To overcome this problem, @athey2019generalized suggest applying local-centering. This means that we use the residuals of the outcome and treatment variable as data instead of the original values. Therefore one has to train two nuisance functions beforehand to predict the conditional mean which is used to create the residuals. While machine learning methods rely on sample splitting to avoid overfitting, the causal random forest integrates this via an honesty condition. A tree is honest if, for each training sample $i$, it only uses the response $Y_i$ to estimate the within-leaf treatment effect or to decide where to place the split, but not both. So far we have looked at how a single tree is build and how the final treatment effect can be estimated. To extend this procedure to multiple trees, let us view a forest as a weighting function: $$\begin{aligned} \hat{\mu}(x)=B^{-1} \sum_{b=1}^{B} \sum_{i=1}^{n} Y_{i} \frac{1\left\{X_{i} \in L_{b}(x)\right\}}{\left|L_{b}(x)\right|}=\sum_{i=1}^{n} Y_{i} \underbrace{B^{-1} \sum_{b=1}^{B} \frac{1\left\{X_{i} \in L_{b}(x)\right\}}{\left|L_{b}(x)\right|}}_{\alpha_{i}(x)} .\end{aligned}$$ Instead of seeing a forest as a double average over observations within a leaf and $B$ single trees, we can integrate the first sum to be a weighted average over all $X_i$ that fall into the leaf $L_b(x)$ and divide by the total number of observations within the leaf ($|L_b(x)|$). This weighted average tells us how often $Y_i$ falls into a certain leaf and hence the weight that we have to apply to control for the different proportions. The weights can be represented as $\alpha_i(x)$. We can now use these weights to weigh each observation in a generalized method of moments estimator where we apply a linear model, regressing the residuals of $D_i$ on the residuals of $Y_i$ and weigh by $\alpha_i$. This is how we get the CATE using a random forest. The algorithm is implemented in the `grf` package. See @friedberg2018local for an extension of this approach to local linear forests. Algorithm \[pseudo:CF\] describes the approach to estimating the CATE for each observation using the causal forest. The results from steps 4-7 are used for local-centering, as described above. If not provided in the causal forest (step 8), the nuisance functions are estimated internally. We use the $\texttt{regression\_forest}$ function to estimate the nuisance parameters. This function uses the honest estimation which means the prediction is based on out-of-bag observations. We state the estimation explicitly since it might be the case that a different ML method is better suited in predicting the conditional mean or the propensity score (e.g a boosting method). When using different methods, we just need to make sure that the predictions (steps 6 and 7) are again based on either out-of-bag observations or a different subsample. Theoretically, if relying completely on the causal forest we do not need to split the sample at all since the honest condition applies to each step (the nuisance parameters and the estimation of the CATE). Since we use $K$ fold sample splitting for all other methods we apply the same subsamples when using the causal forest. \[ht!\] Split sample $Z$ into $K$ random subsets\ **combine** $\hat{\tau}(X_i) = \{\hat{\tau}_1,\hat{\tau}_k, \ldots,\hat{\tau}_K$}\ ### Causal Boosting: An alternative to random forest based causal inference is given by @powers2018methods who introduces boosted trees and causal multivariate adaptive regression splines (MARS). By iteratively fitting weak learners to the residuals of a model, an approximation of the function is build. The idea is to fit a causal tree in the style of @wager2018estimation while setting the basis function $\hat{G}(x,D)$ to zero. Now we estimate the residuals by $Y_i - \epsilon \times \hat{g}_k(X_i,D_i)$ and update $\hat{G}_k = \hat{G}_{k-1} + \epsilon \times \hat{g}_k$. $k$ defines the terminal nodes from the tree and $\epsilon$ is the learning rate parameter. After $K$ iterations we return $\hat{G}_K(x,D)$. Estimating the CATE is done by setting $D$ to 1 for the treated observations and 0 for the control-group observations, such that: $$\begin{aligned} \hat{\tau}(x) = \hat{G}_K(x,1) - \hat{G}_K(x,0).\end{aligned}$$ Like in the causal forest the problem remains how to control for overfitting. Especially boosting methods are prone to overfit the data since the trees are not built independently. While a random forest would benefit from using more trees over which to average, in gradient boosting the number of trees is an important tuning parameter that needs to be controlled. In supervised ML we would ideally apply cross-validation. In our case, the parameter of interest is the CATE and we do not observe the true value for each observation. Hence, cross-validation does not apply here. Instead, we can do something like the honest approach from the causal forest. @powers2018methods propose to split the data into two distinct sets. The training set is used to build the causal boosting. Using the split-points and split-variables from the training set we use the covariates from the validation set, lets call it $X_v$, for validation and get new estimates based on $D_v$ and $Y_v$ for each terminal node. This procedure is done for any of the $K$ trees, using again the residuals (this time from the validation set tree) to reestimate the terminal nodes of the next causal tree. This allows estimating a validation error for each of the original $K$ models. The overall validation error for a causal boosting model is given by the differences of the CATE from the original vs. the validation trees. ### Causal BART: While (causal) boosting relies on multiplying each sequential tree by the learning rate ($\epsilon$), the idea developed by @chipman2010bart is to estimate a posterior distribution of the prediction by explicitly setting priors for the trees and ensemble structure (e.g. the depth of the tree, the probability of a new split). Using a Bayesian approach allows for a broader set of information than the point estimate from regression and classification methods. The Bayesian Additive Regression Trees (BART) approach is a combination of three methods: Using gradient boosting trees, a Bayesian framing for each individual tree, and Markov chain Monte Carlo (MCMC) sampling to do backfitting (using additive and generalized additive models for posterior sampling). @hill2011bayesian proposes to use such nonparametric Bayesian models to estimate treatment effects. Given strong ignorability, one way to estimate treatment effect is to estimate the response function $\mu(X_i,D_i)$. This function is estimated in one step instead of estimating two functions. Hence, the prior is set directly for the response surface. This approach is also called the S-learner - train one function and set $D_i$ to 1 and 0 for each observation to get estimates for both potential outcomes. @hahn2020 extends the idea of using a Bayesian approach to estimate treatment effects but expresses the response surface as: $$\begin{aligned} \mathsf{E}\left[Y_{i} \mid {X}_{i}, D_{i}=d\right]=\mu\left\{{X}_{i}, \hat{e}\left({X}_{i}\right)\right\}+\tau\left({X}_{i}\right) D_{i},\end{aligned}$$ where $\hat{e}(x)$ is the estimated propensity score and the functions $\mu(\cdot)$ and $\tau(\cdot)$ are independent BART priors. The inclusion of the estimated propensity score can be seen as a covariate-dependent prior to control for confounding bias. The method is specially designed to estimate the CATE from observational studies with small effect sizes and heterogeneous effects. The package we use is built on the model by @hill2011bayesian (`install_github(vdorie/bartCause)`). A package that implements the method proposed by @hahn2020 is in development (`install_github(socket778/XBCF)`. This package is also available for Python. Note that the causal BART produces credible intervals as a contrast to confidence intervals. They are estimated from the posterior probability function and hence rely on the prior distribution while confidence intervals are based on data only. We will only use the term confidence interval on all methods, however, we do mean credible intervals for the causal BART and (frequentists) confidence intervals for all other methods. \[ht!\] Split sample $Z$ into $K$ random subsets\ **combine** $\hat{\tau}(X_i) = \{\hat{\tau}_1,\hat{\tau}_k, \ldots,\hat{\tau}_K$}\ Sample splitting and cross-fitting {#sec:sample_splitting} ---------------------------------- To aim for a consistent estimator, we need to assume certain complexity conditions on the nuisance functions. Specifically, we want them to be smooth (i.e. differentiable) and the entropy of the candidate nuisance functions to be small enough to fulfill Donsker conditions (e.g. if we assume Lipschitz parametric functions or VC classes). In high-dimensional settings (p&gt;n) or when using ML methods that are complex or adaptive, the Donsker conditions might not hold; see, for example, @robins2013new, @chernozhukov2016locally and @rotnitzky2017multiply. As @chernozhukov2018double noticed, verification of the entropy condition is so far only available for certain classes of machine learning methods, such as lasso and post-lasso. For classes that employ cross-validation or for hybrid methods (like the SuperLearner), it is likely difficult to verify such conditions. Luckily, there is an easy solution available: sample splitting. When splitting the sample, we can use independent sets for estimating the nuisance functions and constructing the treatment estimation equation. By using different sets, we can treat the nuisance functions as fixed functions which allow avoiding conditions on the complexity. It also allows us to use any ML method such as random forest or boosting or even an ensemble of different methods. The split-sample approach to avoid smoothness conditions dates back at least to @bickel1982adaptive and was extended to also use cross-fitting by @schick1986asymptotically. To overcome a potential loss in efficiency, since only a subset of the data is used when estimating the CATE, cross-fitting is an increasingly popular approach to combine ML methods with semi-parametric estimation problems; see, for example, @chernozhukov2018double, @newey2018cross and @athey2017efficient. We note that there are two definitions of cross-fitting. First, it is defined in the context of estimating the CATE for all observations. For example, we split the data into two folds, subset A and M. We use fold A to train the nuisance functions and then estimate the parameter of interest using subset M. Now we switch the roles of the sets, using subset M for training and subset A for estimation. As a result, we get estimates of the CATE for all observations. The second definition is more in the spirit of averaging CATE estimates obtained from different partitions that are used for the nuisance parameter estimation. For example, let us say we have again the data as above but also an independent test set. Now we can use the procedure as before. First, we train the nuisance function on subset A and predict on subset B to get the pseudo-outcomes. We again train a regression function based on B but predict the CATE using the test set. Now we reverse the roles of A and B and get a second prediction of the CATE for the test set. The two results are now averaged to get the final estimate. In this tutorial, we will combine the two definitions of cross-fitting. First, we estimate the CATE on all observations through reversing roles of samples. Second, we use cross-fitting as an averaging tool over $K$ folds. When referring to cross-fitting we mainly mean the latter definition. We give an example of the benefit from cross-fitting in Figure \[fig:single\_vs\_cross-fit\]. We show the MSE from the true treatment effect for a single estimator and the cross-fit estimator based on a 50:50 sample split. We used the R-learner as the meta-learner and create 50 Monte Carlo replications of the data using the same data generating process (DGP) which simulates a RCT and has the following properties: $N = 2000$, $X = \mathbb{R}^{10}$, $e(X)$ = 0.5, and $\tau(x) = X_1 + \mathds{1}(X_2>0) + W \quad \text{with} \quad W \sim \mathcal{N}(0,0.5)$. Using cross-fitting decreases the MSE compared to the single estimator in about 90% of the cases. We also find that the variance is smaller compared to the single estimator. ![Single vs. cross-fit estimation of CATE. [![image](qletlogo_tr.png){width="2.00000%"}$_{CF}$](https://github.com/QuantLet/Meta_learner-for-Causal-ML/tree/main/Cross-Fitting)<span data-label="fig:single_vs_cross-fit"></span>](Figures/R_single_vs_cross-fit.pdf){width="80.00000%"} In empirical studies we do not have an independent test set and setting aside a partition might not be efficient since we lose observations for the estimation part. In the following, we present an approach to use cross-fitting without an additional test set. We apply 5-fold sample splitting and use 80% of the full data (denoted by $Z$) for training the nuisance functions (denoted by $S_a$) and 20% to for estimation (denoted by $S_k$). We propose to estimate the nuisance parameters for each of the 5 folds, using all folds but $k$ for training and fold $k$ to predict the conditional mean and the propensity score. We then store the estimates. As a result, we have estimates of all nuisance parameters to create the pseudo-outcomes for each observation obtained from independent samples. Hence, the above-mentioned regularity conditions should be fulfilled. Now we want to train a regression model on the pseudo-outcomes (or minimize the R-loss). Instead of using the full sample for training and prediction, we divide the sample into different parts. We assign half of the sample as the test set (denoted by $S_{oob}$, which is short for out-of-bag) and the other half that is used to train the regression model. Let us say we want to rely on 5-fold cross-fitting (taking the average of $S_{oob}$ over 5 folds). We therefore split the other half of the sample into 5 folds (denoted by $S_{train} = \{S_1,S_2,\ldots,S_5\}$). Using each fold to train the regression model and predict on $S_{oob}$ leads to 5 estimates that we average by taking the mean. Now we reverse the role of $S_{train}$ and $S_{oob}$ and proceed as above. We apply this procedure to the DR- and R-learner. The S- and T-learner only needs one estimation step and hence it suffices to only use two different samples (the $S_a$ and $S_k$). In all other methods, we need an additional model (for example, the IPW-learner would also benefit from cross-fitting). The X-learner is quite robust even without cross-fitting. This might be because it only uses the propensity score in the last step. The advantage of the two-step sample splitting approach is that we have more observations to train the nuisance functions (in this example we have $S_a = 0.8Z$ observations instead of $0.8(Z-S_{oob})$). Figure \[fig:two-step-sample-split\] shows the procedure in detail. As above, we denote $S_k$ as the fold which is used to estimate the nuisance parameters (e.g. the propensity score, the pseudo-outcomes). The
{'source': 'arXiv_out/2104'}