Graphical models for mediation analysis
Johan Steen
Department of Intensive Care Medicine, Ghent University Hospital

Stijn Vansteelandt

arXiv:1801.06069v1 [stat.ME] 18 Jan 2018

Department of Applied Mathematics, Computer Science and Statistics, Ghent University
Centre for Statistical Methodology, London School of Hygiene and Tropical Medicine

January 19, 2018
Abstract
Mediation analysis seeks to infer how much of the effect of an exposure on an outcome can be attributed to specific pathways via intermediate variables or mediators.
This requires identification of so-called path-specific effects. These express how a change
in exposure affects those intermediate variables (along certain pathways), and how the
resulting changes in those variables in turn affect the outcome (along subsequent pathways). However, unlike identification of total effects, adjustment for confounding is
insufficient for identification of path-specific effects because their magnitude is also determined by the extent to which individuals who experience large exposure effects on
the mediator, tend to experience relatively small or large mediator effects on the outcome. This chapter therefore provides an accessible review of identification strategies
under general nonparametric structural equation models (with possibly unmeasured
variables), which rule out certain such dependencies. In particular, it is shown which
path-specific effects can be identified under such models, and how this can be done.

1

Introduction

In many applications across a wide range of scientific disciplines, scholars aim to understand
the mechanisms behind established cause-effect relationships, as witnessed by the widespread
usage of mediation analyses. Such understanding may not only be of pure scientific or
etiologic interest, but may also aid policymakers in making informed decisions about public
health interventions or reforms.
The Job Search Intervention Study (JOBS II), for instance, was designed to assess the
effectiveness of a job training intervention to facilitate re-employment and reduce depressive
symptoms in unemployed job seekers [47, 48]. 1,249 randomly assigned job seekers were
invited to participate in several sessions of job search skills workshops (the treatment group),
whereas the remaining 552 unemployed workers received a booklet with job search tips (the
control group). Vinokur and Schul [48] hypothesized that the treatment group would benefit
from the workshops, assuming workshop attendance improves one’s sense of self-efficacy and
increases chances of getting re-employed, which, in turn, lead to a reduction in depressive
symptoms. Researchers thus believed an enhanced sense of mastery and re-employment to
be active ingredients of the intervention’s beneficial effect on mental health.
More generally, interventions or exposures essentially always realize their effects via a
combination of causal chains or mechanisms. Mediation analysis seeks to unravel and to
quantify specific bundles of these pathways. To fix ideas, consider the causal DAG in Figure 1, which may represent hypothesized causal mechanisms underlying the effect of the job
search intervention A on mental health Y . Suppose that workshop attendance increases participants’ sense of self-efficacy L, which may, in turn, exert beneficial effects on mental health,
1

A

L

M

Y

Figure 1: The treatment effect conceived as a combination of the effects along multiple
causal chains.
either by increasing chances of getting re-employed M (along pathway A → L → M → Y )
or by other subsequent (unspecified) mechanisms (along pathway A → L → Y ). The intervention may also positively affect re-employment through other mechanisms before finally
exerting its effect on mental health (along pathway A → M → Y ), or it may reduce depressive symptoms through none of the putative mediators (along pathway A → Y ). Mediation
analysis then aims to answer questions such as “How much of the intervention’s effect on
mental health is mediated by increased chances of re-employment?” It does so by disentangling the indirect effect that captures all pathways along which re-employment status M , the
mediator of interest, transmits the intervention effect (A → M → Y and A → L → M → Y )
from the direct effect that captures all remaining pathways (A → Y and A → L → Y ). More
generally, it aims to assess what effect the exposure realizes along one or multiple pathways.
We will informally refer to this as a path-specific effect and give a precise definition later.
Bias-free estimation of path-specific effects crucially relies on certain structural assumptions and may often be compromised due to the subtle interplay between causal mechanisms.
In this chapter, we therefore aim to further elaborate on identifiability of path-specific effects — that is, whether or not a certain set of causal assumptions suffices (or may even be
deemed necessary) to identify such components from observed data. In contrast to chapter
2, we wish to provide a more comprehensive overview of the literature on causal mediation
analysis.1 We first briefly review definitions of path-specific effects, in particular of natural
direct and indirect effects [18, 24], which are the standard targets of inference in causal
mediation analysis. Through various worked-out examples we next aim to develop intuition
into non-parametric identification2 of this class of path-specific effects and, in particular,
the technical nature of certain assumptions on which mediation analysis relies.

2

Definitions and notation

To enable clear and formal definitions of the target causal estimands, let A denote the
exposure or treatment (e.g. workshop participation) and Y the outcome of interest (e.g.
presence of depressive symptoms). Throughout, we will use counterfactual notation where,
for instance, Y (a) denotes the value of the outcome that would have been observed had
A (possibly contrary to the fact) been set to level a. The (population-)average causal
effect can then be defined as E{Y (a) − Y (a′ )}, where a and a′ correspond to meaningful
levels of treatment. This is essentially identical to the interventional contrast E(Y |do(a)) −
E(Y |do(a′ )) in terms of Pearl’s do-operator. For expositional simplicity, we will restrict
our presentation to binary treatments (with a = 1 and a′ = 0), although definitions and
results extend to multicategorical or continuous treatments. For instance, in our motivating
example, E{Y (1) − Y (0)} expresses the difference in prevalence of depressive symptoms if
all unemployed workers were invited to participate in the job search skills workshop versus
all received a booklet with job search tips.
1 All references to other sections refer to the forthcoming Handbook of Graphical Models, edited by Mathias
Drton, Steffen Lauritzen, Marloes Maathuis, and Martin Wainwright. It will contain this document as
chapter 3 of Part IV.
2 For brevity, we will loosely use terms such as ‘identification’, ‘identify’, ‘recover from observed data’ to
refer to non-parametric identification.

2

2.1

Natural direct and indirect effects

Robins and Greenland [24] laid the foundations for effect decomposition by invoking nested
counterfactuals to conceptualize the intuitive notion of changing treatment assignment along
specific pathways but not others. For instance, the nested counterfactual Y (a, M (a′ )) denotes the outcome that would have been observed had (possibly contrary to the fact) A been
set to level a and M to M (a′ ), the mediator value that had been observed had A been set to
a′ . Consequently, nested counterfactual expressions enable us to isolate and quantify part
of the intervention effect that is transmitted through the mediator M by leaving treatment
unchanged at A = 1, but changing the counterfactual intermediate outcome M (1) to M (0).
This then leads to the definition of the so-called natural indirect effect
E{Y (1) − Y (1, M (0))} = E{Y (1, M (1)) − Y (1, M (0))}
Its complement, the natural direct effect
E{Y (1) − Y (0)} − E{Y (1) − Y (1, M (0))}
= E{Y (1, M (0)) − Y (0)} = E{Y (1, M (0)) − Y (0, M (0))}
captures the notion of blocking the intervention’s effect on the mediator by keeping the
latter fixed at whatever value it would have attained under no intervention.
In our motivating example, the natural direct effect expresses by how much the prevalence
of depressive symptoms would change if all unemployed workers’ treatment assignment status
were to be changed, but their employment status were to be fixed to whatever status would
be observed if they had originally received a booklet with job search tips. In contrast,
the natural indirect effect expresses the change in prevalence of depressive symptoms if all
unemployed workers were to be invited to participate in the workshop, but their employment
status were changed to whatever status would be observed if they had received a booklet
with job search tips.
The main appeal of effect definitions that utilize nested counterfactuals, as opposed to
equivalent formulations in the linear structural equation modeling tradition, is that they are
model-free. That is, they combine to produce the total effect, irrespective of the scale of
interest or presence of interactions or nonlinearities, under the composition assumption that
Y (a, M (a)) = Y (a). For instance, although the above effects are expressed in terms of mean
(or risk) differences, the causal risk ratio of a binary outcome can similarly be expressed as
the product of the natural direct effect risk ratio and the natural indirect effect risk ratio
E{Y (1, M (0))} E{Y (1, M (1))}
E{Y (1)}
=
.
E{Y (0)}
E{Y (0, M (0))} E{Y (1, M (0))}
Consequently, mean nested counterfactuals can be parameterized using a class of marginal
structural models [25] for mediation analyis, so-called natural effect models [10, 11, 12, 34,
45], for instance
E{Y (a, M (a′ ))} = g −1 (β0 + β1 a + β2 a′ + β3 aa′ ),

(1)

for all a and a′ and where g(·) is a known link function. If g(·) is chosen to be the identity link,
β1 captures the natural direct effect and β2 + β3 captures the natural indirect effect on an
additive scale. Similarly, effects can be expressed on multiplicative scales, such as risk or odds
ratios, by choosing g(·) to represent the log or logit link function. Robins and Greenland [24]
originally termed these parameters the pure direct effect and total indirect effect, respectively.
By differently apportioning the interaction term β3 , an alternative decomposition is obtained
in terms of the total direct effect E{Y (1, M (1)) − Y (0, M (1))}, captured by β1 + β3 , and
the pure indirect effect E{Y (0, M (1)) − Y (0, M (0))}, captured by β2 . In accordance with
VanderWeele [40], any of these two decompositions can thus be further refined, leading to
the same unique three-way decomposition into the pure direct effect β1 , the pure indirect
3

effect β2 , and a mediated interactive effect β3 , which can be interpreted to capture the extent
to which direct and indirect pathways interact in their effect on the outcome.
Pearl [18] later adopted the same definitions but named these parameters natural (rather
than pure) direct and indirect effects to emphasize that pure direct effects, as opposed to
controlled direct effects E{Y (1, m) − Y (0, m)}, allow for natural variation in the mediator.
That is, natural direct effects reflect the effect of treatment upon fixing the mediator at
values that would, for each individual, have naturally occurred under no treatment, rather
than at some predetermined level m (uniformly across the population). In the remainder of
this chapter, we will adopt Pearl’s terminology of natural effects.

2.2

Path-specific effects

In graphical terms, the natural indirect effect quantifies the contribution along all pathways
through which a single mediator transmits the treatment’s effect on the outcome. Its counterpart, the natural direct effect, quantifies the contribution along all remaining pathways
from treatment to outcome. Both of their counterfactual definitions refer to specific instances
of nested counterfactuals of the form Y (a, M (a′ )), with a possibly different from a′ . Contributions along other predefined sets of directed paths π from treatment A to outcome Y can
similarly be defined in terms of contrasts of path-specific nested counterfactuals, which we
will denote Y (π, a, a′ ), in accordance with notation in the previous chapter.3 As for natural
effects, these π-specific counterfactuals represent two (possibly incompatible) hypothetical
interventions which, for instance, set A to a for the purpose of all directed paths in π, or to
a′ for the purpose of directed paths not in π. For notational convenience, we denote π to be
the set of directed pathways from A to Y not in π.
Suppose that, in our motivating example, interest lies in the effect of the job search
intervention mediated by re-employment (M in Figure 1) but not by possible prior changes
in perceived self-efficacy (L in Figure 1), as captured by π = {A → M → Y }. This π-specific
effect has been referred to as the partial [6] or semi-natural [19] indirect effect with respect
to M . Its corresponding π-specific nested counterfactual
Y ({A → M → Y }, a, a′ ) = Y (a′ , L(a′ ), M (a, L(a′ )))
can be obtained by recursive substitution, as discussed more formally in the previous chapter
(Equation 1.5). Just as counterfactuals of the form Y (a, M (a′ )) give rise to definitions for
natural effects, recursively nested counterfactuals of the above form enable us to define, for
instance, the pure π-specific effect as
E{Y (0, L(0), M (1, L(0))) − Y (0)}
and the total path-specific effect along pathways not in π (or in π) as
E{Y (1) − Y (0, L(0), M (1, L(0)))}
By symmetry and the composition assumption, these components again combine to produce
the total effect of treatment. The natural effect model
E{Y (a, L(a), M (a′ , L(a)))} = γ0 + γ1 a + γ2 a′ + γ3 aa′ ,
for all a and a′ , maps these path-specific effects to γ1 and γ2 + γ3 , respectively. Natural
effect models that parameterize more fine-grained decompositions of the total causal effect
into k > 2 path-specific effects (along k − 1 ordered mediators) have been discussed in [34].
3 For

expositional simplicity, we will restrict settings to those with A and Y being singletons.

4

3

Cross-world quantities call for cross-world assumptions

Despite the formal and intuitive appeal of path-specific effects, their non-parametric identification is subtle and a source of much controversy. The reason is that the usual consistency
assumptions alone — for instance, that M (a) = M when A = a and that Y (a, m) = Y
when A = a and M = m — do not suffice to link all counterfactuals to observed data. In
particular, nested counterfactual outcomes such as Y (a, M (a′ )) are unobservable whenever
a 6= a′ . Data, whether experimental or observational, thus never carry information about
the distribution of these counterfactuals as they imply a union of two incompatible states a
and a′ that may only seem to coexist ‘across multiple worlds’. Because of their ‘cross-world’
nature, path-specific effects cannot in general be expressed in terms of interventional contrasts, which typically refer to ideal interventions in a single hypothetical world. Mediation
analyses based on natural or path-specific effects are thus bound to rely on assumptions that
cannot be empirically verified or guaranteed by any study design [9, 24, 26].
To gradually develop intuition into non-parametric identification of natural and pathspecific effects we will work through a number of simple but typical illustrative examples,
spanning the next few sections of this chapter. Unless stated otherwise, we shall assume
throughout that treatment A is randomized, in order to exclusively focus on assumptions
characteristic to mediation analysis. In this section, we particularly highlight that the
distinct nature of nested counterfactuals calls for a type of assumption that cannot be
verified empirically but that is, nonetheless, naturally encoded in so-called non-parametric
structural equation models (NPSEMs).

3.1

Imposing cross-world independence

Identification of natural effects in the causal DAG G(V) with V = {A, M, Y }, in Figure 2,
can be obtained if we recover the distribution p(Y (a, M (a′ )) = y) of nested counterfactuals.
This requires summing (or integrating) the joint counterfactual distribution p(Y (a, m) =
y, M (a′ ) = m) over m. When a 6= a′ , observed data carry no information about the
dependence of Y (a, m) on M (a′ ). This articulates why natural effects cannot, in general,
be identified from experimental data without further, untestable assumptions. One such
assumption is that of cross-world independence
Y (a, m) ⊥
⊥ M (a′ ).

(i)

Under this assumption, we can factorize p(Y (a, m) = y, M (a′ ) = m) as a product of interventional distributions, each of which is identified from observed data under the assumptions
encoded in G(V), as follows
X
p(Y (a, M (a′ )) = y) =
p(Y (a, m) = y, M (a′ ) = m)
m

=

X

p(Y (a, m) = y)p(M (a′ ) = m) =

m

X

p(y|a, m)p(m|a′ ),

m

where for arbitrary variables V and W , p(v|w) is shorthand notation for p(V = v|W = w).4
Pearl [18] claimed cross-world assumption (i) to be key to ‘experimental’ identification of
natural effects. With this, he indicated that, if interventional distributions p(Y (a, m) =
y) and p(M (a′ ) = m) were known from previous randomized interventions do(a, m) and
do(a′ ), this assumption could be considered the missing link required to piece together these
distributions in order to recover p(Y (a, M (a′ )) = y).
4 In this chapter, we will mostly use counterfactual notation instead of Pearl’s do-notation, especially when
cross-world counterfactuals cannot be expressed using do-notation. However, we will refer to counterfactual
distributions as interventional distributions, whenever applicable.

5

A

M

Y
Figure 2: A causal DAG representing a simple yet overly simplistic mediation setting.

3.2

Cross-world independence and NPSEMs

Subtleties surrounding cross-world assumptions such as (i) have long been obscured to practitioners because of reliance on stringent parametric constraints or, more recently, on representations of causal DAGs as NPSEMs. In fact, as discussed in more detail in the previous
two chapters, NPSEMs impose restrictions on the joint distribution of all counterfactual
outcomes, including those that inhabit different worlds, that is, worlds under conflicting
hypothetical interventions or treatment assignments such as do(a) and do(a′ ). As a result,
cross-world independencies are naturally encoded by the set of (recursive) structural equations that defines a particular NPSEM. For instance, the NPSEM representation of G(V) in
Figure 2 is characterized by the following set of structural equations:
A := fA (ǫA )
M := fM (A, ǫM )
Y := fY (A, M, ǫY )
where fA , fM and fY are unknown deterministic functions and ǫA , ǫM and ǫY are mutually independent random error terms (representing unobserved background variables). The
assumed invariance of these equations endows them with a causal interpretation and permits us to deduce the counterfactual independencies they encode. For example, under the
interventions do(a, m) and do(a′ ), the structural equations can respectively be written as
A := a′

A := a

M (a′ ) := fM (a′ , ǫM )
Y (a′ ) := fY (a′ , M (a′ ), ǫY )

M (a) := m
Y (a, m) := fY (a, m, ǫY )

Under this representation, the joint distribution of the one-step ahead counterfactuals
V (xpaG (V ) ) := fV (xpaG (V ) , ǫV ),
where paG (V ) denotes the set of parents of V in G(V) and xpaG (V ) the set of values to
which these parents are set via the intervention do(x), is fully determined by the mutually
independent error terms ǫV . It thus follows that all such one-step ahead counterfactuals
are also mutually independent, irrespective of the choice of hypothetical values xpaG (V ) to
which we set the parents of V . As a result, independence of the error terms ǫM ⊥
⊥ ǫY
in the above structural equations not only translates into Y (a, m) ⊥
⊥ M (a) but also into
cross-world independence (i). This may sound reassuring, but also signals the restrictiveness
of NPSEMs, as they inherently seem to encode independence assumptions that can never
be verified from randomized interventions.

3.3

Single world versus multiple worlds models

Robins and Richardson [26] extensively discuss these restrictions encoded by NPSEMs.
They moreover contrast the latter with another class of graphical causal models, Robins’
[23] Finest Fully Randomized Causally Interpretable Structured Tree Graph Model (FFRCISTGM) representation of causal DAGs, which only enforces restrictions that are (in principal) empirically verifiable. Because this less restrictive class of models only imposes independence restrictions on sets of counterfactuals under a single set of (non-conflicting) interventions such models have been referred to as ‘single world models’, as opposed to NPSEMs
6

which were termed ‘multiple worlds models’. A more formal treatment of NPSEMs, FFRCISTGs and Single World Intervention Graphs (SWIGs) [22], which encode counterfactual
independencies implied by a ‘single world model’, is given in chapters 1, 2 and 5.

3.4

Further outline

Because NPSEMs naturally encode cross-world independence assumptions, they have provided a framework for the recent development of a fairly intuitive graphical rule that governs
whether and how nested ‘cross-world’ counterfactual quantities relate to observed variables
[1], even in the presence of unobserved or hidden variables [27]. In section 5, we demonstrate
that specific cross-world independence assumptions can indeed be relatively easily interrogated from a (hidden variable) causal DAG interpreted as an NPSEM by this graphical rule.
As it turns out, this type of assumption forms the extra necessary layer on top of a set of
assumptions that is subject to experimental verification and serves to identify total treatment effects. When combined with complete identification algorithms for total treatment
effects [5, 29, 38], the proposed graphical criterion therefore not only delineates sufficient,
but also necessary conditions for identification of path-specific effects.
Essentially, when it comes to natural direct and indirect effects, this sound and complete
criterion indicates that identification can be obtained under NPSEMs with the aid of two
different types of auxiliary variables, provided that no mediator-outcome confounders are
themselves affected by treatment (and the total treatment effect is identifiable). As the
connection with earlier sufficient identification conditions for natural effects [18] seems to be
somewhat missing from the literature, we choose to review the main (two) graphical identification algorithms in chronological order (in sections 4 and 5, respectively) and to revisit
earlier assumptions in the light of this recent graphical criterion (in section 6). In doing so,
we point out that certain identification strategies have long been concealed because of the
initial and exclusive focus on a single type of auxiliary variable that recovers identification
by establishing a conditional version of cross-world assumption (i). Finally, in section 7, we
provide insights that may help to put a longstanding conceptual discussion regarding the
very nature of mediation analysis into perspective.

4

Identification 1.0

In this section, we further extend the simple causal DAG in Figure 2 to illustrate the logic
and reasoning behind sufficient conditions for identification of natural effects.

4.1

Unmeasured mediator-outcome confounding

In most settings, the assumptions encoded by Figure 2 are unrealistic. Indeed, even if
treatment were randomized, as represented by the absence of back-door paths into A, we
cannot generally assume the absence of confounding of the mediator-outcome relation (other
than by A) because typically M is not randomized. Nonetheless, independence of the error
terms ǫM ⊥
⊥ ǫY , as encoded in the NPSEM representation of Figure 2, critically hinges on the
assumption of no unmeasured mediator-outcome confounding. Because the latter assumption
can be considered unlikely, the assumption of independent error terms is therefore almost
guaranteed to be violated. In this subsection, we will therefore relax assumptions by adding
a hidden node U that captures unmeasured confounding of the mediator-outcome relation
(and induces dependence between their respective error terms when structural equations
are expressed only in terms of observed variables V), as in Figure 3A. More generally, we
will represent unobserved variables H on hidden variable causal DAGs G(V ∪ H) by circled
nodes.

7

(A)
A

(B)
M

U

(C)

A

U

M

A

M

C

U

C
Y

Y

Y

Figure 3: Causal DAGs that reflect more realistic mediation settings with unmeasured
mediator-outcome confounding (A) along with two scenarios where a measured covariate C
may deconfound the mediator-outcome relation (B,C).
By treatment randomization we have that U ⊥
⊥ A such that, not surprisingly, the gformula [23] yields
X
X
p(Y (a) = y) =
p(y|a, m, u)p(m|a, u)p(u) =
p(y|a, m)p(m|a) = p(y|a).
(2)
u,m

m

Unfortunately, U cannot similarly be integrated out ‘across worlds’, which prevents us from
identifying p(Y (a, M (a′ )) = y), even if conditional cross-world independence Y (a, m) ⊥
⊥
M (a′ )|U were to hold. Indeed, we obtain
X
p(Y (a, M (a′ )) = y) =
p(Y (a, m) = y|u)p(M (a′ ) = m|u)p(u)
u,m

=

X

p(y|a, m, u)p(m|a′ , u)p(u),

(3)

u,m

an expression that cannot further be reduced to a functional of observed variables (such as
Equation 2) because of the conflicting treatment assignments in its first two factors.

4.2

Adjusting for mediator-outcome confounding

Issues of non-identifiability of p(Y (a, M (a′ )) = y) may, however, be remedied when one has
available a measured set of prognostic covariates C ⊆ V \ {A, M, Y } for mediator and/or
outcome that renders the mediator-outcome relationship unconfounded given treatment assignment. This can be understood because the availability of such a set C, as, for instance,
in the simplified example DAGs in Figures 3B and 3C where C = {C}, no longer necessitates
stratifying on hidden variables such as U to establish cross-world independence.
For example, in Figure 3B, conditioning on C suffices, since the structural equations
M (a′ ) := fM (a′ , U, ǫM )
Y (a, m) := fY (a, m, C, ǫY ),
indicate that cross-world independence holds within strata of C, that is
Y (a, m) ⊥
⊥ M (a′ )|C.

(ii)

This then implies the same functional as expression (3) but with unobserved U replaced by
the observed adjustment set C
X
p(Y (a, M (a′ )) = y) =
p(Y (a, m) = y|c)p(M (a′ ) = m|c)p(c)
(4)
c,m

=

X

p(y|a, m, c)p(m|a′ , c)p(c).

c,m

8

(5)

(A)
A

(B)
M

L

A

M

Y

Y

(C)
A

L
U

(D)
M

Y

L

A

U

M

Y

L
U

Figure 4: Causal DAGs that reflect mediation settings with treatment-induced confounding
by L.
This functional is commonly referred to as Pearl’s [18] mediation formula.
To appreciate the importance of adjustment for prognostic factors C, reconsider our
motivating example. Randomization of the intervention in itself did not suffice to eliminate
potential confounding between re-employment M and the outcome. It is therefore essential
to adjust for the pretreatment level of depression, a strong prognostic factor of the outcome
of interest and most likely also related to re-employment. Measurements on a range of other
baseline covariates, including demographics, previous occupation and financial strain, were
also collected and adjusted for to strengthen the validity of cross-world assumption (ii).

4.3

Treatment-induced mediator-outcome confounding

The previous example may have led the reader to erroneously conclude that, given treatment
randomization, adjustment for a measured covariate set C that deconfounds the mediatoroutcome relation within treatment arms suffices to establish cross-world independence (ii)
under NPSEMs, thus enabling identification of p(Y (a, M (a′ )) = y). An important additional
requirement is that no prognostic factor L ∈ C is affected by treatment.
Intuitively, if L were a common cause of both M and Y , as in the causal DAG G(V)
with V = {A, L, M, Y }, in Figure 4A, adjustment for L (as in Equation 5 with C = {L})
would block the pathway A → L → M → Y , which makes up part of the natural indirect
effect of interest. Lack of identification can formally be understood as follows. According
to the NPSEM associated with Figure 4A, Y (a, m) ⊥
⊥ M (a′ ) holds conditional on {L(a) =
′
′
l, L(a ) = l } since
M (a′ ) := fM (a′ , L(a′ ), ǫM )
Y (a, m) := fY (a, m, L(a), ǫY ).
Under the remaining assumptions encoded by this NPSEM, this allows us to express p(Y (a, M (a′ )) =
y) as
X
p(Y (a, m) = y|L(a) = l, L(a′ ) = l′ )p(M (a′ ) = m|L(a) = l, L(a′ ) = l′ )
l,l′ ,m
× p(L(a) = l, L(a′ ) = l′ )
X
=
p(y|a, l, m)p(m|a′ , l′ )p(L(a) = l, L(a′ ) = l′ ).
l,l′ ,m

9

As in the example of the previous section, this expression cannot further be reduced to
a functional of the observed data as it requires the joint cross-world counterfactual distribution p(L(a) = l, L(a′ ) = l′ ). Because this distribution again involves conflicting treatment
assignments, strong untestable restrictions (beyond those encoded in NPSEMs) would be
needed to enable identification.
In our motivating example, all available covariates were measured prior to randomization. It may thus be safely assumed that none of them was affected by the intervention.
However, other mediators of the intervention’s effect on mental health, such as an altered
sense of self-efficacy, may well have affected re-employment and thus manifest themselves
as mediator-outcome confounders that are affected by the intervention. In that case, crossworld independence (ii) is likely violated.

4.4

Pearl’s graphical criteria for conditional cross-world independence

Pearl [18] devised two graphical criteria for assessing cross-world independence (ii) under
an NPSEM associated with a certain hidden variable causal DAG G(V ∪ H). The logic for
these criteria can be understood from the previous two examples in sections 4.1 and 4.3.
The first criterion requires the availability of an adjustment set C that is sufficient,
along with treatment A, to adjust for confounding of the mediator-outcome relation. Such
covariate set C should block all back-door paths between M and Y (except those traversing
A) in the sense that
(Y ⊥
⊥ M |C)G(V∪H)AM .

(ii.a)

That is, C d-separates Y from M in G(V ∪ H)AM , the subgraph constructed from the
original graph G(V ∪ H) by deleting all arrows emanating from A and M .
The second criterion requires that
no element of C is affected by treatment.

(ii.b)

We will henceforth refer to this criterion as ‘no treatment-induced confounding’ or ‘no intermediate confounding’.
In the next two subsections, we review sufficient conditions for identifying natural direct
and indirect effects from i) experimental data from studies where treatment is randomized
or from ii) observational data. In doing so, we highlight that identification from purely
observational data typically requires additional assumptions, which (in contrast to crossworld assumption (ii)) are empirically falsifiable. Following Pearl [19], we compare different
formulations of these additonal assumptions in terms of their identification power.

4.5

Sufficient conditions to recover natural effects from experimental data

Equation 4 illustrates that cross-world independence (ii) enables expressing the cross-world
counterfactual distribution p(Y (a, M (a′ )) = y) in terms of ‘single world’ interventional distributions p(M (a′ ) = m|c) and p(Y (a, m) = y|c). It is easily demonstrated that these
interventional distributions are identified under an NPSEM if treatment is randomized and
cross-world independence (ii) holds. In other words, when combined with the ignorability
condition that represents treatment randomization
{Y (a, m), M (a), C} ⊥
⊥ A,

(iii)

conditional cross-world independence (ii) enables identification of p(Y (a, M (a′ )) = y) under an NPSEM from data obtained from a single randomized intervention do(a) [7]. The

10

latter implication could be considered an extension of Pearl’s [18] ‘experimental’ identification, which formulates that marginal cross-world independence (i) suffices to recover
p(Y (a, M (a′ )) = y) from two sequentially randomized interventions do(a, m) and do(a′ ) (see
section 3.1; also see [9]). This extension basically illustrates that, under a single randomized
intervention do(a), we need a measured set of baseline covariates C such that identification can be obtained under conditional cross-world independence (ii), without reliance on
additional ignorability assumptions.

4.6

Sufficient conditions to recover natural effects from observational data

As opposed to randomized trials, assumption (ii) is not sufficient for identifying natural
effects from purely observational data. This is because recovering interventional distributions p(M (a′ ) = m|c) and p(Y (a, m) = y|c) from observational data requires additional
assumptions.
4.6.1

The adjustment criterion for natural effects

The availability of an adjustment set C that simultaneously satisfies assumption (ii) and
the following conditional ignorability assumption
{Y (a, m), M (a)} ⊥
⊥ A|C

(iv)

restores identifiability of p(Y (a, M (a′ )) = y) under an NPSEM from observational data by
the mediation formula (Equation 5) [8, 32]. Moreover, because, under NPSEMs, assumptions (ii) and (iv) are exchangeable (as a set) with the following set of conditional ignorability
assumptions5 [32]
M (a) ⊥
⊥ A|C
Y (a, m) ⊥
⊥ {A, M }|C,

(v.a)
(v.b)

it follows that the search for such a sufficient adjustment set C may as well be restricted to
covariate sets that simultaneously identify p(M (a) = m) and p(Y (a, m) = y) by the backdoor [17] or adjustment formula [33]. This led Shpitser and VanderWeele [32] to develop
a complete graphical criterion for identification of p(Y (a, M (a′ )) = y) by the mediation
formula (under NPSEMs). They instead termed this the adjustment formula for natural
direct and indirect effects as it generalizes the adjustment criterion for total effects [33] to
mediation settings.
Intuitively, this criterion can be thought of as aiming to establish both cross-world independence (ii) and conditions (v.a) and (v.b) solely by means of adjustment for a common
measured covariate set C. First, it demands no unmeasured mediator-outcome confounding
(as in Figure 3A), which, if not met, violates cross-world independence (ii) and, moreover,
hampers identification of p(Y (a, m) = y) by the adjustment formula. Second, it demands
the absence of treatment-induced mediator-outcome confounders (such as L in Figure 4A),
because their presence both violates cross-world independence (ii) and hinders the availability of a common set C that enables identification of both p(M (a) = m) and p(Y (a, m) = y)
by the adjustment formula. Crucially, establishing cross-world independence (ii) and conditions (v.a) and (v.b) by means of this generalized adjustment criterion goes hand in hand.
A relatively simple example Consider the causal DAG G(V∪H) with observed variables
V = {C1 , C2 , C3 , A, M, Y } and hidden variables H = {U1 , U2 , U3 } in Figure 5 (adopted from
[19]; Figure 3B). The search for a candidate covariate set C that satisfies assumption (ii) may
5 Assumption (v.b) encodes cross-world independence, be it in a more subtle way. That is, by the consistency assumption, it implies Y (a, m) ⊥
⊥ M (a′ )|{A = a′ , C}, which is inherently cross-world counterfactual.

11

G(V ∪ H)

G(V ∪ H)AM

C2

C2

U3

U3

U1
A

U1
C1

M

A

U2

C3

C1

M
U2

C3

Y

Y

Figure 5: Hidden variable causal DAG G(V ∪ H) which permits identification of
p(Y (a, M (a′ )) = y) by the adjustment criterion (under its NPSEM representation). The
subgraph G(V ∪ H)AM aids in selecting a candidate covariate set that satisfies cross-world
independence (ii) by Pearl’s graphical criteria described in section 4.4.
be guided by graphical criteria (ii.a) and (ii.b), as discussed in section 4.4. Because G(V∪H)
includes no intermediate confounders, it suffices to search for a set of baseline covariates
that d-separates M from Y in the subgraph G(V ∪ H)AM . These candidate adjustment
sets include {C1 , C2 }, {C1 , C3 } and {C1 , C2 , C3 }. Assessing whether p(Y (a, M (a′ )) = y)
can be recovered from observed data by the mediation formula now boils down to verifying
whether both p(M (a) = m) and p(Y (a, m) = y) are identified by the adjustment formula
upon adjustment for one of these candidate sets. It turns out that only the set {C1 , C2 , C3 }
identifies both of these interventional distributions by the adjustment formula, such that
under the NPSEM representation of G(V ∪ H), we obtain
X
p(y|a, m, c1 , c2 , c3 )p(m|a′ , c1 , c2 , c3 )p(c1 , c2 , c3 ).
p(Y (a, M (a′ )) = y) =
c1 ,c2 ,c3 ,m

In fact, under NPSEMs, any covariate set C that suffices to identify both p(M (a) = m) and
p(Y (a, m) = y) by the adjustment formula, will also satisfy cross-world independence (ii),
such that the initial step can simply be skipped [32].
4.6.2

Identification beyond the adjustment criterion

A major appeal of identification via the adjustment criterion for natural effects is that it
leads to a standard identifying functional. This, in turn, allows for general modeling and
estimation strategies. However, as the following examples illustrate, it may unnecessarily
increase modeling demands and limit the ability to identify p(Y (a, M (a′ )) = y).
A relatively simple example revisited By exploiting the following exclusion restrictions encoded in G(V ∪ H) in Figure 5
M⊥
⊥ C3 |{A, C1 , C2 }
Y ⊥
⊥ C2 |{A, M, C1 , C3 },
the identification result can be simplified as
X
p(y|a, m, c1 , c3 )p(m|a′ , c1 , c2 )p(c1 , c2 , c3 ),
c1 ,c2 ,c3 ,m

thereby reducing modeling demands (although see [7] for a critical discussion on such exclusion restrictions).
12

G(V ∪ H)

G(V)

C1

C1
U2

C2

A

U1

M

C2

Y

C3

A

M

U3

C3
G(V)V\A

Y

G(V)Y∗

C1

C1

C2

M

C3

Y

M

C3

Y

Figure 6: A somewhat more involved hidden variable causal DAG G(V ∪ H), its latent
projection ADMG G(V) and subgraphs G(V)V\A and G(V)Y∗ .
A somewhat more involved example Consider next the causal DAG G(V ∪ H) with
observed variables V = {C1 , C2 , C3 , A, M, Y } and hidden variables H = {U1 , U2 , U3 } in
Figure 6 (adopted from [19]; Figure 5F). To identify p(M (a) = m) we must, in any case,
adjust for C2 . Since C3 is a collider, adjusting for it opens spurious pathways that cannot
be blocked by additionally adjusting for C1 , leaving {C2 } and {C1 , C2 } as the only viable
adjustment sets for identifying p(M (a) = m) by the adjustment formula. However, identification of p(Y (a, m) = y) by the adjustment formula requires that C3 is included in the
adjustment set, because the back-door path from A to Y via U3 can only be blocked by C3 .
As a result, p(Y (a, M (a′ )) = y) cannot be identified by the adjustment criterion for natural
direct and indirect effects.
Nonetheless, identification can be obtained by resorting to an alternative identification
strategy. Such strategy may consist of first listing all sufficient adjustment sets that identify
p(M (a) = m) and all sufficient adjustment sets that identify p(Y (a, m) = y). Progress can
then be made if a subset of the intersection of any two of these respective candidate adjustment sets satisfies assumption (ii). For instance, p(M (a) = m) is identified by adjusting for
{C1 , C2 }, whereas p(Y (a, m) = y) is identified by adjusting for {C1 , C3 }. Moreover, {C1 },
the intersection of these separate adjustment sets, satisfies cross-world independence (ii).
Relying on the conditional independence Y ⊥
⊥ C1 |{A, M, C3 }, p(Y (a, M (a′ )) = y) may then
be identified from observed data by
X
p(Y (a, m) = y|c1 )p(M (a′ ) = m|c1 )p(c1 )
c1 ,m
!
!
X X
X
′
=
p(y|a, m, c1 , c3 )p(c3 |c1 )
p(m|a , c1 , c2 )p(c2 ) p(c1 )
c1 ,m

=

X

c3

c2

′

p(y|a, m, c3 )p(m|a , c1 , c2 )p(c1 )p(c2 )p(c3 |c1 ).

c1 ,c2 ,c3 ,m

13

(6)

The above examples demonstrate that p(M (a′ ) = m|c) and p(Y (a, m) = y|c) in Equation 4 can be identified under a much wider range of scenarios than those that lead to
identification by the adjustment criterion for natural effects [19]. That is, identification of
p(Y (a, M (a′ )) = y) can be obtained if, for any candidate covariate set C that satisfies (ii),
p(M (a′ ) = m|c) and p(Y (a, m) = y|c) are identified by Shpitser’s complete IDC algorithm for conditional treatment effects [28] (see section 1.3.6 of the previous chapter). This
resonates Pearl’s original formulations [18] which state that, to recover ‘non-experimental’
identification, assumption (ii) needs to be complemented with the following two assumptions
p(M (a) = m|c) is identifiable by some means, and
p(Y (a, m) = y|c) is identifiable by some means.

(vi.a)
(vi.b)

One way to increase identification power is by resorting to the ‘divide and conquer’
strategy described in the previous example, which Pearl [19] referred to as piecemeal deconfounding. However, in certain settings, this strategy may still be overly restrictive and
identification may then, instead, sometimes be recovered by exploiting so-called mediating
instruments [19]. That is, if specific instruments can be found that fully mediate certain
crucial but confounded paths (that cannot be deconfounded by observed covariates), further
progress can be made by local application of the front-door formula. Specific examples are
given in [19].

5

Identification 2.0

Widening the scope to also include mediating instruments in our ‘identification toolbox’
still does not enable us to fully characterize all possible settings that enable non-parametric
identification of p(Y (a, M (a′ ))) under NPSEMs. In other words, while the assumption
set (ii)-(vi.a)-(vi.b) may be sufficient for recovering natural effects from observational data, it
is not necessary and, consequently, not complete for identification. This lack of completeness
can be demonstrated using a simple illustrating example.
Cross-world independence (ii) is violated in the causal DAG G(V ∪ H) with V =
{A, L, M, Y } and H = {U }, in Figure 7A, because of unmeasured mediator-outcome confounding. Nonetheless, by exploiting both conditional independencies that are naturally
encoded in G(V ∪ H) and conditional counterfactual independencies implied by the following NPSEM representation of G(V ∪ H)
L(a′ ) := fL (a′ , ǫL )
M (l) := fM (l, U, ǫM )
Y (a, m) := fY (a, m, U, ǫY ),
p(Y (a, M (a′ )) = y) can be identified from observed data as follows
p(Y (a, M (a′ )) = y) = p(Y (a, M (L(a′ ))) = y)
X
=
p(Y (a, m) = y, M (l) = m, L(a′ ) = l)
l,m

=

X

p(Y (a, m) = y|u)p(M (l) = m|u)p(L(a′ ) = l)p(u)

u,l,m

=

X

p(y|a, m, u)p(m|l, u)p(l|a′)p(u)

u,l,m

=

X

p(y|a, l, m, u)p(m|a, l, u)p(l|a′)p(u|a, l)

u,l,m

=

X

p(y|a, l)p(l|a′ ).

l

14

(7)

(A)
A

(B)
L

M

A

U

M

U

Z
Y

Y

(C)
A

(D)
L

M

A

U

Z

L

M

U

Z
Y

Y

Figure 7: Hidden variable causal DAGs with with mediating instruments L for the path
A → M (A), Z for the path A → Y (B), and a combination of both (C). The DAG in panel
(D) is an extended deterministic graph of the DAG in Figure 3A, with thick edges indicating
a deterministic relationship.
This result should not come as a big surprise: since the effect of treatment on the mediator
is (assumed to be) entirely mediated by L, and, in addition, L only affects the outcome
via M , L can simply substitute for M . However, the sufficient conditions outlined so far
(especially assumption (ii)) do not naturally lead to this simple result.
In the remainder of this section, we therefore take a step back. Armed with the tools
and concepts from the previous chapter, we take a closer look at the commonalities that
characterize the key problems in the examples in sections 4.1 and 4.3. The resulting insights
offer a framework that allows to extend complete algorithms for identification of total causal
effects (see sections 1.3.4 to 1.3.6 of the previous chapter) to mediation settings. Moreover,
this extension has produced a complete graphical criterion for identification under NPSEMs,
not only of natural direct and indirect effects, but of path-specific effects in general [27]. In
sections 6 and 7, we highlight that this graphical criterion gives rise to complementary
identification strategies that may, in addition, help to shed new light on an ongoing debate
about the controversial cross-world nature of path-specific effects.

5.1

Building blocks for complete graphical identification criteria

Key to violation of assumptions (ii.a) and (ii.b) in the examples in sections 4.1 and 4.3,
respectively, is the occurrence of conflicting treatment assignments in certain factors of the
identifying functional. It can be shown that this problem arises whenever the conflict is situated in factors involving distributions of ancestors of the outcome — which, by convention,
include the outcome itself — that belong to a common confounded component (abbreviated:
c-component ) [38] or district [21]. In this subsection, we first provide the necessary conceptual background on districts, district factorization and complete identification algorithms
for total treatment effects, as discussed in more technical detail in the previous chapter.
Confounded paths, districts and district factorizations Following [30], we first define a confounded path to be a path where all directed arrowheads point at observed nodes,
and never away from observed nodes. To avoid cluttering causal DAGs G(V ∪ H) with
large numbers of hidden variables, unobserved common causes of any two observed nodes
are often omitted, while their presence is, instead, indicated by bidirected edges (↔). This
latent projection operation, as discussed in previous chapters, gives rise to acyclic directed
mixed graphs (ADMGs) [21], which contain only observed nodes V and both directed and

15

bidirected edges (see section 1.3 of chapter 2). These graphs encode conditional independencies between observed variables via m-separation [21], a graphical criterion closely related to
d-separation for causal DAGs containing only directed edges. Throughout, we will denote
latent projection ADMGs of hidden variable causal DAGs G(V ∪ H) by G(V). In ADMGs,
confounded paths can similarly be defined as paths that contain only bidirected edges.
A district may now be defined as the maximal set of observed nodes that are pairwise
connected by confounded paths. The set of all observed nodes can thus be partitioned into
disjoint districts S ∈ D(G(V)), where D(G(V)) denotes the set of districts in the latent
projection G(V). Each of these districts consists of either a single observed node or a set
of observed nodes that are pairwise connected by confounded paths. The importance of
districts may be appreciated by the fact that their disjointness implies that the marginal
distribution of observed variables p(xV ) in G(V ∪ H) factorizes as the product of their
corresponding kernels or c-factors
Y
Q[S],
p(xV ) =
S∈D(G(V))

where each kernel corresponds to
X Y
Q[S] =
p(xV |xpaG (V ) , xuG (V ) )p(xuG (S) ),
xuG (S) V ∈S

and where paG (V ) and uG (V ) denote the set of observed and unobserved parents of V in
G(V ∪ H), respectively. For each Q[S], the product runs across all observed nodes V ∈ S
and the summation over all possible realisations of unobserved parents of V ∈ S.
For instance, in the causal DAG G(V ∪ H) with V = {A, M, Y } and H = {U }, in
Figure 3A, M and Y are connected by a confounded path as they share an unmeasured
parent U . The set of observed variables can thus be partitioned into two districts: {A} and
{M, Y }. Moreover, p(a, m, y) factorizes as the product of the kernels of these districts:
X
p(a, m, y) = Q[{A}]Q[{M, Y }] = p(a)
p(y|a, m, u)p(m|a, u)p(u).
u

Tian and Pearl [38] pointed out that every Q[S] can be interpreted as p(xS |do(xpaG (S) )),
the distribution of S under an intervention that sets all its observed parents paG (S) to
xpaG (S) . Moreover, they proved that every Q[S], for which S ∈ D(G(V)), is identifiable from
observed data. For example, since A ⊥
⊥ U in Figure 3A, it is relatively trivial to show that
X
Q[{M, Y }] =
p(y|a, m, u)p(m|a, u)p(u) = p(y|a, m)p(m|a).
u

We next illustrate that districts (and their corresponding kernels) form the building blocks
of complete graphical identification algorithms for total causal effects [5, 29, 38], since interventional distributions can always be expressed as the marginal of a truncated district
factorization.
Truncated district factorizations in hidden variable causal DAGs In hidden variable causal DAGs G(V ∪H), observed nodes V ∈ V that are connected by confounded paths
group together into districts S ∈ D(G(V)). Consequently, factorizations of p(xV ) involve
kernels that can be interpreted as multivariate interventional distributions. Because district
factorizations extend usual Markov factorizations for causally sufficient DAGs to hidden
variable causal DAGs with respect to p(xV ), this permits us to express p(Y (a) = y) as the
marginal of a truncated version of the district factorization of p(xV ):
X
Y
p(xD |do(apaG (D)∩A , xpaG (D)\A )),
(8)
p(Y (a) = y) =
xY ∗ \Y D∈D(G(V)Y ∗ )

16

where apaG (D)∩A = a if there exists a directed path of the form A → D → ... → Y (with
D ∈ D), apaG (D)∩A = ∅ if no such path exists, and where Y∗ ≡ anGV\A (Y ) denotes the
set of ancestors of Y (including Y ) in a subgraph G(V)V\A of the latent projection G(V).
Here, the product runs across all districts D ∈ D(G(V)Y∗ ) in a subgraph of G(V), and the
summation is made over all possible realisations of the nodes in these districts, except for
the outcome.
Equation 8 indicates that the original problem of identifiability of p(Y (a) = y) can be
reduced to a set of smaller identification problems within a subgraph of G(V ∪H). Logically,
each district D ∈ D(G(V)Y∗ ) is a subset of a district S ∈ D(G(V)). Even though each kernel
Q[S] is identifiable from observed data, identification of some kernels Q[D] may fail under
certain hidden variable causal models. In fact, p(Y (a) = y) is identifiable if and only if
every kernel Q[D] in Equation 8 can be recovered from observed data. In the absence of
hidden variables, D(G(V)Y∗ ) = D(G(V)), such that p(Y (a) = y) is always identifiable from
observed data and Equation 8 reduces to the well-known g-formula [23].
For instance, in G(V ∪ H) in Figure 3A, Y∗ = V \ A = {M, Y }, such that there is only
one district {M, Y } in G(V)Y∗ . Its corresponding kernel Q[{M, Y }] in Equation 8 perfectly
corresponds to a kernel in the latent projection G(V), such that it can readily be identified
from observed data. Consequently, p(Y (a) = y) is identified by
X
X
X
p(y, m|do(a)) =
p(y|a, m, u)p(m|a, u)p(u) =
p(y|a, m)p(m|a) = p(y|a).
(9)
m

u,m

m

In contrast to this simple example, truncation of district factorizations can become much
more complicated under hidden variable DAGs. That is, whenever a district D ∈ D(G(V)Y∗ )
is a proper subset of a district S ∈ D(G(V)), identification of Q[D] requires (repeated)
application of the ‘fixing operation’ (as described in the previous chapter) and may eventually
fail.6 Problematic graphical structures that hinder identification under hidden variable
causal models have been discussed in detail in [28].

5.2

The central notion of recantation

Having provided the necessary theoretical and conceptual background on complete identification algorithms for total treatment effects, we now pick up where we left off in the beginning
of section 5.1 and provide more formality and generality by introducing the central notion
of recantation. This notion will enable us to map crucial cross-world independencies onto an
intuitive graphical criterion under NPSEMs. It can therefore be viewed to serve as a passkey
that permits easy translations from cross-world quantities, used to define path-specific effects, to ‘single-world’ interventional quantities. Identification of the latter can then simply
be passed on to well-established algorithms for identifying total causal effects. Ultimately,
when combined with such complete algorithms, this graphical criterion does not only delineate complete identification criteria for natural effects, but also for more generally defined
path-specific effects.
5.2.1

The recanting witness criterion

Cross-world counterfactual Y (a, M (a′ )) in Figure 4A corresponds to Y (a, L(a), M (a′ , L(a′ )))
and thus represents the response of the outcome to two hypothetical interventions which set
to A to a for the purpose of π = {A → Y ; A → L → Y }, on the one hand, and to a′ for
the purpose of π = {A → M → Y ; A → L → M → Y }, on the other hand. However, even
though the interventional distribution p(Y (a) = y) is identified by
X
X
p(y|do(a, l, m))p(m|do(a, l))p(l|do(a)) =
p(y|a, l, m)p(m|a, l)p(l|a) = p(y|a),
l,m

l,m

6 Application of the ‘fixing operation’ in a conditional ADMGs (as described in the previous chapter) is
essentially equivalent to the systematic removal of certain non-essential nodes in subgraphs of the latent
projection ADMG G(V).

17

identification of p(Y (a, M (a′ )) = y) is hampered because of conflicting treatment assignments a and a′ in the single node district L. Here, L is called a recanting witness [1], for the
following reason. Identification of the natural indirect effect via M requires a first statement
from L that blocks the path A → L → Y in order to keep treatment from transmitting its
effect on the outcome other than through M (as this path is regarded part of the natural
direct effect). However, L subsequently needs to retract this statement in favor of a new
statement which refrains from blocking the path A → L → M → Y in order to allow treatment to transmit its entire effect on the mediator (as blocking would imply adjusting away
part of the natural indirect effect). Clearly, we can’t have it both ways.
The recanting witness criterion [1] formalizes this requirement of having no such witnesses
along π to enable identification of the π-specific effect. More specifically, a child L of
treatment A is called a recanting witness7 for π (and by symmetry, also for π) if there exists
a directed path in π of the form A → L → ... → Y and another directed path in π of the
form A → L → ... → Y . Avin, Shpitser and Pearl [1] demonstrated that if and only if
there is no recanting witness for π in a causal DAG G(V) without hidden variables, then the
π-specific effect is identified under the NPSEM representation of G(V). More specifically,
the distribution of the corresponding nested counterfactual Y (π, a, a′ ) is then identified from
observed data as
X
Y
p(xV |apaπG (V )∩A , a′paπ (V )∩A , xpaG (V )\A ),
p(Y (π, a, a′ ) = y) =
(10)
G

xV\(A∪Y ) V ∈V\A

where paπG (V ) denotes the set of parents of V in G(V) along an edge which is part of a path
in π. This result has been referred to as the edge g-formula [31] because it generalizes the
ordinary g-formula [23] in that it permits different treatment assignments along separate sets
of edges A → V . More specifically, in different Markov factors, treatment A is set to either a
or a′ depending on whether or not {A → V → ... → Y } ∈ π. The recanting witness criterion
thus implicitly imposes the restriction that a single edge A → V can only be assigned a single
treatment value or, in other words, that treatment assignment must be edge consistent for
p(Y (π, a, a′ ) = y) to be identified [31]. The mediation formula (Equation 5) can be viewed
as a specific case of this more general identifying functional.
5.2.2

The recanting district criterion

Even though the recanting witness criterion gives a complete characterization of settings
when path-specific effects are identified under DAGs without hidden variables, it does not
suffice as a graphical identification criterion in hidden variable DAGs. This can be seen
from the illustrating example in section 3, which suffers from unmeasured mediator-outcome
confounding. For instance, even though p(Y (a) = y) is identified by Equation 9 under the
hidden
variable causal DAG G(V ∪ H) in Figure 3A, p(Y (a, M (a′ )) = y) is not identified by
P
p(y|a,
m)p(m|a′ ), simply because we cannot readily integrate out U under conflicting
m
treatment assignments a and a′ . Identification of p(Y (a, M (a′ )) = y) is thus hindered
because of conflicting treatment assignments a and a′ in the district {M, Y }, where A is set
to a for the purpose of π = {A → Y } and to a′ for the purpose of π = {A → M → Y }.
Inspired by complete algorithms for identifying p(Y (a) = y), based on district factorizations, Shpitser [27] recently extended the recanting witness criterion to hidden variable
DAG settings. This extension is conceptually fairly simple. As districts rather than single
nodes are the building blocks of the factorization of p(xV ), the term ‘witness’ simply needs
to be replaced by the term ‘district’. Informally, this extended criterion requires there to be
no ‘conflict of interest’ between members of a common district within a particular subgraph
G(V)Y∗ . Formally, a district D ∈ D(G(V)Y∗ ) is said to be a recanting district for π if there
7 The definition we provide here is restricted to π-specific effects from A to Y , with both A and Y being
singletons. A more general definition requires making reference to proper causal paths, and is given in the
previous chapter.

18

exists a directed path in π of the form A → D → ... → Y as well as a directed path in π of
the form A → D′ → ... → Y , where D, D′ ∈ D (and possibly D = D′ ).
Shpitser [27], moreover, showed that only in the absence of a recanting district for π, the
cross-world counterfactual distribution p(Y (π, a, a′ ) = y) can be expressed as a functional
of interventional distributions
X
Y
(11)
p(xD |do(apaπG (D)∩A , a′paπ (D)∩A , xpaG (D)\A )).
p(Y (π, a, a′ ) = y) =
G

xY ∗ \Y D∈D(GY ∗ )

This functional is a generalization of the truncated district factorization used to identify
p(Y (a) = y) in hidden variable causal DAGs (Equation 8) via Tian’s ID algorithm [38].
Equation 11 closely matches the formulation in Equation 10 in that it also permits different treatment assignments along separate sets of edges, while replacing single nodes V by
districts D ∈ D(G(V)Y∗ ). It can therefore also be considered an extension of the edge
g-formula to hidden variable causal DAGs. However, an additional condition must hold to
enable further translation of the involved interventional quantities onto observable quantities. That is, whereas each of the kernels in Equation 10 is identifiable from observed data in
causal DAGs without hidden variables, identifiability of kernels in Equation 11 is not guaranteed because of the assumed presence of hidden variables. Nonetheless, if p(Y (a) = y)
is identified via Tian’s ID algorithm, this logically implies that each kernel in the above
functional is expressible in terms of observed data. Importantly, the recanting district criterion thus needs to be complemented by identifiability of the total causal effect p(Y (a) = y)
to give a complete characterization of identification conditions for path-specific effects under hidden variable causal DAGs (interpreted as NPSEMs) [27]. Whereas the recanting
district criterion enables translations from cross-world counterfactual quantities into ‘single
world’ interventional quantities, the ID algorithm then verifies whether these interventional
quantities can be expressed as functionals of the observed data.
A somewhat more involved example revisited Consider again G(V∪H) in Figure 6A,
along with the subgraphs of interest G(V)V\A and G(V)Y∗ . Since there is no recanting
district for the set of pathways that capture the natural (in)direct effect in G(V)Y∗ , the
identifying functional for p(Y (a, M (a′ )) = y) can be expressed as
X
p(y|do(a, m, c3 ))p(m|do(a′ , c1 ))p(c3 |do(c1 ))p(c1 )
(12)
c1 ,c3 ,m

by application of Equation 11. Moreover, because p(Y (a) = y) is identified by
X
p(y|a, m, c3 )p(m|a, c1 , c2 )p(c1 )p(c2 )p(c3 |c1 )

(13)

c1 ,c2 ,c3 ,m

via Tian’s ID algorithm, we can re-express Equation 12 as Equation 6 by simply plugging
in appropriate treatment assignments in the respective factors of Equation 13.
Note that Pearl’s ‘divide and conquer’ approach, as discussed in section 4.6.2, required
searching the space of candidate covariate sets C that not only satisfy cross-world indendence (ii) but also conditions (vi.a) and (vi.b). Shpitser’s identification approach is therefore
not only (more) complete, but arguably also more insightful as it clarifies that the main difficulty in identifying p(Y (a, M (a′ )) = y) is identification of the total treatment effect, which
involves repeated application of the fixing operator.
5.2.3

A new perspective on cross-world independence

From the perspective of the recanting district criterion, the need for an observed covariate
set C that is sufficient to adjust for confounding of the mediator-outcome relation (given
treatment) serves to establish that mediator and outcome belong to separate districts so
19

that conflicting treatment assignments causes no further identification problems (provided
that no member of C is affected by treatment). For instance, in Figures 3B and C, a
sufficient adjustment set {C} enables to pull apart the district {M, Y } and resolve the
conflict in order to ensure the validity of cross-world assumption (ii) that permits factorizing
p(Y (a, m) = y, M (a′ ) = m|c) as p(Y (a, m) = y|c)p(M (a′ ) = m|c).
Importantly, the central notion of recantation thus groups Pearl’s graphical criteria (ii.a)
and (ii.b) for establishing cross-world independence (ii) under NPSEMs by offering a framework that allows their respective violations to be interpreted as distinct instances of essentially the same problem. As will be discussed in the next section, the implications of this
graphical criterion reach beyond Pearl’s [18] sufficient conditions, as discussed in section 4.

6

Complementary identification strategies

The completeness of Shpitser’s [27] new identification approach reveals that Pearl’s [18]
conditions (ii)-(vi.a)-(vi.b) may not be necessary to recover natural effects from observed
data. That is, simply combining cross-world independence (ii) and identifiability of the total
causal effect may suffice to identify p(Y (a, M (a′ )) = y). When cross-world independence (ii)
can thus be established upon adjustment for a covariate set C, one may simply assess
identifiability of p(Y (a) = y) via the ID algorithm instead of assessing identifiability of
both p(M (a) = m|c) and p(Y (a, m) = y|c) via the more complicated IDC algorithm.
Interestingly, the completeness of this novel result also highlights that, in some rare
cases, cross-world independence (ii) — despite being a sufficient condition for ‘experimental’
identification [18] — may not be required either. This was already exemplified by the case
of Figure 7A, which we now revisit.

6.1

Interchanging cross-world assumptions

Careful inspection of the hidden variable causal DAG G(V ∪ H) in Figure 7A yields that
Y∗ = {L, M, Y }, such that the subgraph G(V)Y∗ can be partitioned into districts {L} and
{M, Y }. The absence of a recanting district for π = {A → Y }, which transmits the natural
direct effect with respect to M , and identifiability of p(Y (a) = y), by randomization of
treatment, then leads to the same identification result for p(Y (a, M (a′ )) = y) as obtained
in Equation 7, but derived more elegantly via application of Equation 11:
X
X
X
p(y, m|do(a, l))p(l|do(a′ )) =
p(y|a, l, m)p(m|a, l)p(l|a′ ) =
p(y|a, l)p(l|a′ ).
l,m

l,m

l

This result can be explained by the fact that, in this case, the recanting district criterion
does not serve to establish identifiability via cross-world independence (ii), but, instead,
via the alternative cross-world independence assumption Y (a, m) ⊥
⊥ L(a′ ) encoded in the
NPSEM representation of G(V ∪ H). Indeed, the derivations in Equation 7 illustrate that
the mediating instrument L achieves to prevent the conflict between treatment assignments
a and a′ from taking place within the district {M, Y } by diverting treatment state a′ to
itself, thereby fulfilling its mediating role, literally and figuratively. A crucial insight here
is that when L is assumed to mediate the entire treatment effect on the mediator M , then
the latter is no longer a child of A, and hence, cannot receive any input from A that may
conflict with input to other children of A in the same district.
A mediating instrument on the path between treatment and outcome, such as Z in
Figure 7B, would similarly allow to make progress upon substituting (ii) by cross-world
independence Z(a) ⊥
⊥ M (a′ ).

6.2

Two types of auxiliary variables

The above examples illustrate that, when p(Y (a) = y) is identifiable, further identification
of p(Y (a, M (a′ )) = y) by the recanting district criterion can be achieved under NPSEMs
20

with the aid of two types of auxiliary variables. Each type can be viewed to have its own
distinct strategy for preventing recantation.8
The first type, such as C in Figures 3B and 3C, aims to prevent conflicting treatment
assignments within districts by separating nodes of a common district, such as {M, Y } in
Figure 3A, which is bound to recant due to unmeasured mediator-outcome confounding,
into different districts. Adjustment for this type of covariates specifically aims to strengthen
cross-world assumption (ii). The second type, such as L or Z in Figures 7A, 7B and 7C,
avoids conflicts in a specific district (such as {M, Y }) not by separating its nodes into different districts, but instead hosting one potential ‘troublemaker’ in its own district. Such
mediating instruments therefore do not aspire to establish assumption (ii), but instead target identification by means of alternative cross-world assumptions that may substitute for
assumption (ii).
This result is important because mediating instruments, while useful to identify p(M (a) =
m|c) and/or p(Y (a, m) = y|c) in order to satisfy conditions (vi.a) and (vi.b), cannot aid in
(avoiding recantation by) establishing cross-world independence (ii) because this can only
be achieved by means of covariate adjustment [19]. The recanting district criterion reveals
the extended utility of mediating instruments as auxiliary variables that may, nonetheless,
help to avoid recantation by establishing cross-world independencies that substitute for
cross-world independence (ii).

6.3

Mediating instruments — some reasons for skepticism

Contrary to the long-held belief that identification of p(Y (a, M (a′ )) = y) hinges on the
assumption that no mediator-outcome confounding is left unadjusted, mediating instruments
arm us with additional identification power in the presence of such unmeasured confounding.
This provides researchers different identification strategies, each relative to a specific set of
assumptions. One may use this as a basis for a sensitivity analysis, or adopt the strategy that
corresponds with the most plausible assumptions given a certain research context. However,
some caution is warranted.
First, the recanting district criterion indicates that, when resorting to mediating instruments, the requirement of no unmeasured confounding is simply shifted from the mediatoroutcome relation to both the instrument-mediator and instrument-outcome relations. This
can be seen upon noting that either type of unmeasured confounding results in the instrument being ‘absorbed’ into the district {M, Y }, such that it expands to {L, M, Y } in
Figure 7A or to {Z, M, Y } in Figure 7B, respectively. Both of these would, however, be
recanting with respect to the set of pathways π that transmit either the natural direct or
natural indirect effect. As for mediator-outcome confounding, neither types of unmeasured
confounding can be avoided by treatment randomization.
Second, the assumption that L or Z is a mediating instrument involves strong and
often unrealistic exclusion restrictions.9 For instance, for L in Figure 7A to be a mediating
instrument, it would need to mediate the entire effect of A on M . Despite being a strong
assumption, it is partially testable from observed data when there is no unmeasured L − M
or A − M confounding, as in Figure 7A, for then M must be conditionally independent
of A, given L. Likewise, for Z in Figure 7B to be a mediating instrument for the path
A → Y , it would need to mediate the entire direct treatment effect on the outcome that is
not mediated by M . However, the requirement that Z and M together mediate the entire
treatment effect, which implies Y ⊥
⊥ A|{Z, M }, is untestable in the presence of unmeasured
mediator-outcome confounding. Even though these assumptions cannot directly be verified
8 Note that this classification is analogous to the one often used for auxiliary variables that aid identification of treatment effects, where identification can be achieved via two main strategies: using either
the back-door criterion (i.e. standard adjustment for covariates) or the front-door criterion (i.e. sequential
adjustment by means of a mediating instrument).
9 Importantly, L or Z may also correspond to covariate sets (rather than being singletons) that satisfy
the stated conditions.

21

from observed data, in principle, along with the aforementioned unmeasured confounding
assumptions, they lend themselves to experimental verification.
Third, mediating instruments do not resolve the previously considered identification
problems in the presence of treatment-induced mediator-outcome confounding by a recanting
witness. For instance, the exclusion restriction that L does not directly affect Y in Figure 7A,
or that Z does not affect M in Figure 7B, can be thought of as a constraint that prevents
the instrument from turning into a recanting witness.

7

From mediating instruments to conceptual clarity

Even though the practical use of mediating instruments, as an alternative route to identification of natural effects, may be debatable, their added value is more immediate on a
conceptual level. Such instruments may help to frame some recent conceptual development
that aims to cast mediation analysis into a more strict interventionist paradigm, void of
untestable cross-world assumptions [26]. Before going on to discuss this development, we
briefly sketch some difficulties that may arise when interpreting natural effects, at least from
an interventional point of view.

7.1

In search of operational definitions

When it comes to the interpretation of natural direct effects, critics adhering to the slogan
‘no causation without manipulation’ have repeatedly emphasized the operational question
of how exactly one may go about blocking the treatment’s effect on the mediator, in order
to recover M (0) in treated subjects, without affecting the direct path from treatment to
outcome [e.g. 4]. Inevitably, any answer to this question invokes a mediating instrument,
such as L in Figure 7A, that can be intervened on in order to prevent treatment from
exerting its effect on the mediator. Likewise, it is difficult to imagine an intervention that
would block only the direct path from treatment to outcome, without conceptualizing a
mediating instrument such as Z in Figure 7B.

7.2

Deterministic expanded graphs

It thus seems that mediating instruments provide some sort of necessary extension to the
original causal diagram that permits interventionist interpretations of natural effects. The
conceptual notion of an expanded graph with two mediating instruments, as depicted in
Figure 7C, corresponds very closely to what has been described by Robins and Richardson
[26]. In settings where L and Z can confidently be considered mediating instruments, the
recanting district criterion tells us that, given that p(Y (a) = y) is identifiable, identification of p(Y (a, M (a′ )) = y) may be obtained if the instruments are in separate districts
and if neither of the instruments affects the other. The associated cross-world assumption
Z(a) ⊥
⊥ L(a′ ) indeed formalizes the need for no unmeasured confounding between the two
instruments. However, this cannot be guaranteed unless both L and Z are deterministic
functions of (a randomized) treatment [26]. In that case, both Z(a) and L(a′ ) are constants,
and hence trivially independent.10
Ironically, this required determinism seems to leave us incapable of pulling apart the
pathways that we meant to separate in the first place. However, progress can be made
if one can conceive of separate interventions on L and Z that would enable to break their
perfect correlation. From this perspective, the deterministic characterization of an expanded
causal DAG, such as Figure 7D, gives rise to a specific type of experimental design that
requires one to think of L and Z as inherent but distinct properties of the treatment,
which may be intervened on separately but, when combined, fully capture all of its active
ingredients. The feasibility of such designs primarily mirrors the extent to which different
10 In

addition, as shown in [26], independence of Z(a) and L(a′ ) leads to cross-world assumption (ii).

22

active components of treatment can be conceived of being manipulated in isolation [3].
Moreover, when combined with the aforementioned exclusion restrictions, such designs thus
entail separate manipulations of L and Z, which capture distinct but exhaustive features of
treatment to which, respectively, solely M or Y are (directly) responsive. Importantly, this
characterization enables to interpret natural effects as specific interventional contrasts.
Consider the causal DAG G(V ∪ H) with observed variables V = {A, M, Y } and hidden
variable H = {U } in Figure 3A, and its deterministic expansion G ′ (V′ ∪ H), with V′ = V ∪
{Z, L}, in Figure 7D. Let Z and L be deterministic functions which can be conceived as two
complementary components that fully characterize what we will refer to as the ‘composite’
treatment A such that A ≡ {L, Z}, a ≡ {aL , aZ }, a′ ≡ {a′L , a′Z }, p(l|ã) = 1{l = ãL } and
p(z|ã) = 1{z = ãZ }. As pointed out by Robins and Richardson [26], p(Y (a, M (a′ )) = y)
then corresponds to the interventional distribution p(Y (aZ , a′L ) = y) since
X
p(Y (a, M (a′ )) = y) =
p(y, m|do(z, l))p(z|do(a))p(l|do(a′ ))
z,l,m

=

X

p(y|z, l, m)p(m|l)p(z|a)p(l|a′ )

(14)

z,l,m

=

X

p(y|z, l)1{z = aZ }1{l = a′L }

z,l

= p(y|aZ , a′L ) = p(Y (aZ , a′L ) = y).

(15)

The first equality is obtained by application of Equation 11, the third by conditional independence M ⊥
⊥ Z|L and determinism, and the second and last by Tian’s ID algorithm.
Note that, because Z(a) ⊥
⊥ L(a′ ) holds by determinism rather than by independence restrictions implied by NPSEMs, this result can be obtained under the ‘single world’ model
associated with the causal DAG in Figure 7D.
The above result implies that, if deterministic mediating instruments like L and Z can
be assumed to exist and the aforementioned exclusion restrictions are deemed plausible,
it is not necessary to actually conduct any experiment, nor to assume any cross-world
independencies to identify the interventional distribution p(Y (aZ , a′L ) = y). Instead, if
Y (aZ , a′L ) = Y (a, M (a′ )) under G ′ (V′ ∪ H) with V′ = V ∪ {Z, L}, then identification of
p(Y (aZ , a′L ) = y) is tantamount to identification of p(Y (a, M (a′ )) = y) from observed data
on V under the ‘single world’ causal model representation of G(V ∪ H) [26]. For instance, if
one merely assumes the existence of some (unidentified) deterministic mediating instruments
L and Z in Figure 7D, measurements on L and Z are typically missing and p(a′L , aZ ) = 0
(for a′L 6= aZ ) in the observed sample. Consequently, we can only express Equation 14 in
terms of observable data on V = {A, M, Y } in the absence of unmeasured mediator-outcome
confounding by U . To recover identifiability we will thus generally need to complement V
with an additional set of observable auxiliary variables of the types described in section 6.2.

7.3

Some examples

Some existing designs, such as double-blind placebo-controlled trials, were in fact devised
in the spirit of Robins and Richardson’s [26] deterministic extended graphs [3]. Such trials
aim to isolate part of the effect of the drug A that may be attributed to active chemical
components Z, and is not mediated by the patient’s or doctor’s expectations about the effectiveness of the drug M . In such designs it is often reasonable to assume that expectations are
solely affected by the knowledge of (possibly) being treated L and that the active component
itself does not affect expectations. The natural direct effect of the drug, not mediated by
expectations, could therefore be interpreted as the interventional contrast comparing drug
effectiveness between the treatment and placebo arm. Note that experimental designs that
reflect an expanded deterministic graph do not require any measurements on the mediator
to identify interventional contrasts that correspond to certain natural effects.
23

Unfortunately, success is not always guaranteed. Side effects in the treatment arm may,
for instance, raise suspicions of being on active treatment, thereby violating the crucial
exclusion restriction M ⊥
⊥ Z|L encoded in Figure 7D. To accommodate for known side
effects, active placebos have been designed that mimick side effects of the active treatment
[2, 13], illustrating that the ability to increase the credibility of required exclusion restrictions
may often be highly dependent on the creativity of the researcher [26].11
In other contexts, experimental designs in the spirit of deterministic extended graphs
are more difficult to conceive. For instance, even though the JOBS II study [48] involved
a job search skills workshop that targeted specific component processes grounded in psychological theory, it may still be hard to imagine similar interventions or workshops that
isolate the distinct triggering elements of separate targeted processes, let alone, to conceive
of distinct elements that exclusively affect either re-employment or mental health (via direct
pathways). Any attempt to endow natural direct and indirect effects with an interventionist
interpretation would thus necessarily rely on strong theoretical assertions about the active
components of the job training intervention.

8

Path-specific effects for multiple mediators

The focus of this chapter has hitherto been restricted to identification of natural effects.
The recanting district criterion, however, delineates conditions that permit identification of
any effect along any bundle of pathways that may be of interest. Its utility may thus be
particularly appealing in settings with intertwined pathways along multiple mediators or
longitudinal settings where both treatment and/or mediators may be time-varying.
Because of the inherent cross-world nature of path-specific effects, non-parametric identification necessarily always relies on untestable cross-world independence assumptions. A
major appeal of the recanting district criterion is that it makes explicit formulations of
relevant path-specific cross-world independence assumptions essentially redundant for the
purpose of identification under NPSEM representations of hidden variable DAGs, as illustrated below. In general, decompositions of the treatment effect into path-specific effects
other than natural direct and indirect effects may be motivated by non-identifiability of natural effects or by the simple fact that the primary mediation hypothesis cannot be expressed
in terms of such effects.
Alternative decompositions in the presence of intermediate confounding In our
motivating example, the natural indirect effect with respect to re-employment M is not
identified if re-employment and mental health Y are believed to be subject to treatmentinduced confounding by changes in perceived self-efficacy, as denoted by L in Figure 4A.
If, nonetheless, the main interest is in the mediating role of re-employment, we may either
calculate partial identification bounds for the natural indirect effect (see [14] and references
therein), conduct a sensitivity analysis (see [41] and references therein), or abandon focus on
the natural indirect effect altogether. Instead, shifting focus to less ambitious decompositions
in terms of either the joint natural indirect effect mediated by both perceived self-efficacy and
re-employment (transmitted by π1 = {A → M → Y ; A → L → Y ; A → L → M → Y }), or
the partial indirect effect with respect to re-employment (transmitted by π2 = {A → M →
Y }), may still to some extent enable us to learn about the mediating role of re-employment.
The joint natural indirect effect with respect to {L, M } requires recovering the crossworld counterfactual distribution p(Y (π1 , a, a′ ) = y) = p(Y (a′ , L(a), M (a)) = y) and is
identifiable even if we relax assumptions to allow for unmeasured confounding between sense
of self-efficacy and re-employment, as in Figure 3.4C. This can easily be seen upon noting
11 In a strict sense, active placebo designs also violate the required exclusion restrictions. Nonetheless,
they enable to arrive at a measure of a direct effect that more closely resembles the natural direct effect of
primary interest [2].

24

that the district {L, M } is not recanting with respect to π1 . In this case, the recanting
district criterion serves to establish cross-world independence
Y (a, l, m) ⊥
⊥ {M (a′ , l), L(a′ )}.
When combined with experimentally verifiable identifying assumptions for p(Y (a) = y), as
encoded in the ‘single-world’ representation of the hidden variable DAG, this cross-world
assumption renders p(Y (a′ , L(a), M (a)) = y) identifiable from observed data. In contrast,
districts {M, Y } and {L, Y } are recanting with respect to π1 , as in Figures 3.4B and D
respectively, such that p(Y (a′ , L(a), M (a)) = y) is not identifiable under unmeasured confounding of the relation between the outcome and any of the given intermediate variables
along paths in π1 . Nonetheless, identification can be restored in the presence of a mediating
instrument, for instance, on the edge A → M or A → Y if it is hindered due to unmeasured
M − Y confounding.
The partial indirect effect with respect to re-employment, on the other hand, requires
recovering p(Y (π2 , a, a′ ) = y) = p(Y (a′ , L(a′ ), M (a, L(a′ ))) = y). This cross-world counterfactual distribution remains identifiable if we relax assumptions by allowing for unmeasured
confounding between sense of self-efficacy and mental health, as in Figure 3.4D, because the
district {L, Y } is not recanting with respect to π2 . Allowing for unmeasured confounding
of the relation between re-employment M and either the outcome Y or intermediate confounder L, in contrast, destroys identification because of recantation of the districts {M, Y }
and {L, M } with respect to π2 , in Figures 3.4B and C, respectively. The violated cross-world
independence assumption
{Y (a, l, m), L(a)} ⊥
⊥ M (a′ , l)
which enables identification of p(Y (a′ , L(a′ ), M (a, L(a′ )) = y), can, however, again be interchanged with another assumption which restores identification in the presence of a mediating
instrument on one of the respective edges emanating from A.
Addressing different types of mediation questions In certain cases, the partial indirect effect with respect to a mediator of interest M that is affected by earlier mediators, may,
however, be the primary path-specific effect of interest. For example, Miles and colleagues
[15] aimed to assess the extent to which treatment adherence driven by non-toxicity factors
mediates the effect of antiretroviral therapy (ART) on virological failure in HIV patients
in Nigeria. Ignoring potential baseline confounders, their target of inference corresponds to
the path-specific effect along π = {A → M → Y } in Figure 3.4D, where A denotes ART, L
drug toxicity, M adherence and Y viral load. Their corresponding mediation analysis thus
aimed to answer the question “How much of the medication’s effect is mediated by adherence, if we discard the mediating role of adherence driven by drug toxicity?” Estimation of
this contribution to the total effect of ART on viral load enabled Miles and colleagues to
address questions that are not only etiologically relevant but that may also have important
policy implications. In fact, the corresponding mediation analysis aimed to assess whether
conceivable modifications to the treatment regimen, which may increase adherence (but not
through changes in toxicity), may magnify the net treatment effect and hence increase its
effectiveness. The interpretation of the partial indirect effect as an interventional contrast,
which could be estimated from such a hypothetical experiment, can likewise be represented
via a deterministic expanded DAG. A more detailed discussion of deterministic expanded
DAGs for path-specific effects, as discussed in [26], is, however, beyond the scope of this
chapter.

9

Discussion and further challenges

Most developments on the identification of natural direct and indirect effects have so far
focused on single mediator settings where cross-world independence (ii) holds along with
25

conditional ignorability assumptions (v.a) and (v.b). That is, where the data-generating
mechanism can be described by an NPSEM in which a common set of baseline covariates
C suffices to adjust for confounding of the treatment-mediator, treatment-outcome and
mediator-outcome associations (within levels of treatment), and where moreover none of the
elements of C is affected by treatment. The latter two requirements have, to a large extent,
prohibited extensions to settings with multiple, possibly longitudinal, mediators.
Recently, a complete graphical criterion has been devised for identification of any pathspecific effect of a treatment on an outcome under NPSEMs [27]. Briefly, it shows that
when the total causal effect is identifiable by some means, as can be verified using Tian’s ID
algorithm [38] (as discussed in the previous chapter), then every path-specific effect (along a
set of pathways) for which there is no recanting district is also identifiable. Identification then
essentially proceeds via Tian’s identifying functional (Equation 8), which extends Robins’ gfunctional [23] to (NPSEM representations of) hidden variable causal DAGs, while allowing
treatment assignments to be different across districts (Equation 11).
Increased identification power It is not too hard to come up with examples of settings
where the adjustment criterion for natural effects fails to identify p(Y (a, M (a′ )) = y), due
to violations of assumption set (v.a)-(v.b), yet the recanting district criterion leads to identifiable natural direct and indirect effects (see Figures 6 and 7, respectively). Even so, from
a practitioner’s point of view, it can be argued that the associated increased identification
power is of limited practical relevance in single mediator settings. The reason is that prior
knowledge in practice is often too limited to justify assumptions that could substitute for
failure of assumptions (v.a)-(v.b) [7]. On the other hand, one cannot ignore the potential
of causal structure learning algorithms (see chapter 4), especially in the ‘big data’ age, in
which we have increasing access to massive data sets. In particular, such algorithms may
aid researchers to construct a class of DAGs that are compatible with the observed data distribution and under which identification may be obtained under more general assumptions
as delineated by the ID algorithm and the recanting district criterion.
Importantly, the recanting district criterion has been proposed as a criterion that, given
identifiability of p(Y (a) = y), delineates conditions for identifying marginal (or populationaveraged) path-specific distributions p(Y (π, a, a′ ) = y). Nonetheless, its utility is still unclear (but definitely more subtle) when it comes to identification of conditional (or stratumspecific) path-specific distributions p(Y (π, a, a′ ) = y|c). Generalizations for complete identification criteria for conditional path-specific effects are undoubtedly less straightforward,
and are left as subject for further research. Consequently, the increased identification power
that follows from Shpitser’s results is currently only well-documented for marginal natural direct and indirect effects, since the adjustment criterion for natural direct and indirect effects delineates identical conditions for identifying both p(Y (a, M (a′ )) = y) and
p(Y (a, M (a′ )) = y|c∗ ) whenever C∗ is a subset of a set of baseline covariates C that controls for mediator-outcome confounding.
Estimation If identification of natural effects is achieved under the above ‘traditional’ but
more stringent set of assumptions, this leads to a standard identifying functional, generally
known as the mediation formula, for which a well-established suite of (semi-)parametric
estimators has been developed (see [44] and references therein). Accordingly, estimation
may then proceed via routine application of these methods as implemented in off-the-shelf
statistical software packages (see [35] and references therein). Even though software implementations of complete graphical identification algorithms, such as the ID algorithm, are
now publicly available [39], estimation of more involved or less standard identifying functionals arguably imposes another barrier to routine application of complete identification
algorithms. Inevitably, this may have led to a trade-off between postulating realistic causal
structure, on the one hand, and simple and accessible estimation strategies, on the other
hand. Future research thus needs to focus on the development of a generic and flexible esti-

26

mation framework for more generic identifying functionals. Such framework should not only
incorporate estimation of natural effects, but also of more generally defined path-specific
effects (see [15, 34] for some first promising steps in this direction).
Broadening the scope Even though its added value for identification of natural effects
may be debatable, the recanting district criterion offers a major potential for extensions to
settings with multiple, possibly longitudinal, mediators. The identification of the partial
indirect effect via a given mediator of interest, as in Figure 4D, forms a first step towards
such extension, as it allows for possibly high-dimensional post-treatment confounders to
confound the mediator-outcome association. In particular, it allows for earlier mediators to
be confounders of the association between later mediators and outcome, while at the same
time being confounded with the outcome by unmeasured common causes.
Cross-world contemplations Both the cross-world nature of path-specific effects and
the required cross-world independence assumptions for identification have been the subject
of an ongoing debate [e.g. 16, 26], roughly dividing the field into NPSEM ‘skeptics’ and
‘advocates’. We have tried to shed some light on this controversy, and illustrated the important role of mediating instruments and deterministic expanded graphs [26] in elucidating and
bridging this conceptual and ontological divide. The main objection is that such cross-world
independence assumptions, on which modern causal mediation analysis generally relies (although see [36] for a recent exception), cannot be enforced experimentally and hence are not
falsifiable. However, whether or not researchers should be encouraged to reformulate their
mediational hypotheses in terms of feasible potential interventions on defining features of
treatment, remains an open question.
An alternative approach to avoiding cross-world definitions and assumptions has recently
gained increasing attention. This approach builds on the claim that, even in the absence of
any reference to cross-world quantities or restrictions, certain contrasts based on the mediation formula may still carry empirically meaningful interpretations [4, 20]. This has given
rise to the more formal definition of so-called randomized intervention analogs of natural
effects [43, 42, 46], which conceive of setting the mediator at some level that is randomly
assigned from the conditional counterfactual mediator distribution p(M (a′ ) = m|c) rather
than at the individual counterfactual level (see [13] for a related approach). Importantly,
because their definitions do not employ cross-world counterfactuals strong and unfalsifiable
assumptions, such as cross-world independence (but also ‘no intermediate confounding’)
may be avoided. These estimands also tend to correspond more closely to relevant policy
measures that can be estimated from actual interventions.
Even if one is willing to make untestable cross-world assumptions, identification of natural
effects is typically hindered in the presence of treatment-induced confounding. Accordingly,
several contributions to the field have articulated alternative assumptions that may allow us
to recover natural effects despite treatment-induced confounding. However, such assumptions generally impose additional structure on the joint distribution of counterfactuals, such
as rank preservation [26], monotonicity [37] or parametric constraints [24, 20, 37].

10

Acknowledgements

The authors would like to thank Vanessa Didelez, Yves Rosseel, Karel Vermeulen and one
anonymous referee for helpful suggestions and feedback on an earlier draft, and Ilya Shpitser
for valuable discussions that have led to some improved insights presented in this chapter.
This work was supported by Research Foundation Flanders (FWO Grant G.0111.12).

27

References
[1] Chen Avin, Ilya Shpitser, and Judea Pearl. Identifiability of Path-Specific Effects.
In Proceedings of the 19th International Joint Conference on Artificial Intelligence,
IJCAI’05, pages 357–363, San Francisco, CA, USA, 2005. Morgan Kaufmann Publishers
Inc.
[2] Vanessa Didelez. Basic concepts of causal mediation analysis and some extensions. Talk
at Symposium on causal mediation analysis, Ghent, 2013.
[3] Vanessa Didelez. Discussion on the paper by Imai, Tingley and Yamamoto. Journal of
the Royal Statistical Society. Series A (Statistics in Society), 176(1):39, 2013.
[4] Vanessa Didelez, A Philip Dawid, and Sara Geneletti. Direct and Indirect Effects of
Sequential Treatments. In Rina Dechter and Thomas Richardson, editors, Proceedings
of the Twenty-Second Conference on Uncertainty in Artificial Intelligence (UAI-06),
pages 138–146, Arlington, Virginia, 2006. AUAI Press.
[5] Yimin Huang and Marco Valtorta. Identifiability in causal bayesian networks: A sound
and complete algorithm. Proceedings of the National Conference on Artificial Intelligence, 21(2):1149, 2006.
[6] Martin Huber. Identifying Causal Mechanisms (Primarily) Based on Inverse Probability
Weighting. Journal of Applied Econometrics, 29(6):920–943, sep 2014.
[7] Kosuke Imai, Luke Keele, Dustin Tingley, and Teppei Yamamoto. Comment on Pearl:
Practical Implications of Theoretical Results for Causal Mediation Analysis. Psychological Methods, 19(4):482–487, 2014.
[8] Kosuke Imai, Luke Keele, and Teppei Yamamoto. Identification, Inference and Sensitivity Analysis for Causal Mediation Effects. Statistical Science, 25(1):51–71, feb 2010.
[9] Kosuke Imai, Dustin Tingley, and Teppei Yamamoto. Experimental Designs for Identifying Causal Mechanisms. Journal of the Royal Statistical Society A, 176(1):5–51, jan
2013.
[10] Theis Lange, Mette Rasmussen, and Lau Caspar Thygesen. Assessing Natural Direct
and Indirect Effects Through Multiple Pathways. American Journal of Epidemiology,
179(4):513–8, feb 2014.
[11] Theis Lange, Stijn Vansteelandt, and Maarten Bekaert. A Simple Unified Approach
for Estimating Natural Direct and Indirect Effects. American Journal of Epidemiology,
176(3):190–195, jul 2012.
[12] Tom Loeys, Beatrijs Moerkerke, Olivia De Smet, Ann Buysse, Johan Steen, and Stijn
Vansteelandt. Flexible Mediation Analysis in the Presence of Nonlinear Relations:
Beyond the Mediation Formula. Multivariate Behavioral Research, 48(6):871–894, nov
2013.
[13] Judith J Lok. Defining and estimating causal direct and indirect effects when setting
the mediator to specific values is not feasible. Statistics in Medicine, (October 2015),
2016.
[14] Caleb H Miles, Phyllis Kanki, Seema Meloni, and Eric J Tchetgen Tchetgen. On Partial
Identification of the Natural Indirect Effect. Journal of Causal Inference, jan 2017.
[15] Caleb H Miles, Ilya Shpitser, Phyllis Kanki, Seema Meloni, and Eric J Tchetgen Tchetgen. Quantifying an Adherence Path-Specific Effect of Antiretroviral Therapy in the
Nigeria PEPFAR Program. Journal of the American Statistical Association, 2017.
28

[16] Ashley I Naimi, Jay S Kaufman, and Richard F MacLehose. Mediation misgivings: ambiguous clinical and public health interpretations of natural direct and indirect effects.
International Journal of Epidemiology, 43(5):1656–1661, oct 2014.
[17] Judea Pearl. Causal Diagrams for Empirical Research. Biometrika, 82(4):669–710, dec
1995.
[18] Judea Pearl. Direct and Indirect Effects. In John Breese and Daphne Koller, editors, Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence
(UAI-01), UAI-01, pages 411–420, San Francisco, CA, 2001. Morgan Kaufmann.
[19] Judea Pearl. Interpretation and Identification of Causal Mediation. Psychological Methods, 19(4):459–481, jun 2014.
[20] Maya L Petersen, Sandra E Sinisi, and Mark J van der Laan. Estimation of Direct
Causal Effects. Epidemiology, 17(3):276–84, may 2006.
[21] Thomas S Richardson. Markov Properties for Acyclic Directed Mixed Graphs. Scandinavian Journal of Statistics, 30(1):145–157, mar 2003.
[22] Thomas S Richardson and James M Robins. Single World Intervention Graphs
(SWIGs): A Unification of the Counterfactual and Graphical Approaches to Causality.
Technical Report 128, University of Washington, 2013.
[23] James M Robins. A new approach to causal inference in mortality studies with a
sustained exposure period—application to control of the healthy worker survivor effect.
Mathematical Modelling, 7(9–12):1393–1512, 1986.
[24] James M Robins and Sander Greenland. Identifiability and Exchangeability for Direct
and Indirect Effects. Epidemiology, 3(2):143–155, mar 1992.
[25] James M Robins, Miguel A Hernán, and Babette Brumback. Marginal structural models
and causal inference in epidemiology. Epidemiology, 11(5):550–60, sep 2000.
[26] James M Robins and Thomas S Richardson. Alternative Graphical Causal Models and
the Identification of Direct Effects. In P Shrout, editor, Causality and Psychopathology: Finding the Determinants of Disorders and Their Cures, pages 103–158. Oxford
University Press, Oxford, England, 2010.
[27] Ilya Shpitser. Counterfactual graphical models for longitudinal mediation analysis with
unobserved confounding. Cognitive Science, 37(6):1011–35, aug 2013.
[28] Ilya Shpitser and Judea Pearl. Identification of Conditional Interventional Distributions.
Proceedings of the Twenty-Second Conference on Uncertainty in Artificial Intelligence,
pages 437–444, 2006.
[29] Ilya Shpitser and Judea Pearl. Identification of Joint Interventional Distributions in
Recursive semi-Markovian Causal Models. Proceedings of the Twenty-First AAAI Conference on Artificial Intelligence, pages 1219–1226, 2006.
[30] Ilya Shpitser and Judea Pearl. Complete identification methods for the causal hierarchy.
The Journal of Machine Learning Research, 9:1941–1979, 2008.
[31] Ilya Shpitser and Eric J Tchetgen Tchetgen. Causal inference with a graphical hierarchy
of interventions. The Annals of Statistics, 44(6):2433–2466, dec 2016.
[32] Ilya Shpitser and Tyler J VanderWeele. A Complete Graphical Criterion for the Adjustment Formula in Mediation Analysis. The International Journal of Biostatistics,
7(1):1–24, 2011.
29

[33] Ilya Shpitser, Tyler J VanderWeele, and James M Robins. On the validity of covariate
adjustment for estimating causal effects. Proceedings of the Twenty Sixth Conference
on Uncertainty in Artificial Intelligence (UAI-10), pages 527–536, 2010.
[34] Johan Steen, Tom Loeys, Beatrijs Moerkerke, and Stijn Vansteelandt. Flexible Mediation Analysis With Multiple Mediators. American Journal of Epidemiology, 186(2):184–
193, jul 2017.
[35] Johan Steen, Tom Loeys, Beatrijs Moerkerke, and Stijn Vansteelandt. medflex: An
R Package for Flexible Mediation Analysis using Natural Effect Models. Journal of
Statistical Software, 76(11), 2017.
[36] Eric J Tchetgen Tchetgen and Kelesitse Phiri. Evaluation of Medication-mediated Effects in Pharmacoepidemiology. Epidemiology, 28(3):439–445, may 2017.
[37] Eric J Tchetgen Tchetgen and Tyler J VanderWeele. Identification of Natural Direct
Effects When a Confounder of the Mediator Is Directly Affected by Exposure. Epidemiology, 25(2):282–91, mar 2014.
[38] Jin Tian and Judea Pearl. On the identification of causal effects. Technical report,
Department of Computer Science, University of California, Los Angeles, 2003.
[39] Santtu Tikka and Juha Karvanen. Identifying Causal Effects with the R Package causaleffect. Journal of Statistical Software, 76(12), 2017.
[40] Tyler J VanderWeele. A Three-Way Decomposition of a Total Effect into Direct, Indirect, and Interactive Effects. Epidemiology, 24(2):224–232, 2013.
[41] Tyler J VanderWeele and Yasutaka Chiba. Sensitivity analysis for direct and indirect effects in the presence of exposure-induced mediator-outcome confounders. Epidemiology,
Biostatistics and Public Health, 11(2):1–16, 2014.
[42] Tyler J VanderWeele and Eric J Tchetgen Tchetgen. Mediation analysis with time
varying exposures and mediators. Journal of the Royal Statistical Society: Series B
(Statistical Methodology), jun 2016.
[43] Tyler J VanderWeele, Stijn Vansteelandt, and James M Robins. Effect Decomposition
in the Presence of an Exposure-Induced Mediator-Outcome Confounder. Epidemiology,
25(2):300–306, 2014.
[44] Stijn Vansteelandt. Understanding Counterfactual-Based Mediation Analysis Approaches and Their Differences. Epidemiology, 23(6):889–91, nov 2012.
[45] Stijn Vansteelandt, Maarten Bekaert, and Theis Lange. Imputation Strategies for the
Estimation of Natural Direct and Indirect Effects. Epidemiologic Methods, 1(1):Article
7, jan 2012.
[46] Stijn Vansteelandt and Rhian M Daniel. Interventional effects for mediation analysis
with multiple mediators. Epidemiology, 28(2):258–265, nov 2017.
[47] Amiram D Vinokur, Richard H Price, and Yaacov Schul. Impact of the JOBS intervention on unemployed workers varying in risk for depression. American Journal of
Community Psychology, 23(1):39–74, 1995.
[48] Amiram D Vinokur and Yaacov Schul. Mastery and inoculation against setbacks as
active ingredients in the JOBS intervention for the unemployed. Journal of Consulting
and Clinical Psychology, 65(5):867–877, 1997.

30

