What articles use structural equation modelling in a meta-analytic context to model mediation? I'm interested in structural equation modelling (SEM) meta-analysis and its application to modelling mediation relationships.

Mike W.-L. Cheung has written a few articles on the topic of SEM Meta-Analysis (e.g., Cheung, 2009). He's also written the metaSEM package in R.

One basic approach to SEM meta-analysis is a simple fixed-effect two-step approach. First, you extract a weighted average correlation matrix from the component studies. Second, you analyse that correlation matrix using standard SEM approaches. 

However, in reality, things are more complicated. 


Correlations often vary between studies by more than just sampling error, suggesting that a fixed-effect approach may be invalid. If a random-effects model is adopted, this raises issues of whether to categorise the correlation matrices into groups of fixed-effects matrices or whether to somehow incorporate the random-effects into the overall SEM.
Some studies may not have all the variables of interest, which introduces a  form of missing data, which in turn raises issues of how to define sample size and the standard error.


So in general, I'm interested in reading examples of SEM meta-analyses applied in psychology to get a sense of how others have dealt with the various issues. In particular, I'm interested in meta-analysis SEM that incorporates the modelling of indirect effects (i.e., mediation).

Questions


What studies have applied meta-analysis SEM in psychology to model indirect effects?
How have they dealt with issues such as (a) fixed versus random effects and (b) missing data?


References


Cheung, M.W.L. (2009). Meta-analysis: A Structural Equation Modeling Perspective. PDF
I'd like to address important issues that Jeromy Anglim raised in the "Personal thoughts" section of his answer, namely that correlation parameters (i.e., true, population, or infinite-sample correlations) often vary and covary among studies, and this between-studies\interstudy heterogeneity implies heterogeneity in studies' parameters for a structural equation model (SEM).  I'll describe a method I've proposed to account for this heterogeneity when estimating and making statistical inferences about a SEM, such as a univariate or multivariate regression model, factor model, path model, or model for structural relations among latent variables, or specific quantities based on such a model (e.g., $R^2$, indirect\mediation effect, fit index).  This approach's core idea is simple:


Use random-effects (RE) meta-analysis to estimate attributes of the between-studies distribution of correlation-matrix parameters (e.g., its mean and covariance matrix).
Transform those results to estimate attributes of the between-studies distribution of SEM parameters.


Many commonly used MASEM methods are based on fixed-effects (FE) models, and although meta-analyzing heterogeneous correlation matrices using certain FE methods performs reasonably well in some situations, it's probably not advisable for making unconditional inferences about a larger universe of studies (Hafdahl, 2008a).

Here I'll just give overviews of the problem and a three-step version of my proposed meta-analytic SEM (MASEM) method -- actually a collection of methods based on different choices for critical tasks at each step.  To keep the exposition relatively accessible I'll gloss over several important technical details.  Although this is an active area of my methodological research, the methods I'll sketch here are based largely on unpublished work (I'll cite references).  Because some aspects of this approach would be prohibitively difficult for many applied researchers to implement, one aim of my work in this area is to develop user-friendly software.  Until that's available, feel free to contact me for help with these techniques; such requests may motivate me to devote more time and other resources to this work.

Overview of Problem

In this section I'll describe my view of the problems for which MASEM is typically used.  Suppose we have from each of $k$ independent studies a sample Pearson-r or Fisher-z correlation matrix among $p$ variables of interest (e.g., $X_1, X_2, \ldots, X_p$), and suppose we're interested in a particular SEM for those variables or a related quantity that can be expressed as a function of the correlation matrix (e.g., 1 or more [standardized] path coefficients, indirect or total effect, squared multiple correlation, fit index).  It's useful to distinguish between each study's correlation-matrix parameter and that study's correlation-matrix estimate from a particular sample of subjects.  Meta-analysts are typically interested in using several studies' estimates to understand their corresponding parameters or distributions thereof.

In this overview I'll mostly ignore several interesting but vexing complications that arise in practice.  For instance, studies might use different versions of one or more variables (e.g., different measures similar to $X_1$), some studies might not contribute all $d = p( p - 1) / 2$ distinct correlation-matrix elements (e.g., due to missing variables or unreported elements), some of a study's correlations might be based on different subsets of its sample (e.g., if some subjects are missing some variables), a study might contribute two or more independent or dependent correlation matrices (e.g., from different groups or the same group in different conditions or at different times), studies' correlations might be influenced by so-called artifacts (e.g., [un]reliability, range restriction, dichotomization), and study-level covariates\moderators might partially account for between-studies heterogeneity.

Now, if we take the RE view that correlation-matrix parameters vary among studies -- that is, our studies are from a universe of studies whose correlation-matrix parameters follow some multivariate distribution -- then most functions of that correlation-matrix parameter will also vary among studies.  In particular, an SEM's path-coefficient parameters and other quantities (e.g., indirect effects) will have a distribution over studies.  To be clear, these distributions' between-studies (co)variation is not due to within-study sampling (co)variation caused by finite samples of subjects; instead, it may be viewed as due to studies' different constellations of features that produce different values for their correlation-matrix parameters and, consequently, most functions of those parameters.  (As a simpler example, consider 1 Pearson-r correlation parameter from each of several studies: If it varies among studies, then so will various functions of it -- its square, coefficient of alienation, Fisher z-transform, common language effect size, etc.)

By analogy with typical tasks in conventional RE meta-analyses, we might be interested in the following for our SEM: estimates of the between-studies mean and variance of each SEM path coefficient or other related quantity (e.g., indirect effect) and inferences about each such quantity (e.g., confidence interval [CI], prediction interval [PrI], hypothesis test).  When we're interested in two or more quantities from the SEM, we might wish to obtain bi- or multivariate generalizations of these estimates or inferences (e.g., confidence or prediction region\set); these might include quantities based on two or more distinct SEMs we'd like to compare, such as fit indices from two or more SEMs.  We might also be interested in the entire between-studies distribution of one or more SEM quantities (i.e., not only its mean and [co]variance [matrix]), which we could in some situations depict graphically (e.g., density plot).

Most commonly used MASEM approaches neglect between-studies heterogeneity in the SEM parameters, which seems especially difficult to justify in the presence of heterogeneous correlation(-matrix) parameters.  Certain aspects of the SEM parameters' between-studies (co)variance might be substantively important, such as if we're interested in how much particular path coefficients, indirect or total effects, squared multiple correlations, or fit indices (co)vary among studies.  For instance, if we use RMSEA as a fit index, we might wish to know how much RMSEA varies among studies, a plausible range of RMSEA values (e.g., prediction interval), or what proportion of studies have RMSEA values within some "acceptable" interval (e.g., below .05).

Moreover, there's reason to treat heterogeneity cautiously even if we're interested in only the mean of SEM parameters: Because SEM parameters are typically nonlinear functions of the correlation matrix, it's unclear what's estimated by applying the SEM directly to a mean correlation matrix, as in most MASEM approaches; that might estimate the mean SEM parameter poorly.  As a simpler example, suppose that for a heterogeneous effect-size parameter $Y$ we know its mean and variance over studies, $\mathrm{E}(Y)$ and $\mathrm{Var}(Y)$, but want to know the mean for $Y$'s square: Because for most distributions of $Y$ it's a fact that $\mathrm{E}(Y^2) = [\mathrm{E}(Y)]^2 + \mathrm{Var}(Y)$, simply squaring $\mathrm{E}(Y)$ gives a value lower than the desired $\mathrm{E}(Y^2)$, especially if $\mathrm{Var}(Y)$ is large.  This basic problem with applying a nonlinear transformation to a mean of heterogeneous effect sizes was addressed for the case of a univariate correlation -- focusing on the z-to-r transformation -- by Hafdahl (2009) and Hafdahl and Williams (2009); the analogous situation was addressed for correlation matrices by Hafdahl (2008b, 2009b), for generic univariate effect sizes by Hafdahl (2011), and for generic multivariate effect sizes by Hafdahl (2009c).

Overview of Proposed Method

In the first paragraph above I mentioned the gist of my proposed two-stage MASEM method.  To facilitate explanation, it's useful to divide the second stage -- transformation of results -- into separate steps for estimation and inference.  For instance, if we start with $k$ studies' estimates of a Fisher-z correlation matrix, the first step might entail estimating the correlation-matrix parameters' between-studies mean and covariance matrix, and the second and third steps might entail transforming those results to obtain estimates of and inferences about the between-studies mean and covariance matrix of the SEM's path coefficients.  Below I elaborate a bit on these three steps.

For convenience, let's use the following notation for correlation matrices from Study $i, i = 1, 2, \ldots, k$:


$\theta_i$: vector of the $d$ distinct parameters in a correlation matrix for $p$ variables, in either the Pearson-r or Fisher-z metric
$y_i$: vector of the $d$ distinct estimates in sample correlation matrix (i.e., $y_i$ is an estimate of $\theta_i$)


For example, if we're interested in the correlation matrix for $p = 5$ variables, then both $\theta_i$ and $y_i$ contain $d = 5(5 - 1) / 2 = 10$ correlations.  In RE meta-analysis we typically assume that $y_i$ has a within-study (sampling\conditional) distribution whose mean is approximately $\theta_i$, and that $\theta_i$ (or just $\theta$) has the same between-studies distribution for all studies, with mean $\mu_\theta = \mathrm{E}(\theta)$ and covariance matrix $\Sigma_\theta = \mathrm{Cov}(\theta)$.  (For simplicity I'll slightly abuse notation for random correlation parameters and estimates -- instead of using $\Theta_i$ and $Y_i$ -- and I'll ignore other quantities used in certain meta-analytic procedures, such as each study's sample size and conditional covariance matrix, whose inverse [i.e., precision matrix] is essentially used as a weight matrix.)

In terms of SEM notation, let's similarly denote the SEM parameters of interest in Study $i$ by $\gamma_i = g(\theta_i)$, where the function $g$ transforms a correlation-matrix parameter into SEM parameters.  This $g$ might be a fairly complicated function, such as for the parameters of a SEM with respect to a particular objective\loss criterion (e.g., ML, WLS, ADF) or a fit index for that SEM.  Also, $\gamma_i$ might be just one number (e.g., 1 path coefficient, indirect effect, fit index) or a vector (e.g., 2 or more path coefficients).  At any rate, our MASEM goal might be to estimate $\gamma$'s between-studies mean or (co)variance (matrix), $\mu_\gamma = \mathrm{E}(\gamma)$ and $\Sigma_\gamma = \mathrm{Cov}(\gamma)$, and make inferences about either distributional attribute; we might also wish to estimate $\gamma$'s entire distribution.

Below are the three steps of my proposed method; Steps 2 and 3 presume we have in mind a specific SEM or related quantity that can be expressed as $\gamma_i = g(\theta_i)$.  This is similar to Hafdahl's (2009a, 2011) univariate methods and Hafdahl's (2008b, 2009b, 2009c) multivariate methods but specific to MASEM.

1. Meta-Analysis for $\theta$: Apply multivariate RE meta-analysis to $y_i$ to obtain estimates of at least $\mu_\theta$ and $\Sigma_\theta$, which I'll denote $\hat\mu_\theta$ and $\hat\Sigma_\theta$, and perhaps $\theta$'s full between-studies distribution; perhaps also obtain a covariance matrix for only $\hat\mu_\theta$ or both $\hat\mu_\theta$ and $\hat\Sigma_\theta$, depending on how inference is handled in Step 3.  Among several proposed methods for estimating $\mu_\theta$ and $\Sigma_\theta$, only a few handle incomplete correlation matrices -- nearly unavoidable in MASEM data sets -- in a principled way (e.g., Hafdahl &amp; Wu, 2011; Kalaian, &amp; Raudenbush, 1996; White, 2011).  In particular, Hafdahl and Wu's extension of Becker and Schram's (1994) EM algorithm permits any one or more of $y_i$'s correlations to be missing, doesn't require imputing values for missing correlations, and yields a posterior distribution for each study's entire $\theta_i$ (given its possibly incomplete $y_i$); it also yields an estimate of $\theta$'s full between-studies distribution as a mixture of the studies' posterior distributions.  Depending on the estimation method, a covariance matrix for $\hat\mu_\theta$ may be obtained by generalized least-squares (GLS) or other methods (e.g., based on Hessian matrix for maximum-likelihood estimators), some of which also provide a covariance matrix for $\hat\Sigma_\theta$.  Hafdahl (2004) demonstrated substantial differences in performance among different techniques for multivariate RE meta-analysis applied to correlation matrices.

2. Estimation for $\gamma$: Use an appropriate transformation method -- based on the function $g$ -- to obtain estimates of at least $\mu_\gamma$ and $\Sigma_\gamma$, which I'll denote $\hat\mu_\gamma$ and $\hat\Sigma_\gamma$, and perhaps $\gamma$'s full between-studies distribution.  One strategy is to use a first- or second-order Taylor series approximation, which essentially entails approximating $\gamma = g(\theta)$ by a simpler linear or quadratic function of $\theta$; estimates of this approximating function's mean and covariance can then be computed from Step 1's $\hat\mu_\theta$ and $\hat\Sigma_\theta$.  Another strategy entails simulation: Sample values of $\theta$ from its distribution estimated in Step 1, transform these to values of $\gamma$, and estimate $\mu_\gamma$ and $\Sigma_\gamma$ from this simulated distribution; we might treat $\theta$'s distribution as multivariate normal -- such as $\theta \sim \mathcal{N}_d(\hat\mu_\theta, \hat\Sigma_\theta)$ -- or permit it to take some other form estimated from the data (e.g., mixture of posteriors from EM algorithm).  Either strategy might also be used to estimate other attributes of $\gamma$'s distribution, such as tail or central areas (e.g., probability that 1 or more path coefficients or other quantities are near 0, positive, large, etc.) or $\gamma$ values bounding regions of interest (e.g., quartiles, middle 95%).  (We could in principle transform $\hat\mu_\theta$ and $\hat\Sigma_\theta$ to $\hat\mu_\gamma$ and $\hat\Sigma_\gamma$ via integration, using the definitions of $\mu_\gamma$ and $\Sigma_\gamma$, but that'll often be intractable analytically and infeasible computationally.)

3. Inference for $\gamma$: Make inferences about $\mu_\gamma$ and $\Sigma_\gamma$, such as CIs, PrIs, or hypothesis tests for single-valued parameters or their multivariate generalizations for vector-valued parameters (e.g., confidence or prediction regions).  One strategy is to use the (multivariate) delta method, which essentially entails using derivatives to transform the covariance matrix for $\hat\mu_\theta$ alone or both $\hat\mu_\theta$ and $\hat\Sigma_\theta$ to a covariance matrix for $\hat\mu_\gamma$ or $\hat\Sigma_\gamma$; the latter covariance matrix can be used to construct CIs or PrIs or test hypotheses.  Another strategy, at least for CIs or confidence regions, is to use a bootstrap technique to essentially construct an empirical sampling distribution of $\hat\mu_\gamma$ or $\hat\Sigma_\gamma$; numerous bootstrap options are available, depending largely on how the sample of bootstrap replicates -- $\hat\mu_\gamma$ or $\hat\Sigma_\gamma$ for each resample from the data -- is constructed (e.g., parametric vs. nonparametric) and how it's used to construct confidence intervals or regions (e.g., standard deviation\error vs. percentile, bias correction or not).

Because this is already a fairly long overview, I'll close with a few remarks.  First, despite its advantages over some other MASEM methods, my proposed method also has drawbacks and limitations; I won't elaborate here on these pros and cons, except to caution that my proposed method might perform unacceptably in certain circumstances.  Second, my proposed method would benefit from considerably more work, such as refining aspects of each step and studying its performance in realistic situations defined by characteristics of MASEM studies (e.g., number of primary studies, distribution of sample sizes, distribution of correlation-matrix parameters, pattern and mechanism of missing data, choice of function g).  To date there's been little evaluation of multivariate RE meta-analysis analytically or by simulation, for either correlation matrices (cf. Hafdahl, 2004, 2008b) or other multivariate effect sizes (cf. Riley, 2009; Riley, Abrams, Sutton, Lambert, &amp; Thompson, 2007), and Hafdahl's (2009c) Monte Carlo studies of meta-analysis for functions of multivariate effect sizes did not include correlation matrices.  Third, Bayesian approaches to meta-analysis, such as Prevost, Mason, Griffin, Kinmonth, Sutton, and Spiegelhalter's (2007) proposed method for correlation matrices, might be especially well-suited to MASEM due to their natural -- though computationally challenging -- strategies for constructing posterior distributions for functions of a study's parameters.

References

Becker, B. J., &amp; Schram, C. M. (1994). Examining explanatory models through research synthesis. In H. Cooper &amp; L. V. Hedges (Eds.), The handbook of research synthesis (pp. 357-381). New York: Russell Sage Foundation.

Hafdahl, A. R. (2004, June). Refinements for random-effects meta-analysis of correlation matrices. Paper presented at the meeting of the Psychometric Society, Monterey, CA.

Hafdahl, A. R. (2008a). Combining heterogeneous correlation matrices: Simulation analysis of fixed-effects methods. Journal of Educational and Behavioral Statistics, 33, 507-533. doi:10.3102/1076998607309472

Hafdahl, A. R. (2008b, July). Meta-analysis for functions of heterogeneous correlation matrices. Paper presented at the meeting of the Psychometric Society, Durham, NH.

Hafdahl, A. R. (2009a). Improved Fisher z estimators for univariate random-effects meta-analysis of correlations. British Journal of Mathematical and Statistical Psychology, 62, 233-261. doi:10.1348/000711008X281633

Hafdahl, A. R. (2009b, May). Meta-analysis for functions of dependent correlations. In A. R. Hafdahl (Chair), Advances in meta-analysis for multivariable linear models. Invited symposium presented at the meeting of the Association for Psychological Science, San Francisco, CA.

Hafdahl, A. R. (2009c). Meta-analysis for functions of heterogeneous multivariate effect sizes. Unpublished master's thesis, Washington University in St. Louis, St. Louis, Missouri. http://openscholarship.wustl.edu/etd/439/

Hafdahl, A. R. (2011). Translating meta-analytic results: Techniques to express random-effects estimates in other metrics. Manuscript in preparation, Washington University in St. Louis.

Hafdahl, A. R., &amp; Williams, M. A. (2009). Meta-analysis of correlations revisited: Attempted replication and extension of Field’s (2001) simulation studies. Psychological Methods, 14, 24-42. doi:10.1037/a0014697

Hafdahl, A. R., &amp; Wu, W. (2012, February). An EM algorithm for multivariate random-effects meta-analysis with incomplete effect estimates. Manuscript in preparation, ARCH Statistical Consulting, LLC.

Kalaian, H. A., &amp; Raudenbush, S. W. (1996). A multivariate mixed linear model for meta-analysis. Psychological Methods, 1, 227-235. doi:10.1037/1082-989X.1.3.227

Prevost, A. T., Mason, D., Griffin, S., Kinmonth, A.-L., Sutton, S., &amp; Spiegelhalter, D. (2007). Allowing for correlations between correlations in random-effects meta-analysis of correlation matrices. Psychological Methods, 12, 434-450. doi:10.1037/1082-989X.12.4.434

Riley, R. D. (2009). Multivariate meta-analysis: The effect of ignoring within-study correlation. Journal of the Royal Statistical Society—Series A, 172, 789–811. doi:10.1111/j.1467-985X.2008.00593.x

Riley, R. D., Abrams, K. R., Sutton, A. J., Lambert, P. C., &amp; Thompson, J. R. (2007). Bivariate random-effects meta-analysis and the estimation of between-study correlation. BMC Medical Research Methodology, 7, 3. doi:10.1186/1471-2288-7-3

White, I. R. (2011). Multivariate random-effects meta-regression: Updates to mvmeta. Stata Journal, 11, 255-270.