In a variety of disciplines such as social sciences, psychology, medicine and economics, the recorded data are considered to be noisy measurements of latent variables connected by some causal structure. This corresponds to a family of graphical models known as the structural equation model with latent variables. While linear non-Gaussian variants have been well-studied, inference in nonparametric structural equation models is still underdeveloped. We introduce a sparse Gaussian process parameterization that defines a non-linear structure connecting latent variables, unlike common formulations of Gaussian process latent variable models. The sparse parameterization is given a full Bayesian treatment without compromising Markov chain Monte Carlo efficiency. We compare the stability of the sampling procedure and the predictive ability of the model against the current practice.
@cite_8: Factor analysis and its extensions are widely used in the social and behavioral sciences, and can be considered useful tools for exploration and model fitting in multivariate analysis. Despite its popu- larity in applications, factor analysis has attracted rather limited atten- tion from statisticians. Three issues, identification ambiguity, heavy reliance on normality, and limitation to linearity, may have contributed to statisticians' lack of interest in factor analysis. In this paper, the sta- tistical contributions to the first two issues are reviewed, and the third issue is addressed in detail. Linear models can be unrealistic even as an approximation in many applications, and often do not fit the data well without increasing the number of factors beyond the level explainable by the subject-matter theory. As an exploratory model, the conventional factor analysis model fails to address nonlinear structure underlying multivariate data. It is argued here that factor analysis does not need to be restricted to linearity and that nonlinear factor analysis can be formulated and carried out as a useful statistical method. In particular, for a general parametric nonlinear factor analysis model, the errors- in-variables parameterization is suggested as a sensible way to formu- late the model, and two procedures for model fitting are introduced and described. Tests for the goodness-of-fit of the model are also proposed. The procedures are studied through a simulation study. An example from personality testing is used to illustrate the issues and the methods.
@cite_1: 1. Introduction to probabilities, graphs, and causal models 2. A theory of inferred causation 3. Causal diagrams and the identification of causal effects 4. Actions, plans, and direct effects 5. Causality and structural models in the social sciences 6. Simpson's paradox, confounding, and collapsibility 7. Structural and counterfactual models 8. Imperfect experiments: bounds and counterfactuals 9. Probability of causation: interpretation and identification Epilogue: the art and science of cause and effect.
@cite_15: 
@cite_12: Abstract Structural equation models (SEMs) with latent variables are routinely used in social science research, and are of increasing importance in biomedical applications. Standard practice in implementing SEMs relies on frequentist methods. A simple and concise description of an alternative Bayesian approach is developed. Furthermore, a brief overview of the literature, a description of Bayesian specification of SEMs, and an outline of a Gibbs sampling strategy for model fitting is provided. Bayesian inferences are illustrated through an industrialization and democratization case study from the literature. The Bayesian approach has some distinct advantages, due to the availability of samples from the joint posterior distribution of the model parameters and latent variables, that are highlighted. These posterior samples provide important information not contained in the measurement and structural parameters. As is illustrated using the case study, this information can often provide valuable insight into structural relationships.
@cite_17: Summarising a high dimensional data set with a low dimensional embedding is a standard approach for exploring its structure. In this paper we provide an overview of some existing techniques for discovering such embeddings. We then introduce a novel probabilistic interpretation of principal component analysis (PCA) that we term dual probabilistic PCA (DPPCA). The DPPCA model has the additional advantage that the linear mappings from the embedded space can easily be non-linearised through Gaussian processes. We refer to this model as a Gaussian process latent variable model (GP-LVM). Through analysis of the GP-LVM objective function, we relate the model to popular spectral techniques such as kernel PCA and multidimensional scaling. We then review a practical algorithm for GP-LVMs in the context of large data sets and develop it to also handle discrete valued data and missing attributes. We demonstrate the model on a range of real-world and artificially generated data sets.
Non-linear factor analysis has been studied for decades in the psychometrics literature Another instance of the whatever you do, somebody in psychometrics already did it long before'' law: http: www.stat.columbia.edu @math cook movabletype archives 2009 01 a .html . A review is provided by @cite_8 . However, most of the classic work is based on simple parametric models. A modern approach based on Gaussian processes is the Gaussian process latent variable model of @cite_17 . By construction, factor analysis cannot be used in applications where one is interested in learning functions relating latent variables, such as in causal inference. For embedding, factor analysis is easier to use and more robust to model misspecification than SEM analysis. Conversely, it does not benefit from well-specified structures and might be harder to interpret. @cite_15 discusses the interplay between factor analysis and SEM. Practical non-linear structural equation models are discussed by @cite_12 , but none of such approaches rely on nonparametric methods. Gaussian processes latent structures appear mostly in the context of dynamical systems (e.g., @cite_1 ). However, the connection is typically among data points only, not among variables within a data point, where on-line filtering is the target application.