Invariant Gaussian Process Latent Variable Models and Application in Causal Discovery

In nonlinear latent variable models or dy- namic models, if we consider the latent vari- ables as confounders (common causes), the noise dependencies imply further relations between the observed variables. Such models are then closely related to causal discovery in the presence of nonlinear confounders, which is a challenging problem. However, generally in such models the observation noise is as- sumed to be independent across data dimen- sions, and consequently the noise dependen- cies are ignored. In this paper we focus on the Gaussian process latent variable model (GPLVM), from which we develop an ex- tended model called invariant GPLVM (IG- PLVM), which can adapt to arbitrary noise covariances. With the Gaussian process prior put on a particular transformation of the la- tent nonlinear functions, instead of the origi- nal ones, the algorithm for IGPLVM involves almost the same computational loads as that for the original GPLVM. Besides its poten- tial application in causal discovery, IGPLVM has the advantage that its estimated latent nonlinear manifold is invariant to any nonsin- gular linear transformation of the data. Ex- perimental results on both synthetic and real- world data show its encouraging performance in nonlinear manifold learning and causal dis- covery.