Polya-gamma augmentations for factor models
Proceedings of the Sixth Asian Conference on Machine Learning, PMLR 39:112-128, 2015.
Bayesian inference for latent factor models, such as principal component and canonical correlation analysis, is easy for Gaussian likelihoods. In particular, full conjugacy makes both Gibbs samplers and mean-field variational approximations straightforward. For other likelihood potentials one needs to either resort to more complex sampling schemes or to specifying dedicated forms of variational lower bounds. Recently, however, it was shown that for specific likelihoods related to the logistic function it is possible to augment the joint density with auxiliary variables following a Polya-Gamma distribution, leading to closed-form updates for binary and over-dispersed count models. In this paper we describe how Gibbs sampling and mean-field variational approximation for various latent factor models can be implemented for these cases, presenting easy-to-implement and efficient inference schemas.