Fast Variational Bayesian Inference for Non-Conjugate Matrix Factorization Models


Matthias Seeger, Guillaume Bouchard ;
Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:1012-1018, 2012.


Probabilistic matrix factorization methods aim to extract meaningful correlation structure from an incomplete data matrix by postulating low rank constraints. Recently, variational Bayesian (VB) inference techniques have successfully been applied to such large scale bilinear models. However, current algorithms are of the alternate updating or stochastic gradient descent type, slow to converge and prone to getting stuck in shallow local minima. While for MAP or maximum margin estimation, singular value shrinkage algorithms have been proposed which can far outperform alternate updating, this methodological avenue remains unexplored for Bayesian techniques. In this paper, we show how to combine a recent singular value shrinkage characterization of fully observed spherical Gaussian VB matrix factorization with augmented Lagrangian techniques in order to obtain efficient VB inference for general MF models with arbitrary likelihood potentials. In particular, we show how to handle Poisson and Bernoulli potentials, far more suited for most MF applications than Gaussian likelihoods. Our algorithm can be run even for very large models and is easily implemented in \em Matlab. It outperforms MAP estimation on a range of real-world datasets.

Related Material