An Instability in Variational Inference for Topic Models
[edit]
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:22212231, 2019.
Abstract
Naive mean field variational methods are the state oftheart approach to inference in topic modeling. We show that these methods suffer from an instability that can produce misleading conclusions. Namely, for certain regimes of the model parameters, variational inference outputs a nontrivial decomposition into topics. However for the same parameter values the data contain no actual information about the true topic decomposition, and the output of the algorithm is uncorrelated with it. In particular, the estimated posterior mean is wrong, and estimated credible regions do not achieve the nominal coverage. We discuss how this instability is remedied by more accurate mean field approximations.
Related Material


