Importance Sampled Stochastic Optimization for Variational Inference

Variational inference approximates the poste- rior distribution of a probabilistic model with a parameterized density by maximizing a lower bound for the model evidence. Modern solu- tions fit a flexible approximation with stochastic gradient descent, using Monte Carlo approxima- tion for the gradients. This enables variational inference for arbitrary differentiable probabilis- tic models, and consequently makes variational inference feasible for probabilistic programming languages. In this work we develop more effi- cient inference algorithms for the task by consid- ering importance sampling estimates for the gra- dients. We show how the gradient with respect to the approximation parameters can often be eval- uated efficiently without needing to re-compute gradients of the model itself, and then proceed to derive practical algorithms that use impor- tance sampled estimates to speed up computa- tion. We present importance sampled stochas- tic gradient descent that outperforms standard stochastic gradient descent by a clear margin for a range of models, and provide a justifiable vari- ant of stochastic average gradients for variational inference.