Yes, but Did It Work?: Evaluating Variational Inference

Yuling Yao, Aki Vehtari, Daniel Simpson, Andrew Gelman
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:5581-5590, 2018.

Abstract

While it’s always possible to compute a variational approximation to a posterior distribution, it can be difficult to discover problems with this approximation. We propose two diagnostic algorithms to alleviate this problem. The Pareto-smoothed importance sampling (PSIS) diagnostic gives a goodness of fit measurement for joint distributions, while simultaneously improving the error in the estimate. The variational simulation-based calibration (VSBC) assesses the average performance of point estimates.

Cite this Paper

BibTeX
@InProceedings{pmlr-v80-yao18a, title = {Yes, but Did It Work?: Evaluating Variational Inference}, author = {Yao, Yuling and Vehtari, Aki and Simpson, Daniel and Gelman, Andrew}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {5581--5590}, year = {2018}, editor = {Dy, Jennifer and Krause, Andreas}, volume = {80}, series = {Proceedings of Machine Learning Research}, month = {10--15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v80/yao18a/yao18a.pdf}, url = {https://proceedings.mlr.press/v80/yao18a.html}, abstract = {While it’s always possible to compute a variational approximation to a posterior distribution, it can be difficult to discover problems with this approximation. We propose two diagnostic algorithms to alleviate this problem. The Pareto-smoothed importance sampling (PSIS) diagnostic gives a goodness of fit measurement for joint distributions, while simultaneously improving the error in the estimate. The variational simulation-based calibration (VSBC) assesses the average performance of point estimates.} }
Endnote
%0 Conference Paper %T Yes, but Did It Work?: Evaluating Variational Inference %A Yuling Yao %A Aki Vehtari %A Daniel Simpson %A Andrew Gelman %B Proceedings of the 35th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jennifer Dy %E Andreas Krause %F pmlr-v80-yao18a %I PMLR %P 5581--5590 %U https://proceedings.mlr.press/v80/yao18a.html %V 80 %X While it’s always possible to compute a variational approximation to a posterior distribution, it can be difficult to discover problems with this approximation. We propose two diagnostic algorithms to alleviate this problem. The Pareto-smoothed importance sampling (PSIS) diagnostic gives a goodness of fit measurement for joint distributions, while simultaneously improving the error in the estimate. The variational simulation-based calibration (VSBC) assesses the average performance of point estimates.
APA
Yao, Y., Vehtari, A., Simpson, D. & Gelman, A.. (2018). Yes, but Did It Work?: Evaluating Variational Inference. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research 80:5581-5590 Available from https://proceedings.mlr.press/v80/yao18a.html.

Related Material