Expectation programming: Adapting probabilistic programming systems to estimate expectations efficiently

Tim Reichelt, Adam Goliński, Luke Ong, Tom Rainforth
Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, PMLR 180:1676-1685, 2022.

Abstract

We show that the standard computational pipeline of probabilistic programming systems (PPSs) can be inefficient for estimating expectations and introduce the concept of expectation programming to address this. In expectation programming, the aim of the backend inference engine is to directly estimate expected return values of programs, as opposed to approximating their conditional distributions. This distinction, while subtle, allows us to achieve substantial performance improvements over the standard PPS computational pipeline by tailoring computation to the expectation we care about. We realize a particular instance of our expectation programming concept, Expectation Programming in Turing (EPT), by extending the PPS Turing to allow so-called target-aware inference to be run automatically. We then verify the statistical soundness of EPT theoretically, and show that it provides substantial empirical gains in practice.

Cite this Paper

BibTeX
@InProceedings{pmlr-v180-reichelt22a, title = {Expectation programming: Adapting probabilistic programming systems to estimate expectations efficiently}, author = {Reichelt, Tim and Goli{\'n}ski, Adam and Ong, Luke and Rainforth, Tom}, booktitle = {Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence}, pages = {1676--1685}, year = {2022}, editor = {Cussens, James and Zhang, Kun}, volume = {180}, series = {Proceedings of Machine Learning Research}, month = {01--05 Aug}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v180/reichelt22a/reichelt22a.pdf}, url = {https://proceedings.mlr.press/v180/reichelt22a.html}, abstract = {We show that the standard computational pipeline of probabilistic programming systems (PPSs) can be inefficient for estimating expectations and introduce the concept of expectation programming to address this. In expectation programming, the aim of the backend inference engine is to directly estimate expected return values of programs, as opposed to approximating their conditional distributions. This distinction, while subtle, allows us to achieve substantial performance improvements over the standard PPS computational pipeline by tailoring computation to the expectation we care about. We realize a particular instance of our expectation programming concept, Expectation Programming in Turing (EPT), by extending the PPS Turing to allow so-called target-aware inference to be run automatically. We then verify the statistical soundness of EPT theoretically, and show that it provides substantial empirical gains in practice.} }
Endnote
%0 Conference Paper %T Expectation programming: Adapting probabilistic programming systems to estimate expectations efficiently %A Tim Reichelt %A Adam Goliński %A Luke Ong %A Tom Rainforth %B Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2022 %E James Cussens %E Kun Zhang %F pmlr-v180-reichelt22a %I PMLR %P 1676--1685 %U https://proceedings.mlr.press/v180/reichelt22a.html %V 180 %X We show that the standard computational pipeline of probabilistic programming systems (PPSs) can be inefficient for estimating expectations and introduce the concept of expectation programming to address this. In expectation programming, the aim of the backend inference engine is to directly estimate expected return values of programs, as opposed to approximating their conditional distributions. This distinction, while subtle, allows us to achieve substantial performance improvements over the standard PPS computational pipeline by tailoring computation to the expectation we care about. We realize a particular instance of our expectation programming concept, Expectation Programming in Turing (EPT), by extending the PPS Turing to allow so-called target-aware inference to be run automatically. We then verify the statistical soundness of EPT theoretically, and show that it provides substantial empirical gains in practice.
APA
Reichelt, T., Goliński, A., Ong, L. & Rainforth, T.. (2022). Expectation programming: Adapting probabilistic programming systems to estimate expectations efficiently. Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 180:1676-1685 Available from https://proceedings.mlr.press/v180/reichelt22a.html.

Related Material