Lightweight Implementations of Probabilistic Programming Languages Via Transformational Compilation

David Wingate, Andreas Stuhlmueller, Noah Goodman
Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, PMLR 15:770-778, 2011.

Abstract

We describe a general method of transforming arbitrary programming languages into probabilistic programming languages with straightforward MCMC inference engines. Random choices in the program are “named” with information about their position in an execution trace; these names are used in conjunction with a database of randomness to implement MCMC inference in the space of execution traces. We encode naming information using lightweight source-to-source compilers. Our method enables us to reuse existing infrastructure (compilers, interpreters, etc.) with minimal additional code, implying fast models with low development overhead. We illustrate the technique on two languages, one functional and one imperative: Bher, a compiled version of the Church language which eliminates interpretive overhead of the original MIT-Church implementation, and Stochastic Matlab, a new open-source language.

Cite this Paper

BibTeX
@InProceedings{pmlr-v15-wingate11a, title = {Lightweight Implementations of Probabilistic Programming Languages Via Transformational Compilation}, author = {Wingate, David and Stuhlmueller, Andreas and Goodman, Noah}, booktitle = {Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics}, pages = {770--778}, year = {2011}, editor = {Gordon, Geoffrey and Dunson, David and Dudík, Miroslav}, volume = {15}, series = {Proceedings of Machine Learning Research}, address = {Fort Lauderdale, FL, USA}, month = {11--13 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v15/wingate11a/wingate11a.pdf}, url = {https://proceedings.mlr.press/v15/wingate11a.html}, abstract = {We describe a general method of transforming arbitrary programming languages into probabilistic programming languages with straightforward MCMC inference engines. Random choices in the program are “named” with information about their position in an execution trace; these names are used in conjunction with a database of randomness to implement MCMC inference in the space of execution traces. We encode naming information using lightweight source-to-source compilers. Our method enables us to reuse existing infrastructure (compilers, interpreters, etc.) with minimal additional code, implying fast models with low development overhead. We illustrate the technique on two languages, one functional and one imperative: Bher, a compiled version of the Church language which eliminates interpretive overhead of the original MIT-Church implementation, and Stochastic Matlab, a new open-source language.} }
Endnote
%0 Conference Paper %T Lightweight Implementations of Probabilistic Programming Languages Via Transformational Compilation %A David Wingate %A Andreas Stuhlmueller %A Noah Goodman %B Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2011 %E Geoffrey Gordon %E David Dunson %E Miroslav Dudík %F pmlr-v15-wingate11a %I PMLR %P 770--778 %U https://proceedings.mlr.press/v15/wingate11a.html %V 15 %X We describe a general method of transforming arbitrary programming languages into probabilistic programming languages with straightforward MCMC inference engines. Random choices in the program are “named” with information about their position in an execution trace; these names are used in conjunction with a database of randomness to implement MCMC inference in the space of execution traces. We encode naming information using lightweight source-to-source compilers. Our method enables us to reuse existing infrastructure (compilers, interpreters, etc.) with minimal additional code, implying fast models with low development overhead. We illustrate the technique on two languages, one functional and one imperative: Bher, a compiled version of the Church language which eliminates interpretive overhead of the original MIT-Church implementation, and Stochastic Matlab, a new open-source language.
RIS
TY - CPAPER TI - Lightweight Implementations of Probabilistic Programming Languages Via Transformational Compilation AU - David Wingate AU - Andreas Stuhlmueller AU - Noah Goodman BT - Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics DA - 2011/06/14 ED - Geoffrey Gordon ED - David Dunson ED - Miroslav Dudík ID - pmlr-v15-wingate11a PB - PMLR DP - Proceedings of Machine Learning Research VL - 15 SP - 770 EP - 778 L1 - http://proceedings.mlr.press/v15/wingate11a/wingate11a.pdf UR - https://proceedings.mlr.press/v15/wingate11a.html AB - We describe a general method of transforming arbitrary programming languages into probabilistic programming languages with straightforward MCMC inference engines. Random choices in the program are “named” with information about their position in an execution trace; these names are used in conjunction with a database of randomness to implement MCMC inference in the space of execution traces. We encode naming information using lightweight source-to-source compilers. Our method enables us to reuse existing infrastructure (compilers, interpreters, etc.) with minimal additional code, implying fast models with low development overhead. We illustrate the technique on two languages, one functional and one imperative: Bher, a compiled version of the Church language which eliminates interpretive overhead of the original MIT-Church implementation, and Stochastic Matlab, a new open-source language. ER -
APA
Wingate, D., Stuhlmueller, A. & Goodman, N.. (2011). Lightweight Implementations of Probabilistic Programming Languages Via Transformational Compilation. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 15:770-778 Available from https://proceedings.mlr.press/v15/wingate11a.html.

Related Material