Amortized Probabilistic Conditioning for Optimization, Simulation and Inference

Paul Edmund Chang, Nasrulloh Ratu Bagus Satrio Loka, Daolang Huang, Ulpu Remes, Samuel Kaski, Luigi Acerbi
Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, PMLR 258:703-711, 2025.

Abstract

Amortized meta-learning methods based on pre-training have propelled fields like natural language processing and vision. Transformer-based neural processes and their variants are leading models for probabilistic meta-learning with a tractable objective. Often trained on synthetic data, these models implicitly capture essential latent information in the data-generation process. However, existing methods do not allow users to flexibly inject (condition on) and extract (predict) this probabilistic latent information at runtime, which is key to many tasks. We introduce the Amortized Conditioning Engine (ACE), a new transformer-based meta-learning model that explicitly represents latent variables of interest. ACE affords conditioning on both observed data and interpretable latent variables, the inclusion of priors at runtime, and outputs predictive distributions for discrete and continuous data and latents. We show ACE’s practical utility across diverse tasks such as image completion and classification, Bayesian optimization, and simulation-based inference, demonstrating how a general conditioning framework can replace task-specific solutions.

Cite this Paper

BibTeX
@InProceedings{pmlr-v258-chang25a, title = {Amortized Probabilistic Conditioning for Optimization, Simulation and Inference}, author = {Chang, Paul Edmund and Loka, Nasrulloh Ratu Bagus Satrio and Huang, Daolang and Remes, Ulpu and Kaski, Samuel and Acerbi, Luigi}, booktitle = {Proceedings of The 28th International Conference on Artificial Intelligence and Statistics}, pages = {703--711}, year = {2025}, editor = {Li, Yingzhen and Mandt, Stephan and Agrawal, Shipra and Khan, Emtiyaz}, volume = {258}, series = {Proceedings of Machine Learning Research}, month = {03--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v258/main/assets/chang25a/chang25a.pdf}, url = {https://proceedings.mlr.press/v258/chang25a.html}, abstract = {Amortized meta-learning methods based on pre-training have propelled fields like natural language processing and vision. Transformer-based neural processes and their variants are leading models for probabilistic meta-learning with a tractable objective. Often trained on synthetic data, these models implicitly capture essential latent information in the data-generation process. However, existing methods do not allow users to flexibly inject (condition on) and extract (predict) this probabilistic latent information at runtime, which is key to many tasks. We introduce the Amortized Conditioning Engine (ACE), a new transformer-based meta-learning model that explicitly represents latent variables of interest. ACE affords conditioning on both observed data and interpretable latent variables, the inclusion of priors at runtime, and outputs predictive distributions for discrete and continuous data and latents. We show ACE’s practical utility across diverse tasks such as image completion and classification, Bayesian optimization, and simulation-based inference, demonstrating how a general conditioning framework can replace task-specific solutions.} }
Endnote
%0 Conference Paper %T Amortized Probabilistic Conditioning for Optimization, Simulation and Inference %A Paul Edmund Chang %A Nasrulloh Ratu Bagus Satrio Loka %A Daolang Huang %A Ulpu Remes %A Samuel Kaski %A Luigi Acerbi %B Proceedings of The 28th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2025 %E Yingzhen Li %E Stephan Mandt %E Shipra Agrawal %E Emtiyaz Khan %F pmlr-v258-chang25a %I PMLR %P 703--711 %U https://proceedings.mlr.press/v258/chang25a.html %V 258 %X Amortized meta-learning methods based on pre-training have propelled fields like natural language processing and vision. Transformer-based neural processes and their variants are leading models for probabilistic meta-learning with a tractable objective. Often trained on synthetic data, these models implicitly capture essential latent information in the data-generation process. However, existing methods do not allow users to flexibly inject (condition on) and extract (predict) this probabilistic latent information at runtime, which is key to many tasks. We introduce the Amortized Conditioning Engine (ACE), a new transformer-based meta-learning model that explicitly represents latent variables of interest. ACE affords conditioning on both observed data and interpretable latent variables, the inclusion of priors at runtime, and outputs predictive distributions for discrete and continuous data and latents. We show ACE’s practical utility across diverse tasks such as image completion and classification, Bayesian optimization, and simulation-based inference, demonstrating how a general conditioning framework can replace task-specific solutions.
APA
Chang, P.E., Loka, N.R.B.S., Huang, D., Remes, U., Kaski, S. & Acerbi, L.. (2025). Amortized Probabilistic Conditioning for Optimization, Simulation and Inference. Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 258:703-711 Available from https://proceedings.mlr.press/v258/chang25a.html.

Related Material