Optimizing Likelihoods via Mutual Information: Bridging Simulation-Based Inference and Bayesian Optimal Experimental Design

Vincent D. Zaballa, Elliot E Hui
Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, PMLR 337:7928-7951, 2026.

Abstract

Simulation-based inference (SBI) depends on expensive simulators, so inference and experimental design must operate under fixed simulation budgets. {Bayesian} optimal experimental design (BOED) maximizes expected information gain (EIG), but is often implemented with a separate mutual-information critic, used only as a diagnostic, or restricted to differentiable simulators. We show that the InfoNCE lower bound on EIG becomes a principled SBI training objective when the critic is a normalized conditional density model. Maximizing the bound is equivalent to fitting a surrogate likelihood by minimizing a KL divergence plus a marginal-likelihood term, so each simulator call can improve both inference and design. We propose SBI-BOED, a single stochastic-gradient procedure that jointly trains a conditional normalizing-flow likelihood and optimizes designs without simulator differentiability. The same MI view also yields simulation active learning under fixed designs via epistemic predictive information gain (EPIG). With an InfoNCE-$\lambda$ objective and practical stabilizers, SBI-BOED improves posterior calibration and predictive accuracy over strong BOED baselines on synthetic benchmarks and scientific simulators at matched simulation budgets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v337-zaballa26a, title = {Optimizing Likelihoods via Mutual Information: Bridging Simulation-Based Inference and {Bayesian} Optimal Experimental Design}, author = {Zaballa, Vincent D. and Hui, Elliot E}, booktitle = {Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence}, pages = {7928--7951}, year = {2026}, editor = {Perković, Emilija and Malinsky, Daniel}, volume = {337}, series = {Proceedings of Machine Learning Research}, month = {17--21 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v337/main/assets/zaballa26a/zaballa26a.pdf}, url = {https://proceedings.mlr.press/v337/zaballa26a.html}, abstract = {Simulation-based inference (SBI) depends on expensive simulators, so inference and experimental design must operate under fixed simulation budgets. {Bayesian} optimal experimental design (BOED) maximizes expected information gain (EIG), but is often implemented with a separate mutual-information critic, used only as a diagnostic, or restricted to differentiable simulators. We show that the InfoNCE lower bound on EIG becomes a principled SBI training objective when the critic is a normalized conditional density model. Maximizing the bound is equivalent to fitting a surrogate likelihood by minimizing a KL divergence plus a marginal-likelihood term, so each simulator call can improve both inference and design. We propose SBI-BOED, a single stochastic-gradient procedure that jointly trains a conditional normalizing-flow likelihood and optimizes designs without simulator differentiability. The same MI view also yields simulation active learning under fixed designs via epistemic predictive information gain (EPIG). With an InfoNCE-$\lambda$ objective and practical stabilizers, SBI-BOED improves posterior calibration and predictive accuracy over strong BOED baselines on synthetic benchmarks and scientific simulators at matched simulation budgets.} }
Endnote
%0 Conference Paper %T Optimizing Likelihoods via Mutual Information: Bridging Simulation-Based Inference and Bayesian Optimal Experimental Design %A Vincent D. Zaballa %A Elliot E Hui %B Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2026 %E Emilija Perković %E Daniel Malinsky %F pmlr-v337-zaballa26a %I PMLR %P 7928--7951 %U https://proceedings.mlr.press/v337/zaballa26a.html %V 337 %X Simulation-based inference (SBI) depends on expensive simulators, so inference and experimental design must operate under fixed simulation budgets. {Bayesian} optimal experimental design (BOED) maximizes expected information gain (EIG), but is often implemented with a separate mutual-information critic, used only as a diagnostic, or restricted to differentiable simulators. We show that the InfoNCE lower bound on EIG becomes a principled SBI training objective when the critic is a normalized conditional density model. Maximizing the bound is equivalent to fitting a surrogate likelihood by minimizing a KL divergence plus a marginal-likelihood term, so each simulator call can improve both inference and design. We propose SBI-BOED, a single stochastic-gradient procedure that jointly trains a conditional normalizing-flow likelihood and optimizes designs without simulator differentiability. The same MI view also yields simulation active learning under fixed designs via epistemic predictive information gain (EPIG). With an InfoNCE-$\lambda$ objective and practical stabilizers, SBI-BOED improves posterior calibration and predictive accuracy over strong BOED baselines on synthetic benchmarks and scientific simulators at matched simulation budgets.
APA
Zaballa, V.D. & Hui, E.E.. (2026). Optimizing Likelihoods via Mutual Information: Bridging Simulation-Based Inference and Bayesian Optimal Experimental Design. Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 337:7928-7951 Available from https://proceedings.mlr.press/v337/zaballa26a.html.

Related Material