Implicit Variational Rejection Sampling

Jian Xu, Shigui Li, Wei Chen, Jiacheng Li, Zhiqi Lin, Delu Zeng, Xinghao Ding, John Paisley, Qibin Zhao
Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, PMLR 337:7651-7669, 2026.

Abstract

Variational Inference (VI) is a fundamental inference technique in {Bayesian} machine learning for approximating complex posterior distributions. Traditional VI often relies on the mean-field factorization, which can inadequately capture true posterior complexity. Recent advancements have leveraged neural networks to model implicit distributions, offering increased flexibility. However, the practical constraints of neural network architectures still produces inaccuracies. In this paper, we propose a method called Implicit Variational Rejection Sampling (IVRS), which integrates implicit distributions with rejection sampling to improve the posterior approximation. Our method uses neural networks to construct implicit proposal distributions, and rejection sampling with a discriminator network that estimates the density ratio between the implicit proposal and the true posterior for refining the approximation. Towards this end, we introduce the Implicit Resampling Evidence Lower Bound (IR-{ELBO}) as a metric to characterize the resampled distribution’s quality and derive a tighter variational lower bound. Experimental results demonstrate that our method outperforms traditional variational inference techniques.

Cite this Paper

BibTeX
@InProceedings{pmlr-v337-xu26e, title = {Implicit Variational Rejection Sampling}, author = {Xu, Jian and Li, Shigui and Chen, Wei and Li, Jiacheng and Lin, Zhiqi and Zeng, Delu and Ding, Xinghao and Paisley, John and Zhao, Qibin}, booktitle = {Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence}, pages = {7651--7669}, year = {2026}, editor = {Perković, Emilija and Malinsky, Daniel}, volume = {337}, series = {Proceedings of Machine Learning Research}, month = {17--21 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v337/main/assets/xu26e/xu26e.pdf}, url = {https://proceedings.mlr.press/v337/xu26e.html}, abstract = {Variational Inference (VI) is a fundamental inference technique in {Bayesian} machine learning for approximating complex posterior distributions. Traditional VI often relies on the mean-field factorization, which can inadequately capture true posterior complexity. Recent advancements have leveraged neural networks to model implicit distributions, offering increased flexibility. However, the practical constraints of neural network architectures still produces inaccuracies. In this paper, we propose a method called Implicit Variational Rejection Sampling (IVRS), which integrates implicit distributions with rejection sampling to improve the posterior approximation. Our method uses neural networks to construct implicit proposal distributions, and rejection sampling with a discriminator network that estimates the density ratio between the implicit proposal and the true posterior for refining the approximation. Towards this end, we introduce the Implicit Resampling Evidence Lower Bound (IR-{ELBO}) as a metric to characterize the resampled distribution’s quality and derive a tighter variational lower bound. Experimental results demonstrate that our method outperforms traditional variational inference techniques.} }
Endnote
%0 Conference Paper %T Implicit Variational Rejection Sampling %A Jian Xu %A Shigui Li %A Wei Chen %A Jiacheng Li %A Zhiqi Lin %A Delu Zeng %A Xinghao Ding %A John Paisley %A Qibin Zhao %B Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2026 %E Emilija Perković %E Daniel Malinsky %F pmlr-v337-xu26e %I PMLR %P 7651--7669 %U https://proceedings.mlr.press/v337/xu26e.html %V 337 %X Variational Inference (VI) is a fundamental inference technique in {Bayesian} machine learning for approximating complex posterior distributions. Traditional VI often relies on the mean-field factorization, which can inadequately capture true posterior complexity. Recent advancements have leveraged neural networks to model implicit distributions, offering increased flexibility. However, the practical constraints of neural network architectures still produces inaccuracies. In this paper, we propose a method called Implicit Variational Rejection Sampling (IVRS), which integrates implicit distributions with rejection sampling to improve the posterior approximation. Our method uses neural networks to construct implicit proposal distributions, and rejection sampling with a discriminator network that estimates the density ratio between the implicit proposal and the true posterior for refining the approximation. Towards this end, we introduce the Implicit Resampling Evidence Lower Bound (IR-{ELBO}) as a metric to characterize the resampled distribution’s quality and derive a tighter variational lower bound. Experimental results demonstrate that our method outperforms traditional variational inference techniques.
APA
Xu, J., Li, S., Chen, W., Li, J., Lin, Z., Zeng, D., Ding, X., Paisley, J. & Zhao, Q.. (2026). Implicit Variational Rejection Sampling. Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 337:7651-7669 Available from https://proceedings.mlr.press/v337/xu26e.html.

Related Material