Explicit Density Approximation for Neural Implicit Samplers Using a Bernstein-Based Convex Divergence

José Manuel de Frutos, Pablo M. Olmos, Manuel A. Vázquez, Joaquin Miguez
Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, PMLR 300:1270-1278, 2026.

Abstract

Rank-based objectives such as the invariant statistical loss (ISL) are robust, likelihood-free tools for training implicit generative models. We propose \emph{dual-ISL}, obtained by interchanging the roles of the target $p$ and model density $\tilde p$ within ISL, which induces a \emph{convex} optimization problem over model densities. We show that the associated rank-based discrepancy $d_K$ is \emph{continuous} under weak and $L^1$ convergence and \emph{convex} in its first argument, properties not shared by classical divergences such as KL or Wasserstein distances. Additionally, we prove that $d_K$ admits an $L^2$ interpretation: it is the projection of the density ratio $q=p/\tilde p$ onto a Bernstein polynomial basis. This yields explicit truncation-error bounds, sharp convergence rates, and a closed-form expression for the truncated density approximation. To handle multivariate data, we further introduce a sliced dual-ISL via random one-dimensional projections that preserves both continuity and convexity. Empirically, across several benchmarks, dual-ISL delivers faster and smoother convergence than standard ISL and offers competitive, often superior, mode coverage relative to state-of-the-art implicit models (modern GAN baselines, including multi-critic setups), while providing an explicit density approximation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v300-frutos26a, title = { Explicit Density Approximation for Neural Implicit Samplers Using a Bernstein-Based Convex Divergence }, author = {de Frutos, Jos{\'e} Manuel and Olmos, Pablo M. and V{\'a}zquez, Manuel A. and Miguez, Joaquin}, booktitle = {Proceedings of The 29th International Conference on Artificial Intelligence and Statistics}, pages = {1270--1278}, year = {2026}, editor = {Khan, Emtiyaz and Li, Yingzhen and Solin, Arno and Ramdas, Aaditya}, volume = {300}, series = {Proceedings of Machine Learning Research}, month = {02--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v300/main/assets/frutos26a/frutos26a.pdf}, url = {https://proceedings.mlr.press/v300/frutos26a.html}, abstract = { Rank-based objectives such as the invariant statistical loss (ISL) are robust, likelihood-free tools for training implicit generative models. We propose \emph{dual-ISL}, obtained by interchanging the roles of the target $p$ and model density $\tilde p$ within ISL, which induces a \emph{convex} optimization problem over model densities. We show that the associated rank-based discrepancy $d_K$ is \emph{continuous} under weak and $L^1$ convergence and \emph{convex} in its first argument, properties not shared by classical divergences such as KL or Wasserstein distances. Additionally, we prove that $d_K$ admits an $L^2$ interpretation: it is the projection of the density ratio $q=p/\tilde p$ onto a Bernstein polynomial basis. This yields explicit truncation-error bounds, sharp convergence rates, and a closed-form expression for the truncated density approximation. To handle multivariate data, we further introduce a sliced dual-ISL via random one-dimensional projections that preserves both continuity and convexity. Empirically, across several benchmarks, dual-ISL delivers faster and smoother convergence than standard ISL and offers competitive, often superior, mode coverage relative to state-of-the-art implicit models (modern GAN baselines, including multi-critic setups), while providing an explicit density approximation. } }
Endnote
%0 Conference Paper %T Explicit Density Approximation for Neural Implicit Samplers Using a Bernstein-Based Convex Divergence %A José Manuel de Frutos %A Pablo M. Olmos %A Manuel A. Vázquez %A Joaquin Miguez %B Proceedings of The 29th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2026 %E Emtiyaz Khan %E Yingzhen Li %E Arno Solin %E Aaditya Ramdas %F pmlr-v300-frutos26a %I PMLR %P 1270--1278 %U https://proceedings.mlr.press/v300/frutos26a.html %V 300 %X Rank-based objectives such as the invariant statistical loss (ISL) are robust, likelihood-free tools for training implicit generative models. We propose \emph{dual-ISL}, obtained by interchanging the roles of the target $p$ and model density $\tilde p$ within ISL, which induces a \emph{convex} optimization problem over model densities. We show that the associated rank-based discrepancy $d_K$ is \emph{continuous} under weak and $L^1$ convergence and \emph{convex} in its first argument, properties not shared by classical divergences such as KL or Wasserstein distances. Additionally, we prove that $d_K$ admits an $L^2$ interpretation: it is the projection of the density ratio $q=p/\tilde p$ onto a Bernstein polynomial basis. This yields explicit truncation-error bounds, sharp convergence rates, and a closed-form expression for the truncated density approximation. To handle multivariate data, we further introduce a sliced dual-ISL via random one-dimensional projections that preserves both continuity and convexity. Empirically, across several benchmarks, dual-ISL delivers faster and smoother convergence than standard ISL and offers competitive, often superior, mode coverage relative to state-of-the-art implicit models (modern GAN baselines, including multi-critic setups), while providing an explicit density approximation.
APA
de Frutos, J.M., Olmos, P.M., Vázquez, M.A. & Miguez, J.. (2026). Explicit Density Approximation for Neural Implicit Samplers Using a Bernstein-Based Convex Divergence . Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 300:1270-1278 Available from https://proceedings.mlr.press/v300/frutos26a.html.

Related Material