Rényi Diffusion Models

Yirong Shen, Lu Gan, Cong Ling
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:110870-110911, 2026.

Abstract

The choice of training objective is central to diffusion-based generative modeling in terms of both sample quality and distribution coverage. While standard maximum likelihood training provides a principled objective with strong theoretical grounding, empirical studies indicate that previous training objectives in diffusion models often face an inverse correlation between likelihood optimization and perceptual evaluations. We propose the Rényi diffusion model, a unified generative framework that formulates training objectives using Rényi divergence. This yields a generalized score matching objective providing explicit control over the trade-off between sample quality and distribution coverage. Experiments demonstrate improved balance between density estimation and sample generation performances across multiple datasets without modifying model architectures or sampling procedures.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-shen26g, title = {Rényi Diffusion Models}, author = {Shen, Yirong and Gan, Lu and Ling, Cong}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {110870--110911}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/shen26g/shen26g.pdf}, url = {https://proceedings.mlr.press/v306/shen26g.html}, abstract = {The choice of training objective is central to diffusion-based generative modeling in terms of both sample quality and distribution coverage. While standard maximum likelihood training provides a principled objective with strong theoretical grounding, empirical studies indicate that previous training objectives in diffusion models often face an inverse correlation between likelihood optimization and perceptual evaluations. We propose the Rényi diffusion model, a unified generative framework that formulates training objectives using Rényi divergence. This yields a generalized score matching objective providing explicit control over the trade-off between sample quality and distribution coverage. Experiments demonstrate improved balance between density estimation and sample generation performances across multiple datasets without modifying model architectures or sampling procedures.} }
Endnote
%0 Conference Paper %T Rényi Diffusion Models %A Yirong Shen %A Lu Gan %A Cong Ling %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-shen26g %I PMLR %P 110870--110911 %U https://proceedings.mlr.press/v306/shen26g.html %V 306 %X The choice of training objective is central to diffusion-based generative modeling in terms of both sample quality and distribution coverage. While standard maximum likelihood training provides a principled objective with strong theoretical grounding, empirical studies indicate that previous training objectives in diffusion models often face an inverse correlation between likelihood optimization and perceptual evaluations. We propose the Rényi diffusion model, a unified generative framework that formulates training objectives using Rényi divergence. This yields a generalized score matching objective providing explicit control over the trade-off between sample quality and distribution coverage. Experiments demonstrate improved balance between density estimation and sample generation performances across multiple datasets without modifying model architectures or sampling procedures.
APA
Shen, Y., Gan, L. & Ling, C.. (2026). Rényi Diffusion Models. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:110870-110911 Available from https://proceedings.mlr.press/v306/shen26g.html.

Related Material