Score-of-Mixture Training: One-Step Generative Model Training Made Simple via Score Estimation of Mixture Distributions

Tejas Jayashankar, Jongha Jon Ryu, Gregory W. Wornell
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:27021-27049, 2025.

Abstract

We propose Score-of-Mixture Training (SMT), a novel framework for training one-step generative models by minimizing a class of divergences called the $\alpha$-skew Jensen–Shannon divergence. At its core, SMT estimates the score of mixture distributions between real and fake samples across multiple noise levels. Similar to consistency models, our approach supports both training from scratch (SMT) and distillation using a pretrained diffusion model, which we call Score-of-Mixture Distillation (SMD). It is simple to implement, requires minimal hyperparameter tuning, and ensures stable training. Experiments on CIFAR-10 and ImageNet 64$\times$64 show that SMT/SMD are competitive with and can even outperform existing methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-jayashankar25a, title = {Score-of-Mixture Training: One-Step Generative Model Training Made Simple via Score Estimation of Mixture Distributions}, author = {Jayashankar, Tejas and Ryu, Jongha Jon and Wornell, Gregory W.}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {27021--27049}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/jayashankar25a/jayashankar25a.pdf}, url = {https://proceedings.mlr.press/v267/jayashankar25a.html}, abstract = {We propose Score-of-Mixture Training (SMT), a novel framework for training one-step generative models by minimizing a class of divergences called the $\alpha$-skew Jensen–Shannon divergence. At its core, SMT estimates the score of mixture distributions between real and fake samples across multiple noise levels. Similar to consistency models, our approach supports both training from scratch (SMT) and distillation using a pretrained diffusion model, which we call Score-of-Mixture Distillation (SMD). It is simple to implement, requires minimal hyperparameter tuning, and ensures stable training. Experiments on CIFAR-10 and ImageNet 64$\times$64 show that SMT/SMD are competitive with and can even outperform existing methods.} }
Endnote
%0 Conference Paper %T Score-of-Mixture Training: One-Step Generative Model Training Made Simple via Score Estimation of Mixture Distributions %A Tejas Jayashankar %A Jongha Jon Ryu %A Gregory W. Wornell %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-jayashankar25a %I PMLR %P 27021--27049 %U https://proceedings.mlr.press/v267/jayashankar25a.html %V 267 %X We propose Score-of-Mixture Training (SMT), a novel framework for training one-step generative models by minimizing a class of divergences called the $\alpha$-skew Jensen–Shannon divergence. At its core, SMT estimates the score of mixture distributions between real and fake samples across multiple noise levels. Similar to consistency models, our approach supports both training from scratch (SMT) and distillation using a pretrained diffusion model, which we call Score-of-Mixture Distillation (SMD). It is simple to implement, requires minimal hyperparameter tuning, and ensures stable training. Experiments on CIFAR-10 and ImageNet 64$\times$64 show that SMT/SMD are competitive with and can even outperform existing methods.
APA
Jayashankar, T., Ryu, J.J. & Wornell, G.W.. (2025). Score-of-Mixture Training: One-Step Generative Model Training Made Simple via Score Estimation of Mixture Distributions. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:27021-27049 Available from https://proceedings.mlr.press/v267/jayashankar25a.html.

Related Material