Skipping the Zeros in Diffusion Models for Sparse Data Generation

Phil Ostheimer, Mayank Nagda, Andriy Balinskyy, Gabriel Vicente Rodrigues, Jean Radig, Carl Herrmann, Stephan Mandt, Marius Kloft, Sophie Fellenz
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:94967-94988, 2026.

Abstract

Diffusion models (DMs) excel on dense continuous data, but are not designed for sparse continuous data. They do not model exact zeros that represent the deliberate absence of a signal. As a result, they erase sparsity patterns and perform unnecessary computation on mostly zero entries. With Sparsity-Exploiting Diffusion (SED), we model only non-zero values, preserving sparsity. SED delivers computational savings while maintaining or improving generation quality by skipping zeros during training and inference. Across physics and biology benchmarks, SED matches or surpasses conventional DMs and domain-specific baselines, while vision experiments provide intuitive insights into the limitations of dense DMs and the benefits of SED.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-ostheimer26a, title = {Skipping the Zeros in Diffusion Models for Sparse Data Generation}, author = {Ostheimer, Phil and Nagda, Mayank and Balinskyy, Andriy and Rodrigues, Gabriel Vicente and Radig, Jean and Herrmann, Carl and Mandt, Stephan and Kloft, Marius and Fellenz, Sophie}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {94967--94988}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/ostheimer26a/ostheimer26a.pdf}, url = {https://proceedings.mlr.press/v306/ostheimer26a.html}, abstract = {Diffusion models (DMs) excel on dense continuous data, but are not designed for sparse continuous data. They do not model exact zeros that represent the deliberate absence of a signal. As a result, they erase sparsity patterns and perform unnecessary computation on mostly zero entries. With Sparsity-Exploiting Diffusion (SED), we model only non-zero values, preserving sparsity. SED delivers computational savings while maintaining or improving generation quality by skipping zeros during training and inference. Across physics and biology benchmarks, SED matches or surpasses conventional DMs and domain-specific baselines, while vision experiments provide intuitive insights into the limitations of dense DMs and the benefits of SED.} }
Endnote
%0 Conference Paper %T Skipping the Zeros in Diffusion Models for Sparse Data Generation %A Phil Ostheimer %A Mayank Nagda %A Andriy Balinskyy %A Gabriel Vicente Rodrigues %A Jean Radig %A Carl Herrmann %A Stephan Mandt %A Marius Kloft %A Sophie Fellenz %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-ostheimer26a %I PMLR %P 94967--94988 %U https://proceedings.mlr.press/v306/ostheimer26a.html %V 306 %X Diffusion models (DMs) excel on dense continuous data, but are not designed for sparse continuous data. They do not model exact zeros that represent the deliberate absence of a signal. As a result, they erase sparsity patterns and perform unnecessary computation on mostly zero entries. With Sparsity-Exploiting Diffusion (SED), we model only non-zero values, preserving sparsity. SED delivers computational savings while maintaining or improving generation quality by skipping zeros during training and inference. Across physics and biology benchmarks, SED matches or surpasses conventional DMs and domain-specific baselines, while vision experiments provide intuitive insights into the limitations of dense DMs and the benefits of SED.
APA
Ostheimer, P., Nagda, M., Balinskyy, A., Rodrigues, G.V., Radig, J., Herrmann, C., Mandt, S., Kloft, M. & Fellenz, S.. (2026). Skipping the Zeros in Diffusion Models for Sparse Data Generation. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:94967-94988 Available from https://proceedings.mlr.press/v306/ostheimer26a.html.

Related Material