MAD: Manifold Attracted Diffusion

Dennis Elbrächter, Giovanni S Alberti, Matteo Santacesaria
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:27822-27843, 2026.

Abstract

Score-based diffusion models are a highly effective method for generating samples from a distribution of images. We consider scenarios where the training data comes from a noisy version of the target distribution, and present an efficiently implementable modification of the inference procedure to generate noiseless samples. Our approach is motivated by the manifold hypothesis, according to which meaningful data is concentrated around some low-dimensional manifold of a high-dimensional ambient space. The central idea is that noise manifests as low magnitude variation in off-manifold directions in contrast to the relevant variation of the desired distribution which is mostly confined to on-manifold directions. We introduce the notion of an extended score and show that, in a simplified setting, it can be used to reduce small variations to zero, while leaving large variations mostly unchanged. We describe how its approximation can be computed efficiently from an approximation to the standard score and demonstrate its efficacy on toy problems, synthetic data, and real data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-elbrachter26a, title = {{MAD}: Manifold Attracted Diffusion}, author = {Elbr\"{a}chter, Dennis and Alberti, Giovanni S and Santacesaria, Matteo}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {27822--27843}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/elbrachter26a/elbrachter26a.pdf}, url = {https://proceedings.mlr.press/v306/elbrachter26a.html}, abstract = {Score-based diffusion models are a highly effective method for generating samples from a distribution of images. We consider scenarios where the training data comes from a noisy version of the target distribution, and present an efficiently implementable modification of the inference procedure to generate noiseless samples. Our approach is motivated by the manifold hypothesis, according to which meaningful data is concentrated around some low-dimensional manifold of a high-dimensional ambient space. The central idea is that noise manifests as low magnitude variation in off-manifold directions in contrast to the relevant variation of the desired distribution which is mostly confined to on-manifold directions. We introduce the notion of an extended score and show that, in a simplified setting, it can be used to reduce small variations to zero, while leaving large variations mostly unchanged. We describe how its approximation can be computed efficiently from an approximation to the standard score and demonstrate its efficacy on toy problems, synthetic data, and real data.} }
Endnote
%0 Conference Paper %T MAD: Manifold Attracted Diffusion %A Dennis Elbrächter %A Giovanni S Alberti %A Matteo Santacesaria %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-elbrachter26a %I PMLR %P 27822--27843 %U https://proceedings.mlr.press/v306/elbrachter26a.html %V 306 %X Score-based diffusion models are a highly effective method for generating samples from a distribution of images. We consider scenarios where the training data comes from a noisy version of the target distribution, and present an efficiently implementable modification of the inference procedure to generate noiseless samples. Our approach is motivated by the manifold hypothesis, according to which meaningful data is concentrated around some low-dimensional manifold of a high-dimensional ambient space. The central idea is that noise manifests as low magnitude variation in off-manifold directions in contrast to the relevant variation of the desired distribution which is mostly confined to on-manifold directions. We introduce the notion of an extended score and show that, in a simplified setting, it can be used to reduce small variations to zero, while leaving large variations mostly unchanged. We describe how its approximation can be computed efficiently from an approximation to the standard score and demonstrate its efficacy on toy problems, synthetic data, and real data.
APA
Elbrächter, D., Alberti, G.S. & Santacesaria, M.. (2026). MAD: Manifold Attracted Diffusion. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:27822-27843 Available from https://proceedings.mlr.press/v306/elbrachter26a.html.

Related Material