On the Anisotropy of Score-Based Generative Models

Andreas Floros, Seyed-Mohsen Moosavi-Dezfooli, Pier Luigi Dragotti
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:31189-31202, 2026.

Abstract

We investigate the role of network architecture in shaping the inductive biases of modern score-based generative models. To this end, we introduce the Score Anisotropy Directions (SADs), architecture-dependent directions that reveal how different networks preferentially capture data structure. Our analysis suggests that SADs form adaptive bases aligned with the architecture’s output geometry, providing a principled way to predict generalization ability in score models prior to training. Through both synthetic data and standard image benchmarks, we demonstrate that SADs reliably capture fine-grained model behavior and correlate with downstream performance, as measured by Wasserstein metrics. Our work offers a new lens for explaining and predicting directional biases of generative models.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-floros26a, title = {On the Anisotropy of Score-Based Generative Models}, author = {Floros, Andreas and Moosavi-Dezfooli, Seyed-Mohsen and Dragotti, Pier Luigi}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {31189--31202}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/floros26a/floros26a.pdf}, url = {https://proceedings.mlr.press/v306/floros26a.html}, abstract = {We investigate the role of network architecture in shaping the inductive biases of modern score-based generative models. To this end, we introduce the Score Anisotropy Directions (SADs), architecture-dependent directions that reveal how different networks preferentially capture data structure. Our analysis suggests that SADs form adaptive bases aligned with the architecture’s output geometry, providing a principled way to predict generalization ability in score models prior to training. Through both synthetic data and standard image benchmarks, we demonstrate that SADs reliably capture fine-grained model behavior and correlate with downstream performance, as measured by Wasserstein metrics. Our work offers a new lens for explaining and predicting directional biases of generative models.} }
Endnote
%0 Conference Paper %T On the Anisotropy of Score-Based Generative Models %A Andreas Floros %A Seyed-Mohsen Moosavi-Dezfooli %A Pier Luigi Dragotti %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-floros26a %I PMLR %P 31189--31202 %U https://proceedings.mlr.press/v306/floros26a.html %V 306 %X We investigate the role of network architecture in shaping the inductive biases of modern score-based generative models. To this end, we introduce the Score Anisotropy Directions (SADs), architecture-dependent directions that reveal how different networks preferentially capture data structure. Our analysis suggests that SADs form adaptive bases aligned with the architecture’s output geometry, providing a principled way to predict generalization ability in score models prior to training. Through both synthetic data and standard image benchmarks, we demonstrate that SADs reliably capture fine-grained model behavior and correlate with downstream performance, as measured by Wasserstein metrics. Our work offers a new lens for explaining and predicting directional biases of generative models.
APA
Floros, A., Moosavi-Dezfooli, S. & Dragotti, P.L.. (2026). On the Anisotropy of Score-Based Generative Models. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:31189-31202 Available from https://proceedings.mlr.press/v306/floros26a.html.

Related Material