The Geometry of Diffusion Models: Tubular Neighbourhoods and Singularities

Kotaro Sakamoto, Ryosuke Sakamoto, Masato Tanabe, Masatomo Akagawa, Yusuke Hayashi, Manato Yaguchi, Masahiro Suzuki, Yutaka Matsuo
Proceedings of the Geometry-grounded Representation Learning and Generative Modeling Workshop (GRaM), PMLR 251:332-363, 2024.

Abstract

Diffusion generative models have been a leading approach for generating high-dimensional data. The current research aims to investigate the relation between the dynamics of diffusion models and the tubular neighbourhoods of a data manifold. We propose an algorithm to estimate the injectivity radius, the supremum of radii of tubular neighbourhoods. Our research relates geometric objects such as curvatures of data manifolds and dimensions of ambient spaces, to singularities of the generative dynamics such as emergent critical phenomena or spontaneous symmetry breaking.

Cite this Paper

BibTeX
@InProceedings{pmlr-v251-sakamoto24a, title = {The Geometry of Diffusion Models: Tubular Neighbourhoods and Singularities}, author = {Sakamoto, Kotaro and Sakamoto, Ryosuke and Tanabe, Masato and Akagawa, Masatomo and Hayashi, Yusuke and Yaguchi, Manato and Suzuki, Masahiro and Matsuo, Yutaka}, booktitle = {Proceedings of the Geometry-grounded Representation Learning and Generative Modeling Workshop (GRaM)}, pages = {332--363}, year = {2024}, editor = {Vadgama, Sharvaree and Bekkers, Erik and Pouplin, Alison and Kaba, Sekou-Oumar and Walters, Robin and Lawrence, Hannah and Emerson, Tegan and Kvinge, Henry and Tomczak, Jakub and Jegelka, Stephanie}, volume = {251}, series = {Proceedings of Machine Learning Research}, month = {29 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v251/main/assets/sakamoto24a/sakamoto24a.pdf}, url = {https://proceedings.mlr.press/v251/sakamoto24a.html}, abstract = {Diffusion generative models have been a leading approach for generating high-dimensional data. The current research aims to investigate the relation between the dynamics of diffusion models and the tubular neighbourhoods of a data manifold. We propose an algorithm to estimate the injectivity radius, the supremum of radii of tubular neighbourhoods. Our research relates geometric objects such as curvatures of data manifolds and dimensions of ambient spaces, to singularities of the generative dynamics such as emergent critical phenomena or spontaneous symmetry breaking.} }
Endnote
%0 Conference Paper %T The Geometry of Diffusion Models: Tubular Neighbourhoods and Singularities %A Kotaro Sakamoto %A Ryosuke Sakamoto %A Masato Tanabe %A Masatomo Akagawa %A Yusuke Hayashi %A Manato Yaguchi %A Masahiro Suzuki %A Yutaka Matsuo %B Proceedings of the Geometry-grounded Representation Learning and Generative Modeling Workshop (GRaM) %C Proceedings of Machine Learning Research %D 2024 %E Sharvaree Vadgama %E Erik Bekkers %E Alison Pouplin %E Sekou-Oumar Kaba %E Robin Walters %E Hannah Lawrence %E Tegan Emerson %E Henry Kvinge %E Jakub Tomczak %E Stephanie Jegelka %F pmlr-v251-sakamoto24a %I PMLR %P 332--363 %U https://proceedings.mlr.press/v251/sakamoto24a.html %V 251 %X Diffusion generative models have been a leading approach for generating high-dimensional data. The current research aims to investigate the relation between the dynamics of diffusion models and the tubular neighbourhoods of a data manifold. We propose an algorithm to estimate the injectivity radius, the supremum of radii of tubular neighbourhoods. Our research relates geometric objects such as curvatures of data manifolds and dimensions of ambient spaces, to singularities of the generative dynamics such as emergent critical phenomena or spontaneous symmetry breaking.
APA
Sakamoto, K., Sakamoto, R., Tanabe, M., Akagawa, M., Hayashi, Y., Yaguchi, M., Suzuki, M. & Matsuo, Y.. (2024). The Geometry of Diffusion Models: Tubular Neighbourhoods and Singularities. Proceedings of the Geometry-grounded Representation Learning and Generative Modeling Workshop (GRaM), in Proceedings of Machine Learning Research 251:332-363 Available from https://proceedings.mlr.press/v251/sakamoto24a.html.

Related Material