Universal Joint Approximation of Manifolds and Densities by Simple Injective Flows

Michael Puthawala, Matti Lassas, Ivan Dokmanic, Maarten De Hoop
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:17959-17983, 2022.

Abstract

We study approximation of probability measures supported on n-dimensional manifolds embedded in R^m by injective flows—neural networks composed of invertible flows and injective layers. We show that in general, injective flows between R^n and R^m universally approximate measures supported on images of extendable embeddings, which are a subset of standard embeddings: when the embedding dimension m is small, topological obstructions may preclude certain manifolds as admissible targets. When the embedding dimension is sufficiently large, m >= 3n+1, we use an argument from algebraic topology known as the clean trick to prove that the topological obstructions vanish and injective flows universally approximate any differentiable embedding. Along the way we show that the studied injective flows admit efficient projections on the range, and that their optimality can be established "in reverse," resolving a conjecture made in Brehmer & Cranmer 2020.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-puthawala22a, title = {Universal Joint Approximation of Manifolds and Densities by Simple Injective Flows}, author = {Puthawala, Michael and Lassas, Matti and Dokmanic, Ivan and De Hoop, Maarten}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {17959--17983}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/puthawala22a/puthawala22a.pdf}, url = {https://proceedings.mlr.press/v162/puthawala22a.html}, abstract = {We study approximation of probability measures supported on n-dimensional manifolds embedded in R^m by injective flows—neural networks composed of invertible flows and injective layers. We show that in general, injective flows between R^n and R^m universally approximate measures supported on images of extendable embeddings, which are a subset of standard embeddings: when the embedding dimension m is small, topological obstructions may preclude certain manifolds as admissible targets. When the embedding dimension is sufficiently large, m >= 3n+1, we use an argument from algebraic topology known as the clean trick to prove that the topological obstructions vanish and injective flows universally approximate any differentiable embedding. Along the way we show that the studied injective flows admit efficient projections on the range, and that their optimality can be established "in reverse," resolving a conjecture made in Brehmer & Cranmer 2020.} }
Endnote
%0 Conference Paper %T Universal Joint Approximation of Manifolds and Densities by Simple Injective Flows %A Michael Puthawala %A Matti Lassas %A Ivan Dokmanic %A Maarten De Hoop %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-puthawala22a %I PMLR %P 17959--17983 %U https://proceedings.mlr.press/v162/puthawala22a.html %V 162 %X We study approximation of probability measures supported on n-dimensional manifolds embedded in R^m by injective flows—neural networks composed of invertible flows and injective layers. We show that in general, injective flows between R^n and R^m universally approximate measures supported on images of extendable embeddings, which are a subset of standard embeddings: when the embedding dimension m is small, topological obstructions may preclude certain manifolds as admissible targets. When the embedding dimension is sufficiently large, m >= 3n+1, we use an argument from algebraic topology known as the clean trick to prove that the topological obstructions vanish and injective flows universally approximate any differentiable embedding. Along the way we show that the studied injective flows admit efficient projections on the range, and that their optimality can be established "in reverse," resolving a conjecture made in Brehmer & Cranmer 2020.
APA
Puthawala, M., Lassas, M., Dokmanic, I. & De Hoop, M.. (2022). Universal Joint Approximation of Manifolds and Densities by Simple Injective Flows. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:17959-17983 Available from https://proceedings.mlr.press/v162/puthawala22a.html.

Related Material