The Expressive Power of a Class of Normalizing Flow Models

Zhifeng Kong, Kamalika Chaudhuri
Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, PMLR 108:3599-3609, 2020.

Abstract

Normalizing flows have received a great deal of recent attention as they allow flexible generative modeling as well as easy likelihood computation. While a wide variety of flow models have been proposed, there is little formal understanding of the representation power of these models. In this work, we study some basic normalizing flows and rigorously establish bounds on their expressive power. Our results indicate that while these flows are highly expressive in one dimension, in higher dimensions their representation power may be limited, especially when the flows have moderate depth.

Cite this Paper

BibTeX
@InProceedings{pmlr-v108-kong20a, title = {The Expressive Power of a Class of Normalizing Flow Models}, author = {Kong, Zhifeng and Chaudhuri, Kamalika}, booktitle = {Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics}, pages = {3599--3609}, year = {2020}, editor = {Chiappa, Silvia and Calandra, Roberto}, volume = {108}, series = {Proceedings of Machine Learning Research}, month = {26--28 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v108/kong20a/kong20a.pdf}, url = {https://proceedings.mlr.press/v108/kong20a.html}, abstract = {Normalizing flows have received a great deal of recent attention as they allow flexible generative modeling as well as easy likelihood computation. While a wide variety of flow models have been proposed, there is little formal understanding of the representation power of these models. In this work, we study some basic normalizing flows and rigorously establish bounds on their expressive power. Our results indicate that while these flows are highly expressive in one dimension, in higher dimensions their representation power may be limited, especially when the flows have moderate depth. } }
Endnote
%0 Conference Paper %T The Expressive Power of a Class of Normalizing Flow Models %A Zhifeng Kong %A Kamalika Chaudhuri %B Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2020 %E Silvia Chiappa %E Roberto Calandra %F pmlr-v108-kong20a %I PMLR %P 3599--3609 %U https://proceedings.mlr.press/v108/kong20a.html %V 108 %X Normalizing flows have received a great deal of recent attention as they allow flexible generative modeling as well as easy likelihood computation. While a wide variety of flow models have been proposed, there is little formal understanding of the representation power of these models. In this work, we study some basic normalizing flows and rigorously establish bounds on their expressive power. Our results indicate that while these flows are highly expressive in one dimension, in higher dimensions their representation power may be limited, especially when the flows have moderate depth.
APA
Kong, Z. & Chaudhuri, K.. (2020). The Expressive Power of a Class of Normalizing Flow Models. Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 108:3599-3609 Available from https://proceedings.mlr.press/v108/kong20a.html.

Related Material