Triangular Flows for Generative Modeling: Statistical Consistency, Smoothness Classes, and Fast Rates

Nicholas J. Irons, Meyer Scetbon, Soumik Pal, Zaid Harchaoui
Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, PMLR 151:10161-10195, 2022.

Abstract

Triangular flows, also known as Knöthe-Rosenblatt measure couplings, comprise an important building block of normalizing flow models for generative modeling and density estimation, including popular autoregressive flows such as real-valued non-volume preserving transformation models (Real NVP). We present statistical guarantees and sample complexity bounds for triangular flow statistical models. In particular, we establish the statistical consistency and the finite sample convergence rates of the minimum Kullback-Leibler divergence statistical estimator of the Knöthe-Rosenblatt measure coupling using tools from empirical process theory. Our results highlight the anisotropic geometry of function classes at play in triangular flows, shed light on optimal coordinate ordering, and lead to statistical guarantees for Jacobian flows. We conduct numerical experiments to illustrate the practical implications of our theoretical findings.

Cite this Paper

BibTeX
@InProceedings{pmlr-v151-irons22a, title = { Triangular Flows for Generative Modeling: Statistical Consistency, Smoothness Classes, and Fast Rates }, author = {Irons, Nicholas J. and Scetbon, Meyer and Pal, Soumik and Harchaoui, Zaid}, booktitle = {Proceedings of The 25th International Conference on Artificial Intelligence and Statistics}, pages = {10161--10195}, year = {2022}, editor = {Camps-Valls, Gustau and Ruiz, Francisco J. R. and Valera, Isabel}, volume = {151}, series = {Proceedings of Machine Learning Research}, month = {28--30 Mar}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v151/irons22a/irons22a.pdf}, url = {https://proceedings.mlr.press/v151/irons22a.html}, abstract = { Triangular flows, also known as Knöthe-Rosenblatt measure couplings, comprise an important building block of normalizing flow models for generative modeling and density estimation, including popular autoregressive flows such as real-valued non-volume preserving transformation models (Real NVP). We present statistical guarantees and sample complexity bounds for triangular flow statistical models. In particular, we establish the statistical consistency and the finite sample convergence rates of the minimum Kullback-Leibler divergence statistical estimator of the Knöthe-Rosenblatt measure coupling using tools from empirical process theory. Our results highlight the anisotropic geometry of function classes at play in triangular flows, shed light on optimal coordinate ordering, and lead to statistical guarantees for Jacobian flows. We conduct numerical experiments to illustrate the practical implications of our theoretical findings. } }
Endnote
%0 Conference Paper %T Triangular Flows for Generative Modeling: Statistical Consistency, Smoothness Classes, and Fast Rates %A Nicholas J. Irons %A Meyer Scetbon %A Soumik Pal %A Zaid Harchaoui %B Proceedings of The 25th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2022 %E Gustau Camps-Valls %E Francisco J. R. Ruiz %E Isabel Valera %F pmlr-v151-irons22a %I PMLR %P 10161--10195 %U https://proceedings.mlr.press/v151/irons22a.html %V 151 %X Triangular flows, also known as Knöthe-Rosenblatt measure couplings, comprise an important building block of normalizing flow models for generative modeling and density estimation, including popular autoregressive flows such as real-valued non-volume preserving transformation models (Real NVP). We present statistical guarantees and sample complexity bounds for triangular flow statistical models. In particular, we establish the statistical consistency and the finite sample convergence rates of the minimum Kullback-Leibler divergence statistical estimator of the Knöthe-Rosenblatt measure coupling using tools from empirical process theory. Our results highlight the anisotropic geometry of function classes at play in triangular flows, shed light on optimal coordinate ordering, and lead to statistical guarantees for Jacobian flows. We conduct numerical experiments to illustrate the practical implications of our theoretical findings.
APA
Irons, N.J., Scetbon, M., Pal, S. & Harchaoui, Z.. (2022). Triangular Flows for Generative Modeling: Statistical Consistency, Smoothness Classes, and Fast Rates . Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 151:10161-10195 Available from https://proceedings.mlr.press/v151/irons22a.html.

Related Material