Marginal Tail-Adaptive Normalizing Flows

Mike Laszkiewicz, Johannes Lederer, Asja Fischer
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:12020-12048, 2022.

Abstract

Learning the tail behavior of a distribution is a notoriously difficult problem. By definition, the number of samples from the tail is small, and deep generative models, such as normalizing flows, tend to concentrate on learning the body of the distribution. In this paper, we focus on improving the ability of normalizing flows to correctly capture the tail behavior and, thus, form more accurate models. We prove that the marginal tailedness of an autoregressive flow can be controlled via the tailedness of the marginals of its base distribution. This theoretical insight leads us to a novel type of flows based on flexible base distributions and data-driven linear layers. An empirical analysis shows that the proposed method improves on the accuracy{—}especially on the tails of the distribution{—}and is able to generate heavy-tailed data. We demonstrate its application on a weather and climate example, in which capturing the tail behavior is essential.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-laszkiewicz22a, title = {Marginal Tail-Adaptive Normalizing Flows}, author = {Laszkiewicz, Mike and Lederer, Johannes and Fischer, Asja}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {12020--12048}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/laszkiewicz22a/laszkiewicz22a.pdf}, url = {https://proceedings.mlr.press/v162/laszkiewicz22a.html}, abstract = {Learning the tail behavior of a distribution is a notoriously difficult problem. By definition, the number of samples from the tail is small, and deep generative models, such as normalizing flows, tend to concentrate on learning the body of the distribution. In this paper, we focus on improving the ability of normalizing flows to correctly capture the tail behavior and, thus, form more accurate models. We prove that the marginal tailedness of an autoregressive flow can be controlled via the tailedness of the marginals of its base distribution. This theoretical insight leads us to a novel type of flows based on flexible base distributions and data-driven linear layers. An empirical analysis shows that the proposed method improves on the accuracy{—}especially on the tails of the distribution{—}and is able to generate heavy-tailed data. We demonstrate its application on a weather and climate example, in which capturing the tail behavior is essential.} }
Endnote
%0 Conference Paper %T Marginal Tail-Adaptive Normalizing Flows %A Mike Laszkiewicz %A Johannes Lederer %A Asja Fischer %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-laszkiewicz22a %I PMLR %P 12020--12048 %U https://proceedings.mlr.press/v162/laszkiewicz22a.html %V 162 %X Learning the tail behavior of a distribution is a notoriously difficult problem. By definition, the number of samples from the tail is small, and deep generative models, such as normalizing flows, tend to concentrate on learning the body of the distribution. In this paper, we focus on improving the ability of normalizing flows to correctly capture the tail behavior and, thus, form more accurate models. We prove that the marginal tailedness of an autoregressive flow can be controlled via the tailedness of the marginals of its base distribution. This theoretical insight leads us to a novel type of flows based on flexible base distributions and data-driven linear layers. An empirical analysis shows that the proposed method improves on the accuracy{—}especially on the tails of the distribution{—}and is able to generate heavy-tailed data. We demonstrate its application on a weather and climate example, in which capturing the tail behavior is essential.
APA
Laszkiewicz, M., Lederer, J. & Fischer, A.. (2022). Marginal Tail-Adaptive Normalizing Flows. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:12020-12048 Available from https://proceedings.mlr.press/v162/laszkiewicz22a.html.

Related Material