Scalable Normalizing Flows for Permutation Invariant Densities

Marin Biloš, Stephan Günnemann
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:957-967, 2021.

Abstract

Modeling sets is an important problem in machine learning since this type of data can be found in many domains. A promising approach defines a family of permutation invariant densities with continuous normalizing flows. This allows us to maximize the likelihood directly and sample new realizations with ease. In this work, we demonstrate how calculating the trace, a crucial step in this method, raises issues that occur both during training and inference, limiting its practicality. We propose an alternative way of defining permutation equivariant transformations that give closed form trace. This leads not only to improvements while training, but also to better final performance. We demonstrate the benefits of our approach on point processes and general set modeling.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-bilos21a, title = {Scalable Normalizing Flows for Permutation Invariant Densities}, author = {Bilo{\v{s}}, Marin and G{\"u}nnemann, Stephan}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {957--967}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/bilos21a/bilos21a.pdf}, url = {https://proceedings.mlr.press/v139/bilos21a.html}, abstract = {Modeling sets is an important problem in machine learning since this type of data can be found in many domains. A promising approach defines a family of permutation invariant densities with continuous normalizing flows. This allows us to maximize the likelihood directly and sample new realizations with ease. In this work, we demonstrate how calculating the trace, a crucial step in this method, raises issues that occur both during training and inference, limiting its practicality. We propose an alternative way of defining permutation equivariant transformations that give closed form trace. This leads not only to improvements while training, but also to better final performance. We demonstrate the benefits of our approach on point processes and general set modeling.} }
Endnote
%0 Conference Paper %T Scalable Normalizing Flows for Permutation Invariant Densities %A Marin Biloš %A Stephan Günnemann %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-bilos21a %I PMLR %P 957--967 %U https://proceedings.mlr.press/v139/bilos21a.html %V 139 %X Modeling sets is an important problem in machine learning since this type of data can be found in many domains. A promising approach defines a family of permutation invariant densities with continuous normalizing flows. This allows us to maximize the likelihood directly and sample new realizations with ease. In this work, we demonstrate how calculating the trace, a crucial step in this method, raises issues that occur both during training and inference, limiting its practicality. We propose an alternative way of defining permutation equivariant transformations that give closed form trace. This leads not only to improvements while training, but also to better final performance. We demonstrate the benefits of our approach on point processes and general set modeling.
APA
Biloš, M. & Günnemann, S.. (2021). Scalable Normalizing Flows for Permutation Invariant Densities. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:957-967 Available from https://proceedings.mlr.press/v139/bilos21a.html.

Related Material