A Change of Variables Method For Rectangular Matrix-Vector Products

Edmond Cunningham, Madalina Fiterau
Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, PMLR 130:2755-2763, 2021.

Abstract

Rectangular matrix-vector products (MVPs) are used extensively throughout machine learning and are fundamental to neural networks such as multi-layer perceptrons. However, the use of rectangular MVPs in successive normalizing flow transformations is notably missing. This paper identifies this methodological gap and plugs it with a tall and wide MVP change of variables formula. Our theory builds up to a practical algorithm that envelops existing dimensionality increasing flow methods such as augmented flows. We show that tall MVPs are closely related to the stochastic inverse of wide MVPs and empirically demonstrate that they improve density estimation over existing dimension changing methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v130-cunningham21a, title = { A Change of Variables Method For Rectangular Matrix-Vector Products }, author = {Cunningham, Edmond and Fiterau, Madalina}, booktitle = {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics}, pages = {2755--2763}, year = {2021}, editor = {Banerjee, Arindam and Fukumizu, Kenji}, volume = {130}, series = {Proceedings of Machine Learning Research}, month = {13--15 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v130/cunningham21a/cunningham21a.pdf}, url = {https://proceedings.mlr.press/v130/cunningham21a.html}, abstract = { Rectangular matrix-vector products (MVPs) are used extensively throughout machine learning and are fundamental to neural networks such as multi-layer perceptrons. However, the use of rectangular MVPs in successive normalizing flow transformations is notably missing. This paper identifies this methodological gap and plugs it with a tall and wide MVP change of variables formula. Our theory builds up to a practical algorithm that envelops existing dimensionality increasing flow methods such as augmented flows. We show that tall MVPs are closely related to the stochastic inverse of wide MVPs and empirically demonstrate that they improve density estimation over existing dimension changing methods. } }
Endnote
%0 Conference Paper %T A Change of Variables Method For Rectangular Matrix-Vector Products %A Edmond Cunningham %A Madalina Fiterau %B Proceedings of The 24th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2021 %E Arindam Banerjee %E Kenji Fukumizu %F pmlr-v130-cunningham21a %I PMLR %P 2755--2763 %U https://proceedings.mlr.press/v130/cunningham21a.html %V 130 %X Rectangular matrix-vector products (MVPs) are used extensively throughout machine learning and are fundamental to neural networks such as multi-layer perceptrons. However, the use of rectangular MVPs in successive normalizing flow transformations is notably missing. This paper identifies this methodological gap and plugs it with a tall and wide MVP change of variables formula. Our theory builds up to a practical algorithm that envelops existing dimensionality increasing flow methods such as augmented flows. We show that tall MVPs are closely related to the stochastic inverse of wide MVPs and empirically demonstrate that they improve density estimation over existing dimension changing methods.
APA
Cunningham, E. & Fiterau, M.. (2021). A Change of Variables Method For Rectangular Matrix-Vector Products . Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 130:2755-2763 Available from https://proceedings.mlr.press/v130/cunningham21a.html.

Related Material