Sinusoidal Flow: A Fast Invertible Autoregressive Flow

Yumou Wei
Proceedings of The 13th Asian Conference on Machine Learning, PMLR 157:299-314, 2021.

Abstract

Normalising flows offer a flexible way of modelling continuous probability distributions. We consider expressiveness, fast inversion and exact Jacobian determinant as three desirable properties a normalising flow should possess. However, few flow models have been able to strike a good balance among all these properties. Realising that the integral of a convex sum of sinusoidal functions squared leads to a bijective residual transformation, we propose Sinusoidal Flow, a new type of normalising flows that inherits the expressive power and triangular Jacobian from fully autoregressive flows while guaranteed by Banach fixed-point theorem to remain fast invertible and thereby obviate the need for sequential inversion typically required in fully autoregressive flows. Experiments show that our Sinusoidal Flow is not only able to model complex distributions, but can also be reliably inverted to generate realistic-looking samples even with many layers of transformations stacked.

Cite this Paper


BibTeX
@InProceedings{pmlr-v157-wei21a, title = {Sinusoidal Flow: A Fast Invertible Autoregressive Flow}, author = {Wei, Yumou}, booktitle = {Proceedings of The 13th Asian Conference on Machine Learning}, pages = {299--314}, year = {2021}, editor = {Balasubramanian, Vineeth N. and Tsang, Ivor}, volume = {157}, series = {Proceedings of Machine Learning Research}, month = {17--19 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v157/wei21a/wei21a.pdf}, url = {https://proceedings.mlr.press/v157/wei21a.html}, abstract = {Normalising flows offer a flexible way of modelling continuous probability distributions. We consider expressiveness, fast inversion and exact Jacobian determinant as three desirable properties a normalising flow should possess. However, few flow models have been able to strike a good balance among all these properties. Realising that the integral of a convex sum of sinusoidal functions squared leads to a bijective residual transformation, we propose Sinusoidal Flow, a new type of normalising flows that inherits the expressive power and triangular Jacobian from fully autoregressive flows while guaranteed by Banach fixed-point theorem to remain fast invertible and thereby obviate the need for sequential inversion typically required in fully autoregressive flows. Experiments show that our Sinusoidal Flow is not only able to model complex distributions, but can also be reliably inverted to generate realistic-looking samples even with many layers of transformations stacked.} }
Endnote
%0 Conference Paper %T Sinusoidal Flow: A Fast Invertible Autoregressive Flow %A Yumou Wei %B Proceedings of The 13th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Vineeth N. Balasubramanian %E Ivor Tsang %F pmlr-v157-wei21a %I PMLR %P 299--314 %U https://proceedings.mlr.press/v157/wei21a.html %V 157 %X Normalising flows offer a flexible way of modelling continuous probability distributions. We consider expressiveness, fast inversion and exact Jacobian determinant as three desirable properties a normalising flow should possess. However, few flow models have been able to strike a good balance among all these properties. Realising that the integral of a convex sum of sinusoidal functions squared leads to a bijective residual transformation, we propose Sinusoidal Flow, a new type of normalising flows that inherits the expressive power and triangular Jacobian from fully autoregressive flows while guaranteed by Banach fixed-point theorem to remain fast invertible and thereby obviate the need for sequential inversion typically required in fully autoregressive flows. Experiments show that our Sinusoidal Flow is not only able to model complex distributions, but can also be reliably inverted to generate realistic-looking samples even with many layers of transformations stacked.
APA
Wei, Y.. (2021). Sinusoidal Flow: A Fast Invertible Autoregressive Flow. Proceedings of The 13th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 157:299-314 Available from https://proceedings.mlr.press/v157/wei21a.html.

Related Material