Semi-Supervised Learning with Normalizing Flows

Pavel Izmailov, Polina Kirichenko, Marc Finzi, Andrew Gordon Wilson
Proceedings of the 37th International Conference on Machine Learning, PMLR 119:4615-4630, 2020.

Abstract

Normalizing flows transform a latent distribution through an invertible neural network for a flexible and pleasingly simple approach to generative modelling, while preserving an exact likelihood. We propose FlowGMM, an end-to-end approach to generative semi supervised learning with normalizing flows, using a latent Gaussian mixture model. FlowGMM is distinct in its simplicity, unified treatment of labelled and unlabelled data with an exact likelihood, interpretability, and broad applicability beyond image data. We show promising results on a wide range of applications, including AG-News and Yahoo Answers text data, tabular data, and semi-supervised image classification. We also show that FlowGMM can discover interpretable structure, provide real-time optimization-free feature visualizations, and specify well calibrated predictive distributions.

Cite this Paper

BibTeX
@InProceedings{pmlr-v119-izmailov20a, title = {Semi-Supervised Learning with Normalizing Flows}, author = {Izmailov, Pavel and Kirichenko, Polina and Finzi, Marc and Wilson, Andrew Gordon}, booktitle = {Proceedings of the 37th International Conference on Machine Learning}, pages = {4615--4630}, year = {2020}, editor = {III, Hal Daumé and Singh, Aarti}, volume = {119}, series = {Proceedings of Machine Learning Research}, month = {13--18 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v119/izmailov20a/izmailov20a.pdf}, url = {https://proceedings.mlr.press/v119/izmailov20a.html}, abstract = {Normalizing flows transform a latent distribution through an invertible neural network for a flexible and pleasingly simple approach to generative modelling, while preserving an exact likelihood. We propose FlowGMM, an end-to-end approach to generative semi supervised learning with normalizing flows, using a latent Gaussian mixture model. FlowGMM is distinct in its simplicity, unified treatment of labelled and unlabelled data with an exact likelihood, interpretability, and broad applicability beyond image data. We show promising results on a wide range of applications, including AG-News and Yahoo Answers text data, tabular data, and semi-supervised image classification. We also show that FlowGMM can discover interpretable structure, provide real-time optimization-free feature visualizations, and specify well calibrated predictive distributions.} }
Endnote
%0 Conference Paper %T Semi-Supervised Learning with Normalizing Flows %A Pavel Izmailov %A Polina Kirichenko %A Marc Finzi %A Andrew Gordon Wilson %B Proceedings of the 37th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Hal Daumé III %E Aarti Singh %F pmlr-v119-izmailov20a %I PMLR %P 4615--4630 %U https://proceedings.mlr.press/v119/izmailov20a.html %V 119 %X Normalizing flows transform a latent distribution through an invertible neural network for a flexible and pleasingly simple approach to generative modelling, while preserving an exact likelihood. We propose FlowGMM, an end-to-end approach to generative semi supervised learning with normalizing flows, using a latent Gaussian mixture model. FlowGMM is distinct in its simplicity, unified treatment of labelled and unlabelled data with an exact likelihood, interpretability, and broad applicability beyond image data. We show promising results on a wide range of applications, including AG-News and Yahoo Answers text data, tabular data, and semi-supervised image classification. We also show that FlowGMM can discover interpretable structure, provide real-time optimization-free feature visualizations, and specify well calibrated predictive distributions.
APA
Izmailov, P., Kirichenko, P., Finzi, M. & Wilson, A.G.. (2020). Semi-Supervised Learning with Normalizing Flows. Proceedings of the 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research 119:4615-4630 Available from https://proceedings.mlr.press/v119/izmailov20a.html.

Related Material