Continuous-Time Flows for Efficient Inference and Density Estimation

Changyou Chen, Chunyuan Li, Liqun Chen, Wenlin Wang, Yunchen Pu, Lawrence Carin Duke
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:824-833, 2018.

Abstract

Two fundamental problems in unsupervised learning are efficient inference for latent-variable models and robust density estimation based on large amounts of unlabeled data. Algorithms for the two tasks, such as normalizing flows and generative adversarial networks (GANs), are often developed independently. In this paper, we propose the concept of continuous-time flows (CTFs), a family of diffusion-based methods that are able to asymptotically approach a target distribution. Distinct from normalizing flows and GANs, CTFs can be adopted to achieve the above two goals in one framework, with theoretical guarantees. Our framework includes distilling knowledge from a CTF for efficient inference, and learning an explicit energy-based distribution with CTFs for density estimation. Both tasks rely on a new technique for distribution matching within amortized learning. Experiments on various tasks demonstrate promising performance of the proposed CTF framework, compared to related techniques.

Cite this Paper

BibTeX
@InProceedings{pmlr-v80-chen18d, title = {Continuous-Time Flows for Efficient Inference and Density Estimation}, author = {Chen, Changyou and Li, Chunyuan and Chen, Liqun and Wang, Wenlin and Pu, Yunchen and Duke, Lawrence Carin}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {824--833}, year = {2018}, editor = {Dy, Jennifer and Krause, Andreas}, volume = {80}, series = {Proceedings of Machine Learning Research}, month = {10--15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v80/chen18d/chen18d.pdf}, url = {https://proceedings.mlr.press/v80/chen18d.html}, abstract = {Two fundamental problems in unsupervised learning are efficient inference for latent-variable models and robust density estimation based on large amounts of unlabeled data. Algorithms for the two tasks, such as normalizing flows and generative adversarial networks (GANs), are often developed independently. In this paper, we propose the concept of continuous-time flows (CTFs), a family of diffusion-based methods that are able to asymptotically approach a target distribution. Distinct from normalizing flows and GANs, CTFs can be adopted to achieve the above two goals in one framework, with theoretical guarantees. Our framework includes distilling knowledge from a CTF for efficient inference, and learning an explicit energy-based distribution with CTFs for density estimation. Both tasks rely on a new technique for distribution matching within amortized learning. Experiments on various tasks demonstrate promising performance of the proposed CTF framework, compared to related techniques.} }
Endnote
%0 Conference Paper %T Continuous-Time Flows for Efficient Inference and Density Estimation %A Changyou Chen %A Chunyuan Li %A Liqun Chen %A Wenlin Wang %A Yunchen Pu %A Lawrence Carin Duke %B Proceedings of the 35th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jennifer Dy %E Andreas Krause %F pmlr-v80-chen18d %I PMLR %P 824--833 %U https://proceedings.mlr.press/v80/chen18d.html %V 80 %X Two fundamental problems in unsupervised learning are efficient inference for latent-variable models and robust density estimation based on large amounts of unlabeled data. Algorithms for the two tasks, such as normalizing flows and generative adversarial networks (GANs), are often developed independently. In this paper, we propose the concept of continuous-time flows (CTFs), a family of diffusion-based methods that are able to asymptotically approach a target distribution. Distinct from normalizing flows and GANs, CTFs can be adopted to achieve the above two goals in one framework, with theoretical guarantees. Our framework includes distilling knowledge from a CTF for efficient inference, and learning an explicit energy-based distribution with CTFs for density estimation. Both tasks rely on a new technique for distribution matching within amortized learning. Experiments on various tasks demonstrate promising performance of the proposed CTF framework, compared to related techniques.
APA
Chen, C., Li, C., Chen, L., Wang, W., Pu, Y. & Duke, L.C.. (2018). Continuous-Time Flows for Efficient Inference and Density Estimation. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research 80:824-833 Available from https://proceedings.mlr.press/v80/chen18d.html.

Related Material