Unsupervised representation learning with recognition-parametrised probabilistic models

William I. Walker, Hugo Soulat, Changmin Yu, Maneesh Sahani
Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR 206:4209-4230, 2023.

Abstract

We introduce a new approach to probabilistic unsupervised learning based on the recognition-parametrised model (RPM): a normalised semi-parametric hypothesis class for joint distributions over observed and latent variables. Under the key assumption that observations are conditionally independent given latents, the RPM combines parametric prior and observation-conditioned latent distributions with non-parametric observation marginals. This approach leads to a flexible learnt recognition model capturing latent dependence between observations, without the need for an explicit, parametric generative model. The RPM admits exact maximum-likelihood learning for discrete latents, even for powerful neural network-based recognition. We develop effective approximations applicable in the continuous latent case. Experiments demonstrate the effectiveness of the RPM on high-dimensional data, learning image classification from weak indirect supervision; direct image-level latent Dirichlet allocation; and Recognition-Parametrised Gaussian Process Factor Analysis (RP-GPFA) applied to multi-factorial spatiotemporal datasets. The RPM provides a powerful framework to discover meaningful latent structure underlying observational data, a function critical to both animal and artificial intelligence.

Cite this Paper

BibTeX
@InProceedings{pmlr-v206-walker23a, title = {Unsupervised representation learning with recognition-parametrised probabilistic models}, author = {Walker, William I. and Soulat, Hugo and Yu, Changmin and Sahani, Maneesh}, booktitle = {Proceedings of The 26th International Conference on Artificial Intelligence and Statistics}, pages = {4209--4230}, year = {2023}, editor = {Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem}, volume = {206}, series = {Proceedings of Machine Learning Research}, month = {25--27 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v206/walker23a/walker23a.pdf}, url = {https://proceedings.mlr.press/v206/walker23a.html}, abstract = {We introduce a new approach to probabilistic unsupervised learning based on the recognition-parametrised model (RPM): a normalised semi-parametric hypothesis class for joint distributions over observed and latent variables. Under the key assumption that observations are conditionally independent given latents, the RPM combines parametric prior and observation-conditioned latent distributions with non-parametric observation marginals. This approach leads to a flexible learnt recognition model capturing latent dependence between observations, without the need for an explicit, parametric generative model. The RPM admits exact maximum-likelihood learning for discrete latents, even for powerful neural network-based recognition. We develop effective approximations applicable in the continuous latent case. Experiments demonstrate the effectiveness of the RPM on high-dimensional data, learning image classification from weak indirect supervision; direct image-level latent Dirichlet allocation; and Recognition-Parametrised Gaussian Process Factor Analysis (RP-GPFA) applied to multi-factorial spatiotemporal datasets. The RPM provides a powerful framework to discover meaningful latent structure underlying observational data, a function critical to both animal and artificial intelligence.} }
Endnote
%0 Conference Paper %T Unsupervised representation learning with recognition-parametrised probabilistic models %A William I. Walker %A Hugo Soulat %A Changmin Yu %A Maneesh Sahani %B Proceedings of The 26th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2023 %E Francisco Ruiz %E Jennifer Dy %E Jan-Willem van de Meent %F pmlr-v206-walker23a %I PMLR %P 4209--4230 %U https://proceedings.mlr.press/v206/walker23a.html %V 206 %X We introduce a new approach to probabilistic unsupervised learning based on the recognition-parametrised model (RPM): a normalised semi-parametric hypothesis class for joint distributions over observed and latent variables. Under the key assumption that observations are conditionally independent given latents, the RPM combines parametric prior and observation-conditioned latent distributions with non-parametric observation marginals. This approach leads to a flexible learnt recognition model capturing latent dependence between observations, without the need for an explicit, parametric generative model. The RPM admits exact maximum-likelihood learning for discrete latents, even for powerful neural network-based recognition. We develop effective approximations applicable in the continuous latent case. Experiments demonstrate the effectiveness of the RPM on high-dimensional data, learning image classification from weak indirect supervision; direct image-level latent Dirichlet allocation; and Recognition-Parametrised Gaussian Process Factor Analysis (RP-GPFA) applied to multi-factorial spatiotemporal datasets. The RPM provides a powerful framework to discover meaningful latent structure underlying observational data, a function critical to both animal and artificial intelligence.
APA
Walker, W.I., Soulat, H., Yu, C. & Sahani, M.. (2023). Unsupervised representation learning with recognition-parametrised probabilistic models. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 206:4209-4230 Available from https://proceedings.mlr.press/v206/walker23a.html.

Related Material