Bayesian Deconditional Kernel Mean Embeddings

Kelvin Hsu, Fabio Ramos
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:2830-2838, 2019.

Abstract

Conditional kernel mean embeddings form an attractive nonparametric framework for representing conditional means of functions, describing the observation processes for many complex models. However, the recovery of the original underlying function of interest whose conditional mean was observed is a challenging inference task. We formalize deconditional kernel mean embeddings as a solution to this inverse problem, and show that it can be naturally viewed as a nonparametric Bayes' rule. Critically, we introduce the notion of task transformed Gaussian processes and establish deconditional kernel means as their posterior predictive mean. This connection provides Bayesian interpretations and uncertainty estimates for deconditional kernel mean embeddings, explains their regularization hyperparameters, and reveals a marginal likelihood for kernel hyperparameter learning. These revelations further enable practical applications such as likelihood-free inference and learning sparse representations for big data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-hsu19a, title = {{B}ayesian Deconditional Kernel Mean Embeddings}, author = {Hsu, Kelvin and Ramos, Fabio}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {2830--2838}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/hsu19a/hsu19a.pdf}, url = {https://proceedings.mlr.press/v97/hsu19a.html}, abstract = {Conditional kernel mean embeddings form an attractive nonparametric framework for representing conditional means of functions, describing the observation processes for many complex models. However, the recovery of the original underlying function of interest whose conditional mean was observed is a challenging inference task. We formalize deconditional kernel mean embeddings as a solution to this inverse problem, and show that it can be naturally viewed as a nonparametric Bayes' rule. Critically, we introduce the notion of task transformed Gaussian processes and establish deconditional kernel means as their posterior predictive mean. This connection provides Bayesian interpretations and uncertainty estimates for deconditional kernel mean embeddings, explains their regularization hyperparameters, and reveals a marginal likelihood for kernel hyperparameter learning. These revelations further enable practical applications such as likelihood-free inference and learning sparse representations for big data.} }
Endnote
%0 Conference Paper %T Bayesian Deconditional Kernel Mean Embeddings %A Kelvin Hsu %A Fabio Ramos %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-hsu19a %I PMLR %P 2830--2838 %U https://proceedings.mlr.press/v97/hsu19a.html %V 97 %X Conditional kernel mean embeddings form an attractive nonparametric framework for representing conditional means of functions, describing the observation processes for many complex models. However, the recovery of the original underlying function of interest whose conditional mean was observed is a challenging inference task. We formalize deconditional kernel mean embeddings as a solution to this inverse problem, and show that it can be naturally viewed as a nonparametric Bayes' rule. Critically, we introduce the notion of task transformed Gaussian processes and establish deconditional kernel means as their posterior predictive mean. This connection provides Bayesian interpretations and uncertainty estimates for deconditional kernel mean embeddings, explains their regularization hyperparameters, and reveals a marginal likelihood for kernel hyperparameter learning. These revelations further enable practical applications such as likelihood-free inference and learning sparse representations for big data.
APA
Hsu, K. & Ramos, F.. (2019). Bayesian Deconditional Kernel Mean Embeddings. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:2830-2838 Available from https://proceedings.mlr.press/v97/hsu19a.html.

Related Material