Bayesian Approaches to Distribution Regression

Ho Chung Leon Law, Danica J. Sutherland, Dino Sejdinovic, Seth Flaxman
Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, PMLR 84:1167-1176, 2018.

Abstract

Distribution regression has recently attracted much interest as a generic solution to the problem of supervised learning where labels are available at the group level, rather than at the individual level. Current approaches, however, do not propagate the uncertainty in observations due to sampling variability in the groups. This effectively assumes that small and large groups are estimated equally well, and should have equal weight in the final regression. We account for this uncertainty with a Bayesian distribution regression formalism, improving the robustness and performance of the model when group sizes vary. We frame our models in a neural network style, allowing for simple MAP inference using backpropagation to learn the parameters, as well as MCMC-based inference which can fully propagate uncertainty. We demonstrate our approach on illustrative toy datasets, as well as on a challenging problem of predicting age from images.

Cite this Paper

BibTeX
@InProceedings{pmlr-v84-law18a, title = {Bayesian Approaches to Distribution Regression}, author = {Law, Ho Chung Leon and Sutherland, Danica J. and Sejdinovic, Dino and Flaxman, Seth}, booktitle = {Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics}, pages = {1167--1176}, year = {2018}, editor = {Storkey, Amos and Perez-Cruz, Fernando}, volume = {84}, series = {Proceedings of Machine Learning Research}, month = {09--11 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v84/law18a/law18a.pdf}, url = {https://proceedings.mlr.press/v84/law18a.html}, abstract = {Distribution regression has recently attracted much interest as a generic solution to the problem of supervised learning where labels are available at the group level, rather than at the individual level. Current approaches, however, do not propagate the uncertainty in observations due to sampling variability in the groups. This effectively assumes that small and large groups are estimated equally well, and should have equal weight in the final regression. We account for this uncertainty with a Bayesian distribution regression formalism, improving the robustness and performance of the model when group sizes vary. We frame our models in a neural network style, allowing for simple MAP inference using backpropagation to learn the parameters, as well as MCMC-based inference which can fully propagate uncertainty. We demonstrate our approach on illustrative toy datasets, as well as on a challenging problem of predicting age from images.} }
Endnote
%0 Conference Paper %T Bayesian Approaches to Distribution Regression %A Ho Chung Leon Law %A Danica J. Sutherland %A Dino Sejdinovic %A Seth Flaxman %B Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2018 %E Amos Storkey %E Fernando Perez-Cruz %F pmlr-v84-law18a %I PMLR %P 1167--1176 %U https://proceedings.mlr.press/v84/law18a.html %V 84 %X Distribution regression has recently attracted much interest as a generic solution to the problem of supervised learning where labels are available at the group level, rather than at the individual level. Current approaches, however, do not propagate the uncertainty in observations due to sampling variability in the groups. This effectively assumes that small and large groups are estimated equally well, and should have equal weight in the final regression. We account for this uncertainty with a Bayesian distribution regression formalism, improving the robustness and performance of the model when group sizes vary. We frame our models in a neural network style, allowing for simple MAP inference using backpropagation to learn the parameters, as well as MCMC-based inference which can fully propagate uncertainty. We demonstrate our approach on illustrative toy datasets, as well as on a challenging problem of predicting age from images.
APA
Law, H.C.L., Sutherland, D.J., Sejdinovic, D. & Flaxman, S.. (2018). Bayesian Approaches to Distribution Regression. Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 84:1167-1176 Available from https://proceedings.mlr.press/v84/law18a.html.

Related Material