Scalable Gaussian Process Structured Prediction for Grid Factor Graph Applications

Sebastien Bratieres, Novi Quadrianto, Sebastian Nowozin, Zoubin Ghahramani
Proceedings of the 31st International Conference on Machine Learning, PMLR 32(2):334-342, 2014.

Abstract

Structured prediction is an important and well studied problem with many applications across machine learning. GPstruct is a recently proposed structured prediction model that offers appealing properties such as being kernelised, non-parametric, and supporting Bayesian inference (Bratières et al. 2013). The model places a Gaussian process prior over energy functions which describe relationships between input variables and structured output variables. However, the memory demand of GPstruct is quadratic in the number of latent variables and training runtime scales cubically. This prevents GPstruct from being applied to problems involving grid factor graphs, which are prevalent in computer vision and spatial statistics applications. Here we explore a scalable approach to learning GPstruct models based on ensemble learning, with weak learners (predictors) trained on subsets of the latent variables and bootstrap data, which can easily be distributed. We show experiments with 4M latent variables on image segmentation. Our method outperforms widely-used conditional random field models trained with pseudo-likelihood. Moreover, in image segmentation problems it improves over recent state-of-the-art marginal optimisation methods in terms of predictive performance and uncertainty calibration. Finally, it generalises well on all training set sizes.

Cite this Paper

BibTeX
@InProceedings{pmlr-v32-bratieres14, title = {Scalable Gaussian Process Structured Prediction for Grid Factor Graph Applications}, author = {Bratieres, Sebastien and Quadrianto, Novi and Nowozin, Sebastian and Ghahramani, Zoubin}, booktitle = {Proceedings of the 31st International Conference on Machine Learning}, pages = {334--342}, year = {2014}, editor = {Xing, Eric P. and Jebara, Tony}, volume = {32}, number = {2}, series = {Proceedings of Machine Learning Research}, address = {Bejing, China}, month = {22--24 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v32/bratieres14.pdf}, url = {https://proceedings.mlr.press/v32/bratieres14.html}, abstract = {Structured prediction is an important and well studied problem with many applications across machine learning. GPstruct is a recently proposed structured prediction model that offers appealing properties such as being kernelised, non-parametric, and supporting Bayesian inference (Bratières et al. 2013). The model places a Gaussian process prior over energy functions which describe relationships between input variables and structured output variables. However, the memory demand of GPstruct is quadratic in the number of latent variables and training runtime scales cubically. This prevents GPstruct from being applied to problems involving grid factor graphs, which are prevalent in computer vision and spatial statistics applications. Here we explore a scalable approach to learning GPstruct models based on ensemble learning, with weak learners (predictors) trained on subsets of the latent variables and bootstrap data, which can easily be distributed. We show experiments with 4M latent variables on image segmentation. Our method outperforms widely-used conditional random field models trained with pseudo-likelihood. Moreover, in image segmentation problems it improves over recent state-of-the-art marginal optimisation methods in terms of predictive performance and uncertainty calibration. Finally, it generalises well on all training set sizes.} }
Endnote
%0 Conference Paper %T Scalable Gaussian Process Structured Prediction for Grid Factor Graph Applications %A Sebastien Bratieres %A Novi Quadrianto %A Sebastian Nowozin %A Zoubin Ghahramani %B Proceedings of the 31st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2014 %E Eric P. Xing %E Tony Jebara %F pmlr-v32-bratieres14 %I PMLR %P 334--342 %U https://proceedings.mlr.press/v32/bratieres14.html %V 32 %N 2 %X Structured prediction is an important and well studied problem with many applications across machine learning. GPstruct is a recently proposed structured prediction model that offers appealing properties such as being kernelised, non-parametric, and supporting Bayesian inference (Bratières et al. 2013). The model places a Gaussian process prior over energy functions which describe relationships between input variables and structured output variables. However, the memory demand of GPstruct is quadratic in the number of latent variables and training runtime scales cubically. This prevents GPstruct from being applied to problems involving grid factor graphs, which are prevalent in computer vision and spatial statistics applications. Here we explore a scalable approach to learning GPstruct models based on ensemble learning, with weak learners (predictors) trained on subsets of the latent variables and bootstrap data, which can easily be distributed. We show experiments with 4M latent variables on image segmentation. Our method outperforms widely-used conditional random field models trained with pseudo-likelihood. Moreover, in image segmentation problems it improves over recent state-of-the-art marginal optimisation methods in terms of predictive performance and uncertainty calibration. Finally, it generalises well on all training set sizes.
RIS
TY - CPAPER TI - Scalable Gaussian Process Structured Prediction for Grid Factor Graph Applications AU - Sebastien Bratieres AU - Novi Quadrianto AU - Sebastian Nowozin AU - Zoubin Ghahramani BT - Proceedings of the 31st International Conference on Machine Learning DA - 2014/06/18 ED - Eric P. Xing ED - Tony Jebara ID - pmlr-v32-bratieres14 PB - PMLR DP - Proceedings of Machine Learning Research VL - 32 IS - 2 SP - 334 EP - 342 L1 - http://proceedings.mlr.press/v32/bratieres14.pdf UR - https://proceedings.mlr.press/v32/bratieres14.html AB - Structured prediction is an important and well studied problem with many applications across machine learning. GPstruct is a recently proposed structured prediction model that offers appealing properties such as being kernelised, non-parametric, and supporting Bayesian inference (Bratières et al. 2013). The model places a Gaussian process prior over energy functions which describe relationships between input variables and structured output variables. However, the memory demand of GPstruct is quadratic in the number of latent variables and training runtime scales cubically. This prevents GPstruct from being applied to problems involving grid factor graphs, which are prevalent in computer vision and spatial statistics applications. Here we explore a scalable approach to learning GPstruct models based on ensemble learning, with weak learners (predictors) trained on subsets of the latent variables and bootstrap data, which can easily be distributed. We show experiments with 4M latent variables on image segmentation. Our method outperforms widely-used conditional random field models trained with pseudo-likelihood. Moreover, in image segmentation problems it improves over recent state-of-the-art marginal optimisation methods in terms of predictive performance and uncertainty calibration. Finally, it generalises well on all training set sizes. ER -
APA
Bratieres, S., Quadrianto, N., Nowozin, S. & Ghahramani, Z.. (2014). Scalable Gaussian Process Structured Prediction for Grid Factor Graph Applications. Proceedings of the 31st International Conference on Machine Learning, in Proceedings of Machine Learning Research 32(2):334-342 Available from https://proceedings.mlr.press/v32/bratieres14.html.

Related Material