Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics, PMLR 33:877-885, 2014.
Abstract
We investigate the Student-t process as an alternative to the Gaussian process as a nonparametric prior over functions. We derive closed form expressions for the marginal likelihood and predictive distribution of a Student-t process, by integrating away an inverse Wishart process prior over the covariance kernel of a Gaussian process model. We show surprising equivalences between different hierarchical Gaussian process models leading to Student-t processes, and derive a new sampling scheme for the inverse Wishart process, which helps elucidate these equivalences. Overall, we show that a Student-t process can retain the attractive properties of a Gaussian process – a nonparametric representation, analytic marginal and predictive distributions, and easy model selection through covariance kernels – but has enhanced flexibility, and a predictive covariance that, unlike a Gaussian process, explicitly depends on the values of training observations. We verify empirically that a Student-t process is especially useful in situations where there are changes in covariance structure, or in applications like Bayesian optimization, where accurate predictive covariances are critical for good performance. These advantages come at no additional computational cost over Gaussian processes.
@InProceedings{pmlr-v33-shah14,
title = {{Student-t Processes as Alternatives to Gaussian Processes}},
author = {Amar Shah and Andrew Wilson and Zoubin Ghahramani},
booktitle = {Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics},
pages = {877--885},
year = {2014},
editor = {Samuel Kaski and Jukka Corander},
volume = {33},
series = {Proceedings of Machine Learning Research},
address = {Reykjavik, Iceland},
month = {22--25 Apr},
publisher = {PMLR},
pdf = {http://proceedings.mlr.press/v33/shah14.pdf},
url = {http://proceedings.mlr.press/v33/shah14.html},
abstract = {We investigate the Student-t process as an alternative to the Gaussian process as a nonparametric prior over functions. We derive closed form expressions for the marginal likelihood and predictive distribution of a Student-t process, by integrating away an inverse Wishart process prior over the covariance kernel of a Gaussian process model. We show surprising equivalences between different hierarchical Gaussian process models leading to Student-t processes, and derive a new sampling scheme for the inverse Wishart process, which helps elucidate these equivalences. Overall, we show that a Student-t process can retain the attractive properties of a Gaussian process – a nonparametric representation, analytic marginal and predictive distributions, and easy model selection through covariance kernels – but has enhanced flexibility, and a predictive covariance that, unlike a Gaussian process, explicitly depends on the values of training observations. We verify empirically that a Student-t process is especially useful in situations where there are changes in covariance structure, or in applications like Bayesian optimization, where accurate predictive covariances are critical for good performance. These advantages come at no additional computational cost over Gaussian processes.}
}
%0 Conference Paper
%T Student-t Processes as Alternatives to Gaussian Processes
%A Amar Shah
%A Andrew Wilson
%A Zoubin Ghahramani
%B Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics
%C Proceedings of Machine Learning Research
%D 2014
%E Samuel Kaski
%E Jukka Corander
%F pmlr-v33-shah14
%I PMLR
%J Proceedings of Machine Learning Research
%P 877--885
%U http://proceedings.mlr.press
%V 33
%W PMLR
%X We investigate the Student-t process as an alternative to the Gaussian process as a nonparametric prior over functions. We derive closed form expressions for the marginal likelihood and predictive distribution of a Student-t process, by integrating away an inverse Wishart process prior over the covariance kernel of a Gaussian process model. We show surprising equivalences between different hierarchical Gaussian process models leading to Student-t processes, and derive a new sampling scheme for the inverse Wishart process, which helps elucidate these equivalences. Overall, we show that a Student-t process can retain the attractive properties of a Gaussian process – a nonparametric representation, analytic marginal and predictive distributions, and easy model selection through covariance kernels – but has enhanced flexibility, and a predictive covariance that, unlike a Gaussian process, explicitly depends on the values of training observations. We verify empirically that a Student-t process is especially useful in situations where there are changes in covariance structure, or in applications like Bayesian optimization, where accurate predictive covariances are critical for good performance. These advantages come at no additional computational cost over Gaussian processes.
TY - CPAPER
TI - Student-t Processes as Alternatives to Gaussian Processes
AU - Amar Shah
AU - Andrew Wilson
AU - Zoubin Ghahramani
BT - Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics
PY - 2014/04/02
DA - 2014/04/02
ED - Samuel Kaski
ED - Jukka Corander
ID - pmlr-v33-shah14
PB - PMLR
SP - 877
DP - PMLR
EP - 885
L1 - http://proceedings.mlr.press/v33/shah14.pdf
UR - http://proceedings.mlr.press/v33/shah14.html
AB - We investigate the Student-t process as an alternative to the Gaussian process as a nonparametric prior over functions. We derive closed form expressions for the marginal likelihood and predictive distribution of a Student-t process, by integrating away an inverse Wishart process prior over the covariance kernel of a Gaussian process model. We show surprising equivalences between different hierarchical Gaussian process models leading to Student-t processes, and derive a new sampling scheme for the inverse Wishart process, which helps elucidate these equivalences. Overall, we show that a Student-t process can retain the attractive properties of a Gaussian process – a nonparametric representation, analytic marginal and predictive distributions, and easy model selection through covariance kernels – but has enhanced flexibility, and a predictive covariance that, unlike a Gaussian process, explicitly depends on the values of training observations. We verify empirically that a Student-t process is especially useful in situations where there are changes in covariance structure, or in applications like Bayesian optimization, where accurate predictive covariances are critical for good performance. These advantages come at no additional computational cost over Gaussian processes.
ER -
Shah, A., Wilson, A. & Ghahramani, Z.. (2014). Student-t Processes as Alternatives to Gaussian Processes. Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics, in PMLR 33:877-885
This site last compiled Mon, 16 Jul 2018 07:45:17 +0000