Doubly Mixed-Effects Gaussian Process Regression

Jun Ho Yoon, Daniel P. Jeong, Seyoung Kim
Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, PMLR 151:6893-6908, 2022.

Abstract

We address the multi-task Gaussian process (GP) regression problem with the goal of decomposing input effects on outputs into components shared across or specific to tasks and samples. We propose a family of mixed-effects GPs, including doubly and translated mixed-effects GPs, that performs such a decomposition, while also modeling the complex task relationships. Instead of the tensor product widely used in multi-task GPs, we use the direct sum and Kronecker sum for Cartesian product to combine task and sample covariance functions. With this kernel, the overall input effects on outputs decompose into four components: fixed effects shared across tasks and across samples and random effects specific to each task and to each sample. We describe an efficient stochastic variational inference method for our proposed models that also significantly reduces the cost of inference for the existing mixed-effects GPs. On simulated and real-world data, we demonstrate that our approach provides higher test accuracy and interpretable decomposition.

Cite this Paper

BibTeX
@InProceedings{pmlr-v151-ho-yoon22a, title = { Doubly Mixed-Effects Gaussian Process Regression }, author = {Ho Yoon, Jun and Jeong, Daniel P. and Kim, Seyoung}, booktitle = {Proceedings of The 25th International Conference on Artificial Intelligence and Statistics}, pages = {6893--6908}, year = {2022}, editor = {Camps-Valls, Gustau and Ruiz, Francisco J. R. and Valera, Isabel}, volume = {151}, series = {Proceedings of Machine Learning Research}, month = {28--30 Mar}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v151/ho-yoon22a/ho-yoon22a.pdf}, url = {https://proceedings.mlr.press/v151/ho-yoon22a.html}, abstract = { We address the multi-task Gaussian process (GP) regression problem with the goal of decomposing input effects on outputs into components shared across or specific to tasks and samples. We propose a family of mixed-effects GPs, including doubly and translated mixed-effects GPs, that performs such a decomposition, while also modeling the complex task relationships. Instead of the tensor product widely used in multi-task GPs, we use the direct sum and Kronecker sum for Cartesian product to combine task and sample covariance functions. With this kernel, the overall input effects on outputs decompose into four components: fixed effects shared across tasks and across samples and random effects specific to each task and to each sample. We describe an efficient stochastic variational inference method for our proposed models that also significantly reduces the cost of inference for the existing mixed-effects GPs. On simulated and real-world data, we demonstrate that our approach provides higher test accuracy and interpretable decomposition. } }
Endnote
%0 Conference Paper %T Doubly Mixed-Effects Gaussian Process Regression %A Jun Ho Yoon %A Daniel P. Jeong %A Seyoung Kim %B Proceedings of The 25th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2022 %E Gustau Camps-Valls %E Francisco J. R. Ruiz %E Isabel Valera %F pmlr-v151-ho-yoon22a %I PMLR %P 6893--6908 %U https://proceedings.mlr.press/v151/ho-yoon22a.html %V 151 %X We address the multi-task Gaussian process (GP) regression problem with the goal of decomposing input effects on outputs into components shared across or specific to tasks and samples. We propose a family of mixed-effects GPs, including doubly and translated mixed-effects GPs, that performs such a decomposition, while also modeling the complex task relationships. Instead of the tensor product widely used in multi-task GPs, we use the direct sum and Kronecker sum for Cartesian product to combine task and sample covariance functions. With this kernel, the overall input effects on outputs decompose into four components: fixed effects shared across tasks and across samples and random effects specific to each task and to each sample. We describe an efficient stochastic variational inference method for our proposed models that also significantly reduces the cost of inference for the existing mixed-effects GPs. On simulated and real-world data, we demonstrate that our approach provides higher test accuracy and interpretable decomposition.
APA
Ho Yoon, J., Jeong, D.P. & Kim, S.. (2022). Doubly Mixed-Effects Gaussian Process Regression . Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 151:6893-6908 Available from https://proceedings.mlr.press/v151/ho-yoon22a.html.

Related Material