Source-Target Similarity Modelings for Multi-Source Transfer Gaussian Process Regression

Pengfei Wei, Ramon Sagarna, Yiping Ke, Yew-Soon Ong, Chi-Keong Goh
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:3722-3731, 2017.

Abstract

A key challenge in multi-source transfer learning is to capture the diverse inter-domain similarities. In this paper, we study different approaches based on Gaussian process models to solve the multi-source transfer regression problem. Precisely, we first investigate the feasibility and performance of a family of transfer covariance functions that represent the pairwise similarity of each source and the target domain. We theoretically show that using such a transfer covariance function for general Gaussian process modelling can only capture the same similarity coefficient for all the sources, and thus may result in unsatisfactory transfer performance. This leads us to propose TC$_{MS}$Stack, an integrated strategy incorporating the benefits of the transfer covariance function and stacking. Extensive experiments on one synthetic and two real-world datasets, with learning settings of up to 11 sources for the latter, demonstrate the effectiveness of our proposed TC$_{MS}$Stack.

Cite this Paper

BibTeX
@InProceedings{pmlr-v70-wei17a, title = {Source-Target Similarity Modelings for Multi-Source Transfer {G}aussian Process Regression}, author = {Pengfei Wei and Ramon Sagarna and Yiping Ke and Yew-Soon Ong and Chi-Keong Goh}, booktitle = {Proceedings of the 34th International Conference on Machine Learning}, pages = {3722--3731}, year = {2017}, editor = {Precup, Doina and Teh, Yee Whye}, volume = {70}, series = {Proceedings of Machine Learning Research}, month = {06--11 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v70/wei17a/wei17a.pdf}, url = {https://proceedings.mlr.press/v70/wei17a.html}, abstract = {A key challenge in multi-source transfer learning is to capture the diverse inter-domain similarities. In this paper, we study different approaches based on Gaussian process models to solve the multi-source transfer regression problem. Precisely, we first investigate the feasibility and performance of a family of transfer covariance functions that represent the pairwise similarity of each source and the target domain. We theoretically show that using such a transfer covariance function for general Gaussian process modelling can only capture the same similarity coefficient for all the sources, and thus may result in unsatisfactory transfer performance. This leads us to propose TC$_{MS}$Stack, an integrated strategy incorporating the benefits of the transfer covariance function and stacking. Extensive experiments on one synthetic and two real-world datasets, with learning settings of up to 11 sources for the latter, demonstrate the effectiveness of our proposed TC$_{MS}$Stack.} }
Endnote
%0 Conference Paper %T Source-Target Similarity Modelings for Multi-Source Transfer Gaussian Process Regression %A Pengfei Wei %A Ramon Sagarna %A Yiping Ke %A Yew-Soon Ong %A Chi-Keong Goh %B Proceedings of the 34th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2017 %E Doina Precup %E Yee Whye Teh %F pmlr-v70-wei17a %I PMLR %P 3722--3731 %U https://proceedings.mlr.press/v70/wei17a.html %V 70 %X A key challenge in multi-source transfer learning is to capture the diverse inter-domain similarities. In this paper, we study different approaches based on Gaussian process models to solve the multi-source transfer regression problem. Precisely, we first investigate the feasibility and performance of a family of transfer covariance functions that represent the pairwise similarity of each source and the target domain. We theoretically show that using such a transfer covariance function for general Gaussian process modelling can only capture the same similarity coefficient for all the sources, and thus may result in unsatisfactory transfer performance. This leads us to propose TC$_{MS}$Stack, an integrated strategy incorporating the benefits of the transfer covariance function and stacking. Extensive experiments on one synthetic and two real-world datasets, with learning settings of up to 11 sources for the latter, demonstrate the effectiveness of our proposed TC$_{MS}$Stack.
APA
Wei, P., Sagarna, R., Ke, Y., Ong, Y. & Goh, C.. (2017). Source-Target Similarity Modelings for Multi-Source Transfer Gaussian Process Regression. Proceedings of the 34th International Conference on Machine Learning, in Proceedings of Machine Learning Research 70:3722-3731 Available from https://proceedings.mlr.press/v70/wei17a.html.

Related Material