Covariate Shift in Hilbert Space: A Solution via Sorrogate Kernels

Kai Zhang, Vincent Zheng, Qiaojun Wang, James Kwok, Qiang Yang, Ivan Marsic
Proceedings of the 30th International Conference on Machine Learning, PMLR 28(3):388-395, 2013.

Abstract

Covariate shift is a unconventional learning scenario in which training and testing data have different distributions. A general principle to solve the problem is to make the training data distribution similar to the test one, such that classifiers computed on the former generalizes well to the latter. Current approaches typically target on the sample distribution in the input space, however, for kernel-based learning methods, the algorithm performance depends directly on the geometry of the kernel-induced feature space. Motivated by this, we propose to match data distributions in the Hilbert space, which, given a pre-defined empirical kernel map, can be formulated as aligning kernel matrices across domains. In particular, to evaluate similarity of kernel matrices defined on arbitrarily different samples, the novel concept of surrogate kernel is introduced based on the Mercer's theorem. Our approach caters the model adaptation specifically to kernel-based learning mechanism, and demonstrates promising results on several real-world applications.

Cite this Paper

BibTeX
@InProceedings{pmlr-v28-zhang13b, title = {Covariate Shift in Hilbert Space: A Solution via Sorrogate Kernels}, author = {Zhang, Kai and Zheng, Vincent and Wang, Qiaojun and Kwok, James and Yang, Qiang and Marsic, Ivan}, booktitle = {Proceedings of the 30th International Conference on Machine Learning}, pages = {388--395}, year = {2013}, editor = {Dasgupta, Sanjoy and McAllester, David}, volume = {28}, number = {3}, series = {Proceedings of Machine Learning Research}, address = {Atlanta, Georgia, USA}, month = {17--19 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v28/zhang13b.pdf}, url = {https://proceedings.mlr.press/v28/zhang13b.html}, abstract = {Covariate shift is a unconventional learning scenario in which training and testing data have different distributions. A general principle to solve the problem is to make the training data distribution similar to the test one, such that classifiers computed on the former generalizes well to the latter. Current approaches typically target on the sample distribution in the input space, however, for kernel-based learning methods, the algorithm performance depends directly on the geometry of the kernel-induced feature space. Motivated by this, we propose to match data distributions in the Hilbert space, which, given a pre-defined empirical kernel map, can be formulated as aligning kernel matrices across domains. In particular, to evaluate similarity of kernel matrices defined on arbitrarily different samples, the novel concept of surrogate kernel is introduced based on the Mercer's theorem. Our approach caters the model adaptation specifically to kernel-based learning mechanism, and demonstrates promising results on several real-world applications.} }
Endnote
%0 Conference Paper %T Covariate Shift in Hilbert Space: A Solution via Sorrogate Kernels %A Kai Zhang %A Vincent Zheng %A Qiaojun Wang %A James Kwok %A Qiang Yang %A Ivan Marsic %B Proceedings of the 30th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2013 %E Sanjoy Dasgupta %E David McAllester %F pmlr-v28-zhang13b %I PMLR %P 388--395 %U https://proceedings.mlr.press/v28/zhang13b.html %V 28 %N 3 %X Covariate shift is a unconventional learning scenario in which training and testing data have different distributions. A general principle to solve the problem is to make the training data distribution similar to the test one, such that classifiers computed on the former generalizes well to the latter. Current approaches typically target on the sample distribution in the input space, however, for kernel-based learning methods, the algorithm performance depends directly on the geometry of the kernel-induced feature space. Motivated by this, we propose to match data distributions in the Hilbert space, which, given a pre-defined empirical kernel map, can be formulated as aligning kernel matrices across domains. In particular, to evaluate similarity of kernel matrices defined on arbitrarily different samples, the novel concept of surrogate kernel is introduced based on the Mercer's theorem. Our approach caters the model adaptation specifically to kernel-based learning mechanism, and demonstrates promising results on several real-world applications.
RIS
TY - CPAPER TI - Covariate Shift in Hilbert Space: A Solution via Sorrogate Kernels AU - Kai Zhang AU - Vincent Zheng AU - Qiaojun Wang AU - James Kwok AU - Qiang Yang AU - Ivan Marsic BT - Proceedings of the 30th International Conference on Machine Learning DA - 2013/05/26 ED - Sanjoy Dasgupta ED - David McAllester ID - pmlr-v28-zhang13b PB - PMLR DP - Proceedings of Machine Learning Research VL - 28 IS - 3 SP - 388 EP - 395 L1 - http://proceedings.mlr.press/v28/zhang13b.pdf UR - https://proceedings.mlr.press/v28/zhang13b.html AB - Covariate shift is a unconventional learning scenario in which training and testing data have different distributions. A general principle to solve the problem is to make the training data distribution similar to the test one, such that classifiers computed on the former generalizes well to the latter. Current approaches typically target on the sample distribution in the input space, however, for kernel-based learning methods, the algorithm performance depends directly on the geometry of the kernel-induced feature space. Motivated by this, we propose to match data distributions in the Hilbert space, which, given a pre-defined empirical kernel map, can be formulated as aligning kernel matrices across domains. In particular, to evaluate similarity of kernel matrices defined on arbitrarily different samples, the novel concept of surrogate kernel is introduced based on the Mercer's theorem. Our approach caters the model adaptation specifically to kernel-based learning mechanism, and demonstrates promising results on several real-world applications. ER -
APA
Zhang, K., Zheng, V., Wang, Q., Kwok, J., Yang, Q. & Marsic, I.. (2013). Covariate Shift in Hilbert Space: A Solution via Sorrogate Kernels. Proceedings of the 30th International Conference on Machine Learning, in Proceedings of Machine Learning Research 28(3):388-395 Available from https://proceedings.mlr.press/v28/zhang13b.html.

Related Material