Semi-Generative Modelling: Covariate-Shift Adaptation with Cause and Effect Features

Julius Kügelgen, Alexander Mey, Marco Loog
Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, PMLR 89:1361-1369, 2019.

Abstract

Current methods for covariate-shift adaptation use unlabelled data to compute importance weights or domain-invariant features, while the final model is trained on labelled data only. Here, we consider a particular case of covariate shift which allows us also to learn from unlabelled data, that is, combining adaptation and semi-supervised learning. Using ideas from causality, we argue that this requires learning with both causes, XC, and effects, XE, of a target variable, Y, and show how this setting leads to what we call a semi-generative model, P(Y,XE|XC,θ). Our approach is robust to domain shifts in the distribution of causal features and leverages unlabelled data by learning a direct map from causes to effects. Experiments on synthetic data demonstrate significant improvements in classification over purely-supervised and importance-weighting baselines.

Cite this Paper

BibTeX
@InProceedings{pmlr-v89-kugelgen19a, title = {Semi-Generative Modelling: Covariate-Shift Adaptation with Cause and Effect Features}, author = {von K\"{u}gelgen, Julius and Mey, Alexander and Loog, Marco}, booktitle = {Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics}, pages = {1361--1369}, year = {2019}, editor = {Chaudhuri, Kamalika and Sugiyama, Masashi}, volume = {89}, series = {Proceedings of Machine Learning Research}, month = {16--18 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v89/kugelgen19a/kugelgen19a.pdf}, url = {https://proceedings.mlr.press/v89/kugelgen19a.html}, abstract = {Current methods for covariate-shift adaptation use unlabelled data to compute importance weights or domain-invariant features, while the final model is trained on labelled data only. Here, we consider a particular case of covariate shift which allows us also to learn from unlabelled data, that is, combining adaptation and semi-supervised learning. Using ideas from causality, we argue that this requires learning with both causes, $X_C$, and effects, $X_E$, of a target variable, $Y$, and show how this setting leads to what we call a semi-generative model, $P(Y,X_E|X_C,\theta)$. Our approach is robust to domain shifts in the distribution of causal features and leverages unlabelled data by learning a direct map from causes to effects. Experiments on synthetic data demonstrate significant improvements in classification over purely-supervised and importance-weighting baselines.} }
Endnote
%0 Conference Paper %T Semi-Generative Modelling: Covariate-Shift Adaptation with Cause and Effect Features %A Julius Kügelgen %A Alexander Mey %A Marco Loog %B Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Masashi Sugiyama %F pmlr-v89-kugelgen19a %I PMLR %P 1361--1369 %U https://proceedings.mlr.press/v89/kugelgen19a.html %V 89 %X Current methods for covariate-shift adaptation use unlabelled data to compute importance weights or domain-invariant features, while the final model is trained on labelled data only. Here, we consider a particular case of covariate shift which allows us also to learn from unlabelled data, that is, combining adaptation and semi-supervised learning. Using ideas from causality, we argue that this requires learning with both causes, $X_C$, and effects, $X_E$, of a target variable, $Y$, and show how this setting leads to what we call a semi-generative model, $P(Y,X_E|X_C,\theta)$. Our approach is robust to domain shifts in the distribution of causal features and leverages unlabelled data by learning a direct map from causes to effects. Experiments on synthetic data demonstrate significant improvements in classification over purely-supervised and importance-weighting baselines.
APA
Kügelgen, J., Mey, A. & Loog, M.. (2019). Semi-Generative Modelling: Covariate-Shift Adaptation with Cause and Effect Features. Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 89:1361-1369 Available from https://proceedings.mlr.press/v89/kugelgen19a.html.

Related Material