Semi-supervised learning, causality, and the conditional cluster assumption

Julius Kügelgen, Alexander Mey, Marco Loog, Bernhard Schölkopf
Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI), PMLR 124:1-10, 2020.

Abstract

While the success of semi-supervised learning (SSL) is still not fully understood, Schölkopf et al. (2012) have established a link to the principle of independent causal mechanisms. They conclude that SSL should be impossible when predicting a target variable from its causes, but possible when predicting it from its effects. Since both these cases are restrictive, we extend their work by considering classification using cause and effect features at the same time, such as predicting a disease from both risk factors and symptoms. While standard SSL exploits information contained in the marginal distribution of all inputs (to improve the estimate of the conditional distribution of the target given in-puts), we argue that in our more general setting we should use information in the conditional distribution of effect features given causal features. We explore how this insight generalises the previous understanding, and how it relates to and can be exploited algorithmically for SSL.

Cite this Paper

BibTeX
@InProceedings{pmlr-v124-kugelgen20a, title = {Semi-supervised learning, causality, and the conditional cluster assumption}, author = {von K\"{u}gelgen, Julius and Mey, Alexander and Loog, Marco and Sch\"{o}lkopf, Bernhard}, booktitle = {Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI)}, pages = {1--10}, year = {2020}, editor = {Peters, Jonas and Sontag, David}, volume = {124}, series = {Proceedings of Machine Learning Research}, month = {03--06 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v124/kugelgen20a/kugelgen20a.pdf}, url = {https://proceedings.mlr.press/v124/kugelgen20a.html}, abstract = {While the success of semi-supervised learning (SSL) is still not fully understood, Schölkopf et al. (2012) have established a link to the principle of independent causal mechanisms. They conclude that SSL should be impossible when predicting a target variable from its causes, but possible when predicting it from its effects. Since both these cases are restrictive, we extend their work by considering classification using cause and effect features at the same time, such as predicting a disease from both risk factors and symptoms. While standard SSL exploits information contained in the marginal distribution of all inputs (to improve the estimate of the conditional distribution of the target given in-puts), we argue that in our more general setting we should use information in the conditional distribution of effect features given causal features. We explore how this insight generalises the previous understanding, and how it relates to and can be exploited algorithmically for SSL.} }
Endnote
%0 Conference Paper %T Semi-supervised learning, causality, and the conditional cluster assumption %A Julius Kügelgen %A Alexander Mey %A Marco Loog %A Bernhard Schölkopf %B Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI) %C Proceedings of Machine Learning Research %D 2020 %E Jonas Peters %E David Sontag %F pmlr-v124-kugelgen20a %I PMLR %P 1--10 %U https://proceedings.mlr.press/v124/kugelgen20a.html %V 124 %X While the success of semi-supervised learning (SSL) is still not fully understood, Schölkopf et al. (2012) have established a link to the principle of independent causal mechanisms. They conclude that SSL should be impossible when predicting a target variable from its causes, but possible when predicting it from its effects. Since both these cases are restrictive, we extend their work by considering classification using cause and effect features at the same time, such as predicting a disease from both risk factors and symptoms. While standard SSL exploits information contained in the marginal distribution of all inputs (to improve the estimate of the conditional distribution of the target given in-puts), we argue that in our more general setting we should use information in the conditional distribution of effect features given causal features. We explore how this insight generalises the previous understanding, and how it relates to and can be exploited algorithmically for SSL.
APA
Kügelgen, J., Mey, A., Loog, M. & Schölkopf, B.. (2020). Semi-supervised learning, causality, and the conditional cluster assumption. Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI), in Proceedings of Machine Learning Research 124:1-10 Available from https://proceedings.mlr.press/v124/kugelgen20a.html.

Related Material