Semi-Supervised Learning via Compact Latent Space Clustering

Konstantinos Kamnitsas, Daniel Castro, Loic Le Folgoc, Ian Walker, Ryutaro Tanno, Daniel Rueckert, Ben Glocker, Antonio Criminisi, Aditya Nori
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:2459-2468, 2018.

Abstract

We present a novel cost function for semi-supervised learning of neural networks that encourages compact clustering of the latent space to facilitate separation. The key idea is to dynamically create a graph over embeddings of labeled and unlabeled samples of a training batch to capture underlying structure in feature space, and use label propagation to estimate its high and low density regions. We then devise a cost function based on Markov chains on the graph that regularizes the latent space to form a single compact cluster per class, while avoiding to disturb existing clusters during optimization. We evaluate our approach on three benchmarks and compare to state-of-the art with promising results. Our approach combines the benefits of graph-based regularization with efficient, inductive inference, does not require modifications to a network architecture, and can thus be easily applied to existing networks to enable an effective use of unlabeled data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v80-kamnitsas18a, title = {Semi-Supervised Learning via Compact Latent Space Clustering}, author = {Kamnitsas, Konstantinos and Castro, Daniel and Folgoc, Loic Le and Walker, Ian and Tanno, Ryutaro and Rueckert, Daniel and Glocker, Ben and Criminisi, Antonio and Nori, Aditya}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {2459--2468}, year = {2018}, editor = {Dy, Jennifer and Krause, Andreas}, volume = {80}, series = {Proceedings of Machine Learning Research}, month = {10--15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v80/kamnitsas18a/kamnitsas18a.pdf}, url = {https://proceedings.mlr.press/v80/kamnitsas18a.html}, abstract = {We present a novel cost function for semi-supervised learning of neural networks that encourages compact clustering of the latent space to facilitate separation. The key idea is to dynamically create a graph over embeddings of labeled and unlabeled samples of a training batch to capture underlying structure in feature space, and use label propagation to estimate its high and low density regions. We then devise a cost function based on Markov chains on the graph that regularizes the latent space to form a single compact cluster per class, while avoiding to disturb existing clusters during optimization. We evaluate our approach on three benchmarks and compare to state-of-the art with promising results. Our approach combines the benefits of graph-based regularization with efficient, inductive inference, does not require modifications to a network architecture, and can thus be easily applied to existing networks to enable an effective use of unlabeled data.} }
Endnote
%0 Conference Paper %T Semi-Supervised Learning via Compact Latent Space Clustering %A Konstantinos Kamnitsas %A Daniel Castro %A Loic Le Folgoc %A Ian Walker %A Ryutaro Tanno %A Daniel Rueckert %A Ben Glocker %A Antonio Criminisi %A Aditya Nori %B Proceedings of the 35th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jennifer Dy %E Andreas Krause %F pmlr-v80-kamnitsas18a %I PMLR %P 2459--2468 %U https://proceedings.mlr.press/v80/kamnitsas18a.html %V 80 %X We present a novel cost function for semi-supervised learning of neural networks that encourages compact clustering of the latent space to facilitate separation. The key idea is to dynamically create a graph over embeddings of labeled and unlabeled samples of a training batch to capture underlying structure in feature space, and use label propagation to estimate its high and low density regions. We then devise a cost function based on Markov chains on the graph that regularizes the latent space to form a single compact cluster per class, while avoiding to disturb existing clusters during optimization. We evaluate our approach on three benchmarks and compare to state-of-the art with promising results. Our approach combines the benefits of graph-based regularization with efficient, inductive inference, does not require modifications to a network architecture, and can thus be easily applied to existing networks to enable an effective use of unlabeled data.
APA
Kamnitsas, K., Castro, D., Folgoc, L.L., Walker, I., Tanno, R., Rueckert, D., Glocker, B., Criminisi, A. & Nori, A.. (2018). Semi-Supervised Learning via Compact Latent Space Clustering. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research 80:2459-2468 Available from https://proceedings.mlr.press/v80/kamnitsas18a.html.

Related Material