Message Passing Adaptive Resonance Theory for Online Active Semi-supervised Learning

Taehyeong Kim, Injune Hwang, Hyundo Lee, Hyunseo Kim, Won-Seok Choi, Joseph J Lim, Byoung-Tak Zhang
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:5519-5529, 2021.

Abstract

Active learning is widely used to reduce labeling effort and training time by repeatedly querying only the most beneficial samples from unlabeled data. In real-world problems where data cannot be stored indefinitely due to limited storage or privacy issues, the query selection and the model update should be performed as soon as a new data sample is observed. Various online active learning methods have been studied to deal with these challenges; however, there are difficulties in selecting representative query samples and updating the model efficiently without forgetting. In this study, we propose Message Passing Adaptive Resonance Theory (MPART) that learns the distribution and topology of input data online. Through message passing on the topological graph, MPART actively queries informative and representative samples, and continuously improves the classification performance using both labeled and unlabeled data. We evaluate our model in stream-based selective sampling scenarios with comparable query selection strategies, showing that MPART significantly outperforms competitive models.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-kim21e, title = {Message Passing Adaptive Resonance Theory for Online Active Semi-supervised Learning}, author = {Kim, Taehyeong and Hwang, Injune and Lee, Hyundo and Kim, Hyunseo and Choi, Won-Seok and Lim, Joseph J and Zhang, Byoung-Tak}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {5519--5529}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/kim21e/kim21e.pdf}, url = {https://proceedings.mlr.press/v139/kim21e.html}, abstract = {Active learning is widely used to reduce labeling effort and training time by repeatedly querying only the most beneficial samples from unlabeled data. In real-world problems where data cannot be stored indefinitely due to limited storage or privacy issues, the query selection and the model update should be performed as soon as a new data sample is observed. Various online active learning methods have been studied to deal with these challenges; however, there are difficulties in selecting representative query samples and updating the model efficiently without forgetting. In this study, we propose Message Passing Adaptive Resonance Theory (MPART) that learns the distribution and topology of input data online. Through message passing on the topological graph, MPART actively queries informative and representative samples, and continuously improves the classification performance using both labeled and unlabeled data. We evaluate our model in stream-based selective sampling scenarios with comparable query selection strategies, showing that MPART significantly outperforms competitive models.} }
Endnote
%0 Conference Paper %T Message Passing Adaptive Resonance Theory for Online Active Semi-supervised Learning %A Taehyeong Kim %A Injune Hwang %A Hyundo Lee %A Hyunseo Kim %A Won-Seok Choi %A Joseph J Lim %A Byoung-Tak Zhang %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-kim21e %I PMLR %P 5519--5529 %U https://proceedings.mlr.press/v139/kim21e.html %V 139 %X Active learning is widely used to reduce labeling effort and training time by repeatedly querying only the most beneficial samples from unlabeled data. In real-world problems where data cannot be stored indefinitely due to limited storage or privacy issues, the query selection and the model update should be performed as soon as a new data sample is observed. Various online active learning methods have been studied to deal with these challenges; however, there are difficulties in selecting representative query samples and updating the model efficiently without forgetting. In this study, we propose Message Passing Adaptive Resonance Theory (MPART) that learns the distribution and topology of input data online. Through message passing on the topological graph, MPART actively queries informative and representative samples, and continuously improves the classification performance using both labeled and unlabeled data. We evaluate our model in stream-based selective sampling scenarios with comparable query selection strategies, showing that MPART significantly outperforms competitive models.
APA
Kim, T., Hwang, I., Lee, H., Kim, H., Choi, W., Lim, J.J. & Zhang, B.. (2021). Message Passing Adaptive Resonance Theory for Online Active Semi-supervised Learning. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:5519-5529 Available from https://proceedings.mlr.press/v139/kim21e.html.

Related Material