Adaptive Robust Learning using Latent Bernoulli Variables

Aleksandr Karakulev, Dave Zachariah, Prashant Singh
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:23105-23122, 2024.

Abstract

We present an adaptive approach for robust learning from corrupted training sets. We identify corrupted and non-corrupted samples with latent Bernoulli variables and thus formulate the learning problem as maximization of the likelihood where latent variables are marginalized. The resulting problem is solved via variational inference, using an efficient Expectation-Maximization based method. The proposed approach improves over the state-of-the-art by automatically inferring the corruption level, while adding minimal computational overhead. We demonstrate our robust learning method and its parameter-free nature on a wide variety of machine learning tasks including online learning and deep learning where it adapts to different levels of noise and maintains high prediction accuracy.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-karakulev24a, title = {Adaptive Robust Learning using Latent Bernoulli Variables}, author = {Karakulev, Aleksandr and Zachariah, Dave and Singh, Prashant}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {23105--23122}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/karakulev24a/karakulev24a.pdf}, url = {https://proceedings.mlr.press/v235/karakulev24a.html}, abstract = {We present an adaptive approach for robust learning from corrupted training sets. We identify corrupted and non-corrupted samples with latent Bernoulli variables and thus formulate the learning problem as maximization of the likelihood where latent variables are marginalized. The resulting problem is solved via variational inference, using an efficient Expectation-Maximization based method. The proposed approach improves over the state-of-the-art by automatically inferring the corruption level, while adding minimal computational overhead. We demonstrate our robust learning method and its parameter-free nature on a wide variety of machine learning tasks including online learning and deep learning where it adapts to different levels of noise and maintains high prediction accuracy.} }
Endnote
%0 Conference Paper %T Adaptive Robust Learning using Latent Bernoulli Variables %A Aleksandr Karakulev %A Dave Zachariah %A Prashant Singh %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-karakulev24a %I PMLR %P 23105--23122 %U https://proceedings.mlr.press/v235/karakulev24a.html %V 235 %X We present an adaptive approach for robust learning from corrupted training sets. We identify corrupted and non-corrupted samples with latent Bernoulli variables and thus formulate the learning problem as maximization of the likelihood where latent variables are marginalized. The resulting problem is solved via variational inference, using an efficient Expectation-Maximization based method. The proposed approach improves over the state-of-the-art by automatically inferring the corruption level, while adding minimal computational overhead. We demonstrate our robust learning method and its parameter-free nature on a wide variety of machine learning tasks including online learning and deep learning where it adapts to different levels of noise and maintains high prediction accuracy.
APA
Karakulev, A., Zachariah, D. & Singh, P.. (2024). Adaptive Robust Learning using Latent Bernoulli Variables. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:23105-23122 Available from https://proceedings.mlr.press/v235/karakulev24a.html.

Related Material