MODL: Multilearner Online Deep Learning

Antonios Valkanas, Boris N. Oreshkin, Mark Coates
Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, PMLR 258:4321-4329, 2025.

Abstract

Online deep learning tackles the challenge of learning from data streams by balancing two competing goals: fast learning and deep learning. However, existing research primarily emphasizes deep learning solutions, which are more adept at handling the ”deep” aspect than the ”fast” aspect of online learning. In this work, we introduce an alternative paradigm through a hybrid multilearner approach. We begin by developing a fast online logistic regression learner, which operates without relying on backpropagation. It leverages closed-form recursive updates of model parameters, efficiently addressing the fast learning component of the online learning challenge. This approach is further integrated with a cascaded multilearner design, where shallow and deep learners are co-trained in a cooperative, synergistic manner to solve the online learning problem. We demonstrate that this approach achieves state-of-the-art performance on standard online learning datasets. We make our code available: \url{https://github.com/AntonValk/MODL}

Cite this Paper

BibTeX
@InProceedings{pmlr-v258-valkanas25a, title = {MODL: Multilearner Online Deep Learning}, author = {Valkanas, Antonios and Oreshkin, Boris N. and Coates, Mark}, booktitle = {Proceedings of The 28th International Conference on Artificial Intelligence and Statistics}, pages = {4321--4329}, year = {2025}, editor = {Li, Yingzhen and Mandt, Stephan and Agrawal, Shipra and Khan, Emtiyaz}, volume = {258}, series = {Proceedings of Machine Learning Research}, month = {03--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v258/main/assets/valkanas25a/valkanas25a.pdf}, url = {https://proceedings.mlr.press/v258/valkanas25a.html}, abstract = {Online deep learning tackles the challenge of learning from data streams by balancing two competing goals: fast learning and deep learning. However, existing research primarily emphasizes deep learning solutions, which are more adept at handling the ”deep” aspect than the ”fast” aspect of online learning. In this work, we introduce an alternative paradigm through a hybrid multilearner approach. We begin by developing a fast online logistic regression learner, which operates without relying on backpropagation. It leverages closed-form recursive updates of model parameters, efficiently addressing the fast learning component of the online learning challenge. This approach is further integrated with a cascaded multilearner design, where shallow and deep learners are co-trained in a cooperative, synergistic manner to solve the online learning problem. We demonstrate that this approach achieves state-of-the-art performance on standard online learning datasets. We make our code available: \url{https://github.com/AntonValk/MODL}} }
Endnote
%0 Conference Paper %T MODL: Multilearner Online Deep Learning %A Antonios Valkanas %A Boris N. Oreshkin %A Mark Coates %B Proceedings of The 28th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2025 %E Yingzhen Li %E Stephan Mandt %E Shipra Agrawal %E Emtiyaz Khan %F pmlr-v258-valkanas25a %I PMLR %P 4321--4329 %U https://proceedings.mlr.press/v258/valkanas25a.html %V 258 %X Online deep learning tackles the challenge of learning from data streams by balancing two competing goals: fast learning and deep learning. However, existing research primarily emphasizes deep learning solutions, which are more adept at handling the ”deep” aspect than the ”fast” aspect of online learning. In this work, we introduce an alternative paradigm through a hybrid multilearner approach. We begin by developing a fast online logistic regression learner, which operates without relying on backpropagation. It leverages closed-form recursive updates of model parameters, efficiently addressing the fast learning component of the online learning challenge. This approach is further integrated with a cascaded multilearner design, where shallow and deep learners are co-trained in a cooperative, synergistic manner to solve the online learning problem. We demonstrate that this approach achieves state-of-the-art performance on standard online learning datasets. We make our code available: \url{https://github.com/AntonValk/MODL}
APA
Valkanas, A., Oreshkin, B.N. & Coates, M.. (2025). MODL: Multilearner Online Deep Learning. Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 258:4321-4329 Available from https://proceedings.mlr.press/v258/valkanas25a.html.

Related Material