CD-IMM: The Benefits of Domain-based Mixture Models in Bayesian Continual Learning

Daniele Castellana, Antonio Carta, Davide Bacciu
Proceedings of the 1st ContinualAI Unconference, 2023, PMLR 249:25-36, 2024.

Abstract

Real-world streams of data are characterised by the continuous occurrence of new and old classes, possibly on novel domains. Bayesian non-parametric mixture models provide a natural solution for continual learning due to their ability to create new components on the fly when new data are observed. However, popular class-based and time-based mixtures are often tested on simplified streams (eg class-incremental), where shortcuts can be exploited to infer drifts. We hypothesise that domain-based mixtures are more effective on natural streams. Our proposed method, the CD-IMM, exemplifies this approach by learning an infinite mixture of domains for each class. We experiment on a natural scenario with a mix of class repetitions and novel domains to validate our hypothesis. The experimental results confirm our hypothesis and we find that CD-IMM beats state-of-the-art bayesian continual learning methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v249-castellana24a, title = {CD-IMM: The Benefits of Domain-based Mixture Models in Bayesian Continual Learning}, author = {Castellana, Daniele and Carta, Antonio and Bacciu, Davide}, booktitle = {Proceedings of the 1st ContinualAI Unconference, 2023}, pages = {25--36}, year = {2024}, editor = {Swaroop, Siddharth and Mundt, Martin and Aljundi, Rahaf and Khan, Mohammad Emtiyaz}, volume = {249}, series = {Proceedings of Machine Learning Research}, month = {09 Oct}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v249/main/assets/castellana24a/castellana24a.pdf}, url = {https://proceedings.mlr.press/v249/castellana24a.html}, abstract = {Real-world streams of data are characterised by the continuous occurrence of new and old classes, possibly on novel domains. Bayesian non-parametric mixture models provide a natural solution for continual learning due to their ability to create new components on the fly when new data are observed. However, popular class-based and time-based mixtures are often tested on simplified streams (eg class-incremental), where shortcuts can be exploited to infer drifts. We hypothesise that domain-based mixtures are more effective on natural streams. Our proposed method, the CD-IMM, exemplifies this approach by learning an infinite mixture of domains for each class. We experiment on a natural scenario with a mix of class repetitions and novel domains to validate our hypothesis. The experimental results confirm our hypothesis and we find that CD-IMM beats state-of-the-art bayesian continual learning methods.} }
Endnote
%0 Conference Paper %T CD-IMM: The Benefits of Domain-based Mixture Models in Bayesian Continual Learning %A Daniele Castellana %A Antonio Carta %A Davide Bacciu %B Proceedings of the 1st ContinualAI Unconference, 2023 %C Proceedings of Machine Learning Research %D 2024 %E Siddharth Swaroop %E Martin Mundt %E Rahaf Aljundi %E Mohammad Emtiyaz Khan %F pmlr-v249-castellana24a %I PMLR %P 25--36 %U https://proceedings.mlr.press/v249/castellana24a.html %V 249 %X Real-world streams of data are characterised by the continuous occurrence of new and old classes, possibly on novel domains. Bayesian non-parametric mixture models provide a natural solution for continual learning due to their ability to create new components on the fly when new data are observed. However, popular class-based and time-based mixtures are often tested on simplified streams (eg class-incremental), where shortcuts can be exploited to infer drifts. We hypothesise that domain-based mixtures are more effective on natural streams. Our proposed method, the CD-IMM, exemplifies this approach by learning an infinite mixture of domains for each class. We experiment on a natural scenario with a mix of class repetitions and novel domains to validate our hypothesis. The experimental results confirm our hypothesis and we find that CD-IMM beats state-of-the-art bayesian continual learning methods.
APA
Castellana, D., Carta, A. & Bacciu, D.. (2024). CD-IMM: The Benefits of Domain-based Mixture Models in Bayesian Continual Learning. Proceedings of the 1st ContinualAI Unconference, 2023, in Proceedings of Machine Learning Research 249:25-36 Available from https://proceedings.mlr.press/v249/castellana24a.html.

Related Material