Task-Free Continual Learning via Order-Invariant Linearized Adaptation and Density-Guided Adapter Routing

Hang Thi-Thuy Le, Nam-Quan Nguyen, Lam-Huy Nguyen, Dien Dinh, Minh Hoang, Trong Nghia Hoang
Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, PMLR 337:3274-3297, 2026.

Abstract

Task-free continual learning (TFCL) aims to adapt models to non-stationary data streams without knowing task boundaries. In TFCL, catastrophic forgetting arises from both the evolving data distributions and the order-sensitive nature of batch-streaming optimization. To address these challenges, we propose a new TFCL framework that mitigates optimization-induced forgetting via order-invariant linearized adaptation during learning and accommodates evolving data distributions via density-guided adapter routing for more accurate and effective adapter retrieval during inference. We also provide a theoretical characterization of retrieval error in terms of density estimation quality and cross-adapter embedding separability. Experiments across multiple benchmarks demonstrate that our proposed method consistently achieves higher accuracy and lower forgetting than strong TFCL baselines under both standard and realistic streaming settings. For reproducibility, our experimental code is available at: https://github.com/Alisia0303/HESTIA.git .

Cite this Paper

BibTeX
@InProceedings{pmlr-v337-le26a, title = {Task-Free Continual Learning via Order-Invariant Linearized Adaptation and Density-Guided Adapter Routing}, author = {Le, Hang Thi-Thuy and Nguyen, Nam-Quan and Nguyen, Lam-Huy and Dinh, Dien and Hoang, Minh and Hoang, Trong Nghia}, booktitle = {Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence}, pages = {3274--3297}, year = {2026}, editor = {Perković, Emilija and Malinsky, Daniel}, volume = {337}, series = {Proceedings of Machine Learning Research}, month = {17--21 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v337/main/assets/le26a/le26a.pdf}, url = {https://proceedings.mlr.press/v337/le26a.html}, abstract = {Task-free continual learning (TFCL) aims to adapt models to non-stationary data streams without knowing task boundaries. In TFCL, catastrophic forgetting arises from both the evolving data distributions and the order-sensitive nature of batch-streaming optimization. To address these challenges, we propose a new TFCL framework that mitigates optimization-induced forgetting via order-invariant linearized adaptation during learning and accommodates evolving data distributions via density-guided adapter routing for more accurate and effective adapter retrieval during inference. We also provide a theoretical characterization of retrieval error in terms of density estimation quality and cross-adapter embedding separability. Experiments across multiple benchmarks demonstrate that our proposed method consistently achieves higher accuracy and lower forgetting than strong TFCL baselines under both standard and realistic streaming settings. For reproducibility, our experimental code is available at: https://github.com/Alisia0303/HESTIA.git .} }
Endnote
%0 Conference Paper %T Task-Free Continual Learning via Order-Invariant Linearized Adaptation and Density-Guided Adapter Routing %A Hang Thi-Thuy Le %A Nam-Quan Nguyen %A Lam-Huy Nguyen %A Dien Dinh %A Minh Hoang %A Trong Nghia Hoang %B Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2026 %E Emilija Perković %E Daniel Malinsky %F pmlr-v337-le26a %I PMLR %P 3274--3297 %U https://proceedings.mlr.press/v337/le26a.html %V 337 %X Task-free continual learning (TFCL) aims to adapt models to non-stationary data streams without knowing task boundaries. In TFCL, catastrophic forgetting arises from both the evolving data distributions and the order-sensitive nature of batch-streaming optimization. To address these challenges, we propose a new TFCL framework that mitigates optimization-induced forgetting via order-invariant linearized adaptation during learning and accommodates evolving data distributions via density-guided adapter routing for more accurate and effective adapter retrieval during inference. We also provide a theoretical characterization of retrieval error in terms of density estimation quality and cross-adapter embedding separability. Experiments across multiple benchmarks demonstrate that our proposed method consistently achieves higher accuracy and lower forgetting than strong TFCL baselines under both standard and realistic streaming settings. For reproducibility, our experimental code is available at: https://github.com/Alisia0303/HESTIA.git .
APA
Le, H.T., Nguyen, N., Nguyen, L., Dinh, D., Hoang, M. & Hoang, T.N.. (2026). Task-Free Continual Learning via Order-Invariant Linearized Adaptation and Density-Guided Adapter Routing. Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 337:3274-3297 Available from https://proceedings.mlr.press/v337/le26a.html.

Related Material