Patch-Based Contrastive Learning and Memory Consolidation for Online Unsupervised Continual Learning

Cameron Ethan Taylor, Vassilis Vassiliades, Constantine Dovrolis
Proceedings of The 3rd Conference on Lifelong Learning Agents, PMLR 274:938-958, 2025.

Abstract

In this study, we delve into a learning paradigm known as Online Unsupervised Continual Learning (O-UCL) which involves a learner observing an evolving, unlabeled data stream and progressively learning to identify an increasing number of classes of objects. This new paradigm is meant to more closely model many real world applications such as an AI-powered drone that explores a forest attempting to identify the presence of various wildlife species, both new and previously identified. In contrast with prior works in the areas of unsupervised, continual, or online learning, O-UCL focuses on combining all three of these areas into a single challenging and realistic learning environment where agents are evaluated frequently and must aim to have the best possible representation at any point throughout the stream, instead of at the end of each offline “task”. In this new setting we assess our proposed approach, Patch-Based Contrastive Learning and Memory Consolidation (PCMC). PCMC builds a compositional understanding of data through identifying and clustering patch-level features. Embeddings for these patch-level features are extracted with an encoder, trained via a novel patch-based contrastive learning approach. PCMC incorporates new data into it’s distribution while avoiding catastrophic forgetting and consolidating memory examples during a periodic sleep cycle. We evaluate PCMC’s performance on several different streams created from the ImageNet and Places365 datasets. Additionally, we explore various versions of the PCMC algorithm and compare against several existing and simple baselines.

Cite this Paper

BibTeX
@InProceedings{pmlr-v274-taylor25a, title = {Patch-Based Contrastive Learning and Memory Consolidation for Online Unsupervised Continual Learning}, author = {Taylor, Cameron Ethan and Vassiliades, Vassilis and Dovrolis, Constantine}, booktitle = {Proceedings of The 3rd Conference on Lifelong Learning Agents}, pages = {938--958}, year = {2025}, editor = {Lomonaco, Vincenzo and Melacci, Stefano and Tuytelaars, Tinne and Chandar, Sarath and Pascanu, Razvan}, volume = {274}, series = {Proceedings of Machine Learning Research}, month = {29 Jul--01 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v274/main/assets/taylor25a/taylor25a.pdf}, url = {https://proceedings.mlr.press/v274/taylor25a.html}, abstract = {In this study, we delve into a learning paradigm known as Online Unsupervised Continual Learning (O-UCL) which involves a learner observing an evolving, unlabeled data stream and progressively learning to identify an increasing number of classes of objects. This new paradigm is meant to more closely model many real world applications such as an AI-powered drone that explores a forest attempting to identify the presence of various wildlife species, both new and previously identified. In contrast with prior works in the areas of unsupervised, continual, or online learning, O-UCL focuses on combining all three of these areas into a single challenging and realistic learning environment where agents are evaluated frequently and must aim to have the best possible representation at any point throughout the stream, instead of at the end of each offline “task”. In this new setting we assess our proposed approach, Patch-Based Contrastive Learning and Memory Consolidation (PCMC). PCMC builds a compositional understanding of data through identifying and clustering patch-level features. Embeddings for these patch-level features are extracted with an encoder, trained via a novel patch-based contrastive learning approach. PCMC incorporates new data into it’s distribution while avoiding catastrophic forgetting and consolidating memory examples during a periodic sleep cycle. We evaluate PCMC’s performance on several different streams created from the ImageNet and Places365 datasets. Additionally, we explore various versions of the PCMC algorithm and compare against several existing and simple baselines.} }
Endnote
%0 Conference Paper %T Patch-Based Contrastive Learning and Memory Consolidation for Online Unsupervised Continual Learning %A Cameron Ethan Taylor %A Vassilis Vassiliades %A Constantine Dovrolis %B Proceedings of The 3rd Conference on Lifelong Learning Agents %C Proceedings of Machine Learning Research %D 2025 %E Vincenzo Lomonaco %E Stefano Melacci %E Tinne Tuytelaars %E Sarath Chandar %E Razvan Pascanu %F pmlr-v274-taylor25a %I PMLR %P 938--958 %U https://proceedings.mlr.press/v274/taylor25a.html %V 274 %X In this study, we delve into a learning paradigm known as Online Unsupervised Continual Learning (O-UCL) which involves a learner observing an evolving, unlabeled data stream and progressively learning to identify an increasing number of classes of objects. This new paradigm is meant to more closely model many real world applications such as an AI-powered drone that explores a forest attempting to identify the presence of various wildlife species, both new and previously identified. In contrast with prior works in the areas of unsupervised, continual, or online learning, O-UCL focuses on combining all three of these areas into a single challenging and realistic learning environment where agents are evaluated frequently and must aim to have the best possible representation at any point throughout the stream, instead of at the end of each offline “task”. In this new setting we assess our proposed approach, Patch-Based Contrastive Learning and Memory Consolidation (PCMC). PCMC builds a compositional understanding of data through identifying and clustering patch-level features. Embeddings for these patch-level features are extracted with an encoder, trained via a novel patch-based contrastive learning approach. PCMC incorporates new data into it’s distribution while avoiding catastrophic forgetting and consolidating memory examples during a periodic sleep cycle. We evaluate PCMC’s performance on several different streams created from the ImageNet and Places365 datasets. Additionally, we explore various versions of the PCMC algorithm and compare against several existing and simple baselines.
APA
Taylor, C.E., Vassiliades, V. & Dovrolis, C.. (2025). Patch-Based Contrastive Learning and Memory Consolidation for Online Unsupervised Continual Learning. Proceedings of The 3rd Conference on Lifelong Learning Agents, in Proceedings of Machine Learning Research 274:938-958 Available from https://proceedings.mlr.press/v274/taylor25a.html.

Related Material