A Conformal Martingales Approach for Recurrent Concept Drift

Charalambos Eliades, Harris Papadopoulos
Proceedings of the Fourteenth Symposium on Conformal and Probabilistic Prediction with Applications, PMLR 266:706-724, 2025.

Abstract

In many Concept Drift scenarios, previously seen data distributions reappear. This type of drift is known as Recurrent Concept Drift and is especially common in environments with seasonality, user-behavior cycles, or regime changes. This work extends our previously proposed Inductive Conformal Martingales(ICM) concept drift approach so that it can reuse earlier models, thus saving computational resources and data. Upon drift detection, the proposed approach selects a model from a pool of all earlier models if (i) ICM fails to reject exchangeability between a recent-data window and the window immediately following that model’s training set, and (ii) the model’s F1 score on the current window exceeds a threshold derived from its historical performance. It only trains a new model when no stored model satisfies both criteria. Experiments on three public data streams (STAGGER, Airlines and ELEC) cut retraining events by up to 94% and reduce wasted training instances by 22%-33%, while limiting accuracy loss to less than 3 percentage points relative to always retraining.

Cite this Paper

BibTeX
@InProceedings{pmlr-v266-eliades25a, title = {A Conformal Martingales Approach for Recurrent Concept Drift}, author = {Eliades, Charalambos and Papadopoulos, Harris}, booktitle = {Proceedings of the Fourteenth Symposium on Conformal and Probabilistic Prediction with Applications}, pages = {706--724}, year = {2025}, editor = {Nguyen, Khuong An and Luo, Zhiyuan and Papadopoulos, Harris and Löfström, Tuwe and Carlsson, Lars and Boström, Henrik}, volume = {266}, series = {Proceedings of Machine Learning Research}, month = {10--12 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v266/main/assets/eliades25a/eliades25a.pdf}, url = {https://proceedings.mlr.press/v266/eliades25a.html}, abstract = {In many Concept Drift scenarios, previously seen data distributions reappear. This type of drift is known as Recurrent Concept Drift and is especially common in environments with seasonality, user-behavior cycles, or regime changes. This work extends our previously proposed Inductive Conformal Martingales(ICM) concept drift approach so that it can reuse earlier models, thus saving computational resources and data. Upon drift detection, the proposed approach selects a model from a pool of all earlier models if (i) ICM fails to reject exchangeability between a recent-data window and the window immediately following that model’s training set, and (ii) the model’s F1 score on the current window exceeds a threshold derived from its historical performance. It only trains a new model when no stored model satisfies both criteria. Experiments on three public data streams (STAGGER, Airlines and ELEC) cut retraining events by up to 94% and reduce wasted training instances by 22%-33%, while limiting accuracy loss to less than 3 percentage points relative to always retraining.} }
Endnote
%0 Conference Paper %T A Conformal Martingales Approach for Recurrent Concept Drift %A Charalambos Eliades %A Harris Papadopoulos %B Proceedings of the Fourteenth Symposium on Conformal and Probabilistic Prediction with Applications %C Proceedings of Machine Learning Research %D 2025 %E Khuong An Nguyen %E Zhiyuan Luo %E Harris Papadopoulos %E Tuwe Löfström %E Lars Carlsson %E Henrik Boström %F pmlr-v266-eliades25a %I PMLR %P 706--724 %U https://proceedings.mlr.press/v266/eliades25a.html %V 266 %X In many Concept Drift scenarios, previously seen data distributions reappear. This type of drift is known as Recurrent Concept Drift and is especially common in environments with seasonality, user-behavior cycles, or regime changes. This work extends our previously proposed Inductive Conformal Martingales(ICM) concept drift approach so that it can reuse earlier models, thus saving computational resources and data. Upon drift detection, the proposed approach selects a model from a pool of all earlier models if (i) ICM fails to reject exchangeability between a recent-data window and the window immediately following that model’s training set, and (ii) the model’s F1 score on the current window exceeds a threshold derived from its historical performance. It only trains a new model when no stored model satisfies both criteria. Experiments on three public data streams (STAGGER, Airlines and ELEC) cut retraining events by up to 94% and reduce wasted training instances by 22%-33%, while limiting accuracy loss to less than 3 percentage points relative to always retraining.
APA
Eliades, C. & Papadopoulos, H.. (2025). A Conformal Martingales Approach for Recurrent Concept Drift. Proceedings of the Fourteenth Symposium on Conformal and Probabilistic Prediction with Applications, in Proceedings of Machine Learning Research 266:706-724 Available from https://proceedings.mlr.press/v266/eliades25a.html.

Related Material