DriftSurf: Stable-State / Reactive-State Learning under Concept Drift

Ashraf Tahmasbi, Ellango Jothimurugesan, Srikanta Tirthapura, Phillip B Gibbons
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:10054-10064, 2021.

Abstract

When learning from streaming data, a change in the data distribution, also known as concept drift, can render a previously-learned model inaccurate and require training a new model. We present an adaptive learning algorithm that extends previous drift-detection-based methods by incorporating drift detection into a broader stable-state/reactive-state process. The advantage of our approach is that we can use aggressive drift detection in the stable state to achieve a high detection rate, but mitigate the false positive rate of standalone drift detection via a reactive state that reacts quickly to true drifts while eliminating most false positives. The algorithm is generic in its base learner and can be applied across a variety of supervised learning problems. Our theoretical analysis shows that the risk of the algorithm is (i) statistically better than standalone drift detection and (ii) competitive to an algorithm with oracle knowledge of when (abrupt) drifts occur. Experiments on synthetic and real datasets with concept drifts confirm our theoretical analysis.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-tahmasbi21a, title = {DriftSurf: Stable-State / Reactive-State Learning under Concept Drift}, author = {Tahmasbi, Ashraf and Jothimurugesan, Ellango and Tirthapura, Srikanta and Gibbons, Phillip B}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {10054--10064}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/tahmasbi21a/tahmasbi21a.pdf}, url = {https://proceedings.mlr.press/v139/tahmasbi21a.html}, abstract = {When learning from streaming data, a change in the data distribution, also known as concept drift, can render a previously-learned model inaccurate and require training a new model. We present an adaptive learning algorithm that extends previous drift-detection-based methods by incorporating drift detection into a broader stable-state/reactive-state process. The advantage of our approach is that we can use aggressive drift detection in the stable state to achieve a high detection rate, but mitigate the false positive rate of standalone drift detection via a reactive state that reacts quickly to true drifts while eliminating most false positives. The algorithm is generic in its base learner and can be applied across a variety of supervised learning problems. Our theoretical analysis shows that the risk of the algorithm is (i) statistically better than standalone drift detection and (ii) competitive to an algorithm with oracle knowledge of when (abrupt) drifts occur. Experiments on synthetic and real datasets with concept drifts confirm our theoretical analysis.} }
Endnote
%0 Conference Paper %T DriftSurf: Stable-State / Reactive-State Learning under Concept Drift %A Ashraf Tahmasbi %A Ellango Jothimurugesan %A Srikanta Tirthapura %A Phillip B Gibbons %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-tahmasbi21a %I PMLR %P 10054--10064 %U https://proceedings.mlr.press/v139/tahmasbi21a.html %V 139 %X When learning from streaming data, a change in the data distribution, also known as concept drift, can render a previously-learned model inaccurate and require training a new model. We present an adaptive learning algorithm that extends previous drift-detection-based methods by incorporating drift detection into a broader stable-state/reactive-state process. The advantage of our approach is that we can use aggressive drift detection in the stable state to achieve a high detection rate, but mitigate the false positive rate of standalone drift detection via a reactive state that reacts quickly to true drifts while eliminating most false positives. The algorithm is generic in its base learner and can be applied across a variety of supervised learning problems. Our theoretical analysis shows that the risk of the algorithm is (i) statistically better than standalone drift detection and (ii) competitive to an algorithm with oracle knowledge of when (abrupt) drifts occur. Experiments on synthetic and real datasets with concept drifts confirm our theoretical analysis.
APA
Tahmasbi, A., Jothimurugesan, E., Tirthapura, S. & Gibbons, P.B.. (2021). DriftSurf: Stable-State / Reactive-State Learning under Concept Drift. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:10054-10064 Available from https://proceedings.mlr.press/v139/tahmasbi21a.html.

Related Material