Concept Drift Detection Through Resampling

Maayan Harel, Shie Mannor, Ran El-Yaniv, Koby Crammer
Proceedings of the 31st International Conference on Machine Learning, PMLR 32(2):1009-1017, 2014.

Abstract

Detecting changes in data-streams is an important part of enhancing learning quality in dynamic environments. We devise a procedure for detecting concept drifts in data-streams that relies on analyzing the empirical loss of learning algorithms. Our method is based on obtaining statistics from the loss distribution by reusing the data multiple times via resampling. We present theoretical guarantees for the proposed procedure based on the stability of the underlying learning algorithms. Experimental results show that the detection method has high recall and precision, and performs well in the presence of noise.

Cite this Paper

BibTeX
@InProceedings{pmlr-v32-harel14, title = {Concept Drift Detection Through Resampling}, author = {Harel, Maayan and Mannor, Shie and El-Yaniv, Ran and Crammer, Koby}, booktitle = {Proceedings of the 31st International Conference on Machine Learning}, pages = {1009--1017}, year = {2014}, editor = {Xing, Eric P. and Jebara, Tony}, volume = {32}, number = {2}, series = {Proceedings of Machine Learning Research}, address = {Bejing, China}, month = {22--24 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v32/harel14.pdf}, url = {https://proceedings.mlr.press/v32/harel14.html}, abstract = {Detecting changes in data-streams is an important part of enhancing learning quality in dynamic environments. We devise a procedure for detecting concept drifts in data-streams that relies on analyzing the empirical loss of learning algorithms. Our method is based on obtaining statistics from the loss distribution by reusing the data multiple times via resampling. We present theoretical guarantees for the proposed procedure based on the stability of the underlying learning algorithms. Experimental results show that the detection method has high recall and precision, and performs well in the presence of noise.} }
Endnote
%0 Conference Paper %T Concept Drift Detection Through Resampling %A Maayan Harel %A Shie Mannor %A Ran El-Yaniv %A Koby Crammer %B Proceedings of the 31st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2014 %E Eric P. Xing %E Tony Jebara %F pmlr-v32-harel14 %I PMLR %P 1009--1017 %U https://proceedings.mlr.press/v32/harel14.html %V 32 %N 2 %X Detecting changes in data-streams is an important part of enhancing learning quality in dynamic environments. We devise a procedure for detecting concept drifts in data-streams that relies on analyzing the empirical loss of learning algorithms. Our method is based on obtaining statistics from the loss distribution by reusing the data multiple times via resampling. We present theoretical guarantees for the proposed procedure based on the stability of the underlying learning algorithms. Experimental results show that the detection method has high recall and precision, and performs well in the presence of noise.
RIS
TY - CPAPER TI - Concept Drift Detection Through Resampling AU - Maayan Harel AU - Shie Mannor AU - Ran El-Yaniv AU - Koby Crammer BT - Proceedings of the 31st International Conference on Machine Learning DA - 2014/06/18 ED - Eric P. Xing ED - Tony Jebara ID - pmlr-v32-harel14 PB - PMLR DP - Proceedings of Machine Learning Research VL - 32 IS - 2 SP - 1009 EP - 1017 L1 - http://proceedings.mlr.press/v32/harel14.pdf UR - https://proceedings.mlr.press/v32/harel14.html AB - Detecting changes in data-streams is an important part of enhancing learning quality in dynamic environments. We devise a procedure for detecting concept drifts in data-streams that relies on analyzing the empirical loss of learning algorithms. Our method is based on obtaining statistics from the loss distribution by reusing the data multiple times via resampling. We present theoretical guarantees for the proposed procedure based on the stability of the underlying learning algorithms. Experimental results show that the detection method has high recall and precision, and performs well in the presence of noise. ER -
APA
Harel, M., Mannor, S., El-Yaniv, R. & Crammer, K.. (2014). Concept Drift Detection Through Resampling. Proceedings of the 31st International Conference on Machine Learning, in Proceedings of Machine Learning Research 32(2):1009-1017 Available from https://proceedings.mlr.press/v32/harel14.html.

Related Material