DeltaGrad: Rapid retraining of machine learning models

Yinjun Wu, Edgar Dobriban, Susan Davidson
Proceedings of the 37th International Conference on Machine Learning, PMLR 119:10355-10366, 2020.

Abstract

Machine learning models are not static and may need to be retrained on slightly changed datasets, for instance, with the addition or deletion of a set of data points. This has many applications, including privacy, robustness, bias reduction, and uncertainty quantifcation. However, it is expensive to retrain models from scratch. To address this problem, we propose the DeltaGrad algorithm for rapid retraining machine learning models based on information cached during the training phase. We provide both theoretical and empirical support for the effectiveness of DeltaGrad, and show that it compares favorably to the state of the art.

Cite this Paper

BibTeX
@InProceedings{pmlr-v119-wu20b, title = {{D}elta{G}rad: Rapid retraining of machine learning models}, author = {Wu, Yinjun and Dobriban, Edgar and Davidson, Susan}, booktitle = {Proceedings of the 37th International Conference on Machine Learning}, pages = {10355--10366}, year = {2020}, editor = {III, Hal Daumé and Singh, Aarti}, volume = {119}, series = {Proceedings of Machine Learning Research}, month = {13--18 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v119/wu20b/wu20b.pdf}, url = {https://proceedings.mlr.press/v119/wu20b.html}, abstract = {Machine learning models are not static and may need to be retrained on slightly changed datasets, for instance, with the addition or deletion of a set of data points. This has many applications, including privacy, robustness, bias reduction, and uncertainty quantifcation. However, it is expensive to retrain models from scratch. To address this problem, we propose the DeltaGrad algorithm for rapid retraining machine learning models based on information cached during the training phase. We provide both theoretical and empirical support for the effectiveness of DeltaGrad, and show that it compares favorably to the state of the art.} }
Endnote
%0 Conference Paper %T DeltaGrad: Rapid retraining of machine learning models %A Yinjun Wu %A Edgar Dobriban %A Susan Davidson %B Proceedings of the 37th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Hal Daumé III %E Aarti Singh %F pmlr-v119-wu20b %I PMLR %P 10355--10366 %U https://proceedings.mlr.press/v119/wu20b.html %V 119 %X Machine learning models are not static and may need to be retrained on slightly changed datasets, for instance, with the addition or deletion of a set of data points. This has many applications, including privacy, robustness, bias reduction, and uncertainty quantifcation. However, it is expensive to retrain models from scratch. To address this problem, we propose the DeltaGrad algorithm for rapid retraining machine learning models based on information cached during the training phase. We provide both theoretical and empirical support for the effectiveness of DeltaGrad, and show that it compares favorably to the state of the art.
APA
Wu, Y., Dobriban, E. & Davidson, S.. (2020). DeltaGrad: Rapid retraining of machine learning models. Proceedings of the 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research 119:10355-10366 Available from https://proceedings.mlr.press/v119/wu20b.html.

Related Material