AIMEE: Interactive model maintenance with rule-based surrogates

Owen Cornec, Rahul Nair, Elizabeth Daly, Oznur Alkan, Dennis Wei
Proceedings of the NeurIPS 2021 Competitions and Demonstrations Track, PMLR 176:288-291, 2022.

Abstract

In real-world applications, such as loan approvals or claims management, machine learn-ing (ML) models need to be updated or retrained to adhere to new rules and regulations.But how can a new model be built and new decision boundaries be formed without having new training data available? We present the AI Model Explorer and Editor tool (AIMEE) for model exploration and model editing using human understandable rules. It addresses the problem of changing decision boundaries by leveraging user-specified feedback rules that are used to pre-process training data such that a retrained model will reflect user changes.The pre-processing step uses synthetic oversampling and relabeling and assumes black-box access to the algorithm that retrains the model. AIMEE provides interactive methods to edit rule sets, visualize changes to decision boundaries, and generate interpretable comparisons of model changes so that users see their feedback reflected in the updated model. The demo shows an end-to-end solution that supports the full update lifecycle of an ML model.

Cite this Paper

BibTeX
@InProceedings{pmlr-v176-cornec22a, title = {AIMEE: Interactive model maintenance with rule-based surrogates}, author = {Cornec, Owen and Nair, Rahul and Daly, Elizabeth and Alkan, Oznur and Wei, Dennis}, booktitle = {Proceedings of the NeurIPS 2021 Competitions and Demonstrations Track}, pages = {288--291}, year = {2022}, editor = {Kiela, Douwe and Ciccone, Marco and Caputo, Barbara}, volume = {176}, series = {Proceedings of Machine Learning Research}, month = {06--14 Dec}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v176/cornec22a/cornec22a.pdf}, url = {https://proceedings.mlr.press/v176/cornec22a.html}, abstract = {In real-world applications, such as loan approvals or claims management, machine learn-ing (ML) models need to be updated or retrained to adhere to new rules and regulations.But how can a new model be built and new decision boundaries be formed without having new training data available? We present the AI Model Explorer and Editor tool (AIMEE) for model exploration and model editing using human understandable rules. It addresses the problem of changing decision boundaries by leveraging user-specified feedback rules that are used to pre-process training data such that a retrained model will reflect user changes.The pre-processing step uses synthetic oversampling and relabeling and assumes black-box access to the algorithm that retrains the model. AIMEE provides interactive methods to edit rule sets, visualize changes to decision boundaries, and generate interpretable comparisons of model changes so that users see their feedback reflected in the updated model. The demo shows an end-to-end solution that supports the full update lifecycle of an ML model.} }
Endnote
%0 Conference Paper %T AIMEE: Interactive model maintenance with rule-based surrogates %A Owen Cornec %A Rahul Nair %A Elizabeth Daly %A Oznur Alkan %A Dennis Wei %B Proceedings of the NeurIPS 2021 Competitions and Demonstrations Track %C Proceedings of Machine Learning Research %D 2022 %E Douwe Kiela %E Marco Ciccone %E Barbara Caputo %F pmlr-v176-cornec22a %I PMLR %P 288--291 %U https://proceedings.mlr.press/v176/cornec22a.html %V 176 %X In real-world applications, such as loan approvals or claims management, machine learn-ing (ML) models need to be updated or retrained to adhere to new rules and regulations.But how can a new model be built and new decision boundaries be formed without having new training data available? We present the AI Model Explorer and Editor tool (AIMEE) for model exploration and model editing using human understandable rules. It addresses the problem of changing decision boundaries by leveraging user-specified feedback rules that are used to pre-process training data such that a retrained model will reflect user changes.The pre-processing step uses synthetic oversampling and relabeling and assumes black-box access to the algorithm that retrains the model. AIMEE provides interactive methods to edit rule sets, visualize changes to decision boundaries, and generate interpretable comparisons of model changes so that users see their feedback reflected in the updated model. The demo shows an end-to-end solution that supports the full update lifecycle of an ML model.
APA
Cornec, O., Nair, R., Daly, E., Alkan, O. & Wei, D.. (2022). AIMEE: Interactive model maintenance with rule-based surrogates. Proceedings of the NeurIPS 2021 Competitions and Demonstrations Track, in Proceedings of Machine Learning Research 176:288-291 Available from https://proceedings.mlr.press/v176/cornec22a.html.

Related Material