Cost-sensitive Classifier Selection when there is Additional Cost Information

Ryan Meekins, Stephen Adams, Peter A. Beling, Kevin Farinholt, Nathan Hipwell, Ali Chaudhry, Sherwood Polter, Qing Dong
Proceedings of The International Workshop on Cost-Sensitive Learning, PMLR 88:17-30, 2018.

Abstract

Machine learning models are increasing in popularity in many domains as they are shown to be able to solve difficult problems. However, selecting a model to implement when there are various alternatives is a difficult problem. Receiver operating characteristic (ROC) curves are useful for selecting binary classification models for real world problems. However, ROC curves only consider the misclassification cost of the classifier. The total cost of a classification system includes various other types of cost including implementation, computation, and feature costs. To extend the ROC analysis to include this additional cost information, the ROC Convex Hull with Cost (ROCCHC) method is introduced. This method extends the ROC Convex Hull (ROCCH) method, which is used to select potentially optimal classifiers in the ROC space using misclassification cost, by selecting potentially optimal classifiers using this additional cost information. The ROCCHC method is tested using three binary classification data sets, each of which include real feature costs as the additional cost information. Competing classifiers are created with the CART algorithm by using each combination of features or sensors for each data set. The ROCCHC method reduces the classifier decision space to 4%, 9%, and 0.02%. These results are compared to the current ROCCH method, which misses 91%, 58%, and 6% of potentially optimal classifiers because the method does not include the additional cost information.

Cite this Paper

BibTeX
@InProceedings{pmlr-v88-meekins18a, title = {Cost-sensitive Classifier Selection when there is Additional Cost Information}, author = {Meekins, Ryan and Adams, Stephen and Beling, Peter A. and Farinholt, Kevin and Hipwell, Nathan and Chaudhry, Ali and Polter, Sherwood and Dong, Qing}, booktitle = {Proceedings of The International Workshop on Cost-Sensitive Learning}, pages = {17--30}, year = {2018}, editor = {Torgo, Luís and Matwin, Stan and Weiss, Gary and Moniz, Nuno and Branco, Paula}, volume = {88}, series = {Proceedings of Machine Learning Research}, month = {05 May}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v88/meekins18a/meekins18a.pdf}, url = {https://proceedings.mlr.press/v88/meekins18a.html}, abstract = {Machine learning models are increasing in popularity in many domains as they are shown to be able to solve difficult problems. However, selecting a model to implement when there are various alternatives is a difficult problem. Receiver operating characteristic (ROC) curves are useful for selecting binary classification models for real world problems. However, ROC curves only consider the misclassification cost of the classifier. The total cost of a classification system includes various other types of cost including implementation, computation, and feature costs. To extend the ROC analysis to include this additional cost information, the ROC Convex Hull with Cost (ROCCHC) method is introduced. This method extends the ROC Convex Hull (ROCCH) method, which is used to select potentially optimal classifiers in the ROC space using misclassification cost, by selecting potentially optimal classifiers using this additional cost information. The ROCCHC method is tested using three binary classification data sets, each of which include real feature costs as the additional cost information. Competing classifiers are created with the CART algorithm by using each combination of features or sensors for each data set. The ROCCHC method reduces the classifier decision space to 4%, 9%, and 0.02%. These results are compared to the current ROCCH method, which misses 91%, 58%, and 6% of potentially optimal classifiers because the method does not include the additional cost information.} }
Endnote
%0 Conference Paper %T Cost-sensitive Classifier Selection when there is Additional Cost Information %A Ryan Meekins %A Stephen Adams %A Peter A. Beling %A Kevin Farinholt %A Nathan Hipwell %A Ali Chaudhry %A Sherwood Polter %A Qing Dong %B Proceedings of The International Workshop on Cost-Sensitive Learning %C Proceedings of Machine Learning Research %D 2018 %E Luís Torgo %E Stan Matwin %E Gary Weiss %E Nuno Moniz %E Paula Branco %F pmlr-v88-meekins18a %I PMLR %P 17--30 %U https://proceedings.mlr.press/v88/meekins18a.html %V 88 %X Machine learning models are increasing in popularity in many domains as they are shown to be able to solve difficult problems. However, selecting a model to implement when there are various alternatives is a difficult problem. Receiver operating characteristic (ROC) curves are useful for selecting binary classification models for real world problems. However, ROC curves only consider the misclassification cost of the classifier. The total cost of a classification system includes various other types of cost including implementation, computation, and feature costs. To extend the ROC analysis to include this additional cost information, the ROC Convex Hull with Cost (ROCCHC) method is introduced. This method extends the ROC Convex Hull (ROCCH) method, which is used to select potentially optimal classifiers in the ROC space using misclassification cost, by selecting potentially optimal classifiers using this additional cost information. The ROCCHC method is tested using three binary classification data sets, each of which include real feature costs as the additional cost information. Competing classifiers are created with the CART algorithm by using each combination of features or sensors for each data set. The ROCCHC method reduces the classifier decision space to 4%, 9%, and 0.02%. These results are compared to the current ROCCH method, which misses 91%, 58%, and 6% of potentially optimal classifiers because the method does not include the additional cost information.
APA
Meekins, R., Adams, S., Beling, P.A., Farinholt, K., Hipwell, N., Chaudhry, A., Polter, S. & Dong, Q.. (2018). Cost-sensitive Classifier Selection when there is Additional Cost Information. Proceedings of The International Workshop on Cost-Sensitive Learning, in Proceedings of Machine Learning Research 88:17-30 Available from https://proceedings.mlr.press/v88/meekins18a.html.

Related Material