CP-RA$k$EL: Improving Random $k$-labelsets with Conformal Prediction for Multi-label Classification

Fan Yang, Xiaolu Gan, Huazhen Wang, Lei Feng, Yongxuan Lai
Proceedings of the Sixth Workshop on Conformal and Probabilistic Prediction and Applications, PMLR 60:266-279, 2017.

Abstract

Multi-label conformal prediction has attracted much attention in the conformal predictor (CP) community. In this article, we propose to combine CP with random $k$-labelsets (RA$k$EL) method, which is state-of-the-art multi-label classification method for large number of labels. In the framework of RA$k$EL, the original problem is reduced to a number of small-sized multi-label classification tasks by randomly breaking the initial set of labels into a number of small-sized labelsets, and then label powerset (LP) method is employed on these tasks respectively. In this work, ICP-RF, an inductive conformal predictor based on random forest, is used in each multi-label task in order to get p-values for predictions of the LP model, and then the predictions are aggregated to get a final result. Experimental results on six benchmark datasets empirically demonstrate the calibration property of ICP-RF as LP models, and show that conformal prediction can significantly improve the performances of the proposed approach, which is called RA$k$EL. However, the validity property of CP does not hold in CP-RA$k$EL. In the future work we will study how to use some new CP techniques to calibrate the new method.

Cite this Paper

BibTeX
@InProceedings{pmlr-v60-yang17a, title = {{CP-RA$k$EL}: Improving Random $k$-labelsets with Conformal Prediction for Multi-label Classification}, author = {Yang, Fan and Gan, Xiaolu and Wang, Huazhen and Feng, Lei and Lai, Yongxuan}, booktitle = {Proceedings of the Sixth Workshop on Conformal and Probabilistic Prediction and Applications}, pages = {266--279}, year = {2017}, editor = {Gammerman, Alex and Vovk, Vladimir and Luo, Zhiyuan and Papadopoulos, Harris}, volume = {60}, series = {Proceedings of Machine Learning Research}, month = {13--16 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v60/yang17a/yang17a.pdf}, url = {https://proceedings.mlr.press/v60/yang17a.html}, abstract = {Multi-label conformal prediction has attracted much attention in the conformal predictor (CP) community. In this article, we propose to combine CP with random $k$-labelsets (RA$k$EL) method, which is state-of-the-art multi-label classification method for large number of labels. In the framework of RA$k$EL, the original problem is reduced to a number of small-sized multi-label classification tasks by randomly breaking the initial set of labels into a number of small-sized labelsets, and then label powerset (LP) method is employed on these tasks respectively. In this work, ICP-RF, an inductive conformal predictor based on random forest, is used in each multi-label task in order to get p-values for predictions of the LP model, and then the predictions are aggregated to get a final result. Experimental results on six benchmark datasets empirically demonstrate the calibration property of ICP-RF as LP models, and show that conformal prediction can significantly improve the performances of the proposed approach, which is called RA$k$EL. However, the validity property of CP does not hold in CP-RA$k$EL. In the future work we will study how to use some new CP techniques to calibrate the new method.} }
Endnote
%0 Conference Paper %T CP-RA$k$EL: Improving Random $k$-labelsets with Conformal Prediction for Multi-label Classification %A Fan Yang %A Xiaolu Gan %A Huazhen Wang %A Lei Feng %A Yongxuan Lai %B Proceedings of the Sixth Workshop on Conformal and Probabilistic Prediction and Applications %C Proceedings of Machine Learning Research %D 2017 %E Alex Gammerman %E Vladimir Vovk %E Zhiyuan Luo %E Harris Papadopoulos %F pmlr-v60-yang17a %I PMLR %P 266--279 %U https://proceedings.mlr.press/v60/yang17a.html %V 60 %X Multi-label conformal prediction has attracted much attention in the conformal predictor (CP) community. In this article, we propose to combine CP with random $k$-labelsets (RA$k$EL) method, which is state-of-the-art multi-label classification method for large number of labels. In the framework of RA$k$EL, the original problem is reduced to a number of small-sized multi-label classification tasks by randomly breaking the initial set of labels into a number of small-sized labelsets, and then label powerset (LP) method is employed on these tasks respectively. In this work, ICP-RF, an inductive conformal predictor based on random forest, is used in each multi-label task in order to get p-values for predictions of the LP model, and then the predictions are aggregated to get a final result. Experimental results on six benchmark datasets empirically demonstrate the calibration property of ICP-RF as LP models, and show that conformal prediction can significantly improve the performances of the proposed approach, which is called RA$k$EL. However, the validity property of CP does not hold in CP-RA$k$EL. In the future work we will study how to use some new CP techniques to calibrate the new method.
APA
Yang, F., Gan, X., Wang, H., Feng, L. & Lai, Y.. (2017). CP-RA$k$EL: Improving Random $k$-labelsets with Conformal Prediction for Multi-label Classification. Proceedings of the Sixth Workshop on Conformal and Probabilistic Prediction and Applications, in Proceedings of Machine Learning Research 60:266-279 Available from https://proceedings.mlr.press/v60/yang17a.html.

Related Material