Sparse Extreme Multi-label Learning with Oracle Property

Weiwei Liu, Xiaobo Shen
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:4032-4041, 2019.

Abstract

The pioneering work of sparse local embeddings for extreme classification (SLEEC) (Bhatia et al., 2015) has shown great promise in multi-label learning. Unfortunately, the statistical rate of convergence and oracle property of SLEEC are still not well understood. To fill this gap, we present a unified framework for SLEEC with nonconvex penalty. Theoretically, we rigorously prove that our proposed estimator enjoys oracle property (i.e., performs as well as if the underlying model were known beforehand), and obtains a desirable statistical convergence rate. Moreover, we show that under a mild condition on the magnitude of the entries in the underlying model, we are able to obtain an improved convergence rate. Extensive numerical experiments verify our theoretical findings and the superiority of our proposed estimator.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-liu19d, title = {Sparse Extreme Multi-label Learning with Oracle Property}, author = {Liu, Weiwei and Shen, Xiaobo}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {4032--4041}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/liu19d/liu19d.pdf}, url = {https://proceedings.mlr.press/v97/liu19d.html}, abstract = {The pioneering work of sparse local embeddings for extreme classification (SLEEC) (Bhatia et al., 2015) has shown great promise in multi-label learning. Unfortunately, the statistical rate of convergence and oracle property of SLEEC are still not well understood. To fill this gap, we present a unified framework for SLEEC with nonconvex penalty. Theoretically, we rigorously prove that our proposed estimator enjoys oracle property (i.e., performs as well as if the underlying model were known beforehand), and obtains a desirable statistical convergence rate. Moreover, we show that under a mild condition on the magnitude of the entries in the underlying model, we are able to obtain an improved convergence rate. Extensive numerical experiments verify our theoretical findings and the superiority of our proposed estimator.} }
Endnote
%0 Conference Paper %T Sparse Extreme Multi-label Learning with Oracle Property %A Weiwei Liu %A Xiaobo Shen %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-liu19d %I PMLR %P 4032--4041 %U https://proceedings.mlr.press/v97/liu19d.html %V 97 %X The pioneering work of sparse local embeddings for extreme classification (SLEEC) (Bhatia et al., 2015) has shown great promise in multi-label learning. Unfortunately, the statistical rate of convergence and oracle property of SLEEC are still not well understood. To fill this gap, we present a unified framework for SLEEC with nonconvex penalty. Theoretically, we rigorously prove that our proposed estimator enjoys oracle property (i.e., performs as well as if the underlying model were known beforehand), and obtains a desirable statistical convergence rate. Moreover, we show that under a mild condition on the magnitude of the entries in the underlying model, we are able to obtain an improved convergence rate. Extensive numerical experiments verify our theoretical findings and the superiority of our proposed estimator.
APA
Liu, W. & Shen, X.. (2019). Sparse Extreme Multi-label Learning with Oracle Property. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:4032-4041 Available from https://proceedings.mlr.press/v97/liu19d.html.

Related Material