Consistent Polyhedral Surrogates for Top-k Classification and Variants

Anish Thilagar, Rafael Frongillo, Jessica J Finocchiaro, Emma Goodwill
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:21329-21359, 2022.

Abstract

Top-$k$ classification is a generalization of multiclass classification used widely in information retrieval, image classification, and other extreme classification settings. Several hinge-like (piecewise-linear) surrogates have been proposed for the problem, yet all are either non-convex or inconsistent. For the proposed hinge-like surrogates that are convex (i.e., polyhedral), we apply the recent embedding framework of Finocchiaro et al. (2019; 2022) to determine the prediction problem for which the surrogate is consistent. These problems can all be interpreted as variants of top-$k$ classification, which may be better aligned with some applications. We leverage this analysis to derive constraints on the conditional label distributions under which these proposed surrogates become consistent for top-$k$. It has been further suggested that every convex hinge-like surrogate must be inconsistent for top-$k$. Yet, we use the same embedding framework to give the first consistent polyhedral surrogate for this problem.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-thilagar22a, title = {Consistent Polyhedral Surrogates for Top-k Classification and Variants}, author = {Thilagar, Anish and Frongillo, Rafael and Finocchiaro, Jessica J and Goodwill, Emma}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {21329--21359}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/thilagar22a/thilagar22a.pdf}, url = {https://proceedings.mlr.press/v162/thilagar22a.html}, abstract = {Top-$k$ classification is a generalization of multiclass classification used widely in information retrieval, image classification, and other extreme classification settings. Several hinge-like (piecewise-linear) surrogates have been proposed for the problem, yet all are either non-convex or inconsistent. For the proposed hinge-like surrogates that are convex (i.e., polyhedral), we apply the recent embedding framework of Finocchiaro et al. (2019; 2022) to determine the prediction problem for which the surrogate is consistent. These problems can all be interpreted as variants of top-$k$ classification, which may be better aligned with some applications. We leverage this analysis to derive constraints on the conditional label distributions under which these proposed surrogates become consistent for top-$k$. It has been further suggested that every convex hinge-like surrogate must be inconsistent for top-$k$. Yet, we use the same embedding framework to give the first consistent polyhedral surrogate for this problem.} }
Endnote
%0 Conference Paper %T Consistent Polyhedral Surrogates for Top-k Classification and Variants %A Anish Thilagar %A Rafael Frongillo %A Jessica J Finocchiaro %A Emma Goodwill %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-thilagar22a %I PMLR %P 21329--21359 %U https://proceedings.mlr.press/v162/thilagar22a.html %V 162 %X Top-$k$ classification is a generalization of multiclass classification used widely in information retrieval, image classification, and other extreme classification settings. Several hinge-like (piecewise-linear) surrogates have been proposed for the problem, yet all are either non-convex or inconsistent. For the proposed hinge-like surrogates that are convex (i.e., polyhedral), we apply the recent embedding framework of Finocchiaro et al. (2019; 2022) to determine the prediction problem for which the surrogate is consistent. These problems can all be interpreted as variants of top-$k$ classification, which may be better aligned with some applications. We leverage this analysis to derive constraints on the conditional label distributions under which these proposed surrogates become consistent for top-$k$. It has been further suggested that every convex hinge-like surrogate must be inconsistent for top-$k$. Yet, we use the same embedding framework to give the first consistent polyhedral surrogate for this problem.
APA
Thilagar, A., Frongillo, R., Finocchiaro, J.J. & Goodwill, E.. (2022). Consistent Polyhedral Surrogates for Top-k Classification and Variants. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:21329-21359 Available from https://proceedings.mlr.press/v162/thilagar22a.html.

Related Material