Coherence Functions for Multicategory Margin-based Classification Methods

Zhihua Zhang, Michael Jordan, Wu-Jun Li, Dit-Yan Yeung
Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics, PMLR 5:647-654, 2009.

Abstract

Margin-based classification methods are typically devised based on a majorization-minimization procedure, which approximately solves an otherwise intractable minimization problem defined with the 0-l loss. However, extension of such methods from the binary classification setting to the more general multicategory setting turns out to be non-trivial. In this paper, our focus is to devise margin-based classification methods that can be seamlessly applied to both settings, with the binary setting simply as a special case. In particular, we propose a new majorization loss function that we call the coherence function, and then devise a new multicategory margin-based boosting algorithm based on the coherence function. Analogous to deterministic annealing, the coherence function is characterized by a temperature factor. It is closely related to the multinomial log-likelihood function and its limit at zero temperature corresponds to a multicategory hinge loss function.

Cite this Paper

BibTeX
@InProceedings{pmlr-v5-zhang09a, title = {Coherence Functions for Multicategory Margin-based Classification Methods}, author = {Zhang, Zhihua and Jordan, Michael and Li, Wu-Jun and Yeung, Dit-Yan}, booktitle = {Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics}, pages = {647--654}, year = {2009}, editor = {van Dyk, David and Welling, Max}, volume = {5}, series = {Proceedings of Machine Learning Research}, address = {Hilton Clearwater Beach Resort, Clearwater Beach, Florida USA}, month = {16--18 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v5/zhang09a/zhang09a.pdf}, url = {https://proceedings.mlr.press/v5/zhang09a.html}, abstract = {Margin-based classification methods are typically devised based on a majorization-minimization procedure, which approximately solves an otherwise intractable minimization problem defined with the 0-l loss. However, extension of such methods from the binary classification setting to the more general multicategory setting turns out to be non-trivial. In this paper, our focus is to devise margin-based classification methods that can be seamlessly applied to both settings, with the binary setting simply as a special case. In particular, we propose a new majorization loss function that we call the coherence function, and then devise a new multicategory margin-based boosting algorithm based on the coherence function. Analogous to deterministic annealing, the coherence function is characterized by a temperature factor. It is closely related to the multinomial log-likelihood function and its limit at zero temperature corresponds to a multicategory hinge loss function.} }
Endnote
%0 Conference Paper %T Coherence Functions for Multicategory Margin-based Classification Methods %A Zhihua Zhang %A Michael Jordan %A Wu-Jun Li %A Dit-Yan Yeung %B Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2009 %E David van Dyk %E Max Welling %F pmlr-v5-zhang09a %I PMLR %P 647--654 %U https://proceedings.mlr.press/v5/zhang09a.html %V 5 %X Margin-based classification methods are typically devised based on a majorization-minimization procedure, which approximately solves an otherwise intractable minimization problem defined with the 0-l loss. However, extension of such methods from the binary classification setting to the more general multicategory setting turns out to be non-trivial. In this paper, our focus is to devise margin-based classification methods that can be seamlessly applied to both settings, with the binary setting simply as a special case. In particular, we propose a new majorization loss function that we call the coherence function, and then devise a new multicategory margin-based boosting algorithm based on the coherence function. Analogous to deterministic annealing, the coherence function is characterized by a temperature factor. It is closely related to the multinomial log-likelihood function and its limit at zero temperature corresponds to a multicategory hinge loss function.
RIS
TY - CPAPER TI - Coherence Functions for Multicategory Margin-based Classification Methods AU - Zhihua Zhang AU - Michael Jordan AU - Wu-Jun Li AU - Dit-Yan Yeung BT - Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics DA - 2009/04/15 ED - David van Dyk ED - Max Welling ID - pmlr-v5-zhang09a PB - PMLR DP - Proceedings of Machine Learning Research VL - 5 SP - 647 EP - 654 L1 - http://proceedings.mlr.press/v5/zhang09a/zhang09a.pdf UR - https://proceedings.mlr.press/v5/zhang09a.html AB - Margin-based classification methods are typically devised based on a majorization-minimization procedure, which approximately solves an otherwise intractable minimization problem defined with the 0-l loss. However, extension of such methods from the binary classification setting to the more general multicategory setting turns out to be non-trivial. In this paper, our focus is to devise margin-based classification methods that can be seamlessly applied to both settings, with the binary setting simply as a special case. In particular, we propose a new majorization loss function that we call the coherence function, and then devise a new multicategory margin-based boosting algorithm based on the coherence function. Analogous to deterministic annealing, the coherence function is characterized by a temperature factor. It is closely related to the multinomial log-likelihood function and its limit at zero temperature corresponds to a multicategory hinge loss function. ER -
APA
Zhang, Z., Jordan, M., Li, W. & Yeung, D.. (2009). Coherence Functions for Multicategory Margin-based Classification Methods. Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 5:647-654 Available from https://proceedings.mlr.press/v5/zhang09a.html.

Related Material