Multi-Class Optimal Margin Distribution Machine

Teng Zhang, Zhi-Hua Zhou
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:4063-4071, 2017.

Abstract

Recent studies disclose that maximizing the minimum margin like support vector machines does not necessarily lead to better generalization performances, and instead, it is crucial to optimize the margin distribution. Although it has been shown that for binary classification, characterizing the margin distribution by the first- and second-order statistics can achieve superior performance. It still remains open for multi-class classification, and due to the complexity of margin for multi-class classification, optimizing its distribution by mean and variance can also be difficult. In this paper, we propose mcODM (multi-class Optimal margin Distribution Machine), which can solve this problem efficiently. We also give a theoretical analysis for our method, which verifies the significance of margin distribution for multi-class classification. Empirical study further shows that mcODM always outperforms all four versions of multi-class SVMs on all experimental data sets.

Cite this Paper

BibTeX
@InProceedings{pmlr-v70-zhang17h, title = {Multi-Class Optimal Margin Distribution Machine}, author = {Teng Zhang and Zhi-Hua Zhou}, booktitle = {Proceedings of the 34th International Conference on Machine Learning}, pages = {4063--4071}, year = {2017}, editor = {Precup, Doina and Teh, Yee Whye}, volume = {70}, series = {Proceedings of Machine Learning Research}, month = {06--11 Aug}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v70/zhang17h/zhang17h.pdf}, url = {https://proceedings.mlr.press/v70/zhang17h.html}, abstract = {Recent studies disclose that maximizing the minimum margin like support vector machines does not necessarily lead to better generalization performances, and instead, it is crucial to optimize the margin distribution. Although it has been shown that for binary classification, characterizing the margin distribution by the first- and second-order statistics can achieve superior performance. It still remains open for multi-class classification, and due to the complexity of margin for multi-class classification, optimizing its distribution by mean and variance can also be difficult. In this paper, we propose mcODM (multi-class Optimal margin Distribution Machine), which can solve this problem efficiently. We also give a theoretical analysis for our method, which verifies the significance of margin distribution for multi-class classification. Empirical study further shows that mcODM always outperforms all four versions of multi-class SVMs on all experimental data sets.} }
Endnote
%0 Conference Paper %T Multi-Class Optimal Margin Distribution Machine %A Teng Zhang %A Zhi-Hua Zhou %B Proceedings of the 34th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2017 %E Doina Precup %E Yee Whye Teh %F pmlr-v70-zhang17h %I PMLR %P 4063--4071 %U https://proceedings.mlr.press/v70/zhang17h.html %V 70 %X Recent studies disclose that maximizing the minimum margin like support vector machines does not necessarily lead to better generalization performances, and instead, it is crucial to optimize the margin distribution. Although it has been shown that for binary classification, characterizing the margin distribution by the first- and second-order statistics can achieve superior performance. It still remains open for multi-class classification, and due to the complexity of margin for multi-class classification, optimizing its distribution by mean and variance can also be difficult. In this paper, we propose mcODM (multi-class Optimal margin Distribution Machine), which can solve this problem efficiently. We also give a theoretical analysis for our method, which verifies the significance of margin distribution for multi-class classification. Empirical study further shows that mcODM always outperforms all four versions of multi-class SVMs on all experimental data sets.
APA
Zhang, T. & Zhou, Z.. (2017). Multi-Class Optimal Margin Distribution Machine. Proceedings of the 34th International Conference on Machine Learning, in Proceedings of Machine Learning Research 70:4063-4071 Available from https://proceedings.mlr.press/v70/zhang17h.html.

Related Material