Equivalence of Multicategory SVM and Simplex Cone SVM: Fast Computations and Statistical Theory
[edit]
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:41334140, 2018.
Abstract
The multicategory SVM (MSVM) of Lee et al. (2004) is a natural generalization of the classical, binary support vector machines (SVM). However, its use has been limited by computational difficulties. The simplexcone SVM (SCSVM) of Mroueh et al. (2012) is a computationally efficient multicategory classifier, but its use has been limited by a seemingly opaque interpretation. We show that MSVM and SCSVM are in fact exactly equivalent, and provide a bijection between their tuning parameters. MSVM may then be entertained as both a natural and computationally efficient multicategory extension of SVM. We further provide a Donsker theorem for finitedimensional kernel MSVM and partially answer the open question pertaining to the very competitive performance of OnevsRest methods against MSVM. Furthermore, we use the derived asymptotic covariance formula to develop an inversevariance weighted classification rule which improves on the OnevsRest approach.
Related Material


