A metric learning perspective of SVM: on the relation of LMNN and SVM


Huyen Do, Alexandros Kalousis, Jun Wang, Adam Woznica ;
Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:308-317, 2012.


Support Vector Machines, SVMs, and the Large Margin Nearest Neighbor algorithm, LMNN, are two very popular learning algorithms with quite different learning biases. In this paper we bring them into a unified view and show that they have a much stronger relation than what is commonly thought. We analyze SVMs from a metric learning perspective and cast them as a metric learning problem, a view which helps us uncover the relations of the two algorithms. We show that LMNN can be seen as learning a set of local SVM-like models in a quadratic space. Along the way and inspired by the metric-based interpretation of SVMs we derive a novel variant of SVMs, ε-SVM, to which LMNN is even more similar. We give a unified view of LMNN and the different SVM variants. Finally we provide some preliminary experiments on a number of benchmark datasets in which show that ε-SVM compares favorably both with respect to LMNN and SVM.

Related Material