A metric learning perspective of SVM: on the relation of LMNN and SVM

Huyen Do, Alexandros Kalousis, Jun Wang, Adam Woznica
Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:308-317, 2012.

Abstract

Support Vector Machines, SVMs, and the Large Margin Nearest Neighbor algorithm, LMNN, are two very popular learning algorithms with quite different learning biases. In this paper we bring them into a unified view and show that they have a much stronger relation than what is commonly thought. We analyze SVMs from a metric learning perspective and cast them as a metric learning problem, a view which helps us uncover the relations of the two algorithms. We show that LMNN can be seen as learning a set of local SVM-like models in a quadratic space. Along the way and inspired by the metric-based interpretation of SVMs we derive a novel variant of SVMs, ε-SVM, to which LMNN is even more similar. We give a unified view of LMNN and the different SVM variants. Finally we provide some preliminary experiments on a number of benchmark datasets in which show that ε-SVM compares favorably both with respect to LMNN and SVM.

Cite this Paper

BibTeX
@InProceedings{pmlr-v22-do12, title = {A metric learning perspective of SVM: on the relation of LMNN and SVM}, author = {Do, Huyen and Kalousis, Alexandros and Wang, Jun and Woznica, Adam}, booktitle = {Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics}, pages = {308--317}, year = {2012}, editor = {Lawrence, Neil D. and Girolami, Mark}, volume = {22}, series = {Proceedings of Machine Learning Research}, address = {La Palma, Canary Islands}, month = {21--23 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v22/do12/do12.pdf}, url = {https://proceedings.mlr.press/v22/do12.html}, abstract = {Support Vector Machines, SVMs, and the Large Margin Nearest Neighbor algorithm, LMNN, are two very popular learning algorithms with quite different learning biases. In this paper we bring them into a unified view and show that they have a much stronger relation than what is commonly thought. We analyze SVMs from a metric learning perspective and cast them as a metric learning problem, a view which helps us uncover the relations of the two algorithms. We show that LMNN can be seen as learning a set of local SVM-like models in a quadratic space. Along the way and inspired by the metric-based interpretation of SVMs we derive a novel variant of SVMs, ε-SVM, to which LMNN is even more similar. We give a unified view of LMNN and the different SVM variants. Finally we provide some preliminary experiments on a number of benchmark datasets in which show that ε-SVM compares favorably both with respect to LMNN and SVM.} }
Endnote
%0 Conference Paper %T A metric learning perspective of SVM: on the relation of LMNN and SVM %A Huyen Do %A Alexandros Kalousis %A Jun Wang %A Adam Woznica %B Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2012 %E Neil D. Lawrence %E Mark Girolami %F pmlr-v22-do12 %I PMLR %P 308--317 %U https://proceedings.mlr.press/v22/do12.html %V 22 %X Support Vector Machines, SVMs, and the Large Margin Nearest Neighbor algorithm, LMNN, are two very popular learning algorithms with quite different learning biases. In this paper we bring them into a unified view and show that they have a much stronger relation than what is commonly thought. We analyze SVMs from a metric learning perspective and cast them as a metric learning problem, a view which helps us uncover the relations of the two algorithms. We show that LMNN can be seen as learning a set of local SVM-like models in a quadratic space. Along the way and inspired by the metric-based interpretation of SVMs we derive a novel variant of SVMs, ε-SVM, to which LMNN is even more similar. We give a unified view of LMNN and the different SVM variants. Finally we provide some preliminary experiments on a number of benchmark datasets in which show that ε-SVM compares favorably both with respect to LMNN and SVM.
RIS
TY - CPAPER TI - A metric learning perspective of SVM: on the relation of LMNN and SVM AU - Huyen Do AU - Alexandros Kalousis AU - Jun Wang AU - Adam Woznica BT - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics DA - 2012/03/21 ED - Neil D. Lawrence ED - Mark Girolami ID - pmlr-v22-do12 PB - PMLR DP - Proceedings of Machine Learning Research VL - 22 SP - 308 EP - 317 L1 - http://proceedings.mlr.press/v22/do12/do12.pdf UR - https://proceedings.mlr.press/v22/do12.html AB - Support Vector Machines, SVMs, and the Large Margin Nearest Neighbor algorithm, LMNN, are two very popular learning algorithms with quite different learning biases. In this paper we bring them into a unified view and show that they have a much stronger relation than what is commonly thought. We analyze SVMs from a metric learning perspective and cast them as a metric learning problem, a view which helps us uncover the relations of the two algorithms. We show that LMNN can be seen as learning a set of local SVM-like models in a quadratic space. Along the way and inspired by the metric-based interpretation of SVMs we derive a novel variant of SVMs, ε-SVM, to which LMNN is even more similar. We give a unified view of LMNN and the different SVM variants. Finally we provide some preliminary experiments on a number of benchmark datasets in which show that ε-SVM compares favorably both with respect to LMNN and SVM. ER -
APA
Do, H., Kalousis, A., Wang, J. & Woznica, A.. (2012). A metric learning perspective of SVM: on the relation of LMNN and SVM. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 22:308-317 Available from https://proceedings.mlr.press/v22/do12.html.

Related Material