A metric learning perspective of SVM: on the relation of LMNN and SVM

Huyen Do, Alexandros Kalousis, Jun Wang, Adam Woznica
; Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:308-317, 2012.

Abstract

Support Vector Machines, SVMs, and the Large Margin Nearest Neighbor algorithm, LMNN, are two very popular learning algorithms with quite different learning biases. In this paper we bring them into a unified view and show that they have a much stronger relation than what is commonly thought. We analyze SVMs from a metric learning perspective and cast them as a metric learning problem, a view which helps us uncover the relations of the two algorithms. We show that LMNN can be seen as learning a set of local SVM-like models in a quadratic space. Along the way and inspired by the metric-based interpretation of SVMs we derive a novel variant of SVMs, ε-SVM, to which LMNN is even more similar. We give a unified view of LMNN and the different SVM variants. Finally we provide some preliminary experiments on a number of benchmark datasets in which show that ε-SVM compares favorably both with respect to LMNN and SVM.

Cite this Paper


BibTeX
@InProceedings{pmlr-v22-do12, title = {A metric learning perspective of SVM: on the relation of LMNN and SVM}, author = {Huyen Do and Alexandros Kalousis and Jun Wang and Adam Woznica}, booktitle = {Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics}, pages = {308--317}, year = {2012}, editor = {Neil D. Lawrence and Mark Girolami}, volume = {22}, series = {Proceedings of Machine Learning Research}, address = {La Palma, Canary Islands}, month = {21--23 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v22/do12/do12.pdf}, url = {http://proceedings.mlr.press/v22/do12.html}, abstract = {Support Vector Machines, SVMs, and the Large Margin Nearest Neighbor algorithm, LMNN, are two very popular learning algorithms with quite different learning biases. In this paper we bring them into a unified view and show that they have a much stronger relation than what is commonly thought. We analyze SVMs from a metric learning perspective and cast them as a metric learning problem, a view which helps us uncover the relations of the two algorithms. We show that LMNN can be seen as learning a set of local SVM-like models in a quadratic space. Along the way and inspired by the metric-based interpretation of SVMs we derive a novel variant of SVMs, ε-SVM, to which LMNN is even more similar. We give a unified view of LMNN and the different SVM variants. Finally we provide some preliminary experiments on a number of benchmark datasets in which show that ε-SVM compares favorably both with respect to LMNN and SVM.} }
Endnote
%0 Conference Paper %T A metric learning perspective of SVM: on the relation of LMNN and SVM %A Huyen Do %A Alexandros Kalousis %A Jun Wang %A Adam Woznica %B Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2012 %E Neil D. Lawrence %E Mark Girolami %F pmlr-v22-do12 %I PMLR %J Proceedings of Machine Learning Research %P 308--317 %U http://proceedings.mlr.press %V 22 %W PMLR %X Support Vector Machines, SVMs, and the Large Margin Nearest Neighbor algorithm, LMNN, are two very popular learning algorithms with quite different learning biases. In this paper we bring them into a unified view and show that they have a much stronger relation than what is commonly thought. We analyze SVMs from a metric learning perspective and cast them as a metric learning problem, a view which helps us uncover the relations of the two algorithms. We show that LMNN can be seen as learning a set of local SVM-like models in a quadratic space. Along the way and inspired by the metric-based interpretation of SVMs we derive a novel variant of SVMs, ε-SVM, to which LMNN is even more similar. We give a unified view of LMNN and the different SVM variants. Finally we provide some preliminary experiments on a number of benchmark datasets in which show that ε-SVM compares favorably both with respect to LMNN and SVM.
RIS
TY - CPAPER TI - A metric learning perspective of SVM: on the relation of LMNN and SVM AU - Huyen Do AU - Alexandros Kalousis AU - Jun Wang AU - Adam Woznica BT - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics PY - 2012/03/21 DA - 2012/03/21 ED - Neil D. Lawrence ED - Mark Girolami ID - pmlr-v22-do12 PB - PMLR SP - 308 DP - PMLR EP - 317 L1 - http://proceedings.mlr.press/v22/do12/do12.pdf UR - http://proceedings.mlr.press/v22/do12.html AB - Support Vector Machines, SVMs, and the Large Margin Nearest Neighbor algorithm, LMNN, are two very popular learning algorithms with quite different learning biases. In this paper we bring them into a unified view and show that they have a much stronger relation than what is commonly thought. We analyze SVMs from a metric learning perspective and cast them as a metric learning problem, a view which helps us uncover the relations of the two algorithms. We show that LMNN can be seen as learning a set of local SVM-like models in a quadratic space. Along the way and inspired by the metric-based interpretation of SVMs we derive a novel variant of SVMs, ε-SVM, to which LMNN is even more similar. We give a unified view of LMNN and the different SVM variants. Finally we provide some preliminary experiments on a number of benchmark datasets in which show that ε-SVM compares favorably both with respect to LMNN and SVM. ER -
APA
Do, H., Kalousis, A., Wang, J. & Woznica, A.. (2012). A metric learning perspective of SVM: on the relation of LMNN and SVM. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, in PMLR 22:308-317

Related Material