Differential Geometric Regularization for Supervised Learning of Classifiers

Qinxun Bai, Steven Rosenberg, Zheng Wu, Stan Sclaroff
Proceedings of The 33rd International Conference on Machine Learning, PMLR 48:1879-1888, 2016.

Abstract

We study the problem of supervised learning for both binary and multiclass classification from a unified geometric perspective. In particular, we propose a geometric regularization technique to find the submanifold corresponding to an estimator of the class probability P(y|\vec x). The regularization term measures the volume of this submanifold, based on the intuition that overfitting produces rapid local oscillations and hence large volume of the estimator. This technique can be applied to regularize any classification function that satisfies two requirements: firstly, an estimator of the class probability can be obtained; secondly, first and second derivatives of the class probability estimator can be calculated. In experiments, we apply our regularization technique to standard loss functions for classification, our RBF-based implementation compares favorably to widely used regularization methods for both binary and multiclass classification.

Cite this Paper

BibTeX
@InProceedings{pmlr-v48-baia16, title = {Differential Geometric Regularization for Supervised Learning of Classifiers}, author = {Bai, Qinxun and Rosenberg, Steven and Wu, Zheng and Sclaroff, Stan}, booktitle = {Proceedings of The 33rd International Conference on Machine Learning}, pages = {1879--1888}, year = {2016}, editor = {Balcan, Maria Florina and Weinberger, Kilian Q.}, volume = {48}, series = {Proceedings of Machine Learning Research}, address = {New York, New York, USA}, month = {20--22 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v48/baia16.pdf}, url = {https://proceedings.mlr.press/v48/baia16.html}, abstract = {We study the problem of supervised learning for both binary and multiclass classification from a unified geometric perspective. In particular, we propose a geometric regularization technique to find the submanifold corresponding to an estimator of the class probability P(y|\vec x). The regularization term measures the volume of this submanifold, based on the intuition that overfitting produces rapid local oscillations and hence large volume of the estimator. This technique can be applied to regularize any classification function that satisfies two requirements: firstly, an estimator of the class probability can be obtained; secondly, first and second derivatives of the class probability estimator can be calculated. In experiments, we apply our regularization technique to standard loss functions for classification, our RBF-based implementation compares favorably to widely used regularization methods for both binary and multiclass classification.} }
Endnote
%0 Conference Paper %T Differential Geometric Regularization for Supervised Learning of Classifiers %A Qinxun Bai %A Steven Rosenberg %A Zheng Wu %A Stan Sclaroff %B Proceedings of The 33rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2016 %E Maria Florina Balcan %E Kilian Q. Weinberger %F pmlr-v48-baia16 %I PMLR %P 1879--1888 %U https://proceedings.mlr.press/v48/baia16.html %V 48 %X We study the problem of supervised learning for both binary and multiclass classification from a unified geometric perspective. In particular, we propose a geometric regularization technique to find the submanifold corresponding to an estimator of the class probability P(y|\vec x). The regularization term measures the volume of this submanifold, based on the intuition that overfitting produces rapid local oscillations and hence large volume of the estimator. This technique can be applied to regularize any classification function that satisfies two requirements: firstly, an estimator of the class probability can be obtained; secondly, first and second derivatives of the class probability estimator can be calculated. In experiments, we apply our regularization technique to standard loss functions for classification, our RBF-based implementation compares favorably to widely used regularization methods for both binary and multiclass classification.
RIS
TY - CPAPER TI - Differential Geometric Regularization for Supervised Learning of Classifiers AU - Qinxun Bai AU - Steven Rosenberg AU - Zheng Wu AU - Stan Sclaroff BT - Proceedings of The 33rd International Conference on Machine Learning DA - 2016/06/11 ED - Maria Florina Balcan ED - Kilian Q. Weinberger ID - pmlr-v48-baia16 PB - PMLR DP - Proceedings of Machine Learning Research VL - 48 SP - 1879 EP - 1888 L1 - http://proceedings.mlr.press/v48/baia16.pdf UR - https://proceedings.mlr.press/v48/baia16.html AB - We study the problem of supervised learning for both binary and multiclass classification from a unified geometric perspective. In particular, we propose a geometric regularization technique to find the submanifold corresponding to an estimator of the class probability P(y|\vec x). The regularization term measures the volume of this submanifold, based on the intuition that overfitting produces rapid local oscillations and hence large volume of the estimator. This technique can be applied to regularize any classification function that satisfies two requirements: firstly, an estimator of the class probability can be obtained; secondly, first and second derivatives of the class probability estimator can be calculated. In experiments, we apply our regularization technique to standard loss functions for classification, our RBF-based implementation compares favorably to widely used regularization methods for both binary and multiclass classification. ER -
APA
Bai, Q., Rosenberg, S., Wu, Z. & Sclaroff, S.. (2016). Differential Geometric Regularization for Supervised Learning of Classifiers. Proceedings of The 33rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 48:1879-1888 Available from https://proceedings.mlr.press/v48/baia16.html.

Related Material