Connecting Sphere Manifolds Hierarchically for Regularization

Damien Scieur, Youngsung Kim
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:9399-9409, 2021.

Abstract

This paper considers classification problems with hierarchically organized classes. We force the classifier (hyperplane) of each class to belong to a sphere manifold, whose center is the classifier of its super-class. Then, individual sphere manifolds are connected based on their hierarchical relations. Our technique replaces the last layer of a neural network by combining a spherical fully-connected layer with a hierarchical layer. This regularization is shown to improve the performance of widely used deep neural network architectures (ResNet and DenseNet) on publicly available datasets (CIFAR100, CUB200, Stanford dogs, Stanford cars, and Tiny-ImageNet).

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-scieur21a, title = {Connecting Sphere Manifolds Hierarchically for Regularization}, author = {Scieur, Damien and Kim, Youngsung}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {9399--9409}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/scieur21a/scieur21a.pdf}, url = {https://proceedings.mlr.press/v139/scieur21a.html}, abstract = {This paper considers classification problems with hierarchically organized classes. We force the classifier (hyperplane) of each class to belong to a sphere manifold, whose center is the classifier of its super-class. Then, individual sphere manifolds are connected based on their hierarchical relations. Our technique replaces the last layer of a neural network by combining a spherical fully-connected layer with a hierarchical layer. This regularization is shown to improve the performance of widely used deep neural network architectures (ResNet and DenseNet) on publicly available datasets (CIFAR100, CUB200, Stanford dogs, Stanford cars, and Tiny-ImageNet).} }
Endnote
%0 Conference Paper %T Connecting Sphere Manifolds Hierarchically for Regularization %A Damien Scieur %A Youngsung Kim %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-scieur21a %I PMLR %P 9399--9409 %U https://proceedings.mlr.press/v139/scieur21a.html %V 139 %X This paper considers classification problems with hierarchically organized classes. We force the classifier (hyperplane) of each class to belong to a sphere manifold, whose center is the classifier of its super-class. Then, individual sphere manifolds are connected based on their hierarchical relations. Our technique replaces the last layer of a neural network by combining a spherical fully-connected layer with a hierarchical layer. This regularization is shown to improve the performance of widely used deep neural network architectures (ResNet and DenseNet) on publicly available datasets (CIFAR100, CUB200, Stanford dogs, Stanford cars, and Tiny-ImageNet).
APA
Scieur, D. & Kim, Y.. (2021). Connecting Sphere Manifolds Hierarchically for Regularization. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:9399-9409 Available from https://proceedings.mlr.press/v139/scieur21a.html.

Related Material