Multiclass Neural Network Minimization via Tropical Newton Polytope Approximation

Georgios Smyrnis, Petros Maragos
Proceedings of the 37th International Conference on Machine Learning, PMLR 119:9068-9077, 2020.

Abstract

The field of tropical algebra is closely linked with the domain of neural networks with piecewise linear activations, since their output can be described via tropical polynomials in the max-plus semiring. In this work, we attempt to make use of methods stemming from a form of approximate division of such polynomials, which relies on the approximation of their Newton Polytopes, in order to minimize networks trained for multiclass classification problems. We make theoretical contributions in this domain, by proposing and analyzing methods which seek to reduce the size of such networks. In addition, we make experimental evaluations on the MNIST and Fashion-MNIST datasets, with our results demonstrating a significant reduction in network size, while retaining adequate performance.

Cite this Paper

BibTeX
@InProceedings{pmlr-v119-smyrnis20a, title = {Multiclass Neural Network Minimization via Tropical {N}ewton Polytope Approximation}, author = {Smyrnis, Georgios and Maragos, Petros}, booktitle = {Proceedings of the 37th International Conference on Machine Learning}, pages = {9068--9077}, year = {2020}, editor = {III, Hal Daumé and Singh, Aarti}, volume = {119}, series = {Proceedings of Machine Learning Research}, month = {13--18 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v119/smyrnis20a/smyrnis20a.pdf}, url = {https://proceedings.mlr.press/v119/smyrnis20a.html}, abstract = {The field of tropical algebra is closely linked with the domain of neural networks with piecewise linear activations, since their output can be described via tropical polynomials in the max-plus semiring. In this work, we attempt to make use of methods stemming from a form of approximate division of such polynomials, which relies on the approximation of their Newton Polytopes, in order to minimize networks trained for multiclass classification problems. We make theoretical contributions in this domain, by proposing and analyzing methods which seek to reduce the size of such networks. In addition, we make experimental evaluations on the MNIST and Fashion-MNIST datasets, with our results demonstrating a significant reduction in network size, while retaining adequate performance.} }
Endnote
%0 Conference Paper %T Multiclass Neural Network Minimization via Tropical Newton Polytope Approximation %A Georgios Smyrnis %A Petros Maragos %B Proceedings of the 37th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Hal Daumé III %E Aarti Singh %F pmlr-v119-smyrnis20a %I PMLR %P 9068--9077 %U https://proceedings.mlr.press/v119/smyrnis20a.html %V 119 %X The field of tropical algebra is closely linked with the domain of neural networks with piecewise linear activations, since their output can be described via tropical polynomials in the max-plus semiring. In this work, we attempt to make use of methods stemming from a form of approximate division of such polynomials, which relies on the approximation of their Newton Polytopes, in order to minimize networks trained for multiclass classification problems. We make theoretical contributions in this domain, by proposing and analyzing methods which seek to reduce the size of such networks. In addition, we make experimental evaluations on the MNIST and Fashion-MNIST datasets, with our results demonstrating a significant reduction in network size, while retaining adequate performance.
APA
Smyrnis, G. & Maragos, P.. (2020). Multiclass Neural Network Minimization via Tropical Newton Polytope Approximation. Proceedings of the 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research 119:9068-9077 Available from https://proceedings.mlr.press/v119/smyrnis20a.html.

Related Material