Bounds on the Approximation Power of Feedforward Neural Networks

Mohammad Mehrabi, Aslan Tchamkerten, MANSOOR YOUSEFI
; Proceedings of the 35th International Conference on Machine Learning, PMLR 80:3453-3461, 2018.

Abstract

The approximation power of general feedforward neural networks with piecewise linear activation functions is investigated. First, lower bounds on the size of a network are established in terms of the approximation error and network depth and width. These bounds improve upon state-of-the-art bounds for certain classes of functions, such as strongly convex functions. Second, an upper bound is established on the difference of two neural networks with identical weights but different activation functions.

Cite this Paper


BibTeX
@InProceedings{pmlr-v80-mehrabi18a, title = {Bounds on the Approximation Power of Feedforward Neural Networks}, author = {Mehrabi, Mohammad and Tchamkerten, Aslan and YOUSEFI, MANSOOR}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {3453--3461}, year = {2018}, editor = {Jennifer Dy and Andreas Krause}, volume = {80}, series = {Proceedings of Machine Learning Research}, address = {Stockholmsmässan, Stockholm Sweden}, month = {10--15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v80/mehrabi18a/mehrabi18a.pdf}, url = {http://proceedings.mlr.press/v80/mehrabi18a.html}, abstract = {The approximation power of general feedforward neural networks with piecewise linear activation functions is investigated. First, lower bounds on the size of a network are established in terms of the approximation error and network depth and width. These bounds improve upon state-of-the-art bounds for certain classes of functions, such as strongly convex functions. Second, an upper bound is established on the difference of two neural networks with identical weights but different activation functions.} }
Endnote
%0 Conference Paper %T Bounds on the Approximation Power of Feedforward Neural Networks %A Mohammad Mehrabi %A Aslan Tchamkerten %A MANSOOR YOUSEFI %B Proceedings of the 35th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jennifer Dy %E Andreas Krause %F pmlr-v80-mehrabi18a %I PMLR %J Proceedings of Machine Learning Research %P 3453--3461 %U http://proceedings.mlr.press %V 80 %W PMLR %X The approximation power of general feedforward neural networks with piecewise linear activation functions is investigated. First, lower bounds on the size of a network are established in terms of the approximation error and network depth and width. These bounds improve upon state-of-the-art bounds for certain classes of functions, such as strongly convex functions. Second, an upper bound is established on the difference of two neural networks with identical weights but different activation functions.
APA
Mehrabi, M., Tchamkerten, A. & YOUSEFI, M.. (2018). Bounds on the Approximation Power of Feedforward Neural Networks. Proceedings of the 35th International Conference on Machine Learning, in PMLR 80:3453-3461

Related Material