[edit]

# Non-asymptotic approximations of neural networks by Gaussian processes

*Proceedings of Thirty Fourth Conference on Learning Theory*, PMLR 134:1754-1775, 2021.

#### Abstract

We study the extent to which wide neural networks may be approximated by Gaussian processes, when initialized with random weights. It is a well-established fact that as the width of a network goes to infinity, its law converges to that of a Gaussian process. We make this quantitative by establishing explicit convergence rates for the central limit theorem in an infinite-dimensional functional space, metrized with a natural transportation distance. We identify two regimes of interest; when the activation function is polynomial, its degree determines the rate of convergence, while for non-polynomial activations, the rate is governed by the smoothness of the function.