Towards Empirical Process Theory for Vector-Valued Functions: Metric Entropy of Smooth Function Classes

Junhyung Park, Krikamol Muandet
Proceedings of The 34th International Conference on Algorithmic Learning Theory, PMLR 201:1216-1260, 2023.

Abstract

This paper provides some first steps in developing empirical process theory for functions taking values in a vector space. Our main results provide bounds on the entropy of classes of smooth functions taking values in a Hilbert space, by leveraging theory from differential calculus of vector-valued functions and fractal dimension theory of metric spaces. We demonstrate how these entropy bounds can be used to show the uniform law of large numbers and asymptotic equicontinuity of the function classes, and also apply it to statistical learning theory in which the output space is a Hilbert space. We conclude with a discussion on the extension of Rademacher complexities to vector-valued function classes.

Cite this Paper

BibTeX
@InProceedings{pmlr-v201-park23a, title = {Towards Empirical Process Theory for Vector-Valued Functions: Metric Entropy of Smooth Function Classes}, author = {Park, Junhyung and Muandet, Krikamol}, booktitle = {Proceedings of The 34th International Conference on Algorithmic Learning Theory}, pages = {1216--1260}, year = {2023}, editor = {Agrawal, Shipra and Orabona, Francesco}, volume = {201}, series = {Proceedings of Machine Learning Research}, month = {20 Feb--23 Feb}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v201/park23a/park23a.pdf}, url = {https://proceedings.mlr.press/v201/park23a.html}, abstract = {This paper provides some first steps in developing empirical process theory for functions taking values in a vector space. Our main results provide bounds on the entropy of classes of smooth functions taking values in a Hilbert space, by leveraging theory from differential calculus of vector-valued functions and fractal dimension theory of metric spaces. We demonstrate how these entropy bounds can be used to show the uniform law of large numbers and asymptotic equicontinuity of the function classes, and also apply it to statistical learning theory in which the output space is a Hilbert space. We conclude with a discussion on the extension of Rademacher complexities to vector-valued function classes. } }
Endnote
%0 Conference Paper %T Towards Empirical Process Theory for Vector-Valued Functions: Metric Entropy of Smooth Function Classes %A Junhyung Park %A Krikamol Muandet %B Proceedings of The 34th International Conference on Algorithmic Learning Theory %C Proceedings of Machine Learning Research %D 2023 %E Shipra Agrawal %E Francesco Orabona %F pmlr-v201-park23a %I PMLR %P 1216--1260 %U https://proceedings.mlr.press/v201/park23a.html %V 201 %X This paper provides some first steps in developing empirical process theory for functions taking values in a vector space. Our main results provide bounds on the entropy of classes of smooth functions taking values in a Hilbert space, by leveraging theory from differential calculus of vector-valued functions and fractal dimension theory of metric spaces. We demonstrate how these entropy bounds can be used to show the uniform law of large numbers and asymptotic equicontinuity of the function classes, and also apply it to statistical learning theory in which the output space is a Hilbert space. We conclude with a discussion on the extension of Rademacher complexities to vector-valued function classes.
APA
Park, J. & Muandet, K.. (2023). Towards Empirical Process Theory for Vector-Valued Functions: Metric Entropy of Smooth Function Classes. Proceedings of The 34th International Conference on Algorithmic Learning Theory, in Proceedings of Machine Learning Research 201:1216-1260 Available from https://proceedings.mlr.press/v201/park23a.html.

Related Material