A Unified Framework for Learning with Nonlinear Model Classes from Arbitrary Linear Samples

Ben Adcock, Juan M. Cardenas, Nick Dexter
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:169-202, 2024.

Abstract

This work considers the fundamental problem of learning an unknown object from training data using a given model class. We introduce a framework that allows for objects in arbitrary Hilbert spaces, general types of (random) linear measurements as training data and general types of nonlinear model classes. We establish a series of learning guarantees for this framework, which provide explicit relations between the amount of training data and the model class to ensure near-best generalization bounds. In doing so, we introduce the key notion of the variation of a model class with respect to a distribution of sampling operators. We show that this framework can accommodate many different types of well-known problems of interest, such as matrix sketching by random sampling, compressed sensing with isotropic vectors, active learning in regression and compressed sensing with generative models. In all cases, known results become straightforward corollaries of our general theory. Hence, this work provides a powerful framework for studying and analyzing many different types of learning problems.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-adcock24a, title = {A Unified Framework for Learning with Nonlinear Model Classes from Arbitrary Linear Samples}, author = {Adcock, Ben and Cardenas, Juan M. and Dexter, Nick}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {169--202}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/adcock24a/adcock24a.pdf}, url = {https://proceedings.mlr.press/v235/adcock24a.html}, abstract = {This work considers the fundamental problem of learning an unknown object from training data using a given model class. We introduce a framework that allows for objects in arbitrary Hilbert spaces, general types of (random) linear measurements as training data and general types of nonlinear model classes. We establish a series of learning guarantees for this framework, which provide explicit relations between the amount of training data and the model class to ensure near-best generalization bounds. In doing so, we introduce the key notion of the variation of a model class with respect to a distribution of sampling operators. We show that this framework can accommodate many different types of well-known problems of interest, such as matrix sketching by random sampling, compressed sensing with isotropic vectors, active learning in regression and compressed sensing with generative models. In all cases, known results become straightforward corollaries of our general theory. Hence, this work provides a powerful framework for studying and analyzing many different types of learning problems.} }
Endnote
%0 Conference Paper %T A Unified Framework for Learning with Nonlinear Model Classes from Arbitrary Linear Samples %A Ben Adcock %A Juan M. Cardenas %A Nick Dexter %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-adcock24a %I PMLR %P 169--202 %U https://proceedings.mlr.press/v235/adcock24a.html %V 235 %X This work considers the fundamental problem of learning an unknown object from training data using a given model class. We introduce a framework that allows for objects in arbitrary Hilbert spaces, general types of (random) linear measurements as training data and general types of nonlinear model classes. We establish a series of learning guarantees for this framework, which provide explicit relations between the amount of training data and the model class to ensure near-best generalization bounds. In doing so, we introduce the key notion of the variation of a model class with respect to a distribution of sampling operators. We show that this framework can accommodate many different types of well-known problems of interest, such as matrix sketching by random sampling, compressed sensing with isotropic vectors, active learning in regression and compressed sensing with generative models. In all cases, known results become straightforward corollaries of our general theory. Hence, this work provides a powerful framework for studying and analyzing many different types of learning problems.
APA
Adcock, B., Cardenas, J.M. & Dexter, N.. (2024). A Unified Framework for Learning with Nonlinear Model Classes from Arbitrary Linear Samples. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:169-202 Available from https://proceedings.mlr.press/v235/adcock24a.html.

Related Material