The Non-linear $F$-Design and Applications to Interactive Learning

Alekh Agarwal, Jian Qian, Alexander Rakhlin, Tong Zhang
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:362-396, 2024.

Abstract

We propose a generalization of the classical G-optimal design concept to non-linear function classes. The criterion, termed F -design, coincides with G-design in the linear case. We compute the value of the optimal design, termed the F-condition number, for several non-linear function classes. We further provide algorithms to construct designs with a bounded F -condition number. Finally, we employ the F-design in a variety of interactive machine learning tasks, where the design is naturally useful for data collection or exploration. We show that in four diverse settings of confidence band construction, contextual bandits, model-free reinforcement learning, and active learning, F-design can be combined with existing approaches in a black-box manner to yield state-of-the-art results in known problem settings as well as to generalize to novel ones.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-agarwal24e, title = {The Non-linear $F$-Design and Applications to Interactive Learning}, author = {Agarwal, Alekh and Qian, Jian and Rakhlin, Alexander and Zhang, Tong}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {362--396}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/agarwal24e/agarwal24e.pdf}, url = {https://proceedings.mlr.press/v235/agarwal24e.html}, abstract = {We propose a generalization of the classical G-optimal design concept to non-linear function classes. The criterion, termed F -design, coincides with G-design in the linear case. We compute the value of the optimal design, termed the F-condition number, for several non-linear function classes. We further provide algorithms to construct designs with a bounded F -condition number. Finally, we employ the F-design in a variety of interactive machine learning tasks, where the design is naturally useful for data collection or exploration. We show that in four diverse settings of confidence band construction, contextual bandits, model-free reinforcement learning, and active learning, F-design can be combined with existing approaches in a black-box manner to yield state-of-the-art results in known problem settings as well as to generalize to novel ones.} }
Endnote
%0 Conference Paper %T The Non-linear $F$-Design and Applications to Interactive Learning %A Alekh Agarwal %A Jian Qian %A Alexander Rakhlin %A Tong Zhang %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-agarwal24e %I PMLR %P 362--396 %U https://proceedings.mlr.press/v235/agarwal24e.html %V 235 %X We propose a generalization of the classical G-optimal design concept to non-linear function classes. The criterion, termed F -design, coincides with G-design in the linear case. We compute the value of the optimal design, termed the F-condition number, for several non-linear function classes. We further provide algorithms to construct designs with a bounded F -condition number. Finally, we employ the F-design in a variety of interactive machine learning tasks, where the design is naturally useful for data collection or exploration. We show that in four diverse settings of confidence band construction, contextual bandits, model-free reinforcement learning, and active learning, F-design can be combined with existing approaches in a black-box manner to yield state-of-the-art results in known problem settings as well as to generalize to novel ones.
APA
Agarwal, A., Qian, J., Rakhlin, A. & Zhang, T.. (2024). The Non-linear $F$-Design and Applications to Interactive Learning. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:362-396 Available from https://proceedings.mlr.press/v235/agarwal24e.html.

Related Material