More Flexible PAC-Bayesian Meta-Learning by Learning Learning Algorithms

Hossein Zakerinia, Amin Behjati, Christoph H. Lampert
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:58122-58139, 2024.

Abstract

We introduce a new framework for studying meta-learning methods using PAC-Bayesian theory. Its main advantage over previous work is that it allows for more flexibility in how the transfer of knowledge between tasks is realized. For previous approaches, this could only happen indirectly, by means of learning prior distributions over models. In contrast, the new generalization bounds that we prove express the process of meta-learning much more directly as learning the learning algorithm that should be used for future tasks. The flexibility of our framework makes it suitable to analyze a wide range of meta-learning mechanisms and even design new mechanisms. Other than our theoretical contributions we also show empirically that our framework improves the prediction quality in practical meta-learning mechanisms.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-zakerinia24a, title = {More Flexible {PAC}-{B}ayesian Meta-Learning by Learning Learning Algorithms}, author = {Zakerinia, Hossein and Behjati, Amin and Lampert, Christoph H.}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {58122--58139}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/zakerinia24a/zakerinia24a.pdf}, url = {https://proceedings.mlr.press/v235/zakerinia24a.html}, abstract = {We introduce a new framework for studying meta-learning methods using PAC-Bayesian theory. Its main advantage over previous work is that it allows for more flexibility in how the transfer of knowledge between tasks is realized. For previous approaches, this could only happen indirectly, by means of learning prior distributions over models. In contrast, the new generalization bounds that we prove express the process of meta-learning much more directly as learning the learning algorithm that should be used for future tasks. The flexibility of our framework makes it suitable to analyze a wide range of meta-learning mechanisms and even design new mechanisms. Other than our theoretical contributions we also show empirically that our framework improves the prediction quality in practical meta-learning mechanisms.} }
Endnote
%0 Conference Paper %T More Flexible PAC-Bayesian Meta-Learning by Learning Learning Algorithms %A Hossein Zakerinia %A Amin Behjati %A Christoph H. Lampert %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-zakerinia24a %I PMLR %P 58122--58139 %U https://proceedings.mlr.press/v235/zakerinia24a.html %V 235 %X We introduce a new framework for studying meta-learning methods using PAC-Bayesian theory. Its main advantage over previous work is that it allows for more flexibility in how the transfer of knowledge between tasks is realized. For previous approaches, this could only happen indirectly, by means of learning prior distributions over models. In contrast, the new generalization bounds that we prove express the process of meta-learning much more directly as learning the learning algorithm that should be used for future tasks. The flexibility of our framework makes it suitable to analyze a wide range of meta-learning mechanisms and even design new mechanisms. Other than our theoretical contributions we also show empirically that our framework improves the prediction quality in practical meta-learning mechanisms.
APA
Zakerinia, H., Behjati, A. & Lampert, C.H.. (2024). More Flexible PAC-Bayesian Meta-Learning by Learning Learning Algorithms. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:58122-58139 Available from https://proceedings.mlr.press/v235/zakerinia24a.html.

Related Material