Improving Bi-level Optimization Based Methods with Inspiration from Humans’ Classroom Study Techniques

Pengtao Xie
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:38137-38186, 2023.

Abstract

In humans’ classroom learning, many effective study techniques (e.g., the Feynman technique, peer questioning, etc.) have been developed to improve learning outcomes. We are interested in investigating whether these techniques can inspire the development of ML training strategies to improve bi-level optimization (BLO) based methods. Towards this goal, we develop a general framework, Skillearn, which consists of basic elements such as learners, interaction functions, learning stages, etc. These elements can be flexibly configured to create various training strategies, each emulating a study technique of humans. In case studies, we apply Skillearn to create new training strategies, by emulating the Feynman technique and peer questioning, which are two broadly adopted techniques in humans’ classroom learning. These training strategies are used for improving two BLO based applications including neural architecture search and data weighting. Experiments on various datasets demonstrate the effectiveness of our methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-xie23a, title = {Improving Bi-level Optimization Based Methods with Inspiration from Humans’ Classroom Study Techniques}, author = {Xie, Pengtao}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {38137--38186}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/xie23a/xie23a.pdf}, url = {https://proceedings.mlr.press/v202/xie23a.html}, abstract = {In humans’ classroom learning, many effective study techniques (e.g., the Feynman technique, peer questioning, etc.) have been developed to improve learning outcomes. We are interested in investigating whether these techniques can inspire the development of ML training strategies to improve bi-level optimization (BLO) based methods. Towards this goal, we develop a general framework, Skillearn, which consists of basic elements such as learners, interaction functions, learning stages, etc. These elements can be flexibly configured to create various training strategies, each emulating a study technique of humans. In case studies, we apply Skillearn to create new training strategies, by emulating the Feynman technique and peer questioning, which are two broadly adopted techniques in humans’ classroom learning. These training strategies are used for improving two BLO based applications including neural architecture search and data weighting. Experiments on various datasets demonstrate the effectiveness of our methods.} }
Endnote
%0 Conference Paper %T Improving Bi-level Optimization Based Methods with Inspiration from Humans’ Classroom Study Techniques %A Pengtao Xie %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-xie23a %I PMLR %P 38137--38186 %U https://proceedings.mlr.press/v202/xie23a.html %V 202 %X In humans’ classroom learning, many effective study techniques (e.g., the Feynman technique, peer questioning, etc.) have been developed to improve learning outcomes. We are interested in investigating whether these techniques can inspire the development of ML training strategies to improve bi-level optimization (BLO) based methods. Towards this goal, we develop a general framework, Skillearn, which consists of basic elements such as learners, interaction functions, learning stages, etc. These elements can be flexibly configured to create various training strategies, each emulating a study technique of humans. In case studies, we apply Skillearn to create new training strategies, by emulating the Feynman technique and peer questioning, which are two broadly adopted techniques in humans’ classroom learning. These training strategies are used for improving two BLO based applications including neural architecture search and data weighting. Experiments on various datasets demonstrate the effectiveness of our methods.
APA
Xie, P.. (2023). Improving Bi-level Optimization Based Methods with Inspiration from Humans’ Classroom Study Techniques. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:38137-38186 Available from https://proceedings.mlr.press/v202/xie23a.html.

Related Material