A General Representation Learning Framework with Generalization Performance Guarantees

Junbiao Cui, Jianqing Liang, Qin Yue, Jiye Liang
Proceedings of the 40th International Conference on Machine Learning, PMLR 202:6522-6544, 2023.

Abstract

The generalization performance of machine learning methods depends heavily on the quality of data representation. However, existing researches rarely consider representation learning from the perspective of generalization error. In this paper, we prove that generalization error of representation learning function can be estimated effectively by solving two convex optimization problems. Based on it, we propose a general representation learning framework. And then, we apply the proposed framework to two most commonly used nonlinear mapping methods, i.e., kernel based method and deep neural network (DNN), and thus design a kernel selection method and a DNN boosting framework, correspondingly. Finally, extensive experiments verify the effectiveness of the proposed methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v202-cui23c, title = {A General Representation Learning Framework with Generalization Performance Guarantees}, author = {Cui, Junbiao and Liang, Jianqing and Yue, Qin and Liang, Jiye}, booktitle = {Proceedings of the 40th International Conference on Machine Learning}, pages = {6522--6544}, year = {2023}, editor = {Krause, Andreas and Brunskill, Emma and Cho, Kyunghyun and Engelhardt, Barbara and Sabato, Sivan and Scarlett, Jonathan}, volume = {202}, series = {Proceedings of Machine Learning Research}, month = {23--29 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v202/cui23c/cui23c.pdf}, url = {https://proceedings.mlr.press/v202/cui23c.html}, abstract = {The generalization performance of machine learning methods depends heavily on the quality of data representation. However, existing researches rarely consider representation learning from the perspective of generalization error. In this paper, we prove that generalization error of representation learning function can be estimated effectively by solving two convex optimization problems. Based on it, we propose a general representation learning framework. And then, we apply the proposed framework to two most commonly used nonlinear mapping methods, i.e., kernel based method and deep neural network (DNN), and thus design a kernel selection method and a DNN boosting framework, correspondingly. Finally, extensive experiments verify the effectiveness of the proposed methods.} }
Endnote
%0 Conference Paper %T A General Representation Learning Framework with Generalization Performance Guarantees %A Junbiao Cui %A Jianqing Liang %A Qin Yue %A Jiye Liang %B Proceedings of the 40th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2023 %E Andreas Krause %E Emma Brunskill %E Kyunghyun Cho %E Barbara Engelhardt %E Sivan Sabato %E Jonathan Scarlett %F pmlr-v202-cui23c %I PMLR %P 6522--6544 %U https://proceedings.mlr.press/v202/cui23c.html %V 202 %X The generalization performance of machine learning methods depends heavily on the quality of data representation. However, existing researches rarely consider representation learning from the perspective of generalization error. In this paper, we prove that generalization error of representation learning function can be estimated effectively by solving two convex optimization problems. Based on it, we propose a general representation learning framework. And then, we apply the proposed framework to two most commonly used nonlinear mapping methods, i.e., kernel based method and deep neural network (DNN), and thus design a kernel selection method and a DNN boosting framework, correspondingly. Finally, extensive experiments verify the effectiveness of the proposed methods.
APA
Cui, J., Liang, J., Yue, Q. & Liang, J.. (2023). A General Representation Learning Framework with Generalization Performance Guarantees. Proceedings of the 40th International Conference on Machine Learning, in Proceedings of Machine Learning Research 202:6522-6544 Available from https://proceedings.mlr.press/v202/cui23c.html.

Related Material