Gradient-Based Meta-Learning with Learned Layerwise Metric and Subspace

Yoonho Lee, Seungjin Choi
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:2927-2936, 2018.

Abstract

Gradient-based meta-learning methods leverage gradient descent to learn the commonalities among various tasks. While previous such methods have been successful in meta-learning tasks, they resort to simple gradient descent during meta-testing. Our primary contribution is the MT-net, which enables the meta-learner to learn on each layer’s activation space a subspace that the task-specific learner performs gradient descent on. Additionally, a task-specific learner of an MT-net performs gradient descent with respect to a meta-learned distance metric, which warps the activation space to be more sensitive to task identity. We demonstrate that the dimension of this learned subspace reflects the complexity of the task-specific learner’s adaptation task, and also that our model is less sensitive to the choice of initial learning rates than previous gradient-based meta-learning methods. Our method achieves state-of-the-art or comparable performance on few-shot classification and regression tasks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v80-lee18a, title = {Gradient-Based Meta-Learning with Learned Layerwise Metric and Subspace}, author = {Lee, Yoonho and Choi, Seungjin}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {2927--2936}, year = {2018}, editor = {Dy, Jennifer and Krause, Andreas}, volume = {80}, series = {Proceedings of Machine Learning Research}, month = {10--15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v80/lee18a/lee18a.pdf}, url = {https://proceedings.mlr.press/v80/lee18a.html}, abstract = {Gradient-based meta-learning methods leverage gradient descent to learn the commonalities among various tasks. While previous such methods have been successful in meta-learning tasks, they resort to simple gradient descent during meta-testing. Our primary contribution is the MT-net, which enables the meta-learner to learn on each layer’s activation space a subspace that the task-specific learner performs gradient descent on. Additionally, a task-specific learner of an MT-net performs gradient descent with respect to a meta-learned distance metric, which warps the activation space to be more sensitive to task identity. We demonstrate that the dimension of this learned subspace reflects the complexity of the task-specific learner’s adaptation task, and also that our model is less sensitive to the choice of initial learning rates than previous gradient-based meta-learning methods. Our method achieves state-of-the-art or comparable performance on few-shot classification and regression tasks.} }
Endnote
%0 Conference Paper %T Gradient-Based Meta-Learning with Learned Layerwise Metric and Subspace %A Yoonho Lee %A Seungjin Choi %B Proceedings of the 35th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jennifer Dy %E Andreas Krause %F pmlr-v80-lee18a %I PMLR %P 2927--2936 %U https://proceedings.mlr.press/v80/lee18a.html %V 80 %X Gradient-based meta-learning methods leverage gradient descent to learn the commonalities among various tasks. While previous such methods have been successful in meta-learning tasks, they resort to simple gradient descent during meta-testing. Our primary contribution is the MT-net, which enables the meta-learner to learn on each layer’s activation space a subspace that the task-specific learner performs gradient descent on. Additionally, a task-specific learner of an MT-net performs gradient descent with respect to a meta-learned distance metric, which warps the activation space to be more sensitive to task identity. We demonstrate that the dimension of this learned subspace reflects the complexity of the task-specific learner’s adaptation task, and also that our model is less sensitive to the choice of initial learning rates than previous gradient-based meta-learning methods. Our method achieves state-of-the-art or comparable performance on few-shot classification and regression tasks.
APA
Lee, Y. & Choi, S.. (2018). Gradient-Based Meta-Learning with Learned Layerwise Metric and Subspace. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research 80:2927-2936 Available from https://proceedings.mlr.press/v80/lee18a.html.

Related Material