Unsupervised Embedding Adaptation via Early-Stage Feature Reconstruction for Few-Shot Classification

Dong Hoon Lee, Sae-Young Chung
Proceedings of the 38th International Conference on Machine Learning, PMLR 139:6098-6108, 2021.

Abstract

We propose unsupervised embedding adaptation for the downstream few-shot classification task. Based on findings that deep neural networks learn to generalize before memorizing, we develop Early-Stage Feature Reconstruction (ESFR) — a novel adaptation scheme with feature reconstruction and dimensionality-driven early stopping that finds generalizable features. Incorporating ESFR consistently improves the performance of baseline methods on all standard settings, including the recently proposed transductive method. ESFR used in conjunction with the transductive method further achieves state-of-the-art performance on mini-ImageNet, tiered-ImageNet, and CUB; especially with 1.2% 2.0% improvements in accuracy over the previous best performing method on 1-shot setting.

Cite this Paper

BibTeX
@InProceedings{pmlr-v139-lee21d, title = {Unsupervised Embedding Adaptation via Early-Stage Feature Reconstruction for Few-Shot Classification}, author = {Lee, Dong Hoon and Chung, Sae-Young}, booktitle = {Proceedings of the 38th International Conference on Machine Learning}, pages = {6098--6108}, year = {2021}, editor = {Meila, Marina and Zhang, Tong}, volume = {139}, series = {Proceedings of Machine Learning Research}, month = {18--24 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v139/lee21d/lee21d.pdf}, url = {https://proceedings.mlr.press/v139/lee21d.html}, abstract = {We propose unsupervised embedding adaptation for the downstream few-shot classification task. Based on findings that deep neural networks learn to generalize before memorizing, we develop Early-Stage Feature Reconstruction (ESFR) — a novel adaptation scheme with feature reconstruction and dimensionality-driven early stopping that finds generalizable features. Incorporating ESFR consistently improves the performance of baseline methods on all standard settings, including the recently proposed transductive method. ESFR used in conjunction with the transductive method further achieves state-of-the-art performance on mini-ImageNet, tiered-ImageNet, and CUB; especially with 1.2% 2.0% improvements in accuracy over the previous best performing method on 1-shot setting.} }
Endnote
%0 Conference Paper %T Unsupervised Embedding Adaptation via Early-Stage Feature Reconstruction for Few-Shot Classification %A Dong Hoon Lee %A Sae-Young Chung %B Proceedings of the 38th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2021 %E Marina Meila %E Tong Zhang %F pmlr-v139-lee21d %I PMLR %P 6098--6108 %U https://proceedings.mlr.press/v139/lee21d.html %V 139 %X We propose unsupervised embedding adaptation for the downstream few-shot classification task. Based on findings that deep neural networks learn to generalize before memorizing, we develop Early-Stage Feature Reconstruction (ESFR) — a novel adaptation scheme with feature reconstruction and dimensionality-driven early stopping that finds generalizable features. Incorporating ESFR consistently improves the performance of baseline methods on all standard settings, including the recently proposed transductive method. ESFR used in conjunction with the transductive method further achieves state-of-the-art performance on mini-ImageNet, tiered-ImageNet, and CUB; especially with 1.2% 2.0% improvements in accuracy over the previous best performing method on 1-shot setting.
APA
Lee, D.H. & Chung, S.. (2021). Unsupervised Embedding Adaptation via Early-Stage Feature Reconstruction for Few-Shot Classification. Proceedings of the 38th International Conference on Machine Learning, in Proceedings of Machine Learning Research 139:6098-6108 Available from https://proceedings.mlr.press/v139/lee21d.html.

Related Material