NeurIPS’22 Cross-Domain MetaDL competition: Design and baseline results

Dustin Carrión-Ojeda, Hong Chen, Adrian El Baz, Segio Escalera, Chaoyu Guan, Isabelle Guyon, Ihsan Ullah, Xin Wang, Wenwu Zhu
ECMLPKDD Workshop on Meta-Knowledge Transfer, PMLR 191:24-37, 2022.

Abstract

We present the design and baseline results for a new challenge in the ChaLearn meta-learning series, accepted at NeurIPS’22, focusing on “cross-domain” meta-learning. Meta-learning aims to leverage experience gained from previous tasks to solve new tasks efficiently (i.e., with better performance, little training data, and/or modest computational resources). While previous challenges in the series focused on within-domain few-shot learning problems, with the aim of learning efficiently N-way k-shot tasks (i.e., N class classification problems with k training examples), this competition challenges the participants to solve “any-way” and “any-shot” problems drawn from various domains (healthcare, ecology, biology, manufacturing, and others), chosen for their humanitarian and societal impact. To that end, we created Meta-Album, a meta-dataset of 40 image classification datasets from 10 domains, from which we carve out tasks with any number of “ways” (within the range 2-20) and any number of “shots” (within the range 1-20). The competition is with code submission, fully blind-tested on the CodaLab challenge platform. The code of the winners will be open-sourced, enabling the deployment of automated machine learning solutions for few-shot image classification across several domains.

Cite this Paper

BibTeX
@InProceedings{pmlr-v191-carrion-ojeda22a, title = {NeurIPS’22 Cross-Domain MetaDL competition: Design and baseline results}, author = {Carri\'on-Ojeda, Dustin and Chen, Hong and El Baz, Adrian and Escalera, Segio and Guan, Chaoyu and Guyon, Isabelle and Ullah, Ihsan and Wang, Xin and Zhu, Wenwu}, booktitle = {ECMLPKDD Workshop on Meta-Knowledge Transfer}, pages = {24--37}, year = {2022}, editor = {Brazdil, Pavel and van Rijn, Jan N. and Gouk, Henry and Mohr, Felix}, volume = {191}, series = {Proceedings of Machine Learning Research}, month = {23 Sep}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v191/carrion-ojeda22a/carrion-ojeda22a.pdf}, url = {https://proceedings.mlr.press/v191/carrion-ojeda22a.html}, abstract = {We present the design and baseline results for a new challenge in the ChaLearn meta-learning series, accepted at NeurIPS’22, focusing on “cross-domain” meta-learning. Meta-learning aims to leverage experience gained from previous tasks to solve new tasks efficiently (i.e., with better performance, little training data, and/or modest computational resources). While previous challenges in the series focused on within-domain few-shot learning problems, with the aim of learning efficiently N-way k-shot tasks (i.e., N class classification problems with k training examples), this competition challenges the participants to solve “any-way” and “any-shot” problems drawn from various domains (healthcare, ecology, biology, manufacturing, and others), chosen for their humanitarian and societal impact. To that end, we created Meta-Album, a meta-dataset of 40 image classification datasets from 10 domains, from which we carve out tasks with any number of “ways” (within the range 2-20) and any number of “shots” (within the range 1-20). The competition is with code submission, fully blind-tested on the CodaLab challenge platform. The code of the winners will be open-sourced, enabling the deployment of automated machine learning solutions for few-shot image classification across several domains.} }
Endnote
%0 Conference Paper %T NeurIPS’22 Cross-Domain MetaDL competition: Design and baseline results %A Dustin Carrión-Ojeda %A Hong Chen %A Adrian El Baz %A Segio Escalera %A Chaoyu Guan %A Isabelle Guyon %A Ihsan Ullah %A Xin Wang %A Wenwu Zhu %B ECMLPKDD Workshop on Meta-Knowledge Transfer %C Proceedings of Machine Learning Research %D 2022 %E Pavel Brazdil %E Jan N. van Rijn %E Henry Gouk %E Felix Mohr %F pmlr-v191-carrion-ojeda22a %I PMLR %P 24--37 %U https://proceedings.mlr.press/v191/carrion-ojeda22a.html %V 191 %X We present the design and baseline results for a new challenge in the ChaLearn meta-learning series, accepted at NeurIPS’22, focusing on “cross-domain” meta-learning. Meta-learning aims to leverage experience gained from previous tasks to solve new tasks efficiently (i.e., with better performance, little training data, and/or modest computational resources). While previous challenges in the series focused on within-domain few-shot learning problems, with the aim of learning efficiently N-way k-shot tasks (i.e., N class classification problems with k training examples), this competition challenges the participants to solve “any-way” and “any-shot” problems drawn from various domains (healthcare, ecology, biology, manufacturing, and others), chosen for their humanitarian and societal impact. To that end, we created Meta-Album, a meta-dataset of 40 image classification datasets from 10 domains, from which we carve out tasks with any number of “ways” (within the range 2-20) and any number of “shots” (within the range 1-20). The competition is with code submission, fully blind-tested on the CodaLab challenge platform. The code of the winners will be open-sourced, enabling the deployment of automated machine learning solutions for few-shot image classification across several domains.
APA
Carrión-Ojeda, D., Chen, H., El Baz, A., Escalera, S., Guan, C., Guyon, I., Ullah, I., Wang, X. & Zhu, W.. (2022). NeurIPS’22 Cross-Domain MetaDL competition: Design and baseline results. ECMLPKDD Workshop on Meta-Knowledge Transfer, in Proceedings of Machine Learning Research 191:24-37 Available from https://proceedings.mlr.press/v191/carrion-ojeda22a.html.

Related Material