A Brief Review of the ChaLearn AutoML Challenge: Any-time Any-dataset Learning Without Human Intervention

Isabelle Guyon, Imad Chaabane, Hugo Jair Escalante, Sergio Escalera, Damir Jajetic, James Robert Lloyd, Núria Macià, Bisakha Ray, Lukasz Romaszko, Michèle Sebag, Alexander Statnikov, Sébastien Treguer, Evelyne Viegas
Proceedings of the Workshop on Automatic Machine Learning, PMLR 64:21-30, 2016.

Abstract

The ChaLearn AutoML Challenge team conducted a large scale evaluation of fully automatic, black-box learning machines for feature-based classification and regression problems. The test bed was composed of 30 data sets from a wide variety of application domains and ranging across different types of complexity. Over five rounds, participants succeeded in delivering AutoML software capable of being trained and tested without human intervention. Although improvements can still be made to close the gap between human-tweaked and AutoML models, this challenge has been a leap forward in the field and its platform will remain available for post-challenge submissions at http://codalab.org/AutoML.

Cite this Paper

BibTeX
@InProceedings{pmlr-v64-guyon_review_2016, title = {A brief Review of the ChaLearn AutoML Challenge: Any-time Any-dataset Learning without Human Intervention}, author = {Guyon, Isabelle and Chaabane, Imad and Escalante, Hugo Jair and Escalera, Sergio and Jajetic, Damir and Lloyd, James Robert and Macià, Núria and Ray, Bisakha and Romaszko, Lukasz and Sebag, Michèle and Statnikov, Alexander and Treguer, Sébastien and Viegas, Evelyne}, booktitle = {Proceedings of the Workshop on Automatic Machine Learning}, pages = {21--30}, year = {2016}, editor = {Hutter, Frank and Kotthoff, Lars and Vanschoren, Joaquin}, volume = {64}, series = {Proceedings of Machine Learning Research}, address = {New York, New York, USA}, month = {24 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v64/guyon_review_2016.pdf}, url = {https://proceedings.mlr.press/v64/guyon_review_2016.html}, abstract = {The ChaLearn AutoML Challenge team conducted a large scale evaluation of fully automatic, black-box learning machines for feature-based classification and regression problems. The test bed was composed of 30 data sets from a wide variety of application domains and ranging across different types of complexity. Over five rounds, participants succeeded in delivering AutoML software capable of being trained and tested without human intervention. Although improvements can still be made to close the gap between human-tweaked and AutoML models, this challenge has been a leap forward in the field and its platform will remain available for post-challenge submissions at http://codalab.org/AutoML.} }
Endnote
%0 Conference Paper %T A Brief Review of the ChaLearn AutoML Challenge: Any-time Any-dataset Learning Without Human Intervention %A Isabelle Guyon %A Imad Chaabane %A Hugo Jair Escalante %A Sergio Escalera %A Damir Jajetic %A James Robert Lloyd %A Núria Macià %A Bisakha Ray %A Lukasz Romaszko %A Michèle Sebag %A Alexander Statnikov %A Sébastien Treguer %A Evelyne Viegas %B Proceedings of the Workshop on Automatic Machine Learning %C Proceedings of Machine Learning Research %D 2016 %E Frank Hutter %E Lars Kotthoff %E Joaquin Vanschoren %F pmlr-v64-guyon_review_2016 %I PMLR %P 21--30 %U https://proceedings.mlr.press/v64/guyon_review_2016.html %V 64 %X The ChaLearn AutoML Challenge team conducted a large scale evaluation of fully automatic, black-box learning machines for feature-based classification and regression problems. The test bed was composed of 30 data sets from a wide variety of application domains and ranging across different types of complexity. Over five rounds, participants succeeded in delivering AutoML software capable of being trained and tested without human intervention. Although improvements can still be made to close the gap between human-tweaked and AutoML models, this challenge has been a leap forward in the field and its platform will remain available for post-challenge submissions at http://codalab.org/AutoML.
RIS
TY - CPAPER TI - A Brief Review of the ChaLearn AutoML Challenge: Any-time Any-dataset Learning Without Human Intervention AU - Isabelle Guyon AU - Imad Chaabane AU - Hugo Jair Escalante AU - Sergio Escalera AU - Damir Jajetic AU - James Robert Lloyd AU - Núria Macià AU - Bisakha Ray AU - Lukasz Romaszko AU - Michèle Sebag AU - Alexander Statnikov AU - Sébastien Treguer AU - Evelyne Viegas BT - Proceedings of the Workshop on Automatic Machine Learning DA - 2016/12/04 ED - Frank Hutter ED - Lars Kotthoff ED - Joaquin Vanschoren ID - pmlr-v64-guyon_review_2016 PB - PMLR DP - Proceedings of Machine Learning Research VL - 64 SP - 21 EP - 30 L1 - http://proceedings.mlr.press/v64/guyon_review_2016.pdf UR - https://proceedings.mlr.press/v64/guyon_review_2016.html AB - The ChaLearn AutoML Challenge team conducted a large scale evaluation of fully automatic, black-box learning machines for feature-based classification and regression problems. The test bed was composed of 30 data sets from a wide variety of application domains and ranging across different types of complexity. Over five rounds, participants succeeded in delivering AutoML software capable of being trained and tested without human intervention. Although improvements can still be made to close the gap between human-tweaked and AutoML models, this challenge has been a leap forward in the field and its platform will remain available for post-challenge submissions at http://codalab.org/AutoML. ER -
APA
Guyon, I., Chaabane, I., Escalante, H.J., Escalera, S., Jajetic, D., Lloyd, J.R., Macià, N., Ray, B., Romaszko, L., Sebag, M., Statnikov, A., Treguer, S. & Viegas, E.. (2016). A Brief Review of the ChaLearn AutoML Challenge: Any-time Any-dataset Learning Without Human Intervention. Proceedings of the Workshop on Automatic Machine Learning, in Proceedings of Machine Learning Research 64:21-30 Available from https://proceedings.mlr.press/v64/guyon_review_2016.html.

Related Material