A Brief Review of the ChaLearn AutoML Challenge: Any-time Any-dataset Learning Without Human Intervention

Isabelle Guyon, Imad Chaabane, Hugo Jair Escalante, Sergio Escalera, Damir Jajetic, James Robert Lloyd, Núria Macià, Bisakha Ray, Lukasz Romaszko, Michèle Sebag, Alexander Statnikov, Sébastien Treguer, Evelyne Viegas
; Proceedings of the Workshop on Automatic Machine Learning, PMLR 64:21-30, 2016.

Abstract

The ChaLearn AutoML Challenge team conducted a large scale evaluation of fully automatic, black-box learning machines for feature-based classification and regression problems. The test bed was composed of 30 data sets from a wide variety of application domains and ranging across different types of complexity. Over five rounds, participants succeeded in delivering AutoML software capable of being trained and tested without human intervention. Although improvements can still be made to close the gap between human-tweaked and AutoML models, this challenge has been a leap forward in the field and its platform will remain available for post-challenge submissions at http://codalab.org/AutoML.

Cite this Paper


BibTeX
@InProceedings{pmlr-v64-guyon_review_2016, title = {A brief Review of the ChaLearn AutoML Challenge: Any-time Any-dataset Learning without Human Intervention}, author = {Isabelle Guyon and Imad Chaabane and Hugo Jair Escalante and Sergio Escalera and Damir Jajetic and James Robert Lloyd and Núria Macià and Bisakha Ray and Lukasz Romaszko and Michèle Sebag and Alexander Statnikov and Sébastien Treguer and Evelyne Viegas}, booktitle = {Proceedings of the Workshop on Automatic Machine Learning}, pages = {21--30}, year = {2016}, editor = {Frank Hutter and Lars Kotthoff and Joaquin Vanschoren}, volume = {64}, series = {Proceedings of Machine Learning Research}, address = {New York, New York, USA}, month = {24 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v64/guyon_review_2016.pdf}, url = {http://proceedings.mlr.press/v64/guyon_review_2016.html}, abstract = {The ChaLearn AutoML Challenge team conducted a large scale evaluation of fully automatic, black-box learning machines for feature-based classification and regression problems. The test bed was composed of 30 data sets from a wide variety of application domains and ranging across different types of complexity. Over five rounds, participants succeeded in delivering AutoML software capable of being trained and tested without human intervention. Although improvements can still be made to close the gap between human-tweaked and AutoML models, this challenge has been a leap forward in the field and its platform will remain available for post-challenge submissions at http://codalab.org/AutoML.} }
Endnote
%0 Conference Paper %T A Brief Review of the ChaLearn AutoML Challenge: Any-time Any-dataset Learning Without Human Intervention %A Isabelle Guyon %A Imad Chaabane %A Hugo Jair Escalante %A Sergio Escalera %A Damir Jajetic %A James Robert Lloyd %A Núria Macià %A Bisakha Ray %A Lukasz Romaszko %A Michèle Sebag %A Alexander Statnikov %A Sébastien Treguer %A Evelyne Viegas %B Proceedings of the Workshop on Automatic Machine Learning %C Proceedings of Machine Learning Research %D 2016 %E Frank Hutter %E Lars Kotthoff %E Joaquin Vanschoren %F pmlr-v64-guyon_review_2016 %I PMLR %J Proceedings of Machine Learning Research %P 21--30 %U http://proceedings.mlr.press %V 64 %W PMLR %X The ChaLearn AutoML Challenge team conducted a large scale evaluation of fully automatic, black-box learning machines for feature-based classification and regression problems. The test bed was composed of 30 data sets from a wide variety of application domains and ranging across different types of complexity. Over five rounds, participants succeeded in delivering AutoML software capable of being trained and tested without human intervention. Although improvements can still be made to close the gap between human-tweaked and AutoML models, this challenge has been a leap forward in the field and its platform will remain available for post-challenge submissions at http://codalab.org/AutoML.
RIS
TY - CPAPER TI - A Brief Review of the ChaLearn AutoML Challenge: Any-time Any-dataset Learning Without Human Intervention AU - Isabelle Guyon AU - Imad Chaabane AU - Hugo Jair Escalante AU - Sergio Escalera AU - Damir Jajetic AU - James Robert Lloyd AU - Núria Macià AU - Bisakha Ray AU - Lukasz Romaszko AU - Michèle Sebag AU - Alexander Statnikov AU - Sébastien Treguer AU - Evelyne Viegas BT - Proceedings of the Workshop on Automatic Machine Learning PY - 2016/12/04 DA - 2016/12/04 ED - Frank Hutter ED - Lars Kotthoff ED - Joaquin Vanschoren ID - pmlr-v64-guyon_review_2016 PB - PMLR SP - 21 DP - PMLR EP - 30 L1 - http://proceedings.mlr.press/v64/guyon_review_2016.pdf UR - http://proceedings.mlr.press/v64/guyon_review_2016.html AB - The ChaLearn AutoML Challenge team conducted a large scale evaluation of fully automatic, black-box learning machines for feature-based classification and regression problems. The test bed was composed of 30 data sets from a wide variety of application domains and ranging across different types of complexity. Over five rounds, participants succeeded in delivering AutoML software capable of being trained and tested without human intervention. Although improvements can still be made to close the gap between human-tweaked and AutoML models, this challenge has been a leap forward in the field and its platform will remain available for post-challenge submissions at http://codalab.org/AutoML. ER -
APA
Guyon, I., Chaabane, I., Escalante, H.J., Escalera, S., Jajetic, D., Lloyd, J.R., Macià, N., Ray, B., Romaszko, L., Sebag, M., Statnikov, A., Treguer, S. & Viegas, E.. (2016). A Brief Review of the ChaLearn AutoML Challenge: Any-time Any-dataset Learning Without Human Intervention. Proceedings of the Workshop on Automatic Machine Learning, in PMLR 64:21-30

Related Material