Position: Embracing Negative Results in Machine Learning

Florian Karl, Malte Kemeter, Gabriel Dax, Paulina Sierak
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:23256-23265, 2024.

Abstract

Publications proposing novel machine learning methods are often primarily rated by exhibited predictive performance on selected problems. In this position paper we argue that predictive performance alone is not a good indicator for the worth of a publication. Using it as such even fosters problems like inefficiencies of the machine learning research community as a whole and setting wrong incentives for researchers. We therefore put out a call for the publication of “negative” results, which can help alleviate some of these problems and improve the scientific output of the machine learning research community. To substantiate our position, we present the advantages of publishing negative results and provide concrete measures for the community to move towards a paradigm where their publication is normalized.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-karl24a, title = {Position: Embracing Negative Results in Machine Learning}, author = {Karl, Florian and Kemeter, Malte and Dax, Gabriel and Sierak, Paulina}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {23256--23265}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/karl24a/karl24a.pdf}, url = {https://proceedings.mlr.press/v235/karl24a.html}, abstract = {Publications proposing novel machine learning methods are often primarily rated by exhibited predictive performance on selected problems. In this position paper we argue that predictive performance alone is not a good indicator for the worth of a publication. Using it as such even fosters problems like inefficiencies of the machine learning research community as a whole and setting wrong incentives for researchers. We therefore put out a call for the publication of “negative” results, which can help alleviate some of these problems and improve the scientific output of the machine learning research community. To substantiate our position, we present the advantages of publishing negative results and provide concrete measures for the community to move towards a paradigm where their publication is normalized.} }
Endnote
%0 Conference Paper %T Position: Embracing Negative Results in Machine Learning %A Florian Karl %A Malte Kemeter %A Gabriel Dax %A Paulina Sierak %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-karl24a %I PMLR %P 23256--23265 %U https://proceedings.mlr.press/v235/karl24a.html %V 235 %X Publications proposing novel machine learning methods are often primarily rated by exhibited predictive performance on selected problems. In this position paper we argue that predictive performance alone is not a good indicator for the worth of a publication. Using it as such even fosters problems like inefficiencies of the machine learning research community as a whole and setting wrong incentives for researchers. We therefore put out a call for the publication of “negative” results, which can help alleviate some of these problems and improve the scientific output of the machine learning research community. To substantiate our position, we present the advantages of publishing negative results and provide concrete measures for the community to move towards a paradigm where their publication is normalized.
APA
Karl, F., Kemeter, M., Dax, G. & Sierak, P.. (2024). Position: Embracing Negative Results in Machine Learning. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:23256-23265 Available from https://proceedings.mlr.press/v235/karl24a.html.

Related Material