Position: Key Claims in LLM Research Have a Long Tail of Footnotes

Anna Rogers, Sasha Luccioni
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:42647-42665, 2024.

Abstract

Much of the recent discourse within the ML community has been centered around Large Language Models (LLMs), their functionality and potential – yet not only do we not have a working definition of LLMs, but much of this discourse relies on claims and assumptions that are worth re-examining. We contribute a definition of LLMs, critically examine five common claims regarding their properties (including ’emergent properties’), and conclude with suggestions for future research directions and their framing.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-rogers24a, title = {Position: Key Claims in {LLM} Research Have a Long Tail of Footnotes}, author = {Rogers, Anna and Luccioni, Sasha}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {42647--42665}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/rogers24a/rogers24a.pdf}, url = {https://proceedings.mlr.press/v235/rogers24a.html}, abstract = {Much of the recent discourse within the ML community has been centered around Large Language Models (LLMs), their functionality and potential – yet not only do we not have a working definition of LLMs, but much of this discourse relies on claims and assumptions that are worth re-examining. We contribute a definition of LLMs, critically examine five common claims regarding their properties (including ’emergent properties’), and conclude with suggestions for future research directions and their framing.} }
Endnote
%0 Conference Paper %T Position: Key Claims in LLM Research Have a Long Tail of Footnotes %A Anna Rogers %A Sasha Luccioni %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-rogers24a %I PMLR %P 42647--42665 %U https://proceedings.mlr.press/v235/rogers24a.html %V 235 %X Much of the recent discourse within the ML community has been centered around Large Language Models (LLMs), their functionality and potential – yet not only do we not have a working definition of LLMs, but much of this discourse relies on claims and assumptions that are worth re-examining. We contribute a definition of LLMs, critically examine five common claims regarding their properties (including ’emergent properties’), and conclude with suggestions for future research directions and their framing.
APA
Rogers, A. & Luccioni, S.. (2024). Position: Key Claims in LLM Research Have a Long Tail of Footnotes. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:42647-42665 Available from https://proceedings.mlr.press/v235/rogers24a.html.

Related Material