Position: Insights from Survey Methodology can Improve Training Data

Stephanie Eckman, Barbara Plank, Frauke Kreuter
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:12268-12283, 2024.

Abstract

Whether future AI models are fair, trustworthy, and aligned with the public’s interests rests in part on our ability to collect accurate data about what we want the models to do. However, collecting high-quality data is difficult, and few AI/ML researchers are trained in data collection methods. Recent research in data-centric AI has show that higher quality training data leads to better performing models, making this the right moment to introduce AI/ML researchers to the field of survey methodology, the science of data collection. We summarize insights from the survey methodology literature and discuss how they can improve the quality of training and feedback data. We also suggest collaborative research ideas into how biases in data collection can be mitigated, making models more accurate and human-centric.

Cite this Paper

BibTeX
@InProceedings{pmlr-v235-eckman24a, title = {Position: Insights from Survey Methodology can Improve Training Data}, author = {Eckman, Stephanie and Plank, Barbara and Kreuter, Frauke}, booktitle = {Proceedings of the 41st International Conference on Machine Learning}, pages = {12268--12283}, year = {2024}, editor = {Salakhutdinov, Ruslan and Kolter, Zico and Heller, Katherine and Weller, Adrian and Oliver, Nuria and Scarlett, Jonathan and Berkenkamp, Felix}, volume = {235}, series = {Proceedings of Machine Learning Research}, month = {21--27 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v235/main/assets/eckman24a/eckman24a.pdf}, url = {https://proceedings.mlr.press/v235/eckman24a.html}, abstract = {Whether future AI models are fair, trustworthy, and aligned with the public’s interests rests in part on our ability to collect accurate data about what we want the models to do. However, collecting high-quality data is difficult, and few AI/ML researchers are trained in data collection methods. Recent research in data-centric AI has show that higher quality training data leads to better performing models, making this the right moment to introduce AI/ML researchers to the field of survey methodology, the science of data collection. We summarize insights from the survey methodology literature and discuss how they can improve the quality of training and feedback data. We also suggest collaborative research ideas into how biases in data collection can be mitigated, making models more accurate and human-centric.} }
Endnote
%0 Conference Paper %T Position: Insights from Survey Methodology can Improve Training Data %A Stephanie Eckman %A Barbara Plank %A Frauke Kreuter %B Proceedings of the 41st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Ruslan Salakhutdinov %E Zico Kolter %E Katherine Heller %E Adrian Weller %E Nuria Oliver %E Jonathan Scarlett %E Felix Berkenkamp %F pmlr-v235-eckman24a %I PMLR %P 12268--12283 %U https://proceedings.mlr.press/v235/eckman24a.html %V 235 %X Whether future AI models are fair, trustworthy, and aligned with the public’s interests rests in part on our ability to collect accurate data about what we want the models to do. However, collecting high-quality data is difficult, and few AI/ML researchers are trained in data collection methods. Recent research in data-centric AI has show that higher quality training data leads to better performing models, making this the right moment to introduce AI/ML researchers to the field of survey methodology, the science of data collection. We summarize insights from the survey methodology literature and discuss how they can improve the quality of training and feedback data. We also suggest collaborative research ideas into how biases in data collection can be mitigated, making models more accurate and human-centric.
APA
Eckman, S., Plank, B. & Kreuter, F.. (2024). Position: Insights from Survey Methodology can Improve Training Data. Proceedings of the 41st International Conference on Machine Learning, in Proceedings of Machine Learning Research 235:12268-12283 Available from https://proceedings.mlr.press/v235/eckman24a.html.

Related Material