Position: AI Welfare Is Bullshit

Yunze Xiao, Gordon Dai, Shahan Ali Memon, Jen-Tse Huang, Maarten Sap, Mona T. Diab
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:172649-172665, 2026.

Abstract

In this position paper, we argue that for AI systems, “welfare” is a choice in mechanism and evaluation, rather than an empirically discoverable property, because welfare assessment lacks an external validation channel: there is no independent, intervention-based test that can falsify a welfare metric or adjudicate among competing accounts of what welfare requires. We formalize this diagnosis using evaluation theory, emphasizing that in AI the subject, indicators, and metrics are co-engineered, so proposed welfare evidence can be manufactured or suppressed by ordinary development decisions. We then analyze two institutional failure modes if welfare scorecards are nonetheless used in release and access decisions: they expand procedural gates around routine ML work and they enable organizations to reframe discretionary choices about liability, publicity, and risk posture as moral necessity. We conclude with guidance for research and governance: prohibit welfare scorecards as release gates, disallow appeals to model welfare as a reason to resist auditing and oversight, and require that any restrictions on AI development be justified by externally verifiable harms rather than untestable welfare claims.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-xiao26x, title = {Position: {AI} Welfare Is Bullshit}, author = {Xiao, Yunze and Dai, Gordon and Memon, Shahan Ali and Huang, Jen-Tse and Sap, Maarten and Diab, Mona T.}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {172649--172665}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/xiao26x/xiao26x.pdf}, url = {https://proceedings.mlr.press/v306/xiao26x.html}, abstract = {In this position paper, we argue that for AI systems, “welfare” is a choice in mechanism and evaluation, rather than an empirically discoverable property, because welfare assessment lacks an external validation channel: there is no independent, intervention-based test that can falsify a welfare metric or adjudicate among competing accounts of what welfare requires. We formalize this diagnosis using evaluation theory, emphasizing that in AI the subject, indicators, and metrics are co-engineered, so proposed welfare evidence can be manufactured or suppressed by ordinary development decisions. We then analyze two institutional failure modes if welfare scorecards are nonetheless used in release and access decisions: they expand procedural gates around routine ML work and they enable organizations to reframe discretionary choices about liability, publicity, and risk posture as moral necessity. We conclude with guidance for research and governance: prohibit welfare scorecards as release gates, disallow appeals to model welfare as a reason to resist auditing and oversight, and require that any restrictions on AI development be justified by externally verifiable harms rather than untestable welfare claims.} }
Endnote
%0 Conference Paper %T Position: AI Welfare Is Bullshit %A Yunze Xiao %A Gordon Dai %A Shahan Ali Memon %A Jen-Tse Huang %A Maarten Sap %A Mona T. Diab %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-xiao26x %I PMLR %P 172649--172665 %U https://proceedings.mlr.press/v306/xiao26x.html %V 306 %X In this position paper, we argue that for AI systems, “welfare” is a choice in mechanism and evaluation, rather than an empirically discoverable property, because welfare assessment lacks an external validation channel: there is no independent, intervention-based test that can falsify a welfare metric or adjudicate among competing accounts of what welfare requires. We formalize this diagnosis using evaluation theory, emphasizing that in AI the subject, indicators, and metrics are co-engineered, so proposed welfare evidence can be manufactured or suppressed by ordinary development decisions. We then analyze two institutional failure modes if welfare scorecards are nonetheless used in release and access decisions: they expand procedural gates around routine ML work and they enable organizations to reframe discretionary choices about liability, publicity, and risk posture as moral necessity. We conclude with guidance for research and governance: prohibit welfare scorecards as release gates, disallow appeals to model welfare as a reason to resist auditing and oversight, and require that any restrictions on AI development be justified by externally verifiable harms rather than untestable welfare claims.
APA
Xiao, Y., Dai, G., Memon, S.A., Huang, J., Sap, M. & Diab, M.T.. (2026). Position: AI Welfare Is Bullshit. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:172649-172665 Available from https://proceedings.mlr.press/v306/xiao26x.html.

Related Material