Position: Assistive Agents Need Accessibility Alignment

Jie Hu, Changyuan Yan, Yu Zheng, Ziqian Wang, Jiaming Zhang
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:170557-170568, 2026.

Abstract

Assistive agents, especially those intended to sup- port Blind and Visually Impaired (BVI) users, require accessibility alignment as a first-class de- sign objective. Despite rapid progress in agen- tic AI, most current systems are designed and evaluated under assumptions that implicitly cen- ter sighted users, leading to systematic failures in assistive scenarios that cannot be addressed by model scaling or post-hoc adaptations alone. Based on an analysis of 778 real-world assistance instances involving BVI users, we show that these failures arise from persistent mismatches between agent capabilities and the accessibility-specific needs, risks, and interaction constraints of visu- ally impaired users. We argue that accessibil- ity should be treated as an alignment problem rather than a peripheral usability concern. To this end, we introduce the notion of accessibility alignment and propose a lifecycle-oriented design pipeline for accessibility-aligned assistive agents, spanning user research, system design, and post- deployment iteration. We conclude that BVI users centered assistive tasks provide a critical stress test for agentic AI and motivate a shift toward more inclusive agent design.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-hu26bl, title = {Position: Assistive Agents Need Accessibility Alignment}, author = {Hu, Jie and Yan, Changyuan and Zheng, Yu and Wang, Ziqian and Zhang, Jiaming}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {170557--170568}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/hu26bl/hu26bl.pdf}, url = {https://proceedings.mlr.press/v306/hu26bl.html}, abstract = {Assistive agents, especially those intended to sup- port Blind and Visually Impaired (BVI) users, require accessibility alignment as a first-class de- sign objective. Despite rapid progress in agen- tic AI, most current systems are designed and evaluated under assumptions that implicitly cen- ter sighted users, leading to systematic failures in assistive scenarios that cannot be addressed by model scaling or post-hoc adaptations alone. Based on an analysis of 778 real-world assistance instances involving BVI users, we show that these failures arise from persistent mismatches between agent capabilities and the accessibility-specific needs, risks, and interaction constraints of visu- ally impaired users. We argue that accessibil- ity should be treated as an alignment problem rather than a peripheral usability concern. To this end, we introduce the notion of accessibility alignment and propose a lifecycle-oriented design pipeline for accessibility-aligned assistive agents, spanning user research, system design, and post- deployment iteration. We conclude that BVI users centered assistive tasks provide a critical stress test for agentic AI and motivate a shift toward more inclusive agent design.} }
Endnote
%0 Conference Paper %T Position: Assistive Agents Need Accessibility Alignment %A Jie Hu %A Changyuan Yan %A Yu Zheng %A Ziqian Wang %A Jiaming Zhang %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-hu26bl %I PMLR %P 170557--170568 %U https://proceedings.mlr.press/v306/hu26bl.html %V 306 %X Assistive agents, especially those intended to sup- port Blind and Visually Impaired (BVI) users, require accessibility alignment as a first-class de- sign objective. Despite rapid progress in agen- tic AI, most current systems are designed and evaluated under assumptions that implicitly cen- ter sighted users, leading to systematic failures in assistive scenarios that cannot be addressed by model scaling or post-hoc adaptations alone. Based on an analysis of 778 real-world assistance instances involving BVI users, we show that these failures arise from persistent mismatches between agent capabilities and the accessibility-specific needs, risks, and interaction constraints of visu- ally impaired users. We argue that accessibil- ity should be treated as an alignment problem rather than a peripheral usability concern. To this end, we introduce the notion of accessibility alignment and propose a lifecycle-oriented design pipeline for accessibility-aligned assistive agents, spanning user research, system design, and post- deployment iteration. We conclude that BVI users centered assistive tasks provide a critical stress test for agentic AI and motivate a shift toward more inclusive agent design.
APA
Hu, J., Yan, C., Zheng, Y., Wang, Z. & Zhang, J.. (2026). Position: Assistive Agents Need Accessibility Alignment. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:170557-170568 Available from https://proceedings.mlr.press/v306/hu26bl.html.

Related Material