Text2Interaction: Establishing Safe and Preferable Human-Robot Interaction

Jakob Thumm, Christopher Agia, Marco Pavone, Matthias Althoff
Proceedings of The 8th Conference on Robot Learning, PMLR 270:1250-1267, 2025.

Abstract

Adjusting robot behavior to human preferences can require intensive human feedback, preventing quick adaptation to new users and changing circumstances. Moreover, current approaches typically treat user preferences as a reward, which requires a manual balance between task success and user satisfaction. To integrate new user preferences in a zero-shot manner, our proposed Text2Interaction framework invokes large language models to generate a task plan, motion preferences as Python code, and parameters of a safety controller. By maximizing the combined probability of task completion and user satisfaction instead of a weighted sum of rewards, we can reliably find plans that fulfill both requirements. We find that 83% of users working with Text2Interaction agree that it integrates their preferences into the plan of the robot, and 94% prefer Text2Interaction over the baseline. Our ablation study shows that Text2Interaction aligns better with unseen preferences than other baselines while maintaining a high success rate. Real-world demonstrations and code are made available at [sites.google.com/view/text2interaction](sites.google.com/view/text2interaction).

Cite this Paper

BibTeX
@InProceedings{pmlr-v270-thumm25a, title = {Text2Interaction: Establishing Safe and Preferable Human-Robot Interaction}, author = {Thumm, Jakob and Agia, Christopher and Pavone, Marco and Althoff, Matthias}, booktitle = {Proceedings of The 8th Conference on Robot Learning}, pages = {1250--1267}, year = {2025}, editor = {Agrawal, Pulkit and Kroemer, Oliver and Burgard, Wolfram}, volume = {270}, series = {Proceedings of Machine Learning Research}, month = {06--09 Nov}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v270/main/assets/thumm25a/thumm25a.pdf}, url = {https://proceedings.mlr.press/v270/thumm25a.html}, abstract = {Adjusting robot behavior to human preferences can require intensive human feedback, preventing quick adaptation to new users and changing circumstances. Moreover, current approaches typically treat user preferences as a reward, which requires a manual balance between task success and user satisfaction. To integrate new user preferences in a zero-shot manner, our proposed Text2Interaction framework invokes large language models to generate a task plan, motion preferences as Python code, and parameters of a safety controller. By maximizing the combined probability of task completion and user satisfaction instead of a weighted sum of rewards, we can reliably find plans that fulfill both requirements. We find that 83% of users working with Text2Interaction agree that it integrates their preferences into the plan of the robot, and 94% prefer Text2Interaction over the baseline. Our ablation study shows that Text2Interaction aligns better with unseen preferences than other baselines while maintaining a high success rate. Real-world demonstrations and code are made available at [sites.google.com/view/text2interaction](sites.google.com/view/text2interaction).} }
Endnote
%0 Conference Paper %T Text2Interaction: Establishing Safe and Preferable Human-Robot Interaction %A Jakob Thumm %A Christopher Agia %A Marco Pavone %A Matthias Althoff %B Proceedings of The 8th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2025 %E Pulkit Agrawal %E Oliver Kroemer %E Wolfram Burgard %F pmlr-v270-thumm25a %I PMLR %P 1250--1267 %U https://proceedings.mlr.press/v270/thumm25a.html %V 270 %X Adjusting robot behavior to human preferences can require intensive human feedback, preventing quick adaptation to new users and changing circumstances. Moreover, current approaches typically treat user preferences as a reward, which requires a manual balance between task success and user satisfaction. To integrate new user preferences in a zero-shot manner, our proposed Text2Interaction framework invokes large language models to generate a task plan, motion preferences as Python code, and parameters of a safety controller. By maximizing the combined probability of task completion and user satisfaction instead of a weighted sum of rewards, we can reliably find plans that fulfill both requirements. We find that 83% of users working with Text2Interaction agree that it integrates their preferences into the plan of the robot, and 94% prefer Text2Interaction over the baseline. Our ablation study shows that Text2Interaction aligns better with unseen preferences than other baselines while maintaining a high success rate. Real-world demonstrations and code are made available at [sites.google.com/view/text2interaction](sites.google.com/view/text2interaction).
APA
Thumm, J., Agia, C., Pavone, M. & Althoff, M.. (2025). Text2Interaction: Establishing Safe and Preferable Human-Robot Interaction. Proceedings of The 8th Conference on Robot Learning, in Proceedings of Machine Learning Research 270:1250-1267 Available from https://proceedings.mlr.press/v270/thumm25a.html.

Related Material