Generating Robot Constitutions & Benchmarks for Semantic Safety

Pierre Sermanet, Anirudha Majumdar, Alex Irpan, Dmitry Kalashnikov, Vikas Sindhwani
Proceedings of The 9th Conference on Robot Learning, PMLR 305:4767-4823, 2025.

Abstract

Large vision and language models are being increasingly deployed on real robots, leading to an immediate need for ensuring robot safety under AI-control. In this paper, we develop the ASIMOV Benchmark — a collection of large-scale semantic safety datasets grounded in real-world visual scenes and human injury reports from hospitals (500k situations, 3M instructions). We propose a scalable recipe for data generation leveraging text and image generation techniques to synthesize safety-relevant scenarios. As a second contribution, we develop a framework to automatically generate robot constitutions from real-world data to steer a robot’s behavior using Constitutional AI mechanisms. We report a top alignment rate of 84.3% on the ASIMOV Benchmark using generated constitutions, outperforming no-constitution baselines and human-written constitutions. We argue that human interpretability and modifiability of constitutions inferred from data make them an ideal medium for behavior governance of AI-controlled robots.

Cite this Paper

BibTeX
@InProceedings{pmlr-v305-sermanet25a, title = {Generating Robot Constitutions & Benchmarks for Semantic Safety}, author = {Sermanet, Pierre and Majumdar, Anirudha and Irpan, Alex and Kalashnikov, Dmitry and Sindhwani, Vikas}, booktitle = {Proceedings of The 9th Conference on Robot Learning}, pages = {4767--4823}, year = {2025}, editor = {Lim, Joseph and Song, Shuran and Park, Hae-Won}, volume = {305}, series = {Proceedings of Machine Learning Research}, month = {27--30 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v305/main/assets/sermanet25a/sermanet25a.pdf}, url = {https://proceedings.mlr.press/v305/sermanet25a.html}, abstract = {Large vision and language models are being increasingly deployed on real robots, leading to an immediate need for ensuring robot safety under AI-control. In this paper, we develop the ASIMOV Benchmark — a collection of large-scale semantic safety datasets grounded in real-world visual scenes and human injury reports from hospitals (500k situations, 3M instructions). We propose a scalable recipe for data generation leveraging text and image generation techniques to synthesize safety-relevant scenarios. As a second contribution, we develop a framework to automatically generate robot constitutions from real-world data to steer a robot’s behavior using Constitutional AI mechanisms. We report a top alignment rate of 84.3% on the ASIMOV Benchmark using generated constitutions, outperforming no-constitution baselines and human-written constitutions. We argue that human interpretability and modifiability of constitutions inferred from data make them an ideal medium for behavior governance of AI-controlled robots.} }
Endnote
%0 Conference Paper %T Generating Robot Constitutions & Benchmarks for Semantic Safety %A Pierre Sermanet %A Anirudha Majumdar %A Alex Irpan %A Dmitry Kalashnikov %A Vikas Sindhwani %B Proceedings of The 9th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2025 %E Joseph Lim %E Shuran Song %E Hae-Won Park %F pmlr-v305-sermanet25a %I PMLR %P 4767--4823 %U https://proceedings.mlr.press/v305/sermanet25a.html %V 305 %X Large vision and language models are being increasingly deployed on real robots, leading to an immediate need for ensuring robot safety under AI-control. In this paper, we develop the ASIMOV Benchmark — a collection of large-scale semantic safety datasets grounded in real-world visual scenes and human injury reports from hospitals (500k situations, 3M instructions). We propose a scalable recipe for data generation leveraging text and image generation techniques to synthesize safety-relevant scenarios. As a second contribution, we develop a framework to automatically generate robot constitutions from real-world data to steer a robot’s behavior using Constitutional AI mechanisms. We report a top alignment rate of 84.3% on the ASIMOV Benchmark using generated constitutions, outperforming no-constitution baselines and human-written constitutions. We argue that human interpretability and modifiability of constitutions inferred from data make them an ideal medium for behavior governance of AI-controlled robots.
APA
Sermanet, P., Majumdar, A., Irpan, A., Kalashnikov, D. & Sindhwani, V.. (2025). Generating Robot Constitutions & Benchmarks for Semantic Safety. Proceedings of The 9th Conference on Robot Learning, in Proceedings of Machine Learning Research 305:4767-4823 Available from https://proceedings.mlr.press/v305/sermanet25a.html.

Related Material