Constraint-Aware Reinforcement Learning via Adaptive Action Scaling

Murad Dawood, Usama Ahmed Siddiquie, Shahram Khorshidi, Maren Bennewitz
Proceedings of The 8th Annual Learning for Dynamics and Control Conference, PMLR 331:2109-2122, 2026.

Abstract

Safe reinforcement learning (RL) seeks to mitigate unsafe behaviors that arise from exploration during training by reducing constraint violations while maintaining task performance. Existing approaches typically rely on a single policy to jointly optimize reward and safety, which can cause instability due to conflicting objectives, or they use external safety filters that override actions and require prior system knowledge. In this paper, we propose a modular cost-aware regulator that scales the agent’s actions based on predicted constraint violations, preserving exploration through smooth action modulation rather than overriding the policy. The regulator is trained to minimize constraint violations while avoiding degenerate suppression of actions. Our approach integrates seamlessly with off-policy RL methods such as SAC and TD3, and achieves state-of-the-art return-to-cost ratios on Safety Gym locomotion tasks with sparse costs, reducing constraint violations by up to 126 times while increasing returns by over an order of magnitude compared to prior methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v331-dawood26a, title = {Constraint-Aware Reinforcement Learning via Adaptive Action Scaling}, author = {Dawood, Murad and Siddiquie, Usama Ahmed and Khorshidi, Shahram and Bennewitz, Maren}, booktitle = {Proceedings of The 8th Annual Learning for Dynamics and Control Conference}, pages = {2109--2122}, year = {2026}, editor = {Sukhatme, Gaurav and Lindemann, Lars and Tu, Stephen and Wierman, Adam and Atanasov, Nikolay}, volume = {331}, series = {Proceedings of Machine Learning Research}, month = {17--19 Jun}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v331/main/assets/dawood26a/dawood26a.pdf}, url = {https://proceedings.mlr.press/v331/dawood26a.html}, abstract = {Safe reinforcement learning (RL) seeks to mitigate unsafe behaviors that arise from exploration during training by reducing constraint violations while maintaining task performance. Existing approaches typically rely on a single policy to jointly optimize reward and safety, which can cause instability due to conflicting objectives, or they use external safety filters that override actions and require prior system knowledge. In this paper, we propose a modular cost-aware regulator that scales the agent’s actions based on predicted constraint violations, preserving exploration through smooth action modulation rather than overriding the policy. The regulator is trained to minimize constraint violations while avoiding degenerate suppression of actions. Our approach integrates seamlessly with off-policy RL methods such as SAC and TD3, and achieves state-of-the-art return-to-cost ratios on Safety Gym locomotion tasks with sparse costs, reducing constraint violations by up to 126 times while increasing returns by over an order of magnitude compared to prior methods.} }
Endnote
%0 Conference Paper %T Constraint-Aware Reinforcement Learning via Adaptive Action Scaling %A Murad Dawood %A Usama Ahmed Siddiquie %A Shahram Khorshidi %A Maren Bennewitz %B Proceedings of The 8th Annual Learning for Dynamics and Control Conference %C Proceedings of Machine Learning Research %D 2026 %E Gaurav Sukhatme %E Lars Lindemann %E Stephen Tu %E Adam Wierman %E Nikolay Atanasov %F pmlr-v331-dawood26a %I PMLR %P 2109--2122 %U https://proceedings.mlr.press/v331/dawood26a.html %V 331 %X Safe reinforcement learning (RL) seeks to mitigate unsafe behaviors that arise from exploration during training by reducing constraint violations while maintaining task performance. Existing approaches typically rely on a single policy to jointly optimize reward and safety, which can cause instability due to conflicting objectives, or they use external safety filters that override actions and require prior system knowledge. In this paper, we propose a modular cost-aware regulator that scales the agent’s actions based on predicted constraint violations, preserving exploration through smooth action modulation rather than overriding the policy. The regulator is trained to minimize constraint violations while avoiding degenerate suppression of actions. Our approach integrates seamlessly with off-policy RL methods such as SAC and TD3, and achieves state-of-the-art return-to-cost ratios on Safety Gym locomotion tasks with sparse costs, reducing constraint violations by up to 126 times while increasing returns by over an order of magnitude compared to prior methods.
APA
Dawood, M., Siddiquie, U.A., Khorshidi, S. & Bennewitz, M.. (2026). Constraint-Aware Reinforcement Learning via Adaptive Action Scaling. Proceedings of The 8th Annual Learning for Dynamics and Control Conference, in Proceedings of Machine Learning Research 331:2109-2122 Available from https://proceedings.mlr.press/v331/dawood26a.html.

Related Material