SteinGate: Tail-Sensitive Safe Reinforcement Learning via Stein Discrepancy

Yassine Chemingui, Chenhua Fan, Honghao Wei, Janardhan Rao Doppa
Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, PMLR 337:1081-1104, 2026.

Abstract

Safe reinforcement learning typically enforces safety by bounding expected cumulative costs, a criterion that often fails to detect rare but catastrophic tail events. To overcome these limitations, this paper introduces {\em {SteinGate}}, a boundary-aware distributional safety certificate that replaces fragile tail fitting with a robust consistency check using Kernelized {Stein} Discrepancy while accounting for boundary atoms induced by clipped costs. {SteinGate} evaluates whether observed policy rollout costs remain consistent with a safe reference distribution, providing a non-parametric safety certificate. This certificate is used to dynamically adapt the learning regime: favoring reward-improving policy updates when rollouts remain consistent with the safe reference and switching to recovery behavior when the cost tail deviates. Experiments on continuous-control benchmarks demonstrate that {SteinGate} significantly reduces both the frequency and severity of constraint violations during training while maintaining competitive returns relative to state-of-the-art baselines.

Cite this Paper

BibTeX
@InProceedings{pmlr-v337-chemingui26a, title = {{SteinGate}: Tail-Sensitive Safe Reinforcement Learning via {Stein} Discrepancy}, author = {Chemingui, Yassine and Fan, Chenhua and Wei, Honghao and Doppa, Janardhan Rao}, booktitle = {Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence}, pages = {1081--1104}, year = {2026}, editor = {Perković, Emilija and Malinsky, Daniel}, volume = {337}, series = {Proceedings of Machine Learning Research}, month = {17--21 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v337/main/assets/chemingui26a/chemingui26a.pdf}, url = {https://proceedings.mlr.press/v337/chemingui26a.html}, abstract = {Safe reinforcement learning typically enforces safety by bounding expected cumulative costs, a criterion that often fails to detect rare but catastrophic tail events. To overcome these limitations, this paper introduces {\em {SteinGate}}, a boundary-aware distributional safety certificate that replaces fragile tail fitting with a robust consistency check using Kernelized {Stein} Discrepancy while accounting for boundary atoms induced by clipped costs. {SteinGate} evaluates whether observed policy rollout costs remain consistent with a safe reference distribution, providing a non-parametric safety certificate. This certificate is used to dynamically adapt the learning regime: favoring reward-improving policy updates when rollouts remain consistent with the safe reference and switching to recovery behavior when the cost tail deviates. Experiments on continuous-control benchmarks demonstrate that {SteinGate} significantly reduces both the frequency and severity of constraint violations during training while maintaining competitive returns relative to state-of-the-art baselines.} }
Endnote
%0 Conference Paper %T SteinGate: Tail-Sensitive Safe Reinforcement Learning via Stein Discrepancy %A Yassine Chemingui %A Chenhua Fan %A Honghao Wei %A Janardhan Rao Doppa %B Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2026 %E Emilija Perković %E Daniel Malinsky %F pmlr-v337-chemingui26a %I PMLR %P 1081--1104 %U https://proceedings.mlr.press/v337/chemingui26a.html %V 337 %X Safe reinforcement learning typically enforces safety by bounding expected cumulative costs, a criterion that often fails to detect rare but catastrophic tail events. To overcome these limitations, this paper introduces {\em {SteinGate}}, a boundary-aware distributional safety certificate that replaces fragile tail fitting with a robust consistency check using Kernelized {Stein} Discrepancy while accounting for boundary atoms induced by clipped costs. {SteinGate} evaluates whether observed policy rollout costs remain consistent with a safe reference distribution, providing a non-parametric safety certificate. This certificate is used to dynamically adapt the learning regime: favoring reward-improving policy updates when rollouts remain consistent with the safe reference and switching to recovery behavior when the cost tail deviates. Experiments on continuous-control benchmarks demonstrate that {SteinGate} significantly reduces both the frequency and severity of constraint violations during training while maintaining competitive returns relative to state-of-the-art baselines.
APA
Chemingui, Y., Fan, C., Wei, H. & Doppa, J.R.. (2026). SteinGate: Tail-Sensitive Safe Reinforcement Learning via Stein Discrepancy. Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 337:1081-1104 Available from https://proceedings.mlr.press/v337/chemingui26a.html.

Related Material