Deep QP Safety Filter: Model-free Learning for Reachability-based Safety Filter

Byeongjun Kim, H. Jin Kim
Proceedings of The 8th Annual Learning for Dynamics and Control Conference, PMLR 331:1470-1482, 2026.

Abstract

We introduce Deep QP Safety Filter, a fully data-driven safety layer for black-box dynamical systems. Our method learns a Quadratic-Program (QP) safety filter without model knowledge by combining Hamilton–Jacobi (HJ) reachability with model-free learning. We construct contraction-based losses for both the safety value and its derivatives, and train two neural networks accordingly. In the exact setting, the learned critic converges to the viscosity solution (and its derivative), even for non-smooth values. Across diverse dynamical systems – even including a hybrid system – and multiple RL tasks, Deep QP Safety Filter substantially reduces pre-convergence failures while accelerating learning toward higher returns than strong baselines, offering a principled and practical route to safe, model-free control.

Cite this Paper

BibTeX
@InProceedings{pmlr-v331-kim26c, title = {Deep QP Safety Filter: Model-free Learning for Reachability-based Safety Filter}, author = {Kim, Byeongjun and Kim, H. Jin}, booktitle = {Proceedings of The 8th Annual Learning for Dynamics and Control Conference}, pages = {1470--1482}, year = {2026}, editor = {Sukhatme, Gaurav and Lindemann, Lars and Tu, Stephen and Wierman, Adam and Atanasov, Nikolay}, volume = {331}, series = {Proceedings of Machine Learning Research}, month = {17--19 Jun}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v331/main/assets/kim26c/kim26c.pdf}, url = {https://proceedings.mlr.press/v331/kim26c.html}, abstract = {We introduce Deep QP Safety Filter, a fully data-driven safety layer for black-box dynamical systems. Our method learns a Quadratic-Program (QP) safety filter without model knowledge by combining Hamilton–Jacobi (HJ) reachability with model-free learning. We construct contraction-based losses for both the safety value and its derivatives, and train two neural networks accordingly. In the exact setting, the learned critic converges to the viscosity solution (and its derivative), even for non-smooth values. Across diverse dynamical systems – even including a hybrid system – and multiple RL tasks, Deep QP Safety Filter substantially reduces pre-convergence failures while accelerating learning toward higher returns than strong baselines, offering a principled and practical route to safe, model-free control.} }
Endnote
%0 Conference Paper %T Deep QP Safety Filter: Model-free Learning for Reachability-based Safety Filter %A Byeongjun Kim %A H. Jin Kim %B Proceedings of The 8th Annual Learning for Dynamics and Control Conference %C Proceedings of Machine Learning Research %D 2026 %E Gaurav Sukhatme %E Lars Lindemann %E Stephen Tu %E Adam Wierman %E Nikolay Atanasov %F pmlr-v331-kim26c %I PMLR %P 1470--1482 %U https://proceedings.mlr.press/v331/kim26c.html %V 331 %X We introduce Deep QP Safety Filter, a fully data-driven safety layer for black-box dynamical systems. Our method learns a Quadratic-Program (QP) safety filter without model knowledge by combining Hamilton–Jacobi (HJ) reachability with model-free learning. We construct contraction-based losses for both the safety value and its derivatives, and train two neural networks accordingly. In the exact setting, the learned critic converges to the viscosity solution (and its derivative), even for non-smooth values. Across diverse dynamical systems – even including a hybrid system – and multiple RL tasks, Deep QP Safety Filter substantially reduces pre-convergence failures while accelerating learning toward higher returns than strong baselines, offering a principled and practical route to safe, model-free control.
APA
Kim, B. & Kim, H.J.. (2026). Deep QP Safety Filter: Model-free Learning for Reachability-based Safety Filter. Proceedings of The 8th Annual Learning for Dynamics and Control Conference, in Proceedings of Machine Learning Research 331:1470-1482 Available from https://proceedings.mlr.press/v331/kim26c.html.

Related Material