Orthogonal projection-based regularization for efficient model augmentation

Bendeguz Mate Györök, Jan H. Hoekstra, Johan Kon, Tamas Peni, Maarten Schoukens, Roland Toth
Proceedings of the 7th Annual Learning for Dynamics \& Control Conference, PMLR 283:166-178, 2025.

Abstract

Deep-learning-based nonlinear system identification has shown the ability to produce reliable and highly accurate models in practice. However, these black-box models lack physical interpretability, and a considerable part of the learning effort is often spent on capturing already expected/known behavior of the system, that can be accurately described by first-principles laws of physics. A potential solution is to directly integrate such prior physical knowledge into the model structure, combining the strengths of physics-based modeling and deep-learning-based identification. The most common approach is to use an additive model augmentation structure, where the physics-based and the machine-learning (ML) components are connected in parallel, i.e., additively. However, such models are overparametrized, training them is challenging, potentially causing the physics-based part to lose interpretability. To overcome this challenge, this paper proposes an orthogonal projection-based regularization technique to enhance parameter learning and even model accuracy in learning-based augmentation of nonlinear baseline models.

Cite this Paper

BibTeX
@InProceedings{pmlr-v283-gyorok25a, title = {Orthogonal projection-based regularization for efficient model augmentation}, author = {Gy{\"o}r{\"o}k, Bendeguz Mate and Hoekstra, Jan H. and Kon, Johan and Peni, Tamas and Schoukens, Maarten and Toth, Roland}, booktitle = {Proceedings of the 7th Annual Learning for Dynamics \& Control Conference}, pages = {166--178}, year = {2025}, editor = {Ozay, Necmiye and Balzano, Laura and Panagou, Dimitra and Abate, Alessandro}, volume = {283}, series = {Proceedings of Machine Learning Research}, month = {04--06 Jun}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v283/main/assets/gyorok25a/gyorok25a.pdf}, url = {https://proceedings.mlr.press/v283/gyorok25a.html}, abstract = {Deep-learning-based nonlinear system identification has shown the ability to produce reliable and highly accurate models in practice. However, these black-box models lack physical interpretability, and a considerable part of the learning effort is often spent on capturing already expected/known behavior of the system, that can be accurately described by first-principles laws of physics. A potential solution is to directly integrate such prior physical knowledge into the model structure, combining the strengths of physics-based modeling and deep-learning-based identification. The most common approach is to use an additive model augmentation structure, where the physics-based and the machine-learning (ML) components are connected in parallel, i.e., additively. However, such models are overparametrized, training them is challenging, potentially causing the physics-based part to lose interpretability. To overcome this challenge, this paper proposes an orthogonal projection-based regularization technique to enhance parameter learning and even model accuracy in learning-based augmentation of nonlinear baseline models.} }
Endnote
%0 Conference Paper %T Orthogonal projection-based regularization for efficient model augmentation %A Bendeguz Mate Györök %A Jan H. Hoekstra %A Johan Kon %A Tamas Peni %A Maarten Schoukens %A Roland Toth %B Proceedings of the 7th Annual Learning for Dynamics \& Control Conference %C Proceedings of Machine Learning Research %D 2025 %E Necmiye Ozay %E Laura Balzano %E Dimitra Panagou %E Alessandro Abate %F pmlr-v283-gyorok25a %I PMLR %P 166--178 %U https://proceedings.mlr.press/v283/gyorok25a.html %V 283 %X Deep-learning-based nonlinear system identification has shown the ability to produce reliable and highly accurate models in practice. However, these black-box models lack physical interpretability, and a considerable part of the learning effort is often spent on capturing already expected/known behavior of the system, that can be accurately described by first-principles laws of physics. A potential solution is to directly integrate such prior physical knowledge into the model structure, combining the strengths of physics-based modeling and deep-learning-based identification. The most common approach is to use an additive model augmentation structure, where the physics-based and the machine-learning (ML) components are connected in parallel, i.e., additively. However, such models are overparametrized, training them is challenging, potentially causing the physics-based part to lose interpretability. To overcome this challenge, this paper proposes an orthogonal projection-based regularization technique to enhance parameter learning and even model accuracy in learning-based augmentation of nonlinear baseline models.
APA
Györök, B.M., Hoekstra, J.H., Kon, J., Peni, T., Schoukens, M. & Toth, R.. (2025). Orthogonal projection-based regularization for efficient model augmentation. Proceedings of the 7th Annual Learning for Dynamics \& Control Conference, in Proceedings of Machine Learning Research 283:166-178 Available from https://proceedings.mlr.press/v283/gyorok25a.html.

Related Material