Action-Conditioned Hamiltonian Generative Networks (AC-HGN) for Supervised and Reinforcement Learning

Arne Troch, Kevin Mets, Siegfried Mercelis
Proceedings of the 7th Annual Learning for Dynamics \& Control Conference, PMLR 283:310-322, 2025.

Abstract

This paper introduces Action-Conditioned Hamiltonian Generative Networks (AC-HGN), a physics-informed neural network architecture which learns Hamiltonian dynamics in environments subject to state-dependent external forces. AC-HGN embeds control inputs of any form into an abstract phase space, extending abstract Hamiltonian dynamics with learned external forces. In a supervised setting, results show that AC-HGN surpasses the prediction accuracy of state-of-the-art Lagrangian Neural Networks when trained on a static dataset. Furthermore, AC-HGN can be readily used as a physics-informed world model in a Model-Based Reinforcement Learning (MBRL) setting by embedding policy actions as external forces. Due to the autoencoder structure of AC-HGN, this marks the first Physics-Informed MBRL algorithm which is not reliant on any domain knowledge and is not limited to specific input modalities. Experimental results demonstrate that AC-HGN achieves competitive sample efficiency and asymptotic performance in simple environments, with minimal degradation in more complex environments, while significantly outperforming an uninformed world model. We conclude that the proposed architecture can accurately and efficiently capture environment dynamics and external forces in a Hamiltonian fashion while requiring no domain-specific knowledge, improving the applicability of physics-informed neural networks in supervised and reinforcement learning settings.

Cite this Paper

BibTeX
@InProceedings{pmlr-v283-troch25a, title = {Action-Conditioned Hamiltonian Generative Networks (AC-HGN) for Supervised and Reinforcement Learning}, author = {Troch, Arne and Mets, Kevin and Mercelis, Siegfried}, booktitle = {Proceedings of the 7th Annual Learning for Dynamics \& Control Conference}, pages = {310--322}, year = {2025}, editor = {Ozay, Necmiye and Balzano, Laura and Panagou, Dimitra and Abate, Alessandro}, volume = {283}, series = {Proceedings of Machine Learning Research}, month = {04--06 Jun}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v283/main/assets/troch25a/troch25a.pdf}, url = {https://proceedings.mlr.press/v283/troch25a.html}, abstract = {This paper introduces Action-Conditioned Hamiltonian Generative Networks (AC-HGN), a physics-informed neural network architecture which learns Hamiltonian dynamics in environments subject to state-dependent external forces. AC-HGN embeds control inputs of any form into an abstract phase space, extending abstract Hamiltonian dynamics with learned external forces. In a supervised setting, results show that AC-HGN surpasses the prediction accuracy of state-of-the-art Lagrangian Neural Networks when trained on a static dataset. Furthermore, AC-HGN can be readily used as a physics-informed world model in a Model-Based Reinforcement Learning (MBRL) setting by embedding policy actions as external forces. Due to the autoencoder structure of AC-HGN, this marks the first Physics-Informed MBRL algorithm which is not reliant on any domain knowledge and is not limited to specific input modalities. Experimental results demonstrate that AC-HGN achieves competitive sample efficiency and asymptotic performance in simple environments, with minimal degradation in more complex environments, while significantly outperforming an uninformed world model. We conclude that the proposed architecture can accurately and efficiently capture environment dynamics and external forces in a Hamiltonian fashion while requiring no domain-specific knowledge, improving the applicability of physics-informed neural networks in supervised and reinforcement learning settings.} }
Endnote
%0 Conference Paper %T Action-Conditioned Hamiltonian Generative Networks (AC-HGN) for Supervised and Reinforcement Learning %A Arne Troch %A Kevin Mets %A Siegfried Mercelis %B Proceedings of the 7th Annual Learning for Dynamics \& Control Conference %C Proceedings of Machine Learning Research %D 2025 %E Necmiye Ozay %E Laura Balzano %E Dimitra Panagou %E Alessandro Abate %F pmlr-v283-troch25a %I PMLR %P 310--322 %U https://proceedings.mlr.press/v283/troch25a.html %V 283 %X This paper introduces Action-Conditioned Hamiltonian Generative Networks (AC-HGN), a physics-informed neural network architecture which learns Hamiltonian dynamics in environments subject to state-dependent external forces. AC-HGN embeds control inputs of any form into an abstract phase space, extending abstract Hamiltonian dynamics with learned external forces. In a supervised setting, results show that AC-HGN surpasses the prediction accuracy of state-of-the-art Lagrangian Neural Networks when trained on a static dataset. Furthermore, AC-HGN can be readily used as a physics-informed world model in a Model-Based Reinforcement Learning (MBRL) setting by embedding policy actions as external forces. Due to the autoencoder structure of AC-HGN, this marks the first Physics-Informed MBRL algorithm which is not reliant on any domain knowledge and is not limited to specific input modalities. Experimental results demonstrate that AC-HGN achieves competitive sample efficiency and asymptotic performance in simple environments, with minimal degradation in more complex environments, while significantly outperforming an uninformed world model. We conclude that the proposed architecture can accurately and efficiently capture environment dynamics and external forces in a Hamiltonian fashion while requiring no domain-specific knowledge, improving the applicability of physics-informed neural networks in supervised and reinforcement learning settings.
APA
Troch, A., Mets, K. & Mercelis, S.. (2025). Action-Conditioned Hamiltonian Generative Networks (AC-HGN) for Supervised and Reinforcement Learning. Proceedings of the 7th Annual Learning for Dynamics \& Control Conference, in Proceedings of Machine Learning Research 283:310-322 Available from https://proceedings.mlr.press/v283/troch25a.html.

Related Material