Hierarchical Decomposition of Nonlinear Dynamics and Control for System Identification and Policy Distillation

Hany Abdulsamad, Jan Peters
Proceedings of the 2nd Conference on Learning for Dynamics and Control, PMLR 120:904-914, 2020.

Abstract

Control of nonlinear systems with unknown dynamics is a major challenge on the road to fully autonomous agents. Current trends in reinforcement learning (RL) focus on complex representations of dynamics and policies. Such approaches have yielded impressive results in solving a variety of hard control tasks. However, this new sophistication has come with the cost of an overall reduction in our ability to interpret the resulting policies from a classical perspective, and the need for extremely over-parameterized controllers. In this paper, we take inspiration from the control community and apply the principles of hybrid switching systems, in order to break down complex representations into simpler components. We exploit the rich representational power of probabilistic graphical models and derive a new expectation-maximization (EM) algorithm for learning a generative model and automatically decomposing nonlinear dynamics into stochastic switching linear dynamical systems. Moreover, we show how this framework of probabilistic switching models enables extracting hierarchies of Markovian and auto-regressive locally linear controllers from nonlinear experts in an imitation learning scenario.

Cite this Paper

BibTeX
@InProceedings{pmlr-v120-abdulsamad20a, title = {Hierarchical Decomposition of Nonlinear Dynamics and Control for System Identification and Policy Distillation}, author = {Abdulsamad, Hany and Peters, Jan}, booktitle = {Proceedings of the 2nd Conference on Learning for Dynamics and Control}, pages = {904--914}, year = {2020}, editor = {Bayen, Alexandre M. and Jadbabaie, Ali and Pappas, George and Parrilo, Pablo A. and Recht, Benjamin and Tomlin, Claire and Zeilinger, Melanie}, volume = {120}, series = {Proceedings of Machine Learning Research}, month = {10--11 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v120/abdulsamad20a/abdulsamad20a.pdf}, url = {https://proceedings.mlr.press/v120/abdulsamad20a.html}, abstract = {Control of nonlinear systems with unknown dynamics is a major challenge on the road to fully autonomous agents. Current trends in reinforcement learning (RL) focus on complex representations of dynamics and policies. Such approaches have yielded impressive results in solving a variety of hard control tasks. However, this new sophistication has come with the cost of an overall reduction in our ability to interpret the resulting policies from a classical perspective, and the need for extremely over-parameterized controllers. In this paper, we take inspiration from the control community and apply the principles of hybrid switching systems, in order to break down complex representations into simpler components. We exploit the rich representational power of probabilistic graphical models and derive a new expectation-maximization (EM) algorithm for learning a generative model and automatically decomposing nonlinear dynamics into stochastic switching linear dynamical systems. Moreover, we show how this framework of probabilistic switching models enables extracting hierarchies of Markovian and auto-regressive locally linear controllers from nonlinear experts in an imitation learning scenario.} }
Endnote
%0 Conference Paper %T Hierarchical Decomposition of Nonlinear Dynamics and Control for System Identification and Policy Distillation %A Hany Abdulsamad %A Jan Peters %B Proceedings of the 2nd Conference on Learning for Dynamics and Control %C Proceedings of Machine Learning Research %D 2020 %E Alexandre M. Bayen %E Ali Jadbabaie %E George Pappas %E Pablo A. Parrilo %E Benjamin Recht %E Claire Tomlin %E Melanie Zeilinger %F pmlr-v120-abdulsamad20a %I PMLR %P 904--914 %U https://proceedings.mlr.press/v120/abdulsamad20a.html %V 120 %X Control of nonlinear systems with unknown dynamics is a major challenge on the road to fully autonomous agents. Current trends in reinforcement learning (RL) focus on complex representations of dynamics and policies. Such approaches have yielded impressive results in solving a variety of hard control tasks. However, this new sophistication has come with the cost of an overall reduction in our ability to interpret the resulting policies from a classical perspective, and the need for extremely over-parameterized controllers. In this paper, we take inspiration from the control community and apply the principles of hybrid switching systems, in order to break down complex representations into simpler components. We exploit the rich representational power of probabilistic graphical models and derive a new expectation-maximization (EM) algorithm for learning a generative model and automatically decomposing nonlinear dynamics into stochastic switching linear dynamical systems. Moreover, we show how this framework of probabilistic switching models enables extracting hierarchies of Markovian and auto-regressive locally linear controllers from nonlinear experts in an imitation learning scenario.
APA
Abdulsamad, H. & Peters, J.. (2020). Hierarchical Decomposition of Nonlinear Dynamics and Control for System Identification and Policy Distillation. Proceedings of the 2nd Conference on Learning for Dynamics and Control, in Proceedings of Machine Learning Research 120:904-914 Available from https://proceedings.mlr.press/v120/abdulsamad20a.html.

Related Material