Learning an Internal Dynamics Model from Control Demonstration

Matthew Golub, Steven Chase, Byron Yu
Proceedings of the 30th International Conference on Machine Learning, PMLR 28(1):606-614, 2013.

Abstract

Much work in optimal control and inverse control has assumed that the controller has perfect knowledge of plant dynamics. However, if the controller is a human or animal subject, the subject’s internal dynamics model may differ from the true plant dynamics. Here, we consider the problem of learning the subject’s internal model from demonstrations of control and knowledge of task goals. Due to sensory feedback delay, the subject uses an internal model to generate an internal prediction of the current plant state, which may differ from the actual plant state. We develop a probabilistic framework and exact EM algorithm to jointly estimate the internal model, internal state trajectories, and feedback delay. We applied this framework to demonstrations by a nonhuman primate of brain-machine interface (BMI) control. We discovered that the subject’s internal model deviated from the true BMI plant dynamics and provided significantly better explanation of the recorded neural control signals than did the true plant dynamics.

Cite this Paper

BibTeX
@InProceedings{pmlr-v28-golub13, title = {Learning an Internal Dynamics Model from Control Demonstration}, author = {Golub, Matthew and Chase, Steven and Yu, Byron}, booktitle = {Proceedings of the 30th International Conference on Machine Learning}, pages = {606--614}, year = {2013}, editor = {Dasgupta, Sanjoy and McAllester, David}, volume = {28}, number = {1}, series = {Proceedings of Machine Learning Research}, address = {Atlanta, Georgia, USA}, month = {17--19 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v28/golub13.pdf}, url = {https://proceedings.mlr.press/v28/golub13.html}, abstract = {Much work in optimal control and inverse control has assumed that the controller has perfect knowledge of plant dynamics. However, if the controller is a human or animal subject, the subject’s internal dynamics model may differ from the true plant dynamics. Here, we consider the problem of learning the subject’s internal model from demonstrations of control and knowledge of task goals. Due to sensory feedback delay, the subject uses an internal model to generate an internal prediction of the current plant state, which may differ from the actual plant state. We develop a probabilistic framework and exact EM algorithm to jointly estimate the internal model, internal state trajectories, and feedback delay. We applied this framework to demonstrations by a nonhuman primate of brain-machine interface (BMI) control. We discovered that the subject’s internal model deviated from the true BMI plant dynamics and provided significantly better explanation of the recorded neural control signals than did the true plant dynamics. } }
Endnote
%0 Conference Paper %T Learning an Internal Dynamics Model from Control Demonstration %A Matthew Golub %A Steven Chase %A Byron Yu %B Proceedings of the 30th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2013 %E Sanjoy Dasgupta %E David McAllester %F pmlr-v28-golub13 %I PMLR %P 606--614 %U https://proceedings.mlr.press/v28/golub13.html %V 28 %N 1 %X Much work in optimal control and inverse control has assumed that the controller has perfect knowledge of plant dynamics. However, if the controller is a human or animal subject, the subject’s internal dynamics model may differ from the true plant dynamics. Here, we consider the problem of learning the subject’s internal model from demonstrations of control and knowledge of task goals. Due to sensory feedback delay, the subject uses an internal model to generate an internal prediction of the current plant state, which may differ from the actual plant state. We develop a probabilistic framework and exact EM algorithm to jointly estimate the internal model, internal state trajectories, and feedback delay. We applied this framework to demonstrations by a nonhuman primate of brain-machine interface (BMI) control. We discovered that the subject’s internal model deviated from the true BMI plant dynamics and provided significantly better explanation of the recorded neural control signals than did the true plant dynamics.
RIS
TY - CPAPER TI - Learning an Internal Dynamics Model from Control Demonstration AU - Matthew Golub AU - Steven Chase AU - Byron Yu BT - Proceedings of the 30th International Conference on Machine Learning DA - 2013/02/13 ED - Sanjoy Dasgupta ED - David McAllester ID - pmlr-v28-golub13 PB - PMLR DP - Proceedings of Machine Learning Research VL - 28 IS - 1 SP - 606 EP - 614 L1 - http://proceedings.mlr.press/v28/golub13.pdf UR - https://proceedings.mlr.press/v28/golub13.html AB - Much work in optimal control and inverse control has assumed that the controller has perfect knowledge of plant dynamics. However, if the controller is a human or animal subject, the subject’s internal dynamics model may differ from the true plant dynamics. Here, we consider the problem of learning the subject’s internal model from demonstrations of control and knowledge of task goals. Due to sensory feedback delay, the subject uses an internal model to generate an internal prediction of the current plant state, which may differ from the actual plant state. We develop a probabilistic framework and exact EM algorithm to jointly estimate the internal model, internal state trajectories, and feedback delay. We applied this framework to demonstrations by a nonhuman primate of brain-machine interface (BMI) control. We discovered that the subject’s internal model deviated from the true BMI plant dynamics and provided significantly better explanation of the recorded neural control signals than did the true plant dynamics. ER -
APA
Golub, M., Chase, S. & Yu, B.. (2013). Learning an Internal Dynamics Model from Control Demonstration. Proceedings of the 30th International Conference on Machine Learning, in Proceedings of Machine Learning Research 28(1):606-614 Available from https://proceedings.mlr.press/v28/golub13.html.

Related Material