Learning the Dynamics of Time Delay Systems with Trainable Delays

Xunbi A. Ji, Tamás G. Molnár, Sergei S. Avedisov, Gábor Orosz
Proceedings of the 3rd Conference on Learning for Dynamics and Control, PMLR 144:930-942, 2021.

Abstract

In this paper, we propose a delay learning algorithm for time delay neural networks (TDNNs) based on mini-batch gradient descent. We show that the proposed algorithm is suitable for learning the dynamics of nonlinear time delay systems using TDNNs with trainable delays. The delays are introduced in the input layer and are learned with the same approach as weights and biases. The learned delays are easy to interpret and they are not restricted to discrete values. We demonstrate the method with an example of learning the dynamics of an autonomous time delay system. We show the performance of two proposed network architectures with trainable delays and compare it to a standard TDNN which has a large number of fixed (non-trainable) input delays. We demonstrate that the networks with trainable input delays achieve significantly better performance in closed-loop simulations compared to the standard TDNN. We also highlight that possible undesired local minima may be caused by the delays in the networks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v144-ji21a, title = {Learning the Dynamics of Time Delay Systems with Trainable Delays}, author = {Ji, Xunbi A. and Moln\'ar, Tam\'as G. and Avedisov, Sergei S. and Orosz, G\'abor}, booktitle = {Proceedings of the 3rd Conference on Learning for Dynamics and Control}, pages = {930--942}, year = {2021}, editor = {Jadbabaie, Ali and Lygeros, John and Pappas, George J. and A. Parrilo, Pablo and Recht, Benjamin and Tomlin, Claire J. and Zeilinger, Melanie N.}, volume = {144}, series = {Proceedings of Machine Learning Research}, month = {07 -- 08 June}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v144/ji21a/ji21a.pdf}, url = {https://proceedings.mlr.press/v144/ji21a.html}, abstract = {In this paper, we propose a delay learning algorithm for time delay neural networks (TDNNs) based on mini-batch gradient descent. We show that the proposed algorithm is suitable for learning the dynamics of nonlinear time delay systems using TDNNs with trainable delays. The delays are introduced in the input layer and are learned with the same approach as weights and biases. The learned delays are easy to interpret and they are not restricted to discrete values. We demonstrate the method with an example of learning the dynamics of an autonomous time delay system. We show the performance of two proposed network architectures with trainable delays and compare it to a standard TDNN which has a large number of fixed (non-trainable) input delays. We demonstrate that the networks with trainable input delays achieve significantly better performance in closed-loop simulations compared to the standard TDNN. We also highlight that possible undesired local minima may be caused by the delays in the networks.} }
Endnote
%0 Conference Paper %T Learning the Dynamics of Time Delay Systems with Trainable Delays %A Xunbi A. Ji %A Tamás G. Molnár %A Sergei S. Avedisov %A Gábor Orosz %B Proceedings of the 3rd Conference on Learning for Dynamics and Control %C Proceedings of Machine Learning Research %D 2021 %E Ali Jadbabaie %E John Lygeros %E George J. Pappas %E Pablo A. Parrilo %E Benjamin Recht %E Claire J. Tomlin %E Melanie N. Zeilinger %F pmlr-v144-ji21a %I PMLR %P 930--942 %U https://proceedings.mlr.press/v144/ji21a.html %V 144 %X In this paper, we propose a delay learning algorithm for time delay neural networks (TDNNs) based on mini-batch gradient descent. We show that the proposed algorithm is suitable for learning the dynamics of nonlinear time delay systems using TDNNs with trainable delays. The delays are introduced in the input layer and are learned with the same approach as weights and biases. The learned delays are easy to interpret and they are not restricted to discrete values. We demonstrate the method with an example of learning the dynamics of an autonomous time delay system. We show the performance of two proposed network architectures with trainable delays and compare it to a standard TDNN which has a large number of fixed (non-trainable) input delays. We demonstrate that the networks with trainable input delays achieve significantly better performance in closed-loop simulations compared to the standard TDNN. We also highlight that possible undesired local minima may be caused by the delays in the networks.
APA
Ji, X.A., Molnár, T.G., Avedisov, S.S. & Orosz, G.. (2021). Learning the Dynamics of Time Delay Systems with Trainable Delays. Proceedings of the 3rd Conference on Learning for Dynamics and Control, in Proceedings of Machine Learning Research 144:930-942 Available from https://proceedings.mlr.press/v144/ji21a.html.

Related Material