Learning the Dynamics of Time Delay Systems with Trainable Delays
Proceedings of the 3rd Conference on Learning for Dynamics and Control, PMLR 144:930-942, 2021.
In this paper, we propose a delay learning algorithm for time delay neural networks (TDNNs) based on mini-batch gradient descent. We show that the proposed algorithm is suitable for learning the dynamics of nonlinear time delay systems using TDNNs with trainable delays. The delays are introduced in the input layer and are learned with the same approach as weights and biases. The learned delays are easy to interpret and they are not restricted to discrete values. We demonstrate the method with an example of learning the dynamics of an autonomous time delay system. We show the performance of two proposed network architectures with trainable delays and compare it to a standard TDNN which has a large number of fixed (non-trainable) input delays. We demonstrate that the networks with trainable input delays achieve significantly better performance in closed-loop simulations compared to the standard TDNN. We also highlight that possible undesired local minima may be caused by the delays in the networks.