Balanced reward-inspired reinforcement learning for autonomous vehicle racing

Zhen Tian, Dezong Zhao, Zhihao Lin, David Flynn, Wenjing Zhao, Daxin Tian
Proceedings of the 6th Annual Learning for Dynamics & Control Conference, PMLR 242:628-640, 2024.

Abstract

Autonomous vehicle racing has attracted extensive interest due to its great potential in autonomous driving at the extreme limits. Model-based and learning-based methods are being widely used in autonomous racing. However, model-based methods cannot cope with the dynamic environments when only local perception is available. As a comparison, learning-based methods can handle complex environments under local perception. Recently, deep reinforcement learning (DRL) has gained popularity in autonomous racing. DRL outperforms conventional learning- based methods by handling complex situations and leveraging local information. DRL algorithms, such as the proximal policy algorithm, can achieve a good balance between the execution time and safety in autonomous vehicle competition. However, the training outcomes of conventional DRL methods exhibit inconsistent correctness in decision-making. The instability in decision-making introduces safety concerns in autonomous vehicle racing, such as collisions into track boundaries. The proposed algorithm is capable to avoid collisions and improve the training quality. Simulation results on a physical engine demonstrate that the proposed algorithm outperforms other DRL algorithms in achieving safer control during sharp bends, fewer collisions into track boundaries, and higher training quality among multiple tracks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v242-tian24a, title = {Balanced reward-inspired reinforcement learning for autonomous vehicle racing}, author = {Tian, Zhen and Zhao, Dezong and Lin, Zhihao and Flynn, David and Zhao, Wenjing and Tian, Daxin}, booktitle = {Proceedings of the 6th Annual Learning for Dynamics & Control Conference}, pages = {628--640}, year = {2024}, editor = {Abate, Alessandro and Cannon, Mark and Margellos, Kostas and Papachristodoulou, Antonis}, volume = {242}, series = {Proceedings of Machine Learning Research}, month = {15--17 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v242/tian24a/tian24a.pdf}, url = {https://proceedings.mlr.press/v242/tian24a.html}, abstract = {Autonomous vehicle racing has attracted extensive interest due to its great potential in autonomous driving at the extreme limits. Model-based and learning-based methods are being widely used in autonomous racing. However, model-based methods cannot cope with the dynamic environments when only local perception is available. As a comparison, learning-based methods can handle complex environments under local perception. Recently, deep reinforcement learning (DRL) has gained popularity in autonomous racing. DRL outperforms conventional learning- based methods by handling complex situations and leveraging local information. DRL algorithms, such as the proximal policy algorithm, can achieve a good balance between the execution time and safety in autonomous vehicle competition. However, the training outcomes of conventional DRL methods exhibit inconsistent correctness in decision-making. The instability in decision-making introduces safety concerns in autonomous vehicle racing, such as collisions into track boundaries. The proposed algorithm is capable to avoid collisions and improve the training quality. Simulation results on a physical engine demonstrate that the proposed algorithm outperforms other DRL algorithms in achieving safer control during sharp bends, fewer collisions into track boundaries, and higher training quality among multiple tracks.} }
Endnote
%0 Conference Paper %T Balanced reward-inspired reinforcement learning for autonomous vehicle racing %A Zhen Tian %A Dezong Zhao %A Zhihao Lin %A David Flynn %A Wenjing Zhao %A Daxin Tian %B Proceedings of the 6th Annual Learning for Dynamics & Control Conference %C Proceedings of Machine Learning Research %D 2024 %E Alessandro Abate %E Mark Cannon %E Kostas Margellos %E Antonis Papachristodoulou %F pmlr-v242-tian24a %I PMLR %P 628--640 %U https://proceedings.mlr.press/v242/tian24a.html %V 242 %X Autonomous vehicle racing has attracted extensive interest due to its great potential in autonomous driving at the extreme limits. Model-based and learning-based methods are being widely used in autonomous racing. However, model-based methods cannot cope with the dynamic environments when only local perception is available. As a comparison, learning-based methods can handle complex environments under local perception. Recently, deep reinforcement learning (DRL) has gained popularity in autonomous racing. DRL outperforms conventional learning- based methods by handling complex situations and leveraging local information. DRL algorithms, such as the proximal policy algorithm, can achieve a good balance between the execution time and safety in autonomous vehicle competition. However, the training outcomes of conventional DRL methods exhibit inconsistent correctness in decision-making. The instability in decision-making introduces safety concerns in autonomous vehicle racing, such as collisions into track boundaries. The proposed algorithm is capable to avoid collisions and improve the training quality. Simulation results on a physical engine demonstrate that the proposed algorithm outperforms other DRL algorithms in achieving safer control during sharp bends, fewer collisions into track boundaries, and higher training quality among multiple tracks.
APA
Tian, Z., Zhao, D., Lin, Z., Flynn, D., Zhao, W. & Tian, D.. (2024). Balanced reward-inspired reinforcement learning for autonomous vehicle racing. Proceedings of the 6th Annual Learning for Dynamics & Control Conference, in Proceedings of Machine Learning Research 242:628-640 Available from https://proceedings.mlr.press/v242/tian24a.html.

Related Material