Robust Multi-Agent Reinforcement Learning for Autonomous Vehicle in Noisy Highway Environments

Lilan Lin, Xiaotong Nie, Jian Hou
Proceedings of the 16th Asian Conference on Machine Learning, PMLR 260:1320-1335, 2025.

Abstract

The field of research on multi-agent reinforcement learning (MARL) algorithms in self-driving vehicles is rapidly expanding in mixed-traffic scenarios where autonomous vehicles (AVs) and human-driven vehicles (HDVs) coexist. Most studies assume that all AVs can obtain accurate state information. However, in real-world scenarios, noisy sensor measurements have a significant impact. To address this issue, we propose an effective and robust MARL algorithm Multi-Agent Proximal Policy Optimization with Curriculum-based Adversarial Learning (CA-MAPPO) for situations where the observation perturbations are considered. The proposed approach incorporates adversarial samples during training and adopts a curriculum learning approach by gradually increasing the noise intensity. By evaluating the proposed approach in the ideal environment and scenarios under noise attacks with varying intensities, experiment results demonstrate that the proposed algorithm enables AVs to achieve a success rate of over 70% for the multi-lane highway on-ramp merging task, achieving a maximum average speed of up to over 19 m/s and performing significantly better than the state-of-the-art MARL algorithms such as MAPPO and MAACKTR.

Cite this Paper

BibTeX
@InProceedings{pmlr-v260-lin25b, title = {Robust Multi-Agent Reinforcement Learning for Autonomous Vehicle in Noisy Highway Environments}, author = {Lin, Lilan and Nie, Xiaotong and Hou, Jian}, booktitle = {Proceedings of the 16th Asian Conference on Machine Learning}, pages = {1320--1335}, year = {2025}, editor = {Nguyen, Vu and Lin, Hsuan-Tien}, volume = {260}, series = {Proceedings of Machine Learning Research}, month = {05--08 Dec}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v260/main/assets/lin25b/lin25b.pdf}, url = {https://proceedings.mlr.press/v260/lin25b.html}, abstract = {The field of research on multi-agent reinforcement learning (MARL) algorithms in self-driving vehicles is rapidly expanding in mixed-traffic scenarios where autonomous vehicles (AVs) and human-driven vehicles (HDVs) coexist. Most studies assume that all AVs can obtain accurate state information. However, in real-world scenarios, noisy sensor measurements have a significant impact. To address this issue, we propose an effective and robust MARL algorithm Multi-Agent Proximal Policy Optimization with Curriculum-based Adversarial Learning (CA-MAPPO) for situations where the observation perturbations are considered. The proposed approach incorporates adversarial samples during training and adopts a curriculum learning approach by gradually increasing the noise intensity. By evaluating the proposed approach in the ideal environment and scenarios under noise attacks with varying intensities, experiment results demonstrate that the proposed algorithm enables AVs to achieve a success rate of over 70% for the multi-lane highway on-ramp merging task, achieving a maximum average speed of up to over 19 $m/s$ and performing significantly better than the state-of-the-art MARL algorithms such as MAPPO and MAACKTR.} }
Endnote
%0 Conference Paper %T Robust Multi-Agent Reinforcement Learning for Autonomous Vehicle in Noisy Highway Environments %A Lilan Lin %A Xiaotong Nie %A Jian Hou %B Proceedings of the 16th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Vu Nguyen %E Hsuan-Tien Lin %F pmlr-v260-lin25b %I PMLR %P 1320--1335 %U https://proceedings.mlr.press/v260/lin25b.html %V 260 %X The field of research on multi-agent reinforcement learning (MARL) algorithms in self-driving vehicles is rapidly expanding in mixed-traffic scenarios where autonomous vehicles (AVs) and human-driven vehicles (HDVs) coexist. Most studies assume that all AVs can obtain accurate state information. However, in real-world scenarios, noisy sensor measurements have a significant impact. To address this issue, we propose an effective and robust MARL algorithm Multi-Agent Proximal Policy Optimization with Curriculum-based Adversarial Learning (CA-MAPPO) for situations where the observation perturbations are considered. The proposed approach incorporates adversarial samples during training and adopts a curriculum learning approach by gradually increasing the noise intensity. By evaluating the proposed approach in the ideal environment and scenarios under noise attacks with varying intensities, experiment results demonstrate that the proposed algorithm enables AVs to achieve a success rate of over 70% for the multi-lane highway on-ramp merging task, achieving a maximum average speed of up to over 19 $m/s$ and performing significantly better than the state-of-the-art MARL algorithms such as MAPPO and MAACKTR.
APA
Lin, L., Nie, X. & Hou, J.. (2025). Robust Multi-Agent Reinforcement Learning for Autonomous Vehicle in Noisy Highway Environments. Proceedings of the 16th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 260:1320-1335 Available from https://proceedings.mlr.press/v260/lin25b.html.

Related Material