Faster Target Encirclement with Utilization of Obstacles via Multi-Agent Reinforcement Learning

Yuxi Zheng, Yongjun Zhang, Chenran Zhao, Huanhuan Yang, Tongyue Li, Qianying Ouyang, Ying Chen
Proceedings of the 15th Asian Conference on Machine Learning, PMLR 222:1731-1746, 2024.

Abstract

Multi-agent encirclement refers to controlling multiple agents to restrict the movement of a target and surround it with a specific formation. However, two challenges remain: encirclement in obstacle scenarios and encirclement of a faster target. In obstacle scenarios, we propose the utilization of obstacles for facilitating encirclement and introduce the concept of contributing angle to quantify the contribution of agents and obstacles, which enables agents to effectively utilize obstacles while mitigating the credit assignment problem. To address the challenge of encircling a faster target, we propose a two-stage encirclement method inspired by lions’ hunting strategy, effectively preventing target escape. We design the reward function based on the contributing angle and the lion encirclement method, integrating it with the Multi-Agent Deep Deterministic Policy Gradient (MADDPG). The simulation results demonstrate that our method can utilize obstacles to complete encirclement and has a higher success rate. In some conditions with insufficient numbers of agents, our methods can still accomplish the task. Ablation experiments are conducted to verify the effectiveness of the contributing angle and the lion encirclement method respectively.

Cite this Paper

BibTeX
@InProceedings{pmlr-v222-zheng24b, title = {Faster Target Encirclement with Utilization of Obstacles via Multi-Agent Reinforcement Learning}, author = {Zheng, Yuxi and Zhang, Yongjun and Zhao, Chenran and Yang, Huanhuan and Li, Tongyue and Ouyang, Qianying and Chen, Ying}, booktitle = {Proceedings of the 15th Asian Conference on Machine Learning}, pages = {1731--1746}, year = {2024}, editor = {Yanıkoğlu, Berrin and Buntine, Wray}, volume = {222}, series = {Proceedings of Machine Learning Research}, month = {11--14 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v222/zheng24b/zheng24b.pdf}, url = {https://proceedings.mlr.press/v222/zheng24b.html}, abstract = {Multi-agent encirclement refers to controlling multiple agents to restrict the movement of a target and surround it with a specific formation. However, two challenges remain: encirclement in obstacle scenarios and encirclement of a faster target. In obstacle scenarios, we propose the utilization of obstacles for facilitating encirclement and introduce the concept of contributing angle to quantify the contribution of agents and obstacles, which enables agents to effectively utilize obstacles while mitigating the credit assignment problem. To address the challenge of encircling a faster target, we propose a two-stage encirclement method inspired by lions’ hunting strategy, effectively preventing target escape. We design the reward function based on the contributing angle and the lion encirclement method, integrating it with the Multi-Agent Deep Deterministic Policy Gradient (MADDPG). The simulation results demonstrate that our method can utilize obstacles to complete encirclement and has a higher success rate. In some conditions with insufficient numbers of agents, our methods can still accomplish the task. Ablation experiments are conducted to verify the effectiveness of the contributing angle and the lion encirclement method respectively.} }
Endnote
%0 Conference Paper %T Faster Target Encirclement with Utilization of Obstacles via Multi-Agent Reinforcement Learning %A Yuxi Zheng %A Yongjun Zhang %A Chenran Zhao %A Huanhuan Yang %A Tongyue Li %A Qianying Ouyang %A Ying Chen %B Proceedings of the 15th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Berrin Yanıkoğlu %E Wray Buntine %F pmlr-v222-zheng24b %I PMLR %P 1731--1746 %U https://proceedings.mlr.press/v222/zheng24b.html %V 222 %X Multi-agent encirclement refers to controlling multiple agents to restrict the movement of a target and surround it with a specific formation. However, two challenges remain: encirclement in obstacle scenarios and encirclement of a faster target. In obstacle scenarios, we propose the utilization of obstacles for facilitating encirclement and introduce the concept of contributing angle to quantify the contribution of agents and obstacles, which enables agents to effectively utilize obstacles while mitigating the credit assignment problem. To address the challenge of encircling a faster target, we propose a two-stage encirclement method inspired by lions’ hunting strategy, effectively preventing target escape. We design the reward function based on the contributing angle and the lion encirclement method, integrating it with the Multi-Agent Deep Deterministic Policy Gradient (MADDPG). The simulation results demonstrate that our method can utilize obstacles to complete encirclement and has a higher success rate. In some conditions with insufficient numbers of agents, our methods can still accomplish the task. Ablation experiments are conducted to verify the effectiveness of the contributing angle and the lion encirclement method respectively.
APA
Zheng, Y., Zhang, Y., Zhao, C., Yang, H., Li, T., Ouyang, Q. & Chen, Y.. (2024). Faster Target Encirclement with Utilization of Obstacles via Multi-Agent Reinforcement Learning. Proceedings of the 15th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 222:1731-1746 Available from https://proceedings.mlr.press/v222/zheng24b.html.

Related Material