KURL: A Knowledge-Guided Reinforcement Learning Model for Active Object Tracking

Xin Liu, Jie Tan, Xiaoguang Ren, Weiya Ren, Huadong Dai
Proceedings of the 15th Asian Conference on Machine Learning, PMLR 222:818-833, 2024.

Abstract

Recent studies have shown that active object tracking algorithms based on deep reinforcement learning have the difficulty of model training while achieving favorable tracking outcomes. In addition, current active object tracking methods are not suitable for air-to-ground object tracking scenarios in high-altitude environments, such as air search and rescue. Therefore, we proposed a Knowledge-gUided Reinforcement learning (KURL) model for active object tracking, which includes two embedded knowledge-guided models (i.e., the state recognition model and the world model), together with a reinforcement learning module. The state recognition model utilizes the correlation between the observed states and image quality (as measured by object recognition probability) as prior knowledge to guide reinforcement learning algorithm to improve the observed image quality. The reinforcement learning module actively controls the Pan-Tilt-Zoom (PTZ) camera to achieve stable tracking. Additionally, a world model is proposed to replace the traditional Unreal Engine (UE) simulator for model training, which significantly enhancing the training efficiency (about ten times). The results indicate that the KURL model can significantly enhance the image quality, stability and robustness of tracking, compared with other methods in similar tasks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v222-liu24c, title = {{KURL}: {A} Knowledge-Guided Reinforcement Learning Model for Active Object Tracking}, author = {Liu, Xin and Tan, Jie and Ren, Xiaoguang and Ren, Weiya and Dai, Huadong}, booktitle = {Proceedings of the 15th Asian Conference on Machine Learning}, pages = {818--833}, year = {2024}, editor = {Yanıkoğlu, Berrin and Buntine, Wray}, volume = {222}, series = {Proceedings of Machine Learning Research}, month = {11--14 Nov}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v222/liu24c/liu24c.pdf}, url = {https://proceedings.mlr.press/v222/liu24c.html}, abstract = {Recent studies have shown that active object tracking algorithms based on deep reinforcement learning have the difficulty of model training while achieving favorable tracking outcomes. In addition, current active object tracking methods are not suitable for air-to-ground object tracking scenarios in high-altitude environments, such as air search and rescue. Therefore, we proposed a Knowledge-gUided Reinforcement learning (KURL) model for active object tracking, which includes two embedded knowledge-guided models (i.e., the state recognition model and the world model), together with a reinforcement learning module. The state recognition model utilizes the correlation between the observed states and image quality (as measured by object recognition probability) as prior knowledge to guide reinforcement learning algorithm to improve the observed image quality. The reinforcement learning module actively controls the Pan-Tilt-Zoom (PTZ) camera to achieve stable tracking. Additionally, a world model is proposed to replace the traditional Unreal Engine (UE) simulator for model training, which significantly enhancing the training efficiency (about ten times). The results indicate that the KURL model can significantly enhance the image quality, stability and robustness of tracking, compared with other methods in similar tasks.} }
Endnote
%0 Conference Paper %T KURL: A Knowledge-Guided Reinforcement Learning Model for Active Object Tracking %A Xin Liu %A Jie Tan %A Xiaoguang Ren %A Weiya Ren %A Huadong Dai %B Proceedings of the 15th Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2024 %E Berrin Yanıkoğlu %E Wray Buntine %F pmlr-v222-liu24c %I PMLR %P 818--833 %U https://proceedings.mlr.press/v222/liu24c.html %V 222 %X Recent studies have shown that active object tracking algorithms based on deep reinforcement learning have the difficulty of model training while achieving favorable tracking outcomes. In addition, current active object tracking methods are not suitable for air-to-ground object tracking scenarios in high-altitude environments, such as air search and rescue. Therefore, we proposed a Knowledge-gUided Reinforcement learning (KURL) model for active object tracking, which includes two embedded knowledge-guided models (i.e., the state recognition model and the world model), together with a reinforcement learning module. The state recognition model utilizes the correlation between the observed states and image quality (as measured by object recognition probability) as prior knowledge to guide reinforcement learning algorithm to improve the observed image quality. The reinforcement learning module actively controls the Pan-Tilt-Zoom (PTZ) camera to achieve stable tracking. Additionally, a world model is proposed to replace the traditional Unreal Engine (UE) simulator for model training, which significantly enhancing the training efficiency (about ten times). The results indicate that the KURL model can significantly enhance the image quality, stability and robustness of tracking, compared with other methods in similar tasks.
APA
Liu, X., Tan, J., Ren, X., Ren, W. & Dai, H.. (2024). KURL: A Knowledge-Guided Reinforcement Learning Model for Active Object Tracking. Proceedings of the 15th Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 222:818-833 Available from https://proceedings.mlr.press/v222/liu24c.html.

Related Material