Let Occ Flow: Self-Supervised 3D Occupancy Flow Prediction

Yili Liu, Linzhan Mou, Xuan Yu, Chenrui Han, Sitong Mao, Rong Xiong, Yue Wang
Proceedings of The 8th Conference on Robot Learning, PMLR 270:2895-2912, 2025.

Abstract

Accurate perception of the dynamic environment is a fundamental task for autonomous driving and robot systems. This paper introduces Let Occ Flow, the first self-supervised work for joint 3D occupancy and occupancy flow prediction using only camera inputs, eliminating the need for 3D annotations. Utilizing TPV for unified scene representation and deformable attention layers for feature aggregation, our approach incorporates a novel attention-based temporal fusion module to capture dynamic object dependencies, followed by a 3D refine module for fine-gained volumetric representation. Besides, our method extends differentiable rendering to 3D volumetric flow fields, leveraging zero-shot 2D segmentation and optical flow cues for dynamic decomposition and motion optimization. Extensive experiments on nuScenes and KITTI datasets demonstrate the competitive performance of our approach over prior state-of-the-art methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v270-liu25e, title = {Let Occ Flow: Self-Supervised 3D Occupancy Flow Prediction}, author = {Liu, Yili and Mou, Linzhan and Yu, Xuan and Han, Chenrui and Mao, Sitong and Xiong, Rong and Wang, Yue}, booktitle = {Proceedings of The 8th Conference on Robot Learning}, pages = {2895--2912}, year = {2025}, editor = {Agrawal, Pulkit and Kroemer, Oliver and Burgard, Wolfram}, volume = {270}, series = {Proceedings of Machine Learning Research}, month = {06--09 Nov}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v270/main/assets/liu25e/liu25e.pdf}, url = {https://proceedings.mlr.press/v270/liu25e.html}, abstract = {Accurate perception of the dynamic environment is a fundamental task for autonomous driving and robot systems. This paper introduces Let Occ Flow, the first self-supervised work for joint 3D occupancy and occupancy flow prediction using only camera inputs, eliminating the need for 3D annotations. Utilizing TPV for unified scene representation and deformable attention layers for feature aggregation, our approach incorporates a novel attention-based temporal fusion module to capture dynamic object dependencies, followed by a 3D refine module for fine-gained volumetric representation. Besides, our method extends differentiable rendering to 3D volumetric flow fields, leveraging zero-shot 2D segmentation and optical flow cues for dynamic decomposition and motion optimization. Extensive experiments on nuScenes and KITTI datasets demonstrate the competitive performance of our approach over prior state-of-the-art methods.} }
Endnote
%0 Conference Paper %T Let Occ Flow: Self-Supervised 3D Occupancy Flow Prediction %A Yili Liu %A Linzhan Mou %A Xuan Yu %A Chenrui Han %A Sitong Mao %A Rong Xiong %A Yue Wang %B Proceedings of The 8th Conference on Robot Learning %C Proceedings of Machine Learning Research %D 2025 %E Pulkit Agrawal %E Oliver Kroemer %E Wolfram Burgard %F pmlr-v270-liu25e %I PMLR %P 2895--2912 %U https://proceedings.mlr.press/v270/liu25e.html %V 270 %X Accurate perception of the dynamic environment is a fundamental task for autonomous driving and robot systems. This paper introduces Let Occ Flow, the first self-supervised work for joint 3D occupancy and occupancy flow prediction using only camera inputs, eliminating the need for 3D annotations. Utilizing TPV for unified scene representation and deformable attention layers for feature aggregation, our approach incorporates a novel attention-based temporal fusion module to capture dynamic object dependencies, followed by a 3D refine module for fine-gained volumetric representation. Besides, our method extends differentiable rendering to 3D volumetric flow fields, leveraging zero-shot 2D segmentation and optical flow cues for dynamic decomposition and motion optimization. Extensive experiments on nuScenes and KITTI datasets demonstrate the competitive performance of our approach over prior state-of-the-art methods.
APA
Liu, Y., Mou, L., Yu, X., Han, C., Mao, S., Xiong, R. & Wang, Y.. (2025). Let Occ Flow: Self-Supervised 3D Occupancy Flow Prediction. Proceedings of The 8th Conference on Robot Learning, in Proceedings of Machine Learning Research 270:2895-2912 Available from https://proceedings.mlr.press/v270/liu25e.html.

Related Material