One Model to Translate Them All: Universal Any-to-Any Translation for Heterogeneous Collaborative Perception

Yang Li, Weize Li, Quan Yuan, Shao Congzhang, Guiyang Luo, Yunqi Ba, Xuanhan Zhu, Xinyuan Ding, Xiaoyuan Fu, Jinglin Li
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:68044-68062, 2026.

Abstract

By sharing intermediate features, collaborative perception extends each agent’s sensing beyond standalone limits, but real-world feature modality heterogeneity remains a key barrier to effective fusion. Most existing methods, including direct adaption and protocol-based transforma-tion, typically rely on training adapters for newly emerging feature modalities and often require additional retraining or fine-tuning. Such repeated training is costly and is often infeasible across manufacturers due to model and data privacy con-straints, limiting real-world scalability. To address this issue, we propose UniTrans, a universal any-to-any feature modality translation model that instantiates translators on the fly for arbitrary modalities. UniTrans pretrains a bank of translator expert parameters and learns their combination coefficients as a function of source-to-target modality mapping. The mapping is measured in a modality-intrinsic latent space, where an intrinsic encoder extracts modality-specific yet scene-invariant codes from single-frame intermediate features, enabling UniTrans to instantiate translators in a zero-shot manner. Experiments on OPV2V-H and DAIR-V2X demonstrate that UniTrans consistently outperforms state-of-the-art methods in both simulated and real-world set-tings, enabling efficient any-to-any translation through a universal model. Code will be made available.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-li26cm, title = {One Model to Translate Them All: Universal Any-to-Any Translation for Heterogeneous Collaborative Perception}, author = {Li, Yang and Li, Weize and Yuan, Quan and Congzhang, Shao and Luo, Guiyang and Ba, Yunqi and Zhu, Xuanhan and Ding, Xinyuan and Fu, Xiaoyuan and Li, Jinglin}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {68044--68062}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/li26cm/li26cm.pdf}, url = {https://proceedings.mlr.press/v306/li26cm.html}, abstract = {By sharing intermediate features, collaborative perception extends each agent’s sensing beyond standalone limits, but real-world feature modality heterogeneity remains a key barrier to effective fusion. Most existing methods, including direct adaption and protocol-based transforma-tion, typically rely on training adapters for newly emerging feature modalities and often require additional retraining or fine-tuning. Such repeated training is costly and is often infeasible across manufacturers due to model and data privacy con-straints, limiting real-world scalability. To address this issue, we propose UniTrans, a universal any-to-any feature modality translation model that instantiates translators on the fly for arbitrary modalities. UniTrans pretrains a bank of translator expert parameters and learns their combination coefficients as a function of source-to-target modality mapping. The mapping is measured in a modality-intrinsic latent space, where an intrinsic encoder extracts modality-specific yet scene-invariant codes from single-frame intermediate features, enabling UniTrans to instantiate translators in a zero-shot manner. Experiments on OPV2V-H and DAIR-V2X demonstrate that UniTrans consistently outperforms state-of-the-art methods in both simulated and real-world set-tings, enabling efficient any-to-any translation through a universal model. Code will be made available.} }
Endnote
%0 Conference Paper %T One Model to Translate Them All: Universal Any-to-Any Translation for Heterogeneous Collaborative Perception %A Yang Li %A Weize Li %A Quan Yuan %A Shao Congzhang %A Guiyang Luo %A Yunqi Ba %A Xuanhan Zhu %A Xinyuan Ding %A Xiaoyuan Fu %A Jinglin Li %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-li26cm %I PMLR %P 68044--68062 %U https://proceedings.mlr.press/v306/li26cm.html %V 306 %X By sharing intermediate features, collaborative perception extends each agent’s sensing beyond standalone limits, but real-world feature modality heterogeneity remains a key barrier to effective fusion. Most existing methods, including direct adaption and protocol-based transforma-tion, typically rely on training adapters for newly emerging feature modalities and often require additional retraining or fine-tuning. Such repeated training is costly and is often infeasible across manufacturers due to model and data privacy con-straints, limiting real-world scalability. To address this issue, we propose UniTrans, a universal any-to-any feature modality translation model that instantiates translators on the fly for arbitrary modalities. UniTrans pretrains a bank of translator expert parameters and learns their combination coefficients as a function of source-to-target modality mapping. The mapping is measured in a modality-intrinsic latent space, where an intrinsic encoder extracts modality-specific yet scene-invariant codes from single-frame intermediate features, enabling UniTrans to instantiate translators in a zero-shot manner. Experiments on OPV2V-H and DAIR-V2X demonstrate that UniTrans consistently outperforms state-of-the-art methods in both simulated and real-world set-tings, enabling efficient any-to-any translation through a universal model. Code will be made available.
APA
Li, Y., Li, W., Yuan, Q., Congzhang, S., Luo, G., Ba, Y., Zhu, X., Ding, X., Fu, X. & Li, J.. (2026). One Model to Translate Them All: Universal Any-to-Any Translation for Heterogeneous Collaborative Perception. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:68044-68062 Available from https://proceedings.mlr.press/v306/li26cm.html.

Related Material