MFH-NAS:A Hybrid Neural Architecture Search Framework for Multimodal Fusion Object Detection

Quanwei Gao, Shuqi Zhao, Ruyu Wang, Shuyin Zhang, Cong Liu, Zirui Luo
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:33985-33996, 2026.

Abstract

Multimodal fusion object detection faces a substantial modality gap at the same backbone stage. This makes predefined stage-aligned fusion insufficient for cross-stage interactions. We propose MFH-NAS, a hybrid neural architecture search framework that automatically discovers fusion architectures to better leverage cross-modal complementarity. MFH-NAS searches both local fusion primitives and stage-level fusion connectivity. It targets fusion operator design and fusion stage selection. It couples differentiable search with evolutionary search. Differentiable search learns architecture parameters for local fusion primitives. Evolutionary search explores global fusion topologies, including stage selection and cross-stage connection patterns. The joint search balances exploitation and exploration and mitigates premature convergence. It yields fusion structures that strengthen cross-stage interactions.We evaluate MFH-NAS on three public benchmarks, LLVIP, RGBT-Tiny, and M3FD. MFH-NAS consistently outperforms handcrafted fusion-stage designs and prior stage-searching NAS baselines, improving mAP@0.5 from 85.3% to 88.2% over strong fixed-stage fusion methods and delivering gains across all benchmarks.The code will be released at https://github.com/someboy0/MFH-NAS.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-gao26an, title = {{MFH}-{NAS}:{A} Hybrid Neural Architecture Search Framework for Multimodal Fusion Object Detection}, author = {Gao, Quanwei and Zhao, Shuqi and Wang, Ruyu and Zhang, Shuyin and Liu, Cong and Luo, Zirui}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {33985--33996}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/gao26an/gao26an.pdf}, url = {https://proceedings.mlr.press/v306/gao26an.html}, abstract = {Multimodal fusion object detection faces a substantial modality gap at the same backbone stage. This makes predefined stage-aligned fusion insufficient for cross-stage interactions. We propose MFH-NAS, a hybrid neural architecture search framework that automatically discovers fusion architectures to better leverage cross-modal complementarity. MFH-NAS searches both local fusion primitives and stage-level fusion connectivity. It targets fusion operator design and fusion stage selection. It couples differentiable search with evolutionary search. Differentiable search learns architecture parameters for local fusion primitives. Evolutionary search explores global fusion topologies, including stage selection and cross-stage connection patterns. The joint search balances exploitation and exploration and mitigates premature convergence. It yields fusion structures that strengthen cross-stage interactions.We evaluate MFH-NAS on three public benchmarks, LLVIP, RGBT-Tiny, and M3FD. MFH-NAS consistently outperforms handcrafted fusion-stage designs and prior stage-searching NAS baselines, improving mAP@0.5 from 85.3% to 88.2% over strong fixed-stage fusion methods and delivering gains across all benchmarks.The code will be released at https://github.com/someboy0/MFH-NAS.} }
Endnote
%0 Conference Paper %T MFH-NAS:A Hybrid Neural Architecture Search Framework for Multimodal Fusion Object Detection %A Quanwei Gao %A Shuqi Zhao %A Ruyu Wang %A Shuyin Zhang %A Cong Liu %A Zirui Luo %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-gao26an %I PMLR %P 33985--33996 %U https://proceedings.mlr.press/v306/gao26an.html %V 306 %X Multimodal fusion object detection faces a substantial modality gap at the same backbone stage. This makes predefined stage-aligned fusion insufficient for cross-stage interactions. We propose MFH-NAS, a hybrid neural architecture search framework that automatically discovers fusion architectures to better leverage cross-modal complementarity. MFH-NAS searches both local fusion primitives and stage-level fusion connectivity. It targets fusion operator design and fusion stage selection. It couples differentiable search with evolutionary search. Differentiable search learns architecture parameters for local fusion primitives. Evolutionary search explores global fusion topologies, including stage selection and cross-stage connection patterns. The joint search balances exploitation and exploration and mitigates premature convergence. It yields fusion structures that strengthen cross-stage interactions.We evaluate MFH-NAS on three public benchmarks, LLVIP, RGBT-Tiny, and M3FD. MFH-NAS consistently outperforms handcrafted fusion-stage designs and prior stage-searching NAS baselines, improving mAP@0.5 from 85.3% to 88.2% over strong fixed-stage fusion methods and delivering gains across all benchmarks.The code will be released at https://github.com/someboy0/MFH-NAS.
APA
Gao, Q., Zhao, S., Wang, R., Zhang, S., Liu, C. & Luo, Z.. (2026). MFH-NAS:A Hybrid Neural Architecture Search Framework for Multimodal Fusion Object Detection. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:33985-33996 Available from https://proceedings.mlr.press/v306/gao26an.html.

Related Material