Hamiltonian Asymmetric Fusion: One-Way Safe Directed Refinement under Modality Imbalance

Bingbing Chen, Congcong Liu, Dong Liang, Zhuo-Xu Cui
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:15638-15657, 2026.

Abstract

In RGB–D salient object detection, multimodal fusion is commonly implemented via symmetric token interaction, implicitly allowing information to flow in both directions. Under modality imbalance—when an auxiliary stream is substantially noisier than a designated primary stream—such symmetry permits a backflow channel through which auxiliary noise can affect the primary representation and accumulate across iterative refinement stages. We formulate fusion in this regime as directed refinement with one-way safety: the primary modality defines a guidance field, while the auxiliary representation is iteratively refined and auxiliary-induced primary perturbations are explicitly quantified. We propose Hamiltonian Asymmetric Fusion (HAF), a lightweight unrolled refinement block that updates auxiliary tokens through Hamiltonian-inspired, momentum-regularized dynamics with gated driving. The refinement force is instantiated by FFT-based spectral global correlation and modulated by a shared learnable spectral response to emphasize reliable frequency components with minimal parameters; a leaky momentum gate damps stale updates during multi-step refinement. Experiments on six RGB–D SOD benchmarks show consistent gains and substantially more graceful degradation under controlled auxiliary corruption.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-chen26ch, title = {{H}amiltonian Asymmetric Fusion: One-Way Safe Directed Refinement under Modality Imbalance}, author = {Chen, Bingbing and Liu, Congcong and Liang, Dong and Cui, Zhuo-Xu}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {15638--15657}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/chen26ch/chen26ch.pdf}, url = {https://proceedings.mlr.press/v306/chen26ch.html}, abstract = {In RGB–D salient object detection, multimodal fusion is commonly implemented via symmetric token interaction, implicitly allowing information to flow in both directions. Under modality imbalance—when an auxiliary stream is substantially noisier than a designated primary stream—such symmetry permits a backflow channel through which auxiliary noise can affect the primary representation and accumulate across iterative refinement stages. We formulate fusion in this regime as directed refinement with one-way safety: the primary modality defines a guidance field, while the auxiliary representation is iteratively refined and auxiliary-induced primary perturbations are explicitly quantified. We propose Hamiltonian Asymmetric Fusion (HAF), a lightweight unrolled refinement block that updates auxiliary tokens through Hamiltonian-inspired, momentum-regularized dynamics with gated driving. The refinement force is instantiated by FFT-based spectral global correlation and modulated by a shared learnable spectral response to emphasize reliable frequency components with minimal parameters; a leaky momentum gate damps stale updates during multi-step refinement. Experiments on six RGB–D SOD benchmarks show consistent gains and substantially more graceful degradation under controlled auxiliary corruption.} }
Endnote
%0 Conference Paper %T Hamiltonian Asymmetric Fusion: One-Way Safe Directed Refinement under Modality Imbalance %A Bingbing Chen %A Congcong Liu %A Dong Liang %A Zhuo-Xu Cui %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-chen26ch %I PMLR %P 15638--15657 %U https://proceedings.mlr.press/v306/chen26ch.html %V 306 %X In RGB–D salient object detection, multimodal fusion is commonly implemented via symmetric token interaction, implicitly allowing information to flow in both directions. Under modality imbalance—when an auxiliary stream is substantially noisier than a designated primary stream—such symmetry permits a backflow channel through which auxiliary noise can affect the primary representation and accumulate across iterative refinement stages. We formulate fusion in this regime as directed refinement with one-way safety: the primary modality defines a guidance field, while the auxiliary representation is iteratively refined and auxiliary-induced primary perturbations are explicitly quantified. We propose Hamiltonian Asymmetric Fusion (HAF), a lightweight unrolled refinement block that updates auxiliary tokens through Hamiltonian-inspired, momentum-regularized dynamics with gated driving. The refinement force is instantiated by FFT-based spectral global correlation and modulated by a shared learnable spectral response to emphasize reliable frequency components with minimal parameters; a leaky momentum gate damps stale updates during multi-step refinement. Experiments on six RGB–D SOD benchmarks show consistent gains and substantially more graceful degradation under controlled auxiliary corruption.
APA
Chen, B., Liu, C., Liang, D. & Cui, Z.. (2026). Hamiltonian Asymmetric Fusion: One-Way Safe Directed Refinement under Modality Imbalance. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:15638-15657 Available from https://proceedings.mlr.press/v306/chen26ch.html.

Related Material