MMPD-Bench: Bridging Multimodal Fission with Multi-Polarimetric Modalities Decomposition

Yi He, Zimo Zhao, Yiming Yang, Xiaoyuan Cheng, Chao He, Yukun Hu
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:42298-42333, 2026.

Abstract

Recovering multiple physical parameters from high-dimensional optical measurements remains challenging in computational optics. We present MMPD-Bench, a pioneering benchmark that reframes multi-polarimetric modalities decomposition from Mueller matrix observations as a modality fission problem under the multi-modal learning paradigm. By replacing iterative numerical inversion with deep surrogate models, MMPD-Bench provides data, standardized solutions and evaluations to address the multi-physics modalities generation challenge. We benchmark representative architectures to this problem, including state-space models, vision transformers, conditional diffusion models, and neural operators, under a multi-faceted evaluation protocol that jointly assesses perceptual fidelity, physical consistency, robustness, and computational efficiency. Our analysis reveals non-trivial trade-offs between accuracy and robustness in accelerated high-fidelity polarimetric decomposition, highlighting key limitations of existing surrogates. To support reproducible research, we open-source the full codebase, together with a large-scale dataset of 21,412 high-resolution Mueller matrix observations acquired through extensive polarimetric measurements. We invite the community to further advance the intersection of polarization optics and multimodal representation learning.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-he26ay, title = {{MMPD}-Bench: Bridging Multimodal Fission with Multi-Polarimetric Modalities Decomposition}, author = {He, Yi and Zhao, Zimo and Yang, Yiming and Cheng, Xiaoyuan and He, Chao and Hu, Yukun}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {42298--42333}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/he26ay/he26ay.pdf}, url = {https://proceedings.mlr.press/v306/he26ay.html}, abstract = {Recovering multiple physical parameters from high-dimensional optical measurements remains challenging in computational optics. We present MMPD-Bench, a pioneering benchmark that reframes multi-polarimetric modalities decomposition from Mueller matrix observations as a modality fission problem under the multi-modal learning paradigm. By replacing iterative numerical inversion with deep surrogate models, MMPD-Bench provides data, standardized solutions and evaluations to address the multi-physics modalities generation challenge. We benchmark representative architectures to this problem, including state-space models, vision transformers, conditional diffusion models, and neural operators, under a multi-faceted evaluation protocol that jointly assesses perceptual fidelity, physical consistency, robustness, and computational efficiency. Our analysis reveals non-trivial trade-offs between accuracy and robustness in accelerated high-fidelity polarimetric decomposition, highlighting key limitations of existing surrogates. To support reproducible research, we open-source the full codebase, together with a large-scale dataset of 21,412 high-resolution Mueller matrix observations acquired through extensive polarimetric measurements. We invite the community to further advance the intersection of polarization optics and multimodal representation learning.} }
Endnote
%0 Conference Paper %T MMPD-Bench: Bridging Multimodal Fission with Multi-Polarimetric Modalities Decomposition %A Yi He %A Zimo Zhao %A Yiming Yang %A Xiaoyuan Cheng %A Chao He %A Yukun Hu %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-he26ay %I PMLR %P 42298--42333 %U https://proceedings.mlr.press/v306/he26ay.html %V 306 %X Recovering multiple physical parameters from high-dimensional optical measurements remains challenging in computational optics. We present MMPD-Bench, a pioneering benchmark that reframes multi-polarimetric modalities decomposition from Mueller matrix observations as a modality fission problem under the multi-modal learning paradigm. By replacing iterative numerical inversion with deep surrogate models, MMPD-Bench provides data, standardized solutions and evaluations to address the multi-physics modalities generation challenge. We benchmark representative architectures to this problem, including state-space models, vision transformers, conditional diffusion models, and neural operators, under a multi-faceted evaluation protocol that jointly assesses perceptual fidelity, physical consistency, robustness, and computational efficiency. Our analysis reveals non-trivial trade-offs between accuracy and robustness in accelerated high-fidelity polarimetric decomposition, highlighting key limitations of existing surrogates. To support reproducible research, we open-source the full codebase, together with a large-scale dataset of 21,412 high-resolution Mueller matrix observations acquired through extensive polarimetric measurements. We invite the community to further advance the intersection of polarization optics and multimodal representation learning.
APA
He, Y., Zhao, Z., Yang, Y., Cheng, X., He, C. & Hu, Y.. (2026). MMPD-Bench: Bridging Multimodal Fission with Multi-Polarimetric Modalities Decomposition. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:42298-42333 Available from https://proceedings.mlr.press/v306/he26ay.html.

Related Material