MICE-Bench: A Challenging and Comprehensive Benchmark for Multi-Reference Image Creation and Editing

Siqi Luo, Huayu Zheng, Jianghan Shen, Yi Xin, Luxin Xu, Jiyao Liu, Xinyu Zhang, Hang Zhou, Pengyu Xie, Xiaohui Li, Shuo Cao, Yuandong Pu, Junjun He, Bin Fu, Yihao Liu, Yu Qiao, Guangtao Zhai, Yuewen Cao, Xiaohong Liu
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:83169-83197, 2026.

Abstract

The paradigm of visual generation is rapidly shifting from single-image conditioning toward multi-image conditioning, making the ability to synthesize and edit images based on multiple visual references a critical capability. Despite this trend, existing benchmarks remain largely limited to single-reference scenarios or narrowly defined tasks, leaving model behavior under complex multi-concept composition insufficiently explored. To bridge this gap, we introduce MICE-Bench, a comprehensive benchmark for Multi-reference Image Creation and Editing. The benchmark is designed around three core principles: 1) heterogeneous concept composition across seven visual dimensions; 2) varying levels of constraint density, ranging from dual-concept to seven-concept configurations; 3) concept-centric data construction and benchmark evaluation, enabling fine-grained analysis of interactions among multiple concepts. MICE-Bench consists of 3,119 high-quality test cases within a unified concept space. Using an 8-dimensional evaluation metric, we systematically evaluate 13 state-of-the-art models. Our results show that although closed-source models maintain a clear performance advantage, all models experience notable degradation in concept consistency and physical realism as concept complexity increases.This indicates that current models rely on superficial composition rather than genuine multi-concept synthesis, highlighting substantial room for future improvement.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-luo26ak, title = {{MICE}-Bench: A Challenging and Comprehensive Benchmark for Multi-Reference Image Creation and Editing}, author = {Luo, Siqi and Zheng, Huayu and Shen, Jianghan and Xin, Yi and Xu, Luxin and Liu, Jiyao and Zhang, Xinyu and Zhou, Hang and Xie, Pengyu and Li, Xiaohui and Cao, Shuo and Pu, Yuandong and He, Junjun and Fu, Bin and Liu, Yihao and Qiao, Yu and Zhai, Guangtao and Cao, Yuewen and Liu, Xiaohong}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {83169--83197}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/luo26ak/luo26ak.pdf}, url = {https://proceedings.mlr.press/v306/luo26ak.html}, abstract = {The paradigm of visual generation is rapidly shifting from single-image conditioning toward multi-image conditioning, making the ability to synthesize and edit images based on multiple visual references a critical capability. Despite this trend, existing benchmarks remain largely limited to single-reference scenarios or narrowly defined tasks, leaving model behavior under complex multi-concept composition insufficiently explored. To bridge this gap, we introduce MICE-Bench, a comprehensive benchmark for Multi-reference Image Creation and Editing. The benchmark is designed around three core principles: 1) heterogeneous concept composition across seven visual dimensions; 2) varying levels of constraint density, ranging from dual-concept to seven-concept configurations; 3) concept-centric data construction and benchmark evaluation, enabling fine-grained analysis of interactions among multiple concepts. MICE-Bench consists of 3,119 high-quality test cases within a unified concept space. Using an 8-dimensional evaluation metric, we systematically evaluate 13 state-of-the-art models. Our results show that although closed-source models maintain a clear performance advantage, all models experience notable degradation in concept consistency and physical realism as concept complexity increases.This indicates that current models rely on superficial composition rather than genuine multi-concept synthesis, highlighting substantial room for future improvement.} }
Endnote
%0 Conference Paper %T MICE-Bench: A Challenging and Comprehensive Benchmark for Multi-Reference Image Creation and Editing %A Siqi Luo %A Huayu Zheng %A Jianghan Shen %A Yi Xin %A Luxin Xu %A Jiyao Liu %A Xinyu Zhang %A Hang Zhou %A Pengyu Xie %A Xiaohui Li %A Shuo Cao %A Yuandong Pu %A Junjun He %A Bin Fu %A Yihao Liu %A Yu Qiao %A Guangtao Zhai %A Yuewen Cao %A Xiaohong Liu %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-luo26ak %I PMLR %P 83169--83197 %U https://proceedings.mlr.press/v306/luo26ak.html %V 306 %X The paradigm of visual generation is rapidly shifting from single-image conditioning toward multi-image conditioning, making the ability to synthesize and edit images based on multiple visual references a critical capability. Despite this trend, existing benchmarks remain largely limited to single-reference scenarios or narrowly defined tasks, leaving model behavior under complex multi-concept composition insufficiently explored. To bridge this gap, we introduce MICE-Bench, a comprehensive benchmark for Multi-reference Image Creation and Editing. The benchmark is designed around three core principles: 1) heterogeneous concept composition across seven visual dimensions; 2) varying levels of constraint density, ranging from dual-concept to seven-concept configurations; 3) concept-centric data construction and benchmark evaluation, enabling fine-grained analysis of interactions among multiple concepts. MICE-Bench consists of 3,119 high-quality test cases within a unified concept space. Using an 8-dimensional evaluation metric, we systematically evaluate 13 state-of-the-art models. Our results show that although closed-source models maintain a clear performance advantage, all models experience notable degradation in concept consistency and physical realism as concept complexity increases.This indicates that current models rely on superficial composition rather than genuine multi-concept synthesis, highlighting substantial room for future improvement.
APA
Luo, S., Zheng, H., Shen, J., Xin, Y., Xu, L., Liu, J., Zhang, X., Zhou, H., Xie, P., Li, X., Cao, S., Pu, Y., He, J., Fu, B., Liu, Y., Qiao, Y., Zhai, G., Cao, Y. & Liu, X.. (2026). MICE-Bench: A Challenging and Comprehensive Benchmark for Multi-Reference Image Creation and Editing. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:83169-83197 Available from https://proceedings.mlr.press/v306/luo26ak.html.

Related Material