Benchmarking and Improving Fine-Grained Text-to-Image Alignment via Paired Reinforcement Learning

Kaihang Pan, Wendong Bu, Yuruo Wu, Kai Shen, Yang Wu, Yun Zhu, Zehan Wang, Yunfei Li, Hang Zhao, Juncheng Li, Siliang Tang
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:95696-95713, 2026.

Abstract

While recent autoregressive models have achieved text-to-image generation performance comparable to diffusion models, they significantly struggle with fine-grained semantic alignment. To rigorously evaluate this limitation, we introduce DeltaBench, a benchmark featuring paired prompts with subtle fine-grained differences, which reveals that existing models fail to achieve precise control over visual tokens. To bridge this gap, we propose FocusDiff, a comprehensive framework that enhances alignment by learning from subtle differences in similar text-image pairs. Specifically, we construct FocusDiff-Data, a large-scale dataset of paired samples derived from image editing tasks to capture localized semantic shifts. Furthermore, we introduce Pair-GRPO, an improved reinforcement learning algorithm that extends GRPO to paired samples. Extensive experiments demonstrate that our approach outperforms most prior prominent methods on both DeltaBench and existing benchmarks.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-pan26b, title = {Benchmarking and Improving Fine-Grained Text-to-Image Alignment via Paired Reinforcement Learning}, author = {Pan, Kaihang and Bu, Wendong and Wu, Yuruo and Shen, Kai and Wu, Yang and Zhu, Yun and Wang, Zehan and Li, Yunfei and Zhao, Hang and Li, Juncheng and Tang, Siliang}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {95696--95713}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/pan26b/pan26b.pdf}, url = {https://proceedings.mlr.press/v306/pan26b.html}, abstract = {While recent autoregressive models have achieved text-to-image generation performance comparable to diffusion models, they significantly struggle with fine-grained semantic alignment. To rigorously evaluate this limitation, we introduce DeltaBench, a benchmark featuring paired prompts with subtle fine-grained differences, which reveals that existing models fail to achieve precise control over visual tokens. To bridge this gap, we propose FocusDiff, a comprehensive framework that enhances alignment by learning from subtle differences in similar text-image pairs. Specifically, we construct FocusDiff-Data, a large-scale dataset of paired samples derived from image editing tasks to capture localized semantic shifts. Furthermore, we introduce Pair-GRPO, an improved reinforcement learning algorithm that extends GRPO to paired samples. Extensive experiments demonstrate that our approach outperforms most prior prominent methods on both DeltaBench and existing benchmarks.} }
Endnote
%0 Conference Paper %T Benchmarking and Improving Fine-Grained Text-to-Image Alignment via Paired Reinforcement Learning %A Kaihang Pan %A Wendong Bu %A Yuruo Wu %A Kai Shen %A Yang Wu %A Yun Zhu %A Zehan Wang %A Yunfei Li %A Hang Zhao %A Juncheng Li %A Siliang Tang %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-pan26b %I PMLR %P 95696--95713 %U https://proceedings.mlr.press/v306/pan26b.html %V 306 %X While recent autoregressive models have achieved text-to-image generation performance comparable to diffusion models, they significantly struggle with fine-grained semantic alignment. To rigorously evaluate this limitation, we introduce DeltaBench, a benchmark featuring paired prompts with subtle fine-grained differences, which reveals that existing models fail to achieve precise control over visual tokens. To bridge this gap, we propose FocusDiff, a comprehensive framework that enhances alignment by learning from subtle differences in similar text-image pairs. Specifically, we construct FocusDiff-Data, a large-scale dataset of paired samples derived from image editing tasks to capture localized semantic shifts. Furthermore, we introduce Pair-GRPO, an improved reinforcement learning algorithm that extends GRPO to paired samples. Extensive experiments demonstrate that our approach outperforms most prior prominent methods on both DeltaBench and existing benchmarks.
APA
Pan, K., Bu, W., Wu, Y., Shen, K., Wu, Y., Zhu, Y., Wang, Z., Li, Y., Zhao, H., Li, J. & Tang, S.. (2026). Benchmarking and Improving Fine-Grained Text-to-Image Alignment via Paired Reinforcement Learning. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:95696-95713 Available from https://proceedings.mlr.press/v306/pan26b.html.

Related Material