FreeText: Training-Free Text Rendering via Attention Localization and Spectral Glyph Injection

Ruiqiang Zhang, Hengyi Wang, Chang Liu, Guanjie Wang, Zehua Ma, Weiming Zhang
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:159055-159070, 2026.

Abstract

Large-scale text-to-image (T2I) diffusion models excel at open-domain synthesis but still struggle with precise text rendering, especially for multi-line layouts, dense typography, and long-tailed scripts such as Chinese. Prior solutions typically necessitate costly retraining or impose rigid external layout constraints, often compromising aesthetic quality and flexibility. We propose FreeText, a training-free, plug-and-play framework that improves text rendering by leveraging intrinsic mechanisms of Diffusion Transformer (DiT) models. FreeText decomposes the problem into where to write and what to write. For the former, we localize writing regions by extracting token-wise spatial attribution from image-to-text attention, using sink-like tokens as stable spatial anchors and topology-aware refinement to produce high-confidence masks. For the latter, we introduce Spectral-Modulated Glyph Injection (SGMI), which injects a noise-aligned glyph prior with frequency-domain band-pass modulation to strengthen glyph structure and mitigate semantic leakage (rendering the concept instead of the word). Extensive experiments on Qwen-Image, FLUX.1-dev, and SD3 variants across longText-Benchmark, CVTG, and our CLT-Bench show consistent gains in text readability while maintaining semantic alignment and aesthetic quality, with modest inference overhead.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-zhang26ge, title = {{F}ree{T}ext: Training-Free Text Rendering via Attention Localization and Spectral Glyph Injection}, author = {Zhang, Ruiqiang and Wang, Hengyi and Liu, Chang and Wang, Guanjie and Ma, Zehua and Zhang, Weiming}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {159055--159070}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/zhang26ge/zhang26ge.pdf}, url = {https://proceedings.mlr.press/v306/zhang26ge.html}, abstract = {Large-scale text-to-image (T2I) diffusion models excel at open-domain synthesis but still struggle with precise text rendering, especially for multi-line layouts, dense typography, and long-tailed scripts such as Chinese. Prior solutions typically necessitate costly retraining or impose rigid external layout constraints, often compromising aesthetic quality and flexibility. We propose FreeText, a training-free, plug-and-play framework that improves text rendering by leveraging intrinsic mechanisms of Diffusion Transformer (DiT) models. FreeText decomposes the problem into where to write and what to write. For the former, we localize writing regions by extracting token-wise spatial attribution from image-to-text attention, using sink-like tokens as stable spatial anchors and topology-aware refinement to produce high-confidence masks. For the latter, we introduce Spectral-Modulated Glyph Injection (SGMI), which injects a noise-aligned glyph prior with frequency-domain band-pass modulation to strengthen glyph structure and mitigate semantic leakage (rendering the concept instead of the word). Extensive experiments on Qwen-Image, FLUX.1-dev, and SD3 variants across longText-Benchmark, CVTG, and our CLT-Bench show consistent gains in text readability while maintaining semantic alignment and aesthetic quality, with modest inference overhead.} }
Endnote
%0 Conference Paper %T FreeText: Training-Free Text Rendering via Attention Localization and Spectral Glyph Injection %A Ruiqiang Zhang %A Hengyi Wang %A Chang Liu %A Guanjie Wang %A Zehua Ma %A Weiming Zhang %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-zhang26ge %I PMLR %P 159055--159070 %U https://proceedings.mlr.press/v306/zhang26ge.html %V 306 %X Large-scale text-to-image (T2I) diffusion models excel at open-domain synthesis but still struggle with precise text rendering, especially for multi-line layouts, dense typography, and long-tailed scripts such as Chinese. Prior solutions typically necessitate costly retraining or impose rigid external layout constraints, often compromising aesthetic quality and flexibility. We propose FreeText, a training-free, plug-and-play framework that improves text rendering by leveraging intrinsic mechanisms of Diffusion Transformer (DiT) models. FreeText decomposes the problem into where to write and what to write. For the former, we localize writing regions by extracting token-wise spatial attribution from image-to-text attention, using sink-like tokens as stable spatial anchors and topology-aware refinement to produce high-confidence masks. For the latter, we introduce Spectral-Modulated Glyph Injection (SGMI), which injects a noise-aligned glyph prior with frequency-domain band-pass modulation to strengthen glyph structure and mitigate semantic leakage (rendering the concept instead of the word). Extensive experiments on Qwen-Image, FLUX.1-dev, and SD3 variants across longText-Benchmark, CVTG, and our CLT-Bench show consistent gains in text readability while maintaining semantic alignment and aesthetic quality, with modest inference overhead.
APA
Zhang, R., Wang, H., Liu, C., Wang, G., Ma, Z. & Zhang, W.. (2026). FreeText: Training-Free Text Rendering via Attention Localization and Spectral Glyph Injection. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:159055-159070 Available from https://proceedings.mlr.press/v306/zhang26ge.html.

Related Material