Boost the Identity-Preserving Embedding for Consistent Visual Generation

Zixun Xia, Kai Wang, Shuyu Guo, Boqian Li, Jian Yang, Yaxing Wang
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:138283-138307, 2026.

Abstract

Text-to-image models have advanced high-fidelity content generation, but their inability to maintain subject consistency hampers realistic applications. Existing training-based methods rely on heavy computation and large datasets; while training-free approaches demand excessive memory or complex auxiliary modules. In this paper, we first reveal a key property overlooked in prior works that the identity-relevant signals, termed Identity-Preserving Embeddings (IPemb), are implicitly encoded in textual embeddings of frame prompts. To address the consistent T2I generation with the IPemb embedding, we propose Boost Identity-Preserving Embedding (BIPE), a training-free yet plug-and-play framework that explicitly extracts and enhances the IPemb. Its core innovations are two complementary techniques: First, Adaptive Singular-Value Rescaling (adaSVR) applies singular-value decomposition to the joint embedding matrix of all frame prompts, amplifying identity-centric components while suppressing frame-specific noise. Second, Union Key (UniK) further reinforces consistency by aligning the T2I backbone’s image-text attention across the entire generation sequence. Experiments on the ConsiStory+ benchmark demonstrate BIPE outperforms existing methods in both qualitative and quantitative metrics. To address the gap in evaluating a broader range of scenarios with diversified prompt templates, we introduce a DiverStory benchmark to further confirm our scalability.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-xia26i, title = {Boost the Identity-Preserving Embedding for Consistent Visual Generation}, author = {Xia, Zixun and Wang, Kai and Guo, Shuyu and Li, Boqian and Yang, Jian and Wang, Yaxing}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {138283--138307}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/xia26i/xia26i.pdf}, url = {https://proceedings.mlr.press/v306/xia26i.html}, abstract = {Text-to-image models have advanced high-fidelity content generation, but their inability to maintain subject consistency hampers realistic applications. Existing training-based methods rely on heavy computation and large datasets; while training-free approaches demand excessive memory or complex auxiliary modules. In this paper, we first reveal a key property overlooked in prior works that the identity-relevant signals, termed Identity-Preserving Embeddings (IPemb), are implicitly encoded in textual embeddings of frame prompts. To address the consistent T2I generation with the IPemb embedding, we propose Boost Identity-Preserving Embedding (BIPE), a training-free yet plug-and-play framework that explicitly extracts and enhances the IPemb. Its core innovations are two complementary techniques: First, Adaptive Singular-Value Rescaling (adaSVR) applies singular-value decomposition to the joint embedding matrix of all frame prompts, amplifying identity-centric components while suppressing frame-specific noise. Second, Union Key (UniK) further reinforces consistency by aligning the T2I backbone’s image-text attention across the entire generation sequence. Experiments on the ConsiStory+ benchmark demonstrate BIPE outperforms existing methods in both qualitative and quantitative metrics. To address the gap in evaluating a broader range of scenarios with diversified prompt templates, we introduce a DiverStory benchmark to further confirm our scalability.} }
Endnote
%0 Conference Paper %T Boost the Identity-Preserving Embedding for Consistent Visual Generation %A Zixun Xia %A Kai Wang %A Shuyu Guo %A Boqian Li %A Jian Yang %A Yaxing Wang %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-xia26i %I PMLR %P 138283--138307 %U https://proceedings.mlr.press/v306/xia26i.html %V 306 %X Text-to-image models have advanced high-fidelity content generation, but their inability to maintain subject consistency hampers realistic applications. Existing training-based methods rely on heavy computation and large datasets; while training-free approaches demand excessive memory or complex auxiliary modules. In this paper, we first reveal a key property overlooked in prior works that the identity-relevant signals, termed Identity-Preserving Embeddings (IPemb), are implicitly encoded in textual embeddings of frame prompts. To address the consistent T2I generation with the IPemb embedding, we propose Boost Identity-Preserving Embedding (BIPE), a training-free yet plug-and-play framework that explicitly extracts and enhances the IPemb. Its core innovations are two complementary techniques: First, Adaptive Singular-Value Rescaling (adaSVR) applies singular-value decomposition to the joint embedding matrix of all frame prompts, amplifying identity-centric components while suppressing frame-specific noise. Second, Union Key (UniK) further reinforces consistency by aligning the T2I backbone’s image-text attention across the entire generation sequence. Experiments on the ConsiStory+ benchmark demonstrate BIPE outperforms existing methods in both qualitative and quantitative metrics. To address the gap in evaluating a broader range of scenarios with diversified prompt templates, we introduce a DiverStory benchmark to further confirm our scalability.
APA
Xia, Z., Wang, K., Guo, S., Li, B., Yang, J. & Wang, Y.. (2026). Boost the Identity-Preserving Embedding for Consistent Visual Generation. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:138283-138307 Available from https://proceedings.mlr.press/v306/xia26i.html.

Related Material