Concept-Centric Token Interpretation for Vector-Quantized Generative Models

Tianze Yang, Yucheng Shi, Mengnan Du, Xuansheng Wu, Qiaoyu Tan, Jin Sun, Ninghao Liu
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:71034-71050, 2025.

Abstract

Vector-Quantized Generative Models (VQGMs) have emerged as powerful tools for image generation. However, the key component of VQGMs—the codebook of discrete tokens—is still not well understood, e.g., which tokens are critical to generate an image of a certain concept? This paper introduces Concept-Oriented Token Explanation (CORTEX), a novel approach for interpreting VQGMs by identifying concept-specific token combinations. Our framework employs two methods: (1) a sample-level explanation method that analyzes token importance scores in individual images, and (2) a codebook-level explanation method that explores the entire codebook to find globally relevant tokens. Experimental results demonstrate CORTEX’s efficacy in providing clear explanations of token usage in the generative process, outperforming baselines across multiple pretrained VQGMs. Besides enhancing VQGMs transparency, CORTEX is useful in applications such as targeted image editing and shortcut feature detection. Our code is available at https://github.com/YangTianze009/CORTEX.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-yang25z, title = {Concept-Centric Token Interpretation for Vector-Quantized Generative Models}, author = {Yang, Tianze and Shi, Yucheng and Du, Mengnan and Wu, Xuansheng and Tan, Qiaoyu and Sun, Jin and Liu, Ninghao}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {71034--71050}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/yang25z/yang25z.pdf}, url = {https://proceedings.mlr.press/v267/yang25z.html}, abstract = {Vector-Quantized Generative Models (VQGMs) have emerged as powerful tools for image generation. However, the key component of VQGMs—the codebook of discrete tokens—is still not well understood, e.g., which tokens are critical to generate an image of a certain concept? This paper introduces Concept-Oriented Token Explanation (CORTEX), a novel approach for interpreting VQGMs by identifying concept-specific token combinations. Our framework employs two methods: (1) a sample-level explanation method that analyzes token importance scores in individual images, and (2) a codebook-level explanation method that explores the entire codebook to find globally relevant tokens. Experimental results demonstrate CORTEX’s efficacy in providing clear explanations of token usage in the generative process, outperforming baselines across multiple pretrained VQGMs. Besides enhancing VQGMs transparency, CORTEX is useful in applications such as targeted image editing and shortcut feature detection. Our code is available at https://github.com/YangTianze009/CORTEX.} }
Endnote
%0 Conference Paper %T Concept-Centric Token Interpretation for Vector-Quantized Generative Models %A Tianze Yang %A Yucheng Shi %A Mengnan Du %A Xuansheng Wu %A Qiaoyu Tan %A Jin Sun %A Ninghao Liu %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-yang25z %I PMLR %P 71034--71050 %U https://proceedings.mlr.press/v267/yang25z.html %V 267 %X Vector-Quantized Generative Models (VQGMs) have emerged as powerful tools for image generation. However, the key component of VQGMs—the codebook of discrete tokens—is still not well understood, e.g., which tokens are critical to generate an image of a certain concept? This paper introduces Concept-Oriented Token Explanation (CORTEX), a novel approach for interpreting VQGMs by identifying concept-specific token combinations. Our framework employs two methods: (1) a sample-level explanation method that analyzes token importance scores in individual images, and (2) a codebook-level explanation method that explores the entire codebook to find globally relevant tokens. Experimental results demonstrate CORTEX’s efficacy in providing clear explanations of token usage in the generative process, outperforming baselines across multiple pretrained VQGMs. Besides enhancing VQGMs transparency, CORTEX is useful in applications such as targeted image editing and shortcut feature detection. Our code is available at https://github.com/YangTianze009/CORTEX.
APA
Yang, T., Shi, Y., Du, M., Wu, X., Tan, Q., Sun, J. & Liu, N.. (2025). Concept-Centric Token Interpretation for Vector-Quantized Generative Models. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:71034-71050 Available from https://proceedings.mlr.press/v267/yang25z.html.

Related Material