End-to-End Autoregressive Image Generation with 1D Semantic Tokenizer

Wenda Chu, Bingliang Zhang, Jiaqi Han, Yizhuo Li, Linjie Yang, Yisong Yue, Qiushan Guo
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:20707-20721, 2026.

Abstract

Autoregressive image modeling relies on visual tokenizers to compress images into compact latent representations. We design an end-to-end training pipeline that jointly optimizes reconstruction and generation, enabling direct supervision from generation results to the tokenizer. This contrasts with prior two-stage approaches that train tokenizers and generative models separately. We further investigate leveraging vision foundation models to improve 1D tokenizers for autoregressive modeling. Our autoregressive generative model achieves strong empirical results, including a state-of-the-art FID score of 1.48 without guidance on ImageNet 256$\times$256 generation.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-chu26b, title = {End-to-End Autoregressive Image Generation with 1{D} Semantic Tokenizer}, author = {Chu, Wenda and Zhang, Bingliang and Han, Jiaqi and Li, Yizhuo and Yang, Linjie and Yue, Yisong and Guo, Qiushan}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {20707--20721}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/chu26b/chu26b.pdf}, url = {https://proceedings.mlr.press/v306/chu26b.html}, abstract = {Autoregressive image modeling relies on visual tokenizers to compress images into compact latent representations. We design an end-to-end training pipeline that jointly optimizes reconstruction and generation, enabling direct supervision from generation results to the tokenizer. This contrasts with prior two-stage approaches that train tokenizers and generative models separately. We further investigate leveraging vision foundation models to improve 1D tokenizers for autoregressive modeling. Our autoregressive generative model achieves strong empirical results, including a state-of-the-art FID score of 1.48 without guidance on ImageNet 256$\times$256 generation.} }
Endnote
%0 Conference Paper %T End-to-End Autoregressive Image Generation with 1D Semantic Tokenizer %A Wenda Chu %A Bingliang Zhang %A Jiaqi Han %A Yizhuo Li %A Linjie Yang %A Yisong Yue %A Qiushan Guo %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-chu26b %I PMLR %P 20707--20721 %U https://proceedings.mlr.press/v306/chu26b.html %V 306 %X Autoregressive image modeling relies on visual tokenizers to compress images into compact latent representations. We design an end-to-end training pipeline that jointly optimizes reconstruction and generation, enabling direct supervision from generation results to the tokenizer. This contrasts with prior two-stage approaches that train tokenizers and generative models separately. We further investigate leveraging vision foundation models to improve 1D tokenizers for autoregressive modeling. Our autoregressive generative model achieves strong empirical results, including a state-of-the-art FID score of 1.48 without guidance on ImageNet 256$\times$256 generation.
APA
Chu, W., Zhang, B., Han, J., Li, Y., Yang, L., Yue, Y. & Guo, Q.. (2026). End-to-End Autoregressive Image Generation with 1D Semantic Tokenizer. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:20707-20721 Available from https://proceedings.mlr.press/v306/chu26b.html.

Related Material