CSD: Content-aware Speculative Decoding for Efficient Image Generation

Mingcheng Wang, Junbo Qiao, Yunchen Li, Lingfu Jiang, Wei Li, Jie Hu, Jiao Xie, Zhou Yu, Xinghao Chen, Guixu Zhang, Shaohui Lin
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:129486-129501, 2026.

Abstract

Speculative decoding (SD) has emerged as a key solution to accelerate the inference of autoregressive models. However, in the field of image generation, it faces the challenge of low acceptance rates, and directly relaxing its criteria leads to degradation in image quality. In this paper, we propose a novel content-aware speculative decoding algorithm, termed CSD, which integrates an entropy-based probability relaxation mechanism with an optimal resampling strategy to enhance the inference efficiency for autoregressive image generation. By leveraging the informational uncertainty inherent in different regions of an image, CSD dynamically adjusts the acceptance probability of candidate tokens, increasing the acceptance rate in low-detail areas to accelerate generation. Moreover, a distribution alignment filter is introduced to ensure the output distribution to be aligned with the target model, which significantly improves the generative quality. Experiments conducted on Lumina-mGPT and Janus-Pro demonstrate that the superiority of the proposed CSD. Our source code is available at https://github.com/aderfebr/CSD.

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-wang26gi, title = {{CSD}: Content-aware Speculative Decoding for Efficient Image Generation}, author = {Wang, Mingcheng and Qiao, Junbo and Li, Yunchen and Jiang, Lingfu and Li, Wei and Hu, Jie and Xie, Jiao and Yu, Zhou and Chen, Xinghao and Zhang, Guixu and Lin, Shaohui}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {129486--129501}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/wang26gi/wang26gi.pdf}, url = {https://proceedings.mlr.press/v306/wang26gi.html}, abstract = {Speculative decoding (SD) has emerged as a key solution to accelerate the inference of autoregressive models. However, in the field of image generation, it faces the challenge of low acceptance rates, and directly relaxing its criteria leads to degradation in image quality. In this paper, we propose a novel content-aware speculative decoding algorithm, termed CSD, which integrates an entropy-based probability relaxation mechanism with an optimal resampling strategy to enhance the inference efficiency for autoregressive image generation. By leveraging the informational uncertainty inherent in different regions of an image, CSD dynamically adjusts the acceptance probability of candidate tokens, increasing the acceptance rate in low-detail areas to accelerate generation. Moreover, a distribution alignment filter is introduced to ensure the output distribution to be aligned with the target model, which significantly improves the generative quality. Experiments conducted on Lumina-mGPT and Janus-Pro demonstrate that the superiority of the proposed CSD. Our source code is available at https://github.com/aderfebr/CSD.} }
Endnote
%0 Conference Paper %T CSD: Content-aware Speculative Decoding for Efficient Image Generation %A Mingcheng Wang %A Junbo Qiao %A Yunchen Li %A Lingfu Jiang %A Wei Li %A Jie Hu %A Jiao Xie %A Zhou Yu %A Xinghao Chen %A Guixu Zhang %A Shaohui Lin %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-wang26gi %I PMLR %P 129486--129501 %U https://proceedings.mlr.press/v306/wang26gi.html %V 306 %X Speculative decoding (SD) has emerged as a key solution to accelerate the inference of autoregressive models. However, in the field of image generation, it faces the challenge of low acceptance rates, and directly relaxing its criteria leads to degradation in image quality. In this paper, we propose a novel content-aware speculative decoding algorithm, termed CSD, which integrates an entropy-based probability relaxation mechanism with an optimal resampling strategy to enhance the inference efficiency for autoregressive image generation. By leveraging the informational uncertainty inherent in different regions of an image, CSD dynamically adjusts the acceptance probability of candidate tokens, increasing the acceptance rate in low-detail areas to accelerate generation. Moreover, a distribution alignment filter is introduced to ensure the output distribution to be aligned with the target model, which significantly improves the generative quality. Experiments conducted on Lumina-mGPT and Janus-Pro demonstrate that the superiority of the proposed CSD. Our source code is available at https://github.com/aderfebr/CSD.
APA
Wang, M., Qiao, J., Li, Y., Jiang, L., Li, W., Hu, J., Xie, J., Yu, Z., Chen, X., Zhang, G. & Lin, S.. (2026). CSD: Content-aware Speculative Decoding for Efficient Image Generation. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:129486-129501 Available from https://proceedings.mlr.press/v306/wang26gi.html.

Related Material