Trajectory-Aware Spiking DiTs Conversion via Membrane Potential Error-Feedback

Haoran Fang, Tianxing Man, Xingchen Li, Wanli Shi, Jinjie Fang, Bin Gu
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:29284-29297, 2026.

Abstract

Diffusion Transformers (DiTs) have achieved state-of-the-art generative performance, yet their iterative denoising process remains computationally expensive and energy-intensive. Spiking Neural Networks (SNNs) offer a promising neuromorphic alternative for energy efficiency; however, the non-differentiable nature of spiking neurons makes direct training difficult, positioning ANN-to-SNN conversion as a more practical, training-free solution. In this paper, we identify a critical challenge unique to converting DiTs: standard fixed-scale spiking neurons fail to accommodate the highly dynamic activation ranges inherent across denoising steps. This mismatch leads to cumulative errors that significantly degrade generation fidelity. To resolve this, we propose a novel conversion framework featuring Multi-Threshold (MT) neurons and a Membrane Potential Error-Feedback (MPEF) mechanism. MT neurons expand the expressive capacity of discrete spikes by employing a multi-level firing strategy. Concurrently, MPEF exploits the temporal correlation between successive denoising steps to recycle residual membrane potential, effectively compensating for information loss and mitigating distribution shifts without retraining. Extensive experiments on ImageNet demonstrate that our framework achieves competitive generative quality with superior energy efficiency, establishing a new performance benchmark for spiking Diffusion Transformers.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-fang26n, title = {Trajectory-Aware Spiking {D}i{T}s Conversion via Membrane Potential Error-Feedback}, author = {Fang, Haoran and Man, Tianxing and Li, Xingchen and Shi, Wanli and Fang, Jinjie and Gu, Bin}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {29284--29297}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/fang26n/fang26n.pdf}, url = {https://proceedings.mlr.press/v306/fang26n.html}, abstract = {Diffusion Transformers (DiTs) have achieved state-of-the-art generative performance, yet their iterative denoising process remains computationally expensive and energy-intensive. Spiking Neural Networks (SNNs) offer a promising neuromorphic alternative for energy efficiency; however, the non-differentiable nature of spiking neurons makes direct training difficult, positioning ANN-to-SNN conversion as a more practical, training-free solution. In this paper, we identify a critical challenge unique to converting DiTs: standard fixed-scale spiking neurons fail to accommodate the highly dynamic activation ranges inherent across denoising steps. This mismatch leads to cumulative errors that significantly degrade generation fidelity. To resolve this, we propose a novel conversion framework featuring Multi-Threshold (MT) neurons and a Membrane Potential Error-Feedback (MPEF) mechanism. MT neurons expand the expressive capacity of discrete spikes by employing a multi-level firing strategy. Concurrently, MPEF exploits the temporal correlation between successive denoising steps to recycle residual membrane potential, effectively compensating for information loss and mitigating distribution shifts without retraining. Extensive experiments on ImageNet demonstrate that our framework achieves competitive generative quality with superior energy efficiency, establishing a new performance benchmark for spiking Diffusion Transformers.} }
Endnote
%0 Conference Paper %T Trajectory-Aware Spiking DiTs Conversion via Membrane Potential Error-Feedback %A Haoran Fang %A Tianxing Man %A Xingchen Li %A Wanli Shi %A Jinjie Fang %A Bin Gu %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-fang26n %I PMLR %P 29284--29297 %U https://proceedings.mlr.press/v306/fang26n.html %V 306 %X Diffusion Transformers (DiTs) have achieved state-of-the-art generative performance, yet their iterative denoising process remains computationally expensive and energy-intensive. Spiking Neural Networks (SNNs) offer a promising neuromorphic alternative for energy efficiency; however, the non-differentiable nature of spiking neurons makes direct training difficult, positioning ANN-to-SNN conversion as a more practical, training-free solution. In this paper, we identify a critical challenge unique to converting DiTs: standard fixed-scale spiking neurons fail to accommodate the highly dynamic activation ranges inherent across denoising steps. This mismatch leads to cumulative errors that significantly degrade generation fidelity. To resolve this, we propose a novel conversion framework featuring Multi-Threshold (MT) neurons and a Membrane Potential Error-Feedback (MPEF) mechanism. MT neurons expand the expressive capacity of discrete spikes by employing a multi-level firing strategy. Concurrently, MPEF exploits the temporal correlation between successive denoising steps to recycle residual membrane potential, effectively compensating for information loss and mitigating distribution shifts without retraining. Extensive experiments on ImageNet demonstrate that our framework achieves competitive generative quality with superior energy efficiency, establishing a new performance benchmark for spiking Diffusion Transformers.
APA
Fang, H., Man, T., Li, X., Shi, W., Fang, J. & Gu, B.. (2026). Trajectory-Aware Spiking DiTs Conversion via Membrane Potential Error-Feedback. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:29284-29297 Available from https://proceedings.mlr.press/v306/fang26n.html.

Related Material