Global Credit Assignment via Dynamical Criticality

Wentao Wang, Keren Gao, Guozhang Chen
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:126218-126251, 2026.

Abstract

Efficiently training recurrent neural networks on long sequences remains an open challenge. The standard global paradigm, backpropagation through time (BPTT), suffers from vanishing and exploding gradients and memory costs that scale linearly with sequence length. Conversely, biologically inspired local learning rules are memory-efficient but typically introduce severe bias. To bridge this gap, we introduce Criticality-driven Online Local Alignment (COLA). By leveraging the long-range spatiotemporal correlations inherent to the critical regime, COLA enables a strictly local learning rule to approximate global error propagation, thereby combining online efficiency with gradient descent precision. Theoretically, for a recurrent neural network with $H$ hidden units, COLA requires only an $O(H)$ auxiliary state and constant activation memory, completely independent of sequence length. Empirically, COLA is competitive with BPTT on standard benchmarks and demonstrates superior robustness on stability-sensitive tasks. Finally, we conduct a rigorous analysis of the approximation error to provide a theoretical foundation for reliable online learning. Code is available at https://github.com/Criticality-Cognitive-Computation-Lab/COLA

Cite this Paper


BibTeX
@InProceedings{pmlr-v306-wang26be, title = {Global Credit Assignment via Dynamical Criticality}, author = {Wang, Wentao and Gao, Keren and Chen, Guozhang}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {126218--126251}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/wang26be/wang26be.pdf}, url = {https://proceedings.mlr.press/v306/wang26be.html}, abstract = {Efficiently training recurrent neural networks on long sequences remains an open challenge. The standard global paradigm, backpropagation through time (BPTT), suffers from vanishing and exploding gradients and memory costs that scale linearly with sequence length. Conversely, biologically inspired local learning rules are memory-efficient but typically introduce severe bias. To bridge this gap, we introduce Criticality-driven Online Local Alignment (COLA). By leveraging the long-range spatiotemporal correlations inherent to the critical regime, COLA enables a strictly local learning rule to approximate global error propagation, thereby combining online efficiency with gradient descent precision. Theoretically, for a recurrent neural network with $H$ hidden units, COLA requires only an $O(H)$ auxiliary state and constant activation memory, completely independent of sequence length. Empirically, COLA is competitive with BPTT on standard benchmarks and demonstrates superior robustness on stability-sensitive tasks. Finally, we conduct a rigorous analysis of the approximation error to provide a theoretical foundation for reliable online learning. Code is available at https://github.com/Criticality-Cognitive-Computation-Lab/COLA} }
Endnote
%0 Conference Paper %T Global Credit Assignment via Dynamical Criticality %A Wentao Wang %A Keren Gao %A Guozhang Chen %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-wang26be %I PMLR %P 126218--126251 %U https://proceedings.mlr.press/v306/wang26be.html %V 306 %X Efficiently training recurrent neural networks on long sequences remains an open challenge. The standard global paradigm, backpropagation through time (BPTT), suffers from vanishing and exploding gradients and memory costs that scale linearly with sequence length. Conversely, biologically inspired local learning rules are memory-efficient but typically introduce severe bias. To bridge this gap, we introduce Criticality-driven Online Local Alignment (COLA). By leveraging the long-range spatiotemporal correlations inherent to the critical regime, COLA enables a strictly local learning rule to approximate global error propagation, thereby combining online efficiency with gradient descent precision. Theoretically, for a recurrent neural network with $H$ hidden units, COLA requires only an $O(H)$ auxiliary state and constant activation memory, completely independent of sequence length. Empirically, COLA is competitive with BPTT on standard benchmarks and demonstrates superior robustness on stability-sensitive tasks. Finally, we conduct a rigorous analysis of the approximation error to provide a theoretical foundation for reliable online learning. Code is available at https://github.com/Criticality-Cognitive-Computation-Lab/COLA
APA
Wang, W., Gao, K. & Chen, G.. (2026). Global Credit Assignment via Dynamical Criticality. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:126218-126251 Available from https://proceedings.mlr.press/v306/wang26be.html.

Related Material