In-memory Training on Analog Devices with Limited Conductance States via Multi-tile Residual Learning

Jindan Li, Zhaoxian Wu, Gaowen Liu, Tayfun Gokmen, Tianyi Chen
Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, PMLR 300:3664-3672, 2026.

Abstract

Analog in-memory computing (AIMC) accelerators enable efficient deep neural network computation directly within memory using resistive crossbar arrays, where model parameters are represented by the conductance states of memristive devices. However, effective in-memory training typically requires at least 8-bit conductance states to match digital baselines. Realizing such fine-grained states is costly and often requires complex noise mitigation techniques that increase circuit complexity and energy consumption. In practice, many promising memristive devices such as ReRAM offer only about 4-bit resolution due to fabrication constraints, and this limited update precision substantially degrades training accuracy. To enable on-chip training with these limited-state devices, this paper proposes a \emph{multi-tile residual learning} framework that sequentially learns on multiple crossbar tiles to compensate the residual errors from low-precision weight updates. Our theoretical analysis shows that the optimality gap shrinks with the number of tiles and achieves a linear convergence rate. Experiments on standard image classification benchmarks demonstrate that our method consistently outperforms state-of-the-art in-situ analog training strategies under limited-state settings, while incurring only moderate hardware overhead as confirmed by our cost analysis.

Cite this Paper

BibTeX
@InProceedings{pmlr-v300-li26i, title = { In-memory Training on Analog Devices with Limited Conductance States via Multi-tile Residual Learning }, author = {Li, Jindan and Wu, Zhaoxian and Liu, Gaowen and Gokmen, Tayfun and Chen, Tianyi}, booktitle = {Proceedings of The 29th International Conference on Artificial Intelligence and Statistics}, pages = {3664--3672}, year = {2026}, editor = {Khan, Emtiyaz and Li, Yingzhen and Solin, Arno and Ramdas, Aaditya}, volume = {300}, series = {Proceedings of Machine Learning Research}, month = {02--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v300/main/assets/li26i/li26i.pdf}, url = {https://proceedings.mlr.press/v300/li26i.html}, abstract = { Analog in-memory computing (AIMC) accelerators enable efficient deep neural network computation directly within memory using resistive crossbar arrays, where model parameters are represented by the conductance states of memristive devices. However, effective in-memory training typically requires at least 8-bit conductance states to match digital baselines. Realizing such fine-grained states is costly and often requires complex noise mitigation techniques that increase circuit complexity and energy consumption. In practice, many promising memristive devices such as ReRAM offer only about 4-bit resolution due to fabrication constraints, and this limited update precision substantially degrades training accuracy. To enable on-chip training with these limited-state devices, this paper proposes a \emph{multi-tile residual learning} framework that sequentially learns on multiple crossbar tiles to compensate the residual errors from low-precision weight updates. Our theoretical analysis shows that the optimality gap shrinks with the number of tiles and achieves a linear convergence rate. Experiments on standard image classification benchmarks demonstrate that our method consistently outperforms state-of-the-art in-situ analog training strategies under limited-state settings, while incurring only moderate hardware overhead as confirmed by our cost analysis. } }
Endnote
%0 Conference Paper %T In-memory Training on Analog Devices with Limited Conductance States via Multi-tile Residual Learning %A Jindan Li %A Zhaoxian Wu %A Gaowen Liu %A Tayfun Gokmen %A Tianyi Chen %B Proceedings of The 29th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2026 %E Emtiyaz Khan %E Yingzhen Li %E Arno Solin %E Aaditya Ramdas %F pmlr-v300-li26i %I PMLR %P 3664--3672 %U https://proceedings.mlr.press/v300/li26i.html %V 300 %X Analog in-memory computing (AIMC) accelerators enable efficient deep neural network computation directly within memory using resistive crossbar arrays, where model parameters are represented by the conductance states of memristive devices. However, effective in-memory training typically requires at least 8-bit conductance states to match digital baselines. Realizing such fine-grained states is costly and often requires complex noise mitigation techniques that increase circuit complexity and energy consumption. In practice, many promising memristive devices such as ReRAM offer only about 4-bit resolution due to fabrication constraints, and this limited update precision substantially degrades training accuracy. To enable on-chip training with these limited-state devices, this paper proposes a \emph{multi-tile residual learning} framework that sequentially learns on multiple crossbar tiles to compensate the residual errors from low-precision weight updates. Our theoretical analysis shows that the optimality gap shrinks with the number of tiles and achieves a linear convergence rate. Experiments on standard image classification benchmarks demonstrate that our method consistently outperforms state-of-the-art in-situ analog training strategies under limited-state settings, while incurring only moderate hardware overhead as confirmed by our cost analysis.
APA
Li, J., Wu, Z., Liu, G., Gokmen, T. & Chen, T.. (2026). In-memory Training on Analog Devices with Limited Conductance States via Multi-tile Residual Learning . Proceedings of The 29th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 300:3664-3672 Available from https://proceedings.mlr.press/v300/li26i.html.

Related Material