Convergence Analysis of the Lion Optimizer in Centralized and Distributed Settings

Wei Jiang, Mao Xu, Wenhao Yang, Yibo Wang, Zechao Li, Lijun Zhang
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:53308-53333, 2026.

Abstract

In this paper, we provide a comprehensive convergence analysis for the Lion optimizer. First, we establish that the original Lion achieves a convergence rate of $\mathcal{O}(d^{1/2}T^{-1/4})$, where $d$ denotes the problem dimension and $T$ is the iteration number. To improve this rate, we propose a variance reduction variant of Lion, which attains an enhanced rate of $\mathcal{O}(d^{1/2}T^{-1/3})$ with the average smoothness assumption. Then, we extend our analysis to distributed settings. We demonstrate that the distributed Lion optimizer and its variance reduction counterpart achieve linear speedup with respect to the number of nodes $n$, yielding convergence rates of $\mathcal{O}(d^{1/2}(nT)^{-1/4})$ and $\mathcal{O}(d^{1/2}(nT)^{-1/3})$, respectively. Additionally, we investigate a communication-efficient distributed Lion variant that utilizes sign compression for bidirectional communication. By employing unbiased sign operations, this variant achieves a convergence rate of $\mathcal{O} \left( \max { \frac{d^{1/4}}{T^{1/4}}, \frac{d^{1/10}}{n^{1/5}T^{1/5}} } \right)$, and its variance-reduced counterpart can further improve the rate to $\mathcal{O}\left( \frac{d^{1/4}}{T^{1/4}} \right)$. Finally, we conduct numerical experiments to validate the effectiveness of the proposed methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-jiang26ax, title = {Convergence Analysis of the Lion Optimizer in Centralized and Distributed Settings}, author = {Jiang, Wei and Xu, Mao and Yang, Wenhao and Wang, Yibo and Li, Zechao and Zhang, Lijun}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {53308--53333}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/jiang26ax/jiang26ax.pdf}, url = {https://proceedings.mlr.press/v306/jiang26ax.html}, abstract = {In this paper, we provide a comprehensive convergence analysis for the Lion optimizer. First, we establish that the original Lion achieves a convergence rate of $\mathcal{O}(d^{1/2}T^{-1/4})$, where $d$ denotes the problem dimension and $T$ is the iteration number. To improve this rate, we propose a variance reduction variant of Lion, which attains an enhanced rate of $\mathcal{O}(d^{1/2}T^{-1/3})$ with the average smoothness assumption. Then, we extend our analysis to distributed settings. We demonstrate that the distributed Lion optimizer and its variance reduction counterpart achieve linear speedup with respect to the number of nodes $n$, yielding convergence rates of $\mathcal{O}(d^{1/2}(nT)^{-1/4})$ and $\mathcal{O}(d^{1/2}(nT)^{-1/3})$, respectively. Additionally, we investigate a communication-efficient distributed Lion variant that utilizes sign compression for bidirectional communication. By employing unbiased sign operations, this variant achieves a convergence rate of $\mathcal{O} \left( \max { \frac{d^{1/4}}{T^{1/4}}, \frac{d^{1/10}}{n^{1/5}T^{1/5}} } \right)$, and its variance-reduced counterpart can further improve the rate to $\mathcal{O}\left( \frac{d^{1/4}}{T^{1/4}} \right)$. Finally, we conduct numerical experiments to validate the effectiveness of the proposed methods.} }
Endnote
%0 Conference Paper %T Convergence Analysis of the Lion Optimizer in Centralized and Distributed Settings %A Wei Jiang %A Mao Xu %A Wenhao Yang %A Yibo Wang %A Zechao Li %A Lijun Zhang %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-jiang26ax %I PMLR %P 53308--53333 %U https://proceedings.mlr.press/v306/jiang26ax.html %V 306 %X In this paper, we provide a comprehensive convergence analysis for the Lion optimizer. First, we establish that the original Lion achieves a convergence rate of $\mathcal{O}(d^{1/2}T^{-1/4})$, where $d$ denotes the problem dimension and $T$ is the iteration number. To improve this rate, we propose a variance reduction variant of Lion, which attains an enhanced rate of $\mathcal{O}(d^{1/2}T^{-1/3})$ with the average smoothness assumption. Then, we extend our analysis to distributed settings. We demonstrate that the distributed Lion optimizer and its variance reduction counterpart achieve linear speedup with respect to the number of nodes $n$, yielding convergence rates of $\mathcal{O}(d^{1/2}(nT)^{-1/4})$ and $\mathcal{O}(d^{1/2}(nT)^{-1/3})$, respectively. Additionally, we investigate a communication-efficient distributed Lion variant that utilizes sign compression for bidirectional communication. By employing unbiased sign operations, this variant achieves a convergence rate of $\mathcal{O} \left( \max { \frac{d^{1/4}}{T^{1/4}}, \frac{d^{1/10}}{n^{1/5}T^{1/5}} } \right)$, and its variance-reduced counterpart can further improve the rate to $\mathcal{O}\left( \frac{d^{1/4}}{T^{1/4}} \right)$. Finally, we conduct numerical experiments to validate the effectiveness of the proposed methods.
APA
Jiang, W., Xu, M., Yang, W., Wang, Y., Li, Z. & Zhang, L.. (2026). Convergence Analysis of the Lion Optimizer in Centralized and Distributed Settings. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:53308-53333 Available from https://proceedings.mlr.press/v306/jiang26ax.html.

Related Material