Proximal Sampler with Adaptive Step Size

Bo Yuan, Jiaojiao Fan, Jiaming Liang, Yongxin Chen
Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, PMLR 258:1387-1395, 2025.

Abstract

We consider the problem of sampling from a target unnormalized distribution $\exp(-f(x))$ defined on $\mathbb{R}^d$ where $f(x)$ is smooth, but the smoothness parameter is unknown. As a key design parameter of Markov chain Monte Carlo (MCMC) algorithms, the step size is crucial for the convergence guarantee. Existing non-asymptotic analysis on MCMC with fixed step sizes indicates that the step size heavily relies on global smoothness. However, this choice does not utilize the local information and fails when the smoothness coefficient is hard to estimate. A tuning-free algorithm that can adaptively update stepsize is highly desirable. In this work, we propose an \textbf{adaptive} proximal sampler that can utilize the local geometry to adjust step sizes and is guaranteed to converge to the target distribution. Experiments demonstrate the comparable or superior performance of our algorithm over various baselines.

Cite this Paper

BibTeX
@InProceedings{pmlr-v258-yuan25a, title = {Proximal Sampler with Adaptive Step Size}, author = {Yuan, Bo and Fan, Jiaojiao and Liang, Jiaming and Chen, Yongxin}, booktitle = {Proceedings of The 28th International Conference on Artificial Intelligence and Statistics}, pages = {1387--1395}, year = {2025}, editor = {Li, Yingzhen and Mandt, Stephan and Agrawal, Shipra and Khan, Emtiyaz}, volume = {258}, series = {Proceedings of Machine Learning Research}, month = {03--05 May}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v258/main/assets/yuan25a/yuan25a.pdf}, url = {https://proceedings.mlr.press/v258/yuan25a.html}, abstract = {We consider the problem of sampling from a target unnormalized distribution $\exp(-f(x))$ defined on $\mathbb{R}^d$ where $f(x)$ is smooth, but the smoothness parameter is unknown. As a key design parameter of Markov chain Monte Carlo (MCMC) algorithms, the step size is crucial for the convergence guarantee. Existing non-asymptotic analysis on MCMC with fixed step sizes indicates that the step size heavily relies on global smoothness. However, this choice does not utilize the local information and fails when the smoothness coefficient is hard to estimate. A tuning-free algorithm that can adaptively update stepsize is highly desirable. In this work, we propose an \textbf{adaptive} proximal sampler that can utilize the local geometry to adjust step sizes and is guaranteed to converge to the target distribution. Experiments demonstrate the comparable or superior performance of our algorithm over various baselines.} }
Endnote
%0 Conference Paper %T Proximal Sampler with Adaptive Step Size %A Bo Yuan %A Jiaojiao Fan %A Jiaming Liang %A Yongxin Chen %B Proceedings of The 28th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2025 %E Yingzhen Li %E Stephan Mandt %E Shipra Agrawal %E Emtiyaz Khan %F pmlr-v258-yuan25a %I PMLR %P 1387--1395 %U https://proceedings.mlr.press/v258/yuan25a.html %V 258 %X We consider the problem of sampling from a target unnormalized distribution $\exp(-f(x))$ defined on $\mathbb{R}^d$ where $f(x)$ is smooth, but the smoothness parameter is unknown. As a key design parameter of Markov chain Monte Carlo (MCMC) algorithms, the step size is crucial for the convergence guarantee. Existing non-asymptotic analysis on MCMC with fixed step sizes indicates that the step size heavily relies on global smoothness. However, this choice does not utilize the local information and fails when the smoothness coefficient is hard to estimate. A tuning-free algorithm that can adaptively update stepsize is highly desirable. In this work, we propose an \textbf{adaptive} proximal sampler that can utilize the local geometry to adjust step sizes and is guaranteed to converge to the target distribution. Experiments demonstrate the comparable or superior performance of our algorithm over various baselines.
APA
Yuan, B., Fan, J., Liang, J. & Chen, Y.. (2025). Proximal Sampler with Adaptive Step Size. Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 258:1387-1395 Available from https://proceedings.mlr.press/v258/yuan25a.html.

Related Material