autoMALA: Locally adaptive Metropolis-adjusted Langevin algorithm

Miguel Biron-Lattes, Nikola Surjanovic, Saifuddin Syed, Trevor Campbell, Alexandre Bouchard-Cote
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, PMLR 238:4600-4608, 2024.

Abstract

Selecting the step size for the Metropolis-adjusted Langevin algorithm (MALA) is necessary in order to obtain satisfactory performance. However, finding an adequate step size for an arbitrary target distribution can be a difficult task and even the best step size can perform poorly in specific regions of the space when the target distribution is sufficiently complex. To resolve this issue we introduce autoMALA, a new Markov chain Monte Carlo algorithm based on MALA that automatically sets its step size at each iteration based on the local geometry of the target distribution. We prove that autoMALA has the correct invariant distribution, despite continual automatic adjustments of the step size. Our experiments demonstrate that autoMALA is competitive with related state-of-the-art MCMC methods, in terms of the number of log density evaluations per effective sample, and it outperforms state-of-the-art samplers on targets with varying geometries. Furthermore, we find that autoMALA tends to find step sizes comparable to optimally-tuned MALA when a fixed step size suffices for the whole domain.

Cite this Paper

BibTeX
@InProceedings{pmlr-v238-biron-lattes24a, title = {{autoMALA}: Locally adaptive {M}etropolis-adjusted {L}angevin algorithm}, author = {Biron-Lattes, Miguel and Surjanovic, Nikola and Syed, Saifuddin and Campbell, Trevor and Bouchard-Cote, Alexandre}, booktitle = {Proceedings of The 27th International Conference on Artificial Intelligence and Statistics}, pages = {4600--4608}, year = {2024}, editor = {Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen}, volume = {238}, series = {Proceedings of Machine Learning Research}, month = {02--04 May}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v238/biron-lattes24a/biron-lattes24a.pdf}, url = {https://proceedings.mlr.press/v238/biron-lattes24a.html}, abstract = {Selecting the step size for the Metropolis-adjusted Langevin algorithm (MALA) is necessary in order to obtain satisfactory performance. However, finding an adequate step size for an arbitrary target distribution can be a difficult task and even the best step size can perform poorly in specific regions of the space when the target distribution is sufficiently complex. To resolve this issue we introduce autoMALA, a new Markov chain Monte Carlo algorithm based on MALA that automatically sets its step size at each iteration based on the local geometry of the target distribution. We prove that autoMALA has the correct invariant distribution, despite continual automatic adjustments of the step size. Our experiments demonstrate that autoMALA is competitive with related state-of-the-art MCMC methods, in terms of the number of log density evaluations per effective sample, and it outperforms state-of-the-art samplers on targets with varying geometries. Furthermore, we find that autoMALA tends to find step sizes comparable to optimally-tuned MALA when a fixed step size suffices for the whole domain.} }
Endnote
%0 Conference Paper %T autoMALA: Locally adaptive Metropolis-adjusted Langevin algorithm %A Miguel Biron-Lattes %A Nikola Surjanovic %A Saifuddin Syed %A Trevor Campbell %A Alexandre Bouchard-Cote %B Proceedings of The 27th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2024 %E Sanjoy Dasgupta %E Stephan Mandt %E Yingzhen Li %F pmlr-v238-biron-lattes24a %I PMLR %P 4600--4608 %U https://proceedings.mlr.press/v238/biron-lattes24a.html %V 238 %X Selecting the step size for the Metropolis-adjusted Langevin algorithm (MALA) is necessary in order to obtain satisfactory performance. However, finding an adequate step size for an arbitrary target distribution can be a difficult task and even the best step size can perform poorly in specific regions of the space when the target distribution is sufficiently complex. To resolve this issue we introduce autoMALA, a new Markov chain Monte Carlo algorithm based on MALA that automatically sets its step size at each iteration based on the local geometry of the target distribution. We prove that autoMALA has the correct invariant distribution, despite continual automatic adjustments of the step size. Our experiments demonstrate that autoMALA is competitive with related state-of-the-art MCMC methods, in terms of the number of log density evaluations per effective sample, and it outperforms state-of-the-art samplers on targets with varying geometries. Furthermore, we find that autoMALA tends to find step sizes comparable to optimally-tuned MALA when a fixed step size suffices for the whole domain.
APA
Biron-Lattes, M., Surjanovic, N., Syed, S., Campbell, T. & Bouchard-Cote, A.. (2024). autoMALA: Locally adaptive Metropolis-adjusted Langevin algorithm. Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 238:4600-4608 Available from https://proceedings.mlr.press/v238/biron-lattes24a.html.

Related Material