Adaptive Hamiltonian and Riemann Manifold Monte Carlo

Ziyu Wang, Shakir Mohamed, Nando Freitas
Proceedings of the 30th International Conference on Machine Learning, PMLR 28(3):1462-1470, 2013.

Abstract

In this paper we address the widely-experienced difficulty in tuning Hamiltonian-based Monte Carlo samplers. We develop an algorithm that allows for the adaptation of Hamiltonian and Riemann manifold Hamiltonian Monte Carlo samplers using Bayesian optimization that allows for infinite adaptation of the parameters of these samplers. We show that the resulting sampling algorithms are ergodic, and demonstrate on several models and data sets that the use of our adaptive algorithms makes it is easy to obtain more efficient samplers, in some precluding the need for more complex models. Hamiltonian-based Monte Carlo samplers are widely known to be an excellent choice of MCMC method, and we aim with this paper to remove a key obstacle towards the more widespread use of these samplers in practice.

Cite this Paper

BibTeX
@InProceedings{pmlr-v28-wang13e, title = {Adaptive Hamiltonian and Riemann Manifold Monte Carlo}, author = {Wang, Ziyu and Mohamed, Shakir and Freitas, Nando}, booktitle = {Proceedings of the 30th International Conference on Machine Learning}, pages = {1462--1470}, year = {2013}, editor = {Dasgupta, Sanjoy and McAllester, David}, volume = {28}, number = {3}, series = {Proceedings of Machine Learning Research}, address = {Atlanta, Georgia, USA}, month = {17--19 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v28/wang13e.pdf}, url = {https://proceedings.mlr.press/v28/wang13e.html}, abstract = {In this paper we address the widely-experienced difficulty in tuning Hamiltonian-based Monte Carlo samplers. We develop an algorithm that allows for the adaptation of Hamiltonian and Riemann manifold Hamiltonian Monte Carlo samplers using Bayesian optimization that allows for infinite adaptation of the parameters of these samplers. We show that the resulting sampling algorithms are ergodic, and demonstrate on several models and data sets that the use of our adaptive algorithms makes it is easy to obtain more efficient samplers, in some precluding the need for more complex models. Hamiltonian-based Monte Carlo samplers are widely known to be an excellent choice of MCMC method, and we aim with this paper to remove a key obstacle towards the more widespread use of these samplers in practice.} }
Endnote
%0 Conference Paper %T Adaptive Hamiltonian and Riemann Manifold Monte Carlo %A Ziyu Wang %A Shakir Mohamed %A Nando Freitas %B Proceedings of the 30th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2013 %E Sanjoy Dasgupta %E David McAllester %F pmlr-v28-wang13e %I PMLR %P 1462--1470 %U https://proceedings.mlr.press/v28/wang13e.html %V 28 %N 3 %X In this paper we address the widely-experienced difficulty in tuning Hamiltonian-based Monte Carlo samplers. We develop an algorithm that allows for the adaptation of Hamiltonian and Riemann manifold Hamiltonian Monte Carlo samplers using Bayesian optimization that allows for infinite adaptation of the parameters of these samplers. We show that the resulting sampling algorithms are ergodic, and demonstrate on several models and data sets that the use of our adaptive algorithms makes it is easy to obtain more efficient samplers, in some precluding the need for more complex models. Hamiltonian-based Monte Carlo samplers are widely known to be an excellent choice of MCMC method, and we aim with this paper to remove a key obstacle towards the more widespread use of these samplers in practice.
RIS
TY - CPAPER TI - Adaptive Hamiltonian and Riemann Manifold Monte Carlo AU - Ziyu Wang AU - Shakir Mohamed AU - Nando Freitas BT - Proceedings of the 30th International Conference on Machine Learning DA - 2013/05/26 ED - Sanjoy Dasgupta ED - David McAllester ID - pmlr-v28-wang13e PB - PMLR DP - Proceedings of Machine Learning Research VL - 28 IS - 3 SP - 1462 EP - 1470 L1 - http://proceedings.mlr.press/v28/wang13e.pdf UR - https://proceedings.mlr.press/v28/wang13e.html AB - In this paper we address the widely-experienced difficulty in tuning Hamiltonian-based Monte Carlo samplers. We develop an algorithm that allows for the adaptation of Hamiltonian and Riemann manifold Hamiltonian Monte Carlo samplers using Bayesian optimization that allows for infinite adaptation of the parameters of these samplers. We show that the resulting sampling algorithms are ergodic, and demonstrate on several models and data sets that the use of our adaptive algorithms makes it is easy to obtain more efficient samplers, in some precluding the need for more complex models. Hamiltonian-based Monte Carlo samplers are widely known to be an excellent choice of MCMC method, and we aim with this paper to remove a key obstacle towards the more widespread use of these samplers in practice. ER -
APA
Wang, Z., Mohamed, S. & Freitas, N.. (2013). Adaptive Hamiltonian and Riemann Manifold Monte Carlo. Proceedings of the 30th International Conference on Machine Learning, in Proceedings of Machine Learning Research 28(3):1462-1470 Available from https://proceedings.mlr.press/v28/wang13e.html.

Related Material