Adaptive Metropolis with Online Relabeling

Remi Bardenet, Olivier Cappe, Gersende Fort, Balazs Kegl
Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:91-99, 2012.

Abstract

We propose a novel adaptive MCMC algorithm named AMOR (Adaptive Metropolis with Online Relabeling) for efficiently simulating from permutation-invariant targets occurring in, for example, Bayesian analysis of mixture models. An important feature of the algorithm is to tie the adaptation of the proposal distribution to the choice of a particular restriction of the target to a domain where label switching cannot occur. The algorithm relies on a stochastic approximation procedure for which we exhibit a Lyapunov function that formally defines the criterion used for selecting the relabeling rule. This criterion reveals an interesting connection with the problem of optimal quantifier design in vector quantization which was only implicit in previous works on the label switching problem. In benchmark examples, the algorithm turns out to be fast converging and efficient at selecting meaningful non-trivial relabeling rules to allow accurate parameter inference.

Cite this Paper

BibTeX
@InProceedings{pmlr-v22-bardenet12, title = {Adaptive Metropolis with Online Relabeling}, author = {Bardenet, Remi and Cappe, Olivier and Fort, Gersende and Kegl, Balazs}, booktitle = {Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics}, pages = {91--99}, year = {2012}, editor = {Lawrence, Neil D. and Girolami, Mark}, volume = {22}, series = {Proceedings of Machine Learning Research}, address = {La Palma, Canary Islands}, month = {21--23 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v22/bardenet12/bardenet12.pdf}, url = {https://proceedings.mlr.press/v22/bardenet12.html}, abstract = {We propose a novel adaptive MCMC algorithm named AMOR (Adaptive Metropolis with Online Relabeling) for efficiently simulating from permutation-invariant targets occurring in, for example, Bayesian analysis of mixture models. An important feature of the algorithm is to tie the adaptation of the proposal distribution to the choice of a particular restriction of the target to a domain where label switching cannot occur. The algorithm relies on a stochastic approximation procedure for which we exhibit a Lyapunov function that formally defines the criterion used for selecting the relabeling rule. This criterion reveals an interesting connection with the problem of optimal quantifier design in vector quantization which was only implicit in previous works on the label switching problem. In benchmark examples, the algorithm turns out to be fast converging and efficient at selecting meaningful non-trivial relabeling rules to allow accurate parameter inference.} }
Endnote
%0 Conference Paper %T Adaptive Metropolis with Online Relabeling %A Remi Bardenet %A Olivier Cappe %A Gersende Fort %A Balazs Kegl %B Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2012 %E Neil D. Lawrence %E Mark Girolami %F pmlr-v22-bardenet12 %I PMLR %P 91--99 %U https://proceedings.mlr.press/v22/bardenet12.html %V 22 %X We propose a novel adaptive MCMC algorithm named AMOR (Adaptive Metropolis with Online Relabeling) for efficiently simulating from permutation-invariant targets occurring in, for example, Bayesian analysis of mixture models. An important feature of the algorithm is to tie the adaptation of the proposal distribution to the choice of a particular restriction of the target to a domain where label switching cannot occur. The algorithm relies on a stochastic approximation procedure for which we exhibit a Lyapunov function that formally defines the criterion used for selecting the relabeling rule. This criterion reveals an interesting connection with the problem of optimal quantifier design in vector quantization which was only implicit in previous works on the label switching problem. In benchmark examples, the algorithm turns out to be fast converging and efficient at selecting meaningful non-trivial relabeling rules to allow accurate parameter inference.
RIS
TY - CPAPER TI - Adaptive Metropolis with Online Relabeling AU - Remi Bardenet AU - Olivier Cappe AU - Gersende Fort AU - Balazs Kegl BT - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics DA - 2012/03/21 ED - Neil D. Lawrence ED - Mark Girolami ID - pmlr-v22-bardenet12 PB - PMLR DP - Proceedings of Machine Learning Research VL - 22 SP - 91 EP - 99 L1 - http://proceedings.mlr.press/v22/bardenet12/bardenet12.pdf UR - https://proceedings.mlr.press/v22/bardenet12.html AB - We propose a novel adaptive MCMC algorithm named AMOR (Adaptive Metropolis with Online Relabeling) for efficiently simulating from permutation-invariant targets occurring in, for example, Bayesian analysis of mixture models. An important feature of the algorithm is to tie the adaptation of the proposal distribution to the choice of a particular restriction of the target to a domain where label switching cannot occur. The algorithm relies on a stochastic approximation procedure for which we exhibit a Lyapunov function that formally defines the criterion used for selecting the relabeling rule. This criterion reveals an interesting connection with the problem of optimal quantifier design in vector quantization which was only implicit in previous works on the label switching problem. In benchmark examples, the algorithm turns out to be fast converging and efficient at selecting meaningful non-trivial relabeling rules to allow accurate parameter inference. ER -
APA
Bardenet, R., Cappe, O., Fort, G. & Kegl, B.. (2012). Adaptive Metropolis with Online Relabeling. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 22:91-99 Available from https://proceedings.mlr.press/v22/bardenet12.html.

Related Material