Adaptive Metropolis with Online Relabeling

Remi Bardenet, Olivier Cappe, Gersende Fort, Balazs Kegl
; Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:91-99, 2012.

Abstract

We propose a novel adaptive MCMC algorithm named AMOR (Adaptive Metropolis with Online Relabeling) for efficiently simulating from permutation-invariant targets occurring in, for example, Bayesian analysis of mixture models. An important feature of the algorithm is to tie the adaptation of the proposal distribution to the choice of a particular restriction of the target to a domain where label switching cannot occur. The algorithm relies on a stochastic approximation procedure for which we exhibit a Lyapunov function that formally defines the criterion used for selecting the relabeling rule. This criterion reveals an interesting connection with the problem of optimal quantifier design in vector quantization which was only implicit in previous works on the label switching problem. In benchmark examples, the algorithm turns out to be fast converging and efficient at selecting meaningful non-trivial relabeling rules to allow accurate parameter inference.

Cite this Paper


BibTeX
@InProceedings{pmlr-v22-bardenet12, title = {Adaptive Metropolis with Online Relabeling}, author = {Remi Bardenet and Olivier Cappe and Gersende Fort and Balazs Kegl}, booktitle = {Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics}, pages = {91--99}, year = {2012}, editor = {Neil D. Lawrence and Mark Girolami}, volume = {22}, series = {Proceedings of Machine Learning Research}, address = {La Palma, Canary Islands}, month = {21--23 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v22/bardenet12/bardenet12.pdf}, url = {http://proceedings.mlr.press/v22/bardenet12.html}, abstract = {We propose a novel adaptive MCMC algorithm named AMOR (Adaptive Metropolis with Online Relabeling) for efficiently simulating from permutation-invariant targets occurring in, for example, Bayesian analysis of mixture models. An important feature of the algorithm is to tie the adaptation of the proposal distribution to the choice of a particular restriction of the target to a domain where label switching cannot occur. The algorithm relies on a stochastic approximation procedure for which we exhibit a Lyapunov function that formally defines the criterion used for selecting the relabeling rule. This criterion reveals an interesting connection with the problem of optimal quantifier design in vector quantization which was only implicit in previous works on the label switching problem. In benchmark examples, the algorithm turns out to be fast converging and efficient at selecting meaningful non-trivial relabeling rules to allow accurate parameter inference.} }
Endnote
%0 Conference Paper %T Adaptive Metropolis with Online Relabeling %A Remi Bardenet %A Olivier Cappe %A Gersende Fort %A Balazs Kegl %B Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2012 %E Neil D. Lawrence %E Mark Girolami %F pmlr-v22-bardenet12 %I PMLR %J Proceedings of Machine Learning Research %P 91--99 %U http://proceedings.mlr.press %V 22 %W PMLR %X We propose a novel adaptive MCMC algorithm named AMOR (Adaptive Metropolis with Online Relabeling) for efficiently simulating from permutation-invariant targets occurring in, for example, Bayesian analysis of mixture models. An important feature of the algorithm is to tie the adaptation of the proposal distribution to the choice of a particular restriction of the target to a domain where label switching cannot occur. The algorithm relies on a stochastic approximation procedure for which we exhibit a Lyapunov function that formally defines the criterion used for selecting the relabeling rule. This criterion reveals an interesting connection with the problem of optimal quantifier design in vector quantization which was only implicit in previous works on the label switching problem. In benchmark examples, the algorithm turns out to be fast converging and efficient at selecting meaningful non-trivial relabeling rules to allow accurate parameter inference.
RIS
TY - CPAPER TI - Adaptive Metropolis with Online Relabeling AU - Remi Bardenet AU - Olivier Cappe AU - Gersende Fort AU - Balazs Kegl BT - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics PY - 2012/03/21 DA - 2012/03/21 ED - Neil D. Lawrence ED - Mark Girolami ID - pmlr-v22-bardenet12 PB - PMLR SP - 91 DP - PMLR EP - 99 L1 - http://proceedings.mlr.press/v22/bardenet12/bardenet12.pdf UR - http://proceedings.mlr.press/v22/bardenet12.html AB - We propose a novel adaptive MCMC algorithm named AMOR (Adaptive Metropolis with Online Relabeling) for efficiently simulating from permutation-invariant targets occurring in, for example, Bayesian analysis of mixture models. An important feature of the algorithm is to tie the adaptation of the proposal distribution to the choice of a particular restriction of the target to a domain where label switching cannot occur. The algorithm relies on a stochastic approximation procedure for which we exhibit a Lyapunov function that formally defines the criterion used for selecting the relabeling rule. This criterion reveals an interesting connection with the problem of optimal quantifier design in vector quantization which was only implicit in previous works on the label switching problem. In benchmark examples, the algorithm turns out to be fast converging and efficient at selecting meaningful non-trivial relabeling rules to allow accurate parameter inference. ER -
APA
Bardenet, R., Cappe, O., Fort, G. & Kegl, B.. (2012). Adaptive Metropolis with Online Relabeling. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, in PMLR 22:91-99

Related Material