Variational Inference for Sequential Distance Dependent Chinese Restaurant Process

Sergey Bartunov, Dmitry Vetrov
Proceedings of the 31st International Conference on Machine Learning, PMLR 32(2):1404-1412, 2014.

Abstract

Recently proposed distance dependent Chinese Restaurant Process (ddCRP) generalizes extensively used Chinese Restaurant Process (CRP) by accounting for dependencies between data points. Its posterior is intractable and so far only MCMC methods were used for inference. Because of very different nature of ddCRP no prior developments in variational methods for Bayesian nonparametrics are appliable. In this paper we propose novel variational inference for important sequential case of ddCRP (seqddCRP) by revealing its connection with Laplacian of random graph constructed by the process. We develop efficient algorithm for optimizing variational lower bound and demonstrate its efficiency comparing to Gibbs sampler. We also apply our variational approximation to CRP-equivalent seqddCRP-mixture model, where it could be considered as alternative to one based on truncated stick-breaking representation. This allowed us to achieve significantly better variational lower bound than variational approximation based on truncated stick breaking for Dirichlet process.

Cite this Paper

BibTeX
@InProceedings{pmlr-v32-bartunov14, title = {Variational Inference for Sequential Distance Dependent Chinese Restaurant Process}, author = {Bartunov, Sergey and Vetrov, Dmitry}, booktitle = {Proceedings of the 31st International Conference on Machine Learning}, pages = {1404--1412}, year = {2014}, editor = {Xing, Eric P. and Jebara, Tony}, volume = {32}, number = {2}, series = {Proceedings of Machine Learning Research}, address = {Bejing, China}, month = {22--24 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v32/bartunov14.pdf}, url = {https://proceedings.mlr.press/v32/bartunov14.html}, abstract = {Recently proposed distance dependent Chinese Restaurant Process (ddCRP) generalizes extensively used Chinese Restaurant Process (CRP) by accounting for dependencies between data points. Its posterior is intractable and so far only MCMC methods were used for inference. Because of very different nature of ddCRP no prior developments in variational methods for Bayesian nonparametrics are appliable. In this paper we propose novel variational inference for important sequential case of ddCRP (seqddCRP) by revealing its connection with Laplacian of random graph constructed by the process. We develop efficient algorithm for optimizing variational lower bound and demonstrate its efficiency comparing to Gibbs sampler. We also apply our variational approximation to CRP-equivalent seqddCRP-mixture model, where it could be considered as alternative to one based on truncated stick-breaking representation. This allowed us to achieve significantly better variational lower bound than variational approximation based on truncated stick breaking for Dirichlet process.} }
Endnote
%0 Conference Paper %T Variational Inference for Sequential Distance Dependent Chinese Restaurant Process %A Sergey Bartunov %A Dmitry Vetrov %B Proceedings of the 31st International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2014 %E Eric P. Xing %E Tony Jebara %F pmlr-v32-bartunov14 %I PMLR %P 1404--1412 %U https://proceedings.mlr.press/v32/bartunov14.html %V 32 %N 2 %X Recently proposed distance dependent Chinese Restaurant Process (ddCRP) generalizes extensively used Chinese Restaurant Process (CRP) by accounting for dependencies between data points. Its posterior is intractable and so far only MCMC methods were used for inference. Because of very different nature of ddCRP no prior developments in variational methods for Bayesian nonparametrics are appliable. In this paper we propose novel variational inference for important sequential case of ddCRP (seqddCRP) by revealing its connection with Laplacian of random graph constructed by the process. We develop efficient algorithm for optimizing variational lower bound and demonstrate its efficiency comparing to Gibbs sampler. We also apply our variational approximation to CRP-equivalent seqddCRP-mixture model, where it could be considered as alternative to one based on truncated stick-breaking representation. This allowed us to achieve significantly better variational lower bound than variational approximation based on truncated stick breaking for Dirichlet process.
RIS
TY - CPAPER TI - Variational Inference for Sequential Distance Dependent Chinese Restaurant Process AU - Sergey Bartunov AU - Dmitry Vetrov BT - Proceedings of the 31st International Conference on Machine Learning DA - 2014/06/18 ED - Eric P. Xing ED - Tony Jebara ID - pmlr-v32-bartunov14 PB - PMLR DP - Proceedings of Machine Learning Research VL - 32 IS - 2 SP - 1404 EP - 1412 L1 - http://proceedings.mlr.press/v32/bartunov14.pdf UR - https://proceedings.mlr.press/v32/bartunov14.html AB - Recently proposed distance dependent Chinese Restaurant Process (ddCRP) generalizes extensively used Chinese Restaurant Process (CRP) by accounting for dependencies between data points. Its posterior is intractable and so far only MCMC methods were used for inference. Because of very different nature of ddCRP no prior developments in variational methods for Bayesian nonparametrics are appliable. In this paper we propose novel variational inference for important sequential case of ddCRP (seqddCRP) by revealing its connection with Laplacian of random graph constructed by the process. We develop efficient algorithm for optimizing variational lower bound and demonstrate its efficiency comparing to Gibbs sampler. We also apply our variational approximation to CRP-equivalent seqddCRP-mixture model, where it could be considered as alternative to one based on truncated stick-breaking representation. This allowed us to achieve significantly better variational lower bound than variational approximation based on truncated stick breaking for Dirichlet process. ER -
APA
Bartunov, S. & Vetrov, D.. (2014). Variational Inference for Sequential Distance Dependent Chinese Restaurant Process. Proceedings of the 31st International Conference on Machine Learning, in Proceedings of Machine Learning Research 32(2):1404-1412 Available from https://proceedings.mlr.press/v32/bartunov14.html.

Related Material