Streaming Variational Inference for Dirichlet Process Mixtures

Viet Huynh, Dinh Phung, Svetha Venkatesh
Asian Conference on Machine Learning, PMLR 45:237-252, 2016.

Abstract

Bayesian nonparametric models are theoretically suitable to learn streaming data due to their complexity relaxation to the volume of observed data. However, most of the existing variational inference algorithms are not applicable to streaming applications since they require truncation on variational distributions. In this paper, we present two truncation-free variational algorithms, one for mix-membership inference called TFVB (truncation-free variational Bayes), and the other for hard clustering inference called TFME (truncation-free maximization expectation). With these algorithms, we further developed a streaming learning framework for the popular Dirichlet process mixture (DPM) models. Our experiments demonstrate the usefulness of our framework in both synthetic and real-world data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v45-Huynh15, title = {Streaming Variational Inference for Dirichlet Process Mixtures}, author = {Huynh, Viet and Phung, Dinh and Venkatesh, Svetha}, booktitle = {Asian Conference on Machine Learning}, pages = {237--252}, year = {2016}, editor = {Holmes, Geoffrey and Liu, Tie-Yan}, volume = {45}, series = {Proceedings of Machine Learning Research}, address = {Hong Kong}, month = {20--22 Nov}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v45/Huynh15.pdf}, url = {https://proceedings.mlr.press/v45/Huynh15.html}, abstract = {Bayesian nonparametric models are theoretically suitable to learn streaming data due to their complexity relaxation to the volume of observed data. However, most of the existing variational inference algorithms are not applicable to streaming applications since they require truncation on variational distributions. In this paper, we present two truncation-free variational algorithms, one for mix-membership inference called TFVB (truncation-free variational Bayes), and the other for hard clustering inference called TFME (truncation-free maximization expectation). With these algorithms, we further developed a streaming learning framework for the popular Dirichlet process mixture (DPM) models. Our experiments demonstrate the usefulness of our framework in both synthetic and real-world data.} }
Endnote
%0 Conference Paper %T Streaming Variational Inference for Dirichlet Process Mixtures %A Viet Huynh %A Dinh Phung %A Svetha Venkatesh %B Asian Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2016 %E Geoffrey Holmes %E Tie-Yan Liu %F pmlr-v45-Huynh15 %I PMLR %P 237--252 %U https://proceedings.mlr.press/v45/Huynh15.html %V 45 %X Bayesian nonparametric models are theoretically suitable to learn streaming data due to their complexity relaxation to the volume of observed data. However, most of the existing variational inference algorithms are not applicable to streaming applications since they require truncation on variational distributions. In this paper, we present two truncation-free variational algorithms, one for mix-membership inference called TFVB (truncation-free variational Bayes), and the other for hard clustering inference called TFME (truncation-free maximization expectation). With these algorithms, we further developed a streaming learning framework for the popular Dirichlet process mixture (DPM) models. Our experiments demonstrate the usefulness of our framework in both synthetic and real-world data.
RIS
TY - CPAPER TI - Streaming Variational Inference for Dirichlet Process Mixtures AU - Viet Huynh AU - Dinh Phung AU - Svetha Venkatesh BT - Asian Conference on Machine Learning DA - 2016/02/25 ED - Geoffrey Holmes ED - Tie-Yan Liu ID - pmlr-v45-Huynh15 PB - PMLR DP - Proceedings of Machine Learning Research VL - 45 SP - 237 EP - 252 L1 - http://proceedings.mlr.press/v45/Huynh15.pdf UR - https://proceedings.mlr.press/v45/Huynh15.html AB - Bayesian nonparametric models are theoretically suitable to learn streaming data due to their complexity relaxation to the volume of observed data. However, most of the existing variational inference algorithms are not applicable to streaming applications since they require truncation on variational distributions. In this paper, we present two truncation-free variational algorithms, one for mix-membership inference called TFVB (truncation-free variational Bayes), and the other for hard clustering inference called TFME (truncation-free maximization expectation). With these algorithms, we further developed a streaming learning framework for the popular Dirichlet process mixture (DPM) models. Our experiments demonstrate the usefulness of our framework in both synthetic and real-world data. ER -
APA
Huynh, V., Phung, D. & Venkatesh, S.. (2016). Streaming Variational Inference for Dirichlet Process Mixtures. Asian Conference on Machine Learning, in Proceedings of Machine Learning Research 45:237-252 Available from https://proceedings.mlr.press/v45/Huynh15.html.

Related Material