Matrix-Variate Dirichlet Process Mixture Models

Zhihua Zhang, Guang Dai, Michael I. Jordan
Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, PMLR 9:980-987, 2010.

Abstract

We are concerned with a multivariate response regression problem where the interest is in considering correlations both across response variates and across response samples. In this paper we develop a new Bayesian nonparametric model for such a setting based on Dirichlet process priors. Building on an additive kernel model, we allow each sample to have its own regression matrix. Although this overcomplete representation could in principle suffer from severe overfitting problems, we are able to provide effective control over the model via a matrix-variate Dirichlet process prior on the regression matrices. Our model is able to share statistical strength among regression matrices due to the clustering property of the Dirichlet process. We make use of a Markov chain Monte Carlo algorithm for inference and prediction. Compared with other Bayesian kernel models, our model has advantages in both computational and statistical efficiency.

Cite this Paper

BibTeX
@InProceedings{pmlr-v9-zhang10e, title = {Matrix-Variate Dirichlet Process Mixture Models}, author = {Zhang, Zhihua and Dai, Guang and Jordan, Michael I.}, booktitle = {Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics}, pages = {980--987}, year = {2010}, editor = {Teh, Yee Whye and Titterington, Mike}, volume = {9}, series = {Proceedings of Machine Learning Research}, address = {Chia Laguna Resort, Sardinia, Italy}, month = {13--15 May}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v9/zhang10e/zhang10e.pdf}, url = {https://proceedings.mlr.press/v9/zhang10e.html}, abstract = {We are concerned with a multivariate response regression problem where the interest is in considering correlations both across response variates and across response samples. In this paper we develop a new Bayesian nonparametric model for such a setting based on Dirichlet process priors. Building on an additive kernel model, we allow each sample to have its own regression matrix. Although this overcomplete representation could in principle suffer from severe overfitting problems, we are able to provide effective control over the model via a matrix-variate Dirichlet process prior on the regression matrices. Our model is able to share statistical strength among regression matrices due to the clustering property of the Dirichlet process. We make use of a Markov chain Monte Carlo algorithm for inference and prediction. Compared with other Bayesian kernel models, our model has advantages in both computational and statistical efficiency.} }
Endnote
%0 Conference Paper %T Matrix-Variate Dirichlet Process Mixture Models %A Zhihua Zhang %A Guang Dai %A Michael I. Jordan %B Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2010 %E Yee Whye Teh %E Mike Titterington %F pmlr-v9-zhang10e %I PMLR %P 980--987 %U https://proceedings.mlr.press/v9/zhang10e.html %V 9 %X We are concerned with a multivariate response regression problem where the interest is in considering correlations both across response variates and across response samples. In this paper we develop a new Bayesian nonparametric model for such a setting based on Dirichlet process priors. Building on an additive kernel model, we allow each sample to have its own regression matrix. Although this overcomplete representation could in principle suffer from severe overfitting problems, we are able to provide effective control over the model via a matrix-variate Dirichlet process prior on the regression matrices. Our model is able to share statistical strength among regression matrices due to the clustering property of the Dirichlet process. We make use of a Markov chain Monte Carlo algorithm for inference and prediction. Compared with other Bayesian kernel models, our model has advantages in both computational and statistical efficiency.
RIS
TY - CPAPER TI - Matrix-Variate Dirichlet Process Mixture Models AU - Zhihua Zhang AU - Guang Dai AU - Michael I. Jordan BT - Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics DA - 2010/03/31 ED - Yee Whye Teh ED - Mike Titterington ID - pmlr-v9-zhang10e PB - PMLR DP - Proceedings of Machine Learning Research VL - 9 SP - 980 EP - 987 L1 - http://proceedings.mlr.press/v9/zhang10e/zhang10e.pdf UR - https://proceedings.mlr.press/v9/zhang10e.html AB - We are concerned with a multivariate response regression problem where the interest is in considering correlations both across response variates and across response samples. In this paper we develop a new Bayesian nonparametric model for such a setting based on Dirichlet process priors. Building on an additive kernel model, we allow each sample to have its own regression matrix. Although this overcomplete representation could in principle suffer from severe overfitting problems, we are able to provide effective control over the model via a matrix-variate Dirichlet process prior on the regression matrices. Our model is able to share statistical strength among regression matrices due to the clustering property of the Dirichlet process. We make use of a Markov chain Monte Carlo algorithm for inference and prediction. Compared with other Bayesian kernel models, our model has advantages in both computational and statistical efficiency. ER -
APA
Zhang, Z., Dai, G. & Jordan, M.I.. (2010). Matrix-Variate Dirichlet Process Mixture Models. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 9:980-987 Available from https://proceedings.mlr.press/v9/zhang10e.html.

Related Material