A Stick-Breaking Likelihood for Categorical Data Analysis with Latent Gaussian Models

Mohammad Khan, Shakir Mohamed, Benjamin Marlin, Kevin Murphy
; Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:610-618, 2012.

Abstract

The development of accurate models and efficient algorithms for the analysis of multivariate categorical data are important and long-standing problems in machine learning and computational statistics. In this paper, we focus on modeling categorical data using Latent Gaussian Models (LGMs). We propose a novel stick-breaking likelihood function for categorical LGMs that exploits accurate linear and quadratic bounds on the logistic log-partition function, leading to an effective variational inference and learning framework. We thoroughly compare our approach to existing algorithms for multinomial logit/probit likelihoods on several problems, including inference in multinomial Gaussian process classification and learning in latent factor models. Our extensive comparisons demonstrate that our stick-breaking model effectively captures correlation in discrete data and is well suited for the analysis of categorical data.

Cite this Paper


BibTeX
@InProceedings{pmlr-v22-khan12, title = {A Stick-Breaking Likelihood for Categorical Data Analysis with Latent Gaussian Models}, author = {Mohammad Khan and Shakir Mohamed and Benjamin Marlin and Kevin Murphy}, booktitle = {Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics}, pages = {610--618}, year = {2012}, editor = {Neil D. Lawrence and Mark Girolami}, volume = {22}, series = {Proceedings of Machine Learning Research}, address = {La Palma, Canary Islands}, month = {21--23 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v22/khan12/khan12.pdf}, url = {http://proceedings.mlr.press/v22/khan12.html}, abstract = {The development of accurate models and efficient algorithms for the analysis of multivariate categorical data are important and long-standing problems in machine learning and computational statistics. In this paper, we focus on modeling categorical data using Latent Gaussian Models (LGMs). We propose a novel stick-breaking likelihood function for categorical LGMs that exploits accurate linear and quadratic bounds on the logistic log-partition function, leading to an effective variational inference and learning framework. We thoroughly compare our approach to existing algorithms for multinomial logit/probit likelihoods on several problems, including inference in multinomial Gaussian process classification and learning in latent factor models. Our extensive comparisons demonstrate that our stick-breaking model effectively captures correlation in discrete data and is well suited for the analysis of categorical data.} }
Endnote
%0 Conference Paper %T A Stick-Breaking Likelihood for Categorical Data Analysis with Latent Gaussian Models %A Mohammad Khan %A Shakir Mohamed %A Benjamin Marlin %A Kevin Murphy %B Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2012 %E Neil D. Lawrence %E Mark Girolami %F pmlr-v22-khan12 %I PMLR %J Proceedings of Machine Learning Research %P 610--618 %U http://proceedings.mlr.press %V 22 %W PMLR %X The development of accurate models and efficient algorithms for the analysis of multivariate categorical data are important and long-standing problems in machine learning and computational statistics. In this paper, we focus on modeling categorical data using Latent Gaussian Models (LGMs). We propose a novel stick-breaking likelihood function for categorical LGMs that exploits accurate linear and quadratic bounds on the logistic log-partition function, leading to an effective variational inference and learning framework. We thoroughly compare our approach to existing algorithms for multinomial logit/probit likelihoods on several problems, including inference in multinomial Gaussian process classification and learning in latent factor models. Our extensive comparisons demonstrate that our stick-breaking model effectively captures correlation in discrete data and is well suited for the analysis of categorical data.
RIS
TY - CPAPER TI - A Stick-Breaking Likelihood for Categorical Data Analysis with Latent Gaussian Models AU - Mohammad Khan AU - Shakir Mohamed AU - Benjamin Marlin AU - Kevin Murphy BT - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics PY - 2012/03/21 DA - 2012/03/21 ED - Neil D. Lawrence ED - Mark Girolami ID - pmlr-v22-khan12 PB - PMLR SP - 610 DP - PMLR EP - 618 L1 - http://proceedings.mlr.press/v22/khan12/khan12.pdf UR - http://proceedings.mlr.press/v22/khan12.html AB - The development of accurate models and efficient algorithms for the analysis of multivariate categorical data are important and long-standing problems in machine learning and computational statistics. In this paper, we focus on modeling categorical data using Latent Gaussian Models (LGMs). We propose a novel stick-breaking likelihood function for categorical LGMs that exploits accurate linear and quadratic bounds on the logistic log-partition function, leading to an effective variational inference and learning framework. We thoroughly compare our approach to existing algorithms for multinomial logit/probit likelihoods on several problems, including inference in multinomial Gaussian process classification and learning in latent factor models. Our extensive comparisons demonstrate that our stick-breaking model effectively captures correlation in discrete data and is well suited for the analysis of categorical data. ER -
APA
Khan, M., Mohamed, S., Marlin, B. & Murphy, K.. (2012). A Stick-Breaking Likelihood for Categorical Data Analysis with Latent Gaussian Models. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, in PMLR 22:610-618

Related Material