A Stick-Breaking Likelihood for Categorical Data Analysis with Latent Gaussian Models

Mohammad Khan, Shakir Mohamed, Benjamin Marlin, Kevin Murphy
Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:610-618, 2012.

Abstract

The development of accurate models and efficient algorithms for the analysis of multivariate categorical data are important and long-standing problems in machine learning and computational statistics. In this paper, we focus on modeling categorical data using Latent Gaussian Models (LGMs). We propose a novel stick-breaking likelihood function for categorical LGMs that exploits accurate linear and quadratic bounds on the logistic log-partition function, leading to an effective variational inference and learning framework. We thoroughly compare our approach to existing algorithms for multinomial logit/probit likelihoods on several problems, including inference in multinomial Gaussian process classification and learning in latent factor models. Our extensive comparisons demonstrate that our stick-breaking model effectively captures correlation in discrete data and is well suited for the analysis of categorical data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v22-khan12, title = {A Stick-Breaking Likelihood for Categorical Data Analysis with Latent Gaussian Models}, author = {Khan, Mohammad and Mohamed, Shakir and Marlin, Benjamin and Murphy, Kevin}, booktitle = {Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics}, pages = {610--618}, year = {2012}, editor = {Lawrence, Neil D. and Girolami, Mark}, volume = {22}, series = {Proceedings of Machine Learning Research}, address = {La Palma, Canary Islands}, month = {21--23 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v22/khan12/khan12.pdf}, url = {https://proceedings.mlr.press/v22/khan12.html}, abstract = {The development of accurate models and efficient algorithms for the analysis of multivariate categorical data are important and long-standing problems in machine learning and computational statistics. In this paper, we focus on modeling categorical data using Latent Gaussian Models (LGMs). We propose a novel stick-breaking likelihood function for categorical LGMs that exploits accurate linear and quadratic bounds on the logistic log-partition function, leading to an effective variational inference and learning framework. We thoroughly compare our approach to existing algorithms for multinomial logit/probit likelihoods on several problems, including inference in multinomial Gaussian process classification and learning in latent factor models. Our extensive comparisons demonstrate that our stick-breaking model effectively captures correlation in discrete data and is well suited for the analysis of categorical data.} }
Endnote
%0 Conference Paper %T A Stick-Breaking Likelihood for Categorical Data Analysis with Latent Gaussian Models %A Mohammad Khan %A Shakir Mohamed %A Benjamin Marlin %A Kevin Murphy %B Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2012 %E Neil D. Lawrence %E Mark Girolami %F pmlr-v22-khan12 %I PMLR %P 610--618 %U https://proceedings.mlr.press/v22/khan12.html %V 22 %X The development of accurate models and efficient algorithms for the analysis of multivariate categorical data are important and long-standing problems in machine learning and computational statistics. In this paper, we focus on modeling categorical data using Latent Gaussian Models (LGMs). We propose a novel stick-breaking likelihood function for categorical LGMs that exploits accurate linear and quadratic bounds on the logistic log-partition function, leading to an effective variational inference and learning framework. We thoroughly compare our approach to existing algorithms for multinomial logit/probit likelihoods on several problems, including inference in multinomial Gaussian process classification and learning in latent factor models. Our extensive comparisons demonstrate that our stick-breaking model effectively captures correlation in discrete data and is well suited for the analysis of categorical data.
RIS
TY - CPAPER TI - A Stick-Breaking Likelihood for Categorical Data Analysis with Latent Gaussian Models AU - Mohammad Khan AU - Shakir Mohamed AU - Benjamin Marlin AU - Kevin Murphy BT - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics DA - 2012/03/21 ED - Neil D. Lawrence ED - Mark Girolami ID - pmlr-v22-khan12 PB - PMLR DP - Proceedings of Machine Learning Research VL - 22 SP - 610 EP - 618 L1 - http://proceedings.mlr.press/v22/khan12/khan12.pdf UR - https://proceedings.mlr.press/v22/khan12.html AB - The development of accurate models and efficient algorithms for the analysis of multivariate categorical data are important and long-standing problems in machine learning and computational statistics. In this paper, we focus on modeling categorical data using Latent Gaussian Models (LGMs). We propose a novel stick-breaking likelihood function for categorical LGMs that exploits accurate linear and quadratic bounds on the logistic log-partition function, leading to an effective variational inference and learning framework. We thoroughly compare our approach to existing algorithms for multinomial logit/probit likelihoods on several problems, including inference in multinomial Gaussian process classification and learning in latent factor models. Our extensive comparisons demonstrate that our stick-breaking model effectively captures correlation in discrete data and is well suited for the analysis of categorical data. ER -
APA
Khan, M., Mohamed, S., Marlin, B. & Murphy, K.. (2012). A Stick-Breaking Likelihood for Categorical Data Analysis with Latent Gaussian Models. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 22:610-618 Available from https://proceedings.mlr.press/v22/khan12.html.

Related Material