Diagonal Orthant Multinomial Probit Models

James Johndrow, David Dunson, Kristian Lum
Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics, PMLR 31:29-38, 2013.

Abstract

Bayesian classification commonly relies on probit models, with data augmentation algorithms used for posterior computation. By imputing latent Gaussian variables, one can often trivially adapt computational approaches used in Gaussian models. However, MCMC for multinomial probit (MNP) models can be inefficient in practice due to high posterior dependence between latent variables and parameters, and to difficulties in efficiently sampling latent variables when there are more than two categories. To address these problems, we propose a new class of diagonal orthant (DO) multinomial models. The key characteristics of these models include conditional independence of the latent variables given model parameters, avoidance of arbitrary identifiability restrictions, and simple expressions for category probabilities. We show substantially improved computational efficiency and comparable predictive performance to MNP.

Cite this Paper

BibTeX
@InProceedings{pmlr-v31-johndrow13a, title = {Diagonal Orthant Multinomial Probit Models}, author = {Johndrow, James and Dunson, David and Lum, Kristian}, booktitle = {Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics}, pages = {29--38}, year = {2013}, editor = {Carvalho, Carlos M. and Ravikumar, Pradeep}, volume = {31}, series = {Proceedings of Machine Learning Research}, address = {Scottsdale, Arizona, USA}, month = {29 Apr--01 May}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v31/johndrow13a.pdf}, url = {https://proceedings.mlr.press/v31/johndrow13a.html}, abstract = {Bayesian classification commonly relies on probit models, with data augmentation algorithms used for posterior computation. By imputing latent Gaussian variables, one can often trivially adapt computational approaches used in Gaussian models. However, MCMC for multinomial probit (MNP) models can be inefficient in practice due to high posterior dependence between latent variables and parameters, and to difficulties in efficiently sampling latent variables when there are more than two categories. To address these problems, we propose a new class of diagonal orthant (DO) multinomial models. The key characteristics of these models include conditional independence of the latent variables given model parameters, avoidance of arbitrary identifiability restrictions, and simple expressions for category probabilities. We show substantially improved computational efficiency and comparable predictive performance to MNP. }, note = {Notable paper award} }
Endnote
%0 Conference Paper %T Diagonal Orthant Multinomial Probit Models %A James Johndrow %A David Dunson %A Kristian Lum %B Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2013 %E Carlos M. Carvalho %E Pradeep Ravikumar %F pmlr-v31-johndrow13a %I PMLR %P 29--38 %U https://proceedings.mlr.press/v31/johndrow13a.html %V 31 %X Bayesian classification commonly relies on probit models, with data augmentation algorithms used for posterior computation. By imputing latent Gaussian variables, one can often trivially adapt computational approaches used in Gaussian models. However, MCMC for multinomial probit (MNP) models can be inefficient in practice due to high posterior dependence between latent variables and parameters, and to difficulties in efficiently sampling latent variables when there are more than two categories. To address these problems, we propose a new class of diagonal orthant (DO) multinomial models. The key characteristics of these models include conditional independence of the latent variables given model parameters, avoidance of arbitrary identifiability restrictions, and simple expressions for category probabilities. We show substantially improved computational efficiency and comparable predictive performance to MNP. %Z Notable paper award
RIS
TY - CPAPER TI - Diagonal Orthant Multinomial Probit Models AU - James Johndrow AU - David Dunson AU - Kristian Lum BT - Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics DA - 2013/04/29 ED - Carlos M. Carvalho ED - Pradeep Ravikumar ID - pmlr-v31-johndrow13a PB - PMLR DP - Proceedings of Machine Learning Research VL - 31 SP - 29 EP - 38 L1 - http://proceedings.mlr.press/v31/johndrow13a.pdf UR - https://proceedings.mlr.press/v31/johndrow13a.html AB - Bayesian classification commonly relies on probit models, with data augmentation algorithms used for posterior computation. By imputing latent Gaussian variables, one can often trivially adapt computational approaches used in Gaussian models. However, MCMC for multinomial probit (MNP) models can be inefficient in practice due to high posterior dependence between latent variables and parameters, and to difficulties in efficiently sampling latent variables when there are more than two categories. To address these problems, we propose a new class of diagonal orthant (DO) multinomial models. The key characteristics of these models include conditional independence of the latent variables given model parameters, avoidance of arbitrary identifiability restrictions, and simple expressions for category probabilities. We show substantially improved computational efficiency and comparable predictive performance to MNP. N1 - Notable paper award ER -
APA
Johndrow, J., Dunson, D. & Lum, K.. (2013). Diagonal Orthant Multinomial Probit Models. Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 31:29-38 Available from https://proceedings.mlr.press/v31/johndrow13a.html. Notable paper award

Related Material