A Variational Approach to Bayesian Logistic Regression Models and their Extensions

Tommi S. Jaakkola, Michael I. Jordan
Proceedings of the Sixth International Workshop on Artificial Intelligence and Statistics, PMLR R1:283-294, 1997.

Abstract

We consider a logistic regression model with a Gaussian prior distribution over the parameters. We show that accurate variational techniques can be used to obtain a closed form posterior distribution over the parameters given the data thereby yielding a posterior predictive model. The results are readily extended to (binary) belief networks. For belief networks we also derive closed form posteriors in the presence of missing values. Finally, we show that the dual of the regression problem gives a latent variable density model, the variational formulation of which leads to exactly solvable EM updates.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR1-jaakkola97a, title = {A Variational Approach to {B}ayesian Logistic Regression Models and their Extensions}, author = {Jaakkola, Tommi S. and Jordan, Michael I.}, booktitle = {Proceedings of the Sixth International Workshop on Artificial Intelligence and Statistics}, pages = {283--294}, year = {1997}, editor = {Madigan, David and Smyth, Padhraic}, volume = {R1}, series = {Proceedings of Machine Learning Research}, month = {04--07 Jan}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/r1/jaakkola97a/jaakkola97a.pdf}, url = {https://proceedings.mlr.press/r1/jaakkola97a.html}, abstract = {We consider a logistic regression model with a Gaussian prior distribution over the parameters. We show that accurate variational techniques can be used to obtain a closed form posterior distribution over the parameters given the data thereby yielding a posterior predictive model. The results are readily extended to (binary) belief networks. For belief networks we also derive closed form posteriors in the presence of missing values. Finally, we show that the dual of the regression problem gives a latent variable density model, the variational formulation of which leads to exactly solvable EM updates.}, note = {Reissued by PMLR on 30 March 2021.} }
Endnote
%0 Conference Paper %T A Variational Approach to Bayesian Logistic Regression Models and their Extensions %A Tommi S. Jaakkola %A Michael I. Jordan %B Proceedings of the Sixth International Workshop on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 1997 %E David Madigan %E Padhraic Smyth %F pmlr-vR1-jaakkola97a %I PMLR %P 283--294 %U https://proceedings.mlr.press/r1/jaakkola97a.html %V R1 %X We consider a logistic regression model with a Gaussian prior distribution over the parameters. We show that accurate variational techniques can be used to obtain a closed form posterior distribution over the parameters given the data thereby yielding a posterior predictive model. The results are readily extended to (binary) belief networks. For belief networks we also derive closed form posteriors in the presence of missing values. Finally, we show that the dual of the regression problem gives a latent variable density model, the variational formulation of which leads to exactly solvable EM updates. %Z Reissued by PMLR on 30 March 2021.
APA
Jaakkola, T.S. & Jordan, M.I.. (1997). A Variational Approach to Bayesian Logistic Regression Models and their Extensions. Proceedings of the Sixth International Workshop on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research R1:283-294 Available from https://proceedings.mlr.press/r1/jaakkola97a.html. Reissued by PMLR on 30 March 2021.

Related Material