Deep Hybrid Models: Bridging Discriminative and Generative Approaches

Volodymyr Kuleshov, Stefano Ermon
Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence, PMLR R15:271-280, 2017.

Abstract

Most methods in machine learning are described as either discriminative or generative. The former often attain higher predictive accuracy, while the latter are more strongly regularized and can deal with missing data. Here, we propose a new framework to combine a broad class of discriminative and generative models, interpolating between the two extremes with a multiconditional likelihood objective. Unlike previous approaches, we couple the two components through shared latent variables, and train using recent advances in variational inference. Instantiating our framework with modern deep architectures gives rise to deep hybrid models, a highly flexible family that generalizes several existing models and is effective in the semi-supervised setting, where it results in improvements over the state of the art on the SVHN dataset.

Cite this Paper

BibTeX
@InProceedings{pmlr-vR15-kuleshov17a, title = {Deep Hybrid Models: Bridging Discriminative and Generative Approaches}, author = {Kuleshov, Volodymyr and Ermon, Stefano}, booktitle = {Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence}, pages = {271--280}, year = {2017}, editor = {Elidan, Gal and Kersting, Kristian}, volume = {R15}, series = {Proceedings of Machine Learning Research}, month = {11--15 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/r15/main/assets/kuleshov17a/kuleshov17a.pdf}, url = {https://proceedings.mlr.press/r15/kuleshov17a.html}, abstract = {Most methods in machine learning are described as either discriminative or generative. The former often attain higher predictive accuracy, while the latter are more strongly regularized and can deal with missing data. Here, we propose a new framework to combine a broad class of discriminative and generative models, interpolating between the two extremes with a multiconditional likelihood objective. Unlike previous approaches, we couple the two components through shared latent variables, and train using recent advances in variational inference. Instantiating our framework with modern deep architectures gives rise to deep hybrid models, a highly flexible family that generalizes several existing models and is effective in the semi-supervised setting, where it results in improvements over the state of the art on the SVHN dataset.}, note = {Reissued by PMLR on 04 October 2026.} }
Endnote
%0 Conference Paper %T Deep Hybrid Models: Bridging Discriminative and Generative Approaches %A Volodymyr Kuleshov %A Stefano Ermon %B Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2017 %E Gal Elidan %E Kristian Kersting %F pmlr-vR15-kuleshov17a %I PMLR %P 271--280 %U https://proceedings.mlr.press/r15/kuleshov17a.html %V R15 %X Most methods in machine learning are described as either discriminative or generative. The former often attain higher predictive accuracy, while the latter are more strongly regularized and can deal with missing data. Here, we propose a new framework to combine a broad class of discriminative and generative models, interpolating between the two extremes with a multiconditional likelihood objective. Unlike previous approaches, we couple the two components through shared latent variables, and train using recent advances in variational inference. Instantiating our framework with modern deep architectures gives rise to deep hybrid models, a highly flexible family that generalizes several existing models and is effective in the semi-supervised setting, where it results in improvements over the state of the art on the SVHN dataset. %Z Reissued by PMLR on 04 October 2026.
APA
Kuleshov, V. & Ermon, S.. (2017). Deep Hybrid Models: Bridging Discriminative and Generative Approaches. Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research R15:271-280 Available from https://proceedings.mlr.press/r15/kuleshov17a.html. Reissued by PMLR on 04 October 2026.

Related Material