On Construction of Hybrid Logistic Regression-Naïve Bayes Model for Classification

Yi Tan, Prakash P. Shenoy, Moses W. Chan, Paul M. Romberg
; Proceedings of the Eighth International Conference on Probabilistic Graphical Models, PMLR 52:523-534, 2016.

Abstract

In recent years, several authors have described a hybrid discriminative-generative model for classification. In this paper we examine construction of such hybrid models from data where we use logistic regression (LR) as a discriminative component, and naïve Bayes (NB) as a generative component. First, we estimate a Markov blanket of the class variable to reduce the set of features. Next, we use a heuristic to partition the set of features in the Markov blanket into those that are assigned to the LR part, and those that are assigned to the NB part of the hybrid model. The heuristic is based on reducing the conditional dependence of the features in NB part of the hybrid model given the class variable. We implement our method on 21 different classification datasets, and we compare the prediction accuracy of hybrid models with those of pure LR and pure NB models.

Cite this Paper


BibTeX
@InProceedings{pmlr-v52-tan16, title = {On Construction of Hybrid Logistic Regression-Na\"ive {B}ayes Model for Classification}, author = {Yi Tan and Prakash P. Shenoy and Moses W. Chan and Paul M. Romberg}, booktitle = {Proceedings of the Eighth International Conference on Probabilistic Graphical Models}, pages = {523--534}, year = {2016}, editor = {Alessandro Antonucci and Giorgio Corani and Cassio Polpo Campos}}, volume = {52}, series = {Proceedings of Machine Learning Research}, address = {Lugano, Switzerland}, month = {06--09 Sep}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v52/tan16.pdf}, url = {http://proceedings.mlr.press/v52/tan16.html}, abstract = {In recent years, several authors have described a hybrid discriminative-generative model for classification. In this paper we examine construction of such hybrid models from data where we use logistic regression (LR) as a discriminative component, and naïve Bayes (NB) as a generative component. First, we estimate a Markov blanket of the class variable to reduce the set of features. Next, we use a heuristic to partition the set of features in the Markov blanket into those that are assigned to the LR part, and those that are assigned to the NB part of the hybrid model. The heuristic is based on reducing the conditional dependence of the features in NB part of the hybrid model given the class variable. We implement our method on 21 different classification datasets, and we compare the prediction accuracy of hybrid models with those of pure LR and pure NB models.} }
Endnote
%0 Conference Paper %T On Construction of Hybrid Logistic Regression-Naïve Bayes Model for Classification %A Yi Tan %A Prakash P. Shenoy %A Moses W. Chan %A Paul M. Romberg %B Proceedings of the Eighth International Conference on Probabilistic Graphical Models %C Proceedings of Machine Learning Research %D 2016 %E Alessandro Antonucci %E Giorgio Corani %E Cassio Polpo Campos} %F pmlr-v52-tan16 %I PMLR %J Proceedings of Machine Learning Research %P 523--534 %U http://proceedings.mlr.press %V 52 %W PMLR %X In recent years, several authors have described a hybrid discriminative-generative model for classification. In this paper we examine construction of such hybrid models from data where we use logistic regression (LR) as a discriminative component, and naïve Bayes (NB) as a generative component. First, we estimate a Markov blanket of the class variable to reduce the set of features. Next, we use a heuristic to partition the set of features in the Markov blanket into those that are assigned to the LR part, and those that are assigned to the NB part of the hybrid model. The heuristic is based on reducing the conditional dependence of the features in NB part of the hybrid model given the class variable. We implement our method on 21 different classification datasets, and we compare the prediction accuracy of hybrid models with those of pure LR and pure NB models.
RIS
TY - CPAPER TI - On Construction of Hybrid Logistic Regression-Naïve Bayes Model for Classification AU - Yi Tan AU - Prakash P. Shenoy AU - Moses W. Chan AU - Paul M. Romberg BT - Proceedings of the Eighth International Conference on Probabilistic Graphical Models PY - 2016/08/15 DA - 2016/08/15 ED - Alessandro Antonucci ED - Giorgio Corani ED - Cassio Polpo Campos} ID - pmlr-v52-tan16 PB - PMLR SP - 523 DP - PMLR EP - 534 L1 - http://proceedings.mlr.press/v52/tan16.pdf UR - http://proceedings.mlr.press/v52/tan16.html AB - In recent years, several authors have described a hybrid discriminative-generative model for classification. In this paper we examine construction of such hybrid models from data where we use logistic regression (LR) as a discriminative component, and naïve Bayes (NB) as a generative component. First, we estimate a Markov blanket of the class variable to reduce the set of features. Next, we use a heuristic to partition the set of features in the Markov blanket into those that are assigned to the LR part, and those that are assigned to the NB part of the hybrid model. The heuristic is based on reducing the conditional dependence of the features in NB part of the hybrid model given the class variable. We implement our method on 21 different classification datasets, and we compare the prediction accuracy of hybrid models with those of pure LR and pure NB models. ER -
APA
Tan, Y., Shenoy, P.P., Chan, M.W. & Romberg, P.M.. (2016). On Construction of Hybrid Logistic Regression-Naïve Bayes Model for Classification. Proceedings of the Eighth International Conference on Probabilistic Graphical Models, in PMLR 52:523-534

Related Material