Generalized Linear Rule Models

Dennis Wei, Sanjeeb Dash, Tian Gao, Oktay Gunluk
Proceedings of the 36th International Conference on Machine Learning, PMLR 97:6687-6696, 2019.

Abstract

This paper considers generalized linear models using rule-based features, also referred to as rule ensembles, for regression and probabilistic classification. Rules facilitate model interpretation while also capturing nonlinear dependences and interactions. Our problem formulation accordingly trades off rule set complexity and prediction accuracy. Column generation is used to optimize over an exponentially large space of rules without pre-generating a large subset of candidates or greedily boosting rules one by one. The column generation subproblem is solved using either integer programming or a heuristic optimizing the same objective. In experiments involving logistic and linear regression, the proposed methods obtain better accuracy-complexity trade-offs than existing rule ensemble algorithms. At one end of the trade-off, the methods are competitive with less interpretable benchmark models.

Cite this Paper

BibTeX
@InProceedings{pmlr-v97-wei19a, title = {Generalized Linear Rule Models}, author = {Wei, Dennis and Dash, Sanjeeb and Gao, Tian and Gunluk, Oktay}, booktitle = {Proceedings of the 36th International Conference on Machine Learning}, pages = {6687--6696}, year = {2019}, editor = {Chaudhuri, Kamalika and Salakhutdinov, Ruslan}, volume = {97}, series = {Proceedings of Machine Learning Research}, month = {09--15 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v97/wei19a/wei19a.pdf}, url = {https://proceedings.mlr.press/v97/wei19a.html}, abstract = {This paper considers generalized linear models using rule-based features, also referred to as rule ensembles, for regression and probabilistic classification. Rules facilitate model interpretation while also capturing nonlinear dependences and interactions. Our problem formulation accordingly trades off rule set complexity and prediction accuracy. Column generation is used to optimize over an exponentially large space of rules without pre-generating a large subset of candidates or greedily boosting rules one by one. The column generation subproblem is solved using either integer programming or a heuristic optimizing the same objective. In experiments involving logistic and linear regression, the proposed methods obtain better accuracy-complexity trade-offs than existing rule ensemble algorithms. At one end of the trade-off, the methods are competitive with less interpretable benchmark models.} }
Endnote
%0 Conference Paper %T Generalized Linear Rule Models %A Dennis Wei %A Sanjeeb Dash %A Tian Gao %A Oktay Gunluk %B Proceedings of the 36th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2019 %E Kamalika Chaudhuri %E Ruslan Salakhutdinov %F pmlr-v97-wei19a %I PMLR %P 6687--6696 %U https://proceedings.mlr.press/v97/wei19a.html %V 97 %X This paper considers generalized linear models using rule-based features, also referred to as rule ensembles, for regression and probabilistic classification. Rules facilitate model interpretation while also capturing nonlinear dependences and interactions. Our problem formulation accordingly trades off rule set complexity and prediction accuracy. Column generation is used to optimize over an exponentially large space of rules without pre-generating a large subset of candidates or greedily boosting rules one by one. The column generation subproblem is solved using either integer programming or a heuristic optimizing the same objective. In experiments involving logistic and linear regression, the proposed methods obtain better accuracy-complexity trade-offs than existing rule ensemble algorithms. At one end of the trade-off, the methods are competitive with less interpretable benchmark models.
APA
Wei, D., Dash, S., Gao, T. & Gunluk, O.. (2019). Generalized Linear Rule Models. Proceedings of the 36th International Conference on Machine Learning, in Proceedings of Machine Learning Research 97:6687-6696 Available from https://proceedings.mlr.press/v97/wei19a.html.

Related Material