Rule-Enhanced Penalized Regression by Column Generation using Rectangular Maximum Agreement


Jonathan Eckstein, Noam Goldberg, Ai Kagawa ;
Proceedings of the 34th International Conference on Machine Learning, PMLR 70:1059-1067, 2017.


We describe a learning procedure enhancing L1-penalized regression by adding dynamically generated rules describing multidimensional “box” sets. Our rule-adding procedure is based on the classical column generation method for high-dimensional linear programming. The pricing problem for our column generation procedure reduces to the NP-hard rectangular maximum agreement (RMA) problem of finding a box that best discriminates between two weighted datasets. We solve this problem exactly using a parallel branch-and-bound procedure. The resulting rule-enhanced regression procedure is computation-intensive, but has promising prediction performance.

Related Material