There’s a Hole in My Data Space: Piecewise Predictors for Heterogeneous Learning Problems

Ofer Dekel, Ohad Shamir
Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:291-298, 2012.

Abstract

We study statistical learning problems where the data space is multimodal and heterogeneous, and constructing a single global predictor is difficult. We address such problems by iteratively identifying high-error regions in the data space and learning specialized predictors for these regions. While the idea of composing localized predictors is not new, our approach is unique in that we actively seek out predictors that clump errors together, making it easier to isolate the problematic regions. When errors are clumped together they are also easier to interpret and resolve through appropriate feature engineering and data preprocessing. We present an error-clumping classification algorithm based on a convex optimization problem, and an efficient stochastic optimization algorithm for this problem. We theoretically motivate our approach with a novel sample complexity analysis for piecewise predictors, and empirically demonstrate its behavior on an illustrative classification problem.

Cite this Paper

BibTeX
@InProceedings{pmlr-v22-dekel12, title = {There's a Hole in My Data Space: Piecewise Predictors for Heterogeneous Learning Problems}, author = {Dekel, Ofer and Shamir, Ohad}, booktitle = {Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics}, pages = {291--298}, year = {2012}, editor = {Lawrence, Neil D. and Girolami, Mark}, volume = {22}, series = {Proceedings of Machine Learning Research}, address = {La Palma, Canary Islands}, month = {21--23 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v22/dekel12/dekel12.pdf}, url = {https://proceedings.mlr.press/v22/dekel12.html}, abstract = {We study statistical learning problems where the data space is multimodal and heterogeneous, and constructing a single global predictor is difficult. We address such problems by iteratively identifying high-error regions in the data space and learning specialized predictors for these regions. While the idea of composing localized predictors is not new, our approach is unique in that we actively seek out predictors that clump errors together, making it easier to isolate the problematic regions. When errors are clumped together they are also easier to interpret and resolve through appropriate feature engineering and data preprocessing. We present an error-clumping classification algorithm based on a convex optimization problem, and an efficient stochastic optimization algorithm for this problem. We theoretically motivate our approach with a novel sample complexity analysis for piecewise predictors, and empirically demonstrate its behavior on an illustrative classification problem.} }
Endnote
%0 Conference Paper %T There’s a Hole in My Data Space: Piecewise Predictors for Heterogeneous Learning Problems %A Ofer Dekel %A Ohad Shamir %B Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2012 %E Neil D. Lawrence %E Mark Girolami %F pmlr-v22-dekel12 %I PMLR %P 291--298 %U https://proceedings.mlr.press/v22/dekel12.html %V 22 %X We study statistical learning problems where the data space is multimodal and heterogeneous, and constructing a single global predictor is difficult. We address such problems by iteratively identifying high-error regions in the data space and learning specialized predictors for these regions. While the idea of composing localized predictors is not new, our approach is unique in that we actively seek out predictors that clump errors together, making it easier to isolate the problematic regions. When errors are clumped together they are also easier to interpret and resolve through appropriate feature engineering and data preprocessing. We present an error-clumping classification algorithm based on a convex optimization problem, and an efficient stochastic optimization algorithm for this problem. We theoretically motivate our approach with a novel sample complexity analysis for piecewise predictors, and empirically demonstrate its behavior on an illustrative classification problem.
RIS
TY - CPAPER TI - There’s a Hole in My Data Space: Piecewise Predictors for Heterogeneous Learning Problems AU - Ofer Dekel AU - Ohad Shamir BT - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics DA - 2012/03/21 ED - Neil D. Lawrence ED - Mark Girolami ID - pmlr-v22-dekel12 PB - PMLR DP - Proceedings of Machine Learning Research VL - 22 SP - 291 EP - 298 L1 - http://proceedings.mlr.press/v22/dekel12/dekel12.pdf UR - https://proceedings.mlr.press/v22/dekel12.html AB - We study statistical learning problems where the data space is multimodal and heterogeneous, and constructing a single global predictor is difficult. We address such problems by iteratively identifying high-error regions in the data space and learning specialized predictors for these regions. While the idea of composing localized predictors is not new, our approach is unique in that we actively seek out predictors that clump errors together, making it easier to isolate the problematic regions. When errors are clumped together they are also easier to interpret and resolve through appropriate feature engineering and data preprocessing. We present an error-clumping classification algorithm based on a convex optimization problem, and an efficient stochastic optimization algorithm for this problem. We theoretically motivate our approach with a novel sample complexity analysis for piecewise predictors, and empirically demonstrate its behavior on an illustrative classification problem. ER -
APA
Dekel, O. & Shamir, O.. (2012). There’s a Hole in My Data Space: Piecewise Predictors for Heterogeneous Learning Problems. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 22:291-298 Available from https://proceedings.mlr.press/v22/dekel12.html.

Related Material