There’s a Hole in My Data Space: Piecewise Predictors for Heterogeneous Learning Problems

Ofer Dekel, Ohad Shamir
; Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, PMLR 22:291-298, 2012.

Abstract

We study statistical learning problems where the data space is multimodal and heterogeneous, and constructing a single global predictor is difficult. We address such problems by iteratively identifying high-error regions in the data space and learning specialized predictors for these regions. While the idea of composing localized predictors is not new, our approach is unique in that we actively seek out predictors that clump errors together, making it easier to isolate the problematic regions. When errors are clumped together they are also easier to interpret and resolve through appropriate feature engineering and data preprocessing. We present an error-clumping classification algorithm based on a convex optimization problem, and an efficient stochastic optimization algorithm for this problem. We theoretically motivate our approach with a novel sample complexity analysis for piecewise predictors, and empirically demonstrate its behavior on an illustrative classification problem.

Cite this Paper


BibTeX
@InProceedings{pmlr-v22-dekel12, title = {There's a Hole in My Data Space: Piecewise Predictors for Heterogeneous Learning Problems}, author = {Ofer Dekel and Ohad Shamir}, booktitle = {Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics}, pages = {291--298}, year = {2012}, editor = {Neil D. Lawrence and Mark Girolami}, volume = {22}, series = {Proceedings of Machine Learning Research}, address = {La Palma, Canary Islands}, month = {21--23 Apr}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v22/dekel12/dekel12.pdf}, url = {http://proceedings.mlr.press/v22/dekel12.html}, abstract = {We study statistical learning problems where the data space is multimodal and heterogeneous, and constructing a single global predictor is difficult. We address such problems by iteratively identifying high-error regions in the data space and learning specialized predictors for these regions. While the idea of composing localized predictors is not new, our approach is unique in that we actively seek out predictors that clump errors together, making it easier to isolate the problematic regions. When errors are clumped together they are also easier to interpret and resolve through appropriate feature engineering and data preprocessing. We present an error-clumping classification algorithm based on a convex optimization problem, and an efficient stochastic optimization algorithm for this problem. We theoretically motivate our approach with a novel sample complexity analysis for piecewise predictors, and empirically demonstrate its behavior on an illustrative classification problem.} }
Endnote
%0 Conference Paper %T There’s a Hole in My Data Space: Piecewise Predictors for Heterogeneous Learning Problems %A Ofer Dekel %A Ohad Shamir %B Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2012 %E Neil D. Lawrence %E Mark Girolami %F pmlr-v22-dekel12 %I PMLR %J Proceedings of Machine Learning Research %P 291--298 %U http://proceedings.mlr.press %V 22 %W PMLR %X We study statistical learning problems where the data space is multimodal and heterogeneous, and constructing a single global predictor is difficult. We address such problems by iteratively identifying high-error regions in the data space and learning specialized predictors for these regions. While the idea of composing localized predictors is not new, our approach is unique in that we actively seek out predictors that clump errors together, making it easier to isolate the problematic regions. When errors are clumped together they are also easier to interpret and resolve through appropriate feature engineering and data preprocessing. We present an error-clumping classification algorithm based on a convex optimization problem, and an efficient stochastic optimization algorithm for this problem. We theoretically motivate our approach with a novel sample complexity analysis for piecewise predictors, and empirically demonstrate its behavior on an illustrative classification problem.
RIS
TY - CPAPER TI - There’s a Hole in My Data Space: Piecewise Predictors for Heterogeneous Learning Problems AU - Ofer Dekel AU - Ohad Shamir BT - Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics PY - 2012/03/21 DA - 2012/03/21 ED - Neil D. Lawrence ED - Mark Girolami ID - pmlr-v22-dekel12 PB - PMLR SP - 291 DP - PMLR EP - 298 L1 - http://proceedings.mlr.press/v22/dekel12/dekel12.pdf UR - http://proceedings.mlr.press/v22/dekel12.html AB - We study statistical learning problems where the data space is multimodal and heterogeneous, and constructing a single global predictor is difficult. We address such problems by iteratively identifying high-error regions in the data space and learning specialized predictors for these regions. While the idea of composing localized predictors is not new, our approach is unique in that we actively seek out predictors that clump errors together, making it easier to isolate the problematic regions. When errors are clumped together they are also easier to interpret and resolve through appropriate feature engineering and data preprocessing. We present an error-clumping classification algorithm based on a convex optimization problem, and an efficient stochastic optimization algorithm for this problem. We theoretically motivate our approach with a novel sample complexity analysis for piecewise predictors, and empirically demonstrate its behavior on an illustrative classification problem. ER -
APA
Dekel, O. & Shamir, O.. (2012). There’s a Hole in My Data Space: Piecewise Predictors for Heterogeneous Learning Problems. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, in PMLR 22:291-298

Related Material