Unbiased Objective Estimation in Predictive Optimization

Shinji Ito, Akihiro Yabe, Ryohei Fujimaki
Proceedings of the 35th International Conference on Machine Learning, PMLR 80:2176-2185, 2018.

Abstract

For data-driven decision-making, one promising approach, called predictive optimization, is to solve maximization problems i n which the objective function to be maximized is estimated from data. Predictive optimization, however, suffers from the problem of a calculated optimal solution’s being evaluated too optimistically, i.e., the value of the objective function is overestimated. This paper investigates such optimistic bias and presents two methods for correcting it. The first, which is analogous to cross-validation, successfully corrects the optimistic bias but results in underestimation of the true value. Our second method employs resampling techniques to avoid both overestimation and underestimation. We show that the second method, referred to as the parameter perturbation method, achieves asymptotically unbiased estimation. Empirical results for both artificial and real-world datasets demonstrate that our proposed approach successfully corrects the optimistic bias.

Cite this Paper

BibTeX
@InProceedings{pmlr-v80-ito18a, title = {Unbiased Objective Estimation in Predictive Optimization}, author = {Ito, Shinji and Yabe, Akihiro and Fujimaki, Ryohei}, booktitle = {Proceedings of the 35th International Conference on Machine Learning}, pages = {2176--2185}, year = {2018}, editor = {Dy, Jennifer and Krause, Andreas}, volume = {80}, series = {Proceedings of Machine Learning Research}, month = {10--15 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v80/ito18a/ito18a.pdf}, url = {https://proceedings.mlr.press/v80/ito18a.html}, abstract = {For data-driven decision-making, one promising approach, called predictive optimization, is to solve maximization problems i n which the objective function to be maximized is estimated from data. Predictive optimization, however, suffers from the problem of a calculated optimal solution’s being evaluated too optimistically, i.e., the value of the objective function is overestimated. This paper investigates such optimistic bias and presents two methods for correcting it. The first, which is analogous to cross-validation, successfully corrects the optimistic bias but results in underestimation of the true value. Our second method employs resampling techniques to avoid both overestimation and underestimation. We show that the second method, referred to as the parameter perturbation method, achieves asymptotically unbiased estimation. Empirical results for both artificial and real-world datasets demonstrate that our proposed approach successfully corrects the optimistic bias.} }
Endnote
%0 Conference Paper %T Unbiased Objective Estimation in Predictive Optimization %A Shinji Ito %A Akihiro Yabe %A Ryohei Fujimaki %B Proceedings of the 35th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2018 %E Jennifer Dy %E Andreas Krause %F pmlr-v80-ito18a %I PMLR %P 2176--2185 %U https://proceedings.mlr.press/v80/ito18a.html %V 80 %X For data-driven decision-making, one promising approach, called predictive optimization, is to solve maximization problems i n which the objective function to be maximized is estimated from data. Predictive optimization, however, suffers from the problem of a calculated optimal solution’s being evaluated too optimistically, i.e., the value of the objective function is overestimated. This paper investigates such optimistic bias and presents two methods for correcting it. The first, which is analogous to cross-validation, successfully corrects the optimistic bias but results in underestimation of the true value. Our second method employs resampling techniques to avoid both overestimation and underestimation. We show that the second method, referred to as the parameter perturbation method, achieves asymptotically unbiased estimation. Empirical results for both artificial and real-world datasets demonstrate that our proposed approach successfully corrects the optimistic bias.
APA
Ito, S., Yabe, A. & Fujimaki, R.. (2018). Unbiased Objective Estimation in Predictive Optimization. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research 80:2176-2185 Available from https://proceedings.mlr.press/v80/ito18a.html.

Related Material