Safe Exploration for Efficient Policy Evaluation and Comparison

Runzhe Wan, Branislav Kveton, Rui Song
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:22491-22511, 2022.

Abstract

High-quality data plays a central role in ensuring the accuracy of policy evaluation. This paper initiates the study of efficient and safe data collection for bandit policy evaluation. We formulate the problem and investigate its several representative variants. For each variant, we analyze its statistical properties, derive the corresponding exploration policy, and design an efficient algorithm for computing it. Both theoretical analysis and experiments support the usefulness of the proposed methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v162-wan22b, title = {Safe Exploration for Efficient Policy Evaluation and Comparison}, author = {Wan, Runzhe and Kveton, Branislav and Song, Rui}, booktitle = {Proceedings of the 39th International Conference on Machine Learning}, pages = {22491--22511}, year = {2022}, editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}, volume = {162}, series = {Proceedings of Machine Learning Research}, month = {17--23 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v162/wan22b/wan22b.pdf}, url = {https://proceedings.mlr.press/v162/wan22b.html}, abstract = {High-quality data plays a central role in ensuring the accuracy of policy evaluation. This paper initiates the study of efficient and safe data collection for bandit policy evaluation. We formulate the problem and investigate its several representative variants. For each variant, we analyze its statistical properties, derive the corresponding exploration policy, and design an efficient algorithm for computing it. Both theoretical analysis and experiments support the usefulness of the proposed methods.} }
Endnote
%0 Conference Paper %T Safe Exploration for Efficient Policy Evaluation and Comparison %A Runzhe Wan %A Branislav Kveton %A Rui Song %B Proceedings of the 39th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2022 %E Kamalika Chaudhuri %E Stefanie Jegelka %E Le Song %E Csaba Szepesvari %E Gang Niu %E Sivan Sabato %F pmlr-v162-wan22b %I PMLR %P 22491--22511 %U https://proceedings.mlr.press/v162/wan22b.html %V 162 %X High-quality data plays a central role in ensuring the accuracy of policy evaluation. This paper initiates the study of efficient and safe data collection for bandit policy evaluation. We formulate the problem and investigate its several representative variants. For each variant, we analyze its statistical properties, derive the corresponding exploration policy, and design an efficient algorithm for computing it. Both theoretical analysis and experiments support the usefulness of the proposed methods.
APA
Wan, R., Kveton, B. & Song, R.. (2022). Safe Exploration for Efficient Policy Evaluation and Comparison. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:22491-22511 Available from https://proceedings.mlr.press/v162/wan22b.html.

Related Material