Identification of Subgroups With Similar Benefits in Off-Policy Policy Evaluation

Ramtin Keramati, Omer Gottesman, Leo Anthony Celi, Finale Doshi-Velez, Emma Brunskill
Proceedings of the Conference on Health, Inference, and Learning, PMLR 174:397-410, 2022.

Abstract

Off-policy policy evaluation methods for sequential decision making can be used to help identify if a proposed decision policy is better than a current baseline policy. However, a new decision policy may be better than a baseline policy for some individuals but not others. This has motivated a push towards personalization and accurate per-state estimates of heterogeneous treatment effects (HTEs). Given the limited data present in many important applications such as health care, individual predictions can come at a cost to accuracy and confidence in such predictions. We develop a method to balance the need for personalization with confident predictions by identifying subgroups where it is possible to confidently estimate the expected difference in a new decision policy relative to a baseline. We propose a novel loss function that accounts for the uncertainty during the subgroup partitioning phase. In experiments, we show that our method can be used to form accurate predictions of HTEs where other methods struggle.

Cite this Paper

BibTeX
@InProceedings{pmlr-v174-keramati22a, title = {Identification of Subgroups With Similar Benefits in Off-Policy Policy Evaluation}, author = {Keramati, Ramtin and Gottesman, Omer and Celi, Leo Anthony and Doshi-Velez, Finale and Brunskill, Emma}, booktitle = {Proceedings of the Conference on Health, Inference, and Learning}, pages = {397--410}, year = {2022}, editor = {Flores, Gerardo and Chen, George H and Pollard, Tom and Ho, Joyce C and Naumann, Tristan}, volume = {174}, series = {Proceedings of Machine Learning Research}, month = {07--08 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v174/keramati22a/keramati22a.pdf}, url = {https://proceedings.mlr.press/v174/keramati22a.html}, abstract = {Off-policy policy evaluation methods for sequential decision making can be used to help identify if a proposed decision policy is better than a current baseline policy. However, a new decision policy may be better than a baseline policy for some individuals but not others. This has motivated a push towards personalization and accurate per-state estimates of heterogeneous treatment effects (HTEs). Given the limited data present in many important applications such as health care, individual predictions can come at a cost to accuracy and confidence in such predictions. We develop a method to balance the need for personalization with confident predictions by identifying subgroups where it is possible to confidently estimate the expected difference in a new decision policy relative to a baseline. We propose a novel loss function that accounts for the uncertainty during the subgroup partitioning phase. In experiments, we show that our method can be used to form accurate predictions of HTEs where other methods struggle.} }
Endnote
%0 Conference Paper %T Identification of Subgroups With Similar Benefits in Off-Policy Policy Evaluation %A Ramtin Keramati %A Omer Gottesman %A Leo Anthony Celi %A Finale Doshi-Velez %A Emma Brunskill %B Proceedings of the Conference on Health, Inference, and Learning %C Proceedings of Machine Learning Research %D 2022 %E Gerardo Flores %E George H Chen %E Tom Pollard %E Joyce C Ho %E Tristan Naumann %F pmlr-v174-keramati22a %I PMLR %P 397--410 %U https://proceedings.mlr.press/v174/keramati22a.html %V 174 %X Off-policy policy evaluation methods for sequential decision making can be used to help identify if a proposed decision policy is better than a current baseline policy. However, a new decision policy may be better than a baseline policy for some individuals but not others. This has motivated a push towards personalization and accurate per-state estimates of heterogeneous treatment effects (HTEs). Given the limited data present in many important applications such as health care, individual predictions can come at a cost to accuracy and confidence in such predictions. We develop a method to balance the need for personalization with confident predictions by identifying subgroups where it is possible to confidently estimate the expected difference in a new decision policy relative to a baseline. We propose a novel loss function that accounts for the uncertainty during the subgroup partitioning phase. In experiments, we show that our method can be used to form accurate predictions of HTEs where other methods struggle.
APA
Keramati, R., Gottesman, O., Celi, L.A., Doshi-Velez, F. & Brunskill, E.. (2022). Identification of Subgroups With Similar Benefits in Off-Policy Policy Evaluation. Proceedings of the Conference on Health, Inference, and Learning, in Proceedings of Machine Learning Research 174:397-410 Available from https://proceedings.mlr.press/v174/keramati22a.html.

Related Material