On Validation and Planning of An Optimal Decision Rule with Application in Healthcare Studies

Hengrui Cai, Wenbin Lu, Rui Song
Proceedings of the 37th International Conference on Machine Learning, PMLR 119:1262-1270, 2020.

Abstract

In the current era of personalized recommendation, one major interest is to develop an optimal individualized decision rule that assigns individuals with the best treatment option according to their covariates. Estimation of optimal decision rules (ODR) has been extensively investigated recently, however, at present, no testing procedure is proposed to verify whether these ODRs are significantly better than the naive decision rule that always assigning individuals to a fixed treatment option. In this paper, we propose a testing procedure for detecting the existence of an ODR that is better than the naive decision rule under the randomized trials. We construct the proposed test based on the difference of estimated value functions using the augmented inverse probability weighted method. The asymptotic distributions of the proposed test statistic under the null and local alternative hypotheses are established. Based on the established asymptotic distributions, we further develop a sample size calculation formula for testing the existence of an ODR in designing A/B tests. Extensive simulations and a real data application to a schizophrenia clinical trial data are conducted to demonstrate the empirical validity of the proposed methods.

Cite this Paper

BibTeX
@InProceedings{pmlr-v119-cai20b, title = {On Validation and Planning of An Optimal Decision Rule with Application in Healthcare Studies}, author = {Cai, Hengrui and Lu, Wenbin and Song, Rui}, booktitle = {Proceedings of the 37th International Conference on Machine Learning}, pages = {1262--1270}, year = {2020}, editor = {III, Hal Daumé and Singh, Aarti}, volume = {119}, series = {Proceedings of Machine Learning Research}, month = {13--18 Jul}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v119/cai20b/cai20b.pdf}, url = {https://proceedings.mlr.press/v119/cai20b.html}, abstract = {In the current era of personalized recommendation, one major interest is to develop an optimal individualized decision rule that assigns individuals with the best treatment option according to their covariates. Estimation of optimal decision rules (ODR) has been extensively investigated recently, however, at present, no testing procedure is proposed to verify whether these ODRs are significantly better than the naive decision rule that always assigning individuals to a fixed treatment option. In this paper, we propose a testing procedure for detecting the existence of an ODR that is better than the naive decision rule under the randomized trials. We construct the proposed test based on the difference of estimated value functions using the augmented inverse probability weighted method. The asymptotic distributions of the proposed test statistic under the null and local alternative hypotheses are established. Based on the established asymptotic distributions, we further develop a sample size calculation formula for testing the existence of an ODR in designing A/B tests. Extensive simulations and a real data application to a schizophrenia clinical trial data are conducted to demonstrate the empirical validity of the proposed methods.} }
Endnote
%0 Conference Paper %T On Validation and Planning of An Optimal Decision Rule with Application in Healthcare Studies %A Hengrui Cai %A Wenbin Lu %A Rui Song %B Proceedings of the 37th International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2020 %E Hal Daumé III %E Aarti Singh %F pmlr-v119-cai20b %I PMLR %P 1262--1270 %U https://proceedings.mlr.press/v119/cai20b.html %V 119 %X In the current era of personalized recommendation, one major interest is to develop an optimal individualized decision rule that assigns individuals with the best treatment option according to their covariates. Estimation of optimal decision rules (ODR) has been extensively investigated recently, however, at present, no testing procedure is proposed to verify whether these ODRs are significantly better than the naive decision rule that always assigning individuals to a fixed treatment option. In this paper, we propose a testing procedure for detecting the existence of an ODR that is better than the naive decision rule under the randomized trials. We construct the proposed test based on the difference of estimated value functions using the augmented inverse probability weighted method. The asymptotic distributions of the proposed test statistic under the null and local alternative hypotheses are established. Based on the established asymptotic distributions, we further develop a sample size calculation formula for testing the existence of an ODR in designing A/B tests. Extensive simulations and a real data application to a schizophrenia clinical trial data are conducted to demonstrate the empirical validity of the proposed methods.
APA
Cai, H., Lu, W. & Song, R.. (2020). On Validation and Planning of An Optimal Decision Rule with Application in Healthcare Studies. Proceedings of the 37th International Conference on Machine Learning, in Proceedings of Machine Learning Research 119:1262-1270 Available from https://proceedings.mlr.press/v119/cai20b.html.

Related Material