Safe Sequential Testing and Effect Estimation in Stratified Count Data

Rosanne Turner, Peter Grunwald
Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR 206:4880-4893, 2023.

Abstract

Sequential decision making significantly speeds up research and is more cost-effective compared to fixed-n methods. We present a method for sequential decision making for stratified count data that retains Type-I error guarantee or false discovery rate under optional stopping, using e-variables. We invert the method to construct stratified anytime-valid confidence sequences, where cross-talk between subpopulations in the data can be allowed during data collection to improve power. Finally, we combine information collected in separate subpopulations through pseudo-Bayesian averaging and switching to create effective estimates for the minimal, mean and maximal treatment effects in the subpopulations.

Cite this Paper

BibTeX
@InProceedings{pmlr-v206-turner23a, title = {Safe Sequential Testing and Effect Estimation in Stratified Count Data}, author = {Turner, Rosanne and Grunwald, Peter}, booktitle = {Proceedings of The 26th International Conference on Artificial Intelligence and Statistics}, pages = {4880--4893}, year = {2023}, editor = {Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem}, volume = {206}, series = {Proceedings of Machine Learning Research}, month = {25--27 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v206/turner23a/turner23a.pdf}, url = {https://proceedings.mlr.press/v206/turner23a.html}, abstract = {Sequential decision making significantly speeds up research and is more cost-effective compared to fixed-n methods. We present a method for sequential decision making for stratified count data that retains Type-I error guarantee or false discovery rate under optional stopping, using e-variables. We invert the method to construct stratified anytime-valid confidence sequences, where cross-talk between subpopulations in the data can be allowed during data collection to improve power. Finally, we combine information collected in separate subpopulations through pseudo-Bayesian averaging and switching to create effective estimates for the minimal, mean and maximal treatment effects in the subpopulations.} }
Endnote
%0 Conference Paper %T Safe Sequential Testing and Effect Estimation in Stratified Count Data %A Rosanne Turner %A Peter Grunwald %B Proceedings of The 26th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2023 %E Francisco Ruiz %E Jennifer Dy %E Jan-Willem van de Meent %F pmlr-v206-turner23a %I PMLR %P 4880--4893 %U https://proceedings.mlr.press/v206/turner23a.html %V 206 %X Sequential decision making significantly speeds up research and is more cost-effective compared to fixed-n methods. We present a method for sequential decision making for stratified count data that retains Type-I error guarantee or false discovery rate under optional stopping, using e-variables. We invert the method to construct stratified anytime-valid confidence sequences, where cross-talk between subpopulations in the data can be allowed during data collection to improve power. Finally, we combine information collected in separate subpopulations through pseudo-Bayesian averaging and switching to create effective estimates for the minimal, mean and maximal treatment effects in the subpopulations.
APA
Turner, R. & Grunwald, P.. (2023). Safe Sequential Testing and Effect Estimation in Stratified Count Data. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 206:4880-4893 Available from https://proceedings.mlr.press/v206/turner23a.html.

Related Material