Direct Inference of Effect of Treatment (DIET) for a Cookieless World

Shiv Shankar, Ritwik Sinha, Saayan Mitra, Moumita Sinha, Madalina Fiterau
Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, PMLR 206:1869-1887, 2023.

Abstract

Brands use cookies and device identifiers to link different web visits to the same consumer. However, with increasing demands for privacy, these identifiers are about to be phased out, making identity fragmentation a permanent feature of the online world. Assessing treatment effects via randomized experiments (A/B testing) in such a scenario is challenging because identity fragmentation causes a) users to receive hybrid/mixed treatments, and b) hides the causal link between the historical treatments and the outcome. In this work, we address the problem of estimating treatment effects when a lack of identification leads to incomplete knowledge of historical treatments. This is a challenging problem which has not been addressed in literature yet. We develop a new method called DIET, which can adjust for users being exposed to mixed treatments without the entire history of treatments being available. Our method takes inspiration from the Cox model, and uses a proportional outcome approach under which we prove that one can obtain consistent estimates of treatment effects even under identity fragmentation. Our experiments, on one simulated and two real datasets, show that our method leads to up to 20% reduction in error and 25% reduction in bias over the naive estimate.

Cite this Paper

BibTeX
@InProceedings{pmlr-v206-shankar23a, title = {Direct Inference of Effect of Treatment (DIET) for a Cookieless World}, author = {Shankar, Shiv and Sinha, Ritwik and Mitra, Saayan and Sinha, Moumita and Fiterau, Madalina}, booktitle = {Proceedings of The 26th International Conference on Artificial Intelligence and Statistics}, pages = {1869--1887}, year = {2023}, editor = {Ruiz, Francisco and Dy, Jennifer and van de Meent, Jan-Willem}, volume = {206}, series = {Proceedings of Machine Learning Research}, month = {25--27 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v206/shankar23a/shankar23a.pdf}, url = {https://proceedings.mlr.press/v206/shankar23a.html}, abstract = {Brands use cookies and device identifiers to link different web visits to the same consumer. However, with increasing demands for privacy, these identifiers are about to be phased out, making identity fragmentation a permanent feature of the online world. Assessing treatment effects via randomized experiments (A/B testing) in such a scenario is challenging because identity fragmentation causes a) users to receive hybrid/mixed treatments, and b) hides the causal link between the historical treatments and the outcome. In this work, we address the problem of estimating treatment effects when a lack of identification leads to incomplete knowledge of historical treatments. This is a challenging problem which has not been addressed in literature yet. We develop a new method called DIET, which can adjust for users being exposed to mixed treatments without the entire history of treatments being available. Our method takes inspiration from the Cox model, and uses a proportional outcome approach under which we prove that one can obtain consistent estimates of treatment effects even under identity fragmentation. Our experiments, on one simulated and two real datasets, show that our method leads to up to 20% reduction in error and 25% reduction in bias over the naive estimate.} }
Endnote
%0 Conference Paper %T Direct Inference of Effect of Treatment (DIET) for a Cookieless World %A Shiv Shankar %A Ritwik Sinha %A Saayan Mitra %A Moumita Sinha %A Madalina Fiterau %B Proceedings of The 26th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2023 %E Francisco Ruiz %E Jennifer Dy %E Jan-Willem van de Meent %F pmlr-v206-shankar23a %I PMLR %P 1869--1887 %U https://proceedings.mlr.press/v206/shankar23a.html %V 206 %X Brands use cookies and device identifiers to link different web visits to the same consumer. However, with increasing demands for privacy, these identifiers are about to be phased out, making identity fragmentation a permanent feature of the online world. Assessing treatment effects via randomized experiments (A/B testing) in such a scenario is challenging because identity fragmentation causes a) users to receive hybrid/mixed treatments, and b) hides the causal link between the historical treatments and the outcome. In this work, we address the problem of estimating treatment effects when a lack of identification leads to incomplete knowledge of historical treatments. This is a challenging problem which has not been addressed in literature yet. We develop a new method called DIET, which can adjust for users being exposed to mixed treatments without the entire history of treatments being available. Our method takes inspiration from the Cox model, and uses a proportional outcome approach under which we prove that one can obtain consistent estimates of treatment effects even under identity fragmentation. Our experiments, on one simulated and two real datasets, show that our method leads to up to 20% reduction in error and 25% reduction in bias over the naive estimate.
APA
Shankar, S., Sinha, R., Mitra, S., Sinha, M. & Fiterau, M.. (2023). Direct Inference of Effect of Treatment (DIET) for a Cookieless World. Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 206:1869-1887 Available from https://proceedings.mlr.press/v206/shankar23a.html.

Related Material