A/B testing under Interference with Partial Network Information

Shiv Shankar, Ritwik Sinha, Yash Chandak, Saayan Mitra, Madalina Fiterau
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, PMLR 238:19-27, 2024.

Abstract

A/B tests are often required to be conducted on subjects that might have social connections. For e.g., experiments on social media, or medical and social interventions to control the spread of an epidemic. In such settings, the SUTVA assumption for randomized-controlled trials is violated due to network interference, or spill-over effects, as treatments to group A can potentially also affect the control group B. When the underlying social network is known exactly, prior works have demonstrated how to conduct A/B tests adequately to estimate the global average treatment effect (GATE). However, in practice, it is often impossible to obtain knowledge about the exact underlying network. In this paper, we present UNITE: a novel estimator that relax this assumption and can identify GATE while only relying on knowledge of the superset of neighbors for any subject in the graph. Through theoretical analysis and extensive experiments, we show that the proposed approach performs better in comparison to standard estimators.

Cite this Paper

BibTeX
@InProceedings{pmlr-v238-shankar24a, title = { {A}/{B} testing under Interference with Partial Network Information }, author = {Shankar, Shiv and Sinha, Ritwik and Chandak, Yash and Mitra, Saayan and Fiterau, Madalina}, booktitle = {Proceedings of The 27th International Conference on Artificial Intelligence and Statistics}, pages = {19--27}, year = {2024}, editor = {Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen}, volume = {238}, series = {Proceedings of Machine Learning Research}, month = {02--04 May}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v238/shankar24a/shankar24a.pdf}, url = {https://proceedings.mlr.press/v238/shankar24a.html}, abstract = { A/B tests are often required to be conducted on subjects that might have social connections. For e.g., experiments on social media, or medical and social interventions to control the spread of an epidemic. In such settings, the SUTVA assumption for randomized-controlled trials is violated due to network interference, or spill-over effects, as treatments to group A can potentially also affect the control group B. When the underlying social network is known exactly, prior works have demonstrated how to conduct A/B tests adequately to estimate the global average treatment effect (GATE). However, in practice, it is often impossible to obtain knowledge about the exact underlying network. In this paper, we present UNITE: a novel estimator that relax this assumption and can identify GATE while only relying on knowledge of the superset of neighbors for any subject in the graph. Through theoretical analysis and extensive experiments, we show that the proposed approach performs better in comparison to standard estimators. } }
Endnote
%0 Conference Paper %T A/B testing under Interference with Partial Network Information %A Shiv Shankar %A Ritwik Sinha %A Yash Chandak %A Saayan Mitra %A Madalina Fiterau %B Proceedings of The 27th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2024 %E Sanjoy Dasgupta %E Stephan Mandt %E Yingzhen Li %F pmlr-v238-shankar24a %I PMLR %P 19--27 %U https://proceedings.mlr.press/v238/shankar24a.html %V 238 %X A/B tests are often required to be conducted on subjects that might have social connections. For e.g., experiments on social media, or medical and social interventions to control the spread of an epidemic. In such settings, the SUTVA assumption for randomized-controlled trials is violated due to network interference, or spill-over effects, as treatments to group A can potentially also affect the control group B. When the underlying social network is known exactly, prior works have demonstrated how to conduct A/B tests adequately to estimate the global average treatment effect (GATE). However, in practice, it is often impossible to obtain knowledge about the exact underlying network. In this paper, we present UNITE: a novel estimator that relax this assumption and can identify GATE while only relying on knowledge of the superset of neighbors for any subject in the graph. Through theoretical analysis and extensive experiments, we show that the proposed approach performs better in comparison to standard estimators.
APA
Shankar, S., Sinha, R., Chandak, Y., Mitra, S. & Fiterau, M.. (2024). A/B testing under Interference with Partial Network Information . Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 238:19-27 Available from https://proceedings.mlr.press/v238/shankar24a.html.

Related Material