Estimating the Causal Effect of Early ArXiving on Paper Acceptance

Yanai Elazar, Jiayao Zhang, David Wadden, Bo Zhang, Noah A. Smith
Proceedings of the Third Conference on Causal Learning and Reasoning, PMLR 236:913-933, 2024.

Abstract

What is the effect of releasing a preprint of a paper before it is submitted for peer review? No randomized controlled trial has been conducted, so we turn to observational data to answer this question. We use data from the ICLR conference (2018–2022) and apply methods from causal inference to estimate the effect of arXiving a paper before the reviewing period (early arXiving) on its acceptance to the conference. Adjusting for confounders such as topic, authors, and quality, we may estimate the causal effect. However, since quality is a challenging construct to estimate, we use the negative outcome control method, using paper citation count as a control variable to debias the quality confounding effect. Our results suggest that early arXiving may have a small effect on a paper’s chances of acceptance. However, this effect (when existing) does not differ significantly across different groups of authors, as grouped by author citation count and institute rank. This suggests that early arXiving does not provide an advantage to any particular group.

Cite this Paper

BibTeX
@InProceedings{pmlr-v236-elazar24a, title = {Estimating the Causal Effect of Early ArXiving on Paper Acceptance}, author = {Elazar, Yanai and Zhang, Jiayao and Wadden, David and Zhang, Bo and Smith, Noah A.}, booktitle = {Proceedings of the Third Conference on Causal Learning and Reasoning}, pages = {913--933}, year = {2024}, editor = {Locatello, Francesco and Didelez, Vanessa}, volume = {236}, series = {Proceedings of Machine Learning Research}, month = {01--03 Apr}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v236/elazar24a/elazar24a.pdf}, url = {https://proceedings.mlr.press/v236/elazar24a.html}, abstract = {What is the effect of releasing a preprint of a paper before it is submitted for peer review? No randomized controlled trial has been conducted, so we turn to observational data to answer this question. We use data from the ICLR conference (2018–2022) and apply methods from causal inference to estimate the effect of arXiving a paper before the reviewing period (early arXiving) on its acceptance to the conference. Adjusting for confounders such as topic, authors, and quality, we may estimate the causal effect. However, since quality is a challenging construct to estimate, we use the negative outcome control method, using paper citation count as a control variable to debias the quality confounding effect. Our results suggest that early arXiving may have a small effect on a paper’s chances of acceptance. However, this effect (when existing) does not differ significantly across different groups of authors, as grouped by author citation count and institute rank. This suggests that early arXiving does not provide an advantage to any particular group.} }
Endnote
%0 Conference Paper %T Estimating the Causal Effect of Early ArXiving on Paper Acceptance %A Yanai Elazar %A Jiayao Zhang %A David Wadden %A Bo Zhang %A Noah A. Smith %B Proceedings of the Third Conference on Causal Learning and Reasoning %C Proceedings of Machine Learning Research %D 2024 %E Francesco Locatello %E Vanessa Didelez %F pmlr-v236-elazar24a %I PMLR %P 913--933 %U https://proceedings.mlr.press/v236/elazar24a.html %V 236 %X What is the effect of releasing a preprint of a paper before it is submitted for peer review? No randomized controlled trial has been conducted, so we turn to observational data to answer this question. We use data from the ICLR conference (2018–2022) and apply methods from causal inference to estimate the effect of arXiving a paper before the reviewing period (early arXiving) on its acceptance to the conference. Adjusting for confounders such as topic, authors, and quality, we may estimate the causal effect. However, since quality is a challenging construct to estimate, we use the negative outcome control method, using paper citation count as a control variable to debias the quality confounding effect. Our results suggest that early arXiving may have a small effect on a paper’s chances of acceptance. However, this effect (when existing) does not differ significantly across different groups of authors, as grouped by author citation count and institute rank. This suggests that early arXiving does not provide an advantage to any particular group.
APA
Elazar, Y., Zhang, J., Wadden, D., Zhang, B. & Smith, N.A.. (2024). Estimating the Causal Effect of Early ArXiving on Paper Acceptance. Proceedings of the Third Conference on Causal Learning and Reasoning, in Proceedings of Machine Learning Research 236:913-933 Available from https://proceedings.mlr.press/v236/elazar24a.html.

Related Material