FAB-PPI: Frequentist, Assisted by Bayes, Prediction-Powered Inference

Stefano Cortinovis, Francois Caron
Proceedings of the 42nd International Conference on Machine Learning, PMLR 267:11328-11356, 2025.

Abstract

Prediction-powered inference (PPI) enables valid statistical inference by combining experimental data with machine learning predictions. When a sufficient number of high-quality predictions is available, PPI results in more accurate estimates and tighter confidence intervals than traditional methods. In this paper, we propose to inform the PPI framework with prior knowledge on the quality of the predictions. The resulting method, which we call frequentist, assisted by Bayes, PPI (FAB-PPI), improves over PPI when the observed prediction quality is likely under the prior, while maintaining its frequentist guarantees. Furthermore, when using heavy-tailed priors, FAB-PPI adaptively reverts to standard PPI in low prior probability regions. We demonstrate the benefits of FAB-PPI in real and synthetic examples.

Cite this Paper

BibTeX
@InProceedings{pmlr-v267-cortinovis25a, title = {{FAB}-{PPI}: Frequentist, Assisted by Bayes, Prediction-Powered Inference}, author = {Cortinovis, Stefano and Caron, Francois}, booktitle = {Proceedings of the 42nd International Conference on Machine Learning}, pages = {11328--11356}, year = {2025}, editor = {Singh, Aarti and Fazel, Maryam and Hsu, Daniel and Lacoste-Julien, Simon and Berkenkamp, Felix and Maharaj, Tegan and Wagstaff, Kiri and Zhu, Jerry}, volume = {267}, series = {Proceedings of Machine Learning Research}, month = {13--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v267/main/assets/cortinovis25a/cortinovis25a.pdf}, url = {https://proceedings.mlr.press/v267/cortinovis25a.html}, abstract = {Prediction-powered inference (PPI) enables valid statistical inference by combining experimental data with machine learning predictions. When a sufficient number of high-quality predictions is available, PPI results in more accurate estimates and tighter confidence intervals than traditional methods. In this paper, we propose to inform the PPI framework with prior knowledge on the quality of the predictions. The resulting method, which we call frequentist, assisted by Bayes, PPI (FAB-PPI), improves over PPI when the observed prediction quality is likely under the prior, while maintaining its frequentist guarantees. Furthermore, when using heavy-tailed priors, FAB-PPI adaptively reverts to standard PPI in low prior probability regions. We demonstrate the benefits of FAB-PPI in real and synthetic examples.} }
Endnote
%0 Conference Paper %T FAB-PPI: Frequentist, Assisted by Bayes, Prediction-Powered Inference %A Stefano Cortinovis %A Francois Caron %B Proceedings of the 42nd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2025 %E Aarti Singh %E Maryam Fazel %E Daniel Hsu %E Simon Lacoste-Julien %E Felix Berkenkamp %E Tegan Maharaj %E Kiri Wagstaff %E Jerry Zhu %F pmlr-v267-cortinovis25a %I PMLR %P 11328--11356 %U https://proceedings.mlr.press/v267/cortinovis25a.html %V 267 %X Prediction-powered inference (PPI) enables valid statistical inference by combining experimental data with machine learning predictions. When a sufficient number of high-quality predictions is available, PPI results in more accurate estimates and tighter confidence intervals than traditional methods. In this paper, we propose to inform the PPI framework with prior knowledge on the quality of the predictions. The resulting method, which we call frequentist, assisted by Bayes, PPI (FAB-PPI), improves over PPI when the observed prediction quality is likely under the prior, while maintaining its frequentist guarantees. Furthermore, when using heavy-tailed priors, FAB-PPI adaptively reverts to standard PPI in low prior probability regions. We demonstrate the benefits of FAB-PPI in real and synthetic examples.
APA
Cortinovis, S. & Caron, F.. (2025). FAB-PPI: Frequentist, Assisted by Bayes, Prediction-Powered Inference. Proceedings of the 42nd International Conference on Machine Learning, in Proceedings of Machine Learning Research 267:11328-11356 Available from https://proceedings.mlr.press/v267/cortinovis25a.html.

Related Material