General Synthetic-Powered Inference

Meshi Bashari, Yonghoon Lee, Roy Maor Lotan, Edgar Dobriban, Yaniv Romano
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:6862-6907, 2026.

Abstract

The rapid proliferation of high-quality synthetic data—generated by advanced AI models or collected as auxiliary data from related tasks—presents both opportunities and challenges for statistical inference. This paper introduces a GEneral Synthetic-Powered Inference (GESPI) framework that wraps around any statistical inference procedure to safely enhance sample efficiency by combining synthetic and real data. Our framework leverages high-quality synthetic data to boost statistical power, yet adaptively defaults to the standard method using only real data when synthetic data are of low quality. The error rate of our method remains below a user-specified bound without any distributional assumptions on the synthetic data, and decreases as the quality of the synthetic data improves. This flexibility enables seamless integration with conformal prediction, risk control, hypothesis testing, and multiple testing procedures, all without modifying the base inference method. We demonstrate the benefits of our method on challenging tasks with limited labeled data, including AlphaFold protein structure prediction, and comparing large reasoning models on complex math problems.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-bashari26a, title = {General Synthetic-Powered Inference}, author = {Bashari, Meshi and Lee, Yonghoon and Lotan, Roy Maor and Dobriban, Edgar and Romano, Yaniv}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {6862--6907}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/bashari26a/bashari26a.pdf}, url = {https://proceedings.mlr.press/v306/bashari26a.html}, abstract = {The rapid proliferation of high-quality synthetic data—generated by advanced AI models or collected as auxiliary data from related tasks—presents both opportunities and challenges for statistical inference. This paper introduces a GEneral Synthetic-Powered Inference (GESPI) framework that wraps around any statistical inference procedure to safely enhance sample efficiency by combining synthetic and real data. Our framework leverages high-quality synthetic data to boost statistical power, yet adaptively defaults to the standard method using only real data when synthetic data are of low quality. The error rate of our method remains below a user-specified bound without any distributional assumptions on the synthetic data, and decreases as the quality of the synthetic data improves. This flexibility enables seamless integration with conformal prediction, risk control, hypothesis testing, and multiple testing procedures, all without modifying the base inference method. We demonstrate the benefits of our method on challenging tasks with limited labeled data, including AlphaFold protein structure prediction, and comparing large reasoning models on complex math problems.} }
Endnote
%0 Conference Paper %T General Synthetic-Powered Inference %A Meshi Bashari %A Yonghoon Lee %A Roy Maor Lotan %A Edgar Dobriban %A Yaniv Romano %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-bashari26a %I PMLR %P 6862--6907 %U https://proceedings.mlr.press/v306/bashari26a.html %V 306 %X The rapid proliferation of high-quality synthetic data—generated by advanced AI models or collected as auxiliary data from related tasks—presents both opportunities and challenges for statistical inference. This paper introduces a GEneral Synthetic-Powered Inference (GESPI) framework that wraps around any statistical inference procedure to safely enhance sample efficiency by combining synthetic and real data. Our framework leverages high-quality synthetic data to boost statistical power, yet adaptively defaults to the standard method using only real data when synthetic data are of low quality. The error rate of our method remains below a user-specified bound without any distributional assumptions on the synthetic data, and decreases as the quality of the synthetic data improves. This flexibility enables seamless integration with conformal prediction, risk control, hypothesis testing, and multiple testing procedures, all without modifying the base inference method. We demonstrate the benefits of our method on challenging tasks with limited labeled data, including AlphaFold protein structure prediction, and comparing large reasoning models on complex math problems.
APA
Bashari, M., Lee, Y., Lotan, R.M., Dobriban, E. & Romano, Y.. (2026). General Synthetic-Powered Inference. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:6862-6907 Available from https://proceedings.mlr.press/v306/bashari26a.html.

Related Material