RSA-CP: Efficient Conformal Prediction in Small-Sample Regimes via Random Score Alignment

Pankaj Bhagwat, Zhixian Yang, Yihao Wang, Bei Jiang, Linglong Kong
Proceedings of the 43rd International Conference on Machine Learning, PMLR 306:7856-7883, 2026.

Abstract

Conformal Prediction (CP) provides rigorous finite-sample coverage guarantees, yet its statistical efficiency hinges critically on the size of the calibration set. In data-scarce regimes, CP often suffers from volatile quantile estimation, leading to overly conservative and wide prediction intervals. To address this, we propose Random Score Alignment-Conformal Prediction (RSA-CP), a simple framework designed to improve sample efficiency in small-sample CP. Instead of requiring the computationally intensive generation of full synthetic datasets, RSA-CP enhances calibration by directly aligning real scores with a high-resolution reference score distribution. By employing an optimal transport mapping, our framework refines "step-like" quantile increments through a globally optimal use of reference information. We provide theoretical guarantees establishing that RSA-CP maintains robust coverage without any distributional assumptions on the reference scores. Empirical evaluations demonstrate that RSA-CP consistently produces shorter and more precise prediction intervals while maintaining finite-sample coverage guarantees. Overall, RSA-CP offers a computationally efficient and theoretically grounded solution for robust uncertainty quantification under limited data.

Cite this Paper

BibTeX
@InProceedings{pmlr-v306-bhagwat26a, title = {{RSA}-{CP}: Efficient Conformal Prediction in Small-Sample Regimes via Random Score Alignment}, author = {Bhagwat, Pankaj and Yang, Zhixian and Wang, Yihao and Jiang, Bei and Kong, Linglong}, booktitle = {Proceedings of the 43rd International Conference on Machine Learning}, pages = {7856--7883}, year = {2026}, editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto}, volume = {306}, series = {Proceedings of Machine Learning Research}, month = {06--11 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/bhagwat26a/bhagwat26a.pdf}, url = {https://proceedings.mlr.press/v306/bhagwat26a.html}, abstract = {Conformal Prediction (CP) provides rigorous finite-sample coverage guarantees, yet its statistical efficiency hinges critically on the size of the calibration set. In data-scarce regimes, CP often suffers from volatile quantile estimation, leading to overly conservative and wide prediction intervals. To address this, we propose Random Score Alignment-Conformal Prediction (RSA-CP), a simple framework designed to improve sample efficiency in small-sample CP. Instead of requiring the computationally intensive generation of full synthetic datasets, RSA-CP enhances calibration by directly aligning real scores with a high-resolution reference score distribution. By employing an optimal transport mapping, our framework refines "step-like" quantile increments through a globally optimal use of reference information. We provide theoretical guarantees establishing that RSA-CP maintains robust coverage without any distributional assumptions on the reference scores. Empirical evaluations demonstrate that RSA-CP consistently produces shorter and more precise prediction intervals while maintaining finite-sample coverage guarantees. Overall, RSA-CP offers a computationally efficient and theoretically grounded solution for robust uncertainty quantification under limited data.} }
Endnote
%0 Conference Paper %T RSA-CP: Efficient Conformal Prediction in Small-Sample Regimes via Random Score Alignment %A Pankaj Bhagwat %A Zhixian Yang %A Yihao Wang %A Bei Jiang %A Linglong Kong %B Proceedings of the 43rd International Conference on Machine Learning %C Proceedings of Machine Learning Research %D 2026 %E Tong Zhang %E Miroslav Dudik %E Martin Jaggi %E Alekh Agarwal %E Sharon Li %E Dale Schuurmans %E Jerry Zhu %E Felix Berkenkamp %E Hanze Dong %E Alberto Bietti %F pmlr-v306-bhagwat26a %I PMLR %P 7856--7883 %U https://proceedings.mlr.press/v306/bhagwat26a.html %V 306 %X Conformal Prediction (CP) provides rigorous finite-sample coverage guarantees, yet its statistical efficiency hinges critically on the size of the calibration set. In data-scarce regimes, CP often suffers from volatile quantile estimation, leading to overly conservative and wide prediction intervals. To address this, we propose Random Score Alignment-Conformal Prediction (RSA-CP), a simple framework designed to improve sample efficiency in small-sample CP. Instead of requiring the computationally intensive generation of full synthetic datasets, RSA-CP enhances calibration by directly aligning real scores with a high-resolution reference score distribution. By employing an optimal transport mapping, our framework refines "step-like" quantile increments through a globally optimal use of reference information. We provide theoretical guarantees establishing that RSA-CP maintains robust coverage without any distributional assumptions on the reference scores. Empirical evaluations demonstrate that RSA-CP consistently produces shorter and more precise prediction intervals while maintaining finite-sample coverage guarantees. Overall, RSA-CP offers a computationally efficient and theoretically grounded solution for robust uncertainty quantification under limited data.
APA
Bhagwat, P., Yang, Z., Wang, Y., Jiang, B. & Kong, L.. (2026). RSA-CP: Efficient Conformal Prediction in Small-Sample Regimes via Random Score Alignment. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:7856-7883 Available from https://proceedings.mlr.press/v306/bhagwat26a.html.

Related Material