Conformalized Deep Splines for Optimal and Efficient Prediction Sets

Nathaniel Diamant, Ehsan Hajiramezanali, Tommaso Biancalani, Gabriele Scalia
Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, PMLR 238:1657-1665, 2024.

Abstract

Uncertainty estimation is critical in high-stakes machine learning applications. One effective way to estimate uncertainty is conformal prediction, which can provide predictive inference with statistical coverage guarantees. We present a new conformal regression method, Spline Prediction Intervals via Conformal Estimation (SPICE), that estimates the conditional density using neural- network-parameterized splines. We prove universal approximation and optimality results for SPICE, which are empirically reflected by our experiments. SPICE is compatible with two different efficient-to- compute conformal scores, one designed for size-efficient marginal coverage (SPICE-ND) and the other for size-efficient conditional coverage (SPICE-HPD). Results on benchmark datasets demonstrate SPICE-ND models achieve the smallest average prediction set sizes, including average size reductions of nearly 50% for some datasets compared to the next best baseline. SPICE-HPD models achieve the best conditional coverage compared to baselines. The SPICE implementation is made available.

Cite this Paper

BibTeX
@InProceedings{pmlr-v238-diamant24a, title = {Conformalized Deep Splines for Optimal and Efficient Prediction Sets}, author = {Diamant, Nathaniel and Hajiramezanali, Ehsan and Biancalani, Tommaso and Scalia, Gabriele}, booktitle = {Proceedings of The 27th International Conference on Artificial Intelligence and Statistics}, pages = {1657--1665}, year = {2024}, editor = {Dasgupta, Sanjoy and Mandt, Stephan and Li, Yingzhen}, volume = {238}, series = {Proceedings of Machine Learning Research}, month = {02--04 May}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v238/diamant24a/diamant24a.pdf}, url = {https://proceedings.mlr.press/v238/diamant24a.html}, abstract = {Uncertainty estimation is critical in high-stakes machine learning applications. One effective way to estimate uncertainty is conformal prediction, which can provide predictive inference with statistical coverage guarantees. We present a new conformal regression method, Spline Prediction Intervals via Conformal Estimation (SPICE), that estimates the conditional density using neural- network-parameterized splines. We prove universal approximation and optimality results for SPICE, which are empirically reflected by our experiments. SPICE is compatible with two different efficient-to- compute conformal scores, one designed for size-efficient marginal coverage (SPICE-ND) and the other for size-efficient conditional coverage (SPICE-HPD). Results on benchmark datasets demonstrate SPICE-ND models achieve the smallest average prediction set sizes, including average size reductions of nearly 50% for some datasets compared to the next best baseline. SPICE-HPD models achieve the best conditional coverage compared to baselines. The SPICE implementation is made available.} }
Endnote
%0 Conference Paper %T Conformalized Deep Splines for Optimal and Efficient Prediction Sets %A Nathaniel Diamant %A Ehsan Hajiramezanali %A Tommaso Biancalani %A Gabriele Scalia %B Proceedings of The 27th International Conference on Artificial Intelligence and Statistics %C Proceedings of Machine Learning Research %D 2024 %E Sanjoy Dasgupta %E Stephan Mandt %E Yingzhen Li %F pmlr-v238-diamant24a %I PMLR %P 1657--1665 %U https://proceedings.mlr.press/v238/diamant24a.html %V 238 %X Uncertainty estimation is critical in high-stakes machine learning applications. One effective way to estimate uncertainty is conformal prediction, which can provide predictive inference with statistical coverage guarantees. We present a new conformal regression method, Spline Prediction Intervals via Conformal Estimation (SPICE), that estimates the conditional density using neural- network-parameterized splines. We prove universal approximation and optimality results for SPICE, which are empirically reflected by our experiments. SPICE is compatible with two different efficient-to- compute conformal scores, one designed for size-efficient marginal coverage (SPICE-ND) and the other for size-efficient conditional coverage (SPICE-HPD). Results on benchmark datasets demonstrate SPICE-ND models achieve the smallest average prediction set sizes, including average size reductions of nearly 50% for some datasets compared to the next best baseline. SPICE-HPD models achieve the best conditional coverage compared to baselines. The SPICE implementation is made available.
APA
Diamant, N., Hajiramezanali, E., Biancalani, T. & Scalia, G.. (2024). Conformalized Deep Splines for Optimal and Efficient Prediction Sets. Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, in Proceedings of Machine Learning Research 238:1657-1665 Available from https://proceedings.mlr.press/v238/diamant24a.html.

Related Material