Conformal Risk Minimization with Variance Reduction

Sima Noorani, Orlando Romero, Nicolo Dal Fabbro, Hamed Hassani, George J. Pappas
Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, PMLR 337:4972-5002, 2026.

Abstract

Conformal prediction **({CP})** is a distribution-free framework for achieving probabilistic guarantees on black-box models. **{CP}** is generally applied to a model post-training. Recent research efforts, on the other hand, have focused on optimizing {CP} efficiency **during training**. We formalize this concept as the problem of **conformal risk minimization** (CRM). In this direction, **conformal training** **(ConfTr**) by (Stutz et. al (2022)) is a CRM technique that seeks to minimize the expected prediction set size of a model by simulating **{CP}** in-between training updates. In this paper, we provide a novel analysis for the **ConfTr** gradient estimation method, revealing a strong source of sample inefficiency that introduces training instability and limits its practical use. To address this challenge, we propose **variance-reduced conformal training** **(VR-ConfTr)**, a CRM method that carefully incorporates a novel variance reduction technique in the gradient estimation of the **ConfTr** objective function. Through extensive experiments on various benchmark datasets, we demonstrate that **VR-ConfTr** consistently achieves faster convergence and smaller prediction sets compared to baselines.

Cite this Paper

BibTeX
@InProceedings{pmlr-v337-noorani26a, title = {Conformal Risk Minimization with Variance Reduction}, author = {Noorani, Sima and Romero, Orlando and Dal Fabbro, Nicolo and Hassani, Hamed and Pappas, George J.}, booktitle = {Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence}, pages = {4972--5002}, year = {2026}, editor = {Perković, Emilija and Malinsky, Daniel}, volume = {337}, series = {Proceedings of Machine Learning Research}, month = {17--21 Aug}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v337/main/assets/noorani26a/noorani26a.pdf}, url = {https://proceedings.mlr.press/v337/noorani26a.html}, abstract = {Conformal prediction **({CP})** is a distribution-free framework for achieving probabilistic guarantees on black-box models. **{CP}** is generally applied to a model post-training. Recent research efforts, on the other hand, have focused on optimizing {CP} efficiency **during training**. We formalize this concept as the problem of **conformal risk minimization** (CRM). In this direction, **conformal training** **(ConfTr**) by (Stutz et. al (2022)) is a CRM technique that seeks to minimize the expected prediction set size of a model by simulating **{CP}** in-between training updates. In this paper, we provide a novel analysis for the **ConfTr** gradient estimation method, revealing a strong source of sample inefficiency that introduces training instability and limits its practical use. To address this challenge, we propose **variance-reduced conformal training** **(VR-ConfTr)**, a CRM method that carefully incorporates a novel variance reduction technique in the gradient estimation of the **ConfTr** objective function. Through extensive experiments on various benchmark datasets, we demonstrate that **VR-ConfTr** consistently achieves faster convergence and smaller prediction sets compared to baselines.} }
Endnote
%0 Conference Paper %T Conformal Risk Minimization with Variance Reduction %A Sima Noorani %A Orlando Romero %A Nicolo Dal Fabbro %A Hamed Hassani %A George J. Pappas %B Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2026 %E Emilija Perković %E Daniel Malinsky %F pmlr-v337-noorani26a %I PMLR %P 4972--5002 %U https://proceedings.mlr.press/v337/noorani26a.html %V 337 %X Conformal prediction **({CP})** is a distribution-free framework for achieving probabilistic guarantees on black-box models. **{CP}** is generally applied to a model post-training. Recent research efforts, on the other hand, have focused on optimizing {CP} efficiency **during training**. We formalize this concept as the problem of **conformal risk minimization** (CRM). In this direction, **conformal training** **(ConfTr**) by (Stutz et. al (2022)) is a CRM technique that seeks to minimize the expected prediction set size of a model by simulating **{CP}** in-between training updates. In this paper, we provide a novel analysis for the **ConfTr** gradient estimation method, revealing a strong source of sample inefficiency that introduces training instability and limits its practical use. To address this challenge, we propose **variance-reduced conformal training** **(VR-ConfTr)**, a CRM method that carefully incorporates a novel variance reduction technique in the gradient estimation of the **ConfTr** objective function. Through extensive experiments on various benchmark datasets, we demonstrate that **VR-ConfTr** consistently achieves faster convergence and smaller prediction sets compared to baselines.
APA
Noorani, S., Romero, O., Dal Fabbro, N., Hassani, H. & Pappas, G.J.. (2026). Conformal Risk Minimization with Variance Reduction. Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 337:4972-5002 Available from https://proceedings.mlr.press/v337/noorani26a.html.

Related Material