Distortion and Consistency in Conformal Prediction

Ilia Nouretdinov
Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications, PMLR 329:62-81, 2026.

Abstract

Conformal Prediction (CP) is a framework for reliable machine learning. Its aim is to convert a classification algorithm into a calibrated one with guaranteed validity properties. The core element (metaparameter) of a CP algorithm is a Non-Conformity Measure (CM) function that typically links the framework to an underlying algorithm. Typically, NCM represents an information distance between a data example from an object space and a bag of data examples. In most cases, it has a residual form: the difference between the true value of an example and the label predicted by the underlying algorithm. However, there is also an alternative principle for the NCM construction. Its core is a consistency function that is a function of a bag of data examples only and measures, in some sense, regularity in a bag. Consistency functions can be transformed into NCM functions in a standard way. Therefore, we suggest considering the consistency function as an intermediate chain between the underlying algorithm and the Conformity Measure. In the example of a simple nearest-neighbour approach, we demonstrate that it covers the capabilities of the standard method for defining conformity measures.

Cite this Paper

BibTeX
@InProceedings{pmlr-v329-nouretdinov26a, title = {Distortion and Consistency in Conformal Prediction}, author = {Nouretdinov, Ilia}, booktitle = {Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications}, pages = {62--81}, year = {2026}, editor = {Ahlberg, Ernst and Johansson, Ulf and Boström, Henrik and Carlevaro, Alberto and Hallberg Szabadváry, Johan and Carlsson, Lars}, volume = {329}, series = {Proceedings of Machine Learning Research}, month = {02--04 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v329/main/assets/nouretdinov26a/nouretdinov26a.pdf}, url = {https://proceedings.mlr.press/v329/nouretdinov26a.html}, abstract = {Conformal Prediction (CP) is a framework for reliable machine learning. Its aim is to convert a classification algorithm into a calibrated one with guaranteed validity properties. The core element (metaparameter) of a CP algorithm is a Non-Conformity Measure (CM) function that typically links the framework to an underlying algorithm. Typically, NCM represents an information distance between a data example from an object space and a bag of data examples. In most cases, it has a residual form: the difference between the true value of an example and the label predicted by the underlying algorithm. However, there is also an alternative principle for the NCM construction. Its core is a consistency function that is a function of a bag of data examples only and measures, in some sense, regularity in a bag. Consistency functions can be transformed into NCM functions in a standard way. Therefore, we suggest considering the consistency function as an intermediate chain between the underlying algorithm and the Conformity Measure. In the example of a simple nearest-neighbour approach, we demonstrate that it covers the capabilities of the standard method for defining conformity measures.} }
Endnote
%0 Conference Paper %T Distortion and Consistency in Conformal Prediction %A Ilia Nouretdinov %B Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications %C Proceedings of Machine Learning Research %D 2026 %E Ernst Ahlberg %E Ulf Johansson %E Henrik Boström %E Alberto Carlevaro %E Johan Hallberg Szabadváry %E Lars Carlsson %F pmlr-v329-nouretdinov26a %I PMLR %P 62--81 %U https://proceedings.mlr.press/v329/nouretdinov26a.html %V 329 %X Conformal Prediction (CP) is a framework for reliable machine learning. Its aim is to convert a classification algorithm into a calibrated one with guaranteed validity properties. The core element (metaparameter) of a CP algorithm is a Non-Conformity Measure (CM) function that typically links the framework to an underlying algorithm. Typically, NCM represents an information distance between a data example from an object space and a bag of data examples. In most cases, it has a residual form: the difference between the true value of an example and the label predicted by the underlying algorithm. However, there is also an alternative principle for the NCM construction. Its core is a consistency function that is a function of a bag of data examples only and measures, in some sense, regularity in a bag. Consistency functions can be transformed into NCM functions in a standard way. Therefore, we suggest considering the consistency function as an intermediate chain between the underlying algorithm and the Conformity Measure. In the example of a simple nearest-neighbour approach, we demonstrate that it covers the capabilities of the standard method for defining conformity measures.
APA
Nouretdinov, I.. (2026). Distortion and Consistency in Conformal Prediction. Proceedings of the Fifteenth Symposium on Conformal and Probabilistic Prediction with Applications, in Proceedings of Machine Learning Research 329:62-81 Available from https://proceedings.mlr.press/v329/nouretdinov26a.html.

Related Material