Conformal prediction in manifold learning

The paper presents a geometrically motivated view on conformal prediction applied to nonlinear multi-output regression tasks for obtaining valid measure of accuracy of Manifold Learning Regression algorithms. A considered regression task is to estimate an unknown smooth mapping $\mathbff$ from $q$-dimensional inputs $\mathbfx ∈\mathbfX$ to $m$-dimensional outputs $\mathbfy = \mathbff(\mathbfx)$ based on training dataset $\mathbfZ_(n)$ consisting of “input-output” pairs ${Z_i = (\mathbfx_i, \mathbfy_i = \mathbff(\mathbfx_i))^\textrmT, i = 1, 2, ..., n}$. Manifold Learning Regression (MLR) algorithm solves this task using Manifold learning technique. At first, unknown $q$-dimensional Regression manifold $\mathbfM(\mathbff) = {(\mathbfx,\mathbff(\mathbfx))^\textrmT∈\mathrmR^q+m: \mathbfx ∈\mathbfX ⊂\mathrmR^q}$, embedded in ambient $(q+m)$-dimensional space, is estimated from the training data $\mathbfZ_(n)$, sampled from this manifold. The constructed estimator $\mathbfM_\textMLR$, which is also $q$-dimensional manifold embedded in ambient space $\textrmR^q+m$, is close to $\mathbfM$ in terms of Hausdorff distance. After that, an estimator $\mathbff_\textMLR$ of the unknown function $\mathbff$, mapping arbitrary input $\mathbfx ∈\mathbfX$ to output $\mathbff_\textrmMLR(\mathbfx)$, is constructed as the solution to the equation $\mathbfM(\mathbff_\textrmMLR) = \mathbfM_\textMLR$. Conformal prediction allows constructing a prediction region for an unknown output $\mathbfy = \mathbff(\mathbfx)$ at Out-of-Sample input point $\mathbfx$ for a given confidence level using given nonconformity measure, characterizing to which extent an example $Z = (\mathbfx, \mathbfy)^\textrmT$ is different from examples in the known dataset $\mathbfZ_(n)$. The paper proposes a new nonconformity measure based on MLR estimator using an analog of Bregman distance.