Conformal prediction in manifold learning

Alexander Kuleshov, Alexander Bernstein, Evgeny Burnaev
Proceedings of the Seventh Workshop on Conformal and Probabilistic Prediction and Applications, PMLR 91:234-253, 2018.

Abstract

The paper presents a geometrically motivated view on conformal prediction applied to nonlinear multi-output regression tasks for obtaining valid measure of accuracy of Manifold Learning Regression algorithms. A considered regression task is to estimate an unknown smooth mapping $\mathbff$ from $q$-dimensional inputs $\mathbfx ∈\mathbfX$ to $m$-dimensional outputs $\mathbfy = \mathbff(\mathbfx)$ based on training dataset $\mathbfZ_(n)$ consisting of “input-output” pairs ${Z_i = (\mathbfx_i, \mathbfy_i = \mathbff(\mathbfx_i))^\textrmT, i = 1, 2, ..., n}$. Manifold Learning Regression (MLR) algorithm solves this task using Manifold learning technique. At first, unknown $q$-dimensional Regression manifold $\mathbfM(\mathbff) = {(\mathbfx,\mathbff(\mathbfx))^\textrmT∈\mathrmR^q+m: \mathbfx ∈\mathbfX ⊂\mathrmR^q}$, embedded in ambient $(q+m)$-dimensional space, is estimated from the training data $\mathbfZ_(n)$, sampled from this manifold. The constructed estimator $\mathbfM_\textMLR$, which is also $q$-dimensional manifold embedded in ambient space $\textrmR^q+m$, is close to $\mathbfM$ in terms of Hausdorff distance. After that, an estimator $\mathbff_\textMLR$ of the unknown function $\mathbff$, mapping arbitrary input $\mathbfx ∈\mathbfX$ to output $\mathbff_\textrmMLR(\mathbfx)$, is constructed as the solution to the equation $\mathbfM(\mathbff_\textrmMLR) = \mathbfM_\textMLR$. Conformal prediction allows constructing a prediction region for an unknown output $\mathbfy = \mathbff(\mathbfx)$ at Out-of-Sample input point $\mathbfx$ for a given confidence level using given nonconformity measure, characterizing to which extent an example $Z = (\mathbfx, \mathbfy)^\textrmT$ is different from examples in the known dataset $\mathbfZ_(n)$. The paper proposes a new nonconformity measure based on MLR estimator using an analog of Bregman distance.

Cite this Paper


BibTeX
@InProceedings{pmlr-v91-kuleshov18a, title = {Conformal prediction in manifold learning}, author = {Kuleshov, Alexander and Bernstein, Alexander and Burnaev, Evgeny}, booktitle = {Proceedings of the Seventh Workshop on Conformal and Probabilistic Prediction and Applications}, pages = {234--253}, year = {2018}, editor = {Gammerman, Alex and Vovk, Vladimir and Luo, Zhiyuan and Smirnov, Evgueni and Peeters, Ralf}, volume = {91}, series = {Proceedings of Machine Learning Research}, month = {11--13 Jun}, publisher = {PMLR}, pdf = {http://proceedings.mlr.press/v91/kuleshov18a/kuleshov18a.pdf}, url = {https://proceedings.mlr.press/v91/kuleshov18a.html}, abstract = {The paper presents a geometrically motivated view on conformal prediction applied to nonlinear multi-output regression tasks for obtaining valid measure of accuracy of Manifold Learning Regression algorithms. A considered regression task is to estimate an unknown smooth mapping $\mathbff$ from $q$-dimensional inputs $\mathbfx ∈\mathbfX$ to $m$-dimensional outputs $\mathbfy = \mathbff(\mathbfx)$ based on training dataset $\mathbfZ_(n)$ consisting of “input-output” pairs ${Z_i = (\mathbfx_i, \mathbfy_i = \mathbff(\mathbfx_i))^\textrmT, i = 1, 2, ..., n}$. Manifold Learning Regression (MLR) algorithm solves this task using Manifold learning technique. At first, unknown $q$-dimensional Regression manifold $\mathbfM(\mathbff) = {(\mathbfx,\mathbff(\mathbfx))^\textrmT∈\mathrmR^q+m: \mathbfx ∈\mathbfX ⊂\mathrmR^q}$, embedded in ambient $(q+m)$-dimensional space, is estimated from the training data $\mathbfZ_(n)$, sampled from this manifold. The constructed estimator $\mathbfM_\textMLR$, which is also $q$-dimensional manifold embedded in ambient space $\textrmR^q+m$, is close to $\mathbfM$ in terms of Hausdorff distance. After that, an estimator $\mathbff_\textMLR$ of the unknown function $\mathbff$, mapping arbitrary input $\mathbfx ∈\mathbfX$ to output $\mathbff_\textrmMLR(\mathbfx)$, is constructed as the solution to the equation $\mathbfM(\mathbff_\textrmMLR) = \mathbfM_\textMLR$. Conformal prediction allows constructing a prediction region for an unknown output $\mathbfy = \mathbff(\mathbfx)$ at Out-of-Sample input point $\mathbfx$ for a given confidence level using given nonconformity measure, characterizing to which extent an example $Z = (\mathbfx, \mathbfy)^\textrmT$ is different from examples in the known dataset $\mathbfZ_(n)$. The paper proposes a new nonconformity measure based on MLR estimator using an analog of Bregman distance.} }
Endnote
%0 Conference Paper %T Conformal prediction in manifold learning %A Alexander Kuleshov %A Alexander Bernstein %A Evgeny Burnaev %B Proceedings of the Seventh Workshop on Conformal and Probabilistic Prediction and Applications %C Proceedings of Machine Learning Research %D 2018 %E Alex Gammerman %E Vladimir Vovk %E Zhiyuan Luo %E Evgueni Smirnov %E Ralf Peeters %F pmlr-v91-kuleshov18a %I PMLR %P 234--253 %U https://proceedings.mlr.press/v91/kuleshov18a.html %V 91 %X The paper presents a geometrically motivated view on conformal prediction applied to nonlinear multi-output regression tasks for obtaining valid measure of accuracy of Manifold Learning Regression algorithms. A considered regression task is to estimate an unknown smooth mapping $\mathbff$ from $q$-dimensional inputs $\mathbfx ∈\mathbfX$ to $m$-dimensional outputs $\mathbfy = \mathbff(\mathbfx)$ based on training dataset $\mathbfZ_(n)$ consisting of “input-output” pairs ${Z_i = (\mathbfx_i, \mathbfy_i = \mathbff(\mathbfx_i))^\textrmT, i = 1, 2, ..., n}$. Manifold Learning Regression (MLR) algorithm solves this task using Manifold learning technique. At first, unknown $q$-dimensional Regression manifold $\mathbfM(\mathbff) = {(\mathbfx,\mathbff(\mathbfx))^\textrmT∈\mathrmR^q+m: \mathbfx ∈\mathbfX ⊂\mathrmR^q}$, embedded in ambient $(q+m)$-dimensional space, is estimated from the training data $\mathbfZ_(n)$, sampled from this manifold. The constructed estimator $\mathbfM_\textMLR$, which is also $q$-dimensional manifold embedded in ambient space $\textrmR^q+m$, is close to $\mathbfM$ in terms of Hausdorff distance. After that, an estimator $\mathbff_\textMLR$ of the unknown function $\mathbff$, mapping arbitrary input $\mathbfx ∈\mathbfX$ to output $\mathbff_\textrmMLR(\mathbfx)$, is constructed as the solution to the equation $\mathbfM(\mathbff_\textrmMLR) = \mathbfM_\textMLR$. Conformal prediction allows constructing a prediction region for an unknown output $\mathbfy = \mathbff(\mathbfx)$ at Out-of-Sample input point $\mathbfx$ for a given confidence level using given nonconformity measure, characterizing to which extent an example $Z = (\mathbfx, \mathbfy)^\textrmT$ is different from examples in the known dataset $\mathbfZ_(n)$. The paper proposes a new nonconformity measure based on MLR estimator using an analog of Bregman distance.
APA
Kuleshov, A., Bernstein, A. & Burnaev, E.. (2018). Conformal prediction in manifold learning. Proceedings of the Seventh Workshop on Conformal and Probabilistic Prediction and Applications, in Proceedings of Machine Learning Research 91:234-253 Available from https://proceedings.mlr.press/v91/kuleshov18a.html.

Related Material