Multi-modal conformal prediction regions by optimizing convex shape templates

Renukanandan Tumu, Matthew Cleaveland, Rahul Mangharam, George Pappas, Lars Lindemann
Proceedings of the 6th Annual Learning for Dynamics & Control Conference, PMLR 242:1343-1356, 2024.

Abstract

Conformal prediction is a statistical tool for producing prediction regions for machine learning models that are valid with high probability. A key component of conformal prediction algorithms is a non-conformity score function that quantifies how different a model’s prediction is from the unknown ground truth value. Essentially, these functions determine the shape and the size of the conformal prediction regions. However, little work has gone into finding non-conformity score functions that produce prediction regions that are multi-modal and practical, i.e., that can efficiently be used in engineering applications. We propose a method that optimizes parameterized shape template functions over calibration data, which results in non-conformity score functions that produce prediction regions with minimum volume. Our approach results in prediction regions that are multi-modal, so they can properly capture residuals of distributions that have multiple modes, and practical, so each region is convex and can be easily incorporated into downstream tasks, such as a motion planner using conformal prediction regions. Our method applies to general supervised learning tasks, while we illustrate its use in time-series prediction. We provide a toolbox and present illustrative case studies of F16 fighter jets and autonomous vehicles, showing an up to 68% reduction in prediction region area.

Cite this Paper

BibTeX
@InProceedings{pmlr-v242-tumu24a, title = {Multi-modal conformal prediction regions by optimizing convex shape templates}, author = {Tumu, Renukanandan and Cleaveland, Matthew and Mangharam, Rahul and Pappas, George and Lindemann, Lars}, booktitle = {Proceedings of the 6th Annual Learning for Dynamics & Control Conference}, pages = {1343--1356}, year = {2024}, editor = {Abate, Alessandro and Cannon, Mark and Margellos, Kostas and Papachristodoulou, Antonis}, volume = {242}, series = {Proceedings of Machine Learning Research}, month = {15--17 Jul}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v242/tumu24a/tumu24a.pdf}, url = {https://proceedings.mlr.press/v242/tumu24a.html}, abstract = {Conformal prediction is a statistical tool for producing prediction regions for machine learning models that are valid with high probability. A key component of conformal prediction algorithms is a non-conformity score function that quantifies how different a model’s prediction is from the unknown ground truth value. Essentially, these functions determine the shape and the size of the conformal prediction regions. However, little work has gone into finding non-conformity score functions that produce prediction regions that are multi-modal and practical, i.e., that can efficiently be used in engineering applications. We propose a method that optimizes parameterized shape template functions over calibration data, which results in non-conformity score functions that produce prediction regions with minimum volume. Our approach results in prediction regions that are multi-modal, so they can properly capture residuals of distributions that have multiple modes, and practical, so each region is convex and can be easily incorporated into downstream tasks, such as a motion planner using conformal prediction regions. Our method applies to general supervised learning tasks, while we illustrate its use in time-series prediction. We provide a toolbox and present illustrative case studies of F16 fighter jets and autonomous vehicles, showing an up to 68% reduction in prediction region area.} }
Endnote
%0 Conference Paper %T Multi-modal conformal prediction regions by optimizing convex shape templates %A Renukanandan Tumu %A Matthew Cleaveland %A Rahul Mangharam %A George Pappas %A Lars Lindemann %B Proceedings of the 6th Annual Learning for Dynamics & Control Conference %C Proceedings of Machine Learning Research %D 2024 %E Alessandro Abate %E Mark Cannon %E Kostas Margellos %E Antonis Papachristodoulou %F pmlr-v242-tumu24a %I PMLR %P 1343--1356 %U https://proceedings.mlr.press/v242/tumu24a.html %V 242 %X Conformal prediction is a statistical tool for producing prediction regions for machine learning models that are valid with high probability. A key component of conformal prediction algorithms is a non-conformity score function that quantifies how different a model’s prediction is from the unknown ground truth value. Essentially, these functions determine the shape and the size of the conformal prediction regions. However, little work has gone into finding non-conformity score functions that produce prediction regions that are multi-modal and practical, i.e., that can efficiently be used in engineering applications. We propose a method that optimizes parameterized shape template functions over calibration data, which results in non-conformity score functions that produce prediction regions with minimum volume. Our approach results in prediction regions that are multi-modal, so they can properly capture residuals of distributions that have multiple modes, and practical, so each region is convex and can be easily incorporated into downstream tasks, such as a motion planner using conformal prediction regions. Our method applies to general supervised learning tasks, while we illustrate its use in time-series prediction. We provide a toolbox and present illustrative case studies of F16 fighter jets and autonomous vehicles, showing an up to 68% reduction in prediction region area.
APA
Tumu, R., Cleaveland, M., Mangharam, R., Pappas, G. & Lindemann, L.. (2024). Multi-modal conformal prediction regions by optimizing convex shape templates. Proceedings of the 6th Annual Learning for Dynamics & Control Conference, in Proceedings of Machine Learning Research 242:1343-1356 Available from https://proceedings.mlr.press/v242/tumu24a.html.

Related Material