Confident Object Detection via Conformal Prediction and Conformal Risk Control: an Application to Railway Signaling

Leo Andeol, Thomas Fel, Florence de Grancey, Luca Mossina
Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications, PMLR 204:36-55, 2023.

Abstract

Deploying deep learning models in real-world certified systems requires the ability to provide confidence estimates that accurately reflect their uncertainty. In this paper, we demonstrate the use of the conformal prediction framework to construct reliable and trustworthy predictors for detecting railway signals. Our approach is based on a novel dataset that includes images taken from the perspective of a train operator and state-of-the-art object detectors. We test several conformal approaches and introduce a new method based on conformal risk control. Our findings demonstrate the potential of the conformal prediction framework to evaluate model performance and provide practical guidance for achieving formally guaranteed uncertainty bounds.

Cite this Paper

BibTeX
@InProceedings{pmlr-v204-andeol23a, title = {Confident Object Detection via Conformal Prediction and Conformal Risk Control: an Application to Railway Signaling}, author = {Andeol, Leo and Fel, Thomas and de Grancey, Florence and Mossina, Luca}, booktitle = {Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications}, pages = {36--55}, year = {2023}, editor = {Papadopoulos, Harris and Nguyen, Khuong An and Boström, Henrik and Carlsson, Lars}, volume = {204}, series = {Proceedings of Machine Learning Research}, month = {13--15 Sep}, publisher = {PMLR}, pdf = {https://proceedings.mlr.press/v204/andeol23a/andeol23a.pdf}, url = {https://proceedings.mlr.press/v204/andeol23a.html}, abstract = {Deploying deep learning models in real-world certified systems requires the ability to provide confidence estimates that accurately reflect their uncertainty. In this paper, we demonstrate the use of the conformal prediction framework to construct reliable and trustworthy predictors for detecting railway signals. Our approach is based on a novel dataset that includes images taken from the perspective of a train operator and state-of-the-art object detectors. We test several conformal approaches and introduce a new method based on conformal risk control. Our findings demonstrate the potential of the conformal prediction framework to evaluate model performance and provide practical guidance for achieving formally guaranteed uncertainty bounds.} }
Endnote
%0 Conference Paper %T Confident Object Detection via Conformal Prediction and Conformal Risk Control: an Application to Railway Signaling %A Leo Andeol %A Thomas Fel %A Florence de Grancey %A Luca Mossina %B Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications %C Proceedings of Machine Learning Research %D 2023 %E Harris Papadopoulos %E Khuong An Nguyen %E Henrik Boström %E Lars Carlsson %F pmlr-v204-andeol23a %I PMLR %P 36--55 %U https://proceedings.mlr.press/v204/andeol23a.html %V 204 %X Deploying deep learning models in real-world certified systems requires the ability to provide confidence estimates that accurately reflect their uncertainty. In this paper, we demonstrate the use of the conformal prediction framework to construct reliable and trustworthy predictors for detecting railway signals. Our approach is based on a novel dataset that includes images taken from the perspective of a train operator and state-of-the-art object detectors. We test several conformal approaches and introduce a new method based on conformal risk control. Our findings demonstrate the potential of the conformal prediction framework to evaluate model performance and provide practical guidance for achieving formally guaranteed uncertainty bounds.
APA
Andeol, L., Fel, T., de Grancey, F. & Mossina, L.. (2023). Confident Object Detection via Conformal Prediction and Conformal Risk Control: an Application to Railway Signaling. Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications, in Proceedings of Machine Learning Research 204:36-55 Available from https://proceedings.mlr.press/v204/andeol23a.html.

Related Material