The Uncertain Object: Application of Conformal Prediction to Aerial and Satellite Images

Vicky Copley, Greg Finlay, Ben Hiett
Proceedings of the Thirteenth Symposium on Conformal and Probabilistic Prediction with Applications, PMLR 230:73-89, 2024.

Abstract

Satellites and airborne sensors are critical components of the modern surveillance and reconnaissance capability. A common use case involves the application of object detection models to such images in order to rapidly process the large volumes of data. This optimises use of expensive communications channel bandwidth, reduces the cognitive load on a human interpreter and accelerates the rate at which intelligence can be generated. However there is a clear need for statements of confidence in any predictions in order to provide context and enable trust in model outputs. Our work examines the use of conformal prediction approaches to robustly quantify types of uncertainty in object detection models applied to aerial and satellite imagery for intelligence, surveillance and reconnaissance use cases. We investigate measures of detection and location uncertainty in a YOLO model and indicate how these may be leveraged conformal-wise to provide guarantees on the percentage of objects which aren’t detected and the coverage of predicted bounding boxes. We find that conformal approaches provide a simple and effective means to expose the uncertainty in the outputs of an object detection model and highlight the utility of this knowledge in the intelligence setting.

Cite this Paper

BibTeX
@InProceedings{pmlr-v230-copley24a, title = {The Uncertain Object: Application of Conformal Prediction to Aerial and Satellite Images}, author = {Copley, Vicky and Finlay, Greg and Hiett, Ben}, booktitle = {Proceedings of the Thirteenth Symposium on Conformal and Probabilistic Prediction with Applications}, pages = {73--89}, year = {2024}, editor = {Vantini, Simone and Fontana, Matteo and Solari, Aldo and Boström, Henrik and Carlsson, Lars}, volume = {230}, series = {Proceedings of Machine Learning Research}, month = {09--11 Sep}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v230/main/assets/copley24a/copley24a.pdf}, url = {https://proceedings.mlr.press/v230/copley24a.html}, abstract = {Satellites and airborne sensors are critical components of the modern surveillance and reconnaissance capability. A common use case involves the application of object detection models to such images in order to rapidly process the large volumes of data. This optimises use of expensive communications channel bandwidth, reduces the cognitive load on a human interpreter and accelerates the rate at which intelligence can be generated. However there is a clear need for statements of confidence in any predictions in order to provide context and enable trust in model outputs. Our work examines the use of conformal prediction approaches to robustly quantify types of uncertainty in object detection models applied to aerial and satellite imagery for intelligence, surveillance and reconnaissance use cases. We investigate measures of detection and location uncertainty in a YOLO model and indicate how these may be leveraged conformal-wise to provide guarantees on the percentage of objects which aren’t detected and the coverage of predicted bounding boxes. We find that conformal approaches provide a simple and effective means to expose the uncertainty in the outputs of an object detection model and highlight the utility of this knowledge in the intelligence setting.} }
Endnote
%0 Conference Paper %T The Uncertain Object: Application of Conformal Prediction to Aerial and Satellite Images %A Vicky Copley %A Greg Finlay %A Ben Hiett %B Proceedings of the Thirteenth Symposium on Conformal and Probabilistic Prediction with Applications %C Proceedings of Machine Learning Research %D 2024 %E Simone Vantini %E Matteo Fontana %E Aldo Solari %E Henrik Boström %E Lars Carlsson %F pmlr-v230-copley24a %I PMLR %P 73--89 %U https://proceedings.mlr.press/v230/copley24a.html %V 230 %X Satellites and airborne sensors are critical components of the modern surveillance and reconnaissance capability. A common use case involves the application of object detection models to such images in order to rapidly process the large volumes of data. This optimises use of expensive communications channel bandwidth, reduces the cognitive load on a human interpreter and accelerates the rate at which intelligence can be generated. However there is a clear need for statements of confidence in any predictions in order to provide context and enable trust in model outputs. Our work examines the use of conformal prediction approaches to robustly quantify types of uncertainty in object detection models applied to aerial and satellite imagery for intelligence, surveillance and reconnaissance use cases. We investigate measures of detection and location uncertainty in a YOLO model and indicate how these may be leveraged conformal-wise to provide guarantees on the percentage of objects which aren’t detected and the coverage of predicted bounding boxes. We find that conformal approaches provide a simple and effective means to expose the uncertainty in the outputs of an object detection model and highlight the utility of this knowledge in the intelligence setting.
APA
Copley, V., Finlay, G. & Hiett, B.. (2024). The Uncertain Object: Application of Conformal Prediction to Aerial and Satellite Images. Proceedings of the Thirteenth Symposium on Conformal and Probabilistic Prediction with Applications, in Proceedings of Machine Learning Research 230:73-89 Available from https://proceedings.mlr.press/v230/copley24a.html.

Related Material