MetaCOG: A Heirarchical Probabilistic Model for Learning Meta-Cognitive Visual Representations

Marlene Berke, Zhangir Azerbayev, Mario Belledonne, Zenna Tavares, Julian Jara-Ettinger
Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence, PMLR 244:349-359, 2024.

Abstract

Humans have the capacity to question what we see and to recognize when our vision is unreliable (e.g., when we realize that we are experiencing a visual illusion). Inspired by this capacity, we present MetaCOG: a hierarchical probabilistic model that can be attached to a neural object detector to monitor its outputs and determine their reliability. MetaCOG achieves this by learning a probabilistic model of the object detector’s performance via Bayesian inference{—}i.e., a meta-cognitive representation of the network’s propensity to hallucinate or miss different object categories. Given a set of video frames processed by an object detector, MetaCOG performs joint inference over the underlying 3D scene and the detector’s performance, grounding inference on a basic assumption of object permanence. Paired with three neural object detectors, we show that MetaCOG accurately recovers each detector’s performance parameters and improves the overall system’s accuracy. We additionally show that MetaCOG is robust to varying levels of error in object detector outputs, showing proof-of-concept for a novel approach to the problem of detecting and correcting errors in vision systems when ground-truth is not available.

Cite this Paper

BibTeX
@InProceedings{pmlr-v244-berke24a, title = {MetaCOG: A Heirarchical Probabilistic Model for Learning Meta-Cognitive Visual Representations}, author = {Berke, Marlene and Azerbayev, Zhangir and Belledonne, Mario and Tavares, Zenna and Jara-Ettinger, Julian}, booktitle = {Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence}, pages = {349--359}, year = {2024}, editor = {Kiyavash, Negar and Mooij, Joris M.}, volume = {244}, series = {Proceedings of Machine Learning Research}, month = {15--19 Jul}, publisher = {PMLR}, pdf = {https://raw.githubusercontent.com/mlresearch/v244/main/assets/berke24a/berke24a.pdf}, url = {https://proceedings.mlr.press/v244/berke24a.html}, abstract = {Humans have the capacity to question what we see and to recognize when our vision is unreliable (e.g., when we realize that we are experiencing a visual illusion). Inspired by this capacity, we present MetaCOG: a hierarchical probabilistic model that can be attached to a neural object detector to monitor its outputs and determine their reliability. MetaCOG achieves this by learning a probabilistic model of the object detector’s performance via Bayesian inference{—}i.e., a meta-cognitive representation of the network’s propensity to hallucinate or miss different object categories. Given a set of video frames processed by an object detector, MetaCOG performs joint inference over the underlying 3D scene and the detector’s performance, grounding inference on a basic assumption of object permanence. Paired with three neural object detectors, we show that MetaCOG accurately recovers each detector’s performance parameters and improves the overall system’s accuracy. We additionally show that MetaCOG is robust to varying levels of error in object detector outputs, showing proof-of-concept for a novel approach to the problem of detecting and correcting errors in vision systems when ground-truth is not available.} }
Endnote
%0 Conference Paper %T MetaCOG: A Heirarchical Probabilistic Model for Learning Meta-Cognitive Visual Representations %A Marlene Berke %A Zhangir Azerbayev %A Mario Belledonne %A Zenna Tavares %A Julian Jara-Ettinger %B Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence %C Proceedings of Machine Learning Research %D 2024 %E Negar Kiyavash %E Joris M. Mooij %F pmlr-v244-berke24a %I PMLR %P 349--359 %U https://proceedings.mlr.press/v244/berke24a.html %V 244 %X Humans have the capacity to question what we see and to recognize when our vision is unreliable (e.g., when we realize that we are experiencing a visual illusion). Inspired by this capacity, we present MetaCOG: a hierarchical probabilistic model that can be attached to a neural object detector to monitor its outputs and determine their reliability. MetaCOG achieves this by learning a probabilistic model of the object detector’s performance via Bayesian inference{—}i.e., a meta-cognitive representation of the network’s propensity to hallucinate or miss different object categories. Given a set of video frames processed by an object detector, MetaCOG performs joint inference over the underlying 3D scene and the detector’s performance, grounding inference on a basic assumption of object permanence. Paired with three neural object detectors, we show that MetaCOG accurately recovers each detector’s performance parameters and improves the overall system’s accuracy. We additionally show that MetaCOG is robust to varying levels of error in object detector outputs, showing proof-of-concept for a novel approach to the problem of detecting and correcting errors in vision systems when ground-truth is not available.
APA
Berke, M., Azerbayev, Z., Belledonne, M., Tavares, Z. & Jara-Ettinger, J.. (2024). MetaCOG: A Heirarchical Probabilistic Model for Learning Meta-Cognitive Visual Representations. Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence, in Proceedings of Machine Learning Research 244:349-359 Available from https://proceedings.mlr.press/v244/berke24a.html.

Related Material