# MetaCOG: Learning a Metacognition to Recover What Objects Are Actually There

Marlene D. Berke1, Zhangir Azerbayev2, Mario Belledonne1,
Zenna Tavares3, 4, & Julian Jara-Ettinger1

###### Abstract

Humans not only form representations about the world based on what we see, but
also learn meta-cognitive representations about how our own vision works. This
enables us to recognize when our vision is unreliable (e.g., when we realize
that we are experiencing a visual illusion) and enables us tp question what we
see. Inspired by this human capacity, we present MetaCOG: a model that
increases the robustness of object detectors by learning representations of
their reliability, and does so without feedback. Specifically, MetaCOG is a
hierarchical probabilistic model that expresses a joint distribution over the
objects in a 3D scene and the outputs produced by a detector. When paired with
an off-the-shelf object detector, MetaCOG takes detections as input and infers
the detector’s tendencies to miss objects of certain categories and to
hallucinate objects that are not actually present, all without access to
ground-truth object labels. When paired with three modern neural object
detectors, MetaCOG learns useful and accurate meta-cognitive representations,
resulting in improved performance on the detection task. Additionally, we show
that MetaCOG is robust to varying levels of error in the detections. Our
results are a proof-of-concept for a novel approach to the problem of
correcting a faulty vision system’s errors. The model code, datasets, results,
and demos are available:
https://osf.io/8b9qt/?view˙only=8c1b1c412c6b4e1697e3c7859be2fce6.

##  1 Introduction

Building accurate representations of the world is critical for prediction,
inference, and planning in complex environments (Lake et al. 2017). While the
last decade has witnessed a revolution in the performance of end-to-end object
recognition models, these models can nonetheless suffer from errors. After a
model’s training regime is complete and the model is deployed, how can we
identify when a detected object is not actually there (a hallucination), or
when an object in a scene was not detected (a miss)?

This paper draws inspiration from humans to introduce a proof-of-concept
solution to this problem. While human vision is generally robust and reliable,
it nonetheless suffers from errors, such as when we experience a visual
illusion, like a mirage on a highway or apparent motion in a static image.
Even though these illusions are visually compelling, people recognize that
they shouldn’t be trusted. In these cases, humans are representing that their
own visual system produced an unreliable percept. This representation of our
own vision’s behavior is a form of meta-cognition (Nelson 1990), and it
enables us to build world models that are robust to faults in our own vision.
We propose that this same approach can be used to improve the robustness of
object detection models, by augmenting them with a meta-cognition that
represents the object detector’s behavior (Fig. 1). We present a formalization
of this idea and a proof-of-concept of the approach.

There are two main insights that we draw upon. First, how can we learn what
objects are being missed or hallucinated without access to ground truth? Our
insight is that humans embed object detections in three-dimensional
representations that assume object permanence (i.e., objects in the world
continue to exist even when they cannot be seen; Carey 2009; Spelke and
Kinzler 2007). By grounding an detections in a 3D representation with object
permanence, our model, MetaCOG (Meta-Cognitive Object-detecting Generative-
model), can, without feedback or access to ground-truth object labels, learn
which detections were hallucinated and which objects were missed.

Second, what is a useful meta-cognitive representation of vision? For humans,
our own visual system is an encapsulated black-box into whose internal
computations we have no access (Firestone and Scholl, 2016). Following this,
the meta-cognitive module we develop here does not have access to the
internals of the object detector. The central representation in this meta-
cognition is a joint distribution expressing the relationship between the
objects in a scene and the detections produced by a detector processing that
scene. This distribution therefore represents the object detector’s
performance, capturing which object detections are trustworthy, and which are
not. With access to ground-truth object labels, this distribution could be
calculated directly (i.e. by simply calculating empirical hallucination and
miss rates). The challenge therefore lies in learning an accurate meta-
cognitive representation of the object detector without access to the ground-
truth objects in the scene.

By grounding detections on stable 3D representations, and formulating visual
meta-cognition as a joint distribution between what’s in the world and what’s
detected, MetaCOG learns a meta-cognitive representation of an object
detector, all without access to its internal architecture, performance
metrics, or human supervision. Specifically, given a pre-trained object
detector and a dataset of images (partitioned into scene-specific sets with
corresponding viewpoints), MetaCOG performs joint inference over the objects
in the scenes and the behavior of the object detector.

We evaluate MetaCOG’s ability to 1) infer an accurate representation of the
detector’s performance and 2) use this meta-representation to identify and
correct missed or hallucinated objects. In Exp. 1, we explore MetaCOG’s
performance when paired with three modern neural object detectors, testing
MetaCOG on a dataset of scenes rendered in the ThreeDWorld (TDW) virtual
environment (Gan et al. 2020). In Exp. 2, we explore MetaCOG’s tolerance to
faulty inputs. These experiments are not intended to show advancement over
state-of-the-art computer vision techniques on benchmark datasets, but instead
to demonstrate the promise of meta-cognition for handling a vision system’s
errors.

Figure 1: Conceptual schematic of MetaCOG. Information flows from left to
right. From the left, images of a scene (described by a “World State”) are
processed by a neural “Object Detector,” which produces detections (semantic
labels with bounding boxes in the 2D image). The MetaCOG model takes
detections as input (without any access to the underlying world state or the
ground-truth accuracy of these detections), and jointly infers a meta-
cognitive representation of the detector, and the objects present in the scene
(“Inferred World State”). Specifically, MetaCOG’s meta-cognitive
representation consists of learned category-specific probabilities of the
object detector generating hallucinations (detections of objects that were not
actually there) and missing objects (failures to detect an object that was
actually there). MetaCOG simultaneously infers this meta-cognitive
representation and the world state (i.e., semantic labels and locations in 3D
space).

Our work makes two main contributions. First, we propose meta-cognition as a
novel approach to correcting for the biases of a computer vision system so as
to improve robustness. Second, we present a particular instantiation of meta-
cognition in the context of object detection, called MetaCOG. We show that
MetaCOG can efficiently learn an accurate meta-cognitive representation of an
object detector without feedback and use it to make better inferences about
what objects are where, enabling it to recover a scene in a way that is robust
to the faults of the object detector.

##  2 Related work

##### Meta-cognition in AI.

Previous work has shown the promise of meta-cognition for improving
classification accuracy (Babu and Suresh 2012; Subramanian, Suresh, and
Sundararajan 2013). While that work focused on engineering (rather than
learning) a meta-cognition to guide training, we focus on a complimentary
problem: learning a meta-cognition for correcting outputs from a pre-trained
network.

##### Object knowledge.

Our work is also related to computational models of infant object knowledge
(Smith et al. 2019; Kemp and Xu 2009) and work applying cognitively-inspired
object principles to computer vision (Chen et al. 2022; Tokmakov et al. 2021).
The difference is that our work uses object principles to learn a meta-
cognition, whereas past work focused on modeling object principles themselves.

##### Uncertainty-aware AI.

Several systems have been developed that, when trained end-to-end, can express
uncertainty in their inferences (e.g, Sensoy, Kaplan, and Kandemir 2018;
Kaplan et al. 2018; Ivanovska et al. 2015). In contrast, MetaCOG learns a
model of uncertainty over an already-trained vision system’s outputs ( similar
to the approach in Platt et al. 1999 and Shen et al. 2023). These two
approaches can co-exist even in the same system — humans have both uncertainty
intrinsic to vision (e.g., the blurry percept experienced upon removing
eyeglasses) and meta-cognitive uncertainty (e.g., as when we doubt the
veracity of a mirage).

##### Neurosymbolic AI.

Our work can be seen as an instance of neurosymbolic AI (Garcez and Lamb
2020). Related work uses detections from neural object detectors and a
generative model of scenes to infer a symbolic scene graph (Gothoskar et al.
2021). In the language domain, Nye et al. 2021 uses a symbolic world model to
improve the coherence of a large language model, similar to how MetaCOG’s
symbolic world models and meta-cognition improve the coherence of object
detections. Our work is unique in its formulation of meta-cognition within a
neurosymbolic framework.

##  3 MetaCOG

Throughout, we consider a problem setting where multiple images are taken of a
scene (a room with objects in it). These images are taken from different
viewpoints along a camera trajectory $\vec{cj}$, such that objects may pass in
and out of view. These images are processed by an object detector, producing
detections. The goal of MetaCOG is to take the pattern of detections and learn
(without access to ground-truth objects) an accurate representation of the
detector’s performance and of the world state (i.e., the semantic label and 3D
position of each object in the scene). By repeating this process over many
scenes, we test if and how efficiently MetaCOG can build an accurate
representation of the detector so as to make better inferences about world
states.

Fig. 1 illustrates MetaCOG’s usage and representations. We formalize visual
meta-cognition as a joint distribution over the objects in the scene and the
vision system’s outputs. This is represented by two category-specific
probability distributions: one capturing the probability of hallucinated
detections (detected, but not actually there) for each category, and another
capturing the probability of missed detections (not detected, but actually
there) for each category. The purpose of learning this meta-cognition is to
account for the detector’s faults so as to correctly infer what objects are
where. The rest of this section describes MetaCOG more formally, starting with
the meta-cognitive representation, then world states and detections, and
finally, the inference procedure.

###  3.1 Generative model

#### Meta-cognitive representation

The meta-cognition represents two aspects of the object detector’s
performance: its propensity to hallucinate objects that are not there, and its
propensity to miss (or conversely, accurately detect) objects that are there.
This is represented as two probability distributions per object category. The
first distribution captures the detector’s propensity to hallucinate, modeled
as the number of times objects of category $c\in\mathbb{C}$ will be
hallucinated in a given frame. Under naive assumptions, hallucinations may be
independent of each other and randomly distributed within and across images,
therefore following a Poisson distribution with rate $\lambda_{c}$ (to be
inferred by MetaCOG). The second distribution captures the detector’s
propensity to correctly detect an object that’s in view. Because object
detectors can produce multiple detections from a single object (e.g., if it is
incorrectly parsed as two objects), this representation follows a Geometric
distribution with rate $p_{c}$ encoding a belief over the number of times
($0,1,...,k$) an object of category $c$ will be detected when it is present in
the image. Conveniently, under this formulation, the detector’s miss rate for
an object in category $c$ is $1-p_{c}$.

Because each distribution is captured by a single parameter, the parameters of
the distributions forming the meta-cognition can be represented as a pair of
vectors of length $|\mathbb{C}|$ storing each category’s hallucination rate
$\lambda_{c}$ and miss rate $1-p_{c}$. We call this pair of vectors of
parameters $\theta$. We refer collectively to the distributions that they
parameterize as the $V$ since they express a belief about the veracity of the
object detector. For notation simplicity, $V$ will refer to $V|\theta$.

The generative model described so far captures how MetaCOG represents an
object detector’s performance. However, a detector’s propensity to hallucinate
or miss objects can vary across scenes, so leaving flexibility in $V$ is
desirable. At the same time, experience in a previous scene includes critical
information about the detector that should inform expectations about its
performance in a new scene. Our generative model therefore includes an
evolving kernel, capturing changing priors over the parameters $\theta$ for
the probability distributions $V$ (see A.1). Since beliefs about a detector
evolve from scene to scene, $V_{t}$ denotes the belief at time $t$, and
$\vec{V}=(V_{1},...,V_{T})$ denotes a sequence of evolving beliefs over
timesteps.

#### Object Detector Outputs and Input to MetaCOG

To infer the meta-cognitive representation, MetaCOG uses as input the
detections output by an object detector. Given an image, a detector produces
an unordered list of detections, each consisting of a category label and a
position in the 2D image. Each detection is a tuple $(x,y,c)$ where $x$ and
$y$ are pixel-coordinates (i.e., the centroid of a bounding box) and
$c\in\mathbb{C}$ is an object category. The neural network object detector can
be seen as a function $NN$ applied to an image $i$, producing a collection of
detections, call this $NN(i)$. If $\mathbb{I}_{W_{t}}$ is the collection of
images from world state $W_{t}$, then the collection of detections generated
from all of those images is $NN(\mathbb{I}_{W_{t}})$. For convenience, we will
refer to this collection of detections generated from the world state $W_{t}$
as $D_{t}$. Crucially, while MetaCOG has access to the detections generated by
a detector, the object detector itself is a black box—MetaCOG does not have
any access to its internal state.

#### World States and Object Constraints

The meta-cognition describes how detections relate to the presence or absence
of objects in a world state. We represent an underlying world state as a
collection of objects, each object a 4-tuple of the form $(x,y,z,c)$ where
$x$, $y$ and $z$ are coordinates in 3D space and $c$ is an object category. We
denote by $\vec{W}=(W_{1},\dots,W_{T})$ a sequence of world states.

Our approach implements object constraints into world states, which we
hypothesize will make the visual meta-cognition learnable without access to
ground-truth objects. Inspired by human infants’ representations of objects,
we assume that objects occupy locations in 3D space and that no two objects
can occupy overlapping positions (implemented as a prior over the locations of
the objects in a world state; details available in A.1). We further implement
object permanence as the constraint that each world state has a fixed set of
objects that does not change (as a consequence, an object in a scene is still
there even when it is not observed due to being out of view, occluded, or
missed by the detector).

In our setting, multiple images are taken of each scene in a sequence of world
states $\vec{W}$. We assume access to the camera’s position and orientation
for each image. We also assume that access to the camera trajectory implies
the association of which images were taken of which scenes. While inferring
the camera trajectory may be possible, that is not the focus of our work.
Critically, we do not assume access to the ground-truth objects in the
underlying world states.

###  3.2 Inference procedure

So far we have defined how 2D images taken of a series of scenes are processed
through an object detector to produce collections of detections. These
detections, along with the camera trajectories, are then used as observations
from which to infer world states and the meta-cognitive representation of the
object detector.

Given $\vec{D}$ (collections of detections ($D_{1},...,D_{T}$) from a sequence
of world states $\vec{W}$) and the corresponding camera trajectories
$\vec{cj}$, the goal is to infer $Pr(\vec{V},\vec{W}|\vec{D},\vec{cj})$:

$P\left(\vec{V},\vec{W}\mid\vec{D},\vec{cj}\right)\propto\\\
\prod_{t=1}^{T}P\left(D_{t}|W_{t},V_{t},cj_{t}\right)P(V_{t})P(W_{t})$

The posterior is approximated via Sequential Monte-Carlo using a particle
filter (for background, see Doucet, Johansen et al. 2009). The details of the
algorithms used for inference are described in A.1.

An estimate of the joint posterior can be sequentially approximated via:

$\begin{split}&P\left(\vec{V},\vec{W}|\vec{D},\vec{cj}\right)\approx
P\left(\hat{V_{0}}^{0}\right)*\\\\[-5.0pt]
&\prod_{t=1}^{T}P\left(D_{t}|\hat{V_{t}}^{t},\hat{W_{t}}^{t},cj_{t}\right)P\left(\hat{W_{t}}^{t}\right)P\left({\hat{V_{t}}}^{t}|{\hat{W_{t-1}}^{t}},{\hat{D_{t-1}}^{t}}\right)\end{split}$

where $\hat{W_{1}}^{T},\ldots,\hat{W_{T}}^{T}$ is the estimate of
$W_{1},\ldots,W_{T}$ after detections from $T$ world states have been
observed, and $\hat{V_{1}}^{T},\ldots,\hat{V_{T}}^{T}$ is the estimate of
$V_{1},\ldots,V_{T}$ after $T$ observations. Here the transition kernel,
$Pr(\hat{V_{t}}^{t}|\hat{{W_{t-1}}^{t}},{\hat{D_{t-1}}^{t}})$ is governed by
the meta-cognitive dynamics kernel in A.1.

#### Estimating $V$

After all $T$ world states have been processed, we estimate $V_{T}$ by taking
the expectation of the marginal distribution by averaging across particles
weighted by their likelihood $l$:
$\hat{V}_{T,\mu}^{T}=E[\hat{V}_{T}^{T}|\vec{D}]=\frac{1}{M}\sum_{m=1}^{M}(\hat{V}_{T,m}^{T}*l_{m})$,
where $m$ indexes the particles. This $\hat{V}_{T,\mu}^{T}$ is the final
estimate of the belief about the true $V$ after all detections have been
observed. Given $\hat{V}_{T,\mu}^{T}$, the posterior predictive distribution
is defined as:

$\begin{split}&P\left(\hat{\vec{W}}|\vec{D},\vec{cj},\vec{V}=\hat{V}_{T,\mu}^{T}\right)\propto\\\\[-5.0pt]
&\prod_{t=1}^{T}P\left(D_{t}|\hat{W}_{t},cj_{t},V_{t}=\hat{V}_{T,\mu}^{T}\right)P\left(\hat{W}_{t}\right)\end{split}$

This posterior predictive distribution can be used to better infer world
states for novel scenes, $W_{T+1},...$ or to reassess previous world states
$W_{1},...,W_{T}$ that were originally inferred using a less informed
$\hat{V}$.

##  4 Experiments

Our experiments have two goals: first, to test whether MetaCOG can learn an
accurate meta-cognitive representation of an object detector, and second, to
explore whether this learning confers any benefits in robustness and overall
accuracy. Exp. 1 tests MetaCOG’s performance when processing outputs of three
popular object detection systems (Section 4.1). Exp. 2 presents a robustness
analysis of how MetaCOG performs as a function of an object detector’s
baseline performance (Section 4.2). Throughout, we evaluate MetaCOG by
sampling scenes (arrangements of objects in a room), and then sampling images
taken from different viewpoints from each scene (calling the collection of
images from a scene called a “video”). All materials are available:
https://osf.io/8b9qt/?view˙only=8c1b1c412c6b4e1697e3c7859be2fce6.

###  4.1 Experiment 1: Enhancing neural networks for object detection with a
meta-cognition

Figure 2: Results for MetaCOG and comparison models. Throughout, yellow codes for RetinaNet, magenta for Faster R-CNN, and blue-grey codes for DETR. A) Scatterplot showing MetaCOG’s inferred values for the hallucination rates (circles) and miss rates (triangles) against the ground-truth values. B) The MSE (averaged across categories) of MetaCOG’s inferences about $\theta$ as a function of number of videos observed. C) Comparisons between MetaCOG and the two baseline models on the test set (after conditioning on the meta-cognitive representation that MetaCOG learned on the training set). The left group of bars show the difference between MetaCOG and the NN’s output, and the right shows the difference between MetaCOG and the lesioned model. Positive values indicated MetaCOG outperforms the comparison model. Object Detector | Model | Acc. Training | Acc. Test
---|---|---|---
RetinaNet | MetaCOG |  0.72 (0.66, 0.79) |  0.72 (0.65, 0.77)
| NN Output | 0.56 (0.52, 0.61) | 0.54 (0.50, 0.57)
| Lesioned MetaCOG | 0.64 (0.56, 0.71) | 0.56 (0.50, 0.63)
Faster R-CNN | MetaCOG |  0.79 (0.73, 0.84) |  0.76 (0.71, 0.82)
| NN Output | 0.75 (0.70, 0.80) | 0.69 (0.64, 0.74)
| Lesioned MetaCOG | 0.66 (0.59, 0.73) | 0.58 (0.52, 0.64)
DETR | MetaCOG | 0.32 (0.25, 0.39) |  0.32 (0.24, 0.39)
| NN Output | 0.16 (0.13, 0.18) | 0.16 (0.13, 0.18)
| Lesioned MetaCOG |  0.36 (0.29, 0.44) | 0.31 (0.24, 0.38)
Table 1: Accuracy results for Exp. 1. Values in parentheses are 95%
bootstrapped confidence intervals.

##### Object detection models

To test MetaCOG’s capacity to learn and use a meta-cognition, we tested its
performance when paired with three modern neural networks for object
detection. These networks represent three popular architectures: RetinaNet, a
one-stage detector (Lin et al. 2017); Faster R-CNN, a two-stage detector (Ren
et al. 2015); and DETR, a vision transformer (Carion et al. 2020), all pre-
trained. The networks were validated to ensure that their baseline
performances on our dataset were within their expected ranges. See A.3 for
details.

##### Dataset

We evaluate MetaCOG on a dataset rendered in the ThreeDWorld physical
simulation platform (Gan et al. 2020) using one canonical object model per
category. First, 100 scenes were generated by sampling objects and placing
them in a room with carpeting and windows. For each scene, we then generated a
video by sampling 20 frames from the ego-centric perspective of an agent
moving around the room. The resulting set of 100 videos was then randomly
split into a training and test set (each with n=50 videos). To avoid order
effects, all reported results show averages across four different video
orders. See A.2 for the motivation for this dataset, sample images, and
further details.

##### Comparison models

We compare MetaCOG to two baselines in order to test if the learned meta-
cognition improves performance. First, we compare MetaCOG’s inferences against
the neural network’s post-processed detections, which served as input to
MetaCOG (see A.3 for details). Second, it is possible that the computational
structure of MetaCOG might improve accuracy without meta-cognition mattering
per se. For instance, mapping detections to 3D representations with object
permanence alone might improve accuracy. Alternatively, the meta-cognitive
representation might provide tolerance for suppressing hallucinations or
recovering missed objects, but the exact learned meta-cognition (i.e., the
precise values of the parameters $\theta$ in $V$) might not matter. To test
these possibilities, we compare our results against a Lesioned MetaCOG, where
the values of $\theta$ were set to the mean of the initial prior over $\theta$
(see A.5). Although this model has a meta-cognitive representation $V$, the
values of its parameters $\theta$ are neither learned nor updated in light of
observations. Since this lesioned model contains the same representations as
the full model, it serves as a control for model complexity.

##### Results

We first assessed MetaCOG’s ability to learn an accurate meta-cognitive
representation $V$ (see A.4 for operationalization). Fig. 2A shows the
relationship between the parameters in MetaCOG’s inferred $\hat{\theta}$ and
each network’s true $\theta$, with an overall correlation of $r=0.878$. Fig.
2B shows the learning trajectory, visualizing the MSE of $\hat{\theta}$ for
each network as a function of the number of observed videos. After the 50
training videos, MSEs decreased, on average, by 96.1% of their initial values,
and had already decreased by 94.9% after only 20 videos. This demonstrates
that MetaCOG can learn an accurate meta-cognition $V|\theta$ efficiently and
without access to the ground-truth objects.

Table 1 and Fig. 2C show MetaCOG’s accuracy relative to our comparison models
(see A.8 for operationalization). For all three neural networks, MetaCOG
outperformed their outputs, and for two of the three neural networks, MetaCOG
also outperformed the lesioned model. For DETR, we did not see a significantly
stronger accuracy benefit in MetaCOG against the lesioned version, perhaps
because Lesioned MetaCOG was already performing well due to a coincidence
where the priors over $\theta$ gave an adequate approximation of DETR’s
performance in this dataset (see A.5 for further discussion). Averaged across
NNs, MetaCOG increased accuracy by $13.9\%$ relative to the NN outputs in the
test set. On average, MetaCOG also increased accuracy by $11.4\%$ relative to
Lesioned MetaCOG, confirming that MetaCOG’s success can be partially
attributed to its learned meta-cognition, rather than purely to the 3D object
representation and priors (see A.5 for additional results showing that
MetaCOG’s inferences about the presence and locations of objects at the level
of 3D scenes rather than 2D images also outperform those of Lesioned MetaCOG.
Also see A.5 for results on an additional dataset).

While MetaCOG and Lesioned MetaCOG were matched in architecture, computational
complexity, and input data, MetaCOG and NN Output were not. This raises the
possibility that MetaCOG’s success relative to the neural network’s outputs is
due to the additional structure, computational complexity, and access to
camera trajectories. To create a like-to-like comparison, we trained Faster
R-CNN (initialized to its pre-trained weights) using MetaCOG’s inferences.
Because MetaCOG infers what is where in 3D space, these inferences can be
projected back onto the 2D images and treated as a new synthesized training
set to train the neural network (all without access to ground-truth object
labels; see A.5 for details on the training procedure). Faster R-CNN trained
on MetaCOG’s inferences performed better than the network had off-the-shelf
(Fig. 3, comparing Pre-trained NN to Re-trained NN). Beyond controlling for
computational complexity, these results serve as a proof-of-concept that
MetaCOG can train an object detection system. Furthermore, the improved
detections from re-trained Faster R-CNN can be input into MetaCOG again. With
better inputs, MetaCOG made even better inferences (MetaCOG Round II in Fig.
3), demonstrating MetaCOG’s potential to become a self-supervised learning
system.

Figure 3: The results of MetaCOG training Faster R-CNN. Each point shows the
average accuracy of a model on the test set, and the bars are bootstrapped 95%
CIs. From left to right, the leftmost point shows the accuracy of pre-trained,
off-the-shelf Faster R-CNN. The next point shows the accuracy of MetaCOG, when
paired with Faster R-CNN’s inputs. The difference between these two points is
depicted by the magenta $\Delta$(MetaCOG,NN) bar in Fig. 2C. The third point
shows the accuracy of Faster R-CNN after re-training using MetaCOG’s
inferences. The rightmost point shows the accuracy of MetaCOG with inputs from
re-trained Faster R-CNN. For exact accuracy values, see Table 5 in A.5.

###  4.2 Experiment 2: Robustness analysis

Figure 4: Lightweight MetaCOG’s average performance over 40000 simulated
detectors varying in faultiness, each processing 75 world states. A) MSE
between true and inferred hallucination and miss rates as a function of number
of processed world states. Horizontal dotted line represents the average MSE
for the mean of the prior (the $\theta$ used in Lesioned MetaCOG). B) Mean
accuracy as a function of number of processed world states. Average accuracy
of (C) and difference between (D) MetaCOG and Lesioned MetaCOG as a function
of faultiness ($\zeta$) in the detections.

The results of Exp. 1 show that MetaCOG can efficiently learn a meta-cognition
for an object detector and use it to improve accuracy. This provides proof-of-
concept that MetaCOG can be applied to modern neural network object detection
systems. However, the performance of these three networks does not capture the
full space of possible object detector performances. How robust is MetaCOG to
different degrees of error in its inputs?

To evaluate MetaCOG’s robustness, Exp. 2 tests MetaCOG paired with a large
range of possible simulated object detectors. To achieve this, we created a
synthetic dataset of world states passed through simulated faulty object
detectors. To reduce computational cost, we used an abstraction of world
states and detections (removing all spatial components, detailed below) and
used a lightweight, general-purpose version of MetaCOG (see A.6) to infer a
meta-cognition of the synthetic object detectors and the world states causing
the observations.

##### Dataset

To focus on the contribution of meta-cognition for determining the presence or
absence of different object categories in a way that is robust to the failures
of the object detector, the dataset consisted of hypothetical collections of
objects with no spatial information, passed through simulated object detectors
with varying degrees of faulty performance. Specifically, we sequentially
sampled world states (vectors of 1s and 0s indicating the presence or absence
of five possible object categories), object detectors (miss and hallucination
distributions $V$ with a wide variety of parameters $\theta$), and then
generated faulty detections (vectors of 1s and 0s indicating the detection or
lack of detection of each object category) by passing world states through the
probabilistic artificial object detector. The final dataset consisted of 40000
randomly sampled object detectors, each processing 75 world states, with 5-15
processed frames (detection vectors) per world state. The details of
synthesizing this dataset are left to A.7.

##### Comparison models

As in Exp. 1, we compare MetaCOG to Lesioned MetaCOG, which fixes the
parameters $\theta$ of the meta-cognitive representation $V$ to the
expectation of the prior over $\theta$. This enables a fair comparison where
the two models are matched for complexity. For details on the prior, see A.6,
and for model details, see A.9.

##### Results

Here, we demonstrate that, as in Exp. 1, MetaCOG can learn a detector’s
hallucination and miss rates without feedback, and that its improvements in
accuracy track with its learning of a meta-cognitive representation. See A.8
for definitions of the metrics used.

Fig. 4 shows MetaCOG’s performance over the 40000 sampled object detectors. To
assess MetaCOG’s ability to learn an accurate meta-cognitive representation
$V|\theta$, we examined the MSE of $\hat{\theta}$ as a function of the number
of observed videos. Fig. 4A shows that MetaCOG’s estimates $\hat{\theta}_{t}$
of the hallucination and miss rates rapidly approach the true values $\theta$,
with a final average MSE of 0.0017. To test if learning varied as a function
of the object detector’s errors, we next calculated MSE as a function of the
detector’s faultiness $\zeta$, defined as the detector’s average proportion of
errors per scene (see A.8 for formal definition). A linear regression
predicting MSE as a function of faultiness revealed that MetaCOG’s learned
$\hat{\theta}$ is less accurate for faultier detectors ($\beta=0.002$;
$p<0.001$), although the effect was minimal (predicting a MSE increase of
$0.001$ from a perfect detector with faultiness 0 to a detector with
faultiness 0.5, where detections are equally likely to be true or false).
Together, these analyses confirm that the results of Exp. 1 hold for a wider
set of object detector performances.

Fig. 4B shows the average accuracy of the models’ inferences about world
states. The yellow line shows MetaCOG’s rapid increase in accuracy over the
first 40 world states. During these world states, MetaCOG’s accuracy increases
from 88.8% to 93.6% (with 93.8% final accuracy after the 75th world states).
This increase in accuracy occurs simultaneously with the decline in MSE for
the inferred parameters of the meta-cognitive representation. By comparison,
the blue line shows Lesioned MetaCOG’s performance, which showed an average
performance of 91.2%. The green line shows MetaCOG’s accuracy after
conditioning on the parameters $\hat{\theta}_{T}$ of meta-cognition learned
after observing the detections from all $T$ world states (see A.9 for
details).

To quantify MetaCOG’s robustness to faulty inputs, we next computed accuracy
as a function of the object detector’s faultiness $\zeta$ on a range from 0
(perfect performance) to 0.5 (detections equally likely to be true or false).
Figure 4C shows MetaCOG’s and Lesioned MetaCOG’s mean accuracies as a function
of $\zeta$. When faultiness is low, MetaCOG and Lesioned MetaCOG performed
near ceiling, as the object detector’s output is already highly reliable.
However, MetaCOG outperformed Lesioned MetaCOG in accuracy for detectors with
faultinesses in $\zeta\in[0.12,0.5]$.

Fig. 4D shows the average difference in accuracy between MetaCOG and Lesioned
MetaCOG as a function of faultiness. MetaCOG reaches its highest accuracy
boost over Lesioned MetaCOG at faultiness level 0.42 (with a 21.8%
improvement), and consistently shows a performance boost across a wide range
of faultiness values. Together, these results show that MetaCOG’s success is
not limited to the particular neural networks tested in Exp. 1. Instead,
MetaCOG can learn a good meta-cognitive representation for a wide range of
detectors and use it to improve accuracy.

##  5 Discussion

In order to behave intelligently in a complex and dynamic world, autonomous
systems must be able to account for inevitable errors in perceptual
processing. In humans, meta-cognition provides this critical ability. Inspired
by human cognition, we formalize meta-cognition and apply it to the domain of
object detection. Our work is a proof-of-concept of how a meta-cognitive
representation can improve the robustness of object detectors. The MetaCOG
model presented here can be directly applied to embodied systems, and MetaCOG
may be particularly useful in situations where the vision system may be
unreliable, like if the agent is deployed in an environment that was not
adequately represented in training. Outside of this specific use-case, the
more general MetaCOG approach can support robust AI more broadly.

First, the MetaCOG approach provides a way to quantify uncertainty, even for a
pre-trained system and when ground-truth is not accessible. Sudden changes in
the uncertainty expressed in the meta-cognitive representation could be used
to detect domain or distribution shifts, so as to flag situations of high
uncertainty that pose increased risk of unreliable behavior. This could be
useful for downstream decision-making processes determining when to halt
action.

Second, the MetaCOG approach can be used to tune and improve the underlying
system that it is monitoring, as when MetaCOG trained Faster R-CNN (Fig. 3).
Using meta-cognition to generate a training signal is a novel alternative to
fine-tuning that may enable robustness to distribution shifts. In domains
where the ground-truth labels required for traditional fine-tuning are
difficult to acquire, the MetaCOG approach could prove especially valuable.

Finally, our work focused on learning a meta-cognition for the purpose of
improving accuracy and robustness, but the meta-cognitive representation also
has potential benefits for transparency and interpretability. In our
implementation, MetaCOG’s $V$ captures the object detector’s performance in a
way that is easy for humans to understand. As such, this approach of learning
a meta-cognitive representation may be able to support explainable AI by
generating simplified meta-representations of a black-box system’s
performance.

##  6 Conclusion

We proposed a formalization of meta-cognition for object detectors that
increases robustness by removing hallucinations and filling in missed objects.
Our model, MetaCOG, learns a probabilistic relation between detections and the
objects causing them (thereby representing the detector’s performance and
instantiating a form of meta-cognition). This is achieved as joint inference
over the objects and a meta-cognitive representation of the detector’s
tendency to error. Critically, MetaCOG performs this inference without
feedback or ground-truth object labels, instead using cognitively-inspired
priors about objects. Applying MetaCOG to three modern neural object detectors
(Exp. 1) and to simulated detectors (Exp. 2) showed that MetaCOG can
efficiently learn an accurate meta-cognitive representation for a wide range
of detectors and use it to account for errors, correctly inferring the objects
in a scene in a way that is robust to the faultiness of the detector. This
work is a proof-of-concept of that meta-cognition is a promising approach to
improving robustness in computer vision and perhaps even beyond.

## References

  * Babu and Suresh (2012) Babu, G. S.; and Suresh, S. 2012.  Sequential projection-based metacognitive learning in a radial basis function network for classification problems.  _IEEE transactions on neural networks and learning systems_ , 24(2): 194–206.
  * Canini, Shi, and Griffiths (2009) Canini, K.; Shi, L.; and Griffiths, T. 2009.  Online inference of topics with latent Dirichlet allocation.  In _Artificial Intelligence and Statistics_ , 65–72.
  * Carey (2009) Carey, S. 2009.  _The origin of concepts_.  Oxford university press.
  * Carion et al. (2020) Carion, N.; Massa, F.; Synnaeve, G.; Usunier, N.; Kirillov, A.; and Zagoruyko, S. 2020.  End-to-End Object Detection with Transformers.  In Vedaldi, A.; Bischof, H.; Brox, T.; and Frahm, J.-M., eds., _Computer Vision – ECCV 2020_ , 213–229. Cham: Springer International Publishing.  ISBN 978-3-030-58452-8.
  * Chen et al. (2022) Chen, H.; Venkatesh, R.; Friedman, Y.; Wu, J.; Tenenbaum, J. B.; Yamins, D. L.; and Bear, D. M. 2022.  Unsupervised segmentation in real-world images via spelke object inference.  In _European Conference on Computer Vision_ , 719–735. Springer.
  * Cusumano-Towner et al. (2019) Cusumano-Towner, M. F.; Saad, F. A.; Lew, A. K.; and Mansinghka, V. K. 2019.  Gen: A General-purpose Probabilistic Programming System with Programmable Inference.  In _Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation_ , PLDI 2019, 221–236. New York, NY, USA: ACM.  ISBN 978-1-4503-6712-7.
  * Doucet, Johansen et al. (2009) Doucet, A.; Johansen, A. M.; et al. 2009.  A tutorial on particle filtering and smoothing: Fifteen years later.  _Handbook of nonlinear filtering_ , 12(656-704): 3.
  * Firestone and Scholl (2016) Firestone, C.; and Scholl, B. J. 2016.  Cognition does not affect perception: Evaluating the evidence for “top-down” effects.  _Behavioral and brain sciences_ , 39.
  * Gan et al. (2020) Gan, C.; Schwartz, J.; Alter, S.; Schrimpf, M.; Traer, J.; De Freitas, J.; Kubilius, J.; Bhandwaldar, A.; Haber, N.; Sano, M.; et al. 2020.  Threedworld: A platform for interactive multi-modal physical simulation.  _arXiv preprint arXiv:2007.04954_.
  * Garcez and Lamb (2020) Garcez, A. d.; and Lamb, L. C. 2020.  Neurosymbolic AI: the 3rd wave.  _arXiv preprint arXiv:2012.05876_.
  * Gothoskar et al. (2021) Gothoskar, N.; Cusumano-Towner, M.; Zinberg, B.; Ghavamizadeh, M.; Pollok, F.; Garrett, A.; Tenenbaum, J.; Gutfreund, D.; and Mansinghka, V. 2021.  3DP3: 3D scene perception via probabilistic programming.  _Advances in Neural Information Processing Systems_ , 34: 9600–9612.
  * Ivanovska et al. (2015) Ivanovska, M.; Jøsang, A.; Kaplan, L.; and Sambo, F. 2015.  Subjective networks: Perspectives and challenges.  In _International Workshop on Graph Structures for Knowledge Representation and Reasoning_ , 107–124. Springer.
  * Kaplan et al. (2018) Kaplan, L.; Cerutti, F.; Sensoy, M.; Preece, A.; and Sullivan, P. 2018.  Uncertainty aware AI ML: why and how.  _arXiv preprint arXiv:1809.07882_.
  * Kemp and Xu (2009) Kemp, C.; and Xu, F. 2009.  An ideal observer model of infant object perception.  In _Advances in Neural Information Processing Systems_ , 825–832.
  * Lake et al. (2017) Lake, B. M.; Ullman, T. D.; Tenenbaum, J. B.; and Gershman, S. J. 2017.  Building machines that learn and think like people.  _Behavioral and brain sciences_ , 40.
  * Lin et al. (2017) Lin, T.-Y.; Goyal, P.; Girshick, R. B.; He, K.; and Dollár, P. 2017.  Focal Loss for Dense Object Detection.  _2017 IEEE International Conference on Computer Vision (ICCV)_ , 2999–3007.
  * Lin et al. (2014) Lin, T.-Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Dollár, P.; and Zitnick, C. L. 2014.  Microsoft coco: Common objects in context.  In _European conference on computer vision_ , 740–755. Springer.
  * Nelson (1990) Nelson, T. O. 1990.  Metamemory: A theoretical framework and new findings.  In _Psychology of learning and motivation_ , volume 26, 125–173. Elsevier.
  * Nye et al. (2021) Nye, M.; Tessler, M.; Tenenbaum, J.; and Lake, B. M. 2021.  Improving coherence and consistency in neural sequence models with dual-system, neuro-symbolic reasoning.  _Advances in Neural Information Processing Systems_ , 34: 25192–25204.
  * Platt et al. (1999) Platt, J.; et al. 1999.  Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods.  _Advances in large margin classifiers_ , 10(3): 61–74.
  * Ren et al. (2015) Ren, S.; He, K.; Girshick, R.; and Sun, J. 2015.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.  In Cortes, C.; Lawrence, N.; Lee, D.; Sugiyama, M.; and Garnett, R., eds., _Advances in Neural Information Processing Systems_ , volume 28. Curran Associates, Inc.
  * Sensoy, Kaplan, and Kandemir (2018) Sensoy, M.; Kaplan, L.; and Kandemir, M. 2018.  Evidential deep learning to quantify classification uncertainty.  In _Advances in Neural Information Processing Systems_ , 3179–3189.
  * Shen et al. (2023) Shen, M.; Bu, Y.; Sattigeri, P.; Ghosh, S.; Das, S.; and Wornell, G. 2023.  Post-hoc Uncertainty Learning Using a Dirichlet Meta-Model.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 37, 9772–9781.
  * Smith et al. (2019) Smith, K.; Mei, L.; Yao, S.; Wu, J.; Spelke, E.; Tenenbaum, J.; and Ullman, T. 2019\.  Modeling Expectation Violation in Intuitive Physics with Coarse Probabilistic Object Representations.  In _Advances in Neural Information Processing Systems_ , 8983–8993.
  * Spelke and Kinzler (2007) Spelke, E. S.; and Kinzler, K. D. 2007.  Core knowledge.  _Developmental science_ , 10(1): 89–96.
  * Subramanian, Suresh, and Sundararajan (2013) Subramanian, K.; Suresh, S.; and Sundararajan, N. 2013.  A metacognitive neuro-fuzzy inference system (McFIS) for sequential classification problems.  _IEEE Transactions on Fuzzy Systems_ , 21(6): 1080–1095.
  * Tokmakov et al. (2021) Tokmakov, P.; Li, J.; Burgard, W.; and Gaidon, A. 2021.  Learning to track with object permanence.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 10860–10869.

##  Appendix A Technical Appendix

###  A.1 Additional MetaCOG details

The MetaCOG model and inference were implemented in the Julia-based
probablistic programming language Gen (Cusumano-Towner et al. 2019), licensed
under the Apache License Version 2.0.

#### Meta-cognitive dynamics kernel

Beliefs about each category’s hallucination rate $\lambda_{c}$ is represented
as a Gamma distribution, as Gamma is the conjugate prior of the Poisson
distribution representing hallucinations. At $t=0$, the parameters of each
Gamma prior are initialized to $\alpha=1$ and $\beta=1$ (representing a
relatively uninformative prior with a mean of 1). After inference over a
scene, the Gamma prior over $\lambda_{c}$ evolves by computing the inferred
number of hallucinations at time $t$ (based on the difference between world
state $W_{t}$ and associated detections $D_{t}$, see A.1), and updating the
$\alpha$ and $\beta$ parameters of the Gamma distribution.

Beliefs about the detection rates $p_{c}$ evolve in an analogous manner. As
Beta is the conjugate prior of the Geometric distribution representing
detections, beliefs about the miss rate evolve by updating the $\alpha$ and
$\beta$ parameters of the Beta distribution. At $t=0$, the Beta prior is
initialized with parameters $\alpha=1$ and $\beta=1$ (equivalent to a uniform
distribution on $[0,1]$, a maximally uninformative prior for detection rate).
The parameters from the Beta distribution are then updated based on the
inferred missed detections (by comparing $W_{t}$ against $D_{t}$, see A.1).

Figure 5: Schematic depicting the forward generative model. $G_{t}$ is the
prior over meta-cognition ($V_{t}$) at time t; $W_{t}$ is the world state; and
$D_{t}$ are the detections that are generated. $W_{t}$, $D_{t}$, and $G_{t}$
collectively influence $G_{t+1}$, the prior over $V_{t_{1}}$.

#### Details of procedure for taking the difference between $W_{t}$ and
$D_{t}$

From a world state $W_{t}$ and a corresponding collection of detections
$D_{t}$, MetaCOG tries to infer which detection were hallucinations or misses
so as to update the prior over the hallucination and miss rate. The procedure
for this inference is as follows: first, the objects in $W_{t}$ are projected
from 3D space to the 2D image. Then, each detection is matched to the nearest
projected object of the same category as long as the Euclidean distance
between the pair is within radius $\sigma=200$ pixels (equal to the spatial
noise parameter). After matching, the number of unmatched objects per category
are counted, and considered to have been missed. The number of unmatched
detections are also counted, and considered to be hallucinations. This process
informs the updating of the $\alpha$s and $\beta$s in the priors described
above.

#### Object constraints and generative model parameters

Implementation of object constraints were as follows. World states without
object persistence were not included in $\mathbb{W}$, which is equivalent to
implicitly setting their prior probability to 0. The assumption that two
objects cannot occupy the same region in space was implemented through a prior
over $\mathbb{W}$ where the probability of having one object near another
decreased to 0 according to a Gaussian distribution with $\sigma^{2}=1$.

To account for noise in a detector’s location detection $D_{t}$, each
detection was modeled as having 2D spatial noise, following a Gaussian with
$\sigma_{x,y}=200$ pixels.

Finally, the full generative model requires specifying a prior distribution
over camera positions and focal points (although these are observable), set as
uniform over 3D space, and a prior distribution over expected number of
objects in a scene, sampled from a geometric distribution with parameter
$p=0.9$ (with a uniform prior over category type).

#### Inference procedure implementation

We sequentially approximate the joint posterior given in 3.2 using a particle
filter with 100 particles. The solution space to the inference problem that we
consider is sparse, and Sequential Monte-Carlo methods can suffer from
degeneracy and loss of diversity in these situations. Our inference approach
solves these problems by implementing particle rejuvenation over objects,
locations, and beliefs about $V$. Rejuvenation is conducted using a series of
Metropolis-Hastings (Canini, Shi, and Griffiths 2009) moves with data-driven
proposals designed to obtain samples from
$Pr(\vec{V},\vec{W}|\vec{O},\vec{cj})$.

During object rejuvenation, a new state $W^{\prime}_{t}$ is proposed by either
adding or removing an object from the current state $W_{t}$. The probability
$p_{add}$ of proposing to add an object depends on the belief about $V_{t}$
and the observed detections. The number of detections across the five
categories are summed. Call this sum $k$. The probability $p$ of $k$ or more
detection being produced by hallucinations is calculated as
$p=1-e^{-\lambda_{total}}\sum_{i=0}^{k}\frac{\lambda_{total}^{i}}{i!}$, where
$\lambda_{total}=\sum_{c}(\lambda_{c})$. (That
$e^{-\lambda_{total}}\sum_{i=0}^{k}\frac{\lambda_{total}^{i}}{i!}$ term is the
cdf of $Pois(\lambda_{total})$ from $0$ to $k$.) $p_{add}$ should depend on
$p$ such that, when the belief about the overall hallucination rate is high
relative to the number of detections, the probability $p_{add}$ of proposing
adding a new object to the world state is low, but when the hallucination rate
is low relative to the number of detections, the probability $p_{add}$ of
proposing adding a new objects is high. However, if $p_{add}$ were to be set
to $p$, then there is a danger that, when there are more detections than can
be explained as hallucinations, the proposal function would only ever propose
adding objects, and never remove them. To balance the relative proposals of
adding and removing objects, we set $p_{add}=0.5*p$, effectively setting the
maximum of $p_{add}$ to 0.5.

Objects in $W_{t}$ have a weighted probability of being removed to produce
$W^{\prime}_{t}$, such that their probability of removal is inversely
proportional to the number of times the object was observed in scene $t$. New
objects are added based on a data-driven distribution. This distribution
samples object categories from a categorical distribution biased towards
categories observed in the current scene $t$ (see Data-driven proposal
functions below). With probability 0.5, location is sampled from a 3-D uniform
distribution, and otherwise sampled from a data-driven function with location
biased toward 3D points likely to have caused the 2D detections. The proposal
for adding a new object or removing and existing object from the world state
is accepted or rejected according to the MH algorithm.

After a proposed change to $W$, a second rejuvenation step is performed on
locations, wherein an object in $W$ is randomly selected (with equal
probability) to have a new location proposed. With probability 0.5, the new
location is drawn from a multivariate normal distribution centered on the
previous location, and it is otherwise sampled from a data-driven proposal.
The proposed world-state $W^{\prime}$ with a perturbed location is then
accepted or rejected according to the MH algorithm.

Finally, new parameters $\theta^{\prime}$ for the meta-cognition are proposed
by perturbing $\theta$. Each parameter in the pair of length $|C|$ vectors is
perturbed, with new values generated for $\theta(i,j)_{t}$ and for
$\theta(i,j)_{t-1}$ $\forall i,j$, where $i$ indexes the two vectors and $j$
indexes their elements. The new value for each $\theta(i,j)_{t}$ is sampled
from the appropriate distribution (Beta for the miss rate, Gamma for
hallucination rate) with $\alpha_{t}$ and $\beta_{t}$, while the new value for
$V(i,j)_{t-1}$ is sampled with $\alpha_{t-1}$ and $\beta_{t-1}$. These new
values are accepted or rejected according to the MH algorithm.

This three-step rejuvenation process for world states, locations, and
parameters of the meta-cognition is done for each particle for 200 iterations
and the last state reached in the chain is used as the new rejuvenated
particle.

#### Data-driven proposal functions

During rejuvenation, new objects are proposed to be added to $W_{t}^{\prime}$
using a data-driven distribution. The category of the object comes from a
categorical distribution. A category’s weight in the categorical distribution
depends on the number detections of that category in the current scene, and on
the belief about the probability that those detections reflect a real object
(as opposed to a hallucination). The meta-cognition can be used to calculate
belief about the probability that this detection was generated by a real
object. By Bayes rule, $P(real|obs)=P(obs|real)*P(real)/P(obs)$. Under the
prior, each category has the same probability of appearing, so $P(real)$ will
not affect the relative weighting of the categories. $P(obs|real)$ is simply
the probability of detecting an object when it’s present, $p_{c}$, for that
category. $P(obs)$ is the probability of detecting an object of category c.
Since a detection must be either true or a hallucination,
$P(obs)=P(obs|real)+P(obs|not\;real)=p_{c}+(1-e^{-\lambda_{c}})$, where the
$(1-e^{-\lambda_{c}})$ term is the probability of hallucinating one or more
objects of category $c$. All together, the probability of sampling category
$c$ is proportional to $\frac{k_{c}*p_{c}}{p_{c}+(1-e^{-\lambda_{c}})}$, where
$k_{c}$ is the number of detections of category $c$ in this scene. So that its
possible (though unlikely) to sample an object from an unseen category,
$k_{c}$ is set to have a minimum of one. This probability distribution for
proposing object categories incoorporates both the number of times a category
is detected, and the meta-cognitive belief about whether those detections were
a mere hallucinations or generated from a real object.

In addition to a category, a location must be sampled for the proposed object.
With probability 0.5, the location is sampled from a 3-D uniform distribution
over the whole scene, and otherwise sampled from a data-driven function. Using
the data-driven function, a point is sampled based on proximity to the line-
segment that, when projected onto the 2D image, would result in the point
where the detection was observed. The probability of proposing a particular
point decreases with the distance from this line segment, following a Gaussian
with $\sigma^{2}=0.01$.

In the second rejuvenation step a new location is proposed for an object. With
probability 0.5, the new location is drawn from a multivariate normal
distribution centered on the previous location, with $\sigma_{x,y,z}=0.01$.
Otherwise, it is sampled as described in the previous paragraph (based on
proximity to the line segment that would results in the detection’s 2D
location).

###  A.2 Experiment 1 Dataset

Evaluating Exp. 1 we needed a dataset of videos of the same COCO object
categories (so pre-trained NNs could detect them) across multiple videos.
These videos had to include the camera trajectory and position. Furthermore,
so as to enable use to assess MetaCOG’s accuracy locating objects in 3D space,
we also needed ground-truth labels for the objects, including their locations
in 3D space. Because we did not find any existing datasets that satisfy all
these constraints, we rendered our own, custom dataset.

#### TDW scene rendering

The TDW environment is licensed under the BSD 2-Clause “Simplified” License,
Copyright 2020 Massachusetts Institute of Technology. We used the “box room
2018” model with a footprint with dimensions $\approx 12$ x $8$. For scale,
the largest object model is about 1 unit along any face.

The number of objects in the scene is uniformly drawn from the counting
numbers up to 3 and each object is uniformly assigned one of 5 object
categories: potted plant, chair, bowl, tv, or umbrella. We sequentially place
objects at locations drawn from a uniform distribution, resampling if two
objects are within 1 units of each other or the walls so as to avoid
collisions.

#### Camera trajectory sampling

A camera trajectory consists of a sequence of camera positions and focal
points. We generate frames by querying the camera for an image at 20 linearly-
spaced times.

Camera trajectories consisted of a circular path around the periphery of the
room with noise, generated by a Gaussian process with a radial basis function
(RBF) kernel with parameters $\sigma=0.7$ and $\ell=2.5$. The height of the
camera is held constant at $y=2$. The focal point trajectory is also sampled
from a Gaussian process with an RBF kernel, mean above the center of the floor
and component-wise parameters $\sigma=0.7$ and $\ell=2$. Fig. 6 shows some
example trajectories.

Figure 6: Three sampled trajectories. The wide circular patterns at the top
are camera positions, and the smaller patterns near the bottom are camera
focal points.

The resulting images include objects passing in and out of frame and sometimes
partly occluding each other. Fig. 7 shows five images from a scene in the
dataset.

Figure 7: Five images from a scene in the dataset. The scene has three objects
– a potted plant, a TV, and an umbrella. In the fourth image, the plant partly
occludes the TV, and in the last image, the umbrella is out of view. The rest
of the dataset and demo videos can be found at:
https://osf.io/8b9qt/?view˙only=8c1b1c412c6b4e1697e3c7859be2fce6

#### Video counter-balancing

To avoid order effects, all tests were run with four different videos orders.
Videos were first randomly labeled from 1 to 50, and the four counterbalanced
orders were: {1 … 50}, {50 … 1}, {26 … 50, 1 … 25}, and {25 … 1, 26 … 50}.

###  A.3 Details About Neural Networks Used in Experiment 1

#### Pre-trained weights

For Faster R-CNN, we used the resnet50 pre-trained weights. For RetinaNet, we
used the resnet50 pre-trained weights. And for DETR, we used the resnet101
pre-trained weights. These NNs were pre-trained on the COCO dataset.

Faster R-CNN is licensed under The MIT License (MIT), RetinaNet under Apache
License 2.0, and DETR under Apache License 2.0.

#### Post-processing

We process the outputs of the object detectors by filtering for the 5 object
categories present in the scenes and then performed Non-Maximum Suppression
(NMS) with an IoU (Intersection over Union) threshold of 0.4 (applied only for
RetinaNet and Faster R-CNN, as it is not typically used for DETR). As input to
MetaCOG, we took the top five highest-confidence detections per frame. The
reason for taking the top five highest-confidence detection per frame, rather
than all of the detections per frame is that sometimes some networks,
especially DETR, can output many detections for a single frame. Typically, a
confidence threshold is selected by fitting to ground-truth labels. But our
setting is unsupervised and meant to apply to situations where ground-truth
labels are perhaps unavailable. So, instead of fitting the confidence
threshold using ground-truth, we took the top five highest-confidence
detections. On most frames and with most networks, the NNs rarely output more
than 5 detections, so in practice, taking the top 5 detections rarely affected
the NN’s output. DETR is the exception, and output more than 5 detections on
many frames, resulting in its relatively poor baseline performance, as can be
seen in Table 1.

#### mAP of NNs on our dataset

To demonstrate that the neural networks are performing reasonably on our
dataset, we calculated mAP values for the neural networks when applied to our
dataset. We find that the object detectors perform similarly on our dataset as
on the datasets on which they are typically evaluated. To show this, we
evaluated the object detectors using the standard COCO evaluator (Lin et al.
2014). On the dataset used in Exp. 1, Faster R-CNN has a mAP of 39.4 on the
training set, and 41.1 on the test set. This is similar to the mAPs reported
for the most challenging object categories. For instance, Faster R-CNN’s
reported mAP for the plant category is 39.1 and 40.1 on the PASCAL VOC 2007
and 2012 test sets, respectively (Ren et al. 2015). RetinaNet had a mAP of
35.9 on the training set, and 36.3 on the test set. RetinaNet’s reported mAP
(across all object categories) on the COCO test dataset is 39.1 (Lin et al.
2017), so RetinaNet performs similarly on our dataset as on COCO. DETR had a
mAP of 0.478 on the training set, and 0.492 on the test set. DETR’s reported
mAP on the COCO dataset is 43.5 (Carion et al. 2020). This confirms that the
object detectors are operating within their expected performance ranges.

###  A.4 Metrics for Experiment 1

We evaluate MetaCOG’s performance in three ways: 1) by testing whether it can
learn an accurate meta-cognition of the object detection module, 2) by testing
whether the learned meta-cognition improves inferences about which objects are
where in 3D space, and 3) by testing whether the learned meta-cognition leads
to improved accuracy at detecting objects in the 2D images.

#### MSE for $\theta$

To measure how well MetaCOG learned a meta-cognition, we calculated the MSE
between the inferred parameters $\hat{\theta}$ of the meta-cognition and the
true parameters $\theta$. The true $\theta$ was calculated for each object
detection system as described in the following section A.4. The squared error
was calculated for corresponding values in $\hat{\theta}$ and the true
$\theta$, then summed and divided by the total number of values ($2*|C|$).

#### Calculating ground-truth $\theta$

We needed a way, for a given image, to calculate how many times an object of
each category was missed or hallucinated by the object detector. There are
many possible ways to define and calculate ground-truth misses and
hallucinations, but we chose a definition and procedure to match the
definition and procedure MetaCOG uses (see A.1).

We used the 3D position of the object, the camera, and the camera’s focus to
determine whether and where the object would appear in the image. We
considered an object to be missed if the object was in the image, but not
detected within 200 pixels of its projected 2D location on the image. We
calculated the miss rate for a category $c$ as the number of misses divided by
the number of times an object of category $c$ was in view.

By hallucination, we mean a detection of an object of a particular category
that was not caused by a ground-truth object of that category. To
operationalize this, we considered a detection to be a hallucination if the
detection occurred more than 200 pixels away from the projection of a ground-
truth object of that category. We calculated the hallucination rate for a
category $c$ by counting the number of times an object of category $c$ was
hallucinated in our dataset, and then dividing by the total number of frames
in the dataset.

The 200 pixel radius was set to match the radius used by MetaCOG in matching
$W_{t}$s to $D_{t}$s (see A.1).

#### Jaccard similarity

The Jaccard similarity coefficient is a useful metric for evaluating
inferences about which objects are in a frame or scene. The Jaccard similarity
coefficients measures the similarity between two sets by taking the size of
the intersection of two sets and dividing it by the size of the union of the
two sets. We can apply this measure to our setting by treating the object
labels that were inferred as one set and the objects that were actually
present as the second set. The Jaccard similarity coefficient is given by
$J(\mathbb{I},\mathbb{G})=\frac{|\mathbb{I}\cap\mathbb{G}|}{|\mathbb{I}\cup\mathbb{G}|}$,
where, in our setting, $\mathbb{I}$ is the set of object labels that were
inferred (i.e. $\mathbb{I}$ = {chair, chair, bowl}) and $\mathbb{G}$ is the
set of objects in ground-truth (i.e. $\mathbb{G}$ = {chair, bowl}). (In this
example, $J(\mathbb{I},\mathbb{G})=\frac{2}{3}$.)

This Jaccard Similarity coefficient can be applied either at the frame or
scene level, and is used in both contexts in this paper.

#### 2D inference accuracy

To assess the accuracy of of the both MetaCOG and the NNs in a comparable way,
we wanted an accuracy metric that could incooporate both which object
categories are present, and where they are. The NNs operate over 2D images,
not 3D spaces, so we must make this comparison at the level of 2D frames. The
standard metric for object detection accuracy on images is mean Average
Precision (mAP), which requires both a ground-truth 2D bounding box and an
inferred/detected bounding box. Unfortunately, MetaCOG outputs a point in 3D
space, which can be projected to a point in 2D space, but cannot be used to
make a bounding box. But we still needed a metric by which to compare
MetaCOG’s object detection accuracy to those of the neural networks without a
meta-cognition.

To solve this problem, we projected MetaCOG’s inferences about the 3D location
of objects to a point on each 2D image, and for the neural networks, we took
the centroid of the bounding boxes. This way, MetaCOG and the neural networks’
detections have the same format: an object label and a point on the image, or
a tuple of $(c,x,y)$. Then, for each object category, we used the Hungarian
algorithm with Euclidean distances as cost to pair up the detections with the
centroid of the ground-truth bounding box. For each pairing, we simply coded
whether or not the detected point was within the ground-truth bounding box as
$0$ or $1$. Last, we calculated the Jaccard similarity coefficient (see A.4)
defining pairs as within the intersection of the sets if and only if they pair
received the $1$ coding. In other words, we counting up the number of pairs
where the detection lay within the ground-truth bounding box, and divided by
and the union of the detection and ground-truth sets: the sum of the number of
unpaired detections, unpaired ground-truth bounding boxes, and number of total
pairings. To extrapolate this accuracy metric from frames to videos, we simply
average across frame accuracy for each frame in the video, and we average
across videos to calculate overall accuracy. This accuracy metric encodes, for
2D frames, accuracy at detecting both which objects are present and where they
are.

It is important to note that assessing accuracy at the 2D frame level is
different from assessing accuracy at the 3D scene level. Accuracy at the 2D
level should depend on the objects that are in the frame. For example, it
could be the case that the scene contains a chair and a plant, but the chair
is only in frame once, while the plant is in frame 20 times. In that case, 2D
accuracy should depend more on identifying and locating the plant than the
chair. This distinction will be important when we assess accuracy at the level
of the 3D scene in the next section.

###  A.5 Experiment 1 Supplemental Results and Details

#### 3D inference accuracy

Object Detector | Model | Acc. Training | Acc. Test
---|---|---|---
RetinaNet | MetaCOG |  0.66 (0.58, 0.75) |  0.60 (0.51, 0.68)
| Lesioned MetaCOG | 0.58 (0.49, 0.66) | 0.48 (0.41, 0.55)
Faster R-CNN | MetaCOG |  0.76 (0.67, 0.84) |  0.66 (0.58, 0.74)
| Lesioned MetaCOG | 0.58 (0.49, 0.66) | 0.49 (0.43, 0.56)
DETR | MetaCOG | 0.32 (0.26, 0.38) |  0.35 (0.29, 0.41)
| Lesioned MetaCOG |  0.38 (0.30, 0.46) | 0.34 (0.28, 0.41)
Table 2: Scene-level accuracy results for Exp. 1. Values in parentheses
represent 95% bootstrapped confidence intervals.

Our main results focused on MetaCOG’s accuracy in 2D images. This was
necessary so that we could evaluate its inferences against NN Output, because
the object detection neural networks that we use only provide location on 2D
images. Although not our main focus, here we report supplemental results
evaluating MetaCOG’s capacity to infer 3D world states in Exp. 1. This allows
us to test MetaCOG’s inferences at the scene level (i.e., what objects are in
the room and where are they?), rather than at the frame level (i.e., what
objects are visible at this time point and where are they in this 2D
projection?). Throughout, we compare MetaCOG only to Lesioned MetaCOG, as it
is the only comparison model that also produces results in 3D space.

We first evaluate MetaCOG’s accuracy in determining which objects are present
in a scene (independent of location). To do this, we calculated the Jaccard
similarity coefficent (see A.4) between the object categories inferred to be
present in the scene, and those actually present. Table 2 shows Jaccard
similarity coefficient averaged across the scenes. We find that on the whole,
MetaCOG outperforms Lesioned MetaCOG, except for when paired with DETR, where
the two perform comparably (as can be observed by noting that the confidence
interval of each model’s accuracy contains the mean of the other, indicating
that there is no significant different between the means).

Next, we evaluate MetaCOG’s accuracy in determining where the objects are in
the scene. To do this, we first paired the inferred and ground-truth object
locations of the same category using the Hungarian algorithm with Euclidean
distance as the cost. We then averaged the distance between all of the pairs,
in both the training and test sets, resulting in an average distance between
inferred and ground-truth objects. The results in are shown in Table 3, in the
Avg. Distance All Pairs column.

Object Detector | Model | Avg. Distance All Pairs | Avg. Distance Shared Pairs
---|---|---|---
RetinaNet | MetaCOG |  0.37 (0.31, 0.42) |  0.30 (0.26, 0.34)
| Lesioned MetaCOG | 0.58 (0.49, 0.66) | 0.48 (0.41, 0.55)
Faster R-CNN | MetaCOG | 0.36 (0.30, 0.40) |  0.30 (0.25, 0.34)
| Lesioned MetaCOG |  0.31 (0.25, 0.35) | 0.31 (0.25, 0.35)
DETR | MetaCOG | 1.26 (0.90, 1.58) |  0.71 (0.45, 0.92)
| Lesioned MetaCOG |  0.83 (0.52, 1.11) | 0.81 (0.51, 1.06)
Table 3: Average 3D distance between inferred and ground-truth object
locations.

On the whole, Lesioned MetaCOG’s meta-cognition represents a system with a
relatively high hallucination rate (higher than the rates inferred by
MetaCOG). That belief leads Lesioned MetaCOG to only infer that an object was
present when there were more detections of that object, and more spatial
information, leading to greater location precision for the objects it inferred
to be present. Consequently, Lesioned MetaCOG only infers the presence of the
objects that are easiest to locate.

For a more fair comparison, we can examine the average distance between pairs
of inferred and ground-truth objects only for ground-truth objects that both
MetaCOG and Lesioned MetaCOG’s inferred to be present. These average distances
are reported in the Avg. Distance Shared Pairs column of Table 3. For objects
that both MetaCOG and Lesioned MetaCOG inferred to be present, MetaCOG
consistently infers slightly better locations than does the Lesioned model.

For reference, the footprint of the room is $\approx 12$ x $8$, and the
largest object is 1 unit across.

#### Results on additional dataset

To address questions of how well our results will generalize to a new dataset,
we rendered a new dataset in ThreeDWorld, this time using five different
background rooms and including up to six objects per scene. This collection of
room models varies in room size, texture and color (e.g. carpeting vs wood vs
vinyl floors), windows, and even ceiling beams. With the inclusion of more
objects and sometimes smaller rooms, this means that the new dataset also
includes scenes that are more crowded and complex. As a conservative
evaluation we used Faster R-CNN, which has the highest baseline performance of
all three neural networks that we use, and for which MetaCOG showed the
smallest improvement in the one-room dataset. Table 5 reports the results with
Faster R-CNN and MetaCOG on this new dataset. As this dataset is more
challenging, Faster R-CNN performs worse on this dataset, so the quality of
the inputs to MetaCOG is lower. As the problem is harder, we increased the
computation during inference from 200 iterations of the three-step
rejuvenation process to 1000 iterations, but no other hyperparameters were
changed.

Model | Acc. Training | Acc. Test
---|---|---
MetaCOG |  0.69 (0.64, 0.74) |  0.60 (0.54, 0.67)
Lesioned MetaCOG | 0.49 (0.43, 0.56) | 0.46 (0.39, 0.53)
Faster R-CNN | 0.66 (0.61, 0.70) | 0.57 (0.51, 0.63)
Table 4: Results on dataset with multiple rooms

#### Faster R-CNN re-training details

We trained Faster R-CNN on the training set using MetaCOG’s inferences as
ground-truth labels. Since MetaCOG infers an object category and a 3D point,
rather than a bounding box, we constructed a 50x50 pixel bounding box centered
on the MetaCOG inferred’s 3D point projected onto the 2D image. Then, Faster
R-CNN with pre-trained weights was trained on the 1000 images (50 videos $*$
20 frames) in the training set. The training set was augmented by flipping the
images horizontally. Faster R-CNN was trained for 10 epochs with a SGR
optimizer (learning rate = 0.005, momentum = 0.9, weight decay = 0.0005) and a
learning rate scheduler (step size = 3, gamma = 0.1). The classification head
layer was not replaced because we wished to retain the information about the
COCO object categories Faster R-CNN had been pre-trained on, as the object
categories in our dataset were a subset of COCO categories.

Model | Acc. Training | Acc. Test
---|---|---
Pre-trained NN | 0.75 (0.67, 0.80) | 0.69 (0.63, 0.74)
MetaCOG | 0.79 (0.73, 0.84) | 0.76 (0.70, 0.82)
NN re-trained with MetaCOG | 0.84 (0.81, 0.87) | 0.81 (0.78, 0.84)
MetaCOG Round 2 |  0.87 (0.83, 0.90) |  0.85 (0.82, 0.87)
Table 5: Faster R-CNN re-trained on MetaCOG’s inferred labels.

Recall from 4.1 that all reported results show averages across four runs of
MetaCOG, with four different orderings of the videos. Each run of MetaCOG
produced slightly different inferences, so we trained Faster R-CNN using the
outputs of each of the four runs of MetaCOG, producing four sets of weights
for Faster R-CNN. In the third line of Table 5, we report the accuracy of re-
trained Faster R-CNN by averaging across the accuracy of Faster R-CNN with
each of the four sets of weights.

Finally, the outputs re-trained Faster R-CNN were input again into MetaCOG,
and MetaCOG inferred the objects present. The fourth line of Table 5 reports
the accuracy of MetaCOG, averaged over four runs of MetaCOG, each with input
from Faster R-CNN with different weights.

The results are shown in Fig. 3 and Table 5.

#### Lesioned MetaCOG

To whether learning the correct parameters $\theta$ for the meta-cognition
actually matter, or if the priors (over $\theta$ and over world states in the
form of object constraints) do all the work of improving accuracy, we tested a
lesioned version of MetaCOG fixing the values in $\theta$ to the mean of the
prior. The initial prior over the hallucination rate for each object category
is a Gamma distribution with with $\alpha=1$ and $\beta=1$. The mean of that
prior is then $1.0$, so in Lesioned MetaCOG, the hallucination rates are set
to $1.0$ The initial prior over the miss rate for each object category is a
Beta distribution with $\alpha=1$ and $\beta=1$. The mean of that prior is
then $0.5$, so in Lesioned MetaCOG, the miss rates are set to $0.5$. So this
lesion produces the expectation that an object of each category will be
hallucinated on average once per frame, and that, when an object is in view,
it has a 50% chance of being missed.

#### Explaining Lesioned MetaCOG’s performance when paired with DETR

As described in A.5, the Lesioned MetaCOG model’s meta-cognitive
representation of the behavior of the object detectors is fixed to the mean of
the priors, which is a hallucination rate of $1.0$ and a miss rate of $0.5$
for each object category. It just so happens that, because of the post-
processing procedure used for DETR (described in A.3), that representation are
not too far off the ground-truth. The ground-truth miss rates for DETR are
indeed around $0.5$ for most categories (as can be seen in Fig. 2A), and for
one category (umbrella), the hallucination rate is indeed very high
$\lambda_{umbrella}=0.79$. It’s worth noting that the full MetaCOG model badly
underestimates this category’s hallucination rate (estimating it as
$\hat{\lambda_{umbrella}}=0.25$). It seems that DETR’s poor baseline
performance (Table 1) made it more difficult for MetaCOG to learn an accurate
meta-cognitive representation. Since the Lesioned model’s meta-cognition is
not bad for DETR, and the full MetaCOG model struggled to learn a correct
meta-cognition for DETR, the two models performed comparably when inferring
the objects present. This shows that, in the case of DETR, learning the meta-
cognition from experience did not result in a performance boost above and
beyond MetaCOG’s priors over meta-cognition and about the world (i.e. object
constraints). In the case of DETR, the priors did all of the work.

#### Compute resources

We estimate the compute resources used in producing the final results reported
in Exp. 1. Running the MetaCOG (and Lesioned) model four times per each of the
three neural networks used 36 CPUs with about 5GB memory per CPU on an
internal cluster for approximately 100 hours.

###  A.6 Lightweight MetaCOG

In addition to the full MetaCOG model described in the main text, we also
implemented a simplified version for the simplified setting used in Exp. 2,
eliminating spatial information so as to isolate the contribution of meta-
cognition. This simplified setting is object detection without location,
wherein a black-box object detector generates labels (without associated
locations) for the objects present in a scene. So detections are simply object
labels. Furthermore, the goal of inferences in this simplified setting is not
to infer what objects are where in 3D space, but merely which objects are
present in the scene. Camera trajectories are no longer included, and all
objects are assumed to remain in view at all times.

Additionally, the meta-cognition in this Lightweight MetaCOG is simplified as
well: the prior over the meta-cognition, $G$, does not vary with time, but is
fixed as a Beta distribution with parameters $(\alpha=2,\beta=10)$.
Hallucination and misses are treated as Bernoulli random variables, with their
parameters $\theta$ sampled from the Beta prior, separately for each object
category.

The inference targets and procedure are largely unchanged, with the estimate
of joint posterior now sequentially approximated via:

$Pr\left(\vec{V},\vec{W}|\vec{D}\right)\approx\prod_{t=1}^{T}Pr\left(D_{t}|\hat{V_{t}}^{t},\hat{W_{t}}^{t}\right)Pr\left(\hat{W_{t}}^{t}\right)Pr\left({\hat{V_{t}}}^{t}|{\hat{V_{t-1}}}^{t}\right)$

where the transition kernel, $Pr(\hat{V}^{t}|\hat{V}^{t-1})$, defines the
identity function.

#### Details of inference procedure for Lightweight MetaCOG

The above equation is estimated via particles filtering, with 100 particles.

We implemented rejuvenation using a series of Metropolis-Hastings MCMC
perturbation moves over $\hat{\theta}$. The proposal function is defined as a
truncated normal distribution with bounds $(0,1)$:

$\hat{\theta_{i,j}}^{t^{\prime}}\sim\mathcal{N}(\mu=\hat{\theta_{i,j}}^{t},\sigma^{2}=0.01)$

where $\theta_{i,j}$ is the $j$th element of vector $i$ in the pair of vectors
of parameters $\theta$. A proposal is accepted or rejected according to the
Metropolis-Hastings algorithm. Each element in $\theta$ is rejuvenated
separately and in randomized order.

###  A.7 Experiment 2 Dataset

Here we discuss how we synthesized the dataset for evaluating Lightweight
MetaCOG in Exp. 2.

In this context, the object detector can be represented by its parameters,
$\theta$, which is a pair of vectors of length $|C|$ containing the
hallucination and miss rates for each object category. The parameters for each
object detector was generated by drawing ten independent samples from a beta
distribution, $\sim\mbox{B}(\alpha=2,\beta=10)$. This distribution allows us
to sample object detectors with variable error rates (mean value =
$\frac{1}{6}$) while maintaining a low probability of sampling object
detectors that produced hallucinations or misses more often than chance (0.005
chance of sampling values above 0.5; 0.06 chance that complete sampled object
detector has at least one hallucination or miss rate above 0.5).

For each object detector, we sampled 75 world states. A Poisson distribution
$N\sim Poisson(\lambda=1)$ truncated with bounds $[1,5]$ determined the number
of objects in a world state. The object categories were samples from a uniform
distribution. Each world state was a hypothetical collections of objects,
summarized as a vector of 1s and 0s indicating the presence or absence of each
category of objects. For each world state we used the object detector to
synthesize the detections from $5-15$ simulated frames (number sampled from a
uniform distribution), producing a total of $375-1125$ simulated frames per
object detector. Inferences about the hallucination and miss rates of each
object category are independent, and we thus considered situations with only
five categories.

###  A.8 Metrics for Experiment 2

To measure how well MetaCOG learned a meta-cognition, we calculated the mean
squared error (MSE) between the inferred parameters of the object detector
$\hat{\theta}$ and the true parameters $\theta$ generating the percepts given
by

$\mbox{MSE}=\frac{1}{2|C|}\sum_{c\in
C}\bigg{(}(H_{c}-\hat{H}_{c})^{2}+(M_{c}-\hat{M_{c}})^{2})\bigg{)}$

where $C$ is the the set of object classes, $H_{c}$ is the hallucination rate
for category $c$, and $M_{c}$ is the miss rate for category $c$.

Accuracy was measured using the Jaccard similarity coefficient of the set of
object classes in the ground-truth world state $W_{t}$ and the set of object
classes in the inferred world state $\hat{W}_{t}$, see A.4.

To analyze MetaCOG’s accuracy as a function of faultiness in the detected
labels, we computed the average faultiness in a collection of detections
$D_{t}$ as

$\zeta_{t}=\frac{1}{|D_{t}||C|}\sum_{c\in C}\sum_{x\in
D_{t}}|\mathds{1}_{W_{t}}(c)-\mathds{1}_{x}(c)|$ (1)

where $C$ is the set of object classes, $D_{t}$ is the collection of detected
labels generated from world state $W_{t}$, $\mathds{1}_{W_{t}}(c)$ is an
indicator for whether an object of class $c$ is in $W_{t}$, and
$\mathds{1}_{x}(c)$ is an indicator for whether an object of class $c$ is in
the detection $x$.

Because our sampling-based dataset generation process (A.7) does not guarantee
enough data points for every possible faultiness value, analysis in Fig. 4C
and D were computed using a rolling window such that each point shows average
accuracy on the $[\zeta-.05,\zeta+.05]$ range.

###  A.9 Comparison models for Experiment 2

To test whether learning a meta-cognition improves inferences about the world
state, we compare MetaCOG with and without the meta-cognitive learning. We
call MetaCOG without learned meta-cognition Lesioned MetaCOG. Like the other
MetaCOG models, Lesioned MetaCOG has a meta-cognitive representation of $V$
and uses an assumption of object permanence to infer the world states causing
the detected labels. Lesioned MetaCOG, however, does not learn or adjust the
parameters $\theta$ in its meta-represetation based on the observed labels.
Formally, Lesioned MetaCOG assumes that the hallucination and miss rates for
every category are the mean of the beta prior over hallucination and miss
rates, call it $\hat{\theta}_{0,\mu}$. Lesioned MetaCOG then uses the same
particle filtering process described in A.6, except that it conditions on
$\hat{\theta}_{0,\mu}$ instead of $\hat{\theta}_{T,\mu}$.

As described in A.6, the prior over both the hallucination rate and the miss
rate is a Beta distribution parameters $(\alpha=2,\beta=10)$. The mean of this
prior is then $\frac{1}{6}$. So in the Lesioned MetaCOG for Exp. 2, the
hallucination rate and the miss rate are both set to $\frac{1}{6}$.

We also compare two variations of MetaCOG with learning. MetaCOG Learning
performs a joint inference over $\theta$ and $\vec{W}$ based on collections of
detections $\vec{D}$ generated from the sequence of world states $\vec{W}$.
This model allows us to evaluate how MetaCOG’s inferences improve as a
function of the observations it has received. We name this model observation-
by-observation inference MetaCOG Learning.

After having received all $T$ observations, MetaCOG could retrospectively re-
infer the world states causing the $T$ observations. This model, simply called
MetaCOG, re-infers the world states that caused its observations conditioned
on its estimate $\hat{\theta}^{T}_{T,\mu}$, as described in 3.2.

MetaCOG Learning lets us interpret how MetaCOG learns a meta-cognition, and
MetaCOG lets us test how MetaCOG performs after having learned that meta-
cognition. Lesioned MetaCOG serve as a baseline model for comparison.