Figure 8: t-SNE visualization of episodic instances based on their embeddings
$\\{\mathbf{a}_{t}\\}_{t=1}^{N_{T}}$. One can see that similar scenes are
often in proximity. For example, the circled areas show images of indoor
scenes (bottom) and of buses on streets (right).

###  9.2 Event Memory and Episodic Memory

In this article, we use the terms event memory and episodic memory almost
synonymously. In the narrower sense, we consider event memory to be related to
a single instance in time and episodic memory to a sequence of events that is,
a story. This is in agreement with Mannila et al. (1997), who define an
episode as a collection of events that occur relatively close to each other in
a given partial order. We propose that the distinction between event memory
and episodic memory is blurred. First, we often remember rather static images
of past episodes. Second, even an analysis of a static scene is a sequential
process, where each bounding box is recovered in sequence. Third, during a
single saccade, the scene might have already changed: Consider, for example,
an agent riding a bicycle where the scenery is constantly changing. So the
recovery of a single scene might already describe a dynamic event.

###  9.3 Recall of Episodic Memory Engrams

An activated past episodic instance $t$ restores its embedding
$\mathbf{a}_{t}$, that is, its engram, in the representation layer. Figure 8
shows that the embedding vectors of episodic memories form meaningful maps and
also are organized as a conceptual space.

| $s^{*}$ | Unary labels | Binary labels
---|---|---|---
Model | @10 | @1 | B-Class | P-Class | G-Class | Y/O | Color | Act. | @10 | @1
EM | 99.84 | 37.13 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 90.40
P-noI | 0.0 | 0.0 | 41.19 | 20.83 | 51.76 | 49.03 | 12.97 | 71.05 | 39.34 | 7.79

Table 12: Top row: episodic memory experience using VRD-E data. For randomly
selected past episodic instances $t^{*}$ as input, we determine the highest-
ranked entity indices (first two columns). For the other columns, we set the
correct entity indices ($s^{*}$ and $o^{*}$) and predict unary labels and
binary labels. The performance is, in many cases, 100%. Thus, an agent might
recall seeing a black dog, but it might be less confident that it was Sparky.
Considering the large number of entities in the data set, the performance on
entity prediction (first two columns) is still impressive as well. Bottom row:
In P-noI we removed all entity indices, using only class and unary labels. The
bad performance demonstrates the relevance of representing entities for memory
recall.  Figure 9: Episodic memory experience using VRD-E data. The first row
shows the visual input to perception and then extracted bounding boxes for
entities in the scene. In the second row, we show an episodic recall of this
scene ($t^{*}=t^{\prime}$), which is now a memory. We show the bounding boxes
of the scene entities, which were recalled. We see that, in sampling, episodic
recall recovers entities with correct bounding box content (e.g., fire
hydrant, sky, car). Columns 6 to 8 show incorrect recalls. Note that these
incorrect recalls would still be entirely plausible in the scene’s context.

###  9.4 Episodic Memory Experience: Semantic Decoding and Embodiment

The embedding of an episodic index is all there is, but the treasures of an
episodic memory are only deciphered by the semantic decoding of that
embedding, that is, the symbolic reconstruction of triples involving that
episode (see Table 2). The recall of the episodic embedding, but especially
the semantic decoding into triples, has the character of a simulation:
Episodic memory is a reactivation of a possibly multimodal memory experience
(Evans, 2012) and has been described as a reliving of past experience
(Shapiro, 2010). Figure 9 illustrates an episodic memory experience. Table 12
provides numerical results.

###  9.5 Episodic and Semantic Memory in Perception

We propose that in perception, an agent first uses both the episodic and
semantic attention approximation, which can be executed fast and in parallel.
Only in a second step are specific entities sampled. This permits the
integration of specific multimodal background (see Section 7.8), and permits
associations with past episodic instances and known entities. Thus if
perception poses the hypothesis that an entity in the image is identical to
Sparky, then the semantic memory experience would supplement specific
background information on Sparky, which cannot be derived from the visual
input. Semantic memory reconstructs what is known about the concept (i.e., the
prior information). Similarly, episodic memory would recall relevant past
episodes.

###  9.6 Recent Episodic Memories for Context

Recent episodic memory can provide the agent with information on recent
perceptional experiences. A recall is triggered by the recency of the episodic
instance and relevance, and it contributes to an agent’s sense of the world
state. A recent episodic memory experience is an episodic memory that is
almost treated as a current observation. An agent needs to know about the
state of the world, even for parts that are not currently being perceived. An
example is the “lurking-bear” situation (see Figure 10): “There was a bear
strolling around outside the hideout, $\ldots$. Remember: it might still be
there, although it might not be visible from the hideout.” Figure 11 shows
another example of a recent episodic memory recall. This emphasis on episodic
closeness can be implemented as “time encoding” (Ma et al., 2018b), in a
similar way as “position encoding” is used in the attention literature
(Vaswani et al., 2017). Patients who are unable to form new episodic memory
show great deficits in personal orientation and context understanding. These
deficits are often associated with severe bilateral damage to MTL (Gluck et
al., 2013).

Figure 10: Recent episodic memory experience: An illustration of the effect of
a recent episodic memory experience using VRD-E data. The left image shows a
harmless garden scene, but due to a recall of a recent episodic memory
$t^{*}=684$, the agent is aware of the lurking bear close by (right scene).
Labels for visual entity recovered in the episodic recall (right scene) are
Bear, Mammal, LivingBeing, Old, Black, OtherActivity, Dangerous. Note that
episodic recall is not triggered by closeness in a scene but by recency and
relevance.

s* | Unary labels | o* | Binary label
---|---|---|---
346 | Person, 1.00, Mammal, 1.00, LivingBeing, 1.00 | 345 | on
| Young, 1.00, Other, 1.00, Other 1.00 |  |
348 | Shirt, 1.00, Clothing, 1.00, NonLivingBeing, 1.00 | 347 | on
| Young, 1.00, Orange, 1.00, Other, 1.00 |  |
347 | Person, 1.00, Mammal, 1.00, LivingBeing, 1.00 | 348 | wear
| Young, 1.00, Other, 1.00, Playing, 1.00 |  |
345 | Grass, 1.00, Plant, 1.00 LivingBeing, 1.00 | 346 | under
| Old, 1.00, Green, 1.00, Other, 1.00 |  |

Figure 11: Recent episodic memory experience. (Top) The left image shows
visual input to perception from VRD-EX data. Then a recent $t^{*}$ is sampled.
The right image shows the image belonging to $t^{*}$. (Bottom) The table shows
results from this episodic memory experience without the actual image for
$t^{*}$ being available, only based on memory recall. The first column shows
samples for $s^{*}$. The second column shows unary labels. The third column
shows sampled objects $o^{*}$. The fourth column shows the most likely binary
label.

###  9.7 Remote Episodic Memory for Decision Support

Remote episodic memory can provide the agent with information on remote
perceptional experiences similar to the current perceptual experience, and
this can contribute to decision-making. An episodic recall is triggered by
$t=t^{*}$ (a sample in Equation 17) which reflects the similarity between the
current scene representation with the episodic embedding. It makes sense that
the current event should trigger the same action as in the retrieved episodic
memory —if it led to a good outcome— or an alternative action if not. For
example, if the agent finds the current situation very similar to a previous
one, where it had walked toward a bear and almost got attacked, it would very
likely not do this again! Remote episodic memory provides information about
possible future scenarios and aids the agent in decision-making.

Thus memory guides behavior. Duncan and Shohamy (2016) describe this process
as integration across relational events by imaging possible rewards in the
future. The value associated with a memory (e.g., reward, threat) might be an
integral aspect of episodic memory. The article also states that there is now
extensive empirical data supporting the prevalent use of episodic memory
across various decision-making tasks in humans.

Figure 12 shows that a perceptual scene indeed can activate memories of
similar scenes in remote episodic memory.

Figure 12: (Top) Remote episodic memory experience using VRD-E data. The two
images in the first column show the visual input to perception. Then we sample
$t^{*}$, representing past episodic memories. The images of the scenes
associated with the $t^{*}$ show that, indeed, recalled past episodic memories
are related. (Bottom) The bounding boxes in the first column represent visual
entities $s^{\prime}$ in perception. The right bounding boxes display
retrieved entities with high activation.

###  9.8 Forecasting and Future Episodic Memories

To forecast the state at the next instance $t^{\prime}$, the agent might
consider recent episodic memories to be current, that is, the instance index
for those episodic memories is virtually changed to $t^{\prime}$. The post-
observation model then combines these memories with current observations and
with the semantic memory to estimate the world state.

A future episodic memory is a forecasted event in the future, which at some
point in time is predicted to become a regular episodic memory. This helps
plan future actions. Events might be predicted using, for example, temporal
knowledge graph models (Han et al., 2020a), assuming temporal smoothness and
predictability of the embedding of time instances. Here we consider forecasts
made entirely on a symbolic level, say, by verbal communication or reading.
For example, f the agent has learned by verbal, symbolic communication that
there is a football match in the stadium this evening and the weather will be
bad, embedded reasoning, based on this assumed future information, predicts
that there will be a traffic jam in the city and that driving will be
difficult. Technically, the BTN for future episodic memories is identical to
regular episodic memories, only that the episodic index embedding is
calculated based on assumed events in the future, either without or with
imagined and simulated future sensory input.

###  9.9 Memory Supports the Agent in the Present and the Future

It is of great interest for an agent to estimate the state of the world at the
current instance $t^{\prime}$, and to predict how it will evolve. We propose
that the mind estimates the world’s state using perception and memory: The
memory systems provide information that makes the agent act right but is not
communicated by current perceptual experience.

For example, to illustrate the relevance of memory in daily life, consider a
typical day at the office of agent Mary (see Figure 13). When Mary arrives at
the office in the morning, she expects that everything is as usual, as
modelled by semantic memory. The pre-observation model (semantic memory) will
produce statements describing the state of normality. This sets the stage.

Perception will produce triples describing the actual situation, for example,
the status of the coffee machine, which is broken. Since the coffee machine
has mostly been working, semantic memory by itself would have predicted
(incorrectly) that the coffee machine most likely is working. She immediately
informs Jack in an office nearby: “Jack, can you believe this?! The coffee
machine is broken, again!” (from triples to language). Remote Episodic memory
reminds her that this is not the first time that the coffee machine was
broken. Recent episodic memory will remind Mary all day that the coffee
machine is broken, even when she is not in the same room as the coffee machine
and thus does not have immediate perceptual information on its status. As
discussed, recent episodic memory is an episodic memory that is almost treated
as a current observation. The state change of the coffee machine is only
slowly integrated into semantic memory. Being a long-term average, semantic
memory is a sluggish state estimator and relies on recent episodic memory to
emphasize recent state changes.

Jack might ask Mary if the coffee machine had been working last Tuesday. If
she does not have an episodic recall from last Tuesday about the status of the
coffee machine, she might consult semantic memory, reminding her that the
coffee machine mostly has been working. But if she has an episodic memory that
last Monday, the coffee machine was broken, she might infer that it most
likely was also broken on Tuesday. Here, she consults a memory that is recent
to the past time instance of interest.

Another recent episodic memory might remind Mary that she had met Jane on her
way to work and that she had said that she would drop by at the office
(simulation of a recent episodic memory). Jane is a good friend (semantic
memory).

Mary recognizes that Sparky is in the office (perception), and semantic memory
adds triples describing background on Sparky, for example, that Sparky is
Jack’s dog. Mary might also recall that a while ago, Sparky was at the office
and behaved well (remote episodic memory), although dogs in general can be
quite a nuisance (as concluded by a generalized statement and by employing
embedded symbolic reasoning).

When Mary relaxes, she suddenly recalls that people are repairing the heater
today (episodic memory, not triggered by perception) and that she should call
her roommate to check if they arrived in time.

Then Mary remembers that the football match will be in the stadium this
evening and that the weather has been predicted to be bad (future episodic
event): there will likely be a traffic jam (generalization of future episodic
events), and, due to expected bad weather conditions, she should better drive
carefully (an association stored in semantic memory). Future-state prediction
uses both episodic memory and generalized future episodic memory.

The story illustrates that background is provided by semantic memory. Semantic
memory is mostly relevant as it concerns triple statements, whose states are
static or change rather slowly. Recent episodic memory provides information
that is not yet or will not be absorbed into semantic memory but contributes
to the current state estimate. Episodic memory is mostly relevant in as much
as it concerns state changes (broken coffee machine), singular events (meeting
Jane on her way to work) or information, not yet known. As our story
indicates, narratives often tell causal stories and might also reflect the
events and facts the brain pays attention to for decision-making. Future
episodic memory supports planning. Remote episodic memory guides behavior:
since the last time Sparky behaved well, he might behave well again this time.

The whole story is on a symbolic level. But each index that fires is embedded
and leads to embodiment. We would propose that the storyline involves System-1
type associative reasoning, supported possibly by some background knowledge of
how the world works. As our story suggests: to deal with daily life, agents
might need little effortful System-2 reasoning.

Figure 13: The horizontal axis stands for time and the vertical for some
abstract triple-dimension. We show past, present, and future. The current
instance is $t^{\prime}$ on the right. The light blue background box stands
for the predictions of the pre-observation model of semantic memory. The
observation model of the episodic memory is represented by the horizontal gray
bars; they indicate where, in the past, information was acquired. At episodic
instance $t^{\prime}$, the agent learns about some statements by perception
(orange), and these become part of the episodic memory (recent episodic
memory). There might be an association with another portion of the episodic
memory, that is, the remote episodic memory (red dotted arrow). If the agent
contemplates information unrelated to current perception, it can query on
remote episodic memory (blue dotted lines). The right vertical bar indicates a
future episodic memory.

##  10 To Perceive is to Learn

There is nothing in the mind that was not first in the senses. —John Locke

###  10.1 Overview

In normal operations, a new perceptual event that becomes an episodic memory
requires the establishment of a new episodic index and its associated
embedding vector. This roughly corresponds to the MTL-based fast nonparametric
learning system in the complementary learning systems (CLS) theory (McClelland
et al., 1995; Kumaran et al., 2016). A new perceptual event might contain
novel entities not yet known to the agent. For important ones, the agent needs
to establish new indices and their embedding vectors as well. More rarely, new
indices for attributes, classes, and predicates need to be established. Being
quite stable, those might be represented in neocortex.

The second component of the CLS-theory is the parametric learning system,
where the neocortex is trained by replay in a slow process from data from the
nonparametric learning system. Slow training serves as the basis for the
gradual acquisition of structured knowledge about the environment and its
transfer to neocortex (Kumaran et al., 2016). An agent might get new
information on existing entities, and this information might need to be
integrated into their embeddings. In some applications involving online
adaptation, interference of new knowledge with old ones can be a problem and
might lead to catastrophic forgetting (Kumaran et al., 2016). Catastrophic
forgetting does not show up in our preliminary experiments. The large
dimensionality and modularity of the embedding vector lead to robustness and
stability; for example, learning first about Jack’s size and hair color might
not interfere with the learning of his social network later. And when he later
dyes his hair, the changes in his embedding vector are local, that is, it
affects only a few dimensions. Learning happens as an online process, so more
recent episodes will have a greater effect than more distant ones.

###  10.2 Neuroscience Perspective on Establishing Episodic Memory Engrams

Episodic memories are first formed in the hippocampus, which is part of medial
temporal lobe (MTL). The idea that episodic memory is index based is by now
one of several accepted theories (Tonegawa et al., 2018). It goes back to the
hippocampal memory indexing theory (Teyler and DiScenna, 1986; Teyler and
Rudy, 2007), which was long controversial. The event indices have a relational
memory function in that they bind together different pieces of experience.
Recent research found evidence for the existence of time cells in the
hippocampus (CA1) (Eichenbaum et al., 2012; Eichenbaum, 2014; Kitamura et al.,
2015b, a).

There is some evidence that neurogenesis might be involved in forming new
episodic memories. Neurogenesis by special stem cells has been discovered in
the dentate gyrus (part of the hippocampal formation) and is active throughout
adult life; these new neurons may be preferentially recruited in the formation
of memories. In fact, it has been observed that the adult macaque monkey forms
a few thousand new neurons daily (Gluck et al., 2013; Gould et al., 1999),
possibly to encode new information (Becker, 2005).

The establishment of new indices, together with their embedding vectors, are
some of the most demanding learning tasks in the brain. Functionally, our
model assumes that a new episodic memory engram is quickly stored by
establishing a new index and its connections to the representation layer,
copying the episodic memory trace. Although there exist several theories,
little is known about how exactly new time indices are formed in the brain
anatomically and how they quickly set up the bidirectional connection patterns
with the representation layer, forming a hippocampal–cortical network
(Frankland and Bontempi, 2005). Here, the brain might employ already existing
networks, which might also explain how a single index can influence the
distributed representation layer with potentially numerous neurons. A
structural intra-hub connectivity might facilitate this process.

###  10.3 Experiments with Self-supervised Learning

Self-supervised learning (SSL) concerns learning without teacher-provided
explicit training labels. It is biologically highly relevant. During most of
life, an individual has to learn without explicit supervision (lifelong
learning). In our approach, self-supervised learning works exactly like
supervised training; the difference is that the agent’s predicted winner-take-
all unary and binary labels become the training labels. This form of self-
supervised learning is a type of bootstrap learning; see the bootstrap Widrow-
Hoff rule (Hinton and Nowlan, 1990) and learning with pseudo-labels (Lee et
al., 2013). The generated data is then used to train our model using cross-
entropy cost-function terms derived from perception, episodic, and semantic
memory (cf. Appendix E).

In our SSL experiments (see Table 13), we establish new indices and their
embeddings for perceptional episodes (episodic indices) and for new entities
in those episodes (concept indices); this would be part of the fast
nonparametric learning system. The self-supervised training for a novel
episodic instance and a novel entity666The agent might decide that an entity
is novel if the activation of all elements in $\mathrm{sig}(\mathbf{n}_{S})$
are below a threshold for all known entities. is modular and fast, both
technically and most likely also biologically. Thus, we do not assume domain
closure and consider that new perceptual experiences might require
representations for new entities. The embedding vector of a new episodic
instance $\mathbf{a}_{t^{\prime}}$ is adapted to model the decoded triple
statements. Decoded triple statements, which are more certain, will be decoded
more often, will be more influential in memory formation, and will then be
retrieved more likely in an episodic memory recall. Row 3 in the table shows
that the semantic recall on entities is reasonable, but not as good as on
entities trained with annotator-labeled data (see row 1). This is
understandable since predicted labels are noisier than human-annotator-
provided training labels.

Now consider the slow parametric learning system. Row 2 shows that the
embeddings of already established entities are not negatively affected by SSL.
Rows 4 and 5 show perception performance on unseen entities. We see better
performance after SSL has been applied. This shows that embeddings for classes
and attributes are improved by SSL.

###  10.4 Replay for Episodic Memory Consolidation

As self-supervised learning adds information, the required memory capacity
grows. In our approach, for each salient episode, a new episodic index with
its embedding vector is established. Similarly, for each new entity, a new
concept index with its embedding vector is added. The current thinking in
neuroscience is that this capacity problem can be solved by a consolidation
from MTL, with a limited capacity, to the neocortex, with an essentially
unlimited capacity. This is called systems consolidation of memory (SCM).

We now consider the leading two theories about the consolidation of episodic
memory. The standard theory assumes that at some point, episodic memory
becomes independent of the hippocampus and MTL over a period of weeks to years
(Squire and Alvarez, 1995; Frankland and Bontempi, 2005). In contrast, the
multiple trace theory assumes that both the hippocampus and MTL remain
involved (Nadel and Moscovitch, 1997; Jonides et al., 2008; Greenberg and
Verfaellie, 2010); MTL remains the manager of complex spatial and relational
memories (Whittington et al., 2020). In general, it is assumed that
consolidation involves both the medial prefrontal cortex (mPFC) and the MTL.
The temporal lobe might be where the transferred indices are established
(Frankland and Bontempi, 2005; Tonegawa et al., 2018). After consolidation,
episodic memories might be organized in temporal order or according to a
similarity in representation (see Figure 8). It is assumed that consolation
might be a process executed entirely or partially during sleep (Stickgold,
2005).

We propose consolidation by replay, which, in our model, can be executed as
follows. An episodic instance $t$ in MTL is activated, which activates the
representation layer with vector $\mathbf{a}_{t}$; this activation is then
learned in the connection weights of a newly formed index in the neocortex,
for example, by a form of Hebbian learning. Thus, these index duplicates in
the neocortex would inherit the connection weights. If the index in the
neocortex becomes more distributed, this will lead to greater robustness of
memories after consolidation. For a while, both representations exist in
parallel, and this facilitates the learning and consolidation of new memories;
gradually, the index representation in the neocortex might become dominant.
Consolidation by replay has the advantage that there is no need for direct
interactions of indices in both storage sites, only indirect interactions by a
shared activation of the representation layer. Replay might also be one way of
how the brain implements large-scale structural changes in the brain, in
general, for example, as a consequence of brain damage or as a consequence of
a changing world with new statistics. Following the principle “use it or lose
it,” replay might also be essential such that relevant information in
consolidated memory is not forgotten. From our model’s perspective, the
location of the index is irrelevant. It could be in MTL, in neocortex, or both
at the same time. In either case, decoding relies on the same machinery (see
Figure 2). Cognitive maps might be more pronounced after consolidation in the
neocortex (Binder and Desai, 2011). From a cognitive perspective, it makes
sense that the brain mainly consolidates unusual episodic instances involving
semantic state changes or singular events. It is tempting to assume that
recent episodic memory is represented in MTL, whereas remote episodic memory
is stored in neocortex.

###  10.5 Replay for Semantic Memory Consolidation

Training by episodic memory replay might also be important for the gradual
transition from episodic to semantic memory, in which episodic memory reduces
its sensitivity and association to particular episodes so that the information
can be generalized as semantic memory. It might enable the semantic memory to
adapt more quickly to state changes, for example, to a friend’s status change
from being single to being married. Although this forgetting of the “single”
state might require time, the label “married” might quickly show higher
activity than “single”; the mind might consult episodic memory to resolve the
conflict, discovering that the label “married” was assigned to recent
instances. Semantic memory has a built-in forgetting mechanism since Equation
4 shows that if an observation of a triple is not renewed, dividing by
$N_{\mathrm{total}}$ implies that it will get less significant over time. Some
theories speculate that episodic memory may be the gateway to semantic memory
(Baddeley and Hitch, 1974; Squire, 1987; Baddeley, 1988; Steyvers et al.,
2004; Socher et al., 2009; McClelland et al., 1995; Yee et al., 2014; Kumar et
al., 2015).

Training | Test | Unary labels (accuracy)
---|---|---
Regime | Set | B-Class | P-Class | G-Class | Y/O | Color | Activity | Average
SL | SL-D | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 100.0
+SSL | SL-D | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 | 99.99 | 100.0
+SSL | SSL-D | 85.51 | 88.36 | 93.16 | 48.32 | 59.42 | 79.78 | 75.76
SL | G | 77.43 | 85.79 | 92.63 | 49.08 | 62.35 | 81.39 | 74.78
+SSL | G | 77.54 | 86.15 | 92.83 | 48.58 | 62.62 | 82.78 | 75.08

Table 13:  Self-supervised learning on the VRD-E data. SL stands for
supervised learning, SSL for self-supervised learning. We first trained the
model (SL) in a supervised way with human-provided labels (SL-D) with only 50%
of the training images (2000 images). Columns 3 to 9 show perfect semantic
memory performance for unary labels (first row). Then we continued to train
the model with only the other 50% of the training images (SSL-D) in the SSL-
mode (+SSL). The second row shows that self-supervised learning does not lead
to a deterioration of prediction performance on the entities in SL-D. The
third row shows that the performance on SSL-D is quite good, but of course,
not as good as on SL-D since predicted labels on this data are noisier than
the human-provided training labels. Rows 4 and 5 show performance in
perception on new entities (generalization, G). The performance of +SSL is
better than SL, which shows that self-supervised learning also improves the
detection of classes and attributes, helping to fine-tune their embeddings.

###  10.6 Forgetting and Modularity

Our hypothesis is that for a new instance or a new entity, an index is
established in MTL. Let’s focus the discussion on a past episodic instance. If
an episodic index is never consolidated into neocortex, the instance might
eventually be forgotten since the resources in MTL are limited, and the brain
might need to reuse them. On the other hand, if an index is consolidated into
neocortex, it might be there for the lifetime of the agent. In neuroscience,
it is an open question if consolidated memories really ever are forgotten, or
if only the access to those memories is lost. The proposed index
representation is essential since a particular functional synapse between an
index and a dimension in the representation layer only serves this one purpose
of connecting the index with the corresponding dimension in the representation
layer. It is not reused for any other purpose. Assuming that an index and its
embedding vector are stable, there might still be an “aging” of a memory. One
reason is that the meaning of a dimension in the representation layer might
change over the lifetime of an agent. In other words, the grounding or
embedding might later in life be different from the time the memory trace was
established. A second issue is that the representations of other indices
change over the lifetime of an agent. Thus, if Jack was part of a 10-year-old
episode, the embedding vector for the index “Jack” might have changed since
then. Reactivation of a past memory in combination with self-supervised
learning might stabilize a past episodic memory, associated with the danger of
changing that memory. It is well known that the reactivation of a past memory
does not just strengthen that memory but also might change it. Note that this
stabilization mechanism also only works in the context of index
representations.

##  11 Summary, Conclusions and Future Work

We have shown how perception, memory, reasoning, and foundations for language
and consciousness can all be realized by different functional and operational
modes of the oscillating interactions between an index layer and a
representation layer in the BTN. Whereas the representation layer, as a global
workspace, is prominent in the current discussion on consciousness, the
introduced index layer is an original contribution. We have proposed that an
index representation increases modularity and counteracts forgetting. We have
emphasized the role of episodic and semantic memory in perception. We have
proposed an associative memory where recency is key to the recall of recent
episodic memory and similarity of episodic representations with the present
scene representation for remote episodic memory. We argue that episodic memory
models the observed data, and semantic memory provides background knowledge.
Our approach explains the great similarity between episodic and semantic
memory: semantic memory is the expected episodic memory of a future instance.

We have proposed that perception and memories first produce subsymbolic
representations, which are subsequently decoded semantically to produce
symbolic triple sentences. Our paper contributed to the discussion (a recent
example being (Browning and LeCun, 2022)) of how language, thought, and
subsymbolic processing interact.

As part of future work, we will address more systematically subsumption
hierarchies and how parts bind to form the whole, which is another form of
compositionality addressed, for example, in capsule networks (Sabour et al.,
2017). The agent itself is also an important concept: it is part of each
event. It might be quite relevant if the angry bear looks at a deer or at the
agent itself. Other issues of the “I” are mood, state of mind, current
mission, social contexts, and spatial location.

In this article, we did not focus on spatial representations. It is well known
that MTL is not only instrumental for forming novel episodic memories but also
contains spatial representations, for example, in the form of grid cells and
place cells. It is generally assumed that MTL is central for reconstructing
spatial memories, permitting complex spatial reasoning. We plan to investigate
if instance indices, their embeddings, and their labels might also be the
basis for spatial navigation and reasoning.

The brain does many different things simultaneously and addresses a particular
problem with several strategies. Our work emphasizes the role of perception
and memory, and it might provide some insights into part of the amazing
faculties of human intelligence.

##  A Cost Function Terms

$\mathbb{P}(s,p,o,t)$ has four categorical variables. The number of states is
$N_{C}^{2}N_{P}N_{T}$. So there is one state for each Boolean variable.

#### Observation Model for Episodic Memory

If $i_{s,p,,o,t}=1$, then the contribution to the cross-entropy cost function
for binary labels is

$-(\log\mathbb{P}(p|s,o,t)+\log\mathbb{P}(o|s,t)+\log\mathbb{P}(s|t))$

and for unary labels $-(\log\mathbb{P}(c|s,t)+\log\mathbb{P}(s|t))$.

#### Pre-observation Model for Semantic Memory

If $i_{s,p,,o,t}=1$, then the contribution to the cross-entropy cost function
for binary labels is

$-(\log\mathbb{P}(p|s,o,\bar{t})+\log\mathbb{P}(o|s,\bar{t}))$

and for unary labels $-\log\mathbb{P}(c|s,\bar{t})$ .

#### Perception

In perception (see Section 4.6)

$\mathbb{P}(s^{\prime}=s,p^{\prime}=p,o^{\prime}=o|t^{\prime}=t,\textit{BB}_{\textit{sub}},\textit{BB}_{\textit{obj}},\textit{BB}_{\textit{pred}},\textit{scene}_{t^{\prime}}).$

If $i_{s,p,,o,t}=1$, then the contribution to the cross-entropy cost function
is

$-\log\mathbb{P}(s^{\prime}=s,p^{\prime}=p,o^{\prime}=o|t^{\prime}=t,\textit{BB}_{\textit{sub}},\textit{BB}_{\textit{obj}},\textit{BB}_{\textit{pred}},\textit{scene}_{t^{\prime}}).$

##  B Boolean versus Categorical Models

As discussed, the interface between the multinomial and the two-state
Bernoulli model is via sampling. If the multinomial model generates an
$(s,p,o)$ or $(s,c)$ sample, then the random variable $Y_{s,p,o,t}$ is assumed
true. Here, we show a close mathematical relationship between the
probabilistic scores. The key correspondence is that for entities that were
observed in the episodic experience at $t$,

$\mathbb{P}(s,p,o|t)\approx\mathbb{E}(Y_{s,p,o,t})/{N_{\textit{t}}}.$

From this, one can derive

$\mathbb{P}(p|s,o,t)\approx\mathbb{E}(Y_{s,p,o,t})/{\sum_{p^{\prime}}\mathbb{E}(Y_{s,p^{\prime},o,t})}$
(20)
$\mathbb{P}(c|s,t)\approx{\mathbb{E}(Y_{s,\textit{hA},c,t})}/{\sum_{c^{\prime}}\mathbb{E}(Y_{s,\textit{hA},c^{\prime},t})}$
(21)

For semantic memory, we have

$\mathbb{P}(s,p,o|\bar{t})\approx
N_{s,p,o,\mathrm{known}}\mathbb{E}(Y_{s,p,o,\bar{t}})/{N_{\textit{total}}}.$

Thus, the samples from the pre-observation model reflect the expected-state
model weighted by the number of measurements on the triple statement.
Sometimes it is possible to apply knowledge on mutually exclusiveness. For
example, if we define the complementary of $c$ as $\bar{c}$, then

$\frac{\mathbb{P}(s,\textit{hA},c,\bar{t})}{\mathbb{P}(s,p,\textit{hA},c,\bar{t})+\mathbb{P}(s,\textit{hA},\bar{c},\bar{t})}=\mathbb{E}(Y_{s,\textit{hA},c,\bar{t}}).$

As an example, assume Sparky is observed to be barking in 10 episodic
instances and not-barking in 1000 episodic instances. Then,
$\mathbb{E}(Y_{\textit{Sparky},\textit{hA},\textit{Barking},\bar{t}})\approx
1\%$, so Sparky is rarely barking. But since he is sometimes barking, this
would be reflected in a firing of “Barking”, following
$\mathbb{P}(\textit{Sparky},\textit{hA},\textit{Barking}|\bar{t})$. The agent
would have problems to understand if Sparky is often barking or not. In
contrast, if the agent observes both the firing of “Barking” and “NotBarking”,
the fraction of firing rates (see last equation) would tell the agent, that
Sparky is mostly quiet.

Furthermore,

$\mathbb{P}(p|s,o,\bar{t})\approx\mathbb{E}(Y_{s,p,o,\bar{t}})/\sum_{p^{\prime}}\mathbb{E}(Y_{s,p^{\prime},o,\bar{t}})$
(22)
$\mathbb{P}(c|s,\bar{t})\approx\mathbb{E}(Y_{s,\textit{hA},c,\bar{t}})/\sum_{c^{\prime}}\mathbb{E}(Y_{s,\textit{hA},c^{\prime},\bar{t}}).$
(23)

In the last two equations, we have exploited that due to the LCWA,
$N_{s,p,o,\mathrm{known}}=N_{s,p^{\prime},o,\mathrm{known}}$ and
$N_{s,c,\mathrm{known}}=N_{s,c^{\prime},\mathrm{known}}$.

##  C Visualisation

Figure 14:  Visualization for categorical variables. Here, we show the
situation for unary statements. For binary statements, replace $c$ with
$(p,o)$.

Figure 14 illustrates the situation for categorical variables. The update for
the pre-observation and post-observation models are almost identical. The
difference is that the observed data have a higher weight in the post-
observation model. Figure 15 shows the situation for Boolean variables.

Figure 15: Same, but for Boolean variables.

##  D Social Network

We consider all 4987 persons in the data set and link a person $s$ to persons
$s^{\prime}$, if the score $\mathbf{a}_{s}^{\top}\mathbf{a}_{s^{\prime}}$ is
in the top 5 of all scores related to $s$. Thus links exist between persons
with similar embeddings, simulating homophily. We then determine the link
direction. Considering two entities $s$ and $s^{\prime}$,
$\exp\beta\|\mathbf{a}_{s}\|/(\|\mathbf{a}_{s}+\mathbf{a}_{s^{\prime}}\|)$ is
proportional to the probability that we determine that (s, knows, s’),
otherwise, (s’, knows, s). At a social network episodic time step $t$, all
links to one person $s$ are added. This defines 4987 episodes for the tKG. The
pKG then aggregates the tKG. Overall, we have 24953 knows statements. In
summary, our social network data set has 4987 person entities (along with
their unary labels), 24953 friendship statements, and 4987 episodes of social
events.

##  E Implementation Details

In this section, we focus on the implementation aspects and provide some
details about the network architecture and training hyperparameters. Our
program is written in Python and utilizes PyTorch.

###  E.1 Network Architecture

Our model mainly consists of two layers. The representation layer $\mathbf{q}$
has $r=4096$ neurons. The index layer $\mathbf{n}$ has $N_{T}+N_{C}+N_{P}$
neurons. We use VGG-19 backbone as the deep neural network $\mathbf{f}(\cdot)$
which takes as input a scene or a bounding box and outputs a 4096-dimensional
feature vector. The VGG-19 network consists of a sequence of convolutional
blocks followed by two fully connected hidden layers. Each convolutional block
is a sequence of 2 convolution layers with 3x3 filters, a max-pooling layer,
and another two convolution layers with the same parameters. We use the
activations before the nonlinear transformation from the last hidden layer and
copy them over to $\mathbf{q}$. Thus the layer $\mathbf{q}$ stores pre-
activation values in the range of $\mathbb{R}$. Next, we apply a nonlinear
function (LeakyReLU) to the values in $\mathbf{q}$ and feed them to the index
layer via connection weights $\mathbf{A}^{\top}$. At different decoding steps,
$\mathbf{n}$ covers different sets of indices, namely $\mathbf{n}_{T}$ has
$N_{T}$ units for episodic instances, $\mathbf{n}_{S}$, $\mathbf{n}_{C}$,
$\mathbf{n}_{O}$, all have $N_{C}$ concepts units (entities, classes,
attributes), and $\mathbf{n}_{P}$ has $N_{P}$ units for predicates. The index
layer in turn activates the representation layer via the same weights
$\mathbf{A}$. To calculate the enhanced representation $\mathbf{q}_{T}$,
$\mathbf{q}_{S}$, and $\mathbf{q}_{O}$, we add the pre-activation of layer
$\mathbf{q}$ and corresponding activations from $\mathbf{n}$. To obtain
$\mathbf{n}_{S}$ and $\mathbf{n}_{O}$, we apply softmax on the output with an
inverse temperature $\beta=1$. For $\mathbf{n}_{C}$ we split the concepts into
eight sets and apply softmax on each set. Alternatively, we could also use the
sigmoid function as stated in our algorithm. However, softmax fits here as our
labels are mutually exclusive, and in practice, softmax leads to faster
convergence and slightly better performance. The dynamic context layer
$\mathbf{h}$ contains 500 neurons with a self-connection via weight matrix
$\mathbf{B}$. The input to the dynamic context layer is post-activation of
$\mathbf{q}$, that is, values after a nonlinear transformation. There is a
direct path for the hidden layer between different decoding steps (the dotted
line between $\mathbf{h}$ blocks in figure 2), which stores the state of the
working memory and leads to a slight improvement (1%) in relationship
prediction. For $\bar{\mathbf{a}}$, we use a learnable embedding vector of
length 4096.

###  E.2 Training Scheme

We train a global set of parameters for our BTN with an objective of
minimizing the multi-task loss for perception, episodic, and semantic memory
experience. We set the batch size to 128, the learning rate to 0.0001, and
dropout $p$=0.5. The BTN is optimized using an Adam optimizer for 60 epochs.
During training, we freeze the VGG-19 layers except for its last fully
connected layer so that the knowledge in the pre-trained model will not be
destroyed. For the last layer of VGG-19, we use a smaller learning rate of
0.00001 to allow it to adapt to the new task. Except for the VGG-19 backbone,
we initialize our network using Kaiming uniform initialization proposed in (He
et al., 2015). We minimize the summed cross-entropy loss on $\mathbf{n}_{T}$,
$\mathbf{n}_{S}$, $\mathbf{n}_{C}$, $\mathbf{n}_{O}$, and $\mathbf{n}_{P}$ for
the perception and memory experience. For $\mathbf{n}_{S}$ and
$\mathbf{n}_{O}$, we randomly activate an index from the set of entity indices
and class/attribute labels for learning generalized statements (cf. 3.8). For
$\mathbf{n}_{C}$, we apply cross-entropy loss on each subset. For the social
network, we train the model for attribute prediction and relationship
prediction on the social network data set. Concretely, we minimize the cross-
entropy loss for $\mathbf{n}_{S}$, $\mathbf{n}_{C}$, $\mathbf{n}_{O}$, and
$\mathbf{n}_{P}$. For SSL, we use a batch size of 128, a learning rate of
1e-5, and total training epochs of 20. Except for the embeddings of time and
entities, all weights are frozen. Regarding the ablation studies, we train the
RESCAL model using the implementation of PyKeen (Ali et al., 2021). The rank
of entity embeddings and predicate embeddings are both set to 1000, which
gives a comparable number of learnable parameters as our model. All
experiments are conducted on an Nvidia GTX 1089 Ti GPU with a 4 core CPU of
16G memory.

## Code

The source code related to this article is available at the following link:
https://github.com/hangligit/BTN

## Acknowledgements

We thank the anonymous reviewers for their helpful comments on the manuscript.

## References

  * Ali et al. (2021) Mehdi Ali, Max Berrendorf, Charles Tapley Hoyt, Laurent Vermue, Sahand Sharifzadeh, Volker Tresp, and Jens Lehmann.  Pykeen 1.0: A python library for training and evaluating knowledge graph embeddings.  _Journal of Machine Learning Research_ , 22(82):1–6, 2021.
  * Anderson et al. (2018) Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang.  Bottom-up and top-down attention for image captioning and visual question answering.  In _CVPR_ , 2018.
  * Awh and Vogel (2020) E Awh and EK Vogel.  Online and off-line memory states in the human brain.  _the Cognitive Neurosciences_ , 2020.
  * Baars (1997) Bernard J Baars.  _In the theater of consciousness: The workspace of the mind_.  Oxford University Press, USA, 1997.
  * Baddeley (1988) Alan Baddeley.  Cognitive psychology and human memory.  _Trends in neurosciences_ , 11(4):176–181, 1988\.
  * Baddeley (1992) Alan Baddeley.  Working memory.  _Science_ , 255(5044):556–559, 1992.
  * Baddeley and Hitch (1974) Alan D Baddeley and Graham Hitch.  Working memory.  _The psychology of learning and motivation_ , 8:47–89, 1974\.
  * Baier et al. (2017) Stephan Baier, Yunpu Ma, and Volker Tresp.  Improving visual relationship detection using semantic modeling of scene descriptions.  In _ISWC_. Springer, 2017.
  * Baier et al. (2018) Stephan Baier, Yunpu Ma, and Volker Tresp.  Improving information extraction from images with learned semantic models.  In _Proceedings of the 27th International Joint Conference on Artificial Intelligence_ , pages 5214–5218. AAAI Press, 2018.
  * Barsalou et al. (2008) Lawrence W Barsalou et al.  Grounded cognition.  _Annual review of psychology_ , 59(1):617–645, 2008.
  * Bassett and Sporns (2017) Danielle S Bassett and Olaf Sporns.  Network neuroscience.  _Nature neuroscience_ , 20(3):353, 2017.
  * Becker (2005) Suzanna Becker.  A computational principle for hippocampal learning and neurogenesis.  _Hippocampus_ , 15(6):722–738, 2005.
  * Binder and Desai (2011) Jeffrey R Binder and Rutvik H Desai.  The neurobiology of semantic memory.  _Trends in cognitive sciences_ , 15(11):527–536, 2011.
  * Bloice et al. (2017) Marcus D Bloice, Christof Stocker, and Andreas Holzinger.  Augmentor: an image augmentation library for machine learning.  _arXiv preprint arXiv:1708.04680_ , 2017.
  * Bor and Seth (2012) Daniel Bor and Anil K Seth.  Consciousness and the prefrontal parietal network: insights from attention, working memory, and chunking.  _Frontiers in Psychology_ , 3:63, 2012.
  * Bordes et al. (2013) Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko.  Translating Embeddings for Modeling Multi-relational Data.  In _Advances in Neural Information Processing Systems 26_ , 2013.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901, 2020.
  * Browning and LeCun (2022) Jacob Browning and Yann LeCun.  AI and the limits of language.  _Noema Magazine_ , 2022.
  * Budson et al. (2022) Andrew E Budson, Kenneth A Richman, and Elizabeth A Kensinger.  Consciousness as a memory system.  _Cognitive and Behavioral Neurology_ , 2022.
  * Buschman and Miller (2020) Timothy J Buschman and Earl K Miller.  How working memory works.  _The Cognitive Neurosciences_ , page 357, 2020.
  * Dayan et al. (1995) Peter Dayan, Geoffrey E Hinton, Radford M Neal, and Richard S Zemel.  The Helmholtz machine.  _Neural computation_ , 7(5):889–904, 1995.
  * de Heer et al. (2017) Wendy A de Heer, Alexander G Huth, Thomas L Griffiths, Jack L Gallant, and Frédéric E Theunissen.  The hierarchical cortical organization of human speech processing.  _Journal of Neuroscience_ , 37(27):6539–6557, 2017.
  * Dehaene (2014) Stanislas Dehaene.  _Consciousness and the brain: Deciphering how the brain codes our thoughts_.  Penguin, 2014.
  * Dettmers et al. (2018) Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel.  Convolutional 2d knowledge graph embeddings.  In _Thirty-Second AAAI Conference on Artificial Intelligence_ , 2018\.
  * Duncan and Shohamy (2016) Katherine D Duncan and Daphna Shohamy.  Memory states influence value-based decisions.  _Journal of Experimental Psychology: General_ , 145(11):1420, 2016.
  * Eichenbaum (2014) Howard Eichenbaum.  Time cells in the hippocampus.  _Nature Reviews Neuroscience_ , 15(11), 2014.
  * Eichenbaum et al. (2012) Howard Eichenbaum, Magdalena Sauvage, Norbert Fortin, Robert Komorowski, and Paul Lipton.  Towards a functional organization of episodic memory in the medial temporal lobe.  _Neuroscience & Biobehavioral Reviews_, 36(7):1597–1608, 2012.
  * Evans (2003) Jonathan St BT Evans.  In two minds: dual-process accounts of reasoning.  _Trends in cognitive sciences_ , 7(10):454–459, 2003.
  * Evans (2012) Vyvyan Evans.  Cognitive linguistics.  _Wiley Interdisciplinary Reviews: Cognitive Science_ , 3(2):129–141, 2012.
  * Fellbaum (2010) Christiane Fellbaum.  Wordnet.  In _Theory and applications of ontology: computer applications_ , pages 231–243. Springer, 2010.
  * Fodor (1975) Jerry A Fodor.  _The language of thought_ , volume 5.  Harvard university press, 1975.
  * Frankland and Bontempi (2005) Paul W Frankland and Bruno Bontempi.  The organization of recent and remote memories.  _Nature Reviews Neuroscience_ , 6(2):119–130, 2005.
  * Friston (2010) Karl Friston.  The free-energy principle: a unified brain theory?  _Nature Reviews Neuroscience_ , 11(2):127–138, 2010.
  * Friston et al. (1995) Karl J Friston, G Tononi, O Sporns, and GM Edelman.  Characterising the complexity of neuronal interactions.  _Human Brain Mapping_ , 3(4):302–314, 1995.
  * Gärdenfors (2016) Peter Gärdenfors.  _The Geometry of Meaning: Semantics Based on Conceptual Spaces_.  MIT Press, 2016.
  * Gazzaniga et al. (2013) Michael S Gazzaniga, Richard B Ivry, and George Ronald Mangun.  _Cognitive Neuroscience: The biology of the mind_.  New York: WW Norton, fourth edition edition, 2013.
  * Gluck et al. (2013) Mark A Gluck, Eduardo Mercado, and Catherine E Myers.  _Learning and memory: From brain to behavior_.  Palgrave, 2013.
  * Gould et al. (1999) Elizabeth Gould, Alison J Reeves, Michael SA Graziano, and Charles G Gross.  Neurogenesis in the neocortex of adult primates.  _Science_ , 286(5439):548–552, 1999.
  * Goyal et al. (2021) Anirudh Goyal, Aniket Didolkar, Alex Lamb, Kartikeya Badola, Nan Rosemary Ke, Nasim Rahaman, Jonathan Binas, Charles Blundell, Michael Mozer, and Yoshua Bengio.  Coordination among neural modules through a shared global workspace.  _arXiv preprint arXiv:2103.01197_ , 2021.
  * Graves et al. (2014) Alex Graves, Greg Wayne, and Ivo Danihelka.  Neural turing machines.  _arXiv preprint arXiv:1410.5401_ , 2014.
  * Greenberg and Verfaellie (2010) Daniel L Greenberg and Mieke Verfaellie.  Interdependence of episodic and semantic memory: evidence from neuropsychology.  _Journal of the International Neuropsychological society_ , 16(05):748–753, 2010.
  * Griffiths et al. (2008) Thomas L Griffiths, Charles Kemp, and Joshua B Tenenbaum.  Bayesian models of cognition.  In _The Cambridge Handbook of Computational Psychology_. Cambridge University Press, 2008.
  * Hackbusch (2012) Wolfgang Hackbusch.  _Tensor spaces and numerical tensor calculus_ , volume 42.  Springer, 2012.
  * Halford et al. (1998) Graeme S. Halford, William H. Wilson, and Steven Phillips.  Processing capacity defined by relational complexity: Implications for comparative, developmental, and cognitive psychology.  _Behavioral and Brain Sciences_ , 21(06):803–831, 1998.
  * Halford et al. (2014) Graeme S Halford, William H Wilson, Glenda Andrews, and Steven Phillips.  _Categorizing cognition: Toward conceptual coherence in the foundations of psychology_.  MIT Press, 2014.
  * Han et al. (2020a) Zhen Han, Peng Chen, Yunpu Ma, and Volker Tresp.  Explainable subgraph reasoning for forecasting on temporal knowledge graphs.  In _International Conference on Learning Representations_ , 2020a.
  * Han et al. (2020b) Zhen Han, Yunpu Ma, Yuyi Wang, Stephan Guennemann, and Volker Tresp.  Graph hawkes neural network for forecasting on temporal knowledge graphs.  In _Automated Knowledge Base Construction_ , 2020b.
  * Harnad (1990) Stevan Harnad.  The symbol grounding problem.  _Physica D: Nonlinear Phenomena_ , 42(1-3):335–346, 1990.
  * He et al. (2015) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Delving deep into rectifiers: Surpassing human-level performance on imagenet classification.  In _Proceedings of the IEEE international conference on computer vision_ , pages 1026–1034, 2015.
  * Hearst (1992) Marti A Hearst.  Automatic acquisition of hyponyms from large text corpora.  In _COLING 1992 Volume 2: The 14th International Conference on Computational Linguistics_ , 1992.
  * Hemmer and Steyvers (2009) Pernille Hemmer and Mark Steyvers.  A Bayesian account of reconstructive memory.  _Topics in Cognitive Science_ , 1(1):189–202, 2009.
  * Hinton and Nowlan (1990) Geoffrey E Hinton and Steven J Nowlan.  The bootstrap widrow-hoff rule as a cluster-formation algorithm.  _Neural Computation_ , 2(3):355–362, 1990.
  * Hintzman (1984) Douglas L Hintzman.  Minerva 2: A simulation model of human memory.  _Behavior Research Methods, Instruments, & Computers_, 16(2):96–101, 1984.
  * Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber.  Long short-term memory.  _Neural computation_ , 9(8):1735–1780, 1997.
  * Hommel et al. (2001) Bernhard Hommel, Jochen Müsseler, Gisa Aschersleben, and Wolfgang Prinz.  The theory of event coding (tec): A framework for perception and action planning.  _Behavioral and brain sciences_ , 24(5):849–878, 2001.
  * Hudson and Manning (2019) Drew Hudson and Christopher D Manning.  Learning by abstraction: The neural state machine.  In _Advances in Neural Information Processing Systems_ , pages 5901–5914, 2019.
  * Humphreys et al. (1989) Michael S Humphreys, John D Bain, and Ray Pike.  Different ways to cue a coherent memory system: A theory for episodic, semantic, and procedural tasks.  _Psychological Review_ , 96(2):208, 1989.
  * Huth et al. (2016) Alexander G. Huth, Wendy A. de Heer, Thomas L. Griffiths, Frédéric E. Theunissen, and Jack L. Gallant.  Natural speech reveals the semantic maps that tile human cerebral cortex.  _Nature_ , 2016.
  * Johnson et al. (2015) Justin Johnson, Ranjay Krishna, Michael Stark, Li-Jia Li, David Shamma, Michael Bernstein, and Li Fei-Fei.  Image retrieval using scene graphs.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 3668–3678, 2015.
  * Jonides et al. (2008) John Jonides, Richard L Lewis, Derek Evan Nee, Cindy A Lustig, Marc G Berman, and Katherine Sledge Moore.  The mind and brain of short-term memory.  _Annual review of psychology_ , 59:193, 2008.
  * Jung et al. (2020) Alexander B. Jung, Kentaro Wada, Jon Crall, Satoshi Tanaka, Jake Graving, Christoph Reinders, Sarthak Yadav, Joy Banerjee, Gábor Vecsei, Adam Kraft, Zheng Rui, Jirka Borovec, Christian Vallentin, Semen Zhydenko, Kilian Pfeiffer, Ben Cook, Ismael Fernández, François-Michel De Rainville, Chi-Hung Weng, Abner Ayala-Acevedo, Raphael Meudec, and Matias Laporte.  Augmentor: an image augmentation library for machine learning.  https://github.com/aleju/imgaug, 2020.  Online; accessed 01-Feb-2020.
  * Kahneman (2011) Daniel Kahneman.  _Thinking, fast and slow_.  Macmillan, 2011.
  * Kanerva (1988) Pentti Kanerva.  _Sparse distributed memory_.  MIT press, 1988.
  * Kiefer and Pulvermüller (2012) Markus Kiefer and Friedemann Pulvermüller.  Conceptual representations in mind and brain: theoretical developments, current evidence and future directions.  _cortex_ , 48(7):805–825, 2012.
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ , 2014.
  * Kitamura et al. (2015a) Takashi Kitamura, Christopher J Macdonald, and Susumu Tonegawa.  Entorhinal–hippocampal neuronal circuits bridge temporally discontiguous events.  _Learning & memory (Cold Spring Harbor, NY)_, 22(9):438–443, 2015a.
  * Kitamura et al. (2015b) Takashi Kitamura, Chen Sun, Jared Martin, Lacey J Kitch, Mark J Schnitzer, and Susumu Tonegawa.  Entorhinal cortical ocean cells encode specific contexts and drive context-specific fear memory.  _Neuron_ , 87(6):1317–1331, 2015b.
  * Knill and Pouget (2004) David C Knill and Alexandre Pouget.  The Bayesian brain: the role of uncertainty in neural coding and computation.  _Trends in Neurosciences_ , 27(12):712–719, 2004\.
  * Koch (2014) Christof Koch.  Keep it in mind.  _Scientific American Mind_ , 25(3):26–29, 2014\.
  * Koch et al. (2016) Christof Koch, Marcello Massimini, Melanie Boly, and Giulio Tononi.  Neural correlates of consciousness: progress and problems.  _Nature Reviews Neuroscience_ , 17(5):307, 2016\.
  * Koner et al. (2020) Rajat Koner, Suprosanna Shit, and Volker Tresp.  Relation transformer network.  _arXiv preprint arXiv:2004.06193_ , 2020.
  * Körding et al. (2004) Konrad P Körding, Shih-pi Ku, and Daniel M Wolpert.  Bayesian integration in force estimation.  _Journal of Neurophysiology_ , 92(5):3161–3165, 2004.
  * Kriegeskorte and Douglas (2018) Nikolaus Kriegeskorte and Pamela K Douglas.  Cognitive computational neuroscience.  _Nature neuroscience_ , 21(9):1148–1160, 2018\.
  * Krishna et al. (2017) Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, and David A Shamma.  Visual genome: Connecting language and vision using crowdsourced dense image annotations.  _International Journal of Computer Vision_ , 123(1):32–73, 2017.
  * Kumar et al. (2015) Ankit Kumar, Ozan Irsoy, Jonathan Su, James Bradbury, Robert English, Brian Pierce, Peter Ondruska, Ishaan Gulrajani, and Richard Socher.  Ask me anything: Dynamic memory networks for natural language processing.  _arXiv preprint arXiv:1506.07285_ , 2015.
  * Kumaran et al. (2016) Dharshan Kumaran, Demis Hassabis, and James L McClelland.  What learning systems do intelligent agents need?  _Trends in Cognitive Sciences_ , 20(7):512–534, 2016.
  * Lakoff et al. (1999) George Lakoff, Mark Johnson, and John F Sowa.  Review of philosophy in the flesh: The embodied mind and its challenge to western thought.  _Computational Linguistics_ , 25(4):631–634, 1999\.
  * Lee et al. (2013) Dong-Hyun Lee et al.  Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks.  In _Workshop on challenges in representation learning, ICML_ , 2013\.
  * Leopold et al. (2019) David A Leopold, Peter L Strick, Danielle S Bassett, Randy M Bruno, Hermann Cuntz, Kristen M Harris, Marcel Oberlaender, and Marcus E Raichle.  Functional architecture of the cerebral cortex.  In _The Neocortex_ , pages 141–164. MIT Press, 2019.
  * Lu et al. (2016) Cewu Lu, Ranjay Krishna, Michael Bernstein, and Li Fei-Fei.  Visual relationship detection with language priors.  In _ECCV_. Springer, 2016.
  * Luo et al. (2019) Ruotian Luo, Ning Zhang, Bohyung Han, and Linjie Yang.  Context-aware zero-shot recognition.  _arXiv preprint arXiv:1904.09320_ , 2019.
  * Ma et al. (2018a) Yunpu Ma, Marcel Hildebrandt, Stephan Baier, and Volker Tresp.  Holistic representations for memorization and inference.  _UAI_ , 2018a.
  * Ma et al. (2018b) Yunpu Ma, Volker Tresp, and Erik A Daxberger.  Embedding models for episodic knowledge graphs.  _Journal of Web Semantics_ , 2018b.
  * Mannila et al. (1997) Heikki Mannila, Hannu Toivonen, and A Inkeri Verkamo.  Discovery of frequent episodes in event sequences.  _Data mining and knowledge discovery_ , 1(3):259–289, 1997.
  * McClelland et al. (1995) James L McClelland, Bruce L McNaughton, and Randall C O’Reilly.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory.  _Psychological review_ , 102(3):419, 1995.
  * McClelland et al. (2020) James L McClelland, Felix Hill, Maja Rudolph, Jason Baldridge, and Hinrich Schütze.  Placing language in an integrated understanding system: Next steps toward human-level performance in neural language models.  _Proceedings of the National Academy of Sciences_ , 117(42):25966–25974, 2020.
  * Montague (1970) Richard Montague.  Universal grammar.  _Theoria_ , 36(3):373–398, 1970.
  * Moscovitch et al. (2016) Morris Moscovitch, Roberto Cabeza, Gordon Winocur, and Lynn Nadel.  Episodic memory and beyond: the hippocampus and neocortex in transformation.  _Annual review of psychology_ , 67:105–134, 2016.
  * Nadel and Moscovitch (1997) Lynn Nadel and Morris Moscovitch.  Memory consolidation, retrograde amnesia and the hippocampal complex.  _Current opinion in neurobiology_ , 7(2):217–227, 1997.
  * Nickel et al. (2011) Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel.  A three-way model for collective learning on multi-relational data.  In _Proceedings of the 28th International Conference on International Conference on Machine Learning_ , pages 809–816, 2011.
  * Nickel et al. (2012) Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel.  Factorizing YAGO: scalable machine learning for linked data.  In _Proceedings of the 21st International Conference on World Wide Web_ , WWW ’12, pages 271–280, 2012.
  * Nickel et al. (2014) Maximilian Nickel, Xueyan Jiang, and Volker Tresp.  Reducing the Rank in Relational Factorization Models by Including Observable Patterns.  In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, editors, _Advances in Neural Information Processing Systems 27_ , pages 1179–1187. Curran Associates, Inc., 2014.
  * Nickel et al. (2015a) Maximilian Nickel, Kevin Murphy, Volker Tresp, and Evgeniy Gabrilovich.  A review of relational machine learning for knowledge graphs.  _Proceedings of the IEEE_ , 2015a.
  * Nickel et al. (2015b) Maximilian Nickel, Lorenzo Rosasco, and Tomaso Poggio.  Holographic embeddings of knowledge graphs.  _arXiv preprint arXiv:1510.04935_ , 2015b.
  * Noy et al. (2006) Natasha Noy, Alan Rector, Pat Hayes, and Chris Welty.  Defining n-ary relations on the semantic web.  _W3C working group note_ , 12(4), 2006.
  * Ogden and Richards (1923) Charles Kay Ogden and Ivor Armstrong Richards.  _The meaning of meaning: A study of the influence of thought and of the science of symbolism_.  Harcourt, Brace & World, Inc., 1923.
  * Osth and Dennis (2015) Adam F Osth and Simon Dennis.  Sources of interference in item and associative recognition memory.  _Psychological review_ , 122(2):260, 2015.
  * Pellionisz and Llinás (1980) András Pellionisz and Rodolfo Llinás.  Tensorial approach to the geometry of brain function: Cerebellar coordination via a metric tensor.  _Neuroscience_ , 5(7):1125–1136, 1980.
  * Plate (1997) Tony Plate.  A common framework for distributed representation schemes for compositional structure.  _Connectionist systems for knowledge representation and deduction_ , pages 15–34, 1997.
  * Poggio et al. (2020) Tomaso Poggio, Andrzej Banburski, and Qianli Liao.  Theoretical issues in deep networks.  _Proceedings of the National Academy of Sciences_ , 117(48):30039–30045, 2020.
  * Pollack (1990) Jordan B Pollack.  Recursive distributed representations.  _Artificial Intelligence_ , 46(1):77–105, 1990\.
  * Pope et al. (2021) Maria Pope, Makoto Fukushima, Richard F Betzel, and Olaf Sporns.  Modular origins of high-amplitude cofluctuations in fine-scale functional connectivity dynamics.  _Proceedings of the National Academy of Sciences_ , 118(46):e2109380118, 2021.
  * Popp et al. (2019) Margot Popp, Natalie M Trumpp, and Markus Kiefer.  Processing of action and sound verbs in context: An fmri study.  _Translational Neuroscience_ , 10(1):200–222, 2019.
  * Quiroga et al. (2005) R Quian Quiroga, Leila Reddy, Gabriel Kreiman, Christof Koch, and Itzhak Fried.  Invariant visual representation by single neurons in the human brain.  _Nature_ , 435(7045):1102–1107, 2005.
  * Quiroga (2012) Rodrigo Quian Quiroga.  Concept cells: the building blocks of declarative memory functions.  _Nat Rev Neurosci_ , 13(8), 2012.
  * Ralph et al. (2017) Matthew A Lambon Ralph, Elizabeth Jefferies, Karalyn Patterson, and Timothy T Rogers.  The neural and computational bases of semantic cognition.  _Nature Reviews Neuroscience_ , 18(1):42, 2017\.
  * Rao and Ballard (1999) Rajesh PN Rao and Dana H Ballard.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects.  _Nature neuroscience_ , 2(1):79–87, 1999.
  * Ren et al. (2015) Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks.  In _Advances in neural information processing systems_ , pages 91–99, 2015.
  * Rolls (2016) Edmund T Rolls.  _Cerebral cortex: principles of operation_.  Oxford University Press, 2016.
  * Rueschemeyer et al. (2010) Shirley-Ann Rueschemeyer, Daan van Rooij, Oliver Lindemann, Roel M Willems, and Harold Bekkering.  The function of words: Distinct neural correlates for words denoting differently manipulable objects.  _Journal of cognitive neuroscience_ , 22(8):1844–1851, 2010.
  * Ruffinelli et al. (2019) Daniel Ruffinelli, Samuel Broscheit, and Rainer Gemulla.  You can teach an old dog new tricks! on training knowledge graph embeddings.  In _International Conference on Learning Representations_ , 2019.
  * Russakovsky et al. (2015) Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, and Michael Bernstein.  Imagenet large scale visual recognition challenge.  _International journal of computer vision_ , 115(3):211–252, 2015.
  * Sabour et al. (2017) Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton.  Dynamic routing between capsules.  _Advances in neural information processing systems_ , 30, 2017.
  * Seth and Bayne (2022) Anil K Seth and Tim Bayne.  Theories of consciousness.  _Nature Reviews Neuroscience_ , pages 1–14, 2022.
  * Shapiro (2010) Lawrence Shapiro.  _Embodied cognition_.  Routledge, 2010.
  * Sharifzadeh et al. (2019) Sahand Sharifzadeh, Max Berrendorf, and Volker Tresp.  Improving visual relation detection using depth maps.  _arXiv preprint arXiv:1905.00966_ , 2019.
  * Sharifzadeh et al. (2020) Sahand Sharifzadeh, Sina Moayed Baharlou, and Volker Tresp.  Classification by attention: Scene graph classification with prior knowledge, 2020.
  * Simonyan and Zisserman (2014) Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  _arXiv preprint arXiv:1409.1556_ , 2014.
  * Singer (2001) Wolf Singer.  Consciousness and the binding problem.  _Annals of the New York Academy of Sciences_ , 929(1):123–146, 2001.
  * Smolensky (1990) Paul Smolensky.  Tensor product variable binding and the representation of symbolic structures in connectionist systems.  _Artificial intelligence_ , 46(1):159–216, 1990\.  URL http://www.sciencedirect.com/science/article/pii/000437029090007M.
  * Socher et al. (2009) Richard Socher, Samuel Gershman, Per Sederberg, Kenneth Norman, Adler J Perotte, and David M Blei.  A Bayesian analysis of dynamics in free recall.  In _Advances in neural information processing systems_ , pages 1714–1722, 2009.
  * Socher et al. (2013) Richard Socher, Danqi Chen, Christopher D Manning, and Andrew Ng.  Reasoning With Neural Tensor Networks for Knowledge Base Completion.  In _Advances in Neural Information Processing Systems 26_ , 2013.
  * Sporns (2018) Olaf Sporns.  Graph theory methods: applications in brain networks.  _Dialogues in clinical neuroscience_ , 20(2):111, 2018.
  * Squire (1987) Larry R Squire.  _Memory and brain_.  Oxford UP, 1987.
  * Squire and Alvarez (1995) Larry R Squire and Pablo Alvarez.  Retrograde amnesia and memory consolidation: a neurobiological perspective.  _Current opinion in neurobiology_ , 5(2):169–177, 1995.
  * Steyvers et al. (2004) Mark Steyvers, Richard M Shiffrin, and Douglas L Nelson.  Word association spaces for predicting semantic similarity effects in episodic memory.  _Experimental cognitive psychology and its applications: Festschrift in honor of Lyle Bourne, Walter Kintsch, and Thomas Landauer_ , pages 237–249, 2004.
  * Stickgold (2005) Robert Stickgold.  Sleep-dependent memory consolidation.  _Nature_ , 437(7063):1272–1278, 2005.
  * Sukhbaatar et al. (2015) Sainbayar Sukhbaatar, Jason Weston, and Rob Fergus.  End-to-end memory networks.  In _Advances in neural information processing systems_ , pages 2440–2448, 2015.
  * Sun and Peterson (1996) Ron Sun and Todd Peterson.  Learning in reactive sequential decision tasks: The clarion model.  In _Proceedings of International Conference on Neural Networks (ICNN’96)_ , volume 2, pages 1073–1078. IEEE, 1996.
  * Tenenbaum et al. (2006) Joshua B Tenenbaum, Thomas L Griffiths, and Charles Kemp.  Theory-based Bayesian models of inductive learning and reasoning.  _Trends in cognitive sciences_ , 10(7):309–318, 2006.
  * Teyler and DiScenna (1986) Timothy J Teyler and Pascal DiScenna.  The hippocampal memory indexing theory.  _Behavioral neuroscience_ , 1986.
  * Teyler and Rudy (2007) Timothy J Teyler and Jerry W Rudy.  The hippocampal indexing theory and episodic memory.  _Hippocampus_ , 2007.
  * Tibshirani (1996) Robert Tibshirani.  Regression shrinkage and selection via the lasso.  _Journal of the Royal Statistical Society: Series B (Methodological)_ , 58(1):267–288, 1996.
  * Tolman (1948) Edward C Tolman.  Cognitive maps in rats and men.  _Psychological review_ , 55(4):189, 1948.
  * Tomasello et al. (2017) Rosario Tomasello, Max Garagnani, Thomas Wennekers, and Friedemann Pulvermüller.  Brain connections of words, perceptions and actions: A neurobiological model of spatio-temporal semantic activation in the human cortex.  _Neuropsychologia_ , 98:111–129, 2017.
  * Tonegawa et al. (2018) Susumu Tonegawa, Mark D Morrissey, and Takashi Kitamura.  The role of engram cells in the systems consolidation of memory.  _Nature Reviews Neuroscience_ , 19(8):485, 2018\.
  * Tresp and Ma (2016) Volker Tresp and Yunpu Ma.  The tensor memory hypothesis.  In _NIPS Workshop on Representation Learning_ , 2016.
  * Tresp et al. (2015) Volker Tresp, Cristóbal Esteban, Yinchong Yang, Stephan Baier, and Denis Krompaß.  Learning with memory embeddings.  _arXiv preprint arXiv:1511.07972_ , 2015.
  * Tresp et al. (2017a) Volker Tresp, Yunpu Ma, and Stephan Baier.  Tensor memories.  In _Conference on Cognitive Computational Neuroscience_ , 2017a.
  * Tresp et al. (2017b) Volker Tresp, Yunpu Ma, Stephan Baier, and Yinchong Yang.  Embedding learning for declarative memories.  In _European Semantic Web Conference_ , pages 202–216. Springer, 2017b.
  * Tresp et al. (2019) Volker Tresp, Sahand Sharifzadeh, and Dario Konopatzki.  A model for perception and memory.  In _Conference on Cognitive Computational Neuroscience_ , 2019.
  * Tresp et al. (2020) Volker Tresp, Sahand Sharifzadeh, Dario Konopatzki, and Yunpu Ma.  The tensor brain: Semantic decoding for perception and memory, 2020.
  * Trouillon et al. (2016) Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard.  Complex embeddings for simple link prediction.  In _International Conference on Machine Learning_ , pages 2071–2080. PMLR, 2016.
  * Tulving (1972) Endel Tulving.  Episodic and semantic memory 1.  _Organization of Memory. London: Academic_ , 381(e402):4, 1972.
  * Tulving (1985) Endel Tulving.  _Elements of episodic memory_.  Oxford University Press, 1985.
  * Tulving (2002) Endel Tulving.  Episodic memory: from mind to brain.  _Annual review of psychology_ , 53(1):1–25, 2002\.
  * van den Heuvel and Sporns (2013) Martijn P van den Heuvel and Olaf Sporns.  Network hubs in the human brain.  _Trends in cognitive sciences_ , 17(12):683–696, 2013.
  * Van der Maaten and Hinton (2008) Laurens Van der Maaten and Geoffrey Hinton.  Visualizing data using t-sne.  _Journal of machine learning research_ , 9(11), 2008.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In _Advances in neural information processing systems_ , pages 5998–6008, 2017.
  * Weston et al. (2014) Jason Weston, Sumit Chopra, and Antoine Bordes.  Memory networks.  _arXiv preprint arXiv:1410.3916_ , 2014.
  * Whittington et al. (2020) James CR Whittington, Timothy H Muller, Shirley Mark, Guifen Chen, Caswell Barry, Neil Burgess, and Timothy EJ Behrens.  The tolman-eichenbaum machine: Unifying space and relational memory through generalization in the hippocampal formation.  _Cell_ , 183(5):1249–1263, 2020.
  * Wu et al. (2019) Yuxin Wu, Alexander Kirillov, Francisco Massa, Wan-Yen Lo, and Ross Girshick.  Detectron2.  https://github.com/facebookresearch/detectron2, 2019.
  * Yang et al. (2014) Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng.  Embedding entities and relations for learning and inference in knowledge bases.  _arXiv preprint arXiv:1412.6575_ , 2014.
  * Yang et al. (2018) Jianwei Yang, Jiasen Lu, Stefan Lee, Dhruv Batra, and Devi Parikh.  Graph r-cnn for scene graph generation.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , pages 670–685, 2018.
  * Yee et al. (2014) Eiling Yee, Evangelia G Chrysikou, and Sharon L Thompson-Schill.  _The Cognitive Neuroscience of Semantic Memory_.  Oxford Handbook of Cognitive Neuroscience, Oxford University Press, 2014\.
  * Zellers et al. (2018) Rowan Zellers, Mark Yatskar, Sam Thomson, and Yejin Choi.  Neural motifs: Scene graph parsing with global context.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 5831–5840, 2018.
  * Zhang et al. (2017) Hanwang Zhang, Zawlin Kyaw, Shih-Fu Chang, and Tat-Seng Chua.  Visual translation embedding network for visual relation detection.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 5532–5540, 2017.