# Advancing Perception in Artificial Intelligence through Principles of
Cognitive Science

Palaash Agrawal, Cheston Tan and Heena Rathore P. Agrawal and C. Tan are with
the Center for Frontier AI Research, Agency for Science, Technology and
Research (A*STAR), Singapore (e-mail<EMAIL_ADDRESS>cheston_tan@cfar.a-star.edu.sg).H. Rathore is with the Dept of Computer
Science, Texas State University, Texas, USA (e-mail:
heena.rathore@txstate.edu).

###### Abstract

Although artificial intelligence (AI) has achieved many feats at a rapid pace,
there still exist open problems and fundamental shortcomings related to
performance and resource efficiency. Since AI researchers benchmark a
significant proportion of performance standards through human intelligence,
cognitive sciences-inspired AI is a promising domain of research. Studying
cognitive science can provide a fresh perspective to building fundamental
blocks in AI research, which can lead to improved performance and efficiency.
In this review paper, we focus on the cognitive functions of perception, which
is the process of taking signals from one’s surroundings as input, and
processing them to understand the environment. Particularly, we study and
compare its various processes through the lens of both cognitive sciences and
AI. Through this study, we review all current major theories from various sub-
disciplines of cognitive science (specifically neuroscience, psychology and
linguistics), and draw parallels with theories and techniques from current
practices in AI. We, hence, present a detailed collection of methods in AI for
researchers to build AI systems inspired by cognitive science. Further,
through the process of reviewing the state of cognitive inspired AI, we point
out many gaps in the current state of AI (with respect to the performance of
the human brain), and hence present potential directions for researchers to
develop better perception systems in AI.

###### Index Terms:

cognitive sciences, perception, neuroscience, psychology, linguistics,
artificial intelligence

##  I Introduction

Artificial intelligence (AI) research has seen significant leaps in the last
decade [1]. However, there are still many limitations to the various
approaches used in AI, such as efficient storing and retrieval of vast amounts
of information from limited computational resources [2], the dependence of
present AI methods on large amounts of data and processing power [3], and the
ability to perform complex, layered and abstract reasoning. In order to
overcome these limitations, it is worthwhile to revisit the definition and
characteristics of intelligence. Understanding the nature of intelligence
itself is a crucial building block to the realization of artificial general
intelligence [4, 5].

Among the various definitions of intelligence, one of the most widely accepted
premises is that intelligence is cognitive in nature, i.e., it is defined
through the benchmarking of human performance and behavior [6]. To this end,
cognitive science inspired modeling in AI is seen as a promising approach to
advance the state of AI [7]. Cognitive modeling focuses on deriving functional
and structural mechanisms from the human brain along with its behavior and
applying them to AI models [8, 9] (see Figure 1). By understanding cognitive
modeling principles, one may understand the nature of intelligence in various
forms such as logic, functionality, ability to solve complex problems, social
behavior and ethics [10]. Cognitive theory can help intelligent agents
represent knowledge and reasoning logic more comprehensively [11, 12]. Many
works in AI research have in fact shown significant improvement in performance
by modeling based on cognitive principles. For example, a recent work showed
that a modular model performed well on various spatial navigation tasks [13].

Figure 1: Cognitive-AI is the intersection of cognitive sciences and
artificial intelligence. Cognitive sciences itself is a multidisciplinary
field, and the intersection of all these fields with AI (the green region) is
of great importance to understand intelligence. We study the state of
cognitive AI from a fresh perspective in this paper.

###  I-A Perception and the state of AI

The brain consists of various cognitive functional modules, such as
perception, attention and memory. While all these have their own significance,
perception is the gateway to all information collected by both the brain as
well as AI models. It is, hence, highly fundamental to intelligence, compared
to other cognitive functions. Perception is the process by which an agent
(biological or artificial) gathers raw sensory information from its
environment in various forms (modalities), and processes them in order to
derive meaningful information (features) [14]. In the brain, all other
cognitive functions build on top of sensory signals that are gathered and
processed through the various perceptual pathways. Further, various learning
processes in both biological and AI systems heavily rely on the nature and
quality of the information signals collected from their respective
surroundings, thus making perception the bottleneck to intelligence. Hence, as
AI continues to advance, improving perceptual processes is a critical step
towards creating more capable and intelligent systems [15]. Incorporating
cognitive science principles into models in AI can lead to AI systems that are
not only more capable but also more aligned with human cognition and behavior
[16].

The state of perception in AI is brittle and full of various kinds of open
problems, which limit the scope of application to new problems. From a high
level overview, visual perception (in the field of computer vision) still
suffers from problems such as susceptibility to noisy and adversarial input
data [17], long-tail recognition [18], the inability of models to learn from
limited data [19], and generalizing from two-dimensional visual input to the
semantics of three-dimensional visualizations [20]. Apart from these
fundamental problems, there are many avenues of research that aim to push the
boundaries of current computer vision systems, including perception of visual
input beyond the visible spectrum of light [21], multi-agent and multi-view
integration [22] and understanding of underlying characteristics of physics,
functionality and causality [23]. Similarly, linguistic perception (in the
field of natural language processing) has its own set of open problems that
limit current AI systems, including reasoning abilities [24], ability to
handle arithmetic representations [25], and understanding nuanced cross-
cultural linguistic phenomena such as euphemisms, idioms, and negations [26].

###  I-B Contributions of the paper

The primary goal of this paper is to review AI techniques, theories and
methods related to perception, that share characteristics with cognitive
science, directly or indirectly. We begin by studying prevalent theories from
cognitive science. Cognitive science is a multi-disciplinary field consisting
of many sub-disciplines related to functions of the brain. Of the various sub-
disciplines, we majorly focus on three sub-disciplines of cognitive science –
neuroscience (the study of the brain), psychology (the study of the mind) and
linguistics (the study of language and thought organization) 111Cognitive
science is widely defined as an culmination of five natural sciences –
psychology, neuroscience, linguistics, anthropology and philosophy; and their
intersection with computational intelligence [27]. In this discussion, we do
not cover the fields of philosophy and anthropology, due to their limited
contribution towards the current understanding of cognitive functions and
their underlying mechanisms. However, it should also be kept in mind that,
while philosophy and anthropology are not relevant to this particular
discussion, they may be immensely helpful in other forms of studies, such as
modeling of social behavior and ethics in AI [28, 29]..

From an academic perspective, cognitive science and AI are largely treated as
independent fields of study [30]. The last few decades have seen limited
interdisciplinary efforts, leading to distinct forms of formulating problem
statements and theoretical assumptions [31], modeling, and methods of
application. This also leads to a knowledge gap, where AI practitioners lack
understanding of cognitive science, and conversely, cognitive scientists lack
awareness about the trends in AI. Hence, translation of concepts from
cognitive science to problem-specific AI research involves a degree of
resistance for researchers. This paper aims to bridge this gap by providing a
fresh approach on cognitive AI through a bottom-up approach, which facilitates
the merger of fundamental concepts from cognitive science with the current
prevalent methods in AI, including that of data modeling, model training and
architectural designing.

The main contributions of this paper are as follows.

  1. 1.

Comprehensive state of cognitive AI: Firstly, we provide a fresh perspective
on cognitive science inspired AI, by studying AI methods and techniques that
intentionally or indirectly share characteristics with cognitive science,
while staying aligned with current research trends. This paper, hence, serves
as a collection of various methods at the disposal of researchers to build
effective cognitively inspired AI models.

  2. 2.

Comprehensive state of cognitive science: Secondly, this paper studies the
current state of cognitive science from a contemporary perspective. Since
cognitive science is an extremely wide field of research, we select theories
through the criteria that the theories must be widely accepted and relevant
(i.e., they must be cited by a large number of academic papers, especially in
recent works), must not be in conflict with an equally popular theory, and
must provide an interesting perspective to the field of AI, such as the
prospect of better performance or computational efficiency.

  3. 3.

Gaps in AI research: Finally, through this process, we point out the obvious
gaps in AI research, which can be addressed by studying principles and
theories from cognitive science. Thus, this paper presents many potential
directions of research in AI, that can be addressed through inspiration from
cognitive science.

##  II Background

###  II-A History of cognitive AI

The history of cognitive based AI modeling dates as back as the history of AI
research itself. Primitive developments in artificial neural networks and
learning algorithms were, in essence, a crude replication of the human brain
[32]. Earlier forms of cognitive frameworks for AI included theoretical and
philosophical propositions of the working of the human brain. These included
theories such as the Global Workspace Theory [33], the Society of Mind [34]
and the Pandemomium Theory [35], which essentially formulated cognitive
functions and emotions as a collection of numerous underlying small models,
the combination of which results in sophisticated models. Other forms of
philosophical cognitive frameworks included the CogAff framework [36], which
separates different cognitive processes into distinct layers, organized in a
structure that involves both vertical hierarchy and parallelization. However,
these frameworks were primarily conceptual in nature, and thus had limited
practical applications. These frameworks inspired the next generation of
cognitive architectures, which defined cognitively inspired layers of an AI
system more definitively. These cognitive architectures, such as SOAR [37] and
ACT-R [38], majorly focus on modeling high-level cognitive functions like
memory, learning and attention, in a symbolic fashion. With the rise of
advanced deep learning methods, various hybrid architectures, which combined
symbolic reasoning with data-driven learning are becoming increasingly popular
[39, 40].

Cognitive architectures provide valuable blueprints for defining relations
between various cognitive functions from a broad perspective. However, a
detailed review on the cognitive architectures of the past few decades [41]
revealed that, while a significant number of cognitive architectures had some
form of a cognitive-based layout of various functions, the said functions have
little to no cognitive influence in their core mechanism. Notably, perception
has often been merely treated as a mechanism to capture signals
electronically, where signals are more or less represented as standard
discrete input to subsequent functions. Without levying the advantage of
cognitive sciences at the foundational layers of these architectures, they are
bound to become bulky and inefficient. This inefficiency stems from connecting
various modules, each limited by their own technological constraints.

###  II-B Perception in the brain

Figure 2: Processes of perception in the brain. Perception is the collective
outcome of five major processes – sensory stimulation, modularity and
multisensory integration, bottom-up processing, top-down processing and
interpretation. For each of these processes, we study concepts from
neuroscience, psychology and linguistics, and draw parallels with AI.

We start by taking a deeper look into the meaning of perception in the brain,
and breaking down the process of perception into its various sub-processes.
Perception is the interpretation of the environment by an agent (biological or
artificial) through sensory input. This process involves the reception and
processing of raw information gathered from the surroundings. However, the
interpretation of this raw information is not done on its own; it is
influenced by various factors and undergoes several stages before leading to
meaningful understanding and action. The human brain aggregates perceptual
information from various sensory sources, such as vision, audition and
olfaction, all of which are processed and stored as neuro-electric signals
[42]. Perception consists of three major processes – sensory stimulation (or
the identification of sensory input), organization of information from
different sources [43] and interpretation of processed neural representations
[44]. These three sequential processes involve various sub-processes that are
crucial for efficient information processes. Understanding the organization,
sequence and relation between these different processes is crucial for capable
AI models to be developed.

  1. 1.

Sensory stimulation [45]: The first step in the perception process is to take
in a sensory input signal, which contains various different forms of signals
(such as visual and auditory).

  2. 2.

Organization: The sensory inputs are then processed in various stages in
specialized perceptual pathways.

    1. (a)

Modularity and multisensory integration [46]: Features are hierarchically
processed in different modules or areas of the brain, giving rise to highly
specialized functionality, all of which are integrated together to form a
common concept. Different modalities, such as vision, audition, and touch
consist of complementary information and are integrated together at various
stages, hence improving the robustness of perception.

    2. (b)

Bottom-up processing [47]: The sensory input is passed through various stages
of processing in a bottom-up fashion, where features (attributes) are
extracted and combined to form complex concepts, relations and patterns.

    3. (c)

Top-down processing [48, 49]: The perceptual pathway is not a fixed collection
of processes but is dynamically guided by other high-level cognitive
processes, such as attention and memory, in a top-down fashion of processing
information.

  3. 3.

Interpretation [50]: Processed sensory information is then combined with prior
knowledge and cues through prediction and inference to give more informed
results.

These processes can be visualized in detail through Figure 2. In the
subsequent sections, we explore each one of these five processes (sensory
stimulation, modularity and multisensory integration, bottom-up processing,
top-down processing and interpretation) of perception individually. For each
process, we gather relevant and prevalent theories from the domains of
neuroscience, psychology and linguistics, where we study various theories and
principles, and draw parallels with AI – highlighting the current state-of-
the-art and gaps in research that can potentially be addressed through
following the corresponding cognitive principles.

SENSORY STIMULATION
---
Domain | Topic | Cognitive Theory | Corresponding theory in AI
|  | Retinotopicity [51] | Inherent spatial organization in CNNs [52]
|  | Cortical magnification [53] |  | Fovea-inspired cortical sampling [54, 55] and data
---
transforms [56]
| Signal organization | Tonotopicity [57] | Frequency selective capability in neural networks [58]
|  |  |  | Chromaticity estimation methods for light, color and contrast
---
invariance [59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74,
75, 76, 77, 78]
|  |  | Scale invariant architectures for size constancy [79, 55]
|  | Perceptual constancy [80] |  | Estimation of perturbed view [81, 82] and constrastive learning
---
methods [83] for shape invariance
Neuroscience |  Perceptual constancy and object permanence | Object permanence [84] | Object tracking methods [85, 86, 87]
|  | Sensitivity [88] | Encoder-decoder based signal denoising approaches [89, 90]
Psychology |  Selective filtering of sensory signals | Response criterion [88] | Input signal selectivity [91]
|  |  | Predetermined tokenization methods [92, 93, 94, 95, 96]
Linguistics | Phonological processing | Categorical perception [97, 98] | Token grouping through dynamic learning [99]
MODULARITY AND MULTISENSORY INTEGRATION
Domain | Topic | Cognitive Theory | Corresponding theory in AI
|  |  | Explicit modular implementation in large parametric models [100]
|  Extraction of multimodal features |  Hierarchical feature extraction in specialized regions [101] | Natural modular organization of data in certain architectures [102]
Neuroscience | Integration of modalities | Temporal coincidence theory [103] | Nil
|  | Nature of modular
---
organization
| Bergeron’s theory of
---
modularity [104]
Cross-modal learning [105, 106]
|  |  |  | Classical multimodal feature fusion (late fusion, early fusion)
---
[107, 108]
|  |  |  | Dynamic integration methods (attention-based approaches
---
[109, 110] and mixture-of-experts models [111])
Psychology |  Functional flexibility due to dynamic integration | Dynamical systems theory [112] | Progressive integration of modalities[113]
|  |  | Storing knowledge in parametric spaces [114, 115]
Linguistics | Mental lexicon | Mental lexicon [116, 117] | Independent knowledge base entities [118, 119, 120]
BOTTOM-UP PROCESSING
Domain | Topic | Cognitive Theory | Corresponding theory in AI
|  | Population encoding [121] | Adversarial training for robustness [122]
Neuroscience | Signal propagation | Sparse encoding [123] | Out-of-distribution detection for sensitivity [124]
|  | Feature detection theory [125] | Feature detection based pattern recognition [126]
|  Pattern recognition mechanism | Geon theory [127] | Point cloud representation method [128, 129]
|  | Gestalt properties [130] |  | Gestalt estimation methods in traditional machine
---
learning[131, 132, 133]
Psychology |  Gestalt properties of visual perception |  | Gestalt properties in neural networks [134, 135, 136, 137, 138]
---
|  | Incremental conceptualization
---
and levels of abstractions
| Why-vs-how framework [139, 140]
---
| Complex context-aware reasoning in deep LLM layers [141]
---
|  |  |  | Exploratory creativity methods [142]
---
(e.g. reinforcement learning [143])
|  |  Creative engine in the default mode network [144] |  | Combinatorial creativity methods [142] (e.g.
---
Probabilistic time modeling [145])
Linguistics | Creativity | Spreading activation theory [146] |  | Transformation creativity methods[142](e.g. diffusion
---
models [147], adversarial networks [148])
TOP-DOWN PROCESSING
Domain | Topic | Cognitive Theory | Corresponding theory in AI
Neuroscience |  | Neuronal synchronization
---
as a guiding mechanism
| Neuronal synchronization theory
---
[149]
Nil
Psychology |  | Templates for guiding
---
bottom-up processing
Schema theory [150] | Concept representation framework [151]
|  | Garden path theory [152] |  Transformer based language model families (can exhibit both garden path [153, 154] and constrain based characteristics [155, 156] in different task settings)
Linguistics |  Filtering competing activation |  | Constrain based theory
---
[157]
INTERPRETATION
Domain | Topic | Cognitive Theory | Corresponding theory in AI
|  |  |  | Hierarchical propagation of error signals
---
through predictive coding principles [158]
|  |  Predictive coding theory [159, 160] |  | Recursiveness [161] and bidirectionalism [162] connectionist
---
models
|  |  | Dopamine-based reinforcement learning [163]
| Predictive processes | Prediction error [164] | Spiking errors in spiking neural networks [165]
|  |  | Variational free energy
---
optimization [166]
| Out-of-distribution detection [167] and few-shot
---
learning methods [168]
Neuroscience |  Optimization principles of interpretation |  | Expected free energy
---
optimization [169]
| Future event prediction through bayesian prediction [169] and
---
reinforcement learning [170]
| Nature of prediction | Dual theory process [171] | Estimation methods for heuristical prediction [172, 173, 174, 175, 176]
Psychology | Inference |  | Prior-informed probabilistic
---
inference [177, 178]
Bayesian inference methods [179, 180]
Linguistics | Semantic understanding |  | Discourse representation theory [181]
---
Coreference relation tracking in language models [182, 183]

TABLE I: Summary of methods in AI that derive inspiration from cognitive
science.

##  III Sensory stimulation

In this section, we highlight various properties of the initial stages of
perception in the brain. Specifically, we highlight various interesting
characteristics of the visual pathway, such as the robustness of the retina to
spatial distances, and the of the linguistic pathway, such as hearing a
continuous speech as distinct sounds for efficient understanding. The findings
are highlighted in Figure 3 and Table I.

###  III-A In neuroscience

####  III-A1 Signal organization

The neurons in different sensory pathways are organized systematically,
preserving a structured spatio-temporal input mapping. For example, in the
visual pathway, the signals originating from retinal cells are arranged to
form a spatial map (retinotopicity) [51], such that neuronal distances are
correlated with geometrical distances in the visual receptive field of the
eye. Similarly, in the auditory pathway, the cochlear neurons are arranged
tonotopically in the auditory pathway [57] which provides a structured
frequency gradient map, allowing the brain to distinguish frequencies
efficiently and even localize the source of a sound.

Retinotopicity exhibits some interesting properties, such as cortical
magnification, which allows efficient processing of high-resolution visual
input [53]. This is characterized with a neural density gradient around the
retinotopic center, resulting in high neural density at the center, and
gradually decreasing neural density in the periphery.

Parallel with AI: Similar to retintopic properties of the visual pathway,
convolutional neural networks (CNNs) also demonstrate spatial organizational
[52]. However, convolutional operations, by nature, lack the properties of
cortical magnification [54]. Convolutional operations, instead, sample the
visual field uniformly, which leads to inadequate allocation of computational
resources for visual areas requiring focussed processing. An attempt to
implement such a fovea-inspired cortical sampling was proposed by [54], where
a simple neural layer was introduced on top of a CNN, resulting in many
retinotopic organization properties including cortical magnification. In
general, cortical magnification inspired data transforms have been shown to
improve the performance of computer vision models [56].

Deep learning models aimed towards auditory tasks exhibit tonotopic (frequency
selective) properties as well. Certain layers of deep learning models are
capable of signal discrimination when trained on constrained tasks [58]. Even
though there is no universal rule of selecting the tonotopic regions of a
neural network, developing heuristics to identify task-specific frequency-
selective regions in a given model may be fruitful. Analyzing the frequency
response of neural networks can contribute towards many tonotopic abilities
such as frequency discrimination, sound segregation [184] and spatial
localization [185].

####  III-A2 Perceptual constancy and object permanence

Visual transduction is characterized by perceptual constancy [80] and object
permanence [84]. Perceptual constancy is the ability to perceive objects in
the visual field as static, despite physical variations such as lighting,
color and contrast [186, 187] (carried out at the retinal level), and other
variations such as size, shape, and warping/distortion [188, 189] (carried out
in later stages of visual processing). Object permanence, on the other hand,
is the ability to perceive the presence and state of an object even after its
partial occlusion, or complete disappearance from the visual field.

Both these mechanisms are related, and arise out of the synaptic ability to
access representations of objects with only partial sensory input activation,
and retain this representation even after the visual ceases to persist [190].
This indicates towards a neural mapping mechanism that maps sparse sensory
signals to higher dimensional information-rich representations in the initial
stages of perception.

Parallel with AI In traditional computer vision modeling, light, color and
contrast constancy (i.e. robustness towards physical variations) is achieved
through augmentation [191]. However, this approach easily breaks down on out-
of-distribution data [192]. An interesting alternative approach, referred to
as chromaticity estimation, aims to regulate the effect of physical variations
by estimating and correcting the color map of the visual input. Various types
of chromaticity estimation modeling approaches can be found in existing AI
literature, such as (in increasing order of computational complexity) –

  * •

Statistical approaches (including distribution analysis methods [59, 60] and
edge/filter based methods [61, 62]).

  * •

Traditional machine learning-based approaches (including unsupervised
clustering methods [63], pixel mapping methods [64, 65, 66, 67, 68],
probabilistic methods [69, 70], regression methods [71, 72, 73], and decision-
tree based methods [74]).

  * •

Deep learning based approaches [75, 76, 77].

Selection between these three approaches is a trade-off between computational
complexity and performance. Simpler statistical approaches may not be suitable
for highly accurate perceptual constancy but may be useful to simple AI
perception systems in a computational-friendly manner. On the other hand,
sophisticated approaches like deep learning methods are resource-intensive but
can handle complex distributions and can perform light enhancement, exposure
correction, tone mapping, and noise suppression simultaneously [78].

Previous studies have attempted to achieve size constancy through scale-
invariant architectural approaches in CNN [79]. However, this approach does
not generalize well to scales outside the training data distribution, because
of the inability of convolutional models to process global-scale features
[193]. However, [55] showed that a fovea-inspired convolutional architecture
can perform on previously unseen scales efficiently. Shape constancy methods,
which refers to the invariance of models to perturbations in the viewpoint of
the object (as in rotation, or warping), involve two types of approaches –
estimation of the perturbed view [81, 82], and contrastive learning
approaches, involving architectures that simultaneously learn various shape
variations of an object [83].

Object permanence has been an active area of research in the domain of object
tracking, and is majorly driven by data-centric approaches [85, 86, 87].
However, these solutions perform poorly and are not generalizable.

Overall, it is observed that the current state of perceptual constancy and
permanence in AI is largely reliant on the training distribution of the data
provided to the models. These models perform poorly when partial sensory
signals are provided to them. A higher dimension mapping strategy, similar to
the brain, may be a better alternative to the current data-driven approach.

###  III-B In psychology

####  III-B1 Selective filtering of sensory signals

Sensory stimulation is associated with interesting properties such as the
ability to ignore noise and the preference for certain types of signals.
According to the signal detection theory [88], there are two major criteria
used by the brain to filter the sensory input signal – sensitivity and
response criterion [88].

Sensitivity refers to the ability to filter out true signals from a noisy
environment. Response criterion, on the other hand, suggests that signals with
strength only above a specific threshold are processed. Together, these act as
a selective function that allows only the relevant and meaningful signals to
be processed. The sensitivity criterion and response thresholds are affected
by various factors such as expectation (anticipation) and prior knowledge and
are context-specific in nature [194].

Parallel with AI Similar to sensitivity in the signal detection theory, which
relates to filtering noise from input, various signal denoising approaches
have been proposed using encoder-decoder architectural approaches [89, 90].
However, unlike the brain, where feedback connections guide the magnitude of
signal sensitivity based on cues such as anticipation of a signal, predominant
AI approaches are purely feed-forward. While some early work on attention-
guided signal filtering methods provides foundational work for cue-driven
denoising [195, 196], there is a lack of research on context-driven denoising
based on anticipation of future signals.

There has been some work on input signal selectivity (similar to response
criterion in signal detection theory) [91], which aims to reject poor quality
input signals. However, another important goal of the response criterion is to
reject irrelevant signals in the agent’s surrounding. Most deep learning
models are trained in task-specific settings, which assumes that all input is
relevant and clean. This can however be rectified with minor changes in the
training regime,by training the model on mixture of relevant and irrelevant
signals, optimizing for task-relevant signals [197].

###  III-C In linguistics

####  III-C1 Phonological processing

The first step of speech processing in the language processing pathway
involves breaking down signals into simple and distinct units of sounds, known
as phonemes in the primary auditory cortex [198]. According to the categorical
perception theory [97, 98], phonemes are treated as independent categories,
which are categorized based on efficient phenome recognition [199].

Parallel with AI The major themes of phonological processing can be seen in
fundamental techniques used in natural language processing (NLP) tasks in the
field of AI. Similar to phoneme categorization, which involves breaking down
incoming language signals into small chunks, are also use in NLP tasks in the
form of various tokenization techniques [92, 93, 94]. Many tokenization
techniques such as subword tokenization [95] and wordpiece tokenization [96]
also emulate the efficient grouping of syllables as seen in phonological
processing. This type of grouping ensures that groups of symbols that occur
together more often are assigned an independent phenome category. Over time,
the brain updates phenome categories (for example, when learning new
languages, or during early development stages). Traditionally, NLP tasks use a
fixed token vocabulary, with no ability to dynamically regroup token
characters for efficient processing. Some works have attempted to learn
character grouping techniques using a gradient-based approach [99], allowing
efficient tokenization adapted to the language task at hand. However, updating
the token vocabulary requires models to be retrained from scratch, unlike the
brain, where phoneme recategorization can occur in parallel to linguistic
processing, without any degrading effect on the signal quality.

####  III-C2 Phonological buffer

According to Baddeley’s working memory model [200], the brain stores raw
signals temporarily in a memory element known as phonological buffer during
language comprehension. The buffer helps recall recently heard phrases without
significant neural processing, correct minor errors in the sensory input (such
as wrongly pronounced words, or missing sounds) [201]. It also maintains
mental intermediate states, while allows efficient mental arithmetic and
prediction of future events [202].

Parallel with AI Some characteristics of the phonological buffer are also seen
in the form of hidden states in recurrent models [203] (such as RNNs, GRUs,
and LSTMs) and the self-attention mechanism in transformer-based architectures
[204]. These hidden states store information about previous words and are
updated recursively in each iteration. While some features of the phonological
buffer are satisfied with this approach, such as deriving contextual
information from previous words, some important properties of the phonological
buffer are still not developed in AI models, such as the processing of low
resource language tasks by bypassing the feedforward processing pipeline.

##  IV Modularity and multisensory integration

Modularity in the brain is a highly sophisticated process, where signals are
incrementally divided and directed to different functional parts of the brain.
Signal modularization (that divides signals to perform different processes
independently) and signal integration (collection of the individually
processed sub-signals to obtain a unified representation of information) go
hand-in-hand in a highly structured yet dynamical method. A summary of these
processes in the brain can be visualized in Figure 3 and Table I.

###  IV-A In neuroscience

####  IV-A1 Extraction of multimodal features

The human brain exhibits sophisticated modularity, through the division of
perceptual processing into various specialized regions [101]. Different
modalities apply specific filters to the relevant sensory signal in
hierarchical stages, thus extracting features in progressively increasing
complexities.

Parallel with AI: Over-parameterization of models (in the order of trillions
of parameters in recent architectural approaches) [205] has demonstrated the
ability of recent deep learning models to process a vast range of tasks
simultaneously, without any explicit separation of signals. However, a modular
architecture with separated units of computation for different tasks and
modalities can provide many benefits, including parameter efficiency,
prevention of signal interference between different tasks, and generalization
to new emergent properties [100]. Such a modular approach in AI modeling has
only been explored in limited directions.

Modularity need not be explicitly designed architecturally. Certain
architectures like the transformer naturally result in modular groups of
computation when trained on large general-purpose datasets [102]. However,
identifying and adapting to the specific nature of different modules within a
network can lead to improved overall performance.

####  IV-A2 Integration of modalities

There exist specialized regions that are responsible for the integration of
the processed sensory information from the various modalities [206]. An
interesting theory related to perceptual integration is the temporal
coincidence theory [103], which suggests that temporally aligned neural
features are integrated together. Temporal frequency matching allows unique
patterns of intra-modal and cross-modal integration, allowing only relevant
information to be combined.

Parallel with AI: Majorly, multisensory integration is realized in AI by
processing different sensory signals (such as vision, audio, and text)
independently [207] in a static manner, where a fixed architecture treats each
modality identically. This approach neglects the different computational needs
of different modalities [113]. In contrast, the brain associate different
types of neural features with unique frequencies (as proposed in the temporal
coincidence theory [103]), allowing only similar signals to associate. This
dynamic grouping of signals can lead to improved performance by virtue of
shared processing of different modalities (rather than independent
processing). However, this concept has not been explored well in AI yet.

###  IV-B In psychology

####  IV-B1 Nature of modular organization

Unlike older theories that promote the isolation of cognitive modules in the
brain [208, 209], contemporary theories in cognitive sciences adopt a more
unified approach to modular organization. Bergeron’s theory of modularity
[104] posits a more flexible approach, where cognitive functions are viewed as
domain-specific, but not modular in a strictly physical sense. Each high-level
cognitive function, while fairly independent, consists of various lower-level
functions (sub-modules), which can be shared by other cognitive functions.

Parallel with AI: The existence of lower-level modules collectively shared by,
and progressively unified to isolated higher-level modules (as proposed by
Bergeron’s theory of modularity) remains majorly unexplored in AI. While
cross-modal learning [105, 106] presents some glimpses of sharing features
across modalities in intermediate layers of a network, no major work explores
progressive modularization of the model. Exploring this direction is an
interesting opportunity for AI research, and is expected to enhance AI model
performance and efficiency.

####  IV-B2 Functional flexibility due to dynamic integration

Sharing of cognitive resources across modules gives rise to new emergent
functions [210]. This is formally supported by the dynamical systems theory
[112], which claims that the integration of multisensory information takes
place dynamically through the interaction of various perceptual components in
arbitrary stages. Hence, multi-modal sensory information can be efficiently
combined in a flexible manner.

Parallel with AI: Classical multimodal models treated feature fusion in two
major ways – late fusion [107], where modalities were treated completely
independently, and early fusion, where different features are combined and
normalized before any processing [108]. While both these approaches have their
advantages and limitations, they share a common attribute of the lack of
dynamic integration of modalities, as proposed by the dynamic systems theory.

While some approaches provide crude solutions towards dynamic (i.e. task-
specific learnable) integration of modalities (including attention-based
approaches [109, 110] and mixture-of-expert models [111]), they still majorly
integrate modalities over a single layer. A potentially viable solution is
proposed in [113], a progressive integration of modalities is proposed over
multiple layers. The potential of such an approach can, however, only be
realized by scaling this solution to models with large sizes of parameters and
layers.

###  IV-C In linguistics

####  IV-C1 Mental lexicon

After phonological processing, speech signals are grouped together to form
distinct words, and definitions are derived from a dictionary, known as the
mental lexicon [116]. This mental dictionary acts as a modular system
connecting various related concepts, such as definitions, visual appearance,
spelling and pronunciation [211].

The mental lexicon has various interesting properties. For example, it is
known to be hierarchically organized, where similar concepts are grouped
together [117], forming distinct clusters of shared representations. Another
interesting property is that the mental lexicon is highly dynamic, where new
meanings are constantly learned and representations are constantly updated
[212], without affecting the efficiency of subsequent processes.

Parallel with AI: Unlike the concept of mental lexicon in the brain, which
acts as the storage house of all conceptual knowledge in the brain, the
mainstream approach in AI-based models is store information in parameters of
the model implicitly [114]. This approach of using parameters as a knowledge
base has significantly improved through research in the field of language
modeling [115]. However, there are still some fundamental issues in this
approach. As the size of the knowledge corpus increases, the models have to be
scaled exponentially in terms of the number of parameters [213] to achieve the
same theoretical performance. Larger models are also highly prone to incorrect
language predictions [214], and updating knowledge is highly tricky because of
the interdependent nature of parametric statistics with data distribution,
unless the entire model is retrained, which can be expensive [115].

Many works augment language models in AI to incorporate an independent
knowledge base entity [118]. This approach not only allows the language model
to offload information retrieval to an external module with larger
reliability, but also allows reduction of the model size in terms of
parameters. The knowledge base entity may take many forms, such as a knowledge
graph [119] or a simpler look-up based method [120]. One major advantage of an
independent knowledge base entity is that it can be updated over time, without
compromising the performance of the main prediction model. On the other hand,
models are sensitive to the entity representations of knowledge base, which is
why updating an knowledge base updating is tricky, and requires extensive
analyses before an update.

Figure 3: A visualization of all topics studied in this paper. We select
various relevant theories of perception from the different sub-domains of
cognitive sciences that have valuable insights to offer to AI.

##  V Bottom-up processing

Bottom-up processing, which is the process by which features are extracted
from signals is a multi-stage process, where each stage (or level) is
responsible for extraction of special types of features from sensory signals.
Consequently, it is observed that the various stages of bottom-up processing
have different functional mechanism. A brief summary of these methods, as
understood by cognitive sciences, can be referred to in Figure 3 and Table I.

###  V-A In neuroscience

####  V-A1 Signal propagation

At the core of bottom-up processing in the brain, the propagation of sensory
signals through specific regions of the brain results in the extraction of
meaningful features. The nature of information propagation, as suggested by
neural encoding experiments [215], is highly statistical. Sensory information
is typically propagated simultaneously by neurons within a neuronal group
[121] (known as population encoding), ensuring robustness and generalization
of the signal by suppressing noisy information. However, certain sensory
signals are propagated through sparse encoding [123], which involves the
propagation of signals through a relatively small number of neurons. This
allows the detection of specific signals with high sensitivity.

Parallel with AI: While training AI models to achieve robustness (through
techniques such as adversarial training [122]) allows models to withstand data
perturbations, sensitivity is useful to detect out-of-distribution data [124].
However, training models to achieve both robustness and sensitivity is tricky,
since the two properties are contradictory. While the former optimizes model
to ignore deviation from a statistical average, the latter promotes training
to identify minor deviations with high precision. However, some modeling
techniques have been developed to strike a balance between the two. This
includes stabilization of sensitive parameters [216] and the use of specific
loss functions [217].

###  V-B In psychology

####  V-B1 Pattern recognition mechanism

Many theories have been proposed to explain pattern recognition properties of
the brain [218]. However, the most recent and widely accepted theory of object
recognition mechanisms is the recognition-by-component (RBC) theory. RBC
theory is a general family of theories, which argues that objects are treated
as the collection of their constituent parts and properties.

This includes the feature detection theory [125], according to which
recognition of visual stimuli occurs as a result of various feature
identifiers in the brain, including edges, colors, shapes, and texture.
Another theory in the RBC family is the geon theory [127], which hypothesizes
that object recognition is a result of the clustering of various small
constituent 3D visual representations (geons).

Parallel with AI: Pattern recognition in AI is primarily developed on the
lines of feature detection theory [126]. However, the geon theory currently
has limited applications in AI, such as in point cloud representation methods
in 3D computer vision [128, 129]. Many characteristics of the geon theory such
as breaking down complex objects into a limited number of simple shapes, can
be integrated into mainstream AI research. This can potentially result in
efficient visual relationship detection [219], by virtue of reduced search
spaces for AI models, leading to better relations and patterns between geons.
It can also solve various perceptual constancy problems such as size and shape
invariance.

####  V-B2 Gestalt properties of visual perception

An interesting set of organizational properties of visual bottom-up processing
are the gestalt properties [130]. This set of properties treats the perception
process as a collective sensation of the environment, rather than individual
processing of elements. The gestalt theory describes visual perception to have
various properties such as proximity, similarity, continuity, closure,
symmetry, convexity, pr$\ddot{a}$gnanz, parallelism, and others [220, 221,
222].

Parallel with AI: There have been many attempts to produce generalized gestalt
estimates through machine learning approaches [131, 132]. In general, it has
been demonstrated that many AI models, including neural network based
approaches are capable of demonstrating most gestalt effects in specific
conditions [133].

Specific gestalt properties have been studied in depth in the field of deep
learning. For example, simple neural network-based computer vision approaches
have demonstrated similarity and proximity gestalts [134, 135], as well as
homogeneity [223]. However, the property of closure has proven to be more
difficult to achieve than others [136], where only specific settings illicit
closure properties [137]. Recent studies argue that contour closure should not
be considered as a physical property of a visual space, but rather an internal
relation that guides visual perception [138].

###  V-C In linguistics

####  V-C1 Incremental conceptualization and levels of abstractions

Raw definitions of word sequences are incrementally processed to derive
semantic meaning from speech. From the mental lexicon, multiple related sub-
concepts are derived and processed to form incremental levels of conceptual
abstractions. According to the why-vs-how framework [139], lower level
abstractions, corresponding to “how”, represent the underlying functional
characteristics of the concept, while higher level abstractions in the later
stages of bottom-up processing, corresponding to “why”, represent semantic
context of a thought. This framework also quantifies levels of abstractions
into concrete conceptual parameters [140] , such as concreteness, specificity,
imageability [224] and neutrality of the intended semantic message.

Parallel with AI: In general, deep-learning based models are known to process
incrementally sophisticated features through the various layers [225]. While
the lower level layers are responsible for feature extraction, the higher
level layers are responsible for extraction of complex features, semantics and
various other abstract concepts. Recently, very deep layered models (such as
networks of large language models (LLMs) [141]) have demonstrated the
capability of complex context-aware reasoning as well as alignment with human
values. This is very similar to the why-vs-how framework, where higher order
layers can effectively perform reasoning as well as complex conceptualization.
While certain aspects of abstractness have been individually assessed in these
models (such as neutrality [226] in language models), there is lack of studies
that compares the ability of models to demonstrate the various aspects of
abstractness parallelly, making it difficult to understand the true reasoning
capabilities of AI models.

####  V-C2 Creativity as a result of neural activation

The brain is a highly creative engine, where new ideas are triggered as a
result of forming connections between fragments of information from memory
[144]. This is primarily carried out in an independent internal network in the
brain known as the default mode network, which is primarily driven by the
objective of optimizing future event prediction [227]. Apart from this, an
interesting characteristic of the creativity mechanism is explained by the
spreading activation theory [146]. According to this theory, processing a
particular concept in a neural node activates neighboring neural pathways,
thus leading to creation of novel ideas [228].

Parallel with AI: From a computational perspective, creativity can majorly be
divided into three categories [142], each of which relates to different
methods in AI. This includes

  * •

Combinatorial creativity, which relates to combining priors with sensory
information in order to predict future events. This is seen in various
probabilistic time modeling techniques [145].

  * •

Exploratory creativity, which relates to predicting totally new events through
a trial-and-error approach. This can be seen in techniques such as
reinforcement learning [143], which optimize future event prediction by method
of exploration.

  * •

Transformational creativity, which converts sensory information to a new form
by combining it with other information cues. This can be seen in various
generative methods such as diffusion models [147] and adversarial networks
[148].

These various methods explore different aspects of the creative process of the
brain. For example, while combinatorial and exploratory methods in AI can be
designed for effective future event prediction, they cannot process related
information to generate novel ideas. On the other hand, transformational
creativity can effectively emulate the spreading activation theory, but has
not been yet explored well enough for future event prediction. A more holistic
approach towards creativity is hence required in this domain of research.

##  VI Top-down processing

Higher-order cognitive functions, such as anticipation, expectation, attention
and memory play important roles in guiding, shaping and filtering the signals
during perception. A sensory signal can behave differently in different
settings as a result of top-down processing. A summary is provided in Figure 3
and Table I.

###  VI-A In neuroscience

####  VI-A1 Neuronal Synchronization as a guiding mechanism

Top-down processing refers to the influence of various factors such as context
(sensory information related to a particular perceptual signal), and higher-
order cognitive processes (such as attention and memory), on the outcome of
perception, which guides thought in a particular direction [229]. Several
biological theories indicate mechanisms relating to top-down processing in the
brain. Some important theories include the neuronal synchronization theory
[149], suggesting that higher cognitive functions modulate the neural
oscillations in different regions of the brain, which guides the neural
pathway based on particular frequency channels matched by oscillations,
leading to different outcomes in different situations.

Parallel with AI: Theories of top-down processing from neuroscience, such as
neuronal synchronization, have not been well explored in AI. This theory
associates different perceptual signals to a characteristic firing frequency,
which helps in many cognitive properties such as integration of related
features and coupling of different modalities. The current approach in AI is
to treat each activation from the model as a discrete value, and all neuron
activations are simultaneously generated with each forward pass. This leads to
the treatment of AI models as one big black box. More research is required in
this domain.

###  VI-B In psychology

####  VI-B1 Templates for guiding bottom-up processing

All bottom-up processes are influenced by guiding cues from higher-level
cognitive functions. The nature of these guiding functions is described by the
schema theory [150], which proposes the existence of cognitive guides
(schemas). Schemas are imaginary frameworks that encapsulate generalized ideas
or object representations and are used as templates to guide perceptual
processing. Schemas make processing efficient by providing similar sensory
input signals with a predetermined perceptual path consisting of only relevant
processes.

Parallel with AI: In the context of AI, the schema theory is related to
concept representation in the deeper layers of the model. For example, the
representation of similar objects and concepts should be closer to each other,
irrespective of variations such as viewpoint or illumination. Similarly, it
may be desirable to increase the distance between two related objects in a
discriminatory setting. Understanding the feature representation space is
hence necessary to properly model constraints as well as optimization goals in
AI [230]. In [151], a crude framework for optimizing deep learning models is
presented, based on concept representation analysis.

However, there is still a lot of work that can be done in this field of
research. Feature representation of complex, compositional, or abstract
concepts is still not understood well. There is growing interest in this
domain, especially in the field of explainable AI [231, 232].

###  VI-C In linguistics

####  VI-C1 Filtering competing activations

The association and simultaneous triggering of multiple related subconcepts
involves selection among many competing alternatives in order to carry out
effective semantic understanding. According to the competing alternatives
theory [233], competing concepts such as polysemous (words that have multiple
definitions) and homonymous (words that sound the same but have different
meanings) words are sequentially filtered through higher-order cues via a top-
down feedback path. There are two major theories explaining selection between
competing alternatives in the brain. The garden path theory [152] proposes
that sentences are processed incrementally, and the cognitively simplest
definition is chosen as the interpretation, until it creates a conflict with
the narrative of the discourse. Once, a conflict is encountered, an
alternative interpretation is selected. The constrain based theory [157]
alternatively proposes that this selection process is majorly statistical in
nature, where the pathway with the highest activation is selected and
processed further. This phenomenon is more or less consistent across all
languages, as is observed through the study of implicit causality [234] (i.e.,
the understanding of the source of an action without an explicit mention in a
sentence).

Parallel with AI: While the garden path theory conserves working memory
resources, the constrain based theory is more resource-intensive, but highly
efficient. This presents a efficiency-performance trade-off from a
computational perspective. Depending on resource availability and performance
requirements, transformer based language model families (such as GPT and BERT)
can be modeled to exhibit both garden path based prediction[153, 154] as well
as constrain based prediction [155] in different task settings. Further, the
selection among competing concepts (across languages, tasks, etc.) can be
selectively designed through targeted fine-tuning [156].

##  VII Interpretation

Effective decision making in the brain is a result of combining processed
sensory information with other forms of knowledge, including prior knowledge,
previous states of sensory signals as well as various kinds of heuristics. The
overall interpretation mechanism of the brain can be visualized through Figure
3 and Table I.

###  VII-A In neuroscience

####  VII-A1 Predictive processes

According to a prominent theory in neuroscience in the brain is the predictive
coding theory [159, 160], the brain continuously predicts future sensory
signals in higher-level cortical areas, while also constantly optimizing
prediction in new environments. This mechanism allows for efficient perceptual
processing by reducing the amount of sensory data required to make informed
decisions. Predictive coding theory also acts as the primary method of
learning representations of the world.

Parallel with AI: Current AI algorithms derive many ideas from the predictive
coding mechanism, which postulates the general principles of prediction and
learning in the brain [235]. This includes hierarchical propagation of error
signal [158], similar to layerwise error propagation in backpropagation [236],
widely used in neural network optimization. Certain representations of the
prediction error [164], such as dopamine release, are also translated to
objective functions in reinforcement learning [163]. Other forms of predictive
error modeling, such as spiking error, can be widely seen in the body of
spiking neural network [165]. However, backpropagation operates by minimizing
a global loss function that drives the entire model architecture. Predictive
coding, on the other hand, provides means for learning based on available
local neural activity alone [237]. However, there is a body of work that has
focused on enhancing the current backpropagation algorithm through predictive
coding principles [238].

Predictive coding mechanism in the brain is also characterized by recursive
and bidirectional connections. Many classes of AI models have properties of
recursive connections (as in recurrent neural networks [161]) and
bidirectionalism (as in bidirectional LSTMs [162]). However, recurrent
connections in AI models have majorly been implemented locally about a single
layer. There are many scales of recurrent loops in the brain [239]. These
involve not only recurrent connections between layers, but between modules and
also local node groups with individual layers. This needs to better explored
in AI models.

####  VII-A2 Optimization principles of interpretation

Predictive coding falls under a more general theory known as the free energy
principle [240], according to which, predictive optimization mainly focuses on
reducing two types of energy costs of processing information, (a) variational
free energy [166], related to processing unfamiliar sensory signals, and (b)
expected energy, related to the general prediction of future events [169].

Parallel with AI: According to the free energy principle, the brain tries to
minimize variational free energy, which aims to minimize the surprise when the
brain comes across an unfamiliar input. e is executed by combining sensory
input and prior information. Through this mechanism, the brain performs well
on tasks such as few shot learning, and out-of-distribution detection, which
are particularly challenging tasks for AI models [241]. Many energy
optimization problems have been formulated in AI to tackle the problem of out
of distribution detection [167], as well as few shot learning [168]. The brain
also tries to minimize expected free energy, which aims to predict future
events with maximal certainty, through bayesian prediction [169]. This form of
energy optimization is also frequently observed in the field of reinforcement
learning [170]. There has also been work on frameworks to integrate both
variational free energy and expected free energy in the context of bayesian
modeling for AI model learning [242].

###  VII-B In psychology

####  VII-B1 Nature of prediction

Prediction in the brain is related to combining sensory input with contextual
information derived from mental models or frameworks [243] that are used to
predict future outcomes. These mental models are continuously learned and
manipulated based on the outcomes of perceptual events and other factors such
as rewards.

A contemporary approach towards prediction in the brain is proposed by the
dual theory process [171], which posits that the perceptual system consists of
two parallel modes of thinking - a heuristical method (used to make fast
inferences through cognitive shortcuts), and an active method of thinking(
involving step-by-step reasoning and careful deliberation of various possible
outcomes). The mix between the two types of responses is affected by factors
[244] such as time pressure (high-pressure tasks are carried out without much
deliberation), expected rewards (lower reward tasks are likely to be handled
with less deliberation), and experience (repetitive tasks are more likely
heuristically performed).

Parallel with AI: The dual theory process, which is well studied in cognitive
sciences proposes an interesting framework for AI systems. Current AI models
treat all input signal uniformly, through a systematic processing mechanism.
However, there are some classical approaches that can be used to augment the
current systematic reasoning methods with heuristical approaches, such as the
Order of Magnitude Reasoning framework [172, 173, 174], and various similar
reasoning frameworks [175, 176]. The interplay between both types of modules
may help AI systems to modulate between different time or resource constrained
contexts.

####  VII-B2 Inference

Along with various sensory clues, efficient decision-making also requires
prior information gained through experience [245]. Inference in the brain is
highly probabilistic in nature, where experiences that are more recent, and/or
repetitive (thus represented by a larger neuronal population) have a higher
effect on present perception [246]. While it is largely believed that the
brain follows bayesian-like probabilistic inference [177], recent theories
also claim the possibility of other probabilistic models of the inference
engine in the brain [178].

Parallel with AI: Bayesian approaches have consistently shown improvement in
many AI applications [179, 180]. However, the underlying information provided
by prior (or underlying pieces of information aiding decision making, which is
otherwise hard to learn with limited data) are not well understood, and
heavily underutilized in the context of AI [247]. Understanding the influence
of data that informs further decisions in a hierarchical manner [248] can
allow for better reasoning in AI systems, which currently lack complex
reasoning capabilities [249]. Better prior utilization can also improve
learning efficiency, by reducing the dependency of current AI systems on large
amounts of data for training [250].

###  VII-C In linguistics

####  VII-C1 Semantic understanding

The definitions of various sentences have to be combined to derive semantic
information for deep contextual understanding. According to the discourse
representation theory [181], the brain integrates knowledge across multiple
sentences by keeping track of the states of key entities (or “centers” [251]).
These states are referred to as situation states [252]. The event-indexing
model further divides situation states are into five major classes of
entities, namely protagonist (the center theme of the discourse), temporality,
causality, spatiality and intentionality [253]. During a sequence of events,
one or more entities of the sitation state may be updated in the memory [254].
Related events (i.e., events that differ by a minimal number of entity
classes) are more correlated in the memory than unrelated events.

Parallel with AI: Many language models [182, 183] demonstrate the ability to
perform basic entity tracking (such as capturing coreference relations [255])
through targeted training. While many datasets are specifically designed to
train and evaluate models on entity tracking tasks [256, 257, 258], entity
tracking is also an emergent property observed in large language models, as a
result of pretraining on large text corpora. However, text-based training
alone is incapable of training models for complex entity tracking. Complex
entity tracking is observed as a by-product of training on code-based data
(i.e. computer programs) along with text-based data [259]. This may be a
result of the involvement of structured and systematic transformation of
multiple variables states in code. However, code cannot capture many non-
trivial, unstructured linguistic semantics, which is why there is a need for
text-based datasets that can train models on complex and simultaneous tracking
of multiple entities in a discourse. Further, no dataset yet published
classifies entities based on the event-indexing model.

##  VIII Conclusion

Cognitive sciences clearly has many interesting insights to offer to
perception in AI. In Table I, many potential existing techniques in AI have
been highlighted that can bring about cognitive influence in perception
systems in AI. These methods, however, are only a starting point for AI
researchers, since many cognitively inspired methods currently existing in AI
are tailored to specific settings and assumptions. Thus, they may not be
generalizable to other problems where AI can be applied. Conversely, cognitive
scientists also can immensely benefit from adopting these methods to create
more viable and potent cognitive architectures that can provide quantifiably
more efficient intelligent systems.

###  VIII-A Major gaps in cognitive AI research

Through the process of reviewing the state of cognitive AI, we came across a
large number of themes from neuroscience, psychology and linguistics, that
have no, or minimal counterparts in the field of AI. Here, we summarize,
domain by domain, the major gaps in research, providing many potential
research directions for AI researchers.

##### Research gaps in neuroscience-based AI

We observe two major themes in neuroscience, that are more or less unexplored
in AI.

  * •

Firstly, neural architecture and design, which relates to the study of the
wiring of the brain structurally provides some interesting insights. The brain
is highly modular at various levels of hierarchy. Although modularity is
recognized by the AI community, the application of modularity in mainstream
research is limited to problem-specific architecture design or to separate
different chains of processes [260]. We also observe that concepts like
temporal coincidence, which points towards the ability to dynamically group
similar signals without explicit architectural design is completely unexplored
in AI.

  * •

Secondly, AI lacks insights in many neural functions, which study various
functions that arise specifically out of neuronal cell grouping. Object
Permanence, while one of the most fundamental functions of cognitive
perception, is crudely addressed in the field of AI, primarily through
problem-specific data-driven approaches. The neural nature of object
permanence in the human eye highlights that this function is less learned, and
more mechanistic in nature. Similarly, tonotopicity, which indicates towards
the ability to filter specific kinds of signals is not well analysed in
current models in AI.

##### Research gaps in psychology-based AI

Similar to neuroscience, we also observe two major themes in psychology, that
have limited reach in AI research.

  * •

AI can borrow many interesting insights from the cognitive mechanisms
underlying contextual processing, or the function that augments the knowledge
derived from sensory signals by incorporating prior knowledge cues. AI still
lacks efficient methods to utilize goal-oriented feedback connections, which
also extends to the ability to anticipate future signals. Signal denoising is,
again, highly data-centric in mainstream AI, unlike the computationally
inexpensive mechanistic mechanism of the brain. On the contrary, properties
such as signal selectivity (or the property of discarding irrelevant signals)
is a task to be solved by specific training schemas.

  * •

Psychology also has many insights to offer into architectural design for AI
models, such as dynamic sharing and integration of cognitive resources. As
such, progressive modularization and integration of modalities is not well
explored in the field of AI.

##### Research gaps in linguistics-based AI

Finally, we also observe two major themes in linguistics, that are
underexplored in AI, despite the huge interest in the field due to the rise of
generative language models.

  * •

Many known limitations in natural language processing [261] can be solved by
encoding better linguistic structures in models. A dynamic vocabulary
structure, for example, is a challenge to implement in current AI models, as
changing representations of the vocabulary requires one to retrain the entire
model. However, the brain is capable of enhancing the internal vocabulary
representations without affecting, or specifically downgrading, the quality of
subsequent sensory representations in the linguistic pathway of perception.
Similarly, AI still has not explored the benefits of a structure that emulates
the phonological buffer in the brain.

  * •

Language generation semantics is still an open problem in AI, with lots of
interest in the research community. In general, the most sophisticated
language systems still suffer from the lack of creativity [262]. The reason
for the shortcoming, which relates to the underlying design of autoregressive
models currently used, directly extends to many other problems including
hallucinations (or giving false information) [263] and long-range coherence
[264]. Studying the neural activation method in the brain behind the creative
mechanism of the brain is a worthwhile research direction for researchers in
AI.

While each of the above-mentioned topics have a huge potential upside in
research, an overarching characteristic that all research in cognitive AI
(which requires the collaborative effort of both AI scientists and cognitive
scientists from all related sub-domains) should avoid is designing methods and
modeling techniques tailored for specific problems. Such an approach makes
intelligent systems prone to pitfalls that blocks the path of AI towards
general intelligence. In this spirit, a few directions of research should be
explored actively alongside cognitive AI systems that aim to increase model
efficiency and performance.

###  VIII-B Path towards general intelligence: future directions of research

Figure 4: Schema for General Intelligence

The pursuit of general intelligence [265] has been the ultimate motivation of
the field of AI since its inception. Cognitive AI is, however, only one facet
of general intelligence. The purpose of studying cognitive principles and
borrowing inspiration from them into AI is to improve the learning capacity
and efficiency (in energy, performance, comprehension of the environment and
making better logical decisions). A truly intelligent system has many other
aspects that must be given equal importance in the grand set of motivations
for AI research. Overall, general intelligence is constituted of five major
elements – functional knowledge (that may be learned through cognitive
sciences), ability to generalize to a broad spectrum of tasks and data,
embodied cognition, interpretability and social intelligence. These aspects of
general intelligence can be visualized in Figure 4. As such, research efforts
in AI as well as cognitive sciences should focus on these parallelly. While we
discussed cognitive AI efforts in depth, here we describe in brief some future
directions of research for holistically developing general intelligence
through advanced perception.

####  VIII-B1 Generalization

One of the hardest challenges in AI is to generalize performance of AI models
across various domains without having to explicitly train models many tasks
[266]. Generalization is typically achieved to limited extents through the
technique of transfer learning[267]. Many problems remain unsolved in the
domain of transfer learning, including domain gap (where the target domain is
not benefited by the knowledge of the source domain), and negative transfer
[268], which hinders retrieval of knowledge from the target domain. In order
to better generalize to real world knowledge, models need to be regularized
using psychophysical data [269], that helps models achieve human-like
perception and transfer knowledge in a practically useful manner.

####  VIII-B2 Embodied cognition

Embodiment involves teaching AI agents to interact meaningfully with their
surroundings with real-world constraints such as physics and, rather than a
simulated interactions [270]. Embodied AI is still in very primitive stages,
where models can only handle narrow tasks such as visual object navigation
[269], and fail when extended beyond simple constraints. Cognitive science can
especially aid embodied AI enhance perception beyond simple tasks by defining
a structure for embodied principles on various levels of human interaction
with the environment [271].

####  VIII-B3 Interpretability

In general, the learning patterns and mechanism of deep learning models is not
well understood, which leads to the black-box nature of the large parametric
models of today [272]. Understanding the internal representations of AI models
is crucial to gain control over the learning mechanisms of models and their
behavior in different settings. This particular problem is explored in depth
in the domain of explainable AI [273]. The representation of sensory signals
in perception can be better understood by taking into account cognitive
schemas that separate the various components of explainability explicitly in
an intuitive manner [274].

Apart from understanding parametric representations, having control over model
training is equally important in order to guide learning of models in the
right direction. This domain, commonly known as meta-learning [275] is an
active area of research in AI. For general AI, a unified understanding of
meta-learning is required, and should be studied in depth going forward.

####  VIII-B4 Social intelligence

An indispensable factor in human-like cognition is incorporating social
dynamics in AI models, including understanding of ethics, social norms, social
equality, naturalistic communication, and evolution of semantic usage and
meaning over time [276]. As AI models become more accessible and powerful, the
potential to bring harm to society increases as well [277]. This problem
cannot be simply solved by the power of data scaling, and needs to be
addressed through putting down foundations of societal structures carefully in
AI models. These foundations need to emerge at various levels, including at
global, national and communal level [278].

## References

  * [1] Maria Glenski, Ellyn Ayton, Robin Cosbey, Alyssa Pauline Comising, and Svitlana Volkova.  State of the benchmark: Existing efforts across modalities supporting single and multipurpose ai.
  * [2] Yao Liang, Hongjian Fang, Yi Zeng, and Feifei Zhao.  A brain-inspired memory transformation based differentiable neural computer for reasoning-based question answering.  arXiv preprint arXiv:2301.02809, 2023.
  * [3] Fahad Sarfraz, Elahe Arani, and Bahram Zonooz.  Towards brain inspired design for addressing the shortcomings of anns.  arXiv preprint arXiv:2307.00039, 2023.
  * [4] Jing Wang.  Is artificial intelligence capable of understanding? an analysis based on philosophical hermeneutics.  Cultures of Science, 4(3):135–146, 2021.
  * [5] Lin Zhao, Lu Zhang, Zihao Wu, Yuzhong Chen, Haixing Dai, Xiaowei Yu, Zhengliang Liu, Tuo Zhang, Xintao Hu, Xi Jiang, et al.  When brain-inspired ai meets agi.  Meta-Radiology, page 100005, 2023.
  * [6] Piotr Bołtuć.  Consciousness for agi.  Procedia Computer Science, 169:365–372, 2020.
  * [7] Bin Luo, Jin Tang, and Cheng-Lin Liu.  Special issue on recent advances in cognitive learning and data analysis.  Cognitive Computation, pages 1–2, 2022.
  * [8] Arun Kumar Sangaiah and Patrick Siarry.  Cognitive brain-inspired cyber-physical systems in industrial informatics.  Frontiers in Neurorobotics, 16, 2022.
  * [9] Aaisha Makkar and Jong Hyuk Park.  Securecps: Cognitive inspired framework for detection of cyber attacks in cyber–physical systems.  Information Processing & Management, 59(3):102914, 2022.
  * [10] Christelle Langley, Bogdan Ionut Cirstea, Fabio Cuzzolin, and Barbara J Sahakian.  Theory of mind and preference learning at the interface of cognitive science, neuroscience, and ai: A review.  Frontiers in Artificial Intelligence, 5:62, 2022.
  * [11] John Funge, Xiaoyuan Tu, and Demetri Terzopoulos.  Cognitive modeling: Knowledge, reasoning and planning for intelligent characters.  In Proceedings of the 26th annual conference on Computer graphics and interactive techniques, pages 29–38, 1999.
  * [12] Troy Dale Kelley and Christian Lebiere.  From cognitive modeling to robotics: how research on human cognition and computational cognitive architectures can be applied to robotics problems.  In International Conference on Applied Human Factors and Ergonomics, pages 273–279. Springer, 2018.
  * [13] Theophile Gervet, Soumith Chintala, Dhruv Batra, Jitendra Malik, and Devendra Singh Chaplot.  Navigating to objects in the real world.  Science Robotics, 8(79):eadf6991, 2023.
  * [14] Robert Efron.  What is perception?  In Proceedings of the Boston Colloquium for the Philosophy of Science 1966/1968, pages 137–173. Springer, 1969.
  * [15] Yong-hong Tian, Xi-lin Chen, Hong-kai Xiong, Hong-liang Li, Li-rong Dai, Jing Chen, Jun-liang Xing, Jing Chen, Xi-hong Wu, Wei-min Hu, et al.  Towards human-like and transhuman perception in ai 2.0: A review.  Frontiers of Information Technology & Electronic Engineering, 18:58–67, 2017.
  * [16] Eliezer Yudkowsky.  The ai alignment problem: why it is hard, and where to start.  Symbolic Systems Distinguished Speaker, 4, 2016.
  * [17] Teng Long, Qi Gao, Lili Xu, and Zhangbing Zhou.  A survey on adversarial attacks in computer vision: Taxonomy, visualization and future directions.  Computers & Security, page 102847, 2022.
  * [18] Vitaly Feldman and Chiyuan Zhang.  What neural networks memorize and why: Discovering the long tail via influence estimation.  Advances in Neural Information Processing Systems, 33:2881–2891, 2020.
  * [19] Archit Parnami and Minwoo Lee.  Learning from few examples: A summary of approaches to few-shot learning.  arXiv preprint arXiv:2203.04291, 2022.
  * [20] Anny Yuniarti and Nanik Suciati.  A review of deep learning techniques for 3d reconstruction of 2d images.  In 2019 12th International Conference on Information & Communication Technology and System (ICTS), pages 327–331. IEEE, 2019.
  * [21] Michael Teutsch, Angel D Sappa, and Riad I Hammoud.  Computer vision in the infrared spectrum: challenges and approaches.  Springer, 2022.
  * [22] Binglu Wang, Lei Zhang, Zhaozhong Wang, Yongqiang Zhao, and Tianfei Zhou.  Core: Cooperative reconstruction for multi-agent perception.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8710–8720, 2023.
  * [23] Yixin Zhu.  Visual commonsense reasoning: Functionality, physics, causality, and utility.  University of California, Los Angeles, 2018.
  * [24] Karthik Valmeekam, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati.  Large language models still can’t plan (a benchmark for llms on planning and reasoning about change).  arXiv preprint arXiv:2206.10498, 2022.
  * [25] Sourav Mandal and Sudip Kumar Naskar.  Solving arithmetic mathematical word problems: A review and recent advancements.  Information Technology and Applied Mathematics: ICITAM 2017, pages 95–114, 2019.
  * [26] Chen Cecilia Liu, Fajri Koto, Timothy Baldwin, and Iryna Gurevych.  Are multilingual llms culturally-diverse reasoners? an investigation into multicultural proverbs and sayings.  arXiv preprint arXiv:2309.08591, 2023.
  * [27] Paul Thagard.  Cognitive science.  In The Routledge companion to philosophy of science, pages 597–608. Routledge, 2013.
  * [28] Joffrey Becker.  Anthropology, ai and robotics.  The Routledge Social Science Handbook of AI, pages 107–121, 2021.
  * [29] John McCarthy.  What has ai in common with philosophy?  In IJCAI, pages 2041–2044, 1995.
  * [30] Pablo Contreras Kallens, Rick Dale, and Morten H Christiansen.  Quantifying interdisciplinarity in cognitive science and beyond.  Topics in Cognitive Science, 14(3):634–645, 2022.
  * [31] Rick Dale, Eric Dietrich, and Anthony Chemero.  Explanatory pluralism in cognitive science.  Cognitive science, 33(5):739–742, 2009.
  * [32] RA Kycia, A Niemczynowicz, and M Jaworski.  Biological inspiration of artificial neural networks.  Selected aspects of complex, hypercomplex and fuzzy neural networks, page 7, 2023.
  * [33] Bernard J Baars.  Global workspace theory of consciousness: toward a cognitive neuroscience of human experience.  Progress in brain research, 150:45–53, 2005.
  * [34] Marvin Minsky.  The society of mind.  In The Personalist Forum, volume 3, pages 19–32. JSTOR, 1987.
  * [35] Olin Bjork.  Selfridge’s milton: The legacy of pand (a) emonium in cognitive science and artificial intelligence.  Journal of Literature and Science, 11(2):1–19, 2018.
  * [36] AARON SLOMAN.  Cogaff architecture schema.  Presented at Gatsby Computational Neuroscience Unit, 2002.
  * [37] John E Laird.  The Soar cognitive architecture.  MIT press, 2019.
  * [38] Frank E Ritter, Farnaz Tehranchi, and Jacob D Oury.  Act-r: A cognitive architecture for modeling cognition.  Wiley Interdisciplinary Reviews: Cognitive Science, 10(3):e1488, 2019.
  * [39] Ben Goertzel, Shujing Ke, Ruiting Lian, Jade O’Neill, Keyvan Sadeghi, Dingjie Wang, Oliver Watkins, and Gino Yu.  The cogprime architecture for embodied artificial general intelligence.  In 2013 IEEE symposium on computational intelligence for human-like intelligence (CIHLI), pages 60–67. IEEE, 2013.
  * [40] Tamas Madl, Stan Franklin, Ke Chen, Daniela Montaldi, and Robert Trappl.  Towards real-world capable spatial memory in the lida cognitive architecture.  Biologically Inspired Cognitive Architectures, 16:87–104, 2016.
  * [41] Iuliia Kotseruba and John K Tsotsos.  40 years of cognitive architectures: core cognitive abilities and practical applications.  Artificial Intelligence Review, 53(1):17–94, 2020.
  * [42] Ken A Paller.  Electrical signals of memory and of the awareness of remembering.  Current Directions in Psychological Science, 13(2):49–55, 2004.
  * [43] Sudeep Sarkar.  An introduction to perceptual organization.  In IEMC’03 Proceedings. Managing Technologically Driven Organizations: The Human Side of Innovation and Change (IEEE Cat. No. 03CH37502), pages 330–335. IEEE, 2003.
  * [44] Daniel Schacter, Daniel Gilbert, Daniel Wegner, and Bruce M Hood.  Psychology: European Edition.  Macmillan International Higher Education, 2011.
  * [45] E Bruce Goldstein and James Brockmole.  Sensation and perception.  Cengage Learning, 2016.
  * [46] Ophelia Deroy.  Modularity of perception.  2015\.
  * [47] Karsten Rauss and Gilles Pourtois.  What is bottom-up and what is top-down in predictive coding?  Frontiers in psychology, 4:276, 2013.
  * [48] Jan Theeuwes.  Top–down and bottom–up control of visual selection.  Acta psychologica, 135(2):77–99, 2010.
  * [49] Nadine Dijkstra, Peter Zeidman, Sasha Ondobaka, Marcel AJ van Gerven, and K Friston.  Distinct top-down and bottom-up brain connectivity during visual perception and imagery.  Scientific reports, 7(1):1–9, 2017.
  * [50] Konrad P Körding, Ulrik Beierholm, Wei Ji Ma, Steven Quartz, Joshua B Tenenbaum, and Ladan Shams.  Causal inference in multisensory perception.  PLoS one, 2(9):e943, 2007.
  * [51] Melissa Saenz, Wietske Van Der Zwaag, Jose P Marques, Richard S Frackowiak, Stephanie Clarke, and Sandra E Da Costa.  Striking parallel between tonotopy in auditory cortex and retinotopy in visual cortex: A human fmri study at 7 tesla.  Journal of Vision, 11(11):778–778, 2011.
  * [52] Sakib Mostafa, Debajyoti Mondal, Michael Beck, Christopher Bidinosti, Christopher Henry, and Ian Stavness.  Visualizing feature maps for model selection in convolutional neural networks.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1362–1371, 2021.
  * [53] Richard T Born, Alexander R Trott, and Till S Hartmann.  Cortical magnification plus cortical plasticity equals vision?  Vision research, 111:161–169, 2015.
  * [54] Danny da Costa, Rainer Goebel, and Mario Senden.  Convnets develop organizational principles of the visual cortex when using ganglion cell-based sampling.  bioRxiv, pages 2021–11, 2021.
  * [55] Ylva Jansson and Tony Lindeberg.  Scale-invariant scale-channel networks: Deep networks that generalise to previously unseen scales.  Journal of Mathematical Imaging and Vision, 64(5):506–536, 2022.
  * [56] Binxu Wang, David Mayo, Arturo Deza, Andrei Barbu, and Colin Conwell.  On the use of cortical magnification and saccades as biological proxies for data augmentation.  arXiv preprint arXiv:2112.07173, 2021.
  * [57] Colin Humphries, Einat Liebenthal, and Jeffrey R Binder.  Tonotopic organization of human auditory cortex.  Neuroimage, 50(3):1202–1211, 2010.
  * [58] Alexander JE Kell, Daniel LK Yamins, Erica N Shook, Sam V Norman-Haignere, and Josh H McDermott.  A task-optimized neural network replicates human auditory behavior, predicts brain responses, and reveals a cortical processing hierarchy.  Neuron, 98(3):630–644, 2018.
  * [59] Edwin H Land.  The retinex theory of color vision.  Scientific american, 237(6):108–129, 1977.
  * [60] Brian Funt and Lilong Shi.  The rehabilitation of maxrgb.  In Color and imaging conference, volume 2010, pages 256–259. Society for Imaging Science and Technology, 2010.
  * [61] Joost Van De Weijer and Theo Gevers.  Color constancy based on the grey-edge hypothesis.  In IEEE International Conference on Image Processing 2005, volume 2, pages II–722. IEEE, 2005.
  * [62] Joost Van De Weijer, Theo Gevers, and Arjan Gijsenij.  Edge-based color constancy.  IEEE Transactions on image processing, 16(9):2207–2214, 2007.
  * [63] Nikola Banić, Karlo Koščević, and Sven Lončarić.  Unsupervised learning for color constancy.  arXiv preprint arXiv:1712.00436, 2017.
  * [64] Kobus Barnard.  Improvements to gamut mapping colour constancy algorithms.  In European conference on computer vision, pages 390–403. Springer, 2000.
  * [65] Graham D Finlayson, Steven D Hordley, and Ingeborg Tastl.  Gamut constrained illuminant estimation.  International journal of computer vision, 67(1):93–109, 2006.
  * [66] Joost Van De Weijer, Cordelia Schmid, and Jakob Verbeek.  Using high-level visual information for color constancy.  In 2007 IEEE 11th International Conference on Computer Vision, pages 1–8. IEEE, 2007.
  * [67] Arjan Gijsenij and Theo Gevers.  Color constancy using natural image statistics and scene semantics.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(4):687–698, 2010.
  * [68] Graham D Finlayson.  Corrected-moment illuminant estimation.  In Proceedings of the IEEE International Conference on Computer Vision, pages 1904–1911, 2013.
  * [69] Peter Vincent Gehler, Carsten Rother, Andrew Blake, Tom Minka, and Toby Sharp.  Bayesian color constancy revisited.  In 2008 IEEE Conference on Computer Vision and Pattern Recognition, pages 1–8. IEEE, 2008.
  * [70] Ayan Chakrabarti, Keigo Hirakawa, and Todd Zickler.  Color constancy with spatio-spectral statistics.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(8):1509–1519, 2011.
  * [71] Nikola Banić and Sven Lončarić.  Color rabbit: Guiding the distance of local maximums in illumination estimation.  In 2014 19th International Conference on Digital Signal Processing, pages 345–350. IEEE, 2014.
  * [72] Vivek Agarwal, Andrei V Gribok, and Mongi A Abidi.  Machine learning approach to color constancy.  Neural Networks, 20(5):559–563, 2007.
  * [73] Weihua Xiong and Brian Funt.  Estimating illumination chromaticity via support vector regression.  Journal of Imaging Science and Technology, 50(4):341–348, 2006.
  * [74] Dongliang Cheng, Brian Price, Scott Cohen, and Michael S Brown.  Effective learning-based illuminant estimation using simple features.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1000–1008, 2015.
  * [75] Jueqin Qiu, Haisong Xu, and Zhengnan Ye.  Color constancy by reweighting image feature maps.  IEEE Transactions on Image Processing, 29:5711–5721, 2020.
  * [76] Oleksii Sidorov.  Artificial color constancy via googlenet with angular loss function.  Applied Artificial Intelligence, 34(9):643–655, 2020.
  * [77] Jonathan T. Barron.  Convolutional color constancy, 2015.
  * [78] Kai-Fu Yang, Cheng Cheng, Shi-Xuan Zhao, Hong-Mei Yan, Xian-Shi Zhang, and Yong-Jie Li.  Learning to adapt to light.  International Journal of Computer Vision, pages 1–20, 2023.
  * [79] Yichong Xu, Tianjun Xiao, Jiaxing Zhang, Kuiyuan Yang, and Zheng Zhang.  Scale-invariant convolutional neural networks.  arXiv preprint arXiv:1411.6369, 2014.
  * [80] Jonathan Cohen.  Perceptual constancy.  The Oxford handbook of philosophy of perception, pages 621–639, 2015.
  * [81] Qidong Huang, Xiaoyi Dong, Dongdong Chen, Hang Zhou, Weiming Zhang, and Nenghai Yu.  Shape-invariant 3d adversarial point clouds.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15335–15344, 2022.
  * [82] Praful Agrawal, Ross T Whitaker, and Shireen Y Elhabian.  Learning deep features for shape correspondence with domain invariance.  arXiv preprint arXiv:2102.10493, 2021.
  * [83] Jeffrey Gu and Serena Yeung.  Staying in shape: learning invariant shape representations using contrastive learning.  In Uncertainty in Artificial Intelligence, pages 1852–1862. PMLR, 2021.
  * [84] Pierre Bonzon.  Piaget revisited: object permanence without mental representations.
  * [85] Aviv Shamsian, Ofri Kleinfeld, Amir Globerson, and Gal Chechik.  Learning object permanence from video.  In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVI 16, pages 35–50. Springer, 2020.
  * [86] Pavel Tokmakov, Jie Li, Wolfram Burgard, and Adrien Gaidon.  Learning to track with object permanence.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10860–10869, 2021.
  * [87] Pavel Tokmakov, Allan Jabri, Jie Li, and Adrien Gaidon.  Object permanence emerges in a random walk along memory.  arXiv preprint arXiv:2204.01784, 2022.
  * [88] Don McNicol.  A primer of signal detection theory.  Psychology Press, 2005.
  * [89] Ademola E Ilesanmi and Taiwo O Ilesanmi.  Methods for image denoising using convolutional neural network: a review.  Complex & Intelligent Systems, 7(5):2179–2198, 2021.
  * [90] Madhav Mahesh Kashyap, Anuj Tambwekar, Krishnamoorthy Manohara, and S Natarajan.  Speech denoising without clean training data: A noise2noise approach.  arXiv preprint arXiv:2104.03838, 2021.
  * [91] Stephan Rabanser, Anvith Thudi, Kimia Hamidieh, Adam Dziedzic, and Nicolas Papernot.  Selective classification via neural network training dynamics, 2022.
  * [92] Sabrina J Mielke, Zaid Alyafeai, Elizabeth Salesky, Colin Raffel, Manan Dey, Matthias Gallé, Arun Raja, Chenglei Si, Wilson Y Lee, Benoît Sagot, et al.  Between words and characters: A brief history of open-vocabulary modeling and tokenization in nlp.  arXiv preprint arXiv:2112.10508, 2021.
  * [93] Abigail Rai and Samarjeet Borah.  Study of various methods for tokenization.  In Applications of Internet of Things: Proceedings of ICCCIOT 2020, pages 193–200. Springer, 2021.
  * [94] Lukas Michelbacher.  Multi-word tokenization for natural language processing.  2013\.
  * [95] Ruotian Ma, Yiding Tan, Xin Zhou, Xuanting Chen, Di Liang, Sirui Wang, Wei Wu, Tao Gui, and Qi Zhang.  Searching for optimal subword tokenization in cross-domain ner.  arXiv preprint arXiv:2206.03352, 2022.
  * [96] Xinying Song, Alex Salcianu, Yang Song, Dave Dopson, and Denny Zhou.  Fast wordpiece tokenization.  arXiv preprint arXiv:2012.15524, 2020.
  * [97] Stevan Harnad.  Categorical perception.  2003\.
  * [98] José A Mompeán.  The phoneme as a basic-level category: Experimental evidence from english.  International Journal of English Studies, 6(2), 2006.
  * [99] Yi Tay, Vinh Q Tran, Sebastian Ruder, Jai Gupta, Hyung Won Chung, Dara Bahri, Zhen Qin, Simon Baumgartner, Cong Yu, and Donald Metzler.  Charformer: Fast character transformers via gradient-based subword tokenization.  arXiv preprint arXiv:2106.12672, 2021.
  * [100] Edoardo M Ponti, Ivan Vulić, Ryan Cotterell, Marinela Parovic, Roi Reichart, and Anna Korhonen.  Parameter space factorization for zero-shot learning across tasks and languages.  Transactions of the Association for Computational Linguistics, 9:410–428, 2021.
  * [101] Mike P Hollier, Andrew N Rimell, David S Hands, and Rupert M Voelcker.  Multi-modal perception.  BT Technology Journal, 17(1):35–46, 1999.
  * [102] Zhengyan Zhang, Zhiyuan Zeng, Yankai Lin, Chaojun Xiao, Xiaozhi Wang, Xu Han, Zhiyuan Liu, Ruobing Xie, Maosong Sun, and Jie Zhou.  Emergent modularity in pre-trained transformers.  arXiv preprint arXiv:2305.18390, 2023.
  * [103] P Read Montague and Terrence J Sejnowski.  The predictive brain: temporal coincidence and temporal order in synaptic learning mechanisms.  Learning & Memory, 1(1):1–33, 1994.
  * [104] Vincent Bergeron.  Cognitive architecture and the brain: Beyond domain-specific functional specification.  PhD thesis, University of British Columbia, 2008.
  * [105] Palash Goyal, Saurabh Sahu, Shalini Ghosh, and Chul Lee.  Cross-modal learning for multi-modal video categorization.  arXiv preprint arXiv:2003.03501, 2020.
  * [106] Kaiye Wang, Qiyue Yin, Wei Wang, Shu Wu, and Liang Wang.  A comprehensive survey on cross-modal retrieval.  arXiv preprint arXiv:1607.06215, 2016.
  * [107] Khaled Bayoudh, Raja Knani, Fayçal Hamdaoui, and Abdellatif Mtibaa.  A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets.  The Visual Computer, pages 1–32, 2021.
  * [108] Pradeep K Atrey, M Anwar Hossain, Abdulmotaleb El Saddik, and Mohan S Kankanhalli.  Multimodal fusion for multimedia analysis: a survey.  Multimedia systems, 16(6):345–379, 2010.
  * [109] Arsha Nagrani, Shan Yang, Anurag Arnab, Aren Jansen, Cordelia Schmid, and Chen Sun.  Attention bottlenecks for multimodal fusion.  Advances in Neural Information Processing Systems, 34, 2021.
  * [110] Fan Qian and Jiqing Han.  Multimodal sentiment analysis with temporal modality attention.  Proc. Interspeech 2021, pages 3385–3389, 2021.
  * [111] Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean.  Outrageously large neural networks: The sparsely-gated mixture-of-experts layer.  arXiv preprint arXiv:1701.06538, 2017.
  * [112] Luis H Favela.  Dynamical systems theory in cognitive science and neuroscience.  Philosophy Compass, 15(8):e12695, 2020.
  * [113] Zihui Xue and Radu Marculescu.  Dynamic multimodal fusion.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2574–2583, 2023.
  * [114] Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel.  Language models as knowledge bases?  arXiv preprint arXiv:1909.01066, 2019.
  * [115] Badr AlKhamissi, Millicent Li, Asli Celikyilmaz, Mona Diab, and Marjan Ghazvininejad.  A review on language models as knowledge bases.  arXiv preprint arXiv:2204.06031, 2022.
  * [116] Michael T Ullman.  The biocognition of the mental lexicon.  The Oxford handbook of psycholinguistics, pages 267–286, 2007.
  * [117] Majid Farahian.  Mental lexicon: its organization and representation.  BRAIN. Broad Research in Artificial Intelligence and Neuroscience, 2(4):56–59, 2011.
  * [118] Grégoire Mialon, Roberto Dessì, Maria Lomeli, Christoforos Nalmpantis, Ram Pasunuru, Roberta Raileanu, Baptiste Rozière, Timo Schick, Jane Dwivedi-Yu, Asli Celikyilmaz, et al.  Augmented language models: a survey.  arXiv preprint arXiv:2302.07842, 2023.
  * [119] Qi Liu, Dani Yogatama, and Phil Blunsom.  Relational memory-augmented language models.  Transactions of the Association for Computational Linguistics, 10:555–572, 2022.
  * [120] David Wilmot and Frank Keller.  Memory and knowledge augmented language models for inferring salience in long-form stories.  arXiv preprint arXiv:2109.03754, 2021.
  * [121] Rüdiger Krahe, Gabriel Kreiman, Fabrizio Gabbiani, Christof Koch, and Walter Metzner.  Stimulus encoding and feature extraction by multiple sensory neurons.  Journal of Neuroscience, 22(6):2374–2382, 2002.
  * [122] Hanjie Chen and Yangfeng Ji.  Adversarial training for improving model robustness? look at both prediction and interpretation.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 10463–10472, 2022.
  * [123] Michael Beyeler, Emily L Rounds, Kristofor D Carlson, Nikil Dutt, and Jeffrey L Krichmar.  Neural correlates of sparse coding and dimensionality reduction.  PLoS computational biology, 15(6):e1006908, 2019.
  * [124] Samuel Wilson, Tobias Fischer, Feras Dayoub, Dimity Miller, and Niko Sünderhauf.  Safe: Sensitivity-aware features for out-of-distribution object detection, 2023.
  * [125] Denis G Pelli, Catherine W Burns, Bart Farell, and Deborah C Moore-Page.  Feature detection and letter identification.  Vision research, 46(28):4646–4674, 2006.
  * [126] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton.  Deep learning.  nature, 521(7553):436–444, 2015.
  * [127] Irving Biederman.  Geon theory as an account of shape recognition in mind and brain.  The Irish Journal of Psychology, 14(3):314–327, 1993.
  * [128] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas.  Pointnet: Deep learning on point sets for 3d classification and segmentation.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652–660, 2017.
  * [129] Haoxi Ran, Jun Liu, and Chengjie Wang.  Surface representation for point clouds.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18942–18952, 2022.
  * [130] Max Wertheimer.  Gestalt theory.  1938\.
  * [131] G Kanizsa and Walter Gerbino.  Il completamento amodale tra vedere e pensare.  1981\.
  * [132] Agne Desolneux, Lionel Moisan, and Jean-Michel Morel.  Gestalt theory and computer vision.  In Seeing, Thinking and Knowing, pages 71–101. Springer, 2004.
  * [133] Angelos Amanatiadis, Vasileios G Kaburlasos, and Elias B Kosmatopoulos.  Understanding deep convolutional networks through gestalt theory.  In 2018 IEEE International Conference on Imaging Systems and Techniques (IST), pages 1–6. IEEE, 2018.
  * [134] Markus Hoerhan and Horst Eidenberger.  Gestalt descriptions for deep image understanding.  Pattern Analysis and Applications, 24(1):89–107, 2021.
  * [135] Sungho Kim, Kuk-Jin Yoon, and In So Kweon.  Object recognition using a generalized robust invariant feature and gestalt’s law of proximity and similarity.  Pattern Recognition, 41(2):726–741, 2008.
  * [136] Nicholas Baker, Gennady Erlikhman, Philip J Kellman, and Hongjing Lu.  Deep convolutional networks do not perceive illusory contours.  In CogSci, 2018.
  * [137] Christina M Funke, Judy Borowski, Karolina Stosio, Wieland Brendel, Thomas SA Wallis, and Matthias Bethge.  Five points to check when comparing visual perception in humans and machines.  Journal of Vision, 21(3):16–16, 2021.
  * [138] Been Kim, Emily Reif, Martin Wattenberg, Samy Bengio, and Michael C Mozer.  Neural networks trained on natural scenes exhibit gestalt closure.  Computational Brain & Behavior, pages 1–13, 2021.
  * [139] Antonio L Freitas, Peter Gollwitzer, and Yaacov Trope.  The influence of abstract and concrete mindsets on anticipating and guiding others’ self-regulatory efforts.  Journal of experimental social psychology, 40(6):739–752, 2004.
  * [140] Robert P Spunt, David Kemmerer, and Ralph Adolphs.  The neural basis of conceptualizing the same action at different levels of abstraction.  Social cognitive and affective neuroscience, 11(7):1141–1151, 2016.
  * [141] Xinghua Zhang, Bowen Yu, Haiyang Yu, Yangyu Lv, Tingwen Liu, Fei Huang, Hongbo Xu, and Yongbin Li.  Wider and deeper llm networks are fairer llm evaluators.  arXiv preprint arXiv:2308.01862, 2023.
  * [142] Giorgio Franceschelli and Mirco Musolesi.  Creativity and machine learning: A survey.  arXiv preprint arXiv:2104.02726, 2021.
  * [143] Michael X Cohen and Charan Ranganath.  Reinforcement learning signals predict future decisions.  Journal of Neuroscience, 27(2):371–378, 2007.
  * [144] Ned Herrmann.  The creative brain.  Journal of creative behavior, 25(4):275–95, 1991.
  * [145] Lukas Neumann, Andrew Zisserman, and Andrea Vedaldi.  Future event prediction: If and when.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 0–0, 2019.
  * [146] Allan M Collins and Elizabeth F Loftus.  A spreading-activation theory of semantic processing.  Psychological review, 82(6):407, 1975.
  * [147] Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, and Mubarak Shah.  Diffusion models in vision: A survey.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
  * [148] Ziqiang Li, Beihao Xia, Jing Zhang, Chaoyue Wang, and Bin Li.  A comprehensive survey on data-efficient gans in image generation.  arXiv preprint arXiv:2204.08329, 2022.
  * [149] Ramon Guevara Erra, Jose L Perez Velazquez, and Michael Rosenblum.  Neural synchronization from the perspective of non-linear dynamics.  Frontiers in computational neuroscience, 11:98, 2017.
  * [150] Mary B McVee, Kailonnie Dunsmore, and James R Gavelek.  Schema theory revisited.  Review of educational research, 75(4):531–566, 2005.
  * [151] Daniel T Chang.  Concept representation learning with contrastive self-supervised learning.  arXiv preprint arXiv:2112.05677, 2021.
  * [152] Lyn Frazier and Keith Rayner.  Making and correcting errors during sentence comprehension: Eye movements in the analysis of structurally ambiguous sentences.  Cognitive psychology, 14(2):178–210, 1982.
  * [153] William Jurayj, William Rudman, and Carsten Eickhoff.  Garden-path traversal within gpt-2.  arXiv preprint arXiv:2205.12302, 2022.
  * [154] Tovah Irwin, Kyra Wilson, and Alec Marantz.  Bert shows garden path effects.  In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 3212–3224, 2023.
  * [155] Cristina Garbacea and Qiaozhu Mei.  Why is constrained neural language generation particularly challenging?  arXiv preprint arXiv:2206.05395, 2022.
  * [156] Forrest Davis and Marten van Schijndel.  Uncovering constraint-based behavior in neural models via targeted fine-tuning.  arXiv preprint arXiv:2106.01207, 2021.
  * [157] Sharon L Thompson-Schill, Mark D’Esposito, and Irene P Kan.  Effects of repetition and competition on activity in left prefrontal cortex during word generation.  Neuron, 23(3):513–522, 1999.
  * [158] Fabian A Mikulasch, Lucas Rudelt, Michael Wibral, and Viola Priesemann.  Where is the error? hierarchical predictive coding through dendritic error computation.  Trends in Neurosciences, 46(1):45–59, 2023.
  * [159] Stewart Shipp.  Neural elements for predictive coding.  Frontiers in psychology, 7:1792, 2016.
  * [160] Charlotte Caucheteux, Alexandre Gramfort, and Jean-Rémi King.  Evidence of a predictive coding hierarchy in the human brain listening to speech.  Nature human behaviour, 7(3):430–441, 2023.
  * [161] Hojjat Salehinejad, Sharan Sankar, Joseph Barfett, Errol Colak, and Shahrokh Valaee.  Recent advances in recurrent neural networks.  arXiv preprint arXiv:1801.01078, 2017.
  * [162] Alex Graves, Santiago Fernández, and Jürgen Schmidhuber.  Bidirectional lstm networks for improved phoneme classification and recognition.  In Artificial Neural Networks: Formal Models and Their Applications–ICANN 2005: 15th International Conference, Warsaw, Poland, September 11-15, 2005. Proceedings, Part II 15, pages 799–804. Springer, 2005.
  * [163] Blake A Richards, Timothy P Lillicrap, Philippe Beaudoin, Yoshua Bengio, Rafal Bogacz, Amelia Christensen, Claudia Clopath, Rui Ponte Costa, Archy de Berker, Surya Ganguli, et al.  A deep learning framework for neuroscience.  Nature neuroscience, 22(11):1761–1770, 2019.
  * [164] Hanneke EM Den Ouden, Peter Kok, and Floris P De Lange.  How prediction errors shape perception, attention, and motivation.  Frontiers in psychology, 3:548, 2012.
  * [165] Samanwoy Ghosh-Dastidar and Hojjat Adeli.  Spiking neural networks.  International journal of neural systems, 19(04):295–308, 2009.
  * [166] Karl Friston, Jérémie Mattout, Nelson Trujillo-Barreto, John Ashburner, and Will Penny.  Variational free energy and the laplace approximation.  Neuroimage, 34(1):220–234, 2007.
  * [167] Jingkang Yang, Kaiyang Zhou, Yixuan Li, and Ziwei Liu.  Generalized out-of-distribution detection: A survey.  arXiv preprint arXiv:2110.11334, 2021.
  * [168] Jiang Lu, Pinghua Gong, Jieping Ye, and Changshui Zhang.  Learning from very few samples: A survey.  arXiv preprint arXiv:2009.02653, 2020.
  * [169] Beren Millidge, Alexander Tschantz, and Christopher L Buckley.  Whence the expected free energy?  Neural Computation, 33(2):447–482, 2021.
  * [170] Ryoya Ogishima, Izumi Karino, and Yasuo Kuniyoshi.  Combining imitation and reinforcement learning with free energy principle.  2020\.
  * [171] Bertram Gawronski and Laura A Creighton.  Dual process theories.  2013\.
  * [172] Olivier Raiman.  Order of magnitude reasoning.  In Readings in qualitative reasoning about physical systems, pages 318–322. Elsevier, 1990.
  * [173] Michael L Mavrovouniotis and George Stephanopoulos.  Reasoning with orders of magnitude and approximate relations.  In AAAI, volume 87, pages 626–630, 1987.
  * [174] Kenneth Man-kam Yip.  Model simplification by asymptotic order of magnitude reasoning.  Artificial Intelligence, 80(2):309–348, 1996.
  * [175] Didier Dubois and Henri Prade.  Order-of-magnitude reasoning with fuzzy relations.  In Advanced Information Processing in Automatic Control (AIPAC’89), pages 195–200. Elsevier, 1990.
  * [176] Johan De Kleer and Daniel G Bobrow.  Qualitative reasoning with higher-order derivatives.  In AAAI, pages 86–91, 1984.
  * [177] Ladan Shams and Ulrik Beierholm.  Bayesian causal inference: A unifying neuroscience theory.  Neuroscience & Biobehavioral Reviews, 137:104619, 2022.
  * [178] Laurence Aitchison and Máté Lengyel.  With or without you: predictive coding and bayesian inference in the brain.  Current opinion in neurobiology, 46:219–227, 2017.
  * [179] Hao Wang and Dit-Yan Yeung.  A survey on bayesian deep learning.  ACM computing surveys (csur), 53(5):1–37, 2020.
  * [180] Andrew Gordon Wilson.  The case for bayesian deep learning.  arXiv preprint arXiv:2001.10995, 2020.
  * [181] Hans Kamp, Josef Van Genabith, and Uwe Reyle.  Discourse representation theory.  Handbook of Philosophical Logic: Volume 15, pages 125–394, 2011.
  * [182] Ian Tenney, Dipanjan Das, and Ellie Pavlick.  Bert rediscovers the classical nlp pipeline.  arXiv preprint arXiv:1905.05950, 2019.
  * [183] Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D Manning.  What does bert look at? an analysis of bert’s attention.  arXiv preprint arXiv:1906.04341, 2019.
  * [184] Christophe Micheyl, Paul R Schrater, and Andrew J Oxenham.  Auditory frequency and intensity discrimination explained using a cortical population rate code.  PLoS computational biology, 9(11):e1003336, 2013.
  * [185] Andrew Francl and Josh H McDermott.  Deep neural network models of sound localization reveal how perception is adapted to real-world environments.  Nature human behaviour, 6(1):111–133, 2022.
  * [186] Valerie Nunez, Robert M Shapley, and James Gordon.  Cortical double-opponent cells in color perception: perceptual scaling and chromatic visual evoked potentials.  i-Perception, 9(1):2041669517752715, 2018.
  * [187] Ala Morshedian and Gordon L Fain.  Light adaptation and the evolution of vertebrate photoreceptors.  The Journal of physiology, 595(14):4947–4960, 2017.
  * [188] Gary Hatfield.  Psychological experiments and phenomenal experience in size and shape constancy.  Philosophy of Science, 81(5):940–953, 2014.
  * [189] Peter Wenderoth and Darren Burke.  Testing the tilt-constancy theory of visual illusions.  Perception, 35(2):201–213, 2006.
  * [190] Abigail A Baird, Jerome Kagan, Thomas Gaudette, Kathryn A Walz, Natalie Hershlag, and David A Boas.  Frontal lobe activation during object permanence: Data from near-infrared spectroscopy.  NeuroImage, 16(4):1120–1126, 2002.
  * [191] Connor Shorten and Taghi M Khoshgoftaar.  A survey on image data augmentation for deep learning.  Journal of big data, 6(1):1–48, 2019.
  * [192] Mahmoud Afifi and Michael S Brown.  What else can fool deep learning? addressing color constancy errors on deep neural network performance.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 243–252, 2019.
  * [193] Nicholas Baker, Hongjing Lu, Gennady Erlikhman, and Philip J Kellman.  Deep convolutional networks do not classify based on global object shape.  PLoS computational biology, 14(12):e1006613, 2018.
  * [194] Dobromir Rahnev, Hakwan Lau, and Floris P De Lange.  Prior expectation modulates the interaction between sensory and prefrontal regions in the human brain.  Journal of Neuroscience, 31(29):10741–10748, 2011.
  * [195] Tien-Nam Nguyen, Jean-Christophe Burie, Thi-Lan Le, and Anne-Valerie Schweyer.  On the use of attention in deep learning based denoising method for ancient cham inscription images.  In Document Analysis and Recognition–ICDAR 2021: 16th International Conference, Lausanne, Switzerland, September 5–10, 2021, Proceedings, Part I, pages 400–415. Springer, 2021.
  * [196] Chunwei Tian, Yong Xu, Zuoyong Li, Wangmeng Zuo, Lunke Fei, and Hong Liu.  Attention-guided cnn for image denoising.  Neural Networks, 124:117–129, 2020.
  * [197] Jerry Mannil, Mohammad-Mahdi Bidmeshki, and Roozbeh Jafari.  Rejection of irrelevant human actions in real-time hidden markov model based recognition systems for wearable computers.  In Proceedings of the 2nd Conference on Wireless Health, WH ’11, New York, NY, USA, 2011. Association for Computing Machinery.
  * [198] Nima Mesgarani, Stephen V David, Jonathan B Fritz, and Shihab A Shamma.  Phoneme representation and classification in primary auditory cortex.  The Journal of the Acoustical Society of America, 123(2):899–909, 2008.
  * [199] RJJH Van Son, Louis CW Pols, et al.  How efficient is speech.  In Proceedings of the institute of phonetic sciences, volume 25, pages 171–184. University of Amsterdam, 2003.
  * [200] Alan D Baddeley and Robert H Logie.  Working memory: The multiple-component model.  1999\.
  * [201] Alfonso Caramazza, Gabriele Miceli, and Gianpiero Villa.  The role of the (output) phonological buffer in reading, writing, and repetition.  Cognitive Neuropsychology, 3(1):37–76, 1986.
  * [202] Ineke Imbo and André Vandierendonck.  The role of phonological and executive working memory resources in simple arithmetic strategies.  European Journal of Cognitive Psychology, 19(6):910–933, 2007.
  * [203] Yao Ming, Shaozu Cao, Ruixiang Zhang, Zhen Li, Yuanzhe Chen, Yangqiu Song, and Huamin Qu.  Understanding hidden memories of recurrent neural networks.  In 2017 IEEE conference on visual analytics science and technology (VAST), pages 13–24. IEEE, 2017.
  * [204] Kyungmin Kim, Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Zhicheng Yan, Peter Vajda, and Seon Joo Kim.  Rethinking the self-attention in vision transformers.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3071–3075, 2021.
  * [205] William Fedus, Barret Zoph, and Noam Shazeer.  Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.  The Journal of Machine Learning Research, 23(1):5232–5270, 2022.
  * [206] Dora E Angelaki, Yong Gu, and Gregory C DeAngelis.  Multisensory integration: psychophysics, neurophysiology, and computation.  Current opinion in neurobiology, 19(4):452–458, 2009.
  * [207] Jing Gao, Peng Li, Zhikui Chen, and Jianing Zhang.  A survey on deep learning for multimodal data fusion.  Neural Computation, 32(5):829–864, 2020.
  * [208] Jerry A Fodor.  The modularity of mind.  MIT press, 1983.
  * [209] Bernard J Baars.  Understanding subjectivity: Global workspace theory and the resurrection of the observing self.  Journal of Consciousness Studies, 3(3):211–216, 1996.
  * [210] Lawrence W Barsalou.  Grounded cognition.  Annu. Rev. Psychol., 59:617–645, 2008.
  * [211] Robyn Downie, Dan Milech, and Kim Kirsner.  Unit definition in the mental lexicon.  Australian Journal of Psychology, 37(2):141–155, 1985.
  * [212] Gary Libben.  From lexicon to flexicon: The principles of morphological transcendence and lexical superstates in the characterization of words in the mind.  Frontiers in Artificial Intelligence, 4:788430, 2022.
  * [213] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei.  Scaling laws for neural language models.  arXiv preprint arXiv:2001.08361, 2020.
  * [214] Angeliki Lazaridou, Adhi Kuncoro, Elena Gribovskaya, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d’Autume, Tomas Kocisky, Sebastian Ruder, et al.  Mind the gap: Assessing temporal generalization in neural language models.  Advances in Neural Information Processing Systems, 34:29348–29363, 2021.
  * [215] Edmund T Rolls and Alessandro Treves.  The neuronal encoding of information in the brain.  Progress in neurobiology, 95(3):448–490, 2011.
  * [216] Chongzhi Zhang, Aishan Liu, Xianglong Liu, Yitao Xu, Hang Yu, Yuqing Ma, and Tianlin Li.  Interpreting and improving adversarial robustness of deep neural networks with neuron sensitivity.  IEEE Transactions on Image Processing, 30:1291–1304, 2020.
  * [217] Seungyeon Kim, Daniel Glasner, Srikumar Ramalingam, Cho-Jui Hsieh, Kishore Papineni, and Sanjiv Kumar.  Balancing robustness and sensitivity using feature contrastive learning.  arXiv preprint arXiv:2105.09394, 2021.
  * [218] Wang Shugen.  Framework of pattern recognition model based on the cognitive psychology.  Geo-spatial Information Science, 5:74–78, 2002.
  * [219] Jun Cheng, Lei Wang, Jiaji Wu, Xiping Hu, Gwanggil Jeon, Dacheng Tao, and Mengchu Zhou.  Visual relationship detection: A survey.  IEEE Transactions on Cybernetics, 52(8):8453–8466, 2022.
  * [220] Benjamin B Lahey.  Psychology: an introduction.  Wm C Brown Publishers, 1992.
  * [221] Stephen E Palmer.  Organizing objects and scenes.  Foundations of cognitive psychology: Core readings, pages 189–211, 2002.
  * [222] William Prinzmetal, Deborah Henderson, and Richard Ivry.  Loosening the constraints on illusory conjunctions: Assessing the roles of exposure duration and attention.  Journal of Experimental Psychology: Human Perception and Performance, 21(6):1362, 1995.
  * [223] Yijun Yan, Jinchang Ren, Genyun Sun, Huimin Zhao, Junwei Han, Xuelong Li, Stephen Marshall, and Jin Zhan.  Unsupervised image saliency detection with gestalt-laws guided optimization and visual attention based refinement.  Pattern Recognition, 79:65–78, 2018.
  * [224] John TE Richardson.  Concreteness and imageability.  The Quarterly Journal of Experimental Psychology, 27(2):235–249, 1975.
  * [225] Matthew D Zeiler and Rob Fergus.  Visualizing and understanding convolutional networks.  In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13, pages 818–833. Springer, 2014.
  * [226] Negar Foroutan, Mohammadreza Banaei, Rémi Lebret, Antoine Bosselut, and Karl Aberer.  Discovering language-neutral sub-networks in multilingual language models.  arXiv preprint arXiv:2205.12672, 2022.
  * [227] Roger E Beaty, Mathias Benedek, Robin W Wilkins, Emanuel Jauk, Andreas Fink, Paul J Silvia, Donald A Hodges, Karl Koschutnig, and Aljoscha C Neubauer.  Creativity and the default network: A functional connectivity analysis of the creative brain at rest.  Neuropsychologia, 64:92–98, 2014.
  * [228] Emery Schubert.  Creativity is optimal novelty and maximal positive affect: A new definition based on the spreading activation model.  Frontiers in Neuroscience, 15:612379, 2021.
  * [229] Victoria Rousay.  Top-down processing examples in psychology.  Victoria, 2023.
  * [230] Michael Tamir and Elay Shech.  Machine understanding and deep learning representation.  Synthese, 201(2):51, 2023.
  * [231] Kacper Sokol and Peter Flach.  Interpretable representations in explainable ai: From theory to practice.  arXiv preprint arXiv:2008.07007, 2020.
  * [232] Joaquín Borrego-Díaz and Juan Galán Páez.  Knowledge representation for explainable artificial intelligence: modeling foundations from complex systems.  Complex & Intelligent Systems, 8(2):1579–1601, 2022.
  * [233] Helen E Moss, S Abdallah, Paul Fletcher, Peter Bright, L Pilgrim, K Acres, and Lorraine K Tyler.  Selecting among competing alternatives: selection and retrieval in the left inferior frontal gyrus.  Cerebral Cortex, 15(11):1723–1735, 2005.
  * [234] Joshua K Hartshorne, Yasutada Sudo, and Miki Uruwashi.  Are implicit causality pronoun resolution biases consistent across languages and cultures?  Experimental psychology, 2013.
  * [235] Peter Kok and Floris P de Lange.  Predictive coding in sensory cortex.  An introduction to model-based cognitive neuroscience, pages 221–244, 2015.
  * [236] Timothy P Lillicrap, Adam Santoro, Luke Marris, Colin J Akerman, and Geoffrey Hinton.  Backpropagation and the brain.  Nature Reviews Neuroscience, 21(6):335–346, 2020.
  * [237] Robert Rosenbaum.  On the relationship between predictive coding and backpropagation.  Plos one, 17(3):e0266102, 2022.
  * [238] Beren Millidge, Tommaso Salvatori, Yuhang Song, Rafal Bogacz, and Thomas Lukasiewicz.  Predictive coding: towards a future of deep learning beyond backpropagation?  arXiv preprint arXiv:2202.09467, 2022.
  * [239] Yasuo Kawaguchi.  Pyramidal cell subtypes and their synaptic connections in layer 5 of rat frontal cortex.  Cerebral Cortex, 27(12):5755–5771, 2017.
  * [240] Karl Friston.  The free-energy principle: a unified brain theory?  Nature reviews neuroscience, 11(2):127–138, 2010.
  * [241] Jangho Lee, Jeonghee Jo, Byounghwa Lee, Jung-Hoon Lee, and Sungroh Yoon.  Brain-inspired predictive coding improves the performance of machine challenging tasks.  Frontiers in Computational Neuroscience, 2022.
  * [242] Pietro Mazzaglia, Tim Verbelen, Ozan Çatal, and Bart Dhoedt.  The free energy principle for perception and action: A deep learning perspective.  Entropy, 24(2):301, 2022.
  * [243] Philip N Johnson-Laird.  Mental models and deduction.  Trends in cognitive sciences, 5(10):434–442, 2001.
  * [244] Daniel Kahneman and Gary Klein.  Conditions for intuitive expertise: a failure to disagree.  American psychologist, 64(6):515, 2009.
  * [245] Karl Friston.  Learning and inference in the brain.  Neural Networks, 16(9):1325–1352, 2003.
  * [246] Xaq Pitkow and Dora E Angelaki.  Inference in the brain: statistics flowing in redundant population codes.  Neuron, 94(5):943–953, 2017.
  * [247] Yunfan Lu, Yiqi Lin, Hao Wu, Yunhao Luo, Xu Zheng, and Lin Wang.  All one needs to know about priors for deep image restoration and enhancement: A survey.  arXiv preprint arXiv:2206.02070, 2022.
  * [248] Pang Wei Koh and Percy Liang.  Understanding black-box predictions via influence functions.  In International conference on machine learning, pages 1885–1894. PMLR, 2017.
  * [249] Jie Huang and Kevin Chen-Chuan Chang.  Towards reasoning in large language models: A survey.  arXiv preprint arXiv:2212.10403, 2022.
  * [250] SH Muggleton FREng.  Hypothesizing an algorithm from one example: the role of specificity.  Philosophical Transactions of the Royal Society A, 381(2251):20220046, 2023.
  * [251] Marilyn A Walker, Aravind Krishna Joshi, and Ellen Friedman Prince.  Centering theory in discourse.  Oxford University Press, 1998.
  * [252] Rolf A Zwaan and Gabriel A Radvansky.  Situation models in language comprehension and memory.  Psychological bulletin, 123(2):162, 1998.
  * [253] Rolf A Zwaan, Mark C Langston, and Arthur C Graesser.  The construction of situation models in narrative comprehension: An event-indexing model.  Psychological science, 6(5):292–297, 1995.
  * [254] Jeremy R Reynolds, Jeffrey M Zacks, and Todd S Braver.  A computational model of event segmentation from perceptual prediction.  Cognitive science, 31(4):613–643, 2007.
  * [255] Jerry R Hobbs.  Coherence and coreference.  Cognitive science, 3(1):67–90, 1979.
  * [256] Hong Chen, Zhenhua Fan, Hao Lu, Alan L Yuille, and Shu Rong.  Preco: A large-scale dataset in preschool vocabulary for coreference resolution.  arXiv preprint arXiv:1810.09807, 2018.
  * [257] David Bamman, Olivia Lewke, and Anya Mansoor.  An annotated dataset of coreference in english literature.  arXiv preprint arXiv:1912.01140, 2019.
  * [258] Olga Uryupina, Ron Artstein, Antonella Bristot, Federica Cavicchio, Francesca Delogu, Kepa J Rodriguez, and Massimo Poesio.  Annotating a broad range of anaphoric phenomena, in a variety of genres: the arrau corpus.  Natural Language Engineering, 26(1):95–128, 2020.
  * [259] Najoung Kim and Sebastian Schuster.  Entity tracking in language models.  arXiv preprint arXiv:2305.02363, 2023.
  * [260] Mohammed Amer and Tomás Maul.  A review of modularization techniques in artificial neural networks.  Artificial Intelligence Review, 52:527–561, 2019.
  * [261] Diksha Khurana, Aditya Koli, Kiran Khatter, and Sukhdev Singh.  Natural language processing: State of the art, current trends and challenges.  Multimedia tools and applications, 82(3):3713–3744, 2023.
  * [262] Giorgio Franceschelli and Mirco Musolesi.  On the creativity of large language models.  arXiv preprint arXiv:2304.00008, 2023.
  * [263] Hongbin Ye, Tong Liu, Aijia Zhang, Wei Hua, and Weiqiang Jia.  Cognitive mirage: A review of hallucinations in large language models.  arXiv preprint arXiv:2309.06794, 2023.
  * [264] Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Nick Barnes, and Ajmal Mian.  A comprehensive overview of large language models.  arXiv preprint arXiv:2307.06435, 2023.
  * [265] Ben Goertzel.  Artificial general intelligence: concept, state of the art, and future prospects.  Journal of Artificial General Intelligence, 5(1):1, 2014.
  * [266] Ismaeel Al Ridhawi, Safa Otoum, Moayad Aloqaily, and Azzedine Boukerche.  Generalizing ai: Challenges and opportunities for plug and play ai solutions.  IEEE Network, 35(1):372–379, 2020.
  * [267] Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongbo Xi, Yongchun Zhu, Hengshu Zhu, Hui Xiong, and Qing He.  A comprehensive survey on transfer learning.  Proceedings of the IEEE, 109(1):43–76, 2020.
  * [268] Wen Zhang, Lingfei Deng, Lei Zhang, and Dongrui Wu.  A survey on negative transfer.  IEEE/CAA Journal of Automatica Sinica, 10(2):305–329, 2022.
  * [269] Jiafei Duan, Samson Yu, Hui Li Tan, Hongyuan Zhu, and Cheston Tan.  A survey of embodied ai: From simulators to research tasks.  IEEE Transactions on Emerging Topics in Computational Intelligence, 6(2):230–244, 2022.
  * [270] Ron Chrisley.  Embodied artificial intelligence.  Artificial intelligence, 149(1):131–150, 2003.
  * [271] Yann Coello and Tina Iachini.  Embodied perception of objects and people in space: Towards a unified theoretical framework.  In Perceptual and emotional embodiment, pages 206–228. Routledge, 2015.
  * [272] Davide Castelvecchi.  Can we open the black box of ai?  Nature News, 538(7623):20, 2016.
  * [273] Feiyu Xu, Hans Uszkoreit, Yangzhou Du, Wei Fan, Dongyan Zhao, and Jun Zhu.  Explainable ai: A brief survey on history, research areas, approaches and challenges.  In Natural Language Processing and Chinese Computing: 8th CCF International Conference, NLPCC 2019, Dunhuang, China, October 9–14, 2019, Proceedings, Part II 8, pages 563–574. Springer, 2019.
  * [274] Wencan Zhang and Brian Y Lim.  Towards relatable explainable ai with the perceptual process.  In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pages 1–24, 2022.
  * [275] Joaquin Vanschoren.  Meta-learning.  Automated machine learning: methods, systems, challenges, pages 35–61, 2019.
  * [276] Aimee Van Wynsberghe.  Sustainable ai: Ai for sustainability and the sustainability of ai.  AI and Ethics, 1(3):213–218, 2021.
  * [277] Jonathan Gratch and Nathanael J Fast.  The power to harm: Ai assistants pave the way to unethical behavior.  Current Opinion in Psychology, 47:101382, 2022.
  * [278] Nenad Tomašev, Julien Cornebise, Frank Hutter, Shakir Mohamed, Angela Picciariello, Bec Connelly, Danielle CM Belgrave, Daphne Ezer, Fanny Cachat van der Haert, Frank Mugisha, et al.  Ai for social good: unlocking the opportunity for positive impact.  Nature Communications, 11(1):2468, 2020.