Gianluca Baldassarre1Giovanni Granato1 Laboratory of Computational Embodied
Neuroscience, Institute of Cognitive Sciences and Technologies, National
Research Council of Italy, Rome, ItalyLaboratory of Computational Embodied
Neuroscience, Institute of Cognitive Sciences and Technologies, National
Research Council of Italy, Rome, Italy

# Representation Internal-Manipulation (RIM):
A Neuro-Inspired Computational Theory of Consciousness

###### Abstract

Many theories, based on neuroscientific and psychological empirical evidence
and on computational concepts, have been elaborated to explain the emergence
of consciousness in the central nervous system. These theories propose key
fundamental mechanisms to explain consciousness, but they only partially
connect such mechanisms to the possible functional and adaptive role of
consciousness. Recently, some cognitive and neuroscientific models try to
solve this gap by linking consciousness to various aspects of goal-directed
behaviour, the pivotal cognitive process that allows mammals to flexibly act
in challenging environments. Here we propose the Representation Internal-
Manipulation (RIM) theory of consciousness, a theory that links the main
elements of consciousness theories to components and functions of goal-
directed behaviour, ascribing a central role for consciousness to the goal-
directed manipulation of internal representations. This manipulation relies on
four specific computational operations to perform the flexible internal
adaptation of all key elements of goal-directed computation, from the
representations of objects to those of goals, actions, and plans. Finally, we
propose the concept of ‘manipulation agency’ relating the sense of agency to
the internal manipulation of representations. This allows us to propose that
the subjective experience of consciousness is associated to the human capacity
to generate and control a simulated internal reality that is vividly perceived
and felt through the same perceptual and emotional mechanisms used to tackle
the external world.

11footnotemark: 1

The authors have equally contributed to the paper.

##  1 Introduction

Scientific theories of consciousness often focus on specific key mechanisms
possibly relevant for it such as the neuronal networks that support conscious
processing (Dehaene ., 1998; Dehaene Naccache, 2001; Dehaene Changeux, 2011),
their informational integration (Tononi, 2008; Tononi ., 2016; Koch ., 2016),
the brain hierarchies (Damasio, 1989; Meyer Damasio, 2009), and the
sensorimotor interactions involving brain, body, and the environment (O’Regan
Noe, 2001; O’Regan ., 2005).

Similarly to Baars’ approach stressing the possible functional elements and
adaptive role of conscious cognition (Baars, 1997; Baars ., 2003; Baars, 2005;
Baars ., 2013), our approach to consciousness strongly focuses on the
importance of considering the functional (adaptive) aspects of consciousness
alongside the possible mechanisms that might support them, integrated to form
a coherent system. In particular, here we propose a strong link between the
main features of consciousness highlighted by the aforementioned theories and
the processes and mechanisms involved in goal-directed behaviour (Balleine
Dickinson, 1998; Mannella ., 2013; Hills, 2006). Goal-directed processes allow
complex organisms to adapt to novel challenges of the environment through
complex forms of mental simulation (Gerlach ., 2011; Pezzulo ., 2013) allowing
them to plan actions ahead, to support complex decision making, and to solve
challenging problems. Despite various authors mention possible links between
consciousness and goal-directed processes (e.g., Rossano, 2003; Keller, 2014;
Chiappe MacDonald, 2005; Ressler, 2004; Pennartz, 2017; Bengio, 2017), how the
computational processes underlying goal-directed behaviour might be
specifically related to conscious processing remains elusive.

Here we propose the Representation Internal-Manipulation (RIM) theory of
consciousness. The theory posits that consciousness emerges within the goal-
directed system of brain pivoting on this key main components and processes: a
set of working memories mediating perception and imagination, an abstract
working memory representing the main elements of goal-directed processes, a
selection process (‘internal manipulator’) manipulating internal
representations at all levels, and a motivational system driving the
manipulator. In this proposal, a central role for consciousness is played by
the goal-directed manipulation of internal representations. This manipulation
process performs four classes of computational operations on internal
representations: decomposition, composition, abstraction, and specification.
The key adaptive function played by consciousness through these operations is
to flexibly modify as desired all key elements of goal-directed processes,
from the representations of objects and contexts to those of goals, actions,
and plans. The theory allows the integration of the key elements proposed by
the theories on consciousness while at the same time ascribing to them a
functional/adaptive valence.

Finally, we relate the internal manipulation of representations to the sense
of agency. This allows us to contribute to face the ‘hard problem of
consciousness’ (Chalmers, 1995), in particular to propose a possible
explanation of the subjective experience that accompanies conscious processes
(Dennett, 1988; Haggard, 2017). The contribution rests on a novel concept that
we formulate here, of ‘manipulation agency’. The idea is that the internal
manipulation system underlying consciousness allows humans to actively and
flexibly generate, control, and change as desired an internal
simulated/imagined reality. The sense of agency given by this flexible and
powerful control, together with the vividly sensed and felt experience of the
imagined reality that is processed through the same perceptual and emotional
mechanisms used to tackle the external world, gives rise to the subjective
experience accompanying consciousness.

##  2 Overview of some theories of consciousness

Many theories have been proposed to explain consciousness as a propriety
emergent from specific mechanisms of brain. Here we consider some major
theories that propose articulated systems of concepts on consciousness that
give a reasonable coverage of the relevant ideas on consciousness also
developed in other theories. We start from the theory of Baars that has the
merit to more strongly aim to propose a functional interpretation of conscious
processes, i.e. which cognitive functions useful for adaptation these might
play, an aim that we also follow here.

The global workspace theory of Baars (GWT, Baars 1997) proposes that
consciousness relies on a set of fundamental elements that might
metaphorically correspond to the elements of a theatre, the ‘theatre of
consciousness’. The fundamental elements are the conscious contents such as
percepts, thoughts, and judgements (‘actors in the stage’), the global
workspace of working memory where conscious contents are represented (‘theatre
stage’), the selective attention that allocates limited processing resources
to specific contents (‘theatre spotlight’), the executive functions
(‘director’) that, on the basis of the agent’s current goals, decide which
contents to activate, the unconscious background processes (‘audience’), that
contribute to furnish row information to the conscious contents (e.g.
contextual information). The author recently grounded the model on
neuroscientific evidence (Baars ., 2003; Baars, 2005; Baars ., 2013), in
particular by linking the ‘theatre elements’ with neural structures and
processes. Baars’ theory follows an approach that best fits our functional
approach to consciousness, and in this respect it can be considered a
precursor of our proposal.

A related biologically-grounded theory, the neuronal workspace model (NWM;
Dehaene Changeux 2011), proposes the existence of two computational sub-spaces
of brain networks to explain the emergence of conscious experience. A first
sub-space is formed by many specialised functional modules (e.g., sensory
areas or motor systems) characterised by high-density short/medium range
connections. A second sub-space is formed by a distributed set of neurons,
communicating trough excitatory long-range projections, involving the
associative cortices (prefrontal, parieto-temporal, and cingulate cortex) and
including the fibers of corpus callosum and of the cortico-thalamic system
(Dehaene Changeux, 2011). The NWM has been extended to envisage the existence
of buffers (working memories) between the sensorial cortices and the neuronal
workspace (Raffone ., 2014, 2015). Based on these anatomical organisation, the
model proposes that the reverberating networks formed by the top-down
projections (second interconnected computational space) and the bottom-up
projections (first computational space) implement specific selective processes
that lead to the emergence of a restricted set of relevant contents within
consciousness.

Another influential theory of consciousness, the integrated information theory
(IIT; Tononi 2008; Tononi . 2016), proposes that a high anatomical and
functional integration of a key brain sub-network, namely the thalamo-cortical
system, represents a critical element of consciousness. In particular, this
theory proposes that the representations of the thalamo-cortical system are
characterised by ‘specificity’, i.e. an substantial level of differentiation
and segregation with respect to other competitive representations, and
‘integration’, i.e. a relevant level of information exchanged by the units
that compose the same representation. A major strength of the theory is the
proposal of a quantitative measure of the level of such specificity and
integration, namely the $\Phi$ value. The theory postulates that a high $\Phi$
value represents the neural correlate of a subjective consciousness state. A
recent update of the theory (Koch ., 2016) distinguishes a brain hot zone that
supports the formation of conscious contents localised in the posterior
cortices and a frontal-parietal system that supports task-monitoring
processes, making conscious contents coherent.

Damasio (Damasio, 1989; Meyer Damasio, 2009) has proposed another relevant
theory, the convergence-divergence zones theory (CDZt), that focuses on the
hierarchical organisation of brain systems to explain the difference between
conscious and subconscious representations. In particular, this theory
postulates the existence of minor convergence zones, corresponding to the
sensory cortices, and a major central convergence zone, corresponding to
associative cortical areas (prefrontal cortex, parietal cortex, and temporal
cortex), that together form a hierarchy of convergence zones. The theory
proposes that this architecture supports a progressive integration of
information from perceptual to central areas that supports perceptual
processes, and an inverse flow of information that supports imaginary
processes. The theory also focuses on the emotional dimension of conscious
experience by referring to the somatic marker theory to explain the emotional
nuance of conscious representations (Bechara Damasio, 2005). In particular,
the term ‘somatic marker’ refers to the influence, taking place within the
major associative convergence zones, of somatic signals onto brain
representations that thus receive a prioritisation bias for being consciously
processed (Verdejo-García ., 2006).

O’Regan (O’Regan Noe, 2001; O’Regan ., 2005) proposed a sensorimotor theory of
consciousness that, in line with the approaches of embodied cognition
(Anderson, 2003; Garbarini Adenzato, 2004; Borghi Cimatti, 2010) and
enactivism (Hutto, 2005), goes beyond strictly cognitive processes to involve
the close interaction between brain, body, and environment. In particular, the
theory explains the features of conscious visual experience, proposing that
‘sensorimotor contingencies’ (the perception of world changes caused by
actions) are fundamental features for the emergence of conscious experience.
Furthermore, this theory explains the difference between a sensory experience
and other types of neural processes (e.g. imagination or reasoning) suggesting
the existence of two properties of action-contingent stimuli defined
‘allerting’ and ‘corporality’. While allerting corresponds to the capacity of
changing stimuli to exogenously attract our attention, corporality describes a
phenomenal change of perception that is consequent to a motor act of the
agent.

##  3 The goal-directed system

Goal-directed behaviour relies on sophisticated internal representations, such
as the representations of objects, goals, and world models, and is involved in
many higher order cognitive processes such as planning, reasoning, and problem
solving. In this section we present contributions from different disciplines,
in particular neurobiology, neuropsychology, psychology, and computational
literature, that clarify relevant aspects of the processes that underlie goal
directed behaviour in brain, behaviour, and artificial intelligence systems.
The contributions are in part overlapping; they are also so wide that here we
can only consider few main aspects of them.

###  3.1 Biological contributions

Given a specific condition of the environment, goal-directed behaviour allows
an agent to exhibit different behaviours depending on the goal that it intends
to pursue. In particular, the behaviour is not automatically triggered by
specific stimuli, as in habitual behaviour relying on stimulus-response (S-R)
associations (Yin Knowlton, 2006). Rather, it is guided by the future desired
environment states on the basis of previously acquired action-outcome (A-O)
associations (Balleine Dickinson, 1998). Goal-directed behaviour, initially
studied in animal species such as rats, is phylogenetically more recent than
habitual behaviour and relies on the evolutionary emergence of complex brain
processes (Passingham Wise, 2012). Among these, planning processes allow
animals to flexibly assemble sequences of actions in novel ways (Pfeiffer
Foster, 2013; Delatour Gisquet-Verrier, 2000), reasoning processes allow the
inference of new knowledge from previously acquired information (Blaisdell .,
2006; Aust ., 2008), and problem solving processes allow the accomplishment of
new goals by adjusting previously acquired knowledge (Newman ., 2003).

We now consider some main aspects of the brain goal-directed system (for more
details, see Cisek Kalaska, 2010; Baldassarre, Caligiore Mannella, 2013;
Caligiore ., 2019).

#### States and the cortical pathways

Brain processes perceptual information through multiple hierarchies of
cortical areas partially segregated by sensory modality, e.g. for hearing or
vision (Felleman Van Essen, 1991). For example, visual information is
processed from early occipital cortex, performing basic information processing
such as the detection of image edges, towards higher associative cortices,
processing information for object identification in the infero-temporal cortex
(‘ventral visual pathway’) or for supporting the interaction with object based
on and supporting object interaction, e.g. related to space location and size
encoded in the parietal cortex (‘dorsal visual pathway’; Goodale Milner,
1992). These areas support action performance by sending information to motor
cortices (Rizzolatti ., 1998). They also send information to the multimodal
integrative areas of prefrontal cortex (Passingham Wise, 2012), which then
exert a top-down control both over motor areas and backward towards the
perceptual cortical hierarchies to support attentional processes.

#### Goals and frontal cortex

Goals represent the pivotal components of goal-directed behaviour that
distinguish it from habitual behaviour. In particular, within the behavioural
psychology literature a goal is defined as an action outcome having biological
value, such as a food pellet, that can drive the performance of a behaviour
for the balancing of homeostasis (i.e for the satisfaction of an homeostatic
drive; Balleine Dickinson 1998; Berridge 2004). More in general, in cognitive
sciences a goal is defined as an internal representation of a desirable future
state of the world. This representation can be internally re-activated, in the
absence of its referent in the environment, for triggering actions directed to
accomplish the world state corresponding to the goal (Thill ., 2013). A goal
representation (e.g., the internal representation of a food pellet) can be
supported by different sensory modalities, as the smell and visual appearance,
and by different hierarchical processes with an increasing level of
abstraction and integration of sensory modalities (Mechelli ., 2004). In the
human brain, goals are formed within the orbitofrontal cortex and ventromedial
prefrontal cortex and are given motivational valence by lower-level
representations within amygdala and nucleus accumbens (Mirolli ., 2010; Roy .,
2012; Mannella ., 2013). Dorsolateral prefrontal cortex also encodes the more
complex and abstract rules/situations that are instrumental for the attainment
of biologicallyrelevant outcomes (Tsujimoto ., 2011).

#### Selection and the basal ganglia

A key brain mechanism supporting the selection of goal-directed behaviour
elements, such as states, goals, and actions, is implemented by basal-ganglia
(Redgrave ., 1999). These form basal ganglia-thalamo-cortical loops that
originate from the associative and frontal regions of cortex, and feedback
onto the same regions via the thalamus. These loops are partially segregated
into parts specialised to support selection of cortical contents involving in
particular (Yin Knowlton, 2006): motor cortices; associative cortices
(parietal, temporal, and dorsal prefrontal cortex); and limbic cortices
(orbital and ventromedial preforntal cortex). An overall fundamental pattern
of brain is the one where the cortical pathways discussed above, hosting
bottom-up and top-down information flow, are ‘intercepted’ by the basal-
ganglia loops that perform selection of contents within them (Baldassarre,
Caligiore Mannella, 2013). We will see that this pattern is very important for
the internal manipulation of information of the RIM theory proposed here.

#### Motivations

Motivations represent another component of GDB, strongly related to the
formation and the internal activation of goals. In particular motivations are
internal processes that, based on the organism’s ‘needs’, guide the selection
of certain behaviours and the modulation of other internal processes such as
attention and learning. There are various types of motivations, but in general
literature divided them in extrinsic (Panksepp, 1998) and intrinsic
motivations (Baldassarre, 2011). Extrinsic motivations refer to the basic
physiological motivations linked to survival and the reproduction such as
pain, hunger, and thirst (Panksepp, 1998). These motivations can lead to form
goals, for example in the form of visual/auditive internal representations of
the environment resources that can satisfy them (e.g. food or water). In
brain, extrinsic motivations are supported by areas such as the hypothalamus
and amygdala and involve reward-related systems such as the dopamine system
(Schultz, 2002).

Intrinsic motivation, endowed with a more sophisticated cognition (White,
1959; Baldassarre Mirolli, 2013; Fantz, 1964; Deci ., 1991) drive the
acquisition of knowledge that might be later employed for the acquisition of
biological relevant resources and are related to processes such as curiosity,
exploration, play, novelty, surprise, the setting of own goals and the
competence to accomplish them. Intrinsic motivations involve various brain
structures, for example hippocampus for novelty detection (Lisman Grace, 2005;
Kumaran Maguire, 2007) or the anterior cingulate cortex for surprise (Paus,
2001; O’Reilly ., 2013). Also intrinsic motivations can support the formation
of goals (Baldassarre, Mannella ., 2013).

###  3.2 Psychological contributions

#### Attention

Attention is a relevant process for goal-directed behaviour Attention
processes allow the acquisition of relevant information while discarding
irrelevant stimuli (selective attention; Treisman Gelade, 1980). They are also
fundamental for the formation of goals formation and their internal re-
activation. Attention can be divided in covert vs. overt attention (Lavie .,
2004; Rizzolatti ., 1987), endogenous (‘bottom-up’) vs. exogenous (‘top-down’)
attention (Posner Petersen, 1990).

Overt attention refers to the selection of environment information based on
the control of sensors focusing on a particular portion of the environment or
stimuli (e.g. the view of a food pellet regardless of the context where it is
found). Covert attention refers to an internal selection process that can
mentally isolate the relevant parts of an internal representation (e.g. to
internally focus on a certain portion of an imagined scene). Both covert and
overt attention can support the formation of focused representations of
states, goals, and models of the world.

Regarding the second distinction, exogenous attention refers to externally-
driven shifts of the attention focus, mostly caused by highly salient stimuli
or stimulus changes, while endogenous attention involves a voluntary shift
driven by the agents’ goals (Ognibene Baldassarre, 2015). Endogenous attention
is very important for goal-directed behaviour because it allows the internal
activate of desired goals and also support reasoning processes and planning
processes that rely on a serial scan of mental contents. These processes
strongly rely on top-down driven sensory imagination processes (Mechelli .,
2004; Seepanomwan ., 2015).

#### Planning

Given a selected goal, planning processes use the knowledge on action-outcome
contingencies to search a single action or an action sequence that leads to
accomplish the desired goal (Pfeiffer Foster, 2013; Delatour Gisquet-Verrier,
2000). In humans, this searching process is supported by an ‘internal
simulation’ assembling and testing actions sequences ‘within the mind’ rather
than by acting in the environment in humans (Gerlach ., 2011). These processes
might also involve non-human animals (Balleine Dickinson, 1998; Clayton
Dickinson, 2010). Planning processes are also supported by bottom-up
mechanisms such as affordances that suggest the possible actions that could be
useful and appropriate for a given condition or object (Thill ., 2013; Rounis
Humphreys, 2015; Garbarini Adenzato, 2004).

#### Problem solving

Problem solving is a sophisticated version of planning, considered a
fundamental element of human intelligence in early cognitive-science studies
(Simon, 1975). In early views, problem solving identifies the a flexible
search of the correct sequence of actions to pass from an initial state to a
final desired state (goal), so is analogous to planning. This view still has
an important role in current artificial intelligence models and cognitive
science interpretations of human problem solving (e.g. Condell . 2010; Waltz .
1999), but it overlooks the fact that life problem solving requires not only
to find suitable sequences of actions to achieve the goal, but also to face
knowledge gaps regarding the elements of planning, for examples the relevant
objects, their functions, the actions to act on them, and the possible effects
of actions (world model).

###  3.3 Neuro-psychological contributions

#### Executive functions

The neuro-psychology literature studies a set of processes defined ‘executive
functions’ (Diamond, 2013; Miyake ., 2000) that have a notable overlap with
goal-directed behaviour processes studied in the biological literature but
also add useful information. Executive functions support a coherent feedback-
dependent goal-based top-down control of behaviour and rely on three
fundamental processes, i.e. working memory, inhibitory control, and cognitive
flexibility.

#### Working memory

Working memory, also studied in psychology, is a fundamental process for goal-
directed behaviour referring to the capacity of the brain to keep active
information representations for prolongued periods of time. Without the
persistent maintenance of information by working memory cognition could
support only sensorimotor behaviours. Working memory can store information in
a modality-specific form (Fiehler ., 2008; Raffone ., 2014) or in an highly
integrated multimodal form (Baddeley, 2000; Wolters Raffone, 2008). Working
memory is a fundamental process for the selection and active maintenance of
goals and sub-goals, and for supporting planning and problem solving processes
through the maintenance of representations regarding past or future world
states and their possible causal/temporal links (world models).

#### Inhibitory control

Working memory processes have a synergistic interaction with inhibition
processes, another fundamental key element underlying goal-directed behaviour.
Inhibitory control allowing the focus on the selected goals by performing a
continuous suppression of distractors, such as other goals, intervening
stimuli, and irrelevant actions (Durston ., 2002).

#### Cognitive flexibility

The integrated interaction of working memory and inhibitory control allows the
emergence of a third process defined, cognitive flexibility, related to the
capacity to switch behavioural strategy depending on the external conditions.
For example, in a . The integrated functioning of working-memory, inhibitory
control, and cognitive flexibility allows the emergence of high-order forms of
executive functions such as goal monitoring, planning, and problems solving.

Studies regarding the neural correlate of executive functions (Robinson .,
2014; Goldman-Rakic, 1996; Hartley Speer, 2000) highlight many structures that
support goal-directed behaviour. In particular, working memory is suppo,rted
by basal ganglia-cortical loops, hippocampus and high-order prefrontal
cortices while inhibitory control mostly relies on basal-ganglia, ventromedial
prefrontal cortex and anterior cingulate cortex (Durston ., 2002). Cognitive
flexibility relies on a complex network involving various portions of basal-
ganglia, prefrontal cortex, anterior cingulate cortex, and parietal cortex
(Leber ., 2008). The monitoring of the attainment of sub-goals and final goals
relies on frontal portions of prefrontal cortex (Desrochers ., 2015) while the
anterior cingulate cortex shows a strong involvement in the detection of the
violation of expectations (Benn ., 2014).

###  3.4 Artificial intelligence contributions

Research on artificial intelligence, that initially was closely linked to the
study of natural intelligence, gave an important theoretical foundation to the
study of goal-directed processes (Russell Norvig, 2016). A first relevant
contribution was the one on problem solving introduced above, positing that
human intelligence is primarily directed to accomplish goals by searching
suitable sequences of actions leading from an initial state to a final goal
state passing through a number of intermediate sub-goals (Simon, 1975). These
processes involved the representation of the world states as whole elements,
such as the configuration of the pieces on the board in the chess game.

Later this approach was developed by the research on planning that made the
search of action sequences more efficient by factorising the representation of
states into elements (e.g. ‘objects’) and relations between elements (e.g.,
‘being on’, ‘being part of’) suitably expressed with logic symbols (Russell
Norvig, 2016). On this basis, planning systems could hence focus and reason on
fewer relevant elements and relations at a time.

This thread of research mostly proceeded in parallel with respect to the
research on neural networks (McClelland the PDPResearchGroup, 1986), which has
lately arrived to the great success of machine learning and deep neural
networks (Goodfellow ., 2017). This success is now bringing new interest and
energies to face the high-level problems tackled by planning and problem
solving by using approaches based on neural networks (e.g., Baldassarre, 2001;
Wayne ., 2018), a trend that is expected to deliver good outcomes also to the
research on consciousness (e.g., see Bengio, 2017).

##  4 The Representation Internal-Manipulation (RIM) theory of consciousness:
components and manipulation operations

In this section we present the Representation Internal-Manipulation (RIM)
theory of consciousness, in particular the main ‘components’, or set of
functions, that the theory posits to be at the core of consciousness processes
(Figure 1). As the brain works as a whole, these ‘components’ should be
thought to have a limited segregation in brain, i.e. to have a relative
overlap within the brain structures that implement them (Caligiore ., 2019).

Neuro-Functional Elements of the Representation Internal-Manipulation (RIM)
Theory

Figure 1: Schema showing the components of the RIM theory of consciousness.
The colour gradient (red to blue) indicates a gradual change of computational
functions from those encoding emotional/motivational elements (e.g. goals;
fronto-striatal cortex) to those encoding perceptual elements (e.g. visual
activations; occipital and dorso-caudal cortex)

#### (1) Perceptual working memory

This component is responsible for a bottom-up information processing
supporting modal perception (visual, auditive, tactile, somatosensory, etc.)
and the formation of increasingly abstract representations, e.g. it supports
the processing of visual images from the retina and the formation of
increasingly abstract representations, from low-level visual features (edges,
corners, etc.) to higher level representations (more complex features,
objects, and faces). The same component also supports a top-down information
flow moving in the opposite direction and supporting imagination, i.e. the
central re-activation of peripheral perceptual representations. The component,
supported in brain by reverberating loops involving sensory cortices, has the
capacity to implement modal working memories able to maintain active
perceptual representations having a high level of details and possibly several
dimensions. These activations also represent the conscious contents of which
we are aware.

#### (2) Abstract working-memory

This component supports the active maintenance of goals and behavioural
strategies in more abstract multimodal formats. This component, mostly
supported by fronto-striatal circuits, forming reverberating loops with
thalamus and basal ganglia, has the capacity of maintaining abstract
representations of the world elements (context, objects, other agents), and of
goals and action plans. These representations can be activated or de-activated
as needed on the basis of the current condition, the goals to pursue, and the
plans, decisions, and thoughts formulated to accomplish them.

#### (3) Top-down internal manipulator

This component, guided by motivationally-charged active goals, activates
specific contents within the abstract working memory and the perceptual
working memories, in particular by ‘manipulating’ them so to best use the
agent’s knowledge to pursue the currently desired goals. This component is
actually fully embedded into the other three components. In particular, it is
implemented by the biases from the motivational component, and by the
selection processes internal to the abstract and perceptual working memories.
The ‘component’ in particular relies on the excitatory fronto-parietal
cortical pathways for the top-down guidance, on the disinhibition mechanisms
of basal ganglia for performing macro selections, and on the local inhibitiory
circuits of cortex for performing micro selections. The component allows the
conscious flexible manipulation and morphing of externally-driven percepts
(e.g., visual or somatosensory representations) and of self-activated
representations (imagination). The internal manipulator represents the
keystone of the RIM theory.

#### (4) Motivation system

This component, supported in the brain by sub-cortical emotional systems such
as the hypothalamus, amygdala, hippocampus, and the nucleus accumbens,
furnishes the motivational valence to the representations within the other
components. In particular, it supports the selection/activation of specific
goals within the abstract working memory, thus forming ‘intentions’, and also
supports the selection of other relevant elements, for example enhances the
activation of representations related to objects and sub-goals relevant to
support intentional behaviour.

The four components support the internal manipulation of representations
proposed here to be at the core of consciousness. The manipulator performs
operations that can be grouped into four classes (Figure 2). We extracted
these four classes of operations from neuro-psychological and neuroscientific
literature, for example on executive functions, attention, the selective
processes of basal ganglia, and the hierarchical organisation of cortex. We
think these operations are the main means through which conscious processing
supports flexible goal-directed planning and problem solving. The four classes
of operations are as follows.

#### (1) Decomposition

This operation, linked to the concept of selective attention, allows the
separation of whole representations into pieces, thus allowing the agent to
focus on single elements (e.g., objects in the world or specific sub-goals)
and their parts. This operation supports the focusing on specific portions of
representations based on spatial regularities (space-based) or
functional/visual regularities (e.g., object-based).

#### (2) Composition

A ‘chunk’, or ‘composition’, is a group of elements glued together by a
higher-level semantic link (e.g., the eyes, nose, and mouth together form a
face). Neurally, a chunk can be implemented by connections that link the
neural representations of the different elements and/or connections that link
them to another neural representation of the semantic link. The connections
allow the activation of the neural representation of the semantic link when
needed, and to use it in place of the representations of the elements of the
composition. Alongside such structural chunks, the composition also allows the
formation of chunks through a functional connectivity, for example based on
the synchronous firing of spiking-neurons encoding the items of to be chunked.
Composition supports the assembling of items relevant for many goal-directed
processes, for example the creation of action sequences forming a plan, or to
consider wholes useful for problem solving.

#### (3) Abstraction

This operation allows the activation of internal representations that have
less details or dimensions with respect to some original representations.
There exist multiple possible abstractions of a pattern depending on the
information that is preserved or discarded, so the process of abstraction
needs to be biased in a specific direction. Abstraction is obtained by several
features contributing to form more complex features or to identify the ‘hidden
causes’ of given patterns. Abstraction is at the basis of the capacity of
humans to consider specific aspects of objects while ignoring others, for
example to focus on the colour, or the size, or the shape of an object.
Furthermore, abstraction is strongly linked to the capacity to create
simplified/compact representations of the world.

#### (4) Specification

Specification represents the inverse operation with respect to abstraction, in
particular it adds details and dimensions to original abstract
representations. Since specification implies few-to-many mappings, it requires
that the process of specification is biased towards a desired direction.
Neurally, specification is implemented by passing from abstract to more
specific representations, for example by passing from hidden causes to full
representations, for example to imagine some details of the face of a known
person. This operation, together with composition, is thus behind the capacity
of adding details to ideas and of imagining and creating new elements.

#### Conscious Knowledge Transfer (CKT)

The integrated functioning of the four operations give rise to a super-
ordinate function called here conscious knowledge transfer. CKT is an enhanced
version of generalisation, intended here as the capacity of neural networks to
respond in similar ways to similar stimuli; generalisation is in particular
based on the interpolation between previous experiences or the extrapolation
very proximal to such experiences. For example, an agent might experience an
object on various positions at the centre of the table and on this basis
imagine it a bit outside such centre (Nair ., 2018). Instead, CKT implies the
capacity of extrapolating knowledge far beyond previous experiences, e.g. to
imagine an object anywhere in a known space. Moreover, CKT involves all types
of flexible active construction of knowledge that, starting from experience,
can build fully new knowledge on the basis of the regularities extracted from
such knowledge, for example those related to the physical properties of the
environment, and to naturalistic and social knowledge (Lake ., 2017). This
construction of knowledge relies on knowledge building processes such as
analogy (Gentner Forbus, 2011), induction, and other types of inference.

All these processes allow conscious agents to flexibly ‘warp and morph’ all
elements of planning, such as contexts, objects, goals/sub-goals, and actions,
in order to accomplish the desired goals in the lack of sufficient knowledge
(flexible problem solving). For example, a goal might be split in never-
experienced sub-goals trough decomposition; a new function of an object might
be found by ‘importing’ it from another object through abstraction (which
allows to see similarities), and by linking it to the object by composition; a
plan might be searched at different levels of abstractions through abstraction
and specification of objects/states and actions. Overall, these abilities are
at the core of the flexible solution of new problems. New problems are
challenging exactly because their solution requires knowledge that the agent
initially does not have, and that has to be built through CKT processes. For
example, in the classic problem of Duncker (Duncker, 1945), formulated within
the psychological literature on problem solving, a participant has to solve a
problem of fixing a candle on a wall, and light it with some matches, using
some pins in a box. The solution is to pin the cardboard box of the pins to
the wall, and then to put the candle on it. Finding this solution requires to
use abstraction to realise that the box is made of cardboard, and so can be
pinned onto the wall, and to imagine by composition the never-seen compound
formed by the box pinned to the wall and the candle on it.

Figure 2: The four fundamental classes of operations that can be performed by
the representation internal-manipulator at the core of consciousness.

##  5 Link to the neuroscientific and psychological theories

The components of the RIM theory of consciousness, which are grounded on the
fundamental elements of the brain goal-directed system, can be linked to the
main aspects of the theories of consciousness proposed in the literature and
considered in the previous sections. This section draws some links between
those elements to these aspects as examples, but many other links can be
easily identified.

The global workspace theory (GWT; Baars, 1997) and the neuronal workspace
model (NWM; Dehaene Changeux, 2011) suggest that the active maintenance of
internal representations is a fundamental element of consciousness. This
functionality can be related to both the central working memory component of
the RIM theory, and the perceptual working memories. The two theories also
stress the importance of the amplification of relevant information by higher
order cortices. In particular, the NWM remarks the importance of the frontal-
parietal system for the top-down selection of information. In the RIM theory,
the amplification function relies on attention processes involving the top-
down cortical pathways and also on the selection processes relying on the
basal ganglia-thalamo-cortical loops (Granato Baldassarre, 2019). These
selection processes play a crucial role in the RIM theory as they are at the
basis of the internal manipulation of representations. Both theories stress
the importance of a consciousness bottleneck, for which only one/few elements
at a time enter the focus of attention. In the RIM theory, representations can
take place at multiple levels of abstraction, so the manipulator has to select
only one/few linked representations at a time to avoid interference between
them.

The convergence-divergence zones theory (CDZt; Damasio, 1989) proposes the
existence of a brain multi-level hierarchy for information processing,
highlighting the fundamental difference between a abstract and perceptual
processing and maintenance of information. The RIM theory of consciousness
also gives a relevant importance to the hierarchical processing and
maintenance of information, in particular at the level of abstract working
memory and perceptual working memories, to support the key process of
imagination.

The GWT and the NWM, and even more the somatic-marker hypothesis linked to the
CDZt, assign a notable importance to the motivational/emotional processes for
guiding conscious processes. In the RIM theory, the motivation system plays a
fundamental role to guide the selections performed by the top-down internal
manipulator onto the abstract and perceptual working memories. In this
respect, the motivation system is at the apex of the hierarchy that ultimately
establishes which information should be attended and which manipulations of
internal representations should be performed.

The integrated information theory (IIT; Tononi, 2008) stresses the features of
neural-network and cognitive architectures that can support conscious
processes. We hypothesise that the circuits posited by the RIM theory might
have a high $\Phi$ value (Tononi, 2008). In particular, we posit that in brain
the neural assemblies representing the elements relevant for goal-directed
processes (e.g., in cortex), and the loops supporting their internal selection
and manipulation (e.g., based on the cortical circuits and the basal ganglia-
thalamo-cortical loops), should present a high-level of specificity and
integration.

The sensorimotor theory of consciousness (O’Regan Noe, 2001) emphasises the
relevance for consciousness of the sensorimotor interaction that conscious
agents engage with the environment, in particular for the role played by the
causal action-outcome correlations that conscious agents can sense. The
closest link that the RIM theory has with this view is that the theory is a
functional theory that posits that consciousness is a process having an
adaptive utility for agents as it allows them to produce more adaptive
actions. In particular, conscious processes allow the flexible manipulation of
internal representations and these support a more effective planning and a
more powerful problem solving. The success of these processes requires a close
alignment between the agents’ internally manipulated representations and the
world where they act, and a tight sensation-action coupling with it for action
success, as claimed by the sensorimotor theory of consciousness.

##  6 Link to computational and machine-learning models: a research agenda

In this section we discuss some initial ideas on how the components of the RIM
theory could be implemented in computational models (Figure 3), while also
drafting a possible research agenda to realise them.

Representation Internal-Manipulation (RIM) Implementation Framework

Figure 3: Possible key elements of a RIM theory usable as a framework to build
specific computational models. The graph shows the possible functional
components of the theory, and some of the possible machine learning systems
that might be used to implement them.

A possibility to implement a computational model following the principles of
the RIM theory would be to realise a model capable of performing planning
processes as in classic AI systems, but based on neural networks (Baldassarre,
2001). The use of neural networks, vs. symbolic approaches, to implement the
system would be a condition necessary to allow the quantitative ‘morphing and
warping’ of the internal representations of the system.

Initially, the system could work on the basis of whole state/observations
representations (Russell Norvig, 2016) as commonly done in RL systems (RS.
Sutton Barto, 1998). This would allow the system to perform simple forms of
planning as in Dyna planning (R. Sutton, 1990), i.e. by using RL run on the
basis of an internal model of the world rather than in the actual real world
(Baldassarre, 2003). This would require the system to learn a model of the
environment on the basis of experience in the world, either before or during
the accomplishment of desirable goals. Moreover, the system might be able to
initially perform only one-step plans, involving the processing of the current
state, of the goal, and of actions: this system might already allow the study
of some elements of the internal manipulation of representations that is
central to the theory (we discuss below a model inspired by the RIM theory
exemplifying this).

It would be important to implement this initial system through the elements of
the RIM theory. In particular, the states of the world should be processed
through a generative model implementing a perceptual working memory, e.g. for
processing visual information (we discuss below a model inspired by the RIM
theory exemplifying this). The use of a generative model would allow the
system to abstract information along the bottom-up information flow and to
form neural representations capturing hidden causes of the percepts, possibly
at different levels of abstraction. The top-down manipulation of these
representations would allow the model to generate (imagine) different
percepts. The goal-driven active imagination of different possible input
stimuli could allow the creative solution of problems (as in the example model
discussed below). Different generative models could be used to implement this
component, such as Restricted Boltzmann Machines (RBM; Hinton, 2002),
variational autoencoders (VAE; Kingma Welling, 2013), or generative
adversarial networks (GAN; Goodfellow ., 2014).

An abstract working memory component could store persistent representations of
the pursued goal, sub-goals, and other elements relevant for planning (and
problem solving, see below). This component could be implemented through a
recurrent neural network such as a long-short term memory (LSTM; Hochreiter
Schmidhuber, 1997) or transformers (Vaswani ., 2017). The use of the latter
model could support an easier implementation of the internal attention
selective processes needed to manipulate internal representations, via the
manipulator system, thanks to the internal ‘attention heads’ of such model.

An internal manipulator component would allow the selection of elements needed
for planning such as suitable representations of the initial state, goals, and
actions to perform. This component could learn to select the internal
representations of the system on the basis of reinforcement learning (RL)
processes. Two strategies might be followed to this purpose. A first strategy,
now successfully used in several works (e.g., Mnih ., 2015; Levine ., 2016),
would use an ‘end-to-end’ RL strategy where the reward is used to train by RL
the outer action-oriented part of the model, and then to use a supervised
gradient-based algorithm to train all layers of the system, here including the
internal manipulation processes, down to the layers close to the input. A
second strategy, that we are exploring at the moment, is inspired by the
functioning of brain where reward signals (e.g., based on dopamine or
noradrenaline neuromodulators) directly reach, and affect the learning of, the
‘deep layers of the system. This strategy specifically implies to use reward
signals to learn to select by RL not only the representations of the actions
to perform in the environment, but also the internal representations of the
system that affect the selection of such actions (we are at the moment
developing some models implementing this idea).

A motivation system would furnish the reward signals to guide the learning of
the manipulation processes of the system. These rewards might be generated by
different possible mechanisms. They might just be standard rewards, as those
commonly used in RL (RS. Sutton Barto, 1998), or they might be represented by
‘pseudo-rewards’ indicating if a desired goal has been achieved.
Alternatively, they might be represented by ‘intrinsic rewards’ (Barto .,
2004; Baldassarre Mirolli, 2013), or goal-related ‘intrinsic pseudo-rewards’,
generated by intrinsic motivation mechanisms (Santucci ., 2016).

A further step of this research agenda might aim to implement a multi-step
planning process allowing the system to accomplish goals that require a
sequence of actions, rather than one action, to be accomplished (Wayne .,
2018).

Another fundamental step would be to enhance the system to allow it to do the
‘morphing and warping’ of internal representations advocated by the RIM theory
to actively build lacking knowledge elements that are necessary to solve
challenging problems that go beyond the need for a ‘simple’ planning.
Crucially, the system should be able to ‘search’ solutions to problems not
only by assembling sequences of actions, as in planning, but also by changing
the internal representations of the key elements of planning on the basis of
the four internal-manipulation operations of decomposition, composition,
abstract, and specification. Generative models are relevant for this purpose
as they show to have a notable potential to generate novel imagined
representations by manipulating the internal representations capturing
relevant hidden causes of learned stimuli (Klys ., 2018; Bau ., 2019).

An important means of validation of the principles of the RIM theory would be
to show their capacity to solve problems that require not only to accomplish
goals by planning (and possibly also to autonomously acquire the knowledge on
the world to support such planning) but also to solve problems that require
elements that have not been previously experienced/learned by the agent. The
system might for example be required to modify internal representations of
experienced conditions, objects, and actions to form new ones allowing the
solution of new problems.

The ideas above led us to implement two computational models aiming to
implement some of the principles of the RIM theory. A first model (Baldassarre
., 2019; see Figure 4) uses decomposition (based on overt and covert attention
processes) to segment an overall goal into a number of sub-goals that the
agent is able to solve. In particular, a goal assigned to the system required
the creation of a certain configuration of objects in space (based on a simple
custom simulator of simple 2D coloured circle/triangle/rectangle ‘objects’).
The agent could accomplish the overall goal by decomposing it into sub-
problems thanks to the independence between the sub-goals: each sub-goal could
be accomplished by re-arranging the single object in the position and state
indicated by the overall goal. The model could then accomplish each sub-goal
through one-step planning operations involving the performance of single
(hardwired) actions.

A second ‘companion’ model (Granato Baldassarre, 2019) implements some initial
mechanisms to perform abstraction and specification. The model reproduces the
behaviour, and errors, of human participants of the Wisconsin card sorting
test (WCST; Berg, ; Heaton ., 1993), a standard neuropsychological test used
to study cognitive flexibility. The model is able to actively apply
abstraction on objects (cards to sort), thus extracting either their shape, or
colour, or size, and then to search correspondences between objects, with
respect to one of such dimensions, to solve the WCST. The model is in
particular able to employ specification to imagine objects that are formed by
specific attributes, for example to imagine a small, red, circle. These
processes are based on a restricted Boltzmann machine that perceives cards
and, with its ‘hidden units’, learns (for now based on a supervised algorithm)
to encode the different shape/colour/size attributes of the objects depicted
on them. The different possible attributes of the cards are then used to
compare them along different dimensions and to solve the WCST by inferring the
unknown rule to match the cards through a RL algorithm.

Figure 4:  Key elements of previous computational models that implemented some
elements of the RIM theory. (Left) The model that can perform a decomposition
of some types of goals into sub-goals based on decomposition (Baldassarre .,
2019). (Right) The model that performs selected types of abstraction and
specification of input images (Granato Baldassarre, 2019).

###  6.1 Manipulation agency and the subjective experience of consciousness

The nature of subjective experience is a widely debated issue within the
philosophical literature. Many philosophers propose different ideas to explain
the uniqueness of subjective experience such as the ‘zombie argumentation’ of
Chalmers (Chalmers, 2003), for which any physicalist explanation of subjective
experience is deemed to fail, the red room mental experiment of Searle
(Searle, 2004), warning against the possibility of processing information
without having a deep grasp of the process, or the list of fundamental
features of qualia, i.e. private subjective sensations, of Dennet (Dennett,
1988).

Some scientific theories of consciousness point to what should be the
fundamental elements of subjective experience, for example through the concept
of ‘qualia space’, a computational model capturing the functioning of brain
thalamo-cortical networks at the basis of consciousness (Tononi, 2008); the
drawing of a close link between visual experience and embodied sensorimotor
loops (O’Regan Noe, 2001); the emotional dimension of conscious experience
(Damasio, 1996); or the dynamics of a complex distributed frontal-parietal
brain network proposed within the GWT (Seth ., 2005; Baars ., 2013) and NWM
(Dehaene Naccache, 2001). While we think that each of these proposals
highlights a fundamental aspect of the subjective elements of conscious
experience, it seems that they still remain unclear and that the RIM theory of
consciousness proposed here might contribute to better understand them.

Based on the RIM theory, a conscious agent intentionally controls the
representation internal-manipulator to create an internal simulated reality at
will. This manipulation has a notable power as it relies on the four flexible
primitive operations of manipulations illustrated above that can transform and
warp internal representations in arbitrary ways. This power allows the agent
to also internally simulate itself, e.g. based on images involving other
similar agents: these might be possibly abstracted along various dimensions
and then added, by composition, some features related to oneself, including
elements of own cognition (Fernandez-Duque ., 2000).

An key point of our view is that, based on mechanisms as those operating for
the actions exerted on the external world, conscious agents have a sense of
agency for the actions exerted through the internal manipulator on the
internal reality generated by imagination. We call this manipulation agency.

The internally simulated and manipulated realty relies on imagination that
involves also the lowest-level perceptual representations. The representations
that the internal manipulator acts on are largely overlapped to those
activated by the perception of the real world: this activation thus produces a
vivid perceptual and emotional experiences similar to those produced by
experiences in the real world. These processes are at the basis of the human
intuition that we have an exclusive private access to our conscious thoughts,
a fundamental feature of the concept of Qualia (Dennett, 1988, 2001).

To summarise, the RIM theory posits that the subjective experience of
consciousness relies on the operations of the goal-driven representation
internal-manipulator that allow the agent to intentionally create, and
powerfully manipulate as desired, an internal reality that possibly includes
elements of the agent itself and has the perceptual and emotional vividness of
the outer world.

The concept of manipulation agency could also be used to propose a possible
‘consciousness scale’. Increasing levels of manipulation agency could
characterise three states of: phenomenal consciousness, access consciousness,
and manipulative consciousness (Figure 5). The three levels of manipulation
agency could characterise different neuro-cognitive states accompanying three
typical mental processing modes. In particular, an unexpected event could
cause the transitory perception accompanied by a low level of control and
manipulation agency. Mind-wandering, a common brain mode leading to explore
sequences of conscious thoughts, and accompanying the performance of routine
automatic activities or causing a mental shift from a specific tasks, often
happening without awareness (Schooler ., 2011), might be characterised by a
medium level of manipulation agency. At last, a high attention focus
characterising states of mind as those achieved with mindfulness could be
accompanied by the highest level of control on own cognition and hence of
manipulation agency.

Our proposal could also clarify the difference between pseudo-hallucinations
(which do not mimic real perception and are perceived as unreal), and
hallucinations (perception in the absence of external stimuli, having the
quality of real perception, Telles-Correia ., 2015). In particular, different
alternations of manipulation agency could explain the presence of insight in
pseudohallucinations, and its lack in true hallucinations. These states are
also reported in (Figure 5) to indicate the levels of manipulation agency
possibly corresponding to them. The scale could also characterise dreams,
involving uncontrolled imagination during REM sleep, and lucid dreams,
involving controlled imagination during sleep (Stumbrys ., 2012).

Figure 5: A scale of consciousness based on the concept of manipulation
agency.

##  7 Conclusions

We introduced the representation internal-manipulation (RIM) theory of
consciousness. The theory views conscious processes as playing a key adaptive
function for flexible planning and problem solving. The theory pivots on the
idea that the processes of consciousness support the internal manipulation of
representations of all aspects of goal-directed processes, from the
representations of objects, to those of goals, actions, and plans. This
manipulation relies on four cardinal operations (decomposition, composition,
abstraction, and specification) that allow conscious representation
manipulation to build new knowledge on the basis of previously acquired
experience, a process called here conscious knowledge transfer.

We discussed how the RIM theory integrates various aspects of some main
theories of consciousness, in particular by linking them to different elements
of goal-directed processes at the basis of the theory. We also discussed
preliminary ideas on the possible computational implementation of some
elements of the theory based on current machine learning systems, and we
briefly reviewed initial computational models implemented to capture some of
the theory ideas.

Finally, we used the theory to propose a new interpretation of the subjective
elements of conscious experience by relying on the new concept of manipulation
agency. This indicates the sense that humans have of being able to manipulate
as desired, in powerful ways, an internal imagined reality that possibly
includes oneself and that is vividly sensed and felt as the outer world.

## References

  * Anderson (2003) Anderson2003Anderson, ML.  2003\.  Embodied cognition: A field guide Embodied cognition: A field guide.  Artificial intelligence149191–130.
  * Aust . (2008) aust2008inferentialAust, U., Range, F., Steurer, M. Huber, L.  2008\.  Inferential reasoning by exclusion in pigeons, dogs, and humans Inferential reasoning by exclusion in pigeons, dogs, and humans.  Animal cognition114587–597.
  * Baars (1997) Baars1997Baars, BJ.  1997\.  In the theatre of consciousness. Global workspace theory, a rigorous scientific theory of consciousness In the theatre of consciousness. global workspace theory, a rigorous scientific theory of consciousness.  Journal of Consciousness Studies44292–309.
  * Baars (2005) Baars2005Baars, BJ.  2005\.  Global workspace theory of consciousness: toward a cognitive neuroscience of human experience Global workspace theory of consciousness: toward a cognitive neuroscience of human experience.  Progress in brain research15045–53.
  * Baars . (2013) Baars2013Baars, BJ., Franklin, S. Ramsoy, TZ.  2013\.  Global workspace dynamics: cortical ”binding and propagation” enables conscious contents Global workspace dynamics: cortical ”binding and propagation” enables conscious contents.  Frontiers in psychology4.
  * Baars . (2003) Baars2003Baars, BJ., Ramsøy, TZ. Laureys, S.  2003\.  Brain, conscious experience and the observing self Brain, conscious experience and the observing self.  Trends in neurosciences2612671–675.
  * Baddeley (2000) Baddeley2000TheepisodicbufferanewcomponentofworkingmemoryBaddeley.  2000Nov.  The episodic buffer: a new component of working memory? The episodic buffer: a new component of working memory?  Trends Cogn Sci411417–423.
  * Baldassarre (2001) Baldassarre2002PlanningwithneuralnetworksandreinforcementlearningBaldassarre, G.  2001\.  Planning with neural networks and reinforcement learning Planning with neural networks and reinforcement learning PhD Thesis.  Colchester, UKComputer Science Department, University of Essex.
  * Baldassarre (2003) Baldassarre2003ForwardandbidirectionalplanningbasedonreinforcementlearningandneuralnetworksinasimulatedrobotBaldassarre, G.  2003\.  Forward and bidirectional planning based on reinforcement learning and neural networks in a simulated robot Forward and bidirectional planning based on reinforcement learning and neural networks in a simulated robot.  M. Butz, O. Sigaud P. GÃ©rard (), Anticipatory behaviour in adaptive learning systems Anticipatory behaviour in adaptive learning systems ( 2684, 179–200).  BerlinSpringer Verlag.
  * Baldassarre (2011) Baldassarre2011WhatareintrinsicmotivationsAbiologicalperspectiveBaldassarre, G.  2011\.  What are intrinsic motivations? A biological perspective What are intrinsic motivations? a biological perspective.  A. Cangelosi . (), Proceedings of the International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob-2011) Proceedings of the international conference on development and learning and epigenetic robotics (icdl-epirob-2011) ( e1–8).  New York, NYIEEE.  Frankfurt am Main, Germany, 24–27/08/11
  * Baldassarre, Caligiore Mannella (2013) BaldassarreCaligioreMannella2013ThehierarchicalorganisationofcorticalandbasalgangliasystemsacomputationallyinformedreviewandintegratedhypothesisBaldassarre, G., Caligiore, D. Mannella, F.  2013\.  The hierarchical organisation of cortical and basal-ganglia systems: a computationally-informed review and integrated hypothesis The hierarchical organisation of cortical and basal-ganglia systems: a computationally-informed review and integrated hypothesis.  G. Baldassarre M. Mirolli (), Computational and Robotic Models of the Hierarchical Organisation of Behaviour Computational and robotic models of the hierarchical organisation of behaviour ( 237-270).  BerlinSpringer-Verlag.
  * Baldassarre . (2019) baldassarre2019embodiedBaldassarre, G., Lord, W., Granato, G. Santucci, VG.  2019\.  An embodied agent learning affordances with intrinsic motivations and solving extrinsic tasks with attention and one-step planning An embodied agent learning affordances with intrinsic motivations and solving extrinsic tasks with attention and one-step planning.  Frontiers in neurorobotics1345.
  * Baldassarre, Mannella . (2013) BaldassarreMannellaFioreRedgraveGurneyMirolli2013IntrinsicallymotivatedactionoutcomelearningandgoalbasedactionrecallAsystemlevelbioconstrainedcomputationalmodelBaldassarre, G., Mannella, F., Fiore, VG., Redgrave, P., Gurney, K. Mirolli, M.  2013\.  Intrinsically motivated action-outcome learning and goal-based action recall: A system-level bio-constrained computational model. Intrinsically motivated action-outcome learning and goal-based action recall: A system-level bio-constrained computational model.  Neural Networks41168-187.
  * Baldassarre Mirolli (2013) BaldassarreMirolli2013IntrinsicallymotivatedlearninginnaturalandartificialsystemsBaldassarre, G. Mirolli, M. ().  2013\.  Intrinsically motivated learning in natural and artificial systems Intrinsically motivated learning in natural and artificial systems.  BerlinSpringer.  Cost 91.62 euros, pp. 458, 82 illustrations, 55 illustrations in color
  * Balleine Dickinson (1998) BalleineDickinson1998GoaldirectedinstrumentalactioncontingencyandincentivelearningandtheircorticalsubstratesBalleine, BW. Dickinson, A.  1998\.  Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Goal-directed instrumental action: contingency and incentive learning and their cortical substrates.  Neuropharmacology374-5407–419.
  * Barto . (2004) BartoSinghChentanez2004IntrinsicallyMotivatedLearningofHierarchicalCollectionsofSkillsBarto, AG., Singh, S. Chentanez, N.  2004\.  Intrinsically motivated learning of hierarchical collections of skills Intrinsically motivated learning of hierarchical collections of skills.  J. Triesch T. Jebara (), International Conference on Developmental Learning (ICDL2004) International conference on developmental learning (icdl2004) ( 112–119).  New York, NY.
  * Bau . (2019) bau2019semanticBau, D., Strobelt, H., Peebles, W., Wulff, J., Zhou, B., Zhu, JY. Torralba, A.  2019\.  Semantic photo manipulation with a generative image prior Semantic photo manipulation with a generative image prior.  ACM Transactions on Graphics (TOG)38459.
  * Bechara Damasio (2005) bechara2005somaticBechara, A. Damasio, AR.  2005\.  The somatic marker hypothesis: A neural theory of economic decision The somatic marker hypothesis: A neural theory of economic decision.  Games and economic behavior522336–372.
  * Bengio (2017) bengio2017consciousnessBengio, Y.  2017\.  The consciousness prior The consciousness prior.  arXiv preprint arXiv:1709.08568.
  * Benn . (2014) BennWebbChangSunWilkinsonFarrow2014TheneuralbasisofmonitoringgoalprogressBenn, Y., Webb, TL., Chang, BPI., Sun, YH., Wilkinson, ID. Farrow, TFD.  2014\.  The neural basis of monitoring goal progress. The neural basis of monitoring goal progress.  Front Hum Neurosci8688.
  * Berg () Berg1948ASimpleObjectiveTechniqueforMeasuringFlexibilityinThinkingBerg, EA.  .  A simple objective technique for measuring flexibility in thinking A simple objective technique for measuring flexibility in thinking.  Journal General Psychology3915–22.
  * Berridge (2004) Berridge2004MotivationconceptsinbehavioralneuroscienceBerridge, KC.  2004Apr.  Motivation concepts in behavioral neuroscience Motivation concepts in behavioral neuroscience.  Physiol Behav812179–209.
  * Blaisdell . (2006) blaisdell2006causalBlaisdell, AP., Sawa, K., Leising, KJ. Waldmann, MR.  2006\.  Causal reasoning in rats Causal reasoning in rats.  Science31157631020–1022.
  * Borghi Cimatti (2010) BorghiCimatti2010EmbodiedcognitionandbeyondactingandsensingthebodyBorghi, AM. Cimatti, F.  2010Feb.  Embodied cognition and beyond: acting and sensing the body. Embodied cognition and beyond: acting and sensing the body.  Neuropsychologia483763–773.
  * Caligiore . (2019) CaligioreArbibMiallBaldassarre2019TheSuperLearningHypothesisIntegratingLearningProcessesacrossCortexCerebellumandBasalGangliaCaligiore, D., Arbib, MA., Miall, CR. Baldassarre, G.  201902\.  The super-learning hypothesis: Integrating learning processes across cortex, cerebellum and basal ganglia The super-learning hypothesis: Integrating learning processes across cortex, cerebellum and basal ganglia.  Neuroscience and Biobehavioral Reviews10019–34.
  * Chalmers (1995) Chalmers1995Chalmers, DJ.  1995\.  Facing up to the problem of consciousness Facing up to the problem of consciousness.  Journal of consciousness studies23200–219.
  * Chalmers (2003) Chalmers2003Chalmers, DJ.  2003\.  Consciousness and its place in nature Consciousness and its place in nature.  Blackwell guide to the philosophy of mind102–142.
  * Chiappe MacDonald (2005) ChiappeMacDonald2005TheevolutionofdomaingeneralmechanismsinintelligenceandlearningChiappe, D. MacDonald, K.  2005Jan.  The evolution of domain-general mechanisms in intelligence and learning. The evolution of domain-general mechanisms in intelligence and learning.  J Gen Psychol13215–40.
  * Cisek Kalaska (2010) CisekKalaska2010NeuralMechanismsforInteractingwithaWorldFullofActionChoicesCisek, P. Kalaska, JF.  2010\.  Neural mechanisms for interacting with a world full of action choices Neural mechanisms for interacting with a world full of action choices.  Annu Rev Neurosci33269–298.
  * Clayton Dickinson (2010) clayton2010mentalClayton, NS. Dickinson, A.  2010\.  Mental Time Travel: Can Animals Recall the Past and Plan for the Future? Mental time travel: Can animals recall the past and plan for the future?  Encyclopedia of Animal Behavior438–442.
  * Condell . (2010) condell2010problemCondell, J., Wade, J., Galway, L., McBride, M., Gormley, P., Brennan, J. Somasundram, T.  2010\.  Problem solving techniques in cognitive science Problem solving techniques in cognitive science.  Artificial Intelligence Review343221–234.
  * Damasio (1989) Damasio1989ThebrainbindsentitiesandeventsbymultiregionalactivationfromconvergencezonesDamasio, AR.  1989\.  The brain binds entities and events by multiregional activation from convergence zones The brain binds entities and events by multiregional activation from convergence zones.  Neural Computation11123–132.
  * Damasio (1996) Damasio1996ThesomaticmarkerhypothesisandthepossiblefunctionsoftheprefrontalcortexDamasio, AR.  1996Oct.  The somatic marker hypothesis and the possible functions of the prefrontal cortex. The somatic marker hypothesis and the possible functions of the prefrontal cortex.  Philos Trans R Soc Lond B Biol Sci35113461413–1420.
  * Deci . (1991) DeciVallerandPelletierRyan1991MotivationandeducationTheselfdeterminationperspectiveDeci, EL., Vallerand, RJ., Pelletier, LG. Ryan, RM.  1991\.  Motivation and education: The self-determination perspective Motivation and education: The self-determination perspective.  Educational psychologist263-4325–346.
  * Dehaene Changeux (2011) DehaeneChangeux2011ExperimentalandtheoreticalapproachestoconsciousprocessingDehaene, S. Changeux, JP.  2011Apr.  Experimental and theoretical approaches to conscious processing. Experimental and theoretical approaches to conscious processing.  Neuron702200–227.
  * Dehaene . (1998) Dehaene1998Dehaene, S., Kerszberg, M. Changeux, JP.  199811\.  A neuronal model of a global workspace in effortful cognitive tasks. A neuronal model of a global workspace in effortful cognitive tasks.  Proceedings of the National Academy of Sciences of the United States of America9514529–14534.
  * Dehaene Naccache (2001) Dehaene2001Dehaene, S. Naccache, L.  2001\.  Towards a cognitive neuroscience of consciousness: basic evidence and a workspace framework Towards a cognitive neuroscience of consciousness: basic evidence and a workspace framework.  Cognition7911–37.
  * Delatour Gisquet-Verrier (2000) delatour2000functionalDelatour, B. Gisquet-Verrier, P.  2000\.  Functional role of rat prelimbic-infralimbic cortices in spatial memory: evidence for their involvement in attention and behavioural flexibility Functional role of rat prelimbic-infralimbic cortices in spatial memory: evidence for their involvement in attention and behavioural flexibility.  Behavioural brain research1091113–128.
  * Dennett (1988) Dennett1988Dennett, DC.  1988\.  Quining qualia Quining qualia.  Consciousness in modern science. Consciousness in modern science.  Oxford University Press.
  * Dennett (2001) Dennett2001Dennett, DC.  2001\.  Are we explaining consciousness yet? Are we explaining consciousness yet?  Cognition791221–237.
  * Desrochers . (2015) DesrochersBurkBadreSheinberg2015TheMonitoringandControlofTaskSequencesinHumanandNonHumanPrimatesDesrochers, TM., Burk, DC., Badre, D. Sheinberg, DL.  2015\.  The Monitoring and Control of Task Sequences in Human and Non-Human Primates. The monitoring and control of task sequences in human and non-human primates.  Front Syst Neurosci9185.
  * Diamond (2013) Diamond2013ExecutivefunctionsDiamond, A.  2013\.  Executive functions. Executive functions.  Annu Rev Psychol64135–168.
  * Duncker (1945) Duncker1945OnProblemSolvingDuncker, K.  1945\.  On Problem Solving On problem solving ( 58).  American Psychological Association.
  * Durston . (2002) durston2002neuralDurston, S., Thomas, KM., Yang, Y., Uluğ, AM., Zimmerman, RD. Casey, B.  2002\.  A neural basis for the development of inhibitory control A neural basis for the development of inhibitory control.  Developmental Science54F9–F16.
  * Fantz (1964) FANTZ1964VISUALEXPERIENCEININFANTSDECREASEDATTENTIONTOFAMILIARPATTERNSRELATIVETONOVELONESFantz, RL.  1964Oct.  VISUAL EXPERIENCE IN INFANTS: DECREASED ATTENTION TO FAMILIAR PATTERNS RELATIVE TO NOVEL ONES. Visual experience in infants: Decreased attention to familiar patterns relative to novel ones.  Science1463644668–670.
  * Felleman Van Essen (1991) FellemanEssen1991DistributedhierarchicalprocessingFelleman, DJ. Van Essen, DC.  1991\.  Distributed hierarchical processing in the primate cerebral cortex Distributed hierarchical processing in the primate cerebral cortex.  Cereb Cortex111–47.
  * Fernandez-Duque . (2000) fernandez2000executiveFernandez-Duque, D., Baird, JA. Posner, MI.  2000\.  Executive attention and metacognitive regulation Executive attention and metacognitive regulation.  Consciousness and cognition92288–307.
  * Fiehler . (2008) FiehlerBurkeEngelBienRoesler2008KinestheticworkingmemoryandactioncontrolwithinthedorsalstreamFiehler, K., Burke, M., Engel, A., Bien, S. Rösler, F.  2008Feb.  Kinesthetic working memory and action control within the dorsal stream. Kinesthetic working memory and action control within the dorsal stream.  Cereb Cortex182243–253.
  * Garbarini Adenzato (2004) Garbarini2004Garbarini, F. Adenzato, M.  200410\.  At the root of embodied cognition: cognitive science meets neurophysiology. At the root of embodied cognition: cognitive science meets neurophysiology.  Brain and cognition56100–106.
  * Gentner Forbus (2011) gentner2011computationalGentner, D. Forbus, KD.  2011\.  Computational models of analogy Computational models of analogy.  Wiley Interdisciplinary Reviews: Cognitive Science23266–276.
  * Gerlach . (2011) GerlachSprengGilmoreSchacter2011SolvingfutureproblemsdefaultnetworkandexecutiveactivityassociatedwithgoaldirectedmentalsimulationsGerlach, KD., Spreng, RN., Gilmore, AW. Schacter, DL.  2011Apr.  Solving future problems: default network and executive activity associated with goal-directed mental simulations. Solving future problems: default network and executive activity associated with goal-directed mental simulations.  Neuroimage5541816–1824.
  * Goldman-Rakic (1996) GoldmanRakic1996TheprefrontallandscapeimplicationsoffunctionalarchitectureforunderstandinghumanmentationandthecentralexecutiveGoldman-Rakic, PS.  1996Oct.  The prefrontal landscape: implications of functional architecture for understanding human mentation and the central executive. The prefrontal landscape: implications of functional architecture for understanding human mentation and the central executive.  Philos Trans R Soc Lond B Biol Sci35113461445–1453.
  * Goodale Milner (1992) GoodaleMilner1992SeparateVisualPathwaysforPerceptionandActionGoodale, MA. Milner, AD.  199201\.  Separate visual pathways for perception and action Separate visual pathways for perception and action.  Trends Neurosci15120–25.
  * Goodfellow . (2017) BengioGoodfellowCourville2017DeepLearningGoodfellow, I., Bengio, Y. Courville, A.  2017\.  Deep Learning Deep learning.  Boston, MAThe MIT Press.  http://www.iro.umontreal.ca/ bengioy/dlbook
  * Goodfellow . (2014) GoodfellowPougetAbadieMirzaXuWardeFarleyOzairCourvilleBengio2014GenerativeadversarialnetsGoodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S.Bengio, Y.  2014\.  Generative adversarial nets Generative adversarial nets.  Advances in Neural Information Processing Systems Advances in neural information processing systems ( 2672–2680).  10 Jun 2014
  * Granato Baldassarre (2019) GranatoBaldassarre2019Granato, G. Baldassarre, G.  2019\.  Goal-directed top-down control of perceptual representations: A computational model of the Wisconsin Card Sorting Test Goal-directed top-down control of perceptual representations: A computational model of the wisconsin card sorting test.  Conference on Cognitive Computational Neuroscience. Conference on cognitive computational neuroscience.  13-16 September 2019, Berlin Germany
  * Haggard (2017) haggard2017senseHaggard, P.  2017\.  Sense of agency in the human brain Sense of agency in the human brain.  Nature Reviews Neuroscience184196.
  * Hartley Speer (2000) hartley2000locatingHartley, AA. Speer, NK.  2000\.  Locating and fractionating working memory using functional neuroimaging: storage, maintenance, and executive functions Locating and fractionating working memory using functional neuroimaging: storage, maintenance, and executive functions.  Microscopy research and technique51145–53.
  * Heaton . (1993) heaton1993wcstHeaton, RK., Chelune, GJ., Talley, JL., Kay, GG. Curtiss, G.  1993\.  WCST: Wisconsin card sorting test Wcst: Wisconsin card sorting test.  Psychological Assessment resources.
  * Hills (2006) Hills2006AnimalforagingandtheevolutionofgoaldirectedcognitionHills, TT.  2006Jan.  Animal foraging and the evolution of goal-directed cognition. Animal foraging and the evolution of goal-directed cognition.  Cogn Sci3013–41.
  * Hinton (2002) Hinton2002TrainingproductsofexpertsbyminimizingcontrastivedivergenceHinton, GE.  2002\.  Training products of experts by minimizing contrastive divergence Training products of experts by minimizing contrastive divergence.  Neural computation1481771–1800.
  * Hochreiter Schmidhuber (1997) HochreiterSchmidhuber1997LongShortTermMemoryHochreiter, S. Schmidhuber, J.  1997\.  Long short-term memory. Long short-term memory.  Neural computation91735–1780.
  * Hutto (2005) Hutto2005KnowingwhatRadicalversusconservativeenactivismHutto, DD.  2005\.  Knowing what? Radical versus conservative enactivism Knowing what? radical versus conservative enactivism.  Phenomenology and the Cognitive Sciences44389–405.
  * Keller (2014) Keller2014TheevolutionaryfunctionofconsciousinformationprocessingisrevealedbyitstaskdependencyintheolfactorysystemKeller, A.  2014\.  The evolutionary function of conscious information processing is revealed by its task-dependency in the olfactory system. The evolutionary function of conscious information processing is revealed by its task-dependency in the olfactory system.  Front Psychol562.
  * Kingma Welling (2013) KingmaWelling2013AutoEncodingVariationalBayesKingma, DP. Welling, M.  2013\.  Auto-Encoding Variational Bayes Auto-encoding variational bayes.  arXiv.
  * Klys . (2018) klys2018learningKlys, J., Snell, J. Zemel, R.  2018\.  Learning latent subspaces in variational autoencoders Learning latent subspaces in variational autoencoders.  Advances in Neural Information Processing Systems Advances in neural information processing systems ( 6444–6454).
  * Koch . (2016) koch2016neuralKoch, C., Massimini, M., Boly, M. Tononi, G.  2016\.  Neural correlates of consciousness: progress and problems Neural correlates of consciousness: progress and problems.  Nature Reviews Neuroscience175307.
  * Kumaran Maguire (2007) KumaranMaguire2007MatchmismatchprocessesunderliehumanhippocampalresponsestoassociativenoveltyKumaran, D. Maguire, EA.  2007Aug.  Match mismatch processes underlie human hippocampal responses to associative novelty. Match mismatch processes underlie human hippocampal responses to associative novelty.  J Neurosci27328517–8524.
  * Lake . (2017) LakeUllmanTenenbaumGershman2017Buildingmachinesthatlearnandthinklikepeople_TargetPaperLake, BM., Ullman, TD., Tenenbaum, JB. Gershman, SJ.  2017\.  Building machines that learn and think like people Building machines that learn and think like people.  Brain and Behavioural Sciences401–72.
  * Lavie . (2004) LavieHirstdeFockertViding2004LoadtheoryofselectiveattentionandcognitivecontrolLavie, N., Hirst, A., de Fockert, JW. Viding, E.  2004Sep.  Load theory of selective attention and cognitive control. Load theory of selective attention and cognitive control.  J Exp Psychol Gen1333339–354.
  * Leber . (2008) LeberTurkBrowneChun2008NeuralpredictorsofmomenttomomentfluctuationsincognitiveflexibilityLeber, AB., Turk-Browne, NB. Chun, MM.  2008Sep.  Neural predictors of moment-to-moment fluctuations in cognitive flexibility. Neural predictors of moment-to-moment fluctuations in cognitive flexibility.  Proc Natl Acad Sci U S A1053613592–13597.
  * Levine . (2016) LevineFinnDarrellAbbeel2016EndtoendtrainingofdeepvisuomotorpoliciesLevine, S., Finn, C., Darrell, T. Abbeel, P.  2016\.  End-to-end training of deep visuomotor policies End-to-end training of deep visuomotor policies.  Journal of Machine Learning Research17391–40.
  * Lisman Grace (2005) lisman2005hippocampalLisman, JE. Grace, AA.  2005\.  The hippocampal-VTA loop: controlling the entry of information into long-term memory The hippocampal-vta loop: controlling the entry of information into long-term memory.  Neuron465703–713.
  * Mannella . (2013) MannellaGurneyBaldassarre2013ThenucleusaccumbensasanexusbetweenvaluesandgoalsingoaldirectedbehaviorareviewandanewhypothesisMannella, F., Gurney, K. Baldassarre, G.  2013\.  The nucleus accumbens as a nexus between values and goals in goal-directed behavior: a review and a new hypothesis. The nucleus accumbens as a nexus between values and goals in goal-directed behavior: a review and a new hypothesis.  Frontiers in Behavioral Neuroscience7135e1-29.
  * McClelland the PDPResearchGroup (1986) McClellandPDPResearchGroup1986ParallelDistributedProcessingExplorationsintheMicrostructureofCognitionMcClelland, DE., James L. andRumelhart the PDPResearchGroup.  1986\.  Parallel distributed processing: explorations in the microstructure of cognition Parallel distributed processing: explorations in the microstructure of cognition ( 1).  Cambridge,MAThe MIT Press.
  * Mechelli . (2004) MechelliPriceFristonIshai2004WherebottomupmeetstopdownneuronalinteractionsduringperceptionandimageryMechelli, A., Price, CJ., Friston, KJ. Ishai, A.  2004Nov.  Where bottom-up meets top-down: neuronal interactions during perception and imagery. Where bottom-up meets top-down: neuronal interactions during perception and imagery.  Cereb Cortex14111256–1265.
  * Meyer Damasio (2009) MeyerDamasio2009ConvergenceanddivergenceinaneuralarchitectureforrecognitionandmemoryMeyer, K. Damasio, A.  2009Jul.  Convergence and divergence in a neural architecture for recognition and memory. Convergence and divergence in a neural architecture for recognition and memory.  Trends Neurosci327376–382.
  * Mirolli . (2010) mirolli2010rolesMirolli, M., Mannella, F. Baldassarre, G.  2010\.  The roles of the amygdala in the affective regulation of body, brain, and behaviour The roles of the amygdala in the affective regulation of body, brain, and behaviour.  Connection Science223215–245.
  * Miyake . (2000) MiyakeFriedmanEmersonWitzkiHowerterWager2000TheunityanddiversityofexecutivefunctionsandtheircontributionstocomplexFrontalLobetasksalatentvariableanalysisMiyake, A., Friedman, NP., Emerson, MJ., Witzki, AH., Howerter, A. Wager, TD.  2000Aug.  The unity and diversity of executive functions and their contributions to complex ”Frontal Lobe” tasks: a latent variable analysis. The unity and diversity of executive functions and their contributions to complex ”frontal lobe” tasks: a latent variable analysis.  Cogn Psychol41149–100.
  * Mnih . (2015) MnihKavukcuogluSilverRusuVenessBellemareGravesRiedmillerFidjelandOstrovskiPetersenBeattieSadikAntonoglouKingKumaranWierstraLeggHassabis2015HumanlevelcontrolthroughdeepreinforcementlearningMnih, V., Kavukcuoglu, K., Silver, D., Rusu, AA., Veness, J., Bellemare, MG.Hassabis, D.  201502\.  Human-level control through deep reinforcement learning Human-level control through deep reinforcement learning.  Nature5187540529–533.
  * Nair . (2018) NairPongDalalBahlLinLevine2018VisualReinforcementLearningwithImaginedGoalsNair, A., Pong, V., Dalal, M., Bahl, S., Lin, S. Levine, S.  2018\.  Visual Reinforcement Learning with Imagined Goals Visual reinforcement learning with imagined goals.  The Second Lifelong Learning: A Reinforcement Learning Approach Workshop (LLRLA2018 at FAIM2018). The second lifelong learning: A reinforcement learning approach workshop (llrla2018 at faim2018).  Stockholm, Sweden
  * Newman . (2003) newman2003frontalNewman, SD., Carpenter, PA., Varma, S. Just, MA.  2003\.  Frontal and parietal participation in problem solving in the Tower of London: fMRI and computational modeling of planning and high-level perception Frontal and parietal participation in problem solving in the tower of london: fmri and computational modeling of planning and high-level perception.  Neuropsychologia41121668–1682.
  * Ognibene Baldassarre (2015) OgnibeneBaldassarre2015EcologicalactivevisionfourbioinspiredprinciplestointegratebottomupandadaptivetopdownattentiontestedwithasimplecameraarmrobotOgnibene, D. Baldassarre, G.  2015\.  Ecological active vision: four bio-inspired principles to integrate bottom-up and adaptive top-down attention tested with a simple camera-arm robot Ecological active vision: four bio-inspired principles to integrate bottom-up and adaptive top-down attention tested with a simple camera-arm robot.  IEEE Transactions on Autonomous Mental Development713–25.
  * O’Regan Noe (2001) OReganNoee2001AsensorimotoraccountofvisionandvisualconsciousnessO’Regan, JK. Noe, A.  2001Oct.  A sensorimotor account of vision and visual consciousness. A sensorimotor account of vision and visual consciousness.  Behav Brain Sci245939–73; discussion 973-1031.
  * O’Reilly . (2013) o2013dissociableO’Reilly, JX., Schüffelgen, U., Cuell, SF., Behrens, TE., Mars, RB. Rushworth, MF.  2013\.  Dissociable effects of surprise and model update in parietal and anterior cingulate cortex Dissociable effects of surprise and model update in parietal and anterior cingulate cortex.  Proceedings of the National Academy of Sciences201305373.
  * O’Regan . (2005) o2005sensoryO’Regan, JK., Myin, E. Noë, A.  2005\.  Sensory consciousness explained (better) in terms of corporality and alerting capacity Sensory consciousness explained (better) in terms of corporality and alerting capacity.  Phenomenology and the Cognitive Sciences44369–387.
  * Panksepp (1998) Panksepp1998AffectiveNeurosciencetheFoundationsofHumanandAnimalEmotionsPanksepp, J.  1998\.  Affective neuroscience: the foundations of human and animal emotions Affective neuroscience: the foundations of human and animal emotions.  OxfordOxford Unversity Press.
  * Passingham Wise (2012) PassinghamWise2012TheneurobiologyoftheprefrontalcortexanatomyevolutionandtheoriginofinsightPassingham, RE. Wise, SP.  2012\.  The neurobiology of the prefrontal cortex: anatomy, evolution, and the origin of insight The neurobiology of the prefrontal cortex: anatomy, evolution, and the origin of insight ( 50).  OxfordOxford University Press.
  * Paus (2001) Paus2001PrimateanteriorcingulatecortexwheremotorcontroldriveandcognitioninterfacePaus, T.  2001Jun.  Primate anterior cingulate cortex: where motor control, drive and cognition interface. Primate anterior cingulate cortex: where motor control, drive and cognition interface.  Nat Rev Neurosci26417–424.
  * Pennartz (2017) pennartz2017consciousnessPennartz, CM.  2017\.  Consciousness, representation, action: the importance of being goal-directed Consciousness, representation, action: the importance of being goal-directed.  Trends in cognitive sciences.
  * Pezzulo . (2013) PezzuloRigoliChersi2013ThemixedinstrumentalcontrollerusingvalueofinformationtocombinehabitualchoiceandmentalsimulationPezzulo, G., Rigoli, F. Chersi, F.  2013\.  The mixed instrumental controller: using value of information to combine habitual choice and mental simulation. The mixed instrumental controller: using value of information to combine habitual choice and mental simulation.  Front Psychol492.
  * Pfeiffer Foster (2013) pfeiffer2013hippocampalPfeiffer, BE. Foster, DJ.  2013\.  Hippocampal place-cell sequences depict future paths to remembered goals Hippocampal place-cell sequences depict future paths to remembered goals.  Nature497744774.
  * Posner Petersen (1990) PosnerPetersen1990TheattentionsystemofthehumanbrainPosner, MI. Petersen, SE.  1990\.  The attention system of the human brain. The attention system of the human brain.  Annu Rev Neurosci1325–42.
  * Raffone . (2014) RaffoneSrinivasanvanLeeuwen2014PerceptualawarenessanditsneuralbasisbridgingexperimentalandtheoreticalparadigmsRaffone, A., Srinivasan, N. van Leeuwen, C.  2014May.  Perceptual awareness and its neural basis: bridging experimental and theoretical paradigms. Perceptual awareness and its neural basis: bridging experimental and theoretical paradigms.  Philos Trans R Soc Lond B Biol Sci369164120130203.
  * Raffone . (2015) RaffoneSrinivasanvanLeeuwen2015RapidswitchingandcomplementaryevidenceaccumulationenableflexibilityofanallornoneglobalworkspaceforcontrolofattentionalandconsciousprocessingareplytoWybleetalRaffone, A., Srinivasan, N. van Leeuwen, C.  2015Feb.  Rapid switching and complementary evidence accumulation enable flexibility of an all-or-none global workspace for control of attentional and conscious processing: a reply to Wyble et al. Rapid switching and complementary evidence accumulation enable flexibility of an all-or-none global workspace for control of attentional and conscious processing: a reply to wyble et al.  Philos Trans R Soc Lond B Biol Sci370166120140315.
  * Redgrave . (1999) redgrave1999basalRedgrave, P., Prescott, TJ. Gurney, K.  1999\.  The basal ganglia: a vertebrate solution to the selection problem? The basal ganglia: a vertebrate solution to the selection problem?  Neuroscience8941009–1023.
  * Ressler (2004) ressler2004rewardsRessler, N.  2004\.  Rewards and punishments, goal-directed behavior and consciousness Rewards and punishments, goal-directed behavior and consciousness.  Neuroscience & Biobehavioral Reviews28127–39.
  * Rizzolatti . (1998) RizzolattiLuppinoMatelli1998TheorganizationofthecorticalmotorsystemnewconceptsRizzolatti, G., Luppino, G. Matelli, M.  199804\.  The organization of the cortical motor system: new concepts. The organization of the cortical motor system: new concepts.  Electroencephalography and clinical neurophysiology1064283–96.
  * Rizzolatti . (1987) RizzolattiRiggioDascolaUmilta1987ReorientingattentionacrossthehorizontalandverticalmeridiansevidenceinfavorofapremotortheoryofattentionRizzolatti, G., Riggio, L., Dascola, I. Umiltá, C.  1987\.  Reorienting attention across the horizontal and vertical meridians: evidence in favor of a premotor theory of attention Reorienting attention across the horizontal and vertical meridians: evidence in favor of a premotor theory of attention.  Neuropsychologia25131–40.
  * Robinson . (2014) RobinsonCalamiaGlaescherBrussTranel2014NeuroanatomicalcorrelatesofexecutivefunctionsaneuropsychologicalapproachusingtheEXAMINERbatteryRobinson, H., Calamia, M., Gläscher, J., Bruss, J. Tranel, D.  2014Jan.  Neuroanatomical correlates of executive functions: a neuropsychological approach using the EXAMINER battery. Neuroanatomical correlates of executive functions: a neuropsychological approach using the examiner battery.  J Int Neuropsychol Soc20152–63.
  * Rossano (2003) Rossano2003ExpertiseandtheevolutionofconsciousnessRossano, MJ.  2003Oct.  Expertise and the evolution of consciousness. Expertise and the evolution of consciousness.  Cognition893207–236.
  * Rounis Humphreys (2015) RounisHumphreys2015LimbapraxiaandtheaffordancecompetitionhypothesisRounis, E. Humphreys, G.  2015\.  Limb apraxia and the ”affordance competition hypothesis”. Limb apraxia and the ”affordance competition hypothesis”.  Front Hum Neurosci9429.
  * Roy . (2012) RoyShohamyWager2012VentromedialprefrontalsubcorticalsystemsandthegenerationofaffectivemeaningRoy, M., Shohamy, D. Wager, TD.  2012Mar.  Ventromedial prefrontal-subcortical systems and the generation of affective meaning. Ventromedial prefrontal-subcortical systems and the generation of affective meaning.  Trends Cogn Sci163147–156.
  * Russell Norvig (2016) RussellNorvig2016ArtificialIntelligenceAModernApproachRussell, SJ. Norvig, P.  2016\.  Artificial Intelligence: A Modern Approach Artificial intelligence: A modern approach (Third Edition ).  Harlow, UKPearson Education.
  * Santucci . (2016) SantucciBaldassarreMirolli2016GRAILaGoalDiscoveringRoboticArchitectureforIntrinsicallyMotivatedLearningSantucci, VG., Baldassarre, G. Mirolli, M.  2016\.  GRAIL: A goal-discovering robotic architecture for intrinsically-motivated learning Grail: A goal-discovering robotic architecture for intrinsically-motivated learning.  IEEE Transactions on Cognitive and Developmental Systems83214–231.
  * Schooler . (2011) schooler2011metaSchooler, JW., Smallwood, J., Christoff, K., Handy, TC., Reichle, ED. Sayette, MA.  2011\.  Meta-awareness, perceptual decoupling and the wandering mind Meta-awareness, perceptual decoupling and the wandering mind.  Trends in cognitive sciences157319–326.
  * Schultz (2002) Schultz2002GettingformalwithSchultz, W.  200210\.  Getting formal with dopamine and reward Getting formal with dopamine and reward.  Neuron362241–263.
  * Searle (2004) Searle2004Searle, JR.  2004\.  Mind: a brief introduction Mind: a brief introduction.  Oxford University Press.
  * Seepanomwan . (2015) SeepanomwanCaligioreCangelosiBaldassarre2015GeneralizationdecisionmakingandembodimenteffectsinmentalrotationaneuroroboticarchitecturetestedwithahumanoidrobotSeepanomwan, K., Caligiore, D., Cangelosi, A. Baldassarre, G.  2015\.  Generalization, decision making, and embodiment effects in mental rotation: a neurorobotic architecture tested with a humanoid robot Generalization, decision making, and embodiment effects in mental rotation: a neurorobotic architecture tested with a humanoid robot.  Neural Networks7231–47.
  * Seth . (2005) Seth2005Seth, AK., Baars, BJ. Edelman, DB.  2005\.  Criteria for consciousness in humans and other mammals Criteria for consciousness in humans and other mammals.  Consciousness and cognition141119–139.
  * Simon (1975) simon1975functionalSimon, HA.  1975\.  The functional equivalence of problem solving skills The functional equivalence of problem solving skills.  Cognitive psychology72268–288.
  * Stumbrys . (2012) stumbrys2012inductionStumbrys, T., Erlacher, D., Schädlich, M. Schredl, M.  2012\.  Induction of lucid dreams: A systematic review of evidence Induction of lucid dreams: A systematic review of evidence.  Consciousness and Cognition2131456–1475.
  * R. Sutton (1990) Sutton1990IntegratedarchitecturesforlearningplanningandreactingbasedonapproximatingdynamicprogrammingSutton, R.  1990\.  Integrated architectures for learning, planning, and reacting based on approximating dynamic programming Integrated architectures for learning, planning, and reacting based on approximating dynamic programming.  Proceedings of the seventh international conference on machine learning Proceedings of the seventh international conference on machine learning ( 216, 216–224).
  * RS. Sutton Barto (1998) sutton1998introductionSutton, RS. Barto, AG.  1998\.  Introduction to reinforcement learning Introduction to reinforcement learning ( 135).  MIT press Cambridge.
  * Telles-Correia . (2015) telles2015hallucinationsTelles-Correia, D., Moreira, AL. Goncalves, JS.  2015\.  Hallucinations and related concepts-their conceptual background Hallucinations and related concepts-their conceptual background.  Frontiers in psychology6991.
  * Thill . (2013) ThillCaligioreBorghiZiemkeBaldassarre2013TheoriesandcomputationalmodelsofaffordanceandmirrorsystemsAnintegrativereviewThill, S., Caligiore, D., Borghi, AM., Ziemke, T. Baldassarre, G.  2013\.  Theories and computational models of affordance and mirror systems: An integrative review Theories and computational models of affordance and mirror systems: An integrative review.  Neuroscience and Biobehavioral Reviews37491-521.
  * Tononi (2008) Tononi2008ConsciousnessasintegratedinformationaprovisionalmanifestoTononi, G.  2008Dec.  Consciousness as integrated information: a provisional manifesto. Consciousness as integrated information: a provisional manifesto.  Biol Bull2153216–242.
  * Tononi . (2016) TononiBolyMassiminiKoch2016IntegratedinformationtheoryfromconsciousnesstoitsphysicalsubstrateTononi, G., Boly, M., Massimini, M. Koch, C.  2016Jul.  Integrated information theory: from consciousness to its physical substrate. Integrated information theory: from consciousness to its physical substrate.  Nat Rev Neurosci177450–461.
  * Treisman Gelade (1980) treisman1980featureTreisman, AM. Gelade, G.  1980\.  A feature-integration theory of attention A feature-integration theory of attention.  Cognitive psychology12197–136.
  * Tsujimoto . (2011) tsujimoto2011frontalTsujimoto, S., Genovesio, A. Wise, SP.  2011\.  Frontal pole cortex: encoding ends at the end of the endbrain Frontal pole cortex: encoding ends at the end of the endbrain.  Trends in cognitive sciences154169–176.
  * Vaswani . (2017) VaswaniShazeerParmarUszkoreitJonesGomezKaiserPolosukhin2017AttentionIsAllYouNeedVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, AN.Polosukhin, I.  2017\.  Attention Is All You Need Attention is all you need.  arXiv preprint arXiv:1706.03762.
  * Verdejo-García . (2006) VerdejoGarciaPerezGarciaBechara2006EmotiondecisionmakingandsubstancedependenceasomaticmarkermodelofaddictionVerdejo-García, A., Pérez-García, M. Bechara, A.  2006Jan.  Emotion, decision-making and substance dependence: a somatic-marker model of addiction. Emotion, decision-making and substance dependence: a somatic-marker model of addiction.  Curr Neuropharmacol4117–31.
  * Waltz . (1999) waltz1999systemWaltz, JA., Knowlton, BJ., Holyoak, KJ., Boone, KB., Mishkin, FS., de Menezes Santos, M.Miller, BL.  1999\.  A system for relational reasoning in human prefrontal cortex A system for relational reasoning in human prefrontal cortex.  Psychological science102119–125.
  * Wayne . (2018) WayneHungAmosMirzaAhujaGrabskaBarwinskaRaeMirowskiLeiboSantoroGemiciReynoldsHarleyAbramsonMohamedRezendeSaxtonCainHillierSilverKavukcuogluBotvinickHassabisLillicrap2018UnsupervisedPredictiveMemoryinaGoalDirectedAgentWayne, G., Hung, CC., Amos, D., Mirza, M., Ahuja, A., Grabska-Barwinska, A.Lillicrap, T.  2018\.  Unsupervised Predictive Memory in a Goal-Directed Agent Unsupervised predictive memory in a goal-directed agent.  arXiv.
  * White (1959) WHITE1959MotivationreconsideredtheconceptofcompetenceWhite, RW.  1959Sep.  Motivation reconsidered: the concept of competence. Motivation reconsidered: the concept of competence.  Psychol Rev66297–333.
  * Wolters Raffone (2008) WoltersRaffone2008CoherenceandrecurrencymaintenancecontrolandintegrationinworkingmemoryWolters, G. Raffone, A.  2008Mar.  Coherence and recurrency: maintenance, control and integration in working memory. Coherence and recurrency: maintenance, control and integration in working memory.  Cogn Process911–17.
  * Yin Knowlton (2006) YinKnowlton2006TheroleofthebasalgangliainhabitformationYin, HH. Knowlton, BJ.  200606\.  The role of the basal ganglia in habit formation The role of the basal ganglia in habit formation.  Nature Reviews Neuroscience76464–476.