# Will we ever have Conscious Machines?

Patrick Krauss  Neuroscience Lab, Experimental Otolaryngology, University
Hospital Erlangen, Germany Cognitive Computational Neuroscience Group at the
Chair of English Philology and Linguistics, Department of English and American
Studies, Friedrich-Alexander University Erlangen-Nürnberg (FAU), Germany
Andreas Maier  Chair of Pattern Recognition, Friedrich-Alexander University
Erlangen-Nürnberg (FAU), Germany

###### Abstract

The question of whether artificial beings or machines could become self-aware
or consciousness has been a philosophical question for centuries. The main
problem is that self-awareness cannot be observed from an outside perspective
and the distinction of whether something is really self-aware or merely a
clever program that pretends to do so cannot be answered without access to
accurate knowledge about the mechanism’s inner workings. We review the current
state-of-the-art regarding these developments and investigate common machine
learning approaches with respect to their potential ability to become self-
aware. We realise that many important algorithmic steps towards machines with
a core consciousness have already been devised. For human-level intelligence,
however, many additional techniques have to be discovered.

Corresponding author:
Prof. Andreas Maier
Chair of Pattern Recognition
Friedrich-Alexander University of Erlangen-Nürnberg (FAU), Germany
E-Mail<EMAIL_ADDRESS>

Keywords:
machine consciousness, artificial intelligence, theories of consciousness

## Introduction

The question of understanding consciousness is in the focus of philosophers
and researchers for more than two millennia. Insights range broadly from
_“Ignorabimus” – “We will never know.”_ 111With this simple statement, Emil du
Bois-Reymond concluded his talk on the limits of scientific knowledge about
the relation of brain processes and subjective experience at the 45th annual
meeting of German naturalists and physicians in 1872. to mechanistic ideas
with the aim to construct artificial consciousness following Richard Feynman’s
famous words _“What I cannot create, I do not understand.”_ 222Richard Feynman
left these words on his blackboard in 1988 at the time of his death as a final
message to the world..

The major issue that precludes the analysis of consciousness is its
subjectivity. Our mind is able to feel and process our own conscious states.
By induction, we are also able to ascribe conscious processing to other human
beings. However, once we try to imagine to be another species, as Nagel
describes in his seminal work “What is it like to be a Bat?”[1], we
immediately fail to follow such experience consciously.

Another significant issue is that we are not able to determine consciousness
by means of behavioural observations as Searle demonstrates in his thought
experiment [2]. Searle describes a room that we cannot enter. One can pass
messages written in Chinese to the room and the room returns messages to the
outside world. All the messages and questions passed to the room are answered
correctly as a Chinese person would. A first conclusion would be that there is
somebody in the “Chinese Room” who speaks Chinese and answers the questions.
However, the person in the room could also have simply access to a large
dictionary that contains all possible questions and the respective answers.
When we are not able to understand how the information is actually processed,
we will never be able to determine whether a system is conscious or not.

In this article, we want to explore these and different thoughts in literature
to address the problem of consciousness. We therefore revisit works in
philosophy, neuroscience, artificial intelligence, and machine learning.
Following the paradigm of cognitive computational neuroscience [3], we present
how the convergence of these fields could potentially also lead to new
insights regarding consciousness.

## The Philosophical Perspective

More than two thousand years ago, Aristotle was convinced that only humans are
endowed with a rational soul. All animals, however, live only with the
instincts necessary for survival, like biological automata. Along the same
line, in the statement “Cogito ergo sum” also Descartes realised being self-
aware is reserved for human beings. In his view, this insight is fundamental
for any philosophical approach [4].

Modern philosophy went on to differentiate the problem into an easy and a hard
problem. While the “easy problem” is to explain its function, dynamics, and
structure, the “hard problem of consciousness” [5] is summarised in the
Internet Encyclopedia of Philosophy [6] as:

_“The hard problem of consciousness is the problem of explaining why any
physical state is conscious rather than nonconscious. It is the problem of
explaining why there is “something it is like” for a subject in conscious
experience, why conscious mental states “light up” and directly appear to the
subject.”_

In order to avoid confusion some scientists prefer to speak of “conscious
experience” or only “experience” instead of consciousness [5]. As already
noted, the key problem of deriving models of conscious events is that they can
only be perceived subjectively. As such it is difficult to encode such an
experience in a way that it can be recreated by others. This gives rise to the
so-called “qualia problem” [7] as we can never be sure, e.g. that the color
red consciously looks the same to another person. Extension of this line of
thought leads again to Nagel’s thought experiment [1].

According to [6], approaches to tackle the problem from a philosophical point
of view are very numerous, but none of them can be considered to be
exhaustive:

  * •

Eliminativism [8] demonstrates that the mind is fully functional without the
experience of consciousness. Being nonfunctional, consciousness can be
neglected.

  * •

The view of strong reductionism proposes that consciousness can be
deconstructed into simpler parts and be explained by functional processes.
Such considerations gave rise to the global work space theory [9, 10, 11] or
integrated information theory [12, 13] in neuroscience. The main critique of
this view, is that any mechanistic solution to consciousness that is not fully
understood will only mimic true consciousness, i.e., one could construct
something that appears conscious that simply isn’t as the Chinese Room
argument demonstrates [2].

  * •

Mysterianism proposes that the question of consciousness cannot be tackled
with scientific methods. Therefore any investigation is in vain and the
explanatory gap cannot be closed [14].

  * •

In Dualism the problem is tackled as consciousness being metaphysical that is
independent of physical substance [4]. Modern versions of Dualism exist, but
virtually all of them require to reject that our world can be fully described
by physical principles. Recently, Penrose and Hammeroff tried to close this
gap using quantum theory [15, 16]. We dedicate a closer description of this
view in a later section of this article.

  * •

Assuming that metaphysical world and physical world simply do not interact
does not require to reject physics and gives rise to Epiphenomenalism [17].

There are further theories and approaches to address the hard problem of
consciousness that we do not want to detail here. To the interested reader, we
recommend to study the Internet Encyclopedia of Philosophy [6] as further
reading into the topic.

In conclusion, we observe that a major disadvantage of exploring the subject
of consciousness by philosophical means is that we are never able to explore
the inside of the Chinese Room. Thought alone will not be able to open the
black box. Neuroscience, however, offers means to explore the inside by
various means of measurement and might offer suitable means to address the
problem.

## Consciousness in Neuroscience

### Historical Overview

In 1924, Hans Berger recorded, for the first time, electrical brain activity
using electroencephalography (EEG) [18]. This breakthrough enabled the
investigation of different mental states by means of electrophysiology, e.g.
during perception [19] or during sleep [20]. The theory of cell assemblies,
proposed by Donald Hebb in 1949 [21], marked the starting point for the
scientific investigation of neural networks as the biological basis for
perception, cognition, memory, and action. In 1965, Gazzaniga demonstrated
that dissecting the corpus callosum which connects the two brain hemispheres
with each other results in a split of consciousness [22, 23]. Almost ten years
later, Weiskrantz et al. discovered a phenomenon for which the term
“blindsight” has been coined: following lesions in the occipital cortex,
humans loose the ability to consciously perceive, but are still able to react
to visual stimuli [24, 25]. In 1983, Libet demonstrated that voluntary acts
are preceded by electrophysiological readiness potentials that have their
maximum at about $550\,ms$ before the voluntary behavior [26]. He concluded
that the role of conscious processing might not be to initiate a specific
voluntary act but rather to select and control volitional outcome [27]. In
contrast to the above mentioned philosophical tradition from Aristotle to
Descartes that consciousness is a phenomenon that is exclusively reserved for
humans, in contemporary neuroscience most researchers tend to regard
consciousness as a gradual phenomenon, which in principle also occurs in
animals [28], and several main theories of how consciousness emerges have been
proposed so far.

### Neural Correlates of Consciousness

Based on Singer’s observation that high-frequency oscillatory responses in the
feline visual cortex exhibit inter-columnar and inter-hemispheric
synchronization which reflects global stimulus properties [29, 30, 31] and
might therefore be the solution for the so called “binding problem” [32],
Crick and Koch suggested Gamma frequency oscillations to play a key role in
the emergence of consciousness [33]. Koch further developed this idea and
investigated neural correlates of consciousness in humans [34, 35]. He argued
that activity in the primary visual cortex, for instance, is necessary but not
sufficient for conscious perception, since activity in areas of extrastriate
visual cortex correlates more closely with visual perception, and damage to
these areas can selectively impair the ability to perceive particular features
of stimuli [36]. Furthermore, he discussed the possibility that the timing or
synchronization of neural activity might correlate with awareness, rather than
simply the overall level of spiking [36]. A finding which is supported by
recent neuroimaging studies of visual evoked activity in parietal and
prefrontal cortex areas [37]. Based on these findings, Koch and Crick provided
a framework for consciousness, where they proposed a coherent scheme to
explain the neural activation of visual consciousness as competing cellular
clusters [38]. Finally, the concept of neural correlates of consciousness has
been further extended to an index of consciousness based on brain complexity
[39], which is independent of sensory processing and behavior [40], and might
be used to quantify consciousness in comatose patients [41].

### Consciousness as a Computational Phenomenon

Motivated by the aforementioned findings concerning the neural correlates of
consciousness, Tononi introduced the concept of integrated information, which
according to his “Integrated Information Theory of Consciousness” plays a key
role in the emergence of consciousness [12, 13]. This theory represents one of
two major theories of contemporary research in consciousness. According to
this theory, the quality or content of consciousness is identical to the form
of the conceptual structure specified by the physical substrates of
consciousness, and the quantity or level of consciousness corresponds to its
irreducibility, which is defined as integrated information [42].

Tegmark generalized Tononi’s framework even further from neural-network-based
consciousness to arbitrary quantum systems. He proposed that consciousness can
be understood as a state of matter with distinctive information processing
abilities, which he calls “perceptronium”, and investigates interesting links
to error-correcting codes and condensed matter criticality [43, 44].

Even though, there is large consensus that consciousness can be understood as
a computational phenomenon [45, 46, 47, 48], there is dissent about which is
the appropriate level of granularity of description and modeling [3]. Penrose
and Hameroff even proposed that certain features of quantum coherence could
explain enigmatic aspects of consciousness, and that consciousness emerges
from brain activities linked to fundamental ripples in spacetime geometry. In
particular, according to their model of orchestrated objective reduction (Orch
OR), they hypothesize that the brain is a kind of quantum computer, performing
quantum computations in the microtubeles, which are cylindrical protein
lattices of the neurons’ cytoskeleton [15, 49, 50].

However, Tegmark and Koch argue, that the brain can be understood within a
purely neurobiological framework, without invoking any quantum-mechanical
properties: quantum computations which seek to exploit the parallelism
inherent in entanglement, require that the qubits are well isolated from the
rest of the system, whereas on the other hand, coupling the system to the
external world is necessary for the input, the control, and the output of the
computations. Due to the wet and warm nature of the brain, all these
operations introduce noise into the computation, which causes decoherence of
the quantum states, and thus makes quantum computations impossible.
Furthermore, they argue that the molecular machines of the nervous system,
such as the pre- and post-synaptic receptors, are so large that they can be
treated as classical rather than quantum systems, i.e. that there is nothing
fundamentally wrong with the current classical approach to neural network
simulations [51, 52, 53].

Figure 1: The Global Workspace emerges by connecting different brain areas
according to Dehaene.

### The Global Workspace Theory of Consciousness

In the 1990s, Baars introduced the concept of a virtual “Global Workspace”
that emerges by connecting different brain areas (Figure 1) to describe
consciousness [9, 10, 11, 54]. This idea was taken up and further developed by
Dehaene [55, 56, 57, 58, 59, 60]. Today, besides the Integrated Information
Theory, the Global Workspace Theory represents the second major theory of
consciousness, being intensively discussed in the field of cognitive
neuroscience. Based on the implications of this theory, i.e., that
consciousness arises from specific types of information-processing
computations, which are physically realized by the hardware of the brain,
Dehaene argues that a machine endowed with these processing abilities _“would
behave as though it were conscious; for instance, it would know that it is
seeing something, would express confidence in it,would report it to others,
could suffer hallucinations when its monitoring mechanisms break down, and may
even experience the same perceptual illusions as humans”_ [61].

Figure 2: Simplified view of Damasio’s model of consciousness: The protoself
processes emotions and sensory input unconsciously. Core consciousness arises
from the protoself which allows to put the itself into relation. Projections
of emotions give rise to higher-order feelings. With access to memory and
extended functions such as language processing the extended consciousness
emerges.

### Damasio’s Model of Consciousness

Damasio’s model of consciousness was initially published in his popular
science book “The feeling of what happens” [62]. Later Damasio also published
the central ideas in peer-reviewed scientific literature [63]. With the ideas
being published first in a popular science book, most publications on
consciousness neglect his contributions. However, we believe that his thoughts
deserve more attention. Therefore, we want to introduce his ideas quickly in
this section.

The main idea in Damasio’s model is to relate consciousness to the ability to
identify one’s self in the world and to be able to put the self in relation
with the world. However, a formal definition is more complex and requires the
introduction of several concepts first.

He introduces three levels of conscious processing:

  * •

The fundamental protoself does not possess the ability to recognize itself. It
is a mere processing chain that reacts to inputs and stimuli like an
automaton, completely non-conscious. As such any animal has a protoself
according to this definition. However, also more advanced lifeforms including
humans exhibit this kind of self.

  * •

A second stage of consciousness is the core consciousness. It is able to
anticipate reactions in its environment and adapts to them. Furthermore, it is
able to recognise itself and its parts in its own image of the world. This
enables it to anticipate and to react to the world. However, core
consciousness is also volatile and not able to persist for hours to form
complex plans.

In contrast to many philosophical approaches, core consciousness does not
require to represent representations of the world in words or language. In
fact, Damasio believes that progress in understanding conscious processing has
been impeded by dependence on words and language.

  * •

The extended consciousness enables human-like interaction with the world. It
builds on top of core consciousness and enables further functions such as
access to memory in order to create a autobiographic self. Also being able to
process words and language falls into the category extended consciousness and
can be interpreted as a form of serialisation of conscious images and states.

In Damasio’s theory emotion and feelings are fundamental concepts [64]. In
particular Damasio differentiates emotions from feelings. Emotions are direct
signals that indicate a positive or negative state of the (proto-)self.
Feelings emerge only in conjunction with images of the world and can be
interpreted as a second-order emotion that is derived from the world
representation and future possible events in the world. Both are crucial for
the emergence of consciousness. Fig. 2 schematically puts the described terms
in relation.

After having defined the above concepts, Damasio now goes on to attempt and
describe a model of (core) consciousness. In his theory, consciousness does
not merely emerge from the ability to identify oneself in the world or an
image of the world. For conscious processing, additionally feeling oneself in
the sense of desiring to exist is required. Hence, he postulates a feeling,
i.e., a derived second-order emotion, between the protoself and its internal
representation of the world. Conscious beings as such want to identify oneself
in the world and want to exist. From an evolutionary perspective as he argues,
this is possibly a mechanism to enforce self-preservation.

In the context of this article, Damasio’s theory is interesting for two major
reasons. On the one hand, it describes a biologically plausible model of
consciousness, as he locates all stages of consciousness to structures in the
brain and associates them to the respective function. On the other hand,
Damasio describes a mechanistic model that can at least in theory be
completely implemented as a computer program.

Hence, we can conclude that neuroscience is able to describe fundamental
processes in the brain that give rise to complex phenomena such as
consciousness. However, the means of observation in neuroscience are
insufficient. Neither EEG nor fMRI have a temporal and spatial resolution that
is even close enough to observe what is happening in the brain in-vivo. At
this point, the recent massive progress in artificial intelligence and machine
learning comes to our attention and will be the focus of our next section.

## Consciousness in Machine Learning and AI

In artificial intelligence (AI) numerous theories of consciousness exist [65,
66]. Implementations often focus on the global work space theory with only
limited learning capabilities [67], i.e. most of the consciousness is hard-
coded and not trainable [68]. An exception is the theory by van Hateren which
relates the consciousness to close to simultaneous forward and backward
processing in the brain [69]. Yet, algorithms that were investigated so far
made use of a global work space and mechanistic hard-coded models of
consciousness. Following this line, research on minds and consciousness rather
focuses on representation than on actual self-awareness [70]. Although
representation will be important to create human-like minds and general
intelligence [71, 72, 73], a key factor to become conscious is the ability to
identify a self in one’s environment [61]. A major drawback of pure
mechanistic methods, however, is that the complete knowledge on the model of
consciousness is required in order to realise and implement them. As such, in
order to develop these models to higher forms such as Damasio’s extended
consciousness, a complete mechanistic model of the entire brain including all
connections is required.

### Consciousness in Machine Learning

A possible solution to this problem is machine learning, as it allows to form
and train complex models. The topic of consciousness, however, is neglected in
the field to a large extent. On the one hand, this is because of the concerns
that the brain and consciousness will never be successfully simulated in a
computer system [74, 16]. On the other hand, consciousness is considered to be
an extremely hard problem and current results in AI are still meager [75].

The state-of-the-art in machine learning instead focuses on supervised and
unsupervised learning techniques [76]. Another important research direction is
reinforcement learning [77] that aims at learning of suitable actions for an
agent in a given environment. As consciousness is often associated with an
embodiment, reinforcement learning is likely to be important for modelling of
consciousness.

The earliest work that the authors are aware of attempting to model and create
agents that learn their own representation of the world entirely using machine
learning date back to the early 1990’s. Already in 1990, Schmidhuber proposed
a model for dynamic reinforcement learning in reactive environments [78] and
found evidence for self-awareness in 1991 [79]. The model follows the idea of
a global work space. In particular, future rewards and inputs are predicted
using a world model. Yet, Schmidhuber was missing a theory on how to analyse
intelligence and consciousness in this approach. Similar to Tononi [13],
Schmidhuber followed the idea of compressed neural representation.
Interestingly, compression is also key to inductive reasoning, i.e., learning
from few examples which we typically deem as intelligent behaviour.

Solomonoff’s Universal Theory of Inductive Inference [80] gives a theoretic
framework to inductive reasoning. It combines information and compression
theory and results in a formalisation of Occam’s razor preferring simple
models over complex ones, as simple models are more likely from an information
theoretic point of view [81].

Under Schmidhuber’s supervision, Hutter applied Solomonoff’s theory to machine
learning to form a theory of Universal Artificial Intelligence [82]. In his
theory, intelligent behaviour stems from efficient compression of inputs, e.g.
from an environment, such that predictions and actions are performed
optimally. Again, models capable of describing a global work space play an
important role.

Maguire et al. further expand on this concept to extend Solomonoff’s and
Hutter’s theories to also describe consciousness. Following the ideas of
Tononi and Koch [36] consciousness is understood as data compression, i.e. the
optimal integration of information [81]. The actual consciousness emerges from
binding of information and is inherently complex. As such, consciousness can
also not be deconstructed into mechanical sub-components, as the decomposition
would destroy the sophisticated data compression. Maguire et al. even provide
a mathematical proof to demonstrate that consciousness is either integrated
and therefore cannot be decomposed or there is an explicit mechanistic way of
modelling and describing consciousness [81].

Based on the extreme success of deep learning [83], also several scientists
observed similarities in neuroscience and machine learning. In particular,
deep learning allows to build complex models that are hard to analyse and
interpret at the benefit of making complex predictions. As such both fields
are likely to benefit each other in the ability to understand and interpret
complex dynamic systems [84, 85, 86, 3, 87, 88, 89]. In particular, hard-
wiring following biological ideas might help to reduce the search space
dramatically [90]. This is in line with recent theoretical considerations in
machine learning as prior knowledge allows to reduce maximal error bounds
[91]. Both fields can benefit from these ideas as recent discoveries of e.g.
successor representation show [92, 93, 94]. Several scientists believe that
extension of this approach to social, cultural, economic, and political
sciences will create even more synergy resulting in the field of machine
behaviour [95].

Figure 3: Overview on typical architectures in machine learning. Hard-coded
paths, i.e. recurrent connections, are indicated by solid arrows, trainable
connections by dashed arrows, and training losses are indicated by red double
arrows. While archtiectures A-E do not match theories of consciousness,
architectures F-H implement theories by Schmidhuber and Damasio.

## Can Consciousness emerge in Machine Learning Systems?

After having reviewed philosophy, neuroscience, and the state-of-the-art in AI
and machine learning, we can now analyse the most important concepts in the
field of machine and deep learning to assess whether they have the potential
to create consciousness following one of the previous theories. In particular,
we focus on the ability of the system to represent a symbol of self and how
this self-awareness is constructed, as all theories of consciousness require
at least experiencing the self.

We show a brief overview on important architectures in machine learning in
Fig. 3. We denote hard-wired connections as solid arrows to indicate recurrent
modes of specific network parts. Dashed lines indicate trainable connections.
They could be implemented as a single feed-forward layer, i.e., a universal
function approximator. Without loss of generality, they could also be
implemented by other deep feed-forward architectures [96] and could thus be
inherently complex. Red double arrows indicate training losses to adjust the
trainable weights of the dashed arrows.

Fig. 3 A shows a simple feed-forward architecture that requires external
labelled training data $\bf{y}^{*}$ to adjust its trainable weights given
input $\bf{x}$ to produce output $\bf y$. Fig. 3 B shows a similar setup, for
the recurrent case. Note that we only indicate a simple recurrent cell here
with time-dependent state $\bf h_{t}$. Without loss of generality, this could
also be realised using gated recurrent units [97] or long short term memory
cells [98]. Both models fall into the category of supervised learning.
Consciousness is not supported by any of the theories presented so far.

In Fig. 3 C, we introduce the concept of “Emotion” following Damasio’s
wording. In machine learning terms, this reflects an additional loss. Now, the
system receives an additional input $\bf e$ that is associated to a valence or
value. Without loss of generality, we can assume positive entries in $\bf e$
to be associated to desirable states for the system and negative values to
undesirable states. As such, training using $\bf e$ falls into the category of
reinforcement learning that aims at maximizing future values of $\bf e$. In
order to model competing interests and saturation effects, e.g. a full battery
does not need to be charged further, we introduce a reference $\bf e^{*}$ that
is able to model such effects. Note that we deem the system to be able to
predict the expected future reward $\bf e^{\prime}$ from its current state
$\bf h_{t}$ following a deep Q learning paradigm [99]. Here we use $\bf
e^{\prime}$ and $\bf e^{*}$ to construct a trainable reinforcement loss, to be
able to learn from low-level rewards/emotions $\bf e$. Although being able to
learn, the system still needs supervision to train the weights producing
output $\bf y$ using a reference. This setup also does not match any theory of
consciousness so far.

As self-awareness is a requirement for base consciousness, we deem a world
model to be necessary. Such an approach his shown in Fig. 3 D. Given from the
produced output $\bf y$, the world model is used to create an expected future
input $\bf x^{\prime}$. In the figure, we chose a recurrent model capturing
the state of the world in $\bf w_{t}$ that is independent of the internal
state of the actual agent $\bf h_{t}$. To gain consciousness, this model
misses at least a link from internal to external state and “emotions” that
would guide future decisions. Adding low-level rewards/emotions results in
Fig. 3 E. Again world and self are disconnected inhibiting self-representation
and self-discovery. Approaches like this are already being explored for video
game control [100].

With a world-model being present, we are now able to predict future rewards
$\bf e^{\prime\prime}$ that also take into account the state of the world and
the chosen action. As such Fig. 3 F is the first one that would implement a
trainable version of deep Q learning. Development of consciousness is
debatable, as the model does not feature a link between the state of the world
and the state of the agent. If we would add a trainable connection from $\bf
h_{t}$ to $\bf w_{t}$ and vice versa, we would end up with Schmidhuber’s Model
from 1990 [78] for which Schmidhuber found evidence to develop self
representation [79].

Interestingly, Damasio’s descriptions follow a similar line in [62]. We depict
a model implementing Damasio’s core consciousness in Fig. 3 G. As Schmidhuber,
Damasio requires a connection from the world model $\bf w_{t}$ to the body
control system $\bf h_{t}$. However, in his view, consciousness does not
emerge by itself. It is enforced by a “feeling” that is expressed as a loss in
the world of machine learning. As such, the Damasio model of core
consciousness requires a loss that aims at the recovery of the image of the
self in the world model. If this is implemented as a loss, we are able to
express the desire to exist in the world. If implemented merely as trainable
weights, we arrive at the theory of integrated information that creates
consciousness as maximized compression of the world, the self, and their
interaction. Interestingly, these considerations also allow integration of
attention [101] and other concepts of resolving context information in machine
learning. Realised in a biological learning framework, e.g. using
neuromodulators like Dopamin [102], the notion of loss and connection will
disappear and the models of Damasio, Schmidhuber, Tononi, Koch, and Dehane
turn out to be different descriptions of the same principles.

Note that the models of consciousness that we have discussed so far are very
basic. They do not concern language, memory, or other complex multimodal forms
of processing, planning, induction, or representation. Again, we follow
Damasio at this point in Fig. 3 H in which all of these sophisticated
processes are mapped into a block “Memory / Extended Functions”. Note although
we omit these extended functions, we are able to integrate them using
trainable paths. As such, the model of core consciousness acts as a “neural
operating system” that is able to update also higher order functions according
to the needs of the environment. With increasing “extended functions”, the
degree of complexity and “integrated information” rises measurably as also
observed by Casarotto [39].

This brings us back to the original heading of our section: There are clearly
theories that enable modelling and implementation of consciousness in the
machine. On the one hand, they are mechanistic to the extend that they can be
implemented in programming languages and require similar inputs as humans
would do. On the other hand, even the simple models in Fig.3 are already
arbitrarily complex, as every dashed path in the models could be realised by a
deep network. As such also training will be hard. Interestingly, the models
follow a bottom-up strategy such that training and development can be
performed in analogy to biological development and evolution. The models can
be trained and grown to more complex tasks gradually.

## Discussion

Existence of consciousness in the machine is a hot topic of debate. Even with
respect to the simple core consciousness, we observe opinions ranging from
“generally impossible” [103] through “plausible” [61] to “has already been
done” [79]. Obviously, all of the suggested models cannot solve the qualia
problem or the general problem on how to demonstrate whether a system is truly
conscious. All of the emerging systems could merely be mimicking conscious
behaviour without being conscious at all (even Fig. 3 A). Yet as already
discussed by Schmidhuber [79], we would be able to measure correlates of self
recognition similar to neural correlates of consciousness in humans [35] which
could help to understand consciousness in human beings. However, as long as we
have not solved how to provide proof of consciousness in human beings, we will
also fail to do so in machines as the experience of consciousness is merely
subjective.

Koch and Dehaene discussed the theories of global work space and integrated
information as being opposed to each other [103]. In the models found in Fig.
3, we see that both concepts require a strong degree of interconnection. As
such, we do not see why both concepts are fundamentally opposing. A global
work space does not necessarily have to be encoded in decompressed state.
Also, Maguire’s view of integrated information [81] is not necessarily
impossible to implement mechanistically, as we are able to use concepts of
deep learning to train highly integrated processing networks. In fact, as
observed by neuroscience [3], both approaches might support each other
yielding methods to construct and reproduce biological processes in a modular
way. This allows the integration of representation [71] and processing
theories [65] as long as they can be represented in terms of deep learning
compatible operations [91].

In all theories that we touched in this article, the notion of self is
fundamental. Hence, all presented theories of consciousness require embodiment
as basis for consciousness. As such, a body is required for conscious
processing. Also the role of emotion and feelings is vital. Without emotion or
feelings, the system cannot be trained and thus it can not adopt to new
environments and changes of circumstances. In the machine learning inspired
models, we assume that a disconnection between environment and self would
cause a degradation of the system similar to the one that is observed in human
beings in locked-in state [104]. This homeostatsis was also deemed important
by Man et al. [105].

Similar to the problems identified by Nagel, also the proposed mechanistic
machine learning models will not be able to understand “what it is like” to be
a bat. However, the notion of train-/learnable programs and connections or
adapters might offer a solution to explore this in the future. Analogously,
one cannot describe to somebody “what it is like” to play the piano or to
snowboard on expert level unless one really acquires the ability. As such also
the qualia problem persists in machine consciousness. However, we are able to
investigate the actual configuration of the representation in the artificial
neural net offering entirely new mechanisms of insight.

In Damasio’s theory, consciousness is effectively created by a training loss
that causes the system to “want” to be conscious, i.e., ”Cogito ergo sum”
becomes ”Sentio ergo sum”. Comparison between trainable connections after
[78], attention mechanisms [101], and this approach are within the reach of
future machine learning models which will create new evidence for the
discussion of integrated information and global work spaces. In fact,
Schmidhuber has already taken up the work on combination of his early ideas
with modern approaches from deep learning [106, 107].

With models for extended consciousness, even the notion of the Homunculus
[108] can be represented by extension of the self with another self pointer.
In contrast to common rejection of the Homunculus thought experiment, this
recurrent approach can be trained using end-to-end systems comparable to
AlphaGo [109].

Damasio also presents more interesting and important work that is mostly
omitted in this article for brevity. In [62] he also relates structural brain
damage to functional loss of cognitive and conscious processing. Also the
notion of emotion is crucial in a biological sense and is the driving effect
of homeostasis. In [105], Damasio already pointed out that this concept will
be fundamental for self-regulating robotic approaches.

With the ideas of cognitive computational neuroscience [3] and the approaches
detailed above, we will design artificial systems that approach the mechanisms
of biological systems in an iterative manner. With the iterations, the
artificial systems will increase in complexity and similarity to the
biological systems. However, even if we arrive at an artificial system that
performs identical computations and reveals identical behaviour as the
biological system, we will not be able to deem this system as conscious beyond
any doubts. The true challenge in being perceived as conscious will be the
acceptance by human beings and society. As Alan Turing already proposed in his
imitation game in 1950 to decide whether a machine is intelligent or even
conscious [110], the ascription of such by other humans is a critical factor.
For this purpose, Turing’s Test has already been extended to also account for
embodiment [111]. However, such tests are only necessary, but not sufficient
as Gary Marcus pointed out: Rather simple chat bot models are already able to
beat Turing’s Test in some occasions [112]. Hence, requirements for conscious
machines will comprise, the similarity to biological conscious processes, the
ability to convince human beings, and even the machine itself. As such, we
deem it necessary to look as some ethical implications at this point.

## Ethical implications

Being able to create systems that are indistinguishable from conscious beings
that are potentially conscious also raises ethical concerns. First and
foremost, in the transformation from core consciousness to extended
consciousness, the systems gain the ability to link new program routines. As
such systems following such a line of implementation need to be handled with
care and should be experimented on in a contained environment. With the right
choice of embodiment in a virtual machine or in a robotic body, one should be
able to solve such problems.

Of course there are also other ethical concerns, the more we approach human-
like behaviour. A first set of robotic laws has been introduced in Asimov’s
novels [113]. Even Asimov considered the rules problematic as can be seen from
the plot twists in his novels. Aside this, being able to follow the robotic
laws requires the robot to understand the concepts of “humans”, “harm”, and
“self”. Hence, such beings must be conscious. Therefore, tampering with their
memories, emotions, and feelings is also problematic by itself. Being able to
copy and reproduce the same body and mind does not lead to further
simplification of the issue and implies the problem that we have to agree on
ethics and standards of AI soon [114].

## Conclusion

In this article, we reviewed the state-of-the-art theories on consciousness in
philosophy, neuroscience, AI, and machine learning. We find that all three
disciplines need to interact to push research in this direction further.
Interestingly, basic theories of consciousness can be implemented in computer
programs. In particular, deep learning approaches are interesting as they
offer the ability to train deep approximators that are not yet well understood
to construct mechanistic systems of complex neural and cognitive processes. We
reviewed several machine learning architectures and related them to theories
of strong reductionism and found that there are neural network architectures
from which base consciousness could emerge. Yet, there is still a long way to
form human-like extended consciousness.

## Data availability

All data in this publication are publicly available.

## Code availability

All software used in this article is publicly available.

## Acknowledgments

This work was funded by the Deutsche Forschungsgemeinschaft (DFG, German
Research Foundation): grant KR5148/2-1 to PK – project number 436456810, the
Emergent Talents Initiative (ETI) of the University Erlangen-Nuremberg (grant
2019/2-Phil-01 to PK). Furthermore, the research leading to these results has
received funding from the European Research Council (ERC) under the European
Union’s Horizon 2020 research and innovation programme (ERC grant no. 810316).

## Author contributions

PK and AM contributed equally to this work.

## Ethics declarations

### Competing interests

The authors declare no competing financial interests.

## References

  * [1] Thomas Nagel.  What is it like to be a bat?  The philosophical review, 83(4):435–450, 1974.
  * [2] John R Searle.  Minds, brains, and programs.  Behavioral and brain sciences, 3(3):417–424, 1980.
  * [3] Nikolaus Kriegeskorte and Pamela K Douglas.  Cognitive computational neuroscience.  Nature neuroscience, 21(9):1148–1160, 2018.
  * [4] René Descartes.  Meditations on First Philosophy/Meditationes de Prima Philosophia: A Bilingual Edition.  University of Notre Dame Pess, 1990.
  * [5] David J Chalmers.  Facing up to the problem of consciousness.  Journal of consciousness studies, 2(3):200–219, 1995.
  * [6] Josh Weisberg.  The Hard Problem of Consciousness.  https://www.iep.utm.edu/hard-con/.  Accessed: 2020-02-28.
  * [7] Tim Crane.  The origins of qualia.  In History of the mind-body problem, pages 177–202. Routledge, 2012\.
  * [8] Georges Rey.  A question about consciousness.  In Perspectives on mind, pages 5–24. Springer, 1988.
  * [9] James Newman and Bernard J Baars.  A neural attentional model for access to consciousness: A global workspace perspective.  Concepts in Neuroscience, 4(2):255–290, 1993.
  * [10] Bernard J Baars.  A global workspace theory of conscious experience.  Consciousness in philosophy and cognitive neuroscience, pages 149–171, 1994.
  * [11] Bernard J Baars and James Newman.  A neurobiological interpretation of global workspace theory.  Consciousness in philosophy and cognitive neuroscience, pages 211–226, 1994.
  * [12] Giulio Tononi.  An information integration theory of consciousness.  BMC neuroscience, 5(1):42, 2004.
  * [13] Giulio Tononi.  Consciousness as integrated information: a provisional manifesto.  The Biological Bulletin, 215(3):216–242, 2008.
  * [14] Joseph Levine.  Purple haze: The puzzle of consciousness.  Oxford University Press, 2001.
  * [15] Roger Penrose.  Mechanisms, microtubules and the mind.  Journal of Consciousness Studies, 1(2):241–249, 1994.
  * [16] Stuart Hameroff and Roger Penrose.  Consciousness in the universe: A review of the ’orch or’ theory.  Physics of life reviews, 11(1):39–78, 2014.
  * [17] Keith Campbell.  Body and mind.  University of Notre Dame Pess, 1992.
  * [18] Hans Berger.  Über das elektrenkephalogramm des menschen.  DMW-Deutsche Medizinische Wochenschrift, 60(51):1947–1949, 1934\.
  * [19] Patrick Krauss, Claus Metzner, Achim Schilling, Konstantin Tziridis, Maximilian Traxdorf, Andreas Wollbrink, Stefan Rampp, Christo Pantev, and Holger Schulze.  A statistical method for analyzing and comparing spatiotemporal cortical activation patterns.  Scientific reports, 8(1):1–9, 2018.
  * [20] Patrick Krauss, Achim Schilling, Judith Bauer, Konstantin Tziridis, Claus Metzner, Holger Schulze, and Maximilian Traxdorf.  Analysis of multichannel eeg patterns during human sleep: a novel approach.  Frontiers in human neuroscience, 12:121, 2018.
  * [21] Donald O Hebb.  The organization of behavior.  na, 1949.
  * [22] Michael S Gazzaniga, Joseph E Bogen, and Roger W Sperry.  Observations on visual perception after disconnexion of the cerebral hemispheres in man.  Brain, 88(2):221–236, 1965.
  * [23] Michael S Gazzaniga.  Forty-five years of split-brain research and still going strong.  Nature Reviews Neuroscience, 6(8):653–659, 2005.
  * [24] Lawrence Weiskrantz, Elizabeth K Warrington, MD Sanders, and J Marshall.  Visual capacity in the hemianopic field following a restricted occipital ablation.  Brain, 97(4):709–728, 1974.
  * [25] L Weiskrantz and E WARRINGTON.  Blindsight-residual vision following occipital lesions in man and monkey.  Brain Research, 85(1), 1975.
  * [26] Benjamin Libet, Elwood W Wright Jr, and Curtis A Gleason.  Preparation-or intention-to-act, in relation to pre-event potentials recorded at the vertex.  Electroencephalography and clinical Neurophysiology, 56(4):367–372, 1983.
  * [27] Benjamin Libet.  Unconscious cerebral initiative and the role of conscious will in voluntary action.  Behavioral and brain sciences, 8(4):529–539, 1985.
  * [28] Melanie Boly, Anil K Seth, Melanie Wilke, Paul Ingmundson, Bernard Baars, Steven Laureys, David Edelman, and Naotsugu Tsuchiya.  Consciousness in humans and non-human animals: recent advances and future directions.  Frontiers in psychology, 4:625, 2013.
  * [29] Charles M Gray, Peter König, Andreas K Engel, and Wolf Singer.  Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties.  Nature, 338(6213):334–337, 1989.
  * [30] Andreas K Engel, Peter König, Andreas K Kreiter, and Wolf Singer.  Interhemispheric synchronization of oscillatory neuronal responses in cat visual cortex.  Science, pages 1177–1179, 1991.
  * [31] Wolf Singer.  Synchronization of cortical activity and its putative role in information processing and learning.  Annual review of physiology, 55(1):349–374, 1993.
  * [32] Wolf Singer and Charles M Gray.  Visual feature integration and the temporal correlation hypothesis.  Annual review of neuroscience, 18(1):555–586, 1995.
  * [33] Francis Crick and Christof Koch.  Towards a neurobiological theory of consciousness.  In Seminars in the Neurosciences, volume 2, pages 263–275. Saunders Scientific Publications, 1990.
  * [34] Giulio Tononi and Christof Koch.  The neural correlates of consciousness-an update.  Annals of the New York Academy of Sciences, 2008.
  * [35] Christof Koch, Marcello Massimini, Melanie Boly, and Giulio Tononi.  Neural correlates of consciousness: progress and problems.  Nature Reviews Neuroscience, 17(5):307, 2016.
  * [36] Geraint Rees, Gabriel Kreiman, and Christof Koch.  Neural correlates of consciousness in humans.  Nature Reviews Neuroscience, 3(4):261–270, 2002.
  * [37] Melanie Boly, Marcello Massimini, Naotsugu Tsuchiya, Bradley R Postle, Christof Koch, and Giulio Tononi.  Are the neural correlates of consciousness in the front or in the back of the cerebral cortex? clinical and neuroimaging evidence.  Journal of Neuroscience, 37(40):9603–9613, 2017.
  * [38] Francis Crick and Christof Koch.  A framework for consciousness.  Nature neuroscience, 6(2):119–126, 2003.
  * [39] Silvia Casarotto, Angela Comanducci, Mario Rosanova, Simone Sarasso, Matteo Fecchio, Martino Napolitani, Andrea Pigorini, Adenauer G. Casali, Pietro D Trimarchi, Melanie Boly, et al.  Stratification of unresponsive patients by an independently validated index of brain complexity.  Annals of neurology, 80(5):718–729, 2016.
  * [40] Adenauer G Casali, Olivia Gosseries, Mario Rosanova, Mélanie Boly, Simone Sarasso, Karina R Casali, Silvia Casarotto, Marie-Aurélie Bruno, Steven Laureys, Giulio Tononi, et al.  A theoretically based index of consciousness independent of sensory processing and behavior.  Science translational medicine, 5(198):198ra105–198ra105, 2013\.
  * [41] Anil K Seth, Zoltán Dienes, Axel Cleeremans, Morten Overgaard, and Luiz Pessoa.  Measuring consciousness: relating behavioural and neurophysiological approaches.  Trends in cognitive sciences, 12(8):314–321, 2008.
  * [42] Giulio Tononi, Melanie Boly, Marcello Massimini, and Christof Koch.  Integrated information theory: from consciousness to its physical substrate.  Nature Reviews Neuroscience, 17(7):450–461, 2016.
  * [43] Max Tegmark.  Consciousness is a state of matter, like a solid or gas.  New Scientist, 222(2964):28–31, 2014.
  * [44] Max Tegmark.  Consciousness as a state of matter.  Chaos, Solitons & Fractals, 76:238–270, 2015.
  * [45] Axel Cleeremans.  Computational correlates of consciousness.  Progress in brain research, 150:81–98, 2005.
  * [46] Anil Seth.  Explanatory correlates of consciousness: theoretical and computational challenges.  Cognitive Computation, 1(1):50–63, 2009.
  * [47] James A Reggia, Garrett Katz, and Di-Wei Huang.  What are the computational correlates of consciousness?  Biologically Inspired Cognitive Architectures, 17:101–113, 2016\.
  * [48] Stephen Grossberg.  Towards solving the hard problem of consciousness: The varieties of brain resonances and the conscious experiences that they support.  Neural Networks, 87:38–95, 2017.
  * [49] Stuart Hameroff and Roger Penrose.  Orchestrated reduction of quantum coherence in brain microtubules: A model for consciousness.  Mathematics and computers in simulation, 40(3-4):453–480, 1996\.
  * [50] Stuart Hameroff.  Biological feasibility of quantum approaches to consciousness.  The Physical Nature of Consciousness, Amsterdam: John Benjamins, pages 1–61, 2001.
  * [51] Max Tegmark.  Importance of quantum decoherence in brain processes.  Physical review E, 61(4):4194, 2000.
  * [52] Christof Koch and Klaus Hepp.  Quantum mechanics in the brain.  Nature, 440(7084):611–611, 2006.
  * [53] Christof Koch and Klaus Hepp.  The relation between quantum mechanics and higher brain functions: Lessons from quantum computation and neurobiology.  Citeseer, 2007.
  * [54] Bernard J Baars.  The global workspace theory of consciousness.  The Blackwell companion to consciousness, pages 236–246, 2007.
  * [55] Stanislas Dehaene, Michel Kerszberg, and Jean-Pierre Changeux.  A neuronal model of a global workspace in effortful cognitive tasks.  Proceedings of the national Academy of Sciences, 95(24):14529–14534, 1998.
  * [56] Stanislas Dehaene and Lionel Naccache.  Towards a cognitive neuroscience of consciousness: basic evidence and a workspace framework.  Cognition, 79(1-2):1–37, 2001.
  * [57] Stanislas Dehaene and Jean-Pierre Changeux.  Neural mechanisms for access to consciousness.  The cognitive neurosciences, 3:1145–58, 2004.
  * [58] Claire Sergent and Stanislas Dehaene.  Neural processes underlying conscious perception: experimental findings and a global neuronal workspace framework.  Journal of Physiology-Paris, 98(4-6):374–384, 2004.
  * [59] Stanislas Dehaene, Jean-Pierre Changeux, and Lionel Naccache.  The global neuronal workspace model of conscious access: from neuronal architectures to clinical applications.  In Characterizing consciousness: From cognition to the clinic?, pages 55–84. Springer, 2011.
  * [60] Stanislas Dehaene, Lucie Charles, Jean-Rémi King, and Sébastien Marti.  Toward a computational theory of conscious processing.  Current opinion in neurobiology, 25:76–84, 2014.
  * [61] Stanislas Dehaene, Hakwan Lau, and Sid Kouider.  What is consciousness, and could machines have it?  Science, 358(6362):486–492, 2017.
  * [62] Antonio R Damasio.  The feeling of what happens: Body and emotion in the making of consciousness.  Houghton Mifflin Harcourt, 1999.
  * [63] Antonio Damasio and Kaspar Meyer.  Consciousness: An overview of the phenomenon and of its possible neural basis.  The neurology of consciousness: Cognitive neuroscience and neuropathology, pages 3–14, 2009.
  * [64] Antonio Damasio.  Fundamental feelings.  Nature, 413(6858):781–781, 2001.
  * [65] Ron Sun and Stan Franklin.  Computational models of consciousness: A taxonomy and some examples., 2007\.
  * [66] Janusz A Starzyk and Dilip K Prasad.  Machine consciousness: A computational model.  Brain-inspired Cognitive Systems (BICS 2010), 2010.
  * [67] Stan Franklin and Art Graesser.  A software agent model of consciousness.  Consciousness and cognition, 8(3):285–301, 1999.
  * [68] Artemy A Kotov.  A computational model of consciousness for artificial emotional agents.  Psychology in Russia: state of the aRt, 10(3):57–73, 2017.
  * [69] JH van Hateren.  A theory of consciousness: computation, algorithm, and neurobiological realization.  Biological cybernetics, 113(4):357–372, 2019.
  * [70] Joshua B Tenenbaum, Charles Kemp, Thomas L Griffiths, and Noah D Goodman.  How to grow a mind: Statistics, structure, and abstraction.  science, 331(6022):1279–1285, 2011.
  * [71] Samuel J Gershman, Eric J Horvitz, and Joshua B Tenenbaum.  Computational rationality: A converging paradigm for intelligence in brains, minds, and machines.  Science, 349(6245):273–278, 2015.
  * [72] Brenden M Lake, Tomer D Ullman, Joshua B Tenenbaum, and Samuel J Gershman.  Building machines that learn and think like people.  Behavioral and brain sciences, 40, 2017.
  * [73] Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua B. Tenenbaum, and Jiajun Wu.  The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision, 2019.
  * [74] Roger Penrose.  Consciousness, the brain, and spacetime geometry: an addendum: Some new developments on the orch or model for consciousness.  Annals of the New York Academy of Sciences, 929(1):105–110, 2001\.
  * [75] Emma S Brunette, Rory C Flemmer, and Claire L Flemmer.  A review of artificial intelligence.  In 2009 4th International Conference on Autonomous Robots and Agents, pages 385–392. IEEE, 2009.
  * [76] Christopher M Bishop.  Pattern recognition and machine learning.  springer, 2006.
  * [77] Richard S Sutton, Andrew G Barto, et al.  Introduction to reinforcement learning, volume 135.  MIT press Cambridge, 1998.
  * [78] Jürgen Schmidhuber.  An on-line algorithm for dynamic reinforcement learning and planning in reactive environments.  In 1990 IJCNN international joint conference on neural networks, pages 253–258. IEEE, 1990.
  * [79] Jürgen Schmidhuber.  A possibility for implementing curiosity and boredom in model-building neural controllers.  In Proc. of the international conference on simulation of adaptive behavior: From animals to animats, pages 222–227, 1991.
  * [80] Ray J Solomonoff.  A formal theory of inductive inference. part i.  Information and control, 7(1):1–22, 1964.
  * [81] Phil Maguire, Philippe Moser, and Rebecca Maguire.  Understanding consciousness as data compression.  Journal of Cognitive Science, 17(1):63–94, 2016.
  * [82] Marcus Hutter.  Universal artificial intelligence: Sequential decisions based on algorithmic probability.  Springer Science & Business Media, 2004.
  * [83] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton.  Deep learning.  nature, 521(7553):436–444, 2015.
  * [84] Adam H Marblestone, Greg Wayne, and Konrad P Kording.  Toward an integration of deep learning and neuroscience.  Frontiers in computational neuroscience, 10:94, 2016.
  * [85] Marcel Van Gerven.  Computational foundations of natural intelligence.  Frontiers in computational neuroscience, 11:112, 2017.
  * [86] Demis Hassabis, Dharshan Kumaran, Christopher Summerfield, and Matthew Botvinick.  Neuroscience-inspired artificial intelligence.  Neuron, 95(2):245–258, 2017.
  * [87] David GT Barrett, Ari S Morcos, and Jakob H Macke.  Analyzing biological and artificial neural networks: challenges with opportunities for synergy?  Current opinion in neurobiology, 55:55–64, 2019.
  * [88] Blake A Richards, Timothy P Lillicrap, Philippe Beaudoin, Yoshua Bengio, Rafal Bogacz, Amelia Christensen, Claudia Clopath, Rui Ponte Costa, Archy de Berker, Surya Ganguli, et al.  A deep learning framework for neuroscience.  Nature neuroscience, 22(11):1761–1770, 2019.
  * [89] Neil Savage.  Marriage of mind and machine.  Nature, pages 15–27, 2019.
  * [90] Anthony M Zador.  A critique of pure learning and what artificial neural networks can learn from animal brains.  Nature communications, 10(1):1–7, 2019.
  * [91] Andreas K Maier, Christopher Syben, Bernhard Stimpel, Tobias Würfl, Mathis Hoffmann, Frank Schebesch, Weilin Fu, Leonid Mill, Lasse Kling, and Silke Christiansen.  Learning with known operators reduces maximum error bounds.  Nature machine intelligence, 1(8):373–380, 2019.
  * [92] Kimberly L Stachenfeld, Matthew M Botvinick, and Samuel J Gershman.  The hippocampus as a predictive map.  Nature neuroscience, 20(11):1643, 2017.
  * [93] Samuel J Gershman.  The successor representation: its computational logic and neural substrates.  Journal of Neuroscience, 38(33):7193–7200, 2018.
  * [94] Jesse P Geerts, Kimberly L Stachenfeld, and Neil Burgess.  Probabilistic successor representations with kalman temporal differences.  arXiv preprint arXiv:1910.02532, 2019.
  * [95] Iyad Rahwan, Manuel Cebrian, Nick Obradovich, Josh Bongard, Jean-François Bonnefon, Cynthia Breazeal, Jacob W Crandall, Nicholas A Christakis, Iain D Couzin, Matthew O Jackson, et al.  Machine behaviour.  Nature, 568(7753):477–486, 2019.
  * [96] Andreas Maier, Christopher Syben, Tobias Lasser, and Christian Riess.  A gentle introduction to deep learning in medical image processing.  Zeitschrift für Medizinische Physik, 29(2):86–101, 2019.
  * [97] Kyunghyun Cho, Bart Van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio.  On the properties of neural machine translation: Encoder-decoder approaches.  arXiv preprint arXiv:1409.1259, 2014.
  * [98] Sepp Hochreiter and Jürgen Schmidhuber.  Long short-term memory.  Neural computation, 9(8):1735–1780, 1997.
  * [99] Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al.  Human-level control through deep reinforcement learning.  Nature, 518(7540):529–533, 2015.
  * [100] Lukasz Kaiser, Mohammad Babaeizadeh, Piotr Milos, Blazej Osinski, Roy H Campbell, Konrad Czechowski, Dumitru Erhan, Chelsea Finn, Piotr Kozakowski, Sergey Levine, et al.  Model-based reinforcement learning for atari.  arXiv preprint arXiv:1903.00374, 2019.
  * [101] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In Advances in neural information processing systems, pages 5998–6008, 2017.
  * [102] Evan M Russek, Ida Momennejad, Matthew M Botvinick, Samuel J Gershman, and Nathaniel D Daw.  Predictive representations can link model-based reinforcement learning to model-free mechanisms.  PLoS computational biology, 13(9):e1005768, 2017.
  * [103] Olivia Carter, Jakob Hohwy, Jeroen Van Boxtel, Victor Lamme, Ned Block, Christof Koch, Naotsugu Tsuchiya, et al.  Conscious machines: Defining questions.  Science, 359(6374):400–400, 2018.
  * [104] Andrea Kübler and Niels Birbaumer.  Brain–computer interfaces and communication in paralysis: Extinction of goal directed thinking in completely paralysed patients?  Clinical neurophysiology, 119(11):2658–2666, 2008.
  * [105] Kingson Man and Antonio Damasio.  Homeostasis and soft robotics in the design of feeling machines.  Nature Machine Intelligence, 1(10):446–452, 2019.
  * [106] Jürgen Schmidhuber.  On learning to think: Algorithmic information theory for novel combinations of reinforcement learning controllers and recurrent neural world models.  arXiv preprint arXiv:1511.09249, 2015.
  * [107] Juergen Schmidhuber.  One big net for everything.  arXiv preprint arXiv:1802.08864, 2018.
  * [108] Anthony Kenny.  The homunculus fallacy.  In Investigating psychology, pages 169–179. Routledge, 2016.
  * [109] David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al.  Mastering the game of go with deep neural networks and tree search.  nature, 529(7587):484, 2016.
  * [110] INTELLIGENCE BY AM TURING.  Computing machinery and intelligence-am turing.  Mind, 59(236):433, 1950.
  * [111] Robert M French.  Moving beyond the turing test.  Communications of the ACM, 55(12):74–77, 2012.
  * [112] Moshe Y Vardi.  Would turing have passed the turing test?  Communications of the ACM, 57(9):5–5, 2014.
  * [113] Roger Clarke.  Asimov’s laws of robotics: implications for information technology-part i.  Computer, 26(12):53–61, 1993.
  * [114] Anna Jobin, Marcello Ienca, and Effy Vayena.  The global landscape of ai ethics guidelines.  Nature Machine Intelligence, 1(9):389–399, 2019.