2021

[1]Julian Zubek

[1]Faculty of Psychology, University of Warsaw, Stawki 5/7, Warsaw, 00-183,
Poland

2]Department of Informatics, University of Sussex, UK

# Models of symbol emergence in communication: a conceptual review and a guide
for avoiding local minima

<EMAIL_ADDRESS>Tomasz Korbak  Joanna Rączaszek-Leonardi  *  [

###### Abstract

Computational simulations are a popular method for testing hypotheses about
the emergence of communication. This kind of research is performed in a
variety of traditions including language evolution, developmental psychology,
cognitive science, machine learning, robotics, etc. The motivations for the
models are different, but the operationalizations and methods used are often
similar. We identify the assumptions and explanatory targets of several most
representative models and summarise the known results. We claim that some of
the assumptions – such as portraying meaning in terms of mapping, focusing on
the descriptive function of communication, modelling signals with amodal
tokens – may hinder the success of modelling. Relaxing these assumptions and
foregrounding the interactions of embodied and situated agents allows one to
systematise the multiplicity of pressures under which symbolic systems evolve.
In line with this perspective, we sketch the road towards modelling the
emergence of meaningful symbolic communication, where symbols are
simultaneously grounded in action and perception and form an abstract system.

###### keywords:

emergent communication, signaling games, symbol grounding, compositionality,
computational simulations, linguistics, ecological psychology

##  1 Introduction

The problem of the emergence of communication systems lies at the heart of
theories of cognition. It subsumes the problem of the emergence of symbols
that seem conceptually indispensable for formulating explanations in cognitive
science and artificial intelligence. Over the last two decades, this problem
has been tackled using computational models, which made the theories more
internally coherent and, at the same time, more comparable and classifiable.
Models are grounded in different domains of inquiry – from philosophy to
linguistics, to robotics – and motivated by questions relevant to those
domains; yet, despite this variety, they seem to share some basic assumptions
about the process of communication. We attribute these assumptions to two
different sources.

The first source is the view of communication imposed by the influential work
of Shannon in the mid-last century and strengthened by the facilitation of
computational modelling that such a perspective carries. In most models,
communication is understood as conveying messages from an information source
to a destination via signals transmitted over a communication channel
(Shannon, 1948). In Shannon’s own words: “The fundamental problem of
communication is that of reproducing at one point either exactly or
approximately a message selected at another point”. This highlights the
problem of message transmission as central for communication. The “meaning” of
messages for a long time was in the second plane, usually simplified as a
mapping between symbols and things in the agent environment, which realises
the “grounding” of symbols (Harnad, 1990). The message-object mapping,
representing meaning, mirrors the message-code mapping central to information
theory.

However, the fathers of information theory were convinced that the term
communication should have a broader meaning. As Weaver (1949) put it: “The
word communication will be used here in a very broad sense to include all of
the procedures by which one mind may affect another”. This resonates with more
pragmatic accounts of meaning that focus on influencing agent actions rather
than representing the world. In human communication, this influence is often
based on existing conventions. For example, I can signal a person to come
closer or stay away using conventional gestures. This brings us to the second
source of assumptions concerning communication: Lewis signaling game (from now
on LSG; Lewis (1969)). On the ground of game theory, Lewis demonstrated that a
conventional communication protocol emerges as an equilibrium when two agents
– sender and receiver – are rewarded for successful coordination; it is
rational for the two agents to stick to the convention, even if the convention
itself is arbitrary. This was Lewis’ solution to the problem of meaning
creation by convention. Notably, in the original formulation of LSG
information-theoretical tools were not used, and Shannon’s communication
theory was not cited as a reference. Later, the two approaches were integrated
by Skyrms (2010). Subsuming LSG into the general information-theoretical
schema for communication, he was able to interpret signals as probabilistic
events that carry information and conceptually integrate information quantity
and information content (conventional meaning). However, the consequence of
this move was undermining some of the pragmatic inclinations of the original
Lewis work. Skyrms noted that in many situations signaling can be discussed
equivalently in terms of information flow and in terms of coordination, but he
preferred the first perspective, which reversed the original emphasis of
Lewis.111In Lewis (1969) the word “information” appears 11 times, and the word
“coordination” 138 times, in Skyrms (2010) the word “information” appears 358
times, and the word “coordination” 7 times. This made it easy to once more
conflate signaling with information transmission and to see LSG as part of the
meaning-mapping enterprise.

The resulting signaling game framework, realised by many computational models,
seems to be compatible with the concept of language games, pioneered by
Wittgenstein (Wittgenstein (1953), see also Correia (2020)). A classical
example of a language game would be a builder who uses building stones of
various shapes and asks his assistant to pass a stone of a particular shape.
The language game results in success when the right kind of stone is passed.
These kinds of structured interactions are simulated with artificial agents,
the behaviour of which is governed by some internal adaptive mechanisms (for
example, neural networks). At first, agents do not have any meaningful
predispositions (policy), so their actions and signals that they produce are
accidental. Each episode of the game brings consequences for the agents (for
instance, success or failure) determining their reward, which make them update
their predispositions. Through individual adaptations, agents become more
successful, but at the same time, they change the rules of the game for other
agents. For example, if an agent adopts the word “glud” to refer to a cuboid-
shaped stone, other agents need to pick up on that to successfully interact
with the first agent. In this aspect, the adaptation process resembles
coevolution within a population, rather than a simple optimisation of
individual fitness. Computational simulations demonstrate how this iterative
adaptation can lead to the emergence of stable communication systems.

Existing work on emergent communication has repeatedly demonstrated the
possibility of learning a conventionalised mapping from a set of discrete
signals to a set of objects without any central authority. Optimal mapping can
be learnt under various settings, agent architectures, and adaptation
mechanisms (Spike ., 2017). Natural languages, however, with their richness
and expressiveness, do not resemble simple bidirectional mappings. They
include parts of speech that are not easily mapped to objects or events, they
involve complex syntax and grammar, they tolerate nuances and ambiguities (and
exploit them, as in the case of poetry), they allow speculative conversations
on abstract entities, and they crucially depend on interaction in time. Much
work is still needed to build an adequate account of these phenomena within
the field of language emergence modelling. But the direction in which to
proceed is unclear. In this paper, we claim that the field is in a state of a
local minimum created by a particular set of assumptions. This obscures the
view, taking some properties of the communication process for granted and
leaving crucial aspects of language unexplored.

Getting stuck in the above-mentioned local minimum hinders the efforts to
scale up from simple signaling games to structured communication systems,
which can be used as complex compositional controls in time-dependent
interactions within structured environments. Recent developments in cognitive
science, linguistics, psycholinguistics, and linguistic anthropology point to
multiple processes and properties of communication, which appear to be
overlooked in dominant models and might be crucial for this kind of scaling
up. Theories that ground communication in biological information processes
(Pattee, 1969, 1982; Deacon, 2011), interaction-dominant and pragmatics-
oriented processes (Kempson ., 2000), dynamical systems models of cognition
(Rączaszek-Leonardi Kelso, 2008) and enaction (Di Paolo ., 2018) offer a
richer perspective on the nature and functions of communication. Bringing them
into play on the modelling level may be helpful in overcoming some of the
obstacles.

In this paper, we review several attempts of emergent communication modelling
that we deem representative to the multitude of models that have appeared over
the last two decades and evaluate them in terms of their merits and
shortcomings. This will allow a better grasp of their underlying common
assumptions about the process of communication and identification of those,
which might be responsible for hindering the progress towards modelling the
emergence of structured symbolic systems. The main claim of the paper is that
progress does not hinge on the complexification of the existing models, making
them more efficient computationally or applicable to more complex
environments, nor even in a creative composition of the models. We claim that
in order to leave the local minimum, we need to examine closely and perhaps
suspend some of the core, and very basic, assumptions on which the extant
modelling efforts rest. For this, we perhaps need to back out of some habits
of thinking and seek guidance in those theories which do not assume an easy
reduction of the problem of communication to the engineering problem of
information transmission. From this widened perspective, we can recognise that
language is not a product of a single optimisation process, but is shaped by
multiple factors. Thus, sources of complexity of language are multiple and may
possibly interact, leading to emergent properties. It is possible that
language properties that have simple formal description, for example
recursively nested structures described with formal grammars, could
nevertheless be shaped by multiple interacting pressures. Opening up this
possibility will hopefully set novel explanatory targets for the models and
open new avenues for research on emergent symbolic communication.

##  2 Modelling traditions and explanatory targets

Computational models of the emergence of communication come from a variety of
research traditions, reflecting their theoretical goals, definitions, and
methods. Giving justice to their recent proliferation is beyond any single
publication. We roughly group them by discipline to draw the reader’s
attention to different explanatory roles assigned to the simulations. These
arise from differences in the perspectives on language in general and on the
place, role, and nature of language within human cognition and behaviour. We
will point out how those differences are manifested in computational modelling
and use the presence and prominence of such models in a discipline as a
criterion to include them in this review. An (admittedly simplified) picture
in Figure 1 illustrates some of the commitments and explanatory targets of the
models in different scientific disciplines (the division of the domains is an
idealisation).

Figure 1: Different scientific disciplines and the roles they assign to
emergent communication models.

Philosophical reflection on the nature of language and its relation to thought
is probably as old as philosophy itself. In this context, game-theoretic
models have been used in research of social conventions, normativity, and
semantic information, starting from the influential work of Lewis (1969). He
formalised the notion of convention, which arises in the interaction of
agents. The sender has to communicate the state of the world to the receiver
by emitting discrete signals, upon which the receiver produces an adequate
reaction. Lewis’ original motivation was to provide a game-theoretic
explication for the notion of truth by convention (and defend it against Quine
(1936)). His ideas influenced many other authors.

When implemented as computer simulations, these models are kept abstract and
usually consist of disembodied agents passing discrete tokens. The goal is not
to faithfully portray actual processes but rather to support certain
philosophical positions by demonstrating the theoretical possibility of the
bottom-up emergence of conventions. In recent years, this thread has been
developed to account for more complex communication processes (Skyrms, 2010;
Barrett Skyrms, 2017; Barrett ., 2019; Korbak ., 2021).

In the area of linguistics, computational models of language emergence are
used to ask a variety of questions. Contrary to biological evolution, language
as such (before writing) leaves no “fossils”, so the communicative systems of
early humans (or hominids) are difficult to uncover. In this context, agent-
based simulations became attractive tools for testing hypotheses about the
origins of language and possible evolutionary trajectories, both on the
biological and cultural time scales. The goals are theoretical: models concern
various aspects of language, such as evolution of lexemes and systematic
variation of syntax (Nowak Krakauer (1999); for a review, see, e.g., Ferri .
(2018); Grifoni . (2016)) or evolution of vocabulary. Compared to the
philosophical tradition, simulations developed within evolutionary linguistics
tend to be more focused on language structure. The results of the simulations
are compared with the empirical data to assess their plausibility as models of
language evolution and change (Brighton ., 2005; Parisi ., 2008; De Boer
Zuidema, 2010).

In the field of cognitive psychology, the emphasis is on the connection
between communication and language and other cognitive processes, mainly
internal, such as memory, categorisation, and reasoning. For example, emergent
language, stabilised in language games, has been shown to be a factor
determining the shared structure of individual categories within populations
(Steels Belpaeme, 2005; Baronchelli ., 2015). Some models in this field do
concern the emergence of language on the developmental timescale, although,
curiously and symptomatically, these rarely take the form of computational
simulation. However, an interesting turn can be noticed from the focus on
learning as tuning of the predetermined generative structure, resulting in the
production of a well-formed language (Chomsky, 1957), to a focus on how
developmental and functional processes might be reflected in the language
structure (Deacon, 1997; Brighton ., 2005). In the former approach, which
spurned modern psycholinguistics, cognitive psychology was seen as a
subsidiary to linguistics, responsible for less theoretically noble processes
of realisation (performance; Devitt (2003)), while in the latter it comes to
the fore as a source of processes that structure language (Rączaszek-Leonardi,
2009). Some models address directly how language structure is shaped by and,
in turn, shapes other processes (cognitive, environmental, or social). For
example, models explored the effects of “starting small”, that is, with a
smaller memory capacity (Elman, 1993), or of interactive factors within social
environments or various demographic characteristics such as population size or
social structure on language evolution (Vogt, 2009; Dale Lupyan, 2012).

Emergent communication models have also enjoyed a recent surge of interest in
the machine learning and artificial intelligence (AI) community. Consistently
with the original mission of AI, here research is focused on replicating
cognitive processes which are necessary to develop and use human intelligence,
in this case language. A great deal of machine learning research on emergent
communication addresses fundamental questions concerning the structure of
language. Lazaridou . (2018) demonstrated that reinforcement learning-driven
agents are capable of solving the LSG in which input is given as raw pixel
images, not symbolic tokens. Emerged protocols displayed some degree of
systematic structure, although ad-hoc vocabulary fitted to a particular game
was preferred. Several other studies focused on the circumstances under which
a more structured language appears (Mordatch Abbeel, 2018; Andreas, 2019;
Chaabouni ., 2020). Other models are not motivated directly by replicating
human cognitive processes but arther by engineering problems related to
natural language understanding. Kottur . (2017) work on emergent dialogue is
to some extent motivated by the work on visually-grounded conversational
systems (Mostafazadeh ., 2017). The formulation of the problem in Lazaridou .
(2017) is motivated by image captioning (Karpathy Fei-Fei, 2017). Lee . (2018)
posed the problem of unsupervised visually grounded translation as a LSG. The
potential applications of emergent communication research in explainable AI
(Doshi-Velez Kim, 2017) are also an intense area of research. They focus
mainly on translating a communication protocol emerging between agents in the
context of some task into a natural language such as English. For visually
grounded approaches, this can be done using labelled images by relying on
visual semantics determined by the class hierarchy of the images (Lazaridou .,
2017). Another approach is to align utterances in English and in an emergent
communication protocol with respect to the effects they exert on a trained
receiver. This also paves the way for natural language understanding by
artificial agents.

The trend of making simulations more realistic and embodied is taken even
further in artificial life and cognitive robotics communities. In this
tradition, agents are not abstract entities but autonomous mobile robots
(usually simulated in silico to some extent). Their communication is
intertwined with the rest of their activity, often involving exploration of
the environment and search for food or other agents (Floreano ., 2007; Grouchy
., 2016; Schulz ., 2012). Stress is placed primarily on the robustness of
emerging communication protocols and minimal starting assumptions, as
communication should scale up to realistic environments. More advanced
communication features are built from the bottom-up rather than arbitrarily
imposed. Some works in this tradition aim to explain basic aspects of animal
communication in different evolutionary scenarios (Floreano ., 2007; Mitri .,
2009), while others target phenomena characteristic for human languages, such
as its productivity, presence of grammatical structures, and the focal role of
dialogue (Grouchy ., 2016; Schulz ., 2012). In contrast to other domains,
signals appearing in robotics models are often modulated by multiple factors
and are not fully controlled by agents. For example, in Floreano . (2007)
communication is realised through light signals that have colour regulated by
the sender, but also properties dependent on the physics of the environment
(directionality, possibility of being obscured by obstacles, etc.).

As can be glimpsed from the above, each discipline, which includes
communication in its area of interest, has explanatory targets dependent on
its general nature and goals. Models are designed to address various aspects
of communication and language emergence, sometimes using different techniques.
It seems that one of the problems in constructing more comprehensive models of
language emergence is the compartmentalization of scientific efforts into
different domains. The domain in which we see the greatest proliferation of
models and in which most novel techniques appear is AI, where research is
often motivated by concrete engineering problems and not theory of language
per se. The novel modelling techniques do radiate to the basic problems and
their respective domains, however, already tinted by the specific approach. On
the other hand, recent advances in the respective fields that are interested
in the theory of language emergence (developmental linguistics, language
evolution, enactivism) are rather slow to seep into the computational models.
In consequence, designed models often rely on a simplified treatment of the
basic problems, which are the core of more theoretically rich approaches, and
modelling results cannot be easily incorporated into constructed theories.
Here, cognitive science, integrating multiple disciplines from linguistics to
psychology to anthropology to computer science, has a good chance to serve as
a bridge between the domains. Importantly, due to the recent “ecological turn”
toward action-oriented and situated cognitive science, it has the opportunity
to integrate processes over the biological, developmental and communication
timescales (Rączaszek-Leonardi, 2003; Christiansen Chater, 2022).

We identify ourselves with the positions of such ecological cognitive science.
From this perspective, we point out how this lack of flow of ideas between
domains might slow down progress of computational modelling as a significant
aid for the theoretical models of communication and language emergence. In the
following, we describe concrete examples of models which best illustrate the
various theoretical goals. We will show their advantages and contributions,
but also indicate the possible simplifications, which will be the basis for
conclusions regarding further progress.

##  3 From Lewis signaling game to embodied simulations

Despite different distributions of accents of the above domains, recently they
have increasingly used computational simulations implemented as agent-based
models with explicit agent architectures. This facilitates the recognition of
basic assumptions regarding the communication and language on which they are
based. As a blueprint for the analysis of all kinds of modelling efforts, we
adopt Lewis signaling game (LSG). As a minimal architecture for an interactive
communicative setting, it delimits a broad class of models, capturing the
essential elements of the problem of convention- or signal-based social
coordination. It is recognised and sometimes explicitly built-on across
various disciplines (from philosophy and anthropology to psychology, biology,
economics, etc.). This should further facilitate the comparison of the models
on a conceptual level, in terms of modifications of this basic scheme for
various reasons, which can be evaluated as welcome or unwelcome from the
theoretical point of view. Showing their basic compatibility with the LSG
should also reveal that they share limitations inherent in the blueprint
itself, if indeed such limitations are present.

LSG consists of two agents, a sender and a receiver, a set of states of
affairs (stimuli), a set of signals, and a set of responses. A sender observes
a state of nature, sends a signal to the receiver, and the receiver chooses a
response. There is a unique correct response given the state of affairs
unknown to the receiver. If the receiver chooses the correct response, both
the sender and the receiver are rewarded. In the course of learning (or
evolution), the agents converge to a Nash equilibrium such that the receiver
always chooses the correct response. This means that the agents must have
agreed upon a convention: a mapping from states of nature to signals and from
signals to responses. Lewis called this a _signaling system_ (nowadays it is
also called a communication protocol).

The original interpretation of the LSG involved rational agents maximising
their reward. Skyrms (2010) later reinterpreted it in terms of evolutionary
game theory: he demonstrated that signaling systems may emerge through simple
replicator dynamics (which may represent both biological and cultural
evolution). He proposed that the information present in the signaling system
is sufficient to provide a general account of semantic meaning. According to
Skyrms, meaning emerges in the symmetry breaking of converging to one
signaling system (out of several possible). Each signal can be assigned a
descriptive meaning (when an external observer estimates the probability of
the sender perceiving certain stimulus given the signal) and indicative
meaning (when an external observer estimates the probability of the receiver
performing certain action given the signal). Such meaning is a property of the
signal system as a whole and does not presuppose that the sender and receiver
are rational agents introducing any kind of meaning of their own.

Several other problems in philosophy and social science can be formalised in
terms of LSGs. The labour market can be modelled in terms of employees
(senders) who signal their ability to employers (receivers) through
educational credentials (signals) (Spence, 1973). Voter ignorance and the
spread of misinformation can be modelled in terms of people (senders)
maintaining or spreading false beliefs (signals) to signal their group
identity or political affiliation (Funkhouser, 2017). Barrett Skyrms (2017)
recently considered generalised LSGs consisting of multiple subgames. These
include games that evolved from simpler games through template transfer or
modular composition. Template transfer assumes that some adaptations that
emerged in the context of one signaling game can be transferred to another
game. One of the possibilities would be to reuse the same structure of emitted
signals and adapt them for a new set of stimuli. Template transfer assumes the
historicity of the evolution or learning process giving rise to the signaling
systems – previous adaptations pave the way for the new ones. The games
themselves are independent and, in principle, could also be learnt in another
order or in parallel. In contrast, _modular composition_ is based on
recursively nested games, where a simpler game becomes a part of a more
complex one. A variety of social, cognitive, and semiotic phenomena can be
understood in terms of generalised signaling games, including logical
inference (Barrett Skyrms, 2017; LaCroix, 2022), information integration in a
network of agents (Barrett ., 2019), functional specialisation, and
communication structuring (Barrett ., 2018).

###  3.1 Beyond Lewis signaling game

Most of the recent models of emergent communication may be seen as extensions
of the LSG with respect to particular elements: stimuli, signals, responses,
or reward/learning mechanism. Below, we provide an overview of various models,
from the simplest extensions to complex embodied models, which diverge from
the original structure of LSG significantly. We do not mean to provide a
comprehensive review. Our choice of models is dictated by their popularity and
representativeness for different strands of modelling efforts. In our
overview, we discuss how these models aim to explain different sources of
communication complexity, connected to a) internal structure of signals, b)
structure of the environment, c) structure of actions in the environment and
types of control introduced by signals, and d) structure of rewards. The
selected models and their properties are presented in Table 1. In the next
sections, we discuss them in detail starting from relatively simple formal
models and moving towards embodied simulations in realistic environments.
Throughout, we refer to the LSG as a blueprint and demonstrate how the
presented models deviate from its structure (as reflected by section titles).

Table 1: Discussed models of emergent communication from various traditions and their characteristics in terms of the basic template of LSG. The models were selected in such a way as to display various interaction scenarios considered as communicative situations in the model. We ordered them according to their abstractness: from idealised formal models to embodied simulations/mobile robots. Model |  Discipline |  Environment (stimuli) |  Signals |  Available actions |  Rewards and learning mechanism |  Results/explanatory target
---|---|---|---|---|---|---
Lewis (1969) |  philosophy |  Discrete tokens |  Discrete tokens |  Discrete tokens |  Specific response rewarded |  Stabilization of social conventions
Nowak Krakauer (1999) |  linguistics |  Discrete tokens |  Sets of tokens (compound expressions), risk of confusion |  Discrete tokens |  Different rewards dependent on the response |  Evolutionary origins of phonological and grammatical structures
Smith . (2003) |  linguistics |  Binary vectors: combinations of properties |  Sets of tokens (compound expressions) |  - |  Learning mapping after previous generation in supervised manner |  Origins of systematic structure
Batali (1998) |  linguistics |  Binary vectors: simple predicates |  Sequences of tokens (compound expressions) |  Binary vectors: reconstructed predicates |  Sender tries to predict receiver’s responses internally (obverter). Receiver learns from the sender in a supervised manner. Roles change periodically. |  Emergence of systematic structure in sequences of variable length
Jaques . (2019) |  artificial intelligence |  2D grid world with simple objects encoded as binary vectors |  Discrete tokens |  Movement in the environment |  Number of collected food items rewarded, group reward |  Emergence of communication under intrinsic social influence motivation
Table 1: Continued from the previous page. Model |  Discipline |  Environment (stimuli) |  Signals |  Available actions |  Rewards and learning mechanism |  Results/explanatory target
---|---|---|---|---|---|---
Steels Belpaeme (2005) |  cognitive psychology |  Sets of numerical vectors: CIE LAB colors (available to sender and receiver), marked stimulus (available to sender) |  Discrete tokens from infinite set |  Choosing correct stimulus |  Specific response rewarded, supervised learning of new word forms |  Emerging communication stabilizing category formation
Lazaridou . (2017) |  artificial intelligence |  Sets of numerical vectors: raw RGB images |  Sets of tokens (compound expressions) |  Choosing correct stimulus |  Specific response rewarded |  Signal system grounded in raw visual input
Cangelosi (1999, 2001) |  linguistics & philosophy |  2D grid world with objects encoded as binary vectors (available to sender and receiver, receiver gets perceptual noise) |  Sets of tokens (compound expressions) |  Discrete movement in grid-like environment and discrete response tokens |  Locating food and avoiding poison rewarded in evolution, speaker not rewarded |  Symbolic mode of reference – transfer of symbol–symbol relations to new vocabularies
Schulz . (2012) |  robotics |  2D realistic environment, own position in the environment (available to sender and receiver) |  Sets of tokens (compound expressions) |  Continuous movement in the environment (available to sender and receiver) |  Meeting at the appointed location rewarded |  Constructing abstract location concepts grounded in spatial experience
Table 1: Continued from the previous page. Model |  Discipline |  Environment (stimuli) |  Signals |  Available actions |  Rewards and learning mechanism |  Results/explanatory target
---|---|---|---|---|---|---
Loula . (2010) |  linguistics & philosophy |  2D grid world, 0-1 presence of nearby objects of various classes (available to all agents) |  Discrete tokens from a large set |  Choice of high-level, predefined behavior: wandering, fleeing, etc. (available to all agents) |  Hebbian learning of associations (classical conditioning) |  Shift between different forms of signification within an ecological niche
Mordatch Abbeel (2018) |  artificial intelligence |  2D environment, (shared by sender and receiver), marked target object and action (available to sender) |  Sets of tokens (compound expressions) |  Continuous movement in the environment |  Explicit error feedback |  Systematic signal structure grounded in action structure
Floreano . (2007) |  robotics |  2D environment (shared by all agents) |  Light emitted by physical robots |  Continuous movement in the environment (available to all agents) |  Locating food and avoiding poison rewarded in evolution, speaker not rewarded |  Communication strategies evolving under different kin structures
Grouchy . (2016) |  robotics & linguistics |  2D environment (shared by all agents) |  Continuous scalar |  Continuous movement in the environment (available to all agents) |  No explicit rewards, two agents producing offspring upon contact |  Emergence of complex mode of signification with signal-signal relations

###  3.2 Compound signals and noise

The simple structure of LSG can be easily extended by adopting compound
signals consisting of multiple tokens. This route was taken by Nowak Krakauer
(1999) who simulated the evolution of meaningful words (mono- and
polysyllabic) and two word expressions. They were interested in the dynamics
of language evolution. They investigated whether compound expressions could be
evolutionarily preferred to simple words using methods of evolutionary game
theory. They constructed a fitness function related to communicative success
and identified language structures that optimise this function. The function
included factors such as perceptual noise in the signal space (risk of
confusing similar syllables) and uneven payoffs of successfully communicating
different concepts (for example, signals warning against dangers of different
magnitude). The authors demonstrated that under certain circumstances smaller
and partially ambiguous lexicons are more optimal, and thus there are
evolutionary limits on the growth of vocabulary. Compound expressions formed
according to grammar rules restricting the space of valid utterances helped to
transcend those limits without degrading communicative success. Sources of
complexity here were the need to distinguish noisy signals and uneven payoffs:
they provided evolutionary pressures making compound signals desirable. All
simulations were kept on the abstract level: they demonstrated what sort of
languages are optimal but did not simulate the emergence of actual
communication properties.

The work of Nowak Krakauer (1999) is a valuable demonstration of the role of
noise and the complex structure of tasks that lead to uneven rewards in
shaping the language structure. These factors align well with the ecological
perspective. However, this model focused solely on the problem of signal
recognition (identifying received signals as such and such) and not signal
interpretation. It explains the form of the language and factors important for
its stability under evolutionary pressures, but not the structure of meaning
and not different modes of signification.

###  3.3 Compositional signals via vertical transmission

The interaction between the sender and receiver in a standard LSG can be
called a horizontal interaction. In contrast, Kirby . (2014) develop iterated
learning games, which involve vertical interactions between subsequent
generations of agents developing new vocabulary. In these games there are no
actions or receivers. Instead, the sender alone produces a set of signals for
the set of stimuli presented. Signals are compound multisyllabic utterances,
and stimuli are objects described through the composition of stable
properties. The stimulus-signal pairs produced become a learning sample for
the next sender, which produces a learning sample for yet another sender, etc.
This framework was mathematically analysed using evolutionary game theory
(Brighton, 2002) and implemented in computational simulations. The goal of
iterated learning is to explain the systematic structure of natural languages.

Smith . (2003) constructed a simulation in which they represented agents as
simple neural networks consisting of two layers of binary neurons. The first
layer represented binary object properties (each object either had a
particular property or did not have it), the second layer represented the
presence of specific components in the utterance (each utterance was an
unordered set of components). The connections between neurons were reinforced
on the basis of the object-utterance pairs present in the learning input, and
then a set of utterances for new objects was produced. The authors introduced
a transmission bottleneck between generations: agents from the new generation
observed only a subset of utterances produced by agents of the previous
generation. They demonstrated that under these circumstances compositional
languages – in which individual syllables in an utterance are systematically
mapped to object properties – emerge. Importantly, Kirby . (2008) replicated
the same schema in an experiment involving human participants obtaining
similar results.

Transmission bottleneck introduces evolutionary pressure towards
structurization of meaning. This is different from the compound expressions
with protogrammar analysed by Nowak Krakauer (1999). In that work, grammar
defined only the structure of an utterance, demanding that the word of a
particular class appears at a particular position in a sentence, which was in
principle independent of the word’s meaning. The structure of the compound
expressions in Smith . (2003) was not restricted (there was no grammar), but
the way they referred to the properties of objects was systematic: different
parts of the expression referred to different properties (the principle of
compositionality).

Models based on vertical language transmission can be seen as orthogonal to
horizontal LSG. They do not include a coordination problem, but rather a
memorisation problem: agents learn the signaling protocol by observing
examples rather than through interaction. Vertical and horizontal approaches
can be seen as complementary. Recently, vertical transmission based on
iterated learning is being introduced to horizontal communication models based
on LSG. It was implemented by periodically resetting the weights of the neural
networks of some agents (clearing their memory; Li Bowling (2019); Cogswell .
(2020)) or by having the agents passively observe the communication protocols
of the previous generation (Ren ., 2020).

###  3.4 Social reasoning facilitating communication

A different line of work looks for pressures towards structured communication
in pragmatic social reasoning, which is involved in language generation and
comprehension. Batali (1998) attempted to explain the origin of the grammars
of natural languages. In his simulations, the basic sender-receiver structure
of LSG was preserved, but the sender used its own language comprehension
module to predict the receiver responses. This prediction guided language
generation – an approach dubbed obverter (Oliphant Batali, 1997). Population
of agents learnt in a self-supervised manner: a pair of agents selected at
random interacted in each round, the sender acted as a teacher for the
receiver, and, importantly, the roles were periodically swapped. The emerged
language had compositional properties similar to Smith . (2003): some parts of
the signal sequence had systematic meanings. Furthermore, it was shown that
the agents were able to generalise and communicate novel combinations of
meanings not present in the training set.

The obverter mechanism makes use of the same signal-meaning mapping for
language generation and comprehension. This guarantees that agents are
internally consistent and use the language they learn in the receiver role
when they assume the speaker role. It can be interpreted as a rudimentary form
of social reasoning: an agent’s own cognitive capabilities serve as the model
of another agent. The symmetry of the roles, together with the selection of
speaker-receiver pairs at random from a population of agents, effectively
creates a transmission chain similar to the iterated learning setting Smith .
(2003). This should constitute a pressure for a systematic communication
protocol. The obverter mechanism was implemented in modern neural networks by
Choi . (2018) and Bogin . (2019), who also observed an increase in
compositionality.

The more complex mechanism of social reasoning was explored in Jaques .
(2019). Their research belongs to the domain of artificial intelligence and
multi-agent cooperation. They introduced the concept of social influence as an
auxiliary reward for agents that communicate to solve a sequential social
dilemma. Social influence was measured using counterfactual reasoning: an
agent considered alternative signals it could have sent and assessed their
likely effect on the actions of another agent. In one variant of their
simulations, actions of the other agent were predicted using a specialised
internal model trained in a supervised manner. Unlike in Batali (1998), the
model of the other agent was separated from the main model that controlled the
actions of the agent. The authors did not focus on the signal structure, but
showed that supporting communication with counterfactual reasoning leads to
faster convergence, higher cumulative rewards, and higher mutual information
between signals and actions. Even if agents did not have an explicit
communication channel, influence rewards incentivised them to develop an
ostensive communication protocol by pointing out important locations by
staying still at these locations. This behaviour emerged because agents from
the beginning viewed their actions not only as means to achieve their
immediate goals but also as means to influence others. The dual meaning of the
action followed the dual structure of the reward function.

The results of the simulations involving aspects of social reasoning suggest
that some aspects of communication might be very hard to model if we treat
language emergence as an unconstrained optimisation problem without particular
inductive biases. In other words, agents might require certain cognitive
preadaptations for social cooperation in order to establish a nontrivial
communication protocol.

###  3.5 Bidirectional influences between perception and communication

Steels Belpaeme (2005) addressed the famous symbol grounding problem (Harnad,
1990) by tracking relations between language, categorisation, and perception.
They used a variant of a language game which they called a guessing game.
Similarly to the standard LSG, it involved a sender and a receiver
communicating the properties of the presented stimuli. The stimuli were more
complex than in the vanilla LSG (vectors of real numbers representing
colours), and they were always presented in a context consisting of other
stimuli. The sender was given a set of four random stimuli drawn from the
continuous space with one distinguished target stimulus (the only restriction
was that the target had to be sufficiently distinct from the other stimuli).
Unlike the standard LSG, the receiver was presented with both the sender
signal and the same set of stimuli. The task was to correctly choose the
target stimulus. Each agent used a lexicon module – a mapping between discrete
categories and words – and an adaptive categorisation module responsible for
forming categories in a continuous perceptual space. Successful communication
required coordination of both vocabulary and perceptual categories. In the
simulation, a population of agents interacted randomly with each other freely
exchanging sender and receiver roles. No colour categorisation was designed as
“correct”. The authors demonstrated how a conventionalised categorisation of
the colour stimuli may emerge through linguistic interaction. The vocabulary
was also shared within the population and – through the categorisation module
– grounded in the perceptual properties of the stimuli.

Similar models were used to study how aspects such as innate agents’ biases
(Baronchelli ., 2015) or the topology of a social network (Centola
Baronchelli, 2015; Zubek ., 2017) affect the formation of categories. However,
all these works were interested in the structure of categories rather than the
structure of language; the signals used in these simulations were simple
unstructured tokens.

###  3.6 Signals grounded in raw sensory inputs

The path to ground signals in more realistic visual stimuli was explored
further using deep neural networks. This kind of stimuli is drastically
different from discrete tokens of the basic LSG. The departure from pre-
processed symbolic input (e.g., encoding an object as a binary vector) in
favour of raw visual input (i.e., arrays of RGB pixels) requires the sender to
deal with entangled input and decompose it into relevant factors of variation
(e.g. shape and colour) on its own – essentially performing image
classification (Lazaridou ., 2017, 2018). When compound signals are used,
emergent communication protocols are on average more systematic with visual
input compared to symbolic input. This corroborates the hypothesis that a
structured environment plays a role in structuring language. Interestingly,
communication protocols do not always capture interpretable, concept-level
properties of visual scenes presented to the agents. Rather than that, agents
can learn specifically to successfully distinguish images while overfitting to
inscrutable low-level relational properties of images. Agents can still solve
guessing games with high accuracy with input data that contain no conceptual
content at all, for example, when images are just random noise sampled from a
Gaussian distribution (Bouchacourt Baroni, 2018).

###  3.7 Multi-task learning: simultaneous classification and signaling

The structure of the emerging communication protocol may also be the result of
demands to emit signals while simultaneously performing another action. This
can be reflected by agent architectures capable of multi-task learning. This
setup was explored by Cangelosi (1999, 2001), who constructed agents foraging
mushrooms of different types. Agents were rewarded for correctly identifying a
mushroom type based on unreliable perceptual input and a signal produced by
their peers. Each agent was controlled by its individual neural network.
Networks were evolved through a simple genetic algorithm (population learning)
and additionally fine-tuned using backpropagation (individual learning). The
agent network had separate output layers responsible for identifying mushrooms
and producing signals, and a hidden layer shared between these tasks.
Communication had a structure similar to the basic signaling game, where an
agent foraging for mushrooms was the receiver, and its parent from the
previous genetic algorithm generation was the sender. The sender was presented
with a true vector of perceptual properties of a given mushroom and produced a
two-component signal. The receiver was presented with a noisy version of the
vector of mushroom properties and with the sender’s signal. It had a double
task of correctly identifying the mushroom (rewarded in the evolutionary
timescale) and recreating the sender’s signal (trained with backpropagation in
a supervised manner). This elaborate setup led to the emergence of a
systematic communication protocol with disentangled mushroom properties.

To check the generalisation capabilities of these systems, Cangelosi applied a
particular test based on Deacon (1997) interpretation of the classic
experiments of teaching language to chimpanzees (Savage-Rumbaugh Rumbaugh,
1978; Savage-Rumbaugh ., 1980). The reasoning is as follows: let us assume
that we know some basic vocabulary in one language. Then some new foreign
vocabulary is introduced in a grounded manner, albeit for a limited number of
examples. Next, the learner is presented with combinations of words in the
foreign language that allow them to recognise a familiar grammatical rule.
Afterwards, the learner should be able to correctly use the foreign words in
combinations, even if they were not introduced explicitly for all examples.
This schema was replicated 1-1 with the neural networks. The trained network
represented the properties of mushrooms A and B with independent symbols.
Cangelosi retrained the network using backpropagation to substitute existing
symbols for property A for 2/3 of mushrooms with new symbols. Then, he
introduced new symbols for property B and taught the network how symbols for
properties A and B are composed together in two-component signals. Finally, he
introduced new symbols for property A for the remaining 1/3 of the mushrooms.
At this point, the network was able to correctly associate all symbols A and
B, although this association was explicitly given only for 2/3 of mushrooms.

This kind of generalisation goes beyond generalisation of responses to new
stimuli; this is a systematic transfer of relations formed between one group
of symbols to a new group (in accordance with the Deacon semiotic framework).
In terms of Barrett Skyrms (2017) this could be called an example of template
transfer, where the stimuli remain the same, but the response repertoire
changes. Cangelosi . (2002) considered this an example of “symbolic theft”,
where preexisting grounded symbols and relations between them are used to
ground new ones. Agents were able to learn and transfer this structure of
relations because of their simultaneous participation in the production of
signals and the classification of mushrooms.222In Cangelosi’s simulations
agents were also moving in a 2D environment and had to approach a mushroom
before picking it up. However, since they did not have any form of memory,
their movement trajectory did not influence their classifications and did not
contribute to the structure of their communication.

###  3.8 Modeling an ecological niche for communication

Loula . (2010) decided that to plausibly model phenomena related to language
emergence, they need to broaden the scope of the model and simulate ecological
relationships shaping a niche in which the communication occurs. They
constructed an artificial life simulation inspired by the ecology of vervet
monkeys. In the simulation, a bunch of herbivore creatures explored a 2D
environment looking for edible plants while avoiding predators. Contrary to
the basic LSG, signaling behaviours existed within the natural ecology of the
simulated environment, going beyond static sender-receiver interactions.

Creatures in the simulation had capabilities to see nearby objects, hear
vocalisations, and select predefined actions based on their perception and
internal drives (hunger, fear, boredom, etc.). Actions were high-level
behaviours, such as scanning the environment, wandering, fleeing, hiding,
climbing a tree, vocalising, etc. The communication was not limited to sender-
receiver pairs: any creature within vicinity was able to hear a vocalisation
and react to it (possibly with another vocalisation). No particular reaction
was distinguished as “correct”, systematic associations between vocalisations
(signals) and predator types (meanings) spontaneously self-organised through
Hebbian learning of associations between stimuli in this ecology. The authors
demonstrated how the emergence of signals transforms ecology. At first,
creatures reacted to signals by scanning their environment looking for
potential predators: all signals were interpreted as general alarm calls
signaling danger. Later, when signals were well learnt, the creatures took an
action adequate for the predator type (fleeing, hiding, or climbing a tree)
immediately after hearing the signal without ever seeing the predator. This
marks a shift between different forms of signification, where the same signals
gradually change meaning. This could be interpreted as another example of
template transfer (Barrett Skyrms, 2017). As these simulations demonstrate,
even a very simple association learning mechanism can stabilise communication
if the ecology is rich enough.

###  3.9 Embodied and situated communication

In the vanilla LSG all actions are discrete: the model abstracts from action
dynamics. Mordatch Abbeel (2018) started from the standard sender-receiver
interaction, but provided a minimum embodiment of their agents by placing them
in a continuous environment where action and communication dynamics matter. In
their simulation, a 2D continuous space was populated by agents and coloured
landmarks. Agents learnt through supervised learning to communicate to each
other about actions that they must undertake (e.g., go to) and objects that
actions must target (e.g. blue). The communication protocol that emerged in
throught the simulations was efficient and had a compositional verb-noun
structure. Compositionality was encouraged through a soft limit on vocabulary
size that made agents reuse the already known words (introducing a new word
generated cost in the form of penalty). The ability of an agent to move freely
contrasts with dominant settings, where the sender passively observes a
stimulus (independently of any actions it can undertake) and the receiver is
waiting for a message to undertake an action. The fact that the sender could
move enabled nonverbal communication as the receiver used, for example, gaze
direction or physical movement of the sender to inform its actions. Moreover,
according to the authors, the physical properties of the environment and agent
communication (e.g. actions and utterances that take time to accomplish)
shaped the syntactic structure of the utterances. In the emerged communication
protocol, the GOTO action type symbol was uttered before the landmark symbol,
allowing the receiver to “prepare” for further actions by moving to an
advantageous central position among the landmarks before starting to move
towards the specific one.

Respect for the rich and structured environment as a source of structure
appearing in language is clear in the experiments of Floreano . (2007), who go
further with the embodiment of agents by employing mobile robots. The robots
navigated through a 2D space to discover the food source and avoid the poison
source. The robots used neural controllers evolved through an evolutionary
algorithm, where fitness was determined by their food intake. They had an
ability to emit light signals, helping other robots locate food, but this
created competition for resources: spatial limitation of the environments
allowed only a limited number of robots to feed at the same time. Contrary to
the standard LSG, there were no sender-receiver roles, no turn-taking, no
explicit reward for successful communication. The results indicated that
cooperative communication can emerge among kin groups even if signaling is
costly for individuals. More varied behaviours, such as the emergence of
deceptive communication in groups of nonrelated robots, were also observed.
Importantly, the values and costs of communication in these experiments were
not arbitrarily introduced, but emerged from the properties of the
environment.

###  3.10 Combining concepts across games

Yet another mechanism that leads to more complex communication was explored by
Schulz . (2012), who experimented with mobile robots that navigated a natural
environment (office space). They used language games similar to Steel’s
guessing game, with different roles of sender and receiver. There was an
explicit acquisition of new vocabulary by the receiver through imitation. The
new element was that the agents played a couple of different games and the
concepts acquired through these games were interrelated.

The robots had the ability to move freely in their environment and construct
private maps of spatial experiences based on visual stimuli and self-motion
odometry (using the RatSLAM algorithm (Milford Wyeth, 2010)). When two robots
met, they were able to hear unique signals that they emitted and thus
establish that they are in the same location. They started by playing a
_where-are-we_ game in which they named the location of their meeting. After a
few games, the agents had conventionalised the lexicons of location names
grounded in their private spatial maps. The robots then used grounded location
names to play two new games: _how-far_ and _what-direction_. In the how-far
game two locations were named and robots had to negotiate words to express the
distance between them, while in the what-direction game three locations were
named and robots coordinated terms to express an angle between the two
locations from the point of view of the third. After they coordinated their
lexicons, they were able to use both location names and relational vocabulary
to successfully play _go-to_ games when they arranged to meet at a specific
location. Then, in _where-is-there game_ the agents used the relational
vocabulary to name the locations they had not yet visited. The labels
established during this last game remained abstract until the agents were able
to discover previously unknown locations through exploration.

It is worth stressing that two different kinds of grounding of the vocabulary
were present in this model. The first was a direct grounding in the
experiences of spatial locations. The second was an indirect grounding in the
relations between already established concepts. This can be considered as an
example of building abstractions.

The communication complexity in this model is built incrementally starting
from simpler games, and the games are combined in a modular composition schema
(Barrett Skyrms, 2017). This requires a progressive complexification of agent
interactions (here encoded in game structures) and a memory mechanism linking
different concepts and allowing meaningful operations on them (here spatial
maps of the environment).

###  3.11 Situated coordinative dialogue

The structure of the LSG brings to mind a descriptive function of
communication: the sender describes the state of affairs unknown to the
receiver. In contrast, Grouchy . (2016) modelled an embodied and situated
scenario where there are no external states to communicate, only agents that
coordinate their movements in space to meet each other. They simulated
colonies of mobile robots that navigated continuous 2D spaces using
controllers evolved through genetic programming (evolved controllers were
transferred to physical robots: e-pucks). The controllers took the form of
ordinary differential equations that defined the relationship between robot
sensors and actuators. The robots could not see each other but could hear
continuous signals emitted by nearby peers. Whenever two robots met in space,
they crossed their genetic material and produced an offspring. There was no
explicit reward or fitness function, but the reproduction mechanism favoured
robots that were able to find each other more efficiently.

Most populations of robots evolved communication systems in which the emitted
signals corresponded to the robot’s position in one of the two dimensions.
Some populations used this communication strategy for a while but later on
evolved more complex strategies, where signals emitted by two robots became
codependent: a signal emitted by one robot was modulated by the signal emitted
by the other robot. The established dialogical communication between the two
robots became a dynamical control structure facilitating their meeting. The
two modes of communication are dependent on each other in the sense that
dialogical communication strategy emerges out of the simpler one.
Conceptually, this resembles modular composition of signaling games proposed
by Barrett Skyrms (2017), although here the signaling structure is fully
emergent, without explicit roles, rewards, etc.

Grouchy’s et al. model remains very original as it challenges many assumptions
common in communication emergence models: a) it uses continuous signals
instead of discrete signs, b) it does not reward agents explicitly, and c) it
explores a form of communication that is less about representing states of the
world and more about the dynamic coordination of behaviour (with respect to b)
and c), Floreano . (2007) model is also similar).

##  4 Emergence, function of communication, and the mapping metaphor

Reflection on the structure of the existing models demonstrates noble efforts
to investigate various aspects of the communication emergence problem, yet at
the same time reveals several gross categories of simplifications, which –
while useful in addressing concrete engineering problems or even concrete
isolated theoretical questions – might obscure the essential properties of
human communication that need to be addressed in a successful explanatory
model:

  1. 1.

Lack of emergence. In existing models, communication rarely truly emerges:
symbols are predefined, given as discrete numbers or vectors. They usually
form a separate modality, distinct from the perceptual input, which is already
designed for communication by the model structure. The problems of
conventionalization of replicable forms, tokenization of complex utterances,
etc. are left out. The existence of discrete symbols is presupposed rather
than explained. Agents’ actions also tend to be discrete and devoid of
dynamics, preventing the emergence of new forms of organisation. This is
somewhat relaxed in only a couple of models (pointing and “gaze” direction in
Mordatch Abbeel (2018), movement and use of light properties for communication
in Floreano . (2007), continuous signals and movement in Grouchy . (2016)).

  2. 2.

Limiting the function of communication. In most of the existing models,
communication has a single function which is explicitly rewarded. The focus on
the descriptive function of communication is evident in the settings where
there is knowledge asymmetry between agents, and one agent is rewarded for
passing certain information about the world to the other. The imperative or
coordinative function of communication, concerning the regulation of the
ongoing interaction (for example, by eliciting certain behaviours of the
interaction partner), is subordinate to the descriptive function (as in Schulz
. (2012), where robots first learn descriptive communication and then use it
imperatively). A notable exception is the model of Grouchy . (2016), which
demonstrates how coordination can be independent of the description of the
external environment.

  3. 3.

Mapping metaphor. The meaning of language is reduced to mapping discrete
signals to discrete referents. This is visible in the basic LSG, in which the
reward is maximised when the emerging signals can be uniquely mapped to the
states of the world. Compositionality is understood in terms of more complex
structured mappings (Korbak ., 2020). In the simplified environments adopted
by most simulations, actions performed by agents and action history do not
really contribute to the meaning of communication. An effort to break free
from the mapping metaphor is visible in Loula . (2010), where emerging meaning
is understood as a transformation of the ecological niche rather than simple
mapping. Also in Grouchy . (2016) signals allow functional coordination
without mapping to specific referents.

All these simplifications are in line with the basic structure of LSG as seen
through the lenses of Shannon’s information transmission (Skyrms, 2010).
Although some existing models depart from these simplifications (usually not
all three of them at once), in general, they hold surprisingly strong. We
assume that this is due to a widespread approach to symbols, which treats them
as something “given”, which exists independently of meaning, similar to
unbound variables in a computer program ready to be associated with some
values. Symbols are thus perceived as discrete abstract entities with formal
properties that can be manipulated. Years of dominance of the information
processing and computer metaphor in the cognitive sciences result in thinking
about symbols as necessary assumptions prior to constructing explanations.
This makes it easy to forget that their existence also needs to be explained.
Consequently, the explanatory thrust is towards how we can manipulate symbols,
combine them, and decipher their meaning. But the problem of the emergence of
complex communication essentially includes the problem of symbol emergence:
how do they come into being in the first place? What kinds of meaningful
relations do they require to be described as symbols (Rączaszek-Leonardi .,
2018)?

Assuming abstract symbols with arbitrary meanings as a starting point is also
limiting because it puts the burden of structuring communication on the
conventions created from scratch by the agents and realised by their internal
architecture, neglecting or limiting the role that the environment and
interactions with it perform. In the real world, the sources of forces that
shape and structure communication are multiple. For instance, physical
properties of the signals also take part in determining their meaning (like
the light direction in Floreano . (2007)), situated agents can only
communicate effectively within some distance (as in Loula . (2010)), etc.
Neglecting the ways symbols might be created in interactions within a
particular ecology obscures these sources of complexity, leaving us only with
the simplified view of communication meaning as mapping from symbols to
features (Rączaszek-Leonardi Deacon, 2018).

##  5 Sources of communication complexity

Following the above review and the summary of theoretical considerations, we
identify some of the most important sources of communication complexity and
formulate the desiderata for a successful model that includes them.

#### Perceptual grounding

Grounding language in perception distinguishes most of the emergent
communication models from distributional semantic approaches, where language
meaning is derived only from word co-occurrences. In perceptually grounded
models, the agent’s perception affects the language structure. For example, in
Steels Belpaeme (2005) vocabulary structure depends on the size of just
noticeable difference in agent’s perception. Models aiming to explore aspects
of perceptual grounding should aim for realistic stimuli (for instance,
pictures represented as raw RGB arrays as in Lazaridou . (2018)), but also
realistic perceptual systems including multimodality, inherent limitations,
perceptual noise, etc.

#### Sensorimotor grounding

Another level is grounding language not only in passive perception, but in
experiences gained from active exploration of the environment (active
perception, Gibson (1966)). This requires an embodied agent capable of
nontrivial actions that affect states of the world that later affect the
perception of the agent (closing the sensorimotor loop). Simple extensions of
the signaling game model or machine learning models inspired by image
captioning or question answering tasks preclude that. Sensorimotor grounding
is possible in embodied simulations, such as Mordatch Abbeel (2018), where
agents move in the environment and perform sequences of actions, which in turn
opens up additional possibilities for language complexification.

#### Properties of sign vehicles

Communication complexity can also be induced by the physical properties of
sign vehicles (signals) used in communication. As we observed before, many
models opt for discrete and amodal signs and do not explore this dimension of
communication complexity at all. In embodied models where signals have
physical vehicles, such as the model of Floreano . (2007) where light signals
are used, the properties of vehicles and the properties of the perceptual
systems of agents interact. This creates new constraints on the interaction
scenarios, for example communication being possible only in vicinity. In
addition, physical sign vehicles allow us to explore aspects of
iconicity—similarity between a sign vehicle and its referent (be it an object,
action, event, etc.)—which cannot be studied with amodal signs.

#### Complexity of the environment

It is generally recognised that, in order for complex communication to emerge,
a sufficiently complex environment is needed. Within the basic signaling game
schema, degrees of freedom of signals are limited by the degrees of freedom of
the stimuli. Simulations which deal with natural images as stimuli explore
this dimension of complexity, although as Bouchacourt Baroni (2018)
demonstrated using complex stimuli does not automatically mean that agents
will arrive at meaningful conceptual representations. Another dimension of
environment complexity, which appears in more embodied simulations, is the
inherent environment dynamics: how does the state and/or changes in the the
environment affect the agents and how, in turn, they are affected by it. All
simulations that incorporate agent movement tap into this dimension. Finally,
the environment may be fully observable—when the agent has full access to the
environment state—or only partially observable. Even embodied simulations in
the style of Mordatch Abbeel (2018) tend to prefer fully observable
environments, which are easier to learn by agents and in which communication
is less ambiguous. The ambiguity introduced by partially observable
environments may add to the complexity of communication.

#### Agent architecture

Internal agent’s architecture governs the available learning mechanisms.
Humans are said to exhibit certain biases related, among others, to the
limitations of working memory (Baddeley, 2003). As we know from the No-Free-
Lunch Theorem (Wolpert, 1996), every learning algorithm has a preference
towards a certain class of problems and thus necessarily introduces a bias
favouring certain communication protocols over others. The strength of this
bias is perhaps moderated by the flexibility of the algorithm. A simple neural
network without a hidden layer used in the work of Smith . (2003) could only
represent the additively linear influence of object properties on signal
components, while a multilayer network used in more recent studies could
represent a wider class of relations (Ren ., 2020).

The scope of agent behaviour governed by learning differs between models. In
the works of Mordatch Abbeel (2018) or Grouchy . (2016) an end-to-end learning
is used, where all aspects of behaviour are governed by the same mechanism. In
contrast, Schulz . (2012) and Loula . (2010) use specialised vocabulary
learning mechanisms, representing other aspects of agent activity through
preprogrammed procedures. On the one hand, these preprogrammed procedures
structure the agent’s interaction and, therefore, are a scaffolding and a
source of structure for communication. On the other hand, the separation of
agent activity into modules may preclude deeper interactions between
communication, action, and perception.

There are also different design decisions in the implementations of learning
algorithm depending on the notion of what is realistic in a simulation. For
example, Mordatch Abbeel (2018) use direct weight sharing between agents and
full knowledge of the dynamics of the environment in the optimisation
procedure that represents learning. This makes agents learning efficient but
raises the question whether it does not circumvent some important challenges
in language learning and miss additional sources of structure.

#### Communication function, agent goals and autonomy

Obviously, communication can only evolve if it is useful in some sense. It
does not seem that there is a single universal use of communication in the
natural world. Communication appears in different forms, both when
communicating individuals have a common goal and when the goals are different,
in the context of cooperation between individuals and competition between
them, both as faithful communication and as deceptive (Wacewicz Żywiczyński,
2018). In computational simulations, all these aspects are usually modelled as
the fitness/cost function. What is included explicitly in the cost function is
a nontrivial question. In the basic signaling game, successful communication
is the goal of its own and is rewarded directly and symmetrically, while in
Cangelosi (2001) or Floreano . (2007) it is only a means to successfully
locate the food and the reward is different for the speaker and the receiver.
Most models consider cooperative scenarios, although some explore the
conditions under which deceptive communication appears (Mitri ., 2009). In
Grouchy . (2016) there is no explicit cost function at all. Instead,
artificial agents are allowed to reproduce whenever two of them meet in the
same place (analogue of sexual reproduction); communication arises from
evolutionary pressure. We argue that agents in this kind of simulation have
more autonomy and a higher chance of evolving a communication system beyond
the expectations of the modeler. On the other side of the spectrum there are
simulations in which aspects of the communication protocol are directly
included in the cost function, such as the pressure to keep the vocabulary
small in Mordatch Abbeel (2018).

#### Social adaptations and social relationships

Somewhat related to the above, pressures on the structure of communication
arise from living in a society. Social pressures may come in two varieties: as
special adaptations of individuals to live in a society, or as social
scaffolding shaping the interactions of individuals. Adaptations for social
behaviour in the form of mechanisms loosely inspired by the Theory of Mind are
included in some models of emergent communication (Batali, 1998; Jaques .,
2019; Bard ., 2020). They are reflected in the agent’s internal architecture
and affect the evolved communication systems. Then, social structures in the
form of social networks (Zubek ., 2017), social ecology (Loula ., 2010) or
kinship structures (Floreano ., 2007; Mitri ., 2009) are shown to be important
for the shape of emerging signals. Still, many works relying on the basic
signaling game are limited to a one-sided communication between a pair of
agents and do not study any such aspects of sociality.

#### History of interactions

As with all complex evolving phenomena, natural languages bear the mark of
their long history of evolution. The current shape of the communication system
may be caused by the particular historical trajectory of the changing
conditions that shaped it. On the level of individual language learning, there
is a historical trajectory of language development that goes through different
stages. Historicity understood in this way is explored only by a couple of
models of emergent communication. The paradigm of iterated learning, which
models vertical language transmission, taps into some aspects of historicity
and the information bottleneck associated with it. In Schulz . (2012) language
games were played in a certain order, and the terms grounded using previous
games were used in subsequent games. Zubek . (2017) studied reactions of a
population of agents to a certain trajectory of changes in the environment and
demonstrated that the structure of language categories is affected by history.
Korbak . (2021) presented a developmentally motivated model in which an agent
goes through two stages of learning, which helps it develop a compositional
language.

##  6 Embodied models combine multiple sources of complexity

Regarding different sources of communication complexity, we may treat them as
semi-independent. It is reasonable that some models explicitly focus on a
single source and attempt to isolate it from the others. This makes the model
easier to interpret and helps to demonstrate the importance of the particular
source. For example, Steels Belpaeme (2005) study perceptual grounding with
minimal agents in an environment limited to colour stimuli. However, this
choice should be conscious and well motivated and it should be recognised that
the sources of complexity may be hard to isolate, as they can be easily
conflated when sufficient complexity is introduced. Models which take
seriously the notions of embodiment and situatedness operate under the
assumptions which are of consequence for most sources of communication
complexity listed above.

For instance, in Floreano . (2007) physical space constraints (inability of
multiple robots to occupy the same spot) had consequences for the socio-
ecological function of communication. In this work, the robots had to locate
and attach themselves to a food source, but because of the limited space only
a certain number of robots could feed simultaneously. Thus, communicating the
location of food posed the risk that some other agent would exploit the scarce
resource. This implicit cost of communication, depending on the evolutionary
selection mechanisms, resulted in various communication strategies, from
honest signaling to deceptive communication (Mitri ., 2009)). In some cases,
the robots also learnt to exploit the physical properties of the medium of
communication: they hid emitted light signals from the other robots by hiding
behind the food source (Lehman ., 2018). This is an example of an emergent
behaviour, which could not evolve in a simpler environment.

The environment adopted by Grouchy . (2016) was minimalistic in the sense that
it is devoid of any objects apart from the agents. Still, the agents were
situated in the sense that they could freely move in continuous space and use
their own locations as a context for communication. Importantly, the emitted
signals were continuous and were interpreted as such. This opened new
possibilities for meaning emergence, as the signal intensity could convey
information on the distance in space (basic iconic form of significance). What
emerged was not a static signal-referent mapping, which can be used to
interpret only previously encountered signals, but a dynamical structure using
signals to control agent actions that was capable of generalisation.
Ultimately, that allowed for the dialogical form of communication where the
signals of both agents became entangled to emerge.

Situatedness is also naturally connected with aspects of perceptual and
sensorimotor grounding. It was shown that situating agents in more complex
environments has a positive influence on systematic generalisation. Although
not directly addressing the problem of language emergence, Hill . (2020)
presented a model in which they studied the effects of embodiment on language
comprehension. Their agent had to navigate through the environment and perform
actions according to the instructions given as a sentence in natural language.
They demonstrated that (i) egocentric perspective (as opposed to third-person
view), (ii) 3d simulated environments (as opposed to 2d grid worlds) and (iii)
active perception (as opposed to perceiving the stimuli statically)
incentivise the agent to factorise experience and behaviours into
pragmatically motivated reusable chunks, leading to better generalisation as
shown in experiments isolating factors (i)-(iii). These results provide an
explanation of previous results, showing a low systematic generalisation in
recurrent neural networks trained on symbolically encoded stimuli from a
single modality (Lake Baroni, 2017). Hill . (2020) hypothesise that this
effect arises because richer experience is a form of implicit data
augmentation. This suggests that the human capacity to exploit the structure
of the world, when learning to generalise in systematic ways, could be
replicated in artificial neural networks if those networks are given access to
a rich, interactive, multimodal stream of stimuli that better matches the
experience of an embodied human learner.

##  7 Conclusion

Models of communication emergence come from multiple traditions, which
explains the differences in terminology and emphasis. The lack of systematic
understanding of the assumptions behind various models contributes to a
barrier preventing researchers from one domain from building on the modelling
efforts from another domain. We hope that our review will be helpful in
building such a systematic understanding.

As is made clear by our analysis, the pressures shaping communication signs
and symbols are many, and building a model means bracketing some of them out.
Sometimes, however, the bracketing is too heavy and too hasty, as it may
remove the vital elements of the explanation. Perhaps the fascination with one
particular kind of a language game, involving a sender, a receiver, and
clearly distinguished referents, made us forget that games people play are
multiple (Berne, 1964). We propose a change of perspective to the dynamic one
advocating embodied and situated models, which naturally accommodate multiple
forces, make room for recognising the historicity of communication systems,
and do not presuppose the existence of readily distinguishable symbols. This
requires freeing language from its sole role of world-describer to a broader
function of interaction controller. Under such a view, the mapping metaphor is
no longer adequate to explain the meaning of language and cannot be used as a
foundation for communication emergence models. Instead, we propose to pay more
attention to models where meaning emerges as a transformation of agents
interaction in an ecological niche.

## Declarations

This work was funded by National Science Centre, Poland (OPUS 15 grant,
2018/29/B/HS1/00884). The funding source had no role in data collection and
manuscript preparation. The authors declare no competing interests.

Authors’ contributions: JZ, TK, JRL contributed to the conception, writing and
editing of the manuscript.

## References

  *   * Andreas (2019) andreas2019Andreas, J.  201904\.  Measuring Compositionality in Representation Learning Measuring Compositionality in Representation Learning ( arXiv:1902.07181).  arXiv.
  * Baddeley (2003) baddeley2003Baddeley, A.  200305\.  Working Memory and Language: An Overview Working memory and language: An overview.  Journal of Communication Disorders363189–208.
  * Bard . (2020) bard2020Bard, N., Foerster, J.N., Chandar, S., Burch, N., Lanctot, M., Song, H.F.Bowling, M.  202003\.  The Hanabi Challenge: A New Frontier for AI Research The Hanabi challenge: A new frontier for AI research.  Artificial Intelligence280103216.
  * Baronchelli . (2015) baronchelli2015Baronchelli, A., Loreto, V. Puglisi, A.  201505\.  Individual Biases, Cultural Evolution, and the Statistical Nature of Language Universals: The Case of Colour Naming Systems Individual Biases, Cultural Evolution, and the Statistical Nature of Language Universals: The Case of Colour Naming Systems.  PLOS ONE105e0125019.
  * Barrett Skyrms (2017) barrett2017Barrett, J.A. Skyrms, B.  201706\.  Self-Assembling Games Self-assembling Games.  The British Journal for the Philosophy of Science682329–353.
  * Barrett . (2018) barrett2018Barrett, J.A., Skyrms, B. Cochran, C.  201805\.  Hierarchical Models for the Evolution of Compositional Language. Hierarchical Models for the Evolution of Compositional Language.  PhilSci.
  * Barrett . (2019) barrett2019Barrett, J.A., Skyrms, B. Mohseni, A.  201903\.  Self-Assembling Networks Self-Assembling Networks.  The British Journal for the Philosophy of Science701301–325.
  * Batali (1998) batali1998Batali, J.  1998\.  Computational Simulations of the Emergence of Grammar Computational simulations of the emergence of grammar.  Approaches to the Evolution of Language-Social and Cognitive Bases-.
  * Berne (1964) berne1964Berne, E.  1964\.  Games People Play: The Psychology of Human Relationships Games people play: The psychology of human relationships.  New YorkGrove Press.
  * Bogin . (2019) bogin2019Bogin, B., Geva, M. Berant, J.  201903\.  Emergence of Communication in an Interactive World with Consistent Speakers Emergence of Communication in an Interactive World with Consistent Speakers ( arXiv:1809.00549).  arXiv.
  * Bouchacourt Baroni (2018) bouchacourt2018Bouchacourt, D. Baroni, M.  201809\.  How Agents See Things: On Visual Representations in an Emergent Language Game How agents see things: On visual representations in an emergent language game ( arXiv:1808.10696).  arXiv.
  * Brighton (2002) brighton2002Brighton, H.  200201\.  Compositional Syntax From Cultural Transmission Compositional Syntax From Cultural Transmission.  Artificial Life8125–54.
  * Brighton . (2005) brighton2005Brighton, H., Smith, K. Kirby, S.  200509\.  Language as an Evolutionary System Language as an evolutionary system.  Physics of Life Reviews23177–226.
  * Cangelosi (1999) cangelosi1999Cangelosi, A.  1999\.  Modeling the Evolution of Communication: From Stimulus Associations to Grounded Symbolic Associations Modeling the Evolution of Communication: From Stimulus Associations to Grounded Symbolic Associations.  G. Goos, J. Hartmanis, J. van Leeuwen, D. Floreano, JD. Nicoud F. Mondada (), Advances in Artificial Life Advances in Artificial Life ( 1674, 654–663).  Berlin, HeidelbergSpringer Berlin Heidelberg.
  * Cangelosi (2001) cangelosi2001Cangelosi, A.  200104\.  Evolution of Communication and Language Using Signals, Symbols, and Words Evolution of communication and language using signals, symbols, and words.  IEEE Transactions on Evolutionary Computation5293–101.
  * Cangelosi . (2002) cangelosi2002Cangelosi, A., Greco, A. Harnad, S.  2002\.  Symbol Grounding and the Symbolic Theft Hypothesis Symbol Grounding and the Symbolic Theft Hypothesis.  A. Cangelosi D. Parisi (), Simulating the Evolution of Language Simulating the Evolution of Language ( 191–210).  LondonSpringer London.
  * Centola Baronchelli (2015) centola2015Centola, D. Baronchelli, A.  201502\.  The Spontaneous Emergence of Conventions: An Experimental Study of Cultural Evolution The spontaneous emergence of conventions: An experimental study of cultural evolution.  Proceedings of the National Academy of Sciences11271989–1994.
  * Chaabouni . (2020) chaabouni2020Chaabouni, R., Kharitonov, E., Bouchacourt, D., Dupoux, E. Baroni, M.  202001\.  Compositionality and Generalization In Emergent Languages Compositionality and Generalization In Emergent Languages.  ( 4427–4442).
  * Choi . (2018) choi2018aChoi, E., Lazaridou, A. de Freitas, N.  201804\.  Compositional Obverter Communication Learning From Raw Visual Input Compositional Obverter Communication Learning From Raw Visual Input ( arXiv:1804.02341).  arXiv.
  * Chomsky (1957) chomsky1957Chomsky, N.  1957\.  Syntactic Structures Syntactic Structures.  Mouton.
  * Christiansen Chater (2022) christiansen2022Christiansen, M.H. Chater, N.  2022\.  The Language Game: How Improvisation Created Language and Changed the World The Language Game: How Improvisation Created Language and Changed the World.  New YorkBasic Books.
  * Cogswell . (2020) cogswell2020Cogswell, M., Lu, J., Lee, S., Parikh, D. Batra, D.  202005\.  Emergence of Compositional Language with Deep Generational Transmission Emergence of Compositional Language with Deep Generational Transmission ( arXiv:1904.09067).  arXiv.
  * Correia (2020) correia2020Correia, J.P.F.e.V.  2020\.  Meaningful Play: Signaling Games in Light of Later Wittgenstein Meaningful play: Signaling games in light of later Wittgenstein .  Porto, PortugalUniversidade do Porto.
  * Dale Lupyan (2012) dale2012Dale, R. Lupyan, G.  201205\.  Understanding the Origins of Morphological Diversity: The Linguistic Niche Hypothesis Understanding the origins of morphological diversity: The linguistic niche hypothesis.  Advances in Complex Systems15.
  * Deacon (1997) deacon1997Deacon, T.W.  1997\.  The Symbolic Species The symbolic species.  New York: Norton.
  * Deacon (2011) deacon2011Deacon, T.W.  2011\.  Incomplete Nature : How Mind Emerged from Matter Incomplete nature : How mind emerged from matter.  New York : W.W. Norton & Co.
  * De Boer Zuidema (2010) deboer2010De Boer, B. Zuidema, W.  2010\.  Multi-Agent Simulations of the Evolution of Combinatorial Phonology Multi-agent simulations of the evolution of combinatorial phonology.  Adaptive Behavior182141–154.
  * Devitt (2003) devitt2003Devitt, M.  200310\.  Linguistics Is Not Psychology Linguistics is not Psychology.  A. Barber A. Barber (), Epistemology of Language Epistemology of Language ( 107–139).  Oxford, New YorkOxford University Press.
  * Di Paolo . (2018) dipaolo2018Di Paolo, E.A., Cuffari, E.C. De Jaegher, H.  2018\.  Linguistic Bodies: The Continuity between Life and Language Linguistic Bodies: The Continuity between Life and Language.
  * Doshi-Velez Kim (2017) doshi-velez2017Doshi-Velez, F. Kim, B.  201703\.  Towards A Rigorous Science of Interpretable Machine Learning Towards A Rigorous Science of Interpretable Machine Learning ( arXiv:1702.08608).  arXiv.
  * Elman (1993) elman1993Elman, J.L.  1993\.  Learning and Development in Neural Networks: The Importance of Starting Small Learning and development in neural networks: The importance of starting small.  Cognition48171–99.
  * Ferri . (2018) ferri2018Ferri, F., D’Ulizia, A. Grifoni, P.  201810\.  Computational Models of Language Evolution: Challenges and Future Perspectives Computational Models of Language Evolution: Challenges and Future Perspectives.  JOURNAL OF UNIVERSAL COMPUTER SCIENCE241345–1377.
  * Floreano . (2007) floreano2007Floreano, D., Mitri, S., Magnenat, S. Keller, L.  200703\.  Evolutionary Conditions for the Emergence of Communication in Robots Evolutionary Conditions for the Emergence of Communication in Robots.  Current Biology176514–519.
  * Funkhouser (2017) funkhouser2017Funkhouser, E.  201708\.  Beliefs as Signals: A New Function for Belief Beliefs as signals: A new function for belief.  Philosophical Psychology306809–831.
  * Gibson (1966) gibson1966Gibson, J.J.  1966\.  The Senses Considered as Perceptual Systems The Senses Considered as Perceptual Systems.  Houghton Mifflin.
  * Grifoni . (2016) grifoni2016Grifoni, P., D’Ulizia, A. Ferri, F.  201603\.  Computational Methods and Grammars in Language Evolution: A Survey Computational methods and grammars in language evolution: A survey.  Artificial Intelligence Review453369–403.
  * Grouchy . (2016) grouchy2016Grouchy, P., D’Eleuterio, G.M.T., Christiansen, M.H. Lipson, H.  201610\.  On The Evolutionary Origin of Symbolic Communication On The Evolutionary Origin of Symbolic Communication.  Scientific Reports6134615.
  * Harnad (1990) harnad1990Harnad, S.  199006\.  The Symbol Grounding Problem The symbol grounding problem.  Physica D: Nonlinear Phenomena421335–346.
  * Hill . (2020) hill2020Hill, F., Lampinen, A., Schneider, R., Clark, S., Botvinick, M., McClelland, J.L. Santoro, A.  202002\.  Environmental Drivers of Systematicity and Generalization in a Situated Agent Environmental drivers of systematicity and generalization in a situated agent ( arXiv:1910.00571).  arXiv.
  * Jaques . (2019) jaques2019Jaques, N., Lazaridou, A., Hughes, E., Gulcehre, C., Ortega, P.A., Strouse, D.J.de Freitas, N.  201906\.  Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning ( arXiv:1810.08647).  arXiv.
  * Karpathy Fei-Fei (2017) karpathy2017Karpathy, A. Fei-Fei, L.  201704\.  Deep Visual-Semantic Alignments for Generating Image Descriptions Deep Visual-Semantic Alignments for Generating Image Descriptions.  IEEE Transactions on Pattern Analysis and Machine Intelligence394664–676.
  * Kempson . (2000) kempson2000Kempson, R., Meyer-Viol, W. Gabbay, D.M.  2000\.  Dynamic Syntax: The Flow of Language Understanding Dynamic Syntax: The Flow of Language Understanding (1st edition ).  Oxford England ; Malden, MassWiley-Blackwell.
  * Kirby . (2008) kirby2008Kirby, S., Cornish, H. Smith, K.  200808\.  Cumulative Cultural Evolution in the Laboratory: An Experimental Approach to the Origins of Structure in Human Language Cumulative cultural evolution in the laboratory: An experimental approach to the origins of structure in human language.  Proceedings of the National Academy of Sciences1053110681–10686.
  * Kirby . (2014) kirby2014Kirby, S., Griffiths, T. Smith, K.  201410\.  Iterated Learning and the Evolution of Language Iterated learning and the evolution of language.  Current Opinion in Neurobiology28108–114.
  * Korbak . (2021) korbak2021aKorbak, T., Zubek, J., Kuciński, Ł., Miłoś, P. Rączaszek-Leonardi, J.  202112\.  Interaction History as a Source of Compositionality in Emergent Communication Interaction history as a source of compositionality in emergent communication.  Interaction Studies222212–243.
  * Korbak . (2020) korbak2020Korbak, T., Zubek, J. Rączaszek-Leonardi, J.  2020\.  Measuring Non-Trivial Compositionality in Emergent Communication Measuring non-trivial compositionality in emergent communication.  arXiv preprint arXiv:2010.15058.  arxiv:2010.15058
  * Kottur . (2017) kottur2017aKottur, S., Moura, J.M.F., Lee, S. Batra, D.  201708\.  Natural Language Does Not Emerge ’Naturally’ in Multi-Agent Dialog Natural Language Does Not Emerge ’Naturally’ in Multi-Agent Dialog ( arXiv:1706.08502).  arXiv.
  * LaCroix (2022) lacroix2022LaCroix, T.  202206\.  Using Logic to Evolve More Logic: Composing Logical Operators via Self-Assembly Using Logic to Evolve More Logic: Composing Logical Operators via Self-Assembly.  The British Journal for the Philosophy of Science732407–437.
  * Lake Baroni (2017) lake2017Lake, B. Baroni, M.  201710\.  Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks.  International Conference on Machine Learning. International Conference on Machine Learning.
  * Lazaridou . (2018) lazaridou2018aLazaridou, A., Hermann, K.M., Tuyls, K. Clark, S.  201804\.  Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input ( arXiv:1804.03984).  arXiv.
  * Lazaridou . (2017) lazaridou2017Lazaridou, A., Peysakhovich, A. Baroni, M.  201703\.  Multi-Agent Cooperation and the Emergence of (Natural) Language Multi-Agent Cooperation and the Emergence of (Natural) Language ( arXiv:1612.07182).  arXiv.
  * Lee . (2018) lee2018Lee, J., Cho, K., Weston, J. Kiela, D.  201804\.  Emergent Translation in Multi-Agent Communication Emergent Translation in Multi-Agent Communication ( arXiv:1710.06922).  arXiv.
  * Lehman . (2018) lehman2018Lehman, J., Clune, J. Misevic, D.  201807\.  The Surprising Creativity of Digital Evolution The Surprising Creativity of Digital Evolution.  ALIFE 2018: The 2018 Conference on Artificial Life ALIFE 2018: The 2018 Conference on Artificial Life ( 55–56).  MIT Press.
  * Lewis (1969) lewis1969Lewis, D.K.  1969\.  Convention: A Philosophical Study Convention: A Philosophical Study.  Cambridge, MA, USA: Wiley-Blackwell.
  * Li Bowling (2019) li2019Li, F. Bowling, M.  201910\.  Ease-of-Teaching and Language Structure from Emergent Communication Ease-of-Teaching and Language Structure from Emergent Communication ( arXiv:1906.02403).  arXiv.
  * Loula . (2010) loula2010Loula, A., Gudwin, R., El-Hani, C.N. Queiroz, J.  201006\.  Emergence of Self-Organized Symbol-Based Communication in Artificial Creatures Emergence of self-organized symbol-based communication in artificial creatures.  Cognitive Systems Research112131–147.
  * Milford Wyeth (2010) milford2010Milford, M. Wyeth, G.  201008\.  Persistent Navigation and Mapping Using a Biologically Inspired SLAM System Persistent Navigation and Mapping using a Biologically Inspired SLAM System.  The International Journal of Robotics Research2991131–1153.
  * Mitri . (2009) mitri2009Mitri, S., Floreano, D. Keller, L.  200909\.  The Evolution of Information Suppression in Communicating Robots with Conflicting Interests The evolution of information suppression in communicating robots with conflicting interests.  Proceedings of the National Academy of Sciences1063715786–15790.
  * Mordatch Abbeel (2018) mordatch2018Mordatch, I. Abbeel, P.  201807\.  Emergence of Grounded Compositional Language in Multi-Agent Populations Emergence of Grounded Compositional Language in Multi-Agent Populations.  arXiv:1703.04908 [cs].  arxiv:1703.04908 [cs]
  * Mostafazadeh . (2017) mostafazadeh2017Mostafazadeh, N., Brockett, C., Dolan, B., Galley, M., Gao, J., Spithourakis, G.P. Vanderwende, L.  201704\.  Image-Grounded Conversations: Multimodal Context for Natural Question and Response Generation Image-Grounded Conversations: Multimodal Context for Natural Question and Response Generation ( arXiv:1701.08251).  arXiv.
  * Nowak Krakauer (1999) nowak1999Nowak, M.A. Krakauer, D.C.  199907\.  The Evolution of Language The evolution of language.  Proceedings of the National Academy of Sciences96148028–8033.
  * Oliphant Batali (1997) oliphant1997Oliphant, M. Batali, J.  1997\.  Learning and the Emergence of Coordinated Communication Learning and the Emergence of Coordinated Communication..
  * Parisi . (2008) parisi2008Parisi, D., Antinucci, F., Natale, F. Cecconi, F.  200801\.  Simulating the Expansion of Farming and the Differentiation of European Languages Simulating the expansion of farming and the differentiation of European languages.  B. Laks, Serge Cleuziou, JP. Demoule P. Encreve (), Origin and Evolution of Languages: Approaches, Models, Paradigms. Origin and evolution of languages: Approaches, models, paradigms.  London, UKEquinox.
  * Pattee (1969) pattee1969Pattee, H.  196901\.  How Does a Molecule Become a Message How does a molecule become a message.  Developmental Biology Supplement.
  * Pattee (1982) pattee1982Pattee, H.  1982\.  Cell Psychology: An Evolutionary Approach to the Symbol-Matter Problem Cell psychology: An evolutionary approach to the symbol-matter problem.  Cognition and Brain Theory54325–341.
  * Quine (1936) quine1936Quine, W.V.  1936\.  Truth by Convention Truth by Convention.  Philosophical Essays for Alfred North Whitehead Philosophical Essays for Alfred North Whitehead ( 90–124).  London: Longmans, Green & Co.
  * Ren . (2020) ren2020Ren, Y., Guo, S., Labeau, M., Cohen, S.B. Kirby, S.  202002\.  Compositional Languages Emerge in a Neural Iterated Learning Model Compositional Languages Emerge in a Neural Iterated Learning Model.  ArXiv.
  * Rączaszek-Leonardi (2003) raczaszek-leonardi2003Rączaszek-Leonardi, J.  2003\.  The Interrelation of Time Scales in a Description of Language The interrelation of time scales in a description of language.  Views & Voices1293–108.
  * Rączaszek-Leonardi (2009) raczaszek-leonardi2009Rączaszek-Leonardi, J.  2009\.  Metodologiczne Podstawy Współczesnej Psycholingwistyki Metodologiczne podstawy współczesnej psycholingwistyki.  P. Stalmaszczyk (), Metodologie Językoznawstwa. Współczesne Tendencje i Kontrowersje. Metodologie językoznawstwa. Współczesne tendencje i kontrowersje. ( 236–251).  KrakówLexis.
  * Rączaszek-Leonardi Deacon (2018) raczaszek-leonardi2018cRączaszek-Leonardi, J. Deacon, T.W.  201809\.  Ungrounding Symbols in Language Development: Implications for Modeling Emergent Symbolic Communication in Artificial Systems Ungrounding symbols in language development: Implications for modeling emergent symbolic communication in artificial systems.  2018 Joint IEEE 8th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob) 2018 Joint IEEE 8th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob) ( 232–237).  Tokyo, JapanIEEE.
  * Rączaszek-Leonardi Kelso (2008) raczaszek-leonardi2008Rączaszek-Leonardi, J. Kelso, J.S.  2008\.  Reconciling Symbolic and Dynamic Aspects of Language: Toward a Dynamic Psycholinguistics Reconciling symbolic and dynamic aspects of language: Toward a dynamic psycholinguistics.  New ideas in psychology262193–207.
  * Rączaszek-Leonardi . (2018) raczaszek-leonardi2018bRączaszek-Leonardi, J., Nomikou, I., Rohlfing, K.J. Deacon, T.W.  201801\.  Language Development From an Ecological Perspective: Ecologically Valid Ways to Abstract Symbols Language Development From an Ecological Perspective: Ecologically Valid Ways to Abstract Symbols.  Ecological Psychology30139–73.
  * Savage-Rumbaugh Rumbaugh (1978) savage-rumbaugh1978Savage-Rumbaugh, E.S. Rumbaugh, D.M.  197811\.  Symbolization, Language, and Chimpanzees: A Theoretical Reevaluation Based on Initial Language Acquisition Processes in Four Young Pan Troglodytes Symbolization, language, and chimpanzees: A theoretical reevaluation based on initial language acquisition processes in four young Pan troglodytes.  Brain and Language63265–300.
  * Savage-Rumbaugh . (1980) savage-rumbaugh1980Savage-Rumbaugh, E.S., Rumbaugh, D.M., Smith, S.T. Lawson, J.  198011\.  Reference: The Linguistic Essential Reference: The Linguistic Essential.  Science2104472922–925.
  * Schulz . (2012) schulz2012Schulz, R., Wyeth, G. Wiles, J.  201210\.  Beyond Here-and-Now: Extending Shared Physical Experiences to Shared Conceptual Experiences Beyond here-and-now: Extending shared physical experiences to shared conceptual experiences.  Adaptive Behavior205360–387.
  * Shannon (1948) shannon1948Shannon, C.E.  1948\.  A Mathematical Theory of Communication A Mathematical Theory of Communication.  Bell System Technical Journal273379–423.
  * Skyrms (2010) skyrms2010Skyrms, B.  2010\.  Signals: Evolution, Learning, and Information Signals: Evolution, Learning, and Information.  Oxford, New YorkOxford University Press.
  * Smith . (2003) smith2003Smith, K., Kirby, S. Brighton, H.  2003\.  Iterated Learning: A Framework for the Emergence of Language Iterated learning: A framework for the emergence of language.  Artificial Life94371–386.
  * Spence (1973) spence1973Spence, M.  197308\.  Job Market Signaling Job Market Signaling.  The Quarterly Journal of Economics873355–374.
  * Spike . (2017) spike2017Spike, M., Stadler, K., Kirby, S. Smith, K.  201704\.  Minimal Requirements for the Emergence of Learned Signaling Minimal Requirements for the Emergence of Learned Signaling.  Cognitive Science413623–658.
  * Steels Belpaeme (2005) steels2005Steels, L. Belpaeme, T.  200508\.  Coordinating Perceptually Grounded Categories through Language: A Case Study for Colour Coordinating perceptually grounded categories through language: A case study for colour.  Behavioral and Brain Sciences284469–489.
  * Vogt (2009) vogt2009Vogt, P.  200904\.  Modeling Interactions between Language Evolution and Demography Modeling Interactions between Language Evolution and Demography.  Human Biology812-3237–258.
  * Wacewicz Żywiczyński (2018) wacewicz2018Wacewicz, S. Żywiczyński, P.  201809\.  Language Origins: Fitness Consequences, Platform of Trust, Cooperation, and Turn-Taking Language origins: Fitness consequences, platform of trust, cooperation, and turn-taking.  Interaction Studies191-2167–182.
  * Weaver (1949) weaver1949Weaver, W.  1949\.  Recent Contributions to the Mathematical Theory of Communication Recent contributions to the mathematical theory of communication.  The Mathematical Theory of Communication. The Mathematical Theory of Communication.  Champaign, ILUniversity of Illinois Press.
  * Wittgenstein (1953) wittgenstein1953Wittgenstein, L.  1953\.  Philosophical Investigations Philosophical Investigations.  OxfordBasil Blackwell.
  * Wolpert (1996) wolpert1996Wolpert, D.  199603\.  The Lack of A Priori Distinctions Between Learning Algorithms The Lack of A Priori Distinctions Between Learning Algorithms.  Neural Computation8.
  * Zubek . (2017) zubek2017aZubek, J., Denkiewicz, M., Barański, J., Wróblewski, P., Rączaszek-Leonardi, J. Plewczynski, D.  201708\.  Social Adaptation in Multi-Agent Model of Linguistic Categorization Is Affected by Network Information Flow Social adaptation in multi-agent model of linguistic categorization is affected by network information flow.  PLOS ONE128e0182490.