*format=pnasformat

# Modal clustering on PPGMMGA projection subspace

Luca Scrucca
Department of Economics
Università degli Studi di Perugia
Via A. Pascoli 20, 06123 Perugia, Italy
<EMAIL_ADDRESS>
https://orcid.org/0000-0003-3826-0484

###### Abstract

PPGMMGA is a Projection Pursuit (PP) algorithm aimed at detecting and
visualizing clustering structures in multivariate data. The algorithm uses the
negentropy as PP index obtained by fitting Gaussian Mixture Models (GMMs) for
density estimation and, then, exploits Genetic Algorithms (GAs) for its
optimization. Since the PPGMMGA algorithm is a dimension reduction technique
specifically introduced for visualization purposes, cluster memberships are
not explicitly provided. In this paper a modal clustering approach is proposed
for estimating clusters of projected data points. In particular, a modal EM
algorithm is employed to estimate the modes corresponding to the local maxima
in the projection subspace of the underlying density estimated using
parsimonious GMMs. Data points are then clustered according to the domain of
attraction of the identified modes. Simulated and real data are discussed to
illustrate the proposed method and evaluate the clustering performance.

Abstract

Keywords Gaussian mixtures models; projection pursuit; negentropy index;
genetic algorithms; modal EM; modal clustering.

## 1\. Introduction

Finite mixture models are an important tool in statistical data analysis.
Because of their flexibility, they have been successfully used for different
purposes, from modelling of the heterogeneity to clustering, from supervised
classification to semiparametric density estimation (McLachlan and Peel,
2000a, ; McLachlan et al.,, 2019). Model-based clustering is often used as a
synonym to indicate the use of mixture models to investigate the presence of
clusters in the data.

Projection Pursuit (PP) refers to a collection of algorithms aimed at finding
interesting structures in the data (Friedman and Tukey,, 1974; Friedman,,
1987). Often, the main interest consists in detecting the presence of
homogeneous and well separated groups of data. Recently, a projection pursuit
algorithm based on Gaussian Mixture Models (GMMs) has been proposed by Scrucca
and Serafini, (2019). This PP algorithm employs the negentropy obtained from a
multivariate density estimated by GMMs as the PP index to be maximized.
Optimization of the negentropy index is performed using Genetic Algorithms
(GAs). The resulting PPGMMGA algorithm estimates the optimal orthogonal
projection subspace which is able to show the directions of maximal non
normality, and for this reason it can be effectively employed to detect and
visualize the presence of clustering structures in multivariate data. However,
being PPGMMGA a dimension reduction method, no clustering memberships are
assigned to the projected data points.

The main goal of this contribution is to propose a clustering approach based
on the modal EM algorithm which allows to estimate the modes corresponding to
the local maxima of the underlying density of projected data. Thus, groups are
obtained according to the mode to which each data point converges, thereby
providing a clustering procedure for the PPGMMGA algorithm. The modal
clustering procedure discussed in this paper follows the definition of
clusters proposed by Hartigan, (1975, p. 205), according to which "clusters
may be thought of as regions of high density separated from other such regions
by regions of low density".

Section 2 provides a brief review of parsimonious GMMs and of the model-based
projection pursuit algorithm used to maximize the negentropy of a GMM. Section
3 contains the modal clustering proposal for applying the modal EM algorithm
on the estimated projection subspace. Section 4 describes the empirical
results obtained from the application of the proposed modal EM algorithm to
examples using both synthetic and real datasets. The final section provides
some concluding remarks.

## 2\. Background

### 2.1. Parsimonious Gaussian mixtures

Let $\boldsymbol{x}$ be a random vector in $\mathbb{R}^{p}$ with density
$f(\boldsymbol{x})$, which we assume it can be described by a finite mixture
of Gaussian distributions:

$f(\boldsymbol{x};\boldsymbol{\theta})=\sum^{G}_{g=1}\pi_{g}\phi(\boldsymbol{x};\boldsymbol{\mu}_{g},\boldsymbol{\Sigma}_{g}),$
(1)

where
$\boldsymbol{\theta}=\\{\pi_{1},\pi_{2},\ldots,\pi_{G-1},\boldsymbol{\mu}_{1},\ldots,\boldsymbol{\mu}_{G},\operatorname{vech}(\boldsymbol{\Sigma}_{1}),\ldots,\operatorname{vech}(\boldsymbol{\Sigma}_{G})\\}$
are the parameters of the mixture model with $G$ components,
($\pi_{1},\pi_{2},\ldots,\pi_{G}$) the mixing weights, so that $\pi_{g}>0$ and
$\sum_{g=1}^{G}\pi_{g}=1$, and
$\phi(\boldsymbol{x};\boldsymbol{\mu}_{g},\boldsymbol{\Sigma}_{g})$ the
underlying multivariate Gaussian density function of the $g$th component
having mean $\boldsymbol{\mu}_{g}$ and covariance matrix
$\boldsymbol{\Sigma}_{g}$. Note that $\operatorname{vech}(\cdot)$ is the
vector operator, which forms a vector by extracting unique elements of a
symmetric matrix.

Parsimonious parametrization of covariance matrices for GMMs is available
through the eigen-decomposition
$\boldsymbol{\Sigma}_{g}=\lambda_{g}\boldsymbol{U}_{g}\boldsymbol{\Delta}_{g}\boldsymbol{U}^{\top}_{g}$,
where $\lambda_{g}=|\boldsymbol{\Sigma}_{g}|^{1/p}$ is a scalar which controls
the volume, $\boldsymbol{\Delta}_{g}$ is the diagonal matrix of normalized
eigenvalues of $\boldsymbol{\Sigma}_{g}$, such that
$|\boldsymbol{\Delta}_{g}|=1$, which controls the shape, and
$\boldsymbol{U}_{g}$ is the orthogonal matrix of eigenvectors of
$\boldsymbol{\Sigma}_{g}$ controlling the orientation of the ellipsoid
(Banfield and Raftery,, 1993; Celeux and Govaert,, 1995). Maximum likelihood
estimates of the mixture parameters can be obtained using the Expectation-
Maximisation (EM) algorithm (Dempster et al.,, 1977; McLachlan and Krishnan,,
2008), whereas information criteria, such as the Bayesian Information
Criterion (BIC; Schwartz,, 1978), are typically used for model selection. The
latter concerns the decision on both the number of mixture components and on
which parsimonious parametrization of covariances to adopt. Other approaches
are also available in the literature to reduce the number of unknown
parameters. For instance, mixture of factor analyzers, proposed in McLachlan
and Peel, 2000b  and based on the work of Ghahramani and Hinton, (1997), could
be considered here by allowing each Gaussian component in the mixture to be
represented in a different lower-dimensional manifold.

### 2.2. Model-based projection pursuit

Consider the orthogonal linear projection
$\boldsymbol{z}=\boldsymbol{B}^{\top}\boldsymbol{x}\in\mathbb{R}^{d}$, with
$d<p$, and the basis $\boldsymbol{B}\in\mathbb{R}^{p\times d}$, a matrix
spanning the dimension reduction subspace $\mathcal{S}(\boldsymbol{B})$. A
remarkable result is the _linear transformation property of GMMs_ (see
Appendix A in Scrucca and Serafini,, 2019), which shows that if
$\boldsymbol{x}$ follows a Gaussian mixture distribution as in (1), then any
linear projection also follows a Gaussian mixture distribution, so we can
write

$f(\boldsymbol{z})=\sum^{G}_{g=1}\pi_{g}\phi(\boldsymbol{z};\boldsymbol{B}^{\top}\boldsymbol{\mu}_{g},\boldsymbol{B}^{\top}\boldsymbol{\Sigma}_{g}\boldsymbol{B}).$
(2)

Thus, the distribution of the projected data can still be described by a GMM
with a suitable transformation of the parameters of the Gaussian components.
The basis $\boldsymbol{B}$ of the subspace can be estimated by maximizing the
_negentropy_ , a PP index that summarizes deviations from the least
interesting case represented by the Gaussian distribution. The negentropy is
defined as

$J(\boldsymbol{z})=h\big{(}\phi(\boldsymbol{\mu}_{z},\boldsymbol{\Sigma}_{z})\big{)}-h(\boldsymbol{z})=\frac{1}{2}\log\big{(}(2\pi
e)^{d}|\boldsymbol{\Sigma}_{z}|\big{)}+\int\log\big{(}f(\boldsymbol{z})\big{)}f(\boldsymbol{z})\operatorname{d}\\!{\boldsymbol{z}},$

where $h(\phi\big{(}\boldsymbol{\mu}_{z},\boldsymbol{\Sigma}_{z})\big{)}$ is
the entropy of the marginal multivariate Gaussian distribution with mean
$\boldsymbol{\mu}_{z}$ and covariance $\boldsymbol{\Sigma}_{z}$, and
$h(\boldsymbol{z})$ is the entropy of the mixture density of the projected
data. Efficient approximations for $h(\boldsymbol{z})$ are available, such as
the _unscented transformation_ (UT), so the PP index can be maximized over the
set of all orthonormal bases $\boldsymbol{B}$ using genetic algorithms. For
further details see Scrucca and Serafini, (2019). This briefly described
algorithm is named PPGMMGA, and it is implemented in the R package ppgmmga
(Serafini and Scrucca,, 2019).

## 3\. Modal clustering on PPGMMGA projection subspace

In the modal approach to clustering, clusters are defined as the local maxima
of the underlying probability density function. For a recent review of
different approaches in modal clustering see Menardi, (2016), while for a more
general discussion on the recent renewed interest on modal procedures see
Chacón, (2020). The Modal EM (MEM) algorithm is an iterative algorithm aimed
at identifying the local maxima of a finite mixture density function (Li et
al.,, 2007). Once the modes have been estimated, clustering of data points is
obtained according to the mode to which they converge. Recently, Scrucca,
(2021) generalized the MEM algorithm to efficiently deal with Gaussian
mixtures having any parsimonious covariance matrices of the type described in
Section 2.2.

To effectively apply the MEM approach on the PPGMMGA projection subspace a
density estimate is needed. In principle, the mixture density in equation (2)
could be used as the underlying density to base the MEM procedure. However, if
this is reasonable at the population level, in practice only an estimate of
the density (2) from sample data is available. Even more important, this
estimate is typically chosen among a set of plausible models according to a
model selection criterion, such as the BIC (Fraley and Raftery,, 1998). Since
the latter penalizes model complexity, more parsimonious models tend to be
selected when the sample size of available data is limited compared to the
dimensionality of the problem. However, the adopted parameterization may prove
to be inadequate to represent the density of the data on the projection
subspace.

A limiting case of this situation is presented in Section 4.1, where a
simulated dataset of 100 observations on a 50-dimensional feature space is
considered. In this case the Gaussian mixture density selected by BIC has only
one component, and if this poses no problem for estimating the PPGMMGA
subspace, it clearly fails if used for estimating the modes of the underlying
density on the two-dimensional projection subspace. For this reason, a
different route is needed and, again, a parsimonious Gaussian mixture is
called to the rescue.

### 3.1. MEM algorithm

Consider the PPGMMGA basis $\boldsymbol{B}$ estimated for a fixed
dimensionality $d$. The projected data can be computed as
$\boldsymbol{z}_{i}=\boldsymbol{B}^{\top}\boldsymbol{x}_{i}$ for all
$i=1,\ldots,n$, or equivalently in matrix form as
$\boldsymbol{Z}=\boldsymbol{X}\boldsymbol{B}$. The Gaussian mixture density on
the projection subspace can be written as

$f(\boldsymbol{z};\boldsymbol{\Psi})=\sum^{K}_{k=1}\tau_{k}\phi(\boldsymbol{z};\boldsymbol{\eta}_{k},\boldsymbol{\Gamma}_{k}),$
(3)

where
$\boldsymbol{\Psi}=\\{\tau_{1},\tau_{2},\ldots,\tau_{K-1},\boldsymbol{\eta}_{1},\ldots,\boldsymbol{\eta}_{K},\operatorname{vech}(\boldsymbol{\Gamma}_{1}),\ldots,\operatorname{vech}(\boldsymbol{\Gamma}_{K})\\}$
are the parameters of the mixture model with $K$ components,
($\tau_{1},\tau_{2},\ldots,\tau_{K}$) the mixing weights, so that $\tau_{k}>0$
and $\sum_{k=1}^{K}\tau_{k}=1$, and
$\phi(\boldsymbol{z};\boldsymbol{\eta}_{k},\boldsymbol{\Gamma}_{k})$ the
multivariate Gaussian density function of the $k$th component having mean
$\boldsymbol{\eta}_{k}$ and covariance matrix $\boldsymbol{\Gamma}_{k}$.
Following the details provided in Section 2.1, the best fitting parsimonious
mixture density (3) can be selected by BIC with respect to both the number of
mixture components $K$ and the parsimonious covariance matrices
$\\{\boldsymbol{\Gamma}_{k}\\}_{k=1}^{K}$. By using the estimated density it
is possible to identify the modes in the dimension reduction subspace
$\mathcal{S}(\boldsymbol{B})$, thus enabling a modal clustering approach for
the identification of homogeneous and separated groups. The MEM algorithm is
briefly described below, while further details for interested readers,
including a discussion on the number of modes in Gaussian mixtures, are
provided in Scrucca, (2021).

The MEM algorithm starts at initial data points
$\boldsymbol{z}_{i}^{(0)}=\boldsymbol{z}_{i}$ for $i=1,\ldots,n$, then at each
iteration $t$, assuming the mixture parameters
$\\{\tau_{k},\boldsymbol{\eta}_{k},\boldsymbol{\Gamma}_{k}\\}_{k=1}^{K}$ as
known and fixed, performs the following steps:

E-step – update the posterior conditional probability of the projected data
point $\boldsymbol{z}_{i}$ to belong to the $k$th mixture component using

$\zeta_{ik}^{(t)}=\frac{\tau_{k}\phi(\boldsymbol{z}_{i}^{(t-1)};\;\boldsymbol{\eta}_{k},\boldsymbol{\Gamma}_{k})}{\sum_{j=1}^{K}\tau_{j}\phi(\boldsymbol{z}_{i}^{(t-1)};\;\boldsymbol{\eta}_{j},\boldsymbol{\Gamma}_{j})}\quad\text{for
all }k=1,\ldots,K.$

M-step – update the current value of $\boldsymbol{z}_{i}$ by solving the
optimization problem

$\boldsymbol{z}_{i}^{(t)}=\operatorname*{arg\max}_{\boldsymbol{z}_{i}\in\operatorname{\mathbb{R}}^{d}}\sum_{k=1}^{K}\zeta_{ik}^{(t)}\log\phi(\boldsymbol{z}_{i}^{(t-1)};\;\boldsymbol{\eta}_{k},\boldsymbol{\Gamma}_{k}).$

The procedure above is iterated until a convergence criterion is satisfied, or
a pre-specified maximum number of iterations is reached.

Note that the objective function in the M-step can be written as

$Q(\boldsymbol{z}_{i})=\sum_{k=1}^{K}\zeta_{ik}\log\phi(\boldsymbol{z}_{i};\;\boldsymbol{\eta}_{k},\boldsymbol{\Gamma}_{k}),$

so the gradient and Hessian of this function with respect to the projected
vector $\boldsymbol{z}_{i}$ are, respectively,

$\displaystyle\nabla Q(\boldsymbol{z}_{i})$
$\displaystyle=-\sum_{k=1}^{K}\zeta_{ik}\boldsymbol{\Gamma}_{k}^{-1}(\boldsymbol{z}_{i}-\boldsymbol{\eta}_{k}),\quad\text{and}$
$\displaystyle\nabla^{2}Q(\boldsymbol{z}_{i})$
$\displaystyle=-\sum_{k=1}^{K}\zeta_{ik}\boldsymbol{\Gamma}_{k}^{-1}.$

Therefore, the M-step presents a closed-form solution given by

$\boldsymbol{z}_{i}^{*}=\left(\sum_{k=1}^{K}\zeta_{ik}\boldsymbol{\Gamma}_{k}^{-1}\right)^{-1}\sum_{k=1}^{K}\zeta_{ik}\boldsymbol{\Gamma}_{k}^{-1}\boldsymbol{\eta}_{k}.$
(4)

A direct application of the update equation above may yield large jumps during
the initial iterations of the algorithm for data points lying in low-density
regions, thus escaping from the region of the attracting mode to the domain of
attraction of a different mode. In other word, a data point located in a low-
density region might see large jumps at the beginning of the iterative
algorithm, hence converging to a mode further from its domain of attraction.
For this reason, Scrucca, (2021, Section 3.1) introduced a step size to get
more reliable update at iteration $t$ using a convex linear combination of the
solution at previous iteration, $\boldsymbol{z}_{i}^{(t-1)}$, and the new
proposal $\boldsymbol{z}_{i}^{*}$ from (4), as follows

$\boldsymbol{z}_{i}^{(t)}=(1-\omega)\,\boldsymbol{z}_{i}^{(t-1)}+\omega\,\boldsymbol{z}_{i}^{*},$
(5)

where $\omega=1-\exp\\{-0.1t\\}$ is a parameter that controls the step size.
Following this approach, at earlier iterations the step size is small, hence
updates are done in small steps, but as the number of iterations increase the
step size converges to one and the updates become essentially equal to the
proposal $\boldsymbol{z}_{i}^{*}$. The proposed modification does not break
the monotonicity property of EM (see Appendix for the proof), although it
slows the convergence somewhat by increasing the number of iterations
required.

Finally, we note that the execution time can be greatly reduced by batch
updating all the data points at once instead of working one point at time, as
detailed in Scrucca, (2021).

### 3.2. Some final comments

In the model-based clustering approach, clusters are defined by the underlying
distribution used for the mixture components. For instance, GMMs assume
ellipsoidal group structures. Then, the so-called maximum a posteriori (MAP)
principle is used for clustering. On the contrary, as discussed in Section 1,
modal clustering is an attempt at addressing the definition of clusters
proposed by Hartigan, (1975). If data points projected onto the PPGMMGA
projection subspace show ellipsoidal structures and are well-separated then
the model-based clustering approach using GMMs and the modal-clustering via
MEM algorithm produce equivalent results. However, in case the groups show
non-Gaussian structures or substantially overlap on the projection subspace,
the final clustering might be different. This mostly reflects the different
definition adopted by the two approaches regarding what a cluster is. An
example of such a case is discussed in Section 4.4.

## 4\. Data analysis examples

In this section we apply the proposed methodology to simulated and real
datasets.

### 4.1. Simulated data with clustering on a subset of attributes

We consider a reduced version of the two-group synthetic dataset discussed in
Friedman and Meulman, (2004). For the first group 85 observations are randomly
generated from a 50-dimensional standard Gaussian distribution. The second
group of 15 observations are also drawn from a 50-dimensional Gaussian
distribution, but its first $p_{0}=15$ attributes have mean $1.5$ and standard
deviation $0.2$, whereas the remaining $50-p_{0}$ attributes have zero mean
and unit standard deviation. Thus, the two groups differ only on the first
$p_{0}$ attributes.

Figure 1a shows the data projected onto the 2-dimensional projection subspace
estimated by the PPGMMGA algorithm, which clearly reveals the two-group
clustering structure. It is interesting to note that the GMM selected by BIC
on the original 50-dimensional features space has a single Gaussian component.
In this case, an attempt at clustering on the original features space would
produce a solution consisting of a single cluster. Nonetheless, the estimated
directions that maximize the negentropy are able to show the underlying
structure as shown in Figure 1a.

Iteration paths of MEM algorithm for each data point toward the attracting
mode are depicted in Figure 1b. The resulting modal clustering solution
obtained on the projection subspace is perfectly able to recover the two
groups of observations.

Figure 1: Plot of two-group synthetic data points projected along the PPGMMGA
directions with points marked according to the known class in panel (a), and
using the modal clustering solution on panel (b). The last graph also displays
the MEM iteration paths toward the attracting modes represented using the $+$
symbol.

### 4.2. Rectangular Gaussian block clusters in three dimensions with noise
attributes

In this second example, we consider a simulated dataset made up of 400
observations on 8 variables. The first three variables are generated from an
equal weight mixture of eight Gaussian distributions with mean vectors at the
corners of a rectangular block in three dimensions. The remaining five
variables only contain some Gaussian noise. Details about the mean vector for
each mixture component and the common covariance matrix are provided in Bolton
and Krzanowski, (2003, Appendix A).

We consider the PPGMMGA estimation of the projection subspace for varying
dimensions. The negentropy values for each solution with $d$ from 1 up to 5
suggest a 3-dimensional projection subspace as a reasonable choice (see Table
2). Figure 2 shows some views of the data points projected along the estimated
PPGMMGA directions with modes and data points marked according to the cluster
memberships estimated by the MEM algorithm. The 2-dimensional scatterplots in
panels (a)–(c) show pairwise projections for the estimated PPGMMGA directions,
with arrows indicating the biplot vectors corresponding to the original
features. As it can be seen, only the first three variables contribute to the
definition of the PPGMMGA directions (see also the coefficients of the
estimated projection basis reported in Table 2). Finally, Figure 2d shows a
static view of the data points projected onto the estimated 3-dimensional
projection subspace.

Table 2 contains, in addition to the negentropy for increasing dimensionality
of the projection subspace, the number of modes found by the MEM algorithm and
the accuracy of the proposed modal clustering procedure. This is measured by
the Adjusted Rand Index (ARI; Hubert and Arabie,, 1985), a measure of
agreement between two partitions, which attains unit value when the two
partitions perfectly agree. Overall, the MEM algorithm is able to recover
almost perfectly the true clusters for $d\geq 3$.

Table 1: PPGMMGA negentropy, number of modes ($m$), and adjusted Rand index (ARI) for increasing dimension ($d$) of the projection subspace. $d$ | Negentropy | $m$ | ARI
---|---|---|---
1 | 0.5609 | 3 | 0.4244
2 | 1.0832 | 7 | 0.8653
3 | 1.5075 | 8 | 0.9942
4 | 1.5271 | 8 | 0.9942
5 | 1.5351 | 8 | 0.9942

Table 2: Estimated PPGMMGA projection basis for the $d=3$ solution. | PP1 | PP2 | PP3
---|---|---|---
$X_{1}$ | -0.2885 | -0.9041 | -0.3080
$X_{2}$ | 0.8964 | -0.3706 | 0.2394
$X_{3}$ | 0.3343 | 0.2101 | -0.9168
$X_{4}$ | -0.0097 | -0.0203 | -0.0425
$X_{5}$ | -0.0184 | -0.0149 | -0.0361
$X_{6}$ | 0.0099 | 0.0026 | 0.0111
$X_{7}$ | -0.0297 | -0.0203 | -0.0626
$X_{8}$ | 0.0020 | -0.0097 | -0.0109

Figure 2: Pairwise scatterplots (panels (a) to (c)) and a static view of 3D
scatterplot (panel (d)) of PPGMMGA projections with data points marked
according to the clusters assigned by the MEM procedure and the corresponding
modes, represented using $+$ symbol.

### 4.3. Chironomus larvae data

This dataset comes from a study on morphometric attributes of Chironomus
larvae (Atchley and Martin,, 1971). A sample of 149 larvae was collected from
3 different species: cloacalis, frommeri and staegeri. For each larva, 17
features of the larval head capsula were measured; for a detailed description
of the dataset we refer the reader to Montanari and Viroli, (2010).

Figure 3a shows the data projected from the original 17-dimensional feature
space to the 2-dimensional projection subspace estimated by the PPGMMGA
algorithm. The estimated modes and the corresponding modal clustering
solutions are shown in Figure 3b, together with the paths induced by the MEM
algorithm. The 3-mode clustering structure identified is then compared with
the true larvae species in Figure 3c, showing a quite good agreement with only
one misclassified observation and an adjusted Rand index of $0.981$ (slightly
larger than the ARI reported in Montanari and Viroli, (2010)).

Figure 3: Chironomus larvae data graphs: (a) plot of data points projected
along the PPGMMGA directions; (b) plot of the modal clustering solution
identified by the MEM algorithm with the estimated modes represented by a $+$
symbol; (c) plot of data points marked according to the true species of the
larvae with different symbols, and different colours for the identified
clusters (circled point represents the misclassified observation).

### 4.4. Coffee data

The coffee dataset (Streuli,, 1973) contains 12 chemical measurements on 43
samples of Arabica or Robusta coffee from 29 countries. Scrucca and Serafini,
(2019) analysed the coffee data by fitting a 1-dimensional PPGMMGA model.
Figure 4a shows the data points projected along the estimated direction,
clearly suggesting that coffee varieties can be clustered on this one-
dimensional subspace. Graph on Figure 4b reports the underlying density
estimated by GMM and used by the MEM algorithm, together with the ascending
paths for each projected data point toward the closest mode. The clustering
obtained perfectly recovers the latent coffee varieties.

Figure 4: Histogram of coffee data points projected along the first PPGMMGA
direction marked by the true variety of coffee on panel (a). Underlying
density estimated by GMM employed by the MEM algorithm on panel (b), with step
lines showing the iteration paths toward the attracting modes represented by a
$+$ symbol. Rug points along the $x$-axis on both graphs represent the
projected data points.

In Section 3.2 the modal clustering approach via MEM algorithm and the model-
based clustering approach using the MAP principle were discussed and
contrasted. The considerations made therein can now be illustrated in practice
using the coffee dataset. In particular, Figure 5 reports the clustering
results obtained using the two approaches on the 1-dimensional PPGMMGA
projection. Panel (a) shows the two Gaussian components of the mixture
estimated on the first PPGMMGA direction, with the corresponding MAP
clustering on panel (c). On the contrary, panel (b) contains the estimated
mixture density, with the corresponding MEM clustering on panel (d). According
to the MAP principle, data points are assigned to the cluster whose
corresponding component density (weighted by the mixing proportion) is the
largest. Note that one data point is assigned to the group of Robusta despite
being of type Arabica and quite far from the other data points assigned to the
same cluster. However, this appears reasonable under the model-based
perspective because the estimated conditional probability for that point is
larger for Robusta than for Arabica (and this is due to the large estimated
variance for the second component). By adopting, instead, a modal clustering
perspective, the mixture density must be used to identify the modes, and data
points are assigned to the “closest” clusters represented by such modes. In
this case, the classification obtained perfectly matches the true coffee
varieties. Indeed, the previously misclassified data point is now assigned to
the Arabica cluster because its domain of attraction is the same as for the
other Arabica coffees.

Figure 5: A graphical comparison of model-based clustering vs modal clustering
on the 1-dimensional PPGMMGA projection for the coffee dataset. The two
Gaussian components of the mixture (weighted by the mixing proportion) are
shown in panel (a), whereas the corresponding mixture density is reported in
panel (b). The clustering solutions obtained by applying the MAP principle and
the MEM algorithm are shown, respectively, in panels (c) and (d).

## 5\. Concluding comments

PPGMMGA is a linear projection method introduced by Scrucca and Serafini,
(2019) with the aim of visualizing multivariate data when a clustering
structure is sought. The PPGMMGA algorithm estimates the directions of maximal
nonnormality as measured by the negentropy PP index. Being a linear mapping
from a potential high-dimensional features space to a subspace of reduced
dimensionality, it involves linear transformations, such as rotation and
reflection, of the original data. Thus, if a clustering structure is present
in the original features space, PPGMMGA should preserve it and, hopefully,
magnify it.

However, if visualization is an important step in understanding the underlying
structure of the data, an automatic procedure for the identification of
homogenous groups would be desirable. In this paper we introduce a modal
clustering approach to address this issue. Working on the projection subspace
estimated by the PPGMMGA method, a modal EM algorithm is proposed for
identifying the modes of the density estimated by fitting parsimonious
Gaussian mixture models.

The proposed approach can, in principle, be applied to any dimension reduction
method, such as local linear embedding (LLE; Roweis and Saul,, 2000),
t-distributed stochastic neighbor embedding (t-SNE; Van der Maaten and
Hinton,, 2008) and uniform manifold approximation and projection (UMAP;
McInnes et al.,, 2018). We defer the investigation of these extensions to
future work.

## Appendix

We prove here that the inclusion of the step size in equation (5) does not
break the monotonicity property of MEM, i.e. $f(\boldsymbol{z}_{i}^{(t)})\geq
f(\boldsymbol{z}_{i}^{(t-1)})$ for any iteration $t$. First, we note that by
the ascending property of the MEM algorithm for any mixture we have that
$f(\boldsymbol{z}_{i}^{*})\geq f(\boldsymbol{z}_{i}^{(t-1)})$ (see Li et al.,,
2007, Appendix A), where $\boldsymbol{z}_{i}^{*}$ is the proposal from
equation (4) at current iteration $t$, $\boldsymbol{z}_{i}^{(t-1)}$ is the
solution at iteration $t-1$, and $f()$ is the mixture density (3). Then, at
iteration $t$ of the algorithm we have

$\displaystyle f(\boldsymbol{z}_{i}^{(t)})$
$\displaystyle=f((1-\omega)\boldsymbol{z}_{i}^{(t-1)}+\omega\boldsymbol{z}_{i}^{*})$
$\displaystyle\geq(1-\omega)f(\boldsymbol{z}_{i}^{(t-1)})+\omega
f(\boldsymbol{z}_{i}^{*})$ (by the monotonocity along the learning path)
$\displaystyle=f(\boldsymbol{z}_{i}^{(t-1)})+\omega(f(\boldsymbol{z}_{i}^{*})-f(\boldsymbol{z}_{i}^{(t-1)}))$
$\displaystyle\geq f(\boldsymbol{z}_{i}^{(t-1)})$ (since
$f(\boldsymbol{z}_{i}^{*})-f(\boldsymbol{z}_{i}^{(t-1)})\geq 0$ and
$\omega\geq 0$)

## References

  * Atchley and Martin, (1971) Atchley, W. R. and Martin, J. (1971).  A morphometric analysis of differential sexual dimorphism in larvae of chironomus.  The Canadian Entomologist, 103(3):319–327.
  * Banfield and Raftery, (1993) Banfield, J. and Raftery, A. E. (1993).  Model-based Gaussian and non-Gaussian clustering.  Biometrics, 49:803–821.
  * Bolton and Krzanowski, (2003) Bolton, R. J. and Krzanowski, W. J. (2003).  Projection pursuit clustering for exploratory data analysis.  Journal of Computational and Graphical Statistics, 12(1):121–142.
  * Celeux and Govaert, (1995) Celeux, G. and Govaert, G. (1995).  Gaussian parsimonious clustering models.  Pattern Recognition, 28:781–793.
  * Chacón, (2020) Chacón, J. E. (2020).  The modal age of statistics.  International Statistical Review, 88(1):122–141.
  * Dempster et al., (1977) Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977).  Maximum likelihood from incomplete data via the EM algorithm (with discussion).  Journal of the Royal Statistical Society: Series B (Statistical Methodology), 39:1–38.
  * Fraley and Raftery, (1998) Fraley, C. and Raftery, A. E. (1998).  How many clusters? Which clustering method? Answers via model-based cluster analysis.  The Computer Journal, 41:578–588.
  * Friedman, (1987) Friedman, J. H. (1987).  Exploratory projection pursuit.  Journal of the American Statistical Association, 82:249–266.
  * Friedman and Meulman, (2004) Friedman, J. H. and Meulman, J. J. (2004).  Clustering objects on subsets of attributes.  Journal of the Royal Statistical Society: Series B (Statistical Methodology), 66(4):815–849.
  * Friedman and Tukey, (1974) Friedman, J. H. and Tukey, J. W. (1974).  A projection pursuit algorithm for exploratory data analysis.  IEEE Transactions on Computers, 100(9):881–890.
  * Ghahramani and Hinton, (1997) Ghahramani, Z. and Hinton, G. E. (1997).  The EM algorithm for factor analyzers.  In Technical Report CRG-TR-96-1. University of Toronto.
  * Hartigan, (1975) Hartigan, J. A. (1975).  Clustering Algorithms.  John Wiley & Sons, New York.
  * Hubert and Arabie, (1985) Hubert, L. and Arabie, P. (1985).  Comparing partitions.  Journal of Classification, 2:193–218.
  * Li et al., (2007) Li, J., Ray, S., and Lindsay, B. G. (2007).  A nonparametric statistical approach to clustering via mode identification.  Journal of Machine Learning Research, 8:1687–1723.
  * McInnes et al., (2018) McInnes, L., Healy, J., and Melville, J. (2018).  UMAP: Uniform manifold approximation and projection for dimension reduction.
  * McLachlan and Krishnan, (2008) McLachlan, G. and Krishnan, T. (2008).  The EM Algorithm and Extensions.  Wiley-Interscience, Hoboken, New Jersey, 2nd edition.
  * McLachlan et al., (2019) McLachlan, G. J., Lee, S. X., and Rathnayake, S. I. (2019).  Finite mixture models.  Annual Review of Statistics and Its Application, 6:355–378.
  * (18) McLachlan, G. J. and Peel, D. (2000a).  Finite Mixture Models.  Wiley, New York.
  * (19) McLachlan, G. J. and Peel, D. (2000b).  Mixtures of factor analyzers.  In Proceedings of the Seventeenth International Conference on Machine Learning, pages 599–606. Burlington, MA: Morgan Kaufmann.
  * Menardi, (2016) Menardi, G. (2016).  A review on modal clustering.  International Statistical Review, 84(3):413–433.
  * Montanari and Viroli, (2010) Montanari, A. and Viroli, C. (2010).  Heteroscedastic factor mixture analysis.  Statistical Modelling, 10(4):441–460.
  * Roweis and Saul, (2000) Roweis, S. T. and Saul, L. K. (2000).  Nonlinear dimensionality reduction by locally linear embedding.  Science, 290(5500):2323–2326.
  * Schwartz, (1978) Schwartz, G. (1978).  Estimating the dimension of a model.  Annals of Statistics, 6:31–38.
  * Scrucca, (2021) Scrucca, L. (2021).  A fast and efficient modal EM algorithm for Gaussian mixtures.  Statistical Analysis and Data Mining.
  * Scrucca and Serafini, (2019) Scrucca, L. and Serafini, A. (2019).  Projection pursuit based on Gaussian mixtures and evolutionary algorithms.  Journal of Computational and Graphical Statistics, 28(4):847–860.
  * Serafini and Scrucca, (2019) Serafini, A. and Scrucca, L. (2019).  ppgmmga: Projection Pursuit Based on Gaussian Mixtures and Evolutionary Algorithms.  R package version 1.2.
  * Streuli, (1973) Streuli, H. (1973).  Der heutige stand der kaffeechemie.  In Association Scientifique International du Cafe, 6th International Colloquium on Coffee Chemistry, Bogota, Columbia, volume 61, pages 61–72.
  * Van der Maaten and Hinton, (2008) Van der Maaten, L. and Hinton, G. (2008).  Visualizing data using t-SNE.  Journal of Machine Learning Research, 9(11).


# The Curious Case of Representational Alignment: Unravelling Visio-Linguistic
Tasks in Emergent Communication

Tom Kouwenhoven
LIACS
Leiden University
<EMAIL_ADDRESS>Max Peeperkorn
School of Computing
University of Kent
<EMAIL_ADDRESS>Bram van Dijk
LUMC
Leiden University Medical Center
<EMAIL_ADDRESS>Tessa Verhoef
LIACS
Leiden University
<EMAIL_ADDRESS>
###### Abstract

Natural language has the universal properties of being compositional and
grounded in reality. The emergence of linguistic properties is often
investigated through simulations of emergent communication in referential
games. However, these experiments have yielded mixed results compared to
similar experiments addressing linguistic properties of human language. Here
we address representational alignment as a potential contributing factor to
these results. Specifically, we assess the representational alignment between
agent image representations and between agent representations and input
images. Doing so, we confirm that the emergent language does not appear to
encode human-like conceptual visual features, since agent image
representations drift away from inputs whilst inter-agent alignment increases.
We moreover identify a strong relationship between inter-agent alignment and
topographic similarity, a common metric for compositionality, and address its
consequences. To address these issues, we introduce an alignment penalty that
prevents representational drift but interestingly does not improve performance
on a compositional discrimination task. Together, our findings emphasise the
key role representational alignment plays in simulations of language
emergence.

The Curious Case of Representational Alignment: Unravelling Visio-Linguistic
Tasks in Emergent Communication

Tom Kouwenhoven LIACS Leiden University<EMAIL_ADDRESS>Max
Peeperkorn School of Computing University of Kent<EMAIL_ADDRESS>

Bram van Dijk LUMC Leiden University Medical Center<EMAIL_ADDRESS>Tessa Verhoef LIACS Leiden University<EMAIL_ADDRESS>

##  1 Introduction

Human language bears unique properties that make it a powerful tool for
communication. A well-known property is compositionality: the ability to
combine meaningful words into more complex meanings (Hockett, 1959). The
emergence of compositionality is studied extensively in the field of language
evolution through human experiments (e.g. Selten and Warglien, 2007; Kirby et
al., 2008, 2015; Raviv et al., 2019a). An important finding from this field is
that the unique nature of human language can be explained as a consequence of
biases for simplicity and expressivity imposed during continuous language
learning and use (Smith, 2022). Computational simulations of language
emergence have also been used to study the emergence of linguistic properties
(e.g. de Boer, 2006; Steels and Loetzsch, 2012), and have seen a rising
interest in the field of computational linguistics (Lazaridou and Baroni,
2020). Here, compositionality in the emergent communication protocols is
commonly measured through topographic similarity (topsim; Brighton and Kirby,
2006). This metric was first used in recent computational simulations by
Lazaridou et al. (2018) and has been used in a large body of work since. Yet,
the interpretation of linguistic properties emerging in simulations remains
challenging, since language protocols used among artificial agents often show
critical mismatches with known properties of human languages (Galke et al.,
2022; Lian et al., 2023) such as efficiency, word-order vs. case-marking
biases, or compositional generalisation (see §2). Consequently, it is evident
that their learning biases and signal-meaning mappings differ from those of
humans. This underscores the critical need to obtain deeper insight into
referential games in the language learning setting (Rita et al., 2022).

A possible explanation for these mismatches could stem from representational
alignment, the degree of agreement between the internal representations of two
information processing systems (Sucholutsky et al., 2023). To the best of our
knowledge, representational alignment in emergent communication was first
reported by Bouchacourt and Baroni (2018), who measured the degree to which
agents aligned their internal image interpretations (inter-agent alignment) by
performing Representational Similarity Analysis (rsa; Kriegeskorte et al.,
2008). Using rsa (§3), they showed that agents establish successful
communication artificially by aligning their internal image representations
while losing any relation to the images presented (image-agent alignment),
enabling communication about noise input even though they were trained on real
images. As such, their communication protocol captured not conceptual
properties of the objects depicted in pictures, but most likely focused on
non-human-like spurious image features (e.g., pixel intensities). While inter-
agent alignment is not a problem per se, the loss of image-agent alignment is
problematic for two reasons. First, for emergent communication simulations to
provide meaningful insights into the emergence of natural human language,
agent image representations must be grounded in the content of the images.
Only then can we deduce what the agents communicate about and assess
linguistic properties or their ability to generalise to novel concepts.
Second, emergent communication setups have been proposed to fine-tune pre-
trained (vision-)language models, aiming to enhance machine understanding of
natural human language (Lazaridou and Baroni, 2020; Lowe et al., 2020;
Steinert-Threlkeld et al., 2022; Zheng et al., 2024). In this context,
maintaining substantial alignment between representations and images is
crucial for preserving mutual understanding between machines and humans.

Representational alignment, however, did not receive the necessary attention
since a host of papers appeared after the findings by Bouchacourt and Baroni
in which results on referential games were reported without taking rsa into
account (e.g. Lazaridou et al., 2018; Guo et al., 2019; Li and Bowling, 2019;
Ren et al., 2020; Chaabouni et al., 2020; Dagan et al., 2021; Mu and Goodman,
2021; Chaabouni et al., 2022). Admittedly, some use attribute-value objects
instead of real images as input. But importantly, in nearly all cases, neural
agents must map inputs—whether attribute-value objects or image
representations—onto agent-specific representations. Therefore the problem of
inter-agent alignment can always occur and is agnostic to the input type.
Although this warrants further analysis of earlier results, the field is
already employing referential games in more complex simulations with real
images (e.g. Dessi et al., 2021; Chaabouni et al., 2022; Mahaut et al., 2024).

This work addresses the understudied alignment problem in standard referential
game setups used in emergent communication. We train Reinforcement Learning
(RL) agents equipped with a recent vision module (DinoV2; Oquab et al., 2024)
to communicate about images. In addition to evaluating the agents on MS COCO
(Lin et al., 2014) image pairs, we evaluate on noise pairs and image pairs
sourced from the Winoground dataset (Thrush et al., 2022). The latter is
explicitly created to gauge visio-linguistic compositional reasoning abilities
of vision and language models. We first confirm that effective communication
in the referential game relies on inter-agent alignment and then move on to
our contributions. First, we find a strong correlation between the degree of
inter-agent alignment and the topsim metric. Our second contribution consists
of a solution to the alignment problem by including an alignment penalty term
to the loss, resulting in equivalent communicative success and higher topsim
whilst ensuring that the agents communicate about images instead of spurious
features (Figure 1). We then argue to start evaluating emergent communication
protocols on more strict tasks that directly target the intuition behind
popular metrics to obtain a clearer understanding of the protocols. Overall,
our results highlight the importance of representational alignment in
simulations of language emergence and underscore the need to better understand
the divergence in human and artificial language emergence.

(a) Communicative performance (Accuracy) on discriminating two images.

(b) Inter-agent representational alignment ($\textsc{rsa}_{sl}$) between agent
representations.

(c) Topographic similarity (topsim) between the images and the messages.

Figure 1: Inference results for different datasets after training on MS COCO
images. In (a) we see that agents can discriminate MS COCO images but struggle
with discriminating Winoground images. In (b) we see the effect of the loss
function on the degree of inter-agent representational alignment and (c)
implies that according to the topsim metric, messages are more structured if
the alignment penalty is used. The presented results are across 15 seeds and
use the best-performing parameters resulting from our parameter sweep, dashed
green lines indicate averages.

##  2 Background

Most research in simulating emergent communication is modelled after the Lewis
signalling game (Lewis, 1969) with a speaker and a listener agent. The speaker
observes a state (e.g., an image) and sends a signal to the listener who acts
based on this signal. In the case of the referential game, this means
selecting a target among distractors. Both agents are rewarded for successful
communication, meaning the listener points to the target object. The solution
of this game requires the agents to have a shared protocol (i.e., an
artificial language) which typically emerges when the agents learn based on
trial and error over multiple games. This resembles how for humans, language
learning and use impose constraints like pressures for learnability and
compression that shape our language design (Kirby et al., 2014, 2015).
Importantly, the emergent language in this setup is also shaped by biases
resulting from, for example, the agent architecture, loss function, and
learning protocol Rita et al. (2022). The current work uses the referential
game: a variant of the Lewis signalling game extensively used to explore
language evolution (e.g. Steels and Loetzsch, 2012; Kirby et al., 2015;
Lazaridou et al., 2017; Kottur et al., 2017; Lazaridou et al., 2018;
Kharitonov et al., 2020; Chaabouni et al., 2022).

An important challenge in emergent communication is that artificial learners
often do not behave the same way as human learners in experimental settings.
Some emergent protocols do not follow Zipf’s law and thus are anti-efficient
unless pressures for brevity are introduced (Chaabouni et al., 2019a), others
do not show the word-order vs. case-marking trade-off found in human languages
(Chaabouni et al., 2019b; Lian et al., 2021). Additionally, there is an
ongoing debate on the degree to which the emergent languages allow for
compositional generalisation Lazaridou and Baroni (2020); Conklin and Smith
(2023). It has been suggested to introduce communicative (e.g., alternating
speaker/listener roles) and cognitive (e.g., memory) constraints (Galke et
al., 2022) and use more natural settings to promote more human-like patterns
of language emergence with neural agents (Kouwenhoven et al., 2022). Doing so
changes the learning pressures to which the agents need to adapt and can
recover initially absent linguistic phenomena of natural language in emergent
languages (for a review see Galke and Raviv, 2024). An example of such work,
investigating the word-order vs. case-marking trade-off, has succeeded in
replicating this trade-off for neural learners (Lian et al., 2023). Their
setup differs from other work in that agents first learn a miniature language
via supervised learning, and then optimise it for communicative success via
RL, resulting in emergent languages that share linguistic universals with
human language.

To enhance understanding of emergent communication in the Lewis game, Rita et
al. (2022) decomposed the standard objective in Lewis games into two key
components: a co-adaptation loss and an information loss. In doing so, they
shed light on potential sources of overfitting and how they might hinder the
emergence of structured communication protocols. They demonstrated that
desired linguistic properties (e.g., compositionality and generalisability)
emerge when they control the listener’s ability to converge to the speaker
agent (i.e., control for overfitting on the co-adaptation loss). While the co-
adaptation loss has parallels to inter-agent alignment, their work does not
address the alignment between the agents’ image representation and the input
features, which we deem crucial in developing grounded communication
protocols.

Another challenge in emergent communication is the disentanglement of the
underlying meanings of emergent languages. Earlier studies by Lazaridou et al.
(2017) suggested that agents assign symbols to general conceptual properties
of objects in images, rather than low-level visual features. However, as
previously mentioned, follow-up work from Bouchacourt and Baroni (2018) showed
this is not always the case. They found that agents align their agent-specific
image representations without developing a language that captures conceptual
properties depicted in the images. Moreover, agents lost any sense of
meaningful within-category variation where two similar objects in human
perception (e.g., two avocados) were observed as maximally dissimilar for the
agents. In response to these findings, recent studies have implemented sanity
checks, testing whether trained agents can communicate about noise Dessi et
al. (2021); Mahaut et al. (2024). However, to the best of our knowledge, there
has been little attention to what we consider to be their main result: the
alignment problem.

##  3 Representational alignment

Representational alignment is the degree of agreement between the internal
representations of two information processing systems, whether biological or
artificial. Even though widely recognised in cognitive science, neuroscience,
and machine learning (Sucholutsky et al., 2023), representational alignment
has not seen much interest in the field of emergent communication, except for
the work by Bouchacourt and Baroni who analysed the referential game using
rsa. This metric measures the alignment between two sets of numerical vectors,
for example, image embeddings and agents’ representations thereof. In
practice, it is calculated by taking the pairwise (cosine) distances between
vectors of a set and calculating the Spearman rank correlation between these
distances.

In this paper, we also use rsa to operationalise representational alignment.
Given the speaker image representations $r_{s}$ of the DinoV2 input embeddings
$i$ and $r_{l}$ as the same images represented in the listener representation
space, we compute the pairwise cosine similarity between the representations
for the speaker $s_{s}$ and for the listener $s_{l}$ and calculate Spearman’s
$\rho$ between $s_{s}$ and $s_{l}$. As such, this measures the degree of
inter-agent alignment ($\textsc{rsa}_{sl}$) between image representations
$s_{s}$ and $s_{l}$, relative to their input. Additionally, we use it to
measure image-agent alignment between the speaker/listener image
representations and the DinoV2 embeddings ($\textsc{rsa}_{si}$ and
$\textsc{rsa}_{li}$). Importantly, alignment is agnostic to the type of input,
being either images or attribute-value objects and can always happen when
inputs are projected onto agent-specific representations.

Intuitively, a high inter-agent $\textsc{rsa}_{sl}$ value can be interpreted
as agents with similar representations for similar images. Importantly, this
can have two causes: both agents’ image representations either maintain a
relation to the image input, or lose this relation. While the former is
desirable, the latter means that the agents are not communicating about the
same high-level image features but are likely communicating about non-human-
like spurious features. A low $\textsc{rsa}_{sl}$ value entails that the
agents have developed different interpretations for the same image. While this
may well be similar to the question of whether people have different
perceptual experiences of colour (Locke, 1847), in the case of emergent
communication, the agents should develop a grounded vocabulary with
overlapping concept-level properties if we wish machines to have more natural
understanding of human language. We use rsa 1) as a metric to re-assess
findings from Bouchacourt and Baroni and 2) as an auxiliary loss to mitigate
the alignment problem and ensure that the agents communicate about image
features.

##  4 Methods

The standard referential game is used as provided by the EGG framework
(Kharitonov et al., 2021). Doing so ensures our findings are representative of
the widely-used setup, rather than being influenced by specific design
decisions. The game is implemented as a multi-agent cooperative RL problem
where a speaker and a listener communicate to discriminate a target image from
two shuffled distractor images. The speaker receives a target $t$ and
generates a message $m$ of at most length $L$, using vocabulary $V$.
Importantly, the messages and symbols have no a priori meaning but are assumed
to obtain meaning and become grounded during the game. Once meaningful, the
symbols are ideally combined in a structured manner to create compositional
messages that express more complex meanings. Using message $m$, the listener
guesses the target $\hat{t}$. Communicative success is defined as $\hat{t}=t$,
meaning that the listener has correctly identified the target image among the
candidate images.

###  4.1 Agents

Agents contain a language and a vision module. The latter consists of a frozen
pre-trained visual network (DinoV2) and a learned agent-specific
representation layer. While difficult to know what conceptual image features
are present in DinoV2 embeddings, they have demonstrated capability in
semantic segmentation tasks (Oquab et al., 2024), which is similar to the
agents’ objective. In contrast to the hybrid structure of the vision module,
the language module is entirely trained from scratch.

The speaker agent processes images by applying a linear transformation to the
image embeddings, followed by batch normalisation, to create its agent-
specific image representation $r_{s}$. Its language module embeds this
representation and passes it through a single-layer Gated Recurrent Unit (GRU;
Cho et al., 2014) that spells out messages to describe the target. The
listener receives the message and the distractor images. It encodes the
message into an embedding using another single-cell GRU layer. Additionally, a
listener image representation $r_{l}$ is obtained for each image by applying a
linear transformation and batch normalisation. Subsequently, temperature-
weighted (temperature defaults to 0.1) cosine scores construct a multi-modal
representation between the image and message representation (Dessi et al.,
2021), where a higher probability should be assigned to the target image.

###  4.2 Optimisation

Communicative success ($\hat{t}=t$) is used to optimise the trainable
parameters of both agents. The listener minimises cross-entropy ($ce$) loss
using stochastic gradient descent, amounting to supervised learning. The $ce$
loss is calculated over the listeners’ target distribution, thus providing
direct pressure for communicative success. At inference, the candidate image
with the highest probability is chosen as the target $\hat{t}$. The gradients
required to optimise the speaker are calculated using the REINFORCE (Williams,
1992) update rule as each generated symbol must be assigned a loss. Following
common practice (Rita et al., 2024), entropy regularisation (Mnih et al.,
2016) is added to the loss to maintain exploration in message generation.

In addition to the conventional $ce$ loss, we introduce an alignment loss
($ce+\textsc{rsa}$) that includes an alignment penalty term to enforce high
inter-agent and image-agent alignment. The term

$L_{\textsc{rsa}}=(1-\textsc{rsa}_{sl})+(1-\textsc{rsa}_{si})+(1-\textsc{rsa}_{li})$

is added to the $ce$ loss with equal importance. We use torchsort (Blondel et
al., 2020) to calculate $L_{\textsc{rsa}}$ such that the entire loss term is
differentiable. Importantly, $L_{\textsc{rsa}}$ is not influenced by
communicative success and does not interact with the $ce$ loss (Appendix C).
Only adding $\textsc{rsa}_{sl}$ to the $ce$ loss is not sufficient as high
inter-agent alignment can be achieved while losing image-agent alignment (see
§3). We therefore also include $\textsc{rsa}_{si}$ and $\textsc{rsa}_{li}$ to
ensure that the agents communicate about the content displayed in the images.
Including $\textsc{rsa}_{sl}$ entails that representational information is
shared between the agents, thus differing from how humans interact. Yet,
ranking the speaker and listener representations in calculating
$\textsc{rsa}_{sl}$ bears some resemblance to projecting beliefs upon the
interpretations of the other communicative partner. The current solution
should be seen as a step towards more grounded vocabularies prone to
refinements such as cognitive plausibility. We train for 30 epochs regardless
of the loss used. The hyperparameters (Appendix B) that resulted in the best
validation accuracy across 42 different communication channel capacities
(Appendix A) were used for our findings.

###  4.3 Data

Agents are trained to discriminate MS COCO images but tested on three
different datasets (Figure 2) to assess out-of-distribution (o.o.d.)
performance.

Figure 2: Exemplar pairs of each dataset used for evaluation. Left: an image
pair from MS COCO. Middle: A Winoground example. Right: A Gaussian noise pair.
All images are cropped for display purposes.

MS COCO – We use a subset of 1200 images from the MS COCO 2017 validation set
to train and test the agents using an 80/20 split. To obtain this subset, we
first select the categories that contain more than 100 images (12 categories)
and subsequently sample 100 images for each supercategory present in the
resulting set of images. The distractor images are sampled from the same
category to ensure that there is some relevance to the target image.
Importantly, sampling distractor images is done for each batch, meaning
targets have different distractors at each epoch.

(a) Learning curves for the MS COCO dataset on train and validation data.

(b) Representational alignment between agent image representations (green) and
between the image and the sender/listener representations (purple, red).

Figure 3: In (a) we see that the agents learn to communicate successfully
without overfitting on train data. In (b) we see that the alignment problem
occurs with the $ce$ but not the $ce+\textsc{rsa}$ loss. Line style indicates
the loss type. Data is averaged over 15 seeds, areas indicate the 95%
confidence intervals.

Winoground – The Winoground dataset (Thrush et al., 2022) was created to
assess the visio-linguistic compositional reasoning abilities of vision and
language models. Here, we repurpose it as a proxy for the agents’ ability to
endow in compositional reasoning for image-based settings. The dataset
contains 800 images and corresponding captions, comprising 400 Winoground
pairs. Image-caption pairs were included when the captions share the same
words but are of different compositions, implying completely different
semantics (e.g., “a tree smashed into a car” versus “a car smashed into a
tree” in Figure 2 (middle)). We only use the image pairs, not the captions.
Crucially, this task differs from MS COCO since the image pairs are fixed,
conceptually similar and meant to be discriminative if the agents’ language
allows for compositional reasoning and is grounded in the visual modality.

Noise – Following Bouchacourt and Baroni (2018), we test whether agents can
communicate about Gaussian noise ($\mu=0,\sigma=1$) pairs when trained on real
images. Being able to do so would imply that messages communicate about
spurious instead of high-level concept features.

###  4.4 Metrics

The performance of our agents is assessed by communicative success (accuracy)
and rsa (§3) measures alignment. The degree of compositionality in the
emergent language is assessed through the topsim metric. Other metrics for
compositionality like positional disentanglement, bag-of-symbols
disentanglement (Chaabouni et al., 2020) are not straightforward due to the
continuous nature of the image embeddings.

##  5 Results

###  5.1 Communicative success

Unsurprisingly, results show that agents can successfully disambiguate between
image pairs from MS COCO using an emergent language (Figure 3(a)). Notably, we
also confirm previous observations by (Bouchacourt and Baroni, 2018) that
agents trained on real images can communicate about Gaussian noise (Figure
1(a)). Thus again suggesting that the messages convey spurious features rather
than concept-level information. Interestingly, their performance on Gaussian
noise is comparable to the performance on Winoground pairs, which requires the
messages to capture concept-level properties. Thus revealing the difficulty of
discriminating between strict pairs of conceptually similar images. The
observed decrease in o.o.d. performance aligns with findings from other
studies, such as Lazaridou et al. (2018) and Conklin and Smith (2023).

###  5.2 The alignment problem

The solid lines in Figure 3(b) clearly show that inter-agent alignment
increases while alignment sensitivity to image features decreases for both
agents. In principle, it is not a problem that the agents’ image
representations align. However, it is problematic when the alignment between
the image embeddings and the image representations declines. Ablations across
different channel capacities (§A) and pre-trained vision modules (§D) showed
that these trends appear consistently and are not influenced by the capacity
or type of vision model. In addition to the communicative success on Gaussian
noise, this re-confirms that the agents do not learn to extract concept-level
information from the image embeddings but instead solve this task differently.

###  5.3 topsim and representational alignment

Figure 4: The relationship between topsim and inter-agent alignment
($\textsc{rsa}_{sl}$) for both loss types.

Earlier findings show mixed results on the relationship between topsim and
generalisation in image-based settings, topsim was either related to
generalisation (Chaabouni et al., 2022) or not (Rita et al., 2022). Our
results indicate that generalisation and topsim are correlated with both $ce$
(${r=.856}$, ${p<.001}$) and $ce+\textsc{rsa}$ (${r=.767}$, ${p<.001}$)
losses. Meaning that more structured languages enable better communication on
unseen validation pairs. Moreover, we find a strong positive relationship
between $\textsc{rsa}_{sl}$ and topsim (${r=.838}$, ${p<.001}$) in the $ce$
(Figure 4). This relation is also present in the $ce+\textsc{rsa}$ setup
(${r=.408}$, ${p=.001}$), but is decoupled from topsim given the (very) small
spread (${\sigma=.003}$) of $\textsc{rsa}_{sl}$. We do not observe an
influence of inter-agent alignment on the number of uniquely produced
messages.

###  5.4 Mitigating the alignment problem

We now focus on the $ce+\textsc{rsa}$ setup which was introduced to ensure
that the agents maintain alignment with the image embeddings. Figure 3(b)
shows that this is the case: inter-agent alignment and agent-image alignment
increase during training and remain high at inference. However, there does not
seem to be a benefit for communicative success at inference time (Figure 1).
This is because the alignment penalty only forces agents to represent images
similarly to the image embeddings and is independent of the cross-entropy loss
used to assess the success of communication (Appendix C). In the case of noise
images, we still observe the above-chance performance, suggesting that
communication between the agents still occurs in an artificial manner.

The alignment penalty also leads to increased topsim, indicating a higher
level of structure (Figure 1(c)) and strengthens our finding that topsim and
inter-agent alignment are related. Suggesting that the observed variations in
topsim, whether higher or lower, as noted in previous studies (e.g. Kottur et
al., 2017; Chaabouni et al., 2020), should not be interpreted without
considering alignment since they may be attributable to this underlying
artefact rather than alterations to the original setup.

When tested on more strict Winoground pairs, communicative success does not
improve as a result of using the alignment penalty (Figure 1(a)). Given the
correlation between topsim and generalisation, this is surprising since the
higher degree of topsim should imply that the language is more structured.
Moreover, both, $\textsc{rsa}_{si}$ and $\textsc{rsa}_{li}$ have not drifted
away from the image features. This combination, in theory, should be ideal for
discriminating image pairs from the Winoground dataset since it was designed
to be discriminative with compositional visio-linguistic reasoning. However,
in practice this is not the case.

##  6 Discussion

In this work, we revisited the representational alignment problem in a common
setup used in emergent communication and proposed a solution to this
underrepresented problem. We corroborated earlier findings by showing that
agents align their image representations and rely on spurious image features
instead of human-like concept-level information (Bouchacourt and Baroni,
2018). We then showed that inter-agent alignment strongly correlates with the
commonly used topsim metric. Our solution to the alignment problem involves an
alignment penalty that forces the agents to remain aligned with the input
features and mitigates the alignment problem without decreasing communicative
success. Finally, when agents are tested on more challenging Winoground pairs
we observed reasonable but lower performance regardless of whether image
representations were similar to the image embeddings or not. With this work,
we hope that the alignment problem will receive more attention in the field of
emergent communication, as is already the case in adjacent fields (Sucholutsky
et al., 2023).

###  6.1 Importance of representational alignment

It is common practice in simulations of emergent communication to process
(visual) inputs into an agent-specific hidden representation and update their
weights simultaneously (e.g. Lazaridou et al., 2017; Bouchacourt and Baroni,
2018; Chaabouni et al., 2019a, 2020; Rita et al., 2022). As such, inter-agent
alignment, irrespective of the input form, likely happens in other simulations
too. This phenomenon is therefore potentially widespread and perhaps the cause
for findings that are at odds with experimental findings. While it is not
always the case that the representation structure we expect to help solve a
task will do so (e.g. Montero et al., 2021; Xu et al., 2022), such
discrepancies may hinder the use of emergent communication models in
developing a more natural understanding of human languages and leave them less
suitable for directly simulating language evolution phenomena. Especially if
we want machine representations of natural language to align with human
representations (Sucholutsky et al., 2023). rsa should therefore be used to
rule out, or at the bare minimum report about, representational alignment in
the future.

###  6.2 topsim and representational alignment

Measuring representational alignment using rsa is similar to how topsim
measures the structure in messages. They differ in their inputs but both
calculate the Spearman-ranked correlation between metric-agnostic pairwise
distances. Crucially, the input makes all the difference, the inputs for rsa
are from both agents and are trained independently, whilst topsim only
assesses the relation between the fixed inputs and learned output. Despite the
similarities, the metrics thus describe different phenomena and are rarely
reported simultaneously.

We hypothesise that the relationship between topsim and inter-agent
representational alignment is a by-product of the setup, which in essence
implies that the listener has to align its representation $r_{l}$ to the
speaker representation $r_{s}$ (Rita et al., 2022). It has to do so using only
the speakers’ messages, being an abstraction of $r_{s}$. A solution to this
problem is to align representations, which eases the listeners’ training
objective. If the speaker consistently produces structured messages during
training, aligning $r_{l}$ to $r_{s}$ is easier, thereby causing higher inter-
agent alignment. Essentially, this renders topsim to be an indirect metric for
the rate of alignment, for which $\textsc{rsa}_{sl}$ is a direct metric. In
the context of learnability, the relationship between topsim and inter-agent
alignment and the fact that alignment always occurs can be seen as reasons for
why languages with higher topsim are easier to learn (Li and Bowling, 2019;
Cheng et al., 2023). This underscores the need to report inter-agent
representational alignment to avoid conclusions drawn about the effect of
specific interventions on topsim which may be attributable to inter-agent
alignment.

###  6.3 Targeted o.o.d. evaluations

An important implication of our findings concerns the standard practice of
reporting o.o.d. accuracy where the agents are tested on unseen input after
training (e.g. Auersperger and Pecina, 2022; Conklin and Smith, 2023). This
should inform about the agents’ ability to generalise from one dataset (e.g.,
MS COCO) to another dataset (e.g., the Winoground pairs) much like human
language allows us to talk about an infinite number of situations. Crucially,
this overlooks the representational alignment problem in that we do not know
what the agents are precisely generalising about. This problem can be
mitigated with the alignment penalty to assess generalisation more directly or
at least should be taken into consideration.

We assess o.o.d. performance on the more challenging Winoground pairs as a
proxy for the agents’ ability to endow in compositional reasoning for image-
based settings. Good performance on the Winoground dataset requires a grounded
language that can be used to create compositional messages since the objects
and their underlying relations need to be described. In general, we suggest to
start evaluating simulations of referential games on targeted strict tasks,
like probing state-of-the-art vision language models on e.g., visio-
compositional (Thrush et al., 2022; Diwan et al., 2022; Hsieh et al., 2023;
Ray et al., 2023) or spatial (Kamath et al., 2023) reasoning. Re-purposing
such datasets can reveal more directly whether agents develop the attested
communicative abilities that are trivial to humans without having to rely on
metrics. Our results illustrate this through a shortcoming of the topsim
metric. We observed that agents still struggle with distinguishing pairs of
conceptually similar Winoground images even though topsim is higher with the
alignment penalty. If the language protocol were to communicate concept-level
information and compositional messages were created, we should not observe
this struggle, meaning that the emerged protocols do not enable human-like
communicative success.

Interestingly, the o.o.d. performance remains substantially above chance in
the $ce+\textsc{rsa}$ setting. Given that MS COCO is not a dataset for
learning to model compositionality, this delineates the limits of what can be
achieved qua performance based on MS COCO image features in the Winoground
context. Nevertheless, this leaves open the question of above-chance
performance on Gaussian noise with the $ce+\textsc{rsa}$ loss. A tentative
explanation is that the higher inter-agent alignment on noise input
(${M_{ce}=.428}$, ${M_{ce+\textsc{rsa}_{sl}}=.543}$, ${t=-8.71}$, ${p<.001}$)
alleviates part of the problem (Figure 1(b)). To validate this, future
experiments should involve controlling the prior distributions of the agents’
image encoders by training their vision modules on different data. Doing so
ensures that they have to communicate about novel objects and cannot rely on
similar representations.

##  7 Conclusion

This paper revisits the underrepresented alignment problem present in the
referential game often used in simulations of emergent communication.
Specifically, we focused on the problem of increasing alignment between agent-
image representations in combination with a decreasing alignment between the
input and agent representations. We first confirmed that agents align their
image representations while losing connection to their input, meaning that the
emergent languages do not appear to encode human-like visual features. We then
showed that, in the common setup, inter-agent alignment is related to
topographic similarity, and argued that this renders topsim an indirect metric
of the rate of inter-agent alignment. To further investigate the effects of
alignment, we introduced an alignment penalty to mitigate the alignment
problem and showed that the communicative ability on a strict compositionality
benchmark did not improve, leaving the question of inducing compositional
generalisation in emergent communication for images unsolved. Our findings
underscore the need to better understand the divergence between human and
artificial language emergence within the prevalent referential setup and
highlight the importance and potential impact of representational alignment.
We hope that future work rules out or at least reports about representational
alignment.

##  8 Limitations

Our work has a few notable limitations. First, it only involves the
referential game. Another popular variant, the reconstruction game (e.g.
Chaabouni et al., 2019a, 2020; Lian et al., 2021; Conklin and Smith, 2023),
requires the listener to reconstruct the input object based on the speakers’
message. Since this setup has a different objective and presents different
learning biases, it may have different results. We still expect the results to
be similar as there is no pressure to retain alignment between the image input
and agent representation. It would, however, be interesting to investigate
whether the language protocol in this scenario is more structured than in the
referential game.

Another limitation in our setup is that we only consider the scenario with two
agents, which may be a requirement for alignment to be possible. Since
experiments with human participants show that larger communities create more
systematic languages (Raviv et al., 2019b), simulations on emergent multi-
agent communication with populations of agents are also conducted, but these
yield mixed results. The emergent communication protocols oftentimes do not
evolve to be more structured unless explicit pressures such as population
diversity or emulation mechanisms are introduced Rita et al. (2022); Chaabouni
et al. (2022). Michel et al. (2023) however, showed that population setups can
result in more compositional languages if agent pairs are trained in a
partitioned manner to prevent co-adaptation. Despite the mixed results, we
believe that emergent communication with populations of agents is ecologically
more valid and could result in different alignment effects. Much like how
Tieleman et al. (2019) showed that autoencoders encode better concept category
representations when they learn representations in a community-based setting
with multiple encoders and decoders collectively.

The final limitation of our study regards its scale. While simulations of
emergent communication are typically conducted on relatively small-scale
datasets, human language emergence is accompanied by rich and diverse multi-
modal experiences. Recent results in the field of computer vision suggest that
dataset diversity and scale are the primary drivers of alignment to human
representations (Conwell et al., 2023; Muttenthaler et al., 2023). As such,
this key difference between the setting of artificial emergent communication
and human language emergence can drive the observed differences in
representations. Due to the difficulty of interpreting these representations,
we see this as another reason to evaluate emergent protocols on more strict
datasets with clear pragmatic value for humans.

## References

  * Auersperger and Pecina (2022) Michal Auersperger and Pavel Pecina. 2022.  Defending compositionality in emergent languages.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop_ , pages 285–291, Hybrid: Seattle, Washington + Online. Association for Computational Linguistics.
  * Blondel et al. (2020) Mathieu Blondel, Olivier Teboul, Quentin Berthet, and Josip Djolonga. 2020.  Fast differentiable sorting and ranking.  In _International Conference on Machine Learning_ , pages 950–959. PMLR.
  * Bouchacourt and Baroni (2018) Diane Bouchacourt and Marco Baroni. 2018.  How agents see things: On visual representations in an emergent language game.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 981–985, Brussels, Belgium. Association for Computational Linguistics.
  * Brighton and Kirby (2006) Henry Brighton and Simon Kirby. 2006.  Understanding linguistic evolution by visualizing the emergence of topographic mappings.  _Artificial Life_ , 12(2):229–242.
  * Chaabouni et al. (2020) Rahma Chaabouni, Eugene Kharitonov, Diane Bouchacourt, Emmanuel Dupoux, and Marco Baroni. 2020.  Compositionality and generalization in emergent languages.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 4427–4442, Online. Association for Computational Linguistics.
  * Chaabouni et al. (2019a) Rahma Chaabouni, Eugene Kharitonov, Emmanuel Dupoux, and Marco Baroni. 2019a.  Anti-efficient encoding in emergent communication.  In _Advances in Neural Information Processing Systems_ , volume 32. Curran Associates, Inc.
  * Chaabouni et al. (2019b) Rahma Chaabouni, Eugene Kharitonov, Alessandro Lazaric, Emmanuel Dupoux, and Marco Baroni. 2019b.  Word-order biases in deep-agent emergent communication.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 5166–5175, Florence, Italy. Association for Computational Linguistics.
  * Chaabouni et al. (2022) Rahma Chaabouni, Florian Strub, Florent Altché, Eugene Tarassov, Corentin Tallec, Elnaz Davoodi, Kory Wallace Mathewson, Olivier Tieleman, Angeliki Lazaridou, and Bilal Piot. 2022.  Emergent communication at scale.  In _International Conference on Learning Representations_.
  * Cheng et al. (2023) Emily Cheng, Mathieu Rita, and Thierry Poibeau. 2023.  On the correspondence between compositionality and imitation in emergent neural communication.  In _Findings of the Association for Computational Linguistics: ACL 2023_ , pages 12432–12447, Toronto, Canada. Association for Computational Linguistics.
  * Cho et al. (2014) Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014.  Learning phrase representations using RNN encoder–decoder for statistical machine translation.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1724–1734, Doha, Qatar. Association for Computational Linguistics.
  * Conklin and Smith (2023) Henry Conklin and Kenny Smith. 2023.  Compositionality with variation reliably emerges in neural networks.  In _The Eleventh International Conference on Learning Representations_.
  * Conwell et al. (2023) Colin Conwell, Jacob S. Prince, Kendrick N. Kay, George A. Alvarez, and Talia Konkle. 2023.  What can 1.8 billion regressions tell us about the pressures shaping high-level visual representation in brains and machines? _bioRxiv_.
  * Dagan et al. (2021) Gautier Dagan, Dieuwke Hupkes, and Elia Bruni. 2021.  Co-evolution of language and agents in referential games.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 2993–3004, Online. Association for Computational Linguistics.
  * de Boer (2006) Bart de Boer. 2006.  Computer modeling as a tool for understanding language evolution.  In Nathalie Gontier, Jean Paul Van Bendegem, and Diederik Aerts, editors, _Evolutionary Epistemology, Language and Culture_ , pages 381–406. Springer, Dordrecht.
  * Dessi et al. (2021) Roberto Dessi, Eugene Kharitonov, and Baroni Marco. 2021.  Interpretable agent communication from scratch (with a generic visual processor emerging on the side).  In _Advances in Neural Information Processing Systems_ , volume 34, pages 26937–26949. Curran Associates, Inc.
  * Diwan et al. (2022) Anuj Diwan, Layne Berry, Eunsol Choi, David Harwath, and Kyle Mahowald. 2022.  Why is winoground hard? investigating failures in visuolinguistic compositionality.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 2236–2250, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Galke et al. (2022) Lukas Galke, Yoav Ram, and Limor Raviv. 2022.  Emergent communication for understanding human language evolution: What’s missing? In _Emergent Communication Workshop at ICLR 2022_.
  * Galke and Raviv (2024) Lukas Galke and Limor Raviv. 2024.  Emergent communication and learning pressures in language models: a language evolution perspective.  _arXiv preprint arXiv:2403.14427_.
  * Guo et al. (2019) Shangmin Guo, Yi Ren, Serhii Havrylov, Stella Frank, Ivan Titov, and Kenny Smith. 2019.  The emergence of compositional languages for numeric concepts through iterated learning in neural agents.
  * He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016.  Deep residual learning for image recognition.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 770–778.
  * Hockett (1959) Charles F Hockett. 1959.  Animal" languages" and human language.  _Human Biology_ , 31(1):32–39.
  * Hsieh et al. (2023) Cheng-Yu Hsieh, Jieyu Zhang, Zixian Ma, Aniruddha Kembhavi, and Ranjay Krishna. 2023\.  Sugarcrepe: Fixing hackable benchmarks for vision-language compositionality.  In _Advances in Neural Information Processing Systems_ , volume 36, pages 31096–31116. Curran Associates, Inc.
  * Kamath et al. (2023) Amita Kamath, Jack Hessel, and Kai-Wei Chang. 2023.  What’s “up” with vision-language models? investigating their struggle with spatial reasoning.  In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_ , pages 9161–9175, Singapore. Association for Computational Linguistics.
  * Kharitonov et al. (2020) Eugene Kharitonov, Rahma Chaabouni, Diane Bouchacourt, and Marco Baroni. 2020.  Entropy minimization in emergent languages.  In _Proceedings of the 37th International Conference on Machine Learning_ , volume 119 of _Proceedings of Machine Learning Research_ , pages 5220–5230. PMLR.
  * Kharitonov et al. (2021) Eugene Kharitonov, Roberto Dessì, Rahma Chaabouni, Diane Bouchacourt, and Marco Baroni. 2021.  EGG: a toolkit for research on Emergence of lanGuage in Games.  https://github.com/facebookresearch/EGG.
  * Kirby et al. (2008) Simon Kirby, Hannah Cornish, and Kenny Smith. 2008.  Cumulative cultural evolution in the laboratory: An experimental approach to the origins of structure in human language.  _Proceedings of the National Academy of Sciences_ , 105(31):10681–10686.
  * Kirby et al. (2014) Simon Kirby, Tom Griffiths, and Kenny Smith. 2014.  Iterated learning and the evolution of language.
  * Kirby et al. (2015) Simon Kirby, Monica Tamariz, Hannah Cornish, and Kenny Smith. 2015.  Compression and communication in the cultural evolution of linguistic structure.  _Cognition_ , 141:87–102.
  * Kottur et al. (2017) Satwik Kottur, José Moura, Stefan Lee, and Dhruv Batra. 2017.  Natural language does not emerge ‘naturally’ in multi-agent dialog.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 2962–2967, Copenhagen, Denmark. Association for Computational Linguistics.
  * Kouwenhoven et al. (2022) Tom Kouwenhoven, Tessa Verhoef, Roy de Kleijn, and Stephan Raaijmakers. 2022.  Emerging Grounded Shared Vocabularies Between Human and Machine, Inspired by Human Language Evolution.  _Frontiers in Artificial Intelligence_ , 5.
  * Kriegeskorte et al. (2008) Nikolaus Kriegeskorte, Marieke Mur, and Peter A Bandettini. 2008.  Representational similarity analysis-connecting the branches of systems neuroscience.  _Frontiers in systems neuroscience_ , 2:4.
  * Lazaridou and Baroni (2020) Angeliki Lazaridou and Marco Baroni. 2020.  Emergent multi-agent communication in the deep learning era.  _arXiv preprint arXiv:2006.02419_.
  * Lazaridou et al. (2018) Angeliki Lazaridou, Karl Moritz Hermann, Karl Tuyls, and Stephen Clark. 2018.  Emergence of linguistic communication from referential games with symbolic and pixel input.  In _International Conference on Learning Representations_.
  * Lazaridou et al. (2017) Angeliki Lazaridou, Alexander Peysakhovich, and Marco Baroni. 2017.  Multi-agent cooperation and the emergence of (natural) language.  _5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings_ , pages 1–11.
  * Lewis (1969) David Lewis. 1969.  Convention: A philosophical study.  _Cambridge, MA_.
  * Li and Bowling (2019) Fushan Li and Michael Bowling. 2019.  Ease-of-teaching and language structure from emergent communication.  In _Advances in Neural Information Processing Systems_ , volume 32. Curran Associates, Inc.
  * Lian et al. (2021) Yuchen Lian, Arianna Bisazza, and Tessa Verhoef. 2021.  The effect of efficient messaging and input variability on neural-agent iterated language learning.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 10121–10129, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Lian et al. (2023) Yuchen Lian, Arianna Bisazza, and Tessa Verhoef. 2023.  Communication Drives the Emergence of Language Universals in Neural Agents: Evidence from the Word-order/Case-marking Trade-off.  _Transactions of the Association for Computational Linguistics_ , 11:1033–1047.
  * Lin et al. (2014) Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014.  Microsoft coco: Common objects in context.  In _Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13_ , pages 740–755. Springer.
  * Locke (1847) John Locke. 1847.  _An essay concerning human understanding_, volume 114.  Kay & Troutman.
  * Lowe et al. (2020) Ryan Lowe, Abhinav Gupta, Jakob Foerster, Douwe Kiela, and Joelle Pineau. 2020.  On the interaction between supervision and self-play in emergent communication.  In _International Conference on Learning Representations_.
  * Mahaut et al. (2024) Matéo Mahaut, Francesca Franzon, Roberto Dessì, and Marco Baroni. 2024.  Referential communication in heterogeneous communities of pre-trained visual deep networks.
  * Michel et al. (2023) Paul Michel, Mathieu Rita, Kory Wallace Mathewson, Olivier Tieleman, and Angeliki Lazaridou. 2023.  Revisiting populations in multi-agent communication.  In _The Eleventh International Conference on Learning Representations_.
  * Mnih et al. (2016) Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016.  Asynchronous methods for deep reinforcement learning.  In _Proceedings of The 33rd International Conference on Machine Learning_ , volume 48 of _Proceedings of Machine Learning Research_ , pages 1928–1937, New York, New York, USA. PMLR.
  * Montero et al. (2021) Milton Llera Montero, Casimir JH Ludwig, Rui Ponte Costa, Gaurav Malhotra, and Jeffrey Bowers. 2021.  The role of disentanglement in generalisation.  In _International Conference on Learning Representations_.
  * Mu and Goodman (2021) Jesse Mu and Noah Goodman. 2021.  Emergent communication of generalizations.  In _Advances in Neural Information Processing Systems_ , volume 34, pages 17994–18007. Curran Associates, Inc.
  * Muttenthaler et al. (2023) Lukas Muttenthaler, Jonas Dippel, Lorenz Linhardt, Robert A. Vandermeulen, and Simon Kornblith. 2023.  Human alignment of neural network representations.  In _The Eleventh International Conference on Learning Representations_.
  * Oquab et al. (2024) Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. 2024.  DINOv2: Learning robust visual features without supervision.  _Transactions on Machine Learning Research_.
  * Radford et al. (2021) Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021.  Learning transferable visual models from natural language supervision.  In _International conference on machine learning_ , pages 8748–8763. PMLR.
  * Raviv et al. (2019a) Limor Raviv, Antje Meyer, and Shiri Lev-Ari. 2019a.  Compositional structure can emerge without generational transmission.  _Cognition_ , 182:151–164.
  * Raviv et al. (2019b) Limor Raviv, Antje Meyer, and Shiri Lev-Ari. 2019b.  Larger communities create more systematic languages.  _Proceedings of the Royal Society B_ , 286(1907):20191262.
  * Ray et al. (2023) Arijit Ray, Filip Radenovic, Abhimanyu Dubey, Bryan Plummer, Ranjay Krishna, and Kate Saenko. 2023.  Cola: A benchmark for compositional text-to-image retrieval.  In _Advances in Neural Information Processing Systems_ , volume 36, pages 46433–46445. Curran Associates, Inc.
  * Ren et al. (2020) Yi Ren, Shangmin Guo, Matthieu Labeau, Shay B. Cohen, and Simon Kirby. 2020.  Compositional languages emerge in a neural iterated learning model.  In _International Conference on Learning Representations_.
  * Rita et al. (2024) Mathieu Rita, Paul Michel, Rahma Chaabouni, Olivier Pietquin, Emmanuel Dupoux, and Florian Strub. 2024.  Language evolution with deep learning.  _arXiv preprint arXiv:2403.11958_.
  * Rita et al. (2022) Mathieu Rita, Corentin Tallec, Paul Michel, Jean-Bastien Grill, Olivier Pietquin, Emmanuel Dupoux, and Florian Strub. 2022.  Emergent communication: Generalization and overfitting in lewis games.  In _Advances in Neural Information Processing Systems_ , volume 35, pages 1389–1404. Curran Associates, Inc.
  * Selten and Warglien (2007) Reinhard Selten and Massimo Warglien. 2007.  The emergence of simple languages in an experimental coordination game.  _Proceedings of the National Academy of Sciences_ , 104(18):7361–7366.
  * Smith (2022) Kenny Smith. 2022.  How Language Learning and Language Use Create Linguistic Structure.  _Current Directions in Psychological Science_ , 31(2):177–186.
  * Steels and Loetzsch (2012) Luc Steels and Martin Loetzsch. 2012.  The Grounded Naming Game.  _Experiments in cultural language evolution_ , 3:41–59.
  * Steinert-Threlkeld et al. (2022) Shane Steinert-Threlkeld, Xuhui Zhou, Zeyu Liu, and C. M. Downey. 2022.  Emergent communication fine-tuning (EC-FT) for pretrained language models.  In _Emergent Communication Workshop at ICLR 2022_.
  * Sucholutsky et al. (2023) Ilia Sucholutsky, Lukas Muttenthaler, Adrian Weller, Andi Peng, Andreea Bobu, Been Kim, Bradley C. Love, Erin Grant, Iris Groen, Jascha Achterberg, Joshua B. Tenenbaum, Katherine M. Collins, Katherine L. Hermann, Kerem Oktar, Klaus Greff, Martin N. Hebart, Nori Jacoby, Qiuyi Zhang, Raja Marjieh, Robert Geirhos, Sherol Chen, Simon Kornblith, Sunayana Rane, Talia Konkle, Thomas P. O’Connell, Thomas Unterthiner, Andrew K. Lampinen, Klaus-Robert Müller, Mariya Toneva, and Thomas L. Griffiths. 2023.  Getting aligned on representational alignment.
  * Thrush et al. (2022) Tristan Thrush, Ryan Jiang, Max Bartolo, Amanpreet Singh, Adina Williams, Douwe Kiela, and Candace Ross. 2022.  Winoground: Probing vision and language models for visio-linguistic compositionality.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 5238–5248.
  * Tieleman et al. (2019) Olivier Tieleman, Angeliki Lazaridou, Shibl Mourad, Charles Blundell, and Doina Precup. 2019.  Shaping representations through communication: community size effect in artificial learning systems.  _arXiv preprint arXiv:1912.06208_.
  * Williams (1992) Ronald J Williams. 1992.  Simple statistical gradient-following algorithms for connectionist reinforcement learning.  _Machine learning_ , 8:229–256.
  * Xu et al. (2022) Zhenlin Xu, Marc Niethammer, and Colin A Raffel. 2022.  Compositional generalization in unsupervised compositional representation learning: A study on disentanglement and emergent language.  In _Advances in Neural Information Processing Systems_ , volume 35, pages 25074–25087. Curran Associates, Inc.
  * Zheng et al. (2024) Chenhao Zheng, Jieyu Zhang, Aniruddha Kembhavi, and Ranjay Krishna. 2024.  Iterated learning improves compositionality in large vision-language models.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 13785–13795.

##  Appendix A Channel capacity

To test to what degree communicative success, topsim, and representational
alignment are confounded with the communication channel capacity, we ran
simulations altering the vocabulary size ($V=\\{3,5,10,20,40,50,100\\}$) and
message length ($L=\\{2,3,5,10,50,100\\}$) resulting in 42 parameter settings
per loss type.

Overall, performance is relatively independent of the chosen configuration,
but vocabulary size influences success more than message length (Figure 5).
The hyperparameters that resulted in the best validation accuracy (i.e.,
generalisation; Chaabouni et al., 2022) for the standard $ce$ setup were
$V=40$ and $L=2$. These parameters are used to produce the results in the main
paper. Contra expectations, the vocabulary size also influenced topsim more
than message length. It, especially in the case of $ce+L_{\textsc{rsa}}$, is
higher when messages are shorter but have access to a larger vocabulary
(Figure 6).

Figure 5: The validation accuracy as a dependent factor of the vocabulary size
and maximum message length. Values are averages across 15 seeds. Figure 6:
topsim as a dependent factor of the vocabulary size and maximum message
length. Values are averages across 15 seeds.

Figure 7 shows that, regardless of capacity, inter-agent alignment
($\textsc{rsa}_{sl}$) increases while image-agent alignment
($\textsc{rsa}_{si}$ and $\textsc{rsa}_{li}$) decreases with the $ce$ loss.
Interestingly, $\textsc{rsa}_{sl}$ is agnostic to capacity but a larger
vocabulary size, not message length, reduces the degree of drifting away from
the input. We hypothesise this to result from lower pressure to compress rich
continuous embeddings into smaller discrete vocabulary embeddings.

Figure 7: Representational alignment metrics averaged over 15 simulations with
the standard $ce$ loss. Regardless of channel capacity, representational
alignment always occurs while losing relation to the input.

##  Appendix B Best hyperparameters

The parameters used to run the experiments in the main paper were the
following:

Parameter | Value
---|---
Batch size | 32
Optimiser | Adam
Learning Rate (S & L) | 0.01 & 0.001
Vocabulary size ($V$) | 40
Message length ($L$) | 2
Hidden size (S & L) | 768 & 768
Embedding size | 50
Listener cosine temperature | 0.1
Seeds |  | 16,22,41,56,67,
---
77,14,78,99,23,
82,40,51,37,62
Table 1: Best-performing parameters resulting from the parameter sweep.

##  Appendix C Interaction of the alignment term on the cross-entropy loss

To ensure that there is no impact of the alignment penalty on the pressure for
communicative success, we ablated the $L_{\textsc{rsa}}$ term of our proposed
loss function and found that both, communicative success and $ce$ are not
affected by the alignment penalty (Figure 8). Corroborating that only the $ce$
term provides pressure for successful communication (§5.4).

Figure 8: Learning curves (accuracy) and cross-entropy loss ($ce$) for both
loss settings. There is virtually no effect of the auxiliary term
$L_{\textsc{rsa}}$ on the cross entropy loss or communicative success.

##  Appendix D Pre-trained vision modules

Although it is in principle possible to train the vision module of the agents
from scratch (Dessi et al., 2021), in our work, agents’ perception stems from
a pre-trained vision-language model. Although there is reason to believe that
DinoV2 embeddings capture high-level, conceptual image features useful for
discriminating image pairs Oquab et al. (2024), we assessed the degree to
which the alignment problem occurs for different pre-trained models despite
encoding the same objects. We ran additional simulations using image features
obtained from ResNet (He et al., 2016) and CLIP (Radford et al., 2021) for 6
different parameter settings with the $ce$ loss function. Here we used the
parameters that resulted in the best, worst, mean, and quantile validation
performance from the parameter sweep in appendix A (see Table 2), and a
sensible setup with $V=10$ and $L=5$.

Msg. Length ($L$) | Vocab. Size ($V$) | Vision
---|---|---
2 | 40 |  DinoV2 CLIP ResNet
3 | 10
5 | 5
5 | 10
10 | 3
50 | 100
Table 2: The parameters for running additional simulations with CLIP and
ResNet to assess the robustness of our results. Each combination was run for
15 different seeds. Note: results for the DinoV2 simulations are from the
sweep. Figure 9: Learning curves (accuracy) and rsa metrics for different
vision models averaged over 6 parameter settings with 15 seeds each. The
representational alignment problem always occurs. Line style corresponds to
the vision module used to obtain image embeddings and colour indicates the
metric. Areas indicate the 95% confidence intervals.

Figure 9 shows clearly that inter-agent alignment increases while agent-image
alignment decreases for all models. In addition to the similar results
reported by Bouchacourt and Baroni (2018) for VGG ConvNet embeddings, both
4096 and 1000 layers, we can confirm that the problem is agnostic to the input
embeddings. Interestingly, agent representations drift most for CLIP
embeddings. Nevertheless, the agents still develop a successful communication
strategy, indicating that out-of-the-box CLIP embeddings are the least useful
for agents in finding a (non-grounded) solution. No such differences are seen
when the agents are trained with the additional alignment penalty term, inter-
agent and image-agent alignment remain high for all models.