# UniCon: Unidirectional Split Learning with Contrastive Loss for Visual
Question Answering

Yuwei Sun
University of Tokyo
RIKEN AIP
<EMAIL_ADDRESS>
&Hideya Ochiai
University of Tokyo
<EMAIL_ADDRESS>

###### Abstract

Visual question answering (VQA) that leverages multi-modality data has
attracted intensive interest in real-life applications, such as home robots
and clinic diagnoses. Nevertheless, one of the challenges is to design robust
learning for different client tasks. This work aims to bridge the gap between
the prerequisite of large-scale training data and the constraint of client
data sharing mainly due to confidentiality. We propose the Unidirectional
Split Learning with Contrastive Loss (UniCon) to tackle VQA tasks training on
distributed data silos. In particular, UniCon trains a global model over the
entire data distribution of different clients learning refined cross-modal
representations via contrastive learning. The learned representations of the
global model aggregate knowledge from different local tasks. Moreover, we
devise a unidirectional split learning framework to enable more efficient
knowledge sharing. The comprehensive experiments with five state-of-the-art
VQA models on the VQA-v2 dataset demonstrated the efficacy of UniCon,
achieving an accuracy of 49.89% in the validation set of VQA-v2. This work is
the first study of VQA under the constraint of data confidentiality using
self-supervised Split Learning.

## Introduction

The real-world deployment of multi-modal machine learning (MMML) in safety-
critical applications such as healthcare needs to address a variety of model
vulnerabilities for robust architecture design. The adversarial attacks on
MMML usually include two different types according to the attacking target,
i.e., data poisoning (Bagdasaryan et al., 2020; Bagdasaryan and Shmatikov,
2020; Lin et al., 2020; Wang et al., 2020a; Walmer et al., 2022) and model
poisoning (Wang et al., 2020a; Fang et al., 2020; Zhou et al., 2021). A large
body of works showed the venerability of an MMML model like Visual Question
Answering (VQA) to the attacks (Walmer et al., 2022) but a few of them
addressed the robust architecture design for an MMML model. In this work, we
devise a novel Split Learning-based VQA framework to alleviate the threats
from the two types of poisoning attacks.

Figure 1: The overall architecture of UniCon. UniCon comprises the cross-modal
representation learning, the answer projection network for semantic notion
understanding of answers, and two adapter networks for the contrastive
learning of different model component outputs. UniCon learns refined
representations by encouraging the similarity of the relevant component
outputs while discouraging the irrelevant component outputs.

Moreover, there has been an increasing need for personal models working at the
edge, mainly due to data confidentiality. Nevertheless, a standalone model is
prone to have small training data that decreases the model performance. In
this regard, decentralized machine learning methods (Sun, Ochiai, and Esaki,
2021) like federated learning (McMahan et al., 2017) are used to facilitate
the distributed model training across different data silos without disclosing
the raw data. Furthermore, in MMML tasks like VQA whose goal is to answer
natural language questions according to images, decentralized learning has
been attracting increasing attention for personal applications. For instance,
connected home robots could learn to better generalize to new dialogues based
on knowledge transferred from other robots via model parameters or
representation sharing. So that sensitive personal data will not be disclosed
and used by a third party. Training a VQA model usually necessitates a large
number of multi-modality data, which involves a broad range of personal
interests in both natural language texts and image contents. The privacy-
preserving VQA has not been studied until recently(Liu et al., 2020). However,
the efforts in the previous work using federated learning could only guarantee
data privacy but not model privacy where training was performed via model
parameter sharing.

In this work, we propose a novel contrastive loss-based split learning
framework to train a VQA model without disclosing either the training data or
the model. In particular, we tackle VQA as a self-supervised learning task
instead of a multi-class classification task. The proposed method learns
refined cross-modal representations from different question-image-answer
triplets without the supervision of labels. So that the representations from
relevant triplets stay close while the representations from irrelevant
triplets are far apart (Fig. 1). We termed the proposed framework the
Unidirectional Split Learning with Contrastive Loss (UniCon). UniCon
facilitates the privacy-preserving model training on distributed data silos,
safeguarding vision and natural language data for VQA. We hope that this work
will motivate future research in robust learning for personal multimodal
models.

The main contributions of this work are the following.

1) This work is the first study of Visual Question Answering (VQA) under the
constraint of data confidentiality using Split Learning.

2) We devise a new contrastive loss-based split learning framework, called
Unidirectional Split Learning with Contrastive Loss (UniCon), to overcome
inefficiency in the bidirectional representations and gradients sharing in
classical Split Learning and train a VQA model end-to-end.

3) This paper demonstrates the contrastive learning of different model
components for training a global VQA model on the fly. UniCon aligns different
modality representations and optimizes the entire model by encouraging the
similarity of the relevant model component outputs while discouraging the
irrelevant component outputs.

4) We present an in-depth evaluation on the VQA-v2 dataset using a wide range
of state-of-the-art models including MFB, BUTD, BAN, MMNas, and MCAN. The
empirical results demonstrated the efficacy of UniCon in the different
learning settings achieving an accuracy of 49.89% in the VQA-v2 validation
set.

## Related Work

### Multimodal Machine Learning and Visual Question Answering

Information in the real world usually comes in different modalities.
Multimodal machine learning (MMML) (Alayrac et al., 2020; Radford et al.,
2021; Rouditchenko et al., 2021; Ramesh et al., 2021, 2022) has been
intensively studied with significant progress in cross-modal understanding and
reasoning. In particular, Visual Question Answering (VQA) is the task to
answer a natural language question according to the contents of a presented
image. VQA is actively studied (Yang et al., 2016a; Anderson et al., 2018; Kim
et al., 2018; Yu et al., 2020) with recent years’ progress in the attention
mechanism (Vaswani et al., 2017a). Nevertheless, the vast majority of VQA
studies so far are based on the modality network fusion methods (Yang et al.,
2016a; Kim et al., 2017) where the VQA task is considered a multi-class
classification. Such an assumption hinders the understanding of the semantic
notions embedded in the natural language answers. Moreover, in practice, these
studies do not touch on the privacy of VQA when using multi-modality data to
train a model. In this work, we aim to bridge the gap between the large-scale
training of VQA and the constraint of data sharing mainly due to
confidentiality (Table 1).

Methods | Shared Training Data | Shared Training Model | Learning Framework | Loss Function
---|---|---|---|---
Yu et al. (2020) | $\checkmark$ | ✓ | Single fusion model | Cross entropy
Zhu et al. (2020) | $\checkmark$ | ✓ | Single fusion model | Cross entropy + Contrastive loss
Liu et al. (2020) | $\times$ | $\checkmark$ | Federated learning | Cross entropy
UniCon (Ours) | $\times$ | $\times$ | Unidirectional split leaning | Contrastive loss
Table 1: Summary of existing work that was designed for tackling VQA tasks.
Compared to the previous work, UniCon does not require sharing training data
or models. This is enabled by a unidirectional split learning framework and
contrastive loss.

### Contrastive Learning

The prerequisite of large-scale training tasks like VQA that requires a decent
amount of labeled modality data hinders the efficacy of cross-modal learning
since data collection and labeling processes are usually effortful. Most VQA
research has focused on supervised learning where the one-hot vectors of
answer classes are employed to perform a multi-class classification (Yang et
al., 2016b; Anderson et al., 2018; Kim et al., 2018). Nevertheless, the
learned correlation between the multi-modality input and the answer does not
consider the semantic meaning of the answer.

In contrast, recent research in MMML has emerged in self-supervised learning
(SSL)(Chopra, Hadsell, and LeCun, 2005; Chen et al., 2020a). In particular,
contrastive learning is commonly used to learn a shared embedding space from
unlabelled data, in which similar sample pairs stay close to each other while
dissimilar ones from different pairs are far apart. For instance, Barlow
Twins(Zbontar et al., 2021) learns a cross-correlation matrix by keeping the
representations of different distorted versions of an input sample similar,
while minimizing the redundancy between these representations. Moreover,
Momentum Contrast (MoCo) (He et al., 2020) trains a visual representation
encoder by matching an encoded query to a dictionary of encoded keys based on
the Information Noise Contrastive Estimation (InfoNCE) loss (Oord et al.,
2018a). Furthermore, contrastive learning was also reported usage in training
multimodal models. For example, CLIP(Radford et al., 2021) computes a cosine
similarity matrix between all possible candidates of images and texts within a
batch. Then, the similarity between relevant pairs is maximized and the
similarity between irrelevant ones is minimized. Furthermore, there are also
studies such as MultiModal Versatile Networks (Alayrac et al., 2020) and
AVLnet(Rouditchenko et al., 2021) demonstrating the model training on large
collections of unlabelled video data with the contrastive loss.

The most recent work has also explored the implementation of SSL to VQA.
Question-Image Correlation Estimation (QICE) (Zhu et al., 2020) trains on
relevant and irrelevant image and question pairs in VQA datasets to alleviate
the language prior problem (Kurakin, Goodfellow, and Bengio, 2017; Agrawal et
al., 2018; Goyal et al., 2019) improving the understanding of image contents.
Kim et al. (Kim et al., 2021) studied Video Question Answering using SSL by
measuring the similarity scores between the representations of questions and
ground truth answers. These methods above adopt a two-step training strategy,
i.e., the cross-modality pretraining with SSL and the fusion network fine-
tuning. In contrast, we train a VQA model end-to-end based on the contrastive
learning of different model components and determine a prediction result using
the similarity measurement between modality representations. This work
leverages several projection networks to align the cross-modal representations
and embed the semantic meaning of answers.

### Decentralized Machine Learning

Our method is inspired by Decentralized Machine Learning built upon
distributed learning (Li et al., 2014; Smola and Narayanamurthy, 2010), such
as Federated Learning (McMahan et al., 2017; Sun, Ochiai, and Esaki, 2020;
Kairouz et al., 2021), Split Learning (Hancox and et al., 2020; Thapa et al.,
2022), and Swarm Learning (Warnat et al., 2021). Though methods like Federated
Learning have been intensively studied to tackle tasks of a single modality
(Sun, Ochiai, and Esaki, 2021), the studies in multimodal models such as VQA
are still insufficient. Notably, for the visual language grounding tasks, Liu
et al. (Liu et al., 2020) proposed a Federated Learning-based VQA framework
called the aimNet, which consists of an aligning module, an integrating
module, and a mapping module. AimNet could acquire fine-grained
representations from different clients for improved downstream tasks. However,
aimNet is a supervised learning framework that trains on the hard labels for
different answers. Moreover, though aimNet does not disclose the training data
via Federated Learning, the model parameters are shared to obtain better-
refined representations. Such shared model parameters can be used to mount
adversarial attacks of model poisoning (Wang et al., 2020a; Fang et al., 2020;
Zhou et al., 2021).

To overcome the above challenges, we devise a new split Learning-based VQA
framework leveraging a simpler and faster unidirectional learning flow to
improve classical split learning. Furthermore, neither the training data nor
the complete model’s parameters are disclosed during the training. To the best
of our knowledge, this is the first time any work has attempted to employ
split learning to tackle the VQA task and consolidate split learning and the
contrastive loss.

## Methods

In this section, we first introduce the motivation for introducing the
Unidirectional Split Learning with Contrastive Loss (UniCon) for visual
question answering (VQA) tasks. We then discuss the technical underpinnings of
UniCon comprising the unidirectional split learning framework, semantic notion
understanding with the answer projection network, and contrastive learning of
different model components using two adapter networks.

### Motivation

In VQA, the training data of images, questions, and answers are usually
prepared beforehand for the model training. Nevertheless, previous efforts in
the study of adversarial attacks on neural networks have shown that the use of
large-scale training data can be correlated with the model’s venerability to
poisoning attacks. The poisoning attacks can usually be divided into two
categories depending on the prior knowledge of the adversary (Sun, Ochiai, and
Esaki, 2021), i.e., data poisoning and model poisoning. Moreover, in the most
recent study of attacks on state-of-the-art VQA models, an adversary exploited
the complex fusion mechanisms to successfully embed effective and stealthy
backdoors (Walmer et al., 2022). Such threats are enabled due to the
adversary’s accessibility to either the training data or VQA models.

To this end, the line of work in decentralized machine learning (DML)
approaches such as Federated Learning (FL) have been recently used to
facilitate privacy-preserving learning of VQA models (Liu et al., 2020). FL
trains a global model over the entire data distribution of different tasks
thus attaining better-refined representations for downstream tasks. This is
enabled by the sharing of local model parameters trained on different client
tasks. Intuitively, FL can retain data confidentiality via model sharing,
however, sharing the entire model renders the FL-based framework venerable to
model poisoning attacks. For example, the adversary could reconstruct the
input data from the shared model parameters (Hitaj, Ateniese, and Pérez-Cruz,
2017). Our goal is to alleviate the threats of model poisoning while retaining
the guarantee of data confidentiality based on an adapted method of Split
Learning. The proposed method leverages contrastive learning to align
knowledge between different components of a VQA model such that the model can
be trained without disclosing the entire model architecture.

### Split Learning for Visual Question Answering

Visual Question Answering (VQA) is the task to answer questions according to
given image contents. The VQA problem is usually considered a supervised
learning task with a fixed list of $C$ possible answer options. In particular,
let $f_{\mbox{{VQA}}}$ be the VQA model that takes as the input the pair of an
image $x_{i}\in\mathbb{R}^{V}$ and a question $x_{q}\in\mathbb{R}^{Q}$ and
outputs an answer $\hat{y}\in\\{y_{1},y_{2},...,y_{C}\\}$ where
$y_{c}\in\mathbb{R}^{A}$. The goal of the VQA model is to predict the correct
answer $y$ given the input pair $(x_{i},x_{q})\in D$ where $D$ is the dataset.
$\hat{y}=\underset{y}{\mbox{arg max}}\,p(y|x_{i},x_{q};f_{\mbox{{VQA}}})$
where $p(\cdot|\cdot)$ is the conditional probability.

Figure 2: Split Learning vs. UniCon: Split Learning without label sharing
(left) employs the one-hot vectors of answer classes as labels to train the
model. The propagation direction is bidirectional increasing the client’s
waiting time for the computation of the global component. The Unidirectional
Split Learning with Contrastive Loss (right) considered in this paper,
however, learns the semantic notion of the answer text with contrastive
learning. Moreover, the direction of the propagation is unidirectional,
therefore, the client components can be computed simultaneously without
waiting for the computation of the global components.

Moreover, Split Learning (SL) splits a complete model into different parts and
trains a global model via interactive representation and gradient sharing.
There are mainly two types of SL for different tasks (Thapa et al., 2022). In
particular, we consider Split Learning without label sharing that wraps the
model on the server around the end layer and sends the layer output back to a
client (Fig. 2.a). This architecture could guarantee the data confidentiality
of a client since both the input data and labels are not shared for the
training. Furthermore, a complete model is split into three different
components for the training, i.e., a global component $f_{g}$ and two client
components $\\{f_{c,1},f_{c,2}\\}$. We assume there are $K$ clients. The $k$th
client has its own dataset
$D^{(k)}:=\\{(x_{i,j},x_{q,j},y_{j})\\}_{j=1}^{N^{(k)}}$ where $N^{(k)}$ is
the sample size of dataset $D^{(k)}$. Here, $\cup_{k=1}^{K}D^{(k)}=D$,
$D^{(i)}\cap D^{(j)}=\emptyset\,\,\forall i\neq j$, and
$\sum_{k=1}^{K}N^{(k)}=N$ where $N$ is the sample size of $D$. We suppose that
the model components of different clients share the same architecture in SL
and each client cannot share data mainly due to data confidentiality.

Then, SL proceeds by iterating the following steps: (1) each client $k$
computes the output of the component $f_{c,1}$ with $D^{(k)}$ and sends the
output to the server, (2) the server further forward-propagates the client
input with the global component $f_{g}$ and sends back the output, (3) the
probability distribution is computed with the client component $f_{c,2}$ and
the loss is computed with the ground-truths $\\{y_{j}\\}_{j=1}^{N^{(k)}}$, (4)
the gradients of each client $k$
$(\delta_{k}\theta_{c,1},\delta_{k}\theta_{c,2},\delta_{k}\theta_{g})$ with
respect to each component’s parameters
$(\theta_{c,1},\theta_{c,2},\theta_{g})$ are computed via an inverse path
$f_{c,2}\rightarrow f_{g}\rightarrow f_{c,1}$, (5) the gradients are averaging
aggregated to update the different components,
$\delta\theta_{c,1}=\frac{1}{K}\underset{k\in K}{\sum}\delta_{k}\theta_{c,1}$,
$\delta\theta_{c,2}=\frac{1}{K}\underset{k\in K}{\sum}\delta_{k}\theta_{c,2}$,
$\delta\theta_{g}=\frac{1}{K}\underset{k\in K}{\sum}\delta_{k}\theta_{g}$. The
process above is repeated until a given training goal is achieved. We refer to
the appendix for full details on the SL algorithm.

### Unidirectional Split Learning with Contrastive Loss

Previous work showed the plausible usages and efficacy of Contrastive Learning
(Chopra, Hadsell, and LeCun, 2005) in a single modality (Chen et al., 2020a;
He et al., 2020; Zbontar et al., 2021). In multimodal learning, Contrastive
Learning has recently been used to learn refined cross-modal representations
(Radford et al., 2021; Rouditchenko et al., 2021) by encouraging multimodal
data from a relevant input to have more similar representations compared to
data from irrelevant inputs. For the VQA tasks, one challenge is that most
models are supervised where the one-hot vectors of answer classes are employed
to perform a multi-class classification (Anderson et al., 2018; Kim et al.,
2018). Therefore, the semantic notions of answers are usually not well
correlated with the inputs reducing the generality of the trained model to
unknown samples. To this end, we propose the use of the contrastive loss in
Split Learning to correlate vision contents and language semantic notions such
that each model component learns better-refined representations for the VQA
tasks. Moreover, since the learning flow in classical SL is bidirectional
between a client and the server, the waiting time for computing the next
component is greatly increased (Fig. 2.a). By using the contrastive loss, we
adapt SL to the unidirectional sharing architecture where the representations
are shared by clients to the server and the gradients are shared by the server
to clients. The unidirectional split learning with contrastive loss (UniCon)
allows a client to compute the representations and gradients without waiting
for the computation of the global component, which is more efficient and
simpler compared to SL (Fig. 2.b).

#### Semantic Notion Understanding with Answer Projection Network.

The vast majority of VQA models are supervised where the one-hot vectors of
answer classes are employed to perform a multi-class classification (Yang et
al., 2016b; Anderson et al., 2018; Kim et al., 2018). To learn semantic
notions as well from the answers, we devise an Answer Projection Network (APN)
$f_{\mbox{{APN}}}$ to embed the answer language contexts $y$ into a feature
vector $v_{\mbox{{APN}}}\in\mathbb{R}^{P}$. Notably, APN comprises three
different building blocks including a text preprocessing module, the Word2Vec
using GloVe (Pennington et al., 2014), and a linear projection layer. The
detailed architecture is described in Section Experiments.

#### Adapter Networks and the Shared Projection Space.

We propose the use of two adapter networks to project the outputs from
different model components into a shared projection space. In this regard, a
nonlinear projection head on more complex representations can improve the
performance while for simpler modality representations it is not beneficial to
use the nonlinear projection (Chen et al., 2020b, a; Alayrac et al., 2020). In
particular, we replace a VQA model $f_{\mbox{{VQA}}}$’s output layer with the
Nonlinear Head Adapter (NHA) network $f_{\mbox{{NHA}}}$ that projects the
high-level cross-modal representations from the layer before the output layer
into the shared projection space $v_{\mbox{{NHA}}}\in\mathbb{R}^{S}$.
Furthermore, we devise the Linear Tail Adapter (LTA) to project the low-level
representations $v_{\mbox{{APN}}}$ of APN into the shared projection space
$v_{\mbox{{LTA}}}\in\mathbb{R}^{S}$. Note that $v_{\mbox{{LTA}}}$ and
$v_{\mbox{{NHA}}}$ have the same dimension of $S$. The detailed architecture
of NHA and LTA are described in Section Experiments.

#### Learning with the Information Noise Contrastive Estimation loss.

The Information Noise Contrastive Estimation (InfoNCE) loss is commonly used
for contrastive learning (Oord et al., 2018b) to identify the positive sample
from a set of unrelated negative samples. Notably, UniCon employs the relevant
NHA and LTA outputs in the shared projection space of the same input triplets
within one training batch as positive pairs.
$\\{(v_{\mbox{{NHA}},j},v_{\mbox{{LTA}},j})\\}_{j=1}^{B}$ where $B$ is the
sample size of the training batch. In contrast, given a NHA output
$v_{\mbox{{NHA}},i}$, any irrelevant LTA outputs $\\{v_{\mbox{{LTA}},j}|j\neq
i\\}_{j=1}^{B}$ are employed as the negative keys of the NHA output. Then, we
train the model by aligning the knowledge between the component outputs in
positive pairs while discouraging the similarity between the outputs in
negative pairs (Fig. 1). We formulate the loss $\mathcal{L}$ as follows

$\mathcal{L}=-\sum_{i=1}^{B}\log\frac{\exp(v_{\mbox{{NHA}},i}\cdot
v_{\mbox{{LTA}},i}/\tau)}{\sum_{j=1}^{B}\mathbbm{1}_{[j\neq
i]}\exp(v_{\mbox{{NHA}},i}\cdot v_{\mbox{{LTA}},j}/\tau)},$ (1)

where $\tau$ is the temperature parameter and $\mathbbm{1}_{[j\neq i]}$ is an
indicator function: 1 if $j\neq i$, 0 otherwise.

#### Model Parameter Aggregation.

Parameter aggregation of models trained on different VQA tasks aims to improve
the generality of the models to unseen samples. This is enabled by aggregating
the update gradients of different components after each epoch’s training.
Since the sharing of all components’ gradients to the server for aggregation
might disclose the complete model’s architecture, we employ a dual-server
aggregation strategy where an auxiliary parameter server is adopted to
aggregate the gradients of client components ($f_{\mbox{{APN}}}$ and
$f_{\mbox{{VQA}}}$) and the main server is used to aggregate the gradients of
global components ($f_{\mbox{{NHA}}}$ and $f_{\mbox{{LTA}}}$), respectively.
The dual-server aggregation can limit an adversary’s prior knowledge of the
training model thus alleviating model poisoning attacks such as membership
inference (Nasr et al., 2019) and information stealing (Wang et al., 2020b).
However, since the main focus of this work is not adversarial attacks and
defense of VQA models, we leave the discussion on the model’s robustness
against poisoning attacks to future study. We formulate the parameter
aggregation of APN, VQA, NHA, and LTA by the following

$\delta\theta_{t}=\frac{1}{K}\underset{k\in\\{1,2,\dots,K\\}}{\sum}(\theta_{t+1}^{(k)}-\theta_{t}^{(k)}),$
(2)

where $\theta$ is the parameters of a model component from
$\\{\theta_{\mbox{{APN}}},\theta_{\mbox{{VQA}}},\theta_{\mbox{{NHA}}},\theta_{\mbox{{LTA}}}\\}$.

Furthermore, each client $k$ updates the local components and the main server
updates the global components based on the aggregated gradients, respectively.
We refer to the appendix for full details on the UniCon algorithm.

### Accuracy with Representation Similarity Measurement

The difficulty to measure the accuracy in UniCon is that we do not have a
discriminative model to infer the class of the input. Inspired by (Radford et
al., 2021), we evaluate the product similarity scores between the
representations $v_{\mbox{{NHA}}}$ of an image and question input pair
$(x_{i},x_{q})$ from the hold-out validation dataset $D_{\mbox{{val}}}$ of VQA
and the representations $v_{\mbox{{LTA}},c}$ of $C$ answer options
$y_{c}\in\\{y_{1},y_{2},\dots,y_{C}\\}$, where $v_{\mbox{{LTA}},c}$ denotes
the representation of answer option $y_{c}$. Then, we select the answer with
the highest similarity with the input as the predicted answer$\hat{y}$. There
also exist studies reporting the average accuracy of different batches, i.e.,
measuring the similarity scores within the same batch. However, we found that
such a metric can easily produce a much higher accuracy compared to the
evaluation metric based on all answer options. Therefore, we use as the metric
the similarity scores between the representations of input and all answer
options. We formulate the accuracy by the following

$\hat{y}=\underset{c}{\mbox{arg max}}\,v_{\mbox{{NHA}}}\cdot
v_{\mbox{{LTA}},c}$ (3) $\mbox{ValAcc}=\frac{\sum_{(x_{i},x_{q},y)\in
D_{\mbox{{val}}}}\mathbbm{1}\\{\hat{y}=y\\}}{|D_{\mbox{{val}}}|}.$

## Experiments

### Datasets and Models

##### Datasets.

Our method is evaluated on the benchmark dataset VQA-v2 (Agrawal et al., 2017)
where images are from the COCO dataset (Lin et al., 2014) and we report the
results on its validation split using Eq. 3.

##### Models.

We studied our approach with the following state-of-the-art Visual Question
Answering models: (1) Multi-modal Factorized Bilinear (MFB) (Yu et al., 2017)
combines multi-modal features using an end-to-end network architecture to
jointly learn the image and question attention; (2) Bottom-Up and Top-Down
attention mechanism (BUTD) (Anderson et al., 2018) enables attention to be
calculated at the level of objects and other salient image regions. The
bottom-up mechanism based on Faster R-CNN proposes image regions, while the
top-down mechanism determines feature weightings; (3) Bilinear Attention
Networks (BAN) (Kim et al., 2018) considers bilinear interactions among two
groups of input channels and extracts the joint representations for each pair
of channels; (4) Multimodal neural architecture search (MMNas) (Yu et al.,
2020) uses a gradient-based algorithm to learn the optimal architecture; (5)
Modular Co-Attention Network (MCAN) (Yu et al., 2019b) consists of Modular Co-
Attention (MCA) layers cascaded in depth where each MCA layer models both the
self-attention and the guided-attention of the input channels.

### Implementation Details

The proposed method is model agnostic and can be applied to different VQA
models. We used PyTorch and the OpenVQA platform (Yu et al., 2019a) to
implement the VQA models. We set the hyperparameters of different VQA models
to their default author-recommended values. Due to the training time cost, we
evaluated the model performance with three different seeds and reported the
mean. We conducted experiments on four NVIDIA A100 Tensor Core GPU with 40GB
memory each. The code will be made publicly available upon acceptance.

### Component Architecture and Hyperparameters

We employed the following architecture for the three model components in
UniCon, respectively. For APN, we used the GloVe (Pennington et al., 2014)
trained on Common Crawl to convert the pre-processed texts with a maximum word
of eight into the dimension of $8\times 300$. Then, a fully-connected layer
followed by the ReLU activation function was applied to project the
representations into $8\times 512$. Finally, a Max Pooling layer was applied
to the eight words with respect to each dimension producing a 512-dimension
vector. Moreover, for LTA, we used a fully connected layer to project the
representations received from each client to the shared space with a dimension
of 256. For NHA, we used a two-layer architecture: (1) a fully connected layer
to project the representations to a dimension of 512 followed by the ReLU, (2)
and another fully connected layer to project the middle layer output to a
dimension of 256 followed by the batch normalization. We refer to the appendix
for full details on the model architecture.

Furthermore, we employed a batch size of 128, a total epoch of 20 (693400
steps), the Adam optimizer with parameters $\beta_{1}=0.9$, $\beta_{2}=0.999$,
and $\epsilon=10^{-8}$, and an initial learning rate of 0.0001 with a linear
warmup of 10K steps and a decay rate of 0.2 at the epoch 10 and 15. The
hyperparameters were chosen using the grid search. We found that a smaller
learning rate or a larger batch size would generally hinder the model learning
and the model ceased learning in some cases with larger parameter sizes. In
addition, for the InfoNCE loss, we adopted a temperature of 0.07 as in (Wu et
al., 2018; Patrick et al., 2020; Alayrac et al., 2020).

### Empirical Results

Figure 3: Qualitative analysis of model predictions. We show different success and failure cases for different question types. VQA models trained with the contrastive loss tend to successfully answer the questions about the general contents of an image while poorly answering the counting questions, e.g., counting the pictures on the wall, and the questions about the detail of an image, e.g., what is the man wearing on his head. VQA Models |  Contrastive learning-based VQA (%)
---|---
Overall | Yes/No | Number | Other
BAN | 36.23 | 66.90 | 12.71 | 19.11
BUTD | 45.08 | 75.82 | 29.27 | 25.86
MFB | 46.98 | 73.95 | 32.81 | 30.20
MCAN-s | 53.18 | 81.06 | 41.95 | 34.93
MCAN-l | 53.32 | 81.21 | 42.66 | 34.90
MMNas-s | 51.54 | 78.06 | 39.76 | 34.46
MMNas-l | 53.82 | 80.06 | 42.86 | 36.75
Table 2: Effectiveness of the contrastive learning-based VQA for different
models. The highest reported accuracy under each task is in bold.

We performed extensive experiments based on the five state-of-the-art VQA
models above. In particular, for MMNas and MCAN, we further considered the
effectiveness of different model complexities using MMNas-small (MMNas-s) and
MMNas-large (MMNas-l), and MCAN-small (MCAN-s) and MCAN-large (MCAN-l). The
detailed architecture designs of these models followed the settings in (Yu et
al., 2020, 2019b). Moreover, we evaluated the proposed method’s performance
based on Eq. 3. Notably, for each triplet in the validation set, we input the
image and question pair to UniCon. Then, the output representation of the
nonlinear head adapter is used to measure the similarity scores with the
representations of the 3048 answers from the linear tail adapter. Finally, the
prediction result is the answer text with the highest similarity score and the
accuracy is computed based on the predictions and the ground truths of the
validation set. Note that the label space of VQA-v2 is 3048-dimension which is
much larger than the datasets considered in CLIP (Radford et al., 2021),
hence, the VQA tasks are more challenging to perform the contrastive learning.
The largest label space in CLIP is ImageNet (Deng et al., 2009) with only 1000
classes.

Furthermore, we studied the effectiveness of the contrastive learning-based
VQA. The results are shown in Table 2. The empirical results demonstrate that
the contrastive learning-based approach can be effectively applied to most VQA
models. BAN showed the worst performance, particularly in counting the number.
MMNas-l showed the best overall performance of 53.82% outperforming the other
models for counting the number (Number) and answering the contents of an image
(Other). Nevertheless, MCAN-l performed the best in the Yes/No questions.

VQA Models |  UniCon (%)
---|---
Overall | Yes/No | Number | Other
BAN | 35.11 | 63.84 | 11.06 | 19.61
BUTD | 40.96 | 66.98 | 13.34 | 28.74
MFB | 42.43 | 68.65 | 23.33 | 27.52
MCAN-s | 48.42 | 74.93 | 30.88 | 32.89
MCAN-l | 48.44 | 77.44 | 30.72 | 32.01
MMNas-s | 45.14 | 70.55 | 28.04 | 30.33
MMNas-l | 49.89 | 74.85 | 36.88 | 34.33
Table 3: Effectiveness of UniCon for different models where two clients train
a global model without disclosing their data or models. The highest reported
accuracy under each task is in bold. Figure 4: Similarity scores between
component outputs before and after training in UniCon. The similarity scores
are optimized such that the positive pairs have a higher similarity while the
negative pairs have a lower similarity.

To evaluate the efficacy of UniCon, we randomly divided the training set into
two subsets. Then, we employed the two subsets as the local datasets of two
clients to perform the split learning. The two clients share the same model
component architecture and cannot share data mainly due to data
confidentiality. The performance was evaluated on the global model after each
round’s parameter aggregation. We show the numerical results in Table 3.
Similarly, MMNas-l outperformed the other models, however, MCAN-l performed
the best in the Yes/No questions. Moreover, there exists a trade-off between
the model performance and using split learning for confidentiality. Compared
to the overall accuracy of 53.82% of MMNas-l in the standalone training over
the entire data set, the split learning-based approach obtained an accuracy of
49.89%. Nevertheless, when data sharing becomes an obstacle, the proposed
approach can benefit the model training by leveraging other clients’ knowledge
of different tasks. In addition, in case that data sharing is not allowed, a
standalone model usually only has access to part of the data training over the
partial distribution. UniCon allows clients to train over the entire data
distribution thus improving the global model performance. We aim to study the
efficacy of UniCon to improve the performance of standalone learning over the
partial distribution in our future endeavors. We also presented the similarity
measurement scores between the pairs of $v_{\mbox{{NHA}}}$ and
$v_{\mbox{{LTA}}}$ within one training batch before and after training,
respectively, in Fig. 4.

## Conclusion

We propose a Unidirectional Split Learning with Contrastive loss (UniCon) for
VQA, which learns fine-grained cross-modal representations by aligning model
component outputs of different client tasks. We evaluate the efficacy of the
UniCon framework on five different VQA models based on the VQA-v2 dataset.
Extensive experiments show that our approach can be applied to different
models demonstrating the effectiveness and universality of our approach. This
work can be extended by considering a broader list of answer options using
prompt engineering (Gao et al., 2021; Radford et al., 2021) further improving
the prediction performance. Moreover, the robustness of UniCon against
adversarial attacks will be studied with additional efforts using methods like
Differential Privacy (Abadi et al., 2016). We hope that this work will
motivate future research in robust learning for personal multimodal models.

## References

  * Abadi et al. (2016) Abadi, M.; Chu, A.; Goodfellow, I. J.; and et al. 2016.  Deep Learning with Differential Privacy.  In _ACM Conference on Computer and Communications Security_.
  * Agrawal et al. (2018) Agrawal, A.; Batra, D.; Parikh, D.; and Kembhavi, A. 2018.  Don’t Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering.  In _CVPR_.
  * Agrawal et al. (2017) Agrawal, A.; Lu, J.; Antol, S.; and et al. 2017.  VQA: Visual Question Answering - www.visualqa.org.  _Int. J. Comput. Vis._ , 123(1): 4–31.
  * Alayrac et al. (2020) Alayrac, J.; Recasens, A.; Schneider, R.; and et al. 2020.  Self-Supervised MultiModal Versatile Networks.  In _NeurIPS_.
  * Anderson et al. (2018) Anderson, P.; He, X.; Buehler, C.; and et al. 2018.  Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering.  In _CVPR_.
  * Bagdasaryan and Shmatikov (2020) Bagdasaryan, E.; and Shmatikov, V. 2020.  Blind Backdoors in Deep Learning Models.  _arXiv preprint_.
  * Bagdasaryan et al. (2020) Bagdasaryan, E.; Veit, A.; Hua, Y.; and et al. 2020.  How To Backdoor Federated Learning.  In _AISTATS_.
  * Bahdanau, Cho, and Bengio (2015) Bahdanau, D.; Cho, K.; and Bengio, Y. 2015.  Neural Machine Translation by Jointly Learning to Align and Translate.  In _ICLR_.
  * Chen et al. (2020a) Chen, T.; Kornblith, S.; Norouzi, M.; and Hinton, G. E. 2020a.  A Simple Framework for Contrastive Learning of Visual Representations.  In _ICML_.
  * Chen et al. (2020b) Chen, T.; Kornblith, S.; Norouzi, M.; and Hinton, G. E. 2020b.  A Simple Framework for Contrastive Learning of Visual Representations.  In _ICML_.
  * Chopra, Hadsell, and LeCun (2005) Chopra, S.; Hadsell, R.; and LeCun, Y. 2005.  Learning a Similarity Metric Discriminatively, with Application to Face Verification.  In _CVPR_.
  * Deng et al. (2009) Deng, J.; Dong, W.; Socher, R.; and et al. 2009.  Imagenet: A large-scale hierarchical image database.  In _CVPR_.
  * Fang et al. (2020) Fang, M.; Cao, X.; Jia, J.; and Gong, N. Z. 2020.  Local Model Poisoning Attacks to Byzantine-Robust Federated Learning.  In _USENIX Security Symposium_.
  * Gao et al. (2021) Gao, T.; Fisch, A.; Chen, D.; and et al. 2021.  Making Pre-trained Language Models Better Few-shot Learners.  In _ACL/IJCNLP_.
  * Goyal et al. (2019) Goyal, Y.; Khot, T.; Agrawal, A.; and et al. 2019.  Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering.  _Int. J. Comput. Vis._ , 127(4): 398–414.
  * Hancox and et al. (2020) Hancox, R.; and et al. 2020.  The future of digital health with federated learning.
  * He et al. (2020) He, K.; Fan, H.; Wu, Y.; and et al. 2020.  Momentum Contrast for Unsupervised Visual Representation Learning.  In _CVPR_.
  * Hitaj, Ateniese, and Pérez-Cruz (2017) Hitaj, B.; Ateniese, G.; and Pérez-Cruz, F. 2017.  Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning.  In _ACM Conference on Computer and Communications Security_.
  * Kairouz et al. (2021) Kairouz, P.; McMahan, H. B.; Avent, B.; and et al. 2021.  Advances and Open Problems in Federated Learning.  _Found. Trends Mach. Learn._ , 14(1-2): 1–210.
  * Kim et al. (2018) Kim, J.; Jun, J.; Zhang, B.; and et al. 2018.  Bilinear Attention Networks.  In _NeurIPS_.
  * Kim et al. (2017) Kim, K.; Heo, M.; Choi, S.; and Zhang, B. 2017.  DeepStory: Video Story QA by Deep Embedded Memory Networks.  In _IJCAI_.
  * Kim et al. (2021) Kim, S.; Jeong, S.; Kim, E.; Kang, I.; and Kwak, N. 2021.  Self-supervised Pre-training and Contrastive Representation Learning for Multiple-choice Video QA.  In _AAAI_.
  * Kim (2015) Kim, T. K. 2015.  T test as a parametric statistic.  _Korean journal of anesthesiology_ , 68(6).
  * Kurakin, Goodfellow, and Bengio (2017) Kurakin, A.; Goodfellow, I. J.; and Bengio, S. 2017.  Adversarial Machine Learning at Scale.  In _ICLR_.
  * Li et al. (2014) Li, M.; Andersen, D. G.; Park, J. W.; and et al. 2014.  Scaling Distributed Machine Learning with the Parameter Server.  In _USENIX_.
  * Lin et al. (2020) Lin, J.; Xu, L.; Liu, Y.; and Zhang, X. 2020.  Composite Backdoor Attack for Deep Neural Network by Mixing Existing Benign Features.  In _ACM SIGSAC_ , 113–131. ACM.
  * Lin et al. (2014) Lin, T.; Maire, M.; Belongie, S. J.; and et al. 2014.  Microsoft COCO: Common Objects in Context.  In _ECCV_.
  * Liu et al. (2020) Liu, F.; Wu, X.; Ge, S.; and et al. 2020.  Federated Learning for Vision-and-Language Grounding Problems.  In _AAAI_.
  * McMahan et al. (2017) McMahan, B.; Moore, E.; Ramage, D.; and et al. 2017.  Communication-Efficient Learning of Deep Networks from Decentralized Data.  In _AISTATS_.
  * Nasr et al. (2019) Nasr, M.; Shokri, R.; Houmansadr, A.; and et al. 2019.  Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning.  In _IEEE Symposium on Security and Privacy (SP)_.
  * Oord et al. (2018a) Oord, A.; Li, Y.; Vinyals, O.; and et al. 2018a.  Representation Learning with Contrastive Predictive Coding.  _arXiv Preprint_.
  * Oord et al. (2018b) Oord, A.; Li, Y.; Vinyals, O.; and et al. 2018b.  Representation Learning with Contrastive Predictive Coding.  _arXiv preprint_.
  * Patrick et al. (2020) Patrick, M.; Asano, Y. M.; Fong, R.; and et al. 2020.  Multi-modal Self-Supervision from Generalized Data Transformations.  _arXiv preprint_.
  * Pennington et al. (2014) Pennington, J.; Socher, R.; Manning, C. D.; and et al. 2014.  GloVe: Global Vectors for Word Representation.  In _EMNLP_.
  * Radford et al. (2021) Radford, A.; Kim, J. W.; Hallacy, C.; and et al. 2021.  Learning Transferable Visual Models From Natural Language Supervision.  In _ICML_.
  * Ramesh et al. (2022) Ramesh, A.; Dhariwal, P.; Nichol, A.; and et al. 2022.  Hierarchical Text-Conditional Image Generation with CLIP Latents.  _arXiv preprint arXiv.2204.06125_.
  * Ramesh et al. (2021) Ramesh, A.; Pavlov, M.; Goh, G.; and et al. 2021.  Zero-Shot Text-to-Image Generation.  In _ICML_.
  * Rouditchenko et al. (2021) Rouditchenko, A.; Boggust, A. W.; Harwath, D.; and et al. 2021.  AVLnet: Learning Audio-Visual Language Representations from Instructional Videos.  In _Annual Conference of the International Speech Communication Association_.
  * Smola and Narayanamurthy (2010) Smola, A. J.; and Narayanamurthy, S. M. 2010.  An Architecture for Parallel Topic Models.  _Proc. VLDB Endow._ , 3(1): 703–710.
  * Sun, Ochiai, and Esaki (2020) Sun, Y.; Ochiai, H.; and Esaki, H. 2020.  Intrusion Detection with Segmented Federated Learning for Large-Scale Multiple LANs.
  * Sun, Ochiai, and Esaki (2021) Sun, Y.; Ochiai, H.; and Esaki, H. 2021.  Decentralized Deep Learning for Multi-Access Edge Computing: A Survey on Communication Efficiency and Trustworthiness.  In _IEEE Transactions on Artificial Intelligence_.
  * Thapa et al. (2022) Thapa, C.; Chamikara, M. A. P.; Camtepe, S.; and Sun, L. 2022.  SplitFed: When Federated Learning Meets Split Learning.
  * Vaswani et al. (2017a) Vaswani, A.; Shazeer, N.; Parmar, N.; and et al. 2017a.  Attention is All you Need.  In _NeurIPS_.
  * Vaswani et al. (2017b) Vaswani, A.; Shazeer, N.; Parmar, N.; and et al. 2017b.  Attention is All you Need.  In _NeurIPS_.
  * Walmer et al. (2022) Walmer, M.; Sikka, K.; Sur, I.; and et al. 2022.  Dual-Key Multimodal Backdoors for Visual Question Answering.  _CVPR_.
  * Wang et al. (2020a) Wang, H.; Sreenivasan, K.; Rajput, S.; and et al. 2020a.  Attack of the Tails: Yes, You Really Can Backdoor Federated Learning.  In _NeurIPS_.
  * Wang et al. (2020b) Wang, H.; Sreenivasan, K.; Rajput, S.; and et al. 2020b.  Attack of the Tails: Yes, You Really Can Backdoor Federated Learning.  In _NeurIPS_.
  * Warnat et al. (2021) Warnat, S.; Schultze, H.; Shastry, L.; and et al. 2021.  Swarm Learning for decentralized and confidential clinical machine learning.
  * Wu et al. (2018) Wu, Z.; Xiong, Y.; Yu, S. X.; and Lin, D. 2018.  Unsupervised Feature Learning via Non-Parametric Instance Discrimination.  In _CVPR_.
  * Yang et al. (2016a) Yang, Z.; He, X.; Gao, J.; and et al. 2016a.  Stacked Attention Networks for Image Question Answering.  In _CVPR_.
  * Yang et al. (2016b) Yang, Z.; He, X.; Gao, J.; and et al. 2016b.  Stacked Attention Networks for Image Question Answering.  In _CVPR_.
  * Yu et al. (2019a) Yu, Z.; Cui, Y.; Shao, Z.; Gao, P.; and Yu, J. 2019a.  OpenVQA.  https://github.com/MILVLG/openvqa.
  * Yu et al. (2020) Yu, Z.; Cui, Y.; Yu, J.; and et al. 2020.  Deep Multimodal Neural Architecture Search.  In _ACM Multimedia_.
  * Yu et al. (2019b) Yu, Z.; Yu, J.; Cui, Y.; and et al. 2019b.  Deep Modular Co-Attention Networks for Visual Question Answering.  In _CVPR_.
  * Yu et al. (2017) Yu, Z.; Yu, J.; Fan, J.; and Tao, D. 2017.  Multi-modal Factorized Bilinear Pooling with Co-attention Learning for Visual Question Answering.  In _ICCV_.
  * Zbontar et al. (2021) Zbontar, J.; Jing, L.; Misra, I.; and et al. 2021.  Barlow Twins: Self-Supervised Learning via Redundancy Reduction.
  * Zhou et al. (2021) Zhou, X.; Xu, M.; Wu, Y.; and Zheng, N. 2021.  Deep Model Poisoning Attack on Federated Learning.  _Future Internet_ , 13(3): 73.
  * Zhu et al. (2020) Zhu, X.; Mao, Z.; Liu, C.; Zhang, P.; Wang, B.; and Zhang, Y. 2020.  Overcoming Language Priors with Self-supervised Learning for Visual Question Answering.  In _IJCAI_.

##  Appendix A Appendices

### VQA-v2 Dataset

Our method was evaluated on the benchmark VQA-v2 dataset (Agrawal et al.,
2017) (Fig. 5) that includes different question types and images from the COCO
dataset (Lin et al., 2014).

Figure 5: VQA-v2 dataset including the question types of ”yes/no”, ”number”,
and ”other”.

### Statistical T-test

We performed a paired t-test with two-tails (Kim, 2015) to compare the results
between the results in Table 2 and Table 3. The $n=7$ VQA model variants that
are based on contrastive learning without using split learning compared to the
seven VQA model variants that are based on UniCon demonstrated a $t$-value of
1.357. If we set a $p$-value of 0.05 and a degree of freedom $n-1=6$, then the
$p$-value is 2.477. Since $t=1.357$ is in the range [-2.477, 2.477], there is
no significant effect on the prediction accuracy for using split learning.
Therefore, UniCon achieves a competitive prediction result while safeguarding
a client’s training data and model.

### Attention Map Visualization

The attention mechanism (Bahdanau, Cho, and Bengio, 2015; Vaswani et al.,
2017b) consists of three main components, namely the query $Q$, the key $K$,
and the value $V$. Notably, in Visual Question Answering (VQA), attention
weights are learned to represent the relative importance of visual
representations at different spatial locations with respect to a given
question. The attention layers are updated such that weights are put on the
visual regions that are more relevant to the question. Following (Yang et al.,
2016b), we visualized the attention map by extracting the weight matrix from
the learned attention mechanism (Fig. 6).

Figure 6: The highlighted image regions are of importance to answering the
given question. The attention maps are generated with the weight matrix of the
attention mechanism in the cross-modal representation learning. The questions
are ”What color is the sail” for the samples above and ”How many dogs is the
man walking” for the samples below.

### Model Components Architecture

We show the detailed architecture for the nonlinear head adapter (NHA), the
linear tail adapter (LTA), and the answer projection network (APN) in Fig. 7.

Figure 7: The detailed architecture of NHA, LTA, and APN.

### More Results of the Qualitative Analysis

The qualitative analysis evaluates the different success and failure cases for
the different question types. Here we show more examples of the prediction
results of UniCon in Fig. 8. The prediction results are based on the
similarity measurement approach with a total of 3048 different answers.

Figure 8: More prediction results of UniCon.

### Algorithms

Here we demonstrate the algorithms for Split Learning (without label sharing)
and Unidirectional Split Learning with Contrastive Loss (UniCon) in Algorithm
1 and 2, respectively.

Algorithm 1 Split Learning (without label sharing)

1: $L_{\mbox{{head}},t}^{(k)}$: parameters of $f_{c,1}^{(k)}$

2: $L_{\mbox{{tail}},t}^{(k)}$: parameters of $f_{c,2}^{(k)}$

3: $G_{t}$: parameters of $f_{g}$

4: $\delta_{g,e}$: partial derivatives of $f_{g}$ at local epoch $e$

5: $\delta_{e}^{(k)}$: partial derivatives of $f_{c,2}^{(k)}$ at local epoch
$e$

6: $T$: total training rounds

7: $E$: total local epochs

8: $J$: negative log likelihood loss

9: $\eta$: learning rate

10: $D^{(k)}=(X^{(k)},Y^{(k)})$

11: for each round $t=1,2,\dots,T$ do

12: for each client $k\in\\{1,2,\dots,K\\}$ in parallel do

13: $G^{(k)}_{t}\leftarrow G_{t}$

14: $L_{\mbox{{head}},t}^{(k)}\leftarrow L_{\mbox{{head}},t}$

15: $L_{\mbox{{tail}},t}^{(k)}\leftarrow L_{\mbox{{tail}},t}$

16: for each local epoch $e=1,2,\dots,E$ do

17: $v_{e}^{(k)}=f_{c,1}^{(k)}(L_{\mbox{{head}},t,e}^{(k)},D^{(k)})$

18: $v_{e}^{g}$ = ServerForward$(v_{e}^{(k)},G^{(k)}_{t,e})$

19: $\delta_{e}^{(k)}=\frac{\partial
J(f_{c,2}^{(k)}(L_{\mbox{{tail}},t,e}^{(k)},v_{e}^{g}),Y^{(k)})}{\partial
L_{\mbox{{tail}},t,e}^{(k)}}$

20: $L_{\mbox{{tail}},t,e+1}^{(k)}\leftarrow
L_{\mbox{{tail}},t,e}^{(k)}-\eta\cdot\delta_{e}^{(k)}$

21: $\delta_{g,e}^{(k)},G_{t,e+1}^{(k)}$=
ServerBackprop$(\delta_{e}^{(k)},G^{(k)}_{t,e})$

22: $L_{\mbox{{head}},t,e+1}\leftarrow
L_{\mbox{{head}},t,e}^{(k)}-\eta\cdot\frac{\partial\delta_{g,e}^{(k)}}{\partial
L_{\mbox{{head}},t,e}^{(k)}}$

23: end for

24: end for

25: $G_{t+1}=\frac{1}{K}\sum_{k\in K}G_{t,E+1}^{(k)}$

26: $L_{\mbox{{head}},t+1}=\frac{1}{K}\sum_{k\in
K}L_{\mbox{{head}},t,E+1}^{(k)}$

27: $L_{\mbox{{tail}},t+1}=\frac{1}{K}\sum_{k\in
K}L_{\mbox{{tail}},t,E+1}^{(k)}$

28: end for

29:

30: function ServerForward$(v_{e}^{(k)},G^{(k)}_{t,e})$

31: $v_{e}^{g}\leftarrow f_{g}(G^{(k)}_{t,e},v_{e}^{(k)})$

32: return $v_{e}^{g}$ to client

33:

34: function ServerBackprop$(\delta_{e}^{(k)},G^{(k)}_{t,e})$

35: $\delta_{g,e}^{(k)}=\frac{\partial\delta_{e}^{(k)}}{\partial
G^{(k)}_{t,e}}$

36: $G_{t,e+1}^{(k)}\leftarrow G^{(k)}_{t,e}-\eta\cdot\delta_{g,e}^{(k)}$

37: return $\delta_{g,e}^{(k)}$ to client $k$

Algorithm 2 Unidirectional Split Learning with Contrastive Loss (UniCon)

1: $L_{\mbox{{VQA}},t}^{(k)}$: parameters of $f_{\mbox{{VQA}}}^{(k)}$

2: $L_{\mbox{{APN}},t}^{(k)}$: parameters of $f_{\mbox{{APN}}}^{(k)}$

3: $L_{\mbox{{NHA}},t}$: parameters of $f_{\mbox{{NHA}}}$

4: $L_{\mbox{{LTA}},t}$: parameters of $f_{\mbox{{LTA}}}$

5: $D^{(k)}=(X_{i}^{(k)},X_{q}^{(k)},Y^{(k)})$

6: $T$: total training rounds

7: $E$: total local epochs

8: $\eta$: learning rate

9: $J_{cl}$: Information Noise Contrastive Estimation (InfoNCE) loss

10: for each round $t=1,2,\dots,T$ do

11: for each client $k\in\\{1,2,\dots,K\\}$ in parallel do

12: $L_{\mbox{{NHA}},t}^{(k)}\leftarrow L_{\mbox{{NHA}},t}$

13: $L_{\mbox{{LTA}},t}^{(k)}\leftarrow L_{\mbox{{LTA}},t}$

14: $L_{\mbox{{VQA}},t}^{(k)}\leftarrow L_{\mbox{{VQA}},t}$

15: $L_{\mbox{{APN}},t}^{(k)}\leftarrow L_{\mbox{{APN}},t}$

16: for each local epoch $e=1,2,\dots,E$ do

17: $v_{\mbox{{VQA}},t,e}^{(k)}$ =
$f_{\mbox{{VQA}}}(L_{\mbox{{VQA}},t,e}^{(k)},(X_{i}^{(k)},X_{q}^{(k)}))$

18: $v_{\mbox{{APN}},t,e}^{(k)}$ =
$f_{\mbox{{APN}}}(L_{\mbox{{APN}},t,e}^{(k)},Y^{(k)})$

19: $\delta_{\mbox{{NHA}},e}^{(k)},\delta_{\mbox{{LTA}},e}^{(k)}$ =
Server($v_{\mbox{{VQA}},t,e}^{(k)}$,$v_{\mbox{{APN}},t,e}^{(k)}$)

20: $L_{\mbox{{VQA}},t,e+1}^{(k)}\leftarrow
L_{\mbox{{VQA}},t,e}^{(k)}-\eta\cdot\frac{\partial\delta_{\mbox{{NHA}},e}^{(k)}}{\partial
L_{\mbox{{VQA}},t,e}^{(k)}}$

21: $L_{\mbox{{APN}},t,e+1}^{(k)}\leftarrow
L_{\mbox{{APN}},t,e}^{(k)}-\eta\cdot\frac{\partial\delta_{\mbox{{LTA}},e}^{(k)}}{\partial
L_{\mbox{{APN}},t,e}^{(k)}}$

22: end for

23: end for

24: $L_{\mbox{{NHA}},t+1}=\frac{1}{K}\sum_{k\in
K}L_{\mbox{{NHA}},t,E+1}^{(k)}$

25: $L_{\mbox{{LTA}},t+1}=\frac{1}{K}\sum_{k\in
K}L_{\mbox{{LTA}},t,E+1}^{(k)}$

26: $L_{\mbox{{VQA}},t+1}=\frac{1}{K}\sum_{k\in
K}L_{\mbox{{VQA}},t,E+1}^{(k)}$

27: $L_{\mbox{{APN}},t+1}=\frac{1}{K}\sum_{k\in
K}L_{\mbox{{APN}},t,E+1}^{(k)}$

28: end for

29:

30: function Server($v_{\mbox{{VQA}},t,e}^{(k)}$,$v_{\mbox{{APN}},t,e}^{(k)}$)

31: $v_{\mbox{{NHA}},t,e}^{(k)}\leftarrow
f_{\mbox{{NHA}}}(v_{\mbox{{VQA}},t,e}^{(k)})$

32: $v_{\mbox{{LTA}},t,e}^{(k)}\leftarrow
f_{\mbox{{LTA}}}(v_{\mbox{{APN}},t,e}^{(k)})$

33:
$\mathcal{L}=J_{cl}(v_{\mbox{{NHA}},t,e}^{(k)},v_{\mbox{{LTA}},t,e}^{(k)})$

34: $\delta_{\mbox{{NHA}},e}^{(k)}=\frac{\partial\mathcal{L}}{\partial
L_{\mbox{{NHA}},t,e}^{(k)}}$

35: $\delta_{\mbox{{LTA}},e}^{(k)}=\frac{\partial\mathcal{L}}{\partial
L_{\mbox{{LTA}},t,e}^{(k)}}$

36: $L_{\mbox{{NHA}},t,e+1}^{(k)}\leftarrow
L_{\mbox{{NHA}},t,e}^{(k)}-\eta\cdot\delta_{\mbox{{NHA}},e}^{(k)}$

37: $L_{\mbox{{LTA}},t,e+1}^{(k)}\leftarrow
L_{\mbox{{LTA}},t,e}^{(k)}-\eta\cdot\delta_{\mbox{{LTA}},e}^{(k)}$

38: return $\delta_{\mbox{{NHA}},e}^{(k)},\delta_{\mbox{{LTA}},e}^{(k)}$ to
client $k$


L T

# Flow-Lenia: Towards open-ended evolution in cellular automata through mass
conservation and parameter localization

Erwan Plantec1, Gautier Hamon1, Mayalen Etcheverry1,2, Pierre-Yves Oudeyer1,
Clément Moulin-Frier1 and Bert Wang-Chak Chan1,3

1FLOWERS Team, Inria, Bordeaux, France
2Poietis, Pessac, France
3Brain Team, Google research, Tokyo, Japan
<EMAIL_ADDRESS>
###### Abstract

The design of complex self-organising systems producing life-like phenomena,
such as the open-ended evolution of virtual creatures, is one of the main
goals of artificial life. Lenia, a family of cellular automata (CA)
generalizing Conway’s Game of Life to continuous space, time and states, has
attracted a lot of attention because of the wide diversity of self-organizing
patterns it can generate. Among those, some spatially localized patterns
(SLPs) resemble life-like artificial creatures and display complex behaviors.
However, those creatures are found in only a small subspace of the Lenia
parameter space and are not trivial to discover, necessitating advanced search
algorithms. Furthermore, each of these creatures exist only in worlds governed
by specific update rules and thus cannot interact in the same one. This paper
proposes as mass-conservative extension of Lenia, called Flow Lenia, that
solve both of these issues. We present experiments demonstrating its
effectiveness in generating SLPs with complex behaviors and show that the
update rule parameters can be optimized to generate SLPs showing behaviors of
interest. Finally, we show that Flow Lenia enables the integration of the
parameters of the CA update rules within the CA dynamics, making them dynamic
and localized, allowing for multi-species simulations, with locally coherent
update rules that define properties of the emerging creatures, and that can be
mixed with neighbouring rules. We argue that this paves the way for the
intrinsic evolution of self-organized artificial life forms within continuous
CAs. A notebook with Flow Lenia implementation and demo are available at
https://tinyurl.com/mr2ncy3h.

## Introduction

Complex self-organizing systems have been central to Artificial Life (ALife)
towards the search of emergent life-reminiscent phenomenons. Among those
systems are continuous cellular automata (CAs) like SmoothLife (Rafler,, 2011)
or Lenia (Chan,, 2019). Lenia is a family of CAs generalizing Conway’s Game of
Life (GoL) to continuous space, time and states (Chan,, 2019, 2020). Each
instance of Lenia is given by a specific configuration of parameters defining
its update rule and an initial configuration which map to a specific pattern
and behavior in a deterministic way. Lenia is a generalization of GoL as the
latter corresponds to a specific parameter set in Lenia.

|  |
---|---|---
|  |
|  |

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

(j)

Figure 1: Flow Lenia creatures. (a-c) Samples of creatures found through
random search in Flow Lenia parameter space. (d-f) and (g-i) Timelapses of
patterns found through random search. Colors in (g-i) code for different
channels. (j) Effect of changing temperature in Flow Lenia, temperature is
linearly increasing from left to right. Videos are available at
https://sites.google.com/view/flowlenia/videos.

Previous studies with Lenia have shown the emergence of autopoïetic (i.e self-
produced) spatially localized patterns (SLPs), also called “creatures”, often
resembling microscopic life-forms and displaying various behaviors like
motility or self-replication. Such observations have made Lenia a particularly
interesting system for studying the emergence of life-like phenomenon and even
sensori-motor capabilities (Hamon et al.,, 2022). Importantly, these
artificial creatures bear a deep similarity to their biological counterparts
as enacted agents endowed with constitutive autonomy (i.e creatures are self-
producing) as described by enactive theories of cognition (see Froese and
Ziemke, (2009) for a more complete discussion on enactive artificial
intelligence).
However, those patterns are quite difficult to find, necessitating complex
fine-tuned search algorithms in order to find update rule parameters allowing
the emergence of interesting patterns (e.g spatially localized patterns). For
instance, complex patterns have been found in (Hamon et al.,, 2022) using
intrinsically motivated goal exploration processes and gradient descent.
An other important challenge in ALife and artificial intelligence is about the
design of systems displaying open-ended intrinsic evolution (i.e unbounded
growth of complexity through intrinsic evolutionary processes) (Stanley,,
2019). Such a process is called _intrinsic_ since no final objective (i.e
fixed fitness function) is set by the experimenter, the fitness landscape is
intrinsic to the system and depends only on its current state, as in natural
evolution where there is no final goal (Lehman and Stanley,, 2011). Systems
like EvoLoop (Sayama,, 1999) or an extension of it, SexyLoop (Oros and
Nehaniv,, 2007), allow for evolutionary processes to occur in CAs. However,
such systems rely on hand-defined rules and particular structures ultimately
limiting the kind of patterns that can emerge in the system. On the other
hand, even though Lenia creatures display greater diversity, different
creatures (governed by different update rules) cannot exist in the same world
(i.e the same simulation) and thus cannot interact. Obtaining such an
evolutionary process in a CA could be achieved by embedding information in the
system locally modifying the update rule and so the properties of emerging
creatures, like a genome, enabling multi-species simulations. Such simulations
might set the stage for evolution to occur in populations of patterns each
with their own update rule. However, achieving it in CAs like Lenia is still
an open-problem.
We believe that adding mass conservation is a key ingredient to address the
aforementioned challenges. Such a constraint could (i) constrain emerging
creatures to spatially localized ones, (ii) allow for the design of multi-
species simulations and (iii) provide an important evolutionary pressure
(Hickinbotham and Stepney,, 2015). We propose in this work a mass-conservative
extension to Lenia called _Flow Lenia_ and demonstrate that such conservation
laws effectively facilitate the search fort artificial creatures by
constraining (almost all) emerging patterns to spatially localized ones. We
also show that the update rule parameters can easily be optimized using
vanilla evolutionary strategies (Salimans et al.,, 2017) with respect to some
fitness functions to obtain patterns with specific properties such as directed
motion or angular motion. Importantly, we show that the Flow Lenia formulation
enables the integration of the parameters of the CA update rules within the CA
dynamics, making them dynamic and localized, allowing for multi-species
simulations, with locally coherent update rules that define properties of the
emerging creatures. Finally, we propose a resource collection mechanism which
could act as a selective pressure in such multi-species simulations.

## Model

Let $\mathcal{L}$ be the support of a CA, which is the two-dimensional grid
$\mathbb{Z}^{2}$ in the rest of this work. Let $A^{t}:\mathcal{L}\to S^{C}$ be
CA’s activations at time $t$, with $A^{t}_{i}(x)$ the activation in location
$x\in\mathcal{L}$, channel $i$ and time $t$. $C$ is the number of channels of
the system ans $S$ is the state space (the set of states a cell can take in
each channel).

Figure 2: Top left : Lenia update rule. The growth $U^{t}$ is computed with
kernels $K$ and growth functions $G$ (equation 3) defined by a specific
parameter configuration sampled in Lenia’s parameter space. A small portion of
the growth is then added to activations $A^{t}$ to give the next state
$A^{t+dt}$ (equation 4). Bottom left : Flow Lenia update rule. Affinity map
$U^{t}$ is computed as in Lenia. The flow $F^{t}$ is given by combining the
affinity map and concentration (i.e activations) gradients (equation 5).
Finally, the next state is obtained by “moving” matter in the CA space
according to the flow $F^{t}$ using reintegration tracking (RT) (equations 6
and 7). Right : Patterns generated from randomly sampled parameters sets (no
cherry picking) in Lenia (left) and Flow Lenia (right). Parameters used for
both systems are the same (i.e each row corresponds to one parameters set).

### Lenia

The state space $S$ in Lenia is the unit range $[0,1]$. An instance of Lenia
is defined by a tuple $<K,G,A^{0}>$ where $K$ is a set of convolution kernels
where $K_{i}:\mathcal{L}\to[0,1]$ satisfies $\int_{\mathcal{L}}K_{i}=1$ and
$G$ is a set of growth functions with $G_{i}:[0,1]\to[-1,1]$. Each pair
$(K_{i},G_{i})$ is associated to a source channel $c^{i}_{0}$ it senses and a
target channel $c^{i}_{1}$ it updates. Connectivity can be represented through
a square adjacency matrix
$M_{C,C}=\left[\begin{array}[]{ccc}m_{11}&\cdots&m_{1C}\\\
\vdots&\ddots&\vdots\\\ m_{C1}&\cdots&m_{CC}\end{array}\right]$ where
$m_{ij}\in\mathbb{N}$ is the number of kernels sensing channel $i$ and
updating channel $j$. $A^{0}$ is the initial state of the system. As in Hamon
et al., (2022), kernels are radially symmetrical and defined as a sum of
concentric Gaussian bumps :

$K_{i}(x)=\sum_{j=1}^{k}b_{i,j}\,exp\left(-\frac{(\frac{x}{r_{i}R}-a_{i,j})^{2}}{2w_{i,j}^{2}}\right)$
(1)

Where $a_{i}$, $b_{i}$, $w_{i}$ and $r_{i}$ are parameters defining kernel
$i$. $k$ is a parameter defining the number of rings per kernel (set to 3
here) and $R$ is a parameter common to all kernels defining the maximum
neighborhood radius. Each kernel is then defined by $3\times k+1$ parameters.
Growth functions are defined as Gaussian function scaled in the range
$[-1,1]$:

$G_{i}(x)=2\;exp\left(-\frac{(\mu_{i}-x)^{2}}{2\sigma_{i}^{2}}\right)-1$ (2)

Where $\mu_{i}$ and $\sigma_{i}$ are parameters of growth function $i$ so each
growth function is defined by $2$ parameters. A step in Lenia is defined by
the following steps (see figure 2 (top)) :

  1. 1.

Compute the growth at time $t$ given the actual state $A^{t}$ :

$U^{t}_{j}=\sum_{i}h_{i}\cdot G_{i}(K_{i}\ast
A_{c^{i}_{0}}^{t})\mathbb{1}_{[c^{i}_{1}=j]}$ (3)

Where $h\in\mathbb{R}^{|K|}$ is a vector weighting the impact of each pair
$(K_{i},G_{i})$ on the growth.

  2. 2.

Add a small portion of the growth $U^{t}$ to the actual state $A^{t}$ to get
the state at the next time step and clip results back to the unit range :

$A^{t+dt}_{i}=[A^{t}_{i}+dt\,U^{t}_{i}]_{0}^{1}$ (4)

### Flow Lenia

Flow Lenia is a mass-conservative extension to Lenia. By mass-conservative, we
mean that the sum of activations across all cells and for each channel is
constant over time :

$\sum_{x\in\mathcal{L}}A^{t}_{c}=\sum_{x\in\mathcal{L}}A^{t+dt}_{c}\;,\forall
t,\forall c\in\\{1,...,C\\}$

We propose for this system to interpret activations as concentrations of
“matter” in all cells and to refer to the term $U^{t}$, previously called the
growth in Lenia, as an affinity map. The idea is that the matter will move
towards higher affinity regions by following the local gradient of the
affinity map $U$, $\nabla U:\mathcal{L}\to\mathbb{R}^{2}$. To do so, we define
a flow $F:\mathcal{L}\to(\mathbb{R}^{2})^{C}$, which can be interpreted as the
instantaneous speed of matter, as:

$\begin{cases}F^{t}_{i}=(1-\alpha^{t})\nabla U^{t}_{i}-\alpha^{t}\nabla
A^{t}_{\Sigma}\\\
\alpha^{t}(p)=[({A^{t}_{\Sigma}(p)}/{\theta_{A}})^{n}]_{0}^{1}\end{cases}$ (5)

With $A^{t}_{\Sigma}(p)=\sum_{i=1}^{C}A^{t}_{i}(p)$ the total mass in each
location $p$. Here $\nabla U^{t}_{i}$ is the affinity gradient for channel
$i$. The negative concentration gradient $-\nabla A^{t}_{\Sigma}$ is a
diffusion term to avoid concentrating all the matter in very small regions
akin to the clipping in Lenia which upper bounds concentrations. In practice,
gradients are estimated through Sobel filtering. Map
$\alpha:\mathcal{L}\to[0,1]$ is used to weight the importance of each term
such that $-\nabla A^{t}_{\Sigma}$ dominates when the total mass at a given
location is close to a critical mass $\theta_{A}\in\mathbb{R}_{>0}$.
Intuitively, the result is that matter is mainly driven by concentration
gradients in high concentrations regions and is more free to move along the
affinity gradient in less concentrated areas. We typically use $n>1$ such that
the affinity gradient dominates on a larger range of masses.
Then, we can move matter in space according to flow $F$ giving us the state at
the next time step. To do so we use the reintegration tracking method proposed
in (Moroz,, 2020). Reintegration tracking is a semi-Lagrangian grid based
algorithm thought as a reformulation of particle tracking in screen space (i.e
grid space) aimed at not losing information (i.e particles) which happens when
two particles end up in the same cell. The basic principle is to work with
distribution of particles (i.e infinite number of particles) and conserve the
total mass by adding up masses going on a same cell. Overall, reintegration
tracking can be seen as a grid-based approximation to particle systems with
infinite number of particles having the property to conserve total mass. Thus,
Flow lenia can be seen as a new kind of model at the frontier between
continuous CAs and particle systems. A particle based model directly inspired
by the Flow Lenia formulation has been recently proposed in Mordvintsev et
al., (2022). Figure 3 illustrates how reintegration tracking is used in our
case. The resulting update rule is the following :

$A^{t+dt}_{i}(p)=\sum_{p^{\prime}\in\mathcal{L}}A^{t}_{i}(p^{\prime})I_{i}(p^{\prime},p)$
(6)

Where $I_{i}(p^{\prime},p)$ is the proportion of incoming matter in channel
$i$ going from cell $p^{\prime}\in\mathcal{L}$ to cell $p\in\mathcal{L}$:

$I_{i}(p^{\prime},p)=\int_{\Omega(p)}\mathcal{D}(p_{i}^{\prime\prime},s)$ (7)

With $p_{i}^{\prime\prime}=p^{\prime}+dt\cdot F^{t}_{i}(p^{\prime})$ the
target location of the flow from $p^{\prime}$. $\Omega(p)$ is the domain of
cell at location $p$, which is a square of side $1$. $\mathcal{D}(m,s)$ is a
distribution defined on $\mathcal{L}$ with mean $m$ and variance $s$
satisfying $\int_{\mathcal{L}}\mathcal{D}(m,s)=1$, which is in practice a
uniform square distribution with side length $2s$ centered at $m$. This
distribution emulates a flow of particles from source area
$\Omega(p^{\prime})$ to target area $\mathcal{D}(p^{\prime\prime},s)$, where
the distribution $\mathcal{D}$ emulates Brownian motion at the low level. $s$
is an hyperparameter of the system which can be seen as form of temperature.
The reintegration tracking method is depicted in Fig. 3. Since the
distribution $\mathcal{D}$ integrates to 1, it is clear that a cell cannot
send out more mass than it contains nor less and so the system conserves its
total mass. Mass conservation also implies that cells’ states are no longer
bound to the unit range but can be any positive real valued number
($S\equiv\mathbb{R}_{\geq 0}^{C}$). This model has been implemented in JAX
(Bradbury et al.,, 2018) allowing fast simulation on GPU ($255\mu s\pm 3.11\mu
s$ per step on Tesla T4 GPU with 1 channel, 10 kernels and $128\times 128$
world size).

Figure 3: Calculation of incoming matter to cell $p\in\mathcal{L}$ through
reintegration tracking (Moroz,, 2020). Mass contained in cell at location
$p^{\prime}\in\mathcal{L}$ is moved to a square distribution $\mathcal{D}$
centered on $p^{\prime\prime}=p^{\prime}+dt\cdot F^{t}(p^{\prime})$. The
proportion of mass from $p^{\prime}$ arriving in $p$ is then given by the
integral of $\mathcal{D}$ on the cell domain of $p$, $\Omega(p)$, denoted as
$I(p^{\prime},p)$.

## Results

Neighborhood | Growth functions
---|---
$R$ | $\in[2,25]$ |  | $\mu$ | $\in[0.05,0.5]$ | *
$r$ | $\in[0.2,1]$ | * | $\sigma$ | $\in[0.001,0.2]$ | *
Kernels | Flow
$h$ | $\in[0,1]$ | * | $s$ | $0.65$ |
$a$ | $\in[0,1]^{3}$ | * | $n$ | $2$ |
$b$ | $\in[0,1]^{3}$ | * | $dt$ | $0.2$ |
$w$ | $\in[0.01,0.5]^{3}$ | * |  |  |
Table 1: Flow Lenia explored parameter space. Parameters marked with a * must
be sampled for each kernel-growth function pair.

### Random search

By performing random and manual search of the Flow Lenia parameter and
hyperparameter space described in table 1 we have been able to discover SLPs
with already interesting and complex behaviors. Such patterns are shown in
figure 1 and described hereafter. Initial patterns $A^{0}$ are set with a
$40\times 40$ patch with matter drawn from uniform distribution in the center
of the grid and no matter everywhere else.
Most of the patterns generated in Flow Lenia are SLPs (see figure 2) with rare
exceptions found by manually setting parameters to specific configurations
leading to scattered matter. Using multiple kernels led to the emergence of
SLPs with more complex shapes and behaviors. While part of emerging patterns
tend to be static ones, dynamic patterns are quite common in Flow Lenia. For
instance gyrating SLPs (a) or snake like patterns (b) with complex motion
emerging from attraction/repulsion dynamics can be frequently observed.
Dividing and merging dots (c) resembling reaction-diffusion patterns are also
a common pattern. Timelapse (d-f) shows a creature with complex and
unpredictable dynamics emerging from the interactions of its membrane,
multiple organoids-like structures and a central nuclei ultimately leading to
a phase transition happening in (e). Timelapse (g-e) shows a 2-channels
creature displaying complex division patterns and interesting modular
creatures whose characteristics change depending on their total mass while
being of the same “kind” (i) (see 5 creatures on the leftmost part of (i)).
Note that multi-channel creatures often show more complex dynamics and
patterns with very modular shapes where each channel seems to occupy a
different role. (j) shows the effect of changing the size of the reintegration
tracking distribution $s$ (see equation 7 and figure 3), parameter we call
temperature. Here temperature is linearly increasing from left to right
showing very different phases of the systems. More interestingly, patterns at
the frontier between the Turing-like phase (center) and the equilibrium phase
(right) are much more dynamic and display unpredictable dynamics suggesting a
critical regime.

### Optimizing Flow Lenia creatures

Figure 4: Results of evolutionary optimization. C is the number of channels of
the system and $k$ is the number of kernels and growth functions. When
performing the exact same optimization for directed motion in the original
Lenia system (yellow curve), not only optimization is unstable but it only
discovers exploding patterns.

|
---|---
|

(a)

(b)

(c)

(d)

Figure 5: Creatures found through optimization. (a) Directed motion with 2
channels and 20 kernels. (b) Angular motion with 2 channels and 20 kernels.
(c) Motion through obstacles with 2 channels and 25 kernels. (d) Chemotaxis
with 2 channels and 25 kernels. Videos ara availabe at
https://sites.google.com/view/flowlenia/videos.

In this section, we show that Flow Lenia update rule parameters can also be
easily optimized so to generate patterns with specific behaviors. This is a
difficult task in Lenia as it would require constantly monitoring the
existential status and the spatially-localizedness of evolved creatures. Thus
training creatures in Lenia requires to define characterizations of creatures
accounting for such properties which is a far from trivial problem. Moreover,
even if one can come up with proxies to find spatially localized patterns, the
optimization process remains difficult necessitating advanced optimization
methods like curriculum learning (Hamon et al.,, 2022). In Flow Lenia, the
spatial localization constraint is intrinsic to the system thus removing the
necessity to account for it when searching for creatures. Using evolutionary
strategies (Salimans et al.,, 2017) to optimize the update rule parameters and
the initial configuration ($A^{0}$) with respect to user-defined fitness
functions, we have been able to successfully find good solutions for 4
different tasks : directed motion, angular motion, navigation through
obstacles and chemotaxis.
We used evosax (Lange,, 2022) implementation of the OpenES strategy with
population size of $16$ and adam optimizer (Kingma and Ba,, 2017) with $0.01$
as learning rate. We optimized the Flow Lenia update rule with different
number of kernels and either 1 or 2 channels. For comparison, we also trained
original Lenia on the directed motion task following the same optimization
procedure. The initial pattern is composed, as in random search, of a square
patch with non-zero activations placed at the center of the world and zeros
everywhere else. Results are shown in figure 4.

#### Directed motion

In order to train creatures displaying directed motion, i.e straight line
motion, we used the distance travelled by the creature as the fitness
function. The distance is calculated by computing the center of mass of the
pattern at step 0 and final step 400. Formally, the fitness function is
defined as :

$f(\theta)=dist(\phi(A^{0}),\phi(A^{400}))$

Where $A\equiv\\{A^{0},...,A^{T}\\}$ is the pattern obtained by making a
rollout with parameters $\theta$ for $T$ timesteps (here 500).
$\phi(A^{t})\in[-0.5,0.5]^{2}$ is the center of mass of state $A^{t}$ and
$dist$ is the euclidean distance function. We optimized the system with either
1 or 2 channels and 10 or 20 kernels. We used
$M=\left[\begin{array}[]{cc}5&5\\\ 5&5\end{array}\right]$ as the adjacency
matrix with 2 channels and 20 kernels and $M=\left[\begin{array}[]{cc}3&2\\\
2&3\end{array}\right]$ with 10 kernels.
Results (see fig 4) show that good solutions can be found in the 2 channels
condition but not in the single channel case. However, when running the
algorithm for longer (e.g 5000 generations), we have been able to found single
channel creatures with similar fitness than their 2 channels counterpart.
Increasing the number of kernels led to faster discovery of good solutions.
The best performing creature is shown in figure 5(a). This creature moves
because of attraction/repulsion dynamics between the 2 channels which might
explain why directed motion is much easier to attain with multi-channels
creatures. On the other hand, the optimization of the original Lenia model is
much less stable and discovered patterns are less successful than their mass-
conservative counterparts. Moreover, every Lenia optimized patterns are
exploding ones.

#### Angular motion

In this task, we want emerging creatures to display more complex forms of
motion. More precisely, we want creatures to be able to move and make turns.
As with directed motion, we use the center of mass of the creature through
time to compute its trajectory. The fitness function is the following :

$\displaystyle f(\theta)$ $\displaystyle=dist(\phi(A^{0}),\phi(A^{200}))$
$\displaystyle+dist(\phi(A^{200}),\phi(A^{400}))$
$\displaystyle+\angle[\phi(A^{200})-\phi(A^{0})]\,[\phi(A^{400})-\phi(A^{200})]$

Where $\angle ab$ is the angle between vectors $a$ and $b$. The first two
terms are the distance travelled from step 0 to step 200 and from step 200 to
step 400. The last term is the angle between these two trajectories which is
maximal when they are opposite. In order to avoid large angles to come from
very small movements, the angle is set to 0 when distance traveled either
before or after step 200 is below a given threshold. The optimal behavior for
this fitness function is then to move fast in one direction, make a 180° turn,
and then move fast in the opposite direction. We used 2 channels, 20 kernels
and the same connectivity matrix as for directed motion.
Result are shown in figure 4. The best performing creature, shown in figure 5
(b), displays very complex internal dynamics leading it to periodically make
180° turns while moving in straight line the rest of the time. These dynamics
seem to be generated by attraction repulsion dynamics like the ones observed
in directed motion but here in a more intricate morphology.

#### Navigation through obstacles

In this task, we want to see if creatures can navigate through obstacles as
done in Hamon et al., (2022). To do so, we added walls which are implemented
by adding a strong flow going from the center of walls outwards, thus strongly
repelling the creature and acting as a solid obstacle. At each evaluation of
the optimization process, we randomly sample points on a circle surrounding
the creatures’ initial positions to be walls positions thus making a “forest”
of walls around the creature. We then optimize the creature with the same
fitness function as in the _directed motion_ task so creatures have to go as
far as possible and so through the forest. We made the experiment with 2
channels creatures, walls are defined in a separate third channel. We used 25
kernels and $M=\left[\begin{array}[]{ccc}5&5&0\\\ 5&5&0\\\
5&0&0\end{array}\right]$ as the connectivity matrix so creatures are able to
sense the walls channel (3rd channel).
We have been able to successfully train creatures able to move and stay robust
when making contact with walls such as the one shown in figure 5 (c) which is
able to resist deformation and find a way out the “forest”. In comparison,
solving a similar task in Lenia required complex optimization methods based on
curriculum learning, diversity search and gradient descent over a
differentiable CA (Hamon et al.,, 2022). However, such a comparison is
difficult because Flow Lenia creatures are inherently more robust due to
conservation of mass, whereas Lenia creatures can disappear because of
perturbations.

#### Chemotaxis

Another important feature of natural life-forms is the ability to sense their
environment in order to find food or avoid dangers through chemotaxis. In this
task, we want creatures to be able to sense a “chemical” gradient and climb it
towards its maximum. To do so, we added a separate channel
$\Gamma:\mathcal{L}\to\mathbb{R}_{\geq 0}$ whose activations are defined
following a Gaussian function around a point randomly sampled on a circle
surrounding the center of the CA for each evaluation of the optimization
process ensuring creatures learn to follow gradient and not a fixed direction
while keeping the distance to cover constant. We also added 5 kernels and
growth functions from $\Gamma$ to $A$, which are also optimized, so the
creature is able to sense the chemical. The fitness of an individual is then
computed with the following function :

$f(\theta)=\frac{\sum_{x\in\mathcal{L}}A^{500}_{\Sigma}(x)\times\Gamma(x)}{\sum_{x\in\mathcal{L}}A^{500}_{\Sigma}(x)}$

Since mass is conserved, the optimal behavior for a creature is to concentrate
as much of its mass in the cells where $\Gamma$ is maximal.
We have been able to find good solutions to this task as shown in figure 4.
Best solutions such as the one shown in figure 5 (d) are perfectly able to
climb the gradient towards its maximum.

## Flow Lenia : Towards intrinsic evolution

---
|
|

(a)

(b)

(c)

(d)

(e)

(f)

Figure 6: Multi-species simulations. (a) Sample of a multi-species simulation
where color code for parameters. (b-e) Timelapse of larger scale multi-species
simulation. World is a $1024\times 1024$ grid initialized with $144$ creatures
with distinct parameters represented by colors. We simulate 200k timesteps and
use softmax sampling as the mixing rule and random mutations every $500$
steps. (f) Timelapse of simulation with parameter embedding and food (in blue)
showing division events (highlighted with boxes). Videos are available at
https://sites.google.com/view/flowlenia/videos.

As said in the introduction, Flow Lenia allows to embed the update rule
parameters inside the CA. Intuitively, we can “attach” a vector of parameters
to the matter locally modifying how it behaves (i.e locally modifying how the
affinity map is computed), and let it flow with it. Formally, this comes to
defining a parameter map $P:\mathcal{L}\to\Theta$ where $\Theta$ is a given
parameter set. This map can be used to locally modify the update rule. For
instance, we can embed the $h\in\mathbb{R}^{|K|}$ vector weighting the
importance of each kernel in the affinity map computation (see equation 3),
giving :

$U^{t}_{j}(p)=\sum_{i,k}P^{t}_{k}(p)\cdot G_{k}(K_{k}\ast A^{t}_{i})(p)$ (8)

Then parameters can be moved along with matter. A question is how to mix
parameters arriving in the same cell. Here we propose two different methods
which are respectively _average_ and _softmax sampling_. The former makes a
weighted average of incoming parameters with respect to the quantities of
incoming matter and is formally defined as :

$P^{t+dt}(p)=\frac{\sum_{p^{\prime}\in\mathcal{L}}A^{t}(p^{\prime})I(p^{\prime},p)P^{t}(p^{\prime})}{\sum_{p^{\prime}\in\mathcal{L}}A^{t}(p^{\prime})I(p^{\prime},p)}$
(9)

Softmax sampling on the other hand samples a parameter in the set of incoming
ones following the softmax distribution given by incoming quantities of matter
:

$\mathbb{P}[P^{t+dt}(p)=P^{t}(x)]=\frac{\exp(A^{t}(x)I(x,p))}{\sum_{p^{\prime}\in\mathcal{L}}\exp(A^{t}(p^{\prime})I(p^{\prime},p))}$
(10)

Intuitively, the more represented set of parameters has a greater probability
of being selected in the cell, like simulating in one step a competition
between different parameters in the cell. Note that we could also sample each
element of the vector of parameters independently giving some crossover
mechanism. We can also add mutations in the simulation. For instance, we can,
at a given rate, modify parameters in a randomly sampled zone by adding
Gaussian noise to the parameter map.
The parameters embedding mechanism allows to run multi-species simulations
where all creatures patches are initialized with different parameters as shown
in figure 6(a) where colors code for parameters. We can see that the
creatures, although in the same CA, display different morphologies even though
they have been initialised with the same pattern. We also ran larger scale
simulations (e.g $1024\times 1024$ worlds initialized with 144 creatures and
simulated for 200k timesteps) using parameter embedding as shown in figure
6(b-e). Such a simulation display very interesting dynamics on the large scale
where some “species” can take over large parts of the world by contaminating
other species before reaching more stable states. Also, we can see the
emergence of coherent creatures composed of different parameters where one of
those will compose a sort of membrane around a nuclei composed by different
parameters (see green and pink creature on bottom left of figure 6(e)). Such a
pattern can be seen as a form of symbiosis.
While parameter embedding could lead to the emergence of intrinsic
evolutionary processes where parameters would compete for available matter, we
can also add intrinsic selective pressures in the form of resource collection
mechanisms bootstrapping evolution. The idea is to add food resources that
creatures would need to collect in order to replenish their own constantly
decaying pool of resources. To do so, we let matter decay at a fixed rate
$\rho_{decay}$, and create a food map $\Psi:\mathcal{L}\to\mathbb{R}_{\geq
0}$. When matter is in a cell where there is also food, then food is
transformed into matter at a given rate $\rho_{digest}$ giving the following
update.

$\begin{cases}A^{t+dt}(x)=\cdots+[A^{t}(x)\rho_{digest}]_{0}^{\Psi^{t}(x)}-A^{t}(x)\rho_{decay}\\\
\Psi^{t+dt}(x)=\Psi^{t}(x)-[A^{t}(x)\rho_{digest}]_{0}^{\Psi^{t}(x)}\end{cases}$
(11)

Where $\cdots$ refers to the update equation 6 and $[\cdot]_{a}^{b}$ is the
clip function between $a$ and $b$. We enable creatures to sense food by adding
kernels and growth function from the food map $\Psi$ to creatures’ channels
$A$.
By performing simulations with random set of parameters, parameter embedding
and food, we have been able to observe interesting patterns. First, we have
been able to observe that some creatures, while not having trained for it, are
able to go towards nearby food sources and consume it. We can hypothesize that
creatures with such a capability will survive (and grow) while other will not
leading to intrinsic evolution. Quite interestingly, complex patterns can
emerge from the change of mass induced by decay or food consumption. For
instance, when growing after eating, some creatures will divide in two
identical creatures as shown in figure 6 (f), crucial pattern for evolution to
occur. On the other hand, mass decay also leads to interesting dynamics where
creatures undergo phase transitions, changing their shape and behavior, when
their mass falls below a certain threshold which can lead them to adopt
foraging behaviour for example while being initially static.

## Discussion

Due to the mass conservative nature of Flow Lenia, most of patterns do not
grow indefinitely into spatially global patterns (i.e patterns that diffuse on
the entire grid also called Turing-like patterns), therefore SLPs are much
more common and easier to find. This is an important difference from the
previous versions of Lenia, where one needs to search or evolve for patterns
that are both non-vanishing and non-exploding, and to constantly monitor their
existential status. Here, the mass conservation constraint acts as a
regularizer on the kind of patterns that can emerge. Even though patterns
generated by Flow Lenia are often static or slowly moving, we have been able
to find creatures with complex dynamics from random search only which would be
a difficult task in Lenia as most of the search space corresponds to either
exploding or vanishing patterns. Furthermore, we have shown that the update
rule parameters can be optimized with simple evolutionary strategies to
generate patterns with specific properties and behaviors. Doing so in Lenia is
a difficult task since the spatial localization of emergent patterns is not
guaranteed necessitating more complex algorithms accounting for such a
property.
Finally we showed that the Flow Lenia system allowed for the integration of
the update rule parameters within the CA dynamics allowing for the coexistence
of multiple update rules, and thus different creature or species, within the
same simulation. We argue that such a feature represent an important step
towards the design of emergent microcosms (Arbesman,, 2022) in which could
emerge intrinsic, maybe open-ended, evolutionary processes through inter-
species interactions. Moreover, we also showed that food consumption
mechanisms can be implemented in Flow Lenia which could act as intrinsic
selective pressures in such settings bootstrapping evolutionary processes
though competitive dynamics. Whereas environment design is poorly addressed
and quite challenging in cellular automata systems, we believe that it is
crucial to study the emergence of agency and cognition in those systems as
argued in Godfrey-Smith, (2002) and shown in Hamon et al., (2022). By enabling
the design of complex environmental features like inter-species interactions,
walls, food or temperature, Flow Lenia could represent a particularly
interesting system to study ecological theories of the evolution of complexity
and cognition (Nisioti et al.,, 2021).
Lot of exciting roads remain to be taken in order to fully capture the value
of complex self-organized systems such as Flow Lenia. A major path to explore
is about the detection of agency and cognition in such systems where
everything is emergent with no predefined notion of individuals. Information
theoretical measures of concepts like individuality, autonomy and agency
(Krakauer et al.,, 2020; Bertschinger et al.,, 2008; Negru,, 2021; Kolchinsky
and Wolpert,, 2018) could be of great interest in this perspective even though
they remain difficultly applicable in large systems. Demonstrating the
emergence of intrinsic evolutionary processes has been left out of the scope
of this paper and left for future work. However, evaluating such a system
appears as a difficult task. Quantifying intrinsic evolutionary processes with
measures like evolutionary activity (Bedau and Packard,, 1996; Droop and
Hickinbotham,, 2012) could reveal to be useful in order to get a better grasp
on the complex dynamics taking place in simulations such as the one shown in
figure 6. Ultimately, such simulations could be key for the growth of
complexity inside the system and the emergence of virtual creatures displaying
more complex forms of cognition.

## References

  * Arbesman, (2022) Arbesman, S. (2022).  Emergent Microcosms.
  * Bedau and Packard, (1996) Bedau, M. A. and Packard, N. H. (1996).  Measurement of Evolutionary Activity, Teleology, and Life.
  * Bertschinger et al., (2008) Bertschinger, N., Olbrich, E., Ay, N., and Jost, J. (2008).  Autonomy: An information theoretic perspective.  Bio Systems, 91(2):331–345.
  * Bradbury et al., (2018) Bradbury, J., Frostig, R., Hawkins, P., Johnson, M. J., Leary, C., Maclaurin, D., Necula, G., Paszke, A., VanderPlas, J., Wanderman-Milne, S., and Zhang, Q. (2018).  JAX: Composable transformations of Python+NumPy programs.
  * Chan, (2019) Chan, B. W.-C. (2019).  Lenia - Biology of Artificial Life.  Complex Systems, 28(3):251–286.
  * Chan, (2020) Chan, B. W.-C. (2020).  Lenia and Expanded Universe.  In The 2020 Conference on Artificial Life, pages 221–229.
  * Droop and Hickinbotham, (2012) Droop, A. and Hickinbotham, S. (2012).  A quantitative measure of non-neutral evolutionary activity for systems that exhibit intrinsic fitness.  In ALIFE 2012: The Thirteenth International Conference on the Synthesis and Simulation of Living Systems, pages 45–52. MIT Press.
  * Froese and Ziemke, (2009) Froese, T. and Ziemke, T. (2009).  Enactive artificial intelligence: Investigating the systemic organization of life and mind.  Artificial Intelligence, 173(3):466–500.
  * Godfrey-Smith, (2002) Godfrey-Smith, P. (2002).  Environmental complexity and the evolution of cognition.  In The Evolution of Intelligence, pages 223–249. Lawrence Erlbaum Associates Publishers, Mahwah, NJ, US.
  * Hamon et al., (2022) Hamon, G., Etcheverry, M., Chan, B. W.-C., Moulin-Frier, C., and Oudeyer, P.-Y. (2022).  Learning sensorimotor agency in cellular automata.
  * Hickinbotham and Stepney, (2015) Hickinbotham, S. and Stepney, S. (2015).  Conservation of Matter Increases Evolutionary Activity.  In ECAL 2015: The 13th European Conference on Artificial Life, pages 98–105. MIT Press.
  * Kingma and Ba, (2017) Kingma, D. P. and Ba, J. (2017).  Adam: A Method for Stochastic Optimization.
  * Kolchinsky and Wolpert, (2018) Kolchinsky, A. and Wolpert, D. H. (2018).  Semantic information, autonomous agency and non-equilibrium statistical physics.  Interface Focus, 8(6):20180041.
  * Krakauer et al., (2020) Krakauer, D., Bertschinger, N., Olbrich, E., Flack, J. C., and Ay, N. (2020).  The information theory of individuality.  Theory in Biosciences, 139(2):209–223.
  * Lange, (2022) Lange, R. T. (2022).  Evosax: JAX-based Evolution Strategies.
  * Lehman and Stanley, (2011) Lehman, J. and Stanley, K. O. (2011).  Novelty Search and the Problem with Objectives.  In Riolo, R., Vladislavleva, E., and Moore, J. H., editors, Genetic Programming Theory and Practice IX, Genetic and Evolutionary Computation, pages 37–56. Springer, New York, NY.
  * Mordvintsev et al., (2022) Mordvintsev, A., Niklasson, E., and Randazzo, E. (2022).  Particle Lenia and the energy-based formulation.  https://google-research.github.io/self-organising-systems/particle-lenia/.
  * Moroz, (2020) Moroz, M. (2020).  Reintegration tracking.  https://michaelmoroz.github.io/Reintegration-Tracking/.
  * Negru, (2021) Negru, T. (2021).  Self, Agency and Autonomy in Dynamical Living Systems.  Synthesis philosophica, 36(1):191.
  * Nisioti et al., (2021) Nisioti, E., Jodogne-del Litto, K., and Moulin-Frier, C. (2021).  Grounding an Ecological Theory of Artificial Intelligence in Human Evolution.  In NeurIPS 2021 - Conference on Neural Information Processing Systems / Workshop: Ecological Theory of Reinforcement Learning, virtual event, France.
  * Oros and Nehaniv, (2007) Oros, N. and Nehaniv, C. (2007).  Sexyloop: Self-Reproduction, Evolution and Sex in Cellular Automata.
  * Rafler, (2011) Rafler, S. (2011).  Generalization of Conway’s ”Game of Life” to a continuous domain - SmoothLife.
  * Salimans et al., (2017) Salimans, T., Ho, J., Chen, X., Sidor, S., and Sutskever, I. (2017).  Evolution Strategies as a Scalable Alternative to Reinforcement Learning.  arXiv:1703.03864 [cs, stat].
  * Sayama, (1999) Sayama, H. (1999).  Toward the realization of an evolving ecosystem on cellular automata.  Proc. Fourth Int. Symp. Artificial Life and Robotics, pages 254–257.
  * Stanley, (2019) Stanley, K. O. (2019).  Why Open-Endedness Matters.  Artificial Life, 25(3):232–235.