# Towards evolution of Deep Neural Networks through contrastive Self-
Supervised learning

Adriano Vinhas  University of Coimbra
CISUC/LASI, DEI
Coimbra, Portugal
<EMAIL_ADDRESS>João Correia  University of Coimbra
CISUC/LASI, DEI
Coimbra, Portugal
<EMAIL_ADDRESS>Penousal Machado  University of Coimbra
CISUC/LASI, DEI
Coimbra, Portugal
<EMAIL_ADDRESS>
###### Abstract

Deep Neural Networks (DNNs) have been successfully applied to a wide range of
problems. However, two main limitations are commonly pointed out. The first
one is that they require long time to design. The other is that they heavily
rely on labelled data, which can sometimes be costly and hard to obtain. In
order to address the first problem, neuroevolution has been proved to be a
plausible option to automate the design of DNNs. As for the second problem,
self-supervised learning has been used to leverage unlabelled data to learn
representations. Our goal is to study how neuroevolution can help self-
supervised learning to bridge the gap to supervised learning in terms of
performance. In this work, we propose a framework that is able to evolve deep
neural networks using self-supervised learning. Our results on the CIFAR-10
dataset show that it is possible to evolve adequate neural networks while
reducing the reliance on labelled data. Moreover, an analysis to the structure
of the evolved networks suggests that the amount of labelled data fed to them
has less effect on the structure of networks that learned via self-supervised
learning, when compared to individuals that relied on supervised learning.

###### Index Terms:

Self-supervised learning, NeuroEvolution, Deep Learning, Evolutionary Machine
Learning

##  I Introduction

Deep Learning (DL) has demonstrated its power by achieving new state of the
art results and surpassing previous ones based on more traditional methods.
Part of this success is due to the availability of large-scale labelled
datasets. However, labels are often time-consuming and costly to obtain.
Additionally, one could argue that label-based learning can be limiting, as
the produced labels are not inherent in the data and, therefore, do not
accurately relate to how animals, including humans, tend to learn. For
instance, during early stages of life, young children can learn to
discriminate between common categories (e.g. birds and dogs [1]) without being
explicitly told that they have different labels. This encourages the idea
that, at initial stages, learning is unsupervised and uses perceptual
abilities to build representations of those categories [2].

Inspired on this rationale, Self-Supervised learning (SSL) has been growing on
popularity, due to its ability to leverage vast amounts of unlabelled data
available. Its goal is to learn helpful representations without requiring
extrinsic labels. It was found that DNNs trained using SSL obtained
breakthrough results in fields like Natural Language Processing (NLP),
overshadowing the ones obtained with supervised learning. However, the same
progress is yet to be achieved in computer vision. We believe that there are
two main reasons for this. The first one is that most of the works that apply
SSL on computer vision have used a limited range of network topologies. This
is because the design of a DNN is a time-consuming task, as its
hyperparameters are optimised following a trial-and-error process. The second
reason is because SSL algorithms are manually designed too, which hampers the
discovery of new approaches that might be more suited to the image domain.

Evolutionary Computation (EC) is another biologically inspired field which
borrows ideas from evolution theory. It has been used to automate the search
for the best topology or other hyperparameters of a given DNN. The
intersection of these two fields originated a subfield known as Neuroevolution
(NE). Not only NE helps designing the network that better suits a task
automatically, it also promotes the emergence of increasingly better solutions
by following a bio-inspired approach.

In this paper, we hypothesise that combining SSL algorithms with NE promotes
the emergence of DNNs that are useful for specific tasks (in our case, image
classification). By merging these two areas, we believe that we can take the
best of SSL and NE: reduce the need for labelled data and, at the same time,
automate design aspects that impact the final DNNs. In order to achieve this,
we propose a framework that performs Evolution of Deep Networks through Self
Supervision (EvoDeNSS). The code is publicly available on
GitHub111https://github.com/adrianovinhas/evodenss.

The remainder of the document is structured as follows. In Section II we
survey related works. Next, in Section III we detail the developed approach,
which is then followed by the undertaken experiments and respective results
(Section IV). To end, in Section V, conclusions are drawn and future work is
addressed.

##  II Related work

Our work intersects two main topics - 1) SSL and 2) NE. Therefore, we will
start by outlining the dynamics of this learning paradigm, followed by
reviewing NE works which attempt to learn representations without the use of
extrinsic labels.

###  II-A Self-Supervised learning (SSL)

SSL [3, 4] is a learning paradigm that uses the input data itself to create
pseudo-labels that will guide the learning process. Pseudo-labels are
intrinsically generated from the relation between the original input and a
modified version of it. This modification can come in the form of parts of the
original input, a corrupted version of the input or even different modalities
of data. The goal of SSL is to create feature extractors by predicting the
original inputs based on their modified counterparts.

SSL methods are composed of two stages. In the first one, we train a model
without relying on any labels and without explicitly taking into account our
final goal. Instead, the model is trained to solve a task related to the one
we are interested in, known as pretext task, that will learn how to extract
features based on pseudo-labels. The second stage, known as the downstream
task, transfers the pretrained model by reusing the learned features in order
to train another model that solves the actual task. This step follows a
supervised learning setting, as the features extracted from labelled inputs
are used to train the downstream task. An overview of the 2-step process in
SSL is depicted in Figure 1.

Figure 1: Overview of the SSL process for an image classification problem.

###  II-B Neuroevolution (NE)

NE [5, 6, 7] is a field that encompasses the application of Evolutionary
Algorithms (EAs) to automate design aspects of DNNs. Within the specific
context of SSL, NE literature can be divided into the evolution of generative
and contrastive methods.

Regarding generative methods, David and Greental evolved the weights of an
Auto-Encoder (AE) [8]. Their architecture is fixed, and the weights of the
encoder are tied to the decoder, allowing to substantially reduce the search
space. Lander and Shang [9] proposed a framework which allows more freedom to
the evolution process by targeting the weight and structural space. Assunção
et al. evolved AEs that are not constrained to have the same structure nor
weights on the encoder and decoder [10], aiming to compress the data without
compromising classification performance. Their fitness function is the
aggregation of several objectives into a single one via linear combination: 1)
maximise the accuracy of the learned representations in an image
classification task, 2) minimise the size of the chokepoint, 3) minimise the
number of layers in the decoder. Other works employed NE to the evolution of
Variational Auto Encoders (VAEs) [11]. For instance, Hajewski and Oliveira
evolve VAEs using only fully connected layers [12]. The architecture of their
evolved DNNs was flexible enough to vary the number of layers and the number
of neurons within a layer. Chen et al. employed a block-based search by
allowing any arrangement of layers within each block (without skip
connections) [13], but the way blocks interact which other is predefined. Four
blocks are defined in total. Fitness is calculated based on the training loss,
which may cause generalisation issues due to overfitting.

Generative Adversarial Networks (GANs) [14] can be considered as another form
of SSL, given that a generator $G$ learns representations from a pretext task
that is taught to distinguish images that come from the training data or
generated from the latent space. E-GAN [15] targeted generators in their
evolutionary process and assume that the discriminator will always achieve its
optimal performance. Generators are evolved only through mutations and
separately trained using three predefined objective functions, generating
three new GANs. Gonzalez and Miikkulainen evolved loss functions adapted to
GANs [16]. Individuals represent coefficients of Taylor expansions and are
evolved using an Evolutionary Strategies (ES) algorithm. Costa et al.
approached the evolution of GANs as a co-evolution problem [17], whereby
generators and discriminators follow a similar representation to DeepNEAT [18]
and evolved individuals using a mutation operator only. Each GAN is the result
of pairing each discriminator with each generator to calculate the fitness for
each individual. This work was later extended to use different fitness
functions. One of them is a multiobjective function that takes into account
both FID and a novelty search score that measures diversity [19]. The other
used a skill rating score [20] based on the Glicko-2 rating system [21].

Most modern SSL methods aim to learn representations by contrastive learning.
The idea is to feed a model with multiple views of the same input and
encourage it to represent contrasting versions of an input similarly, thereby
promoting multi-view invariance. Even though contrastive methods obtain state
of the art results, NE literature is very scarce in automating design aspects
learned with these methods.

In order to automate the design of the data augmentation component, Barrett et
al. aimed to evolve the augmentation policy parameters to find the one that
produces the best representations [22].

Outside the image domain, ELo [23] is a network that is trained to minimise a
joint loss composed of losses on $M$ modalities, trained on $T$ self-
supervised tasks. The final loss is divided into two components. One is the
linear combination of all losses $L_{m,t},m\in M,t\in T$. The other is
distillation loss $L_{d}$, which is the result of transferring knowledge from
one modality to the main network. Coefficients associated to each one of these
losses are evolved through an ES algorithm. Finally, Wei et al. use NE to
evolve DNN architectures that were trained using a SSL algorithm [24]. As a
fitness metric, the authors rely on a surrogate model that predicts the
performance of the individuals.

##  III Proposed approach

In this section we detail the most relevant parts of our proposed
neuroevolutionary framework. We start by describing the evolutionary engine,
focusing on how the evaluation of individuals was adapted to the SSL case.
Finally, the dataset partitioning process is specified with regards to both
SSL and supervised learning.

###  III-A Evolutionary Engine

EvoDeNSS is a neuroevolutionary framework that can be considered as an
extension of Fast-DENSER [25]. Therefore, we will describe how Fast-DENSER
works and what relevant extensions were performed to enable individuals to be
evaluated through a SSL algorithm.

Fast-DENSER is a NE framework that allows the emergence of Convolutional
Neural Networks (CNNs) using a $(1+\lambda)$-ES algorithm. The rationale
behind this decision is due to an effort to reduce the number of evaluations,
when compared to its predecessor [26]. Fast-DENSER aims to run with a more
restricted population size, hence limiting the number of evaluations needed to
execute the NE algorithm.

One of the concerns raised by the authors was the impact of this decision in
the diversity of the evolved solutions. Diversity is an important factor in
EAs, as it impacts the ability of converging towards optimal DNNs. To
compensate this possible diversity loss, weight-sharing mechanisms were not
used. Instead, at each generation one parent is selected to create its
descendants, but its weights are discarded. In the next generation, the parent
is forced to be trained from scratch, using another set of initial weights. An
overview of Fast-DENSER is depicted in Figure 2.

Figure 2: Overview of Fast-DENSER.

EvoDeNSS uses the same representation as Fast-DENSER. It is a two-level
representation which can be seen as an array of modules at the macro level
(modules can represent a layer or learning aspects), whereas at the
microscopic level, each module uses a Dynamic Structured Grammatical Evolution
(DSGE) representation [27], which allows the structure of each module to be
constrained by context-free grammars. The flexibility of this approach biases
the search space towards more favourable solutions by injecting a-priori
knowledge. In EvoDeNSS, one does not use fully-connected layers for
representation learning, meaning that we only consider layers within the
context of learning features. The details about the grammars that were used in
EvoDenSS can be checked in the public repository mentioned in Section I.
Moreover, for the learning block we included the possibility of training DNNs
using LARS optimiser [28]. This decision was made as this optimiser was chosen
by the authors of the SSL algorithm we use for evaluation purposes. More
details about this algorithm can be found on Section III-B.

###  III-B Evaluation Module

In order to evaluate each individual, the genotype needs to be converted into
a phenotype, which in this case is a DNN in a format that is trainable with a
Deep Learning framework. Given that we are focusing on an image classification
problem, we define fitness as the ability of an evolved network to correctly
classify the images it is fed with. For this effect, we compute fitness using
the accuracy on a dedicated subset, created only for this purpose.

The process to compute accuracy differs slightly depending on the chosen
learning paradigm. In supervised learning the model needs to be trained using
labelled data (subjected to a maximum training time) and then the accuracy is
measured on the dedicated test set. As for SSL, the evolved network is trained
for the pretext task with unlabelled data to learn representations, bounded by
a maximum training time too. Then, the train for the downstream task happens,
by applying a linear layer on top of the representations and feeding the
network with labelled data. This linear layer is trained with fixed parameters
for a predefined number of epochs, meaning that no aspects of the downstream
task are incorporated into evolution. In this scenario, the fitness metric is
the quality of the learned representations, which could be translated into
finding out if the evolved representations are good predictors. Therefore, we
measure the accuracy on the downstream task by feeding the dedicated test set
data. The differences in calculating fitness for each learning paradigm are
depicted in Figure 3.

(a) Fitness assignment under supervised learning

(b) Fitness assignment under SSL

Figure 3: Overview of individual evaluation under different learning
paradigms.

In order to train representations without using labels, we incorporated Barlow
Twins algorithm [29] within the proposed framework. The rationale of this
method is to build representations from pairs of image views. Learning occurs
by employing a loss function that maximises invariance and minimises the
redundancy between these representation pairs. An overview of Barlow Twins
method is depicted in Figure 4.

Figure 4: Barlow Twins algorithm

Each batch of original images from the dataset is augmented, generating two
different batches, named $Y^{A}$ and $Y^{B}$. Both $Y^{A}$ and $Y^{B}$ contain
unique image views that are created from the same set of original images.
Image views are fed to an encoder network, which is in turn attached to a
projector network. A projector network is commonly a multiple layer neural
network composed by dense and/or batch normalisation layers only. The outputs
of this process are two matrices that represent two batches of output vectors
$Z^{A}$ and $Z^{B}$. The cross-correlation matrix between $Z^{A}$ and $Z^{B}$
is calculated, and is then used to calculate the loss. The rationale is that
we want the same dimensions to be as correlated as possible, making the
representations to be invariant to transformations. At the same time different
dimensions should be untangled from each other. If they are not, two
dimensions can be considered redundant. Therefore, the goal of the
optimisation process is to approximate the cross-correlation matrix as much as
possible to the identity matrix, via Equation 1.

$L_{BT}=\underbrace{\sum_{i}(1-C_{i,i})^{2}}_{\text{invariance
term}}+\lambda\cdot\underbrace{\sum_{i}\sum_{j}C_{i,j}^{~{}~{}2}}_{\text{redundancy
reduction term}},$ (1)

In this loss function, $i$ and $j$ identify dimensions of the representation
vector, $C$ represents the cross-correlation matrix and $\lambda$ is a
parameter that sets the importance of the redundancy reduction term. It should
be noted that the representations to be used for downstream tasks are the ones
produced by the encoder, hence the projector network is only needed during the
pretext task.

Barlow Twins was chosen for these experiments due to the simplicity of its
design when compared to other algorithms. Besides, this algorithm is reported
to work well with lower batch sizes when compared to some of its peers, making
it a suitable candidate in scenarios that do not require extensive GPU memory
resources.

###  III-C Dataset Partitioning

There were two major factors taken into account while partitioning the
dataset. The first one was that even though our fitness function is based on
accuracy metric, we need to ensure that the data that is used to measure the
fitness and the final performance of the networks are disjointed. For
instance, if we were to use the same test set for fitness and performance
purposes, even though individuals would not be trained on that data, the
evolution process would still be guided by a measurement that is based on the
test data. The other one was that the data partitioning process had to be
flexible so that it could be used in supervised and self-supervised scenarios.
Given that most common current benchmark datasets come divided in train and
test sets, we used only data from the train set during the evolutionary
process, whereas the test set was only used at the end of it, to check the
performance of the best DNN in unseen data. During the evolutionary process we
divide the entire train set into three disjoint splits for training,
validation and testing purposes. From now onwards, we will name these splits
as evolutionary training set, evolutionary validation set and evolutionary
test set. In order to test scenarios in which the amount of labelled data is
scarce, we downsampled the train split further, producing an even smaller
training subset which we will name downsampled evolutionary training set. All
divisions and downsampling operations produce balanced sets, i.e., in each set
there is the same number of instances per class. The entire dataset
partitioning process is described in Figure 5.

Figure 5: Set splits created by the dataset partitioning process. Coloured
sets represent the ones used during the evolutionary process by supervised
and/or self-supervised learning, whereas the test set is used at the end to
check the final performance.

The splits that were used vary according to the learning paradigm chosen in
the evolutionary process. In the supervised learning scenario, each candidate
network is trained with the downsampled evolutionary training set. During the
train of each individual, we use an evolutionary validation set to track
network loss. This allows to set an early stopping criteria to preclude the
network from overfitting. Finally, the evolutionary test set is used for
fitness computation.

In the SSL scenario, we use the evolutionary training set to train
representations, without relying on its labels. Following previous SSL works
[29], no evolutionary validation set is used in the pretext task, as it is
harder to check if the network is overfitting. This happens because 1) the
pretext task is not the final task we are aiming to solve, and 2) the loss
produced during the pretext task is derived only from inputs and their
stochastic augmented versions (no labels are used as ground truth). After
representations are learned, the downstream task will make use of the
downsampled evolutionary training set to train a network for the image
classification problem to be solved. Finally, the evolutionary test set is
applied to the trained network in order to compute the fitness of an
individual. The splits that were used for each learning paradigm are described
in Figure 6.

Figure 6: Set splits used breakdown by task and learning paradigm

##  IV Experimentation

The evolutionary engine built for these experiments (version 2.0.0 on GitHub)
uses Python 3.9. As for the fitness assignment phase, Pytorch library was
adopted to create the corresponding phenotype models and optimisers. Models
were trained on the CIFAR-10 dataset [30] using a single NVIDIA 3080 GPU.

###  IV-A Setup

In order to validate the adopted methodology, we conducted experiments that
envisaged two goals. First, we wanted to analyse how would the evolution using
a supervised learning paradigm compare against evolution using SSL. Besides,
we wanted to understand what was the impact of evolving DNNs when the amount
of labelled data is scarce. Therefore, four different scenarios were tested.

  * •

Evolution using supervised learning and 100% of the labelled data for
training;

  * •

Evolution using supervised learning and 10% of the labelled data for training;

  * •

Evolution using Barlow Twins and 100% of the labelled data for training (in
the downstream task);

  * •

Evolution using Barlow Twins and 10% of the labelled data for training (in the
downstream task).

For each of the aforementioned scenarios, 10 different runs were executed.
Each run based on Barlow Twins took an average of 2 days to execute, whereas
each of the supervised learning runs took less than a day. The parameters used
for these experiments are described in Table I.

TABLE I: Experimental parameters. EA Parameters | Supervised | SSL
---|---|---
Number of runs | 10
Generations | 75
Population size | 6
Add layer rate | 0.25
Remove layer rate | 0.25
DSGE mutation rate | 0.15
Macro mutation rate | 0.3
Train longer | 0.03
Dataset Parameters | Supervised | SSL
Evolutionary Training Set | 70%
Evolutionary Validation Set | 20% | -
Evolutionary Test Set | 10% | 30%
Learning Parameters | Supervised | SSL
$\lambda$ | - | 0.0078125
Train time (mins) | 2
Loss Function | Cross-Entropy | Barlow Twins
Downstream epochs | - | 30
Downstream learning rate | - | 0.001
Downstream optimiser | - | Adam
Downstream loss | - | Cross-Entropy
Data Augmentation Parameters | Supervised | SSL
Padding | 4 | 4
Horizontal Flip | 0.5 | 0.5
Color Jitter (Brightness) | - | 0.4
Color Jitter (Contrast) | - | 0.4
Color Jitter (Saturation) | - | 0.4
Color Jitter (Hue) | - | 0.1
Color Jitter (Probability) | - | 0.8
Grayscale | - | 0.2

###  IV-B Results

Firstly, we will focus in analysing the ability of the NE algorithm to promote
the emergence of increasingly better solutions as well as the generalisation
capabilities of the best DNNs. The plot in Figure 7 depicts how the fitness of
the best individual changes throughout generations, averaged by the 10 runs.

Figure 7: Evolution of DNNs using the accuracy on the Evolutionary Test set as
fitness. Results are averages of 10 runs.

From this Figure, it is clear that all four test cases are able to evolve
solutions that are increasingly better. When the evolved networks learn with
supervised learning and without downsampling (Supervised 100), fitness
converges faster towards more optimal regions as the blue line reaches an
average fitness of 0.85. When compared to self-supervised runs that use Barlow
Twins, this is a significantly higher fitness value. This behaviour is
expected, as learning without labels makes the evolution task notoriously more
difficult. Although the combination that produces solutions with the highest
fitness is the Supervised 100 case, it has to be noted that when there is
downsampling, the supervised scenario (Supervised 10) seems to reach a plateau
that suggests that no significant fitness improvements will happen if the
evolution process was to be extended. Conversely, self-supervised based
evolution exhibits evolution curves that suggest that further improvements are
still possible to occur if the evolutionary runs are extended for more
generations. This is particularly evident in the case where DNNs are evolved
without downsampling using a SSL paradigm (BT 100), as the evolution curve
does not flatten at the same pace as the Supervised 100 line.

Another point worth mentioning is that SSL guided evolution seems to be more
resilient to the scarcity of labelled data than its supervised counterpart,
given that both red lines are somewhat close to each other throughout
generations, compared to the blue lines that represent supervised learning
scenarios. This suggests that the evolved solutions are able to leverage
unlabelled data to learn representations. Additionally, when the amount of
labelled data is limited, self-supervised evolution is able to reach the same
fitness levels as its supervised counterpart, despite the extra difficulty of
the task.

Besides performing an evolution analysis, we computed the accuracy of the best
individual of each run on the original test set. These values are summarised
in Table II and they show the ability of the evolved networks to generalise on
unseen data. When analysing the self-supervised runs, the best one is able to
achieve a test accuracy of 77.41%, using Barlow Twins with 100% of labelled
data. Moreover, the average performance of the best solutions evolved with SSL
was not affected by using fewer labelled samples in the downstream task, and
it also obtained comparable results to the evolution runs guided by supervised
learning using 10% of labelled data, showing how competitive SSL based
evolution can be when the labelled data is limited.

TABLE II: Test accuracies (%) of the best networks produced by each testing scenario. |  BT 10% |  Supervised 10% |  BT 100% |  Supervised 100%
---|---|---|---|---
Mean | 66.86 $\pm$ 3.53 | 66.53 $\pm$ 2.83 | 68.35 $\pm$ 5.73 | 84.67 $\pm$ 1.67
Best run | 71.63 | 69.59 | 77.41 | 86.89

The second part of our results involves a structural analysis to the best
individuals whose average fitness was depicted previously. In order to do
that, Figure 8 shows the average number of layers of the best individuals
throughout generations. In the SSL case, we consider the number of layers as
the ones that are utilised to train representations and the final layer
trained during downstream task, thus excluding the projector network.

Figure 8: Number of layers of the best individuals throughout generations.
Results are averages of 10 runs.

Although we acknowledge that a direct comparison between the total number of
layers of Barlow Twins and supervised scenarios cannot be made in a fairly
manner, we can still analyse the impact of data scarcity in each of the
learning paradigms separately. This is due to the fact that the downstream
task is not evolved (only one fixed layer is used); hence, we do not promote
the emergence of fully connected layers under the same circumstances.
Nevertheless, we can still observe that evolved networks using Barlow Twins
showed a similar number of total layers as the evolution proceeded, which can
be seen by both red lines being very close to each other during the
evolutionary process. That is not the case of the supervised scenario. The
Supervised 100 line actually starts with a very similar average number of
layers (5) than Supervised 10 (4.8). However, by the time of the last
generation, there is a discrepancy in this number, as Supervised 100 reaches
an average of 11.7 compared to 8.2 for the Supervised 10 scenario.

A deeper insight to the number of layers breakdown by type is depicted in
Figure 9. This plot shows how there is a common trend when it comes to the
number of convolutional layers. Regardless of the learning paradigm and
percentage of labelled data used, the best individuals will tend to hold an
increasingly number of convolutional layers throughout generations. This shows
the importance of convolutional layers in promoting the emergence of
increasingly better individuals.

Another detail that emerges from Figure 9 is in the number of batch
normalisation layers (green line). In Figures 9(c) and 9(d), one can see that
the green lines and blue lines (convolutional layers) grow together,
suggesting that when the evolved networks learn through supervised learning,
the number of batch normalisation layers has a trend that is correlated to the
number of convolutional layers (even more visible in Figure 9(c)). On the
other hand, if one performs the same analysis for Barlow Twins looking at
Figures 9(a) and 9(b), a different behaviour is observed (particularly in
Figure 9(a)), as the gap between the number of convolutional and batch
normalisation layers seems to grow throughout generations. This is an
interesting result, as the grammars used in supervised learning and Barlow
Twins contain the same derivation rules for the definition of convolutional
and batch normalisation layers.

(a) Barlow Twins with 10% of labelled data.

(b) Barlow Twins with 100% of labelled data.

(c) Supervised learning with 10% of labelled data.

(d) Supervised learning with 100% of labelled data.

Figure 9: Number of layers of the best individuals evolved using a different
combination of learning paradigm and % of labelled data. Results are averages
of 10 runs.

##  V Conclusions and Future Work

The time required to design DNN architectures and to prepare the necessary
datasets can be time-consuming. Moreover, in order to feed these models with
data one needs to label them, which adds extra complexity to an already
laborious task. This highlights the importance of reducing the amount of human
intervention during this process.

The current work aims to alleviate this problem by combining ideas of NE with
SSL, so that aspects of DNN architecture design can be automated while
reducing the reliance on labelled data to train these models. We developed a
neuroevolutionary framework that evolves DNNs which can learn either using
supervised learning, or through Barlow Twins algorithm. In the latter, we
leverage the unlabelled data to learn representations, and then train a single
dense layer on top of them using a percentage of the labelled data. In order
to assess the impact of the chosen learning paradigm within the NE context, we
evolve DNNs that are taught with both supervised learning and SSL paradigms.
We also vary the amount of labelled data that is fed into the evolved networks
(10% or 100%).

Our results confirm the idea that similarly to evolution guided by supervised
learning, the same can be done in a self-supervised context. However, given
the additional constraints imposed by the unavailability of labels, the
optimisation of networks is a harder task. Despite of this, evolved networks
that learn in a self-supervised manner are able to compete with their
supervised counterparts when the amount of labelled data is limited, even
outperforming them in some cases. Additionally, a structural analysis to the
evolved DNNs evidences that the ones that learned via the Barlow Twins
algorithm tend to be structurally more similar (in terms of number of layers)
regardless of the percentage of labelled data, when compared to networks that
learned in a supervised manner.

In order to better understand the impact of NE within a SSL context, there are
other components in our proposed framework which can be targeted by EC. For
instance, we plan to evolve other components like the projector network, as
they are reported to help improving the performance of the downstream task.
However, the reason why it works and its exact role are yet to be uncovered.
Chen et al. concluded that using the layer before the projector as
representations offered better downstream performance [31], whereas Ma et al.
suggest that the projector seeks uniformity by maximising the distance between
dissimilar samples [32]. Similarly, we plan to evolve data augmentation
aspects within Barlow Twins algorithm. They play a vital role during the
pretext task by setting its difficulty to be solved. The network used during
the downstream task can also be evolved to help the EA to converge towards
more optimal solutions. On a different line of thought, we believe that
modifying the search space by adopting a cell-based approach could benefit
both the quality of the evolved networks and the time that takes to evolve
them.

## Acknowledgments

This work is funded by the FCT - Foundation for Science and Technology,
I.P./MCTES through national funds (PIDDAC), within the scope of CISUC R&D Unit
- UIDB/00326/2020 or project code UIDP/00326/2020

## References

  * [1] P. C. Quinn, P. D. Eimas, and S. L. Rosenkrantz, “Evidence for representations of perceptually similar natural categories by 3-month-old and 4-month-old infants,” Perception, vol. 22, no. 4, pp. 463–475, 1993.
  * [2] E. Orhan, V. Gupta, and B. M. Lake, “Self-supervised learning through the eyes of a child,” Advances in Neural Information Processing Systems, vol. 33, pp. 9960–9971, 2020.
  * [3] V. R. de Sa, “Learning classification with unlabeled data,” Advances in neural information processing systems, pp. 112–112, 1994.
  * [4] C. Doersch, A. Gupta, and A. A. Efros, “Unsupervised visual representation learning by context prediction,” in Proceedings of the IEEE international conference on computer vision, pp. 1422–1430, 2015.
  * [5] D. Floreano, P. Dürr, and C. Mattiussi, “Neuroevolution: from architectures to learning,” Evolutionary intelligence, vol. 1, pp. 47–62, 2008.
  * [6] D. J. Montana, L. Davis, et al., “Training feedforward neural networks using genetic algorithms.,” in IJCAI, vol. 89, pp. 762–767, 1989.
  * [7] S. A. Harp and T. Samad, “Genetic optimization of self-organizing feature maps,” in IJCNN-91-Seattle International Joint Conference on Neural Networks, vol. 1, pp. 341–346, IEEE, 1991.
  * [8] O. E. David and I. Greental, “Genetic algorithms for evolving deep neural networks,” in Proceedings of the Companion Publication of the 2014 Annual Conference on Genetic and Evolutionary Computation, pp. 1451–1452, 2014.
  * [9] S. Lander and Y. Shang, “Evoae–a new evolutionary method for training autoencoders for deep learning networks,” in 2015 IEEE 39th annual computer software and applications conference, vol. 2, pp. 790–795, IEEE, 2015.
  * [10] F. Assuncao, D. Sereno, N. Lourenco, P. Machado, and B. Ribeiro, “Automatic evolution of autoencoders for compressed representations,” in 2018 IEEE Congress on Evolutionary Computation (CEC), pp. 1–8, IEEE, 2018.
  * [11] D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” in 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings (Y. Bengio and Y. LeCun, eds.), 2014.
  * [12] J. Hajewski and S. Oliveira, “An evolutionary approach to variational autoencoders,” in 2020 10th Annual Computing and Communication Workshop and Conference (CCWC), pp. 0071–0077, IEEE, 2020.
  * [13] X. Chen, Y. Sun, M. Zhang, and D. Peng, “Evolving deep convolutional variational autoencoders for image classification,” IEEE Transactions on Evolutionary Computation, vol. 25, no. 5, pp. 815–829, 2020.
  * [14] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in Neural Information Processing Systems (Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Weinberger, eds.), vol. 27, Curran Associates, Inc., 2014.
  * [15] C. Wang, C. Xu, X. Yao, and D. Tao, “Evolutionary generative adversarial networks,” IEEE Transactions on Evolutionary Computation, vol. 23, no. 6, pp. 921–934, 2019.
  * [16] S. Gonzalez, M. Kant, and R. Miikkulainen, “Evolving gan formulations for higher quality image synthesis,” arXiv preprint arXiv:2102.08578, 2021.
  * [17] V. Costa, N. Lourenço, and P. Machado, “Coevolution of generative adversarial networks,” in Applications of Evolutionary Computation: 22nd International Conference, EvoApplications 2019, Held as Part of EvoStar 2019, Leipzig, Germany, April 24–26, 2019, Proceedings 22, pp. 473–487, Springer, 2019.
  * [18] R. Miikkulainen, J. Liang, E. Meyerson, A. Rawal, D. Fink, O. Francon, B. Raju, H. Shahrzad, A. Navruzyan, N. Duffy, and B. Hodjat, “Evolving deep neural networks,” in Artificial Intelligence in the Age of Neural Networks and Brain Computing (R. Kozma, C. Alippi, Y. Choe, and F. C. Morabito, eds.), pp. 293–312, Amsterdam: Elsevier, 2019.
  * [19] V. Costa, N. Lourenço, J. Correia, and P. Machado, “Exploring the evolution of gans through quality diversity,” in Proceedings of the 2020 Genetic and Evolutionary Computation Conference, pp. 297–305, 2020.
  * [20] V. Costa, N. Lourenço, J. Correia, and P. Machado, “Using skill rating as fitness on the evolution of gans,” in Applications of Evolutionary Computation: 23rd European Conference, EvoApplications 2020, Held as Part of EvoStar 2020, Seville, Spain, April 15–17, 2020, Proceedings 23, pp. 562–577, Springer, 2020.
  * [21] M. E. Glickman, “Example of the glicko-2 system,” Boston University, vol. 28, 2012.
  * [22] N. Barrett, Z. Sadeghi, and S. Matwin, “Evolutionary augmentation policy optimization for self-supervised learning,” arXiv preprint arXiv:2303.01584, 2023.
  * [23] A. Piergiovanni, A. Angelova, and M. S. Ryoo, “Evolving losses for unsupervised video representation learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 133–142, 2020.
  * [24] C. Wei, Y. Tang, C. N. C. Niu, H. Hu, Y. Wang, and J. Liang, “Self-supervised representation learning for evolutionary neural architecture search,” IEEE Computational Intelligence Magazine, vol. 16, no. 3, pp. 33–49, 2021.
  * [25] F. Assunção, N. Lourenço, P. Machado, and B. Ribeiro, “Fast denser: Efficient deep neuroevolution,” in Genetic Programming (L. Sekanina, T. Hu, N. Lourenço, H. Richter, and P. García-Sánchez, eds.), (Cham), Springer International Publishing, 2019.
  * [26] F. Assunçao, N. Lourenço, P. Machado, and B. Ribeiro, “Denser: deep evolutionary network structured representation,” Genetic Programming and Evolvable Machines, vol. 20, pp. 5–35, 2019.
  * [27] N. Lourenço, F. Assunção, F. B. Pereira, E. Costa, and P. Machado, “Structured grammatical evolution: a dynamic approach,” Handbook of Grammatical Evolution, pp. 137–161, 2018.
  * [28] Y. You, I. Gitman, and B. Ginsburg, “Large batch training of convolutional networks,” arXiv preprint arXiv:1708.03888, 2017.
  * [29] J. Zbontar, L. Jing, I. Misra, Y. LeCun, and S. Deny, “Barlow twins: Self-supervised learning via redundancy reduction,” in International Conference on Machine Learning, pp. 12310–12320, PMLR, 2021.
  * [30] A. Krizhevsky, G. Hinton, et al., “Learning multiple layers of features from tiny images,” 2009.
  * [31] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in International conference on machine learning, pp. 1597–1607, PMLR, 2020.
  * [32] J. Ma, T. Hu, and W. Wang, “Deciphering the projection head: Representation evaluation self-supervised learning,” arXiv preprint arXiv:2301.12189, 2023.