#  Enhancing Robot Learning through Learned Human-Attention Feature Maps

Daniel Scheuchenstuhl  CPS Group
TU Wien
Wien, Austria
0009-0000-6080-2898  Stefan Ulmer  CPS Group
TU Wien
Wien, Austria
0009-0009-7654-7743  Felix Resch  CPS Group
TU Wien
Wien, Austria
0009-0004-3240-3725  Luigi Berducci  CPS Group
TU Wien
Wien, Austria
0000-0002-3497-6007  Radu Grosu  *Indicates authors with equal contributions
CPS Group
TU Wien
Wien, Austria
0000-0001-5715-2142

###### Abstract

Robust and efficient learning remains a challenging problem in robotics, in
particular with complex visual inputs. Inspired by human attention mechanism,
with which we quickly process complex visual scenes and react to changes in
the environment, we think that embedding auxiliary information about focus
point into robot learning would enhance efficiency and robustness of the
learning process. In this paper, we propose a novel approach to model and
emulate the human attention with an approximate prediction model. We then
leverage this output and feed it as a structured auxiliary feature map into
downstream learning tasks. We validate this idea by learning a prediction
model from human-gaze recordings of manual driving in the real world. We test
our approach on two learning tasks - object detection and imitation learning.
Our experiments demonstrate that the inclusion of predicted human attention
leads to improved robustness of the trained models to out-of-distribution
samples and faster learning in low-data regime settings. Our work highlights
the potential of incorporating structured auxiliary information in
representation learning for robotics and opens up new avenues for research in
this direction. All code and data are available online111Code/Data at
https://github.com/CPS-TUWien/learning_human_attention

###### Index Terms:

Robot learning, Human attention

##  I Introduction

Robot learning has seen significant progress in recent years, developing
systems able to perform increasingly complex tasks in a variety of challenging
environments [1, 2]. However, the performance of the learning process often
depends on the quality of representations, which retain the important features
extracted from high-dimensional sensor data. Effective representation learning
is therefore crucial for achieving high performance in robot learning tasks,
and an increasing effort has been invested into this fundamental research area
[3, 4, 5].

Most of the modern approaches to representation learning build on self-
supervised learning and generative models [6, 7, 8], and have shown promising
results in learning effective representations reusable in many different
downstream tasks. However, we believe there is still much to be gained by
taking inspiration from human behavior. Humans are able to process rich visual
scenes and perform complex visual-motor tasks with great efficiency, due in
part to the sophisticated attention mechanisms we employ [9]. Attention allows
us to capture the most important features to accurately perform a task.

In this work, we build on this insight by developing a new model that mimics
the human attention. However, instead of learning representations retaining
the most salient features, we propose to enrich the input data with
attentional feature maps.

To validate this idea, we collect real-world data of human gaze and train a
model to distill human attention maps from a sequence of visual inputs. Having
a model able to accurately predict attention maps, we can use them in unseen
frame sequences, without any human in the loop.

Overall, our work proposes the following contributions:

  * •

A novel model to predict human attention maps from visual input, trained on
real-world data of human gaze collected during manual driving in scaled
miniature cars.

  * •

The integration of the learned representations based on human attention into
two downstream robot-learning tasks of object detection and imitation
learning.

  * •

Experimental evaluation of the effectiveness of our approach by comparing it
to existing methods that do not incorporate human-attention features.

In the rest of this paper, we will explain the proposed methodology and
experimental results.

Figure 1:  Hardware setup to record human-gaze while manually driving a
miniature racecar. It consists of a eye-tracking system, remote control of the
vehicle and data acquisition system.

##  II Related Works

In this section, we include the works related to our contributions.
Considering our application in the context of autonomous racing, we include a
review of the existing work of learning-based approaches used in autonomous
racing.

Attention models. Research on human attention dates back several decades and
has been extensively studied due to its significant impact on learning and
perception. In [9], the authors discuss the fundamental mechanism of attention
and its advantageous effects on learning of humans. More recently, the notion
of focus or attention has been adopted by the machine learning community [10],
leading to many successful applications. Among them, notable mention is the
self-attention mechanism adopted by transformer in natural language processing
[11], emotion detection [12], or image recognition [13]. However, the
connection between artificial attention mechanism commonly adopted in modern
neural architectures and the human or biological attention is not clear. This
motivates a new effort in research to explore how the two are related [14,
15]. In this direction, our contributions try to demonstrate the practical
usability of human-based features.

Representation Learning. In recent years, there has been significant progress
in the field of representation learning [16]. Most of the modern approaches
rely on self-supervised learning and generative models [6, 7, 8], using
variants of auto-encoders [17, 18] to learn a low-dimensional latent
representation. The applications range from computer vision [19, 20], natural
language [21], or multimodal inputs [22, 23]. Compared with these works, we do
not use attention to provide a compact representation of the input, but
instead propose to enrich it with additional attentional features.

Autonomous Racing. Considering our robotics experiments have been framed in
the context of autonomous racing, we provide a review of existing approaches
with focus on learning applications. A complete overview of this research
field is provided in [24]. Despite the wide use of learning-based approaches
for perception, even in racing competitions [25, 26, 27], most of the planning
approaches currently deployed on hardware cars use either traditional control,
such as model predictive control (MPC), or analytical approaches [28, 29].
However, an increase effort on learning-based control raised in the last
years. Some works use learned models in conjunction with MPC [30], in an end-
to-end fashion with imitation or reinforcement learning [31, 32, 33, 34, 35],
or in a hierarchical framework [36, 37, 38] where a deep model generates
trajectories that are then tracked with a low-level controller. In [39], MPC
is combined with a novel attention mechanism that uses the generated
trajectory to identify a region of interest on images.

##  III Human-Attention Model

We first formalize the problem of reproducing the human-attention mechanism.
Considering input images $x_{t}\in\mathbb{R}^{m\times n}$, we define an
attention map $y_{t}\in\mathbb{R}^{m\times n}$ retaining all the focus points
obtained by recording the human gaze.

Figure 2:  The proposed human-attention model is trained to distill attention
feature maps from visual input and learn the most-relevant focus area.

The human-attention model $M$ is then defined as

$y_{t}=M(x_{t})$

Considering the sequential nature of the human-attention mechanism, which
focuses on different areas of the image in a sequential way, and the limited
capabilities of the recording system, which is able to capture up to a fixed
number of points simultaneously, we propose an approximation of the attention
maps with an exponential-decay time processing. Let
$x_{t}\in\mathbb{R}^{m\times n}$ be the frame captured at time $t$,
$P_{t}=\\{p_{j,t}\\}_{j}$ the set of focus points tracked with our system, and
$h_{t}\in\mathbb{R}^{m\times n}$ the heatmap computed by centering a Gaussian
distribution over each focus point $p\in P_{t}$. Since $h_{t}$ only captures
the instantaneous attention map of time step $t$, we aggregate it to produce
$y_{t}$ which resembles the complete attention focus as:

$\displaystyle y_{0}$ $\displaystyle=h_{0}$ $\displaystyle y_{t}$
$\displaystyle=max(h_{t},(1-r)y_{t-1})$

where $max$ refers to the pixel-wise max operation. In practice, we use
$r=0.17$ to cover all the focus point in the last second of recording,
according to the frequency of our system. We normalize the attention maps to
have likelihood values between $0$ and $255$.

Having formulated the input and output to model the human-attention mechanism,
we now introduce the learning of an attention predictor that serves to exploit
the use of attention beyond the data-collection setup. We frame the problem of
learning an attention predictor as an image restoration problem which aims to
produce approximated attention heatmaps. We build on top of the state-of-the-
art model U-Net [40] and use a smooth L1 loss, as commonly adopted for image
segmentation and related application domains. Specifically, we produce each
attention map by centering an isotropic 2D Gaussian distribution with a zero
mean and unit standard deviation over each focus point, and perform the
aggregation described above. The use of Gaussian distribution is a smooth
representation which resembles the likelihood of a driver’s attendance to that
location. The main advantage of it is to have a continuous output space where
adjacent pixels are not independent.

Hardware Setup. In this section, we will provide a detailed description of the
hardware setup used to collect a dataset of human attention while driving a
miniature race car on various tracks. Since the primary objective of this
experiment was to analyze human attention and behavior while driving, we
mounted a camera on the car and asked human drivers to control the car through
it by using either a steering wheel and pedals, or a game controller. To
ensure that the driver’s focus was entirely on the video stream captured by
the camera and rely solely on it to control the car, we seated them away from
the track.

To record the driver’s gaze, we used a VPS 19 System eye-tracker. The eye-
tracker recorded the driver’s gaze points, which were then transformed into
the camera’s frame. All the gaze points outside the camera frame were
discarded, to filter out when the drivers moved their head or looked around.

The miniature vehicle used in this experiment was an F1Tenth race car equipped
with an inertial measurement unit, a 2D LiDAR, and an RGB camera. The LiDAR
Hokuyo 10LX could sense distances up to $10$ meters in a $270$-degree field of
view at a frequency of $40$ Hz. To capture a diverse set of images, we used
two models of RGB camera, either a Logitech C930e or an Intel RealSense d435i.
Both cameras had a resolution of $848\times 480$ pixels and a frequency of
$30$ FPS. All the computation to control the car was carried out on a NVIDIA
Jetson Xavier NX board. The board had six cores and a NVIDIA Volta GPU, making
it powerful enough to run the deep models for inference later on. During the
experiment, all the gaze data, sensor data, and video streams were recorded
and synchronized with the first-person video for further processing.

Model training. The human-attention model introduced in the previous setting
has been trained on a Tesla T4 GPU with 16 GiB VRAM for 60 epochs using a
batch size of 16 and four workers. We compared various network architectures
for semantic segmentation, like Attention U-Net [41], U-Net++ [42], Path
Aggregation Network (PAN) [43] and DeepLabV3+ [44]. Based on the validation
results, U-Net++ with Concurrent Spatial and Channel Excitation (scSE) [45]
performed the best. During preprocessing, the upper third of the input frames
is discarded to bias the learning to the track events. We use horizontal image
flipping and colorspace augmentation, changing brightness, saturation and hue.
The human-attention model is not biased towards out-of-distribution
illumination perturbations, as the magnitude of change applied for colorspace
augmentation is equal to YOLOv7. The optimization is carried on with AdamW
optimizer with a Smooth L1 Loss and the hyperparameters tuned with PyHopper
[46]. We finally select an initial learning rate of $3e^{-4}$ and weight decay
of $5e^{-5}$, and apply automatic mixed precision and learning rate
scheduling.

##  IV Experiments with the Human-Attention Model

In the following, we describe the experiments to demonstrate the benefit of
integrating human attention in autonomous racing tasks, respectively in object
detection and imitation learning.

Object Detection. Considering object detection in the context of F1Tenth
autonomous racing, we consider the detection of the most-common static and
dynamic obstacles, respectively boxes and other cars. We collect a training
dataset consisting of these object classes, and compare the performance of the
state-of-the-art YOLOv7 [47] trained with the predicted attention (Att-YOLOv7)
and without it (YOLOv7). While YOLOv7 expects a 3-channel input, we modified
the Attention-YOLOv7 to additionally receive the attention map, resulting in
4-channels. We trained the models with SGD using learning rate $1e^{-2}$,
momentum $0.937$, and weight decay $5e^{-4}$.

To analyze the robustness of the trained model to out-of-distribution samples,
we evaluate them on images with heavy-perturbed brightness, changing it
between $75\%$ and $185\%$ of the original value. We consider brightness
perturbations because vision models are sensitive to changes in illumination.
While the data augmentation techniques applied during training (i.e.,
horizontal flipping, image translation $\pm 0.2$, scaling $\pm 0.5$, HSV,
mosaic) make the model invariant to many transformations, we still observe
high sensitivity to the perturbed inputs. To assess the impact of various
augmentations, we conduct an ablation study on them and report the results for
Mosaic in Figure 3. We choose Mosaic because it was the agumentation technique
with largest impact on the model performance. For each of the two models, the
performance of training with and without Mosaic ($\pm$Mosaic) is reported as
mAP, a standard metric for this application.

Figure 3:  Robustness evaluation under different brightness perturbations. The
horizontal axis shows the various models and ablations, and the vertical axis
reports the performance in term of mAP@.5:.95 score.

We observe that the introduction of predicted attention as additional input
channel produces more robust models to brightness perturbations. The
performance for models trained with Mosaic are equally good, but the gap is
evident when trained without it. In particular, YOLO-v7 shows a drop of $10\%$
and $25\%$ for perturbations of $185\%$ and $75\%$ respectively. Conversely,
the model trained with attention exhibits the same level of performance for
all the perturbations. This result suggests that feeding human attention as
additional input facilitates learning robust representations for object
detection.

Imitation Learning. We consider the task of imitating the expert driver in
controlling the F1Tenth racecar. The agent is an end-to-end model which
receives RGB-images and predicts the driving commands for steering and
velocity.

To experiment with a different integration of human attention in the input, we
mark the attention points in the image, without adding it as additional
channel. In this experiment, we use the recorded human attention, and give the
marked input to the agent model which encodes it into an embedding with
ResNet18. The representation is then feed into a series of fully-connected
layers which predict the driving commands. We compare the agent model trained
with and without attention and report the performance in term of prediction
error. Figure 4 shows the evaluation for different training budgets, expressed
as a fraction of the available training samples.

We observe a positive impact of marking the input frames with attention
points, especially in low-data regimes. In fact, while using a large amount of
data makes the performance comparable and the introduction of attention does
not degraded the predictions, it results impactful when the training data are
scarce. This result shows that the use of human attention has the potential
for more-efficient learning.

Figure 4:  Evaluation of prediction error for imitation learning under
different training budgets ($1.0$=all available $16\,000$ samples). We report
the average mean squared error over the last $50$ epochs with relative error
margins for the agent trained with (Agent+Att) and without (Agent-Att)
attention features.

##  V Conclusions

In this study, we investigated the impact of human attention on enhancing
robot learning in terms of both robustness and efficiency. To achieve this
goal, we developed a data-collection pipeline to record data from human
interaction in a driving task and propose a novel method to approximate human
attention using a prediction model. This information is then used to enrich
the visual input with attentional feature maps. We assess the impact in two
specific learning tasks. In the object detection task, we observed a
significant improvement in robustness to out-of-distribution samples when
using attention data. In the imitation learning task, we observed lower
prediction error and faster convergence, especially in low-data regimes.

These promising results highlight the potential of integrating human-based
data into machine learning pipelines to improve robot learning. We intend to
continue investigating this approach in future work to further understand its
potential and explore other possible way to integrate human-based features
into learning models. Overall, our findings suggest that the integration of
human attention data can enhance the robustness and efficiency of machine
learning models, which has significant implications for a range of real-world
applications.

## Acknowledgment

Luigi Berducci was supported by the Doctoral College Resilient Embedded System
of TU Wien Informatics. Felix Resch was supported by the Horizon Europe
Research and Innovation programme (Grant Agreement 101070679).

## References

  * [1] O. M. Andrychowicz, B. Baker, M. Chociej, R. Jozefowicz, B. McGrew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray _et al._ , “Learning dexterous in-hand manipulation,” _The International Journal of Robotics Research_ , vol. 39, no. 1, pp. 3–20, 2020.
  * [2] S. Levine, P. Pastor, A. Krizhevsky, J. Ibarz, and D. Quillen, “Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection,” _The International journal of robotics research_ , vol. 37, no. 4-5, pp. 421–436, 2018.
  * [3] Y. Bengio, A. Courville, and P. Vincent, “Representation learning: A review and new perspectives,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 35, no. 8, pp. 1798–1828, 2013.
  * [4] V. Blukis, C. Paxton, D. Fox, A. Garg, and Y. Artzi, “A persistent spatial semantic representation for high-level natural language instruction execution,” in _Conference on Robot Learning_. PMLR, 2022, pp. 706–717.
  * [5] N. Heravi, A. Wahid, C. Lynch, P. Florence, T. Armstrong, J. Tompson, P. Sermanet, J. Bohg, and D. Dwibedi, “Visuomotor control in multi-object scenes using object-aware representations,” _arXiv preprint arXiv:2205.06333_ , 2022.
  * [6] J. Pari, N. M. Shafiullah, S. P. Arunachalam, and L. Pinto, “The surprising effectiveness of representation learning for visual imitation,” _arXiv preprint arXiv:2112.01511_ , 2021.
  * [7] D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” _arXiv preprint arXiv:1312.6114_ , 2013.
  * [8] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,” _Communications of the ACM_ , vol. 63, no. 11, pp. 139–144, 2020.
  * [9] A. Johnson and R. W. Proctor, _Attention: Theory and practice_. Sage, 2004.
  * [10] D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” _arXiv preprint arXiv:1409.0473_ , 2014\.
  * [11] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” p. 11.
  * [12] Z. Peng, X. Li, Z. Zhu, M. Unoki, J. Dang, and M. Akagi, “Speech emotion recognition using 3d convolutions and attention-based sliding recurrent networks with auditory front-ends,” vol. 8, pp. 16 560–16 572, conference Name: IEEE Access.
  * [13] H. Zhao, J. Jia, and V. Koltun, “Exploring self-attention for image recognition,” in _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE, pp. 10 073–10 082. [Online]. Available: https://ieeexplore.ieee.org/document/9156532/
  * [14] G. W. Lindsay, “Attention in psychology, neuroscience, and machine learning,” vol. 14, p. 29. [Online]. Available: https://www.frontiersin.org/articles/10.3389/fncom.2020.00029/full
  * [15] G. W. Lindsay, D. B. Rubin, and K. D. Miller, “A unified circuit model of attention: Neural and behavioral effects.” [Online]. Available: http://biorxiv.org/lookup/doi/10.1101/2019.12.13.875534
  * [16] Y. Bengio, A. Courville, and P. Vincent, “Representation learning: A review and new perspectives,” vol. 35, no. 8, pp. 1798–1828, conference Name: IEEE Transactions on Pattern Analysis and Machine Intelligence.
  * [17] Y. Wang, H. Yao, and S. Zhao, “Auto-encoder based dimensionality reduction,” vol. 184, pp. 232–242. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0925231215017671
  * [18] C. Zhang, C. Zhang, J. Song, J. S. K. Yi, K. Zhang, and I. S. Kweon, “A survey on masked autoencoder for self-supervised learning in vision and beyond.” [Online]. Available: http://arxiv.org/abs/2208.00173
  * [19] M. Chen, A. Radford, R. Child, J. Wu, H. Jun, D. Luan, and I. Sutskever, “Generative pretraining from pixels,” in _Proceedings of the 37th International Conference on Machine Learning_. PMLR, pp. 1691–1703, ISSN: 2640-3498. [Online]. Available: https://proceedings.mlr.press/v119/chen20s.html
  * [20] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale.” [Online]. Available: http://arxiv.org/abs/2010.11929
  * [21] A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever, “Improving language understanding by generative pre-training.”
  * [22] H. Bao, L. Dong, S. Piao, and F. Wei, “BEiT: BERT pre-training of image transformers.” [Online]. Available: http://arxiv.org/abs/2106.08254
  * [23] K. He, X. Chen, S. Xie, Y. Li, P. Dollar, and R. Girshick, “Masked autoencoders are scalable vision learners,” pp. 16 000–16 009.
  * [24] J. Betz, H. Zheng, A. Liniger, U. Rosolia, P. Karle, M. Behl, V. Krovi, and R. Mangharam, “Autonomous vehicles on the edge: A survey on autonomous vehicle racing,” vol. 3, pp. 458–488. [Online]. Available: http://arxiv.org/abs/2202.07008
  * [25] A. Dhall, D. Dai, and L. Van Gool, “Real-time 3d traffic cone detection for autonomous driving,” in _2019 IEEE Intelligent Vehicles Symposium (IV)_ , pp. 494–501, ISSN: 2642-7214.
  * [26] K. Strobel, S. Zhu, R. Chang, and S. Koppula, “Accurate, low-latency visual perception for autonomous racing: Challenges, mechanisms, and practical solutions,” in _2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pp. 1969–1975, ISSN: 2153-0866.
  * [27] N. De Rita, A. Aimar, and T. Delbruck, “CNN-based object detection on low precision hardware: Racing car case study,” in _2019 IEEE Intelligent Vehicles Symposium (IV)_ , pp. 647–652, ISSN: 2642-7214.
  * [28] R. C. Coulter, “Implementation of the pure pursuit path tracking algorithm,” Carnegie-Mellon UNIV Pittsburgh PA Robotics INST, Tech. Rep., 1992.
  * [29] V. Sezer and M. Gokasan, “A novel obstacle avoidance algorithm:“follow the gap method”,” _Robotics and Autonomous Systems_ , vol. 60, no. 9, pp. 1123–1134, 2012.
  * [30] U. Rosolia and F. Borrelli, “Learning model predictive control for iterative tasks. a data-driven control framework,” _IEEE Transactions on Automatic Control_ , vol. 63, no. 7, pp. 1883–1896, 2017.
  * [31] K. Lee, Z. Wang, B. Vlahov, H. Brar, and E. A. Theodorou, “Ensemble bayesian decision making with redundant deep perceptual control policies,” in _2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA)_ , pp. 831–837.
  * [32] Y. Pan, C.-A. Cheng, K. Saigol, K. Lee, X. Yan, E. Theodorou, and B. Boots, “Agile autonomous driving using end-to-end deep imitation learning,” in _Robotics: Science and Systems XIV_. Robotics: Science and Systems Foundation. [Online]. Available: http://www.roboticsproceedings.org/rss14/p56.pdf
  * [33] A. Brunnbauer, L. Berducci, A. Brandstátter, M. Lechner, R. Hasani, D. Rus, and R. Grosu, “Latent imagination facilitates zero-shot transfer in autonomous racing,” in _2022 International Conference on Robotics and Automation (ICRA)_. IEEE, 2022, pp. 7513–7520.
  * [34] P. Cai, H. Wang, H. Huang, Y. Liu, and M. Liu, “Vision-based autonomous car racing using deep imitative reinforcement learning,” vol. 6, no. 4, pp. 7262–7269, conference Name: IEEE Robotics and Automation Letters.
  * [35] L. Berducci, E. A. Aguilar, D. Ničković, and R. Grosu, “Hierarchical potential-based reward shaping from task specifications,” _arXiv preprint arXiv:2110.02792_ , 2021.
  * [36] T. Weiss and M. Behl, “Deepracing: A framework for autonomous racing,” in _2020 Design, automation & test in Europe conference & exhibition (DATE)_. IEEE, 2020, pp. 1163–1168.
  * [37] S. N. Wadekar, B. J. Schwartz, S. S. Kannan, M. Mar, R. K. Manna, V. Chellapandi, D. J. Gonzalez, and A. E. Gamal, “Towards end-to-end deep learning for autonomous racing: On data collection and a uniﬁed architecture for steering and throttle prediction.”
  * [38] Y. Mahmoud, Y. Okuyama, T. Fukuchi, T. Kosuke, and I. Ando, “Optimizing deep-neural-network-driven autonomous race car using image scaling,” vol. 77, p. 04002. [Online]. Available: https://www.shs-conferences.org/10.1051/shsconf/20207704002
  * [39] K. Lee and V. Zakharov, “Perceptual attention-based predictive control.”
  * [40] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” _CoRR_ , vol. abs/1505.04597, 2015. [Online]. Available: http://arxiv.org/abs/1505.04597
  * [41] O. Oktay, J. Schlemper, L. L. Folgoc, M. C. H. Lee, M. P. Heinrich, K. Misawa, K. Mori, S. G. McDonagh, N. Y. Hammerla, B. Kainz, B. Glocker, and D. Rueckert, “Attention u-net: Learning where to look for the pancreas,” _CoRR_ , vol. abs/1804.03999, 2018. [Online]. Available: http://arxiv.org/abs/1804.03999
  * [42] Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh, and J. Liang, “Unet++: A nested u-net architecture for medical image segmentation,” _CoRR_ , vol. abs/1807.10165, 2018. [Online]. Available: http://arxiv.org/abs/1807.10165
  * [43] S. Liu, L. Qi, H. Qin, J. Shi, and J. Jia, “Path aggregation network for instance segmentation,” 2018.
  * [44] L. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoder-decoder with atrous separable convolution for semantic image segmentation,” _CoRR_ , vol. abs/1802.02611, 2018. [Online]. Available: http://arxiv.org/abs/1802.02611
  * [45] A. G. Roy, N. Navab, and C. Wachinger, “Concurrent spatial and channel squeeze & excitation in fully convolutional networks,” _CoRR_ , vol. abs/1803.02579, 2018. [Online]. Available: http://arxiv.org/abs/1803.02579
  * [46] M. Lechner, R. Hasani, P. Neubauer, S. Neubauer, and D. Rus, “Pyhopper – hyperparameter optimization,” 2022. [Online]. Available: https://arxiv.org/abs/2210.04728
  * [47] C.-Y. Wang, A. Bochkovskiy, and H.-Y. M. Liao, “YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors,” 2022\. [Online]. Available: https://github.com/WongKinYiu/yolov7


# Lessons from the Clustering Analysis of a Search Space: A Centroid-based
Approach to Initializing NAS

Kalifou Rene Traore1,2 Andrés Camero1 Xiao Xiang Zhu1,2 1German Aerospace
Center (DLR), Remote Sensing Technology Institute (IMF), Germany
2Technical University of Munich, Data Science in Earth Observation, Germany
{kalifou.traore, andres.camerounzueta<EMAIL_ADDRESS>
###### Abstract

Lots of effort in neural architecture search (NAS) research has been dedicated
to algorithmic development, aiming at designing more efficient and less costly
methods. Nonetheless, the investigation of the initialization of these
techniques remain scare, and currently most NAS methodologies rely on
stochastic initialization procedures, because acquiring information prior to
search is costly. However, the recent availability of NAS benchmarks have
enabled low computational resources prototyping. In this study, we propose to
accelerate a NAS algorithm using a data-driven initialization technique,
leveraging the availability of NAS benchmarks. Particularly, we proposed a
two-step methodology. First, a calibrated clustering analysis of the search
space is performed. Second, the centroids are extracted and used to initialize
a NAS algorithm. We tested our proposal using Aging Evolution, an evolutionary
algorithm, on NAS-bench-101. The results show that, compared to a random
initialization, a faster convergence and a better performance of the final
solution is achieved.

##  1 Introduction

Recently, significant effort has been made to improve the reproducibility in
Neural Architecture Search (NAS), and to democratize its research Elsken et
al. (2019). In this context, the release of performance evaluation databases
Ying et al. (2019); Dong and Yang (2020) has enabled low computational cost
algorithm benchmark. However, so far, few authors have take advantage of these
sources of data to improve the design of NAS algorithms.

This study sets out to answer the following question: can we improve the
performance of a population based NAS algorithm by initializing its population
with a data-driven approach? To this problem, we propose a two-step approach.
First, a tailored clustering analysis of a target search space is performed.
Second, after obtaining satisfying quantitative clustering results, the
centroids are extracted and used to initialize a population based NAS
algorithm.

To validate our proposal we selected a state-of-the-art evolutionary
algorithm, Aging Evolution Real et al. (2019), and we tested the proposed
initialization on NAS-bench-101 Ying et al. (2019). The results show that,
compared to random initialization, our approach improves the convergence and
the performance of the final solution (for 36 epochs), using a compact feature
representation.

The remainder of this article is as follows: The following section introduces
NAS and Aging Evolution Real et al. (2019). Section 3 describes the proposed
methodology. Section 4 introduces the experimental setup. Section 5 presents
the results. Finally, Section 6 outlines the conclusions.

##  2 Related Work

The following Section is dedicated to introducing the general topic of NAS, as
well as _Aging Evolution_ , one of the early evolutionary NAS algorithms
reaching state-of-the-art performances on modern _computer vision_ (CV)
benchmarks.

###  2.1 Neural Architecture Search

NAS is a subfield of Automated Machine Learning (AutoML) Hutter et al. (2019).
It aims at finding architectures providing the best performances on unseen
data for a given task to learn and provided data set Elsken et al. (2019). NAS
has seen successes in various CV tasks including image classification and
object detection; but also other areas of application such as Natural Language
Processing (NLP). Early works on NAS were using Evolutionary-based approaches
to optimize the model (i.e., the architecture, weights, and/or nodes) Ojha et
al. (2017). More recently, the scope of the search algorithms used has been
extended, including such as Bayesian Optimization Camero et al. (2021) and
Differentiable Search Algorithms Hanxiao et al. (2019).

###  2.2 Aging Evolution

In recent years, an Evolutionary Algorithm based approach to NAS achieved
state-of-the-art performance on classical CV benchmarks: the _Aging Evolution_
Real et al. (2019), outlined in Algorithm 1.

This particular baseline 111The implementation of _Aging Evolution_ is
available on NASBench-101 repository. evolves a population of constant size
$P$ and returns a highest performing solution after $C$ generations. The
performance of each candidate solution is evaluated using the _fitness_
function (Train_and_eval). Particularly, the fitness is computed by training a
solution using Stochastic Gradient Descent (SGD), and evaluating its
classification accuracy (Top-1 in validation and test). First, a population is
randomly initialized and its fitness is computed. Then, the population is
evolved until the number of evaluated generations is greater than $C$, by a
tournament selection, mutate operation, and aging replacement. The tournament
selection of size $S$ consists of randomly selecting $S$ solutions (with
replacement) from the current population. Then, the highest performance
solution is selected (_parent_). Later, the _parent_ solution is mutated
(_child.arch_) by a two step process: A hidden state mutation, the connections
between operations in a graph-represented solution (cell) are modified, and an
operation mutation, the operation within the cell is modified. Then, the
_child.arch_ is evaluated. Finally, the _oldest_ solution of the population
(i.e., the earliest evaluated solution in the population) is replaced by the
new candidate solution (_child.arch_). The authors of Agign Evolution claim
that exists a parallel between the introduced age-based removal to a
regularization of the evolution. Once the termination criteria is met, the
best solution of the population is returned.

$population\leftarrow$ empty queue

$history\leftarrow\emptyset$

while  _$\mid population\mid <P$ _ do

$model.arch\leftarrow$ Random_Architecture()

$model.accuracy\leftarrow$ Train_and_Eval($model.arch$)

add (right) $model$ to $population$

add $model$ to $history$

end while

while  _$\mid history\mid <C$ _ do

$sample\leftarrow\emptyset$

while  _$\mid sample\mid <S$ _ do

$candidate\leftarrow$ random_element from $population$

add $candidate$ to $sample$

end while

$parent\leftarrow$ highest_accuracy model $\in sample$

$child.arch\leftarrow$ Mutate($parent.arch$)

$child.accuracy\leftarrow$ Train_and_Eval($child.arch$)

add (right) $child.arch$ to $population$

add $child$ to $history$

remove (left) $dead$ from $population$

discard $dead$

end while

return highest-accuracy $model$ in $history$

Algorithm 1 Aging Evolution

A recent review Kazimipour et al. (2014) on initialization techniques to
Evolutionary Algorithms describes existing methods with three characteristics:
Randomness, Compositionality and Generality. In the case of the Aging
Evolution baseline, its initialization relies on Randomness, i.e $P$ samples
’drawn uniformly at random’. Therefore, it is a non-composite and generic
procedure.

##  3 A Data-driven approach to initializing a NAS Search Strategy

This section introduces the proposed approach of Cluster Analysis for
enhancing the performances of a NAS algorithm. First, we describe the overall
pipeline of the methodology. Second, we detail the feature engineering
essential to the analysis.

###  3.1 Pipeline

This study aims at leveraging the knowledge about a Search Space to help
improve the performances of a Search Strategy. In particular, it sets sets out
to answer the following question: can we improve the convergence of a
population-based NAS algorithm by initializing it with a data-driven approach?

Figure 1:  The pipeline of the proposed data-driven approach to NAS
initialization.

To tackle this problem, we propose an approach consisting of two steps,
depicted in Figure 1. First, we perform a performance-based clustering
analysis of the Search Space. Given the search space and a machine learning
task, we sample a set of $N$ architectures. Each architecture is trained,
evaluated, and encoded using the procedures described in Section 3.2. As the
feature vector consists in an architecture representation and its performance
in test, the resulting clusters should relate to specific behaviors
(performances) on the learned task. Moreover, processing high dimensional and
sparse data can sometimes be uneasy, therefore we propose to facilitate the
clustering by reducing the dimension of input features. With the reduced
samples, we proceed with a clustering analysis composed itself of a sequence
of sub-steps. This sequence is as follows: we reduce the dimension of the
samples, perform the clustering and assessing qualitatively and quantitatively
its results. Besides, calibrating the proper number of clusters play an
important roles in the results retrieved. Thus, we seek to identify values to
this hyperparameter providing satisfying results.

In second step of the proposed methodology, we extract the centroids obtained
in the clustering, initialize a population-based algorithm (Aging Evolution),
and assess it performances.

To summarize, this approach to search initialization is:

  * •

composite: it is multi-step initialization procedure relying on sampling a
search space, clustering it, and initializing an algorithm with the centroids
extracted.

  * •

generic: it is not application-specific, in fact the clustering could be done
on any type of search space given an encoding including a solution
representation and its fitness evaluation.

  * •

deterministic or stochastic: the stochasticity of the procedure depends on the
stochasticity of tool selected for clustering.

###  3.2 Feature representation

To best take advantage of information about the search space when clustering,
we first introduce a minimal feature engineering.

As we look to uncover models and structures relevant to NAS algorithms via
clustering, we seek a feature representation encoding an architecture as well
as its performances. As in Ying et al. (2019), we consider neural
architectures identified by an elementary component repeated in blocks, a
feed-forward cell. Such cell is a DAG, with a maximum number of operations
(nodes), a maximum number of transformations (edges) and a fixed set of
possible operations (e.g Max pool, convolution 3x3) labeling each node. A cell
is in practice represented a list of selected operations and an adjacency
matrix of variable size.

Therefore, we construct two versions of clustering feature representation,
both in the form of vectors. The first one (Original, Short Encoding) consists
in concatenating for each model, its adjacency matrix, the list of operations,
and the list of performances in test for all available training duration
$\\{t_{0},t_{1},t_{2},t_{3}\\}$. Note that this is a variable length feature
representation due to the nature of the adjacency matrix.

Alternatively, the second representation (Binary, Long Encoding) corresponds
to the expanded adjacency matrix, i.e., the matrix that consider all possible
operations (according to the constrains of the search space). This is a fixed
length encoding. Moreover, for both encoding, the vector form of the adjacency
matrix is obtained by a flattening in row-major fashion (C-style).

##  4 Experimental Setup

The experiments performed aim to validate that the initialization of a
population-based NAS Algorithm can benefit from models identified via
Clustering Analysis of a Search Space. In this Section, first, we introduce
the problem used to validate our proposal. Second, we present the parameters
used for performing the experiments on Clustering. Third, we detail the
performance metrics used to assess the quality of the clustering.

###  4.1 NASBench-101

NASBench-101 is a database of neural network architectures and their
performance evaluated on the data set of CIFAR-10. It contains $N=450K$ unique
architectures Ying et al. (2019). Indeed, to tackle the given machine learning
task of CIFAR-10, all contained models use of a classical image classification
structure similar to ResNets. Indeed, the backbone of a model contains a head,
a body and a tail. Its body is made by alternating three (3) times a block
with a down-sampling module. Each block is obtained by repeating three (3)
times a module called ’cell’. A cell is a computational unit that can be
represented by a Directed Acyclic Graph (DAG). It consist in an input node, an
output node and intermediate nodes representing operations (Conv 3x3, Conv
1x1, Maxpool 3x3), and connections indicating features being transformed.
Therefore, each architecture differs by its cell. In practice, the DAG of a
model is encoded by an adjacency matrix and a list of operations labelling the
associated nodes. The constrains on such DAG are the following: the can be at
most $N=7$ nodes and $E=9$ edges in a cell .

Moreover, all models were trained for 108 epochs using the same experimental
setting (learning rate etc), but performance evaluations in training,
validation and test were also provided after 4, 12 and 36 epochs.

###  4.2 Hyperparameters for Clustering

All the clustering experiments were done with a set of $N=10000$ randomly
sampled models. The considered clustering algorithms are K-means, DBSCAN,
BIRCH, Spectral Clustering, and a Bayesian Gaussian Mixture model (BGM). All
were obtained from the latest version (0.24.1) of the Scikit-learn library
Pedregosa et al. (2011). Table 1 shows the hyperparameters selected for each,
including the maximum number of iterations (max iter), the number of samples
used at initialization, or other more algorithm-specific ones. Note that they
are either default ($N/A$) or slightly modified to provide satisfying
clustering performances.

Method | max iter | N init | other
---|---|---|---
KMEANS | 500 | 50 | kmeans++ init
DBSCAN | 500 | 200 | eps=0.30
BIRCH | 500 | NA | threshold=0.12
SPECTRAL | 500 | NA | NA
BGM | 500 | NA | Dirichlet weight distribution, full co-variance
Table 1: Hyperparameters of the clustering algorithms.

###  4.3 Clustering performance evaluation

Moreover, we use various ways of assessing the quality of the results for each
step of the approach. Regarding step one (1), we propose to measure the
Clustering performance using the following three (3) standard metrics: the
Silhouette Coefficient Rousseeuw (1987), the Calinski-Harabasz Caliński and
Harabasz (1974) and the Davies-Bouldin Indexes Davies and Bouldin (1979).
These inform on how well separated and dense are the resulting clusters. They
all apply in the context of Clustering with missing labels, which is relevant
as we seek to investigate relevant clusters and features for NAS algorithms
without prior assumptions. The Silhouette Coefficient is a metric comprised
between -1 and +1, with higher values associated to more dense and separated
clusters The Calinski-Harabasz Index also rates a better defined clusters with
higher values. Similarly, the Davies-Bouldin Index, measures a ’similarity’
between clusters, providing smaller values for better clustering. Additionally
we propose to corroborate the later with a qualitative analysis (visual
assessment) for validation before for step two.

Regarding step two (2), we assess the quality of the centroid-based
initialization using the performances of the algorithm (best obtained accuracy
in test). More importantly, we compare its performances to those of
initializing the same algorithm with random samples, or Random Search.

##  5 Results

In this section, we present results on Clustering for accelerating NAS
algorithms. First, we show results on selecting the proper dimension reduction
tool and hyperparameters for the Clustering. Then, we show results on
identifying the number of clusters providing satisfying Clustering
performances. We also present results on Qualitatively assessing the clusters
quality for various algorithms. Last but not least, we present results on
improving NAS performances using a centroid-based initialization of a
evolutionary NAS algorithm.

###  5.1 Dimension Reduction

To begin our experimental study, we seek to calibrate the dimension reduction
of the input features.

Figure 2 show clustering performances as a function of the number of
components of input features. The blue and red curves displays performances
using respectively the Short (Original) and the Long Encoding (Binary). The
dimension reduction is performed using PCA and clustering with K-means.

Using the Short Encoding (Original), the three metrics are in favor of using a
small number of components for input features via PCA. Indeed the smaller the
number of components the higher the Silhouette and Calinski-Harabasz scores,
and the lower the Davies-Bouldin index, with optimal values for using two (2)
components. The same observations applies when using the Long Encoding
(Binary).

Using the Long Encoding (Binary) yields slightly better performances then the
Short Encoding (Original), with an sensible improvement with larger number of
components with PCA.

Figure 2: Input feature reduction for clustering, with an arbitrary number of
clusters of $N=10$.

As both encoding are rather sparse (lenght of up to 58, or up to 298), we look
know if the dimension reduction tool is affected by such sparsity. Figure 3
show clustering performances as a function of the number of components of
input features, for various reduction tool. The blue and red curves displays
performances using respectively PCA and Truncated SVD as dimension reduction
tools. Plot (a) and (b) display results using respectively the Short
(Original) and the Long Encoding (Binary). The clustering is performed with
K-means.

Trying an alternative dimensional reduction tool (Truncated SVD) more suitable
for higly sparse data does not worsen results on the Short (Original) encoding
(see Figure 3 (a)). Moreover, it allows for a slight improvement over PCA when
using the Long (Binary) Encoding (see Figure 3 (b)).

(a)

(b)

Figure 3: Identifying the proper reduction tool for sparse data, using an
arbitrary number of clusters of $N=10$.

To summarize, the findings show that reducing the dimensions of the input
features to 2D provides the best performances on both encoding. Using the Long
(Binary) encoding improves the results. Also, using Truncated SVD shows slight
improvements as it is more suitable for sparse data. Given these findings, we
following experiments are performed using Truncated SVD for a 2D reduction of
input.

###  5.2 Number of Clusters

Next, we look to identify the number of clusters providing the most satisfying
results.

Figure 4 show clustering performances as a function of the number of clusters.
The blue and red curves displays performances using results using respectively
the Short (Original) and the Long Encoding (Binary). All input features were
reduced to two (2) components using Truncated SVD, and clustering is performed
with K-means.

Figure 4: Identifying the proper number of clusters, using Truncated SVD for
dimension reduction ($N=2$ components) and K-means.

Using both encoding, all performance metrics points towards the use of large
number of clusters. Indeed the higher the number of clusters, the higher the
Silhouette and Calinski-Harabasz scores, and the lower the Davies-Bouldin
index. Additionally, intermediate values around twenty (20) and twenty seven
(27) clusters respectively for the Original and Binary encoding seem to reach
satisfying performance already.

Therefore, results suggest using an intermediate (20,30) to large number of
clusters for improving the K-means clustering performances, with a preference
for the Short encoding.

###  5.3 Qualitative Cluster Analysis

As an additional way to validate the clustering results, we seek to visualize
the obtained clusters and compare them to the natural layout of the reduced
data.

Figure 5 displays visual clustering results for five algorithms: K-means,
Spectral Clustering, DBSCAN, Birch, and a Bayesian Gaussian Mixture of models
(BGM). All input features were reduced to two (2) components using Truncated
SVD. Plot (a) and (b) display results using respectively the Short (Original)
and the Long Encoding (Binary).

(a)

(b)

Figure 5: Qualitative analysis of clustering for both feature representations.
Here we use Truncated SVD for dimension reduction, and $N=2$ components.

When using the Short encoding as depicted in Figure 5 (a), the clusters seem
to have natural horizontal to diagonal (45 degree) layout. This layout is not
well captured by the evaluated algorithms. The Bayesian Gaussian mixture of
models (BGM) seems to provide the most satisfying results, despite little
calibration.

When using the Long encoding as depicted in Figure 5 (b), clusters naturally
layout in well separated and vertical columns. This is also best captured by
the BGM.

Overall, results suggest using BGM for robust clustering on both feature
representations.

###  5.4 Towards Improving NAS Algorithms: A Centroid-based Initialization

In order to assess the quality of the centroids extracted, we use them for
initializing the Aging Evolution.

Figure 6 shows performances in test of various NAS Algorithms. In red appears
Random Search. In blue (Evolution) is Aging Evolution initialized with a
$N=27$ randomly selected configurations. In green (Boosted Evolution) is the
same algorithm but initialized with the $N=27$ centroids collected from
clustering. Both settings of the Aging Evolution use a population of fixed
size $P=N$, and a tournament size $S=10$. The clustering is done with BGM
using the Short Encoding. Each algorithm is ran $M=100$ independent times,
each with distinct random seeds. The top (resp. bottom) figure is for
selecting models trained for 36 (resp 108) epochs.

When selecting models at both 36 or 108 epochs, the Aging Evolution baseline
displays superior results with the centroid-based initialization approach
using the Short encoding. It is indeed better than the Random search baseline,
and demonstrates faster convergence rates then other evaluated algorithms at
36 epochs of training.

(a)

(b)

Figure 6:  Performances in test of various NAS algorithms, when clustering
with the Original Encoding.

Table 2 summarizes the performance of the algorithms benchmarked in Figure 6,
i.e when clustering with the Original Encoding. RS stands for Random Search,
AE for Aging Evolution with randomly initialized population, and B-AE for
Aging Evolution with centroid-based population initialization. To complement
these results, we performed a Wilcoxon rank-sum test to compare RS against
B-AE. The _p_ -value on 36 epochs is equal to 0.0002, and on 108 epochs is
equal to 1.199e-13. Thus, B-AE significantly improves over RS. When repeating
the test with B-AE and AE, the _p_ -value is equal to 0.034 and 0.212 for 36
and 108 epochs respectively. We conclude that B-AE significantly improves over
AE for 36 epochs.

|  | RS | AE | B-AE
---|---|---|---|---
36 epochs | mean | 92.34 | 92.38 | 92.41
median | 92.33 | 92.33 | 92.33
min | 91.98 | 92.25 | 92.18
max | 92.68 | 92.75 | 92.75
std | 0.167 | 0.131 | 0.128
108 epochs | mean | 93.93 | 94.13 | 94.14
median | 93.81 | 94.12 | 94.13
min | 93.80 | 93.70 | 93.70
max | 94.32 | 94.35 | 94.42
std | 0.164 | 0.098 | 0.132
Table 2: Benchmark of NAS algorithm performances after 2000 iterations. The
data-driven initialization technique involve the Original Encoding.

Figure 7 shows results for the same setting with the exception of initializing
Aging Evolution with the $N=27$ centroids collected by BGM on the Long
Encoding. Similarly, the Aging Evolution algorithm shows competitive
performances with the centroid-based initialization, improving over the
baselines when training for long (108).

(a)

(b)

Figure 7:  Performances in test of various NAS algorithms, when clustering
with the Binary Encoding.

Table 3 summarizes the benchmark provided in Figure 7. When comparing final
performances of RS and B-AE, we obtain a _p_ -value at 36 epochs equal to
0.0435, and at 108 epochs equal to 5.105e-11. Thus, B-AE significantly
improves over RS. When repeating the test with B-AE and AE, the _p_ -value is
equal to 0.766 and 0.085 for 36 and 108 epochs respectively. Thus, despite
higher mean and median of test accuracy at 108 epochs, the improvements of
B-AE over AE are non-significant.

To summarize, the centroid extracted from a fitness-based clustering of a the
Search Space seem to be a promising population to initializing an evolutionary
search algorithm. In fact, in the challenging scenario of selecting models
after only 36 epochs of training, using the proposed technique with the Short
Encoding improves significantly over the baseline initialization (_p_
-value=0.034).

The limited improvements when clustering with the Binary Encoding might be
explained by the fact that the Aging Evolution is deployed on models using the
Short Encoding. Indeed, experiments using the Long Encoding were discarded
because of the increased complexity for the search procedure. Future work
might exploring this option, at it could help better exploit the extracted
population.

|  | RS | AE | B-AE
---|---|---|---|---
36 epochs | mean | 92.36 | 92.40 | 92.40
median | 92.33 | 92.33 | 92.33
min | 91.98 | 92.25 | 92.14
max | 92.75 | 92.75 | 92.75
std | 0.174 | 0.138 | 0.141
108 epochs | mean | 93.96 | 94.13 | 94.16
median | 93.81 | 94.12 | 94.17
min | 93.70 | 93.80 | 93.70
max | 94.42 | 94.42 | 94.42
std | 0.191 | 0.118 | 0.128
Table 3: Benchmark of NAS algorithm performances after 2000 iterations. The
data-driven initialization technique involve the Binary Encoding.

##  6 Conclusion

In this study, we seek to gain insights about a search space of image
classification models in order to improve the performance of NAS algorithms.
More precisely, we want to know if the convergence of a search strategy could
be improved using a data-driven initialization technique exploiting the search
space.

For this purpose, we propose a two-step approach to improve the performances
of a NAS search strategy. First, we perform a clustering analysis of the
search space, involving a sequence of sub-tasks. It summarizes as follows: we
sample models from a search space, reduce their dimension, perform a
clustering. After a careful tuning of the clustering pipeline (number of
dimensions, clusters, etc), we select the algorithm providing the best
qualitative and quantitative results. Second, we run a evolutionary algorithm
using as an initial population the centroid extracted previously. Results show
that centroids extracted using BGM for clustering are a promising population
to initialize a state-of-the-art evolutionary algorithm. In the scenario of
selecting models trained only 36 epochs, this appoach shows faster convergence
and long term improvements over a standard initialization (p-value=0.034 at
2000 iterations), when using a Short Encoding.

As future work, we propose to investigate performances of this approach when
selecting models on the Long Encoding. We also propose to study in depth the
obtained clusters to gain more insights on obtained performances. One might
also explore the benefits of such data-driven initialization method on other
families of algorithms (Bayesian Optimization, Local Search, etc).

## Acknowledgments

Authors acknowledge support by the European Research Council (ERC) under the
European Union’s Horizon 2020 research and innovation program (grant agreement
No. [ERC-2016-StG-714087], Acronym: So2Sat), by the Helmholtz Association
through the Framework of Helmholtz AI [grant number: ZT-I-PF-5-01] - Local
Unit “Munich Unit @Aeronautics, Space and Transport (MASTr)” and Helmholtz
Excellent Professorship “Data Science in Earth Observation - Big Data Fusion
for Urban Research”(W2-W3-100), by the German Federal Ministry of Education
and Research (BMBF) in the framework of the international future AI lab ”AI4EO
– Artificial Intelligence for Earth Observation: Reasoning, Uncertainties,
Ethics and Beyond” (Grant number: 01DD20001). and the grant DeToL

## References

  * Caliński and Harabasz [1974] T. Caliński and J Harabasz.  A dendrite method for cluster analysis.  Communications in Statistics, 3(1):1–27, 1974.
  * Camero et al. [2021] Andrés Camero, Hao Wang, Enrique Alba, and Thomas Bäck.  Bayesian neural architecture search using a training-free performance metric.  Applied Soft Computing, page 107356, 2021.
  * Davies and Bouldin [1979] David L. Davies and Donald W. Bouldin.  A cluster separation measure.  IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-1(2):224–227, 1979.
  * Dong and Yang [2020] Xuanyi Dong and Yi Yang.  Nas-bench-201: Extending the scope of reproducible neural architecture search.  In International Conference on Learning Representations (ICLR), 2020\.
  * Elsken et al. [2019] Thomas Elsken, Jan Hendrik Metzen, Frank Hutter, et al.  Neural architecture search: A survey.  J. Mach. Learn. Res., 20(55):1–21, 2019.
  * Hanxiao et al. [2019] Liu Hanxiao, Simonyan Karen, and Yang Yiming.  Darts: Differentiable architecture search.  International Conference on Learning Representations, 2019.
  * Hutter et al. [2019] Frank Hutter, Lars Kotthoff, and Joaquin Vanschoren, editors.  Automated Machine Learning - Methods, Systems, Challenges.  Springer, 2019.
  * Kazimipour et al. [2014] Borhan Kazimipour, Xiaodong Li, and Kai Qin.  A review of population initialization techniques for evolutionary algorithms.  In Proceedings of the 2014 IEEE Congress on Evolutionary Computation, CEC 2014, 07 2014.
  * Ojha et al. [2017] Varun Kumar Ojha, Ajith Abraham, and Václav Snášel.  Metaheuristic design of feedforward neural networks: A review of two decades of research.  Engineering Applications of Artificial Intelligence, 60:97–116, 2017.
  * Pedregosa et al. [2011] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay.  Scikit-learn: Machine learning in Python.  Journal of Machine Learning Research, 12:2825–2830, 2011.
  * Real et al. [2019] Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V. Le.  Regularized evolution for image classifier architecture search.  Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):4780–4789, Jul. 2019.
  * Rousseeuw [1987] Peter J. Rousseeuw.  Silhouettes: A graphical aid to the interpretation and validation of cluster analysis.  Journal of Computational and Applied Mathematics, 20:53–65, 1987\.
  * Ying et al. [2019] Chris Ying, Aaron Klein, Eric Christiansen, Esteban Real, Kevin Murphy, and Frank Hutter.  NAS-bench-101: Towards reproducible neural architecture search.  In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 7105–7114, Long Beach, California, USA, 09–15 Jun 2019. PMLR.