11institutetext:  1Brain Cognitive Computation Lab, Cognitive Science
IIIT, Hyderabad-32
11email<EMAIL_ADDRESS>
11email<EMAIL_ADDRESS>

# BeSt-LeS: Benchmarking Stroke Lesion Segmentation using Deep Supervision

Prantik Deb 11 Lalith Bharadwaj Baru 11 Kamalaker Dadi 11 Bapi Raju S 11

###### Abstract

Brain stroke has become a significant burden on global health and thus we need
remedies and prevention strategies to overcome this challenge. For this, the
immediate identification of stroke and risk stratification is the primary task
for clinicians. To aid expert clinicians, automated segmentation models are
crucial. In this work, we consider the publicly available dataset ATLAS $v2.0$
to benchmark various end-to-end supervised U-Net style models. Specifically,
we have benchmarked models on both 2D and 3D brain images and evaluated them
using standard metrics. We have achieved the highest Dice score of 0.583 on
the 2D transformer-based model and 0.504 on the 3D residual U-Net
respectively. We have conducted the Wilcoxon test for 3D models to correlate
the relationship between predicted and actual stroke volume. For
reproducibility, the code and model weights are made publicly available.

###### Keywords:

Stroke Lesion Segmentation T1 Weighted MRI Deep Supervision ATLAS $v2.0$ Deep
Learning

##  1 Introduction

Brain stroke has become a significant burden on global health with increasing
prevalence in low- and middle-income countries. Therefore there is an urgent
need for targeted prevention strategies and improved healthcare infrastructure
to address this growing public health challenge [20], [22]. Given brain
Magnetic Resonance (MR) images on stroke populations, localizing and detecting
the lesions is crucial for clinicians. However, the automation of the
localization process has achieved significant reach [5] with novel machine-
learning models that can aid clinicians. To fuel these models, we need
datasets that could automatically segment to the level of expertise and doing
so, it could ease the clinician’s task.

In this context, ATLAS (Anatomical Tracings of Lesions After Stroke) v1.2
dataset made progress by creating 304 T1-weighted MRI samples collected from
11 cohorts. This ATLAS v1.2 [24] was released in 2018 and of which, 229
standardized subjects were available with T1-weighted MRI image and its
corresponding lesion mask. Later in 2022, ATLAS v2.0 [25] was released and it
has 1217 T1-weighted MRI samples collected from 44 cohorts of which 655
samples mask were availed to the public. In specific, there was an extra
margin of 426 samples from version 1.2 to 2.0.

There were numerous impressive models that performed well on ATLAS v1.2.
Whereas, ATLAS v2.0 was released recently and therefore, there is quite less
progress. Therefore, we outpace and benchmark ATLAS v2.0 on various standard
U-Net style architectures for both 2D and 3D brain images.

In Table 1, ATLAS v1.2 was applied using distinct U-Net architectures for 2D
modality but, there isn’t any model to date that has been implemented on ATLAS
v2.0. Similarly, in the case of 3D modality, Table 1 illustrates there are few
implementations for ATLAS v2.0 using nnU-Net as their underlying framework
[18]. Thus, we contribute by analyzing the ATLAS v2.0 dataset for both 2D and
3D modality. A brief description of each of the models is detailed in the
supplementary material.

Table 1: The table summarizes the glimpse of 2D and 3D U-Net variants whether they were implemented on the ATLAS v1.2 and v2.0 T1 MR images denoted as Yes or No. The implementation of 2D models on ATLAS v2.0 is sparse. 2D Models | ATLAS v1.2 | ATLAS v2.0
---|---|---
U-Net | Yes [42] [30] [45][43] [33] | No
Residual U-Net | Yes [42] [30] [33] | No
Attention U-Net | Yes [33] | No
Transformer Based | Yes [40] | No
3D Models | ATLAS v1.2 | ATLAS v2.0
U-Net | Yes [45] [43] [28] | Yes [36] [17]
Residual U-Net | Yes [34] | Yes [17]
Attention U-Net | No | No
Transformer Based | No | No

##  2 Contributions of this work

  1. 1.

To the best of our knowledge, ours is the first attempt to benchmark the
standard segmentation models _i.e._ both convolution and transformers-based
architectures on the ATLAS v2.0 dataset.

  2. 2.

We have also conducted experiments for both 2D and 3D-based models. We report
our highest dice score of 0.58 on the 2D transformer-based model. Also, we
have achieved a 0.504 dice score on the 3D residual U-Net.

  3. 3.

Finally, we conduct the Wilcoxon test on 3D models and compare the
relationship between predicted and actual stroke volume.

##  3 Data and Models

This section briefly discusses the dataset and methods considered for
analysis. The organization is as follows: First, we introduce the dataset and
the training strategy implied. Next, we detail the significance of various
U-Net style models.

###  3.1 Dataset

Table 2: The below table describes the train validation and test proportions
divided for training supervised 2D U-Net style architectures. The number of
samples below represents the number of subjects. The slices are unevenly
divided based on the volume of the T1-weighted MRI and the axial plane is
considered while cropping each slice.

Split | % | Samples | Slices
---|---|---|---
Train | 60 | 393 | 15394
Validation | 20 | 131 | 4666
Test | 20 | 131 | 5452

Table 3: In the below tabular data, the train validation and test proportions
are divided for training supervised 3D U-Net style architectures. The number
of samples below represents the number of subjects.

Split | % | Samples
---|---|---
Train | 60 | 393
Validation | 20 | 131
Test | 20 | 131

The ATLAS (Anatomical Tracings of Lesions After Stroke) data consists of
T1-weighted MRI images of subjects having lesions due to stroke. This data has
two versions, ATLAS v1.2 [24], and ATLAS v2.0 [25], respectively. For our
analysis, we solely conduct our experiments on ATLAS v2.0 dataset [25], which
is publicly available. The samples in this data (ATLAS v2.0) were collected
from 44 diverse cohorts with a total sample size of 1271. From these 1271
samples, only 655 samples consist of image-to-mask pairs dedicated to training
the models. Another 300 samples are treated as hidden-set and do not reveal
the masks of T1-weighted MRI images111Additional information such as lesion
numbers, cortical location, and severity of stroke for each subject can be
found in the original paper.

While analyzing ATLAS v2.0 we conduct experiments both on 2D and 3D modality.
As masks for the original test set are inaccessible we divide the training
samples (655) into train validation and test proportions. For 3D modality, the
data can be directly fed to the model. But for 2D modality, each subject’s 3D
T1-weighted MRI images are to be cropped into slices along the z-axis (axial)
and then given as input to 2D U-Net style architectures. For both the
modalities the _Z_ -score Normalization are performed as a pre-processing step
[2]. The information regarding train, test, and validation sets are elucidated
in Table 3 and 3. For the 2D dataset, the discrepancy between the validation
and the test is due to a rejection of 0.1% lesions in the given 2D slice.
Specifically, wherever the slices and their respective mask pairs were
rejected were not included in the study. Now, this data is to be processed
using various segmentation architectures to segment the lesion in the
T1-weighted MRI images in the protocol mentioned in Table 3 and 3.

###  3.2 U-Net Style Architectures

The delineation of a specific organ or certain tissue site or cell nuclei from
given medical images is one of the crucial tasks in medical image analysis.
Various deterministic algorithms were developed to automate this process, some
of which are random walks [15] and SLIC [1]. But later with the aid of deep
learning, more sophisticated and _learnable_ methods were developed [12].
Later, Ronneberger _et al._ [32] proposed U-Net architecture from which the
field of medical image segmentation caught its attention as U-Net was fast,
modulable, and robust. Later there were many methods that were crafted using
the U-Net style as the underlying framework. Thus, we study and benchmark some
of the fundamental U-Net style architectures that achieved significant results
in the field of medical image segmentation.

#### U-Net:

This was the first deep learning architecture that superseded the existing
models with large margins both computationally and performance-wise. This
architecture, thus, was made as a baseline for many medical segmentation tasks
[32]. Soon after the U-Net, the architecture was modified to learn volumetric
data using 3D convolutions [11] with sparse annotations. The architecture is
very similar to that of 2D U-Net except that, 3D can process volumetric
information using 3D convolutions and it is depicted in the last row of Fig.
1.

Figure 1: The figure illustrates various U-Net style architectures. (First
row) shows a diagrammatic view of the convolution-based Transformer models and
(bottom row) shows two novel transformer-based U-Net architectures. We
detailed all the symbols and signs used in the legend block.

The architecture is quite intuitive, as features are downsampled using pooling
layers to a latent space or base and later upsampled to reconstruct the
desired mask image. This latent space is perceived to preserve the crucial
features, while the skip-connections (these are denoted as ($\cdots$)) aid the
reconstruction by guiding to map of the structural information. In the Fig. 1,
you can see each blue circle represents a series of convolution layers, and
the red arrow ($\rightarrow$) indicates that an image of size $h\times w$ is
downsampled (pooled) to $\frac{h}{2}\times\frac{w}{2}$. The downsampling
operation is quadruply performed to get to the latent space. Now, from this
latent space, the acquired features are upsampled quadruply (indicated with a
green arrow ($\rightarrow$)) to produce the desired segmentation.

#### Residual U-Net:

He _et al._[16] proved that residual connections tend to provide refined
representations for downstream tasks with less computation and better
performance. In this regard, Zhang _et al._ [44] proposed a U-Net architecture
with residual connections for extracting road patterns from aerial imagery.
This later was implied in the domain of medical images by Alom _et al._[3]
with additional memory components. Later, the residual connections were
implied between a series of 3D convolutions to produce volumetric segmentation
[41], [7],[19]. The architecture is very similar to that of 2D residual U-Net
except that, 3D can produce volumetric masks from 3D medical data, and the
pictorial interpretations are illustrated in the last row of Figure 1.

The current Residual U-Net style was inspired by Zhang _et al._[44]. In U-Net,
before downsampling at each step, there are a series of convolutions with
residual connections, which are represented by an orange circle. The rest,
upsampling ($\rightarrow$) and downsampling ($\rightarrow$) operations, are
similar to that of traditional U-Net.

#### Attention U-Net:

The fundamental concept of _attention_ was formulated by Bahdanau _et al._[4].
Later Oktay _et al._ [27] applied this mechanism as ’ _Attention Gate_ ’ (AG),
which improved segmentation with detailed localization of multiple organs.

In this architecture, the core component is the attention gate which aids the
U-Net in segmenting desired lesions. The architecture style in downsampling is
similar to U-Net, i.e., the image is quadruply downsampled ($\rightarrow$
$\times 4$) to obtain the latent space. In traditional U-Net, upsampling
($\rightarrow$) is achieved using transpose convolutions, and additional
representations are aggregated from the skip connections. But in Attention
U-Net, the representations from the previous layers and from skip connections
are aggregated using AG, and now these features are concatenated with the
upsampled ($\rightarrow$) features. Thus, they form cascaded connections
resulting in better segmentation. Similarly, this attention gate is applied to
3D convolutions to get volumetric attention [26], [37].

Table 4: The below table illustrates the performance of variants of 2D U-Net architectures. The first three models are pure convolution-based architectures and the remaining two are hybrid networks with convolutions and transformer components. The evaluation criteria implied is the same as Table 3; We report the performance of the model for the test set. Method | Performance Metrics (2D Data)
---|---
Dice Score | IoU Score | Precision | Recall
U-Net [32] | 0.417 | 0.337 | 0.580 | 0.360
Residual U-Net [44] | 0.456 | 0.375 | 0.592 | 0.420
Attention U-Net [27] | 0.487 | 0.396 | 0.636 | 0.439
TransAttn U-Net [10] | 0.572 | 0.477 | 0.660 | 0.565
U-Net Transformer [29] | 0.583 | 0.475 | 0.659 | 0.591

#### TransAttn U-Net:

This architecture was designed by Chen _et al._ [10] in which they propose
SAA: Self-Aware Attention, which is an amalgamation of multi-level and multi-
scale guided attention mechanisms. In specific, after downsampling features to
the embedding space, they perform two attention mechanisms which are
Transformer Self-Attention ($\mathcal{F}_{TSA}$) and Global Spatial Attention
($\mathcal{F}_{GSA}$). Now, these features are combined into a single
convolution block, and with each step of upsampling, the previous layer
features are attached as skip connections using _Bi-linear Upsampling_ (refer
Fig. 1). The significance of each attention mechanism is elucidated below.

Suppose our image is represented as $X\in\mathbb{R}^{t\times h\times w}$,
where $t,h,w$ are time-steps (channels), height, and width of the given image,
respectively. The image is passed to the encoder, and the downsampled
representation is denoted by
$\mathcal{F}_{base}^{t}\in\mathbb{R}^{t\times(h\times w)}$. Now, to achieve
GSA,

$\mathcal{F}_{GSA}(M,N,W)_{i}=\sum_{k=1}^{h\times
w}\left(W_{k}\mathcal{A}_{i,j}\right)$ (1)

Where, $N\in\mathbb{R}^{t^{\prime}\times(h\times w)}$ and
$M\in\mathbb{R}^{(h\times w)\times t^{\prime}}$. Also the
$\mathcal{A}_{i,j}=\frac{e^{(M_{i}N_{j})}}{\sum_{r=1}^{n}e^{(M_{r}N_{j})}}$ .
This $\mathcal{A}_{i,j}$ measures the input of the $i^{\text{th}}$ and
$j^{\text{th}}$ position. Similarly, the TSA attention is calculated as,

$\mathcal{F}_{TSA}(K,Q,V)=soft\left(\frac{QK^{T}}{(d_{k})^{1/2}}\right)V$ (2)

Where $K,Q,V$ are just the features of $\mathcal{F}_{base}^{t}$ added with
positional encoding and $d_{k}$ is the dimensionality of any key or Query or
value sequence (i.e., $d_{k}=|V|or|Q|or|K|$ and $soft(.)$ is the softmax
activation function [14].) These attentions are operated at the latent space
or base, and now, in final step, all these features are amalgamated at the
latent space as,

$\mathcal{F}_{SAA}=\psi_{1}\mathcal{F}_{TSA}+\psi_{2}\mathcal{F}_{GSA}+\mathcal{F}_{base}$
(3)

Where $\psi_{1}$ and $\psi_{2}$ are the scale parameters, respectively, and
they control the importance assigned to each attention mechanism. Initially,
they are assigned with null weights and gradually incremented to obtain a
systematic consistency.

#### U-Net Transformer:

This method originated from the work by Petit _et al._ [29]. The authors
impart self-attention and channel attention modules in this work to produce
interpretative segmentation throughput. Fundamentally, the self-attention
module uses Multi-head Self-Attention (MHSA) which is similar to Vaswani _et
al._ [35], and this aims to acquire long-range structural information from the
images that were downsampled to the latent-space. The underlying operation is
quite similar to equation (2).

Figure 2: 2D visualizations of the benchmarks between ground truth and
predicted lesions for two subjects. As can be seen, we have two different
sizes of stroke lesion subjects included for visualization (Subject ID:
$sub-r027s050$ slice 91 and $sub-r004s004$ slice 68) of which one is small and
the other being large. We display the predicted outputs of convolution and
transformer-based 2D U-Net models for these subjects. All 2D models performed
equally better but the U-Net transformer gave the finest boundaries as visible
for both the subjects.

In the channel-attention module, the representations from skip connections (at
each pooling step) are first applied with MHSA and then concatenate the
features coming from latent space after each upsampling step. Thus this module
is referred to as Multi-Head Channel-Attention (MHCA) as initially the input
is transferred to MHSA and then concatenated with a cross-attention mechanism.
The diagrammatic explanations are elucidated in Fig. 1.

Table 5: The table illustrates the performance of variants of 3D U-Net architectures. These models are pure convolution-based architectures, and the evaluation criteria are the same as in Table 3. We report the performance of the model on the test set. Method | Performance Metrics (3D Data)
---|---
| Dice Score | IoU Score | Precision | Recall
U-Net [11] | 0.450 | 0.350 | 0.584 | 0.444
Residual U-Net [44] | 0.504 | 0.393 | 0.585 | 0.533
Attention U-Net [27] | 0.469 | 0.369 | 0.498 | 0.578

##  4 Results and Discussion

In this section, we experiment with the aforementioned methods as summarized
in Section 3. First, we evaluate the performance of 2D and 3D models. Later,
we conducted the Wilcoxon test using the ground truth and predicted lesion
volume for 3D U-Net models.

###  4.1 Results for 2D

Among the 2D U-Net convolution-based models the attention U-Net has proven to
have a significant Dice score of 0.487 (with an extra verge of 7.0% dice score
from baseline222In this paper we consider, standard U-Net to be our baseline
for both 2D and 3D models respectively.). Whereas the hybrid transformer- and
convolution-based models tend to provide a noteworthy performance of 0.583 and
0.572 dice scores. These models superseded 2D standard U-Net with an
additional 15.5 and 16.6% of dice score respectively. All these results are
illustrated in Table 4.

As there were no additional augmentations applied we can directly infer that
adding self-attention components (latent space) such as TSA, MHSA, and GSA
played a crucial role in providing significant performance with their cascaded
attention [40].

In this regard, we have considered two subjects for visualizing the
performance of each model in predicting lesions. In Figure 2 we have displayed
the potential of each model to segment small lesions (first row) and large
lesions (second row). All the models were near good in segmenting the large
volume of lesions but the U-Net Transformer is able to accurately delineate
the boundaries of tissues from T1-weighted MRI. But, most of the models
struggle to extract the small lesions. Hybrid models such as transformer-based
architecture did perform well (comparatively) to a certain extent but, still,
there is a wide scope for developing novel models.

Figure 3: The above visualization was considered from the test set (ID:
$sub-r001s010$ and $sub-r011s020$ ) and compares three 3D U-Net models
(standard, Residual, and Attention). For each model, the left part remains as
ground truth and the right part is the model’s predictions. In each image, the
visualization elucidates the precise location of stroke in the brain using
three axes (sagittal, coronal, and axial).

###  4.2 Results for 3D

Now, we consider 3D U-Net style architectures which include standard U-Net,
Residual, and Attention U-Net. Due to the added temporal relationship among
the features in 3D convolution, the standard 3D U-Net was able to achieve a
0.45 dice score. Previously, in 2D modality, the convolution-based model did
not achieve more than a 0.47 dice score. However, 3D modalities, both standard
and residual U-Net have a decent increment in performance without any data
augmentation [36]. The shift of modality from 2D to 3D, for Residual U-Net and
standard U-Net, had an increment of 7.7% and 4.3% of dice score respectively.
But, attention U-Net did not succeed in incriminating its performance by
shifting from 2D to 3D. The reason behind it might be due to a lack of
augmentations and an insufficient number of samples for training333The authors
have experimented with various optimizers, learning schedulers, and many more
hyperparameters.. The results for these models under different performance
metrics are illustrated in Table 5.

To specifically study the behaviour of 3D segmentation models we visualized
certain test samples one of which is visualized in Figure 3. We have
considered two test samples and visualized each plot in three different axes.
In each image, the first image describes the Coronal plane and the middle one
is the Sagittal plane. Finally, the terminal one is an axial plane. The
combination of these three views gives us an estimate of the 3-dimensional
pattern of stroke lesions in the brain. As can be seen from Figure 3, U-Net is
unable to provide good segmentation outcomes. Though Residual and Attention
U-Net were able to segment well but not on par with the mask.

Figure 4: The Wilcoxon test is carried out for all three 3D U-Net style
architectures. The first row indicates a scatter plot and the second one
indicates box plots of the predicted and actual stroke volume of the 3D models
respectively. Specifically for the scatter plot. The ideal scenario must be
the gray dots aligned with the red line.

#### Wilcoxon Signed Rank Test:

Now, we also study the prominence of lesion volume using the ground truths
[36]. This is achieved by conducting a statistical test, specifically, the
Wilcoxon Signed Rank test, and studying whether the lesion volume distribution
patterns for each test subject are similar or not 444This test could be
understood as a non-parametric t-test. A detailed premier is provided in the
supplementary material.. Thus, we establish the results for all three 3D
U-Nets (Refer Figure 4). For U-Net and Residual U-Net, the test rejects the
null hypothesis and whereas, for attention U-Net it accepts the test with a
p-value of 0.54. The detailed results of the Wilcoxon test are detailed in the
supplementary material. Also, we visualize box plots to assess the original
and predicted volume distribution for the test samples as in Figure 4.

The distribution of pixels changes after resizing them to a certain shape in
the case of 2D. As 2D models are often shrunk and stretched, based on the
model, and doing so can misguide the volume calculation. Thus, we cannot
estimate the true volumes in such cases and that is the reason we have only
reported for 3D models.

##  5 Limitations and Future Directions

In the analysis, we have considered a set of state-of-the-art models. But,
there are certain limitations in the current work as described below:

  * •

Our motto was to provide standard U-Net style models that are trained on ATLAS
v2.0 without any augmentations. Thus, in this current work, we have not used
any frameworks such as Deep Medic [21], nnU-Net [18], and MONAI [9].

  * •

The current analysis is only done using models that have masks and so we
achieved the results with supervision. Later, this work can be extended with
weak-supervision [39], [8] or self-supervision [17] approaches which can aid
the learning of models in the absence of masks.

  * •

This work does not focus on uncertainty or ambiguity in decision-making using
generative models [6] [23] [31].

  * •

Also, we did not address the issue of very small and disconnected lesions
[17].

The limitations described above can be seen as future directions and they
could contribute to the progressing field of Neuroimaging for Stroke
prediction.

##  6 Conclusion

This paper fundamentally benchmarks variants of U-Net models on the ATLAS v2.0
dataset and deploys standard stroke lesion segmentation models which could be
reproducible both for 2D and 3D brain images. We infer that current 2D or 3D
brain imaging prediction requires much more attention towards developing
hybrid models with the aid of _self-attention_ mechanisms to improve the
performance of the models. In the future, we tend to develop fine-grained
segmentation models with data augmentation, multi-modality (Diffusion Weighted
Imaging and T2-FLAIR), and cascaded attention mechanisms. We hope this
research could progress and contribute to Stroke prediction.

## Acknowledgement

The authors would like to acknowledge Manasa Kondamadugu for her invaluable
coordination efforts throughout the project. Additionally, we extend our
gratitude to IHub-Data, International Institute of Information and Technology,
Hyderabad for their generous funding and support.

## References

  * [1] Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: Slic superpixels compared to state-of-the-art superpixel methods. IEEE transactions on pattern analysis and machine intelligence 34(11), 2274–2282 (2012)
  * [2] Akkus, Z., Galimzianova, A., Hoogi, A., Rubin, D.L., Erickson, B.J.: Deep learning for brain mri segmentation: state of the art and future directions. Journal of digital imaging 30, 449–459 (2017)
  * [3] Alom, M.Z., Yakopcic, C., Hasan, M., Taha, T.M., Asari, V.K.: Recurrent residual u-net for medical image segmentation. Journal of Medical Imaging 6(1), 014006–014006 (2019)
  * [4] Bahdanau, Dzmitry, K.C., Bengio, Y.: Neural machine translation by jointly learning to align and translate. ICLR (2014)
  * [5] Baird, A.E., Warach, S.: Magnetic resonance imaging of acute stroke. Journal of Cerebral Blood Flow & Metabolism 18(6), 583–609 (1998)
  * [6] Baumgartner, C.F., Tezcan, K.C., Chaitanya, K., Hötker, A.M., Muehlematter, U.J., Schawkat, K., Becker, A.S., Donati, O., Konukoglu, E.: Phiseg: Capturing uncertainty in medical image segmentation. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part II 22. pp. 119–127. Springer (2019)
  * [7] Bhalerao, M., Thakur, S.: Brain tumor segmentation based on 3d residual u-net. In: International MICCAI Brainlesion Workshop. pp. 218–225. Springer (2019)
  * [8] Cao, C., Liu, Z., Liu, G., Jin, S., Xia, S.: Ability of weakly supervised learning to detect acute ischemic stroke and hemorrhagic infarction lesions with diffusion-weighted imaging. Quantitative Imaging in Medicine and Surgery 12(1), 321 (2022)
  * [9] Cardoso, M.J., Li, W., Brown, R., Ma, N., Kerfoot, E., Wang, Y., Murrey, B., Myronenko, A., Zhao, C., Yang, D., et al.: Monai: An open-source framework for deep learning in healthcare. arXiv preprint arXiv:2211.02701 (2022)
  * [10] Chen, B., Liu, Y., Zhang, Z., Lu, G., Kong, A.W.K.: Transattunet: Multi-level attention-guided u-net with transformer for medical image segmentation. arXiv preprint arXiv:2107.05274 (2021)
  * [11] Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3d u-net: learning dense volumetric segmentation from sparse annotation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II 19. pp. 424–432. Springer (2016)
  * [12] Ciresan, D., Giusti, A., Gambardella, L., Schmidhuber, J.: Deep neural networks segment neuronal membranes in electron microscopy images. Advances in neural information processing systems 25 (2012)
  * [13] Du, X., Ma, K., Song, Y.: Agmr-net: Attention-guided multiscale recovery framework for stroke segmentation. Computerized Medical Imaging and Graphics 101, 102120 (2022)
  * [14] Elfadel, I.M., Wyatt Jr, J.L.: The” softmax” nonlinearity: Derivation using statistical mechanics and useful properties as a multiterminal analog circuit element. Advances in neural information processing systems 6 (1993)
  * [15] Grady, L.: Random walks for image segmentation. IEEE transactions on pattern analysis and machine intelligence 28(11), 1768–1783 (2006)
  * [16] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90
  * [17] Huo, J., Chen, L., Liu, Y., Boels, M., Granados, A., Ourselin, S., Sparks, R.: Mapping: Model average with post-processing for stroke lesion segmentation. arXiv preprint arXiv:2211.15486 (2022)
  * [18] Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J., Maier-Hein, K.H.: nnu-net: a self-configuring method for deep learning-based biomedical image segmentation. Nature methods 18(2), 203–211 (2021)
  * [19] Isensee, F., Maier-Hein, K.H.: An attempt at beating the 3d u-net. arXiv preprint arXiv:1908.02182 (2019)
  * [20] Johnson, W., Onuma, O., Owolabi, M., Sachdev, S.: Stroke: a global response is needed. Bulletin of the World Health Organization 94(9), 634 (2016)
  * [21] Kamnitsas, K., Ledig, C., Newcombe, V.F., Simpson, J.P., Kane, A.D., Menon, D.K., Rueckert, D., Glocker, B.: Efficient multi-scale 3d cnn with fully connected crf for accurate brain lesion segmentation. Medical image analysis 36, 61–78 (2017)
  * [22] Kim, J., Thayabaranathan, T., Donnan, G.A., Howard, G., Howard, V.J., Rothwell, P.M., Feigin, V., Norrving, B., Owolabi, M., Pandian, J., et al.: Global stroke statistics 2019. International Journal of Stroke 15(8), 819–838 (2020)
  * [23] Kohl, S., Romera-Paredes, B., Meyer, C., De Fauw, J., Ledsam, J.R., Maier-Hein, K., Eslami, S., Jimenez Rezende, D., Ronneberger, O.: A probabilistic u-net for segmentation of ambiguous images. Advances in neural information processing systems 31 (2018)
  * [24] Liew, S.L., Anglin, J.M., Banks, N.W., Sondag, M., Ito, K.L., Kim, H., Chan, J., Ito, J., Jung, C., Khoshab, N., et al.: A large, open source dataset of stroke anatomical brain images and manual lesion segmentations. Scientific data 5(1), 1–11 (2018)
  * [25] Liew, S.L., Lo, B.P., Donnelly, M.R., Zavaliangos-Petropulu, A., Jeong, J.N., Barisano, G., Hutton, A., Simon, J.P., Juliano, J.M., Suri, A., et al.: A large, curated, open-source stroke neuroimaging dataset to improve lesion segmentation algorithms. Scientific data 9(1), 320 (2022)
  * [26] Nodirov, J., Abdusalomov, A.B., Whangbo, T.K.: Attention 3d u-net with multiple skip connections for segmentation of brain tumor images. Sensors 22(17), 6501 (2022)
  * [27] Oktay, O., Schlemper, J., Folgoc, L.L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N.Y., Kainz, B., et al.: Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999 (2018)
  * [28] Paing, M.P., Tungjitkusolmun, S., Bui, T.H., Visitsattapongse, S., Pintavirooj, C.: Automated segmentation of infarct lesions in t1-weighted mri scans using variational mode decomposition and deep learning. Sensors 21(6), 1952 (2021)
  * [29] Petit, O., Thome, N., Rambour, C., Themyr, L., Collins, T., Soler, L.: U-net transformer: Self and cross attention for medical image segmentation. In: Machine Learning in Medical Imaging: 12th International Workshop, MLMI 2021, Held in Conjunction with MICCAI 2021, Strasbourg, France, September 27, 2021, Proceedings 12. pp. 267–276. Springer (2021)
  * [30] Qi, K., Yang, H., Li, C., Liu, Z., Wang, M., Liu, Q., Wang, S.: X-net: Brain stroke lesion segmentation based on depthwise separable convolution and long-range dependencies. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part III 22. pp. 247–255. Springer (2019)
  * [31] Rahman, A., Valanarasu, J.M.J., Hacihaliloglu, I., Patel, V.M.: Ambiguous medical image segmentation using diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 11536–11546 (2023)
  * [32] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. pp. 234–241. Springer (2015)
  * [33] Sheng, M., Xu, W., Yang, J., Chen, Z.: Cross-attention and deep supervision unet for lesion segmentation of chronic stroke. Frontiers in Neuroscience 16, 836412 (2022)
  * [34] Tomita, N., Jiang, S., Maeder, M.E., Hassanpour, S.: Automatic post-stroke lesion segmentation on mr images using 3d residual convolutional neural network. NeuroImage: clinical 27, 102276 (2020)
  * [35] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017)
  * [36] Verma, K., Kumar, S., Paydarfar, D.: Automatic segmentation and quantitative assessment of stroke lesions on mr images. Diagnostics 12(9), 2055 (2022)
  * [37] Wang, X., Han, S., Chen, Y., Gao, D., Vasconcelos, N.: Volumetric attention for 3d medical image segmentation and detection. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part VI 22. pp. 175–184. Springer (2019)
  * [38] Wilcoxon, F., Katti, S., Wilcox, R.A., et al.: Critical values and probability levels for the wilcoxon rank sum test and the wilcoxon signed rank test. Selected tables in mathematical statistics 1, 171–259 (1970)
  * [39] Wu, K., Du, B., Luo, M., Wen, H., Shen, Y., Feng, J.: Weakly supervised brain lesion segmentation via attentional representation learning. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part III 22. pp. 211–219. Springer (2019)
  * [40] Wu, Z., Zhang, X., Li, F., Wang, S., Huang, L., Li, J.: W-net: A boundary-enhanced segmentation network for stroke lesions. Expert Systems with Applications p. 120637 (2023)
  * [41] Yu, W., Fang, B., Liu, Y., Gao, M., Zheng, S., Wang, Y.: Liver vessels segmentation based on 3d residual u-net. In: 2019 IEEE international conference on image processing (ICIP). pp. 250–254. IEEE (2019)
  * [42] Yu, W., Huang, Z., Zhang, J., Shan, H.: San-net: Learning generalization to unseen sites for stroke lesion segmentation with self-adaptive normalization. Computers in Biology and Medicine 156, 106717 (2023)
  * [43] Zhang, Y., Wu, J., Liu, Y., Chen, Y., Wu, E.X., Tang, X.: Mi-unet: multi-inputs unet incorporating brain parcellation for stroke lesion segmentation from t1-weighted magnetic resonance images. IEEE Journal of Biomedical and Health Informatics 25(2), 526–535 (2020)
  * [44] Zhang, Z., Liu, Q., Wang, Y.: Road extraction by deep residual u-net. IEEE Geoscience and Remote Sensing Letters 15(5), 749–753 (2018)
  * [45] Zhou, Y., Huang, W., Dong, P., Xia, Y., Wang, S.: D-unet: a dimension-fusion u shape network for chronic stroke lesion segmentation. IEEE/ACM transactions on computational biology and bioinformatics 18(3), 940–950 (2019)

## Supplementary Material

#### Related Works:

This section explains each model that was detailed in Table 1 that elucidates
their underlying novelty.

Huo _et al._ [17] presented a stroke lesion segmentation based on the nnU-Net
framework and applied it to the ATLAS v2.0 dataset. They also introduced an
effective post-processing strategy to enhance segmentation metrics. Their
method achieved first place in the 2022 MICCAI ATLAS Challenge with impressive
scores, including an average Dice score of 0.6667 and a Lesion-wise F1 score
of 0.5643. Verma _et al._ [36] reported for the first time an automated lesion
segmentation model on the ATLAS v2.0 dataset which was a 3D U-Net architecture
similar to the nnU-Net based framework. The model achieved a Dice similarity
coefficient of 0.65. Yu _et al._ [42] proposed SAN-Net, a self-adaptive
normalization network and it also incorporates symmetry-inspired data
augmentation (SIDA) for improved generalization for unseen sites. Experimental
results show better performance compared to recent methods on the ATLAS v1.2
dataset.

Wu _et al._ [40] present a novel two-stage network called W-Net for lesion
segmentation in ischemic stroke using multi-modal MRI data. W-Net combines
CNN-transformer architecture, introduces a boundary deformation module (BDM)
to approximate the target boundary, and achieves the best performance in terms
of DSC, HD, and F2 metrics, with scores of 61.76%, 32.47, and 64.60%,
respectively, on the ATLAS v1.2 and ISLES2022 datasets. Du _et al._ [13]
proposed AGMR-Net, a novel method for stroke lesion segmentation, addressing
intraclass inconsistency and interclass indistinction challenges. AGMR-Net
utilizes a coarse-grained patch attention module, a cross-dimensional feature
fusion module, and a multiscale deconvolution upsampling module. AGMR-Net
achieved impressive results with a high Dice similarity coefficient of 0.594,
Hausdorff distance of 27.005 mm, and average symmetry surface distance of
7.137 mm, on the ATLAS v1.2 dataset.

Sheng _et al._ [33] proposes CADS-UNet, a novel cross-attention and deep
supervision UNet, for segmenting chronic stroke lesions from T1-weighted MR
images. The model incorporates a cross-spatial attention module to enhance
spatial focus, a channel attention mechanism to highlight channel
characteristics and deep supervision with mixed loss. The model was evaluated
on the ATLAS v1.2 with a Dice Similarity Coefficient (DSC) of 0.5564. Qi _et
al._ [30] introduced X-Net, a depthwise separable convolution-based model with
a Feature Similarity Module (FSM). They evaluated X-Net on the ATLAS v1.2,
achieving encouraging results with a Dice Similarity Coefficient (DSC) of
0.4867 and an Intersection over Union (IOU) of 0.3723. Zhuo _et al._ [45]
introduces D-UNet, a novel architecture combining 2D and 3D convolution in the
encoding stage with a new loss function called Enhance Mixing Loss (EML). The
proposed method was tested on the ATLAS v1.2 dataset, and it achieved a Dice
Similarity Coefficient (DSC) of 0.5349 $\pm$ 0.2763 and a precision of 0.6331
$\pm$ 0.295. Zhang _et al._ [43] developed a novel stroke lesion segmentation
approach called multi-inputs UNet (MI-UNet), which integrates brain
parcellation information (GM, WM, LV) with original MR image and achieves
notable segmentation performance with a Dice score of 56.72%, Hausdorff
distance of 23.94mm, average symmetric surface distance of 7.00mm, and
precision of 65.45%.

#### Evaluation Metrics:

For segmenting brain stroke lesions, we employed the mean Dice Similarity
Coefficient (DSC), Intersection over Union (IOU), Precision, and Recall as our
evaluation metrics. These metrics were utilized to assess the performance of
our segmentation models.

#### Dice Similarity Coefficient (DSC):

For the segmentation problem Dice Similarity Coefficient (DSC) is one of the
important parameters used to assess the similarity between predicted and
ground truth pixel or voxels values by measuring the overlap between two sets.
Where $X$ is the predicted set of pixels and $Y$ is the ground truth.

$\text{DSC}=\frac{2(X\cap Y)}{X+Y}$

#### Intersection Over Union (IOU):

IOU is an evaluation metric commonly used in image segmentation to assess the
performance of the model.

$\text{IOU}=\frac{(X\cap Y)}{(X\cup Y)}$

#### RECALL:

Also known as sensitivity or True Positive Rate $(TPR)$, is an important
evaluation metric used for segmentation. It measures the ability of a model to
correctly identify positive instances. Recall qualifies how well the model
captures the relevant pixel or voxels that belong to the target class.

$\text{Recall}=\frac{TP}{(TP+FN)}$

#### PRECISION:

Precision measures how well the model performs in correctly identifying the
relevant pixels or voxels that belong to the target class out of all the
pixels or voxels predicted as positive.

$\text{Precision}=\frac{TP}{(TP+FP)}$

#### Wilcoxon Signed Rank Test Primer:

The Wilcoxon Signed Rank Test is a non-parametric statistical test used to
compare two related or paired samples. It assesses whether there is a
significant difference between the paired observations. It does this by
ranking the absolute differences between paired values, summing positive and
negative ranks separately, and comparing these sums using a specific
statistical distribution. If the test statistic is significant, it indicates a
significant difference between the two groups. It’s often used when data
doesn’t meet the assumptions of parametric tests like the t-test. The null
hypothesis of the Wilcoxon test is that there is no difference between the
distributions of the two groups, while the alternative hypothesis suggests
that there is a significant difference [38]. The Python library SciPy is used
to generate the results for the Wilcoxon test:
https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.wilcoxon.html.

#### Loss Function

In medical image analysis, we often use a combination of Dice Loss (DICE) and
Binary Cross-Entropy Loss (BCE) for segmentation tasks. Dice Loss quantifies
the overlap between predicted and actual regions of interest, while Binary
Cross-Entropy Loss measures the dissimilarity between predicted probabilities
and ground truth labels. By combining these two losses, we can effectively
train deep learning models to segment medical images, helping in stroke lesion
segmentation accurately. However, we have provided proportionate weightage for
each of the loss functions, and the combined loss is described in the equation
(6).

$\mathcal{L}_{DICE}=1-\frac{2\cdot\sum_{i=1}^{N}p_{i}\cdot
g_{i}}{\sum_{i=1}^{N}p_{i}^{2}+\sum_{i=1}^{N}g_{i}^{2}}$ (4)

$\mathcal{L}_{BCE}=-\frac{1}{N}\sum_{i=1}^{N}[g_{i}\cdot\log(p_{i})+(1-g_{i})\cdot\log(1-p_{i})]$
(5)

$\mathcal{L}_{OURS}=\gamma\mathcal{L}_{DICE}+(1-\gamma)\mathcal{L}_{BCE}$ (6)

Where $p_{i}$ is the predicted probability of a pixel/voxel being in the ROI,
$g_{i}$ is the corresponding ground truth label (1 for ROI, 0 for background),
and $N$ is the number of pixels/voxels. Here, $\gamma$ is the weightage
parameter, and we have obtained the best results for $\gamma=0.9$, and thus,
all the models (both 2D and 3D) have used this value as default for a fair
evaluation.

#### 2D Parameter description:

We implemented five U-Net style architectures for 2D image segmentation,
namely, 2D U-Net, ResU-Net, Attention U-Net, TransAttn U-Net, and U-Net
Transformer. These models possess approximately 31 M, 32 M, 34 M, 25 M, and 11
M parameters, respectively, and have 4 levels of up and down samplings, except
for the U-Net Transformer, which utilizes 3 levels of ups and downsamplings.

A preprocessing step was implemented by cropping the images (coordinates: (10,
40) and (190, 220)) to focus on relevant regions and resizing them to 192x192
pixels via bilinear interpolation, we standardized the input dimensions and
emphasized salient features associated with stroke lesions. During the
training process, we employed the Adam optimizer with an initial learning rate
of 0.001. To optimize learning adaptability, we incorporated the
”ReduceLROnPlateau” learning rate schedule, which dynamically adjusted the
learning. Dropout regularization with a probability of 0.2 was employed to
mitigate overfitting during training and these models were trained for 50
epochs, utilizing a batch size of 32 for all models except for the U-Net
Transformer, which had a batch size of 16. The 2D architecture of each model
consists of multiple convolutional layers with varying numbers of channels
(64, 128, 256, 512, 1024) to effectively capture diverse and intricate
features in the data. However, the U-Net Transformer model has (64, 128, 256,
and 512) channels. Notably, data augmentations were not applied to the all 2D
U-Net models.

#### 3D Parameter description:

Here we implemented three state-of-the-art 3D models for image segmentation:
U-Net 3D, Res U-Net 3D, and Attention U-Net 3D. Each model’s number of
learnable parameters was meticulously assessed, revealing U-Net 3D to possess
1.40 M parameters, Res U-Net 3D with 1.42 M parameters, and Attention U-Net 3D
featuring 1.61 M parameters. These models were intricately designed with 3
levels of up-and-down samplings. Regarding input image and mask dimensions,
both U-Net 3D and Res U-Net 3D were configured with image dimensions of
$144\times 172\times 128$, while Attention U-Net 3D adopted the dimensions of
$144\times 176\times 128$.

The Adam optimizer is employed for training the models, with an initial
learning rate of 0.001 to facilitate convergence. The learning rate schedule
”Cosine AnnealingLR” is applied to all models, with ADAM optimization. For
training and validation batches, a batch size of 4 is utilized for all models.
Additionally, the early stopping criterion is set at 100 epochs for all models
to control training duration. Regarding the number of channels used, the
models incorporate 16, 32, 64, and 128 channels, allowing them to capture
diverse and intricate image features. To prevent overfitting, a weight decay
value of 0.0001 is employed for all models. No data augmentations are applied
during training.

Table 6: Wilcoxon and Pearson correlation test for actual and predicted stroke volumes for 3D standard, Residual, and Attention U-Net architectures. Method | Wilcoxon Test | Pearson Correlation
---|---|---
| U-statistic | p-value | statistic | p-value
U-Net [32] | 6353.0 | $\approx 0.0$ | 0.949 | $\approx 0.0$
Residual U-Net [44] | 5729.5 | 0.001 | 0.962 | $\approx 0.0$
Attention U-Net [27] | 4279.0 | 0.540 | 0.844 | $\approx 0.0$

#### Visualization Source:

For, the 3D visualizations the authors have used Ni-Learn framework:
https://nilearn.github.io/dev/index.html and for 2D visualizations Matplotlib
framework: https://matplotlib.org/.

#### Concerns of HD95 Evaluation Metric:

We are thankful and are obliged from the fruitful reviews provided by
anonymous reviewers. One of the reviewers suggested using the HD95 score as
one of the evaluation metrics. However, the evaluation metrics used in our
study are well-established in the literature and accepted across various
domains in medical image segmentation. Apart from these metrics, HD95 provides
surface-based information and provides additional insights into the
delineation of stroke. Due to time constraints, we were not able to provide
them. But, we are very much willing to add those in our future work.


# Sufficient Conditions for Persistency of Excitation with Step and ReLU
Activation Functions

Tyler Lekang and Andrew Lamperski This work was supported in part by NSF
CMMI-2122856T. Lekang and A. Lamperski are with the department of Electrical
and Computer Engineering, University of Minnesota, Minneapolis, MN 55455, USA
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

This paper defines geometric criteria which are then used to establish
sufficient conditions for persistency of excitation with vector functions
constructed from single hidden-layer neural networks with step or ReLU
activation functions. We show that these conditions hold when employing
reference system tracking, as is commonly done in adaptive control. We
demonstrate the results numerically on a system with linearly parameterized
activations of this type and show that the parameter estimates converge to the
true values with the sufficient conditions met.

##  I Introduction

Persistency of excitation is a fundamental concept employed within contexts
and applications related to parameter learning, such as system identification
and adaptive control. It is often discussed, or at least mentioned, in
adaptive control textbooks, such as in [1, 2, 3].

It was proven in [4, 5] that persistency of excitation is necessary and
sufficient for the global uniform asymptotic stability of the linear time-
varying (LTV) system

$\dot{\tilde{\theta}}_{t}=-V_{t}\,V_{t}^{\top}\tilde{\theta}_{t}$ (1)

where $\tilde{\theta}_{t}\in\mathbb{R}^{n}$ is the system state, and
$V_{t}\in\mathbb{R}^{n\times d}$ is a vector ($d=1$) or matrix ($d\geq 2$)
function of time that is regulated (one-sided limits exist for all
$t\in[0,\infty)$). Consider if $\tilde{\theta}_{t}={\hat{\theta}}_{t}-\theta$
represents the error of a parameter estimate ${\hat{\theta}}_{t}$ from some
fixed, unknown parameter values $\theta$. Then if $V_{t}$ is persistently
exciting, the state of this system (ie, the error of the parameter estimates)
converges globally uniformly asymptotically to zero.

In (non)linear systems with linear parameterizations, the parameter estimate
error dynamics commonly have the form (1). For example, in [2] (sec 8.7), we
see examples of systems that can be formed into a model $y_{t}=V_{t}\,\theta$,
where the vector $y_{t}$ and the matrix $V_{t}$ are measurable, and then using
a simple gradient-based update rule for the parameter estimate
${\hat{\theta}}_{t}$, within an estimator system
$\hat{y}_{t}=V_{t}\,{\hat{\theta}}_{t}$, gives exactly these dynamics for the
parameter estimate error ${\hat{\theta}}_{t}-\theta$. Thus, if $V_{t}$ is
persistently exciting, the parameter estimate error will converge to zero.

There have been many works since [4, 5] which utilize an assumption of
persistency of excitation in order to achieve results in parameter learning.
In [6], a simple parameter learning scheme can be employed for a general class
of nonlinear systems which have some kind of working (nonlinear) feedback
controller. An integral condition similar to persistency of excitation is
assumed to be satisfiable. This inspired the work in [7], which assumes a
similar integral condition in order to identify, and then provide MRAC control
for, an unknown MIMO LTI system. And in [8, 3] we see additional examples of
MRAC control which assume persistency of excitation in order to achieve
parameter convergence, while in [9], persistently exciting assumptions are
made in reinforcement learning applications. In [10], a sufficient condition
for windows of observed behavior of an LTI system (in discrete time) to span
the space of possible windows, is for a component signal (like the input) to
be persistently exciting. In [11], conditions for neural networks excitation
are given to guarantee bounds on the function estimate error. Lastly, in [12]
it is proven that for a general class of nonlinear systems which are feedback
linearizable (see [13, 14]), global uniform asymptotic stability can be
achieved for linearly paramatrized vector functions meeting relaxed
persistency of excitation conditions.

However, in these and other works which utilize persistency of excitation
assumptions, there is often no explicit sufficient conditions provided for how
to ensure that persistency of excitation is satisfied.

On the other hand, there have been some works which do provide these
sufficient conditions. A classic result is that the state of an LTI system
satisfies persistence of excitation if the (stationary) input to that system
contains sufficient frequency content (“sufficient richness”, see [15]). In
[16, 17] this is extended to certain linear time-varying systems, while in
[18] frequency arguments for sufficient conditions for excitation are then
extended to nonlinear systems in parametric-strict-feedback form and in [19]
to the context of adaptive dynamic programming, in which optimal control value
functions are approximated using polynomial basis functions. In [20], a rank
condition is proven sufficient and necessary for the state of time-invariant
systems to be persistently exciting, and in [21], strong Lyapunov functions
are provided that are equivalent to the persistency of excitation condition.
Finally, and closest in spirit to this paper, in [22] a sufficient condition,
based on geometric criteria, is given for satisfying persistency of excitation
with vector functions composed of radial basis functions.

Our primary contributions are sufficient conditions, based on geometric
criteria, for satisfying persistency of excitation with vector functions
$\phi:\mathbb{R}^{n}\to\mathbb{R}^{N}$ which are composed of ReLU or step
activation functions, together with affine transformations of the state space.
We then demonstrate this using a simulated MRAC control application with the
parameter estimates converging to the true values.

The organization of the remaining parts of the paper are as follows. Section
II provides preliminary notation. Section III sets up problem definitions and
geometry. Section IV presents the main theoretical results, while Section V
presents numerical results, and closing remarks are given in Section VI.

##  II Notation

We interpret $w,x\in\mathbb{R}^{n}$ as column vectors and denote their inner
product as $w^{\top}\\!x$. Similarly, we denote the product of matrix
$W\in\mathbb{R}^{n\times N}$ with vector $x$ as $W^{\top}\\!x$, which is a
length $N$ vector where the $i$th (row) element is the inner product
$W_{i}^{\top}\\!x$. Index subscripts on vectors and matrices denote the row
index, for example $W_{i}^{\top}$ is the $i$th row of $W^{\top}$. The standard
Euclidean norm and Frobenius matrix norm are respectively denoted $\|w\|_{2}$
and $\|W\|_{F}$. The integer set $\\{1,\dots,k\\}$ is denoted by $[k]$. We use
$t$ subscripts on time-dependent variables to reduce parentheses, for example
$\phi(x(t))$ is instead denoted $\phi(x_{t})$. The $i$th row of a time-varying
vector or matrix is thus denoted with $t,i$ subscript. For square $n\times n$
matrix $A$, we use $aI_{n}\preceq A\preceq bI_{n}$ to denote that
$\text{eig}(A)\in[a,b]$, where $I_{n}$ denotes the $n\times n$ identity
matrix. We denote the $n$-length zeros vector as $0_{n}$.

##  III Setup

This section formally defines the nonlinear vector function $\phi$ and what
persistency of excitation means with regards to this definition, then
describes the geometry induced on the state space $\mathbb{R}^{n}$ by this
construction.

###  III-A Nonlinear, Positive Semidefinite Activation Functions

Let $\phi:\mathbb{R}^{n}\to\mathbb{R}^{N}$ be a vector function defined as

$\phi(x)=\begin{bmatrix}\phi_{1}(x)\\\ \vdots\\\
\phi_{N}(x)\end{bmatrix}=\begin{bmatrix}\sigma(w_{1}^{\top}\\!x+b_{1})\\\
\vdots\\\ \sigma(w_{N}^{\top}x+b_{N})\end{bmatrix}\ ,$ (2)

where $\phi_{1},\dots,\phi_{N}:\mathbb{R}^{n}\to\mathbb{R}$ are composed of
nonlinear, piecewise continuous functions $\sigma:\mathbb{R}\to\mathbb{R}$
together with affine transformations
$w_{1}^{\top}\\!x+b_{1},\dots,w_{N}^{\top}x+b_{N}:\mathbb{R}^{n}\to\mathbb{R}$.
We allow $w_{i}\in\mathbb{R}^{n}\setminus\\{0_{n}\\}$ and $b_{i}\in\mathbb{R}$
to be arbitrary for all $i\in[N]$, except we assume each
$w_{i}^{\top}\\!x+b_{i}=0$ hyperplane in $\mathbb{R}^{n}$ is unique with
dimension $n-1$. Let $W=\begin{bmatrix}w_{1}&\cdots&w_{N}\end{bmatrix}$ and
$b=\begin{bmatrix}b_{1}&\cdots&b_{N}\end{bmatrix}^{\top}$. For any
$\mathscr{S}\subset[N]$, let $W^{\top}_{\mathscr{S}}$ and be the submatrix of
$W^{\top}$ with rows given by $w_{i}^{\top}$ for $i\in\mathscr{S}$. Define
$b_{\mathscr{S}}$ similarly for $b$.

Note then that (2) is equivalent to the output of a single hidden neural
network layer with $N$ neurons fully connected to the input
$x\in\mathbb{R}^{n}$, having nonlinear (eg, ReLU) activations, and being
initialized with weights and biases defining unique hyperplanes. Hence why we
refer to $\sigma$ as an activation function or simply an activation. This
paper will focus on the following activations:

$\displaystyle\sigma_{cs}(y)$ $\displaystyle=\begin{cases*}0\quad\text{if
}y\leq 0\\\ c\quad\text{if }y>0\end{cases*}\qquad\text{(scaled step)}$ (3)
$\displaystyle\sigma_{r}(y)$ $\displaystyle=\begin{cases*}0\quad\text{if
}y\leq 0\\\ y\quad\text{if }y>0\end{cases*}\qquad\text{(ReLU)}\ \ ,$ (4)

where $c>0$ is an arbitrary positive scalar.

###  III-B Persistency of Excitation

Let $x:[0,\infty)\to\mathbb{R}^{n}$ be some continuous trajectory in the state
space. Then, $\phi(x_{t})$ is a piecewise continuous (and regulated) vector
function of time, mapping $[0,\infty)\to\mathbb{R}^{N}$. For any time window
$t\in[\tau,\tau+T]$ with $\tau\geq 0$ and $T>0$, the integral

$\int_{\tau}^{\tau+T}\phi(x_{t})\,\phi(x_{t})^{\top}\,\text{dt}$ (5)

defines a $N\times N$ Gramian matrix since the corresponding $i,j$-th entry
$\int_{\tau}^{\tau+T}\phi_{i}(x_{t})\,\phi_{j}(x_{t})\,\text{dt}$ is an inner
product of the composition functions
$\phi_{1}(x_{t}),\dots,\phi_{N}(x_{t}):[0,\infty)\to\mathbb{R}$. Gramian
matrices are always positive semidefinite, which can be shown using the bi-
linearity of inner products.

Persistency of excitation is the requirement that the Gramian matrix (5) must
be strictly positive definite, with eigenvalues in some bounded interval
$[\alpha_{1},\alpha_{2}]$, over all shifts $\tau\geq 0$ of the sliding time
window $[\tau,\tau+T]$ for some window length $T>0$. Formally, persistency of
excitation requires existence of constants $\alpha_{1},\alpha_{2},T>0$ such
that

$\alpha_{1}I_{N}\ \preceq\
\int_{\tau}^{\tau+T}\phi(x_{t})\,\phi(x_{t})^{\top}\,\text{dt}\ \preceq\
\alpha_{2}I_{N}$ (6)

holds for all $\tau\geq 0$. An equivalent scalar requirement is that

$\alpha_{1}\|v\|_{2}^{2}\ \leq\
\int_{\tau}^{\tau+T}(v^{\top}\phi(x_{t}))^{2}\,\text{dt}\ \leq\
\alpha_{2}\|v\|_{2}^{2}$ (7)

must hold for all $v\in\mathbb{R}^{N}$ and $\tau\geq 0$. This follows since
$v^{\top}\phi(x_{t})=\phi(x_{t})^{\top}v$.

Note that if these hold for some $T>0$, then they also hold for all
$\widetilde{T}>T$. This follows because the integrals can be broken into a sum
of two integrals over $t\in[\tau,\tau+T]$ and
$t\in[\tau+T,\tau+\widetilde{T}]$, with the former being strictly positive
(definite) and the latter positive semidefinite (nonegative).

###  III-C Activation Geometry

We define the following subsets for all activations $i\in[N]$:

$\displaystyle X_{i}^{+}$ $\displaystyle=\\{x\in\mathbb{R}^{n}\ |\
w_{i}^{\top}\\!x+b_{i}>0\\}\qquad\text{(active)}$ $\displaystyle
X_{i}^{\circ}$ $\displaystyle=\\{x\in\mathbb{R}^{n}\ |\
w_{i}^{\top}\\!x+b_{i}\leq 0\\}\qquad\text{(zero)}\ \ .$

Each is a half-space of $\mathbb{R}^{n}$ formed by the hyperplane
$w_{i}^{\top}\\!x+b_{i}=0$. These are then used to define the activation
regions $\mathcal{A}_{j}$, with indices corresponding to a binary string
indicating which active (binary 1) and zero (binary 0) half-spaces are in the
intersection:

$\displaystyle\mathcal{A}_{0}$ $\displaystyle=X_{N}^{\circ}\ \cap\
X_{N-1}^{\circ}\ \cap\ \dots\ \cap\ X_{2}^{\circ}\cap\ X_{1}^{\circ}$
$\displaystyle\mathcal{A}_{1}$ $\displaystyle=X_{N}^{\circ}\ \cap\
X_{N-1}^{\circ}\ \cap\ \dots\ \cap\ X_{2}^{\circ}\cap\ X_{1}^{+}$
$\displaystyle\hskip 6.0pt\vdots$ $\displaystyle\mathcal{A}_{2^{N}-1}$
$\displaystyle=X_{N}^{+}\ \cap\ X_{N-1}^{+}\ \cap\ \dots\ \cap\ X_{2}^{+}\cap\
X_{1}^{+}\ \ .$

These partition $\mathbb{R}^{n}$ into, at most, $2^{N}$ convex polytopes. It
is likely that some of the regions will be infeasible
($\mathcal{A}_{j}=\emptyset$). For example, in the $n=1$ case, there are
always only $N+1$ feasible activation regions, since there will be $N$ unique
(by assumption) points partitioning the $\mathbb{R}$ line. In higher
dimensions, more feasible regions are possible.

For all $j\in\\{0,\dots,2N-1\\}$, we define the active set $\mathscr{S}_{j}$
as

$\mathscr{S}_{j}=\\{i\in[N]\ |\ w_{i}^{\top}\\!x+b_{i}>0\quad\forall
x\in\mathcal{A}_{j}\\}\ \ .$

This captures which of the $N$ activations are active in a particular
activation region. For any two activation regions $j,k\in\\{0,\dots,2N-1\\}$
with a nonempty intersection
$\mathcal{A}_{j}\cap\mathcal{A}_{k}\neq\emptyset$, their intersection defines
a border between the two regions. We define a nondegenerate border to mean
$\dim(\mathcal{A}_{j}\cap\mathcal{A}_{k})=n-1$. In this case, only one
activation is different (active to zero or vice-versa) between those regions.
This is because the borders between activation regions must be (subsets of)
the hyperplanes that define the half-spaces $X_{i}$, and the intersection of
more than one unique hyperplane with dimension $n-1$ must have a dimension
less than $n-1$. Thus, a nondegenerate border must be a subset of (or equal
to) a single hyperplane, meaning only one $X_{i}$ half-space can flip from
active to zero or vice-versa. We then define a degenerate border to mean
$\dim(\mathcal{A}_{j}\cap\mathcal{A}_{k})<n-1$. In this case, the border is a
(subset of) the lower dimensional intersection of two or more unique
hyperplanes. Thus, multiple activations are different between the regions.

Now consider a continuous state trajectory $x_{t}\in\mathbb{R}^{n}$ visiting a
sequence of activation regions over the time window $[\tau,\tau+T]$, for some
$\tau\geq 0$ and $T>0$. Assume $x_{t}$ only crosses nondegenerate borders and
that the number of regions visited is $\mathscr{L}\geq 2$. Let us denote
$\mathscr{A}_{1},\dots,\mathscr{A}_{\mathscr{L}}$ as the activation region
indices of the visited sequence and
$\mathscr{I}_{1},\dots,\mathscr{I}_{\mathscr{L}-1}$ as the sequence of
activation indices corresponding to the hyperplanes crossed in order to visit
that sequence of activation regions. For all $s\in[\mathscr{L}]$, we define
time window subsets $\mathcal{T}_{s}\subset[\tau,\tau+T]$ as

$\mathcal{T}_{s}=\\{t\in[\tau,\tau+T]\ |\
x_{t}\in\mathcal{A}_{\mathscr{A}_{s}}\\}\ \ .$ (8)

Since we assume $x_{t}$ crosses only nondegenerate borders when visiting the
activation regions during the time window, we have by definition over all
$s\in[\mathscr{L}-1]$ that

$\mathscr{S}_{\mathscr{A}_{s+1}}=\begin{cases}\mathscr{S}_{\mathscr{A}_{s}}\bigcup\
\\{\mathscr{I}_{s}\\}\qquad\text{if}\ \
\mathcal{A}_{\mathscr{A}_{s+1}}\subseteq X_{\mathscr{I}_{s}}^{+}\\\
\mathscr{S}_{\mathscr{A}_{s}}\setminus\ \\{\mathscr{I}_{s}\\}\qquad\text{if}\
\ \mathcal{A}_{\mathscr{A}_{s+1}}\subseteq
X_{\mathscr{I}_{s}}^{\circ}\end{cases}.$ (9)

##  IV Theoretical Results

This section presents our main theoretical results, which provide sufficient
conditions for satisfying persistency of excitation with (scaled) step or ReLU
activations.

###  IV-A Main Results

###### Theorem 1

Let state trajectory $x_{t}$ be continuous and stay within some compact set
$\mathcal{B}\subset\mathbb{R}^{n}$ for all $t\geq 0$, and let
$\phi(x_{t})=[\sigma_{cs}(w_{1}^{\top}x_{t}+b_{1}),\dots,\sigma_{cs}(w_{N}^{\top}x_{t}+b_{N})]$
be composed of (scaled) step functions (3) with positive scalars
$c=[c_{1},\dots,c_{N}]^{\top}$ together with $N$ unique affine transformations
of $\mathbb{R}^{n}$ according to
$w_{1},\dots,w_{N}\in\mathbb{R}^{n}\setminus\\{0_{n}\\}$ and
$b_{1},\dots,b_{N}\in\mathbb{R}$. If $x_{t}$ over $t\geq 0$ is such that there
exists a window length $T^{*}>0$ whereby the sequence $s\in[\mathscr{L}]$ of
activation regions visited during any shift $\tau\geq 0$ of the time window
$[\tau,\tau+T^{*}]$ always satisfies the following two conditions:
1\. all $i\in[N]$ hyperplanes $w_{i}^{\top}x_{t}+b_{i}=0$ are crossed
2\. only nondegenerate borders are crossed,
then $\phi(x_{t})$ satisfies the persistency of excitation conditions (6) and
(7).

###### Proof:

Given in Appendix A. ∎

###### Theorem 2

Let state trajectory $x_{t}$ be continuous and stay within some compact set
$\mathcal{B}\subset\mathbb{R}^{n}$ for all $t\geq 0$, and let
$\phi(x_{t})=[\sigma_{r}(w_{1}^{\top}x_{t}+b_{1}),\dots,\sigma_{r}(w_{N}^{\top}x_{t}+b_{N})]$
be comprised of ReLU functions (4) together with $N$ unique affine
transformations of $\mathbb{R}^{n}$ according to
$w_{1},\dots,w_{N}\in\mathbb{R}^{n}\setminus\\{0_{n}\\}$ and
$b_{1},\dots,b_{N}\in\mathbb{R}$. If $x_{t}$ over $t\geq 0$ is such that there
exists a window length $T^{*}>0$ whereby the sequence $s\in[\mathscr{L}]$ of
activation regions visited during any shift $\tau\geq 0$ of the time window
$[\tau,\tau+T^{*}]$ always satisfies the following three conditions:
1\. all $i\in[N]$ hyperplanes $w_{i}^{\top}x_{t}+b_{i}=0$ are crossed
2\. only nondegenerate borders are crossed
3\. for each $s\in[\mathscr{L}]$, there are times
$t_{1},\hat{t}_{1},\ldots,t_{M},\hat{t}_{M}\in\mathcal{T}_{s}$ such that

$\mathrm{rank}(W^{\top}_{\mathcal{A}_{\mathscr{A}_{s}}})=\mathrm{rank}\left(W^{\top}_{\mathcal{A}_{\mathscr{A}_{s}}}[x_{t_{1}}-x_{\hat{t}_{1}}\
\cdots\ x_{t_{M}}-x_{\hat{t}_{M}}]\right),$

then $\phi(x_{t})$ satisfies the persistency of excitation conditions (6) and
(7).

###### Proof:

Given in Appendix A. ∎

Note that a sufficient condition for property (iii) is that
$\begin{bmatrix}x_{t_{1}}-x_{\hat{t}_{1}}&\cdots&x_{t_{M}}-x_{\hat{t}_{M}}\end{bmatrix}$
has rank $n$. This can be achieved, for example, if $x_{t}$ is the state
trajectory of a system that satisfies a suitable (local) controllability
property.

###  IV-B Proof Sketch

Both proofs rely on the same overall contradiction method. That is, we assume
there exists a nonzero vector $v\in\mathbb{R}^{N}\setminus\\{0_{N}\\}$ such
that

$v^{\top}\left(\int_{\tau}^{\tau+T}\phi(x_{t})\,\phi(x_{t})^{\top}\text{dt}\right)v\
=\ 0\ \ .$ (10)

We show that if the state trajectory $x_{t}$ meets certain requirements over
any shift $\tau\geq 0$ of the window $[\tau,\tau+T^{*}]$, for some window
length $T^{*}>0$, then in fact (10) can only hold if $v=0_{N}$. This is a
contradiction and thus proves that the LHS integral must be strictly positive
definite, uniformly over all windows $[\tau,\tau+T^{*}]$ for all $\tau\geq 0$.

###  IV-C Extension to Other Activation Functions

In Appendix A we discuss how these methods could be applied similarly to other
nonlinear, positive semidefinite activation functions $\sigma$, such that
$\sigma(y)=0$ for all $y\leq 0$ and $\sigma(y)>0$ for all $y>0$.

##  V Numerical Results

In this section we provide a numerical simulation111All code is available at:
https://github.com/tylerlekang/CDC2022 of the theoretical results, using a
MRAC application which is a variation on the setup from Chapter 9 of [3]. The
plant and reference systems have $n=2$ states, allowing convenient
visualization of the hyperplanes and state space.

###  V-A Setup

The plant is given by

$\dot{x}_{t}=A\,x_{t}+B\big{(}u_{t}+\Theta^{\top}\\!\phi(x_{t})\big{)}\ \ ,$
(11)

where $A$ is a known $n\times n$ state matrix for the plant state
$x_{t}\in\mathbb{R}^{n}$, $B$ is a known $n\times\ell$ input matrix for the
input $u_{t}\in\mathbb{R}^{\ell}$, and $\Theta$ is an unknown $N\times\ell$
matrix which linearly parameterizes the known vector function
$\phi:\mathbb{R}^{n}\to\mathbb{R}^{N}$ defined by (2). The setup in [3] also
includes an unknown diagonal scaling matrix $\Lambda$, such that the overall
input matrix is $B\Lambda$. We have omitted this for simplicity.

The control input $u_{t}$ will be designed in order to force the plant states
$x_{t}$ to track the states of a reference system $x_{t}^{r}$ that is driven
by a bounded reference input $r_{t}$. The reference system is given by

$\dot{x}_{t}^{r}=A_{r}\,x_{t}^{r}+B_{r}\,r_{t}\ \ ,$ (12)

where $A_{r}$ and $B_{r}$ are known reference matrices, with $A_{r}$ Hurwitz,
and $r_{t}\in\mathbb{R}^{\ell}$ is a bounded reference input.

We assume there exists an $n\times\ell$ matrix of feedback gains $K_{x}$ and
an $\ell\times\ell$ matrix of feedforward gains $K_{r}$ satisfying the
matching conditions

$A+BK_{x}^{\top}=A_{r}\quad\text{and}\quad BK_{r}^{\top}=B_{r}\ \ .$ (13)

The setup in [3] has $A$ and $\Lambda$ as unknown, and thus $K_{x}$ and
$K_{r}$ need to be estimated. For this simulation, we will assume that $K_{x}$
and $K_{r}$ can be directly calculated from known $A$ and $B$, and used within
the control law.

Next, we introduce parameter estimates ${\widehat{\Theta}}_{t}$, which will be
dynamically updated to estimate true parameter values $\Theta$. Thus, by
applying to the plant (18) the feedback control law
$u_{t}=K_{x}^{\top}x_{t}+K_{r}^{\top}r_{t}-{\widehat{\Theta}}_{t}^{\top}\\!\phi(x_{t})$,
the plant dynamics become

$\dot{x}_{t}=A_{r}\,x_{t}+B_{r}\,r_{t}-B\,({\widehat{\Theta}}_{t}-\Theta)^{\top}\\!\phi(x_{t})\
\ .$ (14)

This in turn gives the dynamics of the state tracking error
$e_{t}=x_{t}-x_{t}^{r}$ as

$\dot{e}_{t}=\dot{x}_{t}-\dot{x}_{t}^{r}=A_{r}\,e_{t}-B\,({\widehat{\Theta}}_{t}-\Theta)^{\top}\\!\phi(x_{t})\
\ .$ (15)

In [3], it is then shown that these state tracking error dynamics
$\dot{e}_{t}$ are globally uniformly asymptotically stable, such that
$\lim_{t\to\infty}\|e_{t}\|_{2}=0$, if the parameter estimates are dynamically
updated as

$\dot{{\widehat{\Theta}}}_{t}=\Gamma\,\phi(x_{t})\,e_{t}^{\top}P_{x}B\ \ .$
(16)

This is shown by analyzing the Lyapunov function
$\mathscr{V}=e_{t}^{\top}P_{x}e_{t}+\text{tr}\big{(}({\widehat{\Theta}}_{t}-\Theta)^{\top}\Gamma^{-1}({\widehat{\Theta}}_{t}-\Theta)\big{)}$,
along with using Barbalat’s lemma, such that the update rule (16) results in
$\dot{\mathscr{V}}\leq-e_{t}^{\top}Q_{x}e_{t}$ for all values of $e_{t}$ and
${\widehat{\Theta}}_{t}-\Theta$. Here, $P_{x}$ is the unique symmetric,
positive-definite $n\times n$ matrix that solves the algebraic Lyapunov
equation $P_{x}A_{r}+A_{r}^{\top}P_{x}=-Q_{x}$ for some symmetric, positive-
definite $n\times n$ matrix $Q_{x}$, and adaptation rates $\Gamma$ is some
symmetric, positive-definite $N\times N$ matrix, where we denote
$\|\Gamma\|_{F}:=\mathscr{G}$.

###### Remark 1

In the case where $K_{x}$ and $K_{r}$ are also being estimated by
$\hat{K}_{x,t}$ and $\hat{K}_{r,t}$ respectively, the true values can be
appended to $\Theta$, the estimates can be appended to
${\widehat{\Theta}}_{t}$, and an overall vector function $\Phi(x_{t},r_{t})$
which combines $x_{t}$, $r_{t}$, and $\phi(x_{t})$, can be formed. However,
such $\Phi$ then do not strictly meet the definition of (2), and are thus
beyond the scope of this paper.

###  V-B Persistency of Excitation

The dynamic update rule (16) only guarantees asymptotic convergence of the
state tracking error $e_{t}$ to zero. We now show that the parameter
estimation error, which we will denote compactly as
$\widetilde{\Theta}_{t}={\widehat{\Theta}}_{t}-\Theta$, also goes to zero if
$\phi(x_{t})$ is persistently exciting.

Since $A_{r}$ is Hurwitz, it is guaranteed invertible. And so, from (15) we
have

$e_{t}^{\top}P_{x}B=\dot{e}_{t}^{\top}{A_{r}^{-1}}^{\top}P_{x}B+\phi(x_{t})^{\top}\widetilde{\Theta}_{t}\,B^{\top}{A_{r}^{-1}}^{\top}P_{x}B$

and then combined with (16) we get

$\displaystyle\dot{\widetilde{\Theta}}_{t}=\dot{{\widehat{\Theta}}}_{t}=\Gamma\,\phi(x_{t})\,e_{t}^{\top}P_{x}B=$
$\displaystyle\Gamma\,\phi(x_{t})\phi(x_{t})^{\top}\widetilde{\Theta}_{t}\,B^{\top}{A_{r}^{-1}}^{\top}P_{x}B+\Gamma\,\phi(x_{t})\,\dot{e}_{t}^{\top}{A_{r}^{-1}}^{\top}P_{x}B\
.$

Note that the second term asymptotically goes to zero with $\dot{e}_{t}$ and
we have in the first term $B^{\top}{A_{r}^{-1}}^{\top}P_{x}B=$
$-\frac{1}{2}B^{\top}{A_{r}^{-1}}^{\top}Q_{x}\,A_{r}^{-1}B$ by the Lyapunov
equation definition for $P_{x}$.

Let us now restrict to the case of $\ell=1$. Since $Q_{x}$ is positive-
definite we have the positive scalar
$c=\frac{1}{2}B^{\top}{A_{r}^{-1}}^{\top}Q_{x}\,A_{r}^{-1}B$, and obtain that
the dynamics of the parameter estimation error vector
$\widetilde{\Theta}_{t}\in\mathbb{R}^{N}$ asymptotically tend to the linear
time-varying system

$\dot{\widetilde{\Theta}}_{t}=-\Gamma_{c}\,\phi(x_{t})\phi(x_{t})^{\top}\widetilde{\Theta}_{t}\
\ .$ (17)

$\Gamma_{c}=c\,\Gamma$ is symmetric, positive-definite with
$\|\Gamma_{c}\|_{F}=c\mathscr{G}$.

Therefore, if $\phi(x_{t})$ satisfies the persistency of excitation condition
(6), then the parameter estimation error dynamics (17) are globally uniformly
asymptotically stable such that we have
$\lim_{t\to\infty}\|\widetilde{\Theta}_{t}\|_{2}=0$. We show explicitly in
Appendix B how this follows from Lemmas 1 and 2 and Theorem 1 of [5].

###  V-C Simulation

We perform the numerical simulation in the $x_{t}\in\mathbb{R}^{2}$ state
space. Euler integration with a timestep of $\mathscr{dt}=0.001$ sec. is used
to obtain all solutions to differential equations. We use a basic controllable
canonical form for the plant:

$\dot{x}_{t}=\begin{bmatrix}0&1\\\
-a_{1}&2a_{2}\end{bmatrix}\\!\begin{bmatrix}x_{t,1}\\\
x_{t,2}\end{bmatrix}+\begin{bmatrix}0\\\
\beta\end{bmatrix}\big{(}u_{t}+\Theta^{\top}\\!\phi(x_{t})\big{)}\ \ ,$ (18)

where $a_{1},a_{2},\beta>0$ are positive scalars, $u_{t}\in\mathbb{R}$ is the
$\ell=1$ dimensional input, and $\Theta$ is a fixed vector of nonzero scalars
which linearly parameterizes the known nonlinear vector function
$\phi(x_{t})=[\sigma_{R}(w_{1}^{\top}x_{t}+b_{1})\ \cdots\
\sigma_{R}(w_{4}^{\top}x_{t}+b_{4})]^{\top}$ with ReLU activation functions,
meeting the definition (2). This plant model is an $n=2$ example of a general
class of single-input systems with dynamics characterized by an $n$th order
nonlinear ordinary differential equation. See section 9.5 in [3] and section
4.1 in [14] for examples of physical systems in this form using various
nonlinear functions.

The reference system is a unity-gain damped harmonic oscillator with a bounded
reference input $r_{t}\in\mathbb{R}$:

$\dot{x}_{t}^{r}=\begin{bmatrix}0&1\\\
-\omega_{0}^{2}&-2\xi\omega_{0}\end{bmatrix}\\!\begin{bmatrix}x_{t,1}^{r}\\\
x_{t,2}^{r}\end{bmatrix}+\begin{bmatrix}0\\\ \omega_{0}^{2}\end{bmatrix}r_{t}\
\ ,$ (19)

where $\omega_{0},\xi>0$ are the natural frequency and damping ratio. This
gives the plant and reference eigenvalues as
$\lambda=a_{2}\pm\sqrt{a_{2}^{2}-a_{1}}$ and
$\lambda^{r}=(-\xi\pm\sqrt{\xi^{2}-1\,}\,)\,\omega_{0}$, thus $A$ is unstable
with oscillations if $a_{1}>a_{2}^{2}$ and $A_{r}$ is always Hurwitz, and
without oscillations if $\xi\geq 1$. Thus, we can always directly calculate
the required feedback and feedforward gains that satisfy the matching
conditions (13) as

$K_{x}^{\top}=\left[-\frac{\omega_{0}^{2}-a_{1}}{\beta}\ \
-\frac{2\xi\omega_{0}+2a_{2}}{\beta}\right]\ \ \text{and}\ \
K_{r}^{\top}=\left[\,\frac{\omega_{0}^{2}}{\beta}\,\right]\ .$

For the plant system, we use $a_{1}=2$, $a_{2}=0.5$, $\beta=0.75$. This
results in an unstable, oscillatory $A$ matrix with $\text{eig}(A)=0.5\pm
j\sqrt{1.75\,}\,$. The reference system uses $\omega_{0}=2$ and $\xi=1$,
ensuring that $A_{r}$ is Hurwitz and nonoscillatory with
$\text{eig}(A_{r})=-2\,$. The matching conditions are thus satisfied with
$K_{x}^{\top}=[-2.667\ \ -6.667]$ and $K_{r}^{\top}=[5.333]$. For the
parameter estimate dynamic updates we use:

$\Gamma=\begin{bmatrix}5&0&0&0\\\ 0&1&0&0\\\ 0&0&5&0\\\
0&0&0&2\end{bmatrix}Q_{x}=\begin{bmatrix}1&0\\\
0&10\end{bmatrix}P_{x}=\begin{bmatrix}5.625&0.125\\\
0.125&1.281\end{bmatrix},$

and for the linearly parameterized nonlinear vector function
$\Theta^{\top}\phi(x_{t})$ we use:

$\Theta=\begin{bmatrix}-1.2\\\ 2.7\\\ 0.8\\\ -3.2\end{bmatrix}\ \
W^{\top}=\begin{bmatrix}2&1\\\ 1&-2\\\ 1.5&-0.5\\\ 0.5&2\end{bmatrix}\ \
b=\begin{bmatrix}1\\\ 2\\\ 2.5\\\ 3\end{bmatrix}\ \ .$

In Figure 1 we plot points to highlight each activation region, partitioning
the state space $\mathbb{R}^{2}$.

Defining parameter estimates ${\widehat{\Theta}}_{t}=[{\hat{\theta}}_{t,1}\
\cdots\ {\hat{\theta}}_{t,4}]^{\top}$ and using the known $K_{x}$ and $K_{r}$,
we apply the feedback control law
$u_{t}=K^{x}x_{t}+K^{r}r_{t}-{\widehat{\Theta}}_{t}^{\top}\\!\phi(x_{t})$ to
the plant and update the parameter estimates according to the update law (16).
We use the following bounded reference inputs to drive the reference system as
two different scenarios:

$r_{t}^{(1)}=10\sin(0.5t)\qquad r_{t}^{(2)}=40+\sum_{k=1}^{2}10\sin(0.25k\,t)\
.$

The resulting plant and reference state trajectories $x_{t}$ and $x_{t}^{r}$
are plotted in state space along with the hyperplanes $W^{\top}x+b=0_{4}$, in
Figure 2 and Figure 3 respectively for the two scenarios. We see clearly that
the limit cycles in both cases stay within a compact set
$\mathcal{B}\in\mathbb{R}^{2}$, and so the ReLU activations within $\phi$ are
bounded on this $\mathcal{B}$, and the trajectories never maintain a linear
path.

In the first case, the state trajectories cross all $N=4$ hyperplanes at
nondegenerate borders persistently, as well as never being parallel to any
subset of active hyperplanes, thus meeting the criteria of Theorem 2. In the
second case, the limit cycle only crosses two of the hyperplanes persistently.

The parameter estimation error $\|{\widehat{\Theta}}_{t}-\Theta\|_{2}$ for
both cases is plotted over simulation time in Fig. 4. For the case with
$r_{t}^{(1)}$, we see that the error converges to zero as the parameter
estimators converge to the true values, while for the case with $r_{t}^{(2)}$
the error does not converge and is well above zero.

Figure 1: State space $x\in\mathbb{R}^{2}$ showing $N=4$ hyper planes
$W^{\top}x+b=0_{4}$. There are $11$ feasible activation regions
$\mathcal{A}_{j}$ (color dots) and $5$ infeasible regions (no dots). Figure 2:
Phase plot of $x_{t}$ (cyan), tracking into $x_{t}^{r}$ (magenta) driven by
$r_{t}^{(1)}$, and crossing all $N$ hyperplanes at nondegenerate borders along
the limit cycle. Figure 3: Phase plot of $x_{t}$ (cyan), tracking into
$x_{t}^{r}$ (magenta) driven by $r_{t}^{(2)}$, but does not cross all $N$
hyperplanes along the limit cycle. Figure 4: Norm of state tracking error
$\|e_{t}\|_{2}$ converges to zero in both cases, while the parameter
estimation error $\|{\widehat{\Theta}}_{t}-\Theta\|_{2}$ converges to zero for
the case with $r_{t}^{(1)}$ and but not for the case with $r_{t}^{(2)}$.

##  VI Conclusion

In this paper, we defined a geometric criteria that leads to sufficient
conditions for persistency of excitation with single hidden-layer neural
networks of step or ReLU activation functions. Future work will focus on using
the function approximation properties of ReLU activations to obtain similar
results when the plant nonlinearity is not known but can be approximated by
$\Theta^{\top}\phi(x_{t})$ with ReLU’s.

## References

  * [1] Shankar Sastry and Marc Bodson  “Adaptive control: stability, convergence and robustness”  Prentice Hall, Inc., 1989
  * [2] Jean-Jacques E Slotine and Weiping Li  “Applied nonlinear control”  Prentice hall Englewood Cliffs, NJ, 1991
  * [3] Eugene Lavretsky and Kevin A Wise  “Robust and Adaptive Control: with Aerospace Applications”  Springer, 2013
  * [4] AP Morgan and KS Narendra  “On the uniform asymptotic stability of certain linear nonautonomous differential equations”  In _SIAM Journal on Control and Optimization_ 15.1 SIAM, 1977, pp. 5–24
  * [5] Brian Anderson  “Exponential stability of linear equations arising in adaptive identification”  In _IEEE Transactions on Automatic Control_ 22.1 IEEE, 1977, pp. 83–88
  * [6] Veronica Adetola and Martin Guay  “Finite-time parameter estimation in adaptive control of nonlinear systems”  In _IEEE Transactions on Automatic Control_ 53.3 IEEE, 2008, pp. 807–811
  * [7] Sayan Basu Roy, Shubhendu Bhasin and Indra Narayan Kar  “Combined MRAC for unknown MIMO LTI systems with parameter convergence”  In _IEEE Transactions on Automatic Control_ 63.1 IEEE, 2017, pp. 283–290
  * [8] Girish Chowdhary, Tansel Yucelen, Maximillian Mühlegg and Eric N Johnson  “Concurrent learning adaptive control of linear systems with exponentially convergent bounds”  In _International Journal of Adaptive Control and Signal Processing_ 27.4 Wiley Online Library, 2013, pp. 280–301
  * [9] Anuradha M Annaswamy et al.  “Online Algorithms and Policies Using Adaptive and Machine Learning Approaches”  In _arXiv preprint arXiv:2105.06577_ , 2021
  * [10] Jan C Willems, Paolo Rapisarda, Ivan Markovsky and Bart LM De Moor  “A note on persistency of excitation”  In _Systems & Control Letters_ 54.4 Elsevier, 2005, pp. 325–329
  * [11] Kamil Nar and S Shankar Sastry  “Persistency of excitation for robustness of neural networks”  In _arXiv preprint arXiv:1911.01043_ , 2019
  * [12] Elena Panteley, Antonio Loria and Andrew Teel  “Relaxed persistency of excitation for uniform asymptotic stability”  In _IEEE Transactions on Automatic Control_ 46.12 IEEE, 2001, pp. 1874–1886
  * [13] Riccardo Marino and Patrizio Tomei  “Global adaptive output-feedback control of nonlinear systems. I. Linear parameterization”  In _IEEE Transactions on Automatic Control_ 38.1 IEEE, 1993, pp. 17–32
  * [14] Hassan K Khalil  “High-gain observers in nonlinear feedback control”  SIAM, 2017
  * [15] Stephen Boyd and Sosale Shankara Sastry  “Necessary and sufficient conditions for parameter convergence in adaptive control”  In _Automatica_ 22.6 Elsevier, 1986, pp. 629–639
  * [16] Iven MY Mareels and Michel Gevers  “Persistency of excitation criteria for linear, multivariable, time-varying systems”  In _Mathematics of Control, Signals and Systems_ 1.3 Springer, 1988, pp. 203–226
  * [17] Gerhard Kreisselmeier and Gudrun Rietze-Augst  “Richness and excitation on an interval-with application to continuous-time adaptive control”  In _IEEE transactions on automatic control_ 35.2 IEEE, 1990, pp. 165–171
  * [18] Jung-Shan Lin and Ioannis Kanellakopoulos  “Nonlinearities enhance parameter convergence in strict feedback systems”  In _IEEE Transactions on Automatic Control_ 44.1 IEEE, 1999, pp. 89–94
  * [19] Philipp Karg, Florian Köpf, Christian Alexander Braun and Sören Hohmann  “Excitation for Adaptive Optimal Control of Nonlinear Systems in Differential Games”  In _IEEE Transactions on Automatic Control_ IEEE, 2022
  * [20] Alberto Padoan, Giordano Scarciotti and Alessandro Astolfi  “A geometric characterization of the persistence of excitation condition for the solutions of autonomous systems”  In _IEEE Transactions on Automatic Control_ 62.11 IEEE, 2017, pp. 5666–5677
  * [21] Juan G Rueda-Escobedo and Jaime A Moreno  “Strong Lyapunov functions for two classical problems in adaptive control”  In _Automatica_ 124 Elsevier, 2021, pp. 109250
  * [22] AJ Kurdila, Francis J Narcowich and Joseph D Ward  “Persistency of excitation in identification using radial basis function approximants”  In _SIAM journal on control and optimization_ 33.2 SIAM, 1995, pp. 625–642

##  Appendix A Proofs of Theorems 1 and 2

Here we will prove the main results provided in Section IV.

###  A-A Proof Methodology and Setup

The proofs for both theorems rely on the same contradiction method and share a
common setup. First, we will assume that the state trajectory $x_{t}$ is
continuous and stays within some compact set
$\mathcal{B}\subset\mathbb{R}^{n}$ for all $t\geq 0$, and that the component
activations
$\sigma_{1}(w_{1}^{\top}x+b_{1}),\dots,\sigma_{N}(w_{N}^{\top}x+b_{N})$ of
$\phi$ are then bounded on that $\mathcal{B}$.

Then we will assume that for any shift $\tau\geq 0$ of the time window
$[\tau,\tau+T]$ of any length $T>0$, there exists a nonzero vector
$v\in\mathbb{R}^{N}\setminus\\{0_{N}\\}$ such that

$v^{\top}\left(\int_{\tau}^{\tau+T}\phi(x_{t})\,\phi(x_{t})^{\top}\text{dt}\right)v\
=\ 0\ \ .$ (20)

Finally, we will show that if the state trajectory $x_{t}$ meets certain
requirements within any shift $\tau\geq 0$ of the window $[\tau,\tau+T^{*}]$,
for some window length $T^{*}>0$, then in fact (20) can only hold if
$v=0_{N}$. This is a contradiction and thus proves that

$\int_{\tau}^{\tau+T^{*}}\phi(x_{t})\,\phi(x_{t})^{\top}\text{dt}$ (21)

must be strictly positive definite, with bounded eigenvalues, uniformly over
all windows $[\tau,\tau+T^{*}]$. Then by the Min-max principle, the
persistency of excitation conditions (6) and (7) are satisfied with setting
$\alpha_{1}$ and $\alpha_{2}$ respectively as the smallest and largest
eigenvalues of (21) over all shifts $\tau\geq 0$ of the window
$[\tau,\tau+T^{*}]$.

And so, let there be such a continuous trajectory $x_{t}$ staying within a
compact set $\mathcal{B}\subset\mathbb{R}^{n}$ for all $t\geq 0$. Within any
particular time window $[\tau,\tau+T]$, the trajectory visits $\mathscr{L}$
activation regions
$\mathcal{A}_{\mathscr{A}_{1}},\dots,\mathcal{A}_{\mathscr{A}_{\mathscr{L}}}$
and the corresponding index sequences
$\mathscr{A}_{1},\dots,\mathscr{A}_{\mathscr{L}}$ and
$\mathscr{I}_{1},\dots,\mathscr{I}_{\mathscr{L}-1}$ are defined as in Section
III.

For any time window $[\tau,\tau+T]$ and over the corresponding sequence
$s\in[\mathscr{L}]$ of visited activation regions, recall the definition of
the subsets $\mathcal{T}_{s}$ from (8):

$\mathcal{T}_{s}=\\{t\in[\tau,\tau+T]\ |\
x_{t}\in\mathcal{A}_{\mathscr{A}_{s}}\\}\ .$

Clearly the union of these subsets gives back the entire window as

$\bigcup_{s=1}^{\mathscr{L}}\mathcal{T}_{s}\ =\ [\tau,\tau+T]\ ,$

meaning (20) is equivalent to

$\int_{\tau}^{\tau+T}(v^{\top}\phi(x_{t}))^{2}\text{dt}=\sum_{s=1}^{\mathscr{L}}\int_{\mathcal{T}_{s}}(v^{\top}\phi(x_{t}))^{2}\text{dt}=0$
(22)

holding for all time windows $[\tau,\tau+T]$ and corresponding
$\mathcal{T}_{s}$ subsets.

Since each integral in the sum is over $t\in\mathcal{T}_{s}$, and by
definition we know that $x_{t}$ is only within $\mathcal{A}_{\mathscr{A}_{s}}$
during that subset of the time window, then by definition only the activation
indices $i\in\mathscr{S}_{\mathscr{A}_{s}}$ of $\phi(x_{t})$ will be nonzero,
which gives

$v^{\top}\phi(x_{t})=\sum_{i=1}^{N}v_{i}\,\phi(x_{t})_{i}=\sum_{\mathscr{S}_{\mathscr{A}_{s}}}v_{i}\,\phi(x_{t})_{i}:=v_{\mathscr{A}_{s}}^{\top}\,\phi(x_{t})_{\mathscr{A}_{s}}\
.$

Here we denote $v_{\mathscr{A}_{s}}$ and $\phi(x_{t})_{\mathscr{A}_{s}}$
respectively as the indices of $v$ and $\phi(x_{t})$ according to the active
set $i\in\mathscr{S}_{\mathscr{A}_{s}}$, for each $s\in[\mathscr{L}]$. This
means (22) is equivalent to

$\sum_{s=1}^{\mathscr{L}}\int_{\mathcal{T}_{s}}(v_{\mathscr{A}_{s}}^{\top}\phi(x_{t})_{\mathscr{A}_{s}})^{2}\text{dt}\
=\ 0$ (23)

holding for all time windows $[\tau,\tau+T]$ and corresponding
$\mathcal{T}_{s}$ subsets. And since each of these integrals is always
nonnegative, (23) is equivalent to

$v_{\mathscr{A}_{s}}^{\top}\phi(x_{t})_{\mathscr{A}_{s}}=0\qquad\forall
t\in\mathcal{T}_{s},\forall s\in[\mathscr{L}]$ (24)

holding for all time windows $[\tau,\tau+T]$ and corresponding
$\mathcal{T}_{s}$ subsets.

###  A-B Proof of Theorem 1

In this case, (24) is equivalent to

$v_{\mathscr{A}_{s}}^{\top}\phi(x_{t})_{\mathscr{A}_{s}}=v_{\mathscr{A}_{s}}^{\top}c_{\mathscr{A}_{s}}=0\qquad\forall
t\in\mathcal{T}_{s},\forall s\in[\mathscr{L}]$ (25)

holding for all time windows $[\tau,\tau+T]$ and corresponding
$\mathcal{T}_{s}$ subsets. Here we denote $c_{\mathscr{A}_{s}}$ as the indices
of $c$ according to the active set $i\in\mathscr{S}_{\mathscr{A}_{s}}$, for
all $s\in[\mathscr{L}]$. This gives a fixed, nonzero
$|\mathscr{S}_{\mathscr{A}_{s}}|$ length vector.

Note we have assumed that $x_{t}$ over $t\geq 0$ is such that there exists
$T^{*}>0$ whereby all $N$ hyperplanes are crossed and only at nondegenerate
borders, within any time window $[\tau,\tau+T^{*}]$. Thus, we can combine (9)
and (25) to get that

$\displaystyle
0=v_{\mathscr{A}_{s+1}}^{\top}c_{\mathscr{A}_{s+1}}\qquad\forall
t\in\mathcal{T}_{s+1}$ (26)
$\displaystyle=\begin{cases}v_{\mathscr{A}_{s}}^{\top}c_{\mathscr{A}_{s}}+v_{\mathscr{I}_{s}}c_{\mathscr{I}_{s}}\qquad\text{if}\
\ \mathcal{A}_{\mathscr{A}_{s+1}}\subseteq X_{\mathscr{I}_{s}}^{+}\\\
v_{\mathscr{A}_{s}}^{\top}c_{\mathscr{A}_{s}}-v_{\mathscr{I}_{s}}c_{\mathscr{I}_{s}}\qquad\text{if}\
\ \mathcal{A}_{\mathscr{A}_{s+1}}\subseteq
X_{\mathscr{I}_{s}}^{\circ}\end{cases}$

must hold over all $s\in[\mathscr{L}-1]$ with
$v_{\mathscr{A}_{1}}^{\top}c_{\mathscr{A}_{1}}=0$ for all
$t\in\mathcal{T}_{1}$, for all time windows $[\tau,\tau+T^{*}]$ and
corresponding $\mathcal{T}_{s}$ subsets. However, we will now show by
induction that this can only hold if $v=0_{N}$.

Starting at $s=1$: it must hold that
$v_{\mathscr{A}_{1}}^{\top}c_{\mathscr{A}_{1}}=0$ for all
$t\in\mathcal{T}_{1}$, and since $v$ and $c$ are fixed it must continue to
hold for all $t\in\mathcal{T}_{2}$, during which it must hold that
$v_{\mathscr{A}_{2}}^{\top}c_{\mathscr{A}_{2}}=0$. And so, these can both hold
only if the additional term is $v_{\mathscr{I}_{1}}c_{\mathscr{I}_{1}}=0$, and
since $c_{\mathscr{I}_{1}}\neq 0$ it must be that $v_{\mathscr{I}_{1}}=0$. We
can then inductively make the same argument over the remaining
$s=2,3,\dots,\mathscr{L}-1$ to obtain
$v_{\mathscr{I}_{1}}=\dots=v_{\mathscr{I}_{\mathscr{L}-1}}=0$. And since we
have assumed all $N$ hyperplanes were crossed at nondegenerate borders over
the $\mathscr{L}-1$ hyperplane crossings, we therefore must have that
$v_{i}=0$ for all $i\in[N]$. This establishes the contradiction from the
original assumption of a nonzero $v$.

Since we assume that all $N$ hyperplanes are crossed and only at nondegenerate
borders during any time window $[\tau,\tau+T^{*}]$, this contradiction must
hold for all such windows. Therefore, (21) must be strictly positive definite
for all $\tau\geq 0$ with this $T^{*}$ window length. And since the (scaled)
step is bounded on any compact set $\mathcal{B}\subset\mathbb{R}^{n}$, we have
proven persistency of excitation holds for any window length of at least
$T^{*}$. a $\blacksquare$

###  A-C Proof of Theorem 2

In this case, (24) is equivalent to

$v_{\mathscr{A}_{s}}^{\top}\phi(x_{t})_{\mathscr{A}_{s}}=v_{\mathscr{A}_{s}}^{\top}(W_{\mathscr{A}_{s}}^{\top}x_{t}+b_{\mathscr{A}_{s}})=0\qquad\forall
t\in\mathcal{T}_{s},\forall s\in[\mathscr{L}]$ (27)

holding for all time windows $[\tau,\tau+T]$ and corresponding
$\mathcal{T}_{s}$ subsets. Here we denote $W^{\top}$ as the $N\times n$ matrix
with $w_{i}^{\top}$ as the $i$th row and $b$ as the length $N$ vector with
$b_{i}$ as the $i$th entry, for all $i\in[N]$, and thus the $\mathscr{A}_{s}$
subscripts are the indices according to the active set
$i\in\mathscr{S}_{\mathscr{A}_{s}}$, for each $s\in[\mathscr{L}]$.

Let $t_{1},\hat{t}_{1},\ldots,t_{M},\hat{t}_{M}\in\mathcal{T}_{s}$ be the
times from condition (iii). Then for $i=1,\ldots,M$ it must hold by (27) that

$v_{\mathscr{A}_{s}}^{\top}(W_{\mathscr{A}_{s}}^{\top}x_{t_{i}}+b_{\mathscr{A}_{s}})-v_{\mathscr{A}_{s}}^{\top}(W_{\mathscr{A}_{s}}^{\top}x_{\hat{t}_{i}}+b_{\mathscr{A}_{s}})=0=v_{\mathscr{A}_{s}}^{\top}W_{\mathscr{A}_{s}}^{\top}(x_{t_{i}}-x_{\hat{t}_{i}})\
.$ (28)

In particular, $v_{\mathscr{A}_{s}}$ must be in the nullspace of

$\begin{bmatrix}(x_{t_{1}}-x_{\hat{t}_{1}})^{\top}\\\ \vdots\\\
(x_{t_{M}}-x_{\hat{t}_{M}})^{\top}\end{bmatrix}W_{\mathscr{A}_{s}}=:\hat{W}_{\mathscr{A}_{s}}\
.$

Condition (iii) implies that $\hat{W}_{\mathscr{A}_{s}}$ and
$W_{\mathscr{A}_{s}}$ have the same rank, and so the rank-nullity theorem
implies that their nullspaces must have the same dimension. Now, since the
nullspace of $\hat{W}_{\mathscr{A}_{s}}$ is a subspace of the nullspace of
$W_{\mathscr{A}_{s}}$, the nullspaces must actually be the same. As a result,
(28) implies that $v_{\mathscr{A}_{s}}^{\top}W_{\mathscr{A}_{s}}^{\top}=0$ and
so $v_{\mathscr{A}_{s}}^{\top}b_{\mathscr{A}_{s}}=0$ as well.

We have just shown that with condition (iii) satisfied, (27) is equivalent to

$v_{\mathscr{A}_{s}}^{\top}[W_{\mathscr{A}_{s}}^{\top}\
b_{\mathscr{A}_{s}}]=0_{n+1}^{\top}\qquad\forall t\in\mathcal{T}_{s},\forall
s\in[\mathscr{L}]$ (29)

holding for all time windows $[\tau,\tau+T]$ and corresponding
$\mathcal{T}_{s}$ subsets.

Note we have assumed that $x_{t}$ over $t\geq 0$ is such that there exists
$T^{*}>0$ whereby all $N$ hyperplanes are crossed and only at nondegenerate
borders, within any time window $[\tau,\tau+T^{*}]$. Thus, we can combine (9)
and (29) to get that

$\displaystyle
0_{n+1}^{\top}=v_{\mathscr{A}_{s+1}}^{\top}[W_{\mathscr{A}_{s+1}}^{\top}\
b_{\mathscr{A}_{s+1}}]\qquad\forall t\in\mathcal{T}_{s+1}$ (30)
$\displaystyle=\begin{cases}v_{\mathscr{A}_{s}}^{\top}[W_{\mathscr{A}_{s}}^{\top}\
b_{\mathscr{A}_{s}}]+v_{\mathscr{I}_{s}}[w_{\mathscr{I}_{s}}^{\top}\
b_{\mathscr{I}_{s}}]\qquad\text{if}\ \
\mathcal{A}_{\mathscr{A}_{s+1}}\subseteq X_{\mathscr{I}_{s}}^{+}\\\
v_{\mathscr{A}_{s}}^{\top}[W_{\mathscr{A}_{s}}^{\top}\
b_{\mathscr{A}_{s}}]-v_{\mathscr{I}_{s}}[w_{\mathscr{I}_{s}}^{\top}\
b_{\mathscr{I}_{s}}]\qquad\text{if}\ \
\mathcal{A}_{\mathscr{A}_{s+1}}\subseteq
X_{\mathscr{I}_{s}}^{\circ}\end{cases}$

must hold over all $s\in[\mathscr{L}-1]$ with
$v_{\mathscr{A}_{1}}^{\top}[W_{\mathscr{A}_{1}}^{\top}\
b_{\mathscr{A}_{1}}]=0_{n+1}^{\top}$ for all $t\in\mathcal{T}_{1}$, for all
time windows $[\tau,\tau+T^{*}]$ and corresponding $\mathcal{T}_{s}$ subsets.
However, we will now show by induction that this can only hold if $v=0_{N}$.

Starting at $s=1$: it must hold that
$v_{\mathscr{A}_{1}}^{\top}[W_{\mathscr{A}_{1}}^{\top}\
b_{\mathscr{A}_{1}}]=0_{n+1}^{\top}$ for all $t\in\mathcal{T}_{1}$, and since
$v$, $W$, and $b$ are fixed it must continue to hold for all
$t\in\mathcal{T}_{2}$, during which it must hold that
$v_{2}^{\top}[W_{\mathscr{A}_{2}}^{\top}\
b_{\mathscr{A}_{2}}]=0_{n+1}^{\top}$. And so, these can both only hold if the
additional term is $v_{\mathscr{I}_{1}}[w_{\mathscr{I}_{1}}^{\top}\
b_{\mathscr{I}_{1}}]=0_{n+1}^{\top}$, and since $[w_{\mathscr{I}_{1}}^{\top}\
b_{\mathscr{I}_{1}}]$ is nonzero it must be that $v_{\mathscr{I}_{1}}=0$. We
can then inductively make the same argument over the remaining
$s=2,3,\dots,\mathscr{L}-1$ to obtain
$v_{\mathscr{I}_{1}}=\dots=v_{\mathscr{I}_{\mathscr{L}-1}}=0$. And since we
have assumed all $N$ hyperplanes were crossed at nondegenerate borders over
the $\mathscr{L}-1$ hyperplane crossings, we therefore must have that
$v_{i}=0$ for all $i\in[N]$. This establishes the contradiction from the
original assumption of a nonzero $v$.

Since we assume that all $N$ hyperplanes are crossed and only at nondegenerate
borders during any time window $[\tau,\tau+T^{*}]$, this contradiction must
hold for all such windows. Therefore, (21) must be strictly positive definite
for all $\tau\geq 0$ with this $T^{*}$ window length. And since the ReLU is
bounded on any compact set $\mathcal{B}\subset\mathbb{R}^{n}$, we have proven
persistency of excitation holds for any window length of at least $T^{*}$. a
$\blacksquare$ a

###  A-D Extending to Other Nonlinear, Positive Semidefinite Activations

The reasoning and methods described in this appendix section should also be
able to be applied similarly to any nonlinear, positive semidefinite
activation function for which it is possible to show the following:

  1. 1.

describe all specific trajectories that $x_{t}$ can take such that
$\phi(x_{t})_{\mathscr{A}_{s}}$ is some constant vector for the entire time
within the activation region $\mathcal{A}_{\mathscr{A}_{s}}$, and

  2. 2.

formulate conditions like (26) from (25) and (30) from (27), such that the
condition is independent of the specific value of $x_{t}$, and instead relies
only on fixed values within the definition of the activation (like $c$ and
$W$, $b$).

If these two requirements are possible to meet, then the induction proof
method will hold as it does in this appendix section for (scaled) steps and
ReLU activation functions.

##  Appendix B Global Uniform Asymptotic Stability of (17)

Here we will show how the linear time-varying (LTV) system

$\dot{y}_{t}=\Gamma\,\phi(x_{t})\phi(x_{t})^{\top}y_{t}$ (31)

is proven globally uniformly asymptotically stable if the vector function
$\phi:\mathbb{R}^{n}\to\mathbb{R}^{N}$ satisfies the persistency of excitation
condition (6). State trajectories $y:[0,\infty)\to\mathbb{R}^{N}$ and
$x:[0,\infty)\to\mathbb{R}^{n}$ are assumed to be continuous (in time), and
$x_{t}$ is assumed to stay within some compact set
$\mathcal{B}\in\mathbb{R}^{n}$ for all $t\geq 0$. The vector function is
defined $\phi(x):=[g_{1}(x)\ \cdots\ g_{N}(x)]^{\top}$, where the piecewise
continuous, nonlinear scalar functions
$g_{1},\dots,g_{N}:\mathbb{R}^{n}\to\mathbb{R}$ are bounded on $\mathcal{B}$,
and $\Gamma$ is some symmetric, positive-definite $N\times N$ matrix with
$\|\Gamma\|_{F}=\mathscr{G}$. Then, $\phi(x_{t})$ is a piecewise continuous
(and regulated) vector function of time, mapping
$[0,\infty)\to\mathbb{R}^{N}$.

Firstly, Lemma 1 of [5] says that if $F:[0,\infty)\to\mathbb{R}^{N\times N}$,
$H:[0,\infty)\to\mathbb{R}^{N}$, and $K[0,\infty)\to\mathbb{R}^{N}$ are
regulated vector or matrix functions of time, and there exists positive
scalars $\alpha_{3},T>0$ such that

$\int_{\tau}^{\tau+T}\|K_{t}\|_{2}^{2}\,\text{dt}\ \leq\ \alpha_{3}$

holds for all $\tau\geq 0$, then $[F_{t},H_{t}]$ is uniformly completely
observable (UCO) if and only if $[F_{t}+K_{t}\,H_{t}^{\top},H_{t}]$ is UCO.

Next, Lemma 2 of [5] then says that the global uniform asymptotic stability of

$\dot{y}_{t}=F_{t}\,y_{t}$

is equivalent to: i) the existence of a symmetric, differentiable matrix
function of time $P:[0,\infty)\to\mathbb{R}^{n\times n}$ satisfying, for some
positive scalars $\beta_{1},\beta_{2}>0$, that

$\beta_{1}I_{N}\preceq
P_{t}\preceq\beta_{2}I_{N}\qquad\text{and}\qquad-\dot{P}_{t}=P_{t}\,F_{t}+F_{t}^{\top}P_{t}+H_{t}H_{t}^{\top}$

both hold for all $t\geq 0$, and ii) that $[F_{t},H_{t}]$ is UCO.

Finally, a slight modification of Theorem 1 of [5], in order to allow for
$\Gamma$ matrices other than the identity $I_{N}$, says that the global
uniform asymptotic stability of (31) holds if $\phi(x_{t})$ satisfies the
persistency of excitation condition (6).
Proof. Suppose that $\phi(x_{t})$ satisfies the persistency of excitation
condition (6) for some positive scalars $\alpha_{1},\alpha_{2},T>0$. This
means, by definition, that $[0_{N\times N},\phi(x_{t})]$ is UCO. Now set
$K_{t}=-\Gamma\,\phi(x_{t})$ and note that (6) gives

$\int_{\tau}^{\tau+T}\|K_{t}\|_{2}^{2}\,\text{dt}\ \leq\
\int_{\tau}^{\tau+T}\|\Gamma\|_{F}^{2}\|\phi(x_{t})\|_{F}^{2}\,\text{dt}=\mathscr{G}^{2}\int_{\tau}^{\tau+T}\\!\text{tr}\left[\phi(x_{t})\phi(x_{t})^{\top}\right]\text{dt}\
\leq\ \mathscr{G}^{2}\alpha_{2}N\ ,$

thus satisfying Lemma 1 and so
$[-\Gamma\,\phi(x_{t})\phi(x_{t})^{\top},\phi(x_{t})]$ is UCO.

Therefore, we have that Lemma 2 is satisfied with
$P_{t}=\frac{1}{2}\Gamma^{-1}$,
$F_{t}=-\Gamma\,\phi(x_{t})\phi(x_{t})^{\top}$, and $H_{t}=\phi(x_{t})$. This
proves (31) is globally uniformly asymptotically stable. a $\blacksquare$

###### Remark 2

Lemma 1 and Theorem 1 of [5] use exponential stability in their statements,
rather than global asymptotic uniform stability. However, for linear systems
these types of stability are equivalent. See Section 1.5.2 of [1] for details.

a

###### Remark 3

Lemma 1 and 2 and Theorem 1 of [5] also hold with $\phi(x_{t})$, $K_{t}$, and
$H_{t}$ being regulated $N\times\ell$ matrix functions of time for any
$\ell\geq 1$, using the induced Euclidean matrix norm when $\ell>1$. We
specified their results for $\ell=1$ and the Euclidean vector norm, in order
to match the application in Section V.