11institutetext: Embedded Systems, University of Tübingen, Tübingen, Germany

# It’s all about PR

Smart Benchmarking AI Accelerators using
Performance Representatives

Alexander Louis-Ferdinand Jung  0000-0001-5702-5768 Jannik Steinmetz
0009-0003-7193-5620 Jonathan Gietz  0009-0000-7843-1781 Konstantin Lübeck
0000-0002-2701-5881 Oliver Bringmann  0000-0002-1615-507X

###### Abstract

Statistical models are widely used to estimate the performance of commercial
off-the-shelf (COTS) AI hardware accelerators. However, training of
statistical performance models often requires vast amounts of data, leading to
a significant time investment and can be difficult in case of limited hardware
availability.

To alleviate this problem, we propose a novel performance modeling methodology
that significantly reduces the number of training samples while maintaining
good accuracy. Our approach leverages knowledge of the target hardware
architecture and initial parameter sweeps to identify a set of Performance
Representatives (PR) for deep neural network (DNN) layers. These PRs are then
used for benchmarking, building a statistical performance model, and making
estimations. This targeted approach drastically reduces the number of training
samples needed, opposed to random sampling, to achieve a better estimation
accuracy.

We achieve a Mean Absolute Percentage Error (MAPE) of as low as
$0.02\text{\,}\mathrm{\char 37\relax}$ for single-layer estimations and
$0.68\text{\,}\mathrm{\char 37\relax}$ for whole DNN estimations with less
than $10\,000$ training samples. The results demonstrate the superiority of
our method for single-layer estimations compared to models trained with
randomly sampled datasets of the same size.

###### Keywords:

Performance Estimation Benchmarking AI Hardware Accelerators Statistical
Modelling Deep Neural Networks

##  1 Introduction

Deep Neural Networks (DNNs) are being used in various tasks ranging from time-
series and image classification to natural language processing. As these
models become part of more and more AI-assisted edge devices, there is an
abundance of commercial off-the-shelf AI accelerator hardware. Determining
which accelerator to use for a specific AI application is difficult, as real
hardware might not yet be available and simulations tend to be very time-
consuming. Therefore, it is important to be able to estimate the performance
of a DNN mapped onto an accelerator platform.

Another area where performance estimation models are needed is Neural
Architecture Search (NAS). Especially for hardware-aware NAS targeting
embedded platforms, a trade-off between a high DNN accuracy and the execution
time has to be considered. However, measuring all possible DNN candidates can
be extremely time-consuming. Therefore, an efficient performance estimator is
needed to accelerate the optimization loop. [4]

Statistical estimation models have been shown to have high accuracy for
execution time estimation if a sufficient amount of training samples is
obtainable [11, 2, 13]. Analytical models, on the other hand, require detailed
knowledge about the architecture to yield accurate estimations [10, 8].

Vendors of commercial off-the-shelf (COTS) accelerators often only provide a
slow functional and timing simulator. Gathering a sufficiently large amount of
training samples for statistical estimation models with these simulators is
infeasible. Another problem with COTS platforms is, that often only general
specifications such as the available functional units, and the memory
bandwidth are publicly known. An accurate analytical performance model cannot
be created from such limited information. But these known hardware
characteristics can indicate interesting behavior with specific DNN layer
parameters, e.g., which tile-sizes yield a good utilization.

This paper proposes a performance modeling methodology paired with a
benchmarking strategy to select significant data points to reduce measurement
runs and consequently measurement time. Depending on the amount of knowledge
that is available for a platform, the known hardware characteristics can be
used to find DNN layer parameters that are representative of the execution
time behavior. If there is not much information about a platform, these
Performance Representatives need to be determined empirically in an initial
benchmark phase. An overview of the proposed approach is depicted in Fig. 1.

Figure 1: The proposed performance modeling methodology and benchmarking
strategy.

This paper makes the following contributions: (1) We describe a methodology to
determine the Performance Representatives of a specific hardware platform. (2)
We then use these Performance Representatives to carry out benchmarks on the
hardware platform under consideration and then use these representative data
points to train statistical models which can then be used to estimate the
execution time of single-layers and whole DNNs. (3) We also show how the
amount of training samples impacts the accuracy when using our approach
compared to random sampling the same amount of data points.

##  2 Related Work

In ANNETTE [11] they use micro-kernel and multi-layer benchmarks of DNN layers
to create machine learning estimation models. These Random Forest models are
then combined with the well-known Roofline model [12] to better capture the
regular execution time behavior of these platforms. They evaluate their mixed
models on two hardware platforms (an FPGA and an ASIC accelerator) and achieve
a mean absolute percentage error (MAPE) of as low as
$12.71\text{\,}\mathrm{\char 37\relax}$ for single-layer estimations and
$3.47\text{\,}\mathrm{\char 37\relax}$ for whole networks. Per platform and
per layer $35\,000$ measurements were conducted.

Bouzidi et al. [2] compare five different machine learning methods for
performance estimation on the NVIDIA Jetson AGX Xavier and TX2 GPUs. They
measured around $200\,000$ whole DNN inferences, by using the execution times
of the executed GPU kernels reported by NVIDIA’s profiling tools. The MAPE of
the best estimation model ranges between $7.67\text{\,}\mathrm{\char
37\relax}$ and $14.73\text{\,}\mathrm{\char 37\relax}$.

The authors of Blackthorn [7] specifically exploit the very regular execution
time behavior of the NVIDIA Jetson TX2 and the Jetson Nano to create their
performance estimation models. They define a set of step-wise functions and
reduce this set of functions gradually by measuring more and more data points.
Each function that cannot explain the new data point is removed from the
function set, resulting in one final step-wise function that is then used for
estimating the performance. This requires around $15\,000$ measurements for
the Convolution layer, resulting in a Root-Mean-Square-Percentage-Error
(RMSPE) of $7.57\text{\,}\mathrm{\char 37\relax}$ for Convolution layers and
an error between $0.45\text{\,}\mathrm{\char 37\relax}$ and
$6.68\text{\,}\mathrm{\char 37\relax}$ for complete DNN execution time
estimation.

In nn-Meter [13] DNN execution kernels are benchmarked, which contain multiple
fused layers and are used for estimating the execution time of various DNNs by
summing the estimated times for the contained kernels. They obtain up to
$39\,968$ benchmark points for e.g. the Convolution execution kernel. These
samples are then used for training Random Forest models, which yield a RMSPE
between $1.35\text{\,}\mathrm{\char 37\relax}$ and
$22.25\text{\,}\mathrm{\char 37\relax}$ depending on the DNN and hardware
platform.

The presented estimators require a vast amount of training samples to achieve
the reported estimation accuracies. Additionally, the usage of step-wise
functions by Blackthorn has some drawbacks. Firstly, a function pattern that
describes the execution time behavior needs to be defined. Secondly, the step-
wise function describing the execution time behavior must be covered by the
parameter space of the function set. This is a prerequisite so that the
approach finds a matching function. We address these shortcomings by, on the
one hand, reducing the number of training samples, and on the other hand,
determining the step width algorithmically instead of choosing from a set of
pre-defined functions for an accurate execution time estimation.

##  3 Smart Benchmarking and Modeling

The execution time behavior of an AI accelerator platform often exhibits a
very regular pattern. This is due to how AI accelerators compute the layers of
a DNN. For example, the UltraTrail accelerator [1] always processes $8\times
8$ output and input channels in one activation of the multiply accumulate
(MAC) array. Therefore, the execution time behavior takes the shape of a step-
wise function while increasing only the input channel parameter or the output
channel parameter of the 1D-Convolution. We call these types of benchmarks
parameter sweeps. But also other hardware platforms, like the NVIDIA Jetson
AGX Xavier GPU exhibit a similar behavior, as can be seen in Fig. 2.

Figure 2: Step-wise execution time behavior of the 2D-Convolution observed for
different parameter sweeps, run on the NVIDIA Jetson AGX Xavier GPU.
Performance Representatives are highlighted in red (last point on step). The
parameters of the 2D-Convolution are described in Eq. 3 and its explanation.

This also means, that many of the data points in fine-grained measurements do
not contain new information about the execution time behavior of the platform.
The aim of this work is to reduce the number of measured data points for
building statistical performance models, while maintaining good accuracy when
estimating the execution time of a single layer as well as of whole DNNs.

###  3.1 Performance Representatives (PR)

In our work, we combine the findings of [7] and [11]. Exploiting the regular
execution time behavior to guide the selection of benchmark points,
consequently reducing the number of points that need to be measured. These
measurements are then used to train statistical models for execution time
estimation. The idea is to only measure one representative data point for a
step. We call this a Performance Representative (PR). We algorithmically
determine the last point of a step as our PR, as these are generally the
points with optimal utilization of the AI accelerator architecture.

###  3.2 Hardware and Mapping Parameters

How the PRs of the execution time behavior can be derived heavily depends on
how much knowledge about the accelerator architecture is available. We
classify the amount of available architecture knowledge for the accelerators
used in this work ranging from white-box to black-box. This spectrum is
depicted in Fig. 3.

Figure 3: Accelerator classification on the white-box to black-box spectrum.

White-box accelerators are architectures where all the needed information
about how the accelerator works and DNN layers are mapped is publicly
available so that the execution time behavior of the architecture can be
derived, for instance from the number of processing elements (PEs) in each
dimension and how the layer is unrolled onto the architecture.

At the other end of the spectrum are the black-box accelerators. For those
architectures, almost no information is available and therefore, an execution
time behavior cannot be determined without measurements.

When some information about the AI accelerator architecture is available, but
not enough to accurately determine the execution time behavior, we speak of
gray-box accelerators. For this category, it is necessary to use the available
information to guide the selection of benchmarks that will help to either
confirm the expected PRs or to determine the execution time behavior from
which the PRs will be identified. Depending on the category, the accelerator
belongs to, the process of finding the PRs differs:

As stated above, for white-box accelerators, all the necessary information is
available, and the PRs can be determined from it. To illustrate this, we use
the UltraTrail accelerator by Bernardo et al. [1] as an example. In the
standard configuration, it only supports 1D-Convolutions:

$Conv1D(C,C_{w},K,F,s,pad)$ (1)

with input channels $C$, input feature width $C_{w}$, output channels $K$,
kernel size $F$, stride $s$ and padding $pad$. In [1], the UltraTrail
accelerator is described in detail so that it is possible to determine the
hardware and mapping characteristics:

⬇

operation Conv1D

operation_params [C, C_w, K, F, s, pad]

dims [8, 8]

mapping [C, K]

This hardware and mapping description specifies that the parameters $C$ and
$K$ of the Conv1D layer are mapped and unrolled on the two spatial dimensions
of the UltraTrail accelerator, each with a size of eight. From this
information, the PRs can be determined as:

$Conv1D_{R}(x_{C}\cdot 8,C_{w},x_{K}\cdot 8,F,s,pad)$ (2)

with $x_{C},x_{K}\in\\{1,2,...,7\\}$. The upper bounds of $x_{C}$ and $x_{K}$
stem from the upper bounds of the parameters $C$ and $K$ as described in [1].

As long as the AI accelerator architecture and mapping can be described like
this, the PRs can be determined by setting the values of the unrolled
parameters to a multiple of the respective dimensions.

In the category of black-box accelerators, we have AI accelerator
architectures, for which there is almost no information available. Therefore,
the PRs can only be determined from measurements. To illustrate this, we look
at the NVIDIA Jetson AGX Xavier GPU. E.g. we do not know how the
TensorRT111https://developer.nvidia.com/tensorrt, last accessed 03/26/2024
backend maps the Convolution layer

$Conv2D(C,C_{h},C_{w},K,F_{h},F_{w},s,pad)$ (3)

with input channels $C$, input feature height $C_{h}$ and width $C_{w}$,
output channels $K$, kernel height $F_{h}$ and width $F_{w}$, stride $s$ and
padding $pad$ onto the Jetson AGX Xavier GPU. Therefore, we need to perform
initial sweep benchmarks over the parameters
$P=\\{C,C_{h},C_{w},K,F_{h}=F_{w}\\}$, that influence the execution time
behavior the most. We only consider quadratic kernel sizes, as those are most
commonly used in state-of-the-art DNNs.

We use the procedure described in Algorithm 1 to determine the step width
$w_{p}\in W$ of each parameter $p\in P$ from the performed sweeps. In general,
the PRs of the Conv2D layer can be determined from the step width $w_{p}$ as
follows:

$\begin{split}Conv2D_{R}(&x_{C}\cdot w_{C},x_{C_{h}}\cdot
w_{C_{h}},x_{C_{w}}\cdot w_{C_{w}},\\\ &x_{K}\cdot w_{K},x_{F_{h}}\cdot
w_{F_{h}},x_{F_{w}}\cdot w_{F_{w}},s,pad)\end{split}$ (4)

where $x_{p}\in\mathbb{N}$ are the factors which determine each step (as an
integer multiple of the step width). $x_{p}$ might have an upper bound
depending on the capabilities of the AI accelerator. If Algorithm 1 detects a
linear influence of a parameter, it sets the step width $w_{p}=1$. The
described procedure works analogously for other layer types as well.

Although we performed parameter sweeps on the NVIDIA Jetson AGX GPU for Fully-
Connected, Pooling, and activation layers as well, only the following layers
exhibited a non-linear execution time behavior from which the following PRs
could be determined:

  * •

$Conv2D_{R}(x_{C}\cdot 8,x_{C_{h}}\cdot 8,x_{C_{w}}\cdot 16,x_{K}\cdot
32,F_{h},F_{w},s,pad)$ with $F_{h}=F_{w}\geq 3$

  * •

$PointwiseConv2D_{R}(x_{C}\cdot 8,x_{C_{h}}\cdot 4,x_{C_{w}}\cdot 4,x_{K}\cdot
32,F_{h}=F_{w}=1,s,pad)$

  * •

$DepthwiseConv2D_{R}(x_{C}\cdot 8,C_{h},C_{w},K=1,F_{h},F_{w})$ with input
channels $C$, input height and width $C_{h}$ and $C_{w}$, depthwise multiplier
$K$ and $F_{h}=F_{w}\geq 3$

Algorithm 1 Determine PRs from DNN layer parameter sweeps.

$S:P\to(\vec{x}$, $\vec{y})\leftarrow$Sweep measurements over relevant layer
parameters $P$,
$threshold_{linear}:\mathbb{R}^{+}$TestLinearBehavior$\vec{x}$, $\vec{y}$,
$threshold_{linear}$

1:$y_{min}\leftarrow min(\vec{y})$, $y_{max}\leftarrow max(\vec{y})$
$\triangleright$ Get min and max of the values in $\vec{y}$

2:$x_{min}\leftarrow min(\vec{x})$, $x_{max}\leftarrow max(\vec{x})$
$\triangleright$ Get min and max of the values in $\vec{x}$

3:$slope_{avg}\leftarrow\frac{y_{max}-y_{min}}{x_{max}-x_{min}}$
$\triangleright$ Compute average slope over the sweep measurements

4:$\hat{\vec{y}}\leftarrow slope_{avg}\cdot\vec{x}+x_{min}$ $\triangleright$
Linear erstimation of the execution time behavior

5:$rmse\leftarrow\mathrm{RMSE}(\vec{y},\hat{\vec{y}})$ $\triangleright$ RMSE
between measured and linear estimation $rmse<threshold_{linear}$

6:True $\triangleright$ Parameter has linear influence on execution time

7:False ExecutionTimeDelta$\vec{y}$

8:$\vec{deltas}\leftarrow[\ ]$ $y_{i},y_{i+1}\in\vec{y}$ $\triangleright$
Pairs of consecutive data points

9:$\vec{deltas}.append(y_{i+1}-y_{i}$)

10:$\vec{deltas}$

11:$W:P\to\mathbb{N}$ $\triangleright$ Step widths with which the parameters
influence the execution time $p,(\vec{x},\vec{y})\in S$ $\triangleright$ Layer
parameter $p$ with corresponding sweep over values $\vec{x}$ and measured
execution time $\vec{y}$ TestLinearBehavior$\vec{x}$, $\vec{y}$,
$threshold_{linear}$

12:$W[p]\leftarrow 1$

13:$\vec{deltas}\leftarrow$ ExecutionTimeDelta$\vec{y}$

14:$\vec{indices}\leftarrow$ FindPeaks$\vec{deltas}$ $\triangleright$ E.g.
scipy.signal.find_peaks()

15:$w_{p}\leftarrow$ PeakDistance$\vec{indices}$ $\triangleright$ Distance
between peaks is the step width

16:$W[p]\leftarrow w_{p}$

17:$W$

Gray-box accelerators is the category that falls in between the two extremes,
for which we provide two example AI accelerator architectures. The first one
is an undisclosed commercial AI accelerator, for which the vendor provided us
with a timing simulator under a non-disclosure agreement. We determined the
execution time behavior by running initial sweep benchmarks and used Algorithm
1 to find the step widths. We found a non-linear execution time behavior for
the following layers, resulting in the PRs:

  * •

$Conv2D_{R}(C,x_{C_{h}}\cdot 8,x_{C_{w}}\cdot 8,x_{K}\cdot
16,F_{h},F_{w},s,pad)$

  * •

$DepthwiseConv2D_{R}(C,x_{C_{h}}\cdot 8,x_{C_{w}}\cdot 8,K=1,F_{h},F_{w})$
with input channels $C$, input height $C_{h}$, input width $C_{w}$, depthwise
multiplier $K$ and $F_{h}=F_{w}\geq 3$

  * •

$FullyConnected_{R}(1,in,x_{out}\cdot 64)$ with inputs $in$ and outputs $out$

  * •

$AveragePool2D_{R}(C,x_{C_{h}}\cdot 8,x_{C_{w}}\cdot 8,F)$ with input channels
$C$, input height, $C_{h}$, input width $C_{w}$, and pooling kernel size $F$

From the information that is available to us, we additionally know that this
AI accelerator contains two separate functional units (FU). One optimized for
big Convolution and Fully-Connected operations and one for different
activation functions, Max- and Average-Pooling, and smaller Convolutions.
These FUs can execute consecutive layers in a DNN in an overlapping fashion.
We therefore constructed multi-layer benchmarks, similar to those in [11], to
verify this specific behavior. This information is used to improve multi-layer
and whole DNN estimations.

The second example is the Versatile Tensor Accelerator (VTA) [9] that is part
of the Apache TVM project222https://tvm.apache.org/, last accessed 03/26/2024.
In this case, an expert could theoretically retrieve all the required
information from the open-source Chisel implementation. But most of the time,
working with only semi-detailed information is necessary due to practical
limitations. To illustrate our methodology, we assume that we only know the
following about VTA: The supported operations are 2D-Convolution, general
matrix multiplication (GeMM), Fully-Connected, and element-wise matrix
operations. In the used configuration of the VTA hardware, the GeMM core can
compute a $(1,16)\times(16,16)$ matrix-matrix multiplication per cycle.
Furthermore, the documentation suggests that for a multiplication of matrices
$A$ and $B$, the columns of $A$, as well as the columns of $B$ have to be an
integer multiple of $16$ or padding needs to be applied. Analogously, for the
2D-Convolution, the parameters input channels $C$ and output channels $K$ have
to be integer multiples of 16 or again padding is necessary. To confirm these
PRs, we performed sweeps for the GeMM and the 2D-Convolution operation, which
revealed that the PRs are indeed:

$Conv2D_{R}(x_{C}\cdot 16,C_{h},C_{w},x_{K}\cdot
16,F_{h},F_{w},s,pad)\text{\quad and}$ (5)

$FullyConnected_{R}(1,x_{in}\cdot 16,x_{out}\cdot 16)$ (6)

###  3.3 Performance Estimation using Performance Representatives

As our goal is to systematically reduce the number of benchmark points for
training an estimator while still maintaining a good estimation accuracy, we
use Random Forest Regression to build the estimator as they have been shown to
be very accurate for DNN execution time estimation as described in [11, 13]
but other machine learning methods would also have been applicable [2].

After determining the PRs of an accelerator, we perform measurements using
only data points from the set of PRs. Most of the time, even this constrained
set of layer configurations is too large to be benchmarked completely. E.g.
the set of PRs of the $Conv1D$ operation supported by UltraTrail, still
contains $1\,493\,520$ possible configurations (the complete parameter space
contains $95\,585\,280$ data points). Consequently, we need to sample from the
set of PRs. In this study, we explore the accuracy of our approach, without
aiming for a specific target accuracy of the estimation. However, the
benchmark sampling and model training could easily be used in an optimization
loop until the desired estimation accuracy is achieved.

A relevant aspect to consider when using Random Forests is that this type of
model cannot extrapolate, but only estimate in the range of values it has seen
during training. Therefore, it is important to know for which DNN and,
consequently, for which layer parameters a model should be able to make
estimations and ensure that the range of benchmark points covers the area of
interest.

####  3.3.1 Single-Layer Execution Time Estimation

When it comes to making an estimation for a layer, we first map the layer
configuration in question to the corresponding PR by selecting the next larger
integer multiple of the step width $w_{p}$ for every parameter. For the
$Conv2D$ operation, this looks like this:

$\begin{split}Conv2D(&C,C_{h},C_{w},K,F_{h},F_{w},s,pad)\rightarrow\\\
Conv2D_{R}(&x_{C}\cdot w_{C},x_{C_{h}}\cdot w_{C_{h}},x_{C_{w}}\cdot
w_{C_{w}},\\\ &x_{K}\cdot w_{K},x_{F_{h}}\cdot w_{F_{h}},x_{F_{w}}\cdot
w_{F_{w}},s,pad)\\\ \end{split}$ (7)
$\text{with\quad}x_{p}=\biggl{\lceil}\frac{p}{w_{p}}\biggl{\rceil}\mathrm{for\
}p\in P.$ (8)

We then make the estimation for this representative layer configuration. Note
that for all parameters which have a linear influence on the execution time
behavior the step width is $w_{p}=1$ and, therefore, $x_{p}=p$ for $p\in P$.

This approach is effective because for the observed step-wise execution time
behavior of the accelerators in question, all configurations within the same
step exhibit very similar execution times.

####  3.3.2 Multi-Layer and Network Execution Time Estimation

To be able to make execution time estimations for whole DNNs, we opted to take
a similar approach to [11] and [13]. The authors of ANNETTE use two different
types of so called ’multi-layer benchmarks’ to capture layer fusion, while
Zhang et al. call a set of operators that are typically fused during model
deployment an ’execution kernel’. When estimating whole DNNs in both papers,
the authors search for those subgraphs in the complete model graph, make
estimations for the subgraphs and sum the individual estimated execution
times.

We also use the most common building blocks of DNNs for which we want to make
an estimation as multi-layer benchmarks. As those building blocks typically
contain a type of Convolution (Conv2D, DepthwiseConv2D, PointwiseConv2D), we
will again sample from the corresponding set of PRs to determine the
parameters of the Convolution, select the resulting shapes of the remaining
layers in the building block and carry out our benchmarks. These measurements
are then used to determine how the runtime of a complete building-block
behaves compared to the sum of individual execution times of the contained
layers.

In the estimation phase for a whole DNN we, firstly, determine which building
blocks are contained in the network and the layers and parameters inside the
building block. Secondly, we map the configuration of the building block’s
convolutional layers to the corresponding PRs as described in Eq. 7. And,
thirdly, we use those PRs to make estimations for the contained layers.

Depending on the platform, the estimation for a building block can differ. For
the undisclosed commercial AI accelerator with two separate FUs which can
execute certain layers in parallel, we estimate the execution time $t_{b}$ of
a building block $b$ as follows:

$t_{b}=\begin{cases}\max\\{t_{l}\mathrm{\ for\ }l\in
b\\}&\begin{split}\mathrm{if\ }b\in\\{&PWConv2D(DWConv2D(x)),\\\
&FullyConnected(Pool(x))\\}\end{split}\\\ \textstyle\sum_{l\in
b}{t_{l}}&\mathrm{else\ }\end{cases}\\\ $ (9)

Note that the undisclosed commercial AI accelerator fuses all ReLU layers with
the preceding layers. Therefore, ReLU layers do not need to be considered in
the sum of execution times.

In the case of the NVIDIA Jetson AGX Xavier GPU, we observed that the runtime
of a building block is shorter than the sum of the individual runtimes of the
contained layers. Therefore, we measured about 500 configurations per
building-block to determine a so called fusing factor. The individually
estimated layer execution times will be summed, corrected by this fusing
factor, resulting in the execution time of a building block:

$t_{b}=\textstyle\sum_{l\in b}{t_{l}}-f_{\beta}(b)$ (10)

The fusing factor $f_{\beta}(b)$ is calculated by a linear model for each type
of building block $\beta$ (see multi-layer benchmarks in Sec. 4.2) according
to Eq. 11. The weights $w_{\beta}$ and $c_{\beta}$ are fitted to the measured
multi-layer benchmarks. $\\#ops(b)$ denotes the total number of operations in
a block $b$.

$f_{\beta}(b)=\\#ops(b)\cdot w_{\beta}+c_{\beta}$ (11)

The execution time of the whole DNN is the sum of the contained building
blocks’ execution times:

$t_{DNN}=\textstyle\sum_{b\in B}{t_{b}}$ (12)

For DNN layers where we did not observe a step-wise execution time behavior,
we sample up to $9000$ data points and use these data points to train a Random
Forest model for usage in the estimation of a building block’s execution time.

##  4 Evaluation

We evaluate our methodology for reducing the number of samples needed for
training statistical estimation models for the UltraTrail accelerator (white-
box), the VTA accelerator (gray-box), the undisclosed commercial AI
accelerator (gray-box), and the NVIDIA Jetson AGX GPU (black-box).

###  4.1 Experimental Setup

We use the open-source tool ANNETTE333https://github.com/embedded-machine-
learning/annette, last accessed 03/26/2024 for generating single-layer and
multi-layer benchmarks, which we extended to use PyTorch instead of TensorFlow
models. These models are then deployed to the different platforms using
different backends. For UltraTrail we use the custom C-interface of the
accelerator together with the RTL simulation. As VTA is part of the Apache TVM
(version 0.15.0-dev0) project, we use TVM to deploy our models on the
accelerator and the Verilator RTL simulation to determine the execution time.
The measurements for the undisclosed commercial AI accelerator are performed
using the timing simulator provided by the vendor. Lastly, for deployment on
the NVIDIA Jetson AGX Xavier GPU running CUDA 11.4, we also use TVM with the
TensorRT 8.5.2 backend and record the execution times reported by TVM.

###  4.2 Benchmarks

As the NVIDIA Jetson AGX Xavier GPU and the undisclosed commercial AI
accelerator support a very broad range of operators, we ran single-layer
benchmarks for the layers that are contained in the DNNs we use for the whole
DNN evaluation. Namely, Conv2D, Depthwise Conv2D, Pointwise Conv2D, Average-
Pooling, Max-Pooling, and Fully-Connected. For UltraTrail we ran the supported
Conv1D and for VTA we benchmarked the two supported layers, Conv2D and Fully-
Connected (via the general matrix multiplication). All benchmarks were sampled
from the respective set of PRs.

As multi-layer benchmarks we selected recurring building-blocks of layers,
which form the majority of our DNNs used for whole DNN evaluation.

  * •

Depthwise Convolution followed by Pointwise Convolution, each followed by ReLU
(also known as Depthwise Separable Convolution), as used in MobileNet [6]:

$ReLU(PWConv2D(ReLU(DWConv2D(x))))$

Note that for the Jetson AGX GPU the influence of the ReLU layer on the
execution time was negligible. Therefore, no dedicated estimation models were
needed for ReLU on this platform.

  * •

A block of two Convolutions with a shortcut connection followed by an element-
wise addition with ReLU, as used in ResNet [5]:

$ReLU(Add(x,Conv2D_{1}(ReLU(Conv2D_{0}(x)))))$

  * •

A block of two Convolutions with a shortcut connection that contains another
Convolution, followed by an element-wise addition with ReLU, as used in ResNet
[5]:

$ReLU(Add(Conv2D_{2}(x),Conv2D_{1}(ReLU(Conv2D_{0}(x)))))$

  * •

A Max- or Average-Pooling layer followed by a Fully-Connected layer:

$FullyConnected(Pool(x))$

The execution time behavior of Max- and Average-Pooling was identical on the
undisclosed commercial AI accelerator and the Jetson AGX GPU. Therefore, a
combined Pooling model could be used. This multi-layer benchmark was only
performed for the undisclosed commercial AI accelerator, in order to verify
the parallel execution on the two separate FUs.

###  4.3 Test Set and Sampling Strategy Comparison

To show that the models we build from the PRs can actually estimate the layer
execution times of state-of-the-art DNNs we chose the layers from the Keras
model zoo444https://keras.io/api/applications/, last accessed 03/26/2024 to
build our test set. These layer parameters have not been seen during training
of our estimation models and will be used for evaluating the estimation
accuracy. For the UltraTrail accelerator, we use the Conv1D layers of the TC-
ResNet8 [3] as our test set.

To compare our method for selecting benchmark points against an uninformed
approach, we sampled different data set sizes randomly from the set of PRs and
randomly from the complete parameter space. We then trained Random Forest
estimation models using both sampling methods and evaluated the accuracy of
the prediction using the test set. If not otherwise specified, we report the
mean absolute percentage error (MAPE) of the estimation over all the test set
layers. An overview of the results is depicted in Table 1 as well as the mean
measurement time of a single benchmark point on the different platforms. In
case of the NVIDIA Jetson AGX Xavier GPU we performed each measurement $500$
times and used the median in order to mitigate the influence of the first few
slower runs, which were due to setup overhead. As the other measurements were
performed using simulators, there was no need to perform multiple runs of the
same benchmark point.

AI Accelerator | Layer Type Size | Dataset Size | RMSPE | MAPE |  Mean Meas. Time [$\mathrm{s}$]
---|---|---|---|---|---
UltraTrail | Conv1D | 9000 | $0.51\text{\,}\mathrm{\char 37\relax}$ | $0.33\text{\,}\mathrm{\char 37\relax}$ |  $0.38\text{\,}$ ($SD=0.07$)
VTA | Conv2D | 8000 | $4.49\text{\,}\mathrm{\char 37\relax}$ | $7.09\text{\,}\mathrm{\char 37\relax}$ |  $220.74\text{\,}$ ($SD=549.60$)
Fully-Connected | 9000 | $0.44\text{\,}\mathrm{\char 37\relax}$ | $0.02\text{\,}\mathrm{\char 37\relax}$ |  $2.21\text{\,}$ ($SD=1.65$)
Undisclosed Commercial AI Accelerator | Conv2D | 9000 | $9.93\text{\,}\mathrm{\char 37\relax}$ | $7.35\text{\,}\mathrm{\char 37\relax}$ |  $5.34\text{\,}$ ($SD=0.36$)
Depthwise Conv2D | 4000 | $11.59\text{\,}\mathrm{\char 37\relax}$ | $3.44\text{\,}\mathrm{\char 37\relax}$ |  $5.29\text{\,}$ ($SD=0.22$)
Fully-Connected | 5000 | $0.34\text{\,}\mathrm{\char 37\relax}$ | $0.35\text{\,}\mathrm{\char 37\relax}$ |  $5.36\text{\,}$ ($SD=0.12$)
2D Average Pooling | 4000 | $6.09\text{\,}\mathrm{\char 37\relax}$ | $2.09\text{\,}\mathrm{\char 37\relax}$ |  $5.29\text{\,}$ ($SD=0.05$)
NVIDIA Jetson AGX GPU | Conv2D | 8000 | $27.06\text{\,}\mathrm{\char 37\relax}$ | $13.13\text{\,}\mathrm{\char 37\relax}$ |  $47.59\text{\,}$ ($SD=13.29$)
Pointwise Conv2D | 9000 | $29.99\text{\,}\mathrm{\char 37\relax}$ | $13.84\text{\,}\mathrm{\char 37\relax}$ |  $46.00\text{\,}$ ($SD=11.22$)
Depthwise Conv2D | 9000 | $11.43\text{\,}\mathrm{\char 37\relax}$ | $7.37\text{\,}\mathrm{\char 37\relax}$ |  $65.12\text{\,}$ ($SD=40.65$)
Table 1: Overview of the dataset size and RMSPE for the best MAPE results for
all AI accelerators and layer types for which the PR sampling and mapping was
applied. The last column shows the mean measurement time of a single benchmark
point for the different platforms. For the Jetson AGX we performed $500$ runs
for each benchmark.

###  4.4 UltraTrail (white-box)

Fig. 4 shows how the estimation accuracy develops when using different amounts
of training data. In this case, sampling a dataset of only $9000$ layers from
the set of PRs, yields an excellent MAPE of only $0.33\text{\,}\mathrm{\char
37\relax}$. This stems from the fact that the execution time behavior of the
UltraTrail accelerator is extremely regular and the measurements of the RTL
simulation do not contain any noise. Training the estimation model with the
same amount of randomly sampled data points only reaches an accuracy of
$10.98\text{\,}\mathrm{\char 37\relax}$.

We do not present whole DNN estimations for UltraTrail because the execution
time of multiple layers is the sum of the single layer execution times [1].

Figure 4: Comparison of the estimation accuracy when sampling from the set of
PRs vs. random sampling from the complete parameter space for the Conv1D layer
on the UltraTrail accelerator.

###  4.5 VTA (gray-box)

Looking at the results of VTA, one of the gray-box accelerators, in Fig. 5 one
can clearly see that our approach achieves a lower MAPE for the execution time
estimation of Conv2D and Fully-Connected layers compared to random sampling
from the complete set of layer parameters. For this accelerator, we achieved
our top result for the Conv2D layer of only $7.09\text{\,}\mathrm{\char
37\relax}$ MAPE and the Fully-Connected layer of only
$0.02\text{\,}\mathrm{\char 37\relax}$ MAPE.

Figure 5: Comparison of the estimation accuracy when sampling from the set of
PRs vs. random sampling from the complete parameter space for the Conv2D and
Fully-Connected layer on the VTA accelerator.

###  4.6 Undisclosed Commercial AI Accelerator (gray-box)

When comparing PR sampling and mapping against random sampling in Fig. 6, one
can again see that our method results in a better estimation accuracy overall,
but additionally the MAPE decreases much faster when increasing the training
dataset size. Only for the Fully-Connected layer, our method does not
outperform the random selection. But as we can see, the MAPE is always below
$1\text{\,}\mathrm{\char 37\relax}$, meaning that the execution time of the
Fully-Connected layer is straightforward to estimate even with very few
training samples.

Figure 6: Comparison of the estimation accuracy when sampling from the set of
PRs vs. random sampling from the complete parameter space for different layers
on the undisclosed commercial AI accelerator.

We also applied our estimation models to estimate the execution time of whole
DNNs using Eq. 9 and 12 to account for overlapping layer execution due to the
two distinct FUs of the undisclosed commercial AI accelerator. The results are
displayed in Table 2. We achieve an excellent estimation accuracy due to the
step-wise execution time behavior of several layers on this accelerator as
well as our knowledge about the parallel execution.

AI Accelerator | DNN |  Meas. time [$\mathrm{ms}$] |  Est. time [$\mathrm{ms}$] | % Error
---|---|---|---|---
Undisclosed Commercial AI Accelerator |  MobileNet [6] | $2.95$ | $2.93$ | $0.68\text{\,}\mathrm{\char 37\relax}$
ResNet18 [5] | $4.88$ | $4.63$ | $5.12\text{\,}\mathrm{\char 37\relax}$
NVIDIA Jetson AGX GPU |  MobileNet [6] | $1.79$ | $2.14$ | $19.55\text{\,}\mathrm{\char 37\relax}$
ResNet18 [5] | $2.90$ | $3.47$ | $19.66\text{\,}\mathrm{\char 37\relax}$
Table 2: Whole DNN estimation results for the undisclosed commercial AI
accelerator and the NVIDIA Jetson AGX Xavier GPU.

###  4.7 NVIDIA Jetson AGX Xavier GPU (black-box)

The NVIDIA Jetson AGX Xavier GPU is not a highly specialized accelerator, but
an embedded GPU. We still wanted to show that our approach can be used with
this kind of computing device.

In Fig. 7 we see the MAPE of the single-layer estimations using PR sampling
and mapping, and random sampling from the complete parameter space. The
estimation accuracy is generally worse for the AGX Xavier GPU compared to the
other AI accelerators, which can be explained by the fact that for this black-
box accelerator we have almost no information to guide the sweep benchmarks or
the selection of the PR. Moreover, the complexity of an embedded GPU is
generally higher than for specialized accelerators. Still, for Conv2D our
approach yields better results compared to random sampling from such a small
training dataset. For the Pointwise Conv2D and the Depthwise Conv2D our
approach does not outperform the random selection.

This can be attributed to the fact that only one parameter (input channels
$C$) has a step-wise influence on the execution time behavior of the Depthwise
Conv2D, resulting in a set of PRs that is similar to the complete set of layer
configurations. For the Pointwise Conv2D this can be explained by the overall
very short execution times of this layer type (longest observed execution time
during parameter sweeps was $0.28\text{\,}\mathrm{ms}$). For such fast layers,
the overhead associated with the execution of the operations amounts to a
bigger portion of the total execution time, while our approach focuses on the
computational part.

Figure 7: Comparison of the estimation accuracy when sampling from the set of
PRs vs. random sampling from the complete parameter space for different layers
on the NVIDIA Jetson AGX Xavier GPU.

The results for the whole DNN estimations can be seen in Table 2. The
estimation accuracy is worse compared to the undisclosed commercial AI
accelerator. This can be attributed again to the fact that the Jetson AGX GPU
is a black-box accelerator, for which we have almost no information,
especially when it comes to how multiple layers are mapped onto the GPU
architecture together.

###  4.8 Comparison to State-of-the-Art

A comparison to other state-of-the-art (SOTA) execution time estimation
approaches is presented in Table 3. If not otherwise stated, it shows the
literature reported RMSPE and MAPE values of the approaches presented in 2
Related Work. Note that often only one of the metrics is available. Our
approach has comparable accuracy to the other SOTA methods, while needing much
fewer training samples, especially compared to the best approach for
estimating execution time on the NVIDIA Jetson AGX Xavier GPU by Bouzidi et
al. [2] who used approximately $200\,000$ whole DNN benchmarks for training
their estimator.

Work | Type | Platform | Dataset Size | RMSPE | MAPE
---|---|---|---|---|---
ANNETTE [11] | Conv2D Layer | NCS2 |  $35\,000$ per platform | $42.60\text{\,}\mathrm{\char 37\relax}$ | $15.57\text{\,}\mathrm{\char 37\relax}$
ZCU102 | $10.55\text{\,}\mathrm{\char 37\relax}$ | $12.71\text{\,}\mathrm{\char 37\relax}$
Whole DNN | NCS2 | $36\,570$ | - | $7.44\text{\,}\mathrm{\char 37\relax}$
ZCU102 | $37\,812$ | - | $3.47\text{\,}\mathrm{\char 37\relax}$
Blackthorn [7] | Conv2D Layer | Jetson Nano |  $15\,000$ per platform | $5.89\text{\,}\mathrm{\char 37\relax}$ | -
Jetson TX2 | $6.10\text{\,}\mathrm{\char 37\relax}$ | -
Whole DNN | Jetson Nano |  no multi-layer models |  $1.71\text{\,}\mathrm{\char 37\relax}$1 |  $2.95\text{\,}\mathrm{\char 37\relax}$1
Jetson TX2 |  $3.06\text{\,}\mathrm{\char 37\relax}$1 |  $4.29\text{\,}\mathrm{\char 37\relax}$1
Bouzidi et al. [2] | Whole DNN | Jetson AGX |  $\sim$$200\,000$ DNNs per platform | - | $7.67\text{\,}\mathrm{\char 37\relax}$
Jetson TX2 | - | $8.37\text{\,}\mathrm{\char 37\relax}$
nn-Meter [13] | Whole DNN | Cortex-A76 |  $15\,824$2 |  [range-phrase= – ]2.765.543 | -
Adreno 640 |  $14\,040$2 |  [range-phrase= – ]1.355.323 | -
NCS2 |  $39\,968$2 |  [range-phrase= – ]4.2622.253 | -
This | Conv2D Layer | Undisclosed | 9000 | $9.93\text{\,}\mathrm{\char 37\relax}$ | $7.35\text{\,}\mathrm{\char 37\relax}$
Jetson AGX | 8000 | $27.06\text{\,}\mathrm{\char 37\relax}$ | $13.13\text{\,}\mathrm{\char 37\relax}$
Whole DNN | Undisclosed |  max. $9000$4 \+ max. $500$5 | $4.53\text{\,}\mathrm{\char 37\relax}$ | $2.90\text{\,}\mathrm{\char 37\relax}$
Jetson AGX | $20.17\text{\,}\mathrm{\char 37\relax}$ | $19.60\text{\,}\mathrm{\char 37\relax}$
Table 3: Comparison with SOTA execution time estimation methods.
1values were computed from the reported times, 2size of the dataset for the
Conv+bn+relu execution kernel, 3the authors reported RMSPE separately for each
DNN, 4per layer type, 5per building-block for the fusing factor on the AGX

##  5 Conclusion and Future Work

This paper presented a performance modeling methodology and associated
benchmarking strategy to reduce the number of training samples for statistical
performance models, and thus reduce the required measurement time while
maintaining comparable estimation accuracy to state-of-the-art estimators. We
leverage knowledge about the AI accelerator architecture and mapping to
determine Performance Representatives (PR) algorithmically for the
benchmarking and, later, estimation phase. We evaluated our approach for four
different AI accelerators and, depending on the available hardware
architecture knowledge, achieved an execution time estimation MAPE between
$0.02\text{\,}\mathrm{\char 37\relax}13.84\text{\,}\mathrm{\char 37\relax}$
for single-layers and $2.90\text{\,}\mathrm{\char
37\relax}19.60\text{\,}\mathrm{\char 37\relax}$ for whole DNNs.

As future work, we plan to use the PRs of an AI accelerator as a search space
constraint in hardware-aware NAS and to extend the PRs to additionally include
the memory characteristics of the AI accelerator.

####  5.0.1 Acknowledgements

This work has been funded by the German Federal Ministry of Education and
Research (BMBF) under grant number 16ES0876 (GENIAL!).

####  5.0.2

The authors have no competing interests to declare that are relevant to the
content of this article.

## References

  * [1] Bernardo, P.P., Gerum, C., Frischknecht, A., Lübeck, K., Bringmann, O.: UltraTrail: A Configurable Ultralow-Power TC-ResNet AI Accelerator for Efficient Keyword Spotting. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 39(11), 4240–4251 (Nov 2020)
  * [2] Bouzidi, H., Ouarnoughi, H., Niar, S., Cadi, A.A.E.: Performance Prediction for Convolutional Neural Networks on Edge GPUs. In: Proceedings of the 18th ACM International Conference on Computing Frontiers. CF ’21, ACM (May 2021)
  * [3] Choi, S., Seo, S., Shin, B., Byun, H., Kersner, M., Kim, B., Kim, D., Ha, S.: Temporal Convolution for Real-time Keyword Spotting on Mobile Devices (2019)
  * [4] Gerum, C., Frischknecht, A., Hald, T., Bernardo, P.P., Lübeck, K., Bringmann, O.: Hardware Accelerator and Neural Network Co-optimization for Ultra-Low-Power Audio Processing Devices. In: 2022 25th Euromicro Conference on Digital System Design (DSD). IEEE (Aug 2022)
  * [5] He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition (2015)
  * [6] Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications (2017)
  * [7] Lechner, M., Jantsch, A.: Blackthorn: Latency Estimation Framework for CNNs on Embedded Nvidia Platforms. IEEE Access 9, 110074–110084 (2021)
  * [8] Lübeck, K., Jung, A.L.F., Wedlich, F., Bringmann, O.: Work-in-Progress: Ultra-fast yet Accurate Performance Prediction for Deep Neural Network Accelerators. In: 2022 International Conference on Compilers, Architecture, and Synthesis for Embedded Systems (CASES). IEEE (Oct 2022)
  * [9] Moreau, T., Chen, T., Vega, L., Roesch, J., Yan, E., Zheng, L., Fromm, J., Jiang, Z., Ceze, L., Guestrin, C., Krishnamurthy, A.: A Hardware-Software Blueprint for Flexible Deep Learning Specialization (2019)
  * [10] Parashar, A., Raina, P., Shao, Y.S., Chen, Y.H., Ying, V.A., Mukkara, A., Venkatesan, R., Khailany, B., Keckler, S.W., Emer, J.: Timeloop: A systematic approach to dnn accelerator evaluation. In: 2019 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). IEEE (Mar 2019)
  * [11] Wess, M., Ivanov, M., Unger, C., Nookala, A., Wendt, A., Jantsch, A.: ANNETTE: Accurate Neural Network Execution Time Estimation with Stacked Models. IEEE Access 9, 3545–3556 (2021)
  * [12] Williams, S., Waterman, A., Patterson, D.: Roofline: an insightful visual performance model for multicore architectures. Communications of the ACM 52(4), 65–76 (Apr 2009)
  * [13] Zhang, L.L., Han, S., Wei, J., Zheng, N., Cao, T., Yang, Y., Liu, Y.: nn-Meter: towards accurate latency prediction of deep-learning model inference on diverse edge devices. In: Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services. MobiSys ’21, ACM (Jun 2021)


# Asymptotic behavior for the radial eigenvalues of $p$-Laplacian in certain
annular domains

Anderson L. A. de Araujo  Departamento de Matemática, Universidade Federal de
Viçosa, 36570-900, Viçosa (MG), Brazil<EMAIL_ADDRESS>
###### Abstract.

In this paper we prove an asymptotic behavior for the radial eigenvalues to
the Dirichlet $p$-Laplacian problem $-\Delta_{p}\,u=\lambda\,|u|^{p-2}u$ in
$\Omega$, $u=0$ on $\partial\Omega$, where $\Omega$ is an annular domain
$\Omega=\Omega_{R,\overline{R}}$ in $\mathbb{R}^{N}$.

###### Key words and phrases:

Annular Domain, $p$-Laplacian, Asymptotics of Eigenvalues

###### 2010 Mathematics Subject Classification:

Primary 35P15; Secondary 49Rxx

##  1\. Introduction

This paper investigates an asymptotic behavior for the radial eigenvalues
$\lambda_{k}=\lambda_{k}(R,\overline{R})$ (when $0<\overline{R}=R+1$ and
$R\to+\infty$) to the following eigenvalue problem

(1.1)
$\left\\{\begin{array}[]{lll}\displaystyle-\Delta_{p}\,u=\lambda\,|u|^{p-2}u&\textup{in}&\Omega_{R,\overline{R}},\\\
\displaystyle
u=0&\textup{on}&\partial\Omega_{R,\overline{R}},\end{array}\right.$

where $\Omega_{R,\overline{R}}=\\{x\in\mathbb{R}^{N}:R<|x|<\overline{R}\\}$,
with $0<R<\overline{R}$ constants in $\mathbb{R}$, is the annular domain, and
we suppose that

$1<p\leq\,N.$

In particular, when $p=2$, we obtain the Dirichlet Laplacian problem

(1.2)
$\left\\{\begin{array}[]{rll}\displaystyle-\Delta\,u=\lambda\,u&\textup{in}&\Omega_{R,\overline{R}},\\\
\displaystyle
u=0&\textup{on}&\partial\Omega_{R,\overline{R}}.\end{array}\right.$

Since we are interested only in the radial eigenvalues of (1.1), we can
rewrite (1.1) as the following $1$-dimensional eigenvalue problem

(1.3)
$\left\\{\begin{array}[]{lll}\displaystyle(r^{N-1}|u^{\prime}(r)|^{p-2}u^{\prime}(r))^{\prime}+\lambda\,r^{N-1}|u(r)|^{p-2}u(r)=0&\textup{in}&(R,\bar{R}),\\\
\displaystyle u(R)=u(\bar{R})=0.&&\end{array}\right.$

We remark that for every $1\leq k\in\mathbb{N}$, if we denote by $\lambda_{k}$
the $k$-th eigenvalue of (1.3) and by $\lambda_{k}^{\mbox{rad}}$ the $k$-th
radial eigenvalue of (1.1),

$\lambda_{k}=\lambda_{k}^{\mbox{rad}}.$

In order to study the solution of (1.1), one can make a standard change of
variables, see for example [1, 5].

If $N>p$, let $t=-\frac{A}{r^{(N-p)/(p-1))}}+B$ and $v(t)=u(r)$, where

$A=\frac{(R\overline{R})^{\frac{N-p}{p-1}}}{\overline{R}^{\frac{N-p}{p-1}}-R^{\frac{N-p}{p-1}}}\,\,\,\mbox{and}\,\,\,B=\frac{\overline{R}^{\frac{N-p}{p-1}}}{\overline{R}^{\frac{N-p}{p-1}}-R^{\frac{N-p}{p-1}}},$

then the problem (1.3) (hence (1.1)) transforms into the boundary value
problem for the nonautonomous ODE

(1.4)
$\left\\{\begin{array}[]{lll}\displaystyle(|v^{\prime}(t)|^{p-2}v^{\prime}(t))^{\prime}+\lambda\,q(t)|v(t)|^{p-2}v(t)=0&\textup{in}&(0,1),\\\
\displaystyle v(0)=v(1)=0.&&\end{array}\right.$

where

(1.5)
$q(t):=q_{R,\overline{R}}(t)=\left(\frac{p-1}{N-p}\right)^{p}\frac{A^{\frac{(p-1)p}{N-p}}}{(B-t)^{\frac{p(N-1)}{N-p}}}.$

In the case $p=N$, one sets $r=R(\frac{\overline{R}}{R})^{t}$ and $v(t)=u(r)$,
obtaining again the problem (1.4), now with

(1.6)
$q(t):=q_{R,\overline{R}}(t)=\left[R\left(\frac{\overline{R}}{R}\right)^{t}\ln\frac{\overline{R}}{R}\right]^{p}.$

Let $\lambda_{k}$ be the $k$-th eigenvalue of (1.4) and let $\varphi_{k}$ be
an eigenfunction corresponding to $\lambda_{k}$. Since $q$ satisfies

$q\in C^{1}([0,1]),\,\,\,q(t)>0\,\,\,\mbox{for}\,\,\,0\leq t\leq 1.$

It is known that

$0<\lambda_{1}<\lambda_{2}<\cdots<\lambda_{k}<\lambda_{k+1}<\cdots,\,\,\,\lim_{k\to\infty}\lambda_{k}=\infty,$

and that $\varphi_{k}$ has exactly $k-1$ zeros in $(0,1)$, see [2, 3].

Motivated by the work of Zhang [7], whose purpose was to compute an estimate
for eigenvalues of the Dirichlet $p$-Laplacian (1.4), $p>1$, we propose to
prove an asymptotic behavior for the $\lambda_{k}(R,\overline{R})$ in the
form:

$\lim_{R\to+\infty}\lambda_{k}(R,R+1).$

The following estimate is known, see Zhang [7, Remark 2.1]. We suppose that

(1.7) $\pi_{p}=\frac{2\pi(p-1)^{1/p}}{p\sin(\pi/p)},$
$\bar{q}_{-}(R,\overline{R})=\int_{0}^{1}\min\\{1,q(t)\\}dt$

and

$\bar{q}_{+}(R,\overline{R})=\int_{0}^{1}\max\\{1,q(t)\\}dt.$

In [7] (with $T=1$), the author proved the inequality

(1.8)
$\left(\frac{k\pi_{p}}{\bar{q}_{+}(R,\overline{R})}\right)^{p}\leq\lambda_{k}(R,\overline{R})\leq\left(\frac{k\pi_{p}}{\bar{q}_{-}(R,\overline{R})}\right)^{p}.$

In S.S. Lin [4, Lemma A.1], the author proves an asymptotic behavior for all
eigenvalues (that is, radial and non-radial eigenvalues), of Dirichlet problem
(1.2), which is the following result.

######  Lemma 1.1 ([4]).

Let $\lambda_{k,j}(R,R+1)$ be the $j$-th eigenvalue of

$\phi^{\prime\prime}+\frac{N-1}{r}\phi^{\prime}-\frac{\alpha_{k}}{r^{2}}\phi=-\lambda_{k,j}(R,R+1)\phi,\mbox{
in }(R,R+1),$ $\phi(R)=\phi(R+1)=0,$

where $\alpha_{k}=k(k+N-2)$, and let $\lambda_{j}=j^{2}\pi^{2}$ be the $j$-th
eigenvalue of

$\phi^{\prime\prime}=-\lambda_{j}\phi,\mbox{ in }(0,1),$ $\phi(0)=\phi(1)=0,$

where $k=0,1,2\cdots$ and $j=1,2,3,\cdots$. Then

$\lim\limits_{R\to+\infty}\lambda_{k,j}(R,R+1)=\lambda_{j}.$

In this paper, we will prove a generalization of the results of [4], for the
radial eigenvalues of $p$-Laplacian, in the cases when $p=r+1$ and $N=2r+1$,
with $r\in\mathbb{N}$; $p=N$ and $p=2<3\leq N$. The last case is another proof
of Lin’s result for the radial eigenvalues. It is noteworthy that it is not
yet known a characterization for all eigenvalues of (1.1). The results of [4]
are consequence of some results of Bessel functions, while in our paper we use
a totally different approach following the work of Zhang [7].

We state the main result.

######  Theorem 1.2.

(i) Suppose that $N=p$. Then

(1.9)
$\lim_{R\to+\infty}\bar{q}_{+}(R,R+1)=\lim_{R\to+\infty}\bar{q}_{-}(R,R+1)=1.$

In particular, by (1.8)

(1.10) $\lim_{R\to+\infty}\lambda_{k}(R,R+1)=\left(k\pi_{p}\right)^{p},$

that is, the eigenvalues $\lambda_{k}(R,R+1)$ converge asymptotically to the
eigenvalues of the problem

(1.11)
$\left\\{\begin{array}[]{lll}\displaystyle(|v^{\prime}(t)|^{p-2}v^{\prime}(t))^{\prime}+\lambda\,|v(t)|^{p-2}v(t)=0&\textup{in}&(0,1),\\\
\displaystyle v(0)=v(1)=0.&&\end{array}\right.$

(ii) Suppose that $p=2$ and $N\geq 3$. Then,

(1.12)
$\lim_{R\to+\infty}\bar{q}_{+}(R,R+1)=\lim_{R\to+\infty}\bar{q}_{-}(R,R+1)=1.$

In particular, by (1.8)

(1.13) $\lim_{R\to+\infty}\lambda_{k}(R,R+1)=\left(k\pi\right)^{2},$

that is, the eigenvalues $\lambda_{k}(R,R+1)$ converge asymptotically to the
eigenvalues of the problem

(1.14) $\left\\{\begin{array}[]{lll}\displaystyle
v^{\prime\prime}(t)+\lambda\,v(t)=0&\textup{in}&(0,1),\\\ \displaystyle
v(0)=v(1)=0.&&\end{array}\right.$

##  2\. Proofs

###### Proof of Theorem 1.2.

(i): By (1.6), we have

(2.1)
$q(t)=\left[R\left(\frac{\overline{R}}{R}\right)^{t}\ln\frac{\overline{R}}{R}\right]^{p}.$

Therefore,

$q^{\prime}(t)=p\left[R\left(\frac{\overline{R}}{R}\right)^{t}\ln\frac{\overline{R}}{R}\right]^{p-1}R\left(\frac{\overline{R}}{R}\right)^{t}\left(\ln\frac{\overline{R}}{R}\right)^{2}>0,$

hence, the function $q$ is increasing at $t$ and

$0<q(0)\leq q(t)\leq q(1),\,\,\ \forall\,\,t\in[0,1],$

that is,

$\left[R\ln\frac{\overline{R}}{R}\right]^{p}\leq
q(t)\leq\left[\overline{R}\ln\frac{\overline{R}}{R}\right]^{p},\,\,\
\forall\,\,t\in[0,1].$

If $R>0$ and $\overline{R}=R+1$, we have

$q(0)=\left[R\ln\left(\frac{R+1}{R}\right)\right]^{p}=\left[\ln\left(1+\frac{1}{R}\right)^{R}\right]^{p}$

and

$q(1)=\left[(R+1)\ln\left(\frac{R+1}{R}\right)\right]^{p}=\left[\ln\left(1+\frac{1}{R}\right)^{R}+\ln\left(1+\frac{1}{R}\right)\right]^{p}.$

Therefore,

(2.2) $\lim_{R\to+\infty}q(0)=1$

and

(2.3) $\lim_{R\to+\infty}q(1)=1,$

where we used the Euler Number

$e=\lim_{R\to+\infty}\left(1+\frac{1}{R}\right)^{R}.$

By (2.2) and (2.3), we obtain that

$\lim_{R\to+\infty}q(t)=1\mbox{ uniformly in }t\in[0,1].$

As

$\bar{q}_{-}(R,R+1)=\int_{0}^{1}\min\\{1,q(t)\\}dt=\int_{0}^{1}{\frac{1+q(t)-|1-q(t)|}{2}}dt,$

we conclude that

(2.4) $\lim_{R\to+\infty}\bar{q}_{-}(R,R+1)=1.$

As

$\bar{q}_{+}(R,R+1)=\int_{0}^{1}\max\\{1,q(t)\\}dt=\int_{0}^{1}{\frac{1+q(t)+|1-q(t)|}{2}}dt,$

we conclude that

(2.5) $\lim_{R\to+\infty}\bar{q}_{+}(R,R+1)=1.$

It follows from (1.8) that

(2.6)
$\left(\frac{k\pi_{p}}{\bar{q}_{+}(R,R+1)}\right)^{p}\leq\lambda_{k}(R,R+1)\leq\left(\frac{k\pi_{p}}{\bar{q}_{-}(R,R+1)}\right)^{p}.$

By (2.4), (2.5) and by limits in (2.6), we obtain

$\lim_{R\to+\infty}\lambda_{k}(R,R+1)=(\pi_{p}k)^{p}.$

The proof of $(\pi_{p}k)^{p}$ is a solution of (1.11), according to Zhang [7]
(see also del Pino and Manasevich [6]).

Proof of (ii): By (1.5), if $N>p$, we have

(2.7)
$q(t)=\left(\frac{p-1}{N-p}\right)^{p}A^{\frac{(p-1)p}{N-p}}(B-t)^{-\frac{p(N-1)}{N-p}}.$

Therefore,

$q^{\prime}(t)=\left(\frac{p-1}{N-p}\right)^{p}A^{\frac{(p-1)p}{N-p}}\frac{p(N-1)}{N-p}(B-t)^{-\frac{p(N-1)}{N-p}-1}>0,$

and the function $q$ is increasing at $t$. Hence,

$0<q(0)\leq q(t)\leq q(1),,\,\,\ \forall\,\,t\in[0,1].$

Since $p=2$ and $N\geq 3$, by(2.7), we have

$q(t)=(N-2)^{-2}\frac{\left(\frac{(R\overline{R})^{N-2}}{\overline{R}^{N-2}-R^{N-2}}\right)^{\frac{2}{N-2}}}{\left(\frac{(\overline{R})^{N-2}}{\overline{R}^{N-2}-R^{N-2}}-t\right)^{\frac{2(N-1)}{N-2}}}.$

In particular,

$q(0)=\frac{1}{(N-2)^{2}}R^{2}\left(\frac{\overline{R}^{N-2}-R^{N-2}}{\overline{R}^{N-2}}\right)^{2}$

and

$q(1)=\frac{1}{(N-2)^{2}}\overline{R}^{2}\left(\frac{\overline{R}^{N-2}-R^{N-2}}{R^{N-2}}\right)^{2}.$

If $R>0$ and $\overline{R}=R+1$, we have

$\begin{array}[]{rcl}q(0)&=&\frac{1}{(N-2)^{2}}R^{2}\left(\frac{(R+1)^{N-2}-R^{N-2}}{(R+1)^{N-2}}\right)^{2}\\\
&=&\frac{1}{(N-2)^{2}}R^{2}(R+1-R)^{2}\left(\frac{(R+1)^{N-3}+(R+1)^{N-4}R+\dots+(R+1)R^{N-4}+R^{N-3}}{(R+1)^{N-2}}\right)^{2}\\\
&=&\displaystyle\frac{1}{(N-2)^{2}}R^{2}\left(\frac{\sum_{j=0}^{N-3}{(R+1)^{N-(3+j)}R^{j}}}{(R+1)^{N-2}}\right)^{2}\\\
\end{array}$

and we conclude that

(2.8) $\lim_{R\to+\infty}q(0)=1,$

where we used that

$R^{2}\left(\frac{\sum_{j=0}^{N-3}{(R+1)^{N-(3+j)}R^{j}}}{(R+1)^{N-2}}\right)^{2}=\left(1-\frac{1}{R+1}\right)^{2}\left(\sum_{j=0}^{N-3}{\left(1-\frac{1}{R+1}\right)^{j}}\right)^{2}\to(N-2)^{2},$

as $R\to+\infty$.

Similarly,

$\begin{array}[]{rcl}q(1)&=&\frac{1}{(N-2)^{2}}(R+1)^{2}\left(\frac{(R+1)^{N-2}-R^{N-2}}{R^{N-2}}\right)^{2}\\\
&=&\frac{1}{(N-2)^{2}}(R+1)^{2}((R+1)-R)^{2}\left(\frac{(R+1)^{N-3}+(R+1)^{N-4}R+\dots+(R+1)R^{N-4}+R^{N-3}}{R^{N-2}}\right)^{2}\\\
&=&\displaystyle\frac{1}{(N-2)^{2}}(R+1)^{2}\left(\frac{\sum_{j=0}^{N-3}{(R+1)^{N-(3+j)}R^{j}}}{R^{N-2}}\right)^{2}\\\
\end{array}$

and we conclude that

(2.9) $\lim_{R\to+\infty}q(1)=1.$

Therefore, by (2.8) and (2.9), we obtain that

$\lim_{R\to+\infty}q(t)=1\mbox{ uniformly in }t\in[0,1].$

As

$\bar{q}_{-}(R,R+1)=\int_{0}^{1}\min\\{1,q(t)\\}dt=\int_{0}^{1}{\frac{1+q(t)-|1-q(t)|}{2}}dt,$

we conclude that

(2.10) $\lim_{R\to+\infty}\bar{q}_{-}(R,R+1)=1.$

As

$\bar{q}_{+}(R,R+1)=\int_{0}^{1}\max\\{1,q(t)\\}dt=\int_{0}^{1}{\frac{1+q(t)+|1-q(t)|}{2}}dt,$

we conclude that

(2.11) $\lim_{R\to+\infty}\bar{q}_{+}(R,R+1)=1.$

Since by (1.7) we have $\pi_{2}=\pi$, it follows from (1.8) that

(2.12)
$\left(\frac{k\pi}{\bar{q}_{+}(R,R+1)}\right)^{2}\leq\lambda_{k}(R,R+1)\leq\left(\frac{k\pi}{\bar{q}_{-}(R,R+1)}\right)^{2}.$

By (2.10), (2.11) and by limits in (2.12), we obtain

$\lim_{R\to+\infty}\lambda_{k}(R,R+1)=\pi^{2}k^{2}.$

This proves the item ($ii$). ∎

##  3\. Additional results

Similar to Theorem 1.2, we can get the following result. Let $r\in\mathbb{N}$.
Suppose that $p=r+1$ and $N=2r+1$. Then,

(3.1)
$\lim_{R\to+\infty}\bar{q}_{+}(R,R+1)=\lim_{R\to+\infty}\bar{q}_{-}(R,R+1)=1.$

In particular, by (1.8)

(3.2)
$\lim_{R\to+\infty}\lambda_{k}(R,R+1)=\left(k\pi_{p}\right)^{p}=(\pi_{r+1}k)^{r+1}.$

Indeed, since $p=r+1$ and $N=2r+1$, then

$\left(\frac{p-1}{N-p}\right)^{p}=1,\frac{N-p}{p-1}=1,\frac{(p-1)p}{N-p}=r+1,\frac{p(N-1)}{N-p}=2(r+1)$

and by (2.7), we have

$q(t)=\frac{\left(\frac{R\overline{R}}{\overline{R}-R}\right)^{r+1}}{\left(\frac{\overline{R}}{\overline{R}-R}-t\right)^{2(r+1)}}.$

If $R>0$ and $\overline{R}=R+1$, we have

$q(t)=\frac{(R^{2}+R)^{r+1}}{(R+1-t)^{2(r+1)}}.$

In particular,

$q(0)=\left(\frac{R}{R+1}\right)^{r+1}<1$

and

$q(1)=\left(\frac{R+1}{R}\right)^{r+1}>1.$

Therefore,

(3.3) $\lim_{R\to+\infty}q(0)=1$

and

(3.4) $\lim_{R\to+\infty}q(1)=1.$

By (3.3) and (3.4), we obtain that

$\lim_{R\to+\infty}q(t)=1\mbox{ uniformly in }t\in[0,1].$

As

$\bar{q}_{-}(R,R+1)=\int_{0}^{1}\min\\{1,q(t)\\}dt=\int_{0}^{1}{\frac{1+q(t)-|1-q(t)|}{2}}dt,$

we conclude that

(3.5) $\lim_{R\to+\infty}\bar{q}_{-}(R,R+1)=1.$

As

$\bar{q}_{+}(R,R+1)=\int_{0}^{1}\max\\{1,q(t)\\}dt=\int_{0}^{1}{\frac{1+q(t)+|1-q(t)|}{2}}dt,$

we conclude that

(3.6) $\lim_{R\to+\infty}\bar{q}_{+}(R,R+1)=1.$

It follows from (1.8) that

(3.7)
$\left(\frac{k\pi_{p}}{\bar{q}_{+}(R,R+1)}\right)^{p}\leq\lambda_{k}(R,R+1)\leq\left(\frac{k\pi_{p}}{\bar{q}_{-}(R,R+1)}\right)^{p}.$

By (3.5), (3.6) and by limits in (3.7) we obtain

$\lim_{R\to+\infty}\lambda_{k}(R,R+1)=(\pi_{p}k)^{p}=(\pi_{r+1}k)^{r+1}.$

## References

  * [1] de Araujo, A. L.A., Infinitely many solutions for the Dirichlet problem involving the p-Laplacian in annulus, Far East Journal of Applied Mathematics, Vol. 96 (2), 77–91 (2017).
  * [2] Dosly, O., Rehak, P., Half-linear differential equations. North-Holland Mathematics Studies, 202. Amsterdam: Elsevier Science B.V. (2005).
  * [3] Kusano, T., Naito, M., Sturm-Liouville eigenvalue problems from half-linear ordinary differential equations. Rocky Mountain J. Math. 31, 1039-1054 (2001).
  * [4] Lin, S.S., Asymptotic behavior of positive solutions to semilinear elliptic equations on expanding annuli, J. Differential Equations, 120, 255–288 (1995).
  * [5] Liu, X. and Yang, Z., Positive Radial Solutions of the p-Laplacian in an Annulus with a Superlinear Nonlinearity with Zeros, British Journal of Mathematics & Computer Science, 5(4), 429-438 (2015).
  * [6] del Pino, M. and Manasevich, R. Multiple solutions for the p-Laplacian under global nonresonance, Proc. Amer. Math. Soc., Vol. 112, n. 1, 131-138 (1991).
  * [7] Zhang, M., Nonuniform Nonresonance of Semilinear Differential Equations. Journal of Differential Equations 166, 33-50 (2000).


* [CMM00] Alan Carey, Jouko Mickelsson, and Michael K. Murray, _Bundle gerbes applied to quantum field theory_ , Reviews in Mathematical Physics 12 (2000), 65–90.
  * [CMW97] Alan Carey, Michael K. Murray, and Bai-Ling Wang, _Higher bundle gerbes and cohomology classes in gauge theories_ , J. Geom. Phys. 21 (1997), no. 2, 183–197, arXiv:hep-th/9511169.
  * [CT21] Alan Carey and Guo Chuan Thiang, _The Fermi gerbe of Weyl semimetals_ , Letters in Mathematical Physics 111 (2021), no. 3, 72.
  * [CW08] Alan L. Carey and Bai-Ling Wang, _Fusion of symmetric D-branes and Verlinde rings_ , Comm. Math. Phys. 277 (2008), no. 3, 577–625. MR 2365446
  * [Fre95] Daniel S. Freed, _Classical Chern–Simons theory. I_ , Adv. Math. 113 (1995), no. 2, 237–303, arXiv:hep-th/9206021.
  * [FSS12] Domenico Fiorenza, Urs Schreiber, and Jim Stasheff, _Čech cocycles for differential characteristic classes: an $\infty$-Lie theoretic construction_, Adv. Theor. Math. Phys. 16 (2012), no. 1, 149–250.
  * [FSS14] Domenico Fiorenza, Hisham Sati, and Urs Schreiber, _Multiple M5-branes, string 2-connections, and 7d nonabelian Chern-Simons theory_ , Adv. Theor. Math. Phys. 18 (2014), no. 2, 229–321. MR 3273315
  * [FSS15a] by same author, _A higher stacky perspective on Chern–Simons theory_ , pp. 153–211, Springer International Publishing, Cham, 2015.
  * [FSS15b] by same author, _Super-Lie n-algebra extensions, higher WZW models and super-p-branes with tensor multiplet fields_ , International Journal of Geometric Methods in Modern Physics 12 (2015), no. 02, 1550018.
  * [FSS19] by same author, _The rational higher structure of M-theory_ , Fortschritte der Physik 67 (2019), no. 8-9, 1910017.
  * [FSS20] by same author, _Twisted cohomotopy implies M-theory anomaly cancellation on 8-manifolds_ , Communications in Mathematical Physics 377 (2020), no. 3, 1961–2025.
  * [FSS21a] by same author, _Twisted cohomotopy implies level quantization of the full 6d Wess–Zumino term of the M5-brane_ , Communications in Mathematical Physics 384 (2021), no. 1, 403–432.
  * [FSS21b] Domenico Fiorenza, Hisham Sati, and Urs Schreiber, _Twisted cohomotopy implies twisted string structure on M5-branes_ , Journal of Mathematical Physics 62 (2021), no. 4, 042301.
  * [Gaw17] Krzysztof Gawędzki, _$2d$ Fu–Kane–Mele invariant as Wess–Zumino action of the sewing matrix_, Letters in Mathematical Physics 107 (2017), no. 4, 733–755.
  * [GHV72] Werner Greub, Stephen Halperin, and Ray Vanstone, _Connections, curvature, and cohomology. Vol. I: De Rham cohomology of manifolds and vector bundles_ , Pure and Applied Mathematics, Vol. 47, Academic Press, New York-London, 1972.
  * [GT21] Kiyonori Gomi and Guo Chuan Thiang, _‘Real’ gerbes and Dirac cones of topological insulators_ , Communications in Mathematical Physics 388 (2021), no. 3, 1507–1555.
  * [Hen08] André Henriques, _Integrating $L_{\infty}$-algebras_, Compositio Mathematica 144 (2008), no. 4, 1017–1045.
  * [Hit03] Nigel Hitchin, _What is … a gerbe?_ , Notices Amer. Math. Soc. 50 (2003), no. 2.
  * [HMSV13] Pedram Hekmati, Michael K. Murray, Danny Stevenson, and Raymond F. Vozzo, _The Faddeev–Mickelsson–Shatashvili anomaly and lifting bundle gerbes_ , Communications in Mathematical Physics 319 (2013), no. 2, 379–393.
  * [HMSV19] Pedram Hekmati, Michael K. Murray, Richard J. Szabo, and Raymond F. Vozzo, _Real bundle gerbes, orientifolds and twisted $KR$-homology_, Adv. Theor. Math. Phys. 23 (2019), no. 8, 2093–2159. MR 4101661
  * [Joh03] Stuart Johnson, _Constructions with bundle gerbes_ , Ph.D. thesis, Adelaide University, Department of Pure Mathematics, 2003, arXiv:math.DG/0312175.
  * [JSSW19] Branislav Jurčo, Christian Sämann, Urs Schreiber, and Martin Wolf, _Higher structures in M-theory_ , Fortschr. Phys. 67 (2019), no. 8-9, Special issue: Proceedings of the LMS/EPSRC Durham Symposium on Higher Structures in M-Theory, 1870076, 1, Held at Durham University, Durham, August 12–18, 2018, arXiv:arXiv:1903.02807.
  * [JY21] Niles Johnson and Donald Yau, _2-Dimensional Categories_ , Oxford University Press, 2021, arXiv:2002.06055.
  * [Kil87] T. P. Killingback, _World-sheet anomalies and loop geometry_ , Nuclear Phys. B 288 (1987), 578–588.
  * [KM21] Chris Kottke and Richard B. Melrose, _Bigerbes_ , Algebr. Geom. Topol. 21 (2021), no. 7, 3335–3399, arXiv:1905.03081.
  * [KS74] G. M. Kelly and Ross Street, _Review of the elements of $2$-categories_, Category Seminar (Proc. Sem., Sydney, 1972/1973), Lecture Notes in Math, vol. 420, 1974, pp. 75–103.
  * [KS20] Hyungrok Kim and Christian Saemann, _Adjusted parallel transport for higher gauge theories_ , J. Phys. A 53 (2020), no. 44, 445206, 52, arXiv:1911.06390.
  * [LM95] Tom Lada and Martin Markl, _Strongly homotopy Lie algebras_ , Comm. Algebra 23 (1995), no. 6, 2147–2161, arXiv:hep-th/9406095.
  * [McL92] Dennis A. McLaughlin, _Orientation and string structures on loop space_ , Pacific J. Math. 155 (1992), no. 1, 143–156.
  * [Mic06] Jouko Mickelsson, _Gerbes and quantum field theory_ , Encyclopedia of Mathematical Physics, vol. 2, Elsevier, 2006, pp. 539–546.
  * [Mic18] Ryan Mickler, _Localization for Chern–Simons on circle bundles via loop groups_ , J. Geom. Phys. 132 (2018), 257–281. MR 3836781
  * [MR06] V. Mathai and D. Roberts, _Yang–Mills theory for bundle gerbes_ , J. Phys. A 39 (2006), no. 20, 6039–6044.
  * [MRSV17] Michael K. Murray, David Michael Roberts, Danny Stevenson, and Raymond F. Vozzo, _Equivariant bundle gerbes_ , Adv. Theor. Math. Phys. 21 (2017), no. 4, 921–975, arXiv:1506.07931.
  * [MRW17] Michael Murray, David Michael Roberts, and Christoph Wockel, _Quasi-periodic paths and a string 2-group model from the free loop group_ , J. Lie Theory 27 (2017), no. 4, 1151–1177.
  * [MS00] Michael K. Murray and Daniel Stevenson, _Bundle gerbes: stable isomorphism and local theory_ , J. London Math. Soc. (2) 62 (2000), no. 3, 925–937, arXiv:math.DG/9908135.
  * [MS03] by same author, _Higgs fields, bundle gerbes and string structures_ , Comm. Math. Phys. 243 (2003), no. 3, 541–555, arXiv:math/0106179.
  * [Mur96] Michael K. Murray, _Bundle gerbes_ , J. London Math. Soc. 54 (1996), no. 2, 403–416, arXiv:dg-ga/9407015.
  * [Mur09] M. K. Murray, _An introduction to bundle gerbes_ , The many facets of geometry, a tribute to Nigel Hitchin (O. Garcia-Prada, J.-P. Bourguignon, and S. Salamon, eds.), Oxford University Press, 2009, [arXiv:math.DG/0712.1651].
  * [NS11] Thomas Nikolaus and Christoph Schweigert, _Equivariance in higher geometry_ , Adv. Math. 226 (2011), no. 4, 3367–3408, arXiv:1004.4558.
  * [NSW13] Thomas Nikolaus, Christoph Sachse, and Christoph Wockel, _A smooth model for the string group_ , Int. Math. Res. Not. IMRN (2013), no. 16, 3678–3721.
  * [NW13] Thomas Nikolaus and Konrad Waldorf, _Four equivalent versions of nonabelian gerbes_ , Pacific J. Math. 264 (2013), no. 2, 355–419.
  * [PS86] Andrew Pressley and Graeme Segal, _Loop groups_ , Oxford Mathematical Monographs, The Clarendon Press, Oxford University Press, New York, 1986, Oxford Science Publications.
  * [Red11] Corbett Redden, _String structures and canonical 3-forms_ , Pacific J. Math. 249 (2011), no. 2, 447–484, arXiv:0912.2086.
  * [Rob14] David Michael Roberts, _Explicit string bundles_ , Notes from a talk delivered at Heriot-Watt University on 26 June 2014 at the Workshop on Higher Gauge Theory and Higher Quantization, available as arXiv:2203.04544, 2014.
  * [RSW22] Dominik Rist, Christian Saemann, and Martin Wolf, _Explicit non-abelian gerbes with connections_ , arXiv:2203.00092, 2022.
  * [Sat10] Hisham Sati, _Geometric and topological structures related to M-branes_ , Superstrings, geometry, topology, and $C^{\ast}$-algebras, Proc. Sympos. Pure Math., vol. 81, Amer. Math. Soc., Providence, RI, 2010, pp. 181–236.
  * [Seg85] Graeme Segal, _Fadeev’s anomaly in Gauss’ law_ , preprint, 1985.
  * [SP11] Christopher J. Schommer-Pries, _Central extensions of smooth 2-groups and a finite-dimensional string 2-group_ , Geom. Topol. 15 (2011), no. 2, 609–676.
  * [SSS12] Hisham Sati, Urs Schreiber, and Jim Stasheff, _Twisted differential string and fivebrane structures_ , Communications in Mathematical Physics 315 (2012), no. 1, 169–213.
  * [ST04] Stephan Stolz and Peter Teichner, _What is an elliptic object?_ , Topology, geometry and quantum field theory, London Math. Soc. Lecture Note Ser., vol. 308, Cambridge Univ. Press, Cambridge, 2004, pp. 247–343.
  * [Ste00] Danny Stevenson, _The geometry of bundle gerbes_ , Ph.D. thesis, Adelaide University, Department of Pure Mathematics, 2000, arXiv:math.DG/0004117.
  * [Ste04] Daniel Stevenson, _Bundle 2-gerbes_ , Proc. Lond. Math. Soc. 88 (2004), no. 3, 405–435.
  * [SW13] Urs Schreiber and Konrad Waldorf, _Connections on non-abelian gerbes and their holonomy_ , Theory Appl. Categ. 28 (2013), 476–540, arXiv:0808.1923.
  * [SW18] Christoph Schweigert and Konrad Waldorf, _Non-abelian gerbes and some applications in string theory_ , Particles, strings and the early universe: The structure of matter and space-time. Research results of the Collaborative Research Centre SFB 676 in Hamburg. July 2006-June 2018 Universität Hamburg and DESY (Johannes Haller and Michael Grefe, eds.), Verlag Deutsches Elektronen-Synchrotron, Hamburg, 2018, pp. 39–55.
  * [SW21] Gabriel Sevestre and Tilmann Wurzbacher, _On the prequantisation map for 2-plectic manifolds_ , Mathematical Physics, Analysis and Geometry 24 (2021), no. 2, 20.
  * [vH21] Kevin S. van Helden, _Classification of 2-term $L_{\infty}$-algebras_, arXiv:2109.10202, 2021.
  * [Wal07] Konrad Waldorf, _More morphisms between bundle gerbes_ , Theory and Appl. Categ. 18 (2007), no. 9, 240–273, arXiv:math/0702652.
  * [Wal12] Konrad Waldorf, _A construction of string 2-group models using a transgression-regression technique_ , Analysis, geometry and quantum field theory, Contemp. Math., vol. 584, Amer. Math. Soc., Providence, RI, 2012, pp. 99–115.
  * [Wal13] by same author, _String connections and Chern–Simons theory_ , Trans. Amer. Math. Soc. 365 (2013), no. 8, 4393–4432, arXiv:0906.0117.
  * [Wal18] by same author, _A global perspective to connections on principal 2-bundles_ , Forum Math. 30 (2018), no. 4, 809–843, arXiv:1608.00401.
  * [Wit85] Edward Witten, _Global anomalies in string theory_ , Symposium on anomalies, geometry, topology (Chicago, Ill., 1985), World Sci. Publishing, Singapore, 1985, pp. 61–99.
  * [Wit88] by same author, _The index of the Dirac operator in loop space_ , Elliptic modular forms in algebraic topology (Princeton, NJ, 1986) (Berlin), Lecture Notes in Mathematics, no. 1326, Springer-Verlag, 1988, pp. 161–181.
  * [WZ85] McKenzie Y. Wang and Wolfgang Ziller, _On normal homogeneous Einstein manifolds_ , Annales scientifiques de l’École Normale Supérieure, Série 4 18 (1985), no. 4, 563–633.