On nonlocal mechanics of curved elastic beams

Preprint of the article published in
International Journal of Engineering Science
144, November 2019, 103140

Raffaele Barretta,
Francesco Marotti de Sciarra,
Marzia Sara Vaccaro

https://doi.org/10.1016/j.ijengsci.2019.103140

ⓒ 2019. This manuscript version is made available under the CC-BY-NC-ND 4.0
license http://creativecommons.org/licenses/by-nc-nd/4.0/

###### Abstract

Curved beams are basic structural components of Nano-Electro-Mechanical-
Sistems (NEMS) whose design requires appropriate modelling of scale effects.
In the present paper, the size-dependent static behaviour of curved elastic
nano-beams is investigated by stress-driven nonlocal continuum mechanics.
Axial strain and flexural curvature fields are integral convolutions between
equilibrated axial force and bending moment fields and an averaging kernel.
The nonlocal integral methodology formulated here is the generalization to
curved structures of the treatment in [Int. J. Eng. Science 115 (2017) 14-27]
confined to straight beams. The corresponding nonlocal differential problem,
supplemented with non-standard boundary conditions, is highlighted and shown
to lead to mathematically well-posed problems of nano-engineering. The
theoretical predictions, exhibiting stiffening nonlocal behaviours, are
therefore appropriate to significantly model a wide range of small-scale
devices of nanotechnological interest. The nonlocal approach is exploited by
analytically establishing size-dependent responses of curved elastic nano-
sensors and nano-actuators that are driven by the small-scale characteristic
parameter.

###### keywords:

Curved beams , Size effects , Integral elasticity , Stress-driven nonlocal
model , Nanotechnology , MEMS/NEMS

##  1 Introduction

Analysis, modelling, design, optimization and realization of smaller and
smaller structural devices is nowadays a topic of major interest in
Engineering Science, due to the recent advancements in Nanoscience and
Nanotechnology.

A large number of contributions have been published on size-dependent
structural behaviour of rods, wires, beams, membranes, plates and shells at
micro- and nano-scales to conceive modern small-scale systems, such as:
varactors (Sedighi et al., 2017), energy harvesters (Lekha et al., 2017),
piezoelectric devices for biomedicine applications (Salim et al., 2018),
resonators (Chorsi and Chorsi, 2018), resonant accelerometers (Ding et al.,
2019), biosensors for detection of malaria protozoan parasites (Kurmendra et
al., 2019) and of cancer (Kurmendra et al., 2019), arch actuators (Ouakad and
Sedighi, 2019). In this context, curved beams are the main structural elements
of Micro/Nano-Electro-Mechanical-Systems (M/NEMS) resonators (Dantas and
Gusso, 2018; Alfosail et al., 2019; Frangi and Gobat, 2019; Nikpourian et al.,
2019; Ouakad and Najar, 2019; Sieberer et al., 2019; Wang and Ren, 2019) that
should be designed by significantly taking small size effects into account.
Only a few papers have been published on the matter and, however, scale
phenomena have been modelled by strain-driven nonlocal or couple stress
methods, which, as discussed below, present some operational difficulties.

  1. 1.

Eringen differential nonlocal model. Such a strategy, recently employed in
(Ganapathi and Polit, 2017; Aya and Tufekci, 2017; Polit et al., 2018;
Rezaiee-Pajand and Rajabzadeh-Safaei, 2018; Arefi et al., 2019), is based on
the differential relation associated with the strain-driven fully nonlocal
integral convolution of the theory of elasticity, originally exploited by
Eringen (1983) to tackle small-scale problems, defined in unbounded domains,
of screw dislocation and wave propagation. Such a model has been shown to
provide, in the special case of straight structures, mechanical paradoxes
(Peddieson et al., 2003; Challamel and Wang, 2008; Li et al., 2015; Fernández-
Sáez et al., 2016) and size-dependent responses which are of limited
applicative interest. Inapplicability of Eringen’s strain-driven fully
nonlocal integral model of elasticity to nanostructures has been highlited by
Romano and Barretta (2016); Romano et al. (2017) and, nowadays, acknowledged
by the scientific community of Engineering Science. As a partial remedy,
Eringen’s strain-driven fully nonlocal integral model has been replaced with
the strain-driven local/nonlocal mixture (Eringen, 1972, 1987) to capture size
effects (Wang et al., 2016; Fernández-Sáez and Zaera, 2017; Zhu et al., 2017;
Khaniki, 2019; Zhang et al., 2019), even if it exhibits singular structural
behaviors for vanishing local fractions (Romano et al., 2017).

  2. 2.

Nonlocal strain gradient model. This strategy, based on the differential law
associated with the strain-driven nonlocal strain gradient integral model, was
originally conceived by Lim et al. (2015) for applications in wave
propagation. In curved structures, the non-standard constitutive boundary
conditions associated with the nonlocal gradient integral convolutions have
been ignored (She et al., 2019a, b; Karami et al., 2019b; Sobhy and Abazid,
2019; Ebrahimi and Barati, 2018, 2017) and non-pertinent higher-order boundary
conditions of strain gradient theory have been enforced. This issue has been
recently addressed by Barretta and Marotti de Sciarra (2018, 2019) for
straight nanobeams, and advantageously exploited by Apuzzo et al. (2018,
2019).

  3. 3.

Modified couple stress model. Such a strategy, proposed by Yang et al. (2002),
has been widely applied to small-scale structures (see e.g. Dehrouyeh-Semnani
et al., 2017a, b; Farokhi et al., 2017; Ghayesh and Farokhi, 2017a, b; Farokhi
and Ghayesh, 2018a, b, c; Ghayesh et al., 2018; Khorshidi, 2018; Taati, 2018;
Ghayesh, 2019; Karami et al., 2019a). Nevertheless, deformation measures
describing couple stress theories are kinematically redundant, as evidenced by
Romano et al. (2016) in the general framework of micromorphic continua.
Redundancy means that the assumed set of kinematical conditions ensuring
rigidity of motion can be substituted with a proper subset of conditions. To
see this, we observe that, by virtue Euler’s kinematic lemma (Romano et al.,
2016, Lemma 1), vanishing of the macro-strain (Ghayesh and Farajpour, 2019,
Eq.(27)) implies constancy and skew-symmetry of the displacement gradient.
This in turn implies that micro-spin (Ghayesh and Farajpour, 2019, Eq.(31)) is
constant and that the microcurvature (Ghayesh and Farajpour, 2019, Eq.(30)) is
vanishing. Adoption of the microcurvature tensor as an independent measure of
deformation is therefore redundant because its vanishing is already implied by
vanishing of the macro-strain. The concept of kinematic redundancy of
deformation measures, adopted in the context of generalized continua, was
first introduced by Romano et al. (2016) and then acknowledged by the
scientific community (see e.g. Barbagallo et al., 2017; Neff et al., 2017;
Sourki and Hosseini, 2017).

##  2 Motivation and outline

The stress-driven nonlocal integral elasticity approach of straight beams by
Romano and Barretta (2017), effectively exploited for static, vibration,
thermal and buckling problems of nanotechnological interest in (Oskouie et
al., 2018; Apuzzo et al., 2017; Barretta et al., 2018a, b, c, 2019c, d, 2019a,
2019b), is generalized in the present paper to curved nanobeams.

The plan is the following. Kinematics and equilibrium of Bernoulli-Euler
curved beams are preliminarily recalled in Sect.3. The stress-driven nonlocal
integral formulation of elasticity for curved structures is developed in
Sect.4. Assuming that the integral model is governed by the special bi-
exponential averaging kernel adopted by Eringen (1983), an equivalent
differential problem, with homogeneous constitutive boundary conditions, is
provided in Proposition 4.1. The stress-driven nonlocal differential problem
for straight beams (Romano and Barretta, 2017, Prop.7.1, p.22) is recovered as
a special case by vanishing the geometric curvature of the nano-structural
axis line. Equations governing the stress-driven nonlocal elastic equilibrium
of curved beams are formulated in Sect.5. Size-dependent elastic behavior of
statically determinate and indeterminate curved beams is established and
evidenced by illustrations in Sect.6. Results, discussion and closing remarks
are summarized in Sect.7.

##  3 Slender curved beams: kinematics and equilibrium

Let us consider a slender curved beam of length L whose axis is detected by a
curve $\,\boldsymbol{\gamma}:[0,\textrm{L}]\mapsto V\,$, parametrized by the
arc length $\,s\in[0,\textrm{L}]\,$, in a plane $\,\boldsymbol{\pi}\,$ endowed
with the linear space of translations $\,V\,$. The cross-section, assumed to
be uniform along $\,s\,$, is modeled by a two-dimensional domain $\,\Omega\,$.
A geometric sketch of curved beam is depicted in fig.1.

Figure 1: Slender curved beam: a geometric sketch.

The field of tangent unit vectors to the beam axis is defined by

$\textbf{t}:=\boldsymbol{\gamma}^{\prime}$ (1)

where the apex $\,(\bullet)^{\prime}\,$ is the derivative with respect to
$\,s\,$. Denoting by R the linear operator which performs the rotation by
$\,\pi/2\,$ counterclockwise in the plane $\,\boldsymbol{\pi}\,$, we introduce
the following field of transversal unit vectors

$\textbf{t}_{\bot}:=\textbf{R}\textbf{t}$ (2)

and the uniform unit vector field
$\,\textbf{k}:=\textbf{t}\times\textbf{t}_{\bot}\,$, with $\,\times\,$ cross
product. According to Bernoulli-Euler’s beam theory, cross-sections are
clamped to the beam axis $\,\boldsymbol{\gamma}\,$, and therefore the
kinematic model is uniquely described by the translation velocity field
$\,\mathbf{v}:[0,\textrm{L}]\mapsto V\,$ of the beam axis. Indeed, in this
framework, the angular velocity field $\,\omega:[0,\textrm{L}]\mapsto\Re\,$ of
cross-sections is provided by the differential relation

$\omega=\mathbf{v}^{\prime}\cdot\textbf{t}_{\bot}.$ (3)

The kinematically compatible deformation field $\,\mathbf{D}_{\mathbf{v}}\,$
associated with the velocity field $\,\mathbf{v}:[0,\textrm{L}]\mapsto V\,$ of
the beam axis is given by

$\mathbf{D}_{\mathbf{v}}=\begin{vmatrix}\varepsilon_{\mathbf{v}}\vspace{4pt}\\\
\chi_{\mathbf{v}}\end{vmatrix}=\begin{vmatrix}\mathbf{v}^{\prime}\cdot\textbf{t}\vspace{4pt}\\\
(\mathbf{v}^{\prime}\cdot\textbf{t}_{\bot})^{\prime}\end{vmatrix}$ (4)

in terms of axial strain $\varepsilon_{\mathbf{v}}:[0,L]\mapsto\Re$ and
flexural curvature $\chi_{\mathbf{v}}:[0,L]\mapsto\Re$ scalar fields. The
curved beam is subjected to an equilibrated force system $\,f\,$ composed of a
distributed vector loading $\,\mathbf{p}:[0,L]\mapsto V\,$ and of boundary
concentrated forces $\,{\mathbf{F}}_{0}\in V\,$ and $\,{\mathbf{F}}_{L}\in
V\,$ and bending couples $\,\mathcal{M}_{0}\in\Re\,$ and
$\,\mathcal{M}_{L}\in\Re\,$. The equilibrated stress $\,\boldsymbol{\sigma}\,$
in a Bernoulli-Euler curved beam is the couple of axial force
$\,N:[0,L]\mapsto\Re\,$ and bending moment $\,M:[0,L]\mapsto\Re\,$ fields

$\boldsymbol{\sigma}=\begin{vmatrix}N\vspace{4pt}\\\ M\end{vmatrix}$ (5)

fulfilling the virtual power condition

$\displaystyle{\langle}\kern 1.00006ptf,\delta\mathbf{v}\kern
1.00006pt{\rangle}=\int_{0}^{L}\boldsymbol{\sigma}(s)\cdot\mathbf{D}_{\delta\mathbf{v}}(s)\,ds\,,$
(6)

with

$\displaystyle{\langle}\kern 1.00006ptf,\delta\mathbf{v}\kern
1.00006pt{\rangle}:=$
$\displaystyle\int_{0}^{L}\mathbf{p}(s)\cdot\delta\mathbf{v}(s)\,ds+\mathcal{M}_{L}\;\bigl{(}\delta\mathbf{v}^{\prime}(L)\cdot\textbf{t}_{\bot}(L)\bigr{)}$
(7)
$\displaystyle+\mathcal{M}_{0}\;\bigl{(}\delta\mathbf{v}^{\prime}(0)\cdot\textbf{t}_{\bot}(0)\bigr{)}+{\mathbf{F}}_{L}\cdot\delta\mathbf{v}(L)+{\mathbf{F}}_{0}\cdot\delta\mathbf{v}(0)\,,$

virtual power of the force $\,f\,$, for any virtual velocity field
$\,\delta\mathbf{v}:[0,L]\mapsto V\,$ fulfilling homogeneous kinematic
boundary conditions. According to Eq.(4), the deformation field
$\,\mathbf{D}_{\delta\mathbf{v}}\,$ kinematically compatible with the virtual
velocity field $\,\delta\mathbf{v}\,$ is expressed by

$\mathbf{D}_{\delta\mathbf{v}}=\begin{vmatrix}\varepsilon_{\delta\mathbf{v}}\vspace{4pt}\\\
\chi_{\delta\mathbf{v}}\end{vmatrix}=\begin{vmatrix}{\delta\mathbf{v}}^{\prime}\cdot\textbf{t}\vspace{4pt}\\\
({\delta\mathbf{v}}^{\prime}\cdot\textbf{t}_{\bot})^{\prime}\end{vmatrix}\,.$
(8)

Integrating by parts Eq.(6), applying a standard localization procedure and
projecting along the axial t and transversal $\,\textbf{t}_{\bot}\,$
directions we get the equivalent equilibrium differential problem in
$\,[0,\textrm{L}]\,$

$\left\\{\begin{aligned}
N^{\prime}+T\,(\textbf{t}_{\bot}^{\prime}\cdot\textbf{t})&=-\mathbf{p}\cdot\textbf{t}\,,\\\
T^{\prime}+N\,(\textbf{t}^{\prime}\cdot\textbf{t}_{\bot})&=-\mathbf{p}\cdot\textbf{t}_{\bot}\,,\end{aligned}\right.$
(9)

equipped with the boundary conditions at the abscissae $\,s=0\,$ and $\,s=L\,$

$\left\\{\begin{aligned}
-N(0)\cdot(\delta\mathbf{v}\cdot\textbf{t})(0)&=({\mathbf{F}}_{0}\cdot\textbf{t}(0))\cdot(\delta\mathbf{v}\cdot\textbf{t})(0)\,,\\\
N(L)\cdot(\delta\mathbf{v}\cdot\textbf{t})(L)&=({\mathbf{F}}_{L}\cdot\textbf{t}(L))\cdot(\delta\mathbf{v}\cdot\textbf{t})(L)\,,\\\
-M(0)\;\delta\omega(0)&=\mathcal{M}_{0}\;\delta\omega(0)\,,\\\
M(L)\;\delta\omega(L)&=\mathcal{M}_{L}\;\delta\omega(L)\,,\\\
-T(0)\cdot(\delta\mathbf{v}\cdot\textbf{t}_{\bot})(0)&=({\mathbf{F}}_{0}\cdot\textbf{t}_{\bot}(0))\cdot(\delta\mathbf{v}\cdot\textbf{t}_{\bot})(0)\,,\\\
T(L)\cdot(\delta\mathbf{v}\cdot\textbf{t}_{\bot})(L)&=({\mathbf{F}}_{L}\cdot\textbf{t}_{\bot}(L))\cdot(\delta\mathbf{v}\cdot\textbf{t}_{\bot})(L)\,,\end{aligned}\right.$
(10)

with $\,T:=-M^{\prime}:[0,\textrm{L}]\mapsto\Re\,$ shear force interaction
field.

In many engineering problems the beam axis is generally regular, so that the
vector $\,\textbf{t}^{\prime}\,$ is not vanishing and leads to the definition
of normal unit vector $\,\mathbf{n}:=\textbf{t}^{\prime}/c\,$, with
$\,c:=\|\textbf{t}^{\prime}\|\,$ geometric curvature of the beam axis. The
relationship between $\,\textbf{t}_{\bot}\,$ and $\,\mathbf{n}\,$ is given by
Romano (2002):
$\,\textbf{t}_{\bot}=(\textbf{t}_{\bot}\cdot\mathbf{n})\,\mathbf{n}\,$.

##  4 Stress-driven nonlocal integral elasticity

Let us preliminarily recall the local elasticity linearized model of
Bernoulli-Euler curved beam. Local elastic axial strains $\,\varepsilon\,$ and
flexural curvatures $\,\chi\,$ are related to axial forces $\,N\,$ and bending
moments $\,M\,$ by Winkler (1858); Baldacci (1983)

$\left\\{\begin{aligned}
\varepsilon(s)&=\biggl{[}\frac{1}{EA}[N-\textit{c}\,M]\biggr{]}(s)\,,\\\
\chi(s)&=\biggl{[}\frac{M}{EJ_{r}}-\frac{\textit{c}}{EA}[N-\textit{c}\,M]\biggr{]}(s)\,,\end{aligned}\right.$
(11)

with $\,E\,$ Euler-Young modulus, $\,A\,$ cross-sectional area and $\,J_{r}\,$
is the reduced moment of inertia

$J_{r}:=\int_{\Omega}y^{2}\frac{1}{1-c\,y}\,dA\,,$ (12)

along the bending axis $\,y\,$ associated with unit normal vector
$\,\mathbf{n}\,$.

Nonlocal effects are described by requiring that elastic axial strains and
flexural curvatures are integral convolutions between an averaging kernel
$\,\phi_{l}:\Re\mapsto[0;+\infty[\,$, described by a nonlocal length parameter
$\,l\in]0;+\infty[\,$, and the local fields formulated in Eq.(11). The stress-
driven nonlocal integral model takes thus the form

$\left\\{\begin{aligned}
\varepsilon(s)&=\int_{0}^{L}{\phi_{l}(s-\xi)\>\biggl{[}\frac{1}{EA}[N-\textit{c}\,M]\biggr{]}(\xi)\>d\xi}\\\
\chi(s)&=\int_{0}^{L}{\phi_{l}(s-\xi)\>\biggl{[}\frac{M}{EJ_{r}}-\frac{\textit{c}}{EA}[N-\textit{c}\,M]\biggr{]}(\xi)\>d\xi}\end{aligned}\right.$
(13)

Equipping Eq.(13) with the special bi-exponential kernel in (Eringen, 1983)

$\phi_{l}(s)=\frac{1}{2l}\exp(-\frac{|s|}{l})\,,$ (14)

the stress-driven integral convolutions Eq.(13) can be inverted to give the
following constitutive equivalence result, extending thus to curved beams the
formulation in (Romano and Barretta, 2017, Prop.7.1, p.22) confined to
straight beams. The proof is analogous.

###### Proposition 4.1

The nonlocal integral formulation for elastic curved beams Eq.(13), equipped
with the bi-exponential kernel Eq.(14), is equivalent to the differential
problem

$\left\\{\begin{aligned}
&\varepsilon(s)-l^{2}\,(\partial_{s}^{2}\varepsilon)(s)=\biggl{[}\frac{1}{EA}[N-\textit{c}\,M]\biggr{]}(s)\,,\\\
&\chi(s)-l^{2}\,(\partial_{s}^{2}\chi)(s)=\biggl{[}\frac{M}{EJ_{r}}-\frac{\textit{c}}{EA}[N-\textit{c}\,M]\biggr{]}(s)\,,\end{aligned}\right.$
(15)

with the constitutive boundary conditions (CBC)

$\left\\{\begin{aligned}
&\partial_{s}\varepsilon(0)=\frac{1}{l}\,\varepsilon(0)\,,\\\
&\partial_{s}\varepsilon(L)=-\frac{1}{l}\,\varepsilon(L)\,,\\\
&\partial_{s}\chi(0)=\frac{1}{l}\,\chi(0)\,,\\\
&\partial_{s}\chi(L)=-\frac{1}{l}\,\chi(L)\,.\end{aligned}\right.$ (16)

##  5 Stress-driven nonlocal elastic equilibrium of curved beams

The elastostatic problem of a Bernoulli-Euler curved beam is formulated in
terms of displacement field $\,\mathbf{u}:[0,\textrm{L}]\mapsto V\,$ of the
structural axis.

In the realm of the geometrically linearized theory, the kinematic relation
Eq.(4) holds true, so that the deformation field $\,\mathbf{D}_{\mathbf{u}}\,$
of the curved nanobeam, associated with the displacement field
$\,\mathbf{u}:[0,\textrm{L}]\mapsto V\,$, writes as

$\mathbf{D}_{\mathbf{u}}=\begin{vmatrix}\varepsilon_{\mathbf{u}}\vspace{4pt}\\\
\chi_{\mathbf{u}}\end{vmatrix}=\begin{vmatrix}\mathbf{u}^{\prime}\cdot\textbf{t}\vspace{4pt}\\\
(\mathbf{u}^{\prime}\cdot\textbf{t}_{\bot})^{\prime}\end{vmatrix}$ (17)

in terms of linearized axial strain $\varepsilon_{\mathbf{u}}:[0,L]\mapsto\Re$
and linearized flexural curvature $\chi_{\mathbf{u}}:[0,L]\mapsto\Re$ scalar
fields.

Resorting to Eq.(3), the field of rotations
$\,\varphi:[0,\textrm{L}]\mapsto\Re\,$ of cross-sections is obtained by
integrating Eq.(17)${}_{2}\,$

$\varphi(s)=(\mathbf{u}^{\prime}\cdot\textbf{t}_{\bot})(s)=\varphi(0)+\int_{0}^{s}\chi_{\mathbf{u}}(\xi)\,d\xi\,.$
(18)

To analytically establish the expression of the vector displacement field, let
us preliminarily consider the additive decomposition

$\mathbf{u}^{\prime}=(\mathbf{u}^{\prime}\cdot\textbf{t})\,\textbf{t}+(\mathbf{u}^{\prime}\cdot\textbf{t}_{\bot})\,\textbf{t}_{\bot}=\varepsilon_{\mathbf{u}}\,\textbf{t}+\varphi\,\textbf{t}_{\bot}\,,$
(19)

where Eqs.(17) and (18) are enforced.

The sought displacement field of the curved nanobeam is obtained by
integrating Eq.(19)

$\mathbf{u}(s)=\mathbf{u}(0)+\int_{0}^{s}\mathbf{u}^{\prime}d\xi\,=\mathbf{u}(0)+\int_{0}^{s}[\varepsilon_{\mathbf{u}}(\xi)\,\textbf{t}(\xi)+\varphi(\xi)\,\textbf{t}_{\bot}(\xi)]\,d\xi\,.$
(20)

The unknown integration constants $\,\varphi(0)\in\Re\,$ and
$\,\mathbf{u}(0)\in V\,$ are univocally evaluated by prescribing the essential
kinematic boundary conditions at hand. In all the case-studies presented in
the next section, the total deformations
$\,\\{\,\varepsilon_{\mathbf{u}},\chi_{\mathbf{u}}\,\\}\,$ are assumed to be
coincident with the elastic deformation fields
$\,\\{\,\varepsilon,\chi\,\\}\,$ provided by the stress-driven nonlocal
integral convolutions Eq.(13).

##  6 Case-studies

The developed stress-driven methodology is illustrated by investigating the
nonlocal behaviour of four static schemes of nanotechnological interest. The
nanobeam axis is assumed to be a circle arc of radius $\,r=10\,$ [nm], so that
length and curvature are given by $\,L=\pi/(2\textit{c})\,$ and
$\,\textit{c}=1/r\,$, respectively. The cross-section $\,\Omega\,$ is a
rectangle of base $\,b=1\,$ [nm] and height $\,h=2\,$ [nm]. The reduced moment
of inertia is evaluated by Eq.(12)

$J_{r}=\int_{-b/2}^{+b/2}{\int_{-h/2}^{+h/2}{\frac{r\>y^{2}}{r-y}\>dy}\>dx}$
(21)

The nanobeam is assumed to be made of silicon carbide, with Euler-Young
modulus $\,E=427\,$ [GPa].

In the illustrations below, the following dimensionless nonlocal parameter

$\lambda=\frac{l}{L}\,,$ (22)

with $\,l\,$ characteristic length parameter and $\,L\,$ nanobeam length, is
adopted.

The solution procedure of the stress-driven nonlocal integral nanostructural
problem, formulated in Sect.5, consists of three steps:

  1. 1.

Step $\,1\,$. Solution of the differential static problem Eqs.(9) and (10).
The interaction fields: axial force $\,N\,$, shear force $\,T\,$ and bending
moment $\,M\,$ are expressed in terms of $\,n\,$ integration constants, with
$\,n\,$ standing for redundancy degree.

For statically determinate beams we have that $\,n=0\,$ and the stress fields
$\,N\,,T\,,M\,$ are univocally determined by equilibrium requirements.

  2. 2.

Step $\,2\,$. Evaluation of axial strain $\,\varepsilon\,$ and bending
curvature $\,\chi\,$ by making recourse to the stress-driven integral
convolutions Eq.(13) or by solving the differential problem Eq.(15) equipped
with the nonlocal boundary conditions Eq.(16), based on the constitutive
equivalency established in Prop.4.1.

  3. 3.

Step $\,3\,$. Prescription of $\,3+n\,$ essential kinematic boundary
conditions and detection of the curved nanobeam nonlocal displacement
$\,\mathbf{u}\,$ Eq.(20).

The size-dependent responses of the following four static schemes are studied.

  1. 1.

Case $\,1\,$. Cantilever curved nanobeam subjected to a uniformly distributed
vertical load $\,q=0.1\,$ [nN]/[nm] directed upwards.

The clamp is placed at the abscissa $\,\textit{s}=0\,$, see Fig.2.

The essential kinematic boundary conditions are

$\mathbf{u}(0)=\mathbf{o}\in V\,,\qquad\varphi(0)=0\in\Re\,.$ (23)

The natural static boundary conditions Eq.(10) take thus the form

$\begin{cases}N(L)&=0\in\Re\,,\\\ T(L)&=0\in\Re\,,\\\
M(L)&=0\in\Re\,.\end{cases}$ (24)

The structural redundancy degree $\,n\,$ is vanishing and therefore the static
problem, recalled at step $\,1\,$ above, provides the equilibrated axial force
$\,N\,$, shear force $\,T\,$ and bending moment $\,M\,$, whose graphic
representations are reported in Figs.3, 4, 5, respectively. A schematic sketch
of the bending axial vector $\,M\textbf{k}\,$ and of the rotated vector
$\,\textbf{R}M\textbf{k}\,$ is given in Fig.6. The nonlocal displacement field
$\,\mathbf{u}\,$ is got by applying the steps $\,1\,$ and $\,2\,$ of the
solution strategy above. Plots of the deformed beam axis is provided in Fig.7
for increasing nonlocal parameter $\,\lambda\,$. Both the deformed and
undeformed beam axis are curved lines obtained by parameterizations in terms
of the arch length s and plotted in a Cartesian plane $\,x\,$ [nm], $\,y\,$
[nm]. Note that the classical local solution of elasticity is obtained as the
nonlocal parameter tends to zero ($\lambda=0^{+}$).

  2. 2.

Case $\,2\,$. Curved nanobeam with slider and roller supports subjected to a
force $\,F=1\,$ [nN] directed upwards, concentrated at the abscissa
$\,\textit{s}=0\,$, shown in Fig.8. The essential kinematic boundary
conditions are

$\mathbf{u}(0)\cdot\textbf{t}(0)=0\in\Re\,,\qquad\varphi(0)=0\in\Re\,,\qquad\mathbf{u}(L)\cdot\textbf{t}(L)=0\in\Re\,.$
(25)

The natural static boundary conditions Eq.(10) take thus the form

$\begin{cases}T(0)&=-F\in\Re\,,\\\ T(L)&=0\in\Re\,,\\\
M(L)&=0\in\Re\,.\end{cases}$ (26)

The nanostructure is statically determinate, so that the redundancy degree
$\,n\,$ is vanishing. The solution method, applied to case $\,1\,$, provides
scalar interactions $\,N\,$, $\,T\,$, $\,M\,$ and vector nonlocal
displacements $\,\mathbf{u}\,$. Graphic outputs are respectively presented in
Figs.9, 10, 11 and 12.

  3. 3.

Case $\,3\,$. Curved nanobeam with slider and clamp supports subjected to a
force $\,F=10\,$ [nN] directed upwards, concentrated at $\,\textit{s}=0\,$, as
shown in Fig.13. The essential kinematic boundary conditions are

$\mathbf{u}(0)\cdot\textbf{t}(0)=0\in\Re\,,\quad\varphi(0)=0\in\Re\,,\quad\mathbf{u}(L)=0\in
V\,,\quad\varphi(L)=0\in\Re\,.$ (27)

The natural static boundary conditions Eq.(10) take thus the form

$T(0)=-F\in\Re\,.$ (28)

The nanostructure is statically indeterminate with redundancy degree
$\,n=2\,$, so that $\,3+2\,$ essential and $\,1\,$ natural boundary scalar
conditions have to be enforced. The closure of the nonlocal structural problem
is therefore guaranteed and the axial force $\,N\,$, shear force $\,T\,$,
bending moment $\,M\,$ solution fields, associated with the scale parameter
$\,\lambda=0.5\,$, are depicted in Figs.14, 15, 16. The nonlocal displacement
solution field $\,\mathbf{u}\,$ is plotted in Fig.17 for increasing scale
parameter $\,\lambda\,$.

  4. 4.

Case $\,4\,$. Doubly clamped curved nanobeam under a uniformly distributed
vertical load $\,q=5\,$ [nN]/[nm] directed upwards, see Fig.18. The
nanostructure is statically indeterminate with redundancy degree $\,n=3\,$, so
that the boundary conditions to be imposed are all of kinematic type
($\,2+1+2+1=6\,$ scalar prescriptions)

$\mathbf{u}(0)=0\in V\,,\quad\varphi(0)=0\in\Re\,,\quad\mathbf{u}(L)=0\in
V\,,\quad\varphi(L)=0\in\Re\,.$ (29)

No natural static boundary conditions have to be inforced. The elastostatic
problem is solved to detect the static and kinematic nonlocal fields
$\,\\{\,N,T,M\,\\},\mathbf{u}\,$ in terms of the scale parameter
$\,\lambda\,$. The static outcomes $\,\\{\,N,T,M\,\\}\,$, corresponding to
$\,\lambda=0.5\,$, are illustrated in Fig.19, 20, 21, while the nonlocal
displacement vector field $\,\mathbf{u}\,$ is depicted in Fig.22 for
increasing scale parameter $\,\lambda\,$.

##  7 Results, discussion and closing remarks

The stress-driven nonlocal integral mechanics of straight elastic beams
conceived in (Romano and Barretta, 2017) has been generalized in the present
research to curved nanobeams, which are the principal structural elements of
new-generation Micro- and Nano-Electro-Mechanical Systems (M/NEMS). The
presented integral methodology has been shown to be equivalent to a convenient
set of ordinary differential equations, equipped with non-standard nonlocal
boundary conditions, generalizing thus to curved nanobeams the constitutive
equivalency proposition 7.1 in (Romano and Barretta, 2017). The complex
nonlocal elastic equilibrium equations governing the structural problems of
engineering interest have been therefore analytically simplified by
transforming integral convolutions into equivalent differential conditions.

Unlike strain-driven nonlocal integral and kinematically redundant couple
stress models, the stress-driven nonlocal integral formulation of curved
elastic nanostructures has been shown to lead to mathematically well-posed
problems. An effective solution procedure has been illustrated and implemented
by Wolfram’s software Mathematica for a variety of nano-engineered structures
exploited as sensors and actuators in M/NEMS applications.

In agreement with the experimental results collected in the paper by Abazari
et al. (2015) exhibited by a wide range of advanced materials and small-scale
devices, a stiffening nano-mechanical behaviour has been predicted by the
stress-driven approach in all case-studies for increasing nonlocal parameter.
The smaller-is-stiffer phenomenon, also predicted in the recent research by
Fuschi et al. (2019), seems to be a peculiar property in Nano-Engineering.

Acknowledgements

The financial support of the Italian Ministry for University and Research
(P.R.I.N. National Grant 2017, Project code 2017J4EAYB; University of Naples
Federico II Research Unit) is gratefully acknowledged.

## References

  * Abazari et al. (2015) Abazari, A.M., Safavi, S.M., Rezazadeh, G., Villanueva, L.G. (2015). Modelling the Size Effects on the Mechanical Properties of Micro/Nano Structures. Sensors, 15, 28543-28562.
  * Alfosail et al. (2019) Alfosail F.K., Hajjaj A.Z., Younis M.I. (2019). Theoretical and Experimental Investigation of Two-to-One Internal Resonance in MEMS Arch Resonators. Journal of Computational and Nonlinear Dynamics,Transactions of the ASME, 14, 011001.
  * Apuzzo et al. (2017) Apuzzo, A., Barretta, R., Luciano, R., Marotti de Sciarra, F., Penna, R. (2017). Free vibrations of Bernoulli-Euler nano-beams by the stress-driven nonlocal integral model. Composites Part B, 123, 105-111.
  * Apuzzo et al. (2018) Apuzzo, A., Barretta, R., Faghidian, S.A., Luciano, R., Marotti de Sciarra, F. (2018). Free vibrations of elastic beams by modified nonlocal strain gradient theory. International Journal of Engineering Science, 133, 99-108.
  * Apuzzo et al. (2019) Apuzzo, A., Barretta, R., Faghidian, S.A., Luciano, R., Marotti de Sciarra, F. (2019). Nonlocal strain gradient exact solutions for functionally graded inflected nano-beams. Composites Part B, 164, 667-674.
  * Arefi et al. (2019) Arefi, M., Bidgoli, E.M.-R., Dimitri, R., Bacciocchi, M., Tornabene, F. (2019). Nonlocal bending analysis of curved nanobeams reinforced by graphene nanoplatelets. Composites Part B 166, 1-12.
  * Aya and Tufekci (2017) Aya, S.A., Tufekci E. (2017). Modeling and analysis of out-of-plane behavior of curved nanobeams based on nonlocal elasticity. Composites Part B 119, 184-195.
  * Baldacci (1983) Baldacci, R. (1983). Scienza delle Costruzioni, Volume II. Torino: UTET.
  * Barbagallo et al. (2017) Barbagallo, G., Madeo, A., D’Agostino, M.V., Abreu, R., Ghiba, I.-D., Neff, P. (2017). Transparent anisotropy for the relaxed micromorphic model: Macroscopic consistency conditions and long wave length asymptotics. International Journal of Solids and Structures, 120, 7-30.
  * Barretta et al. (2018a) Barretta, R., Diaco, M., Feo, L., Luciano, R., Marotti de Sciarra, F., Penna, R. (2018a). Stress-driven integral elastic theory for torsion of nano-beams. Mechanics Research Communications, 87, 35-41.
  * Barretta et al. (2018b) Barretta, R., Canadija, M., Feo, L., Luciano, R., Marotti de Sciarra, F., Penna, R. (2018b). Exact solutions of inflected functionally graded nano-beams in integral elasticity. Composites Part B, 142, 273-286.
  * Barretta et al. (2018c) Barretta, R., Canadija, M., Luciano, R., Marotti de Sciarra, F. (2018c). Stress-driven modeling of nonlocal thermoelastic behavior of nanobeams. International Journal of Engineering Science, 126, 53-67.
  * Barretta et al. (2018d) Barretta, R., Luciano, R., Marotti de Sciarra, F., Ruta, G. (2018d). Stress-driven nonlocal integral model for Timoshenko elastic nano-beams. European Journal of Mechanics - A/Solids, 72, 275-286.
  * Barretta et al. (2019a) Barretta, R., Faghidian, S.A., Marotti de Sciarra, F., (2019a). Stress-driven nonlocal integral elasticity for axisymmetric nano-plates. International Journal of Engineering Science, 136, 38-52.
  * Barretta et al. (2019b) Barretta, R., Fabbrocino, F., Luciano, R., Marotti de Sciarra, F., Ruta, G. (2019b). Buckling loads of nano-beams in stress-driven nonlocal elasticity. In Press, Doi: https://doi.org/10.1080/15376494.2018.1501523
  * Barretta et al. (2019c) Barretta, R., Faghidian, S. A., Luciano R. (2019c). Longitudinal vibrations of nano-rods by stress-driven integral elasticity. Mechanics of Advanced Materials and Structures, 26(15), 1307-1315.
  * Barretta and Marotti de Sciarra (2018) Barretta, R., Marotti de Sciarra, F. (2018). Constitutive boundary conditions for nonlocal strain gradient elastic nano-beams. International Journal of Engineering Science, 130, 187-198.
  * Barretta and Marotti de Sciarra (2019) Barretta, R., Marotti de Sciarra, F. (2019). Variational nonlocal gradient elasticity for nano-beams. International Journal of Engineering Science, 143, 73-91.
  * Challamel and Wang (2008) Challamel, N., Wang, C.M. (2008). The small length scale effect for a non-local cantilever beam: a paradox solved. Nanotechnology, 19, 345703.
  * Chorsi and Chorsi (2018) Chorsi, M.T., Chorsi, H.T. (2018). Modeling and analysis of MEMS disk resonators. Microsystem Technologies, 24(6), 2517-2528.
  * Dantas and Gusso (2018) Dantas, W.G., Gusso, A. (2018). Analysis of the Chaotic Dynamics of MEMS/NEMS Doubly Clamped Beam Resonators with Two-Sided Electrodes. International Journal of Bifurcation and Chaos, 28(10), 1850122.
  * Dehrouyeh-Semnani et al. (2017a) Dehrouyeh-Semnani, A.M., Nikkhah-Bahrami, M., Yazdi, M.R.H. (2017a). On nonlinear vibrations of micropipes conveying fluid. International Journal of Engineering Science, 117, 20-33.
  * Dehrouyeh-Semnani et al. (2017b) Dehrouyeh-Semnani, A.M., Nikkhah-Bahrami, M., Yazdi, M.R.H. (2017b). On nonlinear stability of fluid-conveying imperfect micropipes. International Journal of Engineering Science, 120, 254-271.
  * Ding et al. (2019) Ding, H., Ma, Y., Guan, Y., Ju, B.-F., Xie, J. (2019). Duplex mode tilt measurements based on a MEMS biaxial resonant accelerometer. Sensors and Actuators, A: Physical, 296, 222-234.
  * Ebrahimi and Barati (2017) Ebrahimi, F., Barati, M.R. (2017). A nonlocal strain gradient refined beam model for buckling analysis of size-dependent shear-deformable curved FG nanobeams. Composite Structures, 159, 174-182.
  * Ebrahimi and Barati (2018) Ebrahimi, F., Barati, M.R. (2018). Vibration analysis of piezoelectrically actuated curved nanosize FG beams via a nonlocal strain-electric field gradient theory. Mechanics of Advanced Materials and Structures, 25(4), 350-359
  * Eringen (1972) Eringen, A.C. (1972). Linear theory of nonlocal elasticity and dispersion of plane waves. International Journal of Engineering Science, 10(5), 425-435.
  * Eringen (1983) Eringen, A.C. (1983). On differential equations of nonlocal elasticity and solutions of screw dislocation and surface waves. Journal of Applied Physics, 54, 4703.
  * Eringen (1987) Eringen, A.C. (1987). Theory of nonlocal elasticity and some applications. Res Mechanica, 21, 313-342.
  * Farokhi et al. (2017) Farokhi, H., Ghayesh, M.H., Gholipour, A., Hussain, S. (2017). Motion characteristics of bilayered extensible Timoshenko microbeams. International Journal of Engineering Science, 112, 1-17.
  * Farokhi and Ghayesh (2018a) Farokhi, H., Ghayesh, M.H. (2018a). Nonlinear mechanics of electrically actuated microplates. International Journal of Engineering Science, 123, 197-213.
  * Farokhi and Ghayesh (2018b) Farokhi, H., Ghayesh, M.H. (2018b). Nonlinear mechanical behaviour of microshells. International Journal of Engineering Science, 127, 127-144.
  * Farokhi and Ghayesh (2018c) Farokhi H., Ghayesh M.H., (2018c). Viscoelasticity effects on resonant response of a shear deformable extensible microbeam. Nonlinear Dynamics (2017) 87, 391-406.
  * Fernández-Sáez et al. (2016) Fernández-Sáez, J., Zaera, R., Loya, J.A., Reddy, J.N. (2016) Bending of Euler-Bernoulli beams using Eringen’s integral formulation: A paradox resolved. International Journal of Engineering Science, 99, 107-116.
  * Fernández-Sáez and Zaera (2017) Fernández-Sáez, J., Zaera, R. (2017). Vibrations of Bernoulli-Euler beams using the two-phase nonlocal elasticity theory. International Journal of Engineering Science, 119, 232-248.
  * Frangi and Gobat (2019) Frangi, A., Gobat, G. (2019). Reduced order modelling of the non-linear stiffness in MEMS resonators. International Journal of Non-Linear Mechanic,s 116, 211-218.
  * Fuschi et al. (2019) Fuschi P., Pisano, A.A., Polizzotto C. (2019). Size effects of small-scale beams in bending addressed with a strain-difference based nonlocal elasticity theory. International Journal of Mechanical Sciences, 151, 661-671.
  * Ganapathi and Polit (2017) Ganapathi, M., Polit, O. (2017). Dynamic characteristics of curved nanobeams using nonlocal higher-order curved beam theory. Physica E, 91, 190-202.
  * Ghayesh and Farokhi (2017a) Ghayesh, M.H., Farokhi, H., (2017a). Nonlinear mechanics of doubly curved shallow microshells. International Journal of Engineering Science, 119, 288-304.
  * Ghayesh and Farokhi (2017b) Ghayesh, M.H., Farokhi, H. (2017b). Parametric vibrations of imperfect Timoshenko microbeams. Microsystem Technologies, 23, 4917-4929.
  * Ghayesh et al. (2018) Ghayesh, M.H., Farokhi, H., Gholipour, A., Tavallaeinejad, M. (2018). Nonlinear oscillations of functionally graded microplates. International Journal of Engineering Science, 122, 56-72.
  * Ghayesh (2019) Ghayesh, M.H., (2019). Viscoelastic dynamics of axially FG microbeams. International Journal of Engineering Science, 135, 75-85.
  * Ghayesh and Farajpour (2019) Ghayesh, M.H., Farajpour, A. (2019). A review on the mechanics of functionally graded nanoscale and microscale structures. International Journal of Engineering Science, 137, 8-36.
  * Karami et al. (2019a) Karami, B., Shahsavari, D., Janghorban, M., Li, L. (2019a). Influence of homogenization schemes on vibration of functionally graded curved microbeams. Composite Structures, 216, 67-79.
  * Karami et al. (2019b) Karami, B., Shahsavari, D., Janghorban, M. (2019b). On the dynamics of porous doubly-curved nanoshells. International Journal of Engineering Science, 143, 39-55.
  * Khaniki (2019) Khaniki, H.B. (2019). On vibrations of FG nanobeams. International Journal of Engineering Science, 135, 23-36.
  * Khorshidi (2018) Khorshidi, M.A. (2018). The material length scale parameter used in couple stress theories is not a material constant. International Journal of Engineering Science, 133, 15-25.
  * Kurmendra and Kumar (2019) Kurmendra, Kumar, R. (2019). MEMS based cantilever biosensors for cancer detection using potential bio-markers present in VOCs: a survey. Microsystem Technologies, 25(9), 3253-3267.
  * Kurmendra et al. (2019) Kurmendra, Rahul, J., Kumar, R. (2019). Micro-cantilevered MEMS Biosensor for Detection of Malaria Protozoan Parasites. Journal of Computational Applied Mechanics, 50(1), 99-107.
  * Lekha et al. (2017) Lekha, C.S.C., Kumar, A.S., Vivek, S., Rasi, U.P.M., Saravanan, K.V., Nandakumar, K., Nair, S.S. (2017). High voltage generation from lead-free magnetoelectric coaxial nanotube arrays and their applications in nano energy harvesters. Nanotechnology, 28(5), 055402.
  * Lim et al. (2015) Lim, C.W., Zhang, G., Reddy J.N. 2015. A higher-order nonlocal elasticity and strain gradient theory and its applications in wave propagation. Journal of the Mechanics and Physics of Solids, 78, 298-313.
  * Li et al. (2015) Li, C., Yao, L., Chen W., Li, S. (2015). Comments on nonlocal effects in nano-cantilever beams. International Journal of Engineering Science, 87, 47-57.
  * Neff et al. (2017) Neff, P., Madeo, A., Barbagallo, G., D’Agostino, M.V., Abreu, R., Ghiba, I.-D. (2017). Real wave propagation in the isotropic-relaxed micromorphic model. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 473 (2197), 20160790.
  * Nikpourian et al. (2019) Nikpourian, A., Ghazavi, M.R., Azizi, S. (2019). Size-dependent secondary resonance of a piezoelectrically laminated bistable MEMS arch resonator. Composites Part B, 173,106850
  * Oskouie et al. (2018) Oskouie, M. F., Ansari, R., Rouhi, H. (2018). Bending of Euler-Bernoulli nanobeams based on the strain-driven and stress-driven nonlocal integral models: a numerical approach. Acta Mechanica Sinica. Doi: 10.1007/s10409-018-0757-0
  * Ouakad and Najar (2019) Ouakad, H.M., Najar, F. (2019). Nonlinear dynamics of MEMS arches assuming out-of-plane actuation arrangement. Journal of Vibration and Acoustics, Transactions of the ASME, 141(4), 041010.
  * Ouakad and Sedighi (2019) Ouakad, H.M., Sedighi, H.M. (2019). Static response and free vibration of MEMS arches assuming out-of-plane actuation pattern. International Journal of Non-Linear Mechanics, 110, 44-57.
  * Peddieson et al. (2003) Peddieson J, Buchanan GR, McNitt RP. (2003). Application of nonlocal continuum models to nanotechnology. International Journal of Engineering Science, 41(3-5), 305-312.
  * Polit et al. (2018) Polit, O., Merzouki, T., Ganapathi, M. (2018). Elastic stability of curved nanobeam based on higher-order shear deformation theory and nonlocal analysis by finite element approach. Finite Elements in Analysis and Design, 146, 1-15.
  * Rezaiee-Pajand and Rajabzadeh-Safaei (2018) Rezaiee-Pajand, M., Rajabzadeh-Safaei, N. (2018). Nonlocal static analysis of a functionally graded material curved nanobeam. Mechanics of Advanced Materials and Structures, 25(7), 539-547.
  * Romano (2002) Romano, G. (2002). Scienza delle Costruzioni, Tomo I. Benevento: Hevelius.
  * Romano and Barretta (2016) Romano, G., Barretta, R. (2016). Comment on the paper “Exact solution of Eringen’s nonlocal integral model for bending of Euler-Bernoulli and Timoshenko beams” by Meral Tuna & Mesut Kirca. International Journal of Engineering Science, 109, 240-242.
  * Romano et al. (2016) Romano, G., Barretta, R., Diaco, M. (2016). Micromorphic continua: non-redundant formulations. Continuum Mechanics and Thermodynamics, 28, 1659-1670.
  * Romano and Barretta (2017) Romano, G., Barretta, R. (2017). Nonlocal elasticity in nanobeams: the stress-driven integral model. International Journal of Engineering Science, 115, 14-27.
  * Romano et al. (2017) Romano, G., Barretta, R., Diaco, M. (2017). On nonlocal integral models for elastic nano-beams. International Journal of Mechanical Sciences, 131-132, 490-499.
  * Romano et al. (2017) Romano, G., Barretta, R., Diaco, M., Marotti de Sciarra, F. (2017). Constitutive boundary conditions and paradoxes in nonlocal elastic nano-beams. International Journal of Mechanical Sciences, 121, 151-156.
  * Salim et al. (2018) Salim, M., Salim, D., Chandran, D., Aljibori, H.S., Kherbeet, A.S. (2018). Review of nano piezoelectric devices in biomedicine applications. Journal of Intelligent Material Systems and Structures, 29(10), 2105-2121.
  * Sedighi et al. (2017) Sedighi, H.M., Koochi, A., Keivani, M., Abadyan, M. (2017). Measurement: Journal of the International Measurement Confederation, 111, 114-121.
  * She et al. (2019a) She, G.-L., Yuan, F.-G., Karami, B., Ren, Y.-R., Xiao, W.-S. (2019a). On nonlinear bending behavior of FG porous curved nanotubes. International Journal of Engineering Science, 135, 58-74.
  * She et al. (2019b) She, G.-L., Ren, Y.-R., Yan, K.-M. (2019b). On snap-buckling of porous FG curved nanobeams. Acta Astronautica, 161, 475-484.
  * Sieberer et al. (2019) Sieberer, S., McWilliam, S., Popov, A.A. (2019). Nonlinear electrostatic effects in MEMS ring-based rate sensors under shock excitation. International Journal of Mechanical Sciences, 157-158, 485-497.
  * Sobhy and Abazid (2019) Sobhy, M., Abazid, M.A. (2019). Dynamic and instability analyses of FG graphene-reinforced sandwich deep curved nanobeams with viscoelastic core under magnetic field effect. Composites Part B, 174, 106966.
  * Sourki and Hosseini (2017) Sourki, R., Hosseini, S.A. (2017). Coupling effects of nonlocal and modified couple stress theories incorporating surface energy on analytical transverse vibration of a weakened nanobeam. European Physical Journal Plus, 132(4), 184.
  * Taati (2018) Taati, E. (2018). On buckling and post-buckling behavior of functionally graded micro-beams in thermal environment. International Journal of Engineering Science, 128, 63-78.
  * Wang et al. (2016) Wang, Y., Zhu, X., Dai, H. (2016). Exact solutions for the static bending of Euler-Bernoulli beams using Eringen two-phase local/nonlocal model. AIP Advances, 6(8), 085114. Doi:10.1063/1.4961695
  * Yang et al. (2002) Yang, F, Chong, A, Lam, D.C., Tong, P. (2002). Couple stress based strain gradient theory for elasticity. International Journal of Solids and Structures, 39, 2731-2743.
  * Wang and Ren (2019) Wang, Z., Ren, J. (2019). Three-to-one internal resonance in MEMS arch resonators. Sensors, 19(8),1888.
  * Winkler (1858) Winkler E., (1858). Formänderung und Festigkeit gekrümmter Körper, insbesondere der Ringe Civilingenieur 4, 232-246.
  * Zhang et al. (2019) Zhang, P., Qing H., Gao, C. (2019). Theoretical analysis for static bending of circular Euler-Bernoulli beam using local and Eringen’s nonlocal integral mixed model. ZAMM. In Press. https://doi.org/10.1002/zamm.201800329
  * Zhu et al. (2017) Zhu, X.W., Wang, Y.B., Dai, H.H. (2017). Buckling analysis of Euler-Bernoulli beams using Eringen’s two-phase nonlocal model. International Journal of Engineering Science, 116, 130-140.

Figure 2: Cantilever curved nanobeam. Figure 3: Cantilever curved nanobeam:
plot of the vector field $N\textbf{t}_{\bot}$. Figure 4: Cantilever curved
nanobeam: plot of the vector field $T\textbf{t}_{\bot}$. Figure 5: Cantilever
curved nanobeam: plot of the vector field
$\textbf{R}($M$\textbf{k})=-M\textbf{t}_{\bot}$. Figure 6: Cross section at
abscissa s with the local triad and the vectors $\,M\textbf{k}\,$ and
$\,\textbf{R}M\textbf{k}\,$. Figure 7: Cantilever curved nanobeam: deflection
of the beam axis for $\lambda\in\\{0;0.1;0.5;1\\}$. Figure 8: Curved nanobeam
with slider and roller supports. Figure 9: Curved nanobeam with slider and
roller supports: plot of the vector field $N\textbf{t}_{\bot}$. Figure 10:
Curved nanobeam with slider and roller supports: plot of the vector field
$T\textbf{t}_{\bot}$. Figure 11: Curved nanobeam with slider and roller
supports: plot of the vector field
$\textbf{R}($M$\textbf{k})=-M\textbf{t}_{\bot}$.

*[scale=0.52]def_2

Figure 12: Curved nanobeam with slider and roller supports: deflection of the
beam axis for $\,\lambda\in\\{0;0.1;0.5;1\\}\,$. Figure 13: Curved nanobeam
with slider and clamp supports. Figure 14: Curved nanobeam with slider and
clamp supports: plot of the vector field $N\textbf{t}_{\bot}$. Figure 15:
Curved nanobeam with slider and clamp supports: plot of the vector field
$T\textbf{t}_{\bot}$. Figure 16: Curved nanobeam with slider and clamp
supports: plot of the vector field
$\textbf{R}($M$\textbf{k})=-M\textbf{t}_{\bot}$. Figure 17: Curved nanobeam
with slider and clamp supports: deflection of the beam axis for
$\lambda\in\\{0;0.1;0.5;1\\}$.

Figure 18: Doubly clamped curved nanobeam Figure 19: Doubly clamped curved
nanobeam: plot of the vector field $N\textbf{t}_{\bot}$. Figure 20: Doubly
clamped curved nanobeam: plot of the vector field $T\textbf{t}_{\bot}$. Figure
21: Doubly clamped curved nanobeam: plot of the vector field
$\,\textbf{R}($M$\textbf{k})=-M\textbf{t}_{\bot}$. Figure 22: Doubly clamped
curved nanobeam: deflection of the beam axis for
$\lambda\in\\{0;0.1;0.5;1\\}$.


# Benchmarking Test-Time Unsupervised Deep Neural Network Adaptation on Edge
Devices

Kshitij Bhardwaj, James Diffenderfer, Bhavya Kailkhura, and Maya Gokhale
Lawrence Livermore National Laboratory, Livermore, CA

###### Abstract

The prediction accuracy of the deep neural networks (DNNs) after deployment at
the edge can suffer with time due to shifts in the distribution of the new
data. To improve robustness of DNNs, they must be able to update themselves to
enhance their prediction accuracy. This adaptation at the resource-constrained
edge is challenging as: (i) new labeled data may not be present; (ii)
adaptation needs to be on device as connections to cloud may not be available;
and (iii) the process must not only be fast but also memory- and energy-
efficient. Recently, lightweight prediction-time unsupervised DNN adaptation
techniques have been introduced that improve prediction accuracy of the models
for noisy data by re-tuning the batch normalization (BN) parameters. This
paper, for the first time, performs a comprehensive measurement study of such
techniques to quantify their performance and energy on various edge devices as
well as find bottlenecks and propose optimization opportunities. In
particular, this study considers CIFAR-10-C image classification dataset with
corruptions, three robust DNNs (ResNeXt, Wide-ResNet, ResNet-18), two BN
adaptation algorithms (one that updates normalization statistics and the other
that also optimizes transformation parameters), and three edge devices (FPGA,
Raspberry-Pi, and Nvidia Xavier NX). We find that the approach that only
updates the normalization parameters with Wide-ResNet, running on Xavier GPU,
to be overall effective in terms of balancing multiple cost metrics. However,
the adaptation overhead can still be significant (around 213 ms). The results
strongly motivate the need for algorithm-hardware co-design for efficient on-
device DNN adaptation.††footnotetext: This paper was selected for poster
presentation in International Symposium on Performance Analysis of Systems and
Software (ISPASS), 2022.

##  I Introduction

While deep neural networks (DNNs) are trained and validated extensively using
a large dataset before deployment on edge, these networks are still prone to
degradation in prediction accuracy in the post-deployment real world
operation. The new input data samples that the trained DNNs encounter may have
different distributions than their training dataset (called dataset shifts)
[1]. This change in distributions could be caused due to sensor or
environmental noise [2]. To improve robustness of DNNs, they are trained using
data augmentation [3] and adversarial training [4] techniques. However, these
offline robust training methods may not be enough to completely handle the
real-time noise as they cannot cover the excessively wide range of potential
data shifts that can occur post deployment. Therefore, the neural networks
need to be adapted to improve their prediction accuracy [5, 6, 7].

Various transfer learning techniques exist that can be used for neural network
adaptation on edge devices, most of which however, require use of new labeled
data that is seldom available at test time [8, 9, 10]. This paper targets
challenging, albeit real-world scenarios where labels for the new, potentially
domain-shifted, test data is not present: such as devices operating in remote
places without human intervention [11, 12] or when the cost of annotating the
new data with labels is too high and not feasible [13, 2]. Some key examples
of these scenarios are: (i) DNNs performing human action recognition on drones
without labeled samples [11]; (ii) techniques such as laser-induced breakdown
spectroscopy in extreme environments (e.g., other planets) [12]; and (iii)
medical imaging where noise could be added due to scanners and the DNN for
analysis needs to rapidly adapt without labeled data [13].

In order to continually maintain or improve prediction accuracy of deployed
DNNs and meet tight performance constraints of streaming applications, the
DNNs must be adapted on device based on the new shifted test data. Due to hard
deadlines, adaptation through cloud services may not be always feasible.
Moreover, certain devices can be operating in areas with limited or no
connectivity (e.g., military zones, other planets). This prediction-time (or
test-time) adaptation at the edge is challenging as: (i) new labeled data may
not available; (ii) it should be fast as the networks are typically operating
on streaming data with strict timing deadlines, and (iii) it should be
lightweight and energy-efficient as the edge devices are generally resource-
constrained and could be battery operated.

Recently, test-time unsupervised DNN adaptation techniques have been
introduced that only update the batch-norm (BN) parameters of pre-trained DNNs
based on newly seen data. BN layers are typically added to DNNs for faster and
more robust training and are common in the modern DNNs. One such algorithm,
which we call BN-Norm, recomputes the normalization statistics during test
time [14, 6]. The other approach, which we call BN-Opt, not only re-estimates
these statistics but also optimizes transformation BN parameters using a
single backpropagation pass during prediction [5]. Both of these algorithms
have been shown to be effective in improving robustness for noisy data. The
simplicity of these approaches make them ideal candidates for real-time
adaptation at the edge. However, as these techniques are designed in isolation
without considering/evaluating the hardware cost, it is unclear if they can
run efficiently on edge devices.

Contributions. In this paper, we perform a comprehensive measurement study of
prediction-time unsupervised DNN adaptation techniques at the edge. To the
best of our knowledge, this is the first study aimed at exploring such
unsupervised approaches for resource-constrained devices. In particular, we
consider the following: (i) three robust DNNs that are pre-trained using data
augmentation [3] and adversarial training [4] on CIFAR-10 image classification
dataset (ResNeXt, Wide-ResNet, and ResNet-18). These models are among the top
models on the robustbench leaderboard111Robustbench systematically tracks the
progress being made in adversarial robust machine learning and keeps a record
of the DNN models that perform the best in terms of prediction accuracy for
corrupted/noisy datasets. However, it does not include adaptation approaches..
They use float32 data type as robustness to corruptions has not been well
explored for float16 or lower types; (ii) two adaptation algorithms (BN-Norm
[6, 14] and BN-Opt [5]); (iii) three edge devices with varying compute
capability, memory, and price (an Ultra96-v2 FPGA, a Raspberry-Pi, and Nvidia
Jetson AI acceleration platform: Xavier NX); and (iv) CIFAR-10-C dataset, that
includes a variety of image corruptions, is used to test the models and the
adaptation approaches.

The study answers the following algorithm-hardware co-design questions: (i)
for each device, what is the optimal choice of robust DNN and test-time
adaptation algorithm in terms of three objective functions: prediction
accuracy, adaptation time, and energy dissipated during adaptation? (ii) what
are the bottlenecks faced when executing these algorithms on the various
devices? and (iii) are there potential ML and hardware optimization
opportunities to improve adaptation time and energy? We also select the
overall best DNN, adaptation technique, and device for varying constraints on
the different cost metrics. The results show that there exists a gap between
algorithmic advances in adaptation and edge hardware designs to efficiently
support such advancements. The study reveals interesting and somewhat non-
obvious outcomes and demonstrates critical trade-offs between accuracy,
performance, energy, and memory.

In particular, there are five key outcomes: (i) for all three DNNs, BN-Opt
outperforms BN-Norm in terms of reducing prediction errors during test time
(by 2.45% on average) for CIFAR-10-C. Both are significantly better than no
adaptation (6.67% and 4.02%, respectively) even though the models were trained
offline with robust methods; (ii) while having a high number of BN parameters
in a DNN (ResNeXt) leads to best prediction accuracy after adaptation, such
models cannot be used with BN-Opt on memory-constrained edge devices such as
Ultra96-v2 FPGA. Even when using the GPU on Xavier NX, ResNeXt and BN-Opt
cause out of memory issues; (iii) if for a given application, prediction
accuracy is top priority, ResNeXt with BN-Opt should be used. For this
configuration, Xavier NX leads to the lowest runtime while Raspberry Pi
delivers the lowest energy; (iv) if all three cost metrics are equally
critical, then Wide-ResNet with BN-Norm should be used running on Xavier NX
GPU, which outperforms other devices. Compared to (iii), this configuration is
220$\times$ faster and 114$\times$ more energy-efficient but with 5.6%
increase in prediction error. However, the extra adaptation time is still
significant (213 ms) and can be a bottleneck for tight deadlines; (v) embedded
hardware DNN accelerators in edge devices can certainly improve performance
and energy efficiency during prediction-time adaptation. The overhead for BN-
Opt is more significant than BN-Norm, due to extra backpropagation, which
becomes a considerable bottleneck when run on low-power Arm cores. Therefore,
BN-Opt requires high-performance advanced accelerator, such as Volta GPU on
Xavier, to accelerate this training step effectively, showing up to
$7.89\times$ speedup compared to the CPU cluster on Xavier. The above outcomes
strongly motivate the need for algorithm-hardware co-design when designing on-
device DNN adaptation techniques. We conclude the paper with some
architecture-algorithm insights gained from this study.

##  II Background

This section provides an overview of the techniques to improve robustness of
DNNs as well as the BN-Norm and BN-Opt adaptation approaches that are the
focus of this paper.

###  II-A Improving robustness of DNNs during training

There are two widely used offline training techniques to improve robustness of
DNNs as described below.

####  II-A1 Data augmentation

Data augmentation involves training a neural network using not only the
“clean” original samples (such as images in CIFAR-10) but also with additional
noisy versions of the original samples [3]. This technique greatly improves
the generalization performance of the DNNs. Common examples of augmentation
include: Cutout, where a portion of an image is removed[15] and Cutmix that
replaces a portion of an image with another [16]. A recent augmentation
approach that was shown to achieve better robustness is Augmix, which applies
a series of augmentations to an image (such as rotate, posterize, etc.) then
mixes the transformed images into a new image [3]. Such samples, while quite
different from the original, are not unrealistic and capture various noises.
Augmix is used in this study.

####  II-A2 Adversarial training

This training makes a model robust to any adversarial attacks. The training
involves solving a min-max optimization problem, where: (i) adversarial image
samples are generated from the clean samples by finding imperceptible
perturbations to the clean images such that the prediction loss of a DNN is
maximized, and (ii) the prediction loss for these adversarial samples is
minimized. In our study, we use a state-of-the-art adversarial training
approach where imperceptibility is defined using learned perceptual image
patch similarity (LPIPS) distance [4]. LPIPS distance is based on the
activations of a DNN evaluated on two different images and is more suitable to
common image corruptions.

###  II-B BN-Norm test-time adaptation

BN-Norm adaptation is a lightweight approach to adapt DNNs at test time
without supervision to improve their classification accuracy[6, 14]. BN-Norm
only updates the batch-norm (BN) layers without modifying the other layers.
The BN statistics, mean ($\mu$) and standard deviation ($\sigma$), are
typically fixed after training. However, in BN-Norm, these statistics are
recomputed during test-time using an incoming batch of unlabeled data (other
BN parameters are not modified).

###  II-C BN-Opt test-time adaptation

BN-Opt also only updates the BN parameters during test-time without any
supervision [5]. BN-Opt, however, not only recomputes the mean and variance
parameters for each of the batch-norm layers but also optimizes the
transformation parameters that apply channel-wise scales and shifts to the
features. This approach is efficient as the transformation parameters
constitute $<1\%$ of the total model parameters.

In particular, two steps are performed for each batch-norm layer during test-
time: (i) normalization that standardizes the input $x$ into
$x^{\prime}=(x-\mu)/\sigma$ using its mean and standard deviation, and (ii)
transformation that turns $x^{\prime}$ into $x^{\prime\prime}=\gamma
x^{\prime}+\beta$ using scale ($\gamma$) and shift ($\beta$) parameters. While
the statistics for (i) are recomputed from the unlabeled data, scale and shift
parameters in (ii) are optimized by a loss function. Since, the optimization
is performed using unlabeled data, entropy of model predictions is used as the
loss function. Shannon entropy for a prediction $y$, which can be computed
without any labeled data, is defined as: $H(y)=-\sum_{c}p(y_{c})logp(y_{c})$
for probability of $y$ for class $c$. Predictions with lower entropy are shown
to exhibit lower prediction error rate as well.

##  III Measuring DNN Adaptation at the Edge

This section provides an overview of our measurement study, followed by the
details on the various parameters.

###  III-A Measurement study overview

Figure 1: Measurement study overview

Figure 1 shows an overview of our hardware-algorithm measurement study for
test-time unsupervised DNN adaptation at the edge. The aim is threefold: (i)
for each target device, among the state-of-the-art robust DNN models and
adaptation strategies, find the best model and adaptation algorithm that leads
to low prediction errors as well as low adaptation runtime and energy
consumption for a given noisy data stream. While the former cost metric is
important for robustness, minimizing the latter two are critical to meet the
constrained timing requirements for various streaming applications and be able
to run on low-power devices. We also find the overall best DNN, adaptation
scheme, and device in terms of the three cost metrics; (ii) measure and
provide a detailed analysis of the adaptation time overhead for the different
models and algorithms running on the various edge devices; and (iii) find
algorithm and hardware optimization opportunities.

In this paper, we consider three state-of-the-art robust neural networks, two
unsupervised NN adaptation approaches, a well-known image classification
dataset with corruptions, and three edge devices. The robust DNNs are pre-
trained models: trained with techniques such as Augmix data augmentation or
adversarial training. The two adaptation approaches are: BN-Norm adaptation
(Section II-B) and BN-Opt adaptation (Section II-C). In addition, we also
include no adaptation approach to provide a baseline for comparisons. For each
adaptation algorithm, the amount of recently-seen unlabeled test data used for
model update (batch size) is also varied: 50, 100, and 200 samples. We use
CIFAR-10-C dataset [17] that applies 15 different common corruptions to the
standard CIFAR-10 image classification dataset. Finally, the three edge
devices considered have varying processing capability and memory (as well as
power requirements), and are typically deployed for a variety of streaming
applications. These devices are: (i) with no inherent DNN acceleration
capability (Ultra96-v2 FPGA and Raspberry-Pi), and (ii) with a DNN accelerator
(Nvidia Jetson Xavier NX).

###  III-B Robust DNN models

####  III-B1 ResNet-18

To improve robustness, ResNet-18 is trained using both Augmix data
augmentation technique for CIFAR-10 dataset (Section II-A1, similar to [3]) as
well as using LPIPS-based adversarial training (Section II-A2, similar to
[4]). This trained ResNet-18 has 0.56 Giga multiply-accumulate (GMAC)
operations and 11.17M total parameters, out of which there are 7808 batch-norm
parameters (target for BN-Norm and BN-Opt). Overall it takes 86 MB memory.

####  III-B2 Wide-ResNet

We leverage Wide-ResNet-40-2 model, trained using Augmix data augmentation
technique for CIFAR-10 (Section II-A1, similar to [3]). This model has 0.33
GMAC operations, 2.24M total parameters, 5408 batch-norm parameters, and takes
9 MB memory.

####  III-B3 ResNeXt

We use ResNeXt29_32x4d with a cardinality of 4 and a base width of 32. ResNeXt
is also trained using Augmix for CIFAR-10 (Section II-A1, similar to [3]).
This model has 1.08 GMAC operations, 6.81M total parameters, 25216 batch-norm
parameters, and takes 26 MB memory. ResNeXt includes significantly more batch-
norm parameters than the above two models.

###  III-C Image classification data with corruptions

While the above three models are pre-trained on CIFAR-10, the CIFAR-10-C
dataset is used for testing the models. CIFAR-10-C includes 15 different types
of corruptions, such as gaussian noise, blur, snow, fog, frost, brightness,
etc., with 5 different severity levels (5 being the most severe and 1 being
the least). In our study, all 15 corruptions (level 5) are used.

The test data consist of streaming 10000 unlabeled CIFAR-10-C image samples
for each corruption type. To perform online adaptation during test time,
batches of recently seen image samples are used by the adaptation algorithms.
Three different batch sizes are used which have a direct impact on the
adaptation efficacy as well as performance/energy/memory during adaptation:
50, 100, and 200.

###  III-D Adaptation techniques

In this study, a no adaptation and two DNN adaptation algorithms are used on
the CIFAR-10-C test data: (i) no adaptation, where the three pre-trained
robust DNNs are used without any changes; (ii) BN-Norm adaptation (Section
II-B) that uses the recently seen batch of image samples to simply recompute
the mean and standard deviation BN parameters; and (iii) BN-Opt adaptation
(Section II-C) which recalculates the normalization parameters as well as
optimizes the BN transformation parameters. While BN-Norm does not need to
retrain when adapting the model, BN-Opt, on the other hand, performs a single
backpropagation pass using the collected batch of image samples to optimize
the transformation parameters for the entropy loss function using the Adam
optimizer.

The algorithms are implemented using Pytorch [18]. During test time with
CIFAR-10-C, the no adaptation algorithm keeps the model in eval() mode of
Pytorch (inference only), while BN-Norm and BN-Opt require the model to be in
train() mode (to allow for adaptation with inference). The forward passes for
both BN-Norm and BN-Opt are modified to adapt the DNN: at each adaptation
point, these algorithms first perform inference followed by updating (and
optimizing in case of BN-Opt) the batch-norm parameters based on the collected
recently seen batch of data. Since our focus is on adaptation (or retraining)
on edge devices, optimized frameworks for fast inference on edge, such as
Tensorflow-Lite [19] and TensorRT [20], are not applicable and hence not used.

###  III-E Edge devices

Details on the three edge devices are presented next.

####  III-E1 Ultra96-v2 FPGA

We use an FPGA due to their increased use in streaming application [21], both
due to the support for programmability as well as low-power operations. The
Ultra96-v2 consists of a Xilinx Zynq UltraScale+ MPSoC ZU3EG A484 processing
system (PS) part and 2 GB LPDDR4 memory. The DNN adaptation algorithms are run
on the PS while the programming logic (PL) part is not used in this study. We
used the PS quad-core ARM Cortex A53 multi-processing system that can run up
to 1.5 GHz, with 32 KB L1 caches and a 1 MB L2 cache. On the software side, we
use the Pynq framework [22] with Petalinux to create the Linux image on the
PS. Pytorch-1.8.0 for ARM is used.

####  III-E2 Raspberry-Pi

Another low-power edge device that has become critical for streaming
applications is Raspberry-Pi [23]. We use the Raspberry-Pi 4 Model B that
consist of a quad-core ARM A72 SoC running at up to 1.5 GHz with 32 KB data +
48 KB instruction L1 caches and 1 MB L2 cache. This device also comes with 8
GB LPDDR4 memory. We booted Ubuntu 21.04 and installed Pytorch-1.8.0 for ARM.

####  III-E3 Nvidia Jetson Xavier NX

While the above two devices did not use hardware neural network acceleration
capability, NX includes a GPU to accelerate DNN processing. NX uses a 6-core
Nvidia CARMel ARM SoC (128 KB L1 instruction cache + 64 KB L1 data cache, and
6 MB L2 cache and 4 MB L3 cache) that can run up to 1.9 GHz speed. It
integrates a 384-core Volta GPU (maximum frequency of 1.1GHz) and 8GB LPDDR4
memory. NX uses Jetpack 4.4 software development kit (SDK) that boots
Linux4Tegra operating system (similar to Ubuntu 18.04 but for Nvidia devices)
and comes with CUDA 10.2 and GPU NN acceleration libraries (cuDNN 8.0).
Pytorch-1.8.0 is installed using Nvidia’s pre-built installer.

###  III-F Objective functions

Three objective functions are considered: overall prediction accuracy for the
entire test stream, average forward time per batch (inference + any
adaptation), and average energy consumption per batch. Accuracy and
performance are computed using Pytorch. To compute energy, we measure power
per batch using Kuman wall outlet power meter.

Different applications give priorities to different objectives. For example,
some scenarios may have strict performance constraints, hence adaptation times
should be minimized, while for others maintaining prediction accuracy during
noise is more important. To find the best DNN model and adaptation approach
for a variety of cases, we combine the multiple objective functions into a
single one using a weighted approach. In particular, we use:
$w_{1}*time+w_{2}*energy+w_{3}*pred\\_error$ as the single objective function
to minimize, where $w_{1}$, $w_{2}$, and $w_{3}$ are weights whose sum should
be equal to 1. We find optimal configurations for four cases that cover a wide
variety of scenarios: (i) same importance to all three objectives (each weight
equal to 0.33); (ii) performance is the top priority but other metrics are not
completely discarded and hence assigned much smaller weights ($w_{1}=0.8$ and
others are 0.1 each); (iii) prediction error is more important ($w_{3}$ is
0.8) with other metrics much less critical but still considered (0.1 each);
and (iv) energy is critical with 0.8 weight and others as 0.1.

##  IV Measurement Results and Analysis

This section first presents the prediction accuracy results for the different
robust DNNs and adaptation approaches, followed by their detailed performance
analysis and multi-objective cost trade-offs for each of the three edge
devices. The section concludes with a summary of the overall outcomes of our
study, a comparison with standard MobileNet-V2 model, and the architecture-
algorithm insights gained from this study.

###  IV-A Prediction accuracy

Figure 2: Average prediction errors for CIFAR-10-C

Figure 2 shows the prediction error for the three algorithms: no adaptation,
BN-Norm, and BN-Opt, for an entire stream of 10000 unlabeled CIFAR-10-C image
samples. The error is averaged across the 15 noise types. Three robust NNs are
considered: ResNeXt with AugMix [3] (RXT-AM), Wide-ResNet with AugMix [3]
(WRN-AM), and ResNet-18 with both AugMix and adversarial training [4] (R18-AM-
AT). For each network, three test-time batch sizes are used for online
adaptation: 50, 100, and 200. The average prediction error is found to be
consistent across the edge devices in cases where they do not run out of
memory (RPi, Xavier NX).

As evident, no adaptation algorithm, even when using AugMix and adversarial
training techniques, shows the worst prediction error. BN-Norm, which
recomputes the normalization parameters during test time, is able to improve
on no adaptation by an average of 4.02% (across the 9 cases). BN-Opt, on the
other hand, shows an average improvement of 6.67% over no adaptation and 2.65%
over BN-Norm as it also optimizes the transformation parameters. For both BN-
Norm and BN-Opt, reduction in prediction error going from 50 to 100 batch size
is higher than from 100 to 200 showing that larger batch sizes than 200 will
yield diminishing returns. Additionally, bigger batch sizes for online
adaptation may not be reasonable for memory-constrained devices.

###  IV-B Ultra96-v2 processing system (PS)

Figure 3: Ultra96 PS forward times (inference + any adaptation)

This section presents the performance analysis for the FPGA PS, followed by
the performance-energy-accuracy trade-offs discussion. Average forward time
per batch is used as the performance metric to compare the different neural
networks using no adaptation (No-Adapt), and BN-Norm and BN-Opt. Forward time
for No-Adapt is simply the inference latency while for the others it also
includes any extra adaptation time: re-estimating normalization parameters for
BN-Norm and BN-Opt, and a single backpropagation pass for BN-Opt to optimize
the transformation parameters. The times are averaged over multiple
adaptations performed over an input data stream of CIFAR-10-C samples. Pytorch
is run on the FPGA PS in a multi-threaded setting utilizing all four Arm A53
cores, while exploiting intra-operation parallelism. Similarly, energy is also
measured for No-Adapt (inference only) and BN-Norm and BN-Opt
(inference+adaptation) at the different adaptation points and then averaged.

Performance analysis. As shown in Figure 3, compared to No-Adapt, the extra
average adaptation overhead for BN-Norm is 1.40 secs while for BN-Opt it is
much higher: 30.27 secs. The simple recompute operation of the former makes it
more suitable for test-time NN adaptation on an A53 device (while incurring a
prediction error overhead of around 2.65%). In addition, while BN-Norm is able
to run for all 9 cases on the FPGA PS, BN-Opt however, runs out of memory for
RXT for 100 and 200 batch sizes. So, while RXT-AM-100/200 with BN-Opt achieved
the lowest prediction error (Figure 2), they cannot be used on a constrained
memory device with only 2 GB memory. Even though RXT’s model size is 26 MB,
lower than R18 (86 MB), it runs out of memory as the number of BN parameters
optimized through backpropagation for RXT are significantly higher than the
others (25216 vs. 5408/7808). To perform backpropagation, Pytorch creates a
dynamic computational graph during forward pass that includes the nodes that
are enabled for gradient computation. RXT’s graph takes 3.12 GB and 5.1 GB
memory (based on a memory profiler used for R-Pi and NX) with batch size of
100 and 200, respectively, which is more than the FPGA’s 2GB memory. Moreover,
compared to the other ResNet models, RXT also shows significantly higher
forward time due to many more number of MAC operations (RXT: 1.08 GMACs, WRN:
0.33 GMACs, R18: 0.56 GMACs).

Figure 4 shows the breakdown of performance for BN-Opt, BN-Norm, and No-Adapt
for Wide-ResNet and ResNet-18 in terms of the time spent on forward and
backpropagation backward passes (averaged for convolution and BN layers). This
breakdown is obtained using the Pytorch Autograd profiler with batch size 50
(similar trend is expected for other batch sizes). The profiler runs out of
memory for RXT-AM (hence not included). While the forward times for
convolution layers are almost the same for a given network for each of the
three algorithms, the BN forward time is higher for BN-Norm and BN-Opt (up to
3.68$\times$ for WRN, 4.71$\times$ for R18) compared to no adaptation. The
reason is the re-tuning of the normalization parameters in the forward pass
which adds a significant cost (higher for R18 as it has more number of BN
parameters). Moreover, while there is no backpropagation performed for No-
Adapt and BN-Norm, this backward pass of BN-Opt shows considerable overhead
for both convolution and BN layers compared to the forward passes (up to
2.51$\times$ and 2.78$\times$, respectively). Although this result is expected
as A53s are not designed to run compute-intensive training algorithms, it also
motivates the need to explore potentially offloading training kernels to the
PL side of the FPGA for acceleration. Some of the recent works have started
looking into training acceleration on FPGAs [24].

Figure 4: Forward (fw) and any backpropagation backward (bw) pass performance
on Ultra96-v2 for batch size 50

Performance-energy-accuracy trade-offs. Figure 5 shows all three objectives
for the various cases of adaptation algorithms, DNNs, and batch sizes. Note
that RXT-AM-100/200 for BN-Opt are not present as they ran out of memory.
Three important cases are shown based on the single weighted objective
function ($w_{1}*forward\\_time+w_{2}*energy+w_{3}*error$, Section III-F): (i)
all three objectives are assigned a weight of 0.33 each, which leads to WRN-
AM-50 with BN-Norm to be the optimal point (3.95 secs, 4.93 J, and 15.21%
error) because of lightweight adaptation of BN-Norm, coupled with the smallest
number of BN parameters in Wide-ResNet; (ii) when accuracy is assigned a
higher weight of 0.8 while still giving small weights of 0.1 each to the other
two, WRN-AM-50 with BN-Opt is selected (13.35 secs, 14.35 J, 12.37% error) as
BN-Opt is highly effective at reducing prediction error and WRN balances
accuracy with performance and energy better than other models; and (iii) when
either performance or energy are top priority (0.8 weight), WRN-AM-50 with No-
Adapt (3.58 secs, 4.47 J, 18.26% error) is the reasonable choice.

Figure 5: Performance-energy-accuracy trade-offs: Ultra96-v2 PS

Summary. Even for Ultra96-v2 with A53 cores and 2 GB memory, real-time
unsupervised adaptation is possible although not efficient. Wide-ResNet using
BN-Opt (batch size of 50) is optimal when priority is given to prediction
error. In contrast, BN-Norm for the same network is the best when equal weight
is given to prediction error, performance, and energy. While ResNeXt (highest
number of BN parameters) with BN-Opt achieved the lowest prediction error, it
is unable to run on Ultra96-v2 PS, showing the need for device-aware design of
robust DNN models and adaptation algorithms targeting memory-constrained
devices. BN-Opt incurs significant performance overheads over no adaptation
and BN-Norm as it involves backpropagation. Use of PL side of the FPGA to
offload training kernels can be explored.

###  IV-C Raspberry Pi (RPi) 4

Performance analysis and the various cost trade-offs are now presented for
RPi.

Performance analysis. As shown in Figure 6, all three DNNs, with both BN-Norm
and BN-Opt, are able to run on the RPi as it packs more memory (8GB) than the
FPGA. Pytorch is run multi-threaded on quad-core Arm A72 cores. Compared to
No-Adapt, the extra adaptation time (in addition to the inference, across all
9 cases) for BN-Norm is only 0.86 secs on average but for BN-Opt it is 24.9
secs. Due to the use of A72s, these times are reduced compared to the FPGA.

Figure 6: RPi forward times (inference + any adaptation time)

Figure 7 shows the breakdown of performance for BN-Opt, BN-Norm, and No-Adapt
approaches for all three models in terms of the average time spent on forward
and backpropagation backward passes. Pytorch Autograd profiler is used with
the batch size fixed to 50 (similar trends for other batch sizes). Similar to
Ultra96-v2, forward BN time for BN-Norm and BN-Opt is up to 4.6$\times$ higher
than No-Adapt. Backward times for convolutions and BN in BN-Opt (zero for BN-
Norm and No-Adapt) are also significant and explain the high adaptation times
for BN-Opt in Figure 6.

Figure 7: Forward (fw) and any backpropagation backward (bw) pass performance
for RPi for batch size 50

Performance-energy-accuracy trade-offs. Figure 8 shows all three objectives
for the various cases. The same weighted multi-objective optimization function
is used. The important outcomes are: (i) for all three objectives assigned an
equal weight of 0.33 each, WRN-AM-50 with BN-Norm is the best (2.59 secs, 5.95
J, and 15.21% error) because of lightweight WRN and fast adaptation of BN-
Norm; (ii) when accuracy is assigned a higher weight of 0.8 while the other
two are 0.1 each, WRN-AM-50 with BN-Opt is selected (7.97 secs, 19.12 J,
12.37% error) as BN-Opt is highly effective at reducing prediction error for
noisy data. However, this configuration shows 3.07$\times$ higher forward time
and 3.21$\times$ more energy than (i); (iii) when performance is the top
priority (using 0.8 weight), interestingly, WRN-AM-50 with Norm is again
selected. Although its forward time is very slightly higher than No-Adapt for
WRN-AM-50 (2.59 secs vs. 2.04 secs) but due to the 0.1 weight assigned to
accuracy, BN-Norm leads to the overall better configuration; and (iv) when
energy is assigned the highest weight (0.8), WRN-AM-50 with no adaptation is
the best choice (2.4 secs, 5.04 J, 18.26%).

Figure 8: Performance-energy-accuracy trade-offs analysis: RPi

Summary. Bigger memory in RPi allows models with higher number of BN
parameters, such as ResNeXt, to run with BN-Opt. BN-Norm, with small loss of
accuracy, can be used to perform online adaptation more efficiently than BN-
Opt. Therefore, while Wide-ResNet with BN-Opt is the best choice when the top
priority is accuracy, the same network with BN-Norm should be used when fast
adaptation is key.

Figure 9: Xavier NX forward times (inference + any adaptation)

(a) ResNeXt

(b) Wide-ResNet

(c) ResNet-18

Figure 10: Xavier NX forward (fw) and any backpropagation backward (bw) time
for three DNNs (batch size: 50)

###  IV-D Nvidia Jetson Xavier NX

The forward times of the previous devices are high due to lack of
acceleration. This study evaluates if the adaptation can be accelerated using
embedded GPUs (NX has a 384-core Volta). It also includes 6-core Nvidia Carmel
Arm, and 8GB memory that is shared between the CPU and the GPU. For CPU
results, Arm cores are used to run multi-threaded Pytorch, and for GPU,
Pytorch CUDA is used.

Performance analysis. Figure 9 shows the performance of No-Adapt, BN-Norm, and
BN-Opt for all 9 cases using CPU and GPU. Interestingly, RXT-AM-200 with BN-
Opt runs out of memory when executed on the GPU. The dynamic Pytorch graph
created for RXT-AM-200 takes up around 5.1GB (when profiled on CPU). But
execution on the GPU leads to higher use of memory due to loading of extra
cuDNN libraries by Pytorch that are utilized for GPU acceleration. For all
other cases, speedup using the GPU is observed for the three algorithms and
DNN models: for No-Adapt 90.5% on average, for BN-Norm 68.13%, and for BN-Opt
79.21%.

Figure 10 shows the average forward and backward times for convolution and BN
layers. The convolution backward pass overhead when using GPU for BN-Opt (over
forward pass on GPU) is 2.2$\times$ on average. The same difference for CPU on
Xavier is 2.5$\times$. Faster forward and backward convolution passes for
Xavier explain the speedup obtained for BN-Opt using GPU. However,
interestingly, the forward BN performance is worse for RXT when using GPU over
CPU for both BN-Opt and BN-Norm, but it does not have a major impact on the
overall time.

Performance-energy-accuracy trade-offs. Figure 11 shows these cost metrics for
the various cases. The important outcomes are: (i) when all three costs are
equally weighted, WRN-AM-50 with BN-Norm on GPU leads to overall best solution
(0.31 secs, 2.96 J, 15.21%); (ii) when prediction error has 0.8 weight, WRN-
AM-50 with BN-Opt on GPU should be selected (0.82 secs, 7.96 J, 12.37%). Also,
note that the forward time (inference + adaptation) is under 1 sec. While the
GPU burns more power than CPU (2.2$\times$), the significantly faster
execution of the former makes it more energy-efficient (2.86$\times$); and
(iii) if the focus is more on energy or performance, then WRN-AM-50 with No-
Adapt (on GPU) is the optimal choice (0.10 secs, 1.02 J, 18.26%).

Figure 11: Performance-energy-accuracy trade-offs analysis for Jetson Xavier
NX

Summary. Volta in NX leads to significant speedups over CPU for not only No-
Adapt and BN-Norm but also for BN-Opt. Interestingly, we see out-of-memory
issues with the GPU for RXT. The lightweight Wide-ResNet with BN-Norm
algorithm running on GPU is the best when equal priority to given to all three
metrics. The same network with BN-Opt (using the GPU) is optimal when accuracy
is more important while still giving small weights to performance and energy.

###  IV-E Overall outcomes

Figure 12: Overall results with all the points from Figs. 5, 8, 11. A1/A2:
when accuracy is the only priority, A1 shows the lowest runtime and A2 the
lowest energy (i.e. among all points with 10.15% error). A3: optimal point
when all three costs are equally important (0.31s, 2.96J, 15.21%).

In this section, we find the overall best DNN model, robust training
technique, adaptation approach (with batch sizes), and the device/accelerator.

Figure 12 shows the complete results, plotting all the design points from
Figures 5, 8, 11 for the three costs. For an application where prediction
accuracy is the only priority, RXT-AM-200 with BN-Opt should be used with the
lowest (10.15%) error. This configuration will require at least 8 GB of memory
for execution, therefore Ultra96-v2 is not feasible. Among RPi and NX for this
model and adaptation approach, NX with CPU leads to the smallest runtime (A1:
69.58 secs) as GPU runs out of memory for batch size 200. However, in terms of
energy, RPi is the most efficient (A2: 337.43 J). But as is evident, both the
runtime and energy costs are significant and may not be practical.
Alternatively, if accuracy, performance, and energy are all equally important
for an application, then WRN-AM-50 with BN-Norm, running on NX GPU is the best
(A3: 0.315 secs, 2.96 J, 15.21%). Compared to BN-Opt (on GPU) with WRN-AM-50,
BN-Norm outperforms by 61.6% lower latency and 62.8% lower energy for the same
network. Compared to RXT-AM-200 and BN-Opt that achieved the best accuracy,
BN-Norm and WRN-50 on NX GPU is 220$\times$ faster and 114$\times$ more
energy-efficient. While WRN-AM-50 with BN-Norm shows 3.05% lower prediction
error than the same network with No-Adapt, the extra overhead for the former
is significant for real-time operation: 213 ms and 1.9 J (GPU).

###  IV-F Other edge-based non-robust DNN models

We also evaluated MobileNet-V2, a model which is optimized for low-power edge
systems. The model size is 9 MB, includes 0.096 GMAC operations, but has 34112
batch-norm parameters (larger than the three robust ResNet models in this
work). Closest to MobileNet is WRN (0.33 GMACs, 9 MB, but with 5408 BN
parameters). MobileNet is pretrained on CIFAR-10, but without adversarial
training, and then evaluated with or without adaptation on CIFAR-10-C.
MobileNet showed 81.2% prediction error without adaptation, which was reduced
to 28.1% using BN-Opt (batch size of 200). This error is still high compared
to the three robust models used above (10.15-12.97% with BN-Opt), highlighting
the importance of offline robust training (adversarial training/data
augmentation).

Table I shows the forward times for MobileNet for various approaches on the NX
GPU. For No-Adapt, MobileNet shows, on average, 80.9%, 19.2%, 34.8% better
performance than RXT, WRN, and R18, respectively (performance of these models
in Figure 9). However, for both BN-Norm and BN-Opt, it shows an average
overhead of $2.1\times$ compared to WRN and R18 as it includes much higher
number of BN parameters that are adapted (34112 compared to the two models:
5408, 7808). On the other hand, MobileNet outperforms ResNeXt for BN-Norm and
BN-Opt by $2.7\times$ as ResNeXt includes almost $10\times$ more GMACs (1.08)
and is closer in terms of number of BN parameters (25216). In summary, while
MobileNet is designed for edge, its large number of BN parameters is a
bottleneck for BN-Norm and BN-Opt like adaptation algorithms.

Batch Size | BN-Opt | BN-Norm | No-Adapt
---|---|---|---
50 | 1.63 s | 0.58 s | 0.07 s
100 | 3.7 s | 1.18 s | 0.13 s
200 | 8.28 s | 2.95 s | 0.25 s
Table I: MobileNet forward time on Xavier NX GPU

###  IV-G Architecture-algorithm insights

The key algorithm-hardware insights revealed by our study are: (i) trade-offs
between BN parameters, prediction accuracy, and execution time/memory
requirements must be considered when designing a robust DNN for edge. A model
with smaller number of BN parameters is more amenable to on-device NN
adaptation even though it might not lead to the best improvement in prediction
accuracy after adaptation, compared to a model with higher number of BN
parameters. For example, Wide-ResNet, with smallest number of BN parameters,
is shown to be overall more effective at balancing the various cost metrics
during adaptation than ResNeXt and ResNet-18 (and MobileNet); (ii) simply
updating BN parameters (BN-Norm) is more suited to test-time adaptation on the
edge as it is significantly faster and more energy-efficient than BN-Opt with
only 2.45% increase in prediction error on average. BN-Opt’s single
backpropagation pass is a major bottleneck. Our results motivate the design of
new hardware-aware adaptation algorithms; (iii) we show that embedded GPUs are
effective in speeding up BN-Norm and BN-Opt. However, the adaptation time is
considerable which can lead to real-time constraint violations. For example,
even when using BN-Norm with WRN-50 and NX GPU, the extra adaptation time
overhead is still significant: 213 ms. As these devices are primarily built
for inference, the adaptation overhead can be reduced with custom accelerators
designed to support fast BN-based adaptation (and/or backpropagation); (iv)
while the above results are using unpruned and full precision models, pruning
and quantization should be explored. However, care must be taken that any
model reduction should not compromise the robust accuracy against corruptions;
(v) online adaptation algorithms are not designed for hardware-constrained
edge platforms. Algorithms should minimize memory high water mark (streaming
approaches?), and make efficient use of accelerators while minimizing memory
overhead of doing so. On the hardware design side, additional MACs and routing
fabric would make back propagation less costly, and low power memories
including nonvolatile and 3D would enable larger batch sizes with less energy;
and (vi) as shown by the MobileNet study, online adaptation alone is not
sufficient and offline robust training, such as using data augmentation, is
also needed to improve accuracy against corruptions.

##  V Related Work

There has been use of generative adversarial networks (GANs) to design domain-
adaptation techniques for DNNs but they do not target unsupervised test-time
adaptation [25, 26]. These approaches use a semi-supervised method to improve
DNN robustness by adding a domain classifier that discriminates between the
source data (used for training) and target data (seen during test time), and
tries to maximize the domain classification loss so as to extract features
that are domain-invariant. However, these methods require full retraining of
the DNN with labeled source data and unlabeled target data, and are therefore
not applicable for fast test time adaptation.

There has been considerable work on characterizing DNN performance on a
variety of edge devices, however, they have only targeted inference and not
adaptation. Bianco et al. have benchmarked accuracy and inference time of a
variety of DNNs on Nvidia Jetson TX1 [27], and found some of the DNNs
bottlenecked by the amount of memory available on the device (such as
ResNeXt). Kljucaric et al. characterized the performance of AlexNet and
GoogleNet on several devices: Nvidia jetson AGX Xavier, Intel Neural Compute
Stick, and Google Edge TPU [28]. Their results showed best latency for AlexNet
is achieved by AGX, while for GoogleNet, TPU is faster. Similar analysis is
also targeted for robotics application [29]. An exhaustive characterization of
a variety of DNNs on many devices is performed but only for on-edge inference
without considering adaptation or robustness [30]. This analysis included
performance, energy, and temperature as well as various pruning/quantization
optimizations.

A recent work performed semi-supervised domain adaptation for an internet-of-
things edge device (TX2) for gesture recognition and image classification
applications [31]. A teacher-student technique is used, where knowledge is
transferred from a sophisticated teacher model (trained on a server on labeled
source data) to a simple student model running on the edge while considering
the domain shift in a new environment. However, this method requires initial
labeled source data to adapt the student model as well as assumes connection
to a server. Hence, it is not meant for unsupervised on-device prediction-time
adaptation. Other similar edge-based transfer learning approaches have been
introduced for health monitoring at home, but the fine tuning of the deployed
model is again performed using supervision and cloud connections [8]. There
are also distributed training based DNN update approaches that do not require
connection to a cloud, but they distribute training tasks to multiple
connected edge devices. A recent paper proposed a load balancing approach for
distributed learning on the edge [32].

##  VI Conclusion

This paper performed a comprehensive measurement study of prediction-time
unsupervised DNN adaptation techniques to quantify their performance and
energy on various edge devices. For each device, the study found the optimal
adaptation approach (between BN-Norm and BN-Opt) and the type of robust DNN
(among ResNeXt, Wide-ResNet, and ResNet-18) in terms of different cost
metrics. A detailed bottleneck analysis of the approaches was also presented,
followed by identifying optimization opportunities. Overall, we found that
Wide-ResNet with BN-Norm, running on Xavier NX GPU, to be highly effective at
balancing the various costs. However, the adaptation overhead is still
expensive for meeting tight deadlines. Algorithm-hardware co-design is
required to achieve efficient on-device adaptation.

## Acknowledgment

This work was performed under the auspices of the U.S. Department of Energy by
Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344. IM
number: LLNL-CONF-824925.

## References

  * [1] Saikiran Bulusu, Bhavya Kailkhura, Bo Li, Pramod K Varshney, and Dawn Song.  Anomalous example detection in deep learning: A survey.  IEEE Access, 8:132330–132347, 2020.
  * [2] Garrett Wilson and Diane J Cook.  A survey of unsupervised deep domain adaptation.  ACM Transactions on Intelligent Systems and Technology (TIST), 11(5):1–46, 2020.
  * [3] Dan Hendrycks, Norman Mu, Ekin D Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshminarayanan.  Augmix: A simple data processing method to improve robustness and uncertainty.  arXiv preprint arXiv:1912.02781, 2019.
  * [4] Klim Kireev, Maksym Andriushchenko, and Nicolas Flammarion.  On the effectiveness of adversarial training against common corruptions.  arXiv preprint arXiv:2103.02325, 2021.
  * [5] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell.  Tent: Fully test-time adaptation by entropy minimization.  ICLR Spotlight, 2021.
  * [6] Zachary Nado, Shreyas Padhy, D Sculley, Alexander D’Amour, Balaji Lakshminarayanan, and Jasper Snoek.  Evaluating prediction-time batch normalization for robustness under covariate shift.  arXiv preprint arXiv:2006.10963, 2020.
  * [7] James Diffenderfer, Brian R Bartoldson, Shreya Chaganti, Jize Zhang, and Bhavya Kailkhura.  A winning hand: Compressing deep networks can improve out-of-distribution robustness.  arXiv preprint arXiv:2106.09129, 2021.
  * [8] Abu Sufian, Changsheng You, and Mianxiong Dong.  A deep transfer learning-based edge computing method for home health monitoring.  In 2021 55th Annual Conference on Information Sciences and Systems (CISS), pages 1–6. IEEE, 2021.
  * [9] Peiying Zhang, Hao Sun, Jingyi Situ, Chunxiao Jiang, and Dongliang Xie.  Federated transfer learning for iiot devices with low computing power based on blockchain and edge computing.  IEEE Access, 2021.
  * [10] Phil Meier, Kris Rohrmann, Marvin Sandner, and Marcus Prochaska.  Transfer learning for neuronal networks deployed on the sensors edge.  In 2020 IEEE Sensors, pages 1–4. IEEE, 2020.
  * [11] Jinwoo Choi, Gaurav Sharma, Manmohan Chandraker, and Jia-Bin Huang.  Unsupervised and semi-supervised domain adaptation for action recognition from drones.  In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1717–1726, 2020.
  * [12] Kshitij Bhardwaj and Maya Gokhale.  Semi-supervised on-device neural network adaptation for remote and portable laser-induced breakdown spectroscopy.  On-device Intelligence Workshop (MLSys Conference), 2021.
  * [13] Yufan He, Aaron Carass, Lianrui Zuo, Blake E Dewey, and Jerry L Prince.  Autoencoder based self-supervised test-time adaptation for medical image analysis.  Medical Image Analysis, page 102136, 2021.
  * [14] Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge.  Improving robustness against common corruptions by covariate shift adaptation.  Advances in Neural Information Processing Systems, 33, 2020.
  * [15] Terrance DeVries and Graham W Taylor.  Improved regularization of convolutional neural networks with cutout.  arXiv preprint arXiv:1708.04552, 2017.
  * [16] Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo.  Cutmix: Regularization strategy to train strong classifiers with localizable features.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6023–6032, 2019.
  * [17] Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein.  Robustbench: a standardized adversarial robustness benchmark.  arXiv preprint arXiv:2010.09670, 2020.
  * [18] Pytorch.  Pytorch.  http://www.pytorch.org/.
  * [19] TensorFlow.  TensorFlow Lite.  http://www.tensorflow.org/lite.
  * [20] Nvidia.  Nvidia TensorRT.  http://developer.nvidia.com/tensorrt.
  * [21] Song Wu, Die Hu, Shadi Ibrahim, Hai Jin, Jiang Xiao, Fei Chen, and Haikun Liu.  When FPGA-accelerator meets stream data processing in the edge.  In 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS), pages 1818–1829. IEEE, 2019.
  * [22] Xilinx.  Pynq: Python Productivity.  http://www.pynq.io/.
  * [23] Diyi Hu and Bhaskar Krishnamachari.  Fast and accurate streaming CNN inference via communication compression on the edge.  In 2020 IEEE/ACM Fifth International Conference on Internet-of-Things Design and Implementation (IoTDI), pages 157–163. IEEE, 2020\.
  * [24] Cheng Luo, Man-Kit Sit, Hongxiang Fan, Shuanglong Liu, Wayne Luk, and Ce Guo.  Towards efficient deep neural network training by fpga-based batch-level parallelism.  FCCM, 2019.
  * [25] Yaroslav Ganin and Victor Lempitsky.  Unsupervised domain adaptation by backpropagation.  In International conference on machine learning, pages 1180–1189. ICML, 2015.
  * [26] Lanqing Hu, Meina Kan, Shiguang Shan, and Xilin Chen.  Duplex generative adversarial network for unsupervised domain adaptation.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1498–1507, 2018.
  * [27] Simone Bianco, Remi Cadene, Luigi Celona, and Paolo Napoletano.  Benchmark analysis of representative deep neural network architectures.  IEEE Access, 6:64270–64277, 2018.
  * [28] Luke Kljucaric, Alex Johnson, and Alan D George.  Architectural analysis of deep learning on edge accelerators.  In 2020 IEEE High Performance Extreme Computing Conference (HPEC), pages 1–7. IEEE, 2020.
  * [29] Alexander Biddulph, Trent Houliston, Alexandre Mendes, and Stephan K Chalup.  Comparing computing platforms for deep learning on a humanoid robot.  In International Conference on Neural Information Processing, pages 120–131. Springer, 2018.
  * [30] Ramyad Hadidi, Jiashen Cao, Yilun Xie, Bahar Asgari, Tushar Krishna, and Hyesoon Kim.  Characterizing the deployment of deep neural networks on commercial edge devices.  In 2019 IEEE International Symposium on Workload Characterization (IISWC), pages 35–48. IEEE, 2019.
  * [31] Jianfei Yang, Han Zou, Shuxin Cao, Zhenghua Chen, and Lihua Xie.  Mobileda: Toward edge-domain adaptation.  IEEE Internet of Things Journal, 7(8):6909–6918, 2020.
  * [32] Anirban Bhattacharjee, Ajay Dev Chhokra, Hongyang Sun, Shashank Shekhar, Aniruddha Gokhale, Gabor Karsai, and Abhishek Dubey.  Deep-edge: An efficient framework for deep learning model update on heterogeneous edge.  In 2020 IEEE 4th International Conference on Fog and Edge Computing (ICFEC), pages 75–84. IEEE, 2020.