#  AtmoSpec – A Tool to Calculate Photoabsorption Cross-Sections for
Atmospheric Volatile Organic Compounds

Daniel Hollas<EMAIL_ADDRESS>Basile F. E. Curchod
<EMAIL_ADDRESS>[

###### Abstract

Characterizing the photolysis processes undergone by transient volatile
organic compounds in the troposphere requires the knowledge of their
photoabsorption cross-section – quantities often challenging to determine
experimentally, particularly due to the reactivity of these molecules. We
present a computational tool coined AtmoSpec, which can predict a quantitative
photoabsorption cross-section for volatile organic compounds by using
computational photochemistry. The user enters the molecule of interest as a
SMILES code and, after selecting a level of theory for the electronic
structure (and waiting for the calculations to take place), is presented with
a photoabsorption cross-section for the low-energy conformers and an estimate
of the photolysis rate coefficient for different standardized actinic fluxes.
More specifically, AtmoSpec is an automated workflow for the nuclear ensemble
approach, an efficient technique to approximate the absolute intensities and
excitation wavelengths of a photoabsorption cross-section for a molecule in
the gas phase. This work provides background information on the nuclear
ensemble approach, a guided example of a typical AtmoSpec calculation, details
about the architecture of the code, and the current limitations and future
developments of this tool.

University of Bristol] Centre for Computational Chemistry, School of
Chemistry, University of Bristol, Cantock’s Close, Bristol BS8 1TS, United
Kingdom

##  1 Introduction

Understanding and predicting the photochemical reactivities and lifetimes of
atmospheric volatile organic compounds (VOCs) upon sunlight absorption is of
prime importance in informing chemical models used to simulate the molecular
composition of our atmosphere.1, 2 However, the potential for photolysis of a
large number of VOCs, in particular transient VOCs, remains unknown to date
due to their short lifetime and instability, preventing their isolation or
synthesis and rendering spectroscopic measurements highly challenging.

The photolysis of a compound A following sunlight absorption, A -¿[hν] B + C,
is characterized by the photolysis rate $-\frac{d[A]}{dt}=J[A]$, where $J$ is
the photolysis rate coefficient. This photolysis rate coefficient is defined
as

$J=\int_{\lambda_{\text{min}}}^{\lambda_{\text{max}}}\phi(\lambda)\sigma(\lambda)F(\lambda)d\lambda.$
(1)

Equation (1) provides the story of a photolysis process: the actinic flux
($F(\lambda)$) informs on the number of photons coming from a source (e.g.,
the sun) at a given wavelength per second and per unit area, and the
likelihood of molecule A to absorb one of these photons is given by its
photoabsorption cross-section $\sigma(\lambda)$. If the molecule has absorbed
light at wavelength $\lambda$, the quantum yield $\phi(\lambda)$ gives the
probability that the photoexcited molecule will undergo the photolysis process
described above and form B + C. Integrating the product
$\phi(\lambda)\sigma(\lambda)F(\lambda)$ over a range of wavelength
characteristic of the actinic region (280 - 400 nm) leads to the overall
photolysis rate coefficient $J$. (We note at this point that $J$ often depends
on the temperature, pressure, and the solar zenith angle.)

Equation (1) makes it clear that a key quantity to consider when studying the
potential for photolysis of a VOC is its photoabsorption cross-section
$\sigma(\lambda)$. Photolysis data for a limited number of key VOCs can be
found in databases like MPI-Mainz UV/Vis Spectral Atlas,3 IUPAC,4 NASA (JPL),5
or atmospheric models such as TUV.6 Information about the potential photolysis
of VOCs is crucial to inform atmospheric models like the Master Chemical
Mechanism used to simulate the chemical composition of the atmosphere.7, 8, 9,
10 As stated above, obtaining such quantities experimentally is often hampered
by the instability of the molecule (or the difficulty in isolating it), and
structure-activity relationships (SARs) – regularly used for ground-state
properties of VOCs – often run out of steam when extended to photochemical
processes, notably due to the possible delocalized nature of molecular excited
states and the multichromophoric nature of some VOCs.11

Recently, we proposed to use computational photochemistry to obtain an
estimate of the photoabsorption cross-section of transient VOCs.12, 11 The
computational strategy, based on the nuclear ensemble approach (NEA, see Sec.
2 below for details),13, 14 offers an adequate description of the absolute
cross-section and the position of the maxima in the photoabsorption cross-
section for VOCs. While this technique does not describe vibronic
progressions,13, 15, 16 the width of each band is captured qualitatively well,
which is a prime factor when considering the use of calculated photoabsorption
cross-sections in Equation (1).

Hence, the NEA could act as a doorway to predict the photoabsorption cross-
section of transient VOCs and, hopefully, determine more robust photochemical
SARs rules for compounds of the same family. However, the practical execution
of the NEA workflow is rather tedious (yet not complex), which hinders its
adoption outside of the computational photochemistry community. Here, we
introduce AtmoSpec – an automated workflow and graphical interface to predict
the photoabsorption cross-sections of VOCs based on the NEA and requiring only
a limited number of operations from the user, making the theoretical
prediction of cross-sections more accessible to researchers outside of the
computational photochemistry community.

In the following, we briefly summarize the main conceptual ideas of the NEA,
the different steps required for a typical calculation, and the limitations of
the methods (Sec. 2). We then walk the reader through an example calculation
in AtmoSpec to determine the photoabsorption cross-section of glycolaldehyde
(Sec. 3). We present the design and architecture of AtmoSpec and the details
of the automatized NEA workflow implemented using the AiiDA infrastructure
(Sec. 4). We finally stress the current limitations of AtmoSpec and highlight
its future developments (Sec. 5).

##  2 A brief survey of the nuclear ensemble approach

Different theoretical strategies can be used to predict the photoabsorption of
gas-phase molecules like VOCs (see Ref. 11 for an overview), but here we focus
exclusively on the NEA which represents a good tradeoff between computational
efficiency and qualitatively robust results. Crucially, unlike other more
complicated approaches, the NEA workflow is amenable to automation. The NEA,
which is a numerical realization of the reflection principle,14 proposes to
project the ground-state nuclear density onto the excited electronic state(s)
of interest.13 In practice, the NEA consists of (i) approximating the ground-
state probability density of the molecule of interest, (ii) sampling from this
distribution a set of $N_{p}$ molecular geometries ($N_{p}\approx 500-10000$),
and (iii) calculating, for each molecular geometry $j$ ($\mathbf{R}_{j}$,
where $\mathbf{R}$ is a collective variable for all the nuclear coordinates of
the molecule of interest), the corresponding vertical excitation energies
($\Delta E_{0I}(\mathbf{R}_{j})$) and transition dipole moments
($\boldsymbol{\mu}_{0I}(\mathbf{R}_{j})$) to a subset of $N_{s}$ excited
electronic states of interest (labeled by $I$). For each geometry, each
transition energy is broadened by a Gaussian (or Lorentzian) with a set width
$\delta$. The photoabsorption cross-section is obtained by averaging over all
the $N_{s}$ broadened electronic transitions for all the Np geometries. Hence,
the photoabsorption cross-section within the NEA for a given conformer is
obtained as

$\sigma(E)=\frac{\pi}{3\hbar\epsilon_{0}c}\frac{1}{N_{p}\delta\sqrt{2\pi}}\sum_{j=1}^{N_{p}}\sum_{I=1}^{N_{s}}\Delta
E_{0I}(\mathbf{R}_{j})|\boldsymbol{\mu}_{0I}(\mathbf{R}_{j})|^{2}\exp{\left(-\frac{\left(E-\Delta
E_{0I}(\mathbf{R}_{j})\right)^{2}}{2\delta^{2}}\right)}\,,$ (2)

with $E$ being the photon energy.

If the molecule of interest exists as multiple conformers at a given
temperature, the protocol described above should be repeated for each
conformer, and the total photoabsorption cross-section is obtained by summing
the Boltzmann-weighted photoabsorption cross-section for each conformer.

The central idea of the NEA – to project a ground-state probability density
vertically onto excited electronic state(s) – is justified for dissociative
excited electronic states for which the corresponding photoabsorption cross-
section does not exhibit vibronic progression.14 In other words, the NEA
cannot reproduce vibronic progressions as it does not consider the vibrational
states of the excited electronic states.15, 11 For dissociative excited states
($n\sigma^{\ast}$, $\pi\sigma^{\ast}$, $\sigma\sigma^{\ast}$), which are of
central interest for photolysis processes related to atmospheric
photochemistry, the NEA can provide an accurate description of the
photoabsorption cross-sections (absolute intensities and excitation
wavelengths), given that an adequate electronic-structure method was employed
for excitation energies and transition dipole moments – see Refs.16, 17, 11,
18, 19, 20 for examples. However, the NEA was shown to provide a qualitatively
correct description of the transition bands for photoabsorption cross-sections
involving bound excited electronic states (like $n\pi^{\ast}$), capturing the
proper envelope of each band as well as its location in energy and absolute
intensity. The key ingredient for the success of the NEA in this later case
lies in its ability to capture non-Condon effects by sampling molecular
geometries away from the equilibrium geometry (i.e., the transition dipole
moments in Eq. (2) depends on the nuclear configuration of the sampled
geometry, $\boldsymbol{\mu}_{0I}(\mathbf{R}_{j})$). Hence, the NEA was
successfully used in the context of atmospheric chemistry to describe the low-
energy $n\pi^{\ast}$ band of the photoabsorption cross-section for carbonyl-
containing VOCs.21, 22 Another potential limitation of the NEA is its
inaccurate description of the tail of the photoabsorption cross-section, which
should be stressed in the context of atmospheric photochemistry as a
photolysis rate coefficient can be highly sensitive to a low-energy tail of
the photoabsorption cross-section if it enters artificially deeper in the
actinic region.

The NEA relies on the determination of an (approximate) ground-state nuclear
probability density. The commonly employed approach is to use a harmonic
approximation for the different vibrational modes of the molecule of interest,
as one can determine analytically from this approximation a so-called Wigner
distribution for both nuclear positions and momenta.23, 24, 11 The key
advantage of this approach is that a harmonic Wigner distribution only
requires, for each conformer, an equilibrium geometry in the ground electronic
state and its corresponding vibrational frequencies – two quantities readily
available from any standard quantum-chemical software. Sampling molecular
geometries from a harmonic Wigner distribution (used as a proxy for the
ground-state nuclear probability density) is often called ‘Wigner sampling’.
Wigner sampling is implemented in different codes to perform excited-state
molecular dynamics, as the nuclear positions and momenta it provides can be
used as initial conditions for excited-state dynamics. Using a harmonic Wigner
distribution to provide the molecular geometries (nuclear positions) required
by the NEA leads to accurate photoabsorption cross-sections for VOCs
exhibiting rather harmonic modes (for example, acrolein11). However, care
should be taken for flexible molecules, typically possessing low-frequency
(anharmonic) normal modes, as they may lead to artifacts in the
photoabsorption cross-section when the low-frequency mode is photoactive.20
More advanced strategies can be used to obtain ground-state distributions for
flexible molecules, such as ground-state ab initio molecular dynamics with a
quantum (or colored-noise) thermostat25, 26, 27 (QT) or path-integral
molecular dynamics.28 A detailed discussion of the NEA applied to VOC and its
performances can be found in Ref. 11.

From a practical perspective, calculating the photoabsorption cross-section
for a given VOC requires the following steps. First, one needs to find all
molecular conformers that are meaningfully present at a given temperature
(this step can be done manually or involve dedicated software for conformer
exploration). Once the conformers are located, an electronic-structure method
should be chosen for the subsequent ground- and excited-state calculations,
ensuring that we have the best compromise between computational efficiency and
accuracy (this step requires a quantum-chemical package). Once the electronic-
structure method is selected, the following steps should be executed for each
conformer: (i) geometry optimization and frequency calculation, (ii) sampling
of $N_{p}$ molecular geometries from the constructed harmonic Wigner
distribution (this step often requires an external software), (iii)
calculation of excitation energies and transition dipole moments for each
sampled geometry, (iv) construction of the photoabsorption cross-section for
the conformer (this step requires a bespoke code, sometimes the same as in
step (ii)). The final photoabsorption cross-section is then calculated by
weighting the photoabsorption cross-section of each conformer by its
corresponding Boltzmann factor. The steps described above hopefully make it
clear that determining the photoabsorption cross-section for a given VOC is
not per se difficult, but it implies a rather tedious and repetitive set of
operations, involving the use of different software packages. In the next
Sections, we discuss how these steps have been turned into an automated
workflow within AtmoSpec.

##  3 A guided tour of AtmoSpec

The goal of AtmoSpec is to transform the steps of an NEA calculation
(described in the last paragraph of Sec. 2) into an automated workflow with
the minimum effort for the user. The overall workflow of AtmoSpec is depicted
in Fig. 1 and this Section endeavors to describe each step of an NEA
calculation with AtmoSpec from a user perspective. As an example, we propose
to calculate the photoabsorption of glycolaldehyde, HCOCH2OH.

Figure 1: The different steps of AtmoSpec and its overall workflow. User icons
highlight the steps where the user input is requested. The step numbering
corresponds to the subsections discussed in Sec. 3.

###  3.1 Step 1 – Initial molecular structure and conformer search

Figure 2: Step 1 – (a) the user inputs the structure of the VOC of interest
(SMILES code or an external coordinate file) and (b) AtmoSpec proceeds with a
(low-level) search of the dominant conformers, displaying all the conformers
found and their approximate energy. (c) AtmoSpec creates an entity in a
database for this molecule.

AtmoSpec implements its graphical interface via a web application that can be
accessed using any modern web browser. The user connects to AtmoSpec via its
address on the machine where the code was installed (detailed installation
information is provided on the GitHub page of AtmoSpec29) After launching
AtmoSpec, the user is first requested to provide the molecular structure for
the VOC of interest, here glycolaldehyde (Figure 2a). The simplest way to
provide this information is to use a SMILES (simplified molecular-input line-
entry system) code – a simple ASCII string encoding the structure of a
molecule.30 A SMILES code is easily obtained from a Lewis structure created
with a molecular editor (for example, the SMILES code of glycolaldehyde is
O=CCO). Once the SMILES code is provided, the user can click on the ’Generate
molecule’ button. AtmoSpec will proceed with a fast search of the possible
conformers of the molecule using the ETKDG algorithm31, 32 implemented in the
RDKit package,33 followed by geometry optimization using the MMFF94 force
field.34 AtmoSpec will display all the conformers of the molecule in a ball-
and-stick representation within a window for the user to confirm that this is
indeed the molecule of interest (Figure 2b). AtmoSpec also displays the MMFF94
energies that serve as an initial estimate of their Boltzmann population. Both
the conformer geometries and energetics are further refined in subsequent
steps. As an alternative to the input via SMILES, the user can also upload a
file containing the molecular structure of interest, for example in an XYZ
format. In this case, AtmoSpec does not perform a conformer search.

The user is then invited to provide a description and label for the
calculation to generate an entry in AtmoSpec database (Figure 2c). The use of
a database means that the results of this calculation can be recovered at any
time in the future by selecting the entry via its label.

###  3.2 Step 2 – Computational details for the NEA calculation

Figure 3: Step 2 – The computational details of the AtmoSpec workflow are
provided. The user can specify the molecular details (charge and multiplicity)
as well as the level of theory required. The number of sampled geometries from
the harmonic Wigner distribution can be entered. Information about the
computational resources (number of cores) can be defined.

The second step is critical as it defines the computational details for the
calculation of the photoabsorption cross-section. The user will be invited to
provide information about the molecule (charge, multiplicity) and the level of
electronic-structure theory required. The ground-state electronic structure
information will be used to optimize the different conformers identified in
Step 1 and calculate the vibrational frequencies required to build the
harmonic Wigner distribution for the molecule. The excited-state electronic
structure method defines the level of theory for the calculation of vertical
excitation energies and transition dipole moments that will be required for
building the photoabsorption cross-section based on Eq. (2). In its current
version, AtmoSpec is interfaced with the quantum-chemical software package
ORCA,35 which offers a broad range of capabilities for excited electronic
states and is free for academic use.

While default parameters are provided for the calculation of a photoabsorption
cross-section with a compromise between efficiency and accuracy (based on
earlier experience with the NEA and VOCs), AtmoSpec offers different options
to the users to refine their calculation and, perhaps more importantly,
benchmark the level of electronic-structure theory. Hence, if the ’Number of
Wigner samples’ is set to 0, the user can perform single-point calculations on
top of the optimized geometry of each conformer to test the sensitivity of
vertical transitions to the quantum-chemical method. AtmoSpec gives access to
LR-TDDFT (with and without the Tamm-Dancoff approximation), but allows the
user to perform calculations with the wavefunction-based methods ADC(2) and
EOM-CCSD. While these two electronic-structure methods are likely to be quite
expensive for a full NEA calculation (i.e., with multiple Wigner samples),
they offer an adequate point of comparison for single-point calculations. We
note that implicit solvent can be included too, motivated by recent
experiments where the photoabsorption cross-section of VOCs was deduced from
their extinction coefficients in cyclohexane.36

Once all the computational details are provided, the user can define where the
calculations should be launched (locally or remotely, for example on an HPC
cluster that the user has access to) as well as the number of cores that the
electronic-structure software can use. By clicking on the ’Submit’ button, the
user will launch the fully-automatized NEA calculation.

###  3.3 Step 3 – Status of the calculation and outputs

Figure 4: Step 3 – This menu appears when an NEA calculation with AtmoSpec has
been launched and offers an overview of its progress. Clicking on ’Show
workflow details’ gives access to the underlying output files produced by the
quantum-chemical software. All the output files are saved in the AtmoSpec
database.

Once the user clicks on the ’Submit’ button in Step 2, AtmoSpec distributes
the required calculations on the available resources (either in a serial or
parallel mode). The user sees a status bar appearing (Figure 4) that
summarizes the progress of the NEA workflow. At this stage, the user can close
the AtmoSpec tab from their web browser, as the details of the molecule and
its NEA workflow are saved in the AtmoSpec database under the descriptor
provided in Step 1. The user can simply reopen AtmoSpec and search for the
descriptor name in the AtmoSpec database to reload the current status of the
calculation.

For the user interested in the details of the NEA workflow, Step 3 gives
direct access to all the output files provided by the quantum-chemical
software in use (in the present example, ORCA). The button ’Show workflow
details’ (Figure 4) opens the organization tree of the workflow – geometry
optimization and frequency calculation for each conformer, followed by the NEA
calculation for each conformer – and all the respective output files can be
found at the bottom of this tree. At a higher level of this tree, AtmoSpec
provides a report of the key quantities extracted from the output files.

###  3.4 Step 4 – Results

Figure 5: Photoabsorption cross-section produced by AtmoSpec. (a) The top
panel provides the details of the NEA workflow employed to produce the results
presented in Step 4. (b) Interactive visualization of the calculated
photoabsorption cross-section (blue) superimposed with the experimental cross-
section (dashed orange) if uploaded by the user. (c) The representation of the
photoabsorption cross-section (and its parameters) can be altered directly
from the control panel. (d) The low-energy conformers of the molecule of
interest are represented as ball-and-stick models in an interactive window,
with their respective energy and Boltzmann population calculated at the level
of theory defined in Step 2.

Once all the calculations are done, AtmoSpec summarizes the results in the
’Step 4’ and displays first the calculated photoabsorption cross-section
(Figure 5). AtmoSpec provides a summary of the computational details for the
NEA workflow (Figure 5a). In this particular case, we calculated the
photoabsorption cross-section of glycolaldehyde with LR-TDDFT/TDA, using the
$\omega$B97X-D4 functional and the aug-cc-pVDZ basis set. 500 geometries were
sampled from the harmonic Wigner distribution produced for each conformer (two
conformers considered in total).

The calculated photoabsorption cross-section is provided in an interactive
window, together with any experimental photoabsorption cross-section that can
be imported by the user (Figure 5b). In this particular example, the overall
shape and width of the photoabsorption cross-section are properly captured by
the NEA, while the intensity is slightly overestimated with the level of
electronic-structure theory selected. Different tools are available to
interact with the photoabsorption cross-section, and the user can save the
results either as a CSV text file or a picture. A menu is also provided for
tuning more advanced parameters related to the NEA (Figure 5c), such as the
function used to broaden each vertical excitation within the NEA, its width
($\delta$), or the overall energy unit used to represent the photoabsorption
cross-section. A side window depicts the low-energy conformers considered to
build the photoabsorption cross-section (Figure 5d) and their relative energy,
calculated at the level of theory defined in Step 2.

Figure 6: AtmoSpec contains a tool to highlight the contribution of each
conformer considered to the calculated photoabsorption cross-section. The red
line in the photoabsorption cross-section window symbolizes the contribution
to the photoabsorption cross-section by the selected conformer in the
molecular window.

The analysis of the conformers can be pushed further by clicking on the ’Show
conformers’ button (Figure 5a). In this mode, AtmoSpec shows the contribution
of each conformer to the full photoabsorption cross-section. The user can
select a conformer from the molecular-representation window, and its
photoabsorption cross-section – weighted by the Boltzmann population of the
conformer – is highlighted in red in the photoabsorption cross-section (Figure
6). In the specific case of glycolaldehyde presented here, a low-energy
conformer with a Boltzmann population of 0.6% at 298 K shows a contribution to
the photoabsorption cross-section that is shifted to longer wavelength (red
line in Figure 6) in comparison to the dominant conformer exhibiting an
intramolecular hydrogen bond (blue line in Figure 6).

Figure 7: Qualitative prediction of the photolysis rate constant for the VOC
of interest based on the calculated photoabsorption cross-section. (a) The
user can select three standardized actinic fluxes $F(\lambda)$ – low, medium,
or high flux (see main text for details). (b) The quantum yield is set by the
user and allows them to determine a qualitative value for the photolysis rate
constant by integrating Eq. (1) over the actinic flux. (c) Interactive window
showing the integrand $j(\lambda)=\phi(\lambda)\sigma(\lambda)F(\lambda)$
(blue curve) and the actinic flux (red curve).

A last window gives access to an estimation of the photolysis rate constant
$J$ using Eq. (1) with the calculated photolysis rate constant, a user-defined
value for the photolysis quantum yield, and three options for the actinic
fluxes (Figure 7). The user can select one of the three standardized actinic
fluxes (Figure 7a) – ’high flux’ (solar zenith angle = 0∘, overhead ozone
column = 200 DU), ’medium flux’ (solar zenith angle = 60∘, overhead ozone
column = 350 DU), and ’low flux’ (solar zenith angle = 90∘, overhead ozone
column = 500 DU) for a ground elevation of 0 km above sea level. An
interactive window depicts the integrand of Eq. (1), namely the product of the
calculated photoabsorption cross-section, the quantum yield, and the chosen
actinic flux (Figure 7c). An approximate wavelength-independent quantum yield
can be provided by the user (Figure 7b) and its effect on the calculated $J$
value is adapted in real-time so that the user can determine the upper and
lower bounds for the photolysis rate constant based on the photoabsorption
cross-section predicted by AtmoSpec.

##  4 Software architecture and implementation

In this Section, we provide a brief description of the overall architecture of
AtmoSpec as well as an overview of the software stack and libraries used to
build this tool.

###  4.1 Design goals

The overarching goal of AtmoSpec was to make the calculation of
photoabsorption cross-section with the NEA accessible to non-experts in
computational photochemistry. With that in mind, the following criteria were
key for the development of the architecture and implementation of AtmoSpec:
(a) simple installation, (b) graphical interface that is easy to navigate, (c)
input and output data should be stored, and easily searchable, and (d) the
calculations should be fully reproducible.

A trivial way to address point (a) is to develop a software that does not
necessarily need to be installed on the user’s computer. This can be achieved
by making the graphical interface run in a web browser, while the application
itself may run on a remote computer. This strategy means that the application
itself can be deployed by an expert user or IT service, while the user only
needs to navigate to the correct URL in the web browser to interact with its
graphical interface. As a secondary goal, we also aimed to make the
installation of the application approachable and reproducible. We achieved
that by delivering AtmoSpec as a Linux container (using Docker37), which
bundles all the necessary dependencies together. To achieve points (c) and
(d), we utilized the workflow manager AiiDA (see below). We note that the
points highlighted above align with the FAIR principles (findable, accessible,
interoperable, reusable) for scientific data.38 To further enhance the
reusability of AtmoSpec, the different parts of the software (such as the
harmonic Wigner sampling module) will be made available as standalone Python
packages, usable independently from the full AtmoSpec tool.

###  4.2 Architecture and data flow

The overall architecture of AtmoSpec is depicted in Fig. 8. We have decided to
utilize the existing ecosystem around the workflow manager AiiDA.39, 40 With
this choice, the implementation of the NEA workflow is fairly generic: AiiDA
handles the tasks of submitting individual ab initio calculations to the
quantum-chemical program (here, ORCA), parsing the resulting outputs and
storing all the critical information in a relational database (PostgreSQL) and
a custom file storage. This strategy gives us the flexibility to combine
AtmoSpec with other quantum-chemical programs in the future. Moreover, AiiDA
benefits from broad support for handling automatic job submission to remote
HPC clusters (key to install AtmoSpec locally but utilize a remote HPC
resource for the quantum-chemical calculations).

The other central part of AtmoSpec is its graphical interface (front-end). We
again employed AiiDAlab, which was specifically created to make quantum-
chemical calculations and complex scientific workflows available through a
user-friendly browser interface.41 In AiiDAlab, the front-end is built on top
of a Jupyter notebook interface. While this choice is somewhat unconventional
for a web application, it has the advantage that the graphical part of the
code is written in the same programming language (Python) as the rest of the
program. This strategy makes AtmoSpec (and other AiiDAlab-based applications)
more approachable for potential developers, in a scientific environment where
Python is nowadays perceived as a de facto lingua franca.

Figure 8: Architecture diagram of AtmoSpec. The Graphical interface in the
browser (implemented in the AiiDAlab framework) communicates with the Workflow
manager (AiiDA), which orchestrates the ab initio calculations (using ORCA)
and stores the results in the SQL Database (PostgreSQL). Each rectangle in the
diagram may represent a different physical computer, but the whole system can
be deployed on a single computational server. The individual components can be
deployed together using a Docker container.

The AtmoSpec code is fully open source, as is most of the software stack that
it builds upon, with the major exception of ORCA, which is however available
for free for academic use under a custom EULA license. (The link to the
AtmoSpec GitHub repository can be found in Ref. 29.)

##  5 Caveats and future developments

AtmoSpec provides a simple interface to calculate an approximate
photoabsorption cross-section, but the user needs to keep in mind that this
simplicity leads to a series of strict limitations. We list here the main
current limitations, and we discuss in the following paragraph the future
developments that may address some of these issues. (1) AtmoSpec only offers a
rather modest choice of (single-reference) electronic-structure methods and,
as such, is not equipped to treat molecules exhibiting a challenging
electronic structure (e.g., radicals or metal complexes). (2) AtmoSpec relies
on the NEA, meaning that the produced photoabsorption cross-sections do not
exhibit any vibronic structure. Perhaps more importantly in the atmospheric
context, the (low-energy) tail of the photoabsorption cross-sections is likely
to be too high in intensity, even for purely dissociative states. (3) AtmoSpec
uses a harmonic Wigner distribution to sample the geometries used in the NEA.
This strategy may lead to artifacts when used for flexible VOCs, that is,
molecules exhibiting low-frequency vibrational modes (¡ 200 cm-1), as
discussed in Ref. 11. Such modes can be removed from the sampling by applying
a low-frequency cutoff in Step 2. The use of ab initio molecular dynamics
combined with a quantum thermostat is desirable for sampling the ground-state
probability density of such molecules, but harder to automatize. (4) AtmoSpec
does not offer any speed-up for the calculation of photoabsorption cross-
sections within the NEA. For VOCs, these calculations can take from a few
hours to a few days, depending on the level of theory and the number of
sampled geometries for the NEA.

A series of future developments are planned for AtmoSpec, some of them trying
to address the limitations listed above. (a) To reduce the computational cost
of a typical photoabsorption cross-section calculation, we plan to interface
AtmoSpec with the GPU-accelerated electronic structure package TeraChem. In
addition to this speed-up of the electronic-structure calculations, we will
incorporate the idea of optimal sampling within AtmoSpec.42 The idea of the
optimal sampling is to use a cheap level of electronic-structure theory (for
example, ZIndo/S) to calculate the photoabsorption cross-section of the
molecule of interest within the NEA and a normal number of sampled geometries
(around 500). From this result, the optimal sampling can select a subset
(around 50) of optimal geometries that best represent the overall
photoabsorption cross-sections. The final photoabsorption cross-section can be
obtained by performing the NEA on this subset of geometries using a higher
level of electronic structure theory. (b) As part of the automation in
AtmoSpec, we plan to devise a strategy to pre-select an optimal level of
theory based on the input molecular structure and automatically determine the
number of excited electronic states important to describe the absorption of
the molecule in the actinic region. (c) AtmoSpec could be generalized to other
types of spectroscopies within the NEA framework, such as photoelectron
spectroscopy43, 19 or X-ray absorption spectroscopy.44

##  6 Summary

In summary, this work introduced AtmoSpec, a computational tool devised to
predict the photoabsorption cross-section and other photolysis properties of
transient volatile organic compounds for atmospheric applications. AtmoSpec
offers an automated workflow for the nuclear ensemble approach, only requiring
a minimal input from the (non-expert) user using a web browser interface. This
work describes a typical calculation with AtmoSpec, going through the details
of each step of the workflow. We also highlighted the main limitations of the
current implementation and its future developments. AtmoSpec is open source
and freely available for download – the GitHub link can be found in Ref. 29.

The authors would like to thank Fay Abu Al Timen, Marco Barnfield, Kirstin
Gerrand, Will Hobson, Kon Nomerotski, and Emily Wright – undergraduate
students at the University of Bristol – for their contribution to the
development of the photolysis rate constant calculation widget. DH would like
to thank the AiiDA and AiiDAlab developers that comprise an incredibly
welcoming open source community, in particular Aliaksander Yakutovich, Jusong
Yu, Xing Wang, Marnik Bercx, Sebastiaan Huber, Carl Simon Adorf, Carlo
Pignedoli, and Giovanni Pizzi. DH would also like to thank the many users of
AtmoSpec within the group in Bristol for their feedback and bug reports. The
authors also acknowledge Sasha Madronich for providing the standardized
actinic fluxes implemented in AtmoSpec. This project has received funding from
the European Research Council (ERC) under the European Union’s Horizon 2020
research and innovation programme (Grant agreement No. 803718, project SINDAM)
and the EPSRC Grants EP/V026690/1, EP/X026973/1, and EP/Y01930X/1. This
article is also based upon work from COST Action CA18212 - Molecular Dynamics
in the GAS phase (MD-GAS), supported by COST (European Cooperation in Science
and Technology).

## References

  * Ervens et al. 2024 Ervens, B.; Rickard, A.; Aumont, B.; Carter, W. P. L.; McGillen, M.; Mellouki, A.; Orlando, J.; Picquet-Varrault, B.; Seakins, P.; Stockwell, W. et al. Opinion: Challenges and needs of tropospheric chemical mechanism development. _EGUsphere_ 2024, _2024_ , 1–35
  * Curchod and Orr-Ewing 2024 Curchod, B. F. E.; Orr-Ewing, A. J. Perspective on Theoretical and Experimental Advances in Atmospheric Photochemistry. _The Journal of Physical Chemistry A_ 2024,
  * Keller-Rudek et al. 2013 Keller-Rudek, H.; Moortgat, G.; Sander, R.; Sörensen, R. The MPI-Mainz UV/VIS spectral atlas of gaseous molecules of atmospheric interest. _Earth System Science Data_ 2013, _5_ , 365–373
  * ACP 2021 (accessed July 2024 ACP copernicus website. https://acp.copernicus.org/articles/special_issue8.html, 2021 (accessed July 2024)
  * Burkholder et al. 2020 Burkholder, J. B.; Sander, S. P.; Abbatt, J. P. D.; Barker, J. R.; Cappa, C.; Crounse, J. D.; Dibble, T. S.; Huie, R. E.; Kolb, C. E.; Kurylo, M. J. et al. _Chemical kinetics and photochemical data for use in atmospheric studies; evaluation number 19_ ; 2020
  * TUV 2021 (accessed July 2024 TUV website. https://www2.acom.ucar.edu/modeling/tropospheric-ultraviolet-and-visible-tuv-radiation-model, 2021 (accessed July 2024)
  * Jenkin et al. 1997 Jenkin, M. E.; Saunders, S. M.; Pilling, M. J. The tropospheric degradation of volatile organic compounds: a protocol for mechanism development. _Atmospheric Environment_ 1997, _31_ , 81–104
  * Saunders et al. 2003 Saunders, S. M.; Jenkin, M. E.; Derwent, R. G.; Pilling, M. J. Protocol for the development of the Master Chemical Mechanism, MCM v3 (Part A): tropospheric degradation of non-aromatic volatile organic compounds. _Atmospheric Chemistry and Physics_ 2003, _3_ , 161–180
  * Jenkin et al. 2003 Jenkin, M. E.; Saunders, S. M.; Wagner, V.; Pilling, M. J. Protocol for the development of the Master Chemical Mechanism, MCM v3 (Part B): tropospheric degradation of aromatic volatile organic compounds. _Atmospheric Chemistry and Physics_ 2003, _3_ , 181–193
  * MCM 2021 (accessed July 2024 Master Chemical Mechanism, MCM v3.3.1. http://mcm.york.ac.uk, 2021 (accessed July 2024)
  * Prlj et al. 2021 Prlj, A.; Marsili, E.; Hutton, L.; Hollas, D.; Shchepanovska, D.; Glowacki, D. R.; Slavíček, P.; Curchod, B. F. E. Calculating Photoabsorption Cross-Sections for Atmospheric Volatile Organic Compounds. _ACS Earth and Space Chemistry_ 2021, _6_ , 207–217
  * Prlj et al. 2020 Prlj, A.; Ibele, L. M.; Marsili, E.; Curchod, B. F. E. On the theoretical determination of photolysis properties for atmospheric volatile organic compounds. _Journal of Physical Chemistry Letters_ 2020, _11_ , 5418–5425
  * Crespo-Otero and Barbatti 2012 Crespo-Otero, R.; Barbatti, M. Spectrum simulation and decomposition with nuclear ensemble: formal derivation and application to benzene, furan and 2-phenylfuran. _Theoretical Chemistry Accounts_ 2012, _131_ , 1237
  * Schinke 1995 Schinke, R. _Photodissociation Dynamics: Spectroscopy and Fragmentation of Small Polyatomic Molecules_ ; Cambridge University Press, Cambridge, 1995
  * Crespo-Otero and Barbatti 2018 Crespo-Otero, R.; Barbatti, M. Recent Advances and Perspectives on Nonadiabatic Mixed Quantum–Classical Dynamics. _Chemical Reviews_ 2018, _118_ , 7026–7068
  * McGillen et al. 2017 McGillen, M. R.; Curchod, B. F. E.; Chhantyal-Pun, R.; Beames, J. M.; Watson, N.; Khan, M. A. H.; McMahon, L.; Shallcross, D. E.; Orr-Ewing, A. J. Criegee intermediate–alcohol reactions, a potential source of functionalized hydroperoxides in the atmosphere. _ACS Earth and Space Chemistry_ 2017, _1_ , 664–672
  * Sršeň et al. 2018 Sršeň, Š.; Hollas, D.; Slavíček, P. UV absorption of Criegee intermediates: quantitative cross sections from high-level ab initio theory. _Physical Chemistry Chemical Physics_ 2018, _20_ , 6421–6430
  * McCoy et al. 2021 McCoy, J. C.; Marchetti, B.; Thodika, M.; Karsili, T. N. A simple and efficient method for simulating the electronic absorption spectra of Criegee intermediates: Benchmarking on CH2OO and CH3CHOO. _The Journal of Physical Chemistry A_ 2021, _125_ , 4089–4097
  * Clarke et al. 2022 Clarke, C. J.; Gibbard, J. A.; Hutton, L.; Verlet, J. R. R.; Curchod, B. F. E. Photochemistry of the pyruvate anion produces CO2, CO, CH${}_{3}^{-}$, CH3, and a low energy electron. _Nature Communications_ 2022, _13_ , 937
  * Prlj et al. 2023 Prlj, A.; Hollas, D.; Curchod, B. F. E. Deciphering the Influence of Ground-State Distributions on the Calculation of Photolysis Observables. _The Journal of Physical Chemistry A_ 2023, _127_ , 7400–7409
  * Hutton and Curchod 2022 Hutton, L.; Curchod, B. F. E. Photodynamics of Gas-Phase Pyruvic Acid Following Light Absorption in the Actinic Region. _ChemPhotoChem_ 2022, _6_ , e202200151
  * Marsili et al. 2022 Marsili, E.; Prlj, A.; Curchod, B. F. E. A theoretical perspective on the actinic photochemistry of 2-hydroperoxypropanal. _The Journal of Physical Chemistry A_ 2022, _126_ , 5420–5433
  * Persico and Granucci 2014 Persico, M.; Granucci, G. An overview of nonadiabatic dynamics simulations methods, with focus on the direct approach versus the fitting of potential energy surfaces. _Theoretical Chemistry Accounts_ 2014, _133_ , 1–28
  * Suchan et al. 2018 Suchan, J.; Hollas, D.; Curchod, B. F. E.; Slavíček, P. On the Importance of Initial Conditions for Excited-State Dynamics. _Faraday Discussions_ 2018,
  * Ceriotti et al. 2009 Ceriotti, M.; Bussi, G.; Parrinello, M. Nuclear quantum effects in solids using a colored-noise thermostat. _Physical Review Letters_ 2009, _103_ , 030603
  * Ceriotti et al. 2010 Ceriotti, M.; Bussi, G.; Parrinello, M. Colored-noise thermostats à la carte. _Journal of Chemical Theory and Computations_ 2010, _6_ , 1170–1180
  * Huppert et al. 2022 Huppert, S.; Plé, T.; Bonella, S.; Depondt, P.; Finocchi, F. Simulation of Nuclear Quantum Effects in Condensed Matter Systems via Quantum Baths. _Applied Sciences_ 2022, _12_
  * Markland and Ceriotti 2018 Markland, T. E.; Ceriotti, M. Nuclear quantum effects enter the mainstream. _Nature Reviews Chemistry_ 2018, _2_ , 0109
  * Hollas and Curchod 2024 (accessed July 2024 Hollas, D.; Curchod, B. F. E. AtmoSpec. 2024 (accessed July 2024); https://doi.org/10.5281/zenodo.11075300
  * Weininger 1988 Weininger, D. SMILES, a chemical language and information system. 1. Introduction to methodology and encoding rules. _Journal of Chemical Information and Computer Sciences_ 1988, _28_ , 31–36
  * Riniker and Landrum 2015 Riniker, S.; Landrum, G. A. Better Informed Distance Geometry: Using What We Know To Improve Conformation Generation. _Journal of Chemical Information and Modeling_ 2015, _55_ , 2562–2574
  * Wang et al. 2020 Wang, S.; Witek, J.; Landrum, G. A.; Riniker, S. Improving Conformer Generation for Small Rings and Macrocycles Based on Distance Geometry and Experimental Torsional-Angle Preferences. _Journal of Chemical Information and Modeling_ 2020, _60_ , 2044–2058
  * 33 RDKit: Open-source cheminformatics. https://www.rdkit.org
  * Halgren and Nachbar 1996 Halgren, T. A.; Nachbar, R. B. Merck molecular force field. IV. conformational energies and geometries for MMFF94. _Journal of Computational Chemistry_ 1996, _17_ , 587–615
  * Neese et al. 2020 Neese, F.; Wennmohs, F.; Becker, U.; Riplinger, C. The ORCA quantum chemistry program package. _The Journal of Chemical Physics_ 2020, _152_ , 224108
  * Mapelli et al. 2023 Mapelli, C.; Donnelly, J. K.; Hogan, U. E.; Rickard, A. R.; Robinson, A. T.; Byrne, F.; McElroy, C. R.; Curchod, B. F. E.; Hollas, D.; Dillon, T. J. Atmospheric oxidation of new “green” solvents – Part 2: methyl pivalate and pinacolone. _Atmospheric Chemistry and Physics_ 2023, _23_ , 7767–7779
  * Merkel 2014 Merkel, D. Docker: lightweight linux containers for consistent development and deployment. _Linux journal_ 2014, _2014_ , 2
  * Wilkinson et al. 2016 Wilkinson, M. D.; Dumontier, M.; Aalbersberg, I. J.; Appleton, G.; Axton, M.; Baak, A.; Blomberg, N.; Boiten, J.-W.; da Silva Santos, L. B.; Bourne, P. E. et al. The FAIR Guiding Principles for scientific data management and stewardship. _Scientific Data_ 2016, _3_ , 160018
  * Uhrin et al. 2021 Uhrin, M.; Huber, S. P.; Yu, J.; Marzari, N.; Pizzi, G. Workflows in AiiDA: Engineering a high-throughput, event-based engine for robust and modular computational workflows. _Computational Materials Science_ 2021, _187_ , 110086
  * Huber et al. 2020 Huber, S. P.; Zoupanos, S.; Uhrin, M.; Talirz, L.; Kahle, L.; Häuselmann, R.; Gresch, D.; Müller, T.; Yakutovich, A. V.; Andersen, C. W. et al. AiiDA 1.0, a scalable computational infrastructure for automated reproducible workflows and data provenance. _Scientific Data_ 2020, _7_ , 300
  * Yakutovich et al. 2021 Yakutovich, A. V.; Eimre, K.; Schütt, O.; Talirz, L.; Adorf, C. S.; Andersen, C. W.; Ditler, E.; Du, D.; Passerone, D.; Smit, B. et al. AiiDAlab – an ecosystem for developing, executing, and sharing scientific workflows. _Computational Materials Science_ 2021, _188_ , 110165
  * Sršeň and Slavíček 2021 Sršeň, Š.; Slavíček, P. Optimal Representation of the Nuclear Ensemble: Application to Electronic Spectroscopy. _Journal of Chemical Theory and Computation_ 2021, _17_ , 6395–6404
  * Arbelo-González et al. 2016 Arbelo-González, W.; Crespo-Otero, R.; Barbatti, M. Steady and Time-Resolved Photoelectron Spectra Based on Nuclear Ensembles. _Journal of Chemical Theory and Computation_ 2016, _12_ , 5037–5049
  * Muchova et al. 2023 Muchova, E.; Hollas, D.; Holland, D. M. P.; Bacellar, C.; Leroy, L.; Barillot, T. R.; Longetti, L.; Coreno, M.; de Simone, M.; Grazioli, C. et al. Jahn–Teller effects in initial and final states: high-resolution X-ray absorption, photoelectron and Auger spectroscopy of allene. _Physical Chemistry Chemical Physics_ 2023, _25_ , 6733–6745


# Dynamic Neural Network is All You Need: Understanding the Robustness of
Dynamic Mechanisms in Neural Networks

Mirazul Haque, Wei Yang
University of Texas at Dallas, USA
<EMAIL_ADDRESS>
###### Abstract

Deep Neural Networks (DNNs) have been used to solve different day-to-day
problems. Recently, DNNs have been deployed in real-time systems, and lowering
the energy consumption and response time has become the need of the hour. To
address this scenario, researchers have proposed incorporating dynamic
mechanism to static DNNs (SDNN) to create Dynamic Neural Networks (DyNNs)
performing dynamic amounts of computation based on the input complexity.
Although incorporating dynamic mechanism into SDNNs would be preferable in
real-time systems, it also becomes important to evaluate how the introduction
of dynamic mechanism impacts the robustness of the models. However, there has
not been a significant number of works focusing on the robustness trade-off
between SDNNs and DyNNs. To address this issue, we propose to investigate the
robustness of dynamic mechanism in DyNNs and how dynamic mechanism design
impacts the robustness of DyNNs. For that purpose, we evaluate three research
questions. These evaluations are performed on three models and two datasets.
Through the studies, we find that attack transferability from DyNNs to SDNNs
is higher than attack transferability from SDNNs to DyNNs. Also, we find that
DyNNs can be used to generate adversarial samples more efficiently than SDNNs.
Then, through research studies, we provide insight into the design choices
that can increase robustness of DyNNs against the attack generated using
static model. Finally, we propose a novel attack to understand the additional
attack surface introduced by the dynamic mechanism and provide design choices
to improve robustness against the attack.

##  1 Introduction

Deep Neural Networks (DNNs) are used in multiple applications such as computer
vision and natural language processing. After the rapid growth of IoT and
embedded devices, many real-time systems use DNNs in their applications. As
the real-time systems require faster response time and low energy consumption,
researchers have proposed to incorporate energy-saving dynamic mechanism [34,
24, 36] to popular static DNN (SDNN) models like ResNet [20], VGG [31],
MobileNet [22] etc. Early-exit is one of the dynamic mechanism techniques
where multiple exits are included in SDNNs (creating multiple sub-networks),
and SDNNs can terminate the operation early if a certain sub-network is
confident about the prediction. These types of DNNs are named as early-exit
Dynamic Neural Networks (DyNNs). Although the transition from SDNNs to DyNNs
is preferred in real time systems because of increased efficiency, whether the
use of dynamic mechanism will impact the robustness of the systems is still
unknown. Studying the impact of the dynamic mechanisms on the robustness is
important for developers or users to understand the trade-offs between DyNN
and SDNN.

Figure 1: The overview of the objective of this work.

In this work, we propose to investigate how robustness is impacted through the
addition of dynamic mechanism to the static model and how different dynamic
mechanism designs would impact the robustness in dynamic mechanism (Figure 1).
We propose to investigate two aforementioned topics through three research
questions. These three RQs are based on: Robustness of Dynamic Mechanism,
Robust Design for the Static Model Attack, and Robust Design for the Dynamic
Mechanism Attack.

Robustness of Dynamic Mechanism. We focus on the investigation of robustness
of the dynamic mechanism based on two aspects: transferability and the
efficiency robustness.

First, we investigate the adversarial attack transferability between SDNNs and
DyNNs to evaluate the robustness of the dynamic mechanism in black-box
scenarios. In the black-box scenarios, adversaries normally assume the target
models are always static. However, the target models can be dynamic also.
Hence, it is important to find out if a surrogate SDNN model is used to attack
a target DyNN model or vice-versa, then, to which extent the adversary can be
successful.

To address this issue, in this paper, we first conduct a comparative study on
the adversarial attack transferability between SDNNs and DyNNs (Section 3).
Our study results suggest that adversarial transferability from DyNNs to SDNNs
is better and surprisingly, using DyNNs as surrogate models for attack seems
to be a more efficient and more effective way to generate adversarial samples.
The adversaries are able to generate more adversarial samples in the same
amount of time compared to using SDNNs as the surrogate model, and the
generated adversarial samples often can also attack SDNNs.

The next robustness study is to understand how robust the efficiency of the
dynamic mechanism is against the adversarial samples generated on SDNNs. This
study tries to evaluate whether the original purpose of DyNNs (i.e., saving
inference time) will be impacted by the adversarial samples (Section 3.2)
generated through SDNNs. Our study results suggest that the adversarial
samples generated by existing white-box attacks and black-box attacks do not
increase the inference time significantly, making the dynamic mechanisms
efficiently robust against attacks on SDNNs.

Robust Design for the Static Model Attack Here, we perform a detailed analysis
of which design choices in the dynamic mechanisms or DyNN architectures
(specifically the position of early exits) may impact the robustness of DyNNs
(Section 4). We consider two attack scenarios in this study: first, the output
layer label of an SDNN is modified by a white-box adversarial example, and we
study the impact of the example on corresponding DyNN’s early-exit layers;
second, in a black-box scenario, the output of SDNN is modified by a sample,
and the sample is fed to separate model’s DyNN. We have made multiple findings
based on the empirical results, for example, putting the first exit earlier in
the model architecture can help to improve the robustness of DyNNs.

Robust Design for the Dynamic Mechanism Attack. Last but not least, we
evaluate the design choices of dynamic mechanism against a novel attack on
dynamic mechanism. First, we design an adversarial attack approach to
understand the extra attack surface introduced by the dynamic mechanisms in
neural network (Section 5). In this attack, the synthesized adversarial
examples will not change the prediction of the final output layer’s label, but
will change the prediction of all the early exits. Based on the evaluation
results, we find that the dynamic mechanism is more vulnerable in scenarios
where dependency among DyNN layers is lesser and when the exits are sparse
w.r.t the layers.

Contribution. Our work provides insights about the robustness of the dynamic
mechanism in DyNNs and also evaluates different design choices w.r.t
robustness. The findings of this work would motivate developers/researchers to
increase the usage of DyNNs and find more robust dynamic mechanisms.

##  2 Related Works and Background

Dynamic Neural Networks. The main objective of DyNNs is to decrease the energy
consumption of the model inference for inputs that can be classified with
fewer computational resources. DyNNs can be classified into Conditional-
skipping DyNNs and Early-exit DyNNs. Early-exit DyNNs use multiple exits (sub-
networks) within a single model and because of the model’s working mechanism,
the model is more suited for resource constrained devices. If, at any exit,
the confidence score of the predicted label exceeds user defined threshold,
inference is stopped. The resource-constrained devices usually deploy a
lightweight sub-network of early exit network locally and resort to a server
for further computations if needed [33] . [14], [12], [32], [24] have proposed
Early-termination DyNNs. Specifically [24] and [38] propose early exit
networks based on popular SDNNs. [38] also show that white-box robustness of
the DyNNs is better than SDNNs. In addition to that multiple works [33, 29]
provide practical usability of DyNNs.

Figure 2: Working mechanism of Early-exit DyNN

Figure 2 shows the working mechanism of Early-exit DyNNs. For example, an
Early-exit DyNN has N parts and each part has an exit. $x$ is the input,
$f^{i}_{out}$ represents prediction after the $i^{th}$ part (generated by
specific computation unit), $f_{out}$ represents prediction of the Neural
Network, $C_{i}$ represents confidence score after $i^{th}$ part,
$Hid^{In}_{i}$ represents input of $i^{th}$ part, $Hid^{Out}_{i}$ represents
output of $i^{th}$ part, and $\tau_{i}$ is the predefined threshold to exit
the network after $i^{th}$ part. The working mechanism of the Early-exit DyNN
can be represented as, $f_{out}(x)=f^{i}_{out}(x),\text{if}\
C_{i}(x)\geq\tau_{i}$.

Adversarial Attacks. Adversarial Examples are the input that can change the
prediction of the DNN model when those are fed to the model. [13] propose Fast
Gradient Sign Method (FGSM) that uses single-step first order entropy loss to
generate adversarial inputs. This attack is modified by [27] to add initial
noise to the benign sample. This attack is referred as projected gradient
descent (PGD). Other than that, [10, 2, 9, lin2019nestero] have proposed
white-box attack methods, while [26, 1, 23, 37] have proposed black-box attack
methods. Recently, attacks [16, 19, 17, 18, 15, 4, 7, 6, 3] have been proposed
to decrease the efficiency of the dynamic neural networks.

##  3 How robust are DyNNs against adversarial examples?

In this research question, we investigate the robustness of dynamic mechanism
in DyNNs w.r.t adversarial examples generated on SDNNs. We further divide this
question into two sub-parts: (i) attack transferability between SDNNs and
DyNNs and (ii) efficiency robustness of DyNNs against adversarial examples
generated on SDNNs. Through these RQs, we get an insight about how the
existing adversarial attacks on SDNNs would impact the accuracy and efficiency
of DyNNs. Also, we find out how adversarial examples generated on DyNNs impact
the SDNNs w.r.t accuracy. As the SDNN models are static, adversarial examples
would not have any impact on the efficiency of SDNNs.

###  3.1 Is adversarial example transferability from DyNN to SDNN lower than
adversarial example transferability from SDNN to DyNN?

In this research question, we investigate the “transferability” of adversarial
inputs generated based on SDNN and DyNN, respectively, i.e., whether
adversarial examples generated based on SDNNs are adversarial to DyNNs and
vice versa. Transferability is an important metric for evaluating the
feasibility of black-box attack. To evaluate the transferability, one of the
popular way [28, 26] is creating a similar model (i.e., surrogate model) as
the target model. In a black-box attack, normally, adversaries assume the
underlying model to be SDNN, so for a deployed DyNN, the adversaries may
likely use an SDNN as surrogate model. Hence, this research question (RQ) is
important to evaluate the robustness of DyNNs.

(a) Target DyNN (CIFAR-10)

(b) Target SDNN (CIFAR-10)

(c) Target DyNN (CIFAR-100)

(d) Target SDNN (CIFAR-100)

Figure 3: Transferable Attack Success Rate for CIFAR-10 and CIFAR-100 data

####  3.1.1 Experimental Setup.

Dataset and Models. We use CIFAR-10 and CIFAR-100 [25] datasets for
evaluation. For SDNNs, we use VGG-16 [31], ResNet56 [20], and MobileNet [22]
model. As DyNNs, we use the early exit version of these models [24]. In all
other RQs, we keep the models and dataset setup same.

Black-box Attack. For the attack scenario, we use surrogate model [28, 26]
based black-box attack scenario. Here, we feed a set of inputs to the target
model and collect the output labels. These inputs are generated using 50% of
the held-out validation data, and naturally corrupted versions of those
validation data. As the number of partial held-out validation data is not
significant, adding corrupted inputs would help to increase training data size
for the surrogate model. For natural corruption [21], we use gaussian noise
and brightness. For each type of corruption, we have five intensity levels.
For example, if the number of held out data is 5000, for each corruption, we
generate 25000 additional data. Once the input-output pairs are collected, a
surrogate model is trained based on those pairs. For a target model, we use
both SDNNs and DyNNs. If the target model is SDNN, then an DyNN is trained as
surrogate model and vice-versa.

To make the surrogate-target pairs, we use different types of DNN
architectures. For example, if the the target model is DyNN VGG, we choose
ResNet56 SDNN as the surrogate model. This assumption is valid because the
attacker doesn’t have information about the target model architecture, hence
the possibility of choosing the same architecture as the surrogate model is
less. We define two terms to represent two different types of transferability
based on different types of surrogate model and target model: D2S
transferability and S2D transferability. D2S transferability evaluates DyNN to
SDNN attack transferability, where S2D transferability evaluates SDNN to DyNN
attack transferability. We have chosen following pairs to evaluate S2D
transferability: (SDNN ResNet56 (surrogate), DyNN VGG (target)), (SDNN
MobileNet (surrogate), DyNN MobileNet (target)), (SDNN MobileNet
(surrogate),DyNN ResNet56 (target)). Similarly, for D2S transferability,
earlier mentioned surrogate models become the target model and earlier
mentioned target models become the surrogate model.

Algorithms. We use FGSM [13] and PGD [27] algorithms to attack the surrogate
models.

Metric. We measure percentage of adversarial examples that can mis-classify
the output w.r.t number of generated adversarial examples as the attack
success rate.

####  3.1.2 Evaluation Results

Figure 3 shows the effectiveness of black-box attacks on DyNNs and SDNNs. On
average, it can be noticed that for target SDNN and surrogate DyNN, the attack
success rate is higher than the success rate of target AdNN and surrogate
SDNN. One of the reasons for this behavior is the lower variance of the DyNNs.
DyNNs use lower number of parameters, hence the feature space for adversarial
samples of DyNNs is smaller than the feature space for adversarial samples of
SDNNs [30]. Also, we find that for the target DyNN, FGSM attack performs
better than PGD attack. If only dataset-specific results are considered, then
for CIFAR-100 the attack success rate is higher than for CIFAR-10.

Also, as the DyNN-generated adversarial inputs can attack SDNNs, then it can
be time efficient to create adversarial inputs using DyNN. Through Figure 8
(Appendix), we can find the probability density plots of different exit
numbers of DyNN that have been used to generate adversarial examples. Lower
exit number suggests that lesser number of computations has been used to
generate adversarial examples. It can be noticed that for PGD attack, more
than 70% of adversarial examples are generated from exit 0, 1 and 2 (first -
third exit). Although for FGSM attack, in a few scenarios (CIFAR-10 VGG,
CIFAR-100 MobileNet, and CIFAR-100 ResNet), more than 50% of adversarial
examples are generated through later exits (higher computation required).

We also perform some additional experiments to strengthen our claim. In
appendix, Figure 13 shows the experimental results on MI-FGSM attack, which is
also a highly transferable attack. Also Figure 17 shows the results on Tiny-
ImageNet dataset, which has a higher input size than CIFAR datasets. These two
results show that our claim also holds for addditional attacks and datasets.
Also Figure 14,15 and 16 in appendix shows adversarial examples generated
through SDNNs and DyNNs. We find that both type of adversarial examples are
similar w.r.t quality.

Finding 1: The D2S transferability is higher than the S2D transferability.
Finding 2: Using DyNNs as surrogate models is more efficient and more
effective way to generate adversarial examples than using SDNNs.

###  3.2 Does adversarial examples impact efficiency robustness in DyNNs?

In this section, we investigate if the dynamic mechanism in DyNNs is robust
w.r.t efficiency against the adversarial examples generated on static mdoel.
Through the evaluation, we ensure whether the original purpose of including
dynamic mechanism in DyNNs (i.e., saving inference time) will be impacted by
the adversarial samples. Specifically, we study whether the adversarial inputs
exit earlier or later in a DyNN compared to the original inputs. The main
objective of this investigation is to find out whether the adversarial samples
generated on SDNN can have an impact on the amount of computation of the DyNN.
For this purpose, we conduct both white-box and black-box attacks to find out
the impact of adversarial samples on the amount of computation.

Here, the white-box attack scenario can also be considered a practical
scenario. There have been studies [5, 35] that focus on reverse engineering of
SDNN models from binary code of on-device models, but no techniques have been
proposed to reverse engineer the dynamic mechanisms in the models. Hence,
adversaries are more likely to get SDNN models instead of their dynamic
counterparts. So it is important to find out how adversarial samples affect
the efficiency of the DyNN for both white-box and black-box scenarios.

####  3.2.1 Experimental Setup.

Attack. We use PGD and FGSM for both white-box and black-box attacks. For
black-box setup, we use the same setup as previous RQ. For black-box scenario,
we use DyNNs as target model and SDNNs as surrogate model. In a white-box
setting, we attack the SDNN and evaluate on the performance of corresponding
DyNN.

Metric. We use the difference between the exit number selected by adversarial
input and the exit number selected by benign input. If the difference is
positive, then the latency of the adversarial sample is increased w.r.t benign
input.

####  3.2.2 Evaluation Results.

Figure 4 and Figure 11 (in Appendix) show the impact of adversarial examples
generated on SDNN on changing in exit number in DyNN in a white-box setting.
It can be observed that for the majority of the scenarios, accuracy-based
adversarial samples do not increase the computation significantly in the DyNN.
On average, FGSM-generated examples increase more computation in DyNNs than
PGD-generated examples. For CIFAR-10 data, for MobileNet and VGG-16 DyNN
models, 25%-37% of the FGSM generated examples could increase the number of
exits by more than one. Also, it can be noted that adversarial samples
generated on CIFAR-100 data is more likely to increase computation than
adversarial samples generated on CIFAR-10. Especially, more than 45% of the
FGSM samples generated on CIFAR-100 data can increase the number of exits by
more than one.

Figure 5 and Figure 12 (in Appendix) show the impact of adversarial examples
generated on SDNN on change in exits in DyNN in a black-box setting. It can be
noted that, for CIFAR-10 dataset, black-box attack can generate more
computation-increasing examples than in white-box attack. For ResNet and VGG
model, more than 40% PGD attack generated examples can increase the number of
exits by more than one. For all three models, 35% of the FGSM attack generated
examples can increase the number of exits by more than one. FGSM attack
generates more inputs that induces low confidence in early-exit layers than
PGD attack. For CIFAR-100 dataset, the increase of computation caused by
adversarial attacks is higher than that of CIFAR-10. As CIFAR-100 data uses
larger model, the robustness of the model is reduced. For CIFAR-100 dataset,
more than 40% examples generated through both the attack can increase the
number of exits by more than one. However, increasing the number of exits by
two or three exits does not decrease the efficiency in DyNNs significantly.

Finding 3: Accuracy-based adversarial samples do not decrease the efficiency
significantly in the DyNN. Finding 4: The adversarial examples, whose output
confidences are significantly lower, can perform better in terms of decreasing
the efficiency in DyNNs. Finding 5: Adversarial examples generated on a larger
model (w.r.t model parameters) is more likely to decrease efficiency in DyNNs.

(a) CIFAR-10 VGG

(b) CIFAR-10 MobileNet

(c) CIFAR-10 ResNet

Figure 4: Density plots of change in exit numbers because of PGD and FGSM
attack (For CIFAR-10 data). The x axis represents the change in exit number
while y axis represents the density.

(a) CIFAR-10 VGG

(b) CIFAR-10 MobileNet

(c) CIFAR-10 ResNet

Figure 5: Density plots of change in exit numbers because of PGD and FGSM
black-box attack (For CIFAR-10). The x axis represents the change in exit
number while y axis represents the density.

##  4 What design of DyNNs may impact the robustness against attack generated
on SDNNs?

In this section, we evaluate which architecture design choices (position of
early exits) in DyNNs may impact the robustness of early layers against the
adversarial inputs generated on SDNNs. In this RQ, we consider DyNNs as multi-
exit networks, where each exit will provide an output. For evaluation, we
assess if we attack the final exit (output of the static model), from which
early-exit layer the label modification begins. If the output label is not
modified in any earlier layer, then for that type of model, the robustness is
higher because the model can produce correct results at any layer. This RQ
will provide us an insight into which type of design choice may improve the
robustness of early layers.

###  4.1 Experimental Setup.

Attack Setup. We use same attack setup as previous research questions.
However, while we attack the DyNN, we do not consider one specific exit layer.
Instead of that, we consider each exit layer.

Metric. We use the early exit from which the label modification starts. For
example, there are $N$ exits. First, $N$th exit’s label is modified through
adversarial sample. If till $K$th exit the original prediction was same, then
we report $K+1$th exit in the experimentation.

###  4.2 Evaluation Results.

Figure 6 and Figure 9 (in Appendix) show probability density plot on which
exit the output label is changed using the white-box adversarial examples. It
can be observed that for all the model-dataset pairs, for more than 77% of the
examples, the label is modified in the first exit. For CIFAR-10 data, only for
MobileNet and ResNet models, the label is changed after the first exit for
more than 20% of the examples (using FGSM attack).

Figure 7 and Figure 10 (in Appendix) show probability density plot on which
exit the output label is changed using the black-box adversarial examples.
From results, we can see that the robustness of earlier exits is better
against black-box attack than white-box attack. For CIFAR-10 data, more than
45% of the both attack generated samples could not misclassify the first exit
for VGG-16 model. For CIFAR-100 data (larger model), robustness of early exit
layers is worse than that of CIFAR-10 data (smaller model). For CIFAR-100
data, for both attacks and three models, less than 30% of the adversarial
examples can not mis-classify the first exit.

For black-box attack scenario, VGG-16 model’s first layer on-average shows
better robustness against black-box attack. In the DyNN, the first exit of the
VGG model is placed only after second layer, while for others, more
computations are performed before using the first exit. Hence, having the
first exit in the early layers can increase the robustness of DyNN. Although
lower number of parameters would be used to predict output, but with VGG we
can notice that a significant number of inputs can be classified correctly
through first exit.

Finding 6: We find that having the first exit in the earlier layers can
increase the robustness of early exits of the DyNNs. Finding 7: Black-box
attack success rate is lesser than white-box attack success rate against
early-exit layers. Finding 8: For black-box scenario, early-exit layers are
more robust against adversarial examples generated on a smaller surrogate
model than adversarial examples generated on a larger surrogate model.

(a) CIFAR-10 VGG

(b) CIFAR-10 MobileNet

(c) CIFAR-10 ResNet

Figure 6: Density plots representing during which exit number output label is
changed because of PGD and FGSM attack (For CIFAR-10 data). The x axis
represents the exit number while y axis represents the density.

(a) CIFAR-10 VGG

(b) CIFAR-10 MobileNet

(c) CIFAR-10 ResNet

Figure 7: Density plots representing during which exit number output label is
changed because of PGD and FGSM black-box attack (For CIFAR-10 data). The x
axis represents the exit number while y axis represents the density.

##  5 What design of DyNNs may impact the robustness against attack on dynamic
mechanism?

In this section, we evaluate if specific examples can be generated only to
understand the additional attack surface introduced by the dynamic mechanism
in DyNNs and which design choices would be helpful in DyNNs to improve the
robustness against this specific attack. First, we aim to design an attack
such that the synthesized adversarial examples will not change the prediction
of the final SDNN label, but change the prediction of all the early exits.
This threat model is practical because the attack evades the existing
detection that relies on the final output of SDNN while the attacker creates a
situation where all the early exit networks do not provide correct prediction,
therefore decreasing the usability of DyNNs. However, this attack is also
challenging to be performed successfully because final layer output is
dependent on the earlier exit layers and it is challenging to impact all the
early exits’ prediction without modifying the final exit’s prediction.

Inputs : $x:Input$ $Image$

Outputs : $x^{\prime}:Perturbed$ $Image$

begin

Initialize($w$)

$T=number\\_of\\_iterations$

$iter\\_no=0$

while  _$iter\\_no <T$_ do

$x^{\prime}=\frac{\tanh(w)+1}{2}$

$output=model(x^{\prime})$

if  _${success}(output)$_ then

$return(x^{\prime})$

end if

$L=loss(x,w,c,\alpha)$

$L_{new},w=Optim(L,w)$

$iter\\_no++$

end while

$x^{\prime}=\frac{\tanh(w)+1}{2}$

end

Algorithm 1 Input generation using Early Attack

###  5.1 Problem Formulation

We propose a novel attack technique called Early Attack to evaluate the
effectiveness of the early layers of DyNNs. Let’s assume $f$ is an DyNN with
$N$ exits. Given an input $x$, the output softmax layer in an exit $i$ can be
defined as $y^{i}=f_{i}(x)$, where $i=1,2,3...N$. For synthesizing adversarial
examples, we have two main objectives. First, the initial prediction in the
$N^{th}$ exit (final layer) does not change. Let’s assume, the initial
prediction at final layer is $p$. Second, the prediction of all the other
exits should be different than $p$.

Based on the aforementioned objectives, we can propose an iterative
optimization procedure to optimise a loss function $L$. For each of the
objectives, one loss function component is formulized. For the first
objective, we propose loss function $L1=(-1*\sum_{j\neq
p}max(y^{N}_{p}-y^{N}_{j},0))$. In $L1$, we maximize the difference between
softmax value of label $p$ and other label’s softmax value, therefore the
prediction won’t get changed at the final exit. For the second objective, we
propose loss function $L2=(\sum_{i=1}^{N}y^{i}_{p})$. The $L2$ ensures that
for any other exit than the final exit, $p$’s softmax value would be
minimized. The final loss function $L=\alpha*L1+L2$. Here the $\alpha$ is a
user-defined variable that provides balance between two loss terms.

Finally, we need to ensure the added perturbation to generate adversarial
examples are limited, hence, the final optimization function would be,
$minimize(||\delta||+c\cdot L)$, where, $(x+\delta)\in[0,1]^{n}$. Here,
$\delta$ represents the added perturbation and $c$ is a user-defined variable
to provide weightage on a specific component. $c$ controls the magnitude of
generated perturbation ($||\delta_{i}||$), where a large $c$ makes the loss
function more dependant on the $L$.

This constrained optimization problem in $\delta$ can be converted into a non-
constrained optimization problem in $w$, where the relationship between
$\delta$ and $w$ is: $\delta=\frac{\tanh(w)+1}{2}-x$ The $tanh$ function would
ensure that the generated adversarial input values stay between 0 and 1. The
equivalent optimization problem in $w$ is:

$\displaystyle\underset{w}{\textrm{minimize}}\;\Bigg{|}\Bigg{|}\frac{\tanh(w)+1}{2}-x\Bigg{|}\Bigg{|}+c\cdot
L$ (1)

Algorithm 1 shows the optimization algorithm. The algorithm outputs the
adversarial input $x^{\prime}$ given a benign image $x$ as input. $w$ is
initialized to a random tensor that has equal shape as the input image. For
each iteration, the loss function of the attack is computed (at line 11).
Based on the back-propagated loss, the optimizer updates $w$ with its next
value. Once the iteration threshold ($T$) is reached or the attack is
successful, the algorithm computes and returns the adversarial input
$x^{\prime}$ (at Line 9 and 15).

###  5.2 Evaluation

####  5.2.1 Evaluation SetUp.

Baseline. We use PGD and FGSM attacks as baseline to modify the early exit
label.

Metric. We use attack success rate as metric in the evaluation. If the
adversarial input generated final layer output is same as the output generated
by benign input and all the other exit layers output is different than the
final output label, then we consider attack is successful for that particular
adversarial input.

Hyperparameters. We use $\alpha=\\{0.001,0.01,0.1,1,20,40\\}$ and $c=50$ as
hyperparameters.

####  5.2.2 Effectiveness

Table 1 shows the evaluation results of the attack success rate of Early
Attack and baseline techniques. Except two model-dataset pair, Early Attack’s
success rate is higher than 80% in all other scenarios. PGD and FGSM attacks
are unsuccessful because the loss function only considers early layer output
and does not consider final layer output. Although FGSM’s attack success rate
is higher than PGD’s success rate. Also, w.r.t best performing $\alpha$
values, $\alpha=1$ performs well for 70% of the model-dataset pairs.

We also analyze why early attack fails for VGG16 and MobileNet models for
CIFAR-10 data. First, for CIFAR-10 data, final fully connected layer has lower
number of parameters than the final layer for CIFAR-100 data. Hence, the
dependency between the final layer and earlier layers is higher for CIFAR-10.
However, for CIFAR-10 data, attack success rate is higher for ResNet model.
First we discuss the failed cases for VGG16 and MobileNet model, and then we
discuss why the attack succeeded for ResNet model.

For VGG16 model, the failed examples can be divided into two types. For the
first type, the final output label is changed with all the other early exit
layers. In the second type, first few layers get the output label mis-
classified, but along with the final layer, few previous layers also get the
output correctly classified. For MobileNet, all the layers except the final
and last to final layer get the output label mis-classified. For MobileNet, we
also find that the final two exits are separated by only two layers, which is
significantly lower than other model’s separation layers between final two
exits. Hence, the dependency between final two layers is higher.

For ResNet model, there are 56 layers divided into 27 blocks. The exits are
sparsely divided between these blocks. For Mobilenet and VGG models, the exit
distributions are less sparse. Hence, the dependency between exits is lower
for ResNet and because of that reason, we could successfully attack ResNet
model.

Table 1:  Attack accuracy percentage of Early Attack and the baseline techniques against different model and dataset, along with $\alpha$ value. Dataset | Model | Early Attack |  best $\alpha$ val | PGD | FGSM
---|---|---|---|---|---
CIFAR-10 | VGG | 35 | 1 | 0 | 0
MobileNet | 11 | 0.1 | 0 | 0
ResNet | 81 | 1 | 0 | 0
CIFAR-100 | VGG | 86 | 20 | 0 | 1
MobileNet | 97 | 1 | 0 | 0
ResNet | 96 | 1 | 0 | 2

####  5.2.3 Transferability

In this section, we discuss about the transferability of the Early Attack
examples. As Early Attack has two different components, instead of measuring
attack success rate directly, we measure two parameters $T1$ and $T2$. $T1$
represents the percentage of inputs for which the final output remains same as
the output generated by benign input. $T2$ represents from the examples
selected from $T1$, how many early exit layers on average are mis-classified.
Having both high $T1$ and $T2$ values ensures transferability.

Table 2: $T1$ and $T2$ values for measuring transferability between three models. TM represents Target Model and SM represents Surrogate Model. $T1$ presents the percentage of inputs for which the final output remains same as the output generated by benign input. $T2$ represents from the examples selected from $T1$, how many early exit layers on average is mis-classified. |  | CIFAR-10 | CIFAR-100
---|---|---|---
Type |  SM TM | VGG | MNet | RNet | VGG | MNet | RNet
$T1$ | VGG | – | 85% | 73% | – | 65% | 39%
MNet | 86% | – | 74% | 51% | – | 38%
RNet | 89% | 82% | – | 79% | 75% | –
$T2$ | VGG | – | 0.73 | 1.65 | – | 1.58 | 2.69
MNet | 1.06 | – | 2.3 | 1.52 | – | 3.26
RNet | 0.95 | 0.85 | – | 1.32 | 1.28 | –

We show the transferability results in Table 2. For CIFAR-10 data, the $T1$
values are high, but $T2$ values are significantly low except for MobileNet to
ResNet transferability. For CIFAR-100 data, $T2$ values are higher than of
CIFAR-10 data, however, $T1$ values are low. From the results, we can notice
that the generated examples either can keep the final layer label the same or
can change the early exit layer outputs. Our finding suggests that early
attack transferability is limited.

Finding 9: With increasing number of exits in DyNNs, the dependency between
multiple exits will increase. Hence, more exits in DyNNs can increase the
robustness against the Early Attack. Finding 10: Early Attack transferability
between DyNNs is not significant.

##  6 Conclusion

In this work 111https://github.com/anonymous2015258/Early_Attack/tree/main, we
discuss the robustness of including dynamic mechanism in DNN through three
research questions. We find out that DyNNs are more robust than SDNNs and also
efficient to generate adversarial examples. We also propose DyNN design
choices through final two RQs. Finally, we propose a novel attack to
understand additional attack space in DyNNs.

## References

  * [1] Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein.  Square attack: a query-efficient black-box adversarial attack via random search.  In European Conference on Computer Vision, pages 484–501. Springer, 2020.
  * [2] Nicholas Carlini and David Wagner.  Towards evaluating the robustness of neural networks.  In 2017 ieee symposium on security and privacy (sp), pages 39–57. Ieee, 2017.
  * [3] Simin Chen, Hanlin Chen, Mirazul Haque, Cong Liu, and Wei Yang.  The dark side of dynamic routing neural networks: Towards efficiency backdoor injection.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24585–24594, 2023.
  * [4] Simin Chen, Mirazul Haque, Cong Liu, and Wei Yang.  Deepperform: An efficient approach for performance testing of resource-constrained neural networks.  In Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering, pages 1–13, 2022.
  * [5] Simin Chen, Hamed Khanpour, Cong Liu, and Wei Yang.  Learning to reverse dnns from ai programs automatically.  arXiv preprint arXiv:2205.10364, 2022.
  * [6] Simin Chen, Cong Liu, Mirazul Haque, Zihe Song, and Wei Yang.  Nmtsloth: understanding and testing efficiency degradation of neural machine translation systems.  In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 1148–1160, 2022.
  * [7] Simin Chen, Zihe Song, Mirazul Haque, Cong Liu, and Wei Yang.  Nicgslowdown: Evaluating the efficiency robustness of neural image caption generation models.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15365–15374, 2022.
  * [8] Patryk Chrabaszcz, Ilya Loshchilov, and Frank Hutter.  A downsampled variant of imagenet as an alternative to the cifar datasets.  arXiv preprint arXiv:1707.08819, 2017.
  * [9] Francesco Croce and Matthias Hein.  Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks.  In International conference on machine learning, pages 2206–2216. PMLR, 2020.
  * [10] Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li.  Boosting adversarial attacks with momentum.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 9185–9193, 2018.
  * [11] Fernando A Fardo, Victor H Conforto, Francisco C de Oliveira, and Paulo S Rodrigues.  A formal evaluation of psnr as quality measurement parameter for image segmentation algorithms.  arXiv preprint arXiv:1605.07116, 2016.
  * [12] Michael Figurnov, Maxwell D Collins, Yukun Zhu, Li Zhang, Jonathan Huang, Dmitry Vetrov, and Ruslan Salakhutdinov.  Spatially Adaptive Computation Time for Residual Networks.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1039–1048, 2017.
  * [13] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy.  Explaining and Harnessing Adversarial Examples.  arXiv preprint arXiv:1412.6572, 2014.
  * [14] Alex Graves.  Adaptive Computation Time for Recurrent Neural Networks.  arXiv preprint arXiv:1603.08983, 2016.
  * [15] Mirazul Haque, Christof J Budnik, and Wei Yang.  Corrgan: Input transformation technique against natural corruptions.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 194–197, 2022.
  * [16] Mirazul Haque, Anki Chauhan, Cong Liu, and Wei Yang.  ILFO: Adversarial Attack on Adaptive Neural Networks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14264–14273, 2020.
  * [17] Mirazul Haque, Simin Chen, Wasif Arman Haque, Cong Liu, and Wei Yang.  Nodeattack: Adversarial attack on the energy consumption of neural odes.  2021\.
  * [18] Mirazul Haque, Rutvij Shah, Simin Chen, Berrak Şişman, Cong Liu, and Wei Yang.  Slothspeech: Denial-of-service attack against speech recognition models.  arXiv preprint arXiv:2306.00794, 2023.
  * [19] Mirazul Haque, Yaswanth Yadlapalli, Wei Yang, and Cong Liu.  Ereba: Black-box energy testing of adaptive neural networks.  arXiv preprint arXiv:2202.06084, 2022.
  * [20] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  arXiv preprint arXiv:1512.03385, 2015.
  * [21] Dan Hendrycks and Thomas Dietterich.  Benchmarking neural network robustness to common corruptions and perturbations.  arXiv preprint arXiv:1903.12261, 2019.
  * [22] Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam.  Mobilenets: Efficient Convolutional Neural Networks for Mobile Vision Applications.  arXiv preprint arXiv:1704.04861, 2017.
  * [23] Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin.  Black-box adversarial attacks with limited queries and information.  In International Conference on Machine Learning, pages 2137–2146. PMLR, 2018.
  * [24] Yigitcan Kaya, Sanghyun Hong, and Tudor Dumitras.  Shallow-deep networks: Understanding and mitigating network overthinking.  In International Conference on Machine Learning, pages 3301–3310. PMLR, 2019.
  * [25] Alex Krizhevsky, Geoffrey Hinton, et al.  Learning multiple layers of features from tiny images.  2009\.
  * [26] Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song.  Delving into transferable adversarial examples and black-box attacks.  arXiv preprint arXiv:1611.02770, 2016.
  * [27] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu.  Towards deep learning models resistant to adversarial attacks.  arXiv preprint arXiv:1706.06083, 2017.
  * [28] Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami.  Practical black-box attacks against machine learning.  In Proceedings of the 2017 ACM on Asia conference on computer and communications security, pages 506–519, 2017.
  * [29] Simone Scardapane, Michele Scarpiniti, Enzo Baccarelli, and Aurelio Uncini.  Why should we add early exits to neural networks?  Cognitive Computation, 12(5):954–966, 2020.
  * [30] Lea Schönherr, Katharina Kohls, Steffen Zeiler, Thorsten Holz, and Dorothea Kolossa.  Adversarial attacks against automatic speech recognition systems via psychoacoustic hiding.  arXiv preprint arXiv:1808.05665, 2018.
  * [31] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  arXiv preprint arXiv:1409.1556, 2014.
  * [32] Surat Teerapittayanon, Bradley McDanel, and Hsiang-Tsung Kung.  Branchynet: Fast Inference via Early Exiting from Deep Neural Networks.  In Proceedings of the International Conference on Pattern Recognition, pages 2464–2469, 2016.
  * [33] Surat Teerapittayanon, Bradley McDanel, and Hsiang-Tsung Kung.  Distributed deep neural networks over the cloud, the edge and end devices.  In 2017 IEEE 37th international conference on distributed computing systems (ICDCS), pages 328–339. IEEE, 2017.
  * [34] Xin Wang, Fisher Yu, Zi-Yi Dou, Trevor Darrell, and Joseph E Gonzalez.  Skipnet: Learning Dynamic Routing in Convolutional Networks.  In Proceedings of the European Conference on Computer Vision, pages 409–424, 2018.
  * [35] Ruoyu Wu, Taegyu Kim, Dave (Jing) Tian, Antonio Bianchi, and Dongyan Xu.  DnD: A Cross-Architecture deep neural network decompiler.  In 31st USENIX Security Symposium (USENIX Security 22), pages 2135–2152, Boston, MA, Aug. 2022. USENIX Association.
  * [36] Zuxuan Wu, Tushar Nagarajan, Abhishek Kumar, Steven Rennie, Larry S Davis, Kristen Grauman, and Rogerio Feris.  Blockdrop: Dynamic inference paths in residual networks.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8817–8826, 2018.
  * [37] Guanqun Yang, Mirazul Haque, Qiaochu Song, Wei Yang, and Xueqing Liu.  Testaug: A framework for augmenting capability-based nlp tests.  arXiv preprint arXiv:2210.08097, 2022.
  * [38] Wangchunshu Zhou, Canwen Xu, Tao Ge, Julian McAuley, Ke Xu, and Furu Wei.  Bert loses patience: Fast and robust inference with early exit.  Advances in Neural Information Processing Systems, 33:18330–18341, 2020.

##  Appendix A RQ1 Density Plots

The density plots in Figure 8 shows the probability density plots of exit
numbers in DyNN that are used to generate black-box adversarial inputs using
PGD and FGSM attacks.

(a) CIFAR-10 VGG

(b) CIFAR-10 MobileNet

(c) CIFAR-10 ResNet

(d) CIFAR-100 VGG

(e) CIFAR-100 MobileNet

(f) CIFAR-100 ResNet

Figure 8: Density plots of exit numbers in DyNN that are used to generate
black-box adversarial inputs using PGD and FGSM attacks. The x axis represents
the exit number while y axis represents the density.

(a) CIFAR-100 VGG

(b) CIFAR-100 MobileNet

(c) CIFAR-100 ResNet

Figure 9: Density plots representing during which exit number output label is
changed because of PGD and FGSM attack (For CIFAR-100 data). The x axis
represents the exit number while y axis represents the density.

(a) CIFAR-100 VGG

(b) CIFAR-100 MobileNet

(c) CIFAR-100 ResNet

Figure 10: Density plots representing during which exit number output label is
changed because of PGD and FGSM black-box attack (For CIFAR-100 data). The x
axis represents the exit number while y axis represents the density.

(a) CIFAR-100 VGG

(b) CIFAR-100 MobileNet

(c) CIFAR-100 ResNet

Figure 11: Density plots of change in exit numbers because of PGD and FGSM
attack (For CIFAR-100 data). The x axis represents the change in exit number
while y axis represents the density.

(a) CIFAR-100 VGG

(b) CIFAR-100 MobileNet

(c) CIFAR-100 ResNet

Figure 12: Density plots of change in exit numbers because of PGD and FGSM
black-box attack (For CIFAR-100 data). The x axis represents the change in
exit number while y axis represents the density.

##  Appendix B Early Attack Success Rate with Different $\alpha$ values

Table 3:  Attack accuracy percentage of Early Attack with different $\alpha$ values Dataset | Model | EA($\alpha$=0.001) | EA($\alpha$=0.01) | EA($\alpha$=0.1) | EA($\alpha$=1) | EA($\alpha$=20) | EA($\alpha$=40)
---|---|---|---|---|---|---|---
CIFAR-10 | VGG | 0 | 0 | 0 | 35 | 10 | 4
MobileNet | 0 | 0 | 11 | 4 | 0 | 0
ResNet | 7 | 32 | 73 | 81 | 49 | 32
CIFAR-100 | VGG | 1 | 5 | 48 | 82 | 86 | 70
MobileNet | 0 | 16 | 74 | 97 | 92 | 77
ResNet | 46 | 74 | 92 | 96 | 96 | 93

##  Appendix C Transferability

Table 3 shows the $T1$ and $T2$ values of all three models on two datasets.

##  Appendix D RQ1 results based on MI-FGSM attack

Here, through Figure 13, we show the adversarial transferability results
between DyNNs and SDNNs using MI-FGSM attack. These results again confirm that
adversarial examples from DyNN to SDNN are more transferable than adversarial
examples from SDNN to DyNN.

##  Appendix E Transferability experiments on MI-FGSM attack

Through Figure 13, we show the S2D and D2S transferability with MI-FGSM attack
[10]. The results confirm our claim that D2S transferability is higher than
S2D transferability.

(a) Target DyNN MI-FGSM

(b) Target SDNN MI-FGSM

Figure 13: Transferable Attack Success Rate for MI-FGSM attack

##  Appendix F Comparing different adversarial images

In this section, we show original images, adversarial images generated through
DyNNs and adversarial images generated through SDNNs through Figure 14,Figure
16 and Figure 15. We find that in terms of quality, images generated through
SDNNs (Average PSNR [11] = 23.20) are slightly better than images generated
through DyNNs (Average PSNR = 23.19).

Figure 14: Original Images

Figure 15: Adversarial Images generated on SDNNs

Figure 16: Adversarial Images generated on DyNNs

##  Appendix G RQ1 results based on Tiny Imagenet Images

Here, through Figure 17, we show the adversarial transferability results
between DyNNs and SDNNs for Tiny Imagenet [8] datasets. These images are
larger in size (64 $\times$ 64) than CIFAR images (size 32 $\times$ 32). These
results reconfirm that adversarial examples from DyNN to SDNN are more
transferable than adversarial examples from SDNN to DyNN, even if the input
feature space is larger. Although, we can notice a slight decrease in
transferability for both D2S and S2D.

(a) Target DyNN (Tiny Imagenet)

(b) Target SDNN (Tiny Imagenet)

Figure 17: Transferable Attack Success Rate for Tiny Imagenet Data