# How Electronic Dynamics with Pauli Exclusion Produces Fermi-Dirac Statistics

Triet S. Nguyen  Department of Chemistry and Biochemistry, University of Notre
Dame, Notre Dame, IN 46556  Ravindra Nanguneri  Department of Chemistry and
Biochemistry, University of Notre Dame, Notre Dame, IN 46556  John Parkhill
Department of Chemistry and Biochemistry, University of Notre Dame, Notre
Dame, IN 46556<EMAIL_ADDRESS>
###### Abstract

It is important that any dynamics method approaches the correct population
distribution at long times. In this paper, we derive a one-body reduced
density matrix dynamics for electrons in energetic contact with a bath. We
obtain a remarkable equation of motion which shows that in order to reach
equilibrium properly, rates of electron transitions depend on the density
matrix. Even though the bath drives the electrons towards a Boltzmann
distribution, hole blocking factors in our equation of motion cause the
electronic populations to relax to a Fermi-Dirac distribution. These factors
are an old concept, but we show how they can be derived with a combination of
time-dependent perturbation theory and the extended normal ordering of
Mukherjee and Kutzelnigg. The resulting non-equilibrium kinetic equations
generalize the usual Redfield theory to many-electron systems, while ensuring
that the orbital occupations remain between zero and one. In numerical
applications of our equations, we show that relaxation rates of molecules are
not constant because of the blocking effect. Other applications to model
atomic chains are also presented which highlight the importance of treating
both dephasing and relaxation. Finally we show how the bath localizes the
electron density matrix.

##  I Introduction

A perfect theory of electronic dynamics must relax towards the correct
equilibrium distribution of population at long times. However, satisfying this
condition is not trivial. The two most common approaches for simulating non-
equilibrium electronic dynamics in chemistry, Ehrenfest method and the surface
hopping approach, do not satisfy this condition exactly. In the case of
Ehrenfest dynamics, the violation of detailed balance is severeParandekar and
Tully (2006) and the dynamics can only be taken seriously at short times. In
the case of surface hopping, the violation is so small it is hard to
identifySchmidt, Parandekar, and Tully (2008). Both these methods are usually
formulated in a basis of distinguishable (Boltzmannian) electronic states.
Because the basis must be limited in practice, predictions in the Boltzmannian
picture depend on a choice of adiabatic or quasi-diabatic states. States with
fractional orbital occupations are also extremely expensive and difficult to
treat in a basis of collective electronic states. To address these problems,
we are pursuing relaxation dynamics based on the reduced electron density
matrices without wavefunctions.
In the one-electron picture, non-equilibrium Green’s function theoriesAyik
_et al._ (1998); Lake and Datta (1992); Semkat, Kremp, and Bonitz (2000);
Latini  _et al._ (2014) and Boltzmann transport equationsBurgio, Chomaz, and
Randrup (1991); Lacroix, Chomaz, and Ayik (1998) do satisfy detailed balance.
Master equations which exchange both electrons and energy between a system and
a bath of two-level Fermions have also been recently derived and approach a
Fermi-Dirac (FD) distributionSargsyan  _et al._ (2014); Suess, Strunz, and
Eisfeld (2014). However, for spectroscopy the most important bath is
undoubtedly the bosonic vibrations of the atomsKawai  _et al._ (2014). To our
knowledge, it has never been explicitly shown how a bosonic bath drives
electrons to their correct equilibrium as presented in this paper. Here we
show how a Pauli-blocking effect slows down relaxation, and dephasing of
electrons. The derivation that we use is general and can be applied to
electron correlation as well. We also use our new expressions to show that the
rate of relaxation for a particle-hole excitation is not constant because of
the Pauli-blocking effect.
Ordinary system-bath perturbation theory with a bath of bosons does not tend
towards FD populations. The equations do not consider exchange statistics so
they approach a canonical equilibrium instead. The goal of this paper is to
show how the derivation of ordinary system-bath perturbation theory naturally
leads to a FD equilibrium if expectation values are taken correctly with
respect to a mixed-state, Fermionic vacuum. The resulting kinetic equations
resemble Redfield theoryRedfield (1957) but also depend on the hole density.
The key to this paper is that the Fermionic expectation value is taken using
the extended normal ordering technique of Mukherjee and Kutzelnigg (MK)
Kutzelnigg and Mukherjee (1997) which doesn’t require a wavefunction and only
depends on reduced density matrices (RDMs). The exact equation of motion (EOM)
for many-Fermion RDMs, the BBGKY Akbari  _et al._ (2012), is also non-linear,
and we show that this non-linearity is also a requirement to obey Fermionic
detailed balance.
The application of Mukherjee and Kutzelnigg’s powerful technique to finite
temperature quantum mechanics is natural. They hinted at this application in
their classic paper, and Hirata and He recently invoked the technique to
correct perturbation theory in the thermodynamic limitHirata and He (2013).
Unfortunately, the generalized normal ordering technique is challenging to
derive, understand, and apply. We must refer the reader to the original papers
for details. A related generalized Wick’s theoremWick (1950) concept was
recently developed for Green’s function-based theoriesvan Leeuwen and
Stefanucci (2012).
For linear response away from a determinant, the blocking effect is
irrelevantTempel  _et al._ (2011) and so the literature is basically correct.
A paper using these new formulas in a useful all-electron dynamics code is an
immediate follow up. Electron transport methods based on the density matrix
will also benefit from the main results of this paperZelovich, Kronik, and Hod
(2014); Chen, Hansen, and Franco (2014); Subotnik  _et al._ (2009); Wang and
Thoss (2013). Electronic statistics can be easily enforced as long as kinetic
rates depend on the density matrix.

##  II Detailed balance

In the non-interacting case, the population of an electronic state at long
times should tend to the Fermi-Dirac (FD) distribution:

$\displaystyle\bar{n}_{i}=\frac{1}{1+e^{\beta(\epsilon_{i}-\mu)}}$ (1)

where $\beta=(k_{b}T)^{-1}$, $\epsilon_{i}$ is the energy of a one-electron
state, and $\mu$ the chemical potential satisfying $\sum_{i}\bar{n}_{i}=N$,
the total number of electrons. The bar indicates that this is an equilibrium
quantity, where the number of particles exchanged between any two states are
equal so the corresponding populations do not change (the condition of
detailed balance).
It is known that a kinetic equation can be easily modified by ’Pauli-blocking’
factors $(1-n_{i})$, to reach detailed balance with the FD distribution.
Consider a set of non-negative Markovian transition rates between states
$K_{ij}$ satisfying $\sum_{j}K_{ij}=1$, and canonical detailed balance
$e^{\beta(\epsilon_{j}-\epsilon_{i})}=K_{ij}/K_{ji}$. The blocking factors
close any channel which would disobey exclusion. Inserting FD
$\bar{n}_{i}^{\prime}s$ into a kinetic equation of this form:

$\displaystyle\frac{d}{dt}n_{i}=\sum_{j}K_{ij}(1-n_{i})n_{j}=\sum_{j}K_{ij}\eta_{i}n_{j}$
(2)

one sees the number of particles exchanged between any two states are equal,
meaning the FD distribution is stationary. We have introduced $\eta_{i}$, the
diagonal of the one-hole density matrix. Equation (2) is the form a kinetic
equation must take to satisfy Fermionic detailed balance. Exclusion factors
are an old concept attributed to LandauLandau and Lifshitz (1980), as an ad
hoc correction. They are also important for dephasing ratesCohen  _et al._
(2009). In the remainder of this paper, we derive an expression which reduces
to Eq. (2) with perturbation theory.

##  III Fermionic Master Equation

We begin our derivation with the following Hamiltonian:

$\displaystyle\hat{H}=\hat{H}_{el}+\hat{H}_{b}+\hat{V}=\sum_{i}\epsilon_{i}a^{i}a_{i}+\sum_{\alpha}\omega_{\alpha}b^{\alpha}b_{\alpha}+\sum_{i,j,\nu}V_{i,j}^{\nu}a^{i}a_{j}\hat{B}_{\nu}$
(3)

Where $\hat{B}_{\nu}$ is an coupling-weighted sum of boson operators attached
to $\nu$, defined below. To show that FD distribution is the exact
equilibrium, the electrons must be non-interacting, but the correction we
derive applies in the interacting case as well. We use a notation for second
quantization which follows Ref. Kutzelnigg and Mukherjee (1997) and summation
over repeated indices is implied. In the open-systems approach, the
Hamiltonian is put into this form after diagonalizing the electronic part from
some localized basis $|i\rangle=\sum_{j}\hat{U}_{i\nu}|\nu\rangle$ with
electronic coupling. In the original basis, which is the atomic basis in our
work, the electronic part of the system-bath coupling is usually taken to be
diagonal:

$\displaystyle\hat{V}=\sum_{\alpha}g_{\alpha,\nu}\omega_{\alpha}(b^{\alpha,\nu}+b_{\alpha,\nu})|\nu\rangle\langle\nu|=\hat{B}_{\nu}|\nu\rangle\langle\nu|$
(4)

but leads to off-diagonal coupling:
$V^{\nu}_{i,j}=\sum_{\nu}{U}_{i,\nu}{U}^{\dagger}_{j,\nu}$ in the system
energy eigenbasis used to propagate the dynamics. We discuss how Eq. (3) is
prepared atomistically in the Results section. The equilibrium Boson
expectation values taken in this work are denoted
$\langle\rangle_{\text{eq}}$, otherwise $\langle\rangle$ denotes a Fermionic
expectation value. No restriction is made on the nature of the Fermionic state
which is used to perform the expectation value. In particular, the formulas
are valid for a general mixed Fermionic state which is completely defined by
its RDMs and _no wavefunction is invoked or implied_. We also define a
projection operator $\mathcal{P}$, which replaces Bosonic operators with their
expectation values at equilibrium leaving the Fermion operators untouched,
e.g. $\mathcal{P}a^{a}a_{b}b^{\dagger}b=a^{a}a_{b}\langle
b^{\dagger}b\rangle_{\text{eq}}$, and its complement operator
$\mathcal{Q}=\hat{I}-\mathcal{P}$.
The one-electron reduced density matrix (1-RDM) is defined as
$\gamma^{a}_{b}=\langle a^{a}a_{b}\rangle=\langle a^{a}_{b}\rangle$. We assume
that an initial 1-RDM we would like to propagate is known, perhaps on the
basis of a stationary electronic structure calculation. In the notation of
this work which follows that of Ref. Kutzelnigg and Mukherjee (1997), $\gamma$
is not an operator. Rather it is the expectation value of a one-particle
operator. 1-RDMs are not equivalent to the tight-binding (TB) density matrices
assumed in a master equation. However, Fermionic _operators_ do satisfy
Heisenberg’s EOM like TB density matrices. The steps used to produce master
equations for TB operators generically apply to one-body Fermionic operators
as well. Thus until fermionic expectation values are taken, our derivation is
essentially the same as the textbook derivations of Redfield theory that can
be found in BreuerBreuer and Petruccione (2002) or NitzanNitzan (2006). Our
final expressions also reduce to the commonly used second-order TB theory in
the limit that Fermi-blocking factors are removed.
Working in the interaction picture with respect to the first two terms of
$\hat{H}$, where $\mathcal{L}(t^{\prime})=-i[\hat{V}(t^{\prime}),\cdot]$,
second-order time convolutionless perturbation theoryParkhill  _et al._
(2012); Shibata, Takahashi, and Hashitsume (1977); Breuer and Petruccione
(2002) produces this EOM for the tensor product of $a^{a}_{b}$ with an
equilibrium boson state:

$\displaystyle\frac{d}{dt}\mathcal{P}a^{a}_{b}(t)=\left(\mathcal{PL}(t)\mathcal{P}+\int_{0}^{t}dt^{\prime}\mathcal{PL}(t)\mathcal{Q}\mathcal{L}(t^{\prime})\mathcal{P}\right)a^{a}_{b}(t)+\mathcal{I}(t)$
(5)

Taking the Fermionic expectation value of Eq. 5 will eventually produce an EOM
for $\gamma$, because $\frac{\partial}{\partial
t}\gamma^{a}_{b}=\frac{\partial}{\partial t}\langle a^{a}_{b}\rangle$, but the
equilibrium Boson expectation values in $\mathcal{P}$ must also be
taken111Also note that an equation of motion for $\gamma$ alone is not enough
to exactly propagate an _interacting_ many-Fermion state which would require
equations for higher RDMs. We assume non-interacting electrons, but our
equations would be compatible with a time-dependent Hartree-Fock
approximation.. The first term on the right of Eq. (5) is zero because it is
sandwiched by $\mathcal{P}$ and the expectation value of odd-numbered boson
operators are zero at equilibrium. The last term is an inhomogeneity which
accounts for any initial correlations between electrons and bosons. Our
previous work explored this term in detail Parkhill  _et al._ (2012). The
effects on observables in that work were shown to be small relative to the
Markov approximation, and other papers document how inhomogeneous effects
decay Semin, Sinayskiy, and Petruccione (2012); Breuer and Petruccione (2002).
Because we are interested in the long-time detailed balance limit, we are well
motivated to neglect the inhomogeneityChaudhry and Gong (2013). This leaves
only the second term, reflecting the time-dependent correlation between the
system and the bath, which we denote as $\frac{d}{dt}(a^{a}_{b})^{(2)}$:

$\displaystyle\frac{d}{dt}(a^{a}_{b})^{(2)}(t)=(\frac{i}{\hbar})^{2}\big{\langle}\int_{0}^{t}dt^{\prime}\big{[}[a^{a}_{b}(t),V^{\nu}_{c,d}a^{c}_{d}(t^{\prime})\hat{B}_{\nu}(t^{\prime})],V^{\nu}_{e,f}a^{e}_{f}(t)\hat{B}_{\nu}(t)]\big{\rangle}_{\text{eq}}$
(6)

At this point we move back to the Schrodinger picture. We collect the time-
dependent factors in each term ($e^{i\omega_{cd}t^{\prime}},e^{i\omega_{ef}t}$
where $\hbar\omega_{ab}=\epsilon_{a}-\epsilon_{b}$) and interaction strength
scalars, $V^{\nu}_{a,b}$ together with the boson expectation value to focus on
the Fermionic expectation value. An EOM for $\gamma$ is obtained by taking the
Fermonic expectation value of both sides of this equation. On the left hand
side,
$\frac{d}{dt}e^{i\omega_{ab}t}a^{a}_{b}=i\omega_{ab}e^{i\omega_{ab}t}a^{a}_{b}+e^{i\omega_{ab}t}\frac{d}{dt}a^{a}_{b}$.
Dividing the common factor and taking the expectation value,
$i\omega_{ab}\langle a^{a}_{b}\rangle=i\omega_{ab}\gamma^{a}_{b}$ produces the
first term in equation 10.
Ordinarily, in order to take an expectation value of products of operators
like on the right hand side of Eq. 6, one needs a description of an electronic
state as excitations relative to a determinantal wavefunction. Instead we use
MK’s extended normal ordering which advantageously only depends on RDMs, but
requires too much algebra to publish in detail. We wrote a Mathematica program
to take the expectation value which is available from us upon request. This
double commutator also occurs in MK’s paperKutzelnigg and Mukherjee (1997) as
equation (48). To obtain our expression from theirs, we first delete terms in
which contain operators (whose expectation values are therefore zero), and
also make a simple substitution ($\delta^{a}_{b}=\eta^{a}_{b}+\gamma^{a}_{b}$,
with $\eta$ the hole density matrix and $\gamma$ the particle density matrix)
so that
$\delta^{a}_{b}\gamma^{c}_{d}-\delta^{c}_{d}\gamma^{a}_{b}=\eta^{a}_{b}\gamma^{c}_{d}-\eta^{c}_{d}\gamma^{a}_{b}$,
leaving four terms:

$\displaystyle\big{\langle}\big{[}[a^{a}_{b},a^{c}_{d}],a^{e}_{f}\big{]}\big{\rangle}=\delta^{c}_{b}(\gamma^{a}_{f}\eta^{e}_{d}-\eta^{a}_{f}\gamma^{e}_{d})+\delta^{a}_{d}(\gamma^{e}_{b}\eta^{c}_{f}-\eta^{e}_{b}\gamma^{c}_{f})$
(7)

This expression has two interesting consequences. First, it shows that the
expectation value of this double commutator does not depend on the higher-
order RDMs. Even if the reference had strong, two-particle correlations, or it
was a double-excitation that _does not_ affect this term222See MK’s paper for
details.. This result supports the possibility of an accurate dissipative
time-dependent Hartree-Fock equation since the dissipation of the one-body
density is not coupled to the two-body density _through the bath_.333The two-
body density does couple to the one-body density via the Coulomb interaction
in an atomistic Hamiltonian. Secondly, each factor in this EOM has a naturally
derived Pauli-blocking factor.
The remaining steps in our derivation are equivalent to the ordinary
derivation of a Redfield equation. If $\eta$ is replaced with the identity
matrix (blocking is shut off) even the result is equivalent to an ordinary
Redfield equation, and so readers can consult with one of the excellent
textbooksNitzan (2006). The first assumption is a Markov approximation
($\lim_{t\rightarrow\infty}$), in which the bath remains at Boson equilibrium.
This places a constraint on the boson correlation function:

$\displaystyle\int_{0}^{\infty}\big{\langle}\hat{B}_{\nu}(t)\hat{B}_{\nu}(0)\big{\rangle}_{eq}e^{i\omega_{ij}t}dt=\Gamma_{\nu}(\omega_{ij})=e^{\beta\hbar\omega_{ij}}\Gamma_{\nu}(\omega_{ji})$
(8)

Which leads to the canonical detailed balance relation in the Redfield
equation, but for this equation furnishes canonical K’s like Eq. (2). Putting
the result of the time integration together with the Fermionic expectation
value for each term we obtain:

$\displaystyle\frac{d}{dt}\big{\langle}(a^{a}_{b})^{(2)}\big{\rangle}(t)=(\frac{i}{\hbar})^{2}\Big{\\{}\Gamma_{\nu}(\omega_{cf})V^{\nu}_{ae}V^{\nu}_{fc}\gamma^{c}_{b}\eta^{e}_{f}+\big{(}\Gamma_{\nu}(\omega_{df})V^{\nu}_{be}V^{\nu}_{fd}\big{)}^{\dagger}\gamma^{a}_{d}\eta^{e}_{f}$
(9)
$\displaystyle-\Gamma_{\nu}(\omega_{ca})V^{\nu}_{de}V^{\nu}_{ac}\gamma^{c}_{d}\eta^{e}_{b}-\big{(}\Gamma_{\nu}(\omega_{db})V^{\nu}_{ce}V^{\nu}_{bd}\big{)}^{\dagger}\gamma^{c}_{d}\eta^{a}_{e}\Big{\\}}$

We note an interesting duality between the electron and the hole in this
expression. The third and fourth terms are particle relaxing, but their
blocking factors resemble hole-dephasing terms (they couple to only one index
of $\eta$), whereas the first and second are particle dephasing, but resemble
hole-relaxing terms.
A secular approximation must be made to reach an equation which preserves
trace, by deleting contributions which couple the diagonal and off-diagonal
elements of the $\gamma$. Besides the desired trace preservation, the secular
approximation can be motivated with a rotating wave argument, which is
identical to the Redfield case described in several previous publicationsVogt,
Jeske, and Cole (2013); Palmieri, Abramavicius, and Mukamel (2009). Unit-less
Kronecker deltas enforce this decoupling
$\text{S}_{a,b,c,d}=\delta_{ab}\delta_{cd}+\delta_{bc}\delta_{ad}(1-\delta_{ab}\delta_{cd})$.
The resulting electronic EOM is the key result of our paper:

$\displaystyle\dot{\gamma_{b}^{a}}=\frac{-i}{\hbar}\omega_{ab}\gamma_{b}^{a}+\frac{-1}{\hbar^{2}}\Big{\\{}\text{S}_{a,f,f,c}\Gamma_{\nu}(\omega_{cf})V^{\nu}_{ae}V^{\nu}_{fc}\gamma^{c}_{b}\eta^{e}_{f}+$
(10)
$\displaystyle\text{S}_{b,f,f,d}\big{(}\Gamma_{\nu}(\omega_{df})V^{\nu}_{be}V^{\nu}_{fd}\big{)}^{\dagger}\gamma^{a}_{d}\eta^{e}_{f}-\text{S}_{d,b,a,c}\Gamma_{\nu}(\omega_{ca})V^{\nu}_{de}V^{\nu}_{ac}\gamma^{c}_{d}\eta^{e}_{b}-$
$\displaystyle\text{S}_{c,a,b,d}\big{(}\Gamma_{\nu}(\omega_{db})V^{\nu}_{ce}V^{\nu}_{bd}\big{)}^{\dagger}\gamma^{c}_{d}\eta^{a}_{e}\Big{\\}}$

which is similar to the form of a secular Redfield equation but with blocking
factors. We propagate this non-linear equation using a straightforwards Runge-
Kutta-Fehlberg adaptive integrator. $\eta$ is not propagated separately, it is
simply always set equal to it’s definition in terms of $\gamma$. The equation
rigorously preserves the trace of the 1-RDM and equilibrates to FD
distribution. In the case that the density starts without coherence, $\eta$ is
diagonal and the equation takes the form of Eq. (2). We will now discuss the
importance of these blocking factors in the evolution of populations and
coherences using calculations on model systems.

##  IV Results

First, we will demonstrate that propagation of Eq. (III) reproduces FD
statistics. To produce a model with the form of Eq. (3), we consider a Huckel
model of atomic sites. Each site has an energy $E_{\nu}=1.0$ au and is coupled
to its nearest neighbors with a coupling strength $\hat{V}^{e}_{\nu,\nu}$. The
energy of each site fluctuates due to the system-bath coupling, and the
spectrum of these fluctuations is determined by a spectral density for this
site
$J_{\nu}(\omega)=\sum_{\alpha}\hbar^{2}\omega_{\alpha}^{2}g_{\nu,\alpha}^{2}\delta(\omega-\omega_{\alpha})$
of the Drude-Lorentz form. We have previously shown a method to obtain these
spectral densities atomistically from limited amounts of molecular dynamics,
but in this work we assume a strong, high-frequency bath444Four Drude-Lorentz
functions are chosen with widths, amplitudes and central frequencies of the
following form (au): {(0.0001, 0.00001, 0.0001), (0.001, 0.0017, 0.0017),
(0.013, 0.023, 0.027), (0.01, 0.017, 0.017)} to test positivity and rapidly
achieve relaxation. After assigning a spectral density and coupling in the
real-space $\nu$ basis, the electronic Hamiltonian is diagonalized to recover
a Hamiltonian of the form (3). Some of the dynamics is performed at a very
high temperature so that the exact reproduction of FD-statistics is made
clear. However, blocking has important effects even at room temperature; we
show some dynamics results at 300K to this effect. In all cases, we half-fill
the chain with electrons, and the chemical potential of the system is constant
as per Eq. (3).

Figure 1: A linear tight binding chain with 12 sites, half-filled with 6
electrons, initially at infinite temperature, in which populations of all
states are equal, approaches a FD distribution at 500 K in the long time limit
($\hat{V}^{e}_{\nu,\nu}=0.005$ au). Solid lines denote the energy eigenstate
populations as a function of time; the dashed lines are the exact asymptotes,
the Fermi-Dirac population of each eigenstate given the energy and chemical
potential.

The electronic dynamics resulting from Eq. (III) preserves the trace of the
electronic density matrix and produces a FD distribution at long times.
Experiments with several different coherent and incoherent initial conditions
and a very strong bath all equilibrate to a FD distribution (Fig. 1).
Solutions of the ordinary secular Redfield equation are exponential with time
constants determined by the eigenvalues of $K_{ij}$; however, our trajectories
with Fermi blocking do not take the same form. There are direct implications
for the kinetics of excited state relaxation, and the interpretation of
transient absorption spectra which are ordinarily fit to a superposition of
decaying exponentials. Because of Fermi-blocking, particle-hole excitations
and especially multiple particle-hole excitations do not relax with simple
exponential kinetics. We define a ground-state probability based on the
overlap of the T=0 K ground state and the density during the dynamics:
$P_{g}(t)=\prod_{i}(1-|n_{i}(t)-\Theta(\epsilon_{i}-\mu)|)$, where $\Theta$ is
the step function. Here we simulate an 8-site chain at T=300 K, with otherwise
unchanged parameters. We initialize the density in a particle-hole excited
state by unfilling HOMO-2 and filling HOMO+2 and monitor $P_{g}(t)$ to observe
the time dependence of the relaxation rates (Fig. 2).

Figure 2: An 8-site, 4-electron model is initialized in a particle-hole state
(HOMO-2 $\rightarrow$LUMO-2), and allowed to relax at 300 K
($\hat{V}^{e}_{\nu,\nu}=0.05$ au). (a) Energy eigenstate populations as
functions of time. (b) The rate (right scale) at which the model returns to
the many-electron ground state (population on the left scale) is not constant
as a function of time due to blocking effects.

For single particle-hole states, the rate of relaxation begins rapid as
electrons fill in holes (Fig. 2) and then adopts a slower ’blocked’ and
essentially exponential relaxation rate dominated by LUMO$\rightarrow$HOMO
relaxation at long times. The effective rate only varies by roughly a factor
of ten because of the blocking effect. Consequently, it would be difficult to
separate from other possible rate fluctuations in ultrafast spectroscopic
experiments although we predict that it exists.
Off-diagonal elements of a density matrix are called coherences, and their
decay is called dephasing. As has been recently reported by other authors, the
dephasing rate of Fermions is considerably different from what ordinary
Redfield theory would predictCohen  _et al._ (2009). In distinguishable models
of electronic relaxation, coherence tends to be an ambiguous quantity because
it is dependent on the chosen basis. In both the energy eigenbasis, and the
position eigenbasis coherence has important and measurable implications. In
the former, coherence should be zero in the thermodynamic limit. When they are
nonzero, energy-basis coherences oscillate. Our kinetic model does completely
decay these coherences and also shows why an atomistic and improvable model
for their decay is so important. If we initialize electrons on the left of our
chain to reach detailed balance, they naturally flow to the right (Fig. 3).
However if only populations are allowed to relax, even though the populations
of energy eigenstates rapidly reach a perfect FD distribution, the undecayed
coherences lead to unphysical oscillatory currents in the system.

Figure 3: The atom-basis populations of two trajectories at 5000 K
($\hat{V}^{e}_{\nu,\nu}=0.05$ au) (a) without dephasing and (b) with
dephasing, showing the importance of dephasing in electronic dynamics. In both
cases, the initial condition is half-filling with all electrons forced to the
left of the chain. The populations in the energy eigenbasis of these
trajectories are rigorously FD at final time (the bottom of the plot). Without
dephasing (a) electron currents do not decay, whereas with all the terms in
Eq. (III) (b) they rapidly decay and all electronic motion is halted by
dissipation to the bath after the initial relaxation. Dephasing is as
important to accurate electronic dynamics as population kinetics.

In the position eigenbasis, coherences determine the measurable quantum
delocalization of particles. Real-space coherence also limits the efficiency
of electronic structure theory, since the cost of calculating exchange energy
is completely determined by how slowly the real-space density matrix decays
away from the diagonal. With our new theory, we can predict how dynamics and
finite temperature effects can lead to the long-range decay of the off-
diagonal elements of the electronic density matrix (Fig. 4). This locality can
potentially be exploited to make electronic structure models more efficient.

Figure 4: A 32-atom chain is initialized into its ground state
($\hat{V}^{e}_{\nu,\nu}=0.05$ au), and allowed to warm in contact with a 5000
K bath. $-1/\text{Log}\\{\gamma(x,x^{\prime})\\}$ at two different times are
plotted at the same scale to make the coherence decay visible.

Although kinetic equation (2) is non-linear in the density matrix, in our
experience it faithfully maps any density matrix to FD equilibrium at long
times, as it should since FD density is a known solution of Eq. (III). We have
not proven that FD is the _only possible solution_ , but we strongly suspect
it is.

##  V Discussion and Conclusions

Fermionic relaxation rates must depend nonlinearly on the density matrix to be
correct at long times. Consequently, relaxation rates for Fermionic states are
not constant as a function of time, and tend to slow down over the course of
non-radiative relaxation. This effect is extremely unclear in a Boltzmannian
’exciton’ model. Hole blocking makes rates of electronic relaxation and
dephasing slower than the corresponding rates for distinguishable particles.
These are second-order equations. The rates of transition they predict will
become inaccurate in the strong coupling limit, even though we obtain FD with
very strong couplings. The main feature of these equations is that they are
systematically improvable. One can easily ’Fermi-block’ surface hopping rates
and perform surface hopping for individual electrons. The results of this
paper suggest that with Fermi blocking surface hopping for electrons would,
for all intents and purposes, tend to FD. However, that electronic surface
hopping would beg for a systematically improvable path to dephasing rates,
like its Boltzmann counterpart.
We have found that the MK normal-ordering is an especially useful technique
for dynamics, allowing us to work around the impurity of the state. We used it
to provide a promising EOM for electrons and simulate non-equilibrium vibronic
relaxation. As in our previous work, the perturbation theory could also be
used to develop electronic equations of motion that treat electron correlation
effects, although that will require an automated version of ENO algebra. This
paper opens the door to answering whether interacting equilibrium differs
meaningfully from the FD equilibrium in real materials, and how that
equilibrium is approached. We are currently extending this work towards higher
electronic density matrices.
It is significant that these equations approach Fermionic detailed balance
naturally, as opposed to enforcing detailed balance in an ad hoc way. In a
related vein, sometimes Pauli-blocking factors are taken to be
$\bar{\eta}_{i}$ instead of $\eta_{i}(t)$, which is correct at equilibrium.
However this work suggests that using the equilibrium value is only
appropriate in the long time limit. Applications of this work to predict rates
of non-radiative relaxation in molecules is an active pursuit in our
laboratory.

##  VI Acknowledgements

We thank The University of Notre Dame’s College of Science and Department of
Chemistry and Biochemistry for generous start-up funding and Honeywell
Corporation. T. S. N. gratefully acknowledges the support of the NSF Graduate
Research Fellowship under grant 1313583. We acknowledge valuable conversations
with Liviu Nicolaescu (Notre Dame), Geoff Thompson (Iowa State), Jarrod
McClean, Samuel Blau, and Thomas Markovich (Harvard).

## References

  * Parandekar and Tully (2006) P. V. Parandekar and J. C. Tully, Journal of Chemical Theory and Computation 2, 229 (2006).
  * Schmidt, Parandekar, and Tully (2008) J. R. Schmidt, P. V. Parandekar, and J. C. Tully, The Journal of Chemical Physics 129, 044104 (2008).
  * Ayik  _et al._ (1998) S. Ayik, O. Yilmaz, A. Gokalp, and P. Schuck, Physical Review C 58, 1594 (1998).
  * Lake and Datta (1992) R. Lake and S. Datta, Physical Review B 46, 4757 (1992).
  * Semkat, Kremp, and Bonitz (2000) D. Semkat, D. Kremp, and M. Bonitz, Journal of Mathematical Physics 41, 7458 (2000).
  * Latini  _et al._ (2014) S. Latini, E. Perfetto, A.-M. Uimonen, R. van Leeuwen, and G. Stefanucci, Physical Review B 89, 075306 (2014).
  * Burgio, Chomaz, and Randrup (1991) G. Burgio, P. Chomaz, and J. Randrup, Nuclear Physics A 529, 157 (1991).
  * Lacroix, Chomaz, and Ayik (1998) D. Lacroix, P. Chomaz, and S. Ayik, Physical Review C 58, 2154 (1998).
  * Sargsyan  _et al._ (2014) V. V. Sargsyan, G. G. Adamian, N. V. Antonenko, and D. Lacroix, Physical Review A 90, 022123 (2014).
  * Suess, Strunz, and Eisfeld (2014) D. Suess, W. Strunz, and A. Eisfeld, arXiv:1410.0304 [quant-ph] (2014).
  * Kawai  _et al._ (2014) H. Kawai, K. Yamashita, E. Cannuccia, and A. Marini, Physical Review B 89, 085202 (2014).
  * Redfield (1957) A. G. Redfield, IBM Journal of Research and Development 1, 19 (1957).
  * Kutzelnigg and Mukherjee (1997) W. Kutzelnigg and D. Mukherjee, The Journal of Chemical Physics 107, 432 (1997).
  * Akbari  _et al._ (2012) A. Akbari, M. J. Hashemi, A. Rubio, R. M. Nieminen, and R. van Leeuwen, Phys. Rev. B 85, 235121 (2012).
  * Hirata and He (2013) S. Hirata and X. He, The Journal of Chemical Physics 138, 204112 (2013).
  * Wick (1950) G. C. Wick, Physical Review 80, 268 (1950).
  * van Leeuwen and Stefanucci (2012) R. van Leeuwen and G. Stefanucci, Physical Review B 85, 115119 (2012).
  * Tempel  _et al._ (2011) D. G. Tempel, M. A. Watson, R. Olivares-Amaya, and A. Aspuru-Guzik, The Journal of Chemical Physics 134, 074116 (2011).
  * Zelovich, Kronik, and Hod (2014) T. Zelovich, L. Kronik, and O. Hod, Journal of Chemical Theory and Computation 10, 2927 (2014).
  * Chen, Hansen, and Franco (2014) L. Chen, T. Hansen, and I. Franco, The Journal of Physical Chemistry C 118, 20009 (2014).
  * Subotnik  _et al._ (2009) J. E. Subotnik, T. Hansen, M. A. Ratner, and A. Nitzan, The Journal of Chemical Physics 130, 144105 (2009).
  * Wang and Thoss (2013) H. Wang and M. Thoss, The Journal of Physical Chemistry A 117, 7431 (2013).
  * Landau and Lifshitz (1980) L. Landau and E. M. Lifshitz,  _Statistical Physics, Part I_ (Pergamon Press, 1980).
  * Cohen  _et al._ (2009) D. Cohen, J. von Delft, F. Marquardt, and Y. Imry, Physical Review B 80, 245410 (2009).
  * Breuer and Petruccione (2002) H.-P. Breuer and F. Petruccione,  _The Theory of Open Quantum Systems_ (Oxford University Press, 2002).
  * Nitzan (2006) A. Nitzan,  _Chemical Dynamics in Condensed Phases: Relaxation, Transfer and Reactions in Condensed Molecular Systems_ (Oxford University Press, 2006).
  * Parkhill  _et al._ (2012) J. A. Parkhill, T. Markovich, D. G. Tempel, and A. Aspuru-Guzik, The Journal of Chemical Physics 137, 22A547 (2012).
  * Shibata, Takahashi, and Hashitsume (1977) F. Shibata, Y. Takahashi, and N. Hashitsume, Journal of Statistical Physics 17, 171 (1977).
  * Note (1) Also note that an equation of motion for $\gamma$ alone is not enough to exactly propagate an _interacting_ many-Fermion state which would require equations for higher RDMs. We assume non-interacting electrons, but our equations would be compatible with a time-dependent Hartree-Fock approximation.
  * Semin, Sinayskiy, and Petruccione (2012) V. Semin, I. Sinayskiy, and F. Petruccione, Physical Review A 86, 062114 (2012).
  * Chaudhry and Gong (2013) A. Z. Chaudhry and J. Gong, Physical Review A 88, 052107 (2013).
  * Note (2) See MK’s paper for details.
  * Note (3) The two-body density does couple to the one-body density via the Coulomb interaction in an atomistic Hamiltonian.
  * Vogt, Jeske, and Cole (2013) N. Vogt, J. Jeske, and J. H. Cole, Physical Review B 88, 174514 (2013).
  * Palmieri, Abramavicius, and Mukamel (2009) B. Palmieri, D. Abramavicius, and S. Mukamel, The Journal of Chemical Physics 130, 204512 (2009).
  * Note (4) Four Drude-Lorentz functions are chosen with widths, amplitudes and central frequencies of the following form (au): {(0.0001, 0.00001, 0.0001), (0.001, 0.0017, 0.0017), (0.013, 0.023, 0.027), (0.01, 0.017, 0.017)}.


# RanStop: A Hardware-assisted Runtime Crypto-Ransomware Detection Technique

Nitin Pundir<EMAIL_ADDRESS>University of Florida, US ,  Mark
Tehranipoor<EMAIL_ADDRESS>University of Florida, US and  Fahim
Rahman<EMAIL_ADDRESS>University of Florida, US

(2020)

###### Abstract.

Among many prevailing malware, crypto-ransomware poses a significant threat as
it financially extorts affected users by creating denial of access via
unauthorized encryption of their documents as well as holding their documents
hostage and financially extorting them. This results in millions of dollars of
annual losses of the infected victims worldwide. Multiple variants of
ransomware are growing in number with capabilities of evasion from many anti-
viruses and software-only malware detection schemes that rely on static
execution signatures. In this paper, we propose a hardware-assisted scheme,
called RanStop, for early detection of crypto-ransomware infection in
commodity processors. Specifically, RanStop leverages the information of
hardware performance counters (HPCs) embedded in the performance monitoring
unit (PMU) in modern processors to observe micro-architectural event sets and
detects known and unknown crypto-ransomware variants. In this paper, we train
a recurrent neural network-based machine learning architecture (RNN) using
long short-term memory (LSTM) model for analyzing micro-architectural events
in the hardware domain when executing multiple variants of ransomware as well
as benign programs (goodware). We create timeseries to develop intrinsic
statistical features using the information of related HPCs and improve the
detection accuracy of RanStop and reduce noise by via LSTM and global average
pooling (GAP). As an early detection scheme, RanStop can accurately and
quickly identify ransomware within 2$\mathrm{ms}$ from the start of the
program execution by analyzing HPC information collected for 20 timestamps
each 100$\mathrm{\SIUnitSymbolMicro s}$ apart. This detection time is too
early for a ransomware to make any significant damage, if none. Moreover,
validation against benign programs with behavioral (sub-routine-centric)
similarity with that of a crypto-ransomware shows that RanStop can detect
ransomware with an average of $\mathrm{9}\mathrm{7}\mathrm{\char 37\relax}$
accuracy for fifty random trials.

Ransomware, Hardware Performance Counters (HPC), neural networks, LSTM,
runtime detection.

††copyright: none††doi: ††conference: arXiv; ; US††journalyear: 2020

##  1\. Introduction

Security vulnerabilities in modern computing systems for stand-alone and
networked applications have given rise to numerous cyber attacks and malware
that can cause privacy breach and data loss, compromise critical
infrastructures and national security, cause financial damage, and more.
McAfee quarterly threat report (Jan-Mar 2017) reveals that 176 new cyber
threats are being emerging every minute (mcafee2017threat, ). Among such cyber
threats and attacks, ransomware has gained much attention due to its malicious
nature and subsequent exploitation, loss of data, and financial loss
(hutcherson2018ransomware, ; vamosi2018wannacry, ; o2012ransomware, ;
liptak_2017, ).

Ransomware, comprising of the words ‘ransom’ and ‘malware’, is a class of
malware which asks for ransom/money from the victim via anonymous payment
mechanisms by holding the system/files as hostage, in exchange for restoring
the hijacked functionality. Since the majority of the ransomware use
cryptographic encryption-decryption processes and key (password) exchange
protocols for ‘locking’ (and later releasing) the data stored in the infected
system, this class is also known as crypto-ransomware (as oppose to the
locker-ransomware that causes a denial-of-service on the system to restrict
operational access by the user) (scaife2016cryptolock, ;
sgandurra2016automated, ). Although existed for more than a decade, a massive
rise of crypto-ransomware is observed in the past few years as more devices
with inadequate protection are being connected to the global network and its
multiple variants with stealthy nature are being fast spread out too often. So
far, the family of crypto-ransomware has caused substantial global financial
loss and is continuing to cost tens of millions of dollars in consumer losses
annually (vamosi2018wannacry, ; scaife2016cryptolock, ). The actual cost
incurred by the ransomware is believed to be much higher since numerous
incidents remain unreported from the victims. In addition to financial loss,
crypto-ransomware attacks, especially on business organizations, law
enforcement agencies, and even entertainment groups, have resulted into loss
of critical data, private information, valuable documents, work hours, and
services (arnold2014tennessee, ; hutcherson2018ransomware, ; liptak_2017, ;
radiohead2019ok, ). Ransomware is considered as the main reason of 39% of
malware-related data breaches in 2018, according to Verizon’s annual report
(alison2018ransomware, ). As such, crypto-ransomware represents a major threat
to all classes of users in the modern world.

In the past few years, researchers have put much emphasis on detecting and
preventing ransomware. The majority of the proposed techniques are based on
static software-centric set-up which typically utilizes static methods such as
signature (template) matching in the program control flow, searching for
dominant features of malicious ransomware-like activities, and monitoring
high-level (software) execution, API/system calls, or data to detect
potentially anomalous behavior (sgandurra2016automated, ;
scaife2016cryptolock, ; kharraz2016unveil, ; andronio2015heldroid, ).
Unfortunately, such static and software-only schemes, including commercial
anti-viruses, often fail to provide a comprehensive security against ever-
growing ransomware. Two prime examples of ransomware evading existing
software-only mechanisms are – (1) the latest ransomware attacks on the city
of Atlanta’s online systems shutting down the activities for more than six
days (hutcherson2018ransomware, ); and (2) the WannaCry malware infecting many
business organizations in over 150 different countries(vamosi2018wannacry, ;
liptak_2017, ). In both cases, systems under attack presumably had adequate
protection against potential crypto-ransomware and denial of service attacks.
Software-only schemes suffer from the following intrinsic limitations:

  * •

There are numerous different ransomware with distinct and obscure control flow
signatures that can evade static signature-matching anti-malware schemes.

  * •

Software-based techniques often require binary signature for each variant of
the ransomware (or, in general, malware). It imposes huge overhead on the
database, i.e., the size of anti-malware updates, with ever-growing
polymorphic and metamorphic variants of a class of malware
(bishop2005introduction, ).

  * •

Even in the case of a successful detection, several existing schemes are too
late (in time) at detecting a ransomware such that the victim system and
important files (or directories) may already be maliciously corrupted
(encrypted) and locked, where the possibility of the recovery is extremely
rare.

  * •

The static signature mapping can produce a high rate of false decisions (false
positive/negative) which pose critical impact on the smooth operation of the
system.

Therefore, it is apparent that the existing software-only techniques are not
adequate to thwart crypto-ransomware attacks. To address the existing
challenges and limitations, we propose a hardware-assisted runtime crypto-
ransomware detection scheme, called RanStop, for commodity computing
platforms. RanStop leverages existing hardware performance counters (HPCs) in
the performance monitoring units (PMUs), commonly available in recent
generation processors (intel64manual, ; armA9manual, ), for runtime event-
monitoring at the micro-architectural level and utilizes state-of-the-art
machine learning technique to develop an advanced predictive model for an
early and accurate detection of known and unknown variants of the crypto-
ransomware family.

The main motivation behind developing our hardware-assisted crypto-ransomware
detection technique, RanStop, comes from the fact that HPCs can collect multi-
dimensional hardware event-traces and micro-architectural information during
program execution with zero hardware modification. Although originally
designed for performance monitoring, HPCs (and PMUs) can be intelligently used
for security by analyzing whether a runtime event profile is malicious in
nature. Monitoring the information of hardware micro-architectural events and
developing ML-based machine learning predictive models provide us advantages
over high-level software features as follows.

  1. (1)

Being an integrated part of the hardware, HPCs operate transparently to any
software running on the processor and collect targeted micro-architectural
data irrespective to the program execution mode. Therefore, software-
obfuscated or stealthy ransomware cannot evade them.

  2. (2)

It offers multi-dimensional information from a large set of micro-
architectural sources. Therefore, acquired information can be utilized for
multi-modal analysis techniques such as statistical analysis and machine
learning techniques.

  3. (3)

Being an integrated part of the hardware, it is able to collect information
significantly faster (often in $\mathrm{\SIUnitSymbolMicro s}$ ranges) than
any software-centric trace acquisition approach.

  4. (4)

Developed ML model can be retrained with additional dataset for emerging
ransomwares with little to no modification in the ML framework. Also, the
developed ML model size is significantly smaller compared to the static
signature-based database (alam2019ratafia, ).

Figure 1. Primary workflow for RanStop.

The nature of cryptro-ransomware family itself shows certain dependency on
various sub-routines (e.g., encryption and data movement) that can be common
in regular benign programs (e.g., disk encryption goodwares) and cannot be
accurately distinguished only via static signature analysis or control/data
flow graph (CDFG) (moussaileb2018ransomware, ; alam2019ratafia, ). Our
proposed RanStop technique can identify intrinsic dissimilarities between
crypto-ransomware and goodware from the hardware activity signatures. RanStop,
as shown in Figure 1, first collects runtime micro-architectural event
signatures for all HPC groups for both the ransomware and goodware. Using the
collected hardware information, a machine learning model is then developed
which takes a small runtime data and can detect ransomware with high accuracy.
The contributions of our work are as follows.

  1. (1)

RanStop offers an accurate and noise-free collection of hardware-domain micro-
architectural activities, e.g., branch prediction success/miss,
$\mathrm{L}\mathrm{2}$ cache access, etc. for ongoing program (command)
executions for detecting malicious event traces. This dynamic approach is
applicable for both known and unknown Ransomware with minimal runtime and zero
hardware overhead.

  2. (2)

RanStop utilizes the state-of-the-art machine learning (ML) techniques with
intelligent feature selection scheme to accurately detect ransomware. We carry
out extensive analysis for $\mathrm{8}\mathrm{0}$ crypto-ransomware using
recurrent neural network (RNN) architecture using long short-term memory
(LSTM) and global average pooling methods (hochreiter1997long, ;
gers2002learning, ). Our technique provides $\mathrm{9}\mathrm{7}\%$
prediction accuracy on average for selected hardware performance groups.

  3. (3)

RanStop offers significantly early-detection for ransomware by analyzing the
micro-architectural data collected for 20 timestamps each
$\mathrm{1}\mathrm{0}\mathrm{0}\mathrm{\SIUnitSymbolMicro}\mathrm{s}$ apart
from the start of the execution (2$\mathrm{ms}$ in total). This allows to stop
the malicious execution at a very early stage and protects the system and
files long before undergoing significant, if not none, damage and data loss.

The rest of the paper is organized as follows: Section 2 highlights prior work
and relevant concepts. Section 3 describes the workflow of our proposed
RanStop technique in detail. Section 4 provides the experimental results and
analysis. Finally, we conclude our work in Section 5.

##  2\. Preliminaries

###  2.1. Ransomware Detection: Prior Work

The generic characteristic of the crypto-ransomware family is to maliciously
search and encrypt users’ files and provide decryption key only in exchange of
a ransom. Very often crypto-ransomware performs targeted attack on specific
extensions, e.g., .doc, .jpg, .pdf, or the directories that are more probable
to contain important user data, e.g., My Documents folder in Windows OS.
Prevalent locker/crypto-ransomware detection techniques heavily rely on
inspecting program execution, monitoring system/API calls, signature/template
matching in program control flow, and data monitoring for critical file
modification. For example, Kharraz et al.(kharraz2015cutting, ;
kharraz2016unveil, ) analyzed various ransomware families and showed that the
majority of them implement naive locking or encryption techniques. Therefore,
they proposed a scheme called UNVEIL to detect ransomware infection by using
automatically generated artificial user environments where these artificial
environments are created for any suspicious activities and constantly
monitored. However, this approach is not suitable for lightweight application
since deployment and monitoring of artificial user puts significant overhead
on execution time and resources without considering kernel applications.
Andronio et al. (andronio2015heldroid, ) proposed an android-based locker-
ransomware detection scheme, called HelDroid, using common ransomware
characteristics, such as functions to lock screen for android devices. This
approach is very much platform-oriented and may not be extendable to commodity
processing units for accurate detection. Scaife et al. (scaife2016cryptolock,
) proposed an early-warning scheme by analyzing variable type changes,
similarity measurements, and entropy of user data. This technique, by nature,
requires significant data analysis and is not suitable for early detection.
Sgandurra et al. (sgandurra2016automated, ) presented a dynamic analysis of
ransomware for higher detection accuracy using signature matching and
monitoring dominant features such as API calls. However, such dominant
features can be hidden via obfuscated sub-routines in the malware. Moussaileb
et al. (moussaileb2018ransomware, ) presented a machine learning-based
ransomware detection technique by monitoring file system traversal using decoy
folders, with the observation that the majority of ransomware start their
encryption process from the root of the hard disk. However, this technique is
prone to high false decision in case any goodware mimics the ransomware’s
traversal behavior or show no traversal signature at all.

As we see, these techniques utilize software-centric features to distinguish
ransomware from goodware. In addition to partial coverage, these approaches
also suffer from different challenges, e.g., program and memory overhead for
creating virtual users and environments (kharraz2016unveil, ), latency and
computational overhead for user storage data analysis (spisak2016hardware, ;
sgandurra2016automated, ), etc. In contrast, our proposed technique solely
relies on hardware-level micro-architectural information that remains
unaffected in case of program obfuscation, stealthy execution, and infection
strength, and is readily available in modern processors requiring zero
hardware overhead.

###  2.2. Micro-architectural Event Monitoring for Malware Detection

Hardware-based micro-architectural event monitoring offers a fine-grain
filtering for individual executions, can collect multi-dimensional
information, and provides a faster data collection with respect to the
software-only monitoring schemes. For example, one or more HPCs in the PMU can
sample how many times a pre-defined event (enabled by the associated
architecture), such as cache misses, occurs during the program runtime to
evaluate the performance of the system under test. PMUs in ARM and Intel x86
architectures can be accessed and controlled via lightweight software modules
such as likwid and perf tools (likwidgithub, ; perfgithub, ).

Although, the primary motivation behind having performance monitors in
hardware was to aid software developers by providing real-time feedback to
diagnose bugs or identify bottlenecks in the software for the given the
hardware platform, such hardware insights can aid for malware and anomaly
detection as well. However, micro-architectural characteristics of both
goodware and malware programs can be noisy due to the diffusion of multiple
program executions within a given time window. Therefore, it is extremely
difficult to characterize and distinguish a malicious program just by simple
observation of execution traces. Tang et al. (tang2014unsupervised, ) proposed
anomaly-based malware detection using HPC data. Wang et al. (wang2015confirm,
) proposed a low-cost validation tool, namely ConFirm, to detect the malicious
modifications in the firmware of embedded systems by creating internal check-
points to monitor HPC data. Malone et al. (malone2011hardware, ) analyzed
static and dynamic program modifications to detect malicious firmware by
modeling the architectural characteristics of benign programs using linear
regression. Most of these approaches considered various classes of malware and
rootkits into one class of malicious program and, therefore, combined generic
signature traces without emphasizing the intrinsic nature (and subsequent
micro-architectural activities) of the malware itself. As a result, these
techniques require a large amount of data for analysis, produces relatively
large false decisions, and are not suitable for early detection schemes
focusing specific families of malware, such as crypto-ransomware.

Very recently, Alam et al. (alam2019ratafia, ) presented a scheme, named
Ratafia, that leveraged HPC-provided micro-architectural event traces for
runtime detection of ransomware. This technique utilized the fast-Fourier
transform of event traces to identify prominent features for generating a one-
class classifier and designed a watchdog program with LSTM-based autoencoders
for anomaly detection in program execution trace. However, the technique was
developed and validated only for a handful of ransomware variants and do not
provide any scalability information for such.

Our proposed technique, on the other hand, is extensively analyzed for a large
number of ransomware and goodware for different scenario and provides
successful detection with low false decisions. Our technique requires only 2ms
of execution traces for correctly classifying a ransomware versus a goodware.

###  2.3. Security Enhancement via Advanced Machine Learning Techniques

One major obstacle for HPC-assisted malware detection is that the same micro-
architectural event can occur in a similar manner (i.e., frequency count and
event profile) during a benign (valid) operation. Therefore, it leads to
possible false detection. However, carefully constructed machine-learning (ML)
techniques can learn and differentiate such events to identify anomaly with a
higher confidence (demme2013feasibility, ). Two fundamental requirements for
deploying hardware-assisted anti-malware techniques are: (1) selecting high-
fidelity micro-architectural features and events, and (2) choosing efficient
machine learning techniques for classification.

In this work, we collect micro-architectural event traces from selected HPCs
in a timeseries fashion. We use the recurrent neural network with long short-
term memory (LSTM) architecture (hochreiter1997long, ; gers2002learning, ;
alam2019ratafia, ). LSTM is widely popular in timeseries analysis, especially
in audio-visual domain, since it helps maintaining a constant error for
recurrent networks and continues to learn over multiple timestamps in a series
fashion. The decisions that the LSTM generates depend on the current input,
previous output, and previous memory of the LSTM cell iteslf. The LSTM cells
are usually outside the general flow of the recurrent network and can filter
the signals they receive with their own set of weights. Alam et al.
(alam2019ratafia, ) used LSTM-based auto-encoder for anomaly detection in
event traces. In contrast, our work implements a RNN-based binary classifier
where a LSTM network is followed by Global Average Pooling (GAP) Layers to
reduce any overfitting in the model. Details of the adopted ML architecture is
discussed in Section 3.2.

##  3\. RanStop: A Hardware-assisted Runtime Crypto-ransomware Detector

In this section, we describe our proposed technique, RanStop, for the runtime
detection of crypto-ransomware via micro-architectural event monitoring using
HPCs. We have built our framework utilizing the key observations presented in
Demme et al. (demme2013feasibility, ):

  1. (1)

The semantics of a program (goodware or ransomware) do not change
significantly over different variants of similar functionality and class.

  2. (2)

While accomplishing a particular task (benign or malicious), there exist
subtasks that cannot be radically modified and should exhibit similar micro-
architectural footprints.

Figure 2. HPC data collection scheme for ransomware and goodware. Figure 3.
LSTM-based model generation for crypto-ransomware detection.

This observation exhibits the potential of building a hardware-assisted
crypto-ransomware detector for early recognition by analyzing micro-
architectural events at runtime. Even though the micro-architectural
information collected via HPCs are noisy; it can enable identifying (or,
separating, at least) crypto-ransomware from the benign activities if chosen
correctly with proper optimization. This is possible because there exists a
significant amount of similar semantic characteristics among multiple variants
of crypto-ransomware due to the similarity of their attack behavior
(moussaileb2018ransomware, ). HPCs, being oblivious to the undergoing program,
are able to collect this multi-dimensional signature that can be put under
further scrutiny via machine learning for ransomware versus goodware
identification.

Figure 1 shows the high-level workflow of our porposed RanStop technique. It
consists of following major steps: (1) program database creation; (2) micro-
architectural event monitoring and data collection in a timeseries fashion;
(3) LSTM-based predictive model generation (training); and, finally, (4)
testing, validation, and runtime detection. We discuss the details of each
step in the following subsections.

###  3.1. Program Database Creation

The very first step of RanStop is creating a database of benign (goodware)
programs and publicly available variants of crypto-ransomware, as shown in
Figure 1. This program database is fed to RanStop framework for subsequent
micro-architectural data collection. The platform, source, and size of the
program database is discussed in details in Sections 4.1-2.

We note that each ransomware executable was manually tested to make sure it
did not throw any runtime error; i.e., the ransomeware executables were
compatible with the execution environment and had access to necessary
resources similar to any real-life infection. Similarly, the benign program
database were also tested in a similar fashion so the the acquired HPC dataset
were not corrupted due incompatibility.

One key point for program database creation is that the goodware database
should contain different families of benign programs with various workload.
Especially, one should also consider computationally intensive programs (e.g.,
disk encryption programs such as VeraCrypt (veracrypt2018, )) that perform
legit but similar operations with respect to that of a crypto-ransomware. The
motivation behind is to offer similar semantic characteristics to different
sub-routines of the crypto-ransomware as well as generic user specific benign
programs. Additionally, the non-encryption benign binaries provide resemblance
to silent crypto-ransomware which does not start execution at the very first
moment of infection but resort to stealthy operation in the background to
other legit programs.

The idea behind choosing ransomware and goodware with similar characteristics
is to make sure that the proposed framework is capable of identifying even the
smallest differences and is not over/under-fitted due to noise
(witten2016data, ). For example, as experimented in Alam et al.
(alam2019ratafia, ), a one-class classifier trained with random benign
programs may tend to separate crypto-ransomware more accurately from a text
editor program; but may not distinguish from a disc encryption or file zipping
program. Therefore, we adopt a well-balanced database with significant number
and variants of ransomware and goodware. This well-balanced training scheme
allows to reduce false positive and false negative by the classifier. Note
that the RanStop framework is readily scalable to a larger dataset, as we
discuss in Section 4, and it allows the user to re-train (update) the initial
model for finer detection with emerging threats.

###  3.2. Micro-architectural Event Monitoring and Data Collection

In this work, we have developed our framework for an experimental Linux OS
setup, whereas a majority of the real-life crypto-ransomware are designed for
Windows OS. Therefore, we execute all programs (both ransomware and goodware)
in a Windows OS virtual machine hosted by the experimental Linux system. (For
detailed configuration of the experimental platform, please see Section 4.1.)
This approach is taken because – (1) it averts the risk of cryto-ransomware
encrypting the collected HPC data which is stored in a separate
administrative-privileged directory; (2) the Linux system provided
inhospitable environment in case any ransomware binary manages to escape the
Windows VM, so that the rest of the networked systems (if any) in the
experimental setup is unaffected.

The crypto-ransomware and benign programs from the database are executed in a
random fashion to monitor and collect micro-architectural information from the
processor using HPCs. We use the open-source tool likwid (likwidgithub, ) for
capturing hardware data from embedded HPCs. To make sure that the virtual
machine offers the same workload signature with or without infection, we
collected the HPC data in the following fashion, as shown in Figure 2:

  * •

The Windows VM is hosted and run with a complete library of programs to
replicate a real-life workload.

  * •

The program under test (ransomware or goodware) is pinned to run inside the VM
with no thread/resource limitations.

  * •

$\mathrm{L}\mathrm{i}\mathrm{k}\mathrm{w}\mathrm{i}\mathrm{d}$ is used to
collect and store timestamp data from all the CPU cores in the
$\mathrm{L}\mathrm{i}\mathrm{n}\mathrm{u}\mathrm{x}$ host machine.

  * •

Once the targeted timeseries data is collected (e.g., by completion of the
program or timeout); The VM is destroyed along with its virtual storage
completely wiped to reduce any residual noise.

  * •

A new VM replaces the old one (e.g., corrupted one, if infected by ransomware
while collecting ransomware data) with a backup storage image having the same
state as prior to running the program.

  * •

Multiple iterations are performed to collect all possible micro-architectural
events with randomized execution order, so that there exists no systematic
data and memory correlation, irrespective to ransomware or goodware execution.

The collected micro-architectural events (and respective values) are then
categorized into different performance groups on the basis of event context,
such as BRANCH, L2_DATA, ICACHE, etc., along with associated event counts and
metrics, as shown in Table 1. These associated events are given input to the
next stage LSTM network for ML-based predictive model building. More details
on these hardware features with appropriate optimizers and event counts are
discussed in Section 3.4 and Section 4.3.

Table 1. Performance Event Groups and Associated Metrics for Data Collection via Likwid. No | Group Name | Metrics
---|---|---
1 | BRANCH | Branch rate
Branch misprediction rate
Branch misprediction ratio
Instructions per branch
2 | CLOCK | Uncore Clock [MHz]
3 |  CYCLE ACTIVITY | Cycles without execution [%]
Cycles with stalls due to L1D [%]
Cycles with stalls due to L2 [%]
| Cycles w/o execution due to memory [%]
---
4 | DATA | Load to store ratio
5 | FLOPS_DP | DP MFLOP/s
AVX DP MFLOP/s
Packed MUOPS/s
Scalar MUOPS/s
Vectorization ratio
6 | ICACHE | L1I request rate
L1I miss rate
L1I miss ratio
L1I stall rate
7 | L2_DATA | L2D load bandwidth [MBytes/s]
L2D load data volume [GBytes]
L2D evict bandwidth [MBytes/s]
L2D evict data volume [GBytes]
L2 bandwidth [MBytes/s]
L2 data volume [GBytes]
8 | L2_CACHE | L2 request rate
L2 miss rate
L2 miss ratio
9 | L3_DATA: | L3 load bandwidth [MBytes/s]
L3 load data volume [GBytes]
L3 evict bandwidth [MBytes/s]
L3 evict data volume [GBytes]
L3 bandwidth [MBytes/s]
L3 data volume [GBytes]
10 | L3_CACHE | L3 request rate
L3 miss rate
L3 miss ratio
11 | TLB_DATA | L1 DTLB load misses
L1 DTLB load miss rate
L1 DTLB load miss duration [Cyc]
L1 DTLB store misses
L1 DTLB store miss rate
L1 DTLB store miss duration [Cyc]
12 | TLB_INSTR | L1 ITLB misses
L1 ITLB miss rate
L1 ITLB miss duration [Cyc]
13 | UOPS | Issued UOPs
Executed UOPs
Retired UOPs
14 | UOPS_EXEC | Used cycles ratio [%]
Unused cycles ratio [%]
Avg stall duration [cycles]
15 | UOPS_ISSUE | Used cycles ratio [%]
Unused cycles ratio [%]
Avg stall duration [cycles]
16 | UOPS_RETIRE | Used cycles ratio [%]
Unused cycles ratio [%]
Avg stall duration [cycles]

###  3.3. LSTM-based Predictive Model Generation (training and validation)

For generating a ML-based predictive model, that will be used for detecting
ransomware via runtime event monitoring, we utilize LSTM-based recurrent
neural network (hochreiter1997long, ; gers2002learning, ) for performing a
timeseries classification of the micro-architectural event signatures. As
shown in Figure 3, the collected hardware event values are provided as the
input features to the LSTM layer of the ML architecture. The reason for using
this specific neural model is already discussed in Section 2.3. A standard
LSTM cell can remember values over time interval making it best candidate to
classifying timeseries data of micro-architectural events from the process
execution.

After the LSTM layer, a global average pooling (GPA) layer is used to reduce
the intrinsic training features created by the LSTM layer. This can
significantly improve the model accuracy and prevent over-fitting of the
tensor. For our specific application, it reduces the spatial dimension of a
three dimension model to one dimension as necessary for the following layer.
For example, a tensor of dimension
$\mathrm{a}\times\mathrm{b}\times\mathrm{c}$ is reduced to dimension of
$\mathrm{1}\times\mathrm{1}\times\mathrm{d}$ after the GAP layer. The produced
features at this stage is then fed as input to the fully connected layer of
the neural network architecture for generating the binary classification model
(i.e., goodware versus ransomware).

###  3.4. Model Generation, Validation, and Detection

The final steps of the proposed RanStop is to generation of the predictive
model based on the given dataset, and validation and deployment of the model
for runtime detection. It should be noted that the impacts of different
optimizers and the micro-architectural performance groups collected in
previous steps are not all same for detecting potential crypto-ransomware with
high accuracy (see Section 4). Also, since the number of hardware performance
counters are limited on any system, the real time detection program (watchdog)
can only be trained to work for certain performance groups and may not swap
between monitors to monitor different set of data too often.

##  4\. Experimental Results

###  4.1. Experimental Platform

The system used as the experimental platform was based on Intel Xeon
CPU-E3-1225 Coffeelake processor with a maximum operating frequency of
$3.30\text{\,}\mathrm{GHz}$ and 32GB RAM. It was a single socket quad-core
processor with L1, L2, and L3 Cache of
$\mathrm{3}\mathrm{2}\mathrm{k}\mathrm{B}$,
$\mathrm{2}\mathrm{5}\mathrm{6}\mathrm{k}\mathrm{B}$, and
$\mathrm{8}\mathrm{M}\mathrm{B}$, respectively. The operating system was
Ubuntu 16.04 LTS and likwid-4.3.2 (likwidgithub, ) was used for HPC event
collection.

###  4.2. Program Database Creation

For this work, we considered 80 crypto-ransomware executables and 76 benign
(goodware) programs all of which had a execution time of at least
2$\mathrm{ms}$ or more. Irrespective to total runtime, we collected data for
the first 2$\mathrm{ms}$ only because our primary objective is an early
detection. The crypto-ransomware database was collected from VirusShare
(virusshare, ) which comprised of Windows executable (.exe) files. For the
benign programs, we used combination of encryption algorithms provided by the
OpenSSL (openssl, ) and collection of random C programs from Github
(awesome-c, ; ckatas, ). The benign encryption binaries are used to encrypt a
local directory from govdocs1 (govdocs1, ).

###  4.3. Micro-architectural Event Capture

Figure 4 depicts sample hardware events collected for goodware and crypto-
ransomware for different performance group. Here, HPC information was
collected for 20 timestamps, each being 100$\mathrm{\SIUnitSymbolMicro s}$
apart. Our objective was to accurately detect ransomware within the
2$\mathrm{ms}$ execution threshold; and as we will see in Section 4.4, we were
successful to correctly classify ransomware from a goodware just utilizing
hardware information of these 20 timestamps. As one can see, this detection
time is too early for a ransomware to make any significant damage, if none.

We also saw that the timeseries data differences between the two classes were
not necessarily significantly large to readily distinguish between ransomware
versus goodware. Additionally, the differences (or similarities) at some
timestamps might have occured due to system noise and additional runtime
overhead. Therefore, it was necessary that the developed detection scheme
ccould reduce any noise and optimized the intrinsic features to accurately
identify ransomware threats.

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

(i)

(j)

(k)

(l)

(m)

(n)

(o)

(p)

(q)

(r)

(s)

(t)

Figure 4. Distribution of example micro-architectural events among benign and
ransomwares for different performance groups. X-axis shows 20 timestamps from
the start of the execution, each timestamp are 100$\mathrm{\SIUnitSymbolMicro
s}$ apart. Y-axis shows respective micro-architectural event count in the
embedded hardware performance counter.

It should be noted that each performance group can collect $\mathrm{4}$ or
fewer micro-architectural events due to hardware limitations. Because, for our
experimental system, only four general-purpose HPCs are available in each core
when hyperthreading is enabled (intel64manual, ). In addition to micro-
architectural event count, likwid readily provides scalar information based on
different performance metrics as shown in Table 1. For the ease of analysis,
we consider available pre-processed metric information for feature selection,
training, and testing in the subsequent steps, rather than naively using raw
hardware event counts which are often noisy and require data pre-processing
such as scaling and alignment (tang2014unsupervised, ; demme2013feasibility,
).

Table 2. Accuracy with 70% Training Dataset | Adadelta | Adamax | RMSprop | SGD
---|---|---|---|---
BRANCH | 77.41% | 78.69% | 76.73% | 53.95%
CLOCK | 77.02% | 80.74% | 76.73% | 52.37%
| CYCLE_ACTIVITY
---
77.32% | 80.42% | 77.75% | 54.59%
DATA | 79.07% | 80.01% | 78.56% | 53.86%
FLOPS_DP | 85.67% | 86.90% | 88.89% | 58.75%
ICACHE | 83.54% | 89.02% | 85.78% | 55.44%
L2_DATA | 82.51% | 86.00% | 80.04% | 53.14%
L2_CACHE | 80.31% | 85.50% | 82.27% | 52.63%
L3_DATA | 82.85% | 82.73% | 84.30% | 51.61%
L3_CACHE | 84.68% | 85.40% | 83.27% | 53.61%
TLB_DATA | 96.34% | 95.32% | 95.36% | 52.80%
TLB_INSTR | 48.90% | 48.90% | 48.90% | 48.90%
UOPS | 75.28% | 78.86% | 73.53% | 52.12%
UOPS_EXEC | 76.90% | 78.00% | 76.77% | 55.01%
UOPS_ISSUE | 77.46% | 78.05% | 78.17% | 54.03%
UOPS_RETIRE | 78.31% | 77.54% | 77.96% | 53.39%
Table 3. Accuracy with 80% Training Dataset | Adadelta | Adamax | RMSprop | SGD
---|---|---|---|---
BRANCH | 74.88% | 79.37% | 77.43% | 50.50%
CLOCK | 79.50% | 86.06% | 79.37% | 51.87%
| CYCLE_ACTIVITY
---
77.75% | 82.81% | 78.11% | 53.69%
DATA | 77.12% | 80.24% | 77.61% | 53.12%
FLOPS_DP | 87.00% | 87.07% | 90.51% | 58.94%
ICACHE | 83.12% | 87.38% | 87.38% | 51.06%
L2_DATA | 81.43% | 87.63% | 81.88% | 51.63%
L2_CACHE | 82.49% | 86.50% | 81.12% | 52.68%
L3_DATA | 82.05% | 87.00% | 85.82% | 52.81%
L3_CACHE | 84.87% | 87.38% | 85.06% | 53.44%
TLB_DATA | 96.57% | 95.52% | 96.45% | 52.37%
TLB_INSTR | 50.00% | 50.00% | 50.00% | 50.00%
UOPS | 72.88% | 77.68% | 73.75% | 51.06%
UOPS_EXEC | 78.75% | 77.68% | 76.19% | 52.12%
UOPS_ISSUE | 76.75% | 79.93% | 77.49% | 53.31%
UOPS_RETIRE | 77.06% | 78.81% | 76.93% | 51.94%
Table 4. Accuracy with 90% Training Dataset | Adadelta | Adamax | RMSprop | SGD
---|---|---|---|---
BRANCH | 76.00% | 76.51% | 78.12% | 51.50%
CLOCK | 77.49% | 82.50% | 77.49% | 51.12%
| CYCLE_ACTIVITY
---
79.24% | 78.62% | 79.76% | 53.37%
DATA | 76.38% | 80.50% | 76.25% | 53.50%
FLOPS_DP | 85.50% | 85.99% | 90.88% | 61.00%
ICACHE | 84.75% | 85.62% | 84.13% | 50.75%
L2_DATA | 81.38% | 87.88% | 83.75% | 50.62%
L2_CACHE | 80.12% | 86.50% | 82.00% | 53.62%
L3_DATA | 85.00% | 83.50% | 85.88% | 50.88%
L3_CACHE | 87.14% | 87.87% | 85.88% | 52.38%
TLB_DATA | 97.26% | 96.52% | 98.26% | 51.00%
TLB_INSTR | 50.00% | 50.00% | 50.00% | 50.00%
UOPS | 72.14% | 76.25% | 75.37% | 51.37%
UOPS_EXEC | 77.37% | 78.74% | 78.12% | 51.88%
UOPS_ISSUE | 78.37% | 78.86% | 75.87% | 53.25%
UOPS_RETIRE | 78.87% | 76.50% | 76.87% | 51.37%

###  4.4. Performance Analysis of the ML Classifier

As discussed in Section 3.2 and Section 4.3, we develop the predictive ML
model by training with timeseries dataset of individual performance groups and
associated metrics (see Table 1) for given ransomware and goodware database.
Note that selecting such groups and features depend on multiple factors: (1)
inherent properties of the ML technique that utilizes such features to perform
binary classification and (2) the program behavior that is running on the
system (performing extensive encryption versus simple output printing).

At first, we used Keras python library (chollet2015keras, ) to implement the
neural network under training. Equal distribution of benign and ransomware was
maintained in the training dataset to prevent inclination of ML towards a
specific dataset. The training was done on four different optimizers belonging
to different classes, i.e., SGD, Adamax, Adadelta, and RMSprop, to calibrate
network weights based on error for reducing the validation loss
(kerasoptimizer, ). We used $\mathrm{2}\mathrm{5}\%$ of the training dataset
for validation after each epoch to efficiently calibrate the loss function of
the model. Also, to reduce any bias in the model due to misfitting, the
accuracy analysis was performed over 50 iterations, where each run contained
randomly shuffled executables and trained for 1000 epochs.

For an in-depth analysis of the RanStop technique, we analyzed the accuracy of
the predictive model where it was developed using different sizes of training
dataset, namely 70% (Table 2), 80% (Table 3), and 90% (Table 4) with
previously mentioned optimizers. Here, each value represents the fraction of
the total dataset that was used for training. The remainders of the dataset,
i.e., 30%, 20%, and 10% for respective cases, were used for testing. For each
table, the detection accuracy (averaged over 50 iterations) is listed with the
HPC groups (row) and optimizers (column). As seen from Tables 2, 3, and 4, the
Adadelta performed the best for all HPC groups in minimizing the over fitting.
On the other hand SGD performed the worst. A detailed scrutiny suggested that
the SGD overfitted the model due to the lack of data endpoints. The rest three
of the optimizers performed quite similar to each other. On the other hand,
the most prominent micro-architectural event group was TLB_DATA which provided
the best detection rate, whereas TLB_INSTR provided the worst outcome. The
results show that even with 70% training dataset, Ranstop was able to achieve
as high as $\mathrm{9}\mathrm{6}\%$ accuracy by correctly identifying benign
versus crypto-ransomware. And the accuracy goes to as high as
$\mathrm{9}\mathrm{7}\%$ while the training dataset contains
$\mathrm{9}\mathrm{0}\%$ of the total timeseries data. We also note that the
programs (ransomware/benign) used for testing the model were not any part of
the training dataset, as mentioned previously. Therefore, this supervised
classifier is fully compatible for detecting unknown ransomware, i.e.,
emerging variants with no (or, very limited, if needed at all) retraining.

False Negative Rate and False Positive Rate are also calculated using the
equation 1. We consider both false negatives and false positives as major
drawbacks for any ransomware (or malware, in general) detection technique,
since false positives, i.e., true benign programs deemed as ransomware, cause
inconvenience and probable denial of service, whereas false negatives, i.e.
true ransomware detected as benign program, can cause catastrophic damage to
the system.

(1) $FNRate=\frac{FN}{FN+TP},~{}FPRate=\frac{FP}{FP+TN}$

Tables 5 and 6 show the results for statistical metrics (True Positive, True
Negative, False Positive, and False Negative) for two of the best performing
optimizers, i.e. Adadelta and RMSprop. The results are again averaged over 50
iterations for each performance group to remove any bias or residual count.
The results show that if we remove the outlier case of TLB_INSTR, the false
negative (identifying crypto-ransomware as benign) rate is less then
$\mathrm{1}\%$ for many of the performance groups. The result also shows the
false positive (identifying benign as ransomware) is little high that can be
concluded to the fact that many micro-architectural activities of benign
programs may resemble that of a crypto-ransomware. We expect that the false
positive will significantly reduce with the increase in the dataset size and
diversity as a future work.

Table 5. Statistics for Adadelta with 70% training data | TP | TN | FP | FN
---|---|---|---|---
BRANCH | 49.11% | 29.57% | 19.36% | 1.96%
CLOCK | 48.09% | 32.64% | 16.30% | 2.98%
CYCLE_ACTIVITY | 50.51% | 31.62% | 19.45% | 0.55%
DATA | 48.09% | 31.91% | 17.02% | 2.98%
FLOPS_DP | 47.49% | 39.40% | 9.53% | 3.57%
ICACHE | 48.21% | 40.81% | 8.13% | 2.85%
L2_DATA | 47.87% | 38.13% | 10.81% | 3.19%
L2_CACHE | 50.09% | 35.40% | 13.53% | 0.98%
L3_DATA | 48.00% | 34.72% | 14.21% | 3.06%
L3_CACHE | 46.00% | 39.40% | 9.53% | 5.06%
TLB_DATA | 49.57% | 45.74% | 3.19% | 1.49%
TLB_INSTR | 0.00% | 48.94% | 0.00% | 51.06%
UOPS | 49.02% | 29.83% | 19.11% | 2.04%
UOPS_EXEC | 50.89% | 27.11% | 21.83% | 0.17%
UOPS_ISSUE | 49.83% | 28.21% | 20.72% | 1.23%
UOPS_RETIRE | 50.30% | 27.23% | 21.70% | 0.77%
Table 6. Statistics for RMSprop with 70% training data | TP | TN | FP | FN
---|---|---|---|---
BRANCH | 48.77% | 27.96% | 20.98% | 2.30%
CLOCK | 49.15% | 27.57% | 21.36% | 1.91%
CYCLE_ACTIVITY | 50.04% | 29.36% | 21.70% | 1.02%
DATA | 49.62% | 28.94% | 20.00% | 1.45%
FLOPS_DP | 48.64% | 40.26% | 8.68% | 2.43%
ICACHE | 47.87% | 37.91% | 11.02% | 3.19%
L2_DATA | 44.81% | 35.23% | 13.70% | 6.26%
L2_CACHE | 48.26% | 34.00% | 14.94% | 2.81%
L3_DATA | 46.77% | 37.53% | 11.40% | 4.30%
L3_CACHE | 46.77% | 36.51% | 12.43% | 4.30%
TLB_DATA | 50.17% | 45.19% | 3.74% | 0.89%
TLB_INSTR | 0.00% | 48.94% | 0.00% | 51.06%
UOPS | 48.38% | 25.15% | 23.79% | 2.68%
UOPS_EXEC | 49.96% | 26.81% | 22.13% | 1.11%
UOPS_ISSUE | 50.43% | 27.74% | 21.19% | 0.64%
UOPS_RETIRE | 50.04% | 27.91% | 21.02% | 1.02%

In Table 7, we provide a comparative analysis between our proposed RanStop
scheme and existing state-of-the-art techniques for ransomware detection. The
table lists different detection techniques, dataset sizes, and performance
metric. As it is shown, RanStop has provided signficiantly better result over
a comprehensive database of ransomware and goodware; and can provide an early
detection with very high accuracy.

Table 7. Comparative Analysis of Existing Techniques |  | Kharazz
---
et al. [11]
| Scaife
---
et al. [6]
| Moussaileb
---
et al. [17]
| Alam
---
et al. [16]
RanStop
Detection Type | Static | Static | Static + Dynamic | Dynamic | Dynamic
Key Features | - | API Calls | graph | hardware-assisted | hardware-assisted
Ransomware Database Size | - | - |  $\sim$700 | <5 | 80
Goodware Program Versatility | Random | Random | Random | Random + Computationally Intensive | Random + Computationally Intensive
Signature Collection Time | - | - | - | In order of seconds | In order of milliseconds
Average Accuracy | - | - | in order of 60% | 100% (for selective RW) | 97%

##  5\. Conclusion

In this paper, we present a hardware-assisted crypto-ransomware runtime
detector, called RanStop. Our proposed technique can detect ransomware with an
average of $\mathrm{9}\mathrm{7}\mathrm{\char 37\relax}$ with data collected
as early as $\mathrm{2}\mathrm{m}\mathrm{s}$ from the start of execution of a
ransomware. Such a very early detection technique ensures that a system, even
if somewhat infected, will suffer little or no damage by crypto-ransomware,
and, therefore, robust against any stealthy ransomware attack. The LSTM-based
ML modeling scheme offers a high accuracy for multiple optimizers; giving the
user complete freedom to choose for while deploying the model for runtime
detection. Although fast and accurate, our proposed scheme, like many other
existing techniques, suffers from additional challenges – especially, (1) Like
any other watchdog program, the runtime detection program itself may be
vulnerable to malicious infections and the obtained hardware data may get
corrupted. Hence, the program must run at the highest privilege level, be
independent of any other program, and have access to sufficient resources
(e.g., physical memory). (2) Due to hardware (physical) limitations, it is not
possible to simultaneously access and monitor all micro-architectural events
via available limited number of HPCs. Moreover, the micro-architectural events
collected by HPCs are historically performance-oriented. Therefore; it does
not necessarily provide security-aware micro-architectural events that may
become significant for the detection of emerging threats. This will require
development and implementation of additional HPC-like register for wider event
coverage. We leave this challenges as a future scope to this work.

## References

  * (1) McAfee Labs Threats Report: April 2017. [Accessed: 14 June 2019]. [Online]. Available: https://www.mcafee.com/April2017ThreatsReport
  * (2) K. Hutcherson. Ransomware reigns supreme in 2018, as phishing attacks continue to trick employees. [Accessed: 14 June 2019]. [Online]. Available: https://www.cnn.com/2018/03/27/us/atlanta-ransomware-computers/index.html
  * (3) R. Vamosi. WannaCry ransomware attack takes the world by storm. [Accessed: 14 June 2019]. [Online]. Available: https://www.synopsys.com/blogs/software-security/wannacry-ransomware/
  * (4) G. O’Gorman and G. McDonald, _Ransomware: A growing menace_. Symantec Corporation, 2012.
  * (5) A. Liptak. (2017) The WannaCry ransomware attack has spread to 150 countries. [Accessed: 14 June 2019]. [Online]. Available: https://www.theverge.com/2017/5/14/15637888/authorities-wannacry-ransomware-attack-spread-150-countries
  * (6) N. Scaife, H. Carter, P. Traynor, and K. R. Butler, “Cryptolock (and drop it): stopping ransomware attacks on user data,” in _Distributed Computing Systems (ICDCS), 2016 IEEE 36th International Conference on_. IEEE, 2016, pp. 303–312.
  * (7) D. Sgandurra, L. Muñoz-González, R. Mohsen, and E. C. Lupu, “Automated dynamic analysis of ransomware: Benefits, limitations and use for detection,” _arXiv preprint arXiv:1609.03020_ , 2016.
  * (8) E. Arnold. Tennessee sheriff pays ransom to cybercriminals. [Accessed: 14 June 2019]. [Online]. Available: http://www.bizjournals.com/memphis/blog/2014/11/tennessee-sheriff-pays-ransom-to-cybercriminals-in.html
  * (9) Radiohead refuses to pay ransom, releases stolen music for fans. [Accessed: 14 June 2019]. [Online]. Available: https://www.latimes.com/entertainment/music/la-et-ms-radiohead-ok-computer-demos-hackers-20190611-story.html
  * (10) Ransomware reigns supreme in 2018, as phishing attacks continue to trick employees. [Accessed: 14 June 2019]. [Online]. Available: https://www.techrepublic.com/article/ransomware-reigns-supreme-in-2018-as-phishing-attacks-continue-to-trick-employees/
  * (11) A. Kharraz, S. Arshad, C. Mulliner, W. K. Robertson, and E. Kirda, “Unveil: A large-scale, automated approach to detecting ransomware.” in _USENIX Security Symposium_ , 2016, pp. 757–772.
  * (12) N. Andronio, S. Zanero, and F. Maggi, “Heldroid: Dissecting and detecting mobile ransomware,” in _International Workshop on Recent Advances in Intrusion Detection_. Springer, 2015, pp. 382–404.
  * (13) M. A. Bishop, _Introduction to computer security_. Addison-Wesley Boston, 2005, vol. 50.
  * (14) Intel® 64 and IA-32 Architectures Software Developer’s Manual – Volume 3. [Accessed: 14 June 2019]. [Online]. Available: https://software.intel.com/sites/default/files/managed/a4/60/325384-sdm-vol-3abcd.pdf
  * (15) ARM CortexA9 Technical Reference Manual - Chapter 11 Performance Monitoring Unit. [Accessed: 14 June 2019. [Online]. Available: http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0433c/BEHGGDJC.html
  * (16) M. Alam, S. Bhattacharya, S. Dutta, S. Sinha, D. Mukhopadhyay, and A. Chattopadhyay, “Ratafia: Ransomware analysis using time and frequency informed autoencoders,” in _IEEE International Symposium on Hardware Oriented Security and Trust (HOST)_. IEEE, 2019.
  * (17) R. Moussaileb, B. Bouget, A. Palisse, H. Le Bouder, N. Cuppens, and J.-L. Lanet, “Ransomware’s early mitigation mechanisms,” in _Proceedings of the 13th International Conference on Availability, Reliability and Security_. ACM, 2018, p. 2.
  * (18) S. Hochreiter and J. Schmidhuber, “Long short-term memory,” _Neural computation_ , vol. 9, no. 8, pp. 1735–1780, 1997.
  * (19) F. A. Gers, N. N. Schraudolph, and J. Schmidhuber, “Learning precise timing with lstm recurrent networks,” _Journal of machine learning research_ , vol. 3, no. Aug, pp. 115–143, 2002.
  * (20) A. Kharraz, W. Robertson, D. Balzarotti, L. Bilge, and E. Kirda, “Cutting the gordian knot: A look under the hood of ransomware attacks,” in _International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment_. Springer, 2015, pp. 3–24.
  * (21) M. Spisak, “Hardware-assisted rootkits: Abusing performance counters on the arm and x86 architectures.” in _WOOT_ , 2016.
  * (22) likwid: Performance monitoring and benchmarking suite. [Accessed: 10 August 2018]. [Online]. Available: {https://github.com/RRZE-HPC/likwid}
  * (23) Performance analysis tools based on Linux perf events (aka perf) and ftrace. [Accessed: 14 June 2019]. [Online]. Available: {https://github.com/brendangregg/perf-tools}
  * (24) A. Tang, S. Sethumadhavan, and S. J. Stolfo, “Unsupervised anomaly-based malware detection using hardware features,” in _International Workshop on Recent Advances in Intrusion Detection_. Springer, 2014, pp. 109–129.
  * (25) X. Wang, C. Konstantinou, M. Maniatakos, and R. Karri, “Confirm: Detecting firmware modifications in embedded systems using hardware performance counters,” in _Computer-Aided Design (ICCAD), 2015 IEEE/ACM International Conference on_. IEEE, 2015, pp. 544–551.
  * (26) C. Malone, M. Zahran, and R. Karri, “Are hardware performance counters a cost effective way for integrity checking of programs,” in _Proceedings of the sixth ACM workshop on Scalable trusted computing_. ACM, 2011, pp. 71–76.
  * (27) J. Demme, M. Maycock, J. Schmitz, A. Tang, A. Waksman, S. Sethumadhavan, and S. Stolfo, “On the feasibility of online malware detection with performance counters,” in _ACM SIGARCH Computer Architecture News_ , vol. 41, no. 3. ACM, 2013, pp. 559–570.
  * (28) VeraCrypt. [Accessed: 14 June 2019]. [Online]. Available: https://www.veracrypt.fr/en/Home.html
  * (29) I. H. Witten, E. Frank, M. A. Hall, and C. J. Pal, _Data Mining: Practical machine learning tools and techniques_. Morgan Kaufmann, 2016.
  * (30) VirusShare. [Accessed: 14 June 2019]. [Online]. Available: https://virusshare.com/
  * (31) OpenSSL. [Accessed: 14 June 2019]. [Online]. Available: https://www.openssl.org/
  * (32) Awesome C. [Accessed: 14 June 2019]. [Online]. Available: https://github.com/kozross/awesome-c
  * (33) C Code Katas. [Accessed: 14 June 2019]. [Online]. Available: https://github.com/ankitpokhrel/c-code-katas
  * (34) Digital corpora/govdocs1. [Accessed: 14 June 2019]. [Online]. Available: https://digitalcorpora.org/corpora/files
  * (35) F. Chollet _et al._ , “Keras,” https://keras.io, 2015.
  * (36) Keras Documentation - Usage of Optimizer. [Accessed: 14 June 2019]. [Online]. Available: https://keras.io/optimizers/