 11affiliationtext: Lawrence Berkeley National Laboratory, Energy
Technologies Area, 1 Cyclotron Road, Berkeley, CA 94720, United States
22affiliationtext: Department of Materials Science and Engineering, University
of California, Berkeley, California 94720, United States 33affiliationtext:
Lawrence Berkeley National Laboratory, Materials Sciences Division, 1
Cyclotron Road, Berkeley, CA 94720, United States

# Pawpyseed: Perturbation-extrapolation band shifting corrections for point
defect calculations

Kyle Bystrom  Danny Broberg  Shyam Dwaraknath  Kristin A. Persson  Mark Asta

###### Abstract

Significant progress has been made recently in the automation and
standardization of _ab initio_ point defect calculations. However, the task of
developing, implementing, and benchmarking charge corrections for density
functional theory (DFT) point defect calculations is still an open challenge.
Here we present a high-performance Python package called pawpyseed, which can
read PAW DFT wave functions and calculate the overlap between wavefunctions
from different structures. Using pawpyseed, we implement a new band shifting
correction derived from first order perturbation theory. We benchmark this
method by calculating the transition levels of several point defects in
silicon and comparing to experimental and hybrid functional results. The new
band shifting method can shift single-particle energies to improve transition
level predictions and can be automated and parallelized using pawpyseed,
suggesting it could be a useful method for high-throughput point defect
calculations.

##  1 Introduction

There has recently been significant research interest in automation and high-
throughput computing for the _ab initio_ prediction of point defect properties
[1, 2, 3], as well as the development and standardization of methods for
accurately predicting point defect properties from semi-local density
functional theory (DFT) [4, 5]. Most first principles studies of point defects
use the supercell method: The point defect is embedded in a supercell of the
bulk structure, which is repeated infinitely throughout space and modeled with
DFT. This method allows point defects to be modeled in robust, high-
performance plane-wave DFT codes, but it has several shortcomings. First, the
periodic boundary conditions cause unphysical elastic and electronic
interactions of the defect with periodic images of itself (finite-size
effects). Finite-size effects can be accounted for using electrostatic
corrections such as the Freysoldt correction [6] and Kumagai correction [7].
Second, semi-local DFT functionals drastically underestimate material band
gaps. This “band gap problem” decreases the accuracy of defect calculations by
incorrectly describing the location of defect levels in the band gap [4, 5].
The band gap problem also results in unrealistic delocalization of electron
density [4, 5], which causes electrostatic corrections to break down [8]. The
delocalization issue is generally circumvented using hybrid functional methods
[9, 10, 11], which correct the band gap but are too computationally expensive
for high-throughput applications. Therefore, a method which can correct band
gap-related errors in defect formation energies from semi-local DFT is needed
to enable the prediction of defect properties in high throughput.

The current solutions to this problem are a _band filling_ correction to
account for band dispersion due to charge delocalization and a _band shifting_
correction to account for the inaccurate location of defect levels in the band
structure. In this work, we present a DFT post-processing software package
called pawpyseed that provides tools to develop, automate, and improve band
shifting corrections. Specifically, we implement a perturbation theory-
motivated band shifting correction, with computationally expensive portions of
the code written in C and parallelized for efficiency. To benchmark the
performance of this correction against a more basic band shifting technique
and other common correction schemes, we use various correction methods to
calculate the transition levels of point defects in silicon with important
device applications: the single vacancy [12] and the boron [12], phosphorus
[12], copper [13, 14], and sulfur [15] substitutionals. The objective is to
obtain semi-local DFT transition levels that better approximate those
calculated with hybrid DFT or measured in experiments, which would facilitate
high-throughput screening studies of point defects.

###  1.1 Defect formation energies and band shifting

Band shifting, the focus of this work, attempts to correct the location of
defect levels in the band gap for semi-local DFT calculations [4]. The general
approach is to calculate accurate band edges of the bulk crystal using a
higher level of theory, such as hybrid DFT, and then correct the defect energy
based on the band edge shifts:

$\displaystyle\Delta E_{CBM}$ $\displaystyle=E_{CBM,hybrid}-E_{CBM,GGA}$ (1)
$\displaystyle\Delta E_{VBM}$ $\displaystyle=E_{VBM,hybrid}-E_{VBM,GGA}$ (2)

Here, VBM is the valence band maximum, and CBM is the conduction band minimum.
The most common use of these band edge shifts is Fermi level renormalization.
In the supercell method, the Fermi level is referenced to the VBM, so the
formation energy of each defect of charge state $q$ is shifted by $q\Delta
E_{VBM}$ as a Fermi level correction. This shifts each transition level by
$\Delta E_{VBM}$ relative to the VBM [16]. This correction neglects that
defect levels derived primarily from conduction or valence bands might shift
in energy with those bands when the gap is opened, thereby shifting the
transition levels by more or less than $\Delta E_{VBM}$ [4]. An additional
level of complexity is to assume that perturbed host states (PHS, shallow
levels derived from delocalized bulk valence or conduction bands) shift in
energy with the bands they are derived from when the band edges are shifted to
the hybrid level of theory [16, 4]. Following this reasoning, two terms can be
added to the defect formation energy. The first is $\Delta E_{CBM}$ times the
number of electrons in conduction band PHSs, and the second is $-\Delta
E_{VBM}$ times the number of holes in valence band PHSs.

Without additional data, simple PHS shifting is not compatible with high-
throughput approaches because it requires identifying perturbed host states.
Due to the underestimation of the band gap in GGA, a deep level state derived
from the localized atomic orbitals of an impurity might be located near/above
the CBM or near/below the VBM in GGA. An automated script would identify such
a state as a PHS, but it has been argued that the energy level of a well-
localized state on a defect is well-described by semi-local DFT and does not
shift with the band edges [17]. In addition, a state might have some
delocalized host band character and some localized atomic state character,
which might cause the energy of the state to shift fractionally with the band
edges [17, 4]. These shortcomings motivate a more sophisticated band shifting
correction, which is discussed next.

###  1.2 The perturbation-extrapolation band shifting correction

In order to extrapolate the defect level from the semi-local DFT band gap to
the experimental or hybrid DFT band gap, several methods have been developed
that predict how a defect level changes as the band gap changes [18]. Most of
these methods are based on DFT calculations with a perturbed and unperturbed
system, such as an LDA and LDA+U functional [19]. However, Bogusławski et al.
[20] projected defect levels onto valence and conduction bands and then
shifted the levels a percentage of $\Delta E_{CBM}$ based on the percentage
conduction character. Using the formalism for a perturbation-extrapolation
band gap correction developed by Lany and Zunger [4], we introduce a band
shifting correction based on the strategy of Bogusławski et al. This band
shifting correction adjusts the Kohn-Sham single-particle states, which are
the eigenfunctions of the Kohn-Sham Hamiltonian [21]. These states are not
unique and are only physically significant because they generally give good
approximations to single-particle ionization energies, which allows them to be
treated as orbitals occupied by a single electron (or electron pair) in an
effective potential. Relating shifts in these single-particle energies to
formation energies and transition levels is therefore an approximation.

Because the eigenfunctions of a Hamiltonian form a complete basis, a defect
wavefunction can be expanded in the bulk wavefunctions:

$\psi_{D}(\mathbf{r})=\sum_{n,\mathbf{k}}\braket{\psi_{n,\mathbf{k}}}{\psi_{D}}\psi_{n,\mathbf{k}}(\mathbf{r})=\sum_{n,\mathbf{k}}A_{n,\mathbf{k}}\psi_{n,\mathbf{k}}(\mathbf{r})$
(3)

$\psi_{D}$ is a defect wavefunction, and $\psi_{n,\mathbf{k}}$ are evaluated
in the pristine bulk. Physically, $\psi_{D}$ is a single state, as opposed to
a band of states in k-space, because the defect is not periodic and therefore
does not have dispersion. However, it is evaluated at different k-points in
the supercell method, so the supercell method defect levels
$\psi_{D,\mathbf{k}}$ will be used to ground the band shifting correction in a
practical computational framework. Since all wavefunctions at different
k-points are orthogonal,

$\psi_{D,\mathbf{k}}(\mathbf{r})=\sum_{n}A_{n,\mathbf{k}}\psi_{n,\mathbf{k}}(\mathbf{r})$
(4)

If the single-particle energy level of $\psi_{D,\mathbf{k}}(\mathbf{r})$ in
DFT is $e^{0}_{D,\mathbf{k}}$, first-order perturbation theory can be used as
suggested by Lany and Zunger to calculate a corrected energy [4].

$e_{D,\mathbf{k}}=e^{0}_{D,\mathbf{k}}+\bra{\psi_{D,\mathbf{k}}(\mathbf{r})}\Delta
H\ket{\psi_{D,\mathbf{k}}(\mathbf{r})}$ (5)

Here, $\Delta H$ is a correction term that extrapolates from the DFT picture
to the true quasiparticle energies. Assuming the diagonal elements of $\Delta
H$ in the basis of the bulk wavefunctions are small, Equation 5 can be
expanded [4]:

$e_{D,\mathbf{k}}=e^{0}_{D,\mathbf{k}}+\sum_{n}|A_{n,\mathbf{k}}|^{2}\bra{\psi_{n,\mathbf{k}}(\mathbf{r})}\Delta
H\ket{\psi_{n,\mathbf{k}}(\mathbf{r})}$ (6)

A rough but potentially useful approximation to $\Delta H$ is a “band
shifting” operator that shifts the energy of a bulk conduction band by $\Delta
E_{CBM}$ (Equation 1) and the energy of a bulk valence band by $\Delta
E_{VBM}$ (Equation 2). In fact, not all conduction bands shift by $\Delta
E_{CBM}$ and valence bands by $\Delta E_{VBM}$, but this approximation is
useful because the defect states of interest are in or near the band gap and
most likely composed of states near the band gap. This band shifting operator
can therefore shift defect levels calculated in GGA toward the more accurate
hybrid DFT energy levels based on how they project onto host bands.

It is useful to define the “proportion valence” $v_{D,\mathbf{k}}$ and
“proportion conduction” $c_{D,\mathbf{k}}$ as:

$\displaystyle v_{D,\mathbf{k}}$ $\displaystyle\equiv\sum_{n\in
VB}|\braket{\psi_{D,\mathbf{k}}}{\psi_{n,\mathbf{k}}}|^{2}$ (7) $\displaystyle
c_{D,\mathbf{k}}$ $\displaystyle\equiv\sum_{n\in
CB}|\braket{\psi_{D,\mathbf{k}}}{\psi_{n,\mathbf{k}}}|^{2}$ (8)

Here, VB is the set of valence bands and CB is the set of conduction bands.
Using these definitions, Equation 6 can be expressed simply:

$\displaystyle e_{D,\mathbf{k}}=$ $\displaystyle
e_{D,\mathbf{k}}^{0}+c_{D,\mathbf{k}}\Delta E_{CBM}+v_{D,\mathbf{k}}\Delta
E_{VBM}$ (9)

Since defect levels should not have k-point dispersion, one can also average
$v_{D,\mathbf{k}}$ and $c_{D,\mathbf{k}}$ over k-points, which we choose to do
in this work. While this type of correction is of interest to the research
community based on its presence in recent reviews [4, 5], it does not have a
standard formalism or open-source implementation. This is partly because the
accurate evaluation of the overlap terms
$\braket{\psi_{D,\mathbf{k}}}{\psi_{n,\mathbf{k}}}$ is computationally
difficult in plane-wave DFT with the projector-augmented wave (PAW) method,
which is the most commonly used basis set for modern periodic solid
calculations. Pawpyseed implements these overlap evaluations as described in
Section 3.2. It also implements a band shifting correction based on Equation
9, called the “Projection shift,” which is described in Section 3.3.

##  2 Results and Discussion

Table 1: Errors of predicted transition levels in eV for the DFT calculations performed with $\Gamma$-centered k-point meshes. Format: “level in eV (error as percentage of band gap).” The band gap for the Fr method is 0.61 eV, while for the other methods it is 1.19 eV. The HSE06 band gap in other studies is 1.16-1.19 eV. Fr = Freysoldt finite-size correction; FLR = Fermi Level Renormalization; PrS = Projection Shifting (Equations 11 and 12); DPrS = Delocalized-State Projection Shifting (Equations 11 and 13). Transition Level | Reference | Fr | Fr+FLR | Fr+FLR+PrS | Fr+FLR+DPrS
---|---|---|---|---|---
Vac ++/+ | 0.14 [22] | -0.08 (-25%) | 0.28 (12%) | -0.00 (-12%) | 0.14 (0%)
Vac +/0 | 0.02 [22] | -0.00 (-2%) | 0.36 (28%) | -0.02 (-3%) | -0.01 (-2%)
P +/0 | 1.07 [23] | 0.49 (-15%) | 0.85 (-24%) | 1.07 (-6%) | 1.04 (-8%)
B 0/- | 0.04 [24] | 0.12 (16%) | 0.48 (37%) | 0.13 (7%) | 0.15 (8%)
Cu +/0 | 0.20 [25] | 0.08 (-4%) | 0.44 (20%) | 0.16 (-4%) | 0.35 (12%)
Cu 0/- | 0.54 [25] | 0.32 (7%) | 0.68 (11%) | 0.41 (-12%) | 0.58 (3%)
Cu -/– | 0.97 [25] | 0.63 (20%) | 0.99 (0%) | 0.77 (-19%) | 0.90 (-8%)
S ++/+ | 0.58 [10] | 0.10 (-33%) | 0.46 (-11%) | 0.62 (3%) | 0.53 (-5%)
S +/0 | 0.86 [10] | 0.43 (-4%) | 0.79 (-7%) | 0.97 (8%) | 0.89 (1%)

To benchmark the perturbative band shifting correction, we used PyCDT [1] to
generate structures and charge states for several point defects in silicon. We
calculated the DFT total energy of each defect as described in Section 3.4 and
used PyCDT, pymatgen [26], and pawpyseed post-processing tools to calculate
the formation energies using four different correction schemes (Table 1). The
first is the Freysoldt finite-size correction only, the second adds Fermi
level renormalization, and the third and fourth use two variants of
perturbative band shifting discussed in Section 3.3. Projection Shifting
shifts all defect levels according to Equation 9, while Delocalized-State
Projection Shifting shifts these levels a fraction of that value based on how
delocalized the band is. The justification for the latter approach is that
localized defect levels are well-characterized by atomic or molecular orbitals
and therefore should not shift in energy with delocalized bulk host states
[17]. Band filling corrections are included for all calculations, although
they have a small effect and do not generally improve the transition level
predictions. Correction schemes excluding band filling corrections are
provided in the SI.

The thermodynamic transition levels were determined from the difference
between the formation energies of different charge states. Analysis of
formation energies was not done to avoid the complications of chemical
potentials. The existing literature on these transition levels contains hybrid
DFT (HSE06 [27]) and experimental data [28, 29, 30, 31] in good agreement. The
exceptions are the phosphorus and boron substitutionals, for which we could
not find hybrid DFT calculations, likely because these defect levels are too
shallow to contain in a reasonable-size supercell.

There can be a significant difference between the activation energies measured
by deep-level transient spectroscopy (DLTS) and 0 K ionization energies [32].
Therefore, we choose to calculate transition level errors relative to the
hybrid DFT data except for in the cases of phosphorus and boron, for which we
compare to experimental excitation spectra. Because of the good agreement
between theory and experiment for these defects, this decision does not
significantly impact the interpretation of results. To aid in understanding
the action of the Projection Shift on the defect levels, Figure 1 shows the
valence band character $v_{D}$ and conduction band character $c_{D}$ for
levels near the band gap of the neutral copper, sulfur, and phosphorus
substitutionals.

Figure 1: $v_{D}$ (valence band character) and $c_{D}$ (conduction band
character) for states near the band gap of several neutral defects. The upper
portion of the plot shows $v_{D}$ (blue) and $c_{D}$ (red). The lower part
shows the average energy of each spin-polarized band, with the grey bars
representing the range of energies of that band over the k-point mesh. The
color of the energy levels corresponds to the occupation, with the “rainbow”
color scale provided in Matplotlib [33] used to clearly illustrate partial
occupancies. Violet levels have occupancy 1, and red levels have occupancy 0.
The GGA band gap of 0.61 eV is denoted with the black lines. Note that because
the bulk basis set is finite and therefore incomplete, $v_{D}+c_{D}$ can be
less than 1. The missing character, represented by white space between the red
and blue bars, is due to the difference between the space spanned by the bulk
eigenstates and defect eigenstates. For example, the bulk silicon basis
contains no $d$ states, so the white space in the Cu substitutional projection
diagram might be due to $d$ state character (or to the Cu $s$ and $p$ atomic
orbitals having different shapes than those of Si).

Figure 2: $v_{D}$ and $c_{D}$ for states near the band gap of several charge
states of the silicon vacancy. The highest occupied band of the neutral
vacancy has a larger $v_{D}$ than the same band in the positive charge states,
and this band is lower in energy than in the positive charge states.

###  2.1 Comparison of non-hydrogenic transition levels to hybrid DFT and
experiment

The single vacancy is a double donor with HSE06 transition levels at 0.14 eV
(++/+) and 0.08 eV (+/0) [22]. The ++/+ transition is higher than the +/0
transition due to well-documented and experimentally observed negative-U
behavior [34], with experimental levels at 0.13 eV (++/+) and 0.05 eV (+/0)
[28]. The GGA functional has previously been used to capture the negative-U
behavior of the silicon vacancy [35] without finite-size corrections, but the
Freysoldt correction decreases the ++/+ transition level by 0.2 eV, resulting
in an incorrect picture of the defect. Interestingly, adding the Projection
Shift does result in negative-U behavior for the single vacancy, but the ++/+
level is too low. As shown in Figure 2, the occupied defect level in the
neutral charge state has more valence band character than in the +1 state
(with $v_{C}=0.95$ for neutral and $v_{C}=0.82$ for +1). Therefore, the energy
of that level shifts lower for the neutral defect and recovers the negative-U
character. Because the defect levels in the vacancy are partly localized
around the defect, Delocalized-State Projection Shifting shifts the levels
less than Projection Shifting. This raises the ++/+ transition to the hybrid
DFT value (Table 1).

Substitutional copper has three transition levels, which were calculated with
the HSE06 functional to be at 0.20 eV (+/0), 0.54 eV (0/-), and 0.97 eV (-/–)
[25]. Fermi level renormalization overestimates the +/0 and 0/- transition
levels significantly. Projection shifting fixes the +/0 level but over-
corrects Fermi level renormalization for the 0/- level compared to HSE06.
However, the experimental 0/- level is 0.43 eV [31], which is 0.11 eV below
the HSE06 level and close to the Projection Shift prediction. The -/– level is
predicted accurately by Fermi level renormalization but then shifted -0.22 eV
by Projection Shifting (Table 1). This suggests that the copper defect levels
are too localized to be completely shifted with the host bands. Delocalized-
state Projection Shifting partly remedies this issue, but the +/0 level is
overestimated and the -/– level is underestimated.

The sulfur substitutional is a double donor with DLTS levels at 0.546 eV
(++/+) and 0.325 eV (+/0) below the CBM [29]. HSE06 calculations placed the
levels 0.54 and 0.26 eV below the CBM, respectively [10]. The high conduction
character of the sulfur defect level (Figure 1) matches the expectation from a
simple hydrogenic model. Due to this conduction character, the Projection
Shift method raises the sulfur transition levels compared to Fermi level
renormalization. The Delocalized-State Projection Shifting result is between
the other two because the defect level is partially localized around the
sulfur.

###  2.2 Shallow hydrogenic levels: phosphorus and boron

The shallowness of the phosphorus and boron substitutionals can decrease the
accuracy of supercell calculations due to charge delocalization. These defects
are still included because they provide a simple benchmark of whether band
shifting methods effectively shift shallow levels with the band edges. The
phosphorus donor level is 46 meV below the CBM [23], and the boron acceptor
level is 44 meV above the VBM [24].

As shown in Table 1, Fermi level renormalization does not accurately treat
shallow defect levels because it does not shift the levels with the valence
band edge (for boron/shallow acceptor) or conduction band edge (for
phosphorus/shallow donor). The phosphorus and boron levels are at 0.86 eV and
0.44 eV, respectively. Projection Shifting correctly predicts both the boron
and phosphorus transition levels because the boron level has nearly 100%
valence character and the phosphorus level has nearly 100% conduction
character (Figure 1), so the levels shift with the valence and conduction
bands, respectively. The phosphorus and boron levels are highly delocalized,
so most of the Projection Shift is preserved by the Delocalized-state
Projection Shift.

###  2.3 Summary and Conclusions

We have developed a perturbation-extrapolation correction method for point
defect calculations based on projecting defect levels onto the bulk sates. As
opposed to previous correction schemes, this new Projection Shift method
successfully handles edge cases involving shallow transition levels and
single-particle levels. We have published an open-source, parallel Python/C
package called pawpyseed that performs the computationally expensive
components of the Projection Shift, allowing the method to be further
developed and tested. One possible future project is to develop a more robust
method for assessing what fraction of the Equation 9 shift to perform based on
how localized a state is. Another is to use Equation 9 to shift transition
levels themselves rather than the single-particle energies in the DFT
calculations.

The primary shortcoming of Fermi level renormalization is that the missing
physics in semi-local DFT compared to real physical systems can affect the
electronic structure of the defect as well as the bulk host properties. Fermi
level renormalization only accounts for the latter, which is particularly
detrimental in the case of shallow defect levels that are well-characterized
by delocalized bulk states.

The Projection Shift method is more computationally expensive than other
correction methods, though much less so than the actual DFT calculations for a
defect. However, by shifting energy levels based on projections onto bulk
states, it is possible to roughly estimate the effect of post-DFT physics on
transition levels. The limitation of Projection Shifting is that the
projection of defect levels onto conduction and valence bands clearly does not
provide complete information about how the energy of the defect level will
shift when extrapolating from DFT to a more accurate band structure. The
chemical environment of an electron in a given state is significantly
different in the defect than in the bulk, so the correction to the energy
could be very different. One way to account for the local chemical environment
around the defect is to partly base the shift in energy of defect levels on
the projection onto local atomic wavefunctions around the defect. Local atomic
states can have different energies in semi-local and hybrid DFT, which will
affect the transition level predictions of localized defect levels.

While the abstract formulations of the methods here have been developed
before, pawpyseed is the first implementation of standard tools for performing
perturbation-extrapolation corrections for point defect calculations. These
tools can potentially improve defect transition level predictions of semi-
local DFT point defect calculations enough to be useful in high-throughput
screening applications, for which more accurate approaches like hybrid DFT are
too computationally expensive.

##  3 Methods

###  3.1 Pawpyseed implementation details

Pawpyseed is written in Python and C. All user interface is written in Python,
and computationally expensive tasks are performed by interfacing with a C
extension via Cython [36]. The Intel Math Kernel Library is used for linear
algebra routines in C, and OpenMP [37] is used to implement shared memory
parallelization. Pawpyseed reads the output of VASP calculations [38]. Pseudo
wavefunctions are read from WAVECAR files using a method based on the
WaveTrans program [39], and data about augmentation regions is read from
pseudopotential (POTCAR) files. In addition to the core Python library,
pawpyseed relies heavily on NumPy and SciPy [40] (for numerical operations),
pymatgen [26] (for storing and manipulating structures, reading VASP input and
output files, and performing symmetry analysis), Matplotlib [33] (for
visualization), and SymPy [41] (for calculating and storing Gaunt coefficients
for overlap operator routines). The code also makes use of the spherical
Bessel transform algorithm developed by Talman [42].

Because pawpyseed abstracts all high-performance routines to its C extension,
complex routines can be performed with single Python calls. The following code
snippet reads the pseudo wavefunctions for two VASP calculations (for a bulk
structure and defect structure), sets up the projector function and partial
wave components, and projects band 0 of the defect structure onto each of the
bands of the bulk structure for each k-point and spin. Further examples and
documentation are available on the pawpyseed website (Section 4).

        from pawpyseed.core.projector import Wavefunction, Projector
        bulk = Wavefunction.from_directory(’bulk’)
        defect = Wavefunction.from_directory(’defect’)
        pr = Projector(defect, bulk)
        res = pr.single_band_projection(0)

###  3.2 Overlap operators in pawpyseed

Pawpyseed calculates the overlaps between PAW wavefunctions from different
structures [43]:

$\ket{\psi_{n\mathbf{k}}}=\ket{\widetilde{\psi}_{n\mathbf{k}}}+\sum_{i}(\ket{\phi_{i}}-\ket{\widetilde{\phi}_{i}})\braket{\widetilde{p}_{i}}{\widetilde{\psi}_{n\mathbf{k}}}$
(10)

There are four options for evaluating these overlaps in pawpyseed. The first,
which is used for this project and is the default in pawpyseed, is to
individually project the components $\ket{\widetilde{\psi}_{n\mathbf{k}}}$ and
$(\ket{\phi_{i}}-\ket{\widetilde{\phi}_{i}})\braket{\widetilde{p}_{i}}{\widetilde{\psi}_{n\mathbf{k}}}$
of one wavefunction onto the components of the other. The numerical techniques
for this method are complex, and therefore we relegate them to the SI and the
technical reference on the pawpyseed website. For this method, aliasing in
real space integration is restricted to the overlap between the
$\ket{\phi_{i}}-\ket{\widetilde{\phi}_{i}}$ terms and the pseudo wavefunction
$\ket{\widetilde{\psi}_{n\mathbf{k}}}$.

The second method is a variation on the first, where the sum
$\sum_{i}(\ket{\phi_{i}}-\ket{\widetilde{\phi}_{i}})\braket{\widetilde{p}_{i}}{\widetilde{\psi}_{n\mathbf{k}}}$
is evaluated in real space and then transformed into reciprocal space to
project onto $\ket{\widetilde{\psi}_{n\mathbf{k}}}$. Because the
$\ket{\phi_{i}}-\ket{\widetilde{\phi}_{i}}$ terms are not multiplied by
$\ket{\widetilde{\psi}_{n\mathbf{k}}}$ directly in real space, this method can
be used to prevent integration aliasing at the expense of using more memory.
We chose not to use this method here because it is still in development, but
tests show the choice between these methods does not significantly affect the
transition level results.

The third method is to evaluate Equation 10 in real space and then integrate
the product of the two wavefunctions over the unit cell. This method requires
a denser realspace grid and therefore is more computationally expensive.

The fourth method is to ignore all components except the pseudo wavefunction
$\ket{\widetilde{\psi}_{n\mathbf{k}}}$, which is a sum of plane waves and
therefore easy to integrate. Because the pseudo wavefunctions are not
orthonormal, this method is not recommended and has no guaranteed precision.
It is provided as an option in the code because it is very fast and can be
used for quick preliminary analysis in some cases.

There are several precision considerations for these integrals, particularly
for the first method. For example, any incompleteness in the set of pseudo
partial waves $\ket{\widetilde{\phi}_{i}}$ will lead to normalization errors,
and the high-frequency components of the all electron partial waves
$\ket{\phi_{i}}$ will cause aliasing when integrated in real space for any
reasonable grid size. However, these errors tend to be small, and basic tests
suggest the precision of the first method above is better than 0.01 for all
bands except high conduction bands, which are poorly described by the pseudo
partial waves. This precision is adequate for the corrections presented here,
and the precision can be improved at the cost of computational efficiency for
applications where high precision is necessary.

###  3.3 Perturbative band shifting correction implementation

For the purpose of benchmarking, a very basic perturbative band shifting
method is presented here and referred to as the “Projection Shift.” Because
the perturbative band shifting correction adjusts single-particle energy
levels, it is nontrivial to decide the correction to the total energy, even
for a simplified correction like Equation 9. For this work, it is assumed that
the shifted single-particle energies $e_{n,\mathbf{k}}$ given by Equation 9
only apply if $e_{n,\mathbf{k}}$ is in the band gap after being shifted (i.e.
it is a defect level). Otherwise, the energy shifts entirely with the set of
bands in which it is located (valence or conduction). This is reasonable
because Equation 9 should only be applied to defect levels, but all of the
bands shift in energy going from the GGA to hybrid level of theory.

To avoid counting energy shifts from occupied valence bands, a reference
energy $N_{VB,X,0}\Delta E_{VBM}$ is chosen, where $N_{VB,X,0}$ is the number
of electrons in the valence band of the neutral charge state of the defect.
Note that this reference energy does not affect transition levels because
transition levels are only dependent on the energy difference between charge
states. This method prevents large double counting errors for the Coulomb
energy while accounting for the energy shifts of the valence band, conduction
band, and defect levels.

$\displaystyle E_{corr,shift}=$ $\displaystyle-N_{VB,X,0}\Delta
E_{VBM}+\sum_{n\in BG}\Delta e_{n}$ (11) $\displaystyle+\sum_{n\in
VB,\mathbf{k}}\omega_{\mathbf{k}}f_{n,\mathbf{k}}\Delta E_{VBM}+\sum_{n\in
CB,\mathbf{k}}\omega_{\mathbf{k}}f_{n,\mathbf{k}}\Delta E_{CBM}$
$\displaystyle\Delta e_{n}=$
$\displaystyle\sum_{\mathbf{k}}\omega_{\mathbf{k}}f_{n,\mathbf{k}}(c_{n,\mathbf{k}}\Delta
E_{CBM}+v_{n,\mathbf{k}}\Delta E_{VBM})$ (12)

$\omega_{\mathbf{k}}$ are k-point weights and $f_{n,\mathbf{k}}$ are
occupations. BG, VB, and CB are the band gap, valence bands, and conduction
bands at the hybrid level of theory, respectively. To include all possible
defect levels, each band is categorized by its k-point-averaged energy
relative to the _hybrid DFT_ band edges and _before_ it is shifted by $\Delta
e_{n}$. The averaging is performed because defect levels do not have
dispersion in the true physical band structure. Spin polarization is taken
into account by replacing the band index $n$ with band and spin indices
$n,\sigma$.

An alteration of this method has been implemented that measures how localized
a defect level is and then performs a fraction of the energy shift (Equation
9) which is larger for more delocalized levels. We define $\chi_{n}$ as the
fraction of the density of state $n$ that is localized on the defect atom and
its first set of nearest neighbors. $\chi_{n}$ can be determined from the
site-projected character output from a DFT calculation (PROCAR for VASP). This
function is valued between 0 and 1, so it can serve as a fraction of the
Projection Shift to perform on each level. This gives an adjusted correction:

$\Delta
e_{n}=(1-\chi_{n})\sum_{\mathbf{k}}\omega_{\mathbf{k}}f_{n,\mathbf{k}}(c_{n,\mathbf{k}}\Delta
E_{CBM}+v_{n,\mathbf{k}}\Delta E_{VBM})$ (13)

We refer to this second method as “Delocalized-State Projection Shifting”.

###  3.4 DFT calculation details

All DFT calculations were performed using the VASP code [38, 44, 45, 46] with
the Perdew-Burke-Ernzerhof (PBE) generalized gradient approximation (GGA)
functional [47, 48] and the PAW method [49, 43]. For hybrid functional bulk
calculations, the Heyd-Scuseria-Ernzerhof (HSE) method [50] was used with the
standard HSE06 [27] tuning parameters. All k-point grids used were
$\Gamma$-centered to effectively sample band extrema, and all calculations
were performed with symmetry off to allow the environment of the defect to
relax out of its initial symmetry.

The boron, phosphorus, copper, and sulfur substitutionals and single vacancy
were embedded in a 216-atom supercell of silicon using PyCDT [1]. PyCDT was
also used to generate charge states for analysis. Energy convergence tests
were performed with the number of k-points. For the total cohesive energy of
the bulk crystal, a difference of 0.02 eV ($1\times 10^{-4}$ eV/atom) was
found between a 2x2x2 and 3x3x3 k-point mesh for the bulk supercell, so a
2x2x2 k-point mesh was used for bulk and defect supercell calculations. The
energy cutoff was set to 520 eV. An ionic relaxation and total energy
calculation were performed for each defect and charge state. The static
dielectric constant was also calculated for the bulk silicon crystal using
VASP and found to be 12.52. For the silicon vacancy, even with symmetry turned
off, the structure does not relax to a stable state because it is caught in a
metastable minimum. The final symmetry is different for each charge state, but
as an example, the neutral defect has a tetragonally distorted structure with
$D_{2d}$ symmetry [35]. To attain the correct structure, the positions of the
four atoms nearest the vacancy site were perturbed manually to break the
structure from the metastable minimum and break symmetry before the
calculation was run.

To calculate DFT and hybrid band edges, a DFT calculation was run on a single
2-atom silicon primitive cell with a 7x7x7 k-point mesh, once with the GGA
functional and once with the HSE06 functional. The GGA band gap calculated in
this study is 0.61 eV, in good agreement with the Materials Project GGA band
gap of 0.612 eV [51]. The HSE06 band gap is 1.19 eV, in good agreement with
the experimental 0 K value of 1.17 eV [52]. For our calculations, $\Delta
E_{VBM}=-0.36$ eV and $\Delta E_{CBM}=0.22$ eV.

##  4 Code Availability

Analysis scripts will be published on Github and linked to in a future
manuscript draft. The pawpyseed source code and installation instructions can
be found at https://github.com/kylebystrom/pawpyseed. The software
documentation, along with a technical reference guide and run-time scaling
analysis for computationally expensive routines, can be found at
https://kylebystrom.github.io/pawpyseed/.

##  5 Data Availability

Electronic structure data will be published to the NOMAD database in the near
future, and this manuscript will be updated with access information.

##  6 Acknowledgements

This work was funded by the U.S. Department of Energy, Office of Science,
Office of Basic Energy Sciences, Materials Sciences and Engineering Division
under Contract No. DE-AC02-05-CH11231 (Materials Project program KC23MP). This
research used the Savio computational cluster resource provided by the
Berkeley Research Computing program at the University of California, Berkeley
(supported by the UC Berkeley Chancellor, Vice Chancellor for Research, and
Chief Information Officer).

##  7 Author Contributions

KB wrote pawpyseed and data analysis scripts, performed the DFT calculations,
and wrote the manuscript. DB developed the idea for the project, helped refine
the correction methods, and assisted with interpretation of results. MA
supervised the code development and initial collection of results. SD and KP
supervised code and method refinement and presentation of results. All authors
reviewed the manuscript.

## References

  * [1] Danny Broberg et al.  “PyCDT: A Python toolkit for modeling point defects in semiconductors and insulators”  In _Computer Physics Communications_ , 2018  DOI: 10.1016/j.cpc.2018.01.004
  * [2] Anuj Goyal et al.  “A computational framework for automation of point defect calculations”  In _Computational Materials Science_ 130, 2017, pp. 1 –9  DOI: https://doi.org/10.1016/j.commatsci.2016.12.040
  * [3] Mit H. Naik and Manish Jain  “CoFFEE: Corrections For Formation Energy and Eigenvalues for charged defect simulations”  In _Computer Physics Communications_ 226, 2018, pp. 114 –126  DOI: https://doi.org/10.1016/j.cpc.2018.01.011
  * [4] Stephan Lany and Alex Zunger  “Assessment of correction methods for the band-gap problem and for finite-size effects in supercell defect calculations: Case studies for ZnO and GaAs”  In _Physical Review B - Condensed Matter and Materials Physics_ 78.23, 2008  DOI: 10.1103/PhysRevB.78.235104
  * [5] Christoph Freysoldt et al.  “First-principles calculations for point defects in solids”  In _Reviews of Modern Physics_ 86.1, 2014, pp. 253–305  DOI: 10.1103/RevModPhys.86.253
  * [6] Christoph Freysoldt, Jörg Neugebauer and Chris G. Van De Walle  “Fully Ab initio finite-size corrections for charged-defect supercell calculations”  In _Physical Review Letters_ , 2009  DOI: 10.1103/PhysRevLett.102.016402
  * [7] Yu Kumagai and Fumiyasu Oba  “Electrostatics-based finite-size corrections for first-principles point defect calculations”  In _Physical Review B - Condensed Matter and Materials Physics_ , 2014  DOI: 10.1103/PhysRevB.89.195205
  * [8] Hannu Pekka Komsa, Tapio Rantala and Alfredo Pasquarello  “Comparison between various finite-size supercell correction schemes for charged defect calculations”  In _Physica B: Condensed Matter_ 407.15, 2012, pp. 3063–3067  DOI: 10.1016/j.physb.2011.08.028
  * [9] Enrique R. Batista et al.  “Comparison of screened hybrid density functional theory to diffusion Monte Carlo in calculations of total energies of silicon phases and defects”  In _Phys. Rev. B_ 74 American Physical Society, 2006, pp. 121102  DOI: 10.1103/PhysRevB.74.121102
  * [10] Peter Deák et al.  “Accurate defect levels obtained from the HSE06 range-separated hybrid functional”  In _Phys. Rev. B_ 81 American Physical Society, 2010, pp. 153203  DOI: 10.1103/PhysRevB.81.153203
  * [11] Hannu-Pekka Komsa and Alfredo Pasquarello  “Assessing the accuracy of hybrid functionals in the determination of defect levels: Application to the As antisite in GaAs”  In _Phys. Rev. B_ 84 American Physical Society, 2011, pp. 075207  DOI: 10.1103/PhysRevB.84.075207
  * [12] P. M. Fahey, P. B. Griffin and J. D. Plummer  “Point defects and dopant diffusion in silicon”  In _Rev. Mod. Phys._ 61 American Physical Society, 1989, pp. 289–384  DOI: 10.1103/RevModPhys.61.289
  * [13] Klaus Graff  “Metal Impurities in Silicon-Device Fabrication”  In _Springer Series in Materials Science_ 24 Berlin: Springer, 1995, pp. 143
  * [14] R. Böhm and H. Klose  “Copper in silicon n+-p junctions”  In _physica status solidi (a)_ 9.2, pp. K165–K168  DOI: 10.1002/pssa.2210090259
  * [15] Yina Mo, Martin Z. Bazant and Efthimios Kaxiras  “Sulfur point defects in crystalline and amorphous silicon”  In _Phys. Rev. B_ 70 American Physical Society, 2004, pp. 205210  DOI: 10.1103/PhysRevB.70.205210
  * [16] Clas Persson, Yu-Jun Zhao, Stephan Lany and Alex Zunger  “$n$-type doping of $\mathrm{CuIn}{\mathrm{Se}}_{2}$ and $\mathrm{CuGa}{\mathrm{Se}}_{2}$”  In _Phys. Rev. B_ 72 American Physical Society, 2005, pp. 035211  DOI: 10.1103/PhysRevB.72.035211
  * [17] Audrius Alkauskas, Peter Broqvist and Alfredo Pasquarello  “Defect levels through hybrid density functionals: Insights and applications”  In _physica status solidi (b)_ 248.4, 2011, pp. 775–789  DOI: 10.1002/pssb.201046195
  * [18] S. B. Zhang, S.-H. Wei and Alex Zunger  “Intrinsic n-type versus p-type doping asymmetry and the defect physics of ZnO”  In _Phys. Rev. B_ 63 American Physical Society, 2001, pp. 075205  DOI: 10.1103/PhysRevB.63.075205
  * [19] Anderson Janotti and Chris G. Walle  “Oxygen vacancies in ZnO”  In _Applied Physics Letters_ 87.12, 2005, pp. 122102  DOI: 10.1063/1.2053360
  * [20] P. Bogusławski, E. L. Briggs and J. Bernholc  “Native defects in gallium nitride”  In _Phys. Rev. B_ 51 American Physical Society, 1995, pp. 17255–17258  DOI: 10.1103/PhysRevB.51.17255
  * [21] W. Kohn and L. J. Sham  “Self-consistent equations including exchange and correlation effects”  In _Physical Review_ 140.4A, 1965  DOI: 10.1103/PhysRev.140.A1133
  * [22] Piotr Śpiewak and Krzysztof J. Kurzydłowski  “Formation and migration energies of the vacancy in Si calculated using the HSE06 range-separated hybrid functional”  In _Phys. Rev. B_ 88 American Physical Society, 2013, pp. 195204  DOI: 10.1103/PhysRevB.88.195204
  * [23] C. Jagannath, Z. W. Grabowski and A. K. Ramdas  “Linewidths of the electronic excitation spectra of donors in silicon”  In _Phys. Rev. B_ 23 American Physical Society, 1981, pp. 2082–2098  DOI: 10.1103/PhysRevB.23.2082
  * [24] David W. Fischer and John J. Rome  “Additional structure in infrared excitation spectra of group-III acceptors in silicon”  In _Phys. Rev. B_ 27 American Physical Society, 1983, pp. 4826–4832  DOI: 10.1103/PhysRevB.27.4826
  * [25] Abhishek Sharan, Zhigang Gui and Anderson Janotti  “Hybrid-functional calculations of the copper impurity in silicon”  In _Physical Review Applied_ , 2017  DOI: 10.1103/PhysRevApplied.8.024023
  * [26] Shyue Ping Ong et al.  “Python Materials Genomics (pymatgen): A robust, open-source python library for materials analysis”  In _Computational Materials Science_ , 2013  DOI: 10.1016/j.commatsci.2012.10.028
  * [27] Aliaksandr V. Krukau, Oleg A. Vydrov, Artur F. Izmaylov and Gustavo E. Scuseria  “Influence of the exchange screening parameter on the performance of screened hybrid functionals”  In _Journal of Chemical Physics_ , 2006  DOI: 10.1063/1.2404663
  * [28] G. D. Watkins and J. R. Troxell  “Negative-U properties for point defects in silicon”  In _Physical Review Letters_ , 1980  DOI: 10.1103/PhysRevLett.44.593
  * [29] Gerhard Pensl, G. Roos, C. Holm and P. Wagner  “Chalcogen Double Donors in Silicon”  In _Defects in Semiconductors 14_ 10, Materials Science Forum  Trans Tech Publications, 1986, pp. 911–916  DOI: 10.4028/www.scientific.net/MSF.10-12.911
  * [30] S Knack, J Weber and H Lemke  “Copper–hydrogen complexes in silicon”  In _Physica B: Condensed Matter_ 273-274, 1999, pp. 387 –390  DOI: https://doi.org/10.1016/S0921-4526(99)00486-X
  * [31] Nikolai Yarykin and Jörg Weber  “Deep levels of copper-hydrogen complexes in silicon”  In _Phys. Rev. B_ 88 American Physical Society, 2013, pp. 085205  DOI: 10.1103/PhysRevB.88.085205
  * [32] Darshana Wickramaratne et al.  “Defect identification based on first-principles calculations for deep level transient spectroscopy”  In _Applied Physics Letters_ 113.19, 2018, pp. 192106  DOI: 10.1063/1.5047808
  * [33] John D. Hunter  “Matplotlib: A 2D graphics environment”  In _Computing in Science and Engineering_ , 2007  DOI: 10.1109/MCSE.2007.55
  * [34] George D. Watkins  “Negative-U properties for defects in solids”  In _Advances in Solid State Physics: Plenary Lectures of the 48th Annual Meeting of the German Physical Society (DPG) and of the Divisions “Semiconductor Physics” “Metal Physics” “Low Temperature Physics” “Thermodynamics and Statistical Physics” “Thin Films” “Surface Physics” “Magnetism” “Physics of Polymers” “Molecular Physics” Münster, March 12…17, 1984_ Berlin, Heidelberg: Springer Berlin Heidelberg, 1984, pp. 163–189  DOI: 10.1007/BFb0107450
  * [35] A F Wright  “Density-functional-theory calculations for the silicon vacancy”  In _Phys. Rev. B_ 74.16 American Physical Society, 2006, pp. 165116  DOI: 10.1103/PhysRevB.74.165116
  * [36] S. Behnel et al.  “Cython: The Best of Both Worlds”  In _Computing in Science Engineering_ 13.2, 2011, pp. 31 –39  DOI: 10.1109/MCSE.2010.118
  * [37] Leonardo Dagum and Ramesh Menon  “OpenMP: An Industry-Standard API for Shared-Memory Programming”  In _IEEE Comput. Sci. Eng._ 5.1 Los Alamitos, CA, USA: IEEE Computer Society Press, 1998, pp. 46–55  DOI: 10.1109/99.660313
  * [38] G. Kresse and J. Hafner  “Ab initio molecular dynamics for liquid metals”  In _Phys. Rev. B_ 47 American Physical Society, 1993, pp. 558–561  DOI: 10.1103/PhysRevB.47.558
  * [39] R. M. Feenstra and M. Widom  “WaveTrans: Real-space wavefunctions from VASP WAVECAR file”  URL: https://www.andrew.cmu.edu/user/feenstra/wavetrans/
  * [40] Eric Jones, Travis Oliphant, Pearu Peterson and Others  “SciPy: Open source scientific tools for Python”  In _Computing in Science and Engineering_ , 2007  DOI: 10.1143/JJAP.31.4110
  * [41] Aaron Meurer et al.  “SymPy: symbolic computing in Python”  In _PeerJ Computer Science_ , 2017  DOI: 10.7717/peerj-cs.103
  * [42] J.D. Talman  “NumSBT: A subroutine for calculating spherical Bessel transforms numerically”  In _Computer Physics Communications_ 180.2, 2009, pp. 332 –338  DOI: https://doi.org/10.1016/j.cpc.2008.10.003
  * [43] P. E. Blochl  “Projector augmented-wave method”  In _Phys. Rev. B_ 50 American Physical Society, 1994, pp. 17953–17979  DOI: 10.1103/PhysRevB.50.17953
  * [44] G. Kresse and J. Hafner  “Ab initio molecular-dynamics simulation of the liquid-metal-amorphous-semiconductor transition in germanium”  In _Phys. Rev. B_ 49 American Physical Society, 1994, pp. 14251–14269  DOI: 10.1103/PhysRevB.49.14251
  * [45] G. Kresse and J. Furthmüller  “Efficiency of ab-initio total energy calculations for metals and semiconductors using a plane-wave basis set”  In _Computational Materials Science_ 6.1, 1996, pp. 15 –50  DOI: https://doi.org/10.1016/0927-0256(96)00008-0
  * [46] G. Kresse and J. Furthmüller  “Efficient iterative schemes for ab initio total-energy calculations using a plane-wave basis set”  In _Phys. Rev. B_ 54 American Physical Society, 1996, pp. 11169–11186  DOI: 10.1103/PhysRevB.54.11169
  * [47] John P. Perdew, Kieron Burke and Matthias Ernzerhof  “Generalized Gradient Approximation Made Simple”  In _Phys. Rev. Lett._ 77 American Physical Society, 1996, pp. 3865–3868  DOI: 10.1103/PhysRevLett.77.3865
  * [48] John P. Perdew, Kieron Burke and Matthias Ernzerhof  “Generalized Gradient Approximation Made Simple [Phys. Rev. Lett. 77, 3865 (1996)]”  In _Phys. Rev. Lett._ 78 American Physical Society, 1997, pp. 1396–1396  DOI: 10.1103/PhysRevLett.78.1396
  * [49] G. Kresse and D. Joubert  “From ultrasoft pseudopotentials to the projector augmented-wave method”  In _Phys. Rev. B_ 59 American Physical Society, 1999, pp. 1758–1775  DOI: 10.1103/PhysRevB.59.1758
  * [50] Jochen Heyd, Gustavo E. Scuseria and Matthias Ernzerhof  “Hybrid functionals based on a screened Coulomb potential”  In _Journal of Chemical Physics_ , 2003  DOI: 10.1063/1.1564060
  * [51] Anubhav Jain et al.  “The Materials Project: A materials genome approach to accelerating materials innovation”  In _APL Materials_ 1.1, 2013, pp. 011002  DOI: 10.1063/1.4812323
  * [52] W. Bludau, A. Onton and W. Heinke  “Temperature dependence of the band gap of silicon”  In _Journal of Applied Physics_ 45.4, 1974, pp. 1846–1848  DOI: 10.1063/1.1663501


# Exploring Highly Quantised Neural Networks for Intrusion Detection in
Automotive CAN

Shashwat Khandelwal & Shanker Shreejith  Reconfigurable Computing Systems Lab,
Electronic & Electrical Engineering
Trinity College Dublin, Ireland
Email: {khandels<EMAIL_ADDRESS>
###### Abstract

Vehicles today comprise intelligent systems like connected autonomous driving
and advanced driving assistance systems (ADAS) to enhance the driving
experience, which is enabled through increased connectivity to infrastructure
and fusion of information from different sensing modes. However, the rising
connectivity coupled with the legacy network architecture within vehicles can
be exploited for launching active and passive attacks on critical vehicle
systems and directly affecting the safety of passengers. Machine learning-
based intrusion detection models have been shown to successfully detect
multiple targeted attack vectors in recent literature, whose deployments are
enabled through quantised neural networks targeting low-power platforms.
Multiple models are often required to simultaneously detect multiple attack
vectors, increasing the area, (resource) cost, and energy consumption. In this
paper, we present a case for utilising custom-quantised MLP’s (CQMLP) as a
multi-class classification model, capable of detecting multiple attacks from
the benign flow of controller area network (CAN) messages. The specific
quantisation and neural architecture are determined through a joint design
space exploration, resulting in our choice of the 2-bit precision and the
n-layer MLP. Our 2-bit version is trained using Brevitas and optimised as a
dataflow hardware model through the FINN toolflow from AMD/Xilinx, targeting
an XCZU7EV device. We show that the 2-bit CQMLP model, when integrated as the
IDS, can detect malicious attack messages (DoS, fuzzing, and spoofing attack)
with a very high accuracy of 99.9%, on par with the state-of-the-art methods
in the literature. Furthermore, the dataflow model can perform line rate
detection at a latency of 0.11 ms from message reception while consuming 0.23
mJ/inference, making it ideally suited for integration with an ECU in critical
CAN networks.

###### Index Terms:

Controller Area Network, Intrusion Detection System, Quantised Neural Nets,
Machine Learning, FPGAs

##  I Introduction & Related Works

Automotive networks are evolving rapidly to cater to the high-data needs for
novel/intelligent capabilities that have enabled safety, infotainment, and
comfort for passengers. These interconnected networks (via multiple network
protocols) facilitate high-speed communication/information exchange between
over a hundred electronic computing units (ECUs), sensors, and actuators
present in modern cars. Among the network protocols, Controller Area Network
(CAN) is used for critical communication between ECUs and it continues to be
the most widely used network protocol for in-vehicle networks owing to its
cost-effective nature and ease of use in control applications [1]. Early ECUs
and software functions deployed on the CAN were automatically siloed due to
the limited connectivity of the vehicles then, to the external world. Novel
capabilities in vehicles today rely on connectivity to infrastructure and
other vehicles to enable real-time sensing of the outside environment and also
to enable unique features like remote monitoring and control of specific
capabilities for diagnostics, and over-the-air upgrades. Researchers however
have shown that such interfaces open up new avenues for injecting malicious
code/messages into these previously siloed networks [2, 3, 4]. Figure 1 shows
an example of a simple DoS attack on a standard CAN bus. Such attacks are
largely enabled by the lack of inherent security and authentication mechanisms
in CAN and similar automotive network protocols [5, 6].

((a)) Figure 1: An illustration of a simple DoS attack launched through a
compromised ECU. Part (a) shows the normal communication between the ECUs,
while in (b), the ECU is unable to transmit messages on the CAN bus as the
compromised ECU floods the bus with high priority messages.

]

To detect the onset of such intrusions on CAN, multiple intrusion detection
approaches have been proposed in the literature allowing critical systems to
enter into a ‘safe working’ mode when such threats are detected. Early
intrusion detection systems (IDS) proposed were rule/specification based,
which utilised a set of rules to compare known attack signatures to patterns
captured from current network parameters/messages to detect unusual activity
[7, 8, 9]. Recently, machine learning (ML) approaches have shown significant
improvement in the detection accuracy of such threats and the ability to adapt
to newer attack vectors [10, 11, 12, 13, 14] without incurring overheads of
rule-based approaches. In addition to native precision ML models, quantised
variants of the floating-point models have shown to be viable alternatives,
reducing the computational complexity, resource, and energy consumption at the
expense of a slight reduction in inference accuracy. Quantised models are then
deployed on constrained devices such as low-end FPGAs or dedicated embedded
platforms [15]. Frameworks such as Vitis-AI and FINN can convert the
Pytorch/TensorFlow representations at native precisions to quantised models
and dataflow-style hardware accelerators for deployment on the FPGA [16, 17].
ECU architectures based on hybrid FPGAs are a promising platform for the
deployment of such QNNs as IDS, while also enabling consolidation with
standard ECU function(s), with clear isolation between them on the same die
[18]. However, most methods so far relied on the integration of multiple
accelerators, each fine-tuned to detect a subset of attacks, increasing the
energy cost and area overhead.

In this paper, we explore the case for a custom quantised feed-forward network
that can classify multiple attack vectors simultaneously and a dataflow-style
custom quantised hardware implementation of the model. The dedicated hardware
implements IDS capability as an Advanced eXtensible Interface (AXI) slave
peripheral to the ARM cores, offering complete isolation from the software
tasks on them, mimicking the model ECU architecture for deploying distributed
IDS in CAN networks. Starting from a standard multi-layer perceptron (MLP)
model, we analyse and evaluate different model configurations and
quantisations for weights/biases/activations to arrive at our custom model
with 2-, 3- & 4-bit precisions through a constrained design space exploration.
Subsequently, the dataflow accelerator is generated and deployed using
AMD/Xilinx’s FINN framework [17]. This integration allows the software tasks
on the ECU to invoke and fully control the operation of the IDS accelerator
through APIs, similar to offloads enabled by AUTOSAR abstractions. The key
contributions of this paper are as follows:

  * •

A feed-forward custom quantised multi-layer-perceptron based-IDS for
automotive CAN achieving state-of-the-art classification accuracy across
multiple attack vectors using only a single model.

  * •

Exploits the tightly-coupled ECU architecture with the dataflow implementation
of IDS accelerating IDS operation in full isolation.

  * •

Quantify the performance and energy savings of the proposed ML model and its
integration using the open CAN dataset. Our results show that the proposed IDS
achieves significant improvements in terms of per-message processing latency
and power consumption against the state-of-the-art IDSs in the literature.

We evaluate our approach using the openly available CAR Hacking dataset [13]
for training and validation across multiple attack vectors captured from an
actual vehicle with the entire CAN data frame used as an input feature to
improve the detection performance. We analyse the 2-, 3- & 4-bit precision
models and utilise the inference accuracy and resource utilisation parameters
to arrive at the most optimal model. Our experiments show that the proposed
custom quantised (2-bit) MLP-based IDS (referred to as CQMLP-IDS) achieves an
average accuracy of 99.91% across Denial of Service (DoS), Fuzzing, and RPM-
spoofing attacks among the benign flow of messages, identical to or exceeding
the detection accuracy achieved by state-of-the-art GPU- and CPU-based
implementations. The tightly integrated ECU architecture reduces the per
message execution latency by 2.2$\times$ and the energy consumed by
3.9$\times$ compared to state-of-the-art IDSs proposed in the research
literature. We also show that our (fp32) MLP model on a Jetson Nano [19],
mimicking a dedicated IDS ECU, incurs 12$\times$ higher energy consumption per
inference, compared to the integrated IDS-ECU.

The remainder of the paper is organised as follows: Section II captures
background information and state-of-the-art research in this area; section III
describes the proposed MLP model and design choices for the implementation;
section IV outlines the experiment setup and results; and we conclude the
paper in section V.

##  II Background and Related works

###  II-A Controller Area Network

In-vehicle networks enable distributed ECUs to exchange control and data
messages to achieve the global functions of the vehicle. Multiple protocols
are used in vehicular systems to cater to different functions based on their
criticality and to optimise the cost of E/E systems. CAN [1] and CAN-FD [20]
continue to be the most widely used protocol today due to their lower cost,
flexibility, and robustness. The broadcast CAN bus uses carrier sense multiple
access with the collision avoidance and arbitration priority (CSMA/CA-AP)
access/arbitration mechanism to control access to the bus using the CAN ID
allocated to each message. This enables higher priority messages to be
processed first hence efficiently handling messages from safety-critical
applications. CAN also supports multiple data rates (125 Kbps to 1 Mbps) and
multiple modes of operation (1-wire, 2-wire) to cater to a range of critical
and non-critical functions in vehicles. Despite this robustness, CAN is
inherently insecure: there is no built-in mechanism (like message encryption)
in the network to authenticate the transmitter, receiver, or the message
content itself [21]. This makes CAN vulnerable to simple yet efficient active
message injection attacks like fuzzing, spoofing, replay attacks, and Denial
of Service (DoS) attacks [22, 6, 23, 24].

Researchers have explored different flavours of IDSs from rule-based
approaches to machine learning-based methods to address CAN’s existing
vulnerabilities. Rule-based approaches are further classified into flow-based
and payload-based. While flow-based approaches identify traits like message
frequency and/or interval for the network to detect abnormalities [25],
payload-based approaches use the data segment in CAN frames to detect abnormal
sequences of instructions and or data [26]. Purely payload-based methods,
however, fail to capture the timing/sequence of messages in an attack, leading
to parts of attack messages being flagged as benign. Hybrid schemes use both
the message timing/frequency and the payload information to capture a more
holistic view of the network, allowing them to extract specific signatures of
transmitting ECUs, receiving ECUs, and messages [27]. Machine learning-based
techniques further generalises such techniques when trained with large
datasets through classification approaches, sequential techniques, and deep
learning-based schemes.

###  II-B Machine Learning based IDSs

ML-based IDSs discussed in the literature mostly use a combination of input
features, as shown in table I. In [13], the authors propose a reduced
inception net architecture as an IDS using deep convolutional neural networks.
They achieve over 99% accuracy for DoS, fuzzing & spoofing attacks. In [28],
authors present a graph based intrusion detection protocol and propose
improvements over existing graph based IDS techniques. In [29], a transformer
network-based IDS is proposed which demonstrates very high classification
accuracy at the cost of higher detection latency. In [30], the authors propose
a GAN-based IDS and achieve an average accuracy of 97.5% for the same attacks.
More complex ML architectures like temporal convolution with global attention
[31], a combination of convolutional neural networks (CNN) and long short-term
memory (LSTM) cells using both supervised and unsupervised approaches [32,
33], gated recurrent units (GRU) networks [34] have been shown to improve
detection accuracy. In [35], the authors use an iForest anomaly detection
algorithm as an intrusion prevention system (IPS) to detect fuzzing and
spoofing (RPM & Gear) attacks and mark the message as an error preventing its
propagation to other ECUs; however, this can cause multiple messages to be
dropped from the bus in case of false positives or DoS attacks. The key
challenge of ML-based approaches is their deployment as an in-vehicle ECU.
Most approaches rely on high-performance GPUs to meet the inference deadlines
[30, 13, 36, 34, 32, 31], while others rely on dedicating full ECUs for IDS
[37, 35]; both approaches incur additional overheads in energy budget and
weight, making them less suited for distributing IDS among different network
segments. Similarly, almost all methods presented in the literature require
multiple uniquely trained models to detect all possible threat vectors, which
incur much higher resources and energy than a singular model which can
classify multiple threat vectors.

TABLE I: Input features used by the IDSs & IPSs proposed in the research literature. Models | Input Features Used
---|---
GIDS [30] | CAN ID
DCNN [13] | CAN ID
Rec-CNN [38] | CAN ID
G-IDCS [28] | CAN ID
TAN-IDS [29] | CAN ID
iForest [35] | Data Field
MLIDS [36] | CAN ID + Data Field
NovelADS [32] | CAN ID + Data Field
TCAN-IDS [31] | CAN ID + Data Field
MTH-IDS [37] | CAN ID + Data Field
HyDL-IDS [33] | CAN ID + Data Field + DLC
GRU [34] | CAN ID + Data Field + DLC
CQMLP-IDS (proposed) | CAN ID + Data Field

##  III System architecture

Figure 2: The proposed MLP model as a multi-attack detection IDS.

###  III-A MLP model for IDS

To determine the best ML model for IDS, we profiled different ML architectures
to find a model with high classification accuracy which also offers low
computational complexity. We observed that while CNNs were effective for
classification when using only CAN IDs, MLPs provided more accurate detection
(accuracy, false positives, and false negatives) when using the entire message
contents at much lower computational complexity; since message data can
contain malicious content, we chose MLPs as our choice of architecture and
entire message as the input feature [39, 40]. A concatenation of n = 4
successive messages (CAN IDs + payload) is chosen as the input feature based
on this testing. Note that during training/testing, if the block has even one
attack message, the entire block is treated as an attack block. In a real
system, this means that a window of 4 messages containing a single attack
message will be detected as a malicious window, allowing a sequence of events
leading to and after the attack message as potential threats. Though
conservative, this approach enables the model to better capture normal
operating sequences during training and provides a better balance to the
dataset. We observed that using a block with a higher number of messages ( $>$
4 ) improves the classification accuracy as the model has more input data to
make a decision; however, grouping more messages also affects the detection
latency from the arrival of the ‘infected’ message, leading to our choice of n
= 4.

Our final MLP model consists of 5 _Linear_ layers implemented with the 256,
128, 64, 32 & 4 units at each layer as shown in Figure 2. The time-series data
from the input feature buffer is fed as input to the input layer with 40
units. Subsequent layer(s) operate on the output of the previous layer, with
increasing complexity to perform classification. Batch Normalisation was used
between the _linear_ layers to prevent over-fitting and to improve the
learning efficiency during the training phase. The output _linear_ layer uses
a _softmax_ activation at the output to estimate the probability of the
message being benign, DoS, fuzzing, or RPM-spoofed. The model is defined in
Pytorch using standard functions and nodes with the quantised versions of
linear and relu activation layers imported from the brevitas training library.

###  III-B Design space exploration

To arrive at the optimum bit-precision for the quantised model, we trained our
final MLP model at 2-bit, 3-bit, and 4-bit uniform quantisation on all the
attacks to determine the trade-offs in detection accuracy and resource
consumption (indirectly leading to power consumption and latency). Figures 3 &
4 show the training and validation losses when the model was trained for 1000
epochs at each quantisation. The 2-bit model converges to an optimum detection
and is very close to the 3-bit & 4-bit variants (between epoch 810 and 820) in
terms of validation loss. We observed _147_ misclassifications out of the
total 180,000 messages in the test set for the 2-bit variant. For the same
test set, we found misclassifications to be _74_ & _31_ for the 3-bit & 4-bit
variants respectively.

To capture the inference costs of the models further, we use the FINN utility
function (inference cost) to estimate the memory footprint and operational
complexity of each model. This function captures the memory footprint and
binary operations cost for each version of the model; the costs are then
normalised (against a baseline) and linearly combined to arrive at the
normalised inference cost for each of the three quantisation levels. Table II
shows these costs with the 4-bit version as the baseline. We find the cost of
the 2-bit version is 40% & 27% of the 3-bit & 4-bit version respectively,
without a significant drop in detection accuracy and was hence chosen as the
optimal bitwidth for deploying the CQMLP.

TABLE II: Inference cost of the custom quantised models (CQMLP) inferred
through the FINN library and normalised to the 4-bit model.

CQMLP Models | 2-bit | 3-bit | 4-bit
---|---|---|---
Normalised inference cost | 0.27 | 0.67 | 1

###  III-C Dataset and Training

We use the open Car Hacking dataset for training our model and to test its
performance [41]. The dataset provides a labelled set of benign and attack
messages which were captured via the Onboard Diagnostic (OBD) port in an
actual vehicle, with attack messages injected in real-time. The dataset
includes DoS, Fuzzing, and RPM-Spoofing message injections allowing us to
validate the detection accuracy across these different attacks. We split the
dataset as 85:10:5 for training, validation, and testing respectively,
allocating the large section to training and optimisation of the quantised
network. The performance of the model on the validation set during training
ensures that it is not over-fitting on any of the attacks. We pre-process the
dataset before training to mimic the dataflow the model will obtain as its
input when integrated into the ECU. Each message (ID and payload) is encoded
into INT8 type vector, and an adjacent sequence of n = 4 of these encoded
messages form the input feature buffer content for the IDS (block shape of
{1,10*n} INT8 data).

The different bit models were trained using Brevitas, a quantisation-aware
training (QAT) library from AMD/Xilinx[42]. We used the adam optimizer with
binary cross-entropy as the loss function. The learning rate was set to 0.0001
to allow for slower learning which aids in efficient model training [43]. Each
model has 54,824 learning parameters and was trained for 1000 epochs with a
batch size of 128 for the DoS, Fuzzing, and RPM-Spoof attacks. The best model
in terms of validation loss was saved and exported as an ’onnx’ graph. This
graph was then fed into the FINN hardware generation flow for the CQMLP-IDS IP
generation.

Figure 3: Training loss of the model for different precision of weights and
activations for all the attacks. Figure 4: Validation loss of the model for
different precision of weights and activations for all the attacks.

###  III-D Dataflow hardware generation and integration to ECU ECU-IDS

We utilise the FINN compiler to synthesize and generate the RTL description of
the hardware implementation of the model. Starting at the ’onnx’ graph, FINN
applies a series of streamlining and dataflow transformations to convert the
ML model into a synthesizable graph. We further specify the parallelism to
achieve our target latency (set to 100000 messages per second), matrix-vector
activation unit widths (set to 80), and folding levels to generate the
dataflow hardware with AXI interfaces for input and output. Subsequently, FINN
invokes Vivado tools to stitch the accelerator as an AXI slave to a Zynq
subsystem for our target device (XCZU7EU) and generates the final bitstream
along with the software drivers for our hardware ML engine. This integration
allows us to model the Zynq-based hybrid ECU architecture, where our custom
hardware is accessible from the ECU and performs IDS in complete isolation
from the tasks on the ARM cores as shown in Figure 5. We build on the proposed
architecture from our previous works which contain detailed information of the
same [18, 44]. For our experiments, we use the PYNQ runtime on top of a Linux
kernel on the ARM cores to interact with our hardware model on the PL.

Figure 5: The proposed IDS-ECU architecture. The ML model is accelerated on
the PL part of the FPGA.

##  IV Experimental Results

To quantify the performance and energy benefits of a dedicated dataflow
accelerator implementing our CQMLP-IDS integrated with ECU function(s), we use
a Zynq Ultrascale+ ZCU104 development board as the test ECU platform. The
ZCU104 board features an XCZU7EV Ultrascale+ device with quad-core ARM A53
cores and dual-core ARM R5 cores on the PS as our target platform. The A53
cores on the PS are configured to run at 1.2 GHz peak, and the dataflow
hardware was synthesized for a clock frequency of 200 MHz by FINN. The ARM
cores run Linux OS with petalinux tools and PYNQ runtime to provide APIs to
interface to the CQMLP-IDS IP in the PL. When testing, the hardware CQMLP
model is fed with inputs message from the ECU mimicking the standard message
flow within ECUs, and use our test split from the Car Hacking dataset to
perform inference. We quantify the inference accuracy by evaluating the
precision, recall, and F1 rates of our CQMLP models when inference is
performed on the Zynq-IDS-ECU. We also quantify the per-message processing
latency and power consumption for each incoming CAN message on the Zynq-IDS-
ECU. The results, in terms of inference accuracy and message processing
latency, are compared against the state-of-the-art IDSs/IPSs presented in the
research literature. We also compare the per-block processing latency and the
per-inference energy consumption when the FP32 version of the proposed model
is executed on an NVIDIA Jetson Nano 4GB GPU. In the case of schemes where
inference is performed on a block of CAN messages, we use these metrics along
with the block size for the comparison. We also compare our active power
consumption against ML-based IDS approaches in literature where power
consumption has been reported.

###  IV-A Inference accuracy

We evaluate the classification performance of the model in identifying DoS,
Fuzzy and RPM-Spoofing attacks from the Car Hacking dataset and compare them
against state-of-the-art techniques in the literature. Table III captures the
classification performance of our model in isolation across our test set as a
confusion matrix. The misclassifications in our test set can be attributed to
scenarios where nearly identical attack and benign message patterns are
observed in a block at a given time which leads to an overall false positive
rate (FPR) of 0.03% (_34 messages falsely classified as attacks out of the
total 103176 benign messages_). We compare the inference performance of our
2-bit CQMLP model integrated within the ECU against the state-of-the-art IDSs
and IPSs proposed in the literature: GIDS [30], DCNN [13], MLIDS [36], HyDL-
IDS [33] NovelADS [32], TCAN-IDS [31], IForest [35], MTH-IDS [37], GRU [34],
G-IDCS [28], TAN-IDS [29] and Rec-CNN [38] which are captured in table IV,
comparing them in terms of inference precision, recall, F1 score and false
negative rate. For the DoS attack the 2-bit CQMLP performs equally among the
proposed IDSs in terms of F1 score. In the case of fuzzing attack, the 2-bit
CQMLP model performs identically to DCNN [13] while performing better than
iForest [35], TCAN-IDS [31], and GRU [34] by 2.3%, 0.6% and 0.6% respectively
in terms of F1 score. The 2-bit CQMLP also achieves almost perfect
classification for the RPM-spoofing attack and performs better than or equal
to the other methods proposed in the literature. Our prior work using two
8-bit variants of the feed-forward model achieved 99.96%, 99.76% & 100% F1
scores for the DoS, Fuzzing, and RPM spoofing attacks, when deployed as two
concurrent accelerators for detecting them simultaneously [18]. In contrast,
our 2-bit CQMLP variant achieves almost identical F1 scores (99.90%, 99.81% &
99.98% respectively) for the same attacks (from the same dataset) while
performing multi-class classification using a single inference model. By
adopting QAT training and conservative block-based labelling, our low
precision model is able to match the inference performance of the MLP variant
at 8-bit quantisation discussed in [18].

TABLE III: Confusion matrix capturing the classification results of the CQMLP.

| Predicted Values
---|---
CQMLP | Benign | DoS | Fuzzing | RPM-Spoof
Benign | 103142 | 17 | 17 | 0
DoS | 27 | 23666 | 0 | 0
Fuzzing | 78 | 0 | 28003 | 8
RPM-Spoof | 0 | 0 | 0 | 25042

TABLE IV: Accuracy metric comparison (%) of our CQMLP accelerators against the
reported literature on the DoS, Fuzzing and RPM-Spoof attacks.

Attack | Model | Precision | Recall | F1 | FNR
---|---|---|---|---|---
DoS | GIDS [30] | - | 99.9 | - | -
DCNN [13] | 100 | 99.89 | 99.95 | 0.13
MLIDS [36] | 99.9 | 100 | 99.9 | -
G-IDCS [28] | 99.81 | 98.86 | 99.33 | -
TAN-IDS [29] | 100 | 100 | 100 | -
| HyDL-IDS [33] | 100 | 100 | 100 | 0
| NovelADS [32] | 99.97 | 99.91 | 99.94 | -
| TCAN-IDS [31] | 100 | 99.97 | 99.98 | -
| iForest [35] | - | - | - | -
| GRU [34] | 99.93 | 99.91 | 99.92 | -
| CQMLP-IDS | 99.92 | 99.88 | 99.90 | 0.11
Fuzzing | GIDS [30] | - | 98.7 | - | -
DCNN [13] | 99.95 | 99.65 | 99.80 | 0.5
MLIDS [36] | 99.9 | 99.9 | 99.9 | -
G-IDCS [28] | 99.71 | 99 | 99.35 | -
TAN-IDS [29] | 99.99 | 99.99 | 99.99 | -
| HyDL-IDS [33] | 99.98 | 99.88 | 99.93 |
| NovelADS [32] | 99.99 | 100 | 100 | -
| TCAN-IDS [31] | 99.96 | 99.89 | 99.22 | -
| iForest [35] | 95.07 | 99.93 | 97.44 | -
| GRU [34] | 99.32 | 99.13 | 99.22 | -
| CQMLP-IDS | 99.93 | 99.69 | 99.81 | 0.27
RPM-Spoof | GIDS [30] | - | 99.6 | - | -
DCNN [13] | 99.99 | 99.94 | 99.96 | 0.05
MLIDS [36] | 100 | 100 | 100 | -
G-IDCS [28] | 99.85 | 98.69 | 99.27 | -
TAN-IDS [29] | 99.99 | 99.93 | 99.96 | -
| HyDL-IDS [33] | 100 | 100 | 100 | 0
| NovelADS [32] | 99.9 | 99.9 | 99.9 | -
| TCAN-IDS [31] | 99.9 | 99.9 | 99.9 | -
| iForest [35] | 98.9 | 100 | 99.4 | -
| CQMLP-IDS | 99.96 | 100 | 99.98 | 0

###  IV-B Inference latency

We quantify the per-message processing latency of the model, starting from the
arrival of the CAN message at the interface to determine the detection delay
incurred by the approach. Table V compares our result against other approaches
in the literature, which utilise different platforms (GPUs, Jetson edge
accelerators, Raspberry Pi) and approaches (block of CAN messages v/s
individual messages). It should be noted that the latency metric in the case
of block-based IDS does not consider the delay in acquiring the number of CAN
messages required, and hence could result in potentially larger delays in
detection of the onset of an attack. The tightly integrated CQMLP-IDS achieves
0.11 ms per CAN frame, which is a 2.18$\times$ improvement over the dedicated
line-rate QMLP-IDS ECU proposed in [18]. We also observe the inference latency
of the (fp32) model on the Jetson Nano (modeling a dedicated IDS ECU) to be
1.2 ms when averaged over 10000 runs. This is 10.9$\times$ slower than the
proposed CQMLP-IDS on the integrated IDS-ECU on the Zynq Ultrascale platform.
In terms of raw throughput, our dataflow implementation of CQMLP coupled to
the ECU can process over 9090 messages per second at the highest payload
capacity, achieving near-line-rate detection on high-speed critical CAN
networks.

TABLE V: Per-message latency comparison against other state-of-the-art IDSs
reported in literature.

Models | Latency | Frames | Platform
---|---|---|---
GRU [34] | 890 ms | 5000 CAN frames | Jetson Xavier NX
MLIDS [36] | 275 ms | per CAN frame | GTX Titan X
Rec-CNN [38] | 117 ms | 128 CAN frames | Jetson TX2
NovelADS [32] | 128.7 ms | 100 CAN frames | Jetson Nano
GIDS [30] | 5.89 ms | 64 CAN frames | GTX 1080
TAN-IDS [29] | 11.6 ms | 128 CAN frames | -
DCNN [13] | 5 ms | 29 CAN frames | Tesla K80
TCAN-IDS [31] | 3.4 ms | 64 CAN frames | Jetson AGX
MTH-IDS [37] | 0.574 ms | per CAN frame | Raspberry Pi 3
QMLP-IDS [18] | 0.24 ms | per CAN frame | Zynq Ultrascale+
CQMLP-IDS | 0.11 ms | per CAN frame | Zynq Ultrascale+

###  IV-C Power consumption, Resource Utilisation (PS/PL)

We further quantify the active power consumption of our dataflow accelerator
to determine the average energy consumption per inference and the hardware
resources incurred by the dataflow design on the XCZU7EV device. We observe
that our model consumed 2.15 W when measured directly from the device’s power
rails (using the PYNQ-PMBus package) while performing inference and other
tasks on the ECU (with Linux OS), thus consuming 0.23 mJ of energy per
inference. This is a 3.9$\times$ improvement compared to QMLP-IDS which report
a per-inference energy consumption of 0.9 mJ. In comparison, we also observed
the per-inference energy consumption of our (fp32) CAE model on the Jetson
Nano to be 2.76 mJ when averaged over 10000 inference runs which is 12$\times$
more than the proposed CQMLP-IDS on the tightly integrated IDS-ECU on a hybrid
FPGA device. Among other reported results in the literature, our approach
improves the active power consumption by 4.6$\times$ when compared to the GRU
[34] model on an Nvidia Jetson Xavier as a dedicated IDS node. In terms of
resources, the dataflow CQMLP model consumes less than 2% of resources on the
device, leaving behind enough resources to integrate dedicated accelerators
for the ECU function.

We also quantify the utilisation of the ARM cores for managing the dataflow
CQMLP accelerator to estimate the overhead in consolidating the IDS capability
on the ECU. It was observed that a single core utilisation peaked at 40% when
processing the completion interrupt from the IDS core (at the highest message
rate modelling the worst case scenario), while other cores remained at IDLE
($\leq 2\%$ utilisation). We believe that this could be further refined by
processing the classification results using hardware logic, interrupting the
ECU only when a threat is detected, while also improving the detection
latency.

TABLE VI: Resource utilisation breakdown of our proposed CQMLP (XCZU7EV).

Node | LUT | FF | BRAM/URAM | DSP
---|---|---|---|---
2-bit-CQMLP-IDS | 3999 | 4524 | 4/0 | 0
% Usage | 1.74% | 0.98% | 1.28%/ 0% | 0%

##  V Conclusion

In this paper, we present a dataflow-style hardware accelerator that
implements a custom quantised MLP (CQMLP) model for detecting and classifying
multiple attack vectors on an automotive CAN network. The accelerator
generated through the FINN flow is quantised to 2-bit precision for weights
and activations and is tightly integrated to the ARM core on a Zynq
Ultrascale+ platform, mimicking a standard ECU. We show that the dataflow
integration achieves a 2.2$\times$ speed-up in per-message processing latency
and almost 3.9$\times$ reduction in energy consumption with 0.23mJ per
inference when compared to the state-of-the-art IDSs. The quantised model also
maintains high detection and classification accuracy across a range of attacks
including DoS, fuzzing, and RPM-Spoofing attacks, making it an ideal approach
for distributing IDS capabilities across ECUs in a vehicle network.

## References

  * [1] R. B. GmbH, “CAN Specification, Version 2.0,” 1991.
  * [2] S. Nie, L. Liu, and Y. Du, “Free-fall: Hacking Tesla from wireless to CAN bus,” Briefing, Black Hat USA, vol. 25, pp. 1–16, 2017\.
  * [3] K. Iehira, H. Inoue, and K. Ishida, “Spoofing attack using bus-off attacks against a specific ECU of the CAN bus,” in Proc. IEEE Communications & Networking Conference (CCNC), pp. 1–4, IEEE, 2018\.
  * [4] Z. Cai, A. Wang, W. Zhang, M. Gruffke, and H. Schweppe, “0-days & mitigations: Roadways to exploit and secure connected BMW cars,” Black Hat USA, vol. 2019, p. 39, 2019.
  * [5] C. Miller and C. Valasek, “Remote exploitation of an unaltered passenger vehicle,” Black Hat USA, vol. 2015, no. S 91, 2015.
  * [6] M. Enev, A. Takakuwa, K. Koscher, and T. Kohno, “Automobile Driver Fingerprinting.,” Proc. Priv. Enhancing Technol., vol. 2016, no. 1, pp. 34–50, 2016.
  * [7] U. E. Larson, D. K. Nilsson, and E. Jonsson, “An approach to specification-based attack detection for in-vehicle networks,” in Proc. IEEE Intelligent Vehicles Symposium, pp. 220–225, 2008.
  * [8] C. Miller and C. Valasek, “Adventures in automotive networks and control units,” Def Con, vol. 21, no. 260-264, pp. 15–31, 2013.
  * [9] I. Studnia, E. Alata, V. Nicomette, M. Kaâniche, and Y. Laarouchi, “A language-based intrusion detection approach for automotive embedded networks,” International Journal of Embedded Systems, vol. 10, 2018.
  * [10] S. N. Narayanan, S. Mittal, and A. Joshi, “Using data analytics to detect anomalous states in vehicles,” arXiv preprint arXiv:1512.08048, 2015.
  * [11] A. Alshammari, M. A. Zohdy, D. Debnath, and G. Corser, “Classification approach for intrusion detection in vehicle systems,” Wireless Engineering and Technology, vol. 9, no. 4, pp. 79–94, 2018.
  * [12] L. Yang, A. Moubayed, I. Hamieh, and A. Shami, “Tree-based intelligent intrusion detection system in internet of vehicles,” in 2019 IEEE global communications conference (GLOBECOM), pp. 1–6, IEEE, 2019.
  * [13] H. M. Song, J. Woo, and H. K. Kim, “In-vehicle network intrusion detection using deep convolutional neural network,” Vehicular Communications, vol. 21, p. 100198, 2020.
  * [14] S. Tariq, S. Lee, and S. S. Woo, “CANTransfer: transfer learning based intrusion detection on a controller area network using convolutional LSTM network,” in Proceedings of the 35th Annual ACM Symposium on Applied Computing, pp. 1048–1055, 2020.
  * [15] P. Jokic, S. Emery, and L. Benini, “Binaryeye: A 20 kfps streaming camera system on FPGA with real-time on-device image recognition using binary neural networks,” in 2018 IEEE 13th International Symposium on Industrial Embedded Systems (SIES), pp. 1–7, IEEE, 2018.
  * [16] Xilinx, “Vitis AI User Guide,” 2021.
  * [17] Y. Umuroglu, N. J. Fraser, G. Gambardella, M. Blott, P. Leong, M. Jahre, and K. Vissers, “Finn: A framework for fast, scalable binarized neural network inference,” in Proc. Intl. Symposium on Field-Programmable Gate Arrays (FPGA), pp. 65–74, 2017.
  * [18] S. Khandelwal and S. Shreejith, “A Lightweight FPGA-based IDS-ECU Architecture for Automotive CAN,” in 2022 International Conference on Field-Programmable Technology (ICFPT), pp. 1–9, IEEE, 2022.
  * [19] NVIDIA, “https://developer.nvidia.com/embedded/jetson-nano.”
  * [20] F. Hartwich et al., “CAN with flexible data-rate,” in Proc. iCC, pp. 1–9, Citeseer, 2012.
  * [21] M. Bozdal, M. Samie, and I. Jennions, “A Survey on CAN Bus Protocol: Attacks, Challenges, and Potential Solutions,” in Proc. Intl. Conf. on Computing, Electronics Communications Engineering (iCCECE), pp. 201–205, 2018\.
  * [22] S. Mukherjee, H. Shirazi, I. Ray, J. Daily, and R. Gamble, “Practical DoS attacks on embedded networks in commercial vehicles,” in Proc. Intl Conference on Information Systems Security, Springer, 2016.
  * [23] K. Koscher, S. Savage, F. Roesner, S. Patel, T. Kohno, A. Czeskis, D. McCoy, B. Kantor, D. Anderson, H. Shacham, et al., “Experimental security analysis of a modern automobile,” in Proc. IEEE Sym. on Security and Privacy, pp. 447–462, IEEE Computer Society, 2010.
  * [24] A. Palanca, E. Evenchick, F. Maggi, and S. Zanero, “A stealth, selective, link-layer denial-of-service attack against automotive networks,” in Proc. Intl. Conf. on Detection of Intrusions and Malware, and Vulnerability Assessment, pp. 185–206, Springer, 2017.
  * [25] T. P. Vuong, G. Loukas, and D. Gan, “Performance evaluation of cyber-physical intrusion detection on a robotic vehicle,” in Proc. Intl. Conf. on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing, pp. 2106–2113, IEEE, 2015.
  * [26] D. Stabili, M. Marchetti, and M. Colajanni, “Detecting attacks to internal vehicle networks through hamming distance,” in AEIT Intl. Annual Conference, pp. 1–6, IEEE, 2017.
  * [27] M. Weber, S. Klug, E. Sax, and B. Zimmer, “Embedded hybrid anomaly detection for automotive CAN communication,” in Proc. European Congress on Embedded Real Time Software and Systems (ERTS), 2018.
  * [28] S. B. Park, H. J. Jo, and D. H. Lee, “G-IDCS: Graph-Based Intrusion Detection and Classification System for CAN Protocol,” IEEE Access, vol. 11, pp. 39213–39227, 2023.
  * [29] T. P. Nguyen, H. Nam, and D. Kim, “Transformer-Based Attention Network for In-Vehicle Intrusion Detection,” IEEE Access, vol. 11, pp. 55389–55403, 2023.
  * [30] E. Seo, H. M. Song, and H. K. Kim, “GIDS: GAN based intrusion detection system for in-vehicle network,” in Proc. Conf. on Privacy, Security and Trust (PST), pp. 1–6, IEEE, 2018.
  * [31] P. Cheng, K. Xu, S. Li, and M. Han, “TCAN-IDS: Intrusion Detection System for Internet of Vehicle Using Temporal Convolutional Attention Network,” Symmetry, vol. 14, no. 2, p. 310, 2022.
  * [32] K. Agrawal, T. Alladi, A. Agrawal, V. Chamola, and A. Benslimane, “NovelADS: A Novel Anomaly Detection System for Intra-Vehicular Networks,” IEEE Transactions on Intelligent Transportation Systems, 2022.
  * [33] W. Lo, H. Alqahtani, K. Thakur, A. Almadhor, S. Chander, and G. Kumar, “A hybrid deep learning based intrusion detection system using spatial-temporal representation of in-vehicle network traffic,” Vehicular Communications, vol. 35, p. 100471, 2022.
  * [34] H. Ma, J. Cao, B. Mi, D. Huang, Y. Liu, and S. Li, “A GRU-Based Lightweight System for CAN Intrusion Detection in Real Time,” Security and Communication Networks, vol. 2022, 2022.
  * [35] P. F. De Araujo-Filho, A. J. Pinheiro, G. Kaddoum, D. R. Campelo, and F. L. Soares, “An Efficient Intrusion Prevention System for CAN: Hindering Cyber-Attacks with a Low-Cost Platform,” IEEE Access, vol. 9, pp. 166855–166869, 2021.
  * [36] A. K. Desta, S. Ohira, I. Arai, and K. Fujikawa, “MLIDS: Handling Raw High-Dimensional CAN Bus Data Using Long Short-Term Memory Networks for Intrusion Detection in In-Vehicle Networks,” in Proc. Intl. Telecommunication Networks and Applications Conference (ITNAC), pp. 1–7, IEEE, 2020.
  * [37] L. Yang, A. Moubayed, and A. Shami, “MTH-IDS: A Multitiered Hybrid Intrusion Detection System for Internet of Vehicles,” IEEE Internet of Things Journal, vol. 9, no. 1, pp. 616–632, 2021.
  * [38] A. K. Desta, S. Ohira, I. Arai, and K. Fujikawa, “Rec-CNN: In-vehicle networks intrusion detection using convolutional neural networks trained on recurrence plots,” Vehicular Communications, vol. 35, p. 100470, 2022\.
  * [39] S. Khandelwal, E. Wadhwa, and S. Shreejith, “Deep Learning-based Embedded Intrusion Detection System for Automotive CAN,” in 2022 IEEE 33rd International Conference on Application-specific Systems, Architectures and Processors (ASAP), pp. 88–92, 2022.
  * [40] S. Khandelwal and S. Shreejith, “A Lightweight Multi-Attack CAN Intrusion Detection System on Hybrid FPGAs,” in 2022 32nd International Conference on Field-Programmable Logic and Applications (FPL), pp. 425–429, 2022.
  * [41] CAR Hacking Dataset, “https://ocslab.hksecurity.net/datasets/can-intrusion-dataset,” 2020.
  * [42] A. Pappalardo, “Xilinx/brevitas,” 2021.
  * [43] S. Wu, G. Li, F. Chen, and L. Shi, “Training and inference with integers in deep neural networks,” arXiv preprint arXiv:1802.04680, 2018.
  * [44] S. Khandelwal, A. Walsh, and S. Shreejith, “Quantised Neural Network Accelerators for Low-Power IDS in Automotive Networks,” in 2023 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1–2, IEEE, 2023.