[1]Jiro Kitagawa

1]Department of Electrical Engineering, Faculty of Engineering, Fukuoka
Institute of Technology, 3-30-1 Wajiro-higashi, Higashi-ku, Fukuoka, 811-0295,
Japan

2]Department of Physics, Tokyo Metropolitan University, 1-1, Minami-osawa,
Hachioji, 192-0397, Japan

3]Department of Electrical Engineering, Faculty of Science and Engineering,
Kyushu Sangyo University, 2-3-1 Matsukadai, Higashi-ku, Fukuoka, 813-8503,
Japan

# Effect of annealing in eutectic high-entropy alloy superconductor NbScTiZr

Takeru Seki  Hiroto Arima  Yuta Kawasaki  Terukazu Nishizaki  Yoshikazu
Mizuguchi<EMAIL_ADDRESS>[  [  [

###### Abstract

We investigated the impact of annealing on the structural characteristics and
superconducting critical temperature ($T_{\mathrm{c}}$) of the eutectic high-
entropy alloy (HEA) superconductor NbScTiZr. The HEA manifests an eutectic
microstructure composed of body-centered cubic (bcc) and hexagonal close-
packed phases. Both the lattice parameters of the bcc phase and grain size of
the eutectic structure exhibited pronounced sensitivity to variations in
annealing temperature. The observed dependence of the lattice parameter on
annealing temperature supports the possibility that lattice strain occurs at
lower annealing temperatures. The as-cast sample demonstrated
superconductivity at $T_{\mathrm{c}}$ of 7.9 K, which increased to 9 K after
annealing at 800 ∘C. However, when subjected to annealing at 1000 ∘C,
$T_{\mathrm{c}}$ diminishes to 8.7 K. The annealing-temperature dependence of
$T_{\mathrm{c}}$ cannot be comprehensively elucidated based solely on the
electronic density of states at the Fermi level. It is plausible that the
lattice strain may influence the annealing temperature dependence of
$T_{\mathrm{c}}$. Our results for the critical current density $J_{c}$ reveal
that the self-field $J_{c}$ of the as-cast NbScTiZr at 2 K exceeds 106 A/cm2.

###### keywords:

high-entropy alloy, superconductivity, eutectic structure, critical current
density

##  1 Introduction

High-entropy alloys (HEA) constitute a new research area in materials science,
and this innovative domain has been extensively applied to a diverse array of
alloys and crystalline systems[1]. Unlike their conventional counterparts,
which comprise one or two principal elements, HEAs generally contain at least
four principal elements[2, 3]. HEAs have attracted considerable attention as
promising candidates for structural materials owing to their excellent
mechanical properties[4]. In addition, HEAs exhibit numerous versatile
functionalities, including energy storage, magnetic refrigeration, soft
ferromagnetism, catalysis, thermoelectricity, and biocompatibility[5, 6, 7, 8,
9, 10, 11, 12, 13]. Another feature in HEAs is functionality augmentation by
manipulating microstructure, such as eutectic structures. Notably, instances
arise in which the microstructural features exert a discernible influence on
the manifested physical properties.[2, 14, 15, 16]. One extensively studied
area is structural HEA materials, with eutectic HEAs capturing particular
interest for their potential to surmount the traditional strength-ductility
trade-off encountered in structural materials.[2, 14]. Body-centered cubic
(bcc) HEAs exhibit limited ductility, whereas face-centered-cubic (fcc) HEAs
exhibit high ductility and low strength. Accordingly, the pursuit of
concurrent high strength and ductility is realized by strategically including
bcc and fcc phases within eutectic HEAs. Another example is a magnetic HEA[15]
denoted as Fe15Co15Ni20Mn20Cu30. This particular alloy exhibited spinodal
decomposition after thermal treatment, and controlling the spinodal structure
by changing the annealing temperature increased the Curie temperature.

HEA superconductivity has been extensively investigated [17, 18, 19] since the
discovery of bcc Ta34Nb33Hf8Zr14Ti11. The HEA superconductors are documented
in various structural types, including bcc[20, 21, 22, 23, 24, 25, 26, 27, 28,
29, 30], hexagonal close-packed (hcp)[31, 32, 33], CsCl-type[34], NaCl-
type[35, 36], $\alpha$ (or $\beta$)-Mn-type[37, 38], CuAl2-type[39],
W5Si3-type[40], BiS2-based, and YBCO-based[41, 42] structures. Several
interesting features of the HEAs have been identified. The
(TaNb)0.67(HfZrTi)0.33 HEA with a bcc structure exhibits a robustness of
superconductivity under high pressures[43]. Remarkably, the superconducting
critical temperature ($T_{\mathrm{c}}$) remained nearly invariant at 8 K, even
when subjected to an extreme pressure of 180 GPa. The ”cocktail effect”,
whereby the physical properties of an HEA surpass those of its constituent
elements, is a crucial aspect of HEA materials. This effect is evident in
BiS2-based superconductors, where the diamagnetic signal is enhanced within
the high-entropy state[41]. A recent report revealed that the critical current
density $J_{c}$ of a Hf-Nb-Ta-Ti-Zr film exceeds that of an NbTi alloy
employed in high-field superconducting magnets[44]. Emerging research has
underscored the notable inverse correlation between $T_{\mathrm{c}}$ and the
Debye temperature across several bcc HEA superconductors[27, 28]. This
behavior can be understood using the Bardeen–Cooper–Schrieffer (BCS) theory,
which considers the uncertainty principle between the phonon energy and phonon
lifetime[27, 28].

In several superconductors, the inclusion of eutectic phases affects the
superconducting properties. A salient exemplar of this phenomenon can be
discerned in Sr2RuO4, where the eutectic composite exhibiting a lamellar
arrangement with Ru metal shows a $T_{\mathrm{c}}$ of 3 K, a notable
advancement from the 1.5 K exhibited by pure Sr2RuO4[45]. A similar effect was
observed for Ir [46], which formed a eutectic phase with a small amount of
YIr2. In this case, a strain-induced lattice softening enhances the
$T_{\mathrm{c}}$ from 0.1 to 2.7 K[46]. Our investigations yielded parallel
findings, wherein $T_{\mathrm{c}}$ enhancement in eutectic Zr5Pt3Ox has been
documented[47]. In this compound, a eutectic phase comprising Zr5Pt3O0.5-0.6
and ZrPt emerged for $x$ values exceeding 1.0. The area of the eutectic phase
increases as $x$ is increased above 1.0, and $T_{\mathrm{c}}$ systematically
increases from 3.2 K for $x$=0.6 to 4.8 K for $x$=2.5. In eutectic HEAs, an
alteration in the superconducting properties is expected by controlling the
morphology of the eutectic structure. However, exploring eutectic HEA
superconductors remains a relatively nascent endeavor. In addition, we
consider $J_{\mathrm{c}}$ within the eutectic HEAs. A eutectic structure
possesses many grain boundaries that serve as effective magnetic flux pinning
sites. Therefore, it is conceivable that eutectic HEAs may confer a heightened
$J_{\mathrm{c}}$. The practical superconducting wire employs a
multifilamentary structure, and a eutectic HEA superconductor can be
conceptualized as a built-in multifilamentary configuration. Therefore,
eutectic HEA superconductors are good candidates for high-performance
superconducting-wire applications[19].

Our attention converges on NbScTiZr, which is a eutectic HEA composed of
(NbTiZr)-enriched bcc and (ScZr)-enriched hcp phases[48, 49]. The
superconducting properties of as-cast NbScTiZr were investigated by Krnel et
al.[49], revealing a bulk superconductivity showing $T_{\mathrm{c}}$ of 7.3 K.
Superconductivity occurs in the bcc phase[49]. Although the superconductivity
of the as-cast sample was investigated, the annealing effect was not studied.
We investigated the effect of annealing on $T_{\mathrm{c}}$ in NbScTiZr by
measuring the magnetization and electrical resistivity. The annealing
temperature dependence of $T_{\mathrm{c}}$ was discussed considering the
structural properties and electronic structure calculations. Furthermore, we
show the results for $J_{\mathrm{c}}$. Our new findings include
$T_{\mathrm{c}}$ enhancement by thermal annealing in the eutectic HEA and a
notably high $J_{\mathrm{c}}$ in the as-cast sample.

##  2 Materials and Methods

The as-cast sample (2.5 g) was synthesized using a homemade arc furnace
operating in an Ar atmosphere employing an Nb wire (Nilaco, 99.9 %), Sc chips
(Kojundo Chemical Laboratory, 99 %), Ti wire (Nilaco, 99.9 %), and Zr wire
(Nilaco, 99.5 %). The atomic ratio was Nb:Sc:Ti:Zr=1:1:1:1. Multiple
iterations of sample flipping and remelting ensured sample homogeneity, and
the sample was quenched on a water-cooled Cu hearth. Subsequent heat treatment
at 400 ∘C, 800 ∘C or 1000 ∘C for 4 d was executed within an electronic
furnace, with the sample encapsulated within an evacuated quartz tube.

X-ray diffraction (XRD) patterns were recorded using an X-ray diffractometer
(Shimadzu XRD-7000L) with Cu-K$\alpha$ radiation in the Bragg-Brentano
geometry. Thin slabs cut from the samples were used because of the difficulty
in obtaining fine powders. Scanning electron microscopy (SEM) images were
captured using field-emission SEM (FE-SEM; JEOL JSM-7100F). An energy-
dispersive X-ray (EDX) spectrometer co-located with the FE-SEM was employed to
gain insight into the elemental composition. The microstructures were observed
on the polished surfaces. The surface of each sample was polished using
silicon carbide paper (#240, #400, #600, and #1000 mesh), alumina paste (5, 1,
and 0.1 $\mu$m), and colloidal silica (0.04 $\mu$m).

The temperature-dependent behavior of ac magnetic susceptibility $\chi_{ac}$
($T$) under a 5 Oe ac field at 800 Hz was assessed using a conventional mutual
inductance technique facilitated by a GM refrigerator (UW404, Ulvac
Cryogenics) operating within a temperature interval of 3 to 20 K. Samples
weighing $\sim$ 20 mg were used for this measurement. The temperature
dependence of dc magnetization $M$($T$) and isothermal magnetization curve
spanning from -7 to 7 T were measured using a SQUID magnetometer (MPMS3,
Quantum Design). To measure $M$, we prepared samples with dimensions of
$\sim$2$\times$1$\times$0.15 mm3. $M$($T$) was measured using field-cooled
(FC) and zero-field-cooled (ZFC) procedures under an external field of 0.4 mT.
Electrical resistivity $\rho$ measurements were performed using a four-probe
methodology facilitated by a Quantum Design MPMS apparatus. In the $\rho$
measurements, we used parallelepiped samples with dimensions of
$\sim$5$\times$1$\times$1 mm3.

The theoretical underpinning of the electronic structure was achieved through
coherent potential approximation (CPA). To this end, the Akai-KKR program
package[50] rooted in the Korringa-Kohn-Rostoker (KKR) method with CPA is
harnessed. We used the generalized gradient approximation from Perdew-Burke-
Ernzerhof (PBE) and treated the spin-orbit interaction.

##  3 Results and Discussion

Figure 1(a) shows the XRD patterns of the as-cast and heat-treated NbScTiZr
samples. Simulation patterns of the bcc ($a$=3.359 Å) and hcp ($a$=3.262 Å,
$c$=5.152 Å) structures were juxtaposed. All experimental patterns show the
concurrent presence of bcc and hcp phases, denoted by filled circles and
triangles, respectively. By employing the least-squares method, the lattice
parameters for all the phases were obtained, as listed in Table 1. In both the
bcc and hcp phases, annealing at 400 ∘C reduced each lattice parameter.
However, each lattice parameter increased with increasing heat treatment
temperature above 400 ∘C (see also Fig.1(b)).

Figure 1: (a) XRD patterns of NbScTiZr annealed at 400 ∘C, 800 ∘C, and 1000 ∘C, together with as-cast sample. The simulation patterns of the bcc ($a$=3.359 Å) and hcp ($a$=3.262 Å, $c$=5.152 Å) structures are also displayed. The origin of each pattern is shifted by a value for clarity. (b) Annealing temperature dependence of the lattice parameters of NbScTiZr (filled red circles). The annealing temperature dependent on the lattice parameters calculated using the hard sphere model is also shown (filled blue circles). Table 1: Chemical compositions of bcc and hcp phases, volume fractions, lattice parameters, VEC, and $T_{\mathrm{c}}$ of prepared samples. NbScTiZr | composition | vol.% | lattice parameter (Å) | VEC |  $T_{\mathrm{c}}$ (K)
---|---|---|---|---|---
as-cast | bcc:Nb26.6(4)Sc22.3(5)Ti25.5(5)Zr25.6(4) | 57 |  $a$=3.359(2) | 4.04 | 7.9
| hcp:Nb21.3(4)Sc30(1)Ti23.0(7)Zr25.8(5) | 43 |  $a$=3.262(3), $c$=5.152(3) | 3.91 |
400 ∘C | bcc:Nb27.9(3)Sc21.6(2)Ti25.3(3)Zr25.2(2) | 57 |  $a$=3.308(2) | 4.06 | 8.4
| hcp:Nb22.1(5)Sc32.3(9)Ti21.3(5)Zr24.2(2) | 43 |  $a$=3.252(1), $c$=5.131(1) | 3.90 |
800 ∘C | bcc:Nb33(1)Sc15(1)Ti32(1)Zr20(1) | 50 |  $a$=3.325(1) | 4.18 | 9.0
| hcp:Nb13(1)Sc39.1(9)Ti15.1(4)Zr32.6(6) | 50 |  $a$=3.252(2), $c$=5.138(3) | 3.74 |
1000 ∘C | bcc:Nb39.0(8)Sc4.2(6)Ti38.0(6)Zr18.9(8) | 50 |  $a$=3.360(1) | 4.35 | 8.7
| hcp:Nb2.6(7)Sc56(1)Ti7.3(7)Zr34.1(5) | 50 |  $a$=3.260(1), $c$=5.141(2) | 3.47 |
|  |  |  |  |
Figure 2:  SEM images of NbScTiZr for as-cast sample (a) and heat-treated
samples at (b) 400 ∘C, (c) 800 ∘C, and (d) 1000 ∘C, respectively.

The SEM images of all samples are shown in Figs. 2(a)-(d). The as-cast
NbScTiZr sample exhibits a lamellar-like arrangement characterized by an
approximate thickness of $\sim$ 70 nm ( Fig.2(a)). This fine microstructure is
consistent with the results of two previous studies [48, 49]. According to the
reports by these research groups, the bright and dark phases were bcc and hcp,
respectively. The fine structure remained resolute even after 400 ∘C annealing
(see Fig. 2(b)). By contrast, when subjected to annealing at higher
temperatures, the grain size increased noticeably, as shown in Figs.2(c) and
(d). With an increase in the annealing temperature beyond 400 ∘C, a systematic
increase in the grain size is observed, attaining dimensions exceeding 1
$\mu$m after 1000 ∘C annealing. The volume fractions associated with bcc and
hcp phases across all samples were obtained from SEM images and are summarized
in Table 1. The ratio of the volume fractions of the bcc and hcp phases was
57:43 in the as-cast sample and its heat-treated counterpart at 400 ∘C. For
the samples annealed at 800 ∘C and 1000 ∘C, the bcc phase fraction exhibited a
slight reduction. The chemical compositions determined by EDX for the bcc and
hcp phases in each sample are summarized in Table 1. In the as-cast sample,
the bcc and hcp phases emerged as Sc-deficient and Sc-rich, respectively. This
chemical composition maintained a nearly identical state, even within the
sample annealed at 400 ∘C. Profound shifts in the chemical compositions of the
bcc and hcp phases occur within the samples annealed at 800 ∘C and 1000 ∘C;
the bcc phase shows increased Nb and Ti contents, whereas the hcp phase
enriches the Sc and Zr contents. Drawing on the insights gained from the
chemical analysis, we discuss the annealing temperature dependence of the
lattice parameters of the bcc phase. We invoked the framework of the hard-
sphere model in conjunction with the obtained chemical composition, enabling
the estimation of the ideal lattice parameter across distinct annealing
conditions. By employing atomic radii of 1.429 Å for Nb, 1.641 Å for Sc,
1.4615 Å for Ti, and 1.6025 Å for Zr[51], application of the hard sphere model
yields the lattice parameter, which is expressed as 2.309$\bar{r}$, where
$\bar{r}$ is the composition-weighted atomic radius. The resultant lattice
parameters were compared with experimental data, as shown in Fig. 1(b),
wherein the as-cast sample is regarded as the analog for 0 ∘C annealing. The
calculated lattice parameter closely approximates its experimental counterpart
at 1000 ∘C, implying a weak influence of lattice strain, probably owing to the
annealing effect. By contrast, at lower temperatures, such as the as-cast and
400 ∘C annealing conditions, a substantial disparity between the calculated
and experimental lattice parameters becomes evident, suggesting a plausible
manifestation of lattice strain.

Figure 3:  Temperature dependences of (a) $\chi_{ac}$, (b) $\rho$, and (c) low
temperature $\rho$ of NbScTiZr.

Figure 3(a) shows $\chi_{ac}$ ($T$) values of all the samples. The onset of
the diamagnetic signal is denoted by $T_{\mathrm{c}}$. The as-cast sample
showed a diamagnetic signal below $T_{\mathrm{c}}$=7.9 K, which agrees with
the findings of Krnel et al.[49]. Remarkably, our investigations revealed a
progressive increase in $T_{\mathrm{c}}$ to 9.0 K with increasing annealing
temperatures up to 800 ∘C. However, a subsequent increase in the annealing
temperature yielded a marginal decline in $T_{\mathrm{c}}$ to 8.7 K. We also
confirmed a zero $\rho$ value for each sample, as shown in Figs. 3(b) and (c).
Importantly, although the order of magnitude characterizing the normal state
$\rho$ for each sample suggests a metallic nature, atomic disorder contributes
to the weak temperature dependence. In each sample, $\rho$ drops below
approximately $T_{\mathrm{c}}$, as determined by $\chi_{ac}$ ($T$)
measurement. Although the superconductive transition in $\rho$ ($T$) of the
as-cast sample was relatively broad, it became sharper after annealing ( Fig.
3(c)).

Figure 4:  VEC dependence of $T_{\mathrm{c}}$ of NbScTiZr (red triangle). The
solid line represents the Matthias rule of conventional binary or ternary
transition metal alloys. The data of typical quinary bcc HEA superconductors
are also shown by the filled circles. For the quinary bcc HEAs, the
correspondence between color and HEA is as follows; green: nonequimolar Hf-Nb-
Ta-Ti-Zr, blue: Al-Nb-Ti-V-Zr, black: Hf21Nb25Ti15V15Zr24, orange: HfNbTaTiZr,
HfNbReTiZr, HfNbTaTiV, and HfMoNbTiZr, light blue: Nb-Ta-Mo-Hf-W, Ti-Zr-Nb-
Ta-W, and Ti-Zr-Nb-Ta-V, and red: Ti-Hf-Nb-Ta-Re.

Herein, we discuss the interplay between valence electron concentration (VEC)
and $T_{\mathrm{c}}$. Within the framework of the BCS theory, $T_{\mathrm{c}}$
depends on the density of states (DOS) at the Fermi level ($E_{\mathrm{F}}$),
denoted as $D(E_{\mathrm{F}})$, and the electron-phonon interaction. The VEC
reflects $D(E_{\mathrm{F}})$, and a strong correlation between the VEC and
$T_{\mathrm{c}}$ has been observed in binary or ternary superconducting
transition metal alloys, referred to as the Matthias rule[52]. The
computational derivation of VEC adopts the following equation:

$\mathrm{VEC}=\sum^{4}_{i=1}c_{i}\mathrm{VEC}_{i}$ (1)

where $c_{i}$ and VECi are the molar fraction and VEC value of the $i$th
element, respectively. The specific VECi values are 5 for Nb, 3 for Sc, and 4
for Ti and Zr. Considering the chemical composition of the bcc phase in each
sample, the relationship between $T_{\mathrm{c}}$ and VEC for NbScTiZr samples
is shown in Fig.4 (see also Table 1). The solid line in the figure describes
the Matthias rule intrinsic to conventional binary or ternary transition metal
alloys[52], revealing a broad peak at a VEC value of approximately 4.6. The
dataset of typical quinary bcc HEA superconductors is also shown by filled
circles (green: nonequimolar Hf-Nb-Ta-Ti-Zr, blue: Al-Nb-Ti-V-Zr, black:
Hf21Nb25Ti15V15Zr24, orange: HfNbTaTiZr, HfNbReTiZr, HfNbTaTiV, and
HfMoNbTiZr, light blue: Nb-Ta-Mo-Hf-W, Ti-Zr-Nb-Ta-W, and Ti-Zr-Nb-Ta-V, and
red: Ti-Hf-Nb-Ta-Re.)[21, 22, 23, 24, 27, 28, 57, 53, 54, 55]. Conventionally,
the dataset of HEAs traces a trajectory beneath a solid line. NbScTiZr
annealed at 1000 ∘C resides near the solid line. The sample annealed at 800
∘C, despite having a diminished VEC value compared with its 1000 ∘C
counterpart, experiences an $T_{\mathrm{c}}$ enhancement that is inconsistent
with the anticipation of the Matthias rule. A further strange observation
emerges for the as-cast NbScTiZr and the sample subjected to 400 ∘C annealing,
which deviates from the solid line and HEAs trend. The Matthias rule is based
on an elevated $D(E_{\mathrm{F}})$ at a VEC of approximately 4.6, accompanied
by a relatively low $D(E_{\mathrm{F}})$ on either side of the threshold.
Consequently, at a VEC of approximately 4.0, a considerable decline in
$D(E_{\mathrm{F}})$ was predicted, causing a notable decrease in
$T_{\mathrm{c}}$. To explore the underpinnings of $D(E_{\mathrm{F}})$ in
NbScTiZr, electronic structure calculations were performed, the results of
which are shown in Figs.5(a), offering DOS for NbScTiZr across all annealing
conditions. The experimental lattice parameters and chemical compositions
determined by EDX were used for the electronic structure calculations. In as-
cast NbScTiZr or the sample annealed at 400 ∘C, $D(E_{\mathrm{F}})$ stands for
analogous bcc HEAs[24, 27, 56], suggesting the plausible emergence of
superconductivity with $T_{\mathrm{c}}$ higher than that predicted by the
Matthias rule. We can extract the value of $D(E_{\mathrm{F}})$ for each
annealing condition and plot it as a function of the annealing temperature in
Fig.5(b). As the annealing temperature increased beyond 400 ∘C,
$D(E_{\mathrm{F}})$ exhibited a systematic increase, driven by the concurrent
increase in VEC. Moreover, when transitioning from the as-cast sample to its
400 ∘ C-annealed counterpart, a reduction in $D(E_{\mathrm{F}})$ was observed.
Fig.5(b) further elucidates the annealing temperature-sensitive
$T_{\mathrm{c}}$. Although we expect an enhancement in $T_{\mathrm{c}}$ as
$D(E_{\mathrm{F}})$ increases, such a relationship is partially observed. It
is evident that factors beyond $D(E_{\mathrm{F}})$ interact to delineate the
annealing temperature dependence of $T_{\mathrm{c}}$, which suggests the
potential role of the electron-phonon interaction. As evidenced by Fig.1(b),
the lattice parameter shown in the 1000 ∘C annealing agrees with that
predicted by the hard sphere model. This result underscores the weak influence
of lattice strain. Conversely, for the other samples, conspicuous disparities
between the experimental and hard-sphere model-derived lattice parameters
manifested, signaling the presence of lattice strain. Such an occurrence could
conceivably contribute to alterations in the electron-phonon interactions,
subsequently influencing the behavior of $T_{\mathrm{c}}$. Notably, the
lattice strain occurring at annealing temperatures below 800 ∘C may be
partially responsible for the enhancement in $T_{\mathrm{c}}$, surpassing the
value predicted by the Matthias rule. This phenomenon resembles the
enhancement of $T_{\mathrm{c}}$ reported for Sr2RuO4, Ir, and Zr5Pt3Ox, as
explained in Introduction.

We discuss the impact of annealing on bcc HEA superconductors by drawing upon
a comparative analysis with relevant literature. Two notable studies
extensively investigated Hf-Nb-Ta-Ti-Zr HEA superconductors[57, 58]. In one of
these studies, three distinct HEAs, each characterized by unique chemical
composition (Hf20Nb21Ta20Ti19Zr20, Hf21Nb24Ta22Ti10Zr23, and
Hf26Nb25Ta25Zr24), are subjected to annealing procedures within the
temperature range of 1800 $\sim$ 2000 ∘C[57]. Remarkably, the superconducting
transition manifests robustly during thermal annealing, although the
$T_{\mathrm{c}}$ value depends on chemical composition. Notably, although the
HEAs exhibit no eutectic structures, each sample exhibits a dendritic
microstructure. Nevertheless, the HEAs did not show a definitive correlation
between microstructural morphology and $T_{\mathrm{c}}$. In the other
study[58], the annealing effect of $T_{\mathrm{c}}$ in (TaNb)0.7(HfZrTi)0.5
was explored. At annealing temperatures exceeding 700 ∘C, the phase
segregation becomes apparent. $T_{\mathrm{c}}$ of approximately 7 K remained
nearly unchanged upon annealing, even after the occurrence of phase
segregation. Furthermore, precipitation phenomena emerged above an annealing
threshold of 500 ∘C, serving as effective flux pinning sites, as subsequently
delineated. However, $T_{\mathrm{c}}$ was insensitive to alterations in the
microstructure. In NbScTiZr, the change in chemical composition depending on
the annealing temperature partially affected the $T_{\mathrm{c}}$ value,
paralleling the findings of a previous report[57]. Notably, the influence of
the eutectic structures on $T_{\mathrm{c}}$, mediated by the lattice strain,
was also confirmed in the NbScTiZr system. This starkly contrasts with Hf-Nb-
Ta-Ti-Zr HEA superconductors, which exhibit a conspicuous insensitivity to
alterations in microstructure in the context of $T_{\mathrm{c}}$. Hf-Nb-Ta-Ti-
Zr HEA superconductors are devoid of lattice strain. Consequently, it becomes
evident that the lattice strain plays a pivotal role in the systematic
variation of $T_{\mathrm{c}}$ during the thermal annealing.

Figure 5:  (a) Total electronic density of states for NbScTiZr. (b) Annealing
temperature dependence of $T_{\mathrm{c}}$ and $D(E_{\mathrm{F}})$.

Figure 6(a) shows $M$($T$) under an external field of 0.4 mT for both the as-
cast NbScTiZr and sample annealed at 800 ∘C. In each sample, $M$($T$) measured
under ZFC conditions exhibited a diamagnetic signal, whereas the $M$($T$) data
obtained under FC conditions indicated the manifestation of flux pinning
within the sample. The onset temperatures of the diamagnetic signals were
consistent with $T_{\mathrm{c}}$ values determined from the $\chi_{ac}$ ($T$)
measurements. The results of the isothermal hysteresis loops in $M$–$H$ ($H$:
external field) curves at both 2 and 4 K are shown in Figs.6(b) and (c) for
the as-cast NbScTiZr and the sample annealed at 800 ∘C, respectively. Both
samples exhibited hysteresis loops that are characteristic of type-II
superconductors. The hysteresis loop corresponding to the sample annealed at
800 ∘C demonstrates a small area relative to the as-cast counterpart. For the
as-cast sample at 2 K, abrupt changes in $M$ occur at approximately 5 kOe
during the field-increasing process and at -5 kOe during the field-decreasing
process, as shown in Fig.6(b). By contrast, no such abrupt variations in $M$
were discernible at 4 K, as shown in the same figure. This behavior is often
observed in several superconductors, including HEAs[44, 58, 59]. Abrupt
modulation of $M$ is commonly attributed to the flux jump phenomenon, which is
indicative of pronounced local magnetic instability.

Figure 6:  (a) Temperature dependence of magnetization of NbScTiZr. Field-
dependent magnetization of NbScTiZr at 2 and 4 K for (b) as-cast sample and
(c) sample annealed at 800 ∘C. Field-dependent $J_{\mathrm{c}}$ of as-cast
NbScTiZr and sample annealed at 800 ∘C at (d) 4 K and at (e) 2 K. In (d) and
(e), the data[44, 58] of representative bcc HEA superconductors are also
shown.

$J_{\mathrm{c}}$ is assessed by using the critical state model expressed as
follows:[60]

$J_{\mathrm{c}}=\frac{3\Delta M}{d}$ (2)

where $\Delta M$ is the width of the loop at applied $H$ and $d$ represents
the effective sample width perpendicular to $H$. Thus, the calculated $H$
dependence of $J_{\mathrm{c}}$ is shown in Figs.6(d) and (e). $J_{\mathrm{c}}$
exhibited an ascending trend with decreasing temperature in both the as-cast
and heat-treated samples. Notably, the self-field $J_{\mathrm{c}}$ of the as-
cast sample at 2 K exceeded 106 A/cm2, classifying it as a material with a
high $J_{\mathrm{c}}$. Intriguingly, the $J_{\mathrm{c}}$ value of the as-cast
sample was higher than that of the sample annealed at 800 ∘C by an order of
magnitude under fixed $H$ conditions. This enhanced $J_{\mathrm{c}}$ signifies
pronounced magnetic-flux pinning. As observed in Figs.2, the eutectic
microstructure of the as-cast sample is distinctly fine, in contrast to the
coarser morphology evident in the sample annealed at 800 ∘C. Typically, the
grain boundaries serve as loci for flux pinning. The fine eutectic structure
in the as-cast sample is endowed with a greater abundance of grain boundaries
between the bcc and hcp phases compared to the sample annealed at 800 ∘C,
which is characterized by a coarser microstructure. This difference accounts
for the elevated $J_{\mathrm{c}}$ of the as-cast sample, indicating the
pivotal role of the fine eutectic structure in enhancing $J_{\mathrm{c}}$.
Figures 6(d) and (e) also show data from a Hf-Nb-Ta-Ti-Zr bcc HEA film, which
exhibits the highest self-field $J_{\mathrm{c}}$ among bcc HEA
superconductors[44]. Furthermore, $J_{\mathrm{c}}$ data from a bulk Hf-Nb-Ta-
Ti-Zr ((TaNb)0.7(HfZrTi)0.5) bcc HEA sample were included[58]. Although the
self-field $J_{\mathrm{c}}$ of this bulk HEA is lower by order of magnitude
than that of the Hf-Nb-Ta-Ti-Zr film, it demonstrates relatively higher
$J_{\mathrm{c}}$ values under higher fields ($\mu_{0}H$ $>$ 3 T), which is
attributable to the fishtail effect[58]. The fishtail effect is ascribed to
precipitation during the heat treatment process, which enhances flux pinning.
Notably, at both 2 and 4 K, $J_{\mathrm{c}}$ of the as-cast NbScTiZr sample
closely aligns with that of the Hf-Nb-Ta-Ti-Zr bcc HEA film below 2 T and
surpasses $J_{\mathrm{c}}$ of the film above 2-3 T. At elevated applied
fields, the $J_{\mathrm{c}}$ of the as-cast NbScTiZr becomes competitive with
that of the Hf-Nb-Ta-Ti-Zr bulk bcc HEA. These comparative assessments
highlight the potential of eutectic HEA superconductors to exhibit high
$J_{\mathrm{c}}$ values. Herein, we briefly discuss a comparative analysis of
the field-dependent $J_{\mathrm{c}}$ behaviors of NbScTiZr and
(TaNb)0.7(HfZrTi)0.5. In the (TaNb)0.7(HfZrTi)0.5 system, the precipitate
takes the form of clusters characterized by dimensions of the order of $\sim$
10 nm[58]. This feature confers a pronounced advantage, particularly in the
augmentation of $J_{\mathrm{c}}$ performance at elevated magnetic fields.
Conversely, in NbScTiZr, we encountered a fine eutectic microstructure that
substantially supported the enhancement of $J_{\mathrm{c}}$ at lower magnetic
fields. Hence, the synergy achieved through the presence of both the eutectic
microstructure and cluster-like nanosized precipitates may realize elevated
$J_{\mathrm{c}}$ values over a wide field range.

##  4 Summary

We found that the $T_{\mathrm{c}}$ of the eutectic HEA superconductor NbScTiZr
systematically increases from 7.9 to 9 K with increasing thermal annealing
temperatures up to 800 ∘C. Annealing at 1000 ∘C slightly reduces
$T_{\mathrm{c}}$. Analysis of the lattice parameters using the hard-sphere
model indicates the presence of potential lattice strain, particularly at
lower annealing temperatures. While the lamellar-like structure with a
thickness of approximately 70 nm endured annealing up to 400 ∘C , a systematic
enlargement of the grain size occurred with an increase in the annealing
temperature beyond 400 ∘C. We discussed the annealing temperature dependence
of $T_{\mathrm{c}}$ by employing the $T_{\mathrm{c}}$ vs. VEC plot and
electronic structure calculations. Factors other than $D(E_{\mathrm{F}})$
should be considered to explain the annealing-temperature dependence of
$T_{\mathrm{c}}$. Notably, the electron-phonon interactions are pivotal in
this regard. Furthermore, the conceivable influence of the lattice strain,
deliberated within the context of the annealing-temperature dependence of the
lattice parameter, may affect the annealing-temperature dependence of
$T_{\mathrm{c}}$. For further exploration, we assessed $J_{\mathrm{c}}$ for
both the as-cast NbScTiZr and sample annealed at 800 ∘C and classified the as-
cast NbScTiZr as a material with a notably high $J_{\mathrm{c}}$.

Acknowledgments

T.N. acknowledges the support from a Grant-in-Aid for Scientific Research
(KAKENHI) (Grant No. 20K03867) and the Takahashi Industrial and Economic
Research Foundation. Y.M. acknowledges the support from a Grant-in-Aid for
Scientific Research (KAKENHI) (Grant No. 21H00151). J.K. acknowledges the
support from a Grant-in-Aid for Scientific Research (KAKENHI) (Grant No.
23K04570) and the Comprehensive Research Organization of the Fukuoka Institute
of Technology.

## References

  * [1] Biswas, K., Gurao, N. P., Maiti, T., Mishra, R. S.: High Entropy Materials. Springer, Singapore (2022)
  * [2] Lu, Y., Dong, Y., Jiang, H., Wang, Z., Cao, Z., Guo, S., Wang, T., Li, T., Liaw, P.K.: Promising properties and future trend of eutectic high entropy alloys. Scr. Mater. 187, 202-209 (2020)
  * [3] Yan, X., Liaw, P. K., Zhang, Y.: Order and Disorder in Amorphous and High-Entropy. Materials Metall. Mater. Trans. A 52, 2021-2111 (2021)
  * [4] Li, W., Xie, D., Li, D., Zhang, Y., Gao, Y., Liaw, P.K.: Mechanical behavior of high-entropy alloys. Prog. Mater. Sci. 118, 100777 (2021)
  * [5] Marques, F., Balcerzak, M., Winkelmann, F., Zepon, G., Felderhoff, M.: Review and outlook on high-entropy alloys for hydrogen storage. Energy Environ. Sci. 14, 5191-5227 (2021)
  * [6] Yang, B., Zhang, Y., Pan, H., Si, W., Zhang, Q., Shen, Z., Yu, Y., Lan, S., Meng, F., Liu, Y., Huang, H., He, J., Gu, L., Zhang, S., Chen, L.Q., Zhu, J., Nan, C.W., Lin, Y.H.: High-entropy enhanced capacitive energy storage. Nat. Mater. 21, 1074-1080 (2022)
  * [7] Law, J., Franco, V.: Review on magnetocaloric high-entropy alloys: Design and analysis methods. J. Mater. Res. 38, 37-51 (2023)
  * [8] Nakamura, Y., Takeshita, K., Nishizaki, T., Kitagawa, J.: High-entropy effect at rare-earth site in DyNi. AIP Adv. 13, 095323 (2023)
  * [9] Kitagawa, J., Fukuda, M., Fukuda, S., Fujiki, K., Nakamura, Y., Nishizaki, T.: Discovery of ferromagnetism in new multicomponent alloy Ti-Nb-Cr-Ru. APL Mater. 10, 071101 (2022)
  * [10] Kitagawa, J.: Magnetic properties, electrical resistivity, and hardness of high-entropy alloys FeCoNiPd and FeCoNiPt. J. Magn. Magn. Mater. 563, 170024 (2022)
  * [11] Wu, D., Kusada, K., Yamamoto, T., Toriyama, T., Matsumura, S., Kawaguchi, S., Kubota, Y., Kitagawa, H.: Platinum-Group-Metal High-Entropy-Alloy Nanoparticles. J. Am. Chem. Soc. 142, 13833–13838 (2020)
  * [12] Jiang, B., Yu, Y., Cui, J., Liu, X., Xie, L., Liao, J., Zhang, Q., Huang, Y., Ning, S., Jia, B., Zhu, B., Bai, S., Chen, L., Pennycook, S.J., He, J.: High-entropy-stabilized chalcogenides with high thermoelectric performance. Science 371, 830-834 (2021)
  * [13] Castro, D., Jaeger, P., Baptista, A.C., Oliveira, J.P.: An overview of high-entropy alloys as biomaterials. Metals 11, 648 (2021)
  * [14] Bhardwaj, V., Zhou, Q., Zhang, F., Han, W., Du, Y., Hua, K., Wang, H.: Effect of Al addition on the microstructure, mechanical and wear properties of TiZrNbHf refractory high entropy alloys. Tribol. Int. 160, 107031 (2021)
  * [15] Rao, Z., Dutta, B., Körmann, F., Lu, W., Zhou, X., Liu, C., Kwiatkowski da Silva, A., Wiedwald, U., Spasova, M., Farle, M., Ponge, D., Gault, B., Neugebauer, J., Raabe, D., Li, Z.: Beyond Solid Solution High-Entropy Alloys: Tailoring Magnetic Properties via Spinodal Decomposition. Adv. Funct. Mater. 31, 2007668 (2021)
  * [16] Baba, K., Ishizu, N., Nishizaki, T., Kitagawa, J.: Magnetic and transport properties of new dual-phase high-entropy alloy FeRhIrPdPt. Mater. 14, 2877 (2021).
  * [17] Koželj, P., Vrtnik, S., Jelen, A., Jazbec, S., Jagličić, Z., Maiti, S., Feuerbacher, M., Steurer, W., Dolinšek, J: Discovery of a Superconducting High-Entropy Alloy. Phys. Rev. Lett. 113, 107001 (2014)
  * [18] Sun, L., Cava, R. J.: High-entropy alloy superconductors: Status, opportunities, and challenges. Phys. Rev. Mater. 3, 090301 (2019)
  * [19] Kitagawa, J., Hamamoto, S., Ishizu, N.: Cutting edge of high-entropy alloy superconductors from the perspective of materials research. Metals 10, 1078 (2020)
  * [20] von Rohr, F. O., Cava, R. J.: Isoelectronic substitutions and aluminium alloying in the Ta-Nb-Hf-Zr-Ti high-entropy alloy superconductor. Phys. Rev. Mater. 2, 034801 (2018)
  * [21] Marik, S., Varghese, M., Sajilesh, K. P., Singh, D., Singh, R. P.: Superconductivity in equimolar Nb-Re-Hf-Zr-Ti high entropy alloy. J. Alloys Compd. 769, 1059-1063 (2018)
  * [22] Ishizu, N., Kitagawa, J.: New high-entropy alloy superconductor Hf21Nb25Ti15V15Zr24. Res. Phys. 13, 102275 (2019)
  * [23] Harayama, Y., Kitagawa, J.: Superconductivity in Al-Nb-Ti-V-Zr multicomponent alloy. J. Supercond. Nov. Magn. 34 2787-2794 (2021)
  * [24] Sarkar, N. K., Prajapat, C. L., Ghosh, P. S., Garg, N., Babu, P. D., Wajhal, S., Krishna, P. S. R., Gonal, M. R., Tewari, R., Mishra, P. K.: Investigations on superconductivity in an equi-atomic disordered Hf-Nb-Ta-Ti-V high entropy alloy. Intermetallics 144, 107503 (2022)
  * [25] Motla, K., Meena, P.K., Arushi, D.S., Singh, D., Biswas, P. K., Hillier, A. D., Singh, R.P.: Superconducting and normal-state properties of the high-entropy alloy Nb-Re-Hf-Zr-Ti investigated by muon spin relaxation and rotation. Phys. Rev. B 105, 144501 (2022)
  * [26] Kitagawa, J., Nishizaki, T.: Materials Research on Superconducting or Magnetic High-Entropy Alloys. Rev. High Press. Sci. Technol. 32, 77-85 (2022)
  * [27] Kitagawa, J., Hoshi, K., Kawasaki, Y., Koga, R., Mizuguchi, Y., Nishizaki, T.: Superconductivity and hardness of the equiatomic high-entropy alloy HfMoNbTiZr. J. Alloys Compd. 924, 166473 (2022)
  * [28] Hattori, T., Watanabe, Y., Nishizaki, T., Hiraoka, K., Kakihara, M., Hoshi, K., Mizuguchi, Y., Kitagawa, J.: Metallurgy, superconductivity, and hardness of a new high-entropy alloy superconductor Ti-Hf-Nb-Ta-Re. Journal of Alloys and Metallurgical Systems, 3, 100020 (2023)
  * [29] Li, K., Hu, X., Guo, R., Jiang, W., Zeng, L., Li, L., Yu, R., Wang, K., Zhang, C., Guo, S., Zhong, R., Xie, T., Yao, D. -X., Luo, H.: Superconductivity in the Medium-Entropy Alloy TiVNbTa with a Body-Centered Cubic Structure. J. Phys. Chem. C, 127, 16211–16218 (2023)
  * [30] Zeng, L., Hu, X., Boubeche, M., Li, K., Li, L., Yu, P., Wang, K., Zhang, C., Jin, K., Yao, D. -X., Luo, H.: Extremely strong coupling s-wave superconductivity in the medium-entropy alloy TiHfNbTa. Sci. China Phys. Mech. Astron., 66, 277412 (2023)
  * [31] Lee, Y.S., Cava, R.J.: Superconductivity in high and medium entropy alloys based on MoReRu. Physica C 566, 1353520 (2019)
  * [32] Marik, S., Motla, K., Varghese, M., Sajilesh, K. P., Singh, D., Breard, Y., Boullay, P., Singh, R. P.: Superconductivity in a new hexagonal high-entropy alloy. Phys. Rev. Mater. 3, 060602(R) (2019)
  * [33] Browne, A. J., Strong, D. P., Cava, R. J.: Phase stability and possible superconductivity of new 4d and 5d transition metal high-entropy alloys. J. Solid State Chem. 321, 123881 (2023)
  * [34] Stolze, K., Tao, J., von Rohr, F.O., Kong, T., Cava, R. J.: Sc-Zr-Nb-Rh-Pd and Sc-Zr-Nb-Ta-Rh-Pd high-entropy alloy superconductors on a CsCl-type lattice. Chem. Mater. 30, 906-914 (2018)
  * [35] Mizuguchi, Y.: Superconductivity in High-Entropy-Alloy Telluride AgInSnPbBiTe5. J. Phys. Soc. Jpn. 88, 124708 (2019)
  * [36] Yamashita, A., Jha, R., Goto, Y., Matsuda, T.D., Aoki, Y., Mizuguchi, Y.: An efficient way of increasing the total entropy of mixing in high-entropy-alloy compounds: A case of NaCl-type (Ag,In,Pb,Bi)Te1-xSex ($x$ = 0.0, 0.25, 0.5) superconductors. Dalton Trans. 49, 9118-9122 (2020)
  * [37] Stolze, K., Cevallos, F. A., Kong, T., Cava, R. J.: High-entropy alloy superconductors on an $\alpha$-Mn lattice. J. Mater. Chem. C 6, 10441-10449 (2018)
  * [38] Xiao, G., Yang, W., Zhu, Q., Song, S., Cao, G. H., Ren, Z: Superconductivity with large upper critical field in noncentrosymmetric Cr-bearing high-entropy alloys. Scr. Mater. 223, 115099 (2023)
  * [39] Kasem, M. R., Yamashita, A., Hatano, T., Sakurai, K., Oono-Hori, N., Goto, Y., Miura, O., Mizuguchi, Y.: Anomalous broadening of specific heat jump at $T_{\mathrm{c}}$ in high-entropy-alloy-type superconductor TrZr2. Supercond. Sci. Technol. 34, 125001 (2021)
  * [40] Liu, B., Yang, W., Xiao, G., Zhu, Q., Song, S., Cao, G.H., Ren, Z.: High-entropy silicide superconductors with W5Si3-type structure. Phys. Rev. Mater. 7, 014805 (2023)
  * [41] Sogabe, R., Goto, Y., Abe, T., Moriyoshi, C., Kuroiwa, Y., Miura, A., Tadanaga, K., Mizuguchi, Y.: Improvement of superconducting properties by high mixing entropy at blocking layers in BiS2-based superconductor REO0.5F0.5BiS2. Solid State Commun. 295, 43-49 (2019)
  * [42] Shukunami, Y., Yamashita, A., Goto, Y., Mizuguchi, Y.: Synthesis of RE123 high-Tc superconductors with a high-entropy-alloy-type RE site. Physica C 572, 1353623 (2020)
  * [43] Guo, J., Wang, H., von Rohr, F., Wang, Z., Cai, S., Zhou, Y., Yang, K., Li, A., Jiang, S., Wu, Q., Cava, R.J., Sun, L.: Robust zero resistance in a superconducting high-entropy alloy at pressures up to 190 GPa. Proc. Natl. Acad. Sci. 114, 13144-13147 (2017)
  * [44] Jung, S. -G., Han, Y., Kim, J.H., Hidayati, R., Rhyee, J.-S., Lee, J.M., Kang, W.N., Choi, W.S., Jeon, H.-R., Suk, J., Park, T.: High critical current density and high-tolerance superconductivity in high-entropy alloy thin films. Nat. Commun. 13, 3373 (2022)
  * [45] Maeno, Y., Ando, T., Mori, Y., Ohmichi, E, Ikeda, S, Nishizaki, S, Nakatsuji, S.: Enhancement of Superconductivity of Sr2RuO4 to 3 K by Embedded Metallic Microdomains. Phys. Rev. Lett. 81, 3765-3768 (1998)
  * [46] Matthias, B. T., Stewart, G. R., Giorgi, A. L., Smith, J. L., Fisk, Z., Barz, H. Enhancement of superconductivity through lattice softening. Science 208, 401–402 (!980)
  * [47] Hamamoto, S., Kitagawa, J.: Superconductivity in oxygen-added Zr5Pt3. Mater. Res. Express 5, 106001 (2018)
  * [48] Rogal, Ł, Morgiel, J., Świa̧tek, Z., Czerwiński, F.: Microstructure and mechanical properties of the new Nb25Sc25Ti25Zr25 eutectic high entropy alloy. Mater. Sci. Eng. A 651, 590-597 (2016).
  * [49] Krnel, M., Jelen, A., Vrtnik, S., Luzar, J., Gačnik, D., Koželj, P., Wencka, M., Meden, A., Hu, Q., Guo, S., Dolinšek, J.: The Effect of Scandium on the Structure, Microstructure and Superconductivity of Equimolar Sc-Hf-Nb-Ta-Ti-Zr Refractory High-Entropy Alloys. Materials 15, 1122 (2022)
  * [50] Akai, H.: Electronic Structure Ni–Pd Alloys Calculated by the Self-Consistent KKR-CPA Method. J. Phys. Soc. Jpn. 51, 468-474 (1982)
  * [51] Miracle, D. B., Senkov, O. N.: A critical review of high entropy alloys and related concepts. Acta Mater. 122, 448-511 (2017)
  * [52] Matthias, B. T.: Empirical relation between superconductivity and the number of valence electrons per atom. Phys. Rev. 97, 74-76 (1955)
  * [53] von Rohr, F., Winiarski, M. J., Tao, J., Klimczuk, T., Cava, R. J.: Effect of electron count and chemical complexity in the Ta-Nb-Hf-Zr-Ti high-entropy alloy superconductor. Proc. Natl. Acad. Sci. 113, E7144-E7150 (2016)
  * [54] Sobota, P., Topolnicki, R., Ossowski, T., Pikula, T., Pikul, A., Idczak, R.: Superconductivity in the high entropy alloy (NbTa)0.67(MoHfW)0.33. Phys. Rev. B 106, 184512 (2022)
  * [55] Shu, R., Zhang, X., Rao, S. G., le Febvrier, A., Eklund, P.: Effects of alloying and deposition temperature on phase formation and superconducting properties of TiZrTaNb-based high entropy-alloy films. Appl. Phys. Lett. 120, 151901 (2022)
  * [56] Jasiewicz, K., Wiendlocha, B., Korbeń, P., Kaprzyk, S., Tobola, J.: Superconductivity of Ta34Nb33Hf8Zr14Ti11 high entropy alloy from first principles calculations. Phys. Status Solidi-Rapid Res. Lett. 10, 415-419 (2016)
  * [57] Vrtnik, S., Koželj, P., Meden, A., Maiti, S., Steurer, W., Feuerbacher, M., Dolinšek, J: Superconductivity in thermally annealed Ta-Nb-Hf-Zr-Ti high-entropy alloys. J. Alloys Compd. 695, 3530-3540 (2017)
  * [58] Gao, L., Ying, T., Zhao, Y., Cao, W., Li, C., Xiong, L., Wang, Q., Pei, C., Ge, J.-Y., Hosono, H., Qi, Y.: Fishtail effect and the vortex phase diagram of high-entropy alloy superconductor. Appl. Phys. Lett. 120, 092602 (2022)
  * [59] Dou, S.X., Wang, X.L., Horvat, J., Milliken, D., Li, A.H., Konstantinov, K., Collings, E.W., Sumption, M.D., Liu, H.K.: Flux jumping and a bulk-to-granular transition in the magnetization of a compacted and sintered MgB2 superconductor. Physica C 361, 79-83 (2001)
  * [60] Peterson, R.L.: Magnetization of anisotropic superconducting grains. J. Appl. Phys. 67, 6930–6933 (1990)


# DRL-GAN: A Hybrid Approach for Binary and Multiclass Network Intrusion
Detection

Caroline Strickland1, Chandrika Saha2, Muhammad Zakar3,
Sareh Soltani Nejad4, Noshin Tasnim5, Daniel Lizotte6, Anwar Haque7
Department of Computer Science, The University of Western Ontario, London,
Canada
{1cstrick4, 2csaha, 3mzakar, 4ssolta7, 5ntasnim3, 6dlizotte<EMAIL_ADDRESS>
###### Abstract

Our increasingly connected world continues to face an ever-growing amount of
network-based attacks. Intrusion detection systems (IDS) are an essential
security technology for detecting these attacks. Although numerous machine
learning-based IDS have been proposed for the detection of malicious network
traffic, the majority have difficulty properly detecting and classifying the
more uncommon attack types. In this paper, we implement a novel hybrid
technique using synthetic data produced by a Generative Adversarial Network
(GAN) to use as input for training a Deep Reinforcement Learning (DRL) model.
Our GAN model is trained with the NSL-KDD dataset for four attack categories
as well as normal network flow. Ultimately, our findings demonstrate that
training the DRL on specific synthetic datasets can result in better
performance in correctly classifying minority classes over training on the
true imbalanced dataset.

###### Index Terms:

Network Security, Network Intrusion Detection System, Deep Reinforcement
Learning, Generative Adversarial Networks, NSL-KDD, Machine Learning,
Artificial Intelligence.

##  I Introduction

The increasing volume and sophistication of network-based attacks motivate the
development of effective techniques and tools to prevent service disruption,
unauthorized access, and the disclosure of sensitive information [1]. An
Intrusion Detection System (IDS) is an important defence tool against
sophisticated and increasing network attacks, but these systems, especially
Machine Learning (ML) based systems, require large, reliable, and valid
network traffic datasets to be effective. Although the majority of recently
available datasets cover a range of network attack types and traffic patterns
and include information about the attacking infrastructure, modern networks
are increasingly diversified such that existing datasets are often not enough
to develop effective classification mechanisms. These datasets often suffer
from a lack of traffic diversity and volume or fail to cover the full scope of
known attack types. To cope up with these new changes, we require a more
dynamic dataset that will improve the ability of an IDS to detect intrusions.
Using deep learning techniques such as Generative Adversarial Networks (GANs),
we can fabricate additional data using existing datasets to increase the
classification accuracy of an IDS, especially for rare attack categories.

Two methods of IDS are Signature-based Intrusion Detection Systems (SNIDS) and
Anomaly-based Intrusion Detection Systems (ANIDS). The SNIDS approach is
effective for known threats, as it looks for specific patterns (or
‘signatures’) such as byte sequences in network traffic, or known malicious
instructions sequences used by malware [1]. Conversely, the ANIDS approach
uses ML algorithms to analyze and monitor the network traffic in order to
detect any suspicious activity, thus being an effective method for catching
unknown attacks [2].

The emergence of deep learning and its integration with Reinforcement Learning
(RL) has created a class of Deep Reinforcement Learning (DRL) methods that are
able to detect the most recent and sophisticated types of network attacks. DRL
combines artificial neural networks with a framework of RL that helps software
agents (or ‘learning entities’) learn how to reach their goals. DRL combines
function approximation and target optimization, mapping states and actions to
the rewards they lead to [3]. This results in a ‘policy’ that our learning
agents can follow to make the best decisions given the current state. To
detect network attacks, DRL is used to train an agent such that, given a
‘state’ represented as a collection of feature values, will take the best
‘action’ (which, in our case, acts as a classification of attack type), in
order to recognize an attack.

Each network is different in that its behaviours and patterns evolve
gradually. Naturally, vulnerabilities also evolve. The performance of IDS
classification accuracy suffers as existing datasets gradually become out of
date, invalid, and unreliable. Moreover, reliable data cannot often be shared
due to privacy concerns. Existing publicly available datasets do not include
all of the existing network attack types, let alone the unknown
vulnerabilities and attacks. To resolve this, we need more diverse and up-to-
date datasets that properly reflect the characteristics of network intrusions
in order to increase the performance of the IDS. Knowing this, we propose a
SNIDS using DRL techniques. We use a collection of GAN models to generate
varied datasets, then use DRL to implement an IDS and train the model on the
GAN-generated datasets and compare our results.

We use the open-source dataset NSL-KDD [4]. NSL-KDD is imbalanced with
significantly less attack samples than normal traffic (especially for Probe,
U2R, and R2L attacks). Thus, we used GAN to generate synthetic data so that
there is a more even class balance. We then trained the DRL model on both the
untouched NSL-KDD dataset as well as the GAN-generated data from each of our
unique models for both binary and multiclass classification. Finally, we
assess how training the DRL models using synthetic datasets compares in terms
of IDS performance as well as individual class F1-scores.

Overall, the primary contributions of this paper include:

  1. 1.

Using both conditional and unconditional CTGAN and copulaGAN models to
generate tabular data. This is useful for increasing the minority class
samples in imbalanced datasets, as well as providing large datasets for
training ML models.

  2. 2.

Combining GAN and DRL techniques for the purpose of network intrusion
detection and increasing the precision and recall for classifying
underrepresented class data. We propose a framework that trains a GAN model to
produce synthetic data, and then passes that data to a DRL model that acts as
an IDS and either alerts a user to an attack or classifies the network traffic
as benign.

The remainder of this paper is organized as follows: Section II surveys
related work for the purpose of network intrusion detection and presents the
motivation and novelty behind this work. Section III discusses methodology and
details necessary for implementation of our models. Section IV provides a
comprehensive evaluation of the obtained results. Section V presents an
interpretation of our findings. Finally, Section VI discusses directions for
future work.

##  II Related Work

Hsu and Matsuoka [1] propose a DRL model for anomaly-based network intrusion
detection. This approach treats the network traffic data as the RL environment
state variables and the outcome of intrusion detection as the action. The
correctness of the intrusion recognition result is used to determine the
reward. The novelty of this work is that the DRL agent dynamically alternates
between ‘detection mode’ and ‘learning mode’ based on whether the current
performance of the system is below a predefined threshold. In learning mode,
the performance is evaluated through the reward and the model is updated with
the new traffic data to improve the detection performance. In detection mode,
a dummy reward is used to maintain operation and the true reward of the label
is not calculated. The system was evaluated on pre-established benchmark
datasets, NSL-KDD [4] and UNSW-NB15 [5], and consistently achieved over 90% in
accuracy, recall, and precision performance metrics.

Alavizadeh et al. [6] also propose a DRL-based continuously updating, self-
learning NIDS. Their proposed Deep Q-Learning (DQL) model combines Q-learning
based RL with a deep feed forward neural network to detect network intrusions.
The model uses an ongoing trial and error auto-learning approach to improve
its detection capabilities for different types of network intrusions. The
model was evaluated on the NSL-KDD [4] dataset and outperformed some other ML
techniques with 78% classification accuracy for the intrusion classes. This
work is promising similarly to work in [1] due to its adaptive-learning
capabilities, making it better suited for securing networks from the
inevitably more sophisticated attacks cyber-attacks seen today.

Benaddi et al. [7] developed a DRL-based IDS (DRL-IDS) for Wireless Sensor
Networks (WSNs) [8] and Internet of Things (IoTs) [9]. Networking
architectures like WSNs and IoTs are receiving increasingly more adoption in
many areas such as healthcare, business, and smart cities and cyber-threats
are the primary challenge for these networks [10]. They highlight that these
networks are vulnerable to intrusions due to security flaws commonly found in
IoT and WSN devices, zero-day vulnerabilities, and the openness of these
networks to a large number of users. The DRL-IDS model improves intrusion
detection performance while monitoring real-time network traffic. The model
was evaluated against standard RL and K-Nearest Neighbours (KNN) based
approaches using the NSL-KDD [4] dataset and performed better in terms of
accuracy and detection rate with fewer false negatives.

Lin et al. [11] propose a IDSGAN, a framework that uses GANs to generate
adversarial malicious network traffic to deceive IDS. Their goal was to
leverage GANs to improve IDS by exposing them to new, more combative and
adversarial attack methods and types. This system modeled the black-box
analogy of IDS from the perspective of an attacker that would generally not
know about the internal details of the detection system. A generator
transformed known malicious traffic records into adversarial ones and a
discriminator classified the records to learn about the originally unknown
detection system. The authors demonstrated the validity of their system by
only modifying the nonfunctional features of the records such that the
modified records would still classify as an intrusion and not junk traffic.
They evaluated their system using the standard NSL-KDD [4] dataset on multiple
different detection models including Naive Bayes, Random Forest, and
multilayer perceptron classifiers. IDSGAN achieved excellent results. The
detection rate of the DoS attack type dropped from approximately 80% with
normal records to less than 1% with modified, adversarial records.

Ring et al. [12] used GANs to generate realistic flow-based network traffic
data. They highlight that the ability of GANs to only process continuous
attributes is a key challenge in using GANs to generate network traffic since
network traffic data ultimately contains categorical features like IP
addresses and ports. They propose three preprocessing techniques for
converting categorical values in flow-based network traffic data into
continuous values:  (1) Simply treat features such as IP addresses and ports
as numerical values  (2) Create binary features from the categorical features
(3) Use IP2Vec [13] to represent the categorical features as vectors.  The
authors evaluated these techniques on the CIDDS-001 [14] dataset and found
that techniques (2) and (3) are effective at generating high-quality flow-
based network traffic data. Finally, technique (1) is not well suited for this
task, meaning that straightforward numeric interpretation of categorical
features should be avoided with GANs.

Overall, there have been a handful of studies focused on using DRL to classify
network traffic as normal or intrusion, as well as several that have used GANs
to generate network traffic data. However, no study has combined these two ML
approaches and evaluated the viability and effectiveness of this combination
both in detecting and classifying network traffic as well as increasing the
precision and recall performance for classifying previously underrepresented
classes. Our proposed solution bridges this gap and improves the current state
of knowledge in this field.

##  III Methodology

###  III-A NSL-KDD Dataset

NSL-KDD is an updated version of the KDD’99 dataset [4]. Basic processing has
been done, such as the removal of redundant records preventing classifiers
from becoming biased towards more frequent records. The use of the NSL-KDD
dataset has been very popular in studies on IDS, in a sense, becoming the de
facto standard. It contains information which can help to build a host-based
and network-based intrusion detection model to ensure network security in a
variety of systems.

The training and test set contains 125 973 and 22 544 records, respectively.
This includes 42 features, however we remove ‘Num outbound cmds’ as all
records contain 0, so we are left with 41 features: 9 basic features, 12
content features for the connection, 9 temporal features calculated at two-
second time windows, 10 statistical network traffic features, and the class
label. Table I lists the features present in the dataset. The training set
contains 22 attack types and the test set contains 37 attacks types. The 15
attack types not included in the training set make this dataset excellent for
modelling unknown attacks. We opt to use the common 5-class classification of
network traffic records: normal, DoS, Probe, R2L, and U2R. Table II describes
these 5 classes in further detail. The class ID refers to the numerical
mapping used by DRL and GAN.

TABLE I: NSL-KDD Dataset Features

F# |  Feature |  F# |  Feature
---|---|---|---
F1 |  Duration |  F22 |  Is guest login
F2 |  Protocol_type |  F23 |  Count
F3 |  Service |  F24 |  Srv count
F4 |  Flag |  F25 |  Serror rate
F5 |  Src bytes |  F26 |  Srv serror rate
F6 |  Dst bytes |  F27 |  Rerror rate
F7 |  Land |  F28 |  Srv rerror rate
F8 |  Wrong fragment |  F29 |  Same srv rate
F9 |  Urgent |  F30 |  Diff srv rate
F10 |  Hot |  F31 |  Srv diff host rate
F11 |  Num_failed_logins |  F32 |  Dst host count
F12 |  Logged_in |  F33 |  Dst host srv count
F13 |  Num compromised |  F34 |  Dst host same srv rate
F14 |  Root shell |  F35 |  Dst host diff srv rate
F15 |  Su attempted |  F36 |  Dst host same src port rate
F16 |  Num root |  F37 |  Dst host srv diff host rate
F17 |  Num file creations |  F38 |  Dst host serror rate
F18 |  Num shells |  F39 |  Dst host srv serror rate
F19 |  Num access files |  F40 |  Dst host rerror rate
F20* |  Num outbound cmds |  F41 |  Dst host srv rerror rate
F21 |  Is host login |  F42 |  Class label

  * •

* Removed during data preprocessing.

TABLE II: NSL-KDD Dataset Record Classes ID | Class | Symbol | # of Records | Definition
---|---|---|---|---
0 | Normal | N | 77 054 |  Normal network traffic record
1 | DoS | D | 53 387 |  Denial of Service attack to prevent requests from intended users from being fulfilled
2 | Probe | P | 14 077 |  Probing attack to gather information such as vulnerabilities about the target machine or network
3 | R2L | R | 3880 |  An attacker tries to gain local access by sending packets to a remote machine
4 | U2R | U | 119 |  An attacker with normal access tries to gain access to the root by exploiting system vulnerabilities

###  III-B Machine Learning Performance Evaluation

We used the accuracy and F1-score (which combines precision and recall)
metrics to evaluate the performance our DRL model and other ML algorithms.
While the accuracy score only measures the percentage of correctly classified
samples, this selection of performance metrics allows us to also evaluate the
percentage of samples that were incorrectly classified. This is especially
important for NIDS as the accuracy performance metric is not enough to
evaluate imbalanced datasets such as network traffic data which generally
include significantly more normal traffic. These performance metrics are
derived from the True Positive (TP), True Negative (TN), False Positive (FP),
and False Negative (FN) values. Fig. 1 presents this confusion matrix used by
our evaluation method.

Figure 1: Confusion matrix for NIDS performance evaluation.

####  III-B1 Accuracy

Accuracy measures the number of correct predictions out of the total
predictions made by the model. In this case, accuracy measures the model’s
ability to correctly identify normal and attack traffic records. Equation 1
formalizes the accuracy performance metric.

$\mathit{Accuracy=\frac{TP+TN}{TP+FP+TN+FN}}$ (1)

####  III-B2 Precision

Precision measures the number of correct positive predictions out of the total
number of positive predictions. In this case, precision measures the model’s
degree of correctness in predicting attack records over the total number of
attacks predicted [1, 15, 16]. Equation 2 formalizes the precision performance
metric.

$\mathit{Precision=\frac{TP}{TP+FP}}$ (2)

####  III-B3 Recall

Recall measures the number of correct positive predictions out of the total
number of positive instances in the dataset. In this case, recall measures the
model’s ability to correctly identify attack traffic records. From this
definition, recall is also referred to as the true positive rate, detection
rate, or sensitivity. Equation 3 formalizes the recall performance metric.

$\mathit{Recall=\frac{TP}{TP+FN}}$ (3)

####  III-B4 F1-Score

F1-score is the harmonic mean of the precision and recall values, essentially
a combined measure of the two performance metrics. F1-score quantifies how
discriminative the model is [17] and acts as a good indicator of performance
since a decrease in either precision or recall results in a significant
decrease in the F1-score. In addition, for multiclass classification we
present both the unweighted and weighted F1-scores. The weighted F1-score
accounts for label imbalance by considering the number of instances of each
label when calculating the average F1-score. Equation 4 shows how the F1-score
is calculated.

$\displaystyle\mathit{F1\ Score}$ $\displaystyle=2\cdot\frac{Precision\cdot
Recall}{Precision+Recall}$ (4)
$\displaystyle=\frac{TP}{TP+\frac{1}{2}(FP+FN)}$

###  III-C Statistical Evaluation of Synthetic Data

To evaluate the synthetic data generated by the GAN models against the real
data they were trained on, we used statistical metrics that compare the
columns of the synthetic tables against those in the real tables. These
statistical metrics are as follows:

####  III-C1 CSTest

The CSTest compares columns with discrete values using the Chi-squared test to
compare their distributions. The output of the test is an average of the
CSTest $p$-values for each of the columns, which ultimately quantifies the
probability that the compared columns were sampled from the same distribution.

####  III-C2 KSTest

The KSTest compares columns with continuous values using the two-sample
Kolmogorov–Smirnov test and empirical Cumulative Distributed Function (CDF) to
compare their distributions. The output of the test is an average of 1 minus
the KSTest D statistic for each of the columns. This result quantifies the
maximum distance between the CDF expected and observed values.

####  III-C3 KSTestExtended

The KSTestExtended is an extension of the KSTest that converts all columns to
numerical values using a hyper transformer and then applies the regular
KSTest.

###  III-D Detection-based Evaluation of Synthetic Data

Detection metrics use ML models to determine how distinguishable the synthetic
data is from the real data. To achieve this, both the synthetic and real
tables are shuffled and a flag indicating whether the record is synthetic or
not is added. Next, cross-validation is used with a selected ML model that
predicts the flag, outputting 1 minus the average ROC AUC of all the cross-
validation splits. Because the ROC AUC measures the separability of the
classes from the model, a high detection metric score means that the model is
unable to easily distinguish the synthetic records from the real ones.

###  III-E Generative Adversarial Network Models

Goodfellow et al. [18] first proposed the idea of a GAN in 2014 as an
unsupervised learning method that generates synthetic data using an input of
real data. GANs are used to generate realistic synthetic data using real data,
usually because obtaining more data can be difficult, time consuming, and
costly. GANs use two independent models, a generator and a discriminator. By
detecting patterns or similarity from given input data, the generator
processes input data and produces more data. The discriminator is a classifier
which determines the difference between the real data and the generated data.
It produces a probability between 0 and 1 to define whether an instance
belongs to the real data (closer to 1) or to the generated data (closer to 0).
Fig. 2 highlights the overall workflow of GANs.

Figure 2: Architecture of Generative Adversarial Networks.

###  III-F Deep Reinforcement Learning Model

DRL is a subfield of ML that combines both RL and deep learning. RL considers
the problem of an agent learning to make decisions through trial and error,
while DRL incorporates deep learning, allowing agents to make decisions from
unstructured input data without manual engineering of the state space.

RL problems involve an agent learning how to map situations to actions in
order a maximize a numerical reward signal. It employs five key concepts:

  * •

Environment: The physical world that the agent operates within.

  * •

State: The agent’s belief of a configuration of the environment.

  * •

Reward: Numerical feedback from the environment.

  * •

Policy: A mapping from the agent’s state to actions.

  * •

Value: Expected future reward an agent would receive by taking an action in a
certain state.

Simply put, RL is the process of running an agent through sequences of state-
actions pairs, observing the rewards that result, and using those rewards to
formulate an optimal policy over time.

Figure 3: Architecture of Deep Reinforcement Learning.

For RL problems with small discrete state-actions spaces, the state-action
mapping can be stored in a table to approximate the mapping within a
reasonable error value. However, for problems with large state-actions spaces,
it is difficult to store such large amounts of data and, therefore,
traditional RL methods suffer in terms of memory and performance. To overcome
this, we can incorporate DRL, which is a combination of RL and deep neural
networks. A neural network can be used to approximate a value or policy
function. Essentially, neural nets learn to map states to values rather than
using a lookup table. Thus, a DRL model can independently learn to establish a
successful function for gaining maximum long-term rewards in RL problems with
large state-actions spaces.

We have defined some characteristics within our DRL model in order for it to
act as both a binary and multiclass classifier. For binary classification, we
have defined our action space as follows:

  * •

0 : No Alert (benign)

  * •

1 : Alert (attack)

And the rewards for this model are defined by:

  * •

+1 if agent correctly alerts to the correct type of attack.

  * •

0 if agent does not raise an alert when it is not needed.

  * •

-1 if agent does not raise an alert when there is an attack.

  * •

-1 if agent raises alert when there is not one needed.

For multiclass classification, we have defined our action space, also seen in
Fig. 3, as follows:

  * •

0 : No Alert (benign)

  * •

1 : DoS

  * •

2 : Probe

  * •

3 : R2L

  * •

4 : U2R

And the rewards for this model are defined by:

  * •

+1 if agent correctly alerts to the correct type of attack.

  * •

0 if agent does not raise an alert when it is not needed.

  * •

-1 if agent does not raise an alert when there is an attack.

  * •

-1 if agent raises alert when there is not one needed.

  * •

-1 if agent raises an alert to the incorrect type of attack.

In terms of network security, alerting benign network traffic is typically
safer than not alerting to an actual attack. Thus, we might consider that the
reward for the latter two cases in the above enumeration should be greater
than $-1$. However, this reward function was selected because identifying the
wrong type of attack would lead to misdirection of resources, which we want to
avoid.

Finally, the state space for both binary and multiclass classification is a
collection of 41 features, both numerical and nominal, existing within the
NSL-KDD dataset. Thus, we have an fairly complex and detailed state space. A
visual of this environment can be seen in Fig. 3.

Figure 4: Distribution of NSL-KDD dataset by record classes. Figure 5: The
action and state spaces of the proposed deep reinforcement learning model.

In addition, we have assigned two distinct conditions for terminating an
episode. An episode will be terminated if 1) it reaches a set timestep
threshold, or 2) if an attack is issued and no alert has been made.

##  IV Results

TABLE III: Statistical Metrics for Synthetic Data Synthetic Data |  CSTest |  KSTest |  KSTest Extended
---|---|---|---
CTGAN |  0.9971 |  0.9156 |  0.9181
CTGAN (Conditional) |  0.7468 |  0.8655 |  0.8571
CopulaGAN |  0.9988 |  0.9550 |  0.9574
CopulaGAN (Conditional) |  0.6982 |  0.9000 |  0.8864
TABLE IV: Discrenment Results for Synthetic Data using
Logistic Regression Synthetic Data |  Discernment Metric
---|---
CTGAN |  0.7579
CTGAN (Conditional) |  0.4139
CopulaGAN |  0.6862
CopulaGAN (Conditional) |  0.3948
TABLE V: Machine Learning Performance for Binary Classification Training Data |  Decision Tree |  AdaBoost Classifier |  Logistic Regression Classifier |  MLP Classifier |  Proposed DRL
---|---|---|---|---|---
| Accuracy | F1 | Accuracy | F1 | Accuracy | F1 | Accuracy | F1 | Accuracy | F1
NSL-KDD | 0.8407 | 0.8414 | 0.8221 | 0.8270 | 0.8700 | 0.8802 | 0.8054 | 0.8080 | 0.8951 | 0.9064
CTGAN | 0.8074 | 0.8112 | 0.8404 | 0.8486 | 0.8610 | 0.8710 | 0.8461 | 0.8545 | 0.8572 | 0.8687
CTGAN (Conditional) | 0.8801 | 0.8927 | 0.9086 | 0.9226 | 0.8740 | 0.8853 | 0.9077 | 0.9220 | 0.4662 | 0.1172
CopulaGAN | 0.7735 | 0.7607 | 0.8259 | 0.8246 | 0.8163 | 0.8201 | 0.7918 | 0.7831 | 0.8294 | 0.8375
CopulaGAN (Conditional) | 0.8287 | 0.8333 | 0.8743 | 0.8881 | 0.8256 | 0.8311 | 0.8947 | 0.9074 | 0.4901 | 0.1893
TABLE VI: Machine Learning Performance for Multi-label Classification Training Data |  Decision Tree |  MLP Classifier |  Proposed DRL
---|---|---|---
| Accuracy | F1 | F1 (weighted) | Accuracy | F1 | F1 (weighted) | Accuracy | F1 | F1 (weighted)
NSL-KDD | 0.7685 | 0.5585 | 0.7338 | 0.7856 | 0.6302 | 0.7556 | 0.7300 | 0.4880 | 0.6891
CTGAN | 0.7475 | 0.5297 | 0.7336 | 0.7765 | 0.6467 | 0.7572 | 0.4247 | 0.3033 | 0.4503
CTGAN (Conditional) | 0.6200 | 0.4475 | 0.6525 | 0.7442 | 0.5643 | 0.7791 | 0.5520 | 0.3938 | 0.4533
CopulaGAN | 0.7031 | 0.4165 | 0.6618 | 0.7374 | 0.4606 | 0.6863 | 0.7023 | 0.3967 | 0.6345
CopulaGAN (Conditional) | 0.6116 | 0.3810 | 0.6215 | 0.7088 | 0.4401 | 0.6883 | 0.4839 | 0.2716 | 0.4049

The following subsections describe the experimental results from our proposed
GAN and DRL models, followed by a comparative analysis of our proposed model
with other state-of-the-art ML methods.

###  IV-A GAN Models

For our experiments, we trained two GAN models, CTGAN [19] and CopulaGAN [20],
using the implementations provided by the SDV open-source library [21],
following work by [22] which showed promising results for generating network
traffic data using models from this library. These models were trained on the
NSL-KDD training data for 100 epochs with a batch size of 500. Both models
used discriminator steps of 5, matching WGAN [23], an extended version of
vanilla GAN. For the other hyperparameters, we opted to use the defaults
provided by the SDV library.

The trained CTGAN and CopulaGAN models were each used to generate two
synthetic datasets:

  1. 1.

A dataset containing 200 000 records generated through regular sampling
without conditions. As expected, these datasets contained records that closely
matched the imbalanced class distribution of the original NSL-KDD dataset.

  2. 2.

A dataset containing 20 000 records for each class generated using conditional
sampling through rejection. These datasets were used to explore the efficacy
of using GANs to generate a balanced distribution from an highly imbalanced
training distribution.

The statistical metric results showcased in Table III indicate that both CTGAN
and CopulaGAN model the discrete and continuous features of the NSL-KDD
dataset effectively. As dictated by the KSTest and KSTestExtended, CopulaGAN
models continuous features better than CTGAN and maintains parity for discrete
features as indicated by the CSTest. Table IV highlights the results for a
linear regression classifier used to evaluate detection performance of the
synthetic data. Altogether, the classifier found it challenging to distinguish
the synthetic records from the real ones, which indicates that the GANs are
able to capture aspects of the true dataset. Table V and Table VI showcase the
performance of ML models when trained to distinguish between various real and
synthetic datasets. Across the board, there is comparable performance between
the original real NSL-KDD dataset and the CTGAN and CopulaGAN synthetic
datasets. Thus, there is promise in using synthetic data in place of real
data.

###  IV-B DRL Model

The DRL models were implemented using both OpenAI Gym [24] and Tensorflow
[25]. Training of the model was done in two distinct stages to investigate the
variation in performance – binary classification and multiclass
classification.

####  IV-B1 Binary Classification

We begin with binary classification, using an action space of two (‘alert’ or
‘no alert’). While binary classification offers the user less knowledge on
attack type specifications, it should perform the basic task of an IDS –
alerting the user to an attack with a high accuracy.

Initially, we trained the DRL model on the NSL-KDD training set, described in
detail above. We did this to create a baseline to see how well our synthetic
GAN-generated data performed in comparison. Prior to training our model, we
converted all class labels using a binary mapping. If the class was originally
‘normal’, we assigned it a value of ‘0’, otherwise it was assigned a value of
‘1’, implying that the data point was an attack of some sort.

For each model, proximal policy optimization (PPO2), a policy-gradient
algorithm that directly optimizes the expected reward by estimating the
gradient of the policy from the trajectories taken by the agent, is executed.
We applied a custom multi-layer perceptron, a class of feedforward neural
network [26], of three layers with size 128, 64, and 32. In addition, each
model used a rectified linear unit (ReLU) activation function.

Figure 6: Results measuring the accuracy of binary classification after
training the DRL model on both the original NSL-KDD dataset and each of the
synthetic GAN datasets.

Training on the NSL-KDD training dataset for 100,000 timesteps resulted in an
average accuracy of 89.5% and an F1-score of 0.906 on the test dataset. We
then proceeded to train the DRL model on each of the GAN-generated datasets
one-by-one and evaluate them individually on the NSL-KDD test dataset. The
detailed results of these experiments can be seen in Table V, and viewed in
terms of progressive performance for average accuracy in Fig. 6.

Training on CTGAN synthetic data performs the best after the NSL-KDD trained
model, with 85.7% accuracy and 0.869 F1-score. Training using CopulaGAN
synthetic data trails close behind with 82.9% accuracy and 0.838 F1-score. The
conditional variations of both CopulaGAN and CTGAN perform significantly worse
than the three other datasets, reaching their peak of 70% and 66% respectively
almost immediately and then dropping to just below 50%.

####  IV-B2 Multiclass Classification

(a) NSL-KDD

(b) CTGAN

(c) CTGAN (Conditional)

(d) CopulaGAN

(e) CopulaGAN (Conditional)

(f) Average Accuracy

Figure 7: Results measuring the F1-scores of multiclass classification (fig a-e) as well as the averages (fig f) after training the DRL model for 2 million timesteps on both NSL-KDD as well as each synthetic dataset. TABLE VII: Class-Based F1 Scores for Multi-Label Classification Dataset | Normal | DoS | Probe | R2L | U2R
---|---|---|---|---|---
NSL-KDD | 0.7785 | 0.8072 | 0.4752 | 0.1490 | 0.0
CTGAN | 0.5670 | 0.4618 | 0.3858 | 0.0831 | 0.0192
CTGAN (Conditional) | 0.7662 | 0.0 | 0.4589 | 0.5725 | 0.1716
CopulaGAN | 0.8139 | 0.7101 | 0.4593 | 0.0 | 0.0
CopulaGAN (Conditional) | 0.8039 | 0.0 | 0.2201 | 0.2097 | 0.0512

We then trained the DRL models to perform multiclass classification. Similar
to binary classification, we are still detecting whether there is an attack or
not, however we now attempt to classify which type of attack is taking place.
Instead of ‘0’ or ‘1’, our action space consists of 0, 1, 2, 3, and 4. As
stated previously, 0 maps to ‘benign’, whereas 1, 2, 3, and 4 map to DoS,
Probe, R2L, and U2R respectively. As our action space has increased in
comparison to binary classification, our problem becomes significantly larger
and more challenging.

Like binary classification, we used a ReLU activation function, however for
the conditional versions of both CTGAN and copulaGAN we used a Sigmoid
activation function, as we found that this results in a significant increase
in performance on test data. For each model, we again used a custom multi-
layer perceptron of three layers with size 128, 64, and 32.

Again, we first analyzed the performance of our model after being trained on
the real NSL-KDD dataset in order to create a benchmark. As seen in Table VI,
our DRL model achieved 73% accuracy and a 68.9% weighted F1-score.

We then trained the DRL model on the four GAN-generated synthetic datasets
discussed previously. The most promising results were seen in training the
model on CopulaGAN. The model reaches an accuracy of 70.2% and a weighted
F1-score of 63%. This is just a 2.7% drop in accuracy from training on the
true NSL-KDD data. Training the DRL model on the remaining three synthetic
datasets underperforms when compared to both the decision tree and MLP
classifier.

As discussed previously, an F1-score refers to both precision and recall being
high. When we train on imbalanced datasets, the F1-scores in minority classes
are typically quite low, as the ML model does a poor job of recognizing and
properly classifying that test data. Looking at Table VII, we can see the
F1-scores for each individual class for each of our training sets. Since NSL-
KDD had extremely low records for both R2L and U2R, we can see that the
F1-scores for these classes are also quite low at 0.1490 and 0.0,
respectively.

One of the major goals of our work was to determine if, by generating
synthetic GAN data, we could inflate the F1-scores (more specifically,
precision and recall) of the minority classes from our imbalanced dataset. In
Table VII, we can see that training our DRL model with data generated from
conditional CTGAN and conditional CopulaGAN improved upon the F1-scores for
both R2L and U2R in the same way that we would expect to see if the true
dataset naturally contained more records of these two class types. Training
the DRL model on synthetic data from conditional CTGAN increased the F1-scores
for R2L and U2R by 0.573 and 0.172 respectively. Training on synthetic data
from conditional CopulaGAN improved the F1-scores for R2L and U2R by 0.210 and
0.051 respectively. This demonstrates that the concept of using GAN models to
generate synthetic data for a minority class and artificially inflating the
training set in order to have better performance in classifying
underrepresented classes is a viable option.

##  V Conclusion

In this paper, we have proposed a SNIDS which is able to perform binary and
multiclass classification on network traffic data. We used DRL to implement
this IDS. The model was trained using the NSL-KDD dataset, allowing it to
detect a range of attack types on a network. To enhance the learning
capabilities of our proposed model, GANs were used to fabricate training data.
Our results demonstrate that this system is able to interact with the network
and identify attack classes with competitive accuracy. As well, we show that
generating synthetic data for underrepresented classes can improve the
precision and recall within these classes, thus acting as a solution for
imbalanced datasets.

For binary classification, we obtained an 89.5% accuracy after training on the
NLS-KDD dataset. We consider this our baseline model. When trained on the four
synthetic datasets, data generated from unconditional CTGAN produced an
accuracy of 85.7%, the closest competition to the baseline model.

For multiclass classification, we obtained a 73.0% accuracy after training on
the NSL-KDD dataset. When trained on the four synthetic datasets, data
generated from CopulaGAN produced an accuracy of 70.2%, the closest
competition to the baseline model. Thus, clearly our GAN models generate data
realistic enough to create competitive IDS.

Further, both Table VII and Fig. 7 demonstrate an increase in F1-scores for
minority classes on the IDS trained using GAN-generated data. Thus, while our
overall accuracy decreased, we are getting better precision and recall
performance for the classes without sufficient data in the NSL-KDD dataset.
This points to a solution for other ML models trying to learn from imbalanced
datasets.

##  VI Future Work

While our work demonstrated competitive classifiers and an increase in
individual F1-scores for minority classes, there is still room for
improvement.

When training our GAN models, we have discussed that we used 41 features from
the NSL-KDD dataset as input to our model. There are two major changes that we
aim to implement in our future work. First, passing our input dataset through
a pipeline of feature analysis methods, including (but not limited to) Pearson
correlation, recursive feature elimination, and Lasso, with the aim to reduce
our feature space. This has the potential to increase the quality of our
generated dataset, thus increasing the evaluation metric scores for our DRL
model. Secondly, supplementing the NSL-KDD dataset with data from under-
represented classes in order to balance the dataset. Our work demonstrates
that there is a notable increase in F1-score when the class has a significant
amount of data being given as input to the GAN model. We plan to explore this
idea, and see the limitations of our performance when the GAN is trained on
significant sample sizes from each class, rather than just a small subset.

We also plan to explore GAN models that are trained only on the minority
classes of our true dataset classes. Thus, this generated data could
potentially be merged with the true dataset to allow for heightened overall
performance of the IDS, as we are synthetically creating balance.

Finally, we plan to explore the performance of training both DQN, a value-
iteration based method [27], and A3C [28], a value-iteration and policy-
gradient method, on GAN-generated data to see how it compares with our PPO2
model. Both DQN and A3C are common DRL approaches, and have the potential to
surpass the performance of our current model.

## References

  * [1] H. Ying-Feng and M. Morito, “A deep reinforcement learning approach for anomaly network intrusion detection system,” in _2020 IEEE 9th International Conference on Cloud Networking (CloudNet)_. IEEE, Nov. 2020.
  * [2] M. H. Bhuyan, D. K. Bhattacharyya, and J. K. Kalita, “Network anomaly detection: methods, systems and tools,” _Ieee communications surveys & tutorials_, vol. 16, no. 1, pp. 303–336, 2013.
  * [3] Y. Li, “Deep reinforcement learning: An overview,” _arXiv preprint arXiv:1701.07274_ , 2017.
  * [4] M. Tavallaee, E. Bagheri, W. Lu, and A. A. Ghorbani, “A detailed analysis of the kdd cup 99 data set,” in _2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications_ , 2009, pp. 1–6.
  * [5] N. Moustafa and J. Slay, “The evaluation of network anomaly detection systems: Statistical analysis of the unsw-nb15 data set and the comparison with the kdd99 data set,” pp. 1–14, 01 2016.
  * [6] H. Alavizadeh, H. Alavizadeh, and J. Jang-Jaccard, “Deep q-learning based reinforcement learning approach for network intrusion detection,” _Computers_ , vol. 11, no. 3, p. 41, 2022.
  * [7] H. Benaddi, K. Ibrahimi, A. Benslimane, and J. Qadir, “A deep reinforcement learning based intrusion detection system (DRL-IDS) for securing wireless sensor networks and internet of things,” in _Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering_ , ser. Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering. Cham: Springer International Publishing, 2020, pp. 73–87.
  * [8] H. K. Patil and T. M. Chen, “Chapter 18 - wireless sensor network security: The internet of things,” in _Computer and Information Security Handbook (Third Edition)_ , third edition ed., J. R. Vacca, Ed. Boston: Morgan Kaufmann, 2017, pp. 317–337. [Online]. Available: https://www.sciencedirect.com/science/article/pii/B9780128038437000181
  * [9] S. Kumar, P. Tiwari, and M. Zymbler, “Internet of things is a revolutionary approach for future technology enhancement: a review,” _J. Big Data_ , vol. 6, no. 1, Dec. 2019\.
  * [10] A. Ghosh, O. Khalid, R. N. B. Rais, A. Rehman, S. U. R. Malik, and I. A. Khan, “Data offloading in iot environments: modeling, analysis, and verification,” _EURASIP Journal on Wireless Communications and Networking_ , vol. 2019, no. 1, p. 53, Mar 2019. [Online]. Available: https://doi.org/10.1186/s13638-019-1358-8
  * [11] Z. Lin, Y. Shi, and Z. Xue, “IDSGAN: Generative adversarial networks for attack generation against intrusion detection,” Sep. 2018.
  * [12] M. Ring, D. Schlör, D. Landes, and A. Hotho, “Flow-based network traffic generation using generative adversarial networks,” Sep. 2018.
  * [13] M. Ring, A. Dallmann, D. Landes, and A. Hotho, “Ip2vec: Learning similarities between ip addresses,” in _2017 IEEE International Conference on Data Mining Workshops (ICDMW)_ , 2017, pp. 657–666.
  * [14] M. Ring, S. Wunderlich, D. Grüdl, D. Landes, and A. Hotho, “Flow-based benchmark data sets for intrusion detection,” 2017.
  * [15] F. Alghayadh and D. Debnath, “A hybrid intrusion detection system for smart home security,” in _2020 IEEE International Conference on Electro Information Technology (EIT)_. IEEE, Jul. 2020.
  * [16] ——, “Performance evaluation of machine learning for prediction of network traffic in a smart home,” in _2020 11th IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON)_. IEEE, Oct. 2020.
  * [17] ——, “A hybrid intrusion detection system for smart home security based on machine learning and user behavior,” _Adv. Internet Things_ , vol. 11, no. 01, pp. 10–25, 2021.
  * [18] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, and Others, “Generative adversarial networks,” 2014.
  * [19] L. Xu, M. Skoularidou, A. Cuesta-Infante, and K. Veeramachaneni, “Modeling tabular data using conditional gan,” 2019. [Online]. Available: https://arxiv.org/abs/1907.00503
  * [20] SDV, “CopulaGAN Model.” [Online]. Available: https://sdv.dev/SDV/user_guides/single_table/copulagan.html
  * [21] A. Montanez _et al._ , “Sdv: an open source library for synthetic data generation,” Ph.D. dissertation, Massachusetts Institute of Technology, 2018\.
  * [22] S. Bourou, A. El Saer, T.-H. Velivassaki, A. Voulkidis, and T. Zahariadis, “A review of tabular data synthesis using gans on an ids dataset,” _Information_ , vol. 12, no. 9, 2021. [Online]. Available: https://www.mdpi.com/2078-2489/12/9/375
  * [23] M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein gan,” 2017. [Online]. Available: https://arxiv.org/abs/1701.07875
  * [24] G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “Openai gym,” 2016.
  * [25] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mane, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viegas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, “TensorFlow: Large-scale machine learning on heterogeneous distributed systems,” Mar. 2016.
  * [26] L. Noriega, “Multilayer perceptron tutorial,” _School of Computing. Staffordshire University_ , 2005.
  * [27] S. Yoon and K.-J. Kim, “Deep q networks for visual fighting game ai,” in _2017 IEEE conference on computational intelligence and games (CIG)_. IEEE, 2017, pp. 306–308.
  * [28] M. Babaeizadeh, I. Frosio, S. Tyree, J. Clemons, and J. Kautz, “Ga3c: Gpu-based a3c for deep reinforcement learning,” _CoRR abs/1611.06256_ , 2016\.

Caroline Strickland received a B.Sc in 2017 and M.Sc in 2019 from Memorial
University of Newfoundland. She is currently in the third year of pursuing a
PhD in Computer Science at the University of Western Ontario. Her past
research has focused on using reinforcement learning for pattern formation
within swarm systems, and her current research interests involve the
intersection of hierarchical reinforcement learning with healthcare.
---
Chandrika Saha received a B.Sc in Computer Science and Engineering from the
University of Barishal, in 2019. Currently, she is pursuing her M.Sc. in
Computer Science at the Western University, London, Ontario, Canada. Her
research interest is Machine Learning, more specifically, deep learning and
its application to network security.
---
Muhammad Zakar received a B.Sc. in Computer Science from Western University,
London, Canada, in 2021. Currently, he is pursuing a M.Sc. in Computer Science
at Western University. His current research interests are in the areas of
autonomous drones and vehicles, distributed systems, next-generation networks,
and machine learning.
---
Sareh Soltani Nejad received a B.Sc. in Computer Engineering from Amirkabir
University of Technology (AUT), Tehran, Iran, in 2019. She is currently
pursuing a M.Sc. in Computer Science at the University of Western Ontario,
Canada. Her research interests broadly focus on Machine learning and Internet
of Things applications in Smart Cities, Smart homes and Healthcare.
---
Noshin Tasnim received a B.Sc. in Computer Science and Engineering from BRAC
University, Bangladesh, in 2019. She is currently pursuing a M.Sc. in Computer
Science with the Department of Computer Science, Western University, London,
Canada. Her current research interests are in the areas of network security,
and machine learning.
---
Daniel Lizotte is currently an Associate Professor in the Department of
Computer Science and in the Department of Epidemiology and Biostatistics,
University of Western Ontario, London, ON, Canada. His research group in
collaboration with community partners investigates different aspects of data-
driven decision support in public health and health care. This work aligns
with methodological research in the areas of machine learning, epidemiology,
and biostatistics. He has received funding from the Natural Sciences and
Engineering Research Council of Canada, the Canadian Institutes of Health
Research, and the Social Sciences and Humanities Research Council of Canada,
and he has served in various capacities on committees for the Machine Learning
for Health Care, International Conference on Machine Learning, and NeurIPS
conferences.
---
Anwar Haque is an Assistant Professor in the Deptartment of Computer Science
at the University of Western Ontario, Canada. Before joining Western, he was
an Associate Director at Bell Canada. He is a leading international expert on
next-generation communication network resources and performance management,
cyber security, and smart city applications. Dr. Haque has authored/co-
authored over 80 peer-reviewed research publications in leading journals and
conferences, authored many industry technical papers, and held a number of
patent/licenses. He has been awarded several national/provincial-level
research grants, including NSERC, MITACS, OCE, and SOSCIP. Dr. Haque’s
collaborative research grants are valued at more than $15 million. Dr. Haque
is serving on the inaugural advisory committee for the newly established Bell-
Western 5G research centre, and he established an industry consortium to
promote and support smart systems and digital services research at Western.
Dr. Haque is the director of the Western Information & Networking Group (WING)
Lab at Western.
---