Authors to whom correspondence should be addressed:]<EMAIL_ADDRESS><EMAIL_ADDRESS>
# Observation of Planar Hall Effect in Topological Insulator – Bi2Te3

Archit Bhardwaj  Tata Institute of Fundamental Research, Hyderabad, Telangana
500046, India  Syam Prasad P  Indian Institute of Technology Hyderabad,
Sangareddy, Telangana 502285, India  Karthik V. Raman  Tata Institute of
Fundamental Research, Hyderabad, Telangana 500046, India  Dhavala Suri1 [

###### Abstract

Planar Hall effect (PHE) in topological insulators (TIs) is discussed as an
effect that stems mostly from conduction due to topologically protected
surface states. Although surface states play a critical role and are of utmost
importance in TIs, our present study in Bi2Te3 thin films reflects the need
for considering the bulk conduction in understanding the origin of PHE in TIs.
This necessity emerges from our observation of an unconventional increase in
PHE signal with TI thickness and temperature where the bulk effect takes over.
Here we find an enhancement in PHE amplitude by doubling the Bi2Te3 film-
thickness on Si (111) substrate– from $\approx$ 1.9 n$\Omega$m in 14 quintuple
layer (QL) to $\approx$ 3.1 n$\Omega$m in 30 QL devices at B = 5 T. Also, the
PHE amplitude in the 30 QL Bi2Te3 films grown on two different substrates viz.
Si (111) and Al2O3 (0001) show an increase with temperature. Our experiments
indicate that the contribution of bulk states to PHE in TIs could be
significant.

Topological insulators (TIs) have been the forerunners of research in
condensed matter physics due to the plethora of possibilities that surface
states offer for fundamental exploration Ando (2013); Qi and Zhang (2011);
Hasan and Kane (2010). However, it is worthwhile to investigate the mesoscopic
transport dominated by the bulk states Backes  _et al._ (2017); Velkov  _et
al._ (2018); Barreto  _et al._ (2014); Checkelsky  _et al._ (2011). A
phenomenon that has evoked widespread interest in recent times is the planar
Hall effect (PHE), which is development of transverse voltage in response to
longitudinal current, under external in-plane magnetic field, in a
configuration where transverse voltage due to Lorentz force is zero. The
discovery of PHE dates back to 1954 Goldberg and Davis (1954), and has been
used extensively for its application to Hall sensors Épshtein (2002). PHE
based magnetic random access memory devices are also known for memory storage
applications Bason  _et al._ (2006). Hence, non-magnetic materials exhibiting
PHE are potential candidates for such applications since they eliminate
spurious fringe field effects. PHE has been observed in a variety of systems
such as ferromagnet/normal metal bilayers Jamali  _et al._ (2016); Safranski,
Montoya, and Krivorotov (2019), ferromagnetic semiconductors Lee  _et al._
(2009); You  _et al._ (2019), ferromagnetic metals EPS (2003) and topological
superconductors Huang  _et al._ (2018); Ge  _et al._ (2020). Recently, the
effect is in the spotlight due to its role in probing topological
characteristics such as chirality. In Weyl semimetals, the origin of PHE
coupled with negative magnetoresistance is considered a signature of chiral
anomaly Burkov (2017); Wan  _et al._ (2011); Li  _et al._ (2016); Xiong  _et
al._ (2015); Hirschberger  _et al._ (2016); Li  _et al._ (2019); Nandy  _et
al._ (2017). The origin of non-zero off-diagonal terms in the resistivity
tensor could be attributed to several mechanisms including classical orbital
magnetoresistance Li  _et al._ (2020). However, conventional PHE refers to
transverse and longitudinal resistivity under in-plane magnetic field, given
by eqn (1) below, where periodicity of PHE is $\pi$:

$\displaystyle\begin{split}\rho_{xy}&=(\rho_{\parallel}-\rho_{\perp})\sin\phi\cos\phi,\\\
\rho_{xx}&=\rho_{\perp}+(\rho_{\parallel}-\rho_{\perp})\cos^{2}\phi\end{split}$
(1)

where $\phi$ is the angle between current and magnetic field direction lying
in the same plane, $\rho_{xy}$ and $\rho_{xx}$ are transverse and longitudinal
resistivity respectively. $\rho_{\parallel}$ and $\rho_{\perp}$ are
longitudinal resistivities when $\phi$ is 0∘ and 90∘ respectively.

PHE in topological insulators is believed to stem from surface state
conduction. This was demonstrated by Taskin et. al. Taskin  _et al._ (2017)
where the PHE amplitude measured across the Fermi level exhibited local maxima
on either side of the Dirac point in the surface state regime. Bulk crystals
of Sn doped Bi1.1Sb0.9Te2S Wu  _et al._ (2018) also exhibit an oscillating PHE
which appears only in the topologically protected surface state regime. It is
interesting to note that there is no experimental report of conventional PHE
in Bi2Se3 despite being extensively studied Wiedmann  _et al._ (2016); Xia
_et al._ (2009); Analytis  _et al._ (2010). However, it does exhibit a non-
linear unconventional PHE He  _et al._ (2019), which refers to PHE with
periodicity of 2$\pi$. Such an unconventional PHE is also observed by
Rakhmilevich et. al., Rakhmilevich  _et al._ (2018) in Bi0.22Sb0.78Te3/EuS,
where induced ferromagnetism shows anisotropic magnetoresistance along with
PHE of periodicity 2$\pi$. Following these experiments several groups have
worked on explaining the results theoretically: (i) S-H Zheng et. al. Zheng
_et al._ (2020) show that PHE can arise as a consequence of anisotropic
backscattering from the Dirac cone, tilted due to in-plane magnetic field.
Including non-linear terms in momentum to the surface state Hamiltonian
explains the experimental result by Taskin et. al., without the need to invoke
scattering by impurities. (ii) Further, S. Nandy et. al. Nandy, Taraphder, and
Tewari (2018) model PHE in TIs in the bulk conduction limit using the
Boltzmann transport equation, and notice that PHE arises purely from the Berry
curvature of bulk bands. (iii) A recent calculation Suri and Soori (2021)
using scattering theory shows that PHE in surface states of a TI stems from
the transverse displacement of the dispersions under in-plane magnetic field
and extends the calculations to spin orbit coupled systems as well Soori
(2021). We recognize that a comprehensive satisfactory understanding of PHE in
topological insulators is yet to be developed. We attempt to address this and
present our experimental results.

In this article, we performed PHE experiments in thin films of Bi2Te3 by
rotating the magnetic field in-plane by an angle $\phi$ with respect to the
direction of current. Bi2Te3Li  _et al._ (2010); Zhang  _et al._ (2009);
Michiardi  _et al._ (2014); Fu (2009); Concepción  _et al._ (2019); Krumrain
_et al._ (2011); Zeng  _et al._ (2013); Analytis  _et al._ (2010); Qu  _et
al._ (2010) is a three-dimensional (3-D) TI with a small bulk band gap $<$
0.15eV Hada  _et al._ (2016) owing to which it is experimentally challenging
to separate the bulk and surface contributions to the transport. An in-plane
magnetic field has no orbital effect on the surface of the TI, but does cause
a transverse shift in the Dirac cone. Hence, the development of a response to
in-plane magnetic field is intriguing Zyuzin, Hook, and Burkov (2011);
Pershoguba and Yakovenko (2012). It is an interesting question to ask if PHE
can be observed in a bulk conduction dominated 3D-TI. To answer this question,
we examined the transport dynamics of Bi2Te3 whose relatively small bulk band
gap allows a significant mixing of the bulk and surface state signals. We
investigate PHE in Bi2Te3 films of two different thicknesses grown on
different substrates. The following sections describe the growth of the films,
the results of measurements and discuss the plausible mechanism.

High quality epitaxial films of Bi2Te3 were deposited on Si (111) and Al2O3
(0001) substrates using molecular beam epitaxy technique at a base pressure
$\approx$ 10-9 mTorr. A constant rate ratio of 1:10 was maintained during
growth of all the samples, to ensure Te rich growth atmosphere. A two-step
growth process was employed. Substrate temperature was maintained at 230 ∘C
for Al2O3 (0001) and 200 ∘ C for Si (111) respectively, optimized to attain
epitaxial film for the first 4 QL. The samples were then annealed at 270∘ C
for 30 min and the remaining thickness of the sample was grown at 270∘ C.
Thicknesses of 14 quintuple layers (QL) and 30 QL were grown on both
substrates. All the samples were capped with 2 nm of Te followed by 5 nm of
Al2O3. Transport measurements were performed in a variable temperature insert
cryostat coupled with Attocube rotation stage for angle dependent scans;
samples could reach a stable lowest temperature of 1.5 K. Keithley current
source (6221) and nanovoltmeter (2182A) were used for DC resistance
measurements. All the samples were manually patterned into Hall bars for
measurements, thus avoiding any contamination due to lithographic processes.
For rest of the manuscript we use the nomenclature of the samples: BSi14 and
BSi30 for Bi2Te3 on Si (111) of 14 QL and 30 QL respectively, BA14 and BA30
for Bi2Te3 on Al2O3 (0001) of 14 QL and 30 QL respectively.

Figure 1: (a) XRD spectrum of Bi2Te3 grown on Si (111) and Al2O3 (0001).
Resistivity versus temperature and RHEED images (inset) of 14 QL Bi2Te3 grown
on (b) Al2O3 (0001) (c) Si (111) (d) Mobility versus temperature for BSi14
(blue) and BA14 (green).

Basic characteristics of the films were analysed by XRD spectra (fig.1 (a))
which show sharp characteristic Bi2Te3 peaks. Raman spectra of the films show
characteristic Bi2Te3 phase as well (refer to supplementary info). The
resistivity versus temperature curves (fig. 1 (b)-(c)) of the two films show
metallic nature. The minor upturn in resistance at low temperatures for T $<$
10 K may be attributed to significant contribution from impurity bands Xiong
_et al._ (2012); Ren  _et al._ (2010) or the e-e interaction. RHEED images
shown in the inset exhibit highly epitaxial growth. We also remark that both
films show hexagonal symmetry corresponding to the substrate indicative of
epitaxial growth. Mobility calculated from the Hall measurements for BA14 is
higher than that of BSi14 (fig. 1 (d)). This maybe attributed to compressive
strain on the films due to lattice mismatch which is 9% for Bi2Te3 grown on
Al2O3 and 18% on Si substrates Young  _et al._ (2011); Brahleka  _et al._
(2015).

Figure 2: (a) $\rho_{xy}$ vs azimuthal angle ($\phi$) for BSi14 (blue) and
BA14 (green) with $\sin 2\phi$ fit (black line) at B = 5 T, inset shows the
schematic of measurement geometry. $\rho_{xy}$ vs $\phi$ at different magnetic
field for (b) BA14 (c) BSi14, insets show B2 fit. (d) PHE amplitude
($\rho_{xy}(\phi=45^{\circ})$) for BA14 at B = 5 T. Inset shows the
corresponding $\rho_{xy}$ vs $\phi$ .

The key result of our work is the demonstration of planar Hall effect in
Bi2Te3 and its uncommon response to temperature and film-thickness. We measure
the transverse resistivity as a function of $\phi$ (inset of fig.2 (a)).
$\rho_{xy}$ varies as $\sin 2\phi$ with a period of $\pi$ (fig.2 (a))), as
expected for the case of conventional PHE. We observe that PHE amplitude
($\rho_{xy}$ at $\phi=45$) varies monotonically with field and fits to $B^{2}$
functional form (inset of fig.2 (b-c)). The longitudinal magnetoresistance
($\rho_{xx}$) varies as $\cos^{2}\phi$ consistent with eqn. (1) for
conventional PHE (refer to suppl. info). The PHE amplitude monotonically
decreases with increase in temperature (fig. 2 (d)) indicating that either (i)
surface state contribution to PHE signal is decreasing with temperature or
(ii) bulk disorder-induced thermal excitations decrease the PHE signal from
the bulk states of BA14 and BSi14 devices (refer to supp. info PHE amplitude
vs T of Bi2Te3/Si (111)). In the low mobility BSi14, either or both of the
above effects may dominate causing a decrease in PHE amplitude with
temperature. However, in the case of BA14 devices having 6 times higher
mobility than in BSi14, the contribution of the latter effect may be less,
suggesting a more dominant surface contribution to the total PHE amplitude.
This may also explain the larger magnitude of PHE amplitude in BA14 compared
to BSi14. Experiments by Taskin et. al. Taskin  _et al._ (2017) and B. Wu et.
al. Wu  _et al._ (2018) have demonstrated that PHE dominated by surface states
decay with temperature in Bi2-xSbxTe3 and Bi1.1Sb0.9Te2S, respectively.
Alternatively, the difference in PHE amplitude of the two devices can be
explained from the chemical potential arguments. The chemical potential in
BA14 is lower than BSi14 Taskin  _et al._ (2017) suggesting a larger surface
contribution; calculated using free-electron approximation (m${}^{*}=$0.1 me,
where m${}_{e}=9.1\times 10^{-31}$ kg and m∗ is the effective mass) to be
lying 0.34 eV and 0.58 eV above the conduction band-edge for BA14 and BSi14
devices, respectively. We remark that this is only a crude approximation since
it does not consider the precise dispersion relation for each sample.

Figure 3: $\rho_{xx}$ vs B at out-of-plane and in-plane magnetic field, for
(a) BSi14 (b) BA14. Insets show magneto-conductance with HLN fit. HLN
parameters extracted from out of plane magneto-transport measurement (inset)
for both devices (c) Pre-factor ($\alpha$) vs temperature (d) Coherence length
(lφ) versus temperature. $\Delta\rho_{xx}$ = $\rho(B)-\rho(0)$.

It is fruitful to study the magneto-transport in the out-of-plane magnetic
field direction, to understand the interplay of scattering from scalar
impurities and spin-orbit coupling Liu, Hankiewicz, and Culcer (2017). As
shown in fig. 3 (a) and (b), the resistivity of the sample shows a cusp at low
field when the sample is perpendicular to the plane of magnetic field,
indicating weak anti-localization (WAL). By rotating the sample from out-of-
plane to in-plane magnetic field configuration ($\theta$ is rotated from 90∘
to 0∘), we find that the cusp vanishes and resistivity decreases. The magneto-
conductance cusp (insets of fig. 3) (a)-(b) is fit to Hikami-Larkin-Nagaosa
model (eqn. 2) Hikami, Larkin, and Nagaoka (1980) that best explains the
scattering and quantifies the coherence length (lφ) and $\alpha$ whose values
indicate the number of conducting 2D channels in the transport and the
mechanism responsible for dephasing respectively.

$\Delta G=\sigma(H)-\sigma(0)=-\alpha\frac{e^{2}}{\pi
h}\left\\{\Psi\left(\frac{1}{2}+\frac{B_{\phi}}{H}\right)-\log\left(\frac{B_{\phi}}{H}\right)\right\\}$
(2)

where $B_{\phi}=\frac{h}{8\pi e\text{l}_{\varphi}^{2}}$ and $\Psi$ is the
digamma function 111refer to suppl. info. for details on fitting routine. The
coherence length lφ at 1.7 K is $\approx$ 160 nm and 200 nm for BSi14 and BA14
respectively (fig. 3 (d)). The power law l${}_{\varphi}=aT^{\gamma}$ fit to
the temperature dependence of lφ shows that the dephasing mechanism is likely
to be e-e interaction Liu  _et al._ (2019). $\alpha$ (fig. 3 (c)) is $\approx$
0.19 and 0.23 for BA14 and BSi14, respectively. In the ideal case the value of
$\alpha$ is expected to be 1/2, for conduction due to single coherent channel.
However, the observed lower values of $\alpha$ indicate that the bulk
conduction dominates the transport in our Bi2Te3 samples irrespective of
substrate. WAL signal decays for $T>10K$, however the PHE persists up to much
higher temperatures, implying the origin of the two effects are uncorrelated.
For a comprehensive picture of the various parameters of the Bi2Te3 devices
discussed in the manuscript, we illustrate the comparison in table 1.

Thickness | 14QL | 30QL
---|---|---
Substrate | Al2O3 | Si | Al2O3 | Si
Mobility (cm2V-1s-1) | 401 | 70 | 670 | 418
Carrier conc. ($\times 10^{19}$cm-3) | 2.38 | 7.5 | 1.6 | 2.95
Lattice Constant (a=b) (Å) | 4.37 | 4.35 | 4.34 | 4.39
$\alpha$ | -0.19 | -0.23 | -0.24 | -0.24
lφ (nm) | 213 | 149 | 108 | 156
Table 1: Summary of structural and transport properties of Bi2Te3 samples
discussed in the manuscript. Figure 4: (a) $\rho_{xy}$ vs $\phi$ for BSi30,
inset shows PHE amplitude vs field with B2 fit. (b) PHE amplitude vs B for
BSi14 (blue) and BSi30 (green) with B2 fit (black) (c) $\rho_{xy}$ vs $\phi$
for BSi30 at different temperatures (d) PHE amplitude vs temperature for BSi30
and BA30. The down-turn in PHE amplitude in BSi30 at T $>$ 100 K appear due to
increasing conduction in Si at higher temperatures.

Since it is important to understand if the origin of PHE stems from the
surface or the bulk, we examine PHE in thicker samples (thickness = 30 QL) of
Bi2Te3 maintaining exactly the same growth conditions as the respective
thinner samples. Fig. 4 (a) shows the transverse resistivity as a function of
magnetic field, which exhibits a conventional behavior: PHE amplitude varies
as B2 and is sinusoidal with respect to $\phi$. We compare the relative PHE
amplitudes of the thicker and thinner film devices (fig.4 (b)). It is
intriguing that the PHE amplitude of BSi30 sample is higher than the BSi14 at
B = 5 T. However, the magnitude of PHE amplitude in BA30 drops relative to
BA14 (fig. 4 (d)). Relative to the respective thinner samples, the mobility in
BSi30 enhances $\approx$ 6 times as compared to a marginal enhancement
($\approx$ 1.7 times) in BA30; carrier concentrations of the samples do not
vary significantly (refer table I). This suggests that bulk-disorder is the
primary reason for the above contrasting trends in the magnitude of PHE
amplitude for the two sets of samples (BA14-BA30 & BSi14-BSi30). In BSi30,
with a considerable drop in bulk-disorder, PHE signal increases due to larger
bulk contribution. While in BA30, increase in thickness primarily reduces the
surface contribution to the overall PHE amplitude explaining the drop in PHE
amplitude relative to BA14. However, interestingly in both BSi30 and BA30, we
observe an increase in PHE amplitude with increase in temperature (fig. 4
(c)-(d)). This contrasting trend of PHE amplitude with temperature for thicker
films compared to the thinner films corroborates our inference that here in
thicker films bulk states predominantly contribute to the PHE signal.

We now discuss all plausible mechanisms that lead to PHE in Bi2Te3. Taskin et.
al., Taskin  _et al._ (2017) propose anisotropic lifting of topological
protection of the surface states causing PHE. However, in our case it is
evident that the carrier concentrations of the samples do not correspond to
the energy of the topologically protected surface state regime, where this
theory could be applied. The picture of chiral anomaly is ruled out since
chirality is not a well defined quantity in 3D topological insulators. Nandy
et. al.,Nandy, Taraphder, and Tewari (2018) show that a 3D-TI with negative
magnetoresistance, can exhibit PHE without the need to invoke chiral anomaly.
This occurs purely because of the Berry curvature induced anomalous magneto-
transport in the bulk limit. In our samples, correlation between weak anti-
localization and planar Hall effect seems to be less since WAL exists only at
low temperatures whereas PHE persists up to relatively much higher
temperatures. S-H Zheng et. al., Zheng  _et al._ (2020) propose the existence
of planar Hall voltage arising due to non-linear terms in momentum due to
spin-orbit coupling in the TI. By this theory, the emergence of PHE in 3D-TI
seems to be of non-topological origin and arises purely due to the tilt of the
Dirac cone in the presence of an in-plane magnetic field. From our experiments
it is clear that PHE arises despite not being in the surface state regime of
the band structure. We do not rule out the possibility that PHE is sensitive
to the Fermi level position, however this requires a rigorous calculation of
the PHE specific to the band structure of Bi2Te3. We believe that the PHE in
Bi2Te3 stems from the dominant bulk contribution, as evident from the
observation of increase in PHE signal with in temperature in our thicker
devices. While disentangling the bulk and the surface states has always been a
challenge experimentally, emergence of PHE in bulk dominated TI establishes
that the effect does not necessarily require surface state transport. This
calls for a theoretical calculation that addresses PHE specific to the band
structure of Bi2Te3. Our experiments pave the way for further theoretical
investigation of anisotropies leading to PHE in TIs.

In conclusion, our experiment demonstrates a comprehensive study of PHE in
3D-TI Bi2Te3. The PHE amplitude varies as square of the magnetic field and
agrees well with the conventional PHE equations. Interestingly, the PHE
amplitude enhances in BSi30 as compared to the BSi14. Furthermore, in contrast
to the 14 QL samples, PHE amplitude in the 30 QL samples increases with
temperature. While the origin of PHE in TIs is attributed to arise because of
topologically protected surface states, our experiment provides a different
perspective. We infer that the PHE has a significant contribution from bulk
conduction as well, paving the way for further theoretical modelling of PHE in
TIs that may assist in the future development of quantum technologies.

## Supplementary Material

Contains data of additional devices, fitting routine and description of
calculations used in the manuscript.
Acknowledgments: DS thanks Jagadeesh S. Moodera and Abhiram Soori for
illuminating discussions. All authors acknowledge extra-mural funding from
TIFR-Hyderabad, Department of Atomic Energy, Government of India, under
Project Identification No. RTI 4007, SERB Sanction No. ECR/2015/000199 and
CRG/2019/003810.
Data Availability: The data that support the findings of this study are
available from the corresponding author upon request.

## References

  * Ando (2013) Y. Ando, “Topological insulator materials,” Journal of the Physical Society of Japan 82, 102001 (2013).
  * Qi and Zhang (2011) X.-L. Qi and S.-C. Zhang, “Topological insulators and superconductors,” Rev. Mod. Phys. 83, 1057–1110 (2011).
  * Hasan and Kane (2010) M. Z. Hasan and C. L. Kane, “Colloquium: Topological insulators,” Rev. Mod. Phys. 82, 3045–3067 (2010).
  * Backes  _et al._ (2017) D. Backes, D. Huang, R. Mansell, M. Lanius, J. Kampmeier, D. Ritchie, G. Mussler, G. Gumbs, D. Grützmacher, and V. Narayan, “Disentangling surface and bulk transport in topological-insulator $p-n$ junctions,” Phys. Rev. B 96, 125125 (2017).
  * Velkov  _et al._ (2018) H. Velkov, G. N. Bremm, T. Micklitz, and G. Schwiete, “Transport in topological insulators with bulk-surface coupling: Interference corrections and conductance fluctuations,” Phys. Rev. B 98, 165408 (2018).
  * Barreto  _et al._ (2014) L. Barreto, L. Kühnemund, F. Edler, C. Tegenkamp, J. Mi, M. Bremholm, B. B. Iversen, C. Frydendahl, M. Bianchi, and P. Hofmann, “Surface-dominated transport on a bulk topological insulator,” Nano Letters 14, 3755–3760 (2014).
  * Checkelsky  _et al._ (2011) J. G. Checkelsky, Y. S. Hor, R. J. Cava, and N. P. Ong, “Bulk band gap and surface state conduction observed in voltage-tuned crystals of the topological insulator ${\mathrm{bi}}_{2}{\mathrm{se}}_{3}$,” Phys. Rev. Lett. 106, 196801 (2011).
  * Goldberg and Davis (1954) C. Goldberg and R. E. Davis, “New galvanomagnetic effect,” Phys. Rev. 94, 1121–1125 (1954).
  * Épshtein (2002) É. M. Épshtein, “Planar hall effect in ferromagnets,” Physics of the Solid State 44, 1327–1329 (2002).
  * Bason  _et al._ (2006) Y. Bason, L. Klein, J.-B. Yau, X. Hong, J. Hoffman, and C. H. Ahn, “Planar hall-effect magnetic random access memory,” Journal of Applied Physics 99, 08R701 (2006).
  * Jamali  _et al._ (2016) M. Jamali, Z. Zhao, M. DC, D. Zhang, H. Li, A. K. Smith, and J.-P. Wang, “Planar hall effect based characterization of spin orbital torques in ta/cofeb/mgo structures,” Journal of Applied Physics 119, 133902 (2016), https://doi.org/10.1063/1.4945324 .
  * Safranski, Montoya, and Krivorotov (2019) C. Safranski, E. A. Montoya, and I. N. Krivorotov, “Spin–orbit torque driven by a planar hall current,” Nature Nanotechnology 14, 27–30 (2019).
  * Lee  _et al._ (2009) S. Lee, J.-H. Chung, X. Liu, J. K. Furdyna, and B. J. Kirby, “Ferromagnetic semiconductor gamnas,” Materials Today 12, 14–21 (2009).
  * You  _et al._ (2019) Y. You, Y. Gong, H. Li, Z. Li, M. Zhu, J. Tang, E. Liu, Y. Yao, G. Xu, F. Xu, and W. Wang, “Angular dependence of the topological hall effect in the uniaxial van der waals ferromagnet ${\mathrm{fe}}_{3}{\mathrm{gete}}_{2}$,” Phys. Rev. B 100, 134441 (2019).
  * EPS (2003) “Planar hall effect in thin-film magnetic structures. cobalt films on silicon substrates,” Journal of Magnetism and Magnetic Materials 258-259, 80–83 (2003), second Moscow International Symposium on Magnetism.
  * Huang  _et al._ (2018) H. Huang, J. Gu, P. Ji, Q. Wang, X. Hu, Y. Qin, J. Wang, and C. Zhang, “Giant anisotropic magnetoresistance and planar hall effect in sr0.06bi2se3,” Applied Physics Letters 113, 222601 (2018).
  * Ge  _et al._ (2020) J. Ge, D. Ma, Y. Liu, H. Wang, Y. Li, J. Luo, T. Luo, Y. Xing, J. Yan, D. Mandrus, H. Liu, X. C. Xie, and J. Wang, “Unconventional Hall effect induced by Berry curvature,” National Science Review 7, 1879–1885 (2020).
  * Burkov (2017) A. A. Burkov, “Giant planar hall effect in topological metals,” Phys. Rev. B 96, 041110 (2017).
  * Wan  _et al._ (2011) X. Wan, A. M. Turner, A. Vishwanath, and S. Y. Savrasov, “Topological semimetal and fermi-arc surface states in the electronic structure of pyrochlore iridates,” Phys. Rev. B 83, 205101 (2011).
  * Li  _et al._ (2016) Q. Li, D. E. Kharzeev, C. Zhang, Y. Huang, I. Pletikosić, A. V. Fedorov, R. D. Zhong, J. A. Schneeloch, G. D. Gu, and T. Valla, “Chiral magnetic effect in zrte5,” Nature Physics 12, 550–554 (2016).
  * Xiong  _et al._ (2015) J. Xiong, S. K. Kushwaha, T. Liang, J. W. Krizan, M. Hirschberger, W. Wang, R. J. Cava, and N. P. Ong, “Evidence for the chiral anomaly in the dirac semimetal na3bi,” Science 350, 413–416 (2015).
  * Hirschberger  _et al._ (2016) M. Hirschberger, S. Kushwaha, Z. Wang, Q. Gibson, S. Liang, C. A. Belvin, B. A. Bernevig, R. J. Cava, and N. P. Ong, “The chiral anomaly and thermopower of weyl fermions in the half-heusler gdptbi,” Nature Materials 15, 1161–1165 (2016).
  * Li  _et al._ (2019) P. Li, C. Zhang, Y. Wen, L. Cheng, G. Nichols, D. G. Cory, G.-X. Miao, and X.-X. Zhang, “Anisotropic planar hall effect in the type-ii topological weyl semimetal ${\mathrm{wte}}_{2}$,” Phys. Rev. B 100, 205128 (2019).
  * Nandy  _et al._ (2017) S. Nandy, G. Sharma, A. Taraphder, and S. Tewari, “Chiral anomaly as the origin of the planar hall effect in weyl semimetals,” Phys. Rev. Lett. 119, 176804 (2017).
  * Li  _et al._ (2020) Z. Li, T. Xiao, R. Zou, J. Li, Y. Zhang, Y. Zeng, M. Zhou, J. Zhang, and W. Wu, “Planar hall effect in ptse2,” Journal of Applied Physics 127, 054306 (2020).
  * Taskin  _et al._ (2017) A. A. Taskin, H. F. Legg, F. Yang, S. Sasaki, Y. Kanai, K. Matsumoto, A. Rosch, and Y. Ando, “Planar hall effect from the surface of topological insulators,” Nature Communications 8, 1340 (2017).
  * Wu  _et al._ (2018) B. Wu, X.-C. Pan, W. Wu, F. Fei, B. Chen, Q. Liu, H. Bu, L. Cao, F. Song, and B. Wang, “Oscillating planar hall response in bulk crystal of topological insulator sn doped bi1.1sb0.9te2s,” Applied Physics Letters 113, 011902 (2018).
  * Wiedmann  _et al._ (2016) S. Wiedmann, A. Jost, B. Fauqué, J. van Dijk, M. J. Meijer, T. Khouri, S. Pezzini, S. Grauer, S. Schreyeck, C. Brüne, H. Buhmann, L. W. Molenkamp, and N. E. Hussey, “Anisotropic and strong negative magnetoresistance in the three-dimensional topological insulator ${\mathrm{bi}}_{2}{\mathrm{se}}_{3}$,” Phys. Rev. B 94, 081302 (2016).
  * Xia  _et al._ (2009) Y. Xia, D. Qian, D. Hsieh, L. Wray, A. Pal, H. Lin, A. Bansil, D. Grauer, Y. S. Hor, R. J. Cava, and M. Z. Hasan, “Observation of a large-gap topological-insulator class with a single dirac cone on the surface,” Nature Physics 5, 398–402 (2009).
  * Analytis  _et al._ (2010) J. G. Analytis, R. D. McDonald, S. C. Riggs, J.-H. Chu, G. S. Boebinger, and I. R. Fisher, “Two-dimensional surface state in the quantum limit of a topological insulator,” Nature Physics 6, 960–964 (2010).
  * He  _et al._ (2019) P. He, S. S.-L. Zhang, D. Zhu, S. Shi, O. G. Heinonen, G. Vignale, and H. Yang, “Nonlinear planar hall effect,” Phys. Rev. Lett. 123, 016801 (2019).
  * Rakhmilevich  _et al._ (2018) D. Rakhmilevich, F. Wang, W. Zhao, M. H. W. Chan, J. S. Moodera, C. Liu, and C.-Z. Chang, “Unconventional planar hall effect in exchange-coupled topological insulator–ferromagnetic insulator heterostructures,” Phys. Rev. B 98, 094404 (2018).
  * Zheng  _et al._ (2020) S.-H. Zheng, H.-J. Duan, J.-K. Wang, J.-Y. Li, M.-X. Deng, and R.-Q. Wang, “Origin of planar hall effect on the surface of topological insulators: Tilt of dirac cone by an in-plane magnetic field,” Phys. Rev. B 101, 041408 (2020).
  * Nandy, Taraphder, and Tewari (2018) S. Nandy, A. Taraphder, and S. Tewari, “Berry phase theory of planar hall effect in topological insulators,” Scientific Reports 8, 14983 (2018).
  * Suri and Soori (2021) D. Suri and A. Soori, “Finite transverse conductance in topological insulators under an applied in-plane magnetic field,” (2021), arXiv:2101.07670 [cond-mat.mes-hall] .
  * Soori (2021) A. Soori, “Finite transverse conductance and anisotropic magnetoconductance in spin orbit coupled two dimensional electron gas under an applied in-plane magnetic field,” (2021), arXiv:2103.05901 [cond-mat.mes-hall] .
  * Li  _et al._ (2010) Y.-Y. Li, G. Wang, X.-G. Zhu, M.-H. Liu, C. Ye, X. Chen, Y.-Y. Wang, K. He, L.-L. Wang, X.-C. Ma, H.-J. Zhang, X. Dai, Z. Fang, X.-C. Xie, Y. Liu, X.-L. Qi, J.-F. Jia, S.-C. Zhang, and Q.-K. Xue, “Intrinsic topological insulator bi2te3 thin films on si and their thickness limit,” Advanced Materials 22, 4002–4007 (2010).
  * Zhang  _et al._ (2009) H. Zhang, C.-X. Liu, X.-L. Qi, X. Dai, Z. Fang, and S.-C. Zhang, “Topological insulators in bi2se3, bi2te3 and sb2te3 with a single dirac cone on the surface,” Nature Physics 5, 438–442 (2009).
  * Michiardi  _et al._ (2014) M. Michiardi, I. Aguilera, M. Bianchi, V. E. de Carvalho, L. O. Ladeira, N. G. Teixeira, E. A. Soares, C. Friedrich, S. Blügel, and P. Hofmann, “Bulk band structure of ${\mathrm{bi}}_{2}{\mathrm{te}}_{3}$,” Phys. Rev. B 90, 075105 (2014).
  * Fu (2009) L. Fu, “Hexagonal warping effects in the surface states of the topological insulator ${\mathrm{bi}}_{2}{\mathrm{te}}_{3}$,” Phys. Rev. Lett. 103, 266801 (2009).
  * Concepción  _et al._ (2019) O. Concepción, V. Pereira, A. Choa, S. Altendorf, A. Escobosa, and O. de Melo, “The growth of bi2te3 topological insulator films: Physical vapor transport vs molecular beam epitaxy,” Materials Science in Semiconductor Processing 101, 61–66 (2019).
  * Krumrain  _et al._ (2011) J. Krumrain, G. Mussler, S. Borisova, T. Stoica, L. Plucinski, C. Schneider, and D. Grützmacher, “Mbe growth optimization of topological insulator bi2te3 films,” Journal of Crystal Growth 324, 115–118 (2011).
  * Zeng  _et al._ (2013) Z. Zeng, T. A. Morgan, D. Fan, C. Li, Y. Hirono, X. Hu, Y. Zhao, J. S. Lee, J. Wang, Z. M. Wang, S. Yu, M. E. Hawkridge, M. Benamara, and G. J. Salamo, “Molecular beam epitaxial growth of bi2te3 and sb2te3 topological insulators on gaas (111) substrates: a potential route to fabricate topological insulator p-n junction,” AIP Advances 3, 072112 (2013).
  * Qu  _et al._ (2010) D.-X. Qu, Y. S. Hor, J. Xiong, R. J. Cava, and N. P. Ong, “Quantum oscillations and hall anomaly of surface states in the topological insulator bi2te3,” Science 329, 821–824 (2010).
  * Hada  _et al._ (2016) M. Hada, K. Norimatsu, S. Tanaka, S. Keskin, T. Tsuruta, K. Igarashi, T. Ishikawa, Y. Kayanuma, R. J. D. Miller, K. Onda, T. Sasagawa, S.-y. Koshihara, and K. G. Nakamura, “Bandgap modulation in photoexcited topological insulator bi2te3 via atomic displacements,” The Journal of Chemical Physics 145, 024504 (2016).
  * Zyuzin, Hook, and Burkov (2011) A. A. Zyuzin, M. D. Hook, and A. A. Burkov, “Parallel magnetic field driven quantum phase transition in a thin topological insulator film,” Phys. Rev. B 83, 245428 (2011).
  * Pershoguba and Yakovenko (2012) S. S. Pershoguba and V. M. Yakovenko, “Spin-polarized tunneling current through a thin film of a topological insulator in a parallel magnetic field,” Phys. Rev. B 86, 165404 (2012).
  * Xiong  _et al._ (2012) J. Xiong, Y. Luo, Y. Khoo, S. Jia, R. J. Cava, and N. P. Ong, “High-field shubnikov–de haas oscillations in the topological insulator bi2te2se,” Phys. Rev. B 86, 045314 (2012).
  * Ren  _et al._ (2010) Z. Ren, A. A. Taskin, S. Sasaki, K. Segawa, and Y. Ando, “Large bulk resistivity and surface quantum oscillations in the topological insulator ${\text{bi}}_{2}{\text{te}}_{2}\text{Se}$,” Phys. Rev. B 82, 241306 (2010).
  * Young  _et al._ (2011) S. M. Young, S. Chowdhury, E. J. Walter, E. J. Mele, C. L. Kane, and A. M. Rappe, “Theoretical investigation of the evolution of the topological phase of bi2se3 under mechanical strain,” Phys. Rev. B 84, 085106 (2011).
  * Brahleka  _et al._ (2015) M. Brahleka, N. Koirala, N. Bansal, and S. Oh, “Transport properties of topological insulators: Band bending, bulk metal-to-insulator transition, and weak anti-localization,” Solid State Communications 215-216, 54–62 (2015).
  * Liu, Hankiewicz, and Culcer (2017) W. E. Liu, E. M. Hankiewicz, and D. Culcer, “Weak localization and antilocalization in topological materials with impurity spin-orbit interactions,” Materials 10 (2017).
  * Hikami, Larkin, and Nagaoka (1980) S. Hikami, A. I. Larkin, and Y. Nagaoka, “Spin-Orbit Interaction and Magnetoresistance in the Two Dimensional Random System,” Progress of Theoretical Physics 63, 707–710 (1980).
  * Note (1) Refer to suppl. info. for details on fitting routine.
  * Liu  _et al._ (2019) H. Liu, L. Bao, Z. Zhou, B. Che, R. Zhang, C. Bian, R. Ma, L. Wu, H. Yang, J. Li, C. Gu, C.-M. Shen, S. Du, and H.-J. Gao, “Quasi-2d transport and weak antilocalization effect in few-layered vse2,” Nano Letters 19, 4551–4559 (2019).


# Fast nonthermal processes in pulsed laser deposition

Jeffrey G. Ulbrandt  Xiaozhi Zhang  Department of Physics and Materials
Science Program, University of Vermont, Burlington VT 05405  Rui Liu
Department of Physics and Astronomy, Stony Brook University, Stony Brook NY
11794  Kenneth Evans-Lutterodt  National Synchrotron Light Source II, Upton NY
11967  Matthew Dawber  Department of Physics and Astronomy, Stony Brook
University, Stony Brook NY 11794  Randall L. Headrick<EMAIL_ADDRESS>Department of Physics and Materials Science Program, University of Vermont,
Burlington VT 05405

###### Abstract

Pulsed Laser Deposition (PLD) is widely used to grow epitaxial thin films of
quantum materials such as complex oxides. Here, we use in-situ X-ray
scattering to study homoepitaxy of SrTiO3 by energetic (e-) and thermalized
(th-) PLD. We find that e-PLD suppresses the lateral growth of two-dimensional
islands, which suggests that energetic particles break up smaller islands.
Fast interlayer transport occurs for both e-PLD and th-PLD, implying a process
operating on sub-microsecond timescales that doesn’t depend strongly on the
kinetic energy of the incident particles.

Pulsed Laser Deposition (PLD) is a versatile method for homoepitaxial growth
of ultra-smooth crystalline surfaces,Vasco  _et al._ (2008) heteroepitaxial
interfaces,Ohtomo and Hwang (2004); Reyren  _et al._ (2007); Caviglia  _et
al._ (2008) and superlattices.Lee  _et al._ (2005) Understanding and
controlling the growth processes occurring in PLD and other energetic
deposition methods presents an important challenge for the synthesis of thin
layers with controllable properties.

PLD employs microsecond-scale pulses that occur as the laser plume reaches the
growth surface, resulting in instantaneous deposition rates orders of
magnitude higher than in continuous deposition methods such as Molecular beam
Epitaxy (MBE). Deposited particles typically cannot diffuse a significant
lateral distance on the surface while the rest of the plume is arriving. This
is generally true over a wide range of deposition temperatures in the layer-
by-layer growth regime, which is most relevant to epitaxial growth. Thus, it
becomes possible to study fast microscopic crystallization mechanisms
separately from slower competing thermal relaxation effects by studying the
evolving X-ray scattering intensity on the relevant timescales.

Figure 1:  (a) Schematic of the X-ray scattering experiment. (b) A portion of
the scattering pattern transformed into Q-coordinates. (c) Time-of-flight
(ToF) spectra for ions in the e-PLD process. The inset shows the corresponding
energy distribution for the ionic species. (d) The corresponding ToF for th-
PLD. Figure 2:  Total specular and diffuse scattering comparison between the
energetic process (a) and the thermalized process (b) for a 6 second dwell
time between pulses. The top panels in (a) and (b) show the evolution of the
total specular intensity. and the bottom panes show the corresponding diffuse
scattering as a function of Qr. The insets in (a) and (b) show the thermal
recovery of the specular intensity for a single pulse near 0.8 ML coverage.
Note that the time axis has been converted to layer coverage for convenience,
although coverage increases in discrete steps at each laser pulse.

Previous research has focused on the possibility that arriving particles with
hyperthermal kinetic energies in the range of tens to hundreds of electron
volts typical of PLD produce energetic surface smoothening effects, including
energetic-particle island breakup and non-thermal transient enhanced mobility.
Transient enhanced mobility is thought to involve conversion of hyperthermal
kinetic energy into ballistic motion of deposited species.Fleet  _et al._
(2005); Tischler  _et al._ (2006); Vasco and Sacedón (2007) The motion can be
in-plane – like enhanced surface diffusion – or it can involve material
transferring between layers, such as when material lands on top on a unit-cell
height island and then hops over the step edge to the next lower terrace. For
example, in an experimental study of SrTiO3 homoepitaxy by Tischler et al., it
was observed that inter-layer transport occurs on two different time scales
separated by orders of magnitude.Tischler  _et al._ (2006) Another study by
Fleet et al. deduced the presence of a fast inter-layer relaxation process by
comparing the experimentally-determined X-ray scattering intensity change
immediately after the deposition pulse to the theoretical change for random
deposition without relaxation.Fleet  _et al._ (2005)

An additional energetic mechanism was suggested by Willmott et al.Willmott
_et al._ (2006) for PLD growth of La1-xSrxMnO3 on SrTiO3, in which the
energetic species in the laser plume breaks up islands into smaller daughter
islands. Island break-up is expected to delay the average island size from
reaching the critical size at which the next layer begins to nucleate. This
delay should lead to a relatively smoother growth surface. These results seem
to confirm computational predictions of a break-up effect in which adatoms are
“chipped” from the edges of larger islands.Pomeroy  _et al._ (2002) Supporting
evidence has also been found in an experimental study of Platinum homoepitaxy
by PLD, in which deposited particles with energies above 200 eV result in a
higher island nucleation density, attributed to an increase of adatoms pushed
out of the surface by the impinging energetic particles.Schmid  _et al._
(2009) On the other hand, later studies of SrTiO3 homoepitaxy utilizing X-ray
scattering measurements sensitive to in-plane structure have shown that the
surface in-plane length scale coarsens significantly during the growth.Eres
_et al._ (2016); Ferguson  _et al._ (2009) This was interpreted as evidence
that significant island breakup does not occur.

As we have mentioned above, we define the PLD regime as being the case where
all the material in the laser plume is deposited before it diffuses
significantly or nucleates islands, so that in the absence of fast non-thermal
processes the instantaneous pulse (impulse) approximation is valid. This has
important implications for the length scale of the islands, particularly
during the initial stages of recovery after a deposition pulse. In continuous
deposition the peak island density scales as $F^{\chi}$ where $F$ is the
deposition flux and $\chi$ is an exponent that depends on the size of the
critical nucleus.Venables  _et al._ (1984) On the other hand, in PLD the
nucleation density in the impulse approximation is set by the pulse intensity
$\sigma$, with a length scale of $\ell\approx a/\sigma^{1/2}$ where $a$ is the
lattice constant. This is strictly only true for the first deposited pulse,
while for later pulses there will be a distribution of islands from previous
pulses that have grown larger by aggregation and ripening in addition to
islands from the most recent deposition pulse.

Figure 3: Diffuse scattering profiles for (a) the energetic process, and (b)
the thermalized process at 0.4 ML coverage. (c) Average length scale during
first monolayer deposition. The triangles are for the e-PLD and the circles
are for the th-PLD.

Here, we present in-situ X-ray scattering measurements of homoepitaxial growth
of SrTiO3, by comparing the standard energetic deposition process (e-PLD) with
a thermalized version of PLD (th-PLD). We report two principal results: (i)
The length scale is reduced and the nucleation density is greatly increased in
e-PLD relative to th-PLD. We observe for a pulse intensity $\sigma\approx
1/20$ ML, the in-plane length scale $\ell$ in e-PLD can approach the limiting
value deduced above, which for $a=0.3905$ nm is $\approx$2 nm. Coarsening is
also limited during e-PLD, consistent with island breakup. (ii) We find that
fast interlayer transport occurs during both e-PLD and th-PLD. This result is
remarkable since by result (i) th-PLD also produces larger island sizes, which
implies that deposited particles have to migrate significantly longer
distances to reach the nearest step edge.

The experiments are performed in a custom growth chamber at the In-Situ and
Resonant Scattering (ISR) beamline at the National Synchrotron Light Source II
(NSLS-II). A schematic of the experiment is shown in Figure 1(a). Real time
X-ray scattering with an energy of 11.3 keV is performed during the growth to
extract in-plane and out-of-plane structural information about the growing
films. The measurements are done at the [0 0 1/2] anti-Bragg position to
maximize sensitivity to single unit cell height features. A broad component of
diffuse scattering is visible corresponding to unit-cell height islands on the
surface. This image corresponds to 5 ML of deposition, which results in a
relatively coarse array of islands. The scattering forms a nearly perfect
circle with a radius of $\approx$0.2 nm-1, indicating that islands are
isotropically arranged on the surface with a mean spacing of $\approx$30 nm.
In our experiments, we are sensitive in-plane diffuse scattering intensity out
to about 3 nm-1, corresponding to length scales on the order of 2 nm. That is,
we can access the whole range of length scales over which the two-dimensional
islands form, from the scale of newly-arrived particles in individual laser
pulses through the aggregation, coarsening, and coalescence stages of
monolayer growth as they occur.

Two growth processes have been developed for these experiments, which we refer
to as e-PLD and th-PLD. The energetic process uses parameters similar to
previous studies of the homoepitaxy of SrTiO3.Eres  _et al._ (2011) A
background gas of 2 mTorr O2 is used to ensure proper oxidation of the growing
surface. The laser fluence on the target is 3 J/cm2, which produces a
deposition rate of 18 to 22 pulses per monolayer. The substrate temperature is
600∘C which results in a smooth starting surface and extended layer-by-layer
growth. For the thermalized process,Shin and Aziz (2007) 300 mTorr of Helium
buffer gas is added along with the Oxygen. Time-of-flight curves obtained with
a Langmuir probe placed 8.7 cm from the laser target for each process are
shown in Figure 1(c) and (d).Doggett and Lunney (2009); sup  As the helium
background pressure is increased, the fast component shown in Figure 1(c)
becomes attenuated and it is completely replaced with a much slower component
at 300 mTorr, as shown in Figure 1(d). As a result, particle kinetic energies
are reduced from $\approx$ 100 eV/atom for e-PLD to $\approx$ 0.5 eV/atom for
th-PLD.

Surface X-ray scattering can be divided into two contributions: (i) the
specular reflection, which is only sensitive to the layer coverages, and (ii)
the diffuse intensity, which is additionally sensitive to in-plane structure.
Figure 2(a) shows both specular and diffuse intensity data during e-PLD growth
of SrTiO3 up to a final thickness of 4.5 unit cells. The top panel is the
specular intensity, which oscillates with each layer, indicating layer-by-
layer growth. A defining characteristic of PLD is that the jumps in the
specular intensity coincide with the arrival of the plume due to a sudden
change in the layer coverages. After each pulse, there is a slow evolution of
the intensity due to interlayer transport as monomers descend and attach to
step-edges from above. Comparable results for th-PLD are shown in Fig. 2(b).
The specular intensity is very similar aside from a slightly reduced intensity
at the peaks of the oscillations. The lower panels in Figs. 2 (a) and (b) show
diffuse scattering profiles versus deposited thickness. They reveal an
interesting difference that e-PLD exhibits a broad distribution centered at
$\approx$1 nm-1, which is absent from the th-PLD profiles. This peak position
corresponds to a length scale of $\approx$5 nm. As we discuss below, the
length scale of this broad profile can be as small as 2.5 nm for shorter dwell
times.

In order to gain further insight into this difference and it’s origin, Fig.
3(a,b) shows diffuse scattering profiles for various dwell times at the same
coverage of $\theta=0.4$ ML. For these plots, we performed a circular average
of the diffuse intensity at each $Q_{r}$ where
$Q_{r}=\sqrt{Q_{x}^{2}+Q_{y}^{2}}$ in order to improve the counting
statistics. Background corresponding to scattering from the substrate before
the deposition was also subtracted, as described in the Supplemental
Materials.sup  There is a clear difference in the diffuse scattering profiles,
both as a function of the dwell time and also between e-PLD and th-PLD. In
Fig. 3(c), we plot the length scale $\ell$ derived from the peak position of
the diffuse scattering, $Q_{\mathrm{r,max}}$. Comparison of the curves for
energetic and thermalized deposition reveals the remarkable result that the
thermal coarsening effect is inhibited in e-PLD, which implies that the
nucleation density is greatly increased by energetic effects. For example,
comparing the curves for 6 sec. dwell time in Fig. 3(c), we see that the
length scale for e-PLD is about a factor of three smaller than for th-PLD over
the entire range of coverages. For shorter dwell times, where thermal surface
diffusion and relaxation are less dominant in both cases, we observe that the
length scales become progressively smaller, even approaching the diffusion-
less limit of $\ell\approx$ 2 nm that we deduced from the pulse intensity.

Figure 4:  Analysis of the transient jump in the specular intensity following
each laser pulse for (a) e-PLD and (b) th-PLD. Blue symbols labelled “Jump”
represent the immediate normalized intensity change, while the red symbols
labelled “Jump + recovery” represent the total normalized intensity change at
the end of the 6 s dwell time. The dash-dotted lines are the two limiting
cases of random deposition with no inter-layer relaxation (blue) and perfect
layer-by-layer relaxation (orange).

Based on the discussion above, the first main conclusion of this study is that
the reduced in-plane length scales and coarsening rates for e-PLD are
consistent with the island break-up mechanism.Pomeroy  _et al._ (2002);
Willmott  _et al._ (2006) This process significantly counteracts the effects
of thermal surface diffusion, aggregation, and ripening, leading to an island
density that can be an order of magnitude higher for e-PLD compared to th-PLD
for long dwell times. However, a net coarsening still occurs for e-PLD, albeit
at a slower rate, as shown in Fig. 3(c), and it continues during the first few
monolayers of growth, as observed in Fig. 2(a).

The specular intensity can be expressed as a simple function of the layer
coverages. Assuming layer coverages $\theta_{n}(t)$ for layers
$n=1~{}\rm{to}~{}N$:

$I(t)\propto|F(Q)|^{2}\left|\frac{1}{1-e^{iQa}}+\sum_{n=1}^{N}\theta_{n}(t)e^{-iQna}\right|^{2},$
(1)

where $F(Q)$ is the scattering amplitude of a single unit-cell high layer and
$Q$ is the magnitude of the scattering vector, which is oriented along the
surface normal for the specular reflection.

Eq. 1 can be used to formulate several simple growth models for the intensity
at the anti-Bragg position. In one model called the impulse approximation, it
is assumed that monomers arrive at random positions on the surface during the
pulse and then remain in the same layer. This model predicts a drop in
specular intensity $I$ following each laser pulse given by $\Delta
I/I=-4\sigma(1-\sigma)$.Fleet  _et al._ (2005) The opposite limiting case is
for perfect layer-by-layer (LBL) growth, which is described by $\Delta
I/I=-4\sigma((1-\sigma)-2\theta)/(1-2\theta)^{2}$. Note that the impulse model
predicts that the intensity jump is always negative, so that in the absence of
any relaxation the intensity should continuously decrease. This is clearly
inconsistent with experimental observations of an oscillating intensity during
growth. Therefore, interlayer transport must occur following each laser pulse,
and so the interesting question becomes: on what time scale does it occur? In
order to address this question, Fig. 4 shows plots of the experimental fast
intensity jump along with the total change in the intensity after the dwell
time (labelled “jump + recovery”) for each pulse during the first monolayer of
deposition. The results show that the fast jump amplitude significantly
deviates from the impulse model. This is evidence that there must be a fast
relaxation component, a result that was previously deduced for e-PLD.Fleet
_et al._ (2005) Interestingly, the fast jump amplitudes are almost the same
for th-PLD and e-PLD, which is remarkable since previous work emphasized the
possible role of high kinetic energies ($E_{k}>10$ eV) in promoting interlayer
transport.Tischler  _et al._ (2006); Fleet  _et al._ (2005) In contrast, our
results show that the high kinetic energy of the incident particles in e-PLD
does not enhance interlayer transport relative to the more modest energies
present in th-PLD. The data for the intensity change after the dwell time is
observed to be significantly closer to the LBL model, which indicates that
thermal relaxation also plays a significant role, although it occurs on a much
slower timescale. However, there is still a significant deviation from LBL
near the middle of the growth cycle, which is related to the fact that the
specular intensity $I$ does not disappear completely near $\theta$=0.5 ML.
This effect is likely due to imperfections in the structure of the deposited
layer.

The second main conclusion of this study is that fast interlayer transport
occurs for th-PLD as well as e-PLD. We note that based on our Langmuir probe
studies that the temperature of incidence particles in th-PLD may be as high
as 4000 K. In this range, “hot” particles incident in normal incidence can
scatter nearly elastically from surface atoms, leading to a significant
velocity in the plane of the surface. For example, computational results by
Gao et al. for Pd deposition on MgO showed that Pd arriving near Mg surface
atoms can recoil with a significant lateral kinetic energy without
desorbing.Gao  _et al._ (2012) Particles incident with 0.4 eV kinetic energy
were found to travel up to 2 nm from their impact site. Our results indicate
that partially thermalized particles in SrTiO3 th-PLD travel up to $\approx$10
nm during growth.

In conclusion, our results suggest that incident particles with kinetic
energies $0.1<E_{k}<10$ eV can migrate via transient-enhanced surface
diffusion. Higher kinetic energies can break chemical bonds, leading to
phenomena such as island breakup. These mechanisms should be included in
models of epitaxial growth by hyperthermal and energetic deposition methods,
and they can be exploited to engineer thin films and multilayers with improved
properties.

The authors acknowledge the contributions of Christie Nelson. This material is
based on work that was supported by the National Science Foundation under
grant No. DMR-1506930. JU and RH were partially supported by the U.S.
Department of Energy (DOE) Office of Science under Grant No. DE-SC0017802.
This research used the 4-ID beamline of the National Synchrotron Light Source
II, a U.S. DOE Office of Science User Facility operated for the DOE Office of
Science by Brookhaven National Laboratory under Contract No. DE-SC0012704.

## References

  * Vasco  _et al._ (2008) E. Vasco, C. Polop, and J. Sacedon, Preventing kinetic roughening in physical vapor-phase-deposited films, Phys. Rev. Lett. 100, 016102 (2008).
  * Ohtomo and Hwang (2004) A. Ohtomo and H. Hwang, A high-mobility electron gas at the $\mathrm{LaAlO}_{3}$/$\mathrm{SrTiO}_{3}$ heterointerface, Nature 427, 423 (2004).
  * Reyren  _et al._ (2007) N. Reyren, S. Thiel, A. Caviglia, L. F. Kourkoutis, G. Hammerl, C. Richter, C. Schneider, T. Kopp, A.-S. Rüetschi, D. Jaccard, _et al._ , Superconducting interfaces between insulating oxides, Science 317, 1196 (2007).
  * Caviglia  _et al._ (2008) A. Caviglia, S. Gariglio, N. Reyren, D. Jaccard, T. Schneider, M. Gabay, S. Thiel, G. Hammerl, J. Mannhart, and J.-M. Triscone, Electric field control of the $\mathrm{LaAlO_{3}/SrTiO_{3}}$ interface ground state, Nature 456, 624 (2008).
  * Lee  _et al._ (2005) H. N. Lee, H. M. Christen, M. F. Chisholm, C. M. Rouleau, and D. H. Lowndes, Strong polarization enhancement in asymmetric three-component ferroelectric superlattices, Nature 433, 395 (2005).
  * Fleet  _et al._ (2005) A. Fleet, D. Dale, Y. Suzuki, and J. D. Brock, Observed effects of a changing step-edge density on thin-film growth dynamics, Phys. Rev. Lett. 94, 036102 (2005).
  * Tischler  _et al._ (2006) J. Z. Tischler, G. Eres, B. C. Larson, C. M. Rouleau, P. Zschack, and D. H. Lowndes, Nonequilibrium interlayer transport in pulsed laser deposition, Phys. Rev. Lett. 96, 226104 (2006).
  * Vasco and Sacedón (2007) E. Vasco and J. L. Sacedón, Role of cluster transient mobility in pulsed laser deposition-type growth kinetics, Phys. Rev. Lett. 98, 036104 (2007).
  * Willmott  _et al._ (2006) P. R. Willmott, R. Herger, C. M. Schlepütz, D. Martoccia, and B. D. Patterson, Energetic surface smoothing of complex metal-oxide thin films, Phys. Rev. Lett. 96, 176102 (2006).
  * Pomeroy  _et al._ (2002) J. M. Pomeroy, J. Jacobsen, C. C. Hill, B. H. Cooper, and J. P. Sethna, Kinetic monte carlo–molecular dynamics investigations of hyperthermal copper deposition on $\mathrm{Cu}$(111), Phys. Rev. B 66, 235412 (2002).
  * Schmid  _et al._ (2009) M. Schmid, C. Lenauer, A. Buchsbaum, F. Wimmer, G. Rauchbauer, P. Scheiber, G. Betz, and P. Varga, High island densities in pulsed laser deposition: Causes and implications, Phys. Rev. Lett. 103, 076101 (2009).
  * Eres  _et al._ (2016) G. Eres, J. Z. Tischler, C. M. Rouleau, H. N. Lee, H. M. Christen, P. Zschack, and B. C. Larson, Dynamic scaling and island growth kinetics in pulsed laser deposition of $\mathrm{SrTiO}_{3}$, Phys. Rev. Lett. 117, 206102 (2016).
  * Ferguson  _et al._ (2009) J. D. Ferguson, G. Arikan, D. S. Dale, A. R. Woll, and J. D. Brock, Measurements of surface diffusivity and coarsening during pulsed laser deposition, Phys. Rev. Lett. 103, 256103 (2009).
  * Venables  _et al._ (1984) J. A. Venables, G. D. T. Spiller, and M. Hanbucken, Nucleation and growth of thin films, Reports on Progress in Physics 47, 399 (1984).
  * Eres  _et al._ (2011) G. Eres, J. Z. Tischler, C. M. Rouleau, P. Zschack, H. M. Christen, and B. C. Larson, Quantitative determination of energy enhanced interlayer transport in pulsed laser deposition of $\mathrm{SrTiO}_{3}$, Phys. Rev. B 84, 195467 (2011).
  * Shin and Aziz (2007) B. Shin and M. J. Aziz, Kinetic-energy induced smoothening and delay of epitaxial breakdown in pulsed-laser deposition, Phys. Rev. B 76, 085431 (2007).
  * Doggett and Lunney (2009) B. Doggett and J. G. Lunney, Langmuir probe characterization of laser ablation plasmas, Journal of Applied Physics 105, 033306 (2009).
  * (18) See supplemental material at [url will be inserted by publisher] for details of the $\mathrm{Langmuir}$ probe method and the $\mathrm{X}$-ray diffuse scattering background subtraction.
  * Gao  _et al._ (2012) D. Z. Gao, M. B. Watkins, and A. L. Shluger, Transient mobility mechanisms of deposited metal atoms on insulating surfaces: Pd on $\mathrm{MgO}$ (100), The Journal of Physical Chemistry C 116, 14471 (2012).


# Harnessing the Power of Explanations for Incremental Training: A LIME-Based
Approach

Arnab Neelim Mazumder12, Niall Lyons1, Ashutosh Pandey1, Avik Santra1, and
Tinoosh Mohsenin2  1Infineon Technologies, Irvine, CA, USA,
2University of Maryland Baltimore County, Baltimore, MD, USA
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS>and<EMAIL_ADDRESS>
###### Abstract

Explainability of neural network prediction is essential to understand feature
importance and gain interpretable insight into neural network performance.
However, explanations of neural network outcomes are mostly limited to
visualization, and there is scarce work that looks to use these explanations
as feedback to improve model performance. In this work, model explanations are
fed back to the feed-forward training to help the model generalize better. To
this extent, a custom weighted loss where the weights are generated by
considering the Euclidean distances between true LIME (Local Interpretable
Model-Agnostic Explanations) explanations and model-predicted LIME
explanations is proposed. Also, in practical training scenarios, developing a
solution that can help the model learn sequentially without losing information
on previous data distribution is imperative due to the unavailability of all
the training data at once. Thus, the framework incorporates the custom
weighted loss with Elastic Weight Consolidation (EWC) to maintain performance
in sequential testing sets. The proposed custom training procedure results in
a consistent enhancement of accuracy ranging from 0.5% to 1.5% throughout all
phases of the incremental learning setup compared to traditional loss-based
training methods for the keyword spotting task using the Google Speech
Commands dataset.

###### Index Terms:

keyword spotting, incremental learning, LIME explanations, elastic weight
consolidation, weighted loss, explainable methods

##  I Introduction

Human-Machine interface via voice has become omnipresent in nowadays society.
A distinctive feature of voice assistants is that, in order to be used, they
first have to be activated by means of a spoken Keyword Spotting (KWS),
thereby avoiding computational expenses when it is not required. Thus, KWS can
be defined as the task of identification of keywords in audio streams
comprising speech and has become a fast-growing technology due to the paradigm
shift introduced by deep learning [1]. The earliest approach is based on
continuous KWS. One of the advantages of this approach is the flexibility to
deal with changing/non-predefined keywords. Whereas the main disadvantage of
such KWS systems might reside in the computational complexity dimension and
non-availability of all real-world sequential data at once.

Therefore it is usually desired to have Incremental Learning (IL) algorithms,
commonly referred to as continual learning [2, 3]. In a continual learning
setup, a continuously learning agent at a time step ”$t$” is trained to
recognize the tasks $1,..,t-1,t$ while the data
$\mathcal{D}_{1},\mathcal{D}_{2},\cdots,\mathcal{D}_{t-1}$, for the tasks
$1,\ldots,t-1$ may or may not be available. Such a learning paradigm has two
fundamental trade-offs to overcome. The first of these is Knowledge Transfer
(KT), which measures how incremental learning up to task $t$ influences the
agent’s knowledge about it [2]. In terms of performance, a positive KT
suggests that the agent should deliver better accuracy on the task $t$ if
allowed to learn it incrementally through tasks $1,\ldots,t-1$ while achieving
a low validation error on all of these datasets, assuming that these datasets
are seen in the order
$\mathcal{D}_{1},\mathcal{D}_{2},\cdots,\mathcal{D}_{t}$. On the other hand,
Semantic Transfer (ST) measures the influence that learning a task $t$ has on
the performance of a previous task [4]. A positive ST means that learning a
new task $t$ would increase the performance of the model on the previously
learned tasks $1,\ldots,t-1$. This compromise between learning a new task and
preserving knowledge on previously learned tasks. Multiple methods are
proposed in the literature to find the trade-off between KT and ST
(architecture-based, memory-based, regularization-based). Here, architectural
approaches (e.g., progressive nets [5]) evolve the network size after every
task while assimilating the new knowledge with the past knowledge into the new
network and memory approaches (e.g., gradient episodic memory [6]) store
memory of each of the previous tasks [7] and while learning the new task. In
contrast, the regularization method (in this case, elastic weight
consolidation) typically assumes a fixed network size and learns a new task
while trying to avoid changes to parameters sensitive to previous tasks.

In this work, a regularization-based Elastic Weight Consolidation (EWC)
approach is utilized for continual learning where parameter
$\boldsymbol{\theta}_{1:i-1}^{*}$ configuration is achieved at the end of the
dataset $i$, which is expected to solve all the datasets $\mathcal{D}_{1:i}$
[8]. Thus the posterior maximization over the new task is equivalent to
likelihood maximization for the new dataset, and the posterior maximization on
the previous dataset,

$\displaystyle\max_{\boldsymbol{\theta}}\log
p\left(\boldsymbol{\theta}\mid\mathcal{D}_{1:i}\right)$
$\displaystyle=\max_{\boldsymbol{\theta}}\left[\log
p\left(\mathcal{D}_{1:i}\mid\boldsymbol{\theta}\right)+\log
p(\boldsymbol{\theta})\right]$ (1)
$\displaystyle=\max_{\boldsymbol{\theta}}\left[\log
p\left(\mathcal{D}_{i}\mid\boldsymbol{\theta}\right)+\log
p\left(\boldsymbol{\theta}\mid\mathcal{D}_{1:i-1}\right)\right]$

Such an objective can be minimized by adding a regularization loss, which
prevents $\boldsymbol{\theta}_{1:i}^{*}$ from veering too far away from
$\boldsymbol{\theta}_{1:i-1}^{*}$. Since this regularization loss should
preserve closeness to the previous solution, the KL-divergence between
$p\left(\boldsymbol{\theta}\mid\mathcal{D}_{1:i}\right)$ and
$p\left(\boldsymbol{\theta}\mid\mathcal{D}_{1:i-1}\right)$ as the
regularization loss is used. In practice, EWC proposes using the second-order
approximation of this KL-divergence:

$KL\left(p\left(\boldsymbol{\theta}\mid\mathcal{D}_{1:i}\right)\|p\left(\boldsymbol{\theta}\mid\mathcal{D}_{1:i-1}\right)\right)\approx\frac{1}{2}\sum_{j}F_{jj}\left(\theta_{j}-\theta_{1:i-1,j}^{*}\right)^{2}$
(2)

Here, $F$ refers to the empirical Fisher matrix, only the diagonal of which is
used in the approximation.

As the network is optimized using maximum likelihood estimation, the semantic
transfer highly depends on the examples used during training and their
similarity coefficient. That is, dataset components with less similarity
suffer more semantic loss. To avoid data-dependent optimization, LIME (Local
Interpretable Model-Agnostic Explanations) [9, 10] based continual learning
where the important semantics are learned using weighted LIME scores in
combination with EWC is proposed. The contributions of this paper are:

  * •

A novel and general framework where EWC regularization is combined with model
explainability to enhance the classification performance of any neural network
is proposed.

  * •

Further, LIME scores of miss-classified samples from the previous task are
used as a weighting factor during model optimization to have better semantic
transfer learning between tasks in an IL setting and to build more generalized
models.

Section II details the proposed approach of LIME-based weighted loss IL with
Section III describing the experimental setup and elaborations of the
experiments in Section IV.

##  II Background & Proposed Framework

The existing literature ([11]) on Explainable AI (XAI) is extensive and
focuses particularly on addressing the relationship between model output and
its user. Much of this literature is exploratory in nature, as the definition
of explainability, intertwined with interpretability, is loosely defined.
However, recently some research has been conducted on the use of incorporating
XAI and IL to create robust, reliable, non-human-in-the-loop AI systems [12,
13]. The proposed solution utilizes recent XAI approaches introduced in [9,
14] and incorporates them into an IL framework.

Figure 1: A high-level overview of the proposed training methodology that
integrates explainable weight generation with EWC regularization to maintain
model performance in sequential testing sets.

The framework shown in Figure 1 aims to bridge this gap and provide a training
methodology to enhance the classification accuracy for the KWS task in-midst
of an explainable pipeline that can augment IL for the same task. The
situation where adding new data to the training regime negatively impacts the
learned distribution is known as catastrophic forgetting. In order to prevent
this, [8] proposed EWC, which forces the model to retain previous information
on top of adding new data. However, although EWC acts as a regularizer to
prevent catastrophic forgetting, it also limits the model to learn information
from the new data. To address this, we propose using a weighted loss during
model retraining where the weights from the samples come from the difference
between LIME visuals of the true and predicted classes. As a result, the model
will focus more on rectifying the incorrect predictions with higher weights,
allowing the network to learn the new data better.

###  II-A LIME Visualizations and Feature Scores

Require: $X_{in},\theta^{t}$

$C=$ Cluster$(X_{in})$

$var\leftarrow[null],\sigma=0.25$

for $i$ in $[1,2,....n]$ do

$\quad V=$ Perturbation$(C)$

$\quad var.append(V)$

end for

$pred=$ Predict$(\theta^{t},var)$

$dist=$ Cosine Distance$(X_{in},var)$

$wt=\sqrt{e^{(-dist^{2}/\sigma^{2})}}$

$reg=$ Linear Regression$(var,pred,wt)$

$score=\operatorname{coeff}(reg)$

Algorithm 1 Feature Score Generation Procedure with LIME

Saliency map-based methods like GradCAM [10], ScoreCAM [15], etc., can provide
explainable visuals of the heatmap overlayed on top of the original image
according to either weight or gradient activation. However, in the context of
enhancing the accuracy of the KWS task, determining the importance of segments
within a spectrogram to isolate activity regions is highly important. To this
end, the LIME-based visual explanations [9] to generate sample weight for the
weighted loss is adopted in this work and detailed in Algorithm 1. Initially,
the input is segmented with the slic [16] clustering algorithm. The clustered
input then goes through a perturbation process where the different segments
are turned on or off according to the binomial distribution. Each instance of
the clustered input thus generated is referred to as a variation. Next, the
trained model (for which the explanations are generated) is used to predict
the classes of all these variations. Finally, the scores are created for the
segments by fitting a linear regression classifier on the variations and their
corresponding predictions, where the cosine distance between the variations
and the original input act as weights. Since LIME explanations are based on
segmentations created on the spectrogram, it tries to fit a linear classifier
for generating an importance score for the segment itself. As a result, LIME
provides a qualitative and a quantitative metric to explain the model
prediction on an example as shown in Figure 2 (A) and (B).

Figure 2: A qualitative representation of LIME explanations for an accurate
classification (A) and a misclassification (B). (A) illustrates the five most
influential slic clusters (greyed out clusters) for making the prediction
’yes’, which coincides with the LIME explanation for the true class. (B)
demonstrates the two most influential slic clusters for making the prediction
’no’, which is different compared to the important slic clusters of the true
class ’go’.

###  II-B Weighted Loss

In a traditional training setup, all samples are provided a similar
influence/weight during loss generation. However, some samples are more
difficult to predict than others. More importantly, in an IL setting, it is
imperative to learn the incoming data rigorously while preserving the already
learned information from the distribution. In this context, using a weighted
loss function will force the model to prioritize learning the new samples in
back-propagation during the training. For example, if a sample is $X_{i}$,
with the corresponding model to be $\theta(x_{i},w)$ with $w$ parameter
weights, and the sample loss to be $L(\hat{y_{i}},y)$ then the weighted loss
function for $N$ batches in a feed-forward network is given by,

$\displaystyle Loss=\frac{1}{N}\sum_{i=1}^{N}{w_{i}L(\hat{y_{i}},y)}$ (3)

The straightforward way to generate these above-mentioned sample weights would
be to assign each incorrectly predicted sample with a high value. However,
that process might not represent which samples need more focus than others. To
address this, sample weights are generated from the explainable LIME visuals.
The weights are considered to be the Euclidean distance between the LIME
explanations for the true class $(E_{t})$ and the predicted class $(E_{p})$ as
given by the following:

$\displaystyle w=\sum_{i=1}^{i=n}{(E_{p}^{i}-E_{t}^{i})^{2}}$ (4)

Here, $n$ denotes the number of segments. The choice behind using Euclidean
distance is further justified through Table I, where it is evident that
Euclidean distance-based LIME weights outperform Manhattan and Cosine
distance-based LIME weights in terms of classification accuracy.

TABLE I: Standard error of accuracy for different distance metrics over six separate runs Distance Metric | Accuracy (%)
---|---
Euclidean | 90.92 ±0.23
Manhattan | 90.69 ±0.28
Cosine | 90.65 ±0.24

###  II-C Elastic Weight Consolidation (EWC)

To address catastrophic forgetting in an IL setting, it is imperative to have
some regularization (usually L1 or L2) during training to maintain information
from the previous distributions. However, as demonstrated in [8], L1 or L2
normalization most often constrains each weight with the same coefficient,
meaning the model can only remember the previous task at the expense of not
learning the new one. EWC provides an alternative to this predicament where
learning is slowed down on certain weights based on their importance to the
previous distribution with a quadratic penalty on the loss, as shown in
Equation 5. This scenario further enhances the learning of both tasks and
maintains the model performance on each separately. For example, if the loss
for the current task is $L_{curr}$, then EWC regularization is given by the
following:

$\displaystyle
Loss=L_{curr}+\sum_{i}{\frac{\lambda}{2}F_{i}(\theta_{i}-\theta_{A,i}^{*})^{2}}$
(5)

Here, $\theta_{A,i}$ are the parameters from the previous task, and
$\theta_{i}$ are the parameters from the current task. $F_{i}$ represents the
parameters of the previous task’s Fisher Information Matrix (FIM). $\lambda$
controls the amount of EWC regularization applied to the current loss. A lower
$\lambda$ value favors learning the new task more at the expense of forgetting
the previous task and vice versa.

Require: $D^{m},D^{v},D^{t}$

Initial Training :

1: Initialize model $\theta^{0}$

2: $\theta^{m}=\operatorname{train}\left(D^{m},\theta^{0},D^{v}\right)$

3: $Acc^{m}=\operatorname{eval}\left(\theta^{m},D^{t}\right)$

Generate LIME weights

4: for $D^{v_{i}}$ in $[D^{v_{1}},D^{v_{2}},....D^{v_{n}}]$ do

5: $\quad D^{I_{i}}=\\{(d,y)\in D^{v_{i}}|pred(\theta^{m},d)\neq y\\}$

6: $\quad LIME\\_pred^{I_{i}}=lime_{p}(\theta^{m},D^{I_{i}})$

7: $\quad LIME\\_true^{I_{i}}=lime_{t}(\theta^{m},D^{I_{i}})$

8: $\quad w^{I_{i}}=l\\_wt(LIME\\_pred^{I_{i}},LIME\\_true^{I_{i}})$

9: end for

Incremental Training :

10: Initialize $D=D^{m},\theta^{0}=\theta^{m},w=1$

11: for $D^{I_{i}}$ in $[D^{I_{1}},D^{v_{2}},....D^{I_{n}}]$ do

12: Add new data: $D_{new}=D\cup D^{I_{i}},w=w\cup w^{I_{i}}$

13: $\quad FIM=gen\\_fim(\theta^{i-1}$, 5% of $D_{new})$

14: $\quad\theta^{i}=EWC\\_trn(\theta^{i-1},D_{new},w,D^{v_{i}},FIM)$

15: $\quad Acc^{i}=eval(\theta^{i},D^{t})$

16: end for

Algorithm 2 Incremental Training Procedure with LIME-based Weighted Loss and
EWC.

##  III Experimental Setup

The Google Speech Commands dataset [17] forms the basis of our experimental
protocol. The process starts with normalizing all audio files to 16,000
samples through zero-padding. Following this, we generate spectrograms from
these files, thereby considerably diminishing the computational load for
subsequent deployment of Deep Neural Networks (DNNs). The derived spectrograms
further facilitate LIME-based explanations that serve as valuable feedback in
the learning mechanism. For this study, a VGG-like architecture is leveraged,
albeit with only four blocks and lesser filters across all layers compared to
the conventional VGG structure. This modified VGG structure is henceforth
referenced as the study’s adapted VGG architecture (illustrated in Equation
6). In this notation, 2$\times(8\,Conv_{3\times 3})$ denotes 2 consecutive
convolutional layers with 8 filters and a 3$\times$3 kernel shape,
$32\,pool_{2\times 2}$ signifies a 2$\times$2 maxpooling layer with 32
filters, and $1000\,FC$ represents a fully-connected layer hosting 1000
neurons.

$\displaystyle(in)-2\times(8\,Conv_{3\times
3})\\!-\\!2\times(16\,Conv_{3\times 3})\\!-3\times(32\,Conv_{3\times 3})\\!-$
(6) $\displaystyle(32\,pool_{2\times 2})\\!-3\times(64\,Conv_{3\times
3})\\!-(64\,pool_{2\times 2})\\!-(1000\,FC)\\!-(out)$

The comprehensive training approach is depicted in Figure 1 and elaborated in
Algorithm 2. To ensure viable sequential retraining, we partition the data
into training, validation, and test sets at 80%, 10%, and 10% respectively,
strictly adhering to speaker information for the division. This ensures
complete containment of a speaker’s utterances within a single set, thus
preventing any potential leakage of samples between sets. For the purpose of
retraining, misclassified samples from the validation sets are moved into the
training set. These samples are weighted based on the Euclidean distance
between the LIME explanation of the predicted class ($lime_{p}$) and the
actual class ($lime_{t}$), as denoted in line 8 of Algorithm 2. The stepwise
training procedure then unfolds by introducing these misclassified samples and
corresponding LIME weights into the training data $D_{m}$. This incremental
methodology continues for all sequential training iterations. The choice of
this strategy arises from the notion that the user may not always have access
to previously unseen test data or validation data. It also offers the
advantage of significant reduction in training time, given that only the
misclassified samples are considered for retraining, instead of the entire
validation set. To formulate the Fisher Information Matrix (FIM) during EWC-
based training, we randomly select 5% of the correctly predicted samples from
the expanded training set in each session. In all sessions, the retraining
commences utilizing the weights inherited from the previous session,
leveraging the Adam optimizer (with a learning rate of 0.001) and a batch size
of 512.

##  IV Results & Discussion

Figure 3: (A), (B), and (C) correspond to the error bar for two training
methods over six runs, confusion matrix for weighted loss-based training, and
traditional loss-based training, respectively, on the complete dataset. Only
the first ten classes are represented in the confusion matrix figures. The
rest of the classes follow a similar trend as well.

We perform six separate runs on the complete dataset for weighted loss
training and regular loss training to create an analogy between the two
methods. The weighted loss method results in 90.9% Top-1 accuracy, which is
around 1% better than regular loss-based training (90.2%). Figure 3 shows the
performance of the first ten classes during testing in the form of confusion
matrices. The weighted loss training method adopted in this work performs
better and gets more predictions right in most classes than traditional loss-
based training, proving that our custom loss enhances performance. To further
accommodate EWC according to Equation 5, we aim to choose the optimal
parameter ($\lambda$), which controls the influence of EWC regularization
while generating the loss. A higher $\lambda$ value allows aggressive learning
of the newer task with the caveat of forgetting the older ones. Hence we
experiment with six identical EWC and weighted loss-based IL setups for six
different $\lambda$ values. As per the line graph shown in Figure 4, it is
apparent that $\lambda=1$ provides us with the most stable performance in
terms of test set accuracy throughout all six sessions of IL.

Figure 4: The experimental insight allows us to conclude that $\lambda=1$ provides the best performance in terms of accuracy throughout the six different sessions of IL. TABLE II: Standard Error Comparison among traditional loss, weighted loss and weighted loss with EWC for 16 stage incremental learning over three separate runs # of Sess. | Trad. Loss | W_Loss | W_Loss + EWC
---|---|---|---
1 | 89.37 ±0.75 | 90.17 ±0.71 | 90.43 ±1.08
4 | 89.90 ±0.56 | 90.20 ±0.78 | 91.00 ±0.95
8 | 89.73 ±0.51 | 90.13 ±0.50 | 91.07 ±0.49
12 | 89.93 ±0.75 | 90.03 ±0.51 | 90.63 ±0.70
16 | 89.57 ±0.60 | 90.03 ±0.46 | 90.87 ±0.65

Table II compares the performance of the EWC-based weighted loss IL to
traditional IL and stand-alone weighted loss IL, respectively. The 16-stage IL
experiment suggests that the weighted loss incremental training performs
better than traditional incremental training throughout all the sessions. And,
when coupled with EWC regularization, the accuracy is better for EWC-based
weighted loss IL than the stand-alone weighted loss IL, further providing
evidence that our proposed method retains previous information better than
traditional IL setups. Moreover, we see the test accuracy follows a reducing
trend for all three methods as more incremental stages are trained. This
effect is the result of some information from the earlier stages being lost.
Finally, our proposed method has an improvement of at least 0.5-1.5% over
traditional loss based IL setup for all stages of IL and even with higher
training stages our method maintains better performance compared to both
traditional and weighted loss based IL setups.

##  V Conclusion

This work aims to take advantage of the LIME visuals of a model prediction and
use these explainable insights to improve the performance of DNNs. By
differentiating the explanations for actual and predicted classes,
proportional weights are generated for false predictions and used during the
consequent IL stage retraining. On top of it, the enhancement brought upon by
EWC regularization to maintain neural network performance across IL scenarios
is explored. The experiments suggest that the weighted loss training coupled
with EWC provides at least 0.5-1.5% improvement on all IL stages over the
traditional loss-based IL. Our future work will expand the scope of this
methodology to include weights from explainable visuals in the context of
metric learning to further bolster the classification network’s performance.

## References

  * [1] I. López-Espejo, Z.-H. Tan, J. Hansen, and J. Jensen, “Deep spoken keyword spotting: An overview,” 2021. [Online]. Available: https://arxiv.org/abs/2111.10592
  * [2] M. Riemer, I. Cases, R. Ajemian, M. Liu, I. Rish, Y. Tu, and G. Tesauro, “Learning to learn without forgetting by maximizing transfer and minimizing interference,” 2018. [Online]. Available: https://arxiv.org/abs/1810.11910
  * [3] S. Jha, M. Schiemer, and J. Ye, “Continual learning in human activity recognition: an empirical analysis of regularization,” 2020. [Online]. Available: https://arxiv.org/abs/2007.03032
  * [4] V. V. Ramasesh, E. Dyer, and M. Raghu, “Anatomy of catastrophic forgetting: Hidden representations and task semantics,” 2020. [Online]. Available: https://arxiv.org/abs/2007.07400
  * [5] A. A. Rusu, N. C. Rabinowitz, G. Desjardins, H. Soyer, J. Kirkpatrick, K. Kavukcuoglu, R. Pascanu, and R. Hadsell, “Progressive neural networks,” 2016\. [Online]. Available: https://arxiv.org/abs/1606.04671
  * [6] D. Lopez-Paz and M. Ranzato, “Gradient episodic memory for continual learning,” 2017. [Online]. Available: https://arxiv.org/abs/1706.08840
  * [7] J. Knoblauch, H. Husain, and T. Diethe, “Optimal continual learning has perfect memory and is np-hard,” 2020. [Online]. Available: https://arxiv.org/abs/2006.05188
  * [8] J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska _et al._ , “Overcoming catastrophic forgetting in neural networks,” _Proceedings of the national academy of sciences_ , vol. 114, no. 13, pp. 3521–3526, 2017.
  * [9] M. T. Ribeiro, S. Singh, and C. Guestrin, “” why should i trust you?” explaining the predictions of any classifier,” in _Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining_ , 2016, pp. 1135–1144.
  * [10] R. R. Selvaraju, A. Das, R. Vedantam, M. Cogswell, D. Parikh, and D. Batra, “Grad-cam: Why did you say that?” _arXiv preprint arXiv:1611.07450_ , 2016\.
  * [11] Z. C. Lipton, “The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery.” _Queue_ , vol. 16, no. 3, pp. 31–57, 2018.
  * [12] S. Ede, S. Baghdadlian, L. Weber, A. Nguyen, D. Zanca, W. Samek, and S. Lapuschkin, “Explain to not forget: Defending against catastrophic forgetting with xai,” in _Machine Learning and Knowledge Extraction: 6th IFIP TC 5, TC 12, WG 8.4, WG 8.9, WG 12.9 International Cross-Domain Conference, CD-MAKE 2022, Vienna, Austria, August 23–26, 2022, Proceedings_. Springer, 2022, pp. 1–18.
  * [13] L. Weber, S. Lapuschkin, A. Binder, and W. Samek, “Beyond explaining: Opportunities and challenges of xai-based model improvement,” _Information Fusion_ , 2022.
  * [14] H. Sun, L. Servadei, H. Feng, M. Stephan, R. Wille, and A. Santra, “Utilizing explainable ai for improving the performance of neural networks,” 2022. [Online]. Available: https://arxiv.org/abs/2210.04686
  * [15] H. Wang, Z. Wang, M. Du, F. Yang, Z. Zhang, S. Ding, P. Mardziel, and X. Hu, “Score-cam: Score-weighted visual explanations for convolutional neural networks,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops_ , 2020, pp. 24–25.
  * [16] R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Süsstrunk, “Slic superpixels,” Tech. Rep., 2010.
  * [17] P. Warden, “Speech commands: A dataset for limited-vocabulary speech recognition,” _arXiv preprint arXiv:1804.03209_ , 2018.