††thanks: equal contribution††thanks: equal contribution

# Errors in stimulated-Raman-induced logic gates in 133Ba+

Matthew J. Boguslawski  Department of Physics and Astronomy, University of
California Los Angeles, Los Angeles, CA, USA  Challenge Institute for Quantum
Computation, University of California Los Angeles, Los Angeles, CA, USA
<EMAIL_ADDRESS><EMAIL_ADDRESS>Zachary J. Wall  Department of
Physics and Astronomy, University of California Los Angeles, Los Angeles, CA,
USA  Samuel R. Vizvary  Department of Physics and Astronomy, University of
California Los Angeles, Los Angeles, CA, USA  Isam Daniel Moore  Department of
Physics, University of Oregon, Eugene, OR, USA  Michael Bareian  Department of
Physics and Astronomy, University of California Los Angeles, Los Angeles, CA,
USA  David T. C. Allcock  Department of Physics, University of Oregon, Eugene,
OR, USA  David J. Wineland  Department of Physics, University of Oregon,
Eugene, OR, USA  Eric R. Hudson  Department of Physics and Astronomy,
University of California Los Angeles, Los Angeles, CA, USA  Challenge
Institute for Quantum Computation, University of California Los Angeles, Los
Angeles, CA, USA  Center for Quantum Science and Engineering, University of
California Los Angeles, Los Angeles, CA, USA  Wesley C. Campbell  Department
of Physics and Astronomy, University of California Los Angeles, Los Angeles,
CA, USA  Challenge Institute for Quantum Computation, University of California
Los Angeles, Los Angeles, CA, USA  Center for Quantum Science and Engineering,
University of California Los Angeles, Los Angeles, CA, USA

###### Abstract

${}^{133}\mathrm{Ba}^{+}$ is illuminated by a laser that is far-detuned from
optical transitions, and the resulting spontaneous Raman scattering rate is
measured. The observed scattering rate is lower than previous theoretical
estimates. The majority of the discrepancy is explained by a more accurate
treatment of the scattered photon density of states. This work establishes
that, contrary to previous models, there is no fundamental limit to laser-
driven quantum gates from laser-induced spontaneous Raman scattering.

Trapped ion quantum systems have realized the highest-fidelity single- and
two-qubit gate operations to date [1, 2, 3, 4, 5]. Nonetheless, these
fidelities remain below estimated thresholds for realizing practical, fault-
tolerant quantum computers [6]. As technical limitations on fidelity are
overcome, it becomes increasingly important to understand any _fundamental_
limitations on gate fidelity, which is to say those that are inherent to a
particular architecture.

For the case of stimulated-Raman laser-driven gates on trapped ion qubits,
spontaneous Raman scattering during gates is a chief source of infidelity.
This process occurs when a far-off-resonant laser photon is incident and
another photon is spontaneously emitted, resulting in the ion either returning
to its initial state (spontaneous Rayleigh scattering) or transitioning to a
different state (spontaneous Raman scattering, SRS) [7, 8, 9, 10, 11, 12]. In
SRS, the spontaneously emitted photon carries information about the ion
internal state, thereby limiting the achievable fidelity.

Previous models of this process, here referred to as the constant density of
states approximation (CDA) [9, 10] posited the existence of a fundamental
lower limit, $\varepsilon_{\mathrm{D}\infty}$, to the achievable gate
infidelity due to spontaneous scattering to the metastable
${}^{2}\mathrm{D}_{J}$ states present in commonly used species. However, a
recent treatment (referred to here as the “Moore et al. treatment”) of photon
scattering [13] that includes the frequency-dependence of the density of
states predicts no such fundamental limit.

In this Letter, we measure spontaneous Raman scattering of a trapped 133Ba+
ion qubit under laser illumination (Fig. 1). Our results disagree with CDA
models of the scattering process [9, 10], but agree with the Moore et al.
treatment [13]. In what follows, we detail a model of the scattering process
that reproduces the main features of the experiment, describe the experimental
approach and results, and discuss implications for trapped ion quantum
information processing.

Figure 1: (a) ${}^{133}\mathrm{Ba}^{+}$ level structure. The SRS photon
frequency, $\omega_{\mathrm{sc}}$, varies depending on the laser frequency,
$\omega_{\ell}$, and the decay channel (wavy lines). (b) As the gate laser is
detuned further to the red of the intermediate resonance ($\omega_{\ell i}$,
$i=1\rightarrow 3$), the scattering rate to a metastable state ($\Gamma_{i}$,
proportional to $\omega^{3}_{\mathrm{sc}}$) decreases until the error channel
is closed (see $\omega_{\mathrm{sc}}^{3}\Theta(\omega_{\mathrm{sc}})$ in Eqn.
1).

The salient features of the observed scattering rate can be understood with a
model that stops short of the Moore et al. treatment of Ref. [13]. Using
second-order perturbation theory, the scattering rate from initial state
$|i\rangle$ (with energy $\hbar\omega_{i})$ to final state $|f\rangle$
($\hbar\omega_{f}$) through the intermediate states $|k\rangle$
($\hbar\omega_{k})$ can be estimated as:

$\Gamma_{i\rightarrow
f}=\frac{\mathcal{E}_{0}^{2}}{4\hbar^{2}}\frac{\omega_{\mathrm{sc}}^{3}\Theta(\omega_{\mathrm{sc}})}{3\pi\epsilon_{0}\hbar
c^{3}}\sum_{q}\left|\sum_{k}\frac{\langle f|d_{q}|k\rangle\langle
k|d_{\ell}|i\rangle}{\omega_{ik}-\omega_{\ell}}\right|^{2}.$ (1)

Here, $\Theta(x)$ is the Heaviside step function;
$\omega_{\mathrm{sc}}\equiv\omega_{\ell}-\omega_{if}$ is the scattered photon
frequency, given by the difference between the frequency of the incident laser
photon, $\omega_{\ell}$, and $\omega_{if}\equiv\omega_{f}-\omega_{i}$;
$\omega_{ik}\equiv\omega_{k}-\omega_{i}$;
$d_{\ell}\equiv\mathbf{d}\cdot\mbox{\boldmath$\hat{\epsilon}$}_{\ell}$ and
$d_{q}\equiv\mathbf{d}\cdot\mathbf{\hat{e}}_{q}$ are components of the
electric dipole operator $\mathbf{d}\equiv\sum_{j}e\mathbf{r}_{j}$. The role
of $\Theta(\omega_{\mathrm{sc}})$ is to enforce energy conservation when the
laser is too red to populate a particular potential final state via single-
photon absorption (see Fig. 1(b)). The applied laser, which we model as
monochromatic, produces an electric field at the ion of
$\mathbf{E}(t)=\frac{\mathcal{E}_{o}}{2}(\mbox{\boldmath$\hat{\epsilon}$}_{\ell}\,e^{-i\omega_{\ell}t}+\mbox{\boldmath$\hat{\epsilon}$}^{\ast}_{\ell}e^{i\omega_{\ell}t})$.
The total spontaneous Raman scattering rate is calculated by summing Eqn. 1
over all final states $|f\rangle$ with $f\neq i$ (i.e. omits Rayleigh
scattering). Though this model, which we refer to as the $\omega^{3}$-theory
model, neglects emission-first processes, and we restrict our basis to the
five lowest electronic states, it reproduces the observed scattering rate at
the ten percent level. Explicit expressions for the SRS rates in the CDA and
$\omega^{3}$-theory models in terms of Einstein $A$ coefficients are given in
the Supplemental Information [14].

To empirically compare the scattering behavior of trapped ions in the far-
detuned regime to these theories, we illuminate a trapped ion in a single
initial quantum state with far-detuned light and probe for internal state
changes induced due to SRS. Since far-off-resonance scattering is rare, high-
fidelity state preparation and measurement (SPAM) is desirable to discern
scattering from SPAM errors, and we therefore perform this measurement with
${}^{133}\text{Ba}^{+}$ [4]. Specifically, a single ${}^{133}\text{Ba}^{+}$
ion (see Fig. 1(a)) is confined in a linear Paul trap with a minimum ion-
electrode spacing of 3 mm driven at $\Omega=2\pi\times 2.6$ MHz, resulting in
a radial secular frequency of $\omega_{r}=2\pi\times 230$ kHz. Ions are
detected by imaging the laser induced fluorescence (LIF) of the
${}^{2}\text{S}_{1/2}$ to ${}^{2}\text{P}^{o}_{1/2}$ transition at 493 nm
through an objective with a numerical aperture of $0.28$. The ion is
illuminated by a Continuum Verdi-V10, 532 nm laser focused to a $1/e^{2}$
intensity radius $w_{0}\approx 40\,\,\upmu\mathrm{m}$ centered on the ion with
optical power between 0.3 W and 1.4 W. This wavelength is a suitable choice
for driving gates in both ${}^{2}\mathrm{S}_{1/2}$ $g$-type qubits and
${}^{2}\mathrm{D}_{5/2}$ $m$-type qubits in ${}^{133}\text{Ba}^{+}$ [15]. The
polarization of the light is set to drive $\sigma^{+}$ transitions, with a 0.5
mT magnetic field at the ion aligned antiparallel to the laser beam
$\mathbf{k}$ vector 111Beam polarization and bias field alignment was
determined using measurements of polarization-dependent Stark shifts on the
ion. Beam polarization was left-hand circular with 99.86% degree of circular
polarization at the vacuum viewport, measured with a Thorlabs PAX1000VIS
polarimeter. The bias field was aligned to within an estimated 2∘ from the
$\mathbf{k}$ vector axis. The laser intensity is determined from a
differential AC Stark shift measurement of the $o$-type clock-state qubit
defined on $|^{2}\text{S}_{1/2},F=1,m_{F}=0\rangle$ and
$|^{2}\text{D}_{5/2},F=3,m_{F}=0\rangle$ via narrow-linewidth optical
spectroscopy at 1762 nm.

The total SRS rate measurement proceeds by first preparing the ion in the
$\ket{\uparrow}\equiv\ket{F=0,m_{F}=0}$ $g$-type clock qubit state of the
${}^{2}\mathrm{S}_{1/2}$ manifold via optical pumping [4]. The ion is then
illuminated by the 532 nm laser for a chosen exposure time between 4 ms and 11
ms, during which the ion may spontaneously scatter a photon (see Fig. 1(a)).
Next, a $300\,\,\upmu\mathrm{s}$ pulse of resonant 455 nm, 585 nm and 650 nm
light is used to optically pump (“shelve”) the ion if it is in the
${}^{2}\mathrm{S}_{1/2}(F=1)$ or ${}^{2}\mathrm{D}_{3/2}$ manifolds to
${}^{2}\mathrm{D}_{5/2}$ with high fidelity [4]. Finally, population in
$\ket{\uparrow}$ is measured by monitoring LIF while Doppler cooling via 493
nm and 650 nm illumination for 8 ms, and a measurement that does not yield LIF
indicates that a SRS event occurred. The fraction of such “dark” events
recorded in the same sequence without the 532 nm laser applied is subtracted
from the laser-on experiments to yield a background-corrected probability of
SRS events. Given the small probability of a SRS event, the experiment is
repeated for $>5\cdot 10^{4}$ measurements to collect sufficient statistics,
and the probability of multiple SRS events in a single illumination time is
negligible. The SRS rate to only the 2D5/2 manifold is measured in the same
manner, omitting the shelving step. For the total SRS rate measurement, the
observed background event probability was $6(2)\cdot 10^{-4}$, primarily due
to shelving errors. As the measurement of scattering to 2D5/2 requires no
shelving step, background events are due primarily to state preparation
errors, which occur with probability $4(4)\cdot 10^{-5}$.

Figure 2:  SRS rate, $\Gamma$, from 532 nm laser illumination (green) versus
differential $o$-type Stark shift, $\delta$. The upper grouping (a) shows the
total SRS rate, while the lower grouping (b) shows rate for SRS into only
2D5/2. (Black circles) Measurements of the scattering rate. (Red, dotted) CDA
theory prediction. (Black, solid) Moore et al. treatment [13]. (Blue. dashed)
$\omega^{3}$-theory prediction.

Figure 2(a) shows the measured total SRS rate plotted against the differential
laser AC Stark shift of the $o$-type qubit and Fig. 2(b) shows the measured
the SRS rate into only the ${}^{2}\mathrm{D}_{5/2}$ manifold. The scattering
rate $\Gamma_{\mathrm{meas}}$ is extracted from the laser exposure time,
$\tau_{\ell}$, and the background-corrected, measured probability of
scattering, $P_{\mathrm{meas}}$, as
$\Gamma_{\mathrm{meas}}=-\ln{\left(1-P_{\mathrm{meas}}\right)}/\tau_{\ell}$.
The vertical error bars are calculated from the Wilson score interval for the
measured scattering events. The horizontal uncertainties are the standard
errors of the spectral peak center of the optical spectra with and without the
laser-induced Stark shift added in quadrature (smaller than the width of the
data markers in Fig. 2).

A linear best fit is calculated using an orthogonal distance regression (ODR)
[17], taking into account statistical uncertainty in the measurements of the
scattering rate and the Stark shift frequency. The Stark shift is calculated
as a function of intensity and used to convert to laser intensity [9]. The
results are summarized in Table 1. The data show good agreement with the Moore
et al. treatment of spontaneous Raman scattering [13]. The $\omega^{3}$-theory
prediction for the total SRS rate shows a $10\%$ offset due to the neglect of
the emission-first terms that are present in the Moore et al. treatment. The
measurements are in disagreement with the prediction of the CDA model,
particularly for the SRS rate to ${}^{2}\mathrm{D}_{5/2}$ states; as this
process produces a lower energy photon $\hbar\omega_{\mathrm{sc}}$ than
scattering to a ${}^{2}\mathrm{S}_{1/2}$ state, leading to the observed larger
deviation of the CDA from the data for the ${}^{2}\mathrm{D}_{5/2}$ SRS
process.

Table 1: Comparison of models with the measured value of SRS rate vs. laser intensity. Values are reported in units of SRS rate ($10^{-9}$) s-1 per (W/m2) intensity. SRS Type | CDA [10] |  $\omega^{3}$ [14] | Moore et al. [13] | Measured
---|---|---|---|---
Total ($\Gamma_{\mathrm{SRS}}$) | 1.65 | 1.37 | 1.52 | $\mathbf{1.52(5)}$
$\mathrm{D}_{5/2}\,\,(\Gamma_{\mathrm{D}_{5/2}})$ | 0.48 | 0.25 | 0.29 | $\mathbf{0.28(2)}$

To explore this disagreement further, the SRS branching _fraction_ to the
${}^{2}\mathrm{D}_{5/2}$ manifold,
$\eta_{\mathrm{D}_{5/2}}\equiv\Gamma_{\mathrm{D}_{5/2}}/\Gamma_{\mathrm{SRS}}$
was separately measured, where $\Gamma_{\mathrm{D}_{5/2}}$ and
$\Gamma_{\mathrm{SRS}}$ are the ${}^{2}\mathrm{D}_{5/2}$ and total SRS rates.
Although the Stark shift of the laser was used to monitor the intensity
stability on the ion, the result is independent of the intensity on the ion
and is simply a function of the gate-laser wavelength. Figure 3 illustrates
the striking consequence of the inclusion of the density of states
$\omega_{\mathrm{sc}}^{3}$ factor (black trace). The CDA model (red dashed
trace) shows a large asymptotic value of
$\mathrm{lim}_{\omega_{\ell}\rightarrow 0}\,\eta_{\mathrm{D}_{5/2}}=0.73$,
which implies nearly-static fields would show a significant branching fraction
to ${}^{2}\mathrm{D}_{5/2}$, a consequence that would require violation of
energy conservation. In contrast, the $\omega^{3}$-theory model (Eqn. 1) shows
scatter to ${}^{2}\mathrm{D}_{J}$ smoothly becomes forbidden once the laser
frequency is less than the ${}^{2}\mathrm{D}_{J}$ transition frequency. The
measured value of the SRS branching fraction $\eta_{\mathrm{D}_{5/2}}=0.19(1)$
is reproduced by the $\omega^{3}$-theory model (0.7$\sigma_{E}$), but is
incompatible with the CDA model (8.7$\sigma_{E}$) — here $\sigma_{E}$ is the
standard error.

Figure 3:  The SRS branching fraction to the $\mathrm{D}_{5/2}$ manifold,
$\eta_{\mathrm{D}_{5/2}}$ as a function of laser frequency. (Black, solid)
Moore et al. treatment [13]. (Red, dashed) CDA model. (Blue, dashed)
$\omega^{3}$-theory prediction. The black point shows the result of 21
measurements with the standard error displayed as error bars. The inset shows
a zoom-in near the gate-laser frequency.

For quantum information applications, it is essential to understand the
fundamental limit posed by the far-detuned SRS events. For a stimulated-Raman
transition, the probability of an ion to undergo an SRS event during the gate
time $\tau_{\ell}$ for gate lasers indexed by $j$ is approximated by
$\epsilon\approx 1-e^{-\sum_{j}\tau_{\ell}\Gamma_{j}}$. For a typical single-
qubit gate (modelled here as a $\pi$ pulse) with two Raman beams, the gate
time $\tau_{1q}=\pi/(2|\Omega_{\mathrm{R}}|)$, where $\Omega_{\mathrm{R}}$ is
the resonant, stimulated-Raman Rabi frequency. For a two-qubit gate that
traverses $K$ loops in phase space, the gate time is reduced by the coupling
strength to the motional sideband defined by the Lamb-Dicke parameter $\eta$,
giving $\tau_{2q}=\pi\sqrt{K}/(2\eta|\Omega_{\mathrm{R}}^{\prime}|)$ [18]. For
two-qubit gates, three beams may be used to produce two stimulated-Raman
couplings, with the most efficient coupling realized in a counter-propagating
configuration with two beams in one direction and the third, with twice the
intensity in the opposite direction, increasing the Rabi rate
$\Omega_{\mathrm{R}}^{\prime}$ by $\sqrt{2}$ compared to $\Omega_{R}$ for four
balanced beams. The scattering rates $\Gamma_{j}$ are given by Eqn. 1, leading
to single- and two-qubit gate errors of, respectively:

$\begin{split}\epsilon_{1q}&\approx
2\tau_{1q}\Gamma=\pi\Gamma/|\Omega_{\mathrm{R}}|\\\
\epsilon_{2q}&\approx(2)3\tau_{2q}\tilde{\Gamma}=3\pi\sqrt{K}\tilde{\Gamma}/(\eta|\Omega_{\mathrm{R}}^{\prime}|).\end{split}$
(2)

Here, the expression for $\epsilon_{2q}$ accounts for the fact that SRS by
either ion leads to an error, and $\tilde{\Gamma}$ is the scattering rate from
the average beam intensity.

Figure 4:  Calculated two-qubit gate error versus gate-laser optical
frequency. (Black, solid) Moore et al. theory using a three-beam
configuration, $K=1$ and motional sideband frequency of $2\pi\times 5$ MHz.
(Red, dashed) CDA theory for three beams, which reproduces limit
$\varepsilon_{\mathrm{D}\infty}=(1/2)1.46\cdot 10^{-4}$ for this
configuration. (Blue, dashed) $\omega^{3}$-theory prediction. (Black point)
Inferred achievable two-qubit gate error, calculated from the measured
scattering rate and Stark shifts at 532 nm.

Since the stimulated-Raman Rabi frequency $\Omega_{\mathrm{R}}$ is
proportional to the differential AC Stark shift, the fit results from Fig. 2
can be used to calculate an empirical value for the achievable error
probability for a two-qubit gate at 532 nm 222The slope of the ODR of the
total SRS data in Fig. 4 gives $\Gamma/\delta$ for SRS rate $\Gamma$ and
differential Stark shift $\delta$. At 532 nm, $\delta/\Omega_{\mathrm{R}}$ is
calculated following Eqn. 2.3 and 2.10 of [9]. Using this quantity,
$\pi(\Gamma/\delta)(\delta/\Omega_{\mathrm{R}})(3\sqrt{K}/\eta)=(\Gamma/\Omega_{\mathrm{R}})(3\pi\sqrt{K}/\eta)=\epsilon_{2q}$
from Eq. 2 is calculated at 532 nm.. In Fig. 4, we consider a two-qubit gate
with $K=1$ operating on a $2\pi\times 5$ MHz motional normal mode and find
that the inferred two-qubit gate error is 6.4(2)$\cdot 10^{-5}$, in agreement
with the Moore et al. theory value of 6.6$\cdot 10^{-5}$ [13]. This is in
contrast to the value provided in Table III of Ref. [10], where the CDA
results in the conclusion that errors below
$\varepsilon_{\mathrm{D}\infty}=(1/2)1.46\cdot 10^{-4}=7.3\cdot 10^{-5}$ (for
a three beam configuration) detuned outside the ${}^{2}\mathrm{P}_{J}^{o}$
levels are not possible in $\mathrm{Ba}^{+}$ under _any_ conditions. Further,
for 532 nm, the CDA theory predicts an error of $1.05\cdot 10^{-4}$, also in
disagreement with our measurement-based estimate. Notably, the updated models
as well as the infidelity inferred from measurements with a 532 nm gate laser
reach below the $10^{-4}$ threshold anticipated for efficient error correction
[6]. With the removal of the asymptotic error rate predicted by the CDA model,
additional reductions to the SRS error rate can be found by, for example,
choosing a gate laser further red-detuned from the ${}^{2}\mathrm{P}_{J}^{o}$
levels.

In summary, the Raman scattering rate of ${}^{133}\mathrm{Ba}^{+}$ under
illumination by laser light at 532 nm was studied and found to be smaller than
previously predicted. The observed scattering rate agrees with more recent and
more complete models, with the difference in models largely due to the
inclusion of the scattered photon density of states. This result has important
consequences. For example, in contradiction to previous predictions for laser-
based gates, there is no fundamental limit to achievable gate error from
spontaneous Raman scattering for trapped ion quantum processing since
$\epsilon_{2q}\rightarrow 0$ as $\omega_{\ell}\rightarrow 0$. Also, in
contradiction to previous predictions [10], gate errors below $10^{-4}$ are
achievable in ${}^{133}\mathrm{Ba}^{+}$ at the technologically convenient
wavelength of 532 nm.

We thank Clayton Ho for assistance in calculating Stark shifts. This work was
supported by the ARO under Grants No. W911NF-20-1-0037 and No.
W911NF-19-1-0297 and the NSF under Grants No. PHY-2110421, No. PHY-2207985,
and No. OMA-2016245.

## References

  * Ballance  _et al._ [2016] C. J. Ballance, T. P. Harty, N. M. Linke, M. A. Sepiol, and D. M. Lucas, Phys. Rev. Lett. 117, 060504 (2016).
  * Gaebler  _et al._ [2016] J. P. Gaebler, T. R. Tan, Y. Lin, Y. Wan, R. Bowler, A. C. Keith, S. Glancy, K. Coakley, E. Knill, D. Leibfried, and D. J. Wineland, Phys. Rev. Lett. 117, 060505 (2016).
  * Bruzewicz  _et al._ [2019] C. D. Bruzewicz, J. Chiaverini, R. McConnell, and J. M. Sage, Appl. Phys. Rev. 6, 021314 (2019).
  * Christensen  _et al._ [2020] J. E. Christensen, D. Hucul, W. C. Campbell, and E. R. Hudson, npj Quantum Inf 6, 35 (2020).
  * Clark  _et al._ [2021] C. R. Clark, H. N. Tinkey, B. C. Sawyer, A. M. Meier, K. A. Burkhardt, C. M. Seck, C. M. Shappert, N. D. Guise, C. E. Volin, S. D. Fallek, H. T. Hayden, W. G. Rellergert, and K. R. Brown, Phys. Rev. Lett. 127, 130505 (2021).
  * Knill [2005] E. Knill, Nature 434, 39 (2005).
  * Plenio and Knight [1997] M. B. Plenio and P. L. Knight, Proc. R. Soc. Lond. A. 453, 2017 (1997).
  * Wineland  _et al._ [1998] D. J. Wineland, C. Monroe, W. M. Itano, D. Leibfried, B. E. King, and D. M. Meekhof, J. Res. Natl. Inst. Stand. Technol. 103, 259 (1998).
  * Wineland  _et al._ [2003] D. J. Wineland, M. Barrett, J. Britton, J. Chiaverini, B. DeMarco, W. M. Itano, B. Jelenković, C. Langer, D. Leibfried, V. Meyer, T. Rosenband, and T. Schätz, Phil. Trans. R. Soc. A 361, 1349 (2003).
  * Ozeri  _et al._ [2007] R. Ozeri, W. M. Itano, J. B. R. B. Blakestad, J. Chiaverini, J. D. Jost, C. Langer, D. Leibfried, R. Reichle, S. Seidelin, J. H. Wesenberg, and D. J. Wineland, Phys. Rev. A 75, 042329 (2007).
  * Uys  _et al._ [2010] H. Uys, M. J. Biercuk, A. P. VanDevender, C. Ospelkaus, D. Meiser, R. Ozeri, and J. J. Bollinger, Phys. Rev. Lett. 105, 200401 (2010).
  * Sawyer and Brown [2021] B. C. Sawyer and K. R. Brown, Phys. Rev. A 103, 022427 (2021).
  * Moore  _et al._ [2022] I. D. Moore, W. Campbell, E. R. Hudson, M. J. Boguslawski, D. J. Wineland, and D. T. C. Allcock, “Photon scattering errors during stimulated raman transitions in trapped-ion qubits,” (2022), arXiv:2211.00744 [physics.quant-ph] .
  * [14] See Supplemental Material at [URL will be inserted by publisher] for explicit formulas for scattering rates in terms of Einstein $A$ coefficients.
  * Allcock  _et al._ [2021] D. T. C. Allcock, W. C. Campbell, J. Chiaverini, I. L. Chuang, E. R. Hudson, I. D. Moore, A. Ransford, C. Roman, J. M. Sage, and D. J. Wineland, Appl. Phys. Lett. 119, 214002 (2021).
  * Note [1] Beam polarization and bias field alignment was determined using measurements of polarization-dependent Stark shifts on the ion. Beam polarization was left-hand circular with 99.86% degree of circular polarization at the vacuum viewport, measured with a Thorlabs PAX1000VIS polarimeter. The bias field was aligned to within an estimated 2∘ from the $\mathbf{k}$ vector axis.
  * Boggs  _et al._ [1992] P. T. Boggs, R. H. Byrd, J. E. Rogers, and R. B. Schnabel, “User’s reference guide for odrpack version 2.01: Software for weighted orthogonal distance regression,” (1992).
  * Sørensen and Mølmer [2000] A. Sørensen and K. Mølmer, Phys. Rev. A 62, 022311 (2000).
  * Note [2] The slope of the ODR of the total SRS data in Fig. 4 gives $\Gamma/\delta$ for SRS rate $\Gamma$ and differential Stark shift $\delta$. At 532 nm, $\delta/\Omega_{\mathrm{R}}$ is calculated following Eqn. 2.3 and 2.10 of [9]. Using this quantity, $\pi(\Gamma/\delta)(\delta/\Omega_{\mathrm{R}})(3\sqrt{K}/\eta)=(\Gamma/\Omega_{\mathrm{R}})(3\pi\sqrt{K}/\eta)=\epsilon_{2q}$ from Eq. 2 is calculated at 532 nm.

##  I Supplemental Materials

##  II Scattering models

Two scattering models considered in this paper (which we denote the “CDA” and
“$\omega^{3}$-theory” models) are explicitly given below. To establish a
common set of notation that can be used for ions with similar atomic
structure, we adopt the notation given in Fig. S1 and Table S.I. The Einstein
$A$ coefficients, electron angular momentum quantum numbers, and frequencies
will be denoted with subscripts accordingly (e.g.
$A_{\mathrm{e}^{\prime},\mathrm{g}^{\prime\prime}}$, $J_{\mathrm{g}}$,
$\omega_{\mathrm{e},\mathrm{g}^{\prime}}$, etc.).

###  II.1 CDA theory model

First, the constant density of states approximation (CDA) theory model, which
we base on the previous theory [10], is most valid when the laser detuning
from an intermediate state is small compared to the laser’s frequency. The
total spontaneous scattering rate (during a $g$-type, zero-field hyperfine
clock-state qubit gate) is given by

$\displaystyle\Gamma_{\mathrm{total,CDA}}\equiv$
$\displaystyle\,\,\left(A_{\mathrm{e},\mathrm{g}}+A_{\mathrm{e},\mathrm{g}^{\prime}}\right)\times\frac{g^{2}}{3}\frac{1}{\Delta^{2}}+\left(A_{\mathrm{e}^{\prime},\mathrm{g}}+A_{\mathrm{e}^{\prime},\mathrm{g}^{\prime}}+A_{\mathrm{e}^{\prime},\mathrm{g}^{\prime\prime}}\right)\times\frac{g^{2}}{3}\frac{2}{(\Delta-\omega_{\mathrm{FS}})^{2}}$
(S.1)

where $\Delta=\omega_{\ell}-E_{\mathrm{P}1/2}/\hbar$ is the detuning of the
laser from the
${}^{2}\mathrm{P}^{o}_{1/2}\leftrightarrow{}^{2}\mathrm{S}_{1/2}$ resonance.
The spontaneous Rayleigh scattering rate is

$\displaystyle\Gamma_{\mathrm{Rayleigh,CDA}}\equiv$
$\displaystyle\,\,A_{\mathrm{e}^{\prime},\mathrm{g}}\frac{g^{2}}{3}\left(\frac{3\Delta^{2}-2\Delta\omega_{\mathrm{FS}}+\omega_{\mathrm{FS}}^{2}/3}{\Delta^{2}(\Delta-\omega_{\mathrm{FS}})^{2}}\right),$
(S.2)

and the spontaneous Raman scattering rate is is difference between these two,
$\Gamma_{\mathrm{Raman,CDA}}=\Gamma_{\mathrm{total,CDA}}-\Gamma_{\mathrm{Rayleigh,CDA}}$.
The branching fraction of the spontaneous Raman scattering rate into
${}^{2}\mathrm{D}_{5/2}$ is

$\displaystyle\eta_{\mathrm{D}_{5/2}\mathrm{,CDA}}\equiv$
$\displaystyle\,\,\frac{\Gamma_{\mathrm{D}_{5/2}\mathrm{,CDA}}}{\Gamma_{\mathrm{Raman,CDA}}}=A_{\mathrm{e}^{\prime},\mathrm{g}^{\prime\prime}}\times\frac{g^{2}}{3}\left(\frac{2}{(\Delta-\omega_{\mathrm{FS}})^{2}}\right)/\Gamma_{\mathrm{Raman,CDA}}.$
(S.3)

###  II.2 $\omega^{3}$-theory model

The $\omega^{3}$-theory model closely resembles the CDA theory model near
resonance, but differs significantly far from resonance. The difference
between the two is simply the addition of the dependence of the spontaneous
scattering rate on the frequency of the spontaneously scattered photon. This
necessarily includes a step function to ensure that spontaneous scattering
events to excited final states turn off when the laser photons don’t have
enough energy to populate those states (see Fig. 1b). The total spontaneous
scattering rate in this model is given by

$\displaystyle\Gamma_{\mathrm{total,}\,\omega^{3}}=$
$\displaystyle\,\,\left(\frac{\omega_{\ell}}{\omega_{\mathrm{e},\mathrm{g}}}\right)^{3}A_{\mathrm{e},\mathrm{g}}\times\frac{g^{2}}{3}\left(\frac{1}{\Delta^{2}}\right)$
$\displaystyle+\Theta(\omega_{\ell}-\omega_{\mathrm{g},\mathrm{g}^{\prime}})\left(\frac{\omega_{\ell}-\omega_{\mathrm{g},\mathrm{g}^{\prime}}}{\omega_{\mathrm{e},\mathrm{g}^{\prime}}}\right)^{3}A_{\mathrm{e},\mathrm{g}^{\prime}}\times\frac{g^{2}}{3}\left(\frac{1}{\Delta^{2}}\right)$
$\displaystyle+\left(\frac{\omega_{\ell}}{\omega_{\mathrm{e}^{\prime},\mathrm{g}}}\right)^{3}A_{\mathrm{e}^{\prime},\mathrm{g}}\times\frac{g^{2}}{3}\left(\frac{2}{(\Delta-\omega_{\mathrm{FS}})^{2}}\right)$
$\displaystyle+\Theta(\omega_{\ell}-\omega_{\mathrm{g},\mathrm{g}^{\prime}})\left(\frac{\omega_{\ell}-\omega_{\mathrm{g},\mathrm{g}^{\prime}}}{\omega_{\mathrm{e}^{\prime},\mathrm{g}^{\prime}}}\right)^{3}A_{\mathrm{e}^{\prime},\mathrm{g}^{\prime}}\times\frac{g^{2}}{3}\left(\frac{2}{(\Delta-\omega_{\mathrm{FS}})^{2}}\right)$
$\displaystyle+\Theta(\omega_{\ell}-\omega_{\mathrm{g},\mathrm{g}^{\prime\prime}})\left(\frac{\omega_{\ell}-\omega_{\mathrm{g},\mathrm{g}^{\prime\prime}}}{\omega_{\mathrm{e}^{\prime},\mathrm{g}^{\prime\prime}}}\right)^{3}A_{\mathrm{e}^{\prime},\mathrm{g}^{\prime\prime}}\times\frac{g^{2}}{3}\left(\frac{2}{(\Delta-\omega_{\mathrm{FS}})^{2}}\right).$
(S.4)

The spontaneous Rayleigh scattering rate is similar to the CDA theory,

$\displaystyle\Gamma_{\mathrm{Rayleigh,}\,\omega^{3}}=$
$\displaystyle\,\,\left(\frac{\omega_{\ell}}{\omega_{\mathrm{e}^{\prime},\mathrm{g}}}\right)^{3}A_{\mathrm{e}^{\prime},\mathrm{g}}\,\frac{g^{2}}{3}\left(\frac{3\Delta^{2}-2\Delta\omega_{\mathrm{FS}}+\omega_{\mathrm{FS}}^{2}/3}{\Delta^{2}(\Delta-\omega_{\mathrm{FS}})^{2}}\right)$
(S.5)

and the spontaneous Raman scattering rate is again given by the difference
between the total and Rayleigh rates,

$\displaystyle\Gamma_{\mathrm{Raman,}\,\omega^{3}}=$
$\displaystyle\,\,\Gamma_{\mathrm{total,}\,\omega^{3}}-\Gamma_{\mathrm{Rayleigh,}\,\omega^{3}}$
$\displaystyle=$
$\displaystyle\,\,\left(\frac{\omega_{\ell}}{\omega_{\mathrm{e}^{\prime},\mathrm{g}}}\right)^{3}A_{\mathrm{e}^{\prime},\mathrm{g}}\,\frac{2g^{2}}{9}\left(\frac{\omega_{\mathrm{FS}}}{\Delta(\Delta-\omega_{\mathrm{FS}})}\right)^{2}$
$\displaystyle+\Theta(\omega_{\ell}-\omega_{\mathrm{g},\mathrm{g}^{\prime}})\left(\frac{\omega_{\ell}-\omega_{\mathrm{g},\mathrm{g}^{\prime}}}{\omega_{\mathrm{e},\mathrm{g}^{\prime}}}\right)^{3}A_{\mathrm{e},\mathrm{g}^{\prime}}\times\frac{g^{2}}{3}\left(\frac{1}{\Delta^{2}}\right)$
$\displaystyle+\Theta(\omega_{\ell}-\omega_{\mathrm{g},\mathrm{g}^{\prime}})\left(\frac{\omega_{\ell}-\omega_{\mathrm{g},\mathrm{g}^{\prime}}}{\omega_{\mathrm{e}^{\prime},\mathrm{g}^{\prime}}}\right)^{3}A_{\mathrm{e}^{\prime},\mathrm{g}^{\prime}}\times\frac{g^{2}}{3}\left(\frac{2}{(\Delta-\omega_{\mathrm{FS}})^{2}}\right)$
$\displaystyle+\Theta(\omega_{\ell}-\omega_{\mathrm{g},\mathrm{g}^{\prime\prime}})\left(\frac{\omega_{\ell}-\omega_{\mathrm{g},\mathrm{g}^{\prime\prime}}}{\omega_{\mathrm{e}^{\prime},\mathrm{g}^{\prime\prime}}}\right)^{3}A_{\mathrm{e}^{\prime},\mathrm{g}^{\prime\prime}}\times\frac{g^{2}}{3}\left(\frac{2}{(\Delta-\omega_{\mathrm{FS}})^{2}}\right)$
(S.6)

where we have used
$A_{\mathrm{e},\mathrm{g}}/\omega_{\mathrm{e},\mathrm{g}}^{3}=A_{\mathrm{e},\mathrm{g}^{\prime}}/\omega_{\mathrm{e},\mathrm{g}^{\prime}}^{3}$.
The branching fraction of the spontaneous Raman scattering to the
${}^{2}\mathrm{D}_{5/2}$ manifold is

$\displaystyle\eta_{\mathrm{D}_{5/2}\mathrm{,}\,\omega^{3}}=$
$\displaystyle\,\,\Theta(\omega_{\ell}-\omega_{\mathrm{g},\mathrm{g}^{\prime\prime}})\left(\frac{\omega_{\ell}-\omega_{\mathrm{g},\mathrm{g}^{\prime\prime}}}{\omega_{\mathrm{e}^{\prime},\mathrm{g}^{\prime\prime}}}\right)^{3}A_{\mathrm{e}^{\prime},\mathrm{g}^{\prime\prime}}\times\frac{g^{2}}{3}\left(\frac{2}{(\Delta-\omega_{\mathrm{FS}})^{2}}\right)/\Gamma_{\mathrm{Raman,}\,\omega^{3}}.$
(S.7)

State | Shorthand | $J$ | Energy ($/\hbar$)
---|---|---|---
${}^{2}\mathrm{S}_{1/2}$ | $\mathrm{g}$ | $J_{\mathrm{g}}=\textstyle{\frac{1}{2}}\displaystyle$ | $\equiv 0$
${}^{2}\mathrm{P}_{1/2}^{o}$ | $\mathrm{e}$ | $J_{\mathrm{e}}=\textstyle{\frac{1}{2}}\displaystyle$ | $\omega_{\mathrm{e},\mathrm{g}}$
${}^{2}\mathrm{P}_{3/2}^{o}$ | $\mathrm{e}^{\prime}$ | $J_{\mathrm{e}^{\prime}}=\textstyle{\frac{3}{2}}\displaystyle$ | $\omega_{\mathrm{e}^{\prime},\mathrm{g}}$
${}^{2}\mathrm{D}_{3/2}$ | $\mathrm{g}^{\prime}$ | $J_{\mathrm{g}^{\prime}}=\textstyle{\frac{3}{2}}\displaystyle$ | $\omega_{\mathrm{g}^{\prime},\mathrm{g}}$
${}^{2}\mathrm{D}_{5/2}$ | $\mathrm{g}^{\prime\prime}$ | $J_{\mathrm{g}^{\prime\prime}}=\textstyle{\frac{5}{2}}\displaystyle$ | $\omega_{\mathrm{g}^{\prime\prime},\mathrm{g}}$
Table S.I: Shorthand notation for the five manifolds under consideration.
Figure S1: Level diagram relevant for driving a stimulated Raman transition
between zero-field $g$-type clock states in a half-integer-spin isotope of an
alkaline earth ion with pure $\sigma^{+}$ light. For isotopes with
$I<\frac{5}{2}$, the allowed values of $F$ are constrained to $F\geq 0$.

The $\omega^{3}$-theory model, while being a fairly clear improvement over the
CDA model in the regime explored experimentally in this paper, neglects a
number of effects that should be considered before applying this analysis to
other regimes. As discussed in Ref. [13], there are terms arising from the
counter-rotating part of the electric field (i.e. emission-first processes)
that can play a role, as well as coupling to more excited states than just the
five manifolds considered here (Fig. S1).


# Incipient Fault Detection in Power Distribution System: A Time-Frequency
Embedded Deep Learning Based Approach

Qiyue Li, , Huan Luo, Hong Cheng, Yuxing Deng, Wei Sun, , Weitao Li, Zhi Liu
Qiyue Li, Huan Luo, Hong Cheng, Yuxing Deng, Wei Sun, and Weitao Li are with
the school of Electrical Engineering and Automation, Hefei Unviersity of
Techonology, Hefei, China, (e-mail<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>wtli@hfut.edu.cn).Zhi Liu is with Department of Computer and
Network Engineering, The University of Electro-Communications, Japan (e-mail:
liuzhi@uec.ac.jp).Corresponding author: Zhi Liu.

###### Abstract

Incipient fault detection in power distribution systems is crucial to improve
the reliability of the grid. However, the non-stationary nature and the
inadequacy of the training dataset due to the self-recovery of the incipient
fault signal, make the incipient fault detection in power distribution systems
a great challenge. In this paper, we focus on incipient fault detection in
power distribution systems and address the above challenges. In particular, we
propose an ADaptive Time-Frequency Memory (AD-TFM) cell by embedding wavelet
transform into the Long Short-Term Memory (LSTM), to extract features in time
and frequency domain from the non-stationary incipient fault signals. We make
scale parameters and translation parameters of wavelet transform learnable to
adapt to the dynamic input signals. Based on the stacked AD-TFM cells, we
design a recurrent neural network with ATtention mechanism, named AD-TFM-AT
model, to detect incipient fault with multi-resolution and multi-dimension
analysis. In addition, we propose two data augmentation methods, namely phase
switching and temporal sliding, to effectively enlarge the training datasets.
Experimental results on two open datasets show that our proposed AD-TFM-AT
model and data augmentation methods achieve state-of-the-art (SOTA)
performance of incipient fault detection in power distribution system. We also
disclose one used dataset logged at State Grid Corporation of China to
facilitate future research.

###### Index Terms:

power distribution system, incipient fault detection, wavelet transform,
recurrent neural network, attention mechanism, LSTM, data augmentation

##  I Introduction

Power distribution system delivers electricity from the transmission system to
the individual consumers and is an inseparable part of people’s lives and
society. Real-time fault detection system plays an important role in
maintaining the stability of power equipment [1, 2, 3]. In particular, some
pre-existing anomalies occur before a fault occurs in the power distribution
system, which is called incipient faults [4].

The incipient fault may occur at any time or in any place of the distribution
network. Containing a large number of non-fundamental transient signals, the
voltage and current time series data shows strong randomness and non-
stationary characteristics when incipient fault occurs [5]. In addition, as
the incipient fault of the distribution network is self-recovering and can be
self-concealed, only a small amount of data can be logged by traditional fault
recorder, which makes incipient faults detection a huge challenge[6, 7, 8].

Detection of incipient fault allows maintenance personnel to replace defective
equipment in advance, effectively improving power supply reliability. In
addition, it is a kind of predictive maintenance, where the detection of
failures at an early stage helps to avoid unexpected disruptions. There are
two mainstream methodologies of fault detection in power distribution system.
The first one is traditional fault classification method, in which manually
selected features are extracted from the filtered current and voltage time
series signal, and then matched with pre-set feature thresholds or patterns to
detect the corresponding fault types [9, 10, 11, 12, 13]. For example, in
[14], a method based on human-level concept learning is proposed by selecting
waveform features of current and voltage and decomposing it into primitives to
detect faults. In [15], an online model based on sequential Bayesian approach
is proposed by splitting power quality abnormalities of continuous current.
This kind of methods are easy to implement, however, human selected features
and thresholds rely heavily on expertise knowledge, and are not sufficiently
capable of characterizing complex non-stationary signals to be well applied to
incipient fault detection in distribution system.

With the help of artificial intelligence (AI), data driven methods are also
applied to incipient fault detection in power distribution system [16, 17, 18,
19]. Due to the complex causes and electrical characteristics of incipient
faults in power distribution system, it is difficult to establish a
comprehensive mathematical model. On the other hand, data driven methods are
more effective to deal with incipient fault diagnosis in the power
distribution network, and can detect some unknown faults. For example, Long
Short-Term Memory (LSTM) utilizes memory units instead of hidden layers in
traditional Recurrent Neural Network (RNN), which constructs a more powerful
model over time series using contextual information, and shows good
performance on time series estimation. In particular, regarding the voltage
and current time series data, LSTM cell is utilized to build a deep RNN
architecture to automatically extract features and perform fault detection
[20]. In [21], a LSTM based network is proposed by learning low-resolution
data from a real case study to detect incipient faults.

However, due to lack of the frequency domain analysis, LSTM based schemes
cannot fully extract features of time series data, especially for the non-
stationary incipient fault signal [22]. Besides, incipient faults in power
distribution system are usually of short duration and self-recovering, which
leads to the unavailability of sufficient samples to train the LSTM network.
And thus exacerbates the difficulty of data-driven incipient fault detection
methods.

In this paper, to improve the feature extraction ability from random and non-
stationary signal of incipient faults, we propose an ADaptive Time-Frequency
Memory (AD-TFM) cell which embeds adaptive wavelet transform into LSTM.
Specifically, we first use wavelet transform to decompose different frequency
signals existing in a non-stationary signal into non-overlapping frequency
bands. At each time step, the learned wavelet transform coefficients are
multiplied by the input signal to obtain the time and frequency domain
features. The coefficients are stored in AD-TFM and propagated to next time
step to improve feature extraction abilities. In addition, we make the scale
parameters and translation parameters of wavelet transform learnable to
automatically adapt to the input signal, which can achieve multi-resolution
and multi-dimensional analysis of non-stationary incipient fault signals.

Then we construct an AD-TFM cell based RNN model to perform incipient fault
detection in power distribution system. To focus the neural network on global
hidden information, we strengthen the stacked AD-TFM network by adding an
ATtention layer, i.e., a new AD-TFM-AT model is implemented. The correlation
of the hidden state output at all time steps of AD-TFM cells are calculated in
the attention layer. Then, the correlation degrees are used to make a weighted
average of hidden states of all time steps. By improving the attention of
neural network to the time steps containing fault information and increasing
the feature extraction ability of hidden information for all time steps, the
fault detection accuracy is further improved. To enlarge the training sample
of incipient fault data and improve detection performance of AD-TFM-AT, we
propose two data augmentation methods, namely phase switching and temporal
sliding. Based on the available small incipient fault dataset[23] and a
relatively large dataset logged in State Grid Corporation of China, our
proposed method achieves state-of-the-art (SOTA) performance.

The main contributions of this paper are as follows:

  1. 1.

We propose an AD-TFM cell based on adaptive wavelet transform, which performs
feature extraction at different scales to effectively deal with the non-
stationary incipient fault signals of power distribution system. We design AD-
TFM-AT, an attention assisted AD-TFM based RNN model, which increases the
weight of the most relevant hidden states in fault detection and guides the
feature fusion process.

  2. 2.

We propose two effective data augmentation methods, i.e., phase switching and
temporal sliding, which swap the phases of the voltage and current data of the
fault signal with the remaining phases and intercepts each fault with
different starting points, respectively. These methods effectively expand the
small fault data set in the power distribution system and improve the training
performance.

  3. 3.

We conduct extensive experiments and experimental results on two open
datasets, annd the results show that our proposed AD-TFM-AT model and data
augmentation methods achieve SOTA performance of incipient fault detection in
power distribution system. We also disclose a relatively large dataset logged
at State Grid Corporation of China to facilitate future
research111https://github.com/smartlab-hfut/SGAH-datasets.

The rest of the paper is organized as follows.Section II presents the latest
research related to incipient fault detection. Section III explores the random
and non-stationary characteristic of incipient fault in distribution network
based on a simplified circuit model. Section IV introduces our proposed AD-TFM
cell based on adaptive wavelet transform and LSTM. Section V shows the
hierarchical structure diagram of the AD-TFM-AT, and two methods of data
augmentation are explained in Section VI. In Section VII, we show the
performance of data augmentation and incipient fault detection accuracy
against two datasets. Finally, Section VIII concludes this manuscript.

Note that this paper is an extended version of our previous conference paper
[24]. Different from [24],

this paper analyzes the non-stationary of fault signals based on a simplified
distribution network circuit model. In addition, an adaptive wavelet transform
with learnable wavelet scale parameters and translation parameters is
proposed, which realizes the multi-resolution analysis of fault signals. We
also added an attention mechanism to the neural network to enhance the focus
on time step hidden states that are highly correlated with fault
classification. This further improves the network’s ability to detect
incipient faults.

##  II Related Work

This section explains the latest research related to incipient fault
detection.

###  II-A Faults in power distribution systems

The incipient faults are transient events that occur at random locations and
are pre-emptive hidden faults before permanent faults occur. Incipient faults
in the power distribution network usually occur in underground cables[25, 26,
5], transformer equipment[27, 28], distribution networks with high Distributed
Energy Resources (DERS) penetration[29], and so on. There are many reasons for
incipient faults, such as tree interference, animal contact and coil contact
[14]. When an incipient fault occurs, the fault phases voltage and current
change, waveform is distorted, and the fault transient signal exhibits non-
stationary characteristics. Meanwhile, incipient faults are typically self-
clearing faults and have a short duration, ranging from a quarter of a cycle
(sub-cycle), to up to four cycles (multi-cycle) [14]. Thus, incipient faults
are less well documented, and detection methods for incipient faults are much
needed.

###  II-B Traditional fault detection methods

Traditional fault detection methods, which mainly include similarity
detection[30, 31], waveform eigenvalue decomposition[32, 33, 34], and model
parameter estimation[35, 36, 37], have been used for fault detection in power
distribution system. For example, in[30], an expression for the transient zero
sequence current characteristics under the influence of the inverter is
derived, and a method based on the first-order accumulated generation operator
(AGO) and the improved cosine similarity is proposed to identify the faulty
feeders. In[32], the wavelet singular value decomposition is applied to obtain
the edge components of the normalized fault current amplitude for fault
detection. In[37], the transformer state space model, linear parameter varying
(LPV) observer, primary and secondary voltage values are used to estimate the
primary current at each time step of the transformer. The estimated primary
currents are compared with the actual primary currents to distinguish whether
the transformer is internally or externally faulty.

The above methods rely on the manual extraction of features and then achieve
fault identification based on rules or thresholds set by manual experience. In
distribution networks with different parameters, the threshold values set by
various methods vary greatly. Moreover, these methods lack the analysis of the
non-stationarity of faults, which limits the application and effectiveness of
traditional fault detection methods.

###  II-C AI based fault detection methods

With the wide application of AI, methods based on machine learning and deep
learning are applied to fault detection. From the computer vision point of
view, there are schemes using Convolutional Neural Networks (CNN)[38, 39].
Recently, hybrid approaches for fault recognition have also been proposed [40,
41]. For example, in [42], zero-sequence currents are transformed into
spectrograms by short time Fourier Transform, and then a two-channel CNN is
constructed to achieve fault classification. In [43], a Multi-layer Long
Short-Term Memory Network (MLSTM) is applied to voltage waveform analysis in
order to detect whether a fault occurs in the grid. In [40], a hybrid
statistical learning and machine learning approach is proposed to identify
fault-inducing regions in photovoltaic (PV) farm based on micro Phasor
Measurement Unit (PMU) measurement data. However, the above AI-based methods
lack the analysis of fault signal features, and do not fully take into account
the non-stationary nature of incipient faults in the power distribution
network.

###  II-D Solution with insufficient training data

Fault data scarcity is also an important issue faced by fault detection using
deep learning methods, and several methods are proposed to cope with this
problem [44, 45]. In particular, in [44], various pre-trained models are fine-
tuned on different substations through migration learning and federated
learning. However,[44] requires a large number of deployable substation
resources and edge-cloud communications, which will incur a large cost
overhead. In [45], the fault current data are decomposed into multilayer
wavelet coefficients which are fused into a matrix. Then the matrix is mapped
into a phase space image with three channels (RGB) by colormap indexes as the
input of the classification model. Among them, two data enhancement methods
are proposed. The first one is to change the colormap indexes randomly to
obtain phase space images with different color domains. The second one is to
convert the phase space images from RGB color mode to HSV (Hue, Saturation,
Value) mode. The Hue channel value is changed to generate different images.
Then the different images are changed back to RGB mode to achieve data
enhancement. In [45], only the color mapping index and Hue of the phase space
images of the coefficient matrix are changed to obtain a different color graph
of the same fault data, and no new fault information is actually generated. In
summary, the existing methods mainly address the problem of insufficient
incipient fault data in distribution networks through migration learning and
fault image data enhancement. These methods perform data enhancement in terms
of increasing the fault data acquisition surface or changing the fault data
mapping. And they do not actively generate new fault information nor consider
fault characteristics.

##  III Incipient faults and their features

The faults in power distribution system, can be divided into sub-cycle faults,
multi-cycle faults and permanent fault, according to their durations. Among
them, sub-cycle and multi-cycle faults are called incipient faults[25]. Sub-
cycle incipient faults are characterized by abnormal fault phase voltage and
recovery in one cycle, while multi-cycle incipient faults mainly include
interphase short circuit faults, permanent faults include grounding with high
resistance, single-phase grounding faults and main transform faults. Several
typical fault waveforms are shown in the Fig. 1.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 1: Typical incipient fault signals. (a) Sub-cycle incipient fault. (b)
Multi-cycle incipient fault. (c) High resistance grounding fault. (d) Single-
phase grounding fault. (e) Two-phase grounding fault. (f) Interphase short
circuit fault.

Taking the single-phase grounding fault occurring in overhead lines of power
distribution network as an example, the simplified circuit model contains two
inductors and one capacitor, as illustrated in Fig. 2. Due to the presence of
inductors, the current in the line can not be changed suddenly, which will
cause a short circuit transient process, and there are a large number of
integer and non-integer harmonics in the voltage and current signal. As the
characteristic frequency components in the transient process are not fixed,
the current signal flowing through the capacitor contains fault information,
and is non-stationary.

(a)  (b)

Figure 2: Overhead line in power distribution network and its simplified
circuit model. (a) Overhead line. (b) Simplified circuit model.

###  III-A Transient capacitive current

Based on Fig. 2b, the differential equation of transient capacitance current
can be expressed as:

$\displaystyle
R_{0}i_{C}+L_{0}\frac{di_{C}}{dt}+\frac{1}{C}\int_{0}^{t}i_{C}=U_{m}\sin(\omega
t+\varphi),$ (1)

where $U_{m}$ is the amplitude of the zero sequence voltage. The transient
capacitive current $i_{C}$ is composed of transient free oscillation component
$i_{C.os}$ and steady-state power frequency component $i_{C.st}$. When single-
phase grounding fault occurs, $i_{C.os}+i_{C.st}=0$ and $i_{Cm}=U_{m}\omega
C$. And the transient capacitive current can be calculated as:

$\displaystyle i_{C}$ $\displaystyle=i_{C.os}+i_{C.st}$ (2)
$\displaystyle=I_{Cm}[\left(\frac{\omega_{f}}{\omega}\sin\varphi\sin\omega_{f}t-\cos\omega\cos\omega_{f}t\right)e^{-t\delta}+$
$\displaystyle\cos\omega_{f}t],$

where $I_{Cm}$ is the amplitude of the transient capacitor current.
$\omega_{f}$ is the angular frequency of the transient free oscillation
component. $\delta={1/\tau C}={R/2L_{0}}$ is the attenuation coefficient of
the free oscillation component. $\varphi$ is the phase angle of the phase
voltage when the fault occurs.

If $R_{0}$ is less than $2\sqrt{L_{0}/C}$, the transient process of the loop
current has periodic oscillation and attenuation characteristics. Otherwise,
the loop current has aperiodic oscillation attenuation characteristics, and
gradually tends to be in a stable state.

###  III-B Transient inductive current

The inductive current of arc suppression coil is composed of transient DC
component and steady-state AC component, which is expressed as:

$\displaystyle i_{L}=I_{Lm}[\cos\varphi^{-t/\tau_{L}}-\cos(\omega
t+\varphi)],$ (3)

where $\tau_{L}$ is the time constant of the inductance circuit.
$I_{Lm}=\frac{U_{m}}{\omega_{L}}$, $\phi$ is the phase angle of phase voltage
at fault.

The fault signal contains a large number of non-fundamental transient signals,
which consisting of high frequency components, non-periodic components and a
large number of fault or disturbance information. The transient component in
fault signal is a non-stationary random process, which changes with time, the
location of the fault point, the transition resistance of the fault point and
the different operating conditions of the system. Through the above analysis,
when a single-phase grounding fault occurs in the power grid, it can be seen
from the analysis of the transient process that the fault signal is non-
stationary at that time.

##  IV Time-Frequency Memory Cell Based on Adaptive Wavelet

To extract the dynamic characteristics of fault parameters in power
distribution networks, we introduce wavelet transform that can accurately
analyze non-stationary signals into the LSTM cell, and change the forget gate
of LSTM into the joint forget gate, which decomposes fault information in both
time and frequency domain. Besides we establish an adaptive learning mechanism
for scale parameters and position parameters in wavelet transform, and propose
AD-TFM cell that can accurately model the non-stationary incipient fault
signal.

###  IV-A Basic Idea of AD-TFM

The traditional method, which combines the wavelet transform and the neural
network for fault detection, usually uses the wavelet transform to extract the
fault features, which are then fed into neural network for classification[20].
In this method, the wavelet transform is separated from the neural network,
and the error generated during feature extraction has a greater impact on the
accuracy of fault classification in the later stage. To solve the above-
mentioned problem, we propose AD-TFM by embedding the wavelet transform into
traditional LSTM cell, changing the originally fixed scale parameter and
translation parameter to dynamic parameter that changes with the input fault
information.

###  IV-B Structure of AD-TFM Cell

The structure of AD-TFM cell is shown in Fig. 3, which consists of joint
forget gate, input gate, output gate and cell state updating. It dynamically
models the input, i.e., three-phase current and voltage time series
$\left\\{x_{t}\mid t=1,2,...,T\right\\}$ by continuous time steps. In each
time step of AD-TFM, the hidden state of the previous time step and the input
information of the current time step are decided by the joint forget gate, and
the input gate selects the information to be updated. In the cell state
updating part, the input information after adaptive wavelet transform and the
information retained by the joint forget gate are added to update the cell
state, and then the updated cell state is input to the output gate to obtain
the hidden state at the current time.

In this process, the non-stationary analysis is achieved by converting the
input three-phase voltage and current data into time-frequency features using
efficient time modeling (via LSTM) and non-stationary signal processing (i.e.,
wavelet transform).

Figure 3: Structure of AD-TFM cell.

###  IV-C The Joint Forget Gate

The joint forget gate contains three parts: the state forget gate
$f_{t}^{ste}$, the time forget gate $f_{t}^{tim}$ and the frequency forget
gate $f_{t}^{fre}$, which decompose the input and the hidden state of the
previous time step into the $K$ dimension in the time domain, $J$ dimension in
the frequency domain and $D$ dimension in state domain, respectively.

$\displaystyle
f_{t}^{ste}=sigmoid\left(W_{ste}x_{t}+U_{ste}h_{t-1}+b_{ste}\right)\in\mathbb{R}^{D},$
(4)

$\displaystyle
f_{t}^{tim}=sigmoid\left(W_{tim}x_{t}+U_{tim}h_{t-1}+b_{tim}\right)\in\mathbb{R}^{K},$
(5)

$\displaystyle
f_{t}^{fre}=sigmoid\left(W_{fre}x_{t}+U_{fre}h_{t-1}+b_{fre}\right)\in\mathbb{R}^{J},$
(6)

where $W_{*}$ and $U_{*}$ are weight matrices. $b_{*}$ is a bias vector and
$h_{t-1}$ is the output hidden state at the ($t-1$)th time step. Among them
${*}$ refers to ${ste}$, ${tim}$, ${fre}$.

The output of three forget gates are used to obtain $F^{t}$, by jointly
regulating the state, time and frequency information.

$\displaystyle F_{t}=f_{t}^{ste}\otimes f_{t}^{tim}\otimes
f_{t}^{fre}\in\mathbb{R}^{D\times J\times K},$ (7)

$\displaystyle FC_{t}=F_{t}\circ C_{t-1}\in\mathbb{R}^{D\times J\times K},$
(8)

where $\otimes$ is the outer product operation and $\circ$ is the element-wise
multiplication operation.

The joint forget gate determines the amount of information retained from the
previous time step to the current step. It can be considered as a combination
gate, which controls the information of different frequencies, times and
states flowing into the memory cell.

###  IV-D Input Gate

The formulations of the input gate $i_{t}$ and the input modulation $g_{t}$
are similar as these of LSTM:

$\displaystyle i_{t}=sigmoid\left(W_{i}x_{t}+U_{i}h_{t-1}+b_{i}\right),$ (9)

$\displaystyle g_{t}=tanh\left(W_{g}x_{t}+U_{g}h_{t-1}+b_{g}\right),$ (10)

$\displaystyle ig_{t}=i_{t}\circ g_{t},$ (11)

where the $ig_{t}$ is defined to generate a compatible result for the input
gate.

The input gate decides how much new information should be allowed to enter the
current memory cell to update AD-TFM.

###  IV-E Cell state updating based on adaptive wavelet transform

The state updating procedure of AD-TFM is similar to LSTM. By integrating the
adaptive wavelet transform, the output of the input gate needs to be
multiplied by the coefficients of the adaptive wavelet transform when the AD-
TFM cell is updated. The output of the input gate is decomposed by the wavelet
transform into $K$ and $J$ dimensions in the time domain and frequency domain
respectively.

Taking the Morlet wavelet transform used in this paper as an example, the
implementation function of the adaptive learning of the scale parameters $a$
and translation parameters $b$ are:

$\displaystyle a=tanh\left(W_{a}ig_{t}+b_{a}\right),$ (12) $\displaystyle
b=tanh\left(W_{b}ig_{t}+b_{b}\right).$ (13)

The output of the input gate, which is decomposed by the wavelet transform, is
expressed as follows.

$\displaystyle\psi_{k,j}$
$\displaystyle=exp\left(i\cdot\frac{\omega_{0}}{a}\cdot\left(\frac{t+b}{2^{j}}-k\right)\right)\cdot$
(14) $\displaystyle
exp\left(\left(-\frac{1}{a}\right)\cdot\left(\frac{t+b}{2^{j}}-k\right)^{2}\right).$

Then the cell state after decomposition can be obtained as:

$\displaystyle C_{t}=FC_{t}+ig_{t}\otimes\psi_{k,j}\in\mathbb{R}^{D\times
J\times K},$ (15)

$C_{t-1}$ is the cell state value at the previous time step.
$FC_{t}\in\mathbb{R}^{D\times J\times K}$ and $i_{t}\in\mathbb{R}^{D}$ are
forget and input gates, respectively, controlling the past and current
information on states, time and frequencies that are allowed to update the AD-
TFM at the $t$th time step.

As a complex number can be uniquely represented by its amplitude and phase, we
decompose the update matrix $C_{t}$ of AD-TFM into two parts, amplitude and
phase, which are expressed as:

$\displaystyle A_{t}=\mid
C_{t}\mid=\sqrt{\left(ReC_{t}\right)^{2}+\left(ImC_{t}\right)^{2}}\in\mathbb{R}^{D\times
J\times K},$ (16)

$\displaystyle\angle
C_{t}=arctan\left(\frac{ReC_{t}}{ImC_{t}}\right)\in\left[-\frac{\pi}{2},\frac{\pi}{2}\right].$
(17)

where $Re$ and $Im$ are the functions of taking the real part and taking the
imaginary part, respectively. $arctan\left(\cdot\right)$ is an element-wise
inverse tangent function.

The amplitude will be fed into the memory cell for the next time step.
However, this phase will be ignored because it has no impact on performance
except for higher computation and memory overhead.

At each time step, we calculate the component $A_{t}^{k,j}$ of the amplitude
$A_{t}$ in the $k$th dimensional time domain and the $j$th dimensional
frequency domain. $A_{t}^{k,j}$ will be sent to next time step state cell unit
of AD-TFM, and the forget gate and input gate determine the information that
needs to be updated. After the update, $A_{t}^{k,j}$ is combined into
$\widetilde{c_{t}}$, and enters the output gate, which is expressed as:

$\displaystyle\widetilde{c_{t}}=\sum\nolimits_{k=1}^{K}\sum\nolimits_{j=1}^{J}\left(W_{c}^{k,j}A_{t}^{k,j}+b_{c}^{k,j}\right),$
(18)

###  IV-F Output gate

The output gate determines the information that will be fed into next time
step. The input of the output gate can be expressed as:

$\displaystyle o_{t}=sigmoid\left(W_{o}x_{t}+U_{o}h_{t-1}+b_{o}\right),$ (19)

And the output hidden state $h_{t}$ is computed as:

$\displaystyle h_{t}=o_{t}\circ tanh\left(\widetilde{c_{t}}\right).$ (20)

##  V AD-TFM Based RNN with Attention for Incipient Fault Detection

In this section, we construct a RNN model for incipient fault detection based
on the proposed AD-TFM cell. To focus the neural network on the global hidden
information, we strengthen the stacked AD-TFM network by adding an attention
layer. The hierarchical structure of AD-TFM-AT model is shown in Fig. 4.

Figure 4: AD-TFM-AT model for incipient fault detection.

The fault signal consisting of three phase voltage and current input to AD-TFM
cell will be encoded into a fixed length of hidden information. At each time
step, the hidden state output contains a different amount of fault
information. Direct use of the hidden state of last time step of AD-TFM will
lead to insufficient attention to global hidden information. Therefore, the
amount of fault information contained in the hidden information of each time
step of AD-TFM cell is quantified in the form of similarity through the
attention mechanism, and the final output is calculated by weighting the
matching degree of hidden state of each time step as a weight. Therefore, we
use the attention mechanism to extract the important information from the
hidden states of all time steps and give it larger weights to obtain more
accurate fault feature vectors and improve the fault detection accuracy. Its
specific implementation process is as follows:

Let ${h_{i}}$ represent the hidden layer vector containing the time series
produced by AD-TFM. We convert ${h_{i}}$ to ${u_{i}}$ through a fully
connected layer illustrated as:

$\displaystyle u_{i}=tanh\left(Wh_{i}+b_{o}\right).$ (21)

Then we calculate the similarity between $u_{i}$ and the context vector
$u_{w}$, and convert it to a probability distribution $\alpha_{i}$ through
softmax function.

$\displaystyle\alpha_{i}=\frac{exp(u_{i}^{T}u_{w})}{\Sigma_{i}exp(u_{i}^{T}u_{w})}.$
(22)

The context information $u_{w}$ can be regarded as the contribution of one
time step data to the overall data, and the contribution of each $u_{i}$ to
$u_{w}$ can be obtained by calculating the similarity between $u_{i}$ and
$u_{w}$, where $u_{w}$ is randomly initialized and obtained through training.

As $\alpha_{i}$ represents the importance of the fault hidden state at each
time step to the overall fault hidden state, we use $\alpha_{i}$ as the
weighted summation of the global $h_{i}$ to obtain the tensors and express the
fault type.

$\displaystyle s=\Sigma\alpha_{i}h_{i}.$ (23)

##  VI Data augmentation

###  VI-A Overview

The incipient faults of the power distribution systems are manifested as
waveform distortion of the three-phase voltage and current sinusoidal signals
at the moment of fault occurrence. We intercept the three-phase voltage and
current data before and after the moment of fault occurrence as fault data. In
Section III, we have introduced the types of incipient fault signals in the
power distribution system and showed the waveform of the faults.

Orthogonally, training a neural network usually requires a large amount of
data. However, low incidence of incipient fault makes it a typical small
sample learning problem [14]. According to the characteristics of voltage and
current sinusoidal signal, we use two methods for data augmentation, i.e.,
phase switching and temporal sliding, to obtain a larger training dataset
while keeping the characteristics of fault data unchanged.

###  VI-B Phase Switching

The single-phase grounding fault is one major incipient fault in power
distribution systems, where the faults happens in one phase of the three-phase
voltage and current data. The first data augmentation method we use is phase
switching, which swaps the phase of the voltage and current data of the fault
signal with one of the rest phases. This changes the phase of the fault but
keeps the fault type unchanged, i.e., achieve multiple data from one fault
data.

Taking a single-phase grounding fault as an example, we assume that the fault
occurs in phase A, i.e., phase A voltage and current data contains fault
information, and phase B and C voltage and current data are normal. Then, we
swap the voltage and current data of the fault occurring in phase A with the
normal data of phase B. In this way, the fault occurring in phase A becomes
the fault occurring in phase B, and new data containing fault can then be
obtained. Meanwhile, this operation does not change the characteristics of the
single-phase grounding fault, e.g., the fault does not happen in two or more
phases at the same time. We also exchange the voltage and current data of the
fault occurring in phase A with the normal data of phase C.

###  VI-C Temporal Sliding

Temporal sliding is also used to enlarge the amount of fault data, by sampling
the original data containing the fault data multiple times with different
starting time. The starting time here is selected with equal time sliding
intervals. Comparing with original data (i.e., the case of only one starting
time), the amount of fault data is increased while the characteristics of
fault data is unchanged.

In particular, we select a window of a certain length H to intercept the fault
data, and the window can pick different starting points when sampling the
fault data. In order to enlarge the amount of fault data, we specify that the
sampling window intercepts the data from one starting time point and then
slides backward $T$ time points to intercept the data again. In this way, a
fault can be intercepted multiple times and the amount of fault data is
increased, thus achieving data augmentation. Fig. 5 illustrates the temporal
sliding. With sliding windows at different starting times, one fault data will
be sampled multiple times within different sliding windows. As a result, the
amount of fault data increases, but the type of fault is not changed.

Figure 5: Illustration of temporal sliding.

##  VII Experiments

To verify the performance of our proposed AD-TFM-AT neural network model,
extensive experiments are conducted on two datasets. We use several evaluation
metrics to assess the performance with and without data enhancement. We also
perform ablation experiments to show the performance of adaptive wavelet
transform and attention mechanism.

###  VII-A Experimental Setup

Dataset and Analysis: To train and test the proposed model, we use two
datasets, a small Incipient Fault dataset in Power Distribution (IFPD) system
from [14], and a relatively large dataset logged by State Grid Corporation of
China in AnHui Province (SGAH). In IFPD dataset, there are the Sub-cycle
Incipient Fault (SIF), Multi-cycle Incipient Fault (MIF), Single phase
Grounding Fault (SGF) and High Resistance Grounding Fault (HRGF), each
containing three-phase voltage and three-phase current data, and each cycle
has 82 sampling points. The waveforms are shown in Fig. 1a, Fig. 1b, Fig. 1d
and Fig. 1c, respectively. In the SGAH dataset, there are Inter Phase Short-
circuit Fault (IPSF), Two-phase Ground Fault (TGF), Single-phase Grounding
Fault (SGF), and Main Transformer Fault (MTF), each also containing three-
phase voltage and three-phase current data, with 100 sampling points per
cycle. Both datasets contain groundtruth consisting of three-phase voltage and
current data with fault labels. The waveform of these faults are shown in Fig.
1e, Fig. 1f and Fig. 1d. We also make SGAH dataset available to the public at
GitHub.

Evaluation Metrics: To verify the performance of our proposed method, the
following five metrics are adopted: accuracy, precision, recall, F1-score and
Receiver Operating Characteristic (ROC) curve. To calculate the accuracy, the
fault detection results are compared with ground truth. Accuracy is the ratio
of the number of correct predictions to the total number of samples. Precision
is the ratio of the number of samples correctly classified as faults to the
number of samples in the population that are classified as such. The higher
the precision is, the better performance of the model will gain. Recall rate
refers to the fact that the number of samples that are correctly classified
for a certain type of fault accounts for the actual samples. The higher the
recall rate is, the less the number of faults that are incorrectly classified
into other types of faults will be. To balance the accuracy and recall of our
model, we also calculate the F1-score. The performance of the proposed model
are also evaluated by the size of Area Under ROC Curve (AUC). The larger the
AUC is, the better the performance of the model has.

Data Augmentation Evaluation: To verify the validity of the proposed data
augmentation method, both dataset are divided into original dataset and
augmented dataset. Then we train the proposed AD-TFM-AT network, and test the
performance.

Ablation experiments: we conduct ablation experiments to show the performance
of TFM, AD-TFM and TFM-AT. TFM model is based on the LSTM by changing the
forget gate to a joint forget gate and adding a wavelet transform with fixed
scale parameters and translation parameters. AD-TFM is based on TFM with
wavelet transform of learnable parameters without the attention mechanism.
TFM-AT is based on TFM with the attention mechanism.

Comparison schemes: To validate the classification performance of the proposed
AD-TFM-AT, we use the following five comparision schemes. We train these
models using both IFPD and SGAH datasets with augmentation, and compare the
evaluation metrics on the ground truth.

  1. 1.

Support Vector Machines (SVM): The three-phase voltage and current data are
input into a set of Gaussian kernel functions based SVMs, where each SVM
detects one kind of faults. And the classification results of all SVMs are
combined to achieve fault classification.

  2. 2.

LSTM: The pre-processed three-phase voltage and current data are fed into a
three-layer LSTM for learning. Then feature classification of the LSTM output
is implemented by a fully connected layer.

  3. 3.

Minirocket[46]: Multiple features of three-phase voltage three-phase current
data are extracted using multiple convolution kernels which are represented by
two deterministic values $\\{-1,2\\}$. The multiple features are then used to
train a linear classifier for fault detection.

  4. 4.

SlI-CNN[19]: The three-phase voltage and current data are converted to
synchronous Lissajous images as the input to a CNN. The CNN contains three
convolutional layers and one fully-connected layer. Each convolutional layer
consists of batch normalization, max-pooling, and dropout. The last
convolutional layer connects a fully-connected layer for classification.

  5. 5.

HLCL[14]: The three-phase voltage and current waveform are decomposed into
approximate shapes and residuals by Meyer wavelet, and then further decomposed
into primitive and temporal relationships by Fast Fourier Transform (FFT).
Finally, fault classification is realized by variable probability statistics
and Bayesian hierarchical model.

Implementation details: The proposed model is implemented in Python3.7, and
the experimental code is available at GitHub222 https://github.com/smartlab-
hfut/AD-TFM-AT-Model. The training parameter settings are shown in Table I.
All the experiments are performed on four Nvidia Tesla V100 GPUs. We use a
cycle of three-phase voltage and current data as a data packet.

TABLE I: Experimental parameter settings. Network model | LSTM | TFM | AD-TFM | TFM-AT | AD-TFM-AT
---|---|---|---|---|---
Initial learning rate | 0.001 | 0.001 | 0.001 | 0.001 | 0.001
Batch size | 256 | 256 | 256 | 256 | 256
Optimizer | Adam | Adam | Adam | Adam | Adam
$D$ | 32 | 32 | 32 | 32 | 32
$K$ |  | 4 | 4 | 4 | 4
$J$ |  | 4 | 4 | 4 | 4
$\omega_{0}$ |  | 16 | 16 | 16 | 16

###  VII-B Experimental Results

Data augmentation results: First we perform data augmentation on IFPD and SGAH
datasets, and the results are shown in Table II.

TABLE II: Augmentation of IFPD Dataset and SGAH Dataset. Dataset | Fault type | Original data size | Augmented data size
---|---|---|---
IFPD | SIF | 137 | 3276
MIF | 164 | 2340
SGF | 678 | 7767
HRGF | 28 | 486
Normal | 300 | 2700
SGAH | IPSF | 342 | 2052
TGF | 70 | 420
SGF | 501 | 3006
MTF | 497 | 2982
Normal | 320 | 1920

Then we select the original data and augmented data on a pro-rata basis (8:2)
to train and test our proposed AD-TFM-AT neural network. The performance is
shown in the Table III. And the model classification performance is
represented by ROC curves in Fig. 6 and Fig. 7.

TABLE III: Performance evaluation of AD-TFM-AT with augmented data. Metrics | SGAH Data | IFPD Data
---|---|---
Original | Augmented | Original | Augmented
Accuracy | 0.97 | 0.99 | 0.82 | 0.97
Precision | 0.96 | 0.97 | 0.83 | 0.97
Recall | 0.98 | 0.98 | 0.74 | 0.96
F1-score | 0.97 | 0.98 | 0.75 | 0.96

From Table III, we can see that the model trained on the augmented dataset
generally performs better in terms of the accuracy, precision, and F1-score
than the model trained on the original dataset. In particular, the AD-TFM-AT
model trained on the augmented IFPD data better than trained on the original
dataset, of which the accuracy is increased from 0.82 to 0.97, and the
F1-score is increased from 0.75 to 0.96. The above experimental results show
that our proposed data augmentation method provides an effective support for
model training, and solves the problem that the amount of incipient fault data
of the power distribution network is insufficient.

(a)  (b)

Figure 6: ROC of AD-TFM-AT model on SGAH dataset. (a) SGAN original dataset.
(b) SGAH augmented dataset.

(a)  (b)

Figure 7: ROC of AD-TFM-AT model on IFPD dataset. (a) IFPD original dataset.
(b) IFPD augmented dataset.

The ROC of the data augmentation experiments are shown in Fig. 6 and Fig. 7.
We can see that the proposed AD-TFM-AT model has better classification ability
on both the original and augmented SGAH datasets. In particular, the average
AUC of the AD-TFM-AT model improves from 0.93 to 0.99 in the IFPD dataset, and
the AUC of identifying SGF improves from 0.85 to 1. This indicates that the
proposed data augmentation method allows the model to more sufficiently learn
each fault characteristic and improves the model’s resolution ability to
various faults.

Ablation experiments results: We train and test the four models of TFM, AD-
TFM, TFM-AT, AD-TFM-AT on the augmented IFPD and SGAH dataset, and the
performance is shown in Fig. 8 \- Fig. 11.

Figure 8: Ablation experiments results on SGAH dataset.

(a)  (b)  (c)  (d)

Figure 9: ROC of ablation models on SGAH dataset. (a) ROC of AD-TFM-AT. (b)
ROC of AD-TFM. (c) ROC of TFM-AT. (d) ROC of TFM. Figure 10: Ablation
experiments results on IFPD dataset.

(a)  (b)  (c)  (d)

Figure 11: ROC of ablation models on IFPD dataset. (a) ROC of AD-TFM-AT. (b)
ROC of AD-TFM. (c) ROC of TFM-AT. (d) ROC of TFM.

From Fig. 8, we can see that the model with the adaptive wavelet transform and
the Attention mechanism performs better on SGAH dataset. From Fig. 9, the Are
Under Curve (AUC) of the TFM model for TGF is only 0.74, indicating the poor
ability of the TFM model to detect this kind of fault. With the combination of
adaptive wavelet transform and attention mechanism, the AUC of the TPF exceeds
0.90. Both TGF and SGF belong to the ground fault, in which the three-phase
voltage and current at the time of fault occurrence have similar
characteristics, both of which show a voltage drop in the fault phase and
distortion in the three-phase current. The AD-TFM and AD-TFM-AT models with
the addition of adaptive wavelet transform have higher resolution than the
three-phase voltage and current features extracted by the fixed parameter
wavelet base in TFM. On the other hand, the TGF and SPGF fault durations are
different, and the TFM-AT and AD-TFM-AT models with the added Attention
mechanism focus on different time periods of fault information that the TFM
lacks attention.

From Fig. 10, we can see that the AD-TFM-AT model with the introduction of
adaptive wavelet transform and attention mechanism has significantly improved
in terms of precision, accuracy, recall and F1 score when tested on IFPD data.

Among them, the four evaluation metrics of the AD-TFM-AT model are 0.1 higher
than these of TFM model. From the ROC in Fig. 11, we can see that the AUC
under ROC curve for AD-TFM, TFM-AT and TFM models for HRGF is relatively low
compared to other faults, while that of AD-TFM-AT is as high as 0.97.
Therefore, the proposed method by combining the adaptive wavelet transform and
attention mechanism increases the depth of the network and improves the
accuracy and generalization ability of the model for incipient fault
detection.

The above results show that adding adaptive wavelet transform to extract fault
features at different times and frequencies can well deal with the non-
stationary characteristics of incipient faults such as TGF, and the
characterization of the fault feature vector for key fault information is
enhanced by the attention mechanism, thus enabling the proposed AD-TFM-AT
model to achieve high accuracy fault identification.

Comparison with existing methods: The results of the comparison with existing
methods are shown in Table IV.

TABLE IV: Comparison results with existing models. Models | SGAH Data | IFPD Data
---|---|---
Accuracy | Precision | Recall | F1score | Accuracy | Precision | Recall | F1score
AD-TFM-AT | 0.99 | 0.97 | 0.98 | 0.97 | 0.97 | 0.97 | 0.97 | 0.97
SLI-CNN[19] | 0.96 | 0.77 | 0.80 | 0.78 | 0.93 | 0.93 | 0.82 | 0.84
HLCL[14] | 0.96 | 0.93 | 0.93 | 0.93 | 0.96 | 0.93 | 0.95 | 0.94
Minirocket[46] | 0.96 | 0.81 | 0.97 | 0.80 | 0.91 | 0.84 | 0.93 | 0.88
LSTM | 0.91 | 0.88 | 0.91 | 0.90 | 0.93 | 0.90 | 0.93 | 0.91
SVM | 0.82 | 0.81 | 0.77 | 0.79 | 0.85 | 0.83 | 0.77 | 0.80

From Table IV, we can see that our proposed AD-TFM-AT model has the highest
metrics on both SGAH and IFPD datasets, especially the accuracy reaches 0.99
and 0.97, respectively. In addition, on the SGAH dataset, Minirocket, SLI-CNN,
and HLCL all achieve 0.96 accuracies, and SVM only has 0.82 lowest accuracies.
Besides, Minirocket also has a high Recall up to 0.97, and HLCL also performs
well. Note that SLI-CNN is only 0.77 on precision and F1-score 0.78. On the
IFPD dataset, the accuracy of the proposed AD-TFM-AT model reaches 0.97, which
is the highest among all models. In addition, Minirocket’s and LSI-CNN’s
accuracy reach 0.91 and 0.93, respectively. HLCL’s accuracy and LSTM’s
accuracy reach 0.96 and 0.93, respectively. SVM’s accuracy is 0.85, which is
the lowest. This is because both TGF and SGF are one kind of ground fault,
their three-phase voltage and three-phase current waveform have similar
characteristics. The voltage waveform shows a drop in two phases. And another
phase voltage maintains a normal state. Therefore, the waveform features of
these two faults obtained by using convolution through images have similarity,
which leads to low final classification accuracy and F1-score.

The above results show that among the existing fault detection methods, AD-
TFM-AT has best performance. Instead, the Minirocket method and the SLI-CNN
method use convolution to extract the features of time series, which lacks the
analysis for non-stationary. Besides, they extract fault information accounts
for a small component of the overall information, which may make wrong
decisions. Therefore, it is not advisable to directly apply existing
classification methods for fault classification. The AD-TFM-AT performs
adaptive wavelet transform on the fault waveform data compared to other
methods to achieve analysis of non-stationary. In addition, AD-TFM-AT also
uses the attention mechanism to focus global information on fault. These make
AD-TFM-AT the best performer on both datasets.

##  VIII Conclusion

In this paper, we focus on incipient fault detection in power distribution
systems and analyzed the non-stationary characteristics of incipient faults.
We propose an AD-TFM cell by embedding wavelet transform into the LSTM, to
extract features in time and frequency domain from the non-stationary
incipient fault signals. We make scale parameters and translation parameters
of wavelet transform learnable to follow the dynamic input signals to analyse
incipient fault with multi-resolution and multi-dimension analysis. Based on
the stacked AD-TFM cells, we design a AD-TFM-AT model to obtain more efficient
fault features. In addition, we propose two data augmentation methods, namely
phase switching and temporal sliding, to effectively enlarge the training
datasets. Experimental results on two open datasets show that our proposed AD-
TFM-AT model and data augmentation methods achieve better performance of
incipient fault detection in power distribution system.

## Acknowledgment

This work is supported in part by grants from the National Natural Science
Foundation of China (52077049, 51877060, 62173120), the Anhui Provincial
Natural Science Foundation (2008085UD04, 2108085UD07, 2108085UD11), the 111
Project (BP0719039).

## References

  * [1] Q. Li, Y. Deng, X. Liu, W. Sun, W. Li, J. Li, and Z. Liu, “Autonomous smart grid fault detection,” _IEEE Communications Standards Magazine_ , 2023.
  * [2] Q. Li, H. Tang, Z. Liu, J. Li, X. Xu, and W. Sun, “Optimal resource allocation of 5g machine-type communications for situation awareness in active distribution networks,” _IEEE Systems Journal_ , vol. 16, no. 3, pp. 4187–4197, 2022.
  * [3] J. Li, Y. Deng, W. Sun, W. Li, R. Li, Q. Li, and Z. Liu, “Resource orchestration of cloud-edge–based smart grid fault detection,” _ACM Transactions on Sensor Networks_ , vol. 18, no. 3, aug 2022.
  * [4] W. Zhang, X. Xiao, K. Zhou, W. Xu, and Y. Jing, “Multicycle incipient fault detection and location for medium voltage underground cable,” _IEEE Transactions on Power Delivery_ , vol. 32, no. 3, pp. 1450–1459, 2017.
  * [5] H. Samet, S. Khaleghian, M. Tajdinian, T. Ghanbari, and V. Terzija, “A similarity-based framework for incipient fault detection in underground power cables,” _International Journal of Electrical Power & Energy Systems_, vol. 133, p. 107309, 2021.
  * [6] S. Kulkarni, A. J. Allen, S. Chopra, S. Santoso, and T. A. Short, “Waveform characteristics of underground cable failures,” in _IEEE PES General Meeting_ , 2010, pp. 1–8.
  * [7] K. Manivinnan, C. L. Benner, B. D. Russell, and J. A. Wischkaemper, “Automatic identification, clustering and reporting of recurrent faults in electric distribution feeders,” in _2017 19th International Conference on Intelligent System Application to Power Systems (ISAP)_ , 2017, pp. 1–6.
  * [8] Q. Li, Y. Zhu, J. Ding, W. Li, W. Sun, and L. Ding, “Deep reinforcement learning based resource allocation for cloud edge collaboration fault detection in smart grid,” _CSEE Journal of Power and Energy Systems_ , pp. 1–10, 2022.
  * [9] T. S. Sidhu and Z. Xu, “Detection of incipient faults in distribution underground cables,” _IEEE Transactions on Power Delivery_ , vol. 25, no. 3, pp. 1363–1371, 2010.
  * [10] S. N. Zhang Chao and L. Yudun, “Incipient fault identification of distribution networks based on feature matching of power disturbance data,” _Electrical Engineering_ , vol. 103, pp. 1432–0487, 2021.
  * [11] H. Samet, M. Tajdinian, S. Khaleghian, and T. Ghanbari, “A statistical-based criterion for incipient fault detection in underground power cables established on voltage waveform characteristics,” _Electric Power Systems Research_ , vol. 197, p. 107303, 2021.
  * [12] C. Zhao, K. Li, Y. Li, L. Wang, Y. Luo, X. Xu, X. Ding, and Q. Meng, “Novel method based on variational mode decomposition and a random discriminative projection extreme learning machine for multiple power quality disturbance recognition,” _IEEE Transactions on Industrial Informatics_ , vol. 15, no. 5, pp. 2915–2926, 2019.
  * [13] Q. Xiong, X. Feng, A. L. Gattozzi, X. Liu, L. Zheng, L. Zhu, S. Ji, and R. E. Hebner, “Series arc fault detection and localization in dc distribution system,” _IEEE Transactions on Instrumentation and Measurement_ , vol. 69, no. 1, pp. 122–134, 2020.
  * [14] S. Xiong, Y. Liu, J. Fang, J. Dai, L. Luo, and X. Jiang, “Incipient fault identification in power distribution systems via human-level concept learning,” _IEEE Transactions on Smart Grid_ , vol. 11, no. 6, pp. 5239–5248, 2020.
  * [15] X. Jiang, B. Stephen, and S. McArthur, “A sequential bayesian approach to online power quality anomaly segmentation,” _IEEE Transactions on Industrial Informatics_ , vol. 17, no. 4, pp. 2675–2685, 2021.
  * [16] W. He, Y. He, and B. Li, “Generative adversarial networks with comprehensive wavelet feature for fault diagnosis of analog circuits,” _IEEE Transactions on Instrumentation and Measurement_ , vol. 69, no. 9, pp. 6640–6650, 2020.
  * [17] G.-Y. Kwon, C.-K. Lee, and Y.-J. Shin, “Diagnosis of shielded cable faults via regression-based reflectometry,” _IEEE Transactions on Industrial Electronics_ , vol. 66, no. 3, pp. 2122–2131, 2019.
  * [18] Y. Wang, F. Zhang, and S. Zhang, “A new methodology for identifying arc fault by sparse representation and neural network,” _IEEE Transactions on Instrumentation and Measurement_ , vol. 67, no. 11, pp. 2526–2537, 2018.
  * [19] M. Izadi and H. Mohsenian-Rad, “A synchronized lissajous-based method to detect and classify events in synchro-waveform measurements in power distribution networks,” _IEEE Transactions on Smart Grid_ , vol. 13, no. 3, pp. 2170–2184, 2022.
  * [20] Y. Ma, D. Oslebo, A. Maqsood, and K. Corzine, “Dc fault detection and pulsed load monitoring using wavelet transform-fed lstm autoencoders,” _IEEE Journal of Emerging and Selected Topics in Power Electronics_ , vol. 9, no. 6, pp. 7078–7087, 2021.
  * [21] M. R. Skydt, M. Bang, and H. R. Shaker, “A probabilistic sequence classification approach for early fault prediction in distribution grids using long short-term memory neural networks,” _Measurement_ , vol. 170, p. 108691, 2021.
  * [22] M.-D. Liu, L. Ding, and Y.-L. Bai, “Application of hybrid model based on empirical mode decomposition, novel recurrent neural networks and the arima to wind speed prediction,” _Energy Conversion and Management_ , vol. 233, p. 113917, 2021.
  * [23] X. Siheng, “Incipient fault data in power distribution systems,” 2019. [Online]. Available: https://dx.doi.org/10.21227/bwjy-7e05
  * [24] Q. Li, Y. Deng, J. Ding, W. Sun, and W. Li, “Time-frequency memory recurrent neural networks empowered incipient fault detection in power distribution system,” in _2021 IEEE 5th Conference on Energy Internet and Energy System Integration (EI2)_ , 2021, pp. 3190–3194.
  * [25] M. Jannati, B. Vahidi, and S. H. Hosseinian, “Incipient faults monitoring in underground medium voltage cables of distribution systems based on a two-step strategy,” _IEEE Transactions on Power Delivery_ , vol. 34, no. 4, pp. 1647–1655, 2019.
  * [26] L. A. da Costa, D. S. Gazzana, R. C. Leborgne, and D. W. P. Thomas, “Incipient fault location in underground distribution networks using electromagnetic time reversal,” in _2020 IEEE International Conference on Environment and Electrical Engineering and 2020 IEEE Industrial and Commercial Power Systems Europe (EEEIC / I &CPS Europe)_, 2020, pp. 1–6.
  * [27] N. Ardi, N. A. Setiawan, and T. Bharata Adji, “Analytical incremental learning for power transformer incipient fault diagnosis based on dissolved gas analysis,” in _2019 5th International Conference on Science and Technology (ICST)_ , vol. 1, 2019, pp. 1–4.
  * [28] S. M. de Andrade Lopes, R. A. Flauzino, and R. A. C. Altafim, “Incipient fault diagnosis in power transformers by data-driven models with over-sampled dataset,” _Electric Power Systems Research_ , vol. 201, p. 107519, 2021.
  * [29] G. Lu, Q. Zhang, D. Zheng, S. Mirsaeidi, and P. Zhang, “Detection and location approach to incipient grounding defect in transformer winding based on sequence voltage ratios at pv inverter switching frequency,” _IEEE Transactions on Industrial Electronics_ , pp. 1–9, 2022.
  * [30] X. Wei, X. Wang, J. Gao, D. Yang, K. Wei, L. Guo, and Y. Wei, “Faulty feeder identification method considering inverter transformer effects in converter dominated distribution network,” _IEEE Transactions on Smart Grid_ , pp. 1–1, 2022.
  * [31] Y. Liu, Y. Zhao, L. Wang, C. Fang, B. Xie, and L. Cui, “High-impedance fault detection method based on feature extraction and synchronous data divergence discrimination in distribution networks,” _Journal of Modern Power Systems and Clean Energy_ , pp. 1–12, 2022.
  * [32] Y.-L. Shen and R.-J. Wai, “Wavelet-analysis-based singular-value-decomposition algorithm for weak arc fault detection via current amplitude normalization,” _IEEE Access_ , vol. 9, pp. 71 535–71 552, 2021.
  * [33] R. Bhandia, J. d. J. Chavez, M. Cvetković, and P. Palensky, “High impedance fault detection using advanced distortion detection technique,” _IEEE Transactions on Power Delivery_ , vol. 35, no. 6, pp. 2598–2611, 2020.
  * [34] X. Wang, X. Wei, J. Gao, G. Song, M. Kheshti, and L. Guo, “High-impedance fault detection method based on stochastic resonance for a distribution network with strong background noise,” _IEEE Transactions on Power Delivery_ , vol. 37, no. 2, pp. 1004–1016, 2022.
  * [35] X. Zheng, C. Chao, Y. Weng, H. Ye, Z. Liu, P. Gao, and N. Tai, “High-frequency fault analysis based pilot protection scheme for a distribution network with high photovoltaic penetration,” _IEEE Transactions on Smart Grid_ , pp. 1–1, 2022.
  * [36] H. Mirshekali, R. Dashti, H. R. Shaker, R. Samsami, and A. J. Torabi, “Linear and nonlinear fault location in smart distribution network under line parameter uncertainty,” _IEEE Transactions on Industrial Informatics_ , vol. 17, no. 12, pp. 8308–8318, 2021.
  * [37] A. Ameli, M. Ghafouri, H. H. Zeineldin, M. M. A. Salama, and E. F. El-Saadany, “Accurate fault diagnosis in transformers using an auxiliary current-compensation-based framework for differential relays,” _IEEE Transactions on Instrumentation and Measurement_ , vol. 70, pp. 1–14, 2021.
  * [38] F. Husari and J. Seshadrinath, “Incipient interturn fault detection and severity evaluation in electric drive system using hybrid hcnn-svm based model,” _IEEE Transactions on Industrial Informatics_ , vol. 18, no. 3, pp. 1823–1832, 2022.
  * [39] Z. Li, H. Yang, F. Yang, T. Tan, X. Lu, and J. Tian, “An infrared image based state evaluation method for cable incipient faults,” _Electric Power Systems Research_ , vol. 210, p. 108148, 2022.
  * [40] P. Khaledian and H. Mohsenian-Rad, “Automated event region identification and its data-driven applications in behind-the-meter solar farms based on micro-pmu measurements,” _IEEE Transactions on Smart Grid_ , vol. 13, no. 3, pp. 2094–2106, 2022.
  * [41] X. Jiang, B. Stephen, and S. McArthur, “Automated distribution network fault cause identification with advanced similarity metrics,” _IEEE Transactions on Power Delivery_ , vol. 36, no. 2, pp. 785–793, 2021.
  * [42] H. Liu, S. Liu, J. Zhao, T. Bi, and X. Yu, “Dual-channel convolutional network-based fault cause identification for active distribution system using realistic waveform measurements,” _IEEE Transactions on Smart Grid_ , pp. 1–1, 2022.
  * [43] Q. Li, J. Zhang, J. Zhao, J. Ye, W. Song, and F. Li, “Adaptive hierarchical cyber attack detection and localization in active distribution systems,” _IEEE Transactions on Smart Grid_ , vol. 13, no. 3, pp. 2369–2380, 2022.
  * [44] W. Sima, H. Zhang, and M. Yang, “Edge–cloud collaboration detection approach for small-sample imbalanced faults in power lines,” _IEEE Transactions on Instrumentation and Measurement_ , vol. 71, pp. 1–11, 2022.
  * [45] S. Zhang, N. Qu, T. Zheng, and C. Hu, “Series arc fault detection based on wavelet compression reconstruction data enhancement and deep residual network,” _IEEE Transactions on Instrumentation and Measurement_ , vol. 71, pp. 1–9, 2022.
  * [46] A. Dempster, D. F. Schmidt, and G. I. Webb, “Minirocket: A very fast (almost) deterministic transform for time series classification,” in _Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining_. New York, NY, USA: Association for Computing Machinery, 2021, p. 248–257.