# Low Temperature Domain Wall Freezing and Non-Equilibrium Dynamics in the
Transverse-Field Ising Model Material CoNb2O6

C. L. Sarkis  Department of Physics, Colorado State University, 200 W. Lake
St., Fort Collins, CO 80523-1875, USA  S. Säubert  Department of Physics,
Colorado State University, 200 W. Lake St., Fort Collins, CO 80523-1875, USA
V. Williams  Department of Physics, Colorado State University, 200 W. Lake
St., Fort Collins, CO 80523-1875, USA  E.S. Choi  National High Magnetic Field
Laboratory, Florida State University, Tallahassee, FL 32310-3706, USA.  T. R.
Reeder  Department of Physics and Astronomy and Institute for Quantum Matter,
Johns Hopkins University, Baltimore, Maryland 21218, USA  H. S. Nair
Department of Physics, University of Texas El Paso, 500 W University Ave, El
Paso, Texas 79902  K. A. Ross<EMAIL_ADDRESS>Department of Physics,
Colorado State University, 200 W. Lake St., Fort Collins, CO 80523-1875, USA
Quantum Materials Program, CIFAR, MaRS Centre, West Tower 661 University Ave.,
Suite 505, Toronto, ON, M5G 1M1, Canada

###### Abstract

CoNb2O6 is a rare realization of the transverse field Ising model (TFIM),
making it a useful tool for studying both equilibrium and non-equilibrium
many-body quantum physics. Despite a large body of work dedicated to
characterizing this material, details of the ordered states in the presence of
relatively weak transverse fields have not been discussed in detail. Here, we
present a detailed study of CoNb2O6 via ac susceptibility measurements in
order to further characterize its low temperature behavior in the presence of
a transverse field. Specifically, we call attention to an unconventional
freezing transition in zero-field below TF = 1.2 K, occurring _within_ the
well-known commensurate antiferromagnetic (CAFM) state that onsets at $T_{N2}$
= 1.9 K. We performed a series of transverse-field quenches into this frozen
state, which resulted in a slowly relaxing susceptibility, $\chi^{\prime}(t)$,
that followed a logarithmic decay within the time range measured. We discuss
the frozen state in the context of the freezing of previously discussed “free”
chains arising from domain walls between the four degenerate sublattices of
the CAFM state. We also attempted to observe Kibble-Zurek scaling by quenching
the transverse field into the frozen state at different rates. This produced a
null result; the behavior can be fully explained by coarsening of domains over
the timescale of the quenches. The absence of a clear Kibble-Zurek scaling is
itself surprising, given the proposed ubiquity of the phenomenon for general
second order phase transitions.

##  I Introduction

Recently non-equilibrium condensed matter has become a prominent field of
study as a generator of exotic phenomena, such as many-body localization[1, 2,
3] and Kibble-Zurek scaling[4, 5, 6]. While theory has been propitious in some
areas, much progress in non-equilibrium relies heavily on the guidance of
experiment. Thus, experimental studies which are centered on tractable models
are crucial for the field of non-equilibrium physics.

One of the archetypal models of condensed matter is the transverse field Ising
model (TFIM)[7, 8, 9],

$H=J\sum_{<i,j>}\sigma_{i}^{z}\sigma_{j}^{z}-h_{x}\sum_{i}\sigma_{i}^{x},$ (1)

where $\sigma_{i}^{\alpha}\ (\alpha=x,y,z)$ are Pauli spin matrices, $J$ is
the exchange interaction strength, and $h_{x}$ represents an external magnetic
field transverse to the Ising (z) axis. This model can be exactly solved in 1D
and features a quantum critical point at $J=2h_{x}$[7, 9]. The TFIM provides a
robust theoretical framework to study quantum criticality and non-equilibrium
many-body physics.

While real material analogs of the TFIM with critical fields accessible in
current laboratory settings are rare, one key example is the quasi-1D Ising
material CoNb2O6. This material was extensively studied from the 1970’s to
2000’s, with the goal of characterizing its complex magnetic phase diagram in
the presence of a magnetic field [10, 11, 12, 13, 14, 15, 16], though the
_transverse_ field behavior was not investigated until later [17]. The
electronically insulating CoNb2O6 crystallizes into the orthorhombic Columbite
space group P$bcn$, with room temperature lattice parameters of $a$ =
14.1475$~{}\AA$, $b$ = 5.712$~{}\AA$, and $c$ = 5.045$~{}\AA$ [18]. The Co2+
sublattice is shown in Fig. 1a. Co2+ ions are linked by oxygen octahedra
forming zig-zag chains along the $c$-axis, with $\sim$90∘ superexchange giving
rise to ferromagnetic nearest neighbor interactions $J_{0}$ and a pronounced
1D behavior [19]. Staggered planes of isosceles triangles form in the
$ab$-plane, with antiferromagnetic (AFM) interactions $|J_{1}|>|J_{2}|$
between chains estimated to be an order of magnitude smaller than $J_{0}$[20].
The isosceles triangular arrangement combined with AFM Ising interactions
results in geometric frustration, which when combined with the 1D magnetism of
the chains and strongly Ising-like interactions, leads to a complex magnetic
phase diagram that strongly depends on the field direction [21, 22, 13]. In
zero-field, the system undergoes a phase transition at $T_{N1}$ = 2.9 K into
an incommensurate AFM (ICAFM) phase with temperature-dependent wavevector
$q=(0~{}q_{y}~{}0)$. Below $T_{N2}$ = 1.9 K, the system locks into a
commensurate AFM (CAFM) phase with $q=(0~{}0.5~{}0)$ via a first-order phase
transition. Within the CAFM phase, neutron diffraction showed that CoNb2O6 has
an ordered moment of $\mu\approx 3.2\mu_{\mathrm{B}}$[14]. Heat capacity
measurements showed that magnetic contributions persist up to $T$ = 25 K [13]
and have an associated entropy of $R$ln2, supporting an
$S_{\text{eff}}=\frac{1}{2}$ picture for the low temperature angular momentum
degrees of freedom [13, 23]. In the CAFM state, there are two local moment
directions, canted $\pm 31^{\circ}$ away from the $c$-axis and lying within
the $ac$-plane. A series of papers by Kobayashi, _et al._ reported anisotropic
domain coarsening within the $ab$-plane (the triangular network) at $T$ = 1.5
K exhibiting an anomalously small growth exponent of $n$ = 0.2[24, 25]. This
low growth exponent was attributed to the effects of frustration.

More recently, there has been renewed interest in CoNb2O6 due to its
application to the TFIM[21]. Neutron scattering under a transverse field (i.e.
applied along the $b$-axis) revealed an emergent $\mathbb{E}_{8}$
symmetry[17], which describes the 1D transverse field Ising chain at the
quantum critical point with small but finite longitudinal field[26, 27, 28,
21]. Transverse field heat capacity[23] and nuclear magnetic resonance [29]
measurements revealed that the quantum phase transition (QPT) for an isolated
Ising chain in CoNb2O6 would be at 5.25 T, however, this is buried within the
3D ordered phase which exhibits a QPT at 5.45 T[22], due to weak but non-zero
interchain couplings. Recently it has been argued that additional off-diagonal
exchange interaction terms are important for a quantitative understanding of
CoNb2O6[30, 31], implying that the symmetry is not strictly Ising-like. Even
so, the field-induced quantum critical point of the proposed models still maps
onto the TFIM [30]. Given how well-studied the material now is due to its
behavior in a transverse field, there is a surprising lack of detail in the
literature about its response to low transverse field strengths. In order to
use CoNb2O6 to experimentally investigate non-equilibrium properties of the
TFIM, it is important to fully characterize its low temperature and low
transverse field magnetic properties, particularly since they exhibit slow
relaxation in this regime.

Figure 1: (a) Magnetic structure of CoNb2O6 in the CAFM state. Nearest
neighbors form zig-zag chains along $c$ with ferromagnetic interactions
($J_{0}$). Second ($J_{1}$) and third ($J_{2}$) neighbors (both AFM) connect
isosceles triangles in the $ab$-plane which stabilize 3D order at low
temperatures. $J_{0}$ is roughly an order of magnitude larger than the other
exchange interactions, making CoNb2O6 quasi-1D. (b) Diagram of the ac-
susceptometer at the NHMFL, showing the directions of the ac and dc-fields.
The ac-field can be set at any angle with respect to the dc-field, within a
single plane of rotation. For our experiments, the ac field was oriented along
the $c$-axis (the average Ising direction), and the dc field was usually
oriented along the $b$-axis (transverse to the Ising axes).

Here, we report ac susceptibility measurements on CoNb2O6 taken down to a
temperature of $\it{T}$ = 0.5 K. We used a transverse dc field geometry (Fig.
1 b) to measure the ac susceptibility along the $c$-axis in various scenarios:
1) the zero-field ac susceptibility, 2) the ac susceptibility as a function of
transverse field, and 3) the relaxation of the ac susceptibility in zero-field
after ramping the transverse magnetic field to zero at various rates
($r_{Q}$), spanning 0.1 T/min to 10 T/min. For the zero-field data we find
good agreement with prior measurements [13], including the presence of a
freezing transition which onsets at $T_{F}$ $\sim$ 1.2 K in our sample. This
freezing transition, which occurs within the commensurate ordered phase, is
not usually discussed in studies of CoNb2O6, but appears to be present in
several samples [12, 13, 20, 23]. The transverse-field dependence of the ac
susceptibility at 0.5 K reveals several features which can be identified as
transitions between different ordered states, culminating in the 3D quantum
critical point (QCP) at 5.45 T. Upon quenching the transverse field into the
frozen state, we observe a relaxation in the real ($\chi^{\prime}(t)$) and
imaginary ($\chi^{\prime\prime}(t)$) components of the ac susceptibility,
which is best described by a logarithmic decay. This contrasts with the power
law relaxation of ac susceptibility (and neutron Bragg diffraction) that was
found by Kobayashi, _et al._ at $T$ = 1.5 K after longitudinal ($c$-axis)
field quenches [24, 25], and points to the role of a disordered potential
leading to modified coarsening behavior below $T_{F}$. Varying the quench rate
($r_{Q}$) of the transverse field, we also find a logarithmic dependence of
$\chi^{\prime}$ on $r_{Q}$. Though the scaling could also be fit fairly well
to a power law, which would suggest a connection to the Kibble-Zurek mechanism
(KZM), we find that the dependence can be attributed entirely to a systematic
effect resulting from the relaxing population of domain walls over the course
of the quench time. This highlights the care necessary in showing KZM behavior
for experimental systems in which coarsening occurs on a similar timescale to
the quench time[32, 33]. It appears that KZM is not evident in our
measurements, which in itself is surprising given the ubiquitous nature of the
mechanism for second order transitions[4, 6] and glass transitions[34, 35].

Figure 2: (a) Zero-field susceptibility of CoNb2O6 as a function of probe
frequency. A frequency-dependent peak is seen centered around $T_{F}\sim$ 1.2
K (shown in inset), consistent with Ref. 13. (b) Single crystal heat capacity
of CoNb2O6. A sharp peak at $T_{N1}=2.9$ K coincides with the ICAFM
transition, while a broad shoulder at $T_{N2}=1.9$ K coincides with the CAFM
transition. Heat capacity from Ref. 13 is also shown here in order to
highlight the overall agreement between measurements. (inset) A small shoulder
is observed in the heat capacity near $T=1$ K, which could be due to the
freezing transition observed in the zero-field susceptibility. (c) Dependence
of $\tau_{2}$ on temperature, which is a measure of the sample-to-stage
relaxation processes. $\tau_{2}$ becomes non-zero near the onset of the
frequency dependent low transition found in zero-field ac susceptibility.

##  II Experimental Methods

Small single crystals of CoNb2O6 (2 $\times$ 1 $\times$ 0.5 mm) were prepared
via flux growth, following Ref. 36, using 1.5 g CoNb2O6with 1 g Borax
(Na2B4O7) in a Pt crucible (1250 ∘C for 15 hrs then 750 ∘C for 12 hrs with
temperature ramps of 1.2 K/hr). Representative samples were crushed and
checked for phase purity with a x-ray powder diffractometer. Alignment and
crystallinity were checked with a Laue diffractometer, where the crystal was
aligned within $\sim 0.5^{\circ}$. Two single crystals were used for ac
susceptibility with masses of 13.6 mg and 8.2 mg, both cut into a cuboidal
shape.

Heat capacity measurements on a small single crystal (1.55 mg) grown from the
same batch were performed in zero field from 4 K to 0.06 K. The data were
taken in a Quantum Design Dynacool PPMS with dilution refrigerator insert
using the thermal relaxation method.

Transverse field ac susceptibility data were taken at the National High
Magnetic Field Laboratory (NHMFL) in Tallahassee with a 3He cryostat mounted
in a 20 MW 31 T 50 mm bore resistive solenoid dc magnet[37]. The samples were
subjected to both an ac and dc field, and were oriented such that the ac field
was applied along the average moment direction ($c$-axis), while the dc field
could be oriented anywhere within the $bc$-plane, i.e. between the average
moment direction and the transverse direction (Fig 1b). The ability to change
the angle _in situ_ enabled us to more accurately align the dc field to the
$b$-axis within the plane of rotation, by observing the symmetry of the ac
signal as a function of rotation angle (Appendix B). The resistive magnet
allowed for linear magnetic field ramps of the dc field with rates up to 10
T/min (note that a typical maximum ramp rate for a superconducting magnet is
$\leq$ 1 T/min, which is what motivated us to use a resistive magnet instead).
The ac field had amplitude 1.65 Oe and frequencies ranging between $f=40$ Hz
to 10 kHz, with most dc field-dependent measurements taken at 710 Hz. The dc
field-dependent data were collected at $\it{T}$ = 0.5 K with high temperature
background scans taken at $\it{T}$ = 10 K.

##  III Results & Discussion

###  III.1 Temperature dependence of zero-field susceptibility

The zero-field ac susceptibility data is shown in Fig. 2a. A kink in the
susceptibility at $T_{N1}$ = 2.9 K coincides with the ICAFM transition. An
abrupt decrease in susceptibility is observed at the CAFM transition, $T_{N2}$
= 1.9 K. CoNb2O6 shows a pronounced frequency dependence below $T_{F}$ = 1.2 K
accompanying a small peak in the susceptibility. This feature was also
observed by Hanawa, _et al._ in their ac susceptibility data, who attributed
it to a metamagnetic (first order) transition and noted that it did not appear
when measuring along the crystallographic $b$-axis[13]. The CAFM order is
known to persist to 0.05 K [22], so this frequency-dependent feature appears
to signal a frozen state that co-exists with the CAFM state.

The heat capacity of CoNb2O6 in zero-field is shown in Fig. 2b), which agrees
well with the literature[13, 23]. A sharp anomaly corresponds to $T_{N1}$,
while a low temperature shoulder corresponds to $T_{N2}$. A second smaller
shoulder is also observed around 1 K, which may be related to the freezing
transition discussed above. Significant magnetic heat capacity remains above
the transitions and is known to persist to $\sim$25 K [13], well within the
paramagnetic state, revealing the presence of short range spin correlations up
to this temperature. The thermal relaxation method allows one to analyze the
relaxation curves in terms of a two-time-constant model, where $\tau_{1}$ is
related to the heat capacity of the sample, and $\tau_{2}$ is related to the
heat flowing from within the sample to the sample platform and can be an
indicator for slow relaxation within the sample[38, 39]. The onset of
significant $\tau_{2}$ at $T\sim$ 1.2 K (shown in Fig. 2c) is consistent with
slow thermal relaxation within the sample, as expected based on the zero-field
limit of the freezing transition revealed by ac susceptibility at the same
temperature.

Figure 3:  Typical ac susceptibility data taken at $\it{T}$ = 0.5 K as a
function of transverse field for real component (a) and imaginary component
(b). A shoulder observed near 5.3 T (BUUD) in both real and imaginary
components is consistent with the field induced transition between the
paramagnetic and an “up up down” (UUD) ferrimagnetic phase identified in Ref.
22, which approximately corresponds to the TFIM QCP. A small hysteresis
(history dependence) is evident at this phase boundary, but it is more
pronounced below $\sim$3.7 T (BICAFM), which is a field-induced first-order
transition into the ICAFM state [22]. The transition into the CAFM state
reported in Ref. 22 around 3 T (BCAFM) is more difficult to observe in our
susceptibility data, though a small kink in the decreasing real component can
be observed. A ramp rate dependent discontinuity around $\sim$1.7 T (BF) may
represent the onset of the frozen phase.

Previous studies by Kobayashi, _et al._ have also shown slow relaxation in
CoNb2O6 within the CAFM phase, but at temperatures above the freezing
transition that we report [24, 25]. In those studies, the FM $c$-axis chains
were treated as single super-spins (since $J_{0}$ is by far the dominant
exchange interaction, the spins along each chain were assumed to have the same
orientation), and the defects of the ordering of those super-spins within the
$ab$-plane was investigated. The CAFM state in CoNb2O6 has a four-fold
degeneracy, so domains of this order populate randomly following a quench into
the ordered state, with domain walls forming between them. Kobayashi, _et al._
deduced the presence of “free” chains at certain types of domain walls, which
arise due to the frustration of the isosceles triangular lattice formed by the
chains. A time-dependence of both the correlation length and the magnetic
susceptibility was also reported by Ref. 25 at $\it{T}$ = 1.5 K, just above
the $T_{F}$. In that study, the relaxation behavior was modeled by a power
law, consistent with domain coarsening theory. However the growth exponent was
found to be anomalously low ($n=0.20\pm 0.02$), in contrast to standard
coarsening models of a four-fold degenerate Ising system with curvature-driven
domain growth, which predict a growth exponent of $n$ = 0.5[40, 41], though
within the anisotropic nearest-neighbor Ising (ANNNI) model used in Ref. 25
deviations from this exponent have been predicted[42].

One possible explanation for the freezing transition we observed within the
CAFM state is the influence of disorder on the domain wall dynamics discussed
above. A natural route to disorder in CoNb2O6 comes from the CoO–Nb2O5 binary
phase diagram [43], which shows that CoNb2O6 is not a “line compound”, i.e.
there is a fairly wide range of stoichiometries (within $\sim$ 1% of the ideal
1:1 molar ratio) that lead to the same average crystal structure (the
columbite structure type). The relatively wide compositional space that
stabilizes the columbite structure indicates a low enthalpy for defects
relative to the concomitant gain in configurational entropy (e.g anti-site
disorder or vacancies). Thus, even for perfect stoichiometry, one could expect
a relatively high defect density [44] 111Care must also be taken to convert
cobalt oxide precursors to a single type. When purchased from chemical
suppliers they often contain a mixture of Co3O4 and CoO which, if uncorrected,
can lead to inaccurate Co stoichiometry.. It is already known that CoNb2O6 can
be extremely sensitive to small amounts of disorder [46, 47]; for instance, in
MgxCo1-xNb2O6, less than 1% substitution ($x$=0.008) is enough to entirely
suppress the CAFM state at low temperatures[47].

###  III.2 Transverse-field dependence of susceptibility

Typical ac susceptibility data taken as a function of field at constant
temperature (0.5 K) are shown in Fig 3. The data were taken after cooling in
zero-field from 2 K to 0.5 K. Features in these data align well with phase
transitions identified from neutron scattering data [22] and are indicated in
the figure by arrows. A shoulder near 5.3 T shows the transition between an
ordered state and a field-polarized paramagnet, with a small accompanying
hysteresis between ramps of increasing and decreasing field. The hysteresis is
more dramatic below $\sim$ 3.7 T, where Ref. 22 reported a transition between
a ferrimagnetic phase and a field-induced ICAFM phase. A discontinuity in the
real and imaginary components of the susceptibility appears near 1.7 T, with
the field value at which it occurs weakly depending on quench rate, possibly
signifying the boundary of the frozen state.

Accompanying the development of hysteresis, a slow relaxation is observed in
the susceptibility. The slow relaxation is seen for all fields below the ICAFM
transition (3.7 T) but becomes more pronounced below the CAFM and freezing
transitions ($<3$ T) (see Appendix B.3). In the frozen state, it appears that
it would persist well beyond our maximum measurement time (600 s after
reaching zero-field).

Figure 4: (a) Relaxation in zero-field at $\it{T}$ = 0.5 K after several
quenches from a maximum field of 4 T at different rates, $r_{Q}$. Colors
represent different $r_{Q}$, as quantified by the color bar. (inset) Zero-
field relaxation of several quenches with $r_{Q}=2$ T/min showing a history-
dependence of the initial and final susceptibility, leading us to use the
difference $\chi^{\prime}_{D}$ as a measure of the relaxation at a given
$r_{Q}$ (see Eqn. 3). (b) Zero-field relaxation of 4 T quenches with a
logarithmic time scale, where time is shifted by a critical time “$t_{c}$”
defined as the time at which the field reaches 0.11 T. This produces straight
lines, confirming the logarithm form of relaxation, but indicating an
unusually low field for the onset of relaxation. (c) and (d)
$\chi^{\prime}_{D}$ vs $r_{Q}$ for a representative set of quenches (all
completed after the same cool-down after warming past $T_{F}$) from a maximum
field of 4T (c) and 6T (d). The fits were performed using Bayesian non-linear
regression. The logarithmic form (Eqn. 5) arises from purely systematic
effects, while the power law form could potentially arise from the KZM. Shaded
regions represent the 95% credible intervals of the posterior predictive
distribution. For both sets of quenches, the logarithmic form gives the best
fit.

In zero-field following a field quench, our time-dependent ac susceptibility
is best fit to a logarithmic relaxation form:

$\chi^{\prime}(t)=a\ln{\bigg{(}\frac{t-t_{0}}{\tau}\bigg{)}}+c$ (2)

where $a$ is a negative scale factor, $t_{0}$ is the onset time of the
relaxation, $\tau$ is needed to make the argument of the logarithm
dimensionless and may represent an intrinsic relaxation timescale, and $c$
represents an offset that is partially due to a background contribution. The
form shown in Eqn. 2 produces an unbounded $\chi^{\prime}$ for infinite time
and thus cannot represent the full relaxation curve. We treat it as an early-
time approximation for the relaxation, similar to the intermediate time
logarithmic relaxation proposed by Ref. 48. Thus the $c$ parameter also likely
accounts for some of the late-time behavior of the relaxation. The functional
form above provides a slightly better fit compared to a power law relaxation,
and likewise, the power law fit gives a low exponent (-0.10 $\pm$ 0.03),
hinting at the appropriateness of a logarithmic relaxation for our data (see
Appendix B.4).

The zero-field power law relaxation of ac susceptibility obtained in Ref. 25
was measured at 1.5 K, which is above $T_{F}$, and after different quench
protocols compared to ours (they used temperature quenches and a $c$-axis
field quench). We therefore do not necessarily expect the relaxation forms to
agree, particularly if the frozen state is due to disorder. Indeed, disorder
has been predicted to lead to logarithmic coarsening in Ising systems[49],
while other studies have proposed a crossover from a power law to a
logarithmic form beyond a critical time related to the length scale separating
impurities[50, 41]. Slow relaxation in many systems, including spin glass
systems, have also been modeled through hierarchically constrained dynamics
[51, 52, 53]. Such a dynamical structure has been shown to lead to either a
stretched exponential[54, 55] or logarithmic[48] form of the relaxation.

###  III.3 Quench-rate dependence of susceptibility

One may think of the zero-field ac susceptibility as containing the response
of the domain walls, as well as the intrinsic response of the CAFM ground
state. Generally, we expect any signature which displays a relaxation over
time to be partially attributed to the domain walls of the system (the units
resulting from non-equilibrium). However, one intriguing observation we have
made is the presence of aging effects, i.e. certain properties of the
relaxation depend on the overall history since cooling into the frozen state,
even after the transverse field is brought to 6 T, which is outside any
ordered states. This is discussed in more detail in Appendix B.3 (also see
Fig. B.1d and Fig. B.2). While intriguing, this aging complicates the analysis
of the quench rate dependence, since even identical repeated quenches lead to
different offsets of the susceptibility, as shown in Fig. 4a. To investigate
the _trends_ of the relaxation within the frozen state, we thus characterized
the relaxation by the difference between its initial value and final value

$\chi^{\prime}_{D}=\langle f\rangle-\langle l\rangle$ (3)

where $\langle f\rangle$ describes the average of the first second of zero-
field data after the quench and $\langle l\rangle$ is the average of last
second of collected data (usually at 300s). This represents the initial
density of defects due to each new quench. When $\chi^{\prime}_{D}$ is plotted
as a function of quench rate ($r_{Q}$), a clear trend is seen (Fig. 4c-d). In
both crystals studied, this same type of dependence on quench rate was
observed for all sets of quench data, which include quenches from 6 T across
the QCP as well as quenches from 4 T which do not cross the QCP. Thus, the
dependence on $r_{Q}$ is not due to the either the 1D or 3D QPT. Rather, it
seems to be due to a very low field transition around 0.11 T, which is made
evident by the analysis described in Appendix B.5. This field scale is also
evident in the time-dependent data; when we shift the time axis by the time
this field was reached ($t_{c}$), we see that $\chi^{\prime}(t)$ is linear on
a log-log scale (Fig. 4b). This also further emphasizes the appropriateness of
the log relaxation form (Eqn. 2) discussed above.

|  | 4 T | 6 T
---|---|---|---
| $a_{L}$ | 2.30(6)$\times 10^{-6}$ | 1.9(2)$\times 10^{-6}$
Logarithmic | $B_{c}$ | 0.20(2) | 0.13(6)
| $a_{p}$ | 7.1(3)$\times 10^{-6}$ | 7.0(3)$\times 10^{-6}$
Power law | $b$ | 0.27(2) | 0.21(3)
Table 1: Fitted parameters for the quench-rate dependent data and models shown
in Fig 4 c and d. Bracketed numbers indicate one standard deviation.

Turning to the quench rate dependence of the difference in susceptibility,
$\chi^{\prime}_{D}(r_{Q})$ (Fig. 4c and d), we used a Bayesian non-linear
regression to fit two models; The first is a power law form,

$\chi^{\prime}_{D}(r_{Q})=a_{p}r_{Q}^{b},$ (4)

where $a_{p}$ is a scale factor, and $b$ is the exponent. This was motivated
by the fact that an almost-linear dependence of $\chi^{\prime}_{D}(r_{Q})$
appears on a log-log plot, and would potentially indicate KZM (see Appendix A
for a discussion of the expected values for $b$ for this scenario).

The second analytic form was derived considering a constant initial population
of domain walls produced at a critical field $B_{c}$, which coarsen during the
quenches before the field reaches zero (see Appendix B.5 for more details).
This logarithmic form is,

$\chi^{\prime}_{D}(r_{Q})=a_{L}\ln\left[\frac{B_{c}r_{Q}}{t_{m}+B_{c}r_{Q}}\right],$
(5)

where $t_{m}$ is the measurement time between $\langle f\rangle$ and $\langle
l\rangle$ (300 s for most quenches), $a_{L}$ is a scale factor that would be
expected to be the same as $a$ as in Eqn 2, and $B_{c}$ is a fitting parameter
describing the critical field at which relaxation starts.

Both forms result in a reasonable fit of the data, but the logarithmic form,
which we refer to as “systematics” (since it does not involve any additional
physics such as KZM), does better at the lowest quench rates. The means of the
fitting parameters and their standard deviations of the parameters for the two
models are shown in Table 1, and details of the fitting procedure and
resulting joint distributions of parameters are in Appendix B.5. We note that
the $\chi^{\prime}_{D}(r_{Q})$ dependence of the 6 T quenches was
significantly noisier than that of the 4 T quenches. The reason for this is
currently unknown, but it was a reproducible effect during the experiments.

The power law model would potentially indicate KZM, but with an unexpected
scaling exponent of 0.27(2) for the 4 T data, and 0.21(3) for the 6 T data.
The naively expected exponents are greater than 0.5, but they can be reduced
due to coupling to a bath (see Appendix A). Yet, despite the predicted
ubiquity for KZM across phase transitions, the logarithm explains our data
better, as is evident from Fig 4. The form of the logarithmic scaling can be
completely understood to be a consequence of a systematic effect, namely the
coarsening of domains during the quench. Another way to see this is that the
fitting parameter $a$ in Eqn. 2, which could reasonably be expected to scale
with defect density, is independent of $r_{Q}$ (see Appendix B.3).

Thus, it appears that we have not observed Kibble Zurek scaling in this
experiment, which is somewhat surprising, since it is expected to apply at all
second order phase transitions as well as glass transitions. One clue that may
help to explain this is the low critical field ($B_{c}<0.3$ T) inferred from
the systematics analysis (Eqn. 5), which is well below any of the known phase
transitions for a $b$-axis field applied to CoNb2O6[22]. Further, the time
associated with crossing this field is also the offset in time required to
produce linear dependence of the susceptibility on a logarithmic time scale
(Fig. 4b), i.e. $t_{c}$.

One possibility that could help to explain the low $B_{c}$, which also seems
to explain the absence of Kibble Zurek scaling, is that we are primarily
observing the effects of a transition that is due to a small component of the
field being within the $ac$-plane, due to the slight misorientation of the
sample (estimated to be about 1∘). Indeed, CoNb2O6 is known to exhibit a
complex phase diagram when the field is applied in the $ac$-plane, with
several low field transitions are first-order in nature [13]. Thus, it may be
that the dominant behavior observed in our quench-rate experiments is due to
coarsening effects that occur after crossing a first-order phase transition
induced by a small component of the field being applied in the $ac$-plane,
rather than a second-order or glass transition.

##  IV Conclusion

In summary, we have performed ac susceptibility measurements of the quasi-1D
Ising material CoNb2O6 in zero-field and under predominantly transverse
magnetic fields. At low temperature (0.5 K) we see evidence of several
transverse-field-induced transitions at fields below the known QCP (at 5.3 T),
some of which are similar to those which have been predicted theoretically
[21] and observed in a prior study [22]. We have also observed and
characterized a zero-field transition into an unconventional frozen state at
Tf = 1.2 K, which is within the known commensurate antiferromagnetic state.
This is consistent with earlier reports of extremely slow dynamics in this
temperature range, but has not been discussed in detail before. Within this
frozen state, the $c$-axis ac susceptibility shows a logarithmic relaxation
over time in response to a transverse ($b$-axis) dc field quench. This form of
relaxation is expected for coarsening of domains in the presence of disordered
potentials, and this is an appealing explanation given prior work which
successfully attributed some higher temperature behavior of CoNb2O6 to domain
wall motion and coarsening [24, 25]. However, the presence of lattice disorder
in our (and other group’s) samples of CoNb2O6 remains to be confirmed.

We investigated the effect on the ac susceptibility response of “quenching” a
transverse field across the various field-induced phase transitions of
CoNb2O6. We found a surprising aging effect that indicates that the system is
not reset by applied a field of 6 T, which is above the QPT, within the
“quantum paramagnetic” regime. After accounting for this aging effect, we do
find a distinct dependence of $\chi^{\prime}$ on quench rate, similar to a
power law scaling that would be expected based on the KZM. However, our
observed dependence does not strongly depend on whether the QPT is crossed or
not, and furthermore, we find it can be better fit by a logarithmic function
arising from systematic effect, namely coarsening during the field quench.
Thus, we observe no evidence for the KZM in quenches across the QPT, or across
any of the field-induced transitions including the freezing transition.
Rather, the systematic analysis suggests that the observed effect is due to
crossing a first-order transition brought about by a slight field
misorientation.

This “null result” is somewhat surprising given the proposed ubiquity of the
KZM phenomenon for second order transitions (and generalized to glass
transitions), and the fact that we clearly observe non-equilibrium states
generated by the quenches. However, we cannot rule out that the Kibble Zurek
scaling is being obscured by the aging effect we have observed.

Overall, our results emphasize the complexity of the dynamical behavior of the
well-studied quasi-1D transverse field Ising model material, CoNb2O6 under
relatively weak transverse fields. This weak transverse field regime has until
now not been explored in detail experimentally, but appears to contain a
wealth of intriguing phenomena related to the frustration of the isosceles
triangular lattice, which is likely to be strongly influenced by additional
quantum fluctuations produced by the transverse field.

###### Acknowledgements.

This research was funded by Dept of Energy grant DE-SC0018972. Authors would
like to acknowledge Prof. Tarun Grover for insightful conversations relating
to the Kibble-Zurek mechanism and domain coarsening effects. The authors would
also like to thank Prof. James Neilson for stimulating conversations about
defect chemistry in crystalline materials. A portion of this work was
performed at the National High Magnetic Field Laboratory, which is supported
by the National Science Foundation Cooperative Agreement No. DMR-1644779 and
the state of Florida. Some figures were made using the 3D crystal modeling
software VESTA[56].

##  Appendix A Kibble-Zurek considerations in CoNb2O6

One goal of this study was to investigate the possibility of a Kibble-Zurek
Mechanism (KZM) in CoNb2O6. By quenching a control parameter through a
continuous phase transition the system will inherently be driven out of
equilibrium, a direct result of the divergence of the relaxation time near the
critical point. This results in a non-zero density of defects which accumulate
in the system for any finite quench rate. The density of defects, and all
observables that depend on it, are predicted to scale as a power law with the
quench rate, $r_{Q}$ [6]. The density of defects, $\rho$, is predicted to
scale as,

$\rho\sim r_{Q}^{d\nu/(1+z\nu)}$ (6)

where $r_{Q}$ is the quench rate, $d$ is the dimension, $\nu$ is the
correlation length critical exponent, and $z$ is the dynamical critical
exponent. The interest in Kibble-Zurek scaling is two-fold. First, despite
being an inherently non-equilibrium effect, which are typically challenging to
describe, this phenomenon is described _entirely_ by equilibrium properties
(critical exponents). Second, the only requirement for Kibble-Zurek effects to
occur is to cross a continuous phase transition at a finite rate, with the
scaling relying only on the universality class of the system. While it was
originally formulated by Kibble for defects in the early universe[57, 58], it
was extended to condensed matter systems by Zurek, who proposed it in
superfluid helium[4]. It has since been generalized to quantum phase
transitions[5, 59, 60], and experimental evidence for the KZM has been found
in many systems including superfluid He[61], cold ion chains [62], and Bose
condensates[63, 64]. To this date, no scaling from a KZM due to a quantum
phase transition has been observed in a magnetic system, where a magnetic
field can provide a natural tuning parameter. Due to strong spin-lattice
coupling, typical relaxations in magnetic systems are fast (picoseconds).
However, the timescale for observation of defects is not limited by this spin-
lattice relaxation; rather, it is limited by the coarsening time, which relies
on the mobility of defects (to some extent set by the exchange interactions)
and the dimension. Thus, one should expect that when defect motion is
restricted, perhaps by disorder or frustration, KZ scaling could be readily
observed over experimentally achievable timescales. To accurately assess the
power law exponent, observation of the scaling over several orders of
magnitude of the quench rate is desirable. Conventional superconducting
magnets are limited to slow quench rates ( $<$ 10-1 T/min ) do not offer a
reasonable range. Alternatively, resistive magnets can reach much higher
magnetic field quench rates; the 31 T magnet at NHMFL provided us a range
between 0.1 T/min and 10 T/min. Ultra-fast field ramps can also be achieved
through pulsed magnets, which can reach quench rates of $>$ 103 T/min[65, 66].

The combination of an appropriate range of magnetic field quench rates as well
as the observed slow coarsening dynamics of CoNb2O6 suggests it would have
been a good candidate for observation of KZM. What is less obvious is what
type of scaling should observed in CoNb2O6. It is well-established that near
the QCP CoNb2O6 displays hallmark behavior of the 1D TFIM with small but non-
zero longitudinal field from the intrachain couplings[17, 27, 28]. In the case
of defects being kinks along the Ising chains, scaling could be that of the
QCP of the 1D Ising chain with an exponent of 0.5[9, 67]. However, CoNb2O6
orders into a 3D magnetically ordered ground state with much of its coarsening
behavior at $T$ = 1.5 K attributed to domain walls within the $ab$-plane [24,
25]. These defects would then perhaps be expected to scale according to the
QCP of the 3D Ising model with an exponent of 0.75[68, 69]. With a frozen
state developing below $T_{F}$ = 1.2 K, the system may be described better by
a 3D random field/bond Ising model, where dramatically increased dynamical
exponents can greatly reduce expected scaling exponents.

These predictions also are only inherently true for closed quantum systems,
which we know CoNb2O6 is _not_. Recent theoretical work for 1D Ising systems
show that coupling to a bosonic bath greatly reduces the critical exponent
down to 0.28[70]. Experimental work on quantum simulators of the 1D TFIM have
also estimated the value of the scaling exponent due to the KZM range between
0.20 and 0.33[71]. Coarsening and the KZM have been further explored[33], with
estimates that in $d$ $>$ 2, coarsening does not strongly affect the expected
critical exponents[32].

Finally, we note that the scaling of the initial ac susceptibility with quench
rate, parameterized by $a$ in Equation 2, would likely be a good indicator of
defect density. But as discussed in Appendix B.4 (and shown in Fig. B.2), we
find that $a$ does not depend on quench rate systematically; rather, it
appears to display some aging effects, though not as clearly as the $c$
parameter.

Figure A.1: (a) Real component of ac susceptibility vs linear motor rotation
angle taken under a 1 T dc field. A large peak near 108∘ shows the location of
the average Ising axis ($c$), which is identified based on the symmetric peaks
around it, which likely relate to field-induced phase transitions that are
highly dependent on the field direction. (b) Example of transverse-field
dependent susceptibility, with the background from coil shown, which was taken
at 10 K for the real component (main figure) and imaginary component (inset)
of the susceptibility. The background from the coil shows a frequency
dependence as well as transverse field dependence but little to no quench-rate
dependence at 710 Hz (not shown).

##  Appendix B Transverse Field Susceptibility

###  B.1 Fine-tuning of orientation

For conventional ac susceptibility, one measures with the ac field in the same
direction as the dc field. In contrast, for the susceptometer we used, the ac
component was measured along the crystallographic $c$-axis, while the dc field
could be applied anywhere in the $bc$-plane by an _in situ_ rotation of the ac
coilset. The samples were loaded and aligned into the coilset with use of a
Laue diffractometer, limiting a possible rotation of the sample out of the
$bc$-plane to 1∘. In order to accurately align the dc field to be transverse
to the Ising axis (i.e. $b$), we first located the $c$-axis by rotating the
coilset under a 1 T dc field through a range of angles for which the direction
of $H_{ac}$ is approximately equal to that of $H_{dc}$, and once determined,
set the coilset to 90∘ from there. Figure A.1a shows an approximately
symmetric signal around “108∘”, which identifies the coil angle of 108∘ as the
$c$-axis direction. The several peaks observed around this position are likely
indicating different field-induced transitions as a function of angle (for
fields near the Ising axis, the phase diagram is known to be highly complex:
[13, 20]). The overall background from the coil is not symmetric, but this is
likely due to details of the coil design and centering in the magnet.

Figure B.1: (a-b) Representative ac susceptibility data taken at the end of
intermediate (stepwise) quenches for various points in the phase diagram.
Color scale represents $r_{Q}$. (a) 4 T to 3 T quenches showing a small
relaxation barely larger than instrument noise limit, showing the approach to
the CAFM state near 3 T. (b) 3 T to 2 T quenches show a clear relaxation well
beyond noise limit that exhibit a decrease in signal with increasing run
number. (c) Average susceptibility over last second of measuring time
($\chi^{\prime}_{<l>}$) as function of quench run number for data in panel b.
Above the frozen transition, the system relaxes overall with each subsequent
quench. (d) Average susceptibility over last second of measuring time
(($\chi^{\prime}_{<l>}$)) as function of quench data for zero-field data
showing aging effects within frozen state. Q1 4T and Q2 4T show two separate
sets of quenches with a maximum field of 4 T, while Q1 6T shows a set of
quenches following a maximum field of 6 T.

###  B.2 Background subtraction

In general, for ac susceptometers in a transverse field, an anisotropic
background signal of the coil can develop, which can be both field-dependent
and frequency-dependent[72]. For our setup, an estimate of the coil background
was taken using a high temperature scan at 10 K, well-into the paramagnetic
phase (Fig. A.1b). Interestingly, we note that even at the maximum field of 10
T the imaginary component is still non-zero. This perhaps indicates that the
system still exhibits frustration deep into the field polarized paramagnetic
phase and could help to explain the aging effects we have observed (see next
section). This background is subtracted from all ac susceptibility figures
shown.

###  B.3 Aging effects

In an attempt to separate any possible trends coming from aging effects
compared to quench rate effects, we instituted the following protocol for our
magnetic field quenches: after the system was reset (warmed to 1.8 K then
cooled in zero-field), we ran a series of quenches from 0 T up to a maximum
field (either 6 T or 4 T) and then back down to 0 T. Each of the quench
protocols consisted of an identical field sweep rate of 1 T/min from zero up
to the maximum field, a hold at the maximum field for 10 s, and a field sweep
(quench) rate chosen to be between 0.1 T/min and 10 T/min for the quench back
to 0 T. The system was then reset again before another set of quenches by
warming above $T_{F}$. By doing the quenches “out of order” with respect to
quench rate, we are able to separate any overall aging effects (i.e. effects
that correlate with run number, which loosely represents time since cooling
into the frozen state) from effects depending on the quench rate, $r_{Q}$.

To observe at what field the relaxation begins, we measured the susceptibility
over time for many smaller intermediate magnetic field quenches (e.g. 4 T to 3
T, 3 T to 2 T etc.). A small relaxation on the order of the noise limit of our
experiment is observed in the 4 T to 3 T quenches, shown in Fig. B.1a. The
onset of appreciable relaxation occurs for quenches from 3 T to 2 T, across
the previously-identified transition between the ICAFM and CAFM ordered states
at $\sim$ 3 T (Fig. B.1b). For the final fields above the freezing transition,
relaxations show an overall aging effect such that progressive quenches are
shifted down with run number (Fig. B.1c). For final fields below the anomaly
at Bf $\sim 1.7~{}T$, which we have tentatively associated with a field
boundary of the frozen phase, the relaxation gains a more complicated
dependence on the quench rate, shown in the main paper by the zero-field
relaxation (Fig. 4a). Looking at the average susceptibility at the last second
of measurement time, shown in Fig. B.1d, we notice two different aging
behaviors in the 4 T and 6 T quenches. For the 4 T quenches, the average final
susceptibility increases with initial runs and then saturates, while the 6 T
quenches increase with consecutive quenches. Any trend that appears as a
function of run number (which was not correlated with quench rate, by design)
strongly suggests aging effects are present, and we find that these are
surprisingly _not erased_ by going to 6 T (in the paramagnetic regime). This
may be related to the presence of significant energy dissipation up to 10 T,
as indicated by the high $\chi^{\prime\prime}$ (Fig. 3) at 6 T, and suggests
that one needs to go to higher fields to reset the system.

In order to account for the aging effect, which mainly seemed to produce an
offset to $\chi^{\prime}$, we looked at the difference of the first second and
last second of the susceptibility, $\chi^{\prime}(t)$ (Eqn. 3).

###  B.4 Analysis of time-dependence of $\chi^{\prime}$

After a magnetic field quench (i.e. when the dc field returns to zero), we
observe a decay over time of the real and imaginary components of the
susceptibility at $T$ = 0.5 K, i.e. a non-equilibrium state is generated. This
decay appears for all quenches to zero field, regardless of their starting
field values (which were either 4 T or 6 T), indicating that the non-
equilibrium state is not due to quenching through the QPT associated with the
1D TFIM (at 5.2 T) or the 3D QPT to the field-polarized paramagnet (at 5.4 T).

Figure B.2: (a) Best fit parameters $a$ and $c$ for least squares fit of Eqn.
7c as a function of quench run number, shown with psuedo-random $r_{Q}$
protocol as a function of quench run number. Color overlays shown distinct
sets of quenches: red is Q1 4 T, green is Q2 4 T, and blue is Q1 6 T.
Parameters $a$ and $c$ both show weak trends in run number, suggesting aging.
Parameter $c$ also shows a general offset to the data for each individual
quench set (each set started after warming past $T_{f}$ and cooling back down
in zero field). (b) Plots of best fit parameters $a$ and $c$ for least-squares
fit of Eqn. 7c as a function of $r_{Q}$, where errors plotted are 95%
confidence intervals of the fitted parameters. Although the spread in values
is much larger than the errorbars, they show no obvious dependence on ramp
rate.

Following a field quench at 0.5 K, the time-dependence of the zero-field
relaxation within the frozen state can be fit relatively well to three
different forms: a stretched exponential, a power law decay, and a logarithmic
function given by,

$\chi^{\prime}(t,B=0)=a\cdot\exp\left[-\left(\frac{t-t_{0}}{\tau}\right)^{\beta}\right]+c$
(7a) $\chi^{\prime}(t,B=0)=a\cdot\bigg{(}\frac{t-t_{0}}{\tau}\bigg{)}^{n}+c$
(7b) $\chi^{\prime}(t,B=0)=a\cdot\ln\bigg{(}\frac{t-t_{0}}{\tau}\bigg{)}+c$
(7c)

respectively. In each of the forms, $t_{0}$ represents the time of relaxation
onset, $\tau$ represents an intrinsic time (needed in theory to make the time
arguments dimensionless, but absorbed into the other parameters for fitting
purposes), $a$ is a scale factor that one could expect to represent the
initial population of defects generated by the quench, and $c$ is an offset
due to late-time behavior and/or background. For the stretched exponential,
$\tau$ represents the average relaxation time and $\beta$ represents the
distribution of relaxation times. This form of relaxation is commonly found in
glass systems[73, 74], and has also been found for hierarchically constrained
dynamics[54]. A power law would instead indicate relaxations occurring on all
time scales with growth exponent $n$ describing the coarsening. Power law
relaxation has been observed for CoNb2O6 at higher temperature above the glass
transition where the exponent $n$ was found to be -0.2[24, 25, 47]. Meanwhile,
the logarithmic relaxation function has also been proposed for hierarchical
dynamics[48], and is often seen for domain coarsening in disordered models,
such as the random field Ising model[75, 76].

To determine the best model for the decay, we compared $\chi^{2}$ for fits of
the three models to several relaxation curves. An example of best fits for the
three functional forms are shown in Fig. B.3, where the best fit parameters
used are shown in Table 2. Note that the form of the power law and logarithmic
decay functions have absorbed $\tau$ into the definition of the other
parameters, $a$ and $c$, as appropriate. For the sake of comparison to Ref.
24, 25, we note the data can be well-represented by a power law, but with a
smaller exponent of $n=-0.097(3)$ compared to the value reported in literature
($n=-0.2$). While the $\chi^{2}$ for the three forms were comparable, in the
end we chose the logarithmic form due to the reduced number of free parameters
required to accurately fit the decay, and the very small exponent indicated by
the power law fit.

Figure B.3: Example of least squares best fits to zero-field relaxation for the following functional forms: Power law, Stretched exponential and Logarithmic decay. |  Logarithmic $a\cdot ln(t-t_{0})+c$ |  Power law $a\cdot(t-t_{0})^{n}+c$ |  Stretched Exponential $a\cdot exp\left[-(\frac{t-t_{0}}{\tau})^{\beta}\right]+c$ |
---|---|---|---|---
$a$ | -2.11(2)e-6 | 3.54(6)e-5 | 3.9(5)e-4 |
$t_{0}$ | 368.6(2) | 365.3(4) | 367(1) |
$c$ | 3.02(1)e-5 | -2.2(8)e-6 | -1.2(3)e-4 |
$n$ | - | -0.097(3) | - |
$\tau$ | - | - | 124(8) |
$\beta$ | - | - | 0.01(5) |
$\chi^{2}$ | 3.2e-6 | 2.3e-6 | 7.0e-6 |
Table 2: Table of least squares fits to typical zero-field relaxation of the
susceptibility of CoNb2O6 at $T$ = 0.5 K taken after a 2 T/min quench from a
peak field of 4 T. Best fit parameters are shown for three different forms of
the relaxation: a logarithmic relaxation, a power law relaxation, and a
stretched exponential relaxation. $\tau$ was absorbed into $a$ and $c$ for the
power law and logarithmic forms respectively in order to reduce free
parameters.

From intermediate quench data (quenches ending at a non-zero field), the
system only shows a sizeable decaying population below the CAFM transition,
BCAFM $\sim$ 3 T. Within the frozen state, best fit values of the parameters
$a$ and $c$ show a large distribution of values, but remain largely
insensitive to the quench rate of the magnet, shown in Fig. B.2.

###  B.5 Quench rate dependence of $\chi^{\prime}_{D}$

During our experiments, the magnetic field was decreased linearly at a set
quench rate from 6 T down to 0 T and from 4 T down to 0 T, as well as between
intermediate field values, as described above. The quench rate, $r_{Q}$, is
given by,

$\begin{split}r_{Q}&=\frac{dB}{dt}\\\
r_{Q}&=\frac{B_{i}-B_{f}}{t_{i}-t_{f}}\end{split}$ (8)

where the subscripts $i$ and $f$ refer to the initial and final parameters of
the quench. Note that $r_{Q}$ is defined to be positive for a decreasing
field. In order to analyze the $r_{Q}$ dependence of $\chi^{\prime}_{D}$, we
considered two analytical forms. The first is a power law, which would be
potentially consistent with KZM (Eqn 4 in the main text, repeated here),

$\chi^{\prime}_{D}(r_{Q})=a_{p}r_{Q}^{b},$ (9)

where $a_{p}$ is a scale factor depending on the overall number of defects
generated by the quenches, and $b$ is the power law exponent. The expected
values for $b$ are discussed in Appendix A.

A second analytic form accounts for the effects of domain coarsening during
the quench. This will occur for any open system where coarsening occurs on the
same approximate timescale as the quench time. We refer to the results of
coarsening during the quench as a “systematic effect”, since it does not
require any additional physics such as KZM. To derive the analytic expression
we assumed that the coarsening/relaxation takes place with an identical form
throughout the field range below the relevant transition, following Eqn. 2
(the log relaxation form). We find that this systematic effect would lead to
the following $\chi^{\prime}_{D}(r_{Q})=\langle f\rangle-\langle l\rangle$
form at zero field (Eqn. 5 presented in the main text, repeated here):

$\chi^{\prime}_{D}(r_{Q})=a\ln{\left[\frac{B_{c}r_{Q}}{t_{m}+B_{c}r_{Q}}\right]},$
(10)

where $t_{m}$ is the measurement time between the first few sections of the
measured zero-field relaxation (over which $\chi^{\prime}$ is averaged to
produce $\langle f\rangle$) and the last few seconds (over which
$\chi^{\prime}$ is averaged to produce $\langle l\rangle$) $t_{m}$ was 300 s
(5 min) for all of our data presented here. Meanwhile, $a$ is in principle the
same scale factor as in Eqn 2 (though we let it be a free parameter in our
analysis and compared to the results of fitting Eqn 2 afterwards), and $B_{c}$
is a fitting parameter describing the critical field at which relaxation
starts.

We performed a Bayesian non-linear regression analysis of one set of 4 T
quench data and one set of 6 T data. The analysis was performed using the
Markov Chain Monte Carlo package PyMC3 [77] for Python. The iPython notebook
used to complete the analysis is available in the supplemental material. We
chose to use a Bayesian approach because we encountered great difficulty in
reliably fitting Eqns. 9 and 10 using a traditional least squares (maximum
likelihood estimation) approach, and suspected it was due to highly correlated
parameters. The advantage of Bayesian regression is that it gives access to
the joint probability distribution of the model parameters, which can be
inspected directly to understand the nature of the correlations (these are
shown in Fig. B.4 for the analyses discussed here). Furthermore, Bayesian
analysis allows for a more complete understanding of the uncertainties of the
fitted models. These are indicated by the 95% credible intervals of the mean
of the posterior distributions, shown as shaded colored areas in Fig. 4 panels
c) and d). For our prior distributions, we used uniform distributions for most
parameters, with limits as indicated in Table 3, except for the random error
parameter (which accounts for the standard deviation of the data points away
from the mean value predicted by the model), for which we used half-normal
distribution.

| 4 T | 6 T
---|---|---
log |  $a$: Unif(0, $5\times 10^{-6}$) |  $a$: Unif(0, $5\times 10^{-6}$)
$B_{c}$: Unif(0, 3) |  $B_{c}$: Unif(0, 3)
$\epsilon$: HalfNormal($\sigma=1\times 10^{-5}$) |  $\epsilon$: HalfNormal($\sigma=1\times 10^{-5}$)
power |  $a$: Unif(0, $40\times 10^{-6}$) |  $a$: Unif(0, $40\times 10^{-6}$)
$b$: Unif(0, 2) |  $b$: Unif(0, 2)
$\epsilon$: HalfNormal($\sigma=5$) |  $\epsilon$: HalfNormal($\sigma=5$)
Table 3: Prior distributions for the model parameters used in the Bayesian
regression. $\epsilon$ is the standard deviation associated with the normal
distribution of the data away from the mean (best-fit line). We chose a half-
normal distribution centered at 0 since the standard deviation cannot be
negative. Figure B.4: Joint distributions of the parameters for the two models
fit to the $\chi^{\prime}_{D}(r_{Q})$ data.

## References

  * Pal and Huse [2010] A. Pal and D. A. Huse, Many-body localization phase transition, Physical review b 82, 174411 (2010).
  * Nandkishore and Huse [2015] R. Nandkishore and D. A. Huse, Many-body localization and thermalization in quantum statistical mechanics, Annu. Rev. Condens. Matter Phys. 6, 15 (2015).
  * Friesdorf  _et al._ [2015] M. Friesdorf, A. H. Werner, W. Brown, V. B. Scholz, and J. Eisert, Many-body localization implies that eigenvectors are matrix-product states, Phys. Rev. Lett. 114, 170505 (2015).
  * Zurek [1985] W. H. Zurek, Cosmological experiments in superfluid helium?, Nature 317, 505 (1985).
  * Dziarmaga [2005] J. Dziarmaga, Dynamics of a quantum phase transition: Exact solution of the quantum ising model, Physical review letters 95, 245701 (2005).
  * Chandran  _et al._ [2012] A. Chandran, A. Erez, S. S. Gubser, and S. L. Sondhi, Kibble-zurek problem: Universality and the scaling limit, Physical Review B 86, 064304 (2012).
  * Pfeuty [1970] P. Pfeuty, The one-dimensional ising model with a transverse field, ANNALS of Physics 57, 79 (1970).
  * Elliott  _et al._ [1970] R. Elliott, P. Pfeuty, and C. Wood, Ising model with a transverse field, Physical Review Letters 25, 443 (1970).
  * Dutta  _et al._ [2015] A. Dutta, G. Aeppli, B. K. Chakrabarti, U. Divakaran, T. F. Rosenbaum, and D. Sen,  _Quantum phase transitions in transverse field spin models: from statistical physics to quantum information_ (Cambridge University Press, 2015).
  * Maartense  _et al._ [1977] I. Maartense, I. Yaeger, and B. Wanklyn, Field-induced magnetic transitions of conb2o6 in the ordered state, Solid State Communications 21, 93 (1977).
  * Scharf  _et al._ [1979] W. Scharf, H. Weitzel, I. Yaeger, I. Maartense, and B. Wanklyn, Magnetic structures of conb2o6, Journal of Magnetism and Magnetic Materials 13, 121 (1979).
  * Hanawa  _et al._ [1992] T. Hanawa, M. Ishikawa, and K. Miyatani, Disappearance of ferromagnetism at low temperatures in conb2o6, Journal of the Physical Society of Japan 61, 4287 (1992).
  * Hanawa  _et al._ [1994] T. Hanawa, K. Shinkawa, M. Ishikawa, K. Miyatani, K. Saito, and K. Kohn, Anisotropic specific heat of conb2o6 in magnetic fields, Journal of the Physical Society of Japan 63, 2706 (1994).
  * Heid  _et al._ [1995] C. Heid, H. Weitzel, P. Burlet, M. Bonnet, W. Gonschorek, T. Vogt, J. Norwig, and H. Fuess, Magnetic phase diagram of conb2o6: A neutron diffraction study, Journal of magnetism and magnetic materials 151, 123 (1995).
  * Kobayashi  _et al._ [1995] S. Kobayashi, S. Mitsuda, K. Hosoya, H. Yoshizawa, T. Hanawa, M. Ishikawa, K. Miyatani, K. Saito, and K. Kohn, Competition between the inter-chain interaction and single-ion anisotropy in conb2o6, Physica B: Condensed Matter 213, 176 (1995).
  * Heid  _et al._ [1997] C. Heid, H. Weitzel, P. Burlet, M. Winkelmann, H. Ehrenberg, and H. Fuess, Magnetic phase diagrams of conb2o6, Physica B: Condensed Matter 234, 574 (1997).
  * Coldea  _et al._ [2010] R. Coldea, D. Tennant, E. Wheeler, E. Wawrzynska, D. Prabhakaran, M. Telling, K. Habicht, P. Smeibidl, and K. Kiefer, Quantum criticality in an ising chain: experimental evidence for emergent e8 symmetry, Science 327, 177 (2010).
  * Weitzel [1976] H. Weitzel, Kristallstrukturverfeinerung von wolframiten und columbiten, Zeitschrift für Kristallographie-Crystalline Materials 144, 238 (1976).
  * Mitsuda  _et al._ [1994] S. Mitsuda, K. Hosoya, T. Wada, H. Yoshizawa, T. Hanawa, M. Ishikawa, K. Miyatani, K. Saito, and K. Kohn, Magnetic ordering in one-dimensional system conb2o6 with competing interchain interactions, Journal of the Physical Society of Japan 63, 3568 (1994).
  * Kobayashi  _et al._ [1999a] S. Kobayashi, S. Mitsuda, M. Ishikawa, K. Miyatani, and K. Kohn, Three-dimensional magnetic ordering in the quasi-one-dimensional ising magnet conb 2 o 6 with partially released geometrical frustration, Physical Review B 60, 3331 (1999a).
  * Lee  _et al._ [2010] S. Lee, R. K. Kaul, and L. Balents, Interplay of quantum criticality and geometric frustration in columbite, Nature Physics 6, 702 (2010).
  * da Silva Wheeler [2007] E. M. da Silva Wheeler,  _Neutron scattering from low-dimensional quantum magnets_ , Ph.D. thesis, University of Oxford (2007).
  * Liang  _et al._ [2015] T. Liang, S. Koohpayeh, J. Krizan, T. McQueen, R. J. Cava, and N. P. Ong, Heat capacity peak at the quantum critical point of the transverse ising magnet conb 2 o 6, Nature communications 6, 1 (2015).
  * Kobayashi  _et al._ [1999b] S. Kobayashi, S. Mitsuda, T. Jogetsu, J. Miyamoto, H. Katagiri, and K. Kohn, Anisotropic growth kinetics in the geometrically frustrated isosceles triangular ising antiferromagnet conb 2 o 6, Physical Review B 60, R9908 (1999b).
  * Kobayashi  _et al._ [2004] S. Kobayashi, H. Okano, T. Jogetsu, J. Miyamoto, and S. Mitsuda, Domain growth kinetics in the isosceles triangular ising antiferromagnet conb 2 o 6, Physical Review B 69, 144430 (2004).
  * Zamolodchikov [1989] A. B. Zamolodchikov, Integrals of motion and s-matrix of the (scaled) t= t c ising model with magnetic field, International Journal of Modern Physics A 4, 4235 (1989).
  * Rutkevich [2010] S. Rutkevich, On the weak confinement of kinks in the one-dimensional quantum ferromagnet conb2o6, Journal of Statistical Mechanics: Theory and Experiment 2010, P07015 (2010).
  * Kjäll  _et al._ [2011] J. A. Kjäll, F. Pollmann, and J. E. Moore, Bound states and e 8 symmetry effects in perturbed quantum ising chains, Physical Review B 83, 020407 (2011).
  * Kinross  _et al._ [2014] A. Kinross, M. Fu, T. Munsie, H. Dabkowska, G. Luke, S. Sachdev, and T. Imai, Evolution of quantum fluctuations near the quantum critical point of the transverse field ising chain system conb 2 o 6, Physical Review X 4, 031008 (2014).
  * Fava  _et al._ [2020] M. Fava, R. Coldea, and S. Parameswaran, Glide symmetry breaking and ising criticality in the quasi-1d magnet conb $\\_2$ o $\\_6$, arXiv preprint arXiv:2004.04169 (2020).
  * Morris  _et al._ [2020] C. Morris, N. Desai, J. Viirok, D. Huvonen, U. Nagel, T. Room, J. Krizan, R. Cava, T. McQueen, S. Koohpayeh, _et al._ , Duality and domain wall dynamics in a twisted kitaev chain, arXiv preprint arXiv:2009.14189 (2020).
  * Biroli  _et al._ [2010] G. Biroli, L. F. Cugliandolo, and A. Sicilia, Kibble-zurek mechanism and infinitely slow annealing through critical points, Physical Review E 81, 050101 (2010).
  * Biroli [2015] G. Biroli, Slow relaxations and non-equilibrium dynamics in classical and quantum systems, arXiv preprint arXiv:1507.05858 (2015).
  * Liu  _et al._ [2015] C.-W. Liu, A. Polkovnikov, A. W. Sandvik, and A. Young, Universal dynamic scaling in three-dimensional ising spin glasses, Physical Review E 92, 022128 (2015).
  * Xu  _et al._ [2017] N. Xu, K.-H. Wu, S. J. Rubin, Y.-J. Kao, and A. W. Sandvik, Dynamic scaling in the two-dimensional ising spin glass with normal-distributed couplings, Physical Review E 96, 052102 (2017).
  * Wanklyn  _et al._ [1976] B. Wanklyn, B. Garrard, and G. Garton, Flux growth of crystals of mnb2o6 (m= ni, co, mn, fe), Materials Research Bulletin 11, 1497 (1976).
  * NHM [a] 31 tesla, 50 mm bore magnet (cell 9) - maglab, %****␣CoNbO_v4.bbl␣Line␣500␣****https://nationalmaglab.org/user-facilities/dc-field/instruments-dcfield/resistive-magnets/31-tesla-cell-9 (a).
  * Lashley  _et al._ [2003] J. Lashley, M. Hundley, A. Migliori, J. Sarrao, P. Pagliuso, T. Darling, M. Jaime, J. Cooley, W. Hults, L. Morales, _et al._ , Critical examination of heat capacity measurements made on a quantum design physical property measurement system, Cryogenics 43, 369 (2003).
  * Suzuki  _et al._ [2010] H. Suzuki, A. Inaba, and C. Meingast, Accurate heat capacity data at phase transitions from relaxation calorimetry, Cryogenics 50, 693 (2010).
  * Lifshitz [1962] I. Lifshitz, Kinetics of ordering during second-order phase transitions, Sov. Phys. JETP 15, 939 (1962).
  * Chowdhury and Kumar [1987] D. Chowdhury and S. Kumar, Domain growth in the three-dimensional dilute ising model, Journal of statistical physics 49, 855 (1987).
  * Cheon and Chang [2001] M. Cheon and I. Chang, Anisotropic domain growth of the axial next-nearest-neighbor ising model at low temperatures, Physical review letters 86, 4576 (2001).
  * Burdese  _et al._ [1964] A. Burdese, M. Borlera, and P. Rolando, Systems between niobium oxides and the oxides of nickel and cobalt, Atti Accad. Sci. Torino: I. Classe Sci. Fis., Mat. Nat. 99 (1964).
  * Maier [1993] J. Maier, Defect chemistry: composition, transport, and reactions in the solid state; part i: thermodynamics, Angewandte Chemie International Edition in English 32, 313 (1993).
  * Note [1] Care must also be taken to convert cobalt oxide precursors to a single type. When purchased from chemical suppliers they often contain a mixture of Co3O4 and CoO which, if uncorrected, can lead to inaccurate Co stoichiometry.
  * Sarvezuk  _et al._ [2011] P. W. C. Sarvezuk, E. J. Kinast, C. Colin, M. Gusmao, J. Da Cunha, and O. Isnard, Suppression of magnetic ordering in quasi-one-dimensional fe x co 1- x nb 2 o 6 compounds, Physical Review B 83, 174412 (2011).
  * Nakajima  _et al._ [2014] T. Nakajima, S. Mitsuda, Y. Inomoto, K. Prokes, V. Sikolenko, S. Gerischer, and S. Kobayashi, Magnetic domain growth in geometrically frustrated ising antiferromagnets co 1- x mg x nb 2 o 6 (x= 0 and 0.004) as seen via time-resolved neutron diffraction measurements, Physical Review B 90, 064431 (2014).
  * Brey and Prados [2001] J. Brey and A. Prados, Slow logarithmic relaxation in models with hierarchically constrained dynamics, Physical Review E 63, 021108 (2001).
  * Huse and Henley [1985] D. A. Huse and C. L. Henley, Pinning and roughening of domain walls in ising systems due to random impurities, Physical review letters 54, 2708 (1985).
  * Chowdhury  _et al._ [1987] D. Chowdhury, M. Grant, and J. Gunton, Interface roughening and domain growth in the dilute ising model, Physical Review B 35, 6792 (1987).
  * McKay  _et al._ [1982] S. R. McKay, A. N. Berker, and S. Kirkpatrick, Amorphously packed, frustrated hierarchical models: Chaotic rescaling and spin-glass behavior, Journal of Applied Physics 53, 7974 (1982).
  * Hed  _et al._ [2001] G. Hed, A. K. Hartmann, D. Stauffer, and E. Domany, Spin domains generate hierarchical ground state structure in j=$\pm$1 spin glasses, Physical Review Letters 86, 3148 (2001).
  * Castellana  _et al._ [2010] M. Castellana, A. Decelle, S. Franz, M. Mézard, and G. Parisi, Hierarchical random energy model of a spin glass, Physical review letters 104, 127206 (2010).
  * Palmer  _et al._ [1984] R. G. Palmer, D. L. Stein, E. Abrahams, and P. W. Anderson, Models of hierarchically constrained dynamics for glassy relaxation, Physical Review Letters 53, 958 (1984).
  * Munoz  _et al._ [1998] M. Munoz, A. Gabrielli, H. Inaoka, and L. Pietronero, Hierarchical model of slow constrained dynamics, Physical Review E 57, 4354 (1998).
  * Momma and Izumi [2008] K. Momma and F. Izumi, Vesta: a three-dimensional visualization system for electronic and structural analysis, Journal of Applied Crystallography 41, 653 (2008).
  * Kibble [1976] T. W. Kibble, Topology of cosmic domains and strings, Journal of Physics A: Mathematical and General 9, 1387 (1976).
  * Kibble [1980] T. W. Kibble, Some implications of a cosmological phase transition, Physics Reports 67, 183 (1980).
  * Zurek  _et al._ [2005] W. H. Zurek, U. Dorner, and P. Zoller, Dynamics of a quantum phase transition, Physical review letters 95, 105701 (2005).
  * Polkovnikov [2005] A. Polkovnikov, Universal adiabatic dynamics in the vicinity of a quantum critical point, Physical Review B 72, 161201 (2005).
  * Hendry  _et al._ [1994] P. Hendry, N. S. Lawson, R. Lee, P. V. McClintock, and C. Williams, Generation of defects in superfluid 4he as an analogue of the formation of cosmic strings, Nature 368, 315 (1994).
  * Ulm  _et al._ [2013] S. Ulm, J. Roßnagel, G. Jacob, C. Degünther, S. Dawkins, U. Poschinger, R. Nigmatullin, A. Retzker, M. Plenio, F. Schmidt-Kaler, _et al._ , Observation of the kibble–zurek scaling law for defect formation in ion crystals, Nature communications 4, 1 (2013).
  * Lamporesi  _et al._ [2013] G. Lamporesi, S. Donadello, S. Serafini, F. Dalfovo, and G. Ferrari, Spontaneous creation of kibble–zurek solitons in a bose–einstein condensate, Nature Physics 9, 656 (2013).
  * Anquez  _et al._ [2016] M. Anquez, B. Robbins, H. Bharath, M. Boguslawski, T. Hoang, and M. Chapman, Quantum kibble-zurek mechanism in a spin-1 bose-einstein condensate, Physical review letters 116, 155301 (2016).
  * Campbell  _et al._ [1996] L. Campbell, H. Boenig, D. Rickel, J. Schillig, H. Schneider-Muntau, and J. Sims, The nhmfl long-pulse magnet system- 60–100 t, Physica B: Condensed Matter 216, 218 (1996).
  * NHM [b] 60 tesla, controlled waveform magnet, https://nationalmaglab.org/user-facilities/pulsed-field-facility/instruments-pff/60-tesla-controlled-waveform-magnet (b).
  * Mukherjee  _et al._ [2007] V. Mukherjee, U. Divakaran, A. Dutta, and D. Sen, Quenching dynamics of a quantum x y spin-1 2 chain in a transverse field, Physical Review B 76, 174303 (2007).
  * Zheng and Zhang [1998] G. Zheng and J. Zhang, Determination of dynamical critical exponents from hysteresis scaling, Physical Review E 58, R1187 (1998).
  * Blöte and Swendsen [1980] H. Blöte and R. Swendsen, Critical behavior of the four-dimensional ising model, Physical Review B 22, 4481 (1980).
  * Oshiyama  _et al._ [2020] H. Oshiyama, N. Shibata, and S. Suzuki, Kibble-zurek mechanism in a dissipative transverse ising chain, arXiv preprint arXiv:2005.05621 (2020).
  * Bando  _et al._ [2020] Y. Bando, Y. Susa, H. Oshiyama, N. Shibata, M. Ohzeki, F. J. Gómez-Ruiz, D. A. Lidar, S. Suzuki, A. del Campo, and H. Nishimori, Probing the universality of topological defect formation in a quantum annealer: Kibble-zurek mechanism and beyond, Physical Review Research 2, 033369 (2020).
  * Rucker and Pfleiderer [2019] F. Rucker and C. Pfleiderer, Compact susceptometer for studies under transverse field geometries at very low temperatures, Review of Scientific Instruments 90, 073903 (2019).
  * Mauro and Mauro [2018] J. C. Mauro and Y. Z. Mauro, On the prony series representation of stretched exponential relaxation, Physica A: Statistical Mechanics and its Applications 506, 75 (2018).
  * Potuzak  _et al._ [2011] M. Potuzak, R. C. Welch, and J. C. Mauro, Topological origin of stretched exponential relaxation in glass, The Journal of chemical physics 135, 214502 (2011).
  * Fisher  _et al._ [2001] D. S. Fisher, P. Le Doussal, and C. Monthus, Nonequilibrium dynamics of random field ising spin chains: Exact results via real space renormalization group, Physical Review E 64, 066107 (2001).
  * Corberi [2015] F. Corberi, Coarsening in inhomogeneous systems, Comptes Rendus Physique 16, 332 (2015).
  * pym [2021] Pymc3 probabilistic programming in python (2021).


# END-TO-END ARCHITECTURES FOR ASR-FREE SPOKEN LANGUAGE UNDERSTANDING

###### Abstract

Spoken Language Understanding (SLU) is the problem of extracting the meaning
from speech utterances. It is typically addressed as a two-step problem, where
an Automatic Speech Recognition (ASR) model is employed to convert speech into
text, followed by a Natural Language Understanding (NLU) model to extract
meaning from the decoded text. Recently, end-to-end approaches were emerged,
aiming at unifying the ASR and NLU into a single SLU deep neural architecture,
trained using combinations of ASR and NLU-level recognition units. In this
paper, we explore a set of recurrent architectures for intent classification,
tailored to the recently introduced Fluent Speech Commands (FSC) dataset,
where intents are formed as combinations of three slots (action, object, and
location). We show that by combining deep recurrent architectures with
standard data augmentation, state-of-the-art results can be attained, without
using ASR-level targets or pretrained ASR models. We also investigate its
generalizability to new wordings, and we show that the model can perform
reasonably well on wordings unseen during training.

Index Terms—  spoken language understanding, end-to-end models, recurrent
neural networks, intent classification

##  1 Introduction

Over the past decades voice interfaces have become an integral part of daily
life communication for a wide range of domains like banking, entertainment or
travelling making the development of Spoken Language Understanding (SLU)
systems necessity.

Typical SLU systems employ an ASR module for decoding speech into text and NLU
for estimating the meaning of the utterance, in the form of domain
classification, intent classification, and slot filling [1, 2]. The problem
with these approaches is that the ASR errors are propagated to the next
components of the SLU pipeline. A way to address this problem is by replacing
the single decoding (1-best hypothesis) with word lattices or word confusion
networks [3]. Another approach is to utilize word embeddings that model
acoustic relationships between words in addition to the semantic and syntactic
relations [4]. An alternative approach is to employ end-to-end architectures
capable of learning how to map sequences of acoustic features directly to SLU
recognition units [5, 6, 7, 8]. SLU units that are typically used are
combinations of ASR-level units (e.g. phonemes, characters, word-pieces,
words) with NLU-level units (e.g. intents, slots) [9, 10]. Two-step training
approaches have also been proposed, where the network is pretrained on large
datasets using ASR-level recognition units, and it is subsequently finetuned
on the target dataset using NLU-level recognition units [7, 11].

Intent classification is a crucial task for establishing a successful
communication between the systems and the endpoint users [12]. A system that
detects intents correctly is able to understand the purposes of the users and
interact in the most appropriate way. Most systems handle intents as simple
classes, but looking a step further, intents can be handled as structure or
their names can be something more that labels, e.g., they can convey
semantics. The NLU model of [13] aims to encapsulate the hierarchical
relationship among word, slot and intent, while the joint training on slot
filling and intent classification has been proven to invigorate the NLU
components [13, 2, 14].

In this work, we propose an end-to-end ASR-free architecture for intent
classification, in the sense that it does not make any use of ASR-level
recognition units (e.g. phonemes, characters, words) during training or
evaluation. The architecture handles the intent as a structure of slots, by
combining the predictions of each spot. We also conduct a series of
experiments in order to investigate our architecture’s degree of
generalizability to unseen wordings.

##  2 SLU Architecture

###  2.1 Modeling assumptions

The proposed SLU architecture is trained to predict intents directly from
acoustic features. Rather than considering intents as classes, it handles them
as tuples of slots, each having an associated SoftMax layer. The intent tuple
depends on the dataset, and in the case of [11] a three-slot tuple is defined
by action, object and location. Following such an approach, the single-label
classification task (intent) with a large number of classes is translated to a
multi-class classification task (slots) with reduced number of classes. An
intent is predicted correctly, if all three slots have been predicted
correctly. One way of defining the intent probability is as follows:

$p(A,O,L|D)=p(A|D)\cdot p(O|D)\cdot p(L|D),$ (1)

where $A$, $O$, $L$ stand for action, object and location slots respectively,
and $D$ for the sequence of acoustic features of the utterance. Note that Eq.
(1) assumes conditional independence between slots given $D$. An alternative
way of defining the intent probability -and more consistent with probability
theory- is by assuming conditional dependence between slots as shown below:

$p(A,O,L|D)=p(A|D)\cdot p(O|A,D)\cdot p(L|A,O,D).$ (2)

Note that any ordering of $\\{A,O,L\\}$ is valid. The conditional classifier
consists of one independent and two dependent slots. The independent slot,
similarly with a conditional independent approach, is predicted given $D$. On
the other hand, the dependent slots are predicted given a composite
representation that contains not only $D$, but information from the other
slot(s) as well.

###  2.2 Training and inference

The overall loss function is the summation of the three slot-specific cross
entropy losses. Training the unconditional model is straightforward, since the
model assumes conditional independence. During inference, constraints can be
posed to ignore tuples that do not form a valid intent. On the other hand,
training the conditional model can be performed using teacher-forcing or
scheduled sampling, that are commonly employed for training sequence-to-
sequence models [15]. During inference, one may consider Beam Search in order
to maximize the $p(A,O,L|D)$, although in our experiments with FSC dataset we
observed no significant gains compared to greedy optimization.

###  2.3 Architectural details

Fig. 1 illustrates an overview of the end-to-end Recurrent Neural Network
(RNN) based proposed architecture. It is comprised of three parts: (1) RNN-
stack, (2) Representation-Layer and (3) Conditional classifier.

####  2.3.1 RNN stack

The RNN-stack may consists of $N$ Bidirectional or Unidirectional RNN layers.
Stacking multiple RNN layers makes the architecture deeper and enables the
model to learn more complex input patterns. Connections between stack’s layers
may be a) sequential i.e., the output of the layer $L_{n}$ is the input to the
layer $L_{n+1}$ etc., or b) residual i.e., the input to the layer $L_{n+1}$ is
the summation of the output of $L_{n}$ and $L_{n-1}$ layers. The output of the
stack is fed to the Representation-Layer.

####  2.3.2 Representation layer

The representation layer is responsible for squeezing the temporal information
to enable sequence classification. It contains another RNN layer followed by
average pooling. Apart from a single RNN, it may contain a triple RNN
structure. The latter consists of one RNN for each slot followed by average
pooling. It aims at extracting three slot-specific representations, as opposed
to a single representation for all three SoftMax classifiers.

####  2.3.3 Classifier

In the unconditional model, the three slots are predicted given the
representation layer’s output representation $r$. In the conditional model,
assuming teacher-forcing training, the $object$ slot $s_{o}$ is predicted
given $r$ and the $action$ ground-truth slot $s_{a}$. The input to the
$location$ slot’s classifier is formulated in a similar manner, given $r$ and
ground-truth $s_{a}$, $s_{o}$. Conditioning e.g. on $s_{a}$ is implemented by
concatenating $r$ with the column of the linear layer corresponding to the
ground-truth label of $action$. When scheduled sampling is employed,
conditioning is implemented by alternating between ground-truth and estimated
slots [15]. When triple-RNN is employed, the representation layer extracts
three slot-specific representations, i.e. $r_{a}$, $r_{o}$ and $r_{l}$.
Finally, the predicted intent is formulated by combining the predictions of
action, object and location.

Fig. 1: End-to-end SLU architecture

##  3 Experiments

###  3.1 Dataset and data augmentation

The proposed models are trained from scratch and evaluated on the Fluent
Speech Commands (FSC) dataset [11]. In order to avoid overfitting and to
increase models’ robustness, we train our architectures using five times the
original train set, which we generate using Kaldi’s data augmentation [16].
Data augmentation works as a regularization mechanism preventing the model
from overfitting. The augmented training set is generated by applying four
different data augmentation techniques: Reverberation, Music, Babble and Noise
injection, where the noises are extracted from the MUSAN corpus [17]. After
applying the data augmentation, the number of training utterances increases
from 23132 to 115660. In Table 1 the FSC dataset statistics are summarized.

Some examples of wordings and their intents are depicted in Table 2. If it is
not obvious that a slot is associated to specific words, the slot value is set
to $none$. Sometimes the slots can be inferred by the full context of the
wording, e.g., the $action$ of “ _Too loud_ ” wording is $decrease$. Some
slots, like $heat$ can be expressed by semantically similar words like _heat_
and _temperature_. In such cases, it is hard to understand that both words
refer to the same slot without pretrained word embeddings. The same holds for
phonetically similar words, i.e., “ _on, off_ ”.

Train: (Utterances, Speakers) | (115660, 77)
---|---
Validation: (Utterances, Speakers) | (3118, 10)
Test: (Utterances, Speakers) | (3793, 10)
Unique Intents | 31
Unique: (Actions, Objects, Locations) | (6, 14, 4)
Table 1: FSC dataset statistics. Bathroom$location$: washroom heat$object$:
heat up$action$: increase
---
Bathroom$location$: washroom heat$object$: heat down$action$: decrease
Decrease$action$: decrease the heating$object$: heat in the kitchen$location$:
kitchen
Increase$action$: increase the temperature$object$: heat in the
bedroom$location$: bedroom
Bring$action$: bring me the newspaper$object$: newspaper $location$: none
Lights$object$: lights on$action$: activate $location$: none
Turn the lamp$object$: lamp off$action$: decrease $location$: none
Switch language$action$: change language $object$: none $location$: none
Set my device$action$: change language to Chinese$object$: Chinese $location$:
none
Too loud$action$: decrease $object$: volume $location$: none
Table 2: Examples of wordings and their intents.

###  3.2 Architecture configuration

The architecture shown in Fig. 1 can generate multiple models by configuring
each layer. In the simplest case, RNN-stack consists of one layer, the
representation layer is skipped, and the classifier consists of three SoftMax
layers. In a more complex model the RNN-stack can consist of three layers,
representation layer contains a triple-LSTM and the classifier is conditional.

The models are trained for about 15 epochs on the FSC augmented training
dataset and evaluated on the validation set in order to select the best model.
The final evaluation, that is reported on the results occurs on the unseen
data (test). We experimented with both LSTM-stack and GRU-stack, and with both
sequential or residual connections. The representation layer can be a single
LSTM, a single GRU or a triple LSTM. Bidirectional RNN with hidden size 512
were utilized. The initial learning rate is set to 0.001 and the scheduler
halves it for the first time at the seventh epoch and then at every two
epochs. A dropout layer is applied at the output of the representation layer
and additional dropout layers are applied before the input to the classifiers.
Moreover, batch normalization and average pooling on the representation
layer’s output were also utilized. The conditional models are trained with
scheduled sampling [15]. The probability of selecting ground-truth labels
starts from 1.0 and decreases with the number of epoch using a sigmoid-shaped
function until it reaches 0.5.

We use 40-dimensional MFCCs as acoustic features, extracted every 10ms.
Cepstral mean normalization is applied, which statistics estimated from the
whole utterance.

###  3.3 Generalizing to new wordings

Aiming to investigate how our architecture responds to test data with unseen
wordings we conduct a series of experiments where we remove from the training
data a set of wordings and then we evaluate the model only to wordings unseen
during training. We create three training set versions by randomly choosing
and removing from the training dataset 20 or 50 out of the 248 unique
wordings. We also set up an extreme case, in which we keep in the training set
only the most frequent wording. In every case, we make sure that the unique
intents set do not change. In the Table 3 we list the new sizes of the train
and the test datasets.

Unique wordings |  Train size | Test utterances | Error (%)
---|---|---|---
Unseen | Seen | Unseen | Seen
248 | 23132 | - | 3793 | - | 1.49
228 | 21272 | 306 | 3487 | 15.25 | 1.81
198 | 18577 | 745 | 3048 | 22.45 | 1.59
31 | 3530 | 3228 | 565 | 57.08 | 2.13
Table 3: Train and test dataset sizes used for generalization. The first line
corresponds to the original dataset.

Each reduced train dataset is accompanied with two test sets, one that
contains only the unseen wordings and its complementary set, containing those
seen during training.

###  3.4 Experimental Results

For each architecture, the best epoch is selected based on the performance on
the validation set and the intent classification error achieved on the test
set is reported in the table 4. The results have been derived by training on
the augmented dataset.

RNN-stack | Repr. Layer | Conditional | Error (%)
---|---|---|---
1-LSTM | ✗ | ✗ | 9.38
2-LSTM | triple-LSTM | ✗ | 2.48
3-LSTM | LSTM | ✗ | 1.34
✓ | 1.15
triple-LSTM | ✗ | 1.76
✓ | 1.49
GRU | ✗ | 1.48
| ✓ | 1.28
3-GRU | triple-LSTM | ✗ | 2.54
State-of-the-art without pretraining [11] | 3.40
State-of-the-art with pretraining [11] | 1.20
Table 4: Intent classification error per model.

Examining the results in the Table 4 we observe that all models employing a
3-LSTM stack attain performance comparable to the state-of-the-art. As
presented in [11], the state-of-the-art without for the FSC database is
$3.40\%$ and with pretraining (on LibriSpeech) is $1.20\%$. The best
performing model has triple-LSTM in the representation layer and a conditional
classifier. Apart from the results reported here, other configurations such as
4-LSTM stack, residual RNN-stack connections, unidirectional RNNs, larger and
smaller RNN sizes than 512 were also investigated, without however yielding
results of any statistically significant differences. As regards the
conditional model, the best conditioning order of slots found to be
$action$-$object$-$location$, but with minor differences compared to
$object$-$action$-$location$. We also investigated the impact of the data
augmentation by training models both on clean and augmented dataset. Note that
the best performing model without data augmentation attained classification
error equal to 4.64%.

###  3.5 Comparison with ASR-driven SLU system

The performance of ASR-driven SLU systems was also investigated in [18] and
the results are summarized in table 5. The ASR model is a state-of-the-art
Kaldi recipe (8-layer TDNNs, LF-MMI training [16, 19]). The architecture is
trained on 1832 hours of publicly available datasets (Librispeech, WSJ, a.o.),
while FSC is not included in the training set. The model attains 9.44% WER on
the FSC test set. The ASR-driven SLU models are trained either on the 1-best
or on the N-best hypotheses (where N=10), employing transformer-based
architectures such as BERT, RoBERTa and DistilBERT [20, 21, 22]. N-best
trained models outperform the corresponding 1-best and achieve state-of-the-
art performance, slightly inferior to our end-to-end conditional LSTM model.

###  3.6 Results on unseen wordings and discussion

In Table 3 we summarize the results on wordings unseen during training and the
corresponding statistics on training and test sets. The error rates indicate
that the proposed architecture is to some extent capable of generalizing to
new wordings and attaining decent performance, without further tuning. As
expected, error rates on the unseen test increase with the number of wordings
removed from the training set. We consider this as an inherent limitation of
end-to-end ASR-free SLU approaches. State-of-the-art NLU approaches, based
either on word embeddings or BERT-like architectures are trained on massive
textual corpora and hence they are capable of generalizing easily to new
wordings [20, 23]. Contrarily, our end-to-end SLU method is trained from
scratch on FSC, and has no obvious mechanism for incorporating e.g. word
embeddings. As a result, it can rely only on training sets containing several
wordings per slot or intent for attaining state-of-the-art generalizability.

Model | 1-best (%) | N-best (%)
---|---|---
BERT (base) | 3.69 | 3.62
RoBERTa (base) | 3.87 | 3.34
DistilBERT | 1.64 | 1.44
Table 5: Intent classification error per ASR hypothesis [18].

##  4 Conclusions and Future work

In this work we investigated end-to-end RNN-based SLU architectures for intent
classification, without the need of any ASR supervision. We explored several
architectural variations and experimented with the use of conditional
prediction of the slots composing the intents. Additionally, we demonstrated
that data augmentation is compulsory for attaining state-of-the-art
performance and training deep architectures. We also investigated the
generalizability of the proposed model to wordings unseen during training, and
found that it can attain fairly good results on such wordings.

In the future we plan to train and evaluate the architecture on other SLU
datasets, enhance the architecture with attention mechanism, and replace the
recurrent layers with convolutional or attentive layers.

## References

  * [1] G. Tur and R. De Mori,  Spoken language understanding: Systems for extracting semantic information from speech,  John Wiley & Sons, 2011.
  * [2] A. Bapna, G. Tur, D. Hakkani-Tur, and L. Heck,  “Sequential dialogue context modeling for spoken language understanding,”  in Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, 2017, pp. 103–114.
  * [3] D. Hakkani-Tür, F. Béchet, G. Riccardi, and G. Tur,  “Beyond ASR 1-best: Using word confusion networks in spoken language understanding,”  Computer Speech & Language, vol. 20, no. 4, pp. 495–514, 2006\.
  * [4] P.G. Shivakumar, M. Yang, and P. Georgiou,  “Spoken language intent detection using confusion2vec,”  arXiv preprint arXiv:1904.03576, 2019.
  * [5] Y.-T. Wang, J. Lee, and A. Acero,  “Speech utterance classification model training without manual transcriptions,”  in Proc. of ICASSP. IEEE, 2006, pp. 553–556.
  * [6] Y. Qian, R. Ubale, V. Ramanaryanan, P. Lange, D. Suendermann-Oeft, K. Evanini, and E.e Tsuprun,  “Exploring asr-free end-to-end modeling to improve spoken language understanding in a cloud-based dialog system,”  in Proc. of ASRU. IEEE, 2017, pp. 569–576.
  * [7] Y.-P. Chen, R. Price, and S.s Bangalore,  “Spoken language understanding without speech recognition,”  in Proc. of ICASSP. IEEE, 2018, pp. 6189–6193.
  * [8] P. Haghani, A. Narayanan, M. Bacchiani, G. Chuang, N. Gaur, P. Moreno, R. Prabhavalkar, Z. Qu, and A. Waters,  “From audio to semantics: Approaches to end-to-end spoken language understanding,”  in Proc. of SLT. IEEE, 2018, pp. 720–726.
  * [9] D. Serdyuk, Y. Wang, C. Fuegen, A. Kumar, B. Liu, and Y. Bengio,  “Towards end-to-end spoken language understanding,”  in Proc. of ICASSP. IEEE, 2018, pp. 5754–5758.
  * [10] N. Tomashenko, A. Caubrière, and Y. Estève,  “Investigating adaptation and transfer learning for end-to-end spoken language understanding from speech,”  in Proc. of Interspeech, 2019, pp. 824–828.
  * [11] L. Lugosch, M. Ravanelli, P. Ignoto, V. S. Tomar, and Y. Bengio,  “Speech model pre-training for end-to-end spoken language understanding,”  arXiv preprint arXiv:1904.03670, 2019.
  * [12] M. Firdaus, S. Bhatnagar, A. Ekbal, and P. Bhattacharyya,  “A deep learning based multi-task ensemble model for intent detection and slot filling in spoken language understanding,”  in Proc. of ICONIP, 2018, pp. 647–658.
  * [13] C. Zhang, Y. Li, N. Du, W. Fan, and P. S. Yu,  “Joint slot filling and intent detection via capsule neural networks,”  arXiv preprint arXiv:1812.09471, 2018.
  * [14] C.W. Goo, Y.-K. Gao, G.and Hsu, C.-L. Huo, T.-C.h Chen, K.-W. Hsu, and Y.-N. Chen,  “Slot-gated modeling for joint slot filling and intent prediction,”  in Proc. of the 2018 NAACL-HLT, 2018, pp. 753–757.
  * [15] S. Bengio, O.l Vinyals, N. Jaitly, and N. Shazeer,  “Scheduled sampling for sequence prediction with recurrent neural networks,”  in Advances in Neural Information Processing Systems, 2015, pp. 1171–1179.
  * [16] D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P.r Motlicek, Y. Qian, P. Schwarz, et al.,  “The Kaldi speech recognition toolkit,”  in Proc. of ASRU, 2011.
  * [17] D. Snyder, G. Chen, and D. Povey,  “Musan: A music, speech, and noise corpus,”  arXiv preprint arXiv:1510.08484, 2015.
  * [18] S. Mansalis,  “Natural language understanding for dialogue systems using n-best lists,”  M.S. thesis, Department of Informatics, Athens University of Economics and Business, November 2019.
  * [19] Daniel Povey, Vijayaditya Peddinti, Daniel Galvez, Pegah Ghahremani, Vimal Manohar, Xingyu Na, Yiming Wang, and Sanjeev Khudanpur,  “Purely sequence-trained neural networks for asr based on lattice-free mmi.,”  in Interspeech, 2016, pp. 2751–2755.
  * [20] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova,  “BERT: Pre-training of deep bidirectional transformers for language understanding,”  arXiv preprint arXiv:1810.04805, 2018.
  * [21] Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov,  “Roberta: A robustly optimized BERT pretraining approach,”  arXiv preprint arXiv:1907.11692, 2019.
  * [22] V. Sanh, L. Debut, J. Chaumond, and T. Wolf,  “DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter,”  arXiv preprint arXiv:1910.01108, 2019.
  * [23] J. Pennington, R. Socher, and C. Manning,  “Glove: Global vectors for word representation,”  in Proc. of EMNLP, 2014, pp. 1532–1543.