# Quantum Electrodynamics in Strong Electromagnetic Fields:
Substate Resolved K$\alpha$ Transition Energies in Helium-like Uranium

Ph. Pfäfflein<EMAIL_ADDRESS>Helmholtz Institute Jena, 07743
Jena, Germany  GSI Helmholtzzentrum für Schwerionenforschung GmbH, 64291
Darmstadt, Germany  Institute of Optics and Quantum Electronics,
Friedrich Schiller University Jena, 07743 Jena, Germany  G. Weber  Helmholtz
Institute Jena, 07743 Jena, Germany  GSI Helmholtzzentrum für
Schwerionenforschung GmbH, 64291 Darmstadt, Germany  S. Allgeier  Kirchhoff
Institute for Physics, Heidelberg University, 69120 Heidelberg, Germany  Z.
Andelkovic  GSI Helmholtzzentrum für Schwerionenforschung GmbH, 64291
Darmstadt, Germany  S. Bernitt  Helmholtz Institute Jena, 07743 Jena, Germany
GSI Helmholtzzentrum für Schwerionenforschung GmbH, 64291 Darmstadt, Germany
A. I. Bondarev  Helmholtz Institute Jena, 07743 Jena, Germany  GSI
Helmholtzzentrum für Schwerionenforschung GmbH, 64291 Darmstadt, Germany  A.
Borovik, Jr  I. Physikalisches Institut, Justus-Liebig-Universität Gießen,
35392 Gießen, Germany  Helmholtz Forschungsakademie Hessen für FAIR (HFHF),
Campus Gießen, 35392 Gießen, Germany  L. Duval  Laboratoire Kastler Brossel,
Sorbonne Université, 75005 Paris, France  A. Fleischmann  Kirchhoff Institute
for Physics, Heidelberg University, 69120 Heidelberg, Germany  O. Forstner
Helmholtz Institute Jena, 07743 Jena, Germany  GSI Helmholtzzentrum für
Schwerionenforschung GmbH, 64291 Darmstadt, Germany  Institute of Optics and
Quantum Electronics,
Friedrich Schiller University Jena, 07743 Jena, Germany  M. Friedrich
Kirchhoff Institute for Physics, Heidelberg University, 69120 Heidelberg,
Germany  J. Glorius  GSI Helmholtzzentrum für Schwerionenforschung GmbH, 64291
Darmstadt, Germany  A. Gumberidze  GSI Helmholtzzentrum für
Schwerionenforschung GmbH, 64291 Darmstadt, Germany  Ch. Hahn  Helmholtz
Institute Jena, 07743 Jena, Germany  GSI Helmholtzzentrum für
Schwerionenforschung GmbH, 64291 Darmstadt, Germany  F. Herfurth  GSI
Helmholtzzentrum für Schwerionenforschung GmbH, 64291 Darmstadt, Germany  D.
Hengstler  Kirchhoff Institute for Physics, Heidelberg University, 69120
Heidelberg, Germany  M. O. Herdrich  Helmholtz Institute Jena, 07743 Jena,
Germany  GSI Helmholtzzentrum für Schwerionenforschung GmbH, 64291 Darmstadt,
Germany  Institute of Optics and Quantum Electronics,
Friedrich Schiller University Jena, 07743 Jena, Germany  P.-M. Hillenbrand
GSI Helmholtzzentrum für Schwerionenforschung GmbH, 64291 Darmstadt, Germany
I. Physikalisches Institut, Justus-Liebig-Universität Gießen, 35392 Gießen,
Germany  Helmholtz Forschungsakademie Hessen für FAIR (HFHF), Campus Gießen,
35392 Gießen, Germany  A. Kalinin  GSI Helmholtzzentrum für
Schwerionenforschung GmbH, 64291 Darmstadt, Germany  M. Kiffer  Helmholtz
Institute Jena, 07743 Jena, Germany  GSI Helmholtzzentrum für
Schwerionenforschung GmbH, 64291 Darmstadt, Germany  Institute of Optics and
Quantum Electronics,
Friedrich Schiller University Jena, 07743 Jena, Germany  F. M. Kröger
Helmholtz Institute Jena, 07743 Jena, Germany  GSI Helmholtzzentrum für
Schwerionenforschung GmbH, 64291 Darmstadt, Germany  Institute of Optics and
Quantum Electronics,
Friedrich Schiller University Jena, 07743 Jena, Germany  M. Kubullek
Institute of Optics and Quantum Electronics,
Friedrich Schiller University Jena, 07743 Jena, Germany  P. Kuntz  Kirchhoff
Institute for Physics, Heidelberg University, 69120 Heidelberg, Germany  M.
Lestinsky  GSI Helmholtzzentrum für Schwerionenforschung GmbH, 64291
Darmstadt, Germany  Yu. A. Litvinov  GSI Helmholtzzentrum für
Schwerionenforschung GmbH, 64291 Darmstadt, Germany  B. Löher  GSI
Helmholtzzentrum für Schwerionenforschung GmbH, 64291 Darmstadt, Germany  E.
B. Menz  Helmholtz Institute Jena, 07743 Jena, Germany  GSI Helmholtzzentrum
für Schwerionenforschung GmbH, 64291 Darmstadt, Germany  Institute of Optics
and Quantum Electronics,
Friedrich Schiller University Jena, 07743 Jena, Germany  T. Over  Helmholtz
Institute Jena, 07743 Jena, Germany  Institute of Optics and Quantum
Electronics,
Friedrich Schiller University Jena, 07743 Jena, Germany  N. Petridis  GSI
Helmholtzzentrum für Schwerionenforschung GmbH, 64291 Darmstadt, Germany  S.
Ringleb  Helmholtz Institute Jena, 07743 Jena, Germany  Institute of Optics
and Quantum Electronics,
Friedrich Schiller University Jena, 07743 Jena, Germany  R. S. Sidhu  GSI
Helmholtzzentrum für Schwerionenforschung GmbH, 64291 Darmstadt, Germany
School of Physics and Astronomy, The University of Edinburgh, EH9 3FD
Edinburgh, UK  U. Spillmann  GSI Helmholtzzentrum für Schwerionenforschung
GmbH, 64291 Darmstadt, Germany  S. Trotsenko  GSI Helmholtzzentrum für
Schwerionenforschung GmbH, 64291 Darmstadt, Germany  A. Warczak  Marian
Smoluchowski Institute of Physics, Jagiellonian University, 30-348 Kraków,
Poland  B. Zhu  School of Mechanical and Electrical Engineering, Chuzhou
University, 239000 Chuzhou, China  Ch. Enss  Kirchhoff Institute for Physics,
Heidelberg University, 69120 Heidelberg, Germany  Th. Stöhlker  Helmholtz
Institute Jena, 07743 Jena, Germany  GSI Helmholtzzentrum für
Schwerionenforschung GmbH, 64291 Darmstadt, Germany  Institute of Optics and
Quantum Electronics,
Friedrich Schiller University Jena, 07743 Jena, Germany

###### Abstract

Using novel metallic magnetic calorimeter detectors at the CRYRING@ESR, we
recorded X-ray spectra of stored and electron cooled helium-like uranium
(U90+) with an unmatched spectral resolution of close to $90$ eV. This allowed
for an accurate determination of the energies of all four components of the
K$\alpha$ transitions in U90+. We find good agreement with state-of-the-art
bound-state QED calculations for the strong-field regime. Our results do not
support any systematic deviation between experiment and theory in helium-like
systems, the presence of which was subject of intense debates in recent years.

The study of electrons in extreme electromagnetic fields, as present in heavy
highly-charged ions (HCI), is one of the frontiers in exploring quantum
electrodynamics (QED). At high atomic numbers $Z$, the electron–nucleus
coupling constant $\alpha Z$ approaches unity, where $\alpha\approx 1/137$ is
the fine structure constant. Therefore, for heavy HCI, perturbative treatments
in $\alpha Z$, that enable highly accurate QED predictions for hydrogen and
other light systems, are no longer applicable [1]. Nowadays, challenging non-
perturbative calculations in $\alpha Z$ can be performed up to the second
order of the expansion in $\alpha$, including one-electron two-loop
contributions as well as two-electron QED effects [2, 3, 4, 5].

The most stringent experimental tests of bound-state QED in the presence of
strong electromagnetic fields are provided by measurements of transition
energies [6], hyperfine structure [7, 8] and bound electron g-factors [9]. For
the heaviest HCI, up until now, measurements of transition energies have
delivered the most sensitive tests of various QED contributions. For hydrogen-
like and lithium-like systems, several high-precision measurements have
already been performed, as summarized by Beiersdorfer [10] and Indelicato
[11]. In contrast, precision spectroscopy of helium-like ions heavier than
xenon ($Z=54$) have been virtually non-existent until very recently. It is
important to emphasize that helium-like ions represent the simplest multi-
electron systems where a complex interplay between relativistic,
electron–electron correlation and QED effects can be explored. QED
calculations of the binding energies in helium-like ions for principal quantum
numbers $n\,=\,1$ and $n\,=\,2$ have been accomplished for
$Z=12\,\text{--}\,100$ by Artemyev et al. [3]. Refinements of these
calculations have recently been reported for specific nuclei by Malyshev et
al. [4] and Kozhedub et al. [5]. Furthermore, a successful validation of QED
predictions in this regime of very strong coupling may provide novel
opportunities for tests of fundamental physics and searches for physics beyond
the Standard Model [12, 13, 14]. Recently, groundbreaking results on the
intra-shell transition $1\text{s}2\text{p}\text{ }^{3}\text{P}_{2}\text{
}\rightarrow\text{ }1\text{s}2\text{s}\text{ }^{3}\text{S}_{1}$ (close to 4.5
keV) were reported by Loetzsch et al. [6], confirming theoretical predictions
of the screened and two-loop one-electron QED contributions in helium-like
heavy ions for the first time.

Figure 1: Level scheme of helium-like uranium, where the most prominent
radiative transitions are depicted. The K$\alpha$ transitions studied in this
work are drawn in magenta and blue.

However, a similar test is still missing for the K$\alpha$ transitions
($n\,=\,2\rightarrow n\,=\,1$) which provide access to the most strongly bound
states being most affected by QED effects. In the heaviest systems, precision
spectroscopy of this radiation is particular demanding due to the high X-ray
energies close to 100 keV in combination with the presence of closely spaced
L-shell sublevels, as presented in Fig. 1. Disentangling the individual
transitions that form the K$\alpha$ peaks, called w, x, y and z according to
Gabriel’s notation [15], is hardly possible, due to the limited spectral
resolution of commonly used semiconductor detectors. In U90+, the closest
lines w and x are separated by 74 eV compared to a typical energy resolution
of a few 100 eV full width at half maximum (FWHM). Consequently, precision
spectroscopy of K$\alpha$ transitions has so far been performed only up to
xenon [16]. The need for accurate measurements in high-$Z$ systems is
particular pressing as possible $Z$-dependent deviations between experiment
and theory have been intensely discussed [17, 18, 19, 20, 21, 22, 23, 24, 25,
11].

In this Letter, we present a study using novel high-resolution detectors on
helium-like uranium (U90+) at the GSI/FAIR facility in Darmstadt, Germany. So
far, the most precise studies of these transitions were performed using
semiconductor detectors [26, 27]. The limited resolution provided by this
detector technology hindered accurate comparison of the observed transition
energies with theoretical predictions. In contrast, in the present work we
used an alternative detector technology featuring a superior spectral
performance, similarly to the study of helium-like xenon [16]. At the electron
cooler of the recently installed CRYRING@ESR [28], we employed two maXs-type
detectors (Micro-Calorimeter Arrays for High Resolution X-ray Spectroscopy)
[29, 30]. A spectral resolution of $\leq 90$ eV FWHM over a wide energy range
from a few keV to above 100 keV was achieved. This enabled to disentangle all
four components of the K$\alpha$ radiation of U90+, resulting in an accurate
determination of their transition energies, without relying on assumptions
about the relative line intensities. As a cross-check, we also obtained an
accurate value of the U90+ ground-state ionization energy. While a
comprehensive description of the experimental setup and its operation is
provided in [31], in the following we briefly highlight key features of the
experiment before proceeding to the discussion of the obtained results.

Excited U90+ ions were produced from a primary beam of hydrogen-like U91+ ions
via radiative recombination in the electron cooler of CRYRING@ESR. Starting
with a beam of U91+ at a kinetic energy of 296 MeV/u, in the ESR successive
electron cooling and deceleration steps were applied. At an energy of 10.225
MeV/u the ions were transferred to the low-energy storage ring CRYRING@ESR.
This multi-staged preparation procedure resulted in one injection every 55 s.
The experiment was conducted with an average of about $1\times 10^{6}$ U91+
ions per injection over a week of continuous measurement time.

In CRYRING@ESR, the stored ions were continuously electron-cooled with a
voltage of 5634.5 V applied to the cooler. At an electron current of 30.5 mA
an electron density of approximately $1.2\,\times\,10^{7}\,\text{cm}^{-3}$ was
reached. The repulsive force by the space charge of the electron beam reduced
the acceleration potential to 5609.4 V. We assume an uncertainty of $\pm\,2$ V
to account for the unknown contact potential between the cathode and collector
electrode of the electron cooler. This translates to an expected ion beam
velocity of $\beta=0.14695(3)$.

At relative velocities between electrons and ions close to zero, as is the
case during electron cooling, the recombination process predominantly captures
electrons into Rydberg states. The excited ions subsequently decay to the
ground state via radiative cascades. Thus, each recombination event typically
results in the emission of numerous characteristic photons in addition to the
radiative recombination photon, as discussed in detail for example by Zhu et
al. [32]. The U90+ ions were separated from the main beam in the next dipole
magnet and then registered by a particle detector.

The cooler section features view ports located at 0∘ and 180∘ with respect to
the ion beam axis, that are equipped with 50 µm thin beryllium X-ray windows.
At these angles the Doppler shift gets virtually independent of the
observation angle. Therefore the influence of a slight detector misalignment
on the recorded photon energy is negligible, see [33] for details. This is in
sharp contrast to crystal spectrometer setups with detection perpendicular to
the ion beam axis [34, 6], which require a precise knowledge of the geometry.
At both view ports a maXs-type microcalorimeter was placed. The resulting
distance between the detectors and the center of the interaction zone of
stored ions and cooler electrons was approximately 3.5 m.

Figure 2: Spectral distribution of the K$\alpha$ transitions in U90+, with
photon energies transformed into the emitter frame. Dark blue: Data of the
present study with the spectra of both microcalorimeters added up, counts on
left y-axis. As shown in the inset, the lines z and y have been resolved for
the first time. Light grey: Spectrum recorded with a germanium detector (data
from [27]), counts according to right y-axis.

The detectors are based on the metallic magnetic calorimeter (MMC) technology.
The energy of an incident particle is converted into thermal energy in a small
absorber. The resulting temperature rise leads to a decrease of the
magnetization of a thermally coupled paramagnetic sensor made of sputtered
Ag:Er. The changing magnetization is in turn measured by a low-noise, high-
bandwidth superconducting quantum interference device (SQUID) magnetometer.
The necessary temperatures for operating these quantum sensors of below 20 mK
can be achieved using 3He/4He dilution refrigerator cryostats. This technology
allows for a unique combination of a high spectral resolution with a broad
bandwidth acceptance [35].

In the present study, two identical prototypes of the maXs-100 design were
employed. They feature an $8\times 8$ array of absorbers made of gold. Each
absorber has an area of 1.25 mm $\times$ 1.25 mm and a thickness of 50 µm.
These detectors are tailored for a spectral resolution of $\Delta
E_{\text{FWHM}}\,<\,50$ eV in a broad photon energy range from a few keV to
above 100 keV. The design provides a photopeak efficiency of higher than 10 %
up to 130 keV photon energy. Furthermore, the maXs detectors’ fast signal rise
time was exploited for the first time in this measurement to extract timing
information. Setting a coincidence condition on the simultaneous detection of
photons and down-charged U90+ ions suppressed the background by two orders of
magnitude [36]. This procedure yielded almost background-free spectra of the
radiation emitted in association with the recombination of cooler electrons
with the stored U91+ ions.

The lifetime of the ion beam in CRYRING@ESR was $(7-8)$ s, leaving the ring
virtually empty after 25 s. The remaining 30 s period of the accelerator cycle
was used for energy calibration of the detectors. The isotopes 241Am, 57Co,
109Cd, and 153Gd were selected as reference sources. They offer a set of well-
known $\gamma$ lines in the energy range of interest. The intensity of the
sources was chosen low enough so that a potential input power dependent shift
as observed by Thorn et al. [16] was negligible. A second-order polynomial
function was used to translate pulse amplitudes to incident photon energies,
in accordance with the expected gain behaviour of MMC detectors [37]. For the
0∘ detector the lines near 26 keV and 60 keV of 241Am and the lines near 122
keV and 136 keV of 57Co were used for calibration. In case of the 180∘
detector the lines close to 60 keV of 241Am, close to 88 keV in 109Cd and
close to 122 keV in 57Co were chosen. The energy values of the lines were
taken as reported by Helmer and van der Leun [38].

The use of reference lines that are tens of keV away from the lines of
interest is likely to cause a sizable uncertainty in the determination of
their positions. To quantify this effect, we selected the 97 keV and 103 keV
lines of 153Gd that are sufficiently close to the energy region of interest in
both detectors as a benchmark. The difference between the measured line
positions and the literature values are $0.24\pm 0.99$ eV and $2.0\pm 1.3$ eV
for the 0∘ detector and $-1.32\pm 0.40$ eV and $-2.51\pm 0.50$ eV for the 180∘
detector. From this, we inferred a conservative estimate of $\pm$2.5 eV for
the systematic uncertainty of energy determination in our region of interest.
For the reference lines we measured instrumental line widths between 65 eV and
90 eV FWHM. This degradation in resolution compared to the design value of the
detectors is partly attributed to the coupling of electromagnetic
interferences into the SQUID readout electronics during the beamtime.
Furthermore, mechanical vibrations can introduce an additional heat load into
the cryostats and therefore deteriorate the resolution.

Table 1: Energies of the K$\alpha$ transitions in U90+ obtained in this work ($E_{\text{exp}}$) in comparison with theory ($E_{\text{th}}$) [5]. The lines are referred to by their initial states as well as by Gabriel’s notation [15] in parentheses. Furthermore, the natural line widths $\Delta E_{nat}$ of the initial states are shown. All energies are given in units of eV. An additional systematic uncertainty due to the calibration is estimated to be 2.5 eV for all transitions.  Transition |  | $\Delta E_{\text{nat}}$ |  | $E_{\text{th}}$ |  | $E_{\text{exp}}$
---|---|---|---|---|---|---
$1\text{s}2\text{p}\text{ }^{1}\text{P}_{1}$ | (w) |  | $33.8$ |  | $100610.68$ | (54) |  | $100609$ | $\pm$ | $13$
$1\text{s}2\text{p}\text{ }^{3}\text{P}_{2}$ | (x) |  | $0.2$ |  | $100536.95$ | (54) |  | $100543.1$ | $\pm$ | $7.6$
$1\text{s}2\text{p}\text{ }^{3}\text{P}_{1}$ | (y) |  | $20.1$ |  | $96169.43$ | (54) |  | $96161.0$ | $\pm$ | $6.7$
$1\text{s}2\text{s}\text{ }^{3}\text{S}_{1}$ | (z) |  | $0.1$ |  | $96027.07$ | (54) |  | $96027.9$ | $\pm$ | $5.0$
Figure 3:  Left part: Overview of available experimental data of the four
K$\alpha$ transitions in helium-like systems, taken from [11] and references
therein. Shown is the normalized difference between measured energies and
theoretical predictions [3]. The results of the present work (shown by green
symbols) provide the first accurate data for systems heavier than xenon
($Z=54$). Right part: expanded view of our data points for He-like uranium.

In the following, we focus on the K$\alpha$ transitions in U90+. Each detector
recorded about 150 K$\alpha$ photons, whose energies were Doppler shifted to
the region between 111 keV and 117 keV at 0∘ and 82.5 keV to 87 keV at 180∘,
respectively. The predicted energy splittings of their components, x and w in
case of K$\alpha_{1}$ and z and y for K$\alpha_{2}$, are of similar size as
the detector resolution. Figure 2 shows the combined spectral data in the
K$\alpha$ region from both microcalorimeters, with the photon energies
transformed into the emitter frame. To highlight the improvement in resolution
a spectrum previously recorded with a germanium detector (data from [27]) is
shown as well. The achieved detector resolution proved suitable for separating
the contributions of the z and y lines to the K$\alpha_{2}$ peak. The w and x
transitions are superimposed and form a single peak (K$\alpha_{1}$). However,
a detailed investigation of the peak shape clearly shows that it is formed by
two individual lines.

To obtain the transition energies, four Voigt-shaped peaks were matched to the
spectra recorded by each detector, applying an unbinned maximum-likelihood
adjustment using the iminuit package [39, 40]. The width parameters of the
peaks were fixed: The natural line widths of up to 34 eV, being not negligible
compared to the detector resolution, were taken from theory as listed in Tab.
1. The detectors’ response function was assumed to be Gaussian-shaped, with
width determined from the nearest $\gamma$ reference lines to values of 81 eV
for the $0^{\circ}$ detector and 90 eV for the $180^{\circ}$ detector. The
relative intensities and the centroid positions were used as free parameters
for the adjustment, together with the ion beam velocity to account for the
Doppler shift. The adjustment yields $\beta=0.146987(34)$, in good agreement
with the value estimated from the electron cooler settings. This also serves
as a consistency check for the energy calibration of both detectors.

Our experimental values for the transition energies are given in Tab. 1,
together with theory values taken from [5]. Experimental uncertainties are
dominated by counting statistics and given for $1\sigma$ confidence intervals.
Our measurement provides values for all four components of the K$\alpha$
transitions in U90+ for the first time. An overview of the existing
experimental data for all the K$\alpha$ transition energies for atomic numbers
$Z=12\text{--}92$ is given in Fig. 3. Plotted is the differences between
theoretical and experimental values normalized to $Z^{2}$. While the low- to
medium-$Z$ range was extensively studied, our measurement is the first one at
$Z>54$ to achieve the necessary accuracy for a meaningful test of theory. We
acknowledge that published data exist also for K$\alpha$ transitions in $Z=59$
[41] and $Z=92$ [26, 42]. However, in these studies the determination of the
line position either relied on an assumption about the population of the
excited states and the resulting relative line intensities or used theory
values for some K$\alpha$ transitions as an energy reference for the others.
For the case of U90+, our measurement constitutes an improvement of up to one
order of magnitude of the uncertainty compared to these previous studies. Our
findings are in good agreement with theory and do not suggest any systematic
deviation scaling with 4${}^{\text{th}}$ or higher powers of $Z$ as proposed
in [19, 22].

As a consistency check, we also determined the ground-state ionization energy
$E_{\text{ion}}$ for U90+, by assuming that the L-shell ionization energies
are accurately known from theory [5]. This assumption is supported by a recent
high-precision measurement of the intra-shell transition
$1\text{s}2\text{p}\text{ }^{3}\text{P}_{2}\text{ }\rightarrow\text{
}1\text{s}2\text{s}\text{ }^{3}\text{S}_{1}$ in U90+ [6], showing an agreement
with theory on the level of $0.17$ eV. With the binding energies of the L
shell states being fixed, an adjustment was performed with the ground-state
ionization energy as a free parameter $E_{t}=E_{\text{ion}}-E_{\text{L}i}$
(here $E_{t}$ with $t\in\text{x, y, w, z}$ is the transition energy and
$E_{\text{L}i}$ the ionization energy of the corresponding initial L substate,
see Fig. 1). This evaluation yields $E_{\text{ion}}=129570.0\pm 4.2\pm 2.5$
eV, which is in good agreement with the theoretical value of $129570.09(53)$
eV taken from [5]. Previously, the only experimental determination of this
quantity was a relative measurement to the ground-state binding-energy of U91+
[27]. Combining the relative value of $2248\pm 9$ eV reported there with the
measured ground-state ionization energy of the hydrogen-like system of
$131819.8\pm 4.6$ eV [43] results in a value of $E_{\text{ion}}=129571\pm 10$
eV, which is also in agreement with our value.

In summary, we performed a spectral study of X-ray transitions in helium-like
uranium using novel MMC detectors at the electron cooler of the CRYRING@ESR.
Achieving a resolution of close to 90 eV FWHM for photon energies of up to 150
keV is a major breakthrough for precision spectroscopy of heavy ions at
storage rings. It allowed us to disentangle the components of the K$\alpha$
lines in U90+ for the first time. The measured transition energies are in good
agreement with state-of-the-art bound-state QED calculations. This finding
does not support any systematic deviation that scales with 4${}^{\text{th}}$
or higher powers of $Z$, as was recently proposed and intensely debated within
the community.

We like to highlight that when combining MMC detectors with the beneficial
experimental conditions at the CRYRING@ESR electron cooler, the only
significant systematic uncertainty is the quality of the energy calibration.
By optimizing the calibration procedure and using a broader set of gamma
reference lines, a significant reduction of this uncertainty to below 1 eV is
possible. In the present experiment, the statistical uncertainty even for the
most intense K$\alpha$ transition is close to 5 eV. One reason is that the
number of ions injected into the recently commissioned CRYRING@ESR was about
an order of magnitude below the design value. In future measurements we expect
a significant boost in ion intensity and hence in the counting statistics.
Combined with other already planned incremental improvements, such as thicker
absorber pixels to achieve a higher quantum efficiency, the presented approach
clearly has the potential to reach the 1 eV accuracy level necessary to test
higher-order QED contributions in the heaviest atomic systems.

###### Acknowledgements.

The authors are indebted to the local teams at GSI, in particular of the ESR
and CRYRING@ESR, for making this study possible. We also would like to thank
C. E. Düllmann and D. Renisch for providing us with gamma sources for detector
calibration. This research has been conducted in the framework of the SPARC
collaboration, experiment E138 of FAIR Phase-0 supported by GSI, Darmstadt
(Germany). It is further supported by the European Research Council (ERC)
under the European Union’s Horizon 2020 research and innovation programme
(Grant agreements No. 824109 “EMP” and No. 682841 “ASTRUm”). B. Zhu
acknowledges CSC Doctoral Fellowship 2018.9 - 2022.2 (Grant No. 201806180051).
We also acknowledge the support provided by ErUM FSP T05 - “Aufbau von APPA
bei FAIR” (BMBF n° 05P19SJFAA and n° 05P19VHFA1).

## References

  * Shabaev  _et al._ [2018] V. M. Shabaev, A. I. Bondarev, D. A. Glazov, M. Y. Kaygorodov, Y. S. Kozhedub, I. A. Maltsev, A. V. Malyshev, R. V. Popov, I. I. Tupitsyn, and N. A. Zubova, Hyperfine Interactions 239, 60 (2018).
  * Yerokhin and Shabaev [2015] V. A. Yerokhin and V. M. Shabaev, J. Phys. Chem. Ref. Data 44, 033103 (2015).
  * Artemyev  _et al._ [2005] A. N. Artemyev, V. M. Shabaev, V. A. Yerokhin, G. Plunien, and G. Soff, Phys. Rev. A 71, 062104 (2005).
  * Malyshev  _et al._ [2019] A. V. Malyshev, Y. S. Kozhedub, D. A. Glazov, I. I. Tupitsyn, and V. M. Shabaev, Phys. Rev. A 99, 010501(R) (2019).
  * Kozhedub  _et al._ [2019] Y. S. Kozhedub, A. V. Malyshev, D. A. Glazov, V. M. Shabaev, and I. I. Tupitsyn, Phys. Rev. A 100, 062506 (2019).
  * Loetzsch  _et al._ [2024] R. Loetzsch, H. F. Beyer, L. Duval, U. Spillmann, D. Banaś, P. Dergham, F. M. Kröger, J. Glorius, R. E. Grisenti, M. Guerra, A. Gumberidze, R. Heß, P.-M. Hillenbrand, P. Indelicato, P. Jagodzinski, E. Lamour, B. Lorentz, S. Litvinov, Y. A. Litvinov, J. Machado, N. Paul, G. G. Paulus, N. Petridis, J. P. Santos, M. Scheidel, R. S. Sidhu, M. Steck, S. Steydli, K. Szary, S. Trotsenko, I. Uschmann, G. Weber, T. Stöhlker, and M. Trassinelli, Nature 625, 673 (2024).
  * Ullmann  _et al._ [2017] J. Ullmann, Z. Andelkovic, C. Brandau, A. Dax, W. Geithner, C. Geppert, C. Gorges, M. Hammen, V. Hannen, S. Kaufmann, K. König, Y. A. Litvinov, M. Lochmann, B. Maaß, J. Meisner, T. Murböck, R. Sánchez, M. Schmidt, S. Schmidt, M. Steck, Th. Stöhlker, R. C. Thompson, C. Trageser, J. Vollbrecht, C. Weinheimer, and W. Nörtershäuser, Nat Commun 8, 15484 (2017).
  * Skripnikov  _et al._ [2018] L. V. Skripnikov, S. Schmidt, J. Ullmann, C. Geppert, F. Kraus, B. Kresse, W. Nörtershäuser, A. F. Privalov, B. Scheibe, V. M. Shabaev, M. Vogel, and A. V. Volotka, Phys. Rev. Lett. 120, 093001 (2018).
  * Morgner  _et al._ [2023] J. Morgner, B. Tu, C. M. König, T. Sailer, F. Heiße, H. Bekker, B. Sikora, C. Lyu, V. A. Yerokhin, Z. Harman, J. R. Crespo López-Urrutia, C. H. Keitel, S. Sturm, and K. Blaum, Nature 622, 53 (2023).
  * Beiersdorfer [2010] P. Beiersdorfer, Journal of Physics B: Atomic, Molecular and Optical Physics 43, 074032 (2010).
  * Indelicato [2019] P. Indelicato, Journal of Physics B: Atomic, Molecular and Optical Physics 52, 232001 (2019).
  * Kozlov  _et al._ [2018] M. G. Kozlov, M. S. Safronova, J. R. Crespo López-Urrutia, and P. O. Schmidt, Rev. Mod. Phys. 90, 045005 (2018).
  * Safronova  _et al._ [2018] M. S. Safronova, D. Budker, D. DeMille, D. F. J. Kimball, A. Derevianko, and C. W. Clark, Rev. Mod. Phys. 90, 025008 (2018).
  * Safronova [2019] M. S. Safronova, Annalen der Physik 531, 1800364 (2019).
  * Gabriel [1972] A. H. Gabriel, Monthly Notices of the Royal Astronomical Society 160, 99 (1972).
  * Thorn  _et al._ [2009] D. B. Thorn, M. F. Gu, G. V. Brown, P. Beiersdorfer, F. S. Porter, C. A. Kilbourne, and R. L. Kelley, Phys. Rev. Lett. 103, 163001 (2009).
  * Chantler  _et al._ [2012] C. T. Chantler, M. N. Kinnane, J. D. Gillaspy, L. T. Hudson, A. T. Payne, L. F. Smale, A. Henins, J. M. Pomeroy, J. N. Tan, J. A. Kimpton, E. Takacs, and K. Makonyi, Phys. Rev. Lett. 109, 153001 (2012).
  * Epp [2013] S. W. Epp, Phys. Rev. Lett. 110, 159301 (2013).
  * Chantler  _et al._ [2013] C. T. Chantler, M. N. Kinnane, J. D. Gillaspy, L. T. Hudson, A. T. Payne, L. F. Smale, A. Henins, J. M. Pomeroy, J. A. Kimpton, E. Takacs, and K. Makonyi, Phys. Rev. Lett. 110, 159302 (2013).
  * Payne  _et al._ [2014] A. T. Payne, C. T. Chantler, M. N. Kinnane, J. D. Gillaspy, L. T. Hudson, L. F. Smale, A. Henins, J. A. Kimpton, and E. Takacs, Journal of Physics B: Atomic, Molecular and Optical Physics 47, 185001 (2014).
  * Kubiček  _et al._ [2014] K. Kubiček, P. H. Mokler, V. Mäckel, J. Ullrich, and J. R. Crespo López-Urrutia, Phys. Rev. A 90, 032508 (2014).
  * Chantler  _et al._ [2014] C. T. Chantler, A. T. Payne, J. D. Gillaspy, L. T. Hudson, L. F. Smale, A. Henins, J. A. Kimpton, and E. Takacs, New J. Phys. 16, 123037 (2014).
  * Epp  _et al._ [2015] S. W. Epp, R. Steinbrügge, S. Bernitt, J. K. Rudolph, C. Beilmann, H. Bekker, A. Müller, O. O. Versolato, H.-C. Wille, H. Yavaş, J. Ullrich, and J. R. Crespo López-Urrutia, Phys. Rev. A 92, 020502(R) (2015).
  * Beiersdorfer and Brown [2015] P. Beiersdorfer and G. V. Brown, Phys. Rev. A 91, 032514 (2015).
  * Machado  _et al._ [2018] J. Machado, C. I. Szabo, J. P. Santos, P. Amaro, M. Guerra, A. Gumberidze, G. Bian, J. M. Isac, and P. Indelicato, Phys. Rev. A 97, 032517 (2018).
  * Briand  _et al._ [1990] J. P. Briand, P. Chevallier, P. Indelicato, K. P. Ziock, and D. D. Dietrich, Phys. Rev. Lett. 65, 2761 (1990).
  * Gumberidze  _et al._ [2004] A. Gumberidze, Th. Stöhlker, D. Banaś, K. Beckert, P. Beller, H. F. Beyer, F. Bosch, X. Cai, S. Hagmann, C. Kozhuharov, D. Liesen, F. Nolden, X. Ma, P. H. Mokler, A. Orši ć Muthig, M. Steck, D. Sierpowski, S. Tashenov, A. Warczak, and Y. Zou, Phys. Rev. Lett. 92, 203004 (2004).
  * Lestinsky  _et al._ [2022] M. Lestinsky, E. B. Menz, H. Danared, C. Krantz, E. Lindroth, Z. Andelkovic, C. Brandau, A. Bräuning-Demian, S. Fedotova, W. Geithner, F. Herfurth, A. Kalinin, I. Kraus, U. Spillmann, G. Vorobyev, and Th. Stöhlker, Atoms 10, 141 (2022).
  * Pies  _et al._ [2012] C. Pies, S. Schäfer, S. Heuser, S. Kempf, A. Pabinger, J. P. Porst, P. Ranitsch, N. Foerster, D. Hengstler, A. Kampkötter, T. Wolf, L. Gastaldo, A. Fleischmann, and C. Enss, J. Low Temp. Phys. 167, 269 (2012).
  * Hengstler  _et al._ [2015] D. Hengstler, M. Keller, C. Schötz, J. Geist, M. Krantz, S. Kempf, L. Gastaldo, A. Fleischmann, T. Gassner, G. Weber, R. Märtin, Th. Stöhlker, and C. Enss, Phys. Scr. T166, 014054 (2015).
  * Pfäfflein  _et al._ [2022a] Ph. Pfäfflein, S. Allgeier, S. Bernitt, A. Fleischmann, M. Friedrich, C. Hahn, D. Hengstler, M. O. Herdrich, A. Kalinin, F. M. Kröger, P. Kuntz, M. Lestinsky, B. Löher, E. B. Menz, T. Over, U. Spillmann, G. Weber, B. Zhu, C. Enss, and Th. Stöhlker, Physica Scripta 97, 114005 (2022a).
  * Zhu  _et al._ [2022] B. Zhu, A. Gumberidze, T. Over, G. Weber, Z. Andelkovic, A. Bräuning-Demian, R. J. Chen, D. Dmytriiev, O. Forstner, C. Hahn, F. Herfurth, M. O. Herdrich, P.-M. Hillenbrand, A. Kalinin, F. M. Kröger, M. Lestinsky, Y. A. Litvinov, E. B. Menz, W. Middents, T. Morgenroth, N. Petridis, Ph. Pfäfflein, M. S. Sanjari, R. S. Sidhu, U. Spillmann, R. Schuch, S. Schippers, S. Trotsenko, L. Varga, G. Vorobyev, and T. Stöhlker, Phys. Rev. A 105, 052804 (2022).
  * Kröger  _et al._ [2023] F. M. Kröger, G. Weber, S. Allgeier, Z. Andelkovic, S. Bernitt, A. Borovik, L. Duval, A. Fleischmann, O. Forstner, M. Friedrich, J. Glorius, A. Gumberidze, C. Hahn, F. Herfurth, D. Hengstler, M. O. Herdrich, P.-M. Hillenbrand, A. Kalinin, M. Kiffer, M. Kubullek, P. Kuntz, M. Lestinsky, B. Löher, E. B. Menz, T. Over, N. Petridis, Ph. Pfäfflein, S. Ringleb, R. S. Sidhu, U. Spillmann, S. Trotsenko, A. Warczak, B. Zhu, C. Enss, and Th. Stöhlker, Atoms 11, 10.3390/atoms11020022 (2023).
  * Gassner  _et al._ [2018] T. Gassner, M. Trassinelli, R. Heß, U. Spillmann, D. Banaś, K. H. Blumenhagen, F. Bosch, C. Brandau, W. Chen, D. Chr, E. Förster, R. E. Grisenti, A. Gumberidze, S. Hagmann, P. M. Hillenbrand, P. Indelicato, P. Jagodzinski, T. Kämpfer, K. Chr, M. Lestinsky, D. Liesen, A. L. Yu, R. Loetzsch, B. Manil, R. Märtin, F. Nolden, N. Petridis, M. S. Sanjari, K. S. Schulze, M. Schwemlein, A. Simionovici, M. Steck, Th. Stöhlker, C. I. Szabo, S. Trotsenko, I. Uschmann, G. Weber, O. Wehrhan, N. Winckler, D. F. A. Winters, N. Winters, E. Ziegler, and H. F. Beyer, New J. Phys. 20, 073033 (2018).
  * Kempf  _et al._ [2018] S. Kempf, A. Fleischmann, L. Gastaldo, and C. Enss, J. Low Temp. Phys. 193, 365 (2018).
  * Pfäfflein  _et al._ [2022b] Ph. Pfäfflein, G. Weber, S. Allgeier, S. Bernitt, A. Fleischmann, M. Friedrich, C. Hahn, D. Hengstler, M. O. Herdrich, A. Kalinin, F. M. Kröger, P. Kuntz, M. Lestinsky, B. Löher, E. B. Menz, U. Spillmann, B. Zhu, C. Enss, and Th. Stöhlker, Atoms 11, 5 (2022b).
  * Bates  _et al._ [2016] C. R. Bates, C. Pies, S. Kempf, D. Hengstler, A. Fleischmann, L. Gastaldo, C. Enss, and S. Friedrich, Appl. Phys. Lett. 109, 023513 (2016).
  * Helmer and van der Leun [2000] R. Helmer and C. van der Leun, Nucl. Instrum. Meth. A 450, 35 (2000).
  * James and Roos [1975] F. James and M. Roos, Comput. Phys. Commun. 10, 343 (1975).
  * Dembinski and et al. [2020] H. Dembinski and P. O. et al. 10.5281/zenodo.3949207 (2020).
  * Thorn  _et al._ [2008] D. B. Thorn, G. V. Brown, J. H. Clementson, H. Chen, M. Chen, P. Beiersdorfer, K. R. Boyce, C. A. Kilbourne, F. S. Porter, and R. L. Kelley, Canadian Journal of Physics 86, 241 (2008).
  * Lupton  _et al._ [1994] J. H. Lupton, D. D. Dietrich, C. J. Hailey, R. E. Stewart, and K. P. Ziock, Phys. Rev. A 50, 2150 (1994).
  * Gumberidze  _et al._ [2005] A. Gumberidze, Th. Stöhlker, D. Banas, K. Beckert, P. Beller, H. F. Beyer, F. Bosch, S. Hagmann, C. Kozhuharov, D. Liesen, F. Nolden, X. Ma, P. H. Mokler, M. Steck, D. Sierpowski, and S. Tashenov, Phys. Rev. Lett. 94, 223001 (2005).


# Knowledge Tracing with Sequential Key-Value Memory Networks

Ghodai Abdelrahman and Qing Wang
Research School of Computer Science, Australian National
UniversityCanberraACT0200 ghodai.abdelrahman<EMAIL_ADDRESS>
###### Abstract.

Can machines trace human knowledge like humans? Knowledge tracing (KT) is a
fundamental task in a wide range of applications in education, such as massive
open online courses (MOOCs), intelligent tutoring systems, educational games,
and learning management systems. It models dynamics in a student’s knowledge
states in relation to different learning concepts through their interactions
with learning activities. Recently, several attempts have been made to use
deep learning models for tackling the KT problem. Although these deep learning
models have shown promising results, they have limitations: either lack the
ability to go deeper to trace how specific concepts in a knowledge state are
mastered by a student, or fail to capture long-term dependencies in an
exercise sequence. In this paper, we address these limitations by proposing a
novel deep learning model for knowledge tracing, namely Sequential Key-Value
Memory Networks (SKVMN). This model unifies the strengths of recurrent
modelling capacity and memory capacity of the existing deep learning KT models
for modelling student learning. We have extensively evaluated our proposed
model on five benchmark datasets. The experimental results show that (1) SKVMN
outperforms the state-of-the-art KT models on all datasets, (2) SKVMN can
better discover the correlation between latent concepts and questions, and (3)
SKVMN can trace the knowledge state of students dynamics, and a leverage
sequential dependencies in an exercise sequence for improved predication
accuracy.

Knowledge Tracing; Memory Networks; Deep Learning; Sequence Modelling; Key-
Value Memory

††doi:

##  1\. Introduction

One of the prominent features of human intelligence is the ability to track
their current knowledge states in mastering specific skills or concepts. This
enables humans to identify gaps in their knowledge states to personalise their
learning experience. With the success of artificial intelligence (AI) in
modeling various areas of human cognition (LeCun et al., 2015; Graves et al.,
2013a; Yu et al., 2017; Donahue et al., 2015; Liu et al., 2016), the question
has arisen: can machines trace human knowledge like humans? This motivated the
study of _knowledge tracing_ (KT), which aims to model the knowledge states of
students in mastering skills and concepts, through a sequence of learning
activities they participate in (Corbett and Anderson, 1994; Piech et al.,
2015; Zhang et al., 2017).

Knowledge tracing is of fundamental importance to a wide range of applications
in education, such as massive open online courses (MOOCs), intelligent
tutoring systems, educational games, and learning management systems.
Improvements in knowledge tracing can drive novel techniques to advance human
learning. For example, knowledge tracing can be used to discover students’
individual learning needs so that personalised learning and support can be
provided to fulfill diverse capabilities of each student (Khajah et al.,
2016). It can also be used by human experts to design new measures of student
learning and new teaching materials based on learning strengths and weaknesses
of students (Piech, 2016). Nonetheless, tracing human knowledge using machines
is a rather challenging task. This is due to the complexity of human learning
behaviors (e.g., memorising, guessing, forgetting, etc.) and the inherent
difficulties of modeling human knowledge (i.e. skills and prior background)
(Piech et al., 2015).

Figure 1. An illustration of how a student’s knowledge states are evolving for
a sequence of 50 exercises in the dataset ASSISTments2009 using: (a) DKVMN and
(b) SKVMN. There are 5 concepts $\\{c^{1},\dots,c^{5}\\}$ underlying these 50
exercises. Each row depicts the evolution of the concept states for a specific
concept, and each column depicts the knowledge state at a certain time step.

Existing knowledge tracing models can be generally classified into two
categories: traditional machine learning KT models and deep learning KT
models. Among traditional machine learning KT models, _Bayesian Knowledge
Tracing_ (BKT) is the most popular (Corbett and Anderson, 1994), which models
the knowledge tracing problem as predicting the state of a dynamical system
that has hidden latent variables (i.e. learning concepts). In addition to BKT,
probabilistic graphical models such as Hidden Markov Models (HMMs) (Corbett
and Anderson, 1994; Baker et al., 2008) or Bayesian belief networks (Villano,
1992) have also been used to model knowledge tracing. To keep the inference
computation tractable, traditional machine learning KT models use discrete
random state variables with simple transition regimes, which limits their
ability to represent complex dynamics between learning concepts. Moreover,
these models often assume a first-order Markov chain for an exercise sequence
(i.e. considering the most recent observation to be representing the whole
history) which also limits their ability to model long-term dependencies in an
exercise sequence.

Inspired by recent advances in deep learning (LeCun et al., 2015), several
deep learning KT models have been proposed. A pioneer work by Piech et al.
(Piech et al., 2015) reported _Deep Knowledge Tracing_ (DKT), which uses a
Recurrent Neural Networks (RNN) with Long Short-Term Memory (LSTM) units
(Graves et al., 2013b; Sutskever et al., 2014) to predict student performance
on new exercises given their past learning history. In DKT, a student’s
knowledge states are represented by a sequence of hidden states that
successively encode relevant information from past observations over time.
Although DKT has achieved substantial improvements in prediction performance
over BKT, due to the limitation of representing a knowledge state by one
hidden state, it lacks the ability to go deeper to trace how specific concepts
are mastered by a student (i.e., concept states) in a knowledge state. To deal
with this limitation, _Dynamic Key-Value Memory Networks_ (DKVMN) (Zhang et
al., 2017) was proposed to model a student’s knowledge state as a complex
function over all underlying concept states using a key-value memory. Their
idea of augmenting DKVMN with an auxiliary memory follows the concepts of
Memory-Augmented Neural Networks (MANN) (Santoro et al., 2016; Graves et al.,
2016). However, DKVMN acquires the knowledge growth through the most recent
exercise and thus fails to capture long-term dependencies in an exercise
sequence (i.e., relevant past experience to a new observation).

In this paper, we present a new KT model, called _Sequential Key-Value Memory
Networks_ (SKVMN). This model provides three advantages over the existing deep
learning KT models:

  * •

First, SKVMN unifies the strengths of both recurrent modelling capacity of DKT
and memory capacity of DKVMN for modelling student learning. We observe that,
although a key-value memory can help trace concept states of a student, it is
not effective in modeling long-term dependencies on sequential data. We remedy
this issue by incorporating LSTMs into the sequence modelling for a student’s
knowledge states over time. Thus, SKVMN is not only augmented with a key-value
memory to enhance representation capability of knowledge states at each time
step, but also can provide recurrent modelling capability for capturing
dependencies among knowledge states at different time steps in a sequence.

  * •

Second, SKVMN uses a modified LSTM with hops, called _Hop-LSTM_ , in its
sequence modelling. Hop-LSTM deviates from the standard LSTM architecture by
using a triangular layer for discovering sequential dependencies between
exercises in a sequence. Then, the model may hop across LSTM cells according
to the relevancy of the latent learning concepts. This enables relevant
exercises that correlate to similar concepts to be processed together. In
doing so, the inference becomes faster and the capacity of capturing long-term
dependencies in an exercise sequence is enhanced.

  * •

Third, SKVMN improves the write process of DKVMN in order to better represent
knowledge states stored in a key-value memory. In DKVMN, the current knowledge
state is not considered when calculating the knowledge growth of a new
exercise. This means that the previous learning experience is ignored. For
example, when a student attempts the same question multiple times, the same
knowledge growth would be added to the knowledge state, regardless of whether
the student has previously answered this question or how many times the
answers were correct. SKVMN solves this issue by using a summary vector as
input for the write process, which reflects both the current knowledge state
of a student and the prior difficulty of a new question.

We have extensively evaluated our proposed model SKVMN on five well-
established KT benchmark datasets, and compared it with the state-of-the-art
KT models. The experimental results show that (1) SKVMN outperforms the
existing KT models, including DKT and DKVMN, on all five datasets, (2) SKVMN
can better discover the correlation between latent concepts and questions, and
(3) SKVMN can the knowledge state of students dynamics, and leverage
sequential dependencies between exercises in an exercise sequence for improved
predication accuracy.

(a)

Sequential
---
dependencies
$q_{1}\leftarrow q_{4}$
$q_{2}\leftarrow q_{3}$
$q_{5}\leftarrow q_{3}$
…
$q_{4}\leftarrow q_{t}$

(b)

Figure 2. An illustration of the SKVMN model at time step $\mathrm{t}$, where
$\mathrm{q_{t}}$ is the input question, the exercise history
$\mathrm{X}=\langle\mathrm{\left(q_{1},y_{1}\right),\left(q_{2},y_{2}\right),\dots,\left(q_{t-1},y_{t-1}\right)\rangle}$,
and each $\mathbf{f_{i}}$ is the summary vector of the question
$\mathrm{q_{i}}$ for $\mathrm{i\in[1,t]}$: (a) the model has four layers,
namely the embedding, memory, sequence and output layers; and (b) sequential
dependencies among questions in $\mathrm{X}$.

Figure 1 illustrates how a student’s knowledge states are evolving as the
student attempts a sequence of 50 exercises in DKVMN and SKVMN. We can see
that, compared with the knowledge states of DKVMN depicted in Figure 1.(a),
our model SKVMN provides a smoother transition between two successive concept
states as depicted in Figure 1.(b). Moreover, SKVMN captures a smooth,
progressive evolution of knowledge states over time (i.e., through this
exercise sequence), which more accurately reflects the way a student learns
(will be discussed in detail in Section 5.4).

The reminder of this paper is organized as follows. Section 2 defines the
knowledge tracing problem. Section 3 presents our proposed KT model SKVMN.
Sections 4 and 5 discuss the experimental design and results. The related work
is presented in Section 6. We conclude the paper in Section 7.

##  2\. Problem Formulation

Broadly speaking, knowledge tracing is to track down students’ knowledge
states over time through a sequence of observations on how the students
interact with given learning activities. In this paper, we formulate knowledge
tracing as a sequence prediction problem in machine learning, which learns the
knowledge state of a student based on an exercise answering history.

Let $\mathrm{Q}=\left\\{q_{1},\ldots,q_{|Q|}\right\\}$ be the set of all
distinct question tags in a dataset. Each $\mathrm{q_{i}\in Q}$ may have a
different level of difficulty, which is not explicitly provided. An _exercise_
$\mathrm{x_{i}}$ is a pair $\mathrm{\left(q_{i},y_{i}\right)}$ consisting of a
question tag $\mathrm{q_{i}}$ and a binary variable
$\mathrm{\mathrm{y_{i}\in\\{0,1\\}}}$ representing the answer, where $0$ means
that $\mathrm{q_{i}}$ is incorrectly answered and $1$ means that
$\mathrm{q_{i}}$ is correctly answered. When a student interacts with the
questions in $\mathrm{Q}$, a history of exercises
$\mathrm{X}=\langle\mathrm{x}_{1},x_{2},\ldots,x_{t-1}\rangle$ undertaken by
the student can be observed. Based on a history of exercises
$\mathrm{X}=\langle\mathrm{x}_{1},x_{2},\ldots,x_{t-1}\rangle$, we want to
predict the probability of correctly answering a new question at time step
$\mathrm{t}$ by the student, i.e.,
$\mathrm{p_{t}=\left(y_{t}=1|q_{t},\mathrm{X}\right)}$.

We assume that the questions in $\mathrm{Q}$ are associated with $\mathrm{N}$
latent concepts $\mathrm{C}$. The _concept state_ of each latent concept
$\mathrm{c^{i}\in C}$ is a random variable describing the mastery level of the
student on this latent concept. At each time step $\mathrm{t}$, the _knowledge
state_ of a student is modelled as a set of all concept states of the student,
each corresponding to a latent concept in $\mathrm{C}$ at time step
$\mathrm{t}$.

##  3\. Sequential Key-Value Memory Networks

In this section, we introduce our model _Sequential Key-Value Memory Networks_
(SKVMN). We first present an overview for SKVMN. Then, we show how a key-value
memory can be attended, read and written in our model. To leverage sequential
dependencies among latent concepts for predication, we then present a modified
LSTMs, called _Hop-LSTMs_. Lastly, we discuss the optimisation techniques used
in the model.

###  3.1. Model Overview

The SKVMN model is augmented with a key-value memory
$\langle\mathbf{M^{k}},\mathbf{M^{v}}\rangle$ following the work in (Zhang et
al., 2017), i.e., a pair of one static matrix $\mathbf{M^{k}}$ of size
$\mathrm{N\times d_{k}}$, called the _key matrix_ , and one dynamic matrix
$\mathbf{M^{v}}$ of size $\mathrm{N\times d_{v}}$, called the _value matrix_.
Both the key matrix and the value matrix have the same $N$ memory slots, but
they may differ in their state dimensions $\mathrm{d_{k}}$ and
$\mathrm{d_{v}}$. The key matrix stores the latent concepts underlying
questions, and the value matrix stores the concept states of a student (i.e.,
the knowledge state) which can be changed dynamically based on student
learning.

Given an input question $\mathrm{q_{t}}$ at time step $\mathrm{t}$, the SKVMN
model retrieves the knowledge state of a student from the key-value memory
$\langle\mathbf{M^{k}},\mathbf{M^{v}}\rangle$, and predicts the probability of
correctly answering the question $\mathrm{q_{t}}$ by the student. Figure 2.(a)
illustrates the SKVMN model at time step $\mathrm{t}$, which consists of four
layers: the embedding, memory, sequence and output layers.

  * •

The embedding layer is responsible for mapping an input question at time step
$\mathrm{t}$ into a high-dimensional vector space.

  * •

The memory layer involves two processes: attention and read, where the
attention process provides an addressing mechanism for the input question
$\mathrm{q_{t}}$ to allocate the relevant information from the key-value
memory, and the read process uses the attention vector to retrieve the current
knowledge state of the student from the value matrix $\mathbf{M^{v}_{t}}$. The
details of the attention and read processes will be discussed in Sections
3.2.1 and 3.2.2.

  * •

The sequence layer consists of a set of recurrently connected LSTM cells,
where LSTM cells are connected based on their sequential dependencies
determined by a Triangular layer as depicted in Figure 2.(b). The details of
the sequence layer will be discussed in Section 3.3.

  * •

The output layer generates the probability of correctly answering the input
question $\mathrm{q_{t}}$.

After a student has attempted the input question $\mathrm{q_{t}}$ with the
answer $\mathrm{y_{t}}$, the value matrix in the key-value memory of the SKVMN
model needs to be updated in order to reflect the latest knowledge state of
the student. Figure 3 depicts how the value matrix is transited from
$\mathbf{M_{t}^{v}}$ at time step $\mathrm{t}$ to $\mathbf{M_{t+1}^{v}}$ at
time step $\mathrm{t+1}$ using the write process. The details of the write
process will be discussed in Section 3.2.3.

###  3.2. Attention, Read and Write

There are three processes relating to access to a key-value memory in our
model: attention, read and write. In the following, we elaborate these
processes.

####  3.2.1. Attention

Given a question $\mathrm{q_{t}}\in Q$ as input, the model represents
$\mathrm{q_{t}}$ as a “one-hot” vector of length $\mathrm{|Q|}$ in which all
entries are zero, except for the entry that corresponds to $\mathrm{q_{t}}$.

In order to map $\mathrm{q_{t}}$ into a continuous vector space,
$\mathrm{q_{t}}$ is multiplied by an embedding matrix
$\mathbf{A\mathrm{\in\mathbb{R\mathrm{{}^{|Q|\times d_{k}}}}}}$, which
generates a continuous embedding vector
$\mathbf{k_{t}}\in\mathbb{R\mathrm{{}^{d_{k}}}}$, where $\mathrm{d_{k}}$ is
the embedding dimension. Then, an attention vector $\mathbf{w_{t}}$ is
obtained by applying the Softmax function to the inner product between the
embedding vector $\mathbf{k_{t}}$ and each key slot
$\mathbf{M^{k}\mathrm{(i)}}$ in the key matrix $\mathbf{M^{k}}$ as follows:

(1)
$\mathrm{\mathbf{w_{t}}(i)=Softmax(\mathrm{\mathbf{k_{t}^{T}}}\mathbf{M^{k}}(i))}$

where $\mathrm{Softmax(z_{i})=e^{z_{i}}/\sum_{j}e^{z_{j}}}$. Conceptually,
$\mathbf{w_{t}}$ represents the correlation between the question
$\mathrm{q_{t}}$ and the underlying latent concepts stored in the key matrix
$\mathbf{M^{k}}$.

####  3.2.2. Read

For each exercise $\mathrm{x_{t}=(q_{t},y_{t})}$, the model uses its
corresponding attention vector $\mathbf{w_{t}}$ to retrieve the concept states
of the student with regard to the question $\mathrm{q_{t}}$ from the value
matrix $\mathbf{M_{t}^{v}}$. Specifically, the read process takes the
attention vector $\mathbf{w_{t}}$ as input and yields a read vector
$\mathbf{r_{t}}$ which is the weighted sum of all values being attended by
$\mathbf{w_{t}}$ in the memory slots of the value matrix $\mathbf{M_{t}^{v}}$,
i.e.,

(2)
$\mathbf{r_{t}}=\mathrm{\sum_{i=1}^{N}}\mathbf{\mathrm{w}_{t}}\mathrm{(i)}\mathbf{M_{t}^{v}\mathrm{(i)}}$

The read vector $\mathbf{r_{t}}$ is concatenated with the embedding vector
$\mathbf{k_{t}}$. Then, the combined vector is fed to a Tanh layer to
calculate the summary vector $\mathbf{f_{t}}$:

(3)
$\mathrm{\mathbf{f_{t}}=Tanh(\mathbf{W_{\mathrm{1}}^{\mathrm{T}}\mathrm{\left[\mathbf{r_{t},k_{t}}\right]+b_{1})}}}$

where $\mathrm{Tanh(z_{i})=(e^{z_{i}}-e^{-z_{i}})/(e^{z_{i}}+e^{-z_{i}})}$,
$\mathbf{W}_{1}$ is the weight matrix of the Tanh layer, and $\mathbf{b}_{1}$
is the bias vector. While the read vector $\mathbf{r_{t}}$ represents the
student’s knowledge state with respect to the relevant concepts of the current
question $\mathrm{q_{t}}$, the summary vector $\mathbf{f_{t}}$ adds prior
information of the question (e.g., the level of difficulty) to this knowledge
state. Intuitively, $\mathbf{f_{t}}$ represents how well the student has
mastered the latent concepts relevant to the question $\mathrm{q_{t}}$ before
attempting this question.

Figure 3. An illustration of the write process in the SKVMN model, which
transits the value matrix from $\mathbf{M_{t}^{v}}$ at time step $\mathrm{t}$
to $\mathbf{M_{t+1}^{v}}$ at time step $\mathrm{t+1}$, where
$\mathrm{(\mathbf{f_{t}},y_{t})}$ and $\mathbf{w_{t}}$ are the input to the
write process at time step $\mathrm{t}$.

####  3.2.3. Write

The write process occurs each time after the student has attempted a question.
The purpose of this write process is to update the concept states of the
student in the value matrix $\mathbf{M_{t}^{v}}$ using the knowledge growth
gained through attempting the question $\mathrm{q_{t}}$. This update leads to
the transition of the value matrix from from $\mathbf{M_{t}^{v}}$ to
$\mathbf{{M}_{t+1}^{v}}$ as depicted in Figure 3.

To calculate the knowledge growth, our model considers not only the
correctness of the answer $\mathrm{y_{t}}$ for $\mathrm{q_{t}}$, but also the
student’s mastery level of the concept states $\mathbf{f_{t}}$ before
attempting $\mathrm{q_{t}}$. Each $\mathrm{(\mathbf{f_{t}},y_{t})}$ is
represented as a vector of length $\mathrm{2|\mathrm{Q}|}$ and multiplied by
an embedding matrix $\mathbf{B\mathrm{\in\mathbb{R\mathrm{{}^{2|Q|\times
d_{v}}}}}}$ to get a write vector $\mathbf{v_{t}}$, which represents the
knowledge growth of the student obtained by attempting the question.

Similar to other memory augmented networks (Graves et al., 2016; Zhang et al.,
2017), the write process proceeds with two gates: _erase gate_ and _add gate_.
The former controls what information to erase from the knowledge state after
attempting the latest exercise $\mathrm{x_{t}}$, while the latter controls
what information to add into the knowledge state. From the knowledge tracing
perspective, these two gates capture the forgetting and enhancing aspects of
learning knowledge, respectively.

With the write vector $\mathbf{v_{t}}$ for the knowledge growth, an erase
vector $\mathbf{e_{t}}$ is calculated as:

(4)
$\mathrm{\mathbf{e_{t}}=sigmoid(\mathbf{W_{\mathrm{e}}^{\mathrm{T}}\cdot\mathrm{\mathbf{v_{t}}+b_{e})}}}$

where $\mathrm{\mathrm{Sigmoid(z_{i})=1/(1+e^{-z_{i}})}}$ and $\mathbf{W}_{e}$
is the weight matrix. Using the attention vector $\mathbf{w_{t}}$, the value
matrix $\mathbf{\tilde{M}_{t+1}^{v}}$ after applying the erase vector
$\mathbf{e_{t}}$ is

(5)
$\mathbf{\tilde{M}_{t+1}^{v}\mathrm{(i)}}=\mathbf{M_{t}^{v}}\mathrm{(i)[1-\mathbf{w_{t}}(i)\mathbf{e_{t}}]}.$

Then, an add vector $\mathbf{a_{t}}$ is calculated by

(6)
$\mathrm{\mathbf{a_{t}}=Tanh(\mathbf{W_{\mathrm{a}}^{\mathrm{T}}\cdot\mathrm{\mathbf{v_{t}}+b_{a})}}}$

where $\mathbf{W}_{a}$ is the weight matrix. Finally, the value matrix
$\mathbf{M_{t+1}^{v}}$ for the next knowledge state is updated as:

(7)
$\mathbf{M_{t+1}^{v}\mathrm{(i)}}=\mathbf{\tilde{M}_{t+1}^{v}}\mathrm{(i)+\mathbf{w_{t}}(i)\mathbf{a_{t}}}$

###  3.3. Sequence Modelling

Now we discuss the sequence modelling approach used at the sequence layer for
predicting the probability of correctly answering the question
$\mathrm{q_{t}}$ based on the exercise history.

####  3.3.1. Sequential dependencies

An exercise history $\mathrm{X}$ of a student may contain a long sequence of
exercises, for example, the average sequence length in the ASSISTments2009
dataset is $233\pm 100$ questions per sequence. However, as different
exercises may correlate to different latent concepts, not all exercises in
$\mathrm{X}$ can equally contribute to the prediction of answering a given
question $\mathrm{q_{t}}$. Thus, we observe that, by hopping across irrelevant
exercises in $\mathrm{X}$ with regard to $\mathrm{q_{t}}$, recurrent models
can be applied on a shorter and more relevant sequence, leading to more
efficient and accurate prediction performance.

For each question in $\mathrm{Q}$, since its attention vector reflects the
correlation between this question and the latent concepts in $\mathrm{C}$, we
consider that the similarity between two attention vectors can provide a good
indication of how their corresponding questions are relevant in terms of their
correlations with latent concepts. For example, suppose that we have three
latent concepts $\mathrm{C=\\{c^{1},c^{2},c^{3}\\}}$, and two attention
vectors $\mathrm{\mathbf{w_{1}}=[0.15,0.25,0.6]^{T}}$ and
$\mathrm{\mathbf{w_{2}}=[0.2,0.3,0.5]^{T}}$ that correspond to the questions
$\mathrm{q_{1}}$ and $\mathrm{q_{2}}$, respectively. Then $\mathrm{q_{1}}$ and
$\mathrm{q_{2}}$ are considered as being relevant if both $\mathbf{w_{1}}$ and
$\mathbf{w_{2}}$ are mapped to a vector $\mathrm{[0,0,1]^{T}}$ where $0$, $1$
and $2$ refer to the value ranges “low”, “middle” and “high”, respectively.

Now, the question is: how to identify similar attention vectors? For this, we
use the triangular membership function (Klir and Yuan, 1995):

(8) $\mu(x)=max(min(\frac{x-a}{b-a},\frac{c-x}{c-b}),0),$

where the parameters $a$ and $c$ determine the feet and the parameter $b$
determines the peak of a triangle. We use three triangular membership
functions for three value ranges: low (0), medium (1), and high (2). Each
real-valued component in an attention vector is mapped to one of the three
value ranges. Each attention vector $\mathbf{w_{t}}$ is associated with an
identity vector $\mathbf{d_{t}}$. Similar attention vectors have the same
identity vector, while dissimilar attention vectors have different identity
vectors.

Then, at each time step $\mathrm{t}$, for the current question
$\mathrm{q_{t}}$ and an exercise history $\mathrm{X}$, we say $\mathrm{q_{t}}$
is _sequentially dependent_ on $\mathrm{q_{t-\lambda}}$ in $\mathrm{X}$ with
$\lambda\in(0,t)$, denoted as
$\mathrm{q_{t-\lambda}}\leftarrow\mathrm{q_{t}}$, if the following two
conditions are satisfied:

  * •

The attention vectors of $\mathrm{q_{t-\lambda}}$ and $\mathrm{q_{t}}$ have
the same identity vector, i.e., $\mathbf{d}_{t-\lambda}=\mathbf{d_{t}}$, and

  * •

There is no other $\mathrm{q_{j}}$ in $\mathrm{X}$ such that
$\mathbf{d_{j}}=\mathbf{d_{t}}$ and $\mathrm{j>t}-\lambda$, i.e.,
$\mathrm{q_{t-\lambda}}$ is the most recent exercise in $\mathrm{X}$ that is
relevant to $\mathrm{q_{t}}$.

Over time, given a sequence of questions
$\langle\mathrm{q_{1}},\mathrm{q_{2}},\dots,\mathrm{q_{|Q|}}\rangle$, we thus
have an exercise history $\mathrm{X^{\prime}}$ in which exercises are
partitioned into a set of subsequences
$\mathrm{\\{X^{\prime}_{1},\dots,X^{\prime}_{n}\\}}$ with
$\mathrm{\Sigma_{k=1}^{n}|X^{\prime}_{k}|=|X^{\prime}|}$ and, for any two
consecutive exercises $\mathrm{\left(q_{i},y_{i}\right)}$ and
$\mathrm{\left(q_{j},y_{j}\right)}$ in the subsequence
$\mathrm{X^{\prime}_{k}}$, $\mathrm{q_{j}}$ is sequentially dependent on
$\mathrm{q_{i}}$, i.e. $\mathrm{q_{i}\leftarrow q_{j}}$. In Figure 2, based on
the sequential dependencies presented in Figure 2.(b), $\mathrm{X}$ is
partitioned into $\mathrm{\langle q_{1},q_{4},\dots,q_{t}\rangle}$ and
$\mathrm{\langle q_{2},q_{3},q_{5},\dots\rangle}$, which correspond to the
recurrently connected LSTM cells depicted in Figure 2.(a). We will discuss
further details in the following.

####  3.3.2. Hop-LSTM

Based on sequence dependencies between exercises in a sequence, a modified
LSTM with hops, called Hop-LSTM, is used to predict the probability of
correctly answering a new question by a student. Different from the standard
LSTM architecture, Hop-LSTM allows us to recurrently connect the LSTM cells
for questions based on the relevance of their latent concepts. More precisely,
two LSTM cells in Hop-LSTM are connected only if the input question of one
LSTM cell is sequentially dependent on the input question of the other LSTM
cell. This means that Hop-LSTM has the capability of hopping across the LSTM
cells when their input questions are irrelevant to the current question
$\mathrm{q_{t}}$.

Formally, at each time step $\mathrm{t}$, for the question $\mathrm{q_{t}}$,
if there is an exercise
$\mathrm{\left(q_{t-\lambda},y_{t-\lambda}\right)}\in\mathrm{X}$ with
$\lambda\in(0,t)$ and $\mathrm{q_{t-\lambda}\leftarrow q_{t}}$, then the
current LSTM cell takes the summary vector $\mathbf{f}_{t}$ and the hidden
state $\mathbf{h}_{t-\lambda}$ as input. Moreover, this LSTM cell updates the
cell state $\mathbf{c}_{t-\lambda}$ into the cell state $\mathbf{c}_{t}$, and
generates the new hidden state $\mathbf{h}_{t}$ as output. As in (Hochreiter
and Schmidhuber, 1997), the LSTM cell used in our work has three gates:
_forget gate_ $\mathbf{g}_{t}$, _input gate_ $\mathbf{i}_{t}$, and _output
gate_ $\mathbf{o}_{t}$ in addition to the hidden state $\mathbf{h}_{t}$ and
the cell state $\mathbf{c}_{t}$:

(9)
$\displaystyle\mathbf{g}_{t}=Sigmoid(\mathbf{W}_{g}[\mathbf{h}_{t-\lambda},\mathbf{f}_{t}]+\mathbf{b}_{g})$
(10)
$\displaystyle\mathbf{i}_{t}=Sigmoid(\mathbf{W}_{i}[\mathbf{h}_{t-\lambda},\mathbf{f}_{t}]+\mathbf{b}_{i})$
(11)
$\displaystyle\mathbf{o}_{t}=Sigmoid(\mathbf{W}_{o}[\mathbf{h}_{t-\lambda},\mathbf{f}_{t}]+\mathbf{b}_{o})$
(12)
$\displaystyle\mathbf{\tilde{c}}_{t}=Tanh(\mathbf{W}_{c}[\mathbf{h}_{t-\lambda},\mathbf{f}_{t}]+\mathbf{b}_{c})$
(13)
$\displaystyle\mathbf{c}_{t}=\mathbf{g}_{t}\odot\mathbf{c}_{t-\lambda}+\mathbf{i}_{t}\odot\mathbf{\tilde{c}}_{t}$
(14) $\displaystyle\mathbf{h}_{t}=\mathbf{o}_{t}\odot Tanh(\mathbf{c}_{t})$

Then, the output vector $\mathbf{h}_{t}$ of the curent LSTM cell is sent to a
Sigmoid layer, which calculates the probability $\mathrm{p_{t}}$ of correctly
answering the current question $\mathrm{q_{t}}$ by

(15) $\displaystyle
p_{t}=Sigmoid(\mathbf{W}_{2}^{T}\cdot\mathbf{h}_{t}+\mathbf{b}_{2}).$

###  3.4. Model Optimisation

To optimise the model, we use the cross-entropy loss function between the
predicted probability of being correctly answered $\mathrm{p_{t}}$ and the
true answer $\mathrm{y_{t}}$. The following objective function is defined over
training data:

(16) $\mathcal{L\mathrm{\mathrm{=-\sum_{t}(y_{t}\log
p_{t}+(1-y_{t})\log(1-p_{t}))}}}$

We initialise of the memory matrices ($\mathbf{M_{t}^{k}}$ and
$\mathbf{M_{t}^{v}}$) and embedding matrices ($\mathbf{A}$ and $\mathbf{B}$)
using a random Gaussian distribution $N(0,\sigma)$. While for weights and
biases of the neural layers, we use Glorot uniform random initialization
(Glorot and Bengio, 2010) for a faster convergence. These randomly initialized
parameters are optimized using the stochastic gradient decent (SGD) mechanism
(Bottou, 2012). As we use Hop-LSTM at the sequence layer, during the
backpropagation of the gradients, only the parameters of the connected LSTM
cells (i.e. the ones responsible for the current prediction error) are
updated. Other parameters, such as the embedding matrices, weight matrices,
and bias vectors, are updated in each backpropagation iteration based on loss
function values.

Note that, through training, the model can discover relevant latent concepts
for each question and store their state values in the value matrix
$\mathbf{M^{v}}$.

##  4\. Experiments

In this section, we present the experiments of evaluating our proposed model
SKVMN against the state-of-the-art KT models. These experiments aim to answer
the following research questions:

  * RQ1:

What is the optimal size for a key-value memory (i.e., the key and value
matrices) of SKVMN?

  * RQ2:

How does SKVMN perform on predicting a student’s answers of new questions,
given an exercise history?

  * RQ3:

How does SKVMN perform on discovering the correlation between latent concepts
and questions?

  * RQ4:

How does SKVMN perform on capturing the evolution of a student’s knowledge
states?

###  4.1. Datasets

We use five well-established datasets in the KT literature (Khajah et al.,
2016; Zhang et al., 2017; Piech et al., 2015). Table 2 summarizes the
statistics of the data sets.

  * •

Synthetic-5111Synthetic-5:https://github.com/chrispiech/DeepKnowledgeTracing/tree/master
/data/synthetic: This dataset consists of two subsets: one for training and
one for testing. Each subset contains $50$ distinct questions which were
answered by $4,000$ virtual students. A total number of $200,000$ exercises
(i.e. $\mathrm{(q_{t},y_{t})}$) are contained in the dataset.

  * •

ASSISTments2009222ASSISTments2009:https://sites.google.com/site/assistmentsdata/home/assistment-2009-2010-data/skill-
builder-data-2009-2010: This dataset was collected during the school year
$2009-2010$ using the ASSISTments online education website
333https://www.assistments.org/. The dataset consists of $110$ distinct
questions answered by $4,151$ students which gives a total number of $325,637$
exercises.

  * •

ASSISTments2015444ASSISTments2015:https://sites.google.com/site/assistmentsdata/home/2015-assistments-
skill-builder-data: As an update to the ASSISTments2009 dataset, this dataset
was released in $2015$. It includes $100$ distinct questions answered by
$19,840$ students with a total number of $683,801$ exercises. This dataset has
the largest number of students among the other datasets. Albeit, the average
number of exercises per student is low. The original dataset also has some
incorrect answer values (i.e. $\mathrm{\mathrm{y_{i}\notin\\{0,1\\}}}$), which
are removed during preprocessing.

  * •

Statics2011555Statics2011:https://pslcdatashop.web.cmu.edu/
DatasetInfo?datasetId=507: This datasets was collected from a statistics
course at Carnegie Mellon University during Fall $2011$. It contains $1,223$
distinct questions answered by $333$ undergraduate students with a total
number of $189,297$ exercises. This dataset has the highest exercise per
student ratio among all datasets.

  * •

JunyiAcademy666Junyi2015:
https://datashop.web.cmu.edu/DatasetInfo?datasetId=1198: This dataset was
collected from Junyi Academy 777https://www.junyiacademy.org/, which is an
education website of providing learning materials and exercises on various
scientific courses, on 2015 (Chang et al., 2015). It contains $722$ distinct
questions answered by $199,549$ students with a total number of $25,628,935$
exercises. It is the largest dataset in terms of the number of exercises.

Table 1. Dataset statistics

Dataset | #Questions | #Students | #Exercises | #Exercises
---|---|---|---|---
per student
Synthetic-5 | $50$ | $4,000$ | $200,000$ | $50$
ASSISTments2009 | $110$ | $4,151$ | $325,637$ | $78$
ASSISTments2015 | $100$ | $19,840$ | $683,801$ | $34$
Statics2011 | $1,223$ | $333$ | $189,297$ | $568$
JunyiAcademy | $722$ | $199,549$ | $25,628,935$ | $128$

Table 2. Comparison of SKVMN with DKVMN under different numbers of memory
slots $\mathrm{N}$ and state dimensions $\mathrm{d}$, where $\mathrm{m}$
refers to the number of parameters in each setting.

Dataset | $\mathrm{d}$ | $\mathrm{N}$ | SKVMN | DKVMN
---|---|---|---|---
AUC (%) | m | AUC (%) | m
Synthetic-5 | 10 | 50 | 83.11 | 15K | 82.00 | 12k
50 | 50 | 83.67 | 30k | 82.66 | 25k
100 | 50 | 84.00 | 57k | 82.73 | 50k
200 | 50 | 83.73 | 140k | 82.71 | 130k
ASSISTments2009 | 10 | 10 | 83.63 | 7.8k | 81.47 | 7k
50 | 20 | 82.87 | 35k | 81.57 | 31k
100 | 10 | 82.72 | 71k | 81.42 | 68k
200 | 20 | 82.63 | 181k | 81.37 | 177k
ASSISTments2015 | 10 | 20 | 74.84 | 16k | 72.68 | 14k
50 | 10 | 74.50 | 31k | 72.66 | 29k
100 | 50 | 74.24 | 66k | 72.64 | 63k
200 | 50 | 74.20 | 163k | 72.53 | 153k
Statics2011 | 10 | 10 | 84.50 | 92.8k | 82.72 | 92k
50 | 10 | 84.85 | 199k | 82.84 | 197k
100 | 10 | 84.70 | 342k | 82.71 | 338k
200 | 10 | 84.76 | 653k | 82.70 | 649k
JunyiAcademy | 10 | 20 | 82.50 | 16k | 79.63 | 14k
50 | 10 | 82.41 | 31k | 79.48 | 29k
100 | 50 | 82.67 | 66k | 79.54 | 63k
200 | 50 | 82.32 | 163k | 80.27 | 153k

###  4.2. Baselines

In order to evaluate the performance of our proposed model, we select the
following three KT models as the baselines:

  * –

Bayesian knowledge tracing (BKT) (Corbett and Anderson, 1994), which is based
on Bayesian inference in which a knowledge state is modelled as a set of
binary variables, each representing the understanding of a single concept.

  * –

Deep knowledge tracing (DKT) (Piech et al., 2015) which uses recurrent neural
networks to model student learning.

  * –

Dynamic key-value memory networks (DKVMN) (Zhang et al., 2017) which extends
the memory-augmented neural networks (MANN) by a key-value memory and is
considered as the state-of-the-art model for knowledge tracing.

Our proposed model is referred to as SKVMN in the experiments.

###  4.3. Measures

We use the area under the Receiver Operating Characteristic (ROC) curve,
referred to as AUC (Ling et al., 2003), to measure the prediction performance
of the KT models. The AUC ranges in value from 0 to 1. An AUC score of $0.5$
means random prediction (i.e. coin flipping). The higher an AUC score goes
above $0.5$, the more accurately a predictive model can perform.

Figure 4. The ROC curve results of the four models BKT, DKT, DKVMN, and SKVMN
over five datasets: (a) ASSISTments2009, (b) ASSISTments2015, (c) Statics2011,
(d) Synthetic-5, and (e) JunyiAcademy.

###  4.4. Evaluation Settings

We divided each dataset into $70$% for training and validation and $30$% for
testing, except for Synthetic-5. This is because, as mentioned before,
Synthetic-5 itself contains the training and test subsets of the same size.
For each training and validation subset, we further divided it using the
5-fold cross validation (e.g. $80$% for training and $20$% for validation).
The validation subset was used to determine the optimal values for the
hyperparameters, including the memory slot dimensions
$\mathrm{\mathrm{d}_{k}}$ for the key matrix and $\mathrm{\mathrm{d}_{v}}$ for
the value matrix.

Table 3. The AUC results of the four models BKT, DKT, DKVMN, and SKVMN over
all datasets.

Dataset | BKT | DKT | DKVMN | SKVMN
---|---|---|---|---
Synthetic-5 | $62.0\pm 0.02$ | $80.3\pm 0.1$ | $82.7\pm 0.1$ | $\mathbf{84.0\pm 0.04}$
ASSISTments2009 | $63.1\pm 0.01$ | $80.5\pm 0.2$ | $81.6\pm 0.1$ | $\mathbf{83.6\pm 0.06}$
ASSISTments2015 | $64.2\pm 0.03$ | $72.5\pm 0.1$ | $72.7\pm 0.1$ | $\mathbf{74.8\pm 0.07}$
Statics2011 | $73.0\pm 0.01$ | $80.2\pm 0.2$ | $82.8\pm 0.1$ | $\mathbf{84.9\pm 0.06}$
JunyiAcademy | $65.0\pm 0.02$ | $79.2\pm 0.1$ | $80.3\pm 0.4$ | $\mathbf{82.7\pm 0.01}$

We utilised the Adam optimizer (Kingma and Ba, 2015) for SGD implementation
with momentum of $0.9$ and learning rate $\gamma$ of $0.01$ annealed using a
cosine function every $15$ epochs for $120$ epochs, then it remains fixed at
$0.001$. The LSTM gradients were clipped to improve the training (Pascanu et
al., 2013). For the other baselines, we follow the optimisation procedures
indicated in the original work for each of them (Corbett and Anderson, 1994;
Zhang et al., 2017; Piech et al., 2015).

A mini-batch of $32$ is selected during the training for all datasets, except
Synthetic-5, for which we use a mini-batch of $8$ due to the relatively small
number of training samples (i.e. exercises) in the dataset (Bengio, 2012). For
each dataset, the training process is repeated five times, each time using a
different initialization. We report the average test AUC and the standard
deviation over these five runs.

For the Triangular layer, the hyper-parameter values (${a,b,c}$) of each
triangular membership function are set based on the empirical analysis of each
dataset.

##  5\. Results and Discussion

In this section, we present the experimental results and discuss our
observations from the obtained results.

###  5.1. Hyperparameters $\mathrm{N}$ and $\mathrm{d}$

To explore how the sizes of the key and value matrices can affect the model
performance, we have conducted experiments to compare SKVMN with DKVMN under
different numbers of memory slots $\mathrm{N}$ and state dimensions
$\mathrm{d}$, where $\mathrm{d=d_{k}=d_{v}}$ so as to be consistent with the
previous work (Zhang et al., 2017). In order to allow a fair comparison
between the models SKVMN and DKVMN, we select the same set of state
dimensions, (i.e. $\mathrm{d}=10,50,100,200$) and the same corresponding
numbers of memory slots $\mathrm{N}$ on the datasets Synthetic-5,
ASSISTments2009, ASSISTments2015 and Statics2011 to report the AUC results,
following the settings originally reported in (Zhang et al., 2017). For the
dataset JunyiAcademy, it was not considered in the previous work (Zhang et
al., 2017). Considering that JunyiAcademy has the largest numbers of students
and exercises among all datasets, we use the same settings for the numbers of
memory slots and state dimensions as the ones for the second largest dataset
ASSISTments2015. Table 2 presents the AUC results for all five datasets.

As shown in Table 2, compared with DKVMN, our model SKVMN can produce better
AUC results with comparable parameters on the datasets Synthetic-5,
ASSISTments2015 and Statics2011, and with fewer parameters on the datasets
ASSISTments2009 and JunyiAcademy. Particularly, for the dataset
ASSISTments2009, SKVMN yields an AUC at 83.63% with N=10, d=10 and m=7.8k,
whereas DKVMN yields an AUC at 81.57% with N=20, d=50 and m=31k (nearly 4
times of 7.8k). Similarly, for the dataset JunyiAcademy, SKVMN yields an AUC
at 82.67% with N=50, d=100 and m=66k, whereas DKVMN yields an AUC at 80.27%
with N=50, d=200 and m=153k (more than twice of 66k).

Note that, the optimal value of $\mathrm{N}$ for ASSISTments2015 is higher
than the one for its previous version (i.e. ASSISTments2009). This implies
that the number of latent concepts $\mathrm{N}$ increases in ASSISTments2015
in comparison to ASSISTments2009. Moreover, the optimal value of $\mathrm{d}$
generally reflects the complexity of the exercises in a dataset, and the
dataset JunyiAcademy has exercises of higher complexity than other real-world
datasets.

###  5.2. Prediction Accuracy

We have conducted experiments on comparing the AUC results of our model SKVMN
with the other three KT models: BKT, DKT, and DKVMN. Table 3 presents the AUC
results of all the models. It can be seen that our model SKVMN outperformed
the other models over all the five datasets. Particularly, the SKVMN model
achieved an average AUC value that is at least 2% higher than the state-of-art
model DKVMN on all real-world datasets ASSISTments2009, ASSISTments2015,
Statics2011, and JunyiAcademy. Even for the only synthetic dataset (i.e.
Synthetic-5), the SKVMN model achieved an average AUC value of $84.0\pm 0.04$,
in comparison with $82.7\pm 0.1$ achieved by DKVMN. Note that the AUC values
on ASSISTments2015 are the lowest among all datasets, regardless of the KT
models. This reflects the difficulty of the KT task in this dataset due to its
lowest exercise per student ratio, which not only makes the training process
more difficult but also limits the effective use of sequence information to
enhance the prediction performance. Figure 4 illustrates the ROC curves of
these four models for each dataset.

(a)

(b)

Figure 5. Clustering results of questions in the dataset ASSISTments2009
using: (a) DKVMN, and (b) SKVMN, where questions in the same color are
correlating to the same latent concept. Table 4. Question descriptions in the
dataset ASSISTments2009, where the questions are clustered according to latent
concepts being discovered by SKVMN.

In a nutshell, based on the AUC results in Table 3, we have the following
observations. First, the neural models generally performed better than the
Bayesian inference model (i.e. BKT). This is due to the power of these models
in learning complex student learning patterns without the need to oversimplify
the problem’s assumption to keep it within the tractable computation limits as
the case in BKT. Second, the memory-augmented models DKVMN and SKVMN performed
better than the DKT model that does not utilise an external memory structure.
This has empirically verified the effectiveness of external memory structures
in storing past learning experiences of students, as well as facilitating the
access of relevant information to enhance the prediction performance. Third,
the use of sequential dependencies among exercises in our SKVMN model enhanced
the prediction accuracy in comparison to the DKVMN model which primarily
considers the latest observed exercise.

###  5.3. Clustering Questions

To provide insights on how our proposed model SKVMN can correlate questions to
their latent concepts, In Figure 5.(a)-5.(b), we present the clustering
results of questions based on their correlated concepts in the dataset
ASSISTments2009, generated by using DKVMN and SKVMN, respectively. This
dataset was selected for two reasons. First, it has a reasonable number of
questions (i.e., 110), enabling the visualization of clusters to be readable.
Second, each question in this dataset is provided with a description as
depicted in Table 4, which is useful for validating how well the model
discovers correlations between questions and their latent concepts.

As shown in Figure 5, both DKVMN and SKVMN discover that there are 10 latent
concepts relating to the 110 questions in the dataset ASSISTments2009, where
all questions in one cluster relate to common latent concepts and are labelled
using the same color. It can be noticed that SKVMN performs significantly
better than DKVMN since the overlapping between different clusters in SKVMN is
smaller than in DKVMN. For example, in Figure 5.(a), question 105 is about
curve slop, which is close to the cluster for geometric concepts in brown
colour, while it is placed in the cluster for equation system concepts in blue
colour. Similarly, other overlaps can be observed such as questions 38, 73,
and 26. This indicates that the effectiveness of SKVMN in discovering latent
concepts as well as discovering questions that relate to these latent
concepts.

We can further verify the effectiveness of SKVMN in discovering latent
concepts for questions using the question descriptions in Table 4. For
example, in Figure 5.(b), the questions $13$, $19$, and $30$ fall in the same
cluster in pink (top right corner). Their provided descriptions are
“Equivalent Fractions”,“Multiplication Fractions”, and “Ordering Fractions”,
respectively, which are all relevant to fractions concepts. Similarly, the
questions $1$, $81$, and $92$ have the descriptions “Area Trapezoid”, “Area
Rectangle”, and “Rotations”, respectively. These questions fall in the same
cluster in light blue (bottom right corner) because they are about geometric
concepts, such as area functions and transformations.

Note that, SKVMN depends on identity vectors to aggregate questions with
common concepts together. While the DKVMN depends on the attention vectors to
perform this aggregation.

###  5.4. Evolution of Knowledge States

As previously discussed in Section 1, the knowledge states of a student may
evolve over time, through learning from a sequence of exercises. In order to
illustrate this evolution process, Figure 1 shows a student’s knowledge states
over a sequence of 50 exercises from the ASSISTments2009 dataset. At each time
step, a knowledge state consists of the concept states of five concepts
$\mathrm{\\{c^{1},\dots,c^{5}\\}}$, which are stored in the value matrix of a
key-value memory augmented with DKVMN or SKVMN. Figure 1.(a) shows this
student’s knowledge states captured by DKVMN, while Figure 1.(b) shows this
student’s knowledge states captured by SKVMN.

In SKVMN, relevant questions are identified as shown in Table 4. Comparing
Figure 1.(a) and Figure 1.(b), it can be visually noticed that SKVMN has
smoother updates to the concept states in the value matrix than DKVMN. For
example, considering the questions 23, 33 and 61, the student answered the
first two questions incorrectly and the last one correctly, which result in a
sudden update to the value of $\mathrm{c^{3}}$ (i.e., the concept state of
$\mathrm{c^{3}}$) in the value matrix of DKVMN but a smoother update to the
concept state of $\mathrm{c^{3}}$ in the value matrix of SKVMN. Another
example is the questions 70 and 88 that correlate to same latent concepts, the
student answered the first one incorrectly and the second one correctly, which
resulted in a significant update to the concept state of $\mathrm{c^{3}}$ in
the DKVMN’s memory around indices 18, 19 and 20, while SKVMN’s concept state
of $\mathrm{c^{3}}$ decreased in a smoother manner. This means that SKVMN
considers the past performance of the student in relevance to this concept. At
its core, these differences in capturing concept states are due to the fact
that DKVMN’s write process only takes the question and the answer to calculate
the erase and add vectors, so that the knowledge state of DKVMN is biased to
the most recently observed question. SKVNM has resolved this issue by taking
into account the summary vector (i.e., current knowledge state and the level
of difficulty of the current question) for the write process.

##  6\. Related Work

In this section, we provide a brief review of related research work.

One of the early attempts for developing a KT model was introduced by Corbett
and Anderson (Corbett and Anderson, 1994). Their KT model, called Bayesian
Knowledge Tracing (BKT), assumed a knowledge state to be a binary random
variable (i.e. know or do not know) and followed a Bayesian inference approach
to estimate the values of knowledge states. However, BKT has limitations in
modelling dynamics between different concepts due to its oversimplified
representation to make the Bayesian inference tractable. Baker et al. (Baker
et al., 2008) extended BKT by introducing an additional layer to the Bayesian
inference to represent the contextual information. While their model achieved
better results, it was still considered only the latest observation as a
first-order Markov chain. Several attempts have been made to extend BKT by
individualizing the prior distribution of Bayesian inference parameters
(Pardos and Heffernan, 2010; Yudelson et al., 2013) so as to customize the
model for each individual student. These individualization techniques were
proved to reduce prediction errors of the original BKT model. Pardos and
Heffernan (Pardos and Heffernan, 2011) introduced the use of auxiliary
information to the Bayesian inference process, such as item difficulty, and
showed that it can further enhance the prediction accuracy.

With the rise of deep learning models (LeCun et al., 2015) and their achieved
breakthroughs in sequence modelling (Schmidhuber, 2015), such as natural
language processing (Graves et al., 2013a; Yu et al., 2017), video recognition
(Donahue et al., 2015; Liu et al., 2016), and signal processing (Wang et al.,
2018), recent studies adopted deep learning models to address the KT problem.
Piech et al. (Piech et al., 2015) proposed the deep knowledge tracing (DKT)
model which uses a recurrent neural networks (RNN) (Mandic and Chambers, 2001)
to model dynamics in a past exercise sequence and predicts answers for new
questions. DKT resolved the limitations of Bayesian inference approaches as
RNNs optimization through backpropagation is tractable. Despite this advance,
DKT assumed only one hidden state variable for representing a student’s
knowledge state, which is an unrealistic assumption for real-world scenarios
as a student’s knowledge can significantly vary across different learning
concepts. To address this limitation, Zhang et al. (Zhang et al., 2017)
proposed a model called Dynamic Key-Value Memory Networks (DKVMN), which
followed the concepts of Memory-Augmented Neural Networks (MANN) (Santoro et
al., 2016; Graves et al., 2016). MANN aim at mimicking the human’s brain
functionality which combines neural spiking for computation with memory for
storing past experiences (Gallistel and King, 2011). Inspired by MANN, DKVMN
is augmented with two auxiliary memory structures: the key matrix and the
value matrix. The former is used to keep the concepts underlying exercises,
while the later one is used to store a knowledge state across these concepts.
Results showed that DKVMN outperformed BKT and DKT on standard KT benchmarks,
and therefore it is considered the state-of-the-art KT models. However, DKVMN
only considers the latest exercise embedding when updating the value matrix,
resulting in biased knowledge states that ignore past learning experience. As
an example, if we have three related exercises in a sequence, two being
answered correctly and the latest being answered incorrectly, DKVMN would be
biased to the latest one and update the knowledge state with knowledge loss
abruptly. In addition to this, DKVMN has no model capacity to capture long
dependencies in an exercise sequence. This assumes a first-order Markov chain
to represent a past exercise sequence, which is not satisfactory in many
scenarios. Our proposed KT model has addressed the limitations from both DKT
and DKVMN.

In our proposed KT model, we developed a modified LSTM, called Hop-LSTM, for
sequence modelling. Current recurrent neural network models (RNNs) and their
variants, such as LSTMs (Hochreiter and Schmidhuber, 1997), bi-directional
RNNs (Schuster and Paliwal, 1997), or other gated RNNs (Kusupati et al.,
2018), provide the capacity to effectively ingest dependencies in sequential
data. However, when sequences are long, it is still difficult to capture long
term dependencies. One way to alleviate this issue is to only update a
fraction of hidden states based on the current hidden state and input (Jernite
et al., 2017). For example, Yu, Lee and Le (Yu et al., 2017) proposed a LSTM
model that can jump ahead in a sequence to avoid irrelevant words. The jump
decision was controlled by a policy gradient reinforcement learning algorithm
that works as an active learning technique to sample only important words for
the model. Campos et al. (Campos et al., 2018) proposed a model by augmenting
the standard RNN with a binary state update gate function which is responsible
for deciding whether to update the hidden state or not based on the number of
previous updates performed and a loss term that balances the number of updates
(i.e. learning speed) with achieved accuracy. Different from these models, we
developed Hop-LSTM in relation to a Triangular layer so that only the hidden
states of LSTM cells for relevant exercises are connected. This allows our KT
model to identify relevant skills and prior background from the past learning
activities (e.g., exercise sequences) for improved prediction accuracy.

##  7\. Conclusions

In this paper, we introduced a novel model called Sequential Key-Value Memory
Networks (SKVMN) for knowledge tracing. SKVMN aimed at overcoming the
limitations of the existing KT models. It is augmented with a key-value memory
at the memory layer and a modified LSTM, called Hop-LSTM, at the sequence
layer. The experimental results showed that our proposed model outperformed
the state-of-the-art models over all datasets. Future work will consider
techniques to automatically tune hyper-parameters for Knowledge Tracing
models.

###### Acknowledgements.

This research is supported by an Australian government higher education
scholarship, ANU Vice-Chancellor’s Teaching Enhancement Grant, as well as
NVIDIA for the generous GPU support.

## References

  * (1)
  * Baker et al. (2008) Ryan S. Baker, Albert T. Corbett, and Vincent Aleven. 2008\.  More Accurate Student Modeling Through Contextual Estimation of Slip and Guess Probabilities in Bayesian Knowledge Tracing. In Proceedings of the 9th International Conference on Intelligent Tutoring Systems (ITS). Berlin, Heidelberg, 406–415.
  * Bengio (2012) Yoshua Bengio. 2012\.  Practical recommendations for gradient-based training of deep architectures.  In Neural networks: Tricks of the trade: Second Edition. Berlin, Heidelberg, 437–478.
  * Bottou (2012) Léon Bottou. 2012\.  Stochastic gradient descent tricks.  In Neural networks: Tricks of the trade: Second Edition. Berlin, Heidelberg, 421–436.
  * Campos et al. (2018) Víctor Campos, Brendan Jou, Xavier Giró i Nieto, Jordi Torres, and Shih-Fu Chang. 2018\.  Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks. In 6th International Conference on Learning Representations, (ICLR), Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings.
  * Chang et al. (2015) Haw-Shiuan Chang, Hwai-Jung Hsu, and Kuan-Ta Chen. 2015\.  Modeling Exercise Relationships in E-Learning: A Unified Approach.. In Proceedings of the 8th International Conference on Educational Data Mining, (EDM), Madrid, Spain, June 26-29, 2015. 532–535.
  * Corbett and Anderson (1994) Albert T. Corbett and John R. Anderson. 1994.  Knowledge tracing: Modeling the acquisition of procedural knowledge.  User Modeling and User-Adapted Interaction 4, 4 (01 Dec 1994), 253–278.
  * Donahue et al. (2015) Jeffrey Donahue, Lisa Anne Hendricks, Sergio Guadarrama, Marcus Rohrbach, Subhashini Venugopalan, Kate Saenko, and Trevor Darrell. 2015\.  Long-Term Recurrent Convolutional Networks for Visual Recognition and Description. In The IEEE Conference on Computer Vision and Pattern Recognition, (CVPR) , Boston, MA, USA, June 7-12, 2015. 2625–2634.
  * Gallistel and King (2011) Charles R Gallistel and Adam Philip King. 2011.  Memory and the computational brain: Why cognitive science will transform neuroscience. Vol. 6.  John Wiley & Sons.
  * Glorot and Bengio (2010) Xavier Glorot and Yoshua Bengio. 2010.  Understanding the difficulty of training deep feedforward neural networks. In In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS). Society for Artificial Intelligence and Statistics. Chia Laguna Resort, Sardinia, Italy, 249–256.
  * Graves et al. (2013a) A. Graves, N. Jaitly, and A. Mohamed. 2013a.  Hybrid speech recognition with Deep Bidirectional LSTM. In 2013 IEEE Workshop on Automatic Speech Recognition and Understanding, Olomouc, Czech Republic, December 8-12, 2013. 273–278.
  * Graves et al. (2013b) A. Graves, A. Mohamed, and G. Hinton. 2013b.  Speech recognition with deep recurrent neural networks. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. Vancouver, BC, Canada, 6645–6649.
  * Graves et al. (2016) Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwinska, Sergio Gómez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, Adrià Puigdomènech Badia, Karl Moritz Hermann, Yori Zwols, Georg Ostrovski, Adam Cain, Helen King, Christopher Summerfield, Phil Blunsom, Koray Kavukcuoglu, and Demis Hassabis. 2016.  Hybrid computing using a neural network with dynamic external memory.  Nature 538 (12 Oct 2016), 471.
  * Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber. 1997.  Long Short-Term Memory.  Neural Comput. 9, 8 (Nov. 1997), 1735–1780.
  * Jernite et al. (2017) Yacine Jernite, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017.  Variable computation in recurrent neural networks. In 5th International Conference on Learning Representations, (ICLR), Toulon, France, April 24-26, 2017, Conference Track Proceedings.
  * Khajah et al. (2016) Mohammad Khajah, Robert V Lindsey, and Michael C Mozer. 2016\.  How Deep is Knowledge Tracing?. In Proceedings of the 9th International Conference on Educational Data Mining, (EDM), Raleigh, North Carolina, USA, June 29 - July 2, 2016.
  * Kingma and Ba (2015) Diederik P. Kingma and Jimmy Lei Ba. 2015.  Adam: A Method for Stochastic Optimization. In international conference on learning representations (ICLR).
  * Klir and Yuan (1995) George J. Klir and Bo Yuan. 1995.  Fuzzy Sets and Fuzzy Logic: Theory and Applications.  Prentice-Hall, Inc., Upper Saddle River, NJ, USA.
  * Kusupati et al. (2018) Aditya Kusupati, Manish Singh, Kush Bhatia, Ashish Kumar, Prateek Jain, and Manik Varma. 2018\.  FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems, (NeurIPS), 3-8 December 2018, Montréal, Canada. 9017–9028.
  * LeCun et al. (2015) Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015\.  Deep learning.  nature 521, 7553 (2015), 436.
  * Ling et al. (2003) Charles X. Ling, Jin Huang, and Harry Zhang. 2003\.  AUC: A Statistically Consistent and More Discriminating Measure Than Accuracy. In Proceedings of the 18th International Joint Conference on Artificial Intelligence (IJCAI). San Francisco, CA, USA, 519–524.
  * Liu et al. (2016) Jun Liu, Amir Shahroudy, Dong Xu, and Gang Wang. 2016\.  Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition. In 14th European Conference on Computer Vision, (ECCV) , Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III. Cham, 816–833.
  * Mandic and Chambers (2001) Danilo P. Mandic and Jonathon Chambers. 2001.  Recurrent Neural Networks for Prediction: Learning Algorithms,Architectures and Stability.  John Wiley & Sons, Inc., New York, NY, USA.
  * Pardos and Heffernan (2010) Zachary A. Pardos and Neil T. Heffernan. 2010.  Modeling Individualization in a Bayesian Networks Implementation of Knowledge Tracing. In Proceedings of the 18th International Conference on User Modeling, Adaptation, and Personalization (UMAP). Berlin, Heidelberg, 255–266.
  * Pardos and Heffernan (2011) Zachary A. Pardos and Neil T. Heffernan. 2011.  KT-IDEM: Introducing Item Difficulty to the Knowledge Tracing Model. In Proceedings of the 19th International Conference on User Modeling, Adaption, and Personalization (UMAP). Berlin, Heidelberg, 243–254.
  * Pascanu et al. (2013) Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2013\.  On the Difficulty of Training Recurrent Neural Networks. In Proceedings of the 30th International Conference on International Conference on Machine Learning (ICML). Atlanta, Georgia, USA, III–1310–III–1318.
  * Piech et al. (2015) Chris Piech, Jonathan Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas Guibas, and Jascha Sohl-Dickstein. 2015.  Deep Knowledge Tracing. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada (NeurIPS). Cambridge, MA, USA, 505–513.
  * Piech (2016) Christopher James Piech. 2016\.  Uncovering Patterns in Student Work: Machine Learning to Understand Human Learning.  Ph.D. Dissertation. Stanford University.
  * Santoro et al. (2016) Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. 2016\.  Meta-learning with Memory-augmented Neural Networks. In Proceedings of the 33nd International Conference on Machine Learning, (ICML), New York City, NY, USA, June 19-24, 2016. 1842–1850.
  * Schmidhuber (2015) Jürgen Schmidhuber. 2015\.  Deep learning in neural networks: An overview.  Neural Netw. 61, C (Jan. 2015), 85–117.
  * Schuster and Paliwal (1997) M. Schuster and K. K. Paliwal. 1997.  Bidirectional recurrent neural networks.  IEEE Transactions on Signal Processing 45, 11 (Nov 1997), 2673–2681.
  * Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014\.  Sequence to Sequence Learning with Neural Networks. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems, December 8-13 2014, Montreal, Quebec, Canada (NeurIPS). Cambridge, MA, USA, 3104–3112.
  * Villano (1992) Michael Villano. 1992\.  Probabilistic Student Models: Bayesian Belief Networks and Knowledge Space Theory. In Proceedings of the Second International Conference on Intelligent Tutoring Systems (ITS). London, UK, UK, 491–498.
  * Wang et al. (2018) P. Wang, A. Jiang, X. Liu, J. Shang, and L. Zhang. 2018.  LSTM-Based EEG Classification in Motor Imagery Tasks.  IEEE Transactions on Neural Systems and Rehabilitation Engineering 26, 11 (Nov 2018), 2086–2095.
  * Yu et al. (2017) Adams Wei Yu, Hongrae Lee, and Quoc V. Le. 2017\.  Learning to Skim Text. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, (ACL), Vancouver, Canada, July 30 \- August 4, (Volume 1: Long Papers). 1880–1890.
  * Yudelson et al. (2013) Michael V Yudelson, Kenneth R Koedinger, and Geoffrey J Gordon. 2013.  Individualized bayesian knowledge tracing models. In Artificial Intelligence in Education - 16th International Conference, (AIED), Memphis, TN, USA, July 9-13, 2013. Proceedings. 171–180.
  * Zhang et al. (2017) Jiani Zhang, Xingjian Shi, Irwin King, and Dit-Yan Yeung. 2017\.  Dynamic Key-Value Memory Networks for Knowledge Tracing. In Proceedings of the 26th International Conference on World Wide Web (WWW). Republic and Canton of Geneva, Switzerland, 765–774.