#  Gamma production in neutrino interaction with nuclei

G. Chanfray  Univ Lyon, Univ Claude Bernard Lyon 1, CNRS/IN2P3, IP2I Lyon, UMR
5822, F-69622, Villeurbanne, France  M. Ericson  Univ Lyon, Univ Claude
Bernard Lyon 1, CNRS/IN2P3, IP2I Lyon, UMR 5822, F-69622, Villeurbanne, France
Theory division, CERN, CH-12111 Geneva

###### Abstract

We evaluate the cross-section for gamma production by neutrinos through a
meson exchange effect which derives from the concept of axial-vector mixing.
The resulting cross-section leads to some increase of the gamma production
cross-section by neutrinos, especially at low neutrino energies, which may
influence the understanding of the low energy excess of electron-like events
seen in the MiniBooNE experiment.

##  I Introduction

The problem of gamma emission in the interaction of neutrinos with nuclei is
of a great interest for the interpretation of the low energy excess of
electron-like events seen in the MiniBooNE experiment MiniBooNe2009 ;
MiniBooNe2013 . In this short baseline (541 m) muon neutrino primary beam
experiment a number of low energy electrons have been detected in the target.
The shortness of the baseline excluding oscillations into the known neutrinos,
this observation has been interpreted as an evidence for a new neutrino, the
sterile neutrino, more massive than the known neutrinos and whose existence
has been vividly discussed. This anomalous excess has been confirmed in a
recent analysis MiniBooNe2018 . The importance of this result has triggered a
number of investigations to explore alternative interpretations for the
presence of these electrons. A possibility is that gamma rings produced in the
target have been mistaken for electron rings, as the MiniBooNE detector cannot
distinguish the two, thus artificially increasing the apparent number of
electrons. It is therefore crucial to have a proper evaluation of the gamma
emission background in the interaction of neutrinos.
Several evaluations of this process have been made Hill2010 ; Hill2011 ;
Serot2012 ; Serot2013 ; Wang2014 ; Wang2015 . They involve in particular the
production of a Delta, which decays by gamma emission. In these events where
the gamma rings could be mistaken for electrons, no lepton are emitted and the
process for gamma production involves a neutral current transition: the vector
boson $Z_{0}$ in its interaction with a nucleon excites a Delta which emits a
photon, as shown in Fig 1a. However, even if all these photons are mistaken
for electrons this process has not been able to account for the observed
number of electrons without a sterile neutrino oscillation Katori2020 .

In this work we introduce another source for gamma production by neutrinos
which, to our knowledge, has not been considered in connection with this
problem. It involves a meson exchange effect from a contact vertex where,
together with the gamma, a pion is produced at the $N\Delta$ vertex as
depicted on Fig. 1b. The contact $\langle N|\pi\gamma|\Delta\rangle$ coupling
is obtained from the usual p wave coupling $\langle
N|\partial_{\nu}\pi|\Delta\rangle$ by the minimal substitution where in short
$\partial_{\nu}$ is replaced by $(\partial_{\nu}-q\,A_{\nu})$. Its precise
form will be given below. In the interaction of real photons with a free
nucleon this contact coupling is responsible for an appreciable part of the
photon-nucleon cross-section in the energy region above the Delta energy,
$\omega\simeq 400-500\ MeV$. Notice that the process of $\gamma\pi$
simultaneous production as such would not affect the MiniBooNE interpretation
as the pion produced in this process is detectable, while instead in the
selected events no pions are observed. However in the nucleus the pion is
dressed, in particular by particle-hole ($ph$) excitations, thus acquiring a
broad spectrum which extends on the low energy side of the pion mass. In the
MiniBooNE detector which does not identify nuclear excitations, when a pion
produced is disguised as a $ph$ state it becomes invisible, simulating a
simple gamma production process and contributing to an increase of the gamma
emission cross section. The fact that the energy distribution of the process
where the pion is materialized as a $ph$ excitation is smaller than the one
where it is emitted as a real pion is interesting, as the apparent electron
excess in MiniBooNe occurs below a reconstructed neutrino energy of
approximately $400\,MeV$.
The introduction of the graph of Fig. 1b is not randomly chosen: it embodies
the concept of axial-vector mixing in the nuclear medium introduced by
Chanfray et al CDER99 . This concept consists in the following: the s wave
absorption (or emission) by a nucleon (or a Delta) of a pion emitted by a
neighboring nucleon, as illustrated in Fig 1b, produces a parity change which
transforms a vector correlator into an axial one (or the reverse). It extends
to the nuclear case the concept, introduced by Dey et al DEI90 , of parity
mixing by the thermal pions of a heat bath. In the nuclear medium the virtual
pions of the pionic clouds replace the thermal pions of the heat bath. Drawn
in the perspective of correlator mixing the vector-vector correlator (graph 2c
of Fig. 2 of ref CDER99 ) is associated with the mixing ones represented by
graphs 2d-e-f of the same figure. As a side remark we also point out that the
pion cloud contribution to the broadening of the rho meson observed in
dilepton production in relativistic heavy ion collision CS93 ; CRW96
originates from such an axial-vector mixing effect, as demonstrated first in
this reference CDER99 .
In the simpler case of the interaction of real photons with nuclei the effect
expected from the correlator mixing is a spreading of the gamma nucleus cross
section due to the $\pi N\Delta$ contact term over a larger range of energy,
increasing the nuclear cross section as compared to the nucleonic one in the
low energy transfer region, $\omega\leq 500\,MeV$ and producing some depletion
above, in the energy region where the original $\pi N\Delta$ contact term
acts. These features are remarkably similar to those of the MiniBooNE excess
event but this analogy is premature, as the kinematics of neutrino
interactions is different. The contact $\pi N\Delta$ graph that we have
discussed is not the only mixing term. There is also the one where the gamma
is absorbed by a pion in flight as shown in Fig. 1c. We have also evaluated
this term and it contributes significantly.

Summarizing, the aim of the present work is the evaluation of the cross
section for the process of gamma emission on the ${}^{12}C$ nucleus by
neutrinos through the contact $N\Delta\gamma\pi$ term, leading to a final
$2p-2h$ excited state. We use for the ${}^{12}C$ nucleus a nuclear matter
description with a typical density, $\rho=0.8\,\rho_{0}$, where $\rho_{0}$ is
the normal nuclear matter density. The process which for a free nucleon is :
$Z_{0}+N\,\rightarrow\,\Delta\,\rightarrow\,N\pi\gamma$ leads in the nucleus
to :
$Z_{0}+A\,\rightarrow\,\Delta\,(A-1)\,\rightarrow\,\gamma\pi\,1p1h\,(A-1)\,\rightarrow\,\gamma\,2p2h\,(A-2)$.
As mentioned above, the final state which consists of a gamma plus an
(invisible) $2p-2h$ excitation can be mistaken for a gamma emission and hence
for an electron emission process. This contribution is appreciably different
in different neutrino energy regions, an interesting feature for the MiniBooNE
experiment where the excess is concentrated in a region of neutrino energy
below $\simeq 400MeV.$

Figure 1: Left panel (a): single $\gamma$ emission process considered in
Hill2010 ; Hill2011 ; Serot2012 ; Serot2013 ; Wang2014 ; Wang2015 . Middle
panel (b): contact $\gamma$-pion emission process. Right panel (c): Pion in
flight process.

In the following we denote by $k=(E_{\nu},{\bf k})$ and
$k^{\prime}=(E^{\prime}_{\nu},{\bf k}^{\prime})$ the quadrimomenta of the
incoming and outgoing neutrinos, $\omega=E_{\nu}-E^{\prime}_{\nu}$ and ${\bf
q}={\bf k}-{\bf k}^{\prime}$, the energy and momentum transferred to the
nucleus and $\theta$ the scattering angle. The $Z_{0}N\Delta$ coupling can be
extracted from the isovector piece of the spatial part of the $\bar{q}qZ_{0}$
vertex ; with standard notation it reads:

${\cal
L}_{\bar{q}qZ_{0}}=\frac{g_{2}}{2\cos\theta_{W}}\,\left(V_{\mu}(x)-A_{\mu}(x)\right)\,Z_{0}^{\mu}(x)\simeq\frac{g_{2}}{2\cos\theta_{W}}\bar{q}(x)\,\gamma_{j}\,\left(1-\gamma_{5}\right)\,t_{3}^{q}\,q(x)\,Z_{0}^{j}(x).$
(1)

At low neutrino energy the dominant part is the hadronic axial current, i.e.,
the $\gamma_{5}$ piece. The vector current piece gives a relativistic
correction which generates an axial-vector interference in the cross-section
resulting in a higher cross-section for neutrinos than for antineutrinos. For
the single gamma production discussed below this interference gives typically
a $30\%$ effect compared to the averaged cross- section where only the axial
piece is taken into account. This point is discussed in some details in
section IV of Ref. Hill2010  and is visible on Fig. 4 of Ref. Hill2010 , on
Fig. 3 of Ref. Serot2013  or on Fig. 5 and 8 of Ref. Wang2014 . In this
exploratory work devoted to the comparison of single gamma process with the
pion exchange process we limit ourselves to the axial coupling such that,
strictly speaking, our result applies to the average sum of the neutrino and
antineutrino cross-sections. Hence in the following we take:

${\cal
L}_{\bar{q}qZ_{0}}\simeq-\frac{g_{2}}{2\cos\theta_{W}}\bar{q}(x)\,\gamma_{j}\,\gamma_{5}\,t_{3}^{q}\,q(x)\,Z_{0}^{j}(x).$
(2)

The cross-section for gamma production induced by a neutral current which
excites a Delta from a nucleon with momentum $p$ producing a photon in the
final state writes:

$d\sigma=2G^{2}_{F}\frac{2\pi\,dcos\theta\,k^{\prime}\,E^{\prime}_{\nu}\,dE^{\prime}_{\nu}}{(2\pi)^{3}\,2E_{\nu}\,2E^{\prime}_{\nu}}\,L_{\mu\eta}\,H^{\mu}H^{*\eta}\,2\pi\,\delta\left(E_{f\gamma}+E^{\prime}_{\nu}-E_{p}-E_{\nu}\right),$
(3)

with

$L_{\mu\eta}=Tr\left\\{\not k\gamma_{\mu}(1-\gamma_{5})\not
k^{\prime}\gamma_{\eta}(1-\gamma_{5})\right\\}=8\,\left(k_{\mu}k^{\prime}_{\eta}+k_{\eta}k^{\prime}_{\mu}-g_{\mu\eta}k\cdot
k^{\prime}\pm\,i\varepsilon_{\mu\eta\alpha\beta}k^{\alpha}k^{\prime\beta}\right)$
(4) $H^{\mu}=\langle f\gamma|\int d{\bf x}\,\,{\bf j}_{em}({\bf x})\cdot{\bf
A}({\bf x})\,|\Delta\rangle\,\,G_{\Delta}(\omega,{\bf p}+{\bf
q})\,\,\langle\Delta|\int d{\bf y}\,\bar{q}({\bf
y})\,\gamma^{\mu}\,\gamma_{5}\,t_{3}^{q}\,q({\bf y})\,e^{i{\bf q}\cdot{\bf
y}}|p\rangle.$ (5)

The first form of the leptonic tensor applies to the neutrino case whereas the
second form, with the explicit incorporation of the antisymmetric piece and
where the plus (minus) sign refers to the neutrino (antineutrino), covers both
$\nu$ and $\bar{\nu}$ cases. This antisymmetric piece, when contracted with
the hadronic tensor, gives the axial-vector interference contribution to the
cross-section that we have ignored. For what concerns the hadronic piece
$H^{\mu}$, a summation over spin and isospin states of the intermediate delta
is understood. For the $\Delta$ propagator, neglecting Fermi motion, we take
the simplified form:

$\displaystyle G_{\Delta}(\omega,{\bf q})$ $\displaystyle\simeq$
$\displaystyle\left[\omega-\sqrt{M_{\Delta}^{2}+q^{2}}\,+M_{N}\,+i\,\frac{\Gamma(\omega)}{2}\right]^{-1}$
$\displaystyle\Gamma(\omega)$ $\displaystyle=$
$\displaystyle\frac{1}{6\pi}\left(\frac{g_{A}}{2f_{\pi}}R_{N\Delta}\right)^{2}\left(\omega^{2}-m^{2}_{\pi}\right)^{3/2}\,\,\hbox{with}\,\,\;R_{N\Delta}=\frac{g_{\pi
N\Delta}}{g_{\pi NN}}.$ (6)

In the numerical evaluations we will take for the ratio of the Delta and
nucleon coupling constants : $R_{N\Delta}=2$. As mentioned above, in the non
relativistic limit the spatial part of the axial current dominates. We take it
in the standard form:

$\langle\Delta|\int d{\bf y}\,\bar{q}({\bf
y})\,\gamma^{j}\,\gamma_{5}\,t_{3}^{q}\,q({\bf y})\,e^{-i{\bf q}\cdot{\bf
y}}|p\rangle=\frac{G_{A}(Q^{2}=kk^{\prime}\,\left(\cos\theta-1\right))}{2}R_{N\Delta}\,\langle\Delta|\,S_{j}\,T_{3}\,|p\rangle,$
(7)

where $S_{j}$, ($T_{k}$) are the spin (isospin) transition operators between
the spin-isospin 3/2 $\Delta$ state and the spin-isospin 1/2 nucleon state
with reduced matrix element
$\left\langle\frac{3}{2}\left\|S,T\right\|\frac{1}{2}\right\rangle=2$. In the
right-hand side of Eq. (7) the Delta and proton states refer to the spin-
isospin quantum numbers only. The axial form factor is taken in the standard
dipole form with $G_{A}(0)=g_{A}=1.26$ and a cutoff parameter
$M_{A}=1.032\,GeV$.

##  II Single photon emission cross-sections off nuclei

In the case of a single gamma emission (i.e., $\Delta\rightarrow\gamma N$)
with momentum ${\bf p}_{\gamma}$ and polarisation $\vec{\epsilon}_{\lambda}$,
the $\gamma N\Delta$ vertex, i.e., the electromagnetic matrix element, is
obtained Oset82  by assuming the existence of a scaling law between the
nucleon and Delta axial and magnetic matrix elements:

$\displaystyle\langle N^{\prime}(p^{\prime});\gamma({\bf
p}_{\gamma},\vec{\epsilon}_{\lambda})|\int d{\bf x}\,{\bf j}_{em}({\bf
x})\cdot{\bf A}({\bf x})\,|\Delta\rangle$ $\displaystyle=$
$\displaystyle\frac{e}{\sqrt{2p_{\gamma}V}}\,\langle p^{\prime}|\,\int d{\bf
x}\,\,e^{-i{\bf p}_{\gamma}\cdot{\bf x}}\,\bar{q}({\bf
x})\,\vec{\gamma}\cdot\vec{\epsilon}\left(\frac{1}{6}+t_{3}^{q}\right)\,q({\bf
x})\,|\Delta\rangle$ (8) $\displaystyle=$
$\displaystyle-\frac{ie\,R_{N\Delta}}{\sqrt{2p_{\gamma}V}}\,\frac{\mu_{p}-\mu_{n}}{4M_{N}}\,\langle
p^{\prime}|\left({\bf S}^{\dagger}\times{\bf
p}_{\gamma}\right)\cdot\vec{\epsilon}\,\,T_{3}^{\dagger}\,|\Delta\rangle\,\delta_{{\bf
p}_{\gamma}+{\bf p}^{\prime}-{\bf q}-{\bf p}}.$ (9)

In the evaluation of the cross-section, one has to perform a summation over
the spin-isospin states of the intermediate Delta and of the final emitted
nucleon (see appendix A). Ignoring Fermi motion, the resulting cross section
per nucleon for gamma production writes:

$\frac{d\sigma^{\gamma}}{dE^{\prime}_{\nu}d\cos\theta}=\left[\frac{G^{2}_{F}}{4\pi}\frac{k^{\prime}}{k}\,8\,kk^{\prime}\,\left(3\,-\,\cos\theta\right)\right]\,\left[\frac{4}{9}\,R_{N\Delta}^{2}\,\left(\frac{G_{A}(Q^{2})}{2}\right)^{2}\right]\left[\frac{1}{2\pi}\,\Gamma^{\gamma}(\omega)\right].$
(10)

Neglecting nucleon recoil the radiative Delta width $\Gamma^{\gamma}(\omega)$
is:

$\displaystyle\Gamma^{\gamma}(\omega)$ $\displaystyle=$
$\displaystyle\frac{4}{9}\,R_{N\Delta}^{2}\,\left(\frac{e}{2M_{N}}\right)^{2}\left(\frac{\mu_{p}-\mu_{n}}{2}\right)^{2}\int\frac{d{\bf
p}_{\gamma}}{(2\pi)^{3}\,2p_{\gamma}}p^{2}_{\gamma}\,2\pi\,\delta(p_{\gamma}+\epsilon_{p^{\prime}}-\omega-\epsilon_{p})$
(11) $\displaystyle\simeq$
$\displaystyle\frac{4}{9}\,R_{N\Delta}^{2}\,\left(\frac{e}{2M_{N}}\right)^{2}\left(\frac{\mu_{p}-\mu_{n}}{2}\right)^{2}\,\frac{\omega^{3}}{2\pi}.$

The result of our evaluation for the single gamma emission off ${}^{12}C$ as a
function of the neutrino energy is displayed on Fig. 2. We find a qualitative
agreement with Ref. Hill2010 ; Hill2011 ; Serot2012 ; Serot2013 ; Wang2014 ;
Wang2015 , certainly not perfect but sufficient for the purpose of this
article. For instance, for a neutrino energy of $E_{\nu}=\,0.3\,GeV$, we find
for the cross-section per nucleon
$\sigma^{\gamma}/A\simeq\,0.27\,10^{-42}cm^{2}$ which is twice larger than the
Delta contribution shown on Fig. 4 of Ref. Hill2010 , but the cut of
$200\,MeV$ applied to the photon energy in this reference could explain this
difference. For a neutrino energy of $E_{\nu}=\,0.5\,GeV$, we find
$\sigma^{\gamma}/A\simeq\,2.\,10^{-42}cm^{2}$ which is close, although
slightly larger, to the averaged $\nu\bar{\nu}$ cross-section shown on the
same figure (Fig. 4 of Ref. Hill2010 ) and on Fig. 8 of Ref. Wang2014 . Here
it is apparent that this Delta contribution dominates over other processes
such as ”Compton-like” scattering and omega exchange. At a higher neutrino
energy, $E_{\nu}=\,1\,GeV$, we find $\sigma^{\gamma}/A\simeq
8\,10^{-42}cm^{2}$, again slightly larger than the averaged $\nu\bar{\nu}$
cross-section obtained in Refs. Hill2010 ; Serot2013 ; Wang2014  where it is
apparent that the Delta contribution is largely dominant.

Figure 2: Full line: single $\gamma$ cross section for ${}^{12}C$ (graph of
Fig. 1a) in $10^{-42}cm^{2}$ versus neutrino energy in GeV. Dashed line:
$\gamma$-pion/ph cross section for ${}^{12}C$ with only the contact term
(graph of Fig. 1b). Dotted line: $\gamma$-pion/ph cross section with inclusion
of the pion in flight process (graphs of Fig. 1b + 1c).  Figure 3:  Ratio of
the $\gamma$-pion/ph and single $\gamma$ cross sections versus neutrino energy
in GeV without (full line) and with (dashed line) the pion in flight process
of Fig. 1c.  Figure 4:  The same as Fig. 3 but with the inclusion of the 2p-2h
contribution in the pion self-energy.

##  III Photon-pion emission cross-sections off nuclei

As mentioned previously the specific realization of chiral symmetry in the
hadronic world implies that for any process involving a vector correlator
there is an associated axial correlator process through the emission or the
absorption of an s-wave pion; this is referred as the axial-vector correlator
mixing effect. This mixing implies that there is a $N\Delta\gamma\pi$ vertex
associated with the direct $N\Delta\gamma$ vertex. There are several ways to
derive this vertex. One is to start from an effective chiral theory formulated
at the quark level CE2011  or more directly at the nucleonic level (see Ref.
Holt2016  for a recent review). In both cases the Goldstone pion field
$\vec{\phi}(x)$ is introduced through a matrix
$U(x)=e^{i\,\vec{\tau}\cdot\vec{\phi}(x)/{f_{\pi}}}$ having a perfectly well
defined transformation law under chiral rotations. Nucleons can be also
introduced as heavy sources coupled to pions BKM95 . The leading term is
dictated by chiral symmetry alone :

$\displaystyle{\cal L}^{(1)}_{N}$ $\displaystyle=$
$\displaystyle\bar{\psi}_{N}\left[i\gamma_{\mu}\left(\partial^{\mu}+iv^{\mu}\right)+g_{A}\gamma_{\mu}\gamma^{5}a^{\mu}-M_{N}\right]\psi_{N}$
$\displaystyle\simeq$
$\displaystyle\bar{\psi}_{N}\left(i\gamma_{\mu}\partial^{\mu}-M_{N}\right)\,-\,\frac{g_{A}}{2f_{\pi}}\bar{\psi}_{N}\gamma_{\mu}\gamma_{5}\vec{\tau}\psi_{N}\cdot\partial^{\mu}\vec{\phi}-\,\frac{1}{4f^{2}_{\pi}}\bar{\psi}_{N}\gamma_{\mu}\vec{\tau}\psi_{N}\cdot\vec{\phi}\times\partial^{\mu}\vec{\phi}$
with $\displaystyle\xi=\sqrt{U}\qquad
v^{\mu}=\frac{i}{2}\left(\xi\partial^{\mu}\xi^{\dagger}+\xi^{\dagger}\partial^{\mu}\xi\right)\qquad
a^{\mu}=\frac{i}{2}\left(\xi\partial^{\mu}\xi^{\dagger}-\xi^{\dagger}\partial^{\mu}\xi\right),$
(12)

where the Dirac spinor $\psi_{N}$ denotes the iso-doublet of nucleons. There
are two parameters which are not determined by chiral symmetry : the nucleon
mass (in principle in the chiral limit $M_{0}$) and the axial coupling
constant, $g_{A}=1.26$, known from the analysis of neutron beta decay. The
coupling to the electromagnetic field is simply obtained by gauging the above
lagrangian by the appropriate minimal substitution:

$\partial^{\mu}\xi\rightarrow\partial^{\mu}\xi+ie\frac{1+\tau_{3}}{2}A^{\mu}\xi\qquad\partial^{\mu}\xi^{\dagger}\rightarrow\partial^{\mu}\xi^{\dagger}+ie\frac{1+\tau_{3}}{2}A^{\mu}\xi^{\dagger}.$
(13)

This generates the following Lagrangian:

$\displaystyle{\cal L}_{NN\gamma\pi}$ $\displaystyle=$
$\displaystyle-\frac{e}{2}\,g_{A}\,A_{\mu}\bar{\psi}_{N}\,\gamma_{\mu}\gamma^{5}\left(\xi\frac{\tau_{3}}{2}\xi^{\dagger}\,-\,\xi^{\dagger}\frac{\tau_{3}}{2}\xi\right)\psi_{N}\simeq-e\,g_{A}\,A_{\mu}\bar{\psi}_{N}\,\gamma_{\mu}\gamma^{5}\,\frac{\vec{\tau}}{2f_{\pi}}\cdot\left(\vec{e}_{3}\times\vec{\phi}\right)\psi_{N}$
$\displaystyle=$ $\displaystyle-
ie\frac{g_{A}}{2f_{\pi}}A_{\mu}\,\bar{\psi}_{N}\,\gamma_{\mu}\gamma^{5}\,\left(\tau^{\dagger}_{+}\,\phi_{+}\,+\,\tau^{\dagger}_{-}\,\phi_{-}\right)\psi_{N}$
with $\displaystyle\tau_{\pm}=\mp\frac{\tau_{1}\pm
i\tau_{2}}{\sqrt{2}}\qquad\phi_{\pm}=\frac{\phi_{1}\pm\phi_{2}}{\sqrt{2}}\,\,\hbox{creating
a}\,\pi^{\pm}.$ (14)

Hence the matrix element of the electromagnetic interaction between an initial
nucleon and and a final state made of a nucleon, a photon and a charged pion
$\pi^{\alpha}$ reads:

$\displaystyle\langle N^{\prime}(p^{\prime});\gamma({\bf
p}_{\gamma},\vec{\epsilon}_{\lambda});\pi^{\alpha}({\bf p}_{\pi})|\int d{\bf
x}\,{\bf j}_{em}({\bf x})\cdot{\bf A}({\bf x})\,|N\rangle$ $\displaystyle=$
$\displaystyle\frac{e}{\sqrt{2p_{\gamma}V\,2E_{\pi}V}}\,\frac{g_{A}}{2f_{\pi}}\langle
p^{\prime}|\,\int d{\bf x}\,e^{-i({\bf p}_{\gamma}+{\bf p}_{\pi})\cdot{\bf
x}}\bar{\psi}_{N}({\bf
x})\,\gamma_{\mu}\gamma^{5}\,\tau^{\dagger}_{\alpha}{\psi}_{N}({\bf
x})\,|N\rangle$ (15) $\displaystyle\simeq$
$\displaystyle\frac{e}{\sqrt{2p_{\gamma}V\,2E_{\pi}V}}\,\frac{g_{A}}{2f_{\pi}}\,\langle
p^{\prime}|\vec{\sigma}\cdot\vec{\epsilon}_{\lambda}\,\,\tau^{\dagger}_{\alpha}|N\rangle\,\delta_{{\bf
p}_{\gamma}+{\bf p}_{\pi}+{\bf p}^{\prime}-{\bf q}-{\bf p}},$

where in the non relativistic approximation of the last line the nucleon
states refer to the spin-isospin quantum numbers only. The extension to the
$N\Delta\gamma\pi$ vertex is straightforward with the replacement of the Pauli
matrices by the spin and isospin transition operators with the same rescaling
of the coupling constants as previously:

$\langle N^{\prime}(p^{\prime});\gamma({\bf
p}_{\gamma},\vec{\epsilon}_{\lambda});\pi^{\alpha}\left({\bf
p}_{\pi}\right)|\int d{\bf x}\,{\bf j}_{em}({\bf x})\cdot{\bf A}({\bf
x})\,|\Delta\rangle=\frac{e}{\sqrt{2p_{\gamma}V\,2E_{\pi}V}}\,R_{N\Delta}\,\frac{g_{A}}{2f_{\pi}}\,\langle
p^{\prime}|{\bf
S}^{\dagger}\cdot\vec{\epsilon}_{\lambda}\,\,T^{\dagger}_{\alpha}|\Delta\rangle\,\delta_{{\bf
p}_{\gamma}+{\bf p}_{\pi}+{\bf p}^{\prime}-{\bf q}-{\bf p}}.$ (16)

After performing again the spin-isospin summation over intermediate delta and
final emitted nucleon, we obtain for the photon-pion production cross-section:

$\frac{d\sigma^{\gamma\pi}}{dE^{\prime}_{\nu}d\cos\theta}=\left[\frac{G^{2}_{F}}{4\pi}\frac{k^{\prime}}{k}_{,}8\,kk^{\prime}\,\left(3\,-\,\cos\theta\right)\right]\,\left[\frac{4}{9}\,R_{N\Delta}^{2}\,\left(\frac{G_{A}(Q^{2})}{2}\right)^{2}\right]\left[\frac{1}{2\pi}\,\Gamma^{\gamma\pi}(\omega)\right].$
(17)

The radiative Delta-pion width is:

$\displaystyle\Gamma^{\gamma\pi}(\omega)$ $\displaystyle=$ $\displaystyle
e^{2}\,\frac{2}{9}\,R_{N\Delta}^{2}\,\left(\frac{g_{A}}{2f_{\pi}}\right)^{2}\int\frac{d{\bf
p}_{\gamma}}{(2\pi)^{3}\,2p_{\gamma}}\int\frac{d{\bf
p}_{\pi}}{(2\pi)^{3}\,2E_{\pi}}\,2\pi\,\delta(p_{\gamma}+E_{\pi}+\epsilon_{p^{\prime}}-\omega-\epsilon_{p})$
(18) $\displaystyle\simeq$
$\displaystyle\frac{e^{2}}{(2\pi)^{3}}\,\frac{2}{9}\,R_{N\Delta}^{2}\,\left(\frac{g_{A}}{2f_{\pi}}\right)^{2}\int_{0}^{\infty}dp_{\gamma}\,p_{\gamma}\,\int_{m_{\pi}}^{\infty}dE_{\pi}\,p_{\pi}\,\delta(p_{\gamma}+E_{\pi}-\omega)$
$\displaystyle\simeq$
$\displaystyle\frac{e^{2}}{(2\pi)^{3}}\,\frac{2}{9}\,R_{N\Delta}^{2}\,\left(\frac{g_{A}}{2f_{\pi}}\right)^{2}\int_{0}^{\infty}dp_{\gamma}\,p_{\gamma}\,\int_{m_{\pi}}^{\infty}dE_{\pi}\,p_{\pi}\,\left(-\frac{2E_{\pi}}{\pi}\right)\,ImD_{0\pi}\left(\omega-
p_{\gamma},p_{\pi}=\sqrt{E^{2}_{\pi}-m^{2}_{\pi}}\right).$

In the last two expressions nucleon recoils have been neglected. The in-medium
cross-section is obtained by replacing the bare pion propagator by the in-
medium pion propagator according to :

$ImD_{0\pi}(\Omega,p_{\pi})=Im\left[\Omega^{2}-m_{\pi}^{2}-p_{\pi}^{2}\,+\,i\eta\right]^{-1}\,\,\rightarrow\,\,ImD_{\pi}(\Omega,p_{\pi})=Im\left[\Omega^{2}-m_{\pi}^{2}-p_{\pi}^{2}\,-S(\Omega,p_{\pi})\right]^{-1},$
(19)

where $S(\Omega,p_{\pi})$ is the irreducible pion-self-energy. This quantity
receives contributions from particle-hole and Delta-hole states corrected by
short range screening effects described by three Landau-Migdal parameters
ISW06 :
$g^{\prime}_{NN}=0.7,\,g^{\prime}_{N\Delta}=0.5,\,g^{\prime}_{\Delta\Delta}=0.3$,
according to:

$S(\Omega,p_{\pi})=p^{2}_{\pi}\,\tilde{\Pi}_{0}(\Omega,p_{\pi})=p^{2}_{\pi}\,\frac{\Pi_{0N}+\Pi_{0\Delta}+\left(2g^{\prime}_{NN}-g^{\prime}_{N\Delta}-g^{\prime}_{\Delta\Delta}\right)\,\Pi_{0N}\,\Pi_{0\Delta}}{\left(1-g^{\prime}_{NN}\Pi_{0N}\right)\left(1-g^{\prime}_{\Delta\Delta}\Pi_{0\Delta}\right)-\,g^{\prime
2}_{N\Delta}\Pi_{0N}\,\Pi_{0\Delta}},$ (20)

with

$\displaystyle\Pi_{0N}(\Omega,p_{\pi})=4\,\left({g_{A}\over
2f_{\pi}}\right)^{2}\,v^{2}(p_{\pi})\,\int{d{\bf h}\over(2\pi)^{3}}$
$\displaystyle\bigg{(}{\Theta(p_{F}-h)\,\Theta(|{\bf h}+{\bf
p}_{\pi}|-p_{F})\over\Omega\,-\,\epsilon_{{\bf h}+{\bf
p}_{\pi}}\,+\,\,\epsilon_{h}\,+\,i\eta}$ (21)
$\displaystyle-\,{\Theta(p_{F}-h)\,\Theta(|{\bf h}-{\bf
p}_{\pi}|-p_{F})\over\Omega\,+\,\epsilon_{{\bf h}-{\bf
p}_{\pi}}\,-\,\epsilon_{h}\,-\,i\eta}\bigg{)},$
$\displaystyle\Pi_{0\Delta}(\Omega,{\bf p}_{\pi})={16\over
9}\,R^{2}_{N\Delta}\left({g_{A}\over
2f_{\pi}}\right)^{2}\,v^{2}(p_{\pi})\,\int{d{\bf
h}\over(2\pi)^{3}}\,\Theta(p_{F}-h)$
$\displaystyle\bigg{(}{1\over\Omega\,-\,\epsilon_{\Delta,{\bf h}+{\bf
p}_{\pi}}\,+\,\epsilon_{h}\,+i\Gamma_{\Delta}(\Omega,{\bf h}+{\bf p}_{\pi})}$
(22) $\displaystyle-\,{1\over\Omega\,+\,\epsilon_{\Delta,{\bf h}-{\bf
p}_{\pi}}\,-\,\epsilon_{h}}\bigg{)}.$

$v(p_{\pi})$ is a dipole $\pi NN$ form factor with cutoff $\Lambda=0.98\,GeV$,
as in our previous works CE2007 ; MECM2009 . For the imaginary part of the in-
medium pion propagator we select only those contributions coming from the
dressing of the pion propagator by particle-hole states (see Fig. 1b). Hence
the pion produced is disguised as $p-h$ states which are invisible in the
MiniBoone detector. This simulates a simple gamma process and contributes to
an increase of the gamma emission from the Delta decay. The radiative delta-
pion width entering the cross section (Eq. (17)) thus takes the form:

$\Gamma^{\gamma\pi(ph)}(\omega)\simeq\frac{e^{2}}{(2\pi)^{3}}\,\frac{2}{9}\,R_{N\Delta}^{2}\,\left(\frac{g_{A}}{2f_{\pi}}\right)^{2}\int_{0}^{\infty}dp_{\gamma}\,p_{\gamma}\,\int_{m_{\pi}}^{\infty}dE_{\pi}\,p_{\pi}\,\frac{\left(-\frac{2E_{\pi}}{\pi}\right)\,p^{2}_{\pi}\,Im\tilde{\Pi}_{0}\left(\omega-
p_{\gamma},p_{\pi}=\sqrt{E^{2}_{\pi}-m^{2}_{\pi}}\right)}{\left|\left(\omega-
p_{\gamma}\right)^{2}-m^{2}_{\pi}-p^{2}_{\pi}-p^{2}_{\pi}\,\tilde{\Pi}_{0}\left(\omega-
p_{\gamma},p_{\pi}\right)\right|^{2}}.$ (23)

In the relevant energy domain only the ph bubble has an imaginary part:

$Im\Pi_{0N}(\Omega,{\bf p}_{\pi})=-4\,\pi\,\left({g_{A}\over
2f_{\pi}}\right)^{2}\,v^{2}(p_{\pi})\,\int{d{\bf
h}\over(2\pi)^{3}}\,\Theta(p_{F}-h)\,\Theta(|{\bf h}+{\bf
p}_{\pi}|-p_{F})\,\delta\left(\Omega\,-\epsilon_{{\bf h}+{\bf
p}_{\pi}}\,+\,\epsilon_{h}\right).$ (24)

The inclusion of the pion in flight term (Fig. 1c) can be achieved by
modifying the contact vertex spin operator ${\bf
S}^{\dagger}\cdot\vec{\epsilon}_{\lambda}$ appearing in Eq. (16) according to
Chanfray1983 :

${\bf S}^{\dagger}\cdot\vec{\epsilon}_{\lambda}\qquad\rightarrow\qquad{\bf
S}^{\dagger}\cdot\vec{\epsilon}_{\lambda}-\frac{2\,{\bf S}^{\dagger}\cdot{\bf
t}\,\,{\bf t}\cdot\vec{\epsilon}_{\lambda}}{T^{2}+m_{\pi}^{2}},$ (25)

where ${\bf t}$ and $t_{0}$ are the momentum and energy
($T^{2}=t^{2}-t^{2}_{0}$) of the flying pion. After summation over the photon
polarization states and averaging over the photon momentum direction, the net
effect is the presence in the integrant of $\Gamma^{\gamma\pi}(\omega)$ in Eq.
(18) of a correction factor $F_{Flight}$ given by:

$F_{Flight}=1+\frac{4}{3}\frac{t^{2}}{T^{2}+m_{\pi}^{2}}\left(\frac{t^{2}}{T^{2}+m_{\pi}^{2}}-1\right).$
(26)

If we assume the nucleon recoil momentum to average to zero, we can replace
the momentum ${\bf t}$ of the flying pion by the momentum ${\bf q}$
transferred by the neutrino. As the nuclear response is limited to the low
energy domain we can replace the energy $t_{0}$ of the flying pion by the
photon energy $p_{\gamma}$. The result of the calculation for the photon-pion
emission cross-section without and with the pion in flight terms are shown on
Fig. 2. On Fig. 3 we show their ratio with the single gamma cross-section.

We have also looked at the 2p-2h absorption mode of the exchanged pion, the
dominant mode for physical pions, which would also lead to a simulation of
electron rings. This is achieved by adding to the polarization bubbles
$\Pi_{0N,\Delta}(\omega-p_{\gamma},{\bf p}_{\pi})$ (Eq. 21) a purely imaginary
piece. According to previous studies CS93 ; MECM2009 , we approximate it as
linearly growing in the low energy domain, i.e., $\omega-p_{\gamma}<m_{\pi}$:

$\Pi_{0\,2p2h}(\omega-p_{\gamma},{\bf
p}_{\pi})=-i\,4\pi\,ImC_{0}\,\rho^{2}\left(\frac{\omega-
p_{\gamma}}{m_{\pi}}\right),$ (27)

with $ImC_{0}=0.13\,m_{\pi}^{-6}$ so as to reproduce the absorptive part of
the pion-nucleus optical potential. For $\omega-p_{\gamma}>m_{\pi}$ we keep it
constant up to a maximum value of $400\,MeV$ for $\omega-p_{\gamma}>m_{\pi}$
and zero beyond.

##  IV Discussion

We show on Fig. 2 the results of the calculation of the cross section for
single gamma emission and for gamma-virtual pion/ph emission. We also display
the result of our calculation when the pion in flight process (Fig. 1c) is
introduced on top of the contact term of Fig. 1b. Up to a neutrino energy of
about $0.3\,GeV$ the pion in flight contribution is opposite to that of the
contact term. On Fig. 3 and Fig. 4 we display the ratio between the gamma-pion
and single gamma cross-sections without and with the 2p-2h absorption mode of
the virtual pion. The comparison of these two figures show that the effect of
this mode is moderate.

In conclusion the exchange process that we have introduced is smaller than the
single gamma process but nevertheless significant. Here we have considered
only the emission from a $N\Delta\gamma$ vertex. Other similar meson exchange
terms may also be at work through more complex pion emission processes. Their
complete evaluation should be done before a definite conclusion can be drawn
about the real importance of gamma rings simulating electron rings production
by neutrinos, which could affect some conclusions about the sterile neutrinos.
Finally we remind that our evaluation is restricted to the influence of the
pion exchange effects on the average of the neutrino and antineutrino cross-
sections. The investigation on the influence on their difference, which is
relevant for CP violation, will be the object of a future work.

##  Appendix A Details for the calculation of the single gamma cross-section

The calculation of the single gamma cross-section involves the axial vector
matrix element (Eq. 7) and the electromagnetic matrix element (Eq. 9). When
re-injected in the expression of the cross-section (Eq. 3), this requires a
summation over the spin-isospin states of the intermediate Delta and of the
final emitted nucleon according to :

$\displaystyle\sum_{M^{\prime}_{S},M^{\prime}_{T}}\sum_{M_{S},M_{T}}\sum_{\lambda}\sum_{j,k}\,\langle
p:m_{s},m_{t}|S_{j}^{\dagger}\,T_{3}^{\dagger}|M^{\prime}_{S},M^{\prime}_{T}\rangle\int\frac{d{\hat{p}}_{\gamma}}{4\pi}\sum_{m^{\prime}_{s},m^{\prime}_{t},\lambda}\langle
M^{\prime}_{S},M^{\prime}_{T}|\left({\bf S}\times{\bf
p}_{\gamma}\right)\cdot\vec{\epsilon}^{*}_{\lambda}\,T_{3}\,|m^{\prime}_{s},m^{\prime}_{t}\rangle$
$\displaystyle\langle m^{\prime}_{s},m^{\prime}_{t}|\left({\bf
S}^{\dagger}\times{\bf
p}_{\gamma}\right)\cdot\vec{\epsilon}_{\lambda}\,T_{3}^{\dagger}\,|M_{S},M_{T}\rangle\,\,\,\,\langle
M_{S},M_{T}|S_{k}\,T_{3}|p:m_{s},m_{t}\rangle\,L^{jk}$
$\displaystyle=\sum_{M^{\prime}_{S},M^{\prime}_{T}}\sum_{M_{S},M_{T}}\sum_{j,k}\,\langle
p:m_{s},m_{t}|S_{j}^{\dagger}\,T_{3}^{\dagger}|M^{\prime}_{S},M^{\prime}_{T}\rangle\,\,\,\int\frac{d{\hat{p}}_{\gamma}}{4\pi}\,p^{2}_{\gamma}\,\frac{4}{9}\,\delta_{M_{S},M^{\prime}_{S}}\,\delta_{M_{T},M^{\prime}_{T}}\,\,\,\langle
M^{\prime}_{S},M^{\prime}_{T}|S_{k}\,T_{3}|p:m_{s},m_{t}\rangle\,L^{jk}$
$\displaystyle=\int\frac{d{\hat{p}}_{\gamma}}{4\pi}\,\,\frac{4p^{2}_{\gamma}}{9}\,\,\langle
p:m_{s},m_{t}|\left(\frac{2}{3}\delta_{jk}-\frac{i}{3}\varepsilon{jkl}\,\sigma_{l}\right)\frac{2}{3}|p:m_{s},m_{t}\rangle\,L^{jk}=\int\frac{d{\hat{p}}_{\gamma}}{4\pi}\,\,\frac{4p^{2}_{\gamma}}{9}\,\frac{4}{9}\,(L^{11}+L^{22}+L^{33})$
$\displaystyle=\int\frac{d{\hat{p}}_{\gamma}}{4\pi}\,\,\frac{4p^{2}_{\gamma}}{9}\frac{4}{9}\,\left[8\,kk^{\prime}\,\left(3\,-\,\cos\theta\right)\right].$
(28)

Grouping all the terms together, the cross section per nucleon for gamma
production given in Eq. 10 follows.

## References

  * (1) A. Aguilar-Arevalo, et al., MiniBoone Collaboration, Phys. Rev. Lett. 102, 101802 (2009).
  * (2) A. Aguilar-Arevalo, et al., MiniBoone Collaboration, Phys. Rev. Lett. 110, 161801 (2013).
  * (3) A. Aguilar-Arevalo, et al., MiniBoone Collaboration, Phys. Rev. Lett. 121, 221801 (2018).
  * (4) R.J. Hill, Phys. Rev.D 81, 013008 (2010).
  * (5) R.J. Hill, Phys. Rev.D 84, 017501 (2011).
  * (6) B.D. Serot and X. Zhang, Phys. Rev. C 86, 015501 (2012).
  * (7) X. Zhang and B.D. Serot, Phys. Lett. B, 719 (2013).
  * (8) E. Wang, L. Alvarez-Ruso and J. Nieves, Phys. Rev. C 89, 015503 (2014).
  * (9) E. Wang, L. Alvarez-Ruso and J. Nieves, Phys. Lett. B 740, 16 (2015).
  * (10) T. Katori for the MiniBoone collaboration, arXiv:hep-ex/2010.06015.
  * (11) G. Chanfray, J. Delorme, M. Ericson and M. Rosa-Clot, Phys. Lett. B 455, 39 (1999).
  * (12) M. Dey, V.L. Eletsky and B.L. Ioffe, Phys. Lett. B 252, 620 (1990).
  * (13) G. Chanfray and P. Schuck, Nucl. Phys. A 555, 329 (1993).
  * (14) G. Chanfray, R. Rapp and J. Wambach, Phys. Rev. Lett. 76, 368 (1996). R. Rapp, G. Chanfray and J. Wambach, Nucl. Phys. A 617, 472 (1997).
  * (15) E. Oset, H. Toki and W. Weise, Phys. Rept. 83, 281 (1982).
  * (16) G. Chanfray and M. Ericson, Phys. Rev C 75, 015204 (2011).
  * (17) W. Holt, M. Rho and W. Weise, Phys. Rept. 621, 2 (2016).
  * (18) V. Bernard, N. Kaiser and U.-G. Meissner, Int. J. Mod. Phys. E 4, 193 (1995).
  * (19) M. Ichimura, H. Sakai and T. Wakasa, Prog. Part. Nucl. Phys. 56, 446 (2006).
  * (20) G. Chanfray and M. Ericson, Phys.Rev. C75 015206 (2007).
  * (21) M. Martini, M. Ericson, G. Chanfray and J. Marteau, Phys.Rev. C80 065501 (2009).
  * (22) G. Chanfray and J. Delorme, Phys. Lett. B 129, 167 (1983); G. Chanfray, Nucl. Phys. A 429, 389 (1984).


# Mastery Guided Non-parametric Clustering to Scale-up Strategy Prediction

Anup Shakya
<EMAIL_ADDRESS>
University of Memphis  Anup Shakya, Vasile Rus, Deepak Venugopal

###### Abstract

Predicting the strategy (sequence of concepts) that a student is likely to use
in problem-solving helps Adaptive Instructional Systems (AISs) better adapt
themselves to different types of learners based on their learning abilities.
This can lead to a more dynamic, engaging, and personalized experience for
students. To scale up training a prediction model (such as LSTMs) over large-
scale education datasets, we develop a non-parametric approach to cluster
symmetric instances in the data. Specifically, we learn a representation based
on Node2Vec that encodes symmetries over mastery or skill level since, to
solve a problem, it is natural that a student’s strategy is likely to involve
concepts in which they have gained mastery. Using this representation, we use
DP-Means to group symmetric instances through a coarse-to-fine refinement of
the clusters. We apply our model to learn strategies for Math learning from
large-scale datasets from MATHia, a leading AIS for middle-school math
learning. Our results illustrate that our approach can consistently achieve
high accuracy using a small sample that is representative of the full dataset.
Further, we show that this approach helps us learn strategies with high
accuracy for students at different skill levels, i.e., leveraging symmetries
improves fairness in the prediction model.

## Introduction

The recent pandemic has spurred a remarkable growth in virtual learning and
with it, the necessity to develop personalized learning technologies in the
absence of face-to-face instruction. To this end, Intelligent Tutoring Systems
(ITSs) (Rus et al. 2013) and more broadly Adaptive Instructional systems
(AISs) scale-up one-on-one instruction to large and diverse student
populations. However, for an AIS to better adapt to a learner, it needs to
understand a learner’s problem-solving approach. For example, a student
attempting to solve a math problem shown in Fig. 1 (a) needs to formulate a
plan or a strategy to solve the problem, an example of which is also shown in
the figure. By predicting this likely strategy, an AIS can be personalized to
meet the student’s learning needs such as interventions specific to concepts
utilized in the likely strategy. In this paper, we focus on developing a
scalable model to predict Math learning strategies based on interactions
between students and educational software. In particular, we apply deep neural
network based models such as Long-Short Term Memory (LSTMs) (Hochreiter and
Schmidhuber 1997) to extract sequential patterns that correspond to learner
strategies. However, LSTMs are computationally expensive and notoriously slow
to converge (You et al. 2019) in large datasets. To scale-up, we take
advantage of symmetries in strategies. In particular, when we consider the
domain of math learning for middle-school students, it is highly unlikely that
each student will follow a completely unique strategy. Instead, it is more
likely to have groups of students who adopt similar/symmetrical strategies.
Further, these symmetry groups are based on the skills or mastery level of a
student as illustrated by an example in Fig. 1 (b).

Figure 1:  (a) Example of a strategy used by a student to solve a math
problem. (b) Different strategies to solve linear equations based on
increasing levels of mastery.

To scale-up strategy learning, we develop a non-parametric approach to
partition the data such that each partition is likely to be invariant in
strategies. Thus, training on samples drawn from the partitions effectively
represents strategies over the full data. To identify invariance in
strategies, we apply DP-Means (Kulis and Jordan 2012) where we jointly cluster
students and problems through the Hierarchical Dirichlet Process (HDP)
framework. We adaptively penalize the DP-Means objective such that we have a
coarse-to-fine refinement of the clusters. Specifically, we penalize the
formation of clusters based on symmetries induced by the clusters which we
estimate through positional embeddings (Vaswani et al. 2017). To learn a
representation for students/problems, we develop a mastery-based embedding.
Specifically, strategies are closely related to mastery over skills, i.e., if
a student has mastered a skill, he/she will naturally tend to apply that skill
in a strategy. We estimate mastery in a skill based on the attention given to
that skill in a Transformer model that maps skills to the correct application
of skills.

We evaluate our approach on two datasets from MATHia, a commercial AIS widely
used for math learning in schools. The data is available through the PSLC
datashop (Stamper et al. 2011). The Bridge-to-Algebra 2008-09 and the Carnegie
Learning 2019-20 datasets are both large datasets that consist of millions of
data instances (each instance is a student-problem pair and has multiple
interactions in the dataset). We show that our approach helps scale-up over
large datasets while yielding high accuracy in predicting strategies. Further,
we show that our mastery-based embeddings is fair, i.e., the predictions made
by our model is unbiased over diverse levels of mastery.

## Proposed Approach

### Problem Description

Since strategy is a generic term, we define it more precisely. Specifically,
we consider strategies in the context of structured interaction between
students and tutors. In this case, a student interacts with a tutor and solves
a problem by sequentially solving the steps that lead to the final solution.
Each step is associated with a specific knowledge component (KC) (Koedinger,
Corbett, and Perfetti 2012) which is defined by domain experts and corresponds
to the knowledge required to solve that step. Operationally, a step can be
associated with multiple KCs in which case, we can just unroll the step to
ensure that each step has a single KC. Using this, as in prior work (Ritter et
al. 2019), we define strategies as follows.

###### Definition 1

A strategy used by student $S$ to solve problem $P$ is a sequence of KCs
denoted by ${\bf K}$ $=$ $K_{1}$ $\ldots$ $K_{n}$, where $K_{i}$ is a
knowledge component that $S$ uses to solve the $i$-th step in $P$.

Our task is to predict the likely strategy for a given student-problem pair.
Note that the same problem can have several possible strategies as shown in
Fig. 1 (b). Typically, these strategies are closely related to the skill level
of a student, i.e., students at higher skill levels may use more complex
strategies than students at lower skill levels. We leverage this to learn a
representation (embedding) for students and problems such that similar
embeddings correspond to symmetries in strategies. We use non-parametric
clustering to cluster instances based on this representation and train an LSTM
model by sampling from the clusters instead of training over the full dataset.
Note that, the use of symmetries to learn better representations from data is
in fact a general concept that is formalized under Geometric Deep Learning
(Bronstein et al. 2021). By training deep models over informative instances
from groups of symmetric instances, the Effective Model Complexity is reduced
which improves generalization (Nakkiran et al. 2020).

### Strategy Invariance

We define symmetries over ${\bf S}$, ${\bf P}$ which denote the set of
students and problems respectively as follows.

###### Definition 2

A strategy-invariant partitioning w.r.t $\mathcal{D}$ is a partitioning
$\\{{\bf S}_{i}\\}_{i=1}^{k_{1}}$ and $\\{{\bf P}_{j}\\}_{j=1}^{k_{2}}$ such
that $\forall i,j$, if $S,S^{\prime}\in{\bf S}_{i}$, $P,P^{\prime}\in{\bf
P}_{j}$, $S,S^{\prime}$ follow equivalent strategies for $P,P^{\prime}$
respectively.

where $\mathcal{D}$ is the dataset (sequences of KCs for student-problem
pairs), $k_{1}$ and $k_{2}$ are the number of partitions/clusters for students
and problems respectively.

The benefit of strategy-invariant partitioning is that we can scale up without
sacrificing accuracy by training a prediction model only on samples drawn from
the partitions instead of the full training data. Therefore, our task is to
learn such a partitioning. Since it is hard to know apriori how many
partitions are needed, we formulate this as a non-parametric clustering
problem and use DP-Means (Kulis and Jordan 2012) to learn the clusters.

To formalize our approach, we begin with some notation. Let $x_{i1}$ denote
the $i$-th student and let $x_{j2}$ denote the $j$-th problem. Let $z_{ik}$
indicate the student/problem cluster to which $x_{ik}$ belongs. We refer to
the student and problem clusters as local clusters. A global cluster combines
student and problem clusters. Specifically, $v_{1c}$ $=$ $g$ and
$v_{2c^{\prime}}$ $=$ $g$ indicates that the $c$-th student cluster is
associated with the $c^{\prime}$-th problem cluster through the $g$-th global
cluster. The objective function is defined as follows.

$\sum_{p=1}^{g}\sum_{x_{ij}\in\ell_{p}}||x_{ij},\mu_{p}||_{2}^{2}+\lambda_{\ell}k+\mathcal{S}(\ell_{1}\ldots\ell_{p})g$
(1)

where $\ell_{p}$ is the $p$-th global cluster, $k$ $=$ $k_{1}+k_{2}$ is the
total number of local clusters, $g$ is the total number of global clusters,
$\lambda_{\ell}$ is a local penalty for the formation of a local cluster and
$\mathcal{S}(\ell_{1}\ldots\ell_{p})$ is a global penalty for the formation of
a global cluster. Specifically, $\mathcal{S}$ is a measure of symmetry in
strategies across the global clusters. To minimize Eq. (1), we can directly
apply DP-Means with the Hard Gaussian Hierarchical Dirichlet Process (HDP)
clustering as specified in (Kulis and Jordan 2012). This jointly clusters
students and problems while assigning the optimal number of clusters to each.
However, in our case, computing the global penalty
$\mathcal{S}(\ell_{1}\ldots\ell_{p})$ is challenging, i.e., how can we assign
a global penalty that corresponds to the symmetry in the local clusters? We do
this using a coarse-to-fine clustering as follows. We start with a fixed
(large) value for the global penalty $\lambda_{g}$ that results in a coarse
clustering. Then, depending upon the symmetries that we estimate from the
coarse clusters, we reduce $\lambda_{g}$ resulting in finer-grained clusters
until we reach a fixed point (the change in symmetries is minimal) or for a
fixed number of iterations. Algorithm 1 summarizes our overall approach for
coarse-to-fine clustering. In the next two sections, we describe embeddings
(based on mastery) that we learn to represent the data instances in Algorithm
1 and the quantification of approximate symmetries in a given clustering.

Algorithm 1 Coarse-to-Fine Refinement

0. Student/Problem set: {$x_{ij}$}, Penalty parameter: $\lambda_{\ell}$

0. Global strategy clustering {$\ell_{1},\dots,\ell_{g}$} Initialize : Global cluster penalty $\lambda_{g}=y$ (where $y$ is a large number), $t=0$, cluster coherence $coh_{t-1}=0$.

1. repeat

2. $t=t+1$

3. Perform HDP clustering with parameters: ({$x_{ij}$}, $\lambda_{\ell}$, $\lambda_{g}$)

4. Compute cluster coherence score, $coh_{t}$ = $\mathcal{S}(\ell_{1},\ldots\ell_{g})$

5. Reduce: $\lambda_{g}=\lambda_{g}-\epsilon$

6. until $coh_{t}>coh_{t-1}$

### Mastery-based Embedding

Mastery in a skill implies that a student is able to apply the skill correctly
when they have the opportunity to apply it. For example, if a student has
mastered the skill of addition, then the student should be able to apply this
in an equation or in a word problem. Mastery has been explored in several
well-known learning models, and arguably one of the most well-known methods is
Bayesian Knowledge Tracing (BKT) (Corbett 2001). Here, mastery over a skill is
formalized as a latent variable in an HMM model. This variable is updated
based on the performance of the student at each opportunity given to the
student to apply that skill. More recently, Deep Knowledge Tracing (DKT)
(Piech et al. 2015) has shown that the hidden layers in a deep network can
learn more complex representations of student skills which are much more
expressive than BKT. In particular, the connection between symmetries in
strategies and mastery is that the ability to use a KC $K$ in a strategy is
dependent upon the level of skill in $K^{\prime}$ that is a pre-requisite to
$K$ (also called as pre-requisite structure in DKT). Thus, students with
similar pre-requisite structures will tend to use symmetrical strategies.
Here, we train an attention model to quantify mastery over a KC and use it to
learn an embedding which we refer to as MVec.

Figure 2: Illustrating a graph network of three students, problems, and KCs.
The figure on the right shows some of the sampled random walks/paths.

To learn the MVec embeddings, we use an approach similar to Node2Vec (Grover
and Leskovec 2016). Specifically, we construct a relational graph
$\mathcal{G}$ $=$ (V,E) as follows. Each student, problem, and KC in the
training data is represented as a node $V\in{\bf V}$. For every student $S$
who uses KC $K$ as a step to solve problem $P$, there exists 2 edges
$E,E^{\prime}$ $\in$ E, where $E$ connects the node representing the student
to the node representing the KC and $E^{\prime}$ connects the node
representing the KC to the node representing the problem. An example graph
over 3 students, problems, and KCs is shown in Fig. 2. We now sample paths in
the graph similar to Node2Vec and learn embeddings for these paths using word
embedding models (Word2Vec) (Mikolov et al. 2013). Specifically, the objective
function is as follows.

$\max_{f}\sum_{u\in{\bf V}}logP(N_{Q}(u)|f(u))$ (2)

where $f:u\rightarrow\mathbb{R}^{d}$ is the representation for nodes $u\in{\bf
V}$, $N_{Q}(u)$ denotes the neighbors of $u$ sampled from a distribution $Q$.
As in Node2Vec, we assume a factorized model that gives us the conditional
likelihood that is identical to Word2Vec.

$P(n_{i}|f(u))=\frac{\exp(f(n_{i})\cdot f(u))}{\sum_{v\in{\bf
V}}\exp(f(v)\cdot f(u))}$ (3)

where $n_{i}$ is a neighbor of $u$. The conditional likelihood is optimized by
predicting neighbors of $u$ using $u$ as input in an autoencoder neural
network. The hidden layer learns similar embeddings for nodes with symmetrical
neighborhoods. To do this, we generate walks on $\mathcal{G}$ as shown for the
example in Fig. 2, and in each walk, given a node, we predict neighboring
nodes similar to predicting neighboring words in sentences. To generate these
walks, a simple sampling strategy $Q$ is to randomly sample a neighbor for a
node. However, in our case, it turns out that each neighbor may have different
importance when it comes to determining symmetry. Specifically, if a student
has achieved mastery in applying a KC to a problem, then the corresponding
edges should be given greater importance when determining symmetry between
nodes in $\mathcal{G}$. To do this, we train a Sequence-to-Sequence attention
model (Vaswani et al. 2017) from which we estimate the sampling probabilities
for edges in $\mathcal{G}$.

Figure 3: An example to illustrate the use of attention for mastery
estimation. The bar charts show for each KC, the attention on a KC across
steps that the student solves successfully (CFA=1) normalized by total
attention for that KC. Larger values indicate that the model believes the
student understands the KC as the attentions on it is large when CFA=1 and
vice versa.

The intuitive idea in quantifying mastery is illustrated in Fig. 3 which shows
the opportunities given to 3 students to apply KCs in different problems. For
each sequence of KCs, we predict if the student got the step correct or wrong
in the first attempt (abbreviated as CFA for Correct First Attempt) when given
an opportunity to apply the KC. The CFA values are performance indicators for
the student, i.e., if they have mastered a KC, then they are likely to get the
step correct in every opportunity they get to apply that KC. We train a model
to predict the CFA values (CFA $=$ 1 indicates a correct application of the
KC) given the KCs used in a problem. The predicted values from the model are
shown for each KC. The bar graphs show mastery over the KCs. As seen here, the
first student is inconsistent in applying the skill, find the slope using
points (labeled as $E$) since the predictions for this oscillate between 0 and
1 whenever the student tries to apply this KC. On the other hand, student 2
consistently applies the same skill correctly and therefore the attention
value is higher. We train the attention model from opportunities based on
curriculum structure. Specifically, the curriculum consists of multiple units
and each unit is further subdivided into sections. For each student $S$, from
every unit that the student has completed say $U$, we select a problem $P$
from each section that the student has worked on in $U$ and train the model to
predict the CFA values for each KC used in $P$. We use the standard
architecture described in (Vaswani et al. 2017) for this model. Specifically,
the input consists of the KC sequence, and the encoder maps this sequence to a
latent representation and the decoder decodes the CFA values one at a time.
The attention is given by,

$Attention(Q,Y,V)=softmax\left(\frac{QY^{T}}{\sqrt{d_{k}}}\right)V$ (4)

where $Q$, $Y$, and $V$ are the standard query, key, and value matrices
respectively as defined in (Vaswani et al. 2017), and $d_{k}$ is the dimension
of the embedding that represents the latent representations. We use the
encoder-decoder attention, i.e., the query is the decoder representation and
the key is the encoder representation. The attention weights are an estimate
of the alignment between encoded latent representations of mastery with the
decoded representation of correctly applying a skill at each step in the
problem. The projection of mastery over $K$ based on the attention vectors is
estimated by the following equation.

$\alpha(S,P,K)=\frac{\sum_{i}\sum_{v\in\pi(a_{i})}v}{\sum_{i}\sum_{v\in\pi(a_{i})}v+\sum_{i}\sum_{v\in\bar{\pi}(a_{i})}v}$
(5)

where $\pi(\cdot)$ extracts only those values in the input vector where the
corresponding output for that step is predicted as 1, i.e., the model
predicted that the student could solve the step correctly. $\bar{\pi}(\cdot)$
is the complement of $\pi(\cdot)$, i.e., it extracts attention values
corresponding to steps that were predicted as mistakes. We sample
$\mathcal{G}$ as a factored distribution, i.e., $Q(S)*Q(K|S)*Q(P|K,S)$, where
$Q(S)$ is the probability of sampling a student node, $Q(K|S)$ is the
probability of sampling a KC $K$ given student $S$ and $Q(P|K,S)$ is the
probability of sampling problem $P$ given $K,S$. We assume that $Q(S)$ is a
uniform distribution over students. The conditional distributions are defined
as follows.

$Q(K|S)=1/n\sum_{p}\alpha(S,P,K)$ (6) $Q(P|K,S)=\alpha(S,P,K)$ (7)

where $n$ is the number of opportunities given to student $S$ to apply KC $K$.
The algorithm to generate MVec embeddings is shown in Algorithm 2. As shown
here, we sample a path in the graph as follows. We first sample student $S$
uniformly at random, then we sample a KC $K$ from $Q(K|S)$ and a problem from
$Q(P|K,S)$. We then predict each node in the path using the neighboring nodes
through a standard Word2Vec model. The resulting embeddings are learned in the
hidden layer of this model. Note that for scalability, we do not
construct/store the full graph $\mathcal{G}$ at any point. Instead, we only
sample paths in an online manner as shown in Algorithm 2.

Algorithm 2 Generate MVec embeddings

0. Relation Graph: $\mathcal{G}=(\bf V,\bf E)$ with student, problem and KCs as nodes, Embedding dimension: $d$, pre-trained attention-model $\mathcal{A}$

0. Embeddings for each node $v\in\mathbb{R}^{d}$ Initialize: set of walks, $\mathcal{W}=empty$

1. for all $t$ $=$ 1 to $T$ do

2. Sample a path $<S,K,P>$ in $\mathcal{G}$ from $Q(S)*Q(K|S)*Q(P|K,S)$ using Eq. (6) and (7).

3. $\mathcal{W}$ $=$ $\mathcal{W}$ $\cup$ $<S,K,P>$

4. end for

5. $v_{e}=word2vec(\mathcal{W},d)$

6. return $v_{e}$

Table 1: Illustrating symmetries in strategies where similar colors indicate
similar steps. The three strategies have a different number of steps and other
small differences but the core idea behind them remains the same. A naive
matching will however assume that the three strategies are very different.

Strategy 1 | action add/sub | params add/sub | rm-extra-terms | rm-coeff | act mul/div | params mul/div | rm-extra-terms | find-x
---|---|---|---|---|---|---|---|---
Strategy 2 |  |  | rm-constants | rm-coeff |  | div-coeff-of-x | rm-extra-terms | find-x
Strategy 3 |  | rm-coeff |  | div-coeff-of-x | simplify-eqn | rm-constants | rm-extra-terms | find-x

### Strategy Symmetry

To perform the coarse-to-fine refinement (Algorithm 1), we need to estimate
symmetry in strategies across the global clusters, i.e.,
$\mathcal{S}(\ell_{1},\ldots,\ell_{g})$. A naive approach is to represent the
sequence of KCs as a single vector by taking a mean of their MVec embeddings.
In this case, we can estimate the symmetry between the two strategies based on
the distance of their mean vectors. However, this representation assumes that
all permutations of a sequence of KCs are equivalent to each other. This is
clearly problematic since an essential aspect of reasoning about problems is
to understand the sequence of steps. On the other hand, if we strictly require
the KCs from two strategies to align exactly, this is too restrictive since it
is perfectly acceptable for students to add/delete steps in reasoning though
the core strategy remains equivalent. An example of this is illustrated in
Table 1. In this case, 3 strategies are shown which differ in their steps.
However, the core idea behind each of the strategies is the same and it turns
out that aligning the key steps makes the strategy invariant across the
students as shown in the figure.

To match strategies approximately, we represent a strategy using a combination
of embeddings and positional encodings (Vaswani et al. 2017) and approximately
align two strategies to estimate the symmetry between them. The idea in
positional encodings is to encode positions in a sequence using a continuous
vector space with sine and cosine functions. Given the positional encoding
vectors for say the $i$-th KC in strategy $S_{1}$ and the $j$-th KC in
strategy $S_{2}$, their similarity $\vec{p}_{i}^{\top}\vec{p}_{j}$ will
indicate how close or far apart these positions are. Note that this similarity
takes into account the variability in the number of steps in $S_{1}$ and
$S_{2}$. Thus, the step-positions in the two strategies will have similar
vector representation if they are at similar positions relative to their
overall sequences.

A KC $K$ in the strategy is represented by its positional embedding $\vec{K}$
$=$ $\vec{K_{e}}$ $+$ $\vec{K_{p}}$ where $\vec{K_{e}}$ is the embedding for
$K$ and $\vec{K_{p}}$ is the positional encoding for $K$ in the strategy. To
compute symmetry between strategies, we compute an alignment between their
positional embeddings. Alignment is a fundamental problem in domains such as
Bioinformatics where a classical approach that is often used is the Smith and
Waterman algorithm (SW) (Smith and Waterman 1981). The idea is to perform
local search to compute the best possible alignment between two sequences. SW
requires a similarity function which in our case is the similarity between two
KCs and we set this to be $s(K,K^{\prime})$ $=$
$\vec{K}^{\top}\vec{K}^{\prime}$, i.e., the cosine similarity between the
embeddings of $K$ and $K^{\prime}$. Further, SW also requires a gap penalty
which refers to the cost of leaving a gap in the alignment. In our case, we
set the gap penalty to 0 since we want symmetry between strategies to be
invariant to gaps. That is, if two strategies are symmetric, adding extra
steps in the strategies is acceptable. SW iteratively computes a scoring
matrix based on local alignments. The worst-case complexity to compute the
scoring matrix that gives us scores for the best alignment is equal to
$O(m*n)$ where $m$ and $n$ are lengths of the strategies. Note that in our
case, we are interested in quantifying the symmetry based on the alignment.
Specifically, let ${\bf K}$ and ${\bf K}^{\prime}$ be two strategies of
lengths $n$ and $m$ respectively. SW gives us an alignment between ${\bf K}$
and ${\bf K}^{\prime}$ denoted by $L({\bf K},{\bf K}^{\prime})$. The alignment
can be represented by pairs of aligned KCs in the strategy or a gap. We
compute the symmetry score between ${\bf K}$ and ${\bf K}^{\prime}$ as $r({\bf
K},{\bf K}^{\prime})$ $=$ $\frac{1}{\max(n,m)}$ $\sum_{(K,K^{\prime})\in
L({\bf K},{\bf K}^{\prime})}$ $(\vec{K}^{\top}\vec{K^{\prime}})$, where
$(K,K^{\prime})$ $\in$ $L({\bf K},{\bf K}^{\prime})$ are aligned KCs and
$\vec{K}^{\top}\vec{K^{\prime}}$ is their similarity. We see that $0\leq$
$r({\bf K},{\bf K}^{\prime})$ $\leq 1$. Based on this, we estimate symmetry in
the clustering as follows.

$\displaystyle\mathcal{S}(\ell_{1},\ldots,\ell_{g})=$
$\displaystyle\frac{1}{g}\sum_{p=1}^{g}\mathbb{Z}_{p}\sum_{{\bf K},{\bf
K^{\prime}}\in T(\ell_{p})}r({\bf K},{\bf K}^{\prime})$ (8)

$T(\ell_{p})$ is a set of all strategies in $\ell_{p}$, i.e., $\forall
S,P\in\ell_{p}$, where $S$ is a student and $P$ is a problem, $\exists{\bf
K}\in T(\ell_{p})$, where ${\bf K}$ is the strategy followed by $S$ for $P$
and $\mathbb{Z}_{p}=\frac{2}{|T(\ell_{p})|(|T(\ell_{p})|-1)}$ is the
normalization term. Thus, a larger value of
$\mathcal{S}(\ell_{1},\ldots,\ell_{g})$ implies that the clustering
corresponding to $\ell_{1},\ldots,\ell_{g}$ has a greater degree of symmetry
in strategies.

### Training the Prediction Model

We use the LSTM architecture in (Shakya, Rus, and Venugopal 2021) to predict
strategies. Specifically, the model is a one-to-many LSTM that takes the
embeddings corresponding to a student and a problem as input and generates a
sequence of knowledge components as output. To train the model, we sample
instances from the converged global clusters, and for each sampled student-
problem pair, we concatenate their positional embeddings and feed them as
input to the LSTM. The output corresponds to a sequence of KCs, each of which
is encoded as a one-hot vector. To handle variable-length strategies, a
special stop symbol is used to denote the end of a sequence. The entire model
is trained using the categorical cross-entropy loss.

## Experiments

Table 2: Main parameters for the models.

Transformer-based model | LSTM-based strategy model
---|---
Dimension $\rightarrow$ 512 | Latent Dimension $\rightarrow$ 200
Number of layers $\rightarrow$ 6 | Epochs $\rightarrow$ 60
Number of heads $\rightarrow$ 8 | Batch Size $\rightarrow$ 30
Dimensions of key, value and query $\rightarrow$ 64 | Adam Optimizer with Learning rate 0.01
Max Sequence Length $\rightarrow$ 150 | Dropout $\rightarrow$ 0.1
Dropout $\rightarrow$ 0.1 |
Weight Sharing $\rightarrow$ False |

Figure 4: Illustrating Scalability vs Accuracy. (a), (b) show test accuracy
for strategy prediction for varying training datasize limits. (c), (d) show
accuracy (strategy prediction) for different training time limits. (e) shows
accuracy for different groups of students (based on their performance). The
x-axis denotes different ranges of %s, where a range $a-b$ denotes that
students in this group got $>a$ and $<b$ steps correct in their first attempt.
The y-axis shows accuracy over the groups.

### Experimental Setup

We used two datasets provided by MATHia for evaluating our proposed approach,
Bridge-to-Algebra 2008-09 (BA08) and Carnegie Learning MATHia 2019-20 (CL19).
BA08 consists of about $20$ million interactions for about $6000$ students and
$52k$ unique algebra problems and $1.6$ million instances (student, problem
pairs). CL19 contains $47$ million interactions for $5000$ students, $32k$
problems and $1.9$ million instances. Both datasets are publicly available
through the PSLC datashop. To train the attention model, we used the
implementation in (Vaswani et al. 2017). For the strategy prediction, we used
the LSTM model as described in the previous section. The parameters for the
two models are shown in Table 2. For generating MVec embeddings, we used
Gensim (Mikolov et al. 2013) with an embedding dimension set to 300. We
initialize the local cluster penalty $\lambda_{\ell}=7$ and initial global
cluster penalty $\lambda_{g}=9$ for Coarse-to-Fine refinement. We perform our
experiments with 64 GB RAM and a 16GB quadro GPU.

### Comparision to Baselines

We compared our approach with the following methods. The first one is the
approach in (Shakya, Rus, and Venugopal 2021) ($\mathrm{CS}$) where the LSTM
is trained over samples, but mastery is not used to learn the embeddings. We
also applied a general importance sampling approach for deep learning proposed
in (Katharopoulos and Fleuret 2019) ($\mathrm{IS}$) using their publicly
available implementation. However, $\mathrm{IS}$ failed to output any results
for datasets of our size and therefore we do not show it in our graphs. We
also developed a stratified sampler ($\mathrm{GS}$ for group sampling) where
the distribution is only proportional to the number of problems solved by a
student, i.e., we sample more instances from students that have data
associated with them. The last baseline is a naive Random Sampler
($\mathrm{RS}$) used as a validation check where we sample students and
problems uniformly at random. We refer to our approach as Attention Sampling
($\mathrm{AS}$). In our evaluation, for each approach, we enforce a limit on
the number of training instances and measure the test accuracy of models
trained with this limit. This is similar to effective model complexity
(Nakkiran et al. 2020) which is the number of training samples to achieve
close to zero error. We report the average accuracy based on three training
runs.

### Results

Accuracy and Scalability. The accuracy results for BA08 and CL19 are shown in
Fig. 4 (a) and (b). As shown in (a), for BA08, in our approach
($\mathrm{AS}$), it takes less than around $10\%$ of the data to obtain test
accuracy that is greater than 80$\%$. $\mathrm{CS}$ is the next best performer
but is consistently below $\mathrm{AS}$ for all training sizes. $\mathrm{GS}$
performs significantly worse which illustrates that symmetries are more
complex and a simple grouping based on problems/students is insufficient. The
poor performance of $\mathrm{RS}$ validates that the problem of choosing the
correct samples is indeed a challenging one. As seen in Fig. 4 (b), for a
considerably larger dataset CL19, we can observe similar performance as in
BA08. $\mathrm{AS}$ remains the best performer and here $\mathrm{CS}$ is less
stable since we see a performance drop as we increase the limit on training
samples. This suggests that $\mathrm{CS}$ may not be able to capture all
symmetries (at all mastery levels) and thus may produce a less effective
training sample. The results for $\mathrm{GS}$ and $\mathrm{RS}$ are similar
to those observed in BA08. As mentioned before, $\mathrm{IS}$ failed to
produce any results. Fig. 4 (c) and (d) show the training time required to
obtain a specific accuracy for BA08 and CL19 respectively. Even with the extra
processing that is needed to compute the mastery-based embeddings and the non-
parametric clustering, $\mathrm{AS}$ requires the shortest training time to
achieve an accuracy that is higher than the other approaches. This illustrates
the significance of leveraging symmetries in the data to train the model. As
mentioned before, the full data is infeasible to train and when attempting to
use the full data, the model did not converge even after several days of
training time. For CL19, the training time is larger since it takes longer to
compute the groups using non-parametric clustering due to the larger size of
the dataset. However, considering that CL19 is significantly larger than BA08,
we see that $\mathrm{AS}$ can easily scale big datasets.

Analyzing Bias. The strategy prediction model is fair if it does not
disproportionately favor a sub-group of students, i.e., if it strategy
predictions are not biased for students at a specific mastery-level. To
evaluate fairness, we divided the students in the test data into 5 performance
groups. The performance groups are based on the $\%$ of problem steps the
students solve correctly in their first attempt. For a student $S$ in
performance group ${\bf G}$, we predict the strategies for all problems
attempted by $S$ in the test set and measure the average accuracy $\mu_{S}$.
We then compute the accuracy over a performance group as $1/|{\bf
G}|\sum_{S\in{\bf G}}\mu_{S}$. Fig. 4(e) shows our results for variants, NS
(no symmetries are used), SS (approach in (Shakya, Rus, and Venugopal 2021)
that does not use mastery) and SS+MS (our approach that uses mastery). As
shown here, SS+MS yields a consistent improvement in performance across all
performance groups illustrating that our approach is fair across diverse
mastery levels.

## Conclusion

Mastery over concepts is closely related to problem-solving strategy. Using
this connection, here, we developed embeddings and through non-parametric
clustering on these embeddings, we segregated big datasets into partitions
where strategies are likely to be invariant. Sampling from these partitions
scales up strategy prediction such that it performs equally well for students
at all skill levels. We demonstrated the effectiveness of our approach in Math
learning datasets.

## References

  * Bronstein et al. (2021) Bronstein, M. M.; Bruna, J.; Cohen, T.; and Velickovic, P. 2021.  Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges.  _CoRR_ , abs/2104.13478.
  * Corbett (2001) Corbett, A. T. 2001.  Cognitive Computer Tutors: Solving the Two-Sigma Problem.  In _Proceedings of the 8th International Conference on User Modeling 2001_ , 137–147.
  * Grover and Leskovec (2016) Grover, A.; and Leskovec, J. 2016.  Node2vec: Scalable Feature Learning for Networks.  In _Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ , 855–864.
  * Hochreiter and Schmidhuber (1997) Hochreiter, S.; and Schmidhuber, J. 1997.  Long Short-Term Memory.  _Neural Computation_ , 9(8): 1735–1780.
  * Katharopoulos and Fleuret (2019) Katharopoulos, A.; and Fleuret, F. 2019.  Processing Megapixel Images with Deep Attention-Sampling Models.  In _ICML_ , volume 97 of _Proceedings of Machine Learning Research_ , 3282–3291. PMLR.
  * Koedinger, Corbett, and Perfetti (2012) Koedinger, K. R.; Corbett, A. T.; and Perfetti, C. 2012.  The Knowledge-Learning-Instruction Framework: Bridging the Science-Practice Chasm to Enhance Robust Student Learning.  _Cogn. Sci._ , 36: 757–798.
  * Kulis and Jordan (2012) Kulis, B.; and Jordan, M. I. 2012.  Revisiting k-means: New Algorithms via Bayesian Nonparametrics.  In _Proceedings of the 29th International Conference on Machine Learning_.
  * Mikolov et al. (2013) Mikolov, T.; Sutskever, I.; Chen, K.; Corrado, G. S.; and Dean, J. 2013.  Distributed Representations of Words and Phrases and their Compositionality.  In _Neural Information Processing Systems_ , 3111–3119.
  * Nakkiran et al. (2020) Nakkiran, P.; Kaplun, G.; Bansal, Y.; Yang, T.; Barak, B.; and Sutskever, I. 2020\.  Deep Double Descent: Where Bigger Models and More Data Hurt.  In _ICLR_.
  * Piech et al. (2015) Piech, C.; Bassen, J.; Huang, J.; Ganguli, S.; Sahami, M.; Guibas, L. J.; and Sohl-Dickstein, J. 2015.  Deep Knowledge Tracing.  In _Advances in Neural Information Processing Systems_ , volume 28.
  * Ritter et al. (2019) Ritter, S.; Baker, R.; Rus, V.; and Biswas, G. 2019.  Identifying Strategies in Student Problem Solving.  _Design Recommendations for Intelligent Tutoring Systems_ , 7: 59–70.
  * Rus et al. (2013) Rus, V.; D’Mello, S. K.; Hu, X.; and Graesser, A. C. 2013.  Recent Advances in Conversational Intelligent Tutoring Systems.  _AI Magazine_ , 34(3): 42–54.
  * Shakya, Rus, and Venugopal (2021) Shakya, A.; Rus, V.; and Venugopal, D. 2021.  Student Strategy Prediction using a Neuro-Symbolic Approach.  In _Proceedings of the 14th International Educational Data Mining Conference (EDM 21)_.
  * Smith and Waterman (1981) Smith, T.; and Waterman, M. 1981.  Identification of common molecular subsequences.  _Journal of Molecular Biology_ , 147(1): 195–197.
  * Stamper et al. (2011) Stamper, J. C.; Koedinger, K. R.; de Baker, R. S. J.; Skogsholm, A.; Leber, B.; Demi, S.; Yu, S.; and Spencer, D. 2011.  DataShop: A Data Repository and Analysis Service for the Learning Science Community.  In _AIED_ , volume 6738, 628.
  * Vaswani et al. (2017) Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, L.; and Polosukhin, I. 2017.  Attention is All you Need.  In _NIPS_ , 5998–6008.
  * You et al. (2019) You, Y.; Hseu, J.; Ying, C.; Demmel, J.; Keutzer, K.; and Hsieh, C.-J. 2019.  Large-Batch Training for LSTM and Beyond.  In _International Conference for High Performance Computing, Networking, Storage and Analysis_.