# Scar States in Deconfined $\mathbb{Z}_{2}$ Lattice Gauge Theories

Adith Sai Aramthottil  Instytut Fizyki Teoretycznej, Uniwersytet Jagielloński,
Łojasiewicza 11, PL-30-348 Kraków, Poland  Utso Bhattacharya  ICFO-Institut de
Ciències Fotòniques, The Barcelona Institute of Science and Technology, Av.
Carl Friedrich Gauss 3, 08860 Castelldefels (Barcelona), Spain  Daniel
González-Cuadra  ICFO-Institut de Ciències Fotòniques, The Barcelona Institute
of Science and Technology, Av. Carl Friedrich Gauss 3, 08860 Castelldefels
(Barcelona), Spain  Center for Quantum Physics, University of Innsbruck, 6020
Innsbruck, Austria  Institute for Quantum Optics and Quantum Information of
the Austrian Academy of Sciences, 6020 Innsbruck, Austria  Maciej Lewenstein
ICFO-Institut de Ciències Fotòniques, The Barcelona Institute of Science and
Technology, Av. Carl Friedrich Gauss 3, 08860 Castelldefels (Barcelona), Spain
ICREA, Passeig Lluis Companys 23, 08010 Barcelona, Spain  Luca Barbiero
Institute for Condensed Matter Physics and Complex Systems, DISAT, Politecnico
di Torino, I-10129 Torino, Italy  ICFO-Institut de Ciències Fotòniques, The
Barcelona Institute of Science and Technology, Av. Carl Friedrich Gauss 3,
08860 Castelldefels (Barcelona), Spain  Jakub Zakrzewski  Instytut Fizyki
Teoretycznej, Uniwersytet Jagielloński, Łojasiewicza 11, PL-30-348 Kraków,
Poland  Mark Kac Complex Systems Research Center, Uniwersytet Jagielloński,
Kraków, Poland

###### Abstract

The weak ergodicity breaking induced by quantum many-body scars (QMBS)
represents an intriguing concept that has received great attention in recent
years due to its relation to unusual non-equilibrium behaviour. Here we reveal
that this phenomenon can occur in a previously unexplored regime of a lattice
gauge theory, where QMBS emerge due to the presence of an extensive number of
local constraints. In particular, by analyzing the gauged Kitaev model, we
provide an example where QMBS appear in a regime where charges are deconfined.
By means of both numerical and analytical approaches, we find a variety of
scarred states far away from the regime where the model is integrable. The
presence of these states is revealed both by tracing them directly from the
analytically reachable limit, as well as by quantum quenches showing
persistent oscillations for specific initial states.

#### Introduction.–

The thermalization properties of isolated quantum systems are currently under
intensive investigation in different areas of modern quantum physics [1, 2, 3,
4]. In this context, a huge attention has been recently devoted towards the
study of a large variety of Hamiltonians where the ergodicity is weakly broken
[5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24].
In particular, in such models quenches from carefully designed initial states
reveal persistent many-body revivals that apparently contradict ergodicity [5,
25]. The reason for such an absence of thermalization turns out to be the
presence of specific eigenstates called quantum many-body scars (QMBS)
characterized, in particular, by a subvolume entanglement law [6, 7, 8, 9, 10,
11, 12, 13, 14, 21, 26, 24]. It is worth noting that similar regular states
have been previously identified [27] for chaotic quantum billiards in relation
to semiclassical periodic orbit quantization [28, 29]. Moreover, the concept
of scarred symmetry, closely related to QMBS, has been introduced in studies
of hydrogen atom in a strong magnetic field [30].

Figure 1: Properties of the Ising model (5) in $t-h$ parameter space for
$L=16$ and $\mu=1$. QMBS may exist in chaotic regions with mean gap ratio
$r>0.5$ and low entanglement entropy $S<0.5$ with respect to the GOE value.
The corresponding disconnected blue region lies entirely in charge deconfined
(CD) regime. Charge confinement (CC) region lies mainly within the regime of
chaotic, $r>0.5$, but with lowest entropy states, $S>0.5$, thus with no chance
for QMBS. Pink color marks the mixed dynamics regime, $r<0.5$, $S<0.5$; in
dark blue region while $r<0.5$, $S>0.5$. ‘0’ and ‘I’ denote paths along which
QMBS states are followed using level dynamics.

With the advent of a new generation of cold-atom quantum simulators [31, 32,
33, 34, 35], the weak ergodicity breaking manifested by QMBS has been
experimentally detected in constrained spin [5] and bosonic [36] models.
Crucially, both realizations can be viewed as lattice gauge theories (LGTs)
where the energy constraints are induced by the Gauss law fixing the relation
between gauge and charge variables. Motivated by the recent progress achieved
in the last years in implementing LGTs in quantum simulators [37, 38, 39, 40,
41, 42], an impressive theoretical effort has been devoted towards a better
understanding of simple gauge-invariant theories [43, 44, 45, 46, 47, 48, 49,
50, 17, 18, 51, 52, 53, 54, 55, 56, 57]. In this direction, the connection
between Gauss law and QMBS has recently gained attention both in $U(1)$ [16,
58] and $\mathbb{Z}_{2}$ [13] LGTs. Here indeed, the weak ergodicity breaking
associated with the slow oscillatory dynamics can be interpreted as a string
inversion phenomenon. Crucially, it has to be underlined that all LGTs, where
QMBS have been identified, are characterized by charge confinement. This
regime implies that only particle-antiparticle bound states exist and
therefore charges can be observed in composite structures only. These
effective pairs, together with an emergent new symmetry, generate the slowdown
of quantum dynamics and have been shown to be intricately connected to the
presence of QMBS [59].
Thus it seems natural to wonder whether confinement is a prerequisite to
observe QMBS in LGTs. In this Letter, we tackle this question by investigating
the recently introduced gauged 1D Kitaev model [60, 61, 16], whose ground
state displays a confined phase as well as a regime where charges are not
bounded in pairs, thus describing a deconfined phase. We study its low-
entanglement states and show that QMBS are present in the ergodic deconfined
phase and are absent in the ergodic confined regime of the model. Importantly,
we are able to continuously track QMBS down from the analytic prediction valid
in the quasi-integrable regime and therefore provide their partial
classification.

#### Model and observables.–

The Hamiltonian of the $p$-wave superconducting Kitaev chain minimally coupled
to a $\mathbb{Z}_{2}$ gauge field introduced in [60, 61, 16] reads

$\displaystyle H$ $\displaystyle=$
$\displaystyle-t\sum_{j}\left(c^{\dagger}_{j}-c_{j}\right)\sigma^{z}_{j+1/2}\left(c^{\dagger}_{j+1}+c_{j+1}\right)$
(1)
$\displaystyle-\mu\sum_{j}\left(c^{\dagger}_{j}c_{j}-\frac{1}{2}\right)-h\sum_{j}\sigma^{x}_{j+1/2}.$

Here, $c^{\dagger}_{j}$ ($c_{j}$) denotes the fermionic creation
(annihilation) operator and $t$ describes the tunneling and pair
production/annihilation processes mediated by the $\mathbb{Z}_{2}$ gauge field
$\sigma^{z}_{j+1/2}$ defined on the links between the nearest neighbor sites.
Fluctuations in the gauge field are induced by the electric field
$\sigma^{x}_{j+1/2}$ of strength $h$ and the number of fermions is fixed by
the chemical potential $\mu$. Here $\sigma^{i}_{j+1/2}$ stand for standard
Pauli matrices.

As required in LGTs, the model (1) is invariant under the local gauge
transformation generated by the Gauss operator
$G_{j}=\sigma^{x}_{j-1/2}(-1)^{n_{j}}\sigma^{x}_{j+1/2}$ where $[H,G_{j}]=0$
and $[G_{i},G_{j}]=0$. Therefore, the physical states are those that satisfy
the Gauss law, $G_{j}\ket{\psi}=\pm\ket{\psi}$, for all $j$ sites [62].

The Hamiltonian (1) can be written in terms of gauge-invariant transformed
Pauli operators:

$\displaystyle X_{i+\frac{1}{2}}$ $\displaystyle=$
$\displaystyle\sigma^{x}_{i+\frac{1}{2}}$ (2) $\displaystyle
Y_{i+\frac{1}{2}}$ $\displaystyle=$
$\displaystyle\left(c^{\dagger}_{i}-c_{i}\right)\sigma^{y}_{i+\frac{1}{2}}\left(c^{\dagger}_{i+1}+c_{i+1}\right)$
(3) $\displaystyle Z_{i+\frac{1}{2}}$ $\displaystyle=$
$\displaystyle\left(c^{\dagger}_{i}-c_{i}\right)\sigma^{z}_{i+\frac{1}{2}}\left(c^{\dagger}_{i+1}+c_{i+1}\right).$
(4)

Upon this transformation and with periodic boundary conditions, the gauged
Kitaev model corresponds to the quantum Ising model with both transverse and
longitudinal fields

$H=\sum_{i=1}^{L}\frac{\mu}{2}Z_{i}Z_{i+1}-tX_{i}-hZ_{i}.$ (5)

Notice that the model is now defined on a dual lattice, where the index $i$
corresponds to the links of the original model (1).

For $\mu>0$, the phase diagram of the model (5), as shown in Fig. 1, is
characterized by the presence of both an antiferromagnetic (AFM) order and a
paramagnetic (PM) phase depending on parameters $t$ and $h$ [63]. The AFM
phase turns out to be of great interest, since it supports domain-wall
excitations associated with the antiferromagnetic order caused by the
spontaneous breaking of the $\mathbb{Z}_{2}$ Ising and translational
symmetries. In 1D these last two features imply domain wall deconfinement.
Therefore, in the gauged Kitaev Hamiltonian, (1), such an AFM order
corresponds to charge deconfinement (CD), where fermions are free to expand
without any string tension. On the other hand, the PM regime corresponds to a
phase characterized by charge confinement (CC), where fermions appear only as
bound pairs.

In the limit $t\ll\mu$ it is possible to perform Schrieffer-Wolff
transformation (in agreement with [60], for a higher order expansion see
[64]), with

$\displaystyle S$ $\displaystyle=$
$\displaystyle\frac{-it}{2\mu}\sum_{j}\Bigg{\\{}\left(\frac{1+Z_{j-1}}{2}\right)Y_{j}\left(\frac{1+Z_{j+1}}{2}\right)$
(6)
$\displaystyle-\left(\frac{1-Z_{j-1}}{2}\right)Y_{j}\left(\frac{1-Z_{j+1}}{2}\right)\Bigg{\\}},$

to obtain an effective Hamiltonian $H_{\rm
eff}=e^{S}He^{-S}=H+[S,H]+\mathcal{O}(t^{2})$. Notice that $S$ is chosen in
such a way that the new terms commute with the unperturbed part of the
Hamiltonian $\sum_{i=1}^{L}\frac{\mu}{2}Z_{i}Z_{i+1}$ to the leading order in
the expansion, thus preserving its block-diagonal structure while also
including all virtual processes within each sector. The derived effective
Hamiltonian,

$H_{eff}=\sum_{i=1}^{L}\frac{\mu}{2}Z_{i}Z_{i+1}-hZ_{i}-\frac{t}{2}\left(X_{i}-Z_{i-1}X_{i}Z_{i+1}\right)$
(7)

turns out to be the well-known model discussed in detail in [13], where two
towers of QMBS have been identified. The latter are of the form

$|S_{n}^{k}\rangle=\frac{1}{n!\sqrt{\mathcal{N}(L,n)}}((\mathcal{Q}^{k})^{\dagger})^{n}|\Omega^{k}\rangle,$
(8)

with $k=1,2$, $|\Omega^{1}\rangle=|0\cdots 0\rangle$,
$|\Omega^{2}\rangle=|1\cdots 1\rangle$ and
$(\mathcal{Q}^{k})^{\dagger}=\sum_{i=i_{1}}^{i_{L}}(-1)^{i}P_{i-1}^{k}(X_{i}+(-1)^{k}Y_{i})P_{i+1}^{k}$,
with projection operators $P_{i}^{k}=(1+(-1)^{k}Z_{i})/2$. Such QBMS describe
$n$-magnon and $n$-antimagnons excitations for $k=1$ and $k=2$, respectively
[13].

Figure 2: The gap ratio $r$, (a) and half-chain entanglement
entropy($\mathcal{S}$) along path 0 for (b)– $S^{2}_{4}$ state form antimagnon
family; (c) – $S^{1}_{4}$ and (d) – $S^{1}_{6}$ from the magnon family, all
for $L=18$. States may be followed despite narrow avoided crossings indicated
by spikes of the entanglement entropy. The antimagnon state looses its low EE
feature around $t=0.14$ while magnon-like QMBS may be followed up to $t\approx
0.2$.

As already pointed out, the Schrieffer-Wolff transformation described above
links the models (5) and (1) with the effective spin model (7) only for
$t\ll\mu$. In this limit, the states given by Eq. (8) become true QMBS of
Hamiltonian (7), as it was studied in Ref. [13]. It appears natural to wonder
whether, following the states (8) in the parameter space, it is possible to
also find scarred states in the gauged Kitaev chain beyond the $t\ll\mu$
limit.

#### Level dynamics.–

In order to investigate this point, we first determine in which regime of
parameters the model (5) can be considered as ergodic, and thus where the
regular states may be called QMBS. Ergodicity may be revealed by the adjacent
mean gap ratio $r$ [65] between subsequent level spacings $\Delta_{i}$,

$r_{i}=\frac{min\\{\Delta_{i},\Delta_{i+1}\\}}{max\\{\Delta_{i},\Delta_{i+1}\\}}$
(9)

where $r\simeq 0.531$ corresponds to the fully ergodic regime (as described by
the Gaussian Orthogonal Ensemble (GOE) - [66]) and $r\simeq 0.386$ indicates
the quasi-integrable regime [65, 67]. As shown in Fig. 1, our model shows
strong indications of near-integrable behavior for $h\ll\mu$ or $t\ll\mu$.
Outside this region, the model is expected to be non-integrable. The next step
is then to identify the region where the entanglement entropy (EE) of some
eigenstates is well below the GOE estimate, as QMBS should have low, subvolume
entropy. The half-chain EE is defined in a standard way as
$\mathcal{S}=-Tr[\rho_{L/2}\ln{\rho_{L/2}}]$ [68]. We compare it to the
typical GOE value for a given system size. Once the two lowest EE states have
a relative EE sufficiently low (say half of the GOE value), there is the
chance that those states are indeed scarred. Fig. 1 shows by a light blue
color the domains where both $r>0.5$ and states with a sufficiently low EE
exist.

With the suspected regions identified, we start the analysis with
$|S_{n}^{k}\rangle$ states for the Hamiltonian (5) with parameters
$t,h\ll\mu$, e.g. for $t_{0}=h_{0}=0.001$ (we set $\mu=1$ in the following),
with the aim of tracking such initial states $|S^{k}_{n}(t_{0},h_{0})\rangle$
following their possible deformations induced by making small changes in the
parameters, $t=t_{0}+\delta t$ and $h=h_{0}+\delta h$. As examples we consider
paths indicated by ‘0’ and ‘I’ in Fig. 1.

Every time we update the parameters, we diagonalize the Hamiltonian in the
symmetry sector with momentum $p=0$ and parity $+1$ (we consider $n$ even
only) and find the new candidate $|E_{n}(t,h)\rangle$ by maximizing the
overlap $O=|\langle E_{n}(t,h)|S^{k}_{n})\rangle|$. For an isolated level, the
new state is accepted, $|S^{k}_{n}\rangle$ is updated, and we repeat the
procedure. Special care is taken to diabatically cross narrow avoided
crossings [68].

Figure 3: Panel (a) is like Fig. 2 but for path I and $L=18$. Along this path
only antimagnon excitations may be followed while magnons appear at the edges
of the energy spectrum only. (b) Time dependence of the fidelity relative to
an initial state $|\psi(0)\rangle$ and evolved with (5) at $h=0.5$ and
different values of $t$ for $L=16$ ($\mu=1$).

Figure 2 shows tracking the QMBS states identified as $|S^{k}_{n}(t,h)\rangle$
along path 0. Both magnon and antimagnon excitations, as given by (8), may be
followed. This path lies almost fully in the chaotic regime as revealed by $r$
statistics, so observed low EE states may be truly considered as QMBS,
observed, let us stress, deep in the deconfined region.

The path indicated as I also remains in the deconfined phase, extending to
large values of $h$ \- c.f. Fig. 1. Levels are followed through regular,
weakly perturbed region, along the straight line $t=0.2h$ up to $h=0.5$. Then
path I turns upwards, staying in the chaotic region up to the point
$(t,h)=(0.3,0.5)$. Along this vertical part, the tracked low EE states, shown
in Fig. 3(a), are truly the QMBS embedded in the chaotic spectrum. As seen
from the EE values, the scarred character of the followed states is slowly
lost due to, as verified, [68] various avoided crossings.

As discussed in Introduction, the presence of QMBS can be further revealed by
the persistent time oscillations of an out-of-equilibrium configuration. To
reveal this aspect, we prepare an initial state given by equal superposition
of two states of the form (8) for small $t,h$ values,
$|\psi(0)\rangle=\frac{1}{\sqrt{2}}(|S_{0}^{2}\rangle+|S_{2}^{2}\rangle)$.
After this initialization, we let this state evolve with Hamiltonian (5) and
we calculate the fidelity
$\mathcal{F}(\tau)=|\langle\psi(0)|\psi(\tau)\rangle|^{2}$, where $\tau$ is
the time (in units of inverse $\mu$). At $t=0.25$, we observfor other choices
of finitee persistent oscillations of $\mathcal{F}(\tau)$ as expected for QMBS
states, and this behavior persists until the end of path I - compare Fig.
3(b). Beyond this limit ($t>0.3$), low entanglement states disappear and
$\mathcal{F}(\tau)$ shows irregular oscillations around the mean value of
about $1/L^{2}$, as expected for thermal states.

Figure 4: The half-chain entanglement entropy($\mathcal{S}$) of all the
eigenstates at $t=0.2$, $h=0.5$ for $L=16$. The orange dashed line gives the
$\mathcal{S}_{RMT}$ value. Circles denote different QMBS obtained via our
tracking procedure. Green circles denote antimagnon-like family $S^{2}_{n}$
for $n=0,2,4,6,8$ while red circles magnon-like states, $S^{1}_{n}$ with
$n=0,..,6$ counting from the right hand side. Inset: The half-chain
Entanglement Entropy divided by system size ($\frac{\mathcal{S}}{L}$) for
$S^{2}_{2}$ state showing its sub-volume property as expected for QMBS.

Figure 4 visualizes the presence of QMBS by showing the value of $\mathcal{S}$
for all eigenstates at $t=0.2$ and $h=0.5$, where the system is in the weak
ergodic regime. The entropy still reveals a finger-like structure that
indicates the existence of hidden, unidentified symmetries. The states
enclosed in circles are those tracked up from the near-integrable limit. The
members of the antimagnon-like family $S^{2}_{k}$, denoted by green circles,
have very low entanglement entropies as compared to other states of similar
energy. They are thus truly QMBS. The inset reveals a sub-volume scaling of
the entanglement entropy of the $S^{2}_{2}$ state. The magnon excitations, on
the other hand, dissolve among other states (in the high density of states
region).

In summary, motivated by recent predictions of finding QMBS states in a
confined regime [69, 59] of LGTs, we investigated the thermalization
properties of the gauged Kitaev chain. We observe that deep in the confined
phase even states with the smallest entanglement entropy have relatively
large, volume-law values indicating a lack of QMBS. On the other hand, for
relatively large values of the electric field, some states reveal low
entanglement entropy. We can identify these states by following them from the
very small $t,h$ values for which analytic predictions are available [13].
Such states result to be true QMBS and are found uniquely in the deconfined
phase, which turns out to be a weakly ergodic regime characterized by the GOE-
like mean gap ratio. The presence of QMBS has been further verified by
studying their time dynamics. By building an initial state given by the
superposition of two QMBS, the fidelity of the state shows pronounced
oscillations with no sign of thermalization. The adiabatic following of the
states breaks down at the end points of our chosen paths due to energy-level
mixing, where interestingly QMBS also disappear. In conclusion, our results
unambiguously reveal that QMBS occur even in deconfined regimes of LGTs, thus
paving the way toward a deeper understanding of the connection between lack of
thermalization and local symmetries.

###### Acknowledgements.

We thank A. Bohrdt, U. Borla, F. Grusdt, J.C. Halimeh, P. Hauke. M. Kebric, S.
Moroz and P. Sierant for discussions on this manuscript and related projects.
The numerical computations have been possible thanks to PL-Grid
Infrastructure. The works of A.S.A. and J.Z have been realized within the Opus
grant 2019/35/B/ST2/00034, financed by National Science Centre (Poland). M.L.
acknowledges support from ERC AdG NOQIA, State Research Agency AEI (“Severo
Ochoa” Center of Excellence CEX2019-000910-S, Plan National FIDEUA
PID2019-106901GB-I00/10.13039 / 501100011033, FPI, QUANTERA MAQS
PCI2019-111828-2 / 10.13039/501100011033), Fundacio Privada Cellex, Fundacio
Mir-Puig, Generalitat de Catalunya (AGAUR Grant No. 2017 SGR 1341, CERCA
program, QuantumCAT U16-011424, co-funded by ERDF Operational Program of
Catalonia 2014-2020), EU Horizon 2020 FET-OPEN OPTOLogic (Grant No. 899794),
and the National Science Centre, Poland (Symfonia Grant No.
2016/20/W/ST4/00314), Marie Skłodowska-Curie grant STRETCH No. 101029393, La
Caixa Junior Leaders fellowships (ID100010434), and EU Horizon 2020 under
Marie Skłodowska-Curie grant agreement No 847648 (LCF/BQ/PI19/11690013,
LCF/BQ/PI20/11760031, LCF/BQ/PR20/11770012). D.G.-C. is supported by the
Simons Collaboration on Ultra-Quantum Matter, which is a grant from the Simons
Foundation (651440, P.Z.).

## References

  * Rigol  _et al._ [2008] M. Rigol, V. Dunjko, and M. Olshanii, Thermalization and its mechanism for generic isolated quantum systems, Nature 452, 854 EP (2008).
  * Polkovnikov  _et al._ [2011] A. Polkovnikov, K. Sengupta, A. Silva, and M. Vengalattore, Colloquium: Nonequilibrium dynamics of closed interacting quantum systems, Rev. Mod. Phys. 83, 10.1103/RevModPhys.83.863 (2011).
  * Gogolin and Eisert [2016] C. Gogolin and J. Eisert, Equilibration, thermalization, and the emergence of statistical mechanics in closed quantum systems, Rep. Prog. Phys 79, 056001 (2016).
  * Vidmar and Rigol [2016] L. Vidmar and M. Rigol, Generalized Gibbs ensemble in integrable lattice models, Journal of Statistical Mechanics: Theory and Experiment 2016, 064007 (2016).
  * Bernien  _et al._ [2017] H. Bernien, S. Schwartz, A. Keesling, H. Levine, A. Omran, H. Pichler, S. Choi, A. S. Zibrov, M. Endres, M. Greiner, V. Vuletić, and M. D. Lukin, Probing many-body dynamics on a 51-atom quantum simulator, Nature 551, 579 (2017).
  * Shiraishi and Mori [2017] N. Shiraishi and T. Mori, Systematic construction of counterexamples to the eigenstate thermalization hypothesis, Phys. Rev. Lett. 119, 030601 (2017).
  * Turner  _et al._ [2018a] C. J. Turner, A. A. Michailidis, D. A. Abanin, M. Serbyn, and Z. Papić, Weak ergodicity breaking from quantum many-body scars, Nature Physics 14, 745 (2018a).
  * Turner  _et al._ [2018b] C. Turner, A. Michailidis, D. Abanin, M. Serbyn, and Z. Papić, Quantum scarred eigenstates in a Rydberg atom chain: Entanglement, breakdown of thermalization, and stability to perturbations, Physical Review B 98, 155134 (2018b).
  * Moudgalya  _et al._ [2018a] S. Moudgalya, S. Rachel, B. A. Bernevig, and N. Regnault, Exact excited states of nonintegrable models, Physical Review B 98, 235155 (2018a).
  * Moudgalya  _et al._ [2018b] S. Moudgalya, N. Regnault, and B. A. Bernevig, Entanglement of exact excited states of Affleck-Kennedy-Lieb-Tasaki models: Exact results, many-body scars, and violation of the strong eigenstate thermalization hypothesis, Phys. Rev. B 98, 235156 (2018b).
  * Iadecola and Žnidarič [2019] T. Iadecola and M. Žnidarič, Exact localized and ballistic eigenstates in disordered chaotic spin ladders and the Fermi-Hubbard model, Phys. Rev. Lett. 123, 036403 (2019).
  * Schecter and Iadecola [2019] M. Schecter and T. Iadecola, Weak ergodicity breaking and quantum many-body scars in spin-1 $XY$ magnets, Phys. Rev. Lett. 123, 147201 (2019).
  * Iadecola and Schecter [2020] T. Iadecola and M. Schecter, Quantum many-body scar states with emergent kinetic constraints and finite-entanglement revivals, Physical Review B 101, 024306 (2020).
  * Serbyn  _et al._ [2021] M. Serbyn, D. A. Abanin, and Z. Papić, Quantum many-body scars and weak breaking of ergodicity, Nature Physics 17, 675 (2021).
  * Lee  _et al._ [2020] K. Lee, R. Melendrez, A. Pal, and H. J. Changlani, Exact three-colored quantum scars from geometric frustration, Phys. Rev. B 101, 241111 (2020).
  * Surace  _et al._ [2020] F. M. Surace, P. P. Mazza, G. Giudici, A. Lerose, A. Gambassi, and M. Dalmonte, Lattice Gauge Theories and String Dynamics in Rydberg Atom Quantum Simulators, Phys. Rev. X 10, 021041 (2020).
  * Magnifico  _et al._ [2020] G. Magnifico, M. Dalmonte, P. Facchi, S. Pascazio, F. V. Pepe, and E. Ercolessi, Real Time Dynamics and Confinement in the $\mathbb{Z}_{n}$ Schwinger-Weyl lattice model for 1+1 QED, Quantum 4, 281 (2020).
  * Chanda  _et al._ [2020] T. Chanda, J. Zakrzewski, M. Lewenstein, and L. Tagliacozzo, Confinement and lack of thermalization after quenches in the bosonic Schwinger model, Phys. Rev. Lett. 124, 180602 (2020).
  * Zhao  _et al._ [2020] H. Zhao, J. Vovrosh, F. Mintert, and J. Knolle, Quantum many-body scars in optical lattices, Phys. Rev. Lett. 124, 160604 (2020).
  * Papić [2021] Z. Papić, Weak ergodicity breaking through the lens of quantum entanglement (2021), arXiv:2108.03460 [cond-mat.quant-gas] .
  * Moudgalya  _et al._ [2021] S. Moudgalya, A. Prem, R. Nandkishore, N. Regnault, and B. A. Bernevig, Thermalization and its absence within Krylov subspaces of a constrained hamiltonian, Memorial Volume for Shoucheng Zhang , 147–209 (2021).
  * Surace  _et al._ [2021] F. M. Surace, M. Dalmonte, and A. Silva, Quantum local random networks and the statistical robustness of quantum scars (2021), arXiv:2107.00884 [cond-mat.stat-mech] .
  * Zhao  _et al._ [2021] H. Zhao, A. Smith, F. Mintert, and J. Knolle, Orthogonal quantum many-body scars, Phys. Rev. Lett. 127, 150601 (2021).
  * Szołdra  _et al._ [2022] T. Szołdra, P. Sierant, M. Lewenstein, and J. Zakrzewski, Unsupervised detection of decoupled subspaces: many-body scars and beyond (2022), arXiv:2201.07151 [quant-ph] .
  * Bluvstein  _et al._ [2021] D. Bluvstein, A. Omran, H. Levine, A. Keesling, G. Semeghini, S. Ebadi, T. T. Wang, A. A. Michailidis, N. Maskara, W. W. Ho, S. Choi, M. Serbyn, M. Greiner, V. Vuletić, and M. D. Lukin, Controlling quantum many-body dynamics in driven Rydberg atom arrays, Science 371, 1355 (2021), https://www.science.org/doi/pdf/10.1126/science.abg2530 .
  * Turner  _et al._ [2021] C. J. Turner, J.-Y. Desaules, K. Bull, and Z. Papić, Correspondence principle for many-body scars in ultracold rydberg atoms, Phys. Rev. X 11, 021021 (2021).
  * Heller [1984] E. J. Heller, Bound-state eigenfunctions of classically chaotic hamiltonian systems: Scars of periodic orbits, Phys. Rev. Lett. 53, 1515 (1984).
  * Gutzwiller [1971] M. C. Gutzwiller, Periodic orbits and classical quantization conditions, Journal of Mathematical Physics 12, 343 (1971).
  * Bogomolny [1988] E. Bogomolny, Smoothed wavefunctions of chaotic quantum systems, Physica D 31, 169 (1988).
  * Delande and Gay [1987] D. Delande and J. C. Gay, Scars of symmetries in quantum chaos, Phys. Rev. Lett. 59, 1809 (1987).
  * Jaksch and Zoller [2005] D. Jaksch and P. Zoller, The cold atom Hubbard toolbox, Annals of Physics 315, 52 (2005).
  * Lewenstein  _et al._ [2007] M. Lewenstein, A. Sanpera, V. Ahufinger, B. Damski, A. Sen(De), and U. Sen, Ultracold atomic gases in optical lattices: mimicking condensed matter physics and beyond, Advances in Physics 56, 243 (2007), https://doi.org/10.1080/00018730701223200 .
  * Gross and Bloch [2017] C. Gross and I. Bloch, Quantum simulations with ultracold atoms in optical lattices, Science 357, 995 (2017).
  * Schäfer  _et al._ [2020] F. Schäfer, T. Fukuhara, S. Sugawa, Y. Takasu, and Y. Takahashi, Tools for quantum simulation with ultracold atoms in optical lattices, Nature Reviews Physics 2, 411 (2020).
  * Browaeys and Lahaye [2020] A. Browaeys and T. Lahaye, Many-body physics with individually controlled Rydberg atoms, Nature Physics 16, 132 (2020).
  * Su  _et al._ [2022] G.-X. Su, H. Sun, A. Hudomal, J.-Y. Desaules, Z.-Y. Zhou, B. Yang, J. C. Halimeh, Z.-S. Yuan, Z. Papić, and J.-W. Pan, Observation of unconventional many-body scarring in a quantum simulator (2022), arXiv:2201.00821 [cond-mat.quant-gas] .
  * Martinez  _et al._ [2016] E. Martinez, C. Muschik, P. Schindler, and et al., Real-time dynamics of lattice gauge theories with a few-qubit quantum computer, Nature 534, 516–519 (2016).
  * Schweizer  _et al._ [2019] C. Schweizer, F. Grusdt, M. Berngruber, L. Barbiero, E. Demler, N. Goldman, I. Bloch, and M. Aidelsburger, Floquet approach to $z_{2}$ lattice gauge theories with ultracold atoms in optical lattices, Nat. Phys. 15, 1168–1173 (2019).
  * F.Goörg  _et al._ [2019] F. F.Goörg, K. Sandholzer, J. Minguzzi, R. Desbuquois, M. Messer, and T. Esslinger, Realization of density-dependent peierls phases to engineer quantized gauge fields coupled to ultracold matter, Nat. Phys. 15, 1161–1167 (2019).
  * Mil and et al [2020] A. Mil and et al, A scalable realization of local u(1) gauge invariance in cold atomic mixtures, Science 367, 1128 (2020).
  * Yang  _et al._ [2020a] Z.-C. Yang, F. Liu, A. V. Gorshkov, and T. Iadecola, Hilbert-space fragmentation from strict confinement, Physical review letters 124, 207602 (2020a).
  * Yang  _et al._ [2020b] B. Yang, H. Sun, R. Ott, H. Y. Wang, T. V. Zache, J. C. Halimeh, Z. S. Yuan, P. Hauke, and J. W. Pan, Observation of gauge invariance in a 71-site Bose-Hubbard quantum simulator, Nature , 392 (2020b).
  * Zohar and Reznik [2011] E. Zohar and B. Reznik, Confinement and lattice quantum-electrodynamic electric flux tubes simulated with ultracold atoms, Phys. Rev. Lett. 107, 275301 (2011).
  * Dalmonte and Montangero [2016] M. Dalmonte and S. Montangero, Lattice gauge theory simulations in the quantum information era, Contemporary Physics 57, 388 (2016).
  * Assaad and Grover [2016] F. F. Assaad and T. Grover, Simple fermionic model of deconfined phases and phase transitions, Phys. Rev. X 6, 041049 (2016).
  * González-Cuadra  _et al._ [2017] D. González-Cuadra, E. Zohar, and J. I. Cirac, Quantum simulation of the Abelian-Higgs lattice gauge theory with ultracold atoms, New Journal of Physics 19, 063038 (2017).
  * Gazit  _et al._ [2017] S. Gazit, M. Randeria, and A. Vishwanath, Emergent dirac fermions and broken symmetries in confined and deconfined phases of $z_{2}$ gauge theories, Nat. Phys. 13, 484–490 (2017).
  * Gazit  _et al._ [2018] S. Gazit, F. F. Assaad, S. Sachdev, A. Vishwanath, and C. Wang, Confinement transition of $z_{2}$ gauge theories coupled to massless fermions: Emergent quantum chromodynamics and SO(5) symmetry, Proceedings of the National Academy of Sciences 115, E6987 (2018), https://www.pnas.org/content/115/30/E6987.full.pdf .
  * Barbiero  _et al._ [2019] L. Barbiero, C. Schweizer, M. Aidelsburger, E. Demler, N. Goldman, and F. Grusdt, Coupling ultracold matter to dynamical gauge fields in optical lattices: From flux attachment to $z_{2}$ lattice gauge theories, Sci. Adv. 5, 10.1126/sciadv.aav7444 (2019).
  * Borla  _et al._ [2020] U. Borla, R. Verresen, F. Grusdt, and S. Moroz, Confined phases of one-dimensional spinless fermions coupled to $z_{2}$ gauge theory, Phys. Rev. Lett 124, 120503 (2020).
  * González-Cuadra  _et al._ [2020] D. González-Cuadra, L. Tagliacozzo, M. Lewenstein, and A. Bermudez, Robust topological order in fermionic $z_{2}$ gauge theories: From Aharonov-Bohm instability to soliton-induced deconfinement, Phys. Rev. X 10, 041007 (2020).
  * Bañuls  _et al._ [2020] M. C. Bañuls, R. Blatt, J. Catani, A. Celi, J. I. Cirac, M. Dalmonte, L. Fallani, K. Jansen, M. Lewenstein, S. Montangero, C. A. Muschik, B. Reznik, E. Rico, L. Tagliacozzo, K. Van Acoleyen, F. Verstraete, U.-J. Wiese, M. Wingate, J. Zakrzewski, and P. Zoller, Simulating lattice gauge theories within quantum technologies, The European Physical Journal D 74, 165 (2020).
  * Halimeh and Hauke [2020] J. C. Halimeh and P. Hauke, Reliability of lattice gauge theories, Phys. Rev. Lett. 125, 030503 (2020).
  * Halimeh  _et al._ [2021] J. C. Halimeh, L. Homeier, C. Schweizer, M. Aidelsburger, P. Hauke, and F. Grusdt, Stabilizing lattice gauge theories through simplified local pseudo generators (2021), arXiv:2108.02203 [cond-mat.quant-gas] .
  * Damme  _et al._ [2021] M. V. Damme, J. Mildenberger, F. Grusdt, P. Hauke, and J. C. Halimeh, Suppressing nonperturbative gauge errors in the thermodynamic limit using local pseudogenerators (2021), arXiv:2110.08041 [quant-ph] .
  * Kebrič  _et al._ [2021] M. Kebrič, L. Barbiero, C. Reinmoser, U. Schollwöck, and F. Grusdt, Confinement and mott transitions of dynamical charges in one-dimensional lattice gauge theories, Phys. Rev. Lett. 127, 167203 (2021).
  * Aidelsburger  _et al._ [2022] M. Aidelsburger, L. Barbiero, A. Bermudez, T. Chanda, A. Dauphin, D. González-Cuadra, P. R. Grzybowski, S. Hands, F. Jendrzejewski, J. Jünemann, G. Juzeliūnas, V. Kasper, A. Piga, S.-J. Ran, M. Rizzi, G. Sierra, L. Tagliacozzo, E. Tirrito, T. V. Zache, J. Zakrzewski, E. Zohar, and M. Lewenstein, Cold atoms meet lattice gauge theory, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 380, 20210064 (2022).
  * Banerjee and Sen [2021] D. Banerjee and A. Sen, Quantum scars from zero modes in an abelian lattice gauge theory on ladders, Physical Review Letters 126, 220601 (2021).
  * Chen and Iadecola [2021] I.-C. Chen and T. Iadecola, Emergent symmetries and slow quantum dynamics in a Rydberg-atom chain with confinement, Physical Review B 103, 214304 (2021).
  * Lerose  _et al._ [2020] A. Lerose, F. M. Surace, P. P. Mazza, G. Perfetto, M. Collura, and G. A., Quasilocalized dynamics from confinement of quantum excitations, Phys. Rev. B 102, 041118 (2020).
  * Borla  _et al._ [2021] U. Borla, R. Verresen, J. Shah, and S. Moroz, Gauging the kitaev chain, SciPost Physics 10, 148 (2021).
  * Kogut [1979] J. B. Kogut, An introduction to lattice gauge theory and spin systems, Rev. Mod. Phys. 51, 659 (1979).
  * Ovchinnikov  _et al._ [2003] A. Ovchinnikov, D. Dmitriev, V. Y. Krivnov, and V. Cheranovskii, Antiferromagnetic ising chain in a mixed transverse and longitudinal magnetic field, Physical review B 68, 214406 (2003).
  * Bastianello  _et al._ [2021] A. Bastianello, U. Borla, and S. Moroz, Fragmentation and emergent integrable transport in the weakly tilted Ising chain (2021), arXiv:2108.04845 [cond-mat.str-el] .
  * Oganesyan and Huse [2007] V. Oganesyan and D. A. Huse, Localization of interacting fermions at high temperature, Phys. Rev. B 75, 155111 (2007).
  * Haake [2010] F. Haake,  _Quantum Signatures of Chaos_ (Springer, Berlin, 2010).
  * Atas  _et al._ [2013] Y. Y. Atas, E. Bogomolny, O. Giraud, and G. Roux, Distribution of the ratio of consecutive level spacings in random matrix ensembles, Phys. Rev. Lett. 110, 084101 (2013).
  * [68] See Supplemental Material at [URL will be inserted by publisher] for details on entanglement entropy, the state following analysis and the fate of magnons. It includes Refs. [70, 71, 72, 73, 74, 75, 27, 30].
  * James  _et al._ [2019] A. J. A. James, R. M. Konik, and N. J. Robinson, Nonthermal states arising from confinement in one and two dimensions, Phys. Rev. Lett. 122, 130603 (2019).
  * Vidmar and Rigol [2017] L. Vidmar and M. Rigol, Entanglement entropy of eigenstates of quantum chaotic hamiltonians, Phys. Rev. Lett. 119, 220603 (2017).
  * Huang [2019] Y. Huang, Universal eigenstate entanglement of chaotic local hamiltonians, Nuclear Physics B 938, 594 (2019).
  * Huang [2021] Y. Huang, Universal entanglement of mid-spectrum eigenstates of chaotic local hamiltonians, Nuclear Physics B 966, 115373 (2021).
  * Stöckmann [1999] H. Stöckmann,  _Quantum Chaos: An Introduction_ (Cambridge University Press, Cambridge, 1999).
  * Zakrzewski and Kuś [1991] J. Zakrzewski and M. Kuś, Distributions of avoided crossings for quantum chaotic systems, Phys. Rev. Lett. 67, 2749 (1991).
  * Zakrzewski  _et al._ [1993] J. Zakrzewski, D. Delande, and M. Kuś, Parametric motion of energy levels in quantum chaotic systems. ii. avoided-crossing distributions, Phys. Rev. E 47, 1665 (1993).

##  I Supplementary material for Scar States in Deconfined Lattice Gauge
Theories

###  I.1 Entanglement Entropy

We use the half-chain entanglement entropy, $\mathcal{S}$, as a useful measure
of eigenstates properties. The half-chain entanglement entropy is defined as
the von Neumann entropy of the reduced density matrix $\rho_{L/2}$

$\mathcal{S}=-Tr[\rho_{L/2}\ln{\rho_{L/2}}].$ (S.1)

where $\rho_{L/2}=Tr_{1,\cdots,L/2}|\psi\rangle\langle\psi|$ is obtained by
tracing out half of the system for eigenstate $|\psi\rangle$. The typical
state in the ergodic regime obeys RMT prediction given by
$\mathcal{S}_{RMT}=(L/2)\ln(2)+(1/2+\ln(1/2))/2-1/2$ [70, 71, 72]. A
substantial difference from $\mathcal{S}_{RMT}$ for states in the middle of
the spectrum can be taken as a signature of nonergodic states, although to
really observe a sub-volume property of the entanglement entropy for a given
state one has to study the entropy dependence on the system size (see the
inset in Fig. 4 in the main text.

Figure S1:  The energy levels along the path “0”’. The regularly spaced
multiply degenerate levels at $h=t=0$ split when parameters increase along
$t=3h$ line. The dashed line as well as two dash-dotted lines denote the
position of antimagnon and two magnon excitations whose entanglement entropy
is presented in Fig. 2. As seen all three states lie in the middle, high
density of states, regime of the spectrum. Figure S2:  The energy levels along
the path-I. The regularly spaced levels split following the line $t=0.2h$ up
to $t=0.1$. From there $h$ remains constant, $h=0.5$ while $t$ increases up to
$t\approx 0.3$ which is the point where all the high density states have high
entanglement entropy($\mathcal{S}$).

###  I.2 Breaking of Adiabaticity on path ‘I’ exanple

The initial state $|S_{n}^{k}(t_{0},h_{0})\rangle$ is identified from the
mapping between the Hamiltonian of quantum Ising chain in both transverse and
longitudinal fields (5) and the effective Hamiltonian (7) at
$t_{0},h_{0}=0.001$. The Hamiltonian, $H$, (5) is then diagonalized with the
smaeq:kitll parameter change of $\delta h=0.001$ and $\delta t=0.0002$ and a
state $|E_{n}(t,h)\rangle$ is identified as the state which has a maximum
overlap with $|S_{n}^{k}(t_{0},h_{0})\rangle$ among all the eigenstates of
${H}(t_{0}+\delta t,h_{0}+\delta t)$.

Figure S3: The overlaps along the path I of three antimagnon states. The sharp
overlap drops at the beginning of the path, in almost integrable regions is
due to multiple avoided crossings indicated by vertical dashed lines in the
level diagram (top, left)

Figure S3 shows the exemplary overlaps along path I.

To adiabatically follow the suspected QMBS state $|S_{n}^{k}(t,h)\rangle$ is
updated along the paths. But this is only done so, if for the next 10
diagonalizations (for a parameter change of $\Delta h=0.01$ and $\Delta
t=0.002$) the eigenindex of the state $|E_{n}(t+\Delta t,h+\Delta h)\rangle$
does not change from its value at $|E_{n}(t,h)\rangle$. This procedure allows
us to overcome problems encountered while passing the avoided crossings (see
also below). The following protocol is continued until the final value of $h$
along a given path is reached. Later we continue updating $t$ only with
$\delta t=0.001$ while keeping $h$ constant.

Figure S4: An enlarged image of Fig. S2 – top – restricted to the parameters
where the peaks in entanglement entropy of the followed state and in $r$ value
for the spectrum appear - compare Fig. 1(c). Note that the range of $h$
corresponds via $t=0.2h$ relation defining path I to $t\in[0,0.1]$. The orange
dashed lines represents the location of $t$ values for the peaks. The bottom
plot shows the integrable system states energies obtained by putting $t=0$ in
the Hamiltonian, (S.2).  Figure S5:  Attempt to follow the magnon type state
$S^{1}_{6}$ along path I. Once gap ratio reaches values close to GOE limit the
entropy of the followed state rapidly and significantly increases indicating
that the scarred character is lost.

This procedure runs into difficulties when avoided crossings are met. In the
regime corresponding to mixed regular and chaotic dynamics (as revealed
statistically by the gap ratio, $r$, being between Poissonian and GOE values)
the avoided crossings are isolated. Such avoided crossings may be crossed
diabatically with care. Once we reach the irregular regime multiple avoided
crossings typically arise and only those states can be traced that are almost
decoupled in the process of changing the parameters. Such a situation is well
known from chaotic, single particle, strongly scarred systems such as quantum
billiards or hydrogen atom in a strong magnetic field (see e.g. [73]). While
in generalized time reversal invariant systems small avoided crossings are
highly probable [74, 75] overlapping avoided crossings dissolve the scar
character of the state unless a periodic orbit [27] or symmetry [30]
reinforces this character after the crossing. While this intuition is based on
a single particle systems, we expect the similar qualitative behavior for the
possible preservation of QMBS.

Figure S1 presents the level dynamics along the path “0”, i.e $t=3h$ line.
eq:kit Fig. S2 shows the level dynamics for PATH I. At $t\in[0.0,0.1]$ the $r$
value is nearly Poissonian and all the identified QMBS states can be tracked.
In fact, a simple structure of the tilted transverse Ising Hamiltonian (5)
reproduced here for clarity

$H=\sum_{i=1}^{L}\frac{\mu}{2}Z_{i}Z_{i+1}-tX_{i}-hZ_{i}.$ (S.2)

identifies that for $t,h\ll 1$ the states are ordered by number of dimers as
given by the operator $D=\sum_{i=1}^{L}Z_{i}Z_{i+1}$. Once $h>0$ those states
form a manifold split by $\sum Z_{i}$ operator eigenvalues. Only at the
crossings of the manifolds the term $t\sum_{i}X_{i}$ mixes eigenstates of
$Z_{i}$ leading to nontrivial avoided crossings, compare Fig. S4. It is at
these avoided crossings that sharp spikes visible in the entropy $t$ scans
along the path I appear. Here also the mean gap ratio shows the unusual peaks.

For larger $t\in[0.1,0.3]$ the $r$ value is GOE-like indicating an ergodic
character, in this region only the QMBS states built out of antimagnons are
sustained indicating these states to be characteristically different from the
other states. At $t>0.3$ the levels become strongly mixed and the Hilbert
space splitting into characteristic bands disappears. Then all the tracked
states loose their QMBS characteristics.

###  I.3 Magnon States

The scar states built out of magnon states which were located at the middle of
the spectrum (in the region of a high density of states) tracked along path I
were lost on entering the ergodic regime. This is in contrary to path “0” when
both magnon and antimagnon excitations could be followed. For the path I, only
antimagnon states may be followed and linked to QMBS in the ergodic regime.
The example of our attempt to track a magnon state along path I is summarized
in Fig. S5.


# Measuring the Impact of COVID-19 Induced Campus Closure on Student Self-
Regulated Learning in Physics Online Learning Modules

Tom Zhang<EMAIL_ADDRESS>University of Central Florida4111 Libra
Dr.OrlandoFloridaUSA32816 ,  Michelle Taub<EMAIL_ADDRESS>University
of Central Florida4111 Libra Dr.OrlandoFloridaUSA32816 and  Zhongzhou Chen
<EMAIL_ADDRESS>University of Central Florida4111 Libra
Dr.OrlandoFloridaUSA32816

(2020)

###### Abstract.

This paper examines the impact of COVID-19 induced campus closure on
university students’ self-regulated learning behavior by analyzing click-
stream data collected from student interactions with 70 online learning
modules in a university physics course. To do so, we compared the trend of six
types of actions related to the three phases of self-regulated learning before
and after campus closure and between two semesters. We found that campus
closure changed students’ planning and goal setting strategies for completing
the assignments, but didn’t have a detectable impact on the outcome or the
time of completion, nor did it change students’ self-reflection behavior. The
results suggest that most students still manage to complete assignments on
time during the pandemic, while the design of online learning modules might
have provided the flexibility and support for them to do so.

self-regulated learning, online learning environments, click-stream data

††copyright: acmcopyright††journalyear: 2020††conference: Learning Analytics
and Knowledge ’21; April 12–16, 2021; ††booktitle: Learning Analytics and
Knowledge ’21, April 12–16, 2021††ccs: Applied computing Distance
learning††ccs: Applied computing E-learning††ccs: Applied computing Physics

##  1\. Introduction

In March of 2020, the majority of higher education institutions across the
United States were forced to abruptly close campuses and shift to distance
learning for the remainder of the Spring 2020 semester due to the COVID-19
pandemic. As a result, students were suddenly faced with the unusually
challenging task of self-regulating their learning activities at home, amidst
the disruptions to life brought on by the pandemic. There is widespread
concern amongst instructors and administrators regarding the potential
negative impact on student learning (Dorn et al., 2020; Wilcox and Vignal,
2020), but at the time this manuscript was written, there is little in the way
of published literature which quantitatively measures the magnitude or nature
of this impact (Gonzalez et al., 2020).

As a result of campus closure, click-stream data from online learning systems
has become one of the most reliable sources of data providing information on
learning activities and learning outcomes. Several recent studies have
analyzed click-stream data to investigate students’ self-regulated learning
(SRL) processes (Taub et al., 2018; Maldonado-Mahauad et al., 2018; Li et al.,
2020). This current paper introduces our attempt at measuring the impact of
COVID-19 induced campus closure on multiple aspects of students’ SRL
processes, by analyzing click-stream data collected from students enrolled in
a university introductory physics class interacting with 70 mastery-based
online learning modules (OLMs) as part of the course assignment throughout the
Spring 2020 semester.

We will base our data analysis and results interpretation efforts on the
theoretical framework of SRL, which models students’ SRL processes in three
cyclical phases. In the remainder of this section, we will first briefly
introduce the SRL framework, present predictions of the impact of campus
closure, explain the design of OLMs and OLM sequences, and establish
connections between click-stream data from the OLMs and student actions during
all three phases of SRL.

###  1.1. Frameworks of Self-Regulated Learning

According to theories of SRL (Zimmerman, 2013), a student who is self-
regulating is playing an active role in their learning as opposed to being a
passive recipient of information. According to Zimmerman’s Social Cognitive
Theory (Zimmerman, 2015), SRL is accomplished by engaging in three cyclical
phase during learning: Forethought, Performance, and Self-Reflection. During
each of these phases, students use different strategies to monitor and control
their learning. The Forethought Phase consists of planning and goal setting,
where the student maps out their goals for completing a task and how they are
going to achieve them. These decisions are often impacted by students’
motivations (e.g., achievement goals). In the Performance Phase, students
engage in cognitive learning strategies (e.g., reading content, taking notes)
and metacognitive monitoring processes (e.g., time management) to complete
tasks. Students are thus enacting their plans and self-monitoring their
progress towards those goals. In the Self-Reflection Phase, students evaluate
their progress and understanding of the material being studied and assess the
factors contributing to their performance (e.g., self-testing). Based on these
reflections, students can decide to adapt their behaviors for completing the
current or starting subsequent tasks.

These phases are interdependent and thus they do not need to occur in a
sequential order, nor do they occur only once during a task. For example, if
an online learning module allows multiple attempts at an assessment, a student
may choose to adapt how they engage with the content prior to subsequent
attempts if self-reflection deemed their initial strategy ineffective. This
implies that a student must be aware of their own cognition and performance to
self-regulate efficiently.

COVID-19 induced campus closure can potentially have multiple negative impacts
on a student’s SRL processes by both providing fewer opportunities and placing
a higher demand for different types of cognitive, metacognitive, and adaptive
processes. During the Forethought Phase, a student needs to consider a variety
of extraneous factors such as computer access in a family home when planning
their study. For the Performance Phase, students face a higher barrier for
help seeking (Aleven et al., 2016), while having to more actively monitor the
amount of time they spend on each lesson compared to dedicated class hours. In
terms of self-reflection, students face lower accessibility for external
support such as exchanging notes with classmates or asking questions after
class but are still required to evaluate their progress and make adjustments.

###  1.2. Measuring SRL from Interactions with Online Mastery-Based Learning
Modules (OLMs)

####  1.2.1. Design of OLMs and the OLM sequences

Each OLM is focused on explaining one or two basic concepts, or developing the
skills to solve one kind of problem, designed to be completed between 5 to 30
minutes. The OLM consists of an assessment component (AC) which tests
students’ mastery of the module topic in 1-2 questions, and an instructional
component (IC) with instructional text and practice problems on the topic (see
Figure 1). Upon accessing a module, students are shown the learning objectives
of the current module and asked to make an initial attempt on the AC before
being allowed to access the IC. Students can make additional attempts on the
AC at any time after the first attempt and are not required to access the IC.
This design is motivated in part by the ”mastery-learning” format (Gutmann et
al., [n.d.]; Bloom, 1968) that allow students who are already familiar with
the content to proceed, and by the concept of ”preparation for future
learning” (Schwartz and Bransford, 2005) intending to improve students’
learning from the IC by exposing them to the questions first.

Figure 1. Schematic illustration of OLM design, adapted from (Chen et al.,
2020).

Design diagram of the Online Mastery-Based Learning Modules.

A number of OLM modules form an OLM sequence about a more general topic (e.g.,
conservation of mechanical energy) and students are required to pass the AC or
use up all attempts before moving onto the next OLM in that sequence. A
typical OLM sequence consists of 5-12 modules and are assigned as self-study
homework for students to complete over a period of one to two weeks. In Fall
2019, a total of 44 OLMs were assigned as homework for 7 out of the 10 topics
in a calculus-based introductory physics course, while in Spring 2020, 9 out
of the 10 topics used a total of 70 OLMs as online homework for the same
course. In both semesters, students could earn extra credit by completing some
of the OLMs 2-6 days prior to the due date.

####  1.2.2. Relating Click-Stream Data to SRL Behavior and Actions

Based on the design of the OLMs and OLM sequences, we identify six types of
student actions that can be detected or inferred from one or more patterns in
click-stream data. These actions are related to or indicative of students’
behavior during each of the three phases of SRL, summarized in Table 1.

Table 1. Relating Data to SRL. Phase |  Behavior |  Action/Decision |  Data Signal |  Data Level
---|---|---|---|---
Forethought |  Planning |  Skipping, skimming through, or engaging with the initial attempt |  Fraction of 1st attempts less than 15s, between 15 and 35s, and longer than 35s. |  Module
|  Goal Setting |  Passing or finishing the OLM with minimal effort. |  Fraction of students who adopt a Late Study or No Study strategy. |  Module
|  |  Completing the modules early or close to the sequence due date. |  Fraction of OLMs completed at least 1 or 3 days prior to the due date. |  Sequence
Performance |  Learning |  Passing the assessment after studying the module, or passing on a Brief Attempt. |  Fraction of students passing after accessing the IC and fraction of Brief Passing Attempts (¡35s). |  Module
Self-Reflection |  Reviewing |  Revisiting an upstream module while working within an OLM sequence. |  The average number of revisiting events per OLM sequence |  Sequence
|  |  Revisiting a completed OLM before a midterm exam |  The number of revisiting events within 3 days of a midterm exam. |  Sequence

Regarding the Forethought Phase, data from the OLMs can provide information on
two types of behavior: planning and goal setting. The mandatory first AC
attempt on each OLM requires students to plan on whether to engage with the
problems or randomly submit a response without reading and proceeding to the
IC. Previous studies (Chen et al., 2020; Guthrie et al., 2020) suggest that
attempts submitted under 15 seconds are likely generated by students who
skipped reading the problems in the AC, whereas attempts between 15 and 35
seconds are likely generated by students who read the problems but didn’t know
how to solve them properly. Attempts longer than 35 seconds have a higher
probability of being a genuine attempt at solving the AC problems and are more
frequently observed among high performing students. The decision to skip the
first attempt must be made before the start of the attempt, therefore the
fraction of Short (¡15s) First Attempts on each OLM provides information on
students’ planning actions for each OLM.

Furthermore, when a student fails the initial attempt on an OLM, they can
decide to study the materials in the IC before attempting the AC again or to
make additional attempts immediately. From an SRL perspective, a student with
a goal of mastering the content will likely access the IC after 1 or 2 failed
attempts on the AC, whereas a student with the goal of completing the module
with as little effort as possible are more likely to never access the IC at
all (a ”No Study” strategy) or access the AC after 3 or more fail attempts (a
”Late Study” strategy). Preliminary data analysis suggests that students who
are cramming on multiple modules just prior to the due date are more likely to
adopt these strategies. Measuring the popularity of Late and No Study
strategies among students is used as an indicator for students’ goal setting
behavior upon accessing each individual OLM.

In addition, students’ goal setting action can take place when considering
completing an entire OLM sequence as a larger task. In this context, students
may set a goal to complete modules early and earn extra credit, or decide to
”cram” complete all or most of the modules on or close to the due date.
Detailed investigation of students’ work distribution as a result of extra
credit would require extensive analysis beyond the scope of the current paper
(see for example (Felker and Chen, 2020)). In this paper, we present a quick
estimation by measuring the number of modules completed at least 1 or 3 days
prior to the due date as indicators for students’ goal setting behaviors when
an OLM sequence is viewed as a task.

Regarding the Performance Phase, it is difficult to infer cognitive strategies
adopted by students via click-stream data alone. However, we may
straightforwardly estimate the outcome of learning by measuring the percentage
of passing AC attempts either before or after accessing the IC. In an OLM
sequence, passing attempts before accessing the IC on a later module can be a
measure of learning quality of earlier modules (Chen et al., 2018). In a
previous study (Chen et al., 2020), we found that some fractions of students
pass the AC on a Brief (¡35s) Attempt, which could suggest that students
guessed the answer by chance or obtained the answer from other sources, such
as a classmate. As previously explained, Short (¡15s) Attempts are likely
generated from students who didn’t read the problem body, and Brief Attempts
are more likely generated from students who read the problem body. Therefore,
we use the fraction of Brief and Short Attempts as an indicator for the
quality of students’ learning on each OLM.

Self-reflection is mostly a metacognitive process which doesn’t often generate
direct records in click-stream data. However, we have identified certain types
of behaviors which may be indicative of self-reflective processes. Most
students interact with each OLM only once and move on after passing the AC,
but some will revisit a previously passed module while working on a downstream
OLM in the sequence. Therefore, the average number of modules reviewed by one
student in a given sequence is chosen as an indicator for the frequency of a
self-reflective process. Moreover, self-reflection could take place when
students are reviewing for an upcoming exam, and can be estimated by the
number of OLMs revisited shortly before an exam day. For the current analysis,
we measure the number of modules revisited by a student up to three days
before a midterm exam after campus closure as an indicator for reviewing
behavior.

It must be emphasized that SRL is an interdependent and iterative process,
thus each student action identified is likely influenced by or resulted from
multiple different SRL behaviors in different phases. The reason we associate
each action to one behavior is just to provide an organizational framework for
presenting the results, as well as a baseline for interpreting those results.

###  1.3. Examining the Impact of Campus Closure on Student SRL Processes

In this paper we examine the hypothesis that campus closure resulted in a
significant reduction of productive SRL behavior in the student population.
More specifically, it would result in the following changes in the six data
indicators of SRL actions after campus closure:

  1. (1)

An increase in the frequency of Short 1st Attempts, indicating a reduction in
planning and self-assessment.

  2. (2)

An increase in the fraction of students adopting a Late or No Study strategy,
indicating a shift from mastery-oriented goals to performance-oriented goals.

  3. (3)

A decrease in the number of modules completed 3 or more days prior to the due
date, indicating fewer students setting goals involving completing the modules
early.

  4. (4)

A decrease in passing rate before or after studying the IC, or an increase in
Short Passing Attempts, indicating a reduction in learning outcomes.

  5. (5)

A decrease in the number of revisiting events during each sequence or close to
a midterm exam, indicating a decrease in frequency of self-reflection.

We will examine and compare each type of data due before and after campus
closure following the data analysis schema explained in the Methods, section
(2). We will also present details about OLMs and their implementation in the
physics course as well as operational definitions of actions (e.g., passing a
module). We will present the analysis in the Results, section (3), followed by
discussion on the implications and possibilities of future studies.

##  2\. Methods

###  2.1. OLM Design and Implementation

The OLM modules are created and hosted on the Obojobo learning objects
platform (Center for Distributed Learning, [n.d.]), an open source online
learning platform developed by the Center for Distributed Learning at the
University of Central Florida. In the current iteration, the AC of each OLM
contains 1-2 multiple choice problems and permits a total of 5 attempts. Each
of the first 3 attempts are sets of isomorphic problems assessing the same
content knowledge with different surface features or numbers. On the 4th and
5th attempts, students are presented with the same problems in the 1st and 2nd
attempts respectively and are awarded 90% credit. The IC of each module
contains a variety of learning resources including text, figures, videos, and
practice problems. Access to the IC is locked whenever a student is attempting
the AC. Each OLM sequence contains 3-12 OLMs, which students must complete in
the order given, with completion defined as either passing the AC or using up
all 5 attempts. Readers can access example OLMs in the following URL provided
in (obo, [n.d.]).

###  2.2. Instructional Conditions and Student Population

In the Spring 2020 semester, 70 OLMs in 9 sequences were assigned as online
homework in a calculus-based university introductory physics course, which was
taught in a traditional lecture format before campus closure. In Fall 2019, 44
of the 70 modules were assigned in the same course. The new OLMs added in
Spring 2020 include sequences S1, S2 (modules 1-16), and modules added to S8
(modules 51-57) and S9 (modules 63-66). Each sequence corresponds to classroom
or online instruction for 1-2 weeks with due dates concurrent with lecture
instruction. All OLMs in a sequence are due on the same day. In Spring 2020,
the last three sequences containing 29 modules are due after campus closure.
In Fall 2019, the last 5 modules were due after Thanksgiving break.

In Fall 2019, the OLM sequences accounted for 18% of total course credit, and
online homework from a commercial publisher was used for topics for which no
OLM module was available. In Spring 2020, the OLM sequences accounted for 36%
of course credit, with no additional homework assignments. In Fall 2019,
submissions after the due date received 0 points, while in Spring 2020, late
submissions would receive a 13% daily penalty. In addition, students in both
semesters could earn extra credits by completing some OLMs earlier than the
due date, as explained in more detail in (Felker and Chen, 2020).

In Spring 2020, 276 students were initially enrolled in the class consisting
of 200 males and 76 females. 107 of the students were historically
underrepresented minorities and a total of 263 students passed the course. In
Fall 2019, 289 students registered for the course consisting of 234 males and
54 females. 111 of the students were historically underrepresented minorities
and a total of 247 students passed the course.

###  2.3. Collection and Analysis of Student Log Data

####  2.3.1. Data Collection and Operational Definitions

We list below the operational definition of all key terms related to the data
indicators in Table 1 and section (1.2.2). Readers interested in more nuanced
details of data extraction and cleaning can refer to (Chen et al., 2020).

  * •

AC Attempt Outcome: A student passes an AC attempt by answering every question
on the AC correctly.

  * •

AC Attempt Duration: The time between a student’s click on the start attempt
button and submission button for a given AC.

  * •

Brief and Short Attempt: We will refer to an attempt with duration of less
than 15 seconds as a ”Short Attempt”, and an attempt with duration between 15
and 35 seconds as a ”Brief Attempt”.

  * •

Module Pass: A student will be considered to have passed the module if they
passed the AC within 3 attempts. The distinction arises from the fact that the
4th attempt and beyond were already seen by the student and are given reduced
credit.

  * •

Module Fail: A student will be considered to have failed the module if they
fail on all of the first 3 attempts at the AC.

  * •

Module Complete: A student either passes the module or uses up all attempts.
Time of completion is recorded as the submission time of the first passing
attempt or last failed attempt.

  * •

Late or No Study: A student does not access the IC before the 3rd attempt of
the AC. Students in this category may either access the IC after the 3rd
attempt or not, in which case they will be considered to have adopted the
”Late Study” or ”No Study” strategy respectively.

  * •

Module Revisit: A student interacting with any part of the module for at least
60 seconds after initial completion.

###  2.4. Analysis Scheme

From a data analysis perspective, the six types of actions fall into two
distinct categories: module level action and sequence level action, as listed
in Table 1. Module level actions are actions or decisions made on each module
(e.g., planning to skip the first attempt or not). The proportion of module
level actions on each module is expected to roughly follow a single linear
trend over the semester and to be relatively insensitive to the order of the
module in a given sequence. In comparison, sequence level actions are strongly
influenced by the location of the module within the sequence (e.g., module
completion 3 or more days before the due date) or can only be defined for each
sequence (e.g., number of students who revisited at least 2 previous upstream
modules in a sequence). The disparity in the number of modules (70) to the
number of sequences (9) led us to employ two analysis schemes.

For module level actions, we first calculate the frequency of a given data
indicator in each module, then constructed a linear model from each of the two
semesters in the form:

(1) $y_{i}=y_{0}+\alpha n_{i}+\epsilon_{i}$

where $y_{i}$ is the frequency of observing the data indicator on module $i$
and $n_{i}$ the order in which students complete each OLM in the semester.
$y_{0}$ is the intercept, $\alpha$ the slope, and $\epsilon_{i}$ the noise
term which accounts for all other effects not captured by the linear model.
When constructing the linear models, the module numbers for Spring 2020 were
used for both years as the missing OLM sequences and OLMs were supplemented
with online homework assigned from WebAssign.

In addition, data from 2020 was further divided into two segments, the OLMs
that were due before and after campus closure, A and B respectively. For
comparison, the same partition is applied to the modules in 2019 even though
no campus closure took place. Six linear models of the form (1) were
constructed for each of the data indicators outlined in Table 1.

Table 2. Analysis Acronyms.

Acronym | Type | Year(s) | Segments
---|---|---|---
20A-B | Within Semester | 2019 | A vs. B
19A-B | Within Semester | 2019 | A vs. B
20A-19A | Between Semesters | 2020 vs. 2019 | A vs. A
20B-19B | Between Semesters | 2020 vs. 2019 | B vs. B
20-19 | Between Semesters | 2020 vs. 2019 | Entire Semester

A table tabulating the analysis acronyms to be used later.

Next, we compared the slopes of the linear models for Segments A and B within
the same semester listed as 20A-B and 19A-B in rows 1 and 2 of Table 2. We
tested the homogeneity of the regression slopes using Analysis of Covariance
(ANCOVA) by including the interaction of due date and module number:

(2) $y_{i}=y_{0}+\alpha n_{i}+\beta\delta_{AB}n_{i}+\epsilon_{i}$

where $\delta_{AB}=0$ if module $i$ is in Segment A, and $\delta_{AB}=1$ if
the module is in Segment B. If $\beta$ is not significantly different from
zero (i.e., the slope is similar for the two segments), we performed a second
ANCOVA of the form:

(3) $y_{i}=y_{0}+\alpha n_{i}+\gamma\delta_{AB}+\epsilon_{i}$

If $\gamma$ is significantly different from 0, that indicates that the
intercepts between the two segments are significantly different.

If campus closure had a significant impact on a given SRL signal (i.e., either
$\beta$ or $\gamma$ is significantly different from $0$), we isolate the
effect by comparing the linear models for Segments A and B between the two
semesters, listed as 20A-19A and 20B-19B in rows 3 and 4 of Table 2 using the
subset of modules common to both semesters. If the effect is directly
detectable, we expect that the linear models for Segment B of Spring 2020 to
be significantly different from Segment B of Fall 2019.

If no differences were detected for the linear models of Segments A and B, we
then proceeded to compare the linear models for the entire semester, row 5 of
Table 2. If the slope or intercept was found to be different, it is likely
that either the student population or the instructional condition was
different between the two semesters, but campus closure didn’t have a
detectable impact on the action analyzed.

Analysis of data on sequence-level actions (i.e., early completion and
revisiting) is more straightforward. We first record the observation of the
data indicator (e.g., completing a module 3 days before the due date) for
every student who accessed all modules in a given sequence. The Friedman test
is then performed to observe any differences between the sequences over each
semester. Each sequence can be treated as an independent category since they
cover different topics and have different due dates. To satisfy the complete
block design requirement of the Friedman test, only students who accessed all
9 sequences were retained in the analysis. In the case of revisiting, where we
count the number of students which have at least one revisiting event,
Cochran’s Q test was used in lieu of the Friedman test.

If statistically significant differences are detected between sequences, a
post hoc analysis using pairwise exact tests (Eisinga et al., 2017) (early
completion) or McNemar’s tests (revisiting) is conducted to determine the
precise differences in frequencies between the sequences. If campus closure
had a significant impact on students’ SRL behavior related to the observed
actions, then the observed frequency on sequences due after campus closure
will be significantly different from those due before campus closure. Data
from Fall 2019 was also analyzed and presented for revisiting actions but not
for early completion actions since the first 5 or 6 modules of S8 and S9 were
not available, inevitably resulting in significantly less early completion
events in those sequences.

For the action of revisiting before an exam, we simply recorded the number of
modules revisited over a period of three days leading to an exam by each
student and compared the distribution between the semesters via Wilcoxon test.
All statistical procedures were conducted in R (R Core Team, 2019) with the
tidyverse and PMCMRplus packages (Wickham et al., 2019; Pohlert, 2020).

##  3\. Results

For each figure in this section, the black vertical line separates the modules
due before campus closure (Segment A), from those due after (Segment B). For
figures representing module level data, the blue line visualizes the linear
regression models with the shaded areas representing the 95% confidence
interval. For each table in this section, statistically significant
differences are emboldened and appended with asterisk markings *, **, ***
representing significance at the $\alpha=0.05,\;0.01,\;\text{and}\;0.001$
levels respectively.

###  3.1. Overall Engagement

The fraction of student module access is shown in Figure 1(b) with the linear
models constructed for Segments A and B for each semester. The ANCOVA of the
linear models following the analysis scheme outlined in Table 2 are shown in
Table 3.

(a) Spring 2020

(b) Fall 2019

Figure 2. Fraction of students accessing each module.

Two graphs displaying the fraction of students who access each module, one for
Spring 2020 and one for Fall 2019.

Table 3. ANCOVA statistics for overall engagement.

| 20A-B | 19A-B | 20A-19A | 20B-19B
---|---|---|---|---
F (slope) | 0.003 | 0.895 | 4.996* | 48.339**
F (intercept) | 7.073** | 14.416** | N/A | N/A

A table outlining the ANCOVA statistics for engagement rates.

The fraction of students accessing each module remained remarkably stable over
the entire 2020 semester, between 85% and 100% with no significant difference
detected in the slopes of the linear models. The linear model for Segment B
(post-campus closure) has a higher intercept than that of Segment A (pre-
campus closure), possibly due to modules 29 and 30 having lower than average
access percentage. Similarly, data from Fall 2019 had no significant
difference in the slopes of the linear models between each segment, but had a
higher intercept in the latter half. The regression slopes in 2019 were
significantly more negative than their 2020 counterparts, despite the absence
of campus closure

###  3.2. The Forethought Phase

####  3.2.1. Planning

In Figure 2(b), we plot the fraction of 1st AC attempts on each module under
15 seconds as an indicator for students’ planning action before each OLM. The
results of the comparisons between linear models are listed in Table 4.

(a) Spring 2020

(b) Fall 2019

Figure 3. Fraction of first attempts under 15 seconds.

Two graphs displaying the fraction of first attempts which are under 15
seconds for each module, one for each semester.

Table 4. ANCOVA statistics for Short First Attempts.

| 20A-B | 19A-B | 20A-19A | 20B-19B
---|---|---|---|---
F (slope) | 23.034*** | 1.168 | 1.032 | 0.066
F (intercept) | N/A | 2.418 | 7.809** | 3.297

A table outlining the ANCOVA statistics for short first attempts.

In 2020, the proportion of Short First Attempts increased significantly more
rapidly in Segment B than in Segment A. This rapid shift in slope is not
detected in data from Fall 2019, for which there was no significant difference
observed between the intercepts of the regression lines in each segment. Our
analysis also failed to detect significant differences between the linear
models for Segment B between the two semesters.

In contrast, the fraction of 1st attempts between 15 and 35 seconds showed no
difference in either the slopes or intercepts between either Segments A and B
of each semester, or the overall linear models for both semesters, see Figure
3(b).

(a) Spring 2020

(b) Fall 2019

Figure 4. Fraction of first attempts between 15 and 35 seconds.

Two graphs, one for 2020 and one for 2019, displaying the fraction of first
attempts between 15 and 35 seconds per module.

####  3.2.2. Goal Setting

In Figure 4(b), we plot the fraction of students who adopted either a Late or
No Study strategy on each module as an indicator for students’ goal setting
behavior for each module.

(a) Spring 2020

(b) Fall 2019

Figure 5. Fraction of students adopting a Late Study or No Study strategy.

Two graphs displaying the fraction of students who adopt a Late Study or No
Study strategy per module, one for each semester.

In Spring 2020, the number of students adopting a Late or No Study strategy
increased much faster in Segment B when compared to Segment A,
$F_{1,67}=10.617,\;p<0.05$, while no difference in trend was detected for Fall
2019. Comparing the linear models for each segment between semesters did not
show a statistically significant difference in either the slopes or the
intercepts of each model.

To examine students’ sequence level goals, we plot the average number of
modules completed by a student at least 1 or 3 days before the sequence due
date, see Figure 5(b). Friedman’s test detected that there were significant
differences in the fractions between different sequences in both conditions
($\chi^{2}_{1}(8)=446.121,\;p<0.001$ and $\chi^{2}_{3}(8)=586.571,\;p<0.001$),
but post hoc analysis showed that none of the significantly different
sequences were due after campus closure.

(a) Spring 2020 (1 Day)

(b) Spring 2020 (3 Days)

Figure 6. Average fraction of modules completed in a sequence at least 1 or 3
days before the due date.

Two graphs, one displaying the fraction of modules completed 1 day early per
sequence in 2020, one displaying the fraction of modules completed 3 days
early per sequence in 2019.

###  3.3. The Performance Phase

####  3.3.1. Learning Outcome

The fraction of students passing each module before accessing the IC, as
plotted in Figure 6(b), remained largely stable throughout Spring 2020. No
significant difference was found between the regression slopes for Segments A
and B. However, the intercept of Segment A was found to be higher than that of
Segment B, $F_{1,67}=4.153,\;p<0.05$, likely caused by the slightly negative
slope of the model in Segment A. In 2019, both the regression slopes and
intercepts remain not significantly different between both segments. Comparing
Segment A between semesters, we found no significant difference for the
regression slopes, but the intercept in Spring 2020 was higher than that of
Fall 2019, $F_{1,47}=5.225,\;p<0.05$. No difference was detected in Segment B
between each semester.

(a) Spring 2020

(b) Fall 2019

Figure 7. Fraction of students who passed before study of the IC.

Two graphs displaying the fraction of students passed the module before
studying the instructional components, one for 2020 and one for 2019.

The fraction of students passing each module after accessing the IC is
remarkably stable across both semesters, with no difference detected in all of
our comparisons (see Figure 7(b)).

(a) Spring 2020

(b) Fall 2019

Figure 8. Fraction of students who passed after study of the IC.

Two graphs displaying the fraction of students passed the modules after
studying the instructional components, one for 2020 and one for 2019.

In contrast, the percentage of Short Passing Attempts saw a significant upward
shift in Segment B of Spring 2020,$F_{1,66}=10.667,\;p<0.05$, but not in Fall
2019. No difference was found when comparing the linear models for 20B-19B. No
difference in intercepts was found for comparisons 20A-19A, and 20B-19B. When
the criterion is loosened to include Brief Attempts, no difference in slope
nor intercept was found between the models, see Figure 10.

(a) Spring 2020

(b) Fall 2019

Figure 9. Fraction of Short Passing Attempts.

Two graphs displaying the fraction of passing attempts which are under 15
seconds, one for 2020 and one for 2019.

(a) Spring 2020

(b) Fall 2019

Figure 10. Fraction of Brief Passing Attempts.

Two graphs displaying the fraction of passing attempts which are between 15
and 35 seconds, one for 2020 and one for 2019.

###  3.4. The Self-Reflection Phase

In Figure 10(b), we plot the fraction of students with at least 1 revisiting
event as defined in section (2.3.1). Cochran’s Q test indicated that there
were significant differences between the OLM sequences in each semester,
$\chi^{2}_{2020}(8)=222.514,\;p<0.001$ and
$\chi^{2}_{2019}(8)=92.888,\;p<0.001$. Pairwise McNemar tests determined that
in 2020, the revisiting fraction of S1 and S8 was significantly higher than
the rest of the sequences and that of S5 was lower than all the other
sequences except S4. Additionally, it was found in 2019, S4 and S5 were
significantly lower than the rest of the sequences.

(a) Spring 2020

(b) Fall 2019

Figure 11. Fraction of students with at least 1 revisiting event.

Two graphs displaying the fraction students with at least one revisiting event
by sequence, one for 2020 and one for 2019.

The distribution of modules revisited by each student within a 3 day period
leading to the second midterm exam is plotted in Figure 11(b). There does not
appear to be a statistically significant difference in the distribution of the
number of modules revisited by students within 3 days of the exam according to
the Wilcoxon Test (Z=8115.5, p=0.696).

(a) Spring 2020

(b) Fall 2019

Figure 12. Histograms of the number of modules revisited up to three days
before the second midterm exam.

Two histograms, one for 2020 and one for 2019, which display the distribution
of students among modules revisited.

##  4\. Discussion

###  4.1. Discussion of Results

Results of the current analysis indicate that some SRL actions are impacted
much more by COVID-19 induced campus closure than others; overall, the changes
in SRL action that can be attributed to it were less than expected.

Most notably, we saw a significant increase in trend of the fraction of Late
or No Study events per module for the OLMs due after campus closure, which was
not observed in Fall 2019. Similarly, an increasing trend was observed for the
percentage of Short Attempts, indicative of guessing or copying, for the OLMs
due after campus closure which was again absent in 2019. These observations
suggest that after campus closure, more students are adopting performance-
oriented goals (e.g. completing the module with as little time as possible)
over mastery-oriented goals (e.g. internalizing new content) in the
Forethought Phase and executing those strategies during the Performance Phase.
Notably, there were no sudden shifts in the trend of Brief Attempt
submissions, which are more likely to be generated from students who read the
assessment problem before deciding to guess. Therefore, the abrupt increase in
Short Attempts is more likely a result of change in student strategies, rather
than an increase in content difficulty.

It must be mentioned that in both cases we did not find a significant
difference comparing data from Segment B between the 2020 and 2019 semesters.
It could mean that the impact from campus closure was not strong enough to be
detected, but could also be caused by a lack of statistical power of the
analysis resulting from the smaller number of modules released in Segment B of
2019.

A similar increase in slope after campus closure was observed for the 1st
attempts under 15 seconds for each OLM, which is indicative of students’
planning action during the Forethought Phase. This could suggest that more
students planned to skip the self-assessment opportunities before starting
each new module. However, a similar trend was found for the same modules in
Fall 2019 albeit to a lesser extent. This may imply that the observed change
could in part be due to increase in content difficulty or reduction in
engagement towards the end of the semester unrelated to campus closure.

We found no significant differences comparing a number of data indicators
before and after campus closure: the number of modules completed earlier than
the due date, module passing rate before and after study, the number of module
revisiting events. One exception found was the fraction of students revisiting
sequence S8, which seems to be higher than average in Spring 2020 which is
opposite to the predicted impact of campus closure.

In summary, our results show that COVID-19 related campus closure and distance
learning affected how students completed or planned to complete each module,
pushing more students towards adopting goals and strategies that minimize time
and effort required to pass each module. On the other hand, we found that
there was little or no impact on overall engagement, student performance, or
self-reflective processes.

It is worth noting that the fraction of students accessing each module
decreased more rapidly in both Segment A and B in Fall 2019 when compared to
Spring 2020. Additionally, the fluctuation in module access is much greater in
2019. This difference cannot be caused by the 2020 campus closure and are more
likely due to differences in instructional conditions and student populations
between the semesters, which could have a non-negligible impact on our
analysis.

###  4.2. Implication of Results

Despite the significant disruptions caused by the COVID-19 induced campus
closure, the impact on students’ SRL processes in online learning is less than
what many have feared. Even though an increasing number of students adjusted
their plans and strategies towards preserving time and resources after campus
closure, the fraction of students completing and passing the OLMs on time or
early remained remarkably stable, as well as the fraction of students who
revisited a previously passed module.

This could imply that college students enrolled in a first year physics course
have stronger self-regulatory skills than we previously thought. It could also
suggest that online learning may have provided students with the needed
flexibility to adjust to unexpected disruptions, which is consistent with
findings in (Gonzalez et al., 2020). Furthermore, it is possible that the
mastery-based design of the OLM sequences could have facilitated students’ SRL
processes by providing frequent self-assessment opportunities during learning.
Of course, many other factors such as difference in instructional condition
and student populations could also have contributed to the lack of observed
differences.

###  4.3. Limitations and Future Directions

The current paper provides a quick estimation of the impact of campus closure
on students’ SRL behavior in an online learning environment. Many of our
decisions regarding data selection and analysis methods were prioritized for
quickly detecting trends in the data rather than creating a comprehensive
model. Those choices, while sufficient for the purposes of the current study,
leave much room for improvement in future studies.

First, future studies could include more data such as duration of study and
number of practice problems. While containing rich information about students’
learning, they are not included in this paper as their relation with quality
of learning is less straightforward. Furthermore, incorporation of data from
multiple sources such as the Revised Achievement Goal Questionnaire-Revised
(Elliot and Murayama, 2008) could be used to measure aspects of students’ SRL
(i.e., motivation) that are not well reflected in click-stream data.

Second, future studies should extend beyond the current linear regression
models used to quickly estimate shifts in data. More sophisticated models such
as linear mixture modeling or the ones used in (Chen et al., 2020) could
account for a number of factors overlooked in the current study. Such factors
include the difference in topical difficulty between different OLMs and OLMs
sequences or differences in instructional policy choices in courses.
Additionally, individual students’ shifts in study strategies could be tracked
for analysis on a finer scale.

Third, the validity of comparisons of data between Fall 2019 and Spring 2020
is less than ideal, in large part due to differences in the number of modules
assigned. Future studies involving the latest data from Fall 2020, during
which the entire course was taught online, could provide a better baseline for
comparison.

Finally, the current study paper presents a case study that includes students
from one class, studying one topic, using one type of online instructional
design. A highly valuable direction of future research is to compare and
contrast multiple studies involving different student populations, subject
matter, and online instructional designs to obtain generalizeable knowledge
that will guide the design of future learning environments. Directed by new
insights from such analysis, these systems can not only be more resilient to
disruptions, but also more flexible in accommodating today’s increasingly
diverse student population (U.S Department of Education and Office of
Educational Technology, 2017).

###### Acknowledgements.

This work is supported by NSF Award No. DUE-1845436. We would like to thank
UCF Center for Distributed Learning for creating the UCF Open project and the
Obojobo platform, in particular Dr. Francisca Yonekura, Ian Turgeon and
Zachary Berry.

## References

  * (1)
  * obo ([n.d.]) [n.d.].  https://canvas.instructure.com/courses/1726856
  * Aleven et al. (2016) Vincent Aleven, Jonathan Sewall, Octav Popescu, Michael Ringenberg, Martin van Velsen, and Sandra Demi. 2016.  Embedding Intelligent Tutoring Systems in MOOCs and e-Learning Platforms. In _Intelligent Tutoring Systems_ , Alessandro Micarelli, John Stamper, and Kitty Panourgia (Eds.). Springer International Publishing, Cham, 409–415.
  * Bloom (1968) B Bloom. 1968.  Learning for Mastery. Instruction and Curriculum. Regional Education Laboratory for the Carolinas and Virginia, Topical Papers and Reprints, Number 1.  _Evaluation comment_ 1 (1968), 12.
  * Center for Distributed Learning ([n.d.]) Center for Distributed Learning. [n.d.].  Obojobo.
  * Chen et al. (2018) Zhongzhou Chen, Kyle M. Whitcomb, and Chandralekha Singh. 2018\.  Measuring the effectiveness of online problem-solving tutorials by multi-level knowledge transfer. In _Physics Education Research Conference 2018_ _(PER Conference)_. Washington, DC.
  * Chen et al. (2020) Zhongzhou Chen, Mengyu Xu, Geoffrey Garrido, and Matthew W. Guthrie. 2020.  Relationship between students’ online learning behavior and course performance: What contextual information matters?  _Physical Review Physics Education Research_ 16, 1 (jun 2020), 010138.  https://doi.org/10.1103/PhysRevPhysEducRes.16.010138
  * Dorn et al. (2020) Emma Dorn, Bryan Hancock, Jimmy Sarakatsannis, and Ellen Viruleg. 2020.  COVID-19 and student learning in the United States: The hurt could last a lifetime. (2020), 14 pages.
  * Eisinga et al. (2017) Rob Eisinga, Tom Heskes, Ben Pelzer, and Manfred Te Grotenhuis. 2017.  Exact p-values for pairwise comparison of Friedman rank sums, with application to comparing classifiers.  _BMC Bioinformatics_ 18, 1 (2017), 1–18.  https://doi.org/10.1186/s12859-017-1486-2
  * Elliot and Murayama (2008) Andrew J Elliot and Kou Murayama. 2008.  On the measurement of achievement goals: Critique, illustration, and application.  _Journal of Educational Psychology_ 100, 3 (2008), 613–628.  https://doi.org/10.1037/0022-0663.100.3.613
  * Felker and Chen (2020) Zachary Felker and Zhongzhou Chen. 2020.  The impact of extra credit incentives on students’ work habits when completing online homework assignments. In _2020 Physics Education Research Conference Proceedings_. American Association of Physics Teachers (AAPT), Virtual Conference, 143–148.  https://doi.org/10.1119/perc.2020.pr.felker
  * Gonzalez et al. (2020) T. Gonzalez, M. A. de la Rubia, K. P. Hincz, M. Comas-Lopez, Laia Subirats, Santi Fort, and G. M. Sacha. 2020.  Influence of COVID-19 confinement on students’ performance in higher education.  _PloS one_ 15, 10 (2020), e0239490.  https://doi.org/10.1371/journal.pone.0239490
  * Guthrie et al. (2020) Matthew W. Guthrie, Tom Zhang, and Zhongzhou Chen. 2020\.  A tale of two guessing strategies: interpreting the time students spend solving problems through online log data. In _Physics Education Research Conference Proceedings_. American Association of Physics Teachers (AAPT), Virtual Conference, 185–190.  https://doi.org/10.1119/perc.2020.pr.guthrie
  * Gutmann et al. ([n.d.]) Brianne Gutmann, Gary E. Gladding, Morten Lundsgaard, and Timothy Stelzer. [n.d.].  Mastery-style homework exercises in introductory physics courses: Implementation matters.  _Physical Review Physics Education Research_ ([n. d.]).
  * Li et al. (2020) Qiujie Li, Rachel Baker, and Mark Warschauer. 2020.  Using clickstream data to measure, understand, and support self-regulated learning in online courses.  _Internet and Higher Education_ 45 (apr 2020), 100727\.  https://doi.org/10.1016/j.iheduc.2020.100727
  * Maldonado-Mahauad et al. (2018) Jorge Maldonado-Mahauad, Mar Pérez-Sanagustín, René F. Kizilcec, Nicolás Morales, and Jorge Munoz-Gama. 2018\.  Mining theory-based patterns from Big data: Identifying self-regulated learning strategies in Massive Open Online Courses.  _Computers in Human Behavior_ 80 (2018), 179–196.  https://doi.org/10.1016/j.chb.2017.11.011
  * Pohlert (2020) Thorsten Pohlert. 2020\.  _PMCMRplus: Calculate Pairwise Multiple Comparisons of Mean Rank Sums Extended_.  https://CRAN.R-project.org/package=PMCMRplus R package version 1.6.1.
  * R Core Team (2019) R Core Team. 2019\.  R: A Language and Environment for Statistical Computing.
  * Schwartz and Bransford (2005) Daniel L. Schwartz and John D. Bransford. 2005.  Efficiency and Innovation in Transfer.  In _Transfer of Learning from a Modern Multidisciplinary Perspective (Current Perspectives on Cognition, Learning and Instruction)_ , Jose P. Mestre (Ed.). IAP - Informaiton Age Publishing Inc., Charlotte, 1–51.
  * Taub et al. (2018) Michelle Taub, Roger Azevedo, Amanda E. Bradbury, Garrett C. Millar, and James Lester. 2018\.  Using sequence mining to reveal the efficiency in scientific reasoning during STEM learning with a game-based learning environment.  _Learning and Instruction_ 54 (2018), 93–103.  https://doi.org/10.1016/j.learninstruc.2017.08.005
  * U.S Department of Education and Office of Educational Technology (2017) U.S Department of Education and Office of Educational Technology. 2017\.  Reimagining the Role of Technology in Higher Education. (2017).
  * Wickham et al. (2019) Hadley Wickham, Mara Averick, Jennifer Bryan, Winston Chang, Lucy D’Agostino McGowan, Romain François, Garrett Grolemund, Alex Hayes, Lionel Henry, Jim Hester, Max Kuhn, Thomas Lin Pedersen, Evan Miller, Stephan Milton Bache, Kirill Müller, Jeroen Ooms, David Robinson, Dana Paige Seidel, Vitalie Spinu, Kohske Takahashi, Davis Vaughan, Claus Wilke, Kara Woo, and Hiroaki Yutani. 2019\.  Welcome to the tidyverse.  _Journal of Open Source Software_ 4, 43 (2019), 1686\.  https://doi.org/10.21105/joss.01686
  * Wilcox and Vignal (2020) Bethany R. Wilcox and Michael Vignal. 2020.  Understanding the student experience with emergency remote teaching. In _2020 Physics Education Research Conference Proceedings_. American Association of Physics Teachers (AAPT), Virtual Conference, 581–586.  https://doi.org/10.1119/perc.2020.pr.wilcox
  * Zimmerman (2013) Barry J. Zimmerman. 2013\.  From Cognitive Modeling to Self-Regulation: A Social Cognitive Career Path.  _Educational Psychologist_ 48, 3 (2013), 135–147.  https://doi.org/10.1080/00461520.2013.794676
  * Zimmerman (2015) Barry J. Zimmerman. 2015\.  _Self-Regulated Learning: Theories, Measures, and Outcomes_ (second edi ed.). Vol. 21.  Elsevier. 541–546 pages.  https://doi.org/10.1016/B978-0-08-097086-8.26060-1