#  Overlapping plastic events as a mechanism for irreversible dynamics
in amorphous solids under oscillatory shear

Asaf Szulc  Department of Physics, Ben Gurion University of the Negev, Beer
Sheva 84105, Israel  Ido Regev<EMAIL_ADDRESS>Department of Environmental
Physics, Jacob Blaustein Institutes for Desert Research, Ben-Gurion University
of the Negev, Sede Boqer Campus 84990, Israel

###### Abstract

The origin of the transition from asymptotically reversible to asymptotically
irreversible response in amorphous solids subject to oscillatory shear is
still unknown. It is known that the plastic events that result from shearing
always involve localized particle rearrangements, but it is unclear why some
are reversible while others are not. Here we show, using simulations and
models, that overlaps between particle rearrangements caused by straining the
solid in alternating directions can cause the response to become irreversible
when they occur frequently. As the forcing amplitude increases, plastic events
become more frequent, the number of such overlaps increases, and the
probability of the system returning to previous states diminishes.

##  I Introduction

Since its discovery, the irreversibility transition in amorphous solids under
oscillatory shear has attracted significant interest Regev  _et al._ (2013);
Keim and Arratia (2013); Fiocco  _et al._ (2013); Priezjev (2013); Schreck
_et al._ (2013, 2013); Bhaumik  _et al._ (2021); Leishangthem  _et al._
(2017); Reichhardt  _et al._ (2023); Kawasaki and Berthier (2016). This
transition is observed when amorphous solids are subject to oscillatory shear
at athermal quasi-static (AQS) conditions for different strain amplitudes
$\gamma_{\max}$. For small, sub-yield amplitudes, the system eventually
reaches a periodic plastic state (a limit cycle in the terminology of
dynamical systems), where plastic rearrangements repeat after each cycle and
which stores an exact memory of the particle configurations that are part of
the cycleKeim  _et al._ (2019); Lavrentovich  _et al._ (2017); Keim  _et al._
(2020); Fiocco  _et al._ (2015); Regev  _et al._ (2015). However, reaching
these states requires many irreversible transient cycles, and when
$\gamma_{\max}$ approaches a critical value $\gamma_{c}$, the number of cycles
needed to reach a periodic state diverges. For $\gamma_{\max}>\gamma_{c}$, the
system never reaches a limit cycle and is thus mechanically irreversible. This
transition was demonstrated in experimental systems and atomistic simulations,
where $\gamma_{c}$ was found to be close to or identical to the strain at
which the material plastically yields (the yielding point) Regev  _et al._
(2013); Keim and Arratia (2013); Fiocco  _et al._ (2013); Priezjev (2013);
Schreck  _et al._ (2013). Despite the large interest in the transition from
reversible to irreversible dynamics and several attempts to explain it in
terms of dynamics on a random energy landscape Sastry (2021); Mungan and
Sastry (2021); Szulc  _et al._ (2020), there is still no widespread agreement
as to the nature and origin of this transition and these approaches do not
provide a microscopic origin of irreversibility.

On the microscopic level, it is known that plastic events involve localized
particle rearrangements known as soft spots Argon (1979); Falk and Langer
(1998); Maloney and Lemaître (2006); Manning and Liu (2011). In two
dimensions, a soft spot causes a displacement of four neighboring particles
where two next-nearest neighbors become closer to each other and two nearest
neighbors move further away from each other, resulting in a typical
quadrupolar displacement field Maloney and Lemaître (2006); Tyukodi  _et al._
(2019). The current understanding is that soft spots behave as two-level
systems that can switch back and forth at specific stresses in response to
external strains Maloney and Lemaître (2006); Mungan  _et al._ (2019);
Lindeman and Nagel (2021). This point of view implies that most plastic
transitions are reversible in the sense that there are strain paths that allow
them to repeat. However, most plastic events are actually irreversible in the
sense that they occur only once and can never repeat Lundberg  _et al._
(2008); Mungan  _et al._ (2019); Regev  _et al._ (2021); Lindeman and Nagel
(2021). Nevertheless, the microscopic deformations in both cases seem
qualitatively identical Lundberg  _et al._ (2008); Mungan  _et al._ (2019);
Regev  _et al._ (2021). It is, therefore, not always clear what causes some
events to be reversible while others irreversible.

Figure 1:  (a) A microscopic particle configuration exhibiting a regular
overlap between two soft spots (regions marked by blue and green ellispses).
(b) A microscopic particle configuration exhibiting an irregular overlap
between two soft spots (regions marked by blue and yellow ellipses). Both
examples are taken from Szulc  _et al._ (2022). In the irregular
configurations the coordinates were flipped $x_{i}\rightarrow-x_{i}$ such that
a soft spot that was activated at a negative strain is now activated at a
positive strain and vice-verse, in order to maintain consistency with the
other figure. In both cases the strain at which a soft spot switches or
switches back is shown as a horizontal line on a curve that shows the strain
as a function of simulation steps (the equivalent of time for athermal
quasistatic dynamics).

Recent work on hysteresis and memory in amorphous solids has focused on
modeling amorphous solids as a collection of interacting two-level systems
representing soft spots Lindeman and Nagel (2021); Szulc  _et al._ (2022);
Keim and Paulsen (2021). However, models with a fixed number of soft spots
have a finite number of possible configurations, as will be explained below.
Such models thus, have only transient irreversibility Lindeman and Nagel
(2021) and eventually reach a limit cycle for any amplitude, and cannot
exhibit yield under monotonic straining. To address these issues, recent
elastoplastic models chose to represent deformable regions as multi-level
systems Khirallah  _et al._ (2021); Kumar  _et al._ (2022); Liu  _et al._
(2022). This modeling strategy can allow for an infinite number of soft spots,
and simulations of some of these models have demonstrated a diverging number
of cycles at yield Khirallah  _et al._ (2021); Kumar  _et al._ (2022).
However, even in these models, the question of why the dynamics become
irreversible at yield remains unanswered.

Here we propose that the sharing of particles, or overlaps, between soft spots
is a plausible mechanism for the emergence of sustained irreversibility. By
modifying and extending current models to include overlap effects observed in
atomistic simulations, we show that overlaps caused by soft spots that
switched due to alternating strain directions can prevent the system from
reaching a limit cycle when they become frequent, which is the situation in
the post-yield regime Maloney and Lemaître (2006); Lin  _et al._ (2014a);
Tyukodi  _et al._ (2019).

##  II Overlaps and their significance

As mentioned above, a system comrpised of a finite number of hysterons will
always reach a limit cycle. To understand why, we note that in a system of
$\mathcal{N}$ hysterons, where $\mathcal{N}$ is finite, a state of the system
corresponds to a configuration of the possible states of the $\mathcal{N}$
hysterons. Each configuration of the system can thus be described in terms of
a binary number with $\mathcal{N}$ digits of the form $s=011000101\cdots
010001$. In this number the value of the $n$th digit corresponds to the state
of the $n$th hysteron which in the initial (reference) configuration can be
either $0$, which indicates that it can switch due to an increase in the
strain, or $1$, which indicates that it switches due to a decrease in the
strain. Note that out of all the possible $2^{\mathcal{N}}$ combinations of
$0$s and $1$s, only a small subset represents configurations that are stable
for some strain value, and that each stable configuration is stable for a
range of strains/stresses (See discussion in Mungan and Terzi (2019); Terzi
and Mungan (2020); Mungan  _et al._ (2019)). This means that for a fixed
$\mathcal{N}$, the system can attain a finite number (significantly smaller
than $\mathcal{N}$) of stable states. Since at zero temperature the dynamics
is deterministic, once the system reaches a state that was visited previously,
it continue to other states that were visited previously, and will therefore
start periodic dynamics. Furthermore, since there is only a finite number of
possible states, the system will always reach a state that was previously
visited, eventually, and will thus always enter periodic dynamics (enter a
limit cycle).

In an actual amorphous solid, the number of soft spots is not bounded. Since
each soft spot is made of several particles (typically four), when a soft spot
switches as part of a plastic event, some of its particles may belong to a
soft spot that was switched previously. The new soft spot may thus disable
previously switched soft spots from switching. Such sharing of particles
between soft spots is certain to occur in finite systems at large enough
strain amplitudes since if overlaps are prohibited, the space that was not yet
subject to plastic deformation runs out eventually.

Figure 2:  (a) At post-yield strain amplitudes $\gamma_{\max}$, deformation
during one cycle involves many large avalanches occurring in both straining
directions. This increases the chance that at least one irregular overlap will
occur during each forcing cycle (i.e. after a deformation of the form
$\gamma=0\rightarrow\gamma_{\max}\rightarrow-\gamma_{\max}\rightarrow 0$)
which will prevent the system from ever returning twice to the same zero-
strain configuration, thus preventing the system from reaching a limit cycle.
(b) Top: an illustration of the switching of a single soft spot from state $0$
(dotted) to state $1$ (continuous line) due to strain increase. Bottom: when
irregular overlaps are allowed there is a probability $p_{ir}$ that a soft
spot $\tilde{n}$ with $\sigma_{\tilde{n}}^{+}<\sigma_{n}^{+}$ will switch
before soft spot $n$ switches.

However, identifying such overlaps in experiments or atomistic simulations is
challenging due to the difficulties in identifying individual soft spots,
especially at large strain amplitudes when plastic events involve large
avalanches that can include many different soft spots. In a previous work, we
studied multiperiodic cycles observed in atomistic simulations of a system
comprising 1024 particles subjected to AQS shear. By applying a special
algorithm (which we developed) that employs a network representation of the
dynamics, we identified all the soft spots switching during these cycles Szulc
_et al._ (2022). The goal of this work was to understand the formation of
multiperiodic cycles (i.e limit cycles in which the zero strain configuration
repeats after more than one forcing cycle), as they involve both reversible
and irreversible dynamics. As part of that work, we also identified overlaps
between soft spots that were part of multiperiodic cycles. Nevertheless, the
significance of overlaps to the dynamics was not clear and was thus not
discussed in the manuscript.

A later examination of the aforementioned simulations revealed that overlaps
come in two distinct scenarios. In the first scenario, two or more soft spots
that switch at different strains overlap during a monotonic strain increase.
In most cases, when the strain is reversed, the soft spots switch in the
opposite order as demonstrated in the example shown in Fig.1(a) (the
exceptions will be discussed below). In Fig.1(a), soft spot 8 switches and
following a further increase in the strain, soft spot 1 also switches. When
the strain is decreased, soft spot 1 switches back and upon a further decrease
in the strain, soft spot 8 also switches back. Thus, in this case, the
switching exhibits a Last-In-First-Out (LIFO) ordering and the final zero
strain state is identical to the zero strain state at the beginning of the
cycle. In the second scenario, a soft spot that switches after the straining
direction is reversed, disables a soft spot that switched due to an increase
in the strain, as shown in the example in Fig. 1(b). In this example soft spot
1 switches upon an increase in the strain while soft spot 2 switches upon a
decrease in the strain before soft spot 1 is able to switch back (we have
verified that soft spot 1 is not able to switch back even for significantly
larger negative strains indicated by the dashed line). In the final zero-
strain state, soft spot 1 and 2 are both switched and this state is thus not
identical to the initial zero strain state. We refer to the first type of
overlap as a regular overlap, as it involves sequential switching and
switching back and the second type of overlap as an irregular overlap since it
breaks the switching ordering.

The fact that the irregular overlap caused the system to not return to the
initial zero strain state indicates that irregular overlaps can cause the
system to not be able to reach a periodic cycle. To see this, consider a
situation where the system is subject to $\gamma_{\max}$ that is larger than
the yield strain. In this case, each shearing cycle involves a number of large
avalanche events Nicolas  _et al._ (2018) that occur due to straining in both
positive and negative strains. One thus expects that the probability of
irregular overlaps due to new plastic events will be large and that for large
enough $\gamma_{\max}$, there will be at least one new such overlap during
each forcing cycle, as is illustrated in Fig. 2(a). In this case, the system
will not be able to repeat the same zero-strain configuration and periodic
behavior will not be possible.

Figure 3:  Illustration of the soft spot dynamics of a single site (site $k$
out of $N$ sites) in the model in the case where only regular overlaps are
allowed. At any given time, the site is in a particular configuration in which
the stress is $\sigma_{k}$ and one soft spot can switch due to a stress
increase and another one can switch due to a stress decrease. For example, in
the highlighted reference configuration, soft spot n switches at
$\sigma_{n}^{+}$ while soft spot $n-1$ switches at $\sigma_{n-1}^{-}$ where
$\sigma_{n}^{-}<\sigma_{k}<\sigma_{n}^{+}$. If $\sigma_{n}^{+}$ is reached,
the stress drops to a new value and then soft spot $n$ can switch back at
$\sigma_{n}^{-}$ and soft spot $n+1$ can switch at $\sigma_{n+1}^{+}$. The
former switch takes the system back to the previous (reference) configuration
while the latter switch takes the system to a new configuration. Figure 4:
Illustration of the soft spot dynamics of the same site as in Fig. 3 but where
irregular overlaps are allowed. For any site configuration, there is a
probability $p_{ir}$ that a competing soft spot $\tilde{n}$ with
$\sigma^{+}_{\tilde{n}}<\sigma^{+}_{n}$ will switch before soft spot $n$
switches. In that case, upon a further increase (in the example shown in the
illustration) in the strain, the system will reach configurations that were
not reachable with purely regular overlaps (faded part of the illustration).
However, if the strain is decreased (assuming no interactions with neighboring
sites) the system can return to the configurations that were reachable by
regular overlaps, assuming that no new irregular overlap occurs. Figure 5:
Comparison of the hysteresis loops generated by the model to hysteresis curves
obtained from molecular dynamics simulations: (a) stress-strain curve
generated using the model for forcing amplitude $\gamma_{\max}=0.7$ with
$p_{ir}=0.01$. (b) stress-strain curve generated from atomistic simulations
for forcing amplitude $\gamma_{\max}=0.15$ (data taken from the simulation
results discussed in Regev  _et al._ (2013)).

##  III Model

To better understand the contribution of overlaps to irreversibility, we
developed a model of soft-spot dynamics that includes both regular and
irregular overlaps. In our model, soft spots are represented by interacting,
hysteretic, two-state elements (hysterons), which we and others have used
before to model the response for small strain amplitudes Szulc  _et al._
(2022); Lindeman and Nagel (2021); Keim and Paulsen (2021); Bense and van
Hecke (2021); van Hecke (2021). However, contrary to our previous models that
used the strain as a state variable, here the stress in each hysteron is the
primary variable, and switching is stress-activated, as is the case in other
models of plasticity Hébraud and Lequeux (1998); Bocquet  _et al._ (2009);
Tyukodi  _et al._ (2019); Kumar  _et al._ (2022); Lin  _et al._ (2014b). The
interaction strength between soft spots is of the mean-field type with random,
frustrated, coupling, similar to what we used in a previous work Szulc  _et
al._ (2022) (see detailed description in the Supplementary material). The soft
spots are distributed in $N$ sites, where a site $k$ that belongs to one of
these $N$ sites, is associated with a distinct stress value $\sigma_{k}$ that
is modified at each simulation step due to an applied strain step
$\delta\gamma$ by $\delta\sigma=\mu\delta\gamma$. Here $\mu$ is the shear
modulus, which is assumed to be a material constant. Each site represents
several soft spots that share some of their particles, and thus when one of
them switches, others become disabled, similar to the situation in Figs.
1(a,b).

In the following we first develop a model that allows only regular overlaps
and then extend it to also include irregular overlaps. When only regular
overlaps are allowed, each site has two switchable soft spots for each
configuration of the system. In Fig.2(b,top) we show a schematic demonstrating
the switching of a single soft spot $n$ between two possible states due to an
increase in the stress. When the stress is increased to $\sigma_{n}^{+}$, the
soft spot switches from state $0$ to state $1$. Upon a decrease of the stress
to $\sigma_{n}^{-}$, the soft spot switches back to state $0$. A soft spot can
also have an initial state $1$ and switch to state $0$ upon a decrease of the
strain.

In Fig.3 we show typical dynamics in one site when only regular overlaps are
allowed. We take the configuration in the bold square to be a typical
configuration which we use as a reference configuration (or reference state)
for this specific site. In this configuration there are two switchable soft
spots, $n$ and $n-1$. Soft spot $n$ starts at state $0$ and switches to state
$1$ if the stress is increased to $\sigma_{n}^{+}$. Soft spot $n-1$ starts at
state $1$ and switches to state $0$ if the stress is decreased to
$\sigma_{n-1}^{-}$. When soft spot $n$ switches, it disables soft spot $n-1$
from switching due to an overlap. The site then reaches a new configuration
(box to the right) and a new soft spot $n+1$ that can switch due to an
increase in the stress is added to the site. Starting in this configuration,
soft spot $n$ can switch back from state $1$ to state $0$ if the stress is
decreased to $\sigma_{n}^{-}$ which then disables soft spot $n+1$ but releases
soft spot $n-1$ and brings the site back to the reference configuration.
Similarly, starting from the reference configuration and decreasing the stress
to $\sigma_{n-1}^{-}$, soft spot $n-1$ switches from $1$ to $0$, disables soft
spot $n$ and causes the creation of a new soft spot $n-2$ that can switch if
the strain is decreased further. The dynamics obtained from this scheme is
similar in nature to the single-cell multi-well dynamics used in recent
elastoplastic models Khirallah  _et al._ (2021); Kumar  _et al._ (2022); Liu
_et al._ (2022).

We next modify the model to include the possibility of irregular overlaps. To
allow for such overlaps, whenever the dynamics reaches a configuration which
was not visited before, we add a new soft spot to the site with probability
$p_{ir}$. This is illustrated in Fig. 2(b,bottom) where soft spot $\tilde{n}$
with a random $\sigma_{\tilde{n}}^{+}<\sigma_{n}^{+}$ is added to the cell.
This introduces irregular overlaps since it allows a soft spot to become
overlapped and disabled before switching back to its initial state. When the
system reaches a previously visited system-wide configuration, we do not allow
further soft spot additions (i.e. we set $p_{ir}=0$). This is compatible with
the deterministic nature of the dynamics in the atomistic simulations and
guarantees that once a previously visited state is reached, the system will
enter a limit cycle. Fig. 4 demonstrates how the dynamics is modified when an
irregular overlap is added at the reference configuration. Starting from
$\sigma_{k}$, the stress is increased and at $\sigma_{\tilde{n}}$ soft spot
$\tilde{n}$ switches from $0$ to $1$ which then disables soft spots $n$ and
$n-1$ and causes the creation of a new soft spot $\tilde{n}+1$ that can switch
upon a further increase in the stress. The system thus enters an alternative
path in the configuration space where soft spots $n$, $n+1$, $n+2$, $...$
cannot switch and soft spots $\tilde{n}$, $\tilde{n}+1$, $\tilde{n}+2$, $...$
are switched instead. Upon a decrease of the strain, the system can, in
principle switch $n-1$, $n-2$, $n-3$, $...$ back to their $0$ states. However,
there is a finite probability that another irregular overlap will occur before
any of these soft spots switch back, and in that case at least some of these
soft spots will not switch back, and alternative soft spots will switch
instead.

Figure 6:  The average number of cycles $\tau$ needed to reach a periodic
state as a function of $|\gamma_{\max}-\gamma_{c}|$ for $p_{ir}=0.01$.

##  IV Results

We first verifiy that the model reproduces the salient features of amorphous
solids as observed in experiments and atomistic simulations. In Fig. 5 we
compare the stress-strain curves obtained from the model 5(a) and atomistic
simulations 5(b). It is clear that qualitatively, the behavior is very
similar. In both cases there is observable hysteresis and in both cases large
avalanches become more frequent in the post-yield regime. Other properties of
the stress-strain response and avalanches are discussed in the supplementary
material. To study the contribution of irregular overlaps to irreversibility,
we compared simulations of the model with $p_{ir}=0$ (strictly regular
overlaps) and $p_{ir}=0.01$ (a finite but small probability for irregular
overlaps). In both cases, we simulated $100$ realizations of systems of
$N=250$ sites and studied the response to up to $2000$ forcing cycles. We
found that in both cases the response to monotonically increasing strain was
qualitatively identical (see Supplementary material), but the response to
cyclic shear was extremely different: when $p_{ir}=0$ the system reaches a
limit cycle for any forcing amplitude $\gamma_{\max}$ but when $p_{ir}=0.01$
the number of cycles $\tau$, needed to reach a limit cycle seem to diverge at
a critical amplitude $\gamma_{c}$ as shown in Fig. 6. Fitting to
$\tau\sim|\gamma-\gamma_{c}|^{\alpha}$ leads to a best fit for
$\gamma_{c}=0.97$ and $\alpha=4.65(1)$. The exponent $\alpha$ obtained is
larger than that observed in atomistic simulations, $\sim 2.6$, which could be
attributed to using random, mean-field interactions rather than realistic
quadrupolar interactions.

Figure 7:  (a) The number of new soft spots $\rho$ that are added in each
cycle as a function of the number of applied cycles $n$ for $p_{ir}=0$. The
same plot for $p_{ir}=0.01$. The data is normalized such that $\rho$ is one at
$n=0$ and
$\gamma_{\max}=0.4(\bigcirc),0.6({\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\square}),0.8({\color[rgb]{0,1,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,1,0}\triangle}),1.2({\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\lozenge})$.

To better understand the difference between dynamics with strictly regular
overlaps and dynamics that include irregular overlaps, we recorded the number
of new soft spots added at each cycle $\rho$, which is an indication of the
a-periodicity of the cycle. Fig. 7(a) shows that for the strictly regular
$p_{ir}=0$, the number of new soft spots per cycle decreases rapidly to zero,
irrespective of the driving amplitude $\gamma_{\max}$. However, when
$p_{ir}=0.01$ (Fig. 7(b)), there is a transition as a function of
$\gamma_{\max}$ between transient dynamics in which the number of new soft
spots decays to zero and dynamics in which the number of new soft spots
reaches a constant value. This is consistent with our understanding that an
irreversible steady state requires the number of soft spots to become
effectively infinite (note that $\rho=0$ does not mean that the system
immediately enters a limit cycle, but similarly to a system with a fixed
number of soft spots, it will eventually reach a limit cycle after a finite
number of forcing cycles).

The origin of the difference between dynamics that includes and excludes
irregular overlaps can be understood by studying the number of new soft spots
added per cycle $\rho$ as a function of the strain at which they are switched
$\gamma$, and the number of cycles applied $n$. In Fig. 8(a,b), we show the
average $\rho$ as a function of $n$ and $\gamma$ for a post-yield amplitude
$\gamma_{\max}=1.2$, starting from the second cycle (we are interested in the
transient distribution). When $p_{ir}=0$ (Fig. 8(a)), the number of new soft
spots rapidly decays to zero for most of the strain range except for two
“boundary layers” close to $\pm\gamma_{\max}$, and eventually, the number of
new soft spots decays to zero in the entire strain-range. These boundary
layers are also observed when $p_{ir}=0.01$ (Fig. 8(b)). However, in this
case, the number of new soft spots per cycle reaches a non-zero steady state.
The difference in the steady-state density between $p_{ir}=0$ and $p_{ir}>0$
stems from the fact that in the former, after the first forcing cycle, new
soft spots are added only in the vicinity of the boundaries of the strain
range (i.e., $\gamma=\pm\gamma_{\max}$) due to the hierarchical nature of
regular overlaps, whereas in the latter, new soft spots are also added at
intermediate strains due to the effect of irregular overlaps, which can occur
at any strain value. This allows the number of soft spots added per cycle to
reach a non-zero steady state in which new soft spots are continually added to
the system.

Figure 8: The number of new soft spots added in each cycle $\rho$, as a
function of the strain $\gamma$ and the cycle number $n$ starting from $n=2$,
averaged over $100$ realizations for (a) $p_{ir}=0$ and (b) $p_{ir}=0.01$.
Numerical solution of Eq. (1) as a function of $\gamma$ and $t$ with (c) $c=0$
and $r=-1$ (equivalent to regular dynamics) starting from $t=3$ and (d) $c=10$
and $r=0.5$ (equivalent to irregular dynamics) starting from $t=1$.

To better understand this behavior, we developed a model based on a mean-field
description of the dynamics, which is similar to approaches taken before to
describe the irreversibility transition in dilute systems and the effect of
jamming on such a transition Ness and Cates (2020); Xu and Schwarz (2013). The
primary assumption is that the number of new soft spots added at the $n+1$
cycle depends on how many new soft spots were added at the $n$th cycle. If no
new soft spots are added starting from some forcing cycle, the system
eventually reaches a limit cycle, as explained above. The dynamical variable
in our model is thus $\rho$, which represents the number of new soft spots
added at a time $t$ on a domain $-\gamma_{\max}\leq\gamma\leq\gamma_{\max}$.
The equation describing the change in $\rho$ is a nonlinear diffusion
equation:

$\rho_{t}=D\rho_{\gamma\gamma}+r\rho-c\rho^{2}\,,$ (1)

where the diffusive term $D\rho_{\gamma\gamma}$ takes into account random,
mean field interactions as was shown in other works Parley  _et al._ (2020).
These lead soft spots to switch at different strains in the next cycle (the
so-called scrambling van Hecke (2021)). The case $p_{ir}=0$ is modeled by
taking $c=0$ and $r<0$, which accounts for the hierarchical property of
regular overlaps that prevents the addition of new soft spots at intermediate
strains, which then causes $\rho$ to degrade over time. For $p_{ir}>0$, we
still assume that there is a degradation term proportional to $\rho$, but we
also add a term that takes into account the addition of new soft spots at
strains $-\gamma_{\max}<\gamma<\gamma_{\max}$ due to irregular overlaps. We
assume that the rate of soft spot addition is proportional to the current
number of new soft spots $\rho$, and the number of sites in which new soft
spots were not added at time $t$. This term is thus of the form $\rho(N-\rho)$
where $N$ is the total number of sites in the system. Together these terms
lead to the nonlinear part of Eq. (1). Note that this is the simplest
nonlinear term that leads to a transition between $\rho=0$ and a non-zero
steady-state and that this term is equivalent in form to the mean-field theory
of directed percolation Henkel  _et al._ (2008). To take into account the
addition of new soft spots at $\pm\gamma_{\max}$ (for any $p_{ir}$), we assign
nonlinear boundary conditions:

$\rho_{\gamma}\Bigr{|}_{\begin{subarray}{c}\gamma=\pm\gamma_{\max}\end{subarray}}=\pm[\rho-\rho^{2}]|_{\begin{subarray}{c}\gamma=\pm\gamma_{\max}\end{subarray}}\,,$
(2)

where we have chosen the simplest boundary conditions that allow for steady
states with either zero or non-zero $\rho$ starting from a non-zero $\rho$. In
Fig. 8(c) we show numerical solutions of Eq. (1) for $D=0.75$, $c=0$ and
$r=-1$ whereas in Fig. 8(d) we show the results for $D=0.75$, $c=10$ and
$r=0.5$. We can see that the former exhibits dynamics qualitatively similar to
Fig. 8(a) while the latter shows dynamics qualitatively similar to Fig. 8(b)
in support of our analysis (note that $\gamma_{\max}$ and the initial $\rho$
values are not numerically identical to the values in the discrete model). The
numerical solutions were obtained using the MATLAB package Chebfun Driscoll
_et al._ (2014).

##  V Conclusions

Our simulations and models indicate that soft-spot overlap provides a
mechanism that allows for entirely irreversible dynamics even in a system with
a relatively small number of particles, such as an atomistic simulation. As
the amplitude $\gamma_{\max}$ increases, the number of plastic events
occurring during a cycle increases, and system-spanning avalanches become more
frequent Regev  _et al._ (2015). This increases the probability of having at
least one new irregular overlap during each monotonic part of a forcing cycle,
which, as we saw above, prevents the system from reaching the same zero strain
state after a cycle. When this probability is large enough, the probability of
the system returning to a zero strain configuration after a cycle vanishes,
preventing it from ever reaching a periodic state. In future work, we plan to
extract $p_{ir}$ directly from atomistic simulations and quantify the role
played by avalanches in irregular overlaps and irreversibility. We would also
like to clarify the relationship between our overlap approach, existing
elastoplastic models, and strain compatibility Jagla (2007); Tyukodi  _et al._
(2018). We believe that overlaps between plastic events may play a role in
other mechanical systems, such as crumpled and corrugated sheets Bense and van
Hecke (2021); Shohat  _et al._ (2022), and in the emergence of multi-periodic
cycles as the irreversibility transition is approached Szulc  _et al._ (2022);
Keim and Paulsen (2021). For these reasons, we believe that overlaps between
plastic events will play an important role in understanding plasticity in
mechanical systems.

###### Acknowledgements.

Acknowledgements: IR would like to thank Muhittin Mungan for useful
discussions. AS and IR were supported by the Israel Science Foundation (ISF)
through Grant No. 1301/17 and AS was also supported by the Israel Science
Foundation (ISF) Grant no. 2300/19.

## References

  * Regev  _et al._ (2013) I. Regev, T. Lookman, and C. Reichhardt, Phys. Rev. E 88, 062401 (2013).
  * Keim and Arratia (2013) N. C. Keim and P. E. Arratia, Soft Matter 9, 6222 (2013).
  * Fiocco  _et al._ (2013) D. Fiocco, G. Foffi, and S. Sastry, Phys. Rev. E 88, 020301 (2013).
  * Priezjev (2013) N. V. Priezjev, Physical Review E 87, 052302 (2013).
  * Schreck  _et al._ (2013) C. F. Schreck, R. S. Hoy, M. D. Shattuck, and C. S. O’Hern, Physical Review E 88, 052205 (2013).
  * Bhaumik  _et al._ (2021) H. Bhaumik, G. Foffi, and S. Sastry, Proceedings of the National Academy of Sciences 118 (2021), 10.1073/pnas.2100227118.
  * Leishangthem  _et al._ (2017) P. Leishangthem, A. D. S. Parmar, and S. Sastry, Nature Communications 8 (2017), 10.1038/ncomms14653.
  * Reichhardt  _et al._ (2023) C. Reichhardt, I. Regev, K. Dahmen, S. Okuma, and C. Reichhardt, Physical Review Research 5, 021001 (2023).
  * Kawasaki and Berthier (2016) T. Kawasaki and L. Berthier, Physical Review E 94, 022615 (2016).
  * Keim  _et al._ (2019) N. C. Keim, J. D. Paulsen, Z. Zeravcic, S. Sastry, and S. R. Nagel, Reviews of Modern Physics 91, 035002 (2019).
  * Lavrentovich  _et al._ (2017) M. O. Lavrentovich, A. J. Liu, and S. R. Nagel, Phys. Rev. E 96, 020101 (2017).
  * Keim  _et al._ (2020) N. C. Keim, J. Hass, B. Kroger, and D. Wieker, Physical Review Research 2, 012004 (2020).
  * Fiocco  _et al._ (2015) D. Fiocco, G. Foffi, and S. Sastry, Journal of Physics: Condensed Matter 27, 194130 (2015).
  * Regev  _et al._ (2015) I. Regev, J. Weber, C. Reichhardt, K. A. Dahmen, and T. Lookman, Nature Communications 6 (2015), 10.1038/ncomms9805.
  * Sastry (2021) S. Sastry, Physical Review Letters 126, 255501 (2021).
  * Mungan and Sastry (2021) M. Mungan and S. Sastry, Physical review letters 127, 248002 (2021).
  * Szulc  _et al._ (2020) A. Szulc, O. Gat, and I. Regev, Phys. Rev. E 101, 052616 (2020).
  * Argon (1979) A. Argon, Acta Metallurgica 27, 47 (1979).
  * Falk and Langer (1998) M. L. Falk and J. S. Langer, Phys. Rev. E 57, 7192 (1998).
  * Maloney and Lemaître (2006) C. E. Maloney and A. Lemaître, Phys. Rev. E 74, 016118 (2006).
  * Manning and Liu (2011) M. L. Manning and A. J. Liu, Physical Review Letters 107, 108302 (2011).
  * Tyukodi  _et al._ (2019) B. Tyukodi, D. Vandembroucq, and C. E. Maloney, Physical Review E 100, 043003 (2019).
  * Mungan  _et al._ (2019) M. Mungan, S. Sastry, K. Dahmen, and I. Regev, Phys. Rev. Lett. 123, 178002 (2019).
  * Lindeman and Nagel (2021) C. W. Lindeman and S. R. Nagel, Science Advances 7 (2021), 10.1126/sciadv.abg7133.
  * Lundberg  _et al._ (2008) M. Lundberg, K. Krishan, N. Xu, C. S. O’Hern, and M. Dennin, Physical Review E 77, 041505 (2008).
  * Regev  _et al._ (2021) I. Regev, I. Attia, K. Dahmen, S. Sastry, and M. Mungan, Physical Review E 103, 062614 (2021).
  * Szulc  _et al._ (2022) A. Szulc, M. Mungan, and I. Regev, The Journal of Chemical Physics 156, 164506 (2022).
  * Keim and Paulsen (2021) N. C. Keim and J. D. Paulsen, Science Advances 7 (2021), 10.1126/sciadv.abg7685.
  * Khirallah  _et al._ (2021) K. Khirallah, B. Tyukodi, D. Vandembroucq, and C. E. Maloney, Physical Review Letters 126, 218005 (2021).
  * Kumar  _et al._ (2022) D. Kumar, S. Patinet, C. E. Maloney, I. Regev, D. Vandembroucq, and M. Mungan, The Journal of Chemical Physics 157, 174504 (2022).
  * Liu  _et al._ (2022) C. Liu, E. E. Ferrero, E. A. Jagla, K. Martens, A. Rosso, and L. Talon, J. Chem. Phys. (2022), 10.1063/5.0079460.
  * Lin  _et al._ (2014a) J. Lin, E. Lerner, A. Rosso, and M. Wyart, Proceedings of the National Academy of Sciences 111, 14382 (2014a).
  * Mungan and Terzi (2019) M. Mungan and M. M. Terzi, Annales Henri Poincaré 20, 2819 (2019).
  * Terzi and Mungan (2020) M. M. Terzi and M. Mungan, Phys. Rev. E 102, 012122 (2020).
  * Nicolas  _et al._ (2018) A. Nicolas, E. E. Ferrero, K. Martens, and J.-L. Barrat, Reviews of Modern Physics 90, 045006 (2018).
  * Bense and van Hecke (2021) H. Bense and M. van Hecke, Proceedings of the National Academy of Sciences 118, e2111436118 (2021).
  * van Hecke (2021) M. van Hecke, Phys. Rev. E 104, 054608 (2021).
  * Hébraud and Lequeux (1998) P. Hébraud and F. Lequeux, Physical Review Letters 81, 2934 (1998).
  * Bocquet  _et al._ (2009) L. Bocquet, A. Colin, and A. Ajdari, Physical Review Letters 103, 036001 (2009).
  * Lin  _et al._ (2014b) J. Lin, A. Saade, E. Lerner, A. Rosso, and M. Wyart, EPL (Europhysics Letters) 105, 26003 (2014b).
  * Ness and Cates (2020) C. Ness and M. E. Cates, Physical review letters 124, 088004 (2020).
  * Xu and Schwarz (2013) S.-L.-Y. Xu and J. Schwarz, Physical Review E 88, 052130 (2013).
  * Parley  _et al._ (2020) J. T. Parley, S. M. Fielding, and P. Sollich, Physics of Fluids 32 (2020).
  * Henkel  _et al._ (2008) M. Henkel, H. Hinrichsen, and S. Lübeck,  _Non-Equilibrium Phase Transitions: Volume I: Absorbing Phase Transitions_ (Springer, 2008).
  * Driscoll  _et al._ (2014) T. A. Driscoll, N. Hale, and L. N. Trefethen, “Chebfun guide,” (2014).
  * Jagla (2007) E. Jagla, Physical Review E 76, 046119 (2007).
  * Tyukodi  _et al._ (2018) B. Tyukodi, D. Vandembroucq, and C. E. Maloney, Physical review letters 121, 145501 (2018).
  * Shohat  _et al._ (2022) D. Shohat, D. Hexner, and Y. Lahini, Proceedings of the National Academy of Sciences 119 (2022), 10.1073/pnas.2200028119.


# Designing a Sequential Recommendation System for Heterogeneous Interactions
Using Transformers

Mehdi Soleiman Nejad, Meysam Varasteh, Hadi Moradi and Mohammad Amin Sadeghi
School of Electrical and Computer Engineering
University of Tehran
Tehran, Iran

<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>

###### Abstract

While many production-ready and robust algorithms are available for the task
of recommendation systems, many of these systems do not take the order of
user’s consumption into account. The order of consumption can be very useful
and matters in many scenarios. One such scenario is an educational content
recommendation, where users generally follow a progressive path towards more
advanced courses. Researchers have used RNNs to build sequential
recommendation systems and other models that deal with sequences. Sequential
Recommendation systems try to predict the next event for the user by reading
their history. With the massive success of Transformers in Natural Language
Processing and their usage of Attention Mechanism to better deal with
sequences, there have been attempts to use this family of models as a base for
a new generation of sequential recommendation systems. In this work, by
converting each user’s interactions with items into a series of events and
basing our architecture on Transformers, we try to enable the use of such a
model that takes different types of events into account. Furthermore, by
recognizing that some events have to occur before some other types of events
take place, we try to modify the architecture to reflect this dependency
relationship and enhance the model’s performance.

###### Index Terms:

Recommendation Systems, Transformers, Deep Learning

##  I Introduction

With the massive amount of information available to us in the snap of a
finger, having a system that finds useful and relatable content for us is of
paramount importance. Recommender systems have been trying to predict the
user’s interests by using the information available to us, mostly in the form
of likes and dislikes by the user [1]. While many of these systems don’t
account for the change of taste users have over the course of time, there has
been a push for modeling dynamic sequential behavior in newer recommender
systems thanks to the progress made in Deep Learning [2] [3] [4] [5] and other
fields [6]. There are many instances where the order of interaction with items
should matter to make a good recommendation. One of the areas the users might
benefit greatly from this sequential modeling is educational content. Due to
the progressive nature of many courses where certain topics are covered by
them with different levels of complexity, the order of the consumption by the
user matters. For example, while ”Introduction to Programming” and ”Advanced
Programming” are highly correlated courses and generally draw the same
audience of users, it is unwise to recommend ”Advanced Programming” to a user
that has not passed ”Introduction to Programming”. The user would be confused
when taking the course, because many topics from the other course are needed
to fully understand the advanced material, and it is likely the user will
leave the course before finishing it, thus lowering their engagement with the
platform. Similarly, we should not recommend ”Introduction to Programming” to
a user that has passed ”Advanced Programming”. It is unlikely that there are
any new material in the course for this user, as they have passed the more
advanced course.

To model such behavior, various methods have been proposed [7] [6], focusing
on predicting the user’s next interaction given their interaction history.
Many of these methods integrated Recurrent Neural Networks (RNNs) such as [8]
[9], [10]. These methods try to capture the essence of a user’s experience
into a hidden state vector and predict the next most likely item with said
vector. While these methods have been effective in tackling such modeling,
with the rise of the Transformers family [11] and specially BERT [12] breaking
several records in Natural Language Processing (NLP) tasks, researchers have
focused on using this family of deep learning models to accommodate tasks that
are not traditionally aligned with NLP. A good example would be a recommender
system designed with the mechanisms introduced by the Transformers family [13]
[14] [15] [16]. Furthermore, tech heavyweights Alibaba have also adopted
Transformers for recommendation recently [14]. The main idea behind using
Transformers for the task of recommendation is to create sequences from the
interactions each user has with items, and then use the same mechanisms to
predict the next item that is most likely to appear in the sequence. We can
think of these sequences as sentences and each item in the sequence as a word.

While these models are sophisticated and out-perform their recurrent peers,
they are often designed to work with homogeneous events. For example, [13]
completely disregards the negative reviews and only uses positive reviews to
form a profile for each user and uses this profile to predict the next item
the user will like. This limitation makes the model an unideal solution for
scenarios where predicting other events can also be helpful. We want to have a
model that can consider different types of interactions as well. These events
can include typical user interactions such as leaving a positive or negative
review, registering for a course, or passing it. It is also possible for these
events to be dependent on each other. For example, users can’t review a course
they have not registered for in many learning platforms, so the events can
also form a sequential order. This means we will have to train a model for
each type of event if we want to accommodate different events into our
recommendation. Another direction for improvement that we want to take is to
go beyond simple hyperparameter tuning and slightly modify the architecture to
accommodate specific properties we introduce to our input via preprocessing.

In this work, We first use a simple technique to create heterogeneous
sequences of events that follow certain grammar-like rules for each user from
the user-item interaction matrix. We then modify BERT by altering the
architecture to accommodate the grammar of our synthetic language that
represents our data. This means that the model will have the structural
limitations of our synthetic language as some implicit hyperparameter, and
there is no need to infer them from the dataset and learn them during the
training. This allows the model to use its learning capacity to infer other
useful information from the dataset. Similar to previous models, we also use
the Masked Language (Cloze Test) task to train and evaluate our model.
Ultimately, we measure how much our modifications improve the results compared
to the original architecture and then compare our model to other sequential
models. Due to the method of our transformation of data into sequences, our
proposed model will also be able to predict the type of events for certain
items in addition to the item related to the event.

The rest of this paper is organized into four sections. In section II, we
briefly review related works. In section III, we present our model with
details. In section IV, we evaluate our proposed model on an online learning
platform’s dataset and compare it with other models. Ultimately, in section V
we conclude our work with pointers for future directions.

##  II Related Work

In this section, we discuss some previous work done in recommender systems
briefly.

Recommendation systems have been a field of interest for researchers since the
’90s. Many researchers have opted to study Collaborative Filtering (CF) [17],
a family of methods that tries to learn each user’s preferences by finding
similarities between users and items using the User-Item Matrix. In this
matrix, each row represents a user and each column an item. Since it’s
unlikely that users interact with the majority of items, this matrix is
sparse. The most popular CF method is Matrix Factorization [18] [19], in which
the algorithm tries to break the matrix into two separate matrices, one for
representing a hidden state for each user and one for items. Due to the rise
of Deep Learning, there has been immense progress in creating better
recommender systems [20] [21] [22] [23]. The progress of deep learning meant
that it was now possible to create more complex recommender systems that can
use different sources of data, such as text [24], images [25] or sound [26]
[27] to find the preferences each user has. In addition to the advanced in
using deep learning for a recommendation, some methods also use hybrid
techniques to utilize different models[28] better.

Generally speaking, CF methods have no sequential nature. They do not consider
the order of interaction when predicting an item’s score, and they treat every
given score as if they are given at the same time. As argued previously, the
order of user-item interaction can be good information to have on certain use
cases. Markov Chains (MCs) [29], Factorization Machines [30] [31] and
Recurrent Neural Networks (RNNs) [10] are good choices for dealing with
sequential data and there has been successful efforts to use such
architectures to model a sequential recommender systems. Researchers have used
MCs to create a graph to model the user’s sequential behavior and combine this
with other means to predict the next item. Meanwhile, the main idea in RNN-
based recommender systems is to convert a user’s history into a hidden state
vector and use that vector to predict the next item. Some architectures use
Convolutional Neural Networks (CNNs) to model the sequence [4] [32] where the
sequence is thought of as some image that we have to extract features. Apart
from trying new architectures for better sequential recommendations, there has
also been researching for providing model-agnostic frameworks for sequential
recommendation [33]. After the introduction of Attention mechanism and
Transformer family [11] and later on BERT [12] by Google, the research team
behind BERT managed to break several records in NLP. This major breakthrough
in NLP inspired other researchers to use the new techniques in other fields
that might benefit from modeling sequences.

While BERT itself is a pre-trained model for language processing that can be
fine-tuned to certain datasets or tasks, the architecture can be used in
different scenarios like sequential recommendation systems. SASRec [7], BST
[14], BERT4Rec [13] and S3-Rec [15] are some of the better known models to
utilize Attention mechanism and ultimately Transformers for sequential
recommendation.

##  III Architecture

Figure 1: The overview of the proposed model

In this section we define our proposed architecture to design a sequential
recommender system that deals with heterogeneous item history.

###  III-A Problem Statement

In our task, suppose $U=\\{{u_{1},u_{2},...,u_{N}}\\}$ is the set of the
users, $I=\\{i_{1},i_{2},...,i_{M}\\}$ is the set of the items and also
$E=\\{e_{1},e_{2},...,e_{K}\\}$ is the set of possible events and actions that
each user might perform on an item. We acknowledge that certain events might
only happen after another certain event has taken place. For example, it is
common practice to only let users who have bought the product to leave a
review on it. We define $D(u,\hat{u})$ as a function that defines this
relation between our events. We then define
$S(u)=\\{(e,i)_{1},(e,i)_{2},...,(e,i)_{l}\\}$ where $u\in U$, $e\in E$ and
$i\in I$ as the interaction history of user $u$. These interactions are sorted
by their time of occurrence. Given the interaction history $S(u)$ and either
$e_{l+1}$ or $i_{l+1}$, our recommender system should predict the missing half
of event $l+1$. The proposed model is shown in fig. 1.

###  III-B Learning

####  III-B1 Train/Test Data Split

The data split we perform a bit different from the traditional methods. For
each user, we save the last item in their sequence $S(u)$ as the test item and
use the rest during training. This means that any user with at least two
different interactions in their history is used in the training process.

####  III-B2 Training

We use the Masked Language task to train our model. In this task, some of the
tokens in the sequence are randomly masked, and the model predicts the missing
token. An example of this is shown in fig. 2. In this example, the model
should fill in the space denoted by $<mask>$ with $X_{2}$. This is done by
adding an output layer (also known as a head layer) that maps the
transformer’s output to a distribution over all possible tokens using a fully
connected neural network and applying Softmax. We do many iterations and
randomly mask tokens again each time, so all tokens get a fair chance to be
masked and impact the tunable parameters.

Figure 2: Masked Language example

####  III-B3 Testing

We can do two types of testing on our model. In the first type, we give the
user history and the event type $e_{l+1}$ and mask the item ID. The model will
then predict the appropriate item ID for the given event type and history. We
can output a list of candidates to the model that has given a high score. In
the second type, we mask the item ID $i_{l+1}$ instead, and the model should
predict the appropriate event type for the item.

###  III-C Data Preprocessing

Figure 3: The structure for each user sequence

While most user-item interaction datasets are usually tabular, to feed our
data to BERT, we need to convert the data into sequences for each user. We
present a simple technique to do this transformation. For each user, we gather
all interactions and sort them by the time of their occurrence. We treat each
interaction as an event. We decide on unique symbols for each type of event,
such as like or dislike. We then concatenate the event’s symbol with the
item’s ID to represent this interaction. This makes each of our sequences
follow the structure shown in fig. 3 where $t_{i}$ is the type and $ID_{i}$ is
the associated item ID for event $i$. We then create a tokenizer that
separates $t_{i}$ from $ID_{i}$ and converts the user-item history to a
sequence of integers, ready to be given as input to the embedding layer. This
means that we have effectively transformed $S(u)$ into
$\\{e_{1},i_{1},e_{2},i_{2},...,e_{l},i_{l}\\}$, where each $t_{n}$ is
representing $e_{n}$ and each $ID_{n}$ is representing $i_{n}$.

###  III-D Embedding Layer

Our first component is the embedding layer, which projects each part of our
sequence to fixed-size vectors. These vectors represent their corresponding
tokens and are fed into the next layer. In [11], the writers have suggested
Positional Embedding as their choice for this layer. That means for a given
item $i_{k}$, we will use the following representation:

$h_{i}=v_{i}+p_{i}$ (1)

where $v_{i}$ is the fixed-dimensional embedding of $i_{k}$ and $p_{i}$ is the
positional embedding of $i_{k}$ and of same dimensionality of $v_{i}$.

This choice is to reflect the order of the items in the projections. The
writers notice that there is minimal difference between using a fixed sinusoid
implementation or a tunable layer. We opt to use a tunable layer for
convenience, but we leave two elements from the vector out of the training
process and set them manually. These two elements each emphasize a different
structural limit in of $S(u)$. As stated in the previous section, if we break
$S(u)$ into $\\{e_{1},i_{1},e_{2},i_{2},...,e_{l},i_{l}\\}$, we quickly notice
that the type of tokens in all the odd positions are different from the type
of tokens in the even positions. That is, we only have event types in odd
positions and item IDs in even positions. We use one of the elements to
reflect this fact and label each token as either an event or an item. This
will help the model realize the difference quickly.

$P_{type}(i)=\begin{cases}0,&\text{if $i=2k$}\\\ 1,&\text{if
$i=2k+1$}\end{cases}$ (2)

After breaking the tuples in $S(u)$, we have to find a new way to show the
relationship between each item ID and event type. Using another element and
setting it to the same value for the related event type and item ID token, we
label them together. The model will have a strong clue to figure out the
relationship between each event type and item ID.

$P_{relevance}(i)=\begin{cases}sin(\omega_{k}.i),&\text{if
$\left\lfloor{\frac{i}{2}}\right\rfloor=2k$}\\\ cos(\omega_{k}.i),&\text{if
$\left\lfloor{\frac{i}{2}}\right\rfloor=2k+1$}\end{cases}$ (3)

This slight change helps the model to be more accurate by effectively labeling
each token of the sequence.

###  III-E Transformer Layer

Figure 4: The Encoder layer

The second component is the Transformer layer, which learns a deeper context
for each token by analyzing its relationship with other tokens in the
sequence. This happens by stacking Encoder modules on top of each other. The
structure of each Encoder is shown in fig. 4.

The input to this component is $h_{i}$ produced by the embedding layer. The
input for each layer $l$ is denoted by $h^{l}_{i}$. By stacking these hidden
vectors, we create the matrix $H^{l}$ for denoting the input for each layer.
Each Encoder consists of two main parts, which, as shown in fig
LABEL:fig:_Encoder we use with residual connections followed by a
normalization layer. 1) Multi-Headed Attention and 2) Point-Wise Feed Forward.

####  III-E1 Multi-Headed Attention

The attention mechanism is at the heart of the Transformer architecture and is
regularly used to capture relationships and dependencies in sequences,
regardless of their relative distance to each other. Attention is calculated
as follows:

$Att(Q,K,V)=softmax(\frac{QK^{T}}{\sqrt{d}})V$ (4)

Where $Q$ is the query matrix, $K$ is the key matrix, $V$ is the value matrix,
and $d$ is the dimension of each key vector. These matrices are linear
projections from the original $h_{i}$ produced by the previous layer. As
discussed in [11], it is beneficial to use different representation spaces
together instead of just using one attention module. We’ll adopt the Multi-
Headed Attention mechanism:

$MHA(H)=concat(head_{1}(H),...,head_{n}(H))W^{MH}$ (5)
$head_{i}(H)=Att(HW^{Q}_{i},HW^{K}_{i},HW^{V}_{i})$ (6)

where $W^{MH}$, $W^{Q}_{i}$ , $W^{K}_{i}$ and $W^{V}_{i}$ are tunable
parameters.

####  III-E2 Point-Wise Feed Forward

Since all the attention operations are linear projections, we need to
introduce non-linearity to enhance the model. We do this by using adding a
point-wise feed-forward layer on top of our multi-head attention layer.

$PWFF=concat(FF(h_{1}^{l}),...,FF(h_{n}^{l}))$ (7)
$FF(h)=GeLU(hW^{(1)}+b^{(1)})W^{(2)}+b^{(2)}$ (8)

where $W^{(1)}$, $W^{(2)}$ , $b^{(1)}$ and $b^{(2)}$ are tunable parameters
shared across different points. Following BERT [12], we opt to use GeLU as our
activation function instead of ReLU.

####  III-E3 Stacking Encoders

By stacking encoders on top of each other and feeding the output of the
previous Encoder to the next layer, we introduce further complexity to our
model. We use dropout at the output of each layer to avoid overfitting and
introduce generalization to our model. The output of the last Encoder is used
as input for our final component, the output layer.

###  III-F Output layer

The last component of our architecture is the output layer. This layer is a
fully connected neural network that maps the context vector for the masked
items over target items or events. Using a two-layer fully connected neural
network, we give each token a score for every masked item. As noted before,
since some events can be dependent on other events, we utilize this knowledge
to boost our model’s accuracy by limiting the number of possible candidates.
If the masked token is an item ID (i.e., odd-positioned), we check if the
event type has any dependency. If there is a dependency on another event type,
we increase the score for all the item IDs with said dependency by 1. This
allows our model to differentiate between irrelevant and valuable items and
enhance our model. We then apply the Softmax function over the scores to
distribute the target item IDs and event types. Ultimately, the token with the
highest probability will be chosen as the prediction for the masked item.

##  IV Experiment

This section talks about our dataset, the metrics used for experimenting, and
our configuration for the model. We then measure the impact of our proposed
changes in helping the Transformer model predict the next item. Ultimately, we
compare our proposed models against other models.

###  IV-A Dataset

One of the best examples of sequential behavior is online learning platforms
and digital courses due to the sequential nature of learning topics. The
dataset used for this paper is from an online learning platform. In this
dataset, users register for courses and either like the course or dislike it.
We use $r$, $+$ and $-$ as tokens to represent these events respectively. We
know that liking and disliking a course is also dependent on registering for
the course first. After preprocessing the dataset, there are about 47000 users
available to us. The total number of courses is 630.

###  IV-B Evaluation Metrics

As previously stated, there are two types of tests we can perform using our
model. In the first type where the model predicts the item ID, we use HR@1
(Hit Ratio), HR@5, HR@10, NDCG@5 (Normalized Discounted Cumulative Gain)
NDCG@10. For our specific task, the sequential recommendation, NDCG@1, and
HR@1 are equal, so we only report HR@1. We also use these metrics to compare
our model with other models. In the second type, where the model has to
predict the event type, we use precision, recall, and F1-score and treat the
test as a classification since there are three types of events. We’ll create a
4th class that does not have any correct labels. We use this 4th class to
label anything the model produces that is not associated with the types.

###  IV-C Configuration

TABLE I: Performance comparison of different models Metric |  POP |  End-To-End BERT |  SASRec |  BERT4Rec |  Proposed Model |  Improvement
---|---|---|---|---|---|---
HR@1 |  0.029 |  0.086 |  0.111 |  0.135 |  0.150 |  11.1%
HR@5 |  0.099 |  0.320 |  0.464 |  0.303 |  0.512 |  10.3%
HR@10 |  0.152 |  0.489 |  0.564 |  0.521 |  0.624 |  5.9%
NDCG@5 |  0.063 |  0.208 |  0.311 |  0.220 |  0.344 |  10.6%
NDCG@10 |  0.080 |  0.251 |  0.356 |  0.255 |  0.378 |  6.1%

For this experiment, we’ll use eight heads for the attention module and stack
two layers of encoders on top of each other. Our context vectors’ length will
be 96. The output layer is set to be 128. We mask $p=0.4$ of the tokens
randomly and rerun the training algorithm for 50 epochs with batch sizes of
256. Other models will also feature similar complexity for a fair comparison.
We use the excellent implementations provided by HuggingFace using PyTorch as
our base and apply our modifications to their code. Ultimately, we train our
model on a machine using a single Nvidia GeForce GTX 1070 GPU fully from
scratch, without any pretraining.

###  IV-D Performance Comparison

We compare our models to the following baselines:

  * •

POP: This naive method ranks items according to their popularity and always
recommends them to each and every user.

  * •

End-To-End BERT [12]: We use our preprocessed data to train BERT from scratch
without any pretraining and only using our custom tokenizer. This means that
none of our proposed changes are applied, and we only use our heterogeneous
data.

  * •

SASRec [7]: This method uses a unidirectional Transformer model to predict the
next interaction with items.

  * •

BERT4Rec [13]: This method uses a bidirectional Transformer model, BERT, to
predict the next interaction with items.

With the sole exception of POP, the other methods use Transformers in their
architecture. Table I shows the scores across different metrics for these
models, our proposed model, and the improvement our model has achieved over
the best performing alternative. In each row, the best score for alternative
models is underlined. As is evident in table I, POP is predictably the worst
performer among the models due to a lack of personalization, while the best
performance among alternative models belongs to SASRec, only losing to
BERT4Rec when measured using HR@1. Compared to the best performing model in
each measurement, our model sees an improvement close to 10% and 6% when the
output list size is 5 and 10, respectively. We see the most significant
improvement for HR@1 with 11.1% and the slightest improvement for HR@10 with
5.9%.

###  IV-E Effect of custom positional encoding in event type classification

TABLE II: Performance comparison of different models |  W/ custom encoding |  W/o custom encoding |  Improvement
---|---|---|---
Precision |  0.685 |  0.678 |  1.0%
Recall |  0.577 |  0.555 |  3.9%
F-1 Score |  0.585 |  0.565 |  3.5%

We now study the effect of our proposed positional encoding on correctly
classifying the event type for each event. To study this, we train our model
once with our proposed positional encoding and once with completely tunable
positional encoding from scratch, without any pretraining and any changes over
other parameters. We then use the trained models to predict the event type of
the next interaction. Table II shows the results of this experiment. As
evident, precision does not improve much but recall, and F-1 scores are
improved 3.9% and 3.5% respectively.

##  V Conclusion and Future Work

This paper presented a method for using the Transformer family of models for
sequential recommendation systems. Instead of focusing on only one type of
feedback from the user, such as likes, by simply adding a token to
differentiate between events, we provide a framework that allows other types
of events to be taken into account. We then try to exploit the fact that many
events can only happen if a particular event has happened before them in real
life, such as a user having the ability to positively or negatively review an
item only after ordering that specific item. By modifying the structure of the
model to factor in this rule and event types, we introduce a model that
outperforms other models with similar complexity. While our model does take
order into order, the temporal distance between events is not considered.
Another direction for improvement could be factoring in more complex relations
between events, such as multiple dependencies or mutual exclusion. We can also
leverage Transformers in more complex systems to better encode each user and
what they represent to the system instead of using them directly to predict
the next item.

## Acknowledgment

We would like to thank Maktabkhooneh, the online learning platform, for their
collaboration and providing the dataset to us.

## References

  * [1] P. Resnick and H. R. Varian, “Recommender systems,” Commun. ACM, vol. 40, p. 56–58, Mar. 1997.
  * [2] H.-T. Cheng, L. Koc, J. Harmsen, T. Shaked, T. Chandra, H. Aradhye, G. Anderson, G. Corrado, W. Chai, M. Ispir, R. Anil, Z. Haque, L. Hong, V. Jain, X. Liu, and H. Shah, “Wide and deep learning for recommender systems,” in Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, DLRS 2016, (New York, NY, USA), p. 7–10, Association for Computing Machinery, 2016.
  * [3] P. Covington, J. Adams, and E. Sargin, “Deep neural networks for youtube recommendations,” in Proceedings of the 10th ACM Conference on Recommender Systems, RecSys ’16, (New York, NY, USA), p. 191–198, Association for Computing Machinery, 2016.
  * [4] J. Tang and K. Wang, “Personalized top-n sequential recommendation via convolutional sequence embedding,” in Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, WSDM ’18, (New York, NY, USA), p. 565–573, Association for Computing Machinery, 2018.
  * [5] X. Chen, H. Xu, Y. Zhang, J. Tang, Y. Cao, Z. Qin, and H. Zha, “Sequential recommendation with user memory networks,” in Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, WSDM ’18, (New York, NY, USA), p. 108–116, Association for Computing Machinery, 2018\.
  * [6] S. Wang, L. Hu, Y. Wang, L. Cao, Q. Z. Sheng, and M. Orgun, “Sequential recommender systems: Challenges, progress and prospects,” in Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pp. 6332–6338, International Joint Conferences on Artificial Intelligence Organization, 7 2019.
  * [7] W. Kang and J. J. McAuley, “Self-attentive sequential recommendation,” in IEEE International Conference on Data Mining, ICDM 2018, Singapore, November 17-20, 2018, pp. 197–206, IEEE Computer Society, 2018.
  * [8] T. Donkers, B. Loepp, and J. Ziegler, “Sequential user-based recurrent neural network recommendations,” in Proceedings of the Eleventh ACM Conference on Recommender Systems, RecSys ’17, (New York, NY, USA), p. 152–160, Association for Computing Machinery, 2017.
  * [9] B. Hidasi, A. Karatzoglou, L. Baltrunas, and D. Tikk, “Session-based recommendations with recurrent neural networks,” in 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings (Y. Bengio and Y. LeCun, eds.), 2016\.
  * [10] B. Hidasi and A. Karatzoglou, “Recurrent neural networks with top-k gains for session-based recommendations,” in Proceedings of the 27th ACM International Conference on Information and Knowledge Management, CIKM ’18, (New York, NY, USA), p. 843–852, Association for Computing Machinery, 2018.
  * [11] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems (I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, eds.), vol. 30, Curran Associates, Inc., 2017.
  * [12] J. Devlin, M. Chang, K. Lee, and K. Toutanova, “BERT: pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers) (J. Burstein, C. Doran, and T. Solorio, eds.), pp. 4171–4186, Association for Computational Linguistics, 2019.
  * [13] F. Sun, J. Liu, J. Wu, C. Pei, X. Lin, W. Ou, and P. Jiang, “Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer,” in Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM ’19, (New York, NY, USA), p. 1441–1450, Association for Computing Machinery, 2019.
  * [14] Q. Chen, H. Zhao, W. Li, P. Huang, and W. Ou, “Behavior sequence transformer for e-commerce recommendation in alibaba,” in Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data, DLP-KDD ’19, (New York, NY, USA), Association for Computing Machinery, 2019\.
  * [15] K. Zhou, H. Wang, W. X. Zhao, Y. Zhu, S. Wang, F. Zhang, Z. Wang, and J.-R. Wen, “S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization,” in Proceedings of the 29th ACM International Conference on Information and Knowledge Management, CIKM ’20, (New York, NY, USA), p. 1893–1902, Association for Computing Machinery, 2020\.
  * [16] L. Wu, S. Li, C.-J. Hsieh, and J. Sharpnack, “Sse-pt: Sequential recommendation via personalized transformer,” in Fourteenth ACM Conference on Recommender Systems, RecSys ’20, (New York, NY, USA), p. 328–337, Association for Computing Machinery, 2020.
  * [17] J. B. Schafer, D. Frankowski, J. Herlocker, and S. Sen, Collaborative Filtering Recommender Systems, pp. 291–324.  Berlin, Heidelberg: Springer Berlin Heidelberg, 2007.
  * [18] A. Mnih and R. R. Salakhutdinov, “Probabilistic matrix factorization,” in Advances in Neural Information Processing Systems (J. Platt, D. Koller, Y. Singer, and S. Roweis, eds.), vol. 20, Curran Associates, Inc., 2008.
  * [19] P. O. Hoyer, “Non-negative matrix factorization with sparseness constraints,” J. Mach. Learn. Res., vol. 5, p. 1457–1469, Dec. 2004.
  * [20] X. He, L. Liao, H. Zhang, L. Nie, X. Hu, and T.-S. Chua, “Neural collaborative filtering,” in Proceedings of the 26th International Conference on World Wide Web, WWW ’17, (Republic and Canton of Geneva, CHE), p. 173–182, International World Wide Web Conferences Steering Committee, 2017.
  * [21] X. Li and J. She, “Collaborative variational autoencoder for recommender systems,” in Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’17, (New York, NY, USA), p. 305–314, Association for Computing Machinery, 2017.
  * [22] T. Xiao, S. Liang, W. Shen, and Z. Meng, “Bayesian deep collaborative matrix factorization,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 5474–5481, Jul. 2019.
  * [23] M. Varasteh, M. S. Nejad, H. Moradi, M. A. Sadeghi, and A. Kalhor, “An improved hybrid recommender system: Integrating document context-based and behavior-based methods,” CoRR, 2021.
  * [24] D. Kim, C. Park, J. Oh, S. Lee, and H. Yu, “Convolutional matrix factorization for document context-aware recommendation,” in Proceedings of the 10th ACM Conference on Recommender Systems, RecSys ’16, (New York, NY, USA), p. 233–240, Association for Computing Machinery, 2016.
  * [25] W. Kang, C. Fang, Z. Wang, and J. J. McAuley, “Visually-aware fashion recommendation and design with generative image models,” in 2017 IEEE International Conference on Data Mining, ICDM 2017, New Orleans, LA, USA, November 18-21, 2017 (V. Raghavan, S. Aluru, G. Karypis, L. Miele, and X. Wu, eds.), pp. 207–216, IEEE Computer Society, 2017.
  * [26] A. van den Oord, S. Dieleman, and B. Schrauwen, “Deep content-based music recommendation,” in Advances in Neural Information Processing Systems (C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, eds.), vol. 26, Curran Associates, Inc., 2013.
  * [27] M. Schedl, “Deep learning in music recommendation systems,” Frontiers in Applied Mathematics and Statistics, vol. 5, p. 44, 2019.
  * [28] R. E. Nakhli, H. Moradi, and M. A. Sadeghi, “Movie recommender system based on percentage of view,” in 2019 5th Conference on Knowledge Based Engineering and Innovation (KBEI), pp. 656–660, 2019.
  * [29] S. Rendle, C. Freudenthaler, and L. Schmidt-Thieme, “Factorizing personalized markov chains for next-basket recommendation,” in Proceedings of the 19th International Conference on World Wide Web, WWW ’10, (New York, NY, USA), p. 811–820, Association for Computing Machinery, 2010.
  * [30] N. Wen and F. Zhang, “Extended factorization machines for sequential recommendation,” IEEE Access, vol. 8, pp. 41342–41350, 2020.
  * [31] R. Pasricha and J. McAuley, “Translation-based factorization machines for sequential recommendation,” in Proceedings of the 12th ACM Conference on Recommender Systems, RecSys ’18, (New York, NY, USA), p. 63–71, Association for Computing Machinery, 2018.
  * [32] T. X. Tuan and T. M. Phuong, “3d convolutional networks for session-based recommendation with content features,” in Proceedings of the Eleventh ACM Conference on Recommender Systems, RecSys ’17, (New York, NY, USA), p. 138–146, Association for Computing Machinery, 2017.
  * [33] S. Zhang, D. Yao, Z. Zhao, T.-S. Chua, and F. Wu, “Causerec: Counterfactual user sequence synthesis for sequential recommendation,” in Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’21, (New York, NY, USA), p. 367–377, Association for Computing Machinery, 2021.