smLaTeX-Literature

# Nonlinear random walks optimize the trade-off between cost and prevention in
epidemics lockdown measures : the ESIR model

Bram A. Siebert  MACSI, Department of Mathematics and Statistics, University
of Limerick, Limerick V94 T9PX, Ireland  James P. Gleeson  MACSI, Department
of Mathematics and Statistics, University of Limerick, Limerick V94 T9PX,
Ireland  Malbor Asllani  School of Mathematics and Statistics, University
College Dublin, Belfield, Dublin 4, Ireland

###### Abstract

Contagious diseases can spread quickly in human populations, either through
airborne transmission or if some other spreading vectors are abundantly
accessible. They can be particularly devastating if the impact on individuals’
health has severe consequences on the number of hospitalizations or even
deaths. Common countermeasures to contain the epidemic spread include
introducing restrictions on human interactions or their mobility in general
which are often associated with an economic and social cost. In this paper, we
present a targeted model of optimal social distancing on metapopulation
networks, named $ESIR$ model, which can effectively reduce the disease
spreading and at the same time minimize the impact on human mobility and
related costs. The proposed model is grounded in a nonlinear random walk
process that considers the finite carrying capacity of the network’s
metanodes, the physical patches where individuals interact within mobility
networks. This later constrain is modeled as a _slack compartment_ $E$ for the
classic $SIR$ model and quantifies the density of vacant spaces to accommodate
the diffusing individuals. Formulating the problem as a multi-objective
optimization problem shows that when the walkers avoid crowded nodes, the
system can rapidly approach Pareto optimality, thus reducing the spreading
considerably while minimzing the impact on human mobility as also validated in
empirical transport networks. These results envisage _ad hoc_ mobility
protocols that can potentially enhance policy making for pandemic control.

##  I Introduction

Epidemics have affected humankind many times during history with countless
casualties and enormous economic and social impact. For example, the notorious
Black Death (1331–1353) caused 75–200 million deaths worldwide and led to the
loss of life of approximately one third of Europe’s population [1]. At the
time of writing, the COVID-19 pandemic has drastically influenced our lives
and affected our well-being in many aspects, creating a real challenge in
finding a compromise between curbing the spread of the virus and allowing an
adequate level of normality in human activity. Human behavior is one the
driving factors of epidemic spreading and thus has long been the focus of
scientists, governments, and public health officials [2, 3]. Many attempts
have been made to model measures taken by the public, such as social
distancing [4, 5]. The recent Coronavirus pandemic has reawakened the
necessity for a thorough understanding of how to control the spread of a
highly contagious disease, with many novel studies emerging recently [6, 7,
8]. Many governments have taken measures to slow down the spread of the
COVID-19 virus by restricting, and thus reducing, human mobility [9, 10, 11].
While this has successfully contained the spreading, it has come at an
enormous economic and social cost, for example, the complete closure of
schools, non-essential retail, manufacturing plants, etc. Furthermore,
stopping non-essential medical procedures will unavoidably cause long-term
health issues not related to COVID-19 [12].

The spreading of diseases has been traditionally described using continuous
differential equations such as reaction-diffusion models in spatially extended
systems [13, 14]. These models assume that the human population diffuses
homogeneously in some continuous domain, and contact occurs once two or more
individuals are sufficiently near to each other [15]. This perspective,
however, is far from being realistic, in particular because people moving from
one spatial patch to another follow well-defined paths that a continuous
spatial domain cannot capture. On the other hand, the interactions people have
once they are in the vicinity can hardly be considered homogeneous or well-
mixed. Inspired by that, in recent years, new network-based reaction-diffusion
models have been developed [16, 17, 18, 19, 20] and significant efforts in
understanding the structure of human populations and their movement have been
made with critical outcomes in understanding how an epidemic spreads [4]. From
this novel perspective, network nodes represent the different spatial patches
where individuals can interact, and network edges denote the paths through
which they can move. Modelling and detecting individuals’ movements has been a
major challenge for researchers in human dynamics [21, 22]. Although it might
resemble an oversimplification, random walks are an excellent first
approximation to the diffusion of individuals of a sufficiently large
population [23, 24].

Based on this synopsis, in this work, we present a reaction-diffusion model
that aims to slow down the spread of infection through a targeted reduction of
mobility as opposed to a global restriction on it. At the same time, our goal
is to reduce the adverse effects that a partial or complete lockdown may
cause. At odds with the traditional random walk diffusion, where individuals
diffuse through the network in an “unselective” fashion, in our case, we have
modified the rules such that individual walkers choose which node to hop into
next. Such biased random walk diffusion is based on recent developments that
take into account that each node (i.e., spatial patch) has a finite-size
carrying capacity [25, 26]. This constraint adds a nonlinear term to the
simple (unbiased) random walk process, yielding a different steady-state
configuration. In other words, such a new “microscopic” movement rule forces
the diffusing individuals to avoid nodes where there is already a high density
of individuals, preferring less crowded ones instead. Such a phenomenon when
correlated with the local interactions at the node level restricts the
spreading of the epidemic without necessarily increasing the economic/social
cost, usually an unavoidable consequence of mobility reduction. In fact, the
finite carrying capacity of the nodes can be considered an extra compartment
that can complement the already existing epidemiological models. We will
denote this compartment as $E$, which stands for the _Empty_ compartment.
Therefore, the classic Susceptible-Infected-Recovered ($SIR$) model that we
will consider throughout this paper now becomes an $ESIR$ compartment model
(see Fig. 1 for a schematic presentation. Let us also notice that such a new
slack compartment is artificial and does not represent any particular type of
individual but rather the density of vacant space that the individuals can
occupy. Following this formalism, the conservation property for the $ESIR$
model is written as $E+S+I+R=1$ where the symbols $S$, $I$, $R$, and $E$ will
represent the densities of the respective species of individuals and the empty
space. Grounded in the Pareto efficiency formalism and using the Entropy Rate
[27] to quantify the average process performance, in this work, we show that
it is possible to reduce the infection in the population by redistributing the
individuals differently among the nodes while still keeping the same average
mobility. Therefore, our model indicates that more specific and targeted
policies can be more effective in controlling the spreading of diseases and
simultaneously diminish the adverse effects of reduced mobility.

Figure 1: A schematic representation of the $ESIR$ model across two nodes of
the network. The model consists of a classic $SIR$ model in each of the nodes
(pink ovals), complemented with the Empty compartment $E$. The dynamics
between the different compartments are
$S_{i}\xrightarrow{\beta}I_{i}\xrightarrow{\gamma}R_{i}$ for each node $i$,
indicated by the black solid arrows and where $\beta$ and $\gamma$ represent
respectively the contagion rate and the recovery rate. In addition, we have
also considered (blue arrows) the internode diffusion relations
$S_{i}\xleftrightarrow{D}E_{j}$, $S_{j}\xleftrightarrow{D}E_{i}$,
$I_{i}\xleftrightarrow{D}E_{j}$, $I_{j}\xleftrightarrow{D}E_{i}$, and
$R_{i}\xleftrightarrow{D}E_{j}$, $R_{j}\xleftrightarrow{D}E_{i}$ where $D$
represent the diffusion constant and $i$ and $j$ are the connected nodes. The
double arrows in the these transitions mean that when an individual moves
towards an adjacent node it occupies a vacant space and in turn creates an
empty space in the origin node.

##  II Nonlinear random walks with finite-size nodes

A metapopulation network consists of nodes representing spatial patches where
groups of individuals congregate [28]. The nodes are connected through
preferential paths known as edges through which the individuals diffuse. In
order to model the diffusion process in a metapopulation network it is crucial
to consider the system’s overall behaviour without excluding any of the
walkers. To this aim, the master equation is responsible for the definition of
the stochastic dynamics that governs the global distribution of the
individuals for each node of the network at any instant of time [29, 30]

$\frac{d}{dt}P(\textbf{m},t)=\sum_{\textbf{m}^{\prime}}\big{[}T(\textbf{m}|\textbf{m}^{\prime})P(\textbf{m'},t)-T(\textbf{m}^{\prime}|\textbf{m})P(\textbf{m},t)\big{]}.$
(1)

Here $P(\textbf{m},t)$ denotes the probability that the system is in state m
at time $t$, $T(\textbf{m}|\textbf{m}^{\prime})$ is the transition probability
from state $\textbf{m}^{\prime}$ to m, and the sum is restricted to states
which $\textbf{m}^{\prime}$ can transition from [25]. Since we will initially
limit our discussion to the diffusion process only, the individual-based
dynamics considers the hopping of walkers from one node to another. Thus with
a little abuse in notation the transition probabilities are

$T(m_{i}-1,m_{j}+1|m_{i},m_{j})=\frac{A_{ij}}{k_{i}}f\left(\frac{m_{i}}{M}\right)g\left(\frac{m_{j}}{M}\right),$
(2)

where $m_{i}$ are elements of the vector $\mathbf{m}$, $M$ is total number of
sites available per node (that for simplicity we consider to be the same for
all nodes), $A_{ij}=1$ if there is a link between nodes $i$ and $j$,
$k_{i}=\sum_{j}A_{ij}$, is the degree of node $i$, and $f(\cdot)$ and
$g(\cdot)$ are two nonlinear functions representing the will of each
individual to leave node _i_ , and the will to settle in node _j_
respectively. To keep the discussion simple, throughout this paper, we will
define these functions depending on the concentration ${m_{i}/M}$ as
$f(m_{i}/M)=m_{i}/M\,$, and $g(m_{i}/M)=(1-m_{i}/M)^{\sigma_{i}}$, where the
parameter $\sigma_{i}$ represents the amount of congestion at node $i$, that
we will generally refer to as the _crowding_ parameter. Recall that the term
$1-m_{i}/M$ represents the density of the empty space in node $i$. It is easy
to see that in the limit, ${\sigma_{i}\rightarrow 0,\,\forall i}$, the random
walk with crowding collapses to the simple random walk case. Before we proceed
further, let us emphasise that the crowding phenomenon in terms of molecular
or cellular level occurs for ${\sigma_{i}=1,\,\forall i}$ [31, 32]. In fact,
from the physical point of view, particles are thought to displace
proportionally to the vacant space of the hosting nodes, an example of
interacting particles systems known as asymmetric simple exclusion process
(ASEP) [31]. However, when we extend the idea of the biased random walkers to
the case of humans (the same consideration can also hold for animals), their
perception of _attractiveness_ $g(\rho_{j})$ of the hosting node $j$, can be
different from the amount of the available space in such node, and the
crowding parameter $\sigma_{j}$ aims to quantify such relation. We believe
that this formulation better reflects the properties of human mobility where
the gathering in common areas (e.g., schools, shops, offices, etc.) depends on
the individual’s perception of the attractiveness of vacant space.

Although equation 2 is exact in describing the dynamics, it is, unfortunately,
impractical to deal with. Thus, to describe the time-evolution of the node
density $\rho_{i}(t)$, the implementation of a mean-field approach is
required. Based on that, we first average over the different configurations
${\langle m_{i}\rangle=\sum_{\textbf{m}}m_{i}P(\textbf{m},t)}$ and by then
taking the limit we obtain the node densities
${\rho_{i}=\lim\limits_{M\rightarrow\infty}\langle m_{i}\rangle/M}$. In
conclusion, we obtain the mean-field differential equations

$\frac{d\rho_{i}}{dt}=\sum_{j=1}^{\Omega}\Delta_{ij}\left[f(\rho_{j})g(\rho_{i})-\frac{k_{j}}{k_{i}}f(\rho_{i})g(\rho_{j})\right],\quad\forall
i$ (3)

where $\Omega$ is the number of nodes in the network,
${\Delta_{ij}=A_{ij}/k_{j}-\delta_{ij}}$ is the Linear Random Walk (LRW)
Laplacian, and time is rescaled so that ${t/M\mapsto t}$. Notice that in the
limit of large $M$ we can neglect any correlation ${\langle
f(\cdot)g(\cdot)\rangle=f(\langle\cdot\rangle)g(\langle\cdot\rangle)}$
following the Van Kampen ansatz [29, 30]. For further details on the
derivation and related generalisations the interested reader can refer to
[26]. To make eq. (3) more compact and to reflect our choice for the functions
$f(\cdot)$ and $g(\cdot,\cdot)$, we introduce the nonlinear diffusion operator

$L_{ij}(\rho)=\Delta_{ij}\left[\rho_{j}(1-\rho_{i})^{\sigma_{i}}-\frac{k_{j}}{k_{i}}\rho_{i}(1-\rho_{j})^{\sigma_{j}}\right],$

referring to it as the Nonlinear Random Walk (NLRW) operator, a notation that
we will use for the rest of this paper.

From the mean-field perspective, although we are modeling human populations
and will refer to members as individuals, it is essential to note that the
node populations (densities) are not discrete but rather continuous. The
continuous approximation allows us to easily model individuals’ movement
through the network and is computationally more convenient, thanks to the
deterministic (PDE) formulation. Also, from eq. (3), one can immediately
notice that at variance with the simple random walk diffusion, the diffusion
in our model depends on the state ${g(\cdot)}$ of the hosting node. This
function describes the available space in the destination node, which can be
filled by individuals coming from the origin node.

##  III Epidemic modelling in crowded networks

We will now augment the above formalism to consider also the local contagion
dynamics at the node level. To this aim, we will refer to the celebrated
Susceptible-Infected-Recovered ($SIR$) model [33, 34], a 3-compartment model,
where the (healthy) susceptible individuals, $S$, can potentially become
infected with probability $\lambda$ if they come in contact with infected
individuals $I$. On the other side, infected individuals become recovered,
$R$, at a rate $\gamma$ and participate in the diffusion dynamics as immune to
the infection. Putting together the nonlinear random walk diffusion of the
individuals between adjacent nodes, with the contagion dynamics occurring
within nodes, we have the new mean-field set of equations,

$\displaystyle\frac{dS_{i}}{dt}$ $\displaystyle=$ $\displaystyle-\lambda
S_{i}I_{i}+D\sum_{j=1}^{\Omega}L_{ij}(S),$ $\displaystyle\frac{dI_{i}}{dt}$
$\displaystyle=$ $\displaystyle\lambda S_{i}I_{i}-\gamma
I_{i}+D\sum_{j=1}^{\Omega}L_{ij}(I),$ (4) $\displaystyle\frac{dR_{i}}{dt}$
$\displaystyle=$ $\displaystyle\gamma I_{i}+D\sum_{j=1}^{\Omega}L_{ij}(R),$

where $S_{i}$, $I_{i}$, and $R_{i}$ represent the density of susceptible,
infected, and recovered individuals respectively in node $i$, $\lambda$ is the
infection rate, and $\gamma$ is the recovery rate. The rates are defined under
the standard assumption of well-mixed populations within each node. Finally,
$D$ denotes the diffusion constant, which for simplicity is considered to be
the same for susceptible, infected and recovered individuals, as well as equal
for all nodes. Since the (nonlinear) diffusion is a conservative process [25,
26], the total number of individuals, remains constant over the network,
similar to many other epidemic models [34]. Nevertheless, it is essential to
emphasize that in our model, the densities of the individuals do not sum to
unity. In fact, in the Eqs. (III) above, we have intrinsically introduced
through the diffusion operator $L$ another compartment $E$ to quantify the
density of the vacant space available in each node. Thus in our case the
conservation property for each node reads like $S_{i}+I_{i}+R_{i}+E_{i}=1$. We
name this model the $ESIR$ _compartment model_. However, we note that it is
possible to use other compartment models for more complex spreading mechanisms
by simply augmenting them with the $E$ compartment as above. However, since
our focus is on the role that the spatial interactions have on spreading the
infection rather than the contagion dynamics, we will constrain our discussion
to the $ESIR$ model above.

To shed light on how the perception of crowding nodes affects the spreading of
infection, we will consider a synthetically generated spatial network, known
as the latent spatial network model [35, 36]. It can be briefly described as
an algorithm for generating random geometric graphs; that is, random networks
embedded in a Euclidian space where the probability of two given nodes $i,j$
being connected is given by

$p(i,j)=1\big{/}{\left(1+e^{-\alpha+d\left(i,j\right)}\right)},$

where $\alpha$ is a parameter of the model and $d$ the Euclidian distance
between the nodes, $d=\big{|}x(i)-x(j)\big{|}$, with $x(i)$ representing the
coordinates of node $i$. Thus, nodes that are closer together in this
Euclidean space are more likely to be connected. The outcome of an infection
spreading simulation in the new context of the $ESIR$ model is shown in Fig. 2
where for simplicity we have fixed the crowding parameter to be equal for
every node $\sigma_{i}=\sigma$. It can be observed that increasing the value
of $\sigma$, namely the perception of congestion in the hosting nodes, leads
to a dramatic decrease in the peak of the infection curves. This slowing down
of the spreading of infection is also associated with multiple waves of
infections as well as a general delay in the infection surge. These features
of the infection curves are crucial in addressing hospitalisation capacities.

Figure 2: Plots of the average node-level infection
$\sum_{i}I_{i}/(\Omega\beta)$ and the average node-level recovery
$\sum_{i}R_{i}/(\Omega\beta)$ versus time, on a latent spatial graph for
different values of the uniform crowding parameter $\sigma$. Notice that
increasing the crowding parameter $\sigma$ flattens the infection curves and
breaks the emerging of the infection peak leading to second and third waves.
The latent spatial graph consists of $100$ nodes on a $7.5$ by $7.5$ Euclidean
space, $\alpha=0.3$, and the other parameters of the model are $\lambda=1$,
$\gamma=0.1$, $\beta=0.3$, $D=2$. The infection is seeded at $5$ randomly
selected nodes. These nodes have $S_{i}=0.25$, and $I_{i}=0.05$, ensuring
$S_{i}+I_{i}=\beta=0.3$.

Such an outcome can intuitively be explained because a significant value of
$\sigma$ would slow down the diffusion and decrease mobility, which further
isolates the infections and consequently the spreading. Following this logic,
we analyse eqs. (III) by linearly expanding for early times of the spreading
(see the Appendix for details) for the particular case of regular graphs and
$\sigma_{i}=\sigma,\forall i$. In conclusion, the linearised diffusion
operator is equivalent to, for instance for the susceptible individuals

$D\left(1-S^{*}\right)^{\sigma}\sum_{j=1}^{\Omega}\Delta_{ij}S_{j}$ (5)

as expressed in terms of the LRW Laplacian $\Delta_{ij}$ and where $S^{*}$ is
the initial uniformly distributed fixed point for the susceptible individuals
in absence of infected and recovered individual $I^{*}=R^{*}$=0. Notice that
since, in general, the nodes will not be entirely occupied, the term
$1-S^{*}\leq 1$, thus justifying the consistency of the expression above.
Equation (5) shows that in the absence of the crowding effect $\sigma=0$
(e.g., very diluted conditions $M\rightarrow\infty$), the infection spreads as
if being driven by a LRW Laplacian operator. However, once we take into
consideration the finite capacity of the nodes, with $\sigma>1$, the linear
diffusion will slow down. 111In principle, it is possible to consider the case
$\sigma<1$, which can be interpreted as the scenario when the individuals
underestimate the occupancy of the adjacent nodes. However, this case does not
significantly affect the overall dynamics, therefore it has not been
considered in this study. It is important to emphasise that the linear
diffusion can only predict the dynamical outcome of the model shortly after
the system is initialised, but not for longer time periods. Nevertheless, it
is reasonable to think that if the diffusion of infection has been limited at
early stages, it should decisively influence both the peak of infection and
delay the surge of disease in general. On the other side, it is rational that
a drastic decrease in the pace of diffusion of both infected or healthy
individuals (the recovered ones are passive in the contagion process) should
also strictly reduce the general mobility in the network. Next, we will show
that it is possible to drastically reduce the overall infection while
maintaining high levels of mobility.

##  IV Preventing the infection spread while reducing the mobility cost

Although curbing the peak of infection curve is a highly desired goal of
epidemic control policies, it is often associated with considerable unwanted
effects such economic, social and ironically also health costs [38, 39]. A
real challenge from this perspective is to cut down the bulk of the infection
and at the same time to keep reasonable levels of economic and social
activity, which are also necessary for mental well-being. Before we show that
the $ESIR$ model we propose succeeds in reaching this goal, we first need to
introduce a measure which quantifies the level of mobility of active
individuals in the network spatial domain. To this aim we will refer to an
entropic measure borrowed from information theory, known as the Entropy Rate
(ER) [27], which has been successfully used to assess the mobility of biased
random walkers in complex networks [40]. For a static Markov chain the ER is
defined as $H=-\sum_{ij}\rho_{i}^{*}T_{ij}\log T_{ij}$, where
$\\{\rho_{i}^{*}\\}$ are the equilibrium states and $\textbf{T}=\\{T_{ij}\\}$
is the transition matrix evaluated at equilibrium. In our case this formula is
explicitly written as follows (see also [26])

$H=-D\sum\limits_{ij}\rho_{i}^{*}A_{ij}\frac{\rho_{i}^{*}\left(1-\rho_{j}^{*}\right)^{\sigma_{j}}}{k_{i}}\log\left[A_{ij}\frac{\rho_{i}^{*}\left(1-\rho_{j}^{*}\right)^{\sigma_{j}}}{k_{i}}\right]$
(6)

where it is noted that the ER depends on the the diffusion and crowding
parameters, $D$ and $\sigma_{i},\forall i$, respectively, and on the network
topology via the adjacency matrix $A=\\{A_{ij}\\}$. The equilibria of the
mean-field variables
$\lim\limits_{t\rightarrow\infty}\rho_{i}(t)=\rho_{i}^{*}$ are obtained
numerically once the system fully relaxes its diffusion component—summing eq.
(3) over all three compartments. As anticipated earlier, it is in principle
possible to extend the $SIR$ model we consider here to more complex contagion
dynamics, but in this case, some of the new type of individuals (e.g., the
ones that quarantine) might not contribute to the mobility, and this would
unnecessarily complicate the derivation of eq. (6).

Noe that we have a measure that globally quantifies the mobility of walkers in
a given stochastic process, we turn our attention to the possibility that,
while slowing down the spreading of infection, it is possible to keep a good
amount of efficiency in the overall mobility. To address this question, we
will explore the domain of parameters
$\bm{\mathcal{D}}=\left(D,\bm{\sigma}\right)$ where
$\bm{\sigma}=\left[\sigma_{1},\sigma_{2},\dots,\sigma_{\Omega}\right]$ and
will simultaneously minimize the peak of infection $I_{max}$ and maximize the
overall mobility $H$. Such multi-objective optimization, known as Pareto
optimization [41], can be mathematically formulated as follows:

$\displaystyle\min$
$\displaystyle\left(I_{max}\left(D,\bm{\sigma}\right),H^{-1}\left(D,\bm{\sigma}\right)\right)$
(7) $\displaystyle s.t.\quad\left(D,\bm{\sigma}\right)\in\bm{\mathcal{D}}.$

Notice that, in general, there is not a unique solution to the problem above.
Instead, it can be shown that it exists a set of optimal outcomes denoted as
the Pareto front [41]. Our aim here is to find the points of the Pareto front
in the NLRW case which when compared to the case where crowding is absent,
i.e., $\sigma_{i}=0,\forall i$ (thus the diffusion is modeled as an LRW
process), the peak of infection $I_{max}$ is lower for the same Entropy Rate
$H$. From the mathematical point of view, we expect that by increasing the
degree of freedom of the feasible set through the set of variables,
$\sigma_{1},\sigma_{2},\dots,\sigma_{\Omega}$, we can access optimal points
that are not possible otherwise. The drawback of incrementing the variable
space lies in the high complexity of the Pareto optimization algorithm 222In
the following, we have made use of the pareto() function of the MATLAB Global
Optimization toolbox., making it complicated to compute even for relatively
small networks. To resolve this problem, we will constrain the search of the
Pareto optimal points in a restricted subset of the parameters space. An
immediate choice is to search for optimality in the parametric line
$\sigma_{i}=\sigma$ where now $\sigma$ acts as the line parameter. Another
possibility is to consider the parametric line $\sigma_{i}=k_{i}\sigma$ where
in this case the line coefficient in the $\Omega-$dimensional space is equal
the the node degree $k_{i}$. Thus in this later scenario, we consider more
stringent measures for metanodes with a higher number of connections.

Based on such considerations, in Fig. 3, we show that the task of decreasing
the infection while maximizing mobility is indeed possible. In fact, comparing
the case of LRW diffusion (red curve) vs. both cases of the NLRW one (blue and
green curves) for latent spatial network, one can immediately notice that the
same values of the Entropy Rate $H$ have considerably lower maxima of
infection. Furthermore such maximum values of infection are always lower when
the crowding parameter $\sigma_{i}$ varies proportionally with the nodes’
degree. This result shows that controlling the level of infection in a given
population does not necessarily imply a restriction of mobility via partial or
complete lockdown measures, but that better control on the capacity of all the
spatially distributed sites where people can enter into contact with each
other, is the optimal choice.

We show that the similar results can be obtained in a collection of empirical
spatial networks — the London Tube [43], airline routes [44] and USA
contiguous map network [45, 46] — when we implement the Pareto optimization of
the $SIR$ model with NLRW diffusion. In Fig. 4, it is shown that in each of
these networks, the peak of infection decreases monotonically when the Entropy
Rate $H$ decreases and the gap between the two approaches changes according to
the different network topologies. For the airline routes network, for
instance, we have the same peaks of infections for a very high level of
mobility (no lockdown measures) in both LRW and NLRW cases, suggesting that a
general reduction of mobility is needed for the proposed ad hoc measures to be
effective.

Figure 3: a) Comparison of the evolution of minima of the infection peaks
$I_{max}$ vs. the maxima of Entropy Rate (ER) $H$ between the spreading with
Linear Random Walk (LRW) diffusion (red circles) and Nonlinear Random Walk
(NLRW) diffusion (blue diamonds for a fixed $\sigma$ and green squares
$\sigma_{i}$). b) For the NLRW case, the results were obtained calculating the
Pareto front of the two-objective function $\left(I_{max},1/H\right)$ as a
function of the Pareto set $(D,\sigma)$ (for both cases $\sigma_{i}=\sigma$
and $\sigma_{i}=\sigma\,k_{i}$) in the inset. Instead, the LRW curve was
obtained by simply varying the diffusion rate $D$ by selecting the same ER
values $H$. Thus there is only a single possible curve in the case of the LRW,
as there is only a single parameter to change, D. For any value of $H$, the
infection is always lower when the individuals avoid crowded nodes, and the
simulations were carried with the same graph and parameters of Fig. 2. Figure
4: The minima of the infection peaks $I_{max}$ vs. the maxima of the Entropy
Rate $H$ for the LRW (red circles) and NLRW (blue diamonds for a fixed
$\sigma$ and green squares for a varying $\sigma_{i}$) mobility. The empirical
spatial networks considered are a) London Tube, b) airlines route, and c) USA
contiguous map. In all cases the NLRW mobility performs as well as, and
usually better than, the case of LRW diffusion. The parameters are
$\lambda=1$, $\gamma=0.1$.

##  V Conclusions

In this study, we presented a reaction-diffusion model on a metapopulation
network for optimal control of the infection spread, and tested it on
synthetic networks and real-world data sets. This work aims to find a common
approach to how human mobility should be regulated to keep the same pace of
efficiency of human activity while keeping a low level of infection in the
system. Starting from this premise, we have first reformulated the diffusion
of individuals based on a (biased) Nonlinear Random Walk (NLRW) process
introduced earlier in the literature [25, 26]. The latter differs from the
classical Linear Random Walk (LRW) diffusion by considering a finite carrying
occupancy per node, making the transition probability dependant on the density
of individuals of the neighbour nodes. To further strengthen the social
aspects of how individuals move through different spatial patches represented
by interconnected vertices, we have introduced a node-based parameter that
accounts for individuals’ perception of the amount of crowding in a given
node. Based on the master equation formalism, we show that a nonlinear
deterministic transport operator is derived at the mean-field level. It is
then possible to complement the formalism through a set of contagion
interactions at the node level, where for definiteness, we have considered the
paradigmatic Susceptible-Infected-Recovered model [33].

Based on this formulation, we investigate the role that an NLRW diffusion has
on the spreading of infection, and in particular, we show that not only the
surge of disease is delayed, but also lower peaks can occur, and that the load
of infection is distributed in several smaller waves. Indeed is possible to
prove based on a linear prediction that crowding can slow down the spreading
of the disease. However, at the first moment, this can intuitively be
understood due to the reduction of general mobility. The latter is, of course,
an undesired effect that will negatively impact not only the population from
an economic and social aspect but also the access to the health system in
general [38, 39]. Nevertheless, the nonlinear diffusion model has a higher
complexity regarding the number of parameters influencing the possible
resulting scenario. In fact, starting from this perspective, we implement a
multi-objective (Pareto) optimization on the control parameters,
simultaneously aiming to determine the lowest possible peak of infection
within the maximum possible mobility efficiency. The latter is formulated
based on an information theory concept known as the Entropy Rate and
quantifies the general level of performance of a stochastic process [27, 40].
We demonstrate that in the NLRW setting, it is possible to control the
spreading of epidemics by choosing a set of parameters that reach the Pareto
front, where the peak of infection is considerably lower than in the LRW case
in which no restriction on the diffusion is considered. We have verified the
validity of our result on both synthetic and empirical spatial networks.

In conclusion, the mathematical model we develop in this paper can outline
protocol measures for a targeted set of measures at the level of workplaces,
schools, retail activities, hospitals, etc., by simply imposing a sufficient
maximum occupancy on the latter. Our results suggest novel scenarios where
mitigation policies can be based on a dynamically tuneable limitation of the
capacity of indoor venues in order to contain and control the level of
infection in the society and at the same time minimise the effect that such
measures have on the human mobility. From this viewpoint, it is possible to
better schedule the varying activities that require people to spend a long
time in crowded and poorly ventilated spaces. This would allow a good level of
human activity with less adverse effects of the ongoing epidemics.

## Acknowledgements

This work was partially supported by Irish Research Council [grant number
GOIPG/2018/3026] (BAS) and by Science Foundation Ireland Grant numbers
16/IA/4470, 16/RC/3918 and 12/RC/2289P2 (JPG and MA).

##  Appendix A Local analysis of the epidemic outbreak

Here we use a linear stability analysis approach to predict the initial rate
of infection growth for the NLRW diffusion operator. To proceed with the
explicit calculations, we first need to assume the particular case of regular
graphs, i.e., $k_{i}=k$, and $\sigma_{i}=\sigma,\forall i$.

Let us first recall that the NLRW Laplacian acting on the susceptible species
for regular graphs is given by:

$\displaystyle\sum_{j}L_{ij}(S)=D\sum_{j}\Delta_{ij}$
$\displaystyle\left[S_{j}\left(1-S_{i}-I_{i}-R_{i}\right)^{\sigma}\right.+$
$\displaystyle-\left.S_{i}\left(1-S_{j}-I_{j}-R_{j}\right)^{\sigma}\right].$
(8)

Now consider perturbing from the initial steady state where no infected (and
consequently no recovered) individuals are present, such that
$S_{i}=S^{*}+\delta S_{i}$, $I_{i}=0+\delta I_{i}$, and $R_{i}=0+\delta
R_{i}$, which gives

$\displaystyle D\sum_{j}\Delta_{ij}$ $\displaystyle\left[\left(S^{*}+\delta
S_{j}\right)\left(1-S^{*}-\delta S_{i}-\delta I_{i}-\delta
R_{i}\right)^{\sigma}\right.+$ $\displaystyle-\left.\left(S^{*}+\delta
S_{i}\right)\left(1-\delta S_{j}-\delta I_{j}-\delta
R_{j}\right)^{\sigma}\right].$

Then we linearize the following term via a Taylor approximation,

$\displaystyle\left(S^{*}+\delta S_{j}\right)$
$\displaystyle\left(1-S^{*}-\delta S_{i}-\delta I_{i}-\delta
R_{i}\right)^{\sigma}\approx$ $\displaystyle\approx
S^{*}\left(1-S^{*}\right)^{\sigma}+\delta S_{j}\left(1-S^{*}\right)^{\sigma}+$
$\displaystyle-S^{*}\sigma\left(1-S^{*}\right)^{\left(\sigma-1\right)}\left(\delta
S_{i}+\delta I_{i}+\delta R_{i}\right).$

Subbing this in gives

$\displaystyle
D\left[{S^{*}\left(1-S^{*}\right)^{\sigma}\sum_{j}\Delta_{ij}}+\left(1-S^{*}\right)^{\sigma}\sum_{j}\Delta_{ij}\delta
S_{j}+\right.$ $\displaystyle-{S^{*}\sigma(1-S^{*})^{(\sigma-1)}(\delta
S_{i}+\delta I_{i}+\delta R_{i})\sum_{j}\Delta_{ij}}+$
$\displaystyle-{S^{*}(1-S^{*})^{\sigma}\sum_{j}\Delta_{ij}}-(1-S^{*})^{\sigma}\delta
S_{i}\sum_{j}\Delta_{ij}+$
$\displaystyle+\left.S^{*}\sigma(1-S^{*})^{(\sigma-1)}\sum_{j}\Delta_{ij}\left(\delta
S_{j}+\delta I_{j}+\delta R_{j}\right)\right]$

All these terms, but the second one, vanish since the sum of the rows of the
Laplacian $\Delta_{ij}$ is zero, and the sum of all the perturbations, in each
node, equals zero 333To be more rigorous due to the linear diffusion, the sum
of the perturbation $\delta S_{j}+\delta I_{j}+\delta R_{j}$ does not vanish
completely, but since the initial values of perturbation are small such
contribution is negligible., which leaves us with

$D(1-S^{*})^{\sigma}\sum_{j}\Delta_{ij}\delta S_{j}.$ (9)

The procedure above repeats similarly for all the species, thus fully
justifying eq. (5) of the main text.

## References

  * Benedictow [2004] O. J. Benedictow,  _The Black Death, 1346-1353: the complete history_ (Boydell Press, Woodbridge, Suffolk, UK ; Rochester, N.Y., USA, 2004).
  * Colizza  _et al._ [2006] V. Colizza, A. Barrat, M. Barthélemy, and A. Vespignani, Proceedings of the National Academy of Sciences of the United States of America 103, 2015 (2006).
  * Balcan  _et al._ [2009] D. Balcan, H. Hu, B. Goncalves, P. Bajardi, C. Poletto, J. J. Ramasco, D. Paolotti, N. Perra, M. Tizzoni, W. V. Broeck, V. Colizza, and A. Vespignani, BMC Medicine 7, 45 (2009).
  * Funk  _et al._ [2010] S. Funk, M. Salathé, and V. A. Jansen, Journal of the Royal Society Interface 7, 1247 (2010).
  * Perra  _et al._ [2011] N. Perra, D. Balcan, B. Gonçalves, and A. Vespignani, PLoS ONE 6, 10.1371/journal.pone.0023084 (2011).
  * Kraemer  _et al._ [2020] M. U. Kraemer, C. H. Yang, B. Gutierrez, C. H. Wu, B. Klein, D. M. Pigott, L. du Plessis, N. R. Faria, R. Li, W. P. Hanage, J. S. Brownstein, M. Layan, A. Vespignani, H. Tian, C. Dye, S. Cauchemez, O. G. Pybus, S. V. Scarpino, B. Xu, S. Mekaru, K. Sewalk, L. Goodwin, A. Loskill, E. Cohn, Y. Hswen, S. C. Hill, M. M. Cobo, A. Zarebski, S. Li, E. Hulland, J. Morgan, L. Wang, and K. O’Brien, medRxiv 497, 493 (2020).
  * Vespignani  _et al._ [2020] A. Vespignani, H. Tian, C. Dye, J. O. Lloyd-Smith, R. M. Eggo, M. Shrestha, S. V. Scarpino, B. Gutierrez, M. U. Kraemer, J. Wu, K. Leung, and G. M. Leung, Nature Reviews Physics 2, 279 (2020).
  * Fang  _et al._ [2020] H. Fang, L. Wang, and Y. Yang, Journal of Public Economics 191, 104272 (2020).
  * Aleta and Moreno [2020] A. Aleta and Y. Moreno, medRxiv , 1 (2020).
  * Aleta  _et al._ [2020] A. Aleta, Q. Hu, J. Ye, P. Ji, and Y. Moreno, Chaos, Solitons & Fractals 139, 110068 (2020).
  * Liu  _et al._ [2020] Q. H. Liu, A. I. Bento, K. Yang, H. Zhang, X. Yang, S. Merler, A. Vespignani, J. Lv, H. Yu, W. Zhang, T. Zhou, and M. Ajelli, PLoS Computational Biology 16, 1 (2020).
  * Bonaccorsi  _et al._ [2020] G. Bonaccorsi, F. Pierri, M. Cinelli, A. Flori, A. Galeazzi, F. Porcelli, A. L. Schmidt, C. M. Valensise, A. Scala, W. Quattrociocchi, and F. Pammolli, Proceedings of the National Academy of Sciences of the United States of America 10.1073/pnas.2007658117 (2020).
  * Kermack and McKendrick [1927] W. O. Kermack and A. G. McKendrick, Proceedings of the Royal Society of London. Series A, Containing Papers of a Mathematical and Physical Character 115, 700 (1927).
  * Volpert and Petrovskii [2009] V. Volpert and S. Petrovskii, Physics of Life Reviews 6, 267 (2009).
  * Britton  _et al._ [2020] T. Britton, F. Ball, and P. Trapman, Science 369, 846 (2020).
  * Newman [2010] M. Newman,  _Networks: An Introduction_ (2010) pp. 1–784.
  * Barthélemy [2011] M. Barthélemy, Physics Reports 499, 1 (2011), 1010.0302 .
  * Colizza  _et al._ [2007] V. Colizza, R. Pastor-Satorras, and A. Vespignani, Nature Physics 3, 276 (2007), 0703129 [cond-mat] .
  * Belik  _et al._ [2011] V. Belik, T. Geisel, and D. Brockmann, Physical Review X 1, 1 (2011), 1103.6224 .
  * Chinazzi  _et al._ [2020] M. Chinazzi, J. T. Davis, M. Ajelli, C. Gioannini, M. Litvinova, S. Merler, A. P. Y. Piontti, L. Rossi, K. Sun, C. Viboud, X. Xiong, H. Yu, M. E. Halloran, I. M. Longini, and A. Vespignani, medRxiv 400, 395 (2020).
  * González  _et al._ [2008] M. C. González, C. A. Hidalgo, and A.-L. Barabási, Nature 453, 779 (2008).
  * Song  _et al._ [2010a] C. Song, Z. Qu, N. Blumm, and A.-L. Barabasi, Science 327, 1018 (2010a).
  * Song  _et al._ [2010b] C. Song, T. Koren, P. Wang, and A.-L. Barabási, Nature Physics 6, 818 (2010b).
  * Barbosa  _et al._ [2018] H. Barbosa, M. Barthelemy, G. Ghoshal, C. R. James, M. Lenormand, T. Louail, R. Menezes, J. J. Ramasco, F. Simini, and M. Tomasini, Physics Reports 734, 1 (2018).
  * Asllani  _et al._ [2018] M. Asllani, T. Carletti, F. Di Patti, D. Fanelli, and F. Piazza, Physical Review Letters 120, 158301 (2018), 1711.06733 .
  * Carletti  _et al._ [2020] T. Carletti, M. Asllani, D. Fanelli, and V. Latora, Physical Review Research 2, 033012 (2020).
  * Cover and Thomas [2006] T. M. Cover and J. A. Thomas,  _Elements of information theory_ , 2nd ed. (Wiley-Interscience, Hoboken, N.J, 2006) oCLC: ocm59879802.
  * Calvetti  _et al._ [2020] D. Calvetti, A. P. Hoover, J. Rose, and E. Somersalo, Frontiers in Physics 8, 1 (2020).
  * Gardiner [2004] C. W. Gardiner,  _Handbook of Stochastic Methods_, edited by H. Haken, Springer Series in Synergetics, Vol. 13 (Springer Berlin Heidelberg, Berlin, Heidelberg, 2004).
  * Kampen [2007] N. G. v. Kampen,  _Stochastic processes in physics and chemistry_ , 3rd ed., North-Holland personal library (Elsevier, Amsterdam ; Boston, 2007) oCLC: ocm81453662.
  * Liggett [1999] T. M. Liggett,  _Stochastic Interacting Systems: Contact, Voter and Exclusion Processes_, Grundlehren der mathematischen Wissenschaften, Vol. 324 (Springer Berlin Heidelberg, Berlin, Heidelberg, 1999).
  * Neri  _et al._ [2011] I. Neri, N. Kern, and A. Parmeggiani, Physical Review Letters 107, 068702 (2011).
  * Murray [2002] J. D. Murray,  _Mathematical biology_ , 3rd ed., Interdisciplinary applied mathematics (Springer, New York, 2002).
  * Kiss  _et al._ [2017] I. Z. Kiss, J. C. Miller, and P. L. Simon,  _Interdisciplinary Applied Mathematics_, Vol. 46 (2017) pp. 1–413.
  * Raftery  _et al._ [2012] A. E. Raftery, X. Niu, P. D. Hoff, and K. Y. Yeung, Journal of Computational and Graphical Statistics 21, 901 (2012).
  * Raftery [2017] A. E. Raftery, Journal of the American Statistical Association 112, 1531 (2017).
  * Note [1] In principle, it is possible to consider the case $\sigma<1$, which can be interpreted as the scenario when the individuals underestimate the occupancy of the adjacent nodes. However, this case does not significantly affect the overall dynamics, therefore it has not been considered in this study.
  * Pike  _et al._ [2014] J. Pike, T. Bogich, S. Elwood, D. C. Finnoff, and P. Daszak, Proceedings of the National Academy of Sciences 111, 18519 (2014).
  * Fan  _et al._ [2018] V. Y. Fan, D. T. Jamison, and L. H. Summers, Bulletin of the World Health Organization 96, 129 (2018).
  * Gómez-Gardeñes and Latora [2008] J. Gómez-Gardeñes and V. Latora, Physical Review E - Statistical, Nonlinear, and Soft Matter Physics 78, 1 (2008), 0712.0278 .
  * Miettinen [1998] K. Miettinen,  _Nonlinear Multiobjective Optimization_, edited by F. S. Hillier, International Series in Operations Research & Management Science, Vol. 12 (Springer US, Boston, MA, 1998).
  * Note [2] In the following, we have made use of the pareto() function of the MATLAB Global Optimization toolbox.
  * De Domenico  _et al._ [2014] M. De Domenico, A. Solé-Ribalta, S. Gómez, and A. Arenas, Proceedings of the National Academy of Sciences 111, 8351 (2014), https://www.pnas.org/content/111/23/8351.full.pdf .
  * [44] V. Batagelj and A. Mrvar, .
  * Kunegis [2013] J. Kunegis, Proceedings of the 22nd International Conference on World Wide Web - WWW ’13 Companion , 1343–1350 (2013).
  * Knuth [2005] D. E. Knuth,  _The art of computer Programming. Volume 4, FASCICLE 0_ (Addison-Wesley, 2005).
  * Note [3] To be more rigorous due to the linear diffusion, the sum of the perturbation $\delta S_{j}+\delta I_{j}+\delta R_{j}$ does not vanish completely, but since the initial values of perturbation are small such contribution is negligible.


# Continuous Entailment Patterns for Lexical Inference in Context

Martin Schmitt  Hinrich Schütze
Center for Information and Language Processing (CIS)
LMU Munich, Germany
<EMAIL_ADDRESS>

###### Abstract

Combining a pretrained language model (PLM) with textual patterns has been
shown to help in both zero- and few-shot settings. For zero-shot performance,
it makes sense to design patterns that closely resemble the text seen during
self-supervised pretraining because the model has never seen anything else.
Supervised training allows for more flexibility. If we allow for tokens
outside the PLM’s vocabulary, patterns can be adapted more flexibly to a PLM’s
idiosyncrasies. Contrasting patterns where a “token” can be any continuous
vector vs. those where a discrete choice between vocabulary elements has to be
made, we call our method _CONtinuous pAtterNs_ (conan). We evaluate conan on
two established benchmarks for lexical inference in context (LIiC) a.k.a.
predicate entailment, a challenging natural language understanding task with
relatively small training sets. In a direct comparison with discrete patterns,
conan consistently leads to improved performance, setting a new state of the
art. Our experiments give valuable insights into the kind of pattern that
enhances a PLM’s performance on LIiC and raise important questions regarding
our understanding of PLMs using text patterns.111Our code is publicly
available: https://github.com/mnschmit/conan

##  1 Introduction

Lexical inference in context (LIiC) – also called predicate entailment – is a
variant of natural language inference (NLI) or recognizing textual entailment
(Dagan et al., 2013) with focus on the lexical semantics of verbs and verbal
expressions (Levy and Dagan, 2016; Schmitt and Schütze, 2019). Its goal is to
detect entailment between two very similar sentences, i.e., sentences that
share subject and object and only differ in the predicate, e.g., person$(A)$
_runs_ org$(B)$ $\rightarrow$ person$(A)$ _leads_ org$(B)$. NLI models that
were not specifically trained with lexical knowledge have been reported to
struggle with this task (Glockner et al., 2018; Schmitt and Schütze, 2019),
making LIiC an important evaluation criterion for general language
understanding. Other use cases for this kind of lexical entailment knowledge
include question answering (Schoenmackers et al., 2010; McKenna et al., 2021),
event coreference (Shwartz et al., 2017; Meged et al., 2020), and link
prediction in knowledge graphs (Hosseini et al., 2019).

Although LIiC is an inherently directional task, symmetric cosine similarity
in a vector space, such as word2vec (an, 2013), has long been the state of the
art for this task. Only recently transfer learning with pretrained Transformer
(Vaswani et al., 2017) language models (Devlin et al., 2019), has led to large
improvements for LIiC. Schmitt and Schütze (2021) combine natural language
(NL) patterns with a pretrained language model (PLM) and not only set a new
state of the art but also beat baselines without access to such patterns.

Empirical findings suggest that a good pattern can be worth 100s of labeled
training instances (Le Scao and Rush, 2021), making pattern approaches
interesting for low-resource tasks such as LIiC. But beyond the intuition that
patterns serve as some sort of task instruction (Schick and Schütze, 2021a),
little is known about the reasons for their success. Recent findings that  (i)
PLMs can fail to follow even simple instructions (Efrat and Levy, 2020), that
(ii) PLMs can behave drastically different with paraphrases of the same
pattern (Elazar et al., 2021), and that  (iii) performance increases if we
train a second model to rewrite an input pattern with the goal of making it
more comprehensible for a target PLM (Haviv et al., 2021),  strongly suggest
that patterns do not make sense to PLMs in the same way as they do to humans.

Our work sheds light on the interaction of patterns and PLMs and proposes a
new method of improving pattern-based models fully automatically. On two
popular LIiC benchmarks, our model  (i) establishes a new state of the art
without the need for handcrafting patterns or automatically identifying them
in a corpus and  (ii) does so more efficiently thanks to shorter patterns.
Our best model only uses 2 tokens per pattern.

##  2 The conan Model

Continuous patterns. LIiC is a binary classification task, i.e., given a
premise $\mathbf{p}=p_{1}p_{2}\dots p_{\left|\mathbf{p}\right|}$ and a
hypothesis $\mathbf{h}=h_{1}h_{2}\dots h_{\left|\mathbf{h}\right|}$ a model
has to decide whether $\mathbf{p}$ entails $\mathbf{h}$ ($y=1$) or not
($y=0$). A template-based approach to this task surrounds $\mathbf{p}$ and
$\mathbf{h}$ with tokens $t_{1}t_{2}\dots t_{m}$ to bias the classifier for
entailment detection, e.g., “$\mathbf{p}$, _which means that_
$\mathbf{h}$”.While in most approaches that leverage a PLM $\mathcal{M}$ these
tokens come from the PLM’s vocabulary, i.e., $\forall
i.\,t_{i}\in\Sigma_{\mathcal{M}}$, we propose a model based on CONtinuous
pAtterNs (conan), i.e., surround the embedding representation of $\mathbf{p}$
and $\mathbf{h}$ with continuous vectors that may be close to but do not have
to match the embedding of any vocabulary entry.

For this, we first extend the PLM’s vocabulary by a finite set
$C=\left\\{\,c_{1},c_{2},\dots,c_{\left|C\right|}\,\right\\}$ of fresh tokens,
i.e., $\Sigma=\Sigma_{\mathcal{M}}\cup C$ with
$C\cap\Sigma_{\mathcal{M}}=\emptyset$. Then, we distinguish two methods of
surrounding $\mathbf{p}$ and $\mathbf{h}$ with these special tokens: $\alpha$
sets them both $\alpha$round and between $\mathbf{p}$ and $\mathbf{h}$
(Section 2) while $\beta$ only sets them $\beta$etween the two input sentences
(Eq. 2).

$\displaystyle\alpha_{k}(\mathbf{p},\mathbf{h})$ $\displaystyle=c_{1}\dots
c_{a}\mathbf{p}c_{a+1}\dots c_{a+b}\mathbf{h}c_{a+b+1}\dots c_{k}$
$\displaystyle\text{with }a=$ $\displaystyle\left\lfloor
k/3\right\rfloor,\hskip 1.00006ptb=\left\lfloor k/3\right\rfloor+(k\bmod 3)$
(1) $\displaystyle\beta_{k}(\mathbf{p},\mathbf{h})$
$\displaystyle=\mathbf{p}c_{1}\dots c_{k}\mathbf{h}$ (2)

Note that $\alpha$ divides its $k$ tokens into three parts as equally as
possible where any remaining tokens go between $\mathbf{p}$ and $\mathbf{h}$
if $k$ is not a multiple of $3$. In particular, this means that the same
templates are produced by $\alpha$ and $\beta$ for $k\leq 2$. We chose this
behavior as a generalization of the standard approach to fine-tuning a PLM for
sequence classification (such as NLI) where there is only one special token
and it separates the two input sequences. The template produced by
$\alpha_{1}$ and $\beta_{1}$ is very similar to this. A major difference is
that the embeddings for $C$ tokens are randomly initialized whereas the
standard separator token has a pretrained embedding.

Pattern-based classifier. Given $\gamma\in\left\\{\,\alpha,\beta\,\right\\}$,
we estimate the probability distribution
$P\left(\hat{y}\mid\mathbf{p},\mathbf{h}\right)$ with a linear classifier on
top of the pooled sequence representation produced by the PLM $\mathcal{M}$:

$P\left(\hat{y}\mid\mathbf{p},\mathbf{h}\right)=\sigma(\mathcal{M}(\gamma(\mathbf{p},\mathbf{h}))W+b)$
(3)

where $W\in\mathbb{R}^{d\times 2},b\in\mathbb{R}^{2}$ are learnable
parameters, $\sigma$ is the softmax function, and applying $\mathcal{M}$ means
encoding the whole input sequence in a single $d$-dimensional vector according
to the specifics of the PLM. For BERT (Devlin et al., 2019) and its successor
RoBERTa (Liu et al., 2019), this implies a dense pooler layer with tanh
activation over the contextualized token embeddings and picking the first of
these embeddings (i.e., $\mathtt{[CLS]}$ for BERT and $\mathtt{\langle
s\rangle}$ for RoBERTa).222Cf. Jacob Devlin’s comment on issue 43 in the
official BERT repository on GitHub, https://github.com/google-
research/bert/issues/43. For training, we apply dropout with a probability of
$0.1$ to the output of $\mathcal{M}(\cdot)$.

Inference with multiple patterns. Previous work (Bouraoui et al., 2020;
Schmitt and Schütze, 2021) combined multiple patterns with the intuition that
different NL patterns can capture different aspects of the task. This
intuition makes also sense for conan. We conjecture that an efficient use of
the model parameters requires different continuous patterns to learn different
representations, which can detect different types of entailment. Following the
aforementioned work, we form our final score $s$ by combining the probability
estimates from different patterns $\Gamma$ by comparing the maximum
probability for the two classes $0,1$ over all patterns:

$\displaystyle m_{1}(\mathbf{p},\mathbf{h})$
$\displaystyle=\max_{\gamma\in\Gamma}P\left(\hat{y}=1\mid\gamma(\mathbf{p},\mathbf{h})\right)$
$\displaystyle m_{0}(\mathbf{p},\mathbf{h})$
$\displaystyle=\max_{\gamma\in\Gamma}P\left(\hat{y}=0\mid\gamma(\mathbf{p},\mathbf{h})\right)$
$\displaystyle s(\mathbf{p},\mathbf{h})$
$\displaystyle=m_{1}(\mathbf{p},\mathbf{h})-m_{0}(\mathbf{p},\mathbf{h})$ (4)

In conclusion, a conan model $\gamma^{n}_{k}$ is characterized by three
factors:  (i) The type of pattern
$\gamma\in\left\\{\,\alpha,\beta\,\right\\}$,  (ii) the number of patterns
$n\in\mathbb{N}$, and  (iii) the number of tokens $k\in\mathbb{N}$ per
pattern.

Training. While multiple patterns are combined for decision finding during
inference, we treat all patterns separately during training – as did previous
work (Schmitt and Schütze, 2021). So, given a set of patterns $\Gamma$, we
minimize the negative log-likelihood of the training data $\mathcal{T}$, i.e.,

$\displaystyle\sum_{(\mathbf{p},\mathbf{h},y)\in\mathcal{T}}\sum_{\gamma\in\Gamma}-\log(P\left(\hat{y}=y\mid\gamma(\mathbf{p},\mathbf{h})\right))$
(5)

In practice, we apply mini-batching to both $\mathcal{T}$ and $\Gamma$ and
thus compute this loss only for a fraction of the available training data and
patterns at a time. In this case, we normalize the loss by averaging over the
training samples and patterns in the mini-batch.

| train | dev | test | total
---|---|---|---|---
SherLIiC | 797 | 201 | 2,990 | 3,988
Levy/Holt | 4,388 | 1,098 | 12,921 | 18,407
Table 1: Number of labeled instances per data split.

Figure 1: auc on SherLIiC dev for different conan models; top = $\alpha$,
bottom = $\beta$; white/red/blue = similar to/better than/worse than $n=k=1$.
Note that $\alpha_{k}=\beta_{k}$ for $k\leq 2$.

##  3 Experiments

We conduct experiments on two established LIiC benchmarks, SherLIiC (Schmitt
and Schütze, 2019) and Levy/Holt (Levy and Dagan, 2016; Holt, 2018), using the
data splits as defined in (Schmitt and Schütze, 2021) for comparison. Both
benchmarks contain a majority of negative examples (SherLIiC: 67%, Levy/Holt:
81%), making the detection of the entailment (i.e., the minority) class a
particular challenge. See Table 1 for dataset and split sizes. Note that
Levy/Holt is nearly 5 times bigger than SherLIiC and still has less than 5k
train samples.

Following (Schmitt and Schütze, 2021), we use RoBERTa as underlying PLM and
also use the same hyperparameters whenever possible for comparison. Also
following Schmitt and Schütze (2021), we instantiate the typed placeholders
$A,B$ in SherLIiC with Freebase (Bollacker et al., 2008) entities, making sure
that $A$ and $B$ are not assigned the same entity. See Appendix A for full
training details.

We evaluate model performance with two metrics:  (i) The area under the
precision-recall curve for precision values $\geq$ 0.5 (auc) as threshold-less
metric using only the score $s$ defined in the previous section and  (ii) the
F1 score of actual classification decisions after tuning a decision threshold
$\vartheta$ on the respective dev portion of the data.  Our implementation is
based on (Wolf et al., 2019).

##  4 Results

Choosing $n$ and $k$. The number $n$ of patterns and the number $k$ of $C$
tokens per pattern are essential hyperparameters of a conan model.

Fig. 1 shows the impact on performance (measured in auc) on SherLIiC dev for
different $n$-$k$-combinations. We observe that using too many, too long
patterns harms performance w.r.t. the base case $n=k=1$. Best results are
obtained with either a small number of patterns or tokens or both. Comparing
the $\alpha$ and $\beta$ settings, we notice that, rounded to one decimal,
they produce identical results for both $n=1$ and $n=5$ patterns, suggesting
that the particular position of the $C$ tokens does not matter much in these
settings for SherLIiC. Even with $n=10$ patterns, the two methods only begin
to differ with $k\geq 5$ tokens per pattern. Evaluation on the Levy/Holt data
(see Fig. 2 in Appendix B) shows more variation between $\alpha$ and $\beta$
but, otherwise, confirms the trend that small $n$ and $k$ yield better
performance.

Our results offer an explanation for the empirical finding in (Schmitt and
Schütze, 2021) that patterns retrieved from a corpus lead to worse performance
than handcrafted ones because the latter are generally shorter. conan models
do not only yield better performance, they also provide an automatic way to
test pattern properties, such as length, w.r.t. effect on performance for a
given task.

|  | auc | P | R | F1
---|---|---|---|---|---
RoBERTa-base
manpatΦ | (baseline) | 69.2 | 62.0 | 81.2 | 70.3
autpat${}_{15}^{\Phi}$ | (baseline) | 73.1 | 63.0 | 77.4 | 69.4
conan${}^{5}_{2}$ | ($\vartheta=-0.0768$) | 73.2 | 65.8 | 84.1 | 73.8
conan${}^{50}_{1}$ | ($\vartheta=-0.0108$) | 74.0 | 70.1 | 78.0 | 73.8
conan-$\alpha^{1}_{7}$ | ($\vartheta=-0.1875$) | 66.9 | 68.7 | 68.4 | 68.5
conan-$\beta^{1}_{7}$ | ($\vartheta=-0.1875$) | 66.9 | 68.7 | 68.4 | 68.5
RoBERTa-large
nli | (baseline) | 68.3 | 60.5 | 85.5 | 70.9
$\textsc{manpat}^{\Phi\Psi}$ | (baseline) | 74.4 | 66.0 | 80.8 | 72.6
conan${}^{5}_{2}$ | ($\vartheta=-0.6838$) | 75.9 | 67.9 | 81.1 | 73.9
conan${}^{50}_{1}$ | ($\vartheta=-0.9999$) | 73.9 | 64.8 | 81.5 | 72.2
conan-$\alpha^{1}_{7}$ | ($\vartheta=-0.8797$) | 62.6 | 63.1 | 76.4 | 69.1
conan-$\beta^{1}_{7}$ | ($\vartheta=-0.9556$) | 68.6 | 60.8 | 86.0 | 71.2
Table 2: SherLIiC test. auc denotes the area under the precision-recall curve for precision $\geq$ 0.5. All results in %. Bold means best result per column and block. All baselines from (Schmitt and Schütze, 2021). For $k\leq 2$, we simply write conan${}^{n}_{k}$ because $\alpha=\beta$. |  | auc | P | R | F1
---|---|---|---|---|---
RoBERTa-base
nli | (baseline) | 72.6 | 68.7 | 75.3 | 71.9
$\textsc{manpat}^{\Phi\Psi}$ | (baseline) | 76.9 | 78.7 | 66.4 | 72.0
conan${}^{5}_{2}$ | ($\vartheta=-0.4809$) | 77.6 | 79.1 | 67.5 | 72.8
conan${}^{5}_{1}$ | ($\vartheta=-0.9003$) | 74.9 | 67.1 | 77.5 | 71.9
conan-$\alpha^{5}_{3}$ | ($\vartheta=-0.8985$) | 73.6 | 75.0 | 66.7 | 70.6
conan-$\beta^{5}_{3}$ | ($\vartheta=-0.9289$) | 76.4 | 76.6 | 69.2 | 72.7
RoBERTa-large
$\textsc{manpat}^{\Phi\Psi}$ | (baseline) | 83.9 | 84.8 | 70.1 | 76.7
$\textsc{manpat}^{\Phi}$ | (baseline) | 77.8 | 67.9 | 81.5 | 74.1
conan${}^{5}_{2}$ | ($\vartheta=-0.1315$) | 85.9 | 81.7 | 78.1 | 79.9
conan${}^{5}_{1}$ | ($\vartheta=-0.9750$) | 85.2 | 77.2 | 80.3 | 78.7
conan-$\alpha^{5}_{3}$ | ($\vartheta=-0.9212$) | 84.4 | 82.2 | 74.9 | 78.4
conan-$\beta^{5}_{3}$ | ($\vartheta=-0.9585$) | 85.3 | 78.8 | 77.3 | 78.0
Table 3: Levy/Holt test. All baselines from (Schmitt and Schütze, 2021). See
Table 2 for table format.

Test performance. On both SherLIiC (Table 2) and Levy/Holt (Table 3), and
across model sizes (base and large), conan${}^{5}_{2}$ (using either $\alpha$
or $\beta$ because they are identical for $k=2$) outperforms all other models
including the previous state of the art by Schmitt and Schütze (2021), who
fine-tune RoBERTa both without patterns (nli) and using handcrafted (manpat)
or automatically retrieved corpus patterns (autpat). We report their two best
systems for each benchmark.

We take the performance increase with continuous patterns as a clear indicator
that the flexibility offered by separating pattern tokens from the rest of the
vocabulary allows RoBERTa to better adapt to the task-specific data even with
only few labeled training instances in the challenging LIiC task.

##  5 Analysis and Discussion

Nearest neighbors. To further investigate how RoBERTa makes use of the
flexibility of $C$ tokens, we compute their nearest neighbors in the space of
original vocabulary tokens based on cosine similarity for our models in Tables
2 and 3. We always find the $C$ tokens to be very dissimilar from any token in
the original vocabulary, the highest cosine similarity being $0.15$. And even
among themselves, $C$ tokens are very dissimilar, nearly orthogonal, with
$0.08$ being the highest cosine similarity here. RoBERTa seems to indeed take
full advantage of the increased flexibility to put the $C$ tokens anywhere in
the embedding space. This further backs our hypothesis that the increased
flexibility is beneficial for performance.

Influence of additional parameters. One might argue that the vocabulary
extension and the resulting new randomly initialized token embeddings lead to
an unfair advantage for conan models because the parameter count increases.
While more parameters do generally lead to increased model capacity, the
number of new parameters is so small compared to the total number of
parameters in RoBERTa that we consider it improbable that the new parameters
are alone responsible for the improved performance. Of all models in Tables 2
and 3, conan${}^{50}_{1}$ introduces the most additional model parameters,
i.e., $1\cdot 50\cdot 768=38400$ for RoBERTa-base. Given that even the smaller
RoBERTa-base model still has a total of 125M parameters, the relative
parameter increase is maximally 0.03%, which, we argue, is negligible.

| auc | P | R | F1
---|---|---|---|---
SherLIiC train $\to$ Levy/Holt test
Schm&Schü (2021) (base) | 38.4 | 52.7 | 57.1 | 54.8
Schm&Schü (2021) (large) | 70.4 | 39.6 | 95.3 | 56.0
conan${}^{5}_{2}$ (base) | 54.3 | 38.2 | 89.8 | 53.5
conan${}^{5}_{2}$ (large) | 61.5 | 34.5 | 96.3 | 50.8
Levy/Holt train $\to$ SherLIiC test
Schm&Schü (2021) (base) | 63.3 | 62.8 | 68.4 | 65.5
Schm&Schü (2021) (large) | 62.1 | 68.1 | 57.3 | 62.3
conan${}^{5}_{2}$ (base) | 64.9 | 63.4 | 68.9 | 66.1
conan${}^{5}_{2}$ (large) | 70.1 | 69.0 | 69.5 | 69.2
Table 4: Transfer experiments ($\vartheta=0$). Best models from (Schmitt and
Schütze, 2021) according to F1 score.

Transfer between datasets. The experiments summarized in Table 4 investigate
the hypothesis that conan’s better adaptation to the fine-tuning data might
worsen its generalization abilities to other LIiC benchmarks. For this, we
train our best model conan${}^{5}_{2}$ on SherLIiC to test it on Levy/Holt and
vice versa. In this scenario, we assume that the target dataset is not
available at all. So there is no way to adapt to a slightly different domain
other than learning general LIiC reasoning. We thus set $\vartheta=0$ in these
experiments.

We find that with the very few train samples in SherLIiC the risk of
overfitting to SherLIiC is indeed higher. When trained on Levy/Holt with
around 4.4k train samples, however, conan clearly improves generalization to
the SherLIiC domain.

##  6 Related Work

PLMs and text patterns. GPT-2 (Radford et al., 2019) made the idea popular
that a PLM can perform tasks without access to any training data when prompted
with the right NL task instructions. With GPT-3, Brown et al. (2020) adapted
this idea to few-shot settings where the task prompt is extended by a few
training samples. While this kind of few-shot adaptation with a frozen PLM
only works with very big models, Schick and Schütze (2021b) achieve similar
performance with smaller models by fine-tuning the PLM on the available
training data and putting them into NL templates. Recently, Schmitt and
Schütze (2021) investigated the use of PLMs for LIiC. Compared to a standard
sequence classification fine-tuning approach, they were able to improve the
PLM RoBERTa’s performance by putting an entailment candidate into textual
contexts that only make sense for either a valid or invalid example. Patterns
like “$y$ because $x$.” (valid) or “It does not mean that $y$ just because
$x$.” (invalid) make intuitive sense to humans and outperform standard RoBERTa
on LIiC.

A large problem with all these approaches, however, is to find well-
functioning patterns, for which numerous solutions have been proposed (Shin et
al., 2020; Haviv et al., 2021; Bouraoui et al., 2020; Jiang et al., 2020; Gao
et al., 2021; Reynolds and McDonell, 2021). We argue that it is not optimal to
constrain pattern search to the space of NL sequences if the primary goal is
better task performance, and therefore abandon this constraint.

PLMs and continuous patterns. Li and Liang (2021) and Hambardzumyan et al.
(2021) contemporaneously introduced the idea of mixing the input token
embeddings of a PLM with other continuous vectors that do not correspond to
vocabulary elements. In the spirit of GPT-2 (see above), they keep the PLM’s
parameters frozen and only fine-tune the embeddings of the “virtual tokens” to
the target task. While this line of research offers certain appeals of its
own, e.g., reusability of the frozen PLM weights, this is not the focus of our
work. In pursuit of the best possible performance, we instead compare the use
of continuous vs. NL patterns in the process of fine-tuning all PLM parameters
and find that even carefully chosen NL patterns can be outperformed by our
automatically learned ones.

Contemporaneously to our work, Liu et al. (2021) fine-tune entire PLMs with
continuous patterns for SuperGLUE (Wang et al., 2019). Besides reformulating
the SuperGLUE tasks as cloze tasks, while we keep formalizing our task as
classification, Liu et al. (2021) also add more complexity by computing the
continuous token representations with an LSTM (Hochreiter and Schmidhuber,
1997) and adding certain “anchor tokens”, such as a question mark, at manually
chosen places. conan does not use any manual pattern design and embeds
continuous tokens with a simple lookup table.

Another contemporaneous work by Lester et al. (2021) tests the influence of
model size on the performance of a frozen PLM with trained continuous prompts.
Their prompt ensembling is akin to our combining multiple patterns during
inference (cf. Section 2). The key difference is that, instead of making
predictions with different patterns and taking the majority vote, we rather
compare the scores for different patterns to make our prediction.

##  7 Conclusion

We presented conan, a method that improves fine-tuning performance of a PLM
with continuous patterns. conan does not depend on any manual pattern design
and is efficient as the shortest possible patterns with good performance can
be found automatically. It provides an automatic way of systematically testing
structural properties of patterns, such as length, w.r.t. performance changes.
In our experiments on two established LIiC benchmarks, conan outperforms
previous work using NL patterns and sets a new state of the art.

## Acknowledgments

We gratefully acknowledge a Ph.D. scholarship awarded to the first author by
the German Academic Scholarship Foundation (Studienstiftung des deutschen
Volkes). This work was supported by the BMBF as part of the project MLWin
(01IS18050).

## References

  * an (2013) Tomas Mikolov an. 2013.  Efficient estimation of word representations in vector space.  _ArXiv preprint_ , abs/1301.3781.
  * Bollacker et al. (2008) Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008\.  Freebase: A collaboratively created graph database for structuring human knowledge.  In _Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data_ , SIGMOD ’08, page 1247–1250, New York, NY, USA. Association for Computing Machinery.
  * Bouraoui et al. (2020) Zied Bouraoui, José Camacho-Collados, and Steven Schockaert. 2020.  Inducing relational knowledge from BERT.  In _The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020_ , pages 7456–7463. AAAI Press.
  * Brown et al. (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language models are few-shot learners.  In _Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual_.
  * Dagan et al. (2013) Ido Dagan, Dan Roth, Mark Sammons, and Fabio Massimo Zanzotto. 2013.  _Recognizing textual entailment: Models and applications_.  Morgan & Claypool Publishers.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Efrat and Levy (2020) Avia Efrat and Omer Levy. 2020.  The turking test: Can language models understand instructions? _ArXiv preprint_ , abs/2010.11982.
  * Elazar et al. (2021) Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, and Yoav Goldberg. 2021.  Measuring and improving consistency in pretrained language models.  _ArXiv preprint_ , abs/2102.01017.
  * Gao et al. (2021) Tianyu Gao, Adam Fisch, and Danqi Chen. 2021.  Making pre-trained language models better few-shot learners.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 3816–3830, Online. Association for Computational Linguistics.
  * Glockner et al. (2018) Max Glockner, Vered Shwartz, and Yoav Goldberg. 2018.  Breaking NLI systems with sentences that require simple lexical inferences.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 650–655, Melbourne, Australia. Association for Computational Linguistics.
  * Hambardzumyan et al. (2021) Karen Hambardzumyan, Hrant Khachatrian, and Jonathan May. 2021.  WARP: Word-level Adversarial ReProgramming.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 4921–4933, Online. Association for Computational Linguistics.
  * Haviv et al. (2021) Adi Haviv, Jonathan Berant, and Amir Globerson. 2021.  BERTese: Learning to speak to BERT.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 3618–3623, Online. Association for Computational Linguistics.
  * Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber. 1997.  Long short-term memory.  _Neural Computation_ , 9(8):1735–1780.
  * Holt (2018) Xavier R. Holt. 2018.  Probabilistic models of relational implication.  Master’s thesis, Macquarie University.
  * Hosseini et al. (2019) Mohammad Javad Hosseini, Shay B. Cohen, Mark Johnson, and Mark Steedman. 2019.  Duality of link prediction and entailment graph induction.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 4736–4746, Florence, Italy. Association for Computational Linguistics.
  * Jiang et al. (2020) Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig. 2020.  How can we know what language models know? _Transactions of the Association for Computational Linguistics_ , 8:423–438.
  * Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015.  Adam: A method for stochastic optimization.  In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_.
  * Le Scao and Rush (2021) Teven Le Scao and Alexander Rush. 2021.  How many data points is a prompt worth? In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 2627–2636, Online. Association for Computational Linguistics.
  * Lester et al. (2021) Brian Lester, Rami Al-Rfou, and Noah Constant. 2021.  The power of scale for parameter-efficient prompt tuning.  _ArXiv preprint_ , abs/2104.08691.
  * Levy and Dagan (2016) Omer Levy and Ido Dagan. 2016.  Annotating relation inference in context via question answering.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 249–255, Berlin, Germany. Association for Computational Linguistics.
  * Li and Liang (2021) Xiang Lisa Li and Percy Liang. 2021.  Prefix-tuning: Optimizing continuous prompts for generation.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 4582–4597, Online. Association for Computational Linguistics.
  * Liu et al. (2021) Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. 2021.  Gpt understands, too.  _ArXiv preprint_ , abs/2103.10385.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _ArXiv preprint_ , abs/1907.11692.
  * McKenna et al. (2021) Nick McKenna, Liane Guillou, Mohammad Javad Hosseini, Sander Bijl de Vroe, and Mark Steedman. 2021.  Multivalent entailment graphs for question answering.  _ArXiv preprint_ , abs/2104.07846.
  * Meged et al. (2020) Yehudit Meged, Avi Caciularu, Vered Shwartz, and Ido Dagan. 2020.  Paraphrasing vs coreferring: Two sides of the same coin.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 4897–4907, Online. Association for Computational Linguistics.
  * Radford et al. (2019) Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019.  Language models are unsupervised multitask learners.  Technical report, OpenAI, San Francisco, California, United States.
  * Reynolds and McDonell (2021) Laria Reynolds and Kyle McDonell. 2021.  Prompt programming for large language models: Beyond the few-shot paradigm.  _ArXiv preprint_ , abs/2102.07350.
  * Schick and Schütze (2021a) Timo Schick and Hinrich Schütze. 2021a.  Exploiting cloze-questions for few-shot text classification and natural language inference.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 255–269, Online. Association for Computational Linguistics.
  * Schick and Schütze (2021b) Timo Schick and Hinrich Schütze. 2021b.  It’s not just size that matters: Small language models are also few-shot learners.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 2339–2352, Online. Association for Computational Linguistics.
  * Schmitt and Schütze (2019) Martin Schmitt and Hinrich Schütze. 2019.  SherLIiC: A typed event-focused lexical inference benchmark for evaluating natural language inference.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 902–914, Florence, Italy. Association for Computational Linguistics.
  * Schmitt and Schütze (2021) Martin Schmitt and Hinrich Schütze. 2021.  Language models for lexical inference in context.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 1267–1280, Online. Association for Computational Linguistics.
  * Schoenmackers et al. (2010) Stefan Schoenmackers, Jesse Davis, Oren Etzioni, and Daniel Weld. 2010.  Learning first-order horn clauses from web text.  In _Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing_ , pages 1088–1098, Cambridge, MA. Association for Computational Linguistics.
  * Shin et al. (2020) Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. 2020.  AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 4222–4235, Online. Association for Computational Linguistics.
  * Shwartz et al. (2017) Vered Shwartz, Gabriel Stanovsky, and Ido Dagan. 2017.  Acquiring predicate paraphrases from news tweets.  In _Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017)_ , pages 155–160, Vancouver, Canada. Association for Computational Linguistics.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA_ , pages 5998–6008.
  * Wang et al. (2019) Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019.  Superglue: A stickier benchmark for general-purpose language understanding systems.  In _Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada_ , pages 3261–3275.
  * Wolf et al. (2019) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2019.  Huggingface’s transformers: State-of-the-art natural language processing.  _ArXiv preprint_ , abs/1910.03771.

##  Appendix A Training Details

We train our model for 5 epochs on a single GeForce RTX 2080 Ti GPU, with Adam
(Kingma and Ba, 2015) and a mini-batch size of 10 (resp. 2) training instances
for RoBERTa-base (resp. -large) and mini-batches of 5 patterns. We adopt well-
functioning values from (Schmitt and Schütze, 2021) for all non-conan-specific
hyperparameters, i.e., learning rate $\mathit{lr}$, weight decay $\lambda$,
and accumulated batches $c$ before a gradient update: Concretely, we set
$\mathit{lr}=2.28\cdot 10^{-5},\lambda=6.52\cdot 10^{-2},c=2$ for evaluating
RoBERTa-base on SherLIiC, $\mathit{lr}=1.29\cdot 10^{-5},\lambda=2.49\cdot
10^{-4},c=3$ for RoBERTa-large on SherLIiC, $\mathit{lr}=2.72\cdot
10^{-5},\lambda=1.43\cdot 10^{-3},c=1$ for RoBERTa-base on Levy/Holt, and
$\mathit{lr}=4.55\cdot 10^{-6},\lambda=3.90\cdot 10^{-4},c=2$ for RoBERTa-
large on Levy/Holt.

##  Appendix B More Dev Results

Figure 2: auc on Levy/Holt dev for different conan models. Same format as Fig.
1.

See Fig. 2 for evaluation results on Levy/Holt dev.