11institutetext:  Theoretical Physics, University of Oxford, 1 Keble Road,
Oxford, OX1 3NP, United Kingdom
Institut de Physique Théorique, CEA, IPhT, F-91191 Gif-sur-Yvette, France and
CNRS, URA 2306
Science and Finance, Capital Fund Management, 6 Bd Haussmann, 75009 Paris,
France

Theory and modeling of the glass transition Dynamic critical phenomena
Equilibrium properties near critical points, critical exponents

# Mode-coupling as a Landau theory of the glass transition

A. Andreanov 11 G. Biroli and J.-P. Bouchaud 2233112233

###### Abstract

We derive the Mode Coupling Theory (MCT) of the glass transition as a Landau
theory, formulated as an expansion of the exact dynamical equations in the
difference between the correlation function and its plateau value. This sheds
light on the universality of MCT predictions. While our expansion generates
higher order non-local corrections that modify the standard MCT equations, we
find that the square root singularity of the order parameter, the scaling
function in the $\beta$ regime and the functional relation between the
exponents defining the $\alpha$ and $\beta$ timescales are universal and left
intact by these corrections.

###### pacs:

64.70.Q-

###### pacs:

64.60.Ht

###### pacs:

64.60.F-

The Mode-Coupling Theory of glasses (MCT), developed since the mid-eighties
following the seminal work of Götze [1] and Leutheusser [2], has significantly
contributed to our understanding of the slowing down of supercooled liquids.
One of its cardinal predictions is the appearance of a non trivial
$\beta$-relaxation regime where dynamical correlation functions pause around a
plateau value before finally relaxing to zero. In the vicinity of this plateau
value, the theory predicts two power-law regimes in time (or in frequency),
and the divergence of two distinct relaxation times, $\tau_{\alpha}$ and
$\tau_{\beta}$, at the MCT critical temperature $T_{d}$. Although this
divergence is smeared out by activated events in real liquids, the two-step
relaxation picture suggested by MCT seems to account quite well for
experimental and numerical observations [3], at least in weakly supercooled
liquids and for hard sphere colloidal systems.

Originally, MCT was obtained as an uncontrolled self-consistent approximation
within the Mori-Zwanzig projection operator formalism for Newtonian particles.
This scheme yields an integro-differential equation for the dynamic structure
factor $C({\bf k},t)$ that captures mathematically the slowing down of the
dynamics and the appearance of a two-step relaxation at equilibrium. It
provides very detailed predictions for the scaling properties of $C({\bf
k},t)$ in the vicinity of the plateau value $f_{{\bf k}}S_{{\bf k}}$, where
$S_{{\bf k}}$ is the static structure factor and $f_{{\bf k}}$ is called the
non-ergodicity parameter (akin to the Edwards-Anderson parameter in spin
glasses). Alternative derivations of MCT based on field theory have been
sought for [4] and research on this topic has continued until now [5, 6, 7,
8]. It was also realized that the same integro-differential equations describe
the exact evolution of the correlation function of mean-field p-spin glasses
[9]. This is important for at least two reasons:

  * •

Technically, it shows that the MCT approximation is realizable: there is a
well defined system for which it is exact; hence MCT does not violate basic
physical constraints.

  * •

Physically, it brings in a very useful interpretation of the MCT freezing
transition in the “energy landscape” parlance. Above the transition $T_{d}$,
the dynamics is dominated by unstable saddles that become progressively less
and less unstable as one approaches the transition; below the transition,
there are only local minima that are separated by infinite barriers in mean-
field, so that the system is forever trapped in one of them [9]. For non mean-
field systems, these barriers are finite and the transition is smoothed. MCT
can be naturally embedded within the broader Random First Order Theory [10] of
the glass transition. In this context it describes the high temperature region
where metastable states are still in embryo.

However, this analogy shows that MCT is (at best) an incomplete theory of real
supercooled liquids, and needs to be corrected and enhanced. An important
question is to know to what extent the quantitative predictions of MCT are
stable with respect to the ignored contributions. In fact, it is genrally
accepted that the quantitative value of the critical temperature (or the
critical density) obtained solving MCT equations is incorrect, as is the
interaction parameter $\lambda$, which is a functional of the static structure
factor and fixes the value of critical exponents. However, when MCT is used to
fit empirical data, it is assumed (without much justification) that the
predictions about the critical behavior remain correct if $T_{d}$ and
$\lambda$ are treated as adjustable parameters. This implicitly assumes that
some MCT predictions are universal, e.g. the square root singularity of
$f_{{\bf k}}$ and the relation between the exponents describing the divergence
of $\tau_{\alpha}$ and $\tau_{\beta}$, and others which are not, e.g. $T_{d}$
and the actual value of the exponents!

Clearly, the present theoretical understanding of MCT needs to be improved.
Assessing the degree of structural stability of the theory and its
universality properties is a crucial issue to resolve both for theoretical and
practical purposes. One step in this direction has been performed by Szamel
[11] and then later generalized by Mayer et al., where a schematic version of
MCT including higher order correlations was proposed and analyzed [12]. The
result is that whenever the theory is truncated at any finite order in the
n-boby correlations, the phenomenology of MCT is exactly recovered with a
finite $T_{d}$, whereas $T_{d}=0$ when the theory is treated exactly to all
orders. Another approach was followed in [6], based on a field-theoretical
formulation of MCT consistent with the Fluctuation-Dissipation Theorem, which
suggests closure schemes different from standard MCT.

The aim of this work is to argue that MCT can be rephrased as a Landau theory
of the glass transition, based on general assumptions about the nature of the
dynamical arrest but without relying on any particular model. Therefore, some
predictions of MCT are indeed generic and should be useful in a certain regime
of time and temperature. Note that a Landau theory for the glass transition
has been developed also in [13], but it has a very different starting point
and it does not focus on MCT.

The Landau theory is a general phenomenological approach to equilibrium phase
transitions [14, 15]. It relies on a number of natural hypotheses, such as
symmetry, genericity and regularity. In the classic example of the Ising model
for ferromagnets, the expansion of the free energy as a function of the
magnetisation $m$ reads, in the homogeneous case and subject to external field
$h$: $F[m]=F_{0}-mh+\frac{b}{2}(T-T_{c})m^{2}+\frac{g}{4!}m^{4}+...$, from
which a certain number of well known mean-field properties can be derived.
Including the first gradient corrections in the inhomogeneous case also allows
one to show that close to $T_{c}$, the divergence of the uniform
susceptibility $\chi({\bf q}=0)$ is accompanied by the divergence of the
correlation length $\xi$, over which magnetisation fluctuations are
correlated. The Landau construction can falter in three distinct ways:

  1. 1.

Higher order terms, neglected in the expansion of $F[m]$ as a series of $m$,
could qualitatively change the above predictions (structural instability).
This happens, for example, close to a multicritical point where $g(T_{c})=0$.
But if the transition remains second order, higher order terms are truly
negligible when $\epsilon=|T-T_{c}|\to 0$ and the predictions are universal.

  2. 2.

The non-linear feedback of spatial fluctuations on the divergence of the
susceptibility can change all the critical exponents when the dimension of
space is smaller than $d_{u}=4$ in the case of the Ising model. For $d>d_{u}$,
on the other hand, one can prove that the low-$q$ behaviour of $\chi({\bf q})$
is (close to the critical point) identical to that predicted by Landau’s
theory.

  3. 3.

Non-perturbative effects can wipe out the transition. This is the case for
example of the spinodal transition: the system is not able to reach the
critical point because of nucleation, which is an activated process.

Even the analogue of point (1) is difficult in the case of MCT; the basic
reason being that the order parameter is a not a scalar, but it is a time
dependent function $C({\bf k},t)$. The proof that MCT is structurally stable
with respect to the addition of higher order terms is already quite complex
and this will be the scope of the present paper. Once this is achieved, one
should still worry about points (2) and (3) above. As already mentioned, it
was recently realized that diverging fluctuations and an upper critical
dimension $d_{u}$ also exist for MCT [16, 17, 18] (see also [19, 20] for
earlier insights). In order to complete our proof that MCT is stable, one
should prove that spatial fluctuations can be safely neglected in $d>d_{u}$
and understand how close one can get to the critical point before non-
perturbative (activated) effects impair the transition. We will completely
disregard these issues in the present paper, and focus only on point (1).

The case of the glass transition is quite different from standard critical
phenomena. Several physical and formal problems prevent a direct analogy. The
glass transition seems to be a purely dynamical phenomenon: simple static,
thermodynamical properties do not present any peculiarities as the liquid
freezes into a glass 111It has however been argued that highly non-trivial
static correlations, called point to set correlations, increase approaching
the glass transition [21].. The above Landau construction simply does not make
sense in the absence of the clear analogy of the free energy. This means that
the order parameter in glasses cannot be a one point function (such as the
magnetisation) but, instead, it is likely to be a two point dynamic
correlation. The slowing down of the dynamics in glasses is found to be
accompanied by the appearance of a plateau value $f_{\bf k}$ in the relaxation
pattern of the dynamical structure factor $C({\bf k},\tau)$. Since the
appearance of a plateau coincides with increasing time scales, one expects
that within a very long time interval (to be specified), the correlation
function can be approximately written as:

$C({\bf k},\tau)\approx f_{\bf k}S_{\bf k}+\delta C({\bf k},\tau),\qquad\delta
C({\bf k},\tau)\ll 1.$ (1)

The idea underlying our construction of a Landau theory for glasses is to
consider $\delta C({\bf k},\tau)$ as the analogue of the order parameter $m$
and construct a general, structurally stable, dynamical equation for $\delta
C({\bf k},\tau)$. A way to construct such an equation is to start from the
exact dynamical evolution for $C$ and the response function $R$ that can be
derived in the framework of various dynamical field theories, for example
based on Dean’s equation [22] for Brownian dynamics, on Fluctuating
Hydrodynamics for Newtonian dynamics [4] (see also [5, 6, 7, 8]), or on
Langevin equations for $p$-spin models [9]. Although these theories give very
different sets of equations, they can all be reduced to the following single
equation in the ergodic region:

$\partial_{\tau}C({\bf k},\tau)+TC({\bf
k},\tau)+\int\limits_{0}^{\tau}du\Sigma({\bf k},\tau-u)\partial_{u}C({\bf
k},u)=0$ (2)

with initial condition $C({\bf k},0)=S_{\bf k}$. The self-energy $\Sigma({\bf
k},\tau)$ (or memory kernel in MCT terminology) is given by the sum of
$2$-particle irreducible ($2$-PI) diagrams built with $C$ and $R$ lines, see
e.g. [6]. We do not specify the details of the field theory underlying this
equation, nor the Feynman rules for the diagrams contributing to $\Sigma$: we
just need that such a theory exists. We also stay in the high-$T$ region, so
that the system is at equilibrium: both $C$ and $R$ are then time translation
invariant and the Fluctuation-Dissipation theorem holds at a diagrammatic
level: $T\,R({\bf k},\tau)=-\partial_{\tau}C({\bf k},\tau)$. In this case the
self-energy is a functional of the correlation function only. Eq. (2) has
exactly the structure of the standard MCT equation for liquids, although there
is no well defined prescription to build a consistent approximation for
$\Sigma$; in this sense MCT is rather arbitrary and difficult to improve upon
in a systematic way. The standard MCT results correponds to a self-consistent
$1$-loop approximation for $\Sigma$, $\Sigma({\bf k},t-s)={\int\frac{d^{3}{\bf
k}}{(2\pi)^{3}}}V({\bf k},{\bf p})C({\bf k}-{\bf p},t-s)C({\bf p},t-s)$, where
$V({\bf k},{\bf p})$ is an effective vertex. 222Note however that there are
complications related to fluctuation dissipation relations [5, 6, 7, 8]. When
generalized to higher order diagrams, an important difficulty emerges: the
non-locality in time of the corrections. Our goal is to prove that the main
results of the standard MCT (or $1$-loop) approximation still hold. The proof
is done as for usual theories. We start from some conjectures about the
critical properties, such as the nature of the order parameter and its
critical properties, that are motivated by experimental and numerical
findings. Then we show that they result from a Landau-like expansion, which
allows one to assess their universal character and to fix the value of the
critical exponents. We therefore assume that the order parameter is the
dynamical correlation function and that displays the following features:

  * •

There is structural arrest: below some temperature $T_{d}$,
$\lim\limits_{\tau\to\infty}C({\bf k},\tau)=f_{\bf k}S_{\bf k}$ with $f_{\bf
k}>0$.

  * •

When $\epsilon=(T-T_{d})/T_{d}\ll 1$, the correlation function exhibits a two-
step pattern with three well separated characteristic time scales, see Fig 1.
We assume that there exists a diverging time scale $\tau_{\beta}(\epsilon)$
where the difference $\delta C({\bf k},\tau)$ between $C$ and the plateau
value is small, of order $r(\epsilon)\ll 1$. More precisely, the correlation
function decay is decomposed into: (i) a short time regime,
$\tau\sim\tau_{0}$, where $C({\bf k},\tau)=C_{0}({\bf k},\tau)$, with
$C_{0}({\bf k},\tau\gg 1)\to f_{\bf k}S_{\bf k}$: (ii) a $\beta$-regime,
$\tau=s\tau_{\beta}(\epsilon)$ with $s=O(1)$: $\delta C({\bf
k},s\tau_{\beta})=r(\epsilon)S_{\bf k}(1-f_{\bf k})^{2}G({\bf k},s)$; (iii) an
$\alpha$-regime, $\tau=s^{\prime}\tau_{\alpha}(\epsilon)$ with
$s^{\prime}=O(1)$: $C({\bf k},s^{\prime}\tau_{\alpha})=C_{\alpha}({\bf
k},s^{\prime})$ describes the final fall off of the relaxation.

Figure 1: A set of relaxation curves $C(\tau)$ close to the MCT transition for
the so called schematic model of MCT [3] and several values of $\lambda$,
which quantifies the deviation from the transition. As $\lambda\to 1$ a
shoulder develops in the relaxation pattern corresponding to the emerging
$\beta$-relaxation regime.

We assume further (and justify later) that the function $G({\bf k},s)$ can
itself be expanded in powers of $r(\epsilon)$: $G({\bf
k},s)=\sum_{n=1}^{\infty}r^{n-1}G_{n}({\bf k},s)$. 333One could have also some
regular (in $\epsilon$) contributions. However, as we shall show the only two
possible values of $r(\epsilon)$ are $\sqrt{\epsilon}$ or $\epsilon$. As a a
consequence, regular contributions will be automatically contained in the
expansion. All functions $G_{n}$ are a priori singular at $s=0$ and
$s=\infty$, reflecting the fact that the behaviour of $C$ must match the short
time regime and $\alpha$ regime, where the deviation from the plateau ceases
to be small. A crucial remark for the following is that any function $G_{n}$
will appear with a prefactor $r^{n}(\epsilon)$. These hypotheses turn out to
be sufficient to generalize the MCT results in the $\beta$-regime. First, it
is clear that the above expansion of $G({\bf k},s)$ generates a similar
expansion of the self-energy $\Sigma[C]$ in the $\beta$-regime: $\Sigma({\bf
k},s)=\sum_{n=1}^{\infty}r^{n-1}\Sigma_{n}({\bf k},s)$, where the $\Sigma_{n}$
do not depend on $\epsilon$, but are some functionals of $C({\bf p},\tau)$.
The most generic functional form for $\Sigma$ a priori includes contributions
from all three regimes:

$\Sigma({\bf k},s)=\Sigma[\\{C_{0}({\bf
p},s^{\prime}\tau_{\beta}),r(\epsilon)G({\bf p},s^{\prime}),C_{\alpha}({\bf
p},s^{\prime}\tau_{\beta}/\tau_{\alpha})\\}],$ (3)

but since the $\Sigma_{n}$ should not depend on $\epsilon$, general arguments
can be used to restrict the actual functional form of $\Sigma_{n}$. Note that
we have used the notation $s^{\prime}$ to stress that this equation is a
functional relation which is non-local in time. Also, even if we had assumed
that $\delta C({\bf k},s\tau_{\beta})$ only contains a single term of the
order of $r(\epsilon)$ then we would have generated corrections of all orders
in $r(\epsilon)$ anyway. The reason is that the self-energy will contain all
orders in $r(\epsilon)$ as it can be found by expanding the above equation to
all order in $r(\epsilon)G({\bf p},s)$; this will feed back, via the
Schwinger-Dyson equations, on $G({\bf p},s)$ itself.

We now illustrate how this works for the lowest order terms $\Sigma_{0}$,
$\Sigma_{1}$ and $\Sigma_{2}$. As we shall see higher orders are in fact
irrelevant for our purpose. Clearly, the zeroth order term $\Sigma_{0}({\bf
k},s)$ can only be a function of the wavevector ${\bf k}$ since in the limit
$\epsilon\to 0$ time scales separate: $\tau_{\beta}\to\infty$ and
$\tau_{\beta}/\tau_{\alpha}\to 0$ and therefore the previous equation implies
that in $\Sigma_{0}({\bf k},s)$ all dependence on $s$ drops out. The first
order contribution $\Sigma_{1}({\bf k},s)$ must read:

$\Sigma_{1}({\bf k},s)=\int\limits_{0}^{\infty}du\int_{{\bf p}}K_{1}({\bf
k},{\bf p};s,u)G_{1}({\bf p},u)\quad$ (4)

where, henceforth, we shall use the notation $\int_{{\bf
p}}={\int\frac{d^{3}{\bf p}}{(2\pi)^{3}}}$. Any other combination containing
some $G_{n}$ gives an extra factor $r^{n}(\epsilon)$ and thus it corresponds
to a higher order contribution. In the original time variables, the kernel
$K_{1}$ must have some regular shape with a span fixed by the microscopic time
scale. Therefore, in the rescaled variables $u,s$, $K_{1}$ must be local in
$u-s$ and, to lowest order, is a $\delta$ function; higher derivatives of the
$\delta$ function correspond to corrections smaller by at least a factor
$\tau_{0}/\tau_{\beta}$ which, as we shall see, turn out to be negligible even
at order $r^{2}$. Therefore, $\Sigma_{1}({\bf k},s)={\int\frac{d^{3}{\bf
p}}{(2\pi)^{3}}}K_{1}({\bf k},{\bf p})G_{1}({\bf p},s)$.

The second order term has a richer structure. First, there is a term similar
to the first order one with $G_{2}$ instead of $G_{1}$: ${\int\frac{d^{3}{\bf
p}}{(2\pi)^{3}}}K_{2}({\bf k},{\bf p})G_{2}({\bf p},s)$. But since the kernel
$K_{2}$ is obtained, as $K_{1}$, from the first derivative of the self energy
with respect to $r(\epsilon)G({\bf p},s)$, one finds $K_{2}=K_{1}$. Second,
there are terms quadratic in $G_{1}$:

$\int_{{\bf k}_{1},{\bf
k}_{2}}\int\limits_{0}^{\infty}du\int\limits_{0}^{\infty}dvK_{11}({\bf q},{\bf
k}_{1},{\bf k}_{2};s,u,v)G_{1}({\bf k}_{1},u)G_{1}({\bf k}_{2},v)$

For the same reasons outlined above, time dependence of $K_{11}({\bf q},{\bf
k}_{1},{\bf k}_{2};s,u,v)$ is composed of $\delta$-functions and their
derivatives. Some thinking about the underlying diagrammatic structure of the
theory allows one to be convinced that the general structure of $K_{11}$ is,
to leading order:

$\displaystyle K_{11}(s,u,v)=K_{11,\ell}\delta(s-u)\delta(s-v)+\hskip
82.51282pt$ $\displaystyle
K_{11,n\ell}\delta(u+v-s)(\partial_{u}+\partial_{v})+\tilde{K}_{11,n\ell}(\partial_{u}+\partial_{v})\delta(u+v-s)$

where to simplify the notation we have dropped all wave-vector dependence in
the above equation. The fact that only the combination $u+v-s$ enters comes
from causality and the separation of time scales. The full justification of
the above form and other technical details444In particular one could think
that the separation of timescales not only leads to delta function terms but
also to constants. The latter are absent. This can be shown using the general
field theoretical expression of the self energy, see [23]. are presented in
[23]. The first local term (in $s$), $K_{11,\ell}$, is like the usual MCT
contribution, but the other two terms do not appear within standard MCT. The
third term actually reduces to the second one plus local terms via integration
by parts.

This expansion is the main result in the construction of the Landau theory.
The zeroth order equation in $r(\epsilon)$ fixes the non-ergodic parameter
such that $\frac{T_{d}f_{\bf k}}{1-f_{\bf k}}=\Sigma_{0}({\bf k})$, as in
standard MCT. Substituting the expansion of $\Sigma$ up to second order in $r$
into (2) and using the expansion of $C$ in the $\beta$ timescale, one finally
obtains, for $T=T_{d}(1+\epsilon)$ and in Laplace space within the
$\beta$-regime (we have dropped zeroth order, as discussed above):

$\displaystyle r\left[T_{d}(1+\epsilon)\hat{G}_{1}({\bf k},z)-\int_{\bf
p}K_{1}({\bf k},{\bf p})\hat{G}_{1}({\bf p},z)\right]+$ $\displaystyle
r^{2}\left[T_{d}(1+\epsilon)\hat{G}_{2}({\bf k},z)-\int_{\bf p}K_{1}({\bf
k},{\bf p})\hat{G}_{2}({\bf p},z)\right]+$ $\displaystyle\frac{T_{d}f_{\bf
k}\epsilon}{z(1-f_{\bf k})}+r^{2}T_{d}(1+\epsilon)(1-f_{\bf
k})z\hat{G}_{1}^{2}({\bf k},z)=$ $\displaystyle r^{2}\int_{\bf p}K_{2}({\bf
k},{\bf p})\hat{G}_{2}({\bf p},z)+$ $\displaystyle+$ $\displaystyle
r^{2}\int_{{\bf k}_{1}}\int_{{\bf k}_{2}}K_{11,\ell}({\bf k},{\bf k}_{1},{\bf
k}_{2}){\cal L}[G_{1}({\bf k}_{1},\tau)G_{1}({\bf k}_{2},\tau)](z)$
$\displaystyle+$ $\displaystyle r^{2}\int_{{\bf k}_{1}}\int_{{\bf
k}_{2}}K_{11,n\ell}({\bf k},{\bf k}_{1},{\bf k}_{2})z\hat{G}_{1}({\bf
k}_{1},z)\hat{G}_{1}({\bf k}_{2},z),$

Identifying the coefficients order by order produces a series of equations.
The first order fixes the yet unknown function $r(\epsilon)$: the expansion
(Mode-coupling as a Landau theory of the glass transition) only contains terms
with integer powers of $\epsilon$. They should be matched with powers of
$r(\epsilon)$. Inspection of (Mode-coupling as a Landau theory of the glass
transition) shows that there are two possibilities555Actually, there are other
possibilities that correspond to higher order MCT singularities, which have
been called $A_{n}$ [27]. In a usual Landau theory these correspond to
tricritical, or even higher order, critical points. We will neglect them here
since they require some fine tuning of the coupling constants.: either
$r=\epsilon$, or $r=\sqrt{\epsilon}$. The first choice yields a time
independent solution for $G_{1}$ which is in contradiction with our hypothesis
of a two-step relaxation with diverging time scales. Hence
$r(\epsilon)=\sqrt{\epsilon}$, precisely as for usual MCT, or at $1$-loop
order. This follows from the presence of a regular in $T$ term in (2).

The equation to order $r=\sqrt{\epsilon}$ now reads:

$T_{d}\hat{G}_{1}({\bf k},z)={\int\frac{d^{3}{\bf p}}{(2\pi)^{3}}}K_{1}({\bf
k},{\bf p})\hat{G}_{1}({\bf p},z)$ (6)

This is the standard eigenvalue problem found within MCT that fixes the value
of the critical temperature $T_{d}$. It constrains $G_{1}$ to be a product of
wave-vector dependent and time dependent amplitudes, thus reproducing the
well-know MCT “factorization property”: $\hat{G}_{1}({\bf
k},z)=\hat{g}(z)H_{1}({\bf k})$, where $H_{1}$ is the right eigenvector of
$K_{1}$ with largest eigenvalue $\Lambda=T_{d}$. At this order however the
scaling function $\hat{g}(z)$ remains unfixed. The second order equation is
trickier:

$\displaystyle T_{d}\hat{G}_{2}({\bf k},z)-\int_{\bf k}K_{1}({\bf k},{\bf
p})\hat{G}_{2}({\bf p},z)=$ $\displaystyle-\frac{T_{d}f_{\bf k}}{z(1-f_{\bf
k})}-T_{d}(1-f_{\bf k})z\hat{g}^{2}(z)H_{1}^{2}({\bf k})+$ (7)
$\displaystyle+\int_{{\bf k}_{1}}\int_{{\bf k}_{2}}K_{11,\ell}({\bf k},{\bf
k}_{1},{\bf k}_{2}){\cal L}[g^{2}](z)H_{1}({\bf k}_{1})H_{1}({\bf k}_{2})+$
$\displaystyle+z\hat{g}^{2}(z)\int_{{\bf k}_{1}}\int_{{\bf
k}_{2}}\hat{K}_{11,n\ell}({\bf k},{\bf k}_{1},{\bf k}_{2})H_{1}({\bf
k}_{1})H_{1}({\bf k}_{2})$

Following [24], we now multiply (Mode-coupling as a Landau theory of the glass
transition) by $H_{1}({\bf k})$ and integrate over ${\bf k}$. The $G_{2}$ part
of the equation vanishes and the remainder yields an equation on $\hat{g}(z)$.
After some algebra and a proper rescaling of $z$ and $\hat{g}$ one finds:

$\frac{1}{z}+\frac{z}{\lambda}\hat{g}^{2}(z)={\cal L}[g^{2}](z)$ (8)

where $\lambda$ is a constant that includes a non-local contribution as
compared to MCT. But the structure of the equation on the scaling function $g$
is exactly the same as in standard MCT. The properties of solution are well
known: $g$ has a singular power law asymptotics at $z\to\infty$:
$\hat{g}(z)\sim z^{a-1}$ and $z\to 0$: $\hat{g}(z)\sim z^{-1-b}$. The small
time exponent $a$ and long time exponent $b$ characterize the decay of $\delta
C({\bf k},\tau)$ to and away from the plateau $f_{\bf k}$. The exponents $a$
and $b$ are related by the famous equation:

$\frac{\Gamma^{2}(1-a)}{\Gamma(1-2a)}=\frac{\Gamma^{2}(1+b)}{\Gamma(1+2b)}=\lambda,$
(9)

which is a genuinely non-trivial and clear-cut prediction of MCT that
constrains the range of values of $a$ and $b$ to: $0\leq a<1/2$ and $0\leq
b\leq 1$, in good agreement with experimental and numerical results. We have
thus found that this relation has a much broader degree of validity and
survives the introduction of an arbitrary number of loop corrections. The
values of $a$ and $b$ are however different from the standard MCT (or
$1$-loop) result, but as alluded to above, the parameter $\lambda$ is usually
taken as an adjustable parameter anyway.

The fact that the form of the scaling function $g$ is the same as at $1$-loop
(MCT) has two consequences. First, it fixes the functional dependence of the
time scales exactly as for MCT:

$\tau_{\beta}=\epsilon^{-1/2a};\qquad\tau_{\alpha}=\epsilon^{-\gamma},$ (10)

$\gamma=1/2a+1/2b$. This is clear from the matching of $C({\bf
k},s\tau_{\beta})$ at both ends of the $\beta$ regime. Second, it can be used
to show that some superficial divergencies encountered in the calculation are
in fact innocuous (see [23] for more details). Note that the only extra
contribution that appear in the generic case to second order in $g(s)$, namely
the non-local term proportional to $\int dug^{\prime}(u)g(s-u)$, does not
modify the basic MCT equation, Eq. (8).

The conclusion, which is the main result of this work, is that although
$T_{d}$ and $\lambda$ are modified by taking into account corrections to MCT,
the relation between the exponent $a$ and $b$, the square root singularity as
well as the scaling function $g$ are truly universal properties. This
universality with respect to higher order local (in time) corrections was of
course already shown by Götze long ago; here we have proven that this result
is robust with respect to general non-local corrections as well, and
suggesting that MCT has the status of a Landau theory of the glass transition
666As a consequence, it has a very different status compared to Mode-Coupling
theories developed to compute critical exponents beyond mean field theory for
standard critical phenomena..

The above schematic arguments can be made precise within the context of
specialized model. We have in particular studied in full details the finite
$N$ corrections to mean-field 3-spin glass model, where the structure of the
perturbation theory can be used to check that the above conclusions hold in
that case, see [23] for details.

It was recently understood how MCT equations should be generalized in the
presence of spatial inhomogeneities, where the correlation function $C$ can be
space dependent: $C({\bf k},\vec{r};\tau)$, where $\vec{r}$ is the average of
the two points $\vec{r}_{1},\vec{r}_{2}$ between which the correlation is
computed, and ${\bf k}$ is the Fourier vector corresponding to
$\vec{r}_{1}-\vec{r}_{2}$. When wavelength of inhomogeneities is large, one
can establish a gradient expansion of the MCT equations. In the schematic
limit where all dependence on ${\bf k}$ is discarded, the self-energy reads,
to the lowest order:
$\Sigma[C](s)=C(\vec{r},s)^{2}+w_{1}C(\vec{r},s)\nabla^{2}C(\vec{r},s)+w_{2}\vec{\nabla}C(\vec{r},s)\cdot\vec{\nabla}C(\vec{r},s),$
where $w_{1}$ and $w_{2}$ are some coefficients [17]. As mentioned in the
introduction, these gradient terms are very important because they show how
the MCT transition is in fact associated with a diverging correlation length,
which corresponds to the scale over which a localized perturbation affects the
surrounding dynamics [17]. The long-ranged critical fluctuations renormalize
the value of the MCT exponents in $d<d_{u}=8$ [18]. The above analysis, which
was done in the homogeneous limit $\nabla\to 0$, should be repeated in the
inhomogeneous case to complete our proof. We expect that the same conclusion
will hold, namely that the results about dynamical correlation obtained within
inhomogeneous MCT [17] are stable against the addition of higher order
corrections.

In conclusion, we have shown that MCT, which describes a specific slowing down
mechanism through the progressive disappearance of unstable directions, has
the status of a Landau theory and is therefore expected to make generic
predictions, albeit polluted by activated events and critical fluctuations in
finite dimensions. The interplay between critical fluctuations and activated
events when $d<d_{u}$, and the crossover to low temperature dynamics is still
largely an exciting open problem [28]. Note also that even for the exact MCT
equations, the critical region where the asymptotic scaling predictions are
valid is unusually narrow [25, 26]. It would be interesting to generalize our
Landau approach to the aging regime and show what are the truly universal
properties of the mean-field and MCT-like description of the aging dynamics
[9].

In constructing the Landau theory, we have assumed that the freezing
transition is discontinuous, with a finite value $f_{\bf k}$ of the plateau at
the transition. A viable alternative is of course that of a continuous
transition of the spin-glass type, which leads to a completely different
phenomenology. This raises the question of the possible realization of this
second scenario in the context of supercooled liquids. All short-range
interacting glasses seem to be characterized by rather small Lindemann
parameters at the transition, meaning that it is hard to maintain any kind of
amorphous long range order when individual molecules move substantially, and
that the glass transition is therefore discontinuous [29]. This argument
suggests that continuous glasses can only exist for long-ranged interacting
particles or quantum systems. In the quantum case, it is imaginable that
amorphous density waves can indeed form with a vanishing modulation amplitude
(see [30]). It would be very interesting to find experimental realizations of
such a scenario.

###### Acknowledgements.

We thank A. Lefèvre for useful discussions. GB and JPB are supported by ANR
Grant DYNHET; AA was supported in part by EPSRC Grant No. EP/D050952/1.

## References

  * [1] U. Bengtzelius, W. Götze, A. Sjöilander, J. Phys. C 17, 5915 (1984).
  * [2] E. Leuthesser, Phys. Rev. A 29, 2765 (1984).
  * [3] S. P. Das, Rev. Mod. Phys. 76, 785 (2004).
  * [4] S. P. Das, G. F. Mazenko, S. Ramaswamy, and J. J. Toner, Phys. Rev. Lett. 54, 118 (1985).
  * [5] K. Miyazaki, D. R. Reichman, J. Phys. A: Math. Gen. 38, 20 (2005).
  * [6] A. Andreanov, G. Biroli, A. Lefèvre, J. Stat. Mech., P07008 (2006).
  * [7] B. Kim and K. Kawasaki, J. Stat. Mech. (2008) P02004.
  * [8] T. H. Nishino and H. Hayakawa, Phys. Rev. E 78, 061502 (2008).
  * [9] JP. Bouchaud, L. Cugliandolo, J. Kurchan, M. Mézard, in Spin glasses and Random Fields, A.P. Young Editor (World Scientific) 1998.
  * [10] V. Lubchenko, P. G. Wolynes, Ann. Rev. Phys. Chem. 58 235 (2007).
  * [11] G. Szamel, Phys. Rev. Lett. 90, 228301 (2003).
  * [12] P. Mayer, K. Miyazaki, D. R. Reichman, Phys. Rev. Lett. 97, 095702 (2006).
  * [13] S. N. Majumdar, D. Das, J. Kondev, B. Chakraborty, Phys. Rev. E 70, 060501(R) (2004).
  * [14] L.D. Landau, Zh. Eksper. Teor. Fis. 7, 627 (1937).
  * [15] J.-C. Tolédano, P. Tolédano, The Landau theory of phase transitions, (World Scientific Publishing Co. Pte Ltd) 1987.
  * [16] G. Biroli, J.-P. Bouchaud, Europhys. Lett. 67, 21 (2004).
  * [17] G. Biroli, J.-P. Bouchaud, K. Miyazaki, D.R. Reichman, Phys. Rev. Lett. 97, 195701 (2006) .
  * [18] G. Biroli, J.-P. Bouchaud, J. Phys.: Condens. Matter 19 205101 (2007).
  * [19] T. R. Kirkpatrick and D. Thirumalai, Phys. Rev. A 37, 4439 (1988).
  * [20] S. Franz and G. Parisi, J. Phys.: Condens. Matter 12, 6335 (2000).
  * [21] J.-P. Bouchaud, G. Biroli, J. Chem. Phys. 121, 7347 (2004); G. Biroli, J.-P. Bouchaud, A. Cavagna, T. S. Grigera, P. Verrocchio, Nature Phys. 4 771 (2008); M. Mézard, A. Montanari, J. Stat. Phys. 124 (2006) 1317.
  * [22] D. S. Dean, J. Phys. A: Math. Gen. 29, L613 (1996).
  * [23] A. Andreanov, Ph.D. thesis; http://www.imprimerie.polytechnique.fr/Theses/Files/Andreanov.pdf.
  * [24] W. Götze, Z. Phys. B - Condensed Matter 59, 195 (1985).
  * [25] V. Krakoviack and C. Alba-Simionesco, J. Chem. Phys. 117, 2161 (2002).
  * [26] T. Sarlat, A. Billoire, G. Biroli, J.-P. Bouchaud, in preparation.
  * [27] W. Götze, Sjörgen, Rep. Prog. Phys. 55, 241 (1992).
  * [28] S. M. Bhattacharya, B. Bagchi and P. G. Wolynes, Phys. Rev. E 72, 031509 (2005).
  * [29] For a related argument, see M. P. Eastwood and P. G. Wolynes, Europhys. Lett. 60, 587-593 (2002).
  * [30] M. Tarzia, G. Biroli, Europhys. Lett. 82, 67008 (2008).


#  I2D2: Inductive Knowledge Distillation with
NeuroLogic and Self-Imitation

Chandra Bhagavatula∗, Jena D. Hwang∗, Doug Downey∗, Ronan Le Bras∗, Ximing
Lu∗, Keisuke Sakaguchi‡, Swabha Swayamdipta†, Peter
West${}^{\mathsection}{{}^{*}}$, Yejin Choi${}^{\mathsection}{{}^{*}}$ ∗Allen
Institute for AI, †University of Southern California,
‡Tohoku University, §University of Washington
gengen.apps.allenai.org

###### Abstract

Pre-trained language models, despite their rapid advancements powered by
scale, still fall short of robust commonsense capabilities. And yet, scale
appears to be the winning recipe; after all, the largest models seem to have
acquired the largest amount of commonsense capabilities. Or is it?

In this paper, we investigate the possibility of a seemingly impossible match:
can smaller language models with dismal commonsense capabilities (i.e.,
GPT-2), ever win over models that are orders of magnitude larger and better
(i.e., GPT-3), if the smaller models are powered with novel commonsense
distillation algorithms? The key intellectual question we ask here is whether
it is possible, if at all, to design a learning algorithm that does not
benefit from scale, yet leads to a competitive level of commonsense
acquisition. In this work, we study the _generative_ models of commonsense
knowledge, focusing on the task of generating _generics_ , statements of
commonsense facts about everyday concepts, e.g., birds can fly.

We introduce a novel commonsense distillation framework, I2D2, that loosely
follows the Symbolic Knowledge Distillation of West et al. (2022) but breaks
the dependence on the extreme-scale models as the teacher model by two
innovations: (1) the novel adaptation of NeuroLogic Decoding Lu et al. (2021)
to enhance the generation quality of the weak, off-the-shelf language models,
and (2) self-imitation learning to iteratively learn from the model’s own
enhanced commonsense acquisition capabilities. Empirical results suggest that
scale is not the only way, as novel algorithms can be a promising alternative.
Moreover, our study leads to a new corpus of generics, Gen-A-tomic, that is of
the largest and highest quality available to date.

Figure 1: I2D2 radically improves over off-the-shelf generation from GPT-2 XL
using constrained decoding, and self-imitation.

##  1 Introduction

Language models (LMs) become increasingly better with scale. However, even the
largest LMs continue to fail in unexpected ways due to their lack of
commonsense (Brachman and Levesque, 2021). Knowledge models – custom LMs
trained to generate knowledge—provide on-demand access to task-specific
knowledge to address this gap Bosselut et al. (2019). Today, the best strategy
for training a knowledge model depends on large-scale, albeit noisy knowledge
generated from a large LM West et al. (2022). Are massive-scale LMs the only
way to build commonsense capabilities? In addition to being an interesting
scientific inquiry, if smaller LMs can indeed generate high-quality
commonsense, training knowledge models will become far more efficient and
accessible compared to state-of-the-art.

Figure 2: I2D2 is specifically designed to elicit generics—general statements
about the world. I2D2 works by collecting a list of concepts and generates
generics using Neurologic Decoding to constrain generations at decoding time.
To ensure quality, I2D2 includes the use of a supervised critic to filter out
false generations. The quality of the generations is further improved via
iterative self-imitation learning whereby the language model is finetuned on
the high-quality generics selected by the critic.

We study the generation of commonsense knowledge from GPT-2 (a small LM) and
compare that against GPT-3, a model that is orders of magnitude larger.
Specifically, we focus on the task of generating generics – i.e. statements of
commonsense knowledge about everyday concepts. While generics express general
truths (e.g. “birds can fly”), exceptions abound (e.g. penguins do not fly nor
do sleeping or injured birds). Nonetheless, generics form the basis of how we
express our commonsense about the world (Hampton, 2012; Leslie, 2014).

We present I2D2, a new framework for generating generic statements from GPT-2
(depicted in Fig. 2).111I2D2: Iterative Imitation and Decoding for
Distillation Out of the box, GPT-2 generations are anything but valid generics
– often being repetitive, trivial or resembling narratives. The key
breakthrough for overcoming this challenge comes from (i) constrained
decoding: in which generations are controlled to satisfy manually constructed
lexico-syntactic constraints using Neurologic Decoding Lu et al. (2021), and
(ii) self-imitation learning: in which GPT-2 is iteratively fine-tuned on its
own high-quality generations, automatically identified using a supervised
critic model.

The marked disparity in scale makes the comparison between I2D2 and GPT-3 seem
like an impossible match. However, constrained decoding and self-imitation
enable I2D2 to overcome this limitation and even surpass the quality of
knowledge generated by GPT-3. We formulate a binary-classification task on a
human-annotated test set of generic statements and compare the precision-
recall trade-off between I2D2 and Instruct-GPT-3 by ranking statements using
their critic and perplexity scores, respectively.222We use Instruct-GPT 3’s
text-davinci-001 model in our experiments. In the rest of this paper, GPT-3
refers to this model, unless stated otherwise. I2D2 achieves an average
precision of $0.92$ and outperforms Instruct-GPT-3, with an average precision
of $0.82$.333Average precision is computed as a way of calculating area under
the PR Curve. See §3. Next, we show that iterative self-imitation learning
dramatically improves the accuracy of generations from GPT-2 XL, even before
applying the critic; increasing from 45% $\xrightarrow[]{}$ 58%
$\xrightarrow[]{}$ 62% over three iterations. Finally, we construct Gen-A-
tomic – a knowledge resource of generic statements generated by applying I2D2
to 40K everyday concepts. Compared to GenericsKB Bhakthavatsalam et al.
(2020), Gen-A-tomic is judged by humans to be more accurate (75% GenericsKB
vs. 90% I2D2) while being larger (over 2X) in scale. Moreover, unlike
GenericsKB, which was created through information extraction over text, I2D2
can provide commonsense knowledge for unseen concepts on-demand.

##  2 The I2D2 Framework

GenGen is a new framework for automatically generating generic statements
using pretrained language models. In this work, our language model of choice
is GPT-2 XL. However, any auto-regressive pre-trained language model can be
used within the framework.444In the rest of the paper, I2D2 refers to I2D2
using GPT-2 XL.

I2D2 generates generics in four stages. First, in prompt construction, we
collect seed concepts (e.g. bicycle) and automatically construct several
morpho-syntactically varying prompts (e.g. “A bicycle has …”) (§2.1) for each
concept. The prompts are used as inputs to I2D2. Second, we employ constrained
generation to control the style of text generated from the pre-trained LM at
to mimic the style of generic statements(§2.2). Third, a supervised critic is
used to filter out false and ill-formed generations (§2.3). Finally, the
language model is finetuned on its own high-quality generations selected by
the critic in an iterative self-imitation learning setup(§2.4). Figure 2
illustrates the overall framework.

###  2.1 Prompt Construction

##### Source of seed concepts:

Our first set of concepts for generating generic knowledge are common noun
phrases (e.g. “fruits”), selected from two resources: GenericsKB
(Bhakthavatsalam et al., 2020) and ConceptNet (Speer et al., 2017). From
GenericsKB, we retrieve all noun phrases for which there are at least five
generic statements in the resource, resulting in a total of 8.5K seed
concepts.555GenericsKB was found to consist of uncommon or specialized
terminology (e.g. orpiment) that are not conducive for commonsense generation.
Therefore, we select nouns with at least five generic statements so that the
collected nouns are those that are capable of forming commonsense generics
From ConceptNet, we retrieve noun phrases associated with the types artefact
and human, identified based on hypernymy relationships to the corresponding
WordNet senses. The two lists are then manually vetted to compile a list of
artefacts and humans, totaling 1.4K seed concepts.666We choose human and
artifact as much commonsense knowledge centers around these types. The list of
concepts can be extended to other types as well (e.g. animals, natural
phenomena) in the future.

Our second set of seed concepts are high-level human goals (e.g. “get better
at chess”) obtained from two sources: ProScript (Sakaguchi et al., 2021) and
ATOMIC (Sap et al., 2019). We extract all goals that appear in the ProScript
training data. From ATOMIC, we extract all base events and filter out
hypothetical events (e.g. “PersonX expects to win”) based on a short exclusion
list (see Appendix A.1).

To scale the number of seed concepts we prompt GPT-3 Brown et al. (2020) with
a set-expansion template, which are prompt templates for GPT-3 to generate
items similar to a given set of items; see more details in Appendix A.1.1.
Overall, after GPT-3 based expansion, we have 39K seed concepts, consisting of
26K noun phrases and 13K goals. Note that GPT-3 is only used for seed
expansion and not for the generics generation.

Input prompt | Related Concept |  I2D2 output
---|---|---
In order to get better at chess | - | …you have to practice chess.
In order to get better at chess | improve |  …you have to improve your strategy.
In order to get better at chess | tactics |  …you have to practice tactics.
In order to get better at chess | strategy |  …you have to learn strategy.
Table 1: Example outputs of I2D2 for the concept “get better at chess”. We add
constraints to our constrained generation algorithm to include the related
concept.

##### Morpho-Syntactically Varying Prompts:

We programmatically construct a large number of morpho-syntactically divergent
prompts for each concept to facilitate the generation of a diverse set of
generic statements. Prompts for noun phrases are constructed based on the
template described in Table 2:

$[\text{Generally}|\text{Typically}|\text{Usually}]?\
[\text{a}|\text{an}|\text{the}]?\texttt{ <noun\\_phrase> }\
\texttt{<relational phrase>}$
---
Table 2: Template for automatically constructing morpho-syntactically varying
prompts. ‘$?$’ denotes the group of words is optional and ‘$|$’ denotes the
logical OR operator

Each concept is paired with a relational phrase, e.g. “can be”, “is found in”
etc., from a manually constructed list; Appendix A.1.2 presents more details.
Inspired by Leslie (2008), we prefix adverbs (such as “generally”, “usually”,
and “typically”) to the prompts. We find, empirically, that these prefixes
encourage the language model to generate general statements, instead of long-
form, narrative-like text. An article is optionally prefixed before the
concept for grammaticality. For a given (concept, relational phrase) pair, we
construct all prompt combinations according to the template above and choose
the one with the lowest PLM (GPT-2 XL in our experiments) perplexity. For the
goal seed concepts, from each goal we create four separate prompts by
prepending each of these prefixes: “In order to”, “Before you”, “After you”
and “While you”.

Finally, we filter out all prompts whose per-word perplexity under GPT-2 XL is
above a threshold of 250. This allows us to apriori filter out ill-formed
prompts such as “Typically, a hall are planted at …”. This results in a total
of 1.6M prompts.

###  2.2 Constrained Generation

Given a prompt (e.g. “Generally, a bicycle has”), a key challenge for a
language model is to generate continuations consistent with the linguistic
style of generic statements. Unconstrained, a language model tends to go into
a “story-telling” mode which can result in long sentences that resemble a
narrative. In contrast, generic statements are typically simple and short
Tessler and Goodman (2016). Thus, we aim to control the number of function
words (e.g., “in”, “on”, “of”) in the generated sequence to limit the length
of the sequence. We further wish to disallow the generation of connectives
(e.g., “although”, “since”, “furthermore”) to make the statements short and
succinct. We also want to avoid repetition and trivia in the statements. To
steer the generations to meet these various requirements, we apply Neurologic
Decoding (Lu et al., 2021), a beam-based decoding algorithm that allows us to
impose desired lexical constraints during generation.

We devise negative constraints that limit generation of certain tokens (e.g.,
function words or repeated tokens as above) in the outputs. We devise positive
constraints to encourage the inclusion of the optional related concept to
generate generic statements relating two concepts. We describe all the
constraints in more detail in Appendix A.1.4. Given the 1.6M programmatically
constructed prompts, we generate ten generations for each prompt, resulting in
16M generics which must now be filtered to preserve quality.

Many applications require knowledge that relates two known concepts. For
example, to solve a QA problem, it might be important to have background
knowledge about a the relationship between a “hotel” and a “credit card”, e.g.
“At a hotel, credit cards can be used to make a payment”. Our constrained
decoding algorithm enables us to do that. We obtain concepts related to a seed
concept from GPT-3 using a custom template; see details in Appendix A.1.3. In
I2D2, we aim to generate continuations to a prompt that also mention a concept
related to the seed concept.

###  2.3 Supervised Critic

While our constrained decoding method results in generations that resemble
generic statements, a lot of the generics tend to be invalid or false
statements. This is consistent with the finding that LMs can generate
hallucinations and false statements about the real world (Ji et al., 2022). To
address this, we train a supervised critic model to predict the veracity of a
generation. We create a training set of $\sim$12K statements, containing up to
four sampled generations for each concept from a held-out set of $\sim$3K
concepts. The labels for each generation are collected using the same
procedure as the evaluation data, which is described in Section 3.1. We train
a RoBERTa-Large (Liu et al., 2019) classifier as our critic model and used as
a binary classifier to identify valid generic statements.

###  2.4 Self-Imitation Learning

While the constrained decoding followed by filtering with the supervised
critic can already result in a desired set of generics, we intend to leverage
the strength of these stages to further generate more high quality generics.
We achieve this by finetuning GPT-2 XL on the high-quality subset of generics
in an iterative fashion. In particular, we perform two iterations of fine-
tuning in our experiments. This method can be viewed as self-imitation
learning Oh et al. (2018), such that the ’actor’ corresponds to the base GPT-2
language model and the ’critic’ corresponds to the RoBERTa discriminator. Some
related recent works (Oh et al., 2018; Thoppilan et al., 2022; Haluptzok et
al., 2022) show that language models can become better at generating high-
quality programs if they are fine-tuned on their own previously generated
correct code examples, as evaluated by a compiler. However, previous work did
not test multiple iterations of self-imitation using language models.

##  3 Evaluating the I2D2 Framework

Figure 3: Comparing the PR Curve of I2D2 Critic and the language model
perplexities based on GPT2-XL and GPT-3.

Figure 4: Comparing the PR Curve of I2D2 iterations 0 through 2

###  3.1 I2D2 generates better generics than GPT-3

Our goal in this work is to investigate whether smaller language models like
GPT-2 XL can still offer use, while larger models like GPT-3 continue to
outperform them on virtually all NLP tasks. In particular, we show that our
I2D2 approach for generating generic statements outperforms GPT-3 while being
considerably more cost-effective, accessible and scalable.

##### Systems

We wish to compare how GPT-3, given the same set of prompts, can generate and
identify valid generics. For a given prompt, we generate ten generations from
each system. GPT-3 is prompted in a few-shot manner with an instruction and
six examples. We use different sets of few-shot examples for noun phrases and
goals. Appendix A.1.6 presents more details about the instruction and in-
context examples provided to GPT-3. I2D2, using a supervised critic, assigns a
score to each generated statement. For GPT-3, we use the perplexity assigned
to a generation as an indicator of validity. As an additional baseline, we
also compute perplexity under off-the-shelf GPT-2 XL.

##### Evaluation Data

We set aside 300 concepts for evaluation. Each concept is associated with
several prompts (on average 40). We generate ten generic statements for each
prompt from I2D2 and GPT-3. Next, from all generations for a concept, we
randomly sample four statements generated by each system being compared. A
generic statement is considered valid if it is a generally true statement
about the world. Three annotators on the Amazon Mechanical Turk platform rate
the validity of each generated statement. Details of the annotation template
and instructions are provided in Appendix A.1.5. At least two out of three
annotators agreed on a label 92.5% of the time over all 4 statements.777We
provide pairwise annotation agreement score. Since model generations are
ideally should be valid, we naturally produce a skew towards a single label,
which is problematic for kappa metric Feinstein and Cicchetti (1990). For the
annotation task, we maintain a pay rate of at least $15/hr.

##### Metrics

Given the human-annotated test set of generics, we compare the precision-
recall trade-off between I2D2 and GPT-3. Each system assigns a score to each
generic statement, allowing us to rank the statements from most likely to
least likely to be a generic. Combined with the human annotations of the
validity of a statement, we can plot a precision-recall (PR) curve. PR curves
allow us to evaluate the accuracy of each system’s output as the number of
statements output varies, which is important since different tradeoffs between
quantity and quality of output may be desired for different application
settings.

##### Results

Figure 4 shows the impact of including a supervised critic to identify valid
generic statements. We find that GPT-3, while impressive, lags significantly
behind our supervised critic in identifying which generic statements are
valid. Off-the-shelf GPT-2 XL model is the worst at identifying valid generic
statements. Perplexity alone is not a good indicator of what a valid generic
is.

I2D2 uses both a generator and a discriminator. To evaluate the generator, we
sample from its generations over the test set of prompts. For a given set of
generations, human annotators judge whether the statement is true or false. We
compute accuracy against human labels and use that as a metric to measure the
quality of the generator.

###  3.2 I2D2 gets better through iterative self-imitation learning

##### Systems

For self-imitation learning, we generate a large corpus of generations and
filter out invalid statements using the supervised critic to yield a
“purified” subset. We compare generations from I2D2 using off-the-shelf GPT-2
XL and outputs from two additional iterations of fine-tuning.

##### Evaluation Data

We use the same held-out test set of prompts for this experiment.

##### Metrics

We evaluate the accuracy of the generations before and after applying the
supervised critic.

##### Results

We show that a language model gets iteratively better as it gets finetuned on
its own high-quality generations over each iteration. The raw accuracy of the
generations, before applying the critic, improves from 45% $\rightarrow$ 58 %
$\rightarrow$ 62% over three iterations. We also compare the precision-recall
trade-off between the three iterations. Figure 4 shows the effectiveness of
self-imitation learning over three iterations.

###  3.3 Gen-A-tomic corpus is more accurate than GenericsKB

GenericsKB (Bhakthavatsalam et al., 2020) is a static resource of generic
knowledge created through information extraction over three large text
corpora: the Waterloo corpus, SimpleWikipedia and the ARC corpus. This work
released a large scale dataset of 14M generations and a high-quality subset of
1M generic statements. We compare GenericsKB’s best 1M against our corpus. We
randomly sample 1K generic statements from GenericsKB and I2D2 and ask
annotators on Amazon Mechanical Turk to rate the validity of the generic
statement. We find that while only 76% of statements in GenericsKB were
annotated as accurate, over 90% of statements in I2D2 were judged as valid.
The results show that I2D2 is more accurate than than GenericsKB, in addition
to being larger.

###  3.4 Smaller, better-trained versions of GPT-3 outperform larger ones

We compare three versions of the GPT-3 model available on the OpenAI API –
davinci, curie-instruct and davinci-instruct Ouyang et al. (2022); Brown et
al. (2020). Interestingly, we find that the curie-instruct model, despite
being a much smaller model, generates more valid generic statements compared
to the much larger davinci model. The instruct models (including curie-
instruct) were trained using reinforcement learning on human feedback. The
accuracy (validity) of statements generated by the three GPT-3 models on the
same set of test prompts are 53.3% (davinci), 60.6% (curie-instruct), and
81.9% (davinci-instruct). These results further demonstrate that better
training can result in smaller models performing better than larger models.

Our work adds to the growing body of evidence from recent work that large
language models have not been trained optimally Kaplan et al. (2020) and it
would be worthwhile to look for better training strategies to achieve high
performance using smaller, affordable, greener models.

###  3.5 Analysis

##### Diversity

For diversity analysis we employ a survey method called Mark and Recapture
(MnR) Seber et al. (1982); The U.S. Geological Survey (2018) commonly used by
ecologists to estimate a large population size via sampling. This method
captures individuals of a population in two (or more) stages. In the first
capture, the individuals captured are marked and released. At a later capture,
the number of recaptured individuals are counted and the population size
estimated.

We use MnR to estimate the unique population size of our large datasets
thereby gauging the diversity of the dataset. For our implementation of MnR,
we perform two random captures using a sample size of 30% (of the total
dataset size) at each capture. A generic in the second capture is considered a
recapture (i.e., individual seen in the first capture) if it exceeds a textual
similarity threshold (BLEU score $>0.85$)888The threshold was determined via
several rounds of experimentation and manual evaluation to determine a
reasonable level of textual similarity. with the generics of the same concept
from the first capture as the reference. Then, we employ the Chapman estimator
for MnR Brittain and Böhning (2009); Chapman (1951) to estimate the population
size of unique generics in the dataset.

We compare the estimated per concept average count of unique generics for
GenericsKB and Gen-A-tomic. Overall, we find that Gen-A-tomic is estimated to
include at least triple the amount of generics per concept in comparison to
GenericsKB. We also observe that the estimated unique generics per concept is
higher for the best cuts of the Gen-A-tomic dataset.

A more direct approach for diversity perhaps is an embedding-based method
where the unique generic population is estimated via its textual difference
from its semantic neighbors. However, the process becomes quickly unscalable
for large datasets. In preliminary experiments, however, we find that the
embedding-based approach also results in similar trends.

Figure 5: Compared to GenericsKB, the estimated average unique number of
generics per concept is higher for any version of the GenGen corpus.

##  4 Related Work

##### Generics

Generics like “dogs are friendly” are generalizing statements about observed
“truths” or defaults about the world for which exceptions can be found (e.g.,
not all dogs are friendly in practice). Generics have been studied quite
extensively in philosophy, linguistics, and psychology. While they are clearly
important to human reasoning—especially to non-monotonic reasoning Carlson and
Pelletier (1995); Pelletier and Asher (1997), they have also been long debated
for their puzzling properties which renders them difficult to formally analyze
Leslie (2012, 2008); Hampton (2012); Liebesman (2011). Bhakthavatsalam et al.
(2020) demonstrated the usefulness of generics in language understanding by
feeding large-scale generics knowledge to text models and showing improvement
on question-answering and explanation generation. However, being a static
resource, GenericsKB cannot provide knowledge for unseen concepts. To be
useful across a wide range of tasks and datasets, a more comprehensive
resource of generics is required. I2D2 can generate generics for arbitrary
concepts and generics relating two concepts – a feature unique to I2D2. I2D2
can also be easily extended to generate temporal (“during a cold night, people
need a blanket”) or comparative (“a tennis ball is smaller than an office
chair”) generic knowledge, leading to a commonsense knowledge model that is
more comprehensive.

##### Commonsense Knowledge

Various methods for representing commonsense konwledge have been proposed in
the literature. ConceptNet Speer et al. (2017) focused on the conceptual
commonsense relationship among various concepts and entities in their
knowledge graph. Atomic Sap et al. (2019) and Atomic2020 Hwang et al. (2021)
have offered symbolic commonsense knowledge graphs representing relational
inference focusing on the “If-Then” (cause-effect) reasoning. Fine-tuned on
Atomic, Comet Bosselut et al. (2019) have offered a neural knowledge model
that can reason about situations beyond the symbolic knowledge graphs. Unlike
our current framework however, previous commonsense knowledge models typically
only handled data in the form of structured triples and were predominantly
focused on commonsense about events. I2D2 is the first knowledge model that
has focused on generic knowledge, generated in natural language. Uniquely, we
also provide a critic model that can filter invalid or ill-formed generations.

##### Symbolic Knowledge Distillation

Collecting high-quality knowledge at scale has been a longstanding challenge.
The traditional way is to collect by human annotation Speer et al. (2017); Sap
et al. (2019), which can be time-consuming and expensive. Bhakthavatsalam et
al. (2020) extracted generics by filtering and cleaning based on 1.7B
sentences from three large text corpora. However, manually constructed
resources and resources extracted from large corpora can be difficult to
extend. Recent works showed that pre-trained language models can be a good
source of knowledge West et al. (2022); Zhang et al. (2022). Symbolic
knowledge distillation (SKD) West et al. (2022), for instance, have generated
even-centric inferential knowledge from GPT-3 and distill it into GPT-2. While
these methods presents promising results, they primarily relies on using GPT-3
and only handles knowledge about events in a structured triple format. I2D2,
on the other hand, relies only on GPT-2’s own generations to improve itself
and generates knowledge in natural language.

##### Self-Imitation Learning

Self-imitation learning Oh et al. (2018) was proposed as a reinforcement
learning method in which an agent learns to replicate past good actions. More
recently, a similar approach was applied in dialog models Thoppilan et al.
(2022); Xu et al. (2022) and code generation Haluptzok et al. (2022). However,
recent applications have relied on models much larger than GPT-2 XL used in
I2D2. Moreover, while Haluptzok et al. (2022) have explored the idea of self-
imitation learning in language models, their work crucially relies on a
compiler to identify correct programs. However, compilers are deterministic
and by definition, 100% accurate. Instead, the supervised critic in I2D2 can
be noisy, especially for identifying generics, which have paradoxical
properties that make its formalization very difficult Mari et al. (2012). We
also show that self-imitation learning is beneficial when done over multiple
iterations. In principle, I2D2 could be improved iteratively through a life-
long learning process. But, under what conditions would the performance gains
plateau is an interesting open future research question.

##  5 Conclusion

We present I2D2– a novel framework for generating generic knowledge from
language models using constrained decoding and self-imitation learning. I2D2,
while using orders of magnitude fewer parameters, can still outperform GPT-3
at the task of generating high-quality generic statements. We also show that
I2D2 is higher-quality, larger-scale and more diverse than the static
GenericsKB dataset. I2D2 provides on-demand access to generic knowledge that
can bridge the gap in commonsense knowledge, often observed in even the larget
LMs available today.

##  6 Acknowledgements

We thank our colleagues on the Beaker Team at the Allen Institute for AI for
helping with the compute infrastructure. This work was supported in-part by
DARPA MCS program through NIWC Pacific (N66001-19-2-4031).

## References

  * Bhakthavatsalam et al. (2020) Sumithra Bhakthavatsalam, Chloe Anastasiades, and Peter Clark. 2020.  Genericskb: A knowledge base of generic statements.  _arXiv preprint arXiv:2005.00660_.
  * Bosselut et al. (2019) Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. 2019.  Comet: Commonsense transformers for automatic knowledge graph construction.  _arXiv preprint arXiv:1906.05317_.
  * Brachman and Levesque (2021) Ronald J. Brachman and Hector J. Levesque. 2021.  Toward a new science of common sense.
  * Brittain and Böhning (2009) Sarah Brittain and Dankmar Böhning. 2009.  Estimators in capture–recapture studies with two sources.  _AStA Advances in Statistical Analysis_ , 93(1):23–47.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901.
  * Carlson and Pelletier (1995) Gregory N Carlson and Francis Jeffry Pelletier. 1995.  _The generic book_.  University of Chicago Press.
  * Chapman (1951) Douglas George Chapman. 1951.  Some properties of the hypergeometric distribution with applications to zoological sample censuses. berkeley.  _Calif: University of Catifomia Publications in Statistics_ , 195(1).
  * Feinstein and Cicchetti (1990) Alvan R Feinstein and Domenic V Cicchetti. 1990.  High agreement but low kappa: I. the problems of two paradoxes.  _Journal of clinical epidemiology_ , 43(6):543–549.
  * Haluptzok et al. (2022) Patrick Haluptzok, Matthew Bowers, and Adam Tauman Kalai. 2022.  Language models can teach themselves to program better.  _arXiv preprint arXiv:2207.14502_.
  * Hampton (2012) James A Hampton. 2012.  Generics as reflecting conceptual knowledge.  _Recherches linguistiques de Vincennes_ , pages 9–24.
  * Hwang et al. (2021) Jena D Hwang, Chandra Bhagavatula, Ronan Le Bras, Jeff Da, Keisuke Sakaguchi, Antoine Bosselut, and Yejin Choi. 2021.  (comet-) atomic 2020: On symbolic and neural commonsense knowledge graphs.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 35, pages 6384–6392.
  * Ji et al. (2022) Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. 2022.  Survey of hallucination in natural language generation.
  * Kaplan et al. (2020) Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020.  Scaling laws for neural language models.  _arXiv preprint arXiv:2001.08361_.
  * Leslie (2008) Sarah-Jane Leslie. 2008.  Generics: Cognition and acquisition.  _Philosophical Review_ , 117(1):1–47.
  * Leslie (2012) Sarah-Jane Leslie. 2012.  Generics.  In Gillian Russell and Delia Fara, editors, _Routledge Handbook of Philosophy of Language_ , pages 355–366. Routledge.
  * Leslie (2014) Sarah-Jane Leslie. 2014.  Carving up the social world with generics.  _Oxford studies in experimental philosophy_ , 1.
  * Liebesman (2011) David Liebesman. 2011.  Simple generics.  _Noûs_ , 45(3):409–442.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.
  * Lu et al. (2021) Ximing Lu, Peter West, Rowan Zellers, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021.  Neurologic decoding:(un) supervised neural text generation with predicate logic constraints.  _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics_.
  * Mari et al. (2012) Alda Mari, Claire Beyssade, and Fabio Del Prete. 2012.  _Genericity_ , volume 43.  OUP Oxford.
  * Oh et al. (2018) Junhyuk Oh, Yijie Guo, Satinder Singh, and Honglak Lee. 2018.  Self-imitation learning.
  * Ouyang et al. (2022) Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022\.  Training language models to follow instructions with human feedback.  _arXiv preprint arXiv:2203.02155_.
  * Pelletier and Asher (1997) Francis Jeffry Pelletier and Nicholas Asher. 1997.  Generics and defaults.  In _Handbook of Logic and Language_.
  * Sakaguchi et al. (2021) Keisuke Sakaguchi, Chandra Bhagavatula, Ronan Le Bras, Niket Tandon, Peter Clark, and Yejin Choi. 2021.  proscript: Partially ordered scripts generation via pre-trained language models.  _arXiv preprint arXiv:2104.08251_.
  * Sap et al. (2019) Maarten Sap, Ronan Le Bras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A Smith, and Yejin Choi. 2019.  Atomic: An atlas of machine commonsense for if-then reasoning.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 33, pages 3027–3035.
  * Seber et al. (1982) George Arthur Frederick Seber et al. 1982.  The estimation of animal abundance and related parameters.
  * Speer et al. (2017) Robyn Speer, Joshua Chin, and Catherine Havasi. 2017.  Conceptnet 5.5: An open multilingual graph of general knowledge.  In _Thirty-first AAAI conference on artificial intelligence_.
  * Tessler and Goodman (2016) Michael Henry Tessler and Noah D. Goodman. 2016.  A pragmatic theory of generic language.  _ArXiv_ , abs/1608.02926.
  * The U.S. Geological Survey (2018) The U.S. Geological Survey. 2018.  Capture-mark-recapture science.  https://www.usgs.gov/centers/eesc/science/capture-mark-recapture-science, Last accessed on 2022-09-17.
  * Thoppilan et al. (2022) Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, Dehao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Yanqi Zhou, Chung-Ching Chang, Igor Krivokon, Will Rusch, Marc Pickett, Kathleen S. Meier-Hellstern, Meredith Ringel Morris, Tulsee Doshi, Renelito Delos Santos, Toju Duke, Johnny Soraker, Ben Zevenbergen, Vinodkumar Prabhakaran, Mark Diaz, Ben Hutchinson, Kristen Olson, Alejandra Molina, Erin Hoffman-John, Josh Lee, Lora Aroyo, Ravi Rajakumar, Alena Butryna, Matthew Lamm, Viktoriya Kuzmina, Joe Fenton, Aaron Cohen, Rachel Bernstein, Ray Kurzweil, Blaise Aguera-Arcas, Claire Cui, Marian Croak, Ed H. Chi, and Quoc Le. 2022.  Lamda: Language models for dialog applications.  _CoRR_ , abs/2201.08239.
  * West et al. (2022) Peter West, Chandra Bhagavatula, Jack Hessel, Jena Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, and Yejin Choi. 2022.  Symbolic knowledge distillation: from general language models to commonsense models.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 4602–4625, Seattle, United States. Association for Computational Linguistics.
  * Xu et al. (2022) Jing Xu, Megan Ung, Mojtaba Komeili, Kushal Arora, Y-Lan Boureau, and Jason Weston. 2022.  Learning new skills after deployment: Improving open-domain internet-driven dialogue with human feedback.  _arXiv preprint arXiv:2208.03270_.
  * Zhang et al. (2022) Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. 2022.  Opt: Open pre-trained transformer language models.

##  Appendix A Appendix

###  A.1 Selection of Concepts and Goals

##### ConceptNet concept selection

To select ConceptNet concepts, we first build a list of artefact and human
terms from WordNet by hierarchically traversing the hypernymy hierarchy (by
depth) starting from the artefact%1:03:00 and person%1:03:00, respectively. We
then select ConceptNet concepts that belong to the lists in the same order as
the WordNet list. The concepts in the list are sequentially evaluated manually
to build a total of 1K artefact and 400k human terms. The totaling 1.4k
concepts are then used as seed ConceptNet concepts.

##### ATOMIC goal selection

To select goals from ATOMIC, we obtain the complete list of base events (e.g.
“PersonX adopts a cat”). We drop the “PersonX” prefixation and all mentions of
“Person” (e.g. “PersonY”, “PersonZ”). Additionally, because we want to select
for goals that are achievable, we remove all irrealis or hypothetical
situations (the described situation or event has not taken place). More
specifically, we filter out all events with the verbs ‘need’, ‘want’ ‘wish’,
‘hope’, ‘dream’, ‘expect’, ‘imagine’, ‘mean’, and ‘plan’; negated events (e.g.
“PersonX does not get the job”); and events modified by modals that indicate
permission or obligation (e.g. ‘should’). In this manner we arrive at a list
of 8.5K goals from ATOMIC.

####  A.1.1 GPT3 for Set Expansion

We develop a template for set expansion based on GPT3.

> Generate more concepts.
> 1: $<$sampled concept 1$>$
> 2: $<$sampled concept 2$>$
> 3: $<$sampled concept 3$>$
> 4: $<$sampled concept 4$>$
> 5: $<$sampled concept 5$>$
> 6:

Set expansion is done in several iterations. We define $K$ as the number of
new concepts to be found in each iteration. We construct a prompt as shown
above by sampling five concepts. We get five outputs for each prompt. We skip
concepts whose generation perplexity is lower than a set threshold (8 in our
experiments). Thus, at most five new concepts are found with each call to the
OpenAI API. At the end of each iteration, newly found concepts are added to
the seed list of concepts. This iterative process allows us to slowly expand
the original list with new related concepts.

####  A.1.2 List of relational templates

Noun phrases are combined with one of the following verb phrases if they are
obtained from GenericsKB:

> are
> is
> have
> can
> has
> should
> produces
> may have
> may be
>

If a noun phrase is obtained from ConceptNet, we expand the templates
available in the (file “templates.txt” attached in supplementary material.

####  A.1.3 Template for obtaining related concept

Related concepts for a given concept are also obtained from GPT3. We use the
following prompt:

> Generate five words that are related to the given word.
>
> Word: hotel
> Related Words:
> 1: Credit card
> 2: Fee
> 3: Resort
> 4: Parking lot
> 5: Reception
> ==
> Word: $<$given concept$>$
> Related Words:

GPT-3 generates five related concepts for each given word.

####  A.1.4 Constraints for Neurologic Decoding

####  A.1.5 Amazon Mechanical Turk Annotation Template

Figure 6 shows a screenshot of the annotation template used for collecting the
training set and for evaluation of generic statements.

Figure 6: A screenshot of the template used for obtaining annotations on the
Amazon Mechanical Turk platform.

####  A.1.6 GPT-3 generics generation template

Generics are generated from GPT-3 using the following template:

> Generate statements that are generally true in the real world.
> An apple is a fruit.
> Violins are used for music.
> Aardvarks are mammals.
> Accidents cause injuries.
> Protein is made of amino acids.
> Apples can be red or green.
> $<$ test prompt $>$

We generate ten continuations for the prompt above.