Geometry of Vaidya spacetimes

Armand COUDRAY1 & Jean-Philippe NICOLAS111LMBA, UMR CNRS 6205, Department of
Mathematics, University of Brest, 6 avenue Victor Le Gorgeu, 29200 Brest,
France. Emails<EMAIL_ADDRESS>Jean-Philippe.Nicolas@univ-
brest.fr.

Abstract. We investigate the geometrical structure of Vaidya’s spacetime in
the case of a white hole with decreasing mass, stabilising to a black hole in
finite or infinite time or evaporating completely. Our approach relies on a
detailed analysis of the ordinary differential equation describing the
incoming principal null geodesics, among which are the generators of the past
horizon. We devote special attention to the case of a complete evaporation in
infinite time and establish the existence of an asymptotic light-like
singularity of the conformal curvature, touching both the past space-like
singularity and future time-like infinity. This singularity is present
independently of the decay rate of the mass. We derive an explicit formula
that relates directly the strength of this null singularity to the asymptotic
behaviour of the mass function.

Keywords. Black hole, white hole, evaporation, Vaidya metric, Einstein
equations with matter, null singularity.

Mathematics subject classification. 83C57, 83C20

###### Contents

  1. 1 Introduction
  2. 2 Vaidya’s spacetime, connection, curvature
  3. 3 The incoming principal null congruence
    1. 3.1 General properties
    2. 3.2 The second optical function
  4. 4 Case of a complete evaporation in infinite time
    1. 4.1 The asymptotic null singularity
    2. 4.2 A family of uniformly timelike congruences
    3. 4.3 The global structure of the spacetime

##  1 Introduction

In 1959, P.C. Vaidya published a paper [13] in which he was solving a long
standing open problem in general relativity: finding a modification of the
Schwarzschild metric in order to allow for a radiating mass. His original
derivation of the metric was based on Schwarzschild coordinates. Ten years
later, in [14], he observed that using instead what is now known as Eddington-
Finkelstein coordinates would simplify the construction a great deal. Vaidya’s
metric is a solution to the Einstein equations with matter in the form of null
dust and it has since been the object of numerous studies; see the book by
J.B. Griffiths and J. Podolsky [7], section 9.5, for a very clear presentation
of the metric and an excellent account of the history of these investigations.
Many of these works aimed at gluing some part of Vaidya’s spacetime with other
exact spacetimes like Schwarzschild and Minkowski, in order to construct
models for evaporating or collapsing black holes, see for instance W.A.
Hiscock [9]. A different approach consists in studying the maximal extension
(not necessarily analytical) and the matching of the Vaidya exterior with some
interior metric. This has been done under explicit assumptions on the manner
in which the mass varies with time, for instance in [1]. The general problem
was first tackled by W. Israel [10] and more recently by F. Fayos, M.M.
Martin-Prats and M.M. Senovilla [4] and F. Fayos and R. Torres [5], this last
work studying the possibility of a singularity-free gravitational collapse.

In this paper, we study the geometry of Vaidya’s spacetime itself, without
extension, in as much generality as possible. We treat the case of a radiating
white hole that emits null dust and as a result sees its mass decrease. The
case of a black hole whose mass increases due to incoming null dust is
obtained by merely reversing the arrow of time. We make only minimal natural
assumptions on the behaviour of the mass function. In the existing literature,
many of the precise behaviours of null geodesics are studied numerically. In
contrast, the goal of the present paper is to give a mathematical derivation
of the geometrical features of Vaidya’s spacetime, by analysing precisely the
ordinary differential equation describing the incoming principal null
geodesics, among which are the generators of the past horizon. This equation
is well-known and appears explicitly in I. Booth and J. Martin [3], in which a
notion of distance between the event horizon and an apparent horizon is
introduced. This is done for an evaporating white hole that, asymptotically or
in finite retarded time, stabilises to a black hole or evaporates completely.
We devote special attention to the case of a complete evaporation in infinite
time (the case where the mass function vanishes in finite time has been
studied in details in [5], with a mass function that is possibly non
differentiable at its vanishing point), for which the spacetime is already
maximally extended and, we prove the existence of a light-like singularity of
the Weyl tensor, touching both the past spacelike singularity and future
timelike infinity222Note that F. Fayos and R. Torres [5] also exhibit a null
singularity but under slightly different assumptions. The evaporation ends in
finite time, therefore the singularity is not asymptotic but well present in
the spacetime. Moreover the evaporation is allowed to end brutally: the mass
vanishes in finite retarded time with a possibly non-zero slope, in which case
it is non-differentiable at its vanishing point.. This classical evaporation
in infinite time may perhaps be seen as a classical analogue of the final pop
in black hole quantum evaporation, with the emission of a gravitational wave
propagating along a light-cone.

The article is structured as follows. In Section 2, we recall the construction
of Vaidya’s spacetime as a simple modification of the Schwarzschild metric
expressed in retarded Eddington-Finkelstein coordinates
($u=t-r_{*},r,\theta,\varphi$) and we recall the essential features of the
geometry: Petrov-type D, principal null directions, expression of the
Christoffel symbols, Weyl and Ricci tensors in the Newman-Penrose formalism,
scalar curvature and also the curvature scalar (also referred to as the
Kretschmann scalar). We also present our natural assumptions on the mass
function, which are essentially that it is smooth, decreases and admits finite
limits as the retarded times tends to $\pm\infty$. Finally, we derive the
ordinary differential equation that allows to locate the past event horizon
and gives the complete congruence of incoming principal null geodesics.
Section 3 presents the analysis of the properties of the solutions to this ODE
and the construction of an optical function such that its level hypersurfaces
are spanned by the incoming principal null geodesics. This is the
generalisation of the function $v=t+r_{*}$ in the Schwarzschild case. In
Section 4, we give further properties of the principal null geodesics in the
case of a complete evaporation in infinite time and we prove that they all end
up in the future at a light-like conformal curvature singularity. This
singularity is present independently of the speed at which the mass function
approaches zero in the future, however its strength seems to be directly
related to the decay rate of the mass. We also construct families of timelike
curves that end up either at the null singularity or at timelike infinity
depending of their “mass”, i.e. the rate of their proper time as measured with
respect to the retarded time. The section ends with Penrose diagrams of
Vaidya’s spacetime in the case of a complete evaporation in infinite time,
showing the various congruences.

All formal calculations of connection coefficients and curvature tensors have
been done using Sage Manifolds [6].

Notations. Throughout the paper, we use the formalisms of abstract indices,
Newman-Penrose and $2$-component spinors.

##  2 Vaidya’s spacetime, connection, curvature

The Vaidya metric can be constructed as follows. We start with the
Schwarzschild metric

$g=\left(1-\frac{2M}{r}\right)\mathrm{d}t^{2}-\left(1-\frac{2M}{r}\right)^{-1}\mathrm{d}r^{2}-r^{2}\mathrm{d}\omega^{2}\,,~{}\mathrm{d}\omega^{2}=\mathrm{d}\theta^{2}+\sin^{2}\theta\,\mathrm{d}\varphi^{2}\,.$

We express it in outgoing Eddington-Finkelstein coordinates
$(u,r,\theta,\varphi)$, where $u=t-r_{*}$ and $r_{*}=r+2M\log(r-2M)$:

$g=\left(1-\frac{2M}{r}\right)\mathrm{d}u^{2}+2\mathrm{d}u\mathrm{d}r-r^{2}\mathrm{d}\omega^{2}\,.$

Vaidya’s metric is then obtained simply by allowing the mass $M$ to depend on
$u$:

$g=\left(1-\frac{2M(u)}{r}\right)\mathrm{d}u^{2}+2\mathrm{d}u\mathrm{d}r-r^{2}\mathrm{d}\omega^{2}\,.$
(1)

Throughout the paper, we assume that

$M\mbox{ is a smooth function of the retarded time }u$ (2)

and we denote

$\dot{M}(u):=\frac{\mathrm{d}M}{\mathrm{d}u}\,.$

The non-zero Christoffel symbols for (1) are

$\displaystyle\Gamma_{0\,0}^{0}=-\frac{M(u)}{r^{2}}\,,~{}\Gamma_{2\,{2}}^{\,0}=r\,,~{}\Gamma_{3\,{3}}^{\,0}=r\sin^{2}\theta\,,$
$\displaystyle\Gamma_{0\,0}^{1}=-\frac{r^{2}\dot{M}(u)-rM(u)+2\,M(u)^{2}}{r^{3}}\,,~{}\Gamma_{0\,1}^{1}=\frac{M(u)}{r^{2}}\,,$
$\displaystyle\Gamma_{2\,2}^{1}=-r+2\,M(u)\,,~{}\Gamma_{3\,3}^{1}=-r\sin^{2}\theta+2\,M(u)\sin^{2}\theta\,,$
$\displaystyle\Gamma_{1\,2}^{2}=\frac{1}{r}\,,~{}\Gamma_{3\,3}^{2}=-\sin\theta\,\cos\theta\,,~{}\Gamma_{1\,3}^{3}=\frac{1}{r}\,,~{}\Gamma_{2\,3}^{3}=\frac{\cos\theta}{\sin\theta}\,.$

The Weyl tensor has Petrov type D (see [12] for the Petrov classification of
the Weyl tensor in terms of the multiplicities of its principal null
directions), i.e. it has two double principal null directions that are given
by

$V=\frac{\partial}{\partial r}\,,~{}W=\frac{\partial}{\partial
u}-\frac{1}{2}F\frac{\partial}{\partial r}\,.$ (3)

This is well known (see [7]) and can be checked easily by observing that $V$
and $W$ both satisfy the condition ensuring that they are at least double
roots of the Weyl tensor (see R. Penrose, W. Rindler [11] Vol. 2, p. 224)

$C_{abc[d}V_{e]}V^{b}V^{c}=C_{abc[d}W_{e]}W^{b}W^{c}=0\,.$

We consider a null tetrad built using the principal null vectors above

$\displaystyle l$ $\displaystyle=$ $\displaystyle V\,,$ $\displaystyle n$
$\displaystyle=$ $\displaystyle W\,,$ $\displaystyle m$ $\displaystyle=$
$\displaystyle\frac{1}{r\sqrt{2}}\left(\frac{\partial}{\partial\theta}+\frac{i}{\sin\theta}\frac{\partial}{\partial\varphi}\right)\,,$
$\displaystyle\bar{m}$ $\displaystyle=$
$\displaystyle\frac{1}{r\sqrt{2}}\left(\frac{\partial}{\partial\theta}-\frac{i}{\sin\theta}\frac{\partial}{\partial\varphi}\right)\,.$

It is a normalised Newman-Penrose tetrad, i.e.

$l_{a}l^{a}=n_{a}n^{a}=m_{a}m^{a}=\bar{m}_{a}\bar{m}^{a}=l_{a}m^{a}=n_{a}m^{a}=0\,,~{}l_{a}n^{a}=-m_{a}\bar{m}^{a}=1\,.$

Let $\\{o^{A},\iota^{A}\\}$ be the spin-frame (a local basis of the spin-
bundle $\mathbb{S}^{A}$ that is normalised, i.e. $o_{A}\iota^{A}=1$) defined
uniquely up to an overall sign by

$l^{a}=o^{A}\bar{o}^{A^{\prime}}\,,~{}n^{a}=\iota^{A}\bar{\iota}^{A^{\prime}}\,,~{}m^{a}=o^{A}\bar{\iota}^{A^{\prime}}\,,~{}\bar{m}^{a}=\iota^{A}\bar{o}^{A^{\prime}}\,.$

Since the spacetime has Petrov type D, the Weyl spinor $\Psi_{ABCD}$ has only
one non-zero component which is

$\Psi_{2}=\Psi_{ABCD}\,o^{A}o^{B}\iota^{C}\iota^{D}=-\frac{M(u)}{r^{3}}\,.$

The Ricci tensor is non-zero

$\mathrm{Ric}\left(g\right)=-\frac{2\,\dot{M}(u)}{r^{2}}\mathrm{d}u\otimes\mathrm{d}u$

but trace-free, i.e.

$\mathrm{Scal}_{g}=0\,,$

and the only non-zero Newman-Penrose scalar for the Ricci tensor is

$\Phi_{22}=\frac{1}{2}R_{ab}n^{a}n^{b}=-\frac{\dot{M}(u)}{r^{2}}\,.$

The curvature scalar, or Kretschmann scalar, is the total contraction of the
Riemann tensor with itself. It is related to the analogous invariant for the
Weyl tensor by the following formula (see C. Cherubini, D. Bini, S.
Capozziello, R. Ruffini [2])

$k:=R_{abcd}R^{abcd}=C_{abcd}C^{abcd}+2R_{ab}R^{ab}-\frac{1}{3}\mathrm{Scal}_{g}^{2}\,.$

For Vaidya’s spacetime, we have

$k=R_{abcd}R^{abcd}=C_{abcd}C^{abcd}=\frac{48M(u)^{2}}{r^{6}}\,.$ (4)

The metric (1) is defined on $\mathbb{R}_{u}\times]0,+\infty[_{r}\times
S^{2}_{\omega}$ and describes a radiative white hole whose mass varies with
time as a result of outgoing radiation carried by null dust. It is therefore
natural to assume that $M(u)$ is a non-increasing function of $u$; this
amounts to assuming that the null dust has positive energy. Another natural
assumption is that the mass has finite limits as $u$ tends to $\pm\infty$:

$\lim_{u\rightarrow\pm\infty}M(u)\rightarrow M_{\pm}~{}\mbox{with }0\leq
M_{+}<M_{-}<+\infty\,.$ (5)

In the case where $M$ is constant on $]-\infty,u_{-}]$ and on
$[u_{+},+\infty[$ with $-\infty<u_{-}<u_{+}<+\infty$, we have a Schwarzschild
white hole of mass $M_{-}$ which emits a burst of null radiation between the
retarded times $u_{-}$ and $u_{+}$ and eventually stabilises to a
Schwarzschild black hole of mass $M_{+}<M_{-}$ (unless $M_{+}=0$, in which
case the white hole evaporates completely in finite time). If $M_{+}>0$, the
future event horizon is at $r=2M_{+}$ but the location of the past horizon is
not so clear. For $u<u_{-}$, it is located at $r=2M_{-}$ and it is a null
hypersurface with spherical symmetry. Therefore, it is the hypersurface
generated by the family of curves indexed by $\omega\in\mathrm{S}^{2}$:

$\gamma(u)=(u,r=r(u),\omega)\,,~{}u\in\mathbb{R}\,,$ (6)

that are such that

$r(u)=2M_{-}\mbox{ for }u\leq u_{-}$

and have the property of being null, i.e.

$g(\dot{\gamma}(u),\dot{\gamma}(u))=1-\frac{2M(u)}{r(u)}+2\dot{r}(u)=0\,.$ (7)

Hence, the function $r(u)$ satisfies the following ordinary differential
equation

$\dot{r}(u)=-\frac{1}{2}\left(1-\frac{2M(u)}{r(u)}\right)\,,$ (8)

with $r>0$ and $r(u)=2M_{-}$ for $u\leq u_{-}$.

######  Remark 2.1.

We shall often (starting immediately below) identify the solutions to (8) with
the curves (6) satisfying (7) and simply refer to the integral lines of (8).

If we no longer assume that $M(u)=M_{-}$ in a neighbourhood of $-\infty$, the
past horizon will be spanned by solutions to (8) such that $r>0$ and
$\lim_{u\rightarrow-\infty}r(u)=2M_{-}$.

The ODE (8) is in fact the general equation for a null curve that is
transverse to the level hypersurfaces of $u$ (i.e. to $\nabla u$, that is a
normal and tangent vector field to these hypersurfaces) and orthogonal to the
orbits of the rotation Killing vectors. Vaidya’s spacetime comes equipped with
a null congruence, given by the lines of constant $u$ and $\omega$, which are
the integral lines of

$\nabla u=g^{-1}(\mathrm{d}u)=\frac{\partial}{\partial r}=V\,,$ (9)

where $g^{-1}$ is the inverse Vaidya metric given by

$g^{ab}\partial_{a}\partial_{b}=2\partial_{u}\partial_{r}-\left(1-\frac{2M(u)}{r}\right)\partial_{r}^{2}-r^{-2}\partial_{\omega}^{2}\,,$
(10)

$\partial_{\omega}^{2}$ denoting the euclidean inverse metric on
$\mathrm{S}^{2}$. The integral lines of (8) provide us with a second null
congruence that is transverse to the first one, corresponding to the lines of
constant $v$ and $\omega$ in the case of the Schwarzschild metric.

######  Remark 2.2.

Note that the tangent vector to the integral curves of (8) is exactly

$\frac{\partial}{\partial u}-\frac{1}{2}F\frac{\partial}{\partial r}=W\,.$

The integral curves of (8) are therefore the integral lines of the principal
null vector field $W$. Since the spacetime is not vacuum, we do not have the
Goldberg-Sachs Theorem that would ensure that these are geodesics. However we
shall see in Subsection 3.2 Proposition 3.3 that these curves are indeed
geodesics; they are the family of incoming principal null geodesics and form
the second natural null congruence of Vaidya’s spacetime. Similarly the
integral lines of $V$ are also geodesics (see Proposition 3.2), they are the
outgoing principal null geodesics of Vaidya’s spacetime.

##  3 The incoming principal null congruence

In this section, we analyse the qualitative behaviour of solutions to Equation
(8), with special emphasis on the solutions generating the past horizon. Our
main results are proved under the assumption that

$\dot{M}(u)<0\mbox{ on }]u_{-},u_{+}[\,,~{}-\infty\leq
u_{-}<u_{+}\leq+\infty\,,~{}\dot{M}\equiv 0\mbox{ elsewhere.}$ (11)

This covers the cases where the mass decreases strictly for all retarded times
and where it decreases only on a finite retarded time interval. We dismiss as
physically irrelevant the cases where intervals in $u$ with constant mass
alternate with intervals on which the mass decreases. We also ignore, for
similar reasons, cases where $\dot{M}$ vanishes at isolated points.

###  3.1 General properties

We start with an obvious observation.

######  Lemma 3.1.

On an interval $]u_{0},u_{1}[$ on which $\dot{M}(u)$ does not vanish
everywhere, $r(u)$ cannot be identically equal to $2M(u)$.

Proof. If $r(u)=2M(u)$ satisfies (8) on $]u_{0},u_{1}[$, then

$2\dot{M}(u)=-\frac{1}{2}\left(1-\frac{2M(u)}{2M(u)}\right)=0\mbox{ on
}]u_{0},u_{1}[\,,$

which contradicts the assumption. ∎

Then, we give an important estimate that is a consequence of the local
uniqueness of solutions to the Cauchy problem for (8).

######  Lemma 3.2.

Let $(]u_{1},u_{2}[,r)$ be a solution to (8) such that, for a given
$u_{0}\in]u_{1},u_{2}[$, we have $r(u_{0})\geq 2M(u_{0})$. Let us assume that
$\dot{M}(u)<0$ for all $u\in]u_{1},u_{2}[$, then $r(u)>2M(u)$ on
$]u_{0},u_{2}[$.

Proof. First, note that if $r(u_{0})=2M(u_{0})$, then $\dot{r}(u_{0})=0$,
while $\dot{M}(u_{0})<0$, hence there exists $\varepsilon>0$ such that in
$]u_{0},u_{0}+\varepsilon[$ we have $r(u)>2M(u)$. If $r(u_{0})>2M(u_{0})$,
then we have the same conclusion by continuity.

Now, let $u_{3}$ be the lowest value of $u$ in $]u_{0},u_{2}[$ such that
$r(u)=2M(u)$. Then (8) implies that $\dot{r}(u_{3})=0>2\dot{M}(u_{3})$ and
therefore, there exists $\delta>0$ such that $r(u)<2M(u)$ in
$]u_{3}-\delta,u_{3}[$. By continuity of $r$ and $M$, there exists
$u_{4}\in]u_{0},u_{3}[$ such that $r(u_{4})=2M(u_{4})$. This contradicts the
assumptions on $u_{3}$. It follows that $r(u)>2M(u)$ on $]u_{0},u_{2}[$. ∎

The asymptotic behaviour of maximal solutions to (8) in the past is unstable.
One solution has a finite limit $2M_{-}$; it corresponds to the past event
horizon. All other solutions either end at past null infinity or reach the
past singularity in finite retarded time. The following theorem gives a
complete classification of the solutions to (8) in terms of their behaviour in
the past and also describes precisely their behaviour in the future.

######  Theorem 3.1.

Under Assumptions (2), (5) and (11), there exists a unique maximal solution
$r_{h}$ to (8) such that

$\lim_{u\rightarrow-\infty}r_{h}(u)=2M_{-}\,.$

  * •

If either $M_{+}>0$ or $u_{+}=+\infty$, $r_{h}$ exists on the whole real line,
$r_{h}(u)\rightarrow 2M_{+}$ as $u\rightarrow+\infty$ and any other maximal
solution $r$ to (8) belongs to either of the following two categories:

    1. 1.

$r$ exists on the whole real line, $r(u)>r_{h}(u)$ for all $u\in\mathbb{R}$,
$\lim_{u\rightarrow-\infty}r(u)=+\infty$ and
$\lim_{u\rightarrow+\infty}r(u)=2M_{+}$;

    2. 2.

$r$ exists on $]u_{0},+\infty[$ with $u_{0}\in\mathbb{R}$ and satisfies:
$r(u)<r_{h}(u)$ for all $u\in]u_{0},+\infty[$, $\lim_{u\rightarrow
u_{0}}r(u)=0$ and $\lim_{u\rightarrow+\infty}r(u)=2M_{+}$.

  * •

If $M_{+}=0$ and $u_{+}<+\infty$, $r_{h}$ exists on an interval
$]-\infty,u_{0}[$ with $u_{+}\leq u_{0}<+\infty$ and $\lim_{u\rightarrow
u_{0}}r_{h}(u)=0$. The other maximal solutions are of two types:

    1. 1.

$r$ exists on $]-\infty,u_{1}[$ with $u_{0}\leq u_{1}<+\infty$,
$r(u)>r_{h}(u)$ on $]-\infty,u_{0}[$, $\lim_{u\rightarrow u_{1}}r(u)=0$ and
$\lim_{u\rightarrow-\infty}r(u)=+\infty$;

    2. 2.

$r$ exists on $]u_{1},u_{2}[$ with $-\infty<u_{1}<u_{2}\leq u_{0}$,
$r(u)\rightarrow 0$ as $u$ tends to either $u_{1}$ or $u_{2}$ and
$r(u)<r_{h}(u)$ on $]u_{1},u_{2}[$.

Proof.

Step 1: uniqueness of a maximal solution with finite limit as
$u\rightarrow-\infty$.

First, if a solution $r$ exists on an interval of the form $]-\infty,u_{0}[$
and has a finite limit at $-\infty$, then this limit must be $2M_{-}$. Indeed
let us denote this limit by $l$, using (8),

$\lim_{u\rightarrow-\infty}\dot{r}(u)=-\frac{1}{2}\left(1-\frac{2M_{-}}{l}\right)\,.$

So $\dot{r}$ also has a finite limit at $-\infty$ and this limit must be zero
in order not to contradict the finite limit of $r(u)$, i.e. $l=2M_{-}$.

Then let us show that there is at most one solution to (8) defined on an
interval of the form $]-\infty,u_{0}[$ such that

$\lim_{u\rightarrow-\infty}r(u)=2M_{-}\,.$

Let us assume that there are two such solutions $r_{1}$ and $r_{2}$. Then
$\psi=r_{2}-r_{1}$ satisfies

$\dot{\psi}(u)=\frac{M}{r_{2}}-\frac{M}{r_{1}}=\frac{-M}{r_{1}r_{2}}\psi$ (12)

and

$\lim_{u\rightarrow-\infty}\psi(u)=0\,.$

However, since

$\lim_{u\rightarrow-\infty}\frac{-M}{r_{1}r_{2}}=\frac{-1}{4M_{-}}<0\,,$

if follows that unless $\psi$ is identically zero,

$\left(\log(|\psi|)\right)^{\prime}\longrightarrow\frac{-1}{4M_{-}}\mbox{ as
}u\rightarrow-\infty$

and $\psi$ blows up exponentially fast at $-\infty$. Since we know that $\psi$
tends to zero at $-\infty$, we conclude that $\psi$ is identically zero, i.e.
$r_{1}=r_{2}$.

Step2: construction of the past horizon.

Now we construct a solution to (8) that tends to $2M_{-}$ at $-\infty$. Let us
first consider the case where $u_{-}=-\infty$. For each $n\in\mathbb{N}$, we
define $r_{n}$ to be the maximal solution to (8) such that $r_{n}(-n)=2M(-n)$.
It exists on an interval of the form $]u^{1}_{n},u^{2}_{n}[$,
$u^{1}_{n}<-n<u^{2}_{n}$. Let $u^{3}_{n}=\min\\{u^{2}_{n},u_{+}\\}$. By Lemma
3.2, $r_{n}(u)>2M(u)$ on $]-n,u^{3}_{n}[$, hence $\dot{r}_{n}<0$ there and it
follows that $r_{n}(u)<2M(-n)$. These a priori bounds imply that
$u_{n}^{2}\geq u_{+}$. Therefore, $u^{2}_{n}=+\infty$ in the case where
$u_{+}=+\infty$. If $u_{+}<+\infty$ and $M_{+}>0$, then we could have
$r_{n}(u_{+})=2M_{+}$, in which case $u_{n}^{2}=+\infty$ and $r_{n}(u)=2M_{+}$
for $u\geq u_{+}$, or $r_{n}(u_{+})>2M_{+}$ and then $r_{n}(u)>2M_{+}$ on
$[u_{+},u_{n}^{2}[$ since two solutions cannot cross, whence $\dot{r}_{n}$ is
negative there and we infer $u_{n}^{2}=+\infty$. If $u_{+}<+\infty$ and
$M_{+}=0$ then on $]u_{+},u_{n}^{2}[$, $r_{n}$ satisfies the simple ODE

$\dot{r}_{n}=-\frac{1}{2}\,,$

and $r_{n}$ reaches $0$ in finite retarded time. Hence in this case
$u_{n}^{2}<+\infty$.

Using again the fact that, by uniqueness of solutions to the Cauchy problem
for (8), two solutions cannot cross, we infer that the sequence $u_{n}^{2}$ is
increasing. Let

$u^{2}=\lim_{n\rightarrow+\infty}u^{2}_{n}\,.$

For any compact interval $I$ of $]-\infty,u^{2}[$, there exists
$n_{0}\in\mathbb{N}$ such that the sequence $(r_{n})_{n\geq n_{0}}$ is well-
defined, increasing and bounded on $I$. Hence, Lebesgue’s dominated
convergence theorem implies that the sequence $(r_{n})$ converges in
$L^{1}_{\mathrm{loc}}(]-\infty,u^{2}[)$ towards a positive function $r_{h}$
such that

$2M(u)\leq r_{h}(u)\leq 2M_{-}~{}\forall u\in]-\infty,u^{2}[\,.$ (13)

Moreover, $1/r_{n}$ also converges towards $1/r_{h}$ in
$L^{1}_{\mathrm{loc}}(]-\infty,u^{2}[)$, because, for any given compact
interval $I$, it is a well-defined, decreasing and bounded sequence on $I$ for
$n$ large enough. This implies, by equation (8) for $r_{n}$, that
$\dot{r}_{n}$ converges in $L^{1}_{\mathrm{loc}}(]-\infty,u^{2}[)$ and by
uniqueness of the limit in the sense of distributions, the limit must be
$\dot{r}_{h}$. Consequently $r_{h}$ is a solution to (8) in the sense of
distributions. An easy bootstrap argument then shows that $r_{h}$ is a strong
solution to (8) and is in fact smooth on $]-\infty,u^{2}[$.

Besides, by (13) and the fact that $M(u)\rightarrow M_{-}$ as
$u\rightarrow-\infty$, it follows that

$\lim_{u\rightarrow-\infty}r_{h}(u)=2M_{-}\,.$

In the case where $u_{-}>-\infty$, we simply need to consider the maximal
solution to (8) such that $r(u_{-}-1)=2M_{-}$. This solution exists on an
interval of the form $]-\infty,u^{2}[$ and satisfies (13).

Let us now turn to the value of $u^{2}$ and the behaviour of $r_{h}$ in the
future. If either $M_{+}>0$ or $u_{+}=+\infty$, then by (13), we have
$u^{2}=+\infty$. By (13) again, $\dot{r}_{h}(u)<0$ on $\mathbb{R}$ and it
follows that $r_{h}(u)$ has a finite limit $l$ as $u\rightarrow+\infty$. If
$M_{+}>0$, we have

$\dot{r}_{h}(u)\rightarrow-\frac{1}{2}\left(1-\frac{2M_{+}}{l}\right)\mbox{ as
}u\rightarrow+\infty$

and we must have $l=2M_{+}$ or contradict the finite limit of $r_{h}$. If
$M_{+}=0$ and $u_{+}=+\infty$, then if $l\neq 0$,

$\dot{r}_{h}(u)\rightarrow-\frac{1}{2}\mbox{ as }u\rightarrow+\infty$

which is incompatible with $u^{2}=+\infty$. Finally, if $M_{+}=0$ and
$u_{+}<+\infty$, then (13) implies that $u^{2}\geq u^{+}$. If $r_{h}(u_{+})=0$
then the solution terminates at $u=u_{+}$, $u^{2}=u_{+}$. Otherwise,
$u^{2}>u_{+}$ and on $[u_{+},u^{2}[$ we have

$\dot{r}_{h}(u)=-\frac{1}{2}\,,$

as long as $r_{h}(u)$ remains positive. Therefore, we have

$r_{h}(u)=r_{h}(u_{+})-(u-u_{+})\mbox{ for }u_{+}\leq u\leq
u_{+}+r_{h}(u_{+})$

and the integral curve ends at $u=u_{+}+r_{h}(u_{+})$, i.e. $u^{2}$ is finite
and is equal to $u_{+}+r_{h}(u_{+})$. Hence for $M_{+}=0$ and $u_{+}<+\infty$,
the past event horizon vanishes in finite retarded time $u_{+}+r_{h}(u_{+})$
and there is no future event horizon.

Step 3: classification of the other maximal solutions.

  * •

We begin with the case where either $u_{+}=+\infty$ or $M_{+}>0$. Let
$(]u_{1},u_{2}[,r)$ be a maximal solution to (8). Let $u_{0}\in]u_{1},u_{2}[$
and assume that $0<r(u_{0})<r_{h}(u_{0})$ (resp. $r(u_{0})>r_{h}(u_{0})$). By
uniqueness of solutions to the Cauchy problem for (8), solutions cannot cross,
so for all $u\in]u_{1},u_{2}[$ we have $0<r(u)<r_{h}(u)$ (resp.
$r(u)>r_{h}(u)$).

    1. 1.

Case where $0<r(u_{0})<r_{h}(u_{0})$. Let us first assume that
$r(u_{0})>2M(u_{0})$. If $r(u)>2M(u)$ on its interval of existence, then
$r(u)$ is bounded between $2M(u)$ and $r_{h}(u)$ and we must have
$]u_{1},u_{2}[\,=\mathbb{R}$. However, we then have $r(u)\rightarrow 2M_{-}$
as $u\rightarrow-\infty$ and this contradicts the uniqueness of $r_{h}$. It
follows that there exists $u_{3}\in]u_{1},u_{2}[$ such that
$r(u_{3})=2M(u_{3})$. Therefore $r(u)<2M(u)$ on $]u_{1},u_{3}[$ (the proof is
similar to that of Lemma 3.2), $r$ is an increasing function on this interval
and $\dot{r}$ is decreasing. This implies that $r(u)$ must reach $0$ in finite
time in the past and keep on existing towards the past as long as it has not
reached $0$. Hence $u_{1}>-\infty$ and $r(u)\rightarrow 0$ as $u\rightarrow
u_{1}$. Since $r(u_{3})=2M(u_{3})$, then we have by Lemma 3.2 that
$r(u)>2M(u)$ on $]u_{3},u_{+}[$ and since solutions do not cross, $r(u)\geq
2M(u)$ on $[u_{+},u_{2}[$. Hence

$2M(u)\leq r(u)<r_{h}(u)\mbox{ on }]u_{0},u_{2}[\,.$

This implies that $u_{2}=+\infty$ and $\lim_{u\rightarrow+\infty}r(u)=2M_{+}$.

If $r(u_{0})=2M(u_{0})$, then we can repeat the arguments above, replacing
$u_{3}$ by $u_{0}$; we infer: $u_{1}>-\infty$ and $\lim_{u\rightarrow
u_{1}}r(u)=0$, $u_{2}=+\infty$ and $\lim_{u\rightarrow+\infty}r(u)=2M_{+}$.

If $r(u_{0})<2M(u_{0})$ then $r(u)$ increases as long as $r(u)<2M(u)$. Either
$r(u)<2M(u)$ on its whole interval of existence (note that this requires
$M_{+}>0$), in which case $u_{2}=+\infty$, or there exists
$u_{4}\in]u_{0},u_{2}[$ such that $r(u_{4})=2M(u_{4})$ and we can then use the
same reasoning as before on $]u_{4},u_{2}[$ and infer that $u_{2}=+\infty$. In
the latter case, we have as before $\lim_{u\rightarrow+\infty}r(u)=2M_{+}$. In
the former, $r(u)$ has a finite positive limit $l$ as $u\rightarrow+\infty$
and (recall that we must have $M_{+}>0$)

$\dot{r}(u)\rightarrow-\frac{1}{2}\left(1-\frac{2M_{+}}{l}\right)\mbox{ as
}u\rightarrow+\infty$

and we must have $l=2M_{+}$ in order not to contradict $u_{2}=+\infty$. In
both cases, we have $u_{1}>-\infty$ and $r(u)\rightarrow 0$ as $u\rightarrow
u_{1}$.

    2. 2.

If $r(u_{0})>r_{h}(u_{0})$ then $r$ is a decreasing function on its interval
of existence and is bounded below by $r_{h}$. This implies that
$u_{2}=+\infty$. Moreover, on its whole interval of existence, $r$ satisfies

$-\frac{1}{2}<\dot{r}(u)<0$

and it follows that $u_{1}=-\infty$. Since $r$ is a decreasing function on
$\mathbb{R}$, it has a limit as $u\rightarrow-\infty$ and we have seen above
that this limit cannot be finite, hence

$\lim_{u\rightarrow-\infty}r(u)=+\infty\,.$

The solution $r$ also has a limit $l$ as $u\rightarrow+\infty$ and since $r$
is a decreasing function and $r(u)>r_{h}(u)>2M(u)$ on $\mathbb{R}$, it follows
that $2M_{+}\leq l<+\infty$. If $M_{+}>0$, then $l>0$ and $\dot{r}$ also has a
limit as $u\rightarrow+\infty$ given by

$\lim_{u\rightarrow+\infty}\dot{r}(u)=-\frac{1}{2}\left(1-\frac{2M_{+}}{l}\right)\,.$

This must be zero in order not to contradict the finite limit of $r$. Hence
$l=2M_{+}$. If $M+=0$, then unless $l=0$ we have that

$\lim_{u\rightarrow+\infty}\dot{r}(u)=-\frac{1}{2}$

which implies that $r(u)$ must reach $0$ in finite retarded time and
contradicts $u_{2}=+\infty$. Hence in this case we have $l=0=2M_{+}$.

  * •

In the case where $M_{+}=0$ and $u_{+}<+\infty$, the proof uses exactly the
same arguments as in step 3 and the end of step 2. ∎

If the mass decreases only for a finite range of $u$, we have not been able to
rule out cases for which we have $r_{h}(u_{+})=2M_{+}$, nor have we managed to
find explicit examples of this situation. It is however easy to see that there
are cases for which $r_{h}(u)>2M_{+}$ for all $u\in\mathbb{R}$.

######  Proposition 3.1.

In the case where $u_{\pm}$ are both finite, assume that
$u_{+}-u_{-}<4(M_{-}-M_{+})$, then $r_{h}(u_{+})>2M_{+}$ and therefore
$r_{h}(u)>2M_{+}$ for all $u\in\mathbb{R}$.

Proof. It is a simple observation. We have

$\displaystyle r_{h}(u_{+})-r_{h}(u_{-})$ $\displaystyle=$
$\displaystyle\int_{u_{-}}^{u_{+}}\dot{r}_{h}(u)\mathrm{d}u$ $\displaystyle=$
$\displaystyle-\frac{1}{2}\int_{u_{-}}^{u_{+}}\left(1-\frac{2M(u)}{r_{h}(u)}\right)\mathrm{d}u$
$\displaystyle=$
$\displaystyle-\frac{1}{2}(u_{+}-u_{-})+\int_{u_{-}}^{u_{+}}\frac{M(u)}{r_{h}(u)}\mathrm{d}u>-\frac{1}{2}(u_{+}-u_{-})\,.$

Since $r_{h}(u_{-})=2M_{-}$,

$r(u_{+})>2M_{-}-\frac{1}{2}(u_{+}-u_{-})>2M_{+}\,.$

Since on $]u_{+},+\infty[$, $2M_{+}$ is a solution to (8), then by uniqueness
of solutions we must have $r_{h}(u)>2M_{+}$ for all $u>u_{+}$. This proves the
proposition. ∎

###  3.2 The second optical function

The function $u$ is an optical function, which means that its gradient is a
null vector field, or equivalently that $u$ satisfies the eikonal equation

$g(\nabla u,\nabla u)=0\,.$ (14)

An important property of optical functions is that the integral lines of their
gradient are null geodesics with affine parametrisation. This is established
in [8]. The more complete Propositions (7.1.60) and (7.1.61) in Penrose and
Rindler Vol 2 [11] state that for a null congruence, the following three
properties are equivalent :

  1. 1.

it is hypersurface-orthogonal;

  2. 2.

it is hypersurface-forming;

  3. 3.

it is geodetic and twist-free.

We recall the proof of the fact that the integral curves of an optical
function are null geodesics, as it is a straightforward calculation.

######  Lemma 3.3.

Let $\xi$ be an optical function and denote $\cal L=\nabla\xi$. The integral
curves of $\cal L$ are geodesics and $\cal L$ corresponds to a choice of
affine parameter, i.e.

$\nabla_{\cal L}{\cal L}=0\,.$

Proof. The proof is direct :

$\displaystyle\nabla_{\cal L}{\cal L}^{b}$ $\displaystyle=$
$\displaystyle\nabla_{\nabla\xi}{\nabla^{b}\xi}\,,$ $\displaystyle=$
$\displaystyle\nabla_{a}\xi\nabla^{a}\nabla^{b}\xi\,,$ $\displaystyle=$
$\displaystyle\nabla_{a}\xi\nabla^{b}\nabla^{a}\xi\mbox{ since the connection
is torsion-free,}$ $\displaystyle=$
$\displaystyle\nabla^{b}\left(\nabla_{a}\xi\nabla^{a}\xi\right)-\left(\nabla^{b}\nabla_{a}\xi\right)\nabla^{a}\xi\,,$
$\displaystyle=$ $\displaystyle 0-\nabla_{a}\xi\nabla^{a}\nabla^{b}\xi\mbox{
since }\nabla\xi\mbox{ is null and the connection torsion-free,}$
$\displaystyle=$ $\displaystyle-\nabla_{\nabla\xi}\nabla^{b}\xi\,.\qed$

Since (see (9))

$\nabla u=\frac{\partial}{\partial r}=V$

and $V$ is a principal null direction of the Weyl tensor, a consequence of
Lemma 3.3 and of (14) is the following.

######  Proposition 3.2.

The integral lines of $V$ are affinely parametrised null geodesics; they are
the outgoing principal null geodesics of Vaidya’s spacetime.

We now establish the existence of a second optical function.

######  Proposition 3.3.

There exists a function $v$ defined on
$\mathbb{R}_{u}\times]0,+\infty[_{r}\times S^{2}_{\omega}$, depending solely
on $u$ and $r$, such that $\nabla v$ is everywhere tangent to the integral
lines of (8). This means that $g(\nabla v,\nabla v)=0$, i.e. $v$ is an optical
function. The integral lines of (8), which are also the integral lines of
$\nabla v$, are therefore null geodesics and their congruence generates the
level hypersurfaces of $v$. Since the integral lines of (8) are also tangent
to $W$ (defined in (3)) it follows that they are the incoming principal null
geodesics of Vaidya’s spacetime.

Proof. The metric $g$ can be written as

$g=F\mathrm{d}u\left(\mathrm{d}u+2F^{-1}\mathrm{d}r\right)-r^{2}\mathrm{d}\omega^{2}\,.$

Following the construction of $v$ for the Schwarzschild metric, it is tempting
to put

$\mathrm{d}v=\mathrm{d}u+2F^{-1}\mathrm{d}r=2F^{-1}g^{-1}(W)\,,$

however this $1$-form is not closed since

$\mathrm{d}\left(\mathrm{d}u+2F^{-1}\mathrm{d}r\right)=-2F^{-2}\frac{\partial
F}{\partial
u}\mathrm{d}u\wedge\mathrm{d}r=\frac{4\dot{M}(u)}{rF^{2}}\mathrm{d}u\wedge\mathrm{d}r$

which vanishes identically only if the mass $M$ is constant, i.e. in the
Schwarzschild case. We introduce an auxiliary function $\psi>0$ and we write

$g=\frac{F}{\psi}\mathrm{d}u\left(\psi\mathrm{d}u+2\psi
F^{-1}\mathrm{d}r\right)-r^{2}\mathrm{d}\omega^{2}\,.$

Our purpose is to find conditions on $\psi$ that ensure that the $1$-form
$\alpha:=\psi\mathrm{d}u+2\psi F^{-1}\mathrm{d}r$ is exact. Since we work in
the variables $(u,r)$ on the simply connected domain
$\mathbb{R}_{u}\times]0,+\infty[_{r}$, all that is required is that $\alpha$
be closed, i.e. that

$\mathrm{d}\alpha=2\frac{\partial}{\partial
u}\left(\frac{\psi}{F}\right)-\frac{\partial\psi}{\partial r}=0\,.$

This equation has the more explicit form

$\frac{\partial\psi}{\partial u}-\frac{F}{2}\frac{\partial\psi}{\partial
r}+\frac{2}{F}\frac{\dot{M}}{r}\psi=0\,.$ (15)

This is an ordinary differential equation along the integral lines of the
second principal null direction (defined in (3))

$W=\frac{\partial}{\partial u}-\frac{F}{2}\frac{\partial}{\partial r}$

parametrised by $u$. Let $\gamma(u)=(u,r(u),\omega)$ be an integral line of
$W$ (which is equivalent to $r(u)$ being a solution to (8)), Equation (15)
along $\gamma$ reads

$\frac{\mathrm{d}}{\mathrm{d}u}(\psi\circ\gamma)=\left(-\frac{2\dot{M}}{rF}\psi\right)\circ\gamma\,,$

or equivalently

$\frac{\mathrm{d}}{\mathrm{d}u}\left(\log\left|\psi\circ\gamma\right|\right)=\left(-\frac{2\dot{M}}{rF}\right)\circ\gamma\,.$
(16)

Equation (15) can therefore be integrated as follows. First, we take a
hypersurface transverse to all the integral lines of (8), for instance
$\mathcal{S}=\\{u=0\\}$ and, we fix the value of $\psi$ on $\mathcal{S}$, say
$\psi=1$ on $\mathcal{S}$. Then we evaluate $\psi$ on each integral line of
(8) by solving the ODE (16). Since the integral lines of (8) are a congruence
of $\mathbb{R}_{u}\times]0,+\infty[_{r}\times S^{2}_{\omega}$, this allows to
define $\psi$ on this whole domain as a smooth (by smooth dependence on
initial data) and nowhere vanishing function. The $1$-form $\alpha$ is then
closed on $\mathbb{R}_{u}\times]0,+\infty[_{r}\times S^{2}_{\omega}$. Since
$\alpha$ depends only on $u$ and $r$, we may see it as a closed $1$-form on
$\mathbb{R}_{u}\times]0,+\infty[_{r}$ which is simply connected. It follows
that $\alpha$ is exact on $\mathbb{R}_{u}\times]0,+\infty[_{r}\times
S^{2}_{\omega}$ and modulo a choice of hypersurface $\Sigma$ generated by the
integral lines of (8), we can define a function $v$ on
$\mathbb{R}_{u}\times]0,+\infty[_{r}\times S^{2}_{\omega}$ such that $v=0$ on
$\Sigma$ and $\alpha=\mathrm{d}v$. In particular,

$\mathrm{d}v=2\psi F^{-1}g^{-1}(W)\mbox{ and }\nabla v=2\psi F^{-1}W\,.\qed$

##  4 Case of a complete evaporation in infinite time

We now devote particular attention to the case where $M_{+}=0$ and
$u_{+}=+\infty$. As before, we assume that $\dot{M}<0$ on $]u_{-},+\infty[$.

###  4.1 The asymptotic null singularity

As we have established in Theorem 3.1, the past event horizon ends up at $r=0$
as $u\rightarrow+\infty$ and so do all the integral curves of (8), i.e. all
the incoming principal null geodesics. From this, we infer the following
theorem.

######  Theorem 4.1.

Whatever the speed at which $M(u)\rightarrow 0$ as $u\rightarrow+\infty$, we
have a null singularity of the conformal structure in the future of our
spacetime. More precisely, the Kretschmann scalar $k$ does not remain bounded
as $u\rightarrow+\infty$ along any integral line of (8).

Proof. Consider $(]u_{0},+\infty[,r)$ a maximal solution to (8), with
$u_{0}\in\mathbb{R}\cup\\{-\infty\\}$. Assume that $k$ remains bounded along
the integral line as $u\rightarrow+\infty$. Then, using (4), so does $M/r^{3}$
and it follows that $M/r$ tends to $0$ as $u\rightarrow+\infty$ along the
integral line. This implies in turn that $\dot{r}(u)\rightarrow-1/2$ as
$u\rightarrow+\infty$, which contradicts the fact that $r(u)\rightarrow 0$ as
$u\rightarrow+\infty$. ∎

######  Remark 4.1.

If we assume that along the integral lines of (8), $\dot{r}(u)$ has a limit as
$u\rightarrow+\infty$, this limit is necessarily zero in order not to
contradict the fact that $r(u)\rightarrow 0$ as $u\rightarrow+\infty$. This
implies in turn that along the integral line,

$\frac{M(u)}{r(u)}\rightarrow\frac{1}{2}\mbox{ as }u\rightarrow+\infty\,,$

i.e.

$r(u)\simeq 2M(u)\mbox{ as }u\rightarrow+\infty$ (17)

and

$k\simeq\frac{3}{4M(u)^{4}}\mbox{ as }u\rightarrow+\infty\,.$ (18)

###  4.2 A family of uniformly timelike congruences

Some uniformly timelike curves also end up at $r=0$ as $u\rightarrow+\infty$.
Let us consider a curve

$\gamma(u)=(u,r(u),\theta,\varphi)\,,$

such that

$g(\dot{\gamma}(u),\dot{\gamma}(u))=\varepsilon^{2}\,,~{}\varepsilon>0\,,$
(19)

then $r$ satisfies the differential equation

$\dot{r}(u)=\frac{\varepsilon^{2}}{2}-\frac{1}{2}\left(1-\frac{2M(u)}{r(u)}\right)\,.$
(20)

The tangent vector is

$\tau=\partial_{u}+\left(\frac{\varepsilon^{2}}{2}-\frac{1}{2}\left(1-\frac{2M(u)}{r}\right)\right)\partial_{r}$

and

$\nabla_{\tau}\tau=-\frac{M(u)}{r^{2}}\left(\tau-\varepsilon^{2}\partial_{r}\right)\,,$

so the integral curves of (20) are not geodesics, except for $\varepsilon=0$.
The behaviour of the integral curves of (20) changes radically according to
the value of $\varepsilon$. This is detailed in the next two propositions. The
first one deals with the case where $0<\varepsilon<1$.

######  Proposition 4.1.

Let $0<\varepsilon<1$ be given. There exists a unique maximal solution
$r_{\varepsilon}$ to (20) such that

$\lim_{u\rightarrow-\infty}r_{\varepsilon}(u)=\frac{2M_{-}}{1-\varepsilon^{2}}\,.$

This solution exists on the whole real line and $r_{\varepsilon}(u)\rightarrow
0$ as $u\rightarrow+\infty$. Any other maximal solution $r$ to (8) belongs to
either of the two categories :

  1. 1.

$r$ exists on the whole real line, $r(u)>r_{\varepsilon}(u)$ for all
$u\in\mathbb{R}$, $\lim_{u\rightarrow-\infty}r(u)=+\infty$ and
$\lim_{u\rightarrow+\infty}r(u)=0$ ;

  2. 2.

$r$ exists on $]u_{0},+\infty[$ with $u_{0}\in\mathbb{R}$ and satisfies :
$r(u)<r_{\varepsilon}(u)$ for all $u\in]u_{0},+\infty[$ and $r(u)$ tends to
$0$ as $u\rightarrow u_{0}$ and as $u\rightarrow+\infty$.

Moreover, the Kretschmann scalar $k$ fails to be bounded on the integral lines
of (20) as $u\rightarrow+\infty$. If we assume moreover that $\dot{r}$ has a
limit as $u\rightarrow+\infty$ along the integral lines of (20), then we have
a similar behaviour for $k$ to that described in Remark 4.1 for the integral
lines of (8), namely

$k\simeq\frac{3(1-\varepsilon^{2})^{6}}{4M(u)^{4}}\mbox{ as
}u\rightarrow+\infty\,.$ (21)

The second proposition treats the cases where $\varepsilon\geq 1$.

######  Proposition 4.2.

If $\varepsilon\geq 1$, then all maximal solutions to (20) exist on a interval
$]u_{0},+\infty[$ with $u_{0}>-\infty$, $r$ is strictly increasing on
$]u_{0},+\infty[$ and $r(u)\rightarrow 0$ as $u\rightarrow u_{0}$. Moreover:

  * •

if $\varepsilon=1$, then the limit of $r(u)$ as $u\rightarrow+\infty$ is
finite if and only if $M(u)$ is integrable in the neighbourhood of $+\infty$;

  * •

if $\varepsilon>1$, then $r(u)\rightarrow+\infty$ as $u\rightarrow+\infty$.

######  Remark 4.2.

In view of (19), the proper time along an integral curve of (20) is exactly
given (after an adequate choice of origin) by $\tau=\varepsilon u$. Therefore,
the behaviour of the integral lines of (20) as $u\rightarrow+\infty$ described
in Propositions 4.1 and 4.2 corresponds to their behaviour as proper time
tends to $+\infty$.

Proof of Proposition 4.1. We observe that Equation (20) can be transformed to
(8). We put

$\tilde{r}(u)=\frac{r(u)}{1-\varepsilon^{2}}\,,~{}\tilde{M}(u)=\frac{M(u)}{(1-\varepsilon^{2})^{2}}\,,$

then Equation (20) becomes

$\dot{\tilde{r}}(u)=-\frac{1}{2}\left(1-\frac{2\tilde{M}(u)}{\tilde{r}(u)}\right)\,.$

The classification of maximal solutions therefore follows directly from
Theorem 3.1. The derivation of the behaviour of the Kretschmann scalar along
an integral line is also similar to the null case. The proof of the lack of
boundedness is the same as that of Theorem 4.1 and assuming that $\dot{r}$ has
a limit as $u\rightarrow+\infty$ along an integral line of (20), this limit
must be zero and we infer

$r(u)\simeq\frac{2M(u)}{1-\varepsilon^{2}}\,.$

Then (21) follows from (4). ∎

Proof of Proposition 4.2. Let $(]a,b[,r)$ be a maximal solution to (20).

  * •

Case where $\varepsilon=1$. The differential equation (20) becomes

$\dot{r}(u)=\frac{M(u)}{r(u)}\,,$

or equivalently

$\frac{\mathrm{d}}{\mathrm{d}u}((r(u))^{2})=2M(u)\,.$

The function $r$ is strictly increasing and given $u_{1}\in]a,b[$, we have for
all $u\in]a,b[$

$r(u)^{2}=r(u_{1})^{2}+2\int_{u_{1}}^{u}M(s)\mathrm{d}s\,.$

Then $a$ is finite, strictly lower than $u_{1}$ and is precisely such that

$\int^{u_{1}}_{a}M(s)\mathrm{d}s=\frac{r(u_{1})^{2}}{2}\,.$

Also $b=+\infty$ and

$\lim_{u\rightarrow+\infty}r(u)^{2}=r(u_{1})^{2}+2\int_{u_{1}}^{+\infty}M(s)\mathrm{d}s\,.$

  * •

Case where $\varepsilon>1$. Now for all $u\in]a,b[$,

$\dot{r}(u)>\frac{\varepsilon^{2}-1}{2}>0\,.$ (22)

It follows that $a$ is finite and

$\lim_{u\rightarrow a}r(u)=0\,.$

Moreover, let $u_{1}\in]a,b[$, then using the fact that $r$ is strictly
increasing on $]a,b[$, we have for all $u\in]u_{1},b[$,

$\dot{r}(u)<\frac{\varepsilon^{2}-1}{2}+\frac{M_{+}}{r(u_{1})}\,,$

whence $b=+\infty$ and (22) implies

$\lim_{u\rightarrow+\infty}r(u)=+\infty\,.\qed$

###  4.3 The global structure of the spacetime

The two congruences of null geodesics that we have considered (the curves of
constant $(u,\omega)$ and of constant $(v,\omega)$) are inextendible. The
spacetime is therefore maximally extended and we have two global charts
$\mathbb{R}_{u}\times]0,+\infty[_{r}\times S^{2}_{\omega}$ and
$\mathbb{R}_{v}\times]0,+\infty[_{r}\times S^{2}_{\omega}$. Figures 2 to 4
display the Penrose diagram of Vaidya’s spacetime for a mass function that
decreases strictly on the whole real line, tends to $0$ as
$u\rightarrow+\infty$ and to a finite positive limit $M_{+}$ as
$u\rightarrow-\infty$, with the general forms of various congruences: the
incoming principal null geodesics (lines of constant $(v,\omega)$) in Figure
2, the outgoing principal null geodesics (lines of constant $(u,\omega)$) in
Figure 2, the timelike curves given by the integral lines of (20) for
$0<\varepsilon<1$ in Figure 4 and for $\varepsilon\geq 1$ in Figure 4. The
dashed lines are curvature singularities. The null singularity in the future
is an asymptotic singularity.

Figure 1: Incoming principal null congruence: lines of constant $(v,\omega)$.

Figure 2: Outgoing principal null congruence: lines of constant $(u,\omega)$.

Figure 3: The timelike congruence for $\varepsilon<1$.

Figure 4: The timelike congruence for $\varepsilon\geq 1$.

Note that for the fourth figure, the general form of the congruence is the
same for $\varepsilon>1$ and for $\varepsilon=1$, independently of the
integrability of $M(u)$ near $u=+\infty$, because all curves end up at future
timelike infinity, whether the limit of $r$ along the curve is positive and
finite or infinite.

## References

  * [1] V.A. Berezin, V.I. Dokuchaev and Y.N. Eroshenko, On maximal analytical extension of the Vaidya metric with linear mass function, Classical and Quantum Gravity, 33 (2016), 14, 145003.
  * [2] C. Cherubini, D. Bini, S. Capozziello, R. Ruffini, Second order scalar invariants of the Riemann tensor: application to black hole spacetimes, Int. J. Mod. Phys. D, 11 (2002), 6, 827–841.
  * [3] I. Booth, J. Martin, On the proximity of black hole horizons: lessons from Vaidya, Phys. Rev. D 82 (2010), 124046.
  * [4] F. Fayos, M.M. Martin-Prats and J.M.M. Senovilla, On the extension of Vaidya and Vaidya-Reissner-Nordström spacetimes, Class. Quantum Grav. 12 (1995), 2565–2576.
  * [5] F. Fayos and R. Torres, A class of interiors for Vaidya’s radiating metric: singularity-free gravitational collapse, Class. Quantum Grav., 25 (2008), 175009.
  * [6] E. Gourgoulhon, M. Bejger, M. Mancini, Tensor calculus with open-source software: the SageManifolds project, Journal of Physics: Conference Series 600 (2015), 012002, doi:10.1088/1742-6596/600/1/012002.
  * [7] J.B. Griffiths, J. Podolský, Exact spacetimes in Einstein’s general relativity, Cambridge Monographs on Mathematical Physics, Cambridge University Press, 2009.
  * [8] D. Häfner, J.-P. Nicolas, The characteristic Cauchy problem for Dirac fields on curved backgrounds J. Hyperbolic Differ. Equ. 8 (2011), 3, 437–483.
  * [9] W.A. Hiscock, Models of evaporating black holes: II. Effects of the outgoing created radiation, phys. Rev. D 23 (1981), 12, 2823–2827.
  * [10] W. Israel, Gravitational collapse of a radiating star, Physics Letters 24A (1967), 3, 184–186.
  * [11] R. Penrose, W. Rindler, Spinors and spacetime, Vol. I (1984) and Vol. 2 (1986), Cambridge University Press.
  * [12] A.Z. Petrov, The classification of spaces defining gravitational fields, Scientific Proceedings of Kazan State University (named after V.I. Ulyanov-Lenin), Jubilee (1804-1954) Collection 114 (1954),8 ,55-69, translation by J. Jezierski and M.A.H. MacCallum, with introduction, by M.A.H. MacCallum, Gen. Rel. Grav. 32 (2000), 1661–1685.
  * [13] P.C. Vaidya, The external field of a radiating star in general relativity, Current Sci. (India), 12 (1943), 183.
  * [14] P.C. Vaidya, “Newtonian” time in general relativity, Nature 171 (1953), 260–261.


plus1sp

[1]Pablo Gamallo Pablo Gamallo

# Contextualized Word Senses: From Attention to Compositionality

###### Abstract

The neural architectures of language models are becoming increasingly complex,
especially that of Transformers, based on the attention mechanism. Although
their application to numerous natural language processing tasks has proven to
be very fruitful, they continue to be models with little or no
interpretability and explainability. One of the tasks for which they are best
suited is the encoding of the contextual sense of words using contextualized
embeddings. In this paper we propose a transparent, interpretable, and
linguistically motivated strategy for encoding the contextual sense of words
by modeling semantic compositionality. Particular attention is given to
dependency relations and semantic notions such as selection preferences and
paradigmatic classes. A partial implementation of the proposed model is
carried out and compared with Transformer-based architectures for a given
semantic task, namely the similarity calculation of word senses in context.
The results obtained show that it is possible to be competitive with
linguistically motivated models against the black boxes underlying complex
neural architectures.

##  1 Introduction

Although the success of applying deep neural networks in natural language
processing is undeniable and compelling, there are well-founded criticisms
about its inability to mimic the generalization capacity, which is probably
the main characteristic of human intelligence (Lake etal.,, 2017; Marcus,,
2018). One of the most common and recurrent criticisms is related to the
notion of compositionality as a mechanism to produce systematic
generalizations. The principle of compositionality was introduced by Frege in
the early 20th century and reformulated more recently by Montague, (1970) and
Partee, (2004). It states that the meaning of a complex expression is
determined by the meaning of its component parts and the way in which they are
combined. Even though some kind of compositional behaviour can emerge from the
outputs returned by deep neural networks, it is not clear how these systematic
generalizations are carried out.

The work of Nefdt, (2020), Baroni, (2020) and Dankers etal., (2022) suggests
that large neural models are able to generalize and capture composite meaning
without learning the explicit rules governing the combination of components.
We are not in a position to determine whether these generalization methods are
more or less efficient than those we humans use to construct meaning. They are
probably different, as shown by the enormous amount of information that an
artificial system requires for the properties of generalization and
comprehension to emerge, with respect to the input that the human brain
receives in language acquisition (Warstadt and Bowman,, 2020). Ettinger,
(2020) argues that what probably makes the human brain and artificial neural
models different is that the former extracts the meaning of sentences by
compositional mechanisms to make judgments of truth, while the latter rely on
the technique of prediction in context. In human acquisition, compositional
abilities are at the core of the semantic interpretation process and
prediction is only used to strengthen these abilities. By contrast, in
artificial neural networks, prediction is the core process of self-learning
and compositional abilities only emerges from massive prediction on huge
amounts of textual data. In sum, predictions in large language models are at
the heart of the learning process and are driven mainly by superficial
contextual cues, rather than by robust and systematic representations of
context meaning (Pandia and Ettinger,, 2021).

It has been observed that lack of compositionality is one of the main reasons
why neural networks, unlike humans, rely on large amounts of data to make
correct generalizations (Lake etal.,, 2017). However, even with those large
amounts of training data, the neural networks can be easily fooled by
adversarial examples that pose no problem for humans (Ebrahimi etal.,, 2018).
This situation has led some researchers to emphasize data quality by proposing
strategies that focus more on the quality of the input data than on the big
quantity of information, which makes large language models statistical parrots
(Bender etal.,, 2021).

Given the mentioned weaknesses of neural networks, the question arises as to
whether it is possible to provide them with more modular and structured
knowledge, making them more compositional, while maintaining their strengths:
fast, efficient and with great adaptive capacity. In fact, interest in neuro-
symbolic systems is growing up with the intention of compensating for the
shortcomings of systems based purely on artificial neural models (Marcus and
Davis,, 2019). While the composition rules of symbolic methods are transparent
and easily interpretable, they might be too rigid to cope with noise and fuzzy
constructions of natural language. Neural networks, by contrast, seem much
better adapted to noisy scenarios, but they are incapable of carrying out, as
we have already said, systematic routines of a compositional nature essential
for natural language processing (Marcus,, 2003). The combination of both
strategies, neural networks and symbolic strategies, could be a fruitful step
to rethink large language models and make them more transparent and
interpretable.

The objective of this paper is to propose a new symbolic-based language
modeling strategy to encode the sense of words in context. More precisely, we
define a semantic method that uses compositional rules applied on dependency
trees to both build the meaning of complex expressions and to contextualize
the sense of the constituent words. In addition, we compare our linguistically
motivated method with the attention mechanism of neural-based Transformers in
producing contextualized vectors for each word of the input sentence. The main
difference between the two methods is that our syntax-based strategy makes use
of an explicit compositional approach while the attention mechanism integrated
in the neural architecture does not build contextualized senses with explicit
compositional semantic operations (Wang etal.,, 2017).

The proposed compositional method is not an inscrutable black box, but a
robust, transparent and simple linguistic mechanism. The method is based on a
compositional and incremental interpretation of syntactic dependencies, which
allows to compute semantic representations of complex expressions.

In the present article, an extensive and detailed linguistic description of
the method presented in previous works (Gamallo etal.,, 2021; Gamallo,, 2021)
is given. Taking into account the linguistic profile of the journal's
audience, we have sought to deepen the linguistic concepts of the proposed
method to the detriment of the more technical details of its implementation.
In addition, we have included a discussion of both the principle of
compositionality and the mechanism of attention to make it clearer the
conceptual difference between the proposed compositional method and
Transformers. Finally, we have grouped together in one section all the
experiments that had been carried out in previous works.

This article is organized as follows. In the next section
(LABEL:sec:relatedwork) we explore how both neural networks and symbolic
strategies deal with compositional meaning and word contextualization. Then,
in Section LABEL:sec:method, our dependency-based compositional model is
described. Section LABEL:sec:experiments reports a semantic experiment aimed
at comparing the contextualized vectors produced by our strategy with those
generated by Transformer-based large models. Finally, conclusions and future
work are addressed in Section LABEL:sec:conclusions.

##  2 Neural and Symbolic Based Strategies

sec:relatedwork

The meaning of a complex expression relies on the contextualized senses of its
constituent words. To build the meaning of complex expressions and the senses
of their constituent words as contextualized vectors, two methods can be
followed:

  * –

Methods (mostly neural-based) that combine word vectors without making use of
explicit compositional operations.

  * –

Symbolic-based methods that rely on explicit compositional rules
linguistically motivated by syntactic-semantic functions.

In both strategies, distributional representations encoded in continuous
vectors are a proxy for meaning.

###  2.1 Neural-Based Architectures

In the first type of methods, no explicit linguistic information or no
explicit notion of linguistic trees is required. The basic approach (not
strictly neural) is to combine embeddings of two co-occurring words with
arithmetic operations: addition or component-wise multiplication (Mitchell and
Lapata,, 2008, 2009, 2010). This approach is not compositional because the
syntactic functions that links words is not considered, and so two sentences
with the same constituents but with different functions are represented in the
same way (e.g. _``Federer beat Nadal''_ and _``Nadal beat Federer''_). Much
more sophisticated and complex approaches are found in neural networks, for
instance in the use of recurrent neural networks such as Long Short Term
Models (LSTM) to process sequentially the input sentences, or more recently
networks based exclusively on attention mechanisms (Vaswani etal.,, 2017),
called Transformers. Attention mechanisms replace sequential processing, which
is difficult to parallelize, with highly efficient and parallelizable
distributed processing, designed to account for long-distance word
relationships.

Figure 1: Simplified architecture of the self-attention mechanismfig:attention

FigureLABEL:fig:attention shows a simplified version of the self-attention
mechanism underlying the Transformer architecture, one of the most recent and
powerful neural-based models. On the left, a sentence is processed by
associating to each constituent word a static word embedding, which was pre-
trained from a corpus. Each word is assigned a static embedding out of
context. This non contextual embedding is added up to a positional embedding
that has been built by considering the position of each word in the sentence.
The resulting word embeddings (static + position) for all words of the
sentence represent the input of the attention mechanism, which returns a
contextualized vector for each word as a result.

On the right of FigureLABEL:fig:attention, we show how the contextualized
embedding is built in the attention module for a specific word (_love_). The
attention strategy is inspired from the information retrieval approach as it
evokes the key, value and query concepts. A vector conceived as the query is
compared to a list of other vectors (the keys), so as to find the best
candidates (values) for the initial query. This allows the module to focus on
a word and look at other words (keys) in the input sentence so as to grasp
their relevance with respect to the given word (query). The objective is to
calculate the degree of association (or attention) between each word and the
rest of the sentence by considering their position. In the attention
mechanism, this idea is implemented as follows. Let us start with the sentence
``Give me love not money'' and the word love: the initial static embedding of
love (along with its positional embedding) is projected into three different
vectors: query, key and value vectors. The same is done for the rest of words.
These three vectors are created by multiplying the initial word embedding by
three matrices with weights adjusted during the training phase. Then, the
query vector of love is compared against the key vectors associated to all the
words in the sentence by computing the dot product. The dot product of love
with a key vector gives rise to an attention score. This is done with all
words in the sentence resulting in an attention vector of 5 dimensions, one
for each word. The higher the attention score, the closer the linguistic
proximity in the sentence between the query and the key. In the specific toy
example of FigureLABEL:fig:attention, the highest scores (represented by a
lighter color) correspond to the key vectors of Give and money, given that
love is the direct object of the verb Give and it plays the same semantic role
as money in the sentence. Ideally, each word tends to "attend" to those with
which it would be linguistically linked by means of syntactic, semantic,
discursive, or pragmatic relationships.

The final step of the attention mechanism, represented in the upper right part
of the figure, consists of adding up all the value vectors of the 5 words by
taking into account the weight of the attention score of each word with
respect to love. It results in the contextualized vector of love. The same is
done for the rest of the words in the sentence. The whole process is called
self-attention. The prefix 'self' is used as the attention is directed to
words of the same language. In a translation context with an encoder-decoder
architecture, cross-attention is carried out.

Yet, there are more complex attention architectures. In a multi-head attention
architecture, self-attention is repeated and performed $N$ times in parallel
with different query/key/value vectors for each work of the input sentence. In
addition, the whole multi-head attention process is replicated for each
attention layer of the neural network, which are connected by feed forward
layers.

Considering that in a single self-attention layer an input word is assigned 6
different vectors or embeddings (static, positional+static, query, key, value
and contextualized), in a multi-head architecture of 16 heads with 24 layers
as in BERT-large (Devlin etal.,, 2019), the number of vectors required to
fully encode a word in a particular position of a sentence is 1,178: 2 input
vectors corresponding to the input embedding and its addition with the
positional one, 384 query vectors (16 per head and 24 per layer), 384 key
vectors, 384 value vectors, and 24 contextualized outputs (one per layer). So,
the total number of vectors required to encode the five words of the sentence
in FigureLABEL:fig:attention is 5,890. Each embedding is supposed to be a
linguistic representation of a word, but there are no clear linguistic or
neuro-linguistic clues to interpret what kind of information is being encoded
in each of these many word embeddings. The whole architecture is a black box
made of a continuous adjustment of weights and values (through million or even
billion parameters) which are calculated using the brute force of high-
performance computing through backpropagation. In linguistic terms, we know
that the attention mechanism constructs contextualized embeddings by summing
the value vectors of co-present words in a sequence, giving more weight to
some than to others. It is therefore interested in syntagmatic relations. In
this process of contextualization, words in exclusive paradigmatic opposition
do not intervene directly. So, the attention mechanism does not work directly
with lexical selection preferences.

Even though the architecture of the attention mechanism (and also that of
recurrent neural networks) seem not be motivated by explicit linguistic
knowledge, the neural networks are more successful than the symbolic models on
many Natural Language Processing (NLP) tasks, in part because of their ability
to process very large amounts of data.

###  2.2 Symbolic and Compositional Approaches

The second type of methods to process the meaning of words in context relies
on explicit compositional functions. This gives rise to symbolic architectures
that are compositional by design (Hupkes etal.,, 2020) and are often referred
to as Compositional Distributional Semantics.

The most popular approaches in Compositional Distributional Semantics design
compositional models on the basis of the combinatorial behavior inspired by
Categorial Grammar (Steedman,, 1996). In these approaches, functional words
(verbs, adjectives and adverbs) are represented as high-dimensional tensors
that are applied on word arguments, represented as simple vectors, to modify
and specify their meaning (Coecke etal.,, 2010; Baroni and Zamparelli,, 2010;
Grefenstette etal.,, 2011; Krishnamurthy and Mitchell,, 2013; Kartsaklis and
Sadrzadeh,, 2013; Baroni,, 2013; Baroni etal.,, 2014; Wijnholds etal.,, 2020).
Two problems arise with this type of models: one of efficiency and the other
rather conceptual. On the one hand, these models result in an information
scalability problem, since tensor representations grow exponentially
(Kartsaklis etal.,, 2014). On the other hand, it is not possible to assign a
contextualized meaning to all words, since some are functions that
contextualize others. This contradicts the co-compositional hypothesis
(Pustejovsky,, 1995), which states that two related words influence and
constrain each other in the meaning construction process and, thereby, they
can behave as both functions and arguments at the same time (Gamallo,, 2017,
2019).

Other approaches to Compositional Distributional Semantics, based on
Dependency Grammar (instead of Categorial Grammar) do not make use of n-order
tensors to represent functional words. The work by Erk and Padó, (2008)
proposes a strategy in which the static meanings of two related words are
combined by means of a syntactic dependency to give rise to two contextualized
senses, one per word. The main operation in this dependency-based model lies
in the selection preferences that each word imposes on the other. This
strategy follows the co-compositional hypothesis stated above. The problem
with this approach based on selection preferences is that it cannot be easily
adapted to the interpretation of sentences of any type and size. As we will
show in the next section, the main contribution of our work is, precisely, to
generalize the Erk and Padó's model in order to be able to encode larger
expressions such as sentences of any size. Similarly, in Type Composition
Logic (Asher etal.,, 2016) the construction of the compositional meaning
relies on two lexical functions that shift the meaning (and in some coercitive
cases, the semantic type too) of the two input words. However, this approach
only applies to two-word composition, and is not easily adapted to the
modeling of longer expression. In Weir etal., (2016), a dependency-based graph
captures the full sentential context of the word, but with very sparse word
representations.

The linguistic-based works we have introduced have something in common: they
make use of distributional applications, including word vectorization and
massive learning of lexical semantics from large corpora, to help validate
linguistic hypotheses and formal models of language.

With this same focus, Boleda etal., (2013) take as their starting point a
formal modeling of the semantics of adjectives and design experiments with
distributional models to check hypothesis about some semantic types. The
experiments let them conclude that there is no difference between non-
intensional subsective adjectives and intensional (non-subsective) ones. In a
more theoretical work, McNally, (2014) argues that distributional
representations have potential to serve as models for the semantics of very
complex phenomena, such as the theory of kinds to treat generic uses of
nominals. It is also worth mentioning the approach called Functional
Distributional Semantics (Emerson and Copestake,, 2016), which embeds
distributional information with model-theoretic semantics.

Our work, like all of the aforementioned, make use of empirical distributions
learnt from corpora to either validate or not a semantic hypothesis and
contribute to the design of computational architectures that better model the
semantic interpretation process in a more transparent way. By contrast, other
linguistic-based approaches thoroughly explore the results of the neural
models to find some glimmer of implicit linguistic knowledge in their black
boxes. This task is carried out by defining specific tests to analyze whether
neural-based language models can learn syntactic regularities and are provided
with compositional abilities (Linzen and Leonard,, 2018; Kim and Linzen,,
2020; De-Dios-Flores and Garcia,, 2022). Linzen, (2018) states that the role
of linguists would be to clearly delineate the linguistic capabilities that
can be expected of large language models, by constructing controlled
experimental tests that can determine whether those desiderata have been met.
This new paradigm of linguistic research is of great interest and can be seen
as very different but complementary to the one we follow.

In the present work, we will implement a distributional architecture to check
whether selectional preferences can dynamically build the semantic meaning of
composite expressions.

##  3 A Compositional Strategy Based on Syntactic Dependencies and Selection
Preferences

sec:method The objective of this section is to introduce a compositional
strategy based on the notion of selection preferences in a co-compositional
scenario.

###  3.1 Selectional Preferences and Paradigmatic Classes

Given two syntactically related words, each one restricts the meaning of the
other through the mutual imposition of selection preferences (Erk and Padó,,
2008; Gamallo,, 2017).

More precisely, the combination of two words, $w_{1}$ and $w_{2}$, related by
means of the syntactic dependency $R$, gives rise to two restricted and
contextualized word senses, $w_{1}^{\prime}$ and $w_{2}^{\prime}$, in relation
$R$, where $w_{1}^{\prime}$ is obtained by combining the meaning of $w_{1}$
with the selectional preferences imposed by $w_{2}$ in $R$, and
$w_{2}^{\prime}$ is the result of combining the meaning of $w_{2}$ with the
selectional preferences imposed by $w_{1}$ in $R$. Selectional preferences are
represented as the generic sense associated with the paradigmatic class of
most relevant words appearing in one of the two syntactic roles, either head
or dependent, of the dependency relation $R$.

Figure 2: Co-compositional construction of the contextualized senses of catch
and ball by using their selectional preferences.fig:preferences

FigureLABEL:fig:preferences provides an intuitive example of this semantic
procedure showing how the meaning of two polysemous words is restricted when
they are combined in a particular syntactic dependency. This is the case of
the verb catch, which can refer to either a grabbing action (represented by
means of two open hands in FigureLABEL:fig:preferences) or to the result of
contracting a disease (the drawing of the doll with a cold), combined with the
noun ball, referring to either a physical spherical object or a dancing event.
Given the combination of the verb and the noun in the direct object relation
(obj), two different compositional operations are carried out (the order is
not relevant). First, in the head position, noted
$<\boldsymbol{V}\,\,\,OBJ\\!\\!\\!\rightarrow ball>$, of the direct object
relation, the meaning of the verb catch is combined with the meaning of the
paradigmatic class of those most frequent verbs co-occurring with ball in that
syntactic position, namely {throw, grasp, send, toss,…}. The meaning of the
paradigmatic class, which can be seen as the selectional preferences imposed
by ball, restricts the meaning of the verb by selecting the grabbing sense.
Second, in the dependent position,
$<catch\,\,\,OBJ\\!\\!\\!\rightarrow\boldsymbol{N}>$, the meaning of the noun
ball is combined with the meaning of the paradigmatic class of those most
frequent nouns co-occurring with catch in that syntactic position, namely
{train, fish, cat, thief, …}. The meaning of the paradigmatic class, which
represents the selectional preferences imposed by catch, restricts the meaning
of the noun by selecting the sense referring to the physical spherical object.
In sum, the two words restrict, contextualize and disambiguate each other when
they are combined by means of a specific syntactic dependence.

The meaning of the complex expression would be the result of combining the two
contextualized senses (the picture below in the figure), but, given that
Dependency Grammar does not consider categories for complex expressions, we do
not address the combination between contextualized senses in the present work.
We just propose that the most representative meaning of the complex expression
corresponds to the contextualized sense of the root, which is, in the current
example, the sense of the verb head:
$<\boldsymbol{catch}\,\,\,OBJ\\!\\!\\!\rightarrow ball>$.

In the semantic space, out-of-context lexical words are represented as static
vectors, while selection preferences are dynamic vectors resulting from adding
the static vectors of the words belonging to a paradigmatic class in a
syntactic position. In the syntactic dependency
$<w1\,\,\,R\\!\\!\\!\rightarrow w2>$, the selection preferences imposed on
word $w2$ by $w1$ are computed by adding the vectors of those words belonging
to the paradigmatic class $\mathrm{C1}_{<w1\,\,\,R\\!\rightarrow\,C1>}$, such
that $\\{w\arrowvert w\in\mathrm{C1}_{<w1\,\,\,R\\!\rightarrow\,C1>}\\}$ is
the set of words that can replace $w_{2}$ in relation $R$ with $w_{1}$. The
contextualized sense of $w2$ is the result of combining (by component-wise
multiplication or addition) the vector of $w2$ with the vector constructed
with the words belonging to the class
$\mathrm{C1}_{<w1\,\,\,R\\!\rightarrow\,C1>}$. The contextualized sense of
$w1$, on the other hand, results from combining the vector of $w1$ with the
one built with the words belonging to the class
$\mathrm{C2}_{<C2\,\,\,R\\!\rightarrow\,w2>}$, such that $\\{w\arrowvert
w\in\mathrm{C2}_{<C2\,\,\,R\\!\rightarrow\,w1>}\\}$ is the set of words that
can replace $w_{1}$ in relation $R$ with $w_{2}$. More formally, let
$\vec{w1}$ and $\vec{w2}$ be the static vectors of the two words in dependency
$<w1\,\,\,R\\!\\!\\!\rightarrow w2>$; then, the two contextualized senses,
$\vec{w}_{<w1\,\,\,R\\!\rightarrow\,C1>}$ and
$\vec{w}_{<C2\,\,\,R\\!\rightarrow\,w2>}$ , are two dynamic vectors computed
as follows:

$\displaystyle\vec{w}_{<w1\,\,\,R\\!\rightarrow\,C1>}=\vec{w1}\odot\vec{C1}{eq:comp1}$
(1)
$\displaystyle\vec{w}_{<C2\,\,\,R\\!\rightarrow\,w2>}=\vec{w2}\odot\vec{C2}{eq:comp2}$
(2)

where $\vec{C1}$ and $\vec{C2}$ are the vectors representing the selection
preferences of the corresponding paradigmatic classes C1 and C2, and which are
the result of the following operations:

$\displaystyle\vec{C1}=\sum\limits_{w\in\mathrm{C1}_{<w1\,\,\,R\\!\rightarrow\,C1>}}\vec{w}{eq:head}$
(3)
$\displaystyle\vec{C2}=\sum\limits_{w\in\mathrm{C2}_{<C2\,\,\,R\\!\rightarrow\,w2>}}\vec{w}{eq:dep}$
(4)

It means that selection preferences are constructed by adding the word vectors
belonging to a paradigmatic class. It is not necessary to consider all the
words of the paradigm to get acceptable results, but just the most relevant.

Paradigmatic classes are constructed by massive search on large parsed corpora
and static vectors are initialized as word embeddings pre-trained in tagged
corpora. These embeddings are built in separated semantic spaces representing
lexical categories, i.e., nouns, verbs, adjectives, and adverbs are separated
in different vector spaces. We follow one of the fundamental principles of
Cognitive Grammar, which states that syntactic categories construct different
conceptualizations (Langacker,, 1987, 1991) and so they project different
semantic representations, even if they can evoke the same entities. For
instance, the meaning of the word catch would be found in different semantic
spaces depending on whether it is a verb or a noun. In our model, the
embeddings of catch as a verb and as a noun are in different vector spaces and
therefore cannot be put in the same paradigmatic class. We conceive
paradigmatic classes as semantic categories with the same conceptualization
and thereby constituted by words sharing the same part-of-speech. The use of
paradigmatic classes allows us to align vector spaces, i.e., to make them
compatible. In the expression catch the ball, the vector of ball is not
directly combined with the vector of the verb catch because the two vectors
are made up of incompatible syntactic contexts. Nouns appear in syntactic
positions different from those of verbs. However, thanks to the use of
paradigmatic classes, we can combine ball with the vectors of all nouns (or
the most frequent/relevant ones) appearing as direct object of catch, and
vice-versa, the vector of catch is combined with the vectors of all verbs
having ball as direct object. This is a fundamental difference with regard to
the attention mechanism in Transformers, where attention represents
syntagmatic relationships and all word embeddings are in the same vector
space.

###  3.2 Selectional Preferences in a Dependency Tree

Figure 3: Simplified architecture of the dependency-based compositional
approach to build contextualized word vectors.fig:architecture

In order to build the contextualized senses of words in an incremental way, it
is necessary to allow selectional preferences to be defined in larger
dependency trees representing the syntactic analysis of sentences.

FigureLABEL:fig:architecture shows the architecture of the dependency-based
compositional approach. The input sentence is analyzed in dependencies and
lexical words are initialized with static vectors defined in semantic spaces
according with their lexical category. In the internal layer, selection
preferences are constructed and combined with the static vectors giving rise
to several contextualizations according to the number of dependencies the
words are involved. The intermediate vectors associated with girl and catch in
the figure illustrate these internal operations. Finally, the resulting
contextualized vectors associated with the lexical words are the output of the
compositional mechanism. Given that the verb catch is the root of the input
expression, it can be used to represent the meaning of the sentence.

##  4 Experiments

sec:experiments

On the basis of some of the main ideas depicted in the previous section, we
implemented a compositional dependency-based system, _DepFunc_ ,111Freely
available at https://github.com/gamallo/DepFunc which has been compared to
several BERT-like architectures for the task of sentence similarity in four
languages: English, Portuguese, Spanish and Galician. Contextualized
embeddings derived from autoencoder language models, such as BERT, outperform
in many tasks contextualized embeddings derived from generative autoregressive
models (e.g., GPT architecture) (Lenci etal.,, 2022). The experiments
performed and the results obtained have already been described in detail and
published recently (Gamallo etal.,, 2021; Gamallo,, 2021; Gamallo and
MarcosGarcia,, 2022). For more details (configuration of the systems,
elaboration of the datasets in each language, results of all the
configurations of the evaluated systems, etc.), please refer to the cited
articles.

_Models_ |  $\rho$(en) |  $\rho$(pt) |  $\rho$(es) |  $\rho$(gl) | av.
---|---|---|---|---|---
Baseline (static) | 29 | 28 | 29 | 28 | 29
DepFunc (compositional) | 53 | 55 | 45 | 47 | 50
BERT (attention) | 45 | 43 | 42 | 57 | 47
Table 1: Spearman correlation between the compared models (DepFunc and the
best configurations of BERT) and human judgments on subject-verb-object
sentence pairs. The table also shows a baseline based on just comparing static
vectors (first row).tab:results

TableLABEL:tab:results compares the results obtained by our compositional
strategy (DepFunc) with those returned by the best configurations of the BERT
model for each language.

Concerning the results shown in TableLABEL:tab:results, there are no major
differences between the four languages, which range from 42 to 57 Spearman
correlations. Nor is there much difference between the two types of models
used (compositional and attention), since the compositional one achieves a
macro-average score of 50 and the attention mechanism of 47 (last column in
TableLABEL:tab:results). This proves that the compositional model performs
very competitively with respect to the Transformer-based strategy, by using a
more transparent linguistic architecture with much less parameters and trained
with a smaller corpus. Even though BERT and DepFunc returned similar scores,
we applied the Student's test to compare their score distributions, and the
results indicated that the difference between both systems is statistically
significant, with p-value $\leq 0.005$.

The English dataset was used for the first time in Grefenstette and Sadrzadeh,
(2011). The datasets for the other three languages were built on the basis on
the English one and thereby follow its structure. All are freely available and
can be found in the DepFunc software. Each test dataset consists of
$\approx$200 pairs of subject-verb-object triples as those in
TableLABEL:tab:dataset, along with a similarity score, in the third column,
assigned by a human evaluator (or rather by the average of several ones). The
scores range from 1 to 7 depending on the degree of similarity and semantic
acceptability. Each pair is also assigned a Cosine similarity on the basis of
the system prediction: DepFunc in column 4 and BERT in column 5, after having
normalizing the scores for the range 1-7. The evaluation procedure consists of
comparing the Spearman correlation between the Cosine similarity (system
prediction) and the similarity scores given by human evaluators. Since
Transformer-based models take as input sentences with determiners and
inflected words, the triples of the dataset were expanded into grammatical
sentences.

expr1 (in lemmas) | expr2 (in lemmas) |  human score | DepFunc | BERT
---|---|---|---|---
employ buy property | employ purchase property |  7.0 | 7.0 | 6.3
firm buy politician | firm purchase politician |  4.3 | 6.3 | 6.8
system provide facility | system supply facility |  6.5 | 4.3 | 7.0
mother provide baby | mother supply baby |  5.1 | 3.6 | 5.5
report draw attention | report attract attention |  6.5 | 4.6 | 6.9
child draw picture | child attract picture |  1.5 | 3.7 | 6.0
boy meet girl | boy visit girl |  4.5 | 3.9 | 6.8
system meet requirement | system visit requirement |  1.5 | 2.5 | 6.5
people run round | people move round |  3.3 | 3.9 | 6.8
machine run application | machine move application |  1.6 | 0.0 | 6.7
Table 2: Sample of the English dataset. Each pair is assigned a human score,
along with the similarity scores provided by both DepFunc and BERT.tab:dataset

In the small sample of TableLABEL:tab:dataset, it can be observed that the
scores of BERT tend to have a reduced range to high values: between 6 and 7.
On the contrary, the values of DepFunc range between 0 and 7, a range that is
more similar to that used by human annotators. However, the score 0 returned
by DepFunc reveals a problem of the model. This value appears when no member
of the paradigmatic classes on the basis of which the selection preferences
are constructed could be found in the parsed corpus.

##  5 Conclusions

sec:conclusions We have highlighted in this work the lack of transparency and
explainability of artificial neural architectures, specifically, those based
on the attention mechanism. They are black boxes that manage to emerge
generalizations and apparently compositional behaviors, without explicitly
encoding any knowledge about the principle of compositionality.

In contrast to these large models, we have described an alternative semantic
strategy based on syntactic dependencies and how these are combined to
construct contextual meaning. In this paper, we have focused on how contextual
vectors representing the meaning of words in an input sentence are elaborated.
According to our proposal, the interpretation of a sentence is the process of
building the sense of each constituent word in a recursive and incremental
way, where the contextualized sense of the root word is the most
representative of the whole sentence. The proposed strategy is compositional,
linguistically interpretable and does not require large computational
architectures. It is based on the vectorization of paradigmatic classes, in
contrast to the vectorization of words connected in a sequence by syntagmatic
relations, as the attention mechanism does. In the evaluated task, the results
of our strategy are competitive with those obtained with neural architectures
based on the attention mechanism, even if our models were trained with smaller
corpora.

In future work, we will do a full implementation of the dependency-based
compositional method so as to process sentences with open syntax. As one of
the main limitations of the current DepFunc version is that it cannot be
applied to syntactically unrestricted expressions, we are working to improve
the method so that it can be more general and no dependent of specific
syntactic constructions.

On the other hand, special attention will be paid to the compositional meaning
of referential and grammatical expressions, namely determiners, auxiliary
verbs or deictic adverbs, whose combination with other lexical words is not
driven by selectional preferences but by mechanisms involving grounding
strategies to the discourse situation (Gupta etal.,, 2015). It should be noted
that our approach (as well as the attention mechanism) only addresses the
local lexical meaning, leaving out the global one referring to knowledge about
situations, circumstances and state of affairs (Erk and Herbelot,, 2021). So,
a crucial challenge we will address is to find a way to represent the global
meaning so that it interacts with the local/lexical one.

We will also address challenges related to the degree of compositionality,
which includes the treatment of expressions such as idioms, compounds, or
collocations. It is likely that we need other non-compositional or weakly
compositional semantic mechanisms, other than that based on selection
preferences, to account for this type of expressions.

Another possible improvement to be explored is to propose models that take
into account both types of relations, syntagmatic and paradigmatic, in order
to overcome the syntactic limitations of both the attention mechanism and our
approach based on selectional preferences.

Finally, we will also explore the possibility of elaborating neuro-symbolic
strategies by integrating explicit linguistic information into the attention
mechanism of Transformers. This symbolic information could distill the neural
architecture of large language models, by guiding the readjustment of weights
without resorting to the brute force of an uninterpretable black box, as most
of these giant Transformers operate today.

DepFunc software and the evaluation datasets used in the multilingual
experiments are freely available.222https://github.com/gamallo/DepFunc

This research was funded by: project "Nós - Galician in the society and
economy of artificial intelligence", agreement between Xunta de Galicia and
University of Santiago de Compostela; Horizon Europe, Marie Skłodowska-Curie
Actions (MSCA), Doctoral Networks; European Union; LingUMT, grant
PID2021-128811OA-I00, MEC; DeepR, grant TED2021-130295B-C31, MEC; Big-eRisk,
grant PLEC2021-007662, MEC; and grant ED431G2019/04 by the Galician Ministry
of Education, University and Professional Training, and the European Regional
Development Fund (ERDF/FEDER program), and Groups of Reference: ED431C
2020/21.

## Bibliography

  * Asher etal., (2016) Asher, N., Vande Cruys, T., Bride, A., and Abrusán, M. (2016).  Integrating type theory and distributional semantics: A case study on adjective–noun compositions.  Computational Linguistics, 42(4):703–725.
  * Baroni, (2013) Baroni, M. (2013).  Composition in distributional semantics.  Language and Linguistics Compass, 7:511–522.
  * Baroni, (2020) Baroni, M. (2020).  Linguistic generalization and compositionality in modern artificial neural networks.  Philosophical Transactions of the Royal Society B, 375:1–7.
  * Baroni etal., (2014) Baroni, M., Bernardi, R., and Zamparelli, R. (2014).  Frege in space: A program for compositional distributional semantics.  Linguistic Issues in Language Technology (LiLT), 9:241–346.
  * Baroni and Zamparelli, (2010) Baroni, M. and Zamparelli, R. (2010).  Nouns are vectors, adjectives are matrices: Representing adjective-noun constructions in semantic space.  In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, EMNLP'10, pages 1183–1193, Stroudsburg, PA, USA.
  * Bender etal., (2021) Bender, E.M., Gebru, T., McMillan-Major, A., and Shmitchell, S. (2021).  On the dangers of stochastic parrots: Can language models be too big?  In FAccT, pages 610–623.
  * Boleda etal., (2013) Boleda, G., Baroni, M., Pham, T.N., and McNally, L. (2013).  Intensionality was only alleged: On adjective-noun composition in distributional semantics.  In Proceedings of the 10th International Conference on Computational Semantics (IWCS 2013) – Long Papers, pages 35–46, Potsdam, Germany. Association for Computational Linguistics.
  * Coecke etal., (2010) Coecke, B., Sadrzadeh, M., and Clark, S. (2010).  Mathematical foundations for a compositional distributional model of meaning.  Linguistic Analysis, 36(1–4):345–384.
  * Dankers etal., (2022) Dankers, V., Bruni, E., and Hupkes, D. (2022).  The paradox of the compositionality of natural language: A neural machine translation case study.  In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4154–4175, Dublin, Ireland. Association for Computational Linguistics.
  * De-Dios-Flores and Garcia, (2022) De-Dios-Flores, I. and Garcia, M. (2022).  A computational psycholinguistic evaluation of the syntactic abilities of galician bert models at the interface of dependency resolution and training time.  Procesamiento del Lenguaje Natural, 69:15–26.
  * Devlin etal., (2019) Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019).  BERT: Pre-training of deep bidirectional transformers for language understanding.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1, pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Ebrahimi etal., (2018) Ebrahimi, J., Lowd, D., and Dou, D. (2018).  On adversarial examples for character-level neural machine translation.  In COLING, pages 653–663, Santa Fe, New Mexico, USA.
  * Emerson and Copestake, (2016) Emerson, G. and Copestake, A. (2016).  Functional distributional semantics.  In Proceedings of the 1st Workshop on Representation Learning for NLP, pages 40–52, Berlin, Germany. Association for Computational Linguistics.
  * Erk and Herbelot, (2021) Erk, K. and Herbelot, A. (2021).  How to marry a star: Probabilistic constraints for meaning in context.  In Proceedings of the Society for Computation in Linguistics 2021, pages 451–453, Online. Association for Computational Linguistics.
  * Erk and Padó, (2008) Erk, K. and Padó, S. (2008).  A structured vector space model for word meaning in context.  In 2008 Conference on Empirical Methods in Natural Language Processing (EMNLP-2008), pages 897–906, Honolulu, HI.
  * Ettinger, (2020) Ettinger, A. (2020).  What bert is not: Lessons from a new suiteof psycholinguistic diagnostics for language models.  TACL, 8:34–48.
  * Gamallo, (2017) Gamallo, P. (2017).  The role of syntactic dependencies in compositional distributional semantics.  Corpus Linguistics and Linguistic Theory, 13(2):261–289.
  * Gamallo, (2019) Gamallo, P. (2019).  A dependency-based approach to word contextualization using compositional distributional semantics.  Language Modelling, 7(1):53–92.
  * Gamallo, (2021) Gamallo, P. (2021).  Compositional distributional semantics with syntactic dependencies and selectional preferences.  Applied Sciences, 11(12).
  * Gamallo etal., (2021) Gamallo, P., Corral, M.P., and Garcia, M. (2021).  Comparing dependency-based compositional models with contextualized word embedding.  In 13th International Conference on Agents and Artificial Intelligence (ICAART-2021). SCITEPRESS – Science and Technology Publications, Lda.
  * Gamallo and MarcosGarcia, (2022) Gamallo, P. and MarcosGarcia, I.-d.-D.-F. (2022).  Evaluating contextualized vectors from both large language models and compositional strategies.  Procesamiento del Lenguaje Natural, 69:153–164.
  * Grefenstette and Sadrzadeh, (2011) Grefenstette, E. and Sadrzadeh, M. (2011).  Experimental support for a categorical compositional distributional model of meaning.  In Conference on Empirical Methods in Natural Language Processing (EMNLP 2011), pages 1394–1404.
  * Grefenstette etal., (2011) Grefenstette, E., Sadrzadeh, M., Clark, S., Coecke, B., and Pulman, S. (2011).  Concrete sentence spaces for compositional distributional models of meaning.  In Proceedings of the Ninth International Conference on Computational Semantics, IWCS '11, pages 125–134.
  * Gupta etal., (2015) Gupta, A., Boleda, G., Baroni, M., and Padó, S. (2015).  Distributional vectors encode referential attributes.  In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 12–21, Lisbon, Portugal. Association for Computational Linguistics.
  * Hupkes etal., (2020) Hupkes, D., Dankers, V., Mul, M., and Bruni, E. (2020).  Compositionality decomposed: how do neural networks generalise?  Journal of Artificial Intelligence Research, 67:757–795.
  * Kartsaklis etal., (2014) Kartsaklis, D., Kalchbrenner, N., and Sadrzadeh, M. (2014).  Resolving lexical ambiguity in tensor regression models of meaning.  In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Vol. 2: Short Papers), pages 212–217, Baltimore, USA. Association for Computational Linguistics.
  * Kartsaklis and Sadrzadeh, (2013) Kartsaklis, D. and Sadrzadeh, M. (2013).  Prior disambiguation of word tensors for constructing sentence vectors.  In Conference on Empirical Methods in Natural Language Processing (EMNLP 2013), pages 1590–1601.
  * Kim and Linzen, (2020) Kim, N. and Linzen, T. (2020).  COGS: A compositional generalization challenge based on semantic interpretation.  In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9087–9105. Association for Computational Linguistics.
  * Krishnamurthy and Mitchell, (2013) Krishnamurthy, J. and Mitchell, T. (2013).  Vector space semantic parsing: A framework for compositional vector space models.  In Proceedings of the Workshop on Continuous Vector Space Models and their Compositionality, pages 1–10. Association for Computational Linguistics.
  * Lake etal., (2017) Lake, B.M., Ullman, T.D., Tenenbaum, J.B., and Gershman, S.J. (2017).  Building machines that learn and think like people.  Behavioral and Brain Sciences, 40:1–72.
  * Langacker, (1987) Langacker, R.W. (1987).  Foundations of Cognitive Grammar: Theoretical Prerequisites, volume1.  Stanford University Press, Stanford.
  * Langacker, (1991) Langacker, R.W. (1991).  Foundations of Cognitive Grammar: Descriptive Applications, volume2.  Stanford University Press, Stanford.
  * Lenci etal., (2022) Lenci, A., Sahlgren, M., Jeuniaux, P., Gyllensten, A., and Miliani, M. (2022).  A comparative evaluation and analysis of three generations of distributional semantic models.  Language Resources and Evaluation, 56:1269–1313.
  * Linzen, (2018) Linzen, T. (2018).  What can linguistics and deep learning contribute to each other?  CoRR, abs/1809.04179.
  * Linzen and Leonard, (2018) Linzen, T. and Leonard, B. (2018).  Distinct patterns of syntactic agreement errors in recurrent networks and humans.  In Proceedings of the 40th Annual Conference of the Cognitive Science Society.
  * Marcus, (2003) Marcus, G. (2003).  The algebraic mind: Integrating connectionism and cognitive science.  MIT press.
  * Marcus, (2018) Marcus, G. (2018).  Deep learning: A critical appraisal.  CoRR, abs/1801.00631:1–27.
  * Marcus and Davis, (2019) Marcus, G. and Davis, E. (2019).  Rebooting AI : building artificial intelligence we can trust.  New York : Pantheon Books.
  * McNally, (2014) McNally, L. (2014).  Kinds, descriptions of kinds, concepts, and distributions.  In Balogh, K. and Petersen, W., editors, Bridging formal and conceptual semantics, pages 39–61.
  * Mitchell and Lapata, (2008) Mitchell, J. and Lapata, M. (2008).  Vector-based models of semantic composition.  In Proceedings of the Association for Computational Linguistics: Human Language Technologies (ACL-08: HLT), pages 236–244, Columbus, Ohio.
  * Mitchell and Lapata, (2009) Mitchell, J. and Lapata, M. (2009).  Language models based on semantic composition.  In Proceedings of Empirical Methods in Natural Language Processing (EMNLP-2009), pages 430–439.
  * Mitchell and Lapata, (2010) Mitchell, J. and Lapata, M. (2010).  Composition in distributional models of semantics.  Cognitive Science, 34(8):1388–1439.
  * Montague, (1970) Montague, R. (1970).  Universal grammar.  Theoria, 36(3):373–398.
  * Nefdt, (2020) Nefdt, R.M. (2020).  A puzzle concerning compositionality in machines.  Minds and Machines, 30(1):47–75.
  * Pandia and Ettinger, (2021) Pandia, L. and Ettinger, A. (2021).  Sorting through the noise: Testing robustness of information processing in pre-trained language models.  In Moens, M., Huang, X., Specia, L., and Yih, S.W., editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pages 1583–1596. Association for Computational Linguistics.
  * Partee, (2004) Partee, B.H. (2004).  Compositionality in Formal Semantics.  Oxford: Wiley-Blackwell.
  * Pustejovsky, (1995) Pustejovsky, J. (1995).  The Generative Lexicon.  MIT Press, Cambridge.
  * Steedman, (1996) Steedman, M. (1996).  Surface Structure and Interpretation.  The MIT Press.
  * Vaswani etal., (2017) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L.u., and Polosukhin, I. (2017).  Attention is all you need.  In Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R., editors, Advances in Neural Information Processing Systems, volume30, pages 5998–6008. Curran Associates, Inc.
  * Wang etal., (2017) Wang, R., Liu, W., and McDonald, C. (2017).  A matrix-vector recurrent unit model for capturing compositional semantics in phrase embeddings.  In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, CIKM '17, page 1499–1507, New York, NY, USA. Association for Computing Machinery.
  * Warstadt and Bowman, (2020) Warstadt, A. and Bowman, S.R. (2020).  Can neural networks acquire a structural bias from raw data?  In Proceedings of the Annual Meeting of the Cognitive Science Society - CogSci 2020.
  * Weir etal., (2016) Weir, D.J., Weeds, J., Reffin, J., and Kober, T. (2016).  Aligning packed dependency trees: A theory of composition for distributional semantics.  Computational Linguistics, 42(4):727–761.
  * Wijnholds etal., (2020) Wijnholds, G., Sadrzadeh, M., and Clark, S. (2020).  Representation learning for type-driven composition.  In Proceedings of the 24th Conference on Computational Natural Language Learning, pages 313–324, Online. Association for Computational Linguistics.

lastpage