# A Generative Model for Relation Extraction and Classification

Jian Ni, Gaetano Rossiello, Alfio Gliozzo, Radu Florian
IBM Research AI
1101 Kitchawan Road, Yorktown Heights, NY 10598, USA

###### Abstract

Relation extraction (RE) is an important information extraction task which
provides essential information to many NLP applications such as knowledge base
population and question answering. In this paper, we present a novel
generative model for relation extraction and classification (which we call
GREC), where RE is modeled as a sequence-to-sequence generation task. We
explore various encoding representations for the source and target sequences,
and design effective schemes that enable GREC to achieve state-of-the-art
performance on three benchmark RE datasets. In addition, we introduce negative
sampling and decoding scaling techniques which provide a flexible tool to tune
the precision and recall performance of the model. Our approach can be
extended to extract all relation triples from a sentence in one pass. Although
the one-pass approach incurs certain performance loss, it is much more
computationally efficient.

##  1 Introduction

Relation extraction (RE) is a fundamental information extraction task that
seeks to detect and characterize semantic relationships between pairs of
entities or events from natural language text. It provides important
information for many NLP applications such as knowledge base population Ji and
Grishman (2011) and question answering Xu et al. (2016).

Relation extraction has been studied in two settings. In the first setting,
gold entities are provided, and the RE task is to classify the relationships
between given pairs of entities in sentences. This task is also known as
_relation classification_ Hendrickx et al. (2010); Zhang et al. (2017b).

In the second setting, no gold entities are provided, and one needs to
consider both entity recognition and relation extraction Doddington et al.
(2004). This can be tackled via a _pipeline_ approach: first an entity
recognition model is applied to extract entities Florian et al. (2003); Lample
et al. (2016), and then a relation extraction model is applied to classify the
relationships between all pairs of predicted entities Kambhatla (2004); Chan
and Roth (2011); Zhong and Chen (2021). Alternatively, this can be addressed
by a _joint_ approach where entity recognition and relation extraction are
modeled and solved jointly Li and Ji (2014); Miwa and Bansal (2016); Luan et
al. (2019); Lin et al. (2020); Wang and Lu (2020).

In this paper, we focus on RE with entities provided, which covers both the
relation classification setting where gold entities are provided and the
pipeline relation extraction setting where predicted entities are extracted
via an entity recognition model. Given a sentence with a pair of gold or
predicted entities, RE is naturally formulated as a classification task. It is
a very challenging task since RE relies heavily on both syntactic and semantic
information, with possibly multiple entities and relations existing in one
sentence.

Figure 1: Overview of the Generative Relation Extraction and Classification
(GREC) model.

We present a novel generative model for relation extraction and classification
(named GREC), which treats RE as a sequence-to-sequence (seq2seq) text
generation task. Given an input sentence and a pair of entities in it, the
model generates an output _relation triple_ which consists of the two entities
and a relation type that specifies their relationship. Compared with
classification based RE approaches, the generative approach has the capability
of encoding entity information in the target sequence. Experiment results show
that GREC achieves state-of-the art performance on three benchmark RE
datasets. Moreover, our approach enables the adoption of standard fine-tuning
procedures with pre-trained seq2seq language models Wolf et al. (2020),
without the need of designing ad-hoc architectures, hence facilitating the
deployment in information extraction systems.

While the idea of using seq2seq models for RE was studied before Zeng et al.
(2018, 2020); Nayak and Ng (2020); Zhang et al. (2020), the previous works
focused on end-to-end relation extraction that jointly extracts entities and
relations from sentences. In this paper we focus on relation extraction and
classification with entities provided, and we show that it is beneficial to
encode entity information both in the source and in the target sequences to
achieve the best performance.

We summarize our main contributions as follows:

  * •

We explore various encoding representations for the source and target
sequences, and design effective schemes that enable the GREC model to achieve
state-of-the-art performance on three popular benchmark RE datasets: ACE05
Walker et al. (2006), SemEval 2010 Task 8 Hendrickx et al. (2010) and TACRED
Zhang et al. (2017b).

  * •

We introduce negative sampling during training to improve the recall
performance of the model. We also develop a novel decoding scaling scheme
during inference to improve the precision performance. These together provide
a flexible tool to tune both the precision and the recall performance of the
model.

  * •

We extend the approach to extract all relation triples from a sentence in one
pass. Although the one-pass approach incurs certain performance loss, it
significantly reduces the training and decoding time, as we show in Section
3.5.

##  2 Method

For given pairs or all pairs of entities in a sentence, the RE task is to
detect and characterize the relationships between those pairs of entities
Doddington et al. (2004); Hendrickx et al. (2010); Zhang et al. (2017b). We
present a generative approach for RE, which treats RE as a seq2seq text
generation task. Given an input sentence111Our approach can be extended to
cross-sentential relation extraction if we let the input sequence covers
multiple sentences. with gold or predicted entities, we propose two approaches
of encoding the entity information in the source sequence. With the _entity-
pair_ approach, each time we encode one pair of entities in the source
sequence; while with the _one-pass_ approach, we encoded all the entities in
the source sequence.

In Figure 1 we show the overview of our Generative Relation Extraction and
Classification (GREC) model. First, an input sentence and either i) a pair of
entities (under the entity-pair approach) or ii) all the entities (under the
one-pass approach) in the sentence are encoded to a source sequence
($src\\_seq$) via a source encoding module. Then, the source sequence is
passed to a seq2seq model which generates a target sequence ($tgt\\_seq$) that
includes the textual representation of the relation triple(s) supported by the
input sentence. Finally, the generated target sequence is processed by a
target parsing module to extract either i) a relation triple that consists of
the subject entity, the object entity, and the relation type between the two
entities (under the entity-pair approach) or ii) all the relation triples in
the sentence (under the one-pass approach).

###  2.1 Entity-Pair Approach

Under the entity-pair approach, each time we encode the entity information of
one pair of entities of an input sentence in the source sequence.

Let $\mathbf{x}=(x_{1},x_{2},...,x_{n})$ be an input sentence with $n$ tokens.
Let $\mathbf{s}=(x_{s_{1}},...,x_{s_{2}})$ and
$\mathbf{o}=(x_{o_{1}},...,x_{o_{2}})$ be a pair of entities in the input
sentence $\mathbf{x}$, where $\mathbf{s}$ is the subject (head) entity and
$\mathbf{o}$ is the object (tail) entity, with entity types $T_{s}$ and
$T_{o}$, respectively.

Let $R=\\{r_{1},r_{2},...,r_{K}\\}$ be the set of pre-defined relation types.
We use a null relation type $r_{0}$ (e.g., $r_{0}$=“None”) to indicate that
the two entities under consideration do not have a relationship belonging to
one of the $K$ relation types.

We have explored various schemes to encode the entity information in the
source sequence. The first scheme is to use some special tokens to mark the
start and end of the entities to encode entity location information as in Wu
and He (2019); Baldini Soares et al. (2019):

$\displaystyle f_{1}(\mathbf{x},\mathbf{s},\mathbf{o})$ $\displaystyle=$
$\displaystyle\big{(}x_{1},...,\$,x_{s_{1}},...,x_{s_{2}},\$,...,$ (1)
$\displaystyle\&,x_{o_{1}},...,x_{o_{2}},\&,...,x_{n}\big{)}$

The second scheme is to use the entity type of an entity to mark the start and
end of that entity, in order to encode both the entity location and entity
type information in the source sequence as in Ni et al. (2020); Zhong and Chen
(2021):

$\displaystyle f_{1}(\mathbf{x},\mathbf{s},\mathbf{o})$ $\displaystyle=$
$\displaystyle\big{(}x_{1},...,T_{s},x_{s_{1}},...,x_{s_{2}},T_{s},...,$ (2)
$\displaystyle T_{o},x_{o_{1}},...,x_{o_{2}},T_{o},...,x_{n}\big{)}$

In Equations (1) and (2) we assume that $\mathbf{s}$ appears before
$\mathbf{o}$ (i.e., $1\leq s_{1}\leq s_{2}<o_{1}\leq o_{2}\leq n$). If
$\mathbf{s}$ appears after $\mathbf{o}$, the positions of the two entities
will be switched.

Next, to encode the _direction_ of a relation (i.e., to encode which of the
two entities is the subject entity and which is the object entity), we create
the following sub-sequence:

$\displaystyle f_{2}(\mathbf{s},\mathbf{o})$ $\displaystyle=$
$\displaystyle[\mathbf{s}\;\\#\;T_{s}\;,\;\mathbf{o}\;\\#\;T_{o}]$ (3)

so that the subject entity always appears before the object entity in this
sub-sequence.

In addition to sub-sequences $f_{1}$ and $f_{2}$, we find that adding the list
of relation types is helpful to the GREC model:

$f_{3}(R)=[r_{1}-r_{2}-\cdot\cdot\cdot-r_{K}]$ (4)

The final encoding of the source sequence (i.e., the input to the seq2seq
model) is the concatenation of the three sub-sequences:

$\displaystyle src\\_seq=f_{1}(\mathbf{x},\mathbf{s},\mathbf{o})\oplus
f_{2}(\mathbf{s},\mathbf{o})\oplus f_{3}(R)$ (5)

For the target sequence, we have also explored various choices. First we find
that adding the subject and object entities to the target sequence (i.e.,
generating a relation triple) is better than generating the relation type
only. Among the different orders of the relation triple that we have tried, we
find that generating the relation triple with the order “subject, relation,
object” is the most effective:

$\displaystyle tgt\\_seq$ $\displaystyle=$
$\displaystyle[\mathbf{s}\;|\;r(\mathbf{s},\mathbf{o})\;|\;\mathbf{o}]$ (6)

where $r(\mathbf{s},\mathbf{o})$ is the relation type that specifies the
relationship between the subject entity $\mathbf{s}$ and the object entity
$\mathbf{o}$.

The seq2seq model is trained to generate a target sequence that contains the
special characters ‘[’, ‘]’, and ‘|’, which are used to parse the target
sequence to a relation triple. If $r(\mathbf{s},\mathbf{o})=r_{0}$ (null
relation type), the triple is a _negative_ example; otherwise, the triple is a
_positive_ example.

Examples of the source and target sequence encoding are shown in Table 8
(Appendix A).

####  2.1.1 Improving Recall via Sampling Negative Training Examples

It can be very challenging to achieve a sufficient _recall_ for RE models
Zhang et al. (2017b). Let $R_{gold}$ be the set of gold positive relation
triples in an RE dataset. Let $R_{pred}$ be the set of predicted positive
relation triples of an RE model when applied on the dataset. The _precision_
and _recall_ of the RE model on the dataset are defined as:

$\displaystyle precision$ $\displaystyle=$ $\displaystyle\frac{|R_{pred}\cap
R_{gold}|}{|R_{pred}|}$ $\displaystyle recall$ $\displaystyle=$
$\displaystyle\frac{|R_{pred}\cap R_{gold}|}{|R_{gold}|}$

where $|A|$ is the size (cardinality) of set $A$.

To improve the recall, one can try to let the RE model predict more positive
relation triples to increase the number of true positives (a _true positive_
is a predicted positive relation triple that matches a gold positive relation
triple, i.e., a triple in $|R_{pred}\cap R_{gold}|$).

We find that sampling negative training examples during training is very
effective for improving the recall of the GREC model. Specifically, we keep
all the positive training examples while randomly sampling a fraction $\alpha$
of the total negative training examples for training the GREC model. $\alpha$
is called the _negative sampling ratio_ which is a number between 0 and 1. As
we decrease $\alpha$, the model is trained with fewer negative examples and
higher positive-to-negative ratio, and it will generate more positive relation
triples during inference, hence improving the recall. We observe that sampling
negative training examples, however, might reduce the precision. In the next
subsection we present a scheme to improve the precision.

####  2.1.2 Improving Precision via Decoding Scaling

When the GREC model generates more positive relation triples and gets more
true positives, the recall can be improved. However, this may also increase
the number of false positives and reduce the precision. We propose a novel
_decoding scaling_ scheme that utilizes the sequence scores of the top $N$
generated target sequences to improve the precision.

For an input source sequence $\mathbf{z}$, we let the GREC model generate top
$N$ target sequences (relation triples) $\mathbf{y}_{1},...,\mathbf{y}_{N}$
with the highest sequence scores, where the _sequence score_ of
$\mathbf{y}_{i}$ is the conditional probability of $\mathbf{y}_{i}$ given
$\mathbf{z}$: $P(\mathbf{y}_{i}|\mathbf{z})$. Note that in normal decoding, we
just let the GREC model generate the best target sequence $\mathbf{y}_{1}$ and
use that as the prediction.

If a relation triple includes a non-null relation type in $R$, we call it a
positive triple; otherwise we call it a negative triple. There are two cases
to consider:

  * (1)

If the top $N$ triples are all positive or all negative, the scheme simply
selects the best positive or negative triple with the highest sequence score.
This is the same as in normal decoding.

  * (2)

If the top $N$ triples include both positive and negative triples, let
$\mathbf{y}_{+}^{*}$ and $\mathbf{y}_{-}^{*}$ be the best positive and
negative triple, respectively. We select the triple $\mathbf{y}^{*}$ as the
prediction as follows:

$\mathbf{y}^{*}=\begin{dcases}\mathbf{y}_{+}^{*},&\text{if
}\frac{P(\mathbf{y}_{+}^{*}|\mathbf{z})}{P(\mathbf{y}_{-}^{*}|\mathbf{z})}\geq\beta\\\
\mathbf{y}_{-}^{*},&\text{otherwise}\end{dcases}$ (7)

where $\beta$ is called the _decoding scaling factor_. When $\beta=1$ (no
scaling), the scheme will just select the best generated triple as in normal
decoding. When $\beta>1$, the scheme will select the best positive triple
$\mathbf{y}_{+}^{*}$ only if its score is greater than the score of the best
negative triple $\mathbf{y}_{-}^{*}$ by a margin, so the predicted positive
triple is more likely to be a true positive. Therefore, the total number of
false positives will be reduced, hence improving the precision.

###  2.2 One-Pass Approach

Under the one-pass approach, each time we encode the information of all the
entities of the input sentence $\mathbf{x}$ in the source sequence. The target
sequence also includes all the positive relation triples supported by the
input sentence.

Let $E(\mathbf{x})$ be the set of all entities in $\mathbf{x}$, where an
entity $\mathbf{e}_{i}=(x_{i_{1}},...,x_{i_{2}})\in E(\mathbf{x})$ is a span
in $\mathbf{x}$, with entity type $T_{i}$. Let $R(\mathbf{x})$ be the set of
all positive relation triples supported by $\mathbf{x}$, where a relation
triple $\mathbf{t}_{j}=(\mathbf{s}_{j},r_{j},\mathbf{o}_{j})\in R(\mathbf{x})$
consists of a subject entity $\mathbf{s}_{j}\in E(\mathbf{x})$, an object
entity $\mathbf{o}_{j}\in E(\mathbf{x})$, and their relation type $r_{j}\in
R$.

First we extend the entity type marking to all the entities in the input
sentence as follows:

$\displaystyle f_{1}(\mathbf{x},E(\mathbf{x}))$ $\displaystyle=$
$\displaystyle\Big{(}x_{1},...,T_{i},x_{i_{1}},...,x_{i_{2}},T_{i},...,x_{n},\forall\mathbf{e}_{i}\in
E(\mathbf{x})\Big{)}$

Then we encode the list of entities with their entity types as follows:

$f_{2}(E(\mathbf{x}))=[\mathbf{e}_{i}\;\\#\;T_{i}\;,\forall\mathbf{e}_{i}\in
E(\mathbf{x})]$ (9)

We also include the list of relation types as in (4). The final encoding of
the source sequence is:

$src\\_seq=f_{1}(\mathbf{x},E(\mathbf{x}))\oplus f_{2}(E(\mathbf{x}))\oplus
f_{3}(R)$ (10)

The encoding of the target sequence is:

$tgt\\_seq=\oplus_{(\mathbf{s}_{j},r_{j},\mathbf{o}_{j})\in
R(\mathbf{x})}[\mathbf{s}_{j}\;|\;r_{j}\;|\;\mathbf{o}_{j}]$ (11)

In case there is no positive relation triple in $\mathbf{x}$ (i.e., $R(\mathbf{x})=\emptyset$), we set $tgt\\_seq$ to be “[None | None | None]”.

If no entity information is provided, then the encoding of the source sequence
is the concatenation of the input sentence and the list of relation types:

$src\\_seq=\mathbf{x}\oplus f_{3}(R)$ (12)

The encoding of the target sequence is the same as in (11).

Note that for an input sentence with $m$ entities, the entity-pair approach
will create $m(m-1)$ $src\\_seq$ and $tgt\\_seq$ pairs, while the one-pass
approach will create just one $src\\_seq$ and $tgt\\_seq$ pair, which is much
more computationally efficient.

###  2.3 Seq2Seq Model

For both the entity-pair approach and the one-pass approach, we apply a
seq2seq model such as BART Lewis et al. (2020a) or T5 Raffel et al. (2020) to
convert a source sequence to a target sequence. BART/T5 uses a standard
Transformer based neural machine translation architecture Vaswani et al.
(2017) with multiple bidirectional encoder layers and autoregressive decoder
layers. Both models were pre-trained with large English text corpora as a
denoising autoencoder that maps a corrupted document to the original document.
We convert the RE data to source and target sequence pairs as described in the
previous subsections, and use them to fine-tune the seq2seq models for
relation triple generation.

##  3 Experiments

###  3.1 Datasets

We evaluate the GREC model on 3 popular benchmark relation extraction and
classification datasets: ACE05, SemEval 2010 Task 8, and TACRED.

The ACE05 dataset Walker et al. (2006) is a benchmark relation extraction
dataset developed by the Linguistic Data Consortium for the purpose of
Automatic Content Extraction (ACE) technology evaluation. ACE05 defines 7
entity types and 6 relation types between the entities. We use the same
training, development, and test data split in prior works Li and Ji (2014);
Luan et al. (2019).

The SemEval 2010 Task 8 dataset Hendrickx et al. (2010) is a benchmark dataset
for relation classification. It defines 9 relation types and a null relation
type “Other”. It has 8000 training examples and 2717 test examples. We
randomly select 1000 training examples for development.

TACRED Zhang et al. (2017b) is a large supervised relation classification
dataset obtained via crowdsourcing. It defines 42 relation types (including a
null relation type “no_relation”) and includes over 100K examples. The dataset
was recently revised and improved in Alt et al. (2020) by reducing the
annotation errors. In our experiments we use this revised version, which
includes 68,124 training examples, 22,631 development examples, and 15,509
test examples.

###  3.2 Implementation Details

We build the GREC model on top of Transformer based seq2seq models including
BART and T5, with HuggingFace’s pytorch implementation Wolf et al. (2020). Our
preliminary experiment results (Table 6) show that BART and T5 achieve similar
performance. We choose one model (bart-large) to run all the experiments,
which is a common practice to reduce the total computational cost and energy
consumption.

We use the development sets to tune the hyper-parameters. We learn the model
parameters using Adam Kingma and Ba (2015), with a learning rate $l=3e$-5, a
training batch size of $b=16$ for ACE05 and SemEval 2010 Task 8, and $b=8$ for
TACRED. We train the GREC model for 10 epochs with the entity-pair approach
and 20 epochs with the one-pass approach. All experiments were conducted on a
2 NVIDIA V100 GPUs computer.

Model | Entity | Rel | Rel+
---|---|---|---
Li and Ji (2014) | 80.8 | 52.1 | 49.5
SPTree Miwa and Bansal (2016) | 83.4 | - | 55.6
Katiyar and Cardie (2017) | 82.6 | 55.9 | 53.6
Zhang et al. (2017a) | 83.6 | - | 57.5
MRT Sun et al. (2018) | 83.6 | - | 59.6
Li et al. (2019) | 84.8 | - | 60.2
Dixit and Al-Onaizan (2019) | 86.0 | - | 62.8
DYGIE Luan et al. (2019)∗ | 88.4 | 63.2 | -
DyGIE++ Wadden et al. (2019)∗ | 88.6 | 63.4 | -
Lin et al. (2020) | 88.8 | 67.5 | -
Wang and Lu (2020) | 89.5 | 67.6 | 64.3
TANL Paolini et al. (2021) | 88.9 | 63.7 | -
PURE - single sentence Zhong and Chen (2021) | 89.7 | 69.0 | 65.6
PURE - cross sentence Zhong and Chen (2021)∗ | 90.9 | 69.4 | 67.0
GREC (ours) | 90.4 |  70.2 $\pm$ 0.4 |  68.2 $\pm$ 0.5
Table 1: Micro $F_{1}$ scores on the ACE05 test set. For GREC we report the _mean_ and _standard deviation_ of the performance over 5 runs. ∗These models use cross-sentence information.  Model |  Macro $\mathbf{F_{1}}$
---|---
CNN Zeng et al. (2014) | 82.7
Attention Bi-LSTM Zhou et al. (2016) | 84.0
CR-CNN dos Santos et al. (2015) | 84.1
Bi-LSTM Zhang et al. (2015) | 84.3
Hierarchical Attention RNN Xiao and Liu (2016) | 84.3
Entity Attention Bi-LSTM Lee et al. (2019) | 85.2
Attention CNN Shen and Huang (2016) | 85.9
TRE Alt et al. (2019) | 87.1
SpanRel Jiang et al. (2020) | 87.4
Multi-Attention CNN Wang et al. (2016) | 88.0
KnowBERT-W+W Peters et al. (2019)∗ | 89.1
R-BERT Wu and He (2019) | 89.25
BERTEM Baldini Soares et al. (2019) | 89.2
BERTEM+MTB Baldini Soares et al. (2019)∗ | 89.5
GREC (ours) |  89.9 $\pm$ 0.1
Table 2: Macro $F_{1}$ scores on the SemEval 2010 Task 8 test set. For GREC we report the _mean_ and _standard deviation_ of the performance over 5 runs. ∗These models use additional data derived from Wikipedia/WordNet to pre-train their models. Model |  Micro $\mathbf{F_{1}}$
---|---
LSTM (masked) Zhang et al. (2017b) | 63.9
LSTM + BERT (masked) Alt et al. (2020) | 73.4
CNN (masked) Nguyen and Grishman (2015) | 66.5
CNN + BERT (masked) Alt et al. (2020) | 74.3
TRE Alt et al. (2019) | 75.3
SpanBERT Joshi et al. (2020) | 78.0
KnowBERT-W+W Peters et al. (2019)∗ | 79.3
GREC (ours) |  80.6 $\pm$ 0.6
Table 3: Micro $F_{1}$ scores on the TACRED-Revised test set. For GREC we
report the _mean_ and _standard deviation_ of the performance over 5 runs.
∗This model was pre-trained with additional data derived from Wikipedia and
WordNet.

###  3.3 Main Results

Our best GREC model with the entity-pair approach uses source sequence
encoding (5) with entity type markers (2) and target sequence encoding (6).
The results of the GREC model reported in Tables 1-3 include the _mean_ and
_standard deviation_ of the performance over 5 runs with different random
seeds.

In Table 1 we compare the GREC model with previous approaches on the ACE05
test set. As in prior works we use micro-averaged $F_{1}$ score as the
evaluation metric. For entity recognition, a predicted entity is considered
correct if its predicted entity span and entity type are both correct. For
relation extraction with predicted entities, following Li and Ji (2014); Wang
and Lu (2020); Zhong and Chen (2021), we use two evaluation metrics: 1) _Rel_
: a predicted relation is considered correct if the two predicted entity spans
and the predicted relation type are correct; 2) _Rel+_ : a predicted relation
is considered correct if the two predicted entity spans and entity types as
well as the predicted relation type are all correct. Our entity recognition
model is an ensemble of RoBERTa Liu et al. (2019) based sequence labeling
models with voting.

As shown in Table 1, the GREC model achieves the state-of-the-art performance
on ACE05. GREC improves the previous best model PURE by 1.2 $F_{1}$ points on
the Rel metric and by 2.6 $F_{1}$ points on the Rel+ metric with the single-
sentence setting.

Source | Target | $P$ | $R$ | $F_{1}$
---|---|---|---|---
no entity marker | $[s|r|o]$ | 70.2 | 68.8 | 69.5
special token marker | $[s|r|o]$ | 73.2 | 68.2 | 70.6
entity type marker | $\mathbf{[s|r|o]}$ | 74.4 | 76.1 | 75.2
entity type marker | $[r]$ | 71.2 | 66.9 | 69.0
entity type marker | $[r|s|o]$ | 71.8 | 75.3 | 73.5
entity type marker | $[s|o|r]$ | 73.7 | 72.8 | 73.3
Table 4: Performance (precision, recall, $F_{1}$ score) of the GREC model on the ACE05 development set (with gold entities) under different source and target sequence encoding representations. |  $\beta$ = 1 |  $\beta$ = 1.1 |  $\beta$ = 1.2 |  $\beta$ = 1.3
---|---|---|---|---
| $P$ | $R$ | $F_{1}$ | $P$ | $R$ | $F_{1}$ | $P$ | $R$ | $F_{1}$ | $P$ | $R$ | $F_{1}$
$\alpha=1$ | 72.1 | 75.0 | 73.5 | 74.2 | 73.0 | 73.6 | 76.2 | 71.4 | 73.8 | 77.5 | 69.0 | 73.0
$\alpha=0.9$ | 71.6 | 78.7 | 75.0 | 73.7 | 75.9 | 74.7 | 76.6 | 73.3 | 74.9 | 77.5 | 70.4 | 73.8
$\alpha=0.8$ | 68.2 | 79.3 | 73.4 | 71.7 | 78.2 | 74.8 | 74.4 | 76.1 | 75.2 | 76.4 | 73.3 | 74.8
$\alpha=0.7$ | 66.6 | 78.7 | 72.2 | 69.9 | 76.9 | 73.2 | 72.1 | 74.6 | 73.3 | 74.5 | 72.5 | 73.5
Table 5: Performance of the GREC model on the ACE05 development set (with gold entities) under different negative sampling ratio $\alpha$ and decoding scaling factor $\beta$. Model | Parameters | $F_{1}$
---|---|---
t5-base |  $L$=12, $A$=12, $H$=768 | 72.6
bart-base |  $L$=12, $A$=16, $H$=768 | 73.5
bart-large |  $L$=24, $A$=16, $H$=1024 | 75.2
Table 6: Performance ($F_{1}$ score) of the GREC model on the ACE05 development set (with gold entities) under different pre-trained seq2seq models. $L$ is the number of transformer layers, $A$ is the number of attention heads, and $H$ is the hidden state vector size. Setup | Performance | Computational Cost
---|---|---
Approach | Entities | $P$ | $R$ | $F_{1}$ | Training | Decoding
entity-pair | _gold_ | 74.4 | 76.1 | 75.2 | 559 mins | 459 secs
entity-pair | _predicted_ | 66.7 | 68.3 | 67.5 | 559 mins | 456 secs
one-pass | _gold_ | 66.0 | 65.8 | 65.9 | 90 mins | 32 secs
one-pass | _predicted_ | 63.4 | 56.5 | 59.7 | 90 mins | 28 secs
one-pass | _no_ | 55.8 | 51.9 | 53.8 | 80 mins | 29 secs
Table 7: Performance and computational cost of the GREC model under the
entity-pair approach and the one-pass approach on the ACE05 development set.
Training and decoding time is measured on a 2 NVIDIA V100 GPUs computer with a
batch size of 16, 10 training epochs for entity-pair and 20 training epochs
for one-pass. _gold_ : the gold entities are given during inference.
_predicted_ : the predicted entities are given during inference. _no_ : no
entities are given during inference (and training).

In Table 2 we compare the GREC model with previous approaches on the SemEval
2010 Task 8 test set. As in prior works we use the SemEval 2010 Task 8
official scoring metric which is macro-averaged $F_{1}$ score for the 9
relation types (excluding the null relation type “Other”) and takes
directionality into account. The GREC model achieves the state-of-art
performance. While the BERTEM+MTB model used additional data (600 million
relation statement pairs derived from English Wikipeida) to pre-train the
model, GREC achieves better performance without using any additional data.

In Table 3 we compare the GREC model with previous approaches on the revised
TACRED test set. As in prior works we use micro-averaged $F_{1}$ score as the
evaluation metric. Again, the GREC model achieves the state-of-the-art
performance without using any additional data.

###  3.4 Ablation Studies

In this subsection we study the contributions of different components on the
GREC model.

####  3.4.1 Source and Target Sequence Encoding

In Table 4 we show the performance of the GREC model on the ACE05 development
set under different source and target sequence encoding representations. There
are two observations:

  * •

For the source sequence encoding, it is important to encode the entity
information in the input sentence using entity markers. The special token
markers (1) that encode the entity location information improved the
performance by 1.1 $F_{1}$ points, and the entity type markers (2) that encode
both the entity location and type information improved the performance by 5.7
$F_{1}$ points, compared with not using any entity markers.

  * •

For the target sequence encoding, it is beneficial to add the subject and
object entities in the target sequence, which helps the GREC model to generate
more accurate relation types. This improved the performance by 4+ $F_{1}$
points compared with generating the relation type only ($[r]$). Among the
different orders of the relation triple that we have tried, the order
“subject, relation, object” ($[s|r|o]$) achieved the best performance. The
reason could be that the order “subject, relation, object” (e.g., “Toefting,
works for, Bolton”) is the one that is the most consistent with the English
language SVO order and the seq2seq model (BART) was pre-trained with English
text.

####  3.4.2 Negative Sampling and Decoding Scaling

In Table 5 we show the performance of the GREC model on the ACE05 development
set under different negative sampling ratio $\alpha$ and decoding scaling
factor $\beta$ (we let the GREC model generate top $N$=5 target sequences).
The key observations are:

  * •

For a fixed decoding scaling factor $\beta$ (a column in Table 5), as we
decrease the negative sampling ratio $\alpha$ (i.e., keep fewer negative
training examples during training), the recall is improved. The recall reached
the highest value at $\alpha=0.8$, and further decreasing $\alpha$ could
reduce the recall.

  * •

For a fixed negative sampling ratio $\alpha$ (a row in Table 5), as we
increase the decoding scaling factor $\beta$ (so the predicted positive triple
is more likely to be a true positive triple), the precision is improved.
However, increasing $\beta$ hurts the recall.

  * •

Negative sampling and decoding scaling provide a flexible tool to tune the
precision and recall performance of the GREC model. If we want to achieve a
high recall, we would keep $\beta=1$ (no decoding scaling) and select an
optimal $\alpha$: in this case $\alpha=0.8$ gives the best recall performance
of 79.3. On the other hand, if we want to have a higher precision, we would
keep $\alpha=1$ (no negative sampling) and pick a larger $\beta$. We can also
use the development set to find the optimal $\alpha$ and $\beta$ that achieve
the highest $F_{1}$ score.

####  3.4.3 Pre-trained Seq2Seq Models

In Table 6 we show the performance of the GREC model under 3 pre-trained
seq2seq models: t5-base, bart-base and bart-large. The performance of GREC is
robust across different pre-trained models, with the deeper and larger bart-
large model achieves the best performance.

###  3.5 Entity-Pair vs. One-Pass Approach

A sentence can have multiple entities and relation triples in the ACE05 data,
so we use ACE05 to compare the performance and computational cost of the GREC
model under the entity-pair approach and the one-pass approach.

As shown in Table 7, the entity-pair approach has a clear advantage over the
one-pass approach on performance (nearly 10 $F_{1}$ points gain). On the other
hand, since the one-pass approach creates just one source sequence for a
sentence and extracts all the relation triples from the sentence in one-pass,
it has a much smaller number of training/test examples and hence lower
computational cost (6x faster for training and 15x faster for decoding)
compared with the entity-pair approach. Another key observation is that
encoding entity information (even predicted) can significantly improve the
performance compared with no entities provided.

##  4 Related Work

Many RE models have been developed to improve the performance on benchmark RE
datasets such as ACE05, SemEval 2010 Task 8 and TACRED. Earlier RE models
require extensive feature engineering to derive and combine various lexical,
syntactic and semantic features Kambhatla (2004); Zhou et al. (2005); Chan and
Roth (2011); Li and Ji (2014). Later neural network based RE models have
become dominant, including CNN based models Zeng et al. (2014); dos Santos et
al. (2015); Nguyen and Grishman (2015), RNN based models Zhang et al. (2015);
Xiao and Liu (2016); Miwa and Bansal (2016); Ni and Florian (2019), and most
recently Transformer based models Wu and He (2019); Baldini Soares et al.
(2019); Zhong and Chen (2021).

Seq2seq models have been used for NLP tasks such as machine translation
Sutskever et al. (2014); Cho et al. (2014) and text summarization Rush et al.
(2015); Chopra et al. (2016). Recently, generative approaches based on seq2seq
models have been proven competitive in NLP applications such as question
answering, fact checking, relation linking and intent classification Lewis et
al. (2020b); Petroni et al. (2021); Rossiello et al. (2021); Ahmad et al.
(2021). While seq2seq models were also applied to RE Zeng et al. (2018, 2020);
Nayak and Ng (2020); Zhang et al. (2020); Paolini et al. (2021), the previous
works focused on end-to-end relation extraction that jointly extracts entities
and relations from sentences. Our work is also based on the seq2seq framework.
The main difference is that we focus on relation extraction and classification
with entities provided, and we show it is beneficial to encode entity
information both in the source and target sequences to achieve the best
performance.

##  5 Conclusion

In this paper we presented a novel generative model for relation extraction
and classification. We showed the importance of encoding entity information in
the source and target sequences and designed effective encoding
representations that enable the model to achieve state-of-the-art performance
on three popular benchmark RE datasets. Our model is easy to implement with
standard pre-trained seq2seq models like BART, has components to flexibly tune
the precision and recall performance, and has the potential of significantly
reducing the training and decoding time via the one-pass formulation.

## References

  * Ahmad et al. (2021) Wasi Ahmad, Jianfeng Chi, Tu Le, Thomas Norton, Yuan Tian, and Kai-Wei Chang. 2021\.  Intent classification and slot filling for privacy policies.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 4402–4417, Online. Association for Computational Linguistics.
  * Alt et al. (2020) Christoph Alt, Aleksandra Gabryszak, and Leonhard Hennig. 2020.  TACRED revisited: A thorough evaluation of the TACRED relation extraction task.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1558–1569, Online. Association for Computational Linguistics.
  * Alt et al. (2019) Christoph Alt, Marc Hübner, and Leonhard Hennig. 2019.  Improving relation extraction by pre-trained language representations.  In _1st Conference on Automated Knowledge Base Construction, AKBC 2019, Amherst, MA, USA, May 20-22, 2019_.
  * Baldini Soares et al. (2019) Livio Baldini Soares, Nicholas FitzGerald, Jeffrey Ling, and Tom Kwiatkowski. 2019\.  Matching the blanks: Distributional similarity for relation learning.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 2895–2905, Florence, Italy. Association for Computational Linguistics.
  * Chan and Roth (2011) Yee Seng Chan and Dan Roth. 2011.  Exploiting syntactico-semantic structures for relation extraction.  In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_ , pages 551–560, Portland, Oregon, USA. Association for Computational Linguistics.
  * Cho et al. (2014) Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014.  Learning phrase representations using RNN encoder–decoder for statistical machine translation.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1724–1734, Doha, Qatar. Association for Computational Linguistics.
  * Chopra et al. (2016) Sumit Chopra, Michael Auli, and Alexander M. Rush. 2016.  Abstractive sentence summarization with attentive recurrent neural networks.  In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 93–98, San Diego, California. Association for Computational Linguistics.
  * Dixit and Al-Onaizan (2019) Kalpit Dixit and Yaser Al-Onaizan. 2019.  Span-level model for relation extraction.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 5308–5314, Florence, Italy. Association for Computational Linguistics.
  * Doddington et al. (2004) George Doddington, Alexis Mitchell, Mark Przybocki, Lance Ramshaw, Stephanie Strassel, and Ralph Weischedel. 2004.  The automatic content extraction (ACE) program – tasks, data, and evaluation.  In _Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04)_ , Lisbon, Portugal. European Language Resources Association (ELRA).
  * dos Santos et al. (2015) Cícero dos Santos, Bing Xiang, and Bowen Zhou. 2015.  Classifying relations by ranking with convolutional neural networks.  In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 626–634, Beijing, China. Association for Computational Linguistics.
  * Florian et al. (2003) Radu Florian, Abe Ittycheriah, Hongyan Jing, and Tong Zhang. 2003.  Named entity recognition through classifier combination.  In _Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003_ , pages 168–171.
  * Hendrickx et al. (2010) Iris Hendrickx, Su Nam Kim, Zornitsa Kozareva, Preslav Nakov, Diarmuid Ó Séaghdha, Sebastian Padó, Marco Pennacchiotti, Lorenza Romano, and Stan Szpakowicz. 2010.  SemEval-2010 task 8: Multi-way classification of semantic relations between pairs of nominals.  In _Proceedings of the 5th International Workshop on Semantic Evaluation_ , pages 33–38, Uppsala, Sweden. Association for Computational Linguistics.
  * Ji and Grishman (2011) Heng Ji and Ralph Grishman. 2011.  Knowledge base population: Successful approaches and challenges.  In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_ , pages 1148–1158, Portland, Oregon, USA. Association for Computational Linguistics.
  * Jiang et al. (2020) Zhengbao Jiang, Wei Xu, Jun Araki, and Graham Neubig. 2020.  Generalizing natural language analysis through span-relation representations.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 2120–2133, Online. Association for Computational Linguistics.
  * Joshi et al. (2020) Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020.  SpanBERT: Improving pre-training by representing and predicting spans.  _Transactions of the Association for Computational Linguistics_ , 8:64–77.
  * Kambhatla (2004) Nanda Kambhatla. 2004.  Combining lexical, syntactic, and semantic features with maximum entropy models for information extraction.  In _Proceedings of the ACL Interactive Poster and Demonstration Sessions_ , pages 178–181, Barcelona, Spain. Association for Computational Linguistics.
  * Katiyar and Cardie (2017) Arzoo Katiyar and Claire Cardie. 2017.  Going out on a limb: Joint extraction of entity mentions and relations without dependency trees.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 917–928, Vancouver, Canada. Association for Computational Linguistics.
  * Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015.  Adam: A method for stochastic optimization.  In _Proceedings of the 3rd International Conference on Learning Representations (ICLR)_ , ICLR ’15.
  * Lample et al. (2016) Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016.  Neural architectures for named entity recognition.  In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 260–270, San Diego, California. Association for Computational Linguistics.
  * Lee et al. (2019) Joohong Lee, Sangwoo Seo, and Yong Suk Choi. 2019.  Semantic relation classification via bidirectional LSTM networks with entity-aware attention using latent entity typing.  _CoRR_ , abs/1901.08163.
  * Lewis et al. (2020a) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020a.  BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7871–7880, Online. Association for Computational Linguistics.
  * Lewis et al. (2020b) Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020b.  Retrieval-augmented generation for knowledge-intensive NLP tasks.  In _Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual_.
  * Li and Ji (2014) Qi Li and Heng Ji. 2014.  Incremental joint extraction of entity mentions and relations.  In _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 402–412, Baltimore, Maryland. Association for Computational Linguistics.
  * Li et al. (2019) Xiaoya Li, Fan Yin, Zijun Sun, Xiayu Li, Arianna Yuan, Duo Chai, Mingxin Zhou, and Jiwei Li. 2019.  Entity-relation extraction as multi-turn question answering.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 1340–1350, Florence, Italy. Association for Computational Linguistics.
  * Lin et al. (2020) Ying Lin, Heng Ji, Fei Huang, and Lingfei Wu. 2020.  A joint neural model for information extraction with global features.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7999–8009, Online. Association for Computational Linguistics.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized BERT pretraining approach.  _CoRR_ , abs/1907.11692.
  * Luan et al. (2019) Yi Luan, Dave Wadden, Luheng He, Amy Shah, Mari Ostendorf, and Hannaneh Hajishirzi. 2019.  A general framework for information extraction using dynamic span graphs.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 3036–3046, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Miwa and Bansal (2016) Makoto Miwa and Mohit Bansal. 2016.  End-to-end relation extraction using LSTMs on sequences and tree structures.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1105–1116, Berlin, Germany. Association for Computational Linguistics.
  * Nayak and Ng (2020) Tapas Nayak and Hwee Tou Ng. 2020.  Effective modeling of encoder-decoder architecture for joint entity and relation extraction.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 34(05):8528–8535.
  * Nguyen and Grishman (2015) Thien Huu Nguyen and Ralph Grishman. 2015.  Relation extraction: Perspective from convolutional neural networks.  In _Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing_ , pages 39–48, Denver, Colorado. Association for Computational Linguistics.
  * Ni and Florian (2019) Jian Ni and Radu Florian. 2019.  Neural cross-lingual relation extraction based on bilingual word embedding mapping.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 399–409, Hong Kong, China. Association for Computational Linguistics.
  * Ni et al. (2020) Jian Ni, Taesun Moon, Parul Awasthy, and Radu Florian. 2020.  Cross-lingual relation extraction with transformers.  _CoRR_ , abs/2010.08652.
  * Paolini et al. (2021) Giovanni Paolini, Ben Athiwaratkun, Jason Krone, Jie Ma, Alessandro Achille, Rishita Anubhai, Cicero Nogueira dos Santos, Bing Xiang, and Stefano Soatto. 2021\.  Structured prediction as translation between augmented natural languages.  In _9th International Conference on Learning Representations, ICLR 2021_.
  * Peters et al. (2019) Matthew E. Peters, Mark Neumann, Robert Logan, Roy Schwartz, Vidur Joshi, Sameer Singh, and Noah A. Smith. 2019.  Knowledge enhanced contextual word representations.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 43–54, Hong Kong, China. Association for Computational Linguistics.
  * Petroni et al. (2021) Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, and Sebastian Riedel. 2021\.  KILT: a benchmark for knowledge intensive language tasks.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 2523–2544, Online. Association for Computational Linguistics.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _Journal of Machine Learning Research_ , 21(140):1–67.
  * Rossiello et al. (2021) Gaetano Rossiello, Nandana Mihindukulasooriya, Ibrahim Abdelaziz, Mihaela A. Bornea, Alfio Gliozzo, Tahira Naseem, and Pavan Kapanipathi. 2021.  Generative relation linking for question answering over knowledge bases.  In _The Semantic Web - ISWC 2021 - 20th International Semantic Web Conference, ISWC 2021, Virtual Event, October 24-28, 2021, Proceedings_ , volume 12922 of _Lecture Notes in Computer Science_ , pages 321–337. Springer.
  * Rush et al. (2015) Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015.  A neural attention model for abstractive sentence summarization.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_ , pages 379–389, Lisbon, Portugal. Association for Computational Linguistics.
  * Shen and Huang (2016) Yatian Shen and Xuanjing Huang. 2016.  Attention-based convolutional neural network for semantic relation extraction.  In _Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers_ , pages 2526–2536, Osaka, Japan. The COLING 2016 Organizing Committee.
  * Sun et al. (2018) Changzhi Sun, Yuanbin Wu, Man Lan, Shiliang Sun, Wenting Wang, Kuang-Chih Lee, and Kewen Wu. 2018.  Extracting entities and relations with joint minimum risk training.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 2256–2265, Brussels, Belgium. Association for Computational Linguistics.
  * Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014.  Sequence to sequence learning with neural networks.  In _Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2_ , NIPS’14, page 3104–3112, Cambridge, MA, USA. MIT Press.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _NIPS_ , pages 6000–6010.
  * Wadden et al. (2019) David Wadden, Ulme Wennberg, Yi Luan, and Hannaneh Hajishirzi. 2019.  Entity, relation, and event extraction with contextualized span representations.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 5784–5789, Hong Kong, China. Association for Computational Linguistics.
  * Walker et al. (2006) Christopher Walker, Stephanie Strassel, Julie Medero, and Kazuaki Maeda. 2006.  ACE 2005 multilingual training corpus.
  * Wang and Lu (2020) Jue Wang and Wei Lu. 2020.  Two are better than one: Joint entity and relation extraction with table-sequence encoders.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1706–1721, Online. Association for Computational Linguistics.
  * Wang et al. (2016) Linlin Wang, Zhu Cao, Gerard de Melo, and Zhiyuan Liu. 2016.  Relation classification via multi-level attention CNNs.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1298–1307, Berlin, Germany. Association for Computational Linguistics.
  * Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020.  Transformers: State-of-the-art natural language processing.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 38–45, Online. Association for Computational Linguistics.
  * Wu and He (2019) Shanchan Wu and Yifan He. 2019.  Enriching pre-trained language model with entity information for relation classification.  CIKM ’19, page 2361–2364, New York, NY, USA. Association for Computing Machinery.
  * Xiao and Liu (2016) Minguang Xiao and Cong Liu. 2016.  Semantic relation classification via hierarchical recurrent neural network with attention.  In _Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers_ , pages 1254–1263, Osaka, Japan. The COLING 2016 Organizing Committee.
  * Xu et al. (2016) Kun Xu, Siva Reddy, Yansong Feng, Songfang Huang, and Dongyan Zhao. 2016.  Question answering on Freebase via relation extraction and textual evidence.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 2326–2336, Berlin, Germany. Association for Computational Linguistics.
  * Zeng et al. (2014) Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, and Jun Zhao. 2014.  Relation classification via convolutional deep neural network.  In _Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers_ , pages 2335–2344, Dublin, Ireland. Dublin City University and Association for Computational Linguistics.
  * Zeng et al. (2020) Daojian Zeng, Haoran Zhang, and Qianying Liu. 2020.  Copymtl: Copy mechanism for joint extraction of entities and relations with multi-task learning.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 34(05):9507–9514.
  * Zeng et al. (2018) Xiangrong Zeng, Daojian Zeng, Shizhu He, Kang Liu, and Jun Zhao. 2018.  Extracting relational facts by an end-to-end neural model with copy mechanism.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 506–514, Melbourne, Australia. Association for Computational Linguistics.
  * Zhang et al. (2017a) Meishan Zhang, Yue Zhang, and Guohong Fu. 2017a.  End-to-end neural relation extraction with global optimization.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 1730–1740, Copenhagen, Denmark. Association for Computational Linguistics.
  * Zhang et al. (2020) Ranran Haoran Zhang, Qianying Liu, Aysa Xuemo Fan, Heng Ji, Daojian Zeng, Fei Cheng, Daisuke Kawahara, and Sadao Kurohashi. 2020.  Minimize exposure bias of Seq2Seq models in joint entity and relation extraction.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 236–246, Online. Association for Computational Linguistics.
  * Zhang et al. (2015) Shu Zhang, Dequan Zheng, Xinchen Hu, and Ming Yang. 2015.  Bidirectional long short-term memory networks for relation classification.  In _Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation_ , pages 73–78, Shanghai, China.
  * Zhang et al. (2017b) Yuhao Zhang, Victor Zhong, Danqi Chen, Gabor Angeli, and Christopher D. Manning. 2017b.  Position-aware attention and supervised data improve slot filling.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 35–45, Copenhagen, Denmark. Association for Computational Linguistics.
  * Zhong and Chen (2021) Zexuan Zhong and Danqi Chen. 2021.  A frustratingly easy approach for entity and relation extraction.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 50–61, Online. Association for Computational Linguistics.
  * Zhou et al. (2005) GuoDong Zhou, Jian Su, Jie Zhang, and Min Zhang. 2005.  Exploring various knowledge in relation extraction.  In _Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05)_ , pages 427–434, Ann Arbor, Michigan. Association for Computational Linguistics.
  * Zhou et al. (2016) Peng Zhou, Wei Shi, Jun Tian, Zhenyu Qi, Bingchen Li, Hongwei Hao, and Bo Xu. 2016\.  Attention-based bidirectional long short-term memory networks for relation classification.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 207–212, Berlin, Germany. Association for Computational Linguistics.

##  Appendix A Examples of Source and Target Sequence Encoding

Approach |  Source Sequence |  Target Sequence
---|---|---
Entity-Pair |  Person Toefting Person transferred to Organization Bolton Organization in February 2002 from German club Hamburg. [Toefting # Person , Bolton # Organization] [affiliated to - located at - makes - part of - relationship - works for] |  [Toefting | works for | Bolton]
One-Pass |  Person Toefting Person transferred to Organization Bolton Organization in February 2002 from Geo-political German Geo-political Organization club Organization Organization Hamburg Organization. [Toefting # Person , Bolton # Organization , German # Geo-political , club # Organization , Hamburg # Organization] [affiliated to - located at - makes - part of - relationship - works for] |  [Toefting | works for | Bolton] [Toefting | works for | Hamburg] [club | affiliated to | German]
Table 8: Examples of the source and target sequence encoding under the entity-
pair approach and the one-pass approach. For the source sequence, entity type
markers are shown in bold.


# Churn prediction in online gambling

Merchie Florian***Corresponding author. Tel: +32494460103
Email addresses<EMAIL_ADDRESS>(F. Merchie<EMAIL_ADDRESS>(D.
Ernst) Department of Electrical Engineering and Computer Science, ULiège
Ernst Damien  Department of Electrical Engineering and Computer Science,
ULiège  LTCI, Telecom Paris, Institut Polytechnique de Paris

###### Abstract

In business retention, churn prevention has always been a major concern. This
work contributes to this domain by formalizing the problem of churn prediction
in the context of online gambling as a binary classification task. We also
propose an algorithmic answer to this problem based on recurrent neural
network. This algorithm is tested with online gambling data that have the form
of time series, which can be efficiently processed by recurrent neural
networks. To evaluate the performances of the trained models, standard machine
learning metrics were used, such as accuracy, precision and recall. For this
problem in particular, the conducted experiments allowed to assess that the
choice of a specific architecture depends on the metric which is given the
greatest importance. Architectures using nBRC favour precision, those using
LSTM give better recall, while GRU-based architectures allow a higher accuracy
and balance two other metrics. Moreover, further experiments showed that using
only the more recent time-series histories to train the networks decreases the
quality of the results. We also study the performances of models learned at a
specific instant $t$, at other times $t^{\prime}>t$. The results show that the
performances of the models learned at time $t$ remain good at the following
instants $t^{\prime}>t$, suggesting that there is no need to refresh the
models at a high rate. However, the performances of the models were subject to
noticeable variance due to one-off events impacting the data.

##  1 Introduction

In a business-to-consumer context, customer retention has always been
considered as a key issue. Retaining a loyal customer base is crucial for the
longevity of a company. However, enterprises also face a regular loss of
clients, even those who have a loyal customer base. We usually call this loss
’churn’, and its prevention is a major business challenge. The work proposed
in [1, Ascarza et al. 2016] explains how the behaviour of service providers
can be modified by a rise of churn. This work also studies the effects of
these changes and shows that usual offers such as attractive pricing plans can
sometimes increase churn rather than decreasing it. The recent progress made
in data science has eased the anticipation of churn events. By using
customers’ activity history, it is possible to identify different patterns
associated to those who are the most likely to leave. This issue was addressed
in [2, Wei et al. 2002]. That work proposed a data mining technique able to
detect such behaviors in the context of mobile services. However, in non-
contractual business-to-consumer sectors, such as online gambling, the
activity patterns are much more diverse. Indeed, the customers do not have,
for instance, any monthly fee to pay and therefore will not feel obliged to
use the service. In such contexts, churn detection becomes even more difficult
to achieve. To solve such problems, more advanced technologies for analysing
customer activity patterns, which can be represented as time series, can be
used.

From a machine learning perspective, the analysis of time series for
prediction has been an important research topic for a very long time. Among
the many works in the field, several mention techniques using neural networks,
such as [3, LeCun et al, 1995] and [4, Zhang et al. 2003]. Those techniques
have gained in popularity with the advent of recurrent neural networks (RNN),
which have greatly increased the efficiency of developed models in this
domain. Their ability to capture the temporal relationships within data is a
major asset for prediction tasks requiring memory. Several research works,
such as in [5, Connot et al. 1994], [6, Graves et al. 2013], and [7, Che et
al. 2018], proposed novel and robust solutions to time series-based prediction
tasks using RNNs. This places an emphasis on the relevance of using such
techniques for solving business problems where customers’ activity history can
be described as time series.

In this work, we propose an accurate formalization of the churn prediction
problem as a specific instance of a binary classification task. Then, we
present an algorithmic solution to this problem applied to online gambling.
This algorithm is based on the training of an RNN-based model whose goal is to
predict churn occurrences over a given number of coming days. More
particularly, at some time $t$, the model was learned from past histories. It
was then used to predict the probability of churn of the different online
players at this time $t$, over a given time horizon $T_{pred}$. We also
assessed the ability of the model learned at time $t$ to predict the churn
probability over the same horizon $T_{pred}$, but starting at other instants
$t^{\prime}>t$. These data were collected from Gaming1, a Belgian online
gambling company.

This paper is organized in different sections. Section 2 presents an overview
of the research work performed in churn prediction. Section 3 formalizes the
statement of the churn prediction problem. Section 4 details the methodology
of the proposed solution in terms of hypothesis space, loss function, and
optimization algorithm. Section 5 details the different conducted experiments
and their results. Section 6 discusses the conclusion and future work
perspectives around this paper.

##  2 Related work

Formalising a churn prediction problem is no small matter. This depends on
both the churn definition and the evaluation metrics used for assessing the
performances of the solution to the problem. The research paper [8, Datta et
al.] was the first to address a churn prediction problem with machine learning
techniques. They introduced an automatized procedure applied to churn
prediction in cellular phone services. Churn was defined as a per-customer
monthly indicator based on the use of the services. Regarding other activity
sectors, the research work of [9, Buckinx et al., 2005] proposed a solution
able to identify the potential defection of loyal customers in the insurance
domain. Clients were segmented according to business criteria. Churn was then
associated to loyal customers’ risk of switching to a lower loyalty group. In
the sector of digital music libraries, the research paper [10, Lai et al.,
2014] formulated churn as a situation where a user does not download any music
for a year. They aimed at identifying the periods in a user’s life cycle where
the churn risk was the highest and the customer groups associated to this
risk. In the online gambling sector, the work proposed in [11, Coussement et
al., 2013] evaluated the possibilities of predicting players’ churn based on
their activity history. They formalised churn as a four-month period without
activity. In [12, Suh et al, 2016], churn was customized to each player by
relying on how frequent and recent their activity patterns were. More
specifically, a user was considered as a churner when the number of days since
their last activity was greater than a value depending on the mean and
standard deviation of the number of days between each activity. Concerning the
other component of the problem statement, i.e., the performance metrics, the
research works addressing churn prediction can be split into two categories.
The first category, such as the research detailed in [9, Buckinx et al., 2005]
and [12, Suh et al., 2016], relies on the accuracy and confusion, i.e.,
standard machine learning metrics, to evaluate their solution. On the other
hand, the second category, such as [11, Coussement et al., 2013] for instance,
chose metrics specifically designed for churn detection. Among those, we can
cite the top decile lift (TDL). To compute this value, churn probabilities
need to be associated to customers. Then, customers are sorted according to
this probability and the TDL consists of the ratio between the proportion of
churners in the top 10% of the ordered customers and the proportion of
churners in the entire data set. Globally, we can observe the complexity of
providing a unique definition for the problem of churn prediction, which would
be applicable in any domain, which is mainly due to the diversity of business
concerns in each activity sector.

Regarding the different machine learning techniques that were tested for churn
prediction, the use of decision trees seems quite common, see for example [12,
Suh et al, 2016] and [11, Coussement et al., 2013]. In particular, we note
that in [11, Coussement et al., 2013], the performances of tree-based
techniques were assessed on online sports betting data. The results showed
that an ensemble of trees not only surpassed single-tree techniques, but also,
for both, surpassed the equivalent of their Generalized Additive Model (GAM),
a particular class of linear, equivalents. In the Chinese bank domain, the
research paper of [13] showed that classical tree-based models were
outperformed by a novel supervised learning method, called improved balance
random forests. The model was also successfully challenged against other
state-of-the-art methods such as support vector machine (SVM) and artificial
neural networks (ANNs). ANNs were also used by [9, Buckinx et al., 2005], and
shown to outperform random forests and logistic regression in the insurance
sector. Research work [14, Tsai et al., 2009] introduced new methods using a
combination of ANN and self-organizing maps (SOM) for churn prediction in
American telecommunication services. The research paper of [15, Farquad et
al., 2014] introduced a hybrid approach mixing SVM and rules generated using
naive Bayes tree for customer retention in the bank credit card sector. The
inability to scale up to large data sets was the main weakness of this method.
Some unsupervised techniques were also evaluated, such as in [10, Lai et al.,
2014], where activity-based clustering was performed in order to prevent churn
in online digital libraries.

##  3 Problem statement

This section aims at defining the components of the problem statement,
regarding churn indicator first, and loss function afterwards. In business
terms, churn corresponds to a regular loss of customers, who stop any activity
for a sufficiently long time period. This period can be chosen in an arbitrary
way, depending on the activity sector. In online gambling, any customer can
perform various activities, such as casino playing, sports betting, making
deposits or withdrawals. These activities can be recorded and aggregated
according to some arbitrary time step, and expressed as a multi-dimensional
time series per customer. More formally, for any player
$p\in\mathcal{P}=\\{1,2,\dots,|\mathcal{P}|\\}$, these multi-dimensional time
series can be formulated as
$\mathrm{X}^{p}=\mathrm{x}_{0}^{p},\mathrm{x}_{1}^{p},\mathrm{x}_{2}^{p}\dots$,
with $\mathrm{x}_{i}^{p}\in\mathbb{R}^{n}$, and where $n\in\mathbb{N}_{0}$ is
the number of features used to describe the data. Among these features, only a
subset of them is relevant to define the churn. This subset is denoted as
$\mathrm{X}_{c}^{p}\subset\mathrm{X}^{p}$ and contains the same features for
any player $p$. For each vector $\mathrm{x}^{p}\in\mathrm{X}_{c}^{p}$, we have
$\mathrm{x}^{p}\in\mathbb{R}^{n_{c}}$, where $n_{c}<n$. For these activity
features, we define a minimal number of days $T_{c}\in\mathbb{N}_{0}$ during
which they must stay equal to 0 for churn to occur. Following this condition,
for some player $p$, we can define the churn variable $c_{t}^{p}$ at some time
$t$:

$c_{t}^{p}=\text{max}(0,1-\left\lceil\sum_{i=t-T_{c}+1}^{t}\sum_{j=0}^{n}\left\lvert
x_{i,j}^{p}\right\rvert\right\rceil),~{}\forall
t>t-T_{c},\mathrm{x}\in\mathrm{X}_{c}^{p}$ (3.1)

where $c_{t}^{p}=1$ (resp. $c_{t}=0$) stands for churn occurrence (resp. non
churn). Then, we can define, for player $p$ at time $t$, the churn indicator
$y_{t}^{p}$, which corresponds to a binary flag expressing whether player $p$
will churn at least once in the next $T_{pred}$ time steps after time $t$:

$y_{t}^{p}=\text{min}(1,\sum_{i=t+1}^{t+T_{pred}}\\!\\!c_{i}^{p}),~{}\forall
t\in\mathbb{N}_{0}.$ (3.2)

Having $y_{t}^{p}=1$ (resp. $y_{t}^{p}=0$) means player $p$ will churn (resp.
will be retained) over the time horizon $T_{pred}$ starting at time $t$. In
churn detection, we want to be able to predict this target value at time $t$
for any player $p$ having an activity history $i^{p}_{t}\in X^{p}_{0:t}$,
where $X^{p}_{0:t}$ denotes the subset of the players’ time series going from
time $0$ to $t$. In this paper, we seek to learn a function $h_{t}$ that gives
the probability of having a value of the churn indicator $y_{t}^{p}=1$ for any
player $p$, based on such a trajectory $i^{p}_{t}$. This corresponds to a
binary classification problem, which can be solved using machine learning
techniques.

The quality of such a function $h_{t}$ can be evaluated using a loss function
defining the distance between the probability it outputs and the churn
indicator $y_{t}^{p}$. In this paper, we choose the binary cross-entropy $H$
as loss function, which is most commonly used in classification, expressed as
the following:

$H(y_{t}^{p},h_{t}(i^{p}_{t}))=-(y_{t}^{p}\log
h_{t}(i^{p}_{t})+(1-y_{t}^{p})\log(1-h_{t}(i^{p}_{t}))).$ (3.3)

In practice, we learn this function $h_{t}$ at every time step $t$ and select
it among a hypothesis space $\mathcal{H}$. The next section introduces and
discusses the machine learning methodology chosen in this paper.

##  4 Methodology

This section provides the formalization of the hypothesis space and the loss
function used in the proposed algorithm. Then, it explains how the
optimization procedure navigates through the hypothesis space to select its
best function.

###  4.1 Hypothesis space

The learned function $h_{t}$ introduced in Section 3 is selected among some
hypothesis space $\mathcal{H}$. In this work, every $\mathcal{H}$ that is
used, contains a set of functions that output a probability based on an
activity history $i^{p}_{t}$, for some player $p$. In particular, such a set
consists here of all the neural networks corresponding to an architecture
containing RNNs and feedforward networks, taking time series as inputs. An
architecture is defined by its sequence of layers, where each layer depends on
a set of parameters. A feedforward layer is fully specified by its number of
neurons, weights, and activation function. The same elements are also used for
defining recurrent layers, that, contrary to feedforward networks, also have a
hidden state. This hidden state
$\mathrm{h}_{t}=f(\mathrm{h}_{t-1},\mathrm{x}_{t};\theta)$, where
$\mathrm{h}_{0}$ is a constant and $\theta$ are the parameters of the network,
can be used by the network to store past information of the time series. A
syntax enabling one to easily specify such complex ANN architectures was
proposed by [16, Vecoven et al., 2020, Appendix 3]. With this syntax, a layer
is represented by an acronym referring to the type of neurons it is made of.
This acronym is followed by the symbol $\circlearrowright$ when it is a
recurrent layer. The acronym also takes as subscript the number of neurons in
a layer. A $\rightarrow$ symbol is used to highlight the connections between
the different layers, e.g.:

$LSTM_{10}\\!\\!\,\circlearrowright\>\rightarrow
LSTM_{10}\\!\\!\,\circlearrowright\>\rightarrow sigm_{1},$ (4.1)

defines a three-layer neural network. The first two are recurrent layers
having 10 Long-Short Term Memory (LSTM, [17, Hochreiter et al., 1997]) cells
each. The last one is a single neuron using a sigmoid activation function.
Using this syntax, we can provide the generic architecture that implicitly
defines the hypothesis space $\mathcal{H}$ that will be used in our
experiments:

$Rec_{N}\\!\\!\,\circlearrowright\>\rightarrow
Rec_{M}\\!\\!\,\circlearrowright\>\rightarrow sigm_{1},$ (4.2)

where $Rec_{N}\\!\\!\,\circlearrowright$ denotes a $N$-neuron recurrent layer
and $Rec$ corresponds to either a Gated Recurrent Unit (GRU, [18, Cho et al.,
2014]), LSTM, or neuromodulated Bistable Recurrent Cell (nBRC, [19, Vecoven et
al., 2020]).

###  4.2 Optimization procedure

To find the best function $h_{t}$ at time $t$ among the hypothesis space, the
machine learning algorithm aims at minimizing an empirical loss function
$L_{t}$ over a learning data set $LS_{t}$. This learning data set is composed
of pairs of the type $(i_{t^{\prime}}^{p},y_{t^{\prime}}^{p})$ where
$i_{t^{\prime}}^{p}$ is the history trajectory of an eligible player $p$
available at time $t^{\prime}$ and $y_{t^{\prime}}^{p}$ is the churn indicator
for player $p$ at time $t^{\prime}$. The loss function is based on Equation
3.3 and is equal to the averaged sum of the binary entropy losses computed
over the training set, namely:

$L_{t}(LS_{t})=\frac{1}{|LS_{t}|}\sum_{p=1}^{P}\sum_{t^{\prime}=\max(T_{0},t_{0}^{p})}^{t-T_{pred}}H(y_{t^{\prime}}^{p},h_{t}(i^{p}_{t^{\prime}})).$
(4.3)

A few remarks ought to be made concerning Equation 4.3. First, the upper bound
set for $t^{\prime}$ comes from Equation 3.2, that defines the churn
indicator. This equation stresses that, knowing only $i_{t}$ when learning
$h_{t}$, churn indicators can be computed for $t^{\prime}<=t-T_{pred}$ at
most. Also, we faced a trade-off when choosing the lower bound of
$t^{\prime}$. If we had set it to $t-T_{pred}$, we would have used only the
most recent information, but we would probably have missed some relevant data
contained in older pairs. The other way around, nothing prevents us from
fixing this lower bound to 0. The reason for not doing this is twofold: (i) it
may happen that, at the beginning of the trajectories, very little information
is available to carry out a good prediction, which, by selecting data pairs
for very small values of $t^{\prime}$, would only correspond to adding noise
to the loss function (ii) we prefer having in the training set pairs which
correspond to churn indicators that are not too old. Indeed, even though
Equation 4.3 implicitly assumes that churn patterns are somehow stationary
with time, nothing prevents customers from modifying their behaviour regarding
churn. For example, we could imagine the release of a much more addictive
game. The customers playing that game would then be very likely to have their
churn patterns modified. Finally, this leads us to define two parameters: a
constant $T_{0}$, which is not player dependent, and a constant per player
$t_{0}^{p}$, whose maximum corresponds to the lowest time step at which player
$p$’s data are considered for building $LS$. According to these remarks,
player $p$ intervenes through $t-T_{pred}-\max(T_{0},t_{0}^{p})+1$ terms in
the loss function. The value of $T_{0}$ will depend on the conducted
experiments. Additionally, in our simulations, we will restrict the size of
the time series to its last $T_{h}$ elements.

The optimization procedure enabling the selection the hypothesis function
$h_{t}$ at time $t$ is formulated as follows:

$h_{t}\in\underset{h_{t}\in\mathcal{H}}{\text{argmin}}~{}L_{t}(LS_{t}).$ (4.4)

Finding $h_{t}$ is thus equivalent to choosing the point of the hypothesis
space minimizing $L_{t}$ for the training data available at time $t$. In the
scope of ANN, the problem defined by Equation 4.4 is most commonly solved
using gradient descent algorithms. Such algorithms consist of approximating
the gradient of a given function and iteratively following the opposite
direction of the gradient in order to find a local minimum of that function.
Over the past years, several variants of this algorithm were designed in order
to fit different ANN architectures. Among these, we chose RMSprop for this
work. It was introduced in two steps by [20, Hinton et al, 2012]. They first
discussed the variability of the magnitude of the gradient in function of the
weights of the DNN and of the progression of the learning to introduce the
Rprop algorithm. Then, it was adapted in order to be compatible with mini-
batch learning. As any gradient descent algorithm, RMSprop can be tuned
according to several parameters: mini-batch size $size_{batch}$, number of
epochs $n_{epochs}$ and learning rate $\alpha$. It also has a specific
parameter $\rho$, the decay factor for the gradient. Each parameter is
specified for every experiment, and some of them will be discussed in Section
5. Finally, we note that, for solving the optimization problem 4.4 in our
experiments, we unroll the time-series over all its selected elements for
computing the gradient.

##  5 Experiments

In this section, we test our proposed approach on data from an online gaming
company, Gaming1. In this company, each transactional activity is recorded
into a database, along with the exact time at which it occurred. For this
work, we choose a time step of 1 day. As a result, each time-series element
consists of a multidimensional vector aggregating all considered features over
a whole day. These features, which make up the sets $X^{p}$ and $X^{p}_{c}$,
are listed in Table 5.1 †††The Gross Gaming Revenue (GGR) denotes the revenue
earned by the company from the player’s activity. In practice, it is computed
by subtracting the winnings from the stakes for each player.. The number of
website connections correspond to the number of times the player logged into
the website of the company, independently of their subsequent activity.

Type | Feature | Belongs to the set(s)
---|---|---
Casino | Number of casino plays |  $X^{p}$ and $X^{p}_{c}$
Casino | Casino Gross Gaming Revenue (GGR) | $X^{p}$
Sport betting | Number of sport betting tickets |  $X^{p}$ and $X^{p}_{c}$
Sport betting | Sport betting GGR | $X^{p}$
Transactions | Number of deposits |  $X^{p}$ and $X^{p}_{c}$
Transactions | Number of withdrawals | $X^{p}$
Logging | Number of website connections | $X^{p}$
Retention | Number of days since last active day | $X^{p}$
Table 5.1: Features aggregated from transactional logs

In this company, the retention services identify a customer as a churner when
the number of days since his last active day is least equal to $T_{c}=35$. A
day is considered as active when the player performs at least one gaming
session, sport betting ticket, or deposit within this day. Concerning churn
prediction, Gaming1 wants to assess their customers’ churn probability within
the next $T_{pred}=30$ days. This duration has been chosen to allow retention
campaigns to be scheduled sufficiently in advance. Moreover, we set the
minimum time step per player $t_{0}^{p}$ to the $60^{\text{th}}$ day after
player $p$’s registration date for building the training set. Before this day,
the rules used by Gaming1 for retention are different. Therefore, we have also
chosen to use the last $T_{h}=60$ elements of player $p$’s time series when
collecting data pairs $(i_{t^{\prime}}^{p},y_{t^{\prime}}^{p})$. Table A.1
gathers the values of these parameters, which remain the same for all
conducted experiments. The collected data comprise the activity history of
$10~{}000$ gambling players from the $1^{\text{st}}$ of January 2017 to the
$1^{\text{st}}$ of June 2021. For players registered after $1^{\text{st}}$ of
January 2017, the missing parts of the time-series are filled with 0.

We now explain how we build our training set $LT_{t}$ at time $t$ from the
collected players’ histories. This will be composed of all the pairs
$(i_{t^{\prime}}^{p},y_{t^{\prime}}^{p})$ that can defined at time $t$ and
that also satisfy the following conditions, that we call activity conditions:

  1. (i)

$t^{\prime}\geq\max(T_{0},t_{0}^{p})$.

  2. (ii)

player $p$ must have at least 1 active day within the 30 days that precede
$t^{\prime}$.

We note that, from these conditions, we can derive an upper bound on the
number of samples related to some player $p$ used in $LS_{t}$, which is equal
to $t-T_{pred}-\max(T_{0},t_{0}^{p})+1$. As way of summary, Figure 5.1
displays a timeline explaining which data pairs are used to build $LS_{t}$ for
some player $p$. More specifically, the black-coloured block corresponds to
the time steps $t^{\prime}$ that feed the learning set $LS_{t}$ with pairs
$(i_{t^{\prime}}^{p},y_{t^{\prime}}^{p})$.

Figure 5.1: Timeline displaying the time steps $t^{\prime}$ to which
correspond pairs $(i_{t^{\prime}}^{p},y_{t^{\prime}}^{p})$ used for building
the training set at time $t$.

###  5.1 Standard experiment

In this section, we use the learning set $LS_{t}$ previously characterized for
computing $h_{t}$ with $t$ corresponding to the $1^{\text{st}}$ of January
2020. This model $h_{t}$ will be used for forecasting whether some player $p$
will churn in the 30 days following $t$, or equivalently to predict the value
of $y^{p}_{2020/01/01}$. We note that players for which this evaluation will
take place at time $t$, are those who have registered for at least 60 days and
have at least one active day in the 30 days preceding $t$, two conditions that
are similar to the activity conditions introduced before. We obtain a test set
$TS_{2020/01/01}$ containing $1~{}924$ samples, among which $\sim 22$% have a
value of $y_{t}^{p}$ equal to 1. Those correspond to a churn event. The rest,
having a value $y_{t}^{p}$ of 0, is associated with a retained event.
Concerning the training data, $LT_{2020/01/01}$ is composed of $793~{}779$
elements. Among these pairs, $\sim 22$% (resp. $\sim 78$%) are churn (resp.
retained) samples. We use the precision and recall of the churn class, along
with the accuracy, to evaluate the performances of the model. These metrics
are defined as:

$\text{precision}=\frac{\text{TP}}{\text{TP}+\text{FP}},$ (5.1)
$\text{recall}=\frac{\text{TP}}{\text{TP}+\text{FN}},$ (5.2)
$\text{accuracy}=\frac{\text{TP}+\text{TN}}{\text{TP}+\text{FP}+\text{TN}+\text{FN}},$
(5.3)

where TP is the number of true positives, FP the number of false positives, TN
the number of true negatives, and FN the number of false negatives. Precision
corresponds to the rate of correct predictions for the number of customers
predicted as churners, while the recall denotes the rate of churn customers
that are identified by the model. Results have been generated based on three
random seeds and can be found in Table 5.2. We note that the accuracy is
almost identical for the three types of RNN. The choice of the architecture
depends instead on the objective we want to achieve in retention. When we need
cost-effective campaigns, the architecture using nBRC layers is the best
choice, as it obtains the best precision value for the churn event class. In
contrast, LSTM layers allow for the detection of the highest number of churn
customers by getting the best recall value for the churn class. When all
metrics are equally important, GRU layers prevail. Indeed, it gets the best
accuracy score and the second-best value for the other metrics. For further
experiments, we decide to use the LSTM layers.

Layer | Precision - Churn class | Recall - churn class | Accuracy
---|---|---|---
GRU | 0.636212 ± 0.001635 | 0.574519 ± 0.037549 | 0.836972 ± 0.003461
LSTM | 0.616472 ± 0.008382 | 0.611378 ± 0.061255 | 0.833853 ± 0.007375
nBRC | 0.644099 ± 0.006254 | 0.485577 ± 0.025440 | 0.830735 ± 0.002456
Table 5.2: Performances of the $h_{2020/01/01}$, for different RNN layers and
random seeds. The complete list of parameters is available in Table A.2.
Figure 5.2: Performance metrics of $h_{2020/01/01}$, using LSTM layer. The
metrics are computed for different values of $T_{0}$, leading to different
training sets, and different random seeds. The complete list of parameters is
available in Table A.3

###  5.2 Influence of $T_{0}$

In this section, we want to study the impact of a variation of the value of
the $T_{0}$ parameter on the performances of the model trained in the previous
experiment. This parameter was introduced in Section 4.2 and influences the
number of samples that we can use per player to build the training data set.
Lower values of $T_{0}$ lead to a data set with many samples, with the risk
that some of them may be related to past situations not relevant for churn
prediction at time $t$. In contrast, later values result in smaller data sets
but with more-recent samples. With this experiment, we want to determine
whether it is better to use a greater amount of data, or more-recent ones, to
solve the problem stated in Section 3. The different values of the parameters
used for this experiment can be found in Table A.3. The results of the
experiment are displayed in Figure 5.2 for the different values of $T_{0}$.
Globally, we observe that $T_{0}$ has a relatively small impact on the
precision and accuracy curves. However, we observe a progressive drop of the
churn recall for values of $T_{0}$ greater than January 2018. This means using
only recent data for building the training set decreases the number of
churners detected by the model. This experiment suggests we are in a context
where using only the most recent data is not favourable and that relatively
low values of $T_{0}$ can be used.

###  5.3 Model robustness over time

In this last experiment, we want to evaluate the robustness over time of the
model trained in the experiment detailed in Section 5.1. To do so, we compute
the evolution of the performances of the model trained using
$LS_{2020/01/01}$, i.e. $h_{2020/01/01}$, on several testing sets
$TS_{t^{\prime}}$, with $t^{\prime}$ greater or equal to the $1^{\text{st}}$
of January 2020. Then, for each $t^{\prime}$, we train another model
$h_{t^{\prime}}$ using the set $LT_{t^{\prime}}$, that is built using all the
data pairs satisfying at $t^{\prime}$ the activity conditions mentioned in the
introduction paragraph of Section 5. Using the same metrics as in previous
experiments, we evaluate the performances of $h_{t^{\prime}}$ at
$TS_{t^{\prime}}$, for each chosen $t^{\prime}$. We select the different
values of $t^{\prime}$ to uniformly cover the period going from the
$1^{\text{st}}$ of January 2020, to the $1^{\text{st}}$ of January 2021, with
a time step of two months between each $t^{\prime}$. The parameters used in
the experiment are reported in Table A.4. The random seeds are the same as the
previous two experiments. Figure 5.3 display the values of the different
metrics for the different test dates. While both models offer similar
performances, we may observe significant variations over time, for example, in
the way the recall metric evolves. We conjecture that those could be related
to specific major events experienced by the company. For instance, on the
$1^{\text{st}}$ of March 2020, the company stopped offering several types of
bonus for legal reasons. This may explain the strong discontinuity observed at
that time. In particular, we note that the sharp rise in precision observed at
that time might be a side effect of the recall drop. Indeed, such trends are
probably due to the model making many fewer churn guesses than those that
indeed occur at other time steps. As a consequence, only the most certain
churn events are effectively predicted as they are by the model, causing these
variations in recall and precision. Similarly, at the end of June 2020, a
technical issue made the casino unreachable for several days. In both cases,
the company faced events that most likely impacted their players’ activity
patterns. As a consequence, the past histories used to train the model were
not relevant to predict churn events for these particular months.

Figure 5.3: Performance metrics of the model learned on the $1^{\text{st}}$ of
January 2020, using LSTM layer for several $LS_{t^{\prime}}$ after this date.
The metrics are computed for different random seeds and compared to those
obtained by models trained using $LS_{t^{\prime}}$. The complete list of
parameters is available in Table A.4

##  6 Conclusion and perspectives

As a conclusion, this study shows how to formalise the churn prediction
problem and how recurrent neural networks can be used to solve it. Given a
sufficient volume of data, we demonstrate that RNNs can achieve solid
performances. Indeed, even though the occurrence of churn is relatively rare
in the data used in the different experiments, the models still get honourable
results at detecting this event. Moreover, by comparing the performances
obtained in Section 5.1, we observe that the choice of the architecture could
change according to the desired objectives of the retention campaigns. If we
want to conduct cost-effective retention campaigns, nBRC is probably the best
option, as it had the highest precision values for the churn class. When the
top priority is to detect as many churn customers as possible, we would
probably go for LSTM architectures as these the best recall for the churn
events. Then, Section 5.2 shows that using only more-recent data to train the
model can decrease the performances. In particular, the recall of the churn
events starts to drop when the size of the time window used for training
become lower than one year. Furthermore, with the experiment conducted in
Section 5.3, we concluded that, in our given context, a model could sustain
competitive performances over time compared to other models trained at
posterior time steps. However, we also highlighted some non-negligible
sensitivity to one-off events that can impact the players’ activity patterns
for a short period of time. Eventually, there are still possibilities for
future research work in this domain. The room for improvement in the detection
of churn events is significant because of the rarity of such events.
Techniques based on anomaly detection would be an interesting option to tackle
this problem from another point of view. As an example, the research paper
[21] challenged one class SVM to credit card customers churn data set, showing
it is possible to obtain decent results using anomaly detection methods in
this domain. Also, a gambling company does not necessarily have the same
across-the-board reason for retaining all its churning customers, as some may
be more profitable than others. Combining the player value with the churn
probability could help to increase the return on investment of retention
campaigns.

### Acknowledgements.

The authors would like to thank Gaming1 for providing the data used in the
experiments of this paper, and Gaming1 data science team for their support and
counsel for this work.

## References

  * [1] Eva Ascarza, Raghuram Iyengar, and Martin Schleicher.  The perils of proactive churn prevention using plan recommendations: Evidence from a field experiment.  Journal of Marketing Research, 53(1):46–60, 2016.
  * [2] Chih-Ping Wei and I-Tang Chiu.  Turning telecommunications call details to churn prediction: a data mining approach.  Expert systems with applications, 23(2):103–112, 2002.
  * [3] Yann LeCun, Yoshua Bengio, et al.  Convolutional networks for images, speech, and time series.  The handbook of brain theory and neural networks, 3361(10):1995, 1995.
  * [4] G Peter Zhang.  Time series forecasting using a hybrid arima and neural network model.  Neurocomputing, 50:159–175, 2003.
  * [5] Jerome T Connor, R Douglas Martin, and Les E Atlas.  Recurrent neural networks and robust time series prediction.  IEEE transactions on neural networks, 5(2):240–254, 1994.
  * [6] Alex Graves.  Generating sequences with recurrent neural networks.  arXiv preprint arXiv:1308.0850, 2013.
  * [7] Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu.  Recurrent neural networks for multivariate time series with missing values.  Scientific reports, 8(1):1–12, 2018.
  * [8] Piew Datta, Brij Masand, Deepak R Mani, and Bin Li.  Automated cellular modeling and prediction on a large scale.  Artificial Intelligence Review, 14(6):485–502, 2000.
  * [9] Wouter Buckinx and Dirk Van den Poel.  Customer base analysis: partial defection of behaviourally loyal clients in a non-contractual fmcg retail setting.  European journal of operational research, 164(1):252–268, 2005\.
  * [10] Yuangen Lai and Jianxun Zeng.  Analysis of customer churn behavior in digital libraries.  Program, 48(4):370–382, 2014.
  * [11] Kristof Coussement and Koen W De Bock.  Customer churn prediction in the online gambling industry: The beneficial effect of ensemble learning.  Journal of Business Research, 66(9):1629–1636, 2013.
  * [12] Eunju Suh and Matt Alhaery.  Customer retention: Reducing online casino player churn through the application of predictive modeling.  UNLV Gaming Research & Review Journal, 20(2):63–84, 2016.
  * [13] Yaya Xie, Xiu Li, EWT Ngai, and Weiyun Ying.  Customer churn prediction using improved balanced random forests.  Expert Systems with Applications, 36(3):5445–5449, 2009.
  * [14] Chih-Fong Tsai and Yu-Hsin Lu.  Customer churn prediction by hybrid neural networks.  Expert Systems with Applications, 36(10):12547–12553, 2009.
  * [15] Mohammed Abdul Haque Farquad, Vadlamani Ravi, and S Bapi Raju.  Churn prediction using comprehensible support vector machine: An analytical crm application.  Applied Soft Computing, 19:31–40, 2014.
  * [16] Nicolas Vecoven, Damien Ernst, Antoine Wehenkel, and Guillaume Drion.  Introducing neuromodulation in deep neural networks to learn adaptive behaviours.  PloS one, 15(1):e0227922, 2020.
  * [17] Sepp Hochreiter and Jürgen Schmidhuber.  Long short-term memory.  Neural computation, 9(8):1735–1780, 1997.
  * [18] Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio.  Learning phrase representations using RNN encoder-decoder for statistical machine translation.  arXiv preprint arXiv:1406.1078, 2014.
  * [19] Nicolas Vecoven, Damien Ernst, and Guillaume Drion.  A bio-inspired bistable recurrent cell allows for long-lasting memory.  arXiv preprint arXiv:2006.05252, 2020.
  * [20] Geoffrey Hinton, Nitish Srivastava, and Kevin Swersky.  Neural networks for machine learning lecture 6a overview of mini-batch gradient descent.  Cited on, 14(8), 2012.
  * [21] G Ganesh Sundarkumar, Vadlamani Ravi, and V Siddeshwar.  One-class support vector machine based undersampling: Application to churn prediction and insurance fraud detection.  In 2015 IEEE International Conference on Computational Intelligence and Computing Research (ICCIC), pages 1–7. IEEE, 2015.

##  Appendix A Parameter values

Parameter | Description | Value
---|---|---
$T_{c}$ | Number of inactive days for churn | 35
$T_{pred}$ | Churn prediction period | 30
$T_{h}$ | Size of input time series in days | 60
$t_{0}^{p}$ |  Minimum time step for some player $p$’s data pair to belong to $LS$ |  60 days after player $p$’s registration date
Table A.1: Global parameters for all experiments Parameter | Description | Value
---|---|---
$N$ | Number of neurons of the first recurrent layer | 128
$M$ | Number of neurons of the second recurrent layer | 64
$T_{0}$ | Start of the time interval used to build $LS$ | 2017-01-01
date | Date of the experiment | 2020-01-01
$n_{epochs}$ | Number of epochs per training procedure | 20
$size_{batch}$ | Size of sample batch for each training step | 256
$\alpha$ | Learning rate of the optimizer used in the learning procedure | $10^{-3}$
$\gamma$ | Decay factor of the gradient | 0.9
Table A.2: Parameter values for the standard experiment 5.1 Parameter | Description | Value
---|---|---
$N$ | Number of neurons of the first recurrent layer | 128
$M$ | Number of neurons of the second recurrent layer | 64
RNN | Type of recurrent layer | LSTM
date | Date of the experiment | 2020-01-01
$n_{epochs}$ | Number of epochs per training procedure | 20
$size_{batch}$ | Size of sample batch for each training step | 256
$\alpha$ | Learning rate of the optimizer used in the learning procedure | $10^{-3}$
$\gamma$ | Decay factor of the gradient | 0.9
Table A.3: Parameter values for the experiment 5.2 studying the influence of $T_{0}$ Parameter | Description | Value
---|---|---
$N$ | Number of neurons of the first recurrent layer | 128
$M$ | Number of neurons of the second recurrent layer | 64
RNN | Type of recurrent layer | LSTM
$T_{0}$ | Start of the time interval used to build $LS$ | 2017-01-01
$n_{epochs}$ | Number of epochs per training procedure | 20
$size_{batch}$ | Size of sample batch for each training step | 256
$\alpha$ | Learning rate of the optimizer used in the learning procedure | $10^{-3}$
$\gamma$ | Decay factor of the gradient | 0.9
Table A.4: Parameter values for the time robustness experiment 5.3