# To Whom are You Talking? A Deep Learning Model to Endow Social Robots with
Addressee Estimation Skills
††thanks: For research leading to these results Carlo Mazzola has received
funding from the project titled TERAIS in the framework of the program
Horizon-Widera-2021 of the European Union under the Grant agreement number
101079338; Marta Romeo from the UKRI Node on Trust (EP/V026682/1); Angelo
Cangelosi from the EU projects TRAINCREASE and MUSAE, and the US project
THRIVE++.

1st Carlo Mazzola  CONTACT Unit
Istituto Italiano di Tecnologia
DIBRIS, University of Genoa
Genoa, Italy
ORCID 0000-0002-9282-9873  2nd Marta Romeo  School of Mathematical
and Computer Sciences
Heriot-Watt University
Edinburgh, United Kingdom
ORCID 0000-0003-4438-0255  3rd Francesco Rea  4th Alessandra Sciutti  CONTACT
Unit
Istituto Italiano di Tecnologia
Genoa, Italy
ORCID 0000-0001-8535-223X  CONTACT Unit
Istituto Italiano di Tecnologia
Genoa, Italy
ORCID 0000-0002-1056-3398  5th Angelo Cangelosi  Manchester Centre for
Robotics and AI
University of Manchester
Manchester, United Kingdom
ORCID 0000-0002-4709-2243

###### Abstract

Communicating shapes our social word. For a robot to be considered social and
being consequently integrated in our social environment it is fundamental to
understand some of the dynamics that rule human-human communication. In this
work, we tackle the problem of Addressee Estimation, the ability to understand
an utterance’s addressee, by interpreting and exploiting non-verbal bodily
cues from the speaker. We do so by implementing an hybrid deep learning model
composed of convolutional layers and LSTM cells taking as input images
portraying the face of the speaker and 2D vectors of the speaker’s body
posture. Our implementation choices were guided by the aim to develop a model
that could be deployed on social robots and be efficient in ecological
scenarios. We demonstrate that our model is able to solve the Addressee
Estimation problem in terms of addressee localisation in space, from a robot
ego-centric point of view.

###### Index Terms:

Addressee Estimation, Deep learning, Social Robot, Human activity recognition,
Human-robot interaction

##  I Introduction

Communicating means sharing, might it be a message, a thought, or an inner
state, and is an act that inherently shapes the social world. To properly be
part of the social environment, each agent needs to understand some basic
dynamics of communication, such as to whom a message is directed. Therefore, a
crucial element is understanding who the speaker and who the addressee/es are.
This ability is even more crucial for social robots in situations that go
beyond the mere dyadic interactions. For instance, understanding others’
addressee in a group dynamics could help robots to discern implicitly
expressed robot-directed commands [1], the social dynamics and roles in
multiparty interactions [2], and the correct meaning of sentences that
contains deictic expressions (you, he, she, they…) [3].

The scope of the present study is to implement a model for Addressee
Estimation to enhance Human-Robot Interaction (HRI). Addressee Estimation is
the capability to detect the addressee of a user’s utterance [4]. Endowed with
such skill, robots would be able to estimate _the addressee: to whom an agent
is addressing its message_ [5]. Behavioral studies demonstrated that the human
expression of communicative aspects related to Addressee Estimation (i.e.,
turn yielding and turn taking) involves verbal, para-verbal and non-verbal
channels [4]. Specifically it was proven that, for Addressee Estimation, the
speaker’s bodily cues, such as gaze and gestures, allow listeners to better
understand the speaker’s intentions [6, 7]. Our approach in developing our
Addressee Estimation model for social robots was inspired by these findings.
Hence, we conceive Addressee Estimation as _the ability to understand an
utterance’s addressee by interpreting and exploiting non-verbal/bodily cues
from the speaker._

##  II Related Works

Previous studies have often dealt with the problem of Addressee Estimation by
using a multimodal approach. Jovanovic et al. [8] used an ad-hoc retrieved
dataset, gathered on meetings of groups of 4 humans, to train a Bayesian
Network and Naïve Bayes Classifiers with contextual, lexical, and gaze
features and solve the task as a classification of whom, among the four agents
(or the group), was the addressee of each utterance.

Using the AMI corpus [9], containing data from 100 hours of meetings, Op den
Akker et al. [10] treated Addressee Estimation as a binary problem from the
perspective of each agent and trained different classifiers with data about
the speakers’ focus of attention, dialogue acts, and contextual information.
Using the same dataset, Malik et al. [11] selected several features (textual,
contextual, and focus of attention) to classify the role of the addressee
instead of their identity. In this way, they overcame some limitations in
terms of reducing Addressee Estimation to a mere binary problem. In a later
study, the same authors used similar features on the MULTISIMO Corpus, a
multiparty multimodal dataset involving meetings of 3 participants [12]. In
this work, the authors trained different machine learning and deep learning
algorithms to improve their previous results and develop a real-time Addressee
Estimation model, without information about previous addressees.

In embodied artificial agents, Addressee Estimation models have started to be
developed to go beyond the dyadic and robot-centric structure of HRI [13, 14]
and enable robots to interact in ecological scenarios [15]. However, research
on Addressee Estimation, which directly involved artificial conversational
systems and robots, mostly solved the problem as a binary classification. Bakx
et al. [1] conducted multiparty experiments with two humans at an information
kiosk. By recording participants with an external camera, they used a rule-
based approach to classify whether the participant at the information kiosk
was addressing the system or the human partner, given its focus of attention
and the length of the utterance. Operating with the same scenario, Turnhout et
al. [16] trained a Naïve Bayes Classifier to solve the same task. Katzenmaier
et al. [17] designed a multiparty interaction with two humans (host and guest)
and a simulated robot. They approached the task as a binary classification
(host speaking either to the robot or to the guest) using visual data
(automatically extracted head pose) and speech data. In a multiparty HRI
scenario, Richter et al. [18] opted for a rule-based model taking as input the
human’s lips movement and the mutual gaze between the human and the robot to
understand if an utterance was addressed to the robot or not. After a dataset
collection of multi-user human-virtual agent interaction, Huang et al. [19]
trained an SVM classifier for a binary classification (robot addressed or
human partner addressed) by giving as input several features related to
prosody, utterance length, and head direction and equipped the virtual agent
with a model for real-time Addressee Estimation. The work of Sheikhi et al.
[14] relied on the role contextual information plays in Addressee Estimation.
The authors used context about the utterance, the agents involved in the
interaction, and the objects of interest in the environment to extract
information about the speaker’s and the human partner’s visual focus of
attention to train a model to predict the addressee of each utterance in a
binary classification task.

Addressee Estimation has also been connected to other social communication
problems. Johansson et al. [13] combined turn-taking and addressee detection
and used only automatically extracted features to solve the task as binary
classification and as a linear regression problem (gradual opportunity to take
the turn). Horiguchi et al. [15] trained a LSTM neural network combined with
Logistic Regression with features related to the speaker’s face, audio and
text to solve the problem of response obligation detection. Romeo et al. [20]
implemented a CNN model on a Pepper robot to endow it with the ability to
predict the other agent’s intention to interact using only visual information.

To develop an Addressee Estimation model that could be deployed in social
robots, the approach of our work is to design and train a hybrid deep-neural
network (CNN+LSTM) to interpret the non-verbal behavior of the speaker as a
cue to localize the addressee in the HRI space. The idea is to provide the
robot with the ability to read the speaker’s intentions only with the visual
information on their behaviour. This work considers Addressee Estimation with
the idea of enabling multiparty natural HRI. For this reason, with respect to
previous models implemented on robots, our work answers to three needs: going
beyond the mere binary prediction (robot being addressed or not), using only
data from the robot’s sensors, and taking into account more realistic and
ecological scenarios.

##  III Methods

The core objective of this study is to develop a model for Addressee
Estimation in terms of addressee localisation. We approached the task as a
three-class classification of the addressee position starting from the
speaker’s non-verbal behaviour and we designed a deep neural network composed
of two parts: a CNN and an LSTM network. As Skatnze [4] showed, bodily
information can be crucial to estimate the addressee of an utterance. Based on
such evidences, the model developed in this work leverage two types of visual
data collected from the speaker: 1) Images of the speakers’ face, to retrieve
information about their head direction and visual focus. 2) Vectors
representing the speakers’ body pose, as a proxy of their focus of attention
but also as a way to encapsulate the body language and gestures.

###  III-A The dataset

We decided to use the Vernissage Corpus [21, 22] to train and test our model
for Addressee Estimation. Vernissage is a synchronised multimodal corpus of
multiparty interactions in which two humans converse with a Nao robot
(Aldebaran, United Robotics Group). During the interaction, the robot asks
participants to present themselves to the group, shows them some paintings on
the walls of the room, and asks them some questions about the paintings.
Participants are given time to discuss among themselves before answering the
robot, and are left free to comment with their peer on the situation. The aim
of the corpus was to leave participants as free as possible during the
interaction. All recordings take place in the same room, where participants
are not required to keep a specific absolute position, although most of the
time, because of the configuration of the room and of the interaction, they
are standing in front of Nao and hold a relative position to each other: one
on the left, the other on the right side (see Figure 1 for some examples).
Dialogues about the paintings on the walls ensure the presence of a more
complex and natural interaction. More specifically, triadic interactions (two
agents and a target object) occur when the speaker talks to the addressee
about something in the environment, might it be an object or another agent.
This situation, which is typical of social interactions, modifies the gaze
pattern and the body posture of both agents so that they not only look at each
other but also at the target object [4]. In the Vernissage Corpus,
participants focus their attention on the paintings and even get closer to see
them better when describing them. Therefore, although the dialogue is
controlled by the questions of the robot, the scenario grants sufficient
flexibility and spontaneity to human behavior, as it normally happens in
ecological scenarios.

Figure 1: Illustrative frames from Vernissage Dataset. Examples of multiparty
HRI data recorded from the Nao robot’s cameras. Some pictures show blurred
faces for privacy reasons.

Figure 2: Illustration of a sequence. Aggregation of frames in a sequence of
0.8s and extraction of body poses and face images

Yet another positive feature of the Vernissage Corpus is that the authors
recorded the interactions from the cameras of the Nao robot, a single camera
installed in the head of the robo with a resolution of 640x480 pixels at a
frequency of about 15 fps (mean), and YUV422 color mode. The idea guiding the
development of our model was to be able to carry out Addressee Estimation in
unconstrained interaction settings. With this objective in mind, we chose to
focus on data that could be automatically extracted through the robot’s
sensors without the need for any external device. This way, the model could be
trained with data recorded from the ego-centric perspective of a robot,
extremely important for portability. As a side effect, data result noisier
because of the movements made by the robot (e.g., nodding or turning its
head). However, this could also represent an advantage while training a model
suitable for ecological scenarios.

The Vernissage Corpus is manually annotated to have a Ground Truth about
addressees. More specifically, the addressee is annotated as the target person
of a speech utterance. Five different labels are used: “ROBOT”, “RIGHT”,
“LEFT”, “GROUP”, and “NOLABEL”. “RIGHT” and “LEFT” refer to the person at the
right and the left of the robot, “GROUP” means that both the robot and the
other agent are addressed, whereas “NOLABEL” indicates a silent time interval.

###  III-B Features selection and pre-processing data pipeline

The data chunk on which our neural network for Addressee Estimation has been
trained is a sequence of 10 frames of face images and vectors representing
body poses. The process to obtain the chunks from the Vernissage Corpus
involved the following five steps:

#### Division in utterances

The dataset comprises recordings of 10 interactions between two humans
(different for each interaction) and the Nao robot. Video clips were trimmed
according to the speech detection annotations and extracted from the
recordings of the 10 interactions, leaving out the frames labeled “silence”.
Therefore, utterances were considered as the time intervals in which an agent
continuously speaks without being interrupted by silence pauses longer than
0.08s.

Figure 3: Illustration of an utterance. The utterance is partitioned into
sequences of 0.8s. Utterances were defined as speech intervals addressed to
the same addressee and delimited by silence. Each utterance comprised at least
one sequence.

#### Extraction of body poses and face images

Firstly, utterances were divided into frames of 0.08s. Then, using OpenPose
[23], vectors of 2D coordinates for body poses were extracted for each frame
for all participants. The OpenPose COCO body format was adopted, which
predicts the x and y coordinates for 18 key points of each person (5 for the
head, 3 for each limb, and 1 for the torso). Coordinates ranged from -1 to 1
for both axes. Given the poses, from the five coordinates of each person’s
head-key points, a square-size cropped face image was obtained by the original
frame, resized at 50x50px. Body poses and face images were labeled as
“speaker” or “other”. During the interactions in the Vernissage Corpus, both
human participants play the role of “speaker” and “other”. Hence, since the
corpus comprises 10 interactions, each with two possible speakers, 20
instances of speakers were finally available.

#### Aggregation in sequences

Sequences, thereby, resulted being 0.8s portions of the utterances, consisting
of 10 body poses and 10 face images. Since the objective was to train a
network with only data from the speaker, the speaker’s sequences were saved
separately from those belonging to the other participant. All sequences were
annotated with the addressee (“ROBOT”, “LEFT”, “RIGHT”, “GROUP”). Figure 2
shows an illustrative sequence, whereas Figure 3 shows the difference between
sequences and utterances.

#### Data augmentation

We tackle the implementation of Addressee Estimation as the classification of
the addressee’s position with respect to the speaker and from the ego-centric
perspective of the robot. Therefore, we selected three labels among the ones
already annotated in the original corpus, as the addressee’s position could be
classified as “LEFT”, in case the addressee was at the left of the speaker
(from the robot’s perspective), “RIGHT”, in case the addressee was at the
speaker’s right, or “ROBOT”, in case the addressee was the robot. The
interaction scenario in Vernissage, with the Nao robot asking questions and
managing the interaction, caused an imbalanced representation of classes in
the dataset, with a prevalence of sequences labeled as “ROBOT” (addressed to
the robot). With the twofold objective of augmenting the dataset and balancing
the number of sequences, all frames labeled as “LEFT” and “RIGHT” (and
accordingly, the body poses and the face images extracted by them) were
flipped, and their label inverted. As a consequence, “LEFT” and “RIGHT” data
were doubled, and the resulting dataset was composed of 18190 speaker’s face
images and body poses partitioned in 1819 sequences: 529 for “ROBOT”, 645 for
“LEFT”, and 645 for “RIGHT”.

#### Body pose shifting

As it appears from Figure 1, participants at the left of the robot never spoke
toward a left addressee, and participants at the right never did it toward a
right one. Therefore, even though participants could mildly move, the
coordinates of their bodies could bias the prediction of their addressee. To
overcome this issue, for each sequence, the 10 body poses were shifted along
the x-axis of a random measure ranging from the two extremes of the image.

###  III-C Architecture Design

Our Addressee Estimation model is a CNN + LSTM hybrid architecture that
extracts features through convolutions and then supports learning temporal
sequential patterns through LSTM cells. Previous works related to human
activity recognition combined CNNs with an LSTM final layer. For instance,
Subramaniam et al. [24] used this combination to train a model for classifying
first impressions of personality. Romeo et al. [25] used a similar
architecture to predict apparent personality from body language cues for HRI.
Moreover, Ullah et al. [26] integrated convolutional and LSTM layers for
action recognition from videos, while Nakisa et al. [27] developed a
multimodal neural network with convolutional and LSTM layers for emotion
recognition through physiological signals.

Our model is also designed to exploit and integrate both visual input
modalities: the face images and the body pose vectors. Face images and body
pose vectors pass independently in two parallel streams of convolutional
layers. Consequently, the two embeddings received as output are concatenated
before the LSTM layer. In this way, features are extracted separately in
convolutions and then combined at a higher level of abstraction. This was
inspired by a gradual fusion of modalities at an intermediate level of the
network, which has been demonstrated beneficial [28, 29], and by the training
on joint representations of temporal sequences as, for instance, in Nakisa et
al. [27], which proved fusing streams between the convolutional and LSTM
layers being beneficial rather than a late fusion after temporal training.

Therefore, our Addressee Estimation model implements the intermediate-fusion
approach and consists of two blocks, each including two 2D convolutional
layers (the second followed by a LeakyReLU activation function) and one max-
pooling layer. The two convolutional blocks are followed by two fully
connected layers (the first followed by a LeakyReLU activation function)
providing the embeddings of the input modalities to be concatenated. The
fusion of the two streams is carried out as a simple concatenation, with the
body pose embeddings repeated 29 times so as to balance the information in the
final embedding. The sequence of the 10 fused embeddings is then passed
through the LSTM layer. Eventually, after two final fully connected layers
(the first followed by a LeakyReLU activation function), the output is given
by a LogSoftMax layer, as shown in Figure 4 and described in Table III-D.

Figure 4: Illustration of the Deep Neural Network for Addressee Estimation
employing an intermediate fusion approach (Exp. 1a). Face images and body pose
vectors are passed separately to two blocks of convolution, each including two
2D convolutional and one max-pooling layers. Then, the two embeddings
resulting from fully connected layers are concatenated and sequences of 10
fused embeddings are passed to the LSTM layer. The output is provided after
two others fully connected layers and a LogSoftMax layer. * represents
LeakyReLU activation function.

###  III-D Training Procedure and Experiment Lineup

To train and test our model, a 10-fold cross-validation was established. In
this way, the prediction of the classes (“LEFT”, “RIGHT”, “ROBOT”) could be
evaluated based on the average performance of the model when trained and
tested on different sets of data. To create the 10 different train and test
sets, the dataset of sequences derived from the pre-processing of the
Vernissage Corpus was partitioned along the 10 multiparty interactions of the
original corpus. Each interaction comprised two agents for a total of 20
speakers. The ratio to create the train sets was 9:10, with the remaining 1:10
for the test set. Accordingly, each train set included all the face image and
body pose sequences of 18 participants, whereas the test set the ones of the
remaining 2 participants. From the train sets, 90 sequences (30 for each
class) were randomly extracted, removed, and used for the validation phase
during the training in order to check the trend of the loss function.

As a first experiment (Exp 1.a), the addressee localisation model was thus
trained 10 times, one for each train set, and evaluated on as many test sets.
The model was fed with temporal sequences of data in mini-batches of 10
sequences. Each sequence included 10 face images and 10 body pose vectors. The
convolutional section of the network for face images was trained by employing
Stochastic Gradient Descent (SGD) optimizer whereas the one for body pose
vectors and the LSTM section were trained using the Adam algorithm [30]. Cross
entropy was used as the criterion to compute the loss function. The model was
trained for 50 epochs with a learning rate of 1e-3, with a decay of 0.1
(multiplicative factor) after 40 epochs. To prevent overfitting, early
stopping was implemented to stop the training after 10 trials in which the
loss function of the evaluation phase increased. The model was implemented
using PyTorch 1.12 (Python version 3.8), whereas the training was carried out
through an NVIDIA Quadro RTX 5000 with 16 GB of RAM111the code is available at
the following link
https://gitlab.iit.it/cognitiveInteraction/addressee_estimation_ijcnn23.git.

In order to test the developed intermediate-fusion model, and validate the
results from Exp 1.a, we conducted three additional experiments:

  * •

Exp. 1.b: our original model was modified to a late-fusion approach, to be
compared to our intermediate-fusion model

  * •

Exp. 1.c: our model was trained on face images only, to verify whether the two
input modalities were equally necessary

  * •

Exp. 1.d: single modality: our model was trained on body pose vectors only, to
verify whether the two input modalities were equally necessary

In Exp. 1.b, we trained the model using both visual input modalities but,
differently from the original model, we combined them after the LSTM layer.
The fusion of the two streams was carried out as a concatenation of the two
embeddings resulting from the LSTM layer. In this way, for each sequence, one
fused embedding was passed through the two final fully connected layers (the
first followed by a LeakyReLU activation function) and the LogSoftMax layer.
In Exp. 1.c and 1.d, we trained and tested two mono-stream models, the former
with face images, the latter with body pose vectors. For each modality, the
architecture of the neural network matched the late-fusion approach model
except for the last fully connected layers, which were designed for mono-
modality embeddings. Table III-D provides a description for the four
architectures of Exp. 1.a-b-c-d.

Finally, we carried out a last experiment (Exp. 2) to compare our results from
Exp 1.a with the state-of-the-art model used on Vernissage as a binary
classification, predicting whether either the robot or another user was the
addressee of an utterance [14]. Therefore, we trained and tested our
intermediate-fusion binary model to identify whether the robot was addressed
by the speaker. For this experiment, we used the same architecture employed in
Exp. 1.a, except for the last two layers (last fully connected and
LogSoftMax), changed to give binary predictions. For this reason, the previous
three-class data labeling of the addressee’s position (“LEFT”, “ROBOT”,
“RIGHT”) was replaced by a binary one: “NOT-ADDRESSED” (including data
referred to as “LEFT” and “RIGHT” ) and “ADDRESSED” (including data referred
to as “ROBOT” and “GROUP” in the original Vernissage labeling system).

TABLE I: Description of neural networks in the four three-classes experiments
(Exp. 1.a-b-c-d).

| FACE IMAGE | BODY POSE VECTOR
---|---|---
Layers | Input | Param. | Input | Param.
Layers common to all models (convolutional part of the nn)
Conv | [100,3,160,160] | k=7,s=1 | [100,1,18,3] | k=(3,1),s=1
Conv* | [100,6,154,154] | k=5,s=1 | [100,16,16,3] | k=(3,1),s=1
MPool | [100,8,150,150] | k=2,s=2 | [100,16,14,3] | k=(2,1),s=2,1
Conv | [100,8,75,75] | k=5,s=1 | [100,16,7,3] | k=(3,1),s=1
Conv* | [100,12,71,71] | k=3,s=1 | [100,32,5,3] | k=(3,1),s=1
MPool | [100,16,69,69] | k=2,s=2 | [100,32,3,3] | k=2,s=2
Flatten | [100,16,34,34] |  | [100,32,1,1] |
FC* | [100,18496] |  | [100,32] |
FC | [100,4624] |  | [100,24] |
… | … |  | … |
|  |  |  |
Layers after convolution in Exp. 1.a.: Intermediate fusion model
Concat. | [100,578] |  | [100,20] x 29 times
FC* | [10,256] |  |  |
FC | [10,128] |  |  |
LSoftm. | [10,3] |  |  |
Layers after convolution in Exp. 1.b.: Late fusion model
LSTM | [10,10,578] h_dim=512 | [10,10,20] h_dim=256
FC | [10,512] |  | [10,256] |
Concat. | [10,128] |  | [10,128] |
FC* | [10,256] |  |  |
FC | [10,128] |  |  |
LSoftm. | [10,3] |  |  |
Layers after convolution in Exp. 1.c (Face model) and 1.d (Pose model)
| Exp. 1.c.: Face Model | Exp. 1.d.: Pose Model
LSTM | [10,10,578] h_dim=512 | [10,10,20] h_dim=256
FC* | [10,256] |  | [10,256] |
FC | [10,128] |  | [10,128] |
LSoftm. | [10,3] |  | [10,3] |
| k: kernel size; s: stride; h_dim: hidden layer dimension
---
: + LeakyReLU

###  III-E Evaluation Metrics

To evaluate our model Precision, Recall, and F1-score were computed for each
class and expressed as a percentage. Results for each class were subsequently
weighted for the number of samples of each class and averaged to provide a
performance of each model in terms of Weighted F1-score. Eventually, the
results of the 10 testing from the 10-fold cross-validation were averaged to
obtain a final estimate of the model’s performances.

In all experiments, the training and testing of the model were first achieved
by keeping the 10-frames sequences as data chunks. However, since sequences
were extracted from utterances, the same metrics could be used to verify the
model’s performance in predicting the addressee of an entire utterance. The
utterance classification was computed by averaging the predictions of all the
sequences belonging to that utterance, weighted for the prediction score
provided by the LogSoftMax layer.

For Exp. 2, Precision, Recall (Sensitivity), F1-score, and Specificity (true
negative rate) were calculated considering the positive prediction as “the
robot is addresse” and expressed as a percentage. In addition to these
parameters, for a further measure of the model’s performance, an
overall-F1-score of the two classes (“ADDRESSED” vs “NOT-ADDRESSED”) was
computed as in the three-class model.

##  IV Results

In a first set of experiments (Exp. 1.a-b-c-d), we trained the Addressee
Estimation model on a three-class classification task. The four experiments
(a-b-c-d) were designed to verify the best approach for the Addressee
Estimation model: either multi-feature (and more specifically an intermediate
(1.a) of late-fusion (1.b) approach) or mono-feature (and more specifically
using the face (1.c) or body pose (1.d) information).

TABLE II: Performance of the Addressee Estimation model. Results of the 10-fold cross-validation experiments (Exp. 1.a-b-c-d) are provided in terms of mean and standard deviation of weighted F1-score. |  | considering
---
sequences
| considering
---
utterances
| considering
---
1st seq.
of each utt.
Model | avg | std | avg | std | avg | std
| Exp. 1.a
---
Intermediate Fusion
75.01 | 8.60 | 76.48 | 8.42 | 74.15 | 9.19
| Exp. 1.b
---
Late Fusion
73.18 | 7.57 | 74.19 | 7.97 | 71.88 | 5.03
| Exp. 1.c
---
Only Face
72.83 | 5.86 | 73.22 | 6.93 | 72.07 | 8.89
| Exp. 1.d
---
Only Body Pose
72.60 | 6.75 | 71.05 | 7.76 | 70.77 | 8.84

In each experiment, the model was first tested on single sequences of 0.8s,
hence without combining them in utterances (Figure 3 illustrates the
difference between sequences and utterances). In terms of weighted F1-score,
in Exp. 1.a, the model achieved 75.01% , whereas in Exp. 1.b, 1.c, and 1.d the
performance was, respectively, 73.18%, 72.83% and 72.61% (see Table II and
Figure 5).

However, Addressee Estimation is defined as the ability to understand the
addressee of an utterance. Accordingly, we tested the model considering
sequences of the same utterance together, because each utterance might
comprise several sequences. In this case, in Exp. 1.a the weighted F1-score
increased up to 76.48%, whereas in Exp. 1.b, 1.c and 1.d, the performance
measured, respectively, 74.19%, 73.22% and 71.05%.

An additional score was computed only focusing on the first sequence of each
utterance, which means measuring the model’s performance in providing a
correct prediction at 0.8s from the beginning of the utterance. Considering
the limited amount of time, the model achieved a weighted F1-score of 74.15%
in Exp. 1.a, whereas in 1.b, 1.c, and 1.d, it was, respectively, 71.88%,
72.07% and 70.77%.

Figure 5: Bar plots reporting performance of Addressee Estimation model in the
four 3-class experiments. Results of the 10-fold cross-validation experiments
(Exp. 1.a-b-c-d) are provided in terms of mean and standard deviation (error
bar) of weighted F1-scores. On the y-axis the performance score is expressed
in %.

Figure 6: Spider plot reporting performance of Addressee Estimation model for
each class. Results of the 10-fold cross-validation experiments (Exp. 1.a-b-
c-d) are provided in terms of weighted F1-score (means) for each of the 3
classes and expressed in %.

Precision, Recall, and F1-score have been computed for each class (“LEFT”,
“ROBOT”, “RIGHT”) to observe if the model performances resulting from the 4
experiments (1.a-b-c-s) were equally distributed among classes. Results are
reported in Table III and visually displayed in Figure 6.

TABLE III: Performance of Addressee Estimation models for each class. Results of the 10-fold cross-validation experiments experiments (Exp. 1.a-b-c-d) are computed considering sequences not combined in utterances Model | Class | recall | precision | F1-score
---|---|---|---|---
| LEFT | 80.72 | 79.07 | 78.61
Exp. 1.a | ROBOT | 70.09 | 72.05 | 67.52
| RIGHT | 81.76 | 84.34 | 81.75
| LEFT | 83.34 | 77.28 | 79.38
Exp. 1.b | ROBOT | 54.91 | 72.63 | 59.51
| RIGHT | 89.22 | 75.43 | 80.78
| LEFT | 78.43 | 77.58 | 77.46
Exp. 1.c | ROBOT | 62.83 | 68.59 | 61.21
| RIGHT | 80.73 | 78.11 | 78.13
| LEFT | 71.99 | 76.84 | 72.95
Exp. 1.d | ROBOT | 75.16 | 65.37 | 68.56
| RIGHT | 67.11 | 73.19 | 68.87

Our model was trained on sequences of 0.8s but utterances can be composed of
multiple sequences. For such utterances, fresh updated predictions can be
released every 0.8s, computed by averaging the prediction of each sequence.
Therefore, we could check the performance of utterances’ Addressee Estimation
as time passes. Figure 7 displays the trend of such performances in terms of
weighted F1-score in the four experiments (1.a-b-c-d). For what concerned Exp.
1.a, the weighted F1-score was 74.15% at 0.8s, 76.48% at 1.6s, 76.5% at 2.4s
and 79.8% after 2.4s. At the same time intervals, in Exp 1.b, the performance
measured respectively 71.88%, 72.03%, 75.22%, and 77.22%; in Exp. 1.c, 72.07%,
75.04%, 77.26%, and 78.25%; and in Exp 1.d, 70.77%, 70.23%, 70.44%, and
70.49%.

In Exp. 2, we trained our model to solve Addressee Estimation as a binary
classification task in the shape of the robot being addressed or not
(“ADDRESSED” vs “NOT-ADDRESSED”). Considering the test on single sequences,
our repurposed model’s Recall to the affirmative answer (robot addressed) was
73.78%, whereas Precision was 74.23%, and F1-score 72.73%. Sensitivity
achieved 80.7%. Additionally, the general performance of the model, as
measured by the overall-F1-score was 77.36% if measured on single sequences,
79.7% considering utterances, and 79.97% considering the first 0.8s of each
utterance. The performance of the model in Sheikhi et al. [14] was 76.3%
utterances correctly predicted employing a measure of the speaker’s visual
focus of attention automatically computed as input.

Figure 7: Plot reporting performance of Addressee Estimation model as a
function of the duration of the utterance. Results of the 10-fold cross-
validation experiments (Exp. 1.a-b-c-d) are provided in terms of mean of
F1-score. Performance are computed at 0.8s, 1.6s, 2.4s, and for utterances
lasting 2.4s or more. On the y-axis the performance score is expressed in %.

##  V Discussion

In this work, Addressee Estimation has been conceived as the robot’s ability
to understand an utterance’s addressee by interpreting the speaker’s bodily
cues. We tackle this problem as a three-class classification of the
addressee’s position with respect to the speaker.

###  V-A The impact of the two visual input modalities on the model
performance

To solve this task, we designed a deep-learning model with convolutional
layers and LSTM cells and we trained it with sequences of face images and body
pose vectors of the speaker. As Skantze [4] reported, information from the
speaker’s head pose, as a proxy of visual focus of attention, is highly
relevant for humans when estimating others’ addressee and beneficial when
implementing automatic Addressee Estimation models. Results from the testing
phase of our Addressee Estimation model corroborated this perspective as the
model could predict the position of the addressee.

Interestingly, the model trained with only body pose information proved to be
equally effective. It is true that also the body pose vectors contained
information about the head direction, but this was gathered only by 5 key
points: 1 for the nose, 2 for the eyes, and 2 for the ears. Compared to the
face image, the body pose presented evidence of the speaker’s whole-body
direction.

Though the difference was not substantial, the models trained with both visual
modalities (face and body pose) performed better than the single-modality
ones. This outcome was expected, as well as the fact that the intermediate-
fusion approach resulted to be more effective than the late-fusion, as
literature on the topic suggested [28, 29].

The beneficial effect of combining the two visual input modalities may be
explained by analysing the performance of each class more thoroughly. Although
the overall performance of the two single-feature models was nearly identical,
relevant differences appear when considering each class separately. As it
appears in Figure 6, the only-face model predicts with higher F1-score the
“LEFT” and “RIGHT” classes with respect to the “ROBOT” with a gap greater than
15%. Different is the case for the only-pose model, whose performance is more
stable along the three classes. What impacts more this result is the high
recall for the “ROBOT”, meaning that the model recognises the “ROBOT” more
easily from body poses than face images.

This situation seems to be reflected in the performance of the intermediate-
fusion model, which combines a high performance for the “LEFT” and “RIGHT”
classes with results more balanced for the “ROBOT”. This indicates that beyond
a general increase in performance given by the help of two channels instead of
one, relevant features for “LEFT” and “RIGHT” are provided by the face
modality, whereas for “ROBOT” by the body pose of the speaker. Interestingly,
this pattern is not shared with the late-fusion model, in which the same gap
between “LEFT”/“RIGHT” and “ROBOT” is even more evident than in the only-face
model. Accordingly, this difference between the intermediate and the late-
fusion approach suggests that fully-connected layers are not enough to
optimally balance the two modalities.

In Exp. 2, we trained the model using both visual modalities combined with an
intermediate fusion approach to solve a binary classification task and to
compare our methods with the state-of-the-art model on the Vernissage Corpus
[14]. Results of the overall-F1 score demonstrated that our approach slightly
outperformed previous model. In addition, while in [14] the prediction
required contextual information such as the possible targets of attention in
the environment and could be obtained only at the end of the utterance, our
model relies only on visual information automatically extracted from the
robot’s sensors and reliable results can be obtained just 0.8s after the start
of the utterance.

###  V-B The longer you talk, the better I estimate

Our Addressee Estimation model was developed so as to have predictions
independent from the utterance length and available less than 1s after the
utterance start. The method chosen to solve this task was to focus on
sequences of data lasting 0.8s. For each utterance, this allowed providing a
first prediction about the addressee at 0.8s, as well as other predictions
every 0.8s after that, so that the final utterance prediction was
incrementally weighted on all the predictions of sequences of that utterance.
Longer utterances are formed by a higher number of sequences, hence more ample
evidence for a correct estimate. This is what appears from results in Figure
7. The longer the speaker talks, the better the estimate of the addressee.

The intermediate-fusion, late-fusion, and only-face models share this pattern.
Conversely, this is not the case for the only-pose model. It appears,
therefore, that the two multimodal models inherit this characteristic from the
only-face model. Moreover, one may speculate that the reason underlying this
different behavior is that humans often turn their heads while speaking, in
particular, if they are referring to some objects in the environment, as is
the case in triadic interactions. The situation might be different for body
poses that, although including information about the face pose, are more
stable if considering the whole body, at least in the scenario of the
Vernissage Corpus.

###  V-C Addressee Estimation: a skill for social robots

The present model is planned to be implemented on social robots. With this
aim, we followed three general principles for its design: (i) keep the focus
on speaker’s bodily behavior, crucial to develop human-aware robots; (ii)
integrate the temporal dimension of the task; (iii) produce an ecologically
valid model, suitable for ecological scenarios.

Visual non-verbal information from the speakers’ body (face and body pose)
were used to achieve the prediction of their addressee. Our work highlights
how important non-verbal behavior is to correctly interpret the meaning and
the dynamics underlying verbal communication and advocates for making further
use of this component for developing robots as conversational agents. Non-
verbal behavior offers profound insights into other agents’ intentions.
Targeting that is a valid solution to improve robot’s skills of human
awareness, crucial to enhance natural and effective HRI.

The idea of temporality influenced the design of the classification task and
of the neural network. Firstly, temporality was conceived within each
utterance. Utterances were not considered as a whole but partitioned in
multiple time intervals of 0.8s, each of them generating a prediction. In this
way, predictions about the addressee are available just after 0.8s the start
of the utterance: a feature that reveals to be essential in case of real-time
HRI. Secondarily, temporality was conceived within each sequence, i.e., within
each time interval of 0.8s. The inputs of the neural network were not
snapshots but temporal sequences connecting 10 frames. In this way, the final
estimation does not only rely on instants but also on information related to
the temporal dimension of the sequence, fundamental to interpret human
behavior.

Figure 8: Examples of sequences wrongly predicted. The face images of four
sequences are exhibited reporting the wrong prediction given by the
intermediate-fusion model (Exp. 1.a) and the ground truth (correct addressee).

We chose the Vernissage Dataset to make the model suitable for ecological HRI
scenarios. Since our Addressee Estimation model is designed and trained to be
implemented on social robots, it was important to rely on data acquired
directly through the robot’s sensors and from its first-person perspective for
its training. Moreover, using dataset recorded in embodied interaction with a
physical robot was important because people behave differently in front of a
physical robot [31]. In this respect, the Vernissage Corpus was optimal
because it was also designed on triadic interactions, that is with a large
part of the speech related to objects in the environments (in this case, the
pictures on the wall). Instead of having a fixed conversational scenario
(e.g., everybody sat at a table directly looking at their addressee), the
Vernissage Corpus also includes situations more difficult to predict, but more
common in ecological scenarios. Figure 8 displays some sequences wrongly
predicted by our model in Exp 1.a and shows that the speaker’s head direction
was not always predictive of the addressee’s position. On the contrary, in
certain cases, it even caused errors. Triadic interactions are one of the
causes of such errors because participants could talk to Nao while looking at
a picture on the right or, vice versa, talking to their companion on the left
while looking at a picture collocated over the robot. This demonstrates that
approaches relying only on the speaker’s visual focus of attention may have
difficulties when it comes to provide predictions in ‘in-the-wild’ scenarios.
To overcome this difficulty, our approach combined information about the face
and the whole body pose and balanced the predictions over time. In this way,
our approach could achieve reliable results in situations more representative
of ‘in-the-wild’ interactions and outperform state-of-the-art models.

###  V-D Limitations and Future Work

Our current Addressee Estimation model presents two major limitations. The
model does not envisage the presence of a large number of people in the
environment. A three-class classification of the position of the addressee
would not be enough in case of crowded places such as airports, hotel
reception areas, malls. However, this work is a step in advance with respect
to most of the previous implementations of Addressee Estimation in HRI, which
provide a binary output or require a fixed conversational scenario. Morover,
tackling Addressee Estimation as prediction of the addressee’s position may
bring an additional benefit to the robot in terms of augmented perception of
the external world. Since our model rely only on the speaker and no
information about other agents is required, the information about the
addressee’s position may improve the robot’s awareness and localization of
other agents not yet detected in the environment.

With regards to the second limitation, at the moment, the model has been
designed only for visual information. No auditory or contextual information is
used. By design, the model was conceived not to rely on contextual
information, so that a first prediction of the addressee may be provided with
the only use of visual information, without any knowledge about the number of
agents in the environment, previous addressees, previous speakers, topic of
the dialogue, hot-words, etc. Future works may add such contextual features to
improve the performance of the model. Future implementations could also
envisage the use of auditory cues, which comprise not only verbal information
but also, for instance, prosody. Auditory data were already available in the
Vernissage Corpus, but too noisy to extract reliable prosodic information.

##  VI Conclusion

In this work, we tackled the problem of Addressee Estimation, understanding to
whom a speaker is addressing an utterance, by developing a deep learning
hybrid model (CNN+LSTM) taking as input two visual information: the speaker’s
face and body pose. Our approach aims to develop a model for Addressee
Estimation for real-time and ecological HRI. Our model achieves reliable
performances even before the conclusion of the utterance and without any use
of contextual information. When it comes to develop social skills for robots,
deep learning architectures prove to be a suitable solution but need to be
designed combining the richness of information provided by human bodily
behaviour, integrating the temporal dimension of the task, and considering the
ecological validity of the model. In this direction, the present work
represents a good stepping stone to be able to equip robots with Addressee
Estimation skills and to enhance natural and effective HRI.

## References

  * [1] I. Bakx, K. Van Turnhout, and J. Terken, “Facial orientation during multi-party interaction with information kiosks,” _Proceedings of INTERACT 2003 ZÃ $1/4$rich, Switzerland_, pp. 163–170, 2003.
  * [2] S. Strohkorb, I. Leite, N. Warren, and B. Scassellati, “Classification of children’s social dominance in group interactions with robots,” in _Proceedings of the 2015 ACM on International Conference on Multimodal Interaction_ , ser. ICMI ’15. New York, NY, USA: Association for Computing Machinery, 2015, p. 227–234.
  * [3] K. Gold, M. Doniec, C. Crick, and B. Scassellati, “Robotic vocabulary building using extension inference and implicit contrast,” _Artificial Intelligence_ , vol. 173, no. 1, pp. 145–166, 2009.
  * [4] G. Skantze, “Turn-taking in conversational systems and human-robot interaction: A review,” _Computer Speech & Language_, vol. 67, p. 101178, 2021.
  * [5] R. Jakobson, _Linguistics and Poetics_. Berlin, Boston: De Gruyter Mouton, 1981, pp. 18–51.
  * [6] P. Auer, _Gaze, addressee selection and turn-taking in three-party interaction_. John Benjamins Amsterdam, 2018, pp. 197–231.
  * [7] R. Ishii, K. Otsuka, S. Kumano, and J. Yamato, “Prediction of who will be the next speaker and when using gaze behavior in multiparty meetings,” _ACM Trans. Interact. Intell. Syst._ , vol. 6, no. 1, may 2016.
  * [8] N. Jovanovic, R. op den Akker, and A. Nijholt, “Addressee identification in face-to-face meetings,” in _11th Conference of the European Chapter of the Association for Computational Linguistics_ , 2006, pp. 169–176.
  * [9] J. Carletta, S. Ashby, S. Bourban, M. Flynn, M. Guillemot, T. Hain, J. Kadlec, V. Karaiskos, W. Kraaij, M. Kronenthal, G. Lathoud, M. Lincoln, A. Lisowska, I. McCowan, W. Post, D. Reidsma, and P. Wellner, “The ami meeting corpus: A pre-announcement,” in _Machine Learning for Multimodal Interaction_ , S. Renals and S. Bengio, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2006, pp. 28–39.
  * [10] H. op den Akker and R. op den Akker, “Are you being addressed?-real-time addressee detection to support remote participants in hybrid meetings,” in _Proceedings of the SIGDIAL 2009 Conference_ , 2009, pp. 21–28.
  * [11] U. Malik, M. Barange, J. Saunier, and A. Pauchet, “Using multimodal information to enhance addressee detection in multiparty interaction.” in _ICAART (1)_ , 2019, pp. 267–274.
  * [12] ——, “A novel focus encoding scheme for addressee detection in multiparty interaction using machine learning algorithms,” _Journal on Multimodal User Interfaces_ , vol. 15, no. 2, pp. 175–188, 2021.
  * [13] M. Johansson and G. Skantze, “Opportunities and obligations to take turns in collaborative multi-party human-robot interaction,” in _Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue_ , 2015, pp. 305–314.
  * [14] S. Sheikhi, D. Babu Jayagopi, V. Khalidov, and J.-M. Odobez, “Context aware addressee estimation for human robot interaction,” in _GazeIn ’13: Proceedings of the 6th workshop on Eye gaze in intelligent human machine interaction: gaze in multimodal interaction_ , ser. GazeIn ’13. New York, NY, USA: Association for Computing Machinery, 2013, p. 1–6.
  * [15] S. Horiguchi, N. Kanda, and K. Nagamatsu, “Multimodal response obligation detection with unsupervised online domain adaptation.” in _INTERSPEECH_ , 2019, pp. 4180–4184.
  * [16] K. van Turnhout, J. Terken, I. Bakx, and B. Eggen, “Identifying the intended addressee in mixed human-human and human-computer interaction from non-verbal features,” in _Proceedings of the 7th International Conference on Multimodal Interfaces_ , ser. ICMI ’05. New York, NY, USA: Association for Computing Machinery, 2005, p. 175–182.
  * [17] M. Katzenmaier, R. Stiefelhagen, and T. Schultz, “Identifying the addressee in human-human-robot interactions based on head pose and speech,” in _Proceedings of the 6th International Conference on Multimodal Interfaces_ , ser. ICMI ’04. New York, NY, USA: Association for Computing Machinery, 2004, p. 144–151.
  * [18] V. Richter, B. Carlmeyer, F. Lier, S. Meyer zu Borgsen, D. Schlangen, F. Kummert, S. Wachsmuth, and B. Wrede, “Are you talking to me? improving the robustness of dialogue systems in a multi party hri scenario by incorporating gaze direction and lip movement of attendees,” in _Proceedings of the Fourth International Conference on Human Agent Interaction_ , ser. HAI ’16. New York, NY, USA: Association for Computing Machinery, 2016, p. 43–50.
  * [19] H.-H. Huang, N. Baba, and Y. Nakano, “Making virtual conversational agent aware of the addressee of users’ utterances in multi-user conversation using nonverbal information,” in _Proceedings of the 13th International Conference on Multimodal Interfaces_ , ser. ICMI ’11. New York, NY, USA: Association for Computing Machinery, 2011, p. 401–408.
  * [20] M. Romeo, D. Hernández García, R. Jones, and A. Cangelosi, “Deploying a deep learning agent for hri with potential ”end-users” at multiple sheltered housing sites,” in _Proceedings of the 7th International Conference on Human-Agent Interaction_ , ser. HAI ’19. New York, NY, USA: Association for Computing Machinery, 2019, p. 81–88.
  * [21] D. B. Jayagopi, S. Sheikhi, D. Klotz, J. Wienke, J.-M. Odobez, S. Wrede, V. Khalidov, L. Nguyen, B. Wrede, and D. Gatica-Perez, “The vernissage corpus: A multimodal human-robot-interaction dataset,” L’IDIAP Laboratory, Tech. Rep., 2012.
  * [22] D. B. Jayagopi, S. Sheiki, D. Klotz, J. Wienke, J.-M. Odobez, S. Wrede, V. Khalidov, L. Nyugen, B. Wrede, and D. Gatica-Perez, “The vernissage corpus: A conversational human-robot-interaction dataset,” in _2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI)_ , 2013, pp. 149–150.
  * [23] Z. Cao, G. Hidalgo Martinez, T. Simon, S. Wei, and Y. A. Sheikh, “Openpose: Realtime multi-person 2d pose estimation using part affinity fields,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 2019.
  * [24] A. Subramaniam, V. Patel, A. Mishra, P. Balasubramanian, and A. Mittal, “Bi-modal first impressions recognition using temporally ordered deep audio and stochastic visual features,” in _Computer Vision – ECCV 2016 Workshops_ , G. Hua and H. Jégou, Eds. Cham: Springer International Publishing, 2016, pp. 337–348.
  * [25] M. Romeo, D. Hernández García, T. Han, A. Cangelosi, and K. Jokinen, “Predicting apparent personality from body language: benchmarking deep learning architectures for adaptive social human–robot interaction,” _Advanced Robotics_ , vol. 35, no. 19, pp. 1167–1179, 2021.
  * [26] A. Ullah, J. Ahmad, K. Muhammad, M. Sajjad, and S. W. Baik, “Action recognition in video sequences using deep bi-directional lstm with cnn features,” _IEEE Access_ , vol. 6, pp. 1155–1166, 2018.
  * [27] B. Nakisa, M. N. Rastgoo, A. Rakotonirainy, F. Maire, and V. Chandran, “Automatic emotion recognition using temporal multimodal deep learning,” _IEEE Access_ , vol. 8, pp. 225 463–225 474, 2020.
  * [28] D. Ramachandram and G. W. Taylor, “Deep multimodal learning: A survey on recent advances and trends,” _IEEE Signal Processing Magazine_ , vol. 34, no. 6, pp. 96–108, 2017.
  * [29] S. R. Stahlschmidt, B. Ulfenborg, and J. Synnergren, “Multimodal deep learning for biomedical data fusion: a review,” _Briefings in Bioinformatics_ , vol. 23, no. 2, 01 2022.
  * [30] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” 2014.
  * [31] J. Li, “The benefit of being physically present: A survey of experimental works comparing copresent robots, telepresent robots and virtual agents,” _International Journal of Human-Computer Studies_ , vol. 77, pp. 23–37, 2015\.


# ExpMRC: Explainability Evaluation for Machine Reading Comprehension

Yiming Cui1,2, Ting Liu1, Wanxiang Che1, Zhigang Chen2, Shijin Wang2,3
1Research Center for SCIR, Harbin Institute of Technology, Harbin, China
2State Key Laboratory of Cognitive Intelligence, iFLYTEK Research, China
3iFLYTEK AI Research (Hebei), Langfang, China
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>

###### Abstract

Achieving human-level performance on some of Machine Reading Comprehension
(MRC) datasets is no longer challenging with the help of powerful Pre-trained
Language Models (PLMs). However, it is necessary to provide both answer
prediction and its explanation to further improve the MRC system’s
reliability, especially for real-life applications. In this paper, we propose
a new benchmark called ExpMRC for evaluating the explainability of the MRC
systems. ExpMRC contains four subsets, including SQuAD, CMRC 2018, RACE+, and
C3 with additional annotations of the answer’s evidence. The MRC systems are
required to give not only the correct answer but also its explanation. We use
state-of-the-art pre-trained language models to build baseline systems and
adopt various unsupervised approaches to extract evidence without a human-
annotated training set. The experimental results show that these models are
still far from human performance, suggesting that the ExpMRC is
challenging.111Resources will be available through
https://github.com/ymcui/expmrc

##  1 Introduction

Machine Reading Comprehension is a task that requires the machine to read and
comprehend given passages and answer the questions, which receives wide
attention over the past few years. We have seen tremendous efforts on creating
challenging datasets (Hermann et al., 2015; Hill et al., 2015; Rajpurkar et
al., 2016; Lai et al., 2017; Cui et al., 2019; Sun et al., 2020) and designing
effective models (Kadlec et al., 2016; Cui et al., 2017; Seo et al., 2016).

However, though the state-of-the-art systems could achieve better performance
than the average human on some MRC datasets with the help of pre-trained
language models (Devlin et al., 2019; Liu et al., 2019; Clark et al., 2020),
the explainability of these systems remains uncertain, which raises concerns
in utilizing these models in real-world applications. In a realistic view,
question answering (QA) or MRC systems that only give final prediction could
not convince the users since these results lack explainability. In this
context, Explainable Artificial Intelligence (XAI) (Gunning, 2017) received
much more attention in these years than ever. XAI aims to produce more
explainable machine learning models while preserving a high accuracy of the
model output and let the human understand its intrinsic mechanism.

Understanding the intrinsic mechanism of the neural network is a challenging
issue. There are several hot discussions on the relevant topics, such as
whether the attention could be explanations (Serrano and Smith, 2019; Jain and
Wallace, 2019; Wiegreffe and Pinter, 2019; Bastings and Filippova, 2020).
Nonetheless, we could seek post-hoc explainability approaches, which target
models that are not readily interpretable by design. Post-hoc approaches
resort to diverse means to enhance the model’s interpretability (Barredo
Arrieta et al., 2020). One of the suitable post-hoc approaches for NLP is to
generate text explanations, which is a practical way to alleviate the absence
of the neural network’s explainability. Though the text explanation does not
necessarily interpret the model’s intrinsic mechanism, it is informative to
know both the predicted answer and its text explanation, especially for those
real-life applications.

To better evaluate the MRC model’s explainability, in this paper, we propose a
comprehensive benchmark ExpMRC for the machine reading comprehension in a
multi-lingual and multi-task way, which evaluates the accuracy of both answer
and its explanation. The proposed ExpMRC contains four subsets, including
SQuAD (Rajpurkar et al., 2016), CMRC 2018 (Cui et al., 2019), RACE+, and C3
(Sun et al., 2020) with additional annotations of the evidence spans, covering
span-extraction MRC and multi-choice MRC in both English and Chinese. The MRC
model should not only give an answer span or select a choice for the question
but also give a passage span as the evidence, which creates more challenges to
the machine. The resulting dataset contains 11K human-annotated evidence spans
over 4K questions. The contributions of our paper are as follows.

  * •

We propose a new MRC benchmark called ExpMRC, which aims to evaluate the
accuracy of the final answer prediction and how well it gives for its
explanation.

  * •

We also propose several baseline systems that adopt unsupervised approaches
for ExpMRC.

  * •

The experimental results on ExpMRC show that the current pre-trained language
models are still far from satisfactory in giving explanations for the
predicted answer, suggesting that the proposed ExpMRC is challenging.

##  2 Related Work

Machine reading comprehension has been regarded as an important task to test
how well the machine comprehends human languages. In the earlier stage, as
most of the models (Dhingra et al., 2017; Kadlec et al., 2016; Cui et al.,
2017) are solely trained on the training data of each dataset without much
prior knowledge, their performances are not very impressive. However, as the
pre-trained language models emerged during these years, such as BERT (Devlin
et al., 2019), RoBERTa (Liu et al., 2019), ELECTRA (Clark et al., 2020), etc.,
many systems could achieve better performances than average human on several
MRC datasets, such as SQuAD 1.1 and 2.0 (Rajpurkar et al., 2016, 2018).

After reaching the ‘over-human’ performance, there is another issue to be
addressed. The decision process and the explanation of these artifacts still
remain unclear, raising concerns about their reliability. In this context, XAI
becomes more important than ever not only in NLP but also in various
directions in artificial intelligence. However, most of the cutting-edge
systems are developed on neural networks, and the explainability investigation
of these approaches is non-trivial, which is still an on-going research topic.

In NLP, some researchers conducted analyses to better understand the internal
mechanism of BERT-based architecture. For example, Kovaleva et al. (2019)
discovered that there are repetitive attention patterns across different heads
in the multi-head attention mechanism indicating its over-parametrization. But
perhaps the hottest discussion is whether the attention could be explanations.
Some researchers argue that the attention could not be used as explanations,
such as Jain and Wallace (2019) verify that using completely different
attention weights could also achieve the same prediction. On the contrary,
some works hold positive attitudes about this topic (Wiegreffe and Pinter,
2019; Bastings and Filippova, 2020). These works have brought us different
views of attention-based models, but there is still no consensus about this
important topic.

In MRC, the most relevant effort in explainability is the creation of HotpotQA
(Yang et al., 2018), which is a multi-hop explainable question answering
dataset. HotpotQA requires the machine to retrieve the relevant documents and
extract a passage span as the answer along with its evidence sentences.
Various models (Qiu et al., 2019; Shao et al., 2020) have been proposed to
tackle this task using supervised learning approaches with the labeled
training data. However, unfortunately, almost all works focus on achieving
higher scores in the benchmark without specifically caring about the
explainability.

Above all, though various efforts have been made, we argue that the
explainability is a universal demand for all machine reading comprehension
tasks and different languages but not only restricted to English multi-hop QA.
Another issue is that annotating evidence for each task is not feasible, and
we should also seek unsupervised approaches, which do not rely on any
annotated evidence to minimize the labor cost.

In this context, we propose ExpMRC to specifically focus on evaluating
explainability on four tasks, covering span-extraction and multi-choice MRC in
both English and Chinese. ExpMRC does not provide annotated training data,
which aims to enforce our community to focus on designing unsupervised
approaches to improve their generalizability across different MRC tasks and
even different languages. To the best of our knowledge, this is the first MRC
benchmark in a multi-task and multi-lingual setting, which could be used in
not only explainability evaluation but also various other directions, such as
cross-lingual studies, etc.

Subset |  Passage | Question & Answer
---|---|---
SQuAD |  … Competition amongst employers tends to drive up wages due to the nature of the job, since there is a relative shortage of workers for the particular position. Professional and labor organizations may limit the supply of workers which results in higher demand and greater incomes for members. Members may also receive higher wages through collective bargaining, political influence, or corruption … |  Q: Who works to get workers higher compensation? A: Professional and labor organizations
CMRC 2018 |  … 钩盲蛇（学名：“Ramphotyphlops braminus”）是蛇亚目盲蛇科下的一种无毒蛇种，主要分布在非洲及亚洲，不过现在钩盲蛇的分布已推广至世界各地。钩盲蛇是栖息于地洞的蛇种，由于体型细小，加上善于掘洞，因此经常被误认为蚯蚓，唯一分别就是钩盲蛇的身体并没有分成明显的段节。… |  Q: 钩盲蛇一般生活在什么 地形中？ A: 地洞
RACE+ |  … By making their leaves, flowers, roots and fruits poisonous to enemies, plants can fight back. One such plant is the Golden Wattle tree, British scientist David Caneron has found when an animal eats the tree’s leaves, the amount of poison increase in the other leaves. “It’s like the injured leaves telephoning the others telling them to fight together against the enemy,” he said. The tree also sends defense messages to neighboring plants by giving out a special smell. Golden Wattle trees in the nearby 45 meters will get the message and produce more poison within 10 minutes … |  Q: According to the study, if one Golden Wattle tree is attacked by animals, it can? A: tell other trees to protect it B: produce more poison within 10 minutes C: sent defense messages to the neighboring plants D: kill the animals with its leaves
C3 |  … 大学生活是走上社会的预演，可以说，大学里的处世态度和人际关系的成功与否，直接决定着将来在社会上的成败。人是社会性的动物，生活中的每个人都离不开别人的帮助，同时也在帮助着别人。不管是学习、生活、工作，都要求自己要有良好的处理人际关系的能力。一个人要想有良好的人际关系，就要遵循以下几个原则：一是“主动”。要主动和别人交往，主动帮助别人。二是“诚信”。 … |  Q: 说话人认为什么因素决定 在社会上的成败? A: 工作的态度 B: 朋友的数量 C: 大学里的学习成绩 D: 大学里的人际关系
Table 1:  Examples in ExpMRC. The evidence of the answer (in passage) and correct option (for multi-choice MRC) are marked in boldface, and answer spans (for span-extraction MRC) are marked in blue.  | SQuAD | CMRC 2018 | RACE+ | C3
---|---|---|---|---
Dev | Test | Dev | Test | Dev | Test | Dev | Test
Language | English | Chinese | English | Chinese
Answer Type | passage span | passage span | multi-choice | multi-choice
Domain | Wikipedia | Wikipedia | exams | exams
Passage # | 319 | 313 | 369 | 399 | 167 | 168 | 273 | 244
Question # | 501 | 502 | 515 | 500 | 561 | 564 | 505 | 500
Max Answer # per Question | 3 | 3 | 3 | 3 | 1 | 1 | 1 | 1
Max Evidence # per Question | 2 | 2 | 3 | 3 | 2 | 2 | 4 | 4
Avg/Max Passage Tokens # | 146/369 | 157/352 | 467/961 | 468/930 | 311/514 | 324/603 | 426/1096 | 413/1011
Avg/Max Question Tokens # | 12/28 | 11/28 | 15/37 | 15/37 | 15/39 | 16/55 | 14/28 | 14/31
Avg/Max Answer Tokens # | 3/25 | 3/27 | 6/64 | 5/33 | 6/20 | 6/27 | 7/25 | 7/35
Avg/Max Evidence Tokens # | 26/62 | 28/76 | 43/175 | 52/313 | 23/162 | 23/82 | 37/199 | 41/180
Table 2:  Statistics of the proposed ExpMRC.

##  3 ExpMRC

###  3.1 Subset Selection

The motivation of our dataset is to provide a comprehensive MRC benchmark for
evaluating not only the prediction accuracy but also how well it gives for its
explanation. To this end, our dataset is not completely composed of new data.
We adopt several well-designed MRC datasets as well as newly annotated data to
form our dataset to minimize the repetitive annotations and place our work in
line with previous works.

Specifically, our ExpMRC is partly developed from the following datasets,
including two span-extraction MRC datasets and one multi-choice MRC dataset.

  * •

SQuAD (Rajpurkar et al., 2016) is a well-known dataset for span-extraction
MRC. Given a Wikipedia passage, the MRC system should extract a passage span
as the answer to the question.

  * •

CMRC 2018 (Cui et al., 2019) is also a span-extraction MRC dataset but in
Chinese. Besides the traditional train/dev/test split, they also released a
challenge set that requires multi-sentence inference while keeping the
original span-extraction setting.

  * •

C3 (Sun et al., 2020) is a Chinese multi-choice MRC dataset. The system should
choose the correct option as the answer after reading the passage and
question. To ensure the domain consistency for each subset, we only use non-
dialogue subsets C${}^{3}_{\text{M}}$.

As SQuAD and CMRC 2018 do not release the test set to the public, we are
unable to adopt them directly. Instead, we follow the original dataset
construction steps to replicate two subsets for testing purposes, where the
subsets are annotated from either English or Chinese Wikipedia passages. Note
that, during the subset annotation, we select the passages that do not appear
in the original training and development set.

While we could have used RACE (Lai et al., 2017) as the counterpart of C3, we
decided not to adopt it. We had some in-house collected multi-choice MRC data,
which is similar to RACE and is also designed for the middle and high school
students. More importantly, these data contain additional hints on the
answering process, which are very helpful for the evidence annotation. Thus,
we decided to use our data instead of RACE. We denote this new subset as
RACE+.

At this point, we have four subsets to be annotated, containing both span-
extraction and multi-choice MRC in both English and Chinese.

###  3.2 Evidence Annotation

Before evidence annotation, the annotators are required to consider whether a
question is appropriate for annotation. We skipped some questions based on the
following criteria.

  * •

The evidence span is a simple combination of the question and answer without
much syntactical or semantical variance.

  * •

The questions require external knowledge to be solved and cannot be only
inferred from the passage.

  * •

The conclusive questions of the whole passage, such as ‘what is the best title
of this passage?’, ‘what is the main idea of the passage?’, etc.

After that, we begin the evidence annotation process. First, the annotators
are asked to read the question and the correct answer (passage span or option
text). Then, they should select (copy-and-paste) a passage span from the
passage that can be the evidence of the answer. The evidence should be a
minimal passage span that can support the answer and should not always be a
complete sentence or clause. We encourage the annotators to select the
evidence that needs reasoning skills, though this is not a usual case in these
datasets, especially in span-extraction MRC (most of the questions do not need
reasoning).

###  3.3 Quality Assurance

The annotators are paid about $0.5 per evidence for all types of MRC data. The
annotators are either English-majored or Chinese-majored graduate students
from China, depending on the language of the dataset. Also, to avoid
overworking and decreasing the annotation quality, we set a hard limit on the
number of daily annotations for the evidence. After reaching the limit (300),
the system is automatically locked and will be unlocked the next day.

Besides, as our dataset has two or more referential evidence spans per
question, we do not reveal the annotated evidence span to the current
annotator to increase the diversity and avoid copy-and-paste behavior. After
the preliminary annotation, all evidence spans are checked one-by-one to
ensure a high-quality dataset maximally. Finally, they should verify that we
can pick out the correct answer by only reading the evidence and question to
ensure that the annotation is valid.

###  3.4 Statistics

The statistics of the proposed ExpMRC are listed in Table 2. Note that the
‘token’ in Table 2 represents the character for Chinese and word for English.
For all subsets, we provide $2\sim 4$ referential evidence spans for each
question. The distribution of the question type in each task’s development set
is depicted in Figure 1. As we can see that there are fewer questions of ‘who,
when, where’ in RACE+ and C3, suggesting that these subsets are much more
difficult.

Figure 1:  Distribution of question types.

##  4 Baselines

Given that the proposed ExpMRC is designed to evaluate the explainability in
terms of the system’s explanation text, we mainly focus on the unsupervised
approaches222The term ‘unsupervised’ specifically refers to the annotated
explanation spans. for our baseline systems. We use pre-trained language
models as the backbones to generate answers for the questions. Then we apply
several methods to generate evidence spans, where we classify them into non-
learning and machine learning baselines.

###  4.1 Non-learning Baselines

For non-learning baselines, we mainly use the prediction and question as the
clues for finding evidence. For simplicity, we only consider extracting
sentence-level evidence in these baselines, though the ground truth evidence
may not always be a complete sentence. We first split the passage into several
sentences using ‘.!?’ as delimiters. Then we choose one of the passage
sentences as the evidence prediction. To find much accurate evidence sentence,
we adopt three approaches.

  * •

Most Similar Sentence: We calculate the token-level F1 score between the
predicted answer span (or choice text) and each passage sentence. Then we
select the sentence that has the highest F1 as the evidence prediction. In
span-extraction MRC tasks, the extracted evidence should be the sentence that
contains the prediction span in most of the cases.

  * •

Most Similar Sentence with Question: Similar to the ‘Most Similar Sentence’
setting, but we use both question text and predicted answer span as the key to
finding the most similar passage sentence.

  * •

Answer Sentence: Particularly, in span-extraction MRC tasks, we could directly
extract the sentence that contains the answer prediction as the evidence.

These approaches largely rely on the accuracy of answer prediction, as the
wrong prediction will directly affect the finding process of evidence.

###  4.2 Machine Learning Baselines

As no training data is provided in ExpMRC, we seek a pseudo-training approach
to accomplish a machine learning baseline system. First, we generate pseudo-
evidence for each sample in the respective training set, which has no evidence
annotation. We use the ground truth answer and question text to find the most
similar passage sentence as the pseudo-evidence to form pseudo-training data.
Then we use the pseudo-training data and PLM to train a model that outputs
both answer prediction and evidence prediction. Specifically, we add an
additional task head on top of the PLM’s final hidden representation,
alongside its original answer prediction task, as shown in Figure 2.

  * •

Span-Extraction MRC: The concatenation of the question and passage is fed into
PLM, and we use the final hidden representation with two fully-connected
layers to predict the start and end positions of the answer span.

  * •

Multi-Choice MRC: The concatenation of the passage, question, and each choice
is fed into PLM to get four pooled representations (assuming we have four
candidate choices). Then we use a fully-connected layer with softmax
activation to predict the final choice.

(a) Span-Extraction MRC

(b) Multi-Choice MRC

Figure 2:  Neural network architecture of the baselines.

The evidence prediction is identical to the answer prediction in span-
extraction MRC, where we project the final hidden representation
$\bm{h}\in\mathbb{R}^{n\times h}$ into the start and end probability
distributions $p^{s},p^{e}\in\mathbb{R}^{n}$. Then we calculate standard
cross-entropy loss of the start and end positions for evidence span
prediction.

$\displaystyle p^{s}=\mathbf{softmax}(\bm{h}\mathbf{w^{s}}+b^{s})$ (1)
$\displaystyle p^{e}=\mathbf{softmax}(\bm{h}\mathbf{w^{e}}+b^{e})$ (2)
$\displaystyle\mathcal{L}_{E}=-\frac{1}{2N}\sum\limits_{i=1}^{N}(y^{s}_{i}\log
p^{s}+y^{e}_{i}\log p^{e})$ (3)

The final training loss is the sum of answer prediction loss $\mathcal{L}_{A}$
and the evidence prediction loss $\mathcal{L}_{E}$, where we apply
$\lambda\in[0,1]$ scaling on $\mathcal{L}_{E}$, as the pseudo-training data is
not quite accurate.

$\displaystyle\mathcal{L}=\mathcal{L}_{A}+\lambda\mathcal{L}_{E}$ (4)

##  5 Evaluation Approach

###  5.1 Evaluation Metrics

To evaluate how well the MRC model could generate explanations for the
answers, we use the following metrics, which are divided into answer
evaluation and evidence evaluation.

For answer evaluation, we strictly follow the original evaluation script for
each subset. Specifically, we use the F1-score (F1) to evaluate SQuAD and CMRC
2018. We dump Exact Match (EM) and only evaluated F1 for simplicity. Note
that, as these datasets are in different languages, the evaluation details are
slightly different. For RACE+ and C3, we use accuracy for evaluation.

For evidence evaluation, we also use F1 metrics, as most of the evidence spans
are quite long, and it is hard for the machine to extract the evidence spans
exactly, and thus we do not adopt EM. Also, the central idea of the evidence
is to provide enough information to support the answer, so it is proper to
adopt F1. Note that we only evaluate the correctness of evidence in this
metric, regardless of the correctness of the answer.

Altogether, we also provide an overall F1 metric to give a comprehensive
evaluation of the system. For each instance, we calculate the score of the
answer metric and evidence metric. The overall F1 of each instance is obtained
by multiplying both terms. Finally, the overall F1 of all instances is
obtained by averaging all instance-level F1. The overall F1 reflects the
correctness of both answer and its evidence.

$\mathtt{F1_{overall}}=\mathtt{F1_{answer}}\times\mathtt{F1_{evidence}}$ (5)

###  5.2 Human Performance

Following previous works (Rajpurkar et al., 2016; Lai et al., 2017; Cui et
al., 2019), we also report human performance to estimate how well does human
performs on this dataset. Following Cui et al. (2019), we use a cross-
validation approach that regarding one of the candidates as prediction and
treating the rest of the candidates as ground truths. Final scores are
obtained by averaging all possible combinations. Specifically,

  * •

SQuAD, CMRC 2018: In these datasets, both answer and evidence have multiple
references, and thus we use the cross-validation approach for both and get
their products as instance-level human performance.

  * •

RACE+, C3: As these datasets have only one reference answer, we invite three
annotators to answer a random set of 100 questions in each set to get averaged
human performance of the answer. For the evidence, we directly use the cross-
validation approach for the selected random set. Similarly, the instance-level
human performance is obtained by the product of the answer and evidence score.

Note that as the evidence spans are annotated by referring to either the
answers or additional hints, the actual human performance could be much lower,
and thus these results should be regarded as ceiling human performance
roughly. Finally, we average the scores in all instances to obtain the final
overall human performance.

System | SQuAD (dev) | SQuAD (test) | CMRC 2018 (dev) | CMRC 2018 (test)
---|---|---|---|---
Ans. | Evi. | All | Ans. | Evi. | All | Ans. | Evi. | All | Ans. | Evi. | All
Human Performance | 90.8 | 92.1 | 83.6 | 91.3 | 92.9 | 84.7 | 97.7 | 94.6 | 92.4 | 97.9 | 94.6 | 92.6
PLM base-level Baselines |  |  |  |  |
Most Similar Sent. | 87.4 | 81.8 | 74.5 | 87.1 | 85.4 | 76.1 | 82.3 | 71.9 | 60.1 | 84.4 | 62.2 | 52.9
Most Similar Sent. w/ Ques. | 87.4 | 81.0 | 72.9 | 87.1 | 84.8 | 75.6 | 82.3 | 76.9 | 63.9 | 84.4 | 69.8 | 59.9
Predicted Answer Sent. | 87.4 | 84.1 | 76.4 | 87.1 | 89.1 | 79.6 | 82.3 | 78.0 | 66.8 | 84.4 | 69.1 | 59.8
Pseudo-data Training | 88.1 | 76.9 | 69.7 | 88.1 | 76.3 | 67.5 | 82.7 | 65.1 | 54.7 | 85.7 | 59.7 | 51.2
PLM large-level Baselines |  |  |  |  |
Most Similar Sent. | 93.0 | 83.9 | 79.3 | 92.3 | 85.7 | 80.4 | 82.8 | 71.6 | 60.3 | 88.6 | 63.0 | 55.9
Most Similar Sent. w/ Ques. | 93.0 | 81.9 | 77.4 | 92.3 | 85.1 | 79.8 | 82.8 | 76.3 | 63.6 | 88.6 | 71.0 | 63.2
Predicted Answer Sent. | 93.0 | 85.4 | 81.8 | 92.3 | 89.6 | 83.6 | 82.8 | 77.7 | 66.9 | 88.6 | 70.6 | 63.3
Pseudo-data training | 93.3 | 78.1 | 73.5 | 93.4 | 76.1 | 72.1 | 83.8 | 65.3 | 55.6 | 88.8 | 58.4 | 51.5

System | RACE+ (dev) | RACE+ (test) | C3 (dev) | C3 (test)
---|---|---|---|---
Ans. | Evi. | All | Ans. | Evi. | All | Ans. | Evi. | All | Ans. | Evi. | All
Human Performance | 92.0 | 92.4 | 85.4 | 93.6 | 90.5 | 84.4 | 95.3 | 95.7 | 91.1 | 94.3 | 97.7 | 90.0
PLM base-level Baselines |  |  |  |  |
Most Similar Sent. | 62.4 | 36.6 | 28.2 | 59.8 | 34.4 | 26.3 | 68.7 | 57.7 | 47.7 | 66.8 | 52.2 | 41.2
Most Similar Sent. w/ Ques. | 62.4 | 44.5 | 31.5 | 59.8 | 41.8 | 27.3 | 68.7 | 62.3 | 47.3 | 66.8 | 57.4 | 42.3
Pseudo-data training | 63.6 | 45.7 | 31.7 | 60.1 | 43.5 | 27.1 | 70.9 | 59.9 | 43.5 | 69.0 | 57.5 | 40.6
PLM large-level Baselines |  |  |  |  |
Most Similar Sent. | 69.0 | 37.6 | 29.9 | 68.1 | 36.8 | 28.9 | 73.1 | 59.4 | 49.9 | 72.0 | 52.7 | 43.9
Most Similar Sent. w/ Ques. | 69.0 | 48.0 | 36.8 | 68.1 | 42.5 | 31.3 | 73.1 | 63.2 | 50.9 | 72.0 | 58.4 | 46.0
Pseudo-data training | 69.0 | 45.9 | 32.6 | 70.4 | 41.3 | 30.8 | 76.4 | 64.3 | 50.7 | 74.4 | 59.9 | 47.3

Table 3:  Baseline results on SQuAD, CMRC 2018, RACE+, and C3.

##  6 Experiments

###  6.1 Setups

We use pre-trained language models as backbones of baseline systems.
Specifically, we use BERT-base and BERT-large-wwm (Devlin et al., 2019) for
English, and MacBERT-base/large (Cui et al., 2020) for Chinese tasks. We use a
universal initial learning rate of 3e-5 and iterate two training epochs for
all tasks. The maximum sequence length is set to 512, and a QA length of 128
in all experiments. We use Adam Kingma and Ba (2014) with weight decay
optimizer for training. We set $\lambda=0.1$ in the final loss function to
penalize the evidence pseudo-data training, which we found to be effective,
and further investigation is discussed in Section 6.3.

###  6.2 Baseline Results

Overall, the best-performing baselines are still far behind the human
performance, indicating that the proposed dataset is challenging. Also, the
gaps in multi-choice MRC subsets are bigger than the ones in span-extraction
MRC. For all subsets, adding question text for similarity calculation is more
effective than only using the predicted answer. For span-extraction MRC,
traditional token similarity methods seem to be more effective as the answer
is already a passage span, and its evidence often lies around its context. On
the contrary, the pseudo-data training approach is more effective in multi-
choice MRC, where the options are not composed by the passage span, which is
not capable of direct mapping, and it requires similarity calculation in
semantics but not only in the token-level calculation.

(a) SQuAD (b) CMRC 2018 (c) RACE+ (d) C3

Figure 3:  Effect of the lambda term in the evidence loss for different tasks.

The improvement of both answer and evidence prediction does NOT necessarily
improve the overall score. For example, in the development set of C3, pseudo-
data training large-level baseline yields better performance on both answer
and evidence prediction than the others. However, its overall score of 50.7 is
lower than the best-performing baseline of 50.9. After checking the prediction
file, we discovered that there are more samples that have either better
evidence spans for the wrong answer prediction or worse evidence spans for
correct answer prediction, which decreases the overall score.

Another interesting observation is that though pseudo-data training baselines
do not yield better overall scores mostly, we have seen almost consistent
improvements in the answer prediction accuracy, such as in C3 using large-
level PLM (e.g., dev +3.3, test +2.4). This suggests that using pseudo
evidence helps improve answer prediction, and we expect there will be another
improvement when we use a more efficient way to extract high-quality pseudo
evidence.

###  6.3 Answer and Evidence Balance

For balancing the ratio between the answer and evidence loss, we applied a
lambda term on the evidence loss. To explore the effect of the lambda term, we
select different $\lambda\in[0,1]$, and plot the 5-run average performance of
each task. The results are shown in Figure 3.

Overall, as we can see that by increasing the lambda term, the evidence score
and overall score decrease, suggesting that the pseudo-data training could not
be regarded as important as the original supervised task training (answer
prediction), as the pseudo-data is not constructed by the golden evidence.
However, when it comes to the answer score, we observe that the span-
extraction MRC tasks are less sensitive to the lambda term compared to the
multi-choice MRC tasks.

###  6.4 Upper Bound Test

In the last section, we analyze the possible next steps to achieve better
evidence extraction performance. Besides the ‘Most Similar Sentence with
Question’ (MSS w/ Ques.) and ‘Predicted Answer Sentence’ (PA Sent.), we also
provide two additional baselines. We extract the sentence that contains the
ground truth answer (GA Sent.) or annotated evidence (GE Sent.) to measure the
upper bounds for those systems that only extract sentence-level evidence. The
results are shown in Table 4.

| SQuAD | CMRC | RACE+ | C3
---|---|---|---|---
MSS w/ Ques. | 81.9 | 76.3 | 48.0 | 63.2
PA Sent. | 85.4 | 77.7 | - | -
GA Sent. | 88.2 | 82.1 | 49.9 | 66.8
GE Sent. | 91.6 | 85.2 | 86.9 | 89.1
Human Perf. | 92.1 | 94.6 | 92.4 | 95.7
Table 4:  Evidence performance of development sets.

As we can see, the PA-GA and GA-GE gap in span-extraction MRC is very small
(about 3%$\sim$5%), suggesting that the current system is about to reach the
ceiling performance when only using sentence-level evidence extraction. On the
contrary, in multi-choice MRC, we see a huge gap between GA and GE, indicating
that only using the answer sentence is not enough to achieve strong evidence
extraction performance.

The gap between GE and human performance indicates the gains from expanding
sentence-level evidence to a free-form evidence span. Besides the SQuAD task,
the others yield 5.5%$\sim$9.4% gap, which demonstrates that finding the exact
evidence span in these tasks could still achieve a decent improvement.

##  7 Conclusion

In this paper, we propose a comprehensive benchmark for evaluating the
explainability of machine reading comprehension systems. The proposed ExpMRC
benchmark contains four datasets, covering span-extraction MRC and multiple-
choice MRC in both English and Chinese. ExpMRC aims to evaluate the MRC system
to give not only correct predictions on the final answer but also extract
correct evidence for the answer. We set up several baseline systems to
thoroughly evaluate the difficulties of ExpMRC. The experimental results show
that both traditional and state-of-the-art pre-trained language models still
underperform human performance by a large margin, indicating that more work
should be done on explanation extraction. We hope the release of the dataset
could further accelerate the research of explainability and interpretability
for the MRC systems.

## References

  * Barredo Arrieta et al. (2020) Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garcia, Sergio Gil-Lopez, Daniel Molina, Richard Benjamins, Raja Chatila, and Francisco Herrera. 2020.  Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai.  _Information Fusion_ , 58:82 – 115.
  * Bastings and Filippova (2020) Jasmijn Bastings and Katja Filippova. 2020.  The elephant in the interpretability room: Why use attention as explanation when we have saliency methods? In _Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP_ , pages 149–155, Online. Association for Computational Linguistics.
  * Clark et al. (2020) Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020.  ELECTRA: Pre-training text encoders as discriminators rather than generators.  In _ICLR_.
  * Cui et al. (2020) Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Shijin Wang, and Guoping Hu. 2020\.  Revisiting pre-trained models for Chinese natural language processing.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings_ , pages 657–668, Online. Association for Computational Linguistics.
  * Cui et al. (2017) Yiming Cui, Zhipeng Chen, Si Wei, Shijin Wang, Ting Liu, and Guoping Hu. 2017.  Attention-over-attention neural networks for reading comprehension.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 593–602. Association for Computational Linguistics.
  * Cui et al. (2019) Yiming Cui, Ting Liu, Wanxiang Che, Li Xiao, Zhipeng Chen, Wentao Ma, Shijin Wang, and Guoping Hu. 2019.  A span-extraction dataset for Chinese machine reading comprehension.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 5886–5891, Hong Kong, China. Association for Computational Linguistics.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Dhingra et al. (2017) Bhuwan Dhingra, Hanxiao Liu, Zhilin Yang, William Cohen, and Ruslan Salakhutdinov. 2017.  Gated-attention readers for text comprehension.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1832–1846. Association for Computational Linguistics.
  * Gunning (2017) David Gunning. 2017.  Explainable artificial intelligence (xai).  _Defense Advanced Research Projects Agency (DARPA), nd Web_ , 2.
  * Hermann et al. (2015) Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015.  Teaching machines to read and comprehend.  In _Advances in Neural Information Processing Systems_ , pages 1684–1692.
  * Hill et al. (2015) Felix Hill, Antoine Bordes, Sumit Chopra, and Jason Weston. 2015.  The goldilocks principle: Reading children’s books with explicit memory representations.  _arXiv preprint arXiv:1511.02301_.
  * Jain and Wallace (2019) Sarthak Jain and Byron C. Wallace. 2019.  Attention is not Explanation.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 3543–3556, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Kadlec et al. (2016) Rudolf Kadlec, Martin Schmid, Ondřej Bajgar, and Jan Kleindienst. 2016.  Text understanding with the attention sum reader network.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 908–918. Association for Computational Linguistics.
  * Kingma and Ba (2014) Diederik Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_.
  * Kovaleva et al. (2019) Olga Kovaleva, Alexey Romanov, Anna Rogers, and Anna Rumshisky. 2019.  Revealing the dark secrets of BERT.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 4365–4374, Hong Kong, China. Association for Computational Linguistics.
  * Lai et al. (2017) Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017.  Race: Large-scale reading comprehension dataset from examinations.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 796–805. Association for Computational Linguistics.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Qiu et al. (2019) Lin Qiu, Yunxuan Xiao, Yanru Qu, Hao Zhou, Lei Li, Weinan Zhang, and Yong Yu. 2019\.  Dynamically fused graph network for multi-hop reasoning.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 6140–6150, Florence, Italy. Association for Computational Linguistics.
  * Rajpurkar et al. (2018) Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018.  Know what you don’t know: Unanswerable questions for SQuAD.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 784–789, Melbourne, Australia. Association for Computational Linguistics.
  * Rajpurkar et al. (2016) Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016.  Squad: 100,000+ questions for machine comprehension of text.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_ , pages 2383–2392. Association for Computational Linguistics.
  * Seo et al. (2016) Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hananneh Hajishirzi. 2016.  Bi-directional attention flow for machine comprehension.  _arXiv preprint arXiv:1611.01603_.
  * Serrano and Smith (2019) Sofia Serrano and Noah A. Smith. 2019.  Is attention interpretable? In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 2931–2951, Florence, Italy. Association for Computational Linguistics.
  * Shao et al. (2020) Nan Shao, Yiming Cui, Ting Liu, Shijin Wang, and Guoping Hu. 2020.  Is graph structure necessary for multi-hop question answering? In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 7187–7192, Online. Association for Computational Linguistics.
  * Sun et al. (2020) Kai Sun, Dian Yu, Dong Yu, and Claire Cardie. 2020.  Investigating prior knowledge for challenging chinese machine reading comprehension.  _Transactions of the Association for Computational Linguistics_ , 8:141–155.
  * Wiegreffe and Pinter (2019) Sarah Wiegreffe and Yuval Pinter. 2019.  Attention is not not explanation.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 11–20, Hong Kong, China. Association for Computational Linguistics.
  * Yang et al. (2018) Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018.  HotpotQA: A dataset for diverse, explainable multi-hop question answering.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 2369–2380, Brussels, Belgium. Association for Computational Linguistics.