# Roof-Transformer: Divided and Joined Understanding with Knowledge
Enhancement

Wei-Lin Liao
Academia Sinica

&Cheng-En Su
Academia Sinica
{willsonliao, jimmysu<EMAIL_ADDRESS>
&Wei-Yun Ma
Academia Sinica

###### Abstract

Recent work on enhancing BERT-based language representation models with
knowledge graphs (KGs) and knowledge bases (KBs) has yielded promising results
on multiple NLP tasks. State-of-the-art approaches typically integrate the
original input sentences with KG triples and feed the combined representation
into a BERT model. However, as the sequence length of a BERT model is limited,
such a framework supports little knowledge other than the original input
sentences and is thus forced to discard some knowledge. This problem is
especially severe for downstream tasks for which the input is a long paragraph
or even a document, such as QA or reading comprehension tasks. We address this
problem with Roof-Transformer, a model with two underlying BERTs and a fusion
layer on top. One underlying BERT encodes the knowledge resources and the
other one encodes the original input sentences, and the fusion layer
integrates the two resultant encodings. Experimental results on a QA task and
the GLUE benchmark attest the effectiveness of the proposed model.

##  1 Introduction

Although BERT dominates multiple benchmark datasets, various studies have
conducted to incorporate extra knowledge into language models (LMs) to advance
language modeling Zhang et al. (2019); Liu et al. (2019a); Wang et al. (2020).
Sources of this extra knowledge are mostly knowledge graphs (KGs) and
knowledge bases (KBs) that contain rich knowledge facts and benefit language
understanding. ERNIE Zhang et al. (2019), for example, employs TransE Bordes
et al. (2013) to encode entity information, and concatenates them with token
embeddings for input to a fusion layer. Despite its success on the GLUE
benchmark, ERNIE does not consider textual knowledge representation due to its
token-level concatenation. K-BERT Liu et al. (2019a), converts knowledge
triples into textual forms and injects them into the input sentences, forming
a tree representation for input to BERT. However, this approach considers
little knowledge other than the input sentences due to BERT’s intrinsic input
length limitation (512 tokens).

Accordingly, we propose Roof-Transformer, a model with two underlying BERTs
and a fusion layer, the Transformer encoder Vaswani et al. (2017) as a fusion
layer acting as a “roof” over the two BERTs. Roof-Transformer encodes the text
input with one of the underlying BERTs and uses the other BERT to encode the
knowledge information, and then integrates the two embeddings with the fusion
layer for downstream tasks. This structure allows us to incorporate
information from both the original text and from external knowledge sources.
In addition, if memory permits and long input is needed, more than two BERTs
can be employed using this structure.

Figure 1: The overall architecture of Roof-Transformer: The input of Roof-
Transformer contains input tokens and segmentation tokens. In the case of QA
task, the Prediction Layer directly takes the output of Fusion Layer as input;
while in the case of NLU tasks in GLUE, the Prediction Layer takes the mean of
the output of Fusion Layer as input.

Despite the intuitive nature of the proposed idea, several critical challenges
must to be addressed:

(1) What is an appropriate model for a “roof”, and how does the roof
distinguish individual outputs from the underlying BERTs?

(2) How many layers are needed for the roof to fuse the outputs from the
underlying BERTs? This could involve a trade-off between computational
resources and performance.

(3) Various model complexities may result in convergence times for the roof
that differ from those of the BERTs. How should we address this during
training?

(4) The proposed architecture allows for up to 512 tokens of knowledge.
Although long, this remains a limitation; thus precise knowledge selection and
effective representation are crucial to ensure good performance.

We investigate various factors and propose solutions for these challenges, as
described in the following sections.

We conduct experiments on the QA task Rajpurkar et al. (2016) with Chinese KBs
and the GLUE benchmark with English knowledge corpus. Experimental results
reveal that integrating knowledge using Roof-Transformer outperforms using a
single BERT to integrate both the original input sentences and external
knowledge.

Our contributions are summarized as follows:

  * •

We propose Roof-Transformer, an architecture which uses two distinct BERTs to
encode knowledge and input sentences, and demonstrate promising results on a
QA task and the GLUE benchmark.

  * •

Roof-Transformer addresses BERT’s input length limitation. We believe this
will also facilitate NLP tasks where additional knowledge or long context
comprehension is needed.

  * •

We show that precise knowledge selection and effective representation are
critical to improve the performance for downstream NLP tasks.

##  2 Related Work

Much work has been done to integrate knowledge bases or knowledge graphs for
enhanced language representation.

Before strong pre-trained LMs such as BERT were proposed, research concerned
joint representation learning of words and knowledge. Wang et al. (2014)
combine knowledge embeddings and word vectors, and Toutanova et al. (2015)
utilize a convolutional neural network to capture the compositional structure
of textual relations, and jointly optimize entity, KBs, and textual relation
representations. Both studies are based on the concept of word2vec Mikolov et
al. (2013) and TransE Bordes et al. (2013).

After Google Inc. launched BERT in 2018, studies on KB/KG integration
gradually focused on optimization with pre-trained LMs. ERNIE Zhang et al.
(2019), an early study, encodes knowledge information in KGs via TransE Bordes
et al. (2013), a knowledge embedding model trained on Wikidata, and refines
BERT pre-training using named entity masking and phrase masking. K-BERT Liu et
al. (2019a) injects knowledge into the text to form a sentence tree without
the need to pre-train a model for knowledge embeddings, and adopts soft-
position embeddings and a visibility matrix for structural information and to
prevent diverting the sentence from its correct meaning. Based on these works,
KEPLER Wang et al. (2020) jointly optimizes knowledge embeddings and masks
language modeling objectives on pre-trained LMs.

Other work involves the joint use of dual BERTs. For example, Sentence-BERT
Reimers and Gurevych (2019) uses a model to derive sentence embeddings via
BERTs, and uses a classifier to judge the similarity of two sentences. DC-BERT
Zhang et al. (2020), a decoupled contextual encoding framework to address the
efficiency of information retrieval, uses an online BERT to encode the
question once, and an offline BERT which pre-encodes each document and caches
their encodings.

There are also some solutions to sequence length limitation, such as BigBird
Zaheer et al. (2020), Longformer Beltagy et al. (2020). Both of them address
the quadratic dependency on sequence length by using sparse attention
mechanism; while we adopt the full attention mechanism in our work.

##  3 Methodology

In this section, we describe in detail the framework of Roof-Transformer
presented in Fig. 1, including the input formats for Roof-Transformer, which
are sentences pairs and selected triples from KB.

###  3.1 Model Architecture

As shown in Fig. 1, the model architecture of Roof-Transformer contains three
stacked modules: (1) the underlying BERT model, responsible for encoding
tokens to meaningful representations; (2) the Fusion Layer, responsible for
combining information from the underlying BERT model; and (3) the Prediction
Layer, responsible for downstream tasks—in our case, QA and common natural
language understanding (NLU) tasks.

Underlying BERT model
The underlying BERT model is composed of two independent BERT models: TASK-
BERT and KB-BERT. TASK-BERT encodes the tokenized passages, which are
identical to those input for a single BERT on each downstream task, into
embeddings. KB-BERT encodes the tokenized triples from KBs to embeddings. Both
embeddings are then concatenated and fed to the fusion layer as input.

Fusion layer
We select Transformer Encoder (TE) Vaswani et al. (2017), LSTM Hochreiter and
Schmidhuber (1997) and Linear layer as the candidates of our Fusion layer. The
input of the fusion layer is the concatenation of the output embeddings from
TASK-BERT, $\mathit{Emb}\in\mathbb{R}^{M\times d}$, and the output embeddings
from KB-BERT, $\mathit{Emb}^{\prime}\in\mathbb{R}^{N\times d}$, where $d$ is
the hidden dimension of the word embeddings, $M$ is the length of the
tokenized question and paragraph, and $N$ is the length of the tokenized
triples from the KBs.

Prediction layer
The Prediction Layer is simply a linear NN layer, which is responsible for
transforming high-dimension embeddings into appropriate logits for prediction
and inference. The input of the prediction layer is the output embeddings of
the fusion layer, $\mathit{Emb}^{\prime\prime}\in\mathbb{R}^{(M+N)\times d}$,
whereas the output of the QA task is
$\mathit{logits}\in\mathbb{R}^{(M+N)\times 2}$. The two dimensions of the
output logits in each position are the start and end logits, that is, the
probability of whether the position is the start or the end position of the
answer. In other NLU tasks, the output embeddings of the fusion layer are
compressed to a single sequence length $\mathit{AvgEmb}\in\mathbb{R}^{d}$,
whereas the output is $\mathit{logits}\in\mathbb{R}^{e}$, where $e$ is based
on the number of classes of the predicted label.

The model parameters are updated by minimizing the cross-entropy loss between
the output logits and the ground truths.

###  3.2 TASK-BERT Input Format

In this paper, we test the capability of Roof-Transformer on a QA downstream
task and the GLUE NLU tasks. We follow the format of the input of the major
approach for BERT. Each sentence pair from the dataset contains two passages,
both of which are tokenized and concatenated with a [sep] token for input to
TASK-BERT.

Since BERT has a 512-token limitation on input length, we set a maximum length
for our question passage and paragraph passage. If the length of the passage
is shorter than the maximum length, which is generally the case in the
question passage, [pad] tokens are appended to the concatenated passage to fix
the length of every input. If the passage length exceeds the maximum length,
which is generally the case in paragraph passages, we truncate the passage.

###  3.3 KB-BERT Input Format

For our approach, we choose KBs as our external information. The KB contains
triples, each of which consists of a head, a relation, and a tail,
corresponding to the subject, the relation, and the object in a sentence.

The triples are selected via a heuristic algorithm (string match), which
selects a triple if its head exists in the paragraph passage in the TASK-BERT
input. The selected triples are then concatenated together and separated by
the [sep] token.

As shown in Fig. 2, we propose three expansion types—Exp0, Exp1, and Exp2—for
the selected triples. Exp0 simply concatenates the components in the triple as
a unit, and appends this to the previous unit. For Chinese KBs, Exp1 further
adds [的], a Chinese token, between the head and relation, and adds [是] between
the relation and tail to form a natural sentence Agarwal et al. (2021). For
English KBs, the input format will be equivalent to “head is a relation of
tail”. The sentence, which is also the unit, is then concatenated after the
previous unit. Exp2 is a refined version of Exp1: if the current head of the
selected triple is identical to the head of the previous unit, the head is
replaced by a pronoun and merged with the previous unit with a comma to form a
larger sentence/unit.

Figure 2: Three expansion types.

##  4 Experiments

In this section, we describe the Roof-Transformer training as well as the
fine-tuning results with different KBs and settings. The estimated number of
parameters ranges from 200M to 230M, depending on the depth and of candidate
the fusion layer. (see Appendix 6.2 for parameter setting details)

###  4.1 Dataset

For the QA task, we evaluated Roof-Transformer and the baseline models (Sec.
4.6) on the DRCD dataset Shao et al. (2019), a Chinese QA benchmark. For the
NLU tasks, we evaluated the model on the General Language Understanding
Evaluation (GLUE) benchmark Wang et al. (2018), which was used for ERNIE Zhang
et al. (2019). GLUE is an English multi-task NLU benchmark consisting of 11
tasks, of which we used 8 to evaluate Roof-Transformer and compare it with the
baseline models. These tasks use different evaluation metrics depending on
their purpose. (see Appendix 6.1 for statistical details of each dataset)

DRCD
The Delta Reading Comprehension Dataset is an open-source Chinese QA dataset
composed of paragraphs from Wikipedia articles and questions generated by
annotators. The ground truths of each question-paragraph pair are the start
and end position of the answer. The result is evaluated by the exact match
(EM) score.

GLUE
We selected the following eight English GLUE benchmark tasks: (1) SST-2, a
sentiment task using accuracy as the metric; (2) CoLA, an acceptability task
using Matthews correlation; (3) MRPC, a paraphrase task using the F1 score;
(4) STS-B, a sentence similarity task using Pearson–Spearman correlation; (5)
QNLI, a natural language inference (NLI) task using accuracy; (6) QQP, a
paraphrase task using the F1 score; (7) RTE, a NLI task using accuracy; and
(8) MNLI, a NLI task using accuracy.

###  4.2 Knowledge Base

We employ HowNet and CN-DBpedia, Chinese KBs which are refined and used in
K-BERT Liu et al. (2019a) for QA. Each triple in the KBs includes a head, a
relation, and a tail. We also use the KELM corpus as English KB to evaluate
the model on common NLU tasks in the GLUE benchmark.

CN-DBpedia
CN-DBpedia Xu et al. (2017) is a large-scale structured encyclopedia developed
and maintained by the Knowledge Workshop Laboratory of Fudan University. It
has been extended to fields such as law, industry, finance, and medical care,
providing supporting knowledge services for intelligent applications in
various industries. We use a refined version of CN-DBpedia by eliminating
triples whose entity names are less than 2 in length or contain special
characters like what was done in K-BERT. The refined CN-DBpedia contains
around 5M triples.

HowNet
HowNet Dong et al. (2006) is a large-scale KB containing Chinese concepts and
vocabulary. Each entity is annotated with semantic units called sememes. In
Hownet, sememes refer to some basic unit of senses. The triples in HowNet can
be represented as head, contain, sememes. We adopt the same method in CN-
DBpedia to obtain the refined version of HowNet, which contains a total of
52,576 triples.

KELM
The KELM corpus Lu et al. (2021) consists of the entire Wikidata KG as natural
text sentences. It contains around 15M sentences converted from KG’s triples.
The sentences are like the Exp2 examples in Fig. 2.

###  4.3 KB Format

Knowledge representation clearly effects model performance by influencing the
differentiation quality of language understanding. The knowledge selected also
effects model performance. Therefore, we evaluated 6 different input formats
for KB-BERT, that is, combinations of 3 kinds of knowledge representation with
2 kinds of knowledge selection.

Representation
We represented KB knowledge with 3 types of expansions as mentioned in Section
3.3 (Exp0, Exp1, Exp2) and demonstrated in Fig. 2, and evaluated the model
performance using these representations.

Selection
As mentioned in Section 3.3, a KB triple is selected if its head exists in the
paragraph passage, regardless of whether its tail exists in the paragraph:
this is denoted as as No_Tail. The other kind of selection is Has_Tail,
meaning that the selected triple’s head and tail are both present in the
paragraph passage. If the tail exists in the paragraph passage, the selected
triple is more likely to be related to the paragraph passage.

###  4.4 KB Encoder

Since the computational complexity of self-attention mechanism is _O(N 2)_,
where _N_ is the length of input tokens, acquiring knowledge embeddings can be
computationally expensive. Hence, apart from using KB-BERT with full-
attention, we also investigate the performance of our model with _cached_ KB-
BERT.

In cached KB-BERT experiment, we treated KB-BERT as an off-line BERT by
acquiring knowledge embeddings through a cached (freeze) BERT before training,
and the embeddings are then concatenated with the output of Task-BERT forming
the input of the fusion layer during training and inference. It is worth
noting that different from inputting the concatenated knowledge to KB-BERT all
at once, we feed each triple to cached KB-BERT to get its embedding and
dynamically concatenate all triples’ embeddings to feed to the fusion layer.
For example, take [cls] Bill Gates is a founder of Microsoft [sep] Elon Musk
is a founder of SpaceX [sep] as the original input; thus the individual input
of cached KB-BERT will be [cls] Bill Gates is a founder of Microsoft [sep] and
Elon Musk is a founder of SpaceX [sep]; then both output embeddings will be
concatenated to form the knowledge embeddings as part of the input of the
fusion layer (the other part is the passage embeddings from TASK-BERT).

###  4.5 Other Setting

We also conducted experiments to investigate the following factors which
influence fusion efficiency.

Segmentation
Since the input of the fusion layer contains outputs from both underlying BERT
models, each of which contains its own positional information, it is difficult
for the Fusion Layer to distinguish the two parts. Thus, we add segmentation
tokens to the KB-BERT and TASK-BERT input, and evaluate the two formats, which
are denoted as type-1 and type-2 segmentation, to determine which is more
effective.

The type-1 segmentation of KB tokens and padding tokens in KB-BERT is
different, using [A] and [B] respectively; in TASK-BERT, the segmentation of
question, paragraph, and padding tokens is [A], [B], [A] respectively. The
first type of segmentation aims to separate the content of tokens in a single
BERT.

As shown in Fig. 1, in the type-2 segmentation, every token in KB-BERT is set
to [A]; whereas in TASK-BERT, the segmentation of every token is set to [B].
With the second type of segmentation we seek to separate the content of tokens
of the two BERTs.

Fusion layer
We test different (1) Candidates (Linear, LSTM and Transformer encoder), (2)
Depth, (3) Initialization (pre-trained or not) and (4) Learning rate of Fusion
Layer.

It is worth noting that we modify the learning rate of Fusion Layer by
increasing it comparing to those of Underlying BERTs, e.g. 5, 10, 20 times of
the Underlying BERTs’ learning rate due to different model complexities of
BERTs (high) and Fusion Layer (low).

The investigations in Sec. 4.3 \- 4.5 are not only for better performance but
also aim to answer the the questions mentioned in Sec. 1.

Model |  Max Para. Len. |  Max Know. Len. | KB |  Fusion Layer Init. | EM score
---|---|---|---|---|---
BERTbase-chinese | 450 | 0 | - | - | 76.08
BERTbase-chinese† | 420 | 30 | HowNet | - | 76.20 $\uparrow$
BERTbase-chinese† | 400 | 50 | HowNet | - | 75.94 $\downarrow$
BERTbase-chinese† | 420 | 30 | CN-DBpedia | - | 75.63 $\downarrow$
BERTbase-chinese† | 400 | 50 | CN-DBpedia | - | 76.07 $\downarrow$
Roof-Transformer | 450 | 511 | HowNet | BERTbase-chinese | 77.45 $\uparrow$
Roof-Transformer | 450 | 511 | CN-DBpedia | BERTbase-chinese |  77.59 $\uparrow$
Roof-Transformer | 450 | 511 | HowNet | random weight | 76.31 $\uparrow$
Roof-Transformer | 450 | 511 | CN-DBpedia | random weight | 76.61 $\uparrow$

Table 1: Results of Roof-Transformer and baselines on QA tasks (%) with
different KBs, initialization (init.), max paragraph length (para. len.) and
max knowledge length (know. len.). Note that † indicates using knowledge in
single BERT architecture (baseline).

Model | KB | SST-2 | CoLA | MRPC | STS-B | QNLI | QQP | RTE | MNLI-m
---|---|---|---|---|---|---|---|---|---
BERTbase | - | 93.3 | 52.1 | 88.0 | 85.0 | 90.5 | 71.2 | 66.4 | 84.6
ERNIE | Wikidata | 93.5 | 52.3 | 88.2 | 83.2 | 91.3 | 71.2 | 68.8 | 84.0
Roof-Transformer | KELM | 93.0 | 54.4 | 89.0 | 84.2 | 90.6 | 70.3 | 69.0 | 84.3

Table 2: Results of Roof-Transformer and baselines on eight datasets of GLUE
benchmark (%)

###  4.6 Baseline

In QA task, we compare Roof-Transformer to two baselines: BERTbase-chinese
Devlin et al. (2018) without knowledge and a single BERTbase-chinese with
knowledge. In NLU tasks of GLUE, we compare Roof-Transformer to two baselines:
BERTbase without knowledge, and ERINE Zhang et al. (2019).

BERTbase-chinese is pre-trained on WikiZh; BERTbase, is pre-trained on the
BookCorpus and English Wikipedia; ERNIE is pre-trained on English Wikipedia
for large-scale textual corpora and Wikidata for KGs.

Figure 3: Comparison of performance of different roofs in QA task.

Figure 4: Fusion efficiency results on QA and GLUE tasks: The metrics in (a)
and (b) are both EM scores, whereas the metrics in (c) and (d) are accuracy
for RTE and Matthew’s correlation for CoLA. In (a) and (c), we use 10 times
the learning rate of the underlying BERT model in the TE layers; in (b), we
set the number of TE layers to 4; in (d), we set the number of TE layers to 3.

###  4.7 Results

Necessity of Dual BERT
As mentioned in Sec. 4.6, for QA task, we test baseline with knowledge which
encodes both text and knowledge with single BERT and without Fusion Layer.
That is, a partition of paragraph should be sacrificed for knowledge tokens.
(see Appendix 6.2 for details of input length settings)

As we can see in Table 1, sacrificing information of paragraph in exchange of
knowledge may be slightly beneficial in some cases; however, in most cases, it
results in performance drop. Therefore, the need of second BERT to encode
knowledge separately is guaranteed.

Candidate of Fusion Layer
As shown in Figure 3 (all settings are the same and follow Appendix 6.3), the
performance of every roof candidates beats both baselines (except a dramatic
drop in LSTM with the use of CN-DBpedia). Linear and LSTM both achieve better
performance compared to Transformer without pre-train while using HowNet. The
pre-trained Transformer consistently outperforms its variants with different
KBs (the Cached KB-BERT case will be discussed later). This shows the
effectiveness of our roof-architecture and the power of pre-trained
transformer, which was selected as our Fusion Layer for the rest of the
experiments.

QA performance
The best parameter setting for QA task we found through experiments are
presented in Appendix 6.3. As shown in Table 1, with external information from
KBs, the EM score improvement on the QA task exceeds 1.5%, which attests the
benefits of utilizing KBs. Perhaps the slight difference between the EM scores
of CN-DBpedia and HowNet in Table 1 is because CN-DBpedia has better and more
knowledge than HowNet.

KB format
In Table 3 we compare the results of the 6 different KB formats (setting
follows Appendix 6.3 except KB format). The KB format with Exp2 \+ Has_Tail
clearly yields the best performance. However, a closer look at the
contributions of knowledge representation and selection shows that Exp2 yields
marginally better results among three expansion types, producing finer
language representation; even more crucial, however, is whether the tails of
the selected triples are in the paragraph passage. The knowledge selection
comparison is shown in Table 4 (setting mentioned in Appendix 6.3 except
knowledge selection). The results indicate that selecting Has_Tail
knowledge—with tails in the paragraph passage—improves the EM score even with
far less knowledge, suggesting that adding arbitrary information could harm
performance.

Depth of Fusion Layer
As reported in Figures 4(a) and 4(c), the scores peak when using the last 4
and 3 TE layers from the pre-trained BERT as the Fusion layer. Using more
layers consumes excessive memory and yields lower scores, perhaps due to
overfitting; when using too few layers, the fusion layer is unable to
effectively integrate language representation with the KBs.

Type | Has_Tail | No_Tail
---|---|---
Exp0 | 77.21 | 77.34
Exp1 | 76.92 | 76.77
Exp2 | 77.45 | 77.03

Table 3:  EM scores (%) for 6 types of KB selection and representation on QA
task using HowNet as KB.

KB | Selection | Length | EM score (%)
---|---|---|---
HowNet | Has_Tail | 29 | 77.45
HowNet | No_Tail | 131 | 77.03
CN-DBpedia | Has_Tail | 24 | 77.59
CN-DBpedia | No_Tail | 168 | 76.90

Table 4:  Results with different knowledge selections, where Length is the
average length of the input tokens of KB-BERT (w/o [pad]) during training.

Learning rate of Fusion Layer
As reported in Fig. 4(b), using 10 times the learning rate of the underlying
BERT model’s learning rate yields the highest EM score, for an increase of
over 1% compared to using the same learning rate as the underlying BERT model.
Similarly, as reported in Fig. 4(d), using 10 times learning rate yields an
increase of 2%. This shows that a higher learning rate in the fusion layer
improves the fusion effectiveness and further enhances prediction. This is
because after BERT is pre-trained, a small learning rate is sufficient for
fine-tuning on downstream tasks, and requires fewer epochs for its loss to
converge to a minima (global or local); by contrast, a higher learning rate
could prevent the model from converging. Thus it may be that the learning pace
of the fusion layer should be faster than that of the underlying BERT model in
order to make best use of the information provided by each underlying BERT
instead of being confined to one or the other.

Cached KB-BERT
As shown in Figure 3, cahcing KB-BERT does not lead to performance drop but
even slightly improve around 0.1% (77.67%) with the use of Cn-Dbpedia
comparing to our original best setting. A possible explanation is that without
full self-attention between knowledge triples, it could reduce noise during
encoding the triples. Moreover, without updating parameters of KB-BERT during
fine-tuning, it reduces 33% of the computational cost, making prediction and
inference more efficient.

GLUE
The best parameter setting for NLU tasks are presented in Appendix 6.3. As
shown in Table 2, Roof-Transformer outperforms datasets like RTE, CoLA, and
MRPC, which shows that the proposed model effectively integrates knowledge and
contextual embeddings. Moreover, Roof-Transformer achieves comparable or even
superior results with ERNIE on the GLUE benchmark.

The results also show that Roof-Transformer has no significant effect on tasks
like SST-2, STS-B, and QNLI. It is probably due to the need of external
information for tasks like sentimental analysis: sentence sentiment is
determined by emotional words but not knowledge. This phenomenon is reflected
also in studies like ERNIE and K-BERT. However, for these tasks, the proposed
model still achieves performance comparable to that of BERT. For tasks that
our model beats both ERINE and BERT like RTE, we have conducted case study to
further investigate how knowledge effects prediction.

###  4.8 Case Study

We conduct case studies on DRCD dataset and RTE dataset in GLUE benchmark to
find out how knowledge helps or misleads our model. We select three examples
from dev (validation) data where two of them are positive samples (our model
answers correctly, while the baseline answers incorrectly) and one of them is
a negative samples (our model answers incorrectly, while the baseline answers
correctly) in both case studies. The details of examples and analysis are
presented in Appendix 6.4.

##  5 Conclusion

In this paper, we propose Roof-Transformer to encode knowledge and input
sentences using two underlying BERTs with a fusion layer (transformer encoder)
on top. This architecture relaxes BERT’s length limitation, allowing BERT to
use more knowledge and longer input texts. Experimental results on a QA task
and the GLUE benchmark demonstrate the model’s effectiveness. We also show
that precise knowledge selection is critical under this architecture. Roof-
Transformer is a general and powerful method for language understanding which
can easily be applied to other NLP tasks. It is likely to especially benefit
tasks which require information from a large range of knowledge or long input
texts. For future work, we plan to replace the underlying BERTs to other pre-
trained LMs, such as RoBERTa Liu et al. (2019b) and XLNet Yang et al. (2019)
to to consolidate the effectiveness of the proposed framework.

## References

  * Agarwal et al. (2021) Oshin Agarwal, Heming Ge, Siamak Shakeri, and Rami Al-Rfou. 2021.  Knowledge graph based synthetic corpus generation for knowledge-enhanced language model pre-training.  _arXiv preprint arXiv:2010.12688v2_.
  * Beltagy et al. (2020) Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020.  Longformer: The long-document transformer.  _arXiv preprint arXiv:2004.05150_.
  * Bordes et al. (2013) Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013.  Translating embeddings for modeling multi-relational data.  _Proceedings of NeurIPS_ , pages 2787–2795.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_.
  * Dong et al. (2006) Zhendong Dong, Qiang Dong, and Changling Hao. 2006.  Hownet and the computation of meaning.  _Citeseer_.
  * Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber. 1997.  Long short-term memory.  _Neural Computation_ , pages 1735–1780.
  * Liu et al. (2019a) Weijie Liu, Peng Zhou, Zhe Zhao, Zhiruo Wang, Qi Ju, Haotang Deng, and Ping Wang. 2019a.  K-bert: Enabling language representation with knowledge graph.  _arXiv preprint arXiv:1909.07606_.
  * Liu et al. (2019b) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Lu et al. (2021) Yinquan Lu, Haonan Lu, Guirong Fu, and Qun Liu. 2021.  Kelm: Knowledge enhanced pre-trained language representations with message passing on hierarchical relational graphs.  _arXiv preprint arXiv:2109.04223_.
  * Mikolov et al. (2013) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013.  Efficient estimation of word representations in vector space.  _arXiv preprint arXiv:1301.3781_.
  * Rajpurkar et al. (2016) Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016.  Squad: 100,000+ questions for machine comprehension of text.  _Proceedings of EMNLP_ , pages 2383–2392.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-bert: Sentence embeddings using siamese bert-networks.  _arXiv preprint arXiv:1908.10084v1_.
  * Shao et al. (2019) Chih Chieh Shao, Trois Liu, Yuting Lai, Yiying Tseng, and Sam Tsai. 2019.  Drcd: a chinese machine reading comprehension dataset.  _arXiv preprint arXiv:1806.00920v3_.
  * Toutanova et al. (2015) Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, and Michael Gamon. 2015.  Representing text for joint embedding of text and knowledge bases.  _Proceedings of EMNLP_ , pages 1499–1509.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhi. 2017.  Attention is all you need.  _Proceedings of NeurIPS_ , pages 5998–6008.
  * Wang et al. (2018) Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2018.  Glue: A multi-task benchmark and analysis platform for natural language understanding.  _arXiv preprint arXiv:1804.07461_.
  * Wang et al. (2020) Xiaozhi Wang, Tianyu Gao, Zhaocheng Zhu, Zhengyan Zhang, Zhiyuan Liu, Juanzi Li, and Jian Tang. 2020.  Kepler: A unified model for knowledge embedding and pre-trained language representation.  _arXiv preprint arXiv:1911.06136v3_.
  * Wang et al. (2014) Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014.  Knowledge graph and text jointly embedding.  _Proceedings of EMNLP_ , pages 1591–1601.
  * Xu et al. (2017) Bo Xu, Yong Xu, Jiaqing Liang, Chenhao Xie, Bin Liang, Wanyun Cui, and Yanghua Xiao. 2017.  Cn-dbpedia: A never-ending chinese knowl- edge extraction system.  _International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems_ , pages 428–438.
  * Yang et al. (2019) Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019.  Xlnet: Generalized autoregressive pretraining for language understanding.  _Proceedings of NeurIPS_.
  * Zaheer et al. (2020) Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, and Amr Ahmed. 2020.  Big bird: Transformers for longer sequences.  _Proceedings of NeurIPS_ , pages 17283–17297.
  * Zhang et al. (2020) Yuyu Zhang, Ping Nie, Xiubo Geng, Arun Ramamurthy, Le Song, and Daxin Jiang. 2020\.  Dc-bert: Decoupling question and document for efficient contextual encoding.  _arXiv preprint arXiv:2002.12591v1_.
  * Zhang et al. (2019) Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. 2019.  Ernie: Enhanced language representation with informative entities.  _Proceedings of ACL_ , pages 1441–1451.

##  6 Appendices

###  6.1 Dataset Statistics

Dataset | Train | Validation | Test
---|---|---|---
QA task
DRCD | 26,935 | 3,523 | 3,492
GLUE benchmark
SST-2 | 67,349 | 872 | 1,821
CoLA | 8,551 | 1,043 | 1,063
MRPC | 3,668 | 408 | 1,725
STS-B | 5,749 | 1,500 | 1,379
QNLI | 104,743 | 5,463 | 5,463
QQP | 363,846 | 40,430 | 390,965
RTE | 2,490 | 277 | 3,000
MNLI-m | 392,702 | 9,832 | 9,847

Table 5:  Number of instances in train-dev-test split of different datasets.

###  6.2 Parameter Settings

For QA, given BERT’s input length limit, we set the maximum length of the
question and paragraph to 59 and 450, respectively so that the total length of
the input token in Task-BERT would be len([cls]) + len(Question tokens) +
len([sep]) + len(Paragraph tokens) + len([sep]) = 1 + 59 + 1 + 450 + 1 = 512.
Similarly, we set the maximum length of the knowledge as 511, so that the
total length of the input token in KB-BERT would be len([cls]) + len(Knowledge
tokens) = 1 + 511 = 512.

The following setting values were found suitable for the QA datasets: a batch
size of 16 and an AdamW learning rate of $3e^{-5}$. In addition, we used the
linear learning rate decay scheduler. For the epoch count, as the fine-tuning
was for a downstream QA task, the epoch count was set to 1 with the training
loss and accuracy converging properly.

For the GLUE NLU tasks, we set the maximum length of the sentence pairs to
512, including one [cls] token and two [sep] tokens. If the sentence length
was less than 512, we appended [pad] tokens to the sentences until the
sentence length was 512.

The following setting values were found suitable for the GLUE benchmark: a
batch size of 16 and a learning rate of $2e^{-5}$. We used the cosine learning
rate decay scheduler. The training epochs were set to 5 for fine-tuning.

###  6.3 Experiment Settings

For QA task, we find the following setting performs the best in Roof-
Transformer: (1) KB format with Exp2 and Has_Tail, (2) type-2 segmentation,
(3) using pre-trained Transformer encoder layer (last k layers from BERTbase-
chinese) for Fusion Layer, (4) 4 Fusion Layers (k = 4) , (5) using 10 times of
the Underlying BERTs model’s learning rate in Fusion Layer.

For NLU tasks of GLUE, only the number of Fusion Layers is found to be 3 (k =
3) for best performance; other settings remain the same as QA task.

###  6.4 Case Study

QA task
The Question-Answering is the task of answering a Question from a given
Paragraph.

As shown in Fig. 5. In the first positive example, knowledge provides
information that both the Chahar People’s Anti-Japanese Allied Army and the
Eighth Route Army are kinds of armies, enabling Roof-Transformer to understand
these unseen named entities, which indirectly aids prediction. Similarly, in
the second positive example, Roof-Transformer also benefits from the added
knowledge, which makes it aware of the functions and character of the various
named entities (aircraft carrier, destroyer, …) in the paragraph. In contrast,
without KB knowledge, the baseline BERTbase-chinese fails to understand the
contextual information of the given question and paragraph, leading to
misprediction.

On the other hand, in the negative example, the selected knowledge indicates
that political reform is a kind of issue, which subtly response to the word
"contribution" asked in the question, resulting in a biased prediction. On the
contrary, without the information of knowledge, the baseline model is able to
answer correctly.

RTE task
The Recognizing Textual Entailment is the task of determining whether the
meaning of the Hypothesis is entailed (can be inferred) from given Text.

As shown in Fig. 6. In the first positive example, the knowledge tells the
fact of "Texaco is owned by Chevron Corporation", which helps our model to
infer that sentence 1 entails sentence 2 instead of telling model the answer.
Similarly, in the second positive example, Roof-Transformer also benefits from
the fact related to Romano Prodi that knowledge provides. In contrast, without
the help of knowledge corpus, the baseline BERTbase fails to make inference
only based on sentence 1 and sentence 2.

On the other hand, in the negative example, although the selected knowledge
tells fact about "Lex Lasry", there exista a huge overlap between knowledge
and sentence 2 (i.e., "is a lawyer"), resulting in a misprediction. On the
contrary, without the information of knowledge, the baseline model is able to
make correct inference.

Figure 5: Case study of QA task: Answers with underlines are also the ground
truth of those examples. The answers of Roof-Transformer are align with the
ground truth in positive examples, whereas answer of baseline is align with
the ground truth in negative examples. Note that the content is translated
(without loss in meaning), since the DRCD dataset and HowNet are in Chinese,
and only a part of paragraph and knowledge are present for clarity.

Figure 6: Case study of RTE task: Similarly, the answers with underlines are
the ground truth of those examples. Note that the content is originally in
English, and only a part and knowledge is present for clarity.


# A Study on Social Robot Behavior in Group Conversation

Tung Nguyen1, Eric Nichols1, and Randy Gomez 1 1Honda Research Institute Japan
Co., Ltd., 8-1 Honcho, Wakoshi, Japan
<EMAIL_ADDRESS>
###### Abstract

Recently, research in human-robot interaction began to consider a robot’s
influence at the group level. Despite the recent growth in research
investigating the effects of robots within groups of people, our overall
understanding of what happens when robots are placed within groups or teams of
people is still limited. This paper investigates several key problems for
social robots that manage conversations in a group setting, where the number
of participants is more than two. In a group setting, the conversation
dynamics are a lot more complicated than the conventional one-to-one
conversation, thus, there are more challenges need to be solved.

##  I INTRODUCTION

In this paper, we discuss several key issues in terms of interaction
management for a social robot that facilitates conversation in a group
setting. In contrast to the conventional one-to-one conversation, which is the
major research paradigm in human-robot interaction research, a group
conversation, usually called multi-party dialog, involves three or more
participants. As the number of interlocutors increases, the dynamics of the
conversation also increase in terms of complexity, thus, making the
facilitation task of the robot in a multi-party dialog very challenging.
Despite the increase of recent research studying the effects of robots within
groups of people, the overall understanding of what happens when robots
participate in a conversation within groups of people and what the robot
should do in this situation is still limited.

In principle, the handling of a multi-party dialog involves two phases:
observation and interaction management. In the observation phase, the robot
observes interactions among the other interlocutors to gather visual,
acoustic, and verbal information of the current situation. Depending on the
goal of the robot, the raw information from sensors are processed to extract
the dynamics of the conversation such as who is the speaker, or who is being
addressed, engagement level of each participants [1, 2]. In the management
phase, the robot performs facilitation behaviors accordingly to the perceived
information and its pre-defined goal. Our work in this paper focuses on the
phase of interaction management. The goal of a social robot that facilitates
multi-party dialog is to maintain the flow of the conversation and make all
the participants feel satisfied.

Traum [3] discussed several key issues of a dialog system’s interaction
management in a group conversation setting, which includes: turn management,
channel management, thread/conversation management, initiative management, and
attention management. Each of these issues has been investigated in multiple
existing studies. [4, 5] proposed methods for handling turn taking in multi-
party conversations. Matsuyama et al. [6] proposed a POMDP-based model that
tackles the issue of initiative management to control the conversation flow
among the participants.

In this work, we discuss several important interaction management issues that
a social robot needs to manage in a group conversation. The remainder of this
paper is organized as follows. In Section II, we start by summarizing studies
about facilitation in groups interaction. In Section III, we discuss about key
issues for robot to handle interaction management of group conversations
effectively. Finally, we summarize and conclude this study in Section IV.

##  II RELATED WORKS

There are various research on specially situated facilitation agents in
multiparty conversations. Ishizaki et al. [7] studied about differences in
characteristics between dyadic (one-to-one) and multiparty conversations,
making foundation for the later works. Interestingly, even though there are
more research focus on robot behaviors in dyadic conversations, recent surveys
show that humans are more likely to interact with a robot as a group [8, 9].
Matsusaka et al. [10] were among the first to propose the utilization of a
physical robot to participate in multiparty conversations. Traum [3] provided
the first analysis of problems in handling conversation among group of
interlocutors. Matsuyama et al. [11] developed a multiparty quiz-game-type
facilitation system for elderly. Another study reported the effectiveness of
the existence of a robot in quiz-type group in terms of communication
activation [12]. Sidner et al. [13] focused on engagement, a key factor to
determine level of interaction among participants in conversation, which is
defined as “the process by which two (or more) participants establish,
maintain and end their perceived connection during interactions they jointly
undertake”. Based on this definition, they developed a robot that can maintain
high level of engagement with human users using gaze behavior.

Using the same definition as in Sidner’s work, Bohus and Horvitz [14, 15]
evaluated the effectiveness of multimodality behavior model, which includes
gaze, gesture, and speech, for a multiparty conversation facilitating agent.
Kumar et al. [16] developed a model that selects appropriate dialogue action
for facilitation in a tutoring scenario based on Bales’ Socio-Emotional
Interaction Categories for text-based character agents. Recent studies show
that robot can be effectively utilized to resolve conflicts among group
members. For example, robots have demonstrated success in directly mediating
children fighting over the same toy [17]. Martelaro et al. [18] study showed
that a robot intervening after a team member made a hostile remark toward the
other helps increase the awareness of such conflict. However, the attempts of
the robot to solve the conflict sometimes backfired. Thus, the behavior of the
robot must be designed carefully in such cases to avoid heightening the level
of the conflicts.

##  III ISSUES IN GROUP CONVERSATION MANAGEMENT

In this section, we describe the key issues of interaction management in
multi-party conversation, which are: turn management, thread management,
initiative management, and attention management. In order to handle multi-
party dialog effectively and keep a high level of satisfaction among the human
participants, a robot must be able to handle these key issues. The issues
being discussed in this section also appear in dyadic conversation, however,
they become a lot more complex when being considered in the context of multi-
party conversations.

###  III-A Turn management

Turn management refers to the problem of handling turn-taking in
conversations, which means when to stop and when to start talking to the other
participants. In a conventional setup, we force a rigid turn-taking manner,
when a participant can only start talking when the other has finished their
turn. However, there is another setup where the participant can “barge-in”,
which means to take the turn before the other finish talking. This is an
important issue because turn is the basic unit of a conversation, thus,
managing conversation turns is essential for controlling the flow of the
conversation.

In dyadic conversations, turn taking management is straight-forward because
each participant takes a conversation turn after the other finish. As a
result, the number of turns that each participant take during the conversation
is usually balanced. In a multi-party dialog setting, when someone finish
speaking, there are many participants that can take the turn to reply. As a
result, there are cases where someone takes a lot of turns while the others
have less. For a social robot that handles the conversation, such situation
should be avoid, and the robot should intervene and prevents it from
happening. Table I shows an example of the robot taking turn and joins the
conversation with the other participants. When the robot should take the turn
is important in facilitating the conversation so that all participants do not
feel being interrupted and satisfied.

TABLE I: An example of a robot taking a turn in a multiparty conversation.
Participants: A, B, and the robot
---
A: I am going to visit Italy this summer.
B: Oh, really? I think it’s a beautiful country. Where in Italy are you going
to stay?
A: I’m staying in Rome but I’ll also visit Naples and Venice as well.
B: That sounds like an amazing trip!
A: I’m looking forward to try out all the Italian food there, too.
Robot: If you’re staying in Rome, I recommend restaurant ABC. They have
amazing carbonara there!

###  III-B Thread management

Thread management refers to the management of the conversation topics that are
being discussed at the moment. In a one-to-one conversation, there are only
two participants, thus, there can be only one topic exists at one time.
Therefore, the management of topics is trivial in this case. However, in
multi-party conversations, there can be multiple topics being discussed at the
same time between multiple participants. For example, consider the case where
we have five participants: A, B, C, D, and the robot. A and B are talking
about football while C and D are talking about traveling. The robot needs to
keep track of who is talking about which topic and gives the reply to the
participants appropriately. An example of the robot handling topics of the
conversation is shown in Table III, when the robot joins the discussion of A
and B, it talks about football and not about traveling.

TABLE II: An example of the robot handling topics in multi-party conversation.
Participants: A, B, and the robot
---
Topic: football
A: Do you like football, B?
B: I like watching football but I don’t play it.
A: Who is your favorite football player? I like Messi.
B: I don’t have a favorite player but I like Real Madrid, they’re my favorite
football club since high school.
Robot: I like Real Madrid, too!
Participants: C, D, and the robot
Topic: traveling
C: I’m going to Tokyo next month.
D: That’s nice, what do you plan to do in Tokyo?
C: I like Japanese culture and want to visit the shrines and temples.
Robot: Let me see. Senso-ji in Asakusa is a really famous temple in Tokyo that
you can visit.

###  III-C Initiative management

Initiative management concerns about which participant is currently setting
the agenda for topics of discussions. In a dyadic conversation between robot
and human, it is either robot-initiative (the robot starts the topics), human-
initiative (the human user starts the topics), or mixed-initiative (both human
and robot can start the topics). In multi-party dialogs, another type of
initiative taking called cross-initiative can happen, where one participant
asks another one to start a new topic for the conversation.

For a robot that facilitates multi-party conversations, similar to turn
management, it is important to make sure that there is no severe imbalance in
terms of initiative taking. When necessary, the robot should take initiative
and start a new topic of conversation. By doing this, we can keep all
participants engaged and make the conversation goes smoothly.

TABLE III: An example of the robot handling initiative in multi-party
conversation. Participants: A, B, and the robot
---
Topic: football
A: Do you like football, B?
B: I like watching football but I don’t play it.
A: Who is your favorite football player? I like Messi.
B: I don’t have a favorite player but I like Real Madrid, they’re my favorite
football club since high school.
…
conversation continues about football
…
Robot: B, do you have any favorite kind of food?

###  III-D Attention management

Attention management in a dyadic conversation, the attention of the robot is
always toward the only human user. However, in a multiparty dialog, there are
multiple humans that the robot needs to give a response to while the robot can
only give one response at a time. The task of who should the robot attend to
at which scenario is important to facilitate the conversation successfully.
The robot needs to assess the situations carefully and determine which
participant to respond next. Failure to give appropriate attention to human
participants can lead to dissatisfaction. In some cases, such as managing
conversations among children, failure in attention management can lead to
conflicts among the participants. Table IV shows an example where the robot
needs to handle the attention from two participants.

TABLE IV: An example of the robot handling attention in multi-party
conversation. Participants: A, B, and the robot
---
A: Do you like football, B?
B: I like watching football but I don’t play it.
A: Who is your favorite football player? I like Messi.
B: I don’t have a favorite player but I like Real Madrid, they’re my favorite
football club since high school.
A: Robot, is there any interesting football match this weekend?
B: I am looking for a good Indian restaurant, can you give me some
recommendation, robot?
Robot: There is Namaste restaurant near the station.

##  IV CONCLUSIONS

In this paper, we discuss several key issues that a social robot that
facilitates multi-party conversations needs to take care of. In principle, to
handle a multi-party dialogs, a robot needs to tackles problems in terms of
turn management, thread management, initiative management, and attention
management. Solving these challenges is essential for a robot to successfully
facilitating a multi-party conversation.

In the future, we plan to analyse these key issues in more details and design
behavior strategies for a social robot that manages multi-party conversations.

## References

  * [1] S. Gupta, J. Niekrasz, M. Purver, and D. Jurafsky, “Resolving “you” in multi-party dialog,” in _Proceedings of the 8th SIGdial Workshop on Discourse and Dialogue_ , 2007, pp. 227–230.
  * [2] J.-C. Gu, C. Tao, and Z.-H. Ling, “Who says what to whom: A survey of multi-party conversations,” in _Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI_ , 2022, pp. 5486–5493.
  * [3] D. Traum, “Issues in multiparty dialogues,” in _Advances in Agent Communication: International Workshop on Agent Communication Languages, ACL 2003, Melbourne, Australia, July 14, 2003. Revised and Invited Papers_. Springer, 2004, pp. 201–211.
  * [4] Y. Matsusaka, S. Fujie, and T. Kobayashi, “Modeling of conversational strategy for the robot participating in the group conversation,” in _Seventh European conference on speech communication and technology_ , 2001.
  * [5] D. Bohus and E. Horvitz, “Multiparty turn taking in situated dialog: Study, lessons, and directions,” in _Proceedings of the SIGDIAL 2011 Conference_ , 2011, pp. 98–109.
  * [6] Y. Matsuyama, I. Akiba, S. Fujie, and T. Kobayashi, “Four-participant group conversation: A facilitation robot controlling engagement density as the fourth participant,” _Computer Speech & Language_, vol. 33, no. 1, pp. 1–24, 2015.
  * [7] M. Ishizaki and T. Kato, “Exploring the characteristics of multi-party dialogues,” in _36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume 1_ , 1998, pp. 583–589.
  * [8] R. Gockley, J. Forlizzi, and R. Simmons, “Interactions with a moody robot,” in _Proceedings of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction_ , 2006, pp. 186–193.
  * [9] M. P. Michalowski, S. Sabanovic, and R. Simmons, “A spatial model of engagement for a social robot,” in _9th IEEE International Workshop on Advanced Motion Control, 2006._ IEEE, 2006, pp. 762–767.
  * [10] Y. Matsusaka, T. Tojo, and T. Kobayashi, “Conversation robot participating in group conversation,” _IEICE TRANSACTIONS on Information and Systems_ , vol. 86, no. 1, pp. 26–36, 2003.
  * [11] Y. Matsuyama, H. Taniyama, S. Fujie, and T. Kobayashi, “Designing communication activation system in group communication,” in _Humanoids 2008-8th IEEE-RAS International Conference on Humanoid Robots_. IEEE, 2008, pp. 629–634.
  * [12] Y. Matsuyama, S. Fujie, H. Taniyama, and T. Kobayashi, “Psychological evaluation of a group communication activation robot in a party game,” in _Eleventh Annual Conference of the International Speech Communication Association_ , 2010.
  * [13] C. L. Sidner, C. D. Kidd, C. Lee, and N. Lesh, “Where to look: a study of human-robot engagement,” in _Proceedings of the 9th international conference on Intelligent user interfaces_ , 2004, pp. 78–84.
  * [14] D. Bohus and E. Horvitz, “Models for multiparty engagement in open-world dialog,” in _Proceedings of the SIGDIAL 2009 Conference, The 10th Annual Meeting of the Special Interest Group on Discourse and Dialogue_ , 2009, p. 10.
  * [15] ——, “Facilitating multiparty dialog with gaze, gesture, and speech,” in _International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction_ , 2010, pp. 1–8.
  * [16] R. Kumar, J. Beuth, and C. Rose, “Conversational strategies that support idea generation productivity in groups,” 2011.
  * [17] S. Shen, P. Slovak, and M. F. Jung, “” stop. i see a conflict happening.” a robot mediator for young children’s interpersonal conflict resolution,” in _Proceedings of the 2018 ACM/IEEE international conference on human-robot interaction_ , 2018, pp. 69–77.
  * [18] N. Martelaro, M. Jung, and P. Hinds, “Using robots to moderate team conflict: The case of repairing violations,” in _Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction Extended Abstracts_ , 2015, pp. 271–271.