# Contrastive Learning Reduces Hallucination in Conversations

Weiwei Sun1, Zhengliang Shi1, Shen Gao1, Pengjie Ren1, Maarten de Rijke2,
Zhaochun Ren1 Corresponding author.

###### Abstract

Pre-trained language models (LMs) store knowledge in their parameters and can
generate informative responses when used in conversational systems. However,
LMs suffer from the problem of “hallucination:” they may generate plausible-
looking statements that are irrelevant or factually incorrect. To address this
problem, we propose a contrastive learning scheme, named MixCL. A novel mixed
contrastive objective is proposed to explicitly optimize the implicit
knowledge elicitation process of LMs, and thus reduce their hallucination in
conversations. We also examine negative sampling strategies of retrieved hard
negatives and model-generated negatives. We conduct experiments on Wizard-of-
Wikipedia, a public, open-domain knowledge-grounded dialogue benchmark, and
assess the effectiveness of MixCL. MixCL effectively reduces the hallucination
of LMs in conversations and achieves the highest performance among LM-based
dialogue agents in terms of relevancy and factuality. We show that MixCL
achieves comparable performance to state-of-the-art KB-based approaches while
enjoying notable advantages in terms of efficiency and scalability.

##  1 Introduction

Open-domain dialogue agents have received increasing attention in recent years
(Freitas et al. 2020; Huang, Zhu, and Gao 2020). In an engaging open-domain
dialogue, a large amount of knowledge, such as commonsense (Young et al. 2018)
and factual knowledge (Dinan et al. 2019), is involved. To integrate knowledge
into dialogue agents, _KB-based methods_ have been proposed to explicitly
acquire knowledge from knowledge bases (Young et al. 2018; Dinan et al. 2019).
However, KB-based methods suffer from problems of retrieval error (Liu et al.
2022) and inefficiency (Xu et al. 2022). Meanwhile, recent years have
witnessed a rapid development of pre-trained language models (LMs) (Devlin et
al. 2019; Brown et al. 2020) and their applications to dialogue tasks
(Thoppilan et al. 2022). Large LMs implicitly store knowledge in their
parameters during the pretraining stage (Petroni et al. 2019; Zhou et al.
2020) and thus, to some extent, they can serve as knowledge bases to ground
open-domain dialogues (Zhao, Wu, and Xu 2020). Such approaches, known as _LM-
based methods_ , achieve promising performance in generating informative
responses and obviate the drawbacks of KB-based methods. However, LM-based
methods have the problem of “hallucination” (Shuster et al. 2021; Ji et al.
2022): they generate plausible-looking statements that are irrelevant or
factually incorrect.

Figure 1:  Results of a pilot experiment where annotators were asked to label
200 responses generated by BART on the Wizard-of-Wikipedia dataset for
hallucination.

To understand the severity of hallucinations of LMs, we conduct a pilot
experiment. We sample 200 responses generated by BART (Lewis et al. 2020) on
the Wizard-of-Wikipedia dataset (Dinan et al. 2019) for various topics and
conversation turns. These responses are annotated by three well-informed
experts in terms of knowledge relevancy and factuality. Based on the results,
we group the hallucinations of LMs into two types: _intrinsic hallucinations_
and _extrinsic hallucinations_. Intrinsic hallucinations are non-factual
statements, such as incorrectly predicting a celebrity’s birthday. Extrinsic
hallucinations are irrelevant or out-of-context responses, such as the a
description of the history of football when the user asks the number of teams
currently in the NFL. Fig. 1 summarizes the outcomes: intrinsic and extrinsic
hallucinations account for 24% and 27% of the responses, respectively.

The problem of hallucinations is mainly attributable to the optimization
recipes: the commonly used maximum likelihood estimation (MLE) with teacher
forcing training encourages the model to imitate the training data blindly,
leading to model hallucinations at inference time (Kang and Hashimoto 2020).
Most studies on tackling hallucination in conversations focus on KB-based
methods and use pre-retrieval (Shuster et al. 2021) or post-editing techniques
(Dziri et al. 2021) to improve faithfulness; the hallucination of LM-based
agents in eliciting knowledge inside LMs’ parameters is still underexplored.

In this paper, we propose Mixed Contrastive Learning (MixCL) to alleviate the
hallucinations of LM-based dialogue agents. MixCL explicitly samples the most
confusing knowledge to the model and reduces its generation probability by
contrasting it with the ground-truth. To this end, two novel steps are used by
MixCL:  (i) negative sampling, and  (ii) mixed-contrastive learning.  In the
former, we sample the most confused negative knowledge by retrieving from the
corpus or deriving via model bootstrapping. In the latter, we propose mixed-
contrastive learning under the inspiration of mix-up data augmentation (Zhang
et al. 2018), which mixes the positive and negative at span level. Moreover,
we propose two mixed strategies regarding the two types of hallucination:
entity-based mix-up and constituency-based mix-up. Finally, MixCL is optimized
in an end-to-end manner, thus avoiding the retrieval step during inference and
instead using the knowledge inside its parameters.

We conduct experiments on Wizard-of-Wikipedia (Dinan et al. 2019), an open-
domain, knowledge-grounded dialogue dataset. Extensive experiments show that
MixCL improves the informativeness and relevancy of the responses. Compared
with previous LM-based methods (Zhao, Wu, and Xu 2020; Xu et al. 2022; Liu et
al. 2022), MixCL achieves improvements by 5% to 15% in terms of response
quality and relevancy. Moreover, MixCL achieves comparable performance as
state-of-the-art KB-based methods (e.g., KnowledGPT (Zhao et al. 2020)), while
speeding up 5$\times$ in model inference and showing superior scalability. The
effectiveness of MixCL is also verified through human evaluation and ablation
experiments.

Our contributions are as follows:  (i) We propose MixCL, which reduces
hallucinations of LMs in conversation through contrastive learning.  (ii) We
propose a hard negative sampling strategy to obtain the most confused negative
knowledge (see Section 5.1).  (iii) We propose a mix contrastive objective to
optimize the model at span level (see Section 5.2).  (iv) Experiments on the
Wizard-of-Wikipedia dataset show that MixCL effectively reduces the
hallucinating content produced by the LM and achieves comparable performance
to KB-based approaches.111We release our code at
https://github.com/sunnweiwei/MixCL.

##  2 Related work

###  2.1 Knowledge-grounded dialogues

In open-domain knowledge-grounded dialogues (KGDs), people respond to each
other’s utterances in a meaningful way by integrating knowledge (Young et al.
2018; Huang, Zhu, and Gao 2020). To integrate knowledge, _KB-based methods_
have been explored (Liu et al. 2018; Young et al. 2018; Dinan et al. 2019);
they retrieve knowledge from a corpus through additional information retrieval
(IR) modules. Studies on KB-based methods focus on knowledge selection (Meng
et al. 2020; Shuster et al. 2021) and knowledge-grounded response generation
(Zhao et al. 2020; Zheng and Huang 2021). However, KB-based methods suffer
from the problems of retrieval errors (Liu et al. 2022), inefficiencies (Xu et
al. 2022), and multi-granularity knowledge integration (Wu et al. 2022).

###  2.2 Language models as knowledge bases

Recent years have witnessed a rapid development of language models (LMs)
(Brown et al. 2020) and LM-based dialogue agents (Thoppilan et al. 2022).
Large LMs store knowledge into their parameters during pre-training and can
generate informative responses in conversations (Zhao, Wu, and Xu 2020).
Petroni et al. (2019) show that LMs can serve as knowledge bases for
downstream tasks (e.g., question answering (Roberts, Raffel, and Shazeer
2020)). On this basis, Zhao, Wu, and Xu (2020) show that LMs can ground open-
domain dialogues using their implicit knowledge. Madotto et al. (2020) embed
knowledge bases into model’s parameters for end-to-end task-oriented
dialogues. Roller et al. (2021) finetune LMs on KGD data. Cui et al. (2021)
propose knowledge-enhanced finetuning methods to handle unseen entities. Xu et
al. (2022) propose a topic-aware adapter to adapt LMs in KGDs. Liu et al.
(2022) propose a multi-stage prompting approach for triggering knowledge in
LMs. Wu et al. (2022) propose lexical knowledge internalization to integrate
token-level knowledge into the model’s parameters. However, existing LM-based
methods suffer from the problem of hallucination. In this paper, we optimize
the implicit knowledge eliciting process, i.e., reduce hallucination of LMs in
KGD, via the proposed contrastive learning framework MixCL.

###  2.3 Contrastive learning

Contrastive learning (CL) (Chopra, Hadsell, and LeCun 2005; Chen et al. 2020b)
is based on the idea that similar samples should also be close in
representation space, and has seen applications in NLP (Gao, Yao, and Chen
2021). CL has been used for optimizing knowledge retrieval processes
(Karpukhin et al. 2021; Xiong et al. 2021), where the model learns to identify
positive knowledge from negatives. On the task of neural text generation, CL
(Jiang et al. 2022), a.k.a. unlikelihood training (Welleck et al. 2020) or
negative training (He and Glass 2020), alleviates undesirable properties of
the generated output, e.g., repetition (Shirai et al. 2020; Jiang et al.
2022), maliciousness (He and Glass 2020), dullness (Li et al. 2020b, 2022), or
inconsistency (Li et al. 2020a). Moreover, Cao and Wang (2021) propose a
sentence level contrastive learning method to reduce the hallucinations of
text summarization model. Unlike existing studies, we propose a mixed
contrastive learning framework MixCL that eliminates the hallucination at the
span level with effective negative sampling strategies.

##  3 Problem formulation

Let $x$, $y$, and $k$ be the dialogue context, the corresponding response, and
the ground-truth knowledge, respectively. As illustrated in Fig. 2, given a
knowledge corpus $\mathcal{K}$, a dialogue agent learns to predict an
informative response $y$ based on the dialogue context $x$ using the knowledge
in $\mathcal{K}$. As discussed earlier, two approaches are studied in KGD,
_KB-based methods_ and _LM-based methods_. In this paper, we focus on the
latter one.

(a) KB-based dialogue agents explicitly retrieve text-based knowledge from
corpus.

(b) LM-based dialogue agents store knowledge in LM parameters and generate
responses using implicit knowledge.

Figure 2: Types of dialogue agents.

##### KB-based methods.

KB-based dialogue agents (Dinan et al. 2019) ground the response generation by
explicitly retrieving knowledge from $\mathcal{K}$. Two sub-modules, i.e.,
knowledge retriever and response generator, are employed by KB-based
approaches, as shown in Fig. 2 (a).

##### LM-based methods.

In this paper, we explore language models as knowledge bases for dialogue
agents (Zhao, Wu, and Xu 2020; Xu et al. 2022), as illustrated in Fig. 2 (b).
In LM-based approaches, the LMs are first pre-trained on $\mathcal{K}$ to
store the knowledge in their parameters. Then, the models directly generate
$y$ given $x$ using the knowledge in their parameters and getting rid of the
explicit retrieval step.

##  4 Preliminaries

We propose a LM-based dialogue agent for open-domain KGD. The proposed model
$p_{\theta}(y|x)$ is based on a transformer-based language model with encoder-
decoder architecture. The model is first pre-trained on the corpus
$\mathcal{K}$ and then finetuned on dialogue data to generate informative
responses.

##### Pre-training on knowledge corpus.

We employ BART (Lewis et al. 2020) as the pre-trained transformer, which is
pre-trained by denoising self-supervised learning:

$\mathcal{L}_{\text{LM}}=-\mathbb{E}_{k\sim\mathcal{K}}\leavevmode\nobreak\
\log p_{\theta}(k|\hat{k}),$ (1)

where $\mathcal{K}$ is a text-based knowledge corpus (e.g., Wikipedia), $k$ is
a text sampled from knowledge corpus $\mathcal{K}$, and $\hat{k}$ denotes
corrupted text by corruption functions (e.g., masking, deletion, infilling,
etc.; Lewis et al. (2020)).

##### Finetuning on dialogue datasets.

With the pre-trained LM, the model generates the response $y$ given $x$
without explicit knowledge retrieval step (Zhao, Wu, and Xu 2020; Xu et al.
2022). Maximum likelihood estimation (MLE) training loss on dialogue data with
paired $(x,y)$ is employed by previous methods. In MLE, the model learns to
predict the ground-truth tokens for each step in a teacher forcing paradigm
(Zhao, Wu, and Xu 2020; Xu et al. 2022):

$\mathcal{L}_{\text{MLE}}=-\log p_{\theta}(y|x)=-\sum_{t=1}^{|y|}\log
p_{\theta}(y_{t}|y_{<t},x).$ (2)

However, despite its effectiveness in generating informative responses, MLE
loss encourages the model to imitate the training data blindly and leads to
model hallucination (Kang and Hashimoto 2020). Studies have found that models
trained with standard MLE may over-rely on previously predicted tokens,
exacerbating error propagation (Wang and Sennrich 2020). As a result, during
the inference stage, as the generated sequence grows, the errors accumulate
along the sequence, and the model tends to amplify errors and generate
hallucinating contents. We propose a novel contrastive learning framework
MixCL to address this problem.

Figure 3: Overview of MixCL. MixCL consists of two steps: (i) negative
sampling (Section 5.1), which samples most confusing negative knowledge to the
model, and (ii) mixed contrastive learning (Section 5.2), which reduces the
generation probability of negative tokens through mixed contrastive learning.

##  5 MixCL

Next, we present the proposed MixCL framework for addressing the hallucination
of LMs. MixCL explicitly samples negative knowledge (i.e., non-factual or
irrelevant knowledge) and reduces the generation probability of negative
tokens by LMs through contrastive learning. As illustrated in Fig. 3, MixCL
consists of two steps: _negative sampling_ and _mixed contrastive learning_.
In this section, we first present the negative sampling methods, then the
mixed contrastive learning, and finally our optimization strategies.

###  5.1 Negatives sampling

We sample negative knowledge for the dialogue context to construct training
examples for contrastive learning. Formally, let $z^{+}$ be positive
knowledge, i.e., a factual and relevant knowledge snippet, and let
$\mathcal{Q}_{\text{Pos}}(x)$ be the collection of positive knowledge
regarding $x$, where the $z^{+}\sim\mathcal{Q}_{\text{Pos}}(x)$ is sampled
from it. Here, $\mathcal{Q}_{\text{Pos}}(x)$ can be obtained through human
labeling (Dinan et al. 2019) or heuristic methods (Zhao et al. 2020). We
define $z^{-}$ as negative knowledge, i.e., a non-factual or irrelevant
knowledge snippet for $x$. Then, negative sampling is applied to construct the
snippets $z^{-}$ where the model is most likely to get confused. We introduce
two methods for negative sampling, i.e., _retrieved negatives_ and _model-
generated negatives_ , as illustrated in Fig. 3.

##### Retrieved negatives.

For a given $x$, a retrieval tool $\text{Ret}(*)$ is employed to retrieve
irrelevant but potentially confusing knowledge from knowledge corpus
$\mathcal{K}$:

$\mathcal{Q}_{\text{Ret}}(x)=\\{z^{-}|z^{-}\in\text{Ret}(x,\mathcal{K}),z^{-}\notin\mathcal{Q}_{\text{Pos}}(x)\\},$
(3)

where $\text{Ret}(\cdot,\cdot)$ is implemented as TF-IDF retriever (Dinan et
al. 2019), and $z^{-}\notin\mathcal{Q}_{\text{Pos}}(x)$ imposes the constraint
that negative knowledge snippets should not be included in the positive
knowledge.

##### Model-generated negatives.

We also exploit a model bootstrapping approach, in which we generate knowledge
by a model $p_{\theta}(z|x)$ and retain the examples where hallucination
exist. We define:

$\mathcal{Q}_{\text{Model}}(x)=\\{z^{-}|z^{-}\sim
p_{\theta}(z|x),z^{-}\cap\mathcal{Q}_{\text{Pos}}(x)=\varnothing\\},$ (4)

where $z^{-}\sim p_{\theta}(z|x)$ denotes a negative knowledge snippet sampled
from the LM with $\theta$, and
$z^{-}\cup\mathcal{Q}_{\text{Pos}}(x)=\varnothing$ imposes the constraint that
negative knowledge snippets should not be included in the positive knowledge,
which is implemented with a natural language inference (NLI)
toolkit.222https://huggingface.co/roberta-large-mnli

On the basis of the above two methods, we define the constructed negative
collection $\mathcal{Q}_{\text{Neg}}(x)$ with a hyper-parameter
$\beta\in[0,1]$ to control the relative contribution of the methods:

$\mathcal{Q}_{\text{Neg}}(x)=\beta\mathcal{Q}_{\text{Ret}}(x)+(1-\beta)\mathcal{Q}_{\text{Model}}(x).$
(5)

###  5.2 Mixed contrastive learning

Based on the positive knowledge $z^{+}$ and the sampled negative knowledge
$z^{-}$, we introduce a contrastive learning framework to identify positive
knowledge from negatives:

$\mathcal{L}_{\text{CL}}=\mathbb{E}_{z^{+}\sim\mathcal{Q}_{\text{Pos}}(x),\\{z^{-}_{i}\\}_{i=1}^{M}\overset{\text{iid}}{\sim}\mathcal{Z}}l(x,z^{+},\\{z^{-}_{i}\\}_{i=1}^{M},\theta),$
(6)

where $l$ denotes a contrastive loss function that is typically defined as
cross-entropy loss
$l_{\text{ce}}$333$l_{\text{ce}}(x,z^{+}\\!,\\{z^{-}_{i}\\}_{i=1}^{M},\theta)\\!=\\!-\log\frac{\exp{p_{\theta}(z^{+}|x)}}{\exp{p_{\theta}(z^{+}|x)}+\sum_{i=1}^{M}\exp{p_{\theta}(z^{-}_{i}|x)}}$.
(Gao, Yao, and Chen 2021; Cao and Wang 2021), and $M$ denotes the number of
negatives.

However, $l_{\text{ce}}$ only considers token-level or sentence-level
contrast. It ignores fine-grained _span-level_ contrast even though
hallucinations typically exists at the span level. Therefore, inspired by work
on mix-up data augmentation (Zhang et al. 2018; Kim et al. 2020; Shi, Livescu,
and Gimpel 2021; Zhang, Yang, and Yang 2022), we propose a _mixed contrast
objective_ , which mixes the positive and negative examples into a sequence at
the span level. As illustrated in Fig. 3, the proposed mixed contrastive
learning method has three parts:  (i) _extracting spans_ , which extracts
meaningful spans from both positive and negative knowledge;  (ii) _mixing
examples_ , which mixes positive and negative knowledge using the extracted
spans; and  (iii) _mixed-contrast loss_ , which optimizes the model at the
span level through contrastive learning.

#### Extracting spans.

We extract the key components from the both positive and negative knowledge,
$z^{+}$ and $z^{-}$. Regarding the two types of hallucinations, i.e., the
intrinsic and extrinsic, we design two extraction strategies. As part of the
pilot experiment reported in Section 1, we find that intrinsic hallucinations
are typically associated with confused entities. Therefore, we use _named
entity recognition_ (NER)444https://spacy.io/api/entityrecognizer/ to extract
entities of various types e.g. person and time. Moreover, we find that
extrinsic hallucination is mainly triggered by the emergence of irrelevant
sentence fragments in the text. Therefore, we use _constituency parsing_
(CP)555https://stanfordnlp.github.io/stanza/constituency.html to extract
sentence constituents, e.g., noun and particle. Through the two strategies, we
extract sequence spans from $z^{+}$ and $z^{-}$, respectively.

Example. Consider knowledge snippets about the French soccer player Thierry
Henry. A statement like _He was born and raised in Paris_ would be in $z^{+}$,
while the span “Montreal, Quebec, Canada” could be extracted from a snippet
such as in _He was born in Montreal, Quebec, Canada_ in $z^{-}$.

#### Mixing examples.

Based on the extracted spans, we mix the two examples $z^{+}$ and $z^{-}$ into
a mixed sequence $\tilde{z}$ via a mix-up function:
$\tilde{z}=\text{Mix}(z^{+},z^{-})$. The mix-up function randomly selects a
span in $z^{+}$, and then selects a span with the same type in $z^{-}$ to
substitute it. We define a sequence $\phi$ with the same length of
$\tilde{z}$, which annotates the tokens in $\tilde{z}$ as $1$ if they come
from $z^{+}$ and $0$ if they come from $z^{-}$.

In the earlier Thierry Henry example, the span “Paris” in a snippet in $z^{+}$
can be selected and substituted by the corresponding ones from a snippet in
$z^{-}$, such as “Montreal, Quebec, Canada.”

#### Mixed-contrast loss.

Based on the mixed sequence $\tilde{z}$ and $\phi$, we design a loss function
$l_{mix}$ as follows:

$\begin{split}l_{mix}(z^{+},z^{-})={}&\\\
\textstyle-\sum_{j=1}^{|\tilde{z}_{i}|}[\leavevmode\nobreak\ &\phi_{i,j}\log
p_{\theta}(\tilde{z}_{i,j}|\tilde{z}_{i,<j},x)+{}\\\
&(1-\phi_{i,j})\log(1-p_{\theta}(\tilde{z}_{i,j}|\tilde{z}_{i,<j},x))],\end{split}$
(7)

where $\tilde{z}_{i}=\text{Mix}(z^{+},z^{-}_{i})$ is a mixed sequence of
$z^{+}$ and $z^{-}_{i}$, and $\phi_{i,j}$ denotes the sign of token
$\tilde{z}_{i,j}$, which equals $1$ for positive tokens and $0$ for negative
tokens. Using the negative collection $\mathcal{Q}_{\text{Neg}}(x)$ defined in
Eq. 5, we formalize the mixed contrast objective $\mathcal{L}_{\text{MCL}}$
as:

$\textstyle{\sum}_{z^{+}\sim\mathcal{Q}_{\text{Pos}}(x)}{\sum}_{z^{-}_{i}\sim\mathcal{Q}_{\text{Neg}}(x)}^{i=1,...,M}l_{mix}(x,z^{+},z^{-}_{i},\theta).$
(8)

###  5.3 Optimization

During finetuning, MixCL is optimized by minimizing
$\mathcal{L}_{\text{MCL}}$. Two additional loss are considered in training,
i.e., $\mathcal{L}_{\text{LM}}$, $\mathcal{L}_{\text{MLE}}$.
$\mathcal{L}_{\text{LM}}$ is used to alleviate catastrophic knowledge
forgetting (Devlin et al. 2019; Chen et al. 2020a) and
$\mathcal{L}_{\text{MLE}}$ is used to optimize the response generation
ability. Therefore, the final training objective is defined as:

$\mathcal{J}(\theta)=\alpha_{1}\mathcal{L}_{\text{MLE}}+\alpha_{2}\mathcal{L}_{\text{MCL}}+\alpha_{3}\mathcal{L}_{\text{LM}},$
(9)

where three losses are optimized jointly and $\alpha_{1}$, $\alpha_{2}$,
$\alpha_{3}$ denote the weights of the three losses, respectively.

##  6 Experimental setup

###  6.1 Datasets and evaluation metrics

We conduct experiments on the Wizard of Wikipedia (WoW) dataset. WoW is built
with crowd-sourcing and employs Wikipedia as the knowledge corpus. WoW
consists of 22,311 conversations over 1,365 general topics that range from
e-books to toga parties to showers. The ground-truth knowledge used in each
turn is manually labeled. The WoW test set is split into _test seen_ and _test
unseen_ based on whether the topic appears in the training set. We evaluate
our methods on both test seen and test unseen.

We choose F1, ROUGE, BLEU, MT, Knowledge-F1 (KF1), Entity-F1 (EF1), and
Accuracy (Acc) as metrics. F1 (Dinan et al. 2019) calculates the unigram F1
between the generated text and the ground-truth text. For ROUGE (Lin 2004) we
use ROUGE-L (RL for short) following previous work. BLEU (Papineni et al.
2002) we use BLEU-2 and BLEU-4 (or B2 and B4 for short) and use the
implementation in the NLTK Toolkit. MT (Meteor) (Denkowski and Lavie 2014) is
based on the harmonic mean of unigram precision and recall. Knowledge-F1
(Dinan et al. 2019) (or KF1 for short) calculates the F1 between the generated
response and the ground-truth knowledge sentence, which indicates the
informativeness of a response. Acc measures the knowledge selection accuracy.
As we skip the knowledge selection step, we select knowledge by matching the
generated response with each knowledge candidate in WoW using the F1 score.
Entity-F1 (or EF1 for short) identifies entities in text using Spacy, deletes
the non-entity words, and calculates the F1 score between the modified
generated text and the ground-truth response. EF1 eliminates the impact of the
stop-word and focuses on the accuracy of entities.

In addition, we randomly sample 100 examples from the test seen and test
unseen segments of the test set, respectively, and recruit three experts for
human evaluation. Each annotator is presented with examples that come with
dialogue context and model responses. Four metrics are considered in the human
evaluation: Informativeness, which measures whether the response is knowledge-
inclusive; Relevancy, which measures whether the response’s content is
relevant to the dialogue; Factuality, which measures whether the information
in the response is factually correct;666The human annotators used Google to
check the factuality of the responses. and Humanlikeness, which measures
whether the response is human-like in its fluency and naturalness. The
annotators are asked to assign a score in $\\{0,1\\}$ (representing “non-
factual” and “factual”) for factuality, and a score in $\\{0,1,2\\}$
(representing “bad” “fair”, and “good”) for the others.

Table 1: Evaluation results on Wizard-of-Wikipedia. The first group lists _KB-based methods under realistic conditions_. The second group lists _KB-based methods under oracle conditions_. The third group lists _LM-based methods_ , including MixCL. We highlight the results of MixCL that significantly exceed the previous-best LM-based methods in boldface (t-test, $p<0.05$), and of MixCL that exceed the best KB-based methods under realistic conditions in yellow. We also highlight the best results of previous KB-based methods and LM-based methods by underlining them, respectively.  | Test seen | Test unseen
---|---|---
Method | F1 | RL | B2 | B4 | MT | KF1 | EF1 | Acc | F1 | RL | B2 | B4 | MT | KF1 | EF1 | Acc
_KB-based methods under realistic conditions_ |  |  |  |  |  |  |  |  |  |  |  |  |
TMN (Dinan et al. 2019) | 17.3 | 17.0 | 5.7 | 1.1 | 14.8 | 15.8 |  08.7 | 15.2 | 14.4 | 14.5 | 3.3 | 0.3 | 11.5 |  09.4 |  02.1 |  08.6
DukeNet (Meng et al. 2020) | 18.5 | 17.7 | 6.4 | 1.9 | 16.0 | 18.5 | 12.0 | 20.6 | 15.9 | 15.9 | 4.8 | 1.1 | 13.7 | 14.7 |  08.0 | 14.3
KnowledGPT (Zhao et al. 2020) | 21.1 | 20.1 | 8.9 | 3.4 | 20.0 | 22.2 | 15.5 | 24.3 | 19.5 | 18.4 | 8.0 | 2.6 | 18.3 | 20.0 | 11.7 | 20.2
KnowBART | 21.1 | 18.9 | 8.5 | 3.3 | 17.8 | 21.3 | 16.2 | 24.2 | 21.0 | 18.3 | 8.9 | 3.6 | 17.9 | 22.5 | 16.2 | 24.0
_KB-based methods under oracle conditions_ |  |  |  |  |  |  |  |  |  |  |  |  |
DukeNet (Meng et al. 2020) | 19.3 | 18.7 | 7.5 | 2.5 | 17.2 | 19.6 | 13.2 | 22.1 | 17.1 | 17.0 | 6.0 | 1.7 | 15.2 | 16.5 |  09.2 | 16.8
KnowledGPT (Zhao et al. 2020) | 22.0 | 20.8 | 9.9 | 3.7 | 20.9 | 23.8 | 16.9 | 26.3 | 20.5 | 19.5 | 8.7 | 3.0 | 19.3 | 22.1 | 13.3 | 22.6
KnowBART | 22.1 | 19.6 | 9.1 | 3.7 | 18.1 | 23.1 | 18.0 | 26.8 | 22.7 | 20.1 | 9.8 | 4.3 | 18.7 | 24.1 | 18.4 | 27.5
_LM-based methods_ |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
GPT-2 (Zhao, Wu, and Xu 2020) | 19.6 | 18.5 | 7.8 | 1.4 | 17.8 | 17.9 | 13.3 | 15.4 | 18.3 | 17.3 | 6.5 | 0.8 | 16.1 | 14.6 | 7.2 |  08.4
BlenderBot (Roller et al. 2021) | 18.8 | 19.4 | 7.7 | 2.3 | 18.0 | 18.2 | 13.1 | 16.7 | 17.8 | 16.9 | 5.5 | 0.8 | 15.0 | 15.7 | 7.1 |  09.6
KnowExpert (Xu et al. 2022) | 18.7 | 18.6 | 6.7 | 1.3 | 16.5 | 14.1 |  09.8 | 12.6 | 16.7 | 17.2 | 5.4 | 0.6 | 14.5 | 11.8 | 5.5 |  09.2
MSDP (Liu et al. 2022) | 17.8 | 16.5 | 6.1 | 1.9 | 18.2 | 21.7 | 13.9 | 18.4 | 16.9 | 16.1 | 5.5 | 1.1 | 16.2 | 20.3 | 8.4 | 16.1
Ours | 21.6 | 20.5 | 9.2 | 2.7 | 20.5 | 22.3 | 16.3 | 20.4 | 19.6 | 18.8 | 7.4 | 1.4 | 18.0 | 18.0 | 11.6 | 14.4

###  6.2 Baselines

We compare MixCL with baselines of two categories:  (i) _KB-based methods_
that use additional IR modules for explicit knowledge retrieval, and  (ii)
_LM-based methods_ that use LMs as a knowledge base.  All models are re-
evaluated with the same evaluation function using the official public
checkpoints.

The KB-based methods we consider are: TMN (Dinan et al. 2019) (50M), which
combines a transformer with an external memory network to select knowledge and
generate a response; DukeNet (Meng et al. 2020) (150M), which is the best
performing KB-based method without using pre-trained LMs and which models
knowledge shift with a dual learning scheme; KnowledGPT (Zhao et al. 2020)
(227M), which exploits pre-trained LMs in a KB-based approach, selects
knowledge using BERT, generates responses using GPT-2, and optimizes the two
modules jointly with reinforcement learning; it achieves state-of-the-art
performance. We also introduce KnowBART (600M), a KB-based model that selects
knowledge using RoBERTa and generates responses using BART-Large.

The KB-based methods listed above retrieve knowledge under _oracle conditions_
, i.e., they are given a small subset of Wikipedia with roughly ten passages
that _definitely_ contain the ground-truth knowledge (Dinan et al. 2019; Liu
et al. 2022). We also consider KB-based methods under _realistic experimental
conditions_ , where passages from the full knowledge corpus (i.e., Wikipedia)
are retrieved. We employ the state-of-the-art passage retrieval model GENRE
(Cao et al. 2021) from the KILT leaderboard (Petroni et al. 2021), which is
reported to outperform competitors (e.g., DPR and BM25) by a substantial
margin on WoW.

The LM-based methods that we consider are: GPT-2 (Zhao, Wu, and Xu 2020)
(345M),which finetunes GPT-2 on knowledge-grounded dialogue data; BlenderBot
(Roller et al. 2021) (400M), which pre-trains a transformer with encoder-
decoder architecture on _reddit_ data, and then finetunes the model on KGD
data; KnowExpert (Xu et al. 2022) (117M), which uses a topic-aware adapter
that first clusters Wikipedia using a topic model and then employs a mix-of-
adapter architecture to adapt a GPT-2 model to open-domain dialogues; MSDP
(Liu et al. 2022) (357M), which uses a multi-stage prompting model, designs
task-specific prompts with task instructions and in-context examples, and uses
Megatron-LM (Shoeybi et al. 2019) to produce knowledge and response in a two-
stage process.

###  6.3 Implementation details

We implement MixCL using BART-Large (400M) (Lewis et al. 2020) in
HuggingFace’s Transformers library. We use Wikipedia as the knowledge corpus
$\mathcal{K}$, as it is used as knowledge corpus by WoW. We determine the
hyperparameters through pilot experiments. We set the weight of the language
model loss $\alpha_{3}$ to $0.3$ at initialization and linearly decay until
$0$. We set $\alpha_{1}$ and $\alpha_{2}$, i.e., the weight of the MLE loss
and MCL loss, to $0.4$ and $0.3$, respectively, and linearly increase to $0.5$
and $0.5$. We use greedy decoding in testing. More details are available in
Appendix B or at https://github.com/sunnweiwei/MixCL.

##  7 Experimental results

###  7.1 Results of automatic evaluation

Table 1 shows the results of automatic evaluation metrics. Overall, MixCL
achieves the highest scores of the LM-based methods and competitive results
compared to the KB-based methods under realistic conditions. Compared with
previous LM-based methods (the third group in Table 1), MixCL achieves the
highest scores on almost all metrics. For example, MixCL gets F1${}={}$21.6,
B4${}={}$2.7 on test seen and F1${}={}$19.6, B4${}={}$1.4 on test unseen, with
about 5% to 15% relative improvements over previous-best LM-based baselines.
Moreover, we find a dilemma with the previous LM-based methods in terms of
response quality (e.g., F1, RL, B2) and knowledge relevance (e.g., KF1, EF1,
Acc). For example, MSDP performs well on knowledge relevance at the expense of
response quality, while GPT-2 and BlenderBot show the opposite. MixCL, on the
other hand, performs well on both fronts.

Furthermore, compared with KB-based methods (the first block in Table 1), we
find that MixCL outperforms two non-LM methods (DukeNet and TMN) by a large
margin. Compared to KnowledGPT and KnowBART, which combine LMs with the KB-
based approach, MixCL outperforms them on test seen. On test unseen, MixCL
lags behind the best performing KB-based baselines, probably due to knowledge
forgetting issues.

Finally, under oracle conditions, the KB-based methods (the second group in
Table 1) show better results than MixCL. However, the manually selected
knowledge candidates include the ground-truth, which is unavailable in
realistic scenarios.

Table 2: Human evaluation results. Methods marked with ${}^{\text{K}}$ denote KB-based methods, and those marked with ${}^{\text{L}}$ denote LM-based methods. The four metrics (Info., Rel., Fact., and Hum.) denote informativeness, relevance, factuality, and humanlikeness, respectively. Methods | Test seen | Test unseen
---|---|---
Info. | Rel. | Fact. | Hum. | Info. | Rel. | Fact. | Hum.
DukeNet${}^{\text{K}}$ | 1.44 | 1.22 | 0.71 | 1.16 | 1.21 | 1.08 | 0.72 | 1.03
KnowledGPT${}^{\text{K}}$ | 1.67 | 1.47 | 0.87 | 1.73 | 1.63 | 1.23 | 0.83 | 1.36
KnowBART${}^{\text{K}}$ | 1.67 | 1.57 | 0.89 | 1.70 | 1.68 | 1.56 | 0.91 | 1.44
KnowExpert${}^{\text{L}}$ | 1.45 | 1.36 | 0.62 | 1.45 | 1.49 | 1.26 | 0.59 | 1.15
MSDP${}^{\text{L}}$ | 1.20 | 0.96 | 0.71 | 0.98 | 1.28 | 1.18 | 0.82 | 1.05
BART${}^{\text{L}}$ | 1.51 | 1.45 | 0.76 | 1.58 | 1.50 | 1.47 | 0.82 | 1.40
Ours${}^{\text{L}}$ | 1.71 | 1.55 | 0.89 | 1.77 | 1.67 | 1.53 | 0.87 | 1.47
Human | 1.84 | 1.85 | 0.98 | 1.96 | 1.83 | 1.85 | 0.95 | 1.95

###  7.2 Results of human evaluation

Table 2 shows the human evaluation results. The Fleiss’ kappa value is above
$0.60$, indicating substantial agreement among the annotators. MixCL
consistently outperforms LM-based baselines on all metrics, and also
outperforms KB-based baselines in metrics. MixCL is capable of generating more
informative responses compared to previous LM-based methods. Moreover, MixCL
effectively increases relevance and factuality, demonstrating its
effectiveness in reducing both types of hallucinations. In particular, we find
that KnowledGPT is outperformed by MixCL in terms of knowledge relevance,
probably due to the presence of retrieval errors. Finally, MixCL’s responses
are considered more human-like by the annotators.

Table 3: Ablation study. The base model, MixCL, is compared with several variants. See Section 7.3. Methods | Test seen | Test unseen
---|---|---
F1 | B4 | KF1 | F1 | B4 | KF1
Base model | 21.6 | 2.7 | 22.3 | 19.6 | 1.4 | 18.0
-w/o $\mathcal{L}_{\text{MCL}}$ | 21.0↓0.6 | 2.0↓0.7 | 19.1↓3.2 | 19.1↓0.5 | 1.0↓0.4 | 16.9↓1.1
-w/o $\mathcal{Q}_{\text{Neg}}(*)$ | 20.8↓0.8 | 2.4↓0.3 | 20.8↓1.5 | 19.0↓0.6 | 1.1↓0.3 | 17.4↓0.6
-w/o $\mathcal{Q}_{\text{Model}}(*)$ | 21.3↓0.3 | 2.5↓0.2 | 21.7↓0.6 | 19.4↓0.2 | 1.2↓0.2 | 17.5↓0.5
-w/o $\mathcal{L}_{\text{LM}}$ | 21.3↓0.3 | 2.6↓0.1 | 21.8↓0.5 | 18.6↓1.0 | 1.2↓0.2 | 16.7↓1.3
-Only $\mathcal{L}_{\text{MLE}}$ | 20.9↓0.7 | 1.8↓0.9 | 18.9↓3.4 | 18.8↓0.8 | 0.9↓0.5 | 16.0↓2.0

###  7.3 Ablation studies

In Table 3, we compare MixCL with several ablative variants. The variants and
our findings are as follows:

No $\mathcal{L}_{\text{MCL}}$ – We remove the mixed contrast objective. The
performance of the model shows a notable degradation, especially for the
knowledge relevance metric, i.e., KF1. This suggests that the proposed mixed
contrast objective is effective in increasing the relevance of responses.

No $\mathcal{Q}_{\text{Neg}}(*)$ – We remove the hard negative sampling
process and use randomly sampled instances as negatives. The effectiveness of
the hard negative sampling is evidenced by the decrease in the metric on KF1.

No $\mathcal{Q}_{\text{Model}}$ – We remove the negatives generated by the
model. The results suggest that model-generated negatives provides harder
negative examples for the model, i.e., the knowledge that is more likely to be
confounded by the LMs.

No $\mathcal{L}_{\text{LM}}$ – We remove the LM loss. The effect of the model
is a decline, especially on unseen topics. This results suggests that LM loss
is instrumental in suppressing the catastrophic knowledge forgetting problem
of the LMs in conversations (Chen et al. 2020a).

Only $\mathcal{L}_{\text{MLE}}$ – This variant optimizes the model only by MLE
loss. We observe a substantial performance drop, especially on KF1, which
demonstrates the effectiveness of MixCL in improving the knowledge relevancy
and factuality of LMs.

Figure 4: Latency (minutes) versus response quality (F1 score) on WoW test
seen and test unseen. Gray, blue, and orange indicate LM-based, KB-based, and
the proposed methods, respectively. The size of the circle indicates the
number of parameters of these methods.

###  7.4 Efficiency analysis

In Fig. 4, we compare MixCL against baselines in terms of efficiency and
effectiveness. We adjust the inference efficiency of the models by evaluating
the model with different numbers of parameters (e.g., 140M and 400M). Compared
with KB-based methods, LM-based methods generally have an advantage in terms
of speed as they get rid of the extra IR step. However, previous LM-based
methods are outperformed by KB-based methods regarding response quality. By
explicitly eliminating the hallucinations of LM in conversations, MixCL
significantly improves the response quality of LM-based methods without
compromising efficiency. Notably, MixCL is 5$\times$ more efficient than
state-of-the-art KB-based methods while achieving competitive response
generation performance. Moreover, the improvements of MixCL along with the
model size are more noticeable compared to KB-based methods (see the dashed
lines), indicating its superior ability to utilize the knowledge of pre-
trained model.

###  7.5 Case study

We conduct several case studies and find that MixCL is more effective at
incorporating knowledge and generating more engaging and human-like responses
than baselines. Details about our case studies are available in Appendix C or
https://github.com/sunnweiwei/MixCL.

##  8 Conclusions

In this paper, we have proposed MixCL, a contrastive learning framework aimed
at reducing the hallucination of language models in conversations. MixCL is
enhanced by negative sampling and mixed contrastive objective. Experiments on
the Wizard-of-Wikipedia dataset have shown that MixCL outperforms existing LM-
based methods and achieves comparable performance as state-of-the-art KB-based
methods. Human evaluation and ablative experiments also confirm MixCL’s
effectiveness in eliminating hallucination of LMs. Moreover, MixCL
demonstrates advantages in terms of efficiency and scalability. Hence, we
believe that MixCL provides new insights on using knowledge inside large
language models’ parameters for KGD tasks.

The limitations of this work include the problem of knowledge forgetting. In
future work, we would like to explore practical approaches to avoiding
catastrophic knowledge forgetting. We also plan to reproduce our findings for
other, less resource-rich languages.

## Acknowledgements

This work was supported by the National Key R&D Program of China with grant
No. 2020YFB1406704, the Natural Science Foundation of China (62272274,
62202271, 61902219, 61972234, 62072279, 62102234), the Natural Science
Foundation of Shandong Province (ZR2021QF129), the Key Scientific and
Technological Innovation Program of Shandong Province (2019JZZY010129), and by
the Hybrid Intelligence Center, a 10-year program funded by the Dutch Ministry
of Education, Culture and Science through the Netherlands Organisation for
Scientific Research, https://hybrid-intelligence-centre.nl.

All content represents the opinion of the authors, which is not necessarily
shared or endorsed by their respective employers and/or sponsors.

## References

  * Brown et al. (2020) Brown, T. B.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; Agarwal, S.; Herbert-Voss, A.; Krueger, G.; Henighan, T. J.; Child, R.; Ramesh, A.; Ziegler, D. M.; Wu, J.; Winter, C.; Hesse, C.; Chen, M.; Sigler, E.; Litwin, M.; Gray, S.; Chess, B.; Clark, J.; Berner, C.; McCandlish, S.; Radford, A.; Sutskever, I.; and Amodei, D. 2020.  Language Models are Few-Shot Learners.  In _NIPS_ , 1877–1901.
  * Cao et al. (2021) Cao, N. D.; Izacard, G.; Riedel, S.; and Petroni, F. 2021.  Autoregressive Entity Retrieval.  In _ICLR_.
  * Cao and Wang (2021) Cao, S.; and Wang, L. 2021.  CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization.  ArXiv:abs/2109.09209.
  * Chen et al. (2020a) Chen, S.; Hou, Y.; Cui, Y.; Che, W.; Liu, T.; and Yu, X. 2020a.  Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting.  In _EMNLP_ , 7870–7881.
  * Chen et al. (2020b) Chen, T.; Kornblith, S.; Norouzi, M.; and Hinton, G. E. 2020b.  A Simple Framework for Contrastive Learning of Visual Representations.  In _ICML_ , 1597–1607.
  * Chopra, Hadsell, and LeCun (2005) Chopra, S.; Hadsell, R.; and LeCun, Y. 2005.  Learning a Similarity Metric Discriminatively, with Application to Face Verification.  In _CVPR_ , 539–546 vol. 1.
  * Cui et al. (2021) Cui, L.; Wu, Y.; Liu, S.; and Zhang, Y. 2021.  Knowledge Enhanced Fine-Tuning for Better Handling Unseen Entities in Dialogue Generation.  In _EMNLP_ , 2328–2337.
  * Denkowski and Lavie (2014) Denkowski, M. J.; and Lavie, A. 2014.  Meteor Universal: Language Specific Translation Evaluation for Any Target Language.  In _WMT@ACL_.
  * Devlin et al. (2019) Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.  In _NAACL_ , 4171–4186.
  * Dinan et al. (2019) Dinan, E.; Roller, S.; Shuster, K.; Fan, A.; Auli, M.; and Weston, J. 2019.  Wizard of Wikipedia: Knowledge-Powered Conversational agents.  In _ICLR_.
  * Dziri et al. (2021) Dziri, N.; Madotto, A.; Zaiane, O.; and Bose, A. 2021.  Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding.  In _EMNLP_ , 2197–2214.
  * Freitas et al. (2020) Freitas, D. D.; Luong, M.-T.; So, D. R.; Hall, J.; Fiedel, N.; Thoppilan, R.; Yang, Z.; Kulshreshtha, A.; Nemade, G.; Lu, Y.; and Le, Q. V. 2020.  Towards a Human-like Open-Domain Chatbot.  ArXiv:abs/2001.09977.
  * Gao, Yao, and Chen (2021) Gao, T.; Yao, X.; and Chen, D. 2021.  SimCSE: Simple Contrastive Learning of Sentence Embeddings.  In _EMNLP_ , 6894–6910.
  * He and Glass (2020) He, T.; and Glass, J. R. 2020.  Negative Training for Neural Dialogue Response Generation.  In _ACL_ , 2044–2058.
  * Huang, Zhu, and Gao (2020) Huang, M.; Zhu, X.; and Gao, J. 2020.  Challenges in Building Intelligent Open-domain Dialog Systems.  _ACM TOIS_ , 38: 1 – 32.
  * Ji et al. (2022) Ji, Z.; Lee, N.; Frieske, R.; Yu, T.; Su, D.; Xu, Y.; Ishii, E.; Bang, Y.; Madotto, A.; and Fung, P. 2022.  Survey of Hallucination in Natural Language Generation.  ArXiv:abs/2202.03629.
  * Jiang et al. (2022) Jiang, S.; Zhang, R.; Vakulenko, S.; and de Rijke, M. 2022.  A Simple Contrastive Learning Objective for Alleviating Neural Text Degeneration.  ArXiv:abs/2205.02517.
  * Kang and Hashimoto (2020) Kang, D.; and Hashimoto, T. B. 2020.  Improved Natural Language Generation via Loss Truncation.  In _ACL_ , 718–731.
  * Karpukhin et al. (2021) Karpukhin, V.; Oğuz, B.; Min, S.; Lewis, P.; Wu, L. Y.; Edunov, S.; Chen, D.; and tau Yih, W. 2021.  Dense Passage Retrieval for Open-Domain Question Answering.  In _NAACL_ , 5835–5847.
  * Kim et al. (2020) Kim, S.; Lee, G.; Bae, S.; and Yun, S. 2020.  MixCo: Mix-up Contrastive Learning for Visual Representation.  In _NIPS | Workshop on Self-Supervised Learning_.
  * Lewis et al. (2020) Lewis, M.; Liu, Y.; Goyal, N.; Ghazvininejad, M.; Mohamed, A.; Levy, O.; Stoyanov, V.; and Zettlemoyer, L. 2020.  BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension.  In _ACL_ , 7871–7880.
  * Li et al. (2020a) Li, M.; Roller, S.; Kulikov, I.; Welleck, S.; Boureau, Y.-L.; Cho, K.; and Weston, J. 2020a.  Don’t Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Training.  In _ACL_ , 4715–4728.
  * Li et al. (2020b) Li, X.; Li, P.; Wang, Y.; Liu, X.; and Lam, W. 2020b.  Enhancing Dialogue Generation via Multi-Level Contrastive Learning.  ArXiv:abs/2009.09147.
  * Li et al. (2022) Li, Y.; Feng, S.; Sun, B.; and Li, K. 2022.  Diversifying Neural Dialogue Generation via Negative Distillation.  In _NAACL_ , 407–418.
  * Lin (2004) Lin, C.-Y. 2004.  ROUGE: A Package for Automatic Evaluation of Summaries.  In _ACL_.
  * Liu et al. (2018) Liu, S.; Chen, H.; Ren, Z.; Feng, Y.; Liu, Q.; and Yin, D. 2018.  Knowledge Diffusion for Neural Dialogue Generation.  In _ACL_ , 74–81.
  * Liu et al. (2022) Liu, Z.; Patwary, M. A.; Prenger, R. J.; Prabhumoye, S.; Ping, W.; Shoeybi, M.; and Catanzaro, B. 2022.  Multi-Stage Prompting for Knowledgeable Dialogue Generation.  In _Findings of ACL_ , 1317–1337.
  * Madotto et al. (2020) Madotto, A.; Cahyawijaya, S.; Winata, G. I.; Xu, Y.; Liu, Z.; Lin, Z.; and Fung, P. 2020.  Learning Knowledge Bases with Parameters for Task-Oriented Dialogue Systems.  In _Findings of ACL_.
  * Meng et al. (2020) Meng, C.; Ren, P.; Chen, Z.; Sun, W.; Ren, Z.; Tu, Z.; and de Rijke, M. 2020.  DukeNet: A Dual Knowledge Interaction Network for Knowledge-Grounded Conversation.  In _SIGIR_ , 1151–1160.
  * Papineni et al. (2002) Papineni, K.; Roukos, S.; Ward, T.; and Zhu, W.-J. 2002.  Bleu: a Method for Automatic Evaluation of Machine Translation.  In _ACL_ , 311–318.
  * Petroni et al. (2021) Petroni, F.; Piktus, A.; Fan, A.; Lewis, P.; Yazdani, M.; Cao, N. D.; Thorne, J.; Jernite, Y.; Plachouras, V.; Rocktaschel, T.; and Riedel, S. 2021.  KILT: a Benchmark for Knowledge Intensive Language Tasks.  In _NAACL_ , 2523–2544.
  * Petroni et al. (2019) Petroni, F.; Rocktäschel, T.; Lewis, P.; Bakhtin, A.; Wu, Y.; Miller, A. H.; and Riedel, S. 2019.  Language Models as Knowledge Bases?  In _EMNLP_ , 2463–2473.
  * Roberts, Raffel, and Shazeer (2020) Roberts, A.; Raffel, C.; and Shazeer, N. M. 2020.  How Much Knowledge Can You Pack into the Parameters of a Language Model?  In _EMNLP_ , 5418–5426.
  * Roller et al. (2021) Roller, S.; Dinan, E.; Goyal, N.; Ju, D.; Williamson, M.; Liu, Y.; Xu, J.; Ott, M.; Shuster, K.; Smith, E. M.; Boureau, Y.-L.; and Weston, J. 2021.  Recipes for Building an Open-Domain Chatbot.  In _EACL_ , 300–325.
  * Shi, Livescu, and Gimpel (2021) Shi, H.; Livescu, K.; and Gimpel, K. 2021.  Substructure Substitution: Structured Data Augmentation for NLP.  In _Findings of ACL_ , 3494–3508.
  * Shirai et al. (2020) Shirai, K.; Hashimoto, K.; Eriguchi, A.; Ninomiya, T.; and Mori, S. 2020.  Neural Text Generation with Artificial Negative Examples.  ArXiv:abs/2012.14124.
  * Shoeybi et al. (2019) Shoeybi, M.; Patwary, M. A.; Puri, R.; LeGresley, P.; Casper, J.; and Catanzaro, B. 2019.  Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism.  ArXiv:abs/1909.08053.
  * Shuster et al. (2021) Shuster, K.; Poff, S.; Chen, M.; Kiela, D.; and Weston, J. 2021.  Retrieval Augmentation Reduces Hallucination in Conversation.  In _Findings of EMNLP_ , 3784–3803.
  * Thoppilan et al. (2022) Thoppilan, R.; Freitas, D. D.; Hall, J.; Shazeer, N. M.; Kulshreshtha, A.; Cheng, H.-T.; Jin, A.; Bos, T.; Baker, L.; Du, Y.; Li, Y.; Lee, H.; Zheng, H.; Ghafouri, A.; Menegali, M.; Huang, Y.; Krikun, M.; Lepikhin, D.; Qin, J.; Chen, D.; Xu, Y.; Chen, Z.; Roberts, A.; Bosma, M.; Zhou, Y.; Chang, C.-C.; Krivokon, I. A.; Rusch, W. J.; Pickett, M.; Meier-Hellstern, K. S.; Morris, M. R.; Doshi, T.; Santos, R. D.; Duke, T.; Søraker, J. H.; Zevenbergen, B.; Prabhakaran, V.; Diaz, M.; Hutchinson, B.; Olson, K.; Molina, A.; Hoffman-John, E.; Lee, J.; Aroyo, L.; Rajakumar, R.; Butryna, A.; Lamm, M.; Kuzmina, V. O.; Fenton, J.; Cohen, A.; Bernstein, R.; Kurzweil, R.; Aguera-Arcas, B.; Cui, C.; Croak, M.; Chi, E.; and Le, Q. 2022.  LaMDA: Language Models for Dialog Applications.  ArXiv:abs/2201.08239.
  * Wang and Sennrich (2020) Wang, C.; and Sennrich, R. 2020.  On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation.  In _ACL_.
  * Welleck et al. (2020) Welleck, S.; Kulikov, I.; Roller, S.; Dinan, E.; Cho, K.; and Weston, J. 2020.  Neural Text Generation with Unlikelihood Training.  In _ICLR_.
  * Wu et al. (2022) Wu, Z.; Bi, W.; Li, X.; Kong, L.; and Kao, B. C. 2022.  Lexical Knowledge Internalization for Neural Dialog Generation.  In _ACL_ , 7945–7958.
  * Xiong et al. (2021) Xiong, L.; Xiong, C.; Li, Y.; Tang, K.-F.; Liu, J.; Bennett, P.; Ahmed, J.; and Overwijk, A. 2021.  Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval.  In _ICLR_.
  * Xu et al. (2022) Xu, Y.; Ishii, E.; Liu, Z.; Winata, G. I.; Su, D.; Madotto, A.; and Fung, P. 2022\.  Retrieval-Free Knowledge-Grounded Dialogue Response Generation with Adapters.  In _ACL | Workshop on DialDoc_ , 93–107.
  * Young et al. (2018) Young, T.; Cambria, E.; Chaturvedi, I.; Zhou, H.; Biswas, S.; and Huang, M. 2018\.  Augmenting End-to-End Dialog Systems with Commonsense Knowledge.  In _AAAI_.
  * Zhang et al. (2018) Zhang, H.; Cissé, M.; Dauphin, Y.; and Lopez-Paz, D. 2018.  mixup: Beyond Empirical Risk Minimization.  In _ICLR_.
  * Zhang, Yang, and Yang (2022) Zhang, L.; Yang, Z.; and Yang, D. 2022.  TreeMix: Compositional Constituency-based Data Augmentation for Natural Language Understanding.  In _NAACL_ , 5243–5258.
  * Zhao et al. (2020) Zhao, X.; Wu, W.; Xu, C.; Tao, C.; Zhao, D.; and Yan, R. 2020.  Knowledge-Grounded Dialogue Generation with Pre-trained Language Models.  In _EMNLP_ , 3377–3390.
  * Zhao, Wu, and Xu (2020) Zhao, Y.; Wu, W.; and Xu, C. 2020.  Are Pre-trained Language Models Knowledgeable to Ground Open Domain Dialogues?  ArXiv:abs/2011.09708.
  * Zheng and Huang (2021) Zheng, C.; and Huang, M. 2021.  Exploring Prompt-based Few-shot Learning for Grounded Dialog Generation.  ArXiv:abs/2109.06513.
  * Zhou et al. (2020) Zhou, X.; Zhang, Y.; Cui, L.; and Huang, D. 2020.  Evaluating Commonsense in Pre-trained Language Models.  In _AAAI_.

##  Appendix A Results of pilot experiment

Our pilot human evaluation revealed that 51% of the responses of BART are
hallucinatory on the datasets of Wizard-of-Wikipedia. We group the
hallucinations into two types: _intrinsic hallucinations_ and _extrinsic
hallucinations_. The rest is categorized as _other_. Here, we provide more
detailed results and examples from each type:

  * •

Intrinsic hallucinations that account for 24% of the annotated data, are non-
factual statements. Specifically, intrinsic hallucinations can be further
grouped into three cases.  (i) _Non-factual statements_ : we find that 11.5%
of the annotated examples made a non-factual statement, for example, “ _Deer
hunting is illegal in the United States._ ”  (ii) _Incorrect entities_ : we
find errors in 9% of the examples regarding entities such as time and place;
for example, the model said “ _Jane Eyre is a romantic novel by Charlotte
Bronte. It was published in 1891._ ” but actually, _Jane Eyre_ was published
in 1847.  (iii) _Ambiguous statement_ : we find that 3.5% of the examples had
ambiguous statements like “ _rap lyrics are often delivered in rhyme or in
punctuation._ ”

  * •

Extrinsic hallucinations that account for 27% of the annotated data, are
irrelevant responses, which can be further grouped into three cases.  (i)
_Out-of-context_ : we find that 14.5% of the annotated examples are out-of-
context in the sense that BART uses knowledge that is inconsistent with the
knowledge expected from the dialogue context, for example, a description of
the history of football when the user asks for the number of teams currently
in the NFL.  (ii) _Confusion of similar knowledge_ : we find that 6.5% of the
annotated examples suffer from confusion of knowledge where, for example, BART
said “ _Leonardo DiCaprio is probably the most well-known classical music
composer. He was born in the city of Rome in the 15th century._ ”, in which
BART confuses the actor _Leonardo DiCaprio_ with a classical music composer.
(iii) _Non-specific knowledge_ : we find that in 6% of the annotated examples
BART started the response by “ _I don’t know, but I do know that_ ” and
followed with a non-specific knowledge about the current dialogue topic.

  * •

Others that account for 49%. First of all, 32.5% of the annotated examples
were considered qualified responses. We find that of the remaining responses
8% used knowledge in a non-human-like way resulting in mechanical responses,
5.5% did not use knowledge, and 3% repeated the knowledge used in the previous
dialogue turns.

We summarize these findings in Fig. 5.

##  Appendix B Implementation details

We set the hyperparameters by pilot experiments following the common practice:
First, we set the ratio of the three losses to {0.4,0.3,0.3} to make them
balanced and not oversampled; Second, we use the objective shifting method
(Chen et al., 2020) to vary the scale linearly to {0.5,0.5,0} to avoid
knowledge forgetting and balance the upstream/downstream objectives. We set
the ratio of language model loss $\alpha_{3}$ to $0.3$ at initialization and
linearly decay it until $0$. We set $\alpha_{1}$ and $\alpha_{2}$, i.e., the
ratio of MLE loss and MCL loss, to $0.4$ and $0.3$, respectively, and linearly
increase them to $0.5$ and $0.5$. As for $\beta$, which controls the ratio of
the two span extraction functions, we set it to $0.5$. The number of negative
example ($M$) is set to $8$. We truncate the input sequence to a maximum of
128 tokens, and the output sequence to 64 tokens. We append task-specific
prompts (i.e., “Response generation”, “Knowledge identification”, and
“Wikipedia denosing”) to three loss (i.e., $\mathcal{L}_{\text{MLE}}$,
$\mathcal{L}_{\text{MCL}}$, and $\mathcal{L}_{\text{LM}}$) to distinguish
different training targets separately.

Figure 5:  Detailed results of the pilot annotation experiment.

All models have been trained with the AdamW optimizer with a constant learning
rate of $2e{-}5$. We use gradient clipping with a maximum gradient norm of
$0.1$. We optimize the model for up to 5 epochs with a batch size of 16 on 4
V100 GPUs. During training, we first extracted spans in negatives offline,
which takes about 1 hour using 20 threads. Then, we use 8 threads for getting
the training samples (num_workers=8 in data_loader) and find that this setup
does not increase the training time. We choose the model checkpoints by
evaluating the metrics on the validation set for each 5,000 training steps.
During inference, the responses are decoded using a greedy search. On a single
GPU, the training of the model can be completed within 10h, and the average
latency of the model inference for one example is 0.05s.

Table 4: An example of model outputs on the Wizard-of-Wikipedia test seen dataset. This example demonstrates MixCL’s ability to improve the informativeness of a response through the use of knowledge (highlighted with green). Topic |  Yoga
---|---
Context |  User1: Do you like yoga which is the practice not only physical but mental exercises or disciplines.
User2: I love yoga! I actually practice hot yoga pretty recently, it combines
my two favorite things: Sauna and Yoga.
Model outputs
DukeNet |  i love hatha yoga too ! i like hatha yoga and raja yoga .
KnowledGPT |  i love hatha yoga and raja yoga. i like both.
KnowExpert |  i love hot yoga too, it is a type of yoga that emphasizes physical exercises over emotional ones
MSDP |  Yes ! It ’s very intense , classes are usually an hour and a half and by the first half hour you are feeling it , the positions can be intense .
MixCL |  I love hot yoga as well. It’s been around since the 5th and 6th centuries BCE.
Human |  Oh wow that sounds interesting. I read that Hot Yoga is actually refered to as Bikram Yoga
Table 5: An example of model outputs on the Wizard-of-Wikipedia test unseen dataset. This example shows that MixCL uses knowledge correctly (highlighted with green) but repeatedly asks “ _What’s your favorite store?_ ” (highlighted with red). Topic |  Online shopping
---|---
Context |  User2: I like to shop online, probably a bit too much.
User1: Lol me too! What’s your favorite online store?
User2: I love to shop on amazon. So you shop online a lot too? Where do you
shop?
Model outputs
DukeNet |  i shop in amazon . i shop at the amazon . they are an american multinational technology company based in seattle , washington .
KnowledGPT |  i shop online at amazon. it’s a great online store. i like to shop at amazon.
KnowExpert |  i shop online at amazon. i like to shop online because i can find the best deals online.
MSDP |  Yes I do . I shop online a lot too . I know that Online shopping is a form of electronic commerce which allows consumers to directly buy goods or services from a seller over the Internet using a web browser .
MixCL |  I do! I like to shop online at Amazon.com and Barnes & Noble. What’s your favorite store?
Human |  Amazon as well. I also shop a lot online at Old Navy and Gap. They have great sales.
Table 6: An example of model outputs on the Wizard-of-Wikipedia test unseen dataset. This example shows KB-based methods failing due to a retrieval error (highlighted with red). MixCL generates a desired response that answers the user’s question and incorporates knowledge (highlighted with green). Topic |  Archery
---|---
Context |  User2: So I picked up archery recently and to my surprise, I really enjoy it. Have you ever tried?
User1: Yes, actually! I enjoy archery very much. I actually found out it comes
from (arcus) Latin!
User2: What other Latin words do you know?
|  User1: I don’t know many, but I do know that a significant portion of
English comes from Latinate sources.
|  User2: Huh. When did you start archery? Do you take part in any
competitions?
|  User1: I started when I was ten. I got a winning streak when I was 15,
which means I won a consecutive sequence of games.
|  User2: I think I want to enter one but I have no idea where to start.
Model outputs
DukeNet |  the earliest evidence of use traces to the olmecs ( mexico ) , with evidence of chocolate beverages dating to 1900 bc
KnowledGPT |  you should start by playing chess. it’s played by millions of people worldwide.
KnowExpert |  i’m not sure, but i do know that archery is a sport that involves using a bow and arrow to shoot a long-range shot at targets on a moving target.
MSDP |  I think I want to enter one but I have no idea where to start .
MixCL |  You should start with the World Archery Championship. It’s the biggest one in the world.
Human |  You could enter the Junior Grand Prix! It’s a series of international junior level competitions.
Table 7: An example of model outputs on the Wizard-of-Wikipedia test unseen dataset. This example shows MixCL producing a non-factual response (highlighted with red). Topic |  Paramedic
---|---
Context |  User2: my friend wanted to be a paramedic
User1: I wouldn’t want to be EMS on 911 calls. The scenes would haunt you for
life.
User2: yea thats why my friend quit
|  User1: I imagine it is not as bad not that there is enchanced 911 so
paramedics only have to go to the calls they are absolutely neccessary for
|  User2: oh yea really?
|  User1: In Winnipeg they have was is known as the winnipeg fire paramedic
servic, or WFPS. 6They have 2 different paramedic divisions depending on the
type of 911 call.
|  User2: oh i didnt know that.
|  User1: it started in 1882 with the Winnipeg Fire Department and grew from
there.
|  User2: dang thats so long ago
Model outputs
DukeNet |  it was built in 1738 the first fort in 1738 .
KnowledGPT |  the library has both adult and children’s books in over 30 languages. i think i have a lot of books in my library.
KnowExpert |  it was founded in 1881 by a man named william c. durant.
MSDP |  Wow that’s so long ago . I know that Paramedic is a children ’s book by Dr. Seuss , first published on August 12 , 1960 .
MixCL |  Yes, the Winnipeg Fire Department is one of the largest fire departments in the world.
Human |  It is from our view point but not in the grand scheme of things.
Table 8: An example of model outputs on the Wizard-of-Wikipedia test seen dataset. This example shows MixCL using distinct knowledge (highlighted with green) from human, but it is also appropriate. Topic |  Valedictorian
---|---
Context |  User2: I hope I am Valedictorian of my graduating class.
User1: Valedictorian is an academic title of success used in the United
States, Canada, Central America, and the Philippines for the student who
delivers the closing or farewell statement at a graduation ceremony
User2: I’m not sure what kind of grades I’ll need.
Model outputs
DukeNet |  the chosen valedictorian is the highest ranking grade average
KnowledGPT |  the chosen valedictorian is often the student with the highest ranking (highest grade point average, or gpa for short) among their graduating class
KnowExpert |  valedictorian is also known as a bachelors of arts or bsc
MSDP |  I don’t know what kind of grades I ’ll need . I know that: The valedictory address generally is considered a final farewell to classmates , before they disperse to pursue their individual paths after graduating .
MixCL |  In the United States, the title is sometimes awarded to a member of a graduating university class on the basis of contribution to the school rather than academic success.
Human |  valedictorian is often the student with the highest ranking (highest Grade Point Average, or GPA for short) among their graduating class.

##  Appendix C Case study

We provide examples of model output in Table 4, 5, 6,7, and 8. In Table 4, we
see that MixCL generates an informative response through the use of knowledge
(i.e., the history of yoga). Compared to the baselines, MixCL’s response is
more specific and relevant: DukeNet and KnowledGPT express subjective
preferences rather than knowledge, KnowExpert repeats the knowledge from the
previous round, and MSDP’s response is hallucinating.

In Table 5, we see that despite appropriate integration of knowledge, MixCL
repeatedly asked “ _What’s your favorite store_ ” as it ignores the dialogue
context, leading the annotator to consider its response is non-human. Still,
we find MixCL’s response to be competent compared to the baselines. Besides,
we find that MSDP’s response, although very informative, is wordy and preachy
with its rigid insertion of large paragraphs of knowledge.

In Table 6, we see that KB-based methods (DukeNet and KnowledGPT) suffer from
the problem of retrieval error. Their knowledge retrieval module
misinterpreted the dialogue context and retrieved irrelevant knowledge (e.g.,
chocolate beverages and chess). Meanwhile, we find that MSDP simply repeats
the last utterance. In contrast, MixCL’s response is the best in terms of
relevance, factuality, and humanlikeness, because it responds to the user’s
question, uses the correct knowledge, and is fluent in expression.

In Table 7, we provide an example that MixCL produces a non-factual response.
In Table 8, we provide an example that MixCL uses appropriate knowledge,
albeit distinct from the knowledge in the human response.


# Guidelines for Artifacts to Support Industry-Relevant Research on Self-
Adaptation

Danny Weyns1,2 Ilias Gerostathopoulos3 Barbora Buhnova4 Nicolás Cardozo5
Emilia Cioroaica6
Ivana Dusparic7 Lars Grunske8 Pooyan Jamshidi9 Christine Julien10 Judith
Michael11
Gabriel Moreno12 Shiva Nejati13 Patrizio Pelliccione14 Federico Quin1 Genaina
Rodrigues15
Bradley Schmerl12 Marco Vieira16 Thomas Vogel8,17 Rebekka Wohlrab12

1Katholieke Universiteit Leuven; 2Linnaeus University; 3Vrije Universiteit
Amsterdam;
4Masaryk University; 5Universidad de los Andes; 6Fraunhofer IESE; 7Trinity
College Dublin;
8Humboldt-Universität zu Berlin; 9University of South Carolina; 10University
of Texas Austin;
11RWTH Aachen University; 12Carnegie Mellon University; 13University of
Ottawa;
14Gran Sasso Science Institute; 15Universidade de Brasília;
16University of Coimbra; 17Paderborn University
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Artifacts support evaluating new research results and help comparing them with
the state of the art in a field of interest. Over the past years, several
artifacts have been introduced to support research in the field of self-
adaptive systems. While these artifacts have shown their value, it is not
clear to what extent these artifacts support research on problems in self-
adaptation that are relevant to industry. This paper provides a set of
guidelines for artifacts that aim at supporting industry-relevant research on
self-adaptation. The guidelines that are grounded on data obtained from a
survey with practitioners were derived during working sessions at the 17th
International Symposium on Software Engineering for Adaptive and Self-Managing
Systems. Artifact providers can use the guidelines for aligning future
artifacts with industry needs; they can also be used to evaluate the
industrial relevance of existing artifacts. We also propose an artifact
template.

##  1 Introduction

The term artifact has different meanings in the field of software engineering.
In software development communities, artifacts refer to documents,
deliverables, and work products [7]. Software engineering conferences
increasingly use the term artifact in association with publications. Such
artifacts can range from “functional” i.e., artifacts associated with the
research that are documented and exercisable, and include appropriate evidence
of verification and validation; to “replicated” i.e., the main results of the
paper have been independently obtained in a subsequent study by a person or
team other than the authors, without the use of author-supplied
artifacts.111https://conf.researchr.org/track/icse-2022/icse-2022-artifact-
evaluation In this paper, we refer to an artifact as a tangible object that is
created with the intention to drive research advances and compare and contrast
alternative
approaches.222https://conf.researchr.org/track/seams-2022/seams-2022-papers#Artifacts
Our particular focus is on artifacts that support research in the field of
self-adaptation and its applicability in practice.

Generally, different types of artifacts can be provided. Testbeds refer to
implementations of systems (or parts of systems) that support the study and
evaluation of open challenges in self-adaptive systems. Model problems provide
detailed specifications of systems that pose fundamental or characteristic
challenges that self-adaptive systems should address, either in general or in
specific domains. Frameworks refer to concrete or conceptual platforms with
generic functionality that can be selectively specialized by implementing
self-adaptation techniques or algorithms that are potentially useful in
different contexts and use. Datasets (e.g., logging data, sensor data, system
traces, survey raw data) can be used to develop, evaluate, and compare self-
adaptation approaches. These common types of artifacts are representative
classes of the existing artifact landscape.

Over the past decade, researchers in the field of self-adaptation developed
about 30 artifacts [20]. A pioneering example is Znn.com [1] that provides a
webserver system of a simplified news site. Znn.com’s testing environment
simulates the slash-dot effect, which are periods of abnormally high traffic
that overload the system. Another example artifact is TAS [23], which provides
a service-based system, where services offered by third-party providers are
dynamically composed into workflows delivering complex functionality under
uncertainties such as changing workload and fluctuations in the network.
DARTSim [15], instead, offers an artifact for the evaluation and comparison of
adaptation approaches for smart cyber-physical systems that are subject to
sensing errors among other uncertainties. Yet another example artifact is
RoboMax [3], which provides an extensible repository of robotic mission
adaptation exemplars. A recent study highlighted an increasing use of
artifacts in the evaluation of research results, although the adoption by
teams other than the developing team remains relatively low [2].

A recent analysis of the maturity of the field of self-adaptation [22] showed
that we are currently in the phases of internal and external enhancement and
exploration according to the model of Redwine and Riddle [16]. Hence, the
industrial application of self-adaptation in real-life practical and
commercial applications is of crucial importance to reach full maturity. Yet,
it is currently not clear to what extent the existing artifacts support
research activities of problems in self-adaptation that are relevant to
industry.

To that end, this paper provides a set of guidelines for future artifact
providers that aim at supporting industry-relevant research on self-
adaptation. In addition, the guidelines enable assessing existing artifacts on
their relevance to support industry-driven research. The guidelines are
grounded on the data obtained from a large-scale survey with practitioners.
The guidelines have been derived from the outcome of a series of working
sessions during the in-person day at the 17th International Symposium on
Software Engineering for Adaptive and Self-Managing Systems, SEAMS
2022.333https://conf.researchr.org/track/seams-2022/seams-2022-papers Note
that alignment of research with industry has been topic of study in other
domains, for instance in control [17] or in computing systems in general, see
e.g., [18, 4].

The remainder of this paper is structured as follows. In Section 2, we present
the process we followed to define the guidelines presented in this paper, and
we summarize the data that was used as input for the working sessions. Section
3 then presents the guidelines. In Section 4, we present a template to
describe industry-relevant artifacts on self-adaptation. Finally, we conclude
the paper in Section 5.

##  2 Background

In this section, we give an overview of the process we followed to create the
guidelines for artifacts, together with a brief summary of the data used to
define the guidelines.

###  2.1 Process used to Derive the Guidelines

Figure 1 shows the process we followed to derive the guidelines for artifacts.

Figure 1: Process used to derive guidelines.

In a recent large-scale survey, we collected data from 113 practitioners444The
participants range from different companies across the globe, active in a
variety of domains; they have various roles in software engineering with a
variety of experience (67% have at least 9 years of experience) [24]. about:
(a) the drivers and motivations to apply self-adaptation in industrial
software, (b) the use cases and approaches used to realize self-adaptation,
(c) the problems and challenges practitioners face, and (d) the opportunities
practitioners see for self-adaptation. For a summary of the initial results,
we refer to [24]. When defining the guidelines, we have used raw data on three
specific topics: (a) difficulties and challenges practitioners face when
applying self-adaptation, (b) problems for which they would appreciate support
from researchers, and (c) opportunities for applying self-adaptation. In
addition, we used categories and codes that were obtained from the raw data
through inductive reasoning (i.e., moving from specific fragments in comments
to general concepts) [19].

The coded data (elaborated below) was then used as input for the working
sessions that were organized during the in-person part of SEAMS 2022.
Concretely, we divided the group of 20 participants in three breakout groups.
Each group drafted a set of guidelines during two sessions of 1.5 hours.
Finally, the insights were brought together and discussed during a plenary
session of one hour. After the event, we consolidated the guidelines and
documented the results in this paper.

###  2.2 Survey Data

We summarize the results of the coding of the survey data by highlighting (a)
difficulties and challenges practitioners face when applying self-adaptation,
(b) problems for which they would appreciate support from researchers, and (c)
opportunities for applying self-adaptation.

Of all the surveyed practitioners, 35% regularly face difficulties and
challenges with applying self-adaptation, while 40% report that they sometimes
face problems. Table 1 shows the categories and codes collected from the
answers. The table shows that practitioners face difficulties and challenges
across different aspects of software engineering, from lifecycle issues to
design, engineering processes, and runtime.

Table 1: Difficulties and challenges in self-adaptive systems
as perceived by practitioners Categories and codes | #
---|---
Lifecycle issues | 22
Tuning/debugging | 10
Limitations of tools/methods | 7
System/environment evolution | 5
Design challenges | 20
Reliable/optimal design | 10
Design complexity | 9
Security | 1
People and process issues | 14
Skills/experience | 8
Process and management | 5
Documentation | 1
Runtime challenges | 13
Runtime uncertainty | 6
Data collection/evaluation | 5
Delayed/missing runtime changes | 2

Twenty percent of the practitioners express that they would frequently
appreciate support from researchers to tackle problems they face, while 23%
would sometimes appreciate support. Table 2 shows the categories and codes
collected from the answers. The table shows that practitioners would
appreciate support for tackling a variety of problems, from engineering to
providing guarantees of trust, user interaction-related solutions, management
of data, and the realization of a number of advanced features.

Table 2: Problems for which practitioners would appreciate support from researchers Categories and codes | #
---|---
Engineering | 22
Architecture & reuse | 7
Adoption | 7
Tools | 4
Frameworks | 4
Guarantees | 19
Trustworthiness | 9
Unknowns | 5
Security | 5
User interaction | 18
User experience | 7
Automation | 7
User involvement | 4
Data | 18
Govern data | 8
Assess data | 6
Machine learning | 4
Advanced features | 13
Goals | 6
Mechanisms | 4
Testing | 3

Table 3 shows the opportunities practitioners see for applying self-adaptation
that are currently not exploited. The main opportunities are related to system
activities and system properties. Other opportunities concern engineering
activities and human involvement in the self-adaptation process.

Table 3: Opportunities for self-adaptation in industry that are currently not exploited as mentioned by practitioners Categories and codes | #
---|---
System activities | 26
Autonomous operation | 10
Autoscaling | 8
AutoML & data management | 8
System properties | 20
Qualities | 12
Cyber-security | 8
Engineering activities | 15
DevOps & maintenance | 10
Adaptation patterns & libraries | 5
Human involvement | 5
Personalization | 4
Human-machine interaction | 1

##  3 Guidelines

We now present the consolidated guidelines. Table 4 provides an overview of
the guidelines that are divided in three groups: required, recommended, and
desirable guidelines. We discuss each group now in detail.

Table 4: Overview of guidelines for artifacts to support industry-relevant
research on self-adaptation Required Guidelines
---
P1: Artifacts should take into account relevant surrounding elements
P11: Artifact should take into account management of real-world data
P12: Artifacts should take into account humans-on-the-loop
P13: Artifacts should take into account industrial adoption of newly proposed
solutions
P2: Artifacts should take into account industry scale
P3: Artifacts should provide industry-relevant metrics
P4: Artifacts should use an open policy
Recommended Guidelines
R1: Integration with standard technologies
R2: Process-related artifacts
R3: Generality and extensibility of artifacts
R4: Artifacts that facilitate collaborations across communities
Desirable Guidelines
D1: Artifacts may be built to exist in an ecosystem
D2: Artifacts may leverage open-source projects

###  3.1 Required Guidelines

This first group of guidelines are required for industry-relevant artifacts in
self-adaptation. Yet, which of these guidelines are relevant and apply depends
on the type of artifact and its concrete purpose.

> P1: Artifacts should take into account relevant surrounding elements

Crucial for industry-relevant artifacts is that they take into account the
context in which self-adaptation is applied. We refine this guideline in three
concrete sub-guidelines.

> P1.1: Artifact should take into account management of real-world data

There are different kinds of data that a self-adaptive system must manage
during operation. Some are inherent to the functionality of the system, such
as requests from users to a website and sensor readings in a cyber-physical
system. Others are necessary for the self-adaptation process itself, such as
monitoring data collected for analysis and planning. It is important that
artifacts can manage data with characteristics representative of industrial
use, including volume, rate, noise, etc. Otherwise, research evaluated with
artifacts may not be able to deal with the data needs of industrial use.

> P1.2: Artifacts should facilitate humans-on-the-loop

The initial research in engineering self-adaptive systems was mainly concerned
with automating tasks [12, 9, 21]. In this research, the role of humans was in
essence limited to providing high-level goals that the system should fulfill.
Later, the role of humans-in-the-loop became prominent [5]. In this
perspective, humans can take different roles, from providing input to
supporting decision-making and guiding adaptation actions. Yet, several
researchers pointed out the potential implications and risks of uncertainty
caused by humans-in-the-loop [6, 11]. On the other hand, practitioners point
to the need for humans-on-the-loop in self-adaptive systems. In industrial
applications, humans should not be involved in the adaptation process all the
time to avoid that they become a bottleneck. The preferred role of humans is
instead supervision to achieve trustworthiness, avoid misuse or disuse of the
automated system. In this regard, artifacts should consider different
stakeholder roles: Users, operators, business owners, customers, etc., and
emphasize their concerns such as feasibility, added value, and Return On
Investment (ROI).

> P1.3: Artifacts should take into account industrial adoption of newly
> proposed solutions

Artifacts are developed to evaluate and demonstrate new research results,
ultimately with the goal of advancing the state-of-practice. It is however
well known that industry will only adopt research solutions if they are
demonstrated to be effective in the context of real operational requirements.
As such, transferring research results to the field requires organizations to
conduct a preliminary evaluation (e.g., in the form of an industrial
prototype, testbed or case study), which has a cost that may be seen as a risk
by managers and decision-makers. Artifacts can play a major role in the
reduction of such costs, thereby facilitating the experimentation of new
solutions by industry and promoting the adoption of research results. In
practice, artifacts should be built in such a way that allow industry to
easily develop proofs of concept to test the effectiveness of new research
solutions in their specific settings and adopt the results in their practice.

> P2: Artifacts should take into account industry scale

A representative scale is a key requirement for the industrial relevance of an
artifact. Artifacts to support industry-relevant research should have a
capacity and complexity that is characteristic of industrial practice. Taking
into account real-world data as highlighted in guideline P1.1 is one specific
facet of industry scale. Other aspects include the size of systems (e.g., a
representative topology of an Internet of Things network), requirements of
systems (e.g., representative requirements of health care systems including
privacy, performance, and cost), the load exercised on systems (e.g.,
realistic workloads used by Cloud resources), and underlying infrastructure
(e.g., representative messaging techniques for micro-service architectures).

> P3: Artifacts should provide industry-relevant metrics

To quantify the benefits and costs of self-adaptation, artifacts should
provide industry-relevant metrics. On the one hand, such metrics should
address quality characteristics of software systems to measure the
satisfaction of quality requirements. This allows practitioners and
researchers to evaluate a self-adaptive system built with an artifact from a
technical perspective. On the other hand, such metrics should also address a
business-oriented perspective, for instance, to evaluate a self-adaptive
system built with an artifact economically (e.g., added value and ROI as also
suggested by guideline P1.2). This allows customers and business owners to
assess self-adaptation solutions developed with a specific artifact. Moreover,
practitioners and researchers may select suitable artifacts based on their
technical and business interest by taking these metrics into account. This
allows them to easily pick up connection points for leveraging the self-
adaptation mechanisms.

> P4: Artifacts should use an open policy

To facilitate adoption in industry, the artifacts need to (a) be openly
available under clear licensing schemes, (b) leverage open standards (and
preferably be based on standard technology, see guideline R1, and open-source
projects, see guideline D2), and (c) ensure transparency. The transparency
condition is crucial not only in terms of artifact trustworthiness, but also
to support troubleshooting of defects in self-adaptation processes that are
likely to be common in industry, given the reported shortage of expertise on
the topic in the teams [24], and might require further process-related support
(see guideline R2). The transparency also facilitates repeatability (i.e.,
same team, same experimental setup, for the sake of validation),
reproducibility (i.e., different team, same experimental setup, for the sake
of statistical significance), and replicability (i.e., different team,
different experimental setup, for the sake of wider adoption). As artifacts
may evolve over time, clear guidance on versioning and quality assurance is
key to support industries willing to contribute to their extensions.

###  3.2 Recommended Guidelines

The second group of guidelines are recommended for industry-relevant artifacts
in self-adaptation. We advise these guidelines as they will add to the
relevance for industry.

> R1: Integration with standard technology

Central to industrial relevance of artifacts is the use of standard
technologies. This brings forward the following benefits:

  * •

Realism: Artifacts developed with standard technologies allow for a closer
level of realism due to the fact that the technologies used correspond to the
actual technologies.555Besides technologies used, real-world data traces,
realistic uncertainty profiles, etc. should be used to ensure industry
relevance.

  * •

Engagement: Using standard technologies ensures that a larger share of people
is familiar with how the artifact functions. Examples of standard technologies
(at the time of writing) include: Docker, Real-time Operating System (ROS) for
Robotics, Carla (self-driving simulator), etc.

  * •

Relevance: The demonstrated benefits and evidence of techniques, solutions, or
frameworks by using the artifact maps closer to industrial practice. More
specifically, technology-specific settings of problems or opportunities put
forward by industry connect more to the artifact when using the same
technologies.

> R2: Process-related artifacts

Process-related artifacts are those that are targeted at helping apply self-
adaptation in industrial settings. Rather than tools that implement or support
self-adaptation, these kinds of artifacts focus on software engineering
processes (e.g., checklists) for designing, testing, and deploying them. For
instance:

  * •

Design: Guidance is needed on how to ensure that when applying self-adaptation
in a particular context, that the system is complete and can provide a certain
level of assurances. This requires that the right state is being modeled and
monitored and the feedback loop is properly developed by combining tactics.
E.g., it is not necessary to monitor elements that do not contribute to the
decision-making about the system; similarly, if the self-adaptive system needs
information in the planning or analysis process, this information needs to be
stored in the knowledge that the self-adaptive system has about the system it
is managing, and/or that appropriate monitoring is in place to retrieve that
information in a timely and accurate way.

  * •

Testing: In order to practically apply self-adaptation, we need strategies for
testing self-adaptive systems aligned with current practice, definitions of
standard test metrics like coverage in the context of self-adaptation, and
guidelines on how to provide a testing environment that drives the adaptations
to be tested. Some of the research behind this has been done in [8, 13, 10],
but artifacts that can be used to operationalize self-adaptive testing would
be helpful in guiding industry on how and what to apply.

  * •

Deployment: Once an adaptation decision is made, the system needs to enact the
adaptation. This calls for strategies to apply adaptation actions in a safe
way. Artifacts to integrate the execution of system adaptation aligned with
state of the practice infrastructure for continuous deployment are key in
helping practitioners with closing the feedback loop of self-adaptation.

Process related artifacts may rely on, or take the form of best-practices
documents, checklists for practitioners to use to ensure they deal with
appropriate and common steps for self-adaptation, or test/deployment scripts
that practitioners can use to manage CI/CD pipelines for self-adaptive
systems.

> R3: Generality and extensibility of artifacts

Generality and extensibility of artifacts add to their applicability and
reusability. Therefore, it is important to define artifacts enabling their
reuse both across application domains, such as robotics, automotive, and
telecommunications, and/or axes of research/engineering activities, including
requirements, architecture, verification, analysis, deployment, evolution,
etc. Generality will leverage research results across domains and support the
integration of research/engineering activities. Extensibility, on the other
hand, enables the extension of existing artifacts to different domains and/or
research/engineering activities. Note that this does not imply that domain-
specific artifacts cannot not be effective and useful. For instance, RoboMax
[3], the artifact mentioned before, captures key sources of uncertainty and
adaptation concerns of missions for the domain of robotic systems. Such
domain-specific knowledge and know-how can add to the efficiency of applying
an artifact.

> R4: Artifacts that facilitate collaborations across communities and with
> industry

Industry requires simple and easily composable building blocks that can be
shared for experimentation. The motivation for business gain could come from
the composition of innovative ideas that have been developed by members of
different scientific communities and mapped into robust workflows.
Furthermore, science makes better progress when we have diversity of ideas and
the people who generate the ideas [14]. To enable cross-community
collaborations and integration with industry, researchers should make it easy
for others to build on top of the ideas and artifacts they generate. This
calls for an infrastructure that easily facilitates automatic evaluation of
methods/techniques/algorithms/etc. against benchmarks. Such an infrastructure
will enable others to engage, while providing input and feedback from these
diverse set of people. Several examples exist that have already demonstrated
the added value; we give two examples:

  * •

DLRM666https://ai.facebook.com/blog/dlrm-an-advanced-open-source-deep-
learning-recommendation-model/ is a platform to which researchers from
Computer Architecture and Systems and Recommendation Systems actively
contribute. DLRM was developed by academics while placed as interns at
Facebook.

  * •

MLCommons777https://mlcommons.org/en/ supported the development of multiple
important benchmarks such as MLPerf and TinyML for researchers interested in
machine learning for systems and systems for machine learning.

In sum, we need an infrastructure, preferably open source, that makes it
easier for researchers to share innovative ideas, for developers to experiment
with ideas over a short time window (e.g., a weekend), and for software
companies to make the ideas production-ready. This creates opportunities for
more researchers, within and across different communities, to engage, and for
industry to contribute to industry-relevant research efforts and engage with
the research communities.

###  3.3 Desirable Guidelines

The third group of guidelines are desirable for industry-relevant artifacts in
self-adaptation. These guidelines are worth having as they will add to the
attractiveness for industry.

> D1: Artifacts may be built to exist in an ecosystem

Desirable properties of artifacts for industry-relevant research on self-
adaptation are extensibility and integratability, so that they can accommodate
surrounding elements (compare also guideline R3). In the past, the focus of
self-adaptive systems has been on mechanisms to adapt systems’ behaviors or
structures, whereas surrounding elements (e.g., data management, human-machine
interaction, and integration with basic infrastructure) have received little
attention (see guideline P1). To integrate surrounding elements, preferable in
a lightweight way, it is desirable not to focus on specifics of self-adaptive
systems in isolation, but to design artifacts to co-exist within an ecosystem.
To support this guideline, for example, data management solutions may be
provided as reusable components. Moreover, artifacts should be designed so
that they can be extended and integrated with other systems (see also
guideline R4). This will add to a holistic perspective, which is a key aspect
for industrial relevance of artifacts.

> D2: Artifacts may leverage open-source projects

Table 5: Proposed template for industry-relevant artifacts to support research on self-adaptive systems Element |  Brief description
---|---
Problem |  A brief description of the industry-relevant problem tackled by the artifact
Prerequisites |  Requirements or constraints for using the artifact and the research results obtained with it
Technologies |  Relevant state-of-the-practice technologies associated with the artifact
Outcome |  The expected outcome of using the artifact and adopting the solution in practice (e.g., quantifiable by metrics)
Adoption |  The relevant aspects required to adopt the research results obtained by using the artifact
Example |  An exemplary solution to the problem and how it was adopted
Related artifacts |  Artifacts that address similar problems or that can be combined to enrich the solution
Maintenance |  A version history, and a reference to the maintainer of the artifact who keeps it up-to-date and responds to change requests

To increase the industrial relevance of artifacts and their acceptance by
industry, they would benefit from relying on well-known and robust projects.
The use of open source projects may help artifacts to have a more stable
underlying infrastructure. On the one hand, relying on existing infrastructure
from open-source projects (e.g., containers, continuous integration tools,
frameworks) reduces the learning curve for practitioners, and eases deployment
and integration of the artifacts solutions with their technology base. On the
other hand, using existing projects as a base, can lead artifacts to reach a
readiness level accepted by industry faster. Finally, we note that adopting an
open-source process for the development of artifacts can be beneficial for
their evolution and customization. As artifacts become more accepted,
particular industries can customize a base artifact for their particular
needs. As the artifacts evolve, new features can be integrated easily with
branched and customized development.

##  4 Template for Industry-Relevant Artifacts

To support industry-relevant research on self-adaptation, artifact
descriptions would benefit from a standard structure, similarly to the uniform
descriptions of design patterns. Standard template-based descriptions support
both researchers and practitioners in searching, identifying, understanding,
and comparing artifacts that are relevant for a given problem. Otherwise, the
artifacts themselves have to be investigated, analyzed, and even executed to
assess their relevance for the given problem and technology, which is a
difficult and time-consuming effort.

Table 5 shows the proposed template for industry-relevant artifacts.

The elements of the template were derived from the consolidated guidelines.
The top part considers the basic elements to describe an artifact, from a
description of the problem to the adoption of research results obtained with
the artifact. The elements of the middle part provide an example solution and
related artifacts that put the artifact in a broader context. Finally, the
bottom part captures maintenance of the artifact in face of the rapid progress
of technology in industry. Up-to-date maintained artifacts are essential to
industrial relevance research. We anticipate that different types of templates
might be needed to accommodate the description of different types of
artifacts, such as testbeds, frameworks, datasets, and checklists. Finally,
although the template elements proposed in Table 5 were derived based on data
obtained from practitioners in the context of self-adaptation, the elements of
the template are generic and applicable to industry-relevant research in other
areas. Yet, more study is required to underpin this observation.

##  5 Conclusions

Artifacts are key to systematic evaluation of new research results in a field
of interest. We provided a set of empirically grounded guidelines for
artifacts that aim at supporting industry-relevant research in the field of
self-adaptation. The guidelines can be used by artifact providers for aligning
future artifacts with industry needs as well as to assess the industrial
relevance of existing artifacts. Finally, we proposed a template for
describing industry-relevant artifacts to support research on self-adaptive
systems. We hope that this paper will support the research community in self-
adaptation to further enhance its maturity, the broader software engineering
community to engage across their borders, and practitioners in adopting
research results to build better software systems.

## Acknowledgment

We are grateful to the practitioners that participated in the survey that we
used as the basis for the presented guidelines.

## References

  * [1] Znn.com.  https://github.com/cmu-able/znn, 6/2022.
  * [2] Shubham A.  Analysing and assessing self-adaptive systems and designing a general criterion to evaluate sas artifacts.  MSc Thesis Vrije Universiteit Amsterdam, 2020.
  * [3] M. Askarpour, C. Tsigkanos, C. Menghi, R. Calinescu, P. Pelliccione, S. García, R. Caldas, T. von Oertzen, M. Wimmer, L. Berardinelli, M. Rossi, M. Bersani, and G. Rodrigues.  Robomax: Robotic mission adaptation exemplars.  In International Symposium on Software Engineering for Adaptive and Self-Managing Systems, 2021.
  * [4] N. Boules, K. Douglas, S. Feldman, L. Fix, G. Hager, B. Hailpern, M. Hebert, D. Lopresti, B. Mynatt, C. Rossbach, and H. Wright.  The future of computing research: Industry-academic collaborations.  Computing Community Consortium, Version 2:1–19, 6/2022.
  * [5] J. Cámara, G. Moreno, and D. Garlan.  Reasoning about human participation in self-adaptive systems.  In IEEE/ACM 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, 2015.
  * [6] N. Esfahani and S. Malek.  Uncertainty in self-adaptive software systems.  In R. de Lemos, H. Giese, H. Müller, et al., editors, Software Engineering for Self-Adaptive Systems II: International Seminar, Dagstuhl Castle, Germany, October 24-29, 2010 Revised Selected and Invited Papers, pages 214–238. Springer Berlin Heidelberg, 2013.
  * [7] D. Fernández, W. Böhm, A. Vogelsang, J. Mund, M. Broy, M. Kuhrmann, and T. Weyer.  Artefacts in software engineering: a fundamental positioning.  Softw. Syst. Model., 18(5):2777–2786, 2019.
  * [8] E. Fredericks and B. Cheng.  Automated generation of adaptive test plans for self-adaptive systems.  In IEEE/ACM 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, 2015.
  * [9] D. Garlan, S.W. Cheng, A.C. Huang, et al.  Rainbow: Architecture-based self-adaptation with reusable infrastructure.  Computer, 37(10):46–54, 2004.
  * [10] O. Gheibi and D. Weyns.  Lifelong self-adaptation: Self-adaptation meets lifelong machine learning.  In IEEE/ACM 17th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, 2022.
  * [11] S. Hezavehi, D. Weyns, P. Avgeriou, R. Calinescu, R. Mirandola, and D. Perez-Palacin.  Uncertainty in self-adaptive systems: A research community perspective.  ACM Transactions on Autonomous and Adaptive Systems, 15(4), 2021\.
  * [12] J. Kephart and D. Chess.  The vision of autonomic computing.  Computer, 36(1):41–50, 2003.
  * [13] M. Langford and B. Cheng.  Know what you know: Predicting behavior for learning-enabled systems when facing uncertainty.  In International Symposium on Software Engineering for Adaptive and Self-Managing Systems, 2021.
  * [14] D. Medin and C. Lee.  Diversity makes better science.  APS Observer, 25(5), 2012.
  * [15] G. Moreno, C. Kinneer, A. Pandey, and D. Garlan.  Dartsim: An exemplar for evaluation and comparison of self-adaptation approaches for smart cyber-physical systems.  In IEEE/ACM 14th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, 2019.
  * [16] S. Redwine and W. Riddle.  Software technology maturation.  In 8th International Conference on Software Engineering, page 189–200, Washington, DC, USA, 1985. IEEE.
  * [17] T. Samad, M. Bauer, S. Bortoff, S. Di Cairano, L. Fagiano, P. Odgaard, R. Rhinehart, R. Sánchez-Peña, A. Serbezov, F. Ankersen, P. Goupil, B. Grosman, M. Heertjes, I. Mareels, and R. Sosseh.  Industry engagement with control research: Perspective and messages.  Annual Reviews in Control, 49:1–14, 2020.
  * [18] A. Scandura and S. Iammarino.  Academic engagement with industry: the role of research quality and experience.  Technology Transfer, Preprint:1–37, 2021.
  * [19] K. Stol, P. Ralph, and B. Fitzgerald.  Grounded theory in software engineering research: A critical review and guidelines.  In 38th International conference on Software Engineering (ICSE), pages 120–131, 2016.
  * [20] T. Vogel.  Catalog of SEAMS artifacts: http://self-adaptive.org/exemplars.  6/2022.
  * [21] D. Weyns.  Software engineering of self-adaptive systems.  In Sungdeok Cha, Richard N. Taylor, and Kyochul Kang, editors, Handbook of Software Engineering, pages 399–443. Springer International Publishing, Cham, 2019.
  * [22] D. Weyns.  An Introduction to Self-adaptive Systems: A Contemporary Software Engineering Perspective.  John Wiley & Sons, 2021.
  * [23] D. Weyns and R. Calinescu.  Tele assistance: A self-adaptive service-based system exemplar.  In IEEE/ACM 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems, 2015.
  * [24] D. Weyns, I. Gerostathopoulos, N. Abbas, J. Andersson, S. Biffl, P. Brada, T. Bures, A. Di Salle, P. Lago, A. Musil, J. Musil, and P. Pelliccione.  Preliminary results of a survey on the use of self-adaptation in industry.  In International Symposium on Software Engineering of Adaptive and Self-Managing Systems. https://arxiv.org/abs/2204.06816, 2022.