Top mass shift resulting from the recalibration of flavor-dependent jet energy
corrections in the DO lepton+jets top mass measurement

Hannu Siikonen, T. Mäkelä and M. Voutilainen

Helsinki Institute of Physics
University of Helsinki, FINLAND

> We review the possible effects of the miscalibration of the flavor-dependent
> Jet Energy Corrections ($F_{\mathrm{Corr}}$) on the DO lepton+jets top mass
> measurement. The work is based on a previous study, where the DO
> $F_{\mathrm{Corr}}$ calibration procedure was repeated based on the release
> of the internal DO notes after a 5-year moratorium. The cited study was
> motivated by the extraordinary precision claimed by DO in their top mass
> measurement. Using a Pythia6-based $F_{\mathrm{Corr}}$ recalibration, the
> $m_{t}$ result was shifted from 174.95 to 173.16 GeV. Moreover, utilizing a
> Herwig7-based $F_{\mathrm{Corr}}$ calibration (not accounted for in the DO
> studies), a shift down to 171.84 GeV was observed. We find this both
> convincing and specific evidence for re-reviewing a part of the
> $F_{\mathrm{Corr}}$ calibration process. However, DO has been unwilling to
> open such studies or to provide other convincing counter-evidence.

> PRESENTED AT
>
>
>
>
> $13^{\mathrm{th}}$ International Workshop on Top Quark Physics
> Durham, UK (videoconference), 14–18 September, 2020

##  1 Introduction

The most notable measurements of the top quark mass ($m_{t}$) are those
performed by the Tevatron collaborations CDF and DO and those performed by the
LHC collaborations ATLAS and CMS. The DO $m_{t}$ combination ($174.95\pm 0.75$
GeV) [1] is a distinct outlier in the $m_{t}$ world combination [2]. The
measurements by CDF, ATLAS and CMS tend to produce values ranging from 172 to
173 GeV. Moreover, the DO combination is almost completely driven by the
lepton+jets measurement ($174.98\pm 0.76$ GeV) [3]. This gives motivation for
studying the calibrations behind this measurement.

The Master’s thesis written by Toni Mäkelä [4] made an in-depth study of the
DO flavor-dependent Jet Energy Corrections ($F_{\mathrm{Corr}}$). It was based
on the DO internal notes that were released from their five-year moratorium
early in 2018. Some DO authors were helpfully collaborating with Toni, and
provided him with all the necessary material and communication. As a notable
detail, the DO Run IIb $F_{\mathrm{Corr}}$’s differ considerably from those of
Run IIa. Toni’s most significant finding was that the Run IIb results should
have indeed resembled more those of Run IIa.

In the $m_{t}$ shifting analysis, performed by the main author of this text,
we apply the $F_{\mathrm{Corr}}$ values produced by Toni into a lepton+jets
top mass measurement. The study utilizes a standalone top quark pair
simulation and a detailed mathematical framework for propagating the
$F_{\mathrm{Corr}}$ changes into the $m_{t}$ measurement [5]. In the TOP2020
poster session, the results obtained with this method were presented.

Figure 1:  $F_{\textrm{corr}}$ values in the four DO RunII eras. Row 1: DO
histograms (open markers), DO fits (lines) and our P6 reproduction of the DO
results (closed markers). Row 2: our P6 (continuous line) and H7 (dotted line)
fits vs. DO fits (dashed line).

##  2 Observations and Interpretation

Toni’s analysis consists of two steps: refitting the 3 Single Particle
Response (SPR) parameters and making parametrized $F_{\mathrm{Corr}}$ fits
based on these. The analysis was performed both on Pythia6 (P6) and Herwig7
(H7). P6 was run with the exact same settings that DO used. H7 was used to
probe the possible systematic errors, although no DO tune was available. We
believe that Herwig or other secondary Monte Carlo (MC) event generators have
not been utilized sufficiently in the DO’s Jet Energy Calibration and error
analysis [6].

Toni’s measurements are distilled into Figure 1. On Row 1, we observe DO’s
histogram-data and the fits based on these. The closed markers display
histograms based on Toni’s SPR remakes. It is observed that Toni has found the
same histogram results as DO. In contrast, DO’s histograms and fits agree only
for Run IIa, but not for the Run IIb eras. The histograms also show better
time-stability than the fits.

On Row 2 of Fig. 1, the histograms are not shown, but the same DO fits are
displayed. In addition, Toni’s fits are given. His P6 fits agree with the
histograms and are more stable in time than the DO ones. Both the fits and
histograms for H7 show a great deviation from the P6 results. The H7
histograms are not displayed in the Figure, but these agree with the H7 fits.
This motivates a suspicion of the remarkably small $F_{\mathrm{Corr}}$ errors
stated by DO, even if a DO tune for H7 was not available.

The $m_{t}$ shifting method presented in Ref. [5] utilizes Toni’s P6 and H7
fits. The method takes DO’s original fits as a reference and consequently
probes the changes in the measured $m_{t}$ values introduced by Toni’s fits.
The results are considered run-by-run (Run IIa and Run IIb{1,2,34}) and
separating the electron and muon channels. The $m_{t}$ analysis is performed
purely with P6, utilizing DO’s tuning. We do not expect similar generator-
dependence in the $m_{t}$ measurement, as in the $F_{\mathrm{Corr}}$
measurement.

The final $m_{t}$ shifts using the P6 and H7 $F_{\mathrm{Corr}}$ values are
given in Fig. 2. It is noteworthy that the P6-shifted $m_{t}$ result is quite
close to that of the CDF collaboration. The generator choices of DO and CDF
are similar, and hence also the related systematic errors should agree. The
ATLAS and CMS results agree in a similar fashion as the CDF and the P6-shifted
DO results. Interestingly, a simple linear combination of the P6-shifted and
H7-shifted DO results produces a value close to those of the CMS and ATLAS
collaborations. The LHC collaborations utilize more heavily Herwig in their
jet calibrations than the Tevatron collaborations. This could indicate that
the choice of generators in the jet calibration process sets the rough
baseline for results obtained in a precision measurement of the top mass.

Figure 2: The most prominent top mass measurements and the DO measurements
shifted using Toni’s $F_{\textrm{corr}}$ values. Lepton+jets channel dominates
the DO result.

##  3 Discussion

Toni has presented his results actively during 2018-2019, including a LHC Top
WG meeting. After finishing his Master’s thesis, he left our group in mid-2019
to pursue a PhD in Germany. The final $m_{t}$-shift results were published as
an arXiv-preprint in early 2020, to be later included in the main author’s PhD
thesis. In the spring of 2020, the results were presented and handled in an
LHC Top WG meeting. In this meeting, some of the DO contacts got active, and
were unknowing of Toni’s previous contact efforts. The situation was
extraordinary, as Toni had been proactively in contact with the DO authors.
The agreement of the meeting was that the mass shifting method itself seems
trustworthy. However, the DO contacts disagreed on Toni’s results. Toni
provided further information about his studies, but the contact from DO dried
up.

The $m_{t}$ shifting results were submitted as a poster to the TOP2020
conference. We did not hear again from DO sooner than one day before the
poster presentation in a short letter [7]. Here, it is addressed that _There
were no interactions between the authors of Refs. [1, 4] and the primary D0
authors concerning the details in the internal notes_. This is not how we
observed the process, where Toni has been very active into the direction of
DO. Thanks to his contact with the DO authors, he received all the material
and calibrations reasonably available, making the former statement
questionable. Furthermore, some of the important findings were completely
disregarded in the letter, such as the H7-based systematics treatment.

The letter aims for closing the discussion without needing to check the
$F_{\mathrm{Corr}}$ calibration. The disagreement between DO fits and
histograms is spoken off by mentioning that _the plots in the internal notes
could have been misleading_. It is stated that the DO review process is very
thorough, and hence they do not expect any errors. This is an incomplete
argument for defending the calibration. The final paper releases are less
detailed than the analysis notes. The details ending up in the published paper
are generally reviewed with the help of the more detailed analysis notes. If
there are internal disagreements within the notes, how can the calibrations of
the final paper be trusted? To complete their argument, DO should show that
the final fits were indeed corrected from the ones presented in the note.
Without a visual or numerical confirmation, the statements of Ref. [7] remain
unconvincing.

Despite the tight schedule, the TOP2020 organizers allowed adding a mention of
the DO letter into the poster. Regrettably, a thorough reciprocal discussion
of the DO statements did not enter the poster session. The audience was
generally curious, and the most critical voices of the DO letter were absent.

In short, these $m_{t}$ studies aimed for a DO re-review of a specific part of
their $F_{\mathrm{Corr}}$ calibration. With the release of DO’s internal
notes, our collaboration with DO started auspiciously. Utilizing the available
material, we have completed a detailed study of the DO analysis chain.
However, DO has responded to this only with a short letter including
controversial statements. This is not consistent with good scientific
practice. Going into more detail is infeasible without DO taking the lead.

We wish that DO would pick up the issue and re-review the $F_{\mathrm{Corr}}$
calibration. We have pinpointed the location of the possible issues, so this
should not be labor-intensive. With the amount of work and evidence that we
have provided, failing to do this serves against the self-corrective ideals of
science.

## References

  * [1] V. M. Abazov et al. [D0 Collaboration], Phys. Rev. D 95 (2017) no.11, 112004
  * [2] K. A. Olive et al. (Particle Data Group), “The Review of Particle Physics”, Chin. Phys. C, 38, 090001 (2014) and 2015 update.
  * [3] V. M. Abazov et al. [D0 Collaboration], Phys. Rev. D 91 (2015) no.11, 112003
  * [4] T. Mäkelä, Aalto University Document Server (2019) https://aaltodoc.aalto.fi/handle/123456789/39024
  * [5] H. Siikonen, 2020-02-14, https://arxiv.org/abs/2002.06073
  * [6] V. M. Abazov et al. [D0], Nucl. Instrum. Meth. A 763 (2014), 442-475
  * [7] DO collaboration, https://www-d0.fnal.gov/Run2Physics/WWW/results/final/TOP/T14E/D0_top-JES_statement.pdf


# Diversifying Dialog Generation via Adaptive Label Smoothing

Yida Wang1,2, Yinhe Zheng1,311footnotemark: 1, Yong Jiang2, Minlie Huang1
1 The CoAI group, DCST, Institute for Artificial Intelligence, State Key Lab
of Intelligent
Technology and Systems, Beijing National Research Center for Information
Science and Technology, Tsinghua University, Beijing, China
2 Tsinghua-Berkeley Shenzhen Institute, Tsinghua Shenzhen International
Graduate School,
Tsinghua University, Shenzhen, China
3 Samsung Research China - Beijing (SRC-B)
<EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS>
Equal contribution Corresponding Author<EMAIL_ADDRESS>
###### Abstract

Neural dialogue generation models trained with the one-hot target distribution
suffer from the over-confidence issue, which leads to poor generation
diversity as widely reported in the literature. Although existing approaches
such as label smoothing can alleviate this issue, they fail to adapt to
diverse dialog contexts. In this paper, we propose an Adaptive Label Smoothing
(AdaLabel) approach that can adaptively estimate a target label distribution
at each time step for different contexts. The maximum probability in the
predicted distribution is used to modify the soft target distribution produced
by a novel light-weight bi-directional decoder module. The resulting target
distribution is aware of both previous and future contexts and is adjusted to
avoid over-training the dialogue model. Our model can be trained in an end-to-
end manner. Extensive experiments on two benchmark datasets show that our
approach outperforms various competitive baselines in producing diverse
responses.

##  1 Introduction

The success of neural models has greatly advanced the research of dialog
generation Huang et al. (2020); Wang et al. (2020); Zhang et al. (2020).
However, most of these models suffer from a low-diversity issue where models
tend to generate bland and generic responses such as _I don’t know_ or _I’m
OK_ Li et al. (2016). Although various approaches have been proposed to tackle
this issue Li et al. (2016); Zhao et al. (2017); Du et al. (2018); Zhou et al.
(2018); Welleck et al. (2020); Zheng et al. (2020b), there are still
remarkable gaps between responses generated by neural models and those from
humans Holtzman et al. (2020). Further, some existing methods may even harm
the fluency or coherence when improving the diversity of generated responses.
Ippolito et al. (2019); Massarelli et al. (2020); Zheng et al. (2020a).

Recently, Jiang and de Rijke (2018); Jiang et al. (2019) show that there is a
strong connection between the low-diversity problem and the over-confidence
issue. i.e., over-confident dialogue models tend to produce low-diversity
responses. One of the reasons can be attributed to the supervision target.
Specifically, training a dialogue generation model with the Maximum Likelihood
Estimation (MLE) objective under the hard target (i.e., one-hot distribution
as ground truth) makes the model favor high-frequency tokens and produce over-
confident probability estimation Gowda and May (2020), which ultimately leads
to poor calibration Mukhoti et al. (2020), and thus low diversity Jiang et al.
(2019). Hinton et al. (2015) and Yang et al. (2018) suggest that the ideal
training target should be a soft target that assigns probability mass on
multiple valid candidates (see Figure 1). With such a soft target, the over-
confidence issue can be alleviated Müller et al. (2019), and thus the
diversity of the output responses can be improved.

Figure 1: A dialogue sampled from the OpenSubtitles dataset. We demonstrate
the hard target, label smoothing, and Adaptive Label Smoothing approach when
learning to predict the next word (_“human”_).

Unfortunately, the ideal soft target is challenging to obtain. Early works try
to tackle this issue using label smoothing Szegedy et al. (2016), i.e., a
small probability is uniformly assigned to non-target words. However, the
target distribution constructed in this way is far from ideal: First, the
probability of the target word is chosen manually and fixed, which cannot
adapt to different contexts. However, as Holtzman et al. (2020) demonstrated,
human text distribution exhibits remarkable fluctuations in the per-token
perplexity. We argue that different target probabilities should be used for
different contexts. Second, the uniform assignment of the probability mass on
non-target words ignores the semantic relationship between the context and
each word. Ideally, a word should receive more probability mass if it is more
relevant to the context. For the example shown in Figure 1, word “ _fun_ ” is
more likely to appear behind the context “ _I make the robots seem more __ ”
than word “ _bank_ ”.

To address the above issue, we propose an Adaptive Label smoothing (AdaLabel)
method that can dynamically estimate a soft target distribution at each time
step for different contexts. Specifically, for each target word $y_{t}$ in the
training data, the probability distribution predicted by the current model is
first obtained. The maximum probability $p_{max}$ in this distribution
measures the confidence of the current prediction, i.e., a higher $p_{max}$
means higher confidence for the current prediction. To avoid over-confidence,
we use $p_{max}$ as the supervision signal for the target word $y_{t}$ in the
training process so that the model will not be optimized towards $y_{t}$ when
it correctly predicts $y_{t}$. A word-level factor is also introduced to
facilitate the learning of low-frequency words.

Moreover, we introduce a novel auxiliary decoder module $D_{a}$ to produce the
supervision signals for these non-target words in each training step. $D_{a}$
only contains one transformer block, and it is optimized to predict words
based on bi-directional contexts. A novel Target-Mask attention scheme is
devised to prevent $D_{a}$ from seeing the target word in the training
process. This scheme also enables parallel training and inference of $D_{a}$.

We perform extensive experiments on two benchmark datasets: DailyDialog and
OpenSubtitles. Our method outperforms various competitive baselines and
significantly improves the diversity of generated responses while ensuring
fluency and coherency. Our major contributions are summarized:

1. We propose AdaLabel, a method that can produce a soft target distribution considering the current context and the model’s confidence. Specifically, AdaLabel ensures that the dialogue model will not be optimized toward the target word $y_{t}$ if $y_{t}$ has been correctly predicted. This prevents our model from being over-confident.

2. We introduce a light-weight bi-directional decoder that can produce context-aware supervision signals for non-target words. A novel Target-Mask attention scheme is devised to facilitate the parallel training and inference of this decoder.

3. Extensive experiments on two benchmark dialogue datasets with both automatic and human evaluation results show that our method helps to alleviate the model over-confident issue and significantly improves the model’s diversity.

##  2 Related work

Diversity Promotion: Existing approaches for solving the low diversity issue
of neural dialogue models generally involve two categories:

The first category is training-based, where new training objectives are
designed Li et al. (2016); Zhang et al. (2018); Gao et al. (2019) or latent
variables are introduced Zhao et al. (2017); Zhou et al. (2018) in the
dialogue model. Some methods also try to refine the training target used in
the MLE loss Choi et al. (2020); Jiang et al. (2019); Li et al. (2019), or
directly penalize the trivial responses with auxiliary loss terms Welleck et
al. (2020); Li et al. (2020). Unlike these existing approaches, our method
tries to adaptively adjust the training target by utilizing the current
predictions.

The second category is decoding-based, in which different heuristic decoding
rules are designed Holtzman et al. (2020); Kulikov et al. (2019). Note that
these decoding techniques are independent of the model setting, and our method
can be used in combination with these techniques.

Figure 2: Overview of constructing the adaptive soft target $\bm{q^{\prime}}$
using AdaLabel: The maximum probability $p_{max}$ in the predicted
distribution $\bm{p}$ is used to obtain an adaption factor $\epsilon$, which
is further used to combine the hard target $\bm{q}$ and the auxiliary
distribution $\bm{v}$ to obtain $\bm{q^{\prime}}$. A bi-directional auxiliary
decoder $\mathcal{D}_{a}$ is used to produce $\bm{v}$.

Confidence Calibration: Modern deep neural networks suffer from the over-
confidence issue Guo et al. (2017); Kumar and Sarawagi (2019), and various
remedies are proposed Pereyra et al. (2017); Mukhoti et al. (2020); Lin et al.
(2017). Following the work of Jiang and de Rijke (2018); Jiang et al. (2019),
our method is proposed to tackle the over-confidence issue to improve the
diversity of the generated responses. However, different from existing
approaches, our method enables more flexible controls over the target
distribution.

Knowledge Distillation: Another important technique similar to our work is
knowledge distillation, in which a learned teacher model is distilled to a
student model by minimizing a KL term Hinton et al. (2015); Kim and Rush
(2016).

The most related work comparing to ours is the C-MLM approach Chen et al.
(2020), in which a BERT model is fine-tuned to be a teacher. Our approach and
C-MLM’s primary difference is that our auxiliary decoder $D_{a}$ is a one
layer module that is jointly trained with the dialogue model. However, the
BERT teacher in C-MLM contains much more parameters, and it is trained using
an expensive pre-trained and then fine-tuned process. Moreover, the target-
masked attention scheme in $D_{a}$ enables parallel inferences of $\bm{v}$ for
each training sequence $Y$. In contrast, multiple independent forward passes
are required for the BERT teacher.

##  3 Method

###  3.1 Background: MLE with Hard Target

The goal of generative dialogue modeling is to learn a conditional probability
distribution $p(Y|X)$, where $X$ is the dialogue context, $Y=y_{1},...,y_{T}$
is a response word sequence, and $y_{i}\in\mathcal{V}$ is a word from the
vocabulary $\mathcal{V}$. In an auto-regressive manner, $p(Y|X)$ is factorized
as $\prod_{t}p(y_{t}|y_{<t},X)$. For each target word $y_{t}$ in the training
sequence $Y$, a conventional MLE training approach try to optimize the
following cross entropy loss:

$\mathcal{L}(\bm{q},\bm{p})=-\sum_{w_{k}\in\mathcal{V}}q_{k}{\rm
log}\left[p(w_{k}|y_{<t},X)\right],$ (1)

where $\bm{q}$ is a one-hot distribution (i.e., a hard target) that assigns a
probability of $1$ for the target word $y_{t}$ and 0 otherwise, i.e.,
$q_{k}=1$ only when $w_{k}=y_{t}$. For simplicity of notation, we abbreviate
the dependency of $y_{t}$ in the notation of each distribution in our paper,
i.e., different target word $y_{t}$ in $Y$ corresponds to different values of
$\bm{q}$ and $\bm{p}$.

###  3.2 Method Overview

We propose to adaptively construct a soft target distribution
$\bm{q^{\prime}}$ to replace $\bm{q}$ in Eq. 1. Specifically,

$\bm{q^{\prime}}=\varepsilon\cdot\bm{q}+(1-\varepsilon)\cdot\bm{v},$ (2)

where $\varepsilon\in[0,1]$ is an adaption factor, and $\bm{v}$ is an
auxiliary distribution vector that depends on the current time step. (see
Figure 2 for an overview).

In this study, we constrain $\bm{v}$ to assign zero probability for the target
word $y_{t}$ and non-zero probabilities for these non-target words
$\mathcal{V}_{\neq y_{t}}=\\{y_{i}|y_{i}\in\mathcal{V},y_{i}\neq y_{t}\\}$.
This constraint allows us to explicitly control the supervisions assigned to
$y_{t}$. Specifically, the first term $\varepsilon\cdot\bm{q}$ and the second
term $(1-\varepsilon)\cdot\bm{v}$ in Eq. 2 respectively determines how much
probability $\bm{q^{\prime}}$ assigns to $y_{t}$ and $\mathcal{V}_{\neq
y_{t}}$. This setting differs from conventional knowledge distillation Kim and
Rush (2016) because it facilitates more flexible controls over
$\bm{q^{\prime}}$, so that we can use the factor $\varepsilon$ to determine
the supervision signal provided for the target word $y_{t}$. The following
sections detail how to compute $\varepsilon$ and $\bm{v}$.

###  3.3 Target Word Probability

We control the probability of the target word $y_{t}$ in $\bm{p^{\prime}}$ by
manipulating the adaption factor $\varepsilon$ in Eq. 2. Specifically, for a
training dialogue pair $\langle X,Y\rangle$ and each target word $y_{t}\in Y$,
the current distribution $\bm{p}(\cdot|y_{<t},X)$ is first calculated, and the
maximum probability in this distribution is obtained:

$p_{max}=\max_{w_{k}\in\mathcal{V}}p(w_{k}|y_{<t},X).$ (3)

$\varepsilon$ is then obtained:

$\displaystyle\varepsilon$ $\displaystyle=\textrm{max}(p_{max},\lambda),$ (4)

where $\lambda$ serves as a lower-bound of $\varepsilon$ (i.e.,
$\varepsilon\geq\lambda$).

The basic intuition behind Eq. 4 is to set $\varepsilon=p_{max}$ when
$p_{max}$ is reasonably large. This design prevents our model from receiving
supervisions sharper than $p_{max}$, when the current prediction is confidence
enough.

Further, to ensure that the target word $y_{t}$ always receives the largest
probability in $\bm{q^{\prime}}$, i.e., to ensure
$\varepsilon>(1-\varepsilon)\cdot\textrm{max}(\bm{v})$ (see Eq. 2), in which
$\textrm{max}(\bm{v})$ is the maximum probabilities for non-target words
$\mathcal{V}_{\neq y_{t}}$, we have to enforce
$\varepsilon>\frac{\textrm{max}(\bm{v})}{1+\textrm{max}(\bm{v})}$. Thus we
propose to calculate the lower-bound $\lambda$ of $\varepsilon$ as:

$\displaystyle\lambda=\frac{\textrm{max}(\bm{v})}{1+\textrm{max}(\bm{v})}+\eta,$
(5)

where $\eta>0$ is a hyper-parameter that controls the margin between the
probability of the target word and non-target words in $\bm{p^{\prime}}$.

To facilitate faster converge and better learning of low-probability words, an
empirical factor $\alpha\in[0,1]$ is further introduced to adjust the
calculation of $\varepsilon$ on the basis of Eq. 4:

$\displaystyle\varepsilon=1-\alpha\cdot(1-\textrm{max}(p_{max},\lambda)),$ (6)

where $\alpha$ is calculated as the relative ratio to $p_{max}$:

$\displaystyle\alpha=\left[\frac{p(y_{t}|y_{<t},X)}{p_{max}}\right]^{2},$ (7)

where $p(y_{t}|y_{<t},X)$ is the probability for the target word $y_{t}$. Note
that Eq. 6 and Eq. 4 is equivalent if $\alpha=1$. Intuitively, $\alpha$
accelerates the training of low-frequency words because if $y_{t}$ is of low-
frequency in the corpus, then $y_{t}$ is usually under-trained and thus
$p(y_{t}|y_{<t},X)$ is generally small. This leads to a small $\alpha$ and
thus increases the probability for $y_{t}$ in $\bm{p^{\prime}}$.

Note that $\varepsilon$, $\lambda$ and $\alpha$ are all time-step specific
variables, whereas $\eta$ is a fixed hyper-parameter. This allows the values
adapt to dynamic contexts. In our experiments, Eq. 6 is used to calculate
$\varepsilon$.

Figure 3: (a) The auxiliary decoder $D_{a}$; (b) The target-masked attention
scheme used to compute the auxiliary distribution $\bm{v}$ for the target word
$y_{3}$, specifically, $y_{2}$ is used as the query and $y_{3}$ is masked; (c)
The attention pattern used in the target-masked attention scheme, white dots
represent masked positions.

###  3.4 Non-target Words Probabilities

The auxiliary distribution $\bm{v}$ in Eq. 2 is calculated using an auxiliary
decoder $D_{a}$, which is a single-layer transformer-based decoder that is
jointly optimized with the generation model. Figure 3 shows the structure of
$D_{a}$, in which a novel target-masked attention scheme is devised to mask
each target word $y_{t}$ in the self attention module of the decoder when
calculating the corresponding $\bm{v}$ (see Figure 3b and 3c). In this way,
bi-directional contexts can be utilized when predicting the auxiliary
distribution $\bm{v}$ for $y_{t}$. Moreover, it is important to use only one
decoder layer in $D_{a}$ because stacking multiple layers in $D_{a}$ leaks the
information of $y_{t}$ to $\bm{v}$.

Note that using one layer in $D_{a}$ does not necessarily downgrade its
performance Kasai et al. (2021). Our experiment results in Section 5.1
indicate that with the help of bi-directional contexts, the accuracy of
$D_{a}$ largely outperforms the uni-directional dialogue decoder that is much
deeper than $D_{a}$. Moreover, for a training response $Y$, the structure of
$D_{a}$ enables us infer the auxiliary distribution in parallel for all the
target words in $Y$ within a single forward pass. This differs from the BERT
teacher used by Chen et al. (2020), in which multiple independent forward
passes are needed to get the teacher distributions for all the words in $Y$.

When training $D_{a}$, the following standard MLE loss is optimized for each
target word $y_{t}$:

$\mathcal{L}(\bm{q},\bm{v})=-\sum_{k=1}^{|\mathcal{V}|}q_{k}{\rm log}{v}_{k},$
(8)

in which the notation of $q_{k}$ follows Eq. 1.

The outputs of $D_{a}$ are used as the logits to infer $\bm{v}$ to be further
used in Eq. 2. Specifically, the logit of the target word $y_{t}$ is masked to
$-\infty$ before Softmax to ensure $y_{t}$ always receives zero probability in
$\bm{v}$. Moreover, we also follow the approach used by Tang et al. (2020) to
truncate the head and tail of the remaining logits before inferring $\bm{v}$
in Eq. 2, i.e., all the logits are ranked in a descending order and only the
logits ranked from $n$ to $m$ are kept while the rest logits are masked to
$-\infty$. This masks the head and tail probabilities in $\bm{v}$ to zero. We
argue that truncating the tail probabilities of $\bm{v}$ filters noises, and
truncating the head probabilities of $\bm{v}$ encourages the dialogue model to
focus more on low-probability words. In our experiments, we set $n=2$ and
$m=500$. An extensive hyper-parameter search indicates that our method is not
sensitive to the value of $n$ and $m$.

There are two major differences between our auxiliary decoder $D_{a}$ and the
teacher model used in conventional knowledge distillation approaches: First,
conventional teacher models usually carry more parameters than their students,
whereas $D_{a}$ is rather light-weight. Second, conventional teacher models
are typically pre-trained before being utilized in the distillation process,
whereas $D_{a}$ is trained jointly with our dialogue model.

##  4 Experiments

###  4.1 Dataset

We use two benchmark datasets for open-domain dialogue generation: DailyDialog
Li et al. (2017) is a high-quality multi-turn dialogue dataset that is
collected from daily conversations. OpenSubtitles
111http://opus.nlpl.eu/OpenSubtitles.php contains dialogues collected from
movie subtitles. Moreover, we follow Li et al. (2016) and Jiang et al. (2019)
to focus on short conversations, i.e., dialogues with posts or responses
longer than 100 tokens are removed. See Table 1 for more details.

| Train | Valid | Test
---|---|---|---
DailyDialog | 65.8K | 6.13K | 5.80K
OpenSubtitles | 1.14M | 20.0K | 10.0K
Table 1: Dataset statistics.

###  4.2 Implementation Details

The backbone of our model is the transformer-based sequence to sequence model
Vaswani et al. (2017), and most hyper-parameters follow Cai et al. (2020).
Specifically, the encoder and decoder each contains 6 layers. Each layer has 8
attention heads, and the hidden size is set to 512. The auxiliary decoder
$D_{a}$ follows the same hyper-parameter setting as the dialogue decoder, but
it only contains one layer. The WordPiece tokenizer provided by BERT Devlin et
al. (2019) is used, and the Adam optimizer Kingma and Ba (2015) is employed to
train our model from random initializations with a learning rate of 1e-4.
$\eta$ in Eq. 5 is set to 0.2 for all datasets. See Appendix A for more
details. 222Our code is available at: https://github.com/lemon234071/AdaLabel

Model | DailyDialog | OpenSubtitles
---|---|---
| Dist-1, 2 | Ent-1, 2 | LF | BLEU-2,3,4 | Dist-1, 2 | Ent-1, 2 | LF | BLEU-2,3,4
CE |  1.67 |  9.43 |  4.53 |  6.59 |  2.99 |  7.56 |  4.38 |  2.61 |  2.55 |  9.87 |  4.13 |  5.58 |  0.84 |  7.60 |  4.30 |  2.57
LS |  1.48 |  8.78 |  4.48 |  6.55 |  2.44 |  7.98 |  4.68 |  2.86 |  2.77 |  13.08 |  4.45 |  6.57 |  0.51 |  8.91 |  5.57 |  3.84
FL |  2.38 |  13.42 |  4.7 |  7.04 |  5.05 |  9.74 |  6.12 |  4.11 |  3.19 |  13.16 |  4.42 |  6.50 |  1.04 |  8.06 |  4.79 |  3.08
FACE |  1.62 |  11.04 |  4.96 |  7.27 |  4.11 |  8.78 |  5.06 |  3.06 |  3.31 |  14.06 |  4.77 |  7.05 |  1.33 |  7.69 |  4.40 |  2.70
F2 |  1.40 |  7.91 |  4.35 |  6.28 |  2.32 |  7.78 |  4.45 |  2.60 |  2.89 |  11.40 |  4.24 |  6.14 |  0.99 |  7.52 |  4.30 |  2.62
CP |  2.35 |  12.91 |  4.64 |  6.89 |  4.07 |  9.06 |  5.68 |  3.79 |  3.11 |  12.72 |  4.36 |  6.35 |  0.98 |  8.06 |  4.82 |  3.12
UL |  2.35 |  12.99 |  4.68 |  6.98 |  4.96 |  10.83 |  6.87 |  4.61 |  2.84 |  11.64 |  4.31 |  6.32 |  0.76 |  7.73 |  4.59 |  2.96
NL |  1.66 |  9.18 |  4.47 |  6.58 |  4.30 |  9.83 |  5.83 |  3.60 |  3.24 |  12.98 |  4.42 |  6.49 |  1.08 |  7.56 |  4.38 |  2.71
D2GPo |  1.26 |  8.06 |  4.43 |  6.48 |  2.20 |  8.30 |  4.82 |  2.93 |  2.07 |  11.01 |  4.32 |  6.36 |  0.19 |  8.41 |  5.08 |  3.35
AdaLabel |  3.96 |  23.53 |  5.17 |  8.00 |  8.49 |  17.42 |  13.38 |  11.01 |  4.78 |  22.88 |  4.96 |  7.66 |  1.47 |  9.80 |  6.48 |  4.75
Human |  6.59 |  37.74 |  5.67 |  8.91 |  13.7 |  N/A |  N/A |  N/A |  8.62 |  43.16 |  5.89 |  9.36 |  4.75 |  N/A |  N/A |  N/A
Table 2: Automatic evaluation results (%). Best results among all the models
are in bold.

###  4.3 Baselines

We compared our method with two groups of baselines that try to tackle the
over-confidence issue.

The first group modifies the training target used to compute the loss
function: 1) LS Szegedy et al. (2016): uses the label smoothing approach to
construct a target distribution by adding the one-hot target and a uniform
distribution; 2) FL Lin et al. (2017): uses the focal loss to down-weigh well-
classified tokens in each time step. 3) FACE Jiang et al. (2019): uses the
frequency-aware cross-entropy loss to balance per-token training losses.
Specifically, relative low losses are assigned to high-frequency words to
explicitly tackle the over-confidence issue. We used the best performing “Pre-
weigh” version in our experiments. 4) F2 Choi et al. (2020): factorizes the
target distribution based on the token frequencies.

The second group of baselines add some penalty term to the standard MLE loss:
5) CP Pereyra et al. (2017): a confidence penalty term is added to regularize
the entropy of the model, so that over-confident predictions are penalized; 6)
UL Welleck et al. (2020): an unlikelihood loss term is added to penalize the
frequently generated words. 7) NL He and Glass (2020): works similarly with
baseline UL except a negative loss term is used instead of the unlikelihood
loss term. 8) D2GPo Li et al. (2019): augments the MLE loss with a data-
dependent gaussian prior objective to assign different losses for different
non-target words.

We also compared to: 9) CE: a vanilla Seq2Seq model trained with the cross-
entropy loss. For fair comparisons, the C-MLM model proposed by Chen et al.
(2020) is not used as our baseline since the BERT teacher in C-MLM requires a
large amount of extra data to pre-train. Nevertheless, AdaLabel still
surpasses C-MLM on various metrics (see Appendix F for more analysis).

All our baselines are adapted from the authors’ official codes with the same
backbone architecture and hyper-parameters as our model (see details in
Appendix B). Following the original setting, a train-and-refine strategy is
used in baseline 3, 6, and 7, i.e., these baselines are refined based on CE.
We follow the setting of Jiang et al. (2019) to use deterministic decoding
scheme (particularly, greedy decoding) for our model and all baselines. Note
that our method can be adapted to other decoding schemes such as beam-search
or top-K sampling. See Appendix C for more detailed analysis.

###  4.4 Automatic Evaluation

Metrics: We first used automatic metrics to evaluate our method: 1) _Distinct_
(Dist) Li et al. (2016) calculates the proportion of unique n-grams (n=1, 2)
in the generated responses, which is widely used to measure the response
diversity. 2) _Entropy_ (Ent) Zhang et al. (2018) evaluates how evenly the
empirical n-gram (n=1, 2) distribution is. Higher sores mean more diverse of
the response. 3) _Low-Frequency Token Ratio_ (LF) Li et al. (2019) further
measures the model diversity by counting the ratio of low-frequency words in
the generated responses. We chose words with a frequency less than 100 in each
corpus as low-frequency words. Over-confident models tend to omit low-
frequency words (i.e., get low LF scores) and yield less diversified
responses. 4) _BLEU_ Papineni et al. (2002) measures n-gram (n=2, 3, 4)
overlap between the generated responses and references.

Results: As shown in Table 2, our method AdaLabel outperforms all the
baselines by large margins on all the datasets. We can further observe that:
1) AdaLabel achieves the best diversity scores (Dist-1,2, Ent-1,2, and LF).
This indicates that our method yields better training targets that help to
produce more diverse responses; 2). The models that explicitly tackle the
over-confidence issue (i.e., AdaLabel and FACE) generally outperform other
baselines in diversity-related metrics. For example, FACE obtains the second-
best diversity scores (i.e., Dist, Ent, and LF) on the OpenSubtitles dataset.
This verifies our motivation that alleviating the over-confidence issue helps
to produce more diverse responses.

Note that our method also outperforms all the baselines using the stochastic
decoding scheme. Please refer to Appendix C for more details.

###  4.5 Manual Evaluation

Metrics: Pairwise manual evaluations are conducted to further validate our
method. Specifically, for a given dialogue post, our model’s response is
paired with the one from a baseline. Three individual annotators were employed
to rank each response pair from three aspects: 1) _Fluency_ (Flu.): which
response is more fluent; 2) _Coherency_ (Coh.): which response is more
coherent to the context; 3) _Informativeness_ (Info.): which response contains
more informative content. We also asked the annotator to choose an overall
preferred response (Pref.). Ties were allowed.

Results: 200 posts were randomly sampled from each of these two datasets,
respectively, and totally 3.6K response pairs were generated. The inter-rater
annotation agreement was measured using Fleiss’s kappa $\kappa$ Fleiss (1971).
Particularly, the $\kappa$ value on DailyDialog, OpenSubtitles dataset was
0.59 and 0.55, respectively, indicating moderate agreement.

Comparison | DailyDialog | OpenSubtitles
---|---|---
|  Pref. |  Flu. |  Coh. |  Info. |  Pref. |  Flu. |  Coh. |  Info.
AdaLabel vs CE |  17.00‡ |  1.33 |  12.5‡ |  28.33‡ |  6.33 |  1.17 |  7.33† |  13.67‡
AdaLabel vs LS |  2.67 |  0.17 |  3.33 |  24.83‡ |  5.3 |  -0.67 |  3.17 |  8.50‡
AdaLabel vs FL |  4.50 |  1.67 |  7.00† |  22.0‡ |  8.00† |  1.00 |  6.00 |  5.50
AdaLabel vs FACE |  6.67† |  3.50† |  7.17† |  8.50† |  4.50 |  0.50 |  1.83 |  2.50
AdaLabel vs F2 |  7.67† |  0.33 |  6.83† |  8.67‡ |  4.33 |  -0.50 |  1.67 |  9.50‡
AdaLabel vs CP |  10.50‡ |  -0.17 |  8.00† |  23.83‡ |  8.00† |  1.50 |  6.17 |  16.83‡
AdaLabel vs UL |  7.83† |  0.83 |  6.67† |  17.33‡ |  6.83† |  2.00 |  5.83 |  15.00‡
AdaLabel vs NL |  9.17† |  2.67† |  9.17† |  7.67† |  5.17 |  0.17 |  2.17 |  15.5‡
AdaLabel vs D2GPo |  0.83 |  0.00 |  3.33 |  15.17‡ |  3.17 |  7.33‡ |  1.00 |  6.33†
Table 3: Pairwise human evaluation results (%). The absolute gains of AdaLabel
(i.e., _Win rate_ $-$ _Lose rate_) are reported. †, ‡ indicates significant
improvement with _p_ -value $<0.05$ and $<0.005$, respectively (sign test).

As shown in Table 3, AdaLabel outperforms all the baselines on the
informativeness measure. This means that our method can respond with more
informative content. We can further observe that:

1). All models achieve competitive fluency because it is easy for neural
models to produce fluent responses by yielding trivial responses like “I don’t
know”. However, our model surpasses most baselines in terms of fluency while
ensuring high diversity scores. This demonstrates the superiority of our
method in producing high quality responses.

2). AdaLabel produces more coherent responses comparing to most baselines.
This verifies that our model does not sacrifice the response quality when
achieving high diversity scores. In fact, by controlling the model’s
confidence, more low-frequency words are encouraged, and thus AdaLabel can
produce more relevant and coherent responses. This claim is further verified
by observing that our model achieves the best overall preference score among
all the baselines.

###  4.6 Ablation study

Model | BLEU-3,4 | Dist-1,2 | Ent-1,2 |  LF
---|---|---|---|---
1.w/o $\varepsilon$ |  5.46 |  3.57 |  2.52 |  13.21 |  4.64 |  6.89 |  4.85
2.w/o $\alpha$ |  11.35 |  8.70 |  3.62 |  20.56 |  5.02 |  7.70 |  7.30
3.Orig. $\bm{v}$ |  8.15 |  5.77 |  3.71 |  19.53 |  5.00 |  7.58 |  8.25
4.Uniform |  5.66 |  3.61 |  2.24 |  14.96 |  4.84 |  7.33 |  4.98
5.Rand |  6.27 |  4.07 |  2.03 |  13.47 |  4.7 |  7.08 |  4.56
6.BERT |  11.6 |  9.34 |  3.67 |  20.97 |  5.02 |  7.71 |  7.28
AdaLabel |  13.38 |  11.01 |  3.96 |  23.53 |  5.17 |  8.00 |  8.49
Table 4:  Ablation study results on DailyDialog (%).

Ablation studies were performed to verify the effect of each component in our
method. Specifically, two groups of variants were tested:

The first group validates the effectiveness of the calculated target word
probability, i.e., $\varepsilon$: 1). w/o $\bm{\varepsilon}$ directly sets a
fixed value for $\varepsilon$ in Eq. 2. The specific value of $\varepsilon$ is
searched from 0.1 to 0.7 with a stride of 0.1; 2). w/o $\bm{\alpha}$ omits the
empirical factor $\alpha$ in calculating $\varepsilon$, i.e., the value of
$\varepsilon$ in Eq. 2 is calculated using Eq. 4 in instead of Eq. 6.

The second group validates the effectiveness of the non-target word
probabilities produced by $D_{a}$, i.e., $\bm{v}$: 3). Orig. $\bm{v}$ does not
truncate the head of $\bm{v}$ when inferring from $D_{a}$. Note that the
truncation for the tail of $\bm{v}$ is still applied since its effectiveness
has already been proved in previous studies Tang et al. (2020); Tan et al.
(2019); 4). Uniform uses an uniform distribution as $\bm{v}$ in Eq. 2. Note
that different from the baseline LS, the value of $\varepsilon$ is calculated
using Eq. 6 in this ablation model, whereas the value of $\varepsilon$ in the
baseline LS is fixed ; 5). Rand use a random distributions as $\bm{v}$ in Eq.
2; 6). BERT follows the work of Chen et al. (2020) to fine-tune a pre-trained
BERT model to produce $\bm{v}$. Note that our dialogue model may benefit from
the multi-task training of $D_{a}$ since $D_{a}$ shares the same encoder with
our dialogue model. Optimizing Eq. 8 may help the encoder to capture better
features. For fair comparison, we kept the task of optimizing $D_{a}$ in
ablation models 4-6 although it is not used to infer $\bm{v}$.

Table 4 shows the results of ablation models on the DailyDialog dataset. As
can be seen from the first two rows, our method to adaptively calculate
$\varepsilon$ helps to improve the performance of our model by a large margin,
and the empirical adjustment factor $\alpha$ helps to further improve our
performance by facilitating the learning of low-probability words. The
performance of ablation models 3-6 in Table 4 proves that $\bm{v}$ captures
reliable distribution and helps our model produce more diverse responses.
Moreover, truncating the head distribution of $\bm{v}$ enables the dialogue
model to focus more on the low-frequency words and thus facilitates more
informative responses.

It is also interesting to note that our auxiliary decoder $D_{a}$ surpasses
the BERT teacher used by Chen et al. (2020) in helping the dialogue model to
produce more diverse responses. This further proves the effectiveness of
$D_{a}$ considering that BERT contains 6 times parameters than $D_{a}$ and
consumes much more computation resources.

| DailyDialog | OpenSubtitles
---|---|---
Auxiliary Decoder $D_{a}$ | 64.03 | 64.92
Dialog Decoder in AdaLabel | 44.16 | 43.90
Dialog Decoder in CE | 38.58 | 41.57
Table 5: Prediction accuracy of decoders on test sets. Figure 4: Empirical
distribution of confidence scores for high-frequency words on the
OpenSubtitles dataset. Words occupying the top 40% of the frequency mass in
the training set are regarded as high-frequency words.

##  5 Discussion

###  5.1 Auxiliary Decoder

To further test the performance of $D_{a}$, we evaluated the averaged accuracy
score of $D_{a}$ when predicting each target word in the test set (first row
in Table 5). Specifically, a target word $y_{t}$ in the reference response is
determined to be correctly predicted if it is top-ranked in the predicted
distribution $p(\cdot|y_{<t},X)$. A better decoder is generally believed to
obtain a higher accuracy. Table 5 also reports the uni-directional dialogue
decoders’ accuracy in AdaLabel and CE. It can be seen that $D_{a}$ can make
substantially more accurate predictions with the help of modeling bi-
directional contexts using only one layer. Moreover, the dialogue model’s
decoder in AdaLabel, which is guided by $D_{a}$, achieves better accuracies
than the CE. This further proves that our light-weight $D_{a}$ is capable of
producing effective $\bm{v}$.

###  5.2 Prediction Confidence

We also visualized the distribution of confidence scores assigned by each
dialogue model to high-frequency words. Figure 4 shows the results of four
best performing models on the OpenSubtitles dataset. The spikes of high
confidence score observed in Figure 4b and 4d indicate that CE and FACE assign
extremely high confidence scores to a large number of high-frequency words.
Although the smoothed labels in LS manage to alleviate these high-confidence-
spikes (Figure 4c), a considerable amount of words still receives high
confidence scores in LS. Our model outperforms all the baselines to avoid
assigning over-confidence scores, thus alleviating the over-confidence issue.
A similar trend is also observed on the DailyDialog dataset (see Appendix D
for results of all models on both datasets).

###  5.3 Predicted Rare Word Distribution

Over-confident models produce less diversified responses because they usually
under-estimate rare words. To evaluate the effectiveness of AdaLabel, we
tested whether AdaLabel encourages more “rare words” in its generations.
Specifically, the ratio of generated tokens corresponding to different token
frequency bins is calculated, and the results on the OpenSubtitles dataset are
shown in Figure 5. It can be seen that AdaLabel produces more rare words in
the generated responses than other baselines. Similar results are also
observed on the DailyDialog dataset (see Appendix E).

Figure 5: Ratios of low-frequency tokens in the generated responses on the
OpenSubtitles dataset. Tokens in each group are determined based on the
frequency on the training set.

##  6 Conclusion

We address the low-diversity issue of neural dialogue models by introducing an
adaptive label smoothing approach, AdaLabel. In our method, the probability of
each target word is estimated based on the current dialogue model’s
prediction, and the probabilities for these non-target words are calculated
using a novel auxiliary decoder $D_{a}$. A target-masked attention scheme is
introduced in $D_{a}$ to help capture forward and backward contexts. We
evaluate our method on two benchmark datasets: DailyDialog and OpenSubtitles.
Extensive experiments show that our method effectively alleviates the over-
confidence issue and improves the diversity of the generated responses. As
future work, we believe this method is extensible to other text generation
tasks.

## Acknowledgments

This work was partly supported by the NSFC projects (Key project with No.
61936010 and regular project with No. 61876096). This work was also supported
by the Guoqiang Institute of Tsinghua University, with Grant No. 2019GQG1 and
2020GQG0005. We thank Jinchao Zhang and Yao Qiu for early discussions and
insightful comments of this work.

## References

  * Cai et al. (2020) Hengyi Cai, Hongshen Chen, Yonghao Song, Cheng Zhang, Xiaofang Zhao, and Dawei Yin. 2020.  Data manipulation: Towards effective instance learning for neural dialogue generation via learning to augment and reweight.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 6334–6343, Online. Association for Computational Linguistics.
  * Chen et al. (2020) Yen-Chun Chen, Zhe Gan, Yu Cheng, Jingzhou Liu, and Jingjing Liu. 2020.  Distilling knowledge learned in BERT for text generation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7893–7905, Online. Association for Computational Linguistics.
  * Choi et al. (2020) Byung-Ju Choi, Jimin Hong, David Park, and Sang Wan Lee. 2020.  F^2-softmax: Diversifying neural text generation via frequency factorized softmax.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 9167–9182, Online. Association for Computational Linguistics.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Du et al. (2018) Jiachen Du, Wenjie Li, Yulan He, Ruifeng Xu, Lidong Bing, and Xuan Wang. 2018.  Variational autoregressive decoder for neural response generation.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 3154–3163, Brussels, Belgium. Association for Computational Linguistics.
  * Fleiss (1971) Joseph L Fleiss. 1971.  Measuring nominal scale agreement among many raters.  _Psychological bulletin_ , 76(5):378.
  * Gao et al. (2019) Xiang Gao, Sungjin Lee, Yizhe Zhang, Chris Brockett, Michel Galley, Jianfeng Gao, and Bill Dolan. 2019.  Jointly optimizing diversity and relevance in neural response generation.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 1229–1238, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Gowda and May (2020) Thamme Gowda and Jonathan May. 2020.  Finding the optimal vocabulary size for neural machine translation.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 3955–3964, Online. Association for Computational Linguistics.
  * Guo et al. (2017) Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. 2017.  On calibration of modern neural networks.  In _International Conference on Machine Learning_ , pages 1321–1330. PMLR.
  * He and Glass (2020) Tianxing He and James Glass. 2020.  Negative training for neural dialogue response generation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 2044–2058, Online. Association for Computational Linguistics.
  * Hinton et al. (2015) Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. 2015.  Distilling the knowledge in a neural network.  _CoRR_ , abs/1503.02531.
  * Holtzman et al. (2020) Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020.  The curious case of neural text degeneration.  In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net.
  * Huang et al. (2020) Minlie Huang, Xiaoyan Zhu, and Jianfeng Gao. 2020.  Challenges in building intelligent open-domain dialog systems.  _ACM Transactions on Information Systems (TOIS)_ , 38(3):1–32.
  * Ippolito et al. (2019) Daphne Ippolito, Reno Kriz, João Sedoc, Maria Kustikova, and Chris Callison-Burch. 2019.  Comparison of diverse decoding methods from conditional language models.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 3752–3762, Florence, Italy. Association for Computational Linguistics.
  * Jiang et al. (2019) Shaojie Jiang, Pengjie Ren, Christof Monz, and Maarten de Rijke. 2019.  Improving neural response diversity with frequency-aware cross-entropy loss.  In _The World Wide Web Conference_ , pages 2879–2885.
  * Jiang and de Rijke (2018) Shaojie Jiang and Maarten de Rijke. 2018.  Why are sequence-to-sequence models so dull? understanding the low-diversity problem of chatbots.  In _Proceedings of the 2018 EMNLP Workshop SCAI: The 2nd International Workshop on Search-Oriented Conversational AI_ , pages 81–86, Brussels, Belgium. Association for Computational Linguistics.
  * Kasai et al. (2021) Jungo Kasai, Nikolaos Pappas, Hao Peng, James Cross, and Noah Smith. 2021.  Deep encoder, shallow decoder: Reevaluating non-autoregressive machine translation.  In _International Conference on Learning Representations_.
  * Kim and Rush (2016) Yoon Kim and Alexander M. Rush. 2016.  Sequence-level knowledge distillation.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_ , pages 1317–1327, Austin, Texas. Association for Computational Linguistics.
  * Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015.  Adam: A method for stochastic optimization.  In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_.
  * Klein et al. (2017) Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander Rush. 2017\.  OpenNMT: Open-source toolkit for neural machine translation.  In _Proceedings of ACL 2017, System Demonstrations_ , pages 67–72, Vancouver, Canada. Association for Computational Linguistics.
  * Kulikov et al. (2019) Ilia Kulikov, Alexander Miller, Kyunghyun Cho, and Jason Weston. 2019.  Importance of search and evaluation strategies in neural dialogue modeling.  In _Proceedings of the 12th International Conference on Natural Language Generation_ , pages 76–87, Tokyo, Japan. Association for Computational Linguistics.
  * Kumar and Sarawagi (2019) Aviral Kumar and Sunita Sarawagi. 2019.  Calibration of encoder decoder models for neural machine translation.  _CoRR_ , abs/1903.00802.
  * Li et al. (2016) Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016.  A diversity-promoting objective function for neural conversation models.  In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 110–119, San Diego, California. Association for Computational Linguistics.
  * Li et al. (2020) Margaret Li, Stephen Roller, Ilia Kulikov, Sean Welleck, Y-Lan Boureau, Kyunghyun Cho, and Jason Weston. 2020.  Don’t say that! making inconsistent dialogue unlikely with unlikelihood training.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 4715–4728, Online. Association for Computational Linguistics.
  * Li et al. (2017) Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017.  DailyDialog: A manually labelled multi-turn dialogue dataset.  In _Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 986–995, Taipei, Taiwan. Asian Federation of Natural Language Processing.
  * Li et al. (2019) Zuchao Li, Rui Wang, Kehai Chen, Masso Utiyama, Eiichiro Sumita, Zhuosheng Zhang, and Hai Zhao. 2019.  Data-dependent gaussian prior objective for language generation.  In _International Conference on Learning Representations_.
  * Lin et al. (2017) Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017\.  Focal loss for dense object detection.  In _Proceedings of the IEEE international conference on computer vision_ , pages 2980–2988.
  * Massarelli et al. (2020) Luca Massarelli, Fabio Petroni, Aleksandra Piktus, Myle Ott, Tim Rocktäschel, Vassilis Plachouras, Fabrizio Silvestri, and Sebastian Riedel. 2020.  How decoding strategies affect the verifiability of generated text.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 223–235, Online. Association for Computational Linguistics.
  * Mukhoti et al. (2020) Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal, Stuart Golodetz, Philip H. S. Torr, and Puneet K. Dokania. 2020.  Calibrating deep neural networks using focal loss.  In _Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual_.
  * Müller et al. (2019) Rafael Müller, Simon Kornblith, and Geoffrey E Hinton. 2019.  When does label smoothing help?  In _Advances in Neural Information Processing Systems_ , pages 4694–4703.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002.  Bleu: a method for automatic evaluation of machine translation.  In _Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics_ , pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
  * Pereyra et al. (2017) Gabriel Pereyra, George Tucker, Jan Chorowski, Lukasz Kaiser, and Geoffrey E. Hinton. 2017.  Regularizing neural networks by penalizing confident output distributions.  In _5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Workshop Track Proceedings_. OpenReview.net.
  * Szegedy et al. (2016) Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016.  Rethinking the inception architecture for computer vision.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 2818–2826.
  * Tan et al. (2019) Xu Tan, Yi Ren, Di He, Tao Qin, Zhou Zhao, and Tie-Yan Liu. 2019.  Multilingual neural machine translation with knowledge distillation.  In _7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019_. OpenReview.net.
  * Tang et al. (2020) Jiaxi Tang, Rakesh Shivanna, Zhe Zhao, Dong Lin, Anima Singh, Ed H. Chi, and Sagar Jain. 2020.  Understanding and improving knowledge distillation.  _CoRR_ , abs/2002.03532.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA_ , pages 5998–6008.
  * Wang et al. (2020) Yida Wang, Pei Ke, Yinhe Zheng, Kaili Huang, Yong Jiang, Xiaoyan Zhu, and Minlie Huang. 2020.  A large-scale chinese short-text conversation dataset.  In _CCF International Conference on Natural Language Processing and Chinese Computing_ , pages 91–103. Springer.
  * Welleck et al. (2020) Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. 2020.  Neural text generation with unlikelihood training.  In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net.
  * Yang et al. (2018) Chenglin Yang, Lingxi Xie, Siyuan Qiao, and Alan L. Yuille. 2018.  Knowledge distillation in generations: More tolerant teachers educate better students.  _CoRR_ , abs/1805.05551.
  * Zhang et al. (2020) Rongsheng Zhang, Yinhe Zheng, Jianzhi Shao, Xiaoxi Mao, Yadong Xi, and Minlie Huang. 2020.  Dialogue distillation: Open-domain dialogue augmentation using unpaired data.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 3449–3460.
  * Zhang et al. (2018) Yizhe Zhang, Michel Galley, Jianfeng Gao, Zhe Gan, Xiujun Li, Chris Brockett, and Bill Dolan. 2018.  Generating informative and diverse conversational responses via adversarial information maximization.  In _Advances in Neural Information Processing Systems_ , pages 1810–1820.
  * Zhao et al. (2017) Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. 2017.  Learning discourse-level diversity for neural dialog models using conditional variational autoencoders.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 654–664, Vancouver, Canada. Association for Computational Linguistics.
  * Zheng et al. (2020a) Yinhe Zheng, Zikai Chen, Rongsheng Zhang, Shilei Huang, Xiaoxi Mao, and Minlie Huang. 2020a.  Stylized dialogue response generation using stylized unpaired texts.  In _AAAI_.
  * Zheng et al. (2020b) Yinhe Zheng, Rongsheng Zhang, Minlie Huang, and Xiaoxi Mao. 2020b.  A pre-training based personalized dialogue generation model with persona-sparse data.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 34, pages 9693–9700.
  * Zhou et al. (2018) Ganbin Zhou, Ping Luo, Yijun Xiao, Fen Lin, Bo Chen, and Qing He. 2018.  Elastic responding machine for dialog generation with dynamically mechanism selecting.  In _Thirty-Second AAAI Conference on Artificial Intelligence_.

Model | DailyDialog | OpenSubtitles
---|---|---
| Dist-1, 2 | Ent-1, 2 | LF | BLEU-2,3,4 | Dist-1, 2 | Ent-1, 2 | LF | BLEU-2,3,4
CE |  1.79 |  8.21 |  4.19 |  5.90 |  2.57 |  4.06 |  2.49 |  1.58 |  2.48 |  9.21 |  4.07 |  5.74 |  0.76 |  7.03 |  4.26 |  2.82
LS |  1.71 |  8.01 |  4.16 |  5.89 |  2.17 |  4.13 |  2.55 |  1.65 |  2.89 |  12.79 |  4.27 |  6.24 |  0.47 |  8.24 |  5.57 |  4.20
FL |  2.40 |  11.37 |  4.39 |  6.35 |  4.46 |  6.01 |  3.95 |  2.75 |  3.10 |  12.37 |  4.25 |  6.13 |  0.82 |  7.13 |  4.56 |  3.25
FACE |  1.80 |  9.47 |  4.54 |  6.40 |  3.48 |  5.65 |  3.43 |  2.17 |  3.12 |  12.62 |  4.47 |  6.40 |  1.02 |  5.97 |  3.63 |  2.43
F2 |  1.61 |  7.22 |  4.04 |  5.70 |  2.11 |  4.32 |  2.55 |  1.52 |  2.89 |  10.63 |  4.03 |  5.72 |  0.89 |  6.92 |  4.27 |  2.91
CP |  2.30 |  10.39 |  4.28 |  6.16 |  3.25 |  5.31 |  3.39 |  2.30 |  3.14 |  11.87 |  4.17 |  5.97 |  0.85 |  7.28 |  4.60 |  3.21
UL |  2.42 |  11.0 |  4.40 |  6.42 |  4.55 |  7.94 |  5.26 |  3.69 |  2.77 |  10.43 |  3.98 |  5.62 |  0.62 |  6.89 |  4.36 |  3.03
NL |  1.61 |  7.53 |  4.19 |  6.05 |  4.02 |  7.09 |  4.41 |  2.91 |  2.65 |  10.14 |  4.21 |  6.05 |  0.75 |  7.16 |  4.32 |  2.85
D2GPo |  1.57 |  7.83 |  4.14 |  5.91 |  2.26 |  4.47 |  2.71 |  1.71 |  2.06 |  10.43 |  4.15 |  6.00 |  0.12 |  7.32 |  4.69 |  3.33
AdaLabel |  4.25 |  21.47 |  4.95 |  7.51 |  7.68 |  14.71 |  11.63 |  9.80 |  4.91 |  21.53 |  4.71 |  7.08 |  1.35 |  8.68 |  6.08 |  4.68
AdaLabel (Greedy) |  3.96 |  23.53 |  5.17 |  8.00 |  8.49 |  17.42 |  13.38 |  11.01 |  4.78 |  22.88 |  4.96 |  7.66 |  1.47 |  9.80 |  6.48 |  4.75
Human |  6.59 |  37.74 |  5.67 |  8.91 |  13.7 |  N/A |  N/A |  N/A |  8.62 |  43.16 |  5.89 |  9.36 |  4.75 |  N/A |  N/A |  N/A
Table 6:  Automatic evaluation results (%) using the beam search decoding
scheme (beam size is 5). The best results among all these beam-search-decoded
models are in bold.

##  Appendix A Implementation Details

This appendix describes the implementation details of our model. All our
experiments are implemented with python 3.7.4, PyTorch 1.7.1, and the OpenNMT
package Klein et al. (2017). Training is performed on one TITAN Xp GPU. Our
model’s backbone is the transformer-based sequence to sequence model, the
encoder and decoder each contains 6 transformer layers with 8 attention heads,
and the hidden size is set to 512. The dimension of the feed-forward layer is
also 512. The WordPiece tokenizer provided by BERT-base-uncased is used (the
vocabulary contains 30522 tokens). The total number of parameters in our model
is about 90M. The Adam optimizer is employed to train our model from random
initializations with $\beta_{1}=0.9$, $\beta_{2}=0.999$, $\epsilon=1e-9$ and a
learning rate of 1e-4. The batch size is set to 64 with 2 gradient
accumulation so that 2 * 64 samples are used for each parameter update. The
model is evaluated every 1000 steps on the validation set. We use early-
stopping with patience 10, 30 for DailyDialog and OpenSubtitles, respectively.
Specifically, the model stops training when the evaluation perplexity and
accuracy are not increased for “patience” steps. The model training takes 4
hours and 3 days on DailyDialog and OpenSubtitles, respectively.

The auxiliary distribution produced by the auxiliary decoder is smoothed with
the temperature scaling approach. The temperature used in this process is
searched in [1, 1.5, 2]. The temperature value of 1.5 and 1.0 is used for
DailyDialog, and OpenSubtitles, respectively. The hyper-parameter value of
$\eta$ is set to 0.2 for all datasets. The fixed value of epsilon in our
ablation model w/o $\bm{\epsilon}$ is searched in [0.1, 0.2, 0.3, 0.4, 0.5,
0.6], and we find the value of 0.1 works best.

##  Appendix B Baseline Implementation Details

This appendix contains more implementation details of our baselines. All the
baselines utilize the same backbone architecture and basic hyper-parameter
settings as our model (see Appendix A). The hyper-parameters specialized for
each baseline is determined with the grid search based on the Dist measures on
the validation set: For Label smoothing (LS), we searched the smoothing
parameter in [0.05, 0.1, 0.2, 0.3, 0.4, 0.5], and found 0.1 works best on all
the datasets; For Confidence penalty (CP), we searched the weight of penalty
in [0.0005, 0.001, 0.01, 0.05, 0.1] and found 0.05 works best on all the
datasets while ensuring the loss to be positive; For Focal loss (FL), we
searched the hyper-parameter $\gamma$ in [0.1, 0.5, 1, 2, 3], and found 2
works best on all the datasets. For Unlikelihood loss (UL), we searched the
weight of penalty in [1, 10, 100, 1000], and select 1000 on all the datasets.
For FACE, we experiment with the Output token frequency & PRe-weigh version,
which is reported to be the best version of FACE. For Negative loss (NL),
F2-softmax (F2) and Data-dependent Gaussian Prior objective (D2GPo), the
selection of hyper-parameters follows the author’s suggestion.

##  Appendix C Automatic Evaluation Results with Other Decoding Schemes

Model | DailyDialog | OpenSubtitles
---|---|---
| Dist-1, 2 | Ent-1, 2 | LF | BLEU-2,3,4 | Dist-1, 2 | Ent-1, 2 | LF | BLEU-2,3,4
CE |  2.22 |  19.05 |  5.07 |  7.87 |  4.09 |  6.78 |  3.29 |  1.61 |  3.78 |  20.58 |  5.07 |  7.97 |  1.23 |  5.94 |  2.84 |  1.46
LS |  1.95 |  17.74 |  5.02 |  7.82 |  3.69 |  7.08 |  3.50 |  1.77 |  3.46 |  21.27 |  5.10 |  8.12 |  0.78 |  6.15 |  3.16 |  1.85
FL |  2.71 |  20.98 |  5.19 |  8.17 |  6.44 |  8.09 |  4.13 |  2.24 |  3.82 |  22.14 |  5.15 |  8.25 |  1.27 |  5.34 |  2.54 |  1.34
FACE |  2.29 |  21.14 |  5.36 |  8.3 |  5.73 |  7.07 |  3.47 |  1.82 |  4.25 |  23.95 |  5.30 |  8.37 |  1.51 |  5.34 |  2.54 |  1.33
F2 |  2.16 |  19.33 |  5.04 |  7.85 |  3.97 |  6.31 |  3.12 |  1.58 |  4.10 |  22.53 |  5.13 |  8.11 |  1.32 |  5.27 |  2.51 |  1.31
CP |  3.16 |  22.38 |  5.11 |  7.96 |  6.01 |  8.11 |  4.38 |  2.50 |  4.06 |  22.62 |  5.13 |  8.14 |  1.33 |  6.00 |  2.94 |  1.52
UL |  2.92 |  20.81 |  5.12 |  7.99 |  6.44 |  9.36 |  5.13 |  3.00 |  3.74 |  20.97 |  5.01 |  7.94 |  1.00 |  6.01 |  2.99 |  1.64
NL |  2.39 |  18.35 |  4.99 |  7.79 |  5.72 |  8.71 |  4.64 |  2.63 |  3.57 |  20.36 |  5.05 |  7.97 |  1.06 |  5.84 |  2.86 |  1.46
D2GPo |  1.75 |  17.09 |  5.00 |  7.81 |  3.40 |  7.45 |  3.73 |  1.97 |  2.74 |  19.21 |  5.00 |  7.97 |  0.36 |  6.32 |  3.15 |  1.72
AdaLabel |  4.11 |  32.65 |  5.58 |  8.93 |  10.99 |  8.87 |  4.84 |  2.90 |  4.78 |  29.58 |  5.43 |  8.78 |  1.53 |  5.12 |  2.32 |  1.19
AdaLabel (Greedy) |  3.96 |  23.53 |  5.17 |  8.00 |  8.49 |  17.42 |  13.38 |  11.01 |  4.78 |  22.88 |  4.96 |  7.66 |  1.47 |  9.80 |  6.48 |  4.75
Human |  6.59 |  37.74 |  5.67 |  8.91 |  13.7 |  N/A |  N/A |  N/A |  8.62 |  43.16 |  5.89 |  9.36 |  4.75 |  N/A |  N/A |  N/A
Table 7:  Automatic evaluation results (%) using the top-k sampling decoding
scheme ($k=10$). The best results among all these top-k-decoded models are in
bold.

This appendix reports our model’s automatic evaluation results and all the
baselines when different decoding schemes are used. Specifically, Table 6
shows the results for the beam search decoding scheme (beam size of 5), and
Table 7 shows the results when the top-K decoding scheme ($k=10$) is used.
Note that for the F2-softmax, we use the decoupled top-k sampling as the
authors suggested.

As can be seen from Table 6 and 7, our method outperforms all the baselines on
the diversity-related scores (i.e., Dist, Ent, and LF) by a large margin. This
indicates that our method can produce more diverse responses even with the
stochastic based decoding scheme.

We also include the results of AdaLabel when the greedy decoding scheme is
used in Table 6 and Table 7 (the second line from the bottom). It is
interesting to see that the greedily decoded responses from AdaLabel are more
diverse than some baselines that are decoded using the sampling scheme (see
Table 7). Moreover, our model AdaLabel with the greedy decoding scheme
achieves the best BLEU among all the baselines on both datasets.

##  Appendix D Prediction Confidence

This appendix reports the prediction confidence scores assigned by each model
to high-frequency words. Specifically, words occupying the top 40% of the
frequency mass in the training set of each dataset are regarded as high-
frequency words.

Figure 6 shows the results of our model and all the baselines on the
DailyDialog dataset. Figure 7 shows the results of our model and all the
baselines on the OpenSubtitles dataset. It can be seen that most of our
baselines assign extremely high confidence scores (nearly 1.0) to these high-
frequency words, and thus resulting in a spike of high confidence scores in
the plotted distribution. Our model outperforms all the baselines in avoiding
assigning extremely high confidence scores to these high-frequency words.

Figure 6: Confidence score distributions for high-frequency words on the
DailyDialog dataset. Words occupying the top 40% of the frequency mass in the
training set of DailyDialog are regarded as high-frequency words. Figure 7:
Confidence score distributions for high-frequency words on the OpenSubtitles
dataset. Words occupying the top 40% of the frequency mass in the training set
of OpenSubtitles are regarded as high-frequency words.

##  Appendix E Predicted Rare Word Distribution on DailyDialog

This appendix shows the distribution of rare words in the generated responses
on the DailyDialog dataset (see Figure 8). It can be seen that more “rare
words” are predicted by our method on the DailyDialog dataset. This
observation is in line with the results on the OpenSubtitles dataset as
reported in Section 5.3.

Figure 8: Ratios of low-frequency tokens in the generated responses on the
DailyDialog dataset. Tokens in each group are determined based on the
frequency on the training set.

##  Appendix F Use BERT Model to Obtain $\bm{v}$

Model | BLEU-3,4 | Dist-1,2 | Ent-1,2 |  LF
---|---|---|---|---
1\. CMLM |  6.18 |  4.09 |  2.20 |  11.83 |  4.59 |  6.79 |  4.62
2\. CMLM+$\varepsilon$ |  9.36 |  7.31 |  3.78 |  21.05 |  4.96 |  7.61 |  6.88
3\. CMLM+$\varepsilon$+$D_{a}$ |  11.6 |  9.34 |  3.67 |  20.97 |  5.02 |  7.71 |  7.28
AdaLabel |  13.38 |  11.01 |  3.96 |  23.53 |  5.17 |  8.00 |  8.49
Table 8:  Ablation study results based on BERT on DailyDialog (%).

This appendix provides more experiment results comparing to the CMLM model
Chen et al. (2020): 1). CMLM exactly follows the setting of Chen et al.
(2020), i.e., the teacher distribution produced by the BERT model is merged
with the one-hot distribution using a fixed $\varepsilon$. 2).
CMLM+$\varepsilon$ adaptively adjust the value of $\varepsilon$ using Eq. 6 in
our paper. 3). CMLM+$\varepsilon$+$D_{a}$ add an additional training task to
optimize the auxiliary decoder $D_{a}$ on the basis of CMLM+$\varepsilon$. It
is expected that optimizing $D_{a}$ help our dialogue encoder to capture
better representations. The trained $D_{a}$ is not used in the training and
inference phase of our dialogue model. Note that the last model
CMLM+$\varepsilon$+$D_{a}$ is the same with our ablation model 6\. BERT as
reported in our paper.

As can be seen Table 8, our approach to adaptively change $\varepsilon$ helps
to produce better dialogue responses, and the training of $D_{a}$ helps our
dialogue encoder to learn better representations.

##  Appendix G Case study

We sampled some generated cases on the DailyDialog and OpenSubtitles dataset.
The results of our model and some competitive baselines are shown in Table 9
and Table 10. It can be seen that the responses generated by our method are
coherent to the context and contain richer contents. Moreover, our model also
produces more rare words that make our response more diverse.

| Dialogue
---|---
Context1 | Tell me a little bit about yourself , please .
UL | I am sorry , sir . I have been in a foreign
| currency company for the company .
D2GPo | I ’ m afraid i ca n ’ t . I ’ m a little
| nervous about it .
FACE | Yes , i am in the mood for a few days ago .
AdaLabel | I am studying in the international trade
| company . I am studying english in my company .
Context1 | Oh , they look nice . may i try that black one on ?
Context2 | Of course . the fitting room is right over there .
UL | I think you ’ re right . i think you ’ ll have a try .
D2GPo | I think it ’ s a good idea .
FACE | How much is it ?
AdaLabel | I bought this one . it ’ s the latest fashion .
Context1 | Well , we don ’ t have any pre - existing
| illnesses or conditions , so we should be fine .
UL | And most people are n ’ t so quiet and
| interviewing .
D2GPo | And the company company are a good job in
| the company .
FACE | And your company insurance , or workshops
| are booked in a private company ?
AdaLabel | I wish our company or country provided us
| with healthcare .
Table 9:  Cases on DailyDialog.  | Dialogue
---|---
Context | You ’ re going to love this baby .
LS | I ’ m going to get you .
D2GPo | I ’ m going to go to the bathroom .
FACE | You ’ re going to love this baby .
AdaLabel | You ’ re going to be a good father .
Context | Mr. Matlack can ’ t offend .
LS | This is our official business .
D2GPo | This is the most important thing .
FACE | It ’ s a pleasure to meet you , mr .
| President .
AdaLabel | Timothy Matlack is the official scribe
| of the continental congress .
Context | Brave men discover they are really cowards .
LS | I ’ m a samurai .
D2GPo | I ’ ve never seen such a beast in my life .
FACE | They are not afraid of you .
AdaLabel | Father , you ’ re a coward !
Context | I found this whole operation strange .
LS | I ’ m not going to let you go .
D2GPo | I ’ m not sure i ’ m going to do it .
FACE | I ’ m sorry , but i ’ m not sure i can handle it .
AdaLabel | It ’ s an incredibly delicate system .
Context | If they make it , they ’ re clear into a safe zone
| where they can get medical supplies and food .
LS | We ’ il get them to the safe .
D2GPo | We ’ il have to get back to the hotel .
FACE | They ’ re gon na get us out of here .
AdaLabel | So we can use it as a safe field .
Table 10:  Cases on OpenSubtitles.


# Absence of nematic instability in LiFeAs

Michael Wissmann1,2 Federico Caglieris1,3,4 Xiaochen Hong1,5 Saicharan
Aswartham1 Anna Vorobyova6 Igor Morozov6 Bernd Büchner1,2 Christian Hess1,5,
<EMAIL_ADDRESS>1Leibniz Institute for Solid State and Materials
Research IFW Dresden, 01069 Dresden, Germany  2Institute of Solid State
Physics, TU Dresden, 01069 Dresden, Germany  3University of Genova, via
Dodecaneso 33, I-16146 Genova, Italy  4CNR-SPIN Genova, via Dodecaneso 33,
I-16146 Genova, Italy  5Fakultät für Mathematik und Naturwissenschaften,
Bergische Universität Wuppertal, 42097 Wuppertal, Germany  6Lomonosov Moscow
State University, Moscow 119991, Russia

###### Abstract

The relationship between unconventional superconductivity, antiferromagnetism
and nematic order in iron-based superconductors (FeSCs) is still highly
debated. In many FeSCs superconductivity is in proximity of a nematically and
magnetically ordered state. LiFeAs is an exceptional stoichiometric FeSC
becoming superconducting below 18 K, without undergoing a structural or
magnetic transition. However, some recent experimental studies suggested the
existence of finite nematic fluctuations and even a nematic superconducting
state. In this study, we employ elastoresistance as a measure of nematic
fluctuations in pristine LiFeAs and compare the findings with the
elastoresistance of LiFeAs at low Co and V doping levels as well with that of
magnetically and nematically ordering NaFeAs. We find LiFeAs and cobalt-doped
LiFeAs far away from a nematic instability.

††preprint: APS/123-QED

Electronic nematic order is a state that spontaneously breaks the rotational
symmetry while preserving translational symmetry. It has become a subject of
increasing attention in the context of the rich phase interplay in iron-based
superconductors (FeSCs) [1, 2, 3]. In most FeSCs, nematicity occurs in close
vicinity of an antiferromagnetically ordered state, which was repeatedly
proposed to be intimately connected to the evolution of the unconventional
superconductivity [4, 5, 6]. In order to disentangle the relation of
superconductivity, magnetism and nematicity, it is important to investigate
different FeSCs, especially those which exhibit just one or two of these
phases.
LiFeAs is one of these unusual representatives of the FeSC-family. The role of
nematicity in LiFeAs is to date still strongly debated. It is superconducting
in the undoped parent compound with a superconducting transition temperature
between 16 K and 18 K [7, 8, 9, 10]. However, several experimental probes
including SQUID magnetometry [11, 7] and muon-spin rotation [12, 13] found no
evidence of a magnetic transition or long-range magnetic order. However, it
has been reported that LiFeAs could be in principle susceptible to nematicity
[14, 15] or even that it undergoes an unusual symmetry-breaking transition
[16, 17]. In particular, in Ref. [14], the electric field gradient in NMR
measurements was used as an indirect probe of local anisotropies in the
crystal structure and a finite albeit very small $\eta$ in LiFeAs has
therefore been interpreted as the signature of nematic fluctuations. The
presence of electronic-nematic fluctuations has also been suggested from an
unusual enhancement of the QPI amplitude in STM measurements [15], and even
the stabilization of static nematicity in the superconducting state and of
smectic electronic order under uniaxial strain have been reported in ARPES
[16] and STM [17] measurements.
Therefore, the natural role of the nematicity in LiFeAs calls for futher
investigation, especially with different probes. One very successful
experimental approach to investigate nematic properties in FeSCs is measuring
the strain-susceptibility of the resistivity
$\eta=\frac{\mathrm{d}}{\mathrm{d}\epsilon}\frac{\Delta\rho}{\rho}$. With this
elastoresistance technique, Chu et al. [18] were able to establish the
temperature-dependent behavior of $\eta$ as a proof of the electronic origin
of nematicity in FeSCs and a measure of the strength of nematic fluctuations,
which has since then been confirmed und used by other research groups [19, 20,
21, 22, 23].
In this work, we investigate nematic fluctuations in LiFeAs using
elastoresistance measurements on undoped, electron-doped and hole-doped
LiFeAs. In apparent contradiction to the aforementioned reports [14, 15, 16,
17], our results clearly indicate that pure LiFeAs is far away from a nematic
instability. We compare our results with elastoresistance data on NaFeAs,
which is significantly different from LiFeAs.

Figure 1:  Experimental Set-up: Silver wires glued to the sample with silver
paint function as electrodes. Green dashed lines at the piezo stack illustrate
the expansion under application of a positive voltage $V_{\mathrm{p}}$. Strain
$\epsilon$ is measured using a strain gauge on the backside, figure adapted
from Ref. [22]

NaFeAs single crystals were prepared according to Ref. [24]. Single crystals
of pristine as well as doped LiFeAs were grown by using the self-flux
technique as described in Refs. [25, 8]. Pre-reacted $\mathrm{Li}_{3}$As, FeAs
and $\mathrm{FeAs}_{2}$ were used. The mixture was placed in a graphite
crucible, which was placed in a Nb-container which was later sealed in a
quartz ampule to avoid oxidation. This quartz ampule was then placed
vertically in a furnace and heated to 1100 ∘C. After dwelling for 7 hours, the
furnace was cooled down at a rate of 4.5 K/h to 600 ∘C and later cooled with
100 ∘C/h to room temperature. As-grown crystals (exemplary image of
Li($\mathrm{Fe}_{0.972}\mathrm{V}_{0.028})$As in the inset of Figure 2 (d))
are highly sensitive to air and moisture. Single crystals were characterized
by using SEM/EDX and powder x-ray diffraction, which confirm the stoichiometry
and the phase of the LiFeAs.
All sample preparation steps have been performed in an argon-filled glove box
with $\mathrm{O}_{2}$ and $\mathrm{H}_{2}\mathrm{O}$ content less than 0.3
ppm. The glove box is equipped with a second 2m-long chamber that enables the
introduction of a whole measurement probe into the Ar-atmosphere. Few selected
pieces of as grown single crystals were cut mechanically into rectangular
shapes with lengths $<$ 4 mm and cleaved to thicknesses of the order of 50
$\upmu$m, to ensure a homogeneous transmission of strain through the whole
thickness (compare Refs. [18, 22]). Ag-wires were glued on top of a sample, in
order to perform four-terminal resistance measurements. The samples were glued
(using Devcon 14250 5 Minute Epoxy) along the [110]-crystal axis on top of a
piezoelectric actuator that allowed precise strain-control depending on an
applied voltage $V_{\mathrm{p}}$. The strain $\epsilon$ was measured using a
strain gauge glued onto the backside of the piezo (Figure 1). After making
contacts to the sample, it was mounted directly onto the probe, which then was
closed inside the Ar-box, discharged through the air lock and immediately
connected to a high-vacuum pump, which creates a vacuum of less than 1 $\cdot$
$10^{-6}$ mbar to 1 $\cdot$ $10^{-7}$ mbar. The evacuated probe was inserted
into liquid helium and cooled-down. Due to cryo-pumping, the sample stayed in
cryogenic vacuum conditions until it got warmed-up again.
In Figure 2 (a) and (b), we present the temperature-dependent resistivities
$\rho\left(T\right)$ of LiFeAs and NaFeAs, respectively.

Figure 2: Temperature-dependence of the resistivities $\rho(T)$ of (a) LiFeAs,
(b) NaFeAs, (c) Li($\mathrm{Fe}_{0.948}\mathrm{Co}_{0.052})$As, (d)
Li($\mathrm{Fe}_{0.972}\mathrm{V}_{0.028})$As; Insets of (a), (b), (c):
Superconducting transitions, Inset of (d): As-grown single crystal of
Li($\mathrm{Fe}_{0.972}\mathrm{V}_{0.028})$As Figure 3:  (a) Elastoresistance
$\eta$ of NaFeAs and Li($\mathrm{Fe}_{1-x}\mathrm{TM}_{x})$As (TM = V, Co).
NaFeAs shows a strong divergence, indicating the increasing amount of nematic
fluctuations towards the structural transition temperature. The electronic
response to strain is significantly smaller, almost non-existent in
Li($\mathrm{Fe}_{1-x}\mathrm{TM}_{x})$As (TM = V, Co). (b) The amplitude of
the elastoresistance of Li($\mathrm{Fe}_{0.972}\mathrm{V}_{0.028})$As shows a
small increase towards lower temperatures. The error bars of each data point
are derived from the statistical error of a linear fit through the
$\frac{\Delta\rho}{\rho}$ vs. $\epsilon$ data. In the tetragonal phase, a
negative sign of the nematic susceptibility indicates that the resistance is
smaller along the longer crystal axis a than along the shorter b

The resistivity trend and absolute value of both compounds correspond well to
earlier reports [26, 27]. The structural and magnetic phase transitions of
NaFeAs at $T_{\mathrm{S}}\sim$ 54 K and $T_{\mathrm{N}}\sim$ 43 K are visible
as kinks in $\rho(T)$ (inset in Figure 2 (b)), whereas no kinks or anomalies
are observed in the resistivity of LiFeAs as it approaches the sharp
superconducting transition at $T_{\mathrm{c}}\sim$ 16 K (inset in Figure 2
(a)). This comparably high transition temperature and the small absolute value
of $\rho$ indicate a high quality of the sample.
In LiFeAs, 5.2 % Co-doping increases the resistivity and lowers the
superconducting transition temperature to $T_{\mathrm{c}}\sim$ 11 K (Figure 2
(c)). Apparently, as can be inferred from Figure 2 (d), the impact of V-doping
on the resistivity is much stronger than that of Co-doping. Already at 2.8 %
V-doping the resistivity is higher than that of the Co-doped sample and no
sign of a superconducting transition down to 5 K is observed. The
$\rho-T-$curves of both doped LiFeAs compounds show no indication of a
magnetic or nematic phase transition, consistent with previous reports [28,
29, 30].
In Figure 3 (a), the temperature dependence of the elastoresistance $\eta$ of
all four FeSCs investigated in this study is shown. Remarkably, only the
elastoresistance amplitude of NaFeAs shows a diverging behavior towards low
temperatures typical for FeSCs with a nematic phase transition, indicating the
increase of nematic fluctuations towards $T_{\mathrm{S}}$. In contrast, in
LiFeAs, at no temperature a clear electronic response
$\frac{\Delta\rho}{\rho}$ to strain is observed. This is direct evidence for
the absence of nematic fluctuations, since local structural distortions
connected to local nematic order evidentially change the strain response of
the resistivity. The lack of a measurable increase of the elastoresistance
amplitude towards low temperatures indicates that the material is far away
from a nematic instability.
Having established that pure LiFeAs is far away from a nematic instability
opens up the question, whether it is possible to drive the material towards
nematicity by tuning external parameters. Here, we investigate the effect of
electron- and hole-doping on the nematic properties of LiFeAs by measuring
elastoresistance of Li($\mathrm{Fe}_{0.948}\mathrm{Co}_{0.052})$As and
Li($\mathrm{Fe}_{0.972}\mathrm{V}_{0.028})$As, respectively. In 5.2 % Co-doped
LiFeAs, similar to the undoped parent compound, no electronic response to
strain could be measured which leads to elastoresistance values scattered
around zero. The absence of any sign of local structural distortion/nematic
fluctuation in LiFeAs does not change upon 5.2 % Co-doping.
On the other hand, we report a small divergence of the nematic susceptibility
towards low temperatures in Li($\mathrm{Fe}_{0.972}\mathrm{V}_{0.028})$As
(Figure 3 (b)). Even though the absolute amplitude is low, indicating a small
but finite amount of nematic fluctuations, a trend is clearly visible. Note,
that the low temperature absolute value of $\rho$ of
Li($\mathrm{Fe}_{0.972}\mathrm{V}_{0.028})$As is an order of magnitude higher
compared to LiFeAs, thus enabling a very precise measurement of $\eta$ \- as
also noticable in the comparatively small size of the error bars.
The absence of nematicity in elastoresistance measurements on LiFeAs might
seem at odds with the aforementioned NMR [14], ARPES [16] and STM [15, 17]
reports. However, a closer inspection reveals a good consistency with these
data. More specifically, in Ref. [14], the electric field gradient of LiFeAs
is reported to be more than one order of magnitude smaller than that of NaFeAs
(absolute value smaller than 0.005 even at low temperatures). Such a small-
amplitude sign of electronic anisotropy is in principle consistent with the
small increase of the elastoresistance signal in LiFeAs in Figure 3. However,
also given the significant range of error compared to the small absolute
values, the signal can not unambigously be assigned to nematicity.
Furthermore, the nematic order reported from ARPES in Ref. [16] is interpreted
as a result of the superconductivity and therefore should disappear at the
elastoresistance-relevant temperatures above $T_{\mathrm{c}}$ (within the
limits of superconducting fluctuations). Finally, the QPI measurements and
analysis in Ref. [15] suggest a coupling between electronic and supposedly
nematic lattice modes at energies in the range of 10 meV. The energy scale of
our elastoresistance measurements is expected to be smaller since we measure
in quasistatic conditions in the zero-strain limit. It seems reasonable to
expect that under these condictions the nematic modes are energetically too
far away to affect the transport data.
Therefore in consistency with these previous reports on nematic properties in
LiFeAs, we conclude that the missing strain-response in the elastoresistance
measurements presented above strongly suggests that pure, undoped LiFeAs is
far away from a nematic instability.
The results on Co-doped LiFeAs seem in principle consistent with the
observation on canonical FeSCs with nematic ordering parent compounds (i.a.
Ba$\mathrm{Fe}_{2}\mathrm{As}_{2}$, NaFeAs) that the amplitude of the nematic
fluctuations declines upon increasing Co-doping to the overdoped side (as seen
in elastotransport [20] and NMR [14]). In such a scenario, LiFeAs would be
located deep in the electron doped side of the electronic phase diagram. In
this case, hole doping should in principle drive the system closer to the
nematic phase with a higher amplitude of nematic fluctuations. Indeed, this is
what is observed upon V-doping, where vanadium could act as an effective hole
dopant. In fact, V-doping is known to promote the emergence of strong
antiferromagnetic spin fluctuations, which seem supportive of this scenario
[29, 31, 32]. However, at the same time the V-doping apparently suppresses the
superconductivity, which indicates that this doping scheme does not simply
drive the system towards lower doping levels in the phase diagram of the
canonical electron-doped FeSCs.
In summary, we employed elastoresistance as direct probe of the nematic
properties of undoped, electron-doped and hole-doped LiFeAs. We showed, that
there is no sign of nematic fluctuations detectable in LiFeAs. Furthermore,
Co-doping does not induce nematic fluctuations, indicating that LiFeAs is on
the electron-doped side far away from a nematic instability. Finally, V-doping
of LiFeAs might induce nematic fluctuations, while at the same time
suppressing superconductivity.
This project has been supported by the Deutsche Forschungsgemeinschaft through
the Research Projects CA 1931/1-1 (F.C.) and AS 523/4-1 (S.A.). Furthermore,
this project has received funding from the European Research Council (ERC)
under the European Union’s Horizon 2020 research and innovation programme
(Grant Agreement No. 647276-MARS-ERC-2014-CoG) (X.H., C.H.). I.M., A.V., S.A.
and B.B. thank DFG and RSF for financial support in the frame of the joint
DFG-RSF project “Weyl and Dirac semimetals and beyond – prediction, synthesis
and characterization of new semimetals”.

## References

  * Avci  _et al._ [2011] S. Avci, O. Chmaissem, E. A. Goremychkin, S. Rosenkranz, J.-P. Castellan, D. Y. Chung, I. S. Todorov, J. A. Schlueter, H. Claus, M. G. Kanatzidis, A. Daoud-Aladine, D. Khalyavin, and R. Osborn, Magnetoelastic coupling in the phase diagram of ${\mathrm{Ba}}_{1-x}{\mathrm{K}}_{x}{\mathrm{Fe}}_{2}{\mathrm{As}}_{2}$ as seen via neutron diffraction, Phys. Rev. B 83, 172503 (2011).
  * Fernandes and Schmalian [2012] R. M. Fernandes and J. Schmalian, Manifestations of nematic degrees of freedom in the magnetic, elastic, and superconducting properties of the iron pnictides, Superconductor Science and Technology 25, 084005 (2012).
  * Böhmer and Meingast [2016] A. E. Böhmer and C. Meingast, Electronic nematic susceptibility of iron-based superconductors, Comptes Rendus Physique 17, 90 (2016).
  * Hirschfeld  _et al._ [2011] P. J. Hirschfeld, M. M. Korshunov, and I. I. Mazin, Gap symmetry and structure of Fe-based superconductors, Reports on Progress in Physics 74, 124508 (2011).
  * Fernandes  _et al._ [2014] R. M. Fernandes, A. V. Chubukov, and J. Schmalian, What drives nematic order in iron-based superconductors?, Nature Physics 10, 97 (2014).
  * Chubukov  _et al._ [2016] A. V. Chubukov, M. Khodas, and R. M. Fernandes, Magnetism, Superconductivity, and Spontaneous Orbital Order in Iron-Based Superconductors: Which Comes First and Why?, Physical Review X 6, 041045 (2016).
  * Pitcher  _et al._ [2010] M. J. Pitcher, T. Lancaster, J. D. Wright, I. Franke, A. J. Steele, P. J. Baker, F. L. Pratt, W. T. Thomas, D. R. Parker, S. J. Blundell, and S. J. Clarke, Compositional control of the superconducting properties of LiFeAs, Journal of the American Chemical Society 132, 10467 (2010).
  * Aswartham  _et al._ [2011] S. Aswartham, G. Behr, L. Harnagea, D. Bombor, A. Bachmann, I. V. Morozov, V. B. Zabolotnyy, A. A. Kordyuk, T. K. Kim, D. V. Evtushinsky, S. V. Borisenko, A. U. B. Wolter, C. Hess, S. Wurmehl, and B. Büchner, Suppressed superconductivity in charge-doped Li(Fe1-xCox)As single crystals, Phys. Rev. B 84, 054534 (2011).
  * Putzke  _et al._ [2012] C. Putzke, A. I. Coldea, I. Guillamón, D. Vignolles, A. McCollam, D. LeBoeuf, M. D. Watson, I. I. Mazin, S. Kasahara, T. Terashima, T. Shibauchi, Y. Matsuda, and A. Carrington, de Haas–van Alphen Study of the Fermi Surfaces of Superconducting LiFeP and LiFeAs, Physical Review Letters 108, 047002 (2012).
  * Nag  _et al._ [2016] P. K. Nag, R. Schlegel, D. Baumann, H.-J. Grafe, R. Beck, S. Wurmehl, B. Büchner, and C. Hess, Two distinct superconducting phases in LiFeAs, Scientific Reports 6, 27926 (2016).
  * Chu  _et al._ [2009] C. Chu, F. Chen, M. Gooch, A. Guloy, B. Lorenz, B. Lv, K. Sasmal, Z. Tang, J. Tapp, and Y. Xue, The synthesis and characterization of LiFeAs and NaFeAs, Physica C: Superconductivity 469, 326 (2009).
  * Pratt  _et al._ [2009] F. L. Pratt, P. J. Baker, S. J. Blundell, T. Lancaster, H. J. Lewtas, P. Adamson, M. J. Pitcher, D. R. Parker, and S. J. Clarke, Enhanced superfluid stiffness, lowered superconducting transition temperature, and field-induced magnetic state of the pnictide superconductor LiFeAs, Physical Review B 79, 052508 (2009).
  * Wright  _et al._ [2013] J. D. Wright, M. J. Pitcher, W. Trevelyan-Thomas, T. Lancaster, P. J. Baker, F. L. Pratt, S. J. Clarke, and S. J. Blundell, Magnetic fluctuations and spin freezing in nonsuperconducting LiFeAs derivatives, Physical Review B 88, 060401 (2013).
  * Toyoda  _et al._ [2018] M. Toyoda, Y. Kobayashi, and M. Itoh, Nematic fluctuations in iron arsenides NaFeAs and LiFeAs probed by ${}^{75}\mathrm{As}$ NMR, Phys. Rev. B 97, 094515 (2018).
  * Sun  _et al._ [2019] Z. Sun, P. K. Nag, S. Sykora, J. M. Guevara, S. Hoffmann, C. Salazar, T. Hänke, R. Kappenberger, S. Wurmehl, B. Büchner, and C. Hess, Spectroscopic evidence of nematic fluctuations in LiFeAs, Phys. Rev. B 100, 024506 (2019).
  * Kushnirenko  _et al._ [2020] Y. S. Kushnirenko, D. V. Evtushinsky, T. K. Kim, I. Morozov, L. Harnagea, S. Wurmehl, S. Aswartham, B. Büchner, A. V. Chubukov, and S. V. Borisenko, Nematic superconductivity in LiFeAs, Phys. Rev. B 102, 184502 (2020).
  * Yim  _et al._ [2018] C. M. Yim, C. Trainer, R. Aluru, S. Chi, W. N. Hardy, R. Liang, D. Bonn, and P. Wahl, Discovery of a strain-stabilised smectic electronic order in LiFeAs, Nature Communications 9, 2602 (2018).
  * Chu  _et al._ [2012] J.-H. Chu, H.-H. Kuo, J. G. Analytis, and I. R. Fisher, Divergent Nematic Susceptibility in an Iron Arsenide Superconductor, Science 337, 710 (2012).
  * Hosoi  _et al._ [2016] S. Hosoi, K. Matsuura, K. Ishida, H. Wang, Y. Mizukami, T. Watashige, S. Kasahara, Y. Matsuda, and T. Shibauchi, Nematic quantum critical point without magnetism in ${\mathrm{FeSe}}_{1-x}{\mathrm{S}}_{x}$ superconductors, Proceedings of the National Academy of Sciences of the United States of America 113, 8139 (2016).
  * Kuo  _et al._ [2016] H.-H. Kuo, J.-H. Chu, J. C. Palmstrom, S. A. Kivelson, and I. R. Fisher, Ubiquitous signatures of nematic quantum criticality in optimally doped Fe-based superconductors., Science (New York, N.Y.) 352, 958 (2016).
  * Tanatar  _et al._ [2016] M. Tanatar, A. Böhmer, E. Timmons, M. Schütt, G. Drachuck, V. Taufour, K. Kothapalli, A. Kreyssig, S. Bud’ko, P. Canfield, R. Fernandes, and R. Prozorov, Origin of the Resistivity Anisotropy in the Nematic Phase of FeSe, Physical Review Letters 117, 127001 (2016).
  * Hong  _et al._ [2020] X. Hong, F. Caglieris, R. Kappenberger, S. Wurmehl, S. Aswartham, F. Scaravaggi, P. Lepucki, A. U. B. Wolter, H.-J. Grafe, B. Büchner, and C. Hess, Evolution of the Nematic Susceptibility in ${\mathrm{LaFe}}_{1-x}{\mathrm{Co}}_{x}\mathrm{AsO}$, Phys. Rev. Lett. 125, 067001 (2020).
  * Caglieris  _et al._ [2021] F. Caglieris, C. Wuttke, X. C. Hong, S. Sykora, R. Kappenberger, S. Aswartham, S. Wurmehl, B. Büchner, and C. Hess, Strain derivative of thermoelectric properties as a sensitive probe for nematicity, npj Quantum Materials 6, 1–7 (2021).
  * Steckel  _et al._ [2015] F. Steckel, M. Roslova, R. Beck, I. Morozov, S. Aswartham, D. Evtushinsky, C. G. F. Blum, M. Abdel-Hafiez, D. Bombor, J. Maletz, S. Borisenko, A. V. Shevelkov, A. U. B. Wolter, C. Hess, S. Wurmehl, and B. Büchner, Crystal growth and electronic phase diagram of $4d\text{$-$}\mathrm{doped}$ ${\mathrm{Na}}_{1-\delta}{\mathrm{Fe}}_{1-x}{\mathrm{Rh}}_{x}\mathrm{As}$ in comparison to $3d-\text{doped}{\mathrm{Na}}_{1-\delta}{\mathrm{Fe}}_{1-x}{\mathrm{Co}}_{x}\mathrm{As}$, Phys. Rev. B 91, 184516 (2015).
  * Morozov  _et al._ [2010] I. Morozov, A. Boltalin, O. Volkova, A. Vasiliev, O. Kataeva, U. Stockert, M. Abdel-Hafiez, D. Bombor, A. Bachmann, L. Harnagea, M. Fuchs, H. J. Grafe, G. Behr, R. Klingeler, S. Borisenko, C. Hess, S. Wurmehl, and B. Büchner, Single crystal growth and characterization of superconducting LiFeAs, in  _Crystal Growth and Design_, Vol. 10 (American Chemical Society, 2010) pp. 4428–4432.
  * Zhang  _et al._ [2012] Y. Zhang, C. He, Z. R. Ye, J. Jiang, F. Chen, M. Xu, Q. Q. Ge, B. P. Xie, J. Wei, M. Aeschlimann, X. Y. Cui, M. Shi, J. P. Hu, and D. L. Feng, Symmetry breaking via orbital-dependent reconstruction of electronic structure in detwinned NaFeAs, Physical Review B 85, 085121 (2012).
  * Rullier-Albenque  _et al._ [2012] F. Rullier-Albenque, D. Colson, A. Forget, and H. Alloul, Multiorbital Effects on the Transport and the Superconducting Fluctuations in LiFeAs, Physical Review Letters 109, 187005 (2012).
  * Xing  _et al._ [2014] L. Y. Xing, H. Miao, X. C. Wang, J. Ma, Q. Q. Liu, Z. Deng, H. Ding, and C. Q. Jin, The anomaly Cu doping effects on LiFeAs superconductors, Journal of Physics: Condensed Matter 26, 435703 (2014).
  * Xing  _et al._ [2016] L. Y. Xing, X. Shi, P. Richard, X. C. Wang, Q. Q. Liu, B. Q. Lv, J.-Z. Ma, B. B. Fu, L.-Y. Kong, H. Miao, T. Qian, T. K. Kim, M. Hoesch, H. Ding, and C. Q. Jin, Observation of non-Fermi liquid behavior in hole-doped ${\mathrm{LiFe}}_{1-x}{\mathrm{V}}_{x}\mathrm{As}$, Phys. Rev. B 94, 094524 (2016).
  * Xu [2] Supplementary material of [31].
  * Xu  _et al._ [2020] Z. Xu, G. Dai, Y. Li, Z. Yin, Y. Rong, L. Tian, P. Liu, H. Wang, L. Xing, Y. Wei, R. Kajimoto, K. Ikeuchi, D. L. Abernathy, X. Wang, C. Jin, X. Lu, G. Tan, and P. Dai, Strong local moment antiferromagnetic spin fluctuations in V-doped LiFeAs, npj Quantum Materials 5, 1 (2020).
  * Sheng  _et al._ [2020] J. Sheng, X. Li, C. Tian, J. Song, X. Li, G. Sun, T. Xia, J. Wang, J. Liu, D. Xu, H. Zhang, X. Tong, W. Luo, L. Wu, W. Bao, and P. Cheng, Evolution of superconductivity and antiferromagnetic order in $\mathrm{Ba}{({\mathrm{Fe}}_{0.92-x}{\mathrm{Co}}_{0.08}{\mathrm{V}}_{x})}_{2}{\mathrm{As}}_{2}$, Phys. Rev. B 101, 174516 (2020).