#  CausalDialogue: Modeling Utterance-level Causality in Conversations

Yi-Lin Tuan♣ Alon Albalak♣ Wenda Xu♣ Michael Saxon♣ Connor
Pryor${}^{\vardiamondsuit}$
Lise Getoor${}^{\vardiamondsuit}$ William Yang Wang♣
♣ University of California, Santa Barbara, ${}^{\vardiamondsuit}$ University
of California, Santa Cruz
{ytuan, alon_albalak, wendaxu, saxon<EMAIL_ADDRESS>
{cfpryor<EMAIL_ADDRESS>

###### Abstract

Despite their widespread adoption, neural conversation models have yet to
exhibit natural chat capabilities with humans. In this research, we examine
user utterances as causes and generated responses as effects, recognizing that
changes in a cause should produce a different effect. To further explore this
concept, we have compiled and expanded upon a new dataset called
CausalDialogue through crowd-sourcing. This dataset includes multiple cause-
effect pairs within a directed acyclic graph (DAG) structure. Our analysis
reveals that traditional loss functions struggle to effectively incorporate
the DAG structure, leading us to propose a causality-enhanced method called
Exponential Maximum Average Treatment Effect (ExMATE) to enhance the impact of
causality at the utterance level in training neural conversation models. To
evaluate the needs of considering causality in dialogue generation, we built a
comprehensive benchmark on CausalDialogue dataset using different models,
inference, and training methods. Through experiments, we find that a
causality-inspired loss like ExMATE can improve the diversity and agility of
conventional loss function and there is still room for improvement to reach
human-level quality on this new dataset. 111Our code and dataset are available
at https://github.com/Pascalson/CausalDialogue

##  1 Introduction

Over time, broadly-defined dialogue models have become increasingly prevalent
in society and been integrated in a range of domains from speech assistants
and customer service systems to entertainment products, such as video games,
where the non-playable characters (NPCs) engage in conversation with players.
A core goal of training chatbots is enabling them to interact with humans
naturally Vinyals and Le (2015); Sordoni et al. (2015). This includes, but is
not limited to: considering both the machine and addressee’s personalities Li
et al. (2016b), diversifying responses to be less generic (e.g., the same
response “I don’t know.” is often produced in a traditional setting for
different dialogues) Li et al. (2016a), grounding on external knowledge to be
informative Ghazvininejad et al. (2018), and tailoring responses specific to
nuanced differences in conversation.

To the best of our knowledge, no recent studies have prioritized the ability
to tailor responses for minor differences in conversations. This problem is
currently implicitly approached by training models with larger scale or
cleaner conversation data Zhang et al. (2020); Roller et al. (2021); Thoppilan
et al. (2022) or involving human-in-the-loop Li et al. (2016c); Jaques et al.
(2020). However, the effectiveness of these methods is unclear, the online
rewarding scheme can be expensive, and a suitable testbed for evaluating the
solution to this problem has not yet been identified.

Figure 1: A dialogue DAG example in the new dataset CausalDialogue. As the
conversation progress, each utterance can be continued with multiple responses
(branch-splitting; fork); meanwhile, the same root dialogue with different
middle turns can be continued by the same response (branch-colliding;
collider).

To this end, we propose a benchmark to foster research in tailoring responses
for nuanced differences in conversations by answering the question “if all
prior turns are the same, but the last turns in two conversations are
semantically different, how should future turns differ?” We call this concept
Agility and model it as the utterance-level causes and effects in dialogue
response generation, where the causes are the slightly different prior turns
and the effects are the resulting future turns.

We introduce CausalDialogue, a dataset seeded by expert-written dialogues
containing branching dialogue paths, which we further expand in terms of scale
and linguistic abundance with crowd-sourcing. Each conversation is represented
as a directed acyclic graph (DAG) for ease of storage and causal analysis
Pearl (2009) as shown in Figure 1. As conversations progress, each utterance
can elicit multiple responses, resulting in a split of the conversation
(branch-splitting). Alternatively, multiple conversations that share a common
starting point may sometimes lead to the same response, even if the middle
exchanges differ (branch-colliding). Due to the DAG structure of
CausalDialogue, it is ideal for aiding research on response generation that
requires abundant IF-bases, for instance, causal inference and offline
reinforcement learning, which may improve the response generation quality for
nuanced differences in conversation.

| CausalDialogue | TV Series | MultiTalk | DailyDialog | PersonaChat | LIGHT
---|---|---|---|---|---|---
Branches |  ✓(DAG) | ✗ |  ✓(Tree) | ✗ | ✗ | ✗
Profiles | ✓ | ✓ | ✗ | ✗ | ✓ | ✓
Situated | ✓ | ✓ | ✓ | ✗ | ✗ | ✓
Expert involved | ✓ | ✓ | ✗ | ✓ | ✗ | ✗
Table 1: Compared to current widely used datasets, CausalDialogue contains the
utterance-level graph structure and meanwhile has the features of diverse
speaker profiles, descriptive situations, and high quality scripts written by
experts. The referring dialogue generation datasets are: TV series Tuan et al.
(2019), MultiTalk Dou et al. (2021), DailyDialog Li et al. (2017b),
PersonaChat Zhang et al. (2018), LIGHT Urbanek et al. (2019).

To provide a benchmark for future work on the CausalDialogue dataset, we
conduct experiments with various setups. We include both decoder-only and
encoder-decoder transformer models pretrained on either common or dialogue-
specific corpora, various inference methods, conventional training losses, and
a newly proposed loss, Exponential Maximum Average Treatment Effect (ExMATE),
inspired by Average Treatment Effect Holland (1986); Imai et al. (2008), which
is a method commonly used to approximate the causal effect of a treatment and
its outcome. In this benchmark, we show that existing methods are not
sufficient in tackling the agility issue, and a simple causality-inspired loss
demonstrates improvement.

Our key contributions are:

  * •

A novel dataset, CausalDialogue, including both expert-written scripts and
crowd-sourced utterances with a DAG structure.

  * •

A new training loss, ExMATE, for considering the utterances as causes and
effects in a dialogue, inspired by the average treatment effect in research on
causal inference.

  * •

A benchmark with experiments showing that existing methods need improvement on
the agility problem, and a causality-inspired method can be a promising
direction to improve it.

##  2 Related Work

#### Chit-Chat Dialogue Datasets.

To boost the research of dialogue models, the community has collected
dialogues based on scripts written by experts from movies Danescu-Niculescu-
Mizil and Lee (2011); Banchs (2012); Lison and Tiedemann (2016), TV shows
Poria et al. (2019); Tuan et al. (2019); Yu et al. (2020); Rameshkumar and
Bailey (2020), and education purposes Li et al. (2017b); Cui et al. (2020).
For abundant diversity and real-life scenarios, Ritter et al. (2011); Wang et
al. (2013); Lowe et al. (2015); Pasunuru and Bansal (2018) collected datasets
based on the publicly available data from social media and forums.
Additionally, previous work has explored the idea of collecting data through
crowd-sourcing with added constraints to improve its quality or expand label
types. For example, Zhang et al. (2018) constructed a dataset with workers
imitating a given personal profile. Rashkin et al. (2019) built a dataset by
explicitly asking workers to show their empathy during a conversation. Urbanek
et al. (2019); Narayan-Chen et al. (2019); Ammanabrolu et al. (2021) created
datasets with the assistance of game structures, so the purpose of the
dialogue is to complete a mission or collaborate with other agents. Finally,
recent work by Dou et al. (2021) collected branches of dialogues for 120 self-
written prompts to create dialogue trees. Compared to previous studies, our
dataset is a fusion of the scripts written by experts and responses created by
crowd-sourcers with manual correction, granting it high quality, linguistic
abundance, and extensive metadata. Additionally, our dataset includes both
branch-splitting and branch-colliding instances, which has led us to classify
dialogues as directed acyclic graphs (DAGs) instead of just sequences or
trees.

#### Dialogue Generation Training Objectives.

To train a dialogue response generation model, methods have been developed
from maximizing the likelihood between the hypothesis and the ground-truth
Vinyals and Le (2015); Serban et al. (2016), guiding responses to match a
higher reward in reinforcement learning Li et al. (2016d), and allowing for
extra latent variables to optimize divergence through variational autoencoder
Zhao et al. (2017) or generative adversarial networks Li et al. (2017a); Tuan
and Lee (2019). Recent works have introduced the concept of causal inference
Holland (1986); Imai et al. (2008); Pearl (2009); Cunningham (2021) into
generative adversarial network-based Zhu et al. (2020) and multiple-stage
inference based dialogue generation model Tuan et al. (2020). Utterance-level
offline reinforcement learning has also been explored to optimize response
generation Jaques et al. (2020); Verma et al. (2022). However, they were
studied by expanding available sequence data with imaginations. Now by
providing a chit-chat dialogue DAG structure that is enriched with multiple
if-else cases, CausalDialogue can be studied for causal inference and offline
reinforcement learning on response generation. We also propose a new method
called ExMATE for better optimizing a response generation model on the DAG
data structure.

##  3 CausalDialogue Dataset

In this section, we introduce CausalDialogue, a novel dataset that includes
chit-chat conversations in a Conversational Directed Acyclic Graph (DAG) data
structure. This structure allows for the natural inclusion of various dialogue
flows, such as forks (branch-splitting) and colliders (branch-colliding) Pearl
(2009). Our goal is to offer researchers a valuable resource for studying the
complexities of human conversation and advancing the understanding of causal
inference in dialogue.

To create CausalDialogue, we sourced expert-written dialogues from a role-
playing game (Section 3.1) and expanded upon them with Amazon Mechanical Turk
(MTurk)222https://www.mturk.com and manual correction (Section 3.2). By using
our fused collection method, the dataset contains high-quality, engaging
conversations with abundant linguistic usage that imitates daily life.

###  3.1 Data Collection

CausalDialogue is derived from the English scripts of the popular role-playing
game (RPG) Fire Emblem: Three Houses, which we sourced from the fandom
wikipedia333https://fireemblem.fandom.com/ under the GNU Free Documentation
License(GFDL)444https://fireemblem.fandom.com/wiki/Fire_Emblem_Wiki:Copyrights.
This RPG is well-known for its diverse, story-driven conversations, which mix
the interactions of approximately 40 main characters. In this game, players
have the ability to shape the narrative by making choices that lead to
different dialogue branches.

Table 2 lists the statistics of the two main types of the crawled data, which
are already divided in the raw scripts. We name the first conversation type
Ori.-2S, which are mostly dialogues between two speakers, and generally
include conversations about interpersonal relationships. We name the second
conversation type Multi, which are dialogues between two or more speakers, and
usually describe the current status of the story line. In the following
sections, we will introduce the DAG structure to better describe the dataset,
as well as how we obtained additional examples from crowd-sourcing to create
the Expansion to these expert-written scripts.

Data Partition | Ori.-2S | Multi | Expan. | Total
---|---|---|---|---
# Dialogues† | 794 | 1528 | 623 | 2322
# Branches | 1633 | 1298 | 2378 | 4866
# Utterances | 33247 | 13858 | 15728 | 46109
# Speakers | 41 | 47 | 39 | 51
Avg. utts/dial. | 17.0 | 51.4 | 5.6 | 26.8
Avg. words/utt. | 18.4 | 17.8 | 11.8 | 16.5
Avg. utts/spk. | 801.6 | 268.8 | 402.8 | 878.4
Table 2: The statistics of CausalDialogue dataset, where the columns Ori.-2S
and Multi are the crawled and cleaned original scripts and the column
Expansion is from crowd-sourcing. In total, there are 3457/741/715 dialogues
for train/validation/test sets. $\dagger$ indicates that for Expansion set,
623 is the number of initial dialogues that are parts of the 794 Ori.-2S
dialogues, so the total number of dialogues is 2322.

#### Dialogue DAGs.

Conventional linear dialog data structures can be challenging to create when
dealing with forks and colliders, as they can lead to ambiguity in the form of
duplicated utterances and split responses. To address this issue, we propose
using a conversational DAG to maintain the fidelity of the dialog. We convert
each textual conversation into a DAG, as demonstrated in Figure 1. Formally,
each node is a dictionary containing the text type (utterance/scene
information), text, speaker, and its own id in the dialogue. A directed edge
$(i,j)$ then indicates that a node with id $j$ is a possible response to the
node with id $i$. Saving dialogues as DAGs may introduce some complexity, but
it also offers numerous benefits. For example, it reduces the memory required
to save each dialogue branch independently, enables a natural visualization of
the multiple possible dialogues flows, and fosters the survey of causality on
dialogue utterances.

#### Speaker Profiles.

Prior work has shown the relationship between personality and language uses in
conversations Mairesse et al. (2007). To ensure consistent personality, as
well as to diversify linguistic features across speakers, we leverage the
speaker profiles during the data collection process. The resulting
CausalDialogue dataset comprises 41 main speakers who have been thoughtfully
crafted by the game’s developers. These speakers possess diverse backgrounds,
perspectives, and interests, and their characteristics are both human-like and
distinct. These speaker profiles are simplified for collecting the Expansion
partition to reduce workers’ cognitive load, and a set of examples are
provided in Appendix A.1. Compared with the speaker profiles in
CausalDialogue, previous works have provided limited information (e.g. “I have
a dog.”) Zhang et al. (2018); Urbanek et al. (2019), or have a significantly
smaller number of speakers Poria et al. (2019); Tuan et al. (2019)

Figure 2: The flowchart of our strategy for data expansion with crowd-
sourcing. Figure 3: A dialogue example of the Expansion partition in
CausalDialogue.

###  3.2 Data Expansion

In order to increase the breadth and scope of our dataset, we propose
utilizing a crowd-sourcing approach to add more diverse and current language
as shown in Figure 2 (More details in Appendix A.2).

#### Initial Dialogue Selection.

We first randomly select 1,200 partial dialogues from the Ori.-2S partition,
which is of higher quality after our manual inspection. This can result in
more stable quality when crowd-sourcing responses.

#### Expansion Collection.

Each initial dialogue along with the continuing speaker profile is presented
to 3 workers on MTurk to write the next utterance. A new branch of continued
dialogue will then be presented to another 1-2 workers playing as another
speaker to gather another round of responses. We repeated this process three
times and collect a total of about 13,000 written utterances. Table 2 lists
the detailed statistics of the expanded data in the column Expansion. Note
that the statistics of Expansion in Table 2 include the initial dialogues.
Figure 3 shows an DAG representation of an expanded example.

#### Quality Control.

We adopt three strategies to control for dialogue quality. First, we asked the
workers on MTurk to annotate if they regard a dialogue as already completed or
having too specific of details to continue. The purpose of the first stage of
quality control is to identify conversations which cannot be continued, either
because the conversation has already concluded or because the workers are
lacking enough information about the world to continue the conversation.
Second, we used an off-the-shelf model555https://github.com/unitaryai/detoxify
to label potential ethical issues inside the collected utterances for
reference in the next step. Finally, we invited real players of the game and
machine learning researchers to manually check all the utterances by their
fluency, coherence, and ethics as well as referring to the labels from the
previous two steps to ensure the final Expansion partition is of high quality.

##  4 Task Definition

In this work we consider a conversation among two or more speakers. At each
time step $t$, a speaker $s_{t}$ takes their turn with an utterance $u_{t}$.
The goal, as in conventional response generation, is to train a model
parameterized by $\theta$ that can predict a plausible next utterance given
the speakers and utterances in prior turns as:

$u_{t+1}\sim
P_{\theta}(\cdot|s_{1}u_{1},s_{2}u_{2},...,s_{t}u_{t},s_{t+1})\,.$ (1)

Distinct from prior conversation datasets, CausalDialogue has multiple
dialogue branches. If we consider each branch as an independent conversation
(flatten the branches), many conversations will have large overlaps and thus
bias the dataset. We consider this point and extract triples $(DH,x,y)$ from
CausalDialogue. To simplify notations for following sections, we denote
$s_{t}u_{t}$ as $x$, $s_{t+1}u_{t+1}$ as $y$ and $DH$ is the dialogue history
$s_{1}u_{1},s_{2}u_{2},...,s_{t-1}u_{t-1}$. The key idea is that for a $DH$,
we will not extract duplicated pairs $(x,y)$, but $x$ or $y$ itself can be
shared.

The CausalDialogue response generation task is therefore defined as finding a
possible turn-taking speaker and their response given the dialogue history
$DH$ with an utterance cause $x$.

$y\sim P_{\theta}(\cdot|DH,x)\,.$ (2)

The sequences $x=x_{1}x_{2}...x_{i}...x_{|x|}$ and
$y=y_{1}y_{2}...y_{j}...y_{|y|}$, where $x_{i}$ and $y_{j}$ are tokens, and
$|x|$ and $|y|$ are the length of the sequences $x$ and $y$ respectively.

###  4.1 Agility

While the above task definition resembles the standard dialogue generation
setting with the exception of speaker prediction and conversation overlaps,
our primary interest lies in tailoring responses to minor differences in
conversation history. We refer to this concept as Agility, where a minor
difference in conversations can be a shared $DH$ with different continuation
$x$.

To quantify the idea of agility, we propose a new metric with the following
idea: If the predicted next utterance $y$ and the previous turn $x$ has
causal-effect relationship (i.e., $x_{1}\rightarrow y_{1}$ and
$x_{2}\rightarrow y_{2}$), we anticipate that it is less likely that $y_{2}$
is caused by $x_{1}$. The newly proposed metric, named confidence causal-
effect (CCE) is formally defined as:

$\begin{split}CCE=&E_{(x,y)\in D,(x,y^{\prime})\notin
D,(x^{\prime},y^{\prime})\in D}\\\
&[PPL_{\theta}(y^{\prime}|DH,x)-PPL_{\theta}(y|DH,x)]\,,\end{split}$ (3)

where PPL refers to perplexity. Note that CCE is not a metric that stands by
itself and needs to refer to PPL at the same time. That is, given a similar
PPL score, a model with higher CCE score is better. Additionally, it is
important to acknowledge that the concept of agility has been indirectly
incorporated into conventional dialogue generation models and evaluation
metrics, but it has not been specifically examined in isolation. Our newly
introduced dataset and CCE metric can be seen as an initial step towards
addressing this aspect.

##  5 Methods

In this section, we describe how conventional generative models can be used
and propose a simple yet effective approach to model causal effect.

###  5.1 Maximize Likelihood Estimation

An often used method to train a conditional sequence generation model is
minimizing the negative log likelihood Vinyals and Le (2015); Serban et al.
(2016). The loss function is as following:

$\begin{split}&L_{MLE}=\\\ &\underset{(DH,x,y)\sim
P_{D}}{E}\sum_{j=1}^{|y|}-\log
P_{\theta}(y_{j}|DH,x,y_{1...j-1})\,,\end{split}$ (4)

where $P_{D}$ represents the data distribution. Since the duplication of
dialogue history is already taken in to account in our task definition
(Section 4), this MLE method can be seen as the recently proposed dialogue
tree model Dou et al. (2021). However, this function only models a part of the
cause-effect relationship between the condition and the output sequence. This
neglect may lead to a more vague predicted probability distribution of the
output, thus generating less agile responses.

###  5.2 Maximize Average Treatment Effect

To explicitly model the causal effect in a conversation, we propose the
Exponential Maximum Average Treatment Effect (ExMATE), taking into account the
treatment effect in causal inference Pearl (2009). The treatment effect,
denoted by $\delta$, is defined as the difference between the outcome under
treatment $I=1$, represented by $\mathcal{O}^{I=1}$, and the outcome under
treatment $I=0$, represented by $\mathcal{O}^{I=0}$. This measures the
variation in outcomes when an event $I$ is present or absent. A higher value
of $\delta$ indicates that the event $I$ is more likely to be a true cause of
the outcome. Conversely, a small value of $\delta$ suggests that the event $I$
is unlikely to be a cause of the outcome and may only be correlated. We aim to
utilize this characteristic in dialogue generation modeling to ensure that a
preceding utterance can be considered the genuine cause of the predicted
response.

We consider the fork-like DAGs (as shown in Figure 4) existing in a dataset
such as Figure 1 and Figure 3. Without loss of generality, in a binary case,
this type of DAG involves two triples that share the same $DH$ and can be
simplified as having nodes $DH$, $X_{1}$, $X_{2}$, $Y_{1}$, and $Y_{2}$. Here
we use ($X_{1}$, $Y_{1}$) and ($X_{2}$, $Y_{2}$) to denote two possibilities
of ($x$,$y$) after $DH$. We take $I=1$ as choosing the branch $X_{1}$, and
$I=0$ as choosing an alternative branch $X_{2}$. Therefore, a traditional
definition of the treatment effect
$\delta_{i}=|\mathcal{O}_{i}^{I=1}-\mathcal{O}_{i}^{I=0}|$ for the $i$-th
example in this type of DAG can be rewritten as:

$\delta_{i}\triangleq\underset{\begin{subarray}{c}X_{1}\sim
P_{D}(\cdot|DH_{i}),\\\ X_{2}\sim P_{D}(\cdot|DH_{i}),\\\ X_{1}\neq
X_{2}\end{subarray}}{E}|\mathcal{O}_{i}^{X_{1}}-\mathcal{O}_{i}^{X_{2}}|\,,$
(5)

where $\mathcal{O}^{X_{1}}_{i}$ or $\mathcal{O}^{X_{2}}_{i}$ is the outcome of
an oracle given $X_{1}$ or $X_{2}$ as the input.

Since the outcome of a dialogue model is hard to be mathematically described
only by an input $X$, we instead utilize the uncertainty of predicting the
pair $(x,y)$ by a model $\theta$. We abuse the notation $\mathcal{O}_{i}$ here
and redefine it as,

$\mathcal{O}_{i,Y_{1}}^{X_{1}}\triangleq P_{\theta}(Y_{1}|DH,X_{1})\,.$ (6)

Figure 4: The graphical model of fork-like DAG considered in our proposed ExMATE loss. Model | Loss | Inference | Fluency | Diversity | Agility | Identity
---|---|---|---|---|---|---
PPL ($\downarrow$) | BLEU1 ($\uparrow$) | 2 ($\uparrow$) | 4 ($\uparrow$) | Dist1 | Dist2 | CCE ($\uparrow$) | Acc ($\uparrow$)
Human Written Responses | 1.2 | 48.9 | 34.0 | 25.9 | 1.70 | 11.1 | Inf | 100.0
DG | MLE | Greedy Search | 18.9 | 11.2 | 4.47 | 0.84 | 0.73 | 3.42 | 2.33 | 32.51
DG | MLE | Softmax (T=0.5) | 18.9 | 17.0 | 6.43 | 1.17 | 1.12 | 9.09 | 2.33 | 30.97
DG | MLE | TopK (K=10) | 18.9 | 15.7 | 5.34 | 0.81 | 1.37 | 13.57 | 2.33 | 27.65
DG | ExMATE | Greedy Search | 19.0 | 10.7 | 4.26 | 1.05 | 0.79 | 3.65 | 2.68 | 32.18
DG | ExMATE | Softmax (T=0.5) | 19.0 | 15.5 | 5.70 | 1.06 | 1.25 | 9.71 | 2.68 | 31.18
DG | ExMATE | TopK (K=10) | 19.0 | 13.5 | 4.47 | 0.67 | 1.52 | 14.44 | 2.68 | 28.16
T5 | MLE | Greedy Search | 15.4 | 5.80 | 2.52 | 0.58 | 1.11 | 4.37 | 1.39 | 75.64
T5 | MLE | Softmax (T=0.5) | 15.4 | 12.7 | 5.06 | 0.97 | 1.77 | 10.91 | 1.39 | 74.66
T5 | MLE | TopK (K=10) | 15.4 | 14.1 | 5.09 | 0.82 | 2.07 | 15.49 | 1.39 | 72.79
T5 | ExMATE | Greedy Search | 15.4 | 5.66 | 2.46 | 0.55 | 1.10 | 4.06 | 1.50 | 75.76
T5 | ExMATE | Softmax (T=0.5) | 15.4 | 12.6 | 5.02 | 1.00 | 1.72 | 10.73 | 1.50 | 74.80
T5 | ExMATE | TopK (K=10) | 15.4 | 14.1 | 5.06 | 0.80 | 2.06 | 15.67 | 1.50 | 72.83
Table 3: The test results on CausalDialogue of different fine-tuned backbone
models (DialoGPT (DG) and T5), inference methods (Greedy Search, Softmax,
TopK), and loss functions (MLE and ExMATE). Using ExMATE loss enhances the
agility aspect of dialogue generation models without compromising their
fluency ratings.

After formulating a dialogue generation problem as utterance-level causal
analysis as above, we apply the Average Treatment Effects (ATE) Holland (1986)
to conversational DAGs, which is defined as

$\begin{split}&ATE\triangleq
E_{i}[\delta_{i}]=E_{i}[\delta_{i,Y_{1}}+\delta_{i,Y_{2}}]\\\
&=E_{i}[\mathcal{O}_{i,Y_{1}}^{X_{1}}-\mathcal{O}_{i,Y_{1}}^{X_{2}}+\mathcal{O}_{i,Y_{2}}^{X_{2}}-\mathcal{O}_{i,Y_{2}}^{X_{1}}]\,.\end{split}$
(7)

Recall that our goal is to strengthen the cause-effect relationship of each
pair, ($X_{1}$,$Y_{1}$) and ($X_{2}$,$Y_{2}$) in the binary case. This can be
taken as maximizing the defined ATE in Equation 7 with respect to the model
parameters $\theta$.

Therefore, we substitute the $\mathcal{O}_{i,Y}^{X}$ term in Equation 7 with
its definition stated in Equation 6 and derive:

$\begin{split}\arg\max_{\theta}&ATE=\\\
\arg\max_{\theta}&\underset{(X_{i},Y_{i})\sim
P_{D}(\cdot|DH)}{E}P_{\theta}(Y_{i}|DH,X_{i})\\\
&-\underset{\begin{subarray}{c}X_{i}\sim P_{D}(\cdot|DH),Y_{j}\sim
P_{D}(\cdot|DH)\\\ (DH,X_{i},Y_{j})\notin
D\end{subarray}}{E}P_{\theta}(Y_{j}|DH,X_{i})\,.\end{split}$ (8)

To stabilize the training, we modify it with logarithmic and exponential terms
and call it the ExMATE loss function. Formally, it is written as:

$\begin{split}&L_{ExMATE}=\\\ &\underset{\begin{subarray}{c}(DH,x,y)\sim
P_{D},\\\ x_{c}\sim P_{D}(\cdot|DH),\\\ (DH,x_{c},y)\notin
D\end{subarray}}{E}-\log P_{\theta}(y|DH,x)+\exp(\log
P_{\theta}(y|DH,x_{c}))\,.\end{split}$ (9)

The intuition for this change is that without $\exp(\cdot)$, the gradient of
the second term will dominate the loss function, since $\log(u)$ has much
larger gradient for $u$ close to $0$ than $u$ close to $1$ and an
$\exp(\cdot)$ term can linearize it.

Overall, the idea of ExMATE is to maximize the response generation model’s
causal effects given a specific $X_{i}$ (or $(DH,x)$) as the current cause. At
the end, we found that this ATE-inspired approach turns out to be a
combination of MLE and a subtraction of specific negative samples. This
formulation shares a similar concept with negative sampling and contrastive
learning Goldberg and Levy (2014); Chen et al. (2020), but has different
example selection scheme and is not applied on the embedding space. With this
method, we are interested in the research question: Will a model trained on
the CausalDialogue dataset be affected when using a causality-inspired loss?

##  6 Experiments

We provide a preliminary benchmark for CausalDialogue with often used methods
and a naive causality-inspired loss. We fine-tuned two types of pretrained
language models based on transformers Vaswani et al. (2017): decoder-only
architecture, DialoGPT Zhang et al. (2020) and encoder-decoder architecture,
T5 Raffel et al. (2020), by the conventional MLE loss and the proposed ExMATE
loss, and inferred by various sampling methods. We evaluate three aspects of
the generated responses: Fluency (perplexity (PPL) and BLEU Papineni et al.
(2002)), Diversity (Distinct n-grams Dist1 and Dist2 Li et al. (2016a)), and
our proposed Agility (CCE) in Section 4.1. Furthermore, we use accuracy to
evaluate if the speaker for a given turn is correctly predicted as the one in
the human written responses (Identity Acc). More details are in Appendix A.4
and A.6.

Dialogue History ($DH$) |  Lysithea: Oh, hey. It’s you. Going for a walk again today? Ignatz: No, I’m on cooking duty today, and I have to head into town for some groceries.
---|---
Cont. Conv ($x$) |  case1 Lysithea: That sounds like quite a task! |  case2 Lysithea: Would you like some company?
response($y$) | MLE+S |  Ignatz: You’re right. |  Ignatz: Yes, I’d love to.
MLE+T |  Ignatz: I’ll go for a run. |  Ignatz: No, I’m not.
ExMATE+S |  Ignatz: I’m not sure I can handle it. |  Ignatz: Yes, I’m a bit tired of the whole thing.
ExMATE+T |  Ignatz: Well, I have to find a new name for myself. |  Ignatz: No! It’s just me and I.
Cont. Conv ($x$) |  case3 Lysithea: Allow me to join you so I can help. |  case4 Lysithea: All on your own? Hm, I’d better go with you. I’d be worried if you went by yourself.
response($y$) | MLE+S |  Ignatz: Oh, I’m on cooking duty. |  Ignatz: I’m just a bit worried about you.
MLE+T |  Ignatz: I’m going to help cook. |  Ignatz: Yeah, no, not really! I can’t help you, but I’m just going to help you out.
ExMATE+S |  Ignatz: I’m not working, but I’ll help you. |  Ignatz: I’ve got no choice. I’ll just have to go out alone.
ExMATE+T |  Ignatz: Oh. I’m sorry, I couldn’t be there for you. |  Ignatz: Is it okay?
Table 4: Generated responses by our trained models, T5 models trained by MLE
or ExMATE inferred by Softmax(S) or TopK(T) sampling methods, given a shared
dialogue history but different branches. Using ExMATE loss generally produces
more diverse and agile responses.

###  6.1 Results

The test results of human written responses and models trained and inferred by
different setups are listed in Table 3. We provide details of how we evaluate
human written responses in Appendix A.5.

[Backbone Models] We observe that our trained T5 model is generally better
than DialoGPT model, as evidenced by the significant difference in PPL and
Identity Acc between them. [Inference Methods] We observe that Softmax and
TopK can achieve better results than greedy search in this dataset, as
evidenced by their BLEU and Distinct-N scores. The reason is similar to the
conventional generic response problem in open-domain dialogue generation Li et
al. (2016a); Tuan and Lee (2019), since in a DAG, a ($DH$, $x$) pair have
multiple $y$ as references, causing even an ideal probability distribution to
have high entropy. [Loss Functions] We find that ExMATE improves MLE with
better diversity, agility, and identity accuracy, while maintaining similar
fluency scores. This meets our expectation that ExMATE should not deteriorate
the MLE’s ability in training a model while maximizing the potential causal
effect in response prediction. This result empirically shows that the causal
effect can help to increase diversity and predict the turn-taking speaker as
well. Finally, compared to the evaluation results of human written responses
(a hard-to-reach upper bound), current methods still need improvement, except
for diversity scores.

###  6.2 Human Evaluation

We randomly sample 100 dialogues, present each example to three workers on
MTurk and ask them score the three dimensions, agility, coherence, and
informativeness, scaling from 1 to 5. The evaluation form is provided in
Appendix A.3. For each example, we present one shared dialogue history with
two branches and the corresponding machine generated responses or a human
written response. We randomly mix the human written ones to validate if the
human evaluation is reliable to an extent, by anticipating human written ones
will get higher scores. We list the average ratings in Table 5. The model
trained with ExMATE achieves a similar informativeness level as human written
ones, and gets a higher agility rating, which is its main goal. However,
ExMATE can compromises coherence due to the subtraction of a counter example,
which is a natural sentence, in its objective function. The human evaluation
demonstrates the challenge of models to meet human-level quality in
CausalDialogue featured by conversational DAGs, a portion of the diversed
types of flows in the real world.

Model | Coherence | Informativeness | Agility
---|---|---|---
Human | 3.78 | 3.72 | 3.49
MLE | 3.63 | 3.60 | 3.36
ExMATE | 3.59 | 3.74 | 3.40
Table 5: The human evaluation results (scale 1-5, the higher the better) of
models trained on CausalDialogue (MLE, ExMATE), and human written responses
(Human) for reference.

###  6.3 Qualitative Analyses and Discussion

Table 4 shows an example of a shared dialogue history, four different
continuations (case1-4), and responses generated by the same backbone model,
T5, trained with different objectives and inferred with different sampling
methods. We observe that responses produced by MLE+T (TopK), ExMATE+S
(Softmax), ExMATE+T are generally coherent to the conversation, while ExMATE
often produces more diverse and agile responses to different continuation
cases (different $x$). It is notable that other than the improvements, we find
that all the models have three types of issues: mode collapse, semantic
repetition, and identity misplacement. [Mode Collapse] The problem is often-
seen when inferring a model by greedy search, specifically, the predicted
responses often repeat the same phrase such as “I’m not sure”. While tacking
the issue by adopting inference sampling, we conjecture the reason is that in
a DAG, using a typical loss function can learn a probability distribution with
higher entropy. This also demonstrates the need of a new loss function for
training on a conversational DAG dataset. [Semantic Repetition] An example is
the MLE+T response in Table 4 case 4, where “can’t help you ” and “help you
out” have semantic overlaps. This issue can possibly be mitigated by
repetition reduction techniques, such as unlikelihood training Welleck et al.
(2019) in future work. [Identity Misplacement] The problem happens when a
model is confused about its position in a dialogue. For instance, the MLE+T
response in Table 4 case 3 is more like an utterance of speaker Lysithea
instead of Ignatz. This issue might be soothed by existing persona consistent
techniques Li et al. (2016b); Mazaré et al. (2018); Su et al. (2019) for
building a overall good chatbot, while in this work, we focus on proposing a
new dataset to benchmarking on the agility issue.

##  7 Conclusion

In this paper, we presented a new dataset, CausalDialogue, with novel
conversational DAG structure. With experiments on various model setups with a
newly proposed loss, ExMATE, we demonstrate that there is room for improvement
to reach human-level quality, even though ExMATE improves the diversity,
informativeness, and agility. This dataset serves as a testbed for future
research that needs abundant conversation cases, like causal inference and
offline reinforcement learning. Moreover, with the naturally paired metadata,
future work can use this dataset for other tasks, such as speaker prediction
in multi-speaker scenarioes and personalized dialogue generation.

## Limitations

The introduced dataset has a moderate scale, as it is currently designed for
fine-tuning instead of large model pretraining. Our proposed collection scheme
can be futher applied to enlarge the dataset. Moreover, as we focus on
English, the data source has multiple language versions written by experts.
Hence, extending CausalDialogue to multilingual is straightforward. With
reward labeling, the dataset can be more intuitively used for offline RL.
Meanwhile, the dataset includes personality descriptions that can be used for
personalized dialogue generation, even though is not the focus in this paper.
Finally, training a generative model on dialogue domain can require various
computational costs, depending on the aspects such as lengths of input and
output texts and number of model parameters, as well as special designs to
prevent misuses.

## Ethics Consideration

The dataset is based on RPG game in fantasy world with diverse scenarios,
including wars. To match the story background, a model trained on this dataset
might produce war-related words. We manually looked into each example to
meanwhile keep each speaker’s personality and remove utterances that could
potentially cause negative impact, such as violence, bias, and offensive
words.

For the data annotation part and human evaluation part, we utilized the Amazon
Mechanical Turk platform and required workers to have a HIT Approval Rate of
greater than 95% and be located in CA or the US. We pay the annotators over 16
US dollars per hour on average, which is above the highest state minimum wage.
Given our setting, the workers understood the scenarios and agreed that their
annotations will be used for research. The data annotation part of the project
is classified as exempt by Human Subject Committee via IRB protocols.

## Acknowledgement

This work was supported in part by the National Science Foundation under
#2048122 and an unrestricted gift award from Google Deepmind. The views and
conclusions contained in this document are those of the authors and should not
be interpreted as representing the sponsors.

## References

  * Ammanabrolu et al. (2021) Prithviraj Ammanabrolu, Jack Urbanek, Margaret Li, Arthur Szlam, Tim Rocktäschel, and Jason Weston. 2021.  How to motivate your dragon: Teaching goal-driven agents to speak and act in fantasy worlds.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_.
  * Banchs (2012) Rafael E Banchs. 2012.  Movie-dic: a movie dialogue corpus for research and development.  In _ACL_.
  * Chen et al. (2020) Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020.  A simple framework for contrastive learning of visual representations.  In _International conference on machine learning_ , pages 1597–1607. PMLR.
  * Cui et al. (2020) Leyang Cui, Yu Wu, Shujie Liu, Yue Zhang, and Ming Zhou. 2020.  Mutual: A dataset for multi-turn dialogue reasoning.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1406–1416.
  * Cunningham (2021) Scott Cunningham. 2021.  _Causal inference_.  Yale University Press.
  * Danescu-Niculescu-Mizil and Lee (2011) Cristian Danescu-Niculescu-Mizil and Lillian Lee. 2011.  Chameleons in imagined conversations: A new approach to understanding coordination of linguistic style in dialogs.  In _Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, ACL 2011_.
  * Dou et al. (2021) Yao Dou, Maxwell Forbes, Ari Holtzman, and Yejin Choi. 2021.  Multitalk: A highly-branching dialog testbed for diverse conversations.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 35, pages 12760–12767.
  * Ghazvininejad et al. (2018) Marjan Ghazvininejad, Chris Brockett, Ming-Wei Chang, Bill Dolan, Jianfeng Gao, Wen-tau Yih, and Michel Galley. 2018.  A knowledge-grounded neural conversation model.  In _Thirty-Second AAAI Conference on Artificial Intelligence_.
  * Goldberg and Levy (2014) Yoav Goldberg and Omer Levy. 2014.  word2vec explained: deriving mikolov et al.’s negative-sampling word-embedding method.  _arXiv preprint arXiv:1402.3722_.
  * Holland (1986) Paul W Holland. 1986.  Statistics and causal inference.  _Journal of the American statistical Association_ , 81(396):945–960.
  * Imai et al. (2008) Kosuke Imai, Gary King, and Elizabeth A Stuart. 2008.  Misunderstandings between experimentalists and observationalists about causal inference.  _Journal of the royal statistical society: series A (statistics in society)_ , 171(2):481–502.
  * Jaques et al. (2020) Natasha Jaques, Judy Hanwen Shen, Asma Ghandeharioun, Craig Ferguson, Agata Lapedriza, Noah Jones, Shixiang Gu, and Rosalind Picard. 2020.  Human-centric dialog training via offline reinforcement learning.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 3985–4003.
  * Li et al. (2016a) Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and William B Dolan. 2016a.  A diversity-promoting objective function for neural conversation models.  In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 110–119.
  * Li et al. (2016b) Jiwei Li, Michel Galley, Chris Brockett, Georgios Spithourakis, Jianfeng Gao, and William B Dolan. 2016b.  A persona-based neural conversation model.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_.
  * Li et al. (2016c) Jiwei Li, Alexander H Miller, Sumit Chopra, Marc’Aurelio Ranzato, and Jason Weston. 2016c.  Dialogue learning with human-in-the-loop.  _arXiv preprint arXiv:1611.09823_.
  * Li et al. (2016d) Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao. 2016d.  Deep reinforcement learning for dialogue generation.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_ , pages 1192–1202.
  * Li et al. (2017a) Jiwei Li, Will Monroe, Tianlin Shi, Sébastien Jean, Alan Ritter, and Dan Jurafsky. 2017a.  Adversarial learning for neural dialogue generation.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 2157–2169.
  * Li et al. (2017b) Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017b.  Dailydialog: A manually labelled multi-turn dialogue dataset.  In _IJCNLP_.
  * Lison and Tiedemann (2016) Pierre Lison and Jörg Tiedemann. 2016.  Opensubtitles2016: Extracting large parallel corpora from movie and tv subtitles.  In _Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16)_ , pages 923–929.
  * Liu et al. (2016) Chia-Wei Liu, Ryan Lowe, Iulian Serban, Mike Noseworthy, Laurent Charlin, and Joelle Pineau. 2016.  How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_.
  * Lowe et al. (2015) Ryan Lowe, Nissan Pow, Iulian Vlad Serban, and Joelle Pineau. 2015.  The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems.  In _SIGDIAL_.
  * Mairesse et al. (2007) François Mairesse, Marilyn A Walker, Matthias R Mehl, and Roger K Moore. 2007\.  Using linguistic cues for the automatic recognition of personality in conversation and text.  _Journal of artificial intelligence research_ , 30:457–500.
  * Mazaré et al. (2018) Pierre-Emmanuel Mazaré, Samuel Humeau, Martin Raison, and Antoine Bordes. 2018\.  Training millions of personalized dialogue agents.  In _EMNLP_.
  * Narayan-Chen et al. (2019) Anjali Narayan-Chen, Prashant Jayannavar, and Julia Hockenmaier. 2019.  Collaborative dialogue in minecraft.  In _ACL_.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002.  Bleu: a method for automatic evaluation of machine translation.  In _Proceedings of the 40th annual meeting of the Association for Computational Linguistics_.
  * Pasunuru and Bansal (2018) Ramakanth Pasunuru and Mohit Bansal. 2018.  Game-based video-context dialogue.  In _EMNLP_.
  * Pearl (2009) Judea Pearl. 2009.  _Causality_.  Cambridge university press.
  * Poria et al. (2019) Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, and Rada Mihalcea. 2019.  Meld: A multimodal multi-party dataset for emotion recognition in conversations.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 527–536.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _Journal of Machine Learning Research_ , 21(140):1–67.
  * Rameshkumar and Bailey (2020) Revanth Rameshkumar and Peter Bailey. 2020.  Storytelling with dialogue: A critical role dungeons and dragons dataset.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_.
  * Rashkin et al. (2019) Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019.  Towards empathetic open-domain conversation models: A new benchmark and dataset.  In _ACL_.
  * Ritter et al. (2011) Alan Ritter, Colin Cherry, and Bill Dolan. 2011.  Data-driven response generation in social media.  In _Empirical Methods in Natural Language Processing (EMNLP)_.
  * Roller et al. (2021) Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Eric Michael Smith, Y-Lan Boureau, et al. 2021.  Recipes for building an open-domain chatbot.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 300–325.
  * Serban et al. (2016) Iulian V Serban, Alessandro Sordoni, Yoshua Bengio, Aaron Courville, and Joelle Pineau. 2016.  Building End-to-End Dialogue Systems Using Generative Hierarchical Neural Network Models.  In _Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI-16)_.
  * Sordoni et al. (2015) Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Margaret Mitchell, Jian-Yun Nie, Jianfeng Gao, and Bill Dolan. 2015.  A neural network approach to context-sensitive generation of conversational responses.  In _Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_.
  * Su et al. (2019) Feng-Guang Su, Aliyah R Hsu, Yi-Lin Tuan, and Hung-Yi Lee. 2019.  Personalized dialogue response generation learned from monologues.  In _INTERSPEECH_.
  * Thoppilan et al. (2022) Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. 2022.  Lamda: Language models for dialog applications.  _arXiv preprint arXiv:2201.08239_.
  * Tuan et al. (2019) Yi-Lin Tuan, Yun-Nung Chen, and Hung-Yi Lee. 2019.  Dykgchat: Benchmarking dialogue generation grounding on dynamic knowledge graphs.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_.
  * Tuan and Lee (2019) Yi-Lin Tuan and Hung-Yi Lee. 2019.  Improving conditional sequence generative adversarial networks by stepwise evaluation.  _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , 27(4):788–798.
  * Tuan et al. (2020) Yi-Lin Tuan, Wei Wei, and William Yang Wang. 2020.  Knowledge injection into dialogue generation via language models.  _arXiv preprint arXiv:2004.14614_.
  * Urbanek et al. (2019) Jack Urbanek, Angela Fan, Siddharth Karamcheti, Saachi Jain, Samuel Humeau, Emily Dinan, Tim Rocktäschel, Douwe Kiela, Arthur Szlam, and Jason Weston. 2019.  Learning to speak and act in a fantasy text adventure game.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _Advances in neural information processing systems_ , 30.
  * Verma et al. (2022) Siddharth Verma, Justin Fu, Sherry Yang, and Sergey Levine. 2022.  CHAI: A CHatbot AI for task-oriented dialogue with offline reinforcement learning.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 4471–4491, Seattle, United States. Association for Computational Linguistics.
  * Vinyals and Le (2015) Oriol Vinyals and Quoc Le. 2015.  A neural conversational model.  _arXiv preprint arXiv:1506.05869_.
  * Wang et al. (2013) Hao Wang, Zhengdong Lu, Hang Li, and Enhong Chen. 2013.  A dataset for research on short-text conversations.  In _Proceedings of the 2013 conference on empirical methods in natural language processing_ , pages 935–945.
  * Welleck et al. (2019) Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. 2019.  Neural text generation with unlikelihood training.  In _International Conference on Learning Representations_.
  * Yu et al. (2020) Dian Yu, Kai Sun, Claire Cardie, and Dong Yu. 2020.  Dialogue-based relation extraction.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 4927–4940.
  * Zhang et al. (2018) Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018.  Personalizing dialogue agents: I have a dog, do you have pets too?  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_.
  * Zhang et al. (2020) Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and William B Dolan. 2020.  Dialogpt: Large-scale generative pre-training for conversational response generation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations_ , pages 270–278.
  * Zhao et al. (2017) Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. 2017.  Learning discourse-level diversity for neural dialog models using conditional variational autoencoders.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 654–664.
  * Zhu et al. (2020) Qingfu Zhu, Weinan Zhang, Ting Liu, and William Yang Wang. 2020.  Counterfactual off-policy training for neural dialogue generation.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 3438–3448.

##  Appendix A Appendix

Speaker |  Profile Excerpt |  Example Utterances
---|---|---
Byleth |  Byleth has a very subdued personality and rarely expresses emotion. |  \- It’s all right. // - Not really. // - I’m sorry.
Edelgard |  Edelgard holds herself with a dignified air, but full of melancholy and solemn wistfulness. |  \- That’s exactly right. There will no longer be lords who inherently rule over a particular territory. // - Perhaps not. Still, here you are. Maybe I can trust you with this…
Claude |  Claude is described as easygoing on the surface, but has a side that forces others to keep their guard around him. |  \- Huh? Are you actually reading? I thought you hated studying. // - Was that story really worth bawling your eyes out over?
Table 6: In CausalDialogue dataset, some speakers profiles excerpts and their
example utterances in conversations.

###  A.1 Speaker Profiles

Table 6 provides a few examples of the speakers’ profiles and utterances.

###  A.2 Data Expansion Details

#### Initial Dialogue Selection.

We first randomly select $m$ dialogues with replacement from the Ori.-2S
partition, which is of higher quality after our manual inspection. This can
result in more stable quality when doing crowd-sourcing. For each sampled
dialogue, we randomly select a start time stamp $t$ from $Poisson(\lambda=1)$.
Next, we adjust the sampled time stamp $t$ to make sure it lies in an
appropriate point to continue the dialogue by $t^{*}=\max(\min(t+2,L),2)$,
where $L$ is the maximum time stamp of this dialogue. For each time stamp, if
the original dialogue has multiple possible nodes, we select one randomly from
a uniform distribution. This process results in $m$ initial dialogues $D_{0}$
with various lengths (at least two utterances) for expansion.

#### Expansion Collection.

Each initial dialogue $D_{0}$ along with the continuing speaker profile is
presented to $n$ workers on MTurk to write the next utterance. The new
continued dialogues $D_{1}$ will then be presented to another 1-2 workers,
decided by $p$%, playing as another speaker to gather another round of
responses. This results in about $mn((1+p)^{T}-1)/p$ new utterances for data
expansion, where $T$ is the number of iterations. Our expansion data is set
with $m=1200$, $n=3$, $p=0.2$, and $T=3$. This setting results in about 13,000
written utterances.

###  A.3 Human Annotations

#### Interface - Data Expansion.

We design two user interfaces to launch on MTurk for the first stage and the
remaining stages separately of the data expansion process. The interface used
for the remaining stages is shown in Figure 5. We include detailed
instructions about the step-by-step works, examples, and requirements to obey.
We put some information into a button to reduce cognitive burden when writing
for multiple hits.

#### Interface - Human Evaluation.

Our used human evaluation form is shown in Figure 6.

#### Setup and Payments.

We collect the expanded dataset and evaluate generated responses via MTurk, a
crowdsourcing platform. We obtained consent from workers by showing them the
study purpose before they agree to do the annotations. We set additional
restrictions of location to United States and Canada. We pay the annotators
from 16-18 US dollars per hour according to the difficulty of the collection
stage (remaining stages are more difficult than the first stage). The payments
are made to be higher than the law’s minimum wage 15 US dollars per hour in
California in 2022 and 15.5 US dollars per hour in 2023, which are the highest
among the US states.

###  A.4 Evaluation Metrics

Here we discuss more about our selection of evaluation metrics.

#### Fluency.

The predicted next utterance should be both coherent to the previous turn and
consistent with the dialogue history. We evaluate the extent of coherence by
perplexity and reference-based metric BLEU Papineni et al. (2002). For nodes
with multiple childs we use multiple references when computing BLEU metrics.
Although that BLEU may not be well correlated with human intuition in
conversation Liu et al. (2016), we use it for reference as it is still widely
used in dialogue generation. The perplexity (PPL) is considered to be the less
the better, whereas BLEU is the higher the better.

#### Diversity.

A dialog model can suffer from the generic issue that given different dialogue
history and previous turn, the predicted utterance is similar, such as “I’m
sorry”. We adopt distinct-N scores (Dist1 and Dist2) to evaluate this
dimension by considering the percentage of distinct n-grams within the total
number of n-grams in the corpus-level Li et al. (2016a). However, the
distinct-N scores are not always the higher the better. We can think about
this in a intuitive example, if we randomly sample words from a uniform
distribution, the distinct-N score can be high but meaningless. We anticipate
a good distinct-N score is in a similar range as the score evaluated on human
written responses.

###  A.5 Evaluate Human Responses

The PPL on human written responses are evaluated by an oracle that will
predict a uniform distribution over all human written responses $y$ given the
same $(DH,x)$. The BLEU scores on human written responses are evaluated on
data examples with multiple possible responses and the response to be
evaluated is hold out from the reference set. Otherwise, the BLEU scores will
be 100 since the response to be evaluated is within the reference set.

###  A.6 Experiment Details

#### Model architecture.

We use DialoGPT-small with 117M parameters and T5-base with 250M parameters.
DialoGPT model is based on the GPT model architecture (a transformer decoder)
but pretrained on conversation-like dataset such as Reddits. T5 model uses the
transformer encoder-decoder architecture and is pretrained on web-extracted
text from Common Crawl, which is a publicly-available web archive of scraped
HTML files. The maximum tokens allowed as the input are $256$.

#### Hyperparameters.

For hyperparameter search, we tried the learning rate from
$\\{$5e-5,2e-5,1e-5$\\}$ and the batch size times gradient accumulation steps
from $\\{$32,64,128$\\}$. We found out that using a learning rate 1e-5 and
batch size 64 can generally fine-tuning a model well with different learning
algorithms in our experiments. We train each model with different combinations
of setups for single run.

#### Data preprocessing.

For data preprocessing, we have tried to utilize the original case and
punctuation, transform all words into lower case, or meanwhile remove all
punctuation.

#### Computation Resources.

Each model is train on one Titan RTX or one RTX A6000 and costs around five
hours.

Figure 5: Screenshots of the interface we show the annotators to write
responses for the remaining stages data expansion. We gave detailed
instruction, requirements, and good/bad examples for reference. Figure 6:
Screenshot of the interface we show the annotators to evaluate generated
responses. We gave instruction with five questions each hit.


# Breathing solitons induced by collision in dipolar Bose-Einstein condensates

Peng Gao1,2 Xin Li1,2 Zhan-Ying Yang1,2,3<EMAIL_ADDRESS>Wen-Li Yang1,2,3,4
Su Yi5,6 1School of Physics, Northwest University, Xi’an 710127, China
2Shaanxi Key Laboratory for Theoretical Physics Frontiers, Xi’an 710127, China
3Peng Huanwu Center for Fundamental Theory, Xi’an 710127, China  4Institute of
Modern Physics, Northwest University, Xi’an 710127, China  5CAS Key Laboratory
of Theoretical Physics, Institute of Theoretical Physics, Chinese Academy of
Sciences, Beijing 100190, China  6School of Physics & CAS Center for
Excellence in Topological Quantum Computation, University of Chinese Academy
of Sciences, Beijing 100190, China

###### Abstract

We numerically study the breathing dynamics induced by collision between
bright solitons in the one-dimensional Bose-Einstein condensates with strong
dipole-dipole interaction. This breathing phenomenon is closely related to the
after-collision short-lived attraction of solitons induced by the dipolar
effect. The initial phase difference of solitons leads to the asymmetric
dynamics after collision, which is manifested on their different breathing
amplitude, breathing frequency, and atom number. We clarify that the asymmetry
of breathing frequency is directly induced by the asymmetric atom number,
rather than initial phase difference. Moreover, the collision between
breathing solitons can produce new after-two-collision breathing solitons,
whose breathing amplitude can be adjusted and reach the maximum (or minimum)
when the peak-peak (or dip-dip) collision happens.

##  I Introduction

The condensates of bosonic gases with large magnet dipole moments have been
created Griesmaier-2005 ; Beaufils-2008 ; Lu-2011 ; Tang-2014 ; Aikawa-2012 ;
Stuhler-2005 . Contrary to the contact interaction between atoms, the dipole-
dipole interaction is long-range and anisotropic, so it brings many novel
influences on characters of condensates and the localized structures including
solitons Dauxois-2002 ; Giovanazzi-2002 ; Sinha-2007 ; Kevrekidis-2007 ;
Koch-2008 ; Lahaye-2008 ; Lahaye-2009 ; Kadau-2016 ; Ferrier-Barbut-2016 . In
the dipolar Bose-Einstein condensates (BECs) described by the Gross-Pitaevskii
equation, many efforts have been made to study these localized structures.
Two-dimensional stable bright and vortex solitons were generated in the cases
without external traps, which benefited from the long-range feature of dipolar
effect Pedri-2005 ; Lashkin-2007 ; the formation of anisotropic bright and
vortex solitons was closely related to its anisotropic feature
Raghunandan-2015 ; Tikhonenkov-2008 ; Yi-2006 . Some new localized waves have
been also found in one-dimensional dipolar BECs, like soliton molecules and
the dark soliton with ripples Baizakov-2015 ; Turmanov-2015 ; Pawowski-2015 ;
Edmonds-2016 . Meanwhile, the interaction between dipolar solitons exhibited
various dynamical behaviors. Two solitons may merge into a breathing wave when
the dipolar interaction was considered Cuevas-2009 ; Pedri-2005 ; Edmonds-2017
; they could rebound from each other in some other cases Cuevas-2009 ;
Eichler-2012 . When the initial distance between two solitons is small, two
out-of-phase solitons could form a bound state with a stable oscillating
frequency Bland-2015 ; Baizakov-2015 ; Pawowski-2015 . These phenomena
indicate that the interaction between solitons becomes inelastic under the
dipolar effect.

Recently, breathing solitons also draw the attention of scientists due to
their character of stable breathing. Their dynamical features and generation
mechanism have been discussed in real physical systemsPeng-2019 ; Bao-2018 ;
Lucas-2017 ; Bao-2016 ; Yu-2017 ; Bao-2015 ; Matsko-2012 . As we know, the
breathing dynamics of solitons cannot be induced after bright solitons collide
in one-dimensional non-dipolar BECs. However, the breathing phenomenon appears
after the collision between dipolar solitons. It provides a new way to
generate and control breathing solitons in dipolar BECs so is worthy of more
attention.

In this paper, we numerically study the breathing character of bright solitons
induced by their collision in the one-dimensional BECs with strong dipole-
dipole interaction. From two solitons with identical density distribution,
they have different breathing amplitude, breathing frequency, and atom number
after collision, under different initial phase difference of them. We
demonstrate that the asymmetry of breathing frequency is directly related to
the asymmetric distribution of after-collision atom number, rather than the
initial phase difference. When we make the peaks of two breathing solitons
collide, their breathing amplitude gets larger after the collision, and
conversely a dip-dip collision generates solitons with smaller breathing
amplitude. Thus, one can strengthen or weaken the breathing character by the
collision between breathing solitons. Importantly, with the breathing
amplitude increasing, the derived solitons have lower values of velocity and
mean kinetic energy. It manifests that there is a short-lived attraction of
solitons after collision, which is the major cause of breathing phenomenon.

##  II Model of the dipolar BECs

The dynamics of dipolar BECs can be discribed by the dipolar Gross-Pitaevskii
model Kevrekidis-2007 ; Baranov-2008 ; Lahaye-2009 ,

$\displaystyle\begin{split}i\hbar\frac{\partial\Psi}{\partial
t}=-\frac{\hbar^{2}}{2m}\nabla^{2}\Psi+V(\mathbf{r})\Psi+g|\Psi|^{2}\Psi+\Phi_{\rm{dd}}(\mathbf{r},t)\Psi,\end{split}$
(1)

where $\Psi(\mathbf{r},t)$ is the mean-field wave function of condensates. The
strength of contact interaction is $g=4\pi\hbar^{2}a_{s}/m$ with the atom mass
$m$ and the $s$-wave scattering length $a_{s}$. The external potential
$V(\mathbf{r})$ provides a trap where the cloud is confined, and it can be
assumed as a harmonic form,
$V(\mathbf{r})=m\omega_{\perp}^{2}(y^{2}+z^{2})/2$, where $\omega_{\perp}$ is
the transverse trapping frequency. Here, the axial trap is neglected. The
nonlocal dipolar potential is

$\Phi_{\rm{dd}}(\mathbf{r},t)=\int\frac{d^{2}(1-3\cos^{2}\theta_{rd})}{|\mathbf{r}-\mathbf{r}^{\prime}|^{3}}|\Psi(\mathbf{r}^{\prime},t)|^{2}d\mathbf{r}^{\prime},$

where $d$ is the magnetic dipole moment and $\theta_{rd}$ is the angle between
the vector joining interacting particles and the dipole direction.

In the transverse external trap $V(\mathbf{r})$, the condensates are
compressed to be cigar-shaped and can be considered as quasi-one-dimensional
dipolar BECs. Thus, the 3D model (1) can be reduced into a 1D model. To this
end, an effective way is to assmue the wave function
$\Psi(\mathbf{r},t)=\tilde{\psi}(x,t)\psi_{\perp}(y,z)$, where
$\psi_{\perp}(y,z)=\exp[-(y^{2}+z^{2})/2l_{\perp}^{2}]/(l_{\perp}\sqrt{\pi})$
and $l_{\perp}=\sqrt{\hbar/m\omega_{\perp}}$. By the integration over
transverse directions, the effective 1D potential of dipole-dipole interaction
is obtained,

$\tilde{\Phi}_{\rm{dd}}(x,t)=-\frac{\alpha
d^{2}}{2l_{\perp}^{3}}\int^{+\infty}_{-\infty}\tilde{R}(x-x^{\prime})|\tilde{\psi}(x^{\prime},t)|^{2}dx^{\prime},$

where $\alpha=(1+3\cos 2\theta)/4$ has the range from $-1/2$ to $1$, and
$\theta$ is the angle between dipole and $x$ directions. The nonlocal response
function has the form of $\tilde{R}(x)=(1+2u^{2})\exp(u^{2}){\rm
erfc}(|u|)-2{\pi}^{-1/2}|u|$, where $u=x/\sqrt{2}l_{\perp}$ and erfc is the
complementary error function. (The detailed calculation of 1D dipolar
potential was illustrated in Ref. Deuretzbacher-2010 .) Then, by the
transformation

$\displaystyle\begin{split}t=\frac{2}{\omega_{\perp}}T,\quad
x=\sqrt{2}l_{\perp}X,\quad\tilde{\psi}=\frac{1}{2\sqrt{|a_{s0}|}}\psi,\end{split}$

we can transform the 1D dipolar model into a dimensionless one. For
convenience, the labels $X$ and $T$ are severally replaced by $x$ and $t$, and
the final dimensionless model becomes

$\displaystyle\begin{split}i{\psi}_{t}&+\frac{1}{2}{\psi}_{xx}+g_{c}|{\psi}|^{2}{\psi}\\\
&+g_{d}{\psi}{\int_{-\infty}^{+\infty}R(x-x^{\prime})|{\psi(x^{\prime},t)}|^{2}dx^{\prime}}=0,\end{split}$
(2)

where the responce function is

${R}(x)=(1+2x^{2})\exp(x^{2}){\rm erfc}(|x|)-2{\pi}^{-1/2}|x|.$

In this equation, $g_{c}=-{a_{s}}/{|a_{s0}|}$ and $g_{d}={\sqrt{\pi}\alpha
a_{d}}/{2|a_{s0}|}$ scale the strength of contact and dipolar interactions,
respectively, where $a_{d}=md^{2}/\hbar^{2}$ is the characteristic dipole
length and $a_{s0}$ is the background value of scattering length. The
positive(or negative) values of $g_{c}$ and $g_{d}$ indicates the attractive
(or repulsive) interaction.

In our numerical simulations, the dipolar condensates of 164Dy atoms are
considered, which have strong dipole-dipole and weak contact interactions. By
applying an external magnetic field and Feshbach resonance technique, the
condition $a_{s}\rightarrow 0$ can be satisfied for the case of strongly
dipolar interaction. We have $m=2.7\times 10^{-25}{\rm kg}$,
$\omega_{\perp}=2\pi\times 250{\rm Hz}$, and $a_{s0}=1.17\times 10^{-11}{\rm
m}$, which leads to $l_{\perp}=1.25{\rm\mu m}$. The units of space and time
coordinates are respectively $x_{\rm u}=\sqrt{2}l_{\perp}=1.77{\rm\mu m}$ and
$t_{\rm u}=2/\omega_{\perp}=1.27{\rm ms}$. The realistic atom number is
$\tilde{N}=N\sqrt{2}l_{\perp}/4|a_{s0}|=37718N$, where $N$ is the
dimensionless one. The dipole moment is
$d=10\mu_{B}\sqrt{\mu_{0}/4\pi}=0.02\sqrt{\hbar\omega_{\perp}l_{\perp}^{3}}$,
where $\mu_{B}$ is the Bohr magneton and $\mu_{0}$ is the permeability of
vacuum, so $a_{d}=5.29\times 10^{-10}{\rm m}$ Lu-2011 . The dipolar strength
$g_{d}$ can be changed by adjusting $\alpha$. For examples, we can set
$g_{d}=20$, $30$, and $40$ to consider the cases of $\alpha=1/4$, $3/8$, and
$1/2$, respectively.

Figure 1: (Color online) (a) Density evolution plot of initial condition (3)
when $N=1$, $x_{0}=5$, $v=2$, $\varphi=0$, $g_{c}=0$, and $g_{d}=20$. (b) Its
evolution of maximal density of two solitons. The black point line denotes the
maximal density of initial soliton. The red and blue dashed lines denotes the
time of peak and dip, respectively. The inset plot shows the density profiles
of the initial soliton (black curve), the dip (blue curve) and peak (red
curve) of breathing soliton. (c) Density evolution plot of initial condition
(3) when $\varphi$ is changed into $\pi/2$. (d) Its evolution of maximal
density of Soliton A (dashed curve) and Soliton B (solid curve).

##  III Breathing dynamics of solitons induced by collision

Figure 2: (Color online) Influence of phase difference $\varphi$ on the (a)
breathing amplitude $A_{br}$, (b) breathing frequency $\Omega_{br}$, and (c)
after-collision atom number $N_{br}$ of solitons, when $g_{d}=20$ (red),
$g_{d}=30$ (blue), and $g_{d}=40$ (black). The open and solid circles denote
the results of Soliton A and B, respectively.

The bright solitons in the model (2) have been numerically studied, which
contains the cases of dipolar and local interactions with identical or
opposite types Cuevas-2009 ; Edmonds-2017 . Here, we use the split-step Crank-
Nicolson method and the imaginary-time propagation method Muruganandam-2003 ;
Muruganandam-2009  to obtain the numerical profiles of bright solitons in this
model. They are denoted as $\psi_{0}(x)$, so the atom number of soliton is
$N=\int_{-\infty}^{+\infty}|{\psi_{0}(x)}|^{2}dx$. With the chemical potential
$\mu$, the bright soliton solution in the model (2) can be written as
$\psi_{bs}(x,t)=\psi_{0}(x)\exp(-i{\mu}t)$. To study the collision between the
solitons, we have the initial condition of two solitons with velocity and
phase difference,

$\displaystyle\begin{split}\psi(x,0)=\psi_{0}(x+x_{0})e^{ivx}+\psi_{0}(x-x_{0})e^{-ivx+i{\varphi}},\end{split}$
(3)

where $x_{0}>0$, $v\geq 0$, and $\varphi$ are the initial offset, velocity,
and phase difference of solitons, respectively. Here, we call the initial
soliton on the left-hand side as Soliton A, while the one on the right-hand
side as Soliton B.

We consider the cases of strongly dipolar interaction ($g_{d}=20$) and neglect
the contact one ($g_{c}=0$). The set of initial parameters is $N=1$,
$x_{0}=5$, $v=2$, and $\varphi=0$, which corresponds to the collision between
two in-phase solitons. By the split-step Fourier method Yang-book , its
evolution plot of density $|\psi(x,t)|^{2}$ is shown in Fig. 1 (a). The two
solitons become breathing with a stable frequency after they collide. In the
cases without collision, the breathing mode of dipolar solitons has been
described by the variational approximation in three-dimensional BECs Yi-2001 ;
Yi-2002 . Here, the breathing phenomenon induced by soliton collision provides
a new way to generate breathing solitons in dipolar BECs. It is related to the
dipolar effect, and it doesn’t happen in the cases without dipolar effect
($g_{d}=0$), where the interaction between solitons are elastic. Note that the
solitons changes their positions on the space coordinate after the collision,
namely, the after-collision Soliton A (or B) locates on the right-hand (or
left-hand) side. For the time $t$, we define the space coordinate where the
soliton has the maximal density as $x_{\rm m}(t)$. Thus, the maximal density
of solitons can be written as $|\psi(x_{\rm m},t)|^{2}$. Their evolutions of
$|\psi(x_{\rm m},t)|^{2}$ are shown in Fig. 1 (b). The density profiles of
initial soliton and the breathing solitons at the time of peak and dip are
shown in the inset. The peak has a high and narrow profile while the dip has a
short and wide profile, and the initial soliton has a profile in between.
Then, we change the phase difference into $\varphi=\pi/2$, which leads to an
asymmetric collision. Its evolution plots of density and maximal density of
solitons are shown in Fig. 1 (c) and (d), respectively. The two solitons have
different breathing dynamics after the collision. The breathing amplitude of
Soliton B is higher than the one of Soliton A, though both of them are smaller
than the one induced by in-phase solitons in Figs. 1 (a) and (b).

Figure 3: (Color online) (a) Density evolution plot of initial condition (3)
when $N=1$, $x_{0}=0.7$, $v=-2$, and $\varphi=0$. It describes a process that
two solitons move away from each other. The inset is a part of Fig. 1 (a),
describing the short-lived attracting process after collision. (b) Influence
of phase difference $\varphi$ on the breathing frequency $\Omega_{br}$ of
solitons, when $\Delta N=0$ (black) and $\Delta N=0.12$ (red). (c) Influence
of atom number difference $\Delta N$ on the breathing frequency $\Omega_{br}$
when $\varphi=0$. The open and solid circle denote the results of Soliton A
and B, respectively.

From Fig. 1, one can find that the phase difference has an obvious influence
on the breathing dynamics of solitons. The breathing character of solitons can
be depicted by its breathing amplitude $A_{br}$ and frequency $\Omega_{br}$.
We define the time when peak and dip appear as $t_{\rm peak}$ and $t_{\rm
dip}$, respectively. In the numerical simulation, the breathing period
$T_{br}$ can be measured by the average value of several time differences of
two adjacent peaks of $|\psi(x_{\rm m},t)|^{2}$, i.e., the average value of
$t_{\rm peak}^{(n)}-t_{\rm peak}^{(n+1)}$. Thus, the breathing frequency is
$\Omega_{br}=2\pi/T_{br}$. The breathing amplitude $A_{br}$ can be measured by
the $|\psi(x_{\rm m},t)|^{2}$ difference at a pair of adjacent dip and peak,
i.e., $A_{br}=|\psi(x_{\rm m},t_{\rm peak})|^{2}-|\psi(x_{\rm m},t_{\rm
dip})|^{2}$. The atom number of solitons after collision is denoted by
$N_{br}$. We illustrate the change of $A_{br}$, $\Omega_{br}$, and $N_{br}$
under the different phase difference $\varphi$ and dipolar strength $g_{d}$ in
Figs. 2 (a), (b), and (c), respectively. When $\varphi\neq 0$ and
$\varphi\neq\pi$, the breathing characters of Soliton A and B are different
after they collide. When $0<\varphi<\pi$, both of $A_{br}$, $\Omega_{br}$, and
$N_{br}$ of Soliton A are smaller than the ones of Soliton B; when
$\pi<\varphi<2\pi$, the opposite results are obtained. These results are
obviously different from the cases without dipolar effect. When $\varphi=0$ or
$\varphi=\pi$, the two solitons have identical values of $A_{br}$,
$\Omega_{br}$, and $N_{br}$. Obvious decreases of $A_{br}$ happen when
$\varphi$ is near $\pi$, which indicates that the breathing character is
weakened in these cases. Meanwhile, with the increase of $g_{d}$, both of
$A_{br}$ and $\Omega_{br}$ increase, and the atom number difference
$|N_{br}-1|$ also increases. Note that the dependence of $N_{br}$ on $\varphi$
in Fig. 2 (c) is quite similar to the one when two non-dipolar solitons
collide on a narrow barrier Martin-2012 ; Helm-2012 . Thus, the dipolar effect
may have a barrier-like behavior for soliton collision.

By carefully observing the soliton collision in Fig. 1 (a), there exists a
short-lived attracting process between solitons after collision [see the inset
of Fig. 3 (a)]. The long-range attraction induced by dipolar effect leads to
the widening of solitons and the subsequent breathing dynamics. It indicates
that the breathing solitons can be also generated from two solitons away from
each other. Its initial condition is still Eq. 3, and we set $v<0$ to make
solitons move in the opposite directions. When $N=1$, $x_{0}=0.7$, $v=-2$, and
$\varphi=0$, the evolution plot is shown in Fig. 3 (a). The two solitons
become breathing after they seperate. Note that the after-collision atom
number difference $\Delta N_{br}$ is identical with the initial one $\Delta N$
and is not influenced by the initial phase difference $\varphi$. Thus,
compared with soliton collisions, this way to generate breathing solitons
allows us to severally study the influence of $\varphi$ and $\Delta N_{br}$,
both of which can be set in the initial condition. We set the initial atom
number of two solitons are $1+\Delta N/2$ and $1-\Delta N/2$, namely, their
atom number difference is $\Delta N$. The influence of $\varphi$ on
$\Omega_{br}$ when $\Delta N=0$ and $\Delta N=0.12$ is shown in Fig. 3 (b).
The value of $N_{br}$ is measured at $t=15$. Under different $\varphi$, when
$\Delta N=0$, the breathing frequency $\Omega_{br}$ of two solitons is almost
identical; when $\Delta N=0.12$, their breathing frequency $\Omega_{br}$ is
different and has an almost constant difference $\Delta\Omega_{br}$ between
solitons. Thus, the direct inducement of asymmetric breathing frequency
between Solitons A and B is their atom number difference, rather than their
phase difference. Also, when $\varphi=0$, the dependence of $\Omega_{br}$ on
$\Delta N$ is shown in Fig. 3 (c). The difference of initial atom number
$\Delta N$ has a directly proportional influence on the difference of
breathing frequency $\Omega_{br}$. These results indicate that there is a
positive correlation between $\Delta\Omega_{br}$ and $\Delta N$ (or $\Delta
N_{br}$), but $\Delta\Omega_{br}$ is not influenced by $\varphi$. Besides, it
is found that the breathing amplitude $A_{br}$ is impacted by both of
$\varphi$ and $\Delta N$, so their results are not shown in this paper. Now,
we look back on the process that two solitons collide and become breathing
ones. The change of $\varphi$ leads to the asymmetry of after-collision atom
number $N_{br}$, and then the asymmetry of $N_{br}$ leads to the asymmetry of
breathing frequency $\Omega_{br}$. However, why the phase difference can cause
the asymmetry of $N_{br}$ is still an open problem for the collision of
dipolar solitons.

Figure 4: (Color online) (a) Kinetic energy evolution of soliton collision
when $\varphi=0$ (black solid curve) and $\varphi=\pi$ (red dashed curve). (b)
Phase evolution plot of Soliton A in Fig. 1 (a), where the contribution of
momentum is neglected. The circle and cross signs denote the positions of
peaks and dips, respectively. The parameters are $N=1$, $x_{0}=5$, $v=2$,
$g_{c}=0$, and $g_{d}=20$.

We recall that the velocity of breathing soliton is smaller than the initial
soliton’s (namely $v_{br}<v$). It may be related to the short-lived attracting
process mentioned above. The attraction makes the solitons wider and makes the
soliton centers closer to each other than non-dipolar cases, so they have the
smaller velocity than the initial solitons. The widened solitons then become
narrow and finally form the breathing ones. This process can be also
understood by the evolution of kinetic energy as shown in Fig. 4 (b). The
kinetic energy of breathing solitons is fluctuating and has a lower mean value
than the initial solitons’, which agrees well with the comparison of their
velocity. Besides, it is known from Fig. 2 (a) that the two derived breathing
solitons when $\varphi=0$ have larger breathing amplitude than the ones when
$\varphi=\pi$. The evolution of their kinetic energy is compared in Fig. 4
(b). The kinetic energy when $\varphi=\pi$ has smaller fluctuation but nearly
the same maximal amplitude than the one when $\varphi=0$. Thus, the former has
a higher mean value than the latter, which indicates that the out-of-phase
breathing solitons have a larger velocity than the in-phase one. We speculate
that, with the breathing amplitude increasing, the breathing solitons have
smaller velocity.

Finally, to analyze the fluxion of atoms in breathing solitons, we also study
the evolution of their phase $\phi$. The atomic velocity is the derivative of
phase with respect to space coordinate, i.e., $v_{\rm
atom}=\partial\phi/\partial x$. The value of phase is calculated by $\phi={\rm
Arg}[\psi\exp({-iv_{br}x})]$ to neglect the contribution of momentum, where
$v_{br}$ is the velocity of breathing soliton. The phase evolution of Soliton
A in Fig. 2 (a) is shown in Fig. 4 (a), and we have $v_{br}=1.74$ here. When
$t=t_{\rm peak}$ or $t=t_{\rm dip}$, the phase distribution of soliton is
almost constant, which indicates the motionless atoms. When $t=t_{\rm
peak}+T_{br}/4$, the phase has smaller value at the soliton center and larger
value at two sides, which indicates the expansion of soliton. When $t=t_{\rm
dip}+T_{br}/4$, the phase has larger center value and smaller two-side value,
which indicates the compression of soliton. Thus, in the breathing process,
the atoms of soliton repeatedly gather and then disperse with a stable
frequency. Actually, the colliding process of two solitons is similar to the
movement of a soliton away from the external trap formed by another soliton.
It indicates that the breathing phenomenon also appears when the soliton moves
in a trap, which is observed in our numerical simulations. In our opinion, the
different parts of soliton distributing in trap have different acceleration.
Under the effect of acceleration difference, the soliton will become wider or
narrower with the time evolution, and finally its breathing behavior
manifests.

Figure 5: (Color online) (a) Density evolution plot for the interaction
between two after-one-collision solitons. Two solitons with a larger breathing
amplitude appear after a peak-peak collision happens. (b) Kinetic energy
evolution of soliton collision in the cases of peak-peak (black solid curve)
and dip-dip (red dashed curve) collisions. (c) Influence of initial offset on
the breathing amplitude of after-two-collision solitons. The dashed black line
denotes the breathing amplitude of after-one-collision solitons.

##  IV Interaction between breathing solitons

Since the collision between dipolar solitons can induce their breathing
character, we pay attention to the influence of collision between breathing
solitons on their breathing dynamics. The initial condition is changed into

$\displaystyle\begin{split}\psi(x,0)=\psi_{br}(x+x_{0})e^{ivx}+\psi_{br}(x-x_{0})e^{-ivx},\end{split}$
(4)

where $\psi_{br}(x)$ is the dip profile of breathing solitons in Fig. 1 (a),
and the velocity $v$ is still set as $2$. Note that the dip profile is
filtered by a super-Gaussian function to weaken the radiation waves outside
the solitons. First, we focus on the cases that the collision happens at the
peaks of two breathing solitons. To this end, we have $x_{0}=(n+1/2)vT_{br}$
where $n$ is an integer. The period is measured as $T_{br}=0.98$ in the
numerical evolution. The density evolution of peak-peak collision between two
breathing solitons is shown in Fig. 5 (a) when $n=7$. Interestingly, two
breathing solitons with larger breathing amplitude appear. (As they are
produced from original solitons by two collisions, they are called "after-two-
collision breathing solitons".) Also, it is found that two breathing solitons
after dip-dip collision has very small breathing amplitude. We compare the
kinetic energy evolution in the cases of peak-peak and dip-dip collisions in
Fig. 5 (b). Compared with the initial breathing solitons, the derived ones
have larger (or smaller) breathing amplitude after peak-peak (or dip-dip)
collision. Nevertheless, they have similar values of maximal kinetic energy,
which coincides with the results shown in Fig. 4 (b). It indicates the
velocity of breathing solitons after peak-peak collision is larger than the
one after dip-dip collision. Meanwhile, it is deduced that the collision at
different parts of breathing solitons can produce solitons with different
breathing dynamics. Thus, we study the influence of position where they
collide on the breathing amplitude of after-two-collision solitons, as shown
in Fig. 5 (c). The horizontal variable is $n_{x}=x_{0}/(vT_{br})$. For an
arbitrary integer $n$, when $n_{x}=n$ a peak-peak collision happens, and when
$n_{x}=n+1/2$ a dip-dip collision happens. From the plot, a dip-dip collision
makes the breathing amplitude of after-two-collision solitons approach
minimum, while a peak-peak collision makes it approach maximum. Thus, one can
use the peak-peak (or dip-dip) collision to generate solitons with a larger
(or smaller) breathing amplitude.

Figure 6: (Color online) Density evolution plot of (a) after-one-collision
soliton, (b) after-two-collision soliton, (c) after-three-collision soliton,
(d) after-four-collision soliton, and (e) the peak-peak collision between two
after-four-collision solitons.

Similarly, one can go on the peak-peak collisions for more times to generate
the solitons with a larger breathing amplitude. The density evolutions of
after-one-collision, after-two-collision, after-three-collision, and after-
four-collision solitons are shown in Fig. 6 (a-d). With the increasing number
of collisions, the breathing behavior of solitons become more and more
obvious, and their breathing period is longer and longer. For the after-four-
collision soliton in Fig. 6 (d), its dip has two maximums, which is different
from other solitons. When two after-four-collision solitons collide, its
density evolution is shown in Fig. 6 (e), where three solitons appear after
the collision. Thus, a common after-five-collision soliton cannot be obtained
in this way. The appearance of splitting could be related to the two-maximum
structure of after-four-collision soliton’s dip, as the two maximums may split
and then merge into the center soliton. Among the derived three solitons, the
center one has a breathing frequency changing over time, while the other ones
breath stably. We set the solitons on the sides as new initial solitons to
make them collide at the peaks. Two common solitons are obtained and the
splitting phenomenon dose not appear. Moreover, the robustness of breathing
solitons is considered numerically. The solitons shown in Figs. 6 (a-d) can
keep stable under the noise with amplitude up to $50\%$, and the related
result is not shown in this paper.

##  V Conclusion

In summary, the breathing dynamics of dipolar solitons induced by their
collision is studied numerically. After the collision, there is a short-lived
attracting process between solitons, which is induced by the long-range
feature of dipolar effect. This attraction makes the solitons momently wider
and finally breathing. The phase difference between two initial solitons has
an obvious influence on the breathing amplitude, breathing frequency, and the
transferred atom number of derived breathing solitons. By considering the
cases that two dipolar solitons move away from each other, we find that the
initial phase difference has no influence on the difference of their breathing
frequency. Only when their initial atom number is different, their breathing
frequency is different. Thus, the difference of breathing frequency is
directly induced by the difference of atom number, rather than the phase
difference. Also, the interaction between breathing solitons is studied. The
peak-peak collision between two breathing solitons generates solitons with
larger breathing amplitude, while the dip-dip collision generates solitons
with smaller one. By the peak-peak collision, we successfully generate an
after-four-collision breathing soliton. However, three solitons instead of two
ones appear after their collision, which indicates that an after-five-
collision cannot be generated in the case we consider. Our results pave a way
to study the generation, interaction, and adjustment of breathing solitons in
dipolar BECs and help to understand the influence of long-range feature of
dipole-dipole interaction on solitons.

## Acknowledgement

This work was supported by National Natural Science Foundation of China
(Contacts No. 11875220 and No. 12047502).

## References

  * (1) A. Griesmaier, J. Werner, S. Hensler, J. Stuhler, and T. Pfau, Phys. Rev. Lett. 94, 160401 (2005).
  * (2) J. Stuhler, A. Griesmaier, T. Koch, M. Fattori, T. Pfau S. Giovanazzi, P. Pedri, and L. Santos, Phys. Rev. Lett. 95 150406 (2005).
  * (3) Q. Beaufils, T. Zanon, R. Chicireanu, B. Laburthe-Tolra, E. Maréchal, L. Vernac, J.-C. Keller, and O. Gorceix, Phys. Rev. A 78, 051603 (2008).
  * (4) M. W. Lu, N. Q. Burdick, S. H. Youn, and B. L. Lev, Phys. Rev. Lett. 107, 190401 (2011).
  * (5) K. Aikawa, A. Frisch, M. Mark, S. Baier, A. Rietzler, R. Grimm, and F. Ferlaino, Phys. Rev. Lett. 108, 210401 (2012).
  * (6) Y. J. Tang, N. Q. Burdick, K. Baumann, and B. L. Lev, New J. Phys. 17, 045006 (2014).
  * (7) T. Dauxois, S. Ruffo, E. Arimondo, and M. Wilkens, _Dynamics and thermodynamics of systems with long-range interactions: An introduction_ (Springer Press, Germany, 2002).
  * (8) S. Giovanazzi, A. GöRlitz, and T. Pfau, Phys. Rev. Lett. 89, 130401 (2002).
  * (9) S. Sinha and L. Santos, Phys. Rev. Lett. 99, 140406 (2007).
  * (10) P. G. Kevrekidis, D. J. Frantzeskakis, and R. Carretero-González, _Emergent nonlinear phenomena in Bose-Einstein condensates: theory and experiment_ (Springer Science and Business Media Press, Germany, 2007).
  * (11) T. Koch, T. Lahaye, J. Metz, B. Frohlich, A. Griesmaier, and T. Pfau, Nat. phys. 4, 218 (2008).
  * (12) T. Lahaye, J. Metz, B. Fröhlich, T. Koch, M. Meister, A. Griesmaier, T. Pfau, H. Saito, Y. Kawaguchi, and M. Ueda, Phys. Rev. Lett. 101, 080401 (2008).
  * (13) T. Lahaye, C. Menotti, L. Santos5, M. Lewenstein, and T. Pfau, Rep. Prog. Phys. 72, 126401 (2009).
  * (14) H. Kadau, M. Schmitt, M. Wenzel, C. Wink, T. Maier, I. Ferrier-Barbut, and T. Pfau, Nature 530, 194 (2016).
  * (15) I. Ferrier-Barbut, H. Kadau, M. Schmitt, M. Wenzel, and T. Pfau, Phys. Rev. Lett. 116, 215301 (2016).
  * (16) P. Pedri and L. Santos, Phys. Rev. Lett. 95, 200404 (2005).
  * (17) V. M. Lashkin, Phys. Rev. A 75, 043607 (2007).
  * (18) M. Raghunandan, C. Mishra, K. Łakomy, P. Pedri, L. Santos, and R. Nath, Phys. Rev. A 92, 013637 (2015).
  * (19) I. Tikhonenkov, B. A. Malomed, and A. Vardi, Phys. Rev. A 78, 043614 (2008).
  * (20) S. Yi and H. Pu, Phys. Rev. A 73, 061602(2006).
  * (21) B. B. Baizakov, S. M. Al-Marzoug, and H. Bahlouli, Phys. Rev. A 92, 033605 (2015).
  * (22) B. K. Turmanov, B. B. Baizakov, B. A. Umarov, and F. K. Abdullaev, Phys. Lett. A 379, 1828 (2015).
  * (23) K. Pawłowski, and K. Rza̧żewski, New J. Phys. 17, 105006 (2015).
  * (24) M. J. Edmonds, T. Bland, D. H. J. O’Dell, and N. G. Parker, Phys. Rev. A 93, 063617 (2016).
  * (25) J. Cuevas, B. A. Malomed, P. G. Kevrekidis, and D. J. Frantzeskakis, Phys. Rev. A 79, 053608 (2009).
  * (26) M. J. Edmonds, T. Bland,R. Doran, and N. G. Parker, New J. Phys. 19, 023019 (2017).
  * (27) R. Eichler, D. Zajec, P. Koberle, J. Main, and G. Wunner, Phys. Rev. A 86, 053611 (2012).
  * (28) T. Bland, M. J. Edmonds, N. P. Proukakis, A. M. Martin, D. H. J. O’Dell, and N. G. Parker, Phys. Rev. A 92, 063601 (2015).
  * (29) J. Peng, S. Boscolo, Z. Zhao, and H. Zeng, Sci. Adv. 5, eaax1110 (2019).
  * (30) C. Y. Bao, Y. Xuan, C. Wang, A. Fülöp, D. E. Leaird and V. Torres-Company, Phys. Rev. Lett. 121, 257401 (2018).
  * (31) E. Lucas, M. Karpov, H. Guo, M. L. Gorodetsky, and T. J. Kippenberg, Nat. Commun. 8, 736 (2017).
  * (32) C. Y. Bao, J. A. Jaramillo-Villegas, Y. Xuan, D. E. Leaird, M. H. Qi, and A. M. Weiner, Phys. Rev. Lett. 117, 163901 (2016).
  * (33) M. Yu, J. K. Jang, Y. Okawachi, A. G. Griffith, K. Luke, S. A. Miller, X. Ji, M. Lipson, and A. L. Gaeta, Nat. Commun. 8, 14569 (2017).
  * (34) C. Y. Bao, L. Zhang, L. C. Kimerling, J. Michel, and C. X. Yang, Opt. Express 23, 18665 (2015).
  * (35) A. B. Matsko, A. A. Savchenkov, and L. Maleki, Opt. Lett. 37, 4856 (2012).
  * (36) M. A. Baranov, Phys. Rep. 464, 71 (2008).
  * (37) F. Deuretzbacher, J. C. Cremon, S. M. Reimann, Phys. Rev. A 81, 063616 (2010).
  * (38) P. Muruganandam and S. K. Adhikari S K, J. Phys. B 36, 2501 (2003).
  * (39) P. Muruganandam and S. K. Adhikari, Comput. Phys. Commun. 180, 1888 (2009).
  * (40) J. Yang, _Nonlinear Waves in Integrable and Nonintegrable Systems_ (Siam, Philadelphia, 2010).
  * (41) S. Yi and L. You, Phys. Rev. A 63, 053607 (2001).
  * (42) S. Yi and L. You, Phys. Rev. A 66, 013607 (2002).
  * (43) A. D. Martin and J. Ruostekoski, New J. Phys. 14, 043040 (2012).
  * (44) J. L. Helm, T. P. Billam, and S. A. Gardiner, Phys. Rev. A 85, 053621 (2012).