#  DialogSum: A Real-Life Scenario Dialogue Summarization Dataset

Yulong Chen♠♡ , Yang Liu♣ , Liang Chen◆ , Yue Zhang♡♢
♠ Zhejiang University
♡ School of Engineering, Westlake University
♣ Microsoft Cognitive Services Research
◆ College of Software, Jilin University
♢ Institute of Advanced Technology, Westlake Institute for Advanced Study
<EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS>

###### Abstract

Proposal of large-scale datasets has facilitated research on deep neural
models for news summarization. Deep learning can also be potentially useful
for spoken dialogue summarization, which can benefit a range of real-life
scenarios including customer service management and medication tracking. To
this end, we propose DialogSum, a large-scale labeled dialogue summarization
dataset. We conduct empirical analysis on DialogSum using state-of-the-art
neural summarizers. Experimental results show unique challenges in dialogue
summarization, such as spoken terms, special discourse structures,
coreferences and ellipsis, pragmatics and social common sense, which require
specific representation learning technologies to better deal with.

##  1 Introduction

Text summarization is the task of automatically generating a concise, salient,
coherent and fluent summary of a given set of documents Radev et al. (2002).
Thanks to the advance in neural network models and the availability of large-
scale labeled datasets, recent research has achieved promising progress on
summarizing monologic texts such as news articles Paulus et al. (2018);
Gehrmann et al. (2018); Liu and Lapata (2019); Liu et al. (2020), patents
Pilault et al. (2020) and academic papers Koncel-Kedziorski et al. (2019).

Figure 1: An example from DialogSum dataset compared with an example from SAMSum dataset. Datasets | Lan. style | Domain | Scenario | Dialogs | Data size | #Tokens/dial. | #Tokens/turn | #Comp. rate
---|---|---|---|---|---|---|---|---
AMI | spoken | single | meeting | 137 | 100hrs (video) | 4,757 | 16.5 | 0.07
SAMSum | written | multiple | online | 16,369 | 1.5M (token) | 94 | 8.4 | 0.30
DialogSum | spoken | multiple | daily life | 13,460 | 1.8M (token) | 131 | 13.8 | 0.18
Table 1: Comparison between DialogSum and other public dialogue summarization
datasets. Lan. stands for language. Dial. stands for dialogue. # stands for
the average result. Comp. stands for compression. The compression rate is the
ratio of the length of the summary divided by the length of the original text.

However, dialogue, as an important channel for achieving communicative intents
Bender and Koller (2020), has received significantly less attention from the
summarization research community. One main reason is the paucity of a suitable
summarization dataset built on dialogue texts. Most existing research uses the
AMI meeting corpus Carletta et al. (2005), which consists of $137$ dialogues
obtained from virtual multi-party meeting recordings. However, research on the
corpus is limited to its small scale. SAMSum Gliwa et al. (2019) is a recently
released written online dialogue summarization dataset, which contains $16$k
online chats with corresponding summaries. However, it focuses on
conversations via messenger apps, which are rather short (around $94$ tokens
per conversation) and their language style and topics also differ from spoken
daily dialogues.

A comparison between the real-life scenario dialogue and online chat is shown
in Figure 1. Online-chat messages contain unique tokens (e.g., “BTW”),
emoticons (e.g., “:)”) and emojis (e.g., “ *  X ”). In contrast, daily
conversations have a different and more formal style. In addition, real-life
dialogues have more diverse task-oriented scenarios and topics compared to
online chit-chats. For example, online-chat messages in SAMSum are about
leisure and social chats, but real-life dialogues contain business negotiation
(Figure 1(a)). Intuitively, automatically summarizing such dialogues can help
a business find common needs or complaints from customers. With the rise of
personal assisting chatbots, summarizing dialogues from different aspects of
daily life can also be useful for personal record management and other
applications.

We introduce Real-Life Scenario Dialogue Summarization (DialogSum), a large-
scale summarization dataset for dialogues. Dialogue data for DialogSum are
collected from three public dialogue corpora, namely Dailydialog Li et al.
(2017), DREAM Sun et al. (2019) and MuTual Cui et al. (2020), as well as an
English speaking practice website. These datasets contain face-to-face spoken
dialogues that cover a wide range of daily-life topics, including schooling,
work, medication, shopping, leisure, travel. Most conversations take place
between friends, colleagues, and between service providers and customers. We
clean and preprocess the dialogue data into a unified format, and ask
annotators to summarize them from an observer perspective. Topics are also
manually labeled for each dialogue. An example of DialogSum is shown in Figure
1(a), where the summary expresses the main content in a business conversation.

The contribution of DialogSum can be stated from two perspectives. First, from
the perspective of downstream applications, summarizing daily spoken dialogues
can be useful for both business and personal uses. Dialogue summaries can also
be useful for personal assistants to keep track of important events as such
business negotiation. Second, from the method perspective, DialogSum has a
larger scale of long dialogue data, which can facilitate the study of dialogue
summarization using neural network models. The number of dialogues in
DialogSum is orders of magnitude larger than in AMI, which can be useful for
training large neural network models for dialogue summarization. The average
length of dialogues in DialogSum is $39.8\%$ longer than in SAMSum. To our
knowledge, we are the first to release a large-scale real-life scenario
dialogue summarization dataset.

We empirically investigate the performance of state-of-the-art neural
summarization models on DialogSum, comparing the characteristics of the spoken
daily dialogue summarization dataset with standard news summarization
benchmarks and the online chat summarization benchmark SAMSum. Experimental
results show that DialogSum is more amenable to abstractive summarizers, while
being relatively more challenging compared to the existing summarization
datasets. We find that main difficulties arise from discourse structures in
multi-turn dialogues, as well as the need for book-keeping both entities and
events mentioned in turns of utterances. We release our dataset at
https://github.com/cylnlp/DialogSum.

##  2 The DialogSum Dataset

###  2.1 Dialogue Data Preparation

#### Data Collection

DailyDialog is a dataset consisting of 13k multi-turn dialogues, obtained from
websites that aim to help English learners to practice English speaking. DREAM
and MuTual are dialogue understanding datasets, consisting of 6k and 9k speech
transcripts, respectively, both collected from online English listening exam
materials. In order to further increase the diversity of data, we crawl
additional dialogues from another English speaking practice
website111http://www.tingroom.com which aims to provide English learners with
conversation examples in real life practical circumstances, such as business
negotiation and banking services.

Although dialogues of DialogSum are from different sources, they all share
important characteristics that are in line with what we expect. First, as
mentioned earlier, these dialogues are under rich real-life scenarios. Unlike
chitchats, these conversations have clear communication patterns and intents,
making them more suitable and valuable to serve as summarization sources
Carletta et al. (2005). Moreover, their multi-turn dialogue lengths are within
a reasonable scale and are longer than chitchats222The average numbers of
tokens for multi-turn dialogues are: Dailydialog: $118.8$, DREAM: $124.6$,
MuTual: $136.1$., which comforts the purpose of automatic summarization.
Greater lengths also indicate these dialogues contain more events and
discourse relations between them. Properly selecting vital events and
identifying their relations make summarizing these dialogues more challenging.

Source | Num. dial. | Tokens | % in DialogSum
---|---|---|---
DailyDialog | 7837 | 980,398 | 58.22
DREAM | 2028 | 301,098 | 16.94
MuTual | 1870 | 260,139 | 13.89
Crawled | 1725 | 238,902 | 12.82
Table 2: Proportions of dialogue sources in DialogSum.

#### Data Cleaning and Pre-Processing

We delete non-English characters, correct typos and grammatical errors, and
further filter out duplicated data based on text similarity. After
deduplicating, proportions of the data sources are summarized in Table 2.
Because of different data processing methods and annotation procedures,
original dialogues in DailyDialog, DREAM and MuTual are in different formats.
We follow previous work (Li et al., 2017; Zhang et al., 2018; Budzianowski et
al., 2018; Dinan et al., 2019) and preprocess them into a bi-turn dialogue
flow, merging continuous turns of the same speaker into one utterance. Also,
we add tags (e.g. #Person_1# and #Person_2# in Figure 1(a)) before each
dialogue turn, to distinguish speakers. The final DialogSum dataset contains
13,460 dialogues, which are divided into training (12,460), validation (500)
and test (500) sets.

###  2.2 Annotation

We ask annotators to write dialogue summaries based on following criteria: the
summary should (1) convey the most salient information of the dialogue and;
(2) be brief (no longer than 20% of the conversation length) and; (3) preserve
important named entities within the conversation and; (4) be written from an
observer perspective and; (5) be written in formal language.

We require our annotators to pay extra attention to the following aspects.

Tense Consistency: Annotators should take the moment that the conversation
occurs as the present time, and choose a proper tense to describe events
before and after the ongoing conversation.

Discourse Relation: If summarized events hold important discourse relations,
particularly causal relation, annotators should preserve the relations if they
are also in the summary.

Emotion: Different from newspaper and academic articles, social conversations
in DialogSum are often implied with emotions. Therefore, we ask annotators to
explicitly describe important emotions related to events in the summary.

Intent Identification: Rather than merely summarizing the consequences of
dialogues, annotators should also describe speakers’ intents in summaries, if
they can be clearly identified.

In addition to the above, annotators should use person tags to refer to
different speakers if real names cannot be detected from the conversation.
Annotators are also asked to write a short (around 3 tokens) topic for each
dialogue. Appendix A shows the list of topics.

Human Annotated Summary | R1 | R2 | RL
---|---|---|---
Summary1 to Summary2 | 52.90 | 26.01 | 50.42
Summary1 to Summary3 | 53.85 | 27.53 | 51.65
Summary2 to Summary3 | 53.30 | 26.61 | 50.44
Average | 53.35 | 26.72 | 50.84
Table 3: ROUGE scores between three human annotated summaries in test set. Dataset | % of novel $n$-grams | lead | longest | ext-oracle
---|---|---|---|---
| unigram | bigram | trigram | 4-gram | R1 | R2 | RL | R1 | R2 | RL | R1 | R2 | RL
CNN | 16.75 | 54.33 | 72.42 | 80.37 | 29.15 | 11.13 | 25.95 | - | - | - | 50.38 | 28.55 | 46.58
DailyMail | 17.03 | 53.78 | 72.14 | 80.28 | 40.68 | 18.36 | 37.25 | - | - | - | 55.12 | 30.55 | 51.24
NY Times | 22.64 | 55.59 | 71.93 | 80.16 | 31.85 | 15.86 | 23.75 | - | - | - | 52.08 | 31.5 | 46.72
XSum | 35.76 | 83.45 | 95.50 | 98.49 | 16.30 | 1.61 | 11.95 | - | - | - | 29.79 | 8.81 | 22.65
SAMSum | 32.63 | 77.22 | 89.27 | 94.83 | 31.41 | 8.70 | 30.41 | 32.13 | 10.13 | 29.11 | 44.60 | 17.37 | 39.38
DialogSum | 26.28 | 76.94 | 89.16 | 94.53 | 27.52 | 6.78 | 27.31 | 24.15 | 6.25 | 22.73 | 37.90 | 13.88 | 34.04
Table 4: Corpora statistics and extractive methods on CNN/DailyMail, NY Times,
XSum, SAMSum and DialogSum. Part of results is from Narayan et al. (2018). All
results are computed on test sets. For DialogSum, the results are the average
of multi-reference results.

###  2.3 Quality Control

To ensure quality, before formal annotation, we ask annotators to annotate
training samples until they pass our examination and meet our requirements.
After annotation, we check summaries by cross-validation between different
annotators twice. During the checking process, bonus is paid to checkers who
find unqualified summaries, and penalty is given to annotators whose
annotation is found with mistakes. In case of appeal, we make the final
decision. After the second checking, we sample $10\%$ summaries and manually
check the samples ourselves. If errors are found in an annotation batch, we
ask corresponding annotators to self-check and re-annotate the whole batch and
repeat this checking and sampling processes.

To further control the quality, and to analyze inter-annotator agreement, for
each dialogue in the test set, we provide three summaries written and checked
by different annotators. For each test dialogue, we compare its and compute
their pair-wise ROUGE (Lin, 2004) scores. Table 3 reports their averaged
$F_{1}$ scores of ROUGE-1 (R1), ROUGE-2 (R2) and ROUGE-L (RL). We see R2 is
relatively low while RL is high, which suggests that annotators’ usage of
language is variable, but the main content and logical order are mostly the
same.

###  2.4 Characteristics of DialogSum

We empirically compare DialogSum with existing news summarization datasets and
SAMSum. CNN/DailyMail (Hermann et al., 2015), NY Times (Sandhaus, 2008) and
XSum (Narayan et al., 2018) are large-scale summarization datasets from the
news domain, written in a monologic structure. XSum is a dataset designed
specifically for abstractive summarization.

First, we compare the percentages of novel $n$-grams in the reference summary
against the source document/dialogue. This intuitively reflects the level of
abstraction of annotated summaries. As shown in Table 4, except for XSum,
which is designed to be highly abstractive, dialogue-based summarization
datasets contain more novel $n$-grams in the summaries. We also find that the
percentage of novel unigrams in DialogSum is $26\%$, $6\%$ lower than in
SAMSum, but novel bigrams, trigrams and 4-grams are about the same as SAMSum.
We believe that the relatively lower novel unigram proportion in DialogSum
compared to SAMSum is because of our pre-processing and annotation criteria.
SAMSum’s summaries include real names, third-person singular pronouns, which
can be diverse across the dialogues. In contrast, DialogSum uses tags such as
#Person_1# to refer to persons whatever they are subjective, objective, or
possessive. This constrains the proportion of novel unigrams to be lower.

Second, we compare the datasets using several extractive summarization
methods. Following previous summarization work (Liu, 2019; Pilault et al.,
2020), we report R1, R2 and RL here. lead creates summaries by selecting the
first $n$ sentences from source texts. longest is designed for dialogue
summarization (Gliwa et al., 2019). It selects the $n$ longest utterances as a
summary, which gives better ROUGE scores than lead on SAMSum. ext-oracle
creates summaries by choosing $n$ sentences that have the highest ROUGE
against reference summaries. It can be viewed as an upper bound for extractive
summarization. We report results of lead-3, longest-3 and ext-oracle-2 on
SAMSum, and lead-2, longest-2 and ext-oracle-2 on DialogSum, where $n$ is
searched for each dataset in range of $1$ to $6$.

Model | CNNDM | XSum | SAMSum | DialogSum
---|---|---|---|---
| R1 | R2 | RL | R1 | R2 | RL | R1 | R2 | RL | R1 | R2 | RL
Transformer | 40.21 | 17.76 | 37.09 | 29.41 | 9.77 | 23.01 | 37.20 | 10.86 | 34.69 | 35.91 | 8.74 | 33.50
$\textsc{UniLMv2}_{\textsc{base}}$ | 43.16 | 20.42 | 40.14 | 44.00 | 21.11 | 36.08 | 50.53 | 26.62 | 48.81 | 47.04 | 21.13 | 45.04
$\textsc{BART}_{\textsc{large}}$ | 44.16 | 21.28 | 40.90 | 45.14 | 22.27 | 37.25 | 53.12 | 27.95 | 49.15 | 47.28 | 21.18 | 44.83
Table 5: Results of abstractive models on CNNDM, XSum, SAMSum and DialogSum.
For DialogSum, we give the average of multi-reference results.

The results are shown in Table 4. In terms of lead, DialogSum sees the lowest
R1 and R2 except for XSum, showing that it is in nature a highly abstractive
summarization dataset. SAMSum is less abstractive than DialogSum by all ROUGE
scores, which is likely because the compression rate of SAMSum ($0.30$) is
higher than DialogSum ($0.18$) (Table 1). The higher compression ratio
suggests the summary contains denser information in the original text. The
same conclusion can be found by using the longest method. By using the ext-
oracle method, we find that DialogSum is the most challenging dataset for
extractive summarizers except for XSum, which is carefully designed for
evaluating abstractive summarizers.

##  3 Experiments

We experiment with several abstractive summarization baselines to further
understand the characteristics and challenges of DialogSum. Following Gliwa et
al. (2019), we concatenate utterances of a dialogue as the input. For
pretrained models, we only finetune them on corresponding datasets.

###  3.1 Models

#### Transformer

We take Transformer (Vaswani et al., 2017) as a non-pretrained abstractive
baseline. For dialogue summarization, we follow Gliwa et al. (2019), using the
same hyper-parameters for news summarization333https://opennmt.net/OpenNMT-
py/examples/Summarization.html, but changing the minimum length to 15. We
train the 6-layer Transformer model with Adam (Kingma and Ba, 2014) for
100,000 steps. Copy attention mechanism is applied and the dropout rate is set
to 0.1.

#### UniLMv2

UniLMv2 (Bao et al., 2020) is a recently released pretrained language model
for autoencoding and partially autoregressive language modeling. Here we use
$\textsc{UniLMv2}_{\textsc{base}}$ as a strong abstractive model. For dialogue
summarization, we train the model with Adam for 100,000 steps with 2,000
warmup steps and learning rate is set to $1.5e^{-5}$.

#### BART

BART (Lewis et al., 2020) is an encoder-decoder Transformer model pretrained
on a large corpus using a denoising autoencoder task. We use the large version
of BART and finetune it with 5,000 training steps/200 warmup steps for
dialogue summarization. Learning rate is set to $3e^{-5}$.

###  3.2 Results

Table 5 presents the experimental results. In general, we find that non-
pretrained abstractive models outperform lead (Table 4), and the best results
are achieved by pretrained models, despite the fact that
$\textsc{BART}_{\textsc{large}}$ and $\textsc{UniLMv2}_{\textsc{base}}$ are
pretrained on monologic texts.

#### Extractive Summary vs Abstractive Summary

Transformer gives similar results on CNNDM and better results on XSum, SAMSum
and DialogSum compared to lead, and pretrained models show better performance
than ext-oracle on all datasets except for CNNDM. In particular, pretrained
models outperform Transformer by $13.07\sim 14.24\%$ RL on XSum, $14.12\sim
14.46\%$ RL on SAMSum, and $11.33\sim 11.54\%$ on DialogSum, while only
$3.05\sim 3.81\%$ on CNNDM. We believe that it is because CNNDM is a highly
extractive dataset (Section 2.4). The key to summarizing CNNDM is to correctly
understand intersentence relations within long documents, and extract
important sentences. In contrast, XSum, SAMSum and DialogSum are more
abstractive, which require a model to paraphrase. And the strong generation
capability of pretrained models can bring great improvements on them. We also
see that, for abstractive datasets, model performance decreases as document
length grows (Avg. length: SAMSum - $93.8$, DialogSum \- $131.1$, Xsum -
$431.1$) and compression rate decreases (Comp. rate: SAMSum - $0.30$,
DialogSum \- $0.18$, XSum - $0.05$). This explains why SAMSum is the easiest
dataset.

#### Spoken vs Written

All three models perform better on dialogue summarization datasets, compared
with XSum. This can be potentially because XSum is naturally highly
abstractive, and thus more challenging. We also compare improvement brought by
pretrained models that are trained on large written texts.

Still in Table 5, the improvement on DialogSum is the least.
$\textsc{BART}_{\textsc{large}}$ outperform Transformer by $15.73\%$ R1 on
XSum, $15.92\%$ R1 on SAMSum, but $11.37\%$ R1 on DialogSum. It demonstrates
that SAMSum has overall more written style than DialogSum, and also suggests
that dialogue and monologue are different. This can be explained by the design
of written app-chat annotation in SAMSum (Gliwa et al., 2019).

####  DialogSum vs SAMSum

As shown in Table 5, model performance is steadily lower on DialogSum than
SAMSum. As stated, DialogSum is more abstractive, open-domain, and spoken
analogous. One more possible reason for the lower performance on DialogSum is
the longer input size. To better quantify the difference between these two
dialogue summarization datasets, we further evaluate Transformer trained on
DialogSum when tested on the SAMSum, and vice versa. As shown in Table 6, the
performance of Transformer drops greatly when traiend on DialogSum and tested
on SAMSum, and vice versa. This shows that the two datasets have substantial
differences. In addition, Transformer trained on DialogSum performs better
than on SAMSum, and shows lower performance drop, suggesting that DialogSum
can provide more generalization ability for training dialogue summarization
models.

Trans. Test | R1 | R2 | RL
---|---|---|---
S2D | 31.72(-5.48) | 6.25(-4.61) | 29.72(-4.97)
D2S | 31.74(-4.17) | 5.93(-2.81) | 29.79(-3.71)
Table 6: Difference between DialogSum (D) and SAMSum (S). Trans. stands for
Transferred.

##  4 Human Evaluation

To better understand DialogSum, we take a deeper investigation into the
outputs of Transformer and UniLMv2 on DialogSum by conducting human evaluation
from multiple aspects.

#### Fluency, Consistency, Relevance and Coherence

First, following Kryscinski et al. (2019, 2020), we implement human evaluation
from four dimensions. Fluency evaluates the quality of individual generated
sentences, Consistency evaluates the factual alignment between the source text
and generated summary, Relevance evaluates the importance of summary content,
and Coherence evaluates the collective quality of all sentences.

We randomly select $50$ dialogues and their summaries from DialogSum test, and
ask a judge to give scores in scale from $1$ to $5$ along the four mentioned
dimensions. The higher, the better. The judge also gives scores to human-
annotated summaries to evaluate their quality.

Summary | Fluency | Cons. | Relevance | Coherence
---|---|---|---|---
Summary 1 | 5 | 5 | 4.96 | 5
Summary 2 | 5 | 5 | 4.98 | 5
Summary 3 | 5 | 5 | 5 | 5
Avg. | 5 | 5 | 4.98 | 5
Transformer | 4 | 2.08 | 2.3 | 3.84
$\textsc{UniLMv2}_{\textsc{base}}$ | 4.8 | 3.84 | 4.06 | 4.34
Table 7: Human evaluation on human annotated summaries and model generated
Summaries. Cons. stands for Consistency. Summary 1 - Summary 3 correspond to
three summaries of a dialogue.

As shown in Table 7, human annotated summaries receive the best scores from
all dimensions. $\textsc{UniLMv2}_{\textsc{base}}$ has steadily better scores
than Transformer, but lower than human. Model-generated summaries have the
highest scores on Fluency, while lowest on Consistency. It suggests that
although model-generated summaries are grammatical and fluent, they still
contain factual errors.

#### Discourse Relation

Reasonable summaries should convey important relations between main events,
and identifying discourse relations and using proper phrases to express them
in summaries can be challenging for summarization systems (Xu et al., 2020).
Take Figure 1 (a) for example, the human annotated summary connects two main
events (underlined) using “since” to express their causal relation explicitly.
However, the causal relation between those two events are not explicitly
expressed in the dialogue, and the distance between them is long. Multiple
turns usually correspond to more complicated discourse structure and relation.
Also, similar with Chen and Yang (2020), we find that model performance
decreases when the number of dialogue turns grows (See Appendix B).

To better evaluate model ability to disambiguate discourse relations in
DialogSum, we first collect discourse connectives from Penn Discourse Treebank
(Miltsakaki et al., 2004), and check whether these connectives are included in
summaries in the testset. If the three reference summaries of a dialogue all
contain connectives, we assume that the dialogues have strong discourse
signals. We choose $70$ dialogues from DialogSum in this way.

We then ask linguists who specialize in discourse to evaluate model outputs
and give scores from $\\{-1,0,1\\}$, where $1$ means that the generated
descriptions of main events are reasonable and contain correct discourse
connectives, $0$ means that the descriptions are good but contain no discourse
connectives and $-1$ means that the description is either incorrect or
contains incorrect connectives. We ask the linguists to focus only on clauses
or phrases that are essential to discourse relations, and ignore syntactic
errors. We report the distribution of annotated scores in Table 8.

Model | Human Scores | ROUGE Scores
---|---|---
| -1 | 0 | 1 | Avg. | R1 | R2 | RL
Transformer | 80% | 17% | 3% | -0.77 | 34.35 | 7.01 | 31.13
UniLMv2 | 43% | 37% | 20% | -0.23 | 43.78 | 17.91 | 40.97
Table 8: Human evaluation on discourse relations, with corresponding ROUGE
scores on the sub-test set. Avg. stands for the averaged score here.

We can see that the most summaries generated by Transformer are scored as
$-1$, and their average score is $-0.77$, close to $-1$. This means that
Transformer is not only incapable of identifying discourse relations but also
incapable of generating the main events correctly. UniLMv2 has a relatively
smooth distribution over three categories and a better average score of
$-0.23$, which is closer to $0$, suggesting that UniLMv2 can mostly choose
important events amongst the conversation. But the $-1$ still holds most
proportion and its average result is still far from $1$, indicating its
incapability of understanding relations between events.

Compared to the full test set, the model performance on this sub-set generally
decreases ($1.56\sim 3.26\%$ lower of R1, $1.73\sim 3.22\%$ of R2, $2.37\sim
4.07\%$ of RL), which also suggests complicated discourse relations between
events make summarization more difficult. The results indicate that further
research is necessary for better representing dialogue discourse structures in
order to obtain more reliable summarization systems.

#### Coreference Information

To evaluate model’s ability to distinguish different interlocutors, we ask a
judge to evaluate whether interlocutors’ names and their conversation
actions/contents are correctly associated in the 50 randomly selected data,
and give scores from $\\{-1,0,1\\}$, where $1$ means that all names and
actions/content in the summary are associated correctly, $0$ means partial
incorrectly, and $-1$ means all incorrectly. Here, we only focus on
coreference information in generated summaries, and ignore other errors, such
as incorrect syntax or failing to summarize salient information.

We report the distribution of annotated scores in Table 9. Most Transformer
generated summaries are annotated as $-1$ and the average result is close to
$-1$, suggesting that Transformer cannot generate clauses that express the
same relation between arguments and predicates in original dialogues. The
$\textsc{UniLMv2}_{\textsc{base}}$ has more $0$-scored summaries, and the
result is much higher, yet closer to $0$, which indicates that although
$\textsc{UniLMv2}_{\textsc{base}}$ can generate summaries containing correct
clauses, but still have much inconsistency. The performance of both models
indicates that Transformer is only capable of extracting important word-level
information from dialogues in DialogSum, while
$\textsc{UniLMv2}_{\textsc{base}}$ shows better performance on clause-level —
it can identify the speakers and partially preserve coreference information,
consistent with findings of Levesque et al. (2012) that pretraining is useful
for coreference resolution. However, it is far from human annotations.

Model | Human Scores | ROUGE Scores
---|---|---
| -1 | 0 | 1 | Avg. | R1 | R2 | RL
Transformer | 66% | 28% | 6% | -0.6 | 35.68 | 8.49 | 32.77
UniLMv2 | 4% | 56% | 40% | 0.36 | 47.46 | 21.33 | 44.93
Table 9: Human evaluation on models’ ability of preserving coreference information on DialogSum, with corresponding ROUGE scores. Avg. stands for the averaged score here.  Summary | Human Scores
---|---
| -1 | 1
Summary 1 | 7.7% | 92.3%
Summary 2 | 20.5% | 79.5%
Summary 3 | 17.9% | 82.1%
Avg. | 15.4% | 84.6%
Transformer | 84.6% | 15.4%
UniLMv2 | 30.8% | 69.2%
Table 10: Human evaluation on models’ ability of identifying interlocutors’
intents.  Figure 2: Case study on DialogSum. Dialogue-A \- a doctor and a
patient dialogue, Dialogue-B \- a customer and a hotel service dialogue.

#### Intent Identification

As stated in Section 2.2, we ask annotators to include important intents of
interlocutors in their summaries, addition to the consequences of dialogues.
The intent here refers to the motivation of a speaker to initiate a
conversation, e.g. “want to do an annual physical” (c.f. Figure 2,
Dialogue-A). This can make summaries more comprehensive and readable.
Therefore, we conduct corresponding human evaluation on whether interlocutors’
intents are described in summaries in the $50$ randomly selected data.

We first ask a judge to evaluate whether the intent is important to a
dialogue, and we select $39$ dialogues that contain important intents. Then,
we ask the judge to give scores from $\\{-1,1\\}$, where $1$ means that
intents are identified correctly, $-1$ means incorrectly. Note that we only
focus on intent identification in the summary, and other errors should be
ignored. We also ask the judge to evaluate human annotated summaries.

The distribution of annotated scores is shown in Table 10. We see that most
summaries generated by Transformer are scored as $-1$, which means that
Transformer is incapable of generating summaries that correctly convey
speakers’ intents. $\textsc{UniLMv2}_{\textsc{base}}$ shows much better
performance, however, it is still below human performance.

##  5 Challenges in DialogSum

Compared to written texts, spoken dialogues can be more difficult for models
to understand, and to summarize Goo and Chen (2018). Therefore, we conduct
error analysis and case studies on DialogSum to quantitatively and
qualitatively discuss such challenges.

###  5.1 Error Analysis

We make error analysis on the $50$ selected model-generated summaries (Section
4). Table 11 summarizes the five most frequent error types and their error
rates. In general, $\textsc{UniLMv2}_{\textsc{base}}$ shows better performance
than Transformer, but its error rates are still high. In particular, incorrect
coreference (c.f. Section 4) sees the highest error rates for both models,
indicating that models can be confused because of interactive information
flow. Compared with Transformer, $\textsc{UniLMv2}_{\textsc{base}}$ can
greatly avoid errors regarding unfactual information ($-52\%$) and syntactic
($-50\%$). However, it still suffers from coreference issues, and tends to
generate redundant summaries.

Error Type | Transformer | $\textsc{UniLMv2}_{\textsc{base}}$
---|---|---
Incorrect Coref. | 94% | 60%
Missing Salient Inf. | 64% | 32%
Redundant Inf. | 62% | 44%
Unfactual Inf. | 74% | 22%
Syntactic Error | 72% | 22%
Table 11: Error analysis of model performance on DialogSum. Coref. stands for
coreference, and Inf. stands for Information.

###  5.2 Case Study

We demonstrate two dialogues and their human-annotated/system-generated
summaries in Figure 2.

First, a big challenge posed by spoken dialogues is that their information
flow is different from monologic text, which is intuitively reflected in the
dialogue discourse structures (Wolf and Gibson, 2005). For example, two
utterances can be closely related even where there is a large distance between
them. Such phenomena are common in spoken dialogues such as negotiations and
procedures (e.g., medical consultation and police reports). Due to the unique
structure of the spoken dialogue, important information is rather dispersed
than well-structured monologues and written-dialogues.

Regular greetings can be useless to written dialogue summaries (e.g. SAMSum),
which is reflected by that lead is worse than longest on SAMSum (Table 4). In
contrast, lead outperforms longest by over $3\%$ on DialogSum. This is
because, for spoken dialogues, such utterances sometimes express and indicate
essential intents of speakers (c.f. Section 4). Farewells also express the
dialogue consequence and future plan of the speakers (e.g. dialogues in Figure
2). Besides, interruptions appear frequently in the middle of conversations
(Figure 2, Dialogue-B). These interruptions make other speaker’s utterances
incomplete, adding redundant information, and can also destroy coherent
discourse structures, making dialogues more difficult to encode. These
characteristics also make information in DialogSum dialogues more dispersed
than existing datasets.

Second, coreference and ellipsis are frequent in spoken dialogues (Grosz et
al., 1995; Quan et al., 2019). It is a natural behavior of communication that
humans obey as a rhetorical principle for saving words and avoiding
repetitions. Although it can be trivial for humans, their understanding can be
challenging to a neural model. For example, to correctly generate
“mischarged/wrong” in summary-b1-summary-b3, models need to understand “I
think you have added someone else’s (laundry service on my bill)”, where “my
bill” refers to “#Person_2#’s bill”.

Third, pragmatics and social common sense give a unique challenge for spoken
language understanding and has a significant impact on summarization. From the
last two sentences of Dialogue-B, human could understand that the “Here you
are” is actually “make a payment”, and “Goodbye” indicates that the event
“check out” is finished. It requires commonsense knowledge to fully understand
such dialogues. Beside, dialogues are summarized from a different perspective
(compared with speakers’ perspective), which suggests that summarizing
dialogues needs to go beyond summarizing dialogue contents, but also dialogue
actions at the pragmatic level. For example, “explains” in summary-a1 and
summary-a2 summarizes multiple dialogue actions of #Person_1#, “agree” in
Figure 1 (a) summarizes actions of both speakers. It requires model to not
only summarize what speakers are saying, but also what they are doing.

##  6 Conclusion

We presented DialogSum, a large-scale dialogue summarization dataset,
investigating its characteristics and challenges empirically. Experiments with
typical models show that DialogSum is highly abstractive, and poses unique
challenges in discourse and complex co-references. From these observations, we
made discussion on the uniqueness of spoken dialogue summarization, listing
several key problems to consider in future modeling. To our knowledge, we are
the first to release a large-scale dataset for real-life scenario dialogue
summarization.

##  7 Ethics Consideration

As mentioned, we collect our data from DailyDialog, DREAM and MuTual that all
are public for academic use. The additional data are from www.tingroom.com,
which are available to the public as well. The sources of our dialogue data
are freely accessible online without copyright constraint to academic use.

We hired annotators who have degrees in English Linguistics or Applied
Linguistics. Before formal annotation, we annotated 50 samples randomly
extracted from the dataset, and calculated our average annotation time so we
could set a fair salary for annotators’ training annotation. During the
training annotation process, they were paid as well. We also calculated the
average annotation time for each dialogue during training, based on which we
determined the final salary was around $9.5$ dollars per hour. This hourly
salary was the same for manual checking. All of our annotators took this
annotation as a part-time job.

## Acknowledgments

Yue Zhang is the corresponding author. We would like to thank the anonymous
reviewers for their comments, and Xuefeng Bai, Ming Shen and Bonnie Webber for
insightful discussion and proofreading. Also, we would like to thank Leyang
Cui for sharing original dialogue data of MuTual dataset. Our sincere
appreciation goes to annotators who have contributed to building this dataset.
This work receives funding support from the Westlake University and Bright
Dream Joint Institute for Intelligent Robotics, and a research grant from
Rxhui Inc.

## References

  * Bao et al. (2020) Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Xiaodong Liu, Yu Wang, Jianfeng Gao, Songhao Piao, Ming Zhou, and Hsiao-Wuen Hon. 2020.  Unilmv2: Pseudo-masked language models for unified language model pre-training.  In _Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event_ , volume 119 of _Proceedings of Machine Learning Research_ , pages 642–652. PMLR.
  * Bender and Koller (2020) Emily M. Bender and Alexander Koller. 2020.  Climbing towards NLU: On meaning, form, and understanding in the age of data.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5185–5198, Online. Association for Computational Linguistics.
  * Budzianowski et al. (2018) Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gašić. 2018.  MultiWOZ - a large-scale multi-domain Wizard-of-Oz dataset for task-oriented dialogue modelling.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 5016–5026, Brussels, Belgium. Association for Computational Linguistics.
  * Carletta et al. (2005) Jean Carletta, Simone Ashby, Sebastien Bourban, Mike Flynn, Mael Guillemot, Thomas Hain, Jaroslav Kadlec, Vasilis Karaiskos, Wessel Kraaij, Melissa Kronenthal, et al. 2005.  The ami meeting corpus: A pre-announcement.  In _International workshop on machine learning for multimodal interaction_ , pages 28–39. Springer.
  * Chen and Yang (2020) Jiaao Chen and Diyi Yang. 2020.  Multi-view sequence-to-sequence models with conversational structure for abstractive dialogue summarization.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 4106–4118, Online. Association for Computational Linguistics.
  * Cui et al. (2020) Leyang Cui, Yu Wu, Shujie Liu, Yue Zhang, and Ming Zhou. 2020.  MuTual: A dataset for multi-turn dialogue reasoning.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1406–1416, Online. Association for Computational Linguistics.
  * Dinan et al. (2019) Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2019.  Wizard of wikipedia: Knowledge-powered conversational agents.  In _7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019_. OpenReview.net.
  * Gehrmann et al. (2018) Sebastian Gehrmann, Yuntian Deng, and Alexander Rush. 2018.  Bottom-up abstractive summarization.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 4098–4109, Brussels, Belgium. Association for Computational Linguistics.
  * Gliwa et al. (2019) Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer. 2019.  SAMSum corpus: A human-annotated dialogue dataset for abstractive summarization.  In _Proceedings of the 2nd Workshop on New Frontiers in Summarization_ , pages 70–79, Hong Kong, China. Association for Computational Linguistics.
  * Goo and Chen (2018) Chih-Wen Goo and Yun-Nung Chen. 2018.  Abstractive dialogue summarization with sentence-gated modeling optimized by dialogue acts.  In _2018 IEEE Spoken Language Technology Workshop (SLT)_ , pages 735–742. IEEE.
  * Grosz et al. (1995) Barbara J. Grosz, Aravind K. Joshi, and Scott Weinstein. 1995.  Centering: A framework for modeling the local coherence of discourse.  _Computational Linguistics_ , 21(2):203–225.
  * Hermann et al. (2015) Karl Moritz Hermann, Tomás Kociský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015.  Teaching machines to read and comprehend.  In _Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada_ , pages 1693–1701.
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_.
  * Koncel-Kedziorski et al. (2019) Rik Koncel-Kedziorski, Dhanush Bekal, Yi Luan, Mirella Lapata, and Hannaneh Hajishirzi. 2019.  Text Generation from Knowledge Graphs with Graph Transformers.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 2284–2293, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Kryscinski et al. (2019) Wojciech Kryscinski, Nitish Shirish Keskar, Bryan McCann, Caiming Xiong, and Richard Socher. 2019.  Neural text summarization: A critical evaluation.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 540–551, Hong Kong, China. Association for Computational Linguistics.
  * Kryscinski et al. (2020) Wojciech Kryscinski, Bryan McCann, Caiming Xiong, and Richard Socher. 2020.  Evaluating the factual consistency of abstractive text summarization.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 9332–9346, Online. Association for Computational Linguistics.
  * Levesque et al. (2012) Hector Levesque, Ernest Davis, and Leora Morgenstern. 2012.  The winograd schema challenge.  In _Thirteenth International Conference on the Principles of Knowledge Representation and Reasoning_. Citeseer.
  * Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020.  BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7871–7880, Online. Association for Computational Linguistics.
  * Li et al. (2017) Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017.  DailyDialog: A manually labelled multi-turn dialogue dataset.  In _Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 986–995, Taipei, Taiwan. Asian Federation of Natural Language Processing.
  * Likas et al. (2003) Aristidis Likas, Nikos Vlassis, and Jakob J Verbeek. 2003.  The global k-means clustering algorithm.  _Pattern recognition_ , 36(2):451–461.
  * Lin (2004) Chin-Yew Lin. 2004.  ROUGE: A package for automatic evaluation of summaries.  In _Text Summarization Branches Out_ , pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  * Liu (2019) Yang Liu. 2019.  Fine-tune bert for extractive summarization.  _arXiv preprint arXiv:1903.10318_.
  * Liu and Lapata (2019) Yang Liu and Mirella Lapata. 2019.  Text summarization with pretrained encoders.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3730–3740, Hong Kong, China. Association for Computational Linguistics.
  * Liu et al. (2020) Yang Liu, Sheng Shen, and Mirella Lapata. 2020.  Noisy self-knowledge distillation for text summarization.  _arXiv preprint arXiv:2009.07032_.
  * Miltsakaki et al. (2004) Eleni Miltsakaki, Rashmi Prasad, Aravind Joshi, and Bonnie Webber. 2004.  The Penn Discourse Treebank.  In _Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC’04)_ , Lisbon, Portugal. European Language Resources Association (ELRA).
  * Narayan et al. (2018) Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018.  Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 1797–1807, Brussels, Belgium. Association for Computational Linguistics.
  * Paulus et al. (2018) Romain Paulus, Caiming Xiong, and Richard Socher. 2018.  A deep reinforced model for abstractive summarization.  In _6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings_. OpenReview.net.
  * Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014.  Glove: Global vectors for word representation.  In _Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)_ , pages 1532–1543.
  * Pilault et al. (2020) Jonathan Pilault, Raymond Li, Sandeep Subramanian, and Chris Pal. 2020.  On extractive and abstractive neural document summarization with transformer language models.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 9308–9319, Online. Association for Computational Linguistics.
  * Quan et al. (2019) Jun Quan, Deyi Xiong, Bonnie Webber, and Changjian Hu. 2019.  GECOR: An end-to-end generative ellipsis and co-reference resolution model for task-oriented dialogue.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 4547–4557, Hong Kong, China. Association for Computational Linguistics.
  * Radev et al. (2002) Dragomir R. Radev, Eduard Hovy, and Kathleen McKeown. 2002.  Introduction to the special issue on summarization.  _Computational Linguistics_ , 28(4):399–408.
  * Sandhaus (2008) Evan Sandhaus. 2008.  The new york times annotated corpus.  _Linguistic Data Consortium, Philadelphia_ , 6(12):e26752.
  * Sun et al. (2019) Kai Sun, Dian Yu, Jianshu Chen, Dong Yu, Yejin Choi, and Claire Cardie. 2019.  DREAM: A challenge data set and models for dialogue-based reading comprehension.  _Transactions of the Association for Computational Linguistics_ , 7:217–231.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA_ , pages 5998–6008.
  * Wolf and Gibson (2005) Florian Wolf and Edward Gibson. 2005.  Representing discourse coherence: A corpus-based study.  _Computational Linguistics_ , 31(2):249–287.
  * Xu et al. (2020) Jiacheng Xu, Zhe Gan, Yu Cheng, and Jingjing Liu. 2020.  Discourse-aware neural extractive text summarization.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5021–5031.
  * Zhang et al. (2018) Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018.  Personalizing dialogue agents: I have a dog, do you have pets too? In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 2204–2213, Melbourne, Australia. Association for Computational Linguistics.

##  Appendix A Dialogue Topics

We use $k$-means (Likas et al., 2003) to cluster the dialogue topic with GloVe
embedding (Pennington et al., 2014), where $k=20$. Figure 3 presents the
proportion of clustering results. Table 12 presents the cluster topics with
corresponding id, which is assigned by human.

Figure 3: Proportion of dialogue topics. ID | topic | ID | topic | ID | topic | ID | topic
---|---|---|---|---|---|---|---
1 | interpersonal relation | 6 | education | 11 | personal and business appoinment | 16 | transportation
2 | work and career | 7 | hobby | 12 | housing and apartment | 17 | in-store shopping
3 | causal chitchat | 8 | interview | 13 | consultation | 18 | health and medicine
4 | hotel and restaurant service | 9 | vacation plan | 14 | personal life | 19 | entertainment
5 | sales | 10 | climate | 15 | economics | 20 | food ordering
Table 12: Topic clusters of DialogSum.

##  Appendix B Dialogue Turns

The number of conversation turns can have a direct impact on neural models.
Multi-turn dialogues correspond to more complicated information flow and
discourse structure. Following Chen and Yang (2020), we split test data based
on dialogue turns, with a step size of $3$, and show model performance on
different dialogue turns.

The results are shown in Figure 4. The performance of Transformer and
$\textsc{UniLMv2}_{\textsc{base}}$ decreases when number of turns grows,
suggesting that more interactions between interlocutors and complicated
discourse structures bring challenge. This phenomenon is also observed by
(Chen and Yang, 2020) for SAMSum.

Figure 4: Model performance against the number of dialogue turns. T -
Transformer. UNI - UniLMv2.


# Projective Quantum Phase Difference Estimation Algorithm for the Direct
Computation of Eigenenergy Gaps on a Quantum Computer

Kenji Sugisaki  [<EMAIL_ADDRESS>
###### Abstract

Quantum computers are capable of calculating the energy gap of two electronic
states by using the quantum phase difference estimation (QPDE) algorithm. The
Bayesian inference based implementations for the QPDE have been reported so
far, but this approach is not projective, and the quality of the calculated
energy gap depends on the input wave functions being used. Here, we report the
inverse quantum Fourier transformation based QPDE with $N_{a}$ of ancillary
qubits, which allows us to compute the difference of eigenenergies based on
the single-shot projective measurement. As a proof-of-concept demonstrations,
we report numerical experiments for the singlet–triplet energy gap of hydrogen
molecule and the vertical excitation energies of halogen-substituted
methylenes (CHF, CHCl, CF2, CFCl and CCl2) and formaldehyde (HCHO).

###### keywords:

Quantum computing, Ab initio electronic structure theory, Quantum chemistry

Keio University]Graduate School of Science and Technology, Keio University,
7-1 Shinkawasaki, Saiwai-ku, Kawasaki, Kanagawa 212-0032, Japan [KQCC]Quantum
Computing Center, Keio University, 3-14-1 Hiyoshi, Kohoku-ku Yokohama,
Kanagawa 223-8522, Japan [TCG Crest]Centre for Quantum Engineering, Research
and Education TCG Centres for Research and Education in Science and
Technology, Sector V, Salt Lake, Kolkata 700091, India

##  1 Introduction

Quantum computing is one of the most innovative research fields in current
science, and it is anticipated to bring breakthroughs in quantum chemical
calculations. Quantum chemical calculations are based on the Schrödinger
equation, which governs the dynamics of quantum particles, and accurate
quantum chemical calculations can potentially open the door to predictive
quantum chemistry. However, as Dirac noted, “The underlying physical laws
necessary for the mathematical theory of a large part of physics and the whole
of chemistry are thus completely known, and the difficulty is only that the
exact application of these laws leads to equations much too complicated to be
soluble.”.1 The computational cost of the full configuration interaction
(full-CI) method, which is the variationally best possible wave function in
the Hilbert space spanned by the basis set used, scales exponentially with
system size, and it is impractical except for small molecules with medium-size
basis set.

Since quantum chemical calculation deals with the dynamics of electrons in
atoms and molecules, it is a potentially amenable problem for quantum
computers. In fact, in 2005 Aspuru-Guzik and coworkers reported a method for
performing the full-CI calculation on a quantum computer,2 using a quantum
phase estimation (QPE) algorithm.3 The QPE is a quantum algorithm that is
capable of computing the eigenvalues and corresponding eigenvectors of a
unitary operator $U$. By using the time evolution operator $\exp(-iHt)$ for
$U$, one can calculate the full-CI energies on a quantum computer. However,
because QPE utilizes the projective measurement of the quantum state to obtain
the eigenfunction and the corresponding eigenvalue, it is probabilistic, and
which electronic state is obtained from the QPE depends on the overlap between
the approximate wave function used as the input and the exact eigenfunction.
Importantly, the QPE itself does not guarantee an exponential speedup of
quantum chemical calculations,4 and connecting theoretical methods to generate
sophisticated approximated wave function is necessary.

Quantum chemical calculations can afford to compute total energies of atoms
and molecules, but total energies are generally not available from
experiments. Chemical phenomena relevant to quantum chemical calculations are
usually discussed in terms of the energy gaps between different geometries or
electronic states. Thus, accurate prediction of the energy gaps is crucial for
the practical use of quantum chemical calculations in chemistry research and
development.

Because quantum computers can use quantum superposition states as their
computational resources, it is possible to compute the energy gaps directly on
a quantum computer. In fact, several quantum algorithms for the direct
estimation of energy gaps have been reported, such as the quantum annealing-
based approach,5 the quantum–classical hybrid algorithm,6 using the robust
phase estimation technique,7 and the algorithms for fault-tolerant quantum
computers including the Bayesian exchange coupling parameter calculator with
broken-symmetry wave functions (BxB) algorithm8 and the Bayesian phase
difference estimation (BPDE) algorithm.9, 10, 11, 12 In addition to these
methods, multiple eigenvalue estimation techniques for the simultaneous
determination of the ground and the excited state energies have been
reported.13, 14, 15 In particular, the BPDE algorithm reported by us is an
extension of the Bayesian phase estimation (BPE)16, 17, 18 to the quantum
phase difference estimation using the quantum superposition of two electronic
states. BPDE is in principle applicable to arbitrary electronic states and it
is free from the controlled-time evolution requited in conventional QPE
algorithms for total energy calculations. The BPDE algorithm has been applied
to the direct calculation of ionization energies, singlet–triplet energy gaps
and valence excitation energies,9 total energies by using the quantum
superposition of the desired electronic state and the vacuum state,10 finite
difference based numerical energy gradients,11 and relativistic energy gaps
(fine-structure splitting).12 The BPDE algorithm is a powerful tool for
studying the energy gap of atoms and molecules, but it has two major
weaknesses. (1) It requires many circuit executions to compute the energy gaps
and the calculated energy gaps suffer from shot noise, and (2) it is not
projective, and the calculated energy gap depends on the approximated wave
functions used as the input. The second drawback can be clearly seen in the
total energy calculations of the transition state between trans\- and iso-
isomers of the $\mathrm{N_{2}H_{2}}$ molecule.10 It is desirable to extend the
quantum phase difference estimation (QPDE) algorithm to the projective
methods.

In this work, we propose a projective QPDE algorithm with $N_{a}$ of ancillary
qubits. Hereafter we denote the proposed approach as the “$N$-qubit QPDE”
algorithm. The $N$-qubit QPDE is a natural extension of the $N$-qubit QPE to
the phase difference estimation, and it is able to compute the eigenenergy
difference even if we use approximated wave functions as inputs. Similar to
the $N$-qubit QPE for total energy calculations, which eigenenergy gap is
obtained in the $N$-qubit QPDE depends on the overlap between the approximated
wave functions and the eigenfunctions. To our knowledge, all other approaches
for the direct calculation of the energy gap reported so far are neither
projective nor single-shot protocols, and this is the first attempt to perform
the eigenenergy gap calculation on a quantum computer without repetitive
quantum circuit executions. As the demonstrations, we report numerical quantum
circuit simulations for the singlet–triplet energy gap of $\mathrm{H_{2}}$
molecule, and the vertical excitation energies of halogen-substituted
methylenes (CHF, CHCl, $\mathrm{CF_{2}}$, CFCl, and $\mathrm{CCl_{2}}$) and
formaldehyde (HCHO). We also report the application of an algorithmic error
mitigation (AEM)19 to remove the Trotter–Suzuki decomposition error and to
improve the calculated energy differences.

##  2 Theory

Figure 1:  Quantum circuits for (a) the BPE and (b) the BPDE algorithms. The
definition of quantum gates is given in the main text.

First, we briefly discuss the BPE16, 17, 18 and the BPDE9 algorithms. The
quantum circuits for the BPE and the BPDE algorithms are shown in Figure 1.
Here, H is an Hadamard gate, $Pr(g)$ is an approximate ground state
preparation gate defined in eq 1, $U$ is a time evolution operator given in eq
2, $P$ is a phase shift gate defined in eq 5, $Ex$ is a quantum circuit that
generates an approximated excited state wave function $|\Phi_{1}\rangle$ from
the approximated ground state wave function $|\Phi_{0}\rangle$ as in eq 6,
$\varepsilon$ and $\Delta\varepsilon$ are estimators of the total energy and
the energy gap, respectively. Throughout this paper we have used
$\\{|\Phi\rangle\\}$ for approximated wave functions and $\\{|\Psi\rangle\\}$
for eigenfunctions. $N_{s}$ in eq 1 is the number of qubits used to store the
wave functions. In conventional fermion–qubit mapping methods such as the
Jordan–Wigner transformation (JWT)20 and the Bravyi–Kitaev transformation
(BKT),21 $N_{s}$ is equal to the number of spin orbitals included in the
active space. Roughly speaking, the QPDE can be implemented by replacing the
controlled-time evolution operation in the QPE by controlled-excitation and
subsequent time evolution and controlled-deexcitation operations.

$\displaystyle Pr(g)|0\rangle^{\otimes
N_{s}}=|\Phi_{0}\rangle=\sum_{j}c_{j}|\Psi_{j}\rangle$ (1) $\displaystyle
U=\exp(-iHt)$ (2) $\displaystyle P(\theta)=\left(\begin{array}[]{cc}1&0\\\
0&e^{i\theta}\end{array}\right)$ (5) $\displaystyle
Ex|\Phi_{0}\rangle=|\Phi_{1}\rangle=\sum_{k}d_{k}|\Psi_{k}\rangle$ (6)

By expanding the approximated wave functions by eigenfunctions as given in the
right hand side of eqs 1 and 6, the probability of obtaining the $|0\rangle$
state in the measurement of an ancillary qubit, $Prob(0)$, can be calculated
as in eqs 7 and 8 for the BPE and the BPDE, respectively.

$\displaystyle
Prob(0;\mathrm{BPE})=\frac{1}{2}\left[1+\sum_{j}|c_{j}|^{2}\cos\\{(E_{j}-\varepsilon)t\\}\right]$
(7) $\displaystyle
Prob(0;\mathrm{BPDE})=\frac{1}{2}\left[1+\sum_{j,k}|c_{j}|^{2}|d_{k}|^{2}\cos\\{(\Delta
E_{jk}-\Delta\varepsilon)t\\}\right]$ (8)

These equations insist that if the approximated wave functions $|\Phi\rangle$
have sufficiently large overlaps with the eigenfunctions of the target
electronic states, $Prob(0)$ becomes maximum at $\varepsilon=E_{j}$ and
$\Delta\varepsilon=\Delta E_{jk}$ for the BPE and the BPDE, respectively. In
the BPE and the BPDE algorithms, total energies and energy differences are
computed by searching for the $\varepsilon$ and $\Delta\varepsilon$,
respectively, that give maximum $Prob(0)$, using Bayesian inference.

From eqs 7 and 8, it is clear that the accuracy of the calculated total
energies and energy gaps in the framework of the BPE and the BPDE algorithms
strongly depends on the quality of the approximated wave functions being used
as the inputs. If the approximated wave functions are expressed by linear
combinations of many eigenstates, and contributions from electronic states
other than the target state to $Prob(0)$ become non-negligible, the peak
position of $Prob(0)$ may be shifted from that corresponding to the true
eigenvalues. For typical closed-shell singlet molecules in their equilibrium
geometries, the Hartree–Fock wave function $|\Phi_{\mathrm{HF}}\rangle$ is a
reasonably good approximation of the ground state wave function. However, the
overlap squared value $|\langle\Phi_{\mathrm{HF}}|\Psi_{\mathrm{full-
CI}}\rangle|^{2}$ can decrease exponentially with the system size, and
therefore using $|\Phi_{\mathrm{HF}}\rangle$ as the input wave function is
only valid for small molecules. In addition, the excited state wave function
is usually more complicated than that of the ground state, hence generating a
good approximated wave function for the excited state is in general a
difficult task. One possible solution is to adopt an adiabatic state
preparation (ASP) algorithm to generate correlated wave functions.2, 22, 23,
24 However, the quantum circuit for the ASP is usually deep and it is
unsuitable for the BPE and BPDE frameworks which need repetitive quantum
circuit executions.

Figure 2:  Quantum circuits for (a) the $N$-qubit QPE algorithm, (b) the
$N$-qubit QPDE algorithm in a naive implementation, and (c) the $N$-qubit QPDE
algorithm used in this work.

In the total energy calculations, single-shot projective calculations are
possible by using the QPE algorithm with $N_{a}$ of ancillary qubits.2 The
quantum circuit for the $N$-qubit QPE is shown in Figure 2(a). Here, QFT†
represents the quantum circuit for the inverse quantum Fourier
transformation.25 By assuming eq 1, the quantum state before the measurement
of ancillary qubits in Figure 2(a) is calculated as in eq 9.

$\displaystyle|0\rangle^{\otimes N_{a}}\otimes|0\rangle^{\otimes
N_{s}}\xrightarrow{\mathrm{QPE}}\sum_{j}c_{j}|\phi_{j}\rangle\otimes|\Psi_{j}\rangle$
(9)

Thus, the measurement of ancillary qubits yields an eigenphase
$\phi_{j}=0.x_{1}x_{2}x_{3}...x_{N_{s}}$ in binary fraction with the
probability proportional to $|c_{j}|^{2}$, and the wave function is projected
onto the corresponding eigenfunction $|\Psi_{j}\rangle$. Here, $\\{x_{j}\\}$
are the measurement outcome of the $j$-th ancillary qubit. The eigenenergy $E$
can be calculated by using the equation $E=-2\pi\phi/t$, which is derived from
the equation
$e^{-iHt}|\Psi\rangle=e^{-iEt}|\Psi\rangle=e^{2i\pi\phi}|\Psi\rangle$. From an
analogy of the extension of the BPE algorithm to the BPDE, we can construct
the quantum circuit for the $N$-qubit QPDE algorithm by replacing the
controlled-$U$ gates with the sequence of controlled-$Ex$, $U$, and
controlled-$Ex^{\dagger}$ gates, as in Figure 2(b). Note that the quantum
circuit in Figure 2(b) contains control-free time evolution operations, and
therefore this implementation implies that $|\Phi_{0}\rangle$ is an
eigenfunction of the time evolution operator. Unfortunately, this assumption
is not generally true. This difficulty can be avoided by starting from the
$|0\rangle^{\otimes N_{s}}$ state and using the controlled-$Pr$ gate given in
eq 10 instead of using the controlled-$Ex$ gate, as shown in Figure 2(c).

$\displaystyle\textrm{controlled-}Pr=|0\rangle\langle 0|\otimes
Pr(g)+|1\rangle\langle 1|\otimes Pr(e)$ (10)

Here, $Pr(g)|0\rangle^{\otimes N_{s}}=|\Phi_{0}\rangle$ and
$Pr(e)|0\rangle^{\otimes N_{s}}=|\Phi_{1}\rangle$. Since the
$|0\rangle^{\otimes N_{s}}$ state in JWT and BKT corresponds to the vacuum
state with no electrons, applying $U$ does not change the state,
$U|0\rangle^{\otimes N_{s}}=|0\rangle^{\otimes N_{s}}$. As a result, the
quantum circuit for the $N$-qubit QPDE is slightly deeper than that for the
$N$-qubit QPE. Nevertheless, the $N$-qubit QPDE-based direct calculation of
the energy gap is still advantageous than the putative approach based on the
$N$-qubit QPE for total energy calculations of individual electronic states,
as long as the controlled-$Pr$ gate can be realized by shallow quantum
circuit. In the present study, the approximated wave functions are expressed
by a linear combination of at most two Slater determinants, and thus the
quantum circuits for controlled-$Pr$ are sufficiently shallow.

In the implementation described in Figure 2(c), the bit string obtained from
the measurement of ancillary qubits corresponds to the eigenphase difference
$\Delta\phi_{jk}=0.x_{1}x_{2}x_{3}...x_{N_{s}}$, and the eigenenery gap can be
calculated as $\Delta E_{jk}=-2\pi\Delta\phi_{jk}/t$. The probability of which
eigenenergy gap is obtained is proportional to $|c_{j}|^{2}\times|d_{k}|^{2}$.

It should be noted that the energy gap can be positive or negative, but the
$\Delta\phi_{jk}$ obtained from the $N$-qubit QPDE quantum circuit does not
contain information about the digits above the arithmetic point. Therefore,
$\Delta\phi_{jk}$ and $\Delta\phi_{jk}-k$, where $k$ is an arbitrary integer,
cannot be distinguished in the $N$-qubit QPDE. This means that there is a
possibility to assign a wrong sign to the energy gap. To avoid such a
misassignment, we can appropriately set the evolution time length $t$ so that
$\cos(2\pi\Delta\phi_{jk})>0$, and use $\Delta\phi_{jk}$ when
$0\leq\Phi_{jk}\leq 1/4$ and $\Delta\phi_{jk}-1$ when $3/4\leq\phi_{jk}<1$.

##  3 Computational conditions

As a proof-of-concept demonstration of the proposed $N$-qubit QPDE algorithm,
here we report numerical simulations for the direct calculation of the
singlet–triplet energy gap of H2 molecule, and the vertical excitation
energies of halogen-substituted methylenes (CHF, CHCl, CF2, CFCl, and CCl2)
and formaldehyde (HCHO). We computed the singlet–triplet energy gap as the
excitation energy of the spin-triplet state from the spin-singlet state.
Because all the molecules under study have the spin-singlet ground state, the
singlet–triplet energy gap $\Delta
E_{\mathrm{ST}}=E(\mathrm{T_{1}})-E(\mathrm{S_{0}})$ is positive. Note that
numerical simulations of the BxB and the BPDE-based energy gap calculations
have been reported8, 9 for some of the compounds under study. In our previous
publications, the singlet–triplet energy gap of H2 was defined as $\Delta
E=E(\mathrm{S_{0}})-E(\mathrm{T_{1}})$, and therefore definition of the energy
gap is different. The one- and two-electron integrals required to construct
the electronic Hamiltonian were computed by using the GAMESS-US program
package.26 The full-CI and the CAS-CI calculations as the references were also
performed with GAMESS-US. Numerical quantum circuit simulations were performed
by using our in-house Python code developed with OpenFermion27 and Cirq28
libraries.

The singlet–triplet energy gap of H2 molecule with the atom–atom distance from
0.7 to 3.0 $\mathrm{\AA}$ was studied by using the 6-31G basis set. The wave
function is encoded in 8 qubits using JWT, and 12 ancillary qubits were used
for the phase readout. The evolution time length in the $U=exp(-iHt)$ operator
was set to $t=10$, and the second-order Trotter–Suzuki decomposition given in
eq 11 was adopted to construct the quantum circuit. We examined three
different time lengths for the single Trotter steps, $\Delta t=t/M=0.5,1.0$,
and 1.25.

$\displaystyle
e^{-iHt}=e^{-i(\sum_{j}\omega_{j}P_{j})t}=\left[\prod_{j=1}^{J}e^{-i\omega_{j}P_{j}t/2M}\prod_{j=J}^{1}e^{-i\omega_{j}P_{j}t/2M}\right]^{M}+O(t(\Delta
t)^{2})$ (11)

Here, $P_{j}$ is a Pauli string described by a direct product of Pauli
operators $\\{I,X,Y,Z\\}$, and $\omega_{j}$ is the corresponding coefficient.
It is known that the Trotter–Suzuki decomposition error depends on the term
ordering.29, 30 In this study we used a magnitude ordering29 where the Pauli
string is applied in decreasing order of $|\omega_{j}|$.

As discussed in the previous section, the probability of which energy gap can
be obtained is proportional to $|c_{j}|^{2}\times|d_{k}|^{2}$, where $c_{j}$
and $d_{k}$ are the coefficients defined in eqs 1 and 6, respectively. This
means that the accuracy of the approximated wave functions only affects the
success probability, and if success, the energy gap obtained from the
$N$-qubit QPDE does not depend on the quality of the wave functions. To study
the input wave function dependence in H2 molecule with the 6-31G basis set, we
used two different wave functions for the ground state spin-singlet wave
functions. One is the Hartree–Fock wave function $|\Phi_{\mathrm{HF}}\rangle$,
and the other is the two-configurational wave function $|\Phi_{2c}\rangle$
constructed by using a diradical character $y$. As reported in our precedent
paper,31 the $|\Phi_{2c}\rangle$ defined as in eq 12 can be a qualitatively
good approximated wave function of diradical systems with a large overlap with
the full-CI wave function. The diradical character $y$ was calculated from the
occupation number of the lowest unoccupied natural orbital
($n_{\mathrm{LUNO}}$) constructed from the spin-unrestricted broken-symmetry
Hartree–Fock wave function for the $M_{S}=0$ state, as given in eq 13.32

$\displaystyle|\Phi_{2c}\rangle=\sqrt{1-\frac{y}{2}}|2000\rangle-\sqrt{\frac{y}{2}}|0200\rangle$
(12) $\displaystyle
y=1-\frac{2(1-n_{\mathrm{LUNO}})}{1+(1-n_{\mathrm{LUNO}})^{2}}$ (13)

Here, $|2000\rangle$ is the Hartree–Fock configuration and $|0200\rangle$ is
the HONO–LUNO two-electron excited determinant from the Hartree–Fock
configuration in the H2 molecule with 6-31G basis set (4 molecular orbitals
and 2 electrons). For the spin-triplet state we used the spin-restricted open-
shell Hartree–Fock configuration $|\Phi_{1}\rangle=|\alpha\alpha 00\rangle$,
where $\alpha$ indicates that the molecular orbital is singly occupied by a
spin-$\alpha$ electron. The controlled-$Pr$ gates used for the calculations
are depicted in Figure 3.

Figure 3:  The controlled-$Pr$ gate used for the $N$-qubit QPDE calculations
of H2 and halogen-substituted methylenes. Qubits corresponding to doubly
occupied (HONO$-$1 and below) and unoccupied (LUNO$+$1 and above) orbitals are
omitted for clarity. The Pauli-X gate should be applied to the qubits
corresponding to the doubly occupied orbitals those do not appear in the
Figure. The rotation angle of the $R_{y}$ gate is set to be
$\theta=-2\arccos{\sqrt{1-y^{2}/2}}$, where $y$ is the diradical character
calculated using eqn 13.

The numerical simulations of halogen-substituted methylenes were carried out
at the CAS-CI/6-31G* level, using the B3LYP/6-31G* optimized geometry.
Geometry optimizations were performed using Gaussian 09 software.33 We used
(6e, 4o) active space for CHF and CHCl, and (10e, 6o) for CF2, CFCl, and CCl2,
and the vertical excitation energies of the lowest spin-triplet excited state
($\mathrm{1\ ^{3}B_{1}}$ state) were calculated. Here, (ke, lo) specifies that
the active space contains k-electrons and l-molecular orbitals. We used ten
ancillary qubits for the phase readout, and the evolution time length was set
to be $t=10$. The RHF and ROHF-like single determinant wave functions were
used for $|\Phi_{0}\rangle$ and $|\Phi_{1}\rangle$, respectively, and the
quantum circuit given in Figure 3 with $\theta=0$ is used for the
controlled-$Pr$ gate.

For the direct calculations of the vertical excitation energies of
formaldehyde (HCHO), we focused on the three low-lying excited states
($\mathrm{1\ ^{1}A_{2},1\ ^{1}B_{1}}$, and $\mathrm{2\ ^{1}A_{1}}$ states). We
used the same geometry and active space as our previous publication.9 The CAS-
CI active space consists of 5a1 (C–O $\sigma$), 1b1 (C–O $\pi$), 2b2 (in-plane
2p lone pair of O), 2b1 (C–O $\pi^{*}$), and 9a1 (C–O $\sigma^{*}$) orbitals.
The excited state wave functions are approximated by the spin symmetry adapted
($\mathrm{2b_{2}\rightarrow 2b_{1}}$), ($\mathrm{5a_{1}\rightarrow 2b_{1}}$),
and ($\mathrm{1b_{1}\rightarrow 2b_{1}}$) one-electron excitations from the
$|\Phi_{\mathrm{HF}}\rangle$ for the $\mathrm{1\ ^{1}A_{2},1\ ^{1}B_{1}}$, and
$\mathrm{2\ ^{1}A_{1}}$ states, respectively. The simulations were carried out
using ten ancillary qubits and $t=10$.

##  4 Results and discussion

Effect of input wave function in H2 molecule. First, we checked the effect of
the approximated wave functions on the success probability of the $N$-qubit
QPDE and the calculated eigenphase difference value, by using
$|\Phi_{\mathrm{HF}}\rangle$ and $|\Phi_{2c}\rangle$ as the input wave
functions for the spin-singlet electronic ground state of H2 molecule. We used
two different geometries $R\mathrm{(H-H)}$ = 2.0 and 3.0 $\mathrm{\AA}$ as the
representative examples of an intermediate bond-breaking and a bond-
dissociation regions, respectively. The ground state wave function exhibits
sizable diradical character ($y$ = 0.4398 and 0.8648 for $R\mathrm{(H-H)}$ =
2.0 and 3.0 $\mathrm{\AA}$, respectively), and therefore $|\Phi_{2c}\rangle$
is a much better approximation for the ground state wave function than
$|\Phi_{\mathrm{HF}}\rangle$.

Figure 4:  The phase value vs. probability plot obtained from the $N$-qubit
QPDE simulations of H2 molecule with different input wave functions for the S0
ground state.

Figure 4 summarizes the relationship between the phase value obtained from the
measurement of the ancillary qubits and the probability of occurrence. The
plots have two major peaks when the $|\Phi_{\mathrm{HF}}\rangle$ is used as
the input wave function for the singlet ground state. By converting the phase
values of the peaks at the geometry $R\mathrm{(H-H)}$ = 2.0 $\mathrm{\AA}$ to
the energy unit, we obtained the energy gaps as 0.023163 and $-0.363553$
Hartree for the major and minor peaks, respectively. These values correspond
to the $\mathrm{(T_{1}-S_{0})}$ and the $\mathrm{(T_{1}-S_{2})}$ energy gaps
($\Delta E$(full-CI/6-31G) = 0.027980 and $-0.361740$ Hartree, respectively).
Importantly, the difference in the input wave functions only affects the
success probability, and the phase value giving a peak remains unchanged.
These results support the projective nature of the $N$-qubit QPDE algorithm.

Figure 5:  The $N$-qubit QPDE simulation results of H2 molecule with different
time lengths of the single Trotter step $t/M$. (a) The singlet–triplet energy
gap. Inset is the plot of $\Delta E$ at the geometry $R(\mathrm{H-H})\geq 2.5\
\mathrm{\AA}$. (b) Differences between the singlet–triplet energy gaps from
the quantum circuit simulations and from the full-CI/6-31G calculations. AEM
stands for the energy gap calculated using the algorithmic error mitigation
technique (see text).

The singlet–triplet energy gap of H2 molecule with different bond lengths and
application of the algorithmic error mitigation. Next, we carried out the
$N$-qubit QPDE simulations for the direct calculation of the singlet–triplet
energy gap of H2 molecule by changing the atom–atom distance from 0.7 to 3.0
$\mathrm{\AA}$. The calculated energy gaps are plotted in Figure 5(a) and the
deviations from the full-CI/6-31G energy gap are given in Figure 5(b). Here,
we used the $|\Phi_{2c}\rangle$ as the input wave function of the spin-singlet
state, and the phase value giving the maximum measurement probability was
taken as the eigenphase. From Figure 5, it is clear that the deviation from
the full-CI value becomes smaller when shorter time length is adopted for the
single Trotter step. For the geometry with longer $R\mathrm{(H-H)}$ distances,
the $N$-qubit QPDE simulations with $t/M=1.00$ and 1.25 gave negative $\Delta
E$ values (see inset of Figure 5(a)), which are in contradiction with the
full-CI/6-31G results.

The plot in Figure 5(b) indicates that the $\Delta E$ values obtained from the
$N$-qubit QPDE simulations have systematic errors. We assume that the main
source of the errors is the Trotter–Suzuki decomposition. Theoretically, using
a shorter time length for the single Trotter step can systematically reduce
the error, but there is a trade-off between the Trotter–Suzuki decomposition
error and the computational cost. Instead of using a shorter time length for
the single Trotter step, here we examined an algorithmic error mitigation
(AEM) method.19 The AEM is a technique for mitigating the errors of
algorithmic origin such as the Trotter–Suzuki decomposition. It is known that
the error in the second-order Trotter–Suzuki decomposition scales as
$O(t(\Delta t)^{2})$, as given in eq 11. Since we have fixed the evolution
time length $t$ for different $\Delta t=t/M$ simulations, the energy gap
including the Trotter–Suzuki decomposition error can be fitted by a quadratic
function, $f(\Delta t)=a(\Delta t)^{2}+b$, where $f(\Delta t)$ is the energy
gap obtained from the $N$-qubit QPDE simulation with the single Trotter step
size $t/M=\Delta t$. The residue $b$ of the fitted function corresponds to the
energy gap of the Trotter–Suzuki decomposition error zero limit estimated from
the extrapolation. The results of the AEM have also been plotted in Figure
5(b) (purple stars). It is clear that the AEM efficiently reduces the error in
the energy gap estimation. In the case of H2 molecule, the maximum value of
$|\Delta E\mathrm{(AEM)}-\Delta E(\mathrm{full}$-CI)$|$ is 0.00101 Hartree =
0.633 kcal mol-1, which is below the chemical precision (1.0 kcal mol-1).

Figure 6:  The $N$-qubit QPDE simulation results of the vertical lowest
triplet excitation energies of halogen-substituted methylenes with different
time lengths of the single Trotter step $t/M$. (a) The excitation energy. (b)
Differences between the excitation energies from the quantum circuit
simulations and those from the CAS-CI/6-31G* calculations. AEM stands for the
energy gap obtained by applying the algorithmic error mitigation.

Vertical lowest-triplet excitation energies of halogen-substituted methylenes.
Next, we focused on the vertical excitation energy of the lowest spin-triplet
state ($\mathrm{1\ ^{3}B_{1}}$ state) of halogen-substituted methylenes (CHF,
CHCl, CF2, CFCl, and CCl2). Note that in the precedent paper we reported the
direct calculation of the vertical excitation energies of CF2 and CCl2
molecules using the BPDE algorithm.9

The results of the $N$-qubit QPDE simulations are plotted in Figure 6. We
observed strong system dependence of the Trotter–Suzuki decomposition errors.
In CHF, CHCl, and CCl2 molecules, the $N$-qubit QPDE simulations with three
different $t/M$ values yielded the same energy gap, and the calculated energy
gaps agree with the CAS-CI values with less than 0.01 eV of deviation. In
contrast, the Trotter–Suzuki decomposition errors are significant for CF2 and
CFCl molecules. For example, the deviations of the excitation energies
obtained from the $N$-qubit QPDE from the CAS-CI values in CF2 are 0.056,
0.240, and 0.357 eV for $t/M$ = 0.50, 1.00, and 1.25, respectively. However,
by adopting the AEM to mitigate the Trotter–Suzuki decomposition error, the
departure of the excitation energy from the CAS-CI value becomes 0.002 eV.
This result also illustrates the importance of the AEM in accurately
predicting excitation energies using the $N$-qubit QPDE algorithm.

Figure 7:  The $N$-qubit QPDE simulation results of the vertical excitation
energies of formaldehyde with different time lengths of the single Trotter
step $t/M$. AEM stands for the excitation energies obtained by applying the
algorithmic error mitigation, and Dotted horizontal lines indicate the CAS-CI
excitation energies.

Vertical excitation energies of formaldehyde. The two examples above deal with
the energy gap between the lowest singlet and the lowest triplet states. Here,
we focus on the vertical excitation energies of the spin-singlet excited
states of formaldehyde (HCHO). We have calculated the excitation energies of
three low-lying states, the $\mathrm{1\ ^{1}A_{2}}$, the $\mathrm{1\
^{1}B_{1}}$, and the $\mathrm{2\ ^{1}A_{1}}$ states. As discussed in the
Computational conditions section, these excited states can be approximated by
the spin symmetry adapted ($\mathrm{2b_{2}\rightarrow 2b_{1}}$),
($\mathrm{5a_{1}\rightarrow 2b_{1}}$), and ($\mathrm{1b_{1}\rightarrow
2b_{1}}$) one-electron excitations, respectively, from the
$|\Phi_{\mathrm{HF}}\rangle$ state. Note that direct calculations of the
excitation energies of these states have been reported by us using the BPDE
algorithm.9 The results are summarized in Figure 7. Again, the Trotter–Suzuki
decomposition causes systematic error in the excitation energies, but the
error can be efficiently removed by applying the AEM. The calculated
excitation energies and those obtained from the BPDE simulations9 are listed
in Table 1. Importantly, in the all excited states under study, the excitation
energies obtained from the $N$-qubit QPDE in conjunction with the AEM agreed
with the CAS-CI values with less than 0.006 eV = 0.138 kcal mol-1 of
deviation, which is smaller than that of the $\Delta E_{\mathrm{BPDE}}$ values
listed in Table 1. Note that the BPDE simulations9 were carried out using
$t/M=0.50$. The excitation energies calculated by using the $N$-qubit QPDE and
the BPDE methods with the same $t/M=0.50$ value are very close to each other.
The differences in the excitation energies between two calculations are mainly
due to the shot noise in the BPDE simulations and the non-projective nature of
the BPDE algorithm.

Table 1:  Vertical excitation energies of formaldehyde calculated by using the
$N$-qubit QPDE and the BPDE algorithms, and at the CAS-CI level of theory.

State | $\Delta E\mathrm{{}_{QPDE}/eV}$ | $\Delta E\mathrm{{}_{BPDE}/eV}^{[b]}$ | $\Delta E\mathrm{{}_{CAS-CI}/eV}$
---|---|---|---
$t/M=1.25$ | $t/M=1.00$ | $t/M=0.50$ | AEM[a]
$\mathrm{1\ ^{1}A_{2}}$ | 4.942 | 5.092 | 5.293 | 5.360 | 5.297 | 5.359
$\mathrm{1\ ^{1}B_{1}}$ | 10.135 | 10.268 | 10.469 | 10.530 | 10.466 | 10.525
$\mathrm{2\ ^{1}A_{1}}$ | 11.053 | 11.270 | 11.588 | 11.686 | 11.603 | 11.692

[a]The energy gap obtained by applying the algorithmic error mitigation.
[b]Ref9.

##  5 Conclusions

In this work, we have developed a quantum phase difference estimation
algorithm with $N_{a}$ of ancillary qubits for the direct calculations of
energy gaps on a quantum computer. Although the number of ancillary qubits
required to run the algorithm is larger than the previously proposed Bayesian
inference-based implementations,9, 10, 11, 12 the proposed approach is based
on the projective measurement and therefore it is able to calculate the
difference of energy eigenvalues of two electronic states with the
approximated wave functions as inputs. The quality of the approximated wave
functions only affects the success probability of the calculation, and the
calculated energy gap value is independent of the input wave functions. The
calculated energy gaps show systematic errors mainly caused by the
Trotter–Suzuki decomposition. We demonstrated that the error is effectively
reduced by applying the algorithmic error mitigation technique. For all the
molecules being studied, the error-mitigated energy gap agreed to the full-CI
or CAS-CI reference value with less than 1 kcal mol-1 of deviations. Since
sophisticated wave function preparation methods such as an adiabatic state
preparation require deep quantum circuit, the single-shot protocol developed
in this work is suitable to connect with such approaches. The $N$-qubit QPDE
method with the input wave functions generated by using ASP is in progress and
will be discussed in the forthcoming paper.

This work was supported by the Center of Innovations for Sustainable Quantum
AI (Grant No. JPMJPF2221) from JST, Japan, Quantum Leap Flagship Program
(JPMXS0120319794) from the MEXT, Japan, and Grants-in-Aid for Scientific
Research C (21K03407) and for Transformative Research Area B (23H03819) from
JSPS, Japan. Partial support by the PRESTO project ”Quantum Software”
(JPMJPR1914) from JST, Japan, is also acknowledged.

## References

  * Dirac 1929 Dirac, P. A. M. Quantum mechanics of many-electron systems. _Proc. R. Soc. London, Ser. A_ 1929, _123_ , 714–733
  * Aspuru-Guzik et al. 2005 Aspuru-Guzik, A.; Dutoi, A. D.; Love, P. J.; Head-Gordon, M. Simulated quantum computation of molecular energies. _Science_ 2005, _309_ , 1704–1707
  * Abrams and Lloyd 1999 Abrams, D. S.; Lloyd, S. Quantum algorithm providing exponential speed increase for finding eigenvalues and eigenvectors. _Phys. Rev. Lett._ 1999, _83_ , 5162–5165
  * Lee et al. 2023 Lee, S.; Lee, J.; Zhai, H.; Tong, Y.; Dalzell, A. M.; Kumar, A.; Helms, P.; Gray, J.; Cui, Z.-H.; Liu, W.; Kastoryano, M.; Babbush, R.; Preskill, J.; Reichman, D. R.; Campbell, E. T.; Valeev, E. F.; Lin, L.; Chan, G. K.-L. Evaluating the evidence for exponential quantum advantage in ground-state quantum chemistry. _Nat. Comm._ 2023, _14_ , 1952
  * Matsuzaki et al. 2021 Matsuzaki, Y.; Hakoshima, H.; Sugisaki, K.; Seki, Y.; Kawabata, S. Direct estimation of the energy gap between the ground state and the excited state with quantum annealing. _Jpn. J. Appl. Phys._ 2021, _60_ , SBBI02
  * Yang et al. 2023 Yang, Y.; Li, Y.; Xu, X.; Yuan, X. A resource-efficient quantum-classical hybrid algorithm for energy gap evaluation. arXiv:2305.07382, 2023
  * Russo et al. 2021 Russo, A. E.; Rudinger, K. M.; Morrison, B. C. A.; Baczewski, A. D. Evaluating energy differences on a quantum computer with robust phase estimation. _Phys. Rev. Lett._ 2021, _126_ , 210501
  * Sugisaki et al. 2021 Sugisaki, K.; Toyota, K.; Sato, K.; Shiomi, D.; Takui, T. A quantum algorithm for spin chemistry: a Bayesian exchange coupling parameter calculator with broken-symmetry wave functions. _Chem. Sci._ 2021, _12_ , 2121–2132
  * Sugisaki et al. 2021 Sugisaki, K.; Sakai, C.; Toyota, K.; Sato, K.; Shiomi, D.; Takui, T. Bayesian phase difference estimation: a general quantum algorithm for the direct calculation of energy gaps. _Phys. Chem. Chem. Phys._ 2021, _23_ , 20152–20162
  * Sugisaki et al. 2021 Sugisaki, K.; Sakai, C.; Toyota, K.; Sato, K.; Shiomi, D.; Takui, T. Quantum algorithm for full configuration interaction calculations without controlled time evolutions. _J. Phys. Chem. Lett._ 2021, _12_ , 11085–11089
  * Sugisaki et al. 2022 Sugisaki, K.; Wakimoto, H.; Toyota, K.; Sato, K.; Shiomi, D.; Takui, T. Quantum algorithm for numerical energy gradient calculations at the full configuration interaction level of theory. _J. Phys. Chem. Lett._ 2022, _13_ , 11105–11111
  * Sugisaki et al. 2022 Sugisaki, K.; Prasannaa, V. S.; Ohshima, S.; Katagiri, T.; Mochizuki, Y.; Sahoo, B. K.; Das, B. P. Bayesian phase difference estimation algorithm for direct calculation of fine structure splitting: accelerated simulation of relativistic and quantum many-body effects. arXiv:2212.02058, 2022
  * O’Brien et al. 2019 O’Brien, T. E.; Tarasinski, B.; Terhal, B. M. Quantum phase estimation of multiple eigenvalues for small-scale (noisy) experiments. _New J. Phys._ 2019, _21_ , 023022
  * Gebhart et al. 2021 Gebhart, V.; Smerzi, A.; Pezzè, L. Bayesian quantum multiphase estimation algorithm. _Phys. Rev. Appl._ 2021, _16_ , 014035
  * Ding and Lin 2023 Ding, Z.; Lin, L. Simultaneous estimation of multiple eigenvalues with short-depth quantum circuit on early fault-tolerant quantum computers. arXiv:2303.05714, 2023
  * Wiebe and Granade 2016 Wiebe, N.; Granade, C. Efficient Bayesian phase estimation. _Phys. Rev. Lett._ 2016, _117_ , 010503
  * Paesani et al. 2017 Paesani, S.; Gentile, A. A.; Santagati, R.; Wang, J.; Wiebe, N.; Tew, D. P.; O’Brien, J. L.; Thompson, M. G. Experimental Bayesian quantum phase estimation on a silicon photonic chip. _Phys. Rev. Lett._ 2017, _118_ , 100503
  * Yamamoto et al. 2023 Yamamoto, K.; Duffield, S.; Kikuchi, Y.; Muñoz Ramo, D. Demonstrating Bayesian quantum phase estimation with quantum error detection. arXiv:2306.16608, 2023
  * Endo et al. 2019 Endo, S.; Zhao, Q.; Li, Y.; Benjamin, S.; Yuan, X. Mitigating algorithmic errors in a Hamiltonian simulation. _Phys. Rev. A_ 2019, _99_ , 012334
  * Jordan and Wigner 1928 Jordan, P.; Wigner, E. Über das Paulische äquivalenzverbot. _Z. Phys._ 1928, _47_ , 631–651
  * Seeley et al. 2012 Seeley, J. T.; Richard, M. J.; Love, P. J. The Bravyi–Kitaev transformation for quantum computation of electronic structure. _J. Chem. Phys._ 2012, _137_ , 224109
  * Veis and Pittner 2014 Veis, L.; Pittner, J. Adiabatic state preparation of methylene. _J. Chem. Phys._ 2014, _140_ , 214111
  * Kremenetski et al. 2021 Kremenetski, V.; Mejuto-Zaera, C.; Cotton, S. J.; Tubman, N. M. Simulation of adiabatic quantum computing for molecular ground state. _J. Chem. Phys._ 2021, _155_ , 234106
  * Sugisaki et al. 2022 Sugisaki, K.; Toyota, K.; Sato, K.; Shiomi, D.; Takui, T. Adiabatic state preparation of correlated wave functions with nonlinear scheduling functions and broken-symmetry wave functions. _Comm. Chem._ 2022, _5_ , 84
  * Nielsen and Chuang 2010 Nielsen, M. A.; Chuang, I. L. _Quantum Computation and Quantum Information, 10th Anniversary Edition_ ; Cambridge University Press, 2010
  * Barca et al. 2020 Barca, G. M. J.; Bertoni, C.; Carrington, L.; Datta, D.; De Silva, N.; Emiliano Deustua, J.; Fedorov, D. G.; Gour, J. R.; Gunina, A. O.; Guidez, E.; Harville, T.; Irle, S.; Ivanic, J.; Kowalski, K.; Leang, S. S.; Li, H.; Li, W.; Lutz, J. J.; Magoulas, I.; Mato, J.; Mironov, V.; Nakata, H.; Pham, B. Q.; Piecuch, P.; Poole, D.; Pruitt, S. R.; Rendell, A. P.; Roskop, L. B.; Ruedenberg, K.; Sattasathuchana, T.; Schmidt, M. W.; Shen, J.; Slipchenko, L.; Sosonkina, M.; Sundriyal, V.; Tiwari, A.; Galvez Vallejo, J. L.; Westheimer, B.; Wloch, M.; Xu, P.; Zahariev, F.; Gordon, M. S. Recent developments in the general atomic and molecular electronic structure system. _J. Chem. Phys._ 2020, _152_ , 154102
  * McClean et al. 2020 McClean, J. R.; Rubin, N. C.; Sung, K. J.; Kivlichan, I. D.; Bonet-Monroig, X.; Cao, Y.; Dai, C.; Schuyler Fried, E.; Gidney, C.; Gimby, B.; Gokhale, P.; Häner, T.; Hardikar, T.; Havlìček, V.; Higgott, O.; Huang, C.; Izaac, J.; Jiang, Z.; Liu, X.; McArdle, S.; Neeley, M.; O’Brien, T.; O’Gorman, B.; Ozfidan, I.; Radin, M. D.; Romero, J.; Sawaya, N. P. D.; Senjean, B.; Setia, K.; Sim, S.; Steiger, D. S.; Steudtner, M.; Sun, Q.; Sun, W.; Wang, D.; Zhang, F.; Babbush, R. OpenFermion: the electronic structure package for quantum computers. _Quantum Sci. Technol._ 2020, _5_ , 034014
  * Cirq Developers 2022 Cirq Developers, Cirq (v1.1.0). Zenodo, 2022; https://doi.org/10.5281/zenodo.7465577
  * Tranter et al. 2018 Tranter, A.; Love, P. J.; Mintert, F.; Coveney, P. V. A comparison of the Bravyi–Kitaev and Jordan–Wigner transformations for the quantum simulation of quantum chemistry. _J. Chem. Theory Comput._ 2018, _14_ , 5617–5630
  * Tranter et al. 2019 Tranter, A.; Love, P. J.; Mintert, F.; Wiebe, N.; Coveney, P. V. Ordering of Trotterization: Impact on errors in quantum simulation of electronic structure. _Entropy_ 2019, _21_ , 1218
  * Sugisaki et al. 2019 Sugisaki, K.; Nakazawa, S.; Toyota, K.; Sato, K.; Shiomi, D.; Takui, T. Quantum chemistry on quantum computers: a method for preparation of multiconfigurational wave functions on quantum computers without performing post-Hartree–Fock calculations. _ACS Cent. Sci._ 2019, _5_ , 167–175
  * Döhnert and Kouteckỳ 1980 Döhnert, D.; Kouteckỳ, J. Occupation numbers of natural orbitals as a criterion of biradical character. Different kind of biradicals. _J. Am. Chem. Soc._ 1980, _102_ , 1789–1796
  * Frisch et al. 2010 Frisch, M. J.; Trucks, G. W.; Schlegel, H. B.; Scuseria, G. E.; Robb, M. A.; Cheeseman, J. R.; Scalmani, G.; Barone, V.; Petersson, G. A.; Nakatsuji, H.; Li, X.; Caricato, M.; Marenich, A.; Bloino, J.; Janesko, B. G.; Gomperts, R.; Mennucci, B.; Hratchian, H. P.; Ortiz, J. V.; Izmaylov, A. F.; Sonnenberg, J. L.; Williams-Young, D.; Ding, F.; Lipparini, F.; Egidi, F.; Goings, J.; Peng, B.; Petrone, A.; Henderson, T.; Ranasinghe, D.; Zakrzewski, V. G.; Gao, J.; Rega, N.; Zheng, G.; Liang, W.; Hada, M.; Ehara, M.; Toyota, K.; Fukuda, R.; Hasegawa, J.; Ishida, M.; Nakajima, T.; Honda, Y.; Kitao, O.; Nakai, H.; Vreven, T.; Throssell, K.; Jr., J. A. M.; Peralta, J. E.; Ogliaro, F.; Bearpark, M.; Heyd, J. J.; Brothers, E.; Kudin, K. N.; Staroverov, V. N.; Keith, T.; Kobayashi, R.; Normand, J.; Raghavachari, K.; Rendell, A.; Burant, J. C.; Iyengar, S. S.; Tomasi, J.; Cossi, M.; Millam, J. M.; Klene, M.; Adamo, C.; Cammi, R.; Ochterski, J. W.; Martin, R. L.; Morokuma, K.; Farkas, O.; Foresman, J. B.; Fox, D. J. Gaussian 09, Revision B.01, 2010