# TellMeWhy: A Dataset for Answering Why-Questions in Narratives

Yash Kumar Lal
Stony Brook University
<EMAIL_ADDRESS>
Nathanael Chambers
US Naval Academy
<EMAIL_ADDRESS>Raymond Mooney
University of Texas, Austin
<EMAIL_ADDRESS>
Niranjan Balasubramanian
Stony Brook University
<EMAIL_ADDRESS>
###### Abstract

Answering questions about why characters perform certain actions is central to
understanding and reasoning about narratives. Despite recent progress in QA,
it is not clear if existing models have the ability to answer “why” questions
that may require commonsense knowledge external to the input narrative. In
this work, we introduce TellMeWhy, a new crowd-sourced dataset that consists
of more than 30k questions and free-form answers concerning why characters in
short narratives perform the actions described. For a third of this dataset,
the answers are not present within the narrative. Given the limitations of
automated evaluation for this task, we also present a systematized human
evaluation interface for this dataset. Our evaluation of state-of-the-art
models show that they are far below human performance on answering such
questions. They are especially worse on questions whose answers are external
to the narrative, thus providing a challenge for future QA and narrative
understanding research.

##  1 Introduction

The actions people perform are steps of plans to achieve their desired goals.
When interpreting language, humans naturally understand the reasons behind
described actions, even when the reasons are left unstated Schank and Abelson
(1975). For NLP systems, answering questions about _why_ people perform
actions in a narrative can test this ability. Answering such questions often
requires filling the implicit gaps in the story itself.

Consider this narrative from ROCStories Mostafazadeh et al. (2016b):

Rudy was convinced that bottled waters all tasted the same. He went to the
store and bought several popular brands. He went back home and set them all on
a table. He spent several hours tasting them one by one. He came to the
conclusion that they actually did taste different.

Now try to answer the question, _“Why did he go to the store and buy several
popular brands?”_ The answer _“he wanted to taste test”_ is not explicit in
the narrative and requires us to read between the lines to fill in the gaps
Norvig (1987). While humans can visualise and process the events in a story to
hypothesize why they might have occurred Kintsch and Dijk (1978), current NLP
systems fall well short of exhibiting similar capabilities. They are unable to
adequately formulate the reasons behind actions in specific contexts.

How can we get NLP models to reason about why actions are performed? One way
is to consider theories like script learning Schank and Abelson (1975);
Pichotta and Mooney (2014) or learning from co-occurrence Chambers and
Jurafsky (2009). But they only partially capture this type of knowledge – much
like other forms of commonsense knowledge, the reasons for why actions are
performed are often left implicit in text. Even though there are many large
scale QA datasets, they rarely contain questions about why people perform
actions.

Therefore, we introduce the TellMeWhy dataset, a collection of 30,519 such
why-questions, each with 3 “gold standard” human answers. Each record in
TellMeWhy contains a short story, an associated question, and its 3 possible
answers.

Further, we focus on enabling _human_ evaluation of this dataset; human
evaluation is more reliable than automatic metrics to evaluate such systems
Celikyilmaz et al. (2020); Gatt and Krahmer (2018). However, reliability of
human judgment is substantially impacted by experimental setup Novikova et al.
(2018); Santhanam and Shaikh (2019). There is little consensus on how human
evaluations should be conducted, so results are often incomparable across
evaluations.

To this end, we present a systematized evaluation framework on MTurk for the
TellMeWhy text generation task – and release the framework for future
researchers. The MTurk interface asks annotators to rate generated answers on
their grammaticality and validity. We show that with our interface human
answers are judged to be of high quality (99% grammatical, 96% valid) with
strong inter-annotator agreement at 0.88 Fleiss Kappa. This indicates high
agreement and also confirms the design of our interface.

Finally, we present baseline results for TellMeWhy and compare against our
human ceiling. We finetune two large language models that have proven to be
effective for a variety of tasks, GPT-2 Radford et al. (2019) and T5 Raffel et
al. (2020), and a dedicated question answering model, UnifiedQA Khashabi et
al. (2020), to perform this task. Human evaluation is performed on their
outputs from independent test data. All models significantly under-perform the
human benchmark and are especially worse on questions where the answer cannot
be simply copied over from text in the narrative. The results clearly
demonstrate the difficulty for current models to convincingly answer such why-
questions.

This paper’s contributions are as follows: (1) we introduce TellMeWhy, a large
dataset of English why-questions for narratives derived from ROCStories
Mostafazadeh et al. (2016a) and CATERS Mostafazadeh et al. (2016b) along with
answers from 3 distinct humans, (2) a systematized human evaluation interface
to calibrate model outputs consistently, and (3) show that current models are
ill-equipped to perform this task. We release the dataset and evaluation suite
at http://lunr.cs.stonybrook.edu/tellmewhy.

##  2 Related Work

###  2.1 Datasets containing why-questions

Most of the datasets related to why-questions fall into one or more of the
following categories: (1) very small size, (2) not focused on stories, or (3)
focused on connecting known events instead of answering reasoning questions.

Some corpora of why-questions have been collected manually: corpora described
in Verberne et al. (2006) and Verberne et al. (2007) both comprise fewer than
400 questions and corresponding answers (one or two per question) formulated
by native speakers. Dunietz et al. (2020) demonstrate that it is important to
define what we want models to comprehend when building datasets for machine
reading comprehension (MRC) tasks. They design templates of understanding
corresponding to the four elements identified by Zwaan et al. (1995). For 201
questions, they design multiple-choice questions derived from Lai et al.
(2017) to test understanding of different categories of events. All of these
are very small corpora that cannot be viably used to further a model’s
understanding of why-questions in stories.

Higashinaka and Isozaki (2008) extend an existing factoid QA system to answer
why-questions by integrating corpus based features, calling it NAZEQA. Oh et
al. (2012) extract a set of answer candidates from a web corpus, and perform
re-ranking using SVMs to predict the right answer. Oh et al. (2019) use an
adversarial learning framework to generate a vector representation from the
passage to judge whether the passage actually answers the why-question. These
papers focus on Japanese news Fukumoto et al. (2007); Oh et al. (2012),
including NTCIR-6, and most critically, all these datasets are very small.

Some prior work focuses on knowledge extraction, not the _reasons_ behind the
actions. Mrozinski et al. (2008) built a corpus of why-questions related to
Wikipedia articles. These were general knowledge questions with solicited
answers from paid workers. Dependency parsing can be used to rephrase why-
questions into statements with a ‘because’ prompt to elicit explanations from
models Nie et al. (2019). PhotoshopQuiA Dulceanu et al. (2018) contains
questions and answers specifically about Photoshop.

NarrativeQA Kočiský et al. (2018) provides a dataset of 1,567 stories (books
and movie scripts) containing 46,765 wh-questions written and answered by
human annotators. Unfortunately, only 9.78% are why-questions, which makes for
a small collection. QuAIL Rogers et al. (2020) has a small subset of multiple
choice questions pertaining to causality in user stories. These datasets are
targeted at broad abilities of reading comprehension, not specifically about
explaining actions in stories.

Dataset | Size | Domain
---|---|---
NTCIR-6 | 200 | Japanese news
Mrozinski et al. (2008) | 695 | Wikipedia
PhotoshopQuiA | 2,854 | Product focused
NarrativeQA | 4,573 | Books+Movie scripts
Dunietz et al. (2020) | 201 | Exam questions
Table 1: Previous why-question corpora. NarrativeQA has 46,765 questions of
which 4,573 are why-questions.

Some recent datasets causally connect events in text, but they do not target
answering why-questions. ATOMIC Sap et al. (2019) consists of entries that
describe a likely cause/effect of events. Most notably, ATOMIC is _non-
contextual_ so it is more about general knowledge, not interpreting a specific
story/context. Perhaps most relevant is GLUCOSE Mostafazadeh et al. (2020), a
crowdsourced dataset of implicit commonsense knowledge in the form of causal
mini-theories grounded in narrative context. These theories are semi-
structured inference rules. This dataset is not aimed at answering why-
questions, but at creating direct relationships between events already
mentioned in the story. They focus on capturing specific cause-enable type
relations. Annotators were given a very constrained task – they had to select
options from a drop down menu describing inference rules.

Abductive commonsense reasoning tests whether models can come up with a
plausible explanation to connect a set of events. Bhagavatula et al. (2020)
present ART with two abductive tasks: 1) given two observations, select one
out of two plausible hypotheses, 2) and generate text connecting two events.
This line of work focuses on connecting the dots between two events and does
not address explaining why an action was performed. Our work crucially differs
from these because the answer is often not in the story at all. StrategyQA
Geva et al. (2021) is a new dataset focusing on performing better implicit
reasoning for multi-hop question answering tasks.

We summarize the different why-questions corpora in Table 1. None of them
represent a large dataset focused on answering why-questions about actions in
a narrative.

###  2.2 Human evaluation for NLG tasks

Among language generation tasks, machine translation has received the most
attention in terms of human evaluation. Qualified crowd workers score output
translations given the source or reference text to calibrate MT systems
Sakaguchi and Van Durme (2018); Graham et al. (2013, 2014). WMT conducts
annual evaluation of outputs of systems submitted to the shared task and uses
it as one of the primary metrics (along with BLEU) to rank systems Bojar et
al. (2016, 2017, 2018); Barrault et al. (2019, 2020).

ChatEval Sedoc et al. (2019) is an evaluation platform for chatbots. Zellers
et al. (2020) present a leaderboard for their advice generation task. These
platforms incorporate some manual analysis, but focus on very different tasks.
None of their Mechanical Turk interfaces can be used for our task. We were
unable to find a consistent interface for human evaluation of an open-ended
question answering task. To address this flaw, we propose a standard human
intelligence task (HIT) evaluation scheme for our dataset.

##  3 Dataset Creation

We want to test the abilities of models to understand the reasoning behind
actions in a story. Therefore, we create a dataset of _why_ questions that ask
for explanations for actions performed in a story. Answering these questions
requires an understanding of the events that are explicit in the story as well
as access to implicit common-sense knowledge on how people use actions as
parts of plans to achieve goals. To cover a wide-range of common situations,
we utilize ROCStories Mostafazadeh et al. (2016a), a collection of 45,496
five-sentence commonsense stories. We also develop a small “hidden” test set
that was only used for the final evaluation using the CATERS Mostafazadeh et
al. (2016b) subset of ROCStories.

###  3.1 Why-Question Generation

Our strategy for creating why questions is simple. For each action in the
narrative, we formulate a why question by applying simple template-based
transformations. We dependency parse each sentence using SpaCy’s
en_core_web_sm model Honnibal et al. (2020). We use the generated parse tree
to rephrase the sentence into a question about the action described. The
generated parse tree is used to extract the subject, object, and verb. We
consider 3 types of sentences and design question templates accordingly: (1)
sentences that have a primary and auxiliary verb, (2) sentences that only have
a primary verb, and (3) sentences that only contain an auxiliary verb. For the
first, the question template is: “Why {aux_verb} {subject} {verb} {object}?",
for the second: “Why did {subject} {verb_lemma} {obj}?", and for the third;
“Why {aux_verb} {subject} {obj}?".

This procedure yielded a little over 113k questions from ROCStories, and 489
questions from the CATERS portion. We selected at random 32,165 questions from
stories that had at least three questions111Since we ask annotators to read an
entire story to answer these questions, avoiding stories with fewer questions
optimizes reading time.. We ensure that there is no overlap between the two
subsets.

Split | # stories | # questions
---|---|---
Train | 7558 | 23964
Dev | 944 | 2992
Test | 944 | 3099
Hidden Test | 190 | 464
Total | 9,636 | 30,519
Table 2: Dataset Statistics

###  3.2 Collecting Answers

We crowd-sourced answers to these questions using Amazon Mechanical Turk.
Figure 1(a) shows the interface used to collect these answers. Annotators were
presented a narrative and asked to answer three why questions in free-form.
For each question, they were also asked to provide judgments about the
comprehensibility of the question, and whether the narrative explicitly
contained the answer. They also selected the sentences from the narrative
which influenced their answer (if any). To avoid variability in answer
prefixes, we provide a prompt to start answering the question. We rephrase the
sentence from which the question was generated to create these prompts. We
consider the same categorisation of sentences described in subsection 3.1. For
sentences that have both primary and auxiliary verbs, the answer prompt is of
the form: “{subject} {aux_verb} {verb} {object} because…". When it only
contains a primary verb, it is of the form: “{subject} {verb} {object}
because…". If it only contains an auxiliary verb, it is of the form:
“{subject} {aux_verb} {object} because…". We found, over several iterations of
this HIT, that providing a prompt gave workers an initial direction and
improved the quality of answers collected.

(a) Task 1: Answer collection

(b) Task 2: Answer validation

Figure 1: MTurk interfaces used to curate data from crowd-source workers

We ask three distinct annotators (three-way redundant task) to answer each of
these questions. Annotators are not allowed to copy pieces of text to make up
an answer. We discard questions that were deemed incomprehensible by any
annotator.222On ROCStories we discarded 1,546 questions and on CATERS we
discarded 25 questions With this process, we obtained 3 answers each from
30,055 questions from 9,636 stories (see Appendix B for more details). Table 2
shows basic statistics of the dataset. We refer to annotations from the CATERS
data as the hidden test set. Examples of records in the dataset are presented
in Table 3. The narrative does not explicitly contain an answer for the second
question. We call these types the implicit-answer questions; they require
extra common-sense inference to produce a plausible answer. Questions are
categorised as implicit-answer if at least 2 out of 3 human annotators
indicate that the answer cannot be explicitly found in the narrative. The
annotators indicated as much and, based on their commonsense knowledge,
provided plausible answers.

Story: Sandra got a job at the zoo. She loved coming to work and seeing all of
the animals. Sandra went to look at the polar bears during her lunch break.
She watched them eat fish and jump in and out of the water. She took pictures
and shared them with her friends.
---
Question: Why did Sandra go to look at the polar bears during her lunch break?
Ans: she wanted to take some pictures of them.
Story: Cam ordered a pizza and took it home. He opened the box to take out a
slice. Cam discovered that the store did not cut the pizza for him. He looked
for his pizza cutter but did not find it. He had to use his chef knife to cut
a slice.
Question: Why did Cam order a pizza?
Ans: Cam was hungry.
Table 3: TellMeWhy examples. The first is answerable directly from text in the
story, but the second requires external knowledge. We only show one out of
three available answers here.

###  3.3 Validating Answers

To ensure an even higher-quality test set, we conducted another round of
crowdsourcing to validate the answers by the first set of crowd-workers on the
CATERS portion (464 questions). This validation interface is show in Figure
1(b). It also serves as the base design for our systematized human evaluation.
Annotators are presented a story, a related question, and the three answers
that were collected as described in Section 3.2.

Three new annotators then rated two aspects of each answer:
(1) Grammaticality – Workers are asked to rate the grammaticality of each
answer on 5-point Likert scales, ranging from ‘Strongly Ungrammatical’ to
‘Strongly Grammatical’. An answer is strongly grammatical if it follows all
the rules of English grammar. It is grammatical if there is a mistake in
tense, number, punctuation or something minor. It is comprehensible if there
are clear grammatical mistakes but its meaning can be inferred, and it is then
considered to be neutral on the Likert scale.

(2) Validity – Workers are asked to rate the validity of each answer on a
5-point Likert scale. Given the story and question, the annotators check if
the given answer ‘is valid and makes sense with the story’. An answer is
considered invalid if it does not give a plausible reason relevant to the
question asked and instead states irrelevant information.

Annotators agreed (by majority) that 99.07% of answers are grammatical and
95.47% of answers are valid. On grammaticality, there is some disagreement in
judgment 0.7% of the time, while there is some disagreement in judgment 1% of
the time for answer validity. We measured the inter-rater reliability of
annotators’ judgments using weighted Fleiss’s Kappa Marasini et al. (2016) and
follow the weighting scheme used by Bastan et al. (2020). This measure has a
penalty for each dissimilar classification based on the distance between two
classes. For instance, if two annotators classify a document as a positive,
the agreement weight is 1, but if one classifies as a positive, and the other
classifies as slightly positive the agreement weight is less. The weighted
agreement score for this subset is 0.88 for grammaticality annotations and
0.81 for validity annotations, indicating that the annotations are highly
reliable. More details can be found in Appendix C.2.

##  4 Dataset Analysis

One of the key distinguishing aspects of answering why questions is that, in
addition to understanding explicitly stated events, they also require access
to commonsense explanations that may be external to the narrative. We conduct
some analyses to investigate the prevalence of this phenomenon: (i) We asked
annotators to judge whether the answer to a question could be found stated
explicitly or only implicitly in the narrative and find that at least two out
of three annotators could not find explicit answers in the story 28.82% of the
time. (ii) We also asked crowd-workers to indicate which sentences helped them
answer the question. Out of 91,557 collected answers, we find that 39,661
answers were provided without an influential sentence from the story. (iii)
Last, we observe that there is only a 57.04% lexical overlap between the words
used in answers and the original narrative. This suggests that annotators
included new inferred information in their answers, instead of just copying
something from the story. We calculate lexical overlap as the number of common
tokens in the narrative and the answer divided by the length of the answer.

We hypothesize that questions about the first action in a story are more
difficult to answer since there is no prior information to provide an explicit
answer. We find that 55.03% of such questions were judged to be implicit-
answer questions by a majority of the assigned annotators. Such questions help
test systems’ ability to infer plausible answers rather than just copy answers
from the text.

Evaluated on | Model | BLEU | RG-L F1 | BLEURT | BertScore
---|---|---|---|---|---
Full Test Set | GPT-2-OO | 4.45 | 0.07 | -1.23 | -0.55
GPT2-FT | 3.96 | 0.13 | -0.75 | 0.18
T5-OO | 9.89 | 0.13 | -0.963 | 0.23
T5-FT | 24.53 | 0.24 | -0.28 | 0.48
UnifiedQA | 21.97 | 0.25 | -0.30 | 0.43
Implicit-Answer Qs in Test Set | GPT-2-OO | 4.45 | 0.06 | -1.22 | -0.54
GPT2-FT | 3.89 | 0.12 | -0.805 | 0.17
T5-OO | 8.14 | 0.11 | -0.99 | 0.22
T5-FT | 16.31 | 0.17 | -0.51 | 0.38
UnifiedQA | 14.6 | 0.18 | -0.50 | 0.34
Table 4: Performance of models on the full test set and on implicit-answer
questions in the test set using automated metrics. RG-L denotes ROUGE-L. The
OO suffix denotes the vanilla version of the model while the FT version
denotes the finetuned version.

We also evaluated the diversity of the answers for each question using simple
lexical overlap. Of the 30k questions, only 150 questions had over 90% overlap
in all 3 answers, i.e., essentially, the 3 distinct annotators wrote the same
answer. For 4,243 other questions, two out of three answers had over 90%
overlap. But for the vast majority of 26,068 questions, we obtained 3 fairly
diverse answers. The average overlap between them is 26.12%. On average, the
answers were 7.59 words long.

Overall, this analysis indicates how TellMeWhy differs from prior datasets.
The answers cannot always be retrieved or connected to other events in the
given text.

##  5 Benchmarking

How well do large language models answer why questions on narratives and what
are their failure modes? To answer these, we use TellMeWhy to benchmark the
performance of multiple state-of-the-art models and provide an analysis of
their performance.

Formally, given a story S as context and a related why-question Q, models are
required to generate a plausible answer A for the question. Since the answers
are open-ended texts we compare them on standard automatic evaluation metrics
for generation but also conduct a human evaluation.

###  5.1 QA Models

GPT-2 Radford et al. (2019) is a large transformer-based language model
trained on an enormous web corpus, which has been shown to be effective on a
wide-range of language related tasks including question answering. It was one
of the first models trained on diverse data to outperform domain-specific
language models.

We used Huggingface Wolf et al. (2020) to finetune a pretrained GPT-2 model on
TellMeWhy. As input, the model receives a concatenation of the narrative and
the related question (in that order), and the target is the answer. The input
and target are separated using the ‘[SEP]’ token. We finetune the model with
batch size 16, learning rate 1e-5 and maximum output length 25. The model is
trained until the dev loss fails to improve for 3 iterations.

T5 Raffel et al. (2020) is an encoder-decoder model pre-trained on a mixture
of unsupervised and supervised tasks in a multi-task setting, where each task
is converted into a text-to-text format. It is a text-to-text model, which
means it can be trained on arbitrary tasks involving textual input and output.
T5 has achieved the state of the art on many natural language understanding
(NLU) tasks. More details about hyperparameter sweeps can be found in Appendix
A.

We finetuned a pretrained T5-base model from HuggingFace Wolf et al. (2020) on
TellMeWhy. Since it is a natural language generation task related to a story,
we use the SQuAD format specified in Appendix D.15 of Raffel et al. (2020) to
format our inputs. Our narrative serves as the ‘context’ and the why-question
is used as the ‘question’ in the selected input format. We train the model
with batch size 16, learning rate 5e-5, maximum source length 75 and maximum
answer length 30. The model is trained until the dev loss fails to improve for
3 iterations.

UnifiedQA Khashabi et al. (2020) is a single pre-trained model that performs
well across 20 different question answering datasets. It is built on top of a
T5 model and simplifies finetuning by unifying the various formats used by T5.
Its ability to perform both extractive and abstractive QA tasks makes it a
suitable candidate for calibrating this task. A pretrained version of this
model is available via HuggingFace Wolf et al. (2020) under the name
“allenai/unifiedqa-t5-base". The input format for this model is simple, just
requiring the question and the narrative to be separated by a newline symbol.
We train this model using learning rate 1e-5 (same as the original paper) and
retain other hyperparameters from finetuning T5 as described above.

###  5.2 Automatic Evaluation

We evaluate all of the above models on both the test set and the hidden test
set (questions from CATERS data). For automatic evaluation, we report BLEU
Papineni et al. (2002), ROUGE-L Lin (2004), BLEURT Sellam et al. (2020) scores
using the bluert-base-128 checkpoint, and BertScore Zhang* et al. (2020) using
the default roberta-large checkpoint. These numbers are presented in Table 4.

We select one human answer at a time and (using SacreBLEU Post (2018))
calculate the BLEU scores for model output with all three references, and
select the maximum. Since BLEURT is a sentence level metric, to calculate the
reported BLEURT, we average all the (output, reference) scores to obtain a
corpus score for each reference. We then select the maximum BLEURT corpus
score over all 3 human references. It is important to note that BLEURT was
proposed as a metric for relative comparison, not absolute calibration. We
also report BertScore F1333idf and rescale_with_baseline flags are set to
True. idf is calculated over all gold answers in the test set. Zhang* et al.
(2020) as another semantic automatic evaluation metric. We report a max
BertScore in the same way as BLEURT and BLEU: by taking the maximum score of
the model output with each human answer taken one at a time.

Vanilla model results are obtained by loading an existing pretrained model
from HuggingFace and running inference with the input formats described above.
They are not trained on TellMeWhy. We see that vanilla pretrained models are
unable to perform this task at all. Finetuning a pretrained model results in
improvements since it better models the relationship between the story, the
question, and a possible answer. On the full test set, the finetuned T5 model
performs the best on our task. In Table 4, we also see that models perform a
lot worse on implicit-answer questions.

###  5.3 Human Evaluation

For open-ended text generation tasks like answering why-questions, the absence
of an automatic evaluation that correlates well with human judgments is a
major challenge Chen et al. (2019); Ma et al. (2019); Caglayan et al. (2020);
Howcroft et al. (2020).

We conduct a human evaluation on the hidden test set with a standardized
interface to compare different models. We want to measure whether a model
produces coherent and grammatical output and more importantly, whether the
produced output is a valid answer for the given question. Our validation HIT
subsection 3.3 showed a way to conduct human evaluation of answers provided by
other crowd-workers. We modified this HIT design to evaluate generated answers
from models. For a given question, we present just one answer from a single
model and then ask the crowd-workers to assess its grammaticality and
validity.

For each story, question, and a model’s answer, we ask 3 distinct annotators
to provide judgments about grammaticality and validity. This serves as the
human evaluation interface for our task. A sample HIT can be seen in 1(b).

We perform human evaluation of the fine-tuned versions of T5 and UnifiedQA,
the two models that performed the best on automatic metrics. We evaluate the
outputs of these models on the hidden test set. We calculated inter-annotator
agreement for these judgments using the method described in subsection 3.3,
and they were >80%, indicating high agreement. The models mostly produce
grammatical answers, but fail to adequately explain many actions in the story.
Figure 2 shows that, under human evaluation, models significantly under-
perform humans at producing valid answers to why-questions. Models fare worse
when the answers to the questions are external to the narrative.

Figure 2: Human evaluated performance of answers. The IA prefix denotes
performance on implicit-answer questions in the data.

Human evaluation is slow and expensive, so we performed a correlation analysis
between the automated metrics and human judgments to gauge usefulness of
popular automated metrics. Figure 3 shows that the embedding-based metrics are
only weakly correlated with human validity judgments, while lexical metrics
did even worse. None of the automatic metrics show a strong correlation,
confirming our earlier assertion that human evaluation is the most appropriate
way to analyze model performance on this open-ended generation task. BertScore
and BLEURT have weak correlation with human validity judgments. All three
metrics improve their correlation with human judgments slightly as the number
of human reference answers is increased; however, the increase is not large.

Figure 3: Correlation of automatic metrics with human validity judgment of
model outputs. For each question, we have 3 crowdsourced human answers
available to us. We selected a number of human answers randomly (X-axis) and
calculated scores for each model output across different automatic metrics.
Finally, we obtained Spearman’s correlation (Y-axis) of these scores in
comparison with Likert judgments provided by annotators for each human answer.

Our human judgment interface can serve as a standard human evaluation of any
future model’s performance on our dataset, and we will make code available for
automatically generating HITs for evaluating the outputs of any model. This
standardized evaluation approach is similar in spirit to GENIE Khashabi et al.
(2021), a contemporary work that also presents an evaluation framework for a
large set of generation tasks.

(a) ROUGE F-1 trend

(b) BLEU trend

Figure 4: Model performance on different metrics with change in lexical
overlap between a question’s answers (as provided by humans) and the related
narrative.

###  5.4 Analysis

In order to better understand when models are generating valid answers, we
analyzed the correlation between model performance and a proxy for checking
when human provided answers were in the input narrative. To this end, we
aligned ROUGE F-1 scores with the lexical overlap of human answers and the
story text. 4(a) shows how ROUGE F-1 scores for our models increases as the
lexical overlap also increases between the answers and corresponding story.
The same is presented for BLEU in 4(b). Perhaps not surprisingly, this
empirically shows that models do best when the answer is in the text, and
suffer greatly when it is not (implicit answers). This further illustrates the
value of TellMeWhy, as well as its challenge, that standard models are largely
incapable of performing the reasoning needed to produce plausible answers that
are assumed common knowledge by the story writer.

Table 5 also shows that the best performing models mainly learnt to copy
complete or parts from the narrative to generate answers, treating this
largely as an extractive task. On average, more than three-fourths of T5 and
UnifiedQA’s answers are based on words in the narrative text. T5 is worse
compared to UnifiedQA in terms of copying, with a much larger fraction of
questions (59.44% vs 27.44) with high lexical overlap (i.e. lexical overlap >
90%). In comparison, the average narrative overlap for human answers is much
lower than the best-performing models, since people are able to infer answers
that are not in the text. If the models are to successfully answer why
questions, they need to look beyond copying texts.

System | Copied ans | Avg overlap
---|---|---
Vanilla GPT2 | 0% | 23.09%
Finetuned GPT2 | 0% | 28.94%
Vanilla T5 | 5.50% | 53.71%
Finetuned T5 | 59.44% | 85.91%
UnifiedQA | 27.44% | 76.51%
Human Answers | 35.03% | 57.04%
Table 5: Overlap between answers and the original narrative. This indicate how
much original text models produce.

##  6 Conclusion

This paper introduces a large, novel QA dataset, TellMeWhy, containing
questions about why characters in a narrative perform their depicted actions.
This challenge problem complements the variety of existing QA datasets,
addressing the scarcity of “why” questions. Using both automated metrics and
human evaluation, we show that existing deep-learned language models perform
quite poorly at answering such questions. We also illustrate the uniqueness of
this challenge where the answer is sometimes in the story itself, but often
not, thus requiring a richer model that can draw on commonsense knowledge or
external reasoning abilities.

We believe that progress on answering such questions requires new systems that
can reason about actions, plans, and goals in order to achieve a deeper
understanding of narrative text, as was initially argued over four decades ago
Schank and Abelson (1977). We hope that TellMeWhy encourages further research
in this area.

## Acknowledgement

This material is based on research that is supported in part by the Air Force
Research Laboratory (AFRL), DARPA, for the KAIROS program under agreement
number FA8750-19-2-1003 and in part by the National Science Foundation under
the award IIS #2007290. The authors would like to thank the anonymous
reviewers and the area chair for their feedback on this work. We would also
like to thank Horace Liu for helping us run some experiments for the camera
ready version.

## References

  * Barrault et al. (2020) Loïc Barrault, Magdalena Biesialska, Ondřej Bojar, Marta R. Costa-jussà, Christian Federmann, Yvette Graham, Roman Grundkiewicz, Barry Haddow, Matthias Huck, Eric Joanis, Tom Kocmi, Philipp Koehn, Chi-kiu Lo, Nikola Ljubešić, Christof Monz, Makoto Morishita, Masaaki Nagata, Toshiaki Nakazawa, Santanu Pal, Matt Post, and Marcos Zampieri. 2020.  Findings of the 2020 conference on machine translation (WMT20).  In _Proceedings of the Fifth Conference on Machine Translation_ , pages 1–55, Online. Association for Computational Linguistics.
  * Barrault et al. (2019) Loïc Barrault, Ondřej Bojar, Marta R. Costa-jussà, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Philipp Koehn, Shervin Malmasi, Christof Monz, Mathias Müller, Santanu Pal, Matt Post, and Marcos Zampieri. 2019.  Findings of the 2019 conference on machine translation (WMT19).  In _Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1)_ , pages 1–61, Florence, Italy. Association for Computational Linguistics.
  * Bastan et al. (2020) Mohaddeseh Bastan, Mahnaz Koupaee, Youngseo Son, Richard Sicoli, and Niranjan Balasubramanian. 2020.  Author’s sentiment prediction.  In _Proceedings of the 28th International Conference on Computational Linguistics_ , pages 604–615, Barcelona, Spain (Online). International Committee on Computational Linguistics.
  * Bhagavatula et al. (2020) Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Hannah Rashkin, Doug Downey, Wen tau Yih, and Yejin Choi. 2020.  Abductive commonsense reasoning.  In _International Conference on Learning Representations_.
  * Bojar et al. (2017) Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Shujian Huang, Matthias Huck, Philipp Koehn, Qun Liu, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Raphael Rubino, Lucia Specia, and Marco Turchi. 2017.  Findings of the 2017 conference on machine translation (WMT17).  In _Proceedings of the Second Conference on Machine Translation_ , pages 169–214, Copenhagen, Denmark. Association for Computational Linguistics.
  * Bojar et al. (2016) Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Aurélie Névéol, Mariana Neves, Martin Popel, Matt Post, Raphael Rubino, Carolina Scarton, Lucia Specia, Marco Turchi, Karin Verspoor, and Marcos Zampieri. 2016.  Findings of the 2016 conference on machine translation.  In _Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers_ , pages 131–198, Berlin, Germany. Association for Computational Linguistics.
  * Bojar et al. (2018) Ondřej Bojar, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Philipp Koehn, and Christof Monz. 2018.  Findings of the 2018 conference on machine translation (WMT18).  In _Proceedings of the Third Conference on Machine Translation: Shared Task Papers_ , pages 272–303, Belgium, Brussels. Association for Computational Linguistics.
  * Caglayan et al. (2020) Ozan Caglayan, Pranava Madhyastha, and Lucia Specia. 2020.  Curious case of language generation evaluation metrics: A cautionary tale.  In _Proceedings of the 28th International Conference on Computational Linguistics_ , pages 2322–2328, Barcelona, Spain (Online). International Committee on Computational Linguistics.
  * Celikyilmaz et al. (2020) Asli Celikyilmaz, Elizabeth Clark, and Jianfeng Gao. 2020.  Evaluation of text generation: A survey.  _arXiv preprint arXiv:2006.14799_.
  * Chambers and Jurafsky (2009) Nathanael Chambers and Dan Jurafsky. 2009.  Unsupervised learning of narrative schemas and their participants.  In _Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP_ , pages 602–610, Suntec, Singapore. Association for Computational Linguistics.
  * Chen et al. (2019) Anthony Chen, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. 2019.  Evaluating question answering evaluation.  In _Proceedings of the 2nd Workshop on Machine Reading for Question Answering_ , pages 119–124, Hong Kong, China. Association for Computational Linguistics.
  * Dulceanu et al. (2018) Andrei Dulceanu, Thang Le Dinh, Walter Chang, Trung Bui, Doo Soon Kim, Manh Chien Vu, and Seokhwan Kim. 2018.  PhotoshopQuiA: A corpus of non-factoid questions and answers for why-question answering.  In _Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)_ , Miyazaki, Japan. European Language Resources Association (ELRA).
  * Dunietz et al. (2020) Jesse Dunietz, Greg Burnham, Akash Bharadwaj, Owen Rambow, Jennifer Chu-Carroll, and Dave Ferrucci. 2020.  To test machine comprehension, start by defining comprehension.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7839–7859, Online. Association for Computational Linguistics.
  * Fukumoto et al. (2007) J. Fukumoto, T. Kato, F. Masui, and T. Mori. 2007.  An overview of the 4th question answering challenge (qac-4) at ntcir workshop 6.  In _NTCIR_.
  * Gatt and Krahmer (2018) Albert Gatt and Emiel Krahmer. 2018.  Survey of the state of the art in natural language generation: Core tasks, applications and evaluation.  _J. Artif. Int. Res._ , 61(1):65–170.
  * Geva et al. (2021) Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. 2021.  Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies.  _Transactions of the Association for Computational Linguistics_ , 9:346–361.
  * Graham et al. (2013) Yvette Graham, Timothy Baldwin, Alistair Moffat, and Justin Zobel. 2013.  Continuous measurement scales in human evaluation of machine translation.  In _Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse_ , pages 33–41, Sofia, Bulgaria. Association for Computational Linguistics.
  * Graham et al. (2014) Yvette Graham, Timothy Baldwin, Alistair Moffat, and Justin Zobel. 2014.  Is machine translation getting better over time? In _Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics_ , pages 443–451, Gothenburg, Sweden. Association for Computational Linguistics.
  * Higashinaka and Isozaki (2008) Ryuichiro Higashinaka and Hideki Isozaki. 2008.  Corpus-based question answering for why-questions.  In _Proceedings of the Third International Joint Conference on Natural Language Processing: Volume-I_.
  * Honnibal et al. (2020) Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. 2020.  spaCy: Industrial-strength Natural Language Processing in Python.
  * Howcroft et al. (2020) David M. Howcroft, Anya Belz, Miruna-Adriana Clinciu, Dimitra Gkatzia, Sadid A. Hasan, Saad Mahamood, Simon Mille, Emiel van Miltenburg, Sashank Santhanam, and Verena Rieser. 2020.  Twenty years of confusion in human evaluation: NLG needs evaluation sheets and standardised definitions.  In _Proceedings of the 13th International Conference on Natural Language Generation_ , pages 169–182, Dublin, Ireland. Association for Computational Linguistics.
  * Khashabi et al. (2020) Daniel Khashabi, Sewon Min, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark, and Hannaneh Hajishirzi. 2020.  UNIFIEDQA: Crossing format boundaries with a single QA system.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 1896–1907, Online. Association for Computational Linguistics.
  * Khashabi et al. (2021) Daniel Khashabi, Gabriel Stanovsky, Jonathan Bragg, Nicholas Lourie, Jungo Kasai, Yejin Choi, Noah A. Smith, and Daniel S. Weld. 2021.  Genie: A leaderboard for human-in-the-loop evaluation of text generation.
  * Kintsch and Dijk (1978) W. Kintsch and T. Dijk. 1978.  Toward a model of text comprehension and production.  _Psychological Review_ , 85:363–394.
  * Kočiský et al. (2018) Tomáš Kočiský, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. 2018.  The NarrativeQA reading comprehension challenge.  _Transactions of the Association for Computational Linguistics_ , 6:317–328.
  * Lai et al. (2017) Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017.  RACE: Large-scale ReAding comprehension dataset from examinations.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 785–794, Copenhagen, Denmark. Association for Computational Linguistics.
  * Lin (2004) Chin-Yew Lin. 2004.  ROUGE: A package for automatic evaluation of summaries.  In _Text Summarization Branches Out_ , pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  * Ma et al. (2019) Qingsong Ma, Johnny Wei, Ondřej Bojar, and Yvette Graham. 2019.  Results of the WMT19 metrics shared task: Segment-level and strong MT systems pose big challenges.  In _Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1)_ , pages 62–90, Florence, Italy. Association for Computational Linguistics.
  * Marasini et al. (2016) D. Marasini, P. Quatto, and E. Ripamonti. 2016.  Assessing the inter-rater agreement for ordinal data through weighted indexes.  _Statistical Methods in Medical Research_ , 25:2611 – 2633.
  * Mostafazadeh et al. (2016a) Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. 2016a.  A corpus and cloze evaluation for deeper understanding of commonsense stories.  In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 839–849, San Diego, California. Association for Computational Linguistics.
  * Mostafazadeh et al. (2016b) Nasrin Mostafazadeh, Alyson Grealish, Nathanael Chambers, James Allen, and Lucy Vanderwende. 2016b.  CaTeRS: Causal and temporal relation scheme for semantic annotation of event structures.  In _Proceedings of the Fourth Workshop on Events_ , pages 51–61, San Diego, California. Association for Computational Linguistics.
  * Mostafazadeh et al. (2020) Nasrin Mostafazadeh, Aditya Kalyanpur, Lori Moon, David Buchanan, Lauren Berkowitz, Or Biran, and Jennifer Chu-Carroll. 2020.  GLUCOSE: GeneraLized and COntextualized story explanations.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 4569–4586, Online. Association for Computational Linguistics.
  * Mrozinski et al. (2008) Joanna Mrozinski, Edward Whittaker, and Sadaoki Furui. 2008.  Collecting a why-question corpus for development and evaluation of an automatic QA-system.  In _Proceedings of ACL-08: HLT_ , pages 443–451, Columbus, Ohio. Association for Computational Linguistics.
  * Nie et al. (2019) Allen Nie, Erin Bennett, and Noah Goodman. 2019.  Learning to explain: Answering why-questions via rephrasing.  In _Proceedings of the First Workshop on NLP for Conversational AI_ , pages 113–120, Florence, Italy. Association for Computational Linguistics.
  * Norvig (1987) Peter Norvig. 1987.  Inference in text understanding.  In _Proceedings of the Sixth National Conference on Artificial Intelligence - Volume 2_ , AAAI’87, page 561–565. AAAI Press.
  * Novikova et al. (2018) Jekaterina Novikova, Ondřej Dušek, and Verena Rieser. 2018.  RankME: Reliable human ratings for natural language generation.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_ , pages 72–78, New Orleans, Louisiana. Association for Computational Linguistics.
  * Oh et al. (2019) Jong-Hoon Oh, Kazuma Kadowaki, Julien Kloetzer, Ryu Iida, and Kentaro Torisawa. 2019\.  Open-domain why-question answering with adversarial learning to encode answer texts.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 4227–4237, Florence, Italy. Association for Computational Linguistics.
  * Oh et al. (2012) Jong-Hoon Oh, Kentaro Torisawa, Chikara Hashimoto, Takuya Kawada, Stijn De Saeger, Jun’ichi Kazama, and Yiou Wang. 2012.  Why question answering using sentiment analysis and word classes.  In _Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning_ , pages 368–378, Jeju Island, Korea. Association for Computational Linguistics.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002.  Bleu: a method for automatic evaluation of machine translation.  In _Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics_ , pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
  * Pichotta and Mooney (2014) Karl Pichotta and Raymond Mooney. 2014.  Statistical script learning with multi-argument events.  In _Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics_ , pages 220–229, Gothenburg, Sweden. Association for Computational Linguistics.
  * Post (2018) Matt Post. 2018.  A call for clarity in reporting BLEU scores.  In _Proceedings of the Third Conference on Machine Translation: Research Papers_ , pages 186–191, Brussels, Belgium. Association for Computational Linguistics.
  * Radford et al. (2019) Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019.  Language models are unsupervised multitask learners.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _Journal of Machine Learning Research_ , 21(140):1–67.
  * Rogers et al. (2020) Anna Rogers, Olga Kovaleva, Matthew Downey, and Anna Rumshisky. 2020.  Getting closer to ai complete question answering: A set of prerequisite real tasks.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 34(05):8722–8731.
  * Sakaguchi and Van Durme (2018) Keisuke Sakaguchi and Benjamin Van Durme. 2018.  Efficient online scalar annotation with bounded support.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 208–218, Melbourne, Australia. Association for Computational Linguistics.
  * Santhanam and Shaikh (2019) Sashank Santhanam and Samira Shaikh. 2019.  Towards best experiment design for evaluating dialogue system output.  In _Proceedings of the 12th International Conference on Natural Language Generation_ , pages 88–94, Tokyo, Japan. Association for Computational Linguistics.
  * Sap et al. (2019) Maarten Sap, Ronan Le Bras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A. Smith, and Yejin Choi. 2019.  Atomic: An atlas of machine commonsense for if-then reasoning.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 33(01):3027–3035.
  * Schank and Abelson (1977) R. Schank and R. P. Abelson. 1977.  Scripts, plans, goals and understanding: an inquiry into human knowledge structures.
  * Schank and Abelson (1975) Roger C. Schank and Robert P. Abelson. 1975.  Scripts, plans, and knowledge.  In _Proceedings of the 4th International Joint Conference on Artificial Intelligence - Volume 1_ , IJCAI’75, page 151–157, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc.
  * Sedoc et al. (2019) João Sedoc, Daphne Ippolito, Arun Kirubarajan, Jai Thirani, Lyle Ungar, and Chris Callison-Burch. 2019.  ChatEval: A tool for chatbot evaluation.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations)_ , pages 60–65, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Sellam et al. (2020) Thibault Sellam, Dipanjan Das, and Ankur Parikh. 2020.  BLEURT: Learning robust metrics for text generation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7881–7892, Online. Association for Computational Linguistics.
  * Verberne et al. (2006) Suzan Verberne, Lou Boves, Nelleke Oostdijk, and Peter-Arno Coppen. 2006.  Data for question answering: The case of why.  In _Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06)_ , Genoa, Italy. European Language Resources Association (ELRA).
  * Verberne et al. (2007) Suzan Verberne, Lou Boves, Nelleke Oostdijk, and Peter-Arno Coppen. 2007.  Evaluating discourse-based answer extraction for why-question answering.  In _Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval_ , SIGIR ’07, page 735–736, New York, NY, USA. Association for Computing Machinery.
  * Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020.  Transformers: State-of-the-art natural language processing.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 38–45, Online. Association for Computational Linguistics.
  * Zellers et al. (2020) Rowan Zellers, Ari Holtzman, Elizabeth Clark, Lianhui Qin, Ali Farhadi, and Yejin Choi. 2020.  Evaluating machines by their real-world language use.  _arXiv preprint arXiv:2004.03607_.
  * Zhang* et al. (2020) Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020.  Bertscore: Evaluating text generation with bert.  In _International Conference on Learning Representations_.
  * Zwaan et al. (1995) Rolf A. Zwaan, Mark C. Langston, and Arthur C. Graesser. 1995.  The construction of situation models in narrative comprehension: An event-indexing model.  _Psychological Science_ , 6(5):292–297.

##  Appendix A Hyperparameter Sweep

We describe the hyperparameters and the range of values we experimented with.
The best hyperparameters are chosen on the basis of model loss on the
validation set. For both GPT-2 Radford et al. (2019) and T5 Raffel et al.
(2020), we conduct guided sweeps for learning rate, batch size and epochs. We
experiment with 1e-5, 5e-5 and 1e-4 for learning rate. Batch sizes of 8, 16
and 32 were tried. Models were trained for 20, 30 and 50 epochs, and we found
that models converged between 30 and 50 epochs. In the case of T5, we also
experiment with different lengths of inputs and target outputs. We trained
models with maximum source lengths of 50, 60 and 75 tokens. For target length,
we experimented with 15, 25 and 30 tokens. The maximum output length is
treated as a hyperparameter for GPT-2, and we tried 15, 20, 25 and 30 tokens.

##  Appendix B Dataset Creation

The method described in subsection 3.1 creates 489 questions from the 200
stories in the CATERS dataset – 36 stories with 1 question, 63 with 2, 59 with
3, 30 with 4, and 6 with 5. We collect 3 human answers for all questions. For
ROCStories, this creates 113,213 questions from 45,496 stories – 7,555 stories
with 1 question, 13,431 with 2, 13,349 with 3, 7356 with 4, and 1865 with 5.
We randomly select 32,165 questions from stories with 3 or 5 questions, for
ease and efficiency of collecting annotations. This is the smallest number for
which we could gather 3 answers for at least 30,000 questions, which is a
reasonable-sized dataset for training or fine-tuning large NLP models.

##  Appendix C Mechanical Turk tasks

###  C.1 Instructions

We present the instructions given to annotators for both the tasks in Figure
5. Annotators were given clear direction for both tasks. We restricted both
tasks to master turkers. The second task (answer validity) was also used a
sanity check for answers collected in the first task (answer collection).
Using results of the answer validity task (mentioned in subsection 3.3), we
see that humans provided high quality answers in the answer curation task.

(a) Instructions for answer collection task

(b) Instructions for answer validation task

Figure 5: Instructions for MTurk tasks

###  C.2 Inter-annotator agreement

We use weighted Fleiss Kappa to calculate inter-rater reliability. The weights
between different classes are shown in Table 6 where negative, slightly
negative, neutral, slightly positive, and positive classes are shown with -2,
-1, 0, 1, and 2. We follow the setup used in Bastan et al. (2020) for a
similar multi-class labeling task.

| -2 | -1 | 0 | 1 | 2
---|---|---|---|---|---
-2 | 1 | cos $\pi$/8 | cos $\pi$/4 | cos 3$\pi$/8 | 0
-1 | cos $\pi$/8 | 1 | cos $\pi$/8 | cos $\pi$/4 | cos 3$\pi$/8
0 | cos $\pi$/4 | cos $\pi$/8 | 1 | cos $\pi$/8 | cos $\pi$/4
1 | cos 3$\pi$/8 | cos $\pi$/4 | cos $\pi$/8 | 1 | cos $\pi$/8
2 | 0 | cos 3$\pi$/8 | cos $\pi$/4 | cos $\pi$/8 | 1
Table 6: Inter class weights used for computing inter annotated agreement


# Ill-posedness for the higher dimensional Camassa-Holm equations in Besov
spaces

Min Li1 and Yingying Guo2,111E-mail<EMAIL_ADDRESS><EMAIL_ADDRESS>
1 School of information Technology, Jiangxi University of Finance and
Economics, Nanchang, 330032, China
2 Department of Mathematics, Foshan University, Foshan, Guangdong 528000,
China

Abstract: In the paper, by constructing a initial data $u_{0}\in
B^{\sigma}_{p,\infty}$ with $\sigma-2>\max\\{1+\frac{1}{p},\frac{3}{2}\\}$, we
prove that the corresponding solution to the higher dimensional Camassa-Holm
equations starting from $u_{0}$ is discontinuous at $t=0$ in the norm of
$B^{\sigma}_{p,\infty}$, which implies that the ill-posedness for the higher
dimensional Camassa-Holm equations in $B^{\sigma}_{p,\infty}$.

Keywords: Higher dimensional Camassa-Holm equations; Ill-posedness; Besov
spaces.

MSC (2010): 35A01, 35Q35, 37K10

##  1 Introduction

In this paper, we consider the following initial value problem for the higher
dimensional Camassa-Holm equations:

$\begin{cases}\partial_{t}m+u\cdot\nabla m+\nabla u^{T}\cdot
m+(\mathrm{div}u)m=0,\qquad&(t,x)\in\mathbb{R}^{+}\times\mathbb{R}^{d},\\\
m=(1-\Delta)u,\qquad&(t,x)\in\mathbb{R}^{+}\times\mathbb{R}^{d},\\\
u(0,x)=u_{0},\qquad&x\in\mathbb{R}^{d}.\end{cases}$ (1.1)

According to [44], we can transform Eq. (1.1) into the following form of
transport equations:

$\displaystyle\partial_{t}u+u\cdot\nabla u=Q(u,u)+R(u,u),$ (1.2)

where

$\displaystyle Q(u,u)=-(I-\Delta)^{-1}\mathrm{div}\Big{(}\nabla u\nabla
u+\nabla u\nabla u^{T}-\nabla u^{T}\nabla u-\nabla
u(\mathrm{div}u)+\frac{1}{2}\mathbf{I}|\nabla u|^{2}\Big{)},$ $\displaystyle
R(u,u)=-(I-\Delta)^{-1}\Big{(}u(\mathrm{div}u)+u\cdot(\nabla u)^{T}\Big{)}.$

Eq. (1.1) was investigated as Euler-Poincaré equations describes geodesic
motion on the diffeomorphism group with respect to the kinetic energy norm in
[29], which can also be viewed as higher dimensional generalization of the
following classical one dimensional Camassa-holm equation (CH):

$\displaystyle m_{t}+um_{x}+2u_{x}m=0,\ m=u-u_{xx}.$

The CH equation is completely integrable [8, 17] and has a bi-Hamiltonian
structure [7, 24]. It also has the solitary waves and peak solitons [21, 18].
It is worth mentioning that the peakons show the characteristic for the
traveling waves of greatest height and arise as solutions to the free-boundary
problem for the incompressible Euler equations over a flat bed, see [10, 11,
15, 42]. The local well-posedness, global strong solutions, blow-up strong
solutions of the CH equations were studied in [9, 14, 12, 13, 22, 35, 41]. The
global weak solutions, global conservative solutions and dissipative solutions
also have been investigated in [2, 3, 20, 31, 43]. For the continuity of the
solutions map of the CH equations with respect to the initial data, it was
only proved in the spaces $C([0,T];B^{s^{\prime}}_{p,r}(\mathbb{R}))$ for any
$s^{\prime}<s$ with $s>\max\\{\frac{3}{2},1+\frac{1}{p}\\}$ by many authors.
Moreover, Li and Yin in [40] proved that the index of the continuous
dependence for the solutions to the Camassa-Holm type equations in
$B^{s}_{p,r}(\mathbb{R})\big{(}s>\max\\{\frac{3}{2},1+\frac{1}{p}\\}\big{)}$
can up to $s$, which improved many authors’ results, especially the Danchin’s
results in [22, 23]. Recently, Guo et al. [25] obtained the local ill-
posedness for a class of shallow water wave equations (such as, the CH, DP,
Novikov equations and etc.) in critical Sobolev space
$H^{\frac{3}{2}}(\mathbb{R})$ and even in Besov space
$B^{1+\frac{1}{p}}_{p,r}(\mathbb{R})$ with $p\in[1,+\infty],\
r\in(1,+\infty]$. More recently, by use of the compactness argument and
Lagrangian coordinate transformatiorather rather than the usual techniques
used in [40], Ye et al. [45] proved the CH equation is locally well-posed and
continuous dependence in Besov spaces $B^{1+\frac{1}{p}}_{p,1}(\mathbb{R})$
with $p\in[1,+\infty)$, which implied $B^{1+\frac{1}{p}}_{p,1}(\mathbb{R})$ is
the critical Besov spaces and the index $\frac{3}{2}$ is not necessary for the
Camassa-Holm type equations. Further, the non-uniform continuity of the CH
equation has been investigated in many papers, see [28, 26, 27, 32, 33].

Eqs. (1.1) has numerous remarkable features and has been studies by many
authors. Chae and Liu [6] eatablished the local existence of weak solution in
$W^{2,p}(\mathbb{R}^{d}),\ p>d$ and local existence of unique classical
solutions in $H^{s}(\mathbb{R}^{d}),\ s>\frac{d}{2}+3$ for (1.1). Then, Li, Yu
and Zhai [39] proved that the solutions to (1.1) with a large class of smooth
initial data blows up in finite time or exists globally in time, which settled
an open problem raised by Chae and Liu [6]. Taking advantage of the
Littlewood-Paley decomposition theory, Yan and Yin [44] further discussed the
local existence and uniqueness of the solution to (1.1) in Besov spaces
$B^{s}_{p,r}(\mathbb{R}^{d})$ with $s>\max\\{1+\frac{d}{p},\frac{3}{2}\\}$ and
$s=1+\frac{d}{p},1\leq p\leq 2d,r=1$. Recently, Li, Dai and Zhu [34] shown
that the corresponding solution to (1.1) is not uniformly constinuous
dependence for that the initial data in
$H^{s}(\mathbb{R}^{d}),s>1+\frac{d}{2}$. Also, Li, Dai and Li in [38] have
shown that the data-to-solution map for (1.1) is not uniformly continuous
dependence in Besov spaces
$B^{s}_{p,r}(\mathbb{R}^{d}),s>\max\\{1+\frac{d}{2},\frac{3}{2}\\}$. For more
results of higher dimensional Camassa-Holm equations, see [36, 46].

However, the continuity of the data-to-solution map for the higher dimensional
Camassa-Holm equations in Besov spaces $B^{s}_{p,\infty}(\mathbb{R}^{d}),\
s>\max\\{1+\frac{d}{p},\frac{3}{2}\\},\ 1\leq p\leq+\infty$ has not been
solved yet. In this paper, we will pay our attention to studying the ill-
posedness for the higher dimensional Camassa-Holm equations in Besov spaces.
The key skill is to construct a initial data.

Now let us state our main result of this paper.

######  Theorem 1.1.

Let $d\geq 2$ and $\sigma>2+\max\big{\\{}1+\frac{d}{p},\frac{3}{2}\big{\\}}$
with $1\leq p\leq\infty$. There exists a $u_{0}\in
B^{\sigma}_{p,\infty}(\mathbb{R}^{d})$ and a positive constant
$\varepsilon_{0}$ such that the data-to-solution map $u_{0}\mapsto u(t)$ of
the Cauchy problem (1.1) satisfies

$\displaystyle\limsup_{t\to
0^{+}}\|u(t)-u_{0}\|_{B^{\sigma}_{p,\infty}}\geq\varepsilon_{0}.$

######  Remark 1.1.

Theorem 1.1 demonstrates the ill-posedness of the higher dimensional CH
equation in $B^{\sigma}_{p,\infty}$. More precisely, there exists a $u_{0}\in
B^{\sigma}_{p,\infty}$ such that the corresponding solution to the higher
dimensional CH equation that starts from $u_{0}$ does not converge back to
$u_{0}$ in the sense of $B^{\sigma}_{p,\infty}$-norm as time goes to zero. Our
key argument is to construct a initial data $u_{0}$.

The remainder of this paper is organized as follows. In Section 2, we list
some notations and recall known results. In Section 3, we present the proof of
Theorem 1.1 by establishing some technical lemmas and propositions.

##  2 Preliminaries

###  2.1 General Notation

In the following, we denote by $\star$ the convolution. Given a Banach space
$X$, we denote its norm by $\|\cdot\|_{X}$. For $I\subset\mathbb{R}$, we
denote by $\mathcal{C}(I;X)$ the set of continuous functions on $I$ with
values in $X$. Sometimes we will denote $L^{p}(0,T;X)$ by $L_{T}^{p}X$. For
all $f\in\mathcal{S}^{\prime}$, the Fourier transform $\mathcal{F}f$ (also
denoted by $\widehat{f}$) is defined by

$\mathcal{F}f(\xi)=\widehat{f}(\xi)=\int_{\mathbb{R}^{d}}e^{-ix\cdot\xi}f(x)\mathrm{d}x\quad\text{for
any}\;\xi\in\mathbb{R}^{d}.$

###  2.2 Littlewood-Paley Analysis

Next, we recall some useful properties about the Littlewood-Paley
decomposition and the Besov spaces.

######  Proposition 2.1 (Littlewood-Paley decomposition, See [1]).

Let $\mathcal{B}:=\\{\xi\in\mathbb{R}^{d}:|\xi|\leq\frac{4}{3}\\}$ and
$\mathcal{C}:=\\{\xi\in\mathbb{R}^{d}:\frac{3}{4}\leq|\xi|\leq\frac{8}{3}\\}.$
There exist two radial functions $\chi\in C_{c}^{\infty}(\mathcal{B})$ and
$\varphi\in C_{c}^{\infty}(\mathcal{C})$ both taking values in $[0,1]$ such
that

$\displaystyle\chi(\xi)+\sum_{j\geq
0}\varphi(2^{-j}\xi)=1,\quad\forall\;\xi\in\mathbb{R}^{d},$
$\displaystyle\frac{1}{2}\leq\chi^{2}(\xi)+\sum_{j\geq
0}\varphi^{2}(2^{-j}\xi)\leq 1,\quad\forall\;\xi\in\mathbb{R}^{d}.$

######  Proposition 2.2 (Bernstein’s inequalities, See [1]).

Let $\mathcal{B}$ be a ball and $\mathcal{C}$ be an annulus. A constant $C>0$
exists such that for all $k\in\mathbb{N},\ 1\leq p\leq q\leq\infty$, and any
function $f\in L^{p}(\mathbb{R})$, we have

$\displaystyle
Supp{\widehat{f}}\subset\lambda\mathcal{B}\Rightarrow\|D^{k}f\|_{L^{q}}=\sup\limits_{|\alpha|=k}\|\partial^{\alpha}f\|_{L^{q}}\leq
C^{k+1}\lambda^{k+d(\frac{1}{p}-\frac{1}{q})}\|f\|_{L^{p}},$ $\displaystyle
Supp{\widehat{f}}\subset\lambda\mathcal{C}\Rightarrow
C^{-k-1}\lambda^{k}\|f\|_{L^{p}}\leq\|D^{k}f\|_{L^{q}}\leq
C^{k+1}\lambda^{k}\|f\|_{L^{p}}.$

######  Definition 2.1 (See [1]).

For any $u\in\mathcal{S^{\prime}}(\mathbb{R}^{d})$, the Littlewood-Paley
dyadic blocks ${\Delta}_{j}$ are defined as follows

$\displaystyle\Delta_{j}u=$ $\displaystyle 0,$ if $j\leq-2$;
$\displaystyle\Delta_{j}u=$
$\displaystyle\chi(D)u=\mathcal{F}^{-1}(\chi\mathcal{F}u),$ if $j=-1$;
$\displaystyle\Delta_{j}u=$
$\displaystyle\varphi(2^{-j}D)u=\mathcal{F}^{-1}\big{(}\varphi(2^{-j}\cdot)\mathcal{F}u\big{)},$
if $j\geq 0$.

In the nonhomogeneous case, the following Littlewood-Paley decomposition makes
sense:

$u=\sum_{j\geq-1}{\Delta}_{j}u,\quad\forall\;u\in\mathcal{S^{\prime}}(\mathbb{R}^{d}).$

######  Definition 2.2 (See [1]).

Let $s\in\mathbb{R}$ and $1\leq p,r\leq\infty$. The nonhomogeneous Besov space
$B^{s}_{p,r}(\mathbb{R}^{d})$ is defined by

$\displaystyle
B^{s}_{p,r}(\mathbb{R}^{d}):=\Big{\\{}f\in\mathcal{S}^{\prime}(\mathbb{R}^{d}):\;\|f\|_{B^{s}_{p,r}(\mathbb{R}^{d})}<\infty\Big{\\}},$

where

$\displaystyle\|f\|_{B^{s}_{p,r}(\mathbb{R}^{d})}=$
$\displaystyle\left(\sum_{j\geq-1}2^{sjr}\|\Delta_{j}f\|^{r}_{L^{p}(\mathbb{R}^{d})}\right)^{\frac{1}{r}},$
if $1\leq r<\infty$, $\displaystyle\|f\|_{B^{s}_{p,r}(\mathbb{R}^{d})}=$
$\displaystyle\sup_{j\geq-1}2^{js}\|\Delta_{j}f\|_{L^{p}(\mathbb{R}^{d})},$ if
$r=\infty$.

For simplicity, we always write $u\in B^{s}_{p,r}(\mathbb{R}^{d})$ and $\nabla
u\in B^{s}_{p,r}(\mathbb{R}^{d})$ standing for
$u\in(B^{s}_{p,r}(\mathbb{R}^{d}))^{d}$ and $\nabla
u\in(B^{s}_{p,r}(\mathbb{R}^{d}))^{d^{2}}$, respectively.

######  Remark 2.1.

It should be emphasized that the following embedding will be often used
implicity:

$B^{s_{1}}_{p,q}(\mathbb{R}^{d})\hookrightarrow
B^{s_{2}}_{p,r}(\mathbb{R}^{d})\ \text{for}\;s_{1}>s_{2}\ \text{or}\
s_{1}=s_{2},\ 1\leq q\leq r\leq\infty.$

Finally, we give some important lemmas which will be also often used
throughout the paper.

######  Lemma 2.1 (See [1, 44]).

Let $(p,r)\in[1,\infty]^{2}$ and
$s>\max\big{\\{}1+\frac{d}{p},\frac{3}{2}\big{\\}}$. Then

$\displaystyle\|uv\|_{B^{s-2}_{p,r}(\mathbb{R}^{d})}\leq
C\|u\|_{B^{s-2}_{p,r}(\mathbb{R}^{d})}\|v\|_{B^{s-1}_{p,r}(\mathbb{R}^{d})}.$

Hence, for the terms $Q(u,u),\ Q(v,v),\ R(u,u)$ and $R(v,v)$, we have

$\displaystyle\|{Q}(u,u)-{Q}(v,v)\|_{B^{s-1}_{p,r}(\mathbb{R}^{d})}\leq
C\|u-v\|_{B^{s-1}_{p,r}(\mathbb{R}^{d})}\big{(}\|u\|_{B^{s}_{p,r}(\mathbb{R}^{d})}+\|v\|_{B^{s}_{p,r}(\mathbb{R}^{d})}\big{)},$
$\displaystyle\|{R}(u,u)-{R}(v,v)\|_{B^{s-1}_{p,r}(\mathbb{R}^{d})}\leq
C\|u-v\|_{B^{s-1}_{p,r}(\mathbb{R}^{d})}\big{(}\|u\|_{B^{s}_{p,r}(\mathbb{R})}+\|v\|_{B^{s}_{p,r}(\mathbb{R}^{d})}\big{)}.$

######  Lemma 2.2 (See [1]).

For $(p,r)\in[1,\infty]^{2}$ and $s>0$, $B^{s}_{p,r}(\mathbb{R}^{d})\cap
L^{\infty}(\mathbb{R}^{d})$ is an algebra. Moreover, for any $u,v\in
B^{s}_{p,r}(\mathbb{R}^{d})\cap L^{\infty}(\mathbb{R}^{d})$, we have

$\displaystyle\|uv\|_{B^{s}_{p,r}(\mathbb{R}^{d})}\leq
C\big{(}\|u\|_{B^{s}_{p,r}(\mathbb{R}^{d})}\|v\|_{L^{\infty}(\mathbb{R}^{d})}+\|v\|_{B^{s}_{p,r}(\mathbb{R}^{d})}\|u\|_{L^{\infty}(\mathbb{R}^{d})}\big{)}.$

In the paper, we also need some estimates for the following transport
equation:

$\displaystyle\left\\{\begin{array}[]{l}\partial_{t}f+v\cdot\nabla f=g,\
(t,x)\in\mathbb{R}^{+}\times\mathbb{R}^{d},\\\ f(0,x)=f_{0}(x),\quad
x\in\mathbb{R}^{d}.\end{array}\right.$ (2.3)

######  Lemma 2.3 (See [1]).

Let $d\in\mathbb{N}^{+},\ 1\leq p\leq\infty,\ 1\leq r\leq\infty$ and
$\theta>-\min(\frac{d}{p},1-\frac{d}{p})$. Let $f_{0}\in
B^{\theta}_{p,r}(\mathbb{R}^{d})$, $g\in
L^{1}(0,T;B^{\theta}_{p,r}(\mathbb{R}^{d}))$, $v\in
L^{\rho}(0,T;B^{-M}_{\infty,\infty}(\mathbb{R}^{d}))$ for some $\rho>1$ and
$M>0$, and

$\displaystyle\begin{array}[]{ll}\nabla v\in
L^{1}(0,T;B^{\frac{d}{p}}_{p,\infty}(\mathbb{R}^{d})\cap
L^{\infty}(\mathbb{R}^{d})),&\ \text{if}\ \theta<1+\frac{d}{p},\\\ \nabla v\in
L^{1}(0,T;B^{\theta-1}_{p,r}(\mathbb{R}^{d})),&\ \text{if}\
\theta>1+\frac{d}{p}\ (or\ \theta=1+\frac{d}{p},\ r=1).\end{array}$

Then the problem (2.3) has a unique solution $f$ in

  * -

the space $\mathcal{C}([0,T];B^{\theta}_{p,r}(\mathbb{R}^{d}))$, if
$r<\infty$,

  * -

the space
$\big{(}\bigcap_{{\theta}^{\prime}<\theta}\mathcal{C}([0,T];B^{{\theta}^{\prime}}_{p,\infty}(\mathbb{R}^{d}))\big{)}\bigcap\mathcal{C}_{w}([0,T];B^{\theta}_{p,\infty}(\mathbb{R}^{d}))$,
if $r=\infty$.

######  Lemma 2.4 (See [1, 40]).

Let $d\in\mathbb{N}^{+},\ 1\leq p,r\leq\infty,\
\theta>-\min(\frac{d}{p},1-\frac{d}{p}).$ There exists a constant $C$ such
that for all solutions $f\in L^{\infty}(0,T;B^{\theta}_{p,r}(\mathbb{R}^{d}))$
of (2.3) with initial data $f_{0}$ in $B^{\theta}_{p,r}(\mathbb{R}^{d})$ and
$g$ in $L^{1}(0,T;B^{\theta}_{p,r}(\mathbb{R}^{d}))$, we have, for a.e.
$t\in[0,T]$,

$\|f(t)\|_{B^{\theta}_{p,r}(\mathbb{R}^{d})}\leq\|f_{0}\|_{B^{\theta}_{p,r}(\mathbb{R}^{d})}+\int_{0}^{t}\|g(t^{\prime})\|_{B^{\theta}_{p,r}(\mathbb{R}^{d})}\mathrm{d}t^{\prime}+\int_{0}^{t}V^{\prime}(t^{\prime})\|f(t^{\prime})\|_{B^{\theta}_{p,r}(\mathbb{R}^{d})}\mathrm{d}t^{\prime}$

or

$\|f(t)\|_{B^{\theta}_{p,r}(\mathbb{R}^{d})}\leq
e^{CV(t)}\Big{(}\|f_{0}\|_{B^{\theta}_{p,r}(\mathbb{R}^{d})}+\int_{0}^{t}e^{-CV(t^{\prime})}\|g(t^{\prime})\|_{B^{\theta}_{p,r}(\mathbb{R}^{d})}\mathrm{d}t^{\prime}\Big{)}$

with

$V^{\prime}(t)=\left\\{\begin{array}[]{ll}\|\nabla
v(t)\|_{B^{\frac{d}{p}}_{p,\infty}(\mathbb{R}^{d})\cap
L^{\infty}(\mathbb{R}^{d})},&\text{if}\ \theta<1+\frac{d}{p},\\\ \|\nabla
v(t)\|_{B^{\theta-1}_{p,r}(\mathbb{R}^{d})},&\text{if}\ \theta>1+\frac{d}{p}\
(\text{or}\ \theta=1+\frac{d}{p},\ r=1).\end{array}\right.$

If $f=v$, then for all $\theta>0$, $V^{\prime}(t)=\|\nabla
v(t)\|_{L^{\infty}(\mathbb{R}^{d})}$.

##  3 Proof of Theorem 1.1

###  3.1 Construction of Initial Data

We need to introduce smooth, radial cut-off functions to localize the
frequency region. Precisely, let
$\widehat{\phi}\in\mathcal{C}^{\infty}_{0}(\mathbb{R})$ be an even, real-
valued and non-negative function on $\mathbb{R}$ and satisfy

$\displaystyle\widehat{\phi}(\xi)=$ $\displaystyle 1,$ if
$|\xi|\leq\frac{1}{4}$,  $\displaystyle\widehat{\phi}(\xi)=$ $\displaystyle
0,$ if $|\xi|\geq\frac{1}{2}$.

In [37], it has been verified that for
$f_{n}:=\phi(x_{1})\cos\big{(}\frac{17}{12}2^{n}x_{1}\big{)}\phi(x_{2})\cdot\phi(x_{n})$
and $n\geq 2$,

$\displaystyle\Delta_{j}(f_{n})=$ $\displaystyle f_{n},$ if $j=n$,
$\displaystyle\Delta_{j}(f_{n})=$ $\displaystyle 0,$ if $j\neq n$. (3.1)

We can obtain the similar result:

######  Lemma 3.1.

Let $6\leq k,n\in\mathbb{N}^{+}$. Define the function $g^{k}_{m,n}(x)$ by

$g^{k}_{m,n}(x):=\phi(x_{1})\cos\Big{(}\frac{17}{12}\big{(}2^{kn}\pm
2^{km}\big{)}x_{1}\Big{)}\phi(x_{2})\cdots\phi(x_{d})\quad\text{with}\quad
0\leq m\leq n-1.$

Then we have

$\displaystyle\Delta_{j}(g^{k}_{m,n})=$ $\displaystyle g^{k}_{m,n},$ if
$j=kn$, $\displaystyle\Delta_{j}(g^{k}_{m,n})=$ $\displaystyle 0,$ if $j\neq
kn$.

###### Proof.

The proof is similar to that of in [37], and here we omit it. ∎

######  Lemma 3.2.

Define the initial data $u_{0}(x)$ as

$\displaystyle
u_{0}(x)=\big{(}u_{0}^{1}(x),\cdots,u_{0}^{d}(x)\big{)}:=\Big{(}\sum\limits^{\infty}_{n=0}2^{-kn\sigma}f^{k}_{n}(x),0,\cdots,0\Big{)},$

ce4 where

$f^{k}_{n}(x):=\phi(x_{1})\cos\Big{(}\frac{17}{12}2^{kn}x_{1}\Big{)}\phi(x_{2})\cdots\phi(x_{d}),\quad
n\geq 0.$

Then for any
$\sigma\in\big{(}2+\max\\{\frac{3}{2},1+\frac{d}{p}\\},+\infty\big{)}$ and for
some $k$ large enough, we have

$\displaystyle|u_{0}|^{2}(x)=\big{(}u_{0}^{1}(x)\big{)}^{2},$ $\displaystyle
u_{0}\cdot\nabla
u_{0}=\Big{(}\frac{1}{2}\partial_{1}\big{(}(u_{0}^{1})^{2}\big{)},0,\cdots,0\Big{)}=\Big{(}\frac{1}{2}\partial_{1}\big{(}|u_{0}|^{2}\big{)},0,\cdots,0\Big{)},$
$\displaystyle\|u_{0}\|_{B^{\sigma}_{p,\infty}}\leq C,$
$\displaystyle\|\Delta_{kn}\big{(}|u_{0}|^{2}\big{)}\|_{L^{p}}\geq
c2^{-kn\sigma}.$

###### Proof.

Thanks to the definition of Besov spaces, the support of
$\varphi(2^{-j}\cdot)$ and (3.1), we see

$\displaystyle\|u_{0}\|_{B^{\sigma}_{p,\infty}}$
$\displaystyle=\sup_{j\geq-1}2^{j\sigma}\|\Delta_{j}u_{0}\|_{L^{p}}$
$\displaystyle=||\phi||^{d-1}_{L^{p}}\cdot\sup_{j\geq
0}\Big{\|}\phi(x_{1})\cos\Big{(}\frac{17}{12}2^{j}x_{1}\Big{)}\Big{\|}_{L^{p}}$
$\displaystyle\leq C.$

Notice that the simple fact

$\cos(\mathbf{A}+\mathbf{B})+\cos(\mathbf{A}-\mathbf{B})=2\cos\mathbf{A}\cos\mathbf{B}$

and

$\sum^{\infty}_{n=0}\sum^{\infty}_{m=0,m\neq
n}\mathbf{A}_{n}\mathbf{A}_{m}=2\sum^{\infty}_{n=0}\sum^{n-1}_{m=0}\mathbf{A}_{n}\mathbf{A}_{m},$

then direct computations give

$\displaystyle|u_{0}|^{2}(x)=\big{(}u_{0}^{1}(x)\big{)}^{2}$ $\displaystyle=$
$\displaystyle\frac{1}{2}\sum^{\infty}_{n=0}2^{-2kn\sigma}\phi^{2}(x_{1})\phi^{2}(x_{2})\cdots\phi^{2}(x_{d})+\frac{1}{2}\sum^{\infty}_{n=0}2^{-2kn\sigma}\phi^{2}(x_{1})\cos\Big{(}\frac{17}{12}2^{kn+1}x_{1}\Big{)}\phi^{2}(x_{2})\cdots\phi^{2}(x_{d})$
$\displaystyle+$
$\displaystyle\sum^{\infty}_{n=1}\sum^{n-1}_{m=0}2^{-k(n+m)\sigma}\phi^{2}(x_{1})\Big{[}\cos\Big{(}\frac{17}{12}(2^{kn}-2^{km})x_{1}\Big{)}+\cos\Big{(}\frac{17}{12}(2^{kn}+2^{km})x_{1}\Big{)}\Big{]}\phi^{2}(x_{2})\cdots\phi^{2}(x_{d}).$

Lemma 3.1 yields

$\displaystyle\Delta_{kn}\big{(}|u_{0}|^{2}\big{)}=\Delta_{kn}\big{(}(u_{0}^{1})^{2}\big{)}$
$\displaystyle=$ $\displaystyle
2^{-kn\sigma}\phi^{2}(x_{1})\Big{[}\cos\Big{(}\frac{17}{12}(2^{kn}-1)x_{1}\Big{)}+\cos\Big{(}\frac{17}{12}(2^{kn}+1)x_{1}\Big{)}\Big{]}\phi^{2}(x_{2})\cdots\phi^{2}(x_{d})$
$\displaystyle+\sum^{n-1}_{m=1}2^{-k(n+m)\sigma}\phi^{2}(x)\Big{[}\cos\Big{(}\frac{17}{12}(2^{kn}-2^{km})x_{1}\Big{)}+\cos\Big{(}\frac{17}{12}(2^{kn}+2^{km})x_{1}\Big{)}\Big{]}\phi^{2}(x_{2})\cdots\phi^{2}(x_{d})$
$\displaystyle=:\mathrm{I}_{1}+\mathrm{I}_{2},$

where we denote

$\displaystyle\mathrm{I}_{1}:=2\cdot
2^{-kn\sigma}\phi^{2}(x_{1})\cos\Big{(}\frac{17}{12}2^{kn}x_{1}\Big{)}\cos\Big{(}\frac{17}{12}x_{1}\Big{)}\phi^{2}(x_{2})\cdots\phi^{2}(x_{d}),$
$\displaystyle\mathrm{I}_{2}:=2\sum^{n-1}_{m=1}2^{-k(n+m)\sigma}\phi^{2}(x_{1})\cos\Big{(}\frac{17}{12}2^{kn}x_{1}\Big{)}\cos\Big{(}\frac{17}{12}2^{km}x_{1}\Big{)}\phi^{2}(x_{2})\cdots\phi^{2}(x_{d}).$

For the first term $\mathrm{I}_{1}$, after a simple calculation, we have

$\displaystyle\|\mathrm{I}_{1}\|_{L^{p}(\mathbb{R}^{d})}$ $\displaystyle\geq
2^{-kn\sigma}\Big{\|}\phi^{2}(x_{1})\cos\Big{(}\frac{17}{12}x_{1}\Big{)}\cos\Big{(}\frac{17}{12}2^{kn}x_{1}\Big{)}\Big{\|}_{L^{p}(\mathbb{R})}||\phi||^{2(d-1)}_{L^{2p}}.$
(3.2)

From Lemma 3.2 in [37], we have for some $\delta>0$

$\displaystyle\Big{\|}\phi^{2}(x_{1})\cos\Big{(}\frac{17}{12}x_{1}\Big{)}\cos\Big{(}\frac{17}{12}2^{kn}x_{1}\Big{)}\Big{\|}_{L^{p}(\mathbb{R})}$
$\displaystyle\geq c\big{(}p,\delta,\phi(0)\big{)}.$ (3.3)

Then we obtain from (3.2) that

$\displaystyle\|\mathrm{I}_{1}\|_{L^{p}}$ $\displaystyle\geq c2^{-kn\sigma}.$
(3.4)

For the second term $\mathrm{I}_{2}$, from a straightforward calculation, we
deduce

$\displaystyle\|\mathrm{I}_{2}\|_{L^{p}}\leq
C\|\phi\|^{2d}_{L^{2p}}\sum^{n-1}_{m=1}2^{-k(n+m)\sigma}\leq
C2^{-k(n+1)\sigma}.$ (3.5)

(3.4) and (3.5) together yield that

$\displaystyle\|\Delta_{kn}\big{(}|u_{0}|^{2}\big{)}\|_{L^{p}}\geq(c-C2^{-k\sigma})2^{-kn\sigma}.$

We choose $k\geq 6$ such that $c-C2^{-k\sigma}>0$ and then finish the proof of
Proposition 3.2. ∎

###  3.2 Error Estimates

We first recall the following local-in-time exstence of strong solutions to
(1.1) in [44].

######  Lemma 3.3 (See [44]).

Let $d\geq 2,1\leq p,r\leq\infty$ and $s>\max\\{1+\frac{d}{p},\frac{3}{2}\\}$.
Assume that $u_{0}\in B^{s}_{p,r}(\mathbb{R}^{d})$, then there exists a time
$T=T(s,p,r,\|u_{0}\|_{B^{s}_{p,r}(\mathbb{R}^{d})})>0$ such that (1.1) has a
unique solution $u\in\mathcal{C}([0,T];B^{s}_{p,r}(\mathbb{R}^{d}))$.
Moreover, for all $t\in[0,T]$, there holds

$\displaystyle achm\|u(t)\|_{B^{s}_{p,r}(\mathbb{R}^{d})}\leq
C\|u_{0}\|_{B^{s}_{p,r}(\mathbb{R}^{d})}.$

######  Proposition 3.1.

Let $s=\sigma-2$ and $u_{0}\in B^{\sigma}_{p,\infty}$. Assume that $u\in
L^{\infty}_{T}B^{\sigma}_{p,\infty}$ be the solution to the Cauchy problem
(1.1), we have

$\displaystyle\|u(t)-u_{0}\|_{B^{s-1}_{p,\infty}}\leq
Ct\|u_{0}\|_{B^{s-1}_{p,\infty}}\|u_{0}\|_{B^{s}_{p,\infty}},$
$\displaystyle\|u(t)-u_{0}\|_{B^{s}_{p,\infty}}\leq
Ct\big{(}\|u_{0}\|^{2}_{B^{s}_{p,\infty}}+\|u_{0}\|_{B^{s-1}_{p,\infty}}\|u_{0}\|_{B^{s+1}_{p,\infty}}\big{)},$
$\displaystyle\|u(t)-u_{0}\|_{B^{s+1}_{p,\infty}}\leq
Ct\big{(}\|u_{0}\|_{B^{s}_{p,\infty}}\|u_{0}\|_{B^{s+1}_{p,\infty}}+\|u_{0}\|_{B^{s-1}_{p,\infty}}\|u_{0}\|_{B^{s+2}_{p,\infty}}\big{)}.$

###### Proof.

Due to Lemma 3.3, we know that there exists a positive time
$T=T(s,p,r,\|u_{0}\|_{B^{s}_{p,\infty}}(\mathbb{R}^{d}))$ such that

$\displaystyle\|u(t)\|_{L^{\infty}_{T}B^{s}_{p,\infty}}\leq
C\|u_{0}\|_{B^{s}_{p,\infty}}\leq C.$ (3.6)

Moreover, for $\gamma>\frac{1}{2}$, taking advantage of Lemma 2.4 and (3.6),
we have

$\displaystyle\|u(t)\|_{L^{\infty}_{T}B^{\gamma}_{p,\infty}}\leq
C\|u_{0}\|_{B^{\gamma}_{p,\infty}}.$ (3.7)

By the Mean Value Theorem, we obtain from (1.2) and (3.7) that

$\displaystyle\|u(t)-u_{0}\|_{B^{s}_{p,\infty}}$
$\displaystyle\leq\int^{t}_{0}\|\partial_{\tau}u\|_{B^{s}_{p,\infty}}\mathrm{d}\tau$
$\displaystyle\leq\int^{t}_{0}\|Q(u,u)\|_{B^{s}_{p,\infty}}\mathrm{d}\tau+\int^{t}_{0}\|R(u,u)\|_{B^{s}_{p,\infty}}\mathrm{d}\tau+\int^{t}_{0}\|u\cdot\nabla
u\|_{B^{s}_{p,\infty}}\mathrm{d}\tau$ $\displaystyle\leq
Ct\big{(}\|u\|^{2}_{L_{t}^{\infty}B^{s}_{p,\infty}}+\|u\|_{L_{t}^{\infty}L^{\infty}}\|\nabla
u\|_{L_{t}^{\infty}B^{s}_{p,\infty}}\big{)}$ $\displaystyle\leq
Ct\big{(}\|u\|^{2}_{L_{t}^{\infty}B^{s}_{p,\infty}}+\|u\|_{L_{t}^{\infty}B^{s-1}_{p,\infty}}\|u\|_{L_{t}^{\infty}B^{s+1}_{p,\infty}}\big{)}$
$\displaystyle\leq
Ct\big{(}\|u_{0}\|^{2}_{B^{s}_{p,\infty}}+\|u_{0}\|_{B^{s-1}_{p,\infty}}\|u_{0}\|_{B^{s+1}_{p,\infty}}\big{)},$

where we have used that $B_{p,\infty}^{s-1}\hookrightarrow L^{\infty}$ with
$s-1>\max\\{\frac{d}{p},\frac{1}{2}\\}$.

Following the same procedure as above, according to Lemmas 2.1 and 2.2, we see

$\displaystyle\|u(t)-u_{0}\|_{B^{s-1}_{p,\infty}}$
$\displaystyle\leq\int^{t}_{0}\|\partial_{\tau}u\|_{B^{s-1}_{p,\infty}}\mathrm{d}\tau$
$\displaystyle\leq\int^{t}_{0}\|Q(u,u)\|_{B^{s-1}_{p,\infty}}\mathrm{d}\tau+\int^{t}_{0}\|R(u,u)\|_{B^{s-1}_{p,\infty}}\mathrm{d}\tau+\int^{t}_{0}\|u\cdot\nabla
u\|_{B^{s-1}_{p,\infty}}\mathrm{d}\tau$ $\displaystyle\leq
Ct\|u\|_{L_{t}^{\infty}B^{s-1}_{p,\infty}}\|u\|_{L_{t}^{\infty}B^{s}_{p,\infty}}$
$\displaystyle\leq
Ct\|u_{0}\|_{B^{s-1}_{p,\infty}}\|u_{0}\|_{B^{s}_{p,\infty}}$

and

$\displaystyle\|u(t)-u_{0}\|_{B^{s+1}_{p,\infty}}$
$\displaystyle\leq\int^{t}_{0}\|\partial_{\tau}u\|_{B^{s+1}_{p,\infty}}\mathrm{d}\tau$
$\displaystyle\leq\int^{t}_{0}\|Q(u,u)\|_{B^{s+1}_{p,\infty}}\mathrm{d}\tau+\int^{t}_{0}\|R(u,u)\|_{B^{s+1}_{p,\infty}}\mathrm{d}\tau+\int^{t}_{0}\|u\cdot\nabla
u\|_{B^{s+1}_{p,\infty}}\mathrm{d}\tau$ $\displaystyle\leq
Ct\big{(}\|u\|_{L_{t}^{\infty}B^{s}_{p,\infty}}\|u\|_{L_{t}^{\infty}B^{s+1}_{p,\infty}}+\|u\|_{L_{t}^{\infty}B^{s-1}_{p,\infty}}\|u\|_{L_{t}^{\infty}B^{s+2}_{p,\infty}}\big{)}$
$\displaystyle\leq
Ct\big{(}\|u_{0}\|_{B^{s}_{p,\infty}}\|u_{0}\|_{B^{s+1}_{p,\infty}}+\|u_{0}\|_{B^{s-1}_{p,\infty}}\|u_{0}\|_{B^{s+2}_{p,\infty}}\big{)}.$

Thus, we finish the proof of Proposition 3.1. ∎

######  Proposition 3.2.

Let $s=\sigma-2$ and $u_{0}\in B^{\sigma}_{p,\infty}$. Assume that $u\in
L^{\infty}_{T}B^{\sigma}_{p,\infty}$ be the solution of the Cauchy problem
(1.1), we have

$\displaystyle\|\mathbf{w}(t,u_{0})\|_{B^{s}_{p,\infty}}\leq
Ct^{2}\big{(}\|u_{0}\|^{3}_{B^{s}_{p,\infty}}+\|u_{0}\|_{B^{s-1}_{p,\infty}}\|u_{0}\|_{B^{s}_{p,\infty}}\|u_{0}\|_{B^{s+1}_{p,\infty}}+\|u_{0}\|_{B^{s-1}_{p,\infty}}^{2}\|u_{0}\|_{B^{s+2}_{p,\infty}}\big{)},$

here and in what follows we denote

$\displaystyle\mathbf{w}(t,u_{0}):=u(t)-u_{0}-t\mathbf{v}_{0},$
$\displaystyle\mathbf{v}_{0}:=-u_{0}\cdot\nabla
u_{0}+{Q}(u_{0},u_{0})+{R}(u_{0},u_{0}).$

In particular, we obtain

$\displaystyle\|\mathbf{w}(t,u_{0})\|_{B^{\sigma-2}_{p,\infty}}\leq
C\big{(}\|u_{0}\|_{B^{\sigma}_{p,\infty}}\big{)}t^{2}.$

###### Proof.

Taking advantage of the Mean Value Theorem and (1.2), and then using Lemma 2.1
and Lemma 2.2, we find

$\displaystyle\|\mathbf{w}(t,u_{0})\|_{B^{s}_{p,\infty}}\leq$
$\displaystyle\int^{t}_{0}\|\partial_{\tau}u-\mathbf{v}_{0}\|_{B^{s}_{p,\infty}}\mathrm{d}\tau$
$\displaystyle\leq$
$\displaystyle\int^{t}_{0}\|{Q}(u,u)-{Q}(u_{0},u_{0})\|_{B^{s}_{p,\infty}}+\|{R}(u,u)-{R}(u_{0},u_{0})\|_{B^{s}_{p,\infty}}\mathrm{d}\tau$
$\displaystyle+\int^{t}_{0}\|u\cdot\nabla u-u_{0}\cdot\nabla
u_{0}\|_{B^{s}_{p,\infty}}\mathrm{d}\tau$ $\displaystyle\leq$ $\displaystyle
C\int^{t}_{0}\|u(\tau)-u_{0}\|_{B^{s}_{p,\infty}}\|u_{0}\|_{B^{s}_{p,\infty}}\mathrm{d}\tau+C\int^{t}_{0}\|u(\tau)-u_{0}\|_{B^{s-1}_{p,\infty}}\|u(\tau)\|_{B^{s+1}_{p,\infty}}\mathrm{d}\tau$
$\displaystyle+C\int^{t}_{0}\|u(\tau)-u_{0}\|_{B^{s+1}_{p,\infty}}\|u_{0}\|_{B^{s-1}_{p,\infty}}\mathrm{d}\tau$
$\displaystyle\leq$ $\displaystyle
Ct^{2}\big{(}\|u_{0}\|^{3}_{B^{s}_{p,\infty}}+\|u_{0}\|_{B^{s-1}_{p,\infty}}\|u_{0}\|_{B^{s}_{p,\infty}}\|u_{0}\|_{B^{s+1}_{p,\infty}}+\|u_{0}\|_{B^{s-1}_{p,\infty}}^{2}\|u_{0}\|_{B^{s+2}_{p,\infty}}\big{)},$

where we have used Proposition 3.1 in the last step.

Thus, we complete the proof of Proposition 3.2.

Now we present the proof of Theorem 1.1. ∎

###### Proof of Theorem 1.1:.

Using Proposition 2.2, Proposition 3.2 and Lemma 3.2, we get

$\displaystyle\|u(t)-u_{0}\|_{B^{\sigma}_{p,\infty}}\geq$ $\displaystyle
2^{{kn\sigma}}\big{\|}\Delta_{kn}\big{(}u(t)-u_{0}\big{)}\big{\|}_{L^{p}}=2^{{kn\sigma}}\big{\|}\Delta_{kn}\big{(}t\mathbf{v}_{0}+\mathbf{w}(t,u_{0})\big{)}\big{\|}_{L^{p}}$
$\displaystyle\geq$ $\displaystyle
t2^{{kn\sigma}}\|\Delta_{kn}\big{(}\mathbf{v}_{0}\big{)}\|_{L^{p}}-2^{{2kn}}2^{{kn(\sigma-2)}}\big{\|}\Delta_{kn}\big{(}\mathbf{w}(t,u_{0})\big{)}\big{\|}_{L^{p}}$
$\displaystyle\geq$ $\displaystyle
t2^{{kn}(\sigma+1)}\|\Delta_{kn}\big{(}|u_{0}|^{2}\big{)}\|_{L^{p}}-Ct(\|u_{0}\|_{B^{\sigma-1}_{p,\infty}}^{2}+\|\nabla
u_{0}\|_{B^{\sigma-1}_{p,\infty}}^{2})$
$\displaystyle-C2^{2{kn}}\|\mathbf{w}(t,u_{0})\|_{B^{\sigma-2}_{p,\infty}}$
$\displaystyle\geq$ $\displaystyle
t2^{{kn}(\sigma+1)}\|\Delta_{kn}\big{(}|u_{0}|^{2}\big{)}\|_{L^{p}}-Ct-C2^{2{kn}}t^{2}$
$\displaystyle\geq$ $\displaystyle ct2^{{kn}}-Ct-C2^{2{kn}}t^{2}.$

Then, for $k\geq 6$, taking $n>N$ large enough such that $c2^{{kn}}\geq 2C$,
we deduce that

$\displaystyle\|u(t)-u_{0}\|_{B^{\sigma}_{p,\infty}}\geq
ct2^{{kn}}-C2^{2{kn}}t^{2}.$

Thus, choosing $t2^{kn}\approx\varepsilon$ with small $\varepsilon$, we
finally conclude that

$\displaystyle\|u(t)-u_{0}\|_{B^{\sigma}_{p,\infty}}\geq
c\varepsilon-C\varepsilon^{2}\geq c_{1}\varepsilon.$

This proves Theorem 1.1. ∎

Acknowledgements. Y. Guo was supported by the Guangdong Basic and Applied
Basic Research Foundation (No. 2020A1515111092) and Research Fund of
Guangdong-Hong Kong-Macao Joint Laboratory for Intelligent Micro-Nano
Optoelectronic Technology (No. 2020B1212030010).

Conflict of interest The authors declare that they have no conflict of
interest.

## References

  * [1] H. Bahouri, J. Y. Chemin and R. Danchin, Fourier Analysis and Nonlinear Partial Differential Equations, Grundlehren der Mathematischen Wissenschaften, vol. 343, Springer-Verlag, Berlin, Heidelberg, 2011.
  * [2] A. Bressan and A. Constantin, Global conservative solutions of the Camassa-Holm equation, Arch. Ration. Mech. Anal., 183 (2007), 215-239.
  * [3] A. Bressan and A. Constantin, Global dissipative solutions of the Camassa-Holm equation, Anal. Appl., 5 (2007), 1-27.
  * [4] R. Camassa and D. D. Holm, An integrable shallow water equation with peaked solitons, Phys. Rev. Lett., 71 (1993), 1661-1664.
  * [5] R. Camassa, D. Holm and J. Hyman, A new integrable shallow water equation, Adv. Appl. Mech., 31 (1994), 1-33.
  * [6] D. Chae and J. Liu, Blow-up, zero $\alpha$ limit and the Liouville type theorem for the Euler-Poincaré equations, Comm. Math. Phys., 314 (2012), 671-687.
  * [7] A. Constantin, The Hamiltonian structure of the Camassa-Holm equation, Exposition. Math., 15 (1997), 53-85.
  * [8] A. Constantin, On the scattering problem for the Camassa-Holm equation, R. Soc. Lond. Proc. Ser. A Math. Phys. Eng. Sci., 457 (2001), 953-970.
  * [9] A. Constantin, Existence of permanent and breaking waves for a shallow water equation: a geometric approach, Ann. Inst. Fourier (Grenoble), 50 (2000), 321-362.
  * [10] A. Constantin, The trajectories of particles in Stokes waves, Invent. Math., 166 (2006), 523-535.
  * [11] A. Constantin, Particle trajectories in extreme Stokes waves, IMA J. Appl. Math., 77 (2012), 293–307.
  * [12] A. Constantin and J. Escher, Well-posedness, global existence, and blowup phenomena for a periodic quasi-linear hyperbolic equation, Comm. Pure Appl. Math., 51 (1998), 475-504.
  * [13] A. Constantin and J. Escher, Wave breaking for nonlinear nonlocal shallow water equations, Acta Math., 181 (1998), 229-243.
  * [14] A. Constantin and J. Escher, Global existence and blow-up for a shallow water equation, Ann. Scuola Norm. Sup. Pisa Cl. Sci. (4), 26 (1998), 303-328.
  * [15] A. Constantin and J. Escher, Particle trajectories in solitary water waves, Bull. Amer. Math. Soc., 44 (2007), 423-431.
  * [16] A. Constantin and J. Escher, Analyticity of periodic traveling free surface water waves with vorticity, Ann. of Math., 173 (2011), 559-568.
  * [17] A. Constantin,V. S. Gerdjikov and R. I. Ivanov Inverse scattering transform for the Camassa-Holm equation, Inverse Problems, 22 (2006), 2197–2207.
  * [18] A. Constantin and W. A. Strauss, Stability of a class of solitary waves in compressible elastic rods, Phys. Lett. A, 270 (2000), 140–148.
  * [19] A. Constantin and D. Lannes, The hydrodynamical relevance of the Camassa-Holm and Degasperis-Procesi equations, Arch. Ration. Mech. Anal., 192 (2009), 165-186.
  * [20] A. Constantin and L. Molinet Global weak solutions for a shallow water equation, Comm. Math. Phys., 211 (2000), 45-61.
  * [21] A. Constantin and W. A. Strauss, Stability of peakons, Comm. Pure Appl. Math., 53 (2000), 603-610.
  * [22] R. Danchin, A few remarks on the Camassa-Holm equation. Differential Integral Equations, 14 (2001), 953-988.
  * [23] R. Danchin, A note on well-posedness for Camassa-Holm equation, J. Differential Equations, 192 (2003), 429-444.
  * [24] A. Fokas and B. Fuchssteiner, Symplectic structures, their Bäcklund transformation and hereditary symmetries, Phys. D, 4 (1981/82), 47-66.
  * [25] Z. Guo, X. Liu, M. Luc and Z. Yin, Ill-posedness of the Camassa-Holm and related equations in the critical space, J. Differential Equations, 266 (2019), 1698-1707.
  * [26] A. Himonas and C. Kenig C 2009 Non-uniform dependence on initial data for the CH equation on the line, Diff. Integral Eqns, 22 (2009), 201–224.
  * [27] A. Himonas, C. Kenig and Misiołek Non-uniform dependence for the periodic CH equation, Commun. Partial Diff. Eqns, 35 (2010), 1145–1162.
  * [28] A. Himonas, G. Misiołek and G. Ponce, Non-uniform continuity in $H^{1}$ of the solution map of the CH equation, Asian J. Math., 11 (2007), 141–150.
  * [29] D. D. Holm and M. F. Staley, Wave structure and nonlinear balances in a family of evolutionary PDEs, SIAM J. Appl. Dyn. Syst., 2 (2003), 323–380.
  * [30] A. Himonas and G. Misiołek, Non-uniform dependence on initial data of solutions to the Euler equations of hydrodynamics, Comm. Math. Phys., 296 (2010), 285-301.
  * [31] H. Holden and X. Raynaud, Global conservative solutions of the Camassa-Holm equation—a Lagrangian point of view, Comm. Partial Differential Equations, 32 (2007), 1511–1549.
  * [32] J. Li, Y. Yu and W. Zhu, Non-uniform dependence on initial data for the Camassa-Holm equation in Besov spaces, J. Differential Equations, 269 (2020), 8686–8700.
  * [33] J. Li, X. Wu, Y. Yu and W. Zhu , Non-uniform dependence on initial data for the Camassa-Holm equation in the critical Besov space, J. Math. Fluid Mech., 23 (2021), 1422-6928.
  * [34] J. Li, L. Dai and W. Zhu, Non-uniform continuous dependence on initial data of solutions to the Euler-Poincaré system, J. Math. Phys., 60 (2019), 111510, 9.
  * [35] Y. A. Li and P. J. Olver, Well-posedness and blow-up solutions for an integrable nonlinearly dispersive model wave equation, J. Differential Equations, 162 (2000), 27–63.
  * [36] W. Luo and Z. Yin, Blow-up phenomena, ill-posedness and peakon solutions for the periodic Euler-Poincaré equations, J. of Differential Equations, 268 (2020), 1307–1325.
  * [37] J. Li, Y. Yu and W. Zhu, Ill-posedness for the Camassa-Holm and related equations in Besov spaces, arXiv preprint arXiv:2104.05973,(2021).
  * [38] J. Li, W. Deng and M. Li , Non-uniform dependence for higher dimensional Camassa-Holm equations in Besov spaces, arXiv preprint arXiv::2003.09623,(2020).
  * [39] D. Li, X. Yu and Z. Zhai, On the Euler-Poincare equation with non-zero dispersion, Arch. Ration. Mech. Anal., 210 (2013), 955-974.
  * [40] J. Li and Z. Yin, Remarks on the well-posedness of Camassa-Holm type equations in Besov spaces, J. Differential Equations, 261 (2016), 6125-6143.
  * [41] G. Rodríguez-Blanco, On the Cauchy problem for the Camassa-Holm equation, Nonlinear Anal., 46 (2001), 309-327.
  * [42] J. F. Toland, Stokes waves, Topol. Methods Nonlinear Anal., 7 (1996), 1-48.
  * [43] Z. Xin and P. Zhang, On the weak solutions to a shallow water equation, Comm. Pure Appl. Math., 53 (2000), 1411-1433.
  * [44] K. Yan and Z. Yin, On the initial value problem for higher dimensional Camassa-Holm equations, Discrete Contin. Dyn. Syst., 35 (2015), 1327-1358.
  * [45] W. Ye, Z. Yin and Y. Guo, A new result for the local well-posedness of the Camassa-Holm type equations in critial Besov spaces $B^{1+1/p}_{p,1}$, arXiv preprint arXiv:2101.00803,(2021).
  * [46] Y. Zhao, M. Yang and Y. Li, Non-uniform dependence for the periodic higher dimensional Camassa-Holm equations, J. Math. Anal. Appl., 461 (2018), 59-73.