# Extending the Scope of Out-of-Domain: Examining QA models in multiple
subdomains

Chenyang Lyu† Jennifer Foster† Yvette Graham¶
† School of Computing, Dublin City University, Dublin, Ireland
¶ School of Computer Science and Statistics, Trinity College Dublin, Dublin,
Ireland
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

###### Abstract

Past works that investigate out-of-domain performance of QA systems have
mainly focused on general domains (e.g. news domain, wikipedia domain),
underestimating the importance of subdomains defined by the internal
characteristics of QA datasets. In this paper, we extend the scope of “out-of-
domain” by splitting QA examples into different subdomains according to their
several internal characteristics including question type, text length, answer
position. We then examine the performance of QA systems trained on the data
from different subdomains. Experimental results show that the performance of
QA systems can be significantly reduced when the train data and test data come
from different subdomains. These results question the generalizability of
current QA systems in multiple subdomains, suggesting the need to combat the
bias introduced by the internal characteristics of QA datasets.

##  1 Introduction

Examining the out-of-domain performance of QA systems is an important focus of
the research community due to its direct connection to the generalizability
and robustness of QA systems especially in production environments Jia and
Liang (2017); Chen et al. (2017); Talmor and Berant (2019); Fisch et al.
(2019); Shakeri et al. (2020); Lyu et al. (2021), Even though previous studies
mostly focus on coarse-grained general domains Ruder and Sil (2021), the
importance of finer-grained subdomains defined by the internal characteristics
of QA datasets cannot be neglected. For example, several studies exploring
specific internal characteristics of QA datasets have been carried out,
including Ko et al. (2020), who reveal that the sentence-level answer position
is a source of bias for QA models, and Sen and Saffari (2020) who investigate
the effect of word-level question-context overlap.

Figure 1: We train QA systems on each subdomain and evaluate each system on
all subdomains

Building on this prior work as well as the definition and discussion of
subdomain in Plank and Sima’an (2008); Plank (2016); Varis and Bojar (2021),
we extend the scope of out-of-domain with a view to assessing the
generalizability and robustness of QA systems by investigating their out-of-
subdomain performance. As shown in Figure 1, we split the QA dataset into
different subdomains based on its internal characteristics. Then we use the QA
examples in each subdomain to train corresponding QA systems and evaluate
their performance on all subdomains.

We focus on extractive QA (EQA) as it is not only an important task in itself
Zhang et al. (2020) but also the crucial reader component in the retriever-
reader model for Open-domain QA Chen et al. (2017); Chen and Yih (2020). In
experiments with SQuAD 1.1 Rajpurkar et al. (2016) and NewsQA Trischler et al.
(2017), we split the data into subdomains based on question type, text length
(context, question and answer) and answer position. We then train QA systems
on each subdomain and examine their performance on each subdomain. Results
show that QA systems tend to perform worse when train and test data come from
different subdomains, particularly those defined by question type, answer
length and answer position.

##  2 Experiments

We employ the QA datasets SQuAD1.1 Rajpurkar et al. (2016) and NewsQA
Trischler et al. (2017). For SQuAD1.1 we use the official dataset released by
Rajpurkar et al. (2016) and for NewsQA we use the data from MRQA Fisch et al.
(2019). For question classification, we use the dataset from Li and Roth
(2002). We use the BERT-base-uncased model from Huggingface Wolf et al. (2019)
for both question classification and QA.

We adopt the following setup for training and evaluation: We split the
original training set $D$ into several disjoint subdomains
$D_{a},D_{b},D_{c},\ldots$; Then we sample subsets from each subdomain using
sample sizes $n_{1},n_{2},n_{3},\dots$ in ascending order. The resulting
subsets are denoted
$D_{a}^{n_{1}},D_{a}^{n_{2}},\dots,D_{b}^{n_{1}},D_{b}^{n_{2}},\ldots$. We
train QA systems on each subset $D_{a}^{n_{1}},D_{a}^{n_{2}},\ldots$. The QA
system trained on $D_{a}^{n_{1}}$ is denoted $QA_{a}^{n_{1}}$. We evaluate
each QA system on the test data $T$ which is also split into disjoint
subdomains $T_{a},T_{b},T_{c},\ldots$ similar to the training data $D$.

###  2.1 Question Type

|  | LOC | ENTY | HUM | NUM | DESC
---|---|---|---|---|---|---
SQuAD1.1 | Train set | 11.4 | 27.6 | 20.7 | 24.5 | 15.5
Dev set | 10.5 | 27.6 | 21.0 | 23.0 | 17.4
NewsQA | Train set | 11.4 | 16.9 | 30.0 | 18.8 | 22.6
Dev set | 12.3 | 16.9 | 32.2 | 17.8 | 20.5

Table 1: The percentage(%) of question types in the SQuAD1.1 and NewsQA train
and dev sets.

In this experiment, we investigate how QA models learn from QA examples with
different question types. We adopt question classification data (Li and Roth,
2002) to train a question classifier that categorizes questions into the
following five classes: HUM, LOC, ENTY, DESC, NUM 111Definitions and examples
provided in the Appendix.(Zhang and Lee, 2003). The QA examples in the
training data are then partitioned into five categories according to their
question type. Question type proportions for SQuAD1.1 and NewsQA are shown in
Table 1, with a high proportion of ENTY and NUM questions in SQuAD1.1, while
NewsQA has more HUM and DESC questions. We use QA examples of each question
type to train a QA system increasing the training set size in intervals of 500
from 500 to 8000. We evaluate it on the test data, which is also divided into
five categories according to question type.

Figure 2: Visualization of F-1 learning curves for the QA systems trained on
the subdomains of five question types (HUM,LOC,ENTY,DESC,NUM), tested on the
subdomains for each question type and the original dev set of SQuAD1.1 (top)
and NewsQA (bottom). Figure 3: Visualization of performance (EM and F-1 score)
ratio curves over long and short context, question and answer (from left to
right) on SQuAD1.1 (top) and NewsQA (bottom). The green, red lines represent
the ratio of the performance on long group and short group. The dashed line is
1, indicating that two QA systems have the same performance. When the sample
size increases, curves in context and question length converge to the dashed
line, whereas there are substantial differences in the performance of $QA_{L}$
and $QA_{S}$ in answer length subdomain.

A visualisation of the resulting F-1 scores of the QA systems trained on each
question type subdomain is shown in Figure 2, for both SQuAD1.1 and NewsQA.
The x-axis represents the training set size, the y-axis is the F-1 score. The
results show that a QA system learns to answer a certain type of question
mainly from the examples of the same question type – this is particularly true
for HUM and NUM questions in SQuAD1.1 and HUM, LOC and NUM questions in
NewsQA. Taking NUM questions as an example, the rightmost plots in Figure 2
show that performance on other question types only results in a minor
improvement as the training set size increases compared to performance
improvements on that question type (NUM). In other words, the QA system gets
most of the knowledge it needs to answer NUM questions from the NUM training
examples and a similar pattern is also present for other question types. The
results in Figure 2 show that the subdomains defined by question type is a
source of bias when training and employing QA systems. We suspect that word
use and narrative style varies over question types, which inject bias into QA
systems when learning from QA examples with different question types.
Therefore, we need to improve the diversity of question types when
constructing and organising QA data.

###  2.2 Text Length

Figure 4: Visualization of performance (EM and F-1 score) ratio curves over
front and back answer positions (char-level, word-level and sentence-level
from left to right) on SQuAD1.1 (top) and NewsQA (bottom). The green, red
lines represent the ratio of the performance on front group and back group.
The dashed line is 1, indicating that two QA systems have the same
performance. The curves show that there are substantially difference in the
performance of $QA_{F}$ and $QA_{B}$ in answer position subdomains especially
for character-level and word-level answer positions.

The effect of text length to the performance and generalizability of neural
models has been discussed in text classification and machine translation
Amplayo et al. (2019); Varis and Bojar (2021). As for QA, there are three
components in a QA example: context, question, answer. The length of each
component could potentially introduce additional bias and affect how QA
systems learn from QA data. For example, a short context could be easy since a
shorter context could reduce the search space for QA models to locate the
answer; on the other hand, a short context could be hard as shorter context
contains less information to find the answer. Therefore, the following
question arises naturally: are short and long context/question/answer are
equivalent?

To answer this question, we split the QA datasets into short and long groups
according to the median of the length of context, question, answer 222See more
statistics in the Appendix.. Then we train QA systems on the QA examples
sampled from short ($QA_{S,context},QA_{S,question},QA_{S,answer}$) and long
($QA_{L,context},QA_{L,question},QA_{L,answer}$) groups respectively, again
increasing the training set size in intervals of 500 from 500 to 25000. The
results are shown in Figure 3, where the x-axis is the data size and the
y-axis is the ratio of the performance (EM and F-1 score) of the $QA_{S}$ and
corresponding $QA_{L}$ systems on the text length subdomains of context,
question, answer. If $QA_{L}$ and $QA_{S}$ have no obvious difference in terms
of performance on long and short groups respectively, the ratio of their
performance should be close to $1$. The results show that the performances of
$QA_{L}$ and $QA_{S}$ trained on the subdomains of context and question length
have no obvious difference as all the three curves converge to $1$, although
there are fluctuations when the sample sizes are small. In contrast, $QA_{L}$
and $QA_{S}$ trained on the subdomains of answer length behave differently –
see the subplots in the two rightmost columns of Figure 3. $QA_{L}$ performs
much better than $QA_{S}$ on the test examples with long answers and $QA_{L}$
performs much worse than $QA_{S}$ on the test examples with short answers. The
results in Figure 3 show that the length of answer introduces strong bias to
QA systems. We think this stems from the fact that $QA_{L}$ tends to predict
longer answers, whereas $QA_{S}$ tends to predict shorter answers, and they
thus underperform in the counterpart subdomain. We show the average length of
the predicted answers of QA systems trained on long and short groups in Table
2. Therefore, it is important to control the length distribution of answers
when constructing and organising QA dataset especially using NER tools in the
answer extraction phase since they tend to find shorter answers.

|  | Context |  | Question |  | Answer
---|---|---|---|---|---|---
|  | Long | Short |  | Long | Short |  | Long | Short
SQuAD1.1 |  | 4.03 | 4.13 |  | 4.00 | 4.23 |  | 6.41 | 2.78
NewsQA |  | 5.46 | 5.33 |  | 5.16 | 5.87 |  | 9.57 | 3.51

Table 2: The average length of predicted answers of QA systems trained on long
and short subdomains of context, question and answer on SQuAD1.1 and NewsQA.

###  2.3 Answer Position

Ko et al. (2020) study the effect of sentence-level answer position. Building
on their analysis, we study the effect of two more types of answer position:
character-level position and word-level position. We split the training set
into front and back groups based on the median of the answer start positions
at the character, word and sentence level 333See more statistics in the
Appendix.. Then we train QA systems on the examples sampled from front
($QA_{F,char},QA_{F,word},QA_{F,sent}$) and back
($QA_{B,char},QA_{B,word},QA_{B,sent}$) groups respectively , increasing the
training set size in intervals of 500 from 500 to 25000. We show the results
in Figure 4, where the x-axis is the training set size and the y-axis is the
ratio of the performance (EM and F-1 score) of $QA_{F}$ and $QA_{B}$ on the
answer position subdomains at the character, word and sentence level.

The results show that answer position on all three levels are a source of
bias. $QA_{F}$ performs much better than $QA_{B}$ on test instances with
answer positions in the front, whereas $QA_{B}$ performs much better than
$QA_{F}$ on test instances with answer positions at the back. The effect of
bias is more serious at the character and word level. We think this answer
position bias is happening because words in different positions have different
position embeddings, which could also affect word semantics in transformer
architectures Vaswani et al. (2017); Wang et al. (2020). This suggests the
need to make sure answer position distribution is balanced as well as the need
to develop QA systems that are more robust to answer position variation.

##  3 Discussion and Conclusion

We presented a series of experiments investigating the out-of-subdomain
performance of QA systems on two popular English extractive QA datasets:
SQuAD1.1 and NewsQA. The experimental results show that the subdomains defined
by question type, answer length, answer position inject strong bias into QA
systems, with the result that the performance of QA systems is negatively
impacted when train and test data come from different subdomains. The
experiments provide useful information on how to control question diversity,
answer length distribution as well as the answer positions when constructing
QA datasets and employing QA systems. In future work, we aim to apply our
analysis to multilingual data to explore how QA models behave across different
languages and we plan to investigate other types of QA beyond extractive QA.

## Acknowledgements

This work was funded by Science Foundation Ireland through the SFI Centre for
Research Training in Machine Learning (18/CRT/6183). We thank James Barry for
his proofreading and helpful comments. We also thank the reviewers for their
helpful comments.

## References

  * Amplayo et al. (2019) Reinald Kim Amplayo, Seonjae Lim, and Seung-won Hwang. 2019.  Text length adaptation in sentiment classification.  In _Asian Conference on Machine Learning_ , pages 646–661. PMLR.
  * Chen et al. (2017) Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017.  Reading Wikipedia to answer open-domain questions.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1870–1879, Vancouver, Canada. Association for Computational Linguistics.
  * Chen and Yih (2020) Danqi Chen and Wen-tau Yih. 2020.  Open-domain question answering.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts_ , pages 34–37, Online. Association for Computational Linguistics.
  * Fisch et al. (2019) Adam Fisch, Alon Talmor, Robin Jia, Minjoon Seo, Eunsol Choi, and Danqi Chen. 2019\.  MRQA 2019 shared task: Evaluating generalization in reading comprehension.  In _Proceedings of the 2nd Workshop on Machine Reading for Question Answering_ , pages 1–13, Hong Kong, China. Association for Computational Linguistics.
  * Jia and Liang (2017) Robin Jia and Percy Liang. 2017.  Adversarial examples for evaluating reading comprehension systems.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 2021–2031, Copenhagen, Denmark. Association for Computational Linguistics.
  * Ko et al. (2020) Miyoung Ko, Jinhyuk Lee, Hyunjae Kim, Gangwoo Kim, and Jaewoo Kang. 2020.  Look at the first sentence: Position bias in question answering.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1109–1121, Online. Association for Computational Linguistics.
  * Li and Roth (2002) Xin Li and Dan Roth. 2002.  Learning question classifiers.  In _COLING 2002: The 19th International Conference on Computational Linguistics_.
  * Lyu et al. (2021) Chenyang Lyu, Lifeng Shang, Yvette Graham, Jennifer Foster, Xin Jiang, and Qun Liu. 2021.  Improving unsupervised question answering via summarization-informed question generation.  _arXiv preprint arXiv:2109.07954_.
  * Plank (2016) Barbara Plank. 2016.  What to do about non-standard (or non-canonical) language in NLP.  In _Proceedings of the 13th Conference on Natural Language Processing, KONVENS 2016, Bochum, Germany, September 19-21, 2016_ , volume 16 of _Bochumer Linguistische Arbeitsberichte_.
  * Plank and Sima’an (2008) Barbara Plank and Khalil Sima’an. 2008.  Subdomain sensitive statistical parsing using raw corpora.  In _Proceedings of the International Conference on Language Resources and Evaluation, LREC 2008, 26 May - 1 June 2008, Marrakech, Morocco_. European Language Resources Association.
  * Rajpurkar et al. (2016) Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016.  SQuAD: 100,000+ questions for machine comprehension of text.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_ , pages 2383–2392, Austin, Texas. Association for Computational Linguistics.
  * Ruder and Sil (2021) Sebastian Ruder and Avi Sil. 2021.  Multi-domain multilingual question answering.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: Tutorial Abstracts_ , pages 17–21, Punta Cana, Dominican Republic & Online. Association for Computational Linguistics.
  * Sen and Saffari (2020) Priyanka Sen and Amir Saffari. 2020.  What do models learn from question answering datasets? In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 2429–2438, Online. Association for Computational Linguistics.
  * Shakeri et al. (2020) Siamak Shakeri, Cicero Nogueira dos Santos, Henghui Zhu, Patrick Ng, Feng Nan, Zhiguo Wang, Ramesh Nallapati, and Bing Xiang. 2020.  End-to-end synthetic data generation for domain adaptation of question answering systems.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 5445–5460, Online. Association for Computational Linguistics.
  * Talmor and Berant (2019) Alon Talmor and Jonathan Berant. 2019.  MultiQA: An empirical investigation of generalization and transfer in reading comprehension.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 4911–4921, Florence, Italy. Association for Computational Linguistics.
  * Trischler et al. (2017) Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. 2017.  NewsQA: A machine comprehension dataset.  In _Proceedings of the 2nd Workshop on Representation Learning for NLP_ , pages 191–200, Vancouver, Canada. Association for Computational Linguistics.
  * Varis and Bojar (2021) Dusan Varis and Ondřej Bojar. 2021.  Sequence length is a domain: Length-based overfitting in transformer models.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 8246–8257, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _CoRR_ , abs/1706.03762.
  * Wang et al. (2020) Benyou Wang, Lifeng Shang, Christina Lioma, Xin Jiang, Hao Yang, Qun Liu, and Jakob Grue Simonsen. 2020.  On position embeddings in bert.  In _International Conference on Learning Representations_.
  * Wolf et al. (2019) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R’emi Louf, Morgan Funtowicz, and Jamie Brew. 2019.  Huggingface’s transformers: State-of-the-art natural language processing.  _ArXiv_ , abs/1910.03771.
  * Zhang and Lee (2003) Dell Zhang and Wee Sun Lee. 2003.  Question classification using support vector machines.  In _Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval_ , pages 26–32.
  * Zhang et al. (2020) Zhuosheng Zhang, Hai Zhao, and Rui Wang. 2020.  Machine reading comprehension: The role of contextualized language models and beyond.

##  Appendix A Appendix

###  A.1 Experimental Setup

We use bert-based-uncased as our QA model, the learning rate is set to 3e-5,
the maximum sequence length is set to 384, the doc stride length is set to
128, we run the training process for 2 epochs for training each QA system in
experiments, the training was conducted on one GeForce GTX 3090 GPU, the
training batch size is 48.

###  A.2 Average Text Length and Answer Position for All Question Types

We show the average text length of context, question and answer as well as the
average answer position on character-level, word-level and sentence-level for
QA examples in all question types in SQuAD1.1 and NewsQA in Table 3 and Table
4.

|  | Context | Question | Answer
---|---|---|---|---
SQuAD1.1 | HUM | 123.20 | 9.79 | 2.82
LOC | 117.18 | 9.62 | 2.78
DESC | 119.32 | 9.91 | 5.82
ENTY | 117.43 | 10.54 | 3.04
NUM | 121.09 | 10.11 | 2.08
NewsQA | HUM | 495.79 | 6.55 | 2.82
LOC | 478.84 | 6.34 | 2.87
DESC | 513.00 | 6.25 | 7.62
ENTY | 505.94 | 6.76 | 4.27
NUM | 476.23 | 7.20 | 2.07

Table 3: The average text length of context, question and answer in QA
examples of each question type in the SQuAD1.1 and NewsQA training data.

|  | Char-Level | Word-Level | Sent-Level
---|---|---|---|---
SQuAD1.1 | HUM | 317.85 | 54.90 | 1.61
LOC | 308.81 | 53.71 | 1.53
DESC | 342.97 | 60.00 | 1.79
ENTY | 317.75 | 55.16 | 1.63
NUM | 315.78 | 56.19 | 1.67
NewsQA | HUM | 532.11 | 101.02 | 3.71
LOC | 566.02 | 107.99 | 3.95
DESC | 844.13 | 160.05 | 5.98
ENTY | 751.48 | 143.90 | 5.49
NUM | 763.73 | 145.26 | 5.47

Table 4: The average answer position of character-level, word-level and
sentence-level in QA examples of each question type in the SQuAD1.1 and NewsQA
training data.

| Context | Question | Answer
---|---|---|---
SQuAD1.1 | 110 | 10 | 2
NewsQA | 534 | 6 | 2

Table 5: The median of the context, question, answer length used to partition
long and short subdomains.

|  | LOC | ENTY | HUM | NUM | DESC
---|---|---|---|---|---|---
SQuAD1.1 | Long | 11.11 | 26.68 | 21.65 | 24.8 | 15.43
Short | 11.73 | 28.42 | 19.68 | 24.2 | 15.52
NewsQA | Long | 10.4 | 18.08 | 29.94 | 16.81 | 24.71
Short | 12.38 | 15.86 | 30.24 | 20.9 | 20.55

Table 6: The percentage of each question type in long context and short
context groups.

|  | LOC | ENTY | HUM | NUM | DESC
---|---|---|---|---|---|---
SQuAD1.1 | Long | 10.36 | 28.59 | 20.37 | 24.73 | 15.63
Short | 12.11 | 26.90 | 20.84 | 24.35 | 15.37
NewsQA | Long | 9.45 | 18.29 | 29.70 | 23.66 | 18.90
Short | 12.96 | 15.91 | 30.40 | 14.98 | 25.63

Table 7: The percentage of each question type in long question and short
question groups.

|  | LOC | ENTY | HUM | NUM | DESC
---|---|---|---|---|---|---
SQuAD1.1 | Long | 10.87 | 27.32 | 19.69 | 21.8 | 19.86
Short | 11.79 | 27.72 | 21.29 | 26.29 | 12.55
NewsQA | Long | 9.37 | 19.87 | 23.16 | 9.31 | 38.17
Short | 13.13 | 14.48 | 36.03 | 27.05 | 9.29

Table 8: The percentage of each question type in long answer and short answer
groups.

|  | Context | Question | Answer
---|---|---|---|---
SQuAD1.1 | Long | 84.53 | 9.99 | 3.09
Short | 155.88 | 10.14 | 3.23
NewsQA | Long | 350.44 | 6.54 | 3.79
Short | 641.35 | 6.69 | 4.25

Table 9: The average answer position on character-level, word-level and
sentence-level in QA examples of long context and short context groups.

|  | Context | Question | Answer
---|---|---|---|---
SQuAD1.1 | Long | 119.12 | 7.8 | 3.25
Short | 120.76 | 13.57 | 3.03
NewsQA | Long | 491.15 | 4.96 | 4.45
Short | 501.55 | 8.66 | 3.49

Table 10: The average answer position on character-level, word-level and
sentence-level in QA examples of long question and short question groups.

|  | Context | Question | Answer
---|---|---|---|---
SQuAD1.1 | Long | 119.08 | 10.18 | 1.42
Short | 120.79 | 9.88 | 5.77
NewsQA | Long | 489.32 | 6.82 | 1.5
Short | 503.34 | 6.37 | 6.95

Table 11: The average answer position on character-level, word-level and
sentence-level in QA examples of long answer and short answer groups.

|  | Char | Word | Sent
---|---|---|---|---
SQuAD1.1 | Long | 402.02 | 70.36 | 2.14
Short | 239.75 | 41.78 | 1.17
NewsQA | Long | 864.85 | 165.73 | 6.40
Short | 510.58 | 95.94 | 3.37

Table 12: The average answer position on character-level, word-level and
sentence-level in QA examples of long context and short context groups.

|  | Char | Word | Sent
---|---|---|---|---
SQuAD1.1 | Long | 342.02 | 59.70 | 1.74
Short | 305.65 | 53.45 | 1.58
NewsQA | Long | 726.78 | 138.64 | 5.22
Short | 655.98 | 124.50 | 4.61

Table 13: The average answer position on character-level, word-level and
sentence-level in QA examples of long question and short question groups.

|  | Char | Word | Sent
---|---|---|---|---
SQuAD1.1 | Long | 324.65 | 57.77 | 1.71
Short | 316.70 | 54.65 | 1.60
NewsQA | Long | 795.46 | 150.20 | 5.61
Short | 595.00 | 114.17 | 4.26

Table 14: The average answer position on character-level, word-level and
sentence-level in QA examples of long answer and short answer groups.

###  A.3 Question Type Proportions, Average Text Length and Average Answer
Position for Long and Short Text Length

The median of the context, question, answer is shown in Table 5. We show the
question type proportion, average text length for context, question and answer
as well as the average answer position on character-level, word-level and
sentence-level for QA examples in long and short groups of context, question,
answer in SQuAD1.1 and NewsQA in Table 6, Table 7, Table 8, Table 9 Table 10,
Table 11, Table 12, Table 13, Table 14.

###  A.4 Question Type Proportions, Average Text Length and Average Answer
Position for QA examples with Front and Back Answer Positions

The median of the answer position on character-level, word-level and sentence-
level is shown in Table 15. We show the question type proportion, average text
length for context, question and answer as well as the average answer position
on character-level, word-level and sentence-level for QA examples in front and
back groups of answer positions in character-level, word-level and sentence-
level in SQuAD1.1 and NewsQA in Table 16, Table 17, Table 18, Table 19, Table
20, Table 21, Table 22, Table 23, Table 24.

| Char | Word | Sent
---|---|---|---
SQuAD1.1 | 262 | 46 | 1
NewsQA | 358 | 67 | 2

Table 15: The median of the answer position on character-level, word-level and
sentence-level used to partition front and back subdomains.

|  | LOC | ENTY | HUM | NUM | DESC
---|---|---|---|---|---|---
SQuAD1.1 | Front | 11.74 | 27.8 | 20.25 | 24.97 | 14.81
Back | 11.11 | 27.32 | 21.06 | 24.02 | 16.14
NewsQA | Front | 13.07 | 15.59 | 37.2 | 15.61 | 18.46
Back | 9.71 | 18.36 | 22.97 | 22.1 | 26.8

Table 16: The percentage of each question type in front and back groups on
character-level answer position

|  | LOC | ENTY | HUM | NUM | DESC
---|---|---|---|---|---|---
SQuAD1.1 | Front | 11.76 | 28.05 | 20.28 | 24.49 | 14.99
Back | 11.16 | 27.08 | 21.00 | 24.45 | 15.94
NewsQA | Front | 13.02 | 15.59 | 37.20 | 15.64 | 18.48
Back | 9.74 | 18.43 | 22.85 | 22.11 | 26.81

Table 17: The percentage of each question type in front and back groups on
word-level answer position

|  | LOC | ENTY | HUM | NUM | DESC
---|---|---|---|---|---|---
SQuAD1.1 | Front | 11.72 | 27.83 | 20.60 | 24.48 | 14.95
Back | 11.04 | 27.18 | 20.71 | 24.56 | 16.15
NewsQA | Front | 13.19 | 15.76 | 36.08 | 16.36 | 18.54
Back | 9.56 | 18.54 | 23.11 | 22.06 | 26.67

Table 18: The percentage of each question type in front and back groups on
sentence-level answer position

|  | Char | Word | Sent
---|---|---|---|---
SQuAD1.1 | Front | 116.25 | 20.6 | 0.44
Back | 524.15 | 91.3 | 2.85
NewsQA | Front | 145.24 | 28.72 | 0.61
Back | 1230.24 | 232.96 | 9.15

Table 19: The average answer position on character-level, word-level and
sentence-level in QA examples of front and back groups of character-level
answer position.

|  | Char | Word | Sent
---|---|---|---|---
SQuAD1.1 | Front | 127.4 | 19.34 | 0.44
Back | 515.71 | 93.09 | 2.88
NewsQA | Front | 151.46 | 28.04 | 0.65
Back | 1229.77 | 234.74 | 9.17

Table 20: The average answer position on character-level, word-level and
sentence-level in QA examples of front and back groups of word-level answer
position.

|  | Char | Word | Sent
---|---|---|---|---
SQuAD1.1 | Front | 158.46 | 26.12 | 0.4
Back | 532.52 | 95.11 | 3.28
NewsQA | Front | 183.56 | 35.56 | 0.63
Back | 1280.56 | 242.86 | 9.89

Table 21: The average answer position on character-level, word-level and
sentence-level in QA examples of front and back groups of sentence-level
answer position.

|  | Context | Question | Answer
---|---|---|---|---
SQuAD1.1 | Front | 108.80 | 9.83 | 3.06
Back | 130.77 | 10.30 | 3.26
NewsQA | Front | 473.52 | 6.50 | 3.28
Back | 518.08 | 6.72 | 4.75

Table 22: The average text length of context, question and answer in QA
examples of front and back groups of character-level answer position

|  | Context | Question | Answer
---|---|---|---|---
SQuAD1.1 | Front | 109.21 | 9.84 | 3.03
Back | 130.50 | 10.28 | 3.30
NewsQA | Front | 473.13 | 6.50 | 3.32
Back | 518.72 | 6.72 | 4.72

Table 23: The average text length of context, question and answer in QA
examples of front and back groups of word-level answer position

|  | Context | Question | Answer
---|---|---|---|---
SQuAD1.1 | Front | 110.14 | 9.93 | 3.04
Back | 132.44 | 10.23 | 3.33
NewsQA | Front | 474.28 | 6.52 | 3.58
Back | 521.11 | 6.73 | 4.54

Table 24: The average text length of context, question and answer in QA
examples of front and back groups of sentence-level answer position Figure 5:
Visualization of performance (EM and F-1 score) difference curves over short
and long context, question and answer (from left to right) on SQuAD1.1 (top)
and NewsQA (bottom). The green, red lines represent the difference of the
performance on long group and short group. The dashed line is 0, indicating
that two QA systems have the same performance. When the sample size increases,
curves in context and question length converge to the dashed line, whereas
there are substantial differences in the performance of $QA_{L}$ and $QA_{S}$
in answer length subdomain. Figure 6: Visualization of performance (EM and F-1
score) difference curves over front and back answer positions (char-level,
word-level and sentence-level from left to right) on SQuAD1.1 (top) and NewsQA
(bottom). The green, red lines represent the difference of the performance on
front group and back group. The dashed line is 0, indicating that two QA
systems have the same performance. The curves show that there are
substantially difference in the performance of $QA_{F}$ and $QA_{B}$ in answer
position subdomains especially for character-level and word-level answer
positions.

###  A.5 Question type definition and examples

We show the definitions of question type HUM, LOC, ENTY, DESC, NUM and some
examples from the question classification
data444https://cogcomp.seas.upenn.edu/Data/QA/QC/ and predictions of SQuAD1.1
and NewsQA in Table 25.

Question type |  Definition |  Examples
---|---|---
HUM |  people, individual, group, title |  What contemptible scoundrel stole the cork from my lunch ?
Which professor sent the first wireless message in the USA ?
Who was sentenced to death in February ?
LOC |  location, city, country, mountain, state |  Where is the Kalahari desert ?
Where is the theology library at Notre Dame ?
Where was Cretan when he heard screams ?
ENTY |  animal, body, color, creation, currency, disease/medical, event, food, instrument, language, plant, product, religion, sport, symbol, technique, term, vehicle |  What relative of the racoon is sometimes known as the cat-bear ?
What is the world’s oldest monographic music competition ?
What was the name of the film about Jack Kevorkian ?
DESC |  definition, description, manner, reason |  What is Eagle ’s syndrome styloid process ?
How did Beyonce describe herself as a feminist ?
What are suspects blamed for ?
NUM |  code, count, date, distance, money, order, other, percent, period, speed, temperature, size, weight |  How many calories are there in a Big Mac ?
What year did Nintendo announce a new Legend of Zelda was in the works for
Gamecube ?
How many tons of cereal did Kelloggs donate ?
Table 25: Definition of each question type and corresponding examples in
SQuAD1.1 and NewsQA.

###  A.6 QA examples with long answers and short answers

We give some QA examples with long answers and short answers in Table 26 and
Table 27.

Answer Length |  Question |  Context
---|---|---
Long |  Where was the main focus of Pan-Slavism? |  Pan-Slavism, a movement which came into prominence in the mid-19th century, emphasized the common heritage and unity of all the Slavic peoples. The main focus was in the Balkans where the South Slavs had been ruled for centuries by other empires: the Byzantine Empire, Austria-Hungary, the Ottoman Empire, and Venice. The Russian Empire used Pan-Slavism as a political tool; as did the Soviet Union, which gained political-military influence and control over most Slavic-majority nations between 1945 and 1948 and retained a hegemonic role until the period 1989–1991.
Long |  What is one reason for homologs to appear? |  Genes with a most recent common ancestor, and thus a shared evolutionary ancestry, are known as homologs. These genes appear either from gene duplication within an organism’s genome, where they are known as paralogous genes, or are the result of divergence of the genes after a speciation event, where they are known as orthologous genes,:7.6 and often perform the same or similar functions in related organisms. It is often assumed that the functions of orthologous genes are more similar than those of paralogous genes, although the difference is minimal.
Long |  How does the water vapor that rises in warm air turn into clouds? |  Solar radiation is absorbed by the Earth’s land surface, oceans – which cover about 71% of the globe – and atmosphere. Warm air containing evaporated water from the oceans rises, causing atmospheric circulation or convection. When the air reaches a high altitude, where the temperature is low, water vapor condenses into clouds, which rain onto the Earth’s surface, completing the water cycle. The latent heat of water condensation amplifies convection, producing atmospheric phenomena such as wind, cyclones and anti-cyclones. Sunlight absorbed by the oceans and land masses keeps the surface at an average temperature of 14 °C. By photosynthesis green plants convert solar energy into chemically stored energy, which produces food, wood and the biomass from which fossil fuels are derived.
Table 26: Examples of QA examples with long answers where answers are highlighted. Answer Length |  Question |  Context
---|---|---
Short |  Who led the Exodus? |  According to the Hebrew Bible narrative, Jewish ancestry is traced back to the Biblical patriarchs such as Abraham, Isaac and Jacob, and the Biblical matriarchs Sarah, Rebecca, Leah, and Rachel, who lived in Canaan around the 18th century BCE. Jacob and his family migrated to Ancient Egypt after being invited to live with Jacob’s son Joseph by the Pharaoh himself. The patriarchs’ descendants were later enslaved until the Exodus led by Moses, traditionally dated to the 13th century BCE, after which the Israelites conquered Canaan.
Short |  When did the Duke of Kent die? |  Victoria was the daughter of Prince Edward, Duke of Kent and Strathearn, the fourth son of King George III. Both the Duke of Kent and King George III died in 1820, and Victoria was raised under close supervision by her German-born mother Princess Victoria of Saxe-Coburg-Saalfeld. She inherited the throne aged 18, after her father’s three elder brothers had all died, leaving no surviving legitimate children. The United Kingdom was already an established constitutional monarchy, in which the sovereign held relatively little direct political power. Privately, Victoria attempted to influence government policy and ministerial appointments; publicly, she became a national icon who was identified with strict standards of personal morality.
Short |  What is the evaluator called in a breed show? |  In conformation shows, also referred to as breed shows, a judge familiar with the specific dog breed evaluates individual purebred dogs for conformity with their established breed type as described in the breed standard. As the breed standard only deals with the externally observable qualities of the dog (such as appearance, movement, and temperament), separately tested qualities (such as ability or health) are not part of the judging in conformation shows.
Table 27: Examples of QA examples with short answers where answers are
highlighted.

###  A.7 QA examples with front answers and back answers

We give some QA examples with character-level answer positions in front group
and back group in Table 28 and Table 29.

Answer Position |  Question |  Context
---|---|---
Front |  What are the first names of the men that invented youtube? |  According to a story that has often been repeated in the media, Hurley and Chen developed the idea for YouTube during the early months of 2005, after they had experienced difficulty sharing videos that had been shot at a dinner party at Chen’s apartment in San Francisco. Karim did not attend the party and denied that it had occurred, but Chen commented that the idea that YouTube was founded after a dinner party ẅas probably very strengthened by marketing ideas around creating a story that was very digestible.̈
Front |  Who became Chairman of the Council of Ministers in 1985? |  In the fall of 1985, Gorbachev continued to bring younger and more energetic men into government. On September 27, Nikolai Ryzhkov replaced 79-year-old Nikolai Tikhonov as Chairman of the Council of Ministers, effectively the Soviet prime minister, and on October 14, Nikolai Talyzin replaced Nikolai Baibakov as chairman of the State Planning Committee (GOSPLAN). At the next Central Committee meeting on October 15, Tikhonov retired from the Politburo and Talyzin became a candidate. Finally, on December 23, 1985, Gorbachev appointed Yeltsin First Secretary of the Moscow Communist Party replacing Viktor Grishin.
Front |  During what seasons is fog common in Boston? |  Fog is fairly common, particularly in spring and early summer, and the occasional tropical storm or hurricane can threaten the region, especially in late summer and early autumn. Due to its situation along the North Atlantic, the city often receives sea breezes, especially in the late spring, when water temperatures are still quite cold and temperatures at the coast can be more than 20 °F (11 °C) colder than a few miles inland, sometimes dropping by that amount near midday. Thunderstorms occur from May to September, that are occasionally severe with large hail, damaging winds and heavy downpours. Although downtown Boston has never been struck by a violent tornado, the city itself has experienced many tornado warnings. Damaging storms are more common to areas north, west, and northwest of the city. Boston has a relatively sunny climate for a coastal city at its latitude, averaging over 2,600 hours of sunshine per annum.
Table 28: Examples of QA examples with answers in front group where answers are highlighted. Answer Position |  Question |  Context
---|---|---
Back |  How many murders did Detroit have in 2015? |  Detroit has struggled with high crime for decades. Detroit held the title of murder capital between 1985-1987 with a murder rate around 58 per 100,000. Crime has since decreased and, in 2014, the murder rate was 43.4 per 100,000, lower than in St. Louis, Missouri. Although the murder rate increased by 6% during the first half of 2015, it was surpassed by St Louis and Baltimore which saw much greater spikes in violence. At year-end 2015, Detroit had 295 criminal homicides, down slightly from 299 in 2014.
Back |  Who was leading the Conservatives at this time? |  Despite being a persistent critic of some of the government’s policies, the paper supported Labour in both subsequent elections the party won. For the 2005 general election, The Sun backed Blair and Labour for a third consecutive election win and vowed to give him öne last chanceẗo fulfil his promises, despite berating him for several weaknesses including a failure to control immigration. However, it did speak of its hope that the Conservatives (led by Michael Howard) would one day be fit for a return to government. This election (Blair had declared it would be his last as prime minister) resulted in Labour’s third successive win but with a much reduced majority.
Back |  Who lost the 2015 Nigerian presidential election? |  Nigeria is a Federal Republic modelled after the United States, with executive power exercised by the president. It is influenced by the Westminster System model[citation needed] in the composition and management of the upper and lower houses of the bicameral legislature. The president presides as both Head of State and head of the national executive; the leader is elected by popular vote to a maximum of two 4-year terms. In the March 28, 2015 presidential election, General Muhammadu Buhari emerged victorious to become the Federal President of Nigeria, defeating then incumbent Goodluck Jonathan.
Table 29: Examples of QA examples with answers in back group where answers are
highlighted.

###  A.8 Performance Difference for Text Length and Answer Position
Experiments

We also show the difference of the performance (EM and F-1 score) between QA
systems ($QA_{L}-QA_{S}$ and $QA_{F}-QA_{B}$) on subdomains of text length and
answer position in Figure 5 and Figure 6.


dx_{d}$ $\displaystyle\leq\left\lVert
K\right\rVert_{\infty}^{2(d-4)}\left\lVert
K^{\prime}\right\rVert_{\infty}^{2}\int_{0}^{r_{\text{max}}}\left(\frac{\ln(r_{\text{max}}/r)+1}{\ln(r_{\text{max}}/r_{\text{min}})}\wedge
1\right)^{2}rdr\left(\frac{\prod_{i=3}^{d}h_{i}}{h_{k}^{2}}\right)$

where we used (71). Now by exactly the same computation yielding to (78), we
have,

$I_{3}\leq c\left\lVert K\right\rVert_{\infty}^{2(d-4)}\left\lVert
K^{\prime}\right\rVert_{\infty}^{2}\frac{\prod_{i=3}^{d}h_{i}}{h_{k}^{2}}\frac{r_{\text{max}}^{2}}{\ln(r_{\text{max}}/r_{\text{min}})}.$

Using that $r_{\text{max}}\leq h_{k}$, we deduce

$I_{3}\leq c\left\lVert K\right\rVert_{\infty}^{2(d-4)}\left\lVert
K^{\prime}\right\rVert_{\infty}^{2}\frac{\prod_{i=3}^{d}h_{i}}{\ln(r_{\text{max}}/r_{\text{min}})}$
(82)

Collecting (81), (78) and (82) we get

$\left\lVert b_{k}^{(1)}-b_{k}^{(0)}\right\rVert_{2}^{2}\leq
c\frac{\prod_{i=3}^{d}h_{i}}{M_{T}^{2}\ln(\frac{r_{\text{max}}}{r_{\text{min}}})}.$
(83)

Eventually, the upper bound (73) is a consequence of (80), (83) and
$b_{\theta}^{(1)}=b_{\theta}^{(0)}=0$.

$\bullet$ We now prove (74). Using again
$\pi^{(1)}=\pi^{(0)}+\frac{d_{T}}{M_{T}}$ and (67), we have

$\left\lVert
b^{(1)}-b^{(0)}\right\rVert_{\infty}\leq\frac{c}{M_{T}}\left[\left\lVert
d_{T}\right\rVert_{\infty}+\left\lVert\frac{\partial d_{T}}{\partial
r}\right\rVert_{\infty}+\sum_{k=3}^{d}\left\lVert\frac{\partial
d_{T}}{\partial x_{k}}\right\rVert_{\infty}\right].$

By the definition (68) of $d_{T}$ with (69), we have $\left\lVert
d_{T}\right\rVert_{\infty}\leq c$ and $\left\lVert\frac{\partial
d_{T}}{\partial x_{k}}\right\rVert_{\infty}\leq\frac{c}{h_{k}}$ for $k\geq 3$.
Using (72), we have $\left\lVert\frac{\partial d_{T}}{\partial
r}\right\rVert_{\infty}\leq\frac{c}{r_{\text{min}}\ln(\frac{r_{\text{max}}}{r_{\text{min}}})}$.
The upper bound (74) follows.

$\bullet$ Finally, we prove (75). By (67) and
$\pi^{(1)}=\pi^{(0)}+\frac{d_{T}}{M_{T}}$, we have after some computations

$\left\lVert\nabla b^{(1)}-\nabla
b^{(0)}\right\rVert_{\infty}\leq\frac{c}{M_{T}}\Big{[}\left\lVert
d_{T}\right\rVert_{\infty}+\sum_{y\in\\{r,x_{3},\dots,x_{d}\\}}\left\lVert
d_{T}\right\rVert_{\infty}\left\lVert\frac{\partial d_{T}}{\partial
y}\right\rVert_{\infty}+\\\
\frac{1}{M_{T}}\sum_{y,y^{\prime}\in\\{r,x_{3},\dots,x_{d}\\}}\left\lVert\frac{\partial
d_{T}}{\partial y}\right\rVert_{\infty}\left\lVert\frac{\partial
d_{T}}{\partial
y^{\prime}}\right\rVert_{\infty}+\sum_{y,y^{\prime}\in\\{r,x_{3},\dots,x_{d}\\}}\left\lVert\frac{\partial^{2}d_{T}}{\partial
y\partial y^{\prime}}\right\rVert_{\infty}\Big{]}.$

The upperbound (75) follows from $\left\lVert\frac{\partial^{2}d_{T}}{\partial
x_{i}\partial x_{j}}\right\rVert_{\infty}\leq c/(h_{i}h_{j})$,
$\left\lVert\frac{\partial^{2}d_{T}}{\partial x_{i}\partial
r}\right\rVert_{\infty}\leq
c/(h_{i}r_{\text{min}}\ln(\frac{r_{\text{max}}}{r_{\text{min}}}))$ and
$\left\lVert\frac{\partial^{2}d_{T}}{\partial r^{2}}\right\rVert_{\infty}\leq
c/(r_{\text{min}}^{2}\ln(\frac{r_{\text{max}}}{r_{\text{min}}}))$. ∎

We can now discuss the conditions ensuring that $(\mathbb{I}_{d\times
d},b^{(1)})\in\Sigma(\beta,\mathcal{L},1,a_{0},a_{1},b_{0},b_{1},\tilde{C},\tilde{\rho})$.
Using that $J_{r_{\text{min}},r_{\text{max}}}$ is constant in a neighbourhood
of zero, we deduce that
$(x_{1},\dots,x_{d})\mapsto\pi^{(1)}(\sqrt{x_{1}^{2}+x_{2}^{2}},x_{3},\dots,x_{d})$
is a smooth function and by (68) with (72) we have

$\displaystyle\left|\frac{\partial^{k}}{\partial
x_{1}^{k}}\pi^{(1)}(\sqrt{x_{1}^{2}+x_{2}^{2}},x_{3},\dots,x_{d})\right|+\left|\frac{\partial^{k}}{\partial
x_{2}^{k}}\pi^{(1)}(\sqrt{x_{1}^{2}+x_{2}^{2}},x_{3},\dots,x_{d})\right|\leq\frac{c(k)}{M_{T}\ln(\frac{r_{\text{max}}}{r_{\text{min}}})r_{\text{min}}^{k}},$
$\displaystyle\left|\frac{\partial^{k}}{\partial
x_{l}^{k}}\pi^{(1)}(\sqrt{x_{1}^{2}+x_{2}^{2}},x_{3},\dots,x_{d})\right|\leq\frac{c(k)}{M_{T}h_{l}^{k}},\quad\forall
k\geq 1,\forall l\in\\{3,\dots,d\\}.$

Then, the following condition is sufficient to ensure that
$\pi^{(1)}\in\mathcal{H}(\beta,2\mathcal{L})$:

$\frac{1}{M_{T}}\leq\varepsilon_{0}r_{\text{min}}^{\beta_{l}}\ln(r_{\text{max}}/r_{\text{min}}),\text{for
$l=1,2$
and,}\quad\frac{1}{M_{T}}\leq\varepsilon_{0}h_{l}^{\beta_{l}},\leavevmode\nobreak\
\forall l\in\\{3,\dots,d\\},$ (84)

for some constant $\varepsilon_{0}>0$ small enough. Remark also that we have
$<b^{(1)}(x),x>\leq-\tilde{C}|x|$ for $|x|\geq\tilde{\rho}$ for $T$ large
enough, using that $b^{(1)}$ and $b^{(0)}$ coincide on $K_{T}^{c}$. Moreover,
to get that for $T$ large enough, $\left\lVert
b^{{\color[rgb]{0,0,0}(1)}}\right\rVert_{\infty}\leq b_{0}$ and
$\left\lVert\nabla b^{(1)}\right\rVert\leq b_{1}$ it is sufficient that

$\frac{c}{M_{T}}\left[\frac{1}{r_{\text{min}}^{2}\ln(\frac{r_{\text{max}}}{r_{\text{max}}})}+\frac{1}{h_{3}^{2}}\right]\xrightarrow{T\to\infty}0.$
(85)

Thus, we have $(\mathbb{I}_{d\times
d},b^{(1)})\in\Sigma(\beta,\mathcal{L},1,b_{0},a_{0},a_{1},b_{1},\tilde{C},\tilde{\rho})$
as soon as (84) and (85) are valid.

$\bullet$ Choice of the calibration and proof of (19).

Repeating the proof of (63) we have

$\mathcal{R}_{T}(\beta,\mathcal{L},1,b_{0},a_{0},a_{1},b_{1},\tilde{C},\tilde{\rho})\gtrsim\frac{1}{M_{T}^{2}},$

as soon as $\sup_{T\geq
0}T\int_{\mathbb{R}^{d}}|b^{(1)}(x)-b^{(0)}(x)|^{2}\pi^{\color[rgb]{0,0,0}(0)}(x)\,dx<\infty$.
From (73) this condition is implied by

$\sup_{T\geq
0}T\frac{\prod_{i=3}^{d}h_{i}}{M_{T}^{2}\ln(\frac{r_{\text{max}}}{r_{\text{min}}})}<\infty.$
(86)

Now, we search $1/M_{T}\to 0$ tending to zero as slow as possible subject to
the existence of $0<r_{\text{min}}<r_{\text{max}}/4<r_{\text{max}}\leq
h_{3}\leq\dots\leq h_{d}$ satisfying (84)–(85) and (86). For $l\geq 3$, we set
$h_{l}=(\frac{1}{M_{T}\varepsilon_{0}})^{1/\beta_{l}}$, $r_{\text{max}}=h_{3}$
and $r_{\text{min}}=(\frac{1}{M_{T}\varepsilon_{0}})^{1/\beta_{2}}$. We have
$\log(r_{\text{max}}/r_{\text{min}})\sim_{T\to\infty}(\frac{1}{\beta_{2}}-\frac{1}{\beta_{3}})\ln(M_{T})\to\infty$
using $\beta_{2}<\beta_{3}$. Since $\beta_{1}\leq\beta_{2}$, we deduce that
the conditions
$\frac{1}{M_{T}}\leq\varepsilon_{0}r_{\text{min}}^{\beta_{l}}\ln(r_{\text{max}}/r_{\text{min}})$,
for $l=1,2$ hold true when $T$ is large enough. Hence, (84) is satisfied, and
with these choices (85) becomes a consequence of $\beta_{3}>\beta_{2}>2$.
Replacing $h_{i}$, $r_{\text{min}}$, $r_{\text{max}}$ by their expression in
function of $M_{T}$, the condition (86) writes,

$\sup_{T\geq
0}T\frac{1}{M_{T}^{2+(d-2)/{\overline{\beta}_{3}}}\ln(M_{T})}<\infty,$

and is satisfied for the choice
$M_{T}=\left(T/\ln(T)\right)^{\frac{\overline{\beta}_{3}}{2\overline{\beta}_{3}+(d-2)}}$.
This yields to

$\mathcal{R}_{T}(\beta,\mathcal{L},1,b_{0},a_{0},a_{1},b_{1},\tilde{C},\tilde{\rho})\gtrsim\left(T/\ln(T)\right)^{-\frac{2\overline{\beta}_{3}}{2\overline{\beta}_{3}+(d-2)}}.$

To complete the proof Theorem 19, it remains to show the existence of the
function $J_{r_{\text{min}},r_{\text{max}}}$. This is done in the next lemma.
∎

######  Lemma 6.

There exists a smooth function
$J_{r_{\text{min}},r_{\text{max}}}:\mathbb{R}\to[0,\infty)$ with compact
support on $[0,r_{\max}]$ such that (69)–(72) hold true.

###### Proof.

We let $\varphi:[0,\infty)\to[0,1]$ be a smooth function with
$\varphi{\scriptscriptstyle{|[0,1/4]}}=0$ and
$\varphi{\scriptscriptstyle{|[1/2,\infty)}}=1$. We define piecewise the
function $J_{r_{\text{min}},r_{\text{max}}}$ as follows,

$\displaystyle J_{r_{\text{min}},r_{\text{max}}}(r)=0,\quad\text{ for $r\geq
r_{\text{max}}$,}$ $\displaystyle
J_{r_{\text{min}},r_{\text{max}}}(r)=\frac{1}{\ln(r_{\text{max}}/r_{\text{min}})}\int_{r}^{r_{\text{max}}}\varphi(1-{\color[rgb]{0,0,0}s}/r_{\text{max}}){\color[rgb]{0,0,0}\frac{ds}{s},}\quad\text{
for $r_{\text{max}}/2\leq r\leq r_{\text{max}}$,}$ $\displaystyle
J_{r_{\text{min}},r_{\text{max}}}(r)=J_{r_{\text{min}},r_{\text{max}}}(r_{\text{max}}/2)+\frac{1}{\ln(r_{\text{max}}/r_{\text{min}})}\int_{r}^{r_{\text{max}}/2}{\color[rgb]{0,0,0}\frac{ds}{s},}\quad\text{
for $r_{\text{min}}/2\leq r\leq r_{\text{max}}/2$,}$ $\displaystyle
J_{r_{\text{min}},r_{\text{max}}}(r)=J_{r_{\text{min}},r_{\text{max}}}(r_{\text{min}}/2)+\frac{1}{\ln(r_{\text{max}}/r_{\text{min}})}\int_{r}^{r_{\text{min}}/2}\varphi({\color[rgb]{0,0,0}s}/r_{\text{min}}){\color[rgb]{0,0,0}\frac{ds}{s},}\quad\text{
for $0\leq r\leq r_{\text{min}}/2$.}$

Using the definition of $\varphi$, one can check that the derivative of the
function $J$ is smooth and that $J$ is decreasing with $J^{\prime}(r)=0$ for
$|r|\leq r_{\text{min}}/4$. By simple computation we have,

$\displaystyle
J_{r_{\text{min}},r_{\text{max}}}(r)=\frac{\Phi_{1}(r/r_{\text{max}})}{\ln(r_{\text{max}}/r_{\text{min}})},\quad\text{
for $r_{\text{max}}/2\leq r\leq r_{\text{max}}$,}$ (87) $\displaystyle
J_{r_{\text{min}},r_{\text{max}}}(r)=\frac{\Phi_{1}(1/2)+\ln(\frac{r_{\max}}{2r})}{\ln(r_{\text{max}}/r_{\text{min}})},\quad\text{
for $r_{\text{min}}/2\leq r\leq r_{\text{max}}/2$,}$ (88) $\displaystyle
J_{r_{\text{min}},r_{\text{max}}}(r)=1+\frac{\Phi_{1}(1/2)+\Phi_{2}(r/r_{\text{min}})}{\ln(r_{\text{max}}/r_{\text{min}})},\quad\text{
for $0\leq r\leq r_{\text{min}}/2$,}$ (89)

where $\Phi_{1}(u)=\int_{u}^{1}\varphi(1-s)\frac{ds}{s}$ and
$\Phi_{2}(u)=\int_{u}^{1/2}\varphi(s)\frac{ds}{s}$ are smooth functions on
$[0,1]$.

Thus, using monotonocity of $J_{r_{\text{min}},r_{\text{max}}}$ and
$\varphi\geq 0$, we have $J_{r_{\text{min}},r_{\text{max}}}(0)\geq
J_{r_{\text{min}},r_{\text{max}}}(r_{\text{min}}/2)\geq 1$. We deduce that
(69) is true.

Using (87)–(89) and that $\Phi_{1}$ and $\Phi_{2}$ are bounded on $[0,1]$, we
deduce (70). The condition (71) follows again from (87)–(89) and the
boundedness of $\Phi_{1}^{\prime}$ and $\Phi_{2}^{\prime}$. The condition (72)
is shown by differentiating $k$ times the representations (87)–(89) and using
$r_{\text{min}}\leq r_{\text{max}}$. ∎

######  Remark 2.

The idea underneath the construction of the bump $(r,\theta)\mapsto
J_{r_{\text{min}},r_{\text{max}}}(r)$ in Lemma 6 is the following. Having in
mind (65), we have constructed a smooth bump, with radius $r_{\text{max}}$,
height greater than $1$, and solution of $\Delta
J_{r_{\text{min}},r_{\text{max}}}=0$ on the torus
$r\in[r_{\text{min}}/2,r_{\text{max}}/2]$.

###  7.3 Proof of Theorem 7

###### Proof.

The proof of Theorem 7 heavily relies on the proof of Theorem 19. As done
above, we prove the theorem in the case where $x_{0}=(0,0)$ and $a_{min}=1$.
We will lower bound the risk on the subclass of model (3) given by

$dX_{t}=b(X_{t})dt+dW_{t},$

where $b$ is any drift function such that $(\mathbb{I}_{d\times
d},b)\in{\color[rgb]{0,0,0}\tilde{\Sigma}}(\beta,\mathcal{L},1,b_{0},a_{0},a_{1},b_{1},\tilde{C},\tilde{\rho})$.
Following the proof of Theorem 19 we introduce cylindrical coordinates
$(r,\theta)$ that translates into the Cartesian coordinates
$(r\cos(\theta),r\sin(\theta))$. In two dimensions (66) is written without the
sum over $k$ and the drift $b$ solution of (66) is given by
$b=b_{r}\vec{e}_{r}+b_{\theta}\vec{e}_{\theta}$ with
$\vec{e}_{r}=(\cos(\theta),\sin(\theta))^{T}$,
$\vec{e}_{\theta}=(-\sin(\theta),\cos(\theta),)^{T}$ and

$b_{r}=\frac{1}{\pi}\frac{\partial\pi}{\partial r},\quad b_{\theta}=0.$

$\bullet$ Construction of the priors. Let $\psi:[0,\infty)\to [0,\infty)$ be a
smooth function, vanishing on $[0,1/2]$ and satisfying $\psi(x)=x$ for $x\geq
1$ as in the proof of Theorem 19. We let

$\pi^{(0)}(r)=c_{\eta}e^{-\eta\psi(r)},$

where $\eta>0$ and $c_{\eta}$ is such that
$\int_{[0,\infty)\times[0,2\pi)}\pi^{(0)}(r)r\leavevmode\nobreak\
drd\theta=1$. Again, for $\eta$ small enough we have
$\pi^{(0)}\in\mathcal{H}_{2}(\beta,\mathcal{L})$. Moreover, acting as in the
proof of Theorem 19, it is easy to see that $(\mathbb{I}_{d\times
d},b^{(0)})\in{\color[rgb]{0,0,0}\tilde{\Sigma}}(\beta,\mathcal{L},1,b_{0},a_{0},a_{1},b_{1}/2,\tilde{C},\tilde{\rho})\subset{\color[rgb]{0,0,0}\tilde{\Sigma}}(\beta,\mathcal{L},1,b_{0},a_{0},a_{1},b_{1},\tilde{C},\tilde{\rho})$.

The construction of $\pi^{(1)}$ is more elaborate, as before we add a bump
centered at $0$ to $\pi^{(0)}$. We set

$\pi^{(1)}=\pi^{(0)}+\frac{1}{M_{T}}J_{r_{\text{min}}(T),r_{\text{max}}(T)}(r),$

where $J_{r_{\text{min}}(T),r_{\text{max}}(T)}$ is the function introduced in
the proof of Theorem 19 which satisfies (69), (70), (71) and (72) and whose
existence has been proven in Lemma 6. We recall that
$0<r_{\text{min}}(T)<r_{\text{max}}(T)/4<r_{\text{max}}(T)$ will be calibrated
later and go to zero with a rate polynomial in $1/T$. We denote by $b^{(1)}$
the drift function associated to $\pi^{(1)}$.
Following the proof of Lemma 5 in the bi-dimensional context it is easy to see
that the following bounds hold true.

$\int_{\mathbb{R}^{d}}\left|b^{(1)}(\sqrt{x_{1}^{2}+x_{2}^{2}})-b^{(0)}(\sqrt{x_{1}^{2}+x_{2}^{2}})\right|^{2}dx_{1}dx_{2}\leq
c\frac{1}{M_{T}^{2}\ln(\frac{r_{\text{max}}}{r_{\text{min}}})},$
$\displaystyle\left\lVert b^{(1)}-b^{(0)}\right\rVert_{\infty}$
$\displaystyle\leq\frac{c}{M_{T}}\frac{1}{r_{\text{min}}\ln(\frac{r_{\text{max}}}{r_{\text{max}}})},$
$\displaystyle\left\lVert\nabla b^{(1)}-\nabla b^{(0)}\right\rVert_{\infty}$
$\displaystyle\leq\frac{c}{M_{T}}\frac{1}{r_{\text{min}}^{2}\ln(\frac{r_{\text{max}}}{r_{\text{max}}})}.$

Moreover we still have, for any $k\geq 1$,

$\displaystyle\left|\frac{\partial^{k}}{\partial
x_{1}^{k}}\pi^{(1)}(\sqrt{x_{1}^{2}+x_{2}^{2}})\right|+\left|\frac{\partial^{k}}{\partial
x_{2}^{k}}\pi^{(1)}(\sqrt{x_{1}^{2}+x_{2}^{2}})\right|\leq\frac{c(k)}{M_{T}\ln(\frac{r_{\text{max}}}{r_{\text{min}}})r_{\text{min}}^{k}}.$

Then, the following condition is sufficient to ensure that
$\pi^{(1)}\in\mathcal{H}_{2}(\beta,2\mathcal{L})$:

$\frac{1}{M_{T}}\leq\varepsilon_{0}r_{\text{min}}^{\beta_{l}}\ln(r_{\text{max}}/r_{\text{min}}),\text{for
$l=1,2$ and }$ (90)

for some constant $\varepsilon_{0}>0$ small enough. As before, if (90) holds
true it is sufficient that

$\frac{c}{M_{T}}\frac{1}{r_{\text{min}}^{2}\ln(\frac{r_{\text{max}}}{r_{\text{max}}})}\xrightarrow{T\to\infty}0$
(91)

to obtain $(\mathbb{I}_{d\times
d},b^{(1)})\in\Sigma(\beta,\mathcal{L},1,b_{0},a_{0},a_{1},b_{1},\tilde{C},\tilde{\rho})$.

$\bullet$ Choice of the calibration and proof of (19).

Repeating the proof of (63) we clearly have

$\mathcal{R}_{T}(\beta,\mathcal{L},1,b_{0},a_{0},a_{1},b_{1},\tilde{C},\tilde{\rho})\gtrsim\frac{1}{M_{T}^{2}},$

as soon as $\sup_{T\geq
0}T\int_{\mathbb{R}^{2}}|b^{(1)}(x)-b^{(0)}(x)|^{2}\pi^{\color[rgb]{0,0,0}(0)}(x)\,dx<\infty$,
which is implied by

$\sup_{T\geq
0}T\frac{1}{M_{T}^{2}\ln(\frac{r_{\text{max}}}{r_{\text{min}}})}<\infty.$ (92)

We look for $1/M_{T}$ tending to zero as slow as possible subject to the
existence of $0<r_{\text{min}}<r_{\text{max}}/4<r_{\text{max}}$ satisfying
(90)–(91) and (92). We set
$r_{\text{min}}=(\frac{1}{M_{T}\varepsilon_{0}})^{1/\beta_{2}}$ and
$r_{\text{max}}=(\frac{1}{M_{T}\varepsilon_{0}})^{1/\beta_{2}-\gamma}$ for any
arbitrary $\gamma>0$. It follows
$\log(r_{\text{max}}/r_{\text{min}})\sim_{T\to\infty}(\frac{1}{\beta_{2}}-(\frac{1}{\beta_{2}}-\gamma))\ln(M_{T})\to\infty$
as $\gamma>0$. Since $\beta_{1}\leq\beta_{2}$, we clearly have from the
definition of $r_{\text{min}}$ and $r_{\text{max}}$ that the conditions
$\frac{1}{M_{T}}\leq\varepsilon_{0}r_{\text{min}}^{\beta_{l}}\ln(r_{\text{max}}/r_{\text{min}})$,
for $l=1,2$ hold true when $T$ is large enough. Hence, (90) is satisfied. The
same can be said about (91), recalling we have assumed $\beta_{2}>2$.
Replacing $r_{\text{min}}$ and $r_{\text{max}}$ by their expression in
function of $M_{T}$, condition (92) writes,

$\sup_{T\geq 0}T\frac{1}{M_{T}^{2}\ln(M_{T})}<\infty,$

and is satisfied for the choice $M_{T}=(\frac{T}{\log T})^{\frac{1}{2}}$. It
provides

$\mathcal{R}_{T}(\beta,\mathcal{L},1,b_{0},a_{0},a_{1},b_{1},\tilde{C},\tilde{\rho})\gtrsim\frac{\log
T}{T},$

as we wanted.

∎

## Acknowledgement

The authors are very grateful to the referees for their careful reading, their
suggestions and remarks which improved the paper.

##  Appendix A Appendix: proof of technical results

This section is devoted to the proof of the results which are more technical
and for which some preliminaries are needed.

###  A.1 Proof of Corollary 1

###### Proof.

We start proving the upper bound. Clearly we have $|x-y|^{2}\leq
2|\theta_{t,s}(x)-y|^{2}+2|\theta_{t,s}(x)-x|^{2}$ and thus

$-|\theta_{t,s}(x)-y|^{2}\leq-\frac{1}{2}|x-y|^{2}+|\theta_{t,s}(x)-x|^{2}.$

Then, the proof consists in finding a good bound for $|\theta_{t,s}(x)-x|$,
together with the straightforward application of Lemma 1. We start by noticing
that, when $|\theta_{t,s}(x)|>1$, $|\theta_{t,s}(x)|^{2}$ is a decreasing
function in $t$. Indeed, because of the definition of the flow, its derivative
with respect to time is

${\color[rgb]{0,0,0}2<\theta_{t,s}(x),\dot{\theta}_{t,s}(x)>}=2<\theta_{t,s}(x),b(\theta_{t,s}(x))>\,<0,$

where the last is a consequence of A2. We remark that, if
$|\theta_{t,s}(x)|>1$, it is not possible to have $|x|\leq 1$, as in this case
the flow could increase only until it reaches the value 1. Hence, for
$|\theta_{t,s}(x)|>1$ we have also $|x|>1$ and it is
$|\theta_{t,s}(x)|\leq{\color[rgb]{0,0,0}|\theta_{s,s}(x)|=|x|}$ for any
$t\geq 0$ since $t\mapsto|\theta_{t,s}(x)|$ is decreasing until it possibly
reaches the value $1$. If instead $|\theta_{t,s}(x)|\leq 1$, then we have

$|\theta_{t,s}(x)-x|\leq 1+|x|.$ (93)

The definition of the flow together with the mean value theorem provides

$\displaystyle|\theta_{t,s}(x)-x|$
$\displaystyle=|\int_{s}^{t}b(\theta_{r,s}(x))dr|$
$\displaystyle\leq\int_{s}^{t}|b(0)|dr+\int_{s}^{t}\left\|b^{\prime}\right\|_{\infty}|\theta_{r,s}(x)|dr$
$\displaystyle\leq b_{0}|t-s|+b_{1}|t-s|(|x|+1).$

It concludes the proof of the upper bound on the transition density.
On the other side we have

$-|\theta_{t,s}(x)-y|^{2}\geq-2|x-y|^{2}-2|\theta_{t,s}(x)-x|^{2}.$

Hence, we need to find a lower bound for $-|\theta_{t,s}(x)-x|^{2}$. What we
have already showed, together with the application of Lemma 1, concludes the
proof. ∎

###  A.2 Proof of Lemma 3

Recalling Lemma 2, the main novelty is to prove that the control (8) is
uniform on the class of coefficients $(a,b)\in\Sigma$. To simplify the proof,
we first strengthen the hypothesis on the coefficients of the diffusion by
assuming that they are infinitely differentiable with bounded derivatives or
any order, together with the condition $(a,b)\in\Sigma$.

The proof of (8) relies on the theory of Lyapunov-Poincaré inequalities
introduced in [9]. We recall some definitions related to these functional
inequalities. First, if $f\in\mathcal{C}^{2}(\mathbb{R}^{d},\mathbb{R})$ we
define $\Gamma(f)=A(f^{2})-2fA(f)$ where $A$ is the generator of the
diffusion, and we have $\Gamma(f)=(\nabla f)^{T}\tilde{a}\nabla f=|a^{T}\nabla
f|^{2}$. We let $P^{*,\pi}_{t}$ be the adjoint in
$\mathbf{L}^{2}({\color[rgb]{0,0,0}\mathbb{R}^{d}},\pi)$ of the operator
$P_{t}$, which is given by the expression

$P^{*,\pi}_{t}(f)(x)=\frac{1}{\pi(x)}\int_{\mathbb{R}^{d}}f(y)\pi(y)p_{t}(y,x)dy,$
(94)

and $A^{*,\pi}$ the generator of the semi group $(P^{*,\pi}_{t})_{t\geq 0}$
(details can be found e.g. in [34]). If $f$ is $\mathcal{C}^{2}$ then

$A^{*,\pi}f=\frac{1}{2}\sum_{1\leq i,j\leq
d}\tilde{a}_{i,j}\frac{\partial^{2}f}{\partial x_{i}\partial
x_{j}}+\sum_{i=1}^{d}b^{*}_{i}\frac{\partial f}{\partial x_{i}}$ (95)

with $b^{*}_{i}=-b_{i}+\sum_{j=1}^{d}\frac{\partial\tilde{a}_{j,i}}{\partial
x_{j}}+\frac{1}{\pi}\sum_{i=1}^{d}(\frac{\partial\pi}{\partial
x_{j}})\tilde{a}_{j,i}$ as soon as the expression in the right hand side of
(95) belongs to $\mathbf{L}^{2}(\pi)$.

######  Definition 3.

(Lyapunov-Poincaré Inequality [9]) Let $W$ be a
$\mathcal{C}^{2}(\mathbb{R}^{d},\mathbb{R})$ function such that $W\geq 1$,
$W\in\mathbf{L}^{2}(\pi)$, $AW\in\mathbf{L}^{2}(\pi)$. The probability $\pi$
satisfies a $W$-Lyapunov-Poincaré inequality if there exists $C_{\text{LP}}>0$
such that for all $f\in\mathcal{C}^{2}(\mathbb{R}^{d},\mathbb{R})$, bounded
function in the domain of $A^{*,\pi}$, with $\Gamma(f)W\in\mathbf{L}^{1}(\pi)$
and  $\int_{\mathbb{R}^{d}}f(x)\pi(x)dx=0$ we have

$\int_{\mathbb{R}^{d}}f^{2}(x)W(x)\pi(x)dx\leq
C_{\text{LP}}\int_{\mathbb{R}^{d}}[W(x)\Gamma(f)(x)-f(x)^{2}AW(x)]\pi(x)dx.$
(96)

We can now state a result of [9] which shows that a W-Lyapounov-Poincaré
inequality entails a mixing property of the semi-group. The mixing rate
depends explicitly on the constant $C_{LP}$ and consequently, controlling
$C_{LP}$ is the central point to get the uniform mixing in Lemma 3. For the
sake of completeness, we give a detailed proof adapted to our context of this
result.

######  Lemma 7 (Bakry et al. [9]).

Assume that the $W$-Lyapunov-Poincaré inequality holds true with some constant
$C_{\text{LP}}>0$. Then, for any $\varphi:\mathbb{R}^{d}\to\mathbb{R}$ bounded
function with $\int_{\mathbb{R}^{d}}\varphi(x)\pi(x)dx=0$, we have

$\int_{\mathbb{R}^{d}}[P_{t}(\varphi)(x)]^{2}\pi(x)dx\leq
e^{-t/C_{LP}}\int_{\mathbb{R}^{d}}\varphi^{2}(x)W(x)\pi(x)dx,$ (97)

where $(P_{t})_{t}$ is the semi-group associated to the process $X$.

###### Proof.

$\bullet$ First, we establish some properties on the stationary density $\pi$.
Using Theorem 1.2 in [58], we know that

$\left|\frac{\partial^{j}}{\partial
y^{j}}p_{t}(x,y)\right|\leq\frac{c}{t^{(d+j)/2}}e^{-\frac{\left|\theta_{t,0}(x)-y\right|^{2}}{ct}}$
(98)

for $j\in\\{0,1,2\\}$.  From the invariance of $\pi$,
$\pi(y)=\int_{\mathbb{R}^{d}}\pi(x)p_{t}(x,y)dx$, we deduce that
$\pi\in\mathcal{C}^{2}(\mathbb{R}^{d},\mathbb{R})$, and that $\pi$ is bounded,
with bounded derivatives. We also deduce from Lemma 1 that $\pi(x)>0$ for all
$x\in\mathbb{R}^{d}$.

$\bullet$ We now prove (97). Remark that by a density argument we can assume
that $\varphi$ is $\mathcal{C}^{2}$ and compactly supported. We know that if
$f$ is in the domain of $A^{*,\pi}$ then $t\mapsto P_{t}^{*,\pi}(f)$ is
differentiable in $\mathbf{L}^{2}(\pi)$ and $\frac{\partial}{\partial
t}P_{t}^{*,\pi}(f)=P_{t}^{*,\pi}A^{*,\pi}(f)=A^{*,\pi}P_{t}^{*,\pi}(f)$.

As in [9], we define
$I_{t}=\int_{\mathbb{R}^{d}}(P_{t}^{*,\pi}\varphi(x))^{2}W(x)\pi(x)dx$. By
formal differentiation, we get for $t>0$,

$\frac{\partial I_{t}}{\partial
t}=2\int_{\mathbb{R}^{d}}P_{t}^{*,\pi}\varphi(x)A^{*,\pi}P_{t}^{*,\pi}\varphi(x)W(x)\pi(x)dx.$
(99)

This differentiation can be justified by proving that
$\frac{\partial}{\partial
t}(P_{t}^{*,\pi}(\varphi))^{2}=2P_{t}^{*,\pi}\varphi\times
A^{*,\pi}P_{t}^{*,\pi}\varphi$ is dominated by a function in
$\mathbf{L}^{1}(W\pi)$. Using (94) and that $\pi$ is invariant we get
$|P_{t}^{*,\pi}(\varphi)|\leq\left\lVert\varphi\right\rVert_{\infty}$. Also,
$|A^{*,\pi}P_{t}^{*,\pi}\varphi|=|P_{t}^{*,\pi}A^{*,\pi}\varphi|$ is bounded
by $\left\lVert A^{*,\pi}\varphi\right\rVert_{\infty}<\infty$, using (95) and
the fact that $\varphi$ is compactly supported. Since
$W\in\mathbf{L}^{1}(\pi)$, we deduce (99). Then, using that $A^{*,\pi}$ is a
differential operator by (95), we have
$A^{*,\pi}((P_{t}^{*,\pi}(\varphi))^{2})=2P_{t}^{*,\pi}(\varphi)A^{*,\pi}(P_{t}^{*,\pi}(\varphi))+\Gamma(P_{t}^{*,\pi}(\varphi))$.
Hence,

$\frac{\partial I_{t}}{\partial
t}=\int_{\mathbb{R}^{d}}[A^{*,\pi}((P_{t}^{*,\pi}\varphi)^{2})(x)-\Gamma(P_{t}^{*,\pi}(\varphi))]W(x)\pi(x)dx.$

We now prove that $\Gamma(P_{t}^{*,\pi}\varphi)W\in\mathbf{L}^{1}(\pi)$. Let
$\chi_{M}$ a sequence of $\mathcal{C}^{2}(\mathbb{R},\mathbb{R})$ functions
bounded with bounded derivative and approximating $x\mapsto x^{2}$ in the
sense that $|\chi_{M}(x)|\leq x^{2}$, $|\chi_{M}^{\prime}(x)|\leq 2|x|$,
$\chi_{M}^{\prime\prime}(x)\geq 0$,
$\chi_{M}^{\prime\prime}(x)\xrightarrow{n\to\infty}2$ for all
$x\in\mathbb{R}$. We have,

$A^{*,\pi}(\chi_{M}(P_{t}^{*,\pi}\varphi))=\chi_{M}^{\prime}(P_{t}^{*,\pi}\varphi)A^{*,\pi}(P_{t}^{*,\pi}\varphi)+\frac{1}{2}\chi_{M}{{}^{\prime\prime}}(P_{t}^{*,\pi}\varphi)\Gamma(P_{t}^{*,\pi}\varphi)$

Multiplying by $W$ and integrating with respect to $\pi$, it yields to

$\int_{\mathbb{R}^{d}}\frac{1}{2}\chi_{M}{{}^{\prime\prime}}(P_{t}^{*,\pi}\varphi(x))\Gamma(P_{t}^{*,\pi}\varphi)(x)W(x)\pi(x)dx\leq\left|\int_{\mathbb{R}^{d}}A^{*,\pi}(\chi_{M}(P_{t}^{*,\pi}\varphi))(x)W(x)\pi(x)dx\right|\\\
+\left|\int_{\mathbb{R}^{d}}\chi_{M}^{\prime}(P_{t}^{*,\pi}\varphi(x))A^{*,\pi}(P_{t}^{*,\pi}\varphi)(x)W(x)\pi(x)dx\right|\\\
=\left|\int_{\mathbb{R}^{d}}\chi_{M}(P_{t}^{*,\pi}\varphi)(x)AW(x)\pi(x)dx\right|+\left|\int_{\mathbb{R}^{d}}\chi_{M}^{\prime}(P_{t}^{*,\pi}\varphi(x))A^{*,\pi}(P_{t}^{*,\pi}\varphi)(x)W(x)\pi(x)dx\right|\\\
\leq\left\lVert(P_{t}^{*,\pi}\varphi)^{2}\right\rVert_{\mathbf{L^{2}(\pi)}}\left\lVert
AW\right\rVert_{\mathbf{L^{2}(\pi)}}+\left\lVert
P_{t}^{*,\pi}\varphi\right\rVert_{\mathbf{L^{4}(\pi)}}\left\lVert
A^{*,\pi}(P_{t}^{*,\pi}\varphi)\right\rVert_{\mathbf{L^{4}(\pi)}}\left\lVert
W\right\rVert_{\mathbf{L^{2}(\pi)}}.$

Letting $M\to\infty$ and using Fatou’s lemma, we deduce
$\int_{\mathbb{R}^{d}}\Gamma(P_{t}^{*,\pi}\varphi)(x)W(x)\pi(x)dx<\infty$.
Hence, we can write

$\displaystyle\frac{\partial I_{t}}{\partial t}$
$\displaystyle=\int_{\mathbb{R}^{d}}A^{*,\pi}((P_{t}^{*,\pi}\varphi)^{2})(x)W(x)\pi(x)dx-\int_{\mathbb{R}^{d}}\Gamma(P_{t}^{*,\pi}(\varphi))W(x)\pi(x)dx$
$\displaystyle=\int_{\mathbb{R}^{d}}(P_{t}^{*,\pi}\varphi)^{2}(x){\color[rgb]{0,0,0}AW}(x)\pi(x)dx-\int_{\mathbb{R}^{d}}\Gamma(P_{t}^{*,\pi}(\varphi))W(x)\pi(x)dx$
$\displaystyle=\int_{\mathbb{R}^{d}}[(P_{t}^{*,\pi}\varphi)^{2}(x)AW(x)-\Gamma(P_{t}^{*,\pi}(\varphi))W(x)]\pi(x)dx.$

Now we use the Lyapunov-Poincaré inequality with $f=P_{t}^{*,\pi}(\varphi)$
and get $\frac{\partial I_{t}}{\partial t}\leq-(1/C_{LP})I_{t}$. From
Gronwall’s lemma it yields $I_{t}\leq e^{-t/C_{LP}}I_{0}$ and since $W\geq 1$,

$\int_{\mathbb{R}^{d}}(P_{t}^{*,\pi}(\varphi)(x))^{2}\pi(x)dx\leq I_{t}\leq
e^{-t/C_{LP}}I_{0}=e^{-t/C_{LP}}\int_{\mathbb{R}^{d}}\varphi(x)^{2}W(x)\pi(x)dx.$

By duality between $P_{t}$ and $P_{t}^{*,\pi}$ in $\mathbf{L}^{2}(\pi)$, we
deduce (97). ∎

######  Lemma 8.

Assume that $a$ and $b$ are $\mathcal{C}^{2}$ with bounded derivatives and
$(a,b)\in\Sigma$. Then, for $\varphi$ bounded with
$\int_{\mathbb{R}^{d}}\varphi(x)\pi(x)dx=0$, and all $t>0$,

$\int_{\mathbb{R}^{d}}[P_{t}(\varphi)(x)]^{2}\pi(x)dx\leq
ce^{-t/c}\left\lVert\varphi\right\rVert_{\infty}^{2},$ (100)

where the constant $c>0$ is uniform over the class $\Sigma$.

###### Proof.

The main idea of the proof is that it is possible to show that for all
$(a,b)\in\Sigma$, the stationary probability $\pi$ satisfies a $W$-Lyapunov-
Poincaré inequality with the same function $W$ and same constant $C_{LP}$.
Following [9], the existence of $W$-Lyapunov-Poincaré inequality is related to
the existence of classical Lyapunov functions.

$\bullet$ First, we construct a Lyapunov function $V$ independent of
$(a,b)\in\Sigma$. We let
$\chi\in\mathcal{C}^{\infty}(\mathbb{R}^{d},\mathbb{R})$ such that
$0\leq\chi(x)\leq|x|$ and $\chi(x)=|x|$ for $|x|\geq 1$ and we set
$V(x)=e^{\varepsilon_{0}\chi(x)}$ for some $\varepsilon_{0}>0$ which will be
calibrated later. We have for $|x|\geq 1$, $\nabla
V(x)=\varepsilon_{0}\frac{x}{|x|}V(x)$ and thus,

$\displaystyle AV(x)=\frac{1}{2}\sum_{1\leq i,j\leq
d}\tilde{a}_{i,j}(x)\frac{\partial^{2}V}{\partial x_{i}\partial
x_{j}}(x)+\epsilon_{0}V(x)<\frac{x}{|x|},b(x)>.$

Using that for $|x|\geq 1$, $|\frac{\partial^{2}V}{\partial x_{i}\partial
x_{j}}|=|e^{\varepsilon_{0}|x|}\\{\varepsilon_{0}^{2}\frac{x_{i}x_{j}}{|x|^{2}}-\varepsilon_{0}\frac{x_{i}x_{j}}{|x|^{3}}+\frac{\varepsilon_{0}}{|x|}1_{\\{i=j\\}}\\}|\leq
e^{\varepsilon_{0}|x|}\\{\varepsilon_{0}^{2}+\frac{2\varepsilon_{0}}{|x|}\\}$,
and that $<x,b(x)>\leq-{\color[rgb]{0,0,0}\tilde{C}}|x|$ for
$|x|\geq{\color[rgb]{0,0,0}\tilde{\rho}}$ by Assumption A2, we get,

$AV(x)\leq-{\color[rgb]{0,0,0}\tilde{C}}\varepsilon_{0}V(x)+\frac{\varepsilon_{0}}{2}V(x)\sum_{i,j}|\tilde{a}_{i,j}(x)|\\{\varepsilon_{0}+\frac{2}{|x|}\\},$

for all $|x|\geq{\color[rgb]{0,0,0}\tilde{\rho}}\vee 1$. Using
$|\tilde{a}_{i,j}(x)|=|(aa^{T})_{i,j}(x)|\leq|a(x)|^{2}\leq a_{0}^{2}$ by
Assumption A1, we deduce

$AV(x)\leq-{\color[rgb]{0,0,0}\tilde{C}}\varepsilon_{0}V(x)[1-\frac{d^{2}a_{0}^{2}}{2{\color[rgb]{0,0,0}\tilde{C}}}\\{\varepsilon_{0}+\frac{2}{|x|}\\}].$

Now, we set $\varepsilon_{0}$ as any constant with
$0<\varepsilon_{0}<\frac{{\color[rgb]{0,0,0}\tilde{C}}}{2d^{2}a_{0}^{2}}$, and
deduce for $|x|\geq{\color[rgb]{0,0,0}\tilde{\rho}}\vee
1\vee\frac{4d^{2}a_{0}^{2}}{{\color[rgb]{0,0,0}\tilde{C}}}$:

$AV(x)\leq-\frac{{\color[rgb]{0,0,0}\tilde{C}}\varepsilon_{0}}{2}V(x).$

We then define $\alpha={\color[rgb]{0,0,0}\tilde{C}}\varepsilon_{0}/2$,
$R={\color[rgb]{0,0,0}\tilde{\rho}}\vee
1\vee\frac{4d^{2}a_{0}^{2}}{{\color[rgb]{0,0,0}\tilde{C}}}$ and
$\beta=e^{\varepsilon_{0}R}[\frac{1}{2}d^{2}a_{0}^{2}\varepsilon_{0}^{2}\left\lVert\nabla\chi\right\rVert_{\infty}^{2}+\frac{1}{2}d^{2}a_{0}^{2}\varepsilon_{0}\times\sup_{1\leq
i,j\leq
d}\left\lVert\partial_{x_{i},x_{j}}^{2}\chi\right\rVert_{\infty}+{\color[rgb]{0,0,0}(b_{0}+b_{1}R)}\varepsilon_{0}\left\lVert\nabla\chi\right\rVert_{\infty}+\alpha]$
where the notations $b_{0}$, $b_{1}$ are introduced in Assumption A1. Using
that for $|x|\leq R$, $|b(x)|\leq b_{0}+b_{1}R$, we  can check that
$|AV(x)+\alpha V(x)|\leq\beta$ for $|x|\leq R$, and in turn $V$ is a Lyapunov
function :

$AV(x)\leq-\alpha V(x)+\beta 1_{\\{|x|\leq R\\}},\quad\forall
x\in\mathbb{R}^{d}.$ (101)

$\bullet$ We now prove that (96) holds true with the same function $W$ and
constant $C_{LP}$ for all coefficients $(a,b)\in\Sigma$. Using Proposition 3.6
in [9] with (101), it is sufficient to find $R^{\prime}>0$ large enough such
that

$\displaystyle\\{V\leq 2\beta/\alpha\\}\subset B(0,R^{\prime})$ (102)
$\displaystyle\pi(B(0,R^{\prime}))>1/2$ (103)

and the local Poincaré inequality is valid on $B(0,R^{\prime})$ with some
constant $\kappa_{R^{\prime}}$: for all $f\in\mathcal{C}^{1}$ with bounded
derivative,

$\int_{B(0,R^{\prime})}f^{2}(x)\pi(x)dx\leq\kappa_{R^{\prime}}\int_{\mathbb{R}^{d}}\Gamma(f)\pi(x)dx+\frac{1}{\pi(B(0,R^{\prime}))}\left(\int_{B(0,R^{\prime})}f(x)\pi(x)dx\right)^{2}.$
(104)

If the conditions (102)–(104) are valid, then by Proposition 3.6 in [9] we
deduce the Lyapunov-Poincaré Inequality (96) with
$W=V+(\beta\kappa_{R^{\prime}}-1)_{+}$ and
$1/C_{LP}=2\alpha(1-\frac{\pi(B(0,R^{\prime})^{c})}{\pi(B(0,R^{\prime}))})\times(1+(\beta\kappa_{R^{\prime}}-1)_{+})^{-1}$.

To get (102), using the expression of $V$ it is sufficient to take
$R^{\prime}\geq(\frac{1}{\varepsilon_{0}}\ln(2\beta/\alpha))\vee 1$.
Considering (103), we take the expectation with respect to $\pi$ in (101) and
obtain

$0=\int_{\mathbb{R}^{d}}AV(x)\pi(x)dx\leq-\alpha\int_{\mathbb{R}^{d}}V(x)\pi(x)dx+\beta\int_{\mathbb{R}^{d}}\pi(x)dx,$

and thus

$\int_{\mathbb{R}^{d}}V(x)\pi(x)dx\leq\frac{\beta}{\alpha}.$ (105)

We deduce

$\int_{|x|\geq
1}e^{\varepsilon_{0}|x|}\pi(x)dx\leq\int_{\mathbb{R}^{d}}V(x)\pi(x)dx\leq\frac{\beta}{\alpha}.$
(106)

Using the Markov inequality, this yields, for any $R^{\prime}\geq 1$,
$\pi(\\{x\mid|x|\geq R^{\prime}\\})\leq
e^{-\varepsilon_{0}R^{\prime}}\frac{\beta}{\alpha}$. It entails that
$R^{\prime}>\frac{1}{\varepsilon_{0}}\ln(2\beta/\alpha)$ is sufficient for the
condition (103) to hold true. We set
$R^{\prime}=(\frac{1}{\varepsilon_{0}}\ln(2\beta/\alpha))\vee 1$, and now we
have to check the condition (104). From the Poincaré inequality on the ball
$B(0,R^{\prime})$ endowed with the Lebesgue measure (see e.g. Theorem 4.9 in
[28]) and the Proposition 4.2.7 in [10], we know that if

$1/c_{\pi}\leq\pi(x)\leq c_{\pi},\quad\forall x\in B(0,R^{\prime}),$ (107)

for some $c_{\pi}>0$ then,

$\int_{B(0,R^{\prime})}f^{2}(x)\pi(x)dx\leq
Cc_{\pi}^{3}R^{\prime}\int_{B(0,R^{\prime})}|\nabla
f|^{2}\pi(x)dx+\frac{1}{\pi(B(0,R^{\prime}))}\left(\int_{B(0,R^{\prime})}f(x)\pi(x)dx\right)^{2},$

where $C$ is some universal constant. As the matrix $\tilde{a}$ is lower
bounded by Assumption A1, we have $|\nabla f|^{2}\leq{a_{min}^{-1}\Gamma(f)}$,
and we deduce that (104) holds true with
$\kappa_{R}^{\prime}=CR^{\prime}c_{\pi}^{3}$. Consequently, by Proposition 3.6
in [9], the Lyapunov-Poincaré inequality holds true. Moreover, the constant
$C_{LP}$ in (96) is independent of $(a,b)\in\Sigma$, as soon as we can find a
constant $c_{\pi}$ in (107) independent of $(a,b)$. Using the invariance of
$\pi$ and (4) with $t=1$ and $s=0$, gives for $x\in B(0,R^{\prime})$,

$\displaystyle
C_{0}^{-1}\int_{B(0,R^{\prime})}\pi(y)e^{-\lambda_{0}^{-1}|\theta_{1}(x)-y|^{2}}dy$
$\displaystyle\leq\pi(x)\leq C_{0}\int_{\mathbb{R}^{d}}\pi(y)dy$
$\displaystyle{\color[rgb]{0,0,0}C_{0}^{-1}}\int_{B(0,R^{\prime})}\pi(y)e^{-4\lambda_{0}^{-1}R^{\prime
2}}dy$ $\displaystyle\leq\pi(x)\leq C_{0}\int_{\mathbb{R}^{d}}\pi(y)dy$
$\displaystyle{\color[rgb]{0,0,0}\frac{C_{0}^{-1}}{2}}e^{-4\lambda_{0}^{-1}R^{\prime
2}}$ $\displaystyle\leq\pi(x)\leq C_{0},$ (108)

where in the second line we used $|\theta_{1}(x)|\leq|x|$ for $|x|\geq 1$, and
in the last line we used (103).

Hence, we have proved the W-Lyapunov-Poincaré inequality (96) with the same
function $W$ and constant $C_{LP}$ for all $(a,b)\in\Sigma$. To see that we
are in the scope of the Definition 3 we need to check that $W$ and $AW$ belong
to $\mathbf{L}^{2}(\pi)$. From (106) we know that the stationary measure
integrates the exponential function $x\mapsto e^{\epsilon_{0}|x|}$ for any
constant $\varepsilon_{0}\in(0,\frac{\tilde{C}}{2d^{2}a_{0}^{2}})$. As the
constant $\varepsilon_{0}$ can be chosen arbitrarily small, it suffices to
choose $\varepsilon_{0}<\frac{\tilde{C}}{6d^{2}a_{0}^{2}}$ to deduce that
$W\in\mathbf{L}^{2}(\pi)$. As $b$ has at most linear growth, we also deduce
$AW\in\mathbf{L}^{2}(\pi)$.

$\bullet$ Eventually, we deduce (100) by applying Lemma 7, together with the
upper bound

$\int_{\mathbb{R}^{d}}W(x)\pi(x)dx\leq\int_{\mathbb{R}^{d}}V(x)\pi(x)dx+(\beta\kappa_{R^{\prime}}-1)_{+}\leq\beta/\alpha+(\beta\kappa_{R^{\prime}}-1)_{+}$

where in the last inequality we used (105). ∎

######  Remark 3.

Remark that the theory of W-Lyapounov-Poincaré inequality could be applied to
more general processes than continuous diffusions. Indeed, following the proof
of Lemma 8 we see that the two main ingredients for getting Lyapounov-Poincaré
inequality are the existence of a classical Lyapounov function, and some lower
bound on compact sets for the carré du champ of some operator $\Gamma(f)$ by
$c|\nabla f|^{2}$. Both ingredients are possible to get, for instance, in the
context of jump-diffusion processes. Adaptation of Lemma 7 seems also possible
to the situation of jump-diffusion processes, taking into account that, in
such a context, the carré du champ of the adjoint operator $A^{*,\pi}$ is not
necessary equal to the carré du champ of the operator $A$.

We now prove Lemma 3 as a consequence of Lemma 8.

###### Proof.

First, remark that by a density argument, we can assume in the proof that
$\varphi$ is a smooth function supported on $K$. The inequality (8) is a
consequence of (100) in Lemma 3, but the latter requires that the coefficients
of the S.D.E. are of class $\mathcal{C}^{2}$. Hence, an approximation of the
initial S.D.E. by one with smoother coefficients is required. For $(a,b)$ in
$\Sigma=\Sigma(a_{0},a_{1},b_{0},b_{1},a_{\text{min}},\tilde{C}_{\text{b}},\tilde{\rho}_{\text{b}})$,
we introduce the following smooth approximations of $a$ and $b$. Let $\eta$ be
a smooth function supported on the unit ball of $\mathbb{R}^{d}$ with
$\int_{\mathbb{R^{d}}}\eta(x)dx=1$, and we set $a_{n}=a\star\eta_{n}$,
$b_{n}=b\star\eta_{n}$, where $\star$ is the convolution operator and
$\eta_{n}(\cdot)=n\eta(n\cdot)$. As $(a,b)\in\Sigma$, we deduce that

$\left\lVert a-a_{n}\right\rVert_{\infty}+\left\lVert
b-b_{n}\right\rVert_{\infty}\leq\frac{c}{n}.$ (109)

Moreover, for all $n$ large enough,
$(a_{n},b_{n})\in\Sigma^{\prime}=\Sigma(2a_{0},2a_{1},2b_{0},2b_{1},a_{\text{min}}/2,\tilde{C}_{\text{b}}/2,2\tilde{\rho^{\prime}}_{\text{b}})$.

We denote by $X^{n}$ the solution of the S.D.E. (3) with the coefficients
$(a_{n},b_{n})$ in place of $(a,b)$, and by $\pi_{n}$ the unique stationary
distribution of $X^{n}$. In the sequel of the proof, we also emphasize the
dependence on the initial condition of the process (3) by denoting as
$\mathbb{E}_{x}$ (resp. $\mathbb{E}_{\pi}$) the expectation computed when the
process starts with the initial condition $X_{0}=x$ (resp.
$X_{0}\overset{\text{law}}{=}\pi$). Since $a_{n}$ and $b_{n}$ are smooth
coefficients, by Lemma 8, we have

$\int_{\mathbb{R}^{d}}\left[{\color[rgb]{0,0,0}\mathbb{E}_{x}[\varphi(X_{t}^{n})]}-\pi_{n}(\varphi)\right]^{2}\pi_{n}(x)dx\leq
ce^{-t/c}\left\lVert\varphi\right\rVert_{\infty}^{2}$ (110)

where the constant $c=c_{\Sigma^{\prime}}$ is independent of $n$ and
$(a,b)\in\Sigma$. Using classical estimates for solutions of stochastic
differential equations with (109), it is possible to show

$\sup_{x\in\mathbb{R}^{d}}{\color[rgb]{0,0,0}\mathbb{E}_{x}\left[\left|X^{n}_{t}-X_{t}\right|\right]}\leq\frac{c^{\prime}e^{c^{\prime}t}}{n},\forall
t>0,\forall n\geq 1,$ (111)

and with some constant $c^{\prime}>0$.

Now, we write

$\displaystyle{\color[rgb]{0,0,0}\mathbb{E}_{\pi}\left[\varphi(X_{t})\varphi(X_{0})\right]}$
$\displaystyle=\int_{\mathbb{R}^{d}}P_{t}(\varphi)(x)\varphi(x)\pi(x)dx$
$\displaystyle=\int_{\mathbb{R}^{d}}{\color[rgb]{0,0,0}E_{x}[\varphi(X^{n}_{t})]}\varphi(x)\pi(x)dx+O(\frac{c^{\prime}e^{c^{\prime}t}}{n}\left\lVert\varphi\right\rVert_{\infty}\left\lVert\varphi^{\prime}\right\rVert_{\infty}),$

where we used (111). On the compact $K$ we can find a constant $c_{K}$ such
that $c_{K}^{-1}\leq\pi\leq c_{K}$, $c_{K}^{-1}\leq\pi_{n}\leq c_{K}$ (in the
same way as we obtained (108)), using (110), we deduce

$\int_{K}\left|{\color[rgb]{0,0,0}\mathbb{E}_{x}[\varphi(X_{t}^{n})]}-\pi_{n}(\varphi)\right|^{2}\pi(x)dx\leq
c_{K}^{2}ce^{-t/c}\left\lVert\varphi\right\rVert_{\infty}^{2}.$ (112)

We deduce

$\displaystyle{\color[rgb]{0,0,0}\mathbb{E}_{\pi}\left[\varphi(X_{t})\varphi(X_{0})\right]}$
$\displaystyle=\int_{K}\pi_{n}(\varphi)\varphi(x)\pi(x)dx$
$\displaystyle+O\left(\int_{K}|{\color[rgb]{0,0,0}\mathbb{E}_{x}[\varphi(X^{n}_{t})]}-\pi_{n}(\varphi)|\varphi(x)\pi(x)dx\right)+O(\frac{c^{\prime}e^{c^{\prime}t}}{n}\left\lVert\varphi\right\rVert_{\infty}\left\lVert\varphi^{\prime}\right\rVert_{\infty})$
$\displaystyle=\pi_{n}(\varphi)\pi(\varphi)+O(c_{K}\left\lVert\varphi\right\rVert_{\infty}^{2}\sqrt{c}e^{-t/(2c)})+O(\frac{c^{\prime}e^{c^{\prime}t}}{n}\left\lVert\varphi\right\rVert_{\infty}\left\lVert\varphi^{\prime}\right\rVert_{\infty}){\color[rgb]{0,0,0},}$
(113)

where in the last line we used (112). It remains to find some upper bound on
$\left|\pi_{n}(\varphi)-\pi(\varphi)\right|$. We write for $t>0$

$\pi_{n}(\varphi)-\pi(\varphi)=\pi_{n}(\varphi)-{\color[rgb]{0,0,0}\mathbb{E}_{x}(\varphi(X_{t}^{n}))}+{\color[rgb]{0,0,0}\mathbb{E}_{x}(\varphi(X_{t}^{n}))}-P_{t}(\varphi)(x)+P_{t}(\varphi)(x)-\pi(\varphi)$

and integrate on the compact set $K$ with respect to the Lebesgue measure to
find

$\text{vol}(K)|\pi_{n}(\varphi)-\pi(\varphi)|\leq
c_{K}\left\lVert\pi_{n}(\varphi)-{\color[rgb]{0,0,0}\mathbb{E}_{x}(\varphi(X_{t}^{n}))}\right\rVert_{\mathbf{L}^{1}(\pi_{n})}+\\\
\int_{K}\left|{\color[rgb]{0,0,0}\mathbb{E}_{x}(\varphi(X_{t}^{n}))}-P_{t}(\varphi)(x)\right|dx+c_{K}\left\lVert
P_{t}(\varphi)(x)-\pi(\varphi)\right\rVert_{\mathbf{L}^{1}(\pi)},$

where we used that on $K$, $\pi$ and $\pi_{n}$ are lower bounded by $1/c_{K}$.
From (111), we deduce

$\text{vol}(K)|\pi_{n}(\varphi)-\pi(\varphi)|\leq
c_{K}\left\lVert\pi_{n}(\varphi)-{\color[rgb]{0,0,0}\mathbb{E}_{x}(\varphi(X_{t}^{n}))}\right\rVert_{\mathbf{L}^{1}(\pi_{n})}+\\\
\text{vol}(K)\frac{c^{\prime}e^{c^{\prime}t}}{n}+c_{K}\left\lVert
P_{t}(\varphi)(x)-\pi(\varphi)\right\rVert_{\mathbf{L}^{1}(\pi)},$

In the last equation, we specify $t=\sqrt{\log(n)}$. The first term on the
right hand side goes to zero by (110), the second one goes to zero
immediately, while the last one goes to zero by the mixing property of the
process $X$ (see Lemma 2). We deduce that

$|\pi_{n}(\varphi)-\pi(\varphi)|\leq\varepsilon_{n}(\varphi,K,a,b),$ (114)

for some sequence $\varepsilon_{n}(\varphi,K,a,b)\xrightarrow{n \to\infty}0$
(let us stress that this convergence is not uniform with respect to
$(a,b)\in\Sigma$, $K$, or the function $\varphi$).

Gathering (113) and (114), we have

$\left|{\color[rgb]{0,0,0}\mathbb{E}_{\pi}}\left[\varphi(X_{t})\varphi(X_{0})\right]-
\pi(\varphi)^{2}\right|\leq
c_{K}^{2}\left\lVert\varphi\right\rVert_{\infty}^{2}\sqrt{c}e^{-ct/2}+\frac{c^{\prime}e^{c^{\prime}t}}{n}\left\lVert\varphi\right\rVert_{\infty}\left\lVert\varphi^{\prime}\right\rVert_{\infty}+\left\lVert\varphi\right\rVert_{\infty}\varepsilon_{n}(\varphi,K,a,b)$

where the constant $c$ does not depend on $n$. Letting $n\to\infty$, we deduce
(8).

∎

## References

  * [1] Aeckerle-Willems, C., & Strauch, C. (2021). Concentration of scalar ergodic diffusions and some statistical implications. In Annales de l’Institut Henri Poincaré, Probabilités et Statistiques (Vol. 57, No. 4, pp. 1857-1887). Institut Henri Poincaré.
  * [2] Altmeyer, R. and Reiss, M. (2020). Nonparametric estimation for linear SPDEs from local measurements. Annals of Applied Probabilty to appear.
  * [3] Amorino, C. (2021). Rate of estimation for the stationary distribution of jump-processes over anisotropic Holder classes. Electronic Journal of Statistics, 15(2), 5067-5116.
  * [4] Amorino, C., & Gloter, A. (2021). Invariant density adaptive estimation for ergodic jump–diffusion processes over anisotropic classes. Journal of Statistical Planning and Inference, 213, 106-129.
  * [5] Amorino, C., & Nualart, E. (2021). Optimal convergence rates for the invariant density estimation of jump-diffusion processes. ESAIM: PS 26, 126–151
  * [6] Azencott, R. (1984) Densite des diffusions en temps petit: Developpements asymptotiques. In J. Azema and M. Yor (eds), Seminaire de Probabilites XVIII, Lecture Notes in Math. 1059. Berlin: Springer-Verlag.
  * [7] Bailey, N.T.J. (1957) The Mathematical Theory of Epidemics, Griffin, London.
  * [8] Bacry, E., Delattre, S., Hoffmann, M., Muzy, J.-F. (2013). Some limit theorems for Hawkes processes and application to financial statistics. Stochastic Processes and their Applications 123, 2475-2499.
  * [9] Bakry, D., Cattiaux, P., Guillin, A. (2008). Rate of convergence for ergodic continuous Markov processes: Lyapounov versus Poincaré. Journal of Functional Analysis, 254, 727–759.
  * [10] Bakry, D., Gentil, I., & Ledoux, M. (2014). Analysis and geometry of Markov diffusion operators (Vol. 103). Cham: Springer.
  * [11] Banks, H.T. (1975) Modeling and Control in the Biological Sciences, Lecture Notes in Biomathematics, 6, Springer-Verlag, Berlin.
  * [12] Banon, G. (1978). Nonparametric identification for diffusion processes, SIAM J. Control Optim. 16, 380–395.
  * [13] Bergstrom, A.R. (1990) Continuous Time Econometric Modeling, Oxford University Press, Oxford.
  * [14] Bosq, D. (1998). Nonparametric Statistics for Stochastic Processes. (Second edition), Lecture Notes Statist., 110, New York: Springer-Verlag.
  * [15] Bosq, D. (1998) Minimax rates of density estimators for continuous time processes, Sankhya. Ser. A 60, 18–28.
  * [16] Brown, L. D., & Low, M. G. (1996). A constrained risk inequality with applications to nonparametric functional estimation. The annals of Statistics, 24(6), 2524-2535.
  * [17] Cialenco, I. (2018). Statistical Inference for SPDEs: an Overview. Statistical Inference for Stochastic Processes, 21, 309-329.
  * [18] Cobb, L. (1981) Stochastic differential equations for the social sciences, in Mathematical Frontiers of the Social and Policy Sciences.
  * [19] Comte, F., Lacour, C. (2013). Anisotropic adaptive kernel deconvolution. Annales de l’IHP Probabilités et statistiques, 49(2), 569-609.
  * [20] Comte, F., Genon-Catalot, V. and Rozenholc, Y. (2007). Penalized nonparametric mean square estimation of the coefficients of diffusion processes. Bernoulli, 13, 514-543.
  * [21] Comte, F., Prieur, C., Samson, A. (2017). Adaptive estimation for stochastic damping Hamiltonian systems under partial observation. Stochastic processes and their applications, 127(11), 3689-3718.
  * [22] Dalalyan, A., & Reiß, M. (2007). Asymptotic statistical equivalence for ergodic diffusions: the multidimensional case. Probability theory and related fields, 137(1), 25-47.
  * [23] Delecroix, M. (1980). Sur l’estimation des densités d’un processus stationnaire á temps continu. Publications de l’ISUP, XXV, 1-2, 17-39.
  * [24] Delattre M., Genon-Catalot V. and Larédo, C. (2018). Parametric inference for discrete observations of diffusion processes with mixed effects. Stochastic processes and their Applications, 128, 1929-1957.
  * [25] Delattre, S., Gloter, A., Yoshida, N. (2020). Rate of Estimation for the Stationary Distribution of Stochastic Damping Hamiltonian Systems with Continuous Observations. arXiv preprint arXiv:2001.10423.
  * [26] Dion, C., Lemler, S. (2019). Nonparametric drift estimation for diffusions with jumps driven by a Hawkes process. Statistical Inference for Stochastic Processes, 1-27.
  * [27] Ditlevsen, P.D., Ditlevsen, S., Andersen, K.K. (2002) The fast climate fluctuations during the stadial and interstadial climate states, Ann. Glaciol., 35, 457–462.
  * [28] Evans, Lawrence C. ; Gariepy, Ronald F. Measure theory and fine properties of functions. Revised edition. Textbooks in Mathematics. CRC Press, Boca Raton, FL, 2015. xiv+299 pp. ISBN: 978-1-4822-4238-6
  * [29] Friedman A., (1964). Partial differential equations of parabolic type. Prentice-Hall Inc., Englewood Cliffs, N.J.
  * [30] Genon-Catalot, V. (1990). Maximum contrast estimation for diffusion processes from discrete observations, Statistics, 21, 99-116.
  * [31] Gobet, E. (2001). Local asymptotic mixed normality property for elliptic diffusion: a Malliavin calculus approach. Bernoulli, 7(6), 899-912.
  * [32] Gobet, E., Hoffmann, M. and Reiss, M. (2004). Nonparametric estimation of scalar diffusions based on low frequency data. The Annals of Statistics, 32, 2223-2253.
  * [33] Goldenshluger, A., Lepski, O. (2011). Bandwidth selection in kernel density estimation: oracle inequalities and adaptive minimax optimality. The Annals of Statistics, 39(3), 1608-1632.
  * [34] Hairer, M. (2009). An Introduction to Stochastic PDEs. arXiv: Probability.
  * [35] Has’minskii, R. Z. (1980). Stability of differential equations. Germantown, MD: Sijthoff and Noordhoff.
  * [36] Hoffmann, M. (1999). Adaptive estimation in diffusion processes. Stoch. Proc. and Appl. 79, 135-163.
  * [37] Holden, A.V. (1976) Models for Stochastic Activity of Neurones, Springer-Verlag, New York.
  * [38] Holmes, E.E. (2004) Beyond theory to application and evaluation: diffusion approximations for population viability analysis, Ecol. Appl., 14(4), 1272–1293.
  * [39] Höpfner, R., (2014). Asymptotic Statistics with a view to stochastic processes. Walter de Gruyter, Berlin/Boston.
  * [40] Hull, J. (2000) Options, Futures and Other Derivatives, Prentice-Hall, Englewood Cliffs, NJ.
  * [41] Iacus, S. M., (2010). Simulation and inference for stochastic differential equations. With R examples. Springer.
  * [42] Kessler, M., Lindner, A. and Sørensen, M., Editors (2012). Statistical methods for stochastic differential equations. CRC press. Taylor & Francis Group. Boca Raton.
  * [43] Klein, T., Rio, E. (2005). Concentration around the mean for maxima of empirical processes. The Annals of Probability, 33(3), 1060-1077.
  * [44] Klutchnikoff, N. (2014). Pointwise adaptive estimation of a multivariate function. Mathematical Methods of Statistics, 23(2), 132-150.
  * [45] Kutoyants, Y. A. (1998). Efficient density estimation for ergodic diffusion processes. Statistical Inference for Stochastic Processes, 1(2), 131-155.
  * [46] Kutoyants, Y. A. (2004). On invariant density estimation for ergodic diffusion processes. SORT: statistics and operations research transactions, 28(2), 0111-124.
  * [47] Kutoyants, Y.A., (2004). Statistical inference for ergodic diffusion processes. Springer, London.
  * [48] Kushner, H.J. (1967) Stochastic Stability and Control, Academic Press, New York.
  * [49] Lacour, C., Massart, P., Rivoirard, V. (2017). Estimator selection: a new method with applications to kernel density estimation. Sankhya A, 79(2), 298-335.
  * [50] Lamberton, D., Pages, G., (2002). Recursive computation of the invariant distribution of a diffusion. Bernoulli 8(3), pp.367-405.
  * [51] Lange, K. (2002) Mathematical and Statistical Methods for Genetic Analysis, Springer-Verlag, New York.
  * [52] Larédo, C. (1990). A sufficient condition for asymptotic sufficiency of incomplete observations of a diffusion process. The Annals of Statistics 18, 1158-1171.
  * [53] Lepski, O. V., Levit, B. Y. (1999). Adaptive non-parametric estimation of smooth multivariate functions.
  * [54] Mancini, C., & Renò, R. (2011). Threshold estimation of Markov models with jumps and interest rate modeling. Journal of Econometrics, 160(1), 77-92.
  * [55] Marie, N., & Rosier, A. (2021). Nadaraya-Watson Estimator for IID Paths of Diffusion Processes. arXiv preprint arXiv:2105.06884.
  * [56] Masuda, H. (2007). Ergodicity and exponential beta-mixing for multidimensional diffusions with jumps. Stoch. Proc. and Appl., 117, 35-56.
  * [57] Masuda, H. (2019). Non-Gaussian quasi-likelihood estimation of SDE driven by locally stable Lévy process. Stoch. Proc. Appl. 129, 1013-1059.
  * [58] Menozzi, S., Pesce, A., & Zhang, X. (2021). Density and gradient estimates for non degenerate Brownian SDEs with unbounded measurable drift. Journal of Differential Equations, 272, 330-369.
  * [59] Nguyen, H. T. (1979). Density estimation in a continuous-time Markov processes. Ann. Statist. 7, 341-348.
  * [60] Panloup, F. (2008). Recursive computation of the invariant measure of a stochastic differential equation driven by a Lévy process. The Annals of Applied Probability 18(2), 379-426.
  * [61] Papanicolaou, G. (1995) Diffusions in random media, in Surveys in Applied Mathematics, Keller, J.B., McLaughin, D., Papanicolaou , G., eds, Plenum Press, New York, 205–255.
  * [62] Pardoux, E., & Veretennikov, Y. (2001). On the Poisson equation and diffusion approximation. I. The Annals of Probability, 29(3), 1061-1085.
  * [63] Piccini, U., De Gaetano, A. and Ditlevsen, S. (2010). Stochastic differential mixed-effects models. Scand. J. Statist. 37, 67-90.
  * [64] Ricciardi, L.M. (1977) Diffusion Processes and Related Topics in Biology, Lecture Notes in Biomathematics, Springer, New York.
  * [65] Schmisser, E. (2013). Penalized nonparametric drift estimation for a multidimensional diffusion process. Statistics, 47(1), 61-84.
  * [66] Schmisser, E. (2014). Non-parametric adaptive estimation of the drift for a jump diffusion process. Stoch. Proc. Appl., 124, 883-914.
  * [67] Schuecker, P., B¨ohringer, H., Arzner, K., Reiprich, T.H. (2001) Cosmic mass functions from Gaussian stochastic diffusion processes, Astron. Astrophys., 370, 715–728.
  * [68] Strauch, C. (2018). Adaptive invariant density estimation for ergodic diffusions over anisotropic classes. The Annals of Statistics, 46(6B), 3451-3480.
  * [69] Tsybakov, A. B. (2008). Introduction to nonparametric estimation. Springer Science & Business Media.
  * [70] Tuerlink, F., Maris, E., Ratcliff, R., De Boeck, P. (2001) A comparison of four methods for simulating the diffusion process, Behav. Res. Methods Instrum. Comput., 33(4), 443–456.
  * [71] Veretennikov, A. Y. (1988). Bounds for the mixing rate in the theory of stochastic equations. Theory of Probability & Its Applications, 32(2), 273-281.
  * [72] Yoshida, N. (1992). Estimation for diffusion processes from discrete observation. J. Multivariate Analysis, 41, 220-242.
  * [73] Van Zanten, H. (2001). Rates of convergence and asymptotic normality of kernel estimators for ergodic diffusion processes. Nonparametric Statist. 13 (6), 833-850.
  * [74] Viennet, G. (1997). Inequalities for absolutely regular sequences: application to density estimation. Probability theory and related fields, 107(4), 467-492.