# Evaluating Discourse in Structured Text Representations

Elisa Ferracane Department of Linguistics  Greg Durrett Department of Computer
Science  Junyi Jessy Li Department of Linguistics  Katrin Erk Department of
Linguistics

###### Abstract

Discourse structure is integral to understanding a text and is helpful in many
NLP tasks. Learning _latent_ representations of discourse is an attractive
alternative to acquiring expensive labeled discourse data. Liu and Lapata
(2018) propose a structured attention mechanism for text classification that
derives a tree over a text, akin to an RST discourse tree. We examine this
model in detail, and evaluate on additional discourse-relevant tasks and
datasets, in order to assess whether the structured attention improves
performance on the end task and whether it captures a text’s discourse
structure. We find the learned latent trees have little to no structure and
instead focus on lexical cues; even after obtaining more structured trees with
proposed model modifications, the trees are still far from capturing discourse
structure when compared to discourse dependency trees from an existing
discourse parser. Finally, ablation studies show the structured attention
provides little benefit, sometimes even hurting performance.111Code and data
available at https://github.com/elisaF/structured

##  1 Introduction

Discourse describes how a document is organized, and how discourse units are
rhetorically connected to each other. Taking into account this structure has
shown to help many NLP end tasks, including summarization Hirao et al. (2013);
Durrett et al. (2016), machine translation Joty et al. (2017), and sentiment
analysis Ji and Smith (2017). However, annotating discourse requires
considerable effort by trained experts and may not always yield a structure
appropriate for the end task. As a result, having a model induce the discourse
structure of a text is an attractive option. Our goal in this paper is to
evaluate such an induced structure.

Inducing structure has been a recent popular approach in syntax Yogatama et
al. (2017); Choi et al. (2018); Bisk and Tran (2018). Evaluations of these
latent trees have shown they are inconsistent, shallower than their explicitly
parsed counterparts (Penn Treebank parses) and do not resemble any linguistic
syntax theory Williams et al. (2018).

For discourse, Liu and Lapata (2018) (L&L) induce a document-level structure
while performing text classification with a structured attention that is
constrained to resolve to a non-projective dependency tree. We evaluate the
document-level structure induced by this model. In order to compare the
induced structure to existing linguistically-motivated structures, we choose
Rhetorical Structure Theory (RST) Mann and Thompson (1988), a widely-used
framework for discourse structure, because it also produces tree-shaped
structures.222The Penn Discourse Treebank (PDTB; Prasad et al., 2008) captures
lexically-grounded discourse for individual connectives and adjacent
sentences, and does not span an entire document; Segmented Discourse
Representation Theory Lascarides and Asher (2008) is a graph. We evaluate on
some of the same tasks as L&L, but add two more tasks we theorize to be more
discourse-sensitive: text classification of writing quality, and sentence
order discrimination (as proposed by Barzilay and Lapata (2008)).

Our research uncovers multiple negative results. We find that, contrary to
L&L, the structured attention does not help performance in most cases;
further, the model is not learning discourse. Instead, the model learns trees
with little to no structure heavily influenced by lexical cues to the task. In
an effort to induce better trees, we propose several principled modifications
to the model, some of which yield more structured trees. However, even the
more structured trees bear little resemblance to ground truth RST trees.

We conclude the model holds promise, but requires moving beyond text
classification, and injecting supervision (as in Strubell et al. (2018)).

Our contributions are (1) comprehensive performance results on existing and
additional tasks and datasets showing document-level structured attention is
largely unhelpful, (2) in-depth analyses of induced trees showing they do not
represent discourse, and (3) several principled model changes to produce
better structures but that still do not resemble the structure of discourse.

##  2 Rhetorical Structure Theory (RST)

In RST, coherent texts consist of minimal units, which are linked to each
other, recursively, through rhetorical relations Mann and Thompson (1988).
Thus, the goal of RST is to describe the rhetorical organization of a text by
using a hierarchical tree structure that captures the communicative intent of
the writer. An RST discourse tree can further be represented as a _discourse
dependency tree_. We follow the algorithm of Hirao et al. (2013) to create an
unlabelled dependency tree based on the nuclearity of the tree.

##  3 Models

We present two models: one for text classification, and one for sentence
ordering. Both are based on the L&L model, with a design change to cause
stronger percolation of information up the tree (we also experiment without
this change).

Figure 1: Model of Liu and Lapata (2018) with the document-level portion
(right) that composes sentences into a document representation.

| Yelp | Debates | WQ | WQTC | WSJSO
---|---|---|---|---|---
L&L(orig) |  68.51 $|$ 68.27 (0.19) |  81.82 $|$ 79.48 (2.90) |  84.14 $|$ 82.69 (1.36) |  80.73 $|$ 79.63 (1.03) |  96.17 $|$ 95.29 (0.84)
L&L(ours) |  68.51 $|$ 68.23 (0.23) |  78.88 $|$ 77.81 (1.80) |  84.14 $|$ 82.70 (1.36) |  82.49 $|$ 81.11 (0.95) |  95.57 $|$ 94.76 (1.11)
$-$doc attn |  68.34 $|$ 68.13 (0.17) |  82.89 $|$ 81.42 (1.08) |  83.75 $|$ 82.80 (0.94) |  80.60 $|$ 79.25 (0.94) |  95.57 $|$ 95.11 (0.42)
$-$both attn |  68.19 $|$ 68.05 (0.13) |  79.95 $|$ 77.34 (1.79) |  84.27 $|$ 83.16 (1.25) |  77.58 $|$ 76.16 (1.25) |  95.23 $|$ 94.68 (0.37)
L&L(reported) | 68.6 | 76.5 | - | - | -

Table 1: Max $|$ mean (standard deviation) accuracy on the test set averaged
across four training runs with different initialization weights. Bolded
numbers are within 1 standard deviation of the best performing model.
L&L(orig) uses the original L&L code; L&L(ours) includes the design change and
bug fix. L&L(reported) lists results reported by L&L on a single training run.

Text classification The left-hand side of Figure 1 presents an overview of the
model: the model operates first at the sentence-level to create sentence
representations, and then at the document-level to create a document
representation from the previously created sentence representations. In more
detail, the model composes GloVe embeddings Pennington et al. (2014) into a
sentence representation using structured attention (from which a tree can be
derived), then sentence representations into a single document representation
for class prediction. At both sentence and document level, each object (word
or sentence, respectively) attends to other objects that could be its parent
in the tree. Since the sentence and document-level parts of the model are
identical, we focus on the document level (Figure 1, right), which is of
interest to us for evaluating discourse effects.

Sentence representations $s_{1},\ldots,s_{t}$ are fed to a bidirectional LSTM,
and the hidden representations $[h_{1},\ldots,h_{t}]$ consist of a semantic
part ($e_{t}$) and a structure part ($d_{t}$):
$[\mathbf{e}_{t},\mathbf{d}_{t}]=\mathbf{h}_{t}$. Unnormalized scores
$\mathbf{f}_{ij}$ representing potentials between parent $i$ and child $j$ are
calculated using a bilinear function over the structure vector:

$\mathbf{t}_{p}=\tanh(\mathbf{W}_{p}\mathbf{d}_{i});\hskip
20.0pt\mathbf{t}_{c}=\tanh(\mathbf{W}_{c}\mathbf{d}_{j})\vspace{-0.7em}$ (1)
$\mathbf{f}_{ij}=\mathbf{t}^{T}_{p}\mathbf{W}_{a}\mathbf{t}_{c}\vspace{-0.3em}$
(2)

The matrix-tree theorem allows us to compute marginal probabilities $a_{ij}$
of dependency arcs under the distribution over non-projective dependency trees
induced by $\mathbf{f}_{ij}$ (details in Koo et al. (2007)). This computation
is fully differentiable, allowing it to be treated as another neural network
layer in the model. We importantly note the model only uses the _marginals_.
We can post-hoc use the Chu-Liu-Edmonds algorithm to retrieve the highest-
scoring tree under $\mathbf{f}$, which we call $\mathbf{f_{best}}$ Chu and Liu
(1965); Edmonds (1967).

The semantic vectors of sentences $\mathbf{e}$ are then updated using this
attention. Here we diverge from the L&L model: in their
implementation,333https://github.com/nlpyang/structured each node is updated
based on a weighted sum over its _parents_ in the tree (their paper states
both parents and children).444We found similar results for using both parents
and children as well as using parents only. We instead inform each node by a
sum over its _children_ , more in line with past work where information more
intuitively percolates from children to parents and not the other way Ji and
Smith (2017) (we also run experiments without this design change). We
calculate the context for all possible children of that sentence as:

$c_{i}=\sum^{n}_{k=1}a_{ik}e_{k}\vspace{-0.3em}$ (3)

where $a_{ik}$ is the probability that $k$ is the child of $i$, and $e_{k}$ is
the semantic vector of the child.

The children vectors are then passed through a non-linear function, resulting
in the _updated_ semantic vector $e^{\prime}_{i}$ for parent node $i$.

$e^{\prime}_{i}=\tanh(W_{r}[e_{i},c_{i}])\vspace{-0.7em}$ (4)

Finally, a max pooling layer over $e^{\prime}_{i}$ followed by a linear layer
produces the predicted document class $y$. The model is trained with cross
entropy loss.

Additionally, the released L&L implementation has a bug where attention scores
and marginals are not masked correctly in the matrix-tree computation, which
we correct.

Sentence order discrimination This model is identical, except for task-
specific changes. The goal of this synthetic task, proposed by Barzilay and
Lapata (2008), is to capture discourse coherence. A negative class is created
by generating random permutations of a text’s original sentence ordering (the
positive class). A coherence score is produced for each positive and negative
example, with the intuition that the originally ordered text will be more
coherent than the jumbled version. Because we compare two examples at a time
(original and permuted order), we modify the model to handle paired inputs and
replace cross-entropy loss with a max-margin ranking loss.

##  4 Experiments

We evaluate the model on four text classification tasks and one sentence order
discrimination task.

###  4.1 Datasets

Details and statistics are included in Appendix A.555Of the document-level
datasets used in L&L (SNLI was sentence-level), we omit IMDB and Czech Movies
because on IMDB their model did not outperform prior work, and Czech (a
language with freer word order than English) highlighted the non-projectivity
of their sentence-level trees, which is not the focus of our work.

Yelp (in L&L, 5-way classification) comprises customer reviews from the Yelp
Dataset Challenge (collected by Tang et al. (2015)). Each review is labeled
with a 1 to 5 rating (least to most positive).

Debates (in L&L, binary classification) are transcribed debates on
Congressional bills from the U.S. House of Representatives (compiled by Thomas
et al. (2006), preprocessed by Yogatama and Smith (2014)). Each speech is
labeled with 1 or 0 indicating whether the speaker voted in favor of or
against the bill.

Writing quality (WQ) (not in L&L, binary classification) contains science
articles from the New York Times (extracted from Louis and Nenkova (2013)).
Each article is labeled as either ‘very good’ or ‘typical’ to describe its
writing quality. While both classes contain well-written text, Louis and
Nenkova (2013) find features associated with discourse including sentiment,
readability, along with PDTB-style discourse relations are helpful in
distinguishing between the two classes.

Writing quality with topic control (WQTC) (not in L&L, binary classification)
is similar to WQ, but controlled for topic using a topic similarity list
included with the WQ source corpus.666An analysis in section 4.3 shows the WQ-
trained model focuses on lexical items strongly related to the article topic.

Wall Street Journal Sentence Order (WSJSO) (not in L&L, sentence order
discrimination) is the WSJ portion of PTB Marcus et al. (1993).

###  4.2 Settings

For each experiment, we train the model four times varying only the random
seed for weight initializations. The model is trained for a fixed amount of
time, and the model from the timestep with highest development performance is
chosen. We report accuracies on the test set, and tree analyses on the
development set. Our implementation is built on the L&L released
implementation, with changes as noted in Section 3. Preprocessing and training
details are in Appendix A.

###  4.3 Results

We report accuracy (as in prior work) in Table 1, and perform two ablations:
removing the structured attention at the document level, and removing it at
both document and sentence levels. Additionally, we run experiments on the
original code _without_ the design change or bug fix to confirm our findings
are similar (see L&L(orig) in Table 1).

Document-level structured attention does not help. Structured attention at the
sentence level helps performance for all except WQ, where no form of attention
helps. However, structured attention at the document level yields mostly
negative results, in contrast to the improvements reported in L&L. In Yelp,
WSJSO, and WQ, there is no difference. In Debates, the attention hurts
performance. Only in WQTC does the structured attention provide a benefit.
While a single training run could produce the improvements seen in L&L, the
results across four runs depict a more accurate picture. When inducing
structures, it is particularly important to repeat experiments as the
structures can be highly inconsistent due to the noise caused by random
initialization Williams et al. (2018).

[theme=simple,hide label,label style=font=,thick,edge
style=black!60!black,thick]  1 & 2 & 3 & 4 & 5 & 6 & 7 & 8& 9& 10& 11& 12& 13&
14& 15& 16& 17
[show label,edge unit distance=1.4ex]1ROOT [arc angle=30]21 [arc angle=30]31
[arc angle=30]41 [arc angle=30]51 [arc angle=30]61 [arc angle=30]71 [arc
angle=30]81 [arc angle=30]91 [arc angle=30]101 [arc angle=30]111 [arc
angle=30]121 [arc angle=30]131 [arc angle=30]141 [arc angle=30]151 [arc
angle=30]161 [arc angle=30]171

(1)madam speaker, i rise in opposition to h.r. 3283 on both process and policy
grounds.…(17)look beyond the majority’s smoke and mirrors, and vote against
this ill-timed and ill-conceived legislation.

Figure 2: Learned dependency tree from Debates.

| Yelp | Debates | WQ | WQTC | WSJSO
---|---|---|---|---|---
tree height | 2.049 | 2.751 | 2.909 | 4.035 | 2.288
prop. of leaf nodes | 0.825 | 0.849 | 0.958 | 0.931 | 0.892
norm. arc length | 0.433 | 0.397 | 0.420 | 0.396 | 0.426
% vacuous trees | 73% | 38% | 42% | 14% | 100%

Table 2: Statistics for learned trees averaged across four runs (similar
results without the design change or bug fix are in the Appendix Table 6). See
Table 4 for gold statistics on WQTC.

Trees do not learn discourse. Although document level structured attention
provides little benefit in performance, we probe whether the model could still
be learning some discourse. We visually inspect the learned
$\mathbf{f_{best}}$ trees and in Table 2 we report statistics on them (see
Appendix Table 6 for similar results with the original code).

The visual inspection (Figure 2) reveals shallow trees (also reported in L&L),
but furthermore the trees have little to no structure.777While shallow trees
are expected in PDTB-style discourse, even these trees would exhibit
meaningful structure between adjacent sentences, which is entirely absent
here.  We observe an interesting pattern where the model picks one of the
first two or last two sentences as the root, and all other sentences are
children of that node. We label these trees as ‘vacuous’ and the strength of
this pattern is reflected in the tree statistics (Table 2). The height of
trees is small, showing the trees are shallow. The proportion of leaf nodes is
high, that is, most nodes have no children. Finally, the normalized arc length
is high, where nodes that are halfway into the document still connect to the
root.

We further probe the root sentence, as the model places so much attention on
it. We hypothesize the root sentence has strong lexical cues for the task,
suggesting the model is instead attending to particular words. In Yelp,
reviewers often start or end with a sentiment-laden sentence summarizing their
rating. In Debates, speakers begin or end their speech by stating their stance
on the bill. In WQ and WQTC, the interpretation of the root is less clear. In
WSJSO, we find the root is always the first sentence of the correctly ordered
document, which is reasonable and commonly attested in a discourse tree, but
the remainder of the vacuous tree is entirely implausible.

To confirm our suspicion that the root sentence is lexically marked, we
measure the association between words appearing in the root sentence and those
elsewhere by calculating their positive pointwise mutual information scores
(Table 3).

Yelp |  uuu, sterne, star, rating, deduct, 0, edit, underwhelmed, update, allgemein
---|---
Debates |  oppose, republican, majority, thank, gentleman, leadership, california, measure, president, vote
WQ |  valley, mp3, firm, capital, universal, venture, silicon, analyst, capitalist, street
Table 3: Top 10 words most associated with the root sentence (measured with
PPMI).

In Yelp, we find root words often express sentiment and explicitly mention the
number of stars given (‘sterne’ in German, or ‘uuu’ as coined by a
particularly prolific Yelper), which are clear indicators of the rating label.
For Debates, words express speaker opinion, politeness and stance which are
strong markers for the binary voting label. The list for WQ revolves around
tech, suggesting the model is learning topics instead of writing quality.
Thus, in WQTC we control for topics.

##  5 Learning better structure

We next probe whether the structure in L&L can be improved to be more
linguistically appropriate, while still performing well on the end task. Given
that structured attention helps only on WQTC and learns vacuous trees less
frequently, we focus on this task. We experiment with three modifications.
First, we remove the document-level biLSTM since it performs a level of
composition that might prevent the attention from learning the true structure.
Second, we note equation 3 captures possible children only at one level of the
tree, but not possible subtrees. We thus perform an additional level of
percolation over the marginals to incorporate the children’s children of the
tree. That is, after equation 4, we calculate:

$c^{\prime}_{i}=\sum^{n}_{k=1}a_{ik}e^{\prime}_{i}\hskip 1.0pt;\hskip
20.0pte^{\prime\prime}_{i}=\tanh(W_{r}[e^{\prime}_{i},c^{\prime}_{i}])\vspace{-0.3em}$
(5)

Third, the max-pooling layer gives the model a way of aggregating over
sentences while ignoring the learned structure. Instead, we propose a sum that
is weighted by the probability of a given sentence being the root, i.e., using
the learned root attention score $a_{i}^{r}$:
$y_{i}=\sum_{i=1}^{n}a_{i}^{r}e^{\prime\prime}_{i}$.

| Acc | height | leaf | arc | vacuous
---|---|---|---|---|---
Full | 81.11 | 4.035 | 0.931 | 0.396 | 14%
-biLSTM | 77.80 | 11.51 | 0.769 | 0.353 | 4%
-biLSTM, +w | 75.57 | 7.364 | 0.856 | 0.359 | 3%
-biLSTM, +p | 77.11 | 10.430 | 0.790 | 0.349 | 3%
-biLSTM, +4p | 81.71 | 9.588 | 0.811 | 0.353 | 3%
parsed RST | - | 25.084 | 0.567 | 0.063 | 0%

Table 4: Mean test accuracy and tree statistics on the WQTC dev set (averaged
across four runs). -biLSTM removes the document-level biLSTM, +w uses the
weighted sum, +p performs 1 extra percolation, and +4p does 4 levels of
percolation. The last row are (‘gold’) parsed RST discourse dependency trees.

We include ablations of these modifications and additionally derive RST
discourse dependency trees,888We use the RST parser in Feng and Hirst (2014)
and follow Hirao et al. (2013) to derive discourse dependency trees.
collapsing intrasentential nodes, as an approximation to the ground truth.

The results (Table 4) show that simply removing the biLSTM produces trees with
more structure (deeper trees, fewer leaf nodes, shorter arc lengths, and less
vacuous trees), confirming our intuition that it was doing the work for the
structured attention. However, it also results in lower performance. Changing
the pooling layer from max to weighted sum both hurts performance and results
in shallower trees (though still deeper than Full), which we attribute to this
layer still being a pooling function. Introducing an extra level of tree
percolation yields better trees but also a drop in performance. Finally, using
4 levels of percolation both reaches the accuracy of Full and retains the more
structured trees.999More than 4 levels caused training to become unstable. We
hypothesize accuracy doesn’t surpass Full because this change also introduces
extra parameters for the model to learn.

While our results are a step in the right direction, the structures are
decidedly not discourse when compared to the parsed RST dependency trees,
which are far deeper with far fewer leaf nodes, shorter arcs and no vacuous
trees. Importantly, the tree statistics show the structures do not follow the
typical right-branching structure in news: the trees are shallow, nodes often
connect to the root instead of a more immediate parent, and the vast majority
of nodes have no children. In work concurrent to ours, Liu et al. (2019)
proposes a new iterative algorithm for the structured attention (in the same
spirit as our extra percolations) and applies it to a transformer-based
summarization model. However, even these induced trees are not comparable to
RST discourse trees. The induced trees are multi-rooted by design (each root
is a summary sentence) which is unusual for RST;101010Less than 25% of trees
in the RST Discourse Treebank Carlson et al. (2001) have more than 1 root;
less than 8% have more than 2 roots. their reported tree height and edge
agreement with RST trees are low.

##  6 Conclusion

In this paper, we evaluate structured attention in document representations as
a proxy for discourse structure. We first find structured attention at the
document level is largely unhelpful, and second it instead captures lexical
cues resulting in vacuous trees with little structure. We propose several
principled changes to induce better structures with comparable performance.
Nevertheless, calculating statistics on these trees and comparing them to
parsed RST trees shows they still contain no meaningful discourse structure.
We theorize some amount of supervision, such as using ground-truth discourse
trees, is needed for guiding and constraining the tree induction.

## Acknowledgments

We thank the reviewers for insightful feedback. We acknowledge the Texas
Advanced Computing Center for grid resources. The first author was supported
by the NSF Graduate Research Fellowship Program under Grant No. 2017247409.

## References

  * Barzilay and Lapata (2008) Regina Barzilay and Mirella Lapata. 2008.  Modeling Local Coherence - An Entity-Based Approach.  _Computational Linguistics_ , 34(1):1--34.
  * Bisk and Tran (2018) Yonatan Bisk and Ke Tran. 2018.  Inducing grammars with and for neural machine translation.  In _Proceedings of the 2nd Workshop on Neural Machine Translation and Generation_ , pages 25--35. Association for Computational Linguistics.
  * Carlson et al. (2001) Lynn Carlson, Daniel Marcu, and Mary Ellen Okurovsky. 2001.  Building a discourse-tagged corpus in the framework of rhetorical structure theory.  In _Proceedings of the Second SIGdial Workshop on Discourse and Dialogue_ , pages 1--10. Association for Computational Linguistics.
  * Choi et al. (2018) Jihun Choi, Kang Min Yoo, and Sang-goo Lee. 2018.  Learning to compose task-specific tree structures.  In _Proceedings of the 2018 Association for the Advancement of Artificial Intelligence (AAAI)._
  * Chu and Liu (1965) Y.J. Chu and T.H. Liu. 1965.  On the shortest arborescence of a directed graph.  _Science Sinica_ , 14:1396--1400.
  * Durrett et al. (2016) Greg Durrett, Taylor Berg-Kirkpatrick, and Dan Klein. 2016.  Learning-based single-document summarization with compression and anaphoricity constraints.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1998--2008. Association for Computational Linguistics.
  * Edmonds (1967) Jack Edmonds. 1967.  Optimum branchings.  _Journal of Research of the National Bureau of Standards_ , 71:233--240.
  * Feng and Hirst (2014) Vanessa Wei Feng and Graeme Hirst. 2014.  A linear-time bottom-up discourse parser with constraints and post-editing.  In _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 511--521. Association for Computational Linguistics.
  * Hirao et al. (2013) Tsutomu Hirao, Yasuhisa Yoshida, Masaaki Nishino, Norihito Yasuda, and Masaaki Nagata. 2013.  Single-document summarization as a tree knapsack problem.  In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing_ , pages 1515--1520.
  * Ji and Smith (2017) Yangfeng Ji and Noah A. Smith. 2017.  Neural Discourse Structure for Text Categorization.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 996--1005. Association for Computational Linguistics.
  * Joty et al. (2017) Shafiq Joty, Francisco Guzmán, Lluís Màrquez, and Preslav Nakov. 2017\.  Discourse structure in machine translation evaluation.  _Computational Linguistics_ , 43(4):683--722.
  * Koo et al. (2007) Terry Koo, Amir Globerson, Xavier Carreras, and Michael Collins. 2007.  Structured prediction models via the matrix-tree theorem.  In _Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL)_.
  * Lascarides and Asher (2008) Alex Lascarides and Nicholas Asher. 2008.  Segmented discourse representation theory: Dynamic semantics with discourse structure.  In _Computing meaning_ , pages 87--124. Springer.
  * Liu and Lapata (2018) Yang Liu and Mirella Lapata. 2018.  "Learning Structured Text Representations".  _Transactions of the Association for Computational Linguistics_ , 6:63--75.
  * Liu et al. (2019) Yang Liu, Ivan Titov, and Mirella Lapata. 2019.  Single document summarization as tree induction.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 1745--1755, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Louis and Nenkova (2013) Annie Louis and Ani Nenkova. 2013.  What Makes Writing Great? First Experiments on Article Quality Prediction in the Science Journalism Domain.  _TACL_.
  * Mann and Thompson (1988) William C Mann and Sandra A Thompson. 1988.  Rhetorical structure theory: Toward a functional theory of text organization.  _Text-Interdisciplinary Journal for the Study of Discourse_ , 8(3):243--281.
  * Marcus et al. (1993) Mitchell P Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993.  Building a large annotated corpus of English: The Penn Treebank.  _Computational linguistics_ , 19(2):313--330.
  * Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014.  Glove: Global vectors for word representation.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1532--1543. Association for Computational Linguistics.
  * Prasad et al. (2008) Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind K Joshi, and Bonnie L Webber. 2008.  The Penn Discourse TreeBank 2.0.  In _LREC_.
  * Sandhaus (2008) Evan Sandhaus. 2008.  The New York Times annotated corpus.  _Linguistic Data Consortium, Philadelphia_ , 6(12):e26752.
  * Strubell et al. (2018) Emma Strubell, Patrick Verga, Daniel Andor, David Weiss, and Andrew McCallum. 2018\.  Linguistically-informed self-attention for semantic role labeling.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 5027--5038. Association for Computational Linguistics.
  * Tang et al. (2015) Duyu Tang, Bing Qin, and Ting Liu. 2015.  "Learning Semantic Representations of Users and Products for Document Level Sentiment Classification".  In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 1014--1023. Association for Computational Linguistics.
  * Thomas et al. (2006) Matt Thomas, Bo Pang, and Lillian Lee. 2006.  "Get out the vote: Determining support or opposition from Congressional floor-debate transcripts".  In _Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing_ , pages 327--335. Association for Computational Linguistics.
  * Williams et al. (2018) Adina Williams, Andrew Drozdov, and Samuel R. Bowman. 2018.  Do latent tree learning models identify meaningful structure in sentences? _Transactions of the Association for Computational Linguistics_ , 6:253--267.
  * Yogatama et al. (2017) Dani Yogatama, Phil Blunsom, Chris Dyer, Edward Grefenstette, and Wang Ling. 2017\.  Learning to compose words into sentences with reinforcement learning.  _ICLR_.
  * Yogatama and Smith (2014) Dani Yogatama and Noah A. Smith. 2014.  "Linguistic Structured Sparsity in Text Categorization".  In _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 786--796. Association for Computational Linguistics.

##  Appendix A Appendices

|  | Number of documents |
---|---|---|---
Dataset | Classes | Total | Train | Dev | Test | Vocab.
Yelp | 5 | 333K | 266,522 | 33,333 | 33,317 | 53K
Debates | 2 | 1.5K | 1,050 | 102 | 374 | 21K
WQ | 2 | 7.7K | 6,195 | 775 | 763 | 150K
WQTC | 2 | 7.8K | 6,241 | 777 | 794 | 131K
WSJSO | - | 2.4K | 1,950 (35,165) | 247 (4,392) | 241 (4,383) | 49K
Table 5: Statistics for the datasets used in the text classification and
discrimination tasks (calculated after preprocessing). For WSJSO, the number
of generated pairs are in parentheses.

#### Datasets

Statistics for the datasets are listed in Table 5.

| Yelp | Debates | WQ | WQTC | WSJSO
---|---|---|---|---|---
tree height | 2.049 (2.248) | 2.751 (2.444) | 2.909 (2.300) | 4.035 (2.468) | 2.288 (2.368)
prop. of leaf nodes | 0.825 (0.801) | 0.849 (0.869) | 0.958 (0.971) | 0.931 (0.966) | 0.892 (0.888)
norm. arc length | 0.433 (0.468) | 0.397 (0.377) | 0.420 (0.377) | 0.396 (0.391) | 0.426 (0.374)
% vacuous trees | 73% (68%) | 38% (40%) | 42% (28%) | 14% (21%) | 100% (56%)
Table 6: Statistics for the learned trees averaged across four runs on the L&L(ours) model with comparisons (in parentheses) to results using the original L&L code without the design change or bug fix. | Accuracy | tree height | prop. of leaf | parent entr. | norm. arc length | % vacuous trees
---|---|---|---|---|---|---
Full |  82.49 $|$ 81.11 (0.95) | 4.035 | 0.931 | 0.774 | 0.396 | 14%
-biLSTM |  80.35 $|$ 77.80 (1.72) | 11.51 | 0.769 | 1.876 | 0.353 | 4%
-biLSTM, +p |  78.72 $|$ 77.11 (2.18) | 10.430 | 0.790 | 0.349 | 0.349 | 3%
-biLSTM, +4p |  82.75 $|$ 81.71 (0.70) | 9.588 | 0.811 | 1.60 | 0.353 | 3%
-biLSTM, +w |  78.46 $|$ 75.57 (2.52) | 7.364 | 0.856 | 1.307 | 0.359 | 3%
-biLSTM, +w, +p |  77.08 $|$ 74.78 (2.58) | 8.747 | 0.826 | 1.519 | 0.349 | 4%
parsed RST | - | 25.084 | 0.567 | 2.711 | 0.063 | 0%
Table 7: Max $|$ mean (standard deviation) test accuracy and tree statistics
of the WQTC dev set (averaged across four training runs with different
initialization weights). Bolded numbers are within 1 standard deviation of the
best performing model. +w uses the weighted sum, +p adds 1 extra level of
percolation, +4p adds 4 levels of percolation. The last row are the (‘gold’)
parsed RST discourse dependency trees.

For WQ, the ‘very good’ class was created by Louis and Nenkova (2013) using as
a seed the 63 articles in the New York Times corpus Sandhaus (2008) deemed to
be high-quality writing by a team of expert journalists. The class was then
expanded by adding all other science articles in the NYT corpus that were
written by the seed authors (4,253 articles). For the ‘typical’ class, science
articles by all other authors were included (19,520). Because the data is very
imbalanced, we undersample the ‘typical’ class to be the same size as the
‘very good’. We split this data into 80/10/10 for training, development and
test, with both classes equally represented in each partition.

For WQTC, the original dataset authors provide a list of the 10 most topically
similar articles for each
article.111111http://www.cis.upenn.edu/~nlp/corpora/scinewscorpus.html We make
use of this list to explicitly sample topically similar documents.

#### Preprocessing

For Debates and Yelp, we follow the same preprocessing steps as in L&L, but do
not set a minimum frequency threshold when creating the word embeddings. For
our three datasets, sentences are split and tokenized using Stanford Core NLP.

#### Training

For all models, we use the Adagrad optimizer with a learning rate of 0.05. For
WQ, WQTC, and WSJSO, gradient clipping is performed using the global norm with
a ratio of 1.0. The batch size is 32 for all models except WSJSO uses 16. All
models are trained for a maximum of 8 hours on a GeForce GTX 1080 Ti card.

#### Results

Because our results hinge on multiple runs of experiments each initialized
with different random weights, we include here more detailed versions of our
results to more accurately illustrate their variability. Table 6 supplements
Table 2 with tree statistics from L&L(orig), the model _without_ the design
change or bug fix, to illustrate the derived trees on this model are similar.
Finally, Table 7 is a more detailed version of Table 4, which additionally
includes maximum accuracy, standard deviation for accuracy, as well as the
average parent entropy calculated over the latent trees.


# Diffusion Approximation for Transport Equations with Dissipative Drifts for
Time Dependent Coefficients

First Author Namea∗, Second Author Namebc, etc.ac
aDepartment, University, City, Country
bDepartment, University, City, Country
cDepartment, University, City, Country

∗Corresponding author: first name, initials,  Luca Di Persio 111College of
Mathematics, Department of Computer Science, University of Verona, strada le
Grazie 15 - 37134 Verona - Italy<EMAIL_ADDRESS>Yuri Kondratiev
222Dragomanov University, Kiev<EMAIL_ADDRESS>Viktorya Vardanyan
333Department of Mathematics, University of Trento, Via Sommarive 14-38123
Povo(TN)-Italy<EMAIL_ADDRESS>
###### Abstract

We consider a generalization of classical results of Freidlin and Wentzell to
the case of time dependent dissipative drifts. We show the convergence of
diffusions with multiplicative noise in the zero limit of a diffusivity
parameter to the related dynamical systems. The solution to the associated
transport equation is obtained as an application.

Keywords: diffusion process, dynamical system, perturbation parameter, local
Lipschitz condition, time dependent dissipative drift, zeroth order
approximation, Cauchy problem, transport equation
_AMS Mathematics Subject Classification 2010:_ 47D07, 37P30, 60J65, 60G55

##  1 Introduction

Dynamical systems with small random perturbations are the subject of well know
studies by Freidlin and Wentzell [1]. Their classical results essentially
cover the case of diffusions with regular drifts and small parameters in
multiplicative noise. The main problem within latter scenario, which is the
focus of our study, is the convergence of such diffusions to deterministic
dynamical systems and related evolution equation of transport type.

We study stochastic differential equations(SDEs) with a small perturbation
parameter, in which the coefficients depend on the time. The dependence on
time is the main novelty in our considerations comparing with [1].

Let us also underline that in [1] the authors work under standard
probabilistic assumptions on the drift in the considered diffusions, namely:
global Lipschitz condition and linear bound on the growth. Nevertheless, in
several applications in mathematical physics, these assumptions are too
restrictive, as in the case of, e.g., the stochastic quantisation method.

We consider the dissipative condition on the drift coefficient and the local
Lipschitz condition on the drift and diffusion coefficients formulated for
$0\leq t\leq T$, where $0<T<+\infty$ is a finite time horizon, with constants
depending on $T$. Under latter conditions, we prove both existence and
uniqueness for the perturbed SDE. Moreover, we show the convergence result for
the solution of the perturbed system to the solution of the unperturbed
system, when the perturbation parameter approaches zero. We then consider an
application of the above-mentioned results to the Cauchy problem and the
transport equations.

##  2 The model

Let $(\Omega,\mathcal{F},\\{\mathcal{F}_{t}\\}_{t\in[0,T]},\mathbb{P})$ be a
reference filtered probability space. For the sake of completeness, let us
recall that the latter means that $\Omega$ is a nonempty set, interpreted as
the space of elementary events, while $\mathcal{F}$, is a $\sigma$-algebra of
subsets of $\Omega$, $\mathbb{P}$ is a probability measure on the
$\sigma$-algebra $\mathcal{F}$, and $\\{\mathcal{F}_{t}\\}_{t\in[0,T]}$,
$0<T<+\infty$ being a finite horizon time, is a filtration over
$(\Omega,\mathcal{F},\mathbb{P})$. Finally, we define
$W=\\{W_{t}\\}_{t\in[0,T]}$ as a given $l$-dimensional standard Brownian
motion adapted to the just defined filtration.
We consider an equation

$\frac{d}{dt}x(t)=b(t,x(t)),\ x(0)=x_{0}$ (1)

and the perturbed stochastic differential equation

$dX^{\varepsilon}(t)=b(t,X^{\varepsilon}(t))dt+\varepsilon\sigma(t,X^{\varepsilon}(t))dW_{t},\
X^{\varepsilon}(0)=x_{0}$ (2)

in $\mathbb{R}^{r}$, $\varepsilon$ representing a fixed, small, multiplicative
constant. The functions $b(t,x)$ and $\sigma(t,x)$ are measurable functions on
$[0,T]\times\mathbb{R}^{r}$, taking, respectively, values in $\mathbb{R}^{r}$
and $\mathbb{R}^{r\times l}$, $x_{0}$ is a given $\mathbb{R}^{r}$-valued
random variable independent of $W_{t}$.
The stochastic process $X^{\varepsilon}(t)$ is considered as a result of small
perturbations of (1), with perturbation parameter $\varepsilon$. Solving (2),
means finding a random process $X_{t}=X_{t}(W)$ which satisfies the relation

$X^{\varepsilon}(t)=x_{0}+\int_{0}^{t}b(s,X^{\varepsilon}(s))ds+\varepsilon\int_{0}^{t}\sigma(s,X^{\varepsilon}(s))dW_{s},$

with probability 1 for every $t\in[0,T]$. Let us first assume that the
coefficients of our diffusion are Lipschitz continuous and with linear growth
bound. This implies that (2) admits a uniquely determined solution. Equations
of the form (2) were studied, e.g., by Gihman and Skorohod [4] in Section 2 of
Chapter 2 and in Section 3 of Chapter 3.
We want to prove that such (sequence of) solution(s) of SDE (2) converges to
the solution of (1) uniformly, when $\varepsilon\rightarrow 0^{+}$, also
estimating the convergence rate.

Before proving main results, let us recall the following result that will be
then exploited in our proofs.

######  Lemma 2.1.

[Gronwall] Let $m(t),t\in[0,T]$, be a non-negative function satisfying

$m(t)\leq C+\alpha\int_{0}^{t}m(s)ds,\ t\in[0,T]$ (3)

with $C,\alpha\geq 0$.
Then $m(t)\leq Ce^{\alpha t},$ for $t\in[0,T]$.

##  3 Zeroth Order Approximation

######  Theorem 3.1.

Assume that the coefficients of (2) are linearly bounded and satisfy a uniform
Lipschitz condition, i.e. for every $T>0$ there exist constants $K_{T}$,
$L_{T}$ such that for all $x,y\in\mathbb{R}^{r}$, $t\in[0,T]$:

$|b(t,x)|^{2}+|\sigma(t,x)|^{2}\leq K_{T}^{2}(1+|x|^{2})$ (4)
$|b(t,x)-b(t,y)|^{2}+|\sigma(t,x)-\sigma(t,y)|^{2}\leq L_{T}^{2}|x-y|^{2}$ (5)

Then for all $t>0$ and $\delta>0$ we have:

  1. 1.

$E|X^{\varepsilon}(t)-x(t)|^{2}\leq\varepsilon^{2}a(t)$, and

  2. 2.

$\lim_{\varepsilon\to 0}P\\{\max_{0\leq s\leq
t}|X^{\varepsilon}(s)-x(s)|>\delta\\}=0$

where a(t) is a monotone increasing function expressed in terms of $E|x_{0}|$,
$L_{T}$ and $K_{T}$.

###### Proof.

We start showing that $1+E|X^{\varepsilon}(t)|^{2}$ is bounded uniformly in
$\varepsilon\in[0,1]$. To this end, we use Ito’s formula, the Cauchy-Schwarz
inequality and the condition (4) of linear boundedness for the SDE (2)
coefficients.

$\displaystyle 1+E|X^{\varepsilon}(t)|^{2}$
$\displaystyle=1+E|x_{0}|^{2}+2\int_{0}^{t}E<X^{\varepsilon}(s),b(s,X^{\varepsilon}(s))>ds+\varepsilon^{2}\int_{0}^{t}E|\sigma(s,X^{\varepsilon}(s))|^{2}ds$
$\displaystyle\leq
1+E|x_{0}|^{2}+2\int_{0}^{t}E\sqrt{|X^{\varepsilon}(s)|^{2}b(s,X^{\varepsilon}(s))^{2}}ds+\varepsilon^{2}\int_{0}^{t}E|\sigma(s,X^{\varepsilon}(s))|^{2}ds$
$\displaystyle\leq
1+E|x_{0}|^{2}+2\int_{0}^{t}E\sqrt{|X^{\varepsilon}(s)|^{2}K_{T}^{2}(1+|X^{\varepsilon}(s)|^{2})}ds$
$\displaystyle+\varepsilon^{2}\int_{0}^{t}E[K_{T}^{2}(1+|X^{\varepsilon}(s)|^{2})]ds$
$\displaystyle\leq
1+E|x_{0}|^{2}+(2K_{T}+\varepsilon^{2}K_{T}^{2})\int_{0}^{t}(1+E|X^{\varepsilon}(s)|^{2})ds$

Then, in view of applying the Gronwall Lemma (3), we choose
$m(t)=1+E|X^{\varepsilon}(t)|^{2}$, $C=1+E|x_{0}|^{2}$ and
$\alpha=2K_{T}+\varepsilon^{2}K_{T}^{2}$, then obtaining

$1+E|X^{\varepsilon}(t)|^{2}\leq(1+E|x_{0}|^{2})\exp[(2K_{T}+\varepsilon^{2}K_{T}^{2})t].$
(6)

In what follows, we use the above result to show that
$E|X^{\varepsilon}(t)-x(t)|^{2}\leq\varepsilon^{2}a(t)$. Proceeding as before,
using Lipschitz condition (5) and the boundedness of
$1+E|X^{\varepsilon}(t)|^{2}$ i.e.(6), we have:

$\displaystyle E|X^{\varepsilon}(t)-x(t)|^{2}$ $\displaystyle\leq
2\int_{0}^{t}E<X^{\varepsilon}(s)-x(s),b(s,X^{\varepsilon}(s))-b(s,x(s))>ds$
$\displaystyle+\varepsilon^{2}\int_{0}^{t}E|\sigma(s,X^{\varepsilon}(s))|^{2}ds$
$\displaystyle\leq
2\int_{0}^{t}E\sqrt{|X^{\varepsilon}(s)-x(s)|^{2}|b(s,X^{\varepsilon}(s))-b(s,x(s))|^{2}}ds$
$\displaystyle+\varepsilon^{2}\int_{0}^{t}E|\sigma(s,X^{\varepsilon}(s))|^{2}ds$
$\displaystyle\leq
2\int_{0}^{t}E\sqrt{|X^{\varepsilon}(s)-x(s)|^{2}L_{T}^{2}|X^{\varepsilon}(s)-x(s)|^{2}}ds$
$\displaystyle+\varepsilon^{2}K_{T}^{2}\int_{0}^{t}(1+E|X^{\varepsilon}(s)|^{2})ds$
$\displaystyle\leq
2L_{T}\int_{0}^{t}E|X^{\varepsilon}(s)-x(s)|^{2}ds+\varepsilon^{2}K_{T}^{2}\int_{0}^{t}(1+E|X^{\varepsilon}(s)|^{2})ds$

By Lemma (3), choosing $m(t)=E|X^{\varepsilon}(t)-x(t)|^{2}$, $\alpha=2L_{T}$,
$C=\varepsilon^{2}K_{T}^{2}\int_{0}^{t}(1+E|X^{\varepsilon}(s)|^{2})ds$, we
derive the following inequality

$\displaystyle E|X^{\varepsilon}(t)-x(t)|^{2}$ $\displaystyle\leq
e^{2L_{T}t}\varepsilon^{2}K_{T}^{2}\int_{0}^{t}(1+E|X^{\varepsilon}(s)|^{2})ds$
$\displaystyle\leq
e^{2L_{T}t}\varepsilon^{2}K_{T}^{2}\int_{0}^{t}(1+E|x_{0}|^{2})\exp[(2K_{T}+\varepsilon^{2}K_{T}^{2})s]ds$
$\displaystyle\leq\varepsilon^{2}a(t)\,,$

and we conclude choosing $a(t)$ such that it is a monotone increasing
function.
Concerning the second part of the theorem, we first use the definition of our
diffusion processes (1) and (2) to get

$\displaystyle\max_{0\leq s\leq t}|X^{\varepsilon}(s)-x(s)|$
$\displaystyle=\max_{0\leq s\leq
t}|\int_{0}^{s}b(v,X^{\varepsilon}(v))dv+\varepsilon\int_{0}^{s}\sigma(v,X^{\varepsilon}(v))dw_{v}-\int_{0}^{s}b(v,x(v))dv|$
$\displaystyle\leq\max_{0\leq s\leq
t}|\int_{0}^{s}b(v,X^{\varepsilon}(v))dv-\int_{0}^{s}b(v,x(v)dv|$
$\displaystyle+\varepsilon\max_{0\leq s\leq
t}|\int_{0}^{s}\sigma(v,X^{\varepsilon}(v))dw_{v}|$
$\displaystyle\leq\int_{0}^{t}|b(s,X^{\varepsilon}(s))-b(s,x(s))|ds+\varepsilon\max_{0\leq
s\leq t}|\int_{0}^{s}\sigma(v,X^{\varepsilon}(v))dw_{v}|.$ (7)

Then, by the Chebyshev inequality, setting

$\xi(\omega)=\int_{0}^{t}|b(s,X^{\varepsilon}(s))-b(s,x(s))|ds,\
a=\frac{\delta}{2},\ f(x)=x^{2}\,,$

we estimate the first term on the right hand side of (7), i.e.:

$\displaystyle
P\\{\int_{0}^{t}|b(s,X^{\varepsilon}(s))-b(s,x(s))|ds>\frac{\delta}{2}\\}$
$\displaystyle\leq\frac{E[\int_{0}^{t}|b(s,X^{\varepsilon}(s))-b(s,x(s))|ds]^{2}}{(\frac{\delta}{2})^{2}}$
$\displaystyle\leq
4\delta^{-2}tE\int_{0}^{t}|b(s,X^{\varepsilon}(s))-b(s,x(s))|^{2}ds$
$\displaystyle\leq
4t\delta^{-2}L_{T}^{2}\int_{0}^{t}E|X^{\varepsilon}(s)-x(s)|^{2}ds$
$\displaystyle\leq 4tL_{T}^{2}\delta^{-2}\varepsilon^{2}\int_{0}^{t}a(s)ds$
$\displaystyle=\varepsilon^{2}\delta^{-2}a_{1}(t)$ (8)

where in the second step we used the Cauchy-Schwarz inequality, in the third
we applied the Lipschitz condition for the drift, then concluding exploiting
the first assertion of the theorem. Having obtained an estimate for the first
term on the right hand side, we move forward to estimate the second term on
the right hand side of (7). To this aim we first use the generalized
Kolmogorov inequality for stochastic integrals, namely:

$P\\{\sup_{a\leq t\leq
b}|\int_{a}^{t}f(s,\omega)dw_{s}|>C|\mathcal{N}_{a}\\}\leq\frac{1}{c^{2}}E(\int_{a}^{b}f^{2}(s,\omega)ds|\mathcal{N}_{a})\,,$
(9)

then, setting $f(v,\omega)=\sigma(v,X^{\varepsilon}(v)),\
C=\frac{\delta}{2\varepsilon},\mathcal{N}_{a}=\Omega,\ a=0,b=t$, we can
rewrite (9) as follows

$\displaystyle P\\{\varepsilon\max_{0\leq s\leq
t}|\int_{0}^{s}\sigma(v,X^{\varepsilon}(v))dw_{v}|>\frac{\delta}{2}\\}$
$\displaystyle=P\\{\sup_{0\leq s\leq
t}|\int_{0}^{s}\sigma(v,X^{\varepsilon}(v))dw_{v}|>\frac{\delta}{2\epsilon}\\}$
$\displaystyle\leq\frac{1}{(\frac{\delta}{2\varepsilon})^{2}}E(\int_{0}^{t}|\sigma(s,X^{\varepsilon}(s))|^{2}ds)$
$\displaystyle\leq\frac{4\varepsilon^{2}}{\delta^{2}}\int_{0}^{t}K_{T}^{2}(1+E|X^{\varepsilon}(s)|^{2})ds$
$\displaystyle\leq
4\varepsilon^{2}\delta^{-2}K_{T}^{2}(1+E|x_{0}|^{2})\int_{0}^{t}\exp[(2K_{T}+\varepsilon^{2}K_{T}^{2})s]ds$
$\displaystyle=\varepsilon^{2}\delta^{-2}a_{2}(t)\,$ (10)

so that, combining (7), (8) and (10), we have:

$\displaystyle\lim_{\varepsilon\to 0}P\\{\max_{0\leq s\leq
t}|X^{\varepsilon}(s)-x(s)|>\delta\\}$ $\displaystyle\leq\lim_{\varepsilon\to
0}P\\{\int_{0}^{t}|b(s,X^{\varepsilon}(s))-b(s,x(s))|ds$
$\displaystyle+\varepsilon\max_{0\leq s\leq
t}|\int_{0}^{s}\sigma(v,X^{\varepsilon}(v))dw_{v}|>\delta\\}$
$\displaystyle\leq\lim_{\varepsilon\to
0}P\\{\int_{0}^{t}|b(s,X^{\varepsilon}(s))-b(s,x(s))|ds>\frac{\delta}{2}\\}$
$\displaystyle+\lim_{\varepsilon\to 0}P\\{\varepsilon\max_{0\leq s\leq
t}|\int_{0}^{s}\sigma(v,X^{\varepsilon}(v))dw_{v}|>\frac{\delta}{2}\\}$
$\displaystyle\leq\lim_{\varepsilon\to
0}[\varepsilon^{2}\delta^{-2}a_{1}(t)+\varepsilon^{2}\delta^{-2}a_{2}(t)]=0.$

which concludes the proof. ∎

##  4 Zeroth Order Approximation for Dissipative Case

###  4.1 Existence and Uniqueness of a Solution of the SDE with Dissipative
Drift

We aim at generalizing what obtained within the previous section to consider
weaker conditions on the coefficients of eq. (2). Namely, we aim at finding
that eq. (2) still admits a unique solution when considering dissipativity for
the drift and local Lipschitz conditions for all the other coefficients.
Moreover, after proving both existence and uniqueness, we will also derive the
zeroth order approximation for the process solving (2), under the same
aforementioned assumptions on the drift as well as for the other coefficients.
Let us start proving a result similar to Theorem 2, Chapter 2, Section 6, [2],
where the Authors considered a similar SDE-problem in dimension $r=1$. In what
follows, we will use it to derive our existence result, see Theorem (4.2).

######  Theorem 4.1.

Assume that the coefficients $b_{1}(t,x)$, $b_{2}(t,x)$, $\sigma_{1}(t,x)$,
$\sigma_{2}(t,x)$ of the equations

$d{X}_{i}^{\varepsilon}(t)=b_{i}(t,X_{i}^{\varepsilon}(t))dt+\varepsilon\sigma_{i}(t,X_{i}^{\varepsilon}(t))dW_{t},\
i=1,2$ (11)

satisfy Lipschitz condition and linear growth condition, i.e., for every $T>0$
there exist constants $K_{T},L_{T}$ such that for $t\in[0,T]$,
$x,y\in\mathbb{R}^{r}$, $i=1,2$

$|b_{i}(t,x)-b_{i}(t,y)|+|\sigma_{i}(t,x)-\sigma_{i}(t,y)|\leq L_{T}|x-y|,$
$|b_{i}(t,x)|^{2}+|\sigma_{i}(t,x)|^{2}\leq K_{T}^{2}(1+|x|^{2})$

and that for some $N>0$ with $|x|\leq N$ , $b_{1}(t,x)=b_{2}(t,x)$ and
$\sigma_{1}(t,x)=\sigma_{2}(t,x)$.
If $X_{1}^{\varepsilon}(t)$ and $X_{2}^{\varepsilon}(t)$ are solutions of (11)
with the same initial condition
$X_{1}^{\varepsilon}(0)=X_{2}^{\varepsilon}(0)=x_{0}$, $M[x_{0}^{2}]<\infty$,
and $\tau_{i}$ is the largest t for which $\sup_{0\leq s\leq
t}|X_{i}^{\varepsilon}(s)|\leq N$, then $P\\{\tau_{1}=\tau_{2}\\}=1$ and

$P\\{\sup_{0\leq
s\leq\tau_{1}}|X_{1}^{\varepsilon}(s)-X_{2}^{\varepsilon}(s)|=0\\}=1.$

###### Proof.

Define $\gamma_{1}(t):=1$, if $\sup_{0\leq s\leq
t}|X_{1}^{\varepsilon}(s)|\leq N,$ and $\gamma_{1}(t):=0$ otherwise.
Then we get

$\displaystyle\gamma_{1}(t)[X_{1}^{\varepsilon}(t)-X_{2}^{\varepsilon}(t)]$
$\displaystyle=\gamma_{1}(t)\int_{0}^{t}[b_{1}(s,X_{1}^{\varepsilon}(s))-b_{2}(s,X_{2}^{\varepsilon}(s))]ds$
$\displaystyle+\gamma_{1}(t)\varepsilon\int_{0}^{t}[\sigma_{1}(s,X_{1}^{\varepsilon}(s))-\sigma_{2}(s,X_{2}^{\varepsilon}(s))]dW_{s}$
$\displaystyle=\gamma_{1}(t)\int_{0}^{t}[b_{1}(s,X_{1}^{\varepsilon}(s))-b_{2}(s,X_{1}^{\varepsilon}(s))]ds$
$\displaystyle+\gamma_{1}(t)\varepsilon\int_{0}^{t}[\sigma_{1}(s,X_{1}^{\varepsilon}(s))-\sigma_{2}(s,X_{1}^{\varepsilon}(s))]dW_{s}$
$\displaystyle+\gamma_{1}(t)\int_{0}^{t}[b_{2}(s,X_{1}^{\varepsilon}(s))-b_{2}(s,X_{2}^{\varepsilon}(s))]ds$
$\displaystyle+\gamma_{1}(t)\varepsilon\int_{0}^{t}[\sigma_{2}(s,X_{1}^{\varepsilon}(s))-\sigma_{2}(s,X_{2}^{\varepsilon}(s))]dW_{s}$
$\displaystyle=\gamma_{1}(t)\int_{0}^{t}[b_{2}(s,X_{1}^{\varepsilon}(s))-b_{2}(s,X_{2}^{\varepsilon}(s))]ds$
$\displaystyle+\gamma_{1}(t)\varepsilon\int_{0}^{t}[\sigma_{1}(s,X_{1}^{\varepsilon}(s))-\sigma_{2}(s,X_{1}^{\varepsilon}(s))]dW_{s}.$

Thus

$\displaystyle\gamma_{1}(t)[X_{1}^{\varepsilon}(t)-X_{2}^{\varepsilon}(t)]^{2}$
$\displaystyle\leq
2\gamma_{1}(t)\Big{[}\int_{0}^{t}[b_{2}(s,X_{1}^{\varepsilon}(s))-b_{2}(s,X_{2}^{\varepsilon}(s))]ds\Big{]}^{2}$
$\displaystyle+2\gamma_{1}(t)\varepsilon^{2}\Big{[}\int_{0}^{t}[\sigma_{1}(s,X_{1}^{\varepsilon}(s))-\sigma_{2}(s,X_{1}^{\varepsilon}(s))]dW_{s}\Big{]}^{2}.$

Taking into account that $\gamma_{1}(t)=1$ implies $\gamma_{1}(s)=1$ for
$s\leq t$ we can write the $\gamma_{1}(s)$ inside the brackets. Taking the
expectation and then using the Lipschitz condition and Cauchy-Schwarz-
inequality, we can show that for $\varepsilon\leq 1$ there exists a constant
$l$ for which

$\displaystyle
E\Big{[}\gamma_{1}(t)[X_{1}^{\varepsilon}(t)-X_{2}^{\varepsilon}(t)]^{2}\Big{]}$
$\displaystyle\leq
E\Big{[}4[\int_{0}^{t}\gamma_{1}(s)L_{T}|X_{1}^{\varepsilon}(s)-X_{2}^{\varepsilon}(s)|ds]^{2}\Big{]}$
$\displaystyle\leq
4L_{T}^{2}t\int_{0}^{t}E[\gamma_{1}(s)|X_{1}^{\varepsilon}(s)-X_{2}^{\varepsilon}(s)|^{2}]ds$
$\displaystyle=l\int_{0}^{t}E[\gamma_{1}(s)|X_{1}^{\varepsilon}(s)-X_{2}^{\varepsilon}(s)|^{2}]ds\,,$

then, by Lemma 2.1, for which C=0, we have:

$E\Big{[}\gamma_{1}(t)|X_{1}^{\varepsilon}(t)-X_{2}^{\varepsilon}(t)|^{2}\Big{]}=0.$

Considering the continuity of $X_{1}^{\varepsilon}(t)$ and
$X_{2}^{\varepsilon}(t)$ we can then establish

$P\\{\sup_{0\leq t\leq
T}\gamma_{1}(t)|X_{1}^{\varepsilon}(t)-X_{2}^{\varepsilon}(t)|^{2}=0\\}=1$

On the interval $[0,\tau_{1}]$ the processes $X_{1}^{\varepsilon}(t)$ and
$X_{2}^{\varepsilon}(t)$ coincide with probability 1. Hence
$P\\{\tau_{2}\geq\tau_{1}\\}=1.$ Interchanging the indices 1 and 2 in the
proof of the theorem, we analogously show that
$P\\{\tau_{1}\geq\tau_{2}\\}=1.$ ∎

######  Theorem 4.2.

Let the coefficients of (2) be defined and measurable for $t\in[0,T]$, and
satisfy the conditions:

  1. 1.

for every $T>0$, there exists a constant $K_{T}$ such that for all
$(t,x)\in[0,T]\times\mathbb{R}^{r}$

$<x,b(t,x)>+|\sigma(t,x)|^{2}\leq K_{T}^{2}(1+|x|^{2});$ (12)

  2. 2.

for every N and $T>0$ there exists an $L_{N,T}$ for which

$|b(t,x)-b(t,y)|+|\sigma(t,x)-\sigma(t,y)|\leq L_{N,T}|x-y|$ (13)

with $|x|\leq N$, $|y|\leq N$.

Then (2) has a unique solution in the sense that for two solutions
$X_{1}^{\varepsilon}(t)$ and $X_{2}^{\varepsilon}(t)$ it holds

$P\\{\sup_{0\leq t\leq
T}|X_{1}^{\varepsilon}(t)-X_{2}^{\varepsilon}(t)|=0\\}=1.$

###### Proof.

We will first start showing the existence of a solution, then we prove its
uniqueness. Let us define

$b_{N}(t,x)=\begin{cases}b(t,x)\quad if\quad|x|\leq N\\\ b(t,Nsign(x))\quad
if\quad|x|>N\end{cases}$

$\sigma_{N}(t,x)$ is defined similarly. By $X_{N}^{\varepsilon}(t)$ we denote
the solution of

$dX_{N}^{\varepsilon}(t)=b_{N}(t,X_{N}^{\varepsilon}(t))+\varepsilon\sigma_{N}(t,X_{N}^{\varepsilon}(t))dW_{t},\
X_{N}^{\varepsilon}(0)=x_{0}\,.$ (14)

For the latter equation all conditions for existence and uniqueness are given,
since can count on a growth condition and global Lipschitz condition. Let
$\tau_{N}$ be the largest value of t for which $\sup_{0\leq s\leq
t}|X_{N}^{\varepsilon}(s)|\leq N$. Let $N^{{}^{\prime}}>N$. Since
$b_{N}(t,x)=b_{N^{{}^{\prime}}}(t,x)$ and
$\sigma_{N}(t,x)=\sigma_{N^{{}^{\prime}}}(t,x)$ for all $|x|\leq N$ ,now we
apply Theorem 4.1 and obtain
$X_{N}^{\varepsilon}(t)=X_{N^{{}^{\prime}}}^{\varepsilon}(t)$, with
probability 1 for $t\in[0,\tau_{N}]$.
Hence for $N^{{}^{\prime}}>N$:

$P\\{\sup_{0\leq t\leq
T}|X_{N}^{\varepsilon}(t)-X_{N^{{}^{\prime}}}^{\varepsilon}(t)|>0\\}\leq
P\\{\tau_{N}>T\\}=P\\{\sup_{0\leq t\leq T}|X_{N}^{\varepsilon}(t)|>N\\}.$

If we show that the probability on the right hand side converges to zero for
$N\rightarrow\infty$, then it will follow that $X_{N}^{\varepsilon}$ converges
uniformly with probability 1 to some limit $X^{\varepsilon}(t)$ as
$N\rightarrow\infty$.
Going to the limit in

$X_{N}^{\varepsilon}(t)=x_{0}+\int_{0}^{t}b_{N}(s,X_{N}^{\varepsilon}(s))ds+\varepsilon\int_{0}^{t}\sigma_{N}(s,X_{N}^{\varepsilon}(s))dW_{s}$

we convince ourselves that $X^{\varepsilon}(t)$ is equal with probability 1 to
the continuous solution of (2). Therefore, to finish the proof of the
existence of a solution, we are left with proving that:

$\lim_{N\rightarrow\infty}P\\{\sup_{0\leq t\leq
T}|X_{N}^{\varepsilon}(t)|>N\\}=0\,,$ (15)

and to this aim, we first define the function $\psi(y)=\frac{1}{1+|y|^{2}}$,
to then use the Ito formula. We obtain

$\displaystyle
E[|X_{N}^{\varepsilon}(t)|^{2}\psi(x_{0})]-E[|x_{0}|^{2}\psi(x_{0})]$
$\displaystyle=$ $\displaystyle
E\Big{[}\int_{0}^{t}[2\psi(x_{0})<X_{N}^{\varepsilon}(s),b(t,X_{N}^{\varepsilon}(s))>+\varepsilon^{2}\psi(x_{0})[\sigma_{N}(s,X_{N}^{\varepsilon}(s)]^{2}]ds\Big{]}$
$\displaystyle\leq$ $\displaystyle
E\Big{[}\psi(x_{0})\int_{0}^{t}[2K_{T}^{2}(1+|X_{N}^{\varepsilon}(s)|^{2})+\varepsilon^{2}K_{T}^{2}(1+|X_{N}^{\varepsilon}(s)|^{2})]ds\Big{]}$
$\displaystyle\leq$
$\displaystyle(2K_{T}^{2}t+\varepsilon^{2}K_{T}^{2}t)E\psi(x_{0})+(2K_{T}^{2}+\varepsilon^{2}K_{T}^{2})\int_{0}^{t}E[\psi(x_{0})|X_{N}^{\varepsilon}(s)|^{2}]ds$

By having that we can use Lemma 2.1 to get

$E[\psi(x_{0})|X_{N}^{\varepsilon}(t)|^{2}]\leq[(2K_{T}^{2}t+\varepsilon^{2}K_{T}^{2}t)E\psi(x_{0})+E(|x_{0}|^{2}\psi(x_{0}))]e^{(2K_{T}^{2}+\varepsilon^{2}K_{T}^{2})t}.$

Which means we have

$E[\psi(x_{0})\sup_{0\leq t\leq T}|X_{N}^{\varepsilon}(t)|^{2}]\leq C_{1}$

where $C_{1}$ is independent of N.
We can moreover write

$\displaystyle P\\{\sup_{0\leq t\leq T}|X_{N}^{\varepsilon}(t)|>N\\}$
$\displaystyle=P\\{\psi(x_{0})\sup_{0\leq t\leq
T}|X_{N}^{\varepsilon}(t)|^{2}>N^{2}\psi(x_{0})\\}$ $\displaystyle\leq
P\\{\psi(x_{0})\sup_{0\leq t\leq T}|X_{N}^{\varepsilon}(t)|^{2}>\delta
N^{2}\\}+P\\{\psi(x_{0})\leq\delta\\}$ $\displaystyle\leq\frac{C_{1}}{\delta
N^{2}}+P\\{\psi(x_{0})\leq\delta\\},$

where the last inequality follows from the Chebychev one.
Consequently

$\overline{\lim\limits_{N\rightarrow\infty}}P\\{\sup_{0\leq t\leq
T}|X_{N}^{\varepsilon}(t)|>N\\}\leq P\\{\psi(x_{0})\leq\delta\\}.$

Since $\delta$ is an arbitrary positive number and

$P\\{\psi(x_{0})=0\\}=0,$

(15) results from the preceding relation, which completes the proof of the
existence of a solution to (2).
Next we want to prove the uniqueness of the solution. Let
$X_{1}^{\varepsilon}(t)$ and $X_{2}^{\varepsilon}(t)$ be two solutions of (2).
Denoting by $\phi(t)$ the variable equal to 1 if $\sup_{0\leq s\leq
t}|X_{1}^{\varepsilon}(s)|\leq N$ and $\sup_{0\leq s\leq
t}|X_{2}^{\varepsilon}(s)|\leq N$ and equal to 0 otherwise, then by the second
condition, we have:

$\displaystyle E|X_{1}^{\varepsilon}(t)-X_{2}^{\varepsilon}(t)|^{2}\phi(t)$
$\displaystyle\leq
2E[\phi(t)(\int_{0}^{t}|b(s,X_{1}^{\varepsilon}(s))-b(s,X_{2}^{\varepsilon}(s))|ds)^{2}]$
$\displaystyle+2E[\phi(t)(\int_{0}^{t}|\sigma(s,X_{1}^{\varepsilon}(s))-\sigma(s,X_{2}^{\varepsilon}(s))|dW_{s})^{2}]$
$\displaystyle\leq
2tE[(\int_{0}^{t}\phi(s)|b(s,X_{1}^{\varepsilon}(s))-b(s,X_{2}^{\varepsilon}(s))|^{2}ds)]$
$\displaystyle+2E[(\int_{0}^{t}\phi(s)|\sigma(s,X_{1}^{\varepsilon}(s))-\sigma(s,X_{2}^{\varepsilon}(s))|^{2}ds)]$
$\displaystyle\leq(2T+2)L_{N,T}^{2}\int_{0}^{t}E(\phi(s)|X_{1}^{\varepsilon}(s)-X_{2}^{\varepsilon}(s)|^{2})ds\,,$

where we used that $(a-b)^{2}\leq 2a^{2}+2b^{2}$, then we applied the Cauchy-
Schwarz inequality and the It integral properties, afterwards exploiting the
local Lipschitz continuity. Then, by Lemma 2.1 with $C=0$, we obtain
$E|X_{1}^{\varepsilon}(t)-X_{2}^{\varepsilon}(t)|^{2}\phi(t)=0$, implying:

$P\\{X_{1}^{\varepsilon}(t)\neq X_{2}^{\varepsilon}(t)\\}\leq P\\{\sup_{0\leq
t\leq T}|X_{1}^{\varepsilon}(t)|>N\\}+P\\{\sup_{0\leq t\leq
T}|X_{2}^{\varepsilon}(t)|>N\\}\,,$

indeed, $\phi(t)$ is zero for $\sup_{0\leq s\leq t}|X_{1}^{\varepsilon}(s)|>N$
or $\sup_{0\leq s\leq t}|X_{2}^{\varepsilon}(s)|>N$. Hence the probability on
the right side of this inequality tend to zero as N $\rightarrow\infty$, i.e.,
for all $t\in[0,T]$: $P\\{X_{1}^{\varepsilon}(t)=X_{2}^{\varepsilon}(t)\\}=1$,
implying uniqueness in the sense that $P\\{\sup_{0\leq t\leq
T}|X_{1}^{\varepsilon}(t)-X_{2}^{\varepsilon}(t)|=0\\}=1.$ ∎

###  4.2 Zeroth Order Approximation for Dissipative Case

Considering the existence and uniqueness result proved in the previous section
we aim to prove that the solution of the perturbed SDE (2) converges to the
solution of the unperturbed one (1) as the perturbation parameter
$\varepsilon\to 0$, under dissipativity and dissipativity for differences for
the drift vector and the local Lipschitz condition for all coefficients.

######  Theorem 4.3.

Assume that the coefficients of (2) satisfy a local Lipschitz condition,
$\sigma$ increases no faster than linearly and b satisfies dissipativity and
dissipativity for the differences:

  1. 1.

for every $T>0$ there exists a constant $K_{T}$ such that for all
$x,y\in\mathbb{R}^{r}$ and $t\in[0,T]$

$\displaystyle<x,b(t,x)>+|\sigma(t,x)|^{2}$ $\displaystyle\leq
K_{T}^{2}(1+|x|^{2});$ (16) $\displaystyle<x-y,b(t,x)-b(t,y)>$
$\displaystyle\leq K_{T}^{2}(1+|x-y|^{2});$ (17)

  2. 2.

for every N and $T>0$ there exists an $L_{N,T}$ for which

$|b(t,x)-b(t,y)|+|\sigma(t,x)-\sigma(t,y)|\leq L_{N,T}|x-y|$ (18)

with $|x|\leq N$, $|y|\leq N$.

Then for all $t>0$ and $\delta>0$ we have:

  1. 1.

$E|X^{\varepsilon}(t)-x(t)|^{2}\leq\varepsilon^{2}a(t)$, and

  2. 2.

$\lim_{\varepsilon\to 0}P\\{\max_{0\leq s\leq
t}|X^{\varepsilon}(s)-x(s)|>\delta\\}=0$

where a(t) is a monotone increasing function expressed in terms of $E|x_{0}|$,
$L_{N,T}$ and $K_{T}$.

###### Proof.

We start showing that $E|X^{\varepsilon}(t)|^{2}$ is bounded uniformly in
$\varepsilon\in[0,1]$. Applying Ito’s formula and mathematical expectation we
get

$1+E|X^{\varepsilon}(t)|^{2}=1+E|x_{0}|^{2}+2\int_{0}^{t}E<X^{\varepsilon}(s),b(s,X^{\varepsilon}(s))>ds+\varepsilon^{2}\int_{0}^{t}E|\sigma(s,X^{\varepsilon}(s))|^{2}ds$

Using the fact that $\sigma$ increases no faster than linearly and the
dissipativity for b, the last relation implies the estimate

$\displaystyle 1+E|X^{\varepsilon}(t)|^{2}$
$\displaystyle=1+E|x_{0}|^{2}+2\int_{0}^{t}E<X^{\varepsilon}(s),b(s,X^{\varepsilon}(s))>ds+\varepsilon^{2}\int_{0}^{t}E|\sigma(s,X^{\varepsilon}(s))|^{2}ds$
$\displaystyle\leq
1+E|x_{0}|^{2}+2\int_{0}^{t}E[K_{T}^{2}(1+|X^{\varepsilon}(s)|^{2})]ds+\varepsilon^{2}\int_{0}^{t}E[K_{T}^{2}(1+|X^{\varepsilon}(s)|^{2})]ds$
$\displaystyle\leq
1+E|x_{0}|^{2}+(2K_{T}^{2}+\varepsilon^{2}K_{T}^{2})\int_{0}^{t}(1+E|X^{\varepsilon}(s)|^{2})ds\,.$

Next we use Lemma 2.1 choosing $m(t)=1+M|X^{\varepsilon}(t)|^{2}$,
$C=1+E|x_{0}|^{2}$ and $\alpha=(2K_{T}^{2}+\varepsilon^{2}K_{T}^{2})$, to
obtain:

$1+E|X^{\varepsilon}(t)|^{2}\leq(1+E|x_{0}|^{2})\exp[(2K_{T}^{2}+\varepsilon^{2}K_{T}^{2})t].$
(19)

Inequality (19) proves that $E|X^{\varepsilon}(t)|^{2}$ is bounded uniformly
in $\varepsilon\in[0,1]$ and by previously stated result we show, proceeding
as before, that $E|X^{\varepsilon}(t)-x(t)|^{2}\leq\varepsilon^{2}a(t)$.
Indeed, applying the Ito’s formula to the function
$|X^{\varepsilon}(t)-x(t)|^{2}$, then taking the mathematical expectation on
both sides of the equality, we get:

$\displaystyle E|X^{\varepsilon}(t)-x(t)|^{2}$
$\displaystyle=2\int_{0}^{t}E<X^{\varepsilon}(s)-x(s),b(s,X^{\varepsilon}(s))-b(s,x(s))>ds$
$\displaystyle+\varepsilon^{2}\int_{0}^{t}E|\sigma(s,X^{\varepsilon}(s))|^{2}ds.$

In the proof of the existence we proved (15). Since $X_{N}^{\varepsilon}(t)$
converges to $X^{\varepsilon}(t)$ as $N\to\infty$, moreover we know that:

$\lim\limits_{N\to\infty}P\\{\sup_{0\leq s\leq T}|X^{\varepsilon}(s)|>N\\}=0$
(20)

and

$\lim\limits_{N\to\infty}P\\{\sup_{0\leq s\leq T}|x(s)|>N\\}=0\,.$ (21)

The latter implies that there exists an N, such that

$P\\{\sup_{0\leq s\leq
T}|X^{\varepsilon}(s)|>N\\}\leq\frac{\varepsilon^{2}}{2}$ (22)

and

$P\\{\sup_{0\leq s\leq T}|x(s)|>N\\}\leq\frac{\varepsilon^{2}}{2}.$ (23)

In the following calculations we first split up our mathematical expectation
in two different cases, then we use the Cauchy-Schwarz inequality and
condition (16). Afterwards, we apply the local Lipschitz condition (18) and
the dissipativity for the differences for the drift coefficient (17). Then we
use inequalities (22) and (23) for the probabilities:

$\displaystyle E|X^{\varepsilon}(t)-x(t)|^{2}$

$\displaystyle=\scalebox{0.85}{\mbox{$\displaystyle
2\int_{0}^{t}E<X^{\varepsilon}(s)-x(s),b(s,X^{\varepsilon}(s))-b(s,x(s))>ds$}}$
$\displaystyle+\scalebox{0.85}{\mbox{$\displaystyle\varepsilon^{2}\int_{0}^{t}E|\sigma(s,X^{\varepsilon}(s))|^{2}ds$}}$
$\displaystyle\leq\scalebox{0.85}{\mbox{$\displaystyle P\\{\max\\{\sup_{0\leq
s\leq T}|X^{\varepsilon}(s)|,\sup_{0\leq s\leq T}|x(s)|\\}\leq N\\}\times$}}$
$\displaystyle+\scalebox{0.85}{\mbox{$\displaystyle P\\{\max\\{\sup_{0\leq
s\leq T}|X^{\varepsilon}(s)|,\sup_{0\leq s\leq T}|x(s)|\\}>N\\}\times$}}$
$\displaystyle+\scalebox{0.85}{\mbox{$\displaystyle\varepsilon^{2}K_{T}^{2}\int_{0}^{t}(1+E|X^{\varepsilon}(s)|^{2})ds$}}$
$\displaystyle+\scalebox{0.85}{\mbox{$\displaystyle P\\{\max\\{\sup_{0\leq
s\leq T}|X^{\varepsilon}(s)|,\sup_{0\leq s\leq
T}|x(s)|\\}>N\\}2\int_{0}^{t}K^{2}(1+E|X^{\varepsilon}(s)-x(s)|^{2})ds$}}$
$\displaystyle+\scalebox{0.85}{\mbox{$\displaystyle\varepsilon^{2}K_{T}^{2}\int_{0}^{t}(1+E|X^{\varepsilon}(s)|^{2})ds$}}$
$\displaystyle\leq\scalebox{0.85}{\mbox{$\displaystyle
2L_{N,T}\int_{0}^{t}E|X^{\varepsilon}(s)-x(s)|^{2}ds$}}$
$\displaystyle+\scalebox{0.78}{\mbox{$\displaystyle(P\\{\sup_{0\leq s\leq
T}|X^{\varepsilon}(s)|>N\\}+P\\{\sup_{0\leq s\leq
T}|x(s)|>N\\})[2K_{T}^{2}t+2K_{T}^{2}\int_{0}^{t}E|X^{\varepsilon}(s)-x(s)|^{2}ds]$}}$
$\displaystyle+\scalebox{0.85}{\mbox{$\displaystyle\varepsilon^{2}K_{T}^{2}\int_{0}^{t}(1+E|X^{\varepsilon}(s)|^{2})ds$}}$
$\displaystyle\leq\scalebox{0.85}{\mbox{$\displaystyle
2L_{N,T}\int_{0}^{t}E|X^{\varepsilon}(s)-x(s)|^{2}ds+\varepsilon^{2}2K_{T}^{2}t$}}$
$\displaystyle+\scalebox{0.85}{\mbox{$\displaystyle\varepsilon^{2}2K_{T}^{2}\int_{0}^{t}E|X^{\varepsilon}(s)-x(s)|^{2}ds+\varepsilon^{2}K_{T}^{2}\int_{0}^{t}(1+E|X^{\varepsilon}(s)|^{2})ds$}}$
$\displaystyle\leq\scalebox{0.85}{\mbox{$\displaystyle(2L_{N,T}+\varepsilon^{2}2K_{T}^{2})\int_{0}^{t}E|X^{\varepsilon}(s)-x(s)|^{2}ds$}}$
$\displaystyle+\scalebox{0.85}{\mbox{$\displaystyle\varepsilon^{2}2K_{T}^{2}t+\varepsilon^{2}K_{T}^{2}\int_{0}^{t}(1+E|X^{\varepsilon}(s)|^{2})ds$}}.$

By lemma 2.1, choosing $m(t)=E|X^{\varepsilon}(t)-x(t)|^{2},\
\alpha=2L_{N,T}+2\varepsilon^{2}K_{T}^{2},\
C=\varepsilon^{2}2K_{T}^{2}t+\varepsilon^{2}K_{T}^{2}\int_{0}^{t}(1+E|X^{\varepsilon}(s)|^{2})ds$,
we have:

$\displaystyle E|X^{\varepsilon}(t)-x(t)|^{2}$ $\displaystyle\leq
e^{(2L_{N,T}+2\varepsilon^{2}K_{T}^{2})t}[\varepsilon^{2}2K_{T}^{2}t+\varepsilon^{2}K_{T}^{2}\int_{0}^{t}(1+E|X^{\varepsilon}(s)|^{2})ds]$
$\displaystyle\leq
e^{(2L_{N,T}+2\varepsilon^{2}K_{T}^{2})t}\varepsilon^{2}2K_{T}^{2}t$
$\displaystyle+e^{(2L_{N,T}+2\varepsilon^{2}K_{T}^{2})t}\varepsilon^{2}K_{T}^{2}\int_{0}^{t}(1+E|x_{0}|^{2})\exp[(2K_{T}^{2}+\varepsilon^{2}K_{T}^{2})s]ds$
$\displaystyle\leq\varepsilon^{2}2K_{T}^{2}te^{(2L_{N,T}+2\varepsilon^{2}K_{T}^{2})t}$
$\displaystyle+\varepsilon^{2}K_{T}^{2}e^{(2L_{N,T}+2\varepsilon^{2}K_{T}^{2})t}(1+E|x_{0}|^{2})\int_{0}^{t}\exp[(2K_{T}^{2}+\varepsilon^{2}K_{T}^{2})s]ds$
$\displaystyle\leq\varepsilon^{2}a(t).$

Where we used the result (19) and $a(t)$ has been chosen such that it is a
monotone increasing function.

Now we want to prove the second assertion of the theorem. We will now use the
Chebyshev inequality according which $P\\{\xi(\omega)\geq
a\\}\leq\frac{Ef(\xi)}{f(a)}$.

By setting $\xi(\omega)=\max_{0\leq s\leq t}|X^{\varepsilon}(s)-x(s)|,\
a=\delta,\ f(x)=x^{2}$ and applying the first assertion of the theorem we
obtain

$\displaystyle P\\{\max_{0\leq s\leq t}|X^{\varepsilon}(s)-x(s)|>\delta\\}$
$\displaystyle\leq\frac{E[\max_{0\leq s\leq
t}|X_{s}^{\epsilon}-x_{s}|]^{2}}{\delta^{2}}$
$\displaystyle\leq\frac{\varepsilon^{2}a(t)}{\delta^{2}}$

Taking limits on both sides in (4.2) , we get

$\lim_{\varepsilon\to 0}P\\{\max_{0\leq s\leq
t}|X^{\varepsilon}(s)-x(s)|>\delta\\}\leq\lim_{\varepsilon\to
0}\frac{\epsilon^{2}a(t)}{\delta^{2}}=0$

∎

###  4.3 Parabolic Differential equations with a Small Parameter: Cauchy
Problem, Transport Equation

We aim at deriving results concerning the behavior of solutions of Cauchy
problem as $\varepsilon\to 0$, from the behavior of $X^{\varepsilon}(t)$ as
$\varepsilon\to 0$. Within preceding section we have obtained a result
concerning the behavior of solutions $X^{\varepsilon}(t)$ as $\varepsilon\to
0$, which we will exploit in what follows. In particular, we consider the
Cauchy problem:

$\frac{\partial v^{\varepsilon}(t,x)}{\partial
t}=L^{\varepsilon}v^{\varepsilon}(t,x)+c(x)v^{\varepsilon}(t,x)+g(x),\
v^{\varepsilon}(0,x)=f(x),$ (25)

$t>0,\ x\in\mathbb{R}^{r}$ for $\varepsilon>0$, together with the problem for
the first-order operator obtained considering $\varepsilon=0$:

$\frac{\partial v^{0}(t,x)}{\partial t}=L^{0}v^{0}(t,x)+c(x)v^{0}(t,x)+g(x),\
v^{0}(0,x)=f(x)\,,$ (26)

where $L^{\varepsilon}$ is a differential operator with a small parameter at
the derivatives of highest order:

$L^{\varepsilon}=\frac{\varepsilon^{2}}{2}\sum_{i,j=1}^{r}a^{ij}(t,x)\frac{\partial^{2}}{\partial
x^{i}x^{j}}+\sum_{i=1}^{r}b^{i}(t,x)\frac{\partial}{\partial x^{i}}\,.$

Every operator $L^{\varepsilon}$ (whose coefficients are assumed to be
sufficiently regular) has an associated diffusion process
$X^{\varepsilon,x}(t)$. This diffusion process can be represented by mean of a
stochastic equation of the following type:

$dX^{\varepsilon}(t)=b(t,X^{\varepsilon}(t))dt+\varepsilon\sigma(t,X^{\varepsilon}(t))dW_{t},\
X^{\varepsilon}(0)=x$ (27)

where $\sigma(t,x)\sigma^{*}(t,x)=(a^{ij}(t,x)),\
b(t,x)=(b^{1}(t,x),...,b^{r}(t,x)).$
Solutions of partial differential equations(PDEs) can be represented in terms
of the solutions of stochastic differential equations, which is no one else
but the well known Feyman-Kac formula.
Hence, assuming the following conditions are satisfied:

  1. 1.

the function $c(x)$ is uniformly continuous and bounded for
$x\in\mathbb{R}^{r}$;

  2. 2.

the coefficients of $L^{\varepsilon}$ satisfy a local Lipschitz condition, b
satisfies dissipativity and dissipativity for the differences;

  3. 3.

$k^{-2}\sum\lambda_{i}^{2}\leq\sum_{i,j=1}^{r}a^{ij}(t,x)\lambda_{i}\lambda_{j}\leq
k^{2}\sum\lambda_{i}^{2}$ for any real
$\lambda_{1},\lambda_{2},...,\lambda_{r}$
and $x\in\mathbb{R}^{r}$, $t\in[0,T]$ where $k^{2}$ is a positive constant.

the solutions to problems (25) and (26), both exist and are unique, moreover
the solution $v^{\varepsilon}(t,x)$ of (25) is given by

$\displaystyle v^{\varepsilon}(t,x)$
$\displaystyle=E^{x}[f(X^{\varepsilon,x}(t))\exp\\{\int_{0}^{t}c(X^{\varepsilon,x}(s))ds\\}]$
$\displaystyle+E^{x}[\int_{0}^{t}g(X^{\varepsilon,x}(s))\exp\\{\int_{0}^{s}c(X^{\varepsilon,x}(r))dr\\}ds]\,.$
(28)

To prove formula (4.3), let us consider the stochastic process

$Y_{s}=exp\big{\\{}\int_{0}^{s}c(X^{\varepsilon,x}(r))dr\big{\\}}v^{\varepsilon}(t-s,X^{\varepsilon,x}(s))+\int_{0}^{s}g(X^{\varepsilon,x}(r))exp\big{\\{}\int_{0}^{r}c(X^{\varepsilon,x}(\tau))d\tau\big{\\}}dr\,,$
(29)

them applying the product rule to the process (29), along with the Ito’s
lemma, we get:

$\displaystyle dY_{s}$
$\displaystyle=d\bigg{(}exp\big{\\{}\int_{0}^{s}c(X^{\varepsilon,x}(r))dr\big{\\}}\bigg{)}v^{\varepsilon}(t-s,X^{\varepsilon,x}(s))$
$\displaystyle+exp\big{\\{}\int_{0}^{s}c(X^{\varepsilon,x}(r))dr\big{\\}}dv^{\varepsilon}(t-s,X^{\varepsilon,x}(s))$
$\displaystyle+d\bigg{(}exp\big{\\{}\int_{0}^{s}c(X^{\varepsilon,x}(r))dr\big{\\}}\bigg{)}dv^{\varepsilon}(t-s,X^{\varepsilon,x}(s))$
$\displaystyle+d\bigg{(}\int_{0}^{s}g(X^{\varepsilon,x}(r))exp\big{\\{}\int_{0}^{r}c(X^{\varepsilon,x}(\tau))d\tau\big{\\}}dr\bigg{)}$
$\displaystyle=c(X^{\varepsilon,x}(s))v^{\varepsilon}(t-s,X^{\varepsilon,x}(s))\big{(}exp\big{\\{}\int_{0}^{s}c(X^{\varepsilon,x}(r))dr\big{\\}}\big{)}ds$
$\displaystyle+exp\big{\\{}\int_{0}^{s}c(X^{\varepsilon,x}(r))dr\big{\\}}dv^{\varepsilon}(t-s,X^{\varepsilon,x}(s))$
$\displaystyle+g(X^{\varepsilon,x}(s))exp\big{\\{}\int_{0}^{s}c(X^{\varepsilon,x}(\tau))d\tau\big{\\}}ds$
$\displaystyle=exp\big{\\{}\int_{0}^{s}c(X^{\varepsilon,x}(r))dr\big{\\}}\bigg{(}c(X^{\varepsilon,x}(s))v^{\varepsilon}(t-s,X^{\varepsilon,x}(s))$
$\displaystyle+g(X^{\varepsilon,x}(s))-\frac{\partial
v^{\varepsilon}}{\partial
s}+\frac{\varepsilon^{2}}{2}\sum_{i,j=1}^{r}a^{ij}(s,X)\frac{\partial^{2}v^{\varepsilon}}{\partial
X^{i}X^{j}}+\sum_{i=1}^{r}b^{i}(s,X)\frac{\partial v^{\varepsilon}}{\partial
X^{i}}\bigg{)}ds$
$\displaystyle+exp\big{\\{}\int_{0}^{s}c(X^{\varepsilon,x}(r))dr\big{\\}}\varepsilon\sigma(s,X)\frac{\partial
v^{\varepsilon}}{\partial
X}dW_{s}=exp\big{\\{}\int_{0}^{s}c(X^{\varepsilon,x}(r))dr\big{\\}}\varepsilon\sigma(s,X)\frac{\partial
v^{\varepsilon}}{\partial X}dW_{s}$

Since $v$ satisfies partial differential equation (25) we will have

$dY_{s}=exp\big{\\{}\int_{0}^{s}c(X^{\varepsilon,x}(r))dr\big{\\}}\varepsilon\sigma(s,X)\frac{\partial
v^{\varepsilon}}{\partial X}dW_{s}\,,$

therefore $Y_{s}$ is a martingale and it follows that

$\displaystyle v^{\varepsilon}(t,x)$ $\displaystyle=Y_{0}=E^{x}Y_{t}$
$\displaystyle=E^{x}\bigg{[}f(X^{\varepsilon,x}(t))exp\big{\\{}\int_{0}^{t}c(X^{\varepsilon,x}(s))ds\big{\\}}$
$\displaystyle+\int_{0}^{t}g(X^{\varepsilon,x}(s))exp\big{\\{}\int_{0}^{s}c(X^{\varepsilon,x}(r))dr\big{\\}}ds\bigg{]}\,.$

Using obtained formula for the solution of the PDE (25), we have the following
result:

######  Theorem 4.4.

If conditions (1)-(3) are satisfied, then the limit
$\lim\limits_{\varepsilon\to 0}v^{\varepsilon}(t,x)=v^{0}(t,x)$ exists for
every bounded continuous initial function f(x), $x\in\mathbb{R}^{r}$.
The function $v^{0}(t,x)$ is a solution of problem (26).

###### Proof.

As proved above, the solution of (25) can be represented as:

$v^{\varepsilon}(t,x)=E^{x}[f(X^{\varepsilon,x}(t))exp\big{\\{}\int_{0}^{t}c(X^{\varepsilon,x}(s))ds\big{\\}}+\int_{0}^{t}g(X^{\varepsilon,x}(s))exp\big{\\{}\int_{0}^{s}c(X^{\varepsilon,x}(r))dr\big{\\}}ds]\,.$
(30)

We use the convergence of $X^{\varepsilon,x}(s)$ to $X^{0,x}(s)$ in
probability on the interval $[0,t]$ as $\varepsilon\to 0$ (Theorem 4.3).
Having under the sign of mathematical expectation in (30) a bounded continuous
functional of $X^{\varepsilon,x}(s)$, we can also use the Lebesgue dominated
convergence theorem to derive:

$\displaystyle\lim\limits_{\varepsilon\to 0}v^{\varepsilon}(t,x)$
$\displaystyle=\lim\limits_{\varepsilon\to
0}E^{x}[f(X^{\varepsilon,x}(t))exp\big{\\{}\int_{0}^{t}c(X^{\varepsilon,x}(s))ds\big{\\}}]$
$\displaystyle+\lim\limits_{\varepsilon\to
0}E^{x}[\int_{0}^{t}g(X^{\varepsilon,x}(s))exp\big{\\{}\int_{0}^{s}c(X^{\varepsilon,x}(r))dr\big{\\}}ds]$
$\displaystyle=E^{x}[\lim\limits_{\varepsilon\to
0}f(X^{\varepsilon,x}(t))exp\big{\\{}\int_{0}^{t}c(X^{\varepsilon,x}(s))ds\big{\\}}]$
$\displaystyle+E^{x}[\lim\limits_{\varepsilon\to
0}\int_{0}^{t}g(X^{\varepsilon,x}(s))exp\big{\\{}\int_{0}^{s}c(X^{\varepsilon,x}(r))dr\big{\\}}ds]$
$\displaystyle=E^{x}\bigg{[}f(X^{0,x}(t))exp\big{\\{}\int_{0}^{t}c(X^{0,x}(s))ds\big{\\}}$
$\displaystyle+\int_{0}^{t}g(X^{0,x}(s))exp\big{\\{}\int_{0}^{s}c(X^{0,x}(r))dr\big{\\}}ds\bigg{]}\,,$

the function on the right side of the equality is a solution of (26), hence
concluding the proof. ∎

Considering $c(x)\equiv g(x)\equiv 0$ we obtain the special case of the
transport equation, namely:

$\frac{\partial v^{\varepsilon}(t,x)}{\partial
t}=L^{\varepsilon}v^{\varepsilon}(t,x),\ v^{\varepsilon}(0,x)=f(x),$

whose solution can be written in the following form:

$v^{\varepsilon}(t,x)=E^{x}[f(X^{\varepsilon,x}(t))].$ (31)

Therefore, as in the case of the Cauchy problem, passing to the limit when
$\varepsilon\to 0$, we get $\lim\limits_{\varepsilon\to
0}v^{\varepsilon}(t,x)=v^{0}(t,x)$ which represents the solution of

$\frac{\partial v^{0}(t,x)}{\partial t}=L^{0}v^{0}(t,x),\ v^{0}(0,x)=f(x).$

##  5 Acknowledgments

Yu.K. is very thankful to Mark Freidlin for an instructive discussion.

The financial support by the Ministry for Science and Education of Ukraine
through Project 0122U000048 is gratefully acknowledged.

## References

  * [1] Freidlin M. and Wentzell A. (2012) Random Pertubations of Dynamical Systems, Third Edition, Springer Berlin, Heidelberg.
  * [2] Gihman I.I. and Skorohod A.V. (1972) Stochastic Differential Equations, Springer-Verlag, New York.
  * [3] Gihman I.I. and Skorohod A.V. (1979) Controlled Stochastic Processes, Springer-Verlag, New York.
  * [4] Gihman I.I. and Skorohod A.V. (1979) The Theory of Stochastic Processes III, Springer-Verlag, New York.
  * [5] Kallenberg O. (2021) Foundations of Modern Probability, Third Edition, Springer Cham.
  * [6] Mao X. (2008) Stochastic Differential Equations and Applications, Second Edition, Horwood Publishing, Chichester, UK.
  * [7] Willems J.C.,  Dissipative dynamical systems part I: General theory. Arch. Rational Mech. Anal. 45, 321 351 (1972). https://doi.org/10.1007/BF00276493