# Incorporating Singletons and Mention-based Features in Coreference
Resolution via Multi-task Learning for Better Generalization

Yilun Zhu Department of Linguistics, Georgetown University  Siyao Peng Center
for Information and Language Processing (CIS), LMU Munich  Sameer Pradhan Amir
Zeldes Department of Linguistics, Georgetown University

###### Abstract

Previous attempts to incorporate a mention detection step into end-to-end
neural coreference resolution for English have been hampered by the lack of
singleton mention span data as well as other entity information. This paper
presents a coreference model that learns singletons as well as features such
as entity type and information status via a multi-task learning-based
approach. This approach achieves new state-of-the-art scores on the OntoGUM
benchmark (+2.7 points) and increases robustness on multiple out-of-domain
datasets (+2.3 points on average), likely due to greater generalizability for
mention detection and utilization of more data from singletons when compared
to only coreferent mention pair matching.111The code is publicly available at
https://github.com/yilunzhu/coref-mtl.

##  1 Introduction

Coreference is a linguistic phenomenon that occurs when two or more
expressions in a text refer to the same entity (e.g. the Vice President… She).
Conceptually, resolving coreference takes two steps: identifying all mention
candidates from a text as opposed to non-referring expressions, and linking
identified mentions into clusters. However, in a given document, some mentions
are never referred back to: these are called singletons, i.e. mentions that,
unlike non-referring expressions, could be referred back to in principle, but
are not involved in any coreference relations in context. Singletons are
important to coreference resolution since they represent true negatives in
cluster linking (Kübler and Zhekova, 2011), but also to how humans understand
discourse from a theoretical perspective (Grosz et al., 1995), since they also
constitute mentioned entities (i.e. clusters of size 1).

However, due to the lack of singleton annotation in the most frequently used
coreference dataset for English, i.e. OntoNotes V5.0 (Weischedel et al., 2011;
Pradhan et al., 2013), previous attempts have either ignored singletons (Lee
et al., 2017, 2018; Wu et al., 2020; Dobrovolskii, 2021) or incorporated
pseudo-singletons into the model (Wu and Gardner, 2021; Toshniwal et al.,
2021). The first approach is commonly used in contemporary end-to-end (e2e)
systems which train directly on detecting coreferring mentions, but causes
problems in that models cannot differentiate singleton spans from non-
referring or random/meaningless spans, i.e. penalizing these two types
equally. Though e2e has achieved significant progress on OntoNotes, it does
not align with linguistic theories on how humans resolve the task. The second
approach attempts to amend the model with pseudo-singletons by predicting non-
coreferring mentions, but the accuracy gap between gold and generated
singletons is unknown and ultimately leads to degradation.

Previous work has also shown that recent coreference models struggle with
domain generalization (Moosavi and Strube, 2017; Zhu et al., 2021). To
alleviate the problem, Moosavi and Strube (2018) proposed a novel algorithm to
incorporate linguistic features and showed improvement in out-of-domain (OOD)
data. Subramanian and Roth (2019) applied adversarial training to improve
generalization. However, the first approach requires carefully designed
linguistic features, and both papers evaluated generalization only on one
single-genre dataset, limiting the validity of the results.

To tackle these challenges, we introduce a novel coreference model. Our
contributions can be summarized as follows: First, we propose a multi-task
learning (MTL) based neural coreference model with constrained mention
detection, which jointly learns several mention-based tasks, including
singleton detection, entity type recognition, and information status
classification. Second, experiments demonstrate that the proposed model
achieves new state-of-the-art performance on the OntoGUM test set. Third, we
show that our model outperforms strong baselines on two OOD datasets, showing
it generalizes more reliably to unseen data than plain e2e. We release all
code and provide a system that detects and links all mentions, including
singletons, and outputs predicted entity types.

##  2 Related Work

#### MTL for coreference

Multitask learning Caruana (1997); Collobert and Weston (2008) uses a single
model with shared parameters trained to perform multiple tasks, with potential
benefits arising from synergies between related objectives. Previous work has
investigated the use of MTL for coreference by harnessing related pre-training
tasks. Yu and Poesio (2020); Kobayashi et al. (2022) applied an MTL framework
to a more specific bridging resolution problem, with standard coreference
resolution as the additional task. Luan et al. (2018) used MTL with
coreference resolution, entity recognition, and relation extraction for
scientific knowledge graph construction. Lu and Ng (2021) used five MTL tasks
for event coreference resolution.

#### Neural coreference resolution

The e2e approach jointly learns mention detection and coreferent pair scoring
(Lee et al., 2017), and achieved SOTA scores on the OntoNotes test set before
several extensions were proposed. Lee et al. (2018); Kantor and Globerson
(2019) improved span representations to improve pair matching. Joshi et al.
(2020) added better pre-trained language models to gain additional score
boosting. Wu et al. (2020) adapted a question-answering framework into the
task and improved both span detection and coreference matching scores.
Dobrovolskii (2021) also improved performance by initially matching
coreference links via words instead of spans.

##  3 Methods

###  3.1 Model

Let $N$ be the number of possible spans in a document $D$. The coreference
task can be formulated as assigning an antecedent span $y_{i}$ for each span
$i$, where the set of possible antecedents for each span $i$ contains a dummy
antecedent $\epsilon$ and all preceding spans:
$\mathcal{Y}(i)=\\{\epsilon,1,...,i-1\\}$.

$s(i,j)=\begin{cases}0,&j=\epsilon\\\
s_{m}(i)+s_{m}(j)+s_{c}(i,j),&j\neq\epsilon\end{cases}$

where $s_{m}(i)$ and $s_{m}(j)$ are the mention scores that determine how
likely the selected text span is a mention candidate. Previous work utilizes a
scoring function to measure how likely the span is a coreference markable.
However, singletons in the training data are ignored and thus weaken the
model’s generalization capability. Therefore, our proposed model uses two
scoring functions to represent the distributions of markables and mentions
better. The mention scoring function uses two feed-forward networks fed by the
representation of each span: one part is a markable score that calculates the
score of the span being a coreferent markable in the document; the other is
the mention candidate score that determines how likely a span is a mention
candidate. The formula is represented as follows:

$s_{m}(i)=\beta_{1}\cdot s_{markable(i)}+\beta_{2}\cdot s_{mention(i)}$
$s_{markable(i)}=w_{markble}\cdot\textsc{ffnn}(g_{i})$
$s_{mention(i)}=w_{mention}\cdot\textsc{ffnn}(g_{i})$

where $\cdot$ denotes a dot product, ffnn denotes a feed-forward neural
network, $\beta_{1}$ and $\beta_{2}$ denote model parameters that adjust the
weights of markable scores and mention candidate scores, and $g_{i}$ denotes
the represented embeddings of the span (we use the same span representing
method as in Lee et al. (2017)). The two scoring functions are computed via
two standard feed-forward neural networks. The purpose of this design is to
prevent random text spans being fed to the pair-matching step. Following the
e2e approach (Lee et al., 2017, 2018; Joshi et al., 2020), we concatenate the
boundary representations, the soft head vector and an additional feature
vector $\phi$ containing speaker information, and feed the resulting vector
into separate feed-forward neural networks to calculate markable scores and
mention candidate scores.

In addition to the main pair-matching task, our model adds three mention-based
tasks: a (possibly singleton) mention span detection task, entity type
recognition, and information status classification (see below). For each task,
the span vector is fed into a separate feed-forward network for
classification. Each task is assigned a weight to calculate the total loss
score:

$\mathcal{L}_{total}=\sum_{c=1}^{C}\mathcal{W}_{c}\cdot\mathcal{L}_{c}$

where $\mathcal{W}_{c}$ is the weight for task $c$. See Appendix A for an
overview of the model architecture.

| Markble Detection | MUC |  B3 |  CEAFϕ4 | Avg. F1
---|---|---|---|---|---
P | R | F1 | P | R | F1 | P | R | F1 | P | R | F1
In-domain \- OntoGUM
Joshi et al. (2019) | 91.0 | 71.9 | 80.3 | 83.3 | 69.7 | 75.9 | 70.8 | 59.2 | 64.5 | 70.5 | 45.8 | 55.5 | 65.5
MTL (sg) | 90.2 | 75.0 | 81.9 | 82.7 | 72.8 | 77.4 | 70.4 | 63.1 | 66.5 | 71.5 | 49.2 | 58.3 | 67.6
MTL (sg+ent) | 90.0 | 75.1 | 81.9 | 82.8 | 72.9 | 77.6 | 71.2 | 63.6 | 67.2 | 71.9 | 50.2 | 59.1 | 68.2
MTL (sg+ent+infs.) | 90.0 | 75.0 | 81.8 | 82.1 | 72.3 | 76.9 | 70.0 | 62.3 | 65.9 | 70.0 | 48.6 | 57.3 | 66.9
Out-of-domain \- OntoNotes
Joshi et al. (2019) | 83.9 | 76.9 | 80.3 | 77.6 | 72.7 | 75.1 | 66.9 | 60.6 | 63.6 | 64.3 | 54.5 | 59.0 | 65.9
MTL (sg+ent) | 82.2 | 80.2 | 81.2 | 77.0 | 76.1 | 76.5 | 67.1 | 64.0 | 65.5 | 63.6 | 59.5 | 61.5 | 67.8
Out-of-domain \- WikiCoref
Joshi et al. (2019) | 79.9 | 58.8 | 67.7 | 73.7 | 60.1 | 66.2 | 66.4 | 43.4 | 52.4 | 56.6 | 31.6 | 40.5 | 53.0
MTL (sg+ent) | 80.4 | 60.0 | 68.7 | 74.5 | 61.8 | 67.5 | 67.8 | 45.3 | 54.4 | 59.0 | 33.0 | 42.4 | 55.6

Table 1: Comparison between Joshi et al. (2019) and our model on test sets of
both in-domain (OntoGUM 8.0) and out-of-domain datasets (OntoNotes and
WikiCoref). The overall F1 score is the average of F1s from three evaluation
metrics MUC, B3, and CEAFϕ4. All models are trained on OntoGUM.

###  3.2 Task Selection

Since OntoNotes does not contain singletons, we choose a corpus for which
singleton information is available but follows the same annotation scheme as
OntoNotes. The OntoGUM corpus (Zhu et al., 2021) is an adapted version of the
GUM corpus Zeldes (2017), a multi-layer corpus with a range of annotations at
the word level (part-of-speech, morphology), phrase level (phrase trees,
entity recognition, and linking), dependency level (Universal Dependencies
syntax) and document-level (discourse parses and coreference). Although
OntoGUM uses the same singleton-free coreference scheme as OntoNotes,
information about singletons can be recovered from the original GUM corpus. We
therefore select three annotations from GUM and investigate whether they are
helpful for coreference resolution on OntoGUM: nested mention span detection,
entity type, and information status.

#### Mention detection

As outlined in Section 1, we integrate gold nested mentions, including
singletons (sg), into our model to improve mention detection and coreference.
The task aims to recognize meaningful referential text spans and makes more
information available to the model than the plain e2e approach that only
trains on coreferring mentions ($\sim$39% of mentions in GUM are singletons).

#### Entity type

GUM assigns one of ten entity types (ent) to each mention -- person,
organization, etc. (see Figure 2 in Appendix D). Since a cluster usually has
one entity type, this feature instructs the model regarding which mentions
belong to the same semantic class.

#### Information status

Information status (infs.) indicates how an entity was introduced into
discourse, e.g. new, previously mentioned or inferrable from other mentions
(Prince, 1981). Each mention is assigned one of six labels (see Appendix C).
This task is expected to inform the model about the likelihood and how an
entity was previously introduced.

##  4 Experiments

###  4.1 Datasets

OntoGUM (Zhu et al., 2021) is a coreference dataset following the same
annotation scheme as OntoNotes. This paper adds other layers to the
coreference annotation, such as mention spans (including singletons), aligned
entity types, and information status, automatically extracted from the GUM
corpus. We train the model with GUM v8.0, which includes 193 documents across
12 written and spoken genres with $\sim$180K tokens.

We also evaluate our model on two OOD datasets of the same annotation scheme:
OntoNotes and WikiCoref. OntoNotes includes richly annotated documents with
layers including syntax, propositions, named entities, word senses, and
coreference, but no singleton mentions or aligned (non-named) entity types
(Pradhan et al., 2013). Its test set includes 348 documents with 170K tokens.
WikiCoref (Ghaddar and Langlais, 2016) is a manually annotated corpus from
English Wikipedia, containing 30 documents with $\sim$60K tokens.

###  4.2 Baseline

Combining the e2e approach with a contextualized language model (LM) and span
masking is one of the best models on OntoNotes. Following Joshi et al. (2020),
we use large SpanBERT embeddings as the LM and the improved coarse-to-fine
(Lee et al., 2018) SOTA model as our baseline model (see Appendix B for
implementation details).

###  4.3 Task Weights

The task weights are a list of parameters that controls the relative
importance of various tasks in our model, which are optimized via
hyperparameter search on the OntoGUM dev set to achieve the best performance.
In the optimal setting with 2 auxiliary tasks, the loss weight for the major
task coreference relation identification is set to 0.4 and the weights for
singleton detection and entity type recognition are set to 0.2 each. The
weights are 0.15 for each auxiliary task when information status is added to
training.

###  4.4 Results

#### In-domain Evaluation

We train the model on OntoGUM and evaluate it in-domain. As shown in the first
part of Table 1, our model with the best setting improves average F1 by 2.7
points and achieves new SOTA performance on the OntoGUM benchmark, indicating
the benefit of the MTL tasks. We also note that recall scores of both mention
detection and coreference matching show a significant increase by 3.2 and 4.0
points, respectively, which suggests that the MTL approach helps the model
capture more non-trivial markable spans and coreference relations than the
baseline model, with little or no precision cost. In addition, though
information status contributes to the result as a sole auxiliary task (see
Table 2), it is harmful when training with other tasks.

#### Out-of-domain Evaluation

To test the robustness of our model, we evaluate on two OOD datasets sharing
the same annotation scheme with OntoGUM. The second part of Table 1 shows that
our best in-domain model with mention detection and entity type as auxiliary
tasks outperforms the baseline model on both datasets by 2.3 points on
average. For OntoNotes, though our model has slightly lower precision, the
recall results in substantially better performance; for WikiCoref, our model
performs better on both precision and recall. These results indicate that the
knowledge gained from the multiple mention-based tasks can be transferred to
unseen text types, and is likely a combination of more training data (since
singletons include instances not considered by the baseline training) and the
learning of features distinguishing non-mentions from mentions and ones
corresponding to semantic types.

###  4.5 Ablation Study

To show the importance of each task in our model, we ablate each task in the
architecture and report the average F1 on the OntoGUM development set. In
Table 2, singleton scores and the mention detection task contribute 1.3 points
to the final result, indicating that this feature is the most important one.

| Avg. F1 | $\Delta$
---|---|---
Base model | 67.0 |
w/ singleton detection (=sg) | 68.3 | +1.3
w/ sg + entity type (=et) | 68.7 | +0.4
w/ sg + et + information status | 67.8 | -0.9
Table 2: Comparison of various tasks included in the coreference model on the
OntoGUM development data.

With the addition of the nested entity type recognition task, the model brings
a smaller increase (0.4 points) to the final result. There could be several
reasons for this: one is that the LM has already learned entity types
latently, so giving this as an explicit feature is redundant; the other reason
is that the baseline model rarely groups mentions with different entity types
into clusters so that entity type features can only correct few errors.

When only integrating information status into the model, the result (avg. F1
67.6) outperforms the baseline model, showing the effectiveness of this type
of information. However, when all three tasks are incorporated, the overall
score (67.8) is lower than excluding information status classification (68.7),
which shows that information status is redundant when other mention-based
features are specified.

##  5 Error analysis

We conduct quantitative and qualitative error analyses to illustrate how our
model differs from the baseline. Firstly we conduct a quantitative analysis
following Lu and Ng (2020), who classify resolution errors into 13 classes.
Following their approach, we merge coreference errors into 6 groups. Table 3
displays the distribution of errors observed in the OntoGUM development set.
These errors are present in the baseline e2e model but correctly resolved by
our proposed MTL model (e2e errors) or vice versa (mtl errors).

Error type | mtl errors | e2e errors
---|---|---
Pronouns |  |  |  |
\- 1st & 2nd person pronouns | 6 | 3.6% | 12 | 5.0%
\- 3rd person pronouns | 20 | 12.1% | 68 | 28.3%
Definiteness |  |  |  |
\- Definite nouns | 63 | 38.2% | 98 | 40.8%
\- Indefinite nouns | 13 | 7.9% | 13 | 5.4%
Proper nouns | 23 | 14.0% | 19 | 7.9%
Others | 40 | 24.2% | 30 | 12.5%
Total | 165 | 100.0% | 240 | 100.0%

Table 3: Number and percentage of errors by class that are produced by e2e but
avoided by the MTL model (e2e errors) and produced by the MTL model but
resolved by the e2e model (mtl errors).

The majority of mtl errors involve definite nominals, revealing the challenge
of resolving cherry-picked cases that must be memorized within a multi-genre
context. However, our proposed model demonstrates its ability to correctly
identify relations when multiple clusters are involved. Furthermore, nearly
16% of resolved errors are associated with pronouns, indicating that our model
is more capable of accurately identifying coreference relationships within the
context of third-person pronouns and demonstrates a slight improvement in
handling pronouns in dialogue, particularly first and second-person pronouns.

We also observe that our proposed model reduces errors across nearly all types
compared to the baseline model, particularly in the case of third-person
pronouns. This result suggests that integrating entity type recognition and
mention detection in the MTL framework enables accurate recognition of noun-
pronoun relations, particularly for pronouns that do not provide explicit
entity type information, e.g., it. Additionally, the MTL model demonstrates
improved error avoidance with definite nouns. These findings highlight the
enhanced performance of our proposed model in identifying coreference
relations within the local context.

Entity type errors
---
1 |  he did represent [the school]1 during the very first Eton v [Harrow]1 cricket match
2 |  Who cut [the grass]1? Marlena did [it]2. Marlena did [it]2 a long time ago, but [it]1 hasn’t been watered. [It]1’s dying.
3 |  I made [noises]1 with my heels but [they]1 were too loud so I stopped.
Singleton errors
4 |  The main reason attributed for the pollution of Athens is because the city is enclosed by mountains in [a basin which does not let the smog leave]1 ... have greatly contributed to better atmospheric conditions in [the basin]1.
5 |  This means that if [the govt]1 decided to print 1 quadrillion dollars in the span of a week ... we ’re loaning [the US govt]1 the very money it prints
Table 4: A qualitative analysis of OntoGUM dev errors that appear in the e2e
model but are avoided by our MTL model. MTL predictions (gold) are represented
by [brackets]x. E2e predictions (errors) are highlighted in colored text and
each color in an example denotes a coreference cluster.

We also identify several errors that illustrate the impact of singleton
detection and entity type recognition. Examples in Table 4 demonstrate how
including singletons and mention-based features improves the retrieval of
accurate mention spans and enhances coreference relationships. The first three
examples highlight how entity-type recognition contributes to resolution by
avoiding type mismatches. In example (1), the pressure from entity type
recognition likely aids in identifying Harrow as a school (an organization).
In example (2), the MTL model recognizes it as an event, thereby correctly
creating two distinct groups and avoiding coreference with the grass (a plant
entity). Similarly, example (3) presents pressure to recognize that they is
not an inanimate object, so it correctly prefers noises as the antecedent.
Examples (4) and (5) illustrate how mention detection identifies missing
mentions in the baseline model or improves boundary recognition. These
representative examples provide valuable insights into the significance of
incorporating singletons and auxiliary mention-based tasks into a coreference
model.

##  6 Conclusion

This paper presents a neural coreference model that connects singletons and
other mention-based features to coreference relation matching via an MTL
architecture, which (1) outperforms a strong baseline and achieves new SOTA
results on OntoGUM and (2) beats the baseline model on two unseen datasets.
The results show the effect of singletons and mention features and indicate
improvements in model robustness when transferring to unseen data rather than
overfitting distributions in the training data. In addition, our resulting
system can output all mentions (incl. singletons) with entity types out-of-
the-box, which benefits a series of downstream applications such as Entity
Linking, Dialogue Systems, Machine Translation, Summarization, and more, since
our single model already outputs typed spans for all entities mentioned in a
text (see Figure 2(b) in Appendix D for an illustration).

## Limitations

In this work, we have experimented with training our model on OntoGUM. Due to
the lack of singletons and other mention-based annotations, we do not train
the model on the most frequently used and one of the largest coreference
datasets. Thus the proposed model has not been tested on a large-scale dataset
and compared with other coreference models on OntoNotes.

We evaluate the model on two English OOD datasets to investigate the model
generalization. Several coreference datasets in other languages share the same
annotation scheme as OntoGUM, such as Arabic (Pradhan et al., 2013), and
Chinese (Pradhan et al., 2013). The proposed model needs to be evaluated on
datasets in other languages and demonstrate the model generalization across
languages. However, this would require singleton annotated data in those
languages as well. With recent releases such as CorefUD Nedoluzhko et al.
(2022) promoting standardization of multilingual coreference annotations and
singleton annotations, we are hopeful that such experiments will be possible
in the near future.

## References

  * Caruana (1997) Rich Caruana. 1997.  Multitask learning.  _Mach. Learn._ , 28(1):41–75.
  * Collobert and Weston (2008) Ronan Collobert and Jason Weston. 2008.  A unified architecture for natural language processing: Deep neural networks with multitask learning.  In _Proceedings of the 25th International Conference on Machine Learning_ , ICML ’08, pages 160–--167, New York, NY, USA. Association for Computing Machinery.
  * Dobrovolskii (2021) Vladimir Dobrovolskii. 2021.  Word-level coreference resolution.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 7670--7675, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Ghaddar and Langlais (2016) Abbas Ghaddar and Phillippe Langlais. 2016.  WikiCoref: An English coreference-annotated corpus of Wikipedia articles.  In _Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16)_ , pages 136--142, Portorož, Slovenia. European Language Resources Association (ELRA).
  * Grosz et al. (1995) Barbara J. Grosz, Aravind K. Joshi, and Scott Weinstein. 1995.  Centering: A framework for modeling the local coherence of discourse.  _Computational Linguistics_ , 21(2):203--225.
  * Hou (2020) Yufang Hou. 2020.  Bridging anaphora resolution as question answering.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1428--1438, Online. Association for Computational Linguistics.
  * Joshi et al. (2019) Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2019.  SpanBERT: Improving pre-training by representing and predicting spans.  _CoRR_ , abs/1907.10529.
  * Joshi et al. (2020) Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020.  SpanBERT: Improving pre-training by representing and predicting spans.  _Transactions of the Association for Computational Linguistics_ , 8:64--77.
  * Kantor and Globerson (2019) Ben Kantor and Amir Globerson. 2019.  Coreference resolution with entity equalization.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 673--677, Florence, Italy. Association for Computational Linguistics.
  * Kobayashi et al. (2022) Hideo Kobayashi, Yufang Hou, and Vincent Ng. 2022.  Constrained multi-task learning for bridging resolution.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 759--770, Dublin, Ireland. Association for Computational Linguistics.
  * Kübler and Zhekova (2011) Sandra Kübler and Desislava Zhekova. 2011.  Singletons and coreference resolution evaluation.  In _Proceedings of the International Conference Recent Advances in Natural Language Processing 2011_ , pages 261--267, Hissar, Bulgaria. Association for Computational Linguistics.
  * Lee et al. (2017) Kenton Lee, Luheng He, Mike Lewis, and Luke Zettlemoyer. 2017.  End-to-end neural coreference resolution.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 188--197, Copenhagen, Denmark. Association for Computational Linguistics.
  * Lee et al. (2018) Kenton Lee, Luheng He, and Luke Zettlemoyer. 2018.  Higher-order coreference resolution with coarse-to-fine inference.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)_ , pages 687--692, New Orleans, Louisiana. Association for Computational Linguistics.
  * Lu and Ng (2020) Jing Lu and Vincent Ng. 2020.  Conundrums in entity coreference resolution: Making sense of the state of the art.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 6620--6631, Online. Association for Computational Linguistics.
  * Lu and Ng (2021) Jing Lu and Vincent Ng. 2021.  Constrained multi-task learning for event coreference resolution.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 4504--4514, Online. Association for Computational Linguistics.
  * Luan et al. (2018) Yi Luan, Luheng He, Mari Ostendorf, and Hannaneh Hajishirzi. 2018.  Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 3219--3232, Brussels, Belgium. Association for Computational Linguistics.
  * Moosavi and Strube (2017) Nafise Sadat Moosavi and Michael Strube. 2017.  Lexical features in coreference resolution: To be used with caution.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 14--19, Vancouver, Canada. Association for Computational Linguistics.
  * Moosavi and Strube (2018) Nafise Sadat Moosavi and Michael Strube. 2018.  Using linguistic features to improve the generalization capability of neural coreference resolvers.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 193--203, Brussels, Belgium. Association for Computational Linguistics.
  * Nedoluzhko et al. (2022) Anna Nedoluzhko, Michal Novák, Martin Popel, Zdeněk Žabokrtský, Amir Zeldes, and Daniel Zeman. 2022.  CorefUD 1.0: Coreference meets Universal Dependencies.  In _Proceedings of the Thirteenth Language Resources and Evaluation Conference_ , pages 4859--4872, Marseille, France. European Language Resources Association.
  * Pradhan et al. (2013) Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Hwee Tou Ng, Anders Björkelund, Olga Uryupina, Yuchen Zhang, and Zhi Zhong. 2013.  Towards robust linguistic analysis using OntoNotes.  In _Proceedings of the Seventeenth Conference on Computational Natural Language Learning_ , pages 143--152, Sofia, Bulgaria. Association for Computational Linguistics.
  * Prince (1981) Ellen F. Prince. 1981.  Toward a taxonomy of given-new information.  In P. Cole, editor, _Syntax and semantics: Vol. 14. Radical Pragmatics_ , pages 223--255. Academic Press, New York.
  * Roesiger et al. (2018) Ina Roesiger, Arndt Riester, and Jonas Kuhn. 2018.  Bridging resolution: Task definition, corpus resources and rule-based experiments.  In _Proceedings of the 27th International Conference on Computational Linguistics_ , pages 3516--3528, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  * Subramanian and Roth (2019) Sanjay Subramanian and Dan Roth. 2019.  Improving generalization in coreference resolution via adversarial training.  In _Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019)_ , pages 192--197, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Toshniwal et al. (2021) Shubham Toshniwal, Patrick Xia, Sam Wiseman, Karen Livescu, and Kevin Gimpel. 2021.  On generalization in coreference resolution.  In _Proceedings of the Fourth Workshop on Computational Models of Reference, Anaphora and Coreference_ , pages 111--120, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Weischedel et al. (2011) Ralph Weischedel, Eduard Hovy, Mitchell Marcus, Martha Palmer, Robert Belvin, Sameer Pradhan, Lance Ramshaw, and Nianwen Xue. 2011.  OntoNotes: A large training corpus for enhanced processing.  In Joseph Olive, Caitlin Christianson, and John McCary, editors, _Handbook of Natural Language Processing and Machine Translation: DARPA Global Autonomous Language Exploitation_. Springer.
  * Wu et al. (2020) Wei Wu, Fei Wang, Arianna Yuan, Fei Wu, and Jiwei Li. 2020.  CorefQA: Coreference resolution as query-based span prediction.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 6953--6963, Online. Association for Computational Linguistics.
  * Wu and Gardner (2021) Zhaofeng Wu and Matt Gardner. 2021.  Understanding mention detector-linker interaction in neural coreference resolution.  In _Proceedings of the Fourth Workshop on Computational Models of Reference, Anaphora and Coreference_ , pages 150--157, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Yu and Poesio (2020) Juntao Yu and Massimo Poesio. 2020.  Multitask learning-based neural bridging reference resolution.  In _Proceedings of the 28th International Conference on Computational Linguistics_ , pages 3534--3546, Barcelona, Spain (Online). International Committee on Computational Linguistics.
  * Zeldes (2017) Amir Zeldes. 2017.  The GUM corpus: Creating multilayer resources in the classroom.  _Language Resources and Evaluation_ , 51(3):581--612.
  * Zhu et al. (2021) Yilun Zhu, Sameer Pradhan, and Amir Zeldes. 2021.  OntoGUM: Evaluating contextualized SOTA coreference resolution on 12 more genres.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_ , pages 461--467, Online. Association for Computational Linguistics.

##  Appendix A Model architecture oveview

Figure 1 shows the architecture of the model proposed in this paper.

##  Appendix B Implementation details

We use Pytorch and the pre-trained SpanBERT-large (Joshi et al., 2020) model
from HuggingFace222https://huggingface.co/ for token representations.
Experiments run on Nvidia RTX A6000 GPUs with 64GB RAM. Following previous
work (Lee et al., 2018; Joshi et al., 2020), we use a batch size of 1 document
for training and evaluation. The coreference task uses the same loss strategy
as the baseline model (Joshi et al., 2020) and each auxiliary task uses Cross
Entropy loss. We use AdamW to optimize coreference loss and Adam to optimize
auxiliary loss. We train 14,500 steps with task_learning_rate of 0.0003 for
baselines and our models.

Figure 1: An overview of the proposed MTL model architecture. Only selected
spans with high mention scores (in blue) are considered in the three auxiliary
tasks.

##  Appendix C Information Status

There are six types of information status in the data:

  * •

new (first, unmediated mention of an entity)

  * •

given:active (subsequent mention after a recent previous mention)

  * •

given:inactive (subsequent mention of a non-recently mentioned entity)

  * •

accessible:inferrable (new entity whose existence could be inferred from other
mentions, e.g. via bridging anaphora Roesiger et al. (2018); Hou (2020), as in
a house … [the door])

  * •

accessible:commonground (entities accessible to speakers in the situation,
e.g. pass [the salt]!)

  * •

accessible:aggregate (new entities referring back to multiple entities, i.e.
split antecedents as in Kim … Yun … [they]).

When information status is included in the auxiliary tasks, our model is
trained to predict the label for each mention.

##  Appendix D Sample data

Figure 2(b) shows the extent of annotations available to the MTL model for
training, compared to the data restricted to coreferring pairs in Figure 2(a),
as used by the baseline e2e approach. Since OntoNotes-style data, such as
OntoGUM, does not contain singletons, mention types or information status,
only coreferring mentions and their spans can be used for learning by the
baseline model. The information in the bottom panel, by contrast, is much
richer and covers all referring expressions with information status and one of
ten entity types: abstract, animal, event, object, organization, person,
place, plant, substance and time.

While each information type (coreference, mention boundaries, mention types,
and information status) is not totally predictable from others, they overlap
to some extent and exhibit different information densities: mention boundaries
are available for many spans and are densely attested. Mention types are
available for each mention, but some types are rare, e.g. abstract mentions
marked by  in Figure 2 are the most common. Information status is mostly
predictable from coreference, e.g. singletons and chain-initial mentions are
new, and chain-medial or final mentions are given (given:active if recently
mentioned, otherwise given:inactive). Accessible mentions are less trivial and
comparatively rare (about 7.1% of mentions in GUM), indicating whether they
are accessible in the common ground, their identity is inferrable from some
other mention (accessible:inferrable), or by aggregating information from
multiple mentions (accessible:aggregate). This information could help systems
to learn whether a span is likely to have an antecedent.

(a) Information available to the baseline e2e model.

(b) Information available to the MTL model for the same document.

Figure 2: Training data from an OntoGUM article in the news genre.


.tiffpdf.pdfconvert #1 .tiff

# Learning and Leveraging Features in Flow-Like Environments to Improve
Situational Awareness

Tahiya Salam1, Victoria Edwards1, and M. Ani Hsieh1 Manuscript received:
September 9, 2021; Revised December 3, 2021; Accepted December 30, 2021.This
paper was recommended for publication by Editor Youngjin Choi upon evaluation
of the Associate Editor and Reviewers’ comments. This work was supported by
ARL DCIST CRA W911NF-17-2-0181 and NSF IIS 1812319.1All authors are with the
General Robotics Actuation Sensing and Perception (GRASP) Laboratory at the
University of Pennsylvania, Philadelphia, PA, USA {tsalam, vmedw,
<EMAIL_ADDRESS>Object Identifier (DOI): see top of this page.

###### Abstract

This paper studies how global dynamics and knowledge of high-level features
can inform decision-making for robots in flow-like environments. Specifically,
we investigate how coherent sets, an environmental feature found in these
environments, inform robot awareness within these scenarios. The proposed
approach is an online environmental feature generator which can be used for
robot reasoning. We compute coherent sets online with techniques from machine
learning and design frameworks for robot behavior that leverage coherent set
features. We demonstrate the effectiveness of online methods over offline
methods. Notably, we apply these online methods for robot monitoring of
pedestrian behaviors and robot navigation through water. Environmental
features such as coherent sets provide rich context to robots for smarter,
more efficient behavior.

###### Index Terms:

Environment Monitoring and Management, Marine Robotics, Surveillance Robotic
Systems

##  I Introduction

Robots operating in flow-like environments benefit from a greater
understanding of the characteristics of their surroundings. Flow-like
environments, such as ocean and air currents, movements of vehicles and
pedestrians, or transport of blood in the heart, are often complex and hard to
model. However, robots conducting environmental monitoring, autonomous
driving, crowd monitoring, or surgical procedures all require knowledge of
these flow-like environments in their tasks. The ability to deduce and
incorporate the important features of these environments into planning and
decision-making allows robots to operate more successfully. We propose a
strategy for robots to learn high-level features in flow-like environments and
show how these features can be incorporated into existing planning frameworks.

(a)  (b)

Figure 1: Time-dependent double gyre flow at times (a) $t=0$ and (b) $t=5.1$.
The double gyre model is a prototypical model in the study of coherent sets.
The black particles, not in a region of coherence, disperse quickly. The pink
particles within a coherent structure follow the flow and stay within a
region.

The typical study of flow-like environments requires knowledge of the flow map
or vector field that describes the evolution of the entire state of the system
under a specified time frame. Robotics application within these environments
usually involve detailed knowledge and representation of the underlying flow
velocity [1, 2, 3], which is not always feasible and may be computationally
expensive. The methods in this paper allows us to represent key features of
the flow-like environment without explicitly estimating or tracking the
equations of motions. Here, we rely on the extraction of the global dynamics
of the system, namely coherent sets, from noisy data to capture high-level,
overarching spatial trends of the particles.

Coherent sets are a global characteristic of turbulent flows that describe
groups of particles that move together robustly and do not disperse much [4].
This means particles within coherent sets tend to stay within the sets (see
Fig. 1). Lagrangian coherent structures (LCS) are separatices that delineate
dynamically distinct regions in dynamical systems [5]. LCS provide different
but complementary information when compared to coherent sets [6]. Knowledge of
coherent sets and LCS allows robots to compactly represent key characteristics
of these flow-like environments and leverage them in application-specific
ways. For example, surgical robots could understand vasculature conditions
such as flow stagnation and separation during medical procedures [7]. Aerial
vehicles are affected by coherent features during takeoff and landing [8].
Robots used for environmental monitoring may be interested in atmospheric and
oceanic transport pathways that play a role in the movement of pollutants [9].
Marine vehicles can plan energy efficient trajectories in the ocean by
leveraging coherent structures [10, 11, 12] and maintaining sensors in their
desired monitoring regions [13, 14]. Knowledge of the coherent sets improves
robots’ situational awareness in context-specific ways by offering a
lightweight feature of the complex environment that can be used for decision-
making and high-level, coarse planning.

There are many methods for studying coherent sets through transfer operators
which provide a framework for analyzing the global behaviors of dynamical
systems. These methods identify which geometric structures are the strongest
barriers to global mixing [15, 16, 17, 18]. There are also methods for
identifying LCS relying on detailed flow field data and integration over a
parametrized time interval [19]. While these approaches elucidate coherence
that can be used for robot planning and control, none of these methods take
into account the complex interdependencies within the data sets and require
detailed environmental knowledge. Additionally, for many robotic applications,
online algorithms are preferred, especially when the time parameterization of
the system is unknown in the absence of prior models or historical data.

In recent years, there has been a surge of machine learning approaches relying
on kernels (or feature maps) and transfer operators to estimate dynamical
systems and their key characteristics [20, 21, 22, 23, 24, 25, 26]. Kernels
map the environmental data to a higher dimensional feature space with greater
expressivity for learning models. New advances in the detection of coherent
sets leverage techniques from machine learning [23, 24]. Though these
techniques highlight the expressiveness of data-driven, kernel-based methods,
they still suffer from the same limitations in their assumptions about
centralized, offline data, and time parameterization.

Despite the field’s growing interest in transfer operators [27, 28, 29, 30],
to the best of our knowledge, these advances at the intersection of machine
learning and dynamical systems have not been explored in a robotics context
using online algorithms. Our work builds upon these recent advances to develop
strategies that allow robots to compute coherent set features online and
contextualize them in the robots’ environment for increased situational
awareness and to facilitate decision-making. The contributions of this paper
are:

  * •

online feature detection methods within complex environmental structures and

  * •

demonstrations of how coherent set features can be used for decision-making
and planning in two application settings.

We propose using the coherent set detection framework as a map for greater
situational awareness for operating within flow-like environments that exhibit
this notion of coherence. First, we provide the relevant background in
dynamical systems and machine learning related to coherent sets and show how
kernel methods can represent coherent sets from data. Next, we demonstrate the
usefulness of computing coherent sets in an online setting with simulations
and analysis of known benchmarks. We apply these online methods for computing
coherent sets for overhead data in an urban monitoring environment, where the
coherent sets correspond to regions of interest for situational awareness.
Lastly, we show how sparse representations of coherent sets in a fluid flow
can be leveraged for energy-efficient navigation of surface vehicles.

##  II Methodology

Figure 2: Diagram of interplay between data, transfer operators, kernel
methods, and environmental features. Transfer operators represent dynamical
systems, where a state $x\in\mathbb{X}$ is lifted to a space
$L^{\infty}\mathbb{X}$ and $g(x)$ provides physical properties of the system.
Many systems are defined by data exhibiting complex patterns, such as two
nested rings, flows in oceans, taxi trajectories, and biological behaviors.
Kernel methods transform this data to an alternative space with the use of
kernel functions. Data is then easier to interpret, such as by separating two
nested rings or by creating a Gram matrix for use in a kernel algorithm.
Transfer operators are represented through kernel methods by embedding
dynamical systems into a kernel space, $\mathbf{R}$. Kernel algorithms extract
environmental features from transfer operators, such as where humans tend to
congregate in crowds, areas of gyres in oceans, or patterns of blood flow in
hearts.

###  II-A Dynamical Systems and Transfer Operators

Coherent sets are global properties of a dynamical system, extracted through
constructing transfer operators. In this section, we discuss the connection
between dynamical systems and transfer operators.

Define a discrete dynamical system as $x_{i+1}=F(x_{i})$ where $F:X\rightarrow
X$ is a measurable map. Transfer operators allow for a systematic study of the
global properties of dynamical systems and alleviates computing the map $F$,
which is often difficult to model and estimate for unknown environments
exhibiting complex, nonlinear dynamics. For example, the Koopman operator is
an infinite dimensional linear transfer operator that fully captures the
nonlinear dynamics of the system by acting on the space of scalar-valued
functions operating on the observables of the system, $g(x)$.

Transfer operators can more generally be defined for stochastic dynamical
systems to infer statistical aspects of a dynamical system. Define
$\\{X_{t}\\}_{t\geq 0}$ as a stochastic process on the state space
$\mathbb{X}\in\mathbb{R}^{d}$. $\\{X_{t}\\}_{t\geq 0}$ is a stationary and
ergodic Markov decision process. A transition density function, $p_{\tau}$, is
then defined by the relation
$\mathbb{P}[X_{t+\tau}\in\mathbb{A}|X_{t}=x]=\int_{\mathbb{A}}p_{\tau}(y|x)dx$
for some measurable set $\mathbb{A}$.

Given a probability density $p_{t}\in L^{1}(\mathbb{X})$, an observable of the
system $f_{t}\in L^{\infty}(\mathbb{X})$, and a specified lag time $\tau$, we
can define the following transfer operators that provides information about
iterated maps. Specifically, the Perron-Frobenius and Koopman transfer
operators serve as adjoints to one another. Furthermore, the eigenvalues of
transfer operators are real and their eigenfunctions form an orthogonal basis
with respect to the corresponding scalar product. The Perron-Frobenius,
$\mathcal{P}$, and Koopman, $\mathcal{K}$, transfer operators are defined as
follows: $\mathcal{P}:L^{1}(\mathbb{X})\rightarrow L^{1}(\mathbb{X})$ where
$\mathcal{P}p_{t}(y)=\int p_{\tau}(y|x)p_{t}(x)dx$, while
$\mathcal{K}:L^{\infty}(\mathbb{X})\rightarrow L^{\infty}(\mathbb{X})$ where
$\mathcal{K}f_{t}(x)=\int
p_{\tau}(y|x)f_{t}(y)dy=\mathbb{E}[f_{t}(X_{t+\tau})|X_{t}=x]$.

Transfer operators and their decompositions are often approximated using
methods acting directly on observations of the dynamical systems, i.e., data
[31, 23, 32, 33].

###  II-B Kernel Methods

Instead of constructing models using data directly, kernel methods rely on
transforming data into a feature representation that elucidates interesting,
meaningful patterns. Here, we present the relevant background for kernel
methods, which later serves as the foundation for estimating transfer
operators.

Kernel functions $k:X\times X\rightarrow\mathbf{R}$ are applied to elements of
some space, $X$, to measure the similarity between any pairs of elements.
However, this similarity metric can be found by computing the inner product of
features $\phi(x)$ acting on elements of the input space in some high-
dimensional, possibly infinite feature space in $\mathbf{R}^{N}$. Thus,
kernels allow us to compare objects based on their features. Formally, for
inputs $x,x^{\prime}\in X$, a feature map $\phi:X\rightarrow\mathbf{R}^{N}$,
and some valid inner product $\langle\cdot{,}\cdot\rangle_{\mathcal{V}}$, a
kernel $k$ is defined as

$k(x,x^{\prime})=\langle\,\phi(x),\phi(x^{\prime})\rangle_{\mathcal{V}}.\vspace{-2.5mm}$
(1)

By using the relation in Eq. (1), commonly referred to as the kernel trick,
operations in the high-dimensional feature space can be computed without ever
explicitly computing the coordinates of $\phi(x)$ and instead using an
efficient kernel function in inner product space [34].

Kernels on features spaces that are positive definite can be used to define a
function $f$ on $X$. The space of such functions $f$, such that the evaluation
of $f$ at $x$ can be represented as the inner product
$\langle\,\phi(x),\phi(x^{\prime})\rangle$ in feature space, is referred to as
the Reproducing Kernel Hilbert Space (RKHS). An RKHS is a space $H$ of
functions mapping $X$ to $\mathbf{R}$ with two defining features:

$\displaystyle\forall x\in X,\phi(x)\in H,$ (2) $\displaystyle\forall x\in
X,\forall f\in H\phi(x)\in H,\langle\,f,\phi(x)\rangle_{H}=f(x).$ (3)

These two properties show that in an RKHS the feature map of every point is in
the feature space as in (2) and that the space exhibits the reproducing
property as in (3). The reproducing property defines a functional evaluation
at a point as the inner product between the function and evaluated feature map
[35].

Let $\phi$ be the feature map associated with kernel $k$ defined on $X$ and
$\psi$ be the feature map associated with kernel $l$ defined on $Y$. Then,
define feature matrices as $\Phi=[\phi(x_{1})\dots\phi(x_{n})]$ and
$\Psi=[\psi(y_{1})\dots\psi(y_{n})]$, with Gram matrices $G_{XX}=\Phi^{T}\Phi$
and $G_{YY}=\Psi^{T}\Psi$. $C_{XX}$ is then the covariance operator and
$C_{YX}$ is the cross-covariance operator, which can be thought of as a
nonlinear generalization of standard covariance and cross-covariance matrices
[23]. $C_{XX}$ and $C_{YX}$ can be estimated as

$\displaystyle\hat{C}_{XX}=\frac{1}{n}\Phi\Phi^{T},$ (4)
$\displaystyle\hat{C}_{YX}=\frac{1}{n}\Psi\Phi^{T}.$ (5)

The covariance and cross-covariance operators are one of the most important
and established tools in RKHS theory. These operators are used in the
computation of kernel principal components analysis (PCA), the kernel Fisher
discriminant, kernel partial least squares, the kernel canonical correlation
analysis (CCA), and many other learning algorithms [36]. Some learning
algorithms relying on the use of (4) and (5), can be written in terms of their
corresponding Gram matrices. For Gram matrices, only the inner products
between $\phi(x)$ and $\phi(x^{\prime})$ and $\psi(x)$ and $\psi(x^{\prime})$
need to be computed. Thus, by computing Gram matrices on a finite number of
data points, we can alleviate the need to explicitly compute the possibly
infinite-dimensional covariance and cross-covariance operator. The ability to
construct algorithms in the inner product space using kernels, and more
specifically the kernel trick, lends them to be particularly useful in a
variety of learning scenarios [34]. In the next section, we will lay the
framework for use of transfer operator theory in the RKHS and later show how
this expression in RKHS can be used as a learning technique for modeling
coherent sets and decision-making with this awareness.

###  II-C Connecting Transfer Operators to Kernel Methods

Transfer operator approximation in the RKHS allows for the use of kernel-based
methods, acting on feature maps of the data, while still capturing the
properties of the transfer operators. Transfer operators can be expressed in
terms of the covariance and cross-covariance operators defined on some RKHS
[23]. Assume the input and output space are the same, i.e., $X=Y$, then the
kernels and resulting Hilbert spaces are also the same. In this scenario, we
assume $g(x)=x$, meaning full-state observables are simply the states
themselves.

The kernel Koopman operator is defined as

$\mathcal{K}_{k}f(x)=\langle C_{XX}^{-1}C_{XY}f\,,k(x,\cdot)\rangle$ (6)

and pushes forward the observables. Using the approximation of $C_{XX}$ from
(4) and $C_{XY}$ from the transpose of (5), the empirical estimate of the
Koopman operator is

$\hat{\mathcal{K}}_{k}=C_{XX}^{-1}C_{XY}=(\Phi\Phi^{T})^{-1}(\Phi\Psi^{T})=\Phi
G_{XX}^{-1}\Psi^{T}.$ (7)

This value can be calculated empirically from training data. Similarly, the
kernel Perron-Frobenius operator is defined as

$\mathcal{P}_{k}p(x)=\langle C_{XX}^{-1}C_{YX}p\,,k(x,\cdot)\rangle$ (8)

with the empirical estimate of the Perron-Frobenius as

$\hat{\mathcal{P}}_{k}=C_{XX}^{-1}C_{YX}=(\Phi\Phi^{T})^{-1}(\Psi\Phi^{T})=\Psi
A\Phi^{T}$ (9)

with $A=G_{XY}^{-1}G_{XX}^{-1}G_{XY}$. For complete derivations, see [23].

We now have a representation of the transfer operators in terms of the
covariance and cross-covariance operators that can be estimated using the
kernel versions (4) and (5). Thus, we now have a framework unifying existing
techniques for approximating transfer operators and their eigendecompositions
in RKHS [23], as shown in Fig. 2. In our work, we can use additional
techniques from statistics to extract the dominant characteristics from the
transfer operator representation of the environment, specifically coherent
sets.

###  II-D Kernel Methods for Coherent Set Detection

Here, we will provide the mathematical definition of coherent sets, show their
representation through transfer operators, and demonstrate how this transfer
operator representation can be computed using their kernel versions. We refer
the reader to [24] for a detailed discussion on these methods and the
intuition behind how these methods relate to coherence. Let
$\mathcal{F}:L_{\mu}^{2}(\mathbb{X})\rightarrow L^{2}(\mathbb{Y})$ be the
forward operator, for some $\mu$ that is a reference density of interest.
Given $\nu=\mathcal{F}\mathds{1}$ as the image density obtained by mapping the
indicator function on $\mathbb{X}$ forward in time. If we normalize
$\mathcal{F}$ with respect to $\nu$, we can define a new operator
$\mathcal{A}:L^{2}_{\mu}(\mathbb{X})\rightarrow L^{2}_{\nu}(\mathbb{Y})$ and
its adjoint $\mathcal{A^{*}}:L^{2}_{\nu}(\mathbb{Y})\rightarrow
L^{2}_{\mu}(\mathbb{X})$ as

$\displaystyle\vspace{-2.5mm}\vspace{-2.5mm}(\mathcal{A}f)(y)=\int{\frac{p_{\tau}(y|x)}{\nu(y)}}f(x)\mu(x)dx,$
$\displaystyle(\mathcal{A}^{*}g)(x)=\int{p_{\tau}(y|x)g(y)dx}.$

Here, $\mathcal{A}$ is the analogue of the reweighted Perron-Frobenius
operator, or the Koopman operator with the roles of $X$ and $Y$ reversed, and
$\mathcal{A}^{*}$ is the analogue of the Koopman operator. The operator
$\mathcal{A^{*}\mathcal{A}}$ is used to detect coherent sets. To detect the
coherent sets, the following eigenvalue problems needs to be solved:

$\mathcal{A}^{*}\mathcal{A}f=\rho^{2}f.$ (10)

In general, the inverses of the covariance and cross covariance operators do
not exist. Thus, the regularized inverses are used, specifically we use
$(C_{XX}+\epsilon\mathcal{I})^{-1}$ and $(C_{YY}+\epsilon\mathcal{I})^{-1}$.
We can then define the RKHS approximation of $\mathcal{A}$ as
$(C_{XX}+\epsilon\mathcal{I})^{-1}C_{XY}$ and $\mathcal{A}^{*}$ as
$(C_{YY}+\epsilon\mathcal{I})^{-1}C_{YX}$ from the kernel formulations of
transfer operators in (6) and (8).

Using these regularized inverses to solve (10) gives

$(C_{XX}+\epsilon\mathcal{I})^{-1}C_{XY}(C_{YY}+\epsilon\mathcal{I})^{-1}C_{YX}f=\rho^{2}f.$
(11)

Using the empirical estimates from (7) and (9), the eigenvalue problem of the
RKHS version of the transfer operators in Eq. (11) becomes

$\vspace{-2.5mm}\Phi B\Phi^{T}\hat{f}=\rho^{2}\hat{f}$

for $B=(G_{XX}+n\epsilon I)^{-1}(G_{YY}+n\epsilon I)^{-1}G_{YY}$. Instead of
solving this eigenvalue problem directly, we can instead solve a surrogate
eigenvalue problem

$\begin{split}\vspace{-2.5mm}\vspace{-2.5mm}(G_{XX}+n\epsilon
I)^{-1}(G_{YY}+n\epsilon I)^{-1}G_{YY}G_{XX}v&=\rho^{2}v\text{, }\\\
\text{with }\hat{f}&=\Phi v.\end{split}$ (12)

In practice, we will solve this eigenvalue problem to compute the coherent
sets of a system. In order to compute the coherent sets, we first compute the
dominant eigenfunctions, as in Eq. (12). Then, a $k$-means clustering is
applied to the $k$ dominant eigenfunctions to determine the coherent sets.

###  II-E Online Computation of Coherent Sets

Robots collect data in real time and knowledge of the dynamics of the
environment is limited; thus, we desire a way to detect coherent sets online
using $\mathcal{A}^{*}\mathcal{A}$ and solving the eigenvalue problem,
estimated as Eq. (10). The operator $\mathcal{A}^{*}\mathcal{A}$ can be
represented as a function of covariance operators, $\mathcal{C}_{XX}$ and
$\mathcal{C}_{YY}$, and cross covariance operators, $\mathcal{C}_{XY}$ and
$\mathcal{C}_{YX}$. Instead of using covariance and cross covariance operators
directly, the eigenvalue problem is solved using a surrogate formulation (12).
The surrogate formulation relies on Gram matrices $G_{XX}$ and $G_{YY}$, where
$G_{XX}$ captures correlations between the initial positions of the tracked
objects and $G_{YY}$ captures correlations between the final positions of the
objects. However, robots collect sensor measurements in real time. Other works
assume knowledge of the fixed lag time $\tau$, such that analyzing the
operator $\mathcal{A}^{*}\mathcal{A}$ with data at $t_{0}$ and $\tau$ computes
coherent sets [24]. Our paper analyzes how coherence can be computed in an
online setting with no prior assumptions about $\tau$. In this section, we
extend the work from [24] and use insights in dynamical systems theory from
[18, 37, 38] to offer a mathematically sound method for computing coherent
sets online using kernel-based techniques from machine learning.

For coherence at several time instances, studies have shown it is sufficient
to average the operators $\mathcal{A}^{*}\mathcal{A}$ for all the different
time instances and compute the dominant eigenfunctions of the resulting
operator [18, 37, 38]. Let $\mathcal{A}^{*}_{t_{0},t}$ correspond to the
Koopman operator and $\mathcal{A}_{t_{0},t}$ correspond to the Perron-
Frobenius operator for the dynamics from $t_{0}$ to $t$. Then, we are
interested in the time-averaged quantity

$\frac{1}{T}\sum\limits_{i=0}^{T-1}\mathcal{A}^{*}_{0,i}\mathcal{A}_{0,i}.$
(13)

Averaging the operator $\mathcal{A}^{*}\mathcal{A}$ by using the
representations of the operators in (11) and the corresponding surrogate
eigenvalue problem in (12), we are interested, at each time step $t$, the
eigenvalue decomposition of

$\vspace{-2.5mm}\frac{1}{t}\sum\limits_{i=0}^{t}(G_{XX}+n\epsilon
I)^{-1}(G_{Y_{i}Y_{i}}+n\epsilon I)^{-1}G_{Y_{i}Y_{i}}G_{XX},$

which can be rewritten as

$\vspace{-2.5mm}(G_{XX}+n\epsilon
I)^{-1}\left[\frac{1}{t}\sum\limits_{i=0}^{t-1}(G_{Y_{i}Y_{i}}+n\epsilon
I)^{-1}G_{Y_{i}Y_{i}}\right]G_{XX}.$ (14)

The inner sum is computed as the standard optimal online averaging solution
with the addition of more data. Then, the eigenvalue decomposition of the full
resulting operator for the surrogate problem, Eq. (14), is computed at each
time step. Finally, $k$-means clustering is applied, as before.

##  III Simulations and Analysis

We present two examples of fluid flows, where the coherent sets are well-
studied. We perform an analysis on the quality of the features generated from
our online method compared to those from offline methods. We use known systems
to analyze the performance of our online methods since it is hard and often
infeasible to obtain ground truth for coherent sets for real-world
applications, especially when the underlying dynamics are poorly understood.

###  III-A Time-Dependent Double Gyre

A simple model of the wind-driven, time-dependent double gyre flow, as in
[39], is described by

$\displaystyle f(x,t)$ $\displaystyle=\epsilon*\sin{(\omega
t)}*x^{2}+(1-2\epsilon\sin{(\omega t}))*x$ $\displaystyle\frac{\partial
f}{\partial x}$ $\displaystyle=2\alpha*\sin{(\omega
t)}*x+(1-2\alpha\sin{(\omega t}))$ $\displaystyle\dot{x}$ $\displaystyle=-\pi
A\sin{(\pi f(x,t))}\cos{(\pi y)}$ $\displaystyle\dot{y}$ $\displaystyle=\pi
A\cos{(\pi f(x,t))}\sin{(\pi y)}*\frac{\partial f}{\partial x}.$

(a)  (b)

Figure 3: (a) $20\,000$ particles are initialized in a $[0,2]\times[0,1]$
environment. Particles are colored by their $x$-coordinate to visualize their
movement. (b) The position of the particles are tracked under a time-dependent
double gyre flow at time $t=12.1$. Two dynamically distinct regions can be
seen as a result of this type of flow, which correspond to coherent sets.

(a)  (b)

Figure 4: The Bickley jet is an idealized model for fluid flows, such as in
the ocean or in the atmosphere. (a) $9900$ particles are initialized in a
$[0,20]\times[-3,3]$ environment. (b) The movement of the particles are
tracked under a Bickley jet model until $t=40$, shown here at $t=10.1$.

For this paper, the parameters are set to
$\epsilon=0.25,\alpha=0.25,A=0.25,\text{and }\omega=2\pi$. We simulate
$20\,000$ points uniformly sampled on grid $[0,2]\times[0,1]$. From time
$t\in[0,20]$ with step size $0.1$, we use a differential equations solver
based on an explicit Runge-Kutta (4,5) formula on the velocities to calculate
the trajectories of the points, as shown in Fig. 3.

With a Gaussian radial basis function kernel
$k(x,x^{\prime})=\exp(-(\left\lVert x-x^{\prime}\right\rVert)/(2\sigma^{2}))$
for $\sigma=0.75$, we compute the Gramian matrices for $G_{XX}$ for the
initial position of the particles and $G_{Y_{i}Y_{i}}$ for the position of the
particles at time $i$, as in Eq. (14). In an online setting, only the inner
sum needs to be recomputed as a moving average. The total product and its
eigenvalue decomposition is performed. A $k$-means clustering for $k=3$ is
computed on the $3$ dominant eigenvalues of the estimated time-averaged matrix
Eq. (13).

###  III-B Bickley Jet

The Bickley jet model is a prototypical model in the study of coherence that
is a meandering zonal jet, flanked both above and below by counter rotating
vertices. The Bickley jet is used as an idealized model for the Gulf Stream in
the ocean and polar night jets in the atmosphere [40, 41].

The stream function for the Bickley jet model is

$\displaystyle\psi(x,y,t)$ $\displaystyle=\psi_{0}(y)+\psi_{1}(x,y,t)$
$\displaystyle\psi_{0}(y)$
$\displaystyle=-U_{0}L_{0}\tanh\left(\frac{y}{L_{0}}\right)$
$\displaystyle\psi_{1}(x,y,t)$
$\displaystyle=U_{0}L_{0}\operatorname{sech}^{2}\left(\frac{y}{L_{0}}\right)\Re\left({\sum_{n=1}^{3}f_{n}(t)\exp(iknx)}\right),$

with $f_{n}(t)=\epsilon_{n}\exp(-ik_{n}c_{n}t)$. The velocities can be
computed as $\dot{x}={\partial{\psi}}/{\partial{x}}$ and
$\dot{y}={\partial{\psi}}/{\partial{y}}$. We use scaled parameters from [42]
resulting in
$U_{0}=5.4138,L_{0}=1.77,c_{1}=0.1446U_{0},c_{2}=0.2053U_{0},c_{3}=0.4561U_{0},\epsilon_{1}=0.075,\epsilon_{2}=0.4,\epsilon_{3}=0.3,r_{0}=6.371,k_{1}=2/r_{0},k_{2}=4/r_{0},k_{3}=6/r_{0}$.

We sample $9900$ points uniformly on a grid $[0,20]\times[-3,3]$. From time
$t\in[0,40]$ with step size $0.1$, we calculate the trajectories of the points
using a variable-step, variable-order Adams-Bashforth-Moulton solver of orders
1 to 13 on the velocity $[\dot{x},\dot{y}]$, as shown in Fig. 4. Some
particles escape the grid $[0,20]\times[-3,3]$, and the positions of these
particles wrap around such that they stay within the grid. The true time lag
parameter $\tau$, as introduced in Sec. II-E, for this map is known to be $40$
for this simulation.

For this example, we also use a Gaussian kernel, with $\sigma=1$. We compute
the Gramian matrices for $G_{XX}$ for the initial position of the particles
and $G_{Y_{i}Y_{i}}$, as before for the time-varying Eq. (14) and perform
$k$-means clustering for $k=9$ with $9$ dominant eigenvalues of the operator,
similar to [24].

###  III-C Analysis of Online Coherent Sets Computation

TABLE I: Comparison of Evaluation Metrics on Methods for Computing Coherent Sets Environment |  Method | Evaluation Metric
---|---|---
RI | H | V
Time-Dependent Double Gyre |  Offline Online | 0.981 | 0.965 | 0.964
0.967 | 0.940 | 0.945
Bickley Jet |  Offline Online | 0.696 | 0.559 | 0.628
0.745 | 0.585 | 0.663

We compare the online algorithm, using Eq. (14), to the offline algorithm,
where we naively compute surrogate eigenvalues of the operator
$\mathcal{A}^{*}\mathcal{A}$ at each snapshot. We compare methods for external
cluster validation, where evaluation is based on comparing the online and
offline clustering to a known true clustering. The true clustering is found by
constructing the operator $\mathcal{A}^{*}\mathcal{A}$ for the true time
parameter $\tau$ and using the $k$-means algorithm for the known number of
clusters over multiple random initializations of cluster centers and averaging
the clusters. The results are shown in Table I. We use the Rand index adjusted
for chance (RI), homogeneity (H), and the V-measure (V). The Rand index
measures similarity of the assignments, homogeneity indicates if each cluster
contains only members of a single class, and V-measure is a combination of
homogeneity and completeness, where completeness indicates if all members of a
given class are assigned to the same cluster. The evaluation metric is found
by comparing clusters from each time step against the true clusters. This is
then averaged across all time steps.

(a)  (b)  (c)  (d)

Figure 5: Comparison of offline and online clustering algorithms for the time-
dependent double gyre flow at time $t=17.6$ in (a) and (b) and Bickley jet at
time $t=39.9$ in (c) and (d). (a) The offline version does not take into
account prior measurements of the double gyre. (b) The online version
incorporates historical data about the flow and consistently computes the true
coherent sets. (c) The offline clustering of the Bickley jet incorrectly
computes the coherent sets, even when close to the true time parameter. (d)
The online method produces robust estimates of the coherent sets. The true
coherent sets associated with the double gyre and Bickley jet almost exactly
align with the coherent sets detected in (b) and (d).

Across all metrics, the online method for the double gyre performs slightly
worse than the offline method. In both the offline and online clustering, the
clustering are qualitatively similar to the true clustering in size and shape.
For the more complex and realistic Bickley jet, the online method outperforms
the offline method across all metrics. In both scenarios, the online method is
less sensitive to noise and invalid clustering, as shown in Fig. 5. The
simulations and experiments conducted in this paper rely on the online method,
which requires less knowledge about the environment and works better in
complicated, real world scenarios due to its incorporation of data over time.

##  IV Environmental Feature Generation for Aerial and Surface Robots
111Accompanying video can be found at https://youtu.be/D-4S6hnFr2E

In this section, we discuss how computing coherent sets in real time
facilitates robots’ awareness. We then demonstrate preliminary ideas as to how
this information can be incorporated into the robots’ decision-making.

###  IV-A Aerial vehicle based crowd monitoring

Robots operating in dense environments, such as urban landscapes or roadways,
rely on persistent monitoring of crowds for scene understanding and safe
navigation. Much of the literature in this area surrounds tracking individual
trajectories of pedestrians or other objects. However, understanding global
information about pedestrian movement allows for greater awareness of high-
level behaviors and facilitates informed planning. With aerial vehicles
monitoring an environment overhead, we can collect trajectory information of
pedestrians, which can be thought of as a flow-like environment [43]. This
information can be used within our framework to elucidate key features, such a
high congregation areas, of the underlying, persistent crowd behaviors.

Overhead data of pedestrians at a train station stop was collected from the
$4$th floor of an hotel in Bahnhofstrasse, Zurich [44]. The original sequence
is processed with a time step of $0.4s$. We found and analyzed the dataset
using tools from OpenTraj [45]. We compute coherent sets using trajectories of
$116$ pedestrians over $51$ time steps, with a $2$-degree polynomial kernel
and $3$-means clusters on $3$ eigenvalues.

Figure 6: Aerial view of pedestrians walking through train station with
coherent sets overlaid. Coherent sets correspond to regions of interest
including train platform, entry areas, and exit areas, indicated by different
colored circles. Regions could be used for decision-making in urban or dense
environments.

In Fig. 6, we see coherent sets illustrate areas of congregation or high
movement for pedestrians. Pedestrian data can be collected in real time to
track coherent sets online, where sets would then be used to navigate through
crowds, such as by avoiding the busy train platform, or deploy sensors for
situational awareness, such as by monitoring the entrance for anomalous
behavior or notable changes in coherence patterns. The regions of interest are
areas where if groups of pedestrians start within a specified region, they
tend to exhibit similar behaviors. The proposed indications of various areas
of interest, such as regions of waiting, entry, and exit, provide a semantic
interpretation of the coherent sets in this specific context.

###  IV-B Energy-efficient trajectories for surface vehicles

Currents in rivers, oceans, and in the atmosphere are other examples of flow
fields. While global information is complex and hard to analyze, local
information does not provide an accurate description of environment features.
Here, we leverage coherent sets from a single gyre to demonstrate the
usefulness of path planning under awareness of these global features.

We create flows and deploy surface vehicles using the multirobot Coherent
Structure Testbed (mCoSTe) which consists of a micro Autonomous Surface
Vehicle (mASV) and a Multi Robot flow tank (MR tank). The MR tank is a
$4.5m\times 3.0m\times 1.2m$ tank of water equipped with an OptiTrack motion
capture system providing localization of the mASV at $120Hz$. The mASV is a
differential drive vehicle with a maximum forward speed of approximately
$0.2m/s$. Two flow driving cylinders placed horizontally and rotating at
approximately $200rpm$ creates a single gyre flow in the MR tank, as shown in
Fig 7.

Figure 7: Experimental test bed for surface vehicle navigation in controlled
flows within a tank. The mCoSTe consists of a mASV and a water tank, where two
flow driving cylinders were placed horizontally in the tank, circled in
orange, to create a single gyre. Figure 8: Comparing trajectories of a robot
operating in a single gyre flow and using a coherent set in decision-making,
with region of entry (blue), region of exit (green), coherent set (pink), and
selected waypoints (stars). The trajectory of the mASV is lightest where it
began and darkest where it ends. (Left) The trajectory of an mASV stuck in the
single gyre while attempting to reach the goal. (Right) The trajectory of an
mASV leveraging the coherent set to move through the environment efficiently
and escape when close to the goal.

We assume one or more robots are collecting velocity or trajectory data. This
may be aerial vehicles collecting overhead imagery of the flow field or
passive gliders directly sampling the velocity. This information is then
processed in real time to compute the coherent sets. Instead of surface
vehicles tracking and modeling individual trajectories within the flow field,
the robot maintains the coherent sets as a global description of the dynamics
of its environment. The coherent sets are calculated using an approximate flow
model for the gyres in the tank for $3591$ simulated particles over $150$ time
steps, where the flow model is a noisy estimate for the true flow induced in
the tank by the flow driving cylinders. Our coherent set algorithm uses a
Gaussian kernel with $\sigma=1.25$ and $3$-means clusters on $3$ eigenvalues.
Then, the sets are used to construct energy-efficient paths. We plan paths
from a specified start location to a goal location by picking waypoints near
coherent sets, exerting control effort to travel to and escape from coherent
sets, and remaining in coherent sets until the next waypoint is close. This is
ideologically similar to the waypoint selection method from [12] using LCS.

The robot navigates the flow induced by a single gyre model and completes its
mission when planning a trajectory based on the coherent set. During
execution, the robot saves energy by turning off its motors when the boundary
of the coherent set is reached and floating through the minimal dispersion
region. However, a boat unaware of the coherent set drives into the gyre,
causing the robot to become stuck and ultimately fail to reach its goal, as
seen in Fig. 8. Modeling coherent sets gives the robot a descriptive framework
to reason about flows in the environment, without maintaining detailed
knowledge.

##  V Conclusion

Robots operating in complex scenarios necessitates greater awareness of their
environment. In flow-like environments, global dynamics may be more useful
than tracking individual trajectories. Transfer operator theory provides a
systematic framework for reasoning about global dynamics. Recent advances in
machine learning and transfer operators combine powerful data representation
with physical system knowledge. Here, we present a framework where coherent
sets, defined by transfer operators, are learned with kernel methods,
estimated online, and integrated into decision-making for robots. We foresee
robotics applications incorporating awareness of environments in novel,
meaningful ways with these connections. For example, our framework allows
scene segmentation according to pedestrian movement instead of static objects
and efficient navigation of strong flows with little information. Future work
includes automated kernel function selection, a distributed framework for
sensor information collected by multi-robot teams, development of more
sophisticated decision-making schemes, and analysis in real environments.

## References

  * [1] D. Kularatne, S. Bhattacharya, and M. A. Hsieh, “Optimal path planning in time-varying flows using adaptive discretization,” _IEEE Robotics and Automation Letters_ , vol. 3, no. 1, pp. 458–465, 1 2018.
  * [2] T. Lolla, M. P. Ueckermann, K. Yiǧit, P. J. Haley, and P. F. Lermusiaux, “Path planning in time dependent flow fields using level set methods,” _Proceedings - IEEE International Conference on Robotics and Automation_ , pp. 166–173, 2012.
  * [3] L. Palmieri, T. P. Kucner, M. Magnusson, A. J. Lilienthal, and K. O. Arras, “Kinodynamic motion planning on Gaussian mixture fields,” in _Proceedings - IEEE International Conference on Robotics and Automation (ICRA)_ , 2017, pp. 6176–6181.
  * [4] H. E. Fiedler, “Coherent structures in turbulent flows,” _Progress in Aerospace Sciences_ , vol. 25, no. 3, pp. 231–269, 1 1988.
  * [5] D. H. Kelley, M. R. Allshouse, and N. T. Ouellette, “Lagrangian coherent structures separate dynamically distinct regions in fluid flows,” _Physical Review E - Statistical, Nonlinear, and Soft Matter Physics_ , vol. 88, no. 1, 2013.
  * [6] P. Tallapragada and S. D. Ross, “A set oriented definition of finite-time Lyapunov exponents and coherent sets,” _Communications in Nonlinear Science and Numerical Simulation_ , 2013.
  * [7] S. C. Shadden and C. A. Taylor, “Characterization of coherent structures in the cardiovascular system,” _Annals of Biomedical Engineering_ , vol. 36, no. 7, pp. 1152–1162, 2008.
  * [8] W. Tang, P. W. Chan, and G. Haller, “Lagrangian Coherent Structure Analysis of Terminal Winds Detected by Lidar. Part II: Structure Evolution and Comparison with Flight Data,” _Journal of Applied Meteorology and Climatology_ , vol. 50, no. 10, pp. 2167–2183, 2011.
  * [9] P. J. Nolan, H. Foroutan, and S. D. Ross, “Pollution transport patterns obtained through generalized lagrangian coherent structures,” _Atmosphere_ , vol. 11, no. 2, p. 14, 2020.
  * [10] T. Inanc, S. C. Shadden, and J. E. Marsden, “Optimal Trajectory Generation in ocean flows,” _Proceedings of the American Control Conference_ , vol. 1, pp. 674–679, 2005.
  * [11] C. R. Heckman, M. A. Hsieh, and I. B. Schwartz, “Controlling basin breakout for robots operating in uncertain flow environments,” in _Springer Tracts in Advanced Robotics_ , 2016.
  * [12] A. G. Ramos, V. J. García-Garrido, A. M. Mancho, S. Wiggins, J. Coca, S. Glenn, O. Schofield, J. Kohut, D. Aragon, J. Kerfoot, T. Haskins, T. Miles, C. Haldeman, N. Strandskov, B. Allsup, C. Jones, and J. Shapiro, “Lagrangian coherent structure assisted path planning for transoceanic autonomous underwater vehicle missions,” _Scientific Reports_ , vol. 8, no. 1, pp. 1–9, 3 2018.
  * [13] M. A. Hsieh, K. Mallory, E. Forgoston, and I. B. Schwartz, “Distributed allocation of mobile sensing agents in geophysical flows,” in _Proceedings of the American Control Conference_ , 2014.
  * [14] C. Wei, H. G. Tanner, X. Yu, and M. A. Hsieh, “Low-range interaction periodic rendezvous along lagrangian coherent structures,” in _Proceedings of the American Control Conference_ , 2019.
  * [15] M. Dellnitz and O. Junge, “On the approximation of complicated dynamical behavior,” _SIAM Journal on Numerical Analysis_ , vol. 36, no. 2, pp. 491–515, 1999.
  * [16] G. Froyland, S. Lloyd, and A. Quas, “Coherent structures and isolated spectrum for Perron–Frobenius cocycles,” _Ergodic Theory and Dynamical Systems_ , vol. 30, no. 3, p. 729–756, 9 2009.
  * [17] G. Froyland, S. Lloyd, and N. Santitissadeekorn, “Coherent sets for nonautonomous dynamical systems,” _Physica D: Nonlinear Phenomena_ , 2010\.
  * [18] G. Froyland, N. Santitissadeekorn, and A. Monahan, “Transport in time-dependent dynamical systems: Finite-time coherent sets,” _Chaos_ , vol. 20, no. 4, p. 043116, 2010.
  * [19] G. Haller, “Lagrangian coherent structures from approximate velocity data,” _Physics of Fluids_ , vol. 14, no. 6, pp. 1851–1861, 2002.
  * [20] M. O. Williams, C. W. Rowley, and I. G. Kevrekidis, “A kernel-based method for data-driven Koopman spectral analysis,” _Journal of Computational Dynamics_ , vol. 2, no. 2, 2015.
  * [21] B. Hamzi, R. Maulik, and H. Owhadi, “Simple, low-cost and accurate data-driven geophysical forecasting with learned kernels,” _Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences_ , vol. 477, no. 2252, p. 20210326, 2021.
  * [22] J. Bouvrie and B. Hamzi, “Kernel methods for the approximation of some key quantities of nonlinear systems,” _Journal of Computational Dynamics_ , vol. 4, no. 1, pp. 1–19, 4 2017.
  * [23] S. Klus, I. Schuster, and K. Muandet, “Eigendecompositions of Transfer Operators in Reproducing Kernel Hilbert Spaces,” _Journal of Nonlinear Science_ , vol. 30, no. 1, pp. 283–315, 2020.
  * [24] S. Klus, B. E. Husic, and M. Mollenhauer, “Kernel canonical correlation analysis approximates operators for the detection of coherent structures in dynamical data,” _arXiv preprint arXiv:1904.07752_ , 2019.
  * [25] N. Takeishi, “Kernel Learning for Data-Driven Spectral Analysis of Koopman Operators,” _Proceedings of Machine Learning Research_ , vol. 101, pp. 956–971, 2019.
  * [26] S. Klus, F. Nüske, and B. Hamzi, “Kernel-based approximation of the koopman generator and Schrödinger operator,” _Entropy_ , vol. 22, no. 7, 2020.
  * [27] I. Abraham, G. De La Torre, and T. D. Murphey, “Model-Based Control Using Koopman Operators,” _Robotics: Science and Systems_ , vol. 13, 9 2017.
  * [28] I. Abraham and T. D. Murphey, “Active Learning of Dynamics for Data-Driven Control Using Koopman Operators,” _IEEE Transactions on Robotics_ , vol. 35, no. 5, pp. 1071–1083, 10 2019.
  * [29] D. Bruder, B. Gillespie, C. David Remy, and R. Vasudevan, “Modeling and Control of Soft Robots Using the Koopman Operator and Model Predictive Control,” in _arXiv preprint arXiv:1902.02827_ , 2019.
  * [30] C. Folkestad, D. Pastor, and J. W. Burdick, “Episodic Koopman Learning of Nonlinear Robot Dynamics with Application to Fast Multirotor Landing,” _Proceedings - IEEE International Conference on Robotics and Automation_ , pp. 9216–9222, 5 2020.
  * [31] I. Mezić, “Spectral properties of dynamical systems, model reduction and decompositions,” _Nonlinear Dynamics_ , vol. 41, no. 1, pp. 309–325, 2005.
  * [32] S. Das and D. Giannakis, “Koopman spectra in reproducing kernel Hilbert spaces,” _Applied and Computational Harmonic Analysis_ , vol. 49, pp. 573–607, 2020.
  * [33] Y. Kawahara, “Dynamic Mode Decomposition with Reproducing Kernels for Koopman Spectral Analysis,” _Advances in Neural Information Processing Systems_ , vol. 29, no. 1, pp. 911–919, 2016.
  * [34] T. Hofmann, B. Schölkopf, and A. J. Smola, “Kernel methods in machine learning,” _Annals of Statistics_ , vol. 36, no. 3, pp. 1171–1220, 2008\.
  * [35] A. Gretton, “Introduction to rkhs, and some simple kernel algorithms,” _Adv. Top. Mach. Learn. Lecture Conducted from University College London_ , vol. 16, pp. 5–3, 2013.
  * [36] ——, “Notes on mean embeddings and covariance operators,” 2015.
  * [37] G. Froyland, “An analytic framework for identifying finite-time coherent sets in time-dependent dynamical systems,” _Physica D: Nonlinear Phenomena_ , vol. 250, pp. 1–19, 2013.
  * [38] R. Banisch and P. Koltai, “Understanding the geometry of transport: Diffusion maps for lagrangian trajectory data unravel coherent sets,” _Chaos_ , vol. 27, no. 3, 2017.
  * [39] E. Forgoston, L. Billings, P. Yecko, and I. B. Schwartz, “Set-based corral control in stochastic dynamical systems: Making almost invariant sets more invariant,” _Chaos_ , 2011.
  * [40] D. Del-Castillo-Negrete and P. J. Morrison, “Chaotic transport by Rossby waves in shear flow,” _Physics of Fluids A_ , vol. 5, no. 4, pp. 948–965, 1992.
  * [41] F. J. Beron-Vera, M. J. Olascoaga, M. G. Brown, H. Koçak, and I. I. Rypina, “Invariant-tori-like Lagrangian coherent structures in geophysical flows,” _Chaos_ , vol. 20, no. 1, 2010.
  * [42] A. Hadjighasem, M. Farazmand, D. Blazevski, G. Froyland, and G. Haller, “A critical comparison of Lagrangian methods for coherent structure detection,” _Chaos_ , vol. 27, no. 5, pp. 1–25, 2017.
  * [43] S. Ali and M. Shah, “A Lagrangian particle dynamics approach for crowd flow segmentation and stability analysis,” _Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition_ , 2007\.
  * [44] S. Pellegrini, A. Ess, K. Schindler, and L. Van Gool, “You’ll never walk alone: Modeling social behavior for multi-target tracking,” in _2009 IEEE 12th International Conference on Computer Vision_. IEEE, 2009, pp. 261–268.
  * [45] J. Amirian, B. Zhang, F. V. Castro, J. J. Baldelomar, J.-B. Hayet, and J. Pettre, “OpenTraj: Assessing Prediction Complexity in Human Trajectories Datasets,” in _Asian Conference on Computer Vision (ACCV)_ , no. CONF. Springer, 2020.