# Conjunct Resolution in the Face of Verbal Omissions

Royi Rassin1 Yoav Goldberg1,2 Reut Tsarfaty1
1Bar-Ilan University 2Allen Institute for Artificial Intelligence
{rassinroyi, yoav.goldberg, reut.tsarfaty} @gmail.com

###### Abstract

Verbal omissions are complex syntactic phenomena in VP coordination
structures. They occur when verbs and (some of) their arguments are omitted
from subsequent clauses after being explicitly stated in an initial clause.
Recovering these omitted elements is necessary for accurate interpretation of
the sentence, and while humans easily and intuitively fill in the missing
information, state-of-the-art models continue to struggle with this task.
Previous work is limited to small-scale datasets, synthetic data creation
methods, and to resolution methods in the dependency-graph level. In this work
we propose a conjunct resolution task that operates directly on the text and
makes use of a split-and-rephrase paradigm in order to recover the missing
elements in the coordination structure. To this end, we first formulate a
pragmatic framework of verbal omissions which describes the different types of
omissions, and develop an automatic scalable collection method. Based on this
method, we curate a large dataset, containing over 10K examples of naturally-
occurring verbal omissions with crowd-sourced annotations of the resolved
conjuncts. We train various neural baselines for this task, and show that
while our best method obtains decent performance, it leaves ample space for
improvement. We propose our dataset, metrics and models as a starting point
for future research on this topic.

##  1 Introduction

Figure 1: A demonstration of the conjunct resolution task. At the top, the
input sentence with the omitted words in bold and the coordinating element is
underlined. Below, the expected rewrite, consisting of a single fully-
specified standalone statement for each of the conjuncts. The rewrite must not
include the coordinating element.

Natural language is economic, and many elements in the message are not spelled
out but omitted by speakers, left for the receiver of the message to complete.
The hearer, either a human or an algorithm, should then complete the missing
information and recover the intended meaning. This kind of omission and
recovery occurs at all levels of conversation, from syntax to pragmatics, and
is performed naturally and intuitively by humans, to the extent that they
often don’t even realize that something was missing in the message they
received.

An important class of omission phenomena — and the focus of this work —
involves verbs and their arguments, especially around coordination structures.
A verb and some of its arguments that appear in an initial clause may be
omitted in subsequent clauses. For example, in the sentence “Josh likes wine
and Jane water”, the verb _likes_ is omitted from the phrase “Jane likes
water” and has to be inferred. We find that state-of-the-art syntactic parsers
(Honnibal and Montani, 2017; Nguyen et al., 2021) fail consistently even on
toy examples such as this one, and assign a structure in which “Jane water” is
interpreted as a noun-compound which is the object of Josh’s liking, together
with water (Figure 2).

[column sep=0.5em] Josh & likes & wine & and & Jane & water.
[edge height=1.5ex]21nsubj [edge height=1.5ex]23dobj [edge height=3.5ex]34cc
[edge height=5ex]36conj [edge height=1.5ex]65compound

Figure 2: Compound relation indicating a misinterpretation of the sentence
"Josh likes wine and Jane water" instead of "Josh likes wine and Jane likes
water"

Downstream applications, such as, Google
translate111https://translate.google.com/, accessed on Jan 18 2023. also fail.
Google translate translates the above sentence to French as “Josh aime le vin
et l’eau de Jane”. When back-translated to English using the same engine, it
results in “Josh likes Jane’s wine and water”. Significantly more complex
sentences involving verbal omissions are of course also possible, and NLP
systems routinely fail around them.

Linguistically, such verbal omissions are studied under the terms _ellipsis_
and _gapping_ , and the research broadly aims at categorizing the verbal
omission cases into sub-categories, carefully documenting the condition under
which different omissions can or cannot occur (Hudson, 1989; Ross, 2014;
Jackendoff, 1971). For language technology purposes, however, we would like to
focus on detection, resolution and usability, not categorization. The need to
recover empty elements automatically and robustly in large-scale calls for a
unifying approach, by which to consider the different cases of verbal
omissions as instances of a single phenomenon, that can be effectively
addressed by a broad-coverage NLP system.

Due to failures of even language technology applications on such
constructions, in this paper, we suggest that these verbal omission phenomena
should be addressed explicitly by systems, rather than hoping it will be
caught on as a side product of end-to-end neural training. To this end, our
aim is to establish a verbal-omission recovery task, and to create a
supporting large-scale corpus documenting such verbal omissions in naturally
occurring English sentences, together with automatic annotation models that
recover the implicit information and make it explicit.

How should a system that resolves such missing information look like? What are
its outputs? In the syntactic parsing and treebanking literature, there has
been an ongoing debate regarding the best way to represent recovery of
omissions in coordinated structures (Marcus et al., 1999; Nivre et al., 2020;
Schuster et al., 2017; Hudson, 1973; Nielsen, 2004; Anand and Hardt, 2016;
Park and Kang, 2007; Park, 2009; Ficler and Goldberg, 2016; Kato and
Matsubara, 2020; Droganova et al., 2018b, a).

However, none of the solutions are fully satisfactory. First, all of them are
highly technical in nature, and require significant linguistic expertise in
order to even understand the notation. Often they assume some formal tree or
graph notation, which makes their adoption unlikely by people who work on NLP
systems but who may lack the specific syntactic expertise. Moreover, how can
one feed such theory-loaded graph-based annotation into NLP systems for
downstream, user-facing tasks?

To mitigate this, our proposed solution is strictly within the text-to-text
paradigm, and involves a re-writing task where the input is a sentence
exhibiting a coordinated structure, and the output is the set of sentences
which contain the same information, but where the implicit information is made
explicit. We illustrate an input and output example in Figure 1. As we further
discuss below, this text-enrichment approach has several appealing properties:
first, it is natural and easily comprehensible to any competent speaker of the
language. Second, this caters for both large-scale annotation efforts and task
adoption by potential users. Lastly, it produces an output which can trivially
be fed into any language processing system which takes natural language text
as input.

We collect a corpus of 10,206 sentences involving a wide range of verbal
omissions in coordinated structures, annotated via this text-rewriting task to
recover the missing elements. Using this corpus, we conduct experiments on the
omission-recovery task. Training a T5-based model to perform the task shows
that the performance saturates after seeing only 10% of the data, but the
accuracy of the neural model is far from being perfect, leaving ample room for
future modeling and improvements.222We make our code and dataset publicly
available at https://github.com/RoyiRa/conjunct-resolution-task

##  2 Related Work

Previous research on verbal omissions in coordination structures has
classified such phenomena into (at least) six categories: (1) conjunction
reduction (Hudson, 1973); (2) gapping (Ross, 2014; Jackendoff, 1971; Hudson,
1989); (3) VP Ellipsis (Nielsen, 2004); (4) sluicing (Anand and Hardt, 2016;
Park and Kang, 2007); (5) pseudo-gapping (Park, 2009); and (6) argument
clusters. All of these phenomena are forms of ellipsis and are considered as
ways to use language efficiently.

How do NLP systems cope with such complex syntactic phenomena? Work around
gapping in the syntactic parsing community has focused on representation
schemes for this phenomena in dependency graphs. Various representation has
been proposed. For instance, the Universal Dependencies (UD) framework (Nivre
et al., 2020) introduces the concept of an "orphan" dependency to indicate the
presence of an ellipsis, and (Schuster et al., 2017) provides a detailed
analysis of how gapping constructions could be represented using UD. Related
work (Schuster et al., 2018; Droganova et al., 2018a, b) utilize such schemes
by promoting a new head of the clause in cases of gapping and attaching all
remnants to it.

However, having a representation schema for a construction does not mean that
an automatic parser is able to accurately predict it. To bridge this gap (no
pun intended), several efforts have been made to increase the training data
size for these constructions by enriching existing data or creating artificial
datasets. Droganova et al. (2018b) proposed data enrichment methods that
utilize existing annotated parse trees to mimic the structure of elliptical
constructions, and Droganova et al. (2018a) trained a parser on artificial
elliptical treebanks, achieving an F1 score of 36% on a small dataset (an
improvement relative to prior work). Moreover, Schuster et al. (2018); Kato
and Matsubara (2020) proposed a reconstruction algorithm at the dependency
graph level, but relied on an oracle to identify the gapped sentences.

Despite these efforts, current approaches do not fully address the issue of
verbal omissions in coordination structures, and remain scattered.
Additionally, these methods are not suitable for large-scale applications and
are not ready for use in downstream tasks. In this work we take a more
realistic, consistent and unified approach, wherein all verbal omissions are
treated within the same framework, for which we provide a gold benchmark, a
crowdsourcing interface, models for the tasks demonstrating feasibility and
efficacy.

##  3 The Conjunct Resolution Task

###  3.1 Desiderata

We seek a unifying approach to handle many types of verb-related omissions in
coordination structures, and which is targeted primarily at users of language
technologies. Concretely, our approach should allow for (1) an annotation
scheme that is scalable to multiple annotators while maintaining quality; (2)
a comprehensible task to non-linguists that is simple enough that no
specialized linguistic expertise is required by a user of the resulting
annotations; (3) amenable to automatic annotation by models; (4) useful in the
context of a downstream language processing applications. Additionally, it
would be preferable if the approach is language-agnostic, and that the
approach is not constrained by any particular linguistic theory.

###  3.2 The Task

In order to meet the aforementioned desiderata (Section 3.1), our approach
steps away from traditional syntactic representation and instead represents
the output as natural language text, which is an enriched version of the input
text. This offers several advantages. First, the resulting annotation task is
intuitive for annotators as they now need to read a sentence and rewrite it,
which is a natural and familiar setting; second, language models are designed
to learn from, operate on, and produce natural text representation; finally,
the output can be consumed by any process that takes text as input, so
applications can benefit from the enrichment without requiring a change in
their design.

Intuitively, such a task could be to take a sentence with missing information,
and rewrite it by completing in all the missing verbs and their arguments,
e.g. rewriting “Josh likes wine and Jane water” to “Josh likes wine and Jane
likes water”. However, we found this task to be notoriously hard to explain
both to non-linguists (who did not realize a verb was missing in this sentence
in the first place) and to linguists (who also do not identify some of the
verbs as “missing”, for example in “Jane likes wine and water”, which is not
technically a gapping construction but a verb taking a coordinated NP as its
object — but which we aim to reconstruct as two distinct conjuncts333In this
work we use the word “conjuncts” to mean expressions linked by the
conjunctions “and”, “or”, or “but”. nonetheless).

Instead, we build on the split-and-rephrase paradigm (Narayan et al., 2017),
which involves breaking down a complex sentence into smaller, simpler
sentences. Specifically, we propose to decompose sentences that potentially
involve verbal omissions around coordination into a set of independent
sentences, that together capture the meaning of the original sentence, and do
not add to it. In contrast to the original split-and-rephrase work, where no
information is actually missing,

the rewritten sentences make the implicit arguments explicit in each conjunct,
and when they are taken together, these sentences retain the meaning of the
original complex sentence.

Concretely, we define the conjunct resolution task as follows: given a
sentence containing one of the conjunctions “or", “and", or “but" as input,
the sentence has to be rewritten into a set of sentences while adhering to the
following constraints:

  * (1) the set of sentences must not include the marked conjunction;
(2) the sentences should introduce a minimal number of new content words;
(3) the sentences set should preserve the meaning of the original sentence and
not add to it;

If it is not possible to rewrite the sentence under the preceding constraints
without altering its meaning, the sentence should be left unchanged.

The first constraint drives the annotation: by not allowing to use the
conjunction, the sentence must be split, and all the verbs and their arguments
must be spelled out. The two other constraints keep the resulting sentence set
both minimal and complete.

To illustrate our task, consider the following sentence. The underlines
indicate omitted elements, and are not part of the input. The focused
conjunction "and" is marked in bold:

  * •

“ _As of January 2013, The Times has a circulation of 399,339, The Sunday
Times __ of 885,612, and The New York Times __ of 9,512,132._”

The sentence is rewritten into a set of three sentences as each clause
describes a unique event:

  1. (1)

As of January 2013, The Times has a circulation of 399,339.

  2. (2)

As of January 2013, The Sunday Times has a circulation of 885,612.

  3. (3)

As of January 2013, The New York Times has a circulation of 9,512,132.

A core challenge of this task is to rewrite the sentence to a correct number
of sentences while faithfully retaining the meaning of each clause (for
instance, including the opening span "As of January 2013," is crucial in
retaining the overall meaning of the sentence, however, it should not be a
sentence on its own, as it is not a coordinated clause).

The closing clause refers to sentences that can not be rewritten to
independent sentences, but still contain verbal omissions. For instance,
consider

  * •

“ _Amla made 133 and Roussow __ 132 with the pair combining to put on 247 for
the third wicket._”

while this is indeed a case of verbal omissions, the span “ _with the pair
combining to put on 247_ ” binds the two clauses together in a way that will
lose its meaning if rewritten to a set of two sentences.

This paradigm fits our objective well and addresses the desiderata we put
forth in Section 3.1, as rewriting a sentence to a set of sentences indirectly
resolves the verbal omissions. It is amicable to non-linguist annotators and
straightforward to scale while maintaining quality, as there is little room
for variance when breaking down a complex sentence. Finally, users of the
process can clearly and intuitively understand its intended behavior and can
analyze its correctness, without requiring any linguistic training.

##  4 Data Collection Process

We collect a dataset of 10,206 examples of a wide array of omission cases,
which can serve for both training and evaluation. We aim for the collected
sentences to cover a wide range of omission cases. The underlying data was
sourced from three publicly available datasets: SQuAD 2.0, Dailymail, and CNN
(Rajpurkar et al., 2018; Hermann et al., 2015), with a roughly equal
proportion of sentences from each one. Our proposed collection protocol
involves two steps: (1) automatic collection of sentences that are likely to
contain interesting omission phenomena; (2) manual annotation of the sentences
via crowd-sourcing.

###  4.1 Sentences Collection

Instead of trying to explicitly target and identify specific types of verbal
omissions, we instead rely on the observation that these verbal omission
constructions affect both manual and automatic syntactic analysis in various
ways, either due to constructions that are hard or impossible to represent, or
due to parsing mistakes. We thus do not look directly for specific verbal
omissions, but rather, identify their side-effects as manifested in the graph
outputs of a dependency parser. For example, here:

[column sep=0.5em] 32% & had & brown & and & 21% & black.
[edge height=1.5ex]21nsubj [edge height=1.5ex]23dobj [edge height=3.5ex]24cc
[edge height=5.0ex]26conj [edge height=1.5ex]65nsubj the omission of _had_
manifests in two “suspicious” structures: a conj relation between two
different part-of-speech tags, and an nsubj dependent of a word which is not a
verb.

We collect cases based on 21 such patterns, applied to sentences that include
coordination.

By considering a sample of the sentences identified in this manner, we
verified that roughly 92% of the resulting cases are indeed non-trivial verbal
omission cases.

###  4.2 Annotation and Curation

We devise a scalable annotation procedure that can be performed by non-expert
annotators.444We rely on a pool of trained crowd workers in the controlled
crowd-sourcing paradigm Roit et al. (2019). The procedure is based on the
conjunction resolution task (Section 3), which does not require annotators to
possess advanced linguistic knowledge. Instead, it relies on their intuitive
understanding of language.

##### Crowdsourcing Infrastructure.

We set up an Amazon Mechanical Turk (AMT) task in which workers were given a
coordination structure with suspected omissions and a highlighted conjunction,
and were asked to rewrite the sentence into multiple independent sentences,
according to our task’s rules. Each AMT assignment (known as a "HIT") begins
with a brief description of the task and two examples: one example of a
rewritable sentence and the other of a not rewritable sentence. Workers were
also given an option to view five rewritable and five non-rewritable examples
with detailed explanations.

A HIT consisted of seven pairs consisting of a coordination sentence and a
highlighted conjunction in it. The annotators were requested to rewrite the
sentences in the order in which the clauses are read in the sentence. We
encouraged annotators to review their work by joining the set of sentences
with the highlighted conjunction and comparing the meaning between the input
and the sum of their sentences. Moreover, when annotators submitted a sentence
unchanged, they were prompted to explain why. This not only facilitated
critical thinking on their part, but also allowed for potential revisions to
their annotation and provided valuable insight on the data, being a useful
resource on its own.

To ensure the quality of the annotations, we implemented several checks: (1)
We ensured that no two sentences in the rewritten set were identical. (2) We
verified that the highlighted conjunction was not present in any of the
sentences in the set. (3) We confirmed that no new content words were added,
while still allowing for inflectional variations in verb and noun forms to
maintain grammatical accuracy. Additionally, to handle unexpected cases and
gain further understanding of the task, annotators were given the option to
indicate uncertainty in their annotation and to specify if the sentence was a
“long list” requiring more than ten rewrites. In case of the latter, sentences
were removed. Annotators were also given the option to provide any feedback.
section A.3 shows the user interface of this task.

##### Annotations Consolidation.

The final annotations were determined by majority agreement among annotators
on the number of sentences in a set and the exact match for each submission.
In cases where no majority agreement was reached, the answer provided by the
highest-performing annotator was chosen.

##### Inter-Annotator Agreement.

We assessed the level of unanimous agreement on factors such as rewrite
agreement (the number of sentences required to accurately rewrite a given
sentence), exact match, and average Jaccard Similarity555Jaccard Similarity is
defined as the size of the intersection of the sets divided by the size of the
union of the sets.. The initial annotation phase involved 64 native English
speakers with a high approval rate (99%) and significant experience on the AMT
platform (over 5,000 completed HITs). Our analysis of the first 10% of the
data revealed rewrite agreement, exact match, and average Jaccard Similarity
scores of 56%, 67.5%, and 94%, respectively, with approximately 5% of the data
being unusable due to corrupted annotations. In order to continually improve
the quality of our annotations, we narrowed our pool of annotators to the top
five performers (based on activity and IAA performance) and provided personal
feedback and bonuses based on the execution of each batch. As a result of
these efforts, unanimous rewrite agreement, exact match, and average Jaccard
Similarity all increased to 85%, 82%, and 97%, respectively, and less than 1%
of the data required corrections.

##  5 Conjunct Resolution Dataset

Our Conjunct Resolution dataset consists of 10,206 verbal omission sentences,
each paired with one of the conjunctions: "and", "or", and "but" (table 2
reports conjunction distribution) coupled with human annotations. By
subtracting the number of verbs in the verbal omissions to those in the gold
annotations, we find that 42% of the verbs are omitted (see table 1).
Furthermore, the majority (95.2%) of sentences were found to be rewritable,
with 82% of the sentences being expressed in two sentences, 9.8% being
expressed in three sentences, 3.4% expressed as four sentences or more, and
only 4.8% being classified as not rewritable.

Split | Explicit | Omitted | Total
---|---|---|---
Train | 29,447 | 21,355 | 50,802
Validation | 3,630 | 2,631 | 6,261
Test | 3,611 | 2,517 | 6,128
Full | 36,688 | 26,502 | 63,190
Table 1: Count of explicit and omitted verbs in each split of the dataset, and the total count for each split. The full dataset contains a total of 10,206 instances Split | and | or | but | Total
---|---|---|---|---
Train | 6,508 | 798 | 860 | 8,166
Validation | 805 | 108 | 108 | 1,021
Test | 811 | 90 | 118 | 1,019
Full | 8,124 | 996 | 1,086 | 10,206
Table 2: Distribution of conjunctions (and, or, but) in each split of the
dataset, and the total count for each split. The full dataset contains a total
of 10,206 instances

##### Non-rewritable Sentences.

491 of the 10,206 (4.9%) were marked as non-rewritable. Of these, 445 contain
an explanation.666We did not collect explanations during the first few
batches. The reasoning behind deciding whether a sentence is rewritable or not
seems to be non-trivial. For instance, consider the following two sentence:

  1. (1)

I’d say Adam will win four majors and Justin __ three, but I wouldn’t be
surprised if it was the other way round.

  2. (2)

The pair are tied at the top after McIlroy shot 67 – his 25th score under par
out of his last 27 rounds - and Horschel __ a 69.

Despite that both sentences are cases of gapping, only the second is
rewritable. To recognize this, the reader (human or model) needs to be able to
identify when two events are bound together by another piece of information.
In the first sentence, the phrase "but I wouldn’t be surprised if it was the
other way round" lacks context when appearing in the rewritten sentences.
However, for the second sentence, there is no such issue, and is thus
rewritable.777In linguistics and formal semantics, when a coordinated
structure refers to the plurality of events as a whole, it is said to have a
collective (as opposed to distributive) reading. The non-rewritable sentences
in our set are those with collective readings. Their annotation and resolution
is beyond the scope of this paper, and we reserve them for future work.

##  6 Evaluation Metric

As no task is complete without an evaluation metric, we propose an automatic
evaluation metric for the proposed conjunct resolution as rephrasing task. For
evaluation, we are interested in measuring three things: (1) how accurate the
model is at resolving verbal omissions, (2) how often does the model omit
other information after resolution, and (3) how often the model generates
extra information. To address these three criteria, we propose to measure
recall and precision over the _predicate-argument relations_ recovered by a
dependency parser on the generated compared to the gold sentence set.

##### High Level Description.

The task revolves around making verbs and their arguments explicit. Our main
object of interest is thus the “verb nucleus”, an instance comprising of a
verb and its arguments, as reflected in the dependency tree. We measure to
what extent the nuclei extracted from the generated sentences overlap with the
nuclei extracted from the gold sentences. Neglecting to resolve an argument,
or adding an extra argument to a given verb, will result in a mismatch between
the gold and generated nucleus of that verb, hurting both recall and
precision. Neglecting to spell out a verb completely will result in a missing
nucleus (recall error), and over generating will result in spurious nuclei
(precision error).

To calculate these metrics, we first produce dependency graphs for both the
model’s generated set of sentences and the gold annotation’s set of sentences.
From these graphs, we extract the verbs, and for each verb a subset of its
dependents that we consider as arguments (based on a set of dependency
labels). Each such set of verb+argument is a “verb nucleus”. We treat the
collection of verb nuclei over all sentences as a set (each element in the set
is a collection of verb+arguments), and compute precision and recall over this
set.

##### Details.

Denote the (automatically produced) syntactic dependency graphs of the $m$
gold sentences as $G={\\{g_{1},g_{2},\text{\ldots},g_{m}\\}}$, and the
dependency graphs of the $n$ generated sentences as
$H={\\{h_{1},h_{2},\dots,h_{n}\\}}$. We extract verb nuclei from these graphs,
where each nucleus is a sub-graph containing a verb, its subject, object and
lexicalized prepositional modifiers, as well as prepositional modifiers of the
object and associated negations, if they exist.888The prepositional modifiers
of the object are needed to handle pp-attachment errors of the parser. We
refer the reader to section A.5 for a comprehensive account of the parser and
included arguments. We represent a nucleus as a bag of
$(w_{1},\textit{dep},w_{2})$ triplets where $w_{1}$ and $w_{2}$ are words and
dep is a dependency label, and consider two nuclei to be the same if their
bags are the same. Denote by $N_{G}$ the bag of gold nuclei, by $N_{H}$ the
bag of generated nuclei, and by $N_{I}$ the bag of identity nuclei, obtained
by extracting nuclei from the input sentence. We obtain the subsets
${N^{\prime}}_{H}=N_{H}\setminus N_{I}$ and ${N^{\prime}}_{G}=N_{G}\setminus
N_{I}$, which strictly contain nuclei with omitted verbs. Then
$\textit{precision}=|{N^{\prime}}_{H}\cap{N^{\prime}}_{G}|\setminus|{N^{\prime}}_{H}|$
and
$\textit{recall}=|{N^{\prime}}_{H}\cap{N^{\prime}}_{G}|\setminus|{N^{\prime}}_{G}|$.
In cases where there is only one sentence in the gold set and the generated
set, we skip the interaction with the identity nucleus, as to not punish the
model for correctly not rewriting.

##### Calibration.

The input sentence contains some verbs and arguments that are repeated
throughout the sentence. Thus, an approach that copies the full sentence two
or more times will also yield some success under our metric, due to repeated
verb nuclei, despite not being meaningful. To calibrate for this, we provide
two additional numbers: (a) the precision and recall obtained by a model that
spits the original sentence as output, unmodified; and (b) the precision and
recall of a model that has access to the correct number $k$ of sentences in a
gold annotation set, and which uses this information by spitting out $k$
copies of the input sentence. Model performance should always be judged in
comparison to these baselines.

Figure 3: A distribution over the number of omitted arguments in a verb
nucleus

##  7 Experiments

We evaluate neural models on the task, both in supervised fine-tuning and in
in-context learning setups. For the supervised fine-tuning case, we measure
both task performance as well as the dependence of performance on dataset
size. As a concluding experiment, we take the best-performing model and
manually evaluate it against the task definition.

##### Dataset Split and Preprocessing.

We shuffle and then split our dataset to train (80%), validation (10%), and
test (10%) sets. Each input instance contains a sentence and a marked
conjunction. Each output instance is a sequence of sentences. For feeding the
text to the models, we mark the conjunction using a special token999In T5,
sentinel tokens were employed, and in GPT3, the ”¡SPLIT¿” marker was utilized.
and separate the output sentences using special tokens.101010In T5, sentinel
tokens were employed again, and in GPT3, each sentence was written in a
separate line.

##### Supervised Fine-tuning.

We fine-tune ten T5-large (Raffel et al., 2020) models, using increasingly
larger subsets of the training data, from 10% to 100% in increments of ten.
All models were trained with the same hyperparameters and fine-tuned for five
epochs, with the best performing model on the validation set being saved and
subsequently evaluated on the test-set (for the detailed training
configuration, see A.2).

##### In-context Learning / Prompting.

We evaluate the state-of-the-art GPT text-davinci-3 model from OpenAI, in an
in-context learning (prompting) fashion (Brown et al., 2020).

To obtain in-context examples, we randomly sampled for each test instance
three re-writable and one non-rewritable sentence, sharing the conjunction
with the test instance. Further details of the prompt and parameters are
available in the appendix A.2.

##### Manual Evaluation.

Our evaluation metric cannot fully evaluate the semantic correctness of
results. To overcome this, we perform a manual evaluation in which a human
annotator is requested to examine the system’s inputs (sentences containing
omissions) together with outputs of the best performing-model, and assess
whether the generated set of sentences has the same meaning as the input
sentence, or a different one.

###  7.1 Results and Discussion

The results of all models are summarized in table 3. Results improve rapidly,
but then saturate on around 82% F1 already with 40% ($\sim 3,200$) training
samples, reaching a peak of 82.4% F1, indicating that the key to the task may
not be “more data”.

In our experiments, the $T5_{40\%}$ and $T5_{80\%}$ models demonstrated the
best performance. However, the $T5_{80\%}$ model had a more balanced
performance across the different metrics, making it the preferred model for
further analysis and interpretation of results. When examining performance on
specific conjunctions, the best performing model, $T5_{80\%}$, scored 83.7% on
“and” sentences, 76.3% on “or” sentences, and on “but” sentences, it scored
77.7% F1. $GPT3$ performed similarly, but to a lesser extent, scoring 73.5%,
70.4%, and 65.4% F1 on average, in the aforementioned order.

In terms of quantity, at a minimum, models learned that the resolution is
centered around the verb. $T5_{80\%}$ generates 5,514 out of the 6,128 (89.9%)
verbs in the gold annotations, while only over-generating 349 verbs, a mere
5.7% of the total verbs in the test set. Similarly, $GPT3$ generates 4,680 out
of the 6,128 (75%) and over-generates 273 verbs (4.4%).

Finally, the manual evaluation evaluating the semantic correctness of the
$T5_{80\%}$ system’s outputs with respect to the input, reveals that in 87.9%
of the cases the meaning is preserved, and in 12.1% it is not. Although
measuring different aspects of the answer, these numbers are similar to the
automatic F1 results, establishing some additional trust in it as an automated
metric.

Model | Recall | Precision | F1
---|---|---|---
Calibration1 | 5.1 | 5.1 | 5.1
Calibrationk | 49.8 | 41.8 | 45.5
$T5_{10\%}$ | 75.9 | 43.2 | 55.1
$T5_{20\%}$ | 77.2 | 78.2 | 77.7
$T5_{30\%}$ | 79.5 | 79.6 | 79.5
$T5_{40\%}$ | 82.4 | 81.8 | 82.1
$T5_{50\%}$ | 81.6 | 82.1 | 81.8
$T5_{60\%}$ | 81.2 | 81.5 | 81.3
$T5_{70\%}$ | 81 | 82.2 | 81.6
$T5_{80\%}$ | 82 | 82.7 | 82.3
$T5_{90\%}$ | 81.2 | 82.1 | 81.6
$T5_{100\%}$ | 81.8 | 82.7 | 82.2
$GPT3$ | 68.7 | 76.2 | 72.3
Table 3: Comparison of model performance in terms of recall, precision, and F1
score on the test set. The T5 models trained on 40% of the training set or
higher generally perform similarly, with $T5_{80\%}$ achieving the highest
overall F1 score of 82.3. The GPT3 model has a lower overall performance, with
an F1 score of 72.3. Calibration1 and Calibrationk are the results of models
that output back the input sentence, or $k$ instances of the input sentence,
where $k$ is the number of sub-sentences in the rewritten text.

###  7.2 Error Examples

Below are some examples of errors by the best-performing T5 model.

Attributing the correct arguments to a different verb. Here, the gold
annotation indicates the verb “playing” as the omission, while the model
wrongly chose “gain” (crossed items are missing in output):

Players _gain_ points from _playing_ in the four grand slams, ATP World Tour
events including eight Masters Series tournaments, and World Group singles
matches in the Davis Cup teams competition.

  1. (a) Players gain points from playing in the four grand slams.
(b) Players gain points from playing in ATP World Tour events including eight
Masters Series tournaments.
(c) Players gain points from playing in World Group singles matches in the
Davis Cup teams competition.

Incorrectly attaching two arguments to the same verb. Here the model should
have produced a different sentence for “over 300,000 injured”:

The January 12 quake left over 220,000 dead, over 300,000 injured, and over a
million homeless.

  1. (a) The January 12 quake left over 220,000 dead, over 300,000 injured.
(b) The January 12 quake left over a million homeless.

Failure to resolve verb-phrase ellipsis.

Germany knows how to do it, and we don’t.

  1. (a) Germany knows how to do it.
(b) We don’t know how to do it.

Considering a different coordination. Here, the model split around ‘;’ rather
than around ‘and’, missing information in the process:

Johnston’s one shot went through her front door and over the officers’ heads;
they responded with 39 shots, hitting Johnston five times.

  1. (a) Johnston’s one shot went through her front door __.
(b) the officers responded with 39 shots, hitting Johnston five times.

Missing conjuncts from the resolution. Here, the crossed item did not appear
in the model’s output, but should have:

At the 2001 Census, 92.4% of the city’s populace was White—including one
percent White Irish—3.8% were South Asian, 1.0% Black, 1.3% Chinese or other
ethnic groups, and 1.5% were of Mixed Race.

  1. (a) At the 2001 Census, 92.4 % of the city’s populace was White—including one percent White Irish.
(b) At the 2001 Census, 3.8 % of the city’s populace was South Asian.
(c) At the 2001 Census, 1.0 % of the city’s populace was Black.
(d) At the 2001 Census, 1.3 % of the city’s populace was Chinese or other
ethnic groups.
(e) At the 2001 Census, 1.5 % of the city’s populace was of Mixed Race.

Resolving creates a factual inaccuracy. Here, the model splits a sentence to
create factually incorrect sentences:

Heist had been sentenced to three years of probation for the identity theft
and for giving false information to a law enforcement officer.

  1. (a) Heist had been sentenced to three years of probation for the identity theft.
(b) Heist had been sentenced to three years of probation for giving false
information to a law enforcement officer.

This error example illustrates an inherent limitation of our approach, as the
correct semantics cannot be represented as a set of sentences. The correct
behavior under our representation would have been to not split this sentence
at all.

##  8 Conclusions

We present a novel approach for studying verbal omissions in coordination
structures. Previous research in this area has been fragmented, focusing on
individual phenomena. In contrast, we propose a unified approach which
considers all conjunction related verbal omissions under the same framework,
by introducing a text-to-text conjunct-resolution task, to resolve omitted
verbs and their arguments. We compiled and curated a large dataset of
conjunction related verbal omissions, consisting of over 10,000 sentences and
human annotations, which serves as a valuable resource for further research in
this area. Our results using state-of-the-art models as neural baselines
demonstrate that this task is challenging and merits further work.

## Limitations

One unsatisfying aspect of proposed task is that it accounts for distributive
coordination structures, but is not able to handle sentences with collective
reading where the main predicate applies to the plurality of conjuncts as a
whole. In our data collection these account for about 4.9% of the verbal
omission cases, and such sentences are left “non-rewritable”. In future work,
we would like a solution that allows to resolve also such sentences in a
consistent yet easy-to-annotate manner.

Additionally, in the GPT prompting experiment we experimented with a few
different prompts, but did not do exhaustive prompt engineering, and it is
possible that with more aggressive prompt engineering GPT can perform better
on the task than our results indicate. Similarly for the fine-tuning
experiments with T5-large, in which we did some hyperparameter tuning, but not
aggressively so.

## Ethics Statement

##### Worker Qualification and Compensation for Annotation.

To collect annotations on our dataset, we used Amazon Mechanical Turk (AMT).
All workers had the following qualifications: (1) over 5,000 completed HITs;
(2) 99% approval rate or higher; (3) Native English speakers from England, New
Zealand, Canada, Australia, or United States. Workers were paid $0.75 per HIT,
and on average completed a batch within four hours of work. In addition, $10
was given upon completing a batch (73 HITs), raising the hourly pay to $16.2.

##### Data Collection and Usage Policy for Annotation.

Workers were informed that their annotations would be collected for research
purposes and would be used to train and evaluate language-related models, and
that the annotations would eventually be made publicly available.
Additionally, our task and the annotations collected were of objective nature
and did not contain any personal information. Furthermore, all data sources
used in the study were publicly available.

##  9 Acknowledgements

This project received funding from the Europoean Research Council (ERC) under
the Europoean Union’s Horizon 2020 research and innovation programme, grant
agreement No. 802774 (iEXTRACT) and grant agreement No. 677352 (NLPRO). The
third author is also funded by a grant from the Israeli Ministry of Science
and Technology (MOST), grant number 3-17992.

## References

  * Anand and Hardt (2016) Pranav Anand and Daniel Hardt. 2016.  Antecedent selection for sluicing: Structure and content.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_ , pages 1234–1243, Austin, Texas. Association for Computational Linguistics.
  * Brown et al. (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language models are few-shot learners.
  * Droganova et al. (2018a) Kira Droganova, Filip Ginter, Jenna Kanerva, and Daniel Zeman. 2018a.  Mind the gap: Data enrichment in dependency parsing of elliptical constructions.  In _Proceedings of the Second Workshop on Universal Dependencies (UDW 2018)_ , pages 47–54, Brussels, Belgium. Association for Computational Linguistics.
  * Droganova et al. (2018b) Kira Droganova, Daniel Zeman, Jenna Kanerva, and Filip Ginter. 2018b.  Parse me if you can: Artificial treebanks for parsing experiments on elliptical constructions.  In _Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)_ , Miyazaki, Japan. European Language Resources Association (ELRA).
  * Ficler and Goldberg (2016) Jessica Ficler and Yoav Goldberg. 2016.  Improved parsing for argument-clusters coordination.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 72–76, Berlin, Germany. Association for Computational Linguistics.
  * Hermann et al. (2015) Karl Moritz Hermann, Tomás Kociský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015.  Teaching machines to read and comprehend.  _CoRR_ , abs/1506.03340.
  * Honnibal and Montani (2017) Matthew Honnibal and Ines Montani. 2017.  spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing.  To appear.
  * Hudson (1973) R. A. Hudson. 1973.  Conjunction-reduction.  _Journal of Linguistics_ , 9(2):303–305.
  * Hudson (1989) Richard A Hudson. 1989.  Gapping and grammatical relations.  _Journal of Linguistics_ , 25:57–94.
  * Jackendoff (1971) Ray S Jackendoff. 1971.  Gapping and related rules.  _Linguistic inquiry_ , 2(1):21–35.
  * Kato and Matsubara (2020) Yoshihide Kato and Shigeki Matsubara. 2020.  Parsing gapping constructions based on grammatical and semantic roles.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 2747–2752, Online. Association for Computational Linguistics.
  * Marcus et al. (1999) Mitchell P. Marcus, Beatrice Santorini, Mary Ann Marcinkiewicz, and Ann Taylor. 1999\.  Treebank-3.  LDC Catalog No.: LDC99T42, ISBN: 1-58563-163-9, ISLRN: 141-282-691-413-2.
  * Narayan et al. (2017) Shashi Narayan, Claire Gardent, Shay B. Cohen, and Anastasia Shimorina. 2017.  Split and rephrase.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 606–616. Association for Computational Linguistics.
  * Nguyen et al. (2021) Minh Van Nguyen, Viet Dac Lai, Amir Pouran Ben Veyseh, and Thien Huu Nguyen. 2021\.  Trankit: A light-weight transformer-based toolkit for multilingual natural language processing.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations_ , pages 80–90, Online. Association for Computational Linguistics.
  * Nielsen (2004) Leif Arda Nielsen. 2004.  Verb phrase ellipsis detection using automatically parsed text.  In _COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics_ , pages 1093–1099, Geneva, Switzerland. COLING.
  * Nivre et al. (2020) Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Jan Hajič, Christopher D. Manning, Sampo Pyysalo, Sebastian Schuster, Francis Tyers, and Daniel Zeman. 2020.  Universal Dependencies v2: An evergrowing multilingual treebank collection.  In _Proceedings of the Twelfth Language Resources and Evaluation Conference_ , pages 4034–4043, Marseille, France. European Language Resources Association.
  * Park (2009) Dong-woo Park. 2009.  On pseudogapping in HPSG.  In _Proceedings of the 23rd Pacific Asia Conference on Language, Information and Computation, Volume 1_ , pages 425–434, Hong Kong. City University of Hong Kong.
  * Park and Kang (2007) Myung-Kwan Park and Jung-Min Kang. 2007.  Multiple sluicing in English.  In _Proceedings of the 21st Pacific Asia Conference on Language, Information and Computation_ , pages 394–404, Seoul National University, Seoul, Korea. The Korean Society for Language and Information (KSLI).
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020.  Exploring the limits of transfer learning with a unified text-to-text transformer.
  * Rajpurkar et al. (2018) Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018.  Know what you don’t know: Unanswerable questions for squad.  _CoRR_ , abs/1806.03822.
  * Roit et al. (2019) Paul Roit, Ayal Klein, Daniela Stepanov, Jonathan Mamou, Julian Michael, Gabriel Stanovsky, Luke Zettlemoyer, and Ido Dagan. 2019.  Controlled crowdsourcing for high-quality qa-srl annotation.
  * Ross (2014) John Robert Ross. 2014.  _Gapping and the order of constituents_.  De Gruyter Mouton.
  * Schuster et al. (2017) Sebastian Schuster, Matthew Lamm, and Christopher D. Manning. 2017.  Gapping constructions in Universal Dependencies v2.  In _Proceedings of the NoDaLiDa 2017 Workshop on Universal Dependencies (UDW 2017)_ , pages 123–132, Gothenburg, Sweden. Association for Computational Linguistics.
  * Schuster et al. (2018) Sebastian Schuster, Joakim Nivre, and Christopher D. Manning. 2018.  Sentences with gapping: Parsing and reconstructing elided predicates.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 1156–1168, New Orleans, Louisiana. Association for Computational Linguistics.

##  Appendix A Appendix

###  A.1 Parser

Throughout this project, we use the spacy parser with the out-of-the-box
en_core_web_trf model.

###  A.2 Models

##### T5 Configuration.

To ensure reproducibility of our results, we provide the specific
configuration of T5-large model used in our study. The model had 770M
parameters and was trained with a batch size of 8. The optimizer used was
AdamW with a adam_eps value of 1e-8. The maximum input and output length were
set to 256 and weight decay was set to 0. The learning rate was set according
to the recommendations of the Hugging Face library, with a value of 3e-4.
These configurations were used consistently across all variations of the model
in our study.

##### GPT-3 Configuration.

In our approach, we use OpenAI’s text-davinci-3 model, a large language model
based on the GPT-3 architecture. The temperature is set to 0 and top_p remains
1, resulting in conservative and less-deviant text. The maximum number of
tokens generated is set to 256. These parameters have been fine-tuned to
control the generated text.

##### GPT-3 Prompt Format.

Here we show a concrete illustration of the instructions provided to GPT-3.

Q: 58.1% of the population described themselves in the 2011 census return as
being at least nominally Christian <SPLIT> and </SPLIT> 0.7% as Muslim with
all other religions represented by less than 0.5% each.
A:
Cannot re-write this sentence.

Q: Federal education assistance offered affordable loans to Americans who
wanted to attend college  <SPLIT> and </SPLIT> money for local schools to
ensure that all children received an adequate education.
A:
Federal education assistance offered affordable loans to Americans who wanted
to attend college.
Federal education assistance offered money for local schools to ensure that
all children received an adequate education.

Q: He was subsequently asked to repeat the program at the American Asylum for
Deaf - mutes in Hartford, Connecticut, <SPLIT> and </SPLIT> the Clarke School
for the Deaf in Northampton, Massachusetts.
A:
He was subsequently asked to repeat the program at the American Asylum for
Deaf - mutes in Hartford, Connecticut.
He was subsequently asked to repeat the program at the Clarke School for the
Deaf in Northampton, Massachusetts.

Q: The plan, a grid with two main axes meeting at a central square <SPLIT> and
</SPLIT> an additional square in each corner, was based on Thomas Holme’s 1682
plan for Philadelphia.
A:
The plan, a grid with two main axes meeting at a central square, was based on
Thomas Holme’s 1682 plan for Philadelphia.
The plan, a grid with an additional square in each corner, was based on Thomas
Holme’s 1682 plan for Philadelphia.

Q: Example alternative schools include Montessori schools, Waldorf schools,
Friends schools, Sands School, Summerhill School, The Peepal Grove School,
Sudbury Valley School, Krishnamurti schools, <SPLIT> and </SPLIT> open
classroom schools.

###  A.3 Crowdsourcing Task

Here we show the instructions in our crowdsourcing annotation task.

Figure 4: Crowdsourcing task UI

###  A.4 Crowdsourcing Task Instructions

###  A.5 Dependencies in Verb Nucleus.

As detailed in section 6, a verb nucleus contains a verb and its arguments.
While to identify the nucleus root (the verb), we look if their part-of-speech
tag is one of (“VB”, “VBD”, “VBG”, “VBN”, “VBP”, “VBZ”), the rest of the
nucleus is defined over the dependency graph:

  * •

subjects – (“nsubj”, “nsubjpass”, “expl”).

  * •

object – (“dobj”, “obj”, “pobj”, “iobj”, “attr”, “oprd”).

  * •

prepositions and their prepositional modifiers – (“prep”, “agent”) and
(“pobj”, “pcomp”).

  * •

negations – “neg”.

Negations are included to account for cases where the model correctly predicts
most verb arguments, but fails to account for negation, thus, breaking the
original meaning of the sentence. For instance, “The governor urged the public
not to panic and to follow his reports closely” is resolved to:

  * •

The governor urged the public not to panic

  * •

The governor urged the public not to follow his reports closely

###  A.6 Additional Results

To put the performance of the models in context, we provide results over each
conjunct. Moreover, we include exact matching over the sets of sentences,
here, punctuation is removed and the sets are assumed to be aligned. See table
4.

Model | F1${}_{\text{and}}$ | F1${}_{\text{but}}$ | F1${}_{\text{or}}$ | F1 | Exact Match
---|---|---|---|---|---
Calibration1 | 4.7 | 0.0 | 15.6 | 5.1 | 2.4
Calibrationk | 45.2 | 30.9 | 64.9 | 45.5 | 2.4
$T5_{10\%}$ | 59.7 | 28.9 | 45.2 | 55.1 | 37.3
$T5_{20\%}$ | 81.2 | 57.7 | 72.2 | 77.7 | 66.6
$T5_{30\%}$ | 82.9 | 60.4 | 74.2 | 79.5 | 69.1
$T5_{40\%}$ | 84.3 | 70.9 | 76.3 | 82.1 | 72.4
$T5_{50\%}$ | 83.5 | 75.7 | 74.2 | 81.8 | 72.6
$T5_{60\%}$ | 82.6 | 76.2 | 76.9 | 81.3 | 72.7
$T5_{70\%}$ | 82.8 | 76.4 | 76.7 | 81.6 | 73.8
$T5_{80\%}$ | 83.7 | 77.7 | 76.3 | 82.3 | 73.9
$T5_{90\%}$ | 82.9 | 77.5 | 75.3 | 81.6 | 72.6
$T5_{100\%}$ | 82.6 | 79.3 | 82.4 | 82.2 | 74.9
$GPT3$ | 73.5 | 65.4 | 70.4 | 72.3 | 53.1

Table 4: F1 performance of all models based on their conjunction. For context,
an exact match over the sets of sentences is provided.

###  A.7 Additional Examples of Patterns Indicating Suspicious Sentences

In collecting sentences, we broadly look for three types of "suspicious"
structures:

  * •

_part-of-speech mismatch_ – two words with a different part-of-speech are
linked by a conj.

  * •

_dependency relation mismatch_ – an inconsistency between a word’s part-of-
speech tag and its relations to its dependents.

  * •

_subtree mismatch_ – two words with the same part-of-speech, but different
subtrees.

While most patterns involve more than one suspicious structure, we group the
examples to match the above list.

The full sentences demonstrating these patterns can be seen below. We denote
common-nouns, proper-nouns, adjectives, and numerical values with “NON-VERB”,
and a verb or an auxiliary verb, with “VERB”. When a pattern checks for one of
few possible relations between two words, we use “/” to separate them (e.g.,
advcl/xcomp indicates the pattern accepts an advcl or an xcomp relation). The
relation any indicates the pattern accepts any type of relation between two
words, and obj indicates the pattern accepts any type of object.

####  A.7.1 Part-of-speech Mismatch

[column sep=0.5em, row sep =.5em] …& * & …& * & …
& VERB & & NON-VERB &
[edge height=5ex]24conj
Example sentence: Koreans made up 1.2% of the city’s population, and Japanese
0.3%.

[column sep=0.5em, row sep =.5em] …& * & …& * & …
& VERB & & NON-VERB
[edge height=5ex]24conj
Example sentence: Ten chapters are devoted to body issues and how to cover
them.

[column sep=0.5em, row sep =.5em] …& * & …& * &…
& ADV & & ADP &
[edge height=5ex]24conj
Example sentence: Desormeaux has won the Preakness twice: once aboard Real
Quiet in 1998 and again 10 years later on Big Brown.

[column sep=0.5em, row sep =.5em] …& * & …& * & …
& ADP/PART & & NON-VERB &
[edge height=5ex]24conj
Example sentence: Tell us in the comments below or @CNNFilms on Twitter.

[column sep=0.5em, row sep =.5em] …& * & …& * & …
& ADJ & & ADP &
[edge height=5ex]24conj
Example sentence: Southwest said all customers were safe and at the terminal.

####  A.7.2 Dependency Relation Mismatch

[column sep=0.5em, row sep =.5em] …& * & * & …
& NON-VERB & NON-VERB &
[edge height=1.5ex]32nsubj
Example sentence: To idealists, spirit or mind or the objects of mind are
primary, and matter secondary.

####  A.7.3 Subtree Mismatch

[column sep=0.5em, row sep =.5em] …& * & …& * & …& * & …& * & …
& VERB & & CCONJ & & ADP & & ADP &
[edge height=1.5ex]28prep

Example sentence: John was born to Henry II of England and Eleanor of
Aquitaine on 24 December 1166.

[column sep=0.5em, row sep =.5em] * & …& * & …& *
ADP & & ADV & & VERB
[edge height=1.5ex]13advmod [edge height=3.ex]15prep
Example sentence: From the 1880s onward neighbourhoods such as Oud- wijk,
Wittevrouwen, Vogelenbuurt to the East, and Lombok to the West were developed.

[column sep=0.5em, row sep =.5em] …& * & * & …& * & …& * & …
& & & & & & VERB &
[edge height=2.5ex]23xcomp [edge height=5ex]25conj [edge height=5ex]57conj
Example sentence: 19 soldiers, policemen reported wounded, and some attackers
killed, wounded or captured.

[column sep=0.5em, row sep =.5em] …& * & * & …& * & …& * & …
& VERB & & & & & NON-VERB &
[edge height=1.5ex]23obj [edge height=1.5ex]75prep/agent [edge
height=5ex]37conj

Example sentence: You send out these sound waves, and when they bounce off of
objects, the reflection of the waves tells you – or in this case, the animal –
where the objects are.

[column sep=0.5em, row sep =.5em] …& * & * & * & * & …& * & * & …
& & & & & & PART & VERB &
[edge height=1.5ex]23any [edge height=1.5ex]34any [edge height=1.5ex]45any
[edge height=5ex]58conj [edge height=1.5ex]87aux

Example sentence: Some runners started raising money for charity or to help
with relief efforts.

[column sep=0.5em, row sep =.5em] …& * & * & * & …& * & * & …
& VERB & & & & NON-VERB & &
[edge height=3.5ex]23agent/prep [edge height=1.5ex]34pobj/pcomp [edge
height=5ex]46conj [edge height=1.5ex]67prep

Example sentence: Every day, someone new is introduced to the hardships of
wartime military service or the horrors of combat.

[column sep=0.5em, row sep =.5em] * & …& * & …& * & *
NON-VERB & & NON-VERB & & &
[edge height=1ex]65subj [edge height=3ex]16ccomp [edge height=5ex]13conj

Example sentence: Progress in the Business District but lingering blight in
poorer neighborhoods, he says.

[column sep=0.5em, row sep =.5em] …& * & * & * & * & * & * & *
& VERB & & & & NON-VERB & &
[edge height=1.5ex]23obj [edge height=5ex]36conj [edge height=1.5ex]34prep
[edge height=1.5ex]45pobj [edge height=1.5ex]67prep [edge height=1.5ex]78pobj

Example sentence: In 1995, material costs were 30 cents for the jewel case and
10 to 15 cents for the CD.

[column sep=0.5em, row sep =.5em] …& * & …& * & * & * & …
& VERB & & & &
[edge height=5ex]24conj [edge height=1.5ex]45prep [edge height=1.5ex]56pobj

Example sentence: Neesham would make 85 from 80 and Kane Williamson a more
considered 54 from 98 as Sri Lanka toiled.

[column sep=0.5em, row sep =.5em] …& * & * & …& * & …
& VERB & & & ADP &
[edge height=1.5ex]23prep [edge height=5ex]35conj

Example sentence: It is also used in woodcut printmaking, and for engraving.

[column sep=0.5em, row sep =.5em] …& * & * & * & …& …& * & …
& VERB & & & & & ADP &
[edge height=1.5ex]23obj [edge height=1.5ex]34prep [edge height=5ex]47conj

Example sentence: This is The Joker’s war on Batman and even more so, on his
family.

[column sep=0.5em, row sep =.5em] …& * & * & * & * & * & …& * & …
& VERB & & & & & & ADP &
[edge height=1.5ex]23obj [edge height=1.5ex]34any [edge height=1.5ex]45any
[edge height=1.5ex]56prep [edge height=5ex]68conj

Example sentence: They’ve been major players in the uprisings in Yemen and in
Syria.

[column sep=0.5em, row sep =.5em] * & * & …& * & …& * & …
NON-VERB & & & ADP & & VERB &
[edge height=1.5ex]12prep [edge height=6ex]24conj [edge height=3ex]61subj

Example sentence: Government control of the economy and of expression is much
reduced, he says.

[column sep=0.5em, row sep =.5em] …& * & * & …& * & …& * & …
& VERB & & & & & ADP &
[edge height=1.5ex]23prep [edge height=1.5ex]35pobj [edge height=5ex]57conj

Example sentence: They concentrated in trade, services, and especially in
money lending.

[column sep=0.5em, row sep =.5em] …& * & * & …& * & * & …
& VERB & NON-VERB & & NON-VERB & NON-VERB &
[edge height=1.5ex]23obj [edge height=5ex]36conj [edge height=1.5ex]65compound

Example sentence: The meteor show is entertainment for most, but a research
chance for NASA.


# Ultra High Energy Cosmic Rays an overview

Roberto Aloisio  Gran Sasso Science Institute, L’Aquila, Italy  INFN -
Laboratori Nazionali Gran Sasso, Assergi (AQ), Italy<EMAIL_ADDRESS>
###### Abstract

We review the main experimental evidences on ultra high energy cosmic rays and
their implications in the physics of these extremely energetic particles, also
in connection with dark matter and cosmology. We discuss the basis of
theoretical models aiming at explaining observations, highlighting the most
relevant open questions in this fascinating field of research.

##  1 Introduction

Ultra High Energy Cosmic Rays (UHECR) are the most energetic particles ever
observed, with detected energies that range from $10^{17}$ eV up to energies
larger than $10^{20}$ eV.

The two running experiments devoted to the observation of UHECR are the Pierre
Auger Observatory (Auger) in Argentina and Telescope Array (TA) in the US.
UHECR can be observed only indirectly through the detection of their
interaction products with the Earth’s atmosphere. An UHE particle interacting
with a nucleus of the atmosphere produces both hadronic and electromagnetic
cascades of particles, collectively called Extensive Air Shower (EAS). In both
Auger and TA the EAS detection is performed by directly detecting particles
that reach the ground and by the observation of the fluorescence emission
produced in the atmosphere by the EAS particles (due to excitation/de-
excitation of atmosphere’s Nitrogen molecules).

Auger and TA, while sharing the same detection technique, based on the
simultaneous observation of the EAS particles on ground and their fluorescence
emission in air (hybrid events), exhibit a relevant difference in the
acceptance, being that of Auger larger by roughly one order of magnitude.

As always in the case of cosmic rays, the information we can gather
experimentally concerns: energy spectrum, mass composition and anisotropy.
Since the first observation of an EAS with energy $~{}10^{20}$ eV back in 1962
[1], the experimental study of UHECR clarified several important facts (see
[2, 3] and references therein): (i) UHECR are charged particles, with a limit
on photon and neutrino fluxes at $10^{19}$ eV at the level of few percent and
well below respectively, (ii) the spectrum observed at the Earth shows a
slight flattening at energies around $5\times 10^{18}$ eV (called the ankle)
with (iii) an instep at $10^{19}$ eV and (iv) a steep suppression at the
highest energies, (v) mass composition is dominated by light particles (proton
and helium) at energies around $10^{18}$ eV becoming progressively heavier
starting from energies around $3\times 10^{18}$ eV toward the highest
energies.

In order to interpret the observations at the Earth, it is important a
detailed modelling of UHECR propagation in the intergalactic medium, which is
mainly conditioned by the interaction with astrophysical photon backgrounds
and magnetic fields. These interactions shape the spectrum observed at the
Earth and are responsible for the production of secondary (cosmogenic)
particles: photons and neutrinos. This secondary radiation can be observed
through ground-based or satellite experiments and contributes to the
understanding of UHECR physics, bringing important information about the mass
composition of UHECR and, possibly, on their sources.

Sources of UHECR are still a mystery, we do not know which kind of
astrophysical object is responsible for the production of these particles.
There are basically two different classes of astrophysical mechanisms that
could be invoked to accelerate UHECR [2, 3, 4, 5]. The first class is based on
the transfer of energy from a macroscopic object (that can move
relativistically or not) through repeated interactions of particles with
magnetic inhomogeneities; belongs to this class the diffusive shock
acceleration mechanism. The second class is based on the interaction with
electric fields that, through high voltage drops, can accelerate particles (at
once) until the highest energies; belongs to this class the unipolar induction
mechanism in fast spinning neutron stars [2, 3, 4, 5].

The extreme energies of UHECR, as high as $10^{11}$ GeV, eleven orders of
magnitude above the proton mass and ”only” eight below the Planck mass, are a
unique workbench to probe new ideas, models and theories beyond the Standard
Model (SM) of particle physics, which show their effects at energies much
larger than those ever obtained, or obtainable in the future, in accelerator
experiments. This is the case of theories with Lorentz invariance violations
[6, 7, 8, 9] or models of Super-Heavy Dark Matter (SHDM) [6, 10, 11, 12, 13,
14, 15], that connect UHECR observation with the dark sector and cosmological
observations.

The paper is organised as follows. In section 2 it is given an overview of the
current status of experimental measurements. In section 3 are discussed the
general requirements, in order to fit experimental evidences, of the
astrophysical sources and acceleration mechanisms of UHECR. In section 4 it is
discussed the impact of UHECR observations on models of super-heavy relic
particles aimed at solving the Dark Matter (DM) problem. Conclusions take
place in section 5.

##  2 UHECR Observations

As discussed in the Introduction, nowadays the most advanced experiments
observing UHECR are Auger and TA, in the present paper we will refer only to
the results of these two experiments.

The high precision measurement of the spectrum of UHECR is of paramount
importance to falsify theoretical models on the acceleration and propagation
of UHECR. The integrated exposures reached so far by Auger and TA are
respectively $12.2\times 10^{4}$ km2 sr y and $0.8\times 10^{4}$ km2 sr y.
Both experiments are based on the detection of hybrid events, with a set of
fluorescence telescopes (FD) and a surface array of particles detectors (SD).

The determination of the UHECR energy in both Auger and TA is obtained by
calibrating the fluorescence detection given the EAS fluorescence yield
expected for a given energy of the primary particle. The ways Auger and TA
implement this technique show a relevant difference. While Auger uses the
absolute light yield and its wavelength behaviour as measured by the Airfly
Collaboration (see [2, 3] and references therein), TA uses the absolute
florescence yield measured at fixed wavelength (337 nm) by Kakimoto et al. and
the wavelength behaviour of the yield as determined by FLASH (see [2, 3] and
references therein). Another important difference between the two experimental
techniques is related to the way the invisible energy (i.e. the portion of EAS
energy not detectable) is treated. In the case of Auger this estimation is
based on the Airfly measurements, while in the case of TA is obtained through
MC simulations [2, 3].

Figure 1: In the left panel the comparison of the fluxes as measured by Auger
and TA, in the central panel the fluxes with energy rescaled by +4.5% (Auger)
and -4.5% (TA) and in the right panel the comparison of Auger and TA fluxes
observed in the common declination band with energy rescaled as in central
panel. Figure taken from [2].

The joint working group on the spectrum measurement of the Auger and TA
collaborations estimated the relative difference in the energy determination
of the two experiments at the level of $6\%$ (see [2, 3] and references
therein), that is perfectly compatible with the quoted systematic
uncertainties of Auger $(14\%)$ and TA $(21\%)$. The UHECR fluxes measured by
the two experiments are in a remarkable agreement up to energy around $3\times
10^{19}$ eV, see left panel of figure 1. Rescaling the Auger energies of
$+4.5\%$ and the TA ones of $-4.5\%$, well within the aforementioned
uncertainties, the two observed fluxes show an excellent agreement apart from
the highest energies, where the differences seem not conciliable (central
panel of figure 1). To understand if this discrepancy has a physical origin
related to the different sky portions observed by the two experiments, in the
right panel of figure 1 we have plotted the spectra observed using only UHECR
events arriving in the common declination band of Auger and TA
($-15^{\circ}\leq\delta\leq 24.8^{\circ}$). Restricting the analysis to this
sky portion the agreement improves, but at energies larger than $3\times
10^{19}$ eV the differences remain sizeable and cannot be explained by an
independent scaling of the energy reconstruction in the two datasets. It
should also be remarked that the energy spectrum measured by Auger does not
show any dependence on the declination, while that observed by TA does, with
some tension related to the expected UHECR anisotropy [2, 3]. Given the
uncertainties in the energy determination and the poor statistics, it is too
early to draw any definite conclusion on the reason of the differences between
Auger and TA in the spectrum measurement at the highest energies.

Both experiments show a suppression in the energy spectrum at the highest
energies. This effect is still not understood, as it can arise from UHECR
propagation (through the processes of photo-pion production and photo-
disintegration on astrophysical photons backgrounds) or it can be related to
the maximum acceleration energy that the sources can provide [16, 17].

The technique used by Auger and TA to determine the mass composition of UHECR
is based on the measurement of the depth $X_{max}$ in the atmosphere at which
the number of particles in the EAS reaches its maximum and the relative
primary particle energy. These measurements are obtained by the FD
observation. While in the case of Auger, thanks to enough statistics, the
$X_{max}$ determination is obtained independently of detector effects, in the
case of TA it is impossible, due to the reduced statistics, and the detector
effects should be properly subtracted to compare with Auger results (see [2,
3] and references therein).

Figure 2: In the left panel the mean of the distribution of the shower maximum
and in the right panel its standard deviation as observed by Auger and TA. The
TA data are corrected for detector effects by shifting $X_{max}$ by $+5$ g/cm2
and by subtracting an $X_{max}$-resolution of 15 g/cm2 (see [2]). Figures
taken from [2].

In the left panel of figure 2 we plot the mean $\langle X_{max}\rangle$ as
measured by Auger and TA and in the right panel we plot the event-by-event
fluctuations of the shower maximum distribution $\sigma(X_{max})$, the TA data
are corrected subtracting detectors effects as discussed in [2]. For
comparison the prediction of $\langle X_{max}\rangle$ and $\sigma(X_{max})$ in
the case of a pure proton and a pure iron composition are displayed. The
expected behaviour of $\langle X_{max}\rangle$ and $\sigma(X_{max})$ differs
depending on the hadronic interaction model used, in figure 2 we plot three
different models: EPOS-LHC (solid lines), Sybil 2.3 (dashed lines) and
QGSJetII-04 (dotted lines) (see [2, 3] and references therein).

The determination of mass composition can also be pursued using the SD by
measuring the number of muons of the EAS that reach the ground. This detection
possibility is affected by larger uncertainties respect to the $X_{max}$
observation, due to theoretical unknowns in the hadronic physics at the
highest energies of the EAS development. Experimentally, both Auger and TA
observe a mismatch between the number of muons observed with energies above
$10^{9.5}$ GeV and the number of muons predicted by the EAS models [2, 3].

On very general grounds we can state that the determination of the primary
mass composition cannot be obtained in a model-independent way, the
description of hadronic multi-particle production at the highest energies
depends on the extrapolation of the accelerator data on cross sections up to
center of mass energies $\sqrt{s}\simeq 400$ TeV. Moreover, the QCD
predictions for the bulk of hadronic production at these energies are
considerably affected by the hadronic interactions model assumed (see [2, 3]
and references therein).

The most relevant signals of UHECR anisotropies are those connected with large
scales. Auger observed, with a remarkable statistical significance, a dipole
anisotropy at energies above $8\times 10^{18}$ eV, with and amplitude at the
level of $6\%$ and the phase pointing toward the galactic anti-center,
signalling a clear extragalactic origin of these particles [2, 3]. At lower
energies, down to $10^{18}$ eV, the dipole amplitude has only upper limits at
the level of $1\%$, with a phase that rotates with decreasing energy reaching
the galactic center direction below $10^{18}$ eV, where the transition between
galactic and extragalactic cosmic rays seems placed [2, 3, 4, 5].

Studies of small scale anisotropies aimed at identifying sources are not
conclusive, with hints of clustering not yet at the required statistical
significance (see [2, 3] and references therein).

##  3 Sources of UHECR

To constrain the basic features of the sources of UHECR it is useful to adopt
a purely phenomenological approach in which sources are homogeneously and
isotropically distributed, characterised by few parameters: the injection of
particles by the sources with a power law behaviour in energy $\propto
E^{-\gamma_{g}}$, the maximum acceleration energy $E_{max}$ and the source
emissivity ${\cal L}_{S}(A)$ , i.e. number of particles injected per unit
volume and energy for each nuclei species labeled by the atomic mass number
$A$. These parameters, that univocally identify a class of sources, are fitted
to experimental data of both spectrum and mass composition (see [2, 3, 4, 5]
and references therein).

Figure 3: Comparison of the flux (left panel), elongation rate (central panel)
and its dispersion (right panel) as observed by Auger and computed assuming
the model with two classes of different extragalactic sources (see text).
Coloured bands show the uncertainties related to the hadronic interaction
model. Figures taken from [17].

Following [16, 17], taking into account all possible channels of energy losses
and solving the transport equations for UHECR (protons or nuclei) we can
determine the theoretical flux and mass composition expected at the Earth, to
be compared with observations. While the flux directly follows solving the
transport equations, mass composition is inferred from the mean value of the
depth of shower maximum (elongation rate) $\langle X_{max}\rangle$ and its
dispersion $\sigma(X_{max})$, computed as shown in [18]. Once the TA data are
corrected for the detector effects, as follows from figure 2, there is a
substantial agreement between the results on mass composition of Auger and TA.
Therefore, in the following discussion we will refer only to Auger data.

The qualitative new finding that mass composition might be mixed has served as
a stimulus to build models that can potentially explain the phenomenology of
Auger data. These models all show that the Auger spectrum and mass composition
at $E\geq 3\times 10^{18}$ eV can be fitted at the same time only at the price
of requiring very hard injection spectra for all nuclei and a maximum
acceleration energy $E_{max}\leq 5Z\times 10^{18}$ eV [16, 17, 19], being $Z$
the charge of the nucleus. The need for hard spectra can be understood taking
into account that the low energy tail of the flux of UHECR reproduces the
injection power law. Therefore, taking $\gamma\geq 2$ cause the low energy
part of the spectrum to be polluted by heavy nuclei thereby producing a
disagreement with the light composition observed at low energy.

One should appreciate the change of paradigm that these findings imply: while
in the case of a pure proton composition it is needed to find sources and
acceleration mechanisms able to energise CR protons up to energies larger than
$10^{20}$ eV with steep injection [20] ($\gamma_{g}\simeq 2.5\div 2.7$), the
Auger data require that the high energy part of the spectrum ($E>3\times
10^{18}$ eV) has a flat injection ($\gamma_{g}\simeq 1.0\div 1.6$) being
dominated by heavy nuclei with maximum energy not exceeding a few$\times
Z\times 10^{18}$ eV [16]. In order to reproduce Auger observations, the
additional light contribution to the flux at energies below $3\times 10^{18}$
eV should exhibit a steep power law injection with $\gamma_{g}\simeq 2.6\div
2.7$ and a maximum acceleration energy not exceeding a few$\times 10^{18}$ eV,
as for the heavier component [16, 21, 22].

The origin of UHECR can be modelled essentially in two ways: (i) assuming the
presence of different classes of sources: one injecting heavy nuclei with hard
spectrum and the other only proton and helium nuclei with soft spectrum [16,
21] or (ii) identifying a peculiar class of sources that could provide at the
same time a steep light component and a flat heavy one [22, 23, 24]. The
second approach is based on a specific hypothesis on the sources that should
be surrounded by an intense radiation field that, through photo-disintegration
of heavy nuclei in the source neighbourhood, can provide a light component of
(secondary) protons with a steep spectrum together with a hard and heavier
component [22, 23]. It should be also noted that the diffusive shock
acceleration mechanism produces steep injections of particles with
$\gamma_{g}>2$, while acceleration due to potential drops typically imply
$\gamma_{g}\simeq 1$ (see [4, 5] and references therein).

In figure 3 we plot the flux and mass composition as observed by Auger and as
reproduced theoretically assuming two classes of extragalactic sources with
different injection characteristics as discussed above.

##  4 Super Heavy Dark Matter

The leading paradigm to explain DM observations is based on the Weakly
Interactive Massive Particle (WIMP) hypotheses, a stable DM particle with mass
in the range of $0.1\div 10$ TeV [25], as it follows in the framework of
super-symmetry solutions of the ”naturalness problem”. Searches for WIMPs are
ongoing through three different routes: direct detection, indirect detection,
and accelerator searches. None of these efforts have discovered a clear WIMP
candidate so far. In addition, no evidence for new physics in the $0.1\div 10$
TeV range has been observed at the Large Hadron Collider (LHC). Although not
yet conclusive, the lack of evidence for WIMP may imply a different solution
for the DM problem outside of the WIMP paradigm.

To solve the DM puzzle, an alternative to WIMP models is represented by the
scenarios based on long lived super-heavy relics, that can be produced by
several mechanisms taking place during the inflationary phase or just after,
in the re-heating phase (see [10, 11, 12, 14, 15] and references therein).
Once created in the early universe, the abundance of the long-lived super-
heavy particles can evolve to match the DM density observed today, the so-
called Super Heavy Dark Matter (SHDM). This conclusion can be drawn under
three general hypotheses: (i) SHDM in the early universe never reaches local
thermal equilibrium; (ii) SHDM particles have mass $M_{X}$ of the order of the
inflaton mass or higher; and (iii) SHDM particles are long-living particles
with a lifetime exceeding the age of the universe, $\tau_{X}\gg t_{0}$.

Recently, using the LHC measurements of the masses of the Higgs boson and Top
quark, has been pointed out that the Higgs potential is stable until very high
energies [26, 27, 28]. The scale $\Lambda_{I}$ at which the quartic coupling
$\lambda$ of the Higgs potential becomes negative at large field values turns
out to be $\Lambda_{I}\simeq 10^{10}\div 10^{12}$ GeV, moreover, being the
$\lambda$ running extremely slow, this instability develops slowly and it
might be possible to extrapolate the SM to even higher energies up to the
Plank mass $M_{P}=10^{19}$ GeV, with no need for new physics until ultra high
energies [26]. This fact implies that the mass spectrum of the Dark Sector
(DS) is restricted to ultra high energies and SHDM could play a role. In the
following we will consider SHDM with masses in the range between $10^{8}$ GeV
up to the Plank mass.

Apart from the gravitational interaction, SHDM can be coupled to ordinary
matter also through some super-weak coupling, driven by a high energy scale
$\Lambda>\Lambda_{I}$ as the Grand Unification Scale ($\Lambda_{GUT}\simeq
10^{16}$ GeV). To assure long-lived particles, the interaction SHDM-SM should
be suppressed by some power $n$ of the high energy scale $\Lambda$, with a
decay SHMD $\rightarrow$ SM characterised by a lifetime
$\tau_{X}\simeq(M_{X}\alpha_{X\Theta})^{-1}(\Lambda/M_{X})^{2n-8}$, being
$\alpha_{X\Theta}$ the reduced coupling constant between SHDM and SM particles
(see [10] and references therein).

In the case in which SHDM interacts with SM particles only through the
gravitational interaction, assuming the very general case of a DS
characterised by its own non-abelian gauge symmetry, SHDM can decay only
through non-perturbative effects as the instanton-induced decay (see [10] and
references therein). In this case the lifetime of SHDM follows from the
corresponding instanton transition amplitude that, being exponentially
suppressed, provides long-living particles. Considering the zeroth oder
contribution, the instanton-induced lifetime can be written as $\tau_{X}\simeq
M_{X}^{-1}\exp{(4\pi/\alpha_{X})}$, being $\alpha_{X}$ the reduced coupling
constant of the hidden gauge interaction in the DS.

Figure 4: Limits fixed by the Auger observations on the SHDM mass $M_{X}$ and:
(left panel) the coupling SHDM-SM $\alpha_{X\Theta}$ for different values of
the suppression scale order $n$, (central panel) the coupling of the hidden DS
gauge interaction $\alpha_{X}$ in the case of the instanton-induced decay,
(right panel) the universe expansion rate at the end of inflation $H_{inf}$
for different values of the reheating efficiency $\epsilon$ (see text).
Figures taken from [10].

Under very general assumptions on the underlying theory (see [6, 10, 11, 12,
13, 14, 15] and references therein), we can determine the composition and
spectra of the standard model particles produced by the SHDM decay. Typical
decay products are couples of quark and anti-quark that, through a cascading
process (jets), give rise to neutrinos, gamma rays and nucleons with a flat
spectrum, that at the relevant energies can be approximated as $dN/dE\propto
E^{-1.9}$, independently of the particle type, with a photon/nucleon ratio of
about $\gamma/N\simeq 2\div 3$ and a neutrino nucleon ratio $\nu/N\simeq 3\div
4$, quite independent of the energy range [12]. The most constraining limits
on the SHDM mass and lifetime are those coming from the (non) observation of
UHE photons and, even to a lesser extent, neutrinos in the UHECR regime. Auger
observations provide us with very stringent limits on the photon and neutrino
fluxes in the energy range $10^{8}\div 10^{11}$ GeV, enabling far the best
limits on SHDM models [10, 11].

In the left and central panels of figure 4 we plot the limits fixed by the
Auger observations on the SHDM mass $M_{X}$ and the coupling SHDM-SM
$\alpha_{X\Theta}$, in the case of direct coupling (left panel), and the
coupling of the hidden DS gauge interaction $\alpha_{X}$ in the case of the
instanton-induced decay (central panel). Requiring that the SHDM density today
fits the observed DM density, the limits on $M_{X}$ and $\tau_{X}$ can be
rewritten in terms of the cosmological parameters characterising the
generation mechanism of SHDM. In the right panel of figure 4 we plot the
limits coming from Auger data on the age of the universe at the end of
inflation $H_{inf}^{-1}$ and the efficiency of the reheating phase
$\epsilon=(\Gamma_{\phi}/H_{inf})^{1/2}$, being
$\Gamma_{\phi}=g_{\phi}^{2}M_{\phi}/8\pi$ the inflaton (of mass $M_{\phi}$)
decaying amplitude (see [10, 11] and references therein).

##  5 Conclusions

Thanks to the measurements of Auger and TA, in the past 20 years the physics
of UHECR experienced a paradigm shift. The simple picture of protons at the
highest energies has been replaced by a more complex and (phenomenologically)
richer one with nuclei dominating the spectrum already at energies around
$5\times 10^{18}$ eV. Moreover, at the highest energies, it was established
the existence of a strong suppression in the spectrum of UHECR. Despite these
important achievements some long-standing problems in the physics of UHECR
remain unsolved: what are the sources and which astrophysical mechanism
accelerate particles? What is the mass composition at the highest energies?
What is the flux of neutrinos and gamma rays produced by UHECR interactions at
the source and during propagation?

In the next decade the upgrades of the Auger and TA observatories can
contribute answering these questions [2, 3]. Yet, to reach this goal, it is
necessary to refine both experimental analysis and theoretical understandings.
Mass determination is currently limited by the uncertainties in the
predictions of hadronic models and muon content in the EAS, the planned Auger
upgrade with an independent measurement of the electromagnetic and muonic
content of the EAS could help solve this problem. Precise mass determination
till the highest energies will be of paramount importance in the future. The
necessary step forward should be disentangling the all-particle energy
spectrum into that of individual mass groups (as for instance p, He, CNO,
MgAlSi, Fe). On more general grounds, the clearest path to event-by-event
primary mass reconstruction lies in a high-resolution independent
reconstruction of both $X_{max}$ and muon content coupled to a high-resolution
energy measurement.

The behaviour of the spectrum at the highest energies is not univocally
determined, as the results of Auger and TA, even in the same declination band,
seem not compatible, with an earlier position of the flux suppression energy
in the Auger dataset. In order to clarify this point it would be very useful
if the two collaborations could use the same fluorescence yield and invisible
energy to analyse the data. Moreover, the origin of the flux suppression at
the highest energies is still uncertain. Solving the puzzle between a
propagation generated suppression or an acceleration generated one requires
individual mass groups energy spectra and a detailed theoretical modelling of
particles acceleration and propagation in the source environment.

As follows from the observed mass composition a clear identification of
individual sources would be difficult. This goal requires a thorough
theoretical modelling of the effects of galactic and extragalactic magnetic
fields on UHECR propagation and a shower-by-shower determination of the mass
and energy of the primary particle. The study of UHECR has an impact also in
fundamental physics, as it involves tests of models and theories that extend
beyond the standard model of particle physics. This is the case of SHDM that,
being a viable alternative to the WIMP paradigm for DM, can be tested only
through the observations of UHECR eventually connected with those of cosmology
[6, 14], as follows from the right panel of figure 4.

We conclude stating that a complete picture of the ultra high energy universe
needs to account for all messengers till the highest energies. The sensitivity
required to detect UHE neutrinos and gamma rays, as well as the highest energy
tail of CR, is still far from being reached, new technologies are needed and
future space observatories, with improved photon detection techniques,
allowing the needed statistics, might open a new era in the physics of UHE
cosmic radiation [29, 30, 31].

## References

## References

  * [1] J. Linsley, Phys. Rev. Lett. 10, 146-148 (1963)
  * [2] A. Coleman, et al. [arXiv:2205.05845 [astro-ph.HE]].
  * [3] R. Alves Batista, et al. Front. Astron. Space Sci. 6, 23 (2019) [arXiv:1903.06714 [astro-ph.HE]].
  * [4] R. Aloisio, P. Blasi, I. De Mitri and S. Petrera, Springer Cham 2018, [arXiv:1707.06147 [astro-ph.HE]].
  * [5] R. Aloisio, PTEP 2017, no.12, 12A102 (2017) [arXiv:1707.08471 [astro-ph.HE]].
  * [6] R. X. Adhikari, et al. [arXiv:2209.11726 [hep-ph]].
  * [7] A. Addazi, et al. Prog. Part. Nucl. Phys. 125, 103948 (2022) [arXiv:2111.05659 [hep-ph]].
  * [8] R. Aloisio, P. Blasi, et al. Phys. Rev. D 62, 053010 (2000) [arXiv:astro-ph/0001258 [astro-ph]].
  * [9] R. Aloisio, P. Blasi, A. Galante, et al. Astropart. Phys. 19, 127-133 (2003) [arXiv:astro-ph/0205271 [astro-ph]].
  * [10] P. Abreu et al. [Pierre Auger], [arXiv:2208.02353 [astro-ph.HE]].
  * [11] P. Abreu, et al. [Pierre Auger], [arXiv:2203.08854 [astro-ph.HE]].
  * [12] R. Aloisio, V. Berezinsky, et al. Phys. Rev. D 74, 023516 (2006) [arXiv:astro-ph/0604311 [astro-ph]].
  * [13] C. Guépin, R. Aloisio, et al. Phys. Rev. D 104, no.8, 083002 (2021) [arXiv:2106.04446 [hep-ph]].
  * [14] R. Aloisio, S. Matarrese and A. V. Olinto, JCAP 08, 024 (2015) [arXiv:1504.01319 [astro-ph.HE]].
  * [15] R. Aloisio and F. Tortorici, Astropart. Phys. 29, 307-316 (2008) [arXiv:0706.3196 [astro-ph]].
  * [16] R. Aloisio, V. Berezinsky and P. Blasi, JCAP 10, 020 (2014) [arXiv:1312.7459 [astro-ph.HE]].
  * [17] R. Aloisio, D. Boncioli, et al. JCAP 10, 006 (2015) [arXiv:1505.04020 [astro-ph.HE]].
  * [18] P. Abreu, et al. [Pierre Auger], JCAP 02, 026 (2013) [arXiv:1301.6637 [astro-ph.HE]].
  * [19] R. Aloisio, V. Berezinsky, et al. Astropart. Phys. 34, 620-626 (2011) [arXiv:0907.5194 [astro-ph.HE]].
  * [20] R. Aloisio, V. Berezinsky, et al. Astropart. Phys. 27, 76-91 (2007) [arXiv:astro-ph/0608219 [astro-ph]].
  * [21] A. M. Taylor, Astropart. Phys. 54, 48-53 (2014) [arXiv:1401.0199 [astro-ph.HE]].
  * [22] N. Globus, D. Allard and E. Parizot, Phys. Rev. D 92, no.2, 021302 (2015) [arXiv:1505.01377 [astro-ph.HE]].
  * [23] M. Unger, G. R. Farrar, et al. Phys. Rev. D 92, no.12, 123001 (2015) [arXiv:1505.02153 [astro-ph.HE]].
  * [24] P. Blasi, E. Amato, et al. Phys. Rev. Lett. 115, no.12, 121101 (2015) [arXiv:1508.02866 [astro-ph.HE]].
  * [25] L. Bergström, Rept. Prog. Phys. 63, 793 (2000) [arXiv:hep-ph/0002126 [hep-ph]].
  * [26] D. Buttazzo, G. Degrassi, et al. JHEP 12, 089 (2013) [arXiv:1307.3536 [hep-ph]].
  * [27] S. Alekhin, A. Djouadi and S. Moch, Phys. Lett. B 716, 214-219 (2012) [arXiv:1207.0980 [hep-ph]].
  * [28] A. V. Bednyakov, B. A. Kniehl, et al. Phys. Rev. Lett. 115, no.20, 201802 (2015) [arXiv:1507.08833 [hep-ph]].
  * [29] A. L. Cummings, R. Aloisio, et al. Phys. Rev. D 103, no.4, 043017 (2021) [arXiv:2011.09869 [astro-ph.HE]].
  * [30] A. Cummings, R. Aloisio, et al. Phys. Rev. D 104, no.6, 063029 (2021) [arXiv:2105.03255 [astro-ph.IM]].
  * [31] A. V. Olinto, et al. [POEMMA], JCAP 06, 007 (2021) [arXiv:2012.07945 [astro-ph.IM]].


# The population of Galactic centre filaments III:
candidate radio and stellar sources

F. Yusef-Zadeh1, R. G. Arendt2, M. Wardle3, I. Heywood4, & W. Cotton5
1 Department of Physics and Astronomy Northwestern University, Evanston, IL
60208
2Code 665, NASA/GSFC, 8800 Greenbelt Road, Greenbelt, MD 20771, USA &
UMBC/CRESST 2
3School of Mathematical and Physical Sciences, Research Centre for Astronomy,
Astrophysics
and Astrophotonics, Macquarie University, Sydney NSW 2109, Australia
4Royal Astronomical Society, Burlington House, Piccadilly, London W1J 0BQ, UK
and Department of Physics and Electronics, Rhodes University, PO Box 94,
Makhanda, 6140, South Africa
5 National Radio Astronomy Observatory, Charlottesville, VA, USA E-mail:
<EMAIL_ADDRESS>
(Accepted XXX. Received YYY; in original form ZZZ)

###### Abstract

Recent MeerKAT radio continuum observations of the Galactic center at 20 cm
show a large population of nonthermal radio filaments (NRFs) in the inner few
hundred pc of the Galaxy. We have selected a sample of 57 radio sources,
mainly compact objects, in the MeerKAT mosaic image that appear to be
associated with NRFs. The selected sources are about 4 times the number of
radio point sources associated with filaments than would be expected by random
chance. Furthermore, an apparent correlation between bright IR stars and NRFs
is inferred from their similar latitude distributions, suggesting that they
both co-exist within the same region. To examine if compact radio sources are
related to compact IR sources, we have used archival 2MASS, and Spitzer data
to make spectral energy distribution of individual stellar sources coincident
or close to radio sources. We provide a catalogue of radio and IR sources for
future detailed observations to investigate a potential 3-way physical
association between NRFs, compact radio and IR stellar sources. This
association is suggested by models in which NRFs are cometary tails produced
by the interaction of a large-scale nuclear outflow with stellar wind bubbles
in the Galactic center.

††pubyear: 2019††pagerange: The population of Galactic centre filaments III:
candidate radio and stellar sources–LABEL:lastpage