# ASER: Towards Large-scale Commonsense Knowledge Acquisition via Higher-order
Selectional Preference over Eventualities

Hongming Zhang<EMAIL_ADDRESS>Xin Liu<EMAIL_ADDRESS>Haojie Pan
<EMAIL_ADDRESS>Haowen Ke<EMAIL_ADDRESS>Jiefu Ou<EMAIL_ADDRESS>Tianqing Fang<EMAIL_ADDRESS>Yangqiu Song<EMAIL_ADDRESS>CSE, HKUST,
HKSAR, China

###### Abstract

Commonsense knowledge acquisition and reasoning have long been a core
artificial intelligence problem. However, in the past, there has been a lack
of scalable methods to collect commonsense knowledge. In this paper, we
propose to develop principles for collecting commonsense knowledge based on
selectional preference, which is a common phenomenon in human languages that
has been shown to be related to semantics. We generalize the definition of
selectional preference from one-hop linguistic syntactic relations to higher-
order relations over linguistic graphs. Unlike previous commonsense knowledge
definition (e.g., ConceptNet), the selectional preference (SP) knowledge only
relies on statistical distribution over linguistic graphs, which can be
efficiently and accurately acquired from the unlabeled corpus with modern
tools, rather than human-defined relations. As a result, acquiring SP
knowledge is a much more salable way of acquiring commonsense knowledge.
Following this principle, we develop a large-scale eventuality (a linguistic
term covering activity, state, and event)-based knowledge graph ASER, where
each eventuality is represented as a dependency graph, and the relation
between them is a discourse relation defined in shallow discourse parsing. The
higher-order selectional preference over collected linguistic graphs reflects
various kinds of commonsense knowledge. For example, dogs are more likely to
bark than cats as the eventuality “dog barks” appears 14,998 times in ASER
while “cat barks” only appears 6 times. “Be hungry” is more likely to be the
reason rather than result of “eat food” as the edge $\langle$“be hungry”,
Cause, “eat food”$\rangle$ appears in ASER while $\langle$“eat food”, Cause,
“be hungry”$\rangle$ does not. Moreover, motivated by the observation that
humans understand events by abstracting the observed events to a higher level
and can thus transferring their knowledge to new events, we propose a
conceptualization module on top of the collected knowledge to significantly
boost the coverage of ASER. In total, ASER contains 438 million eventualities
and 648 million edges between eventualities. After conceptualization with
Probase, a selectional preference based concept-instance relational knowledge
base, our concept graph contains 15 million conceptualized eventualities and
224 million edges between them. Detailed analysis is provided to demonstrate
its quality. All the collected data, APIs, and tools that can help convert
collected SP knowledge into the format of ConceptNet are available at
https://github.com/HKUST-KnowComp/ASER.

###### keywords:

Commonsense Acquisition , Selectional Preference , Eventualities

fn1fn1footnotetext: The first three authors make equally important
contributions to this work. Detailed contributions are in the end of this
paper.

##  1 Introduction

Knowledge is crucial to understanding natural language. When reading, in
addition to linguistic knowledge of the vocabulary and grammar of a language,
readers need to have knowledge about the structure of texts, knowledge about
the subject, and background or commonsense knowledge about the world in order
to comprehend the text. For example, when a user says “I am hungry” to a
chatbot at 1:00 pm, the chatbot should be able to understand that the user may
want to have lunch rather than breakfast and recommend some nearby
restaurants. This requires the chatbot to understand the complex commonsense
knowledge about user’s states and potential consequent activities (i.e., being
hungry can motivate the user to eat) and the implications of location and time
(i.e., compared with breakfast, lunch is more likely to appear at 1:00 pm.
Thus the chatbot should recommend some real food rather than just a cup of
coffee).

Commonsense reasoning has long been a challenging problem in the artificial
intelligence field. As discussed in [1], commonsense knowledge refers to
“millions of basic facts and understanding possessed by most people.” Unlike
factual knowledge like “London is the capital of UK,” which is always true,
commonsense knowledge is often not inevitably true and only reflects a kind of
contextual preference. For example, in most cases, rocks are not used for
eating, but some birds do eat rocks to digest. Such kind of knowledge is also
called factoids [2, 3]. To effectively represent such preference-like
commonsense knowledge, selectional preference [4] was proposed, which was
traditionally defined on top of single dependency connections (e.g., nsubj,
dobj, and amod). Given a word and a dependency relation, humans have
preferences for which words are likely to be connected. For instance, when
seeing the verb “sing,” it is highly plausible that its object is “a song,”
and when seeing the noun “air,” it is highly plausible that its modifier is
“fresh.” However, such selectional preference can only represent commonsense
inside an event or state (e.g., which event/state is more likely to happen)
and cannot represent commonsense between events/states. One such example is
discussed by [5] and similar examples are frequently observed the Winogrande
Schema Challenge [6]:

  * 1.

The soldiers fired at the women, and we saw several of them fall.

To resolve the pronoun “them” in the above example, Wilks argued that machines
need to access the partial information (in MaCarthy’s phrase) “hurt things
tending to fall down,” which can be translated into the following form: (hurt,
X) $\xrightarrow{\rm ResultIn}$ (X, fall).

In history, many efforts have been devoted to acquiring commonsense knowledge
in the form of multi-relational factoids. For example, the Cyc project
initiated in the 1980s [7] and ConceptNet (originated from Open Mind Common
Sense, OMCS) initiated in 2002 [1], tried to use experts or ordinary people to
annotate commonsense knowledge collectively. However, as aforementioned, two
properties of commonsense knowledge determine that we cannot acquire all
commonsense knowledge with such approaches. First, the scale of commonsense
knowledge could be enormous and it is infeasible to perform crowd-sourcing for
commonsense knowledge acquisition in such a huge scale. Second, commonsense
knowledge is often a kind of preference rather than fixed fact, and thus it is
not suitable to represent commonsense knowledge with fixed triplets (e.g.,
$\langle$“rock”, NotUsedFor, “eat”$\rangle$) as used in Cyc and ConceptNet.
Recently, pre-trained language representation models (e.g., BERT [8] and
RoBERTa [9]) have been developed to acquire rich human knowledge implicitly
and have demonstrated promising results on many downstream tasks. However, as
shown in LAMA [10] and TransOMCS [11], even though these models are good at
capturing factual knowledge about named entities, they still struggle at
capturing commonsense knowledge, especially those complex commonsense
knowledge between eventualities (a linguistic term covering activities,
states, and events after [12], e.g., “I am hurt”). One possible explanation is
that compared with tokens or named entities, the distribution of eventualities
is generally much more sparse. More importantly, as discussed by [1], much
commonsense knowledge, which is trivial for humans, is typically not discussed
in our daily language at all. As a result, even though these deep pre-trained
language representation models are good at acquiring knowledge from textual
data, they could not effectively acquire or reason commonsense knowledge they
rarely or never see in the form of word sequences.

Figure 1: ASER Demonstration. Eventualities are connected with weighted
directed edges. Each eventuality is a dependency graph.

To explore a scalable way of acquiring commonsense knowledge, in this paper,
we propose an approach to constructing a large-scale weighted eventuality
knowledge graph, ASER (Activities, States, Events, and their Relations), by
extending the traditional definition of selectional preference to higher-order
selectional preference over eventualities. The eventualities (i.e., nodes of
ASER) are extracted using selected dependency patterns. The edges are based on
discourse relations (e.g., Result) in discourse analysis. As shown in Figure
1, both nodes and edges are associated with frequency-based weights to reflect
higher-order selectional preferences given a specific linguistic (either
dependency and discourse) pattern. As discussed by [4, 13], such frequency
distribution can serve as a good fit for humans’ selectional preference, which
is indeed the commonsense knowledge. An example is shown in Figure 1. In ASER,
“I eat plate” and “I eat fork” never appear in ASER while “I eat pizza”
appears 57 times. We can infer that “plate” and “fork” are not subjects that
can be eaten while “pizza” is. Similarly, the frequencies of edges can be used
to reflect higher-order selectional preference between eventualities. For
example, by observing that $\langle$“Person be hungry”-Result-“Person
eat”$\rangle$ appear at least 12 times while $\langle$“Person be
hungry”-Reason-“Person eat”$\rangle$ appears only once, we can know that
“Person be hungry” is more likely to result in rather than be caused by
“Person eat.” We argue that the higher-order selectional preference in ASER
can be scalable and effective to represent previously defined commonsense
knowledge types in ConceptNet [1] and potentially many other types of
commonsense knowledge.

To build such a large-scale eventuality knowledge graph, we first leverage
unsupervised algorithms and existing tools (e.g., dependency/discourse
parsing) to extract eventualities and their relations from raw documents. For
the eventuality extraction, considering that the English language’s syntax is
relatively fixed and consistent across domains and topics, instead of defining
complex triggers and role structures of events, we use syntactic patterns to
extract all possible eventualities. We do not distinguish between semantic
senses or categories of particular triggers or arguments in eventualities but
treat all extracted words with their dependency relations as hyperedge in a
graph to define an eventuality as a primitive semantic unit in our knowledge
graph. For eventuality relations extraction, we adopt an end-to-end discourse
parser [14] to determine the discourse relations between eventuality spans
automatically and then create edges based on the predicted relation. Compared
with previous commonsense knowledge acquisition methods, acquiring selectional
preference knowledge with linguistic patterns and discourse relation
prediction models is much cheaper and scalable. Thus, it can be used to
extract large-scale selectional preference knowledge from the unlabeled
corpus. After that, to overcome the challenge that a large portion of the
commonsense knowledge is rarely expressed in textual corpus and motivated by
the observation [15] that human beings often conceptualize the events to a
more abstract level such that they can be applied to new events, we propose to
leverage existing conceptualization techniques [16, 17] to automatically
generalize the knowledge we observed and extracted to those unseen
eventualities.

As a result, we create ASER, which contains 438,648,952 unique eventualities
and 648,514,465 edges. Table 1 provides a size comparison between three
variations of ASER111ASER (core) includes all extracted eventualities that
appear more than once, ASER (full) includes all extracted eventualities, and
ASER (concept) includes all conceptualized eventualities. (i.e., core, full,
and concept) and existing eventuality-related (or simply verb-centric)
knowledge bases. Essentially, they are not large enough as modern knowledge
graphs and inadequate for capturing the richness and complexity of
eventualities and their relations. FrameNet [18] is considered the earliest
knowledge base defining events and their relations. It provides annotations
about relations among about 1,000 human-defined eventuality frames, which
contain 27,691 eventualities. However, given the fine-grained definition of
frames, the scale of the annotations is limited. ACE [19] (and its follow-up
evaluation TAC-KBP [20]) reduces the number of event types and annotates more
examples in each of event types. PropBank [21] and NomBank [22] build frames
over syntactic parse trees, and focus on annotating popular verbs and nouns.
TimeBank focuses only on temporal relations between verbs [23]. While the
aforementioned knowledge bases are annotated by domain experts,
OMCS/ConceptNet222Following the original definition, we only select the four
relations (“HasPrerequisite”, “HasFirstSubevent”, “HasSubEvent”, and
“HasLastSubEvent”) that involve eventualities. [1], Event2Mind [24], ProPora
[25], ATOMIC [26], ATOMIC-2020 [27], and GLUECOSE [28] leveraged crowdsourcing
platforms or the general public to annotate commonsense knowledge about
eventualities, in particular the relations among them. Furthermore, KnowlyWood
[29] uses semantic parsing to extract activities (verb+object) from movie/TV
scenes and novels to build four types of relations (parent, previous, next,
similarity) between activities using inference rules. Compared with all these
eventuality-related KGs, ASER is larger by one or more orders of magnitude in
terms of the numbers of eventualities and relations it contains.

Table 1:  Size comparison of ASER and existing eventuality-related resources. # Eventuality, # Relation, and # Relation Types are the number of eventualities, relations between these eventualities, and relation types. For KGs containing knowledge about both entity and eventualities, we report the statistics about the eventualities subset. ASER (core) filters out eventualities that appear only once and thus has better accuracy while ASER (full) can cover more knowledge. ASER (concept) runs conceptualization to aggregate diverse relations from a much cleaner ASER, resulting in a much denser commonsense knowledge graph. | # Eventuality | # Relation | # Relation Types
---|---|---|---
FrameNet | 27,691 | 1,709 | 7
ACE | 3,290 | 0 | 0
PropBank | 112,917 | 0 | 0
NomBank | 114,576 | 0 | 0
TimeBank | 7,571 | 8,242 | 1
OMCS (Only include edges about eventualities) | 74,989 | 116,097 | 4
Event2Mind | 24,716 | 57,097 | 3
ProPora | 2,406 | 16,269 | 1
ATOMIC | 309,515 | 877,108 | 9
ATOMIC-2020 | 638,128 | 1,331,113 | 23
GLUECOSE | 286,753 | 304,099 | 10
Knowlywood | 964,758 | 2,644,415 | 4
ASER (core) | 52,940,258 | 52,296,498 | 14
ASER (full) | 438,648,952 | 648,514,465 | 14
ASER (concept) | 15,640,017 | 224,213,142 | 14

In summary, our contributions are as follows.

  1. 1.

Represent commonsense knowledge with higher-order selectional preference: We
extend the original definition of selectional preference to higher-order
selectional preference between eventualities. By showing that we can cheaply
acquire selectional preference knowledge from the unlabeled corpus and convert
such knowledge into commonsense knowledge in the format of other commonsense
knowledge bases such as ConceptNet and ATOMIC.

  2. 2.

Definition of ASER: We define a brand new knowledge graph (KG) where the
primitive units of semantics are eventualities. We organize our KG as a
relational graph of hyperedges. Each eventuality instance is a hyperedge
connecting several vertices, which are words. A relation between two
eventualities in our KG represents one of the 14 relation types defined in
PDTB [30] or a co-occurrence relation.

  3. 3.

Scalable Extraction of ASER: We perform eventuality extraction over large-
scale corpora. We design several high-quality patterns based on dependency
parsing results to extract all eventualities that match these patterns and
then apply a discourse parsing system to extract the eventuality relations. In
the end, we leverage a conceptualization module to generalize the extracted
knowledge to unseen eventualities.

  4. 4.

Inference over ASER: We also provide several ways of commonsense inference
over ASER. We show that both eventuality and relation retrieval over one-hop
or multi-hop relations can be modeled as conditional probability inference
problems.

  5. 5.

Evaluation and Applications of ASER: We conduct an extensive evaluation to
demonstrate the quality of extracted eventuality knowledge and the
transferability from such linguistic-based knowledge to commonsense knowledge.

The rest of the paper is organized as follows. In Section 2, we introduce
related works. In Section 3, 4, 5, and 6, we introduce the details about the
design principles, definition, construction, and statistics of ASER. After
that, we present extensive evaluations to demonstrate the quality of extracted
knowledge in Section 7. In Section 8, we present various kinds of inference
methods ASER can support. Last but not least, we demonstrate the
transferability from SP knowledge to commonsense knowledge defined in
ConceptNet [1] and ATOMIC [26] in Section 9. The applications of ASER in
downstream tasks are discussed in Section 10. In the end, we conclude this
paper with Section 11.

##  2 Related Works

In his conceptual semantics theory, Ray Jackendoff, a Rumelhart Prize333The
David E. Rumelhart Prize is funded for contributions to the theoretical
foundations of human cognition. winner, describes semantic meaning as “a
finite set of mental primitives and a finite set of principles of mental
combination [31]”. The primitive units of semantic meanings include Thing (or
Object), Activity,444In his original book, he called it Action. But given the
other definitions and terminologies we adopted [32, 12], it means Activity.
State, Event, Place, Path, Property, Amount, etc. Understanding the semantics
related to the world requires the understanding of these units and their
relations. Traditionally, linguists and domain experts built knowledge graphs
(KGs)555Traditionally, people used the term “knowledge base” to describe the
database containing human knowledge. In 2012, Google released its knowledge
graph where vertices and edges in a knowledge base are emphasized. We discuss
in the context of the knowledge graph, as our knowledge is also constructed as
a complex graph. For more information about terminologies, please refer to
[33]. to formalize these units and enumerate categories (or senses) and
relations of them. Typical KGs include WordNet [34] for words, FrameNet [18]
for events, and Cyc [7] and ConceptNet [1] for commonsense knowledge. However,
their small scales restricted their usage in real-world applications.

Nowadays, with the growth of Web contents, computational power, and the
availability of crowdsourcing platforms, many modern and large-scale KGs, such
as Freebase [35], KnowItAll [36], TextRunner [37], YAGO [38, 39], BabelNet
[40], DBpedia [41], NELL [42], Probase [43], and Google Knowledge Vault [44],
have been built based on semi-automatic mechanisms. Most of these KGs are
designed and constructed based on facts about Things or Objects, such as
instances and their concepts, named entities and their categories, as well as
their properties and relations. On top of them, a lot of semantic
understanding problems such as question answering [45] can be supported by
grounding natural language texts on knowledge graphs, e.g., asking a bot for
the nearest restaurants for lunch. Nevertheless, these KGs may fall short in
circumstances that require not only fact knowledge about Things or Objects,
but also the commonsense knowledge about Activities, States, and Events.
Consider the aforementioned utterance that a human would talk to the bot at 1
PM: “I am hungry”, which may also imply one’s need for a restaurant
recommendation. This, however, will not be possible unless the bot can
identify that the consequence of being hungry would be “having lunch” at noon.

In this paper, we propose to leverage higher-order selectional preference to
discover and store commonsense knowledge about Activities (or process, e.g.,
“I sleep”), States (e.g., ”I am hungry”), Events (e.g., “I make a call”), and
their Relations (e.g., “I am hungry” may result in “I have lunch”), for which
we call ASER. In fact, Activities, States, and Events, which are expressed by
verb-related clauses, are all eventualities following the commonly adopted
terminology and categorization proposed by Mourelatos [32] and Bach [12].
Previous literature on eventualities mostly focuses on extracting
eventualities from text with pre-defined event schemas, which enumerates
triggers with senses and arguments with roles, defined in FrameNet [18] or ACE
[19]. However, as the pre-defined event ontology is often domain-specific and
small (e.g., ACE contains 33 event types), the extracted events cannot cover
all commonsense. Different from them, instead of using a small event ontology,
we use patterns over the dependency graphs, which could contain multiple words
and dependency edges, to extract eventualities. Any events that satisfy the
pre-defined patterns will be extracted, and thus we achieve much broader
coverage. Besides the eventuality extraction, extracting relations between
eventualities is another vital research problem in the NLP community. For
example, HieVe [46] focuses on extracting super-sub event relations, and
TimeBank [23] focuses on the temporal relations. These works typically focus
on identifying implicit relations, which is a very challenging task, and the
state-of-the-art models can only achieve 59.5 F1 [47] and 75.5 F1 [48] on the
HieVe and TimeBank datasets, respectively. As a result, current models are
still not ready to be used to extract high-quality relations between events.
As an alternative, we discard the implicit relations and only focus on
explicit discourse relations between events. By doing so, we sacrifice the
recall but make sure the high-quality of the collected knowledge. For example,
the used discourse parser proposed by [14] can guarantee 90.14% F1 on explicit
discourse relation classification. Simultaneously, we try to scan a huge
corpus to guarantee the resulting knowledge graph’s overall coverage.

##  3 Design Principles

As aforementioned, ASER is a large-scale eventuality-based knowledge graph.
Here by eventuality, we mean Activities, States, and Events, which are defined
based on the commonly adopted terminology and categorization proposed by
Mourelatos [32] and Bach [12]:

  * 1.

Activity:  An activity is also called a process [32, 12]. Both activity and
event are occurrences (actions) described by active verbs. An example is “The
coffee machine is brewing coffee.”

  * 2.

State: A state is usually described by a stative verb and cannot be qualified
as actions. A typical state expression is “The coffee machine is ready for
brewing coffee”.

  * 3.

Event: An event is defined as an occurrence that is inherently countable [32].
For example, “The coffee machine brews a cup of coffee once more” is an event
because it admits a countable noun “a cup” and cardinal count adverbials
“once,” while “The coffee machine brews coffee” is not an event with an
imperfective aspect which is not countable.

Unlike the previous works [49, 50], we do not distinguish activities (or
processes), states, and events. Instead, we use dependency patterns to
represent all the eventualities that can be activities, states, and events and
also discourse relations [30] such as COMPARISON.Contrast and
CONTINGENCY.Cause as the relation types between eventualities based on the
following two design principles.

Figure 2: Principle Demonstration. When we fix the grammar, humans’ preference
over the linguistic description are reflecting the commonsense. The examples
are based on ones used in [51].

###  3.1 The Lower Bound of a Semantic Theory

As discussed by the lower bound of a semantic theory [51], understanding human
language requires both the knowledge about the language (i.e., grammar) and
knowledge about the world. As a result, if we fix the grammar structure of the
linguistic descriptions, their difference will be the semantics. An example is
shown in Figure 2. There are three sentences that share the same grammar
structure but describe different events, which may have different reasons,
effect, and sub-events. Give that the previous context is “It is dangerous,”
humans normally will prefer the second sentence to appear in this context
because lion is a dangerous animal. And such preference can reflect the
commonsense knowledge we are after. Motivated by this, we propose to use
dependency patterns to categorize eventualities and discourse relations as the
relations between eventualities. As a result, the frequency about
eventualities and edges can be naturally used to represent humans’ preference
when the grammar structure is fixed.

SP Pair | Plausibility
---|---
(eat, meal) | 10.00
(close, door) | 8.50
(convince, people) | 7.75
(touch, food) | 5.50
(hate, investment) | 4.00
(confront, impulse) | 2.78
(eat, mail) | 0.00
(a) dobj

SP Pair | Plausibility
---|---
(singer, sing) | 10.00
(law, permit) | 7.78
(women, pray) | 5.83
(realm, remain) | 3.06
(victim, contain) | 2.22
(bar, act) | 1.39
(textbook, eat) | 0.00
(b) nsubj

SP Pair | Plausibility
---|---
(fresh, air) | 9.77
(new, method) | 8.89
(young, people) | 6.82
(medium, number) | 4.09
(immediate, food) | 2.50
(eager, price) | 1.36
(secret, wind) | 0.75
(c) amod

SP Pair | Plausibility
---|---
(lift, heavy object) | 9.17
(design, new object) | 8.00
(recall, previous object) | 7.05
(attack, small object) | 5.23
(drag, drunk object) | 4.25
(inform, weird object) | 3.64
(earn, rubber object) | 0.63
(d) dobj_amod

SP Pair | Plausibility
---|---
(friendly subject, smile) | 10.00
(evil subject, attack) | 9.00
(recent subject, demonstrate) | 6.00
(random subject, bear) | 4.00
(happy subject, steal) | 2.25
(stable subject, understand) | 1.75
(sunny subject, make) | 0.56
(e) nsubj_amod

Table 2: Examples of first-order and second-order selectional preference and
their plausibility ratings provided by human annotators [13]. Object and
subject are place holders to help understand the second-hop selectional
preference relations.

Historically, such grammar-based semantics is called selectional preference
[52], which is a relaxation of selectional restrictions [51]. Initially, the
research on selectional preference focuses on the IsA hierarchy in WordNet
[34] and verb-object dependency relations. Later on, the idea of selectional
preference was extended to verb-subject dependency relations. Several first-
order selectional preference examples are as follows.

  * 1.

SP(Cat, IsA, Animal) $>$ SP(Cat, IsA, Plant)

  * 2.

SP(Eat, dobj, Food) $>$ SP(Eat, dobj, Rock)

  * 3.

SP(Sing, nsubj, Singer) $>$ SP(Sing, nsubj, House)

Recently, to represent more complex commonsense knowledge, the principle of
selectional preference was extended to the second-order [13]. The motivation
is that humans tend to have a strong preference over the property of certain
verbs’ subjects and objects. For example, we can formalize the commonsense
that the subject of eat is more likely to be “hungry” rather than “tasty” with
the following second-order selectional preference:

  * 1.

SP(Eat, Nsubj-amod, Hungry) $>$ SP(Eat, Nsubj-amod, Tasty)

More examples are shown in Table 2. Higher plausibility scores indicate that
the annotators have a stronger preference for the combination. For the first-
order selectional preference, people most likely to select “meal” rather than
“mail” as the object of “eat.” Similarly, we can see that “heavy” is a common
property of the object of “lift.” As shown by the experiments in [13], such
selectional preference knowledge is crucial for solving commonsense reasoning
tasks such as Winograd Schema Challenge [6].

In this work, we further extend the idea of selectional preference to
discourse relations between eventualities, which is denoted as higher-order
selectional preference over eventualities.

###  3.2 The Need of Aggregating “Partial Information” in Commonsense
Reasoning

As discussed by [5], to effectively represent the selectional preference over
linguistic relations and use that knowledge for language inference, we need to
do aggregation over the “partial information,” which may “not be invariably
true” but “tends to be of a very high degree of generality indeed” [1]. For
example, Wilks used the following sentence as an example.

  * 1.

The soldiers fired at the women, and we saw several of them fall.

We know that them should refer to women rather than soldiers because we have
the partial information that “hurt things tending to fall down.” Formally, it
can be translated into the following form:

  * 1.

(hurt, X) $\xrightarrow{\rm ResultIn}$ (X, fall).

There are many ways to find such representations of knowledge, e.g., first or
even second-order logic. However, existing logic-based or semantic frame-based
methods such as combinatory categorial grammar [53] or semantic role labeling
[18, 54] require large amounts of annotation. Moreover, the semantic roles
defined in labeled frames [18, 54] are too coarse-grained to support fine-
grained conceptual reasoning. An efficient way of acquiring such partial
information is to do the aggregation over collected selectional preference
about the instance-level eventualities and their conceptualizations. We have
shown that using such higher-order selectional preference, we can solve a
subset of Winograd Schema Challenge (WSC) [6] with 70% accuracy [13]. For
example, to solve the WSC example:

  * 1.

The fish ate the worm. It was tasty.

  * 2.

The fish ate the worm. It was hungry.

we can merge all subjects and object, and get the following frequency
information:

  * 1.

Frequency(‘X eats Y’, co-occur, ‘X is hungry’) = 18 and
Frequency(‘X eats Y’, co-occur, ‘Y is hungry’) = 1;

  * 2.

Frequency(‘X eats Y’, co-occur, ‘X is tasty’) = 0 and
Frequency(‘X eats Y’, co-occur, ‘Y is tasty’) = 7.

These numbers reflect the aforementioned second-order selectional preferences
based on which we can solve the questions. Although such aggregation has been
shown to be useful for the Winograd Schema Challenge, the collected partial
information can be too coarse. It only aggregates all information to be X or
Y. However, in real-world applications, we also need to know the following
question for fine-grained concepts other than humans

  * 1.

Frequency(‘Company acquires Startups’, ResultIn, ‘Stock increases’)=?

Therefore, a principled way of performing the conceptualization of instances
and partial concept information aggregation is needed. Thus, we propose to
leverage another existing knowledge base, Probase [43], to perform
conceptualization [16, 17] over entities that Probase can recognize. For
example, after observing that both “having a cat” and “having a dog” can cause
“being happy,” and with the help of Probase, we can conceptualize and
aggregate “having a cat” and “having a dog” to be “having a pet,” we can then
conclude that “having a pet” can cause “being happy.”

One thing worth mentioning is that the main methodology of ASER is that after
the aggregation, the more heavily weighted (i.e., frequent) eventualities or
edges make more sense than the less heavily weighted ones. As a result, when
we conduct the conceptualization, we do not need to consider the context
because other eventualities exist, and after the aggregation, the more heavily
weighted ones will still make more sense. For example, given the eventuality
“I eat apple,” we do not need to worry about which one of “fruit” and
“company” we should conceptualize “apple” to because we will see many other
eventualities related to fruit such as “I eat banana” and “I eat orange.” In
the end, the overall weight of “I eat fruit” will still be much higher than “I
eat company.”

##  4 Overview of ASER

ASER is a hybrid graph combining a hypergraph
$\\{{\mathcal{V}},{\mathcal{E}}\\}$ where each hyperedge is constructed over
vertices, and a traditional graph $\\{{\mathcal{E}},{\mathcal{R}}\\}$ where
each edge is built among eventualities. For example, $E_{h}$=(I, am, hungry)
and $E_{t}$=(I, eat, anything) are eventualities, where we omit the internal
dependency structures for brevity. They have a relation $\langle
E_{h},\texttt{Result},E_{t}\rangle$, where Result is the relation type. We
devise the formal definition of ASER as below.

###### Definition 1

ASER KG is a hybrid graph ${\mathcal{H}}$ of eventualities $E$’s. Each
eventuality $E$ is a hyperedge linking to a set of vertices $v$’s. Each vertex
$v$ is a word in the vocabulary. We define $v\in{\mathcal{V}}$ in the vertex
set and $E\in{\mathcal{E}}$ in the hyperedge set.
${\mathcal{E}}\subseteq{\mathcal{P}}({\mathcal{V}})\setminus\\{\emptyset\\}$
is a subset of the power set of ${\mathcal{V}}$. We also define a relation
$R_{i,j}\in{\mathcal{R}}$ between two eventualities $E_{i}$ and $E_{j}$, where
${\mathcal{R}}$ is the relation set. Each relation has a type
$T\in{\mathcal{T}}$ where ${\mathcal{T}}$ is the type set. Overall, we have
ASER KG
${\mathcal{H}}=\\{{\mathcal{V}},{\mathcal{E}},{\mathcal{R}},{\mathcal{T}}\\}$.

###  4.1 Eventuality

Unlike named entities or concepts, which are noun phrases, eventualities are
usually expressed as verb phrases, which are more complicated in structure.
Our definition of eventualities is built upon the following two assumptions:
(1) Syntactic English patterns are relatively fixed and consistent; (2) The
eventuality’s semantic meaning is determined by the words it contains. To
avoid the extracted eventualities being too sparse, we use words fitting
specific patterns rather than a whole sentence to represent an eventuality.
Also, to make sure the extracted eventualities have complete semantics, we
retain all necessary words extracted by patterns rather than those simple
verbs or verb-object pairs in sentences. The selected patterns are shown in
Table 3. For example, for the eventuality (dog, bark), we have a relation
nsubj between the two words to indicate that there is a subject-of-a-verb
relation in between. We now formally define an eventuality as follows.

###### Definition 2

An eventuality $E$ is a hyperedge linking multiple words
$\\{v_{1},\ldots,v_{N}\\}$, where $N$ is the number of words in eventuality
$E$. Here, $v_{1},\ldots,v_{N}\in{\mathcal{V}}$ are all in the vocabulary. A
pair of words in $E$ $(v_{i},v_{j})$ may follow a syntactic relation
$e_{i,j}$. The weight of $E$, denoted as $w_{E}^{(e)}$, is defined by the
frequencies of appearance in the whole corpora.

Table 3: Selected eventuality patterns (“v” stands for normal verbs other than “be”, “be” stands for “be” verbs, “n” stands for nouns, “a” stands for adjectives, and “p” stands for prepositions.), Code (to save space, we create a unique code for each pattern and will use that in the rest of this paper), and the corresponding examples.  Pattern | Code |  Example
---|---|---
$n_{1}$-nsubj-$v_{1}$ | s-v |  “The dog barks”
$n_{1}$-nsubj-$v_{1}$-dobj-$n_{2}$ | s-v-o |  “I love you”
$n_{1}$-nsubj-$v_{1}$-xcomp-$a$ | s-v-a |  “He felt ill”
$n_{1}$-nsubj-$v_{1}$-xcomp-$v_{2}$ | s-v-v |  “I want to go”
$n_{1}$-nsubj-($v_{1}$-iobj-$n_{2}$)-dobj-$n_{3}$ | s-v-o-o |  “You give me the book”
$n_{1}$-nsubj-$v_{1}$-xcomp-$v_{2}$-dobj-$n_{2}$ | s-v-v-o |  “I want to eat the apple”
$n_{1}$-nsubj-($v_{1}$-dobj-$n_{2}$)-xcomp-$v_{2}$-dobj-$n_{3}$ | s-v-o-v-o |  “I ask you to help us”
$n_{1}$-nsubj-($v_{1}$-dobj-$n_{2}$)-xcomp-($v_{2}$-iobj-$n_{3}$)-dobj-$n_{4}$ | s-v-o-v-o-o |  “president urges the congress to make her citizen”
$n_{1}$-nsubj-$a_{1}$-cop-$be$ | s-be-a |  “The dog is cute”
$n_{1}$-nsubj-$n_{2}$-cop-$be$ | s-be-o |  “He is a boy”
$n_{1}$-nsubj-$v_{1}$-xcomp-$n_{2}$-cop-$be$ | s-v-be-o |  “I want to be a hero”
$n_{1}$-nsubj-$v_{1}$-xcomp-$a_{1}$-cop-$be$ | s-v-be-a |  “I want to be slim”
$n_{1}$-nsubj-($v_{1}$-iobj-$n_{2}$)-xcomp-$n_{3}$-cop-$be$ | s-v-o-be-o |  “I want her to be hero”
$n_{1}$-nsubj-($v_{1}$-iobj-$n_{2}$)-xcomp-$a_{1}$-cop-$be$ | s-v-o-be-a |  “I want her to be happy”
$there$-expl-$be$-nsubj-$n_{1}$ | there-be-o |  “There is an apple”
$n_{1}$-nsubjpass-$v_{1}$ | spass-v |  “The bill is paid”
$n_{1}$-nsubjpass-$v_{1}$-dobj-$n_{2}$ | spass-v-o |  “He is served water”
$n_{1}$-nsubjpass-$v_{1}$-xcomp-$v_{2}$-dobj-$n_{2}$ | spass-v-v-o |  “He is asked to help us”

We use patterns from dependency parsing to extract eventualities $E$’s from
unstructured large-scale corpora. Here $e_{i,j}$ is one of the relations that
dependency parsing may return. Although the recall is sacrificed in this way,
our patterns are of high precision, and we use massive corpora to extract as
many eventualities as possible. This strategy is also shared with many other
modern KGs [36, 37, 42, 43].

Table 4: Eventuality relation types between two eventualities $E_{h}$ and $E_{t}$ and explanations. Relation |  Explanation
---|---
$\langle E_{h},\texttt{Precedence},E_{t}\rangle$ |  $E_{h}$ happens before $E_{t}$.
$\langle E_{h},\texttt{Succession},E_{t}\rangle$ |  $E_{h}$ happens after $E_{t}$.
$\langle E_{h},\texttt{Synchronous},E_{t}\rangle$ |  $E_{h}$ happens at the same time as $E_{t}$.
$\langle E_{h},\texttt{Reason},E_{t}\rangle$ |  $E_{h}$ happens because $E_{t}$ happens.
$\langle E_{h},\texttt{Result},E_{t}\rangle$ |  If $E_{h}$ happens, it will result in the happening of $E_{t}$.
$\langle E_{h},\texttt{Condition},E_{t}\rangle$ |  Only when $E_{t}$ happens, $E_{h}$ can happen.
$\langle E_{h},\texttt{Contrast},E_{t}\rangle$ |  $E_{h}$ and $E_{t}$ share a predicate or property and have significant difference on that property.
$\langle E_{h},\texttt{Concession},E_{t}\rangle$ |  $E_{h}$ should result in the happening of $E^{\prime}$, but $E_{t}$ indicates the opposite of $E^{\prime}$ happens.
$\langle E_{h},\texttt{Conjunction},E_{t}\rangle$ |  $E_{h}$ and $E_{t}$ both happen.
$\langle E_{h},\texttt{Instantiation},E_{t}\rangle$ |  $E_{t}$ is a more detailed description of $E_{h}$.
$\langle E_{h},\texttt{Restatement},E_{t}\rangle$ |  $E_{t}$ restates the semantics meaning of $E_{h}$.
$\langle E_{h},\texttt{Alternative},E_{t}\rangle$ |  $E_{h}$ and $E_{t}$ are alternative situations of each other.
$\langle E_{h},\texttt{ChosenAlternative},E_{t}\rangle$ |  $E_{h}$ and $E_{t}$ are alternative situations of each other, but the subject prefers $E_{h}$.
$\langle E_{h},\texttt{Exception},E_{t}\rangle$ |  $E_{t}$ is an exception of $E_{h}$.
$\langle E_{h},\texttt{Co-Occurrence},E_{t}\rangle$ |  $E_{h}$ and $E_{t}$ appear in the same sentence.

###  4.2 Eventuality Relation

For relations among eventualities, as introduced in Section 1, we follow
PDTB’s [30] definition of relations between sentences or clauses but simplify
them to eventualities. Following the CoNLL 2015 discourse parsing shared task
[55], we select 14 discourse relation types and an additional co-occurrence
relation to build our knowledge graph.

###### Definition 3

A relation $R$ between a pair of eventualities $E_{h}$ and $E_{t}$ has one of
the following types $T\in{\mathcal{T}}$ and all types can be grouped into five
categories: Temporal (including Precedence, Succession, and Synchronous),
Contingency (including Reason, Result, and Condition), Comparison (including
Contrast and Concession), Expansion (including Conjunction, Instantiation,
Restatement, Alternative, ChosenAlternative, and Exception), and Co-
Occurrence. The detailed definitions of these relation types are shown in
Table 4. The weight of $R=\langle E_{h},T,E_{t}\rangle$, which is denoted as
$w_{R}^{(r)}$, is defined by the sum of weights of $\langle
E_{h},T,E_{t}\rangle$ that appear in the whole corpora.

###  4.3 ASER Conceptualization

As aforementioned, to overcome the challenge that trivial commonsense is often
omitted in humans’ communication, we propose to leverage the conceptualization
to generalize the knowledge about observed eventualities to unseen ones. For
each eventuality $E\in{\mathcal{E}}$, whose weight is $w_{E}^{(e)}$, and we
can conceptualize $E$ to $E^{\prime}$ with confidence
$w_{E,E^{\prime}}^{(c)}$, we will get a new conceptualized eventuality ${\bf
E}^{\prime}$ with the weight $w_{E^{\prime}}^{(c)}=w_{E}^{(e)}\cdot
w_{E,E^{\prime}}^{(c)}$. Similarly, assume that an edge $R\in{\mathcal{R}}$ is
$\langle E_{h},T,E_{t}\rangle$ and its weight is $w_{R}^{(r)}$, and $E_{h}$
and $E_{t}$ can be conceptualized to $E_{h}^{\prime}$ and $E_{t}^{\prime}$
with the confidence $w_{E_{h},E_{h}^{\prime}}^{(c)}$ and
$w_{E_{t},E_{t}^{\prime}}^{(c)}$, respectively. We can then get a new
conceptualized edge $\langle E_{h}^{\prime},T,E_{t}^{\prime}\rangle$ with the
weight $w_{R}^{(r)}\cdot w_{E_{h},E_{h}^{\prime}}^{(c)}\cdot
w_{E_{t},E_{t}^{\prime}}^{(c)}$. Details about how to leverage an external
hypernym knowledge base to get the conceptualized eventualities and determine
the confidence scores are presented in Section 5.

###  4.4 KG Storage

In total, we use the following three tables of the SQLite database to store
ASER.

  * 1.

Eventuality: As aforementioned, all eventualities in ASER are dependency
graphs, where vertices are the words and edges are dependency relations. We
generate unique “eids” for eventualities by hashing their words, pos-tags, and
dependencies and store eventualities in an Eventuality table with SQLite
database where “eids” is the key, and patterns, verb(s), skeleton words,
words, pos-tags, dependencies, and frequencies are the other attribute
columns.

  * 2.

Concept: To effectively distinguish the eventualities before and after the
conceptualization, we store eventualities created by the conceptualization
step in another Concept table and denote the id as “cid.” As the dependency
edges are inherited from the original eventualities, we only hash the
conceptualized words to generate the “cids.” For each conceptualized
eventuality, we store its “cid,” pattern, frequency, and “eids” of the
original eventualities.

  * 3.

Relations: We store the relations between eventualities in the Relations
table. For each pair of eventualities (i.e., $E_{h}$ and $E_{t}$), if there is
at least an edge between them, we will create an instance and generate a “rid”
for them by hashing the concatenation of their “eids.” For the storage
efficiency and retrieval feasibility, we store all edges and the associated
weights between $E_{h}$ and $E_{t}$ as well as the eventuality ids of $E_{h}$
and $E_{t}$ in that instance.

##  5 ASER Construction

In this section, we introduce the ASER construction details.

Figure 3: ASER construction framework. The extraction and the
conceptualization process are shown in the orange dash-dotted and green dashed
boxes, respectively. The blue database is Probase and three gray databases are
the resulted ASER.

###  5.1 System Overview

The overall framework of our extraction system is shown in Figure 3. After
collecting the raw corpora, we first preprocess the texts with the dependency
parser. Then we perform eventuality extraction with pattern matching. We
collect sentences and adjacent sentence pairs that contain more than two
eventualities into an instance collection. After that, we extract discourse
relations from these candidate instances with the help of an explicit
discourse parser [14]. Considering that the discourse parser’s discourse
argument span might not be identical to the extracted eventualities, we apply
token-based Simpson’s similarity between the arguments spans and eventualities
to determine whether the discourse arguments are enough to represent the
meaning of the extracted eventualities. We only keep the extraction results
with the Simpson’s similarity larger than 0.8. After the initial ASER
construction, we leverage the IS-A relations between nouns and named entities
from Probase [43] to conduct the conceptualization. In the end, we aggregate
relations between conceptualized eventualities by retrieving head and tail
eventualities from the conceptualized eventuality database and the eventuality
relation database. In the following sub-sections, we will introduce each part
of the system separately.

###  5.2 Corpora

To ensure the broad coverage of ASER, we select corpora from different
resources (reviews, news, forums, social media, movie subtitles, e-books) as
the raw data. The details of these datasets are as follows.

$\bullet$ Yelp: Yelp is a social media platform where users can write reviews
for businesses (e.g., restaurants). The latest release of the Yelp
dataset666https://www.yelp.com/dataset/challenge contains over five million
reviews.

$\bullet$ New York Times (NYT): The NYT [56] corpus contains over 1.8 million
news articles from the NYT throughout 20 years (1987 - 2007).

$\bullet$ Wiki: Wikipedia is one of the largest free knowledge datasets. To
build ASER, we select the English version of
Wikipedia.777https://dumps.wikimedia.org/enwiki/

$\bullet$ Reddit: Reddit is one of the largest online forums. In this work, we
select the anonymized post
records888https://www.reddit.com/r/datasets/comments/3bxlg7 over one period
month.

$\bullet$ Movie Subtitles: The movie subtitles corpus was collected by [57],
and we select the English subset, which contains subtitles for more than 310K
movies.

$\bullet$ E-books: The last resource we include is the free English electronic
books from Project Gutenberg.999https://www.gutenberg.org/

We merge these resources as a whole to perform the knowledge extraction. The
detailed statistics are presented in Table 5.

Table 5: Statistics of used corpora. (M means millions and G means Gigabytes.) Name | # Sentences | # Tokens | Corpus Size | Category
---|---|---|---|---
YELP | 54.5 M | 838.8 M | 2.5G | Reviews
NYT | 49.8 M | 1,179.4 M | 3G | News
Wiki | 110.6 M | 2,435.4 M | 13G | Knowledge
Reddit | 253.6 M | 3,371.3 M | 21G | Forum
Subtitles | 444.6 M | 3,229.4 M | 13G | Movie Scripts
E-books | 210.6 M | 3,610.0 M | 21G | Stories
Overall | 1,123.7 M | 14,664.2 M | 73.5G | -

###  5.3 Preprocessing

For each document, we aim to extract eventualities, relations between
eventuality, conceptualized eventualities, and relations between
conceptualized eventualities. Based on the consideration of the text parsing
complexity and quality, we parse each paragraph101010As the discourse parser
extracts discourse relations by the constituency tree of a sentence or trees
of adjacent sentences, parsing sentences one by one would miss or misclassify
some discourse relations. instead of a whole document with the CoreNLP
tool111111https://stanfordnlp.github.io/CoreNLP to acquire the lemmatized
tokens, pos-tags, named entities, the dependency graph, and the constituency
tree. Before parsing, we replace URLs with a special token
$\langle$URL$\rangle$ and drop tables in Reddit data.

###  5.4 Eventuality Extraction

Algorithm 1 Eventuality Extraction with One Pattern $p$

INPUT: Parsed dependency graph ${\mathcal{D}}$, center verb $v$, positive
dependency edges ${\mathcal{P}}_{p}^{(pos)}$, optional edges
${\mathcal{P}}_{p}^{(opt)}$, and negative edges ${\mathcal{P}}_{p}^{(neg)}$.
OUTPUT: extracted eventuality $E$.

1:Initialize eventuality edge list ${\mathcal{D}}^{\prime}$.

2:Set the center verb $v$ as the $v_{1}$ in the pattern $p$

3:for each connection $d$ (a relation and the associated word) in positive
dependency edges ${\mathcal{P}}_{p}^{(pos)}$ do

4: if find $d$ in ${\mathcal{D}}$ then

5: Append $d$ in ${\mathcal{D}}^{\prime}$.

6: else

7: Return null.

8: end if

9:end for

10:for each connection $d$ in optional dependency edges
${\mathcal{P}}_{p}^{(opt)}$ do

11: if find $d$ in ${\mathcal{D}}$ then

12: Append $d$ in ${\mathcal{D}}^{\prime}$.

13: end if

14:end for

15:for each connection $d$ in negative dependency edges
${\mathcal{P}}_{p}^{(neg)}$ do

16: if find $d$ in ${\mathcal{D}}$ then

17: Return null.

18: end if

19:end for

20:Build eventuality instance $E$ from ${\mathcal{D}}^{\prime}$.

21:Return $E$.

To ensure that all the extracted eventualities are semantically complete
without being too complicated, we design 18 patterns to extract the
eventualities via pattern matching. Each of the patterns contains three kinds
of dependency edges: positive dependency edges, optional dependency edges, and
negative dependency edges. All the positives edges are shown in Table 3. Six
more dependency relations (advmod, amod, nummod, aux, compound, and neg) are
optional dependency edges that can associate with any of the selected
patterns. We omit all optional edges in the table because they are the same
for all patterns. All other dependency edges are considered negative
dependency edges, designed to ensure all the extracted eventualities are
semantically complete and all the patterns are exclusive with each other. Take
sentence “I have a book” as an example, we will only select $\langle$“I,”
“have,” “book”$\rangle$ rather than $\langle$“I,” “have”$\rangle$ as the valid
eventuality, because “have”-dobj-“book” is a negative dependency edge for
pattern “s-v.”

To extract eventualities from sentence $s$, considering that $s$ may contain
multiple eventualities, we first split it into simple clauses based on the
constituency tree. To do so, besides the commonly used SBAR node, we also
follow previous discourse parsing systems [14] to use a connective classifier
to detect possible separators. As a result, we split sentences based on both
the subordinate conjunctions and connectives. After that, for each verb $v$ in
sentence $s$, we find the dependency graph ${\mathcal{D}}$ of the simple
clause that contains $v$. We then try to match ${\mathcal{D}}$ with all
patterns one by one. For each pattern, we put the verb $v$ as the starting
point (i.e., $v_{1}$ in the pattern) and then try to find all the positive
dependency edges. If we can find all the positive dependency edges around the
center verb, these match edges and words linked by these edges are considered
as potential edges and words of a valid eventuality. Next, other edges and
words are added via optional dependency edges. In the end, we will check if
any negative dependency edge can be found in the dependency graph. If not, we
will keep current edges and words as a valid eventuality. Otherwise, we will
disqualify it. The pseudo-code of the eventuality extraction algorithm is in
Algorithm 1. The time complexity of eventuality extraction is
$\mathcal{O}(|{\mathcal{S}}|\cdot\overline{|{\mathcal{D}}|}\cdot\overline{|{\mathcal{V}}^{(v)}|})$
where $|{\mathcal{S}}|$ is the number of sentences,
$\overline{|{\mathcal{D}}|}$ is the average number of dependency edges in a
dependency parse tree, and $\overline{|{\mathcal{V}}^{(v)}|}$ is the average
number of verbs in a sentence.

###  5.5 Eventuality Relation Extraction

Algorithm 2 Eventuality Relation Extraction

INPUT: Parsed constituency trees ${\mathcal{K}}_{1}$ and ${\mathcal{K}}_{2}$
from adjacent sentences.
OUTPUT: Extracted relations ${\mathcal{R}}$.

1:Initialize relation list ${\mathcal{R}}$ as empty.

2:Extract possible connectives ${\mathcal{C}}$ by a connective extractor given
${\mathcal{K}}_{1}$ and ${\mathcal{K}}_{2}$.

3:for each possible connective $c\in{\mathcal{C}}$ do

4: if two arguments of $c$ in the same sentence then

5: Extract $A_{1}$ and $A_{2}$ by a SS arguments extractor given $c$ and the
sentence.

6: else

7: Extract $A_{1}$ by a PS argument1 extractor given $c$ and
${\mathcal{K}}_{1}$.

8: Extract $A_{2}$ by a PS argument2 extractor given $c$ and
${\mathcal{K}}_{2}$.

9: end if

10: if $A_{1}$ is not null and $A_{2}$ is not null then

11: Classify the relation $y$ by a explicit relation classifier given $c$,
${\mathcal{K}}_{1}$, and ${\mathcal{K}}_{2}$

12: Find eventualities ${\mathcal{E}}_{h}$ that are extracted from $A_{1}$.

13: Find eventualities ${\mathcal{E}}_{t}$ that are extracted from $A_{2}$.

14: Set weight $w$ as $1/(|{\mathcal{E}}_{h}|\cdot|{\mathcal{E}}_{t}|)$.

15: for each eventuality $E_{h}$ in ${\mathcal{E}}_{h}$ do

16: for each eventuality $E_{t}$ in ${\mathcal{E}}_{t}$ do

17: Build relation instance $R=\langle E_{h},y,E_{t}\rangle$ with a weight
$w$.

18: Append $R$ in ${\mathcal{R}}$.

19: end for

20: end for

21: end if

22:end for

23:Return ${\mathcal{R}}$.

We then introduce how to extract the relations between eventualities.
Specifically, we employ an end-to-end discourse parser to extract the
discourse relations. The discourse parser’s job is to parse a piece of text
into a set of discourse relations between two adjacent or non-adjacent
discourse units. Take the sentence “I have a story book, but it is not
interesting.” as an example. Ideally, a good discourse parser extracts “I have
a story book” as arg1, “it is not interesting” as arg2, “but” as the
connective, and annotate the relation as “Contrast.” In our current pipeline,
we use the state-of-the-art discourse parser [14], which is pretrained on the
CoNLL 2015 Shared Task data (PDTB) [55]. From CoNLL 2015
results,121212https://www.cs.brandeis.edu/~clp/conll15st/results.html we can
find out that this discourse parser can achieve 90.00% and 90.79% F1 scores on
the test data from PDTB and the blind test data from Wikinews respectively on
the explicit relation classification, but performance drops to 42.72% and
34.45% on the implicit relation classification. Hence, to guarantee the
extraction quality, we only consider the explicit discourse relations. In
explicit discourse parsing, there are two situations: both arguments are in
the same sentence or not. As statistics shows that less than 0.1% cases that
arguments are located in non-adjacent sentences in the explicit part, we
simply assume arguments of which argument1 is located in the same sentence
(SS) and the previous sentence (PS). Specifically, the explicit discourse
parser is consist of five components: (1) connective extractor to identify
whether a word is a possible connective, (2) arg1 position classifier to
decide whether the arg1 is located in the same sentence as the connective $c$
or the previous sentence of $c$; (3) SS argument extractor to extract the
spans of two arguments in the same sentence; (4) PS argument extractor to
extract the spans of two arguments in adjacent sentences; (5) explicit
relation classifier to classify the relation type of $c$. Extractors in this
system are essentially binary classifiers to identify whether a word is a
connective or a part of any argument. The pseudo-code of eventuality relation
extraction algorithm is shown in Algorithm 2.

As the extracted arguments might not be identical as the extracted
eventualities, we use the Simpson’s similarity to determine whether the
discourse relations between arguments can be assigned to the extracted
eventualities:

$\displaystyle
w_{A,E}^{(sim)}=\text{Simpson}(A,E)=\frac{|{\mathcal{W}}_{A}\cup{\mathcal{W}}_{E}|}{\text{min}\\{|{\mathcal{W}}_{A}|,|{\mathcal{W}}_{E}|\\}},$
(1)

where $A$ is an argument, $E$ is an eventuality, ${\mathcal{W}}_{A}$ and
${\mathcal{W}}_{E}$ are token sets of $A$ and $E$, $|\cdot|$ is the size of a
token set. If the similarity $\text{Simpson}(A,E)\geq 0.8$, we consider the
argument-level relations relevant to $A$ can be assigned to the eventuality
$E$ with a weight $w_{A,E}^{(sim)}$, which is inversely proportional to the
size of all matched eventualities $|{\mathcal{E}}|$.

###  5.6 Enriching ASER with Conceptualization

We then introduce the conceptualization details. For each noun or pronoun in
the extracted eventualities, we will try to conceptualized it to a higher
level with the following steps. If it is a named entity, we will
conceptualized it to the corresponding NER tags. Specifically, we include the
13 NER types: “Time,” “Date,” “Duration,” “Money,” “Percent,” “Number,”
“Country,” “State or Province,” “City,” “Nationality,” “Person,” “Religion,”
“URL.” If it is a personal pronoun (e.g., “I”, “you”, or “they”), we will
conceptualize it to “PersonX.”131313If there are multiple people in the same
edge, we will distinguish them with “PersonX” and “PersonY” etc. As all
aforementioned conceptualization is designed by experts, we set the
conceptualization probability to be 1. If it is a regular noun, we will try to
conceptualize it with Probase [43]. Specifically, for each noun, we will
retrieve its top-five hypernyms (i.e., concepts) and the associated
probability from Probase.

Given an eventuality $E$ with $m$ tokens ${t_{1},t_{2},\cdots,t_{m}}$ to be
mapped into concept tokens, we conceptualize it to a conceptualized
eventuality $C$ with the probability:

$\text{Pr}(C|E)=\prod_{i=1}^{m}\text{Pr}(t_{i}^{(c)}|t_{i}^{(e)}).$ (2)

Here $t_{i}^{(c)}$ is the corresponding token-level concept for token
$t_{i}^{(e)}$. And $\text{Pr}(t_{i}^{(c)}|t_{i}^{(e)})$ is the likelihood for
$\langle t_{i}^{(e)},\texttt{IS-A},t_{i}^{(c)}\rangle$ provided by Probase or
1.0 if $t_{i}^{(e)}$ can be conceptualized with rules. For each conceptualized
eventuality $C$, we would have a list of eventualities ${\mathcal{E}}_{C}$
that can be conceptualized to it. We can then compute the overall weight of
$C$ with Eq. 3, where $w_{E}^{(e)}$ is the weight of $E$.

$w_{C}^{(c)}=\sum_{E\in{\mathcal{E}}_{C}}\text{Pr}(C|E)\cdot w_{E}^{(e)}.$ (3)

Figure 4: Demonstration of the conceptualized ASER. The eventualities are
conceptualized and connected with weighted edges. Each concept contains it’s
projections to specific eventualities.

We then introduce how to construct the edges between a conceptualized
eventuality $C$ and an original eventuality $E$. For any
$E^{\prime}\in{\mathcal{E}}_{C}$, if there is an edge $\langle
E^{\prime},T,E\rangle$ or $\langle E,T,E^{\prime}\rangle$, we can then
construct a new edge $\langle C,T,E\rangle$ or $\langle E,T,C\rangle$ with the
weight based on Eq. 4 or Eq. 5, respectively, where $w_{R}^{(r)}$ means of
weight of the relation $R$.

$\displaystyle w_{\langle C,T,E\rangle}^{(r)}$
$\displaystyle=\sum_{E^{\prime}\in{\mathcal{E}}_{C}}{\text{Pr}(C|E^{\prime})\cdot
w_{\langle E^{\prime},T,E\rangle}^{(r)}},$ (4) $\displaystyle w_{\langle
E,T,C\rangle}^{(r)}$
$\displaystyle=\sum_{E^{\prime}\in{\mathcal{E}}_{C}}{w_{\langle
E,T,E^{\prime}\rangle}^{(r)}\cdot\text{Pr}(C|E^{\prime})}.$ (5)

As each conceptualized eventuality is correlated with a set of original
eventualities, we need to aggregate the edges between the original
eventualities to build the connections between the conceptualized ones.
Formally, given two conceptualized eventualities $C_{h}$ and $C_{t}$, we first
retrieve all related original edges $\\{\langle
E_{h},T,E_{t}\rangle|E_{h}\in{\mathcal{E}}_{C_{h}},E_{t}\in{\mathcal{E}}_{C_{t}}\\}$.
Then we calculate the weight $\text{Pr}(C_{h}|E_{h})\cdot w_{\langle
E_{h},T,E_{t}\rangle}^{(r)}\cdot\text{Pr}(C_{t}|E_{t})$ for each related edge.
Finally, we aggregate all weights to construct the weight as Eq. 6 for the
edge between $C_{h}$ and $C_{t}$ associated with the relation type $T$.

$w_{\langle
C_{h},T,C_{t}\rangle}^{(r)}=\sum_{E_{h}\in{\mathcal{E}}_{C_{h}}}\sum_{E_{t}\in{\mathcal{E}}_{C_{t}}}{\text{Pr}(C_{h}|E_{h})\cdot
w_{\langle E_{h},T,E_{t}\rangle}^{(r)}\cdot\text{Pr}(C_{t}|E_{t})}$ (6)

An illustration of the conceptualized ASER is shown in Figure 4. We can get
the conceptualized eventuality “PersonX be hungry” from “I am hungry,” “they
are hungry,” and other extracted eventualities with $Pr(C|E)=1.000$ because
their subjects (pronouns or names) are mapped to the token-level concept
“PersonX.” As a comparison, “PersonX order Meat” is not a deterministic
eventuality: it can be conceptualized from “I order chicken” with
$\text{Pr}(\textit{PersonX}\text{ order }\textit{Meat}|\text{I order
chicken})=\text{Pr}(\textit{PersonX}|\text{I})\cdot\text{Pr}(\textit{Meat}|\text{chicken})=1.000\cdot
0.069=0.069$, “I order pork rib” with $\text{Pr}(\textit{PersonX}\text{ order
}\textit{Meat}|\text{I order pork
rib})=\text{Pr}(\textit{PersonX}|\text{I})\cdot\text{Pr}(\textit{Meat}|\text{pork
rib})=1.000\cdot 0.120=0.120$, or other extracted ones. Based on Eq. 3, after
aggregating all weights together, we can get the concept weights for “PersonX
be hungry” and “PersonX order Meat” are 1389.000 and 27.705, respectively. As
for the relations between the two conceptualized eventualities, we find
$w^{(r)}_{\langle\text{I am hungry},\texttt{Result},\text{I order orange
chicken}\rangle}=0.125$ and $w^{(r)}_{\langle\text{I am too
hungry},\texttt{Result},\text{I order the fried chicken}\rangle}=1.000$, so
the relation weight in the concept-level is calculated as follows:

$\displaystyle w^{(r)}_{\langle\textit{PersonX}\text{ be
hungry},\texttt{Result},\textit{PersonX}\text{ order }\textit{Meat}\rangle}$
$\displaystyle=\text{Pr}(\textit{PersonX}\text{ be hungry}|\text{I am
hungry})\cdot 0.125\cdot\text{Pr}(\textit{PersonX}\text{ order
}\textit{Meat}|\text{I order orange chicken})$
$\displaystyle\quad+\text{Pr}(\textit{PersonX}\text{ be hungry}|\text{I am too
hungry})\cdot 1.000\cdot\text{Pr}(\textit{PersonX}\text{ order
}\textit{Meat}|\text{I order the fried chicken})$ $\displaystyle=1.000\cdot
0.125\cdot 0.069+1.000\cdot 1.000\cdot 0.069$ $\displaystyle=0.077.$

Similarly, we can calculate all weights among conceptualized eventualities
associated with different relation types. One thing worth mentioning is that
the relation weights depend not only on the relation weights in the extracted
knowledge bases but also on the conceptualization probabilities.

###  5.7 ASER Building Example

Figure 5: ASER building example. The eventuality extraction, the relation
extraction, and the conceptualization process are shown in green, violet, and
orange colors, respectively. For the clear representation, for each
conceptualized eventuality, we only show the skeleton words and hide the
optional ones. We only show the two conceptualized eventualities with the
highest probabilities for each extracted eventuality. As a demonstration, the
weights of eventualities and relations are calculated from this example
instead of the whole KG.

At the end of this section, we use an example to demonstrate the whole
extraction pipeline. As shown in Figure 5, given a sentence “My army will find
your boat. In the meantime, I’m sure we could find you suitable
accommodations.,”141414This case comes from Movie Subtitles. our system will
first detect the possible connective “meantime” and split this text into four
simple clauses: “My army will find your boat,” “In the,” “I’m sure,” and “we
could find you suitable accommodations” with the constituency parsing. After
that, our system will leverage the patterns designed in Table 3 to extract
eventualities from the raw text by Algorithm 1. Simultaneously, two arguments
“My army will find your boat” and “we could find you suitable accommodations”
are extracted by argument extractors. The discourse parsing system predicts
the corresponding discourse relation as Synchronous. As the first and last
extracted eventualities can perfectly match the extracted arguments, we then
create an edge $\langle$ “my army will find your boat,” Synchronous, “we could
find you suitable accommodations” $\rangle$. We also create an edge $\langle$
“I am sure,” Co-Occurrence, “we could find you suitable accommodations”
$\rangle$ because the two eventualities appear in the same sentence. After
extracting the original eventualities and edges, we then try to expand it with
the conceptualization.151515In the real system, we first extract the original
ASER, and then apply the conceptualization step over the whole KG. The
presented single sentence example is just for the demonstration. For example,
“I am sure” can be directly conceptualized as “PersonX be sure” directly
because “I” is a personal pronoun. As both of the other two eventualities
contain regular nouns (i.e., “army”), these eventualities can be
conceptualized to multiple eventualities. After checking Probase, we find out
that “army” can be conceptualized to “Institution” and “Organization” with the
weights 0.058 and 0.038, “boat” can be conceptualized to “Vehicle” and “Item”
with the weights 0.059 and 0.049, “accommodation” can be conceptualized to
“Service” and “Facility” with the weights 0.056 and 0.019, respectively. We
show the two most likely results for each original eventuality (if it has
multiple possible conceptualization results) in Figure 5. In the end, we can
construct edges between conceptualized eventualities, where the weights are
the product of conceptualization probabilities, e.g., $\langle$ “Institution
find boats,” Synchronous, “PersonX find PersonY Service” $\rangle$ with the
weight $0.058\times 0.056=0.003$, $\langle$ “PersonX be sure,” Co-Occurrence,
“PersonX find PersonY Facility” $\rangle$ with the weight $1.000\times
0.019=0.019$.

##  6 ASER Statistics

Table 6: Statistics of the eventuality extraction. # Eventuality and # Unique mean the total number and the unique number of extracted eventualities using corresponding patterns or conceptualized eventualities from them.  Pattern | ASER (full) | ASER (core) | ASER (concept)
---|---|---|---
# Eventuality | # Unique | # Eventuality | # Unique | # Unique
s-v | 351,082,855 | 100,645,728 | 260,663,083 | 14,337,769 | 1,022,415
s-v-o | 284,103,317 | 159,948,356 | 139,031,585 | 18,100,360 | 8,252,653
s-v-a | 11,546,768 | 6,149,584 | 5,951,980 | 752,468 | 139,087
s-v-v | 24,549,946 | 11,129,566 | 14,624,526 | 1,591,424 | 216,413
s-v-o-o | 6,154,685 | 3,789,253 | 2,765,728 | 460,526 | 514,084
s-v-v-o | 29,445,708 | 18,659,717 | 12,720,497 | 2,187,577 | 1,482,783
s-v-o-v-o | 3,863,478 | 2,674,229 | 1,462,883 | 288,326 | 522,613
s-v-o-v-o-o | 91,532 | 59,290 | 40,428 | 8,499 | 18,461
s-be-a | 79,235,136 | 29,845,112 | 52,068,570 | 3,733,978 | 465,747
s-be-o | 98,411,474 | 53,503,410 | 49,979,659 | 6,337,042 | 2,312,209
s-v-be-a | 1,927,990 | 982,438 | 1,035,864 | 123,263 | 29,738
s-v-be-o | 2,322,890 | 1,574,896 | 909,250 | 184,298 | 139,239
s-v-o-be-a | 277,087 | 191,973 | 100,917 | 18,793 | 6,151
s-v-o-be-o | 307,031 | 231,289 | 95,815 | 22,411 | 32,796
there-be-o | 16,021,849 | 6,642,438 | 10,013,628 | 953,041 | 39,500
spass-v | 61,524,872 | 38,270,144 | 25,935,769 | 3,498,516 | 276,817
spass-v-o | 5,519,982 | 4,129,709 | 1,677,244 | 330,229 | 154,410
spass-v-v-o | 257,004 | 221,820 | 46,475 | 11,738 | 14,901
Overall | 976,643,604 | 438,648,952 | 579,123,901 | 52,940,258 | 15,640,017

(a) Extracted eventualities

(b) Conceptualized eventualities

Figure 6: Eventuality distributions.  Table 7: Statistics of the eventuality relation extraction.  Relation | ASER (full) | ASER (core) | ASER (concept)
---|---|---|---
Precedence | 14,058,213 | 1,790,016 | 4,798,015
Succession | 4,939,291 | 663,183 | 1,963,820
Synchronous | 19,464,898 | 3,123,042 | 8,013,943
Reason | 9,775,829 | 2,205,076 | 6,439,128
Result | 16,153,925 | 2,012,311 | 6,718,666
Condition | 18,052,484 | 3,160,271 | 8,063,967
Contrast | 59,333,901 | 8,655,661 | 24,978,311
Concession | 5,684,395 | 477,155 | 1,499,276
Conjunction | 82,121,343 | 13,978,907 | 45,597,200
Instantiation | 1,278,381 | 18,496 | 93,266
Restatement | 1,304,095 | 65,753 | 242,301
Alternative | 3,539,892 | 583,174 | 123,883
ChosenAlternative | 647,228 | 35,406 | 1,843,140
Exception | 106,000 | 20,155 | 93,412
Co-Occurrence | 412,054,590 | 49,232,161 | 113,744,814
Overall | 648,514,465 | 86,020,767 | 224,213,142

In total, we collect 976,643,604 eventualities from the raw documents. We
filter those low-frequency eventualities that only appear once and retain
52,940,258 unique eventualities in ASER (core). From Table 6, we can find the
“s-v” and “s-v-o” are the most frequent patterns. On the other hand, even
though those complex patterns appear relatively less frequently, thanks to the
large scale of ASER, they still appear thousands to millions of times.

The original eventuality distribution is presented in Figure 6(a). In general,
the distribution follows Zipf’s law, where only a small number of
eventualities appear many times while the majority of eventualities appear
only a few times. To better illustrate the distribution of eventualities, we
also show several representative eventualities along with their weights, and
we have two observations. First, eventualities which can be used in general
cases, like “I think (7,501,444)” and “I know” (4,267,911) appear much more
times than other eventualities. Second, eventualities in ASER are more closely
related to our daily life like “I sleep (18,347)” or “food is tasty (1,828)”
rather than domain-specific ones such as “I learn python (16).”

To achieve the balance between the quality and quantity of conceptualization
results, we apply the conceptualization over eventualities whose frequencies
are no less than five. As shown in Table 6, after the conceptualization, we
get 15,640,017 more unique eventualities. It is obvious that patterns with
more nouns (e.g., “s-v-o,” “s-v-v-o,” “s-be-o”) dominate the conceptualized
eventualities. The reason is that the conceptualization is only designed for
nouns, and each noun phrase would be replaced with a general noun phase if
such hypernym relation appears in Probase. For conceptualized eventualities,
we can observe a similar distribution in Figure 6(b). The top three
conceptualized eventualities are “PersonX know” (16,478,603.0), “PersonX
think” (14,117,254.0), and “PersonX say” (12,113,913.0). Although “I think”
(7,501,444) appears the most in the raw data (“I know” appears 3,447,429 times
in the raw data), but “you think” (1,444,333), “he thinks” (314,806), “they
thinks” (205,432), “we think” (196,633), “it thinks” (174,729), “she thinks”
(142,628), etc. appear much less than “you know” (4,726,264), “he knows”
(396,013), “they know” (260,656), “we know” (457,115), “it knows” (247,409),
“she knows” (190,803), etc., respectively. Finally, the weight of “PersonX
know” exceeds that of “PersonX think.”

For relations, as shown in Table 7, we collect 648,514,465 unique relations
from six data resources across different categories. To reduce noises in
parsing and extraction, we also filter out relations that
$\sum_{T^{\prime}\in{\mathcal{T}}}{w_{\langle
E_{h},T^{\prime},E_{t}\rangle}^{(r)}}<=1$ where $E_{h}$ and $E_{t}$ are the
head eventuality and the tail eventuality. Furthermore, if the head or the
tail is filtered out by eventuality filtering, the relation is also dropped.
Finally, we keep 86,020,767 unique relations in ASER (core), among which there
are 36,788,606 relations belonging to 14 discourse relation types depending on
the connectives and arguments, like Conjunction (e.g., “and”), Contrast (e.g.,
“but”), Condition (e.g., “if”), Synchronous (e.g., “meanwhile”), Reason (e.g.,
“because”), Result (e.g., “so”). When we filter out more low-frequency
eventualities, the number of relations decreases slightly. For example, when
we keep high-frequency eventualities whose frequencies are no less than five,
26.0% of eventualities (13,766,746) and 61.5% of relations (88,629,385) are
preserved. We apply the conceptualization over these preserved eventualities
and relations based on quantity and quality considerations. Finally, we obtain
15,640,017 unique conceptualized eventualities and 224,213,142 relations
between these conceptualized eventualities. In total, we have about 26 times
more relations between conceptualized eventualities than original
eventualities.

To better understand the distributions of extracted and conceptualized
knowledge, we show the number of eventualities and edges over different
filtering thresholds in Figure 7(a) and Figure 7(b), respectively. For the
extracted knowledge, the number of eventualities and relations decreases
exponentially when the threshold ranges from 5 to 100. For the conceptualized
knowledge, the rate of diminishing is even larger. When the threshold is less
than 10, the conceptualized eventuality size is greater than the original
size. But it is significantly less than the size of extracted eventualities as
the threshold is larger than 10. On the other hand, the number of
conceptualized eventuality relations consistently exceeds the original
relation size, which results in a denser conceptualized knowledge graph.

(a) Eventualities

(b) Relations

Figure 7: Distributions of eventualities and relations.

##  7 Evaluation

In this section, we leverage human annotation to evaluate the quality of ASER
from the following perspectives:

  1. 1.

Eventuality Extraction: We first evaluate how well the extracted eventualities
can represent the original sentence’s semantics. For example, if the original
sentence is “The kid goes to study,” eventuality “kid-go-to-study” with
pattern “s-v-v” can fully represent the semantics, but eventuality “kid-go”
with the pattern “s-v” cannot. We show the percentage of all extracted
eventualities that can fully represent the original sentences’ core semantic
based on different eventuality patterns.

  2. 2.

Lower-order Selectional Preference: Besides the extraction quality, we also
care about how well the eventuality statistics in ASER can reflect human’s
selectional preference. For example, the frequency of “I eat food” should be
higher than “I eat house.” As such preference appears inside eventualities, we
denote them as the lower-order selectional preference.

  3. 3.

Discourse Extraction: After evaluating the eventualities, we assess how well
the extracted edges can correctly represent the discourse relations in the
original sentence. For example, assume that the original sentence is “he went
to school while I was still preparing the breakfast.” and we have successfully
extracted two eventualities “he went to school” and “I was preparing
breakfast,” the correct discourse relation between them should be
“Synchronous” rather than “Contrast.” In this evaluation, we report the
accuracy based on different discourse relations.

  4. 4.

Higher-order Selectional Preference: Last but not least, we annotate whether
edge frequencies in ASER can reflect the higher-order selectional preference
among eventualities or not. For example, the frequency of “I am
hungry”-Result-“I eat food” should be larger than “I am hungry”-Reason-“I eat
food.”

Evaluation details and result analysis are as follows.

###  7.1 Eventuality Extraction

Figure 8: Human annotation of eventuality extraction quality.

To evaluate the correctness of the selected eventuality patterns and the
effectiveness of the extraction algorithm, we first employ the Amazon
Mechanical Turk platform (MTurk)161616https://www.mturk.com/ to evaluate the
quality of eventuality extraction. We randomly select 50 extracted
eventualities for each eventuality pattern and then provide these extracted
eventualities along with their original sentences to the annotators. For each
pair of eventuality and sentences, the annotators are asked to label whether
the extracted eventuality phrase can fully and precisely represent the
original sentences’ semantic meaning. If so, they should label them with
“Valid.” Otherwise, they should label it with “Not Valid.” For each
eventuality, we invite six workers to label, and if at least four of them
label it as “Valid,” we will consider it valid. In total, we collected 5,400
annotations. To ensure high annotation quality, we require all the annotators
to be the master annotator on the MTurk.

The annotation results are shown in Figure 8. From the result, we can see that
all patterns achieve over 90% accuracy, which demonstrates the high quality of
the selected patterns and the association extraction algorithm. As introduced
in Algorithm 1, to guarantee the quality of extracted eventualities, we
require the extraction algorithm to be strict and selective. Specifically, if
there is an extra dependency edge not in the positive or possible relations of
a corresponding pattern, we will discard the whole sentence. By doing so, even
though we sacrifice the overall recall, we guarantee high accuracy. Luckily,
as our approach is unsupervised, we can remedy the recall problem with a
larger-scale corpus. Among the 18 patterns, we notice that the more complex
patterns tend to have relatively lower accuracy. This makes sense because the
more complex an extracted eventuality is, the more likely that some of the
words are redundant to the eventuality semantics. Thus the annotator may think
that the extracted eventuality is not elegant enough.

###  7.2 Low-order Selectional Preference

(a) High Frequency vs. Low Frequency (before).

(b) Exist vs. Non-exist (before).

(c) High Frequency vs. Low Frequency (after).

(d) (C) Exist vs. Non-exist (after).

Figure 9: Human annotation of lower selectional preference in ASER.
Experiments on the eventualities before and after the conceptualization are
denoted with (before) and (after), respectively. The green color indicates the
number of eventuality pairs that the more frequent eventuality makes more
sense. The purple color indicates the number of eventualities pairs the less
frequent eventuality makes more sense.

To evaluate whether the eventuality frequencies in ASER can reflect human’s
low-order selectional preference, we first compare the plausibility of more
frequent eventualities versus less frequent ones. For each eventuality
pattern, we randomly select 50 eventuality pairs such that they only have one-
word difference but with a significant frequency difference. Specifically, we
require the frequency of the high frequency one to be larger than five, which
is the medium frequency of all eventualities, and the frequency of the high
frequency one must be at least five times larger than the frequency of the low
frequency one. For each eventuality, we invite six annotators from MTruk to
ask them which one of the eventuality seems more plausible to them. If more
annotators agree that the more frequent one makes more sense, we will label
that pair as a positive correlation. On the other hand, if more annotators
agree that the less frequent makes more sense, we will label that pair as a
negative correlation. If the voting draws, we will label it as similar. As
this evaluation fails to consider the eventualities with the frequency zero
(i.e., they do not exist in ASER) and whether an eventuality exist or not is
also a good preference indicator, we add another evaluation to prove that. For
each eventuality pattern, we randomly select 50 eventualities. Then for each
of the eventualities, we randomly select a negative example by randomly
changing a word inside the eventuality with another word of the same postag
label such that their grammar structure is the same. We also conduct filtering
to guarantee the negative examples do not appear in ASER. Last but not least,
to show the influence of the conceptualization, we conduct the aforementioned
two experiments on both the original ASER before the conceptualization and the
final one after the conceptualization.171717For the experiment on the ASER
after the conceptualization, we only sample the conceptualized eventualities
and ignore the original ones.

We present the annotation results in Figure 9 The green color indicates the
number of eventuality pairs that the more frequent eventuality makes more
sense, and the purple color indicates the number of eventualities pairs the
less frequent eventuality makes more sense. From the result in Figure 9 (a),
we can see that more than 70% of the eventuality pairs as positively
correlated, which is consistent with the previous study on the correlation
between frequency and selectional preference [13]. At the same time, we also
observe that about 30% of the less frequent eventualities are also quite
plausible, which is mainly because the frequency of an eventuality is also
severely influenced by the rareness of the words inside the eventuality. For
example, the eventuality “I eat avocado” appears much less than “I eat apple”
because avocado is much rarer than apple rather than “I eat apple” makes more
sense than “eat avocado.” The results in Figure 9 (b) help prove that the low-
frequent eventualities still contain rich low-order selectional preference
because, for more than 90% of the pairs, the randomly extracted pairs in ASER
makes more sense than those out of ASER. Furthermore, the experimental results
in Figure 9 (c) and (d) show that even though the conceptualization process
significantly improves the coverage of ASER, it would not hurt the overall
quality. This is mainly because, during the conceptualization step, we
carefully design the new weights based on the original weight and the
confidence scores provided by Probase [43].

###  7.3 Relation Extraction

Figure 10: Human annotation of discourse relation extraction quality.

Besides the eventuality extraction, we also care about the extraction quality
of the discourse relations between eventualities. For each relation type, we
randomly select 50 edges and the corresponding sentences. We generate a
question for each pair of them by asking the annotators if they think the
extracted discourse relation can represent the correct relation in the
original sentence. If so, they should label it as “Valid.” Otherwise, they
should label it as “Not Valid.” Similar to the eventuality extraction
experiment, we invite six annotators for each edge. If more than four of them
agree that the extracted relation is “Valid,” we will consider it to be
“Valid.”

From the results in Figure 10 we can see that the overall accuracy is about
80%, which is consistent with the reported performance of the used discourse
relations extraction system [14]. Besides that, we also notice that the model
performance varies on different relation types. For example, the model tends
to perform well on simple types such as “Reason” and “Alternative” because the
popular connectives (i.e., “because” and “or”) are less ambiguous. As a
comparison, when the connective is more ambiguous (e.g., “while” for
“Synchronous”), the overall performance will drop.

###  7.4 Higher-order Selectional Preference

(a) High Frequency vs. Low Frequency (before).

(b) Exist vs. Non-exist (before).

(c) High Frequency vs. Low Frequency (after).

(d) Exist vs. Non-exist (after).

Figure 11: Human annotation of the higher selectional preference in ASER.
Experiments on the eventualities before and after the conceptualization are
denoted with (before) and (after), respectively. Green color indicates the
number of edge pairs that the more frequent edge makes more sense, and purple
color indicates the number of edge pairs the less frequent edge makes more
sense.

Finally, we evaluate whether the edge frequency in ASER can be used to reflect
human’s high-order selectional preference about eventualities. Similar to the
evaluation on the lower-order selectional preference, we conduct two
experiments (i.e., (1) High frequency vs. Low Frequency; (2) Exist vs. None-
exist) on ASER before and after the conceptualization.181818For the experiment
on the ASER after the conceptualization, we only sample the conceptualized
eventualities and ignore the original ones. For the “High frequency vs. Low
Frequency” experiment, we randomly sample 50 edge pairs for each relation type
such that the two edges in each pair share the same head eventuality, relation
type, but different tail eventuality (e.g., $\langle$“I am hungry”, Result, “I
eat food”$\rangle$ versus $\langle$“I am hungry”, Result, “I
exercise”$\rangle$). More importantly, the two sampled edges should have
significantly different frequencies. Specifically, we require the frequency of
the high frequency one to be larger than five, and the frequency of the high
frequency one must be at least five times larger than the frequency of the low
frequency one. For the “Exist vs. None-exist” experiment, for each relation
type, we first randomly sample 50 edges. Then for each edge, we randomly
replace a single word of the tail eventuality such that the new tail is very
similar to the original one but the created edge does not exist in ASER.

The annotations results are presented in Figure 11. In general, we can make
similar observations as the lower SP that the correlation is more significant
when we compare the existing and non-existing edges. Besides that, the
experiments on the conceptualized ASER help demonstrate that the
conceptualization module will not influence the overall quality of edge
frequencies.

##  8 Inference over ASER

In this section, we first introduce two kinds of inferences (eventuality
retrieval and relation retrieval) based on ASER. For each of them, inferences
over both one-hop and multi-hop are provided. Complexities of these two
retrieval algorithms are $\mathcal{O}(n^{k})$, where $n$ is the number of
average adjacent eventualities per eventuality and $k$ is the number of hops.
In this section, we show how to conduct these inferences over one-hop and two-
hop as the demonstration. ASER is composed of eventualities and concepts. In
line with the settings, we conduct case studies over extracted sub-graphs of
eventuality and concept graph. After that, we investigate the rule and meta-
path based inferences on ASER. For the rule based inference, we leverage AMIE+
[58], a rule mining system on ontological knowledge bases (KBs), to discover
closed and connected Horn rules on ASER. For the meta-path based inference, we
obtain the frequent meta-paths using statistical methods and perform case
studies by instantiating the meta-paths in both eventuality and concept
graphs.

###  8.1 Eventuality Retrieval

The eventuality retrieval inference is defined as follows. Given a head
eventuality191919ASER also supports the prediction of head eventualities given
tail eventualities and relations. We omit it in this section for a clear
presentation. $E_{h}$ and a relation list ${\mathcal{L}}$ =
($T_{1},T_{2},...,T_{k}$), find related eventualities and their associated
probabilities such that for each eventuality $E_{t}$ we can find a path, which
contains all the relations in ${\mathcal{L}}$ in order from $E_{h}$ to
$E_{t}$.

####  8.1.1 One-hop Inference

Head | Relation | Tail | Probability
---|---|---|---
You drink alcohol | Synchronous | You drown | 0.50
I drink coffee | Result | I calm down | 0.33
You are an employee | Contrast | You get fired | 0.50
I am programmer | Result | I have free time | 1.00
You go to restaurant | Precedence | You get sick | 0.50
I am frightened | Reason | Dog barks | 0.80
I order chicken | Concession | I am a vegan | 1.00
It is my birthday | Result | We go to zoo | 0.20
It is a cat | Condition | It is a tiger | 0.67
The surgery goes well | Result | There is no complication | 0.50
Table 8: Cases of one-hop eventuality inference in the eventuality graph. In the tables of the case study of eventualities, the words in eventualities are stored as lemmas in KBs. However, to clarify the examples, we manually correct the grammar mistakes. Head | Relation | Tail | Probability
---|---|---|---
Company be Stakeholder-Group | Condition |  PersonX be successful | 0.53
PersonX have Issue | Reason |  PersonX be proud | 0.52
PersonX get Symptom | Synchronous |  PersonX be Vulnerable-Group | 0.50
PersonX be Emotion | Succession |  PersonX marry | 0.51
AnimalX bark | Result |  AnimalX kill AnimalY | 0.33
PersonX be Predator | Result |  PersonX tease PersonY | 0.25
PersonX do Academic-Misconduct | Contrast |  PersonX tell Institute | 0.52
PersonX play Sport | Reason |  PersonX love Activity | 0.27
PersonX hurt Insect | Condition |  PersonX help Insect | 0.83
PersonX have Social-Medium | Result |  PersonX post it | 0.72
Table 9: Cases of one-hop eventuality inference in the concept graph. The
concepts are marked as italic texts.

For the one-hop inference, we assume the target relation is $T_{1}$. We then
define the probability of any potential tail node $E_{t}$ as:

$\text{Pr}(E_{t}|E_{h},T_{1})=\frac{w_{\langle
E_{h},T_{1},E_{t}\rangle}^{(r)}}{\sum_{E_{t}^{\prime},s.t.,(E_{h},T_{1},E_{t}^{\prime})\in{\mathcal{R}}}{w_{\langle
E_{h},T_{1},E_{t}^{\prime}\rangle}^{(r)}}},$ (7)

where $w_{\langle E_{h},T_{1},E_{t}\rangle}^{(r)}$ is the relation weight,
which is defined in Definition 3. If no node is connected with $E_{h}$ via
$T_{1}$, $\text{Pr}(E^{\prime}|E_{h},T_{1})$ will be 0 for any
$E^{\prime}\in{\mathcal{E}}$.

Several interesting inference examples are observed. In Table 8, we list the
reasonable examples of one-hop eventuality inference in eventuality graph. We
also list some of them as follows for discussion:

  * 1.

$\langle$“I drink coffee”, Reason, “I enjoy the flavor”$\rangle$

  * 2.

$\langle$“You go to restaurant”, Precedence, “You got sick”$\rangle$

  * 3.

$\langle$“It is a cat”, Condition, “It is a tiger”$\rangle$

It is observed that “I enjoy the (coffee) flavor” is likely to be the reason
for “I drink coffee.” It is also common that if you eat in an unhygienic
restaurant, you would probably get sick after you go to the restaurant. Given
the fact that the tiger is the largest cat species, it is reasonable to say
that if “it is a tiger,” “it is a cat.”

The following examples in Table 9 show the results of one-hop eventuality
inference in concept graph.

  * 1.

$\langle$“Company be Stakeholder-Group”, Condition, “PersonX be
successful”$\rangle$

  * 2.

$\langle$“PersonX hurt Insect”, Condition, “PersonX help Insect”$\rangle$

  * 3.

$\langle$“PersonX be Emotion”, Succession, “PersonX marry”$\rangle$

For instance, if someone is successful, his/her company is likely a big
corporation of stakeholders. The second one shows a situation that if an
unprofessional person helps insects out of good wills, he/she probably hurts
them in reverse. We could also infer from the last case that people tend to be
emotional when they get married.

####  8.1.2 Two-hop Inference

Head | Relation1 | Middle | Relation2 | Tail | Probability
---|---|---|---|---|---
I go to school | Reason | [I admire] | Synchronous | I am grown up | 0.50
I go to bed | Conjunction | [I sleep early] | Result | I am healthy | 0.86
We have dinner | Conjunction | [Food is very good] | Contrast | Service is not | 0.95
You go to restaurant | Condition | [They do something right] | Reason | There is a line-up | 0.50
We have lunch | Conjunction | [We really hit it off] | Contrast | She has a boyfriend at time | 0.50
You drink alcohol | Contrast | [You are fine] | Contrast | You have no work | 0.75
I am a vegan | Result | [I do not eat fish] | Contrast | We are hungry | 0.73
I go to bar | Precedence | [Our table is ready] | Result | We take seats | 0.35
I go to restaurant | Reason | [I have a coupon] | Contrast | It is expired | 0.36
I go to gym | Precedence | [I go on a date] | Contrast | We have nothing in common | 0.25

Table 10: Cases of two-hop eventuality inference in the eventuality graph. In
the table, we provide a typical example of middle nodes (embraced by brackets)
to create a scenario for better understanding.

Head | Relation1 | Middle | Relation2 | Tail | Probability
---|---|---|---|---|---
PersonX wait for PersonY | Precedence | [PersonX be tired] | Result |  PersonX go to sleep | 0.50
PersonX hate Animal | Contrast | [PersonX be harmless] | Contrast |  PersonX be Symptom | 0.40
PersonX be cranky | Synchronous | [PersonX be hungry] | Result |  PersonX order Meat | 0.23
PersonX be Artist | Contrast | [PersonX play Sport] | Reason |  PersonX be strong | 0.33
PersonX regret | Condition | [PersonX despise PersonY] | Reason |  PersonY be Performer | 0.20
PersonX pull gun | Reason | [PersonX startle] | Synchronous |  Domestic-Animal bark | 0.50
Predator take down Animal | Reason | [It be Predator] | Synchronous |  PersonX shoot | 0.32
PersonX be Academic-Title | Result | [PersonX be right] | Contrast |  PersonY doubt it | 0.28
PersonX hear it | Synchronous | [PersonY play Musical-Instrument] | Synchronous |  PersonY be blue | 0.65
PersonX be Artist | Condition | [PersonX strike PersonY] | Synchronous |  PersonY interview PersonX | 0.40

Table 11: Cases of two-hop eventuality inference in the concept graph. In the
table, we provide a typical example of middle nodes (embraced by brackets) to
create a scenario for better understanding. The concepts are marked as italic
texts.

On top of Eq. (7), it is easy for us to define the probability of $E_{t}$ on
two-hop setting. Assume the two relations are $T_{1}$ and $T_{2}$ in order. We
can define the probability as follows:

$\text{Pr}(E_{t}|E_{h},T_{1},T_{2})=\sum_{E_{m}\in{\mathcal{E}}_{m}}{\text{Pr}(E_{m}|E_{h},T_{1})\text{Pr}(E_{t}|E_{m},T_{2})},$
(8)

where ${\mathcal{E}}_{m}$ is the set of intermediate node $E_{m}$ such that
$(E_{h},T_{1},E_{m})$ and $(E_{m},T_{2},E_{t})\in{\mathcal{R}}$.

We list the intuitive examples of two-hop eventuality inference in eventuality
and concept graph in Table 10 and Table 11. To better understand the two
relations between the head node and the tail node, a typical middle node
embraced by brackets is provided. In the eventuality graph, three examples in
Table 10 are given for further explanation.

  * 1.

$\langle$“I go to bed”, Conjunction, [“I sleep early”], Result, “I am
healthy”$\rangle$

  * 2.

$\langle$“We have lunch”, Conjunction, [“We really hit it off”], Contrast,
“She has a boyfriend at time”$\rangle$

  * 3.

$\langle$“I go to restaurant”, Reason, [“I have a coupon”], Contrast, “It is
expired”$\rangle$

The first example illustrates that “I go bed” and “I sleep early” tend to
result in “I am healthy.” The second one describes a common social situation
that I have lunch with a girl and we really hit it off. But she has a
boyfriend at that time. Also, it is inferred that the reason why I go to that
restaurant is that I have a coupon. However, I find out that the coupon is
expired.

Leveraging the same method, we perform two-hop eventuality inference in the
concept graph and the results are presented in Table 11.

  * 1.

$\langle$“PersonX wait for PersonY”, Precedence, [“PersonX be tired”], Result,
“PersonX go to sleep”$\rangle$

  * 2.

$\langle$ “PersonX be cranky”, Synchronous, [“PersonX be hungry”], Result,
“PersonX order Meat” $\rangle$

  * 3.

$\langle$ “PersonX be Artist”, Condition, [“PersonX strike PersonY”],
Synchronous, “PersonY interview PersonX” $\rangle$

An interesting example shows that someone is waiting for his/her friend for
such a long time that he/she is tired and decides to go to sleep. We also
observe that the result of someone is cranky and hungry is most likely to be
that he/she orders meats. The last one shows that The artifacts of an artist
PersonX strikes PersonY and it happens at the same time as PersonY interviews
with PersonX.

###  8.2 Relation Retrieval

Relation | Head | Tail | Probability
---|---|---|---
Result | You drink alcohol | You have to pee | 1.00
Result | I drink coffee | I order a cappuccino | 0.50
Alternative | You are a employee | You will be fired | 0.50
Contrast | I eat meat | I am not a steak lover | 1.00
Precedence | You go to sleep | you wake up | 1.00
Contrast | I go to school | I drop out | 0.50
Reason | I am not picky | I go to restaurant | 0.43
Contrast | I love to cook | I go to restaurant | 0.57
Precedence | He waves his hat | The train stops | 1.00
Concession | I go to gym | I am tired | 0.83
Table 12: Cases of one-hop relation inference in the eventuality graph.  Relation | Head | Tail | Probability
---|---|---|---
Condition |  Company be Stakeholder-Group |  PersonX do Local-Ad | 0.10
Contrast |  PersonX call Agency |  It take Duration | 0.18
Reason |  PersonX be Public-Figure |  PersonX be professional | 0.30
Synchronous |  Animal bite |  Animal be frightened | 0.23
Precedence |  PersonX be Vulnerable-Group |  PersonX quit Activity | 0.88
Synchronous |  It be Domestic-Animal |  It be Mammal | 0.77
Contrast |  Bird catch Animal |  Animal get cheese | 0.67
Synchronous |  PersonX whistle |  Animal bark | 1.00
Condition |  PersonX give lecture |  PersonX be Academic-Title | 0.21
Result |  PersonX play Sport |  PersonX be fit | 0.87
Table 13: Cases of one-hop relation inference in the concept graph. The
concepts are marked as italic texts.

The relation retrieval inference is defined as follows. Given two nodes
$E_{h}$ and $E_{t}$, find all relation lists and their probabilities such that
for each relation list ${\mathcal{L}}$ = ($T_{1},T_{2},...,T_{k}$), we can
find a path from $E_{h}$ to $E_{t}$, which contains all the relations in
${\mathcal{L}}$ in order.

####  8.2.1 One-hop Inference

Assuming that the path length is one, we define the probability of one
relation $R=\langle E_{h},T,E_{t}\rangle$ given $E_{h}$ and $E_{t}$ as:

$\text{Pr}(R|E_{h},E_{t})=\text{Pr}(T|E_{h},E_{t})=\frac{w_{\langle
E_{h},T,E_{t}\rangle}^{(r)}}{\sum_{T^{\prime}\in{\mathcal{T}}}{w_{\langle
E_{h},T^{\prime},E_{t}\rangle}^{(r)}}},$ (9)

where ${\mathcal{T}}$ is the relation type set.

In Table 12 and 13, we perform one-hop relation inference in eventuality and
concept graph separately. The relations between head nodes and tail nodes are
retrieved to present the commonsense in daily life. In Table 12, the
eventuality relations are mined to show frequent patterns in eventuality
graph.

  * 1.

$\langle$ Result, “I drink coffee”, “I order cappuccino”$\rangle$

  * 2.

$\langle$ Contrast, “I love to cook”, “I go to restaurant” $\rangle$

  * 3.

$\langle$ Concession, “I go to gym”, “I am tired” $\rangle$

For example, “You drink alcohol” usually leads to “You have to pee”. Another
intriguing case illustrates that if someone loves to cook, he/she tends not to
go to the restaurant regularly. The last one describes a diligent and
determined person who decides to go to the gym, although he/she is tired.

In the concept graph, the same process is used and the results are stored in
Table 13.

  * 1.

$\langle$ Contrast, “Bird catch Animal”, “Animal get cheese” $\rangle$

  * 2.

$\langle$ Condition, “PersonX give lecture”, “PersonX be Academic-Title”
$\rangle$

  * 3.

$\langle$ Result, “PersonX play Sport”, “PersonX be fit” $\rangle$

We learn from the first example that if birds do not catch these animals
(e.g., rats), they would probably get cheese. The second one shows that if
someone has an academic title (e.g., professor), he/she will deliver a
lecture. The third example tells that the result of PersonX plays sport is
he/she is fit.

####  8.2.2 Two-hop Inference

Similarly, given two nodes $E_{h}$ and $E_{t}$, we define the probability of a
two-hop connection ($T_{1}$, $T_{2}$) between them as follows:

$\displaystyle\text{Pr}(T_{1},T_{2}|E_{h},E_{t})$
$\displaystyle=\sum_{E_{m}\in{\mathcal{E}}_{m}}\text{Pr}(T_{1},T_{2},E_{m}|E_{h},E_{t})$
$\displaystyle=\sum_{E_{m}\in{\mathcal{E}}_{m}}\text{Pr}(T_{1}|E_{h})\text{Pr}(E_{m}|T_{1},E_{2}|E_{m},E_{t}),$
(10)

where $\text{Pr}(T|E_{h})$ is the probability of a relation type $T$ given a
head eventuality $E_{h}$, which is defined as follows:

$\text{Pr}(T|E_{h})=\frac{\sum_{E_{t},s.t.,(E_{h},T,E_{t})\in{\mathcal{R}}}{w_{\langle
E_{h},T^{\prime},E_{t}\rangle}^{(r)}}}{\sum_{T^{\prime}\in{\mathcal{T}}}\sum_{E_{t},s.t.,(E_{h},T^{\prime},E_{t})\in{\mathcal{T}}}{w_{\langle
E_{h},T^{\prime},E_{t}\rangle}^{(r)}}}.$ (11)

The two-hop relations are inferred from the eventuality and concept graph in
Table 14 and 15. Some reasonable results are listed below. In line with two-
hop eventuality inference, we give a typical middle node embraced by brackets
to show the circumstance more clearly.

  * 1.

$\langle$ Synchronous, Conjunction, “We have breakfast”, [“Our room is
ready”], “The front desk staff is friendly” $\rangle$

  * 2.

$\langle$ Synchronous, Reason, “I sit on chair”, [“I get my hair washed”],
“Stylist tells me” $\rangle$

  * 3.

$\langle$ Reason, Result, “I go to supermarket”, [“I have a coupon”], “The
price is great” $\rangle$

In the eventuality graph, we find that some tourists visit a hotel. The guests
have breakfast when their room is cleaned and ready. Meanwhile, they find the
staff at the front desk is friendly and nice. The second example shows a
common thing at the haircut salons. “I sit on chair” to get my hair washed
because the stylist tells me to do so before haircut. We also find that
someone goes to a supermarket because he/she has a coupon to lower the prices
of grocery.

Relation1 | Relation2 | Head | Middle | Tail | Probability
---|---|---|---|---|---
Conjunction | Synchronous | I go to bed | [I would sleep] | I heal | 0.25
Result | Reason | I go to bed | [I fall right to sleep] | I am drunk | 0.30
Synchronous | Conjunction | We have breakfast | [Our room is ready] | The front desk staffs are friendly | 0.12
Contrast | Precedence | You are an employee | [You get fired] | The contract ends | 0.11
Reason | Conjunction | I drink coffee | [I have severe ADHD] | I do not get any help | 0.33
Synchronous | Reason | I sit on chair | [I get my hair wash] | Stylist tell me | 0.25
Contrast | Reason | I am a vegan | [I eat meat] | It tastes good | 0.33
Reason | Result | I go to supermarket | [I have a coupon] | The price is great | 0.23
Reason | Contrast | I go to restaurant | [The service is great] | The food is mediocre | 0.49
Contrast | Contrast | The surgery goes well | [She is in a coma] | She is stabilized | 0.53

Table 14: Cases of two-hop relation inference in the eventuality graph. In the
table, we provide a typical example of middle nodes (embraced by brackets) to
create a scenario for better understanding.

Relation1 | Relation2 | Head | Middle | Tail | Probability
---|---|---|---|---|---
Precedence | Precedence |  PersonX be Stakeholder | [PersonX tell PersonX] |  PersonX sign Document | 0.85
Precedence | Precedence |  PersonX wait for PersonX | [PersonX send Information] |  PersonX drag PersonX away | 0.40
Result | Precedence |  PersonX be Vulnerable-Population | [PersonX be homeless] |  Organization help PersonX | 0.73
Precedence | Result |  Predator catch Herbivore | [Predator eat Meat] |  Mammal live | 0.79
Synchronous | Result |  PersonX be thirsty | [PersonX be hungry] |  PersonX order Meat | 0.56
Contrast | Condition |  PersonX be Musician | [PersonX play Sport] |  PersonX be tall | 0.42
Result | Synchronous |  PersonX excite | [PersonX imagine Emotion] |  PersonX answer Electronic-Device | 0.58
Reason | Contrast |  PersonX eat Animal-Product | [PersonX enjoy it] | It be spicy | 0.25
Synchronous | Condition |  PersonX hear it | [PersonX play Musical-Instrument] |  PersonX be Extracurricular-Activity | 0.16
Alternative | Result |  PersonX eat Meat | [PersonX be vegetarian] |  PersonX starve | 0.50

Table 15: Cases of two-hop relation inference in the concept graph. In the
table, we provide a typical example of middle nodes (embraced by brackets) to
create a scenario for better understanding. The concepts are marked as italic
texts.

In Table 15, the two-hop relations among head, middle, and tail nodes are
extracted to show some insights behind the relation inference.

  * 1.

$\langle$ Precedence, Precedence, “PersonX wait for PersonY”, [“PersonY send
Information”], “PersonZ drag PersonX away” $\rangle$

  * 2.

$\langle$ Result, Synchronous, “PersonX excite”, [“PersonX imagine Emotion”],
“PersonX answer Electronic-Device” $\rangle$

  * 3.

$\langle$ Synchronous, Condition, “PersonX hear it”, [“PersonX play Musical-
Instrument”], “PersonX be Extracurricular-Activity” $\rangle$

In the first example, PersonX waits for PersonY before PersonY sends
information (e.g., a letter) to inform PersonX not wait for him. PersonX is
reluctant to leave until his/her friends drag him/her away. In the latter
example, the result of a person is excited is that he/she imagines the
situation and answers the phone. The third case shows that PersonX hears that
PersonY plays instrument since PersonY is having extracurricular activities.

###  8.3 Rule Mining

Rule | $\langle E_{b}\xrightarrow{\texttt{Concession}}E_{f}\rangle\land\langle E_{a}\xrightarrow{\texttt{Result}}E_{f}\rangle\Rightarrow\langle E_{a}\xrightarrow{\texttt{Contrast}}E_{b}\rangle$
---|---
Instances |  $\langle$ I do not know $\rightarrow$ I guess $\rangle\land\langle$ I believe $\rightarrow$ I guess $\rangle\Rightarrow\langle$ I believe $\rightarrow$ I do not know $\rangle$
|  $\langle$ I am not sure $\rightarrow$ I guess $\rangle\land\langle$ I hope
so $\rightarrow$ I guess $\rangle\Rightarrow\langle$ I hope so $\rightarrow$ I
am not sure $\rangle$
|  $\langle$ I understand $\rightarrow$ I can not speak $\rangle\land\langle$
I am not a lawyer $\rightarrow$ I can not speak $\rangle\Rightarrow\langle$ I
am not a lawyer $\rightarrow$ I understand $\rangle$
Rule | $\langle E_{f}\xrightarrow{\texttt{Contrast}}E_{b}\rangle\land\langle E_{a}\xrightarrow{\texttt{Instantiation}}E_{f}\rangle\Rightarrow\langle E_{a}\xrightarrow{\texttt{Contrast}}E_{b}\rangle$
Instances |  $\langle$ I remember $\rightarrow$ I could not find it $\rangle\land\langle$ I get $\rightarrow$ I remember $\rangle\Rightarrow\langle$ I get $\rightarrow$ I could not find it $\rangle$
|  $\langle$ I would say $\rightarrow$ I might be wrong $\rangle\land\langle$
I hope $\rightarrow$ I would say $\rangle\Rightarrow\langle$ I hope
$\rightarrow$ I might be wrong $\rangle$
|  $\langle$ It have been suggested $\rightarrow$ This is unlikely
$\rangle\land\langle$ It is possible $\rightarrow$ It have been suggested
$\rangle\Rightarrow\langle$ It is possible $\rightarrow$ This is unlikely
$\rangle$
Rule | $\langle E_{e}\xrightarrow{\texttt{ChosenAlternative}}E_{b}\rangle\land\langle E_{a}\xrightarrow{\texttt{ChosenAlternative}}E_{e}\rangle\Rightarrow\langle E_{a}\xrightarrow{\texttt{ChosenAlternative}}E_{b}\rangle$
Instances |  $\langle$ I will not go $\rightarrow$ You come here $\rangle\land\langle$ I want to see $\rightarrow$ I will not go $\rangle\Rightarrow\langle$ I want to see $\rightarrow$ You come here $\rangle$
|  $\langle$ I want $\rightarrow$ It is $\rangle\land\langle$ I wish
$\rightarrow$ I want $\rangle\Rightarrow\langle$ I wish $\rightarrow$ It is
$\rangle$
|  $\langle$ I want $\rightarrow$ I get $\rangle\land\langle$ I do not get
that $\rightarrow$ I want $\rangle\Rightarrow\langle$ I do not get that
$\rightarrow$ I get $\rangle$
Rule | $\langle E_{a}\xrightarrow{\texttt{Reason}}E_{e}\rangle\land\langle E_{e}\xrightarrow{\texttt{Restatement}}E_{b}\rangle\Rightarrow\langle E_{a}\xrightarrow{\texttt{Reason}}E_{b}\rangle$
Instances |  $\langle$ I have ever see $\rightarrow$ I know $\rangle\land\langle$ I know $\rightarrow$ They are $\rangle\Rightarrow\langle$ I have ever see $\rightarrow$ They are $\rangle$
|  $\langle$ I am curious $\rightarrow$ I think $\rangle\land\langle$ I think
$\rightarrow$ It seems $\rangle\Rightarrow\langle$ I am curious $\rightarrow$
It seems $\rangle$
|  $\langle$ It is not $\rightarrow$ You are lying $\rangle\land\langle$ You
are lying $\rightarrow$ I do not believe you $\rangle\Rightarrow\langle$ It is
not $\rightarrow$ I do not believe you $\rangle$
Rule | $\langle E_{a}\xrightarrow{\texttt{Concession}}E_{f}\rangle\land\langle E_{b}\xrightarrow{\texttt{Reason}}E_{f}\rangle\Rightarrow\langle E_{a}\xrightarrow{\texttt{Contrast}}E_{b}\rangle$
Instances |  $\langle$ I have no clue $\rightarrow$ I hope $\rangle\land\langle$ It be $\rightarrow$ I hope $\rangle\Rightarrow\langle$ I have no clue $\rightarrow$ It is $\rangle$
|  $\langle$ I reckon $\rightarrow$ I do not know $\rangle\land\langle$ I can
not talk about it $\rightarrow$ I do not know $\rangle\Rightarrow\langle$ I
reckon $\rightarrow$ I can not talk about it $\rangle$
|  $\langle$ You do not understand it $\rightarrow$ You are admitted
$\rangle\land\langle$ That is $\rightarrow$ You are admitted
$\rangle\Rightarrow\langle$ You do not understand it $\rightarrow$ That is
$\rangle$
Rule | $\langle E_{b}\xrightarrow{\texttt{Alternative}}E_{f}\rangle\land\langle E_{a}\xrightarrow{\texttt{Result}}E_{f}\rangle\Rightarrow\langle E_{a}\xrightarrow{\texttt{Contrast}}E_{b}\rangle$
Instances |  $\langle$ I am going $\rightarrow$ I am not going $\rangle\land\langle$ I do not care $\rightarrow$ I am not going $\rangle\Rightarrow\langle$ I do not care $\rightarrow$ I am going $\rangle$
|  $\langle$ You do $\rightarrow$ I do $\rangle\land\langle$ I suppose
$\rightarrow$ I do $\rangle\Rightarrow\langle$ I suppose $\rightarrow$ You do
$\rangle$
|  $\langle$ I reckon $\rightarrow$ I guess $\rangle\land\langle$ I wonder
$\rightarrow$ I guess $\rangle\Rightarrow\langle$ I wonder $\rightarrow$ I
reckon $\rangle$
Rule | $\langle E_{a}\xrightarrow{\texttt{Reason}}E_{f}\rangle\land\langle E_{b}\xrightarrow{\texttt{Succession}}E_{f}\rangle\Rightarrow\langle E_{a}\xrightarrow{\texttt{Reason}}E_{b}\rangle$
Instances |  $\langle$ I ask $\rightarrow$ I am not sure $\rangle\land\langle$ I do not know $\rightarrow$ I am not sure $\rangle\Rightarrow\langle$ I ask $\rightarrow$ I do not know $\rangle$
|  $\langle$ We are lucky $\rightarrow$ We notice $\rangle\land\langle$ We
order $\rightarrow$ We notice $\rangle\Rightarrow\langle$ We are lucky
$\rightarrow$ We order $\rangle$
|  $\langle$ I remember it $\rightarrow$ I see it $\rangle\land\langle$ I
realize $\rightarrow$ I see it $\rangle\Rightarrow\langle$ I remember it
$\rightarrow$ I realize $\rangle$
Rule | $\langle E_{a}\xrightarrow{\texttt{Concession}}E_{f}\rangle\land\langle E_{b}\xrightarrow{\texttt{Precedence}}E_{f}\rangle\Rightarrow\langle E_{a}\xrightarrow{\texttt{Contrast}}E_{b}\rangle$
Instances |  $\langle$ I am unconscious $\rightarrow$ I wake up $\rangle\land\langle$ I see $\rightarrow$ I wake up $\rangle\Rightarrow\langle$ I am unconscious $\rightarrow$ I see $\rangle$
|  $\langle$ I swear $\rightarrow$ I guess $\rangle\land\langle$ I do not know
$\rightarrow$ I guess $\rangle\Rightarrow\langle$ I swear $\rightarrow$ I do
not know $\rangle$
|  $\langle$ I can not believe $\rightarrow$ It is great $\rangle\land\langle$
I think $\rightarrow$ It is great $\rangle\Rightarrow\langle$ I can not
believe $\rightarrow$ I think $\rangle$
Rule | $\langle E_{a}\xrightarrow{\texttt{Alternative}}E_{e}\rangle\land\langle E_{e}\xrightarrow{\texttt{Exception}}E_{b}\rangle\Rightarrow\langle E_{a}\xrightarrow{\texttt{Exception}}E_{b}\rangle$
Instances |  $\langle$ It is not $\rightarrow$ It is wrong $\rangle\land\langle$ It is wrong $\rightarrow$ It is $\rangle\Rightarrow\langle$ It is not $\rightarrow$ It is $\rangle$
|  $\langle$ I really want $\rightarrow$ I think $\rangle\land\langle$ I think
$\rightarrow$ I know $\rangle\Rightarrow\langle$ I really want $\rightarrow$ I
know $\rangle$
|  $\langle$ It is not $\rightarrow$ I suppose $\rangle\land\langle$ I suppose
$\rightarrow$ You know $\rangle\Rightarrow\langle$ It is not $\rightarrow$ You
know $\rangle$
Rule | $\langle E_{a}\xrightarrow{\texttt{ChosenAlternative}}E_{f}\rangle\land E_{b}\langle\xrightarrow{\texttt{ChosenAlternative}}E_{f}\rangle\Rightarrow\langle E_{a}\xrightarrow{\texttt{Restatement}}E_{b}\rangle$
Instances |  $\langle$ I am hoping $\rightarrow$ We get $\rangle\land\langle$ I think $\rightarrow$ We get $\rangle\Rightarrow\langle$ I am hoping $\rightarrow$ I think $\rangle$
|  $\langle$ I suppose $\rightarrow$ He is $\rangle\land\langle$ I think
$\rightarrow$ He is $\rangle\Rightarrow\langle$ I suppose $\rightarrow$ I
think $\rangle$
|  $\langle$ I am glad $\rightarrow$ I think $\rangle\land\langle$ The food is
good $\rightarrow$ I think $\rangle\Rightarrow\langle$ I am glad $\rightarrow$
The food is good $\rangle$

Table 16: Cases of AMIE+ rule mining in the eventuality graph. For the
simplicity of formatting, we represent $\langle E_{h},T,E_{t}\rangle$ triples
as $\langle E_{h}\xrightarrow{\texttt{T}}E_{t}\rangle$.

AMIE+ [58] aims at mining close and connected Horn Rules in the form of

$\langle E_{a},T_{1},E_{b}\rangle\land\langle
E_{b},T_{2},E_{c}\rangle\Rightarrow\langle E_{a},T_{3},E_{b}\rangle$

where $E_{a},E_{b},E_{c}$ and $T_{1},T_{2},T_{3}$ are eventuality variables
and relation variables, respectively. As demonstrated through experiments on
different KBs, AMIE+ provides an effective approach for the investigation and
inference over KB from a logic-rule perspective. We therefore apply AMIE+ on
ASER to probe whether it preserves logical properties among multi-hop
eventualities and relations. To make the notation consistent with AMIE+, we
denote a fact triple with variables at head and/or tail eventuality positions,
such as $\langle E_{a},T_{1},E_{b}\rangle$, as an atom. A rule of interests
comprise of a body of a set of atoms $B_{1},B_{2},\cdots,B_{n}$ (n = 2 in this
case), and a head of a single atom $\langle E_{h},T,E_{t}\rangle$. We
abbreviate the rule as $\vec{B}\,\Rightarrow\,\langle E_{h},T,E_{t}\rangle$

The Algorithm of AMIE+ adopts an iterative procedure: starting with a queue of
all possible items (rule of size 1), it dequeues a rule in each iteration, and
outputs/grows/prunes the rule with certain criterion (listed below), and
enqueues the grown new rules back. Throughout the iteration, AMIE+ sets the
following significance criterion:

Head coverage:

$hc(\vec{B}\,\Rightarrow\,\langle
E_{h},T,E_{t}\rangle):=\frac{supp(\vec{B}\,\Rightarrow\,\langle
E_{h},T,E_{t}\rangle)}{size(T)},$

where

$supp(\vec{B}\,\Rightarrow\,\langle
E_{h},T,E_{t}\rangle):=\\#(E_{h},E_{t}):\exists\,z_{1},\cdots,z_{m}:\vec{B}\land\langle
E_{h},T,E_{t}\rangle:$

denotes the support, i.e., the number of correct prediction yielded with the
rule in the current KB, and $size(T)$ denotes the number of facts with $T$ as
relations.

Standard confidence:

$conf(\vec{B}\,\Rightarrow\,\langle
E_{h},T,E_{t}\rangle):=\frac{supp(\vec{B}\,\Rightarrow\,\langle
E_{h},T,E_{t}\rangle)}{\\#(E_{h},E_{t}):\exists\,z_{1},\cdots,z_{m}:\vec{B}},$

where $\\#(E_{h},E_{t}):\exists z_{1},\cdots,z_{m}:\vec{B}$ denotes all
possible predictions of the rule.

PCA confidence:

$conf_{pca}(\vec{B}\,\Rightarrow\,\langle
E_{h},T,E_{t}\rangle):=\frac{supp(\vec{B}\,\Rightarrow\,\langle
E_{h},T,E_{t}\rangle)}{\\#(E_{h},E_{t}):\exists\,z_{1},\cdots,z_{m}:\vec{B}\,\land\langle
E_{h},T,E^{\prime}_{t}\rangle},$

where $\\#(E_{h},E_{t}):\exists\,z_{1},\cdots,z_{m}:\vec{B}\,\land\langle
E_{h},T,E^{\prime}_{t}\rangle$ denotes the number of pairs of $(E_{h},E_{t})$
predicted by corresponding relation body $\vec{B}$ but with an existing pair
of $\langle E_{h},T,E^{\prime}_{t}\rangle$ in KB.

AMIE+ focuses on RDF Knowledge Bases, where an RDF KB could be represented as
a set of facts in the form
$\langle{Subject},\texttt{Relation},{Object}\rangle$. To pair ASER Graph with
AMIE+, we extract all $\langle E_{1},\texttt{Relation},E_{2}\rangle$ triples
from the relation table as our set of facts. To preserve the frequency
information, we duplicate each extracted fact for $f$ times where $f$ is the
corresponding triple frequency in the relation table. During our experiments,
we set the threshold of minimal PCA confidence minPCA=0.1 and minimal head
coverage minHC=0.01, and run AMIE+ on both the eventuality graph and concept
graph of ASER.

Some of the mined rules and instantiated cases are shown in Table 16 and 17,
respectively. For the eventuality graph, the rule “$\langle
E_{b}\xrightarrow{\texttt{Concession}}E_{f}\rangle\land\langle
E_{a}\xrightarrow{\texttt{Result}}E_{f}\rangle\Rightarrow\langle
E_{a}\xrightarrow{\texttt{Contrast}}E_{b}\rangle$” demonstrates that if the
result of some event is opposite to what should happen, then the reason that
induces this “opposite” result is likely to contrast the original event in
some core properties. The last instance “$\langle$ I understand $\rightarrow$
I can not speak $\rangle\land\langle$ I am not a lawyer $\rightarrow$ I can
not speak $\rangle\Rightarrow\langle$ I am not a lawyer $\rightarrow$ I
understand $\rangle$” illustrates this rule with a lawsuit scenario where the
“opposite” result ”I can not speak” is induced by the reason “I am not a
lawyer”, which contrasts the original event “I understand”. And the rule
“$\langle E_{a}\xrightarrow{\texttt{ChosenAlternative}}E_{f}\rangle\land
E_{b}\langle\xrightarrow{\texttt{ChosenAlternative}}E_{f}\rangle\Rightarrow\langle
E_{a}\xrightarrow{\texttt{Restatement}}E_{b}\rangle$” captures that if the
subject consistently prefers two events as alternative to a third event, then
it is possible that the first two events have very similar semantic meaning.
This is illustrated with its last instance “$\langle$ I am glad $\rightarrow$
I think $\rangle\land\langle$ The food is good $\rightarrow$ I think
$\rangle\Rightarrow\langle$ I am glad $\rightarrow$ The food is good
$\rangle$” where both “I am glad” and “The food is good” express similar
meanings in which the subject consistently prefers to “I think”.

For the concept graph, the rule “$\langle
E_{e}\xrightarrow{\texttt{Instantiation}}E_{a}\rangle\land\langle
E_{e}\xrightarrow{\texttt{Instantiation}}E_{b}\rangle\Rightarrow\langle
E_{a}\xrightarrow{\texttt{Conjunction}}E_{b}\rangle$” shows that the concepts
that both describe a shared concept in details are likely to both happen. Its
first instance “$\langle$ PersonX realize $\rightarrow$ PersonX point out
$\rangle\land\langle$ PersonX realize $\rightarrow$ PersonX have Information
$\rangle\Rightarrow\langle$ PersonX point out $\rightarrow$ PersonX have
Information” demonstrates this through a information-capture process, where
the two concepts “PersonX point out” and “PersonX have Information $\rangle$”
that both describe the concept “PersonX realize” happen together. The rule
“$\langle E_{e}\xrightarrow{\texttt{Exception}}E_{b}\rangle\land\langle
E_{e}\xrightarrow{\texttt{Succession}}E_{a}\rangle\Rightarrow\langle
E_{a}\xrightarrow{\texttt{Contrast}}E_{b}\rangle$” shows the association
between Exception and Contrast that is bridged via Succession. This is
demonstrated with its first instance “$\langle$ Item be ready $\rightarrow$
PersonX wait $\rangle\land\langle$ Item be ready $\rightarrow$ PersonX check
$\rangle\Rightarrow\langle$ PersonX check $\rightarrow$ PersonX be wait
$\rangle$” that shows a scenario of a customer checking and waiting for
products.

Rule | $\langle E_{e}\xrightarrow{\texttt{Restatement}}E_{a}\rangle\land\langle E_{e}\xrightarrow{\texttt{Restatement}}E_{b}\rangle\Rightarrow\langle E_{a}\xrightarrow{\texttt{Conjunction}}E_{b}\rangle$
---|---
Instances |  $\langle$ PersonX laugh $\rightarrow$ PersonX smile $\rangle\land\langle$ PersonX laugh $\rightarrow$ PersonX open Facial-Feature $\rangle\Rightarrow\langle$ PersonX smile $\rightarrow$ PersonX open Facial-Feature $\rangle$
|  $\langle$ PersonX love it $\rightarrow$ It be good $\rangle\land\langle$
PersonX love it $\rightarrow$ It be tasty $\rangle\Rightarrow\langle$ It be
good $\rightarrow$ It be tasty $\rangle$
|  $\langle$ PersonX wish $\rightarrow$ PersonX need $\rangle\land\langle$
PersonX wish $\rightarrow$ PersonX need $\rangle\Rightarrow\langle$ PersonX
need $\rightarrow$ PersonX need $\rangle$
Rule | $\langle E_{e}\xrightarrow{\texttt{Instantiation}}E_{a}\rangle\land\langle E_{e}\xrightarrow{\texttt{Instantiation}}E_{b}\rangle\Rightarrow\langle E_{a}\xrightarrow{\texttt{Conjunction}}E_{b}\rangle$
Instances |  $\langle$ PersonX realize $\rightarrow$ PersonX point out $\rangle\land\langle$ PersonX realize $\rightarrow$ PersonX have Information $\rangle\Rightarrow\langle$ PersonX point out $\rightarrow$ PersonX have Information $\rangle$
|  $\langle$ PersonX have $\rightarrow$ PersonX get $\rangle\land\langle$
PersonX have $\rightarrow$ PersonX own $\rangle\Rightarrow\langle$ PersonX get
$\rightarrow$ PersonX own $\rangle$
|  $\langle$ PersonX know $\rightarrow$ PersonX be sure $\rangle\land\langle$
PersonX know $\rightarrow$ PersonX remember $\rangle\Rightarrow\langle$
PersonX be sure $\rightarrow$ PersonX remember $\rangle$
Rule | $\langle E_{e}\xrightarrow{\texttt{Concession}}E_{b}\rangle\land\langle E_{e}\xrightarrow{\texttt{Restatement}}E_{a}\rangle\Rightarrow\langle E_{a}\xrightarrow{\texttt{Contrast}}E_{b}\rangle$
Instances |  $\langle$ PersonX order Dish $\rightarrow$ PersonX be hungry $\rangle\land\langle$ PersonX order Dish $\rightarrow$ PersonX order $\rangle\Rightarrow\langle$ PersonX order $\rightarrow$ PersonX be hungry $\rangle$
|  $\langle$ PersonX wish $\rightarrow$ PersonX doubt $\rangle\land\langle$
PersonX wish $\rightarrow$ PersonX need $\rangle\Rightarrow\langle$ PersonX
doubt $\rightarrow$ PersonX need $\rangle$
|  $\langle$ PersonX love it $\rightarrow$ PersonX hate it
$\rangle\land\langle$ PersonX love it $\rightarrow$ It be good
$\rangle\Rightarrow\langle$ PersonX hate it $\rightarrow$ It be good $\rangle$
Rule | $\langle E_{e}\xrightarrow{\texttt{Exception}}E_{b}\rangle\land\langle E_{e}\xrightarrow{\texttt{Succession}}E_{a}\rangle\Rightarrow E_{a}\langle\xrightarrow{\texttt{Contrast}}E_{b}\rangle$
Instances |  $\langle$ Item be ready $\rightarrow$ PersonX wait $\rangle\land\langle$ Item be ready $\rightarrow$ PersonX check $\rangle\Rightarrow\langle$ PersonX check $\rightarrow$ PersonX be wait $\rangle$
|  $\langle$ PersonX say $\rightarrow$ PersonX be sorry $\rangle\land\langle$
PersonX say $\rightarrow$ PersonX be surprised $\rangle\Rightarrow\langle$
PersonX be sorry $\rightarrow$ PersonX be surprised $\rangle$
|  $\langle$ It be $\rightarrow$ PersonX guess $\land$ It be $\rightarrow$ It
be factor $\rangle\Rightarrow\langle$ PersonX guess $\rightarrow$ It be factor
$\rangle$
Rule | $\langle E_{a}\xrightarrow{\texttt{Restatement}}E_{f}\rangle\land\langle E_{b}\xrightarrow{\texttt{Restatement}}E_{f}\rangle\Rightarrow\langle E_{a}\xrightarrow{\texttt{Synchronous}}E_{b}\rangle$
Instances |  $\langle$ PersonX love it $\rightarrow$ It be good $\rangle\land\langle$ PersonX feel $\rightarrow$ It be good $\rangle\Rightarrow\langle$ PersonX love it $\rightarrow$ PersonX feel $\rangle$
|  $\langle$ It be cool $\rightarrow$ It be good $\rangle\land\langle$ PersonX
think $\rightarrow$ It be okay $\rangle\Rightarrow\langle$ It be cool
$\rightarrow$ It be okay $\rangle$
|  $\langle$ PersonX like it $\rightarrow$ It be good $\rangle\land\langle$
PersonX be happy $\rightarrow$ It be good $\rangle\Rightarrow\langle$ PersonX
like it $\rightarrow$ PersonX be happy $\rangle$

Table 17: Cases of AMIE+ rule mining in the concept graph. For the simplicity
of formatting, we represent $\langle E_{h},T,E_{t}\rangle$ triples as $\langle
E_{h}\xrightarrow{\texttt{T}}E_{t}\rangle$.

###  8.4 Meta-path Mining

ASER is a complex heterogeneous graph that encodes the commonsense knowledge.
ASER is composed of two types of nodes (i.e., extracted eventuality and
conceptualized eventuality) and 15 types of edges (e.g., Reason and
Precedence). We leverage meta-path [59] mining which studies the semantic
meanings behind paths to tackle the heterogeneity of ASER. A meta-path is a
path that consists of a sequence of different relations defined among various
node types. Formally, a meta-path $P$ is defined as a path
$E_{1}\xrightarrow{T_{1}}E_{2}\xrightarrow{T_{2}}\cdots\xrightarrow{T_{l-1}}E_{l}$,
in which $T=T_{1}\circ T_{2}\circ\cdots\circ T_{l}$ is the composite relation
between $N_{1}$ and $N_{l}$. Take an example from Table 18, the meta-path
“$E_{1}\xrightarrow{\texttt{Conceptualization}}C_{1}\xrightarrow{\texttt{
ConceptInstantiation}}E_{2}$” defines a composite relation in which the two
eventuality $E_{1}$ and $E_{2}$ are conceptualized to the same concept
$C_{1}$.

To automatically select the most frequent and influential meta-paths, we first
perform a random walk on the hybrid graph. Specifically, 50,000 seed nodes are
chosen independently and uniformly from the nodes of ASER. Starting from each
seed node, a random walk is used to generate 50 multi-hop paths of different
nodes and relations. The nodes in a path are represented by their types rather
than their contents. For example, a path “I drink coffee
$\xrightarrow{\texttt{Result}}$ I stay up late” is converted into a meta-path
“$E_{1}\xrightarrow{\texttt{Result}}E_{2}$”. After collecting meta-paths, we
search for the frequent patterns of 2-hop and 3-hop meta-paths. The appearance
of metapaths is counted. The 2-hop and 3-hop metapaths are later ranked by
their frequencies. The frequent metapaths are selected for the further case
study. We list the intriguing instances from these meta-paths in Table 18.

For 2-hop metapaths, the results are very similar to the ones of
eventuality/relation retrieval inference. For example,
“$E_{1}\xrightarrow{\texttt{Reason}}E_{2}\xrightarrow{\texttt{Result}}E_{3}$”
describes paths following cause and effect relations. A typical instance in
daily life is “I am in pain$\rightarrow$I am alone$\rightarrow$I sit at bar,”
describing a scenario in which a man suffers from loneliness and goes to the
bar to numb the pains. In addition to relations among eventualities, the
interaction between concepts and eventualities are also discovered by the
meta-paths. In the cases of
“$E_{1}\xrightarrow{\texttt{Conceptualization}}C_{1}\xrightarrow{\texttt{ConceptInstantiation}}E_{2}$,”
two semantically distinct eventualities are unified in the concept-level. For
example, “He is psychiatrist” and “I am attorney” follows the same pattern,
“PersonX be specialist.”

For 3-hop metapaths, the reasoning paths are longer and illustrates the daily
life in more details. For example, in the meta-path
“$E_{1}\xrightarrow{\texttt{Result}}E_{2}\xrightarrow{\texttt{Contrast}}E_{3}\xrightarrow{\texttt{Conjunction}}E_{4}$,”
an instantiated example, “I have you number$\rightarrow$I call
you$\rightarrow$I have a meeting$\rightarrow$I have a presentation,” shows
that a person wants to call his friends with the phone number. However, he has
to do a presentation in the coming meeting and decides to call his friend
later. As for the hybrid meta-path with extracted and conceptualized
eventualities,
“$E_{1}\xrightarrow{\texttt{Conjunction}}E_{2}\xrightarrow{\texttt{Conceptualization}}C_{1}\xrightarrow{\texttt{ConceptInstantiation}}E_{3}$,”
we find out that someone is sweating because of the hot weather while someone
is unfortunately in a coma. Both of them are unified under the concept
“PersonX be Symptom.”

#Hop | meta-path | Instances
---|---|---
2 | $E_{1}$ $\xrightarrow{\texttt{Conjunction}}$ $E_{2}$ $\xrightarrow{\texttt{Contrast}}$ $E_{3}$ | I go to bed $\rightarrow$ I go to sleep $\rightarrow$ I wake up
I have breakfast $\rightarrow$ I have milk $\rightarrow$ I feel sick
I take bus $\rightarrow$ I go to work $\rightarrow$ I go home
$E_{1}$ $\xrightarrow{\texttt{Precedence}}$ $E_{2}$ $\xrightarrow{\texttt{Precedence}}$ $E_{3}$ | You go to sleep $\rightarrow$ You wake up $\rightarrow$ You hit the ground
You drink alcohol $\rightarrow$ You go to toilet $\rightarrow$ You have to pee
You go to restaurant $\rightarrow$ You are sick $\rightarrow$ You go to
hospital
$E_{1}$ $\xrightarrow{\texttt{Conceptualization}}$ $C_{1}$ $\xrightarrow{\texttt{ ConceptInstantiation}}$ $E_{2}$ | He is psychiatrist $\rightarrow$ PersonX is Specialist $\rightarrow$ I am attorney
I want milk $\rightarrow$ PersonX want Animal-Product $\rightarrow$ He wants
burgers
You make reservation $\rightarrow$ PersonX make Service $\rightarrow$ He makes
statement