# CEP3: Community Event Prediction with Neural Point Process on Graph

Xuhong Wang<EMAIL_ADDRESS>Shanghai Jiao Tong
UniversityShanghaiChina ,  Sirui Chen<EMAIL_ADDRESS>The University
of Hong KongHong KongChina43017-6221 ,  Yixuan He<EMAIL_ADDRESS>University of OxfordUnited Kingdom ,  Minjie Wang<EMAIL_ADDRESS>AmazonShanghaiChina ,  Quan Gan<EMAIL_ADDRESS>AmazonShanghaiChina ,  Yupu
Yang<EMAIL_ADDRESS>Shanghai Jiao Tong UniversityShanghaiChina and
Junchi Yan<EMAIL_ADDRESS>Shanghai Jiao Tong UniversityShanghaiChina

(2018)

###### Abstract.

Many real world applications can be formulated as event forecasting on
Continuous Time Dynamic Graphs (CTDGs) where the occurrence of a timed event
between two entities is represented as an edge along with its occurrence
timestamp in the graphs.However, most previous works approach the problem in
compromised settings, either formulating it as a link prediction task on the
graph given the event time or a time prediction problem given which event will
happen next. In this paper, we propose a novel model combining Graph Neural
Networks and Marked Temporal Point Process (MTPP) that jointly forecasts
multiple link events and their timestamps on communities over a CTDG.
Moreover, to scale our model to large graphs, we factorize the jointly event
prediction problem into three easier conditional probability modeling
problems.To evaluate the effectiveness of our model and the rationale behind
such a decomposition, we establish a set of benchmarks and evaluation metrics
for this event forecasting task. Our experiments demonstrate the superior
performance of our model in terms of both model accuracy and training
efficiency.

Temporal Point Process, Graph Neural Network, Temporal Graph, Event
Forecasting, Dynamic Graph, Community

††copyright: acmcopyright††journalyear: 2018††doi:
10.1145/1122445.1122456††conference: Make sure to enter the correct conference
title from your rights confirmation emai; June 03–05, 2018; Woodstock,
NY††price: 15.00††isbn: 978-1-4503-XXXX-X/18/06††ccs: Computing methodologies
Neural networks††ccs: Information systems Data mining††ccs: Theory of
computation Dynamic graph algorithms

##  1\. Introduction

Modeling and learning from dynamic interactions of entities has become an
important topic and inspired a great interest in wide application fields. In
particular, studying the evolution of community social events can enable
preemptive intervention for the pandemic (e.g., COVID-19) spreading (Zhang and
Nawata, 2018). Monitoring and forecasting the traffic congestion (Li et al.,
2018b) spreading can help the police prevent congestion from spreading outside
the community. The community needs more attention and help if there is a
sudden change in the state of the economic (Capistrán et al., 2010) or
political leanings (Sulakshin, 2010). In some cases, some entities, such as
those with dense connections, or with similar characteristics, may form
certain communities, and communities could also be defined by users based on
their criteria. In reality, people may only be interested in a specific
community of entities’ interactions in some practical applications, such as
community behavior modeling (Gu et al., 2017), dynamic community discovery
(Rossetti and Cazabet, 2018) and community outliers detection (Gao et al.,
2010).

Continuous Temporal Dynamic Graph (CTDG) is a common representation paradigm
for organizing dynamic interaction events over time, with edges and nodes
denoting the events and the pairwise involved entities, respectively. Each
edge also contains information about the timestamp of an event (assuming that
the event occurs instantaneously). For better understanding and forecasting
the events in a community, we propose the community event forecasting task
(Fig. 1) in a CTDG to predict a series of future events about not only _which_
two entities they will involve but also _when_ they will occur.

Figure 1. Community event forecasting on a CTDG: Given a community (nodes and
edges with blue strokes) in a historical CTDG, predict where and when the next
interaction event (green arrows) will happen. This process can be repeated to
forecast to distant future.

Problem Formulation. Formally, denote a CTDG as $G=(V,E_{T})$, where
$E_{T}=\\{\varepsilon_{i}:i=1,\cdots,T\\}$ is the set of edges,
$\varepsilon_{i}=(u_{i},v_{i},t_{i})$ with source node $u_{i}$, destination
node $v_{i}$, and timestamp $t_{i}$. The edges are ordered by timestamps,
i.e., $t_{i}\leq t_{j}$ given $1\leq i\leq j\leq T$. We further denote a CTDG
within a temporal window as $G_{i:j}=(V,E_{i:j})$ where
$E_{i:j}=\\{\varepsilon_{k}:i\leq k<j\\}$. Given the queried community (or
node candidates that people are interested in) $C_{q}\subset V$, predicting
$K$ future events within the community given $n$ observed events requires to
model the following conditional distribution:

(1) $p(\varepsilon_{n+1},\cdots,\varepsilon_{n+K}\mid G_{1:n},C_{q})$

where the distribution of each edge $\varepsilon_{n+i}$ is further a triple
joint probability distribution of its source and destination nodes as well as
its timestamp. Compared with traditional time series prediction, event
forecasting on a CTDG jointly consider the spatial information characterized
by the graph and the temporal signal characterized by the event stream in
order to make a more accurate prediction.

There have been several lines of work to approach the problem but all in
compromised settings. Graph Neural Networks (GNNs) (Wu et al., 2020b) is an
emerging family of deep neural networks for embedding structural information
via its message passing mechanism. Temporal Graph Neural Networks (TGNNs)
extends GNNs to CTDGs by incorporating temporal signals into the message
passing procedure. Among them, Jodie (Kumar et al., 2019), TGN (Rossi et al.,
2020) and TGAT (Xu et al., 2020) are designed for the _temporal link
prediction_ where the timestamp and one entity of the future event are given,
i.e., modeling the conditional distribution $p(v_{n+1}\mid
G_{1:n},u_{n+1},t_{n+1})$.  Know-Evolve (Trivedi et al., 2017) and DyRep
(Trivedi et al., 2019) combine GNNs with methods for multi-variate continuous
time series such as Temporal Point Process (TPP) for _event timestamp
prediction_ which predicts the time of the next event but requires the
entities to be known, i.e., modeling the conditional distribution
$p(t_{n+1}\mid G_{1:n},u_{n+1},v_{n+1})$. All these models are not directly
applicable to the community event forecasting problem on CTDGs.

Another line of work extend TPP methods to incorporate extra signals. Notably,
Marked TPP (MTPP) associates each event with a marker and jointly predicts the
marker as well as the timestamp of future events. Recurrent Marked Temporal
Point Process (RMTPP) (Du et al., 2016) proposes to use recurrent neural
networks to learn the conditional intensity and marker distribution. MTPP and
RMTPP are capable of predicting CTDG events by treating the entity pair
$(u_{i},v_{i})$ as the event marker. However, these kinds of MTPP-based
methods face three major drawbacks. To begin with, MTPP methods treat edges as
individual makers, which are unable to utilize the community and relationship
information, resulting in a suboptimal training solution. Besides, individual
makers also bring an $O(|V|^{2})$ marker distribution space, making model
unscalable to giant graphs. The last difficulty is that RMTPP is further
constrained by its recurrent structure, which must process each event
sequentially for keeping events’ contextual correlation.

Our contributions in this paper are:

i) We consider the community event forecasting task on a CTDG that jointly
predicts the next event’s incident nodes and timestamp within a certain
community, which is significantly harder than both temporal link prediction
and timestamp prediction. To the best of our knowledge, we are the first who
carefully consider and evaluate this task.

ii) We handle the Community Event Predicting task with a graph Point Process
model (CEP3), which incorporates both spatial and temporal signals using GNNs
and TPPs and can predict event entities and timestamps simultaneously. To
scale to large graphs, we factorize the mark distribution of MTPP and reduce
the computational complexity from $O(|V|^{2})$ of previous TPP attempts (Du et
al., 2016; Trivedi et al., 2019) to $O(|V|)$. Moreover, we employ the a time-
aware attention model to replace the TPP model’s recurrent structure,
significantly shortening the sequence length of each training step and
enabling training in mini-batches.

iii) We propose new benchmarks for the community event forecasting task on a
CTDG. Specifically, we design new evaluation metrics measuring prediction
quality of both entities and timestamps. For baselines, we collect and
carefully adapt state-of-the-art models from time series prediction, temporal
link prediction and timestamp prediction. Our evaluation shows that CEP3 is
superior across all four real-world graph datasets.

##  2\. Related Work

###  2.1. Temporal Graph Learning

Temporal Graph Learning aims at learning node embeddings using both structural
and temporal signals, which gives rise to a number of works. CTDNE (Nguyen et
al., 2018) and CAWs (Wang et al., 2021) incorporate temporal random walks into
skip-gram model for capturing temporal motif information in CTDGs. JODIE
(Kumar et al., 2019) and TigeCMN (Zhang et al., 2020) adopt recurrent neural
networks (RNNs) and attention-based memory module respectively to update node
embedding dynamically. Temporal Graph Neural Networks like TGAT (Xu et al.,
2020) and TGN (Rossi et al., 2020) enhance the attention-based message passing
process from Graph Neural Networks with Fourier time encoding kernel. These
attempts focus on the temporal link prediction task. Besides that, other works
(Li et al., 2017; Wu et al., 2019) focus on information diffusion task which
aims at predicting whether a user will perform an action at time $t$. None of
them is designed for event forecasting.

RE-Net (Jin et al., 2019) and CoNN (Gu et al., 2017) study a similar event
forecasting setting on Discrete Temporal Dynamic Graphs (DTDGs). However,
continuous time prediction is much harder and their methods cannot be directly
applied.

###  2.2. Temporal Point Process

A temporal point process (TPP) (Scargle, 2004) is a stochastic process
modeling the distribution of a sequence of events associated with continuous
timestamps $t_{1},\cdots,t_{n}$. A TPP is mostly characterized by a
conditional intensity function $\lambda(t)$, from which it computes the
conditional probability of an event occurring between $t$ and $t+dt$ given the
history $\\{t_{i}:t_{i}<t\\}$ as $\lambda(t)dt$. According to (Aalen et al.,
2008), the log conditional probability density of an event occurring at time
$t$ can be formulated as

(2) $f(t)=\log\lambda(t)-\int_{t_{n}}^{t}\lambda(\tau)d\tau$

Additionally, a marked temporal point process (MTPP) associates each event
with a _marker_ $y_{i}$ which is often regarded as the _type_ of the event.
MTPP thus not only models when an event would occur, but also models what type
of event it is. Event forecasting over CTDGs can also be modeled as a MTPP if
treating the event’s incident node pair as its marker. The conditional
intensity of the entire MTPP can be modeled as a sum of conditional
intensities of each individual marker: $\lambda=\sum_{m}\lambda_{m}$. This
allows it to first make the prediction of event time, and then predict the
marker conditioned on time via sampling from a categorical distribution:
$m\sim\text{Categorical}(\lambda_{m})$. This avoids modeling time and marker
jointly. Such idea has been widely used in follow-up works such as Recurrent
Marked Temporal Point Process (RMTPP) (Du et al., 2016). RMTPP parametrizes
the conditional intensity and the marker distribution with a recurrent neural
network (RNN). RMTPP’s variants include CyanRNN (Wang et al., 2017) and ARTPP
(Xiao et al., 2017b).

We also demonstrate several other relevant works and applications related to
TPPs (Shchur et al., 2021a). CoEvolving (Wang et al., 2016), a variant model
of MTPP, uses Hawkes processes to model the user-item interaction,
respectively. NeuralHakwes (Mei and Eisner, 2017) relaxes the positive
influence assumption of the Hawkes process by introducing a self-modulating
model. DeepTPP (Li et al., 2018a) models the event generation problem as a
stochastic policy and applied inverse reinforcement learning to efficiently
learn the TPP. However, these methods cannot be directly applied to event
forecasting on CTDGs due to the following reasons. First, a CTDG is
essentially a _single_ event sequence, whose length ranges from tens of
thousands to hundreds of millions. RNNs are known to have trouble dealing with
very long sequences. One may consider dividing the sequence into multiple
shorter windows, which will make the events disconnected within the given
window and discard all the data before it. This fails to explore the
dependencies between events that are distant over time but topologically
connected (i.e. sharing either of the incident nodes). One may also consider
training an RMTPP with Truncated BPTT (Jaeger, 2002) on the long sequence as a
whole. However, this is inefficient because parallel training is impossible
due to its recurrent nature, which means that one has to unroll the sequence
one event at a time. Second, directly modeling the marker generation
distribution with a unit softmax function will produce a vector with space
complexity of $O(|V|^{2})$, as the event markers will be essentially the
events’ incident node pairs. This is undesirable for large graphs.

(Farajtabar et al., 2017) models link and retweet generation on a social
network with a TPP, and also provides a simulation algorithm that generates
from the TPP model. It is very similar to our task except that it is focused
on a specific social network setting, and the authors did not quantitatively
evaluate the quality of the simulation model.

###  2.3. Temporal Point Process on Dynamic Graph

Previous works, such as (Trivedi et al., 2019; Wu et al., 2020a; Hall and
Willett, 2016), use kinds of recurrent architecture to approximate temporal
point process over graphs. However, recurrent architecture prevents the model
from parallelized minibatch training, which is undesirable especially on
large-scale graphs. This is because learning long-term dependencies using
recurrent architecture requires the model to traverse the event sequence one
by one instead of randomly mini-batch selection.

MMDNE (Lu et al., 2019), HTNE (Zuo et al., 2018) and DSPP (Cao et al., 2021)
employ the attention mechanism to avoid the inefficiency of the recurrent
structure in training with large CTDGs. However, these works are restrictively
dedicated to link prediction or timestamp prediction task. Adapting the two
models to event forecasting requires non-trivial changes since neither of them
handles efficient joint forecasting of the event’s incident nodes.

##  3\. Model

Our model is depicted in Fig. 2. To predict the next $K$ events
$\varepsilon_{n+1},\cdots,\varepsilon_{n+K}$ given the history graph
$G_{1:n}$, we first obtain an initial representation $h_{0}^{(\cdot)}$ for
every node using an GNN Encoder, as well as an initial graph $\tilde{G}_{0}$.
Then for the $i$-th step, we predict
$\varepsilon_{n+i}=(u_{n+i},v_{n+i},t_{n+i})$, i.e. the source node,
destination node, and timestamp for the next $i$-th event. The predicted event
is then added into $\tilde{G}_{i-1}$ to form $\tilde{G}_{i}$, to keep track of
what we have predicted so far. The hidden states $h_{i}^{(\cdot)}$ are then
updated from the new graph $\tilde{G}_{i}$ and $h_{i-1}^{(\cdot)}$. This
generally follows the framework of RMTPP (Du et al., 2016), except that i) we
initialize the recurrent network states with a time-aware GNN, which allows
our recurrent module to traverse over a much shorter sequence without losing
historical information, ii) we update the recurrent network states with a GNN
to model the topological dependencies between entities caused by new events,
and iii) we forecast the nodes and the timestamp for an event by decomposing
the joint probability distribution. We give specific details of each component
as follows.

Figure 2. The overall architecture of proposed CEP3 model. Red arrows
represent the predicted events $\varepsilon_{n+i}$.

###  3.1. Structural and Temporal GNN Encoder

The Encoder GNN should be capable of encoding relational dependencies,
timestamps, and optionally edge features at the same time. Therefore, it has
the following form:

(3)
$z_{l}^{(v)}=f_{\text{agg}}^{\text{init}}(\\{g_{\text{msg}}^{\text{init}}(z_{l-1}^{(u)},z_{l-1}^{(v)},e,\phi(t_{n}-t))\\\
\qquad:(u,v,t,e)\in\mathcal{N}^{v}[0:n]\\})$

where $\mathcal{N}^{v}[0,n]$ represents the neighborhood of $v$ on the graph
$G_{1:n}$ and $\phi(t)$ are learnable time encodings used in (Xu et al., 2020;
Rossi et al., 2020; Shchur et al., 2021b). $f_{\text{agg}}^{\text{init}}$ and
$g_{\text{msg}}^{\text{init}}$ can be any aggregation and message functions of
a GNN. $z_{0}^{(v)}$ could be either node $v$’s feature vector if available or
a zero vector otherwise. We then initialize $h_{0}^{(v)}=z_{L}^{(v)}$.

We use a GNN to initialize the recurrent network’s states because it takes all
the historical events within a topological local neighborhood, including the
incident nodes, the timestamps, and the feature of events together, as input,
while enabling us to train on multiple history graphs in parallel. In
particular, we use neighborhood graph temporal attention based method for
encoding, whose detailed formulation is as follows. Temporal Graph Attention
Module is a self attention based node embedding method inspired by (Xu et al.,
2020), the detailed formulation is as below:

(4) $\displaystyle\textbf{z}^{(l),t}_{i}=$
$\displaystyle\texttt{MLP}(\textbf{z}^{(l-1),t}_{i}||\tilde{\textbf{z}}_{i}^{(l),t})$
$\displaystyle\tilde{\textbf{h}}^{(l),t}_{i}=$
$\displaystyle\texttt{MultiHeadAttn}^{(l)}(q^{(l),t}_{i},K^{(l),t}_{i},V^{(l),t}_{i})$
$\displaystyle\textbf{q}^{(l),t}_{i}=$
$\displaystyle[z^{(l-1),t}_{i}||\phi(0)]$
$\displaystyle\textbf{K}^{(l),t}_{i}=$
$\displaystyle\textbf{V}^{(l),t}_{i}=\textbf{C}^{(l),t}_{i}$
$\displaystyle\textbf{C}^{(l),t}_{i}=$
$\displaystyle[h_{j}^{(l-1),t}||e_{ij}^{t_{j}}||\phi(t_{\text{src}}-t_{j}),j\in\mathcal{N}]$

For each source node at time $t_{\text{src}}$, we sample its two hop neighbors
whose timestamp is prior to $t_{\text{src}}$. The multi-head attention is
computed as:

(5)
$\displaystyle\tilde{\textbf{z}}_{(l)}^{t}=\sum_{a=0}^{n}\text{SoftMax}\left(\frac{(W_{Q,a}^{(l)}\textbf{q}_{i}^{(l),t})(W_{K,a}^{(l)}\textbf{K}_{i}^{(l),t})}{\sqrt{d_{k}}}\right)\left(W_{V,a}^{(l)}\textbf{V}_{i}^{(l),t}\right)$

The temporal encoding module is the same as in (Xu et al., 2020; Rossi et al.,
2020) original paper:

(6)
$\begin{gathered}\phi(\Delta{t})=\cos(\vec{\omega}{t}+\vec{b})\end{gathered}$

Where $\omega$ and b are learnable parameters.

Although a GNN cannot consider events outside the neighborhood, we argue that
such an impact is minimal. We empirically verify our belief by comparing
against the variant that uses both attention and RNN based memory module in
the training phase (named CEP3 w RNN), which can incorporate historical events
and time-aware information by the view of topological locality and recursive
impact, respectively. However, RNN takes drastically more memory and time in
training because of the same reason in Section 2.3.

###  3.2. Hierarchical Probability-Chain Forecaster

Figure 3. The hierarchical probability-chain forecaster and its workflow
relationship with the auto-regressive message passing module. The node
embeddings are learned from the GNN Encoder described in Section 3.1. Note
that the ‘selected community’ refers to an application-dependent collection of
candidate nodes for query.

Fig. 3 demonstrate the details of our event forecaster. We can see that the
forecaster predicts future events only according to node embeddings and
historical connections in the selected candidates (or communities). It means
that we do not have to be concerned about a large number of communities which
are likely to slow down the process, because CEP3 model can handle numerous
communities simultaneously during training and inference.

From the superposition property(Çinlar and Agnew, 1968) of an MTPP described
in Section 2.2, we forecast the next event
$\varepsilon_{n+i}=(u_{n+i},v_{n+i},t_{n+i})$ by a triple probability-chain:

(7) $p(u_{n+i},v_{n+i},t_{n+i})=p(t_{n+i})\\\ p(u_{n+i}\mid
t_{n+i})p(v_{n+i}\mid t_{n+i},u_{n+i})$

It means that we can predict first the timestamp, then the source node, and
finally the destination node. We predict $t_{n+i}$ by modeling the
distribution of the time difference $\Delta t_{n+i}=t_{n+i}-t_{n+i-1}$ as
follows:

(8)
$\begin{gathered}\lambda^{(v)}_{i}=\mathrm{Softplus}(\mathrm{MLP}_{\text{t}}(h_{i-1}^{(v)}))\\\
\lambda_{i}=\sum_{v}\lambda^{(v)}_{i}\\\ \Delta t_{n+i}\sim
Exponential(\lambda_{i})\end{gathered}$

where $\mathrm{Softplus}$ ensures that $\lambda_{i}^{(v)}$ is above zero and
the gradient still exists for negative $\lambda_{i}^{(v)}$ values.
$\mathrm{MLP}_{t}$ represents a multilayer perceptron to generate the
intensity value of time. Since $\Delta t_{n+i}$ obeys exponential
distribution, we simply sample the mean value of time intensity distribution,
$\frac{1}{\lambda_{i}}$, as the final $\Delta t_{n+i}$ output during training.
The conditional intensity of the next event occurring on _one_ of the nodes in
$V$ is thus the sum of all $\lambda_{i}^{(v)}$ (Çinlar and Agnew, 1968). Other
conditional intensity choices are also possible.

We then generate the source node $u_{n+i}$ of event $\varepsilon_{n+i}$ from a
categorical distribution conditioned on $t_{n+i}$, parametrized by another
MLP:

(9)
$p(u_{n+i})=\mathrm{Softmax}(\mathrm{MLP}_{\text{src}}(h_{i-1}^{(u)}\|\phi(\Delta
t_{n+i})))$

where $\|$ means concatenation and $\phi$ has the same form as in Eq. 3. We
then generate the destination node $v_{n+i}$ similarly, conditioned on
$t_{n+i}$ and $u_{n+i}$:

(10)
$p(v_{n+i})=\mathrm{Softmax}(\mathrm{MLP}_{\text{dst}}(h_{i-1}^{(v)}\|h_{i-1}^{(u_{n+i})}\|\phi(\Delta
t_{n+i})))$

Note that the formulation above will only generate two distributions that have
$|V|$ elements, instead of $|V|^{2}$ as in RMTPP (Du et al., 2016). The
implication is that during inference the strategy will be _greedy_ : we first
pick whatever source node that has the largest probability, then we pick the
destination node conditioned on the picked source node. To verify the impact
of this design choice, we also explored a variant of our method where we
generate a joint distribution of the pair $(u_{n+i},v_{n+i})$ with
$O(|V|^{2})$ elements, which we name CEP3 w/o HRCHY (short for Hierarchy).

If a dataset has a lot of nodes, evaluating $p(u_{n+i})$ and $p(v_{n+i})$ will
incur a linear projection with $O(|V|)$ complexity. This is another obstacle
of scaling up to larger datasets.

###  3.3. Auto-Regressive Message Passing

As shown in Fig. 3, we assume that an event’s occurrence will directly
influence the hidden states of its incident nodes. Moreover, the influence
will propagate to other nodes along the links created by historical
interactions. Therefore, after generating the new event $\varepsilon_{n+i}$,
we would like to update the nodes’ hidden states by message passing on the
graph with the new events. We achieve that by maintaining another graph
$\tilde{G}_{i}$ that keeps track of the graph with the historical interactions
$G_{1:n}$ and the newly predicted events up to $\varepsilon_{n+i}$.

Specifically, we initialize $\tilde{G}_{0}$ with the candidate node set $C$ as
its nodes. Two candidate nodes are connected in $\tilde{G}_{0}$ if their
distance is within $L$ hops. The resulting graph encompasses the dependency
between candidate nodes during the encoding stage. Every time a new event
$\varepsilon_{n+i}$ is predicted, we add the event back in:
$\tilde{G}_{i}=\tilde{G}_{i-1}\cup\varepsilon_{n+i}$.

Afterwards, we update the nodes’ hidden states using a message passing network
such as GCN (Kipf and Welling, 2017) for spatial propagation and a GRU (Cho et
al., 2014) for temporal propagation:

(11) $\displaystyle w_{i,0}^{(v)}$ $\displaystyle=h_{i-1}^{(v)}$
$\displaystyle w_{i,l}^{(v)}$
$\displaystyle=f_{\text{agg}}^{\text{upd}}(\\{g_{\text{msg}}^{\text{upd}}(w_{i,l-1}^{(u)},w_{i,l-1}^{(v)}):u\in\mathcal{N}_{\tilde{G}_{i}}^{v}\\})$
$\displaystyle h_{i}^{(v)}$
$\displaystyle=\mathrm{GRU}(\left[w_{i,P}^{(v)}\|\phi(\Delta
t_{n+1})\right],h_{i-1}^{(v)})$

where $\mathcal{N}_{\tilde{G}_{i}}^{v}$ is the neighboring events of node $v$
in $\tilde{G}_{i}$, $f_{\text{agg}}^{\text{upd}}$ can be any message
aggregation function and $g_{\text{msg}}^{\text{upd}}$ can be any message
function.

To verify the necessity of updating the community using message passing after
a event, we also explore a variant where we do not update all the node’s
hidden states in the community, but only the incident nodes $u_{n+i}$ and
$v_{n+i}$. We name this variant CEP3 w/o AR.

###  3.4. Loss Function and Prediction

The forecasting module outputs the next event’s timestamp $t_{n+i}$ and
indicent nodes $u_{n+i}$ and $v_{n+i}$ for all events $\varepsilon_{n+i}$,
which are minimized via negative log likelihood. Specifically, the loss
function goes as follows:

(12)
$\mathcal{L}_{\text{time}}=\sum_{i=1}^{K}[\underbrace{-\log(\lambda_{i})+\Delta
t_{n+i}\lambda_{i}}_{\text{time loss}}\\\ \underbrace{-\log p(u_{n+i})-\log
p(v_{n+i})}_{\text{entity loss}}]$

where the first two terms within summation are log survival probability from
Eq. 2 and the last two terms are log probabilities for source and destination
node prediction. The time integration term
$\int_{t_{n}}^{t}\lambda(\tau)d\tau$ in Eq. 2 is approximated using first
order integration method by $\lambda(t)\Delta t$ for the ease of computation.

##  4\. Experiments

###  4.1. Datasets

In this section, we test the performance and efficiency of the proposed method
against multiple baselines on four public real-world temporal graph datasets:
Wikipedia, MOOC (Kumar et al., 2019), GitHub (Trivedi et al., 2019), and
SocialEvo (Madan et al., 2011). Table 1 shows summary statistics of the
datasets used in our experiments. A detailed description is put in the below.

Table 1. Statistics of the datasets used in our experiments.

Level | Statistics | Wikipedia | MOOC | Github | SocialEvo
---|---|---|---|---|---
Graph level | Edges | 157,474 | 411,749 | 20,726 | 62,009
Nodes | 9,227 | 7,145 | 282 | 83
Max Degree | 1,937 | 19,474 | 4,790 | 15,356
Aver. Degree | 34 | 115 | 147 | 1,310
Edge Feat. Dim. | 172 | 4 | 10 | 10
Is Bipartite | True | True | False | False
Timespan | 31days | 30days | 1years | 74days
Edges/hour | 211.66 | 576.30 | 2.36 | 7.79
Data Spilt | 70%-15%-15% by timestamp order
Community level | Communities | 142 | 25 | 17 | 10
Max Nodes | 396 | 990 | 46 | 18
Aver. Nodes | 50.27 | 264.96 | 15.94 | 7.7
Max Edges | 4799 | 11686 | 3221 | 12199
Aver. Edges | 778.28 | 2560.00 | 534.71 | 3420.90
Min Edges | 77 | 16 | 34 | 863
Max Edges/hour | 6.47 | 33.33 | 0.36 | 1.99
Aver. Edges/hour | 1.11 | 5.36 | 0.06 | 0.56
Min Edges/hour | 0.14 | 0.34 | 0.01 | 0.15

Wikipedia (Kumar et al., 2019) dataset is widely used in temporal-graph-based
recommendation systems. It is a bipartite graph consisting of user nodes, page
nodes and edit events as interactions. We convert the text of each editing
into a edge feature vector representing their LIWC categories (Andrei et al.,
2014).

MOOC (Kumar et al., 2019) dataset, collected from a Chinese MOOC learning
platform XuetangX, consists of students’ actions on MOOC courses, e.g.,
viewing a video, submitting an answer, etc.

Github (Trivedi et al., 2019) dataset is a social network built from GitHub
user activities, where all nodes are real GitHub users and interactions
represent user actions to the other’s repository such as Watch, Fork, etc.
Note that we do not use the interaction types as we follow the same setting as
(Trivedi et al., 2019).

SocialEvo (Trivedi et al., 2019; Madan et al., 2011) dataset is a small social
network collected by MIT Human Dynamics Lab.

Since the public dataset SocialEvo and Github have no edge feature, we
generate a 10-dimensional edge feature using following attributes, including
the current degrees of the two incident nodes of an edge, and the time
differences between current timestamp and the last updated timestamps of the
two incident nodes. Note that the time differences are described in the
numbers of days, hours, minutes and seconds, respectively.

###  4.2. Baselines

Table 2. Comparison of model capabilities. Note that the usage of RNN prevents a model from parallel training as is mentioned in Section 2.3. ∗Requires non-trivial adaptation. Taxnomy | GNN+TPP | RNN+TPP | GNN | TPP
---|---|---|---|---
Methods | CEP3 | DyRep | RMTPP | TGAT | Poisson | Hawkes
Predicts Link (u,v) | $\surd$ | $\surd$ | $\surd$ | $\surd$ | $\surd$ | $\surd$
Predicts Continuous Time t | $\surd$ | $\surd$ | $\surd$ |  | $\surd$ | $\surd$
Jointly Predicts Event (u,v,t) | $\surd$ | $\surd^{*}$ | $\surd$ |  | $\surd$ | $\surd$
Explicitly Models Topological Dependency | $\surd$ | $\surd$ |  | $\surd$ |  |
Complexity of Node Prediction | $O(|V|)$ | $O(|V|^{2})$ | $O(|V|^{2})$ | $O(|V|^{2})$ | $O(|V|^{2})$ | $O(|V|^{2})$
Parallel Training | $\surd$ |  |  | $\surd$ | $\surd$ |
Captures Sequential Info with | Attention | RNN+Attention | RNN | Attention | Poisson Process | Hawkes Process

In addition to CEP3, CEP3 w RNN, and CEP3 w/o AR mentioned in Section 3, we
compare against the following baselines: a Seq2seq model with a GRU (Cho et
al., 2014), a Poisson Process (TPP-Poisson), a Hawkes Process (TPP-Hawkes)
(Hawkes, 1971), RMTPP (Du et al., 2016) and its variant with the same two-
level hierarchical factorization as in Eq. 9 and 10 (RMTPP w HRCHY), an
adaptation of DyRep (Trivedi et al., 2019) and an auto-regressive variant
(named DyRep and DyRep w AR). Notably, RMTPP and its variants are SOTA models
for MTPPs in general, and DyRep is SOTA in temporal link prediction and time
prediction on CTDG. Since our task is new, we made adaptations to the
baselines above, with details of each baseline is as follows:

Time series Methods: For baseline model of sequential prediction, we build an
RNN model, Gated Recurrent Unit (GRU). Each source and destination cell has a
hidden state, the output of the model will be predicted time mean and variance
as well as probability for each class to interact, the time will be formulated
as Gaussian distribution and source and destination node will be formulated as
categorical distribution. This formulation forces GRU predicting timestamp of
upcoming events only depending on the hidden state in RNN, whereas other
baselines adapt TPP function as a stochastic probability process, obtaining a
better modeling capability. We use a GRU (Cho et al., 2014) as a simple
baseline without using TPP to model time distribution, treating event
forecasting on CTDG as a sequential modeling task. It takes in the event
sequence and outputs the next $K$ event in a Seq2seq fashion (Sutskever et
al., 2014). The loss term for time prediction is mean squared error and the
loss term for source and destination prediction is the negative log
likelihood. This formulation forces GRU to predict the timestamp of upcoming
events only depending on the hidden state in RNN, whereas other baselines
adapt TPP for better modeling capabilities.

Temporal Point Process Methods: Following the benchmark setting of (Du et al.,
2016), we compared our model against other traditional TPP models and deep TPP
models.

  * •

TPP-Poisson: We assume that the events occurring at each node pair $(u,v)$
follows a Poisson Process with a constant intensity value $\lambda_{u,v}$,
which are learnt from data via Maximum Likelihood Estimation (MLE).

  * •

TPP-Hawkes (Hawkes, 1971): We assume that the events occurring at each node
pair $(u,v)$ follows a Hawkes Process with a base intensity value $\mu_{u,v}$
and an excite parameter $\alpha_{u,v}$, which are learnt from data via MLE.

  * •

RMTPP (Du et al., 2016): We directly consider each source and destination node
pair as a unique marker. We note that this formulation will exhaust memory and
time on graphs with more than a few thousand nodes, since RMTPP will assign a
learnable embedding for each node pair, resulting in $O(|V|^{2})$ (Here $V$ is
total number of nodes in the entire CTDG) parameters which is too expensive to
update.

  * •

RMTPP w HRCHY: We consider a variant of RMTPP where we replace the source-
destination node prediction with our hierarchical formulation: we first select
the source node, then condition on the source node we select the destination
node. The latter formulation can also serves as an ablation study to
demonstrate the usage of considering the graph structure.

  * •

DyRep (Trivedi et al., 2019): To the best of our knowledge, DyRep is the most
popular work that combine the temporal point process with graph learning
techniques to model both temporal and spatial dependencies. Since the original
DyRep formulation only handled temporal link prediction and time prediction,
but not autoregressive forecasting, we compute an intensity value
$\lambda_{u,v}$ with DyRep for each node pair and assumed a Poisson Process
afterwards.

  * •

DyRep w AR: We made a trivial adaptation to the original formulation of DyRep
by updating the source and destination node involved once a new edge is added
to the graph. The update function is identical to DyRep updating function
during embedding. This benchmark is designed to demonstrate that the
propagation from newly forecast event to local neighborhood is necessary in
getting better performance.

We also crave our proposed model for having the ability to implement minibatch
training. As described in the beginning of Section 3, our CEP3 achieves large-
scale and parallelized training by utilizing Hierarchical TPP and GNN based
updating module, respectively. A summary of the mentioned baselines is shown
in Table 2, the proposed model CEP3 satisfies all the desirable properties.

###  4.3. Configuration

We provide the details of our network architecture, the hyper-parameters and
the selected community detection method for better reproducibility. Table 3
summarized other key parameters in our model. From hardware perspective, for
all the experiments we train our model and benchmark models on Intel(R)
Xeon(R) Platinum 8375C CPU @ 2.90GHz. Our code is based on PyTorch and Deep
Graph Library (Wang et al., 2019), and will be published after acceptation.

Table 3. Configurations for our CEP3 and all baselines. Name | Value
---|---
Hidden Dim in Encoder | 100
Hidden Dim in Forecaster | 50
Community Dim | 10
Layers in MLPs | 2
K-hops | 2
Sampled neighbors/Hop | 15
Learning rate | 0.0001
Optimizer | Adam
# of attn head | 4
Recurrent Module | GRU
Epochs | 100
Forecasting Window | 200 Steps
Community Detection Method | Louvain

###  4.4. Evaluation Metrics

One of the most important contributions of our study is the concept of
community event prediction, which is a novel task in the field of temporal
graphs. To the best of our knowledge, we are the first to consider forecasting
events sampled from a temporal point process model, given a historical graph
and a candidate node set in which the user is interested. For evaluation on a
specific dataset, we utilize the communities segmented by the conventional
community detection algorithm Louvain (Blondel et al., 2008) as the candidate
node set, and we report the average result of all communities.

For each community $C_{q}$, we measure the perplexity ($PP_{C_{q}}$) of the
ground truth source and destination node sequence for evaluating the node
predicting performance, and evaluate the mean absolute error ($MAE_{C_{q}}$)
of the predicted timestamps. Using our MAE to evaluate the quality of auto-
regressive forecasted sequence over multiple timesteps can also be seen in
traffic flow prediction (Li et al., 2018b).

Perplexity (PP) (Meister and Cotterell, 2021) is a concept in information
theory that assesses how closely a probability model’s projected outcome
matches the real sample distribution. The less perplexity the situation, the
higher the model’s prediction confidence. In the field of natural language
processing, perplexity is also commonly used to evaluate a language model’s
quality, i.e., to evaluate how closely the sentences generated by the language
model match real human language samples. A language model predicts the next
word from the word dictionary, whereas our event predicting model selects
nodes from the node candidates (community). Therefore, it is reasonable to
employ perplexity as a metric in our task.

Specifically, suppose we have the communities’ ground truth event sequence
$(u_{i},v_{i},t_{i})$ and the prediction sequence
$(\hat{u}_{i},\hat{v}_{i},\hat{t}_{i})$ where $i=1,\cdots,K$. For we compute
per step PP as

(13) $\small PP=\exp{\left(-\frac{1}{K}\sum_{i=1}^{K}\left[\log p(u_{i})+\log
p(v_{i}\mid u_{i})\right]\right)}$

In order to measure the distance between two sequence with difference length,
we compute MAE following (Xiao et al., 2017a) as:

(14)
$MAE=\frac{1}{K(t_{K}-t_{0})}\sum_{i=1}^{K}\left[\left|t_{i}-\min(t_{K},\hat{t}_{i})\right|\right]$

To keep it comparable in diverse datasets, the MAE is divided by the max time
span $t_{K}-t_{0}$ and the sequence length $K$. We report the average
$PP_{C_{q}}$ of all communities as the $PP$ of a certain dataset, and $MAE$ is
calculated in the same way. Smaller values of both metrics indicate the better
model performance.

###  4.5. Result Analysis

From Table 4 we can see the obvious advantage of CEP3 compared to other
baselines in different datasets under both MAE and perplexity. The MAE
difference between GRU and RMTPP show the effectiveness of temporal point
process in predicting timestamps. Comparing CEP3 with sequence based TPP
models RMTPP, we can see that using GNN to capture historical interaction
information can improve the forecasting performance. Further more, when
comparing DyRep w AR versus DyRep and CEP3 w/o AR versus pure CEP3, we can
conclude that auto-regressive updates can better capture the impact of newly
predicted events.

Our model performs better than DyRep w AR because in our CEP3, during the
auto-regressive update, the newly predicted event not only influences the node
involved in the event but also propagates to other nodes via message passing.

###  4.6. Forecasting Visualization

(a) Ground Truth.

(b) CEP3.

(c) DyRep.

Figure 4. The prediction visualizations of certain communities in the whole
timespan of the test phase. The sizes of plotted nodes indicate their degrees,
whereas the colors of edges represent the connection frequencies. Note that
the edge colors and the node sizes can only be compared in the same row. The
communities are from Github, Wikipedia and MOOC datasets, respectively, from
top to bottom.  Table 4. Comparison of the average and standard deviation of
perplexity of incident node prediction and mean absolute error of time
prediction. The smaller the MAE and Perplexity, the better the model. Numbers
are ranked by mean followed by standard deviation. The best result is
highlighted in bold and second best is highlighted with underline. The Rank
column shows the average ranking in each metric and dataset (the lower, the
better).

Datasets | Wikipedia | Github | MOOC | SocialEvo | Rank
---|---|---|---|---|---
Metric | Perplexity | MAE | Perplexity | MAE | Perplexity | MAE | Perplexity | MAE
GRU+Gaussian | 131.06 $\pm$ 11.27 | 54.54 $\pm$ 1.19 | 68.53 $\pm$ 1.18 | 59.05 $\pm$ 1.72 | 457.40 $\pm$ 6.25 | 36.49 $\pm$ 2.01 | 33.85 $\pm$ 0.27 | 131.71 $\pm$ 7.09 | 8.00
Hawkes | 108.00 $\pm$ 3.73 | 56.84 $\pm$ 0.31 | 74.40 $\pm$ 2.47 | 55.21 $\pm$ 0.12 | 502.31 $\pm$ 12.30 | 36.67 $\pm$ 0.29 | 45.33 $\pm$ 5.35 | 139.35 $\pm$ 0.17 | 9.50
Poisson | 119.19 $\pm$ 1.11 | 56.70 $\pm$ 0.11 | 61.49 $\pm$ 0.96 | 55.21 $\pm$ 0.31 | 438.61 $\pm$ 7.05 | 36.61 $\pm$ 0.78 | 40.48 $\pm$ 1.99 | 139.3 $\pm$ 1.15 | 8.25
RMTPP w HRCHY | 133.68 $\pm$ 2.31 | 34.15 $\pm$ 0.89 | 62.19 $\pm$ 0.88 | 55.05 $\pm$ 1.02 | 616.79 $\pm$ 25.74 | 32.29 $\pm$ 1.59 | 41.37 $\pm$ 6.55 | 140.02 $\pm$ 2.06 | 8.88
RMTPP | 121.67 $\pm$ 1.01 | 32.91 $\pm$ 1.90 | 67.97 $\pm$ 1.02 | 54.79 $\pm$ 0.47 | 664.07 $\pm$ 11.05 | 32.83 $\pm$ 2.40 | 37.05 $\pm$ 0.77 | 138.9 $\pm$ 2.30 | 8.00
DyRep w AR | 116.07 $\pm$ 4.98 |  28.74 $\pm$ 0.37 | 54.57 $\pm$ 1.82 |  28.46 $\pm$ 0.65 | 431.18 $\pm$ 1.18 | 29.92 $\pm$ 1.48 | 29.6 $\pm$ 1.93 | 99.96 $\pm$ 6.18 | 3.38
DyRep | 119.13 $\pm$ 1.02 | 30.04 $\pm$ 0.14 | 64.05 $\pm$ 0.78 | 36.97 $\pm$ 1.74 | 438.61 $\pm$ 9.28 | 13.41 $\pm$ 1.42 | 36.59 $\pm$ 3.02 | 103.01 $\pm$ 3.49 | 4.75
CEP3 w RNN |  104.87 $\pm$ 8.70 | 41.94 $\pm$ 1.89 | 60.18 $\pm$ 1.04 | 39.22 $\pm$ 2.93 |  374.77 $\pm$ 24.59 | 20.09 $\pm$ 0.33 |  30.37 $\pm$ 4.56 | 95.12 $\pm$ 2.25 | 3.88
CEP3 w/o HRCHY | 98.98 $\pm$ 7.61 | 28.69 $\pm$ 0.70 |  52.04 $\pm$ 3.33 | 26.8 $\pm$ 0.89 | 365.68 $\pm$ 28.01 | 31.87 $\pm$ 0.18 | 28.66 $\pm$ 2.74 | 79.58 $\pm$ 5.39 | 1.75
CEP3 w/o AR | 125.51 $\pm$ 7.64 | 39.31 $\pm$ 2.59 | 61.03 $\pm$ 1.03 | 34.03 $\pm$ 0.37 | 448.37 $\pm$ 4.34 | 21.4 $\pm$ 0.47 | 38.59 $\pm$ 1.02 | 95.21 $\pm$ 4.44 | 6.13
CEP3 | 118.82 $\pm$ 4.30 | 32.41 $\pm$ 0.58 | 50.42 $\pm$ 0.70 | 30.93 $\pm$ 1.67 | 401.64 $\pm$ 7.06 |  17.69 $\pm$ 2.68 | 36.8 $\pm$ 1.00 |  94.54 $\pm$ 7.31 |  3.25

From top to bottom, Fig. 4 visualizes the circle layout of a certain community
within the graphs of the Github, Wikipedia and MOOC datasets, respectively. We
plot the ground truth, our model’s prediction and DyRep’s prediction for
comparison. The visualized graphs are generated as follows: We first apply the
learned forecasting model to predict the edges using Monte Carlo sampling.
This generation process is repeated for three times. We then discard generated
edges that are not within the 33% highest predicted edge probabilities and
obtain the final generated graph. Generating multiple times and then
discarding the less possible edges is to reduce uncertainty in each one-time
generated graphs.

In the first row, both CEP3 and DyRep capture the triangle connection in this
small community. However, the triangle is lighter in the ground truth, which
means that DyRep over-reinforces this connection in its predictions. In the
second row, the prediction result of CEP3 is more similar to the truth,
whereas DyRep generates a high-frequency purple edge which does not exist in
the original graph. In the third row, CEP3 successfully learns the two black
edges in ground truth, but DyRep predicts more than two links in darker
colors.

We can see that our method successfully recognises the high-degree nodes
captured many patterns of interactions as well as the evolution dynamics of
the interactions graph, which includes the nodes with higher degrees. The
fundamental goal of community event prediction, rather than focusing on a
single local node, is to anticipate if a certain high-frequency pattern will
emerge in the community from a global perspective. Exploring money laundering
patterns (Savage et al., 2017) in finance and disease transmission patterns
(Das et al., 2020) in communities are two of the most important examples.

###  4.7. Ablation Study

In Section 3 we have mentioned three variants: CEP3 w RNN to trade
parallelized training for long-term dependency modeling with RNN based memory
module, CEP3 w/o HRCHY to compare hierarchical (HRCHY) prediction versus joint
prediction of incident nodes, and CEP3 w/o AR to compare using auto-regressive
module versus not using.

CEP3 w/o HRCHY. The formulation without hierarchy structure yields a slower
training and inference speed as shown in Fig. 5. We demonstrate that CEP3 is
more efficient on large communities compared to CEP3 w/o HRCHY. As is shown in
Fig. 5, computing intensity function of each possible node pair would take
more time by orders of magnitude. We relax this issue by decomposing the node
pair prediction problem into two node prediction sub-problems, which can be
solved quicker especially in large scale communities.

Figure 5. To demonstrate the efficiency of proposed CEP3 hierarchical
probability chain, we show 1000 steps’ inference time cost against the node
scale. In this scatter figure, each data point represents a community in the
Wikipedia dataset.

CEP3 w/o AR. The model performance dropped significantly and yielded similar
result as DyRep w AR. From Fig. 6, we can see that the AR forecasting is not
helpful in all varying numbers of prediction steps. When the number of
“prediction steps” is small (such as 10), CEP3 could just use the node
embeddings at time $t_{n}$ to predict the events. When the number of steps
becomes larger, the systematic accumulated errors from AR gradually
accumulate, leading to low MAE accuracy. And as the number of steps increases,
the initial node embedding would have little effect in distant future events.
This indicates that without AR, it may be difficult for the CEP3 model to
accurately predict long-term occurrences.

Figure 6. The number offorecasting step is an important parameter in almost
all forecasting models. To further investigate the effect of the number of
forecasting steps and the AR module on the CEP3 model, we run experiments with
different numbers of forecasting steps. The small MAE, the better the model.

CEP3 w RNN. The results show that the memory module brings performance
improvement compared with pure CEP3, except on the Github dataset. The reason
is that the Github dataset is a small dataset with a small number of nodes and
edges, and meanwhile it has a significantly longer timespan than other
datasets. It means that the interaction is low-frequency in this situation,
causing the insufficient memory updating.

###  4.8. Parallel Training

Figure 7. Training loss curve of different parallel sizes.

Inspired by the Transformer (Vaswani et al., 2017) models in NLP, we believe
it is essential to use a pure attention-based model in training temporal graph
encoders. This is because using a pure attention-based GNN as an encoder
enables us to train multiple time windows in minibatches as described in
Section 3.1, whereas the model such as Dyrep and RMTPP cannot utilize parallel
training due to their RNN structures. This property allows our model to
benefit from mini-batch training such as gradient stabilization and faster
convergence. 7 shows our experiment result on the Wikipedia dataset with
different numbers of parallel processes, which suggests that using parallel
training can increase the speed significantly without losing accuracy.

##  5\. Conclusion and Future Work

In this paper, we present a novel community event forecasting task on a
continuous time dynamic graph, and set up benchmarks using adaptation of the
previous work. We further propose a new model to tackle this problem utilizing
graph structures. We also address the scalability problem when formulating the
temporal point process on graph and reduce complexity with a hierarchical
formulation.

For future work, we are aware that current evaluation metrics of this task
focus only on either time or event type. Although we perform a community
visualization to show the event type prediction in a time window, a
comprehensive joint metric is preferred to better evaluate the event
forecasting task.

## References

  * (1)
  * Aalen et al. (2008) Odd Aalen, Ornulf Borgan, and Hakon Gjessing. 2008.  _Survival and event history analysis: a process point of view_.  Springer Science & Business Media.
  * Andrei et al. (2014) Amanda Andrei, Alison Dingwall, Theresa Dillon, and Jennifer Mathieu. 2014.  Developing a Tagalog Linguistic Inquiry and Word Count (LIWC) ’Disaster’ Dictionary for Understanding Mixed Language Social Media: A Work-in-Progress Paper. In _LaTeCH@EACL_. The Association for Computer Linguistics, 91–94.
  * Blondel et al. (2008) Vincent D Blondel, Jean-Loup Guillaume, Renaud Lambiotte, and Etienne Lefebvre. 2008.  Fast unfolding of communities in large networks.  _Journal of statistical mechanics: theory and experiment_ 2008, 10 (2008), P10008.
  * Cao et al. (2021) Jiangxia Cao, Xixun Lin, Xin Cong, Shu Guo, Hengzhu Tang, Tingwen Liu, and Bin Wang. 2021.  Deep Structural Point Process for Learning Temporal Interaction Networks. In _ECML/PKDD_. Springer, 447–457.
  * Capistrán et al. (2010) Carlos Capistrán, Christian Constandse, and Manuel Ramos-Francia. 2010.  Multi-horizon inflation forecasts using disaggregated data.  _Economic Modelling_ 27, 3 (2010), 666–677.
  * Cho et al. (2014) Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014\.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation. In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_. Association for Computational Linguistics, Doha, Qatar, 1724–1734.  https://doi.org/10.3115/v1/D14-1179
  * Das et al. (2020) Ashis Kumar Das, Shiba Mishra, and Saji Saraswathy Gopalan. 2020\.  Predicting CoVID-19 community mortality risk using machine learning and development of an online prognostic tool.  _PeerJ_ 8 (2020), e10083.
  * Du et al. (2016) Nan Du, Hanjun Dai, Rakshit Trivedi, Utkarsh Upadhyay, Manuel Gomez-Rodriguez, and Le Song. 2016\.  Recurrent marked temporal point processes: Embedding event history to vector. In _Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_. 1555–1564.
  * Farajtabar et al. (2017) Mehrdad Farajtabar, Yichen Wang, Manuel Gomez-Rodriguez, Shuang Li, Hongyuan Zha, and Le Song. 2017\.  Coevolve: A joint point process model for information diffusion and network evolution.  _The Journal of Machine Learning Research_ 18, 1 (2017), 1305–1353.
  * Gao et al. (2010) Jing Gao, Feng Liang, Wei Fan, Chi Wang, Yizhou Sun, and Jiawei Han. 2010\.  On community outliers and their efficient detection in information networks. In _KDD_. ACM, 813–822.
  * Gu et al. (2017) Yupeng Gu, Yizhou Sun, and Jianxi Gao. 2017.  The Co-Evolution Model for Social Network Evolving and Opinion Migration. In _KDD_. ACM, 175–184.
  * Hall and Willett (2016) Eric C. Hall and Rebecca M. Willett. 2016.  Tracking Dynamic Point Processes on Networks.  _IEEE Trans. Inf. Theory_ 62, 7 (2016), 4327–4346.
  * Hawkes (1971) Alan G Hawkes. 1971\.  Spectra of some self-exciting and mutually exciting point processes.  _Biometrika_ 58, 1 (1971), 83–90.
  * Jaeger (2002) Herbert Jaeger. 2002\.  _Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the” echo state network” approach_. Vol. 5.  GMD-Forschungszentrum Informationstechnik Bonn.
  * Jin et al. (2019) Woojeong Jin, Changlin Zhang, Pedro A. Szekely, and Xiang Ren. 2019.  Recurrent Event Network for Reasoning over Temporal Knowledge Graphs.  _CoRR_ abs/1904.05530 (2019).  arXiv:1904.05530 http://arxiv.org/abs/1904.05530
  * Kipf and Welling (2017) Thomas N. Kipf and Max Welling. 2017.  Semi-Supervised Classification with Graph Convolutional Networks. In _ICLR (Poster)_. OpenReview.net.
  * Kumar et al. (2019) Srijan Kumar, Xikun Zhang, and Jure Leskovec. 2019\.  Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks. In _KDD_. ACM, 1269–1278.
  * Li et al. (2017) Dong Li, Shengping Zhang, Xin Sun, Huiyu Zhou, Sheng Li, and Xuelong Li. 2017\.  Modeling Information Diffusion over Social Networks for Temporal Dynamic Prediction.  _IEEE Trans. Knowl. Data Eng._ 29, 9 (2017), 1985–1997.
  * Li et al. (2018a) Shuang Li, Shuai Xiao, Shixiang Zhu, Nan Du, Yao Xie, and Le Song. 2018a.  Learning Temporal Point Processes via Reinforcement Learning. In _NeurIPS_. 10804–10814.
  * Li et al. (2018b) Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. 2018b.  Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting. In _International Conference on Learning Representations_.
  * Lu et al. (2019) Yuanfu Lu, Xiao Wang, Chuan Shi, Philip S. Yu, and Yanfang Ye. 2019.  Temporal Network Embedding with Micro- and Macro-dynamics. In _CIKM_. ACM, 469–478.
  * Madan et al. (2011) Anmol Madan, Manuel Cebrian, Sai Moturu, Katayoun Farrahi, et al. 2011\.  Sensing the” health state” of a community.  _IEEE Pervasive Computing_ 11, 4 (2011), 36–45.
  * Mei and Eisner (2017) Hongyuan Mei and Jason Eisner. 2017.  The Neural Hawkes Process: A Neurally Self-Modulating Multivariate Point Process. In _NIPS_. 6754–6764.
  * Meister and Cotterell (2021) Clara Meister and Ryan Cotterell. 2021.  Language Model Evaluation Beyond Perplexity. In _ACL/IJCNLP (1)_. Association for Computational Linguistics, 5328–5339.
  * Nguyen et al. (2018) Giang Hoang Nguyen, John Boaz Lee, Ryan A Rossi, Nesreen K Ahmed, Eunyee Koh, and Sungchul Kim. 2018.  Continuous-time dynamic network embeddings. In _Companion Proceedings of the The Web Conference 2018_. 969–976.
  * Rossetti and Cazabet (2018) Giulio Rossetti and Rémy Cazabet. 2018.  Community Discovery in Dynamic Networks: A Survey.  _ACM Comput. Surv._ 51, 2 (2018), 35:1–35:37.
  * Rossi et al. (2020) Emanuele Rossi, Ben Chamberlain, Fabrizio Frasca, Davide Eynard, Federico Monti, and Michael M. Bronstein. 2020.  Temporal Graph Networks for Deep Learning on Dynamic Graphs.  _CoRR_ abs/2006.10637 (2020).
  * Savage et al. (2017) David Savage, Qingmai Wang, Xiuzhen Zhang, Pauline Chou, and Xinghuo Yu. 2017.  Detection of Money Laundering Groups: Supervised Learning on Small Networks. In _AAAI Workshops_ _(AAAI Technical Report, Vol. WS-17)_. AAAI Press.
  * Scargle (2004) Jeffrey D. Scargle. 2004\.  An Introduction to the Theory of Point Processes, Vol. I: Elementary Theory and Methods.  _Technometrics_ 46, 2 (2004), 257.
  * Shchur et al. (2021a) Oleksandr Shchur, Ali Caner Türkmen, Tim Januschowski, and Stephan Günnemann. 2021a.  Neural Temporal Point Processes: A Review. In _IJCAI_. ijcai.org, 4585–4593.
  * Shchur et al. (2021b) Oleksandr Shchur, Ali Caner Türkmen, Tim Januschowski, and Stephan Günnemann. 2021b.  Neural Temporal Point Processes: A Review. In _IJCAI_. ijcai.org, 4585–4593.
  * Sulakshin (2010) Stepan S Sulakshin. 2010\.  A Quantitative Political Spectrum and Forecasting of Social Evolution.  _International Journal of Interdisciplinary Social Sciences_ 5, 4 (2010).
  * Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014\.  Sequence to Sequence Learning with Neural Networks. In _Advances in Neural Information Processing Systems_ , Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Q. Weinberger (Eds.), Vol. 27. Curran Associates, Inc.
  * Trivedi et al. (2017) Rakshit Trivedi, Hanjun Dai, Yichen Wang, and Le Song. 2017\.  Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphs. In _ICML_ _(Proceedings of Machine Learning Research, Vol. 70)_. PMLR, 3462–3471.
  * Trivedi et al. (2019) Rakshit Trivedi, Mehrdad Farajtabar, Prasenjeet Biswal, and Hongyuan Zha. 2019.  DyRep: Learning Representations over Dynamic Graphs. In _ICLR (Poster)_. OpenReview.net.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017\.  Attention is All you Need. In _NIPS_. 5998–6008.
  * Wang et al. (2019) Minjie Wang, Lingfan Yu, Da Zheng, Quan Gan, Yu Gai, Zihao Ye, Mufei Li, Jinjing Zhou, Qi Huang, Chao Ma, Ziyue Huang, Qipeng Guo, Hao Zhang, Haibin Lin, Junbo Zhao, Jinyang Li, Alexander J Smola, and Zheng Zhang. 2019.  Deep Graph Library: Towards Efficient and Scalable Deep Learning on Graphs.  _ICLR Workshop on Representation Learning on Graphs and Manifolds_ (2019).
  * Wang et al. (2021) Yanbang Wang, Yen-Yu Chang, Yunyu Liu, Jure Leskovec, and Pan Li. 2021.  Inductive Representation Learning in Temporal Networks via Causal Anonymous Walks. In _International Conference on Learning Representations_.
  * Wang et al. (2016) Yichen Wang, Nan Du, Rakshit Trivedi, and Le Song. 2016\.  Coevolutionary Latent Feature Processes for Continuous-Time User-Item Interactions. In _NIPS_. 4547–4555.
  * Wang et al. (2017) Yongqing Wang, Huawei Shen, Shenghua Liu, Jinhua Gao, and Xueqi Cheng. 2017.  Cascade Dynamics Modeling with Attention-based Recurrent Neural Network. In _Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19-25, 2017_ , Carles Sierra (Ed.). ijcai.org, 2985–2991.  https://doi.org/10.24963/ijcai.2017/416
  * Wu et al. (2019) Qitian Wu, Yirui Gao, Xiaofeng Gao, Paul Weng, and Guihai Chen. 2019.  Dual Sequential Prediction Models Linking Sequential Recommendation and Information Dissemination. In _KDD_. ACM, 447–457.
  * Wu et al. (2020a) Weichang Wu, Huanxi Liu, Xiaohu Zhang, Yu Liu, and Hongyuan Zha. 2020a.  Modeling Event Propagation via Graph Biased Temporal Point Process.  _IEEE Transactions on Neural Networks and Learning Systems_ (2020), 1–11.
  * Wu et al. (2020b) Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip. 2020b.  A comprehensive survey on graph neural networks.  _IEEE transactions on neural networks and learning systems_ 32, 1 (2020), 4–24.
  * Xiao et al. (2017a) Shuai Xiao, Mehrdad Farajtabar, Xiaojing Ye, Junchi Yan, Le Song, and Hongyuan Zha. 2017a.  Wasserstein learning of deep generative point process models.  _arXiv preprint arXiv:1705.08051_ (2017).
  * Xiao et al. (2017b) Shuai Xiao, Junchi Yan, Xiaokang Yang, Hongyuan Zha, and Stephen M. Chu. 2017b.  Modeling the Intensity Function of Point Process Via Recurrent Neural Networks. In _Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA_ , Satinder P. Singh and Shaul Markovitch (Eds.). AAAI Press, 1597–1603.
  * Xu et al. (2020) Da Xu, Chuanwei Ruan, Evren Körpeoglu, Sushant Kumar, and Kannan Achan. 2020.  Inductive representation learning on temporal graphs. In _ICLR_. OpenReview.net.
  * Zhang and Nawata (2018) J Zhang and K Nawata. 2018\.  Multi-step prediction for influenza outbreak by an adjusted long short-term memory.  _Epidemiology & Infection_ 146, 7 (2018), 809–816.
  * Zhang et al. (2020) Zhen Zhang, Jiajun Bu, Martin Ester, Jianfeng Zhang, Chengwei Yao, Zhao Li, and Can Wang. 2020.  Learning Temporal Interaction Graph Embedding via Coupled Memory Networks. In _WWW_. ACM / IW3C2, 3049–3055.
  * Zuo et al. (2018) Yuan Zuo, Guannan Liu, Hao Lin, Jia Guo, Xiaoqian Hu, and Junjie Wu. 2018\.  Embedding Temporal Network via Neighborhood Formation. In _KDD_. ACM, 2857–2866.
  * Çinlar and Agnew (1968) E. Çinlar and R. A. Agnew. 1968.  On the Superposition of Point Processes.  _Journal of the Royal Statistical Society: Series B (Methodological)_ 30, 3 (1968), 576–581.