# Multimodal Graph Learning for Modeling Emerging Pandemics with Big Data

Khanh-Tung Tran  Department of Computing Science, Umeå University, Sweden  AI
Center, FPT Software, Hanoi, Vietnam  Truong Son Hy  Department of Mathematics
and Computer Science, Indiana State University, USA  Lili Jiang  Department of
Computing Science, Umeå University, Sweden  Xuan-Son Vu  Department of
Computing Science, Umeå University, Sweden  Corresponding author. Email:
<EMAIL_ADDRESS>
###### Abstract

Accurate forecasting and analysis of emerging pandemics play a crucial role in
effective public health management and decision-making. Traditional approaches
primarily rely on epidemiological data, overlooking other valuable sources of
information that could act as sensors or indicators of pandemic patterns. In
this paper, we propose a novel framework called MGL4MEP that integrates
temporal graph neural networks and multi-modal data for learning and
forecasting. We incorporate big data sources, including social media content,
by utilizing specific pre-trained language models and discovering the
underlying graph structure among users. This integration provides rich
indicators of pandemic dynamics through learning with temporal graph neural
networks. Extensive experiments demonstrate the effectiveness of our framework
in pandemic forecasting and analysis, outperforming baseline methods across
different areas, pandemic situations, and prediction horizons. The fusion of
temporal graph learning and multi-modal data enables a comprehensive
understanding of the pandemic landscape with less time lag, cheap cost, and
more potential information indicators.

## Introduction

Pandemics are global outbreaks of infectious diseases that affect many people
across continents. The COVID-19 pandemic is one of the most significant
pandemics of our time, impacting millions of individuals worldwide and causing
lasting effects on our society. In order to _combat_ pandemics, it is crucial
to develop efficient solutions that facilitate the comprehension of their
transmission and containment. This requires tracking and evaluating the
evolution of pandemics through efficient monitoring and analysis of online
resources that provide rich information, reflecting public knowledge and
perceptions in a timely manner. For instance, the volumes of social media
interests can serve as early indicators of COVID-19 waves [1, 2], and users’
content can unveil diverse perspectives on regulations, such as quarantine
measures or vaccination strategies. Understanding these signals can help
policymakers to combat pandemics by recognizing trends in their spread and
impact on the population, as well as the efficacy of current countermeasures.

Traditional pandemic monitoring involves tracking hospital admissions,
laboratory testing, and death rates, but can be expensive and lag in providing
real-time disease spread updates. Compartmental models like SIR [3] and
statistical analyses such as ARIMA [4] and Prophet [5] use past data for
predictions, are common approaches. However, these statistical models rely on
assumptions and might lack data to precisely estimate factors like
reproduction number in pandemic planning and forecasting.

Figure 1: Examples showing different stances on social media reacting to the
pandemic and government regulations [6].

Time-series forecasting with deep learning is one of the most effective
methods for tracking pandemics’ evolution. Because of their data-driven
learning process, deep learning-based methods are highly accurate in analyzing
time-series data to identify patterns and trends that can help predict future
outbreaks through historical statistics. By using deep learning algorithms, we
can analyze large amounts of data quickly and accurately, making it easier to
identify patterns and trends that might be missed by other methods. Recent
works leveraged deep learning-based methods to learn statistics from earlier
time stamps as prediction to forecast the COVID-19 pandemic incidence,
achieving better performance comparing to traditional methods [7].

One of the limitations of previous works in pandemic forecasting is that they
frequently rely entirely on epidemiological data and ignore other information
that might act as sensors or indicators of the pandemic’s patterns and
evolution. Data from search engines, for example, can be used to monitor how
individuals are looking for information about pandemics [8, 9]. More
crucially, social media data can be utilized to monitor how people are
reacting to and feeling about a pandemic [10, 11]. Although previous research
has examined the connection between social media usage and pandemic trends
[12], there has been little use of deep learning techniques to predict and
track the spread of the epidemic. We can gain a more complete view of how an
epidemic is evolving and how effective various treatments might be by
including external knowledge from social media into pandemic forecasting
models. For instance, by monitoring social media data, we may pinpoint public
health campaigns to regions where people are most worried about a pandemic.
Fig. 1 illustrates our motivated examples, which show different stances of
social media users on the COVID-19 pandemic and on government regulations.

During pandemics, social media has emerged as a key source of information,
offering real-time updates on the spread of diseases and people’s responses to
them. Therefore, we investigate how pandemic tracking and analysis using deep
learning algorithms can benefit from external knowledge from multi-modality,
including social media and government regulations. More specifically, we
construct graph-structured data from social media, treating each user as a
node representing the current epidemic status. We dynamically capture
interactions between users using temporal graph learning. Graph learning,
particularly Graph Neural Networks (GNNs) [13, 14], is an important branch of
machine learning that deals with learning from and representing a variety of
real-world relational data, including citation networks, social networks,
knowledge graphs, etc. Incorporating graph learning techniques and graph-
structured representations offers a promising approach to overcome limitations
of previous works in pandemic forecasting, where graphs can capture the
structural and semantic information of the pandemic domain [15, 16, 17].

In this work, we introduce MGL4MEP, a neural framework for forecasting and
analyzing developing pandemics using big data sources and deep learning
methods, including graph neural networks. We utilize the extremely recent
COVID-19 pandemic and their effects on multiple areas as a case study. In
order to trace and predict the evolution of the pandemic, we investigate the
relationship between the pandemic risk factors and all other relevant data
sources such as social media. Our framework will support many end users like
politicians, policy makers, and general population for reference by providing
complementary analysis and forecast information, leading to more effective
crisis preventive and reaction times. Our contributions in this work are
summarized as follows:

  * •

We propose a multi-modal neural framework named MGL4MEP for COVID-19 pandemic
tracking and prediction,

  * •

We extract and combine data from multiple sources, including social signals
and government stringency signals as additional indicators to monitor pandemic
trends and predict future evolution,

  * •

We investigate the correlation and impacts of these multi-modal data on
pandemic forecasting using deep learning and graph learning methods,

  * •

We conduct extensive experiments on multiple areas affected by the pandemics
to show the usefulness and effectiveness of our proposed framework.

Source codes of our framework and reproducible baselines are made publicly
available at https://github.com/KhanhTungTran/MGL4MEP for future research and
benchmarking purposes.

## Results

### Baselines

We evaluate our proposed approach against several baselines that employ
different techniques, including statistical, machine learning, and deep
learning approaches.

  * •

Numerical analysis: (i) AVG: average of the whole history are used to predict
the future; (ii) AVG_WINDOW: average statistics of current prediction window
are utilized to predict the future; and (iii) LAST DAY: the statistics of the
current day are used as prediction.

  * •

Machine learning-based models: (i) LIN_REG: Ordinary least squares Linear
Regression fits a line to training samples for predicting future cases; (ii)
GP_REG: Gaussian Process Regressor is a non-parametric regression model
utilizes Gaussian processes; (iii) RAND_FOREST and (iv) XGBOOST: tree-based
models.

  * •

Statistical models: (i) ARIMA [4], a simple autoregressive moving average
model leverage the entire history sequence as input; and (ii) PROPHET [5],
similar to ARIMA but with strong seasonality characteristics.

  * •

Deep learning models without graph topology: (i) A straightforward LSTM model,
uses the sequence of the most recent $d$ days as its input, (ii) SEtransformer
and (iii) SREtransformer, baseline models using the popular transformer
architecture for learning on extracted text embeddings from social media data.
Self-attention is calculated between tokens of different users (extracted
using the same pre-trained language model as MGL4MEP models). Then, the final
embeddings are fused with LSTM for processing the time-series and making the
final predictions.

  * •

MGL4MEPSR, MGL4MEPSE, MGL4MEPSRE: our proposed models (i) MGL4MEPSR are
baseline LSTM with an additional input features of regulations (R)
information, (ii) MGL4MEPSE is the model with temporal graph learning on pre-
processed social media data (E - entity) from 1500 users for the default
setting, and (iii) MGL4MEPSRE is our final model with input from three
different modality, including statistics as in traditional model, and
regulations and social media data.

### Implementation details

The proposed framework was implemented in PyTorch [18], and experiments were
carried out on an NVIDIA 3090Ti GPU. We train the model for a maximum of 300
epochs with early stopping. All models are optimized with AdamW optimizer
[19], $10^{-3}$ initial learning rate, a batch size of 16, and input sequence
length of 7. These hyperparameters are set empirically through grid search.
All experiments are repeated 5 times with different seeds. The last 20% time
steps of the dataset are used as hold-out test set. Details regarding the data
collected and used are described in following sections. Mean Absolute Error
(MAE), Root Mean Squared Error (RMSE), Mean Absolute Percentage Error (MAPE),
and R squared ($R^{2}$) are metrics used to evaluate and compare between
models.

$MAE=\frac{\sum_{i=1}^{n}|y_{i}-\hat{y}_{i}|}{n}$ (1)

where $y_{i}$ and $\hat{y}_{i}$ denote the $i$th statistics from ground truth
data and predicted values of the models, and $n$ is the total number of
samples in the test set. The MAE metric indicates the average variance between
the predicted values and the ground truth in the dataset (lower is better).

$RMSE=\sqrt{\frac{\sum_{i=1}^{n}(y_{i}-\hat{y}_{i})^{2}}{n}}$ (2)

The RMSE metric in Equation 2 is the standard deviation of the residuals
(prediction error) (lower is better).

$MAPE=\frac{1}{n}\sum_{i=1}^{n}\frac{|y_{i}-\hat{y}_{i}|}{y_{i}}\times 100$
(3)

The MAPE given in Equation 3 tells us about the mean of the total percentage
errors (lower is better).

$R^{2}=1-\frac{RSS}{TSS}$ (4)

Finally, the Coefficient of Determination (R-squared metric) provides an
insight into the similarity between real and predicted data, where the closer
to 1 the R squared value is, the better. Here,
$RSS=\sum_{i=1}^{n}(y_{i}-\hat{y}_{i})^{2}$ and
$TSS=\sum_{i=1}^{n}(y_{i}-\overline{y})^{2}$ denote the Residual Sum of
Squares and the Total Sum of Squares, respectively.

Table 1: Results on California State for short-term predictions.

Number of days ahead | 1 | 3 | 7
---|---|---|---
| MAE$\downarrow$ | RMSE$\downarrow$ | MAPE$\downarrow$ | R${}^{2}\uparrow$ | MAE$\downarrow$ | RMSE$\downarrow$ | MAPE$\downarrow$ | R${}^{2}\uparrow$ | MAE$\downarrow$ | RMSE$\downarrow$ | MAPE$\downarrow$ | R${}^{2}\uparrow$
AVG | 1735.92 | 2543.38 | 41.75 | -0.0729 | 1739.06 | 2552.49 | 42.13 | -0.0899 | 1726.68 | 2547.36 | 42.76 | -0.1460
LAST_DAY | 200.24 | 450.19 | 3.23 | 0.9820 | 381.77 | 730.72 | 6.44 | 0.9451 | 638.60 | 1231.95 | 12.18 | 0.8110
AVG_WINDOW | 396.87 | 753.38 | 7.26 | 0.9352 | 527.41 | 990.43 | 10.08 | 0.8787 | 767.76 | 1337.56 | 15.76 | 0.7324
LIN_REG | 239.45 | 511.21 | 3.59 | 0.9761 | 465.00 | 853.38 | 6.61 | 0.9264 | 848.42 | 1649.43 | 13.13 | 0.6523
GP_REG | 236.90 | 509.94 | 3.55 | 0.9762 | 457.30 | 843.30 | 6.50 | 0.9281 | 812.09 | 1585.71 | 12.53 | 0.6769
RAND_FOREST | 224.04 | 475.28 | 3.46 | 0.9785 | 504.18 | 906.01 | 7.44 | 0.9135 | 1251.66 | 2639.35 | 18.71 | 0.1089
XGBOOST | 265.20 | 581.56 | 4.46 | 0.9657 | 633.48 | 1166.90 | 10.35 | 0.8193 | 1511.12 | 3244.66 | 22.97 | -0.3955
ARIMA | 1536.31 | 2286.63 | 57.07 | -1.1243 | 1653.35 | 2431.21 | 60.55 | -1.3625 | 1874.52 | 2718.03 | 68.02 | -2.0135
PROPHET | 1919.07 | 3661.94 | 44.74 | -2.0139 | 1868.22 | 3618.93 | 44.44 | -1.9664 | 1732.04 | 3443.21 | 43.50 | -1.9019
LSTM | 207.62 | 403.71 | 4.41 | 0.9780 | 349.35 | 663.60 | 6.46 | 0.9395 | 831.11 | 1468.21 | 13.76 | 0.6564
SEtransformer | 1026.56 | 1700.72 | 20.89 | 0.4242 | 975.09 | 1634.44 | 19.59 | 0.5352 | 1258.66 | 2136.51 | 23.93 | 0.1751
SREtransformer | 1437.27 | 2326.74 | 26.91 | 0.075 | 1193.21 | 1996.99 | 22.67 | 0.1598 | 1101.02 | 1845.40 | 21.61 | 0.3398
MGL4MEPSR (ours) | 265.77 | 470.65 | 5.16 | 0.9725 | 460.13 | 783.48 | 8.13 | 0.9217 | 825.72 | 1283.63 | 14.40 | 0.7568
MGL4MEPSE (ours) | 510.36 | 854.95 | 9.73 | 0.9039 | 651.32 | 1081.28 | 12.59 | 0.8242 | 577.07 | 1005.08 | 11.07 | 0.8160
MGL4MEPSRE (ours) | 479.11 | 788.15 | 9.41 | 0.9022 | 534.51 | 913.40 | 10.69 | 0.8755 | 636.86 | 1086.27 | 12.08 | 0.7960

### Results

Table 2: Results on New York State for short-term predictions.

Number of days ahead | 1 | 3 | 7
---|---|---|---
| MAE$\downarrow$ | RMSE$\downarrow$ | MAPE$\downarrow$ | R${}^{2}\uparrow$ | MAE$\downarrow$ | RMSE$\downarrow$ | MAPE$\downarrow$ | R${}^{2}\uparrow$ | MAE$\downarrow$ | RMSE$\downarrow$ | MAPE$\downarrow$ | R${}^{2}\uparrow$
AVG | 549.85 | 842.34 | 32.40 | -3.8537 | 557.52 | 850.98 | 32.42 | -3.7875 | 566.90 | 861.35 | 33.01 | -3.9149
LAST_DAY | 714.36 | 1243.29 | 21.06 | -0.5183 | 694.63 | 1146.65 | 22.49 | -0.6301 | 514.94 | 917.25 | 15.75 | 0.1593
AVG_WINDOW | 493.52 | 824.75 | 15.68 | 0.2165 | 505.44 | 839.23 | 16.18 | 0.1809 | 538.03 | 881.71 | 17.01 | 0.1386
LIN_REG | 521.59 | 897.58 | 15.95 | 0.2221 | 485.24 | 806.84 | 15.54 | 0.2991 | 502.77 | 855.60 | 16.31 | 0.1978
GP_REG | 523.64 | 899.53 | 15.86 | 0.2199 | 486.08 | 807.60 | 15.40 | 0.3038 | 504.14 | 855.73 | 16.22 | 0.2073
RAND_FOREST | 1057.99 | 1996.79 | 29.76 | -2.0817 | 1134.78 | 2433.44 | 32.10 | -3.4917 | 607.52 | 1074.55 | 21.19 | -0.2863
XGBOOST | 1307.55 | 3071.69 | 35.31 | -5.9887 | 1445.72 | 3411.18 | 40.63 | -7.7175 | 931.61 | 1587.23 | 31.59 | -1.7561
ARIMA | 534.56 | 904.68 | 17.02 | -0.1033 | 543.39 | 912.94 | 17.13 | -0.0939 | 552.26 | 924.48 | 17.40 | -0.1036
PROPHET | 1565.38 | 2269.08 | 64.46 | -13.9960 | 1578.53 | 2339.01 | 63.30 | -14.6835 | 1582.25 | 2305.86 | 64.34 | -13.9820
LSTM | 186.66 | 301.33 | 7.47 | 0.2098 | 259.98 | 417.70 | 9.43 | -0.4364 | 366.56 | 593.31 | 12.51 | -0.8850
SEtransformer | 324.64 | 497.77 | 10.84 | -1.312 | 446.15 | 685.79 | 15.72 | -2.8800 | 377.39 | 573.80 | 13.36 | -1.4352
SREtransformer | 269.19 | 419.66 | 10.65 | -0.8072 | 339.15 | 543.18 | 11.72 | -1.3120 | 353.17 | 556.10 | 11.47 | -1.4280
MGL4MEPSR (ours) | 203.35 | 320.73 | 7.77 | -0.0129 | 386.10 | 575.82 | 11.22 | -0.9012 | 462.88 | 696.32 | 13.02 | -1.6530
MGL4MEPSE (ours) | 160.63 | 278.49 | 5.97 | 0.2927 | 242.02 | 409.45 | 8.46 | -0.1808 | 336.77 | 549.74 | 11.40 | -0.7996
MGL4MEPSRE (ours) | 179.52 | 285.52 | 7.70 | 0.1352 | 294.04 | 461.59 | 9.91 | -0.5289 | 565.34 | 836.36 | 14.77 | -2.5251

The evaluation results for short-term predictions are presented in Table 1,
where it can be observed that for horizon values of 1 or 3, our proposed
approaches underperform the baseline LSTM neural network method, which is the
closest competitor to our proposed models. However, the performance of our
models significantly improve on the New York state dataset, as shown in Table
2. The reason for this improvement may lie in the fact that the $R^{2}$ score,
which is calculated on the predictions for each state, being close to
perfection ($0.9820$ for the horizon of 1) on the California dataset, but the
$R^{2}$ score on New York dataset is significantly lower, indicating a higher
level of difficulty for learning and prediction on the latter dataset. In
general, the MGL4MEPSE method achieves the most impressive results for short-
term forecasting, followed by the MGL4MEPSRE model. This can be explained as
the time lag between government efforts and their impact on the real-world
situation spans multiple weeks. Incorporating this information for short-term
prediction may adversely affect the effectiveness of the model. Furthermore,
upon comparison with SEtransformer and SREtransformer, which utilize the
transformer architecture without correlation matrices for processing social
media data, we observe that our methodology incorporating graph neural
networks featuring spatial-temporal characteristics distinctively surpass
these approaches. This outcome highlights the efficacy and suitability of our
novel proposed approaches, both in constructing input graph structures and in
learning algorithms, for addressing these types of multi-modality domains.

Table 3: Results on California State for long-term predictions.

Number of days ahead | 14 | 21 | 28
---|---|---|---
| MAE$\downarrow$ | RMSE$\downarrow$ | MAPE$\downarrow$ | R${}^{2}\uparrow$ | MAE$\downarrow$ | RMSE$\downarrow$ | MAPE$\downarrow$ | R${}^{2}\uparrow$ | MAE$\downarrow$ | RMSE$\downarrow$ | MAPE$\downarrow$ | R${}^{2}\uparrow$
AVG | 1625.33 | 2397.80 | 43.58 | -0.4115 | 1550.94 | 2299.96 | 44.92 | -1.2001 | 1620.44 | 2367.44 | 47.82 | -3.4092
LAST_DAY | 1016.07 | 1679.24 | 22.26 | 0.3751 | 1454.70 | 2418.62 | 34.59 | -1.2151 | 1854.50 | 3011.52 | 47.58 | -5.9595
AVG_WINDOW | 1190.22 | 1943.52 | 26.61 | 0.1444 | 1582.72 | 2578.15 | 38.55 | -1.6106 | 1988.98 | 3196.71 | 52.41 | -6.9129
LIN_REG | 1073.60 | 1882.31 | 20.53 | 0.0937 | 1393.40 | 2469.55 | 28.95 | -2.1887 | 1668.01 | 2917.30 | 36.16 | -7.5977
GP_REG | 1004.46 | 1740.18 | 19.67 | 0.1889 | 1357.33 | 2433.31 | 28.26 | -2.1518 | 1629.51 | 2837.48 | 35.48 | -7.5078
RAND_FOREST | 2800.75 | 6404.53 | 44.94 | -8.4948 | 5310.35 | 11285.29 | 96.23 | -61.9137 | 6253.21 | 11610.95 | 123.79 | -119.0820
XGBOOST | 2905.03 | 6253.72 | 51.53 | -9.3011 | 6129.52 | 12290.41 | 112.19 | -70.4298 | 6426.62 | 11945.64 | 123.59 | -119.2901
ARIMA | 2249.92 | 3203.26 | 81.65 | -4.2380 | 2660.01 | 3697.61 | 96.55 | -9.7618 | 3057.48 | 4167.76 | 111.46 | -23.8195
PROPHET | 1417.82 | 2917.99 | 41.58 | -2.0017 | 1081.00 | 2227.87 | 39.39 | -2.4306 | 848.07 | 1726.27 | 37.21 | -3.4281
LSTM | 962.75 | 1464.46 | 21.91 | 0.3687 | 855.98 | 1324.40 | 20.05 | -0.2023 | 981.55 | 1557.89 | 22.00 | -1.2480
SEtransformer | 1250.80 | 2248.76 | 23.58 | -0.2571 | 950.71 | 1792.47 | 20.24 | -0.3664 | 879.30 | 1532.81 | 18.36 | -1.0662
SREtransformer | 1044.82 | 1938.24 | 20.73 | 0.0852 | 1136.17 | 1988.57 | 21.95 | -0.5690 | 658.05 | 1199.98 | 16.37 | -0.3351
MGL4MEPSR (ours) | 771.67 | 1236.55 | 17.04 | 0.5435 | 836.22 | 1285.58 | 17.24 | 0.1384 | 866.02 | 1440.28 | 18.65 | -0.4867
MGL4MEPSE (ours) | 583.17 | 1035.97 | 12.10 | 0.6798 | 563.19 | 932.89 | 13.08 | 0.4467 | 588.17 | 1036.76 | 13.55 | 0.1767
MGL4MEPSRE (ours) | 555.80 | 981.07 | 12.15 | 0.6866 | 568.03 | 948.07 | 12.85 | 0.4457 | 634.88 | 1149.32 | 14.52 | -0.1925

With respect to long-term prediction, our models achieve the best results
across all three horizons with significant gaps compared to all other methods,
as illustrated in Table 3 and 4. Generally, the best performed approaches are
MGL4MEPSRE and MGL4MEPSE models, achieving 42.47%, 34.21%, and 10.62% lower
MAE with horizon equals 14, 21, 28 days ahead than the best basline methods
for California dataset, and 11.94%, and 7.50% lower MAE for horizon 14 and 21,
on New York dataset. Moreover, for long-term prediction, MGL4MEPSR models are
able to obtain better results than the simple baseline LSTM models, compared
to the results and analysis on short-term predictions. The model’s capability
on forecasting the long-term trajectory of the pandemic means that it can
provide valuable information and insights for government and policy makers on
planning ahead and making informed, timely response to the pandemic.

Table 4: Results on New York State for long-term predictions.

Number of days ahead | 14 | 21 | 28
---|---|---|---
| MAE$\downarrow$ | RMSE$\downarrow$ | MAPE$\downarrow$ | R${}^{2}\uparrow$ | MAE$\downarrow$ | RMSE$\downarrow$ | MAPE$\downarrow$ | R${}^{2}\uparrow$ | MAE$\downarrow$ | RMSE$\downarrow$ | MAPE$\downarrow$ | R${}^{2}\uparrow$
AVG | 576.57 | 878.21 | 36.03 | -7.4231 | 577.12 | 878.64 | 37.71 | -9.4983 | 555.38 | 854.57 | 38.84 | -9.9887
LAST_DAY | 608.37 | 1096.63 | 19.75 | -0.5469 | 568.43 | 1041.60 | 21.58 | -0.9110 | 649.36 | 1128.66 | 25.78 | -1.2729
AVG_WINDOW | 577.85 | 950.80 | 19.97 | -0.3342 | 561.28 | 918.91 | 21.37 | -0.6601 | 576.23 | 941.92 | 23.50 | -0.9414
LIN_REG | 550.95 | 901.68 | 20.44 | -0.5076 | 542.15 | 917.15 | 23.91 | -1.5887 | 577.39 | 984.80 | 29.15 | -3.5869
GP_REG | 550.54 | 900.15 | 20.24 | -0.4716 | 539.68 | 913.79 | 23.59 | -1.5046 | 575.04 | 979.01 | 28.76 | -3.4214
RAND_FOREST | 962.95 | 1649.01 | 34.08 | -3.4251 | 1267.22 | 2077.25 | 42.03 | -5.2131 | 1339.23 | 2165.87 | 51.17 | -10.8534
XGBOOST | 1242.85 | 2078.66 | 49.98 | -10.5987 | 1753.07 | 2790.31 | 65.61 | -15.9673 | 1688.39 | 2611.17 | 87.04 | -63.6253
ARIMA | 554.70 | 927.95 | 22.08 | -0.8887 | 546.87 | 918.12 | 23.62 | -1.3794 | 529.60 | 902.33 | 25.27 | -1.6065
PROPHET | 1579.78 | 2307.45 | 64.20 | -13.8898 | 1575.09 | 2298.21 | 63.94 | -15.1467 | 1584.90 | 2306.69 | 63.77 | -15.9274
LSTM | 270.61 | 434.96 | 11.08 | -0.6286 | 288.95 | 452.27 | 10.12 | -0.4052 | 280.53 | 443.59 | 11.53 | -1.022
SEtransformer | 393.33 | 596.95 | 12.67 | -1.2141 | 318.92 | 506.33 | 11.41 | -0.8533 | 324.25 | 511.64 | 11.89 | -1.1520
SREtransformer | 393.19 | 643.84 | 12.71 | -1.6740 | 366.67 | 560.50 | 13.42 | -1.5570 | 368.12 | 564.89 | 11.52 | -0.7978
MGL4MEPSR (ours) | 267.15 | 430.25 | 10.42 | -0.5040 | 254.03 | 411.54 | 9.74 | -0.3239 | 315.59 | 504.57 | 10.86 | -0.7395
MGL4MEPSE (ours) | 238.29 | 382.46 | 9.47 | -0.2762 | 267.29 | 431.71 | 10.04 | -0.3655 | 311.72 | 503.53 | 11.29 | -0.8625
MGL4MEPSRE (ours) | 278.71 | 460.88 | 10.44 | -0.3482 | 287.07 | 479.69 | 9.73 | -0.3946 | 324.18 | 520.57 | 10.56 | -0.6624

We conduct comprehensive ablation studies to investigate the impact of the
size of the input social media graph on our COVID-19 prediction model. In
particular, we reduce the number of users selected for building the graph from
the original 1,500 users to 1,000 and 500, respectively. Our hypothesis is
that as we decrease the number of users, the amount of information provided by
the social media data would be significantly reduced. The results in Table 5
for the California region confirm our hypothesis, showing a degradation in
performance with a decrease in the number of nodes for both short-term and
long-term forecasting. The findings have implications for future research in
that it is critical to take the size of the input social media network into
account when developing a model for predicting COVID-19 instances. The size of
the social media graph directly influences the richness and diversity of the
data captured, allowing the model to capture a more nuanced understanding of
public sentiments, behaviors, and trends related to the pandemic.

Table 5: Ablation study on sufficient amount of social media users for
modeling society interaction and factor. We train MGL4MEPSE with different
number of nodes (users) for social network data and evaluate each model’s
performances on California data.

Number of days ahead | 7 | 14
---|---|---
| MAE$\downarrow$ | RMSE$\downarrow$ | MAPE$\downarrow$ | R${}^{2}\uparrow$ | MAE$\downarrow$ | RMSE$\downarrow$ | MAPE$\downarrow$ | R${}^{2}\uparrow$
LSTM - without social network info. | 831.11 | 1468.21 | 13.76 | 0.6564 | 962.75 | 1464.46 | 21.91 | 0.3687
MGL4MEPSE \- 500 nodes | 687.43 | 1139.88 | 13.74 | 0.7783 | 630.78 | 1049.50 | 13.75 | 0.6631
MGL4MEPSE \- 1000 nodes | 626.24 | 1047.17 | 12.74 | 0.7918 | 595.77 | 1051.18 | 12.74 | 0.6636
MGL4MEPSE \- 1500 nodes | 577.07 | 1005.08 | 11.07 | 0.8160 | 583.17 | 1035.97 | 12.10 | 0.6798

Table 6 illustrates the ablation study on different number of nodes for the
input social media graph of our COVID-19 prediction model for New York
dataset. The results are consistent with experiment on California dataset,
where a degradation in performance with a decrease in the number of nodes for
both short-term and long-term forecasting can be seen.

Table 6: Ablation study on sufficient amount of social media users for
modeling society interaction and factor. We train our MGL4MEPSE model with
different number of nodes (users) for social network graph and evaluate each
model’s performances on New York data.

Number of days ahead | 7 | 14
---|---|---
MAE$\downarrow$ | RMSE$\downarrow$ | MAPE$\downarrow$ | R${}^{2}\uparrow$ | MAE$\downarrow$ | RMSE$\downarrow$ | MAPE$\downarrow$ | R${}^{2}\uparrow$
LSTM - without social network info. | 366.56 | 593.31 | 12.51 | -0.8850 | 270.61 | 434.96 | 11.08 | -0.6286
MGL4MEPSE \- 500 nodes | 371.54 | 596.19 | 12.40 | -1.3730 | 240.97 | 395.27 | 9.49 | -0.3487
MGL4MEPSE \- 1000 nodes | 346.40 | 570.16 | 12.65 | -1.5580 | 249.31 | 405.01 | 9.68 | -0.3383
MGL4MEPSE \- 1500 nodes | 336.77 | 549.74 | 11.40 | -0.7996 | 238.29 | 382.46 | 9.47 | -0.2762

In order to assess the usefulness and effectiveness of our proposed methods in
different stages of the pandemic, we perform an additional experiment by
collecting data for another 150 days, thereby increasing the total amount of
data collected. We then train and evaluate the same models and baselines on
this new dataset, the results of which are presented in Table 7 and visualized
in Fig. 2. It is worth noting that this new test set for California state
exhibits a higher variance compared to the previous forecasting range, where
statistics gradually decrease with sharp changes. Nevertheless, our models
outperform the baselines and achieve the best performance on both test sets.
These results indicate the robustness and generalizability of our approaches
in combating the COVID-19 pandemic in different stages of its evolution.

Table 7: Results on California State when the data collection period went up
to May, 2022.

Number of days ahead | 7 | 14
---|---|---
| MAE$\downarrow$ | RMSE$\downarrow$ | MAPE$\downarrow$ | R${}^{2}\uparrow$ | MAE$\downarrow$ | RMSE$\downarrow$ | MAPE$\downarrow$ | R${}^{2}\uparrow$
AVG | 5409.71 | 8863.60 | 194.54 | -0.2504 | 4563.77 | 6692.37 | 206.27 | -1.0961
LAST_DAY | 3357.62 | 9021.44 | 34.76 | 0.3118 | 5453.62 | 13683.49 | 76.84 | -3.5250
AVG_WINDOW | 5179.47 | 13189.69 | 52.49 | -0.4659 | 7428.87 | 18219.47 | 104.02 | -6.8496
LIN_REG | 3930.85 | 9618.96 | 20.27 | 0.9333 | 5852.20 | 14373.83 | 38.48 | 0.7726
GP_REG | 3942.71 | 9627.01 | 20.22 | 0.9331 | 5858.40 | 14406.63 | 38.23 | 0.7701
RAND_FOREST | 8364.92 | 22422.34 | 28.10 | 0.6728 | 7599.95 | 19360.56 | 45.53 | 0.6717
XGBOOST | 8051.32 | 21248.67 | 27.10 | 0.7062 | 7646.05 | 18724.51 | 48.16 | 0.6739
ARIMA | 9568.85 | 17111.98 | 146.64 | -1.1930 | 11804.89 | 21882.42 | 203.03 | -9.3336
PROPHET | 35891.74 | 52742.08 | 1009.45 | -20.6104 | 36453.48 | 52415.34 | 1051.09 | -57.9110
LSTM | 1384.07 | 2801.17 | 29.70 | 0.7748 | 932.91 | 1691.75 | 32.58 | 0.5906
SEtransformer | 1549.09 | 3709.52 | 56.29 | 0.4954 | 1096.73 | 2052.00 | 45.93 | 0.2553
SREtransformer | 1844.22 | 5042.73 | 50.15 | 0.1378 | 1285.99 | 2359.96 | 50.56 | 0.0341
MGL4MEPSR (ours) | 1099.96 | 2303.54 | 28.36 | 0.7907 | 1124.82 | 2744.64 | 38.89 | -0.0915
MGL4MEPSE (ours) | 1057.29 | 2085.78 | 30.51 | 0.7972 | 734.44 | 1250.40 | 37.20 | 0.6592
MGL4MEPSRE (ours) | 901.99 | 1881.81 | 28.95 | 0.7818 | 828.84 | 1375.84 | 37.45 | 0.5682

## Discussion

Forecasting results. In the real world, COVID-19 is a complex pandemic, and
many factors that cannot be seen by previous statistics can lead to different
future scenarios. For example, if a government implements a strict lockdown,
the number of new cases will likely decrease. However, if a government lifts
all restrictions, the number of new cases will likely increase. This is why it
is important to consider multiple information sources when forecasting the
spread of COVID-19. Our proposed method, MGL4MEP and its variants incorporate
multiple information sources effectively, leading to better performances,
lower errors, and sustained accuracy, particularly in long-term predictions,
compared to other popular forecasting models. MGL4MEP enjoys these benefits
due to it being able to learn the dynamic relationships between various
factors that affect the spread of COVID-19, such as official government
policy, and social stances against the pandemic or situation. Moreover, our
ablation results clearly demonstrate that the availability of more information
significantly enhances the reliability of our forecasting models. Comparisons
between our methods and baselines that do not utilize the graph structure of
social media data also highlight the efficacy of our graph-structure
generation process and temporal graph learning framework. The results not only
underline the effectiveness in learning and extracting information of the
models, but also emphasize on the usefulness of input multimodal data.

Additionally, our experiment results show that MGL4MEP is adaptable and robust
to different situations and history. This means that it can be used to
forecast the spread of COVID-19 in different countries and regions, even if
the pandemic is evolving rapidly. The predictions made by MGL4MEP can be
leverage by different beneficial group such as the authorities to develop
appropriate strategies in order to deal with the spread of this pandemic. An
example is that MGL4MEP can be used to forecast the impact of different
government policies on the spread of the virus.

Finally, it’s important to highlight the automated nature of the forecasting
process with MGL4MEP. The entire process can be automated and seamlessly
updated whenever new information becomes available. This automation is
possible due to our framework’s reliance on openly accessible data from the
Internet, which can be efficiently gathered through automated web crawling.

(a)  (b)

Figure 2: Forecasting results on test set for the newly collected period of
California state dataset (a) Infectious cases. (b) Hospitalized cases.

Model limitations. Although some of the proposed models performed well
according to certain metrics, we found several shortcomings in the models that
we tested. One limitation is that MGL4MEP takes time to realize the trend
depending on the dynamic at the considered area, as information from different
sources, such as the effects of government policies takes time to be reflected
in the data. For example, in the case of New York state, it takes effect
immediately while in the case of California, it takes about 14 days.
Additionally, the underlying factors that affect the infection of COVID-19 are
diverse, and it can be difficult to capture all of them through the multiple
data sources used by MGL4MEP. Another limitation of MGL4MEP is that similar to
other deep learning methods, it is a black-box model. This means that we
cannot easily understand how it makes its predictions, and can make it
difficult to trust or explain the model’s predictions.

Future research directions. There are several ways in which MGL4MEP can be
improved in the future. An interesting future research direction is to enrich
our framework with more information regarding the pandemic situation, such as
regional age population, mobility, or virus variants. Another direction can be
interested in explainability methods, such as identifying important nodes or
features through temporal graph learning, or understanding the most valuable
factors that affect the forecasting results. This would make us more confident
in the predictions of the model and would help us to better understand the
dynamics of the pandemic.

## Related Work

### Pandemic forecasting

Traditional approaches leverage statistical models have been widely used to
forecast COVID-19. These approaches involve analyzing past epidemic data using
statistical and time-series methodologies to identify patterns and trends,
which can then be used to forecast future outbreaks. Methods like
autoregressive integrated moving average (ARIMA) [4] and Prophet [5] are
effective at identifying trends in stationary time-series data and handling
periodic patterns, respectively. One alternative to forecast pandemics, such
as COVID-19 pandemic, is through the use of compartmental models [20, 21, 22].
These models divide a population into compartments, such as susceptible,
exposed, and infected individuals (SIR model), and use mathematical equations
to describe the dynamic and transitions between each group. However, it is
important to acknowledge that leveraging statistical models for pandemic
forecasting has its limitations, as they presumptively assume a linear
relationship between past and future time-series. Such methods rely on certain
assumptions and may lack the data necessary to accurately address all the
relevant issues.

Deep learning have been applied to make predictions about the spread of
COVID-19 pandemic and achieve high performances [23, 7]. With the enormous
dataset of records such as infected and hospitalized cases collected on a
daily basis, deep learning is considered a suitable approach, as neural
networks can learn and update from data effectively. Sequential models such as
Recurrent Neural Network (RNN) and Long Short Term Memory (LSTM) [24] have
been applied and seen high performances for forecasting COVID-19 pandemic,
both at world-wide or country level [25, 26], and more fine-grained levels
including state and county levels [27, 28, 29, 30]. Unlike conventional
approaches, deep learning can incorporate external knowledge and adapt to
changing circumstances, improving their predictive capabilities. These
approaches, or fusion of multiple neural network models, can incorporate a
wide range of data sources, including social media, to provide a more
comprehensive view of the pandemic and its potential impacts.

To our best knowledge, previous research has only attempted to integrate basic
indicators or indices, overlooking the dynamic, intricate information
contained in user-generated content [8, 31, 12]. Incorporating these valuable
signals into neural networks remains a challenge but has the potential to
provide a more comprehensive view of the pandemic and its potential impacts
[9, 32, 1].

### Leveraging external resources for time-series forecasting

Previous studies have explored the connection between social media interests
and pandemic trends. In [9], the authors highlight a strong correlation
between peak of search volume on COVID-19 pandemic and the development of the
pandemic, upto 20 days earlier than the issuance of official warnings. The
authors of [1] also discovered a close connection between the evolution of the
COVID-19 crisis and social media user’s sentiments toward different phases of
the pandemic. Another work [33] makes use of the social impact of media
coverage to support the compartment model for pandemic prediction. Post-
processed indicators such as internal movement index and economic response
have been incorporated as additional input features to sequence models for
forecasting future statistics [34, 35]. Differing from them, our method
considers every aspect of user response through social media and government
regulations against the pandemic. To achieve greater accuracy in pandemic
forecasting, we analyze individual tweets and search for relevant social
events.

There has been a significant amount of interest in effectively leveraging
social media as an external knowledge source for more accurate pandemic
forecasting. In [31], the authors used tweet count (the amount of tweets
related to COVID-19) per day as an additional input to an LSTM model and
achieve better results than using statistics only. Taking a step further, in
[12], the collected tweets are then further extracted into two main features,
representing user sentiment and topic of interest. These features are used as
additional input features to an ARIMAX model, which is an extension of ARIMA.
Furthermore, in [36], important keywords are extracted and curated into a
keyword cloud to present the most important information for each day and input
to a MLP module for pandemic prediction. Perhaps the most relevant works to
this paper are [16, 17] where the authors extract the most popular keywords
per day and view them as a graph structure and employ graph algorithms to
learn on those representations.

In this study, in contrast to prior works, we incorporate data from multiple
different sources, with social media as an important knowledge source where we
build graph structure with each user as node, or an indicator on the current
status of the epidemic, and dynamically represent the interaction between them
through temporal graph neural networks [13, 37, 38]. Our approach
comprehensively considers various aspects of user responses on social media
and government regulations pertaining to the pandemic.

### Temporal graph neural networks forecasting models

Graph neural networks (GNNs) have gained significant attention in various
learning tasks, such as image recognition [39, 40], estimating quantum
chemical computation [41, 42, 43], predicting protein interfaces [44], etc.
GNNs generalize the concept of convolution neural networks to non-Euclidean
domains, allowing for local operations on the nodes and edges of a graph [13,
45]. The most popular GNNs is Message Passing Neural Networks (MPNNs) [42] in
which the graph convolution is defined via the message passing scheme that
propagates and then aggregates the vectorized information between each node
and its local neighborhood.

To handle evolving features and connectivity over time, temporal graph neural
networks have been introduced. Unlike static graphs, temporal graphs are
usually represented by a sequence of node interactions over continuous time
instead of an adjacency matrix. Temporal GNNs aim to capture both the temporal
and structural information of the temporal graphs by introducing a node memory
that represents the state of the node at a given time, acting as a compressed
representation of the node’s past interactions. Temporal GNNs combine graph
encoding techniques with time-series encoding architectures such as LSTM and
Transformers, forming a powerful deep learning framework. They find
applications in various domains, such as traffic prediction, where they
outperform traditional methods by incorporating spatial relationships of road
networks and temporal dynamics of traffic conditions [46, 47, 48]. In the
analysis of brain networks, temporal GNNs utilize invasive techniques like
electrocorticography (ECoG) to uncover temporal patterns and gain insights
into brain network dynamics [48].

In our apporach, by leveraging the temporal and structural aspects of graph
representation in social media data, temporal GNNs enhance modeling
capabilities for understanding evolving complex systems and forecasting
pandemic statistics.

## Preliminaries

### Time-series forecasting

Originally proposed in [24], Long short-term memory (LSTM) has been the
dominant recurrent network architecture for learning from sequences of data.
Unlike standard feedforward neural networks, LSTM can process and retain the
temporal correlations between adjacent time steps, due to its feedback
connections. For a historical time step $t$, the output $y_{t}$ will not only
depend on $x_{t}$ but also from previous iterations through hidden state
$h_{t-1}$ and memory variable $c_{t-1}$:

$\displaystyle\Gamma_{u}=\sigma(\bm{W}_{hu}h_{t-1}+\bm{W}_{xu}x_{t}+b_{u})$
(5a)
$\displaystyle\Gamma_{f}=\sigma(\bm{W}_{hf}h_{t-1}+\bm{W}_{xf}x_{t}+b_{f})$
(5b)
$\displaystyle\tilde{c}_{t}=\tanh(\bm{W}_{hc}h_{t-1}+\bm{W}_{xc}x_{t}+b_{c})$
(5c) $\displaystyle c_{t}=\Gamma_{u}\odot\tilde{c}_{t}+\Gamma_{f}\odot
c_{t-1}$ (5d)
$\displaystyle\Gamma_{o}=\sigma(\bm{W}_{ho}h_{t-1}+\bm{W}_{xo}x_{t}+b_{o})$
(5e) $\displaystyle h_{t}=\Gamma_{o}\odot\tanh(c_{t})$ (5f)

where $\Gamma_{u}$ and $\Gamma_{f}$ are “update gate” and “output gate”,
calculated through a sigmoid ($\sigma$) activation function to determine the
percentage of new memory $\tilde{c}_{t}$ to keep and the percentage of old
memory $c_{t-1}$ to forget, respectively. The “output gate” $\Gamma_{o}$
allows information to be revealed appropriately due to the sigmoid function
then the weights are updated by the element-wise multiplication of
$\Gamma_{o}$ and memory cell $c_{t}$ activated by the non-linear $\tanh$
function.

A simpler, more intuitive version of LSTM called Gated-Recurrent Unit (GRU)
[49], combined the cell memory and the hidden state variable into $h_{t}$ to
transfer information. Therefore, a GRU only has two gates, a “reset gate” and
an “update gate”.

$\displaystyle\Gamma_{u}=\sigma(\bm{W}_{hu}h_{t-1}+\bm{W}_{xu}x_{t}+b_{u})$
(6a) $\displaystyle\Gamma_{f}=1-\Gamma_{u}$ (6b)
$\displaystyle\tilde{h}_{t}=\tanh(\bm{W}_{hh}h_{t-1}+\bm{W}_{xh}x_{t}+b_{h})$
(6c) $\displaystyle h_{t}=\Gamma_{u}\odot\tilde{h}_{t}+\Gamma_{f}\odot
h_{t-1}$ (6d)

Finally, the last hidden state variable $h_{t}$ can be used to predict the
corresponding output value $\hat{y}_{t}$ through a fully connected layer with
$softmax$ activation function:

$\hat{y}_{t}=softmax(\bm{W}_{hy}h_{t})$ (7)

### Temporal graph learning algorithms

Graph neural networks (GNNs) are a class of neural networks that operate on
graph-structured data. Graphs are a powerful method to represent many types of
data, such as social networks, biological networks, and traffic flows. GNNs
are capable of learning the relationships between nodes in a graph. They
generalize the concept of convolutional neural networks to non-Euclidean
domains by defining local operations on the nodes and edges of a graph. A
typical GNN layer operate on input graph $\mathcal{G}=(\bm{X},\bm{E},\bm{A})$
can be formulate as in Equation 8.

$\bm{Y}=g_{\bm{W}}\star\bm{X}\approx(\bm{I_{N}}+\bm{D}^{-\frac{1}{2}}\bm{A}\bm{D}^{-\frac{1}{2}})\bm{X}\bm{W}$
(8)

where $\bm{X}\in\mathbb{R}^{N\times D_{X}}$ represents node matrix, each of
the $N$ nodes has $D_{X}$ features, and $\bm{A}\in\mathbb{R}^{N\times N}$ is a
weighted adjacency matrix encoding set of edges $\bm{E}$. The graph
convolution operator $\star$ can be approximated by first-order Chebyshev
polynomial expansian and generalized to high-dimensional [45, 50] with
learnable parameter $\bm{W}\in\mathbb{R}^{D_{X}\times D_{Y}}$.

Temporal Graph Neural Networks are an extension of GNNs that can handle
temporal graphs, i.e., graphs that change over time. Unlike static graphs,
temporal graphs are usually represented by a sequence of node interactions
over continuous time instead of an adjacency matrix. Temporal GNNs aim to
capture both the temporal and structural information of the temporal graphs by
introducing a node memory that represents the state of the node at a given
time, acting as a compressed representation of the node’s past interactions.
In this work, we follow the framework of recent studies, including GCRN [37],
AGCRN [47], and MPNN LSTM [15], that utilize recurrent neural network on top
of graph convolution operators. We leverage a simplified approach and use GRU
as the recursive network architecture.

$\displaystyle\bm{\Gamma}_{U}=\sigma(g_{\bm{W}_{HU}}\star\bm{H}_{t-1}+g_{\bm{W}_{XU}}\star\bm{X}_{t}+\bm{b_{U}})$
(9a) $\displaystyle\bm{\Gamma}_{F}=1-\bm{\Gamma_{U}}$ (9b)
$\displaystyle\bm{\tilde{H}}_{t}=\tanh(g_{\bm{W}_{HH}}\star\bm{H}_{t-1}+g_{\bm{W}_{XH}}\star\bm{X}_{t}+\bm{b_{H}})$
(9c)
$\displaystyle\bm{H}_{t}=\bm{\Gamma}_{U}\odot\bm{\tilde{H}}_{t}+\bm{\Gamma}_{F}\odot\bm{H}_{t-1}$
(9d)

This framework allows MGL4MEP to learn the dynamic interactions between
entities, act as nodes, or indicators to the current status of the pandemic
and between different time stamps throughout the evolution of the pandemic.

### Pre-trained Language models

Since we are dealing with free-text data to capture population’s reactions to
the pandemic, more specifically, user-generated content sourced from social
media, it is crucial to extract meaningful information before constructing the
graph-structured representation of the data. In recent years, large pre-
trained language models have revolutionized the field of natural language
processing such as Bidirectional Encoder Representations from Transformers
(BERT) [51]. These models have demonstrated exceptional capabilities in
understanding and generating human language. BERT’s underlying architecture,
based on Transformer [52], employs self-attention mechanisms to capture
dependencies between words or tokens in a sentence. This enables BERT to
comprehend the contextual information of a word based on its surrounding
words, leading to more accurate language understanding and representation. By
pre-training BERT on massive amounts of textual data and a wide variety of
tasks, such as masked language modeling and next sentence prediction, the
model learns a rich language representation that can be fine-tuned for
specific downstream tasks.

Building upon recent advancements in applying large pre-trained models to
domain-specific data [53, 54], we leverage BertTweet [55], a variant of BERT
as our main feature extractor for text embeddings. The model has been trained
on a large amount of Twitter data, especially including a sub-set of COVID-19
related data, and its effectiveness in capturing the nuanced meanings and
signals conveyed in text data has been well-established. By leveraging
BertTweet, we can obtain high-quality features that accurately represent the
semantic content of social media posts surrounding the COVID-19 pandemic, and
enables us to discover valuable patterns and trends that contribute to a
comprehensive understanding of the social media landscape during the global
health crisis.

## Methodology

### MGL4MEP

Figure 3: Overall architecture of MGL4MEP - a multi-modal framework for
enhanced pandemic forecasting with external resources. MGL4MEP incorporates
both pandemic related metrics and population’s reactions on social media into
the forecasting to better capture the dynamic properties of emerging
pandemics.

(a)

(b)

Figure 4: Comparison between amount of tweets posted and number of new
COVID-19 cases per day. (a) in California state. (b) in New York state.

In this section, we present the multi-modality framework and techniques
employed in our study to effectively extract and model multi-modal data for
COVID-19 forecasting. Our approach aims to harness the power of social media
data, specifically user-generated content, and government stringency index, to
gain valuable insights to the evolution of the pandemic. We describe the key
components of our framework, including data pre-processing, feature
extraction, graph-based representation with temporal graph learning, and
multi-modal learning. The main components of our proposed framework are
depicted in Figure 3.

One goal of our multi-modality framework is to effectively incorporate signals
from human-generated text data through social media platform, offering
valuable reflections of the population’s response to the pandemic, as exampled
in Figure 1. This importance can also be underscored by the sheer amount of
Covid-19 discussion over time, strongly correlating with pandemic statistics,
as shown in Figure 4. Details about our social media data collection process
is provided in the next Section. Extracting meaningful features from text data
is crucial for constructing a comprehensive understanding of the information
shared on social media platforms. While other works proposed another direction
of extracting indices like sentiment, this might discard necessary
information, such as sentiment about COVID-19, but not about current
government response to the pandemic. As discussed in previous sections, by
employing pre-trained language model, specifically BertTweet, as text feature
extractor, we ensure to capture rich insights contained within user-generated
content, especially in relation to COVID-19 pandemic. Our framework enables
integrating various modalities to capture the complex and temporal dynamics of
emerging pandemics. We obtain temporal embedding for each user by utilizing
BertTweet on their text data as follows:

$x_{t,user_{i}}=\sum_{j\in(t,user_{i})}\text{BertTweet}(post_{j}),$ (10)

where $t$ denotes the timestamp, $user_{i}$ denotes the i-th user in our
social media data, and $post_{j}$, $j\in(t,user_{i})$ denotes the text
obtained from the tweets of the i-th user at time $t$. The inclusion of user
interactions and shared information across users is crucial for further
analysis. In order to capture the correlations and dependencies between users,
it is imperative to construct a graph-structured representation of the pre-
processed data. This graph-based approach allows us to model the interactions
and information flow through graph neural networks, capturing the dynamics and
valuable insights related to the ongoing pandemic from social media signals.
Hence, we introduce an end-to-end learning algorithm to discover the
underlying graph structure that captures the correlation among time-series in
a data-driven manner. More specifically, define node embeddings extracted from
pre-trained language model as
$\bm{X}^{\mathcal{G}}_{t}:=[x_{t,user_{1}},x_{t,user_{2}},\ldots,x_{t,user_{N}}]^{T}\in\mathbb{R}^{N\times
D_{X}},$ and the continuous adjacency matrix can be calculated as the dot-
product similarity matrix of the node embeddings:
$\bm{A}^{\mathcal{G}}_{t}=\bm{X}^{\mathcal{G}}_{t}\cdot{\bm{X}^{\mathcal{G}}_{t}}^{T}\in\mathbb{R}^{N\times
N}$. However, to enable effective learning with temporal graph learning
algorithms, there are two downsides with this approach: first, large embedding
dimension lead to incorrect adjacency matrix calculation. Second, may include
information not directly related to our downstream task, and take up resources
in training and evaluating. Inspired from AGCRN [47], we employed a node-
specific learnable embeddings that allows us to map input dimension to a lower
intermediate embedding dimension:

$g_{\bm{W},\bm{E}}\star\bm{X}=(\bm{I_{N}}+\text{softmax}(\text{ReLU}(\bm{E}\cdot\bm{E}^{T}))\bm{X}\bm{E}\bm{W},$
(11)

where $g$ denotes the filter parameterized by W and E, while $\star$ denotes
the graph convolution operator. $\bm{E}$ is a learnable intermediate node
embedding matrix, $\bm{E}\in\mathbb{R}^{N\times D_{emb}}$. The input node
matrix is multiplied with the node embedding $\bm{E}$, resulting in an updated
representation, where $\bm{E}$ is learnable, meanings that the representation
is specific for each node and its pattern. Then, the integrated node
embeddings are further multiplied by the weight matrix $\bm{W}$ to incorporate
the influence of the node-specific features. Moreover, we replace the
normalized graph Laplician matrix [50] by computing the inner product of the
intermediate node embedding matrix $\bm{E}$ with its transpose $\bm{E}^{T}$.
This operation captures the pairwise relationships between node embeddings and
produces a matrix of shape $N\times N$. We apply the rectified linear unit
(ReLU) activation function to introduce non-linearity and ensure positive
values in the resulting matrix. The softmax function is then applied to
normalize the values across matrix row, ensuring that the row sums to 1. This
step allows us to obtain a valid probability distribution representing the
importance or relevance of each node, or each user, with respect to others.

This graph convolution operation is plugged into the framework in Equation 9,
and the final temporal graph learning algorithm is shown in Equation 12:

$\displaystyle\bm{\Gamma}_{U}^{\mathcal{G}}=\sigma(g_{\bm{W}_{HU},\bm{E}_{H}}\star\bm{H}_{t-1}^{\mathcal{G}}+g_{\bm{W}_{XU},\bm{E}_{X}}\star\bm{X}_{t}^{\mathcal{G}}+\bm{b}_{U})$
(12a)
$\displaystyle\bm{\Gamma}_{F}^{\mathcal{G}}=1-\bm{\Gamma_{U}}^{\mathcal{G}}$
(12b)
$\displaystyle\bm{\tilde{H}}_{t}^{\mathcal{G}}=\tanh(g_{\bm{W}_{HH},\bm{E}_{H}}\star\bm{H}_{t-1}^{\mathcal{G}}+g_{\bm{W}_{XH},\bm{E}_{X}}\star\bm{X}_{t}^{\mathcal{G}}+\bm{b}_{H})$
(12c)
$\displaystyle\bm{H}_{t}^{\mathcal{G}}=\bm{\Gamma}_{U}^{\mathcal{G}}\odot\bm{\tilde{H}}_{t}^{\mathcal{G}}+\bm{\Gamma}_{F}^{\mathcal{G}}\odot\bm{H}_{t-1}^{\mathcal{G}}$
(12d)

where $g$ denotes learnable weights with respect to different embeddings. To
complement the multi-modal nature of our framework, we incorporate government
stringency features that provide valuable insights into the pandemic response
at a regional level. Government stringency features capture the level of
restrictions, policies, and interventions implemented by authorities to
mitigate the spread of COVID-19. These features serve as an important
contextual signal to enhance the understanding of the evolving dynamics in our
model.

Specifically, we utilize the raw data and formula proposed in [56] to compute
an indicator on the level of government stringency. However, recognizing the
complexity of this domain, we compare and analyze each individual indicator,
as well as the averaged general stringency index, to identify the most
suitable indicator for the current pandemic situation. In Figure 5, we present
the correlation levels between index and the number of new COVID cases for two
indicators, with different time lags. The results suggest a strong
relationship between the restriction on internal movement to the status of the
pandemic. Hence, in the refined version of our framework, we leverage this
specific indicator as a measure of government stringency.

Since this indicator can be represented as a vector for each day, similar to
the statistical metrics of the pandemic, we can employ a sophisticated
recurrent neural network (i.e., Equation 5 and Equation 6) to learn solely on
this feature. Alternatively, we have the option to combine, or concatenate it
with the pandemic statistics and learn through a unified recurrent network.
Through extensive experimentation, we have found that the latter approach
yields superior performance, and thus, it is our final choice for
incorporating the government stringency indicator into our framework.

Finally, in order make accurate predictions, it is crucial to integrate the
information from multiple modalities in our framework. We achieve this by
fusing the embeddings obtained from different modalities, namely statistical
features, government stringency features, and social-media graph-based
features. The fusion process is performed using the equation:

$\hat{y}_{t+T}=\text{softmax}(\bm{W}(\bm{H}_{t+T}^{\\{stat,reg\\}}\oplus\bm{H}_{t+T}^{\mathcal{G}}))$
(13)

where $\bm{H}_{t+T}^{\\{stat,reg\\}}$ is the learned embeddings of recurrent
neural network for statistical metrics, $\hat{y}_{t+T}$ represents the
predicted value for the time step $t+T$, where $T$ is the forecasting horizon.
Embeddings from various domain, capturing the relevant information for each
modality, are fused using the concatenation operator $\oplus$ to create a
unified feature representation.

Using the aforementioned equation to integrate the embeddings from multi-
modality, our system successfully combines a variety of information sources
while utilizing the complementary nature of different modalities for enhanced
forecasting performance. This comprehensive approach enables us to capture the
intricate dynamics and interdependencies within the data, leading to more
accurate and reliable forecasts for the future evolution of the pandemic.

### Multimodal data collection process

In this study, as shown in Table 8, we utilized three different types of data
sources to gain insights into the COVID-19 pandemic and its development.

Table 8: Input features used for proposed approaches Data Source |  Features
---|---
JHU CSSE COVID-19 Data [57] |  Daily COVID-19 Statistics
Oxford Covid-19 Government Response Tracker [56] |  Government Stringency Index
|  Rate of Change of Stringency Index over a time period
|  Restrictions on Internal Movement Indicator
|  Rate of Change of Restrictions on Internal Movement Indicator over a time
period
Twitter [58] |  Daily user-generated contents from users with topics-of-interest related to COVID-19

  * •

COVID-19 Statistic Data. We leverage the statistic dataset from Johns Hopkins
University [57] with 450 data points from August 1, 2020, to November 30,
2021. Each data point is represented as the number of confirmed COVID-19
infections or serious, hospitalized cases in a given area per day. Our final
task is time-series forecasting on this multi-variate statistics with
different horizons to predict the trajectory of the pandemic. Then, trained
models can be a valuable tool in responding to the pandemic, as it can support
policymakers give better decisions about how to allocate resources, implement
public health measures, and prepare for the future.

  * •

COVID-19 Government Responses and Regulations Data. The stringency index data
[56] is a valuable resource in understanding the level of government response
to the COVID-19 pandemic. The index is represented as a numeric value between
$0$ to $100$ and includes nine different indicators, such as the closure of
schools and workplaces, cancellation of public events, restrictions on
gatherings, and orders to shelter in place. Fig. 5 displays the correlation
values between the stringency index and record restrictions on internal
movement between regions and the daily statistics of new infected cases.
Interestingly, both time-lag horizons exhibit a clear trend of correlation
values peaking at around 30 days. Moreover, the correlation of record
restrictions on internal movement with new infected cases is consistently
higher than that of the stringency index. This is also observed when
considering new hospitalized cases. The found correlation trends imply that
the current government response can act as a valuable indication to forecast
how the epidemic will develop in the future.

  * •

Social Media Data. We crawl a total amount of more than 74 million tweets
using Twitter API and tweets ids of all tweets related to COVID-19 released by
Banda et al. [58]. The original authors leveraged Twitter Stream for
collecting all tweets in the category of COVID-19 chatter, with over 4 million
tweets a day. We filtered out tweets with geo-location tags in either
California state or New York state in this exploratory study. Moreover, we
filtered out all tweets that are not in English. We randomly keep all tweets
from 1,500 different users for each location. The distributions of tweets over
time with respect to statistics of newly confirmed cases are illustrated in
Fig. 4. A strong correlation between the two time-series can be recognized,
although there is a noisy period at the start. This is likely due to the
initial confusion and fear surrounding the appearance of COVID-19, which led
to a high volume of discussions about the virus worldwide. As the situation
became more stable and people gained a better understanding of the pandemic
affection on their own regions, the amount of tweets posted became more
relevant and had a higher correlation with users’ areas of residence. To
account for this, we excluded data from the initial few months of the pandemic
and only collected data starting from August 1, 2020.

Figure 5: Correlations between Stringency Index or Restrictions on internal
movement to daily infected cases of COVID-19 across different time-lags.

## Conclusion

In this work, we present a novel framework named MGL4MEP that combines
temporal graph neural networks and multi-modal data for accurate pandemic
forecasting. By integrating various big data sources, including social media
content, we effectively capture the complex dynamics of emerging pandemics.
Our framework outperforms traditional approaches by leveraging the potential
of pre-trained language models and generating graph-structured data. Extensive
experiments conducted with multiple variants of our proposed method
demonstrate the effectiveness of our framework in providing timely and
comprehensive insights into the pandemic landscape. The fusion of temporal
graph learning and multi-modal data enables a deeper understanding of the
evolving patterns and indicators, leading to more informed public health
management and decision-making. Our approach offers a promising direction for
leveraging big data in pandemic research and provides a foundation for future
advancements in the field.

## References

  * [1] Quach, H.-L. _et al._ Using ‘infodemics’ to understand public awareness and perception of sars-cov-2: A longitudinal analysis of online information about covid-19 incidence and mortality during a major outbreak in vietnam, july—september 2020.  _PLOS ONE_ 17, e0266299 (2022).
  * [2] Comito, C.  How covid-19 information spread in u.s.? the role of twitter as early indicator of epidemics.  _IEEE Transactions on Services Computing_ 15, 1193–1205 (2022).
  * [3] Kermack, W. O. & Mckendrick, À. G.  A contribution to the mathematical theory of epidemics.  _Proceedings of The Royal Society A: Mathematical, Physical and Engineering Sciences_ 115, 700–721 (1927).
  * [4] Chatfield, C.  _Time-Series Forecasting_ (Chapman and Hall/CRC, 2000).
  * [5] Taylor, S. J. & Letham, B.  Forecasting at scale.  _The American Statistician_ 72, 37–45 (2018).
  * [6] Tran, T. K., Vu, X.-S. & Jiang, L.  Sobigdemicsys: A social media based monitoring system for emerging pandemics with big data.  In _2022 IEEE Eighth International Conference on Big Data Computing Service and Applications (BigDataService)_ , 103–107 (2022).
  * [7] Kırbaş, İ., Sözen, A., Tuncer, A. D. & Kazancıoğlu, F. Ş.  Comparative analysis and forecasting of COVID-19 cases in various european countries with ARIMA, NARNN and LSTM approaches.  _Chaos, Solitons & Fractals_ 138, 110015 (2020).
  * [8] Dai Y, W. J.  Identifying the outbreak signal of covid-19 before the response of the traditional disease monitoring system.  _I PLoS Negl Trop Dis._ 14 (2020).
  * [9] Higgins, T. S. _et al._ Correlations of online search engine trends with coronavirus disease (COVID-19) incidence: Infodemiology study.  _JMIR Public Health and Surveillance_ 6, e19702 (2020).
  * [10] Tsao, S.-F. _et al._ What social media told us in the time of COVID-19: a scoping review.  _The Lancet Digital Health_ 3, e175–e194 (2021).
  * [11] Li, L., Ma, Z., Lee, H. & Lee, S.  Can social media data be used to evaluate the risk of human interactions during the COVID-19 pandemic?  _International Journal of Disaster Risk Reduction_ 56, 102142 (2021).
  * [12] Lamsal, R., Harwood, A. & Read, M. R.  Twitter conversations predict the daily confirmed covid-19 cases.  _Applied Soft Computing_ 129, 109603 (2022).
  * [13] Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M. & Monfardini, G.  The graph neural network model.  _IEEE Transactions on Neural Networks_ 20, 61–80 (2009).
  * [14] Zhou, J. _et al._ Graph neural networks: A review of methods and applications.  _AI Open_ 1, 57–81 (2020).
  * [15] Panagopoulos, G., Nikolentzos, G. & Vazirgiannis, M.  Transfer Graph Neural Networks for Pandemic Forecasting.  In _Proceedings of the 35th AAAI Conference on Artificial Intelligence_ (2021).
  * [16] Zhou, Y., Jiang, J.-Y., Chen, X. & Wang, W.  #stayhome or #marathon? social media enhanced pandemic surveillance on spatial-temporal dynamic graphs.  In _Proceedings of the 30th ACM International Conference on Information & Knowledge Management_, CIKM ’21, 2738–2748 (Association for Computing Machinery, New York, NY, USA, 2021).
  * [17] Jiang, J.-Y. _et al._ COVID-19 surveiller: toward a robust and effective pandemic surveillance system based on social media mining.  _Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences_ 380 (2021).
  * [18] Paszke, A. _et al._ Pytorch: An imperative style, high-performance deep learning library.  In Wallach, H. _et al._ (eds.) _Advances in Neural Information Processing Systems_ , vol. 32 (Curran Associates, Inc., 2019).
  * [19] Loshchilov, I. & Hutter, F.  Decoupled weight decay regularization.  In _International Conference on Learning Representations_ (2019).
  * [20] Postnikov, E. B.  Estimation of COVID-19 dynamics “on a back-of-envelope”: Does the simplest SIR model provide quantitative parameters and predictions?  _Chaos, Solitons & Fractals_ 135, 109841 (2020).
  * [21] Fernández-Villaverde, J. & Jones, C. I.  Estimating and simulating a SIRD model of COVID-19 for many countries, states, and cities.  _Journal of Economic Dynamics and Control_ 140, 104318 (2022).
  * [22] Tang, B. _et al._ Estimation of the transmission risk of the 2019-nCoV and its implication for public health interventions.  _Journal of Clinical Medicine_ 9, 462 (2020).
  * [23] Liu, F. _et al._ Predicting and analyzing the COVID-19 epidemic in china: Based on SEIRD, LSTM and GWR models.  _PLOS ONE_ 15, e0238280 (2020).
  * [24] Hochreiter, S. & Schmidhuber, J.  Long short-term memory.  _Neural Comput._ 9, 1735–1780 (1997).
  * [25] Masum, M., Shahriar, H., Haddad, H. M. & Alam, M. S.  r-lstm: Time series forecasting for covid-19 confirmed cases with lstmbased framework.  In _2020 IEEE International Conference on Big Data (Big Data)_ , 1374–1379 (2020).
  * [26] Chandra, R., Jain, A. & Chauhan, D. S.  Deep learning via LSTM models for COVID-19 infection forecasting in india.  _PLOS ONE_ 17, e0262708 (2022).
  * [27] Nikparvar, B., Rahman, M. M., Hatami, F. & Thill, J.-C.  Spatio-temporal prediction of the COVID-19 pandemic in US counties: modeling with a deep LSTM neural network.  _Scientific Reports_ 11 (2021).
  * [28] Lucas, B., Vahedi, B. & Karimzadeh, M.  A spatiotemporal machine learning approach to forecasting COVID-19 incidence at the county level in the USA.  _International Journal of Data Science and Analytics_ 15, 247–266 (2022).
  * [29] Hy, T. S., Nguyen, V. B., Tran-Thanh, L. & Kondor, R.  Temporal multiresolution graph neural networks for epidemic prediction.  In Xu, P. _et al._ (eds.) _Proceedings of the 1st Workshop on Healthcare AI and COVID-19, ICML 2022_ , vol. 184 of _Proceedings of Machine Learning Research_ , 21–32 (PMLR, 2022).
  * [30] Panagopoulos, G., Nikolentzos, G. & Vazirgiannis, M.  Transfer graph neural networks for pandemic forecasting.  _Proceedings of the AAAI Conference on Artificial Intelligence_ 35, 4838–4845 (2021).
  * [31] Tran, V. & Matsui, T.  Tweet analysis for enhancement of covid-19 epidemic simulation: A case study in japan.  _Frontiers in Public Health_ 10 (2022).
  * [32] Ibrahim, M. R. _et al._ Variational-LSTM autoencoder to forecast the spread of coronavirus across the globe.  _PLOS ONE_ 16, e0246120 (2021).
  * [33] Bae, S., Sung, E. C. & Kwon, O.  Accounting for social media effects to improve the accuracy of infection models: combatting the COVID-19 pandemic and infodemic.  _European Journal of Information Systems_ 30, 342–355 (2021).
  * [34] Khan, F. M., Kumar, A., Puppala, H., Kumar, G. & Gupta, R.  Projecting the criticality of COVID-19 transmission in india using GIS and machine learning methods.  _Journal of Safety Science and Resilience_ 2, 50–62 (2021).
  * [35] Xu, L., Magar, R. & Farimani, A. B.  Forecasting COVID-19 new cases using deep learning methods.  _Computers in Biology and Medicine_ 144, 105342 (2022).
  * [36] Chew, A. W. Z., Pan, Y., Wang, Y. & Zhang, L.  Hybrid deep learning of social media big data for predicting the evolution of covid-19 transmission.  _Knowledge-Based Systems_ 233, 107417 (2021).
  * [37] Seo, Y., Defferrard, M., Vandergheynst, P. & Bresson, X.  Structured sequence modeling with graph convolutional recurrent networks.  In Cheng, L., Leung, A. C. S. & Ozawa, S. (eds.) _Neural Information Processing_ , 362–373 (Springer International Publishing, Cham, 2018).
  * [38] Wu, Z., Pan, S., Long, G., Jiang, J. & Zhang, C.  Graph WaveNet for deep spatial-temporal graph modeling.  In _Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence_ (International Joint Conferences on Artificial Intelligence Organization, 2019).
  * [39] Chen, Z.-M., Wei, X.-S., Wang, P. & Guo, Y.  Multi-label image recognition with graph convolutional networks.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 5177–5186 (2019).
  * [40] Nguyen, H. D., Vu, X.-S. & Le, D.-T.  Modular graph transformer networks for multi-label image classification.  _Proceedings of the AAAI Conference on Artificial Intelligence_ 35, 9092–9100 (2021).
  * [41] Duvenaud, D. K. _et al._ Convolutional networks on graphs for learning molecular fingerprints.  _Advances in neural information processing systems_ 28 (2015).
  * [42] Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O. & Dahl, G. E.  Neural message passing for quantum chemistry.  In _Proceedings of the 34th International Conference on Machine Learning - Volume 70_ , ICML’17, 1263–1272 (JMLR.org, 2017).
  * [43] Hy, T. S., Trivedi, S., Pan, H., Anderson, B. M. & Kondor, R.  Predicting molecular properties with covariant compositional networks.  _The Journal of Chemical Physics_ 148, 241745 (2018).
  * [44] Fout, A., Byrd, J., Shariat, B. & Ben-Hur, A.  Protein interface prediction using graph convolutional networks.  In Guyon, I. _et al._ (eds.) _Advances in Neural Information Processing Systems_ , vol. 30 (Curran Associates, Inc., 2017).
  * [45] Defferrard, M., Bresson, X. & Vandergheynst, P.  Convolutional neural networks on graphs with fast localized spectral filtering.  _Advances in neural information processing systems_ 29 (2016).
  * [46] Li, Y., Yu, R., Shahabi, C. & Liu, Y.  Diffusion convolutional recurrent neural network: Data-driven traffic forecasting.  In _International Conference on Learning Representations_ (2018).
  * [47] Bai, L., Yao, L., Li, C., Wang, X. & Wang, C.  Adaptive graph convolutional recurrent network for traffic forecasting.  In _Proceedings of the 34th International Conference on Neural Information Processing Systems_ , NIPS’20 (Curran Associates Inc., Red Hook, NY, USA, 2020).
  * [48] Nguyen, D. T., Nguyen, M. D. T., Hy, T. S. & Kondor, R.  Fast temporal wavelet graph neural networks.  _arXiv preprint arXiv:2302.08643_ (2023).
  * [49] Cho, K. _et al._ Learning phrase representations using RNN encoder–decoder for statistical machine translation.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , 1724–1734 (Association for Computational Linguistics, Doha, Qatar, 2014).
  * [50] Kipf, T. N. & Welling, M.  Semi-supervised classification with graph convolutional networks.  In _International Conference on Learning Representations_ (2017).
  * [51] Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K.  Bert: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North_ (Association for Computational Linguistics, 2019).
  * [52] Vaswani, A. _et al._ Attention is all you need.  In _Proceedings of the 31st International Conference on Neural Information Processing Systems_ , NIPS’17, 6000–6010 (Curran Associates Inc., Red Hook, NY, USA, 2017).
  * [53] Lee, J. _et al._ BioBERT: a pre-trained biomedical language representation model for biomedical text mining.  _Bioinformatics_ 36, 1234–1240 (2019).
  * [54] Gu, Y. _et al._ Domain-specific language model pretraining for biomedical natural language processing.  _ACM Trans. Comput. Healthcare_ 3 (2021).
  * [55] Nguyen, D. Q., Vu, T. & Nguyen, A. T.  BERTweet: A pre-trained language model for English Tweets.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , 9–14 (2020).
  * [56] Hale, T. _et al._ A global panel database of pandemic policies (oxford COVID-19 government response tracker).  _Nature Human Behaviour_ 5, 529–538 (2021).
  * [57] Dong, E., Du, H. & Gardner, L.  An interactive web-based dashboard to track COVID-19 in real time.  _The Lancet Infectious Diseases_ 20, 533–534 (2020).
  * [58] Banda, J. M. _et al._ A large-scale covid-19 twitter chatter dataset for open scientific research—an international collaboration.  _Epidemiologia_ 2, 315–324 (2021).

## Acknowledgements

The project is partially funded by the STINT Mobility Grants for
Internationalisation in Sweden (no: MG2020-8848). The computations was enabled
by the supercomputing resource Berzelius provided by National Supercomputer
Centre at Linköping University and the Knut and Alice Wallenberg foundation.

## Author contributions

Conceptualization: T.S.H., L.J., and X.S.V. Methodology: K.T.T., L.J., and
X.S.V. Software: K.T.T. Analysis: K.T.T., T.S.H. Validation: L.J., and X.S.V.
Paper preparation: K.T.T., T.S.H., L.J., and X.S.V. All authors reviewed the
manuscript.

## Data availability

The COVID-19 statistics leveraged in this study are available from Johns
Hopkins University Center for Systems Science and Engineering in the COVID-19
repository, https://github.com/CSSEGISandData/COVID-19. The COVID-19
Government Responses and Regulations Data is sourced from The Oxford Covid-19
Government Response Tracker (OxCGRT), https://github.com/OxCGRT/covid-policy-
dataset. The social media data are crawled from Twitter social media platform
by tweets related to COVID-19 released by Covid-19 Twitter chatter dataset for
scientific use, https://github.com/thepanacealab/COVID19_twitter.

## Competing interests

The authors declare no competing interests.