# Ask “Who”, Not “What”: Bitcoin Volatility Forecasting with Twitter Data

M. Eren Akbiyik<EMAIL_ADDRESS>ETH ZurichRamistrasse 50ZurichSwitzerland ,
Mert Erkul<EMAIL_ADDRESS>ETH ZurichRamistrasse 50ZurichSwitzerland ,
Killian Kaempf<EMAIL_ADDRESS>ETH ZurichRamistrasse 50ZurichSwitzerland ,
Vaiva Vasiliauskaite<EMAIL_ADDRESS>ETH ZurichRamistrasse
50ZurichSwitzerland and  Nino Antulov-Fantulin<EMAIL_ADDRESS>ETH
ZurichRamistrasse 50ZurichSwitzerland

###### Abstract.

Understanding the variations in trading price (volatility), and its response
to external information is a well-studied topic in finance. In this study, we
focus on volatility predictions for a relatively new asset class of
cryptocurrencies (in particular, Bitcoin) using deep learning representations
of public social media data from Twitter. For the field work, we extracted
semantic information and user interaction statistics from over 30 million
Bitcoin-related tweets, in conjunction with 15-minute intraday price data over
a 144-day horizon. Using this data, we built several deep learning
architectures that utilized a combination of the gathered information. For all
architectures, we conducted ablation studies to assess the influence of each
component and feature set in our model. We found statistical evidences for the
hypotheses that: (i) temporal convolutional networks perform significantly
better than both autoregressive and other deep learning-based models in the
literature, and (ii) the tweet author meta-information, even detached from the
tweet itself, is a better predictor than the semantic content and tweet volume
statistics.

††conference: WebConf ’18: The ACM Web Conference; April 25–29, 2022; Lyon,
France

##  1\. Introduction

Cryptocurrency market, after its surge in 2009, gained immense popularity
among not only small-scale investors, but also large hedge funds. A study in
2017 indicated that there were approximately 6 million unique users who held
cryptocurrencies (crypto_bench, ), whereas in 4 years this number has risen to
around 300 million (crypto_owner, ). Acknowledging this exponential increase
in popularity, investors began constructing portfolios using cryptocurrencies,
however, the vast majority of the market share still belongs to individuals
(crypto_owner, ). The importance of individual investors in cryptocurrency
markets incentivizes studying alternative sources of information, such as
social media, to which the market participants may respond.

With its widely available API, ease of use, and its primary purpose of
information sharing, Twitter is both relevant and convenient online resource,
utilized in many past studies where relation between financial markets and
social media was investigated (tt_snp, ; rel_2_Bitcoin_resp, ;
rel_3_cnn_lstm_crypto, ; rel_4_predict_stock_market, ;
rel_5_influence_social_snp, ; rel_6_social_media_forecast, ;
rel_7_twitter_pred_btc, ; rel_8_rnn, ; rel_9_comp_study, ). Among
cryptocurrencies, Bitcoin is arguably the most appropriate candidate to
analyse the relationship between social media and cryptocurrency markets.
Although more than 1000 cryptocurrencies are currently available, Bitcoin
remains in the focus of attention, likely due to its accessibility, maturity,
and dominance in market capitalization.

Investigating the relationship between a financial asset and some exogenous
factor requires a clear definition of the targeted properties of this asset.
Among the main considerations for investors in forming portfolios
(volatility_deriv, ), realized volatility is one such property, that was
previously examined in the context of relationship between Bitcoin and Twitter
(rel_7_twitter_pred_btc, ; nino_garch, ). Understanding the key drivers of
inter-day realized volatility of cryptocurrencies is becoming more essential
with the increasing popularity of high-frequency trading of Bitcoin
(rise_of_machines, ). Such analyses must be conducted with great care to
ensure the discovery of proper causal connections between price changes, that
also behaves steadily under unpredictable conditions of the assessed exogenous
factors.

Building upon these insights, our goal in this study is to discover
information, publicly available on Twitter, that may explain some components
of the daily realized volatility of Bitcoin prices. It is important to note
that in this study, we do not aim to find the best state-of-the-art model that
can predict the price volatility of Bitcoin over some test dataset. Instead,
we intend to systematically compare the classical financial models with deep
learning-based approaches for volatility forecasting, and to understand the
key exogenous (social media) features or representations that may drive
realized volatility, when combined with endogenous market features. To achieve
this goal, we utilize a modular and testable architecture to which social
media and other features can be progressively added.

For the prediction task, we consider the intraday logarithmic returns, tweet
counts over 15-minute time frames, and other publicly available Twitter data.
To analyse the impact of social media, we extracted semantic information and
user interaction statistics from over 30 million Bitcoin-related tweets. The
text content of tweets has to be “interpreted”, i.e., assigned emotional
sentiment, or some other machine-comprehensive meaning. Here we used Valence
Aware Dictionary for Sentiment Reasoning (VADER) (vader, ), a well-known
natural language processing tool for extracting emotional sentiment of textual
data. Noting that certain players in both cryptocurrency market, as well as in
social media are more influential than others, we also incorporated tweet
owner’s account statistics, such as follower count, verified status, and
activity to our feature space. This allowed us to separately examine the
importance of individual tweets and the potential influence of notable agents.

To perform a comparative analysis of different feature sets, we designed a
deep learning architecture that can flexibly make use of different feature
spaces. We conducted ablation studies to statistically compare the information
content of different feature sets, while controlling for the added complexity
of model architectures and parameter spaces. We first showed that the deep
learning models are promising tools for Bitcoin volatility forecasting,
outperforming classic financial forecasting methods. We then upgraded the most
promising deep learning model to allow for Twitter-related features. Our
results suggested that certain, but not all features of social media are
accurate and useful predictors of inter-day Bitcoin volatility.

This paper is organized as follows. In Section 2, we discuss the previous
literature. In Section 3 we present the data collection process, followed by a
definition of a volatility prediction task in Section 4. In Section 5 and
Section 6, we describe and study deep learning architectures that incorporate
various amounts of price and social media information. We conclude and discuss
the implications of our study and results in Section 7.

##  2\. Related Work

Over the years, many studies suggested forecasting prices and volatility by
utilizing historic prices alone. Such forecasting techniques are usually based
on econometric models, such as Auto-Regressive Conditional Heteroskedasticity
model (ARCH) (engle1982autoregressive, ), Generalized Auto-Regressive
Conditional Heteroskedasticity model (GARCH) (bollerslev1986generalized, ),
Heterogeneous Auto-Regressive model of Realized (HAR-RV) Volatility (har, ).
The difference in these models stems from accounting for various stylized
facts observed in real markets. We also note that models based on the realized
volatility appear to be a more precise in forecasting volatility than
conditional variance models (ARCH_HAR, ).

To apply state-of-the-art time series analysis tools to financial contexts,
machine learning and, in particular, deep learning models, have been
extensively researched over the recent years. Recurrent Neural Networks (RNNs)
and, specifically, Long Short-Term Memory networks (LSTMs) (liu2019novel, ;
nguyen2019long, ) were shown to be reliable and informative for a diverse set
of tasks (rel_9_comp_study, ). Recently, convolutional neural networks (CNNs)
for causal temporal analysis, also known as Temporal Convolutional Networks
(TCNs), have also gained traction (mcnally2018predicting, ; saad2019toward, ).
Models that combine these two paradigms have also been studied
(guo2021mrclstm, ; rel_3_cnn_lstm_crypto, ).

Stylized facts are not fully known in cryptocurrency markets, and it is not
entirely clear what the economic value of cryptocurrencies is (Hayes2015, ;
Bolt2016, ; nadarajah2017inefficiency, ), which econometric models are
relevant, and how price discovery (cheah2015speculative, ; kristoufek2015main,
; dimpfl2021nothing, ) or bubbles (BouchaudBTC, ; sornetteBTC, ) occur.
Therefore, cryptocurrency markets have been studied from various alternative
perspectives, ranging from volatility and volume forecasting using standard
econometric models (Katsiampa2017, ; GuoBifetAntulov18, ; antulov2021temporal,
), to employing tools from systems dynamics (Ron2013BTC, ; ElBahrawy2017, ;
AntTol18, ). Various studies analysed the effect of external data from social
media, news, search queries, sentiment, comments, replies on forums, and
blockchain (Garcia2015, ; DeyBlock2020, ; nino_garch, ; beck2019sensing, ).
Typically, sentiment analysis is performed and combined with econometric
models such as GARCH (rel_5_influence_social_snp, ) or deep learning models
such as CNN (rel_2_Bitcoin_resp, ). The impact of social media and other
exogenous factors on price in both cryptocurrency and conventional markets
suggests that prices respond to more than price history. The inclusion of data
from social media, news feed or blockchain to improve forecasts has been
studied widely (rel_9_comp_study, ; rel_5_influence_social_snp, ;
rel_6_social_media_forecast, ; rel_7_twitter_pred_btc, ). While it seems the
impact of positive and negative sentiment is different (epstein, ), literature
suggests that this conclusion is not fully understood and may be case-
dependent.

##  3\. Data Pipeline

###  3.1. Data Collection

The data collection pipeline consists of two different components, namely of
the tweet data and of the Bitcoin price data.

####  3.1.1. Tweet data

Using the publicly available Twitter API, the tweets were acquired via a real-
time streamer (_stream-watcher_). All _relevant_ tweets between 10.10.2020 and
3.3.2021 were stored as _JSON line objects_ on an AWS MongoDB server. A tweet
was considered relevant, if its textual body contained one of the following
strings: “BTC”, “$BTC”, or “Bitcoin”. Such a tweet was automatically appended
to the database by the streamer.

####  3.1.2. Price Data

We also considered 15-minute closing prices in Bitfinex exchange for the
prediction task. The data was acquired using Cryptocurrency eXchange Trading
Library.

Overall, the dataset used in this study consisted of approximately 14000
Bitcoin price snapshots and 30 million Bitcoin-related tweets.

###  3.2. Data Preprocessing

The collected data was preprocessed, in order to reduce the memory footprint
and homogenize the dataset schema.

##### Refactoring

A JSON line object for a single tweet consists of nested objects that are
different for different types of tweets: a _general_ tweet, a _quote_ tweet, a
_retweet_ , or a _reply_ to a general tweet. Replies, quotes, and retweets
include an additional data entry that contains the original tweet’s content.
Therefore, these types of tweets may be larger in size than general tweets. To
make all types of tweets equivalent in our database and to reduce overall
redundancy, the fields, such as media and retweeted tweet’s body, were
converted to flags or counts, so that the refactored structure for all types
of tweets is identical.

##### Pruning

After the refactorization, the JSON line object consists of approximately 120
fields, including information about its location and source (i.e., URLs for
media-including tweets). For the study, some of these fields were redundant,
therefore they were excluded. The selected fields could be partitioned into
_user-related_ and _tweet-related_ fields. The schema of the pruned and
refactored tabular format is shown in Tab. 1.

##### User-related fields

Assuming that some data related to the author of the tweet could be
informative, we considered parts of it. In particular, hypothesizing that
popular accounts may have large influence, we used data fields that relate to
the popularity of a user.

##### Tweet-related fields

The previously mentioned tweet streamer collects the sample tweet upon its
creation. At the time, retweet count, favourites count, and quote count are
zero and they would not aid the predictions. Acknowledging this, we selected
tweet-related fields that describe the tweet’s content rather than its
popularity. These fields include number of media objects in the tweet, whether
it is a quote tweet or a retweet, whether it contains sensitive language, and,
finally, the main body of the tweet.

After refactoring and pruning, the entire dataset was sorted and indexed by
JSON line objects’ creation timestamps, after which the dataset was stored in
a Parquet format. The pipeline ensured the tweets are stored in a tabular
manner (rather than a tree structure), making columnar processing,
compression/decompression efficient.

Table 1. Pruned and refactored JSON object schema of the preprocessed Twitter
data.

User-related | Tweet-related
---|---
Name | Data type | Name | Data type
favourites_count | integer | created_at | timestamp
followers_count | integer | gif_count | integer
friends_count | integer | photo_count | integer
listed_count | integer | video_count | integer
verified | boolean | is_quote_status | boolean
default_profile | boolean | possibly_sensitive | boolean
default_profile_image | boolean | tweet_text | string

###  3.3. Semantic Embedding of Tweet Text

Each tweet’s text (tweet_text) was fed through a well-known sentiment tool,
namely Valence Aware Dictionary for Sentiment Reasoning (VADER) (vader, ),
implemented in NLTK. VADER was used to obtain an impression of tweet author’s
stance towards the cryptocurrecy at the time of tweeting (created_at). One of
the several advantages that VADER provides with is multilingualism, therefore
the whole corpora of tweets need not be translated to one language. The VADER
output, a four-dimensional vector, contains sentiment scores for positive,
neutral, and negative emotions, and the compound score. The compound score is
not a simple linear sum of the three raw sentiment scores; it also
incorporates these scores via a heuristic function, whose output is normalized
to range between $[-1,1]$. We used the compound score instead of the raw main
body text to represent the textual content of the tweet in the forecasting
model.

##  4\. Prediction Task

Incorporating semantic information, tweet and user metadata into a single
realized volatility model is a challenging task in and of itself. Although
numeric data can be directly employed via, e.g., linear, autoregressive models
(time_series, ) given the stationarity of the inputs, the task is more
challenging when multiple data types are considered, such as text and other
media.

##### Target

We aim to determine whether Twitter data, alongside with log-returns, can be
used to predict the Bitcoin price volatility. Log-returns are defined as the
change in logarithm of price as observed at two consecutive timesteps, namely,

(1) $r_{t}=\log{P_{t}}-\log{P_{t-1}}.$

_Realized volatility_ is an empirical measure of return variability, defined
as the square-root of summed squared log-returns in a time period $\Delta t$
(realized_volatility, ):

(2) $\displaystyle RV_{\Delta t}$ $\displaystyle=\sqrt{\sum_{t\in\Delta
t}r_{t}^{2}}.$

The _target_ for all models is the next day’s realized volatility, obtained
from 96 segments of 15-minute log-return predictions, and aggregated using
standard formula (2) for realized (future) volatility (ARCH_HAR, ).

##### Evaluation Metrics

We considered a set of standard metrics to compare the real $RV$ with the
predicted volatility, and to provide a single score that displays the quality
of the model under certain assumptions. First metric used was the _mean
absolute percentage error_ (MAPE), which is the mean absolute deviation of the
predicted volatility value from the actual realized volatility. Secondly, the
_mean absolute error (MAE)_ , which evaluates the mean absolute deviation in
terms of the units of the actual realized volatility. In addition, _root mean
square error (RMSE)_ was used to examine how the selected loss functions
influenced the result. Since MSE is a staple loss function for regression
tasks, RMSE is one of the important metrics that indicates the performance-
loss relationship. Lastly, we considered _mean squared logarithmic error
(MSLE)_ , often used to see the forecasting performance on the logarithmic
scale. Detailed equations for the metrics can be found in Appendix A.

##  5\. Deep Learning Volatility Forecasting

As the first step our study, we studied deep learning architectures that may
surpass the classical financial models used in the literature for volatility
forecasting.

As a general setup of forecasting models, historic data is used to predict the
next value of a target variable. Typically, volatility is predicted based on
feature vectors that relate to past price returns, e.g., as is done in GARCH
(nino_garch, ), or to volatility, as in (H)AR-RV (ARCH_HAR, ). In this part of
the study, we considered GARCH and AR-RV models as _classic econometric
baselines_ , and compared their performance to deep learning models that
utilize the same financial information.

###  5.1. Time-series forecasting with deep learning

RNN-based models are popular for financial applications. They have also been
successfully explored as models that utilize Twitter data to analyze the
cryptocurrency market (rel_3_cnn_lstm_crypto, ; rel_9_comp_study, ). However,
convolutional networks have consistently shown similar or better results in a
variety of tasks with fewer number of parameters (RNNvsTCN, ). RNNs also seem
to have a shorter effective memory than their convolutional counterparts
(RNNvsTCN, ). Despite theoretically infinite receptive fields, RNNs suffer
from inherent exploding/vanishing gradient problems, and are susceptible to
neuron saturation because of the sigmoid activations. In order to select the
most appropriate deep learning architecture that can be further expanded to
incorporate social media data in the incoming tasks, we first compared LSTM,
GRU and TCN using solely the price data as the input. As the autoregressive
models have already proven to be remarkably successful in volatility
forecasting (ARCH_HAR, ), we expect deep architectures to at least perform as
well as these baselines.

RNN-based models and TCN-based models that were considered in this study
utilize similar design, illustrated in Fig. 1. The receptive field of all
models is fixed for a single batch, and the inputs are directly fed to the
respective time-series models without any preceding layers. Both input and
output features are in log-return space, and the metric scores are calculated
after aggregating the 15 minute-level output predictions for a day to estimate
the daily volatility using (2). The losses are directly estimated from the
true log-returns (rather than realized volatilities) to provide more fine-
grained gradient estimates to the network.

Figure 1. Baseline TCN model. Causal convolution over time with bottleneck
interpolator. For RNN-based models, the TCN block is replaced by LSTM or GRU
block.

##### TCN mechanism

TCNs were introduced in order to capture temporal information using
1-dimensional causal convolutions that avoid information leakage from the
future time steps (tcn_paper, ; stcn, ). However, without the reduction of
dimensionality in subsequent layers, convolution operation becomes very costly
for long input signals. To bypass this potential limitation, dilated
convolutions for long-history dependent sequence tasks were proposed (wavenet,
; RNNvsTCN, ). For a 1-dimensional signal x at time $t$ of a sequence $s$ to
be mapped to values y, and a filter $f$, one can define the dilated
convolution operation $F$ as follows:

(3) $F(s)=(\textbf{x}*_{d}f)(s)=\sum_{i=0}^{k-1}f(i)\textbf{x}_{s-d\times i}$

with $d$ the dilation factor, $k$ the kernel (filter) size, and $s-d\times i$
accounting for the signals acquired earlier in the sequence (RNNvsTCN, ). It
is important to note that for $d=1$, (3) is equal to a standard 1-dimensional
convolution.

##### Bottleneck layer

Noting that encoder-decoder architectures were more prone to overfitting on
high-frequency artefacts in our data, we relied on a simpler decoding scheme
with a linear layer. This design choice ensured the model focuses more on the
low-frequency components of the output signal while providing the desired
output variability. Another potential explanation for the effectiveness of
this approach is that the linear interpolation estimates intraday stationarity
at no cost, therefore allowing the TCN to train on variance-inducing
components of the input data.

##### Issue of mean collapse in training

An obvious local minimum of this task is the constant returns that can explain
a quasi-linear trend of increase in Bitcoin prices. Mean squared error loss,
which is quite often used in the regression tasks in training neural networks,
is observed to trap the network in this minimum as small log-returns dominate
the loss landscape and prevent the network from learning about the meaningful
changes. To resolve this issue, we resorted to squared epsilon insensitive
loss that ignores the prediction error when the value is “close enough”:

(4)
$\displaystyle\mathcal{L}_{\epsilon}(r_{t},\hat{r}_{t})=\max\\{0,(r_{t}-\hat{r}_{t})^{2}-\epsilon\\}.$

Here $r_{t}$ stands for the log-return at the timepoint $t$, $\hat{r}_{t}$ is
the prediction of the model for the same value, and $\epsilon$ is the upper
end of the interval $[0,\epsilon]$ in which the loss is ignored.

Table 2. Performances of the autoregressive baselines and deep learning models
across 20 runs.

| MAPE | MAE | RMSE | MSLE
---|---|---|---|---
AR-RV | 0.32 | 1.91 | 2.78 | 0.12
GARCH | 0.35 | 2.21 | 2.93 | 0.14
LSTM | 0.30 ± 0.08 | 1.70 ± 0.26 ** | 2.37 ± 0.19 *** | 0.09 ± 0.01 ***
GRU | 0.57 ± 0.40 | 2.92 ± 2.3 | 4.23 ± 5.61 | 0.20 ± 0.20
TCN | 0.23 ± 0.01 *** | 1.63 ± 0.06 *** | 2.58 ± 0.07 *** | 0.11 ± 0.01 ***

For the deep learning models: One-sided t-test significances with respect to
AR-RV
Significance codes: 0 - ‘***’ - 0.001 - ‘**’ - 0.01 - ‘*’ - 0.05 - ‘.’ - 0.1

###  5.2. Experiment Setup

The experimental setup for the autoregressive models and the deep learning
architectures have a common training horizon of 96 days and a testing horizon
of 48 days. AR-RV model uses daily RV values as inputs, whereas GARCH model
uses log-returns. The input data is scaled to reside between $[-0.25,0.25]$ by
subtracting the minimum from the data and dividing the result with the data
range, extracted from the training set. In all subsequent experiments, daily
realized volatility is predicted using the available information from the
previous day, and training is conducted in 1-day strides.

##### Hyperparameter optimization

Deep learning models often suffer from bad hyper-parametrization and the
determination of such parameters can be a critical source of bias. In order to
prevent any imbalance of attention from affecting the results of the
experiment, we conducted thorough hyperparameter searches for each model with
predetermined resources. Given the $72-24$ day split of the training data into
train and validation sets, all three models were tuned to minimize MAPE metric
with a multivariate Bayesian optimizer for 250 different configurations using
Python library optuna (optuna, ). The optimized parameters include the
individual components of each model such as recurrent unit count and kernel
size, as well as common parameters such as the learning rate and the loss
epsilon $\epsilon$. The epoch count is fixed to 30 for all configurations.
Full details and results of the optimization process can be found in Appendix
B.

All experiments were conducted on Nvidia Titan X GPUs with Pascal architecture
wherever possible for the deep learning models. Autoregressive baseline models
were evaluated using CPU. Each non-deterministic model was trained 20 times,
and the results were compared using the appropriate one-sided statistical
tests. For all models, we used AdamW optimizer (adamw, ) with optimal learning
rate and weight decay as found by the hyperparameter optimization.

(a) AR-RV and GARCH

(b) LSTM

(c) GRU

(d) TCN

Figure 2. Comparison of the ground truth test time series and predictions
obtained using a) AR-RV and GARCH, b) LSTM, c) GRU, and d) TCN. For subfigures
b-d, shaded blue areas indicate 95% confidence intervals, estimated via
bootstrapping (using 1000 bootstrap samples).

###  5.3. Results

We compared the performances of the econometric baselines, RNN, TCN, GRU, and
LSTM-based models over the 48 days testing horizon. Here we primarily focus on
MAPE metric, as it ensures that we do not inflate errors due to high-
volatility data points (such high-volatility incidents in Bitcoin data are
common and are often caused by exogenous and highly unpredictable events).

The autoregressive baseline results in Tab. 2 indicate the superiority of AR-
RV over GARCH based on all analysed metrics, therefore we selected AR-RV as
the baseline to compare the deep learning models to. We conducted one-sided
t-tests with respect to the AR-RV results to test if the mean of an error for
a given model is statistically different from the error achieved by the AR-RV.

Tab. 2 shows that TCN model significantly outperforms the baseline AR-RV in
all metrics, whereas GRU performs worse than both of the autoregressive
models. LSTM is not significantly better in terms of MAPE and shows high
levels of standard deviation in different runs. Among the deep learning
models, TCN is also superior in MAPE and MAE metrics (recall that these
metrics are outlier-robust), yet LSTM appears to outperform TCN in MSLE and
RMSE metrics. However, it should also be noted that MSLE is not a symmetric
metric, as under-predictions are punished more than over-predictions, and
might mislead in comparing the accuracy of the two models.

In Fig. 2, we show predictions obtained by each model for a 48 day test set.
Based on the confidence intervals achieved by the neural network models, we
observe that TCN model is much more stable than RNN-based models. TCN
consistently provides better estimates on low-volatility days. The figure also
suggests that RMSE performance of LSTM can be due to overall higher mean of
its predictions that improves the resulting performance metrics at the extreme
data points, such as the three prominent peaks at 28.01, 07.02 and 22.02.

##### Bias distribution of the bottleneck layer

As per architecture shown in Fig. 1, the final layer of the TCN explicitly
models the output distribution in a way that can be visualized as the expected
log-return value for each 15-minute period in one day. The distribution of the
bias values, shown in Fig. 3, appears stationary with two significant downward
spikes at 5:00 and 20:30. In an attempt to explain these phenomenon, we have
annotated the market closing times of the two countries, USA and Japan, that
houses three biggest stock markets in the world, namely NYSE, NASDAQ and Japan
Exchange Group. However, we do not believe that the explanation of these
artefacts falls within the scope of our study, and invite the researchers to
analyse or discard this phenomenon.

Figure 3. TCN bottleneck layer bias distribution (actual scale). The values
are in the log-return space. Time of day is reported in UTC.

Following the results of the experiments, we conclude that TCN is a promising
candidate model for analysis and prediction of Bitcoin volatility. Therefore,
in the following section where we also consider Twitter data, we build our
models upon the TCN base model.

##  6\. Incorporating Twitter Data with TCN

A statistical assessment of the effectiveness of Twitter data requires control
of all other factors that might induce additional variance. For this purpose,
we are proposing a modular architecture with two temporal convolutions and
flexible concatenation layers as depicted in Fig. 4. This model can be used to
perform log-return predictions for different sets of data with minimal change
to parameter count at and after the points of connection. We dub this model
_Double-TCN_ (D-TCN), for the usage of two fully separate TCN pipelines in the
architecture.

Figure 4. D-TCN model. A modular architecture with two temporal convolutions
and flexible concatenation layers.

The upper TCN pipeline is identical to the model illustrated in Fig. 1, and,
as before, its purpose is to utilize previous day’s log-return values. The
lower TCN pipeline, on the other hand, analyzes any additional numeric data
obtained from tweets. The preprocessed Twitter data (see Section 3.2) is
aggregated to 15-minute intervals that correspond to the 15-minute intervals
of log-returns. The data is then fed into a Dense+ReLU stack before the TCN.
Regardless of the number of the input features, the input dimension of the TCN
layer is fixed across all experiments.

##### Single versus double TCN

We propose this D-TCN design both for performance and comparability: removal
and addition of features is easy, and done in a way that preserves the
dimensions in the rest of the network. On the contrary, in a single TCN
structure the log-returns and Twitter data are concatenated and fed to the
same temporal convolutional network. This may lead to a sharp performance
drop, likely because of inherent noise of the tweet-based data. Therefore, a
prior dimension reduction is required to gather meaningful evidence of
volatility instead of a further parametric expansion of the network.
Furthermore, comparison of different features become an intractable task, as
the network itself changes drastically to the level that the information
passed down from the log-returns becomes inseparable from the tweet-based
data.

###  6.1. Feature Sets

We split the pruned and refactored tweet data features indicated in Tab. 1
into Tweet, User and VADER feature sets (the VADER compound score of the
tweet_text from Tab. 1). Additionally, we aggregated the tweet counts within
each 15-minute interval as Count feature. The split for Tweet features and
User features is listed in Tab. 3. Except for the Count, all of the features
are the averages of the tweets in their respective intervals.

Table 3. Feature sets of Twitter data

Tweet | User | VADER | Count
---|---|---|---
gif_count | favourites_count | VADER | Total
photo_count | followers_count | compound | number
video_count | friends_count | score | of tweets
is_quote_status | listed_count |  |
possibly_sensitive | verified |  |
| default_profile |  |
| default_profile_image |  |
Averaged over set of tweets in the interval |

##### Tweet features

The tweet features in our feature set uses the media information contained in
the tweet by enumerating the number of objects contained in the tweet body,
such as GIFs, photos, and videos. is_quote_status is the saved tweet being a
reply, retweet, or quote retweet of a different tweet, and possibly_sensitive
indicates if the tweet contains sensitive language or not.

##### User features

We use favourites count (number of tweets the user has marked as favourite),
followers count (number of followers the user has), friends count (number of
accounts the user follows), listed count (number of lists the user is a member
of), verified (whether the user is verified or not) and finally default
profile flags indicating whether the user has altered his image and background
image in the profile page.

Table 4. Performances of D-TCNs with different feature sets across 40 runs.

| MAPE | MAE | RMSE | MSLE
---|---|---|---|---
TCN | 0.23 ± 0.01 | 1.63 ± 0.06 | 2.58 ± 0.07 | 0.11 ± 0.01
$\text{D-TCN}_{\text{User}}$ | 0.20 ± 0.01 *** | 1.44 ± 0.03 *** | 2.36 ± 0.05 *** | 0.08 ± 0.01 ***
$\text{D-TCN}_{\text{Tweet}}$ | 0.21 ± 0.01 *** | 1.59 ± 0.08 *** | 2.45 ± 0.09 ** | 0.09 ± 0.01 **
$\text{D-TCN}_{\text{VADER}}$ | 0.26 ± 0.02 | 1.72 ± 0.08 | 2.70 ± 0.09 | 0.13 ± 0.01
$\text{D-TCN}_{\text{Count}}$ | 0.32 ± 0.05 | 1.97 ± 0.19 | 3.27 ± 0.47 | 0.16 ± 0.02
$\text{D-TCN}_{\text{VADER, Tweet, User}}$ | 0.21 ± 0.01 *** | 1.47 ± 0.05 *** | 2.38 ± 0.06 *** | 0.09 ± 0.01 ***
$\text{D-TCN}_{\text{VADER, Tweet}}$ | 0.22 ± 0.01 | 1.56 ± 0.09 | 2.51 ± 0.09 | 0.10 ± 0.01
$\text{D-TCN}_{\text{VADER, User}}$ | 0.22 ± 0.01 ** | 1.51 ± 0.05 *** | 2.43 ± 0.09 *** | 0.10 ± 0.01 **
$\text{D-TCN}_{\text{Tweet, User}}$ | 0.21 ± 0.01 *** | 1.49 ± 0.06 *** | 2.41 ± 0.08 *** | 0.09 ± 0.01 ***
$\text{D-TCN}_{\text{Count, Tweet, User}}$ | 0.22 ± 0.01 *** | 1.46 ± 0.05 *** | 2.36 ± 0.10 *** | 0.08 ± 0.01 ***
$\text{D-TCN}_{\text{Count, Tweet}}$ | 0.23 ± 0.01 | 1.59 ± 0.08 | 2.56 ± 0.11 | 0.11 ± 0.01
$\text{D-TCN}_{\text{Count, User}}$ | 0.22 ± 0.01 | 1.53 ± 0.08 * | 2.46 ± 0.11 * | 0.10 ± 0.01
$\text{D-TCN}_{\text{Count, VADER, Tweet, User}}$ | 0.22 ± 0.01 * | 1.48 ± 0.07 *** | 2.43 ± 0.10 ** | 0.10 ± 0.01 **
$\text{D-TCN}_{\text{Count, VADER, Tweet}}$ | 0.24 ± 0.02 | 1.60 ± 0.07 | 2.59 ± 0.09 | 0.11 ± 0.01
$\text{D-TCN}_{\text{Count, VADER, User}}$ | 0.22 ± 0.01 ** | 1.47 ± 0.07 *** | 2.39 ± 0.10 *** | 0.10 ± 0.01 ***
$\text{D-TCN}_{\text{Count, VADER}}$ | 0.29 ± 0.02 | 1.82 ± 0.08 | 2.95 ± 0.18 | 0.15 ± 0.01

For D-TCNs: One-sided Welch’s t-test significances with respect to TCN.
Significance codes: 0 - ‘***’ - 0.001 - ‘**’ - 0.01 - ‘*’ - 0.05 - ‘.’ - 0.1

###  6.2. Experiment Setup

The setup is the same as in Section 5.2. The model is fitted separately for
each subset of the feature sets, listed in Tab. 3. Differently from Section 5,
we train and test each model 40 times to gather more accurate estimates of the
expected performance in each case.

##### Hyperparameters

A separate hyperparameter optimization is not conducted for the D-TCN setups
to ensure comparability of the results with the baseline and also with each
other. We used the same parameters as noted in Appendix B for both TCNs, and
reduced the input dimensionality of the features to four via the Dense layer
in the lower TCN.

###  6.3. Results

Here we compare the results of a single TCN (our _baseline_ here), analyzed in
Section 5, to the results of the D-TCN models trained with different feature
subsets from Tab. 3. The distribution of fits per each metric and model are
compared with the baseline using a one-sided Welch’s t-test. The results are
reported in Tab. 4.

In a comparative feature analysis, it is important to consider the value of
some new information in the context of a bias-variance problem. The additional
information provided with each good feature set is expected boost the
performance of the model, while the variance introduced by the increased set
of parameters is expected to decrease the performance. Therefore, we assess
the features not only by the potential explanation they provide, but also with
the noise content they possess.

####  6.3.1. Performance of individual feature sets

We observe that only two feature sets, when used individually, performs better
the baseline TCN, namely User and Tweet. The former feature set both surpasses
the baseline TCN performance ($p<0.001$) and on average creates the best model
in our study. The Tweet feature set also boosts the baseline results (with
$p<0.001$ in MAPE and MAE, $p<0.01$ in RMSE and MSLE). In Fig. 5, we further
observe that the predictions of Count and VADER feature sets have a tendency
to over-value in some days, severely hurting the aggregate metrics in the
displayed results. As we observe full convergence in all models during
training, these artifacts can be regarded as the effects of overfitting on the
training data.

Figure 5. Mean predictions of the D-TCNs with single feature sets from Tab. 3:
_Count_ in blue dashed line, _VADER_ in green dotted line, _User_ in purple
dash-dot line, and _Tweet_ in orange hyphen-dot line.

####  6.3.2. Performance of collections of feature sets

The best performing models based on the significance levels and metric results
were trained with the User feature set. However, the feature sets VADER and
Count, while performing poorly by themselves, experience a performance boost
when combined with other feature sets. Examples of such cases are the
performance obtained by the $\text{D-TCN}_{\text{VADER, Tweet, User}}$ and
$\text{D-TCN}_{\text{Count, Tweet, User}}$. Although individually the Tweet
and User sets are superior to others, when combined together they benefit from
the addition of VADER in MAE metric, and Count in RMSE and MSLE metrics. These
performance changes, albeit not very significant, might be indications that
with proper architectures, the interaction effects between the metadata
information and other data sources such as semantic content and communication
volume can be utilized to improve the results in volatility prediction.

(a) $\text{D-TCN}_{\text{User}}$

(b) $\text{D-TCN}_{\text{Tweet}}$

(c) $\text{D-TCN}_{\text{Count, User, Tweet}}$

(d) $\text{D-TCN}_{\text{VADER, User, Tweet}}$

Figure 6. Predicted vs. true realized volatilities on test sets. Shaded
regions stand for the 95% confidence intervals, estimated via bootstrapping
(using 1000 bootstrap samples).

####  6.3.3. Performances of best combinations at different label quantiles

Since performance metrics relate to an average achieved performance across the
full dataset, we further look into behaviors of D-TCN models for different
target volatilities. In Fig. 7, we observe that despite the good performance
of the User feature set in the aggregate metrics, it does not provide absolute
performance in all target percentiles: Tweet feature set is more useful in the
lowest percentile, while the addition of Count appears to improve the MAPE
metric in higher RV periods. Tweet volume has been previously connected with
high volatility in Bitcoin prices (count_and_volatility, ), and our results
corroborate with this insight. It is also worthwhile to note that when
compared to TCN, $\text{D-TCN}_{\text{User}}$ has better performance in the
second, third, and fourth quantiles of the true RV, conveying again that peaks
are more precisely forecasted with the introduction of user information.

(a) Best D-TCN models

(b) Classical vs. deep models

Figure 7. Mean MAPE values of each model for the given percentile of the true
labels. Bands stand for 95% confidence intervals.

##  7\. Conclusion

In this study, our aim was threefold: (i) to analyze the deep learning-based
alternatives to standard econometric models used for realized volatility
forecasting; (ii) to design and develop a deep learning architecture for
realized volatility forecasting that can utilize social media data; and (iii)
to understand the social media factors that help in predicting realized
volatility.

To achieve our goals, we employed Twitter API in a _stream-watcher_ fashion,
so that the tweets which contained Bitcoin-related strings were saved
instantly. We then utilized the obtained textual and other accompanying
information of Bitcoin-related tweets and used it together with intraday
Bitcoin price data in the prediction task. The gathered data was first
preprocessed by pruning and refactoring, so that all social media data entries
had the same format, and contained only that information that could
potentially aid the realized volatility forecast.

The double-TCN architectural design was developed via a thorough three-stage
analysis. First, by comparing several deep learning models (namely, LSTM, GRU,
and TCN) to econometric baseline models, we observed that TCN architecture is
the most promising. Following this step, we built a two-level TCN model which
allows for concurrent use of log-returns and information obtained from social
media as its inputs. Finally, to assess the influence of the information
added, we trained the D-TCN with combinations of different feature sets and
compared the results obtained from each combination with the TCN which was
trained using only the log-returns.

Overall, our experiments produced an unexpected outcome: in contrast with what
would be intuitive, semantic content of the tweet’s text is _not_ the most
informative for the predictions of realized volatility. On the contrary,
information about authors of the tweets (e.g., follower counts and friend
counts) convey much more predictive power. Feature sets that combined User
information and Tweet information together with information about the total
tweet volume (Count) were more predictive than Count would be on its own,
particularly in high realized volatility regimes.

As future work, we believe that a higher prediction accuracy could be achieved
by forming a dataset that uses tweet information after a pre-dedicated
duration so that interaction statistics regarding retweet count, favourite
count and reply count for individual tweets can also be used as additions to
Tweet feature set. Incorporating a more complex embedding structure like BERT
(devlin2019bert, ) instead of VADER may also improve the precision of realized
volatility forecasting.

## References

  * [1] Michel Rauchs and Garrick Hileman.  Global Cryptocurrency Benchmarking Study.  Cambridge Centre for Alternative Finance, Cambridge Judge Business School, University of Cambridge, 2017.
  * [2] Triple-A.  Crypto ownership table, 2020.
  * [3] Yuexin Mao, Wei Wei, Bing Wang, and Benyuan Liu.  Correlating S&P 500 stocks with Twitter data.  In Proceedings of the First ACM International Workshop on Hot Topics on Interdisciplinary Social Networks Research, HotSocial ’12, page 69–72, New York, NY, USA, 2012. Association for Computing Machinery.
  * [4] Svitlana Galeshchuk, O. Vasylchyshyn, and Andriy Krysovatyy.  Bitcoin response to Twitter sentiments.  In ICTERI Workshops, 2018.
  * [5] Ioannis Livieris, Niki Kiriakidou, Stavros Stavroyiannis, and P. Pintelas.  An advanced CNN-LSTM model for cryptocurrency forecasting.  Electronics, 01 2021.
  * [6] Thársis T.P. Souza and Tomaso Aste.  Predicting future stock market structure by combining social and financial network information.  Physica A: Statistical Mechanics and its Applications, 535:122343, Dec 2019.
  * [7] Juan Piñeiro, M. López-Cabarcos, Ada M. Pérez-Pico, and Belén Ribeiro-Navarrete.  Does social network sentiment influence the relationship between the S&P 500 and gold returns?  International Review of Financial Analysis, 57, 02 2018.
  * [8] Jia-Yen Huang and Jin-Hao Liu.  Using social media mining technology to improve stock price forecast accuracy.  Journal of Forecasting, 39(1):104–116, 2020.
  * [9] Dehua Shen, Andrew Urquhart, and Pengfei Wang.  Does Twitter predict Bitcoin?  Economics Letters, 174:118–122, 2019.
  * [10] Dibakar Raj Pant, Prasanga Neupane, Anuj Poudel, Anup Kumar Pokhrel, and Bishnu Kumar Lama.  Recurrent neural network based bitcoin price prediction by twitter sentiment analysis.  In 2018 IEEE 3rd International Conference on Computing, Communication and Security (ICCCS), pages 128–132, 2018.
  * [11] Suhwan Ji, Jongmin Kim, and Hyeonseung Im.  A comparative study of Bitcoin price prediction using deep learning.  Mathematics, 7:898, 09 2019.
  * [12] S. Howison, Avraam Rafailidis, and Henrik Rasmussen.  On the pricing and hedging of volatility derivatives.  Applied Mathematical Finance, 11:317–346, 12 2004.
  * [13] Irena Barjašić and Nino Antulov-Fantulin.  Time-varying volatility in bitcoin market and information flow at minute-level frequency.  Frontiers in Physics, 9:273, 2021.
  * [14] Alla A. Petukhina, Raphael C. G. Reule, and Wolfgang Karl Härdle.  Rise of the machines? Intraday high-frequency trading patterns of cryptocurrencies.  The European Journal of Finance, 27(1-2):8–30, Jul 2020.
  * [15] C. Hutto and Eric Gilbert.  Vader: A parsimonious rule-based model for sentiment analysis of social media text.  Proceedings of the International AAAI Conference on Web and Social Media, 8(1):216–225, May 2014.
  * [16] Robert F Engle.  Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation.  Econometrica: Journal of the econometric society, pages 987–1007, 1982.
  * [17] Tim Bollerslev.  Generalized autoregressive conditional heteroskedasticity.  Journal of econometrics, 31(3):307–327, 1986.
  * [18] Fulvio Corsi.  A simple long memory model of realized volatility.  Journal of Financial Econometrics, 7:174–196, 02 2009.
  * [19] Daniele Mastro.  Forecasting realized volatility: Arch-type models vs. the har-rv model.  SSRN Electronic Journal, 01 2014.
  * [20] Yang Liu.  Novel volatility forecasting using deep learning–long short term memory recurrent neural networks.  Expert Systems with Applications, 132:99–109, 2019.
  * [21] Nghia Nguyen, Minh-Ngoc Tran, David Gunawan, and Robert Kohn.  A long short-term memory stochastic volatility model.  arXiv preprint arXiv:1906.02884, 2019.
  * [22] Sean McNally, Jason Roche, and Simon Caton.  Predicting the price of Bitcoin using machine learning.  In 2018 26th euromicro international conference on parallel, distributed and network-based processing (PDP), pages 339–343. IEEE, 2018.
  * [23] Muhammad Saad, Jinchun Choi, DaeHun Nyang, Joongheon Kim, and Aziz Mohaisen.  Toward characterizing blockchain-based cryptocurrencies for highly accurate predictions.  IEEE Systems Journal, 14(1):321–332, 2019.
  * [24] Qiutong Guo, Shun Lei, Qing Ye, and Zhiyang Fang.  MRC-LSTM: A hybrid approach of multi-scale residual CNN and LSTM to predict Bitcoin price, 2021.
  * [25] Adam Hayes.  Cryptocurrency value formation: An empirical analysis leading to a cost of production model for valuing Bitcoin.  SSRN Electronic Journal, 2015.
  * [26] Wilko Bolt.  On the value of virtual currencies.  SSRN Electronic Journal, 2016.
  * [27] Saralees Nadarajah and Jeffrey Chu.  On the inefficiency of Bitcoin.  Economics Letters, 150:6–9, 2017.
  * [28] Eng-Tuck Cheah and John Fry.  Speculative bubbles in Bitcoin markets? An empirical investigation into the fundamental value of Bitcoin.  Economics Letters, 130:32–36, 2015.
  * [29] Ladislav and Kristoufek.  What are the main drivers of the Bitcoin price? Evidence from wavelet coherence analysis.  PloS One, 10(4):e0123923, 2015.
  * [30] Thomas Dimpfl and Franziska J Peter.  Nothing but noise? price discovery across cryptocurrency exchanges.  Journal of Financial Markets, 54:100584, 2021.
  * [31] Jonathan Donier and Jean-Philippe Bouchaud.  Why do markets crash? Bitcoin data offers unprecedented insights.  PloS One, 10:1–11, 2015.
  * [32] Spencer Wheatley, Didier Sornette, Tobias Huber, Max Reppen, and Robert N. Gantner.  Are Bitcoin bubbles predictable? Combining a generalized Metcalfe’s law and the LPPLS model, 2018.
  * [33] Paraskevi Katsiampa.  Volatility estimation for Bitcoin: A comparison of GARCH models.  Economics Letters, 158:3–6, 2017.
  * [34] Tian Guo, Albert Bifet, and Nino Antulov-Fantulin.  Bitcoin volatility forecasting with a glimpse into buy and sell orders.  In 2018 IEEE International Conference on Data Mining (ICDM), pages 989–994, Nov 2018.
  * [35] Nino Antulov-Fantulin, Tian Guo, and Fabrizio Lillo.  Temporal mixture ensemble models for probabilistic forecasting of intraday cryptocurrency volume.  Decisions in Economics and Finance, pages 1–36, 2021.
  * [36] Dorit Ron and Adi Shamir.  Quantitative analysis of the full Bitcoin transaction graph.  In International Conference on Financial Cryptography and Data Security, pages 6–24. Springer, 2013.
  * [37] Abeer ElBahrawy, Laura Alessandretti, Anne Kandler, Romualdo Pastor-Satorras, and Andrea Baronchelli.  Evolutionary dynamics of the cryptocurrency market.  Royal Society Open Science, 4(11):170623, 2017.
  * [38] Nino Antulov-Fantulin, Dijana Tolic, Matija Piskorec, Zhang Ce, and Irena Vodenska.  Inferring short-term volatility indicators from the Bitcoin blockchain.  In Complex Networks and Their Applications VII, pages 508–520, Cham, 2019. Springer International Publishing.
  * [39] David Garcia and Frank Schweitzer.  Social signals and algorithmic trading of Bitcoin.  Royal Society Open Science, 2(9):150288, 2015.
  * [40] Asim K. Dey, Cuneyt G. Akcora, Yulia R. Gel, and Murat Kantarcioglu.  On the role of local blockchain network features in cryptocurrency price formation.  Canadian Journal of Statistics, 48(3):561–581, 2020.
  * [41] Johannes Beck, Roberta Huang, David Lindner, Tian Guo, Zhang Ce, Dirk Helbing, and Nino Antulov-Fantulin.  Sensing social media signals for cryptocurrency news.  In Companion proceedings of the 2019 World Wide Web conference, pages 1051–1054, 2019.
  * [42] Larry G. Epstein and Martin Schneider.  Ambiguity, information quality, and asset pricing.  The Journal of Finance, 63(1):197–228, 2008.
  * [43] James D. Hamilton and Princeton University Press.  Time Series Analysis.  Number 10. c. in Time Series Analysis. Princeton University Press, 1994\.
  * [44] Torben G. Andersen, Tim Bollerslev, Francis X. Diebold, and Paul Labys.  Modeling and forecasting realized volatility.  Econometrica, 71(2):579–625, 2003.
  * [45] Shaojie Bai, J Zico Kolter, and Vladlen Koltun.  An empirical evaluation of generic convolutional and recurrent networks for sequence modeling.  arXiv preprint arXiv:1803.01271, 2018.
  * [46] Colin Lea, Rene Vidal, Austin Reiter, and Gregory D Hager.  Temporal convolutional networks: A unified approach to action segmentation.  In European Conference on Computer Vision, pages 47–54. Springer, 2016.
  * [47] Emre Aksan and Otmar Hilliges.  STCN: Stochastic temporal convolutional networks.  arXiv preprint arXiv:1902.06568, 2019.
  * [48] Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu.  Wavenet: A generative model for raw audio.  arXiv preprint arXiv:1609.03499, 2016.
  * [49] Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama.  Optuna: A next-generation hyperparameter optimization framework.  In Proceedings of the 25rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2019.
  * [50] Ilya Loshchilov and Frank Hutter.  Decoupled weight decay regularization.  arXiv preprint arXiv:1711.05101, 2017.
  * [51] Martina Matta, Ilaria Lunesu, and Michele Marchesi.  Bitcoin spread prediction using social and web search media.  In UMAP workshops, pages 1–10, 2015.
  * [52] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  BERT: Pre-training of deep bidirectional transformers for language understanding.  arXiv preprint arXiv:1810.04805, 2018.
  * [53] James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl.  Algorithms for hyper-parameter optimization.  Advances in neural information processing systems, 24, 2011.

##  Appendix A Metrics

The following metrics are defined in terms of i) $y_{i}$, the actual realized
volatility on the $i$th day, and ii) $\hat{y}_{i}$, the predicted realized
volatility on the $i$th day. $N$ denotes the total number of predicted
datapoints in a given horizon.

(5)
$\displaystyle\textrm{MAPE}=\frac{1}{N}\sum_{i=1}^{N}\left|{\frac{y_{i}-\hat{y}_{i}}{y_{i}}}\right|$

(6) $\displaystyle\textrm{MAE}=\frac{1}{N}\sum_{i=1}^{N}|{y_{i}-\hat{y}_{i}}|\
$

(7)
$\displaystyle\textrm{RMSE}=\sqrt{\frac{1}{N}\sum_{i=1}^{N}(y_{i}-\hat{y}_{i})^{2}}$

(8)
$\displaystyle\textrm{MSLE}=\frac{1}{N}\sum_{i=1}^{N}(\log(y_{i}+1)-\log(\hat{y}_{i}+1))^{2}$

##  Appendix B Hyperparameter optimization

Python library optuna [49] is used to conduct hyperparameter optimization for
all three baseline deep learning models, namely, LSTM, GRU, and TCN. A
multivariate Tree-structured Parzen Estimator sampler (TPE) [53] is used in a
Bayesian setting to minimize the MAPE error on a training set of 72 days and a
validation set of 24 days (from the 96 days that is initially determined as
the full training set of the experiments). The epoch count is kept constant at
30. All models are optimized for 250 iterations. The parameter search spaces
and the best parameter values for LSTM/GRU and TCN are shown in Tab. 5 and
Tab. 6 respectively. Fig. 8 demonstrate the error versus trial count plot for
each model. Fig. 9 show the importance scores with respect to the objective
value (MAPE). As depicted in Fig. B, LSTM achieves 0.2588 MAPE on the
validation set at the best trial, GRU scores 0.2513, and TCN 0.1977.

Table 5. Hyperparameter search space for LSTM and GRU Parameter | Low | High | Best LSTM | Best GRU
---|---|---|---|---
Recurrent Dimension | 32 | 512 | 261 | 43
Dropout | 0 | 0.5 | 0.0237 | 0.0544
$\epsilon$ (Loss epsilon) | 0.01 | 0.1 | 0.0364 | 0.0431
Learning Rate | $10^{-7}$ | $10^{-2}$ | 0.00521 | 0.00973
Weight Decay | $10^{-9}$ | $10^{-2}$ | $9.25\times 10^{-7}$ | $1.59\times 10^{-8}$
Table 6. Hyperparameter search space for TCN Parameter | Low |  High | Best
---|---|---|---
Filter count | 32 |  512 | 287
Dropout | 0 |  0.5 | 0.217
$\epsilon$ (Loss epsilon) | 0.01 |  0.1 | 0.0913
Learning Rate | $10^{-7}$ |  $10^{-2}$ | $6.49\times 10^{-5}$
Weight Decay | $10^{-9}$ |  $10^{-2}$ | $5.93\times 10^{-6}$
Kernel Size | 2 |  6 | 5
Dilation rate | 2 |  4 | 4
Skip Connections | False |  True | True
Normalization | None |  One of Batch, Layer or Weight | None

(a) TCN

(b) LSTM

(c) GRU

Figure 8. Trial-error plots for TCN, LSTM and GRU

(a) TCN

(b) LSTM

(c) GRU

Figure 9. Importance charts of hyperparameters for TCN, LSTM and GRU

##  Appendix C Additional Results

In Section 6, the individual feature sets VADER, Count, User and Tweet were
presented at Fig. 5, and best performing four combinations were presented at
Fig. 6, leaving the results for 9 feature set combinations unseen. For
completeness, results for the remaining 9 feature set combination plots can be
observed in Figures 10 and 11.

(a) $\text{D-TCN}_{\text{VADER, User}}$

(b) $\text{D-TCN}_{\text{User, Tweet}}$

(c) $\text{D-TCN}_{\text{Count, User}}$

(d) $\text{D-TCN}_{\text{Count, Tweet}}$

Figure 10. Results of feature set combinations not presented in Section 6.

(e) $\text{D-TCN}_{\text{VADER,Count, Tweet}}$

(f) $\text{D-TCN}_{\text{VADER, Tweet}}$

(g) $\text{D-TCN}_{\text{VADER, Count, User}}$

(h) $\text{D-TCN}_{\text{VADER, Count}}$

(i) $\text{D-TCN}_{\text{Count, VADER, Tweet, User}}$

Figure 11. Continuation of results of feature set combinations not presented
in Section 6.