XAI

    explainability
CAV

    Concept Activation Vector
MIR

    Music Information Retrieval
NLP

    natural language processing

# Learning Unsupervised Hierarchies of Audio Concepts

###### Abstract

Music signals are difficult to interpret from their low-level features,
perhaps even more than images: e.g. highlighting part of a spectrogram or an
image is often insufficient to convey high-level ideas that are genuinely
relevant to humans. In computer vision, concept learning was therein proposed
to adjust explanations to the right abstraction level (e.g. detect clinical
concepts from radiographs). These methods have yet to be used for MIR.

In this paper, we adapt concept learning to the realm of music, with its
particularities. For instance, music concepts are typically non-independent
and of mixed nature (e.g. genre, instruments, mood), unlike previous work that
assumed disentangled concepts. We propose a method to learn numerous music
concepts from audio and then automatically hierarchise them to expose their
mutual relationships. We conduct experiments on datasets of playlists from a
music streaming service, serving as a few annotated examples for diverse
concepts. Evaluations show that the mined hierarchies are aligned with both
ground-truth hierarchies of concepts – when available – and with proxy sources
of concept similarity in the general case.

##  1 Introduction

Music signals are challenging to interpret [1]. For instance, inspecting a
spectrogram by parts – e.g. think of attention maps [2] – does not convey much
meaning with respect to the high-level descriptions that are useful to humans
– e.g. the mood of a song. Fortunately, a solution was proposed in a field
that shares similar problems: computer vision. As pixels of images are too
low-level to be understandable, concept learning was introduced to rationalise
the abstractions steps involved in a feed-forward model (e.g. detection of
colours, shapes, patterns, …), which in turn can be used to describe the
content of an image directly, and to align models with human reasonings
better. Indeed, this field has had successful applications in explainability
(XAI) – dissecting a model’s predictions into human-grounded concepts [3, 4,
5], interactive learning [6] or knowledge transfer between tasks [7]. However,
these methods have yet to be used for Music Information Retrieval (MIR).

Music has particularities that are not captured by the current formulation of
concept learning. For instance, let us consider genres as common musical
concepts we would like to detect from spectrograms. Genres are typically
blended as they influence or result from other genres (e.g. Punk Rock). By
contrast, most literature on concept learning has considered sets of few
identified and disentangled concepts, which does not apply to genres.
Moreover, one trend in MIR is to incorporate multiple types of descriptors
into a shared space (e.g. mood, genre, instruments [8, 9]), thus requiring
handling even more entanglements.

We thus propose to study a novel problem for concept learning that is relevant
for music: learning hierarchies of concepts. Indeed, a hierarchy enables
navigating thousands of non-independent concepts without cognitive overload by
uncovering their mutual relationships with a structure, thus rationalising
which concepts share the same class or are derived from one another. In
addition, the use of taxonomy – i.e. hierarchy – is common in musicology and
feels like a natural representation in this context [10, 11].

The problem we study is interesting for music representation learning because
research is often limited by data labelling, which is labour-intensive and can
be ambiguous or tied to a specific dataset. Meanwhile, concepts can be learned
with few shots, which open the doors to new sources of music descriptors. In
particular, we propose to leverage a dataset of playlists from a music
streaming service as a folksonomy of concepts. Playlists are often built with
a specific concept; there exists one for any genre, instrument, mood, and many
more niche ideas. Yet, there is no overall ground-truth organisation of
playlists, making their use cumbersome. That is why we propose to solve this
task by first adapting a method from concept learning research to learn music
concepts from playlists in a few-shot manner and then, building on top of this
method, automatically derive a hierarchy from the learned concepts.

The paper proceeds as follows: we provide background on concept and hierarchy
learning (section 2); we adapt the literature of concept-based explanations to
the realm of MIR (3); we present a novel way to organise the detected concepts
into a hierarchy in an unsupervised manner (4); finally, we experiment and
discuss our attempt to provide a new audio concepts hierarchy with mixed types
(5).

Figure 1: Overview of our method (a) CAVs learning (section 3.1); (b) t-SNE
projection of concepts $\mathcal{V}_{\textrm{genre}}$ learned on a small
dataset with 219 music genres; (c) resulting extracted hierarchy
$H(S(\mathcal{V}_{\textrm{genre}}))$. Nodes colours denote ground-truth
clusters. Experiments on a bigger concept dataset are conducted in section 5.

##  2 Preliminaries and related work

We first present the literature on concept learning and hierarchy mining, then
frame our problem through XAI to uncover common pitfalls to avoid when
interpreting data.

###  2.1 Concept learning

Concept learning has witnessed growing interest in recent years [4]. The
introduction of concepts stems from the inadequacy of usual explanation spaces
with human mental models of problems: tabular features, pixels, and words in
sentences are sometimes too low-level to build relevant explanations or to
interact with. This idea is far from novel from a purely technical standpoint:
a stream of papers aim to learn a set of sub-attributes in a fully supervised
manner (e.g. facial features [12], or animal attributes [13]), that are then
combined to predict some end-task labels, typically through a linear
regression to track the importance of each attribute. This idea has been
revamped recently for concept bottleneck learning [14] and prototypical part
learning [15]. In the same vein, disentanglement techniques [16] have sought
to realign latent representations to interpretable attributes. All those works
rely on the assumption that a somewhat complete set of useful concepts exists,
well-defined and rather disentangled, and that a corresponding labelled
dataset is readily available to train on.

This is rarely the case in practice. Kim et al. introduced CAVs for images [3]
and demonstrated that concepts could be learned reliably from few annotated
examples – technical details are provided in section 3.1. Going beyond the
previously few datasets annotated for concepts, this work has led to new and
diverse applications: skin lesion interpretations [17], emotion recognition
[18], interactivity [6, 19], automatic concept learning [20], batch
normalisation [21], sufficiency of explanations with shapley values [22, 23],
causal inference [24]. To our knowledge, there exists no application in the
music domain.

###  2.2 Unsupervised Hierarchy Mining

Using knowledge graphs and taxonomies is frequent leverage in MIR [25].
Unsupervised learning of hierarchy is, however, less so. Part of it lies in
the difficulty of adequately defining the meaning of the hierarchical
relations, learning the hierarchy in a tractable way, and evaluating the found
hierarchy without ground-truth.

Fortunately, there is active literature coming from  natural language
processing (NLP) research. Many work proposed variants of topic modelling [26]
to enable sampling hierarchised structures [27, 28]. However, these methods
get computationally intractable as the number of nodes, topics, and
hierarchical levels grows. Many successes thus came from scalable greedy
methods such as hierarchical clustering [29, 30], rule-mining [31, 32] and
graph-based analyses [33, 34]. Outside of NLP, hierarchies have been learned
based on the latent manifold geometry [35], predefined structure [36], and for
images [37, 38].

To our knowledge, these techniques have never been combined with concept
learning, as we propose.

###  2.3 Explainability

Our end goal is to interpret music signals, which is linked to explainability
(XAI). Before jumping into our method to learn a hierarchy of concepts, we
need to acknowledge that many works have debated the sanity of explanation
methods [39, 40, 41, 42, 43] and the fact that these methods largely disagree
with one another [44]. XAI has vast boundaries, which has also percolated to
its music sub-field for which there exists many explanation paradigms and
evaluations [45]. Explanations for MIR are thus also prone to exhibiting
pathologies that hamper knowledge mining: e.g. shortcut learning [46], lack of
clear definitions [39, 47], flawed evaluations [48], out-of-distribution
artifacts [49, 50] or misalignment with human reasonings [51, 52, 53, 54].

Uncovering these failure cases sharpens our understanding of how models make
decisions, and shows that we have to go beyond evasive considerations on
explainability to make them relevant [41] since trustworthy AI is not achieved
automatically with XAI [55]. One taxonomic distinction we would like to
underline is whether methods aim to explain models or data [56]. Our method is
intended to belong to the latter category for which the purpose is not to
explain how models work (e.g. troubleshooting learned filters) but rather to
use them as proxies to gain new insights about reality (e.g. causal inference
[57]).

With all these aspects in mind and based on previous literature, we formalise
our three explanation goals:

  1. 1.

Attribution. Identify musical concepts associated with a given track’s audio
signal.

  2. 2.

Transferability. Detected concepts should be applicable to various settings
and tasks.

  3. 3.

Generality. The hierarchy should be independent of the signals and make sense
for various settings.

Our first goal is linked to the traditional explanation literature supporting
transparency [47, 45] and is common in concept-learning. We have defined our
second and third goals to stress that the finality of our work is to be as
universal as possible.

##  3 Learning music concepts

In this section, we first adopt the few-shot supervision setting of concept
learning to learn the music concept $\vec{v}_{C}$ of each playlist, viewed as
a set $C$ of audio signals examples.

###  3.1 Background on CAVs

We review some essential notions from the concept learning framework we use to
learn playlist concepts [3].

We denote $\vec{f}^{\,[\ell]}$ the output of the $l$-th layer of a trained
neural network $f$ taking as input some samples $x\sim X$. A "concept
activation vector" $\vec{v}_{C}$ is defined for a set of positive examples $C$
as the normal to a separating hyperplane between $\\{\vec{f}^{\,[\ell]}(x)\mid
x\in C\\}$ versus negative examples – usually $\\{\vec{f}^{\,[\ell]}(x)\mid
x\not\in C\\}$, we provide an illustration in Figure 1(a). In practice, the
hyperplane is learned through a logistic regression on a subset of $C$, while
holding out the remaining samples for validation and testing111By definition,
$\vec{v}_{C}$ is the learned parameter of a logistic regression..

As additional insights, this setting is "sound" for few-shot learning because
it uses a pretrained model $f$ with prior knowledge on the domain – relating
to transfer learning [58] – and learns a linear mapping on its space, hence
reducing the risk of over-parametrisation and justifying why more complex
mappings are not considered.

###  3.2 Sets of examples $C$

The work of Kim et al. [3] lends itself naturally to learning concepts from
playlists. Indeed, playlists can be seen as small sets of positive music
examples built around a particular musical idea. As it will be further
detailed in our experiments, we can use data from music streaming services
where playlists are abundant. We will denote $\mathcal{C}=\\{C_{i}\\}_{i}^{K}$
the set of $K$ sets of music tracks that will enable to learn a CAV set
$\mathcal{V}=\\{\vec{v}_{i}\\}_{i}^{K}$ discriminating $\mathcal{C}$ in the
$\vec{f}^{\,[\ell]}$ space.

It must be noted that playlists are more than a set of tracks, we usually have
access to textual data with a title and a description that help rationalise
the learnt concepts.

###  3.3 Backbone model $\vec{f}^{\,[\ell]}$

As often in few-shot learning, CAVs learning makes use of a pretrained model
$f$ – referred to as backbone – to embed samples into a latent space with a
higher level of abstraction. In this paper, our working hypothesis is that
music concepts can be described solely through audio signals. We wanted to
fully explore the potential of audio signals, which is a known challenging
setting in MIR and recommendation [59], and leave its combination with other
sources of features for future work. In addition, having audio inputs enables
using our model with any music track from any dataset, hence supporting goal
2, while relying on features such as collaborative filtering or other usage
data would have tied our model to a given dataset.

After having experimented with several models, as the recent self-supervised
model CLMR [60], we have chosen to use MusiCNN [61] as the backbone since it
had demonstrated consistent performances across various musical tasks, which
we deemed on par with our goal 2 and 3. MusiCNN was trained on the Million
Song Dataset [62].

###  3.4 Preliminary results

We provide some experimental observations to fix ideas and prepare the next
section. If we train CAVs as in the original paper [3] – further detailed in
section 5.2, we obtain a set of vectors $\mathcal{V}$ that allows to detect a
set of concepts given any music input spectrogram. An example result of
learned concepts is given in Figure 1(b) for one of the studied datasets.
Quite unsurprisingly, we find that many associated concepts end up having
close vector weights222In the figure, proximity is defined as the cosine
similarity.. If we were to use our concept detectors directly, we would have
many concepts activated simultaneously, which could make predictions
unfathomable if $K$ is very big. On the positive side, note that vector
proximity could be used as a means to quantify concept similarities. With this
idea, we go a step further and next present our method to build a hierarchy to
help rationalise concept dependencies.

##  4 Hierarchising music concepts

Having learned numerous non-independent concepts
$\mathcal{V}=\\{\vec{v}_{i}\\}^{K}_{i}$, we next elaborate on how to organise
them with a structure to make them usable (goals 2, 3).

Many classes of structures are possible, and it would be a mistake to select
one with purely theoretical considerations since cognitive and social sciences
[63, 64] have underlined that explanatory preferences are shaped by diverse
factors that cannot be reduced to a set of so-called golden explanation
principles. From an empirical standpoint however, taxonomies seem to be
frequent with music concepts (e.g. genres [8], instruments [25]). We thus
argue that this class of graph is rather natural for humans and chose to
consider it only.

###  4.1 Similarity graph computation

Before mining a hierarchy, it is necessary to determine how concepts – now
acting as graph nodes – relate to one another. Essentially, we are trying to
define a similarity measure between concepts.

When sampling uniformly from a music dataset, concept activations are rare
signals overshadowed by uninformative negative detection. The straightforward
solution of estimating the empirical covariance of concept activations is thus
not a well-behaved measure of similarity. Fortunately, there is another
reliable, interpretable (and faster) way to do it. We propose to consider the
positive examples of each concept playlist – embedded in $\vec{f}^{\,[\ell]}$,
and to check on what side of other concept hyperplanes their centroid lies.
Formally, given $\mathcal{V}=\\{\vec{v}_{i}\\}_{1}^{K}$, we compute the
matrices of concept similarities $S$ and adjacency $A$ (i.e. $A$ is a
binarised matrix representing graph edges):

$\displaystyle S_{i,j}(\mathcal{V})$
$\displaystyle=\mathbb{E}_{x\sim\mathcal{C}_{i}}\left[\sigma(\langle\vec{v}_{j},\vec{f}^{\,[\ell]}(x)\rangle)\right]$
(1) $\displaystyle A_{i,j}(\mathcal{V})$
$\displaystyle=\left[S_{i,j}(\mathcal{V})\geq\frac{1}{2}\right]$ (2)

where $\sigma$ denotes the logistic sigmoid function and $\mathcal{C}_{i}$ a
random variable sampling spectrogram excerpts from tracks of the playlist
$C_{i}$. Bias is integrated in $\vec{v}_{j}$ for simplicity. The threshold at
$\frac{1}{2}$ for $A$ translates having each playlist centroid on the positive
side of the concept hyperplanes. Because $\mathcal{V}$ is learned through
logistic regressions that return well-calibrated probabilities, $S$ and $A$
return similarities with well-defined interpretations.

###  4.2 Hierarchy extraction

Having defined similarities between concepts, we next simplify $S$ and $A$ to
a hierarchy that highlights what concepts are similar to many others and what
others represent unique and niche ideas.

For our problem and its scale, we propose to adapt the greedy hierarchy
construction of Heymann et Garcia-Molina [33] that makes use of the notion of
betweenness graph centrality [65]. In this method, given an unweighted graph
of similarities, a tree is greedily grown by picking each node in decreasing
order of centrality in the similarity graph and linking it to the most similar
and already-added node in the tree. We denote by $H(S(\mathcal{V}))$ the
result of this algorithm on the similarity graph given by the adjacency matrix
$A(\mathcal{V})$ and using $S(\mathcal{V})$ as similarity.

Though this algorithm was originally applied to word tags, we argue that the
inductive bias of the targeted hierarchy is applicable to our problem. In
particular, the use of centrality is justified by the existence of noisy links
in the similarity graph, said links are assumed to be more frequent higher up
in the hierarchy (general-general assumption). For us, this means that general
concepts – e.g. Pop and Rock – should be more likely to be similar than niche
concepts – Bedroom Pop and Goth Rock. This sounds reasonable for music content
since we expect niche concepts to relate to a specific musical idea, style, or
instrument.

Figure 2: Obtained hierarchy on the Deezer dataset. An interactive plot of this figure, as well as many others, are available at  research.deezer.com/concept_hierarchy/. Source | Audio ($\downarrow$) | CF ($\downarrow$) | BERT | W2V1 | W2V2 ($\uparrow$)
---|---|---|---|---|---
$H(S(\mathcal{V}_{D}))$ | 2.449 $\pm$ 0.022 | 0.845 $\pm$ 0.013 | 0.345 $\pm$ 0.007 | 0.286 $\pm$ 0.009 | 0.542 $\pm$ 0.007
$H(S_{\textrm{CF}})$ | 2.413 $\pm$ 0.021 | 0.345 $\pm$ 0.007 | 0.416 $\pm$ 0.008 | 0.336 $\pm$ 0.010 | 0.601 $\pm$ 0.008
$H(S_{\textrm{BERT}})$ | 2.858 $\pm$ 0.028 | 0.868 $\pm$ 0.013 | 0.726 $\pm$ 0.005 | 0.505 $\pm$ 0.011 | 0.652 $\pm$ 0.008
$H(S_{\textrm{W2V-1}})$ | 2.952 $\pm$ 0.028 | 0.932 $\pm$ 0.012 | 0.523 $\pm$ 0.008 | 0.804 $\pm$ 0.005 | 0.721 $\pm$ 0.007
$H(S_{\textrm{W2V-2}})$ | 2.843 $\pm$ 0.026 | 0.847 $\pm$ 0.012 | 0.531 $\pm$ 0.008 | 0.596 $\pm$ 0.009 | 0.836 $\pm$ 0.004
Random | 3.388 $\pm$ 0.027 | 1.104 $\pm$ 0.006 | 0.239 $\pm$ 0.004 | 0.142 $\pm$ 0.006 | 0.452 $\pm$ 0.006
Table 1: Evaluation of the full hierarchy on audio, user-logs and title
semantic similarities with 95% confidence. Audio and CF source are endowed
with an Euclidean similarity measure, the last three semantic sources use
cosine similarities.

##  5 Experiments

We first detail our experimental setups, then validate that concepts can be
reliably learned from audio, and finally show that our proposed structuring
method leads to consistent hierarchies for MIR applications.

###  5.1 Datasets

The set of playlists $\mathcal{C}_{D}$ we use was exported with the API of the
streaming platform Deezer333https://developers.deezer.com/api. We have
filtered 3635 playlists of mixed types to be used as concepts: all 1498
editorial playlists available – considered thoughtfully curated but biased by
popularity, and 2137 public user playlists with the lower artist popularity –
to improve diversity. Playlists have a mean length of 75 tracks and a minimum
of 40. This dataset includes the playlist ids, titles, descriptions, and
public 30s audio preview for each track. There are 245074 unique tracks from
116497 unique artists: playlists overlap is thus very limited.

In order to compare our computed hierarchy to existing taxonomies, we make use
of the APM Music dataset [66]. This is a more traditionally tagged dataset,
but its relevance for our task is to include a ground-truth two-level
hierarchy of the music genres (219) and moods (165) tag types. We aggregate
sets of positively tagged examples as $\mathcal{C}_{\textrm{genre}}$ for each
genre tags and respectively $\mathcal{C}_{\textrm{mood}}$ for moods.

###  5.2 Validating concept learning

We first present experimental evidence related to learning sets of concepts
(section 3), to validate goal 1.

####  5.2.1 Experimental setup

Concept examples are split in a 70% (train) / 10% (validation) / 20% (test)
fashion. According to the requirements of our backbone [61], we convert audio
inputs to mel-spectrograms with log-magnitudes. Representations are extracted
from the penultimate layer of the backbone, denoted as $f^{[\ell]}$ – which
gave us the best performances for concept learning overall. The complete
training code with exact parameters and training regularisers is provided on
our code repository for reproducibility444
github.com/deezer/concept_hierarchy.

####  5.2.2 Learning metrics evaluation

On the Deezer dataset $\mathcal{C}_{D}$, the mean accuracy of learning
concepts is 83.8% $\pm$ 0.3, with a 95% confidence interval. In detail, we
display a histogram of test accuracies in Figure 3, showing that a majority of
playlist concepts can be learned reliably from audio.

As a rationalisation of failure cases, the hypothesis we made in section 3.4
to detect concepts from audio may not always hold. For instance, it fails when
playlist concepts rely on factors extraneous to audio (e.g. playlist of a
movie soundtrack) or when the backbone space is not expressive enough to
discriminate concepts. This actually happens in these experiments with
concepts similar in all respects but for their singing languages: e.g. the
model makes no difference between Japanese and British jazz. We filter
concepts $v_{i}$ below a given test accuracy threshold to account for this
effect. Fixed at 70%, 192 concepts are filtered out, leading to $K=3443$
remaining concepts. This threshold was set empirically by checking that those
left-out concepts were indeed unclear from audio (e.g. OST, new releases,
charts). We denote the resulting concept set $\mathcal{V}_{D}$.

The two smaller APM datasets lead to similar results with a 78.3% $\pm$ 0.8
for genres, denoted $\mathcal{V}_{\textrm{genre}}$, and a 71.3% $\pm$ 0.6
accuracy for mood tags, denoted $\mathcal{V}_{\textrm{mood}}$.

As a wrap-up of this section, goal 1 of detecting concepts from audio seems
satisfied in our experiments.

Figure 3: Concept learning performances. The left line indicates the cut-off
at 70% accuracy, the right line is the mean accuracy of the remaining CAVs.

###  5.3 Hierarchy Mining

We conduct a three-fold evaluation of our proposed hierarchy extraction method
(section 4) in order to evaluate goal 3: structure; ground-truth comparison;
alignment with other sources of concept similarities.

####  5.3.1 Structure priors

We first provide evidence supporting our choice to extract a hierarchical tree
graph over other choices of structures. Though we do not have access to a
ground-truth graph of the relations between concepts in $\mathcal{C}_{D}$, we
have some priors on what a satisfying result should be like. For instance, we
know that musical concepts are often blended (e.g. genres). It is thus safe to
assume that a good graph of concepts should be connected. Extending this
property, isolated nodes should strongly be discouraged. As another prior
assumption on our target graph, we know that we can usually find several
similar playlists to a given one, but that very strongly connected nodes
should be infrequent, sparsity should thus also be valued.

With these two priors in mind, we inspect various graph baselines that could
straightforwardly be obtained from $\mathcal{V}_{D}$ without our hierarchy
extraction step: the similarity graph $A(\mathcal{V}_{D})$ from equation (2);
a similarity graph obtained from $S(\mathcal{V}_{D})$ with an adjusted
threshold to match the sparsity of $H(S(\mathcal{V}_{D}))$ (Sparse $A$); a
top-1 most similar neighbours graph of each node; reference random graphs.
Their respective sparsity and connectivity is shown in Table 2. It appears
that baseline graphs become disconnected as soon as sparsity drops. We also
underline that random baselines provide better-behaved graphs than their
counterparts, indicating that similarity links are not evenly distributed
among nodes, thus pointing to the existence of communities in the graph that
justifies the use of betweenness [67]. Using a hierarchy extraction algorithm
helps maintain a low sparsity while connecting every node in the graph.

Graph | #Edges ($\downarrow$) | #CC ($\downarrow$) | #IN ($\downarrow$)
---|---|---|---
$H$ | 3442 (0.03%) | 1 | 0
$A$ | 1309163 (11%) | 1 | 0
Sparse $A$ | 3443 | 2766 | 2735
Top-1 | 3443 | 134 | 0
Random Sparse $A$ | 3443 | 516 | 430
Random Top-1 | 3443 | 7 | 0
Table 2: Structure evaluation. We count the number of edges in each graph, of
connected components (CC), and of isolated nodes (IN).

This study does not prevent using sparse and connected graphs other than
hierarchies. We believe that other relevant structures may exist. Nonetheless,
as additional benefits of using hierarchies, we note that tree-like graphs are
always planar, which eases their visualisation and navigability by having only
one most-relevant parent per node.

####  5.3.2 Ground-truth evaluation

To validate our generated concept hierarchy, we compare
$H(S(\mathcal{V}_{\textrm{mood}}))$ and $H(S(\mathcal{V}_{\textrm{genre}}))$
to the expert tag hierarchy provided with the APM dataset. Note that this
unavailable for $\mathcal{V}_{D}$. As those hierarchies are two-levelled –
i.e. clustering of tags, we evaluate the accuracy of the mined edges at
linking neighbours from the same cluster and compute a silhouette score [68].
Judging from the results given in Table 3, the generation is promising but far
from perfect.

The estimated hierarchy of genres is illustrated in Figure 1(c) to help
interpret the results. As a typical failure case, the orange cluster
("Electronica") ends up in two separate branches, which can happen because of
the greedy construction of our hierarchy. In other cases, however, the ground-
truth may be questioned: "Latin influence", "Latin", "Ethnic Dance" and
"Calypso" form a link in our hierarchy, but belong to four different clusters
in the ground-truth – due to non-musical taxonomic considerations – despite
being musically similar. It is not easy to quantify the performance upper-
bound we have between our musical concept detection and the practical expert
taxonomies we compare to. Nevertheless, seeing that our generated hierarchies
roughly align with ground-truth structures is a good sanity check.

Tags | Accuracy (%) | Silhouette
---|---|---
$H_{\textrm{mood}}$ | 45.1 | -0.09 $\pm$ 0.05
$H_{\textrm{genre}}$ | 49.1 | -0.17 $\pm$ 0.04
Table 3: Evaluation of our unsupervised hierarchy against ground-truth
clustering of moods and genres tags.

####  5.3.3 Alignment to various sources

Coming back to the difficult setting of $\mathcal{V}_{D}$, we finally evaluate
our hierarchy of concepts with mixed types, illustrated in Figure 2. As we do
not have a ground-truth in this general case, we evaluate whether the edges of
our hierarchy – built from $S(\mathcal{V})$ – make sense on other sources of
similarity. Specifically, since our data is collected from a streaming
service, collaborative filtering embeddings [69] based on listening logs are
available to estimate playlist similarities ($S_{\textrm{CF}}$). Playlists
similar in concepts could indeed be expected to be co-listened by users,
though popularity biases are also at play. We also leverage playlist titles
and expect neighbours to be rather close semantically. To that end, we use a
large language model [70] that can embed any text prompt
($S_{\textrm{BERT}}$), and two music-specialised word embedders for which we
average representations of in-vocabulary words of each concept names:
$S_{\textrm{W2V-1}}$ [9] and $S_{\textrm{W2V-2}}$ [71]. For reference, we
generate hierarchies from each domain-specific source, include a random
hierarchy, and a Audio measure based on CAVs weights, following our
observations made in section 3.4.

We compute the average similarity on each hierarchy’s edges given those
several sources of similarities. Results are provided in Table 1. By
construction, each hierarchy maximises performance on the source it was built
on. We rather inspect how one source of knowledge transfers to other sources.
Collaborative filtering being the canonical way of estimating similarities in
the streaming industry our dataset is extracted from and in general [45], we
underline that our hierarchy – solely based on audio – closely matches its
transfer performances, which is a good result. We have thus shown that our
hierarchy transfers to other sources of knowledge (goal 3).

###  5.4 Qualitative discussion

We retrieve many associations that make sense for humans in
$H(S(\mathcal{V}_{D}))$: blues and jazz; rock and pop; motivation, dancing,
running, and party are close to one another – which aligns with previous work
[72]. More interestingly, we observe that "LSD Trip" is the parent of "Surf",
"Summer of love", and "Stoner Rock", effectively associating an activity to a
sport, an historic phenomenon, and a music genre. Yet, we also find some
association that make sense for the model but not for humans: e.g. "Rock
Christmas" is also a child of "LSD Trip". This phenomenon is hard to avoid
with fully unsupervised methods [53, 73]. We can find many more interesting
associations, but we have to beware of confirmation biases, as often in XAI
[74, 52, 51]. We provide online visualisations of the results555
research.deezer.com/concept_hierarchy/.

As observable limitations, $\mathcal{V_{D}}$ is biased by the streaming
platform usage towards French, English, and Brazilian content, which could be
addressed in future work with importance sampling. Then, some concept titles
are misleading, e.g. playlists "City sounds: <city>" are confounded by the
taste of their curator, resulting in city concepts being close to the 60’s
genre, despite this not being obvious solely judging from the title. Finally,
as already mentioned, the backbone fails to discriminate linguistic
information: e.g. "Queer Pop" and "[Current] Pop" differ by their lyric theme
but are neighbours in the hierarchy.

##  6 Conclusion

Spectrograms are hard to interpret. We propose to extend concept learning to
learn hierarchies of music concepts from playlists, with greater flexibility
than usual music taggers. Our results should be viewed as complementary to
expert music ontologies, as a means to witness how music is organically
described by users and editors, and thus to capture new or evolving salient
aspects of music.

This topic is novel and further steps are necessary in order to to overcome
cultural biases of our data, and to discover causal relations for our
structure. Future work include leveraging the found hierarchy for dynamic
music recommendation – e.g. exploring and switching branches.

## References

  * [1] A. Liberman, F. S. Cooper, D. P. Shankweiler, and M. Studdert-Kennedy, “Why are speech spectrograms hard to read?” _American Annals of the Deaf_ , pp. 127–133, 1968.
  * [2] M. Won, S. Chun, and X. Serra, “Visualizing and understanding self-attention based music tagging,” _Machine Learning for Music Discovery Workshop, ICML_ , 2019.
  * [3] B. Kim, M. Wattenberg, J. Gilmer, C. Cai, J. Wexler, F. Viegas, and R. sayres, “Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV),” in _International Conference on Machine Learning (ICML)_. PMLR, 2018, pp. 2668–2677.
  * [4] C.-K. Yeh, B. Kim, and P. Ravikumar, “Human-centered concept explanations for neural networks,” in _Neuro-Symbolic Artificial Intelligence: The State of the Art_ , vol. 342, 2022, pp. 337–352.
  * [5] A. Ghandeharioun, B. Kim, C.-L. Li, B. Jou, B. Eoff, and R. W. Picard, “Dissect: Disentangled simultaneous explanations via concept traversals,” _International Conference on Learning Representations (ICLR)_ , 2022.
  * [6] C. J. Cai, E. Reif, N. Hegde, J. Hipp, B. Kim, D. Smilkov, M. Wattenberg, F. Viegas, G. S. Corrado, M. C. Stumpe _et al._ , “Human-centered tools for coping with imperfect algorithms during medical decision-making,” in _CHI conference on human factors in computing systems_ , 2019, pp. 1–14.
  * [7] P. Arendsen, D. Marcos, and D. Tuia, “Concept discovery for the interpretation of landscape scenicness,” _Machine Learning and Knowledge Extraction_ , vol. 2, no. 4, pp. 397–413, 2020.
  * [8] R. Hennequin, J. Royo-Letelier, and M. Moussallam, “Audio based disambiguation of music genre tags,” _International Society of Music Information Retrieval Conference (ISMIR)_ , 2018.
  * [9] S. Doh, J. Lee, T. H. Park, and J. Nam, “Musical word embedding: Bridging the gap between listening contexts and music,” _ICML Workshop: Machine Learning for Music Discovery_ , 2020.
  * [10] E. M. Von Hornbostel and C. Sachs, “Classification of musical instruments: Translated from the original german by anthony baines and klaus p. wachsmann,” _The Galpin Society Journal_ , pp. 3–29, 1961.
  * [11] M. Antović, “From expectation to concepts: Toward multilevel grounding in musical semantics,” _Cognitive Semiotics_ , vol. 9, no. 2, pp. 105–138, 2016\.
  * [12] N. Kumar, A. C. Berg, P. N. Belhumeur, and S. K. Nayar, “Attribute and simile classifiers for face verification,” in _12th International Conference on Computer Vision (ICCV)_. IEEE, 2009, pp. 365–372.
  * [13] C. H. Lampert, H. Nickisch, and S. Harmeling, “Learning to detect unseen object classes by between-class attribute transfer,” in _Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE, 2009, pp. 951–958.
  * [14] P. W. Koh, T. Nguyen, Y. S. Tang, S. Mussmann, E. Pierson, B. Kim, and P. Liang, “Concept bottleneck models,” in _International Conference on Machine Learning (ICML)_. PMLR, 2020, pp. 5338–5348.
  * [15] C. Chen, O. Li, D. Tao, A. Barnett, C. Rudin, and J. K. Su, “This looks like that: deep learning for interpretable image recognition,” _Advances in Neural Information Processing Systems (NeurIPS)_ , vol. 32, 2019.
  * [16] B. Zhou, Y. Sun, D. Bau, and A. Torralba, “Interpretable basis decomposition for visual explanation,” in _European Conference on Computer Vision (ECCV)_ , 2018, pp. 119–134.
  * [17] A. Lucieri, M. N. Bajwa, S. A. Braun, M. I. Malik, A. Dengel, and S. Ahmed, “On interpretability of deep learning based skin lesion classifiers using concept activation vectors,” in _International Joint Conference on Neural Networks (IJCNN)_. IEEE, 2020, pp. 1–10.
  * [18] A. R. Asokan, N. Kumar, A. V. Ragam, and S. S. Sharath, “Interpretability for multimodal emotion recognition using concept activation vectors,” _arXiv preprint arXiv:2202.01072_ , 2022.
  * [19] C. Göpfert, Y. Chow, C.-w. Hsu, I. Vendrov, T. Lu, D. Ramachandran, and C. Boutilier, “Discovering personalized semantics for soft attributes in recommender systems using concept activation vectors,” _The Web Conference (WWW)_ , 2022.
  * [20] A. Ghorbani, J. Wexler, J. Y. Zou, and B. Kim, “Towards automatic concept-based explanations,” _Advances in Neural Information Processing Systems (NeurIPS)_ , vol. 32, 2019.
  * [21] Z. Chen, Y. Bei, and C. Rudin, “Concept whitening for interpretable image recognition,” _Nature Machine Intelligence_ , vol. 2, no. 12, pp. 772–782, 2020.
  * [22] E. Strumbelj and I. Kononenko, “An efficient explanation of individual classifications using game theory,” _The Journal of Machine Learning Research_ , vol. 11, pp. 1–18, 2010.
  * [23] C.-K. Yeh, B. Kim, S. Arik, C.-L. Li, T. Pfister, and P. Ravikumar, “On completeness-aware concept-based explanations in deep neural networks,” _Advances in Neural Information Processing Systems (NeurIPS)_ , vol. 33, pp. 20 554–20 565, 2020.
  * [24] Y. Goyal, A. Feder, U. Shalit, and B. Kim, “Explaining classifiers with causal concept effect (cace),” _arXiv preprint arXiv:1907.07165_ , 2019.
  * [25] H. F. Garcia, A. Aguilar, E. Manilow, and B. Pardo, “Leveraging hierarchical structures for few-shot musical instrument recognition,” _International Society of Music Information Retrieval Conference (ISMIR)_ , 2021.
  * [26] D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent dirichlet allocation,” _Journal of machine Learning research_ , vol. 3, no. Jan, pp. 993–1022, 2003\.
  * [27] T. Griffiths, M. Jordan, J. Tenenbaum, and D. Blei, “Hierarchical topic models and the nested chinese restaurant process,” _Advances in Neural Information Processing Systems (NIPS)_ , vol. 16, 2003.
  * [28] V.-A. Nguyen, J. L. Ying, P. Resnik, and J. Chang, “Learning a concept hierarchy from multi-labeled documents,” _Advances in Neural Information Processing Systems (NIPS)_ , vol. 27, 2014.
  * [29] G. N. Lance and W. T. Williams, “A general theory of classificatory sorting strategies: 1. hierarchical systems,” _The computer journal_ , vol. 9, no. 4, pp. 373–380, 1967.
  * [30] X. Liu, Y. Song, S. Liu, and H. Wang, “Automatic taxonomy construction from keywords,” in _SIGKDD international conference on Knowledge discovery and data mining_ , 2012, pp. 1433–1441.
  * [31] J. Völker and M. Niepert, “Statistical schema induction,” in _Extended Semantic Web Conference_. Springer, 2011, pp. 124–138.
  * [32] V. Anoop, S. Asharaf, and P. Deepak, “Unsupervised concept hierarchy learning: a topic modeling guided approach,” _Procedia computer science_ , vol. 89, pp. 386–394, 2016.
  * [33] P. Heymann and H. Garcia-Molina, “Collaborative creation of communal hierarchical taxonomies in social tagging systems,” Stanford, Tech. Rep., 2006\.
  * [34] D. Benz, A. Hotho, and G. Stumme, “Semantics made by you and me: Self-emerging ontologies can capture the diversity of shared knowledge,” in _Web Science Conference (WebSci09)_ , 2010.
  * [35] A. Ross and F. Doshi-Velez, “Benchmarks, algorithms, and metrics for hierarchical disentanglement,” in _International Conference on Machine Learning (ICML)_. PMLR, 2021, pp. 9084–9094.
  * [36] R. Jenatton, J. Mairal, G. Obozinski, and F. R. Bach, “Proximal methods for sparse hierarchical dictionary learning,” in _International Conference on Machine Learning (ICML)_ , 2010.
  * [37] E. Bart, I. Porteous, P. Perona, and M. Welling, “Unsupervised learning of visual taxonomies,” in _Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE, 2008, pp. 1–8.
  * [38] P. Hase, C. Chen, O. Li, and C. Rudin, “Interpretable image recognition with hierarchical prototypes,” in _AAAI Conference on Human Computation and Crowdsourcing_ , vol. 7, 2019, pp. 32–40.
  * [39] F. Doshi-Velez and B. Kim, “Towards a rigorous science of interpretable machine learning,” _arXiv preprint arXiv:1702.08608_ , 2017.
  * [40] J. Adebayo, J. Gilmer, M. Muelly, I. Goodfellow, M. Hardt, and B. Kim, “Sanity checks for saliency maps,” _Advances in neural information processing systems (NeurIPS)_ , 2018.
  * [41] Z. C. Lipton, “The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery.” _Queue_ , vol. 16, no. 3, pp. 31–57, 2018.
  * [42] D. Janzing, L. Minorics, and P. Blöbaum, “Feature relevance quantification in explainable ai: A causal problem,” in _International Conference on artificial intelligence and statistics_. PMLR, 2020, pp. 2907–2916.
  * [43] A. Mahinpei, J. Clark, I. Lage, F. Doshi-Velez, and P. Weiwei, “Promises and pitfalls of black-box concept learning models,” _ICML Workshop on Theoretic Foundation, Criticism, and Application Trend of Explainable AI_ , 2021\.
  * [44] S. Krishna, T. Han, A. Gu, J. Pombra, S. Jabbari, S. Wu, and H. Lakkaraju, “The disagreement problem in explainable machine learning: A practitioner’s perspective,” _arXiv preprint arXiv:2202.01602_ , 2022.
  * [45] D. Afchar, A. B Melchiorre, M. Schedl, R. Hennequin, E. V. Epure, and M. Moussallam, “Explainability in music recommender systems,” in _AI Magazine_ , 2022.
  * [46] R. Geirhos, J.-H. Jacobsen, C. Michaelis, R. Zemel, W. Brendel, M. Bethge, and F. A. Wichmann, “Shortcut learning in deep neural networks,” _Nature Machine Intelligence_ , vol. 2, no. 11, pp. 665–673, 2020.
  * [47] A. B. Arrieta, N. Díaz-Rodríguez, J. Del Ser, A. Bennetot, S. Tabik, A. Barbado, S. García, S. Gil-López, D. Molina, R. Benjamins _et al._ , “Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai,” _Information Fusion_ , vol. 58, pp. 82–115, 2020.
  * [48] D. Afchar, V. Guigue, and R. Hennequin, “Towards rigorous interpretations: a formalisation of feature attribution,” in _International Conference on Machine Learning (ICML)_. PMLR, 2021, pp. 76–86.
  * [49] P. Hase, H. Xie, and M. Bansal, “The out-of-distribution problem in explainability and search methods for feature importance explanations,” _Advances in Neural Information Processing Systems (NeurIPS)_ , vol. 34, 2021\.
  * [50] D. Slack, S. Hilgard, E. Jia, S. Singh, and H. Lakkaraju, “Fooling lime and shap: Adversarial attacks on post hoc explanation methods,” in _AAAI/ACM Conference on AI, Ethics, and Society_ , 2020, pp. 180–186.
  * [51] H. Kaur, H. Nori, S. Jenkins, R. Caruana, H. Wallach, and J. Wortman Vaughan, “Interpreting interpretability: understanding data scientists’ use of interpretability tools for machine learning,” in _CHI conference on human factors in computing systems_ , 2020, pp. 1–14.
  * [52] J. Dinu, J. Bigham, and J. Z. Kolter, “Challenging common interpretability assumptions in feature attribution explanations,” _NeurIPS Workshop: ML Retrospectives, Surveys & Meta-Analyses (ML-RSA)_, 2020.
  * [53] F. Locatello, S. Bauer, M. Lucic, G. Raetsch, S. Gelly, B. Schölkopf, and O. Bachem, “Challenging common assumptions in the unsupervised learning of disentangled representations,” in _International Conference on Machine Learning (ICML)_. PMLR, 2019, pp. 4114–4124.
  * [54] L. Sixt, M. Schuessler, O.-I. Popescu, P. Weiß, and T. Landgraf, “Do users benefit from interpretable vision? a user study, baseline, and dataset,” in _International Conference on Learning Representations (ICLR)_ , 2022.
  * [55] C. Rudin, C. Chen, Z. Chen, H. Huang, L. Semenova, and C. Zhong, “Interpretable machine learning: Fundamental principles and 10 grand challenges,” _Statistics Surveys_ , 2021.
  * [56] H. Chen, J. D. Janizek, S. Lundberg, and S.-I. Lee, “True to the model or true to the data?” _ICML Workshop on Human Interpretability in Machine Learning_ , 2020.
  * [57] J. Pearl, “Causal inference in statistics: An overview,” _Statistics Surveys_ , vol. 3, pp. 96 – 146, 2009.
  * [58] S. Bozinovski and A. Fulgosi, “The influence of pattern similarity and transfer of learning upon training of a base perceptron b2.(original in croatian: Utjecaj slicnosti likova i transfera ucenja na obucavanje baznog perceptrona b2),” in _Proc. Symp. Informatica_ , 1976, pp. 3–121.
  * [59] O. Celma, P. Herrera, and X. Serra, “Bridging the music semantic gap,” _International Semantic Web Conference_ , 2006.
  * [60] J. Spijkervet and J. A. Burgoyne, “Contrastive learning of musical representations,” _International Society of Music Information Retrieval Conference (ISMIR)_ , 2021.
  * [61] J. Pons and X. Serra, “musicnn: Pre-trained convolutional neural networks for music audio tagging,” _Late breaking/demo session of the International Society for Music Information Retrieval Conference (LBD-ISMIR)_ , 2019.
  * [62] T. Bertin-Mahieux, D. P. Ellis, B. Whitman, and P. Lamere, “The million song dataset,” in _International Conference on Music Information Retrieval (ISMIR)_ , 2011.
  * [63] T. Lombrozo, “Explanatory preferences shape learning and inference,” _Trends in Cognitive Sciences_ , vol. 20, no. 10, pp. 748–759, 2016.
  * [64] T. Miller, “Explanation in artificial intelligence: Insights from the social sciences,” _Artificial intelligence_ , vol. 267, pp. 1–38, 2019.
  * [65] S. P. Borgatti and M. G. Everett, “A graph-theoretic perspective on centrality,” _Social networks_ , vol. 28, no. 4, pp. 466–484, 2006.
  * [66] B. Gao, E. Dellandréa, and L. Chen, “Music sparse decomposition onto a midi dictionary of musical words and its application to music mood classification,” in _International Workshop on Content-Based Multimedia Indexing (CBMI)_. IEEE, 2012, pp. 1–6.
  * [67] M. Girvan and M. E. Newman, “Community structure in social and biological networks,” _Proceedings of the national academy of sciences_ , vol. 99, no. 12, pp. 7821–7826, 2002.
  * [68] P. J. Rousseeuw, “Silhouettes: a graphical aid to the interpretation and validation of cluster analysis,” _Journal of computational and applied mathematics_ , vol. 20, pp. 53–65, 1987.
  * [69] Y. Hu, Y. Koren, and C. Volinsky, “Collaborative filtering for implicit feedback datasets,” in _International Conference on Data Mining_. IEEE, 2008, pp. 263–272.
  * [70] N. Reimers and I. Gurevych, “Sentence-bert: Sentence embeddings using siamese bert-networks,” in _Conference on Empirical Methods in Natural Language Processing_. ACL, 2019.
  * [71] M. Won, S. Oramas, O. Nieto, F. Gouyon, and X. Serra, “Multimodal metric learning for tag-based music retrieval,” in _International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2021, pp. 591–595.
  * [72] K. M. Ibrahim, J. Royo-Letelier, E. V. Epure, G. Peeters, and G. Richard, “Audio-based auto-tagging with contextual tags for music,” in _International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2020, pp. 16–20.
  * [73] F. Dalvi, A. R. Khan, F. Alam, N. Durrani, J. Xu, and H. Sajjad, “Discovering latent concepts learned in bert,” in _International Conference on Learning Representations (ICLR)_ , 2022.
  * [74] R. S. Nickerson, “Confirmation bias: A ubiquitous phenomenon in many guises,” _Review of General Psychology_ , vol. 2, no. 2, pp. 175–220, 1998.

  *[XAI]: explainability
  *[MIR]: Music Information Retrieval
  *[CAVs]: Concept Activation Vector
  *[NLP]: natural language processing
  *[CAV]: Concept Activation Vector


# Characterizing Social Movement Narratives in Online Communities: The 2021
Cuban Protests on Reddit

Brian Keith Norambuena<EMAIL_ADDRESS>Virginia TechBlacksburgVirginiaUSA
Universidad Católica del NorteDepartment of Computing & Systems
EngineeringAntofagastaChile1270709 ,  Tanushree Mitra<EMAIL_ADDRESS>University of WashingtonSeattleWashingtonUSA and  Chris North<EMAIL_ADDRESS>Virginia TechBlacksburgVirginiaUSA

###### Abstract.

Social movements are dominated by storytelling, as narratives play a key role
in how communities involved in these movements shape their identities. Thus,
recognizing the accepted narratives of different communities is central to
understanding social movements. In this context, journalists face the
challenge of making sense of these emerging narratives in social media when
they seek to report social protests. Thus, they would benefit from support
tools that allow them to identify and explore such narratives. In this work,
we propose a narrative extraction algorithm from social media that
incorporates the concept of community acceptance. Using our method, we study
the 2021 Cuban protests and characterize five relevant communities. The
extracted narratives differ in both structure and content across communities.
Our work has implications in the study of social movements, intelligence
analysis, computational journalism, and misinformation research.

narrative extraction, online communities, narrative maps

††copyright: none††conference: C+J Conference 2022; June 9 – 11, 2022; New
York, NY, USA††ccs: Applied computing Publishing††ccs: Computing methodologies
Information extraction

##  1\. Introduction

Social movements are dominated by storytelling (Davis, 2002). In fact,
narratives play a key role in social change and social movements (Bell, 2010),
as people use them to learn and exercise agency, shape identity, and motivate
actions (Dimond et al., 2013; Ganz, 2001). Social movements are made possible
by the construction of collective narratives within free spaces (Couto, 1993).
In particular, online communities in social media provide free spaces for
people to develop such narratives (Kow et al., 2016). Thus, studying how
narratives evolve in these online communities is key to understanding social
movements.

Journalists face unique challenges when reporting social movements (Mattoni,
2013) and making sense of the emerging narratives in social media. In
particular, the fluid nature and rapid evolution of social movements (Koltsova
and Selivanova, 2019) makes it hard to keep track of its goals and
motivations. In this context, identifying the driving narratives in the
different communities involved in a social movement would aid journalists in
their sensemaking process. The accepted narratives of a community provide a
cognitive roadmap by which community members are to live and define the
collective identity of the group and their relations with other groups
(Kaufman, 2009; Larsen and Johnson, 2006; Kartchner and Johnson, 2009),
providing insight towards their motivations and goals.

In this work, we propose a narrative extraction algorithm that incorporates
the concept of community acceptance to model and extract the accepted
narratives from social media communities. Our proposed algorithm leverages the
narrative maps approach of Keith and Mitra (Keith and Mitra, 2021) to
construct a complex narrative representation via optimization. The extracted
narratives could provide journalists with an overview of the evolving
narratives surrounding the social movement, giving them a window into these
communities. In particular, we seek to answer the following research question:

  * •

RQ: How can we model community acceptance in social media and extract the
accepted narratives of a social media community?

To test our method, we study the start of the 2021 Cuban protests—the biggest
protests in decades in Cuba (Robles, 2021)—and the narratives surrounding this
movement that emerged in online communities. Cuba has a unique sociopolitical,
economic, and cultural landscape (Dye et al., 2018). Its long-standing
communist regime faces unique challenges (Allahar and Valdés, 2013). The
continued US embargo (White, 2019) has undermined the Cuban economy (Kapcia,
2020) and its health care system (Akbarpour Roshan and Abbasi, 2018). The
protests were triggered by food and medicine shortages (Fletcher, 2021), and
the inefficient handling of the COVID-19 pandemic by the government (Faiola,
2021).

Our approach should be able to capture these relevant events and storylines,
allowing journalists to easily identify key elements of the narratives
espoused by different communities in social media. In particular, we focus on
a specific social media platform—Reddit—to explore how narratives change
across communities. Specifically, we characterize the accepted narratives in a
number of communities with a high level of activity related to the 2021 Cuban
protests (e.g., r/cuba and r/politics). Thus, we make the following
contributions:

  1. (1)

A narrative extraction algorithm that models community acceptance in online
communities, which could be used by journalists and other practitioners to
explore social media narratives.

  2. (2)

An analysis of the accepted narratives espoused by online communities
surrounding the 2021 Cuban protests to showcase the application of our method.

We believe our approach has widespread implications, both for the study of
social movement narratives online and narratives in general. At its core, our
methods allow users to understand the accepted narratives of online
communities. Furthermore, expert users, such as intelligence analysts,
journalists, and misinformation researchers could use our methods to explore
the narrative landscape of social movements emerging in online communities.

##  2\. Related Work

Narratives are systems of stories with coherent themes (Halverson et al.,
2011), where each story is represented as a sequence of events (Abbott, 2008).
Most narrative extraction methods use an event-based representation, where
each event is associated with a single document in a data set (Shahaf and
Guestrin, 2010; Keith and Mitra, 2021). We follow this approach, as event-
based representations have strong theoretical foundations in narratology
(Abbott, 2008). Narrative extraction approaches usually work by optimizing
different criteria (Wang et al., 2015; Shahaf and Guestrin, 2010). We use a
narrative extraction algorithm based on the criteria of coherence maximization
through linear programming, an extension of previous work in (Keith and Mitra,
2021). However, unlike previous models, we integrate the concept of community
acceptance in our optimization model to select important events and
storylines.

Narratives are key to the formation of social movements (Bell, 2010; Davis,
2002). In this context, social media plays a key role in shaping the narrative
(Kahne et al., 2016; Kow et al., 2016) and mobilizing people (Leenders and
Heydemann, 2012). Thus, several works have sought to model these narratives.
Computational methods to study social movement narratives in social media
usually rely on keyword and hashtag analysis (Xiong et al., 2019), topic
modeling (Smoliarova et al., 2019), and user network analysis (Vicari, 2017).
Other approaches include qualitative research, focusing on interviews and the
experiences of people (Crivellaro et al., 2014; Kou et al., 2017). However,
none of these approaches consider the underlying structure of the narrative,
which depends on temporal and causal relationships between events. In
contrast, our approach extracts the narrative structure, allowing for a
detailed model of the social movement narrative via narrative maps—an event-
based directed acyclic graph narrative representation.

##  3\. Materials and Methods

Data Set Description. We used Pushshift (Baumgartner et al., 2020) to extract
Reddit data. In particular, we extracted all submissions that contained the
word “Cuba” from 07/07/2021 to 07/17/2021, covering a few days before the
protests started on July 11th and the first week of the protests, allowing us
to capture any useful signals on social media prior to the start of protests
and the early narratives that emerged just after they started. We retrieved
4810 submissions. We focused on the top five communities according to their
number of posts (see Table 1). We stopped at five because the sixth community
(r/FreeKarma4U) was a karma farming group (Alizadeh et al., 2020) unrelated to
the protests that did not have a cohesive ideology or group identity and the
number of posts rapidly declined in all subsequent communities. We manually
analyzed all the submissions to ensure that they were relevant to the
protests, finding that roughly 95% out of 550 were relevant.

Community |  Description | Posts | Subscribers
---|---|---|---
r/cuba |  The biggest Cuban community in Reddit. | 168 | 36k
r/Conservative |  US-centric right-leaning community focused on political issues. | 126 | 833k
r/GenZedong |  Pro-communist community focused on humor and world news. | 99 | 33k
r/worldnews |  Community focused on world news (excluding US news). | 79 | 26.6M
r/politics |  US-centric left-leaning community focused on political issues. | 78 | 7.6M

Table 1. Summary of the analyzed Reddit communities, including the number of
subscribers and submissions retrieved.

Narrative Extraction Method. We use narrative maps (Keith and Mitra, 2021) as
our representation (see Figure 2). Narrative maps provide a generic foundation
to encode different types of narratives extracted from data, requiring only
the existence of a total ordering (e.g., timestamps) and text representation
of the event (e.g., a news headline or post title). Based on a route map
metaphor, they consist of a directed acyclic graph of event nodes, where each
path represents a route connecting events. These nodes are called landmarks,
based on the idea of route maps having a series of landmarks to highlight
important elements. Certain events are highlighted—called representative
landmarks—at the point of highest width in the graph to capture all parallel
storylines. Finally, by finding the path of maximum coherence we find the main
route that contains the core events of the narrative. Narrative maps are
extracted by optimizing coherence relations between events—based on event
similarity and topical consistency—subject to coverage constraints.

In this work, we propose a new optimization model that accounts for community
acceptance in addition to coherence and story coverage. We include new
parameters to account for the submission score (i.e., the difference between
upvotes and downvotes) and upvote ratio as a way to measure community
acceptance of a submission. In particular, posts that have a high score and
upvote ratio in a subreddit are considered accepted by that community (Buz and
de Melo, 2021; Li, 2020) and should appear in the narrative. We model this
with a constraint that requires the average score percentile to be above a
certain threshold. Note that this does not necessarily discard all the low
score submissions, as their presence might be necessary to construct a
coherent narrative. Furthermore, we incorporate community acceptance into the
optimization objective itself, extracting maps by maximizing both coherence
and acceptance.

We show the optimization model in Figure 1. Our goal is to maximize the
minimum edge strength ($\mathsf{minedge}$), which depends on
$\mathsf{coherence}$—a measure of how much sense it makes to join two events
together, based on event similarity and topical similarity—and
$\mathsf{acceptance}$—a measure of community acceptance that depends on the
upvote ratios and submission score in the community. Table 3 in the appendix
presents the details of our model.

Figure 1. Linear program formulation. Parts highlighted in red are our
contributions to the model.

For the parameters, we set $\mathsf{mincover}=0.5$, ensuring that we cover at
least half of the stories without excessively restricting the narrative. We
set $K=8$ to get main stories with around 8 events, based on (Keith and Mitra,
2021; Shahaf and Guestrin, 2010). We set $\mathsf{minscore}=0.85$, ensuring
that the average community acceptance is at least 85%—intuitively, this means
that roughly 85% of the community regards the selected submissions positively.
Lower thresholds allowed too many low-acceptance events to be on the map. We
encode submissions with Multilingual USE embeddings (Yang et al., 2020), as
some communities use English and Spanish. The rest of the implementation
follows (Keith and Mitra, 2021). We note that all parameters can be easily
modified, allowing journalists and other users to explore their influence on
the narrative.

##  4\. Results

Now we present the accepted narratives extracted from each community. Due to
space limits, we offer details of only one community (r/cuba) while briefly
mentioning the rest. We also analyze how the narratives espoused by each
community differ from each other and manually categorize the relevant
storylines and events.

Figure 2. Narrative map for r/cuba. The numbers beside each submission
represent the upvote ratio and score percentile. The main storyline is
represented by the blue dashed lines. Highlighted nodes correspond to
representative landmarks

. Edge weights represent coherence. Manual annotations describing specific
storylines have been added with markers

,

, and

.

r/cuba: Figure 2 shows the narrative map extracted from r/cuba. The main
storyline starts with a submission criticizing the standing of Cuba as a
leader in health care. Moreover, it highlights economical issues and political
repression. Next, a submission discusses the state of the pandemic in Cuba on
the same day that the protests start. Overall, the main storyline gives the
impression that the protests started due to health care issues, alongside
other catalysts.

Afterward, we see a simple submission with the rallying call “Viva Cuba
libre!” (translated as “long live free Cuba”) with an image about tyranny in
its internal contents. We note that this submission was locked by the
moderators of the community. The next day there is a post criticizing the
moderators for taking up anything about the protests. Then, we see a rallying
call with a video showing the protests. After this, there are two submissions
seeking information about the protests, asking about counter-protests and
whether the internet has been shut down. Finally, the story ends with reports
of abuse by Cuban police and a call against the regime.

Second, we analyze the representative landmarks highlighted in green and their
storylines. Storyline  and its representative landmark are focused on the US.
This story discusses the US embargo, a potential intervention, and a migratory
crisis. The second representative landmark is a rallying call in the main
storyline, which makes sense as a key narrative event that highlights the
protests. The third representative landmark is another rallying call with a
photo of protesters carrying a Cuban flag. The rest of the storyline  focuses
mostly on repression in the protests, with Spain demanding the release of a
reporter, and the police and military shooting protesters. However, it ends on
a better note with changes in customs restrictions by the Cuban government due
to the protests. The final representative landmark and side storyline  focus
on the counter-protests and support for the regime, with people mobilizing in
defense of the communist revolution.

Overall, the side storylines provide three key points that complement the main
storyline and point towards potential future events. First, the US is likely
to be a key player in this narrative, although it is too early at the time of
this writing how it might respond. Second, the protests have at least some
limited success so far, such as the lifting of customs restrictions. Finally,
there is a strong response by the government and its forces, with police and
military forces being used, as well as a counter-movement that backs the
government.

r/Conservative: We present the narrative of r/Conservative—a right-leaning
community centered on US politics (Van Koevering et al., 2020)—in Figure 3.
The main story starts with the protests and how Cubans seek to end the regime.
However, the rest of the narrative focuses mostly on US political actors. The
narrative of r/Conservative uses the protests as a tool to criticize its
political opponents (left-leaning media, Democrats, Black Lives Matter, etc.),
rather than focusing on the protests themselves. Regarding the representative
landmarks, the first one is a satirical article criticizing communist
supporters and their views on the regime after the protests. The second one is
part of the main story and highlights how celebrities are also stepping in to
support the protests against communism.

r/GenZedong: We present the narrative of r/GenZedong—a pro-communist
community—in Figure 4. Due to the linear nature of this narrative, extracting
representative landmarks is not useful, as they rely on the existence of
parallel stories (Keith and Mitra, 2021). The storyline starts with an
unrelated submission criticizing the presence of an anti-communist Cuban exile
in a documentary about Cuba. Although not related to the protests, it aligns
with the pro-communist stance of the community. The rest of the timeline
supports the regime in Cuba with mostly satirical renditions of anti-communist
and pro-US submissions. Overall, this narrative supports the Cuban regime and
criticizes its opponents.

r/worldnews: Figure 5 shows the narrative of r/worldnews—a neutral community
focused on global news (Horne et al., 2017; Bahgat et al., 2020). The main
storyline starts with an unrelated event on Cuba developing effective
vaccines. This is an interesting parallel with the r/cuba narrative that also
started with a health care-related post but in a negative light. The next
event is the Mexican president calling for an end to the US embargo—a
representative landmark of this map. Next, the storyline focuses on how the
protests developed in Cuba. Then, the Miami mayor is discussing bombing Cuba.
Immediately after this, we see Cuba lifting the customs restrictions, heavily
implying that fear of US intervention led to removing the restrictions, unlike
the r/cuba narrative that presented this as a result of the protests. The side
story on the left focuses on the international response (e.g., Russia, China,
Iran, and the US). Overall, this narrative has two main themes: the protests
themselves and the responses by international actors. This makes sense as the
purpose of the community is to discuss world news. Moreover, this external
perspective is an interesting complement to the r/cuba narrative, as it
provides a bigger picture view rather than an emotional one. For example, note
the lack of rallying call submissions. This makes sense as this community is
neutral and should have a more objective view.

r/politics: We present the narrative of r/politics—a left-leaning community
centered on US politics (Van Koevering et al., 2020; Shepherd, 2020)— in
Figure 6. The narrative starts with a report on the protests, then it mostly
focuses on calls to lift the embargo. The storyline then shifts towards the
protests in Miami, with a focus on the anti-riot law. The narrative uses this
to criticize the political opponents of the community, in a similar way to
r/Conservative.

Relevant Storylines | r/cuba | r/Conservative | r/GenZedong | r/worldnews | r/politics
---|---|---|---|---|---
Causes of the Protest | $\times$ |  |  | $\times$ |
Cuban Government Response | $\times$ |  |  | $\times$ | $\times$
Police Brutality and Repression | $\times$ |  |  | $\times$ |
Supporting the Protests | $\times$ | $\times$ |  |  | $\times$
Supporting the Regime (counter-protests) | $\times$ |  | $\times$ |  |
International Response |  |  |  | $\times$ |
US Response and Potential Intervention | $\times$ |  | $\times$ | $\times$ | $\times$
Ending the US Trade Embargo | $\times$ |  | $\times$ | $\times$ | $\times$
Criticizing Political Opponents | $\times$ | $\times$ | $\times$ |  | $\times$

Table 2. Relevant storylines & events in each narrative map.

Narrative Comparison: We show the relevant storylines and events in each
narrative in Table 2. We note that r/cuba covers almost all the aspects of the
protest except the international response, save for the US, given its
relevance in the historical and geopolitical context of Cuba. In contrast, the
other communities tend to focus on specific storylines based on their agenda.

##  5\. Discussion

Social Movement Narratives. Our work unravels the different narratives
espoused by online communities pertaining to the 2021 Cuban protest movement.
In general, our work provides a computational framework to analyze the
accepted narratives that emerge in online communities in the context of social
movements, allowing analysts and social scientists to understand the
motivations and relations between different groups in the context of a social
movement. In particular, we found that r/cuba had the most complex narrative
with a tree-like structure and diverse content covering multiple storylines.
In contrast, r/GenZedong had the simplest narrative—a single linear
storyline—and had pro-communist content. Meanwhile, both the US-based
political communities—r/politics and r/Conservative—used the protests to
criticize their opponents and further their own agendas, rather than focusing
on the protests themselves. Finally, r/worldnews showed the most neutral
presentation, focusing on factual content rather than emotional content, with
news discussing the international response and the events of Cuba itself.

Implications. In the context of computational journalism (Cohen et al.,
2011)—which provides tools for various computational needs in journalism—our
approach could serve as a support tool for journalists. For example, narrative
maps could be part of a narrative exploration suite, which could also include
text summaries, data plots, and references to trustworthy sources. Such a tool
could aid analysts, journalists, and other users in trying to understand the
narratives espoused by online communities (Keith and Mitra, 2021). Our work
could also be applied by social scientists interested in a narrative analysis
of how social movements are evolving, as the accepted narratives provide a
window into what different communities are thinking and their views.

Furthermore, our community acceptance approach could be enhanced with
additional attributes. In particular, we could model the credibility and
political bias of Reddit submissions (e.g., by analyzing the source they are
linking). These attributes could aid in modeling how misinformation spreads
during social movements. Another possible extension would be using sentiment
analysis (Ravi and Ravi, 2015) in conjunction with our narrative model to
obtain more nuanced narratives that explicitly model how communities feel with
respect to specific storylines and events. Finally, we note that our methods
are grounded in theoretical constructs of narratology (Keith and Mitra, 2021)
and could be applied beyond social movement narratives.

Ethical Considerations. There are two stakeholder groups in this research:
members of the studied communities and potential users of the system (e.g.,
analysts, journalists, and researchers). As with any system that could be used
to monitor social movements, there is a risk that malicious actors leverage
the system for nefarious uses, such as tracking activists involved in
organizing or participating in these movements or designing targeted
information operations to counter or support specific narratives. Regarding
tracking issues, the system works at an aggregate level, without user-level
details, and the system uses non-confidential and publicly available data and
does not require any specific user information. Thus, it would be hard to
perform user re-identification. Next, we note that the system does not
distinguish between the truth and misinformation, which could allow users to
explore how misinformation spreads in online communities, but also lead to
incorrect interpretations of the narratives. We could reduce the amount of
potential misinformation in maps by giving control to the analysts over the
information sources used.

Nevertheless, we note that the narrative extraction tool is still in the
prototyping phase. We expect to address any issues that arise as we work with
stakeholders in user studies and pilot testing. Through this process, we will
ensure that the system is resistant to these issues. Finally, we note that our
approach does not involve any direct interaction with the studied communities,
it only analyzes publicly available content from these communities. Thus, we
do not anticipate any direct harm resulting from this research.

Limitations. Our work is not without limitations. The current embedding model
only considers the text of a submission, disregarding images and videos. Thus,
future work could consider multimedia embeddings to model these elements.
Moreover, we note that our data spans a relatively small period of time. Thus,
the analysis does not capture all the narratives surrounding the movement.

##  6\. Conclusions

This paper presents an algorithmic approach to extract the accepted narratives
of online communities in the context of social movements, specifically the
2021 Cuban Protests during its first few days. Building upon existing
narrative extraction methods based on linear optimization, we designed a novel
approach that integrates the concept of community acceptance with pre-existing
definitions of coherence and coverage. Our technique allows us to identify the
major themes surrounding the protests in five different online communities.
Recognizing the accepted narratives of different communities is central to
understanding social movements. Thus, our method could aid journalists in
making sense of emerging narratives in social media in the context of
reporting social movements. In general, our work has potential applications in
the study of social movements, intelligence analysis, computational
journalism, and misinformation research. Moreover, it opens up multiple
avenues for future research, such as adapting the method to find rejected
narratives or modeling acceptance in other social networks.

## Acknowledgments

This work was supported in part by NSF I/UCRC CNS-1822080 via the NSF Center
for Space, High-performance, and Resilient Computing (SHREC), the NSF grants
CNS-1915755 and DMS-1830501, and by ANID/Doctorado Becas Chile/2019 -
72200105.

## References

  * (1)
  * Abbott (2008) H Porter Abbott. 2008\.  _The Cambridge introduction to narrative_.  Cambridge University Press, NY, USA.
  * Akbarpour Roshan and Abbasi (2018) Narges Akbarpour Roshan and Mohsen Abbasi. 2018.  The Impact of the US Economic Sanctions on Health in Cuba.  _Int J of Resistive Econ_ 6, 2 (2018), 17–36.
  * Alizadeh et al. (2020) Meysam Alizadeh, Jacob N. Shapiro, Cody Buntain, and Joshua A. Tucker. 2020.  Content-based features predict social media influence operations.  _Science Advances_ 6, 30 (2020), 13 pages.
  * Allahar and Valdés (2013) Antón L Allahar and Nelson P Valdés. 2013.  The Bureaucratic Imperative: Economic and Political Challenges to Cuban Socialism in the Early 21st Century.  _The CLR James Journal_ 19, 1/2 (2013), 392–422.
  * Bahgat et al. (2020) Mohamed Bahgat, Steve Wilson, and Walid Magdy. 2020\.  Towards Using Word Embedding Vector Space for Better Cohort Analysis.  _Proc. of the Intl. AAAI Conf. on Web and Social Media_ 14, 1 (May 2020), 919–923.
  * Baumgartner et al. (2020) Jason Baumgartner, Savvas Zannettou, Brian Keegan, Megan Squire, and Jeremy Blackburn. 2020.  The Pushshift Reddit Dataset.  _Proc. of the Intl. AAAI Conf. on Web and Social Media_ 14, 1 (May 2020), 830–839.
  * Bell (2010) Lee Anne Bell. 2010\.  _Storytelling for social justice: Connecting narrative and the arts in antiracist teaching_.  Routledge, NY, USA.
  * Buz and de Melo (2021) Tolga Buz and Gerard de Melo. 2021.  Should You Take Investment Advice From WallStreetBets? A Data-Driven Approach.
  * Cohen et al. (2011) Sarah Cohen, James T Hamilton, and Fred Turner. 2011\.  Computational journalism.  _Commun. ACM_ 54, 10 (2011), 66–71.
  * Couto (1993) Richard A Couto. 1993\.  Narrative, free space, and political leadership in social movements.  _The Journal of Politics_ 55, 1 (1993), 57–79.
  * Crivellaro et al. (2014) Clara Crivellaro, Rob Comber, John Bowers, Peter C. Wright, and Patrick Olivier. 2014.  A Pool of Dreams: Facebook, Politics and the Emergence of a Social Movement. In _Proc. of CHI 2014_. ACM, NY, USA, 3573–3582.
  * Davis (2002) Joseph E Davis. 2002\.  _Stories of change: Narrative and social movements_.  SUNY Press, Albany, NY.
  * Dimond et al. (2013) Jill P. Dimond, Michaelanne Dye, Daphne Larose, and Amy S. Bruckman. 2013.  Hollaback! The Role of Storytelling Online in a Social Movement Organization. In _Proc. of the 2013 Conf. on CSCW_. ACM, NY, USA, 477–490.
  * Dye et al. (2018) Michaelanne Dye, David Nemer, Josiah Mangiameli, Amy S. Bruckman, and Neha Kumar. 2018\.  El Paquete Semanal: The Week’s Internet in Havana. In _Proc. of the 2018 CHI Conf. on Human Factors in Computing Systems_. ACM, NY, USA, 1–12.
  * Faiola (2021) Anthony Faiola. 2021\.  Cubans take to the streets for the biggest anti-government protests in decades.  https://web.archive.org/web/20210712033912if_/https://www.washingtonpost.com/world/2021/07/11/cuba-protests/
  * Fletcher (2021) Pascal Fletcher. 2021\.  Cuba protests: Frustration at government runs deep.  https://web.archive.org/web/20210714110213/https://www.bbc.com/news/world-latin-america-57823130
  * Ganz (2001) Marshall Louis Ganz. 2001\.  The power of story in social movements.  In _Proc. of the Annual Meeting of the ASA_. American Sociological Association, CA, USA.
  * Halverson et al. (2011) Jeffry Halverson, Steven Corman, and H Lloyd Goodall. 2011\.  _Master narratives of Islamist extremism_.  Springer, NY, USA.
  * Horne et al. (2017) Benjamin D. Horne, Sibel Adali, and Sujoy Sikdar. 2017\.  Identifying the Social Signals That Drive Online Discussions: A Case Study of Reddit Communities. In _Proc. of the ICCCN 2017_. IEEE, Vancouver, BC, Canada, 1–9.
  * Kahne et al. (2016) Joseph Kahne, Erica Hodgin, and Elyse Eidman-Aadahl. 2016\.  Redesigning civic education for the digital age: Participatory politics and the pursuit of democratic engagement.  _Theory & Research in Social Education_ 44, 1 (2016), 1–35.
  * Kapcia (2020) Antoni Kapcia. 2020\.  _A short history of revolutionary Cuba: revolution, power, authority and the state from 1959 to the present day_.  Bloomsbury, London, UK.
  * Kartchner and Johnson (2009) Kerry Kartchner and Jennie Johnson. 2009.  _Strategic culture and weapons of mass destruction: culturally based insights into comparative national security policymaking_.  Springer, NY, USA.
  * Kaufman (2009) Stuart J Kaufman. 2009\.  Narratives and symbols in violent mobilization: the Palestinian-Israeli case.  _Security Studies_ 18, 3 (2009), 400–434.
  * Keith and Mitra (2021) Brian Keith and Tanushree Mitra. 2021.  Narrative Maps: An Algorithmic Approach to Represent and Extract Information Narratives.  _Proc. of the ACM on Human-Computer Interaction_ 4, CSCW3 (2021), 1–33.
  * Koltsova and Selivanova (2019) Olessia Y. Koltsova and Galina Selivanova. 2019.  Explaining Offline Participation in a Social Movement with Online Date: The Case of Observers for Fair Elections.  _Mobilization: An International Quarterly_ 24, 1 (03 2019), 77–94.
  * Kou et al. (2017) Yubo Kou, Yong Ming Kow, Xinning Gui, and Waikuen Cheng. 2017\.  One social movement, two social media sites: A comparative study of public discourses.  _Computer Supported Cooperative Work_ 26, 4 (2017), 807–836.
  * Kow et al. (2016) Yong Ming Kow, Yubo Kou, Bryan Semaan, and Waikuen Cheng. 2016\.  Mediating the Undercurrents: Using Social Media to Sustain a Social Movement. In _Proc. of CHI 2016_. ACM, NY, USA, 3883–3894.
  * Larsen and Johnson (2006) Jeffrey A Larsen and Jeannie L Johnson. 2006.  _Comparative strategic cultures curriculum project: assessing strategic culture as a methodological approach to understanding WMD decision-making by states and non-state actors_.  Technical Report. Science Applications International Corp, McLean, VA.
  * Leenders and Heydemann (2012) Reinoud Leenders and Steven Heydemann. 2012.  Popular mobilization in Syria: opportunity and threat, and the social networks of the early risers.  _Mediterranean politics_ 17, 2 (2012), 139–159.
  * Li (2020) Vicki Li. 2020.  The Race Talk on Reddit:“China Virus” and its Effect on Asian-Americans.
  * Mattoni (2013) Alice Mattoni. 2013\.  _Journalism and Social Movements_.  Wiley, NJ, USA, 339–340.
  * Ravi and Ravi (2015) Kumar Ravi and Vadlamani Ravi. 2015.  A survey on opinion mining and sentiment analysis: tasks, approaches and applications.  _Knowl-based Syst_ 89 (2015), 14–46.
  * Robles (2021) Frances Robles. 2021\.  Cubans Denounce ‘Misery’ in Biggest Protests in Decades.  https://web.archive.org/web/20210711212619/https://www.nytimes.com/2021/07/11/world/americas/cuba-crisis-protests.html
  * Shahaf and Guestrin (2010) Dafna Shahaf and Carlos Guestrin. 2010.  Connecting the Dots between News Articles. In _Proc. of the 2010 SIGKDD Conf._ ACM, NY, USA, 623–632.
  * Shepherd (2020) Ryan P Shepherd. 2020\.  Gaming Reddit’s algorithm: R/the_donald, amplification, and the rhetoric of sorting.  _Computers and Composition_ 56 (2020), 102572.
  * Smoliarova et al. (2019) Anna S. Smoliarova, Svetlana S. Bodrunova, Alexandr V. Yakunin, Ivan Blekanov, and Alexey Maksimov. 2019.  Detecting Pivotal Points in Social Conflicts via Topic Modeling of Twitter Content. In _Internet Science_ , Svetlana S. Bodrunova, Olessia Koltsova, Asbjørn Følstad, Harry Halpin, Polina Kolozaridi, Leonid Yuldashev, Anna Smoliarova, and Heiko Niedermayer (Eds.). Springer, Cham, 61–71.
  * Van Koevering et al. (2020) Katherine Van Koevering, Austin R. Benson, and Jon Kleinberg. 2020\.  Frozen Binomials on the Web: Word Ordering and Language Conventions in Online Text. In _Proc. of The Web Conf. 2020_. ACM, NY, USA, 606–616.
  * Vicari (2017) Stefania Vicari. 2017\.  Twitter and Non-Elites: Interpreting Power Dynamics in the Life Story of the (#)BRCA Twitter Stream.  _Social Media + Society_ 3, 3 (2017), 14 pages.
  * Wang et al. (2015) Lu Wang, Claire Cardie, and Galen Marchetti. 2015.  Socially-Informed Timeline Generation for Complex Events. In _Proc. of the Conf. of the North American Chapter of the ACL: Human Language Technologies_. ACL, Denver, Colorado, 1055–1065.
  * White (2019) Nigel D White. 2019\.  Ending the US embargo of Cuba: international law in dispute.  _Journal of Latin American Studies_ 51, 1 (2019), 163–186.
  * Xiong et al. (2019) Ying Xiong, Moonhee Cho, and Brandon Boatwright. 2019.  Hashtag activism and message frames among social movement organizations: Semantic network analysis and thematic analysis of Twitter during the #MeToo movement.  _Public relations review_ 45, 1 (2019), 10–23.
  * Yang et al. (2020) Yinfei Yang, Daniel Cer, Amin Ahmad, Mandy Guo, Jax Law, Noah Constant, Gustavo Hernandez Abrego, Steve Yuan, Chris Tar, Yun-hsuan Sung, Brian Strope, and Ray Kurzweil. 2020\.  Multilingual Universal Sentence Encoder for Semantic Retrieval. In _Proc. of the 58th Annual Meeting of the ACL_. ACL, Online, 87–94.

## Appendix A: Linear Model Details

Type | Name |  Description
---|---|---
Variables | $\mathsf{minedge}$ |  Represents the lowest coherence value over all active edges.
| $\mathsf{event}_{i}$ |  Represents the presence of event $i$ in the narrative map.
| $\mathsf{edge}_{ij}$ |  Represents the strength of the connection between events $i$ and $j$ in the narrative map.
| $\mathsf{cluster}_{k}$ |  Represents the average presence of cluster $k$ in the narrative map.
Data-based Parameters | $\mathsf{num\mathchar 45\relax clusters}$ |  Represents the number of clusters of events.
$\mathsf{membership}_{ijk}$ |  Represents whether the connection of events $i$ and $j$ belongs to cluster $k$. Defined as $\sqrt{\mathsf{membership}_{ik}\cdot\mathsf{membership}_{jk}}$—the geometric mean between the probabilities of the events belonging to cluster $k$.
$\mathsf{coherence}_{ij}$ |  Represents the coherence score of the connection between events $i$ and $j$. Defined as $\sqrt{\mathsf{angular\mathchar 45\relax similarity}_{ij}\cdot\mathsf{cluster\mathchar 45\relax similarity}}_{ij}$—the geometric mean between the angular similarity of the underlying event embeddings and the clustering similarity based on comparing the cluster membership probability distributions of each event using Jensen-Shannon divergence.
$\mathsf{score\mathchar 45\relax percentile}_{i}$ (*) |  Represents the score percentile of event (submission) $i$ in the community.
$\mathsf{upvote\mathchar 45\relax ratio}_{i}$ (*) |  Represents the upvote ratio of event (submission) $i$ in the community.
$\mathsf{acceptance}_{ij}$ (*) |  Represents the acceptance score of the connection between events $i$ and $j$. Defined as $\sqrt{\mathsf{score\mathchar 45\relax percentile}_{i}\cdot\mathsf{score\mathchar 45\relax percentile}_{j}}\cdot\sqrt{\mathsf{upvote\mathchar 45\relax ratio}_{i}\cdot\mathsf{upvote\mathchar 45\relax ratio}_{j}}$—the product between the geometric means of submission score percentiles and upvote ratios.
User-defined Parameters | $K$ |  Defines the expected length of the main storyline in the narrative map.
$\mathsf{mincover}$ |  Defines the minimum average coverage over all clusters in the narrative map.
$\mathsf{minscore}$ (*) |  Defines the minimum average acceptance over all events (submissions) in the narrative map.

Table 3. Description of the variables and parameters of the proposed
extraction algorithm. All elements take values between $0$ and $1$, except $K$
and $\mathsf{num\mathchar 45\relax clusters}$, which are positive integers.
Most elements of the model are based on the extraction algorithm of Keith and
Mitra (2021). The (*) symbol represents new additions to the linear problem
formulation designed to incorporate community acceptance in social media into
the model.

## Appendix B: Narrative Map Figures

Figure 3. Narrative map for r/Conservative. Figure 4. Narrative map for
r/GenZedong. Figure 5. Narrative map for r/worldnews. Figure 6. Narrative map
for r/politics.