# The Face of Populism: Examining Differences in Facial Emotional Expressions
of Political Leaders Using Machine Learning

Sara Major  Department of Sociology, University of Novi Sad, Serbia,
<EMAIL_ADDRESS>Aleksandar Tomašević  Department of Sociology,
University of Novi Sad, Serbia<EMAIL_ADDRESS>
###### Abstract

Populist rhetoric employed on online media is characterized as deeply
impassioned and most often imbued with strong emotions. The aim of this paper
is to empirically investigate the differences in affective non-verbal
communication of political leaders. We use a deep-learning approach to process
a sample of 220 YouTube videos depicting political leaders from 15 different
countries, analyze their facial expressions of emotion and then examine
differences in average emotion scores representing the relative presence of 6
emotional states (anger, disgust, fear, happiness, sadness, and surprise) and
a neutral expression for each frame of the processed YouTube video. We observe
statistically significant differences in the average score of expressed
negative emotions between groups of leaders with varying degrees of populist
rhetoric. Overall, our contribution provides insight into the characteristics
of non-verbal emotional expression among political leaders, as well as an
open-source workflow for further computational studies of their affective
communication.

##  1 Introduction

In the growing body of literature on populism, the category of emotion has
most frequently been utilized in discrete ways to describe its emergence and
tenacity (Demertzis 2006)—by reference to the disillusioned silent majority,
heightened emotions surrounding political leaders, and emotionally charged
divisions between the people and the elite (Moffitt & Tormey 2014, Mudde 2017,
Norris & Inglehart 2019). Compared to pluralist parties, who are commonly
portrayed as more affectively neutral, populist rhetoric is most often
characterized as deeply impassioned and imbued with strong (negative) emotions
(Bonansinga 2020a, Salmela & von Scheve 2018). Although the nature of the
relationship between emotionality and populist discourse has rarely been
subject to systematic empirical scrutiny (Widmann 2021), this link has become
one of the main focal points of research on political communication in recent
years. Comprehensive research on the matter is thus crucial for gaining
insight into the extent to which populist narratives and performances are
rooted in and shaped by emotion, and by extension, how emotion may be utilized
as a tool for building and maintaining political support (Nguyen 2019).

In the past, studies of populist communication have generally focused on its
verbal elements, by analyzing the affective content of the words used in oral
or written addresses of political leaders (Bobba 2019, Breeze 2019, 2020,
Cochrane et al. 2022, Jacobs et al. 2020, Martella & Bracciale 2022, Widmann
2021). While advantageous in its own way, this type of analysis can pose a
challenge to researchers primarily due to language barriers and the fact that
written text usually contains only a small amount of words that have an
inherent affective connotation (Widmann 2021). At the same time, currently
available sentiment analysis tools are predominantly based on and developed
for the English language, while the output for other languages is often of
lower quality (Mohammad 2016). These circumstances therefore limit the
possibility of result generalization and cross-cultural comparison.

Visual analyses, although insufficiently represented in the given field of
research, may provide further insight into the characteristics of affective
populist communication as they are not hindered by the same limitations as
text-based studies (Moffitt 2022b) and allow for analysis of the visual
rhetoric constructed and utilized by political leaders (Mendonça & Caetano
2021). Within the scope of visual research on political emotion display, two
main schemes of analysis have shown to be especially valuable: the ethological
approach (Masters et al. 1986) and the Facial Action Coding System (FACS)
(Ekman & Friesen 1976).

Studies within the former largely focus on the analysis of political
candidates’ non-verbal representations in news coverage, their displays during
political campaigns and debates, as well as the impact of different forms of
representation on viewers and success in political elections (Bucy et al.
2020, Bucy & Grabe 2007, 2008, Grabe & Bucy 2009, Sullivan & Masters 1988).
Conversely, the Facial Action Coding System allows for a more in-depth
analysis of the (micro)expressive behavior of political leaders during
speeches and public performances. Due to its complexity, Ekman’s
classification system has been applied in a limited number of studies (Stewart
et al. 2009, 2015, Stewart & Ford Dowe 2013) but as the process of visual
analysis gradually becomes automated (Joo et al. 2019) this research method is
becoming more accessible to researchers from various fields of interest (Ash &
Caliskan 2022, Bossetta & Schmøkel 2023).

This study expands on the existing framework and applies a computer vision
approach for detecting emotions from the facial expressions of political
leaders. Since the introduction of the FER dataset (Goodfellow et al. 2013),
different machine learning algorithms have been developed for solving the
challenge of facial expression recognition, which essentially requires
performing two tasks: (1) detection of facial expression, and (2) emotion
recognition from the detected facial expressions. We develop a reproducible
and principled computational workflow in Python for the analysis of video
content of political leaders’ public performances. Our work relies on the
existing Python library _fer_ (Shenk et al. 2021) which bundles a pre-trained
convolutional neural network (CNN) Keras models for face detection (Zhang et
al. 2016a, based on) and emotion recognition (Arriaga et al. 2017, 2020).

We find that populist leaders on average express more negative emotions and
that the emotionally neutral facial expression is less frequently detected in
their public appearances compared to non-populist leaders. Overall, our work
points towards a new direction in computational studies of the affective
aspects of populist political communication and provides a roadmap for
overcoming the limitations of the current approach in future studies.

##  2 Populism as performance

Defining populism represents a challenging task for researchers as there is
barely any consensus on what such a definition should (or should not) include,
and the concept itself becomes increasingly muddled as the term is used to
describe a growing number of phenomena (Gidron & Bonikowski 2013). Most
conceptualizations of populism can, however, be classified (Rovira Kaltwasser
et al. 2017) into one of three main approaches: (1) the ideational approach
(Mudde 2004), (2) the discursive approach (Norris & Inglehart 2019, Ostiguy
2017) and (3) the political-strategic approach (Weyland 2017). For the
purposes of this study, we adopt the discursive approach, which conceives
populism primarily as a rhetorical style and focuses on the ways in which
populist messages are framed and communicated.

From this perspective, the populist disposition has widely been characterized
by the use of “dramatization, polarization, moralization, directness,
ordinariness, colloquial and vulgar language” (Engesser et al. 2017), and low-
order appeals within which leaders “use a language that includes slang or
folksy expressions and metaphors, are more demonstrative in their bodily or
facial expressions as well as in their demeanor, and display more raw,
culturally popular tastes” (Ostiguy 2017). Because it represents a form of
symbolic resistance to the more formalized and composed self-representation of
mainstream pluralist leaders, i.e., the “establishment” and “political elite”,
the populist performance is subversive to the normative communication
structures of political dialogue. As such, it appeals to “common sense” (Bucy
et al. 2020) and claims to give a voice to “the ordinary people” (Inglehart &
Norris 2016).

This mode of communication has shown to be highly effective as a means of
garnering media attention and cultivating a dedicated base of
supporters—especially in the new media environment—and populist leaders seem
to be overall more adept at capitalizing on it than other, non-populist
political figures (Bobba 2019, Davidson & Enos 2022). Since digital media
hinges on the attention economy (Wells et al. 2020) and thus tends to favor
“dark participation” (Larsson 2022), populist leaders and political parties
may fare well online due to the transgressive nature of their self-
representation and the simplification, emotionalization, and negativity
inherent in populist messaging (Bucy et al. 2020, Engesser et al. 2017).
Social media is a particularly convenient tool for the proliferation of
populist messages and imagery as it allows political actors to circumvent the
mainstream media, reach large audiences, and communicate and interact directly
with ‘the people’ (Lalancette & Raynauld 2019, Moffitt 2022a). At the same
time, online platforms allow for an instant audience response, as the
“feedback loop between producer and consumer is much more intense and rapid
than with television and radio (Finlayson 2022). This form of ‘mediatized
populism’ (Mazzoleni 2014) allows for aspecific type of political performance,
one which is largely unmitigated and relies heavily on portraying political
leaders favorably to (online) audiences by framing them as expressive,
dominant, and disruptive figures who are in tune with “the will of the
people”.

In recent years, much attention has been paid to the strategies of (affective)
populist communication online, especially through analyses of verbal content
shared by populist leaders and parties on Twitter and Facebook (e.g. Bracciale
& Martella 2017, Gründl 2022, Jacobs et al. 2020, Martella & Bracciale 2022,
Widmann 2021). Given the rise of visual communication in politics and the
rapid developments of Web 2.0 practices (Grabe & Bucy 2009, Larsson 2022,
Messaris 2019), our primary aim was to expand the scope of research on the
affective self-representation of populist leaders online by examining the
characteristics of their non-verbal emotional communication. The facial
expressions of political leaders are a particularly valuable area of research
in this context, since digital media provide a breadth of material for
analysis—well-lit, high-quality close-up shots of political leaders are now
more prevalent than ever. We used this abundance of available data sources to
our advantage and processed 220 YouTube videos uploaded to official accounts
of political parties and their leaders, amounting to a total of 77 hours of
video material. Our analysis examines potential differences in facial
expressions between populist and non-populist political leaders during public
performances, and more specifically, the characteristics of their negative
emotional expressions and neutrality in emotional stance (i.e., absence of
emotional expression).

##  3 Data & Method

To create our sample, we first had to differentiate and categorize political
parties (and, by extension, their leading figures) as either populist or
pluralist. For this purpose, we used data from the Global Party Survey (GPS)
(Norris 2020), an expert survey from 2019 whichoffers an overview of key
ideological and issue positions for 1043 parties from 163 countries worldwide.
Among other data, such as ideological framework and party size, the survey
also offers an estimate of the degree to which existing parties employ
populist rhetoric. In the questionnaire design, populist rhetoric is defined
as “language which typically challenges the legitimacy of established
political institutions and emphasizes that the will of the people should
prevail” (Norris 2020), whereas pluralist rhetoric “rejects these ideas,
believing that elected leaders should govern, constrained by minority rights,
bargaining and compromise, as well as checks and balances on executive power”
(Norris 2020). Populist rhetoric scale in (V8 in GPS dataset) is based on
expert assessments of rhetorical style given these parameters. Variable
Type_Populism categorizes this scale into four ordinal groups: strongly
pluralist, moderately pluralist, moderately populist, and strongly populist.

From the GPS dataset, fifteen countries with competitive popular elections
were chosen: Argentina, Australia, Brazil, Croatia, France, Hungary, India,
Italy, Japan, the Netherlands, Serbia, South Africa, Turkey, the United
Kingdom, and the United States. This sample, although limited, provides
somewhat broad coverage of various cultures and political systems globally.
Three parties with varying degrees of populist rhetoric—strongly populist,
strongly pluralist, and either moderately populist or moderately
pluralist—were selected from each country. We primarily focused on minor and
major political parties as defined by the GPS variable categorizing parties
based on their share of the vote in contests for the lower house of the
national parliament. Fringe parties (less than 3% of the vote) were generally
excluded from the sample since their audience reach and influence is
considerably less than that of their larger counterparts competing at the same
scale in their respective countries. Likewise, due to this criteria, only two
parties from the US were included in the sample.

We then identified one leader per party who has held a prominent political
position over the past decade (up to 2019, covered by the GPS), and found
their official YouTube channel or their party’s official YouTube channel. From
these channels, five videos were selected for each leader based on the
following criteria:

  1. (1)

_View count_. Videos that reached a larger audience were given priority over
videos that did not perform comparatively as well on the platform. View count
was used as a filtering mechanism to single out videos with large and
potentially diverse audiences, because the most viewed videos on the official
YouTube channels of major political parties usually refer to topics of wider
public interest. This is partially due to the “popularity bias” on YouTube,
meaning it has a tendency to recommend political content that aligns with the
interests of the majority (Heuer et al. 2021).

  2. (2)

_Unmediated representation_. Only direct and unmediated representations of the
given political leaders were included, i.e., videos which were not in any way
previously modified by third parties such as news outlets. These consisted
mainly of speeches, press conferences, ads, and promotional videos. This was
done to ensure that the given results would reflect the leaders’ self-
representation, rather than their representation in news coverage.

  3. (3)

_Consistent framing_. A preference was given to videos where the face of the
leader was in focus for an extended period of time and which did not
frequently switch between disparate camera angles. This allowed for an overall
consistent frame of reference for each given video.

  4. (4)

_Video quality_. High-quality, well-lit videos were selected whenever possible
so as to improve analysis accuracy.

This selection process resulted in a sample of 220 videos, representing 44
political leaders from 15 different countries. The sample is not gender-
balanced as only five female leaders are present, which reflects the global
underrepresentation of women in leadership positions within major political
parties. As a result, our analysis does not account for gender, which we
acknowledge as a limitation of our approach. We further discuss this
limitation in the closing paragraphs of the paper. Additionally, because the
sample is limited by the timeframe of the currently available GPS dataset, it
hasn’t captured more recent global developments on the populist-pluralist
spectrum (e.g. Giorgia Meloni and the Brothers of Italy movement or the
Servant of the People party in Ukraine are not present in the dataset). Given
these limitations, our sample is conceived as a diverse snapshot of the state
of populist and pluralist rhetoric in the second decade of the 21st century
and does not capture the full scope or evolution of populism today.

We processed each video in the sample by extracting 300 frames uniformly
distributed over the length of the video, which were the basis for our main
analysis. To improve the robustness of the overall approach, we also included
an alternative frame processing approach in the supplementary material, based
on processing every 50th frame of the video. Given that the majority of the
YouTube videos have 24–30 frames per second, this is equivalent to selecting a
frame every 1.67–2.00 seconds, which is a reasonable time window if we expect
to capture potential changes in facial expression. The average number of
frames per video extracted using this approach was 273.03, which makes the
resulting time-series of emotion scores comparable in length with the first
approach.

For each selected frame of every video in the sample the following procedure
is applied. First, face detection is applied to the extract frames using a
multi-task cascaded framework based on convolutional neural networks (CNN)
(Zhang et al. 2016b). This framework is suitable for detecting faces in videos
because it performs simultaneous face detection and alignment, which is
important as we cannot expect that political leaders will be looking straight
at the camera in each extracted video frame. This approach was shown to have a
95% true positive rate and up to 85.1% precision for face detection in easy
data sets containing well-lit frontal face images. In our approach, the
minimum required size for a face to be detected in the frame of the video was
50x50 pixels, which is larger than the input size of images in the training
data. This ensures that small faces in the background of the video (e.g. the
audience) are not detected. We therefore suppose this face-detecting task to
be easier than what the CNN is trained for, since the faces of the leaders are
comparatively large relative to the size of the frame. After face detection
was applied to all frames, there were videos in which multiple faces were
detected throughout the frame. These videos were manually checked to confirm
which of the detected faces corresponds to the selected political leader.

The second step is emotion detection within the verified detected face box. We
relied on implementing the mini-Xception architecture of CNN (Arriaga et al.
2017) aimed at detecting 6 basic facial expressions of emotion, as well as a
neutral expression. Similar to the face detection approach outlined above,
this implementation is aimed at real-time emotion detection from video input
for robotic applications. This model achieves 66% accuracy (see Arriaga et al.
2017, p. 3 for more details) on the FER2013 dataset (Goodfellow et al. 2013)
which is a standard benchmark for emotion detection. The dataset consists of
30,000 small images (48x48 pixels) showing diverse facial expressions of
different emotions. 111FER2013 was constructed using Google Images API queries
for emotion-related keywords like “blissful” and “enraged”, combined with
queries introducing diversity in gender, age and ethnicity represented in the
images. Images were then labeled by human coders (Goodfellow et al. 2013, pp.
3-4). Unfortunately, no information is provided by the authors of the dataset
regarding the number of human coders nor their background, or the labeling
methodology applied to extract training labels for every image included in the
FER2013 dataset. Each emotional expression has approximately 6000 images,
except disgust, which is only present in 600 images. According to the authors,
human classification accuracy is in the range of $65\%\pm 5\%$, but no
additional information about the sample size, number of coders, or detection
procedure is given. While mini-Xception performs worse than the state-of-the-
art methods (up to 93.7% with additional training data, see Ezerceli & Eskil
2022), it stays within range of reported human classification accuracy and is
a fast, memory-saving, and open-source solution that can be deployed on
personal computers and therefore has more potential for widespread use in
computational communication research. Additionally, since we were motivated to
investigate mean trends in expressed emotion scores with a large number of
frames for each video, single-frame accuracy was not a primary concern for
this study.

The result of the emotion detection procedure on a single video frame is an
array of 7 emotion scores, each representing the predicted proportion of an
expressed emotion. Each score is matched with one of 7 labels: “angry”,
“disgust”, “fear”, “happy”, “sad”, “surprise” and “neutral”. The lowest
possible score indicating the absence of a particular emotion is 0, while the
highest score, which indicates a “clean” expression of a single emotion, is 1.
In a statistical sense, the resulting data are compositional as they do not
carry absolute, but rather relative information about the expressed emotions.
This data is organized as a data frame consisting of 7 columns and
approximately 300 rows, storing emotional scores for each selected frame of
the video.

Because we were primarily interested in neutral emotional expressions and
expressions of negative emotions, we created an additional variable which sums
the scores of negative emotions: anger, fear, disgust, and sadness. Figure 1
shows examples of faces with a high detected score of negative emotion and a
high score of neutral expression.

Figure 1: Example frames showing a high detected score of negative emotions
(top: angry 0.78, disgust 0.00, fear 0.09, happy 0.00, sad 0.07, surprise
0.00, neutral 0.04) and a high detected score of neutral expression (bottom:
angry 0.07, disgust 0.00, fear 0.04, happy 0.00, sad 0.02, surprise 0.08,
neutral 0.79).

The entire analysis was performed in Python and the code is publicly available
in the GitHub repository: https://github.com/atomashevic/face-of-populism. The
code is organized into three sections. The data processing part of the code
can be used to reproduce emotion score data files from the list of YouTube
URLs for the videos included in the sample. It can easily be modified to
expand the sample to a larger number of videos. Face and emotion detection
relies heavily on the _fer_ Python library and that part of the code is
presented in the Jupyter notebook. Finally, we have data analysis scripts for
performing data wrangling and statistical tests as well as producing Figures
2, 3, and 4.

##  4 Results

In the entire sample, there are 203 successfully processed videos using the
approach of selecting 300 frames uniformly distributed across the entire
length of the video clip. For 17 videos, multiple faces were detected without
consistent classification of each person as a unique face, and these videos
were discarded from the analysis. Pluralist leaders are present in 96 videos
(31 strongly pluralist and 65 moderately pluralist) and populist leaders are
present in the remaining 107 videos (69 strongly populist and 38 moderately
populist). For every video, the emotion recognition procedure returns the
score of the 6 detected emotions (anger, disgust, fear, happiness, sadness,
surprise) as well as the score of the neutral expression for each of its
frames. These scores sum up to 1 and they can be viewed as proportions of each
emotion present in the compound facial expression detected in the frame. For
each video, we calculated the mean score of each emotional state for all
frames, as well as the mean of neutral expression, and the mean of negative
emotional expression for the entire video. Descriptive statistics of these
values are presented in Table 1.

| Mean | SD | Min. | Median | Max.
---|---|---|---|---|---
Anger | 0.227 | 0.134 | 0.034 | 0.200 | 0.726
Disgust | 0.003 | 0.008 | 0.000 | 0.001 | 0.066
Fear | 0.127 | 0.091 | 0.014 | 0.099 | 0.548
Happiness | 0.077 | 0.083 | 0.000 | 0.051 | 0.414
Sadness | 0.206 | 0.110 | 0.023 | 0.181 | 0.568
Surprise | 0.031 | 0.037 | 0.000 | 0.020 | 0.273
Neutral | 0.328 | 0.169 | 0.020 | 0.300 | 0.807
Negative emotions | 0.563 | 0.185 | 0.156 | 0.570 | 0.964
Table 1: Descriptive statistics for extracted emotions for the entire sample,
$N=203$ videos

Results suggest that negative emotions are more frequently expressed in the
entire sample, with anger (0.227) as the emotion having the highest average
score, followed by sadness (0.206) and fear (0.127). As a consequence, the
average proportion of negative emotions is greater than 0.5, which means that
on average negative emotions are detected more frequently than positive
emotions (e.g., happiness) and neutral expressions combined. There are clear
issues with the detection of disgust and surprise, where a low maximum level
suggests low positive predictive value for these emotions. However, our focus
is on differences in the expression of negative emotions and in scores of
neutral states. Aggregating negative emotions enables us to surpass the false
positive issues within the group of negative emotions (e.g., expression of
fear being classified as sadness, disgust as anger, Khaireddin & Chen 2021)
and minimizes the impact of the low positive predictive value of disgust and
surprise.

Looking at the maximum values of average detected scores, the largest average
scores of specific negative emotions were found in videos of populist leaders.
The highest proportion of anger was detected in a video of Andrej Plenković
(HDZ, Croatia), disgust in a video of Narendra Modi (BJP, India), fear in a
video of Saša Radulović (DJB, Serbia) and sadness in a video of Donald Trump
(GOP, USA). The video with the highest aggregate proportion of negative
emotions, where one of the negative emotions is dominant in almost every
analyzed frame, is the video titled “Special Message from President Trump”,
published on December 23, 2020. On the other hand, the highest average score
of neutral expression is found in a video of Yukio Edano (CDP, Japan), whose
party belongs, in contrast with previous examples, to the moderately pluralist
group.222This does not imply that, on average, there is a tendency for
Japanese leaders to be more neutral. In fact, in Japan, we observe the same
tendency as seen in the entire sample—the speeches of Shinzo Abe (LDP) from
the populist group have the highest average score of negative emotions (0.58),
while Natsuo Yamaguchi (NKP) from the moderately pluralist group has the
lowest average score (0.30). Conversely, the average scores of neutrality show
the opposite trend. These results are not extreme compared to the sample
distributions shown in Figure 2.

To further examine differences between pluralist and populist leaders, we used
the GPS measure of populist rhetoric in a binary way to split the sample into
groups of pluralist (_Party Populism_ 1-2) and populist (_Party Populism_ 3-4)
leaders. The distribution of the average scores of negative emotion and
neutral expression is shown in Figure 2.

Figure 2: Distribution of average scores of negative emotions (left) and
neutral expression (right) of videos featuring populist and pluralist leaders.

Overall, all average values have a large spread in both groups, with pluralist
leaders having several videos with less than a 0.2 average negative emotion
score and more than a 0.75 average score of neutral expression. In terms of
central tendency, we see denser grouping of pluralist leaders between the 0.4
and 0.6 negative emotions score, while populists tend to cluster in areas
between 0.6 and 0.8. In terms of neutral expression, we have a thicker cluster
of points for populists in the range between 0.2 and 0.4. Figure 1 suggests
that, for populist leaders, we are more likely to observe lower average scores
of neutral expression and higher average scores of negative emotion. The
reverse tendency can be observed in the case of pluralist leaders. Negative
emotions are dominant in 73.8% of populist videos, compared to 45.8% of
pluralist videos.

The mean average score of negative emotions in the populist group is 0.616
(95% CI 0.584–0.649), compared to the 0.500 mean average score for pluralists
(95% CI 0.464–0.537). The difference between the two groups is statistically
significant ($t=4.691$, $p=0.000005$). This finding suggests that pluralists
generally tend to have an average score of negative emotions around 0.5, which
roughly translates to the situation where negative emotion is the prevailing
expression in one half of the video (in fact, that percentage is 46.1%
compared to 66.7% for populists; see Table 3 in the Supplement). On the other
hand, for populists, more than one half of each video where they appear is
dominated by negative emotions.

Figure 3: Raincloud plot showing the distribution of average scores of
negative emotions across four degrees of populism. Every data point is
presented as a colored dot and above them boxplots and density plots are
displayed. The dashed vertical bar indicates the score of 0.5.

The mean average score of neutral expression for populists is 0.289 (95% CI
0.260–0.318), compared to the 0.374 mean for pluralists (95% CI 0.338–0.409).
While this difference in means is statistically significant ($t=-3.636$,
$p=0.00035$), it is smaller than the one observed in the case of negative
emotions.

To investigate the robustness of these differences, we took into account the
differences between moderate and strong pluralists/populists provided by the
GPS. The distributions of average scores in these four groups are compared and
shown in Figures 3 and 4.

Figure 4: Raincloud plot showing the distribution of average scores of neutral
expression across four degrees of populism. Every data point is presented as a
colored dot and above them boxplots and density plots are displayed. The
dashed vertical bar indicates the score of 0.5.

In the case of negative emotions (Figure 3), we observe four flat
distributions with thick tails. However, the median point of all distributions
progressively moves towards more negative emotions as we move from strong
pluralists to strong populists. What is apparent in these differences is the
low frequency of videos featuring populist leaders where the average score of
negative emotions is below 0.5. It is worth pointing out that examples of
extremely high scores of negative emotions are present also in pluralist
groups.

Similarly, in the case of neutral expression (Figure 4), we have flat
distributions (except in the case of moderate populists) with long right tails
containing videos of leaders who maintain a neutral expression throughout most
of their recorded appearance. Again, median values move progressively towards
lower neutrality as we move from pluralists towards populists, except that no
large difference can be seen when comparing strong and moderate populists. In
the case of strong populists, however, we have a larger number of videos where
the neutral expression is almost completely absent.

Overall, the results of one-way ANOVA suggest that the differences between the
group means are statistically significant both in the case of negative
emotions ($F=7.109$ $p=0.000152$), with strong effect size measured by eta-
squared (0.104), and neutral expression ($F=5.625$ $p=0.001038$), with
moderate effect size (0.084). Results from Tukey HSD post-hoc tests reveal
significant differences between leaders coming from parties employing strong
populist rhetoric and leaders coming from pluralist parties (mean difference
$0.1559$ $p=0.0005$ in comparison with strong pluralists, and mean difference
$0.1148$ $p=0.0029$ in comparison with moderate pluralists). For neutral
expression we find the opposite case, as strong pluralists differ from both
groups of populists (mean difference $0.135$, $p=0.0056$ for moderate
populists, and mean difference $0.129$ $p=0.0032$ for strong populists).

Overall, these results reveal differences in the average scores of negative
emotion and neutrality between populist and pluralist leaders on a modest
sample. These differences are more pronounced when we focus on candidates
coming from parties who strongly rely on either pluralist or populist
rhetoric, with strong pluralists on average favoring more neutral expressions,
while strong populists more frequently express negative emotions—meaning that,
for the majority of their public appearance, we can observe facial expressions
of negative emotions.

##  5 Discussion

The present study examines facial emotional expressions of populist and non-
populist political leaders using a deep-learning-based computer-vision
algorithm to investigate differences in their expressions of negative emotions
(anger, disgust, fear, and sadness) and neutral stances, i.e., facial
expressions which lack emotional disposition. Our findings indicate that
populist leaders tend to express negative emotions more frequently than non-
populist leaders, and conversely, that neutral expressions are less common in
their public performances than they are in those of more conventional,
pluralist political figures. These findings align with the theoretical
framework on the characteristics of affective populist communication
(Bonansinga 2020b, Salmela & von Scheve 2018, Engesser et al. 2017), as well
as with previous, mostly text-based research which indicates that populist
leaders employ language which is associated with negative emotion at a higher
rate than mainstream, pluralist politicians (Breeze 2019, Caiani & Di Cocco
2023, Macagno 2022, Martella & Bracciale 2022, Widmann 2021).

Overall, the results point to the existence of a style of affective, non-
verbal communication which is more commonly found among political leaders
whose rhetoric is populist-leaning. This provides support for the idea that
populism is to an extent performative in nature. As such, because it is
subversive in relation to established political conventions, the populist
performance is especially well-suited for the environment of social media
where the principle of “any press is good press” often applies—that is, the
quantity of audience attention bears more weight than its quality (Wells et
al. 2020). On one hand, the populist use of negative emotional displays in
video may therefore be a mechanism through which they maintain relevance,
considering that controversial content online ordinarily generates more
engagement among consumers (Larsson 2022). On the other, populists’ facial
expressions can provide affective heuristics for audiences, influencing their
evaluation of candidates’ political competence (Boussalis & Coan 2021) and
allowing them to quickly reduce complex political issues into simple,
emotional interpretations by visually framing them as negative. Further
analysis of the role of non-verbal cues in political communication—such as
facial expressions of emotion—would be beneficial to gaining a better
understanding of the populist performance as a whole.

Given the exploratory nature of the study and its reliance on machine learning
algorithms for emotional detection, there are several notable limitations with
significant impact on the results presented in this paper. There is a group of
limitations inherent to the way we constructed the sample, and another related
to the machine learning approach to detecting emotions from facial
expressions.

Firstly, the sample of YouTube videos is not globally representative and does
not provide ample coverage of the global political spectrum in regard to the
use of populist or pluralistic rhetoric. As we have shown in the supplementary
analysis (see Supplement C: Country-level differences), the differences
between populist and pluralist leaders are not consistent in all investigated
countries. Larger research infrastructure is required to process a more
substantial number of videos, leaders and countries and the current
computational toolset needs to be extended to enable seamless automatic
processing of videos. This also relates back to the gender imbalance of the
current sample. While it is reasonable to hypothesize that gender influences
emotional expression among political leaders (Boussalis et al. 2021), in
thiscase, including more women without significantly increasing the overall
size of the sample and the number of leaders would mean focusing only on
countries where there is at least one female leader of a non-fringe political
party. This creates the risk of introducing bias towards Western countries in
the sample and disregarding the global scope.

Secondly, a key component of our analysis is the measure of populist rhetoric
supplied by the Global Party Survey database (Norris 2020) which significantly
limits the timeframe of the analysis (no leaders, parties and videos after
2019 were taken into consideration), but also imposes the inherent limits of
an expert survey. While the GPS is the only available source of multi-
dimensional measurements of populist rhetoric on a global scale, the experts
surveyed may operate within different understandings of what populist rhetoric
is. This is particularly challenging in developing countries, where fewer
experts answered the survey, as well as in autocratic regimes where experts
may not have been able to provide a fair assessment of the (ruling) political
parties (or refused to do so). By focusing on large countries with free
elections, we tried to mitigate some of the risks, but there is the
possibility that some measurements used in this study reflect expert biases,
especially since we used a single, aggregate measure of populist rhetoric.
Future studies should aim for more granularity and robustness in the
measurement of populist rhetoric, which may be obtained by following a text-
as-data approach and obtaining the measures using computational and machine
learning tools, where expert survey scores provide external validation
(Jankowski & Huber 2023).

Furthermore, since we lacked the resources necessary to process every
political speech from each party’s official YouTube channel, the first
criterion of selection is the view count of the videos. Our expectation is
that this metric serves as a proxy for the relevance of the speech content to
a wider audience over a longer period of time. However, since we have no
information about the audience, it is possible we are observing a demand-side
effect, where audiences of certain political parties might prefer more
emotional content. This could introduce a specific bias in the sample.

One of the major limitations of our current workflow is that the video-
processing algorithm does not focus on one singular face and does not
consistently label the same faces between frame jumps, which requires
researchers to select videos with consistent framing and a small number of
different faces. Solving this problem will allow for faster and more reliable
collection of larger samples.

The next major limitation is a result of the machine learning model deployed
for emotion recognition. As with all models trained on FER2013 data, low
classification accuracy is found for disgust and fear, which is a consequence
of their lower frequency in the training set. The output of the emotion
recognition procedure was not validated by human coders in our study and our
confidence in its accuracy is based on previous work (Khaireddin & Chen 2021).
However, because we aggregate negative emotions and focus on average aggregate
scores, we do not expect our results to be significantly distorted by possible
misclassifications of emotions in individual frames—this is demonstrated by
the robustness of results shown by comparing two different methods of frame
selection, and therefore different frames being analyzed by the emotion
recognition algorithm.

Overall confidence in the presented results would surely be higher if we based
our analysis on a computer-vision approach which was shown to have higher
accuracy, at least on FER2013 data. One of the possible alternatives would be
to follow recent work (Bossetta & Schmøkel 2023, Schmøkel & Bossetta 2022) and
use proprietary API such as Amazon’s Rekognition. Nonetheless, the aim of this
paper was to introduce an open-source approach focusing on free computing
resources and the deployment of machine learning algorithms on systems that
are accessible to individual researchers in social sciences, including
graduate students without significant institutional support for computational
work.

This study was focused on exploring basic mean-level differences between the
given groups. It is worth noting that the full output of our emotion detection
workflow includes time series data which are suitable for multivariate
analysis. Even though time series are compositional in nature, they can be
transformed (Greenacre 2021) so that traditional time series methods can be
applied to explore affective dynamics (Ryan et al. 2023), find additional
differences in relation to other factors (such as type of public address, the
proximity of elections, ideology,etc.) and cluster political leaders according
to patterns of correlations in time-series. Furthermore, multivariate change
point detection methods (James & Matteson 2015) could be used to identify the
key moments in speech where significant changes in emotional expression occur,
which can further be linked to textual analyses of transcripts. This would
allow researchers to identify the rhetorical elements which trigger switches
in the emotional dynamics of leaders’ political addresses.

As computational studies of political communication rapidly embrace machine
learning tools and algorithms, this study offers an extension of an open-
source toolbox for the analysis of affective non-verbal political
communication. Analyses presented in this paper are based on a novel dataset
which is only a starting point for further studies, and which will hopefully
expand in the future to encompass a more diverse sample of political leaders
in videos obtained from different platforms. Gaining this kind of insight into
the visual aspects of emotional political communication, coupled with the
existing analyses of its verbal elements, holds the potential for a more
comprehensive understanding of how leaders convey political messages, persuade
audiences, and present themselves to the public using digital media platforms.

##  6 Data availability

Data files are stored as CSV files. Processed data used for both the main and
supplementary analyses can be found at: https://github.com/atomashevic/face-
of-populism. Repository contains a CSV file with a list of URLs of all YouTube
videos used in the analysis as well as Python code that can be used to
reproduce the CSV files with processed data. CSV file extracted from GPS data
is available in the repistory and the original GPS dataset is available at:
https://dataverse.harvard.edu/dataverse/GlobalPartySurvey.

## References

  * (1)
  * Arriaga et al. (2020) Arriaga, O., Valdenegro-Toro, M., Muthuraja, M., Devaramani, S. & Kirchner, F. (2020), ‘Perception for Autonomous Systems (PAZ)’.
  * Arriaga et al. (2017) Arriaga, O., Valdenegro-Toro, M. & Plöger, P. (2017), ‘Real-time Convolutional Neural Networks for Emotion and Gender Classification’.
  * Ash & Caliskan (2022) Ash, E. & Caliskan, C. (2022), ‘Fight or Flight on Fox?: Partisan Fear Responses on U.S. Cable News Shows’.
  * Bobba (2019) Bobba, G. (2019), ‘Social media populism: Features and ‘likeability’ of Lega Nord communication on Facebook’, European Political Science 18(1), 11–23.
  * Bonansinga (2020a) Bonansinga, D. (2020a), ‘Who Thinks, Feels. The Relationship Between Emotions, Politics and Populism’.
  * Bonansinga (2020b) Bonansinga, D. (2020b), ‘Who Thinks, Feels. The Relationship Between Emotions, Politics and Populism’, PArtecipazione e COnflitto 13(1), 83–106.
  * Bossetta & Schmøkel (2023) Bossetta, M. & Schmøkel, R. (2023), ‘Cross-Platform Emotions and Audience Engagement in Social Media Political Campaigning: Comparing Candidates’ Facebook and Instagram Images in the 2020 US Election’, Political Communication 40(1), 48–68.
  * Boussalis & Coan (2021) Boussalis, C. & Coan, T. G. (2021), ‘Facing the Electorate: Computational Approaches to the Study of Nonverbal Communication and Voter Impression Formation’, Political Communication 38(1-2), 75–97.
  * Boussalis et al. (2021) Boussalis, C., Coan, T. G., Holman, M. R. & Müller, S. (2021), ‘Gender, candidate emotional expression, and voter reactions during televised debates’, The American political science review 115(4), 1242–1257.
https://www.cambridge.org/core/journals/american-political-science-
review/article/gender-candidate-emotional-expression-and-voter-reactions-
during-televised-debates/B259E3277178F68F52CCEADF5AE0CD40

  * Bracciale & Martella (2017) Bracciale, R. & Martella, A. (2017), ‘Define the populist political communication style: The case of Italian political leaders on Twitter’, Information, Communication & Society 20(9), 1310–1329.
  * Breeze (2019) Breeze, R. (2019), ‘Emotion in politics: Affective-discursive practices in UKIP and Labour’, Discourse & Society 30(1), 24–43.
  * Breeze (2020) Breeze, R. (2020), ‘Angry tweets: A corpus-assisted study of anger in populist political discourse’, Journal of Language Aggression and Conflict 8(1), 118–145.
  * Bucy et al. (2020) Bucy, E. P., Foley, J. M., Lukito, J., Doroshenko, L., Shah, D. V., Pevehouse, J. C. & Wells, C. (2020), ‘Performing populism: Trump’s transgressive debate style and the dynamics of Twitter response’, New Media & Society 22(4), 634–658.
  * Bucy & Grabe (2007) Bucy, E. P. & Grabe, M. E. (2007), ‘Taking Television Seriously: A Sound and Image Bite Analysis of Presidential Campaign Coverage, 1992-2004’, Journal of Communication 57(4), 652–675.
  * Bucy & Grabe (2008) Bucy, E. P. & Grabe, M. E. (2008), “‘Happy warriors” Revisited: Hedonic and agonic display repertoires of presidential candidates on the evening news’, Politics and the Life Sciences 27(1), 78–98.
  * Caiani & Di Cocco (2023) Caiani, M. & Di Cocco, J. (2023), ‘Populism and emotions: a comparative study using machine learning’, pp. 1–16.
  * Cochrane et al. (2022) Cochrane, C., Rheault, L., Godbout, J.-F., Whyte, T., Wong, M. W.-C. & Borwein, S. (2022), ‘The Automatic Analysis of Emotion in Political Speech Based on Transcripts’, Political Communication 39(1), 98–121.
  * Davidson & Enos (2022) Davidson, T. & Enos, J. (2022), ‘Are populist political parties more popular on social media? evidence from europe, 2010-2020’.
https://osf.io/preprints/socarxiv/kdyc6/

  * Demertzis (2006) Demertzis, N. (2006), Emotions and Populism, in S. Clarke, P. Hoggett & S. Thompson, eds, ‘Emotion, Politics and Society’, Palgrave Macmillan UK, London, pp. 103–122.
  * Ekman & Friesen (1976) Ekman, P. & Friesen, W. V. (1976), ‘Measuring facial movement’, Environmental Psychology and Nonverbal Behavior 1(1), 56–75.
  * Engesser et al. (2017) Engesser, S., Fawzi, N. & Larsson, A. O. (2017), ‘Populist online communication: introduction to the special issue’, 20(9), 1279–1292.
https://www.tandfonline.com/doi/full/10.1080/1369118X.2017.1328525

  * Ezerceli & Eskil (2022) Ezerceli, Ö. & Eskil, M. T. (2022), Convolutional Neural Network (CNN) Algorithm Based Facial Emotion Recognition (FER) System for FER-2013 Dataset, in ‘2022 International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME)’, pp. 1–6.
  * Finlayson (2022) Finlayson, A. (2022), ‘YouTube and political ideologies: Technology, populism and rhetorical form’, 70(1), 62–80.
http://journals.sagepub.com/doi/10.1177/0032321720934630

  * Gidron & Bonikowski (2013) Gidron, N. & Bonikowski, B. (2013), ‘Varieties of Populism: Literature Review and Research Agenda’, SSRN Electronic Journal .
  * Goodfellow et al. (2013) Goodfellow, I. J., Erhan, D., Carrier, P. L., Courville, A., Mirza, M., Hamner, B., Cukierski, W., Tang, Y., Thaler, D., Lee, D.-H., Zhou, Y., Ramaiah, C., Feng, F., Li, R., Wang, X., Athanasakis, D., Shawe-Taylor, J., Milakov, M., Park, J., Ionescu, R., Popescu, M., Grozea, C., Bergstra, J., Xie, J., Romaszko, L., Xu, B., Chuang, Z. & Bengio, Y. (2013), ‘Challenges in Representation Learning: A report on three machine learning contests’.
  * Grabe & Bucy (2009) Grabe, M. E. & Bucy, E. P. (2009), Image Bite Politics: News and the Visual Framing of Elections, Series in Political Psychology, Oxford University Press, Oxford ; New York.
  * Greenacre (2021) Greenacre, M. (2021), ‘Compositional Data Analysis’, Annual Review of Statistics and Its Application 8(1), 271–299.
  * Gründl (2022) Gründl, J. (2022), ‘Populist ideas on social media: A dictionary-based measurement of populist communication’, 24(6), 1481–1499.
http://journals.sagepub.com/doi/10.1177/1461444820976970

  * Heuer et al. (2021) Heuer, H., Hoch, H., Breiter, A. & Theocharis, Y. (2021), Auditing the biases enacted by YouTube for political topics in germany, in ‘Proceedings of Mensch und Computer 2021’, MuC ’21, Association for Computing Machinery, New York, NY, USA, pp. 456–468.
https://doi.org/10.1145/3473856.3473864

  * Inglehart & Norris (2016) Inglehart, R. & Norris, P. (2016), ‘Trump, Brexit, and the Rise of Populism: Economic Have-Nots and Cultural Backlash’, SSRN Electronic Journal .
  * Jacobs et al. (2020) Jacobs, K., Sandberg, L. & Spierings, N. (2020), ‘Twitter and Facebook: Populists’ double-barreled gun?’, New Media & Society 22(4), 611–633.
  * James & Matteson (2015) James, N. A. & Matteson, D. S. (2015), ‘Ecp: An R Package for Nonparametric Multiple Change Point Analysis of Multivariate Data’, Journal of Statistical Software 62, 1–25.
  * Jankowski & Huber (2023) Jankowski, M. & Huber, R. A. (2023), ‘When correlation is not enough: Validating populism scores from supervised Machine-Learning models’, Political analysis: an annual publication of the Methodology Section of the American Political Science Association 31(4), 591–605.
https://www.cambridge.org/core/services/aop-cambridge-
core/content/view/8CB3DCBECD1E37872074E7F8A9DE20BA/S1047198722000328a.pdf/div-
class-title-when-correlation-is-not-enough-validating-populism-scores-from-
supervised-machine-learning-models-div.pdf

  * Joo et al. (2019) Joo, J., Bucy, E. P. & Seidel, C. (2019), ‘Automated Coding of Televised Leader Displays: Detecting Nonverbal Political Behavior With Computer Vision and Deep Learning’, International Journal of Communication 13(0), 23.
  * Khaireddin & Chen (2021) Khaireddin, Y. & Chen, Z. (2021), ‘Facial Emotion Recognition: State of the Art Performance on FER2013’.
  * Lalancette & Raynauld (2019) Lalancette, M. & Raynauld, V. (2019), ‘The Power of Political Image: Justin Trudeau, Instagram, and Celebrity Politics’, American Behavioral Scientist 63(7), 888–924.
  * Larsson (2022) Larsson, A. O. (2022), ‘Picture-perfect populism: Tracing the rise of european populist parties on facebook’, 24(1), 227–245.
http://journals.sagepub.com/doi/10.1177/1461444820963777

  * Macagno (2022) Macagno, F. (2022), ‘Argumentation profiles and the manipulation of common ground. The arguments of populist leaders on Twitter’, Journal of Pragmatics 191, 67–82.
  * Martella & Bracciale (2022) Martella, A. & Bracciale, R. (2022), ‘Populism and emotions: Italian political leaders’ communicative strategies to engage Facebook users’, Innovation: The European Journal of Social Science Research 35(1), 65–85.
  * Masters et al. (1986) Masters, R., Sullivan, D., Lanzetta, J., Mchugo, G. & Englis, B. (1986), ‘The facial displays of leaders: Toward an ethology of human politics’, Journal of Social and Biological Systems 9(4), 319–343.
  * Mazzoleni (2014) Mazzoleni, G. (2014), Mediatization and Political Populism, in F. Esser & J. Strömbäck, eds, ‘Mediatization of Politics’, Palgrave Macmillan, London.
  * Mendonça & Caetano (2021) Mendonça, R. F. & Caetano, R. D. (2021), ‘Populism as Parody: The Visual Self-Presentation of Jair Bolsonaro on Instagram’, The International Journal of Press/Politics 26(1), 210–235.
  * Messaris (2019) Messaris, P. (2019), The digital transformation of visual politics, in A. Veneti, D. Jackson & D. G. Lilleker, eds, ‘Visual Political Communication’, Springer International Publishing, pp. 17–36.
  * Moffitt (2022a) Moffitt, B. (2022a), ‘How Do Populists Visually Represent ‘The People’? A Systematic Comparative Visual Content Analysis of Donald Trump and Bernie Sanders’ Instagram Accounts’, The International Journal of Press/Politics p. 19401612221100418.
  * Moffitt (2022b) Moffitt, B. (2022b), ‘Taking Account of the Visual Politics of Populism’, Polity 54(3), 557–564.
  * Moffitt & Tormey (2014) Moffitt, B. & Tormey, S. (2014), ‘Rethinking Populism: Politics, Mediatisation and Political Style’, Political Studies 62(2), 381–397.
  * Mohammad (2016) Mohammad, S. M. (2016), Sentiment Analysis: Detecting Valence, Emotions, and Other Affectual States from Text, in H. L. Meiselman, ed., ‘Emotion Measurement’, Elsevier ; Woodhead Publishing, Amsterdam.
  * Mudde (2004) Mudde, C. (2004), ‘The Populist Zeitgeist’, Government and Opposition 39(4), 541–563.
  * Mudde (2017) Mudde, C. (2017), Populism: An Ideational Approach, in C. Rovira Kaltwasser, P. A. Taggart, P. Ochoa Espejo & P. Ostiguy, eds, ‘The Oxford Handbook of Populism’, Oxford Handbooks, Oxford University Press, Oxford, United Kingdom ; New York, pp. 27–47.
  * Nguyen (2019) Nguyen, C. (2019), ‘Emotions and populist support’.
https://osf.io/preprints/socarxiv/e2wm6/

  * Norris (2020) Norris, P. (2020), ‘Measuring populism worldwide’, Party Politics 26(6), 697–717.
  * Norris & Inglehart (2019) Norris, P. & Inglehart, R. (2019), Cultural Backlash: Trump, Brexit, and Authoritarian Populism, Cambridge University Press.
  * Ostiguy (2017) Ostiguy, P. (2017), Populism: A Socio-Cultural Approach, in C. Rovira Kaltwasser, P. A. Taggart, P. Ochoa Espejo & P. Ostiguy, eds, ‘The Oxford Handbook of Populism’, Oxford Handbooks, Oxford University Press, Oxford, United Kingdom ; New York, pp. 73–95.
  * Rovira Kaltwasser et al. (2017) Rovira Kaltwasser, C., Taggart, P. A., Ochoa Espejo, P. & Ostiguy, P., eds (2017), The Oxford Handbook of Populism, Oxford Handbooks, Oxford University Press, Oxford, United Kingdom ; New York.
  * Ryan et al. (2023) Ryan, O., Dablander, F. & Haslbeck, J. M. B. (2023), Towards a Generative Model for Emotion Dynamics, Preprint, PsyArXiv.
  * Salmela & von Scheve (2018) Salmela, M. & von Scheve, C. (2018), ‘Emotional Dynamics of Right- and Left-wing Political Populism’, Humanity & Society 42(4), 434–454.
  * Schmøkel & Bossetta (2022) Schmøkel, R. & Bossetta, M. (2022), ‘FBAdLibrarian and Pykognition: Open science tools for the collection and emotion detection of images in Facebook political ads with computer vision’, Journal of Information Technology & Politics 19(1), 118–128.
  * Shenk et al. (2021) Shenk, J., CG, A., Arriaga, O. & Owlwasrowk (2021), ‘Justinshenk/fer: Zenodo’, Zenodo.
  * Stewart et al. (2015) Stewart, P. A., Bucy, E. P. & Mehu, M. (2015), ‘Strengthening bonds and connecting with followers: A biobehavioral inventory of political smiles’, Politics and the Life Sciences 34(1), 73–92.
  * Stewart & Ford Dowe (2013) Stewart, P. A. & Ford Dowe, P. K. (2013), ‘Interpreting President Barack Obama’s Facial Displays of Emotion: Revisiting the Dartmouth Group: Interpreting Obama’s Facial Displays of Emotion’, Political Psychology 34(3), 369–385.
  * Stewart et al. (2009) Stewart, P. A., Salter, F. K. & Mehu, M. (2009), ‘Taking leaders at face value: Ethology and the analysis of televised leader displays’, Politics and the Life Sciences 28(1), 48–74.
  * Sullivan & Masters (1988) Sullivan, D. G. & Masters, R. D. (1988), ‘”Happy Warriors”: Leaders’ Facial Displays, Viewers’ Emotions, and Political Support’, American Journal of Political Science 32(2), 345–368.
  * Wells et al. (2020) Wells, C., Shah, D., Lukito, J., Pelled, A., Pevehouse, J. C. & Yang, J. (2020), ‘Trump, twitter, and news media responsiveness: A media systems approach’, 22(4), 659–682.
http://journals.sagepub.com/doi/10.1177/1461444819893987

  * Weyland (2017) Weyland, K. (2017), Populism: A Political-Strategic Approach, in C. Rovira Kaltwasser, P. A. Taggart, P. Ochoa Espejo & P. Ostiguy, eds, ‘The Oxford Handbook of Populism’, Oxford Handbooks, Oxford University Press, Oxford, United Kingdom ; New York, pp. 48–72.
  * Widmann (2021) Widmann, T. (2021), ‘How Emotional Are Populists Really? Factors Explaining Emotional Appeals in the Communication of Political Parties’, Political Psychology 42(1), 163–181.
  * Zhang et al. (2016a) Zhang, K., Zhang, Z., Li, Z. & Qiao, Y. (2016a), ‘Joint face detection and alignment using multitask cascaded convolutional networks’, IEEE Signal Processing Letters 23(10), 1499–1503.
  * Zhang et al. (2016b) Zhang, K., Zhang, Z., Li, Z. & Qiao, Y. (2016b), ‘Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks’, IEEE Signal Processing Letters 23(10), 1499–1503.

##  Appendix A Supplementary: Alternative frame processing approach

In our main analysis we presented the results based on the frame-processing
approach of selecting 300 uniformly distributed frames from each video. An
alternative, which yields roughly the same number of frames per video (273.03
on average), is to select and process every 50th frame of each given video.
With this approach, we arrive at 209 successfully processed videos—six more
than in the main analysis, because we find more consistent face selection in
several videos when the algorithm receives this type of input. For shorter
videos, this reduces the probability that, when selecting exactly 300 frames,
a significant proportion of these frames captures cutaways to other people or
the audience instead of the leader delivering their speech. Pluralist leaders
are present in 101 videos (31 strongly pluralist and 70 moderately pluralist)
and populist leaders are present in the remaining 108 videos (70 strongly
populist and 38 moderately populist). Descriptive statistics for emotion
variables obtained from this frame processing approach are presented in Table
2.

| Mean | SD | Min. | Median | Max.
---|---|---|---|---|---
Anger | 0.223 | 0.130 | 0.031 | 0.195 | 0.728
Disgust | 0.003 | 0.006 | 0.000 | 0.001 | 0.051
Fear | 0.122 | 0.087 | 0.015 | 0.093 | 0.530
Happiness | 0.084 | 0.099 | 0.001 | 0.053 | 0.594
Sadness | 0.204 | 0.109 | 0.021 | 0.183 | 0.573
Surprise | 0.030 | 0.037 | 0.000 | 0.018 | 0.275
Neutral | 0.332 | 0.167 | 0.021 | 0.309 | 0.812
Negative emotions | 0.552 | 0.182 | 0.147 | 0.558 | 0.965
Table 2: Descriptive statistics for extracted emotions for the entire sample,
$N=209$ videos

These results are quite similar to those presented in Table 1, with no major
deviations in terms of central tendency or variability of the 8 variables.
Anger, sadness, and fear are the most common emotions and negative emotions
have an average score higher than 0.5.

Again, we observe statistically significant differences in the means of
negative emotions and neutral expression between populists and pluralists
(negative emotions: $t=3.742$, $p=0.00024$; neutral expression:
$t=-3.222,p=0.00148$). The sample mean differences are smaller than in the
main analysis, but with no changes in the statistical significance (negative:
0.116 in main analysis, 0.091 alternative; neutral: -0.075 in main analysis, -
0.074 in alternative). The distribution of average scores is shown in Figure
5.

Figure 5: Distribution of average scores of negative emotions (left) and
neutral expression (right) of videos featuring populist and pluralist leaders.

Figures 6 and 7 illustrate the differences in the distribution of mean emotion
scores for four subsamples according to the degree of pluralist/populist
rhetoric. The results of one-way ANOVA remain in accordance with the main
analysis, but with weaker effect size in the case of negative emotions. The
differences between the group means are statistically significant both in the
case of negative emotions ($F=4.886$, $p=0.002712$) with moderate effect size
measured by eta-squared (0.073), and neutral expression ($F=5.180$
$p=0.001846$) with moderate effect size (0.077).

Figure 6: Raincloud plot showing the distribution of average scores of
negative emotions across four degrees of populism. Every data point is
presented as a colored dot and above them boxplots and density plots are
displayed. The dashed vertical bar indicates the score of 0.5 Figure 7:
Raincloud plot showing the distribution of average scores of neutral
expression across four degrees of populism. Every data point is presented as a
colored dot and above them boxplots and density plots are displayed. The
dashed vertical bar indicates the score of 0.5.

The results from Tukey HSD post-hoc tests reveal significant differences
between leaders coming from parties employing strong populist rhetoric and
leaders coming from pluralist parties (mean difference 0.1298 $p=0.0057$ in
comparison with strong pluralists, and mean difference 0.0961 $p=0.0166$ in
comparison with moderate pluralists). For neutral expression we again find the
opposite case, as strong pluralists differ from both groups of populists (mean
difference 0.138 $p=0.0037$ for moderate populists, and mean difference 0.1131
$p=0.0122$ for strong populists).

Overall, we conclude that using an alternative frame processing approach does
not change the outcomes of statistical tests, but the reduced effect size in
the case of negative emotions and overall shrinkage of the absolute
differences between groups should certainly be noted.

##  Appendix B Supplementary: Dominant emotions per frame

Another way of conceptualizing differences in emotional expression is to look
at the proportions of frames in which a certain emotion or type of emotional
expression is dominant among the groups (Table 3).

| Anger | Disgust | Fear | Happiness | Sadness | Surprise | Neutral | $\begin{array}[]{l}\text{ Negative }\\\ \text{ emotions }\end{array}$
---|---|---|---|---|---|---|---|---
Main analysis |  |  |  |  |  |  |  |
Pluralists | 0.162 | 0.001 | 0.094 | 0.072 | 0.123 | 0.019 | 0.428 | 0.461
Populists | 0.262 | 0.004 | 0.084 | 0.062 | 0.234 | 0.010 | 0.345 | 0.667
Supplementary |  |  |  |  |  |  |  |
Pluralists | 0.176 | 0.000 | 0.096 | 0.073 | 0.132 | 0.020 | 0.438 | 0.484
Populists | 0.248 | 0.000 | 0.080 | 0.076 | 0.225 | 0.011 | 0.358 | 0.640

Table 3: Mean proportion of all frames in which emotion or group of emotions
has the highest score, and mean proportion of frames in which negative
emotions have a sum score $\geq 0.5$, per category of leader and split by
frame processing approach.

##  Appendix C Supplement C: Country-level comparison

The sample of YouTube videos and leaders who appear in them is not
representative for any country included. We have selected only a few political
parties and deliberately excluded small parties from the analysis.
Nonetheless, here we provide a brief overview of raw and mean scores of
negative emotions and neutral expression for every country in the sample,
split by populist and pluralist leaders. Both Figure 8 and 9 show that the
differences between populists and pluralists are not always consistent between
the countries.

Figure 8: Mean and raw scores of negative emotions for every country (ISO
country codes on x-axis) in the sample and for populist and pluralist leaders.
Transparent points are raw scores, thicker points are means for every group.

For example, in the case of Serbia, Croatia and South Africa, pluralists show
more negative emotions (average score $>0.5$ in all three cases). On the other
hand, in Serbia, Croatia, Australia, the Netherlands and Brazil, we have cases
where populists are on average more neutral in their public appearances.

Figure 9: Mean and raw scores of neutral expression for every country (ISO
country codes on x-axis) in the sample and for populist and pluralist leaders.
Transparent points are raw scores, thicker points are means for every group.