# Cloud Classification with Unsupervised Deep Learning

Takuya Kurihana1, Ian Foster1,2, Rebecca Willett1,4, Sydney Jenkins1,5,
Kathryn Koenig1,6, Ruby Werman7, Ricardo Barros Lourenco1, Casper Neo1,
Elisabeth Moyer3 Corresponding author: I Foster,
<EMAIL_ADDRESS>of Computer Science, University of Chicago2Data
Science and Learning Division, Argonne National Lab4Department of Statistics,
University of Chicago5Department of Physics, University of Chicago6Harris
School of Public Policy, University of Chicago 7College of Letters & Science,
University of California, Berkeley3Department of the Geophysical Sciences,
University of Chicago

###### Abstract

We present a framework for cloud characterization that leverages modern
unsupervised deep learning technologies. While previous neural network-based
cloud classification models have used supervised learning methods,
unsupervised learning allows us to avoid restricting the model to artificial
categories based on historical cloud classification schemes and enables the
discovery of novel, more detailed classifications. Our framework learns cloud
features directly from radiance data produced by NASA’s Moderate Resolution
Imaging Spectroradiometer (MODIS) satellite instrument, deriving cloud
characteristics from millions of images without relying on pre-defined cloud
types during the training process. We present preliminary results showing that
our method extracts physically relevant information from radiance data and
produces meaningful cloud classes.

##  I Motivation

Clouds play a dominant role in the Earth’s radiation budget, both reflecting
sunlight and trapping infrared radiation. Their responses are the principal
source of uncertainty in numerical simulations of future climate, because even
state-of-the-art climate models cannot accurately resolve cloud formation and
evolution on scales from sub-kilometers to thousands of kilometers [1]. NASA
satellite instruments have observed cloud behavior for several decades,
providing us with a rich dataset that can potentially inform understanding of
cloud dynamics and feedbacks, but these large datasets have not yet been fully
employed, in part because computing power has only recently approached the
necessary scale. Clouds are therefore a timely target for large scale
computational analyses that can automate detection of cloud attributes and
identify scientifically relevant cloud classes.

Figure 1: MODIS Terra satellite visible imagery on December 1, 2015 over the
East Pacific ocean, off the coast of California, capturing stratocumulus
clouds at a variety of scales. Clouds at lower right are higher in altitude.
We use this scene in all examples that follow.

Cloud classification effectively reduces the dimensionality of information in
satellite images, rendering them tractable to analysis. Attempts to use neural
network methods for this purpose date to the 1990s, when Lee et al. [2] used
human-labeled images to train shallow, fully connected networks to recognize
stratocumulus, cumulus, and cirrus clouds. Similar efforts have continued
sporadically up to the present (e.g., [3, 4, 5]), all involving supervised
learning based on images labeled with historically defined classes. However,
none have produced an operational tool for automated analysis of cloud images.
Supervised learning generally falls short because historical classes are
artificial and are well-defined only for “classic” examples that make up a
small fraction of total data. Furthermore, those classes do not distinguish
scales of features that in nature may vary by an order of magnitude or more.
For example, in the MODIS example image of Figure 1, stratocumulus clouds show
a wide range of cell sizes, and cloud textures and patterns vary in complex
ways.

Unsupervised learning methods may be a more appropriate means of making use of
the complex information in large multi-spectral satellite datasets. Such
methods allow novel data-driven cloud types to emerge from imagery data, and
in principle can track changes in cloud textures and patterns over time, by
identifying both changing frequencies of individual cloud classes and evolving
characteristics within a class.

A scientifically useful operational classification system would:

  1. 1.

produce physically reasonable classes with scientifically relevant
distinctions

  2. 2.

capture information on cloud spatial distributions, i.e., be not reproducible
using only mean properties over the target area

  3. 3.

produce classes that in high-dimensional space are cohesive within each class
but separated between classes

  4. 4.

be rotationally invariant, i.e., insensitive to the orientation of an image

  5. 5.

be stable, i.e., produce similar or identical classes when different subsets
of the data are used.

We describe here the construction of a prototype data-driven workflow for
cloud classification based on unsupervised deep learning. In the following, we
introduce our model architecture and clustering procedure, apply it to images
from the MODIS satellite instrument, and evaluate its ability to meet some of
the key criteria listed above.

##  II Method

###  II-A Model Architecture

We leverage recent work in self-supervised learning, in which an encoder-
decoder network is trained to recover an input image. We use a deep
convolutional autoencoder [6] to obtain dimensionally reduced information from
input data. Autoencoders have been widely used to retrieve dimensionally
reduced information from high-dimensional input data. The resulting lower-
dimensional latent representations incorporate important input features,
simplifying the classification task. In the general framework of an
autoencoder, the learning process minimizes the loss function $L$:

$\displaystyle\min L(x)=\min||x-F(x)||_{p}\hskip 5.69054pt,$ (1)

where $x$ is the input image; $F(\cdot)$ is a function which maps the input
image on the dimension-reduced representation and then reconstructs the image
from the intermediate information, meaning $F(x)$ is the reconstructed input
image; and $||\cdot||_{p}$ denotes the $p$-norm of the two images.

Our loss function combines four metrics: L1 and L2 loss, corresponding to $p$
= 1 and $p$ = 2 in Equation 1; the high frequency error norm after passing
through the Sobel filter to detect edges of input clouds; and the multi-scale
structure similarity index (MSSIM) [7], a multi-band version of SSIM [8], an
index often used in computer vision to assess image similarity. We use the
Adam optimizer [9], a combination of RMSprop and stochastic gradient descent
with momentum, to optimize our loss function, with a learning rate of 10-4.

We also include a convolution layer in our model in order to preserve spatial
structure of the input image. The convolution operation implements a small-
size filter to subset the entire image iteratively, with specified stride and
width. The filter kernel operation extracts local features and parses the
activation layer. The convolutional layer with activation function is
described as

$\displaystyle h^{l}$ $\displaystyle=$ $\displaystyle
f\left(\sum_{i}\sum_{j}x_{(i+w-1)(j+s-1)}^{l}\otimes
W^{l}_{ij}+b^{l}\right)\hskip 5.69054pt,$ (2)

where $h^{l}$ is the $l$th layer’s latent representation; $f$ is a nonlinear
activation function; $x_{(i+w)(j+s)}$ is a $w\times s$ domain for the
convolutional filter; $W_{ij}$ is the weight at the $i$th column and $j$th
row; $\otimes$ is the convolutional operation; and $b$ denotes the bias. We
set the filter size to 3$\times$3 and use Leaky Rectified Linear Unit (Leaky
ReLU) as the activation function $f(x)=\max(0.3x,x)$, as that performs better
than common ReLU. Additionally, we build a residual connection every two
convolutional layers to improve network performance, and add batch
normalization after each residual connection. Between residual blocks, the
size of an input image is scaled by a factor of two. In the encoder, the width
and the stride are halved at each block, while the depth is doubled. In the
decoder, these transformations are reversed, with the minor modification that
we apply a transposed convolution kernel to map each input pixel to 3$\times$3
pixels for up-sampling. The overall model architecture is illustrated in
Figure 2.

We implement the convolutional autoencoder in the TensorFlow deep learning
library [10] and use the Horovod framework [11] for data parallelization. Our
encoder-decoder architecture stacks 20 convolutional layers and has $297\,232$
trainable parameters, and our latent representation has size 8 $\times$ 8
$\times$ 128\. Training took $100\,000$ steps and 17 hours to converge the
loss function on four NVIDIA K80 GPUs on the University of Chicago’s Midway
compute cluster. We chose a batch size of 32 in accordance with common deep
learning parameter settings and processed 789 GB of training images. We
describe the input data and its pre-processing below.

Figure 2: Our autoencoder architecture. The encoder consists of four blocks,
each with two convolutional layers activated by Leaky ReLU with residual
connections; the decoder has the mirror structure of the encoder. The arrows
represent the flow of input images, and each bracketed triple gives the
height, width, and channel dimensions of the layer(s) above.

###  II-B Dataset

We test our workflow to perform cloud feature extraction on multi-spectral
data from the MODIS instrument. Our input data is MODIS level 1B calibrated
radiance imagery at 1 km resolution (MOD021KM; hereafter MOD02). This product
has 36 spectral bands, from the visible to the thermal infrared; we work with
bands 6, 7, 20, 28, 29, and 31, as these are the most important for the
MODIS06 Level 2 algorithms that characterize cloud properties [12, 13]. Bands
6, 7, and 20 ($1.6\text{\,}\mathrm{,}$ $2.1\text{\,}\mathrm{,}$ and
$3.7\text{\,}\mathrm{\SIUnitSymbolMicro m}$) are encoded in the algorithm to
estimate cloud optical properties (e.g., optical thickness and effective
radius), and the brightness temperatures at bands 28, 29, and 31
($7.3\text{\,}\mathrm{,}$ $8.5\text{\,}\mathrm{,}$ and
$11\text{\,}\mathrm{\SIUnitSymbolMicro m}$) are used in the separation of high
and low clouds and the detection of cloud phase. Note that because we seek to
discover aspects of these physics variables in our classification, we do not
use derived properties such as brightness temperature, but instead input
radiance data directly.

To enable efficient learning of cloud features, we define the unit of our
input data, a “patch,” as a small subset of a typical MODIS image: 128 km
$\times$ 128 km $\times$ 6 selected bands, out of an image of 2030 km $\times$
1354 km $\times$ 36 bands. To select input patches that contain clouds, we
align the MOD02 data to its corresponding MODIS35 Level 2 cloud flag product
(“MOD35”), and define a patch to be valid if more than 30$\%$ of the patch is
comprised of cloud pixels as detected by MOD35. We then train the network
using $\sim$1.01 million patches: about 1% of the full 19-year dataset from a
single MODIS satellite instrument.

###  II-C Clustering

We use hierarchical agglomerative clustering (HAC) to merge data points by
minimizing cluster variance, thus building a tree structure during the merging
process. We choose HAC because it exhibits greater stability with respect to
initialization than does k-means clustering. Our linkage metric is Ward’s
method, formulated as following

$\displaystyle\delta\text{dist}(X_{A},X_{B})=\frac{n_{A}n_{B}}{n_{A}+n_{B}}||C_{A}-C_{B}||^{2}\hskip
5.69054pt,$ (3)

where the distance between two clusters $X_{A}$ and $X_{B}$ is evaluated as
the squared distance between the centroids of merged clusters $C_{A}$ and
$C_{B}$ weighted by the number of patches in these clusters $n_{A}$ and
$n_{B}$.

To choose the number of clusters for an analysis, we would ideally determine
the number for which clustering results are stable (allowing the permutation
of clustering categories). As an approximate measure of stability, we measure
the similarity of clusters using the Adjusted Mutual Information score (AMI).
We first obtain pseudo ground-truth labels by applying clustering to
$\sim$320,000 patches, and then conduct tests with varying subsets of patches
(chosen at random from the full dataset) and varying numbers of clusters, and
compute the AMI score between the ground truth labels and subsets. The AMI
score typically stabilizes at cluster numbers of about 10 and higher. Most
demonstration analyses shown below use 12 clusters; a larger number is likely
desirable for eventual science use.

Fig. 3 shows the complete pipeline from input data to resulting clusters.

Figure 3: Cartoon of the framework used in this work. M02 and M35 are the
MOD02 and MOD35 satellite data products used as inputs. Orange and blue arrows
show the paths taken by the training and test data, respectively; the red
arrow depicts the clustering process.

##  III Evaluation

We report on three initial evaluations of our framework’s capabilities.

As a first, we evaluate the physical reasonableness of assigned cluster labels
in our full workflow. That is, we ask whether clusters are associated with
reasonable distributions of patch-mean values of physical variables. Fig. 4
shows results for the representative MODIS swath also shown in Figure. 1. Left
panel shows the cluster labels assigned to each patch in the image; right
panel shows the raw visible image (band 1) and highlights patches assigned to
two selected clusters (#0 and #2); and bottom panels show the distributions
for these patches of four derived cloud physics parameters. Clustering is
clearly correlated with meaningful physical cloud attributes: cluster #2
(blue) is stratocumulus and cluster #0 is cirrus, likely convective outflow.

Figure 4: Results of clustering results via autoencoder of MOD02 level 1B
radiances in a representative MODIS swath image (December 1st, 2015, 11-44 N,
144-112 E; center of scene is shown in Fig. 1). Orbital coverage gaps leave
missing data on sides of swath. Top left: labeled patches, classified into 12
clusters. Color bar shows cluster number in the 491 patches used; white
indicates no data or invalid data; black indicates patches with $<$30$\%$
cloud pixels. Top right: raw visible image (band 1, which is not used as input
to the autoencoder), with clusters #0 (violet) and #2 (blue) highlighted.
Bottom: histograms of path-mean values of four derived cloud physics
parameters in clusters #0 and #2: optical thickness, phase, cloud top
pressure, and effective radius. Cluster #2 captures stratocumulus and Cluster
#0 two instances of high-altitude cirrus.

Figure 5: Results of clustering based on patch-mean values of five MOD06
parameters (the 4 shown in Fig. 5 plus cloud water path) in the same swath as
in Fig. 4. Left: labeled patches, classified into 12 clusters, with same
figure conventions as in Fig. 5. Right: spatial distribution (heat maps) of
values of four MOD06 cloud physics parameters. Clustering based on patch-mean
parameter value produces less spatially coherent assigned classes than in Fig.
4 and does not capture important physical gradients, e.g. the sharp transition
in effective radius at lower right.

We then conduct a simple test of whether our clustering via autoencoder
captures richer and more meaningful information on cloud distributions and
properties than can be provided by the deterministic algorithms used to
produce derived cloud parameters. To have scientific value, our framework must
produce information beyond that encoded in MOD06 products. We apply
agglomerative clustering directly to MOD06 physics parameters and evaluate how
well patches are classified without the guidance of dimension-reduced MOD02
radiance information. Results suggest that clustering via autoencoder produces
classes that are spatially more cohesive and that better capture important
physical transitions. (Compare left panels of Figures 4 and 5.)

Finally, we examine the spatial distribution of the latent representation
itself using t-Distributed Stochastic Neighbor Embedding (t-SNE). This
nonlinear dimensionality reduction technique maps each point in a high-
dimensional space to a two-dimensional point such that similar objects are
placed near to each other and dissimilar objects far apart, with high
probability [14]. Resulting patch clusters are cohesive and distinct,
suggesting that agglomerative clustering within our latent representation
meaningfully separates different patch types (Figure 6).

Figure 6: A t-SNE visualization of the latent representations of the MOD02
patches of Figure 4, with cluster assignments represented by the same colors
as in that figure. Patches in each cluster are projected near to each other in
the t-SNE map.

##  IV Conclusions

We describe here a prototype application of unsupervised learning to the
problem of automated classification of clouds in multi-spectral satellite
imagery. Our convolutional autoencoder generates a latent representation that,
when clustered, yields physically meaningful cloud classes that pass a number
of requisite tests for a scientifically useful tool. Assigned classes
appropriately produce spatially coherent classifications, and capture
meaningful aspects of cloud physics without being reproducible from mean
values of physics parameters alone. This work supports the possibility of
using unsupervised data-driven frameworks for automated cloud classification
and pattern discovery without requiring the prior hypothesis of ground-truth
labeled data. While results here are preliminary, they suggest that similar
frameworks can be used to analyze multi-year, global data to track short-term
evolution during cloud lifecycles and long-term trends in the distribution of
cloud features and characteristics.

More generally, unsupervised learning methods have broad potential
applicability in the Earth sciences. In the satellite era, a primary challenge
to environmental scientists is not gathering data but finding meaning in
overwhelmingly large datasets. Unsupervised learning has the potential to
reveal patterns directly learned from observation, which can then provide new
insights and help diagnose drivers of system behavior.

## Acknowledgments

This work was supported by the Center for Robust Decision-making on Climate
and Energy Policy (RDCEP), NSF award SES-1463644, and used computers at the
U.Chicago Research Computing Center and the Argonne Leadership Computing
Facility, a DOE Office of Science User Facility, contract DE-AC02-06CH11357.

## References

  * [1] M. Yoshiaki, K. Yoshiyuki, Y. Ryuj, Y. Tsuyoshi, Y. Hisashi, and T. Hirofumi, “Deep moist atmospheric convection in a subkilometer global simulation,” Geophysical Research Letters, vol. 40, no. 18, pp. 4922–4926, 2013.
  * [2] L. R. Jonathan, C. Weger, K. S. Sailes, and M. W. Ronaldo, “A neural network approach to cloud classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 28, no. 5, pp. 846–855, 1990.
  * [3] T. Bin, A. S. Mukhtiar, R. A. S. Mahmood, H. V. H. Thomas, and L. R. Donald, “A study of cloud classification with neural networks using spectral and textural features,” IEEE Transactions on Neural Networks, vol. 10, no. 1, pp. 846–855, 1999.
  * [4] W. Robert and L. H. Dennis, “Spatial variability of liquid water path in marine low cloud: The importance of mesoscale cellular convection,” Journal of Climate, vol. 19, no. 9, pp. 1748–1764, 2005.
  * [5] Z. Jinglin, L. Pu, Z. Feng, and S. Qianqian, “CloudNet: Ground-based cloud classification with deep convolutional neural network,” Geophysical Research Letters, vol. 45, no. 16, pp. 8665–8672, 2018.
  * [6] G. E. Hinton and S. Z. Richard, “Autoencoders, minimum description length and Helmholtz free energy,” in Advances in Neural Information Processing Systems 6, pp. 3–10, Morgan-Kaufmann, 1994.
  * [7] Z. Wang, E. P. Simoncelli, and A. C. Bovik, “Multiscale structural similarity for image quality assessment,” in 37th Asilomar Conference on Signals, Systems & Computers, vol. 2, pp. 1398–1402, Ieee, 2003.
  * [8] Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli, et al., “Image quality assessment: from error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004.
  * [9] D. P. Kingma and J. L. Ba, “Adam: A method for stochastic optimization,” International Conference on Learning Representations, 2015.
  * [10] M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, et al., “Tensorflow: A system for large-scale machine learning,” in 12th USENIX Symposium on Operating Systems Design and Implementation, pp. 265–283, 2016.
  * [11] A. Sergeev and M. Del Balso, “Horovod: Fast and easy distributed deep learning in Tensorflow,” arXiv preprint arXiv:1802.05799, 2018.
  * [12] S. Platnick, K. G. Meyer, M. D. King, B. Marchan, T. G. Arnold, Z. Zhang, P. A. Hubanks, R. E. Holz, P. Yang, W. L. Ridgway, and J. Riedi, “The MODIS cloud optical and microphysical products: Collection 6 updates and examples from Terra and Aqua,” IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 1, pp. 502–525, 2017.
  * [13] B. A. Baum, P. W. Menzel, R. A.Frey, D. C. Tobin, R. E. Holz, S. A. Ackerman, A. K. Heidinger, and P. Yang, “MODIS cloud-top property refinements for collection 6,” Journal of Applied Meteorology and Climatology, vol. 51, no. 6, pp. 1145–1163, 2012.
  * [14] L. van der Maaten and G. Hinton, “Visualizing high-dimensional data using t-SNE,” Journal of Machine Learning Research, no. 9, pp. 2579–2605, 2008\.


# Online Motion Style Transfer for Interactive Character Control

Yingtian Tang<EMAIL_ADDRESS>University of Pennsylvania ,  Jiangtao
Liu<EMAIL_ADDRESS>The Chinese University of Hong Kong ,  Cheng Zhou
<EMAIL_ADDRESS>Tencent Robotics X and  Tingguang Li
<EMAIL_ADDRESS>Tencent Robotics X

###### Abstract.

Motion style transfer is highly desired for motion generation systems for
gaming. Compared to its offline counterpart, the research on online motion
style transfer under interactive control is limited. In this work, we propose
an end-to-end neural network that can generate motions with different styles
and transfer motion styles in real-time under user control. Our approach
eliminates the use of handcrafted phase features, and could be easily trained
and directly deployed in game systems. In the experiment part, we evaluate our
approach from three aspects that are essential for industrial game design:
accuracy, flexibility, and variety, and our model performs a satisfying
result.

motion synthesis, style transfer, interactive control

††ccs: Computing methodologies Motion processing††ccs: Computing methodologies
Neural networks

##  1\. Introduction

Character motion synthesis is a major part of game design. The generation of
character motions requires natural joint placements and fluent movements. In
video games, there are multiple styles of character motions that correspond to
various personalities, ages, and etc. For example, there might be a monster
who walks in a zombie-like manner, or a warrior who runs while holding a
weapon.

Traditional methods for motion synthesis (Kovar et al., 2008; Büttner and
Clavet, 2015) construct a database of motion clips. At runtime, the algorithm
searches in the database for the best match, according to the user control and
the current character motion, and then interpolates to generate the motion for
the next frame. These frameworks consider different styles separately and
record different motion clips for them. As a result, they require large
amounts of motion capture data, and the generated motions are restricted to
the preset styles.

Data-driven methods empowered the recent developments for character motion
synthesis, of which the neural network methods played a critical role. A
series of studies (Holden et al., 2017b; Zhang et al., 2018; Starke et al.,
2019, 2020) initiated the exploration of the application of neural networks in
motion synthesis. These methods avoid searching through a large database, but
still producing character motions with high quality in various scenarios, such
as quadruped motions (Zhang et al., 2018) and character interactions (Starke
et al., 2020). These motion generation models mostly possess a specialized
architecture called Mix-Of-Expert (MOE) to modulate their model parameters
according to the phase (timing of the cyclic movements) and other properties
of the current motion. These model structures are suitable for online motion
generation with user control. However, in this case, a single model cannot
produce motions with different styles, so that multiple models have to be
trained.

There are multiple research studies (Aberman et al., 2020; Smith et al., 2019;
Holden et al., 2017a; Yumer and Mitra, 2016) on the topic of motion style
transfer. In motion style transfer, the original style of a given motion clip
could be transferred to another style, where the content of that motion clip
(for example, walking) is preserved. Nevertheless, these methods only focused
on motion style transfer in an offline fashion, which means they transferred
the style of a static motion clip. In this work, we study the methods for
online motion style transfer with interactive user controls, which could be
directly deployed in game systems. With per-frame user controls, the algorithm
generates new stylized motions on the fly, in an autoregressive fashion. A
similar work on this topic is (Mason et al., 2022).

For game design, the styles provide diversity for the designers. Motions with
different styles could be used for different characters, or splitting and
stitching to create impressive visual effects. Based on this, we design an
evaluation pipeline and focus on three aspects of the model: the accuracy, the
flexibility, and the variety.

First, the model should accurately reproduce the original motions in the
training data, given the same controls. This ensures that following different
paths, the model generates diverse motions learned from the datasets. Second,
the model should flexibly make online transition between styles according to
the control, which further enhances the diversity. For example, this empowers
a character to have actions with different respective style. Third, the model
should create new styles by merging existing styles. This makes the stylized
motions not limited to preset ones. With these considerations, we device both
qualitative and quantitative evaluations, as will be shown in Sec. 5.

We propose the Motion Temporal Convolution Network (MTCN) for multi-style
online motion synthesis with interactive controls. A single MTCN model can be
directly used in games for generating a character with multiple styles and
actions, according to the user control. Moreover, the model performs well on
all three aforementioned style-related functionalities.

Our proposed model has two essential augmentations on the previous online
motion generation models (Holden et al., 2017b; Zhang et al., 2018; Starke et
al., 2019, 2020). The first is to re-modulate the MOE with the style. The MOE
architecture is modulated by a set of parameters called expert weights, which
is further re-modulated in our model by the assigned style. The expert weights
change according to different style assignments. The second is to replace the
phase with learned features. The phase of motions is a key input to generate
expert weights in the previous models. In the recent work (Starke et al.,
2020), the phase is computed through fitting a sinuous curve to the foot/hand
contact labels. We replace the phase input with Temporal Convolution Network
(TCN) motion features, which better accommodate complex and multi-style
motions.

In summary, we have the following contributions:

  * •

We propose an end-to-end model for online and interactive motion style
transfer. With our model, the user could control the game character to switch
between styles or to switch to a merged style fluently and naturally. The
model could be directly deployed in game systems.

  * •

We replace the commonly used handcrafted phase with learned features, which
make the model better suited to multi-style tasks and has simpler data-
preprocessing procedure.

The later sections are structured as follows: Section 2 reviews the
development of MOE-based models and the advance of offline motion style
transfer. Then, Section 3 demonstrates how we modify the MOE so that it is
conditioned on the style assignment, and then proposes our method MTCN which
is based on learned TCN motion features. The data preparation is described in
Section 4. In Section 5, we compare the MTCN with the baseline model, in terms
of the accuracy, the flexibility, and the variety. Finally, Section 6
concludes the paper and indicates our future work.

##  2\. Related Works

In this section, we review the series of neural network methods that utilized
the MOE architecture for online motion synthesis, as well as the style
transfer approaches in image and motion generation. Later in Sec. 3, we will
demonstrate how the style transfer techniques could be combined with the MOE
for online multi-style motion synthesis.

###  2.1. Mix-Of-Expert for Online Motion Synthesis

The series of neural network methods (Holden et al., 2017b; Zhang et al.,
2018; Starke et al., 2019, 2020) use the Mix-Of-Expert (MOE) architecture for
accommodating different phases, controls, and interactions in motion
synthesis. Denote the character motion (i.e., the joints positions and
rotations in the local coordinate) at time $t$ as $m_{t}$, these works
formulate the synthesis task as a sequence generation task. That is, to learn
a mapping $m_{t+1}=\psi(g_{t},m_{t},m_{t-1},...)$, where $g_{t}$ is the user
control given at time $t$. More specifically, they claimed that an
autoregressive formulation $m_{t+1}=\psi(g_{t},m_{t})$ would benefit the
performance.

A simple realization of the mapping $m_{t+1}=\psi(g_{t},m_{t})$ is to directly
use a feed-forward neural network and train it with regression losses.
However, (Holden et al., 2017b) argued that this led to sliding movements.
They had the investigation that the network parameters should be modulated
according to the phase of the motions (denoted as $p_{t}$), which refers to
the cyclic left-right feet movements in their case. Specifically, it is a
periodic variable ranging from 0 to 1, determined by the cyclic movement of
feet. Then they proposed the MOE structure and used it in their Phase-
Functioned Neural Network (PFNN). The MOE is conditioned on a set of ”expert
weights”, denoted by
$\boldsymbol{\alpha}=\\{\alpha_{n}|n=1,2,...,N_{A},\sum_{n}\alpha_{n}=1,\alpha_{n}\geq
0\\}$, where $N_{A}$ is the total number of experts. For a neuron in layer
$i$, the forward-propagation is changed from the original
$y=\phi(w_{i}*x+\beta_{i})$ to:

(1)
$y=\phi\left[~{}\sum_{n=1}^{N_{A}}(w_{n}*\alpha_{n})x+(\beta_{n}*\alpha_{n})~{}\right]$

where $w_{n}$ and $\beta_{n}$ are the network weight and bias for different
experts. $\phi$ is the non-linear activation function. Compared with the
simple linear layer, the structure of MOE parameters could naturally possess
stronger capacity. With the change of conditional parameters $\alpha_{n}$, the
MOE produces continuously changing parameters for networks. The constraints
$\sum_{n}\alpha_{n}=1$ and $\alpha_{n}\geq 0$ are required, out of the
intuition that the combination of experts should be convex.

The network weights of PFNN cyclically change according to the phase value
$p_{t}$, which is generated by a phase function. The subsequent studies
further developed this idea. (Zhang et al., 2018) proposed to use the velocity
of the current motion to replace the the simple phase $p_{t}$, so that the MOE
can adapt to different modes of quadruped motions. (Starke et al., 2019)
developed the Neural State Machine (NSM) model that takes $p_{t}$ along with
action labels and environment interaction variables to produce
$\boldsymbol{\alpha}$. (Starke et al., 2020) proposed to use multiple local
phases instead of a single global phase to capture complex interactions
involving multiple contacts.

We choose NSM as our baseline model, since it has strong capacity and does not
consider multiple contacts, which is our case. In this work, we mainly
consider locomotion data, i.e. standing, walking, and running, so that we
discard the environment interaction variables in the original NSM.

The inputs of the NSM model at time $t$ consist of four components, namely the
joint positions and rotations $m_{t}$, root trajectory $r_{t}$, gait
trajectory $\lambda_{t}$ and the movement phase $p_{t}$, such
$\boldsymbol{X_{t}}=\\{m_{t},r_{t},\lambda_{t},p_{t}\\}$.

Specifically, to produce $\boldsymbol{\alpha}$ for MOE, NSM used a multi-layer
network (called gating network) that takes as input the Kronecker Product of
the phase $p_{t}$ and the gait trajectory $\gamma_{t}$
($p_{t}\otimes\gamma_{t}$). We denote the root trajectory at $t$ as $r_{t}$.
The two trajectories, $\gamma_{t}$ and $r_{t}$, consist of the sampled gaits
(for example, stand, walk, or run) and root positions (the positions of past
and future local coordinates, related to the current local coordinate) of the
character from the past 1s to the future 1s. The sampling rate is $1/6$
seconds. We can modify $r_{t}$ to control to which direction the character
moves.

Controlled by expert weights $\boldsymbol{\alpha}$, the MOE then takes the
${{m}_{t}},{r}_{t},\gamma_{t}$ to regress the
${{m}_{t+1}},{r}_{t+1},\gamma_{t+1}$. Denote the gating network as
$\mathcal{G}$, and the MOE as $\mathcal{M}$, the overall NSM model is shown in
Eq. 2.

(2)
$\begin{split}\boldsymbol{\alpha}&=\mathcal{G}(~{}p_{t}\otimes\gamma_{t}~{})\\\
{m}_{t+1},{r}_{t+1},\gamma_{t+1}&=\mathcal{M}({m}_{t},{r}_{t},\gamma_{t};\boldsymbol{\alpha})\end{split}$

Here, for discussing style-related operations, we shall expand the description
of a motion ${m}$ to ${m}^{s}$, with a style $s\in\mathcal{S}$. We shall also
denote a sequence of motion as ${m_{t_{0},t_{1}}^{s}}$ which contains the
motions from $t_{0}$ to $t_{1}$.

These models based on the MOE realize ${m^{s}_{t+1}}=\psi(g_{t},{m^{s}_{t}})$,
and generate motions with high quality for different scenarios and the same
style. In this work, we extend this mechanism for multi-style online motion
generation, which is
${m^{s^{\prime}}_{t+1}}=\psi(g_{t},{m^{s}_{t}},s^{\prime})$, with an online
target style $s^{\prime}\in\mathcal{S}$.

###  2.2. Motion Style Transfer

Unlike the online motion style transfer tasks, the offline motion style
transfer is provided with the whole extracted content sequence
$c_{t_{0},t_{1}}$, and generates ${m^{s^{\prime}}_{t_{0},t_{1}}}$ with the
target style $s^{\prime}$ given.

This kind of offline style transfer has been well-studied in the field of
computer vision. (Gatys et al., 2016; Johnson et al., 2016) started the
research of image style transfer using neural networks, by noticing that the
style of an image could be described by the inter-layer feature statistics of
a pre-trained Convolutional Neural Network (CNN). (Ulyanov et al., 2016)
further showed that the second order statistics of the aforementioned inter-
layer feature distributions are enough for style descriptions. They proposed
an Instance Normalization (IN) which normalizes the means and variances of
inter-layer feature distributions, so that the style of the image is discarded
and the content is preserved. With IN, their image style transfer outperformed
the previous methods. (Huang and Belongie, 2017) realized that a de-
normalization operation right after IN with a pair of learned mean and
variance (or standard deviation) could finish style transfer process. Their
method named adaptive Instance Normalization (AdaIN), learned different pairs
of mean and variance for different styles, which achieved arbitrary image
style transfer.

To briefly summarize, the change or elimination of styles of images is
directly related to the manipulation of inter-layer feature distributions in
the neural networks.

Recently, (Aberman et al., 2020) applied the concept of content and style of
images on motion clips. The original 2-D convolution kernels in a normal CNN
is replaced by 1-D convolution kernels, which change the CNN into a TCN. They
extracted the content sequence from a motion clip using an TCN encoder
$\boldsymbol{E_{c}}$ with Instance Normalization (IN, (Ulyanov et al., 2016)),
and also extracted the style with a TCN encoder $\boldsymbol{E_{s}}$. Then,
they applied AdaIN on a TCN decoder $\boldsymbol{G}$ to generate unseen
content-style combination, with Generative Adversarial Loss (Goodfellow et
al., 2014). Overall, their method is formulated as the following:

(3)
$\begin{split}c_{t_{0},t_{1}}=&~{}\boldsymbol{E_{c}}(m^{s}_{t_{0},t_{1}})\\\
s^{\prime}=&~{}\boldsymbol{E_{s}}(m^{s^{\prime}}_{t_{0},t_{1}})\\\
{m^{s^{\prime}}_{t_{0},t_{1}}}=&~{}\boldsymbol{G}(c_{t_{0},t_{1}},s^{\prime})\end{split}$

However, their work only focused on this kind of offline whole-sequence motion
style transfer, while for real game systems the online counterpart would be
much more desired.

##  3\. Method

Figure 1. The Style-NSM is a modified version of NSM. It takes the generated
expert weights $\boldsymbol{\alpha}$ and modulate them according to the style
$s$(as shown in the dashed box part in the above image). Then, the modulated
expert weights are used to control the MOE.

In this section, we propose our method Motion Temporal Convolution Network
(MTCN) for multi-style online motion synthesis. Before introducing MTCN, we
first demonstrate how the MOE mechanism (introduced in Sec. 2.1) could be
combined with the ideas from motion style transfer (introduced in Sec. 2.2).
This direct combination leads to Style-NSM (SNSM), which will be our baseline
for developing MTCN. MTCN further uses the learned TCN motion features to
replace the phase input $p_{t}$ in NSM, which results in better performance on
multi-style online motion generation.

###  3.1. Style-NSM

As mentioned before, the MOE is an important mechanism for generating motions
in an online manner. The expert weights change according to the phase and gait
trajectory, adjusting the whole network weights, as is reflected in Eq. 1.
This equation could be also re-written as the following:

(4) $y=\phi\left[~{}\sum_{n=1}^{N_{A}}(w_{n}x+\beta_{n})*\alpha_{n}~{}\right]$

In this form, the MOE could be regarded as modulating the outputs of several
neurons and then summing them up. Here the modulation operation is exactly the
same as it is in the de-normalization operation in AdaIN. More specifically,
it changes the variance of the inter-layer feature distribution. With this
observation, we can see the similarity between MOE and AdaIN.

We can further take the advantage of this property, and re-modulate the expert
weights by style label so that they also adapts to different styles. We change
the forward-propagation for each layer from Eq.1 to the following:

(5)
$y=\phi\left[~{}\sum_{n=1}^{N_{A}}(w_{n}x+\beta_{n})*r(\alpha_{n},s)~{}\right]$

where $s$ is the style specification and $r$ is the modulation function. There
are multiple choices of $r$, here we use a simple element-wise affine
transformation, which is the following:

(6) $r(\alpha_{n},s)=\sigma(s)*\alpha_{n}+\mu(s)$

where $\sigma(s)$ and $\mu(s)$ are neural nets which take the one-hot encoding
of style $s$. The reason of this choice is that this architecture is
empirically more stable than the neural net counterpart, and it introduces
minimal number of extra parameters. For the modulation to work better, we
shall remove the constraint imposed on expert weights, that is,
$\sum_{n}\alpha_{n}=1$ and $\alpha_{n}\geq 0$. Then, the $\alpha_{n}$ should
take arbitrary real values and so do the modulated results.

Overall, the SNSM model is different from NSM with an additional modulation to
generate the expert weights (the comparison is shown in Fig. 1):

(7)
$\begin{split}\boldsymbol{\alpha}&=r(\mathcal{G}(~{}p_{t}\otimes\gamma_{t}~{}),s^{\prime})\end{split}$

Figure 2. In SNSM, the expert weight blending coefficients (the four “alpha”s
values) of four different styles are shown here. Each plot visualizes four
weights’ values of one specific style, where the horizontal axis indicates the
frame range.

The SNSM thereby implements the mapping
$\boldsymbol{m^{s^{\prime}}_{t+1}}=\psi(g_{t},\boldsymbol{m^{s}_{t}},s^{\prime})$.
There is a concern that how the original style $s$ is removed so that the new
style $s^{\prime}$ could take place. Empirically, we find that the outputs of
autoregressive models have small dependency on the original style of
$\boldsymbol{m^{s}_{t}}$. When there is an online style transition, we
interpolate between the original and the target style to gradually make the
transition. This process is also shown in Fig. 2.

###  3.2. Motion Temporal Convolution Network

Figure 3. The MTCN takes the past sequence of motions to generate expert
weights, but still uses the MOE in an autoregressive fashion. MOE only takes
the last frame of motion as the input.

The NSM inputs the phase $p_{t}$ and gait $\gamma_{t}$ to the gating network
to generate the expert weights. The idea of using $p_{t}$ was first described
in PFNN (Holden et al., 2017b). Indeed, we empirically find that $p_{t}$ is a
critical input variables for enabling the MOE for online motion synthesis.
However, a simple handcrafted global phase has limited expressive capability.
For some stylized motions, for example, the zombie-like walking, the phase of
the motions is complex and irregular. Besides, the handcrafted process adds to
the complexity of the whole system. As a result, we propose to use a Temporal
Convolutional Network (TCN) for direct end-to-end feature extraction. The
extracted motion features can replace the original handcrafted phase inputs.

Since we target at online motion synthesis, we have to make the TCN causal,
which means it does not take the future information as inputs. The common form
of TCN is to directly apply symmetric kernels along the time dimension, i.e.,
$v^{l+1}_{t}=\sum_{k=-\tau}^{\tau}v^{l}_{t-k}w_{k}$. ($v_{t}^{l}$ is the value
of channel $l$ at time $t$, $w$ is the kernel weight, and $\tau$ is the scope
length) In our case, it is more natural to limit the scope of the kernel so
that it only takes the past inputs. So, we instead use the Causal Convolution
proposed in (Oord et al., 2016), which takes the form of
$v^{l+1}_{t}=\sum_{k=0}^{2\tau}v^{l}_{t-k}w_{k}$.

Replace the inputs to the gating network of SNSM with the proposed motion
features extracted by a TCN encoder $\mathcal{T}$, we have the expert weights
of our new model MTCN to be:

(8) $\begin{split}\theta_{t}&=\mathcal{T}({{m}^{s}_{t-\tau,t}})\\\
\boldsymbol{\alpha}&=r(\mathcal{G}(\theta_{t}),s^{\prime})\end{split}$

where $\theta_{t}$ is the motion features extracted from the motion. ($\tau$
is a preset window size) To ensure the modulation capacity, the gating network
generates $\alpha_{n}^{l}$ for each layer $l$ separately, i.e.,
$\boldsymbol{\alpha}=\\{\alpha_{n}^{l},n=1,2,...,N_{A},l=1,2,...,L\\}$,
instead of generating $\alpha_{n}$ that are shared by all layers, where $L$ is
the total number of layers of MOE. As shown in Fig. 3, at each frame, the MTCN
takes the past sequence to generate TCN motion features, which determine the
expert weights. Then, the expert weights are re-modulated according to the
assigned style, which finally controls the MOE.

Scheduled Sampling. We find that Scheduled Sampling (Bengio et al., 2015) is a
crucial tool for training MTCN. Scheduled Sampling aims to expose the
regression error at runtime to the model at training time. More specifically,
for each training iteration, a sampling probability $p$ is chosen to decide
whether the inputs of the next frame are the real data or they are the outputs
from the network for the previous frame. With probability $1-p$, the inputs
become the outputs from the network, which can be regarded as samples from
runtime distribution. This sampling process takes place for multiple frames,
and the errors are accumulated and backpropagated at once. In practice, we
gradually increase $p$ from 0 to 1 within the first 10 training iterations.

Window Instance Normalization. With the introduction of the temporal motion
features, the network has much more temporal dependency on past styles
compared to the fully autoregressive SNSM. This is also confirmed in our
experiments as we will show in Sec. 5. Consequently, we also find it necessary
to add the Instance Normalization (IN) to the encoder $\mathcal{T}$, so it
becomes $\mathcal{T}_{IN}$. We refer to the MTCN after this modification as
MTCN-IN. However, there is a subtle problem regarding the behavior of IN at
runtime. As mentioned, IN works by taking a set of instances, in our case, a
sequence of motions. At runtime, the motions are generated continuously. At
time stamp $T$, it is undesirable to feed all the motions from the beginning
to $T$ to the IN, because this whole sequence might already contain multiple
styles and different motion types, which clearly differs from the short and
single-styled motion sequences when training. Here we proposed a Window
Instance Normalization (WIN) to handle this problem. Basically, the WIN takes
sequences of motions within a pre-defined window size, $\tau$. At runtime,
instead of normalizing over motions from beginning to $T$, WIN normalizes
motions from $T-\tau$ to $T$. For a given sequence of motion
${m}_{{t_{0},t_{1}}}^{s}$, for a specific intermediate layer, the neural net
produces feature sequence ${f}_{{t_{0},t_{1}}}^{s}$. The WIN takes place by:

(9)
$\begin{split}\mu&=\frac{\sum_{t=t_{0}}^{t_{1}}{f}_{t}^{s}}{t_{1}-t_{0}+1}\\\
\sigma&=\sqrt{\frac{\sum_{t=t_{0}}^{t_{1}}({f}_{t}^{s}-\mu)^{2}}{t_{1}-t_{0}}}\\\
\widetilde{f}_{t}^{s}&=\frac{{f}_{t}^{s}-\mu}{\sigma},~{}for~{}all~{}t\\\
\end{split}$

where the $s$ information is reduced in $\widetilde{f}_{t_{0},t_{1}}^{s}$.
After several WIN operations, the style $s$ of ${{m}^{s}_{t_{0},t_{1}}}$ is
largely reduced and finally produces the content sequence $c_{t_{0},t_{1}}$.

Modification of IN & WIN. There is a critical issue that applies to both IN
and WIN. Consider the standing action, the character stands still on the
ground with some very small yet natural movements. For Instance Normalization,
these natural tiny movements are also normalized, thus generating undesirable
content signals. Ideally, these movements should be represented with their
original small values, instead of the normalized values. In fact, the standing
pose is not only the case where this phenomenon happens. The consequence of
these is the generated motions are shaking at runtime, instead of standing
still or keeping the pose. As a result, we change the standard normalization
procedure (last line in Eq.9) by imposing a minimum for standard deviation:

(10)
$\begin{split}\widetilde{f}_{t}^{s}&=\frac{{f}_{t}^{s}-\mu}{max(\sigma,\epsilon)},~{}for~{}all~{}t\\\
\end{split}$

where $\epsilon$ is the minimal value of standard deviation $\sigma$.
Movements that are essentially small remain their small-value property after
the WIN, such as standing, while other movements are normalized as usual. In
practice, the choice of $\epsilon$ is set to be $0.3$, but generally works
well within an empirical range of $0.1\sim 0.3$.

##  4\. DATA PREPARATION

The data preparation and character skeleton are described in this section,
including the motion capture, the data labelling and bone re-targeting.

###  4.1. Motion Capture and Labelling

We collect data with different styles. The whole data consist of 8 styles in
total, including three different zombie styles (NPC-1, NPC-2, and NPC-3), a
gun-holding style (NPC-Gun), and four distinct female styles (FEM-1, FEM-2,
FEM-3, FEM-4). The three zombie styles include a common zombie, a collapsing
zombie, and a jumpy zombie. The four female styles cover styles of being
injured, feeling cold, normal, and sexy. For each style, there are two
separate BVH files that respectively record clockwise and counterclockwise
circular walks in that style. In total, there are 16 BVH files and
equivalently 40383 frames for the training. For all data, the bones are
calibrated to be the same, and the exact same preprocessing pipeline is used,
which include extracting root and gait trajectories as well as the phase
computation.

Each frame of data is annotated with a global phase value calculated by
manually labelled footstep of four joints, ”right angle”, ”right toe”, ”left
angle” and ”left toe”. The periodically varied phase value describes the
motion condition at current frame. Apart from the phase labels, we also label
a action type for each frame,”stand” and ”walk”, which is the motion type of
the character.

###  4.2. Bone re-targeting

The character we used is similar to the Humanoid used by (Peng et al., 2018),
it has 18 joints including the root joint set as base coordinate. To fit to a
general representing and animation platform, we need to re-target the captured
motion sequences, which have 48 joints. The scheme is composed of the
following three steps:

  * •

The joints size is firstly scaled to target joints of character, according to
labelled skeleton. In details, we fit a template bones into positions and
heights of the original motion sequences frame by frame.

  * •

The scaled ”body” joints were re-targeted using dampened least squares inverse
jacobian method. We set the damping parameter to be 2.0 to get fairly stable
and effective end effectors after 20 iterations.

  * •

The last step is to adjust the toe height to make the character keep contact
with the ground with manually calculated value.

After the labeling and re-targeting process, for the final prepared training
set we find it has stable and smooth movement. The above mentioned process
method can adapt to a wide range of characters without concerning the
difference of motion capture systems.

##  5\. Evaluations

As mentioned in Sec. 1, we demand the model to have three properties, i.e.,
the accuracy, the flexibility, and the variety, for providing the diversity
for game design. We therefore propose three corresponding evaluations for the
online motion style transfer task:

  1. (1)

Stylized Motion Replay: given the original control sequences, the model should
accurately regenerate the training data for all styles.

  2. (2)

Online Style Transfer: the model should transfer to the assigned style by the
user at runtime.

  3. (3)

Style Interpolation: the model should merge different learned styles at
runtime.

In this section, we show the results of all of the three evaluations. We use
SNSM as our baseline, as it is the most straightforward modification of the
NSM that enables multi-style motion generation. We then demonstrate the
superiority of our proposed model MTCN. All models are set to have 4 expert
weights and layers with 256 hidden units. The models are trained with batch-
size 64 and Adam optimizer (Kingma and Ba, 2014) (with weight decay $1\times
10^{-5}$ and initial learning rate $1\times 10^{-3}$) for 100 iterations. When
training, the value of dropout (Srivastava et al., 2014) is set to be 40%.

It should be noticed that the first evaluation is the most important, since
the accuracy of motion reproduction is strongly associated with the online
motion generation performance and is the foundation for other style-related
operations.

###  5.1. Stylized Motion Replay

Currently there are no widely used quantitative measures of the accuracy of a
learned motion generation model. The stylized motion replay measure we used
here is empirically a reasonably good indicator of the stylized motion quality
in our experiments.

While in the training we use Scheduled Sampling with a fixed window of
prediction, we require the model to reproduce the exact same motion sequences
at evaluation time, given the original control sequences.

More specifically, at evaluation time, we iteratively input the pre-computed
control signals $\gamma_{t}$ and $r_{t}$ to the model ($t$ starts from the
beginning of the Biovision Hierarchy (BVH) file), and thus run the
autoregression until the end. Then, we compute the Squared Error (SE) at each
frame between the prediction ${{m}_{t}}^{s}$ and the ground truth (denoted as
$\boldsymbol{\hat{m}^{s}_{t}}$). We then have a sequence of SEs, which is
$e(t)=\|{{m}_{t}}^{s}-\boldsymbol{\hat{m}^{s}_{t}}\|^{2}$. To ensure that
various motions of all the styles are accurately learned, we evaluate the
models on all its training data, which include different styles and contents.
We also average the errors by $(\sum_{t=1}^{T}e(t))/T$ ($T$ is the total
number of frames) for a BVH to get a Mean Squared Error (MSE), which is an
overall error measure for the performance on that BVH.

Figure 4. The error timeseries for a specific BVH for different sequence
models. The horizontal axis is the time stamps (frames), and the vertical axis
is the error.

Style | SNSM | MTCN | MTCN-IN
---|---|---|---
NPC-1 | 0.0772 | 0.0268 | 0.0303
NPC-2 | 0.2080 | 0.1068 | 0.1022
NPC-3 | 0.4443 | 0.1120 | 0.2309
NPC-Gun | 0.0300 | 0.0138 | 0.0137
FEM-1 | 0.1876 | 0.0796 | 0.0977
FEM-2 | 0.1989 | 0.1332 | 0.1366
FEM-3 | 0.2529 | 0.1202 | 0.1957
FEM-4 | 0.0887 | 0.0640 | 0.0656

Table 1. MSE for Sequence Models on walking motions. Figure 5. The series of
motions represent the online style transferring process of the model MTCN.
From left to right, the upper image shows style changing from FEM-3 to FEM-4,
then to NPC-GUN and stand at last. The lower figure shows motions styles
varying from Fem-1 to Fem-2, then to NPC-2 and stand still.

As is suggested in the Fig. 4 and Table 1, MTCN significantly outperforms the
baseline model SNSM. When looking into the replayed motions, we found two
reasons for this improvement of performance. First, the SNSM sometimes
produces motions that have slight lags in terms of the phases. This would
further lead to lags in the future frames, thus causing a misalignment of
phases between the generated motions and the ground truth. In Fig. 4, the
fluctuation of errors is mainly caused by this lagging effect. Second,
compared to MTCN, SNSM generates motions with lower accuracy. This can also be
associated with the phase inputs to the gating network. The reason is that in
a complex and diverse motion clip, even if the phase $p_{t}$ of the motions is
the same, the motions could have natural variations. The simple phase inputs
are unable to capture these. On the other hand, the learned motion features
provide MTCN with better descriptions of the motion at the current frame (as
an improved substitution of the phase $p_{t}$), as well as stronger capacity
of capturing the intrinsic variations in the data.

Consequently, at the frame where the misalignment is likely to happen, MTCN
possesses better performance. Moreover, the overall error of MTCN is much
lower than that of SNSM. This suggests that MTCN has decent accuracy of
reproducing learned motions, which would benefit the diversity of the online
motion generation. On the other hand, SNSM performs poorly in the multi-style
setting, which indicates the quality of the generated motions is not adequate
for gaming and any other style-related operations.

As is discussed in Sec. 3.2, MTCN-IN uses WIN for better elimination of style
information in the past motion sequence. This certainly causes performance
degradation. But as suggested in Table 1, comparatively this degradation is
small in most of the styles.

###  5.2. Online Style Transfer

Ability | SNSM | MTCN | MTCN-IN
---|---|---|---
Replay BVH | Poor | Good | Good
Online Style Transition | Yes | Yes | Yes
Style Interpolation | Yes | No | Yes
Table 2. Different Abilities of Models.

In real game design, it is often required to combine different styles or
transiting between them. In Sec. 5.1, we only evaluate the accuracy of
reproducing the training data, but leave the transitions between styles
untested. This requires the model to produce a series of intermediate motions
between two styles, and continuously change between them. The ability to
interpolate between styles is further evaluated in Sec. 5.3. In this section,
we focus on the continuous transition itself.

Here we device a qualitative evaluation for the models. The evaluation tests
whether the model can flexibly transit between two arbitrary styles that it
was trained on. Suppose the user assigns the transition from style $s_{1}$ to
$s_{2}$. Once the control is issued, we linearly interpolate to change from
$s_{1}$ to $s_{2}$. More specifically, we have a factor $\lambda$, which mixes
the embeddings $s_{1}$ and $s_{2}$ to be $s_{M}=(1-\lambda)s_{1}+\lambda
s_{2}$. The $\lambda$ gradually increase from $0$ to $1$ as time goes on. The
generated $s_{M}$ is set to be the style embedding for the model at that
frame. We set the whole process to happen within $1$ second. Then, we observe
whether the transition happens successfully within this $1$ second.

All the models can successfully perform the arbitrary (shown in Tab. 2). In
Fig. 5, we have two images showing the online style transfer process using
MTCN model. From left to right, the the motion sequence frames varying from
FEM-3 to FEM-4, then to NPC-GUN and stand still for upper image. Lower figure
shows style transfer from FEM-1 to FEM-2 then to NPC-2 and stand at last.

###  5.3. Style Interpolation

Finally, we further evaluate the ability of the models to interpolate between
different styles, to create a new style. This is similar to the evaluation in
(Aberman et al., 2020), but they did it in an offline fashion.

This evaluation is also qualitative. We choose two styles $s_{1}$ and $s_{2}$
from the training data, and linearly interpolated between their one-hot
embeddings (each 50%), which produce $s_{I}={(s_{1}+s_{2})}/{2}$. Then, we
directly use the interpolated $s_{I}$ at each frame for the autoregressive
motion generation. We then specify the user controls and observe the motions
from the new style $s_{I}$.

For this task, only the SNSM and MTCN-IN succeed (shown in Tab. 2). This is
expected for the reasons mentioned in Sec. 3.2. Unlike MTCN-IN, which depends
on the content sequence, MTCN has strong dependency on the raw motions. This
makes it hard for MTCN to generate the motions based on a new sequence of
motions, which is from the the interpolated style.

Overall, MTCN-IN succeeds as the model with both accurate motion replay and
the ability to flexibly transit between and interpolate styles.

##  6\. Conclusion

In this work, we target the online motion style transfer task with interactive
control. We proposed a novel neural network model, namely MTCN. It could
handle large number of motion types and extract motion features using TCN
feature extractors, which replace the commonly used phase features.

We also propose a baseline model, namely SNSM, which extends the NSM model to
accommodate different styles. To make MTCN better transit between and
interpolate styles, we augment it to be MTCN-IN. Both MTCN and MTCN-IN have
significant improvement over SNSM in terms of motion replay error. MTCN-IN
turns out to have the most satisfying performance in our online motion style
transfer evaluation pipeline. It is more flexible than MTCN, with small loss
of accuracy. MTCN-IN could be directly used for multi-style motion generation
in real time applications like games.

## References

  * (1)
  * Aberman et al. (2020) Kfir Aberman, Yijia Weng, Dani Lischinski, Daniel Cohen-Or, and Baoquan Chen. 2020.  Unpaired motion style transfer from video to animation.  _ACM Transactions on Graphics (TOG)_ 39, 4 (2020), 64–1.
  * Bengio et al. (2015) Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015.  Scheduled sampling for sequence prediction with recurrent neural networks.  _arXiv preprint arXiv:1506.03099_ (2015).
  * Büttner and Clavet (2015) Michael Büttner and Simon Clavet. 2015.  Motion Matching-The Road to Next Gen Animation.  _Proc. of Nucl. ai_ (2015).
  * Gatys et al. (2016) Leon A Gatys, Alexander S Ecker, and Matthias Bethge. 2016\.  Image style transfer using convolutional neural networks. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 2414–2423.
  * Goodfellow et al. (2014) Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014\.  Generative adversarial networks.  _arXiv preprint arXiv:1406.2661_ (2014).
  * Holden et al. (2017a) Daniel Holden, Ikhsanul Habibie, Ikuo Kusajima, and Taku Komura. 2017a.  Fast neural style transfer for motion data.  _IEEE computer graphics and applications_ 37, 4 (2017), 42–49.
  * Holden et al. (2017b) Daniel Holden, Taku Komura, and Jun Saito. 2017b.  Phase-functioned neural networks for character control.  _ACM Transactions on Graphics (TOG)_ 36, 4 (2017), 1–13.
  * Huang and Belongie (2017) Xun Huang and Serge Belongie. 2017.  Arbitrary style transfer in real-time with adaptive instance normalization. In _Proceedings of the IEEE International Conference on Computer Vision_. 1501–1510.
  * Johnson et al. (2016) Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016.  Perceptual losses for real-time style transfer and super-resolution. In _European conference on computer vision_. Springer, 694–711.
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ (2014).
  * Kovar et al. (2008) Lucas Kovar, Michael Gleicher, and Frédéric Pighin. 2008\.  Motion graphs.  In _ACM SIGGRAPH 2008 classes_. 1–10.
  * Mason et al. (2022) Ian Mason, Sebastian Starke, and Taku Komura. 2022\.  Real-Time Style Modelling of Human Locomotion via Feature-Wise Transformations and Local Motion Phases.  _arXiv preprint arXiv:2201.04439_ (2022).
  * Oord et al. (2016) Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. 2016.  Wavenet: A generative model for raw audio.  _arXiv preprint arXiv:1609.03499_ (2016).
  * Peng et al. (2018) Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne. 2018.  DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills.  _ACM Trans. Graph._ 37, 4, Article 143 (jul 2018), 14 pages.  https://doi.org/10.1145/3197517.3201311
  * Smith et al. (2019) Harrison Jesse Smith, Chen Cao, Michael Neff, and Yingying Wang. 2019\.  Efficient neural networks for real-time motion style transfer.  _Proceedings of the ACM on Computer Graphics and Interactive Techniques_ 2, 2 (2019), 1–17.
  * Srivastava et al. (2014) Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014.  Dropout: A Simple Way to Prevent Neural Networks from Overfitting.  _Journal of Machine Learning Research_ 15, 56 (2014), 1929–1958.  http://jmlr.org/papers/v15/srivastava14a.html
  * Starke et al. (2019) Sebastian Starke, He Zhang, Taku Komura, and Jun Saito. 2019\.  Neural state machine for character-scene interactions.  _ACM Trans. Graph._ 38, 6 (2019), 209–1.
  * Starke et al. (2020) Sebastian Starke, Yiwei Zhao, Taku Komura, and Kazi Zaman. 2020\.  Local motion phases for learning multi-contact character movements.  _ACM Transactions on Graphics (TOG)_ 39, 4 (2020), 54–1.
  * Ulyanov et al. (2016) Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. 2016\.  Instance normalization: The missing ingredient for fast stylization.  _arXiv preprint arXiv:1607.08022_ (2016).
  * Yumer and Mitra (2016) M Ersin Yumer and Niloy J Mitra. 2016.  Spectral style transfer for human motion between independent actions.  _ACM Transactions on Graphics (TOG)_ 35, 4 (2016), 1–8.
  * Zhang et al. (2018) He Zhang, Sebastian Starke, Taku Komura, and Jun Saito. 2018\.  Mode-adaptive neural networks for quadruped motion control.  _ACM Transactions on Graphics (TOG)_ 37, 4 (2018), 1–11.


# Inno at SemEval-2020 Task 11: Leveraging Pure Transformer for Multi-Class
Propaganda Detection

Dmitry Grigorev
Innopolis University
<EMAIL_ADDRESS>
Vladimir Ivanov
Innopolis University
<EMAIL_ADDRESS>

###### Abstract

The paper presents the solution of team ”Inno” to a SEMEVAL 2020 task 11
”Detection of propaganda techniques in news articles”. The goal of the second
subtask is to classify textual segments that correspond to one of the 18 given
propaganda techniques in news articles dataset. We tested a pure Transformer-
based model with an optimized learning scheme on the ability to distinguish
propaganda techniques between each other. Our model showed $0.6$ and $0.58$
overall F1 score on validation set and test set accordingly and non-zero F1
score on each class on both sets.

##  1 Introduction

00footnotetext: This work is licensed under a Creative Commons Attribution 4.0
International Licence. Licence details:
http://creativecommons.org/licenses/by/4.0/.

Modern society is experiencing an overflow of different kinds of information.
Thousands of news media are publishing a giant number of articles, blogposts,
twits and other kind of content every day and even every hour. People who want
to stay informed about global situation are often not able to distinguish
which source is reliable and provides relevant content. Therefore, such topics
as automatic fake news detection, sentiment analysis and propaganda detection
are gaining increasing attention.

The authors of [Jowett and O’donnell, 2018] defined propaganda in the most
neutral sense as dissemination or promotion of a particular idea.

This paper provides our solution to SEMEVAL 2020 task 11 “Detection of
propaganda techniques in news articles”. The task consists of two subtasks:
(i) a propaganda sample span identification and bounding and (ii) a
classification of these spans into one of the 18 classes (or techniques). Some
techniques in the second subtask were merged into one class to get 14 classes
for prediction in the end. All the techniques, as well as the other details
about the task, are described by the task organizers in [Da San Martino et
al., 2020].

In this paper we present a solution to the second subtask. The suggested
system111Source code can be found here https://github.com/oxxford/SEMEVAL-2020
consists of a Transformer-based classifier with additional learning
optimization techniques such as undersampling, cost-sensitive learning and
context addition, aimed at robustness. This approach allowed us to achieve
$0.6$ and $0.58$ overall F1 scores with non-zero F1 score for each class on
validation set and test set accordingly. We took the 7th place out of 31 teams
on the second subtask.

##  2 Background

One of the first classifications of propaganda techniques was proposed in 1936
by Clyde R. Miller [Edwards, 1938] after a presidential election in the USA.
This categorization consists of description of 7 propaganda techniques: “Name
calling”, “Glittering generalities”, “Transfer”, “Testimonial”, “Plain folks”,
“Card stacking” and “Band wagon”. Since then a lot more techniques have been
identified, but this classification is still relevant (some of the techniques
have been included in the SEMEVAL 2020). Probably, the most promising state-
of-art strategies of dealing with propaganda and filtering it from
sentimentally neutral texts is the one of viewing the task as a text
classification problem. Some of the research efforts in propagandist texts
classification are described further in this section.

[Rashkin et al., 2017] investigated news sources of four categories -
truthful, propagandist, hoax and satiric - to show what language features
correspond to each news source category. They examined 50 highest weighted
n-gram features in the MaxEntropy classifier for each class and showed that
truthful sources heavily use specific places (e.g. “London”) as well as exact
time (e.g. “on Tuesday”). For propagandist news features the sources employ
abstract generalities (e.g. “truth”, “freedom”) and specific issues (e.g.
“election”, “shooting”), while for satiric articles vaguely facetious hearsay
(e. g. “reportedly”, “confirmed”) are employed. Finally, hoax publishers
utilize divisive topics (e.g. “liberals”, “trump”) and dramatic cues (e.g.
“breaking”). The authors also proposed that hoax and propagandist sources
often use videos as a reference since words like “video” and “youtube” are
also included in the highest weighted for these categories.

[Barrón-Cedeno et al., 2019] have significantly contributed to the field. They
not only collected a novel dataset that overall contains more than 35000 news
articles both with and without propagandist content, but also compared what
text features allow to efficiently classify articles on propagandist and non-
propagandist. As a result they concluded that stylistic features like
character n-grams show better results than topic-related features. The authors
also demonstrated a service for news articles propaganda level assessment.

[Martino et al., 2019] have proposed another corpus for propaganda
classification. It consists of 451 articles from 48 sources. Every article was
manually annotated sentence by sentence by identifying the presence or absence
of one of 18 propaganda techniques (grouped into 14 classes). The authors also
presented baseline models for binary and multi-class classification based on
BERT [Devlin et al., 2018] and launched a task on propaganda detection that
will be described later.

A major step in the field of propaganda detection was an establishment of
Fine-Grained Propaganda Detection challenge [Da San Martino et al., 2019].
This challenge, similar to the current one, had two tasks:

  * •

The first one was to classify sentences into propaganda and non-propaganda
classes. The sentences were taken from the articles from propagandist and non-
propagandist sources. The metric was F1 score.

  * •

The second task consisted of two parts - extract spans that contain propaganda
from article text and classify what propaganda technique is used. The metric
took into account both span borders and class predictions. In the current
version of the challenge the task was presented by two separate tasks, span
identification and classification.

Overall, 25 teams submitted their solutions of the first task and 12 teams
submitted their solutions of the second task. Leading teams used Transformer-
based [Vaswani et al., 2017] models that are built on a powerful encoder-
decoder architecture having self-attention mechanism as its core. This
mechanism allows to take into account inter-sequence word dependencies,
providing rich semantic representation. Almost all teams used some
Transformer-based models (especially BERT [Devlin et al., 2018]) either to get
embeddings or as a pretrained model [Yoosuf and Yang, 2019] [Hou and Chen,
2019]. Other teams often used ensembles with different features and models
inside: LSTM-CRF [Gupta et al., 2019], XGBoost [Tayyar Madabushi et al.,
2019], BiLSTM [Vlad et al., 2019] and others. Table 1 summarizes approaches
extensively used in the solutions.

Approach | Times it was used
---|---
BERT | 13
LSTM | 7
Language features | 5
ELMo | 3
Logistic regression | 3
Table 1: How often different approaches were used

As Table 1 shows, Transformer-based models (BERT in particular) have become a
leading architecture in propaganda detection field.

##  3 System overview

Considering the top-performing systems of the previous challenge, we decided
to use Transformer-based model as a solution baseline - mainly focusing on
BERT [Devlin et al., 2018] and RoBERTa [Liu et al., 2019]. However, since a
lot of new models have been published since the previous challenge, we also
tested several different Transformer-based models, including DistilBERT [Sanh
et al., 2019], ALBERT [Lan et al., 2019], XLnet [Yang et al., 2019]. After
calculating F1 scores for all of them, we chose the one with the highest
performance - RoBERTa. We also decided to use a model pretrained on a huge
dataset and fine-tune it on the given dataset by adding a fully-connected
classification layer. This strategy is highly suitable for our case because
the training dataset consisted of 371 articles and contained 6129 propaganda
samples in total for all classes, which is not enough for training a good-
performing Transformer model from scratch, but is reasonable for fine-tuning.
The following sections describe the techniques we used for optimizing the
robustness of our model.

###  3.1 Imbalanced Data and Undersampling

One essential feature of the dataset is high class imbalance. Figure 1
demonstrates class distribution among samples.

A34.6%B17.3%C10.1%D8.0%E7.6%F22.4% Figure 1: Class distribution in the train
data, where A is “Loaded Language”, B is “Name Calling or Labeling”, C is
“Repetition”, D is “Doubt”, E is “Exaggeration or Minimisation”, and F
represents all the remaining 9 classes.

As our first attempt to overcome class imbalance we used undersampling
technique, which implies reducing the portion of more frequent class in the
data by removing some samples. In our case, the most frequent classes were
’Loaded Language’ (2123 out of 6129 samples in train set), and ’Name
Calling,Labeling’ (1058 out of 6129 samples in train set). Our aim was to
retain approximately few hundred samples for each of these classes. To obtain
this we experimented with different coefficients: 0.35, 0.3, 0.2 for ’Loaded
Language’ and 0.4, 0.5, 0.6 for ’Name Calling, Labeling’. The best combination
with respect to final F1 score was 0.2 for ’Loaded Language’ and 0.5 for ’Name
Calling,Labeling’.

###  3.2 Cost-sensitive learning

Another technique we tried to deal with class imbalance was cost-sensitive
learning. The idea of this method is to assign different ”cost” to classes
with different inclusion percentage in the dataset. We implemented the method
by assigning label weights during loss calculation. The weight $w_{i}$ for
each label was calculated using formula (formula 3.2):

$w_{i}=\frac{c_{i}}{\sum\limits_{j}{c_{j}}},c_{i}=\frac{\sum\limits_{j}{l_{j}}}{l_{i}}$

where $l_{i}$ is number of samples of label $i$ in the dataset. This formula
(formula 3.2) can be interpreted like this: class weight is inversely
proportional to the portion of this class in the dataset.

###  3.3 Context addition

Having performed the dataset analysis we found out that span samples may
highly vary in length: from one-word long samples to several sentences long
sentences. Moreover, this significant difference may occur within the same
class. Figure 2 shows box-plots of sample lengths grouped by classes. It is
interesting that three most frequent classes (8, 9, 10) have the shortest
average lengths.

Figure 2: Box-plots of length distribution in words inside classes

Our expectation was that spans with such a diverse lengths cannot efficiently
represent all techniques and that some useful information can be left outside
the span boundaries. To check this, we decided to expand the spans with the
surrounding context. We tried two ways of setting the new sample boundaries:
to complete a sentence (any text surrounded by ’.’, ’?’, ’!’ signs or ”end of
line” symbol on both sides) and to complete a sentence or a subsentence (same
as complete a sentence but with ’,’ sign included). However, this spans
expansion strategy was not efficient as it introduced more noise and, thus,
reduced the score.

##  4 Experimental Setup and Results

To choose between different Transformer-based models we trained each one of
them on a single epoch and compared their validation scores. The result of
this evaluation are shown in Table 2.

Model | F1 val. score
---|---
ALBERT | 0.498
DistillBERT | 0.502
XLnet | 0.526
BERT | 0.553
RoBERTa | 0.573
Table 2: Baseline models performance

According to Table 2, RoBERTa showed the best performance results among all
the other models, therefore we chose it as a model for future improvement. It
is also worth mentioning that all models showed similar patterns on the same
classes:

  * •

High performance ($F_{1}\geq 0.5$) on ”Flag-Waving”, ”Loaded Language” and
”Name Calling, Labeling”;

  * •

Intermediate performance ($0.1\leq F_{1}<0.5$) on ”Appeal to fear-prejudice”,
”Doubt”, ”Exaggeration, Minimisation” and ”Repetition”;

  * •

Low performance ($F_{1}<0.1$) on ”Appeal to Authority”, ”Bandwagon, Reductio
ad hitlerum”, ”Black-and-White Fallacy”, ”Causal Oversimplification”,
”Slogans”, ”Thought-terminating Cliches” and ”Whataboutism, Straw Men, Red
Herring”.

As the next step, we tuned the hyperparameters of the model including the
number of epochs, batch size and maximum sequence length. This step allowed us
to improve F1 score for 2 points and provided us with a ready-to-run model so
that we could proceed to optimization techniques described in previous
section. Table 3 shows the result of applying this strategy.

Strategy | F1 val. score
---|---
Baseline | 0.57384
Tuned | 0.59172
Tuned+Undersampling | 0.59266
Tuned+Cost-sensitive learning | 0.60113
Tuned+Context addition | 0.55691
Table 3: Performance with different techniques applied

###  4.1 Our submission

Our final system submitted for this task is based on RoBERTa model, combined
with cost-sensitive learning strategy. We took pretrained weights from
Transformers [Wolf et al., 2019] library by Hugging Face, and fine-tuned the
model on 5 epochs with batch size 8 and maximum sequence length of 512. We
used Simple
Transformers222https://github.com/ThilinaRajapakse/simpletransformers library
for training and inference.

The submitted model showed $0.58$ F1 score on the test set taking the 7th
place among 31 teams in the second track. The system also showed non-zero F1
score on each class (some models with better final performance did not achieve
this), which means that the model can capture some meaning of each propaganda
technique. It is also worth mentioning that the difference between validation
performance and test performance of our model is 2 points, which is less than
that of most models with better test performance and, moreover, it is less
than that of models in top three. Thus, we achieved less overfit compared to
the competitors.

##  5 Conclusion

In this paper we described a system that we developed to participate in
SEMEVAL 2020 Task 11 “Detection of propaganda techniques in news articles”. We
trained a pure Transformer-based model with preprocessing and achieved $0.58$
F1 score on test data and non-zero F1 score for each class. The approach
demonstrated the effectiveness of combining a Transformer-based baseline with
learning optimization techniques including undersampling, cost-sensitive
learning and context augmentation.

We also demonstrated that compared to other NLP tasks that are being actively
solved, propaganda techniques classification task is far behind, and both
models and data are to be improved. However, non-zero scores for each class
show that various baseline models are able to capture propagandist semantics
and stronger models for this field can be built.

## References

  * [Barrón-Cedeno et al., 2019]  Alberto Barrón-Cedeno, Israa Jaradat, Giovanni Da San Martino, and Preslav Nakov.  2019\.  Proppy: Organizing the news based on their propagandistic content.  Information Processing & Management.
  * [Da San Martino et al., 2019]  Giovanni Da San Martino, Seunghak Yu, Alberto Barrón-Cedeño, Rostislav Petrov, and Preslav Nakov.  2019\.  Fine-grained analysis of propaganda in news articles.  In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, EMNLP-IJCNLP 2019, Hong Kong, China, November.
  * [Da San Martino et al., 2020]  Giovanni Da San Martino, Alberto Barrón-Cedeño, Henning Wachsmuth, Rostislav Petrov, and Preslav Nakov.  2020\.  SemEval-2020 task 11: Detection of propaganda techniques in news articles.  In Proceedings of the 14th International Workshop on Semantic Evaluation, SemEval 2020, Barcelona, Spain, September.
  * [Devlin et al., 2018]  Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  2018\.  Bert: Pre-training of deep bidirectional transformers for language understanding.  arXiv preprint arXiv:1810.04805.
  * [Edwards, 1938]  Violet Edwards.  1938\.  Group leader’s guide to propaganda analysis: Revised edition of experimental study materials for use in junior and senior high schools, in college and university classes, and in adult study groups.  Institute for propaganda analysis, Incorporated.
  * [Gupta et al., 2019]  Pankaj Gupta, Khushbu Saxena, Usama Yaseen, Thomas Runkler, and Hinrich Schütze.  2019\.  Neural architectures for fine-grained propaganda detection in news.  In Proceedings of the Second Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda, pages 92–97, Hong Kong, China, November. Association for Computational Linguistics.
  * [Hou and Chen, 2019]  Wenjun Hou and Ying Chen.  2019\.  CAUnLP at NLP4IF 2019 shared task: Context-dependent BERT for sentence-level propaganda detection.  In Proceedings of the Second Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda, pages 83–86, Hong Kong, China, November. Association for Computational Linguistics.
  * [Jowett and O’donnell, 2018]  Garth S Jowett and Victoria O’donnell.  2018\.  Propaganda & persuasion.  Sage Publications.
  * [Lan et al., 2019]  Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut.  2019\.  Albert: A lite bert for self-supervised learning of language representations.  arXiv preprint arXiv:1909.11942.
  * [Liu et al., 2019]  Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov.  2019\.  Roberta: A robustly optimized bert pretraining approach.  arXiv preprint arXiv:1907.11692.
  * [Martino et al., 2019]  Giovanni Da San Martino, Seunghak Yu, Alberto Barrón-Cedeno, Rostislav Petrov, and Preslav Nakov.  2019\.  Fine-grained analysis of propaganda in news articles.  arXiv preprint arXiv:1910.02517.
  * [Rashkin et al., 2017]  Hannah Rashkin, Eunsol Choi, Jin Jang, Svitlana Volkova, and Yejin Choi.  2017\.  Truth of varying shades: Analyzing language in fake news and political fact-checking.  pages 2931–2937, 01.
  * [Sanh et al., 2019]  Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf.  2019\.  Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.  arXiv preprint arXiv:1910.01108.
  * [Tayyar Madabushi et al., 2019]  Harish Tayyar Madabushi, Elena Kochkina, and Michael Castelle.  2019\.  Cost-sensitive BERT for generalisable sentence classification on imbalanced data.  In Proceedings of the Second Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda, pages 125–134, Hong Kong, China, November. Association for Computational Linguistics.
  * [Vaswani et al., 2017]  Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  2017\.  Attention is all you need.  In Advances in neural information processing systems, pages 5998–6008.
  * [Vlad et al., 2019]  George-Alexandru Vlad, Mircea-Adrian Tanase, Cristian Onose, and Dumitru-Clementin Cercel.  2019\.  Sentence-level propaganda detection in news articles with transfer learning and BERT-BiLSTM-capsule model.  In Proceedings of the Second Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda, pages 148–154, Hong Kong, China, November. Association for Computational Linguistics.
  * [Wolf et al., 2019]  Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R’emi Louf, Morgan Funtowicz, and Jamie Brew.  2019\.  Huggingface’s transformers: State-of-the-art natural language processing.  ArXiv, abs/1910.03771.
  * [Yang et al., 2019]  Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V Le.  2019\.  Xlnet: Generalized autoregressive pretraining for language understanding.  arXiv preprint arXiv:1906.08237.
  * [Yoosuf and Yang, 2019]  Shehel Yoosuf and Yin Yang.  2019\.  Fine-grained propaganda detection with fine-tuned BERT.  In Proceedings of the Second Workshop on Natural Language Processing for Internet Freedom: Censorship, Disinformation, and Propaganda, pages 87–91, Hong Kong, China, November. Association for Computational Linguistics.