# Noisy Self-Training with Data Augmentations for Offensive and Hate Speech
Detection Tasks

João A. Leite1 Carolina Scarton1 Diego F. Silva2

1Department of Computer Science, The University of Sheffield, Sheffield (UK)
2Instituto de Ciências Matemáticas e de Computação, Universidade de São Paulo,
São Carlos (Brazil)
{jaleite1<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Online social media is rife with offensive and hateful comments, prompting the
need for their automatic detection given the sheer amount of posts created
every second. Creating high-quality human-labelled datasets for this task is
difficult and costly, especially because non-offensive posts are significantly
more frequent than offensive ones. However, unlabelled data is abundant,
easier, and cheaper to obtain. In this scenario, self-training methods, using
weakly-labelled examples to increase the amount of training data, can be
employed. Recent “noisy” self-training approaches incorporate data
augmentation techniques to ensure prediction consistency and increase
robustness against noisy data and adversarial attacks. In this paper, we
experiment with default and noisy self-training using three different textual
data augmentation techniques across five different pre-trained BERT
architectures varying in size. We evaluate our experiments on two
offensive/hate-speech datasets and demonstrate that (i) self-training
consistently improves performance regardless of model size, resulting in up to
+1.5% F1-macro on both datasets, and (ii) noisy self-training with textual
data augmentations, despite being successfully applied in similar settings,
decreases performance on offensive and hate-speech domains when compared to
the default method, even with state-of-the-art augmentations such as
backtranslation.

##  1 Introduction

Online social media platforms are widely used by modern society for many
productive purposes. However, they are also known for intensifying offensive
and hateful comments, attributed in part to factors such as user anonymity
Mondal et al. (2017). Manual identification of hate speech is impractical at
scale due to the massive number of posts generated every second and the
potential harm to the mental health of moderators. Therefore, there is a need
for automatic approaches to detect offensive and hateful speech.

In recent years, research on this topic has increased, resulting in new models
and datasets published in various languages and sources Fortuna and Nunes
(2018). A common characteristic among available datasets is label skewness
towards the negative class (non-offensive/hateful), which is usually more
frequent than the positive class (offensive/hateful). Apart from traditional
ways of dealing with imbalanced classes (e.g. under or oversampling or
applying class weighting), semi-supervised techniques such as self-training
can be used to extend the training set with unseen examples that introduce new
learning signals without the costly burden of manual data labeling.

Self-training is a technique that involves iteratively training models using
both labelled and unlabelled data. The process begins by training a model
using human-labelled data only, which is then used to infer labels for a set
of unlabelled data, creating a weakly-labelled dataset. The weakly-labelled
dataset and the human-labelled dataset are then aggregated and used to retrain
the model. This iterative process is repeated for a fixed number of steps or
until no performance improvement is observed. Self-training can be
particularly useful when labelled data is scarce or expensive to obtain, and
was successfully applied in a variety of domains such as computer vision
Schiappa et al. (2022), audio and speech processing Liu et al. (2022), and
natural language processing He et al. (2019).

Several variants of self-training have been proposed over the years (Amini et
al., 2022). One common approach is to use a teacher-student framework, in
which the “student” model learns from the output generated by the “teacher”
model (Blum and Mitchell, 1998; Xie et al., 2020b; Chen et al., 2021;
Karamanolakis et al., 2021). Additionally, a confidence threshold filter may
be applied to remove examples that are too ambiguous or non-informative. This
process is summarised in Figure 1.

Figure 1: Teacher-student self-training loop

Recent research on self-training has reported further improvements in
performance by introducing perturbations directly into the raw input or to its
latent representation, improving generalisation and convergence Rasmus et al.
(2015); Laine and Aila (2017); Miyato et al. (2018); He et al. (2019); Xie et
al. (2020a). These perturbations are often introduced in the form of data
augmentations, which are widely applied in Computer Vision tasks but are less
commonly explored in Natural Language Processing tasks, especially in the
context of self-training. These “noisy self-training” methods can be
particularly useful in settings where the input data is noisy or subject to a
high degree of variation, improving prediction consistency and adversarial
robustness (Carmon et al., 2019; Alayrac et al., 2019; Najafi et al., 2019).

Bayer et al. (2022) argue that data augmentation depends on the underlying
classification task, thus it cannot be effectively applied in all
circumstances. Previous work focusing solely on data augmentation methods, not
coupled with self-training, has shown mixed results for the domain of
offensive/hate speech classification (Section 2.1). This indicates that there
may not be a best method, while some may even negatively impact performance.

An open question is whether noisy self-training with text data augmentations
can contribute to text classification tasks using state-of-the-art transfer-
learning BERT models that have been shown to be invariant to various data
transformations (Longpre et al., 2020). The task of offensive/abusive speech
detection poses a difficult challenge for generating high-quality semantic
invariant augmented examples, since it is a domain that is intrinsically
associated with specific keywords that, if modified, can completely change the
semantics of the text. In this paper, we innovate by providing an extensive
experimentation setup using three different data augmentation techniques -
backtranslation, random word swap, and random synonym substitution - in a
self-training framework, with five different pre-trained BERT architectures
varying in size, on two different datasets.

We demonstrate that self-training, either with or without data noising,
outperforms default fine-tuning regardless of model size, on both datasets.
However, when comparing self-training without data noising vs ‘noisy’ self-
training, we find that data augmentations decrease performance, despite the
literature reporting the superiority of noisy self-training in other domains.
We further investigate how the augmentation methods fail to create label-
invariant examples for the offensive/hate speech domain. Finally, we discuss
future research ideas to address the limitations found in this work.

##  2 Related Work

###  2.1 Data Augmentation

Bayer et al. (2022) present a survey on data augmentation methods for NLP
applications, reporting performance gains on various tasks.

In the domain of offensive/hate speech classification, Ibrahim et al. (2018)
experiment with three different text augmentation techniques to expand and
balance their Wikipedia dataset by augmenting negative (non-offensive)
examples. From a binary view of the dataset, more than 85% of their examples
are labelled as non-offensive, and from a multi-label view of the dataset,
three of the six offensive classes are represented by less than 7% of the
dataset. They report F1-score increases of +1.4% with unique words
augmentation, +2.9% with unique words and random mask, and +3.6% with unique
words, random mask, and synonym replacement.

Mosolova et al. (2018) use a custom synonym replacement augmentation method to
experiment with a ‘toxic’ dataset with 6 classes from a Kaggle
competition111https://www.kaggle.com/c/jigsaw-toxic-comment-classification-
challenge. They experiment with character and word embeddings with a CNN
architecture, and report a +3.7% and +5.1% ROC-AUC increase when applying
their augmentation method with character embeddings on the public and private
scores222Public scores are computed over a smaller portion of the test set. At
the end of the competition, private scores are computed with the remainder of
the test set., respectively. However, when coupled with word embeddings, they
find that their augmentations result in a decrease of -0.09% and -0.21% ROC-
AUC scores on the public and private scores, respectively.

Rizos et al. (2019) propose three text-based data augmentation techniques to
address the class imbalance in datasets, and apply them on three English hate
speech datasets named HON Davidson et al. (2017), RSN-1 Waseem and Hovy (2016)
and RSN-2 Waseem (2016). Their augmentation methods include (i) synonym
replacement based on word embedding, (ii) warping of the token words along the
padded sequence, and (iii) class-conditional RNN language generation. They
compare the three methods on different architectures combining word
embeddings, CNNs, GRUs, and LSTMs, and they report an average across four
different architecture configurations of -6.3% F1-Macro using (i), +5%
F1-Macro using (ii), and -4% F1-Macro using (iii).

Marivate and Sefara (2020) experiment with four different data augmentation
techniques: WordNet synonym substitution, backtranslation between German and
English, word embedding substitution according to cosine similarity, and mixup
Zhang et al. (2018). Authors experiment with three datasets from different
domains: Sentiment 140 Go et al. (2009), AG News Zhang et al. (2015) and a
Hate Speech dataset Davidson et al. (2017). They observe performance increases
on both Sentiment 140 and AG News across different augmentation methods, up to
+0.4% and +0.5% accuracy score on AG News and Sentiment 140, respectively.
However, they report performance decreases with all methods on the Hate Speech
dataset, with decreases of 0.0% with mixup, -0.3% with embedding similarity,
-0.8% with synonym substitution, and -2.3% with backtranslation.

###  2.2 Self-Training

Xie et al. (2020b) present a method called noisy student, which achieves
state-of-the-art results on the ImageNet dataset (Deng et al., 2009) by
performing self-training with a teacher-student approach, using student models
that are equal or larger-sized than the teacher models, and adding noise both
to the input data through random image augmentations and to the model via
dropout.

He et al. (2019) apply a similar idea using textual data augmentation methods
such as backtranslation (Edunov et al., 2018) and token modifications to a
self-training LSTM architecture for the tasks of machine translation and text
summarization. They find that both model noise, in the form of dropout, and
data noise, in the form of data augmentations, are crucial to their observed
increase in performance on both tasks.

Xie et al. (2020a) use six text classification and two image classification
benchmark datasets to experiment with different types of noise-inducing
techniques for self-training. They argue that state-of-the-art augmentations
like backtranslation for text classification and RandAugment (Cubuk et al.,
2020) for image classification, outperform simple noise inducing techniques,
such as additive Gaussian noise.

The use of noisy self-training approaches in the domain of offensive/hate
speech classification is still limited, but default ‘non-noisy’ self-training
has been successfully applied in some recent works. Alsafari and Sadaoui
(2021) collect unlabelled Arabic tweets and perform semi-supervised
classification with self-training for the domain of Offensive and Hate Speech
detection using multiple text representations such as N-grams, Word2Vec,
AraBert and Distilbert, and multiple model architectures such as SVM, CNN and
BiLSTM. They report up to 7% performance increase in low resource settings
where only a few labelled examples are available.

Leonardelli et al. (2020) apply self-training in their submission to the
HaSpeeDe shared task on Italian hate speech detection (task A). They fine-tune
an AlBERTo model with the human-labelled dataset provided by the task
organisers and extend it with a weakly-labelled dataset using self-training.
Additionally, they oversample the human-labelled set in an attempt to make the
model more robust to inconsistencies in the weakly-labelled set. Their
submission achieve an F1-macro score of 75.3% on tweets, placing 11th out of
29 teams, and 70.2% on news headlines, placing 5th out of 29 teams.

Pham-Hong and Chokshi (2020) report experiments with the noisy student method
from Xie et al. (2020b) in the OffensEval 2020 shared task, achieving 2nd
place at subtask B (Automatic categorization of offense types). In their
setup, although dropout is applied to a BERT-large model, no noise is injected
into the data, which is a crucial component of the noisy student method.
Because of this, we argue that this work is actually applying a default self-
training method instead of a noisy self-training method. Also, OffensEval
2020’s training data does not contain human-labelled data333In OffensEval
2020, the labels in the training data are the average confidence score and
confidence standard deviation aggregated from an ensemble of models., thus
both their weakly-labelled dataset and ground-truth dataset consist of
inferred examples.

Richardson et al. (2022) detect hate speech on Twitter in the context of the
Covid-19 pandemic. They employ a simple approach, utilizing a bag-of-words
representation combined with an SVM classifier. Authors demonstrate that by
employing self-training with only 20% of the training data, they manage to
improve accuracy by +1.55% compared to default training using 80% of the
training data.

To the best of our knowledge, Santos et al. (2022) is the only previous work
in which a noisy self-training approach was attempted on an offensive/hate
speech classification task. They propose an ensemble of two semi-supervised
models to create FIGHT, a Portuguese hate speech corpus. Authors combine GANs,
a BERT-based model, and a label propagation model, achieving 66.4% F1-score.
They attempt to increase performance using backtranslation as data
augmentation, but ultimately observe no performance gains, thus their best
model is obtained with default self-training, not with noisy self-training.

##  3 Materials and Methods

This section presents the description of the datasets, data augmentation
methods and self-training architectures used throughout our experiments. Our
code is available at GitHub444https://github.com/JAugusto97/Offense-Self-
Training.

###  3.1 Data Description

We use two English binary offensive/hate speech detection datasets in our
experiments. Table 1 presents their target class distributions.

OLID
---
| Train | Dev | Test
Not-Offensive | 8,840 | 0 | 620
Offensive | 4,400 | 0 | 240
ConvAbuse
| Train | Dev | Test
Not-Offensive | 2,163 | 719 | 725
Offensive | 338 | 112 | 128
Table 1: Target class distribution for OLID and ConvAbuse.

#### Offensive Language Identification Dataset (OLID)

(Zampieri et al., 2019) contains a collection of annotated tweets following
three levels: Offensive Language Detection, Categorization of Offensive
Language, and Offensive Language Target Identification. This work only uses
the first level - Offensive Language Detection. The dataset was normalised by
replacing URLs and user mentions with placeholders. The best model in
(Zampieri et al., 2019) achieves 80% macro-$F1$ using convolutional neural
networks, with 70% and 90% of $F1$-Score for the positive and negative
classes, respectively.

#### ConvAbuse

(Cercas Curry et al., 2021) is a dataset on abusive language towards three
conversational AI systems: an open-domain social bot, a rule-based chatbot,
and a task-based system. Authors find that the distribution of abuse towards
conversational systems differs from other commonly used datasets, with more
than 50% of the instances containing sexism or sexual harassment. To normalise
the data, web addresses were replaced with a placeholder. Authors provide
standard train, development, and test sets and achieve up to 88.92% macro-$F1$
using a fine-tuned BERT model. In our experiments, we concatenate the
interactions between the user and the chatbot into a single text document
divided by new line separators, and we use majority voting between the
annotations to consolidate the binary abusive vs. non-abusive label.

#### Unlabelled data

We collected 365,456 tweets in English with the Twitter API using an unbiased
query rule: random tweets mentioning stop-words like “in”, “on”, “a”, “is”,
“not”, “or” and so on. We also preprocess the data by removing user mentions,
urls, punctuations, extra whitespace and accents.

###  3.2 Self-Training Architecture

Our noisy self-training system is similar to that introduced by Xie et al.
(2020b) and Xie et al. (2020a), and works as follows:

  1. 1.

A teacher model is trained to minimise the cross-entropy loss on the human-
labelled training set exclusively.

  2. 2.

The teacher model infers weak labels from the unlabelled dataset.

     * •

A confidence threshold filter is applied, and examples that fall below this
threshold are removed.

     * •

Apply downsampling on the inferred examples, ending up with a perfectly
balanced weakly-labelled dataset.

  3. 3.

All the examples selected from the previous step are augmented once with one
of the data augmentation methods, doubling the amount of weakly-labelled
examples. The labels obtained with the ‘clean/without noise’ text in step 2
are replicated for the augmented texts.

  4. 4.

An equal-sized student model minimises the combined cross-entropy loss on
human-labelled and weakly-labelled datasets:

$L=\frac{1}{n}\sum_{i=1}^{n}L_{\mathrm{labelled}}+\frac{1}{m}\sum_{i=1}^{m}L_{\mathrm{inferred}}$
(1)

  5. 5.

Repeat from step 2 using the current student model as the teacher model.

In our experiments, we compare this noisy self-training framework against the
default ’non-noisy’ self-training method, which simply skips step 3, meaning
we do not apply any form of data augmentation.

###  3.3 Data Augmentation Methods

In each noisy self-training experiment we use
nlpaug555https://github.com/makcedward/nlpaug to apply one of the three
following data augmentation methods for textual data:

#### Random Synonym Substitution

Uses WordNet Miller (1995) to randomly replace tokens by one of its synonyms.
For each sentence, 30% of its tokens will be replaced.

#### Random Word Swap

Randomly swaps adjacent tokens in a sentence. For each sentence, 30% of its
tokens are swapped.

#### Backtranslation

First translates the original texts into a second language, then translates
them back from the second language to the original language. We use the
backtranslation model from nlpaug, which uses the two different transformer
models from Ng et al. (2019) to translate the data from English to German,
then from German back to English.

##  4 Experimental Setup

Firstly, we experiment with each dataset to estimate the hyperparameters for
the base models, which is the first teacher models in the self-training loop.
We use a batch size of 128, maximum sequence length of 128, learning rate of
0.00001, 15% of the training set as warm-up batches, weight decay of 0.001 and
20 training epochs. We apply a dropout rate of 10% for both the attention and
classification layers. The model with highest validation F1-macro
score666Lowest training loss in the case of OLID, since no development set is
provided. obtained during training is loaded at the end of the last epoch. For
the hyperparameters associated with the self-training method, we set the
number of teacher-student iterations to 4 (including the first teacher model)
and a confidence threshold filter of 80%, similarly to Xie et al. (2020a).
Also, we experiment with five different pre-trained BERT models: DistilBERT,
BERT-base-cased, BERT-large-cased, RoBERTa-base and RoBERTa-large, aiming to
investigate the impact of model size in performance gains associated with
self-training.

From the above-listed configurations, we designed two main classification
scenarios. The first scenario accounts for a regular self-training loop
without data noise injection through augmentations, while the second scenario
uses the noisy self-training approach, introducing data noise with one of the
three augmentation methods described in Section 3.3.

Finally, we conduct a deeper analysis of each augmentation method. We use the
first teacher model, trained exclusively with the human-labelled data of each
dataset, to infer both the ’clean/without augmentation’ and the
’noisy/augmented’ versions of the unlabelled dataset and verify the following:
(i) Does the augmentation method create new tokens that are not present in the
vocabulary of the ’clean/without augmentation’ unlabelled dataset? and (ii)
Are the augmentations semantically invariant, meaning both the ’clean’ and
’noisy’ pairs of examples are assigned the same label?

##  5 Results

OLID
---
Architecture | DF | ST | ST + BT | ST + SS | ST + WS
DistilBERT | 78.4 ± 0.1 | 79.2 ± 0.2 | 79.0 ± 0.3 | 79.0 ± 0.3 | 79.0 ± 0.3
BERT-base-cased | 77.2 ± 0.3 | 78.7 ± 0.1 | 78.1 ± 0.1 | 78.3 ± 0.3 | 78.3 ± 0.3
BERT-large-cased | 79.2 ± 0.2 | 80.0 ± 0.3 | 79.4 ± 0.1 | 79.3 ± 0.3 | 79.3 ± 0.3
RoBERTa-base | 79.4 ± 0.7 | 80.1 ± 0.3 | 80.0 ± 0.4 | 80.0 ± 0.4 | 80.0 ± 0.4
RoBERTa-large | 79.8 ± 0.3 | 80.4 ± 0.4 | 80.3 ± 0.4 | 80.7 ± 0.7 | 80.7 ± 0.7
ConvAbuse
Architecture | DF | ST | ST + BT | ST + SS | ST + WS
DistilBERT | 85.7 ± 0.5 | 86.8 ± 0.3 | 87.1 ± 0.3 | 87.2 ± 0.3 | 87.2 ± 0.3
BERT-base-cased | 86.8 ± 0.8 | 87.6 ± 0.1 | 87.2 ± 0.5 | 87.2 ± 0.5 | 87.2 ± 0.5
BERT-large-cased | 87.1 ± 0.6 | 87.9 ± 0.5 | 87.4 ± 0.2 | 87.9 ± 0.5 | 87.9 ± 0.5
RoBERTa-base | 84.5 ± 0.3 | 85.5 ± 0.4 | 85.3 ± 0.8 | 85.4 ± 0.5 | 85.4 ± 0.5
RoBERTa-large | 86.0 ± 0.1 | 86.2 ± 0.3 | 86.6 ± 0.3 | 86.9 ± 0.1 | 86.8 ± 0.1
Table 2: Mean ± 1 std F1-Macro scores obtained over three random seed
initializations.
DF=Default Fine-Tuning, ST=Self-Training, BT=Backtranslation, SS=Synonym
Substitution, WS=Word Swap

###  5.1 Default Fine-Tuning vs. Self-Training

Table 2 displays the mean and standard deviation $F1$-macro scores computed
over three different random seed initializations for each experiment. Note
that self-training, regardless of whether coupled with data augmentation
methods or not, improves over default fine-tuning for every model
architecture, increasing the F1-macro score from +0.7% up to +1.5% on OLID and
+0.8% up to +1.5% on ConvAbuse depending on the pre-trained model
architecture.

Also, we highlight how self-training can make smaller models, which require
fewer resources to maintain in practical applications, achieving the same
performance as larger and more costly models that are trained with default
fine-tuning. Self-training on a DistilBERT (66M parameters) outperforms a
BERT-large-cased (340M parameters) with default fine-tuning on both OLID and
ConvAbuse. On OLID, a RoBERTa-base architecture (125M parameters) with self-
training outperforms a RoBERTa-large (354M parameters) architecture with
default fine-tuning, although this does not hold true for ConvAbuse.

Furthermore, we point out that OLID and ConvAbuse’s data come from different
sources, the first being Twitter, and the second one representing
conversations between humans and chatbots, thus their structure differs
significantly. Since our unlabelled dataset is composed of Twitter data, it
would be fair to assume that the benefits of self-training in our experiments
would be more prominent for the OLID dataset, but our results do not show
this, since models trained with ConvAbuse benefited from self-training with
our Twitter-originated unlabelled dataset just as much as models trained with
OLID.

###  5.2 Default Self-Training vs. Noisy Self-Training

After verifying that self-training is beneficial to both datasets on all model
architectures, we compare default self-training with noisy self-training, and
the impacts of adding data noise in the form of data augmentations. We find
that introducing data augmentations to the self-training pipeline increases
performance against default self-training only for RoBERTa-large on both OLID
and ConvAbuse, with DistilBERT also showing improvements for ConvAbuse, but
not for OLID. On all other architectures, for both datasets, default self-
training without data augmentations achieves the highest scores.

In our results for offensive/hate speech classification, backtranslation does
not achieve the highest score in any setup, while synonym substitution and
word swap tie for highest score in three scenarios: ConvAbuse with DistilBERT,
ConvAbuse with BERT-large-cased, and OLID with RoBERTa-large. Synonym
substitution outperforms all the remaining methods on ConvAbuse with RoBERTa-
large.

An important remark is that our results diverge from He et al. (2019), which
finds that state-of-the-art data augmentation methods such as backtranslation
outperform simpler methods on self-training for machine translation and text
summarization. However, our results align with Marivate and Sefara (2020),
although their work is not focused on self-training, but instead on how
different data augmentation techniques impact their models on three datasets
from different domains. They report backtranslation as their worst
augmentation method on a hate speech dataset, decreasing accuracy by -2.3%.
Our findings bridge this gap and reveal that backtranslation has significant
limitations in the domain of offensive/hate speech detection, even when used
in a noisy self-training approach.

###  5.3 Data Augmentation Analysis

Our first data augmentation analysis is to understand if the augmented text
introduces new unseen tokens to the vocabulary of the ‘clean’ unlabelled set
when both are combined. We find a vocabulary size increase of 39.5%, 9.0% and
4.7% averaging across all different pre-trained architectures for
backtranslation, synonym substitution and word swap777Word swap is
unintuitively capable of creating new tokens depending on how a sentence is
split into tokens and then merged back after swapping the tokens.
respectively. This indicates that backtranslation is heavily superior in terms
of introducing new unseen tokens, but this is not correlated with performance
increase, as backtranslation appears as the worst augmentation method for
noisy self-training in our classification experiments.

Next, in order to verify the performance of the data augmentation methods in
generating semantically invariant examples, we use the base models trained
exclusively with the human-labelled data from each dataset, on each pre-
trained architecture, and use them to perform inference on both the ‘clean’
and the noisy/augmented unlabelled set. We then compare both predictions and
analyse how augmentations may shift the underlying target class. We will refer
to positive shift when a non-offensive example is classified as offensive
after being augmented, and negative shift when an offensive example is
classified as non-offensive after being augmented.

Table 3 presents the total class shift percentage for each augmentation
method, averaging across both datasets and all model architectures, of which
we further divide into positive and negative label shift percentages. Notice
that backtranslation is the method that produces the highest amount of label
shifting at 23.8%, of which 54.7% are negative shifts, which is a 6.6%
increase over synonym substitution and a 4.8% increase word swap.

Augmentation | Total Shift | Positive Shift | Negative Shift
---|---|---|---
BT | 23.8% | 46.7% | 54.7%
SS | 23.5% | 48.7% | 51.3%
WS | 23.3% | 47.8% | 52.2%

Table 3: Average target class shift percentage on the weakly-labelled set.
BT=Backtranslation, SS=Synonym Substitution, WS=Word Swap

It is fair to assume that not all of the class shifting occurs from the
augmentation changing the semantic that defines if an example is either
offensive or not-offensive. In most cases, class shifting may occur because of
small perturbations that are semantically invariant, meaning both the ’clean’
and the augmented text’s true underlying classes are still the same, even if
the classifier predicted them as different classes. In these cases, when we
set the label of the augmented text to be the same as the one obtained when
inferring the ’clean’ version of the text, as presented in section 3.2, we are
reinforcing the model to be more robust against these small perturbations,
which is one of the main benefits of noisy self-training. However, when
augmentation methods create semantically different versions of the original
texts, replicating the inferred label from the original text to the augmented
text results in the addition of incorrect ground-truth labels to the train
set, which may degrade performance.

Currently, to the best of our knowledge, there is no dataset annotated for
offense/hate speech before and after applying data augmentation, which would
enable a more accurate estimation of semantic variations produced by them. In
tables 4 and 5 we show two examples for each augmentation method that suffered
from positive shift (not-offensive to offensive) and negative shift (offensive
to not-offensive), respectively.

Text | Augmented Text | Method
---|---|---
I HATE ALL OF YOU | ALL I HATE OF YOU | WS
Maybe I dont respect all women | Maybe I respect dont women all | WS
Bitches and sports | Females and Sport | BT
Wooooow what the fuck | Wooooow, what the hell? | BT
Bitch you better be joking | Gripe you good be joking | SS
The NYT has been showing its whole ass […] | The NYT has follow showing its whole butt […] | SS
Table 4: Examples of Offensive to Not-Offensive semantic shift created by data
augmentation.
BT=Backtranslation, SS=Synonym Substitution, WS=Word Swap Text | Augmented Text | Method
---|---|---
Is that Fat Albert | That Fat is Albert | WS
Man that is terrible | That man is terrible | WS
damn white people oppressing the blacks | fucking white people who oppress the blacks | BT
That damn staircase be beating my ass […] | That fucking staircase will bang my ass […] | BT
i will not get over this | i will not fuck off ended this | SS
Send me the link and Ill love you forever | Send pine tree state the link and Ill fuck you forever | SS
Table 5: Examples of Not-Offensive to Offensive class shift created by data
augmentation.
BT=Backtranslation, SS=Synonym Substitution, WS=Word Swap

An example of a recurrent theme among various target shifted examples is the
substitution of the keywords ‘fuck’ with ‘damn’ or ‘hell’, indicating that
despite these keywords being semantically similar, they are not always
interchangeable with respect to the target class, and the mere replacement of
one for another is enough to shift the target class. This could be expected,
as offense detection is highly impacted by the mere presence or absence of
offensive keywords.

##  6 Conclusion

In this work, we analysed the impact of self-training on offensive and hate
speech classification tasks using five different pre-trained BERT models of
varying sizes and two different datasets. We also experimented with noisy
self-training using three different data augmentation techniques for textual
data. We found that self-training improves classification performance for all
model architectures on both datasets, with an increase in F1-Macro of up to
+1.5%. However, our experiments comparing default self-training versus noisy
self-training showed that noisy self-training does not improve performance,
despite its success in other domains. Finally, we investigated the three data
augmentation methods and showed that the domain of offensive/hate speech
classification is highly sensitive to semantic variances produced by them, and
we discussed future research ideas to mitigate these problems.

##  7 Future Work

We understand that some of the semantic variations discussed in this work
could be mitigated by data augmentation methods that both preserve existing
offensive keywords, and do not introduce new offensive keywords randomly, as
these are often conditional to the underlying ground-truth class. For some
languages, most of these keywords are extensively
documented888https://hatebase.org/, thus they can be known a priori by these
methods, and be treated differently, such as only substituting an offensive
keyword by another offensive keyword, or not allowing a non-offensive keyword
to be substituted by an offensive keyword. This custom approach can
theoretically help mitigate semantic variations in this domain, but
offensive/hateful comments can still be made without making use of a single
offensive/hateful keyword. In these more subtle cases, a system would have to
detect the offensive/hateful context without relying solely on keywords, and
modify the example while still maintaining this context. We see potential
benefits of using recent instruction-tuned large language models Ouyang et al.
(2022) as specialised data augmentation methods that are task-specific, and
can be able to preserve the semantics associated with the task when modifying
a given text. In this scenario, an instruction prompt can be designed to
inform the system of the context of the task, and make it aware that this
semantic must be preserved when modifying the given text. In the future, we
aim towards extending this work with the above-mentioned research ideas.

## Acknowledgments

We thank Olesya Razuvayevskaya and Freddy Heppell for their valuable feedback.
This research has been funded by ”SoBigData++: European Integrated
Infrastructure for Social Mining and Big Data Analytics” (EU H2020, Grant
Agreement n.871042 (http://www.sobigdata.eu)).

## References

  * Alayrac et al. (2019) Jean-Baptiste Alayrac, Jonathan Uesato, Po-Sen Huang, Alhussein Fawzi, Robert Stanforth, and Pushmeet Kohli. 2019.  Are labels required for improving adversarial robustness? In _Advances in Neural Information Processing Systems_ , volume 32. Curran Associates, Inc.
  * Alsafari and Sadaoui (2021) Safa Alsafari and Samira Sadaoui. 2021.  Semi-supervised self-training of hate and offensive speech from social media.  _Applied Artificial Intelligence_ , 35(15):1621–1645.
  * Amini et al. (2022) Massih-Reza Amini, Vasilii Feofanov, Loic Pauletto, Emilie Devijver, and Yury Maximov. 2022.  Self-training: A survey.  _arXiv preprint arXiv:2202.12040_.
  * Bayer et al. (2022) Markus Bayer, Marc-André Kaufhold, and Christian Reuter. 2022.  A survey on data augmentation for text classification.  _ACM Computing Surveys_ , 55(7):1–39.
  * Blum and Mitchell (1998) Avrim Blum and Tom Mitchell. 1998.  Combining labeled and unlabeled data with co-training.  In _Proceedings of the Eleventh Annual Conference on Computational Learning Theory_ , COLT’ 98, page 92–100, New York, NY, USA. Association for Computing Machinery.
  * Carmon et al. (2019) Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy S Liang. 2019.  Unlabeled data improves adversarial robustness.  In _Advances in Neural Information Processing Systems_ , volume 32. Curran Associates, Inc.
  * Cercas Curry et al. (2021) Amanda Cercas Curry, Gavin Abercrombie, and Verena Rieser. 2021.  ConvAbuse: Data, analysis, and benchmarks for nuanced abuse detection in conversational AI.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 7388–7403, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Chen et al. (2021) X. Chen, Y. Yuan, G. Zeng, and J. Wang. 2021.  Semi-supervised semantic segmentation with cross pseudo supervision.  In _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 2613–2622, Los Alamitos, CA, USA. IEEE Computer Society.
  * Cubuk et al. (2020) E. D. Cubuk, B. Zoph, J. Shlens, and Q. V. Le. 2020.  Randaugment: Practical automated data augmentation with a reduced search space.  In _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)_ , pages 3008–3017, Los Alamitos, CA, USA. IEEE Computer Society.
  * Davidson et al. (2017) Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017.  Automated hate speech detection and the problem of offensive language.  _Proceedings of the International AAAI Conference on Web and Social Media_ , 11(1):512–515.
  * Deng et al. (2009) J. Deng, W. Dong, R. Socher, L. Li, Kai Li, and Li Fei-Fei. 2009.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops)_ , pages 248–255, Los Alamitos, CA, USA. IEEE Computer Society.
  * Edunov et al. (2018) Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018.  Understanding back-translation at scale.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 489–500, Brussels, Belgium. Association for Computational Linguistics.
  * Fortuna and Nunes (2018) Paula Fortuna and Sérgio Nunes. 2018.  A survey on automatic detection of hate speech in text.  _ACM Comput. Surv._ , 51(4).
  * Go et al. (2009) Alec Go, Richa Bhayani, and Lei Huang. 2009.  Twitter sentiment classification using distant supervision.  _CS224N project report, Stanford_ , 1(12):2009.
  * He et al. (2019) Junxian He, Jiatao Gu, Jiajun Shen, and Marc’Aurelio Ranzato. 2019.  Revisiting Self-Training for Neural Sequence Generation.  _arXiv e-prints_ , page arXiv:1909.13788.
  * Ibrahim et al. (2018) Mai Ibrahim, Marwan Torki, and Nagwa El-Makky. 2018.  Imbalanced toxic comments classification using data augmentation and deep learning.  In _2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA)_ , pages 875–878.
  * Karamanolakis et al. (2021) Giannis Karamanolakis, Subhabrata (Subho) Mukherjee, Guoqing Zheng, and Ahmed H. Awadallah. 2021.  Self-training with weak supervision.  In _NAACL 2021_. NAACL 2021.
  * Laine and Aila (2017) Samuli Laine and Timo Aila. 2017.  Temporal ensembling for semi-supervised learning.  In _5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings_. OpenReview.net.
  * Leonardelli et al. (2020) Elisa Leonardelli, Stefano Menini, and Sara Tonelli. 2020.  Dh-fbk@ haspeede2: Italian hate speech detection via self-training and oversampling.  In _Proceedings of the Seventh Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2020)_ , volume 2765.
  * Liu et al. (2022) Shuo Liu, Adria Mallol-Ragolta, Emilia Parada-Cabaleiro, Kun Qian, Xin Jing, Alexander Kathan, Bin Hu, and Björn W. Schuller. 2022.  Audio self-supervised learning: A survey.  _Patterns_ , 3(12):100616.
  * Longpre et al. (2020) Shayne Longpre, Yu Wang, and Chris DuBois. 2020.  How effective is task-agnostic data augmentation for pretrained transformers? In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 4401–4411, Online. Association for Computational Linguistics.
  * Marivate and Sefara (2020) Vukosi Marivate and Tshephisho Sefara. 2020.  Improving short text classification through global augmentation methods.  In _Machine Learning and Knowledge Extraction: 4th IFIP TC 5, TC 12, WG 8.4, WG 8.9, WG 12.9 International Cross-Domain Conference, CD-MAKE 2020, Dublin, Ireland, August 25–28, 2020, Proceedings 4_ , pages 385–399. Springer.
  * Miller (1995) George A Miller. 1995.  Wordnet: a lexical database for english.  _Communications of the ACM_ , 38(11):39–41.
  * Miyato et al. (2018) Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii. 2018.  Virtual adversarial training: a regularization method for supervised and semi-supervised learning.  _IEEE transactions on pattern analysis and machine intelligence_ , 41(8):1979–1993.
  * Mondal et al. (2017) Mainack Mondal, Leandro Araújo Silva, and Fabrício Benevenuto. 2017.  A measurement study of hate speech in social media.  In _Proceedings of the 28th ACM Conference on Hypertext and Social Media_ , HT ’17, page 85–94, New York, NY, USA. Association for Computing Machinery.
  * Mosolova et al. (2018) Anna Mosolova, Vadim Fomin, and Ivan Bondarenko. 2018.  Text augmentation for neural networks.  _AIST (Supplement)_ , 2268:104–109.
  * Najafi et al. (2019) Amir Najafi, Shin-ichi Maeda, Masanori Koyama, and Takeru Miyato. 2019.  Robustness to adversarial perturbations in learning from incomplete data.  In _Advances in Neural Information Processing Systems_ , volume 32. Curran Associates, Inc.
  * Ng et al. (2019) Nathan Ng, Kyra Yee, Alexei Baevski, Myle Ott, Michael Auli, and Sergey Edunov. 2019\.  Facebook FAIR’s WMT19 news translation task submission.  In _Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1)_ , pages 314–319, Florence, Italy. Association for Computational Linguistics.
  * Ouyang et al. (2022) Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022\.  Training language models to follow instructions with human feedback.  _Advances in Neural Information Processing Systems_ , 35:27730–27744.
  * Pham-Hong and Chokshi (2020) Bao-Tran Pham-Hong and Setu Chokshi. 2020.  PGSG at SemEval-2020 task 12: BERT-LSTM with tweets’ pretrained model and noisy student training method.  In _Proceedings of the Fourteenth Workshop on Semantic Evaluation_ , pages 2111–2116, Barcelona (online). International Committee for Computational Linguistics.
  * Rasmus et al. (2015) Antti Rasmus, Harri Valpola, Mikko Honkala, Mathias Berglund, and Tapani Raiko. 2015\.  Semi-supervised learning with ladder networks.  In _Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2_ , NIPS’15, page 3546–3554, Cambridge, MA, USA. MIT Press.
  * Richardson et al. (2022) Caitlin Richardson, Sandeep Shah, and Xiaohong Yuan. 2022.  Semi-supervised machine learning for analyzing covid-19 related twitter data for asian hate speech.  In _2022 21st IEEE International Conference on Machine Learning and Applications (ICMLA)_ , pages 1643–1648. IEEE.
  * Rizos et al. (2019) Georgios Rizos, Konstantin Hemker, and Björn Schuller. 2019.  Augment to prevent: Short-text data augmentation in deep learning for hate-speech classification.  In _Proceedings of the 28th ACM International Conference on Information and Knowledge Management_ , CIKM ’19, page 991–1000, New York, NY, USA. Association for Computing Machinery.
  * Santos et al. (2022) Raquel Bento Santos, Bernardo Cunha Matos, Paula Carvalho, Fernando Batista, and Ricardo Ribeiro. 2022.  Semi-Supervised Annotation of Portuguese Hate Speech Across Social Media Domains.  In _11th Symposium on Languages, Applications and Technologies (SLATE 2022)_ , volume 104 of _Open Access Series in Informatics (OASIcs)_ , pages 11:1–11:14, Dagstuhl, Germany. Schloss Dagstuhl – Leibniz-Zentrum für Informatik.
  * Schiappa et al. (2022) Madeline C. Schiappa, Yogesh S. Rawat, and Mubarak Shah. 2022.  Self-supervised learning for videos: A survey.  _ACM Comput. Surv._ Just Accepted.
  * Waseem (2016) Zeerak Waseem. 2016.  Are you a racist or am I seeing things? annotator influence on hate speech detection on Twitter.  In _Proceedings of the First Workshop on NLP and Computational Social Science_ , pages 138–142, Austin, Texas. Association for Computational Linguistics.
  * Waseem and Hovy (2016) Zeerak Waseem and Dirk Hovy. 2016.  Hateful symbols or hateful people? predictive features for hate speech detection on Twitter.  In _Proceedings of the NAACL Student Research Workshop_ , pages 88–93, San Diego, California. Association for Computational Linguistics.
  * Xie et al. (2020a) Qizhe Xie, Zihang Dai, Eduard Hovy, Minh-Thang Luong, and Quoc V. Le. 2020a.  Unsupervised data augmentation for consistency training.  In _Proceedings of the 34th International Conference on Neural Information Processing Systems_ , NIPS’20, Red Hook, NY, USA. Curran Associates Inc.
  * Xie et al. (2020b) Qizhe Xie, Minh-Thang Luong, Eduard Hovy, and Quoc V Le. 2020b.  Self-training with noisy student improves imagenet classification.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 10687–10698.
  * Zampieri et al. (2019) Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019.  Predicting the type and target of offensive posts in social media.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 1415–1420, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Zhang et al. (2018) Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. 2018.  mixup: Beyond empirical risk minimization.  In _International Conference on Learning Representations_.
  * Zhang et al. (2015) Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015.  Character-level convolutional networks for text classification.  In _Advances in Neural Information Processing Systems_ , volume 28. Curran Associates, Inc.


$F_{1}(\boldsymbol{\theta})/\bar{\mathrm{F}}=0.23,F_{2}(\boldsymbol{\theta})/\bar{\mathrm{F}}=2.31,F_{3}(\boldsymbol{\theta})/\bar{\mathrm{F}}=0.46$.
These centered inputs are finally transformed into bounded values as in Table
A1.

Table A1: Example: Effects of CDF Transformations | Transformed Responses
---|---
| Weibull CDF
---
$\texttt{CDF}(x)=1-e^{(-x^{2})}$
0.05 / 1.00 / 0.19
| Frechet CDF
---
$\texttt{CDF}(x)=e^{(-1/x)}$
0.01 / 0.65 / 0.11
| Gumbel CDF
---
$\texttt{CDF}(x)=e^{\left(-e^{(-(x-1))}\right)}$
0.12 / 0.76 / 0.18
| Exponential CDF
---
$\texttt{CDF}(x)=1-e^{(-x)}$
0.21 / 0.90 / 0.37
| Logistic CDF
---
$\texttt{CDF}(x)=\frac{1}{1+e^{(-(x-1))}}$
0.32 / 0.79 / 0.37
| Normal CDF
---
$\texttt{CDF}(x)=\frac{1}{2}\left[1+\operatorname{erf}\left(\frac{x-1}{\sqrt{2}}\right)\right]$
0.22 / 0.90 / 0.29

While all losses become bounded values in $[0,1]$, the maximum local loss
(i.e., $F_{2}(\boldsymbol{\theta})=0.10$) is transformed into different values
by each CDF (see underlined figures in the ‘Transformed Responses’ column of
Table A1). When using the Weibull CDF, the maximum local loss is translated
into $1.00$, which means that there may be no value greater than 0.10 (i.e.,
$0.10$ is the largest one in $100\%$ probability) given current local losses.
Meanwhile, when using the Frechet CDF, the maximum local loss is translated
into $0.65$, which means that there still is a 35% chance that some other
local losses are greater than $0.10$ when provided with other losses similar
to 0.01 and 0.02. This implies that each CDF treats a maximum value
differently. When a transformation is easily inclined to the maximum value,
thereby returning 1 (i.e., maximum of CDF), it may yield a degenerate
decision, e.g., $\boldsymbol{p}\approx[0,1,0]^{\top}$.

Fortunately, most of the listed CDFs are designed for modeling maximum values.
For example, the three distributions, Gumbel, Frechet, and Weibull, are
grouped as the Extreme Value Distribution (EVD) (Kotz & Nadarajah, 2000). As
its name suggests, it models the behavior of extreme events, and it is well
known that any density modeling a minimum or a maximum of independent and
identically distributed (IID) samples follows the shape of one of these three
distributions (by the Extreme Value Theorem (Rusnock & Kerr-Lawson, 2005)). In
other words, EVDs can reasonably measure how a certain value is close to a
maximum. Thus, they can estimate whether a certain value is relatively large
or small. Otherwise, the Exponential distribution is a special case of Weibull
distribution, and the logistic distribution is also related to the Gumbel
distribution. Last but not least, although it is not a family of EVD, the
Normal distribution is also considered due to the central limit theorem, since
the local loss is the sum of errors from IID local samples. We expect the CDF
transformation can appropriately measure a relative magnitude of local losses,
and it should be helpful for decision making.

##### Effects of Response Transformation

We also illustrated that the response should be bounded (i.e., Lipschitz
continuous) in section 4.2.1, to have non-vacuous regret upper bound. To
acquire bounded response, we compare the cumulative values of a global
objective in (1), i.e.,
$\sum_{t=1}^{T}\sum_{i=1}^{K}p^{(t)}_{i}F_{i}(\boldsymbol{\theta}^{(t)})$ for
the cross-silo setting, and $\sum_{t=1}^{T}\sum_{i\in
S^{(t)}}p^{(t)}_{i}F_{i}(\boldsymbol{\theta}^{(t)})$ for the cross-device
setting.

Figure A1: Cumulative values of a global objective according to different CDFs
(smaller is better). (Left) Berka dataset (cross-silo setting; $K=7,T=100$).
(Right) Reddit dataset (cross-device setting; $K=817,T=300,C=0.00612$)

In the cross-silo experiment with the Berka dataset, the Normal CDF shows the
smallest cumulative values, while in the cross-device experiment with the
Reddit dataset, the Weibull CDF yields the smallest value. For the Berka
dataset, the Normal CDF yields an average performance (AUROC) of 79.37 with
the worst performance of 43.75, but the Weibull CDF shows an average
performance of 73.02 with the worst performance of 25.00. The same tendency is
also observed in the Reddit dataset. The Normal CDF presents an average
performance (Acc. 1) of 14.05 and the worst performance of 4.26, while the
Weibull CDF shows an average of 12.62 with the worst of 3.35. From these
observations, we can conclude that an appropriate choice of CDF is necessary
for better sequential decision making, and suitable transformation helps
minimize a global objective of FL. Note also that these behaviors are also
directly related to the global convergence of the algorithm w.r.t.
$\boldsymbol{\theta}$.

###  B.2 Choice of a Response Range

In regard to determining the range of a response vector, i.e.,
$[C_{1},C_{2}]$, we can refer to the Lipschitz continuity in Lemma 4.1 and
Lemma A.2. For the cross-silo setting, we can set arbitrary constants so that
$L_{\infty}=\mathcal{O}(1)$ according to Lemma 4.1. Thus, for all experiments
of AAggFF-S, we set $C_{1}=0,C_{2}=\frac{1}{K}$.

For the cross-device setting, the Lipschitz constant is changed into
$\breve{L}_{\infty}$, since it is influenced by the client sampling
probability ($C$). In detail, $C$ is located in the denominator of the
Lipschitz constant, $\breve{L}_{\infty}$, which inflates the resulting
gradient value as $C$ is a constant close to 0. (e.g., when 10 among 100,000
clients are participating in each round, ${1}/{C}=10^{4}$) This is problematic
and even causes an overflow problem empirically in updating a new decision.
Thus, we propose a simple remedy — setting $C_{1}$ and $C_{2}$ to be a
multiple of $C$, so that the $C$ in the denominator is to be canceled out,
according to Lemma A.2. For instance, when $C_{1}=0,C_{2}=C$, resulting
Lipschitz constant simply becomes
$\|\breve{\boldsymbol{g}}^{(t)}\|_{\infty}\leq\breve{L}_{\infty}=\frac{C}{1+0}+\frac{2(C-0)}{C(1+0)}=C+2\approx
2$, which is a constant far smaller than $T$ and $K$, typically assumed in the
practical cross-device FL setting. Therefore, for all experiments of AAggFF-D,
we set $C_{1}=0,C_{2}=C$.

##  Appendix C Experimental Details

Table A2: Statistics of Federated Benchmarks Dataset | Clients | Samples | Avg. | Std.
---|---|---|---|---
Berka | 7 | 621 | 88.71 | 24.78
MQP | 11 | 3,048 | 277.09 | 63.25
ISIC | 6 | 21,310 | 3551.67 | 3976.16
CelebA | 9,343 | 200,288 | 21.44 | 7.63
Reddit | 817 | 97,961 | 119.90 | 229.85
SpeechCommands | 2,005 | 84,700 | 42.24 | 36.69

###  C.1 Datasets

For the main experiments in Table 2 and Table 3, we used 3 datasets for the
cross-silo setting (Berka (Berka, 1999), MQP (McCreery et al., 2020), and ISIC
(Codella et al., 2018; Ogier du Terrail et al., 2022)), and 3 other datasets
for the cross-device setting (CelebA, Reddit (Caldas et al., 2019), and
SpeechCommands (Warden, 2018)). In this section, we describe details of each
dataset about its task, metrics, and client partitioning. The statistics of
all federated benchmarks are summarized in Table A2. Note that Avg. and Std.
in the table refer to the average and a standard deviation of a sample size of
each client in the FL system

First, we present details of the federated benchmark for the cross-silo
setting.

  1. $\bullet$

Berka is a tabular dataset containing bank transaction records collected from
Czech bank (Berka, 1999). Berka accompanies the loan default prediction task
(i.e., binary classification) of each bank’s customers. It is fully anonymized
and is originally composed of 8 relational tables: accounts, clients,
disposition, loans, permanent orders, transactions, demographics, and credit
cards. We merged all 8 tables into one dataset by joining the primary keys of
each table, and finally have 15 input features. From the demographics table,
we obtain information on the region: Prague, Central Bohemia, South Bohemia,
West Bohemia, North Bohemia, East Bohemia, South Moravia, and North Moravia.
We split each client according to the region and excluded all samples of North
Bohemia since it has only one record of loan default, thus we finally have 7
clients (i.e., banks). Finally, we used the area under the receiver operating
characteristic (ROC) curve for the evaluation metric.

  2. $\bullet$

MQP is a clinical question pair dataset crawled from medical question
answering dataset (Ben Abacha & Demner-Fushman, 2019), and labeled by 11
doctors (McCreery et al., 2020). All paired sentences are labeled as either
similar or dissimilar, thereby it is suitable for the binary classification
task. As a pre-processing, we merge two paired sentences into one sentence by
adding special tokens: [SEP], [PAD], and [UNK]. We set the maximum token
length to 200, thus merged sentences less than 200 are filled with [PAD]
tokens, and otherwise are truncated. Then, merged sentences are tokenized
using pre-trained DistilBERT tokenizer (Sanh et al., 2019). We regard each
doctor as a separate client and thus have 11 clients. Finally, we used the
area under the ROC curve for the evaluation metric.

  3. $\bullet$

ISIC is a dermoscopic image dataset for a skin cancer classification,
collected from 4 hospitals. (Codella et al., 2018; Ogier du Terrail et al.,
2022) The task contains 8 distinct melanoma classes, thus designed for the
multi-class classification task. Following (Ogier du Terrail et al., 2022), as
one hospital has three different imaging apparatus, its samples are further
divided into 3 clients, thus we finally have 6 clients in total. Finally, we
used top-5 accuracy for the evaluation metric.

Next, we illustrate details of the federated benchmark for the cross-device
setting.

  1. $\bullet$

CelebA is a vision dataset containing the facial images of celebrities (Liu et
al., 2015). It is curated for federated setting in LEAF benchmark (Caldas et
al., 2019), and is targeted for the binary classification task (presence of
smile). We follow the processing of (Caldas et al., 2019), thereby each client
corresponds to each celebrity, having 9,343 total clients in the FL system.
Finally, we used top-1 accuracy for the evaluation metric following (Caldas et
al., 2019).

  2. $\bullet$

Reddit is a text dataset containing the comments of community users of Reddit
in December 2017, and a part of LEAF benchmark (Caldas et al., 2019).
Following (Caldas et al., 2019), we build a dictionary of vocabularies of size
10,000 from tokenized sentences and set the maximum sequence length to 10. The
main task is tailored for language modeling, i.e., next token prediction,
given word embeddings in each sentence of clients. Each client corresponds to
one of the community users, thus 817 clients are presented in total. Finally,
we used the top-1 accuracy for the evaluation metric following (Caldas et al.,
2019).

  3. $\bullet$

SpeechCommands is designed for a short-length speech recognition task that
includes one second 35 short-length words, such as “Up”, “Down”, “Left”, and
“Right” (Warden, 2018). It is accordingly a multi-class classification task
for 35 different classes. While it is collected from 2,618 speakers, we rule
out all samples of speakers having too few samples when splitting the dataset
into training and test sets. (e.g., exclude speakers whose total sample counts
are less than 3) As a result, we have 2,005 clients, and each client has
16,000-length time-domain waveform samples. Finally, we used top-5 accuracy
for the evaluation metric.

###  C.2 Models

For each dataset, we used task-specific model architectures which are already
used in previous works, or widely used in reality, to simulate the practical
FL scenario as much as possible. For the experiment of the cross-silo setting,
we used the following models.

  1. $\bullet$

Logistic Regression is used for the Berka dataset. We used a simple logistic
regression model with a bias term, and the output (i.e., logit vector) is
transformed into predicted class probabilities by the softmax function.

  2. $\bullet$

DistilBERT (Sanh et al., 2019) is used for the MQP dataset. We used a pre-
trained DistilBERT model, from BookCorpus and English Wikipedia (Sanh et al.,
2019). We also used the corresponding DistilBERT tokenizer for the pre-
processing of raw clinical sentences. For a fine-tuning of the pre-trained
DistilBERT model, we attach a classifier head next to the last layer of the
DistilBERT’s encoder, which outputs an embedding of $768$ dimension. The
classifier is in detail processing the embedding as follows: ($768$-ReLU-
Dropout-$2$), where each figure is an output dimension of a fully connected
layer with a bias term, ReLU is a rectified linear unit activation layer, and
Dropout (Srivastava et al., 2014) is a dropout layer having probability of
$0.1$. In the experiment, we trained all layers including pre-trained weights.

  3. $\bullet$

EfficientNet (Tan & Le, 2019) is used for the ISIC dataset. We also used the
pre-trained EfficientNet-B0 model from ImageNet benchmark dataset (Deng et
al., 2009). For fine-tuning, we attach a classifier head after the convolution
layers of EfficientNet. The classifier is composed of the following
components: (AdaptiveAvgPool($7,7$)-Dropout-$8$), where
AdaptiveAvgPool($cdot,cdot$) is a 2D adaptive average pooling layer outputs a
feature map of size $7\times 7$ (which are flattened thereafter), Dropout is a
dropout layer with a probability of 0.1, and the last linear layer outputs an
8-dimensional vector, which is the total number of classes.

Next, for the cross-device setting, we used the following models.

  1. $\bullet$

ConvNet model used in LEAF benchmark (Caldas et al., 2019) is used for the
CelebA dataset. It is composed of four convolution layers, of which components
are: 2D convolution layer without bias term with 32 filters of size $3\times
3$ (stride=1, padding=1), group normalization layer (the number of groups is
decreased from 32 by a factor of 2: 32, 16, 8, 4), 2D max pooling layer with
$2\times 2$ filters, and a ReLU nonlinear activation layer. Plus, a classifier
comes after the consecutive convolution layers, which are composed of:
(AdaptiveAvgPool($5,5$)-$1$), which are a 2D adaptive average pooling layer
that outputs a feature map of size $5\times 5$ (which are flattened
thereafter), and a linear layer with a bias term outputs a scalar value since
it is a binary classification task.

  2. $\bullet$

StackedLSTM model used in LEAF benchmark (Caldas et al., 2019) is used for the
Reddit dataset. It is composed of an embedding layer of which the number of
embeddings is 200, and outputs an embedding vector of 256 dimensions. It is
processed by consecutive 2 LSTM (Hochreiter & Schmidhuber, 1997) layers with
the hidden size of 256, and enters the last linear layer with a bias term,
which outputs a logit vector of 10,000 dimensions, which corresponds to the
vocabulary size.

  3. $\bullet$

M5 (Dai et al., 2017) model is used for the SpeechCommands dataset. It is
composed of four 1D convolution layers followed by a 1D batch normalization
layer, ReLU nonlinear activation, and a 1D max pooling layer with a filter of
size 4. All convolution layers EXCEPT the input layer have a filter of size
$3$, and the numbers of filters are 64, 128, and 256 (all with stride=1 and
padding=1). The input convolution layer has 64 filters with a filter of size
$80$, and stride of $4$. Lastly, one linear layer outputs a logit vector of 35
dimensions.

###  C.3 Hyperparameters

Before the main experiment, we first tuned the hyperparameter of all baseline
fair FL algorithms from a separate random seed. The hyperparameter of each
fair algorithm is listed as follows.

  1. $\bullet$

AFL (Mohri et al., 2019) — a step size of a mixing coefficient $\in${0.01,
0.1, 1.0}

  2. $\bullet$

q-FedAvg (Li et al., 2019) — a magnitude of fairness, $\in${0.1, 1.0, 5.0}

  3. $\bullet$

TERM (Li et al., 2020a) — a tilting constant, $\lambda$, $\in${0.1, 1.0, 10.0}

  4. $\bullet$

FedMGDA (Hu et al., 2022) — a deviation from static mixing coefficient
$\in${0.1, 0.5, 1.0}

  5. $\bullet$

PropFair (Zhang et al., 2022) — a baseline constant $\in${2, 3, 5}

Each candidate value is selected according to the original paper, and we fix
the number of local epochs, $E=1$ (following the set up in (Li et al., 2019)),
along with the number of local batch size $B=20$ in all experiments. For each
dataset, a weight decay (L2 penalty) factor ($\psi$), a local learning rate
($\zeta$), and variables related to a learning rate scheduling (i.e., learning
rate decay factor ($\phi$), and a decay step ($s$)) are tuned first with
FedAvg (McMahan et al., 2017) as follows.

  1. $\bullet$

Berka: $\psi=10^{-3},\zeta=10^{0},\phi=0.99,s=10$

  2. $\bullet$

MQP: $\psi=10^{-2},\zeta=10^{-\frac{5}{2}},\phi=0.99,s=15$

  3. $\bullet$

ISIC: $\psi=10^{-2},\zeta=10^{-4},\phi=0.95,s=5$

  4. $\bullet$

CelebA: $\psi=10^{-4},\zeta=10^{-1},\phi=0.96,s=300$

  5. $\bullet$

Reddit: $\psi=10^{-6},\zeta=10^{\frac{7}{8}},\phi=0.95,s=20$

  6. $\bullet$

SpeechCommands: $\psi=0,\zeta=10^{-1},\phi=0.999,s=10$

This is intended under the assumption that all fair FL algorithms should at
least be effective in the same setting of the FL algorithm with the static
aggregation scheme (i.e., FedAvg). Note that client-side optimization in all
experiments is done by the Stochastic Gradient Descent (SGD) optimizer.

###  C.4 Implementation Details

All code is implemented in PyTorch (Paszke et al., 2019), simulating a central
parameter server that orchestrates a whole FL procedure and operates AAggFF.
We further simulate $K$ participating clients having their own local samples,
and a communication scheme with the central server. All experiments are
conducted on a server with 2 Intel®Xeon®Gold 6226R CPUs (@ 2.90GHz) and 2
NVIDIA®Tesla®V100-PCIE-32GB GPUs.

##  Appendix D Pseudocode for AAggFF

###  D.1 Pseudocode for ClientUpdate

Algorithm 1 ClientUpdate

Input: number of local epochs $E$, local batch size $B$, local learning rate
$\zeta$, global model $\boldsymbol{\theta}$

Procedure:

Evaluate the received global model on training set according to eq. (94) to
yield $F_{i}\left(\boldsymbol{\theta}\right)$.

Set local model $\boldsymbol{\theta}^{(0)}\leftarrow\boldsymbol{\theta}$

for $e=0$ to $E-1$ do

$\mathcal{B}_{e}$ $\leftarrow$ Split the client training dataset into batches
of size $B$.

for mini-batch $\Xi$ in $\mathcal{B}_{e}$ do

Update the model
$\boldsymbol{\theta}^{(e)}\leftarrow\boldsymbol{\theta}^{(e)}-\frac{\zeta}{B}\sum\limits_{j=1}^{B}\nabla_{\boldsymbol{\theta}}\mathcal{L}\left(\Xi;\boldsymbol{\theta}^{(e)}\right)$.

end for

Set $\boldsymbol{\theta}^{(e+1)}\leftarrow\boldsymbol{\theta}^{(e)}$

end for

Return: $F_{i}\left(\boldsymbol{\theta}\right)$,
$\boldsymbol{\theta}-\boldsymbol{\theta}^{(t+1)}_{i}$.

For generating a response vector, each client is requested to evaluate the
received global model on its training samples, $\\{\xi_{k}\\}_{k=1}^{n_{i}}$,
before the local update. As a result of the evaluation, the local loss of
client $i$ at round $t$, i.e., $F_{i}\left(\boldsymbol{\theta}^{(t)}\right)$,
is calculated as follows.

$\displaystyle
F_{i}\left(\boldsymbol{\theta}^{(t)}\right)=\frac{1}{n_{i}}\sum_{k=1}^{n_{i}}\mathcal{L}\left(\xi_{k};\boldsymbol{\theta}^{(t)}\right),$
(94)

where $n_{i}$ is the total sample size of client $i$, $l$ is a loss function
specific to the task, and $h_{\boldsymbol{\theta}}$ is a hypothesis realized
by the parameter $\boldsymbol{\theta}$.

###  D.2 Pseudocode for AAggFF-S

Algorithm 2 AAggFF-S

Input: number of clients $K$, total rounds $T$, transformation $\rho$, minimum
and maximum of a response $[C_{1},C_{2}]$.

Initialize: mixing coefficients
$\boldsymbol{p}^{(1)}=\frac{1}{K}\boldsymbol{1}_{K}$, global model
$\boldsymbol{\theta}^{(1)}\in\mathbb{R}^{d}$

Procedure:

for $t=0$ to $T-1$ do

for each client $i=1,...,K$ in parallel do

$F_{i}\left(\boldsymbol{\theta}^{(t)}\right),\boldsymbol{\theta}^{(t)}-\boldsymbol{\theta}^{(t+1)}_{i}\leftarrow\texttt{ClientUpdate}\left(\boldsymbol{\theta}^{(t)}\right)$

end for

Return $\boldsymbol{r}^{(t)}$ according to eq. (8) and $C_{1},C_{2}$.

Suffer decision loss $\ell^{(t)}(\boldsymbol{p}^{(t)})$ according to eq. (4).

Return a gradient
$\boldsymbol{g}^{(t)}=\nabla\ell^{(t)}\left(\boldsymbol{p}^{(t)}\right)$.

Return a mixing coefficient $\boldsymbol{p}^{(t+1)}$ according to eq. (7).

Update a global model
$\boldsymbol{\theta}^{(t+1)}=\boldsymbol{\theta}^{(t)}-\sum\limits_{i=1}^{K}p^{(t+1)}_{i}\left(\boldsymbol{\theta}^{(t)}-\boldsymbol{\theta}^{(t+1)}_{i}\right)$.

end for

Return: $\boldsymbol{\theta}^{(T)}$

###  D.3 Pseudocode for AAggFF-D

Algorithm 3 AAggFF-D

Input: number of clients $K$, client sampling ratio $C\in(0,1)$, total rounds
$T$, transformation $\rho$, range of a response $[C_{1},C_{2}]$.

Initialize: mixing coefficients
$\boldsymbol{p}^{(0)}=\frac{1}{K}\boldsymbol{1}_{K}$, global model
$\boldsymbol{\theta}^{(0)}\in\mathbb{R}^{d}$

Procedure:

for $t=0$ to $T-1$ do

$S^{(t)}\leftarrow\text{Wait until }\min(1,\lfloor C\cdot K\rfloor)\text{
clients are active in a network}$.

for each client $i\in S^{(t)}$ in parallel do

$F_{i}\left(\boldsymbol{\theta}^{(t)}\right),\boldsymbol{\theta}^{(t)}-\boldsymbol{\theta}^{(t+1)}_{i}\leftarrow\texttt{ClientUpdate}\left(\boldsymbol{\theta}^{(t)}\right)$

end for

Return $\breve{\boldsymbol{r}}^{(t)}$ according to eq. (8), eq. (10), and
$C_{1},C_{2}$.

Suffer decision loss $\ell^{(t)}(\boldsymbol{p}^{(t)})$ according to eq. (4).

Return a gradient estimate $\breve{\boldsymbol{g}}^{(t)}$ according to eq.
(11).

Return mixing coefficients $\boldsymbol{p}^{(t+1)}$ according to eq. (12)).

Acquire selected coefficients $\tilde{\boldsymbol{p}}^{(t+1)}$ according to
eq. (95).

Update a global model
$\boldsymbol{\theta}^{(t+1)}=\boldsymbol{\theta}^{(t)}-\sum\limits_{i\in
S^{(t)}}\tilde{p}^{(t+1)}_{i}\left(\boldsymbol{\theta}^{(t)}-\boldsymbol{\theta}^{(t+1)}_{i}\right)$.

end for

Return: $\boldsymbol{\theta}^{(T)}$

After updating whole entries of a decision variable, the server only exploits
mixing coefficients of which indices correspond to selected clients. Denoting
as $\tilde{\boldsymbol{p}}^{(t+1)}\in\Delta_{|S^{(t)}|-1}$, each selected
entry is normalized as follows.

$\displaystyle\tilde{p}_{i}^{(t+1)}=\frac{p_{i}^{(t+1)}}{\sum_{j\in
S^{(t)}}p_{j}^{(t+1)}},i\in S^{(t)}$ (95)

This ensures that selected coefficients also satisfy the condition for being a
probability vector (i.e., sum up to 1).


# Angular momentum conservation in counter-propagating vectorially structured
light

Hang Li  School of Optical and Electronic Information, Huazhong University of
Science and Technology, Wuhan, China  Valeria Rodriguez-Fajardo  School of
Physics, University of the Witwatersrand, Johannesburg, South Africa  Peifeng
Chen  School of Optical and Electronic Information, Huazhong University of
Science and Technology, Wuhan, China  Andrew Forbes  School of Physics,
University of the Witwatersrand, Johannesburg, South Africa

###### Abstract

It is well-known that electric spin angular momentum and electric orbital
angular momentum are conserved under paraxial propagation of travelling waves
in free-space. Here we study the electric and magnetic angular momentum in
counter-propagating waves and show both theoretically and experimentally that
neither component alone is conserved except in special cases. We attribute
this non-conservation to spin-spin and orbit-orbit coupling between the
electric and magnetic fields. This work generalises previous findings based on
travelling waves, explains the apparent spin-orbit coupling in counter-
propagating paraxial light, and broadens our understanding of angular momentum
conservation in arbitrary structured light waves.

The angular momentum (AM) of light has contributions from both the spin
angular momentum (SAM) and the orbital angular momentum (OAM) components. The
former is dependent on the polarization of the light, with a purely left- and
right-handed polarized beam carrying SAM of $\hbar$ and $-\hbar$ per photon,
respectively. The OAM is related to the spatial structure of the beam,
specifically, a vortex phase of topological charge $\ell$ results in OAM of
$\ell\hbar$ per photon Allen et al. (1992); O’Neil et al. (2002); Yao and
Padgett (2011). Over the past few years, there has been enormous interest in
the interactions between SAM and OAM due to its fundamental importance
Marrucci et al. (2011); Jisha and Alberucci (2017); Zhu et al. (2014) and
emerging applications in nanophotonics Bliokh et al. (2010); Cardano and
Marrucci (2015); Padgett and Bowman (2011). Spin-orbit (SO) interaction often
occurs in non-paraxial beams including evanescent waves Petersen et al.
(2014); O’connor et al. (2014), scattering Haefner et al. (2009); Brasselet et
al. (2009); Nechayev et al. (2019), and tight focusing processes Han et al.
(2018); Li et al. (2018). Recently, SO interaction was reported in counter-
propagating paraxial beams in free-space Otte et al. (2018), but could not
explain why the total electric AM was not conserved. In this paper, we propose
that the non-conservation of electric angular momentum, including SAM, OAM and
total AM, is a general phenomenon in counter-propagating beams. We attribute
this phenomenon to spin-spin and orbit-orbit interaction between the electric
and magnetic fields. Previously it has been shown that the electric and
magnetic fields contribute equally in travelling paraxial beams Aiello and
Berry (2015), but now we make clear that while this is true, it is only so for
travelling waves; counter-propagating beams do not obey this rule. We support
our hypothesis by a theoretical framework that is validated by experiment, and
offer a new perspective on spin-spin and orbit-orbit coupling in propagating
light.

Theory. To begin, we consider a monochromatic electromagnetic field in free-
space; by adopting the so-called electric-magnetic democracy formalism Berry
(2009), the SAM density $\mathbf{s}$ and OAM density $\mathbf{l}$ can be
written as Aiello et al. (2015)

$\displaystyle\mathbf{s}$
$\displaystyle=\frac{1}{4\omega}\mathrm{Im}[\epsilon_{0}\mathbf{E}^{*}\times\mathbf{E}+\mu_{0}\mathbf{H}^{*}\times\mathbf{H}],$
(1) $\displaystyle\mathbf{l}$ $\displaystyle=\mathbf{r}\times\mathbf{p_{o}},$

for a radius vector of $\mathbf{r}$, with the orbital linear momentum density
$\mathbf{p_{o}}$ defined as

$\mathbf{p_{o}}=\frac{1}{4\omega}\mathrm{Im}[\epsilon_{0}\mathbf{E}^{*}\cdot(\nabla)\mathbf{E}+\mu_{0}\mathbf{H}^{*}\cdot(\nabla)\mathbf{H}],$
(2)

and the spin linear momentum density as

$\mathbf{p_{s}}=\frac{1}{2}\nabla\times\mathbf{s},$ (3)

for an angular frequency of $\omega$. For compactness we have used the
notation $\mathbf{A}\cdot(\nabla)\mathbf{B}=A_{x}\nabla B_{x}+A_{y}\nabla
B_{y}+A_{z}\nabla B_{z}$, and all other terms have their usual meaning. The
total linear momentum is found from the sum of the orbital and spin
contributions, $\mathbf{p}=\mathbf{p_{o}}+\mathbf{p_{s}}$. Since the total
linear momentum is proportional to the time-averaged Poynting vector (energy
flow density), $\mathbf{p_{o}}$ and $\mathbf{p_{s}}$ can be regarded as the
orbital and spin parts of this energy flow density: orbital flow density (OFD)
and spin flow density (SFD), respectively Bekshaev et al. (2011); Bekshaev and
Soskin (2007). We see from Eqs. (1)-(3) that the SAM, OAM, OFD and SFD consist
of two separate terms: the first is only dependent on the electric field while
the second term is only dependent on the magnetic field. Therefore, in the
following, we will use superscripts $e$ and $m$ to distinguish between these
electric and magnetic contributions. Now we ask if there is coupling between
these terms in counter-propagating light.

In a travelling wave, the magnitudes of the electric and magnetic fields are
proportional, so the spin and orbital angular momentum contributions by the
electric and magnetic fields are equal Berry (2009). However, the situation
changes in counter-propagating beams. To illustrate this, we consider a
general counter-propagating beam in which the slow dependence of the
electromagnetic field on $z$ is neglected (we assume without loss of
generality that the beams are collimated). In cylindrical coordinates this can
be expressed in a general form as

$\displaystyle\mathbf{E}_{\perp}$
$\displaystyle=\mathbf{u}^{+}(r,\phi)\exp(ikz)+\mathbf{u}^{-}(r,\phi)\exp(-ikz),$
(4) $\displaystyle E_{z}$
$\displaystyle=\frac{i}{k}\nabla_{\perp}\cdot[\mathbf{u}^{+}\exp(ikz)-\mathbf{u}^{-}\exp(-ikz)],$

where $\nabla_{\perp}=\mathbf{e}_{x}\partial x+\mathbf{e}_{y}\partial y$,
$k=\omega/c$ is the wavenumber, and $c$ is the speed of light. The total
electric field is $\mathbf{E}=\mathbf{E}_{\perp}+E_{z}\mathbf{e}_{z}$, in
which $E_{z}$ is obtained by $\nabla\cdot\mathbf{E}=0$. The first term with
$\exp(ikz)$ is the part whose direction of propagation is parallel to the unit
vector $\mathbf{e}_{z}$, while the second term with $\exp(-ikz)$ is
antiparallel to $\mathbf{e}_{z}$. In this notation, $\mathbf{u}^{+}$ and
$\mathbf{u}^{-}$ are positive- and negative-propagating modes, respectively,
and can be any structured light field: Laguerre-Gaussian, Hermite-Gaussian,
cylindrical vector vortex beams, and so on. Since the wavenumber, $k$, is very
large, the longitudinal component of the electric field is much smaller than
transverse component. Small but not negligible, as we will show that the
longitudinal electromagnetic field plays a significant role in transverse SFD
in counter-propagating beams. Using the first-order approximation Bekshaev et
al. (2011), the magnetic field
$\mathbf{H}=\mathbf{H}_{\perp}+H_{z}\mathbf{e}_{z}$ is

$\displaystyle\mathbf{H}_{\perp}$
$\displaystyle=\frac{1}{c\mu_{0}}\mathbf{e}_{z}\times[\mathbf{u}^{+}\exp(ikz)-\mathbf{u}^{-}\exp(-ikz)],$
(5) $\displaystyle H_{z}$
$\displaystyle=\frac{i}{\omega\mu_{0}}\nabla_{\perp}\cdot(\mathbf{e}_{z}\times\mathbf{E}_{\perp}).$

It should be noted that there is an extra minus sign in the second term of the
transverse magnetic field. If the two parts of transverse electric field are
in phase, the two parts of transverse magnetic field will be exactly out of
phase, which means they contribute unequally to the momentum.

In paraxial beams, only the $z$-component of the SAM and OAM is of primary
relevance: $S_{z}$ and $L_{z}$, respectively. They are the sum of the electric
and magnetic contributions, $S_{z}=S_{z}^{e}+S_{z}^{m}$ and
$L_{z}=L_{z}^{e}+L_{z}^{m}$, and found from the SAM and OAM densities

$\displaystyle S_{z}^{e}$ $\displaystyle=\frac{\int\int
s_{z}^{e}r\mathrm{d}r\mathrm{d}\phi}{\tfrac{1}{\hbar\omega}\int\int
wr\mathrm{d}r\mathrm{d}\phi},S_{z}^{m}$ $\displaystyle=\frac{\int\int
s_{z}^{m}r\mathrm{d}r\mathrm{d}\phi}{\tfrac{1}{\hbar\omega}\int\int
wr\mathrm{d}r\mathrm{d}\phi},$ (6) $\displaystyle L_{z}^{e}$
$\displaystyle=\frac{\int\int
l_{z}^{e}r\mathrm{d}r\mathrm{d}\phi}{\tfrac{1}{\hbar\omega}\int\int
wr\mathrm{d}r\mathrm{d}\phi},L_{z}^{m}$ $\displaystyle=\frac{\int\int
l_{z}^{m}r\mathrm{d}r\mathrm{d}\phi}{\tfrac{1}{\hbar\omega}\int\int
wr\mathrm{d}r\mathrm{d}\phi},$

with the energy density $w$ defined as

$\displaystyle w$
$\displaystyle={\textstyle\frac{1}{4}}\left(\epsilon_{0}\mathbf{E}^{*}\cdot\mathbf{E}+\mu_{0}\mathbf{H}^{*}\cdot\mathbf{H}\right)$
(7)
$\displaystyle={\textstyle\frac{1}{2}}\epsilon_{0}\left(|\mathbf{u}^{+}|^{2}+|\mathbf{u}^{-}|^{2}\right).$

In Eq. (6), $s_{z}^{e}$, $s_{z}^{m}$, $l_{z}^{e}$ and $l_{z}^{m}$ are the
longitudinal electric SAM density, magnetic SAM density, electric OAM density
and magnetic OAM density, respectively. The numerators in Eq. (6) are the
global SAM/OAM while the denominators represent the number of photons.
Equation (7) shows that the energy density $w$ is independent of $z$, which
means the number of photons (or the energy) is conserved. Therefore, whether
the angular momenta per photon ($S_{z}^{e}$, $S_{z}^{m}$, $L_{z}^{e}$ and
$L_{z}^{m}$) are conserved is equivalent to whether the global angular momenta
are conserved.

After a little algebra (see Supplementary Information), one finds

$\displaystyle s_{z}^{e}\mathbf{e}_{z}$
$\displaystyle\propto\mathbf{E}_{\perp}^{*}\times\mathbf{E}_{\perp}=\mathbf{u}^{+*}\times\mathbf{u}^{+}+\mathbf{u}^{-*}\times\mathbf{u}^{-}$
(8)
$\displaystyle\quad+\mathbf{u}^{+*}\times\mathbf{u}^{-}\exp(-i2kz)+\mathbf{u}^{-*}\times\mathbf{u}^{+}\exp(i2kz)$
$\displaystyle S_{z}$
$\displaystyle=\frac{\hbar\mathrm{Im}\left\\{\int\int\mathbf{e}_{z}\cdot
r(\mathbf{u}^{+*}\times\mathbf{u}^{+}+\mathbf{u}^{-*}\times\mathbf{u}^{-})\mathrm{d}r\mathrm{d}\phi\right\\}}{\int\int(|\mathbf{u}^{+}|^{2}+|\mathbf{u}^{-}|^{2})r\mathrm{d}r\mathrm{d}\phi}$
$\displaystyle L_{z}$
$\displaystyle=\frac{\hbar\mathrm{Im}\left\\{\int\int(\mathbf{u}^{+*}\cdot\partial_{\phi}\mathbf{u}^{+}+\mathbf{u}^{-*}\cdot\partial_{\phi}\mathbf{u}^{-})r\mathrm{d}r\mathrm{d}\phi\right\\}}{\int\int(|\mathbf{u}^{+}|^{2}+|\mathbf{u}^{-}|^{2})r\mathrm{d}r\mathrm{d}\phi}.$

The first two terms of $\mathbf{E}_{\perp}^{*}\times\mathbf{E}_{\perp}$ give
the contribution of the positive- and negative-propagating parts of the
electric SAM, respectively. The last two terms are cross contributions, which
are not zero except in some special cases. We conclude that the electric SAM
is not conserved, which seems contrary to intuition. The total OAM and SAM are
both independent of $z$, so they are conserved. In the special case that
$\mathbf{u}^{-}=0$, the field degenerates into a one-way travelling beam; the
cross contribution vanishes and the electric SAM is again conserved,
consistent with intuition and prior reports Aiello and Berry (2015). Similar
conclusions can be also reached on the magnetic SAM, electric OAM and magnetic
OAM (see Supplementary Information).

Example. To make this practical for testing, consider a vector beam that is
composed of a superposition of two orthogonally polarized Laguerre-Gaussian
modes, $\text{LG}_{p,\ell}(r,\phi)$ (carrying OAM of $\ell\hbar$ per photon)
given by

$\displaystyle\mathbf{u}^{+}$
$\displaystyle={\textstyle\frac{1}{2}}[\exp(i\alpha)\text{LG}_{p_{1},\ell_{1}}\mathbf{e_{R}}+\exp(-i\alpha)\text{LG}_{p_{2},\ell_{2}}\mathbf{e_{L}}],$
(9) $\displaystyle\mathbf{u}^{-}$
$\displaystyle={\textstyle\frac{1}{2}}[\exp(i\beta)\text{LG}_{p_{1},\ell_{1}}\mathbf{e_{R}}+\exp(-i\beta)\text{LG}_{p_{2},\ell_{2}}\mathbf{e_{L}}],$

where $\mathbf{e_{R}}=(\mathbf{e_{x}}-i\mathbf{e_{y}})/\sqrt{2}$ and
$\mathbf{e_{L}}=(\mathbf{e_{x}}+i\mathbf{e_{y}})/\sqrt{2}$ represent the unit
vector of right- and left-circularly polarized states, respectively, and
$\alpha$ and $\beta$ define the phase relation between the two states. When
$\alpha=\beta$, the polarization states of the positive-propagating and
negative-propagating components are the same. When $|\alpha-\beta|=\pi/2$,
their states are orthogonal. In Eq. (9), the weight factors of the right- and
left-circularly polarized components are equal (for simplicity, but can easily
be generalized). The SAM and OAM per photon carried by the right-circularly
polarized component is $-\hbar$ and $\ell_{1}\hbar$, respectively, and $\hbar$
and $\ell_{2}\hbar$ for the left-circularly polarized component. Hence, the
total SAM and OAM per photon are 0 and $(\ell_{1}+\ell_{2})\hbar/2$,
respectively.

Figure 1:  (a) Conceptual sketch of the counter-propagating beam, with
intensity and phase distributions for $p_{1}=p_{2}=0$, $\alpha=0,\beta=\pi/2$
and $\ell_{1}=1$ and $\ell_{2}=3$ at $z=0$ shown. (b) Simulated (lines) and
experimental (points) results of global electric SAM, OAM and total AM per
photon. The insets show the intensity and phase distributions of the field at
$z=\lambda/8$ (trough) and $z=3\lambda/8$ (peak). Error bars are too small to
be seen.

To test the theory, we engineer our modes with no radial content
($p_{1}=p_{2}=0$) since global SAM and OAM are independent of radial modes, a
Rayleigh length of $\approx 6$ m so that the fields can be regarded as
collimated, and set $\alpha=0,\beta=\pi/2$ and $\ell_{1}=1$ and $\ell_{2}=3$,
shown schematically in Fig. 1(a), with full details of the experiment given in
the Supplementary Information. The electric SAM, OAM and AM per photon were
measured and compared to theory, shown in Fig. 1(b). As predicted, they are
all non-conserved qualities which increase and decrease in phase resulting in
no global spin-orbit coupling (in this case). The right- and left-circularly
polarized components are both standing waves with nodes at $z=\lambda/8$ and
$z=3\lambda/8$, respectively, where the superposition is a scalar mode (see
Supplementary Information). Moving along the axis from $z=\lambda/8$ to
$z=3\lambda/8$, a purely $\text{LG}_{0,1}$ mode with right-circular
polarization is gradually transformed into a left-circularly polarized
$\text{LG}_{0,3}$ mode, causing both OAM and SAM to increase, with the process
reversed from $z=3\lambda/8$ to $z=5\lambda/8$. The explanation for this non-
conservative requires consideration of the magnetic field component.

Figure 2: Spin-spin coupling and orbit-orbit coupling between the electric and
magnetic fields. We show the electric, magnetic and total (a) SAM and (b) OAM
per photon, theory (lines) and experimental electric data (points). In (c) and
(d) we show the corresponding longitudinal SAM and OAM densities (false colour
plots), respectively, overlaid with their transverse SFD and OFD (black
arrows) in the XY-plane. The first row of each shows the experimentally
reconstructed densities and flows for the electric component, while the
second, third and fourth show the simulations.

Spin-spin coupling. Using the earlier analysis, the electric SAM and magnetic
SAM per photon for our experimental example can be expressed as

$\displaystyle S_{z}^{e}$
$\displaystyle={\textstyle\frac{1}{2}}\hbar\sin(\alpha-\beta)\sin(2kz),$ (10)
$\displaystyle S_{z}^{m}$
$\displaystyle={\textstyle\frac{1}{2}}\hbar\sin(\beta-\alpha)\sin(2kz).$

Figure 2 (a) shows the calculated electric, magnetic and total SAM per photon
as a function of $z$, with experimental electric data in good agreement with
theory. We find that the electric and magnetic SAM vary while the total SAM is
always zero, which we interpret as spin-spin coupling between the electric and
magnetic fields. This coupling results from the polarization variation in our
vectorial beam: we have a standing wave in polarization structure rather than
intensity. According to Eq. (10), the coupling will vanish when the
polarization states of the positive-propagating part and the negative-
propagating part are the same, that is, when $|\alpha-\beta|=0$ or $\pi$,
because the polarization structure is then invariant along the $z$-axis. When
their polarized states are orthogonal, that is, $|\alpha-\beta|=\pi/2$, the
coupling will be the strongest. In other cases, the coupling still exists but
varies in strength.

The longitudinal SAM density and transverse SFD (superimposed arrows) are
shown in Fig. 2 (c), with the experimentally reconstructed electrical
component in the first row and the simulation in the second, third and fourth
rows. We see that the electric longitudinal SAM density and transverse SFD
from $z=0$ to $z=\lambda/2$ are the same as the magnetic part from
$z=\lambda/2$ to $z=0$. The opposite variability of electric and magnetic SAM
density causes invariant total SAM density, as shown in the last row of Fig. 2
(c). The transverse electric SFD was calculated from

$\mathbf{p}_{s\perp}^{e}={\textstyle\frac{1}{2}}\left[\nabla_{\perp}s_{z}^{e}\times\mathbf{e}_{z}+\mathbf{e}_{z}\times\partial_{z}\mathbf{s}_{\perp}^{e}\right].$
(11)

In a one-way travelling collimated beam, the second term is always zero, but
not so in our theoretical example. In our experimental measurement, we used a
constructed counter-propagating beam, resulting in the disparity in the
theoretical and experimental SFDs. The second term in Eq. (11) is dependent on
the longitudinal electric field while the longitudinal electric SAM density is
circularly symmetrical (because of the circular symmetry of LG modes), so the
first term only has an azimuthal component. Thus, the reconstructed
experimental electric SFD, first row of Fig. 2 (c), is either clockwise or
anticlockwise. In counter-propagating beams, the second term is no longer zero
because of the cross contribution from the positive-propagating and the
negative-propagating beams. This term makes the modes of counter-propagating
beams vary between left-circularly polarized ($\text{LG}_{0,3}$) and right-
circularly polarized ($\text{LG}_{0,1}$) modes. For example, from
$z=\lambda/8$ to $z=3\lambda/8$, the electric mode is transformed from an
$\text{LG}_{0,1}$ mode into a $\text{LG}_{0,3}$ mode, and vice versa for the
magnetic mode.

Orbit-orbit coupling. For our test example, the electric and magnetic OAM can
be expressed as

$\displaystyle L_{z}^{e}$
$\displaystyle={\textstyle\frac{1}{2}}\hbar\left[l_{1}\cos^{2}(kz+\tfrac{\alpha-\beta}{2})+l_{2}\cos^{2}(kz+\tfrac{\beta-\alpha}{2})\right],$
(12) $\displaystyle L_{z}^{m}$
$\displaystyle={\textstyle\frac{1}{2}}\hbar\left[l_{1}\sin^{2}(kz+\tfrac{\alpha-\beta}{2})+l_{2}\sin^{2}(kz+\tfrac{\beta-\alpha}{2})\right].$

In Fig. 2 (b) we show simulations for the electric, magnetic and total OAM per
photon as function of $z$, with the measured electric OAM shown as data
points, in excellent agreement with theory. This confirms orbit-orbit coupling
between the electric and magnetic fields. This coupling is not guaranteed, and
instead is dependent on both the spin states and the (OAM) topological
charges, $\ell_{1}$ and $\ell_{2}$. There are three cases in which the
coupling will vanish: (1) when $\ell_{1}=\ell_{2}=0$, neither the right- nor
the left-circularly polarized components of the beam have a vortex phase,
leading to zero electric OAM and zero magnetic OAM, so no possibility of
orbit-orbit coupling; (2) when $\alpha=\beta$ and $\ell_{1}=-\ell_{2}$, the
polarization is structurally invariant along the $z$-axis and the OAM
contributed by the two component cancel: both the electric and magnetic OAM
will be zero; (3) when $|\alpha-\beta|=\pi/2$ and $\ell_{1}=\ell_{2}$, the
spin states of the counter-propagating components are orthogonal, and the
electric energy per unit length is conserved. Thus $\ell_{1}=\ell_{2}$ results
in a balance between the electric OAM contributed by the two components - the
electric and magnetic OAM are both conserved independently and the coupling
appears to vanish.

Finally, the longitudinal OAM density and transverse OFD (superimposed arrows)
are shown in Fig. 2 (d), with the experimentally reconstructed electrical
components (first row) in excellent agreement with theory (second row). We see
that the electric longitudinal OAM density and transverse OFD from $z=0$ to
$z=\lambda/2$ are the same as the magnetic part from $z=\lambda/2$ to $z=0$.
The opposite trend of electric and magnetic OAM density results in an
invariant total OAM density, as shown in the last row of Fig. 2 (d). The
transverse electric OFD,
$\mathbf{p}_{o\perp}^{e}=l_{z}^{e}\mathbf{e}_{\phi}/r$, has no radial
component and is independent of the longitudinal electric field. Further,
since the topological charges $\ell_{1}$ and $\ell_{2}$ are both positive, the
transverse OFD (magnetic or electric) is always anticlockwise, resulting in a
longitudinal OAM density that is always positive. If one topological charge is
positive and the other is negative, OFD reversal occurs.

Conclusion. We have demonstrated that generally held conservation rules appear
to be violated in counter-propagating vectorially structured light beams,
which we confirm by experiment. We reconcile this by considering terms that
are traditionally held to be zero or ‘the same’, and interpret the results as
spin-spin and orbit-orbit coupling. With this we are able to explain some
anomalies previously reported, while reducing to the well-known results for
travelling waves under special cases. We find that in general the total
longitudinal SAM and OAM are both conserved, while neither the electric part
nor magnetic part are conserved independently. This work generalises previous
findings based on travelling waves, explains the apparent spin-orbit coupling
in counter-propagating paraxial light, and broadens our understanding of
angular momentum conservation in arbitrary structured light waves.

## References

  * Allen et al. (1992) L. Allen, M. W. Beijersbergen, R. Spreeuw, and J. Woerdman, Phys. Rev. A 45, 8185 (1992).
  * O’Neil et al. (2002) A. T. O’Neil, I. MacVicar, L. Allen, and M. J. Padgett, Phys. Rev. Lett. 88, 053601 (2002), URL https://link.aps.org/doi/10.1103/PhysRevLett.88.053601.
  * Yao and Padgett (2011) A. M. Yao and M. J. Padgett, Adv. Opt. Photon. 3, 161 (2011).
  * Marrucci et al. (2011) L. Marrucci, E. Karimi, S. Slussarenko, B. Piccirillo, E. Santamato, E. Nagali, and F. Sciarrino, J. Opt. 13, 064001 (2011).
  * Jisha and Alberucci (2017) C. P. Jisha and A. Alberucci, Opt. Lett. 42, 419 (2017).
  * Zhu et al. (2014) J. Zhu, Y. Chen, Y. Zhang, X. Cai, and S. Yu, Opt. Lett. 39, 4435 (2014).
  * Bliokh et al. (2010) K. Y. Bliokh, M. A. Alonso, E. A. Ostrovskaya, and A. Aiello, Phys. Rev. A 82, 063825 (2010), URL https://link.aps.org/doi/10.1103/PhysRevA.82.063825.
  * Cardano and Marrucci (2015) F. Cardano and L. Marrucci, Nat. Photonics 9, 776 (2015).
  * Padgett and Bowman (2011) M. Padgett and R. Bowman, Nat. Photonics 5, 343 (2011).
  * Petersen et al. (2014) J. Petersen, J. Volz, and A. Rauschenbeutel, Science 346, 67 (2014).
  * O’connor et al. (2014) D. O’connor, P. Ginzburg, F. J. Rodríguez-Fortuño, G. A. Wurtz, and A. V. Zayats, Nat. Commun. 5, 5327 (2014).
  * Haefner et al. (2009) D. Haefner, S. Sukhov, and A. Dogariu, Phys. Rev. Lett. 102, 123903 (2009), URL https://link.aps.org/doi/10.1103/PhysRevLett.102.123903.
  * Brasselet et al. (2009) E. Brasselet, N. Murazawa, H. Misawa, and S. Juodkazis, Phys. Rev. Lett. 103, 103903 (2009), URL https://link.aps.org/doi/10.1103/PhysRevLett.103.103903.
  * Nechayev et al. (2019) S. Nechayev, J. S. Eismann, G. Leuchs, and P. Banzer, Phys. Rev. B 99, 075155 (2019), URL https://link.aps.org/doi/10.1103/PhysRevB.99.075155.
  * Han et al. (2018) L. Han, S. Liu, P. Li, Y. Zhang, H. Cheng, and J. Zhao, Phys. Rev. A 97, 053802 (2018), URL https://link.aps.org/doi/10.1103/PhysRevA.97.053802.
  * Li et al. (2018) M. Li, Y. Cai, S. Yan, Y. Liang, P. Zhang, and B. Yao, Phys. Rev. A 97, 053842 (2018), URL https://link.aps.org/doi/10.1103/PhysRevA.97.053842.
  * Otte et al. (2018) E. Otte, C. Rosales-Guzmán, B. Ndagano, C. Denz, and A. Forbes, Light. Sci. & Appl. 7, 18009 (2018).
  * Aiello and Berry (2015) A. Aiello and M. Berry, J. Opt. 17, 062001 (2015).
  * Berry (2009) M. V. Berry, J. Opt. A: Pure Appl. Opt. 11, 094001 (2009).
  * Aiello et al. (2015) A. Aiello, P. Banzer, M. Neugebauer, and G. Leuchs, Nat. Photonics 9, 789 (2015).
  * Bekshaev et al. (2011) A. Bekshaev, K. Y. Bliokh, and M. Soskin, J. Opt. 13, 053001 (2011).
  * Bekshaev and Soskin (2007) A. Y. Bekshaev and M. Soskin, Optics Commun. 271, 332 (2007).