# Energy-Efficient Uncertainty-Aware Biomass
Composition Prediction at the Edge

Muhammad Zawish†, Paul Albert‡, Flavio Esposito§, Steven Davy¶, and Lizy
Abraham†

†Walton Institute, South East Technological University, Ireland
‡Center for Augmented Reasoning, Australian Institute of Machine Learning,
Australia
§Department of Computer Science, Saint Louis University, USA
¶Centre for Sustainable Digital Technologies at Technological University
Dublin, Ireland
<EMAIL_ADDRESS>
###### Abstract

Clover fixates nitrogen from the atmosphere to the ground, making grass-clover
mixtures highly desirable to reduce external nitrogen fertilization. Herbage
containing clover additionally promotes higher food intake, resulting in
higher milk production. Herbage probing however remains largely unused as it
requires a time-intensive manual laboratory analysis. Without this
information, farmers are unable to perform localized clover sowing or take
targeted fertilization decisions. Deep learning algorithms have been proposed
with the goal to estimate the dry biomass composition from images of the grass
directly in the fields. The energy-intensive nature of deep learning however
limits deployment to practical edge devices such as smartphones. This paper
proposes to fill this gap by applying filter pruning to reduce the energy
requirement of existing deep learning solutions. We report that although
pruned networks are accurate on controlled, high-quality images of the grass,
they struggle to generalize to real-world smartphone images that are blurry or
taken from challenging angles. We address this challenge by training filter-
pruned models using a variance attenuation loss so they can predict the
uncertainty of their predictions. When the uncertainty exceeds a threshold, we
re-infer using a more accurate unpruned model. This hybrid approach allows us
to reduce energy-consumption while retaining a high accuracy. We evaluate our
algorithm on two datasets: the GrassClover and the Irish clover using an
NVIDIA Jetson Nano edge device. We find that we reduce energy reduction with
respect to state-of-the-art solutions by 50% on average with only 4% accuracy
loss.

Figure 1: A typical workflow of deep learning based biomass estimation from
phone / camera images.

##  1 Introduction

Evaluating the dry biomass composition of herbage delivers insights that can
in turn be used by farmers to improve the quality of their grass. The usual
biomass composition process requires cutting grass on a delimited surface in
the field. The grass sample is then taken back to a laboratory where it is
dried in a low-temperature oven before being manually separated in its
components [18, 12]. This process is constrained by time and human labor which
hinders its practical application. Computer vision techniques have the
potential to reduce this laborious process by predicting dry biomass
composition from in situ images. Using image processing via deep learning is
also non-destructive for the soil, and a faster method to estimate the biomass
can facilitate the adoption of fine-grained herbage probing. Figure 1
illustrates such a process when images are collected with smartphones.

Computer vision systems have been previously developed on a variety of images
including high-resolution camera systems mounted on All Terrain Vehicles
(ATVs) [19] or on manually operated camera tripods [12]. Other more practical
approaches include using phones [3] or drone images [2]. These systems,
however, were not designed to address the limited resources of the targeted
edge devices, especially in terms of the energy consumption of large neural
networks that can limit the granularity of biomass estimation on battery-
powered devices.

Model compression, particularly structured (filter-based) pruning has proved
to be a viable solution for reducing model size and computations [4]. It
typically measures the saliency of filters in $conv$ layers and prunes them
according to a predefined threshold. The idea of mapping saliency with the
loss function makes these approaches dependent on training data, hence
limiting their practicability on specific computer vision tasks, such as image
classification. Recent work has shown that even without measuring filter
saliency, it is possible to achieve the same level of performance with random
pruning [16]. Furthermore, pruning approaches in the literature mostly focus
on pruning post-training to effectively cut down on inference costs, while
inevitably increasing training and re-training overhead. Recently, a new
method has emerged to prune models at initialization (before training) [9,
23], and show similar performances to the post-training pruning approaches. We
find energy-aware pruning at initialization attractive for biomass estimation
because although prior studies demonstrate the accuracy of deep learning
models to solve the task, these large models are intended for deployment on
energy-constrained edge devices like smartphones. Energy-aware pruning indeed
offers the potential for large energy savings, however, end-users require
reliable results to facilitate technology adoption. We aim in this paper to
answer the need for low energy consumption while retaining biomass prediction
accuracy.

When training state-of-the-art biomass composition estimation models [2] using
initialization-pruned models, we observe a significant decrease in accuracy
concentrated in a population of harder images. These images are clover-heavy
or were taken with hand-held smartphones, leading to lower image quality and
complicating the biomass estimation task. To preserve a high prediction
accuracy, we propose to identify these hard images at test-time so that they
can be re-predicted on using an unpruned model, inherently more robust to
these challenging examples. We achieve this by training the pruned models to
predict a normally distributed probability distribution over biomass values
rather than predicting a single scalar biomass value. The probability
distribution is characterized by a mean $\mu$ and a variance $\sigma$. We find
that $\sigma$ is positively correlated with the prediction error of the pruned
network on the hard images. At test-time and given the prediction of the
pruned network on a target image, if we detect that $\sigma$ exceeds a
threshold, i.e., the prediction is under-confident, we offset such predictions
using the unpruned network to increase accuracy. In the case where $\sigma$ is
small enough, $\mu$ is used as the energy-efficient and reliable predictions.
In effect, during the deployment of our algorithm on edge devices, we can
confidently save energy on simpler images and call for the more robust yet
energy-hungry unpruned model for the detected challenging images.

Consequently, the proposed approach conserves better accuracy than the pruned
model while yielding higher energy efficiency than the unpruned model.
In particular, in this paper, we present the following key contributions:

  * •

We perform stochastic filter pruning at initialization as in [23], iteratively
removing filters until the pruning quota is met, with high-energy $conv$
layers being more likely to be pruned. We consequently present an evaluation
of the detrimental effect of pruning rate on prediction accuracy.

  * •

We introduce a uncertainty-guided approach to accommodate performance loss in
pruned models. A high variance $\sigma$ of the biomass probability
distribution predicted by the pruned models informs the algorithm on the
subset of harder images that need to be more accurately predicted by the
unpruned model.

  * •

We compare with uncompressed deep learning baselines from the state-of-the-art
[2] using two network architectures [13, 17] on two grass biomass estimation
datasets [19, 12] where we show we can maintain a comparable accuracy while
decreasing energy usage.

  * •

We measure the real-world energy efficiency of our algorithm on a NVIDIA
Jetson Nano, a resource-constrained edge device where we realise a $40\%$ to
$60\%$ energy consumption decrease depending on the network and dataset. We
hope that this large decrease in energy consumption further motivates the
deployment of biomass estimation algorithms to edge devices without
compromising much on accuracy.

##  2 Related Work

This section presents into existing literature on dry biomass estimation from
images and model compression. While significant research exists on model
pruning and accurate biomass estimation using deep learning, we notice a
notable gap in integrating both for biomass estimation at the edge.

###  2.1 Grass and clover dry biomass estimation

We first detail datasets and research relevant to grass and clover dry biomass
estimation using computer vision. The GrassClover dataset [18, 19] proposes to
predict the dry biomass composition of grass from images taken from a high
resolution camera mounted on an ATV. $179$ images of grass-clover mixtures are
collected, oven-dried and manually labeled to obtain the ground-truth. The
baseline biomass estimation proposed by the authors is a two stage approach
where a large semantic segmentation model trained on synthetic data is used to
predict the species of each pixel in the grass canopy. Once the canopy
composition is estimated, a linear regression infers the dry biomass
composition. In a further iteration [20], the authors utilize a style transfer
generative network [22] to generate data augmentations on the collected images
to simulate different weather conditions. This effectively increases the
variety of the training dataset and enables more robust predictions. Because
the GrassClover dataset contains a set of partially annotated images,
researchers have proposed to use a one stage regression from image to dry
biomass using a mean-imputation approach using a VGG16 [17] neural network
[12]. Other solutions utilize the semantic segmentation into linear separation
baseline to approximately label unlabeled images so they can be introduced in
the training set [1], adapt the MixMatch semi-supervised classification
algorithm [6] to the regression problem [2] or to use them for self-supervised
pre-training [3]. In addition to the GrassClover dataset, the IrishClover [12]
dataset consists of $424$ training images captured using cameras mounted on
tripods and proposes a similar challenge of dry biomass estimation with added
difficulty of unbalanced species representation and the presence of handheld
phone images in the test set.

###  2.2 Lightweight neural networks for edge AI in agriculture

Generally, majority of the deep learning-based agricultural applications
discussed above are cloud-centric. This means that during the test-time, the
incoming data from mobile edge devices would be offloaded to a remote cloud
server to perform the prediction process. For example, authors in [5] proposed
a cloud-based AI solution for predicting biomass yield maps from apple
orchards using images captured through UAV. Cloud has remained a viable
solution for these tasks, since deep learning models are computationally-
intensive and consume significant energy and storage capacity. However, it
requires continuous and stable internet connection to approach cloud for deep
learning tasks, which is hard to realize in remote farming environments.
Recently, researchers have proposed to process the data directly on the edge
devices to achieve fast responses while ensuring data privacy. For example,
authors in [8] propose to directly classify the crop types using a deep
learning model deployed on an embedded board over a UAV. Similarly, Camargo et
al. [7] customized a lightweight version of ResNet model to fit on an NVIDIA
Jetson Xavier board for automatic weed mapping. However, these customized
lightweight approaches do not provide flexible and scalable models, since
these models are only developed for specific tasks. For these edge
applications, model compression approaches have recently emerged, particularly
pruning to achieve compression from coarse (low compression) to fine (high
compression) scales [10, 4, 9, 23]. In this paper, we implement filter pruning
as a type of model compression to realize lightweight models for biomass
estimation on different scales. We also show how to achieve energy efficiency
on a NVIDIA Jetson Nano edge device while preserving accuracy using the best
of both pruned and unpruned models.

Figure 2: Our proposed framework based on variance-guided energy efficient
deep learning for biomass estimation on edge.

##  3 Methodology

In this section, we first motivate the idea applying model compression to
biomass composition estimation algorithms. We then show how to circumvent the
performance loss of pruned models using our variance-guided approach.

###  3.1 Motivation

Biomass estimation using deep learning is generally carried out on a high-end
remote GPU / CPU cloud servers [1]. This is because SoTA deep learning models
are resource-intensive in nature, requiring significant amounts of storage and
power consumption. Using cloud-based inference implies that input images
captured through mobile phones need to be offloaded – via a wireless network –
to the cloud. This, however, is not considered reliable for making real-time
predictions, since continuous and stable internet connectivity is not
guaranteed in remote agricultural farms. Moreover, offloading raw images from
farms to cloud poses privacy concerns, which can be avoided by leveraging on-
device processing capabilities of mobile edge devices. Numerous research
efforts have been in place to accommodate deep models directly on mobile
phones. Model compression, particularly filter pruning, has been widely used
to enable on-device AI for several computer vision applications but at the
cost of slight reduction in accuracy [10]. Moreover, most of the compression
approaches are evaluated on image classification which raises questions about
their functionality for other tasks. This paper proposes to apply model
pruning at initialization on a vision-based regression task for biomass
estimation. In the subsequent section, we further discuss the pruning
approach.

###  3.2 Pruning at Initialization

Pruning in neural networks is typically of two types; i) structured and ii)
unstructured. Early works on pruning favoured unstructured approach due to its
capacity to provide high compression ratios at a finer granularity such as
individual weight-level [10]. However, recent works have practically proven
that unstructured sparsity merely achieves realistic acceleration due to
(weight) index storage overhead, and requires specialized libraries for matrix
operation [15]. In contrast, removing whole filters/kernels from convolution
layers is a structured form of pruning since it preserves the structure of the
network. Therefore, filter pruning has gained attention in reducing the model
size and computations while being implementation-friendly [4]. For this
reason, we opt for filter pruning to induce structured sparsity in the
baseline (unpruned) models.

Traditionally, filter pruning observes a three-stage procedure: – train, prune
and re-train [4]. Although this setting is widely adopted, it introduces
computational overhead characterized by multiple rounds of training and re-
training. An alternative paradigm has however emerged namely the – Lottery
Ticket Pruning, where pruning is conducted prior to training [9]. In the case
of lottery ticket pruning, sparse models are created at initialization from a
dense (unpruned) model without incurring a pre-training overhead and while
realising similar accuracy guarantees [23]. Additionally, during the pruning
phase, the saliency of filters is assessed using predefined criteria such as
the l1-norm [14], enabling the removal of filters with saliency scores below a
certain threshold. In contrast, “randomly” pruning filters has recently been
proposed to also match the performance of saliency-based approaches, without
introducing unnecessary computations of measuring saliency [16]. Nevertheless,
it is important to note that the success of random pruning lies in careful
selection of candidate convolution layers for filter pruning [23, 16].

###  3.3 Energy-driven layer selection

Typically, in the majority of pruning scenarios, networks are pruned uniformly
where each convolution layer end up with the same proportion of removed
filters. This not only results in a larger loss of performance but also leads
to an unfair pruning regime, since every layer exhibits different level of
underlying complexity. Inspired from [23], we select the candidate layer for
random filter-pruning in an energy-driven manner, until the desired
compression scale ($\varepsilon$) is met. In particular, during each pruning
iteration, the probability ${P_{k}}$ of $conv$ layer $k$ to be selected is
determined by its relative energy consumption as shown in Eq. 1:

$\mathit{P_{k}}=\frac{E_{k}}{\sum\limits_{k\in K}E_{k}},$ (1)

where $(E_{k})$ is total energy requirement of a layer $k$, which is sum of
computational energy $(E^{FLOPs}_{k})$ and energy for memory access
$(E^{Access}_{k})$. In this way, layers are penalized (pruned) in order of
their contribution to the overall energy consumption of the model. This
approach was shown in [23] to not only restricts the layer collapse but also
keeps sufficient capacity within the model to capture important patterns from
the data. Below we elaborate further on the individual terms $E^{FLOPs}_{k}$
and $E^{Access}_{k}$.

Energy for computations. The computational energy $(E^{FLOPs}_{k})$
requirement within a convolution layer emerges from its underlying matrix
multiplications and additions. These arithmetic operations are represented
theoretically as Floating-point operations (FLOPs). For a convolution layer
$k$, the number of FLOPs can be captured using Eq. 2:

$k_{flops}={[C_{in}\times(\Omega)^{2}\times C_{out}\times S_{out}]},$ (2)

then, the energy consumption for performing these computations can be modelled
using Eq. 3:

$E^{FLOPs}_{k}=k_{flops}\times A,$ (3)

where $C_{in}$ and $C_{out}$ represent the input and output channels or
filters, respectively. $\Omega^{2}$ indicates the filter dimension, $S_{out}$
is the feature size of the output layer, and $A$ denotes the energy required
for a single FLOP, i.e., 2.3pJ [11].

Energy for memory access. Accessing data stored in the memory also contributes
to significant amount of energy consumption. For a convolution layer $k$, the
amount of memory required can be formulated using Eq. 4:

$k_{mem}={[(k_{params}\times 3)+\\{(S_{out}\times C_{out}^{k})\times
2\\}]}\times 4,$ (4)

then, the energy consumption for accessing this amount of data can be modelled
using Eq. 5:

$E^{Access}_{k}=k_{mem}\times B,$ (5)

where $k_{params}$ are the number of parameters, and $B$ denotes the energy
required to access 1MB of data from DRAM i.e., 640pJ [11]. In Eq. 4, the
constant 3 accounts for gradients and momentum variables apart from
parameters, and 2 accounts for storing the activations and their errors during
the backpropagation. Finally, 4 refers to number of bytes required to store in
a 32-bit system.

###  3.4 Uncertainty and biomass estimation

Although model compression reduces the energy consumption of neural networks,
this cannot be attained without sacrificing on accuracy. In the case of
biomass composition estimation from phone images, we specifically observe that
the loss in accuracy is constrained to specific input images that become
harder to predict for the less generic energy-constrained models. Therefore,
we propose to identify unreliable predictions of the pruned model on the fly
and to refine them with accurate predictions from the unpruned model at test
time. This enables high energy efficiency by using the energy-pruned model on
most images while retaining high accuracy using the unprunned model on the
most challenging ones.

Figure 3: Examples of predicted high and low variance images from the Irish
dataset, ResNet18 $80\%$ pruned. Figure 4: Examples of predicted high and low
variance images from the GrassClover dataset, ResNet18 $80\%$ pruned.

Variance attenuation loss. Since Figure 5(b) and Figure 6(b) evidence that the
errors made by the pruned models are concentrated in a few images, we aim to
detect these harder samples. Although prediction confidence is often linked to
prediction accuracy, estimating confidence in the context of regression is non
evident. We propose to train our network to expect the ground-truth biomass
target $y$ for an input image $x$ to not be a scalar biomass value but to be
an observation from a normal distribution, centered around a mean $y(x)$ and
corrupted by some input dependant noise $\sigma(x)$:
$y\sim\mathcal{N}(\mu(x),\sigma(x))$. $\mu(x)$ here is the most likely value
for $y$ (mean of the distribution) predicted by the network from the input
image $x$.

We train our network to maximize the following likelihood function over the
dataset $\mathcal{D}=\\{x_{i},y_{i}\\}_{i=1}^{N}$

$\prod_{i=1}^{N}\frac{1}{\sigma(x_{i})\sqrt{2\pi}}e^{-\frac{1}{2}\left(\frac{y_{i}-\mu(x_{i})}{\sigma(x_{i})}\right)^{2}}$
(6)

equivalent to

$\frac{1}{(\sqrt{2\pi})^{2}}e^{-\frac{1}{2}\sum_{i=1}^{N}2\ln{\sigma(x_{i})}+\left(\frac{y_{i}-\mu(x_{i})}{\sigma(x_{i})}\right)^{2}}$
(7)

which amounts to minimizing the exponent

$\sum_{i=1}^{N}2\ln(\sigma(x_{i}))+\left(\frac{(y_{i}-\mu(x_{i}))}{\sigma(x_{i})}\right)^{2}$
(8)

Note that when considering $\sigma$ is consent, minimizing $L_{uncert}$
amounts to minimizing $(y-\mu(x))^{2}$, the MSE.

In practice, when training the neural network using a batch size $B<N$ and
over multiple biomass catergories $C$, we minimize

$L_{uncert}=\frac{1}{B}\sum_{i=1}^{B}\frac{1}{C}\sum_{c=1}^{C}2\ln(\sigma_{c}(x_{i}))+\left(\frac{(y_{i,c}-\mu_{c}(x_{i}))}{\sigma_{c}(x_{i})}\right)^{2}$
(9)

also known as the variance attenuation loss [21].

Intuitively, if $(y_{i,c}-\mu_{c}(x_{i}))$ is close to zero, $L_{uncert}$
encourages the network to predict a small value for $\sigma_{c}(x_{i})$
effectively minimizing the $2\ln(\sigma_{c}(x_{i}))$ term. However, if the
prediction for $\mu_{c}(x_{i})$ is inaccurate, a large $\sigma_{c}(x_{i})$ is
encouraged so that the
$\left(\frac{(y_{i,c}-\mu_{c}(x_{i}))}{\sigma_{c}(x_{i})}\right)^{2}$ fraction
can be minimized while the $2\ln\sigma_{c}(x_{i})$ balances naive high
$\sigma_{c}(x_{i})$ solutions.

In practice, we predict $\mu(x_{i})$ and $\ln\sigma(x_{i})$ (for stability
purposes) using two linear prediction heads sharing the same feature
extractor, see Figure 2. At test time, we can evaluate the confidence of a
given prediction by averaged spread of the predicted distributions, indicated
by $\sigma(x_{i})=\sum_{c=1}^{C}\sigma_{c}(x_{i})$.

Figure 5: (a) showing the correlation between RMSE and Variance, (b) showing
the RMSE distribution over test set of Irish clover dataset for VGG-16. Figure
6: (a) showing the correlation between RMSE and Variance, (b) showing the RMSE
distribution over test set of Irish clover dataset for ResNet-18.

Visualizing high variance images. Figure 3 displays examples of low and high
variance phone images predicted at test-time on the Irish clover dataset.
While the low variance images in the top row of Figure 3 appears to be
composed mostly of tall grass, higher variance images contain more clover. It
is also interesting to notice that the detected high error images in the
bottom row contain non uniform herbage height (left), many clover and clover
flowers (all images) or foreign objects (piece of red rope in the rightmost
image). As for the GrassClover dataset in Figure 4, low variance images in the
top row appear to contain mostly grass with low amounts of visible soil. High
loss images in the bottom row however contain more clover and present a denser
herbage with no visible soil.

For both datasets, clover-heavy images appear harder to correctly predict,
especially for a pruned model. This problem is expectedly exacerbated for the
GrassClover dataset that expects the algorithm to further differentiate
between red and white clover species. It is also possible that the clover
leaves in the herbage canopy grow above and occult a part of the grass
underneath (middle and rightmost image in Figure 4’s bottom row). In the rest
of this paper, we refer to images where the pruned model outputs high error
predictions as the hard images.

Correlation between variance and RMSE. Figure 5(a) and Figure 6(a) shows the
correlation between the test RMSE and variance ($\sigma$) over different held-
out phone images from the Irish dataset. We observe that the test time RMSE
and variance are positively correlated with each other, even on images unseen
during training. For example, VGG-16 in Figure 5(a) exhibits a significant and
positive pearson correlation of $0.71$. Given this information, we can re-
infer high variance predictions using the unpruned model which yield a lower
error. For example, as shown in Figure 2, initially all incoming images in a
batch are inferred through pruned (energy-efficient) model. Then, for images
where model’s output variance does not fall under given threshold, they are
re-inferred using original (unpruned) model. This approach circumvent the
direct usage of the energy-intensive unpruned model for all images,
effectively reducing energy consumption.

###  3.5 Hybridizing Models for Energy-efficient and Accurate Biomass
Prediction

To utilize the strengths of pruned and unpruned models jointly, we propose to
leverage both settings by creating a hybrid system. Given a test image, we
first predict biomass composition and variance from the pruned (energy-
efficient) model, then if the variance exceeds a hyper-parameter threshold we
re-infer using the unpruned (energy-intensive) model to obtain the final
prediction. In essence, the threshold on the log-variance effectively acts as
a control knob to balance the energy-accuracy trade-off in real time. Our
hybrid approach is illustrated in Figure 2.

##  4 Evaluation

We conduct here experiments to quantify the effect of various pruning ratios
$\epsilon$ on the regression accuracy of a state-of-the-art prediction
algorithm [2]. We run experiments on the IrishClover and the GrassClover
datasets using ResNet18 [13] and VGG16 [17] architectures where although
energy consumption diminishes, we observe an accuracy drop after aggressive
pruning ($\epsilon>50$). We then evaluate the performance of our
pruned/unpruned hybrid algorithm on an edge device where we demonstrate that
we can maintain accuracy while decreasing energy consumption from $40\%$ to
$60\%$.

Table 1: Results for pruned models, averaged over 5 runs $\pm$ std Irish
dataset . Unpruned results ($\varepsilon=0$) are obtained using publicly
available code from [2]

Network | $\varepsilon$ | RMSE Camera | RMSE Phone
---|---|---|---
ResNet18 | 0 | $3.11\pm 0.09$ | $4.81\pm 0.68$
20 | $4.38\pm 0.19$ | $5.55\pm 1.32$
40 | $4.44\pm 0.05$ | $5.32\pm 0.16$
60 | $4.45\pm 0.16$ | $5.75\pm 1.02$
80 | $4.44\pm 0.10$ | $5.40\pm 0.44$
VGG16 | 0 | $3.84\pm 0.12$ | $5.28\pm 0.37$
20 | $3.75\pm 0.10$ | $6.82\pm 1.57$
60 | $3.84\pm 0.22$ | $6.11\pm 0.74$
90 | $3.66\pm 0.20$ | $7.50\pm 2.96$

Table 2: Results for pruned models, averaged over 5 runs $\pm$ std.
GrassClover dataset . Unpruned results ($\varepsilon=0$) are obtained using
publicly available code from [2]

Network | $\varepsilon$ | RMSE Camera
---|---|---
ResNet18 | 0 | $10.06\pm 0.28$
20 | $10.91\pm 0.42$
40 | $11.25\pm 0.37$
60 | $11.28\pm 0.35$
80 | $11.21\pm 0.10$
VGG16 | 0 | $11.83\pm 0.32$
20 | $13.16\pm 0.42$
60 | $13.67\pm 1.10$
90 | $13.52\pm 0.80$

###  4.1 Evaluation Settings

We conduct biomass composition estimation experiments on the GrassClover [19]
and Irish clover [12] datasets. We propose to train two different network
architectures: ResNet18 [13] and VGG-16 [17]. We run experiments for $100$
epochs and adopt a cosine learning rate decay. Since we do not need to
estimate uncentainty for the unpruned models, we use the work of [2] as the
uncompressed baseline without making use of any unlabeled images. The
uncompressed baseline is trained using a Root Mean Square Error (RMSE) from a
learning rate of $0.1$ for ResNet18 and $0.03$ for VGG16. For the pruned
models, we train with $L_{uncert}$ and an initial learning rate of $0.0003$.
For all experiments we train at a resolution of $224\times 224$ and utilize
data augmentations that preserve all the information in the image: random
horizontal and vertical flipping as well as random grayscaling with a
probability of $0.2$. These training schedules follow previous implementations
[2, 1, 3].

###  4.2 Compression rate vs prediction accuracy

We first study the effect of the compression rate $\varepsilon$ on the
prediction error of the network. To do so, we generate five independent pruned
models for each compression percentage $\varepsilon$. We then train the pruned
networks using $L_{uncert}$ from equation 9 while the unpruned models are
trained using a RMSE loss. For the Irish dataset, we train the network on the
high quality camera images and evaluate on a held out set of camera or phone
images to simulate edge conditions. For the GrassClover dataset, we report our
results on the validation set which only contains high quality images. We
repeat the experiments for VGG16 and ResNet18. Results are available in Tables
1 and 2. We observe in the case of the Irish dataset that the pruned models
manage to maintain good accuracy results on the higher quality camera images
despite the compression but that the performance drops on the more challenging
camera images. We observe a similar increase in RMSE when training pruned
networks on the GrassClover dataset in Table 2 even if all the images we test
on are of high quality.

Interestingly, increasing $\varepsilon$ from $20\%$ to $90\%$ does not
significantly increase the test error for ResNet18 or VGG16 on either dataset.
These results may hint toward dry biomass composition being a task not
requiring large amounts of parameters to solve, at least regarding high
quality camera images. The accuracy difference between unpruned and pruned
models, especially in the case of the phone images does however motivate a
hybrid unpruned-pruned approach.

###  4.3 Energy efficiency and RMSE on edge

We evaluate the energy efficiency and RMSE on a real edge device to compare
the proposed approach with pruned-only and unpruned-only.

Edge Device. We performed experiments for inference on a resource-constrained
edge device i.e., NVIDIA Jetson Nano. This is a single-board device, similar
to Raspberry Pi, but with a GPU to handle edge AI workloads. This edge device
comprises a GPU with 128 NVIDIA CUDA® cores, a Quadcore ARM Cortex-A57 CPU,
4GB of RAM, and 16GB of microSD storage.

Proposed vs pruned / Unpruned. Figures 8 and 7 show how our approach can
reduce energy consumption by $50\%$ on average with respect to the original
unpruned model, while only losing an average of $4\%$ in accuracy. These
experiments are repeated over 5 runs, and the mean results are presented with
$95\%$ confidence interval.

In Figures 8(a) and 8(b), we show a performance evaluation over different
threshold values for ResNet-18 and VGG-16 models respectively on the
GrassClover dataset [19]. Similarly, in Figures 7(a) and 7(b), different
threshold values are evaluated for VGG16 and ResNet18 models respectively on
the Irish clover [12] dataset. In particular, we indicate the optimal
threshold value for each model on both datasets to achieve high energy
efficiency and low RMSE we achieve using our approach.

Figure 7: Comparison of proposed approach with pruned-only and unpruned-only
methods for energy efficiency and RMSE on Irish clover dataset, and (a)
VGG-16, and (b) ResNet-18 models. Figure 8: Comparison of proposed approach
with pruned-only and unpruned-only methods for energy efficiency and RMSE on
GrassClover dataset, and (a) ResNet-18, and (b) VGG-16 models.

Number of re-inferred images. Table 3 shows the proportion of camera and phone
images of the Irish dataset being refined by unpruned model according to
different variance thresholds. We observe that phone images are re-inferred
more often than the high resolution camera images. This indicates that the
pruned algorithm struggles to generate to these harder images, unseen during
training. Although the unpruned model was not seen phone images during
training, it manages to generalize better that the pruned one, resulting in
smaller prediction errors.

##  5 Conclusion

Evaluating the composition of dry biomass in grass provides valuable
information for farmers. This assessment helps farmers enhance the quality of
their grass and make informed decisions about local fertility improvement.
Although computer vision algorithms based on deep learning have shown great
potential in predicting accurate composition of dry biomass content, these
algorithms are energy-intensive hence hampering their usage on resource-
constrained edge devices. We first propose to apply filter pruning via energy-
driven layer selection to reduce energy consumption. We however find that
pruned model can suffer to generalize to challenging samples containing large
amounts of clover as well as imperfect real world phone images. To retain
accuracy while reducing energy usage, we propose to evaluate the uncertainty
of the regression prediction using a variance attenuation loss. At test-time,
only harder images whose predicted variance exceeds a threshold are refined by
the more accurate unpruned network. We observe that this hybrid approach only
slightly increases the error rate of previous algorithms while largely
reducing the energy requirements of deep learning biomass estimation
algorithms. Finally, generating a variance metric for the network’s
predictions, which correlate with the actual biomass prediction error, is
crucial for encouraging acceptance among human experts. In practical
scenarios, identifying a high predicted variance would prompt end users to
either take a clearer picture of the grass or to approach the network’s
predictions with due caution.

Table 3: Number of Camera and Phone images being re-inferred by unpruned model
on Irish dataset. Each validation image has a phone and a camera view for a
total of $214$ images ($107$ in each view)

Network | Threshold | # of Camera | # of Phone | Total
---|---|---|---|---
ResNet18 | 0.033 | $16$ | $32$ | $48$
0.035 | $9$ | $18$ | $27$
0.037 | $9$ | $17$ | $26$
0.039 | $6$ | $13$ | $19$
0.041 | $4$ | $11$ | $15$
VGG16 | 0.08 | $28$ | $32$ | $60$
0.10 | $20$ | $15$ | $35$
0.12 | $8$ | $9$ | $17$
0.14 | $2$ | $5$ | $7$
| 0.16 | $0$ | $2$ | $2$

## Acknowledgment

This work was supported by the Science Foundation Ireland (SFI), by the
Department of Agriculture, Food, and Marine on behalf of the Government of
Ireland VistaMilk research centre under award 16/RC/3835, and in part by SFI
under Grant 21/FFP-A/9174, by NGI Enrichers program under Horizon Europe
grant: 101070125, and by the National Science Foundation (NSF) under the CPS
award # 2133407.

## References

  * Albert et al. [2021] Paul Albert, Mohamed Saadeldin, Badri Narayanan, Brian Mac Namee, Deirdre Hennessy, Aisling H. O’Connor, Noel E. O’Connor, and Kevin McGuinness.  Semi-supervised dry herbage mass estimation using automatic data and synthetic images.  In _IEEE International Conference on Computer Vision (ICCV)_ , 2021.
  * Albert et al. [2022a] Paul Albert, Mohamed Saadeldin, Badri Narayanan, Brian Mac Namee, Deirdre Hennessy, Noel E O’Connor, and Kevin McGuinness.  Unsupervised domain adaptation and super resolution on drone images for autonomous dry herbage biomass estimation.  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , 2022a.
  * Albert et al. [2022b] Paul Albert, Mohamed Saadeldin, Badri Narayanan, Brian Mac Namee, Noel E. O’Connor, Deirdre Hennessy, Aisling H. O’Connor, and Kevin McGuinness.  Utilizing unsupervised learning to improve sward content prediction and herbage mass estimation.  In _European Grassland Federation Symposium_ , 2022b.
  * Anwar et al. [2017] Sajid Anwar, Kyuyeon Hwang, and Wonyong Sung.  Structured pruning of deep convolutional neural networks.  _ACM Journal on Emerging Technologies in Computing Systems (JETC)_ , 13(3):1–18, 2017.
  * Apolo-Apolo et al. [2020] Orly Enrique Apolo-Apolo, Manuel Pérez-Ruiz, Jorge Martinez-Guanter, and Joao Valente.  A cloud-based environment for generating yield estimation maps from apple orchards using uav imagery and a deep learning technique.  _Frontiers in plant science_ , 11:1086, 2020.
  * Berthelot et al. [2019] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel.  Mixmatch: A holistic approach to semi-supervised learning.  _Advances in Neural Information Processing Systems (NeurIPS)_ , 2019.
  * de Camargo et al. [2021] Tibor de Camargo, Michael Schirrmann, Niels Landwehr, Karl-Heinz Dammer, and Michael Pflanz.  Optimized deep learning model as a basis for fast uav mapping of weed species in winter wheat crops.  _Remote Sensing_ , 13(9):1704, 2021.
  * Der Yang et al. [2020] Ming Der Yang, Hsin Hung Tseng, Yu Chun Hsu, and Wei Chen Tseng.  Real-time crop classification using edge computing and deep learning.  In _2020 IEEE 17th Annual Consumer Communications & Networking Conference (CCNC)_, pages 1–4. IEEE, 2020.
  * Frankle and Carbin [2018] Jonathan Frankle and Michael Carbin.  The lottery ticket hypothesis: Finding sparse, trainable neural networks.  In _International Conference on Learning Representations_ , 2018.
  * Han et al. [2015] Song Han, Huizi Mao, and William J Dally.  Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding.  _arXiv preprint arXiv:1510.00149_ , 2015.
  * Han et al. [2016] Song Han, Xingyu Liu, Huizi Mao, Jing Pu, Ardavan Pedram, Mark A Horowitz, and William J Dally.  Eie: Efficient inference engine on compressed deep neural network.  _ACM SIGARCH Computer Architecture News_ , 44(3):243–254, 2016.
  * Hennessy et al. [2021] Deirdre Hennessy, Mohamed Saadeldin, Brian Mac Namee, Noel E. O’Connor, Kevin. McGuinness, Paul Albert, Badri Narayanan, and Aisling H. O’Connor.  Using image analysis and machine learning to estimate sward clover content.  _Grassland Science in Europe, Vol. 26 – Sensing – New Insights into Grassland Science and Practice_ , pages 132–134, 2021.
  * Kaiming et al. [2016] H. Kaiming, Z. Xiangyu, R. Shaoqing, and S. Jian.  Deep Residual Learning for Image Recognition.  In _IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2016.
  * Li et al. [2016] Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf.  Pruning filters for efficient convnets.  In _International Conference on Learning Representations_ , 2016.
  * Ma et al. [2021] Xiaolong Ma, Sheng Lin, Shaokai Ye, Zhezhi He, Linfeng Zhang, Geng Yuan, Sia Huat Tan, Zhengang Li, Deliang Fan, Xuehai Qian, et al.  Non-structured dnn weight pruning—is it beneficial in any platform?  _IEEE transactions on neural networks and learning systems_ , 33(9):4930–4944, 2021.
  * Mittal et al. [2019] Deepak Mittal, Shweta Bhardwaj, Mitesh M Khapra, and Balaraman Ravindran.  Studying the plasticity in deep convolutional neural networks using random pruning.  _Machine Vision and Applications_ , 30(2):203–216, 2019.
  * Simonyan and Zisserman [2014] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  In _IEEE International Conference on Learning Representations (ICLR)_ , 2014.
  * Skovsen et al. [2018] Søren Skovsen, Mads Dyrmann, Jørgen Eriksen, René Gislum, Henrik Karstoft, and Rasmus Nyholm Jørgensen.  Predicting dry matter composition of grass clover leys using data simulation and camera-based segmentation of field canopies into white clover, red clover, grass and weeds.  In _International Conference on Precision Agriculture_ , 2018.
  * Skovsen et al. [2019] Soren Skovsen, Mads Dyrmann, Anders K Mortensen, Morten S Laursen, René Gislum, Jorgen Eriksen, Sadaf Farkhani, Henrik Karstoft, and Rasmus N Jorgensen.  The GrassClover image dataset for semantic and hierarchical species understanding in agriculture.  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , 2019.
  * Skovsen et al. [2021] Søren Kelstrup Skovsen, Morten Stigaard Laursen, Rebekka Kjeldgaard Kristensen, Jim Rasmussen, Mads Dyrmann, Jørgen Eriksen, René Gislum, Rasmus Nyholm Jørgensen, and Henrik Karstoft.  Robust species distribution mapping of crop mixtures using color images and convolutional neural networks.  _Sensors_ , 2021.
  * Valdenegro-Toro and Mori [2022] Matias Valdenegro-Toro and Daniel Saromo Mori.  A deeper look into aleatoric and epistemic uncertainty disentanglement.  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)_ , 2022.
  * Wang et al. [2020] Zhizhong Wang, Lei Zhao, Sihuan Lin, Qihang Mo, Huiming Zhang, Wei Xing, and Dongming Lu.  GLStyleNet: exquisite style transfer combining global and local pyramid features.  _IET Computer Vision_ , 2020.
  * Zawish et al. [2024] Muhammad Zawish, Steven Davy, and Lizy Abraham.  Complexity-driven model compression for resource-constrained deep learning on edge.  _IEEE Transactions on Artificial Intelligence_ , pages 1–15, 2024.


# On isomorphism to a free group and beyond

Vladimir Shpilrain  Department of Mathematics, The City College of New York,
New York, NY 10031<EMAIL_ADDRESS>
###### Abstract.

The isomorphism problem for infinite finitely presented groups is probably the
hardest among standard algorithmic problems in group theory. It has been
completely solved only in the classes of nilpotent groups, hyperbolic groups,
and limit groups. In this short paper, we address the problem of isomorphism
to particular groups, including free groups and subgroups of limit groups.

In memory of Ben Fine

##  1\. Introduction

The isomorphism problem has been completely solved in the class of finitely
generated nilpotent groups in [8].

Later, it was solved in the class of hyperbolic groups [15] (torsion-free
case), [5] (general case), although it is difficult (if at all possible) to
“computerize” these algorithms, i.e., to code them in one of the known
programming languages.

Then, the isomorphism problem was also solved in the class of limit groups
(a.k.a. fully residually free groups) [4].

In the class of finitely generated one-relator groups, although the
isomorphism problem is still open in general, it has been settled for “most”
one-relator groups (in a precise formal sense) in [9]. More specifically, for
any $r\geq 2$, there is a subset $\mathcal{G}$ of elements of the free group
$F_{r}$ such that: (1) $\mathcal{G}$ has asymptotic density 1 in $F_{r}$; (2)
it is algorithmically possible to find out whether or not a given element
$u\in F_{r}$ is in $\mathcal{G}$; (3) for any two elements
$u,v\in\mathcal{G}$, it is algorithmically possible to find out whether or not
two one-relator groups (with the relators $u$ and $v$, respectively) are
isomorphic.

The “next in line” class of groups where the isomorphism problem may be
solvable is the class of finitely presented metabelian groups (see [3],
Problem (M1)), where “most” algorithmic problems have solutions by now [2].

We note that the isomorphism problem has a reasonable chance to be solvable
only in classes of groups where all groups have solvable word problem. This
rules out, for example, the class of finitely presented solvable groups of
derived length $\geq 3$ since this class has groups with unsolvable word
problem [10].

In this paper, we address an apparently easier problem of isomorphism to a
particular group. Using a simple trick, we establish here the following result
that appears to be useful in some situations.

######  Proposition 1.

Let $G$ be a group with $n$ given generators. Suppose that $G$ has solvable
word problem. Let $H$ be a finitely presented group, and suppose either $G$ or
$H$ is Hopfian. If one can decide whether or not there is an epimorphism from
$G$ onto $H$ and find it as an explicit map on the generators in case it
exists, then one can decide whether or not $G$ is isomorphic to $H$.

Recall that a group is Hopfian if any onto endomorphism of this group is also
one-to-one, i.e., is an automorphism. Note that in Proposition 1 we do not
require that $H$ has solvable word problem or that $G$ is finitely presented.

Our main goal actually was to address the problem of isomorphism to the
(absolutely) free group $F_{n}$ of rank $n$. There is a classical result of
Adyan [1] saying that given an arbitrary (finitely presented) group $B$, there
is no algorithm that would decide, given any (finitely presented) group $G$,
whether or not $G$ is isomorphic to $B$. However, if we require solvability of
the word problem in $G$, then the problem of isomorphism of $G$ to the free
group $F_{n}$ becomes algorithmically solvable:

######  Theorem 1.

Let $G$ be a finitely presented group with $m$ generators and an algorithm for
solving the word problem in $G$. Then it is algorithmically possible to find
out whether or not $G$ is isomorphic to a free group of rank $n\leq m$.

There is a “detour” that leads to this result, see [7, Corollary 4.3] for an
explicit mention of this result. Specifically, there is an algorithm that,
given a finitely presented group $G$ with solvable word problem, decides
whether or not $G$ is a limit group [6]. If not, then $G$ cannot be isomorphic
to a free group because any finitely generated free group is a limit group. If
$G$ is a limit group, then one can use an algorithm, due to [4], that decides
if there is an isomorphism between two limit groups.

Our proof is more straightforward, but it still uses a “big gun”, namely
Razborov’s work on solving (systems of) equations in a free group.

It appears that solvability of equations in groups should inevitably be an
important ingredient in any solution of the isomorphism problem for infinite
groups. However, this is typically not enough. In our proof of Theorem 1, we
actually establish an isomorphism (or non-isomorphism) of the group $G$ to a
subgroup of a given fixed finitely generated free group, and then we use the
fact that every nontrivial subgroup of a free group is itself free. This is
not the case with hyperbolic groups, say; moreover, a finitely generated
subgroup of a hyperbolic group may not even be finitely presented, and this
makes our method inapplicable in that situation. One class of groups where our
method does work is the class of limit groups since every finitely generated
subgroup of a limit group is a finitely presented limit group. Also, finitely
generated limit groups are Hopfian because they are residually free and
therefore residually finite. The following result may be of interest:

######  Theorem 2.

Let $G$ be a finitely presented group with a given algorithm for solving the
word problem in $G$. Let $H$ be a limit group with a given algorithm for
solving the word problem in $H$. Then it is algorithmically possible to find
out whether or not $G$ can be embedded in $H$.

##  2\. Proof of Proposition 1

Let $g_{1},\ldots,g_{n}$ be the given generators of the group $G$, and
$h_{1},\ldots,h_{n}$ generators of the group $H$. Needless to say, if there is
no epimorphism from $G$ onto $H$, then $G$ and $H$ are not isomorphic.

Now suppose the map $\varphi:g_{i}\to h_{i}$ can be extended to an epimorphism
from $G$ onto $H$. Then run two algorithms in parallel:

1. Algorithm $\mathcal{A}$ will detect non-isomorphism by looking for an element in the kernel of $\varphi$. To that effect, it goes over nontrivial elements of $G$ one at a time (this is possible since the word problem in $G$ is solvable) and checks if $\varphi$ takes them to the trivial element of $H$.

Here the reader may say: wait, you do not require that the word problem in $H$
is solvable. Indeed, but here we only need the “yes” part of the word problem
(i.e., detecting that the element is trivial), and this part works in any
recursively presented group. Specifically, to detect that $w=1$ one can go
over all finite products of conjugates of defining relators and (graphically)
compare them to $w$.

We note that if the kernel of $\varphi$ is nontrivial, then $H$ is isomorphic
to a proper factor group of $G$ and therefore cannot be isomorphic to $G$
since we assumed that either $G$ or $H$ was Hopfian.

2. Algorithm $\mathcal{B}$ will detect isomorphism by looking for a map $\psi$, given on the generators $h_{i}$ of $H$, such that $\psi(\varphi(g_{i}))=g_{i}$ for all generators $g_{i}$ of the group $G$. To that effect, $\mathcal{B}$ will go over $n$-tuples $(y_{1},\ldots,y_{n})$ of elements of $G$, one at a time, and define $\psi$ by $\psi(h_{i})=y_{i}$.

First check if $\psi$ is a homomorphism by computing $\psi(r_{j})$ for every
defining relator $r_{j}$ of the group $H$ and checking if $\psi(r_{j})=1$.
This is possible since $G$ has solvable word problem, although we do not
really need this because again, here we only need the “yes” part of the word
problem.

If $\psi$ is a homomorphism, then just check if $\psi(\varphi(g_{i}))=g_{i}$
for all $g_{i}$, again using the “yes” part of the word problem in $G$. If $H$
is isomorphic to $G$, then eventually a map $\psi$ like that will be found.

Eventually one of the algorithms, $\mathcal{A}$ or $\mathcal{B}$, will stop
and give an answer. $\Box$

We note that the only place in the proof where we used solvability of the word
problem in $G$ was where we were trying to detect non-isomorphism by looking
for a nontrivial element in the kernel of $\varphi$.

##  3\. Proof of Theorem 1

Let $g_{1},\ldots,g_{m}$ be the given generators of the group $G$, and let
$r_{1},\ldots,r_{s}$ be all defining relators of $G$. Let $F_{n}$ be a free
group of rank $n$, and let $\alpha:g_{i}\to x_{i}$ for some $x_{i}\in F_{n}$,
$i=1,\ldots,m$. This map extends to a homomorphism $\alpha:G\to F_{n}$ if and
only if $\alpha(r_{j})=1$ for all $j=1,\ldots,s$. This translates into a
system of $s$ equations in the group $F_{n}$.

First, we will run Razborov’s algorithm $\mathcal{R}$ [14] to see if this
system of equations has a solution tuple $(a_{1},\ldots,a_{m})$ that generates
a free subgroup of rank $r\geq n$ in $F_{n}$; in other words, if there is an
epimorphism of $G$ onto a free group of rank $r\geq n$. Denote this free group
by $H_{r}$ (recall that every nontrivial subgroup of a free group is free). If
the system has no such solutions, then $G$ is not isomorphic to a free group
of rank $n$.

If there is an epimorphism of $G$ onto $H_{r}$, then there is also an
epimorphism of $G$ onto a free group of rank $n$, denote this group by
$H_{n}$. To find an explicit epimorphism of $G$ onto $H_{n}$ (as a map on the
generators), one can first find generators of $H_{n}$ and an epimorphism of
$H_{r}$ onto $H_{n}$ by using Nielsen reduction, see e.g. [13].

After one finds an epimorphism of $G$ onto $H_{n}$, Proposition 1 applies
(since any finitely generated free group is Hopfian), and this completes the
proof. $\Box$

We note that Razborov’s results [14] were crucial for this proof. We also note
that we used not only an algorithm for solving systems of equations in a free
group, but also the fact (due to [14] as well) that it is algorithmically
possible to find a subgroup of $F_{n}$ of the maximum rank generated by a
solution tuple of the given system of equations.

##  4\. Proof of Theorem 2

For the most part, the proof is similar to that of Theorem 1. Again, let
$g_{1},\ldots,g_{m}$ be the given generators of the group $G$, and Let
$r_{1},\ldots,r_{s}$ be all defining relators of $G$. Let $\alpha:g_{i}\to
x_{i}$ for some $x_{i}\in H$. This map extends to a homomorphism $\alpha:G\to
H$ if and only if $\alpha(r_{i})=1$ for all $i=1,\ldots,s$. This translates
into a system of $s$ equations in the group $H$.

There are known algorithms for solving systems of equations in limit groups
(see e.g. [11]). Moreover, the results of [11] imply that in a limit group
$H$, different $m$-tuples of solutions of a system of equations generate only
finitely many subgroups $H_{i}$ of the group $H$ up to isomorphism, and a
(finite) presentation of each subgroup $H_{i}$ can be algorithmically computed
according to [12, Theorem 30].

We will therefore first run an algorithm from [11] to see if the system of
equations mentioned in the first paragraph of this section has solutions. If
not, then $G$ cannot be embedded in $H$. If it does have solutions, then we
find generating $m$-tuples $(h_{i1},\ldots,h_{im})$ of subgroups $H_{i}$.
Then, using an algorithm from [11], we find (finitely many) defining relations
for each subgroup $H_{i}$ representing an isomorphism class mentioned in the
previous paragraph.

Thus, if $G$ can be embedded in $H$, it should be isomorphic to one of the
subgroups $H_{i}$. Suppose there are $k$ of them. We will then run $k$
algorithms $\mathcal{C}_{i}$ in parallel, where each $\mathcal{C}_{i}$, in
turn, is a pair of algorithms $(\mathcal{A}_{i},\mathcal{B}_{i})$ running in
parallel.

As in the proof of Theorem 1, algorithm $\mathcal{A}_{i}$ will detect non-
isomorphism by looking for a nontrivial element in the kernel of
$\varphi:g_{j}\to h_{ij}$. If the kernel is nontrivial, then the subgroup
$H_{i}$ is isomorphic to a proper factor group of the group $G$ and therefore
cannot be isomorphic to $G$ itself because all finitely generated subgroups of
a limit group are Hopfian.

At the same time, algorithm $\mathcal{B}_{i}$ will detect isomorphism of the
subgroup $H_{i}$ to the group $G$ by looking for a map $\psi$, given on the
generators $h_{ij}$ of $H_{i}$, such that $\psi(\varphi(g_{i}))=g_{i}$ for all
generators $g_{i}$ of the group $G$. This is done the same way as in the proof
of Theorem 1, but there is one more ingredient needed here. To check if $\psi$
is a homomorphism, we see if $\psi$ takes each defining relation of $H_{i}$ to
the identity element of $G$.

Eventually one of the algorithms, $\mathcal{A}$ or $\mathcal{B}$, will stop
and give an answer about isomorphism (or non-isomorphism) of $H_{i}$ to $G$.

Acknowledgement. I am grateful to Olga Kharlampovich and Alexei Myasnikov for
useful discussions on equations in groups and on various properties of limit
groups.

## References

  * [1] S. I. Adyan, Algorithmic unsolvability of problems of recognition of certain properties of groups, Dokl. Akad. Nauk SSSR 103 (1955), 533–535.
  * [2] G. Baumslag, F. Cannonito, D. Robinson, The algorithmic theory of finitely generated metabelian groups, Trans. Amer. Math. Soc. 344 (1994), 629–648.
  * [3] G. Baumslag, A. G. Myasnikov, V. Shpilrain, Open problems in combinatorial group theory. Second edition, Contemp. Math., Amer. Math. Soc. 296 (2002), 1–38.
  * [4] I. Bumagin, O. Kharlampovich, A. Miasnikov, The isomorphism problem for finitely generated fully residually free groups, J. Pure Appl. Algebra 208 (2007), 961–977.
  * [5] F. Dahmani, V. Guirardel, The isomorphism problem for all hyperbolic groups, Geom. Funct. Anal. 21 (2011), 223–300.
  * [6] D. Groves, Limit groups for relatively hyperbolic groups, I: The basic tools, Algebr. Geom. Topol. 9 (2009), 1423–1466.
  * [7] D. Groves, H. Wilton, Enumerating limit groups, Groups Geom. Dyn. 3 (2009), 389–399.
  * [8] F. Grunewald, D. Segal, Some general algorithms II. Nilpotent groups, Ann. of Math. 112 (1980), 585–617.
  * [9] I. Kapovich, P. Schupp, V. Shpilrain, Generic properties of Whitehead’s algorithm and isomorphism rigidity of random one-relator groups, Pacific J. Math. 223 (2006), 113–140.
  * [10] O. G. Kharlampovich, A finitely presented solvable group with unsolvable word problem. (Russian) Izv. Akad. Nauk SSSR Ser. Mat. 45 (1981), 852–873.
  * [11] O. G. Kharlampovich, A. Myasnikov, V. N. Remeslennikov, D. Serbin, Subgroups of fully residually free groups: algorithmic problems, in: Group theory, statistics, and cryptography, Contemp. Math. Amer. Math. Soc. 360 (2004), 63–101.
  * [12] O. G. Kharlampovich, A. Myasnikov, Elementary theory of free non-abelian groups, J. Algebra 302 (2006), 451–552.
  * [13] R. Lyndon, P. Schupp, Combinatorial Group Theory, (Reprint of the 1977 edition). In Classics in Mathematics, Springer-Verlag, Berlin (2001).
  * [14] A. A. Razborov, Systems of equations in a free group, Izv. Akad. Nauk SSSR Ser. Mat. 48 (1984), 779–832.
  * [15] Z. Sela, The isomorphism problem for hyperbolic groups. I, Ann. of Math. 141 (1995), 217–283.


# HERDPhobia: A Dataset for Hate Speech against Fulani in Nigeria

Saminu Mohammad Aliyu1,2,3, Gregory Maksha Wajiga2, Muhammad Murtala2,
Shamsuddeen Hassan Muhammad1,3,4, Idris Abdulmumin3,5, Ibrahim Said Ahmad1,3
1Bayero University, Kano, Nigeria, 2Modibbo Adama University, Yola, 3HausaNLP,
4LIAAD-INESC TEC, Porto, Portugal, 5Ahmadu Bello University, Zaria, Nigeria
<EMAIL_ADDRESS>
###### Abstract

Social media platforms allow users to freely share their opinions about issues
or anything they feel like. However, they also make it easier to spread hate
and abusive content. The Fulani ethnic group has been the victim of this
unfortunate phenomenon. This paper introduces the HERDPhobia—the first
annotated hate speech dataset on Fulani herders in Nigeria—in three languages:
English, Nigerian-Pidgin, and Hausa. We present a benchmark experiment using
pre-trained languages models to classify the tweets as either hateful or non-
hateful. Our experiment shows that the XML-T model provides better performance
with 99.83% weighted F1. We released the
dataset111https://github.com/hausanlp/HERDPhobia for further research.

##  1 Introduction

Hate speech is any form of communication that promotes hatred against an
individual or a group based on religion, sexual orientation, race, ethnicity,
gender, nationality, age, and disability Schmidt and Wiegand (2017). Recently,
the Natural Language Processing (NLP) community has seen an increase in
research related to the task of detecting hate speech Alkomah and Ma (2022);
Poletto et al. (2021); Celli et al. (2021); Moy et al. (2021).

The Fulani herdsmen in Nigeria are known for their migration, along with their
herds from one location to another for grazing purposes Enor et al. (2019).
Recently, there has been an exponential increase in hate rhetoric against the
Fulani tribe Udanor and Anyanwu (2019). Hence, there is a need to develop an
automatic system for the detection of hate speech.

In this paper, we present HERDPhobia, a Twitter dataset for hate speech
detection against Fulani herdsmen. To our knowledge, this is the first dataset
created for the detection of hate speech against Fulani herdsmen in Nigeria.

##  2 Related work

Some of the work done to create hate dataset for automatic detection includes:
a corpus of 16,914 English tweets to detect sexist or racial slur by Waseem
and Hovy (2016); Davidson et al. (2017) collected 85.5 million tweets from
Twitter using a hate speech lexicon by Hatebase.org, manually annotating a
random 25,000; Poletto et al. (2017) created a corpus of hate against
immigrants, Muslims, and Roma; Ibrohim and Budi (2018) developed a dataset to
detect abusive language in Indonesian tweets using a machine learning
approach; Ousidhoum et al. (2019) annotated multilingual hate corpus in
English, French, and Arabic. and del Arco et al. (2021) created a corpus of
comments from Twitter, YouTube, and Instagram for offensive language detection
in Spanish.

##  3 Data Collection and Annotation

Following a similar approach for tweet collection used in Muhammad et al.
(2022), we collected tweets using three keywords: Fulani, cow and herdsmen
from Nigeria in three languages: English (97.2%), Hausa (1.8%) and Nigerian-
Pidgin (1%).

For data annotation, we adopted the guidelines from Warner and Hirschberg
(2012), and we annotated tweets into three categories: Hate (HT), Not-Hate
(NHT), and Indeterminate(IND). A tweet was labelled as hate if it contains
words that attack or disparage an individual or group that belongs to the
Fulani tribe. All factual and non-sentimental tweets were classified as Not-
Hate. Tweets that are sarcastic or ambiguous were classified as indeterminate.
After the annotation, we obtained $1,131$ HT, $5,007$ NHT, and $36$ IND
tweets, with a Fleiss kappa inter-annotator agreement score of 57%.

S/N |  Tweet |  A1 |  A2 |  A3
---|---|---|---|---
1 |  @user it’s true that all  fulani herdsmen are terrorist think wisely b4 answer |  HT |  HT |  HT
2 |  @user how much for the whole.cow boss |  NH |  NH |  NH
3 |  @user na so all for fulani malu |  IND |  HT |  NH

Table 1: Hate Tweets Exmaples, A1 = Annotator1, A2 = Annotator2, A3 =
annotator3

Three annotators annotated each tweet, and we used a simple majority vote to
select the tweet to train our model from two classes (HT and NHT). Table 1
provides examples of annotated the tweets. The first two were straightforward
cases were the three annotators unanimously agree on their labels as Hate and
Not Hate respectively. The last is a hard-case annotation scenario where the
three annotators could not agree due to its ambiguity. Figure 1 shows
distribution of hateful tweets, with Lagos having the highest. Overall,
hateful tweets are more prevalent in the southern part of Nigerian states, as
also reported in Udanor and Anyanwu (2019).

##  4 Baseline Model

To prepare the training and validation data, we removed all tweets that are
partially or completely annotated as indeterminate. This resulted in 892 Hate
and 3,523 Not Hate tweets. We splitted the dataset as shown in Table 2.

LagosAbujaOthersAnambraOyoDeltaEkitiOsunOgun$100$$200$$300$$400$$500$$598$$177$$28$$18$$14$$13$$13$$12$$10$ Figure 1: Distribution of hateful tweets by states. 12 states have less than 10 tweets and are group as Others. Data | Train | Dev | Test
---|---|---|---
# of tweets | 3,090 | 441 | 884
Table 2: Training and Validation Data Statistics.

We trained three different models on the HERDPhobia dataset by fine-tuning 3
pretrained language models: mBERT, XLM-T and AfriBERTa to create a baseline.
The models were trained using the same hyper-parameters as in Shode et al.
(2022): a total of 20 epochs using a batch size of 32 and a maximum sentence
length of 128. The weighted f1-score obtained for each model on the test set
is shown in Table 3. The best performance was obtained from XLM-T with 99.83%
weighted F1. This may be because HERDPhobia is dominated by English tweets,
the same language that was used to pretrain XLM-T.

Model | F1-score
---|---
XLM-T Barbieri et al. (2022) | 99.83
mBERT Devlin et al. (2019) | 80.96
AfriBERTa Ogueji et al. (2021) | 78.07
Table 3: Benchmark results with weighted F1 scores.

##  5 Conclusion and Future Work

While there are many forms of hate based on religious and ethnic stereotypes
in Nigeria, the Fulani tribe suffers the most. In this paper, we present a new
dataset for hate speech against Fulani. The dataset—HERDPhobia—consists of a
total of 6,174 tweets that were manually annotated into three classes, hate,
not-hate and indeterminate. Our baseline experiment with three pre-trained
language models (XLM-T, mBert, AfriBerta) shows that XLM-T performs best with
99.83% weighted F1. In future work, we plan to extend our work to detecting
hate speech to more ethical groups in Nigeria. We also plan to include
offensive language in the classification category and label hate tweets
according to hate types and intensity.

##  6 Ethical Consideration

We replaced all personally identifiable information from the dataset. Mentions
are replaced with user.

## References

  * Alkomah and Ma (2022) Fatimah Alkomah and Xiaogang Ma. 2022.  A literature review of textual hate speech detection methods and datasets.  _Information_ , 13(6):273.
  * Barbieri et al. (2022) Francesco Barbieri, Luis Espinosa Anke, and Jose Camacho-Collados. 2022.  Xlm-t: Multilingual language models in twitter for sentiment analysis and beyond.  In _Proceedings of the Language Resources and Evaluation Conference_ , pages 258–266, Marseille, France. European Language Resources Association.
  * Celli et al. (2021) Fabio Celli, Mirko Lai, Armend Duzha, Cristina Bosco, and Viviana Patti. 2021.  Policycorpus xl: An italian corpus for the detection of hate speech against politics.  In _CLiC-it_.
  * Davidson et al. (2017) Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017.  Automated hate speech detection and the problem of offensive language.  In _Proceedings of the 11th International AAAI Conference on Web and Social Media_ , ICWSM ’17, pages 512–515.
  * del Arco et al. (2021) Flor Miriam Plaza del Arco, Arturo Montejo-Ráez, L Alfonso Urena Lopez, and María-Teresa Martín-Valdivia. 2021.  Offendes: A new corpus in spanish for offensive language research.  In _Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)_ , pages 1096–1108.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Enor et al. (2019) Frank N Enor, Stephen E Magor, and Charles E Ekpo. 2019.  Contending perspectives and security implications of herdsmen activities in nigeria.  _International Journal of Research-GRANTHAALAYAH_ , 7(7):265–286.
  * Ibrohim and Budi (2018) Muhammad Okky Ibrohim and Indra Budi. 2018.  A dataset and preliminaries study for abusive language detection in indonesian social media.  _Procedia Computer Science_ , 135:222–229.
  * Moy et al. (2021) Tian Xiang Moy, Mafas Raheem, and Rajasvaran Logeswaran. 2021.  Hate speech detection in english and non-english languages: A review of techniques and challenges.  _Technology_.
  * Muhammad et al. (2022) Shamsuddeen Hassan Muhammad, David Ifeoluwa Adelani, Sebastian Ruder, Ibrahim Sa’id Ahmad, Idris Abdulmumin, Bello Shehu Bello, Monojit Choudhury, Chris Chinenye Emezue, Saheed Salahudeen Abdullahi, Anuoluwapo Aremu, Alípio Jorge, and Pavel Brazdil. 2022.  NaijaSenti: A nigerian Twitter sentiment corpus for multilingual sentiment analysis.  In _Proceedings of the Thirteenth Language Resources and Evaluation Conference_ , pages 590–602, Marseille, France. European Language Resources Association.
  * Ogueji et al. (2021) Kelechi Ogueji, Yuxin Zhu, and Jimmy Lin. 2021.  Small data? no problem! exploring the viability of pretrained multilingual language models for low-resourced languages.  In _Proceedings of the 1st Workshop on Multilingual Representation Learning_ , pages 116–126, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Ousidhoum et al. (2019) Nedjma Ousidhoum, Zizheng Lin, Hongming Zhang, Yangqiu Song, and Dit-Yan Yeung. 2019\.  Multilingual and multi-aspect hate speech analysis.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 4675–4684, Hong Kong, China. Association for Computational Linguistics.
  * Poletto et al. (2021) Fabio Poletto, Valerio Basile, Manuela Sanguinetti, Cristina Bosco, and Viviana Patti. 2021.  Resources and benchmark corpora for hate speech detection: a systematic review.  _Language Resources and Evaluation_ , 55(2):477–523.
  * Poletto et al. (2017) Fabio Poletto, Marco Stranisci, Manuela Sanguinetti, Viviana Patti, and Cristina Bosco. 2017.  Hate speech annotation: Analysis of an italian twitter corpus.  In _4th Italian Conference on Computational Linguistics, CLiC-it 2017_ , volume 2006, pages 1–6. CEUR-WS.
  * Schmidt and Wiegand (2017) Anna Schmidt and Michael Wiegand. 2017.  A survey on hate speech detection using natural language processing.  In _Proceedings of the fifth international workshop on natural language processing for social media_ , pages 1–10.
  * Shode et al. (2022) Iyanuoluwa Shode, David Ifeoluwa Adelani, and Anna Feldman. 2022.  YOSM: A new Yorùbá Sentiment Corpus for Movie Reviews.  _AfricaNLP 2022 @ICLR_.
  * Udanor and Anyanwu (2019) Collins Udanor and Chinatu C Anyanwu. 2019.  Combating the challenges of social media hate speech in a polarized society: A twitter ego lexalytics approach.  _Data Technologies and Applications_.
  * Warner and Hirschberg (2012) William Warner and Julia Hirschberg. 2012.  Detecting hate speech on the world wide web.  In _Proceedings of the second workshop on language in social media_ , pages 19–26.
  * Waseem and Hovy (2016) Zeerak Waseem and Dirk Hovy. 2016.  Hateful symbols or hateful people? predictive features for hate speech detection on twitter.  In _Proceedings of the NAACL student research workshop_ , pages 88–93.