11institutetext: Institute for Intelligent Systems, University Of
Johannesburg, South Africa 11email<EMAIL_ADDRESS>22institutetext: Institute for Intelligent Systems, University Of
Johannesburg, South Africa 22email<EMAIL_ADDRESS>
# Comparing Male Nyala and Male Kudu Classification using Transfer Learning
with ResNet-50 and VGG-16

T.T Lemani 11 T.L. van Zyl 22

###### Abstract

Reliable and efficient monitoring of wild animals is crucial to inform
management and conservation decisions. The process of manually identifying
species of animals is time-consuming, monotonous, and expensive. Leveraging on
advances in deep learning and computer vision, we investigate in this paper
the efficiency of pre-trained models, specifically the VGG-16 and ResNet-50
model, in identifying a male Kudu and a male Nyala in their natural habitats.
These pre-trained models have proven to be efficient in animal identification
in general. Still, there is little research on animals like the Kudu and
Nyala, who are usually well camouflaged and have similar features. The method
of transfer learning used in this paper is the fine-tuning method. The models
are evaluated before and after fine-tuning. The experimental results achieved
an accuracy of 93.2% and 97.7% for the VGG-16 and ResNet-50 models,
respectively, before fine-tuning and 97.7% for both models after fine-tuning.
Although these results are impressive, it should be noted that they were taken
over a small sample size of 550 images split in half between the two classes;
therefore, this might not cater to enough scenarios to get a full conclusion
of the efficiency of the models. Therefore, there is room for more work in
getting a more extensive dataset and testing and extending to the female
counterparts of these species and the whole antelope species.

###### Keywords:

Transfer learning Fine-tuning VGG-16 ResNet-50

##  1 Introduction

Zoology has long been fascinated with detecting animals, but manually
identifying them can be daunting due to the vast number of different species.
Fortunately, an algorithm that can classify animals based on images can make
it easier for scientists and researchers to monitor or study animals more
efficiently. Automated animal detection has many potential applications, such
as preventing theft, ensuring the safety of animals in reserves and zoos, and
preventing animal-vehicle accidents [12].

When capturing images of animals in the wild, it can be difficult to classify
them due to their appearance in different backgrounds, climate conditions,
poses, and viewpoints. Furthermore, animals from different classes may have
similar appearances. As a result, an effective classification algorithm is
crucial for dealing with these various challenges [12, 2, 1].

Research on animal identification has shown that Artificial Neural Networks
(ANN) can accurately solve this problem for certain species. Nguyen et al.
(2017) utilized transfer learning with AlexNet, VGG-16, GoogleNet, and
ResNet-50 [9]. Additionally, Van Zyl et al. (2020) employed a Deep Transfer
Learning Siamese Network with a pre-trained ResNet-50 model [15].

Research has been conducted on identifying various animals, but limited
studies focus on distinguishing between antelopes such as the Kudu and the
Nyala. These antelopes blend seamlessly into their surroundings, making it
challenging to differentiate them due to their similar physical
characteristics [4].

In this study, we aim to demonstrate the effectiveness of pre-trained models,
namely VGG-16 and ResNet-50, in accurately identifying male Kudu and Nyala
antelopes. The results of this research have the potential to aid in the
identification of challenging-to-spot animals, such as Kudus and Nyalas.

A technique that has been proven successful in image recognition, in general,
is using transfer learning using the fine-tuning method [3, 5, 10]. The
approach taken in this study is using transfer learning to train the model on
a dataset of male
Nyala111https://github.com/tvanzyl/wildlife_reidentification/tree/main/Nyala_Data_Zero
and male Kudu with pictures taken In different settings, then retraining the
model (fine-tuning) with some of the pre-trained layers of the model unfrozen
[8].

This study offers insight into how efficiently the models perform on the
dataset before and after fine-tuning, using a custom top layer with
hyperparameters obtained using a Keras hyperband tuner. The efficiency of the
models is measured using different metrics: accuracy, precision, recall, and
f1-score.

Figure 1: Research flow of the proposed method.

##  2 Transfer learning using artificial neural networks

Transfer learning research focuses on reusing knowledge obtained in the source
domain and imposing this knowledge on the target domain to solve a new problem
[7]. Transfer learning methods are divided into transfer feature, transfer
sample, and transfer parameter [7].

The transfer sample method is used when the target and source domains show
high similarity. In this method, sample migration joins target and source
samples, and then the source weights are adequately adjusted to get target
domain weights [7]. In feature migration, feature associations are located
between the target and the source domain by rebuilding features and minimizing
their differences. Parameter migration parameters are shared between the
target and the source domains, and weights are automatically adjusted to get
optimal results [7]. Some advantages of using transfer learning are it allows
for training on a much smaller dataset, less computational power is required,
and learning over a shorter time because most of the hard work to recognize
patterns is already done in the pre-trained model[7].

The description of transfer learning above warrants us to decide regarding the
transfer learning method, pre-trained model, and the optimiser.

In this research, we have used pre-trained models, namely the VGG-16 model and
the ResNet-50 model trained on the ImageNet dataset, as base networks for
transfer learning. The method used for transfer learning is fine-tuning, and
the optimiser is Adaptive Moment Estimation (Adam).

###  2.1 VGG-16

All convolutional layers of the VGG-16 network have the same configuration.
They have a convolutional core six of 3x3 with a step size of 1 [14]. There
are five max-pooling layers, of which all have a step size of 2 and are of
size 2x2. There are three fully connected layers. The first two have the same
number of layers as the third, 4096 and 1000, respectively. 1000 in the third
fully connected layer represents the number of label categories [14]. The last
layer is a SoftMax layer. The ReLU nonlinear activation function follows all
the hidden layers [14].

###  2.2 ResNet-50

The 50 in ResNet-50 depicts the number of layers in the architecture. The
ResNet-50 model has 50 residual blocks, which consist of max pooling,
convolution, average pooling, softmax, and fully connected layers [11]. The
idea behind the ResNet-50 architecture is the skip connection. This aims to
ensure that as the layer gets deeper, the gradient in the previous layer will
not decrease in performance. The ResNet-50 nodes also use a bottleneck (use
1x1 convolutions) design for the building block[11]. This reduces the number
of matrix multiplications and parameters [11].

###  2.3 Fine-Tuning method

Fine-tuning aims to allow a section of the pre-trained layers to retrain. This
entails adding a new portion to the top part of the model, specifically the
output and fully connected layers [8]. The pre-trained convolution layers are
frozen, allowing them to convolve visual features as usual. The last few pre-
trained layers are then unfrozen and then trained on the custom data again,
and the layers update according to the fully connected layers predictions [8].
In this research for the fine-tuning step, 2 of the model’s layers are
unfrozen, and the model weights are updated.

Table 1: Training and test results. Here, we compare the results of the two models before and after fine-tuning, as well as comparing the models with each other | Before fine-tuning | After fine-tuning
---|---|---
Metric | VGG-16 | ResNet-50 | VGG-16 | ResNet-50
Test accuracy | 0.93 | 0.98 | 0.98 | 0.98
Precision | 1.00 | 1.00 | 0.98 | 1.00
Recall | 0.86 | 0.95 | 0.98 | 0.95
F1-Score | 0.93 | 0.98 | 0.98 | 0.98
Training time | 11min 25s | 11min 37s | 12min 13s | 10min 22s
Table 2: Best hyperparameters from hyperband tuner

| Best Hyperparameters
---|---
Model |  Number of layers |  Units layer$\\_$0 |  Units layer$\\_$1 |  Units layer$\\_$2 |  Activation function |  Dropout
VGG-16 | 2 | 3560 | 2696 | - | Relu | No
ResNet-50 | 3 | 1128 | 3464 | 1584 | tanh | Yes (0.2)

###  2.4 Adam optimizer

The Adam optimiser is an adaptive learning rate optimisation technique,
meaning the learning rate is not treated as a hyperparameter, and this
optimiser finds individual learning rates for various parameters [13]. This
optimiser was specifically designed for deep learning. The optimiser uses a
gradient’s first and second-moment estimation for a deep neural network to
adapt the learning rate [13]. These moments are mean and variance,
respectively. Exponential moving averages estimate moments in each batch after
every iteration [13].

The rules for updating the Adam optimiser are given:

$m_{t}=\beta_{1}m_{t-1}+(1-\beta_{1})_{gt}$ (1)

$v_{t}=\beta_{2}v_{t-1}+(1-\beta_{2})_{g_{t}^{2}}$ (2)

Where m and v are moving averages, $\beta$1 and $\beta$2 are hyperparameters,
g is the gradient of the current batch, and t is the number of iterations. Eq
(1) and Eq (2) represent the gradient and squared gradient of the moving
averages.

$w_{t}=w_{t-1}-\alpha\frac{\widehat{m_{t}}}{\sqrt{\widehat{v_{t}}}+\epsilon}$
(3)

The weights are updated according to the equations above.

##  3 Method

###  3.1 Dataset

The dataset comprises 550 samples equally split between male Nyala and male
Kudu. The median resolution of the images is 1280x1102, and the average image
size is 1.33 mp (megapixels), with the lowest being 0.04 mp and the highest
being 16.58 mp. Data augmentation is applied to the dataset in rotation,
brightness, width and height shift, and horizontal and vertical flipping. For
the VGG-16 model, the images were resized to 244x244, and for ResNet-50, the
images were resized to 180x180. For the images to conform to the ImageNet
dataset to which the model was trained, an inbuilt function is used to convert
the images from RGB to BGR and zero-center (-1.0 to 1.0) for each colour
channel.

###  3.2 Experimental Design

For the dataset, an 85% and 15% random split was used for training and
testing, respectively. All the images were resized and made to conform with
the ImageNet dataset. Only the training images were augmented.

The pre-trained layers of the model are frozen, and a top model of fully
connected layers is added to the pre-trained layers [8]. The top model
contains a flattened layer to flatten the outputs, Dense layers for
classification, a dropout layer to help with overfitting, and an output layer
(softmax layer) so that the outputs are probabilities. Category cross entropy
was used for the loss function since this classification problem uses a
softmax output layer. An Adam optimiser was used, and a learning rate of
1x$10^{-3}$ was used to initialize the optimizer. The rest of the
hyperparameters were left in their default state ($\beta$1 = 0.9, $\beta$1 =
0.99, $\epsilon$ = 1x$10^{-7}$).

The number of dense layers, the number of units in the dense layer, the
activation function, and whether to include a dropout layer were treated as
hyperparameters, and these hyperparameters were obtained using a Keras
hyperband tuner. These values are shown in Table 2.

After initial training, the model was fine-tuned by unfreezing the last two
pre-trained layers. This allows backpropagation to update the last two layers
of the pre-trained model [8]. The learning rate was reduced on retraining the
model to 1x$10^{-4}$ since the last two layers have been unfrozen. The fully
connected layers might start picking apart more minute details of the images
in the dataset instead of learning robust patterns previously learned [8].

To evaluate the performance of the models before and after fine-tuning and
with each other, the following computations were applied: recall, precision,
accuracy, and F1-Score.

###  3.3 Hardware and Libraries

All results were obtained on an NVIDIA Tesla K80 GPU 24GB, 2 Intel Haswell
CPUs, 24GB RAM, running Python 3.7 with Keras 2.9 on Google Colab.

##  4 Results And Discussion

Figure 2: Confusion matrix for VGG-16 model.

Figure 3: Confusion matrix for ResNet-50 model

Figure 4: VGG – 16 curves before fine-tuning.

Figure 5: VGG –16 curves after fine-tuning.

Figure 6: ResNet-50 curves before fine-tuning.

Figure 7: ResNet-50. curves after fine-tuning.

The result of this study shown in Table 1 shows that before fine-tuning, the
VGG-16 model achieved an accuracy of 93.2%, and the ResNet-50 model achieved
an accuracy of 97.73%. After fine-tuning, the accuracy of the two models went
to 97.73% and 97.73%, respectively. What is surprising is the two models have
the same accuracy after fine-tuning, but what is different is the model’s
precision, recall, and f1-score. The accuracy for the ResNet-50 stayed the
same after fine-tuning. This outcome is contrary to that of Gupta et al.
(2020), who found that unfreezing the lower layers causes the accuracy to
increase [6]. The confusion matrix of VGG-16 Fig. 2 shows a significant
reduction in False Negatives (FN) after fine-tuning. Though the models have
the same accuracy after fine-tuning, it can be observed from the confusion
matrix that the models have different FN and False positive classification,
but in general, the two models misclassify a male Kudu as a male Nyala. An
explanation for this might be insufficient training data or the test image’s
complex background.

Though the classification results are impressive, they were produced from a
small sample size. Caution must be applied as these findings might not hold
for a bigger sample size, especially for this problem where the animals can be
seen in different environments. The effect of the small sample size can be
seen from the accuracy and loss curves in Fig. 4-7, where the curves show the
characteristics of an unrepresentative validation dataset, especially looking
at the accuracy curves [6].

##  5 Conclusion

The present research aimed to assess the effectiveness of transfer learning
using the VGG-16 and ResNet-50 on identifying a male Kudu or Nyala in their
natural habitat. The results of this investigation show that these models
achieve high accuracy for this task, 93.2% and 97.7% for the VGG-16 and
ResNet-50 models, respectively, before fine-tuning and 97.7% for both models
after fine-tuning. After fine-tuning, the difference between the two models is
the number of false negative and false positive predictions. A limitation of
this study was that the dataset was small, which was evident in the
unrepresentative nature of the accuracy curves. Having a small dataset also
implies that there are not enough scenarios to conclude that the models
produce highly accurate results. Still, from this study, it can be ascertained
that these models have a high potential to accomplish this task. More research
using a much bigger dataset is required. This dataset should extend to the
antelope species and their female counterparts, which are generally more
difficult to distinguish.

## References

  * [1] Dlamini, N., Van Zyl, T.L.: Comparing class-aware and pairwise loss functions for deep metric learning in wildlife re-identification. Sensors 21(18), 6109 (2021)
  * [2] Dlamini, N., van Zyl, T.L.: Automated identification of individuals in wildlife population using siamese neural networks. In: 2020 7th international conference on soft computing & machine intelligence (ISCMI). pp. 224–228. IEEE (2020)
  * [3] Ghosal, S., Sarkar, K.: Rice leaf diseases classification using cnn with transfer learning. In: 2020 IEEE Calcutta Conference (CALCON). pp. 230–236. IEEE (2020)
  * [4] Gibbon, G.E., Bindemann, M., Roberts, D.L.: Factors affecting the identification of individual mountain bongo antelope. PeerJ 3, e1303 (2015)
  * [5] Guo, Y., Shi, H., Kumar, A., Grauman, K., Rosing, T., Feris, R.: Spottune: transfer learning through adaptive fine-tuning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 4805–4814 (2019)
  * [6] Gupta, A., Gupta, M.: Transfer learning for small and different datasets: fine-tuning a pre-trained model affects performance. J. Emerg. Res 3(5) (2020)
  * [7] Liu, X., Jia, Z., Hou, X., Fu, M., Ma, L., Sun, Q.: Real-time marine animal images classification by embedded system based on mobilenet and transfer learning. In: OCEANS 2019-Marseille. pp. 1–5. IEEE (2019)
  * [8] McDermott, J.: Hands-on transfer learning with keras and the vgg16 model (2021)
  * [9] Nguyen, H., Maclagan, S.J., Nguyen, T.D., Nguyen, T., Flemons, P., Andrews, K., Ritchie, E.G., Phung, D.: Animal recognition and identification with deep convolutional neural networks for automated wildlife monitoring. In: 2017 IEEE international conference on data science and advanced Analytics (DSAA). pp. 40–49. IEEE (2017)
  * [10] Palani, S.S., Dev, M., Mogili, G., Relan, D., Dey, R.: Face mask detector using deep transfer learning and fine-tuning. In: 2021 8th International Conference on Computing for Sustainable Global Development (INDIACom). pp. 695–698. IEEE (2021)
  * [11] Raihan, M., Suryanegara, M.: Classification of covid-19 patients using deep learning architecture of inceptionv3 and resnet50. In: 2021 4th International Conference of Computer and Informatics Engineering (IC2IE). pp. 46–50. IEEE (2021)
  * [12] Ravoor, P.C., Sudarshan, T.: Deep learning methods for multi-species animal re-identification and tracking–a survey. Computer Science Review 38, 100289 (2020)
  * [13] ŞEN, S.Y., ÖZKURT, N.: Convolutional neural network hyperparameter tuning with adam optimizer for ecg classification. In: 2020 Innovations in intelligent systems and applications conference (ASYU). pp. 1–6. IEEE (2020)
  * [14] Tao, J., Gu, Y., Sun, J., Bie, Y., Wang, H.: Research on vgg16 convolutional neural network feature classification algorithm based on transfer learning. In: 2021 2nd China international SAR symposium (CISS). pp. 1–3. IEEE (2021)
  * [15] Van Zyl, T.L., Woolway, M., Engelbrecht, B.: Unique animal identification using deep transfer learning for data fusion in siamese networks. In: 2020 IEEE 23rd international conference on information fusion (FUSION). pp. 1–6. IEEE (2020)