# Lexical Squad@Multimodal Hate Speech Event Detection 2023: Multimodal Hate
Speech Detection using Fused Ensemble Approach

Mohammad Kashif, Mohammad Zohair, Saquib Ali
Jamia Millia Islamia, New Delhi, India
<EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS>

###### Abstract

With a surge in the usage of social media postings to express opinions,
emotions, and ideologies, there has been a significant shift towards the
calibration of social media as a rapid medium of conveying viewpoints and
outlooks over the globe. Concurrently, the emergence of a multitude of
conflicts between two entities has given rise to a stream of social media
content containing propaganda, hate speech, and inconsiderate views. Thus, the
issue of monitoring social media postings is rising swiftly, attracting major
attention from those willing to solve such problems. One such problem is Hate
Speech detection. To mitigate this problem, we present our novel ensemble
learning approach for detecting hate speech, by classifying text-embedded
images into two labels, namely "Hate Speech" and "No Hate Speech". We have
incorporated state-of-art models including InceptionV3, BERT, and XLNet. Our
proposed ensemble model yielded promising results with 75.21 and 74.96 as
accuracy and F-1 score (respectively). We also present an empirical evaluation
of the text-embedded images to elaborate on how well the model was able to
predict and classify. We release our codebase here
https://github.com/M0hammad-Kashif/MultiModalHateSpeech

##  1 Introduction

Political events have been a perpetual part of governance to date and serve as
a medium of expression for those involved directly or indirectly with the
process. But at times, this medium of communication might turn out to be a
source of unfortunate insensitive expressions, hate speeches, etc, through
verbal forms, visual representations, and physically inconsiderate actions
among others. In such cases, it becomes crucial to monitor political events
and other potential contributors to the circulation of hate speech and
insensitive content.

According to legal publications, hate speech is defined as an expression that
seeks to malign an individual for their immutable characteristics, such as
their race, ethnicity, national origin, religion, gender, gender identity,
sexual orientation, age or disability (Carlson, 2021). Hate speech detection
is one of the most important aspects of event identification during political
events like invasions (Bhandari et al., 2023; Parihar et al., 2021). As is
evident in today’s scenario, the incorporation of multimodal data to meet
incentives is highly prevalent and is a major concern for hate speech
detection and analysis.

In this paper, we elaborate on our submission for the shared
task111https://emw.ku.edu.tr/case-2023/, for multimodal hate speech detection
through text-embedded images from the Russia-Ukraine war, which is a part of
the bigger picture leading to a significantly demanding issue Thapa et al.
(2023). Multimodal content being advertised through physical spaces, social
media, etc, is a mode of spreading hate speech and spiteful views being used
extensively in the current scenario. A significant contributor to this
phenomenon is the sharing of text-embedded images, representing the views of
an individual or a group of individuals, either directly or indirectly. In
accordance with this fact, we aim to categorically determine if a given text-
embedded image conveys hate speech in any possible form or not.

The rest of the paper is structured as follows: Section 2 illustrates the
existing work which has been carried out in this field of research; Section 3
describes the dataset and task for our research study; Section 4 elaborates
our proposed model architecture including the individual blocks incorporated
in the same; Section 5 states the results obtained from this work along with
its empirical analysis; Section 6 provides a view of the future scope in this
domain besides concluding the paper.

##  2 Literature Review

Extensive work has been carried out to survey the extent of incorporating
technology for hate speech detection. For instance, in (Schmidt and Wiegand,
2017), a survey has been carried out on the scope of hate speech detection
using natural language processing. Through this study, the features,
terminologies, existing approaches, and techniques in this context have been
highlighted.

Another similar research work (Abro et al., 2020), shows the comparison of the
performance of three feature engineering techniques and eight machine learning
algorithms on a publicly available dataset having three distinct classes. The
results of this research work showed that the bigram features when used with
the support vector machine algorithm best performed with 79% off overall
accuracy.

In another study (Badjatiya et al., 2017), an experiment has been performed to
emphasize the usage of deep learning for hate speech detection in tweets. A
Twitter dataset containing relevant tweets has been used to classify them as
being racist, sexist, or neither. As per the results obtained in this study,
the deep learning methods outperform state-of-the-art char/word n-gram methods
by $\sim$18 F1 points.

In recent times, multiple attempts have been made to deal with the concern of
intelligently determining the spread of hate speech and related expressions
through multimodal data. For instance, as a part of the research study (Gomez
et al., 2020), it was attempted to jointly analyze textual and visual
information for hate speech detection, using a large-scale dataset from
Twitter, MMHS150K. The researchers associated with this study have compared
the implementation of models working on multimodal data with those on unimodal
data.

Another research work (Das et al., 2020), featuring the detection of hate
speech in multimodal memes, forms its basis for categorizing a meme as hateful
or non-hateful. As a part of this, the visual modality using object detection
and image captioning models to fetch the “actual caption” has been explored
and combined with the multi-modal representation to perform binary
classification. Along with this, an effort has been made to enhance the
predictions using sentiment analysis.

Another instance of research work (Velioglu and Rose, 2020), has been carried
out on a dataset containing more than 10000 examples of multimodal content,
wherein VisualBERT, which is meant to be the “BERT of vision and language” was
trained multimodally on images and captions and was augmented with Ensemble
Learning.

##  3 Dataset and Task

As a part of The 6th Workshop on Challenges and Applications of Automated
Extraction of Socio-political Events from Text (CASE @ RANLP 2023), the Sub-
task A for this research experimentation is to identify whether the given
text-embedded image contains hate speech or not Thapa et al. (2023).

The dataset Thapa et al. (2022) provided for this task consists of around 4700
text-embedded images, having annotations for the prevalence of hate speech. As
a two-way classification task, the two classes in the given dataset correspond
to “Hate Speech” and “No Hate Speech”, with 2665 and 2058 samples
corresponding to the respective classes. The training data consists of 1942
and 1658 samples against the "Hate Speech" and "No Hate Speech" labels
(respectively). Concurrently, the evaluation and testing data consists of 443
random samples each. All the images have a unique identifier called "index".

In the training data, the classes are well-balanced, implying the occurrence
of 50 text-embedded images each against the two labels, that is, “Hate Speech”
as well as “No Hate Speech”.

##  4 Model Architecture

This section describes the proposed model architecture for classifying text-
embedded images as "Hate Speech" or "No Hate Speech".

Figure 1: Proposed ensemble architecture

As depicted in Figure 1, we hereby propose an ensemble approach for this
binary classification problem. Due to the multimodal nature of the data, it is
necessary to extract both visual and textual features from the provided
content containing the same. To comprehend the context of a text-embedded
image, it is necessary to map the textual context to its visual context. So as
to have both of these contexts, we propose an ensemble model that extracts
both of these characteristics from an image.

We have incorporated respective models based on convolutional neural networks
(CNN), and pre-trained transformer models, which provided good results on the
given dataset. InceptionV3 optimizes the neural network for better adaptation
as it has a deeper network compared to its predecessors and uses auxiliary
classifiers as regularizers. Eventually, it serves as an enhanced option for
visual comprehension required for this task. Since transformer models are
based on the mechanism of self-attention and differentially weigh the
significance of each part of the input textual data, they serve as an ideal
option for the textual comprehension of this task.

Our model is comprised of three backbones, one of which (InceptionV3) extracts
visual information while the other two (BERT and XLNet) extract textual
information.

###  4.1 InceptionV3

Inception-v3 is a convolutional neural network architecture from the Inception
family that makes several improvements including using Label Smoothing,
Factorized 7 x 7 convolutions, and the use of an auxiliary classifier to
propagate label information lower down the network (along with the use of
batch normalization for layers in the sidehead) Szegedy et al. (2016).

The InceptionV3 architecture uses a novel "Inception module" that extracts
multi-scale features using various-sized convolutional filters in the same
layer Szegedy et al. (2015). In order to improve the learning of
representations, this module enables the network to capture both local and
global contextual information.

Apart from that, the Inception module employs 1x1 convolutions along with
dimensionality reduction strategies to lessen computational complexity. The
inception block takes the input image

$I\in R^{C*H*W}$

and outputs a feature vector of

$F1\in R^{1*512}$

###  4.2 BERT

BERT’s model architecture is a multi-layer bidirectional Transformer encoder,
designed to create state-of-the-art models for a wide range of tasks, such as
question answering and language inference, without substantial task-specific
architecture modifications Devlin et al. (2018).

We have incorporated BERT into our proposed ensemble model to extract textual
features. As a result of being trained on a large corpus of unlabeled text,
BERT has a solid language foundation and a better understanding of general
language representation. We have extracted text from the image using Google’s
Tesseract-OCR Engine Smith (2007), which is eventually tokenized and fed to
the BERT model.

BERT outputs a feature vector of size 1x768 which is then provided to the
linear layer to generate feature vector

$F2\in R^{1*512}$

###  4.3 XLNet

XLNet is a pre-trained transformer model, which includes segments recurrence,
introduced in Transformer-XL Yang et al. (2019); Dai et al. (2019), allowing
it to digest longer documents Shaheen et al. (2020).

In order to pay greater attention to text features, we have incorporated a
language model into our ensemble-learning model, yet again. XLNet surpasses
the limitations of conventional autoregressive models by taking into account
all possible permutations of words in a sentence, resulting in enhanced
language representation and comprehension.

XLNet is based on the pretraining and fine-tuning paradigm and utilizes the
Transformer architecture. The extracted text, from the OCR Engine Smith
(2007), is fed to the tokenizer and XLNet, which generate a 1x768-dimensional
feature vector, which is then fed to the linear layer, which generates a

$F3\in R^{1*512}$

###  4.4 Ensemble Model

Ensemble learning or ensemble model is the combination of numerous different
predictions from different models to make the final prediction Ganaie et al.
(2022). This has always been an elegant way of enhancing the performance of
models.

Stacking is one of the ensemble learning integration approaches in which the
meta-learning model is utilized to integrate the output of base models
Džeroski and Ženko (2004); Zohair et al. (2022). Following this strategy, we
incorporated our implemented individual models into the blueprint of a stacked
ensemble model. This required the generation of individual embeddings from
respective models as described in the preceding subsections.

The embeddings F1, F2, and F3 as obtained from the InceptionV3, BERT, and
XLNet (respectively), are concatenated to form F4 as the final embedding for
the meta-layer Sesmero et al. (2015).

$F4\in R^{1*1536}$

After the final embedding (F4), has been processed through the linear layer, a
feature vector of size 128 is eventually produced. This feature vector is then
forwarded to the final linear layer for classification, which eventually
enhances the accuracy of the predictions. After every linear layer, a ReLU
non-linearity is applied Agarap (2018).

###  4.5 Hyperparameter

Some of the hyperparameters were kept constant in all models, namely a
learning rate of 3e-4, regularization factor of 3e-5, vocab size of 512, and
StepLR as the learning rate scheduler.

For our training, we utilized the Adam optimizer Zhang (2018) and trained the
model for 100 epochs. All experiments were conducted on a system equipped with
an NVIDIA-A100 GPU, augmented by 64 GB of RAM, with Ubuntu 20.04 as the
operating system. The implementation was carried out using the PyTorch
framework.

###  4.6 Loss and Metric Used

We have used the weighted cross-entropy loss to penalize the ensemble model
with more effectiveness during its training for multimodal classification Phan
and Yamamoto (2020). In addition, we have used accuracy as a standard
performance metric for comparing models.

##  5 Results and Discussion

We have mentioned the results obtained on the validation data in Table 1. The
results corresponding to the submission were obtained on the test data
provided for this sub-task, which have been reflected in Table 2. As it is
evident from the quoted metrics in Table 1, achieved after careful
experimentation for the desired task, the proposed ensemble model outperformed
various individual models which have been brought into usage for classifying
hate speech in the provided dataset.

Model | Accuracy | F1 Score
---|---|---
BERT | 69.65 | 69.51
XLNET | 71.80 | 71.56
InceptionV3 | 48.12 | 48.11
MobileNetV3 | 42.41 | 42.20
ResNet 152 | 44.47 | 44.38
BERT + XLNET | 73.51 | 73.39
Ensemble Model | 75.21 | 74.96
Table 1: Metric Comparison for proposed ensemble model with conventional models # | User | <Rank> | Recall | Precision | F1 | Accuracy
---|---|---|---|---|---|---
1 | arc-nlp | 1.0000 | 0.8567 (1) | 0.8563 (1) | 0.8565 (1) | 0.8578 (1)
2 | bayesiano98 | 2.0000 | 0.8561 (2) | 0.8528 (2) | 0.8528 (2) | 0.8533 (2)
3 | karanpreet_singh | 3.0000 | 0.8508 (3) | 0.8476 (3) | 0.8463 (3) | 0.8465 (3)
4 | DeepBlueAI | 4.0000 | 0.8356 (4) | 0.8335 (4) | 0.8342 (4) | 0.8352 (4)
5 | csecudsg | 5.0000 | 0.8252 (5) | 0.8244 (5) | 0.8248 (5) | 0.8262 (5)
6 | Jesus_Armenta | 6.0000 | 0.8121 (6) | 0.8094 (6) | 0.8097 (6) | 0.8104 (6)
7 | Avanthika | 7.0000 | 0.7878 (7) | 0.7881 (7) | 0.7880 (7) | 0.7901 (7)
8 | Sarika22 | 8.0000 | 0.7806 (8) | 0.7849 (8) | 0.7821 (8) | 0.7856 (8)
9 | rabindra.nath | 9.0000 | 0.7768 (9) | 0.7842 (9) | 0.7788 (9) | 0.7833 (9)
10 | md_kashif_20 | 10.0000 | 0.7270 (10) | 0.7372 (10) | 0.7287 (10) | 0.7359 (10)
11 | lueluelue | 11.7500 | 0.5219 (12) | 0.5219 (12) | 0.5219 (11) | 0.5260 (12)
12 | pakapro | 12.7500 | 0.4938 (13) | 0.4939 (13) | 0.4936 (12) | 0.4966 (13)
13 | Sathvika.V.S | 11.5000 | 0.5334 (11) | 0.7240 (11) | 0.4294 (13) | 0.5779 (11)
Table 2: Rank Table (Sub-Task A)

###  5.1 Metric Comparison

The text-based models, including BERT and XLNet, gave an accuracy of 69.65 and
71.80 (respectively) when implemented individually with respect to the given
dataset. On the other hand, the image-based models including InceptionV3,
MobileNetV3, and ResNet 152, gave accuracy levels of 48.12, 42.41, and 44.47
(respectively) for the same set of data. The combination of BERT and XLNet
(without the visual component) gave an accuracy of 73.51.

With regard to this, our proposed ensemble model, developed with InceptionV3,
BERT, and XLNet as its individual blocks, provided promising results with an
accuracy of 75.21 and an F-1 score of 74.96 on the given dataset, as quoted in
Table 1.

Our model outperforms the existing works oriented towards multimodal hate
speech detection, with an overall accuracy of 75.21. For instance, in Das et
al. (2020), the proposed system achieved the best accuracy of 68.4. In
Velioglu and Rose (2020), the proposed model, VisualBERT achieved an accuracy
of 70.93.

The baseline accuracy and F-1 score for the given sub-task are 79.8 and 78.6
(respectively). The proposed model’s performance metrics are comparable to the
median accuracy and F-1 score of 79.01 and 78.8 (respectively). The same has
been mentioned in Table 3.

Model | Accuracy | F1 Score
---|---|---
Baseline | 79.8 | 78.6
Median | 79.01 | 78.8
Proposed | 75.21 | 74.96
Table 3: Metric Comparison for proposed ensemble model with median and
baseline scores

The variation of accuracy level with respect to the number of epochs taken for
model training has been depicted in Figure 2. Along with this, the variation
of the loss function with respect to the number of epochs taken for model
training has been depicted in Figure 3.

Figure 2: Variation of accuracy with respect to epochs for the proposed
ensemble model Figure 3: Variation of loss function with respect to epochs for
the proposed ensemble model

###  5.2 Empirical Analysis

In this section, we provide an empirical analysis of our model’s predictions
for the sample instances of text-embedded images to elaborate on the precision
yielded by the model as per the desired task.

Figure 4 illustrates the samples of text-embedded images corresponding to the
label "Hate Speech", while Figure 5 illustrates the samples of text-embedded
images corresponding to the label "No Hate Speech". An empirical comparison
between the actual labels and the predicted labels for the respective image
instances quoted in Figures 4 and 5 has been mentioned in Table 4.

With respect to the images corresponding to "Hate Speech", it has been
observed for instances 4b and 4d that the labels have been predicted
accurately, suggesting the correct prediction capabilities of the model.
Similarly, as far as the images corresponding to "No Hate Speech" are
concerned, the correct prediction of the labels for instances 5a and 5b
reemphasize the correct working of the model.

On the contrary, for image instance 4a, the model fails to recognize the
implicit attempt of spreading hate speech through visual sarcasm, resulting in
a false prediction by the model. Along with, for image instance 4c, hate
speech has been embedded in visual form, which was incorrectly detected by the
model, leading to another erroneous prediction. This has been precisely due to
the lack of ability of the model to decipher the historical context required
to detect hate speech in the respective images.

| Image
---
Instance
Actual Label | Predicted Label
4a | Hate Speech | No Hate Speech
4b | Hate Speech | Hate Speech
4c | Hate Speech | No Hate Speech
4d | Hate Speech | Hate Speech
5a | No Hate Speech | No Hate Speech
5b | No Hate Speech | No Hate Speech
5c | No Hate Speech | Hate Speech
5d | No Hate Speech | Hate Speech
Table 4: Empirical Evaluation of predictions with respect to sample image
instances

|  |
---|---|---
|  |

(a)

(b)

(c)

(d)

Figure 4: Sample hate speech images

|  |
---|---|---
|  |

(a)

(b)

(c)

(d)

Figure 5: Sample no-hate speech images

As far as the instances in Figure 5 are concerned, although instances 5c and
5d correspond to the label "No Hate Speech", however, the model faced
difficulty in making accurate predictions when confronted with the sarcasm in
the image. The prime reason for the model’s inaccurate prediction of hate
speech in the images is the presence of specific words and phrases that might
seem to cause the same at first sight. For instance, 5c features the words
"explosion" and "kills", while 5d contains the phrase "invasion of Ukraine",
which is believed to have been a major cause for this erroneous prediction.

This suggests that the model may require further training or refinement to
navigate the nuances of language and accurately identify instances of hate
speech more efficiently, even when presented in a lesser straightforward
manner.

##  6 Conclusion and Future Scope

In this paper, we present our system paper submission for Lexical
Squad@Multimodal Hate Speech Event Detection 2023. We aim to classify text-
embedded images, indicating whether they contain hate speech or not. The
proposed system is an ensemble learning model with fine-tuned InceptionV3,
BERT, and XLNet serving as the individual blocks of the proposed model. Given
text-embedded images and their respective extracted text through the OCR
model, the submitted model classifies each image instance into one of the two
labels: "Hate Speech" and "No Hate Speech". The system performs quite well to
accomplish the desired task with an accuracy of 75.21%.

The proposed system can be incorporated for further applications including
recommendation systems, personalized content viewing, etc. Along with, it can
find usage in further research studies centered on the overlooking field of
interest.

In the future, we intend to work on a multitask learning framework to handle
social media postings related to other concerns pertaining to sentiment
analysis, apart from Hate Speech detection. We also aim to develop models for
multi-lingual postings featuring similar scenarios.

## References

  * Abro et al. (2020) Sindhu Abro, Sarang Shaikh, Zahid Hussain Khand, Ali Zafar, Sajid Khan, and Ghulam Mujtaba. 2020.  Automatic hate speech detection using machine learning: A comparative study.  _International Journal of Advanced Computer Science and Applications_ , 11(8).
  * Agarap (2018) Abien Fred Agarap. 2018.  Deep learning using rectified linear units (relu).  _arXiv preprint arXiv:1803.08375_.
  * Badjatiya et al. (2017) Pinkesh Badjatiya, Shashank Gupta, Manish Gupta, and Vasudeva Varma. 2017.  Deep learning for hate speech detection in tweets.  In _Proceedings of the 26th international conference on World Wide Web companion_ , pages 759–760.
  * Bhandari et al. (2023) Aashish Bhandari, Siddhant B Shah, Surendrabikram Thapa, Usman Naseem, and Mehwish Nasim. 2023.  Crisishatemm: Multimodal analysis of directed and undirected hate speech in text-embedded images from russia-ukraine conflict.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 1993–2002.
  * Carlson (2021) Caitlin Ring Carlson. 2021.  _Hate speech_.  MIT Press.
  * Dai et al. (2019) Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V Le, and Ruslan Salakhutdinov. 2019.  Transformer-xl: Attentive language models beyond a fixed-length context.  _arXiv preprint arXiv:1901.02860_.
  * Das et al. (2020) Abhishek Das, Japsimar Singh Wahi, and Siyao Li. 2020.  Detecting hate speech in multi-modal memes.  _arXiv preprint arXiv:2012.14891_.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_.
  * Džeroski and Ženko (2004) Saso Džeroski and Bernard Ženko. 2004.  Is combining classifiers with stacking better than selecting the best one?  _Machine learning_ , 54:255–273.
  * Ganaie et al. (2022) Mudasir A Ganaie, Minghui Hu, AK Malik, M Tanveer, and PN Suganthan. 2022.  Ensemble deep learning: A review.  _Engineering Applications of Artificial Intelligence_ , 115:105151.
  * Gomez et al. (2020) Raul Gomez, Jaume Gibert, Lluis Gomez, and Dimosthenis Karatzas. 2020.  Exploring hate speech detection in multimodal publications.  In _Proceedings of the IEEE/CVF winter conference on applications of computer vision_ , pages 1470–1478.
  * Parihar et al. (2021) Anil Singh Parihar, Surendrabikram Thapa, and Sushruti Mishra. 2021.  Hate speech detection using natural language processing: Applications and challenges.  In _2021 5th International Conference on Trends in Electronics and Informatics (ICOEI)_ , pages 1302–1308. IEEE.
  * Phan and Yamamoto (2020) Trong Huy Phan and Kazuma Yamamoto. 2020.  Resolving class imbalance in object detection with weighted cross entropy losses.  _arXiv preprint arXiv:2006.01413_.
  * Schmidt and Wiegand (2017) Anna Schmidt and Michael Wiegand. 2017.  A survey on hate speech detection using natural language processing.  In _Proceedings of the fifth international workshop on natural language processing for social media_ , pages 1–10.
  * Sesmero et al. (2015) M Paz Sesmero, Agapito I Ledezma, and Araceli Sanchis. 2015.  Generating ensembles of heterogeneous classifiers using stacked generalization.  _Wiley interdisciplinary reviews: data mining and knowledge discovery_ , 5(1):21–34.
  * Shaheen et al. (2020) Zein Shaheen, Gerhard Wohlgenannt, and Erwin Filtz. 2020.  Large scale legal text classification using transformer models.  _arXiv preprint arXiv:2010.12871_.
  * Smith (2007) Ray Smith. 2007.  An overview of the tesseract ocr engine.  In _Ninth international conference on document analysis and recognition (ICDAR 2007)_ , volume 2, pages 629–633. IEEE.
  * Szegedy et al. (2015) Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015.  Going deeper with convolutions.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 1–9.
  * Szegedy et al. (2016) Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016.  Rethinking the inception architecture for computer vision.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 2818–2826.
  * Thapa et al. (2023) Surendrabikram Thapa, Farhan Ahmad Jafri, Ali Hürriyetoğlu, Francielle Vargas, Roy Ka-Wei Lee, and Usman Naseem. 2023.  Multimodal hate speech event detection - shared task 4, case 2023.  In _Proceedings of the 6th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE)_.
  * Thapa et al. (2022) Surendrabikram Thapa, Aditya Shah, Farhan Jafri, Usman Naseem, and Imran Razzak. 2022.  A multi-modal dataset for hate speech detection on social media: Case-study of russia-Ukraine conflict.  In _Proceedings of the 5th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE)_ , pages 1–6, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
  * Velioglu and Rose (2020) Riza Velioglu and Jewgeni Rose. 2020.  Detecting hate speech in memes using multimodal deep learning approaches: Prize-winning solution to hateful memes challenge.  _arXiv preprint arXiv:2012.12975_.
  * Yang et al. (2019) Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019.  Xlnet: Generalized autoregressive pretraining for language understanding.  _Advances in neural information processing systems_ , 32.
  * Zhang (2018) Zijun Zhang. 2018.  Improved adam optimizer for deep neural networks.  In _2018 IEEE/ACM 26th international symposium on quality of service (IWQoS)_ , pages 1–2. Ieee.
  * Zohair et al. (2022) Mohammad Zohair, Nidhir Bhavsar, Aakash Bhatnagar, and Muskaan Singh. 2022.  Innovators@ smm4h’22: An ensembles approach for self-reporting of covid-19 vaccination status tweets.  In _Proceedings of The Seventh Workshop on Social Media Mining for Health Applications, Workshop & Shared Task_, pages 123–125.


# BIOPAK FLASHER: Epidemic Disease Monitoring and detection in Pakistan using
Text Mining

Muhammad Nasir
Department of CS and IT
University of Balochistan
<EMAIL_ADDRESS>
Maheen Bakhtyar
Department of CS and IT
University of Balochistan
<EMAIL_ADDRESS>
Junaid Baber
Department of CS and IT
University of Balochistan
<EMAIL_ADDRESS>
Sadia Lakho
Department of CS and IT
SBK Women’s University Balochistan
<EMAIL_ADDRESS>
Bilal Ahmed
Department of CS and IT
University of Balochistan
<EMAIL_ADDRESS>
Waheed Noor
Institute of Management Sciences
University of Balochistan
<EMAIL_ADDRESS>
Use footnote for providing further information about author (webpage,
alternative address)— _not_ for acknowledging funding agencies. Muhammad
Nasir1 Maheen Bakhtyar1 Junaid Baber1 Sadia Lakho1 Bilal Ahmed2 Waheed Noor3

###### Abstract

Infectious disease outbreak has a significant impact on morbidity, mortality
and can cause economic instability of many countries. As global trade is
growing, goods and individuals are expected to travel across the border, an
infected epidemic area carrier can pose a great danger to his hostile. If a
disease outbreak is recognized promptly, then commercial products and
travelers (traders/visitors) will be effectively vaccinated, and therefore the
disease stopped. Early detection of outbreaks plays an important role here,
and beware of the rapid implementation of control measures by citizens, public
health organizations, and government. Many indicators have valuable
information, such as online news sources (RSS) and social media sources
(Twitter, Facebook) that can be used, but are unstructured and bulky, to
extract information about disease outbreaks. Few early warning outbreak
systems exist with some limitation of linguistic (Urdu) and covering areas
(Pakistan). In Pakistan, few channels are published the outbreak news in Urdu
or English. The aim is to procure information from Pakistan’s English and Urdu
news channels and then investigate process, integrate, and visualize the
disease epidemic. Urdu ontology is not existed before to match extracted
diseases, so we also build that ontology of disease.

_Keywords_ Pakistan epidemic coverage $\cdot$ Extraction information in Urdu
$\cdot$ RSS $\cdot$ NLP.

##  1 Introduction

In past many infectious diseases hampered humankind, like the Whooping cough
outbreak in U.S. during the summer of 2012, Periodic outbreaks of H7N9 in
China during 2013-2014 and emerging MERS outbreak in Saudi Arabia, Severe
Acute Respiratory Syndrome (SARS), West Nile Virus and drug resistant
tuberculosis Zhao et al. (2012). Diseases are spread exponentially within a
population. Peoples and products travel across the border in high volume so
that an infection can rapidly attain a global scale.
A pandemic HiNi (“Swine Flu”) appeared in 2009 Szomszor et al. (2011). To
increase this outbreak, a globally open online web-based framework gave
warning to everyone concerned. The disease was catered for in response to this
warning, and public understanding of emerging infection has increased. Online
web-based early warning system’s traffic rate was increased about 100 % in 2-3
days. It gives birth to online early warning globally infection awareness.
Hence, globally feels that there must be a system that alerts the early
warning infectious disease timely vaccinate and cure the victims. Presently,
nearly all major outbreaks, investigated by the World Health Organization
(WHO) are first identified through these informal online sources.

The emergence of ubiquitous internet service and build up different internet-
based resources like blogs, online newspapers (Google News), social media
(Twitter), and discussion forums shape the world dynamic fast and very
efficiently. Such services are highly useful in helping to diagnose, assess,
incorporate and forecast outbreaks of infectious diseases or diseases
spreading to new regions. Intelligent modeling of such sources using text-
mining methods such as topic models, deep learning and dependency parsing can
lead to automated generation of the mentioned surveillance tools..
The above resources produced big data at very high speed, and the quantity of
that data is increasing day by day. Despite big data problem, these resources
have the unstructured type of data, which is difficult to process, and extract
useful information. Reading and assimilating a large number of reports daily
basis is a hectic and burdensome task. .
. With BIO PAK FLASHER, a web-based solution focusing primarily on URDU Jabbar
et al. (2016), Pakistani Urdu news outlets, and covering Pakistani rulers and
urban areas, we plan to solve these challenges with the automated framework
for extraction, analysis, query, screening, incorporation, and visualization
of unstructured and bulky disease outbreaks. Our system collects outbreak data
from many sources such as news through the Google News aggregator, Pakistani
local news channels published in Urdu, expert-curetted accounts such as ProMED
Morse et al. (1996) Mail, and validated official alerts such as World Health
Organization announcements. By using the text processing algorithms, the
system classifies outbreak in location and disease, after that overlay such
data on an interactive geographic map. We measure the accuracy of the
classification algorithms based on the level of human curation necessary to
correct misclassification and examine geographic coverage. With state-of-the-
art NLP technology, there is a need to compare all Urdu terms with a disease
database called ontology to figure out the disease when the sentence is
tokenized and all POS deleted. There is no disease ontology Schriml et al.
(2012) exist before to do such work. Here we also build Urdu-based Ontology
with different patterns for further reference. .
Google Maps API consists of a set of classes within a JavaScript container
that opens within an XHTML page. Each time the Google Maps web page opened,
these classes are loaded from Google, and it has embedded JavaScript object on
which all the functionality of the Google Maps is based. To visualize
outbreaks, we use Google Maps.BioPak Flasher has a user- friendly interface
and providing important disease outbreak information with text processing
algorithms and also referred user to view full article with web link
integration. The rest of the paper is composed of Section 2 Literature review
presents the background of infectious disease outbreaks and basic
terminologies used in this research. Section 3 is about the Methodology used
in this research, Section 4 Describes the implementation and Analysis of the
proposed methodology. Section 5 presents the conclusion and future work for
this research.

##  2 Literature Review

There are few traditional disease surveillance systems like Centers for
Disease Control (CDC) Berros-Torres et al. (2017), ProMED Morse et al. (1996),
which released data of infectious disease outbreaks with delay due to their
implemented setup (take news mail, get expert opinion then release outbreak
announcement). In the modern period, the epidemic must be published in real-
time. To cater to evolving diseases, take control steps quickly, and avoid
them from spreading further, an early warning/detection of the outbreak system
is important. These surveillance systems have some limitations. These programs
do not protect all languages, so the epidemic news reported in state and local
news would be missed. The system must be capable to process these languages so
that any news published in theses languages will also be covered to give
authenticated real-time system. Urdu is the national language of Pakistan and
speaks in many other countries as well in non-official manners. Urdu is also
not covered by these systems. Many local publishers like VOA-Urdu, Geo,
Express, Jang, etc. published their news in Urdu. Moreover, these systems have
some geographical limitations. These systems are not covering the ruler/ urban
areas of some countries, especially Pakistan.
Infectious disease is increasing day by day due to unhygienic environment,
people are in close contact with livestock or wildlife and chemicals, etc.
Infectious diseases lead to mortality so awareness is a very essential part of
the current era. Delayed detection of the outbreak may cause some mortality
and may be created hinders in controlling and eradication process. Such
mortality rate can be minimized if timely recognized. If we know in real-
time, then the control and eradication measures also taken time to reduce
mortality rate.
Moreover, when we want to visit somewhere or required to do trade with some
area, we must consider emerging pathogens that causes the epidemic. Shifting
of emerging product or human can cause the spreading of that epidemic in the
hosting area the traditional outbreak system has a manual alarm that often
requires a delay, but our, alert system, swift, and covering Pakistani news is
our disease outbreak system written in Urdu. This system also helps in general
awareness of infectious outbreaks and its remedies measures timely.

###  2.1 Epidemic Breakdowns In Pakistan

A government of Pakistan Official Body releases the statistics in 44th dated
Mar 2019 to June 2019. According to these, Figure 1 alerts has been generated
in the country, Figure 2 shows the statistical results of diseases outbreaks
in Pakistan.

####  2.1.1 Epidemic (A Communicate-able Disease)

An epidemic Newman (2002) occurs when an infectious disease spreads rapidly in
contact with too many people. An epidemic is an outbreak of a disease among
members of a specific population that exceeds the extent of occurrence of the
disease normally found in that population. Epidemics affect those members of
the population who do not have an acquired or inherent immunity to the
disease. Although infectious organisms cause most epidemics, the term can be
applied to an outbreak of any chronic disease, such as lung cancer or heart
disease. For example, in 2003, the severe acute respiratory syndrome (SARS)
epidemic took the lives of nearly 800 people worldwide. Statistics and big
outbreaks in history with casualty.

####  2.1.2 RSS Feed:

RSS stands for "Really Simple Syndication". It is a way to easily distribute a
list of headlines, update notices, and sometimes content to a wide number of
people Johnson et al. (2009). Computer programs that organize those headlines
and notices for easy reading use it. Most people are interested in many
websites whose content changes on an unpredictable schedule. News sites,
community, and religious group information pages, product information pages,
medical websites, and weblogs are examples of such websites. Repeatedly
checking each website to see if there is any new content can be very tedious.

Figure 1: Recent tutbreaks in Pakistan. Figure 2: Statistics of different
disease outbreak in Pakistan.

####  2.1.3 How It Works:

RSS works by having the website author maintain a list of notifications on
their web-site in a standard way. This list of notifications is called an "RSS
Feed". People who are interested in finding out the latest headlines or
changes can check this list. Special computer programs called "RSS
aggregators" have been developed that automatical-ly access the RSS feeds of
websites you care about on your behalf and organize the results for you. (RSS
feeds and aggregators are sometimes called "RSS Channels" and "RSS Readers".)

####  2.1.4 RSS Feed Structure:

An XML File provides very basic information to do its notification. It is made
up of a list of items presented in order from newest to oldest. Each item
usually consists of a simple title describing the item along with a more
complete description and a link to a web page with the actual information
being described. Sometimes this description is the full information you want
to read (such as the content of a weblog post) and sometimes it is just a
summary.

##  3 Methodology

The different aspects of this research including data collection were the
compulsory requirement for further utilization of the data. The essentials
tasks are explained figure 3.
In this figure, we presented the layer view of our model. We preferred to work
in layer because it is easy to wok easily if we have to do any changing. We
divide our model into four layers; each layer is dependent on its precedence
layer. Each layer work is defined clearly. The first layer is the Data
acquisition layer, which retrieves data from the RSS, feed and puts it into
the database for further processing. The second layer is the Data filtering
and preprocessing layer in which we eliminate and filter out unnecessary
diacritics and punctuations then performing some NLP (Natu-ral Language
Processing) techniques like tokenization, stemming, and stop words removal of
Urdu language. Then our data has epidemic, city, date, and some garbage or
unnecessary data. The third layer of Event Recognition is designed to find out
the epidemic disease and concerned area with latitude and longitude. Our last
layer is the Visualization layer, which deals with visualization of epidemic
outbreak in the con-cerned area will show on Google map with marker view and
store events in database for displaying it next 90 days. User can also view
source evidence by simply clicking on a pinpoint map as well as associated
headlines for the event.

Figure 3: Abstract overview of the BioPak flasher.

###  3.1 Data Acquisition and Storage Layer:

In this layer. We acquire the data from Pakistani news channels and newspapers
that are offering online resources (RSS feed) in Urdu and store these data
into a database for further processing. Many online actors can provide news
like RSS, tweeter, LinkedIn, Facebook, etc. However, our choice is RSS feed.
RSS stand for Really Simple Syndication or sometimes refer as Rich Site
Summary. Actually, this technol-ogy was introduced to avoid bulky and
uninterested news, which is being pushed by a provider. This is also used to
keep updated user with upcoming news, blogs, financial information, health,
sports, classified sites, and government alerts. RSS feed is written in XML
code which is easy to process by computer. It is a structured data type.
Figure 4 shows a sample of newsreader who acquired the news from Voice of
America in Urdu. We can select any topic in which we are interested, in our
case health topic is most prominent and maintained by almost every RSS feed
provider. So we subscribe to the health RSS feed. When we click on any news
feed, it will refer us to its description web page shows in Figure 5, where we
can see all detail of the news. In this figure, we refer to the voice of
America page where we can find the detail of said news.

Figure 4: Sample overview of RSS feeds. Figure 5: Sample screenshot of the
News.

Our focus of research covered Pakistan and its national language Urdu Pakistan
has several outlets that include printed media, news channels, several online
podcasts, and internet-based TV channels used for consumer information. These
all channels are published in different languages according to a location like
English, Urdu, Sindhi, etc. Therefore, Urdu RSS feed newspaper or channels are
our main actor for getting information. However, very few channels are
maintained their RSS feed in Urdu. Because our research is focused on the RSS
mechanisms so that we only focus that news, channels, which have their RSS,
feed in Urdu and store it.

####  3.1.1 Data filtering and Pre-Processing:

There are many types of data which is published in the RSS feed. Input news
items are not considered to be cleaned so making them ready for further
processing is im-portant. Raw data, if obtained from the web normally includes
HTML and or XML tags so they should be removed for further processing. In
addition, text format (utf-8) for special characters is important so that no
error occurs while processing.

####  3.1.2 Removal of Diacritics:

The Urdu language is using diacritics to clarify the pronunciation of Urdu
words. Commonly used diacritics are Zabar, Zer, Pesh, Jazam, and shad, etc.
Our raw data received from the database has many unnecessary elements that are
required to be filtered shows in figure 6. In this stage of filtering, it is
necessary that data must be filtered from diacritics. If it is not done then
in processing, different result will be seen and our final output is very
different from the actual.

####  3.1.3 Removal of Punctuation

Urdu language sentences have different punctuations, which used in the
construction of sentences. These punctuations are undesired in our processing
so that we removed these. Some punctuations which mostly used in the Urdu
language are “ “ ” ‘’ – [](),.;: “

####  3.1.4 Data Preparation (Tokenization with NLTK Library)

Urdu tokenization is not easy as English due to its structure. However,
Unicode recommends using Zero Width Non-Joiner character in this context (.,
4) There would be an alignment of tags with the terms present at the simple
stage, so tokenization is also an essential aspect of sorting. The text will
be tokenized into sentences and phrases into words in the process.

Tokenization is a very common feature of NLP. Tokenization is a process to
break a text string into single entities that may include words, symbols,
phrases, etc. In our case, we apply this technique to divide the string into
words, e.g.

Further processing is taken place on the resulted array after tokenization.

####  3.1.5 Building up of Urdu Stop Words (USW) Dictionary

After tokenization, now many unwanted words have no meaning when we tokenize
but these words have more worth when used in combination and clear the concept
of sentences. Unfortunately, there is not much work done in Urdu, so we used
stop word dictionary Stop Words Removal (Based on USW Dictionary) These
dictionaries are used as a reference for the removal of stop words. This is
another technique of NLP, which are tags, or separates various parts of speech
in the text. For filtering, stop words are also attached with this technique
to filter indifferent data. Epidemic disease, city, and date of occurrences
are needed to be extracted from the text string. Everything else in the string
is irrelevant and should not be extracted or in other words, should be deleted
from the string. Tokenization has been done in the previous step. Keeping in
view the resulted string, now the irrelevant data should be filtered out from
the string. At this stage, we have the tokenized string as:

After deletion of Stop word from string, we will be left with the following
array:

Now the leftover data included only disease, city, and date or in some cases
some garbage or irrelevant data.

####  3.1.6 Data Correction (Data Stemming on Tokenize word to their basic
form)

It is also known as lemmatization in which the word is changed to its very
basic form by removing suffixes and will be converted to its root form. This
makes the handling of words very easily. The significance of stem is very well
explained e.g. may be written as  Failed to do so sometimes create problems in
processing which ultimately affects our result.

####  3.1.7 Data Correction (Different Format of Single Word)

Urdu is a versatile language and words can be written in different styles.
Like  to ,  to  in this case our dictionary is unable to find these types of
word and failed to produce results. Then we refer these cases to our format
pattern dictionary where we describe the different patterns of the same word.
If a word is not recognized by “Urdu based epidemic and city” dictionaries
then we will move to this dictionary for finding the diversion of said word.
This dictionary is flexible and regularly updated on the feed-back and custom
detection by users. If we checked every word in this dictionary and found that
maximum words exist in their standard shape, then it is useless and resource-
consuming that we checked every word for its reformatting. This dictionary is
optional.

###  3.2 Event Recognition Layer

After necessary filtering and preprocessing, we have now reduced the array of
string. In this layer, we recognize the epidemic and also its occurrence city
with the date stamp. This layer is also eliminating duplicate news from
different channels and newspapers.

####  3.2.1 Epidemic Directory / Ontology in Urdu (Creation and updating of
said Directory / Ontology on regular basis)

To find out disease from the preprocessed array, we must have a dictionary or
data-base from where we can match and find the result. These types of
dictionaries are available in English but in the Urdu language, it does not
exist. We prepared URDU Directory / Ontology Dictionary which contains
initially approx. 50 diseas-es/Pathogens/ Infections with its English version.

####  3.2.2 Event Recognition (Epidemic)

In this stage, we extracted the disease from the news feed in connection with
epidem-ic disease ontology. There may be some confusion regarding disease
nomenclature like  or  or fever, we removed this confusion with us reformat
dictionary. We made them in simple and basic level, so that the exact disease
is to be identified.

####  3.2.3 City Database in Urdu (Enlisted the all cities of Pakistan in
URDU)

Pakistan has 374 cities. There is no database exist which describe the
Pakistan cities names in Urdu, so for our purpose of city recognition, we will
make a database. This database/ dictionary will use to detect the city of an
epidemic outbreak. Urdu pub-lishers are not always following standard patterns
and published their news in very random ways, sometimes the name of the city
is written as  and sometimes  or sometimes “Burewala”. This versatility in
city names creates confusion for the program to find out the exact city. We
will make our dictionary/ database in such a way that caters all these
problems. We put different patterns of a city in Urdu and English also. So,
when querying this dictionary, it will return us the basic and standard format
of the city. We will also make this database relaxant so that names of cities
or their different formats may include later on. After determining the
disease, it is neces-sary to find out the epidemic area so that we can
visualize it on a map.

####  3.2.4 Entity Recognition (City) (Find City of concerned Epidemic)

In the light of the city named dictionary, we extract city names from the RSS
title. If the city is not identified in the RSS feed title, then we moved on
to the description to find out the city. There may be variants of city names
published in RSS feed that caters as described above. When we query the city
database it will return our city name in that format which is recognized by
our latitude and longitude stage.

####  3.2.5 City Lat-Long Database in Urdu (Enlisted Latitude and Longitude of
Pakistan cities in Urdu)

The world is divided into latitude and longitude. Latitude and Longitude are
the units that represent the coordinates at the geographic coordinate system.
To pinpoint any location or refer to this on the map, we must know the
latitude and longitude of said area. The map only understands the latitude and
longitude. So that to visualize the city on the map we must know the latitude
and longitude of said city. Before this, there is no database that describes
the latitude and longitude of Pakistan cities in Urdu. We prepared a database
that will return the latitude and longitude of each city of Pakistan when
inquired. These are the fixed parameters and not changed with the passage of
time so that we make it robust.

####  3.2.6 Recognition of City Lat-Long (Find Latitude and Longitude of
Concerned City)

After identification of city, we will extract its latitude and longitude from
the Lat-Long database and will store this information in a database for
further processing of JSON to visualize the epidemic on the map.

###  3.3 Visualization Layer (JSON)

In this layer, we visualize the epidemic outbreak Newman (2002) with the
concerned area and also used the cluster marker to simplify the map vision
which removed the so many pins clutter and combined the same area with a
marker. Epidemic disease, city, city lat-long, date, and link are stored in a
dedicated separate database.

####  3.3.1 Database Storage (All IDs, Epidemic, City, Lat-Long stored in
Database with spe-cific ID)

After extracting all necessary information regarding the disease, city, lat-
long, date, link and system generated ID will be stored in a dedicated
database. This infor-mation is accessed by JSON and will be used to view on
map, we will set this data-base frequency up to 90 days which will be reduced
on user-customized query.

####  3.3.2 Web Site Backend (Fetch data from DB with respect to Location/
Disease)

Website required certain types of data to be used on a map. These data
included the epidemic disease and area with lat long. The Database is store in
the list and is used in the visualization.

####  3.3.3 Web site Backend Fetching Types:

  * •

Fetch data from DB with respect to Location/ Disease

  * •

Fetch customized data as per query: database can be customized as per user
requirements.

####  3.3.4 Visualization (In term of pathogen, syndrome, or text type)

For visualization purpose, we have used the JSON. First, we module our data
into JSON pattern and then this data are used by map to visualize. Wherever
the disease outbreak occurred, a marker is placed there and a short note will
appear there.

####  3.3.5 Marker Clustering Google Map Marker Clustering (For more clarity
in visuali-zation)

If any place has many markers Netek et al. (2019) then these markers are
combined and displayed as a single marker, which simplified the view.

####  3.3.6 Source Evidence (URL will open detail for that epidemic)

Source evidence is a feature to facilitate the user with full detail of the
news. If some-one wants to know further details of the concerned epidemic,
then it will just click and web refers him to the concerned page for more
detail. Different colors are used to make it more comprehensive for
visualization.

##  4 Experiments & Results

Our developed model has four layers and each layer has different stages. Here
we describe the output of each stage.

###  4.1 RSS Feed Output:

We used almost all available resources that are being offered the Urdu RSS
Feed. Here we only see the results of SUCHTV news. We automate it to run after
the calendar day has been changed. SUCHTV news almost issued 10-20 RSS feed
daily when the calendar date changed. We also tune our system so that it will
not miss any feed. If we want to run manually or mistakenly run the developed
system then it captured the RSS feed but did not put it in the database
because there is already that feed available with access No control. Figure 7
shows the RSS feed which is being captured by our system.

Figure 6: SACH TV News RSS Feed.

###  4.2 RSS Feed in Database

Here we captured the title, date of news which is being provided by RSS
provider (SUCHTV news), description, access No, and flag. The flag is placed
here to lessen the burden of the system. When our system placed the new RSS
feed in the database then the flag is set to “1”. But when the system
processed that news and found the results then it assigned it “0” so that next
time when the system approaches the data-base then only a new RSS feed will be
processed, hence we less the burden of the system. Figure 8 shows the RSS
feeds are placed in the database.

Figure 7: RSS Feed stored in Database.

###  4.3 Epidemic Outbreak Detection

The accuracy of our system is depending on the detection of epidemic diseases
from every RSS feed. For this purpose, we maintained and directory of epidemic
disease as reference because Urdu based ontology is not exist before. Figures
9 shows the detected disease from RSS feed at system run time, figure 10 show
the summary of detected disease with their database ID for easy tracking.

Figure 8: Run time Epidemic Detection. Figure 9: Summary of detected disease
with database ID.

###  4.4 City Detection and city latitude City Latitude and Longitude
Detection:

Figure 11 shows the latitude and longitude of the detected city.

Figure 10: Detection of Latitude and Longitude of Detected City.

###  4.5 JSON Data for Visualization

To give geolocation parameters to Google Map, We convert our data in JSON so
that map can take latitude and longitude from there. Figure 12 shows the JSON
file with all detected disease, their location name, latitude, longitude, and
Data-base ID for easy tracking and source verification.

Figure 11: JSON File.

###  4.6 Google Map Visualization

Finally, all the epidemic outbreaks and locations are shown in the Google Map
with marker feature which is used to give more clarity on a map, sometime also
refers as clustering, when many diseases is spread in nearby areas then
clustering/ marker feature marked that area with a single marker and shows
numeric to give more clarity shows in figure 13.

Figure 12: Google Map with Clustering Marker.

###  4.7 Removal of Diacritics:

Not all the RSS feed has diacritics but very few, So identify it with Expert
Identi-fication and figure it out that 55 RSS feeds which have diacritics. Now
we analy-sis our system performance with Expert Identification, and In Table
1, we can see how many accuracies, precision, recall and F-Score will obtain

Table 1: Performance Evaluation of Diacritics Removal Removal of Diacritics (Confusion Matrix) |  |  |
---|---|---|---
Expert Identification | BIOPAK Flasher | Total |
| Removed | Not Removed |  |
$-$Removed |  $-$55 (TP) |  $-$0(FN) |  $-$55 |
Not Removed |  $-$0(FP) |  $-$0(TN) |  $-$0 |
$-$Total |  $-$55 | 2 |  $-$55 |

###  4.8 Removal of Punctuations:

Every sentence ends up with (Full stop). Others are used in sentences. It is
too much difficult and vague to quantify this performance test on the whole
existing database, for this purpose we choose 50 RSS feeds that have (90)
punctuations. Now we analysis our system performance with Expert
Identification, and we can see in Table 2 that how many accuracy, precision,
recall, and F-Score will obtain.

Table 2: Performance Evaluation of Punctuation Removal Removal of Punctuation (Confusion Matrix) |  |  |
---|---|---|---
Expert Identification | BIOPAK Flasher | Total |
| Removed | Not Removed |  |
$-$Removed |  $-$88 (TP) |  $-$2(FN) |  $-$90 |
Not Removed |  $-$0(FP) |  $-$0(TN) |  $-$0 |
$-$Total |  $-$88 | 2 |  $-$90 |

###  4.9 Tokenization Correctness:

A very important step in NLP Mullen et al. (2018), we have 301 RSS Feed for
evaluation purposes. Now we analysis our system performance with Expert
Identification, and see how much accuracies, precision, recall, and F-Score
will obtain shows in Table 3.

Table 3: Tokenization Correctness (Confusion Matrix) Removal of Diacritics (Confusion Matrix) |  |  |
---|---|---|---
Expert Identification | BIOPAK Flasher | Total |
| Tokenize | Not Tokenize |  |
$-$Tokenize |  $-$273 (TP) |  $-$16(FN) |  $-$289 |
Not Tokenize |  $-$11(FP) |  $-$1(TN) |  $-$12 |
$-$Total |  $-$284 | 17 |  $-$301 |

###  4.10 Overall System Accuracy

We conclude all the results and find the overall performance of our system.
Starting from first layer shows in Table 4:

Table 4: In terms of overall performance of BIOPAK Flasher, it performed above 80% for almost all stages. BIOPAK Flasher Stages | Accuracy | Precision | Recall | F- Score
---|---|---|---|---
Removal of Diacritics | 100 % | 100 % | 100 % | 100 %
Removal of Punctuation | 97.8% | 100% | 97.8% | 98.9 %
Tokenization Correctness | 91% | 96.1% | 94.5% | 95.3%
Stop Word Removal | 89.6% | 92.6% | 89.8% | 91.2%
Epidemic outbreak detection | 93.9% | 97.4% | 95.7% | 96.5%
City detection | 96.4% | 97.8% | 92.3% | 98.1%
City Lat Long Detection | 99.4% | 100% | 99.4% | 99.7%
Overall BIOPAK Flasher | 95.4 % | 97.7% | 95.6% | 97.1%

Pathogen Lazcka et al. (2007) is the main cause of the epidemic outbreaks.
These pathogens are required objects to survive and can spread from human to
human. These pathogens can also spread through trade. It is necessary to stop
these types of the pathogen to spread more and confined them. A carrier of
such pathogen may cause sources of spread for others if travel other places.
To escape and confined this outbreak, it is necessary to stop other people to
visit that place or stop trade with that place, howev-er, if necessary, to
continue relations then proper vaccination will be carried out.

###  4.11 Conclusion

Our research points out the areas where the epidemic outbreak has occurred
using Urdu language. To know such type of information, many actors are playing
a vital role e.g., RSS Feed, Twitter, Social media, Printed Media, e-paper and
News channels, etc. These resources are available in printed form (hard copy)
or the e-electronic form (soft copy) in English and Urdu. To get news of the
epidemic outbreaks, we choose the RSS feed in Urdu as a resource.
Urdu language is ignored and not used for epidemic detection. Many web sites
are working, but most of the sites are in English. Whereas, Urdu is widely
spoken language in Asia and also the national language of Pakistan. Majority
people can better understand if the information is spread in Urdu. Very few
media groups are maintained their RSS feeds in Urdu, it may be increased with
the passage of time.
Our proposed model is using state of the art technologies of NLP for the ex-
traction of data from these feeds. To implement this, we used the different
NLP li-braries. Some libraries have good support for Urdu but some have less
support, which creates a problem for us. There is no full matured library is
available for the NLP tasks on Urdu language. Every library has basic level
support for Urdu, restriction of these libraries lies on the minimum level of
Urdu data corpus for training different models. However, we extracted the
epidemic diseases and cities from these RSS feed with the help of these
libraries and some custom developed algorithm.
We divided our model into four layers, so that maintenance, troubleshoot-ing,
and updating is so easy, we can add, modify or delete any layer. The first
layer is used to acquire RSS feed from RSS provider and saved it into the
database. RSS feed duplication from same or different sources removed. In the
second layer, we performed NLP techniques to filter the data, we take a single
RSS feed’s title, removed diacritics, and punctuations in the very first
stage. After these basic filtering, we to-kenized the data and removed stop
words. At the end of the second layer, we have filtered unnecessary data, now
it is easy to perform the next steps with great perfor-mance. Our third layer
is used to extract epidemic diseases and their area of occur-rences. For this
purpose, we developed three customized dictionaries with the en-hanced
features of correction, and updating. These dictionaries have epidemic dis-
ease ontology, Pakistani cities in Urdu, and their geo locations. These
dictionaries are used as a reference, if any epidemic pathogen is detected
after deployment of our system, then it is very easy to include this pathogen
in our database. On detection of epidemic disease, our system then detect city
from feed, if city is not found here then move our pointer to description
portion for finding occurrence area. After finding the city, it is necessary
to get its geo location which is required by Google Map to display occurrence
areas. In the fourth layer, we visualized the disease on Google Map. The Map
will be too messy if we display all the information there, to clutter down and
give better visualization, we used the marker technique. In this layer, we
also stored detected disease, city, and geo location in a database for
displaying it up to 90 days. W̌e checked the performance of each layer with
accuracy, precision, recall, and F-score. The overall system accuracy is
95.4%, Precision 97.7%, Recall 95.6% and F-score is 97.1%. In terms of the
performance of BIO PAK Flasher, it performed above 80% for almost all stages.
We also give a comparison of different tools and libraries, which are
available to perform said task. BIOPAK Flasher also gives re-source evidence,
if any viewer is required to see the full news, our system can direct him to
the concerned web page for a detail study. Our research gives a breakthrough
in the field of Urdu language when working with online actors. However, more
work and efforts are needed to mature Urdu NLP.

## References

  * Zhao et al. [2012] Yanlin Zhao, Shaofa Xu, Lixia Wang, Daniel P Chin, Shengfen Wang, Guanglu Jiang, Hui Xia, Yang Zhou, Qiang Li, Xichao Ou, et al.  National survey of drug-resistant tuberculosis in china.  _New England Journal of Medicine_ , 366(23):2161–2170, 2012.
  * Szomszor et al. [2011] Martin Szomszor, Patty Kostkova, and Connie St Louis.  Twitter informatics: tracking and understanding public reaction during the 2009 swine flu pandemic.  In _2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology_ , volume 1, pages 320–323. IEEE, 2011.
  * Jabbar et al. [2016] Abdul Jabbar, Sajid Iqbal, and Muhammad Usman Ghani Khan.  Analysis and development of resources for urdu text stemming.  _Language and Technology_ , 1, 2016.
  * Morse et al. [1996] Stephen S Morse, Barbara Hatch Rosenberg, Jack Woodall, and ProMED Steering Committee Drafting Subgroup.  Promed global monitoring of emerging diseases: design for a demonstration program.  _Health Policy_ , 38(3):135–153, 1996.
  * Schriml et al. [2012] Lynn Marie Schriml, Cesar Arze, Suvarna Nadendla, Yu-Wei Wayne Chang, Mark Mazaitis, Victor Felix, Gang Feng, and Warren Alden Kibbe.  Disease ontology: a backbone for disease semantic integration.  _Nucleic acids research_ , 40(D1):D940–D946, 2012\.
  * Berros-Torres et al. [2017] Sandra I Berros-Torres, Craig A Umscheid, Dale W Bratzler, Brian Leas, Erin C Stone, Rachel R Kelz, Caroline E Reinke, Sherry Morgan, Joseph S Solomkin, John E Mazuski, et al.  Centers for disease control and prevention guideline for the prevention of surgical site infection, 2017.  _JAMA surgery_ , 152(8):784–791, 2017.
  * Newman [2002] Mark EJ Newman.  Spread of epidemic disease on networks.  _Physical review E_ , 66(1):016128, 2002.
  * Johnson et al. [2009] Stephen M Johnson, Andrew Osmond, and Rebecca J Holz.  Developing a current awareness service using really simple syndication (rss).  _Journal of the Medical Library Association: JMLA_ , 97(1):52, 2009.
  * Netek et al. [2019] Rostislav Netek, Jan Brus, and Ondrej Tomecka.  Performance testing on marker clustering and heatmap visualization techniques: a comparative study on javascript mapping libraries.  _ISPRS International Journal of Geo-Information_ , 8(8):348, 2019.
  * Mullen et al. [2018] Lincoln A Mullen, Kenneth Benoit, Os Keyes, Dmitry Selivanov, and Jeffrey Arnold.  Fast, consistent tokenization of natural language text.  _Journal of Open Source Software_ , 3(23):655, 2018.
  * Lazcka et al. [2007] Olivier Lazcka, F Javier Del Campo, and F Xavier Munoz.  Pathogen detection: A perspective of traditional methods and biosensors.  _Biosensors and bioelectronics_ , 22(7):1205–1217, 2007.