rightsretained 978-1-4503-6819-3/20/04 https://doi.org/10.1145/3334480.XXXXXXX

# Context, Utility and Influence of an Explanation

Minal Suresh Patil

Kary Främling

Umeå universitet<EMAIL_ADDRESS>Umeå universitet<EMAIL_ADDRESS>
(2020)

###### Abstract

Contextual utility theory integrates context-sensitive factors into utility-
based decision-making models. It stresses the importance of understanding
individual decision-makers’ preferences, values, and beliefs and the
situational factors that affect them. Contextual utility theory benefits
explainable AI. First, it can improve transparency and understanding of how AI
systems affect decision-making. It can reveal AI model biases and limitations
by considering personal preferences and context. Second, contextual utility
theory can make AI systems more personalized and adaptable to users and
stakeholders. AI systems can better meet user needs and values by
incorporating demographic and cultural data. Finally, contextual utility
theory promotes ethical AI development and social responsibility. AI
developers can create ethical systems that benefit society by considering
contextual factors like societal norms and values. This work, demonstrates how
contextual utility theory can improve AI system transparency, personalization,
and ethics, benefiting both users and developers.

###### keywords:

Explainable AI; Decision Theory; Contextual Utility Theory

††conference: CHI’20, April 25–30, 2020, Honolulu, HI, USA

<ccs2012> <concept> <concept_id>10003120.10003121.10011748</concept_id>
<concept_desc>Human-centered computing Empirical studies in HCI</concept_desc>
<concept_significance>300</concept_significance> </concept> </ccs2012>

[300]Human-centered computing Empirical studies in HCI

##  1 Introduction

Decision theory and utility theory are important concepts in the field of
explainable artificial intelligence (XAI)[7, 17, 8]. These theories provide a
framework for making informed and rational decisions in uncertain
environments.

Decision theory is a branch of mathematics that deals with the process of
making decisions in situations where there is uncertainty about the outcomes
of different choices[6, 4, 16, 15]. It provides a systematic approach for
analyzing and choosing between different options based on the available
information and the decision-maker’s preferences. Decision theory considers
factors such as risk, uncertainty, and the potential outcomes of each choice
in order to determine the best course of action.

Utility theory is a sub field of decision theory that focuses on the
quantification of preferences[14, 5, 3, 2]. It assumes that individuals have a
preference ordering over different outcomes, and that these preferences can be
represented as a numerical value, known as utility. The utility of an outcome
is calculated based on the individual’s preferences and the decision-maker’s
assessment of the likelihood of different outcomes. By combining decision
theory and utility theory, organisations can make informed and rational
decisions that reflect their goals and values[11, 10, 12].

To understand the concept of utility, it is helpful to consider an example.
Suppose you are trying to decide between two job offers, one that pays more
but requires a longer commute and one that pays less but is closer to your
home. The decision involves two attributes, salary and commute time, and you
must decide which attribute is more important to you. To evaluate the job
offers, you must first assign utility scores to each attribute at each level.
For example, you might assign a utility score of $0.8$ to a salary of
$\$80,000$ and a utility score of $0.5$ to a salary of $\$60,000$. Similarly,
you might assign a utility score of $0.5$ to a commute time of $30$ minutes
and a utility score of $0.3$ to a commute time of $60$ minutes.

Once you have assigned utility scores to each attribute at each level, you can
calculate the overall utility of each job offer. This involves weighting the
utility scores for each attribute by their relative importance, which is
typically expressed as a percentage. For example, you might assign a weight of
$60\%$ to salary and a weight of $40\%$ to commute time, reflecting your
priorities.

The overall utility of each job offer is then calculated as a weighted sum of
the utility scores for each attribute, using the weights assigned to each
attribute. For example, the overall utility of the first job offer might be
calculated as follows:

$0.8*0.6+0.5*0.4=0.68$ (1)

This means that the first job offer has an overall utility of $0.68$, which
reflects how well it satisfies your preferences and goals.

The process of assigning utility scores and weights to attributes is
subjective, and different decision-makers may assign different scores and
weights based on their preferences and goals. This subjectivity is one of the
strengths of utility theory, as it allows decision-makers to express their
priorities in a meaningful way.

In the context of XAI, decision theory and utility theory play an important
role in ensuring that AI systems are transparent and explainable[9]. By
incorporating these theories into AI systems, organisations can ensure that
decisions are made in a systematic and consistent manner, and that the
reasoning behind decisions is easily understood. This helps to build trust in
the AI system and ensures that decisions are aligned with the goals and values
of the organisation.

##  2 Importance and Influence

In utility theory, importance and influence are both key concepts that are
used to make decisions based on multiple criteria.

Importance refers to the degree to which each attribute is valued or weighted
in the decision-making process. In other words, it represents the decision
maker’s preferences or priorities for each attribute. Importance is typically
expressed as a numerical value or a weight, and it can be subjective or
objective depending on the decision maker’s goals and objectives.

For example, in choosing a new car, one person may consider fuel efficiency to
be the most important attribute, and so they might assign a higher weight to
it in the decision-making process. Another person may prioritize safety, and
so they would assign a higher weight to the car’s safety rating.

Influence refers to the degree to which each attribute affects the overall
utility or value of the decision. In other words, it represents the objective
or subjective impact of each attribute on the decision. Influence can be
expressed as a number or a ranking, and it is usually determined based on
empirical data, expert opinions, or the decision maker’s past experience.

For example, in the car-buying scenario, fuel efficiency might have a greater
influence on the overall value of the car than the safety rating or the price.
This would mean that changes in the fuel efficiency would have a bigger impact
on the decision than changes in the safety rating or price.

In summary, importance and influence are both used in utility theory to help
decision-makers make choices based on multiple criteria. Importance reflects
the decision maker’s priorities, while influence reflects the objective or
subjective impact of each attribute on the decision.

##  3 Contextual Utility Theory in XAI

Contextual Utility Theory (CUT) is a framework that allows for the development
of XAI models. CUT focuses on the context in which decisions are made, and how
this context can be used to provide explanations for the output of an AI
model. In this work, we will explore the principles of CUT, how it can be used
to develop XAI models, and provide examples of its application in various
domains.

###  3.1 Principles of Contextual Utility Theory

The fundamental principle of CUT is that the utility of a decision is not
fixed, but rather a function of the context in which the decision is made.
This means that the same decision can have different utilities in different
contexts. For example, the decision to purchase a car may have a different
utility for a single person living in the city compared to a family living in
a rural area. This is because the context in which the decision is being made
affects the value that the decision provides. By integrating user-specific
information, such as demographic or cultural factors, AI systems can tailor
their outputs to better align with the needs and values of their users.
Finally, contextual utility theory can help promote ethical decision-making
and social responsibility in AI development. By accounting for contextual
factors such as societal norms or values, AI developers can design systems
that align with broader ethical principles and promote positive social
outcomes and this has been further elaborated in this position paper from the
previous year’s edition [13] which emphasis that explainability should be
grounded in social contexts.

CUT provides a way to account for context in decision-making by using
contextual variables. Contextual variables are variables that describe the
context in which a decision is made. For example, in the loan approval
example, contextual variables may include the credit score of the applicant,
the applicant’s employment status, and the purpose of the loan. These
variables are used to calculate the utility of a decision, which is then used
to determine the best course of action. The CUT framework also incorporates
the principle of explainability. Explainability refers to the ability of an AI
model to provide a clear and understandable explanation for its decisions. By
using contextual variables to calculate utility, CUT allows for the creation
of models that can provide explanations that are tied to the specific context
in which the decision is being made. This makes it easier for users to
understand the decision-making process of the model and to trust its output.
CUT can be mathematically represented using a utility function that takes into
account the individual’s preferences and the context in which the decision is
being made. The utility function:

$U(x,c)=u(x)+v(c)$ (2)

where $U$ is the overall utility, x is the decision variable, $c$ is the
contextual variable, $u(x)$ is the individual’s preference function for the
decision variable, and $v(c)$ is the contextual value function.

For example, consider a restaurant recommendation AI system that takes into
account the user’s preferences and the contextual factors such as the time of
day and the user’s location. The utility function for this AI system could be
represented as:

$U(x,c)=u(x)+v(c)=\alpha 1(r)+\alpha 2(p)+\beta 1(t)+\beta 2(d)$ (3)

where $x$ is the recommendation (i.e., a particular restaurant), $c$ is the
context, $r$ is the restaurant’s rating, $p$ is the price, $t$ is the time of
day, and $d$ is the distance to the restaurant. The parameters $\alpha$ and
$\beta$ represent the weights assigned to each factor based on the user’s
preferences.

The user’s preference function $u(x)$ can be represented as:

$u(x)=\alpha 1(r)+\alpha 2(p)$ (4)

where $\alpha 1$ and $\alpha 2$ are the weights assigned to the restaurant’s
rating and price, respectively. The contextual value function $v(c)$ can be
represented as:

$v(c)=\beta 1(t)+\beta 2(d)$ (5)

where $\beta 1$ and $\beta 2$ are the weights assigned to the time of day and
distance to the restaurant, respectively.

Using this utility function, the AI system can recommend restaurants that have
the highest utility value based on the user’s individual preferences and the
specific context in which the decision is being made. By providing an
interpretable and transparent decision-making process, this approach can help
users understand why certain recommendations are being made and build trust in
the AI system.

###  3.2 Using Contextual Utility Theory in Explainable AI

CUT can be used to develop XAI models in a variety of domains, including
finance, healthcare, and autonomous vehicles. In each domain, contextual
variables can be used to calculate the utility of a decision and provide
explanations for the model’s output.

  * •

Loan Approval \- One example of the application of CUT in finance is the loan
approval process. In the loan approval process, a lender must decide whether
to approve or reject a loan application. To make this decision, the lender
considers various factors such as the applicant’s credit score, employment
status, and the purpose of the loan. To develop an XAI model using CUT, the
lender could use these factors as contextual variables. The model could then
use these variables to calculate the utility of approving or rejecting the
loan. The output of the model could then be explained based on the context of
the application. For example, the model may explain that a loan was rejected
because the applicant’s credit score was too low or because the purpose of the
loan did not align with the lender’s policies.

  * •

Medical Diagnosis \- Another example of the application of CUT is in the field
of healthcare. In healthcare, a physician must make a diagnosis based on the
patient’s symptoms and medical history. The physician may use a variety of
diagnostic tests to help with this process. To develop an XAI model using CUT,
the physician could use the patient’s medical history, age, and other relevant
factors as contextual variables. The model could then use these variables to
calculate the utility of different diagnoses. The output of the model could
then be explained based on the context of the patient. For example, the model
may explain that a particular diagnosis was made because the patient had a
history of similar symptoms or because the patient was of a certain age.

##  4 Conclusion

In conclusion, contextual utility theory offers a valuable framework for
developing explainable AI systems. By taking into account the various
contextual factors that influence decision-making, such as user preferences
and the nature of the task at hand, this theory provides a more nuanced
approach to modeling and interpreting AI outputs.

One of the key benefits of contextual utility theory is that it enables AI
systems to generate more personalized recommendations and decisions. By
incorporating individual preferences and situational factors, these systems
can better adapt to the needs of different users and deliver more tailored
outputs. Moreover, contextual utility theory can help to identify potential
biases and limitations in AI systems, and enable developers to mitigate these
issues in order to improve overall performance and reliability.

Overall, contextual utility theory represents a promising direction for the
development of explainable AI systems. As the field continues to evolve, it
will be important to explore further how this theory can be applied in
practice and refined to improve the accuracy, transparency, and ethical
implications of AI decision-making. By embracing these principles, we can work
towards creating more trustworthy and effective AI systems that benefit
society as a whole.

## Acknowledgements

The author thanks Timotheus Kampik for guidance and valuable insights in this
work. This work was partially funded by the Knut and Alice Wallenberg
Foundation.

## References

  * [1]
  * [2] David E Bell and Peter H Farquhar. 1986.  OR forum—perspectives on utility theory.  Operations Research 34, 1 (1986), 179–183.
  * [3] James S Dyer. 2016.  Multiattribute utility theory (MAUT).  Multiple criteria decision analysis: State of the art surveys (2016), 285–314.
  * [4] James S Dyer, Peter C Fishburn, Ralph E Steuer, Jyrki Wallenius, and Stanley Zionts. 1992.  Multiple criteria decision making, multiattribute utility theory: the next ten years.  Management science 38, 5 (1992), 645–654.
  * [5] José Figueira, Salvatore Greco, Matthias Ehrogott, and James S Dyer. 2005.  MAUT—multiattribute utility theory.  Multiple criteria decision analysis: state of the art surveys (2005), 265–292.
  * [6] Peter C Fishburn. 1968.  Utility theory.  Management science 14, 5 (1968), 335–378.
  * [7] Kary Främling. 2020.  Decision theory meets explainable AI. In Explainable, Transparent Autonomous Agents and Multi-Agent Systems: Second International Workshop, EXTRAAMAS 2020, Auckland, New Zealand, May 9–13, 2020, Revised Selected Papers 2. Springer, 57–74.
  * [8] Kary Främling. 2022.  Contextual importance and utility: a theoretical foundation. In AI 2021: Advances in Artificial Intelligence: 34th Australasian Joint Conference, AI 2021, Sydney, NSW, Australia, February 2–4, 2022, Proceedings. Springer, 117–128.
  * [9] Kary Främling12. 2021.  Contextual importance and utility in R: the ‘ciu’package.  (2021).
  * [10] Ole Hagen. 1995.  Risk in Utility Theory, in Business and in the World of Fear and Hope.  Revolutionary Changes in Understanding Man and Society: Scopes and Limits (1995), 191–210.
  * [11] Joseph E Haring and Gorman C Smith. 1959.  Utility theory, decision theory, and profit maximization.  The American Economic Review 49, 4 (1959), 566–583.
  * [12] Terrence C Sebora and Jeffrey R Cornwall. 1995.  Expected utility theory vs. prospect theory: Implications for strategic decision makers.  Journal of Managerial Issues (1995), 41–61.
  * [13] Deepa Singh, Michal Slupczynski, Ajit G Pillai, and Vinoth Pandian Sermuga Pandian. 2022.  Grounding Explainability Within the Context of Global South in XAI.  arXiv preprint arXiv:2205.06919 (2022).
  * [14] George J Stigler. 1950.  The development of utility theory. I.  Journal of political economy 58, 4 (1950), 307–327.
  * [15] Detlof Von Winterfeldt and Gregory W Fischer. 1975.  Multi-attribute utility theory: models and assessment procedures. In Utility, Probability, and Human Decision Making: Selected Proceedings of an Interdisciplinary Research Conference, Rome, 3–6 September, 1973. Springer, 47–85.
  * [16] Jyrki Wallenius, James S Dyer, Peter C Fishburn, Ralph E Steuer, Stanley Zionts, and Kalyanmoy Deb. 2008.  Multiple criteria decision making, multiattribute utility theory: Recent accomplishments and what lies ahead.  Management science 54, 7 (2008), 1336–1349.
  * [17] Danding Wang, Qian Yang, Ashraf Abdul, and Brian Y Lim. 2019.  Designing theory-driven user-centric explainable AI. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–15.


utf8

# Comparing the Accuracy of Deep Neural Networks (DNN) and Convolutional
Neural Network (CNN) in Music Genre Recognition (MGR): Experiments on Kurdish
Music

Hossein Hassani University of Kurdistan Hewlêr Kurdistan Region - Iraq
{aza.zuhairamin<EMAIL_ADDRESS>
###### Abstract

Musicologists use various labels to classify similar music styles under a
shared title. But, non-specialists may categorize music differently. That
could be through finding patterns in harmony, instruments, and form of the
music. People usually identify a music genre solely by listening, but now
computers and Artificial Intelligence (AI) can automate this process. The work
on applying AI in the classification of types of music has been growing
recently, but there is no evidence of such research on the Kurdish music
genres. In this research, we developed a dataset that contains 880 samples
from eight different Kurdish music genres. We evaluated two machine learning
approaches, a Deep Neural Network (DNN) and a Convolutional Neural Network
(CNN), to recognize the genres. The results showed that the CNN model
outperformed the DNN by achieving 92% versus 90% accuracy.

##  1 Introduction

Music is classified into genres that share the same style, melody, and
culture. Genre is a conceptual tool to classify music and other forms of art
[Lena and Peterson (2008]. According to ?), automated music genre
classification or music genre recognition (MGR) is to structure and organize a
very large music archive using computers. MGR is used in various applications
such as Music Information Retrieval (MIR) [Downie (2003] and recommendation
systems [Lorince (2016], for example, Tidal, Spotify, and Apple Music.

In the absence of automated approaches, musicologists classify pieces of music
into different genres based on lyrics and melody just by listening to them.
However, with the growing collection of music genres and music itself, the
need for automated music classification has also increased [Puppala et al.
(2021]. Consequently, studying the usage of advanced machine learning
techniques such as neural networks in the development of more efficient and
accurate music genre classification has gained more attention in the computer
science domain.

In this research, we apply and evaluate the accuracy of two neural network
approaches, Deep Neural Network (DNN) and a Convolutional Neural Network
(CNN), in automatically classifying Kurdish music into its different genres.
The rest of this paper is organized as follows. Section 2 reviews the related
work. Section 3 provides the method of data collection, training, and
evaluation. In Section 4, we present and discuss the outcomes of the
experiments. Finally, Section 5 concludes the paper.

##  2 Related Work

?) compared two different machine learning models, CNN and a Long Short-Term
Memory (LSTM), and for music classification. They compared the performance of
the models on different types of features such as Mel-Spectrogram, Mel
Coefficients, and Tonnetz Features.

?) used Deep Belief Network (DB), an unsupervised machine learning, to
recognize two to four music genres. They used the GTZAN dataset that consisted
of 1000 pieces of music from 10 different genres to train the model. By
extracting the Mel Frequency Cepstral Coefficient (MFCC) from the music, they
generated 15 samples per music and created a dataset of 15000 samples to train
and test the model (60% for training and 40% for testing). Their DBF model
consisted of five layers. The related layers were trained over Restricted
Boltzmann Machine (RBM) iteratively. The model achieved 98.15% accuracy in
recognizing two genres, 69.16% in recognizing three genres, and 51.88% in
recognizing four genres.

?) used an ensemble learning method for MGR based on space and time
decomposition and evaluated the results of different segment parts. They
divided each music into three different segments: the music’s beginning,
middle, and end. Ensemble learning is a combination of different learning
algorithms to enhance the classification result. The learning algorithms in
the mentioned experiment were Decision Tree, K-Nearest Neighbours (KNN),
Naive-Bayes, Multi-Layer Perceptron neural network, and Support Vector Machine
(SVM). The results showed the middle segment of the music performing better
than the other two segments.

?) applied Gaussian Mixture Model and KNN for MGR. They used three features of
the audio, pitch, rhythm, and timbral texture in their model.

?) developed a multidimensional classifier based on several models, such as
Maximum a Posteriori (MAP), Gaussian Mixture Model (GMM), and KNN in their MGR
experiments in which they incorporated 13 different features of the music.
They trained the classifier on a frame-by-frame audio segment basis that
achieved an error rate of 5.8%. However, by extending the length of the
segments the error rate dropped down to 1.4%.

To summarize, the literature shows using a wide range of machine learning
methods in MGR. It also shows the variety of methods in data segmentation and
datasets preparation. We also observe growing attention to MGR in general, but
we did not notice any study on Kurdish MGR during the literature review and
while compiling this paper.

##  3 Methodology

###  3.1 Data Collection

We reach out to fine art teachers, students, and artists, to explore various
Kurdish music genres and understand them. Because there is no ready-to-use
dataset, we collect music from various sources online and offline, such as
Youtube, Soundcloud, and Music CDs. We organize the collected music into
various genres. ?) mentioned some of the Kurdish music genres, for example,
Bend, Gorani Meqam, and Hayran. We collect the data in Waveform Audio File
Format (WAVE or WAV) where they are available, otherwise in MPEG-1 Audio
Player III (MP3) format. However, we convert the MP3s into WAV to have a
decompressed version of the music.

###  3.2 Feature Extraction

For feature extraction, we used Librosa version (0.8.1) [McFee et al. (2015],
a python library for analyzing music and audio that is often used in the field
of MRI. We use MFCC111https://musicinformationretrieval.com/mfcc.html features
as the sole attribute for classifying the Kurdish genres. MFCC is for
describing the timbre of the music, and it consists of a small set of features
that describe the shape of the spectral envelope. Using Librosa, we segment
the 30-second audio files into 1200 segments, which results in the audio files
being much smaller but the dataset much bigger for training. However,
according to ?), longer segments perform better at classifying audio, and
therefore, the number of segments extracted from the music might change,
depending on the result of the experiments.

###  3.3 Data Preparation

Initially, we randomize the dataset and split it into training, validation,
and testing datasets that receive 70%, 10%, and 20% of the original dataset
accordingly. However, the splitting ratio might change depending on the length
of the collected dataset and the results that we obtain.

###  3.4 Architecture of the Models

We use two architectures using TensorFlow222https://www.tensorflow.org/
libraryand Keras333https://keras.io/ API. The architectures are based on those
that ?) suggests. We may modify the architecture and the number of epochs
during the experiments to improve the performance of the models.

The first model shown in Figure 1 is a Deep Neural Network that consists of
four layers. For the input layer, we flatten the data with the shape of (1,
13), and then we fed it into the first layer that has 512 units. We use
Rectified Linear (ReLu) as the activation function. The second layer has 256
units, and it uses the same activation function as the first layer. Between
the second layer and the third layer, we insert a dropout layer with a rate of
0.3 to prevent overfitting in the model. The third layer consists of 64 units,
and it uses the same activation function as the first and second layers.
Finally, the output layer uses a softmax activation function to classify the
given data.

Figure 1: Modified Deep Neural Network architecture.

Figure 2 shows the second model, CNN, which consists of five layers. The first
layer is a two-dimensional (2D) convolutional layer with 32 filters, kernel
size of (3, 3), and it uses ReLu as the activation function. The model also
contains a 2D Max Pooling layer with a pool size of (3, 3) and strides of (2,
2). We fed the data into a batch normalization layer to normalize the inputs.
The second layer is the same as the first layer, consisting of 2D
convolutional, 2D max pooling, and batch normalization layers. The third layer
contains a 2D convolutional layer with 32 filters, kernel size of (2, 2), and
uses ReLu as the activation function. A 2D max pooling layer with a pool size
of (2, 2) comes after the 2D convolutional layer. We fed the output of the
max-pooling layer into a batch normalization layer. The fourth layer flattens
the output of the last layer and feeds it into a dense layer. The dense layer
consists of 64 units and uses ReLu as the activation function. The last layer
uses a softmax function to predict the genre.

Figure 2: Modified Convolutional Neural Network architecture.

###  3.5 Evaluation and Testing

We evaluate the accuracy of the models using a TensorFlow metric that is
called
Accuracy444https://www.tensorflow.org/api_docs/python/tf/keras/metrics/Accuracy.
Formula 1 shows the metric, in which accuracy is the calculated accuracy of
the model, frequency is the frequency of the predictions that match the true
label, and total samples is the number samples examined in the experiment.

$accuracy=frequency/totalsamples$ (1)

##  4 Experiments and Result

###  4.1 Data Collection

We collected 208 pieces of music from eight genres. The length of the pieces
is between one to 30 minutes. We trimmed the music into 30-second audio files.
We followed the method suggested by ?), and we took the samples from the
middle to keep the overall timbre of the music. The trimming process provided
2293 samples. However, the data was still imbalanced because Gorani Classic
genre had a larger portion of data in comparison to the other genres. That is
why only an equal amount of music in each genre had to be selected for
training and testing the models. Therefore, we randomly selected 110 pieces
because the minimum number of samples that we had was 110 samples for one of
the genres Halparke Se-pey genre that had 110 pieces of music after the
trimming process.

Figure 3: Bar chart representing the number of music in each genre after pre-
processing.

During our experimentation, we observed the results and made modifications to
improve the models. Tables 1, 2, and 3 show those modifications along with the
attributes for training and evaluating the models for each experiment. Table 1
summarizes the experiments we did on the DNN model, with the results of each
experiment.

Table 1: Experiments and results of the DNN model. No. | Input Shape | Total Samples | Randomization Algorithm | Epochs | Accuracy | Loss
---|---|---|---|---|---|---
1 | (2, 13) | 1,056,000 | Random Split | 30 | 74% | 72%
2 | (2, 13) | 200,000 | Random Split | 30 | 67% | 90%
3 | (2, 13) | 1,056,000 | k-Fold Cross-validation | 30 | 74% | 71%
4 | (2, 13) | 1,056,000 | k-Fold Cross-validation | 100 | 75% | 69%
5 | (44, 13) | 26,400 | k-Fold Cross-validation | 30 | 90% | 39%
6 | (44, 13) | 26,400 | k-Fold Cross-validation | 30 | 90% | 32%

Table 2 is the summary of the experiments and results on the CNN model.

Table 2: Experiments and results of the CNN model. No. | Input Shape | Total Samples | Randomization Algorithm | Epochs | Accuracy | Loss
---|---|---|---|---|---|---
1 | (2, 13, 1) | 1,056,000 | Random Split | 30 | 67% | 88%
2 | (2, 13, 1) | 200,000 | Random Split | 30 | 64% | 98%
3 | (2, 13, 1) | 1,056,000 | k-Fold Cross-validation | 30 | 67% | 88%
4 | (2, 13, 1) | 1,056,000 | k-Fold Cross-validation | 100 | 69% | 85%
5 | (44, 13, 1) | 26,400 | k-Fold Cross-validation | 30 | 92% | 23%
6 | (44, 13, 1) | 26,400 | k-Fold Cross-validation | 30 | 92% | 25%

Table 3 is the details of the datasets that are used for each experiment on
both models. As the results show, we achieved 90% accuracy for the DNN model
and 92% accuracy for the CNN model.

Table 3: Details of the dataset for each experiments. No. | Total Samples | Splitting Ratio | Sample Length | Train Set | Valid Set | Test Set
---|---|---|---|---|---|---
1 | 1,056,000 | 7:1:2 | 25ms | 739,200 | 105,600 | 211,200
2 | 200,000 | 5:2.5:2.5 | 25ms | 100,000 | 50,000 | 50,000
3 | 1,056,000 | 7:1:2 | 25ms | 739,200 | 105,600 | 211,200
4 | 1,056,000 | 7:1:2 | 25ms | 739,200 | 105,600 | 211,200
5 | 26,400 | 8:1:1 | 1s | 21,120 | 2,640 | 2,640
6 | 26,400 | 8:1:1 | 1s | 21,120 | 2,640 | 2,640

Fig. 4 shows the training performance of the DNN model in the sixth
experiment, and Figure Fig. 5 shows it for the CNN model in the fifth
experiment.

Figure 4: Sixth experiment training process for the DNN model. Figure 5: Fifth
experiment training process for the CNN model.

###  4.2 Discussion

Although we achieved 90% accuracy for DNN and 92% for CNN, when we look back
at the first four experiments of the models, we can see that they performed
poorly on the testing dataset. For both models, the first experiment did not
perform well on the testing dataset, and the loss was more than 70% in both
cases.

To investigate the problem, in the second experiment, we trained and tested
the models on a smaller dataset, but the results did not improve; the accuracy
was decreased, and the loss was increased. While training the models in the
first and second experiments, the validation accuracy was higher than the
training accuracy, indicating that the dataset is not well-randomized.

In the third experiment, we changed the algorithm for randomizing the dataset
to the cross-validation algorithm mentioned in the methodology. In this
experiment, the dataset was distributed evenly for each genre using the
StratifiedKFold class in the Scikit-learn library555https://scikit-
learn.org/stable/. We used the last distributed group (Fold) as the dataset
for conducting the rest of the experiments. This time, the results showed a
higher validation accuracy than the training accuracy similar to the second
experiment.

?) states that if the model’s accuracy does not decrease across the epochs,
increasing the number of epochs could increase the accuracy. For the fourth
experiment, we increased the number of epochs to 100 for both models. The DNN
model validation accuracy was less than the training, but it did not perform
well on detecting the genres. Unlike the DNN, the CNN model result was similar
to the previous experiment, the validation accuracy being higher than the
training accuracy.

Also, according to ?), the classifier has less error rate when working with
longer samples, for example, samples of 2.4 seconds length. To extract longer
samples, we needed to decrease the amount of the segmentation. Therefore, we
went back to feature extraction in the fifth experiment and changed the number
of samples to 30 from each 30-second music. Segmenting each 30-second music
into 30 samples made each sample 1 second long. We did not extract samples
longer than 1 second because the dataset would be much smaller for
experimenting. Additionally, because the length of the dataset would have
become less than the previous experiments, the model could overfit. Thus, we
changed the number of epochs back to 30.

The fifth experiment showed a promising result, with 90% accuracy for DNN and
92% for CNN. However, The training accuracy was higher than the validation
accuracy by seven percent, indicating a slight overfit in the model. To fix
the minor overfit, we made a slight modification to the architecture of the
models, adding one dropout layer with the rate of 0.3 to both models. After
that, both of the models showed no overfit.

Looking back at the experiments, the best result for the DNN model was in the
sixth experiment. We achieved 90% accuracy with a 32% loss. Additionally, the
best result for the CNN model was in the fifth experiment because it had less
error rate than the sixth error rate, which means it made fewer errors on the
testing dataset. The model was 92% accurate with a 23% loss.

##  5 Conclusion

As music becomes more available in large collections, their genre recognition
gets more attention. As a result, Music Genre Recognition (MGR) has become a
field of study. Scientists have attempted to classify music genres using
machine learning and Artificial Intelligence (AI). However, so far, using
those techniques and methods has not been practiced. In this research, we
applied machine learning techniques to recognize Kurdish music genres by using
two artificial neural network methods, DNN and CNN, using machine learning. We
collected and built a dataset of 880 music from eight different Kurdish music
genres. Each genre has 110 music, and each piece of music is 30 seconds long.
We developed the models and evaluated them. The CNN model scored 92% accuracy,
and the DNN model achieved 90% accuracy, which states that the CNN model
outperformed the DNN model at recognizing the Kurdish music genres.

The Kurdish MGR requires extensive efforts in the future. Some of the areas
could be of immediate interest., for example, to train the models on better
quality music. Some of the music that we used in this research were not of
high-quality sound. Also, working on the recognition of low-quality music
could be a research area because one could find a considerable amount of
Kurdish music that has been recorded using low-quality instruments. To
separate the songs from the music for each genre could be another working area
that might positively affect genre recognition. Furthermore, training the
models with longer samples could also be investigated. Finally, the classified
dataset used for this research can be used for building a recommender system
for Kurdish music genres.

## References

  * [Ahmadi et al. (2020]  Ahmadi, S., Hassani, H., and Abedi, K.  (2020).  A corpus of the sorani kurdish folkloric lyrics.  In Proceedings of the 1st Joint Workshop on Spoken Language Technologies for Under-resourced languages (SLTU) and Collaboration and Computing for Under-Resourced Languages (CCURL), pages 330–335.
  * [Downie (2003]  Downie, J. S.  (2003).  Music information retrieval.  Annual review of information science and technology, 37(1):295–340.
  * [Feng (2014]  Feng, T.  (2014).  Deep learning for music genre classification.  private document.
  * [Ghosal and Kolekar (2018]  Ghosal, D. and Kolekar, M. H.  (2018).  Music genre recognition using deep neural networks and transfer learning.  In Interspeech, pages 2087–2091.
  * [Lena and Peterson (2008]  Lena, J. C. and Peterson, R. A.  (2008).  Classification as culture: Types and trajectories of music genres.  American sociological review, 73(5):697–718.
  * [Lorince (2016]  Lorince, J.  (2016).  Consumption of content on the web: An ecologically inspired perspective.  Ph.D. thesis, Indiana University.
  * [McFee et al. (2015]  McFee, B., Raffel, C., Liang, D., Ellis, D. P., McVicar, M., Battenberg, E., and Nieto, O.  (2015).  librosa: Audio and music signal analysis in python.  In Proceedings of the 14th python in science conference, volume 8, pages 18–25. Citeseer.
  * [Norowi et al. (2005]  Norowi, N. M., Doraisamy, S., and Wirza, R.  (2005).  Factors affecting automatic genre classification: an investigation incorporating non-western musical forms.  In Proceedings of the International Conference on Music Information Retrieval, pages 13–20.
  * [Puppala et al. (2021]  Puppala, L. K., Muvva, S. S. R., Chinige, S. R., and Rajendran, P.  (2021).  A novel music genre classification using convolutional neural network.  In 2021 6th International Conference on Communication and Electronics Systems (ICCES), pages 1246–1249.
  * [Scheirer and Slaney (1997]  Scheirer, E. and Slaney, M.  (1997).  Construction and evaluation of a robust multifeature speech/music discriminator.  In 1997 IEEE international conference on acoustics, speech, and signal processing, volume 2, pages 1331–1334. IEEE.
  * [Silla et al. (2008]  Silla, C. N., Koerich, A. L., and Kaestner, C. A.  (2008).  A machine learning approach to automatic music genre classification.  Journal of the Brazilian Computer Society, 14(3):7–18.
  * [Taub (2021]  Taub, D.  (2021).  Must accuracy increase after every epoch?  Available at https://stackoverflow.com/questions/45605003/must-accuracy-increase-after-every-epoch.
  * [Tzanetakis and Cook (2002]  Tzanetakis, G. and Cook, P.  (2002).  Musical genre classification of audio signals.  IEEE Transactions on speech and audio processing, 10(5):293–302.
  * [Velardo (2021]  Velardo, V.  (2021).  DeepLearningForAudioWithPython.  Available at https://github.com/musikalkemist/DeepLearningForAudioWithPython.