# Music Genre Classification with ResNet and Bi-GRU Using Visual Spectrograms

Junfei Zhang The University of Melbourne
E-mail<EMAIL_ADDRESS>
###### Abstract

Music recommendation systems have emerged as a vital component to enhance user
experience and satisfaction for the music streaming services, which dominates
music consumption. The key challenge in improving these recommender systems
lies in comprehending the complexity of music data, specifically for the
underpinning music genre classification. The limitations of manual genre
classification have highlighted the need for a more advanced system, namely
the Automatic Music Genre Classification (AMGC) system. While traditional
machine learning techniques have shown potential in genre classification, they
heavily rely on manually engineered features and feature selection, failing to
capture the full complexity of music data. On the other hand, deep learning
classification architectures like the traditional Convolutional Neural
Networks (CNN) are effective in capturing the spatial hierarchies but struggle
to capture the temporal dynamics inherent in music data. To address these
challenges, this study proposes a novel approach using visual spectrograms as
input, and propose a hybrid model that combines the strength of the Residual
neural Network (ResNet) and the Gated Recurrent Unit (GRU). This model is
designed to provide a more comprehensive analysis of music data, offering the
potential to improve the music recommender systems through achieving a more
comprehensive analysis of music data and hence potentially more accurate genre
classification.

###### Index Terms:

Machine Learning, Deep Learning, Neural Networks

##  1 Introduction

The landscape of music consumption has significantly shifted in recent years
with the advent of music streaming services [26], such as Spotify, Apple
Music, Amazon Music and YouTube Music. As a result, music recommendation
systems have become an essential part of these platforms to enhance user
experience and satisfaction, which is seen to have a significant entertainment
and commercial value. Despite these substantial advancements, music
recommendation systems often struggle with the inherently complexity of music
data [26]. This complexity stems largely from the challenges involved in
comprehending and classifying the intricacies embedded within musical genres
[5]. Current systems heavily rely on user input and manual genre
classification, often leading to recommendations that fail to accurately
capture the listener’s preferences [6]. Furthermore, with the vast number of
music files currently exist and continuously emerging on the internet, manual
classification becomes increasingly unsustainable, leaving numerous songs
unclassified or inaccurately labeled, as the industrial standard currently is
incomplete [10]. These challenges underline the critical need for a more
advanced, automatic system for music genre classification, necessitating a
shift towards Music Information Retrieval (MIR)-based solutions. An Automatic
Music Genre Classification (AMGC) system becomes imperative. Such a system
could autonomously analyze and classify music into genres, considering a
broader range musical features and their complexities. This would not only
streamline the classification process, but also greatly improve the quality
and personalisation of recommendations.

Music genre classification presents a significant challenge due to the lack of
strict boundaries between genres [3], which is compounded by the complexity of
music’s interaction with the public and the unique vibe it carries [3]. In the
realm of AMGC, traditional machine learning techniques, such as k-Nearest
Neighbors (k-NN), Support Vector Machines (SVM), and Random Forest (RF) have
demonstrated considerable potential. However, these algorithms heavily rely on
the effectiveness of manually engineered features and feature sets, which are
typically defined by the music’s instrumentation (eg presence of electric
guitars), rhythmic structure (eg beat), and harmonic content (eg chords,
tones) [3]. The reliance on the analysis of feature importance is a
significant limitation as it fails to fully capture the complexity of music
data. To address this limitation, deep learning methods such as Convolutional
Neural Networks (CNN) have shown significant promise in handling this task
[11]. While they also often operate on engineered features, such as the Short-
time Fourier transform (STFT)[12] and the Mel Spectrograms [17], their ability
to learn hierearchical representations allows them to extract complex patterns
from this transformed representation of the audio, potentially improving the
genre classification performance. Traditional CNNs are primarily designed to
deal with spatial or hierarchical data mostly present in image data, whereas
it does not inherently model the sequence of data with temporal dynamic
features [18], thereby possibly limiting their effectiveness for time-series
data commonly used to represent audio. Given the shortcomings of traditional
machine learning and acknowledging the limitations and potential of deep
learning methods, there is a strong motivation to design a more effective
approach. This study aims to employ deep learning to manage the complexity of
the task and propose a deep learning architecture that can effectively extract
both hierarchical and temporal information from music data effectively. This
paper presents an innovative approach of using visual spectrograms as image-
based inputs, and propose a novel hybrid model that combines the strength of
the Residual Neural Network (ResNet) and Gated Recurrent Unit (GRU). This
hybrid model processes image-based input with ResNet while the GRU block
handles sequence-like data from the reshaped image inputs, effectively
leveraging the strength of both architectures to provide a more comprehensive
analysis of the music data. The implications of this research could
significantly improve the music recommendation systems by offering a more
precise genre classification, thereby potentially enhancing user experience on
music streaming platforms.

This paper outlines the development and evaluation of the proposed
architecture. The organisation of the paper is as follows: Section 2 gives a
review of current research related to the task. Section 3 describes the
dataset used for training and testing the model, as well as the data
augmentation. Section 4 details the methodology and the classification
pipeline. Section 5 analysis the results obtained, while Section 6 discusses
the implications of the study. Section 7 summarises the conclusions drawn from
the research, as well as a suggestion of potential future work. Appendix A
showcases a deployment of the classifier to a music recommendation system.

##  2 Related Work

###  2.1 Problem Definition

Music genre classification is a difficult task in nature due to the ambiguity
of genre boundaries. Highlighting this, a research study by Robert et al. [7]
investigated manual music classification among college students. Participants
were asked to listen to music pieces and classify them into one of ten genres:
blues, country, classical, dance, jazz, latin, pop, R&B, rap, and rock. The
research found that participants were able to achieve only 70% classification
with 3 seconds musical pieces, and the accuracy did not improve for longer
pieces. This is always referenced as a benchmark in machine genre
classifications that sets the stage of the AMGC task.

The AMGC problem was initially defined by Tzanetakis and Cook [3] in 2002. In
current context, the original sound wave is sampled and transformed into a
sequence of numeric values $S=<s_{1},s_{2}...s_{N}>$ [10], where $s_{i}$
represents the sample at instant $i$, and $N$ represents the total number of
samples. The features that contained in the sequence can be extracted to get
the feature vector $\bar{X}=<x_{1},x_{2}...x_{D}>$. The AMGC problem concerns
about finding the genre from a set of music genres that best represents the
genre of the music [10]. It can be expressed using a statistical perspective
by this equation:

$\hat{g}=argmax_{g\in G}P(g|\bar{X})$ (1)

where $P(g|\bar{X})$ is the probability that the music belong to the genre $g$
given the features expressed by $\bar{X}$. This equation is applicable to both
machine learning and deep learning approaches. This section presents the
related works and solutions pertinent to the AMGC task, providing insights on
the evolution of this field to its current state. A summary of the performance
is presented in Table I.

TABLE I: A summary of current methods and accuracy for the AMGC task (10 genres) Methods |  Accuracy
---|---
Machine Learning
KNN |  0.54[1], 0.61[3], 0.63[38]
SVM |  0.60[1], 0.65 [10], 0.73[38]
NB |  0.57[38]
DT |  0.55[38]
RF |  0.81[39], 0.66[38]
Neural Networks
CNN |  0.69[38], 0.72[40], 0.82[1]
YOLOv4 |  0.945[26]
BBNN |  0.939[18]

###  2.2 Machine Learning Approach

For traditional machine learning methods, Equation 1 models the relationship
between input features and output labels. Examples of such techniques include
K-Nearest Neighbors (KNN), Support Vector Machine (SVM), Naive Bayes (NB),
Decision Tree (DT), and Random Forest (RF). These techniques have shown
considerable promise in the domain of music genre classification. For
instance, a study [1] demonstrated testing accuracy of 0.54 using KNN and 0.60
using SVM. Another comparative evaluation [38] reported accuracy scores of
0.63 for KNN, 0.73 for SVM, 0.57 for NB, 0.55 for DT, and 0.66 for RF.
However, it’s crucial to remember that these performances are largely
dependent on the effectiveness of the selected features or feature sets. Thus,
the quality and relevance of feature selection is a crucial consideration in
traditional machine learning approaches and remains an open question in the
Automatic Music Genre Classification (AMGC) problem [10].

Further studies have tried to delve deeper into the significance of different
feature sets. The noteworthy study by George et al. [3] explored the KNN model
using three sets of features representing timbral texture, rhythmic content,
and pitch content. The study found that timbral texture features outperformed
other sets, underscoring their relevance in the music classification process.
In another work [10], researchers introduced space and time decomposition
strategies which used One-Against-All (OAA) and Round-Robin (RR) approaches to
obtain the final genre label based on an ensemble method applied to the
initial, middle, and end parts of the music. The optimal result, which was an
accuracy of 0.65, was achieved with the SVM classifier and space-time
decomposition using Round Robin (RR) approach. The study shows that 12 out of
the 14 most selected features are timbral texture related, such as 1 st. MFCC
mean, 3 rd. MFCC mean, Standard deviation for spectral flow, Standard
deviation for 1 st. MFCC, etc. However, the authors noted that different
features might have varying importance based on the classifier used (KNN, NB,
SVM), as well as different classification approaches on different segments of
music. This complexity underlines a fundamental challenge in traditional
machine learning for AMGC: despite its promising performance, its
effectiveness heavily depends on the quality of feature selection and its
ability to capture the underlying structure of music data, motivating the
exploration of alternative approaches such as deep learning.

###  2.3 Deep Learning Approach

Deep learning approaches use a layered structure of artificial neurons to
model complex patterns in music data. They do not explicitly calculate the
probability in Equation 1 but implicitly learn to approximate $P(g|\bar{X})$
through their architecture. The training process adjusts the weights of the
connections in the network to minimize a loss function. This can potentially
lead to more robust and nuanced feature extraction.

Convolutional Neural Networks (CNN) have been recognized for their superior
accuracy in AMGC tasks compared with the machine learning approaches. A
comparative study showcased CNN’s supremacy over other machine learning
models, including Support Vector Machines (SVM), Random Forests, and eXtreme
Gradient Boosting (XGB) [35]. While CNNs are traditionally known for their
image pattern recognition capabilities, their application to AMGC tasks often
involves time-series data as the input since it is a straightforward
representation of audio signals. However, current research highlights
promising results when using image-based inputs such as the visual
spectrograms. Two studies have reported high accuracies when applying CNN to
spectrogram images for music genre classification, one achieving an accuracy
of 0.874 using 2.56-second slices and 0.918 using full song data [34], while
the other using the CNN based YOLOv4 architecture on visual Mel Spectrograms
reported an accuracy as high as 0.945 [26]. Using graphical spectrogram
representations not only offers high generalization performance but also
eliminates the need for constructing a specialized audio model [26].

There are multiple time-frequency image formation proposed for CNN. For tasks
like speech emotion recognition [15], the traditional spectrogram
representation formed using the short-time Fourier transform (STFT)
[frequencyrepresentation] is often employed [11]. However, this method can be
computationally expensive if the goal is to uncover unique frequency
characteristics by using a large number of points in the Fourier transform
[11]. As a more efficient alternative, methods have been proposed to utilize a
larger transform length and compute the filterbank energies in frequency
subbands [16]. The mel-filter is one such frequently used filter in this
context, which utilizes the mel-scale [17] to space the frequency bands. Image
representations employing the mel-filter, such as Mel Spectrograms, have been
widely used with CNN in various acoustic classification challenges [11] [24].
Current advancements in the AMGC task leverage deep learning approaches,
specifically CNN, for their remarkable accuracy and efficiency. The following
summary presents these high performing approaches in detail.

####  2.3.1 Bottom-up Broadcast Neural Network (BBNN) [18]

This study focuses on addressing a key limitation of traditional CNN
architectures in the context of music genre classification, which is the
inability for CNN to learn adequate and comprehensive features for music genre
classification in temporal dynamics. Their innovative CNN structure, Bottom-up
Broadcast Neural Network (BBNN), effectively utilizes both high-level semantic
information and low-level features from music data. It achieves this by
constructing a strategic arrangement of blocks and connections among them.
This arrangement is designed to maximize the use of and preserve low-level
information as it is transmitted to higher layers of the network. With this
design, the proposed BBNN achieved an impressive accuracy of 0.939 on the
GTZAN dataset, signifying a significant advance in the AMGC realm.

While the BBNN model represents a significant advancement in automatic music
genre classification, there is still room to apply transfer learning and
leverage the power of well-established pretrained models. Among them, the
Residual Networks (ResNet) and You Only Look Once (YOLO) architectures stand
out for their efficiency and depth, facilitating the training of much deeper
networks, which could potentially enhance the learning process.

####  2.3.2 You Only Look Once Version 4 (YOLOv4) [26]

The study under discussion introduces an innovative approach to the AMGC
problem, employing the YOLOv4 architecture using visual Mel Spectrograms. The
promising result of an accuracy of 0.945 signifies the potential of YOLOv4 for
such tasks. Originating from the field of computer vision, YOLO (You Only Look
Once) models are typically deployed for object detection tasks [19]. The
fourth iteration of this model series, YOLOv4 [20], has demonstrated
enhancements in speed and accuracy over its predecessors. The architecture of
YOLOv4 is composed of five modules based on CNN. Although the conventional
application of YOLOv4 is object recognition in images, such as vehicle
detection for autonomous driving [autodrive], or face detection [22] and
recognition in surveillance systems [23], its success in the AMGC task
suggests a potential for transferring pre-trained computer vision models to
audio classification tasks.

##  3 Dataset

This study utilizes the GTZAN Dataset[37], which is a highly-regarded public
dataset frequently employed for evaluation in machine listening research,
specifically for music genre recognition. The dataset sources vary widely,
encompassing personal CDs, radio transmissions, and microphone recordings. It
is assembled by Tzanetakis and Cook [37] and has served as a significant
resource in this research area. The details of the GTZAN Dataset are
summarized in Table II.

TABLE II: GTZAN Dataset Details Item |  Data |  Details
---|---|---
Genre |  10 |  blues, classical, country, disco, hiphop, jazz, metal, pop, reggae, rock
Length |  $\sim$30s per clip |  Minimum length: 29.93s; Maximum length: 30.65s; Mean length: 30.02s
Sample rate |  22,050 Hz |
Song format |  WAV |
Total |  1000 |  100 files/genre

###  3.1 Data Augmentation

To make the input more robust, this study applies data augmentation to the
GTZAN dataset. The original GTZAN dataset comprises 1,000 audio clips of
slightly varying lengths, a format not ideally suited for model input.
Therefore, to rectify this as well as increasing the number of data, each
audio clip in the original dataset was randomly sampled within a contiguous
3-second window at five non-overlapping random locations. This process
augmented the dataset to 5,000 audio clips, each lasting precisely three
seconds. These details are summarized in Table III.

TABLE III: Augmented Dataset Details Item |  Data |  Details
---|---|---
Genre |  10 |  blues, classical, country, disco, hiphop, jazz, metal, pop, reggae, rock
Length |  3s per clip |
Sample rate |  22,050 Hz |
Song format |  WAV |
Total |  5000 |  500 files/genre

##  4 Method

###  4.1 Preprocessing

Figure 1: Raw audio waveform for jazz class

This study utilizes visual Mel Spectrograms as its primary data. Developed by
Davis and Mermelstein [36] in 1980, the mel-scale prioritises specific
frequency components and filtering less significant ones, which effectively
models the logarithmic frequency resolution of human auditory perception. To
convert raw audio data (Fig. 1) into Mel Spectrograms, the initial step is to
shift the audio from the time domain to the frequency domain using the Short
Time Fourier Transform (STFT). The $i$-th feature of this transformation is
defined by the following equation:

$x_{i}=STFT\\{s\\}(i,\omega)=\sum_{n}s_{n}\cdot w[n-i]\cdot e^{-j\omega n}$
(2)

Figure 2: STFT Spectrogram for jazz class

In this equation, $s_{n}$ signifies the $n$-th sample in the sequence $S$,
$w[n-i]$ represents a window function centered on the $i$-th sample, and
$\omega$ denotes frequency. The outcome, $\bar{X}_{stft}$, is a sequence of
time-frequency representations derived via the STFT at various moments within
the audio signal. The transitioned visual STFT Spectrogram is presented in
Fig. 2, demonstrating the transformation of the raw audio from the time to
time-frequency domain. The subsequent phase is the computation of
$\bar{X}_{mel}$, the Mel Spectrogram. If $\bar{X}_{stft,i}$ symbolizes the
$i$-th STFT feature, the corresponding Mel frequency feature $\bar{X}_{mel,i}$
can be calculated as:

$\bar{X}_{mel,i}=M\cdot|\bar{X}_{stft,i}|^{2}$ (3)

Figure 3: Mel Spectrogram for jazz class

In this formula, $M$ refers to a matrix depicting the Mel filter bank, and
squaring equates to the power spectrum. Each row in $M$ pertains to a unique
Mel filter. This matrix is derived based on the mel-scale; the conversion from
Hertz ($f_{Hz}$) to Mel ($m$) scale can be described by the following
equation:

$m=2595\log_{10}(1+\frac{f_{Hz}}{700})$ (4)

Multiplying this with the power spectrum yields the energy within the Mel-
scaled frequency bands. This process is repeated for each STFT feature to
attain a corresponding Mel-frequency feature. The logarithm of $\bar{X}_{mel}$
is then computed to acquire the log-Mel-spectrum. Fig. 3 portrays the Mel
Spectrogram. All these operations are implemented via the Librosa library. The
parameters are displayed in Table IV.

TABLE IV: Parameter Values for Mel Spectrograms Parameter | Value
---|---
Audio Length (seconds) | 3
Sampling Rate (hertz) | 22,050
Window Length (frames) | 2,048
Overlap Length (frames) | 512
FFT Length (frames) | 2,048
Num Bands (filters) | 128

To optimize the image for the architecture, scales and whitespaces are
eliminated from the spectrograms, and images are reshaped to the dimensions
$(224,224)$ to be compatible with ResNet input. The finalized input images are
presented in Fig 4.

Figure 4: Image inputs

###  4.2 Baseline

The study starts with implementing two simpler machine learning models as
baselines. K-Nearest Neighbors (KNN) and Support Vector Machines (SVM) are
being implemented on time-series data as the input. The performance of these
two models can be seen via Table V.

TABLE V: Machine Learning Method Accuracy Methods | Raw data | STFT Spectrogram | Mel Spectrogram
---|---|---|---
KNN: |  |  |
n = 10 | 0.21 | 0.51 | 0.52
n = 15 | 0.21 | 0.51 | 0.51
n = 20 | 0.21 | 0.51 | 0.48
SVM | 0.09 | 0.60 | 0.57

Overall, both KNN and SVM perform significantly better on STFT and Mel
Spectrograms than on raw data. The performance on STFT and Mel Spectrograms is
relatively similar for KNN, whereas for SVM, the STFT Spectrogram yields
better accuracy. These results suggest that the spectrogram representations
are much more informative for the KNN and SVM models than raw data. However,
the KNN approach is not significantly sensitive to the human-aligned features
introduced by the mel-scale, and the SVM model works better with STFT
Spectrograms.

###  4.3 Convolutional Neural Networks (CNNs) & Resudual Networks (ResNets)

CNN [27] is the base architecture for ResNet [45], which is used in this
study. Conventional CNN consists of convolutional layer, pooling layer, and
fully connected layer [26].

The Convolutional Layer is the core building block of a CNN. A set of
learnable kernels are consisted in the layer’s parameters, which is convoluted
across the width and height of the input volume during the forward pass. Each
kernel is convoluted across the width and height of the input volume, and the
dot produce between the entries of the kernel and the input is calculated,
producing a 2-dimensional activation map of the kernel. The network learns the
kernel that activate when it detects some specific type of feature at some
spatial position in the input. The primary purpose is to detect local
conjunctions of features from the previous layer. An illustration is shown at
Fig. 5.

Figure 5: Convolutional Layer

The Pooling Layer (also known as down-sampling) reduces the dimensionality of
each feature map while retaining the most important information. Max pooling
(Fig. 7) is preferred in this study due to its ability to highlight dominant
features. The important rhythmic and harmonic patterns are often distinctive
for music genres, which can be preserved by max pooling.

Figure 6: Max Pooling Layer

The Fully Connected layer has its neuron in the previous layer connect to
every neuron in the next layer. The purpose of the fully connected layer is to
perform classification on the features extracted by the convolutional layers
which are downsampled by the pooling layers.

Figure 7: ResNet: Skip Connections [45]

ResNets [45] are a type of CNNs and is used in this study. The core idea
behind ResNets is the ”skip connections”, which allow the gradient to be
directly back-propagated to earlier layers. The purpose is the deal with the
problem of conventional CNNs; they are hard to train due to the ”vanishing
gradients” problem. This means that the earlier layers learn very slowly and
have difficulty making significant changes, which hinges to train very deep
networks. ResNet addresses the problem by adding shortcut connections. This
allows the model to carry forward the previously learned information through
shortcut connections and directly add it to the later layers. As shown by Fig.
7, instead of trying to learn an underlying function $H(x)$, the stacked
nonlinear layers are fitted to another mapping of $F(x):=H(x)-x$. The original
function thus becomes $H(x)=F(x)+x$. The ResNet structure makes it possible to
train very deep networks which can improve the performance in various computer
vision tasks. In this study, ResNet18 is found to be the best performing
ResNets. ResNet18 is a member of the Resnet family that has 18 layers with
weights. The Layers are structured as follows:

  * •

Initial Convolutional Layer (7x7, stride 2)

  * •

Max Pooling Layer (3x3, stride 2)

  * •

2 x Convolutional Block (2 layers with 64 filters)

  * •

2 x Convolutional Block (2 layers with 128 filters)

  * •

2 x Convolutional Block (2 layers with 256 filters)

  * •

2 x Convolutional Block (2 layers with 512 filters)

  * •

Average Pooling Layer

  * •

Fully Connected Layer

CNNs are suitable for the AMGC task when using visual spectrograms as inputs
due to their capacity to effectively learn hierarchical patterns in image-like
data [27]. The spectrograms convert the audio signals into 2-dimensional
representations which show how the spectral density of frequencies varies with
time, essentially transforming the music data into image-like structures. The
hierarchical nature of CNNs makes them able to learn the complex patterns in
the spectrograms that can potentially represent basic musical features such as
chords and timbre to more intricate structures such as melody and rhythm [18].
However, while conventional CNNs are excellent at capturing spatial
hierarchies and patterns, they inherently lack the ability to effectively
capture temporal dynamics over time [18]. These temporal dynamics are crucial
in identifying musical features such as tempo, rhythm, beat, meter, and the
form and structure. This motivates the need of other architecture that are
more suitable for processing sequential data.

###  4.4 Recurrent Neural Networks (RNN) & Bidirectional Gated Recurrent Unit
(Bi-GRU)

Recurrent Neural Networks (RNNs) [28] are a type of neural network that are
designed to recognize patterns in sequences of data, which is often being used
in speech recognition [29] and sound event detection [32]. RNNs make use of a
”memory”, which performs the same task for every element of a sequence, with
the output depended on the previous computations. This is achieved by a
looping structure that passes the information from one step of the network to
the next step. This can be explained by the mathematical expression:

$h_{t}=\tanh{(W_{hh}\cdot h_{t-1}+W_{xh}\cdot x_{t})}$ (5)

where $h_{t}$ is the hidden state at time $t$, $x_{t}$ is the input at time
$t$, and $W_{hh}$ and $W_{xh}$ are weight matrices (learned parameters).
However, basic RNNs also suffer from the ”vanishing gradient” problem and
struggle to learn long-term dependencies due to the decline of gradient during
back-propagation, making them hard to carry information through many time
steps. As a solution, architectures like Long Short-Term Memory (LSTM)[31]
units and Gated Recurrent Units (GRU)[30] were developed, which is able to
selectively forget and remember information over long periods. For this study,
the GRU unit is employed due to its efficiency over LSTMs. The GRU unit is
more complex than the conventional RNNs in terms of the hidden state update
mechanism. It has two gates: a reset gate and an update gate. The reset gate
is used to determine the previous hidden state that will be forgotten, whereas
the update gate determines how much of the new state is the old state and how
much is a learned state. Bidirectional GRUs (Bi-GRU) are an extension of the
basic GRU which allows the model to have both backward and forward information
about the sequence at every time step. The input sequence is first fed in
normal time order for the Forward GRU, and then in reverse time order for the
Backward GRU. The outputs of the two GRUs are concatenated at each time step.
Therefore, for each element in the sequence, Bi-GRU can access the previous
information as well as the future information. The Bi-GRU unit can be
explained by these equations [30] [32]:

Forward GRU:

  * •

Forward update gate: $z^{f}_{t}=sigmoid(W^{f}_{z}\cdot[h^{f}_{t-1},x_{t}])$

  * •

Forward reset gate: $r^{f}_{t}=sigmoid(W^{f}_{r}\cdot[h^{f}_{t-1},x_{t}])$

  * •

Forward candidate hidden state:
$\tilde{h}^{f}_{t}=tanh(W^{f}\cdot[r^{f}_{t}\cdot h^{f}_{t-1},x_{t}])$

  * •

Forward final hidden state: $h^{f}_{t}=(1-z^{f}_{t})\cdot
h^{f}_{t-1}+z^{f}_{t}\cdot\tilde{h}^{f}_{t}$

Backward GRU:

  * •

Backward update gate: $z^{b}_{t}=sigmoid(W^{b}_{z}\cdot[h^{b}_{t+1},x_{t}])$

  * •

Backward reset gate: $r^{b}_{t}=sigmoid(W^{b}_{r}\cdot[h^{b}_{t+1},x_{t}])$

  * •

Backward candidate hidden state:
$\tilde{h}^{b}_{t}=tanh(W^{b}\cdot[r^{b}_{t}\cdot h^{b}_{t+1},x_{t}])$

  * •

Backward final hidden state: $h^{b}_{t}=(1-z^{b}_{t})\cdot
h^{b}_{t+1}+z^{b}_{t}\cdot\tilde{h}^{b}_{t}$

where in these equations:

  * •

The superscript $f$ refers to parameters or states associated with the forward
GRU, and the superscript $b$ refers to those associated with the backward GRU.

  * •

$x_{t}$ is the input at time $t$

  * •

$h^{f}_{t-1}$ and $h^{b}_{t+1}$ are the hidden states of the previous time
step in the forward and backward GRUs, respectively

  * •

$W^{f}_{z}$, $W^{f}_{r}$, $W^{f}$, $W^{b}_{z}$, $W^{b}_{r}$, $W^{b}$ are
weight matrices learned during training for the forward and backward GRUs,
respectively

  * •

The sigmoid function squashes the output between 0 and 1, and the $tanh$
function squashes the output between -1 and 1

The Bi-GRU offers an advantage for music genre classification tasks. By
processing data in both directions, it captures interdependencies between
musical elements across time. This context is key in music, for example, the
understanding of a musical note may depend on the notes that came before it as
well as the notes after it. Therefore, Bi-GRU is expected to supplement the
CNN architecture to provide a more comprehensive features for genre
classification.

###  4.5 Proposed Architecture

The key idea of the proposed architecture is to combine two robust deep
learning architectures, Residual Network (ResNet) and Bidirectional Gated
Recurrent Unit (Bi-GRU), to provide a richer set of features for
classification which can recognise both spatial hierarchical dependencies and
temporal dependencies of the input. The architecture leverages the strengths
of both ResNet18 and Bi-GRU, targeting two crucial aspects of the complex data
structure. ResNet excels at extracting the intricate spatial hierarchical
dependencies from input data which makes it excellent at handling visual
spectrograms. Bi-GRU, on the other hand, capture temporal dependencies which
is important with music data. By combining two structures, a musical piece can
be studied with more comprehensive features for genre classification. The
architecture combines ResNet and GRU in a parallel way where the input
spectrogram is processed by two distinct pathways: a ResNet pathway and a Bi-
GRU pathway before being concatenated and sent to a final fully connected
classification layer. The model architecture is illustrated as Fig. 8. A
summary of the architecture is as follows:

  * •

ResNet pathway

    * –

ResNet18 (pre-trained) model (excluding the last layer)

    * –

Adaptive Max Pooling Layer (1x1)

    * –

Fully Connected Layer (512 input features, 256 output features)

    * –

Dropout Layer (0.5 dropout rate)

    * –

Batch Normalization Layer

  * •

Bi-GRU pathway

    * –

Reshape

    * –

GRU Layer (hidden size 256, 1 layer, bidirectional)

    * –

Concatenation of forward output and backward output (512 features)

    * –

Fully Connected Layer (512 input features, 256 output features)

  * •

Concatenation of ResNet and GRU outputs

  * •

Dropout Layer (0.5 dropout rate)

  * •

Fully Connected Layer (512 input features, classify into 10 genres)

Figure 8: Parallel ResNet-BiGRU

For the ResNet pathway, the model uses a pre-trained ResNet18 model as the
feature extractor to process the visual spectrograms. The last layer of the
original ResNet18 is excluded, since the last layer classifies the input. This
pathway begins by applying the ResNet model to the input data. The output is
then passed through a max pooling 2-dimensional layer, which reduces the
spatial dimensions to $1\times 1$ while preserving the number of feature maps.
After the pooling, the output tensor is flattened and passed through a fully
connected layer to reduce the number of features to 256. A dropout layer is
applied for regularisation and a batch normalisation layer is applied for
stabilising learning and reducing generalisation error. The ResNet pathway
outputs a tensor with 256 features.

For the Bi-GRU pathway, the input dimensions are permuted to take the width of
the image as a time sequence, where each column of pixels is treated as a
timestep. The output is reshaped and then fed into a Bi-GRU. The pathway then
takes the last step of the forward direction and the first step of the
backward direction and concatenates them to form the final output, which has
256 features.

The outputs of the ResNet and Bi-GRU pathways are concatenated along the
feature dimension, resulting in 512 features. This is then passed through a
fully connected layer to classify the input to 10 genres. The results of the
classification will be analysed in the next section.

##  5 Results

###  5.1 Metrics

The metrics used to evaluate the performance is as follows:

  * •

Accuracy: accuracy refers to the total number of correct predictions divided
by the total number of predictions.

$Accuracy=\frac{TP+TN}{TP+TN+FP+FN}$ (6)

  * •

Precision: precision is the total number of correct positive predictions
divided by the total number of positive predictions.

$Precision=\frac{TP}{TP+FP}$ (7)

  * •

Recall: recall is the total number of correct positive predictions divided by
the total number of actual positives.

$Recall=\frac{TP}{TP+FN}$ (8)

  * •

F1-score: F1 score is the harmonic mean of precision and recall.

$F1\textnormal{-}Score=2\cdot\frac{Precision\cdot Recall}{Precision+Recall}$
(9)

The weighted average is employed given the context of multi-class
classification, which calculates the above metrics for each class individually
and then takes an average.

###  5.2 Train/test Split

To prepare the data into training and testing, a $DataLoader$ object is first
created to load the images and labels. Subsequently, since the data is being
augmented, $GroupShuffleSplit$ is implemented to prevent ’data leakage’, which
leads to overly optimistic performance during training but don’t generalize
well to unseen data (as shown by Table VI). Rather than distributing the data
across both the training and testing group, $GroupShuffleSplit$ segregate the
images from the same song into either the training or testing group. This
ensures a more robust attempt to train a model that learns general musical
features, which gives a more realistic evaluation of how the model would
perform in real world (Table VII). The training set accounts for 80% of the
data whereas the testing set accounts for 20% of the data. The reproducibility
is maintained by setting a random see ($random\\_state=42$).

TABLE VI: Model Performance Without GroupShuffleSplit Models | STFT Spectrogram | Mel Spectrogram
---|---|---
| Precision | Recall | F1-score | Accuracy | Precision | Recall | F1-score | Accuracy
CNN | 0.61 | 0.61 | 0.61 | 0.61 | 0.66 | 0.66 | 0.66 | 0.66
Bi-GRU | 0.71 | 0.71 | 0.71 | 0.71 | 0.78 | 0.78 | 0.78 | 0.78
ResNet18 | 0.70 | 0.70 | 0.70 | 0.70 | 0.87 | 0.87 | 0.87 | 0.87
ResNet-BiGRU | 0.86 | 0.86 | 0.86 | 0.86 | 0.90 | 0.90 | 0.90 | 0.90

###  5.3 Results & Analysis

The training and testing process were conducted using early stopping. This
study explores four architectures: CNN, Bi-GRU, ResNet18, and the hybrid model
ResNet-BiGRU. Each model was trained and evaluated based on two types of
spectrogram representations, namely, the STFT Spectrogram and Mel Spectrogram.
The results are summarized in Table VII.

TABLE VII: Model Performance Comparison With GroupShuffleSplit Models | STFT Spectrogram | Mel Spectrogram
---|---|---
| Precision | Recall | F1-score | Accuracy | Precision | Recall | F1-score | Accuracy
CNN | 0.53 | 0.53 | 0.52 | 0.53 | 0.59 | 0.58 | 0.58 | 0.58
Bi-GRU | 0.50 | 0.50 | 0.49 | 0.50 | 0.55 | 0.55 | 0.54 | 0.55
ResNet18 | 0.55 | 0.55 | 0.54 | 0.55 | 0.79 | 0.79 | 0.78 | 0.79
ResNet-BiGRU | 0.75 | 0.75 | 0.75 | 0.75 | 0.82 | 0.81 | 0.81 | 0.81

Across all models, the results highlight a superior performance when utilizing
Mel Spectrogram inputs, underscoring the advantage of this representation.
Notably, the ResNet18 model demonstrated a substantial increase when moving
from STFT Spectrogram to Mel Spectrogram. The highest performing model was the
hybrid ResNet-BiGRU, achieving an accuracy of 0.75 for STFT Spectrogram inputs
and 0.81 for the Mel Spectrogram inputs. For the STFT Spectrogram input, CNN,
Bi-GRU, and ResNet18 showed comparable performance, indicating that the
inherent differences among these architectures might not contribute to
significant performance variations for STFT Spectrograms. Nevertheless, the
hybrid model outperformed the others. This suggests that combining the
strength of both convolutional and recurrent networks can have significant
benefits by concurrently managing hierarchical and temporal dependencies
within the data. On the other hand, for the Mel Spectrogram inputs, ResNet18
and the hybrid model performs similar and significantly outperforms the rest.
This observation hints that Mel Spectrograms may provide a more effective
representation of spatial data in the context of these models.

(a) CNN

(b) Bi-GRU

(c) ResNet18

Figure 9: Accuracy and loss curves (input: Mel Spectrograms)

(a) Accuracy curve

(b) Loss curve

Figure 10: Accuracy and loss curves for ResNet-BiGRU (input: Mel Spectrograms)

To better analyse the model’s learning patterns, the accuracy and loss across
epochs is monitored. Fig. 9 and Fig. 10 illustrate the accuracy and loss
curves for the four models when using Mel Spectrogram as input. All models
implement early stopping with a patience setting of 10 epochs. From the
results, it can be seen that all of the three baseline models (CNN, Bi-GRU,
ResNet18) exhibit signs of overfitting, especially with the CNN and ResNet18.
This could be due to the complexity of the models with the given amount of
data. The Bi-GRU model appears to manage overfitting somewhat better.
Therefore, for the hybrid model, a combination of batch normalisation and
dropout layer is introduced for the ResNet path to deal with the overfitting
issue. This addition allows the hybrid model to run through all 100 epochs
instead of triggering early stopping. However, the accuracy plateaus at
approximately 0.80. It is also worth noticing that the accuracy did not
increase for more complex model using ResNet50, and increasing the number of
layers for the Bi-GRU. This may indicate that the number of data is not large
enough for the complexity of the model, or that the data itself may not be
suitable for more complex architecture.

(a) CNN

(b) Bi-GRU

(c) ResNet18

(d) ResNet-BiGRU

Figure 11: Confusion Matrices (input: STFT Spectrograms)

(a) CNN

(b) Bi-GRU

(c) ResNet18

(d) ResNet-BiGRU

Figure 12: Confusion Matrices (input: Mel Spectrograms)

Fig. 11 and Fig. 12 display the confusion matrices for the two input
representations. Overall, the model performs best when classifying the ’metal’
genre and struggles the most with the ’rock’ genre. This disparity may stem
from the distinctive characteristics of these two genres. The ’metal’ genre
typically features highly amplified distortion, extended guitar solos,
emphatic beats, and overall loudness [42]. These elements create unique
patterns in the spectrograms, leading to better classification performance for
this genre. In contrast, the ’rock’ genre encompasses a range of styles,
including rock and roll, electric blues, jazz music, country rock, even heavy
metal[43]. This variety of sub-genres within ’rock’ can have more varied
patterns in the spectrograms, making it more challenging for the model to
generalise. This might also explains the observation that a lot of the ’rock’
class is being classified as ’disco’ or ’country’. The models also struggled
to distinguish between ’reggae’ and ’hiphop’. This confusion could arise from
the overlapping musical elements and structure present in these genres. They
share similar rhythmic components, often featuring a strong, syncopated
bassline and rhythmic spoken or chanted lyrics[44]. These commonalities could
result in analogous patterns in the spectrograms, leading to classification
errors. However, the ResNet-BiGRU hybrid model demonstrates better performance
in distinguishing between ’reggae’ and ’hiphop’ genres when compared to the
standalone CNN, Bi-GRU, and ResNet18 models. This could indicate that ’reggae’
and ’hiphop’, despite having overlapping elements, might exhibit subtle
differences that the hybrid model can pick up.

##  6 Discussion

###  6.1 Deep Learning Approach

In this study, the application of deep learning models in the AMGC task has
shown promising results when compared with the KNN and SVM baselines. The
comparatively poor performance of the machine learning models (Table V) could
be due to the absence of feature selection, such as using Principal Component
Analysis or Random Projection. This highlights a key advantage of deep
learning neural networks, which is their capability to automatically learn
meaningful features from data, eliminating the need for manual feature
engineering. This makes them more suitable for complex data, such as music
data.

###  6.2 Transfer Learning of Image Knowledge on Audio Data

Another unique aspect of this study involves transferring knowledge from the
image classification field to audio data analysis. By treating spectrograms as
images, the deep learning architectures leveraged the power of established
image classification networks (ResNet). As seen in the results, this approach
is particularly effective. The finding underscores the potential of applying
transfer learning to image and audio data.

###  6.3 Hybrid Approach

The hybrid ResNet-BiGRU model exhibited superior performance when using STFT
Spectrograms as input compared to CNN, Bi-GRU, and ResNet. It also
significantly outperforms CNN and Bi-GRU, and shows comparable performance
with ResNet18 when Mel Spectrograms were used as input. This highlights the
benefit of a hybrid approach which addresses a complex task by using a
combination of architectures, each of which is suitable for a different aspect
of the complexity inherent in the input data. This attests to the potential of
dividing a complex task into parts and using different architectures to tackle
different aspects of features inherent in the input data.

###  6.4 Human Centered

Another intriguing observation is that the difference in performance between
using Mel Spectrograms and STFT Spectrograms depending on the input format and
the underlying classification mechanism. As indicated by Table V, when using
time series data as numerical inputs for machine learning models, the use of
STFT Spectrogram as input features outperforms the use of Mel Spectrograms for
the SVM model, while the KNN model is comparatively unaffected by these two
representations. However, as shown by Table VII, when image inputs are being
used on deep learning models, Mel Spectrograms as input significantly
outperform STFT Spectrograms. Since Mel Spectrograms are scaled using mel-
scale, which better aligns with human auditory perception than the linearly-
spaced frequency bands used in STFT Spectrograms [17], the deep learning
approach using image input is more human centered. This supports the potential
of integrating more human-aligned aspects into the AMGC task.

##  7 Conclusion & Future Work

In conclusion, this study presents an innovative approach to the Automatic
Music Genre Classification (AMGC) task by using visual spectrograms, and
proposes a hybrid model that leverages the strengths of both Residual Networks
(ResNet) and Gated Recurrent Units (GRU). The results reveal an accuracy of
0.81 when using visual Mel Spectrograms as input, significantly outperforming
traditional machine learning models like KNN and SVM, and reaching the 70%
human classification benchmark. This underscores the suitability of deep
learning for the AMGC task. The superior performance of the hybrid model
compared to the others highlights its ability to better capture the spatial
and temporal dynamics inherent in music data, showcasing the benefits of using
a hybrid architecture to tackle different aspects of data complexity. The
results also attest to the potential use of visual input in the AMGC task,
underscoring the effectiveness of transfer learning from the field of image
classification to audio data analysis. Further, the findings demonstrate that
visual Mel Spectrogram input outperforms visual STFT Spectrogram input across
all four deep learning architectures (CNN, Bi-GRU, ResNet18, ResNet-BiGRU),
inspiring a more human-centered approach to the AMGC task.

Some future works can be inspired from this study:

  * •

Input representations: investigate other possible visual representations of
audio data that resembles human auditory system (eg cochleagram)

  * •

Explore additional architectures: investigate other deep learning models to
further attest to the benefit of using visual spectrograms as input

  * •

Multitask learning: the model could be extended to perform other tasks such as
music mood detection, or could be transformed into multi-purpose music
analysis tool

  * •

Generative model: the learned features from the model can be used to perform
generative tasks such as genre shift

  * •

Interpretability: using techniques to interpret the neural network’s decisions
is beneficial in improving the model, as well as reveal novel insights about
music genres.

## Acknowledgments

This research was conducted as part of the SCIE30001 course at The University
of Melbourne. I am grateful for the opportunity to undertake this project and
for the resources provided by the school, which were instrumental in the
completion of this research paper.

I would also like to express my deepest appreciation to my supervisor, Prof.
Lars Kulik, and Nestor Cabello. Their invaluable guidance, continuous support,
and constructive feedback have been instrumental in shaping this research.

##  Appendix A Deployment of a music recommender using deep learning
classification

An intriguing product of this project is a small-scale music recommendation
system WebApp, available at https://deeplearnmuse-3t5mgrwzwa-km.a.run.app/.
The WebApp prompts the user to provide an audio input, which is then converted
into a visual Mel Spectrogram and fed into a pretrained deep learning model
for classification. The classification result, as well as the probabilities
for the 10 genres, are returned to the user. Subsequently, five songs that
have the most similar class distribution to the predicted probability
distribution are recommended to the user. A screenshot of the WebApp is shown
in Fig 13.

Figure 13: Deployment

## References

  * [1] Derek A. Huang, Arianna A. Serafini, Eli J. Pugh. Music Genre Classification. http://cs229.stanford.edu/proj2018/report/21.pdf.
  * [2] Mingwen Dong. Convolutional neural network achieves human-level accuracy in music genre classification. CoRR, abs/1802.09697, 2018.
  * [3] George Tzanetakis and Perry Cook. Musical genre classification of audio signals. IEEE Transactions on speech and audio processing, 10(5):293–302, 2002.
  * [4] F. Pachet and D. Cazaly, “A classification of musical genre,” in Proc. RIAO Content-Based Multimedia Information Access Conf., Paris, France, Mar. 2000.
  * [5] S. Lippens, J. P. Martens and T. De Mulder, ”A comparison of human and automatic musical genre classification,” 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing, Montreal, QC, Canada, 2004, pp. iv-iv, doi: 10.1109/ICASSP.2004.1326806.
  * [6] S. Hacker. MP3: The Definitive Guide. O’Reilly Publishers, 2000.
  * [7] D. Perrot and R. Gjerdigen, “Scanning the dial: An exploration of factors in identification of musical style,” in Proc. Soc. Music Perception Cognition, 1999, p. 88, (abstract).
  * [8] Changsheng Xu, N. C. Maddage, Xi Shao, Fang Cao, and Qi Tian. Musical genre classification using support vector machines. In 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP ’03)., volume 5, pages V–429, April 2003.
  * [9] Y. Panagakis, C. Kotropoulos, and G. R. Arce. Music genre classification via sparse representations of auditory temporal modulations. In 2009 17th European Signal Processing Conference, pages 1–5, Aug 2009.
  * [10] Carlos N. Silla Jr., Alessandro L. Koerich and Celso A. A. Kaestner. A Machine Learning Approach to Automatic Music Genre Classification (2008).
  * [11] Roneel V. Sharan , Hao Xiong and Shlomo Berkovsky. Benchmarking Audio Signal Representation Techniques for Classification with Convolutional Neural Networks (2021).
  * [12] Ervin Sejdić, Igor Djurović, Jin Jiang, Time–frequency feature representation using energy concentration: An overview of recent advances, Digital Signal Processing, Volume 19, Issue 1, 2009, Pages 153-183, ISSN 1051-2004, https://doi.org/10.1016/j.dsp.2007.12.004.
  * [13] Hertel, L.; Phan, H.; Mertins, A. Classifying variable-length audio files with all-convolutional networks and masked global pooling. arXiv 2016, arXiv:1607.02857.
  * [14] Hertel, L.; Phan, H.; Mertins, A. Comparing time and frequency domain for audio event recognition using deep learning. In Proceedings of the International Joint Conference on Neural Networks (IJCNN), Vancouver, CO, Canada, 24–29 July 2016; pp. 3407–3411.
  * [15] Badshah, A.M.; Ahmad, J.; Rahim, N.; Baik, S.W. Speech emotion recognition from spectrograms with deep convolutional neural network. In Proceedings of the International Conference on Platform Technology and Service (PlatCon), Busan, Korea, 13–15 February 2017; pp. 1–5.
  * [16] Davis, S.; Mermelstein, P. Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Trans. Acoust. Speech Signal Process. 1980, 28, 357–366.
  * [17] Stevens, S.S.; Volkmann, J.; Newman, E.B. A scale for the measurement of the psychological magnitude pitch. J. Acoust. Soc. Am. 1937, 8, 185–190.
  * [18] Liu, C., Feng, L., Liu, G. et al. Bottom-up broadcast neural network for music genre classification. Multimed Tools Appl 80, 7313–7331 (2021). https://doi.org/10.1007/s11042-020-09643-6.
  * [19] Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 779-788
  * [20] Alexey Bochkovskiy, Chien-Yao Wang and Hong-Yuan Mark Liao. YOLOv4: Optimal Speed and Accuracy of Object Detection (2020).
  * [21] Mahto, Pooja and Garg, Priyamm and Seth, Pranav and Panda, J, Refining Yolov4 for Vehicle Detection (June 16, 2020). International Journal of Advanced Research in Engineering and Technology (IJARET), 11(5), 2020, pp. 409-419., Available at SSRN: https://ssrn.com/abstract=3628439
  * [22] Yu J, Zhang W. Face Mask Wearing Detection Algorithm Based on Improved YOLO-v4. Sensors. 2021; 21(9):3263. https://doi.org/10.3390/s21093263
  * [23] C. Kumar B., R. Punitha and Mohana, ”YOLOv3 and YOLOv4: Multiple Object Detection for Surveillance Applications,” 2020 Third International Conference on Smart Systems and Inventive Technology (ICSSIT), Tirunelveli, India, 2020, pp. 1316-1321, doi: 10.1109/ICSSIT48917.2020.9214094.
  * [24] Mesaros, A.; Heittola, T.; Benetos, E.; Foster, P.; Lagrange, M.; Virtanen, T.; Plumbley, M.D. Detection and classification of acoustic scenes and events: Outcome of the DCASE 2016 challenge. IEEE/ACM Trans. Audio Speech Lang. Process. 2018, 26, 379–393.
  * [25] Wang, H.; Zou, Y.; Chong, D. Acoustic scene classification with spectrogram processing strategies. In Proceedings of the Detection and Classification of Acoustic Scenes and Events 2020 Workshop (DCASE2020), Tokyo, Japan, 2–4 November 2020; pp. 210–214.
  * [26] Cheng, Y.-H.; Kuo, C.-N. Machine Learning for Music Genre Classification Using Visual Mel Spectrum. Mathematics 2022, 10, 4427. https://doi.org/10.3390/math10234427.
  * [27] Alex Krizhevsky, Ilya Sutskever and Geoffrey E. Hinton. ImageNet Classification with Deep Convolutional Neural Networks (2012).
  * [28] Zachary Chase Lipton. A Critical Review of Recurrent Neural Networks for Sequence Learning (2015). CoRR. abs/1506.00019. http://arxiv.org/abs/1506.00019.
  * [29] A. Graves, A. -r. Mohamed and G. Hinton, ”Speech recognition with deep recurrent neural networks,” 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada, 2013, pp. 6645-6649, doi: 10.1109/ICASSP.2013.6638947.
  * [30] Junyoung Chung, Çaglar Gülçehre, KyungHyun Cho and Yoshua Bengio. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. CoRR (2014). http://arxiv.org/abs/1412.3555.
  * [31] S. Hochreiter and J. Schmidhuber, ”Long Short-Term Memory,” in Neural Computation, vol. 9, no. 8, pp. 1735-1780, 15 Nov. 1997, doi: 10.1162/neco.1997.9.8.1735.
  * [32] Rui Lu and Zhiyao Duan. BIDIRECTIONAL GRU FOR SOUND EVENT DETECTION. Detection and Classification of Acoustic Scenes and Events 2017.
  * [33] Bahuleyan, H. Music genre classification using machine learning techniques. arXiv 2018, arXiv:1804.01149.
  * [34] Pelchat, N.; Gelowitz, C.M. Neural network music genre classification. Can. J. Electr. Comput. Eng. 2020, 43, 170–173.
  * [35] Shah, M.; Pujara, N.; Mangaroliya, K.; Gohil, L.; Vyas, T.; Degadwala, S. Music Genre Classification using Deep Learning. In Proceedings of the 2022 6th International Conference on Computing Methodologies and Communication (ICCMC), Erode, India, 29–31 March 2022; pp. 974–978.
  * [36] Davis, S.; Mermelstein, P. Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Trans. Acoust. Speech Signal Process. 1980, 28, 357–366.
  * [37] Tzanetakis, G.; Cook, P. Musical genre classification of audio signals. IEEE Trans. Speech Audio Process. 2002, 10, 293–302.
  * [38] A. Elbir, H. Bilal Çam, M. Emre Iyican, B. Öztürk and N. Aydin, ”Music Genre Classification and Recommendation by Using Machine Learning Techniques,” 2018 Innovations in Intelligent Systems and Applications Conference (ASYU), Adana, Turkey, 2018, pp. 1-5, doi: 10.1109/ASYU.2018.8554016.
  * [39] Medium. (n.d.). Medium. Retrieved June 22, 2023, from https://medium.com/hackerdawn/music-genre-classification-using-random-forest219fc2446666#:~:text=If%20Music%20is%20a%20Place.
  * [40] S. Sugianto and S. Suyanto, ”Voting-Based Music Genre Classification Using Melspectogram and Convolutional Neural Network,” 2019 International Seminar on Research of Information Technology and Intelligent Systems (ISRITI), Yogyakarta, Indonesia, 2019, pp. 330-333, doi: 10.1109/ISRITI48646.2019.9034644.
  * [41] BrianMcFee,MattMcVicar,StefanBalke,CarlThomé,VincentLostanlen,ColinRaffel,DanaLee,OriolNieto,EricBattenberg, Dan Ellis, Ryuichi Yamamoto, Josh Moore, WZY, Rachel Bittner, Keunwoo Choi, Pius Friesch, Fabian-Robert Stöter, Matt Vollrath, Siddhartha Kumar, nehz, Simon Waloschek, Seth, Rimvydas Naktinis, Douglas Repetto, Curtis ”Fjord” Hawthorne, CJ Carr, João Felipe Santos, JackieWu, Erik, and Adrian Holovaty. librosa/librosa: 0.6.2, August 2018.
  * [42] Walser, Robert (1993). Running with the Devil: Power, Gender, and Madness in Heavy Metal Music. Wesleyan University Press. p. 10. ISBN 0-8195-6260-2.
  * [43] Rock. (2020, May 15). Wikipedia. https://en.wikipedia.org/wiki/Rock
  * [44] Wikipedia Contributors. (2019, October 12). Reggae. Wikipedia; Wikimedia Foundation. https://en.wikipedia.org/wiki/Reggae.
  * [45] Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun. Deep Residual Learning for Image Recognition (2015).
  * [46] Tjandra, A.; Sakti, S.; Neubig, G.; Toda, T.; Adriani, M.; Nakamura, S. Combination of two-dimensional cochleogram and spectrogram features for deep learning-based ASR. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brisbane, Australia, 19–24 April 2015; pp. 4525–4529.