# SEGAA: A Unified Approach to Predicting Age, Gender, and Emotion in Speech

Aron R
School of Computer Science and Engineering
Vellore Institute of Technology
Chennai, India
<EMAIL_ADDRESS>
Indra Sigicharla
School of Computer Science and Engineering
Vellore Institute of Technology
Chennai, India
<EMAIL_ADDRESS>
Dr K Mohanaprasad
School of Electronics Engineering
Vellore Institute of Technology
Chennai, India
<EMAIL_ADDRESS>
Sourabh Tiwari
Voice Intelligence Group
Samsung R&D Institute
Bengaluru, India
<EMAIL_ADDRESS>
Shivani Arora
Voice Intelligence Group
Samsung R&D Institute
Bengaluru, India
<EMAIL_ADDRESS>
Dr Nithya Darisini P S
School of Electronics Engineering
Vellore Institute of Technology
Chennai, India
<EMAIL_ADDRESS>
Chirag Periwal
School of Electronics Engineering
Vellore Institute of Technology
Chennai, India
<EMAIL_ADDRESS>

###### Abstract

The interpretation of human voices holds importance across various
applications. This study ventures into predicting age, gender, and emotion
from vocal cues, a field with vast applications. Voice analysis tech
advancements span domains, from improving customer interactions to enhancing
healthcare and retail experiences. Discerning emotions aids mental health,
while age and gender detection are vital in various contexts. Exploring deep
learning models for these predictions involves comparing single, multi-output,
and sequential models highlighted in this paper. Sourcing suitable data posed
challenges, resulting in the amalgamation of the CREMA-D and EMO-DB datasets.
Prior work showed promise in individual predictions, but limited research
considered all three variables simultaneously. This paper identifies flaws in
an individual model approach and advocates for our novel multi-output learning
architecture Speech-based Emotion Gender and Age Analysis (SEGAA) model. The
experiments suggest that Multi-output models perform comparably to individual
models, efficiently capturing the intricate relationships between variables
and speech inputs, all while achieving improved runtime.

_Keywords_ SER $\cdot$ Speech processing $\cdot$ predictive analysis $\cdot$
Deep learning

##  1 Introduction

In our increasingly digital world, the ability to glean profound insights from
the nuances of human voices has assumed paramount importance. This paper
delves into the captivating domain of predicting age, gender, and emotion
based on vocal cues, a multidisciplinary field teeming with far-reaching
applications.

Voice analysis technologies have rapidly advanced, bringing transformative
breakthroughs to various fields [1]. These advancements not only optimize
customer interactions but also hold the potential to revolutionize healthcare
diagnostics, fundamentally altering our comprehension and engagement with
human communication. In mental healthcare, the ability to discern emotions
offers an opportunity to improve emotional and behavioral disorders [2, 3].
Moreover, the application of this technology extends to the retail sector,
where it enhances the consumer experience [4]. Simultaneously, the ability to
detect age and gender variables finds relevance in evaluating the mental
health requirements of distinct demographic groups [5]. It also plays a
crucial role in e-services and policy formulation [6]. Thus, the ability to
discern age, gender, and emotion from voice data emerges as a multifaceted
field with a plethora of discernible applications.

###  1.1 Prior art

This research endeavour embarks on a comprehensive exploration of advanced
deep learning architectures tailored for the prediction of age, gender, and
emotional states. Additionally, it undertakes an exhaustive comparative
analysis, meticulously examining diverse methodologies. These methodologies
include individual models, where a single model is used to predict a single
variable; multi-output models, which employ a single model to predict all
three variables simultaneously; and sequential models, which cascade
individual models to create a sequence representing the three variables at
distinct stages. This scrutiny aims to elucidate the effectiveness of these
approaches in the vital task of detecting these three pivotal variables.

Before beginning the experiments, addressing the critical challenge of
sourcing a dataset that encompasses all three target labels was necessary.
Consequently, a rigorous review of popular, openly available speech datasets
was undertaken to test the models. Datasets like RAVDESS and IEMOCAP [7, 8]
were found to have emotion labels but lacked age and gender annotations. The
TESS dataset [9] featured only two speakers, rendering it unsuitable for our
purposes despite including all three variables. Similarly, the DES dataset
[10] comprised just four speakers, limiting its utility. The Common Voice
dataset [11] had age and gender labels but omitted emotion labels. As a
result, the CREMA-D and EMO-DB datasets [12, 13] were identified as the only
suitable sources containing all three labels and adequate data. An aggregate
dataset was created by amalgamating these two datasets to facilitate our
experiments.

Prior research has produced remarkable results in predicting individual
variables using dedicated models. For instance, [14, 15] achieved impressive
accuracy rates of 90.47% and 92.73% on CREMA-D for emotion prediction using 1D
CNNs and other deep learning architectures like GRU and LSTMs. Meanwhile, [16]
attained an accuracy of 76.83% using their copy-pasta architecture.
Furthermore, [17] detailed a multilingual speech-based gender classification
method employing time-frequency features and the SVM classifier, achieving 81%
classification accuracy on the EMO-DB dataset for gender classification.
Several studies [18, 15] have explored the prediction of age and gender from
speech, employing diverse methodologies involving MLPs and CNNs. Notably,
limited research has been conducted on predicting all three variables—namely,
age, emotion, and gender—from speech data.

While [19] proposed a "one-source-to-detect-all" solution for predicting age,
gender, and emotion from one model, we identified methodological flaws in
their approach. Specifically, these studies utilized three separate datasets
to train three distinct tasks, a strategy that may not be optimal for enabling
a model to discern all three variables from a single speech source.

The motivation behind employing multi-output models lies in their ability to
concurrently specify the relationships between multiple outcome variables (Y)
and feature variables (X). As underscored by [20], this approach acknowledges
the potential interdependencies between these outcome variables, resulting in
enhanced statistical power or improved predictive accuracy. Moreover, [20]
observed that variations in dependency structures within the covariates
influenced the performance of predictive methods. Notably, this impact was
consistent for both univariate and multivariate approaches, indicating that
neither approach held a definitive advantage concerning the dependency
structures within the covariates.

Given these considerations, this study initiates a series of experiments to
determine whether multi-output models or univariate approaches more
effectively classify age, gender, and emotion from speech. This paper aims to
contribute valuable insights to the ongoing discourse surrounding the optimal
approach for addressing this multifaceted predictive task.

The rest of the article is structured as follows: Section 2 details the
methodology of various models used in this experiment, discusses the database
and explains the feature extraction along with our novel SEGAA model. Section
3 elaborates on the results based on the experimental study. Section 4
presents the conclusion of this article.

##  2 Methodology

###  2.1 Dataset Description

Given these considerations, this study initiates a series of experiments to
determine whether multi-output models or univariate approaches more
effectively classify age, gender, and emotion from speech. This paper aims to
contribute valuable insights to the ongoing discourse surrounding the optimal
approach for addressing this multifaceted predictive task.

The EMO-DB database, a publicly accessible German emotional database, was
developed by the Institute of Communication Science at the Technical
University in Berlin, Germany. The data collection involved ten proficient
speakers, equally distributed between genders (five males and five females).
The database comprises 535 speech utterances covering seven distinct emotions:
anger, boredom, anxiety, happiness, sadness, disgust, and neutrality.

Along with the EMO-DB dataset, the CREMA-D dataset was also used. CREMA-D
consists of 7,442 authentic speech clips from a diverse group of 91 actors.
The actors include 48 males and 43 females, with ages ranging from 20 to 74.
They represent diverse racial and ethnic backgrounds, including African
American, Asian, Caucasian, Hispanic, and Unspecified. During the recording
process, the actors delivered a set of 12 distinct sentences, each expressed
with one of six specific emotions: Anger, Disgust, Fear, Happiness,
Neutrality, and Sadness. The study focused on emotions, and data samples
labelled "neutral" were integrated into the "Neutrality" class.

The primary goal with respect to the data augmentation part of this study is
to enhance the diversity and quality of the dataset to improve the performance
of emotion recognition models. The age and gender of the speakers were
extracted from the speaker information provided and appended to the
corresponding data sample. Consequently, our comprehensive dataset comprised a
fusion of the EMO-DB and CREMA-D datasets.

The study focused on emotions, and data samples labelled "neutral" were
integrated into the "Neutrality" class. Simultaneously, we omitted data
samples with the label "boredom" as we narrowed our scope to include the
following emotions: Anger, Disgust, Fear, Happiness, Neutrality, and Sadness.

Figure 1: Workflow Diagram

###  2.2 Feature Extraction / Pre-processing

We implemented audio processing functions to augment the original data
samples. These functions include noise addition, time stretching, pitch
shifting, and signal shifting. The noise addition process introduces random
noise to the audio data, while time stretching alters the duration of the
samples. Pitch shifting, on the other hand, modifies the audio pitch, and
signal shifting shifts the audio signal along the time axis.

Our feature extraction pipeline involves calculating essential audio features
such as the Zero-Crossing Rate (ZCR), Root Mean Square Energy (RMSE), and Mel-
frequency cepstral coefficients (MFCC). These features are widely used in
audio and speech processing to capture relevant characteristics of the audio
signals.

The augmentation techniques were applied to the original audio samples to
construct the final dataset. Audio features were extracted for each sample,
including the original data and its augmented versions. This process enabled
the generation of diverse features, encompassing pitch, duration, and signal
properties variations. The resulting dataset showcases a significant increase
in size and diversity.

We split the dataset into input features (X) and target labels (Y). Y was one-
hot encoded to represent the categorical variables. The dataset was then
divided into training, testing, and validation sets, using a stratified
approach to ensure class distribution balance. Input features were
standardized using the Standard Scaler to enhance model convergence and
performance. To maintain consistency, the testing and validation sets were
transformed using the scaler fitted to the training set. This is then used to
train and test the MLP and the SEGAA models, which finally output the age,
gender and emotion results as described in Figure 1. The results of the
respective models will then be compared and inferred from.

Sigmoid activation is used for gender prediction with binary cross-entropy
loss, while softmax activation is employed for emotion and age prediction with
categorical cross-entropy. The optimization process relies on Stochastic
Gradient Descent (SGD) with a learning rate of 0.0005, a decay rate of 1e-6,
momentum set to 0.9, and Nesterov momentum. The model undergoes training for
200 epochs with a batch size of 32, during which validation accuracy metrics
for gender, emotion, and age predictions are continuously monitored.

###  2.3 Models

####  2.3.1 Individual Models

##### Individual MLP:

The designed multi-layer perceptron architecture incorporates fully connected
layers. It commences with an input layer customized to accommodate the
extracted features, followed by several hidden layers. The hidden layers
consist of 2048, 1024, 512, and 64 neurons as described in figure 2, equipped
with Rectified Linear Unit (ReLU) activation functions. A dropout layer with a
rate of 0.25 is introduced to address the overfitting issue. For making
predictions specific to each label, distinct activation functions are applied
in the output layers.

Sigmoid activation is used for gender prediction with binary cross-entropy
loss, while softmax activation is employed for emotion and age prediction with
categorical cross-entropy. The optimization process relies on Stochastic
Gradient Descent (SGD) with a learning rate of 0.0005, a decay rate of 1e-6,
momentum set to 0.9, and Nesterov momentum. The model undergoes training for
200 epochs with a batch size of 32, during which validation accuracy metrics
for gender, emotion, and age predictions are continuously monitored.

Figure 2: Model architecture for Individual MLP

##### Individual SEGAA (Gen - 0):

The network comprises several key layers in this architecture. Initially,
there is a convolutional layer featuring 256 filters, a kernel size of 5, and
a stride of 1. This layer is augmented with batch normalization and is
succeeded by max pooling with a pool size of 5 and a stride of 2. Following
this, another convolutional layer has 128 filters, a kernel size of 5, and a
stride of 1. This layer incorporates batch normalization, max pooling with a
pool size of 5 and a stride of 2, and dropout at a rate of 20%. Finally, the
architecture culminates with a convolutional layer employing 64 filters, a
kernel size of 5, and a stride of 1. Similar to the previous layers, it
includes batch normalization and max pooling.

After these convolutional layers, a flattening operation is performed, leading
to a shared densely connected layer comprised of 32 neurons. Each of these
neurons benefits from batch normalization and dropout, with a rate set at 20%.
The hyperparameters include a convolutional kernel size of 5, a dropout rate
of 20%, the utilization of the Adam optimizer for efficient optimization, and
the implementation of an early stopping strategy with a patience of 5 epochs.
A learning rate reduction strategy was also implemented, featuring a patience
of 3 epochs and a reduction factor of 0.5.

The models employed for ’emotion’ and ’age’ predictions utilize softmax
activation functions in their final layers, each comprising six neurons. In
contrast, the ’gender’ prediction model employs binary softmax activation in
its final layer, which consists of 2 neurons.

##### Individual SEGAA:

This model architecture is an improvement made upon the previously mentioned
SEGAA Gen-0. The architecture comprises three convolutional blocks, each
followed by Batch Normalization, Max Pooling, and Dropout layers, facilitating
feature extraction and dimensionality reduction. Subsequently, a Flatten layer
consolidates the extracted information, leading to a densely connected layer
with 64 neurons, further normalized and regularized using Batch Normalization
and Dropout as described in figure 3. Key hyperparameters include a kernel
size of 3 and a stride of 1 for the Convolutional layers, dropout rates set at
0.3, and the adoption of the Nadam optimizer for efficient optimization. The
model undergoes training for 200 epochs with a batch size of 16, and
validation accuracy metrics are continuously monitored. Early stopping and
learning rate reduction callbacks are strategically used to ensure optimal
convergence.

The models employed for ’emotion’ and ’age’ predictions utilize softmax
activation functions in their final layers, comprising six neurons. In
contrast, the ’gender’ prediction model employs binary softmax activation in
its final layer, which consists of 2 neurons.

Figure 3: Model architecture for Individual SEGAA

####  2.3.2 Multi-output Models

##### Multi-output MLP:

The devised multi-layer perceptron architecture integrates fully connected
layers. The input layer is tailored to accommodate the extracted features,
leading to subsequent hidden layers, each with 2048, 1024, 512, and 64 neurons
as described in figure 4, and employing Rectified Linear Unit (ReLU)
activation functions. A dropout layer with a rate of 0.25 is introduced to
alleviate overfitting. For label-specific predictions, output layers employ
distinct activation functions: sigmoid for gender prediction and softmax for
both emotion and age prediction, reflecting the multi-class nature of these
tasks. A stochastic Gradient Descent (SGD) optimizer is adopted, featuring a
learning rate of 0.0005, a decay rate of 1e-6, momentum set to 0.9, and
Nesterov momentum. The model is trained over 200 epochs with a batch size of
32 while monitoring accuracy metrics for gender, emotion, and age predictions
on both training and validation datasets.

Figure 4: Model architecture for Multi-output MLP

##### Multi-output SEGAA (Gen - 0):

In this architecture, the following layers are used: first, a convolutional
layer with 256 filters, kernel size of 5, and stride of 1, augmented with
batch normalization and then max pooling with pool size 5 and stride 2.
Subsequently, a convolutional layer with 128 filters, kernel size of 5, and
stride of 1, combined with batch normalization, max pooling with pool size 5
and stride 2, and drop out at a 20% rate. Lastly, a convolutional layer with
64 filters, kernel size of 5, and stride of 1, along with batch normalization
and max pooling, completes the cascade.

Following these layers, a flatten operation leads to a shared densely
connected layer composed of 32 neurons, each enhanced with batch normalization
and dropout set at a rate of 20%. Three distinct output layers then come into
play, with softmax activation applied for ’emotion’ and ’age’ predictions and
binary softmax activation for ’gender’ predictions. The hyperparameters
governing the model’s effectiveness were diligently selected: a convolutional
kernel size of 5, dropout rate of 20%, utilization of the Adam optimizer for
efficient optimization, and an early stopping strategy with a patience of 5
epochs. A learning rate reduction strategy was also embedded, with patience of
3 epochs and a reduction factor of 0.5.

##### Multi-output SEGAA:

The architecture consists of three convolutional blocks, each followed by
Batch Normalization, Max Pooling, and Dropout layers, promoting feature
extraction and dimensionality reduction. Subsequently, a flattened layer
aggregates the information, leading to a densely connected layer with 64
neurons, further normalized and regularized with Batch Normalization and
Dropout as described in figure 5. Three separate output layers cater to each
label category: ’emotion,’ ’age,’ and ’gender,’ utilizing softmax activation
for the former two and binary softmax for the latter.

Hyperparameters include a kernel size of 3 and stride of 1 for the
Convolutional layers, dropout rates of 0.3, and a Nadam optimizer for
optimization. The model is trained over 200 epochs using a batch size of 16,
monitored by validation accuracy metrics. Early stopping and learning rate
reduction callbacks are utilized to ensure optimal convergence.

Figure 5: Model architecture for Multi-output SEGAA (Speech-based Emotion,
Gender, Age Analysis)

####  2.3.3 Sequential Models

In this experimental setup, the individual models described in Section 3.3.1
were cascaded sequentially to predict the first variable from the speech
features, followed by predicting the second variable from the speech features
and the previously predicted first variable, and finally predicting the third
variable from the speech features and the previously predicted second variable
as described in figure 5. This experiment was conducted in three different
sequences: first for the variables emotion, gender, and age; then for gender,
age, and emotion; and lastly for age, emotion, and gender.

Figure 6: Model architecture for Sequential models

##  3 Results

This research investigates the task of speech-based detection of emotion,
gender, and age using a diverse range of machine-learning models. Our primary
objective is to develop accurate and robust models for classifying these three
attributes from audio data. The experimental framework includes individual and
multi-output Multi-Layer Perceptron (MLP) as well as SEGAA architectures.
Additionally, we investigate unique sequences for predicting these attributes
with our SEGAA models.

###  3.1 Individual Models

####  3.1.1 Emotion Detection

For emotion detection, both individual SEGAA and MLP models exhibit robust
performance. The SEGAA model attains an accuracy of 96%, while the MLP model
achieves 94% accuracy. These models demonstrate well-balanced precision,
recall, and F1 scores, ranging from 0.94 to 0.96.

####  3.1.2 Gender Detection

The individual models demonstrate exceptional accuracy for gender detection.
The SEGAA model attains a flawless accuracy of 100%, while the MLP model
achieves 98%. Precision, recall, and F1 scores consistently reach 1.00 for
both models.

####  3.1.3 Age Detection

Age detection, too, benefits from the individual SEGAA and MLP models, with
the SEGAA model achieving 95% accuracy and the MLP model reaching 92%. These
models display high precision, recall, and F1 scores, ranging from 0.92 to
0.95.

###  3.2 Multi-output Models

Our investigation extends to multi-output SEGAA and MLP models, which tries to
simultaneously predict emotion, gender, and age in that order. These models
yield competitive results with those of the individual models, with accuracy
values spanning from 84% to 99% across generations. Although the multi-output
SEGAA Gen-0 model maintains strong precision and F1 scores, there is a slight
decrease in recall for certain attributes, particularly in emotion detection.
In contrast, the proposed SEGAA model effectively addresses these limitations,
demonstrating exceptional performance across all aspects, encompassing
emotion, age, and gender detection, which is highlighted in Figure 7.

###  3.3 Sequential Models

Exploration of various sequences for predicting emotion, gender, and age using
SEGAA models reveals distinct strengths across different orderings. The EGA
sequence (Emotion → Gender → Age) notably excels in gender detection,
delivering respectable accuracy in emotion detection. This sequence
demonstrated has consistent F1 scores above 0.90 for all three attributes
which talks extensively about the sequential model metrics.

Table 1: MLP & SEGAA Model Metrics Target | Accuracy | Precision | Recall | F1 Score
---|---|---|---|---
Individual MLP
Age | 92% | 0.92 | 0.92 | 0.92
Emotion | 94% | 0.94 | 0.94 | 0.94
Gender | 98% | 0.98 | 0.98 | 0.98
Multi-output MLP
Age | 94% | 0.94 | 0.94 | 0.94
Emotion | 95% | 0.95 | 0.95 | 0.95
Gender | 98% | 0.98 | 0.98 | 0.98
Individual SEGAA
Age | 95% | 0.95 | 0.95 | 0.95
Emotion | 96% | 0.96 | 0.96 | 0.96
Gender | 100% | 1 | 1 | 1
Multi-output SEGAA (Gen-0)
Age | 90% | 0.93 | 0.9 | 0.9
Emotion | 84% | 0.86 | 0.84 | 0.84
Gender | 98% | 0.98 | 0.98 | 0.98
Multi-output SEGAA
Age | 94% | 0.96 | 0.94 | 0.95
Emotion | 95% | 0.95 | 0.95 | 0.95
Gender | 99% | 0.99 | 0.99 | 0.99

Table 2: Sequential Model Metrics Target | Accuracy | Precision | Recall | F1 Score
---|---|---|---|---
CNN Sequence 1 (Gender → Age → Emotion)
Gender | 99% | 0.99 | 0.99 | 0.99
Age | 94% | 0.95 | 0.95 | 0.95
Emotion | 94% | 0.94 | 0.94 | 0.94
CNN Sequence 2 (Age → Emotion → Gender)
Age | 94% | 0.94 | 0.94 | 0.94
Emotion | 91% | 0.91 | 0.91 | 0.91
Gender | 99% | 0.99 | 0.99 | 0.99
CNN Sequence 3 (Emotion → Gender → Age)
Emotion | 95% | 0.95 | 0.95 | 0.95
Gender | 99% | 0.99 | 0.99 | 0.99
Age | 92% | 0.93 | 0.92 | 0.92
MLP Sequence (Emotion → Gender → Age)
Emotion | 91% | 0.91 | 0.91 | 0.91
Gender | 90% | 0.91 | 0.9 | 0.91
Age | 92% | 0.93 | 0.92 | 0.92

Table 1 encapsulates the performance metrics of individual and multi-output
models, providing a detailed assessment of accuracy, precision, recall, and F1
scores for age, emotion, and gender detection. For individual models, both
SEGAA and MLP models showcase high accuracy, with the SEGAA model achieving
96% accuracy in emotion detection and demonstrating gender detection at 100%,
with more information on the models highlighted in Figure 6. Notably, the
multi-output models, particularly the SEGAA model, exhibit competitive results
across all attributes, achieving a remarkable 99% accuracy in gender
detection. While the Gen-0 displays slightly lower scores in emotion
detection, SEGAA addresses this limitation effectively as the Nadam optimizer
present in SEGAA uses Nesterov’s Accelerated Gradient as an extension to the
Adam optimizer used in Gen-0. The multi-output MLP model also maintains high
accuracy and balanced metrics, emphasizing the performance of this model
category in simultaneous prediction tasks.

While all models consistently deliver competitive performance across accuracy
and other metrics, it is noteworthy that the proposed model (Multi-output
SEGAA) emerges as the most efficient choice. These models offer commendable
accuracy while preserving efficiency, an important consideration in real-world
applications where computational resources and low latency are necessary.

(a) Age Confusion Matrix

(b) Emotion Confusion Matrix

(c) Gender Confusion Matrix

Figure 7: Confusion Matrices for Individual SEGAA

Predicting gender through audio has been a pretty easy task for the individual
models. This can be seen in Figure 7a and 7b, where there are barely any
misclassifications, and a high number of true positives for each of the
categories. The same can be said for the task of emotion prediction for these
models, as there aren’t many misclassifications, compared to the number of
right predictions. However, they struggle a bit more when it comes to
identifying "neutral" and "anxiety/fear" emotions. The models also excel at
predicting the age accurately, but it’s important to note that the dataset is
imbalanced, as there are fewer examples of people in their "sixties" and
"seventies" in the dataset compared to other age groups.

(a) Age Confusion Matrix

(b) Emotion Confusion Matrix

(c) Gender Confusion Matrix

Figure 8: Confusion Matrices for Multi-output SEGAA

Comparing the results for Individual SEGAA and Multi-output SEGAA shows that
the model itself is stable, since there is relatively little to no change in
the classifications made by the two models. The Multi-output SEGAA is
comparable to the Individual SEGAA in terms of performance. Individual SEGAA
outperforms the Multi-output SEGAA models marginally in all the tasks, as
shown in Figure 8a and 8b, but the training time taken by the multi-output
SEGAA outweighs this merit, as they are already extremely close in
performance.

(a) Age Confusion Matrix

(b) Emotion Confusion Matrix

(c) Gender Confusion Matrix

Figure 9: Confusion matrices for Sequence 3: (Emotion → Gender → Age)

The performance of Emotion → Gender → Age sequence showcased in Figure 9a and
9b is better than the rest, but it still isn’t as good as the individual or
multi-output SEGAA models, which is because of the sum of errors due to error
from one model propagating to the next, which adds up. This is apparent in the
confusion matrices, where the model hasn’t been able to classify the age of
seventies as well as the SEGAA model, along with the other classes.

##  4 Conclusion

In our experimentation, we conducted a rigorous comparative assessment of
univariate and multi-output models to predict gender, age, and emotion from
speech data, a crucial task with applications spanning various domains. Our
analysis unveiled a noteworthy phenomenon associated with sequentially
chaining univariate models: this approach increased error propagation as the
inaccuracies generated by preceding models were amplified in subsequent
stages. Despite this, univariate models exhibited slightly superior accuracy
and F1 scores performance compared to multi-output models.

It is essential to emphasize that our experimental findings suggest that SEGAA
demonstrates a level of predictive capability comparable to univariate models.
Notably, it excels in capturing the intricate interrelationships between the
variables and speech inputs with commendable efficiency. Furthermore, SEGAA
achieves this without compromising runtime efficiency, making them an
attractive alternative for addressing the complexities of predicting gender,
age, and emotion from speech data. These insights, we believe, hold valuable
implications for both researchers and practitioners in the field.

## References

  * Elena-Bucea et al. [2021] Anca Elena-Bucea, Frederico Cruz-Jesus, Tiago Oliveira, and Pedro Simões Coelho.  Assessing the Role of Age, Education, Gender and Income on the Digital Divide: Evidence for the European Union.  _Information Systems Frontiers_ , 23(4):1007–1021, August 2021.  ISSN 1387-3326, 1572-9419.  doi:10.1007/s10796-020-10012-9.  URL https://link.springer.com/10.1007/s10796-020-10012-9.
  * Jiao et al. [2020] Wen Yan Jiao, Lin Na Wang, Juan Liu, Shuan Feng Fang, Fu Yong Jiao, Massimo Pettoello-Mantovani, and Eli Somekh.  Behavioral and emotional disorders in children during the COVID-19 epidemic.  _The Journal of pediatrics_ , 221:264–266, 2020.  URL https://www.jpeds.com/article/S0022-3476(20)30336-X/abstract.  Publisher: Elsevier.
  * McTeague et al. [2020] Lisa M. McTeague, Benjamin M. Rosenberg, James W. Lopez, David M. Carreon, Julia Huemer, Ying Jiang, Christina F. Chick, Simon B. Eickhoff, and Amit Etkin.  Identification of Common Neural Circuit Disruptions in Emotional Processing Across Psychiatric Disorders.  _American Journal of Psychiatry_ , 177(5):411–421, May 2020.  ISSN 0002-953X, 1535-7228.  doi:10.1176/appi.ajp.2019.18111271.  URL http://ajp.psychiatryonline.org/doi/10.1176/appi.ajp.2019.18111271.
  * Cachero-Martínez and Vázquez-Casielles [2021] Silvia Cachero-Martínez and Rodolfo Vázquez-Casielles.  Building consumer loyalty through e-shopping experiences: The mediating role of emotions.  _Journal of Retailing and Consumer Services_ , 60:102481, 2021.  URL https://www.sciencedirect.com/science/article/pii/S0969698921000473.  Publisher: Elsevier.
  * Gonzales et al. [2020] Gilbert Gonzales, Emilio Loret de Mola, Kyle A. Gavulic, Tara McKay, and Christopher Purcell.  Mental health needs among lesbian, gay, bisexual, and transgender college students during the COVID-19 pandemic.  _Journal of Adolescent Health_ , 67(5):645–648, 2020.  URL https://www.sciencedirect.com/science/article/pii/S1054139X20304882.  Publisher: Elsevier.
  * Bădîrcea et al. [2021] R. M. Bădîrcea, A. G. Manta, N. M. Florea, J. Popescu, F. L. Manta, and S. Puiu.  E-Commerce and the Factors Affecting Its Development in the Age of Digital Technology: Empirical Evidence at EU–27 Level. Sustainability 2022, 14, 101, 2021.  URL https://www.academia.edu/download/79805943/pdf.pdf.
  * Busso et al. [2008] Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette N. Chang, Sungbok Lee, and Shrikanth S. Narayanan.  IEMOCAP: interactive emotional dyadic motion capture database.  _Language Resources and Evaluation_ , 42(4):335–359, December 2008.  ISSN 1574-020X, 1574-0218.  doi:10.1007/s10579-008-9076-6.  URL http://link.springer.com/10.1007/s10579-008-9076-6.
  * Livingstone and Russo [2018] Steven R. Livingstone and Frank A. Russo.  The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English.  _PloS one_ , 13(5):e0196391, 2018.  URL https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0196391&fbclid=IwAR0pMF9vaxEvCucqjm2DJ1TH6CUv7JpBD79vi8qJcCAdHzJjJ4X2pFGDv_E.  Publisher: Public Library of Science.
  * Dupuis and Pichora-Fuller [2010] Kate Dupuis and M. Kathleen Pichora-Fuller.  _Toronto emotional speech set (TESS)_.  University of Toronto, Psychology Department, 2010.
  * Engberg et al. [1997] Inger S. Engberg, Anya Varnich Hansen, Ove Andersen, and Paul Dalsgaard.  Design, recording and verification of a Danish emotional speech database.  In _Fifth European conference on speech communication and technology_ , 1997.
  * Ardila et al. [2020] Rosana Ardila, Megan Branson, Kelly Davis, Michael Henretty, Michael Kohler, Josh Meyer, Reuben Morais, Lindsay Saunders, Francis M. Tyers, and Gregor Weber.  Common Voice: A Massively-Multilingual Speech Corpus, March 2020.  URL http://arxiv.org/abs/1912.06670.  arXiv:1912.06670 [cs].
  * Burkhardt et al. [2005] Felix Burkhardt, Astrid Paeschke, Miriam Rolfes, Walter F. Sendlmeier, and Benjamin Weiss.  A database of German emotional speech.  In _Interspeech_ , volume 5, pages 1517–1520, 2005.  URL https://www.researchgate.net/profile/Felix-Burkhardt-2/publication/221491017_A_database_of_German_emotional_speech/links/00b7d5226f45d66e38000000/A-database-of-German-emotional-speech.pdf.
  * Cao et al. [2014] Houwei Cao, David G. Cooper, Michael K. Keutmann, Ruben C. Gur, Ani Nenkova, and Ragini Verma.  Crema-d: Crowd-sourced emotional multimodal actors dataset.  _IEEE transactions on affective computing_ , 5(4):377–390, 2014.  URL https://ieeexplore.ieee.org/abstract/document/6849440/.  Publisher: IEEE.
  * Ahmed et al. [2023] Md Rayhan Ahmed, Salekul Islam, AKM Muzahidul Islam, and Swakkhar Shatabda.  An ensemble 1D-CNN-LSTM-GRU model with data augmentation for speech emotion recognition.  _Expert Systems with Applications_ , 218:119633, 2023.  URL https://www.sciencedirect.com/science/article/pii/S0957417423001343.  Publisher: Elsevier.
  * Tursunov et al. [2021] Anvarjon Tursunov, Mustaqeem, Joon Yeon Choeh, and Soonil Kwon.  Age and gender recognition using a convolutional neural network with a specially designed multi-attention module through speech spectrograms.  _Sensors_ , 21(17):5892, 2021.  URL https://www.mdpi.com/1424-8220/21/17/5892.  Publisher: MDPI.
  * Pappagari et al. [2021] Raghavendra Pappagari, Jesús Villalba, Piotr Żelasko, Laureano Moro-Velazquez, and Najim Dehak.  Copypaste: An augmentation method for speech emotion recognition.  In _ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 6324–6328. IEEE, 2021.  URL https://ieeexplore.ieee.org/abstract/document/9415077/.
  * Wani et al. [2021] Taiba Majid Wani, Teddy Surya Gunawan, Hasmah Mansor, Syed Asif Ahmad Qadri, Ali Sophian, Eliathamby Ambikairajah, and Eko Ihsanto.  Multilanguage Speech-Based Gender Classification Using Time-Frequency Features and SVM Classifier.  In Jessnor Arif Mat Jizat, Ismail Mohd Khairuddin, Mohd Azraai Mohd Razman, Ahmad Fakhri Ab. Nasir, Mohamad Shaiful Abdul Karim, Abdul Aziz Jaafar, Lim Wei Hong, Anwar P. P. Abdul Majeed, Pengcheng Liu, Hyun Myung, Han-Lim Choi, and Gian-Antonio Susto, editors, _Advances in Robotics, Automation and Data Analytics_ , volume 1350, pages 1–10. Springer International Publishing, Cham, 2021.  ISBN 978-3-030-70916-7 978-3-030-70917-4.  doi:10.1007/978-3-030-70917-4_1.  URL http://link.springer.com/10.1007/978-3-030-70917-4_1.  Series Title: Advances in Intelligent Systems and Computing.
  * Goyal et al. [2020] Saksham Goyal, Vinay Vasanth Patage, and Sourabh Tiwari.  Gender and age group predictions from speech features using multi-layer perceptron model.  In _2020 IEEE 17th India Council International Conference (INDICON)_ , pages 1–6. IEEE, 2020.  URL https://ieeexplore.ieee.org/abstract/document/9342434/.
  * Zaman et al. [2021] Syed Rohit Zaman, Dipan Sadekeen, M. Aqib Alfaz, and Rifat Shahriyar.  One source to detect them all: gender, age, and emotion detection from voice.  In _2021 IEEE 45th Annual Computers, Software, and Applications Conference (COMPSAC)_ , pages 338–343. IEEE, 2021.  URL https://ieeexplore.ieee.org/abstract/document/9529731/.
  * Schmid et al. [2022] Lena Schmid, Alexander Gerharz, Andreas Groll, and Markus Pauly.  Machine Learning for Multi-Output Regression: When should a holistic multivariate approach be preferred over separate univariate ones?, January 2022.  URL http://arxiv.org/abs/2201.05340.  arXiv:2201.05340 [cs, stat].