# Energy Efficiency Challenges of 5G Small Cell Networks

Xiaohu Ge1, Jing Yang1, Hamid Gharavi2, , Yang Sun1
1School of Electronic Information and Communications,
Huazhong University of Science and Technology, Wuhan 430074, Hubei, P. R.
China.
Email: {xhge, yang_jing<EMAIL_ADDRESS>
2National Institute of Standards and Technology (NIST),
Gaithersburg, MD 20899-8920 USA.
Email<EMAIL_ADDRESS>
Submitted to Green Communications and Computing Networks Series of IEEE
Communications Magazine.

###### Abstract

The deployment of a large number of small cells poses new challenges to energy
efficiency, which has often been ignored in fifth generation (5G) cellular
networks. While massive multiple-input multiple outputs (MIMO) will reduce the
transmission power at the expense of higher computational cost, the question
remains as to which computation or transmission power is more important in the
energy efficiency of 5G small cell networks. Thus, the main objective in this
paper is to investigate the computation power based on the Landauer principle.
Simulation results reveal that more than 50% of the energy is consumed by the
computation power at 5G small cell BS’s. Moreover, the computation power of 5G
small cell BS can approach 800 watt when the massive MIMO (e.g., 128 antennas)
is deployed to transmit high volume traffic. This clearly indicates that
computation power optimization can play a major role in the energy efficiency
of small cell networks.

##  I Introduction

With the anticipated high traffic, small cell networks are emerging as an
inevitable solution for 5G cellular networks [1]. In particular, the massive
multiple input multiple-output (MIMO) and millimeter wave technologies are
expected to be deployed towards improving the transmission rate and reduce the
transmission power of 5G mobile communication systems [2]. On the other hand,
more computation power will be required to process anticipated heavy traffic
at small cell base stations (BS’s). Under these conditions, a tradeoff between
computation and transmission power needs to be thoroughly evaluated in order
to achieve energy efficiency optimization for 5G small cell networks.

This has been widely investigated in [3, 4, 5, 6]. Compared with transmission
power, computation power was obviously smaller and usually fixed as a constant
in a traditional energy efficiency evaluation of BS’s [4]. As a consequence,
the energy efficiency investigation of small cell networks has focused on the
optimization of transmission power at BS’s [5]. Furthermore, the BS sleeping
scheme has been considered to improve energy efficiency where the radio
frequency (RF) chains and transmitters of BS’s are closed to save transmission
power [6]. In addition, the computation power of small cell BS’s has been
improved by the volume and complexity of signal processing, which is weighted
by massive MIMO and millimeter wave technologies [7].

When small cell BS’s are ultra-densely deployed in 5G cellular networks[8],
there exist scenarios in which the computation power of BS’s will become
larger than the transmission power of BS’s despite lower power transmission
requirements for small cell BSs.

The transmission rate of 5G mobile communication systems is expected to reach
to an average of 1 Gbps (10 Gbps at the peak rate) [2]. Hence, the huge
traffic has to be handled at the base band units (BBU’s) of small cell BS’s
and then the computation power of signal processing has to be accordingly
improved at BBU’s. Moreover, the cache communications and cloud computing
network architecture will strengthen functions of signal processing and
computing at small cell BS’s. Nonetheless, the computation power of 5G small
cell networks could be predicted to increase in the near future. All the above
reasons trigger us to rethink the roles of computation and transmission power
in 5G small cell networks.

Based on the Landauer principle, we first proposed a computation power model
for 5G small cell networks. Considering that the massive MIMO and millimeter
wave technologies are adopted at small cell BS’s, the impact of the number of
antennas and bandwidths on the computation power of 5G small cell networks is
investigated. Simulation results indicate that the computation power will
consume more than 50% of the energy at 5G small cell BS’s. It is a surprising
result for the energy efficiency optimization of 5G small cell networks.
Finally, future challenges of energy efficiency optimization are discussed for
5G small cell networks and conclusions are drawn in the last section.

##  II Power Consumption at BSs

To evaluate roles of computation and transmission power for BS’s, the total BS
power consumption needs to be analyzed in detail. Therefore, in this section
5G transmission technologies, such as massive MIMO and millimeter wave
technologies will be incorporated for analyzing the power consumption of small
cell BS’s.

###  II-A BS Power Consumption Types

Considering functions and architectures of BS’s, the power consumption at BS’s
is typically classified into three types: transmission power, computation
power and additional power which are described as follows.

  * •

The transmission power corresponds to the energy used by power amplifiers
(PAs) and RF chains, which perform the wireless signals change, i.e., signal
transforming between the base band signals and the wireless radio signals.
Besides, the power consuming at feeders is included as a part of the
transmission power.

  * •

The computation power represents the energy consumed at base band units
(BBU’s) which includes digital single processing functions, management and
control functions for BS’s and the communication functions among the core
network and BS’s. All these operations are executed by softwares and realized
at semiconductor chips.

  * •

The additional power represents the BS power, except for the transmission and
computation power, e.g., the power consumed for maintaining the operation of
BS’s. More specifically, the additional power includes the power lost at the
exchange from the power grid to the main supply, at the exchange between
different direct-current to direct-current (DC-DC) power supply, and the power
consumed for active cooling at BS’s.

The values of three types of consumed power are different depending on the
types of BS. For example, unlike the macro cell BS the small cell BS normally
does not have the active cooling system.

###  II-B Total BS Power Consumption Model

The EARTH project has promoted energy efficiency optimization for wireless
access networks and proposes a framework for the power consumption at BS’s[9].
Based on this energy efficiency framework, the BS is divided into five parts
(see Fig. 1): the antenna interface, the power amplifier, the RF chains, the
BBU, the mains supply, and cooling and direct-current to direct-current (DC-
DC). The power consumed at the power amplifier and the antenna interface
occupy the largest proportion of the total power consumption in macro cell
BS’s, i.e., 57%. The portion of RF chains and BBU s is about 10% and 13%,
respectively. And the proportion of remaining parts is about 20%. To analyze
this in greater detail, the total BS power consumption model is presented as
follows: When the BS is equipped with ${N_{TRX}}$ antennas, the total BS power
consumption ${P_{in}}$ is calculated by
${P_{in}}=\frac{{{P_{PA}}\cdot{N_{TRX}}+{P_{RF}}\cdot{N_{TRX}}+{P_{BB}}}}{{(1-{\sigma_{DC}})(1-{\sigma_{MS}})(1-{\sigma_{cool}})}}$,
where ${P_{PA}}$ is the power of PA per antenna, ${P_{RF}}$ is the RF chain
power per antenna, ${P_{BB}}$ is the power consumed at the BBU,
${\sigma_{DC}}$ is the power loss rate of the DC-DC converter, ${\sigma_{MS}}$
is the power loss rate of the alternating current supply, and
${\sigma_{cool}}$ is the power loss rate of cooling. Based on the expression
of the total BS power consumption,
${P_{PA}}\cdot{N_{TRX}}+{P_{RF}}\cdot{N_{TRX}}$ is the transmission power,
${P_{BB}}$ is the computation power,
$(1-{\sigma_{DC}})(1-{\sigma_{MS}})(1-{\sigma_{cool}})$ is the relationship
between the power loss rate and the total power consumption. The power of PA
per antenna is calculated by
${P_{PA}}=\frac{{{P_{out}}}}{{{\eta_{PA}}\cdot(1-{\sigma_{feed}})}}$, where
${P_{out}}$ is the transmission power at every antenna, ${\eta_{PA}}$ is the
exchange efficiency of PA, and feeder loss is configured as
${\sigma_{feed}}=-3$ dB. For macro cell BS’s and small cell BS’s, the values
of ${\sigma_{DC}}$, ${\sigma_{MS}}$ and ${\sigma_{cool}}$ are configured as as
6%, 7%, 9% and 8%, 10%, 0%, respectively[9]. To simplify calculation, in this
paper the RF chain power per antenna is usually fixed as different constants
corresponding to different types of BS’s. Since ${P_{BB}}$ is obviously less
than the power consumed at other parts of BS’s, the power consumed at the BBU
is fixed as constant in a traditional BS power model[10]. Bear in mind that,
as small cells are expected to be widely deployed in 5G cellular networks, the
distance between BS’s and users will be much shorter, resulting in a
considerable reduction of transmission power. Under these conditions, the BBU
becomes the dominant source of power consumption.

##  III Computation Power Model

Because of the extensive traffic processing at 5G small cell BS’s, the volume
of data processing at 5G small cell BS’s is evaluated by the operation per
second at BBU’s. Furthermore, Landauer’s principle is used to estimate the
computation power consumed for data processing in this section. In this
section we also study the impact of massive MIMO and millimeter wave
technologies on the computation power of 5G small cell BS’s.

###  III-A Computation Power Types

In traditional macro cell BS s the power used at BBU s (BBU is the core unit
of a BS) is small compared with the power consumed by PA s. With the recent
advances of 5G of the massive MIMO and millimeter wave technologies, small
cell BS’s are replacing macro cell BS’s to perform the function of wireless
data transmission in 5G cellular networks. Moreover, the power consumed at
BBU’s is expected to gradually increase because of the massive traffic in 5G
small cell BS’s.

Fig. 1 is a typical logistical architecture of eNodeB BS, i.e., a macro cell
BS in a cellular network. Without a loss of generality, the BBU of a macro
cell BS includes four systems: the base band system, the control system, the
transfer system, and the power system. The detailed functions of these systems
in BBU are described as follows.

  * •

The functions of a base band system include signal filtering, fast Fourier
transform/inverse fast Fourier transform (FFT/IFFT), modulation and
demodulation, digital-pre-distortion (DPD) processing, signal detection, and
wireless channel coding/decoding. Note that, the function of signal processing
used for transmitters and receivers is performed by the BBU.

  * •

The control system takes charge of controlling and managing resource
allocation at BS’s in order to provide control interface between the BS and
other network units. Moreover, communication control protocols are run at the
control system. The control system also provide an interface of man-machine
language (MML) for the local maintain terminal (LMT) to configure the resource
allocation of BBU’s.

  * •

The transfer system connects with the mobility management entity/serving-
gateway (MME/S-GW) of the core network by the S1 interface(see Fig. 1).
Moreover, the control and management information among BS’s are forwarded by
the X2 interface of the transfer system in BBU’s.

  * •

The power system is responsible for power supply, cooling, and monitoring at
BBU’s.

For small cell BS’s, most functions are integrated into a few semiconductor
chips and there is not a single power system. Therefore, the systems of BBU’s
at small cell BS’s is simpler than the systems of BBU’s at macro cell BS’s.

###  III-B Computation Power Model

Based on the four systems in the logistical architecture shown in Fig.1, the
main difficulty is how to calculate the computation power for every logistical
system in BBU’s. To achieve this, we partition a BBU into different parts
based on the hardware architecture as shown in Fig. 2. These consist of DPD,
Filter, CPRI, OFDM, FD, FEC, and CPU where DPD is the digital-pre-distortion
processing part, filter is the hardware used for up/down signal sampling and
filtering, CPRI is the common public radio interface part for connecting to
the core network and RF chains by serial links, OFDM is the hardware used for
FFT and orthogonal frequency-division multiplexing (OFDM)-specific signal
processing, FD is the frequency-Domain processing part, which includes, symbol
mapping/demapping and MIMO equalization, FEC is the forward error correction
which includes the channel coding and decoding, and CPU is the BBU platform
control processor. Based on Landauer’s principle, we estimate the computation
power of semiconductor chips using Giga operations per second (GOPS) and
considering different semiconductor chip techniques. The computation power of
BBU is summed up by the computation power of every hardware part, i.e., every
semiconductor chip at BBU.

Landauer’s principle was proposed in 1961 by Rolf Landauer who attempted to
apply the thermodynamic theory to digital computers. Landauer’s principle
elaborates the relationship between the information process and energy
consumption from the viewpoint of a microscopic degree of freedom in
statistical physics. This is based on a physical principle pertaining to the
lower theoretical limit of energy consumption that corresponds to the
computation. Bear in mind that the concept of entropy in information theory
introduced by Claude Shannon is borrowed from the thermodynamic theory.
Similarly, Landauer’s principle connects these two concepts of information and
energy by using the thermodynamic theory and statistical physics. Therefore,
in this paper Landauer’s principle is first used to analyze the computation
power consumption in 5G small cell networks. More specifically Landauer’s
principle points out that any logically irreversible manipulation of
information, such as the erasure of a bit or the merging of two computation
paths, must be accompanied by a corresponding entropy increase in non-
information-bearing degrees of freedom of the information-processing apparatus
or its environment[11]. In other words, erasing a bit information will consume
more than $kT\ln(2)$ energy in a computing system, where $k$ is the Boltzmann
constant, i.e., $1.38\times{10^{-23}}Joule/Kelvin$, $T$ is the kelvin
temperature[12]. According to Landauer’s principle, the lower bound of
computation power for a computing system can be obtained. Compared with the
value of computation power at real semiconductor chips, there exists a
difference of three orders of magnitude for the values of computation power
derived by Landauer’s principle [13]. Moreover, the values of computation
power are different when different semiconductor chip techniques are adopted
at BBU’s. Under these conditions, the main difficulty is how to accurately
calculate the computation power of small cell BS’s using the Landauer’s
principle.

To overcome the gap of computation power estimated by Landauer’s principle and
real semiconductor chips, we propose a power coefficient $\varepsilon$ is that
can represent the level of the semiconduct chip technique in BBU’s. Moreover,
the power coefficient $\varepsilon$ is defined as the ratio of the active
switching power of a transistor and the limit of Landauer’s principle. From
Fig. 3, the power coefficient $\varepsilon$ reflects the distance between
semiconductor chip techniques and the limit of Landauer’s principle. Bear in
mind that up till now the development of semiconductor chip techniques still
follows Moore’s law. However, the international technology roadmap for
semiconductors (ITRS) predicts that the development of semiconductor chip
techniques will deviate from Moore’s law when the power coefficient approaches
the limit of Landauer’s principle. For example, when nanomagnetic Logic is
used for chips, the computation power is expected to approach the limit of
Landauer’s principle [14]. Considering the development of current chip
techniques, we focus our attention on the computation power of semiconductor
chips.

Without a loss of generality, in this paper the power coefficient is
configured as $\varepsilon={10^{3}}$ when the 22 nanometer semiconductor
technique is assumed to be adopted for chip manufacture in BBU’s. Moreover,
the active switching power of a transistor is approximated by
${E_{FET}}\approx\varepsilon kT\ln(2)$, which is used to calculate the power
for operating 1 bit information at the semiconductor chip of BBU’s.

In general, the data processing rate of semiconductor chips is represented by
the instructions per second (IPS). Based on the definition of GOPS, in this
paper the relationship between the IPS and the GOPS is expressed by
$IPS=\frac{{GOPS\times{{10}^{9}}}}{{64}}$ when the logistical architecture of
semiconductor chips is assumed to be 64 bit. According to the experimental
results in [15], the information throughput of semiconductor chips is denoted
by $\rho={(\frac{{IPS}}{\omega})^{\frac{1}{\gamma}}}$, where $\omega$ and
$\gamma$ are configured as 0.1 and 0.64, respectively. As a consequence, the
computation power of different parts of a BBU is calculated by the product of
the information throughput of semiconductor chips and the active switching
power of transistors considering different values of GOPS at different parts
of the BBU.

Since different types of BS’s have different hardware components at BBU’s, it
is difficulty to directly build a uniform model to evaluate the computation
power of BBU’s in different types of BS’s. Therefore, we first build a
reference BS with typical parameters. By comparing different types of BS’s
with reference BS, we can derive the computation power of different BBU’s for
different types of BS’s. Without a loss of generality, the system parameters
are represented by $i\in\\{BW,Ant,M,R,dt,df\\}$, where $BW$ is the bandwidth
parameter, $Ant$ is the number of antennas parameter, $M$ is the modulation
coefficient parameter, $R$ is the parameter of coding rate, $dt$ is the
parameter of time-domain duty-cycling, and $df$ is the parameter of frequency-
domain duty-cycling. To simplify symbols in this paper, $X_{i}^{ref}$ is
denoted as the reference BS. When the subscript $i$ of $X_{i}^{ref}$ is
replaced by different symbols, the new variable represents the corresponding
system parameter in the reference BS, e.g., $X_{BW}^{ref}$ is the bandwidth of
the reference BS. Similarly, $X_{i}^{real}$ is denoted for a real BS. When the
subscript $i$ of $X_{i}^{real}$ is replaced by different symbols, the new
variable represents the corresponding system parameter in the real BS, e.g.,
$X_{BW}^{real}$ is the bandwidth of the real BS.

Considering different computation powers at different hardware parts of a BBU,
the computation power of DPD (Digital Pre-Distortion), Filter, CPRI (Common
Public Radio Interface), OFDM, FD (Frequency-Domain), FEC (Forward Error
Correction) and CPU at reference BS are denoted by $P_{DPD}^{ref}$,
$P_{Filter}^{ref}$, $P_{CPRI}^{ref}$, $P_{OFDM}^{ref}$, $P_{FD}^{ref}$,
$P_{FEC}^{ref}$ and $P_{CPU}^{ref}$, respectively. The different hardware
parts of a BBU depend on the different system parameters of BS. ${S_{i}}$,
$i\in\\{BW,Ant,M,R,dt,df\\}$ signifies the ratio of the different hardware
parts of the BBU and the system parameters of the BS. When the relationship
between the hardware part of BBU and the system parameter of BS is linear, the
corresponding ${S_{i}}$ is configured as 1. If such a relationship is non-
linear, the corresponding ${S_{i}}$ is set to 2. When the relationship between
the hardware part of BBU and the system parameter of BS is independent, the
corresponding ${S_{i}}$ is configured as 0. The detailed configuration
parameters of ${S_{i}}$ are illustrated in Table I.

Based on measurement results from the reference BS, the computation power of
the reference BBU can be obtained by
$P_{BB}^{ref}=P_{DPD}^{ref}+P_{Filter}^{ref}+P_{CPRI}^{ref}+P_{OFDM}^{ref}+P_{FD}^{ref}+P_{FEC}^{ref}+P_{CPU}^{ref}$.
To calculate the computation power of real BBU’s, the reference coefficient
$\alpha$ is defined by
$\alpha=\prod\limits_{i}{{{\left({\frac{{X_{i}^{real}}}{{X_{i}^{ref}}}}\right)}^{{S_{i}}}}}={\left({\frac{{X_{BW}^{real}}}{{X_{BW}^{ref}}}}\right)^{{S_{BW}}}}\cdot{\left({\frac{{X_{Ant}^{real}}}{{X_{Ant}^{ref}}}}\right)^{{S_{Ant}}}}\cdot{\left({\frac{{X_{M}^{real}}}{{X_{M}^{ref}}}}\right)^{{S_{M}}}}\cdot{\left({\frac{{X_{R}^{real}}}{{X_{R}^{ref}}}}\right)^{{S_{R}}}}\cdot{\left({\frac{{X_{dt}^{real}}}{{X_{dt}^{ref}}}}\right)^{{S_{dt}}}}\cdot{\left({\frac{{X_{df}^{real}}}{{X_{df}^{ref}}}}\right)^{{S_{df}}}}$.
Finally, the computation power of real BBU’s is calculated by
$P_{BB}^{real}=\alpha\cdot P_{BB}^{ref}$.

##  IV Evaluations of Computation Power

Considering that 5G small cell networks with massive MIMO’s and millimeter
wave techniques have not yet been commercially deployed, it is difficult to
compare our simulation results with real 5G small cell networks. To validate
the performance of the proposed power consumption model, we first compare the
results of the proposed model with those of the EARTH project [9], which
measures the power consumption of macro cell and small cell BS’s from real
wireless networks. Without a loss of generality, the two wireless
communication systems are configured with 10 MHz and 2$\times$2 antennas at
BSs and terminals. Based on the results from the EARTH project, the total
power consumption and computation power of a macro cell BS are 321.6 W and
29.68 W, respectively. Similarly, for our proposed power consumption model the
total power consumption and the computation power of a macro cell BS are
317.84 W and 24.78 W, respectively. In the case of a small cell BSS for the
EARTH project, the total power consumption and computation power of a small
cell BS are 6.2 W and 2.4 W, respectively. For the proposed power consumption
model, the total power consumption and computation power are 7.22 W and 3.6 W,
respectively. Compared with the above, the results of the proposed power
consumption model are in agreement with the results of real wireless networks.
Therefore, our proposed power consumption model is shown to be capable of
estimating the power consumption of 5G small cell networks.

Without loss of generality, the system parameters of the reference BS are
configured as $X_{BW}^{ref}=20$ MHz, $X_{Ant}^{ref}=1$, $X_{M}^{ref}=6$,
$X_{R}^{ref}=1$, $X_{dt}^{ref}=100\%$, $X_{df}^{ref}=100\%$. Based on the
configuration parameters of the BS’s in Table I, the computation power of BS’s
is simulated for 5G small cell networks. Since the massive MIMO and millimeter
wave technologies are the core technologies for 5G mobile communication
systems, in this section the impact of the number of antennas and bandwidths
on macro cell BS’s and small cell BS’s are simulated in detail.

Generally speaking, the PA’s of macro cell BS and small cell BS are configured
as 102.6 W and 1.0 W. Fig. 4 illustrates the computation power of BS with
respect to the number of antennas and bandwidths. The default system
parameters of real BS’s are configured as follows: the bandwidth is 20 MHz,
the modulation is 64-quadrature amplitude modulation (QAM), the coding rate is
$\frac{5}{6}$, the time-domain duty-cycling is 100% and the frequency-domain
duty-cycling is 100%. Fig. 4(a) shows the computation power of BS’s with
respect to the number of antennas. Based on the results in Fig. 4(a), the
computation power of BS’s quickly increases with the increase the number of
antennas. The reason is that the computation power consumed for frequency-
Domain processing is in proportion to the square of the number of antennas.
Moreover, the computation power of macro cell BS’s is always larger than the
computation power of small cell BS’s when the number of antennas is increased.
When the number of antennas is equal to 128, i.e., adopting the massive MIMO
technology, the computation power of macro cell BS is larger than 3000 W and
the computation power of small cell BS is larger than 800 W.

In general, with the adaptation of millimeter wave techniques, 5G
communication systems will be able to support large bandwidths (e.g., 400
MHz), or more precisely, high transmission rates. Consequently, this would
require more processing at the BBU, hence further increasing the computation
power at BS’s. Therefore, in this paper, the impact of a millimeter wave
technique on the computation power of BS’s is based on a wireless
communication bandwidth. When the number of antennas is configured as 4, Fig.
4(b) depicts the computation power of BS’s with respect to bandwidths. Based
on the results in Fig. 4(b), the computation power of BS increases with the
increase of bandwidths. Moreover, the computation power of macro cell BS’s is
always larger than the computation power of small cell BS’s when the bandwidth
is increased. When the bandwidth is 400 MHz, i.e., adopting the millimeter
wave technology, the computation power of macro cell BS is larger than 1000 W
and the computation power of small cell BS is larger than 200 W. Based on
results in Fig. 4, small cell BS’s can save more computation power for BBU’s
than macro cell BSs in 5G mobile communication systems.

To evaluate the role of computation power in the BS, the computation power
ratio is defined by the computation power over the total power at a BS. Fig. 5
illustrates the computation power ratio with respect to the number of antennas
and bandwidths for small cell BS’s and macro cell BS’s. Fig. 5(a) shows the
computation power with respect to the number of antennas. Based on the results
in Fig. 5(a), the computation power ratio increases with the increased number
of antennas. Moreover, the computation power ratio of small cell BS’s is
always larger than the computation power ratio of macro cell BS’s. In
addition, the computation power ratio of small cell BS’s is obviously larger
than 50%. Fig. 5(b) depicts the computation power ratio with respect to
bandwidths. Based on the results in Fig. 5(b), the computation power ratio
increases with the increase of bandwidths. Moreover, the computation power of
small cell BS’s is always larger than the computation power of macro cell
BS’s. When millimeter wave technology is adopted, i.e., the bandwidth is
larger than or equal to 20 MHz, and the computation power ratio of small cell
BS’s is obviously larger than 50%.

##  V Future Challenges

Based on the results in Fig. 4 and Fig. 5, the computation power will play a
more important role than other power consumptions, including the transmission
of power at 5G small cell BS’s, no matter what the level of the absolute
volume and the ratio for 5G small cell networks is. On the other hand, energy
efficiency of 5G mobile communication systems is expected to improve 100 to
1000 times, compared with the energy efficiency of 4G mobile communication
systems. However, most studies involving the energy efficiency of 5G cellular
networks still focus on the transmission power optimization of BS’s. To face
the role of computation power in 5G small cell networks, some potential
challenges are presented here.

The first challenge is the impact of 5G network architectures on the
computation power in 5G small cell networks. Based on the results in Fig. 5,
the importance of computation power is improved for energy efficiency
optimization of 5G small cell networks. One obvious reason is that the
transmission power is reduced in 5G small cell networks that adopt the massive
MIMO and millimeter wave technologies. With cloud/fog computing and cache
communications emerging for 5G networks, more and more data storage and
computation will be performed at 5G small cell BS’s. Therefore, it is possible
to predict that computation power, no matter what the power consumption level
of the absolute volume and the ratio will be shall further improve for 5G
cellular networks. In this case, the energy efficiency optimization of 5G
cellular networks will not only consider the transmission power. But also the
power consumed for data computation and storage at BS’s.

The second challenge is optimization of computation power at BS’s with massive
MIMO and millimeter wave transmission technologies. Existing studies usually
fix the value of computation power at BS’s. Moreover, the impact of 5G
wireless transmission technologies, such as the massive MIMO and millimeter
wave technologies on the computation power, is ignored at BS’s. Based on the
results in Fig. 4, the massive MIMO and millimeter wave technologies have a
greater impact on the computation power of 5G small cell BS’s. Considering the
role of computation power at 5G small cell BS’s, it is inadvisable to ignore
the impact of 5G transmission technologies on the computation power of 5G
small cell BS’s. When massive MIMO and millimeter wave technologies are
adopted by 5G small cell BS’s, a large number of antennas and bandwidths can
be scheduled for resource optimization in 5G small cell networks. How to
schedule the number of antennas and bandwidths for the optimization of
computation power at 5G small cell BS’s.

The third challenge is the tradeoff between computation power and transmission
power in 5G networks. Based on the analysis in Section II, the additional
power of BS’s depends on the computation and transmission powers of BS’s. When
the additional power of BS’s is combined into the computation and transmission
power of BS’s, the energy efficiency of 5G networks can be calculated by the
energy efficiency of computation and transmission powers at BS’s. However, 5G
transmission technologies have different effects on the energy efficiency of
computation and transmission power of small cell BS’s. In some specific
scenarios, the effects on energy efficiency of computation and transmission
powers are contradictory at 5G small cell BS’s. Hence, the relationship
between the computation and communication powers needs to be further
investigated for 5G networks. Moreover, the tradeoff between computation and
transmission power needs to be optimized for 5G small cell BS’s.

To face the above challenges in the energy efficiency optimization of 5G small
cell networks, some potential research directions are summarized to solve
these issues:

  * •

The new energy efficiency model of 5G small cell networks considering
computation and transmission power needs to be investigated. Moreover, the
software-defined networks (SDN) could be used to trade off computation and
transmission powers at 5G small cell BS’s with cloud/fog computing functions.

  * •

To improve the energy efficiency of 5G small cell BS’s, joint optimization
schemes and algorithms should be developed to save computation and
transmission power at BBU’s and RF chains together.

  * •

Based on the simulation results in Fig. 4, lot of computation power of BBU’s
has to be changed into heat and more cooling systems need to be designed to
support computation functions at BBU’s. To save energy at BBU’s, we should
take the energy cycle into account and some potential technologies are
expected to change the heat from BBU’s into electrical energy based on the
pyroelectric effect.

##  VI Conclusions

Until recently, the computation power of BS’s was ignored or just fixed as a
small constant in the energy efficiency evaluation of cellular networks. In
this paper, the power consumption of BS’s is analyzed for 5G small cell
networks adopting massive MIMO and millimeter wave technologies. Considering
the massive traffic in 5G small cell networks, the computation power of 5G
small cell BS’s is first estimated based on Landauer’s principle. Moreover,
simulation results show that the computation power of BS’s increases as the
number of antennas and bandwidths increases. Compared with transmission power,
computation power will play a more important role in the energy efficiency
optimization of 5G small cell networks. Therefore, we conclude that the energy
efficiency optimization of 5G small cell networks should consider computation
and transmission power together. How to converge computation and transmission
technologies to optimize the energy efficiency of 5G networks is still an open
issue. If this is accomplished, a different challenge would indeed emerge in
the next round of the transmission and computation revolution.

## Acknowledgment

The authors would like to acknowledge the support from the NSFC Major
International Joint Research Project (Grant No. 61210002), the Fundamental
Research Funds for the Central Universities under the grant 2015XJGH011. This
research is partially supported by the EU FP7-PEOPLE-IRSES, project acronym
WiNDOW (grant no. 318992) and project acronym CROWN (grant no. 610524), China
international Joint Research Center of Green Communications and Networking
(No. 2015B01008).

## References

  * [1] X. Ge, S. Tu, G. Mao, C.-X. Wang and T. Han, “5G Ultra-Dense Cellular Networks,” _IEEE Wireless Communications_ , vol. 23, No. 1, Feb. 2016, pp. 72-79.
  * [2] J. G. Andrews, S. Buzzi, C. Wan, et al., “What Will 5G Be?,” _Selected Areas in Communications, IEEE Journal on_ , vol. 32, No. 6, June 2014, pp. 1065-1082.
  * [3] X. Ge, S. Tu, T. Han, Q. Li, and G. Mao, “Energy efficiency of small cell backhaul networks based on Gauss-Markov mobile models,” _IET Networks_ , vol. 4, March 2015, pp. 158-167.
  * [4] S. Samarakoon, M. Bennis, W. Saad, M. Debbah and M. Latva-aho, “Ultra Dense Small Cell Networks: Turning Density Into Energy Efficiency,” _IEEE Journal on Selected Areas in Communications_ , vol. 34, May 2016, pp. 1267-1280.
  * [5] X. Ta, G. Mao and B. D. O. Anderson, “On the Giant Component of Wireless Multihop Networks in the Presence of Shadowing,” _IEEE Transactions on Vehicular Technology_ , vol. 58, no. 9, Nov. 2009, pp. 5152-5163.
  * [6] C. Liu, B. Natarajan and H. Xia, “Small Cell Base Station Sleep Strategies for Energy Efficiency,” _IEEE Transactions on Vehicular Technology_ , vol. 65, March 2016, pp. 1652-1661.
  * [7] J. Choi, “Energy Efficiency of A Heterogeneous Network Using Millimeter-Wave Small-Cell Base Stations,” _in Personal, Indoor, and Mobile Radio Communications (PIMRC), 2015 IEEE 26th Annual International Symposium on_ , 2015, pp. 293-297.
  * [8] S. Chen, F. Qin, B. Hu, X. Li and Z. Chen, “User-centric ultra-dense networks for 5G: challenges, methodologies, and directions,” _IEEE Wireless Communications_ , vol. 23, no. 2, April 2016, pp. 78-85.
  * [9] M. Imran, E. Katranaras, G. Auer, et al., “Energy Efficiency Analysis of The Reference Systems, Areas of Improvements and Target Breakdown,” _Tech. Rep. ICT-EARTH deliverable_ , Tech. Rep, 2011.
  * [10] C. Desset, B. Debaillie, V. Giannini, et al., “Flexible Power Modeling of LTE Base Stations,” _in Wireless Communications and Networking Conference (WCNC), 2012 IEEE_ , 2012, pp. 2858-2862.
  * [11] R. Landauer, “Irreversibility and Heat Generation in The Computing Process,” _IBM Journal of Research and Development_ , vol. 5, 1961, pp. 183-191.
  * [12] A. Berut, A. Arakelyan, A. Petrosyan, et al., “Experimental Verification of Landauer/’s Principle Linking Information and Thermodynamics,” _Nature_ , vol. 483, March 2012, pp. 187-189.
  * [13] W. P. Cockshott, P. Cockshott, L. M. Mackenzie and G. Michaelson, _Computation and Its Limits_. Oxford University Press, 2012.
  * [14] B. Lambson, D. Carlton and J. Bokor, “Exploring The Thermodynamic Limits of Computation in Integrated Systems: Magnetic Memory, Nanomagnetic Logic, and The Landauer Limit,” _Physical Review Letters_ , vol. 107, no. 1, July 2011, p. 010604.
  * [15] L. Zhirnov, R. Cavin and L. Gammaitoni, “Minimum Energy of Computing, Fundamental Considerations,” _ICT-Energy-Concepts Towards Zero-Power Information and Communication Technology_ , vol. 7, 2014.

## Biographies

XIAOHU GE is currently a full professor with the School of Electronic
Information and Communications at Huazhong University of Science and
Technology (HUST), China, and an adjunct professor with the Faculty of
Engineering and Information Technology at the University of Technology Sydney
(UTS), Australia. He received his Ph.D. degree in communication and
information engineering from HUST in 2003. His research interests include
green communications, vehicular communications and wireless networks. He is
the director of the China International Joint Research Center of Green
Communications and Networking. He serves as associate editors for IEEE
Transaction on Greening Communications and Networking, IEEE Access.

JING YANG received his B.E. degrees in communication engineering from HUST in
2014. He is currently pursuing the Ph.D. degree in the School of Electronic
Information and Communications at HUST. His research interests mainly include
computation power of wireless communication systems, green communication and
energy efficiency of wireless cellular networks.

HAMID GHARAVI (Life Fellow, IEEE) received his Ph.D. degree from Loughborough
University, United Kingdom, in 1980. He joined the Visual Communication
Research Department at AT&T Bell Laboratories, Holmdel, New Jersey, in 1982.
He was then transferred to Bell Communications Research (Bellcore), where he
became a Distinguished Member of Research Staff. In 1993, he joined
Loughborough University as a professor and chair of communication engineering.
Since September 1998, he has been with the National Institute of Standards and
Technology (NIST). His research interests include smart grid, wireless
multimedia, mobile communications and wireless systems, mobile ad hoc
networks, and visual communications. He served as a member of the Editorial
Board of Proceedings of the IEEE from January 2003 to December 2008. From
January 2010 to December 2013 he served as Editor-in-Chief of IEEE
Transactions on CAS for Video Technology. He is currently serving as Editor-
in-Chief of IEEE Wireless Communications.

YANG SUN received his B.S. degrees with honor in electronic and information
engineering from Huazhong university of science and technology and now is
pursuing his master degree in electronic information and communications,
Huazhong university of science and technology

Figure 1: Logistical architecture of eNodeB BS.

Figure 2: Hardware architecture of BBU

Figure 3: The power coefficient with respect to the development of chip
techniques

Figure 4: Computation power of BSs with respect to the number of antennas and
bandwidths

Figure 5: Computation power ratio with respect to the number of antennas and
bandwidths.

TABLE I: Configuration parameters of the BBU BBU |  GOPS of |  GOPS of |  ${S_{BW}}$ |  ${S_{M}}$ |  ${S_{R}}$ |  ${S_{Ant}}$ |  ${S_{dt}}$ |  ${S_{df}}$
---|---|---|---|---|---|---|---|---
parameters |  Macro cell |  Small cell |  |  |  |  |  |
DPD |  160 |  0 |  1 |  0 |  0 |  1 |  1 |  0
Filter |  400 |  250 |  1 |  0 |  0 |  1 |  1 |  0
CPRI/SERDES |  720 |  0 |  1 |  1 |  1 |  1 |  1 |  1
OFDM |  160 |  120 |  1 |  0 |  0 |  1 |  1 |  0
FD |  90 |  50 |  1 |  0 |  0 |  1 |  1 |  1
(liner) |  |  |  |  |  |  |  |
FD |  30 |  15 |  1 |  0 |  0 |  2 |  1 |  1
(non-liner) |  |  |  |  |  |  |  |
FEC |  140 |  130 |  1 |  1 |  1 |  1 |  1 |  1
CPU |  400 |  40 |  0 |  0 |  0 |  1 |  0 |  0


# WildDESED: An LLM-Powered Dataset for Wild Domestic Environment
Sound Event Detection System

###### Abstract

This work aims to advance sound event detection (SED) research by presenting a
new large language model (LLM)-powered dataset namely wild domestic
environment sound event detection (WildDESED). It is crafted as an extension
to the original DESED dataset to reflect diverse acoustic variability and
complex noises in home settings. We leveraged LLMs to generate eight different
domestic scenarios based on target sound categories of the DESED dataset. Then
we enriched the scenarios with a carefully tailored mixture of noises selected
from AudioSet and ensured no overlap with target sound. We consider widely
popular convolutional neural recurrent network to study WildDESED dataset,
which depicts its challenging nature. We then apply curriculum learning by
gradually increasing noise complexity to enhance the model’s generalization
capabilities across various noise levels. Our results with this approach show
improvements within the noisy environment, validating the effectiveness on the
WildDESED dataset promoting noise-robust SED advancements.

Index Terms—  sound event detection, DESED, noisy scenario, noise robust SED,
curriculum learning

##  1 Introduction

Sounds play a vital role in our lives, helping us understand our surroundings
and notice changes. Sound event detection (SED) [1, 2] is essential for
interpreting and responding to our environment, with applications ranging from
urban noise management to smart-home technologies [3] and security systems
[4]. SED has made great strides [5, 6, 7], thanks to diverse datasets [8]
tailored for specific scenarios. Google AudioSet [9] provides a wide array of
sounds, and MAVD [10] focuses on traffic noise. Among various SED datasets,
DESED [11, 12] is well known for its focus on domestic environments, which
makes it the most utilized dataset for home sound event research. However,
DESED faces challenges in comprehensively representing the unpredictable and
complex nature of household sounds. Hence, there exists scope for covering a
wide range of domestic scenarios with common background noises that can occur
in a household.

The quest for noise robustness in SED has led to the development of new
methodologies and datasets [13, 14] aimed towards improving performance under
challenging conditions such as noisy urban environments. Innovations by
researchers like Neri et al.[15], Serizel et al.[16], and Wan et al. [17] have
pushed the boundaries of SED systems by integrating deep learning and audio
enhancement techniques. These studies, however, predominantly address
controlled or semi-controlled environments, leaving a gap for SED systems to
effectively detect sound events in the less predictable, ‘wild’ conditions in
domestic environments.

Addressing this gap, our research contributes to the field by introducing a
new dataset namely, wild domestic environment sound event detection
(WildDESED). We proposed carefully selecting noise types from the AudioSet
that accurately represent real home environments but are distinct from DESED’s
target sounds. This artificial selection could be challenging because of the
bias and unnatural correlations. Large language models (LLMs) [18] such as
GPT-4, ChatGPT, and Llama have demonstrated remarkable potential to perform
various tasks [19, 20, 21] in recent years. In this regard, we utilized the
strong capabilities of LLMs to analyze and summarize acoustic data for
selecting specific noises. This helped us to design eight different scenarios
that blend the noises with target sounds, simulating authentic domestic
environments. The noises are divided into four categories based on their
sources and acoustic properties, allowing for a diverse and realistic
combination with target sounds. This novel approach has culminated in the
creation of WildDESED dataset, specifically designed to enhance SED research
in dynamic and natural home environments.

Building on this foundation, our research not only introduces the WildDESED
dataset, but also explores the application of curriculum learning in the
context of SED to tackle the challenges posed by domestic noisy environments.
Curriculum learning [22, 23, 24] is a training approach that improves models
for noisy speech [25, 26] and audio by starting with simpler, less noisy data
and gradually increasing the noise level. This method is similar to the way
how the humans learn and helps models adjust from clean to noisy sounds more
effectively. In this work, we applied curriculum learning to the baseline
convolutional recurrent neural network (CRNN) [27, 28, 29] model using the
WildDESED dataset for our studies. The novelty of this work lies in the
proposal of a new in-the-wild dataset for advancing SED research and exploring
curriculum learning as an approach to develop noise-robust SED systems. We
will publicly release the WildDESED dataset in due course.

##  2 Related work

The WildDESED dataset is an extension to the original DESED dataset, which is
a foundational resource featuring 10 target sound classes pivotal for
understanding the sounds in home environments. The DESED dataset consists of
the following subsets: The weak set, with 1,578 real recordings labeled with
weak annotations, captures the presence of sound classes without temporal
specifics. The unlabeled training set includes 14,412 real, unlabeled
recordings. The test set comprises of 1,168 real recordings with strong
annotations to assess model performance. These three subsets are real-world
recordings from AudioSet. The training synth set contains 10,000 synthetic
recordings with strong annotations [30], detailing exact temporal boundaries.
The synth validation set has 2,500 synthetic recordings with strong
annotations for model validation during development. These two synthetic
subsets are generated with the Scaper. Their background files are extracted
from SINS [31], TUT [32], MUSAN [33], or YouTube and have been selected
because they contain a very low amount of our sound event classes. We propose
to simulate more diverse and complex noisy scenarios that are not covered by
the original DESED dataset and also introduce a controlled variability for
testing.

##  3 WildDESED

We extend DESED to the WildDESED for in-the-wild scenarios for domestic
environments by considering three primary set of questions to address as
follows:

  * •

What type of background noises do we use?

  * •

What are the domestic scenarios we choose?

  * •

How do we mix the background noises to the scenarios?

Table 1: A summary of different background noises used in WildDESED dataset.

Noise | Occurrences | Duration (Second)
---|---|---
Bird chirping outside | 9,847 | 7,523
Car passing by outside | 311 | 862
Chair moving | 343 | 359
Clock ticking | 2,5777 | 2,662
Coffee machine | 6 | 30
Door closing | 335 | 196
Fan noise | 117 | 958
Footsteps | 6,243 | 2,101
Light rain | 159 | 1,379
Refrigerator humming | 58 | 456
TV playing in the background | 805 | 7,191
Wind blowing | 5,467 | 48,648
Total | 49,468 | 72,365

GPT-4 is an advanced language model that builds on the GPT-3 architecture but
uses a larger amount of training data. It includes the latest techniques to
enhance understanding of natural language. In the following subsections, we
will detail how we leverage GPT-4 to address each of these questions,
outlining the methodology behind the creation of the WildDESED dataset. This
new dataset aims to bridge the gap between the controlled environment of
existing datasets and the dynamic, often unpredictable nature of real-world
domestic soundscapes, thus expanding the potential for noise-robust SED
research in truly ‘wild’ home scenarios.

###  3.1 What type of background noises do we use?

To construct the WildDESED dataset, we initiated our process with the
foundational DESED dataset, which identifies 10 distinct sound events in
10-second audio clips. The events in DESED include diverse household sounds
like alarms, appliances, pets, and running water. We input the total 356
classes from the strongly annotated subset of AudioSet to the GPT-4 together
with the 10 DESED classes. Then we guide GPT-4 by the following prompt:

“Select noise classes from the 356 strongly annotated AudioSet classes,
alongside the 10 DESED classes ensuring clear delineation and no overlap with
DESED’s sound events. Further, apply thorough filtering to exclude any
AudioSet classes similar to DESED target classes, preserving the
distinctiveness of the dataset.”

Considering the output of GPT-4, we enhanced DESED with selected events from
the strongly annotated subset of AudioSet, ensuring clear delineation and no
overlap with DESED’s sound events. A thorough filtering process was applied to
exclude any AudioSet classes that are very similar to target classes of DESED
dataset, preserving the distinctiveness of our dataset. Table 1 displays the
outcome of our selection process, listing the types and quantities of noise
clips integrated into WildDESED. We included a spectrum of sounds both indoor,
like the clock ticking, and outdoor, such as birds chirping that capture the
essence of a domestic environment. The ‘clock ticking’ class, for instance,
has the largest event count, while ‘wind blowing’ spans the greatest duration,
together reflecting the continuous and transient nature of home sounds.

This dataset construction ensures WildDESED encompasses a rich and authentic
array of domestic noises, ready to challenge and advance SED systems in
recognizing the events under complex acoustic home environments.

###  3.2 What are the domestic scenarios we choose?

For the WildDESED dataset, we still have to map the selected 12 noise classes
with our 10 target classes. We input them to GPT-4 and use the following
prompt:

“Create eight different domestic scenarios so that they should map 12 selected
noise classes to the 10 target classes from the DESED dataset, crafting
authentic household soundscapes. Ensure the scenarios reflect typical sounds
one would encounter in a household environment.”

Considering the output of LLM, we crafted eight different domestic scenarios,
each mapping to target classes from the DESED dataset to create authentic
soundscapes one would encounter in a household. These scenarios are
constructed to reflect the typical activities and the accompanying sounds in a
domestic environment.

  * •

Morning Routine: Associated with ‘Blender’ target sounds, this scenario
captures the essence of the morning with ‘Light rain’, ‘Refrigerator humming’,
‘Clock ticking’, and ‘TV playing in the background’.

  * •

Home Office: Linked to ‘Speech’ as the target class, it includes background
sounds of ‘Car passing by’, ‘Fan noise’, and ‘Footsteps’, emulating a work-
from-home setting.

  * •

Household Chores: Representing ‘Vacuum cleaner’ noises as the target, this
scenario combines ‘Door closing’, ‘Chair moving’, and ‘Footsteps’ as
background to depict cleaning activities.

  * •

Late-night: Tied to the ‘Electric shaver toothbrush’ target sound, offering
the ‘Clock ticking’ and ‘Light rain’ as a backdrop for night-time routines.

  * •

Cooking: Merging the target sounds of ‘Frying’ and ‘Dishes’ with ‘Coffee
machine’ buzzes and ‘Refrigerator humming’, this scenario is bustling with
culinary activity.

  * •

Pet Care: Incorporating target sounds of ‘Cat’ and ‘Dog’, this setting is
further brought to life with ‘Bird chirping outside’ and ‘TV playing in the
background’.

  * •

Bathroom Routine: Linked to ‘Running water’ as the target sound, with added
‘Fan noise’ and ‘Wind blowing’, simulating personal care sounds.

  * •

Emergency: Associated with the ‘Alarm bell ringing’ target sound, it layers
urgent sounds like ‘Refrigerator humming’ and ‘Fan noise’ with ‘Clock ticking’
and ‘Car passing by’.

Figure 1: Illustration222Figures generated using DALL-E-2
(https://openai.com/dall-e-2)of Morning Routine Scenario out of the total
eight scenarios in WildDESED dataset. In the scenario, key target sound events
are written in bold fonts, along with added different background noises to
simulate real-life settings.

Each scenario’s sound design is a thoughtful blend of target and noise
classes, chosen to challenge the detection capabilities of SED systems within
the rich and varied auditory context of a home environment. To illustrate our
scenarios, we present a Figure 2 that showcases two typical scenarios out of
the eight: the ‘Pet Care Scenario’ and the ‘Morning Routine Scenario’. This
figure highlights key target sound events within each scenario, incorporating
strategically placed background noises to simulate the real-life acoustic
challenges found in domestic settings.

###  3.3 How do we mix the background noises to the scenarios?

In the WildDESED dataset, the integration of background noises into the
selected domestic scenarios is meticulously structured around a quadrant based
on the acoustic characteristics of the noises. The quadrant categorizes noises
into four groups: Ambient Environmental Sounds, Human-related and Intermittent
Sounds, Mechanical and Electronic Sounds, and Nature and Outdoor Sounds, as
illustrated in Figure 2.

  * •

For Ambient Environmental Sounds, such as ‘Light rain’ and ‘Wind blowing’, we
repeated these sounds to cover the entire duration of the audio clip from the
original DESED dataset. These sounds are mixed at a low intensity to ensure
they provide a consistent background atmosphere without overpowering the
primary sound events. The rationale behind this is to create an unobtrusive
ambient layer that emulates the continuous presence of these sounds in a
typical home environment.

  * •

Sounds like ‘Footsteps’, ‘Door closing’, and ‘Chair moving’ fall into the
Human-Related and Intermittent Sounds category. These are inserted at random
intervals to simulate the sporadic nature of human movement and activities
within a home. The volume and frequency of these sounds are varied to reflect
the realistic and unpredictable nature of their occurrence in daily life.

  * •

Mechanical sounds, including ‘Clock ticking’ and ‘Coffee machine’, are
inserted at specific points to coincide with the actions they represent, such
as a coffee machine being used during morning routines. The volume is set to
be noticeable but not overwhelming, ensuring the sound is recognized as a part
of the scenario without becoming a large distraction.

  * •

Lastly, Nature and Outdoor Sounds  like ‘Car passing by outside’ and ‘Bird
chirping outside’ are incorporated randomly to enhance the realism of external
environmental influences. The volume may fluctuate to mimic the variable
volume of these sounds in real settings, contributing to the unpredictability
and diversity of the overall soundscape.

Each noise type and its corresponding mixing approach are tailored to maintain
the authenticity of the domestic scenarios. This methodical and scenario-
specific approach to mix noises ensures that the WildDESED dataset not only
presents a challenge for SED systems but also closely reflects the complex
acoustic environments of actual domestic settings.

Figure 2: Quadrant showing four groups of noise types based on their acoustic
characteristics considered in the WildDESED. Figure 3: Statistics of noises in
the WildDESED subsets. Table 2: Performance in PSDS1 (P1), PSDS2 (P2) and
PSDS1 + PSDS2 (P1 + P2) of the proposed curriculum learning (CL) approach on
the DESED devtest set and our proposed WildDESED (W) dataset with SNR in dB.

Model | Performance on DESED | Performance on WildDESED
---|---|---
10dB | 5dB | 0dB | -5dB
P1 | P2 | P1 + P2 | P1 | P2 | P1 + P2 | P1 | P2 | P1 + P2 | P1 | P2 | P1 + P2 | P1 | P2 | P1 + P2
CRNN | 0.344 | 0.543 | 0.887 | 0.222 | 0.409 | 0.631 | 0.148 | 0.302 | 0.450 | 0.064 | 0.174 | 0.238 | 0.017 | 0.078 | 0.095
CRNN (W) | 0.200 | 0.329 | 0.529 | 0.175 | 0.337 | 0.512 | 0.135 | 0.303 | 0.438 | 0.087 | 0.242 | 0.329 | 0.048 | 0.174 | 0.222
CRNN (W+ CL) | 0.265 | 0.461 | 0.726 | 0.212 | 0.443 | 0.655 | 0.175 | 0.390 | 0.565 | 0.114 | 0.317 | 0.431 | 0.049 | 0.211 | 0.260

In finalizing the composition of the WildDESED dataset, special consideration
was given to the representation of the ‘speech’ sound class due to its
prevalence and significance in domestic environments. For the ‘Home Office’
scenario in synth set and synth val set, we exclusively selected clips that
featured the ‘speech’ class in isolation, omitting any clips where ‘speech’
occurred alongside other sound events.

Figure 3 displays class-wise statistics for different background noises in
each subset of the WildDESED dataset, indicating the prevalence of each noise
type, within synth, synth val, weak, and test subsets. Figure 4 shows
scenario-wise statistics for the scenarios in the WildDESED dataset,
quantifying how frequently each scenario appears in each subset. Through this
detailed dataset structure, WildDESED positions itself as a crucial resource
for developing and evaluating SED systems, equipping researchers with the
means to advance the field of SED in naturalistic home environments.

##  4 Curriculum Learning for Noise-robust SED

We use a curriculum learning [22, 26] method to develop noise-robust SED
systems. This approach introduces complexity in stages, starting with simple
tasks and gradually integrating noise at various signal-to-noise ratios (SNR),
aligning with our goal to augment the model’s resilience to noise.

We have five stages in our methodology, each with an increasing level of noise
difficulty. Initially, the model learns from clean audio samples. This
foundational step is crucial for establishing an understanding of the sound
events without the confounding presence of noise. We then incrementally
introduce noise, decreasing the SNR by 5dB in subsequent stages. Let $N$ be
the total number of training samples. Given $k$ noise levels
$L=[L_{1},L_{2},\ldots,L_{k}]$, the dataset $D$ is composed as follows:

$D=\bigcup_{i=1}^{k}\left\\{D_{i}\right\\},D_{i}=\frac{N}{k}\text{ samples at
noise level }L_{i}\vspace{-2mm}$ (1)

The $k$ in our experiment here is 5 including the clean DESED, and noise
levels 10dB, 5dB, 0dB, and -5dB are considered. The model’s progress is
meticulously monitored, and a validation metric $c$ is used to evaluate
learning at each epoch. In our approach, the $c$ is the intersection f1-score.
If $c$ fails to improve for ten consecutive epochs [34], the best-performing
model state is reloaded, and the training progresses to the next noise level.

Figure 4: Statistics of the scenarios in the WildDESED subsets.

##  5 Experimental Settings

###  5.1 Dataset and Evaluation Metric

We considered the DESED dataset and our proposed WildDESED dataset, featuring
10-second audio clips across various subsets. All clips were resampled to 16
kHz mono and segmented using a 2048-sample window and 256-sample hop length
for spectrogram extraction and log-mel spectrogram generation. Our systems
were evaluated using the threshold-independent polyphonic sound event
detection scores (PSDS) [35] in two scenarios following DCASE 2023 Challenge
Task 4A protocol. Scenario-1 focuses on prompt reaction and temporal
localization, while Scenario-2 emphasizes on reducing class confusion for SED.

###  5.2 Implementation Details

For our experiments, following the DCASE 2023 Task 4A baseline [28], we
utilized a batch size of 48 and employed the Adam optimizer with an initial
learning rate of 0.001, coupled with an exponential warmup scheduler applied
across the first 50 epochs out of a total 200 epochs. To stabilize training,
we implemented a mean teacher model with an exponential moving average [36]
factor set at 0.999. We consider the CRNN [28] baseline system from DCASE 2023
Task 4A, featuring approximately 1.2 million parameters, ensuring a robust
comparison for our curriculum learning approach.

##  6 Results and Discussion

Table 2 shows the results of our studies on DESED and newly created WildDESED
datasets. It is observed that the performance of the baseline CRNN model
trained using DESED dataset drops significantly as the noise levels are
increased on WildDESED dataset compared to that on the original DESED dataset.
We then explore the baseline CRNN model trained using WildDESED data, which we
refer to as CRNN (W). We find that CRNN (W) performs better than the original
CRNN model when the noise levels on WildDESED are on the higher end (0 dB and
-5 dB). However, the performance is comparable for both models when noise
level is 5 dB and then the original CRNN model performs better for less noisy
scenario of 10dB on WildDESED and on the clean DESED dataset.

We now focus on the studies for curriculum learning approach applied on the
CRNN model trained using WildDESED dataset. We refer this model as CRNN (W+CL)
and find that it outperforms both CRNN as well as CRNN (W) models for all
noise levels on the WildDESED dataset. This highlights the scope of curriculum
learning approach for developing noise-robust SED systems using WildDESED
dataset for unseen complex domestic settings. We also note that the CRNN model
trained on the clean DESED performs the best on the DESED test due to the
matched conditions. However, the model CRNN (W+CL) with curriculum learning
certainly helps to boost the performance of the CRNN (W) model trained on
WildDESED dataset to bring it closer that of the CRNN model on DESED test set.
The future work will focus on reducing this performance gap on the clean
scenario for noise-robust SED models.

##  7 Conclusion

In this work, we have presented a new dataset referred to as WildDESED to
advance SED research under noisy home settings and also explored a preliminary
curriculum learning method to develop noise-robust SED systems. We used 12
noises from Audioset to craft the WildDESED dataset considering 8 different
scenarios depicting complex home environments by considering assistance from
an LLM. The studies conducted showed the scope of curriculum learning approach
for developing noise-robust SED systems using the WildDESED dataset. We
believe this WildDESED dataset will be useful for future horizons of noise-
robust SED research.

## References

  * [1] A. Mesaros, T. Heittola, T. Virtanen, and M. D. Plumbley, “Sound Event Detection: A Tutorial,” _IEEE Signal Processing Magazine_ , vol. 38, no. 5, pp. 67–83, 2021.
  * [2] T. Khandelwal, R. K. Das, and E. S. Chng, “Sound Event Detection: A Journey Through DCASE Challenge Series,” _APSIPA Transactions on Signal and Information Processing_ , vol. 13, 2024.
  * [3] R. Radhakrishnan, A. Divakaran, and A. Smaragdis, “Audio Analysis for Surveillance Applications,” in _Proc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics._ , 2005, pp. 158–161.
  * [4] J. P. Bello, C. Silva, O. Nov, R. L. Dubois, A. Arora, J. Salamon, C. Mydlarz, and H. Doraiswamy, “SONYC: A System for Monitoring, Analyzing, and Mitigating Urban Noise Pollution,” _Communications of the ACM_ , vol. 62, no. 2, pp. 68–77, 2019.
  * [5] Y. Xiao and R. K. Das, “Dual Knowledge Distillation for Efficient Sound Event Detection,” _arXiv:2402.02781_ , 2024.
  * [6] Y. Xiao, T. Khandelwal, and R. K. Das, “FMSG Submission for DCASE 2023 Challenge Task 4 on Sound Event Detection with Weak Labels and Synthetic Soundscapes,” DCASE 2023 Challenge, Tech. Rep., 2023.
  * [7] F. Ronchini and R. Serizel, “Performance and Energy Balance: A Comprehensive Study of State-of-the-art Sound Event Detection Systems,” _arXiv:2310.03455_ , 2023.
  * [8] H. Dinkel, M. Wu, and K. Yu, “Towards Duration Robust Weakly Supervised Sound Event Detection,” _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , vol. 29, pp. 887–900, 2021.
  * [9] J. F. Gemmeke, D. P. W. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, and M. Ritter, “Audio Set: An Ontology and Human-Labeled Dataset for Audio Events,” in _Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2017, pp. 776–780.
  * [10] P. Zinemanas, P. Cancela, and M. Rocamora, “MAVD: A Dataset for Sound Event Detection in Urban Environments.” in _Proc. the Detection and Classification of Acoustic Scenes and Events Workshop (DCASE)_ , 2019.
  * [11] N. Turpault, R. Serizel, A. P. Shah, and J. Salamon, “Sound Event Detection in Domestic Environments with Weakly Labeled Data and Soundscape Synthesis,” in _Proc. the Detection and Classification of Acoustic Scenes and Events Workshop (DCASE)_ , 2019.
  * [12] N. Turpault and R. Serizel, “Training Sound Event Detection on a Heterogeneous Dataset,” in _Proc. the Detection and Classification of Acoustic Scenes and Events Workshop (DCASE)_ , 2020.
  * [13] Y. Choi, O. Atif, J. Lee, D. Park, and Y. Chung, “Noise-robust Sound-event Classification System with Texture Analysis,” _Symmetry_ , vol. 10, no. 9, p. 402, 2018.
  * [14] I. McLoughlin, H. Zhang, Z. Xie, Y. Song, and W. Xiao, “Robust Sound Event Classification Using Deep Neural Networks,” _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , vol. 23, no. 3, pp. 540–552, 2015.
  * [15] M. Neri, F. Battisti, A. Neri, and M. Carli, “Sound Event Detection for Human Safety and Security in Noisy Environments,” _IEEE Access_ , vol. 10, pp. 134 230–134 240, 2022.
  * [16] R. Serizel, N. Turpault, A. Shah, and J. Salamon, “Sound Event Detection in Synthetic Domestic Environments,” in _Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2020, pp. 86–90.
  * [17] T. Wan, Y. Zhou, Y. Ma, and H. Liu, “Noise Robust Sound Event Detection Using Deep Learning and Audio Enhancement,” in _Proc. IEEE International Symposium on Signal Processing and Information Technology (ISSPIT)_ , 2019, pp. 1–5.
  * [18] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, _et al._ , “Language Models are Few-shot Learners,” in _Proc. Advances in Neural Information Processing Systems (NeurIPS)_ , vol. 33, 2020, pp. 1877–1901.
  * [19] “(toolqa: A dataset for llm question answering with external tools.”
  * [20] Y. Gong, H. Luo, A. H. Liu, L. Karlinsky, and J. R. Glass, “Listen, Think, and Understand,” in _Proc. International Conference on Learning Representations (ICLR)_ , 2023.
  * [21] J. Bai, H. Yin, M. Wang, D. Shi, W.-S. Gan, J. Chen, and S. Rahardja, “AudioLog: LLMs-Powered Long Audio Logging with Hybrid Token-Semantic Contrastive Learning,” _arXiv preprint:2311.12371_ , 2023.
  * [22] Y. Bengio, J. Louradour, R. Collobert, and J. Weston, “Curriculum Learning,” in _Proc. Annual International Conference on Machine Learning (ICML)_ , 2009, pp. 41–48.
  * [23] X. Wang, Y. Chen, and W. Zhu, “A Survey on Curriculum Learning,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 44, no. 9, pp. 4555–4576, 2021.
  * [24] A. Pentina, V. Sharmanska, and C. H. Lampert, “Curriculum Learning of Multiple Tasks,” in _Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2015, pp. 5492–5500.
  * [25] S. Braun, D. Neil, and S.-C. Liu, “A Curriculum Learning Method for Improved Noise Robustness in Automatic Speech Recognition,” in _Proc. European Signal Processing Conference (EUSIPCO)_ , 2017, pp. 548–552.
  * [26] D. Ng, Y. Xiao, J. Q. Yip, Z. Yang, B. Tian, Q. Fu, E. S. Chng, and B. Ma, “Small Footprint Multi-channel Network for Keyword Spotting with Centroid Based Awareness,” in _Proc. INTERSPEECH_ , 2023, pp. 296–300.
  * [27] E. Çakir, G. Parascandolo, T. Heittola, H. Huttunen, and T. Virtanen, “Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detection,” _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , vol. 25, no. 6, pp. 1291–1303, 2017.
  * [28] F. Ronchini, S. Cornell, R. Serizel, N. Turpault, E. Fonseca, and D. P. W. Ellis, “Description and Analysis of Novelties Introduced in DCASE Task 4 2022 on the Baseline System,” in _Proc. the Detection and Classification of Acoustic Scenes and Events Workshop (DCASE)_ , 2022.
  * [29] T. Khandelwal and R. K. Das, “A Multi-Task Learning Framework for Sound Event Detection using High-level Acoustic Characteristics of Sounds,” in _Proc. INTERSPEECH_ , 2023, pp. 1214–1218.
  * [30] F. Ronchini, R. Serizel, N. Turpault, and S. Cornell, “The Impact of Non-Target Events in Synthetic Soundscapes for Sound Event Detection,” in _Proc. the Detection and Classification of Acoustic Scenes and Events Workshop (DCASE)_ , 2021, pp. 115–119.
  * [31] G. Dekkers, S. Lauwereins, B. Thoen, M. W. Adhana, H. Brouckxon, B. Van den Bergh, T. Van Waterschoot, B. Vanrumste, M. Verhelst, and P. Karsmakers, “The sins database for detection of daily activities in a home environment using an acoustic sensor network,” in _Proc. the Detection and Classification of Acoustic Scenes and Events Workshop (DCASE)_ , 2017, pp. 1–5.
  * [32] A. Mesaros, T. Heittola, and T. Virtanen, “Tut database for acoustic scene classification and sound event detection,” in _Proc. European Signal Processing Conference (EUSIPCO)_. IEEE, 2016, pp. 1128–1132.
  * [33] D. Snyder, G. Chen, and D. Povey, “Musan: A music, speech, and noise corpus,” _arXiv preprint:1510.08484_ , 2015.
  * [34] L. Prechelt, “Early Stopping-but When?” _Neural Networks: Tricks of the trade_ , pp. 55–69, 2002.
  * [35] J. Ebbers, R. Haeb-Umbach, and R. Serizel, “Post-Processing Independent Evaluation of Sound Event Detection Systems,” in _Proc. the Detection and Classification of Acoustic Scenes and Events Workshop (DCASE)_ , 2023.
  * [36] A. Tarvainen and H. Valpola, “Mean Teachers are Better Role Models: Weight-averaged Consistency Targets Improve Semi-supervised Deep Learning Results,” in _Proc. Advances in Neural Information Processing Systems (NeurIPS)_ , vol. 30, 2017.