format=plain,labelformat=simple, labelsep=period

#  Adaptive Resource Management for a Virtualized Computing Platform within
Edge Computing

Thembelihle Dlamini12, Ángel Fernández Gambín1 {dlamini,
<EMAIL_ADDRESS>1Department of Information Engineering, University of
Padova, Padova, Italy  2Athonet, Bolzano Vicentino, Vicenza, Italy

###### Abstract

In virtualized computing platforms, energy consumption is related to the
computing-plus-communication processes. However, most of the proposed energy
consumption models and energy saving solutions found in literature consider
only the active Virtual Machines, thus the overall operational energy
expenditure is usually related to solely the computation process. To address
this shortcoming, in this paper we consider a computing-plus-communication
energy model, within the  Multi-access Edge Computing (MEC) paradigm, and then
put forward a combination of a traffic engineering- and MEC Location Service-
based online server management algorithm with  Energy Harvesting (EH)
capabilities, called Automated Resource Controller for Energy-aware Server
(ARCES), for autoscaling and reconfiguring the computing-plus-communication
resources. The main goal is to minimize the overall energy consumption, under
hard per-task delay constraints (i.e., Quality of Service (QoS)). ARCES
jointly performs (i) a short-term server demand and harvested solar energy
forecasting, (ii) VM soft-scaling, workload and processing rate allocation and
lastly, (iii) switching on/off of transmission drivers (i.e., fast tunable
lasers) coupled with the location-aware traffic scheduling. Our numerical
results reveal that ARCES achieves on average energy savings of $69$%, and an
energy consumption ranging from $31$%-$45$%and from $21$%-$25$% at different
values of per-VM reconfiguration cost, with respect to the case where no
energy management is applied.

###### Index Terms:

Multi-access edge computing, energy harvesting, soft-scaling, adaptive
control.

††publicationid: pubid: 978-1-7281-2294-6/19/$31.00 ©2019 IEEE

##  I Introduction

The data growth generated by pervasive mobile devices and the Internet of
Things, couple with the demand for ultra-low latency, requires high
computation resources which are not available at the end-user device.
Undoubtedly, offloading to a powerful computational resource-enriched server
located closer to mobile users is an ideal solution. Thus, Multi-access Edge
Computing (MEC [1]) has recently emerged to enable low-latency and location-
aware data processing at the network edge, i.e., in close proximity to mobile
devices, sensors, actuators and connected things. Despite the potential
presented by MEC, the computing resources (i.e., VMs) plus communication
within the MEC server (node) raises concerns related to energy consumption, in
the effort to build greener networks and reduce carbon emissions into the
atmosphere. Nonetheless, with the integration of EH into the edge (or
computing) system [2], resulting into an EH-powered MEC (EH-MEC) system, the
carbon footprint and the dependence on the power grid can be minimized.

The energy drained in the computing platform due to the computing-plus-
communication processes is associated with (i) the running VMs [3][4] and (ii)
the communication within the server’s virtual network [5] (see Fig. 3.5 in
this reference). It is observed in the literature that most of the existing
Energy Saving (ES) studies have involved autoscaling [6][7][8][9][10] (scaling
up/down the number of computing nodes/servers or VMs), VM migration [11]
(movement of a VM from one host to another) and soft resource scaling [12]
(shortening the access time to physical resources), all hereby referred to as
VM soft-scaling, i.e., the reduction of computing resources per time instance.
Hence, the proposed energy models (see [13] for a summary of the proposed
models) and the overall operational expenditure of the computing node is
usually related to the computation process (i.e., the running VMs),
overlooking the communication processes within the server.

Regarding energy consumption related to communication within a computing node,
it is shown in [14][15] that having the least number of data transmission
drivers (fast tunable lasers) can yield significant amount of ESs. It is worth
observing that both works, [14][15], are not along the direction of MEC, but
they propose the tuning of the transmission drivers as one of the ES
strategies within the Mobile Network (MN) infrastructure. Thus, ESs within the
MEC server can be jointly achieved by launching an optimal number of VMs for
computing, and transmission drivers coupled with the location-aware traffic
routing for real-time data transfer.

###  I-A Motivation

The virtualized MEC node is equipped with higher computational and storage
resources compared to the end-users devices, in order to handle the
computation workload being generated at the network edge. However, while MEC
tries to meet the computational demand and the guarantee of low-latency, the
issue of energy consumption is still a challenge within the virtualized
computing node [3][4]. To address this challenge: (i) it is expected that the
Network Function Virtualization (NFV) framework can exploit the benefits of
virtualization technologies to significantly reduce the energy consumption in
MN infrastructure; (ii) the current trends in battery and solar module costs
show an expected reduction. This two points motivate the integration of MEC
and EH systems towards green computing [2][16].

###  I-B Related work

For several years, great effort has been devoted to study energy savings in
computing environments with the aim of minimizing the energy consumption.
Procedures for the dynamic on/off switching of servers have been proposed as a
way of minimizing energy consumption in computing platforms. In [6], computing
resources are provisioned depending on the expected server workloads via a
reinforcement learning-based resource management algorithm, which learns the
optimal policy for dynamic workload offloading and servers autoscaling. Our
previous works in [7] and [17], focus on the provision of computing resources
(VMs) based on a Limited Lookahead Control (LLC) policy and the network impact
(the use of traffic load as a performance metric [18]), after forecasting the
future workloads and harvested energy. A single Base Station (BS) optimization
case is considered for an off-grid site in [7], and a multiple BS optimization
case, each BS site powered by hybrid energy sources, is studied in [17] where
the edge management procedures are enabled by an edge controller. This work
differs from our previous works as the MEC server is placed in proximity to a
BS cluster, and not one co-located for each BS. Moreover, here we focus on the
integration of communication-related energy consumption by considering the
tuning of the transmission drivers, which is a novel concept within the MEC
paradigm. In [11], Central Processing Unit (CPU) utilization thresholds are
used to identify over-utilized servers. VMs are migrated to servers that will
accept them without incurring in high energy costs. Subsequently, the idle
servers are turned-off.

Energy management is also of interest in data centers using virtualization
technologies. Along the same lines of VM soft-scaling, in [8] a traffic
engineering-based adaptive approach is presented with the aim of minimizing
energy consumption induced by computing, communication and reconfiguration
costs of virtualized clouds. An iterative method is used to obtain ESs within
a server that transmits wirelessly to clients. Then, in [9] the computing-
plus-communication is also considered towards a goal of saving energy through
an adaptive transmission rate for a Fog node. An automated server provisioning
algorithm that aims to meet workload demand while minimizing energy
consumption in data centers is presented in [10]. Here, energy-aware server
provisioning is performed by taking into account trade-offs between cost,
performance, and reliability. Lastly, a soft resource scaling mechanism is
proposed in [12] where the scheduler shortens the maximum resource usage time
for each VM, i.e., the time slice allocated for using the underlying physical
resources, in order to compensate for the low energy savings achieved with
Dynamic Voltage and Frequency Scaling (DVFS).

###  I-C Objective and Contributions

The main contributions of this work are as follows:

  * •

we consider the aforementioned scenario, where MEC and EH are combined into a
single system located close to a BS cluster, towards energy self-
sustainability in MNs. The EH-MEC system is equipped with solar panels for EH
and an Energy Buffer (EB) for energy storage.

  * •

We consider a computing-plus-communication energy model within the MEC
paradigm, formulating a constrained optimization problem. Due to the non-
linear behavior of the rate-vs-power relationship, the optimization problem is
non-convex. To solve it, we convexify the function by using Geometric
Programming (GP [19]) and then employing the CVXOPT toolbox111M. Andersen and
J. Dahl. CVXOPT: Python Software for Convex Programming, 2019. [Online].
Available: https://cvxopt.org/ and approximations.

  * •

We forecast the short-term future server workload and harvested energy, by
using a Long Short-Term Memory (LSTM) neural network [20], to enable
foresighted optimization.

  * •

Lastly, we develop an online controller-based algorithm called Automated
Resource Controller for Energy-aware Server (ARCES) for the MEC server
management based on LLC theory [21] and energy management procedures. The main
goal is to minimize the overall energy consumption, under hard per-task delay
constraints (i.e., QoS), through the joint consideration of VM soft-scaling
and the tuning of transmission drivers coupled with the location-aware traffic
routing. To the best of our knowledge, this is a novel concept within the MEC
paradigm. ARCES considers future server workloads, onsite green stored energy,
and target BS (based on the Location Service (LS) [22]), and then enable ES
procedures.

The proposed optimization strategy is able to reduce the energy consumption
under the guidance of the online resource controller and the energy management
procedures.

The rest of the paper is organized as follows. The system model is presented
in Section II. In Section III, we detail the optimization problem and the
proposed LLC-based online algorithm. Simulation results are discussed in
Section IV. Lastly, we conclude our work in Section V.

Figure 1: Virtualized computing system powered by hybrid energy sources: on-
grid power and green energy. The electromechanical switch (SW) selects the
appropriate source of energy.

##  II System Model

As a major deployment of MEC [1], the considered network scenario is
illustrated in Fig. 1. It consists of a cache-enabled, TCP/IP offload-enabled
(partial computation at the network adapter), virtualized MEC server hosting
$M$ VMs and it is assumed to be deployed at an aggregation point [1][2], i.e.,
a point in proximity to a group of BSs interconnected to the MEC server for
computation offloading. The MEC node is assumed to be equipped with higher
computational and storage resources compared to the end-user device. The
server clients are assumed to be mobile users moving in groups and they are
represented by the Reference Point Group Mobility Model (RPGM [23]). Their
current locations are known through the LS Application Programmable Interface
(API) [22], in the MEC platform, which is a service that supports UE’s
location retrieval mechanism, and then passing the information to the
authorized applications within the server. The computing site is empowered
with EH capabilities through a solar panel and an EB that enables energy
storage. Energy supply from the power grid is also available for backup. The
Energy Manager (EM) is an entity responsible for selecting the appropriate
energy source and for monitoring the energy level of the EB. The virtualized
Access Control Router (ACR) of Fig. 1 acts as an access gateway, responsible
for routing, and it is locally hosted as an application. Moreover, we consider
a discrete-time model, whereby time is discretized as $t=1,2,\dots$, and each
time slot $t$ has a fixed duration $\tau$.

###  II-A Server Workload and Energy Consumption

For many MN services, the workload demand exhibits a diurnal behavior, thus it
suffices to forecast the short-term server workload (using historical datasets
[2][10]) and then enable dynamic resource management within the server. In
this work, anonymized real server workload traces obtained from [24] are used
to emulate server workloads due to the difficulties in obtaining relevant open
source datasets containing computing requests. A trace file consist of the
file size, session duration, total number of packets and average transmission
rate, over one day. In our numerical results, we use the total number of
packets, denoted by $L_{\rm in}(t)$ ([bits]), to represent the buffered (or
admitted) computation workload at the input buffer at time slot $t$ (see red
curve in Fig. 2). In addition, we assume that the upper-bounded input/output
(I/O) queue’s of Fig. 1 are loss-free and they implement the First-In First-
Out (FIFO) service discipline, thus $L_{\rm in}(t)=L_{\rm out}(t)$, where
$L_{\rm out}(t)$ is the amount of the aggregate computation result stored at
the output buffer.

The total energy consumption ([$\text{\,}\mathrm{J}$]) for the virtualized
computing platform is formulated as follows, inspired by [7][8] and the
virtualization knowledge from [5]:

$\mbox{$\theta_{\rm MEC}(t)=\theta_{\rm COMP}(t)+\theta_{\rm COMM}(t)$}\,,$
(1)

where $\theta_{\rm COMP}(t)$ is the energy drained due to computation and
$\theta_{\rm COMM}(t)$ due to intra-communications processes in the MEC server
at time slot $t$.

$0$$0.2$$0.4$$0.6$$0.8$$1$1:005:0010:0015:0020:00

Normalized values

Time [h]ServerSolar

Figure 2: Example traces for server workloads from [24] and harvested solar
energy from [25].

#### Computing energy

$\theta_{\rm COMP}(t)=\theta_{\rm CPU}(t)+\theta_{\rm SC}(t)+\theta_{\rm
TOE}(t)$, where $\theta_{\rm CPU}(t)$ is the energy drained due to the running
VMs, w.r.t CPU utilization, and $\theta_{\rm SC}(t)$ is the energy drained due
to VM switching the processing rates $f_{m}(t)\in[0,f_{\rm max}]$. $f_{\rm
max}$ [(bit/s)] is the maximum processing rate for VM $m$. $\theta_{\rm
TOE}(t)$ is the energy induced by the TCP/IP offload on the network interface
card (NIC, e.g., TCP/IP checksum offload). In practice, the VMs are
instantiated on top of the CPU cores and each VM processes the currently
allotted task by managing its own local virtualized computing resources, thus
we model the processing rates to be between $f_{0}=0$ (represents zero speed
of the VM, e.g., deep sleep or shutdown) and $f_{\rm max}$. Here, we assume
that real-time processing of computation workloads is performed in parallel
over the VMs interconnected by a power-limited and rate-adaptive switched
Virtual Local Area Network (VLAN).

Considering that $\theta_{\rm CPU}(t)$ is related to the number of VMs running
in time slot $t$, named $M(t)\leq M$, and on the CPU frequency that is
allotted to each VM, $\theta_{\rm CPU}(t)$ is obtained using the linear
relationship between the CPU utilization contributed by VM $m$, and the energy
drained is, inspired by [7]:

$\mbox{$\theta_{\rm CPU}(t)=\sum_{m=1}^{M(t)}\theta_{{\rm
idle},m}(t)+\theta_{{\rm dyn},m}(t)$},$ (2)

where $\theta_{{\rm idle},m}(t)$ represents the static energy drained by VM
$m$ in the idle state, and the quantity $\theta_{{\rm
dyn},m}(t)=\alpha_{m}(t)(\theta_{{\max},m}(t)-\theta_{{\rm idle},m}(t))$
represents the dynamic energy component of VM $m$, where
$\alpha_{m}(t)=(f_{m}(t)/f_{\max})^{2}$ [21] is a load dependent factor and
$\theta_{{\max},m}(t)$ is the maximum energy that VM $m$ can drain.

Next, we remark that the VM switching cost $\theta_{\rm SC}(t)$ depends on the
frequency reconfiguration, i.e., the transition from $f_{1}(t)$ (current
processing rate for VM $m$) to $f_{2}(t)$ (the next processing rate), as an
example. In short, the energy cost depends on the absolute processing rate
gap, $|f_{2}(t)-f_{1}(t)|$. Thus, $\theta_{\rm SC}(t)$ is defined as [8]:

$\mbox{$\theta_{\rm
SC}(t)=\sum_{m=1}^{M(t)}\kappa_{e}(f_{2}(t)-f_{1}(t))^{2}$},$ (3)

where $\kappa_{e}$ is the the per-VM reconfiguration cost caused by a unit-
size frequency switching. Typically, $\kappa_{e}$ is limited to a few hundreds
of $\text{\,}\mathrm{mJ}$ per $($\text{\,}\mathrm{MHz}$)^{2}$.

At this regard, we put forward the following: at the beginning of time slot
$t$, the online resource controller adaptively allocates the available virtual
resources and thus determines the VMs demanded, $M(t)$, the workload allotted
to VM $m$, denoted by $\lambda_{m}(t)$, and $f_{m}(t)$ for VM $m$ that will
yield the desired or expected processing time,
$\chi_{m}(t)=\lambda_{m}(t)/f_{m}(t)$. Note that $L_{\rm
in}(t)=\sum_{m=1}^{M(t)}\lambda_{m}(t)$. Moreover, in practical application
scenarios, the maximum per-VM computation load to be computed is generally
limited up to an assigned value, named $\lambda_{\rm max}$. Lastly, the VM
provisioning and workload allocation is discussed in Section III-B2, and
$f_{m}(t)\stackrel{{\scriptstyle\Delta}}{{=}}\lambda_{m}(t)/\Delta$.

Along the same lines of computation, advancement in TCP/IP Offload Engine
(TOE) technology enables partial computation in the server’s NIC [26], i.e.,
some TCP/IP processing (e.g., checksum computation) is offloaded to a
specialized hardware on the network adapter, relieving the host CPU from the
overhead of processing TCP/IP. Thus, $\theta_{\rm TOE}(t)$ is obtained by
using the fact that it is data volume dependent and then determined using the
workload volume received. Due to the lack of an existing TOE energy model, we
rely on the performance measure for the Broadcom (Fibre) $10$ Gbps NIC [26],
which is considered here as an example of a TCP/IP offload-capable device.
Then, $\theta_{\rm TOE}(t)$ is obtained as:

$\mbox{$\theta_{\rm TOE}(t)=\zeta(t)\,\theta_{\rm idle}^{\rm
TOE}(t)\,+\theta_{\rm max}^{\rm TOE}(t)$},$ (4)

where $\theta_{\rm idle}^{\rm TOE}(t)>0$ is the energy drained by the TOE when
powered, with all links connected without any data transfer. This motivates
the idea of tuning even the NIC so that the energy drained is always zero when
there is no data transfer. For this, we have $\zeta(t)=(0,1)$ as the NIC
switching status indicator ($1$ for active state and $0$ for idle state).
$\theta_{\rm max}^{\rm TOE}(t)=\frac{L_{\rm in}(t)}{\eta}$ is the maximum
energy drained by the TOE. $\eta$ is a fixed value measured in
[Gbit/$\text{\,}\mathrm{J}$].

#### Communication energy

$\theta_{\rm COMM}(t)=\theta_{\rm VLAN}(t)+\theta_{\rm WCOM}(t)$, where
$\theta_{\rm VLAN}(t)$ is the energy drained due to the communication links
(to-and-from each VM), and $\theta_{\rm WCOM}(t)$ is the energy drained due to
the number of transmission (optical) drivers used for the data transfer to
target BS(s).

The communication energy within the VLAN is obtained by using the Shannon-
Hartley exponential analysis. Here, we assume that each VM $m$ communicates
with the resource controller through a dedicated reliable link, that operates
at the transmission rate of $r_{m}(t)$ [(bit/s)]. Thus, the energy needed for
sustaining the two-way $m^{\rm th}$ link is defined as, inspired by [16]:

$\mbox{$\theta_{\rm
VLAN}(t)=2\,\sum_{m=1}^{M(t)}P_{m}(r_{m}(t))(\lambda_{m}(t)/r_{m}(t))$},$ (5)

where $P_{m}(r_{m}(t))=\Gamma_{m}(2^{r_{m}(t)/W_{m}}-1)$ is the power drained
by the $m^{\rm th}$ communication link and $\Gamma_{m}=\frac{W_{m}\times
N_{0}^{(m)}}{g_{m}}$.
$N_{0}^{(m)}($\text{\,}\mathrm{W}\mathrm{/}\mathrm{Hz}$)$ is the noise
spectral power density, $W_{m}$ is the bandwidth, and $g_{m}$ is the (non-
negative) gain of the $m^{\rm th}$ link. In practical application scenarios,
the maximum per-slot communication rate within the intra-VLAN is generally
limited up to an assigned value $r_{\rm max}$. Thus, the following hard
constraint must hold: $\sum_{m=1}^{M(t)}r_{m}(t)\leq r_{\rm max}$.

Before proceeding, we consider the two-way per-task execution delay ([s]). We
have the $m=\\{1,\dots,M(t)\\}$ link connection delays, each denoted by
$\Omega_{m}(t)=\lambda_{m}(t)/r_{m}(t)$, and $\chi_{m}(t)\leq\Delta$, where
$\Delta$ is the maximum per-slot and per-VM processing time ([s]). At this
regard, we note that $\Delta$ is also the server’s response time, i.e., the
maximum time allowed for processing the total computation load and it is fixed
in advance regardless of the task size allocated to VM $m$. Since parallel
real-time processing is assumed in this work, the overall communication
equates to $2\,\Omega_{m}(t)+\Delta$. Therefore, the hard per-task delay
constraint on the computation time is:
$\max\\{2\,\Omega_{m}(t)\\}+\Delta\leq\tau_{\rm max}$, where $\tau_{\rm max}$
is the maximum tolerable delay, which is fixed in advance.

Finally, $\theta_{\rm WCOM}(t)$ depends on the number of laser (optical)
drivers, named $Y(t)\leq Y$ ($Y$ is the total number of them), that are
required for transferring $\ell_{y}(t)\in L_{\rm out}(t)$ in time slot $t$
($\ell_{y}(t)$ is the downlink traffic volume ([bits] of the driver at slot
$t$). $L_{\rm out}(t)$ is accumulated over a fixed period of time to form a
batch at the output buffer. At this regard, we note that a large number of
drivers yield large transmission speed while at the same time resulting into
high energy consumption [15]. Therefore, the energy consumption can be
minimized by launching an optimal number of drivers for the data transfer.
Moreover, for every mobile client who offloaded their task into the MEC server
associated with the radio nodes, i.e. BSs, its location and the computation
result is known through the UE subscription procedure (i.e., through the LS),
thus enabling the location-aware traffic routing and obtaining $Y(t)$.

The energy drained during the data transmission process consists of the
following: a constant energy for utilizing each fast tunable driver denoted by
$O_{{\rm opt},y}(t)$ ([$\text{\,}\mathrm{J}$/s]), the target transmission rate
$r_{0}$ [bits/s] and $L_{\rm out}(t)$. Thus, the energy is, inspired by
[14][27]:

$\mbox{$\theta_{\rm WCOM}(t)=\sum_{y=1}^{Y(t)}\frac{O_{{\rm
opt},y}(t)\,l_{y}(t)}{r_{0}}$},$ (6)

where the parameter $Y(t)$ is obtained using the total number of target BSs as
$Y(t)=\big{\lceil}\frac{1}{\alpha}\cdot(\frac{\omega(t)+1}{\omega(t)})^{2}\big{\rceil}$
(see [14]), where $\omega(t)=\sqrt{\frac{\Upsilon}{\sigma N_{\rm BS}(t)}}$.
$\alpha\in(0,1]$ is a controllable factor that determines the delay constraint
of optical networks, $\sigma$ ([$\text{\,}\mathrm{ms}$]) is the
reconfiguration cost for tuning the transceivers, $N_{\rm BS}(t)$ is an
integer value representing the total number of target BSs at time slot $t$,
and $\Upsilon$ is the number of time slots at which the computed workload is
accumulated at the output buffer. $\alpha,\sigma,$ and $\Upsilon$ are fixed
values. $L_{\rm out}(t)$ is equally distributed over the $Y(t)$ drivers.

###  II-B Energy Patterns and Storage

The energy buffer of Fig. 1 is characterized by its maximum energy storage
capacity $B_{\rm max}$, and power charging/discharging and leaking losses are
not assumed. At each time slot $t$, the EM provides the energy level report to
the MEC server, through the pull mode procedure (e.g., File Transfer Protocol
[28]), thus the EB level $B(t)$ is known, enabling the provision of the
required computation and communication resources, i.e., the VMs and laser
drivers.

In this work, the amount of harvested energy $H(t)$ in time slot $t$ is
obtained from open-source solar traces within a solar panel farm located in
Armenia [25] (see green curve in Fig. 2), where the dataset time scale matches
our time slot duration ($1\text{\,}\mathrm{min}$). The dataset is the result
of daily environmental records for a place assumed to be free from surrounding
obstructions (e.g., buildings, shades). In our numerical results, $H(t)$ is
obtained by picking one day data from the dataset and then scaling the solar
energy to fit the EB capacity $B_{\rm max}$ of $490\text{\,}\mathrm{kJ}$.
Thus, the available EB level $B(t+1)$ at the beginning of time slot $t+1$ is
calculated as follows:

$B(t+1)=B(t)+H(t)-\theta_{{\rm MEC}}(t)+E(t),$ (7)

where $B(t)$ is the energy level in the battery at the beginning of time slot
$t$, $\theta_{\rm MEC}(t)$ is the energy consumption of the computing platform
over time slot $t$, see Eq. (1), and $E(t)\geq 0$ is the amount of energy
purchased from the power grid. We remark that $B(t)$ is updated at the
beginning of time slot $t$ whereas $H(t)$ and $\theta_{\rm MEC}(t)$ are only
known at the end of it.

For decision making by the resource controller, the received EB level reports
are compared with the following thresholds: $B_{\rm low}$ and $B_{\rm up}$,
respectively termed the lower and the upper energy threshold with $0<B_{\rm
low}<B_{\rm up}<B_{\rm max}$. $B_{\rm up}$ corresponds to the desired energy
buffer level and $B_{\rm low}$ is the lowest EB level that the MEC server
should ever reach. The suitable energy source at each time slot $t$ is
selected based on the forecast expectations, i.e., the expected harvested
energy $\hat{H}(t)$. If $\hat{H}(t)$ is enough to reach $B_{\rm up}$, no
energy purchase is needed. Otherwise, the remaining amount up to $B_{\rm up}$,
i.e., $E(t)=B_{\rm up}-B(t)$ is bought from the electrical grid. Our
optimization framework in Section III-A makes sure that $B(t)$ never falls
below $B_{\rm low}$ and guarantees that $B_{\rm up}$ is reached at every time
slot.

##  III Problem Formulation

In this section, we formulate an optimization problem to obtain reduced energy
consumption through short-term server workload and harvested solar energy
forecasting along with server management procedures. The optimization problem
is defined in Section III-A, and the online server management procedures are
presented in Section III-B.

###  III-A Optimization Problem

On per time slot basis, the online controller adaptively schedules the
communication and computing resources, at the same time receiving the energy
level report from the EM. The goal is to minimize the overall resulting
communication-plus-computing energy, i.e., the energy consumption related to
the MEC server’s VMs and transmission drivers. To achieve this, for
$t=1,\dots,T$, where $T$ is the optimization horizon, we define the
optimization problem as:

P1 $\displaystyle:$
$\displaystyle\min_{\mathcal{E}}\mbox{$\sum_{t=1}^{T}\theta_{\rm MEC}(t)$}$
subject to: $\displaystyle{\rm C1}$ $\displaystyle:$ $\displaystyle d\leq
M(t)\leq M,$ $\displaystyle{\rm C2}$ $\displaystyle:$ $\displaystyle B_{\rm
low}\leq B(t)\leq B_{\rm max},$ $\displaystyle{\rm C3}$ $\displaystyle:$
$\displaystyle 0\leq f_{m}(t)\leq f_{\rm max},$ $\displaystyle{\rm C4}$
$\displaystyle:$ $\displaystyle 0\leq\lambda_{m}(t)\leq\lambda_{\rm max},$
$\displaystyle{\rm C5}$ $\displaystyle:$ $\displaystyle\chi_{m}(t)\leq\Delta,$
$\displaystyle{\rm C6}$ $\displaystyle:$
$\displaystyle\mbox{$\sum_{m=1}^{M(t)}r_{m}(t)\leq r_{\rm max}$},$
$\displaystyle{\rm C7}$ $\displaystyle:$
$\displaystyle\max\\{2\,\Omega_{m}(t)\\}+\Delta\leq\tau_{\rm max},$

where
$\mathcal{E}\stackrel{{\scriptstyle\Delta}}{{=}}\\{M(t),\\{\alpha_{m}(t)\\},\\{P_{m}(t)\\},\\{\lambda_{m}(t)\\},\zeta(t),Y(t)\\}$
is the set of objective variables to be configured at slot $t$ in the MEC
server, for the computing-plus-communication processes. Regarding the
constraints, C1 forces the required number of VMs, $M(t)$, to be always
greater than or equal to a minimum number $d\geq 1$: the target of this is to
be always able to handle mission critical communications. C2 makes sure that
the EB level is always above or equal to a preset threshold $B_{\rm low}$, to
guarantee energy self-sustainability over time. C3 and C4, bound the maximum
processing rate and workloads of each running VM $m$. Constraint C5 represents
a hard-limit on the corresponding per-slot and per-VM processing time.
Furthermore, C6 bounds the aggregate communication rate sustainable by the
VLAN to $r_{\rm max}$ and C7 forces the server to process the offloaded tasks
within the set value $\tau_{\rm max}$.

From P1, we note that $\theta_{\rm MEC}(t)$ consists of a non-convex
component, i.e., Eq. (5), while the others are convex and non-decreasing.
Then, Eq. (5) can be convexified into a convex function using GP theory [19],
by introducing alternative variables and approximations. In this, we introduce
fixed parameters (i.e., $\mu_{m},\nu_{m}$) and approximations. Dropping the
index $t$ for convenience, we let $r_{m}=2\,\lambda_{m}/(\tau_{\rm
max}-\Delta)$. We obtain $P_{m}(r_{m})$ in terms of $\lambda_{m}$, by
rearranging the Shannon-Hartley expression and substituting the value of
$r_{m}$, as: $\hat{P}_{m}(r_{m})=\frac{((2\,\lambda_{m}/(\tau_{\rm
max}-\Delta))-\nu_{m}\,W_{m})\ln 2}{\mu_{m}W_{m}}+\ln(N_{0}^{(m)})-\ln g_{m}$.
From the Shannon-Hartley expression, we simply observed the presence of the
log-sum-exp function as it has been proven to be convex in [29] and recall
that $P_{m}(r_{m})=\exp(\hat{P}_{m}(r_{m}))$.

To solve P1, we leverage the use of LLC [21], GP [19], and heuristics,
obtaining the feasible system control inputs
$\psi(t)=(M(t),\\{\alpha_{m}(t)\\},\\{P_{m}(t)\\},\\{\lambda_{m}(t)\\},\zeta(t),Y(t))$,
that yield the best system behavior within $T$.

###  III-B Resource Controller Design and Server Management

In this subsection, a server workload and energy harvesting forecasting
method, and an online resource management algorithm are proposed to solve the
previously stated problem P1. In subsection III-B1, we discuss the LSTM neural
network used to predict the short-term future server workloads and harvested
energy, then in subsection III-B2, we solve P1 by using LLC principles, GP
theory, and heuristics, and lastly, in subsection III-B3 we put forward the
ARCES algorithm.

####  III-B1 Server workload and energy prediction

in order to estimate the system workload over the prediction horizon $T$, we
perform time series prediction, i.e., we obtain $T=3$ estimates of $\hat{\rm
L}(t+1)$ and $\hat{H}(t+1)$, by using an LSTM network developed in Python
using TensorFlow deep learning libraries (Keras, Sequential, Dense, LSTM),
with a hidden layer of $4$ LSTM neurons, and an output layer that makes a
single value prediction. The dataset is split as $67\%$ for training and
$33\%$ for testing. As for the performance measure of the model, we use the
Root Mean Square Error (RMSE). In this work, prediction steps similar to [7]
are adopted (see Table I in this reference), and Fig. 3 shows the prediction
results that will be discussed in Section IV-B.

####  III-B2 Edge system dynamics

we denote the system state vector at time $t$ by
$\textupsilon(t)=(M(t),Y(t),B(t))$, which contains the number of active VMs,
$M(t)$, transmission drivers, $Y(t)$, and the EB level, $B(t)$. The input
vector
$\psi(t)=(M(t),\\{\alpha_{m}(t)\\},\\{P_{m}(t)\\},\\{\lambda_{m}(t)\\},\zeta(t),Y(t))$
drives the MEC server behavior (handles the joint VM soft-scaling and the
tuning of transmission drivers) at time $t$. Note that $\\{P_{m}^{*}(t)\\}$ is
obtained with CVXOPT, and $\\{\lambda_{m}^{*}(t)\\}$ is obtained by following
remark $1$.

The system behavior is described by the discrete-time state-space equation,
adopting the LLC principles [21][30]:

$\textupsilon(t+1)=\phi(\textupsilon(t),\psi(t))\,,$ (9)

where $\phi(\cdot)$ is a behavioral model that captures the relationship
between $(\textupsilon(t),\psi(t))$, and the next state $\textupsilon(t+1)$.
Note that this relationship accounts for the amount of energy drained
$\theta_{\rm MEC}(t)$, that harvested $H(t)$ and that purchased from the
electrical grid $E(t)$, which together lead to the next buffer level $B(t+1)$
through Eq. (7). The online resource management algorithm, ARCES, finds the
best control action vector that yields the desired energy savings within the
computing environment. Specifically, for each time slot $t$, problem (III-A)
is solved, obtaining control actions for the prediction horizon $T$. The
control action that is applied at time $t$ is $\psi^{*}(t)$, which is the
first one in the retrieved control sequence. This control amounts to setting
the number of instantiated VMs, $M^{*}(t)$ (along with their obtained
$\\{\alpha_{m}^{*}(t)\\}$, $\\{P_{m}^{*}(t)\\}$, $\\{\lambda_{m}^{*}(t)\\}$
values), NIC status to either active or not, $\zeta^{*}(t)\in(0,1)$, and the
optimal transmission drivers, $Y^{*}(t)$. The entire process is repeated every
time slot $t$ when the controller can adjust the behavior given the new state
information.

Since the actual values for the system input cannot be measured until the next
time instant when the controller adjusts the system behavior, the
corresponding system state for $t+1$ can only be estimated as:

$\hat{\textupsilon}(t+1)=\phi(\textupsilon(t),\psi(t))\,.$ (10)

For these estimations we use the forecast values of load $\hat{L}_{\rm in}(t)$
and harvested energy $\hat{H}(t)$, from the LSTM forecasting module.

#### Remark 1 (VM provisioning and load distribution)

a remark on the provisioned VMs at slot $t$, $M(t)$, is in order. The number
of active VMs depends on the forecasted server workload, $\hat{L}_{\rm
in}(t+1)$, and each VM can compute an amount of up to $\lambda_{\max}$
(considering that virtualization technologies specify the minimum and maximum
amount of resources that can be allocated per VM [31]). Then, the projected
number of VMs that shall be active in slot $t$ to serve the forecasted server
workloads is hereby obtained as: $M(t)=\big{\lceil}(\hat{L}_{\rm
in}(t+1)/\lambda_{\max})\big{\rceil}$, where $\big{\lceil}\cdot\big{\rceil}$
returns the nearest upper integer. We heuristically split the workload among
VMs by allocating a workload $\lambda_{m}(t)=\lambda_{\max}$ to the first
$M(t)-1$ VMs, $m=1,\dots,M(t)-1$, and the remaining workload
$\lambda_{m}(t)=\hat{L}_{\rm in}(t+1)-(M(t)-1)\lambda_{\max}$ to the last one.
This load distribution is motivated by the shares feature [31] that is
inherent in virtualization technologies. This enables the resource scheduler
to efficiently distribute resources amongst contending VMs, thus guaranteeing
the completion of the computation process within the expected time.

####  III-B3 The ARCES algorithm

Input: |  $\textupsilon(t)$ (current state)
---|---
Output: |  $\psi^{*}(t)$ (control input vector)
01: | Parameter initialization
| ${\mathcal{S}}(t)=\\{\textupsilon(t)\\}$
02: |  for ($n$ within the prediction horizon of depth $T$) do
|  \- $\hat{L}_{\rm in}(t+n)$:= forecast the workload
|  \- $\hat{\rm H}(t+n)$:= forecast the energy
|  \- ${\mathcal{S}}(t+n)=\emptyset$
03: |  for (each $\textupsilon(t)$ in ${\mathcal{S}}(t+n)$) do
|  \- generate all reachable states $\hat{\textupsilon}(t+n)$
|  \- ${\mathcal{S}}(t+n)={\mathcal{S}}(t+n)\cup\\{\hat{\textupsilon}(t+n)\\}$
04: |  for (each $\hat{\textupsilon}(t+n)$ in $\mathcal{S}(t+n)$) do
|  \- calculate the corresponding $\theta_{\rm MEC}(\hat{\textupsilon}(t+n))$
|  end for
|  end for
|  end for
05: | \- obtain a sequence of reachable states yielding
|  minimum energy cost
06: |  $\psi^{*}(t):=$ control leading from $\textupsilon(t)$ to $\hat{\textupsilon}_{\min}$
07: | Return $\psi^{*}(t)$

Algorithm 1 ARCES Pseudocode

in order to obtain the best control action that will adjust the computing
system behavior at time $t$, with negligible computational overhead, the
controller explores the prediction horizon of comprising discrete states and
comes up with the feasibility action set that yields the minimum energy cost,
i.e.,
$\psi(t)=(M(t),\\{\alpha_{m}(t)\\},\\{P_{m}(t)\\},\\{\lambda_{m}(t)\\},\zeta(t),Y(t))$.

The algorithm pseudocode is outlined in Algorithm 1 and it follows the
technique from [21]. Starting from the initial state, the controller
constructs, in a breadth-first fashion, a tree comprising all possible future
states up to the prediction depth $T$. The algorithm proceeds as follows: A
search set $\mathcal{S}$ consisting of the current system state is initialized
(line 01), and it is accumulated as the algorithm traverse through the tree
(line 03), accounting for predictions, accumulated workloads at the output
buffer, past outputs and controls. The set of states reached at every
prediction depth $t+n$ is referred to as $\mathcal{S}(t+n)$ (line 02). Given
$\textupsilon(t)$, we first estimate the workload $\hat{L}_{\rm in}(t+n)$ and
harvested energy $\hat{H}(t+n)$ (line 02), and generate the next set of
reachable control actions by applying the input workload and energy harvested
(line 03). The energy cost function corresponding to each generated state
$\hat{\textupsilon}(t+n)$ is then computed (line 04). Once the prediction
horizon is explored, a sequence of reachable states yielding minimum energy
consumption is obtained (line 05). The control action $\psi^{*}(t)$
corresponding to $\hat{\textupsilon}(t+n)$ (the first state in this sequence)
is provided as input to the system while the rest are discarded (line 06). The
process is repeated at the beginning of each time slot $t$.

##  IV Performance Evaluation

This section present some selected numerical results of the ARCES algorithm
for real server workloads. The parameters that were used for the simulations
are listed in Table I.

TABLE I: System Parameters. Parameter | Value
---|---
Max. number of VMs, $M$ | $10$
Min. number of VMs, $d$ | $1$
Time slot duration, $\tau$ | $1\text{\,}\mathrm{min}$
Idle state energy for VM $m$, $\theta_{{\rm idle},m}(t)$ | $10\text{\,}\mathrm{J}$
Max. energy for VM $m$, $\theta_{{\max},m}(t)$ | $60\text{\,}\mathrm{J}$
per-VM reconfiguration cost, $\kappa_{e}$ | $$0.005\text{\,}\mathrm{J}$/($\text{\,}\mathrm{MHz}$)^{2}$
TOE in idle state, $\theta_{\rm idle}^{\rm TOE}(t)$ | $13.1$\text{\,}\mathrm{J}$$
Max. allowed processing time, $\Delta$ | $0.8\text{\,}\mathrm{s}$
Processing rate set, $\\{f_{m}(t)\\}$ | $\\{0,50,70,90,105\\}$
Bandwidth, $W_{m}$ | $1\text{\,}\mathrm{MHz}$
Max. number of drivers, $Y$ | $6$
Max. tolerable delay, $\tau_{\rm max}$ | $2\text{\,}\mathrm{s}$
Noise spectral density, $N_{0}^{(m)}$ | -$174$ dBm/$\text{\,}\mathrm{Hz}$
Max. VM $m$ load, $\gamma^{\max}$ |  $5$ Mbit
Driver energy, $O_{{\rm opt},y}(t)$ | $1\text{\,}\mathrm{J}\mathrm{/}\mathrm{s}$
Target transmission rate, $r_{0}$ |  $1$ Mbps
Controllable factor of delay, $\alpha$ | $0.96$
Reconfiguration overhead, $\sigma$ | $20\text{\,}\mathrm{ms}$
Energy storage capacity, $\beta_{\rm max}$ | $490\text{\,}\mathrm{kJ}$
Lower energy threshold, $\beta_{\rm low}$ |  $30$% of $\beta_{\rm max}$
Upper energy threshold, $\beta_{\rm up}$ |  $70$% of $\beta_{\rm max}$

###  IV-A Simulation Setup

00.20.40.60.811:005:0010:0015:0020:00

Normalized values

Time [h]Server (predict)Server (real)Solar (real) Solar (predict)

Figure 3: Forecast mean value for $L(t)$ and $H(t)$.

As one of the MEC deployment scenarios, we assume that the MEC server is
placed at an aggregation point where BSs in proximity can offload their
computation workload following the random real-valued arrival process. Our
time slot duration $\tau$ is set to $1\text{\,}\mathrm{min}$ and the time
horizon is set to $T=3$ time slots. The simulations are carried out by
exploiting the Python programming language.

###  IV-B Numerical Results

In Fig. 3, we show real and predicted values for the server workloads (Server)
and harvested energy (Solar) over time. We track the one-step predictive mean
value at each step of the online forecasting routine. The obtained average
prediction error (RMSE) for the server workloads and harvested energy
processes, both normalized in [0,1] for $T\in\\{1,2,3\\}$, are $L_{\rm
in}(t)=\\{0.017,0.019,0.021\\}$ and $H(t)=\\{0.038,0.039,0.039\\}$. Note that
the predictions for $L_{\rm in}(t)$ are more accurate than those of $H(t)$
(confirmed by comparing the average RMSE), due to differences in the used
dataset granularity. However, the measured accuracy is deemed good enough for
the proposed optimization.

$0$$10$$20$$30$$40$$50$$60$$70$$80$$90$123456789101112131415161718192021222324

Mean energy savings $[\%]$

Time [h]ARCESIRS

Figure 4: Mean energy savings within the MEC server.

$10$$20$$30$$40$$50$$60$$1$$2$$3$$4$$5$$6$$7$$8$$9$$10$

Average per-task consumed energy [J]

VMs, $M(t)$ARCES ($\kappa_{e}=0.005$)IRS ($\kappa_{e}=0.005$)ARCES
($\kappa_{e}=0.001$)IRS ($\kappa_{e}=0.001$)

Figure 5: Average per-task consumed energy vs VMs.

Our online server management algorithm (ARCES) is benchmarked with another
one, named Iterative-based Resource Scheduler (IRS), which is inspired by the
iterative approach from [8]. In IRS, the optimum computing-plus-communication
parameters are obtained in an iterative manner: at the end of each cycle,
convergence conditions are used to determine if the found solution is
acceptable or the optimization process should continue. The observed
conditions are as follows: (i) to ensure that the total load $L_{\rm in}(t)$
has been fully allocated with accuracy
$\frac{\sum_{m=1}^{M(t)}\lambda_{m}(t)-L_{\rm in}(t)}{L_{\rm
in}(t)}\leq\epsilon$, with $\epsilon=0.01$; (ii) to verify if the selected
working frequency $f_{m}(t)$ is able to cope with the input load $L_{\rm
in}(t)$, guaranteeing that the computation processing time is within the
server’s response time limit $\Delta$, i.e., $L_{\rm in}(t)\leq
f_{m}(t)\Delta$ . The average energy savings obtained by ARCES are shown in
Fig. 4. The average results for ARCES
($\kappa_{e}=0.005,\Gamma_{m}=$0.5\text{\,}\mathrm{mW}$,\eta=1.4$
Gbit/$\text{\,}\mathrm{J}$) show energy savings of $69$%, while IRS achieves
$56$% on average, in both cases with respect to the case where no energy
management procedures are applied; i.e., the MEC server provisions the
computing resources for maximum expected computation workload (maximum value
of $\theta_{\rm MEC}(t)$, with $M=10$, $\forall t$). As expected, the highest
energy savings peak is observed at $9\text{\,}\mathrm{h}$ as the aggregate
computation requests/workload was at its lowest with an expected increase in
the computation workload and harvested solar energy in the near future. The
effectiveness of the joint VM soft-scaling and tuning of transmission drivers,
coupled with foresighted optimization is observed in the obtained numerical
results.

In Fig. 5, we show the effects of the per-VM reconfiguration cost on
$\theta_{\rm MEC}(t)$ at $\kappa_{e}=\\{0.001,0.005\\}$
($\Gamma_{m}=$0.5\text{\,}\mathrm{mW}$,\eta=1.4$ Gbit/$\text{\,}\mathrm{J}$),
taking into account the performance of ARCES when compared with IRS for
$M(t)=1,\dots,M$. It can be observed that $\theta_{\rm MEC}(t)$ increases with
large $\kappa_{e}$ only for small values of $M(t)$, and as $M(t)$ increases
the energy consumption decreases for large $\kappa_{e}$. Moreover, ARCES leads
to an energy consumption reduction with respect IRS from $25$% to $7$% (case
of $\kappa_{e}=0.005$) and from $7$% to $5$% (case of $\kappa_{e}=0.001$).
When ARCES is compared with the case where no energy management is applied
(maximum value of $\theta_{\rm MEC}(t)$, with $M=10$, $\forall t$), the
obtained energy reduction ranges from $45$% to $31$% (case of
$\kappa_{e}=0.005$) and from $25$% to $21$% (case of $\kappa_{e}=0.001$).
These numerical results confirm that jointly autoscaling the available
computing-plus-communication resources within the computing platform provides
remarkable energy savings. These results conforms to our expectations from
[16].

##  V Conclusions

In this paper, we have envisioned a hybrid-powered MEC server placed in
proximity to a BS cluster for handling the offloaded computation workload.
Moreover, the use of green energy promotes energy self-sustainability within
the network. We have considered a computing-plus-communication energy model,
within the MEC paradigm, and then have put forward a combination of a traffic
engineering- and MEC Location Service-based online server management algorithm
with EH capabilities, called Automated Resource Controller for Energy-aware
Server (ARCES), for autoscaling and reconfiguring the computing-plus-
communication resources. The main goal is to minimize the overall energy
consumption, under hard per-task delay constraints (i.e., QoS). ARCES jointly
performs (i) a short-term server demand and harvested solar energy
forecasting, (ii) VM soft-scaling, workload and processing rate allocation and
lastly, (iii) switching on/off of transmission drivers (i.e., fast tunable
lasers) coupled with the location-aware traffic scheduling. Numerical results,
obtained with real-world energy and server workload traces, demonstrate that
the proposed algorithm (ARCES) achieves energy savings of $69$%, on average,
with an energy consumption ranging from $31$%-$45$% at high per-VM
reconfiguration cost and from $21$%-$25$% at low per-VM reconfiguration cost,
with respect to the case where no energy management techniques are applied.

## Acknowledgements

This work has received funding from the European Union’s Horizon 2020 research
and innovation programme under the Marie Sklodowska-Curie grant agreement No.
675891 (SCAVENGE).

## References

  * [1] S. Kekki, W. Featherstone, Y. Fang, P. Kuure, A. Li, A. Ranjan, D. Purkayastha, F. Jiangping, D. Frydman, G. Verin, K. Wen, K. Kim, R. Arora, A. Odgers, L. M. Contreras, and S. Scarpina, “MEC in 5G Networks,” ETSI, Sophia-Antipolis, France, Tech. Rep., Jun 2018.
  * [2] D. Thembelihle, M. Rossi, and D. Munaretto, “Softwarization of Mobile Network Functions towards Agile and Energy Efficient 5G Architectures: A Survey,” _Wireless Communications and Mobile Computing_ , 2017.
  * [3] R. Morabito, “Power Consumption of Virtualization Technologies: An Empirical Investigation,” in _IEEE International Conference on Utility and Cloud Computing (UCC)_ , Limassol, Cyprus, Dec 2015.
  * [4] Y. Jin, Y. Wen, and Q. Chen, “Energy efficiency and server virtualization in data centers: An empirical investigation,” in _IEEE Conference on Computer Communications Workshops_ , Orlando, USA, Mar 2012.
  * [5] M. Portnoy, _Virtualization essentials_. John Wiley & Sons, 2012.
  * [6] J. Xu and S. Ren, “Online Learning for Offloading and Autoscaling in Renewable-Powered Mobile Edge Computing,” in _IEEE GLOBECOM_ , Washington, USA, Dec 2016.
  * [7] T. Dlamini, Á. F. Gambín, D. Munaretto, and M. Rossi, “Online Resource Management in Energy Harvesting BS Sites through Prediction and Soft-Scaling of Computing Resources,” in _IEEE PIMRC_ , Bologna, Italy, Sep 2018.
  * [8] M. Shojafar, N. Cordeschi, D. Amendola, and E. Baccarelli, “Energy-saving adaptive computing and traffic engineering for real-time-service data centers,” in _IEEE International Conference on Communication Workshop (ICCW)_ , London, UK, Jun 2015.
  * [9] M. Shojafar, N. Cordeschi, and E. Baccarelli, “Energy-efficient Adaptive Resource Management for Real-time Vehicular Cloud Services,” _IEEE Transactions on Cloud Computing_ , 2016.
  * [10] B. Guenter, N. Jain, and C. Williams, “Managing cost, performance, and reliability tradeoffs for energy-aware server provisioning,” in _Proceedings IEEE INFOCOM_ , Shanghai, China, Apr. 2011.
  * [11] A. Beloglazov, J. Abawajy, and R. Buyya, “Energy-aware Resource Allocation Heuristics for Efficient Management of Data Centers for Cloud Computing,” _Future Generation Computer Systems_ , vol. 28, no. 5, pp. 755–768, 2012\.
  * [12] R. Nathuji and K. Schwan, “VirtualPower: coordinated power management in virtualized enterprise systems,” in _Proceedings of ACM SIGOPS symposium on Operating systems principles_ , Washington, USA, Oct 2007.
  * [13] C. Gu, H. Huang, and X. Jia, “Power Metering for Virtual Machine in Cloud Computing-Challenges and Opportunities,” _IEEE Access_ , vol. 2, pp. 1106–1116, 2014.
  * [14] S. Fu, H. Wen, J. Wu, and B. Wu, “Cross-Networks Energy Efficiency Tradeoff: From Wired Networks to Wireless Networks,” _IEEE Access_ , vol. 5, pp. 15–26, 2017.
  * [15] B. Wu, S. Fu, X. Jiang, and H. Wen, “Joint Scheduling and Routing for QoS Guaranteed Packet Transmission in Energy Efficient Reconfigurable WDM Mesh Networks,” _IEEE Journal on Selected Areas in Communications_ , vol. 32, no. 8, pp. 1533–1541, 2014.
  * [16] N. Cordeshi, M. Shojafar, and E. Baccarelli, “Energy-saving self-configuring network data centers,” _Computer Networks_ , vol. 57, no. 17, pp. 3479–3491, 2013.
  * [17] T. Dlamini, Á. F. Gambín, D. Munaretto, and M. Rossi, “Online Supervisory Control and Resource Management for Energy Harvesting BS Sites Empowered with Computation Capabilities,” _Wireless Communications and Mobile Computing_ , 2019.
  * [18] E. Oh, K. Son, and B. Krishnamachari, “Dynamic Base Station Switching-On/Off Strategies for Green Cellular Networks,” _IEEE Transactions on Wireless Communications_ , vol. 12, no. 5, pp. 2126–2136, 2013.
  * [19] W.-C. Ho, L.-P. Tung, T.-S. Chang, and K.-T. Feng, “Enhanced component carrier selection and power allocation in LTE-advanced downlink systems,” in _2013 IEEE WCNC_ , Shanghai, China, Apr 2013.
  * [20] I. Goodfellow, Y. Bengio, and A. Courville, _Deep Learning_. MIT Press, 2016.
  * [21] J. P. Hayes, “Self-Optimization in Computer Systems via On-Line Control: Application to Power Management,” in _Proceedings of the First International Conference on Autonomic Computing_ , May 2004.
  * [22] “Mobile Edge Computing (MEC): Location API,” ETSI, Sophia-Antipolis, France, Tech. Rep., Jul 2017.
  * [23] B. Fan and H. Ahmed, “A survey of mobility models,” _Wireless Adhoc Networks_ , vol. 206, pp. 147–176, 2011.
  * [24] “150 Megabit Ethernet anonymized packet traces.” [Online]. Available: http://mawi.wide.ad.jp/mawi/ditl/ditl2009/
  * [25] “Solar Radiation Measurement Data.” [Online]. Available: https://energydata.info/dataset/armenia-solar-radiation-measurement-data-2017
  * [26] S. Ripduman, R. Andrew, A. W. Moore, and M. Kieran, “Characterizing 10 Gbps network interface energy consumption,” in _IEEE 35th Conference on Local Computer Networks (LCN)_ , Colorado, USA, Oct 2010.
  * [27] Chen, Lixing and Zhou, Sheng and Xu, Jie, “Computation peer offloading for energy-constrained mobile edge computing in small-cell networks,” _IEEE/ACM Transactions on Networking_ , vol. 26, no. 4, pp. 1619–1632, 2018\.
  * [28] “3GPP TS 32.2.297, Charging Data Rececord (CDR) file format and transfer,” ETSI, Sophia-Antipolis, France, Tech. Rep., Aug 2016.
  * [29] S. Boyd and L. Vandenberghe, _Convex Optimization_. Cambridge University Press, 2004.
  * [30] S. Abdelwahed, N. Kandasamy, and S. Neema, “Online control for self-management in computing systems,” in _IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS)_ , Ontario, Canada, May 2004.
  * [31] M. Cardosa, M. R. Korupolu, and A. Singh, “Shares and utilities based power consolidation in virtualized server environments,” in _IFIP/IEEE International Symposium on Integrated Network Management_ , New York, USA, June 2009.

  *[MEC]: Multi-access Edge Computing
  *[EH]: Energy Harvesting
  *[QoS]: Quality of Service
  *[VM]: Virtual Machine
  *[VMs]: Virtual Machine
  *[ES]: Energy Saving
  *[MN]: Mobile Network
  *[NFV]: Network Function Virtualization
  *[LLC]: Limited Lookahead Control
  *[BS]: Base Station
  *[CPU]: Central Processing Unit
  *[ESs]: Energy Saving
  *[EB]: Energy Buffer
  *[LSTM]: Long Short-Term Memory
  *[LS]: Location Service
  *[BSs]: Base Station
  *[API]: Application Programmable Interface
  *[EM]: Energy Manager


# Face: Fast, Accurate and Context-Aware Audio Annotation and Classification

M. Mehrdad Morsali
Department of Electrical Engineering
Sharif University of Technology
<EMAIL_ADDRESS>
Hoda Mohammadzade
Department of Electrical Engineering
Sharif University of Technology
<EMAIL_ADDRESS>
Saeed Bagheri Shouraki
Department of Electrical Engineering
Sharif University of Technology
<EMAIL_ADDRESS>
This paper presents a context-aware framework for feature selection and
classification procedures to realize a fast and accurate audio event
annotation and classification. The context-aware design starts with exploring
feature extraction techniques to find an appropriate combination to select a
set resulting in remarkable classification accuracy with minimal computational
effort. The exploration for feature selection also embraces an investigation
of audio Tempo representation, an advantageous feature extraction method
missed by previous works in the environmental audio classification research
scope. The proposed annotation method considers outlier, inlier, and hard-to-
predict data samples to realize context-aware Active Learning, leading to the
average accuracy of $90\%$ when only $15\%$ of data possess initial
annotation. Our proposed algorithm for sound classification obtained average
prediction accuracy of $98.05\%$ on the UrbanSound8K dataset. The notebooks
containing our source codes and implementation results are available at
https://github.com/gitmehrdad/FACE.

_Keywords_ Context-aware Machine Learning $\cdot$ Active Learning $\cdot$
Sound Classification $\cdot$ Audio Annotation $\cdot$ Audio Tempo
Representation (Tempogram) $\cdot$ Semi-supervised Audio Classification
$\cdot$ Feature Selection

##  1 Introduction

In recent years, sound event classification has gained attention due to its
application in audio analysis, music information retrieval, noise monitoring,
animal call classification, and speech enhancement [1, 2, 3]. The development
in the mentioned fields requires annotated recordings [3] and larger datasets
instead of the smaller ones [4]. Since labeling audio is time-consuming and
expensive, establishing a large-scale labeled audio dataset is arduous [5].
So, it seems necessary to develop reliable methods to label the datasets.
There are several methods of determining the labels of unlabeled data. Some
papers, including [1, 4, 5, 6, 7, 8] pursue Semi-supervised Learning, and some
works, including [2, 3, 9, 10, 11, 12] seek to utilize Active Learning.
Regarding the superior annotation accuracy of Active Learning-based
approaches, this paper also focuses on Active Learning to determine the labels
of audio samples. In certainty-based Active learning, a small portion of data
with deterministic labels trains a classifier at first. Then, the classifier
labels the unlabeled data, and audio samples with the highest classification
certainties join the training set. The cycle of classifier training,
classification, and updating the training set continues until the complete
annotation of all audio samples.

Solutions proposed for the problem of audio event classification examine
several methods. In [1, 4, 5, 7, 8], they suggested utilizing deep multi-layer
neural networks to classify the data. The main issue challenging some Deep
Learning solutions is the low prediction accuracy, in the case of the model
overfitting, due to the scarcity of training data. In the audio classification
task, where usually a propitious amount of training samples are available, we
won’t face the model overfitting; however, in the audio annotation task, where
the labels are available only for a small portion of data, the overfitting
problem makes the deep multi-layer neural networks less attractive. In [2, 3,
6], they have employed classic machine learning classifiers to classify the
data. These methods are potentially less susceptible to overfitting since they
have fewer unknown parameters that should get determined during the training.
Nevertheless, the classic machine learning approaches suffer from the fixed
input size issue.

Regarding the considerable achievements of Context-aware Machine Learning
[13], this paper proposes some context-aware solutions for the addressed
problems. A context-aware design considers the current situation for decision-
making. As a case in point, based on the system’s status, a context-aware
Machine Learning system might dynamically renew the model fine-tuning and
hyperparameter adjustment. The first research contribution of this paper
includes a heuristic approach for context-aware feature selection to provide
summarized fixed-size input vectors to gain a fast, accurate, and overfitting-
resistant classification model. The investigation of the audio signal’s Cyclic
Tempogram feature would be another contribution of this work in the audio
signal classification task. The proposal of a context-aware Active Learning-
based method for audio signal annotation is the third research contribution of
this paper. The fourth research contribution is the context-aware classifier
design to obtain optimum classification results. Section 2 describes the
mentioned contributions in data preparation and feature extraction, data
annotation, and audio classification. Section 3 discusses the implementation
results for the proposed methods. Section 4 concludes the paper.

##  2 The proposed Method

###  2.1 Data Preparation and Feature Extraction

Assume a dataset of audio signals with a duration of four seconds under the
sampling rate of $22.05KHz$. Since each audio signal consists of $4\times
22050$ samples, the input vector of a classifier operating on that dataset
should contain $88200$ variables. These variables might distinguish audio
signals from each other; however, processing those lengthy input vectors
requires large amounts of memory and expensive computational resources.
Besides, as the classifier’s input vector’s length grows, the trained model
tends to be more susceptible to overfitting [14]. So, instead of regarding the
time contents of an audio signal as the classifier’s input vector, some
distinctive features extracted from the signal’s time contents would form the
input vector. A context-aware feature extraction provides good classification
accuracy and low model overfitting possibility.

Some previous works, including [15, 16, 17, 18], discussed the various feature
extraction methods applicable to the sound event classification task.
Following a context-aware approach, we evaluate different feature extraction
methods to find the most effective selection among that methods. Since this
paper desires fast classification, we overlooked the evaluation of some
methods with remarkable classification accuracy gain, like the Scattering
Transform, because of their low feature extraction speed. The setup for the
ablation study on the features contains a dataset, a feature extraction
method, and a classifier.

The employed dataset is UrbanSound8K (US8K) presented in [19], which consists
of $10$ classes of urban environment’s audio events, including air
conditioner, car horn, children playing, dog bark, drilling, engine idling,
gunshot, jackhammer, siren, and street music. Each class consists of various
counts of audio samples, and different-class data are arduous to distinguish
for at least half the data classes. There are $8732$ audio samples with
different sampling rates and a maximum duration of $4$ seconds. In this paper,
the first step of data preparation is resampling the audio signals with the
$22.05KHz$ sampling rate to obtain a uniform dataset. Data compression often
ensues signal resampling since the sampling rate of audio signals usually
exceeds $22.05KHz$. Since the US8K dataset does not specify a validation set,
we regard $10\%$ of the training samples as the validation samples. All
context-aware decisions in this paper are made based on the mentioned
validation set. The examined features contain local autocorrelation of the
onset strength envelope (Tempogram), Chromagram, Mel-scaled Spectrogram, Mel-
frequency cepstral coefficients (MFCCs), Spectral Contrast, Spectral flatness,
Spectral bandwidth, Spectral centroid, Roll-off frequency, RMS value for each
frame of audio samples, tonal centroid features (Tonnetz), and zero-crossing
rate of an audio time series. Regarding the lack of investigation history in
the environmental sound classification papers, this paper studies the
Tempogram, proposed in [20], for the first time.

Most of the mentioned feature extraction methods operate based on windowing an
audio signal, so the extracted features are referred to as static since they
represent the information of the signal’s specific windows. The delta and
delta-delta coefficients of the audio signal feature, which respectively
indicate the first-order and second-order derivatives of the features, are
referred to as dynamic features. The delta features show the speech rate, and
the delta-delta features indicate the speech’s acceleration [21]. Some feature
extraction methods generate a Spectrogram, a visual representation of the
audio signal’s spectrum of frequencies. Regarding the Spectrogram as an image,
we can exert some feature extraction properties from the Computer Vision
scope, so we apply a Gaussian filter to the feature vectors before taking
derivatives. In [14], feature selection helps to overcome the overfitting
problem of the model by reducing the input vector’s dimensions and to
obliviate the fixed input size issue mentioned in the previous section. The
input vector’s length depends on the number of windows used during the feature
extraction. When the windows’ count gets too high, the information represented
by consecutive windows won’t differ substantially, and when the windows’ count
gets too low, the amount of the extracted features won’t suffice to
distinguish different audio samples easily. To reduce the data dimensions, the
mean and the variance of the extracted features per window would replace the
original data. A similar replacement happens for the delta and delta-delta
vectors. So, the final feature vector embraces six vectors of floating-point
numbers, containing the mean and the variance values for the original feature
and first-order and second-order derivatives of it. In this paper, we used the
feature extraction tools available by Librosa, a Python package, where most
arguments of the feature extraction function are left untouched with their
default values. Nevertheless, some function arguments, including the window
length and the number of samples per window, got adjusted using the validation
data. Before the experiments, we transformed features using Quantiles
information into the standard vectors where the value of all elements lies in
the range of (-1,1).

A context-aware choice for the classifier would be the one holding the best
capability to distinguish the data. There is a dilemma in feature and
classifier selection: a classifier might hide the distinction some features
provide, while a feature set might hide the resolution some classifiers
provide. So, initially, we explore various extracted features using a specific
classifier chosen based on our speculation of the data distribution. After
feature selection, we try to find the best classifier to distinguish the data.

As the t-SNE plot for the UrbanSound8K dataset samples represented in [22]
shows, the data distribution in the features space is heterogeneous and
sparse. In a heterogeneous data distribution, the neighborhood of each data
sample embraces some data samples with different class labels. Sparse data
have a distribution in which the same-class data samples don’t shape dense
areas. Assuming that the feature selection in our work also results in
heterogeneous and sparse data distribution, we search for a classifier to
separate the data classes. Classifiers drawing a linear decision boundary fail
to separate sparse and heterogeneous data, and classifiers utilizing non-
linear kernels require expensive computational effort. So, a decision tree
classifier with an appropriate depth seems profitable. The high potential of
model overfitting utilizing the decision tree classifier necessitates using an
ensemble of tree classifiers. Despite the improvements, the model overfitting
still overshadows the accuracy of the mentioned classifier, so the XGBoost
classifier proposes a solution by moving the decision boundaries utilizing
regularization [14]. Inspired by the One-vs-All approach that avails the
application of binary classifiers for multi-class classification, we propose
to employ the One-vs-All XGBoost classifier. In fact, instead of a single
multi-class XGBoost classifier, multiple binary XGBoost classifiers draw the
decision boundaries, where each classifier tries to separate the members of a
specific class from all other data. Regarding the light duty of a binary
XGBoost compared to a multi-class XGBoost, the computational overhead of the
proposed method would not be an issue.

Figure 1: Feature selection resutls based on the proposed heuristic approach

Figure 1 depicts the feature selection results. To avoid the examination of
all features permutations, we followed a heuristic approach, which consists of
adding one new feature to each feature set at each phase, reporting the
validation sample’s classification accuracy, moving the top-two feature sets
to the next study phase, and dropping further investigations when the
classification accuracy stopped enhancement. In Figure 1, the results of Phase
III don’t show improvement compared to Phase II; therefore, we pick the
MFC+CNT combination from Phase II as our final selection. Figure 1 depicts
methods’ abbreviations in Phase II and Phase III of the experiment and shows
the time-domain features in blue and the frequency-domain features in red.
Figure 1 doesn’t represent the Mel-scaled Spectrogram feature extraction
results since MFCC features are not only convertible to the Mel-scaled
Spectrogram features but also capable of providing more classification
accuracy. Some methods like Spectral flatness, Spectral bandwidth, Spectral
centroid, and Roll-off frequency got excluded from Figure 1 because of their
poor classification accuracy results. Please note that the reported accuracies
belong to a simple classification procedure where no Active Learning or other
accuracy optimization techniques are applied. As seen in Figure 1, the
remarkable accuracy the MFCC feature provides alone clarifies the reason
behind the widespread use of that feature in most previous works. The feature
vector of the selected combination consists of $810$ floating-point numbers,
substantially reducing the computational resources required for the annotation
and the classification tasks because it is sufficient to check $810$ features
instead of $88200$ features.

Figure 2: The classification accuracy vs. the utilized classifier

Utilizing various classifiers, Figure 2 shows the classification results where
no Active Learning or other accuracy optimization techniques are applied. The
experiment focuses on finding the best classifier for the annotation task, so
it explores only the classic machine-learning methods. In Figure 2, XGB, HGB,
SVC, MLP, RF, RC, KNC, and SGD, respectively, denote the XGBoost, the
Histogram-based Gradient Boosting Classifier, the Support Vector Machine, the
Multi-layer Perceptron, the Random Forest, the Ridge Classifier, the
K-nearest-neighbors Classifier, and the Regularized Linear SVM with Stochastic
Gradient Descent learning. In Figure 2, the asterisk sign after the
classifier’s name indicates using the classifier in the One-vs-All scheme.
Each classifier uses our defined validation set as the test set and the rest
of the samples in the US8K dataset as the training set. Figure 2 shows the
results in which classifiers succeed in obtaining at least $80\%$
classification accuracy, so it does not contain classifiers like Decision
Tree, Gaussian Naive Bayes, AdaBoost, and QDA, which fail to reach $80\%$
classification accuracy. The results of Figure 2 corroborate our speculation
of the data distribution and the context-aware classifier selection to
separate the classes. As shown in Figure 2, the HGB classifier outperforms the
XGB classifier in terms of classification accuracy, but the XGB classifier
operates faster. So, for the annotation tasks, we will use a One-vs-All
XGBoost classifier, a context-aware choice that provides satisfying
classification accuracy on the validation set.

###  2.2 Data Annotation

Data: $dataset$: A dataset with unlabeled data

Result: $Label$: The predicted labels for the dataset

1 $Label\leftarrow null$;

2 $L\leftarrow dataset.SamplesCount$;

3 $LocalOutlierFactor(neighbors=1).Fit(dataset)$;

4 $I\leftarrow LocalOutlierFactor.Inlier(size=L/40)$;

5 $R\leftarrow RandomSelection(data=dataset,size=3L/40)$;

6 $Q\leftarrow ActiveLearner.query(data=dataset,size=L/20)$;

7 $T\leftarrow Concatenate(I,R,Q)$;

8 $Label[T]\leftarrow AskLabel(T)$;

9 for  _$i\leftarrow 1$ to $4$ _ do

10 $Labeller.fit(dataset[T],P[T])$;

11 $Prediction\leftarrow Labeller.Predict(dataset)$;

12 $Score\leftarrow Labeller.PredictionScore(dataset)$;

13 for  _$j\leftarrow 1$ to $L$ _ do

14 $Outlier\leftarrow LocalOutlierFactor.IsOutlier(dataset[j])$;

15 if  _$Label[j]=null$ and $Outlier=False$ and $Score[j]>0.7$_ then

16 $Label[j]\leftarrow Prediction[j]$;

17 $T.Append(j)$;

18

19 end if

20

21 end for

22

23 end for

24$Labeller.fit(dataset[T],P[T])$;

25 $Prediction\leftarrow Labeller.Predict(dataset)$;

26 for  _$j\leftarrow 1$ to $L$ _ do

27 if  _$Label[j]=null$_ then

28 $Label[j]\leftarrow Prediction[j]$;

29

30 end if

31

32 end for

Algorithm 1 The proposed method for audio event annotation

For the annotation task, conventionally, it is assumed that a human annotator
provides the labels for a small proportion of the data. Such an assumption
leads us to avoid using deep neural network classifiers to annotate new
samples because the low number of training samples and the high number of
model parameters expose the model to overfitting, which in turn causes the
prediction accuracy to drop. So, based on the results observed in Section 2.1,
we will annotate the data using a One-vs-All XGBoost classifier, a context-
aware choice that provides good classification accuracy. Another manifestation
of a context-aware framework, realized in our proposed annotation method, is
specifying the data the human labeler should annotate. Algorithm 1 shows our
proposed annotation method. The Local Outlier Factor (LOF) method, proposed in
[23], finds the inlier samples having the least Euclidean distances from their
nearest neighbor. The Active Learner, a Python tool from the modAL package,
detects hard-to-predict data that are samples with minimum prediction
certainty. According to what is shown from line $1$ to line $8$ of Algorithm
1, a human annotator determines the labels for inlier samples, hard-to-predict
samples, and some random samples, uniformly chosen from every different class.
The maximum number of selected inlier, hard-to-predict, and random samples are
respectively limited to $2.5\%$, $5\%$, and $7.5\%$ of the total sample count
in the datatset. The philosophy behind choosing inlier samples for initial
annotation is that there is a good chance that the neighborhood of the inlier
samples embraces the same class samples. The reason behind adding some random
data samples to the inlier samples is the possible imbalance in the count of
data samples of different classes in the inlier samples. Another reason for
adding random samples is to find different-class data samples adjacent to the
inlier samples.

In Algorithm 1, line $9$ to line $27$ indicate the proposed context-aware
Active Learning method. The method consists of five stages of data labeling
where the predicted samples with reliable prediction accuracy also help
determine the labels. In this work, a reliable prediction possesses at least
$70\%$ prediction accuracy score. The number of Active Learning stages and the
threshold of accepting a prediction as a reliable one are the problem’s
hyperparameters, depending on various factors, including the audio samples
count, the data separability, the classifier’s accuracy, and the
classification’s certainty. The problem’s hyperparameters got adjusted using a
validation set, $10\%$ of the unlabeled samples. The major innovation in our
method is that it overlooks labeling outlier samples until the last stage of
labeling because we believe the presence of outlier data samples among the
training samples decreases the labeling accuracy. In the first stage of Active
Learning-based labeling, we train a classifier using human-labeled data. Then
the trained classifier predicts the labels of dataset samples. Next, the
reliable predictions reach the final prediction list and the set of training
samples for the classifiers of the further labeling stages. In the last stage
of the labeling procedure, the labels predicted for the outlier samples and
the remainder of unlabeled data reach the final prediction list
unconditionally.

###  2.3 Data Classification

Figure 3 shows the proposed neural network for audio classification. Figure 3
demonstrates the convolutional blocks in blue and the locally connected
convolutional blocks in green. The locally connected convolutional blocks work
similarly to convolutional blocks; however, unlike convolutional blocks, a
different set of filters is applied at each input patch selected by the kernel
window. The number on each block indicates the kernel size for that block, and
the number above each column of blocks shows the number of filters. All blocks
enjoy Batch Normalization, followed by a LeakyReLU activation function. We
merge the outputs of five parallel streams, depicted in Figure 3, by an
addition operation. The last layers of the neural network are two fully-
connected layers, joined together through a dropout layer. We adjusted the
neural network’s hyperparameters, including filter count, kernel sizes,
dropout ratio, and the number of training epochs, using a validation set
containing $10\%$ of training samples. In the neural network design, we
preferred in-width extension to in-depth expansion, which makes the neural
network operate faster by providing the opportunity for parallel computations
of five parallel streams.

Figure 3: The proposed Neural Network for the audio classification task

##  3 Implementation Results

Following the convention of this paper’s references, [3, 6, 9, 10, 11, 12], we
have tested our proposed context-aware annotation method on the UrbanSound8K
dataset. As mentioned in Section 2, we assume a labeling budget of $15\%$,
meaning that the labels are deterministic for only $15\%$ of the data before
the labeling process. Table 1 shows the labeling accuracy results, indicating
$90\%$ of accuracy, which is improved by $29\%$ compared to the best
previously reported works.

Table 1: Annotation Accuracy Results with a Labeling Budget of 15% Method | Approach | Accuracy
---|---|---
SSCDE[6] | Semi-supervised Learning | 35
CRTAL+SSL[9, 10] | Active Learning and Semi-supervised Learning | 50
LDAL[11] | Dictionary-based Active Learning | 55
DBAL[12] | Dictionary-based Active Learning | 57
MAL[3] | Active Learning | 61
This work (FACE) | Context-aware Active Learning | 90

For the sound classification task, we trained our neural network in $25$
epochs using a $SGD$ optimizer with a learning rate of $0.01$. We determine
the maximum number of training epochs and the optimizer settings such that the
classification accuracy on the validation data, $10\%$ of the training set,
reaches its highest value. Regarding the popularity of the UrbanSound8K
dataset for the audio classification task, Table 2 shows the classification
accuracy results on the UrabanSound8K dataset. As the results of Table 2
indicate, our proposed method achieved the classification accuracy of
$98.05\%$, which is higher than any other reported works to date. Presenting
another measure for our classification method’s quality assessment, Figure 4
depicts the classification confusion matrix for the UrbanSound8K dataset. To
study the effect of Active Learning on classification accuracy enhancement, we
let the Active Learner, a Python tool from the modAL package, inquire about
the labels of some test set samples. We observed that it is possible to
achieve $100\%$ classification accuracy by providing the data labels for only
$0.9\%$ of the test data.

The notebooks containing our source codes and implementation results are
available at https://github.com/gitmehrdad/FACE.

Table 2: Classification Accuracy Results Method | Accuracy
---|---
MAL[3] | 65
CRTAL+SSL[9, 10] | 65
DBAL[12] | 67
LDAL[11] | 68
CDSS[5] | 78
AemNet-DW[24] | 83.6
Graph wo/ FF Ontology[22] | 89.0
TFCNN[25] | 93.1
SacNet-8[18] | 95.3
TSCNN[26] | 95.7
TSCNN-DS[17] | 97.2
MHAT[15] | 97.5
This work (FACE) | 98.05
This work (FACE + Active Learning) | 100
Figure 4: The classification confusion matrix on the US8K dataset

##  4 Conclusion

This paper presented a fast, accurate, and context-aware method that can get
employed to annotate and classify the audio signals. A context-aware approach
utilized in feature selection, classifier design, and annotation algorithm
design remarkably improves our methods’ accuracy in both annotation and
classification tasks. The compact extracted features vector and the
computationally efficient classification scheme avail a fast labeler,
annotating nearly $8000$ audio samples with $90\%$ of accuracy in less than a
minute using a free computing platform of Google Colaboratory. Our proposed
method obtained an average classification accuracy of $98.05\%$ on the US8K
dataset. The achieved classification accuracy is the best-reported one
compared to previous works. An Active Learning procedure boosts the
classification accuracy to the value of $100\%$ by asking for the labels of
only $0.9\%$ of the test data.

## References

  * Gururani and Lerch [2021] Siddharth Gururani and Alexander Lerch.  Semi-supervised audio classification with partially labeled data.  In _2021 IEEE International Symposium on Multimedia (ISM)_ , pages 111–114, 2021.  doi:10.1109/ISM52913.2021.00027.
  * Wang et al. [2019] Yu Wang, Ana Elisa Mendez Mendez, Mark Cartwright, and Juan Pablo Bello.  Active learning for efficient audio annotation and classification with a large amount of unlabeled data.  In _ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 880–884, 2019.  doi:10.1109/ICASSP.2019.8683063.
  * Shuyang et al. [2017] Zhao Shuyang, Toni Heittola, and Tuomas Virtanen.  Active learning for sound event classification by clustering unlabeled data.  In _2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 751–755, 2017.  doi:10.1109/ICASSP.2017.7952256.
  * Fonseca et al. [2020] Eduardo Fonseca, Shawn Hershey, Manoj Plakal, Daniel P. W. Ellis, Aren Jansen, and R. Channing Moore.  Addressing missing labels in large-scale sound event recognition using a teacher-student framework with loss masking.  _IEEE Signal Processing Letters_ , 27:1235–1239, 2020.  doi:10.1109/LSP.2020.3006378.
  * Lee and Lee [2021] Donmoon Lee and Kyogu Lee.  Cross-domain semi-supervised audio event classification using contrastive regularization.  In _2021 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)_ , pages 196–200, 2021.  doi:10.1109/WASPAA52581.2021.9632721.
  * Takagi et al. [2011] Jun Takagi, Yasunori Ohishi, Akisato Kimura, Masashi Sugiyama, Makoto Yamada, and Hirokazu Kameoka.  Automatic audio tag classification via semi-supervised canonical density estimation.  In _2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 2232–2235, 2011.  doi:10.1109/ICASSP.2011.5946925.
  * Chan et al. [2021] Teck Kai Chan, Cheng Siong Chin, and Ye Li.  Semi-supervised nmf-cnn for sound event detection.  _IEEE Access_ , 9:130529–130542, 2021.  doi:10.1109/ACCESS.2021.3113903.
  * De Benito-Gorrón et al. [2021] Diego De Benito-Gorrón, Daniel Ramos, and Doroteo T. Toledano.  A multi-resolution crnn-based approach for semi-supervised sound event detection in dcase 2020 challenge.  _IEEE Access_ , 9:89029–89042, 2021.  doi:10.1109/ACCESS.2021.3088949.
  * Hakkani-Tür et al. [2002] Dilek Hakkani-Tür, Giuseppe Riccardi, and Allen Gorin.  Active learning for automatic speech recognition.  In _2002 IEEE International Conference on Acoustics, Speech, and Signal Processing_ , volume 4, pages IV–3904–IV–3907, 2002.  doi:10.1109/ICASSP.2002.5745510.
  * Darnstädt et al. [2014] Malte Darnstädt, Hendrik Meutzner, and Dorothea Kolossa.  Reducing the cost of breaking audio captchas by active and semi-supervised learning.  In _2014 13th International Conference on Machine Learning and Applications_ , pages 67–73, 2014.  doi:10.1109/ICMLA.2014.16.
  * Qin et al. [2019] Xiao Qin, Wanting Ji, Ruili Wang, and ChangAn Yuan.  Learnt dictionary based active learning method for environmental sound event tagging.  _Multimedia Tools and Applications_ , 78, 2019.  doi:10.1007/s11042-018-7139-2.
  * Ji et al. [2019] Wanting Ji, Ruili Wang, and Junbo Ma.  Dictionary-based active learning for sound event classification.  _Multimedia Tools and Applications_ , 78, 2019.  doi:10.1007/s11042-018-6380-z.
  * Zeng [2019] Yun Zeng.  Context aware machine learning.  2019\.  doi:10.48550/ARXIV.1901.03415.
  * Hastie et al. [2009] Trevor Hastie, Robert Tibshirani, and Jerome Friedman.  _The Elements of Statistical Learning_.  Springer Series in Statistics. Springer New York Inc., 2009.
  * Song et al. [2022] Shen Song, Cong Zhang, and Zhihui Wei.  Research on scattering transform of urban sound events detection based on self-attention mechanism.  _IEEE Access_ , 10:120804–120822, 2022.  doi:10.1109/ACCESS.2022.3222495.
  * Theodorou et al. [2015] Theodoros Theodorou, Iosif Mporas, and Nikos Fakotakis.  Automatic sound recognition of urban environment events.  In Andrey Ronzhin, Rodmonga Potapova, and Nikos Fakotakis, editors, _Speech and Computer_ , pages 129–136. Springer International Publishing, 2015.  doi:10.1007/978-3-319-23132-7_16.
  * Su et al. [2019] Yu Su, Ke Zhang, Jingyu Wang, and Kurosh Madani.  Environment sound classification using a two-stream cnn based on decision-level fusion.  _Sensors_ , 19(7), 2019.  doi:10.3390/s19071733.
  * Li et al. [2022] Minze Li, Wu Huang, and Tao Zhang.  Attention based convolutional neural network with multi-frequency resolution feature for environment sound classification.  _Neural Processing Letters_ , 2022.  doi:10.1007/s11063-022-11041-y.
  * Salamon et al. [2014] Justin Salamon, Christopher Jacoby, and Juan Pablo Bello.  A dataset and taxonomy for urban sound research.  In _Proceedings of the 22nd ACM international conference on Multimedia_. ACM, 2014.  doi:10.1145/2647868.2655045.
  * Grosche et al. [2010] Peter Grosche, Meinard Müller, and Frank Kurth.  Cyclic tempogram—a mid-level tempo representation for musicsignals.  In _2010 IEEE International Conference on Acoustics, Speech and Signal Processing_ , pages 5522–5525, 2010.  doi:10.1109/ICASSP.2010.5495219.
  * Sreenivasa Rao and Manjunath [2017] K. Sreenivasa Rao and K E Manjunath.  _Speech Recognition Using Articulatory and Excitation Source Features_.  SpringerBriefs in Speech Technology. Springer Cham, 2017.  doi:10.1109/ICASSP.2012.6288282.
  * Sun and Ghaffarzadegan [2020] Yiwei Sun and Shabnam Ghaffarzadegan.  An ontology-aware framework for audio event classification.  In _ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 321–325, 2020.  doi:10.1109/ICASSP40776.2020.9053389.
  * Breunig et al. [2000] Markus M. Breunig, Hans-Peter Kriegel, Raymond T. Ng, and Jörg Sander.  Lof: Identifying density-based local outliers.  _SIGMOD Rec._ , 29(2):93–104, may 2000.  doi:10.1145/335191.335388.
  * Lopez-Meyer et al. [2021] Paulo Lopez-Meyer, Juan A. del Hoyo Ontiveros, Hong Lu, and Georg Stemmer.  Efficient end-to-end audio embeddings generation for audio classification on target applications.  In _ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 601–605, 2021.  doi:10.1109/ICASSP39728.2021.9414229.
  * Mu et al. [2021] Wenjie Mu, Bo Yin, Xianqing Huang, Jiali Xu, and Zehua Du.  Environmental sound classification using temporal-frequency attention based convolutional neural network.  _Scientific Reports_ , 11, 2021.  doi:10.1038/s41598-021-01045-4.
  * Dong et al. [2020] Xifeng Dong, Bo Yin, Yanping Cong, Zehua Du, and Xianqing Huang.  Environment sound event classification with a two-stream convolutional neural network.  _IEEE Access_ , 8:125714–125721, 2020.  doi:10.1109/ACCESS.2020.3007906.