# DeF-DReL: Systematic Deployment of Serverless Functions in Fog and Cloud
environments using Deep Reinforcement Learning

Chinmaya Kumar Dehury<EMAIL_ADDRESS>Shivananda Poojara<EMAIL_ADDRESS>Satish Narayana Srirama<EMAIL_ADDRESS>Institute of Computer
Science, University of Tartu, Tartu, Estonia  School of Computer and
Information Sciences, University of Hyderabad, Hyderabad 500046, India

###### Abstract

Fog computing is introduced by shifting cloud resources towards the users’
proximity to mitigate the limitations possessed by cloud computing. Fog
environment made its limited resource available to a large number of users to
deploy their serverless applications, composed of several serverless
functions. One of the primary intentions behind introducing the fog
environment is to fulfil the demand of latency and location-sensitive
serverless applications through its limited resources. The recent research
mainly focuses on assigning maximum resources to such applications from the
fog node and not taking full advantage of the cloud environment. This
introduces a negative impact in providing the resources to a maximum number of
connected users. To address this issue, in this paper, we investigated the
optimum percentage of a user’s request that should be fulfilled by fog and
cloud. As a result, we proposed DeF-DReL, a Systematic Deployment of
Serverless Functions in Fog and Cloud environments using Deep Reinforcement
Learning, using several real-life parameters, such as distance and latency of
the users from nearby fog node, user’s priority, the priority of the
serverless applications and their resource demand, etc. The performance of the
DeF-DReL algorithm is further compared with recent related algorithms. From
the simulation and comparison results, its superiority over other algorithms
and its applicability to the real-life scenario can be clearly observed.

###### keywords:

Serverless computing, fog computing, cloud computing, deep reinforcement
learning, serverless function deployment, function offloading.

††journal: Journal of XYZ

##  1 Introduction

Fog computing becomes an integral part of the cloud computing environment by
complementing the practical limitations of the cloud, such as reducing the
network overhead, network latency, enabling real-time service, leveraging the
QoS by providing location-sensitive services etc [1, 2, 3]. This is achieved
by shifting the computing resource to the end-user’s proximity. Fog computing
primarily differs from cloud computing in terms of its distance from the users
and the resource capacity. The fog environment is generally set up with
limited computing and storage resources and can provide service to a limited
number of users. Further, very selected small scale services are offered using
those limited resources, such as Internet of Thing (IoT) services, smart city
services, transportation and logistic services, smart irrigation and
agriculture services and many more [1, 4, 5].

The limited fog computing resources are also used to deploy customized users’
applications. For efficient and cost-effective utilization of the resources,
the fog environment provides a serverless platform that enables a user to
deploy a serverless application composed of numerous serverless functions,
thanks to container and microservice technology advancement [6]. Several
software solutions, such as OpenFaaS111https://www.openfaas.com/,
OpenWhisk222https://openwhisk.apache.org/, Kubeless333https://kubeless.io/,
etc., are available in the market to set up a serverless platform. These
serverless functions are generally developed for performing small tasks and
hence consume fewer resources and can be run within a container [7]. However,
in the case of the higher resource demand of these functions, the fog
environment may offload the corresponding serverless functions to the cloud
computing environment [8]. As a result, some portion of the whole applications
is deployed in fog, and the other portion is in the cloud environment. For
instance, a user sends a request to deploy a serverless application, as shown
in Figure 1, which consists of six serverless functions. Upon receiving the
application (Cloud App 1) by the fog serverless platform, two functions (SF1
and SF2) can be deployed on the fog environment and four functions (SF2, SF3,
SF4, and SF6) can be deployed on cloud environment, as shown in Figure 2.

Figure 1: An example of serverless cloud application .

To efficiently distribute the functions among fog and cloud environments,
several approaches are followed, such as the heuristic and AI-based
approaches. It is observed that heuristic approaches are mainly difficult when
it comes to considering a number of real-life parameters. On the contrary,
deep machine learning methods can analyse and process substantial complex
problems considering a large number of associated parameters [9]. Parameters
in this context include resource constraints of serverless platforms at both
fog and cloud environment, user’s distance, priority, functions resource
demand, energy consumption, the latency of the serverless applications, cost
of application deployment etc. In this paper, we take advantage of the
capability of Deep Reinforcement Learning (DRL), a division of artificial
intelligence, while deciding the hosting environment of each serverless
function.

###  1.1 Motivation and goal

It is observed from the recent literature survey that the existing workload
offloading solutions mainly focuses on assigning the maximum workload of a
particular application to the fog environment [10, 11, 12]. By doing so, the
quality of that application is further improved. Here, workload offloading
strategy mainly refers to the mechanism to distribute the workload of an
application among multiple computing environments. In this work, we consider
the deployment of serverless applications atop fog and cloud computing
environment. It is assumed that both the computing environments are equipped
with the required serverless platform. When the users request the nearby fog
computing environment (or the fog node) for their application deployment, the
fog node may offload some of the workloads to the cloud computing environment
(or cloud node) without violating the SLA (Service Level Agreement)
parameters. Assigning more workload to the fog node will significantly reduce
the network latency, where assigning more workload to the cloud node
significantly reduce the computation latency.

As given in the Figure 2 example, the serverless function SF1 and SF5 of Cloud
App 1 are assigned to the fog node, and the rest of the functions (SF2, SF3,
SF4, and SF6) are assigned to the cloud node. It is possible that all six
serverless functions are assigned to either the fog node or cloud node only.
By doing so, the Cloud App 1 may suffer from larger computation latency or
larger network latency, respectively. While deciding the hosting environment
of each serverless function, several parameters, such as resource availability
of fog node and resource demand of serverless functions, number of connected
users, etc., need to be taken into account. This leverages the optimization of
function deployment problem more complex.

Figure 2: A motivational example of serverless functions deployment in fog and
cloud.

Further assigning more number of serverless functions of a particular
application to the fog node makes the fog resources unavailable for other
serverless applications, which would become a biased solution. Such biased
solutions ignore the primary intention of a fog node, which is providing the
resources to the applications of the maximum number of connected users at
their proximity. This hinted at us to design a novel algorithm that tries to
assign the minimum number of serverless functions of an application to the fog
node and offload the maximum number of functions to the cloud node without
compromising the quality of service or the users’ constraints, which is the
goal of this paper. By doing so, the number of benefited users can further be
maximized. By doing so, the number of benefited users can further be
maximized. This motivates us to propose a systematic deployment of serverless
functions in fog and cloud environment, as discussed in further sections.
Towards achieving the goal, our contributions to this paper can be summarized
as below:

  * 1.

The problem of efficient deployment of serverless applications on fog and
cloud environments is investigated and formulated.

  * 2.

A novel strategy is designed and developed using the deep reinforcement
learning that tries to assign minimum number of serverless functions to fog
node.

  * 3.

A number of real-life parameters are incorporated while developing the agent
for the DRL method.

  * 4.

The developed DRL-based deployment strategy is implemented and compared for
the performance evaluation.

The rest of the paper is organized as follows: A systematic survey related to
the discussed research issue is presented in Section 2. The mathematical
formulation of the concerned serverless application deployment problem is
formulated in Section 3, followed by the proposed deployment strategy with the
detailed description in Section 4. The detailed implementation and performance
results are discussed in Section 5, followed by the concluding remarks and the
scope for improvements in Section 6.

##  2 Related works

Many research attempts have shown that serverless frameworks deployed on the
fog computing environment have several benefits over state of art resource
deployment strategies [13, 14, 15]. However, serverless function placement
over fog or cloud can be decided based on end-user expectations [16, 17, 18].
Along the side, recent advancements in Artificial Intelligence (AI) algorithms
can deliver the desired intelligent decisions [19]. Based on this context, we
now describe the literature survey and outline the works relevant to our
proposed work.

DRL is being a part of AI, applied in the diverse areas likely in the area of
edge and fog computing fields such as resource management [20, 21], service
orchestration [22, 23], resource protection [24, 25] and energy-efficient
resource management [26, 12]. However, the proposed work aligns in a similar
direction focusing on optimizing the placement of users’ serverless
application on fog and cloud environments based on real-life parameters, such
as priority of the serverless applications, resource constraints of each
serverless platform, distance and latency of the users’ from nearby fog node,
users’ priority, and their resource demand, etc.

Bermbach et al. [27], proposed auction-based function placement for optimal
decision to execute serverless functions on edge and cloud to maximize the
revenue considering the execution cost and arrival rate of users’ requests.
The major objective in [27] is to reduce cost without considering the latency
and priority needs of the user applications and resource demands. Similarly,
Cheng et al. [28] proposed a data-centric programming model called Fog
Function for placement and scaling of serverless functions based on data
context, system context, and usage context considering latency and flexibility
factors. This work is focused on latency and data-oriented requirements.
However, it lacks in considering nearby fog nodes and the priority of users’
requests.

Serverless function placement resembles the problem of service placement in
fog and clouds. Several attempts have been made to address this problem using
heuristic and machine learning-based approaches. Hamed et al. [11] proposed a
game theory to allocate the resource to IoT users’ in hierarchical computing
paradigm, including fog and remote cloud computing services. They formulated a
computation offloading game to model the competition between IoT users’ and
allocate the limited processing power of fog nodes efficiently. The concerned
research problem is purely modelled with the game theory-based heuristic
approach, which might be less sustainable in mobile and stochastic IoT
environments. Similarly, Du et al. [10] proposed a computation offloading
problem in a mixed fog/cloud system by jointly optimizing the offloading
decisions and the allocation of computation resource, transmit power, and
radio bandwidth while guaranteeing user fairness and maximum tolerable delay.

In our previous work [29], a novel DRL mechanism for optimized service
delivery by slicing the user requests is proposed that distribute the workload
among fog and cloud without comprising the service quality. The major
limitation of the proposed mechanism lies in its inability to take the user’s
priority and its inapplicability to the modern serverless platform. A deep
reinforcement learning method is used in Chen et al. [30] to improve the
performance of mobile edge computing. The problem of offloading is modelled
using Markov Decision Problem (MDP) and offloads the workload to cloud
computing if the mobile edge devices do not possess enough resources to handle
the workload. The Deep Sequential Model-Based on Reinforcement Learning method
is used by Wang et al. [31] to decide whether to offload the workload to cloud
computing. The latency, task dependency and the communication topology are
taken into consideration in the proposed mechanism. However, the proposed
strategy may not be able to take advantage of the available resources at the
nearby fog node, which could have further reduce the latency.

To the best of our knowledge and from the literature survey, it is observed
that the related recent works focus on using game theory, auction-based
approach, heuristic approaches etc. for strategic deployment of users’
applications considering a limited number of parameters making them not enough
matured and hard to apply in real life. To fill this gap between the
scientific research and its practical applicability, we propose a novel DRL-
based systematic deployment of users application on both nearby fog and cloud
environment. Unlike the existing works, the proposed strategy considers the
number of users connected to the fog node, their distance and network latency,
serverless computing and the associated constraints, such as short-run
function execution, limited resource demand, environment-specific resource
constraints, etc.

##  3 Problem formulation

With the above motivation and the goal we envisioned to achieve, in this
paper, we have considered an environment where many users send their resource
requests to a nearby fog node. The fog node is further backed by the
hypothetically infinite amount of resources of the cloud environment. However,
the number of fog nodes and cloud nodes can be increased to fit into the
desired situation. The fog node is responsible for providing service to all
the connected users with the intention to fulfil their requirements. The
user’s request can be interpreted as the deployment of their serverless
applications on the fog node. The serverless application is composed of
several serverless functions that can be deployed on either the fog
environment or cloud environment. In the following subsections, the detailed
modelling of deployment environments, the users, and their requests are
presented. For quick reference, we have included a list of all the notations
used in this manuscript in chronological order in Table-1.

Table 1: List of Notation. Notation |  Description
---|---
$\dot{A}$ |  The action space.
$C$ |  cloud environment
$\mathbbm{c}^{i}_{j}$ |  Computation latency of the serverless function $\hat{s}^{i}_{j}$
$\mathbbm{C}_{i}$ |  Computation latency of the SSR $s_{i}$
$d_{i}$ |  User’s distance from the Fog node
$D$ |  Maximum distance a user is allowed from Fog node
$F$ |  The fog node
$l^{f}$ |  Communication Latency between fog and cloud
$l_{i}$ |  Communication Latency between fog and user
$\hat{l}^{i}_{j}$ |  Communication latency of serverless function $\hat{s}^{i}_{j}$
$n$ |  Total number of users connected to the fog node
$P_{i}$ |  Priority of User $u_{i}$
$p^{\omega}$ |  Importance factor of latency and distance.
$\mathbbm{P}_{j}^{i}$ |  Priority of serverless function $\hat{s}^{i}_{j}$
$K_{j}^{i}$ |  Size of the serverless function code $\hat{s}^{i}_{j}$.
$\hat{K}^{f}$ |  Function code size limit in the fog’s serverless platform.
$\hat{K}^{c}$ |  Function code size limit in the cloud’s serverless platform.
$\kappa_{j}^{i}$ |  Represent the size of the INPUT to the function $\hat{s}^{i}_{j}$.
$\hat{\kappa}^{f}$ |  The input limit to any function in the fog’s serverless platform.
$\hat{\kappa}^{c}$ |  The input limit to any function in the cloud’s serverless platform.
$\hat{R}^{f}(x)$ |  Upper limit of resource demand of serverless function in FOG
$\hat{R}^{c}(x)$ |  Upper limit of resource demand of serverless function in Cloud
$\hat{r}^{i}_{j}(x)$ |  Upper limit resource demand of a serverless function $\hat{s}^{i}_{j}$
$r^{i}_{j}(x)$ |  Total resource demand of a serverless function $\hat{s}^{i}_{j}$
$S$ |  SSR Bucket or set of SSRs from at most n number of users; maximum of one from each user
$s_{i}$ |  SSR from the user $u_{i}$
$s_{j}^{i}$ |  Serverless function belong to SSR $s_{i}$

###  3.1 Deployment environments: Fog and Cloud

Let $F$ be the Fog Node (FN) equipped with a limited amount of resources. The
resource here refers to CPU, RAM, Storage, and Network I/O. The FN is
responsible for providing services/resources to a maximum number of connected
users. The cloud node, denoted by $C$, is equipped with a large number of
interconnected high-end servers. The FN is responsible for receiving the
request from the users to deploy their serverless application that composed of
a number of serverless functions. Upon receiving the request, the fog node
decides the deployment environment of each function. In such a decision
process, the communication latency, denoted by $l^{f}$, is considered as one
of the major parameters.

###  3.2 Modelling of Users

Let $n$ be the total number of users connected to the FN.
$U=\\{u_{1},u_{2},\dots,u_{n}\\}$ represents the set of all connected users.
Each user is associated with two major parameters: (a) geographic distance
from the FN and (b) the communication latency from the FN. $(x,y)$ and
$(p_{i},q_{i})$ denotes the location of FN $F$ and user $u_{i}$, respectively.
The shortest possible path through space between fog and the user can be
calculated using the Euclidean distance method, as follows:

$d_{i}=\sqrt{(p_{i}-x)^{2}+(q_{i}-y)^{2}}$ (1)

Let $D$ be the maximum distance that a user can access the service from the
nearby fog node. Based on $D$ value and the current distance of the user from
the FN, the distance-based priority can be calculated as follows:

$P_{i}^{d}=\frac{d_{i}}{D}$ (2)

It can be seen that the value of $P_{i}^{d}$ always lies between $0$ and
$1$(inclusive). A higher value of $P_{i}^{d}$ indicates the longer user’s
distance from FN. Like the distance, the communication latency plays a major
role in the deployment environment selection process for each serverless
function. The communication latency is nothing but the Round Trip Delay (RTD)
for a single data packet between the user and the FN. It is evident that
latency increases when the distance between the user and FN increases.
However, this may not hold due to the local environmental characteristics and
physical obstructions and hence the users with a shorter distance from the FN
may not experience lower latency [32]. Let $l_{i}$ be the latency for the user
$u_{i}$. Based on the latency, the priority of a user $u_{i}$ can be
calculated as

$P_{i}^{l}=\frac{l_{i}}{max\\{l_{j}\\}},\forall u_{j}$ (3)

By taking the distance-based priority, as in Equation 2 and the latency-based
priority, as in Equation 3, into account, the overall priority, $P_{i}$, of
the user $u_{i}$, can be calculated as below.

$P_{i}=[p^{\omega}*P_{i}^{d}]+[P_{i}^{l}*(1-p^{\omega})]$ (4)

where, $p^{\omega}$ represents the importance factor of latency and the
distance-based priority. $p^{\omega}$ is useful for the fog environment to
give priority to the latency parameter and distance. If $p^{\omega}$ is high,
the fog environment gives higher priority to the “distance” parameter and will
have negligible importance to the “latency” parameter. In other words, users
will be assigned with the priority value based on their distance from the fog
node. This is useful for the service provider when more number of users are
far from the fog node (towards the edge of the communication range of the fog
node). Further, the service requests from the users with high priority, i.e.
high $P_{i}$ value, will be preferably handled by the fog environment instead
of the cloud environment. It is assumed that the value of $p^{\omega}$ will be
determined by the service provider. From the above Equation 4, it can be
concluded that the value of $P_{i}$ is dynamic and depends on user’s current
location and latency and will be the part of SLA.

###  3.3 Serverless Service Request (SSR)

As discussed before, users send their requests to the FN to deploy their
serverless applications. Each such request is called Serverless Service
Request (SSR). Let $S=\\{s_{i},s_{2},\dots,s_{n}\\}$ be the set of such SSRs,
also known as SSR bucket. The SSR bucket may contain a maximum of $n$ number
of requests, and each user can send a maximum of one SSR. The term $s_{i}$
represents a SSR from user $u_{i}$. Further, each SSR $s_{i}$ is nothing but a
serverless cloud application that is to be deployed on FN and cloud
environment. The SSR $s_{i}$ is composed of a set of serverless functions and
can be defined as
$s_{i}=\\{\hat{s}_{1}^{i},\hat{s}_{2}^{i},\hat{s}_{3}^{i},\dots\\}$. It is
also possible that the set $s_{i}$ can be an empty set representing no request
from the user $u_{i}$. All the SSRs may composed of different number of
serverless functions, i.e. $|s_{i}|\neq|s_{k}|,i\neq k,\forall s_{i},s_{k}\in
S$.

The example in Figure 3 further illustrates the SSR and SSR bucket. In the
example, the SSR bucket contains the three SSRs ($SSR1,SSR2,$ and $SSR3$) from
users $USER1,USER2,$ and $USER3$, respectively. $SSR1$ serverless cloud
application from $USER1$ is composed of four serverless functions ($Fun11$,
$Fun12$, $Fun13,$ and $Fun14$). Similarly, the SSRs from $USER2$ and $USER3$
are composed of two and three serverless functions, respectively. As a whole,
upon receiving the SSR bucket, the FN needs to decide the hosting serverless
platform (in fog or in the cloud) of all the nine serverless functions.

Figure 3: An example of SSR bucket.

The boolean variable $f(\hat{s}_{j}^{i})$, where $f(\hat{s}_{j}^{i})=1$,
indicates that the serverless function $\hat{s}_{j}^{i}$ is deployed on FN,
else deployed on cloud. Similarly, the counterpart boolean variable
$c(\hat{s}_{j}^{i})$ indicates if the function $\hat{s}_{j}^{i}$ is deployed
on cloud environment. Both the boolean variables are counterpart to each other
and must follow condition $f(\hat{s}^{i}_{j})+c(\hat{s}^{i}_{j})=1$.

####  3.3.1 Resource demand

The functions are deployed on the either fog or cloud serverless platforms as
discussed earlier. Each serverless platform posses different resource limits.
In other words, each serverless functions have the limit on use of different
resource types. It is assumed that, the resource limit in fog serverless
platform is less than that of in the cloud serverless platform. Let
$\hat{R}^{f}(x)$ and $\hat{R}^{c}(x)$ be the upper limit of type $x$ resource
consumption for any serverless function in fog and cloud environment,
respectively, where $x\in\\{CPU,RAM,Disk,NetworkI/O\\}$. The value of
$\hat{R}^{f}(x)$ and $\hat{R}^{c}(x)$ are decided by serverless platform
service provider. For any serverless function $\hat{s}^{i}_{j}\in s_{i}$, the
maximum amount of type $x$ resource demand can be defined as

$\hat{r}^{i}_{j}(x)=\left[f(\hat{s}^{i}_{j})*\hat{R}^{f}(x)\right]+\left[c(\hat{s}^{i}_{j})*\hat{R}^{c}(x)\right]$
(5)

The value of $\hat{r}^{i}_{j}(x)$ depends on its deployment environment. If
the serverless function is deployed on fog environment, the maximum amount of
type $x$ resource it can demand would be $\hat{R}^{f}(x)$, $\hat{R}^{c}(x)$
otherwise.

However, to maintain the workload balance on different resources, the Resource
Importance Factor, denoted by $r^{\omega}(x)$, is introduced to handle the
imbalanced resource availability in both environments. Imbalanced resource
availability refers to the nonuniform percentage of resource availability in a
particular computing environment. It is assumed that the value of
$r^{\omega}(x)$ will be decided by the service provider and may need to update
on a timely basis, based on resource availability.

The resource demand is not fixed in today’s serverless computing. A serverless
function may need more resource on the fly. The resource requirement of a
serverless function is of two types: (a) Base resource requirement, denoted by
$\check{r}^{i}_{j}(x)$ and (b) supplementary resource requirement, denoted by
$\bar{r}^{i}_{j}(x)$. The base resource demand refers to the minimum demand of
type $x$ resource by the function $\hat{s}^{i}_{j}$. Each serverless function
may process a variable amount of data given as the input. A serverless
function may demand additional resource, also known as a supplementary
resource, to process that input data. Upon combining the base and the
supplementary resource demand, the total resource demand of a serverless
function can be calculated as below.

$r^{i}_{j}(x)=\left[\check{r}^{i}_{j}(x)+\bar{r}^{i}_{j}(x)\right]\leq\hat{r}^{i}_{j}(x),\forall
x$ (6)

The total base resource requirement of a SSR $s_{i}$ can be calculated as the
sum of base resource requirement of all serverless functions. Mathematically,

$\check{R}_{i}(x)=\sum_{\forall\hat{s}^{i}_{j}\in
s_{i}}\check{r}^{i}_{j}(x),\quad\forall x$ (7)

Similarly, the total supplementary resource requirement of the SSR $s_{i}$ can
be calculated as below.

$\bar{R}_{i}(x)=\sum_{\forall\hat{s}^{i}_{j}\in
s_{i}}\bar{r}^{i}_{j}(x),\quad\forall x$ (8)

The total resource requirement of a SSR can be calculated either by summing up
the total resource requirements of all serverless functions or by adding the
value in Equation 7 and 8. As discussed before that, each serverless function
is designed with a dedicated functionality having a major or minor
contribution to the whole serverless application.

####  3.3.2 Priority of SSR

Based on the functionality, users can decide how critical each function is,
denoted by $\hat{\mathbbm{q}}^{i}_{j}$. Eventually, this critical value of the
serverless functions will also play a major role in deciding the deployment
environment. The value of $\hat{\mathbbm{q}}^{i}_{j}$ ranges from $1$ through
$5$. Function with critical value $5$ can be interpreted as a high priority
function and must be deployed on the serverless platform of nearby FN,
provided the function’s resource demand is within the limit. On the contrary,
a function with a critical value $1$ should be deployed on the serverless
platform in the cloud environment. Similar to the criticality of a function,
$\mathbbm{q}_{i}$ denotes the criticality of the SSR $s_{i}$.

Serverless platforms are most suitable for the short run and small-sized
functions. In other words, it is not recommended to deploy very large
functions that take hours to finish their intended execution. To fit into such
a scenario, the size of the serverless functions and the size of their input
are incorporated. The size of the serverless function refers to the size of
the code and the dependent library files. On the other hand, input size mainly
refers to the size of the data passed to the function as an argument.
$K_{j}^{i}$ and $\kappa_{j}^{i}$ denotes the size of the serverless function
$\hat{s}^{i}_{j}$ and its input size, respectively. Input size represents the
approximate amount of data passed to the function as an argument. On the other
hand, the serverless platform setup in fog and cloud environment possess their
own limitations. Let $\hat{K}^{f}$ and $\hat{K}^{c}$ denote the limit on
function code size in fog and in the cloud environment, respectively.
Similarly, $\hat{\kappa}^{f}$ and $\hat{\kappa}^{c}$ denote the limit on input
size to the serverless functions in fog and in cloud environments,
respectively. Smaller function size and functions having smaller input size
have a higher preference to be deployed in the fog environment. The priority
of a serverless function, denoted by $\mathbbm{P}^{i}_{j}$, is derived from
it’s size ($K_{j}^{i}$), it’s input size ($\kappa_{j}^{i}$), and user-
specified critical value ($\hat{\mathbbm{q}}^{i}_{j}$). Mathematically,

$\displaystyle\mathbbm{P}^{i}_{j}=1\bigg{/}\left(\Delta+\left[\frac{5-\hat{\mathbbm{q}}^{i}_{j}}{5}\right]*\left[\frac{\max_{\forall\hat{s}^{i}_{j}\in
s_{i}}\\{K_{j}^{i}\\}-K_{j}^{i}}{\max_{\forall\hat{s}^{i}_{j}\in
s_{i}}\\{K_{j}^{i}\\}}\right]\right.$
$\displaystyle\left.*\left[\frac{\max_{\forall\hat{s}^{i}_{j}\in
s_{i}}\\{\kappa_{j}^{i}\\}-\kappa_{j}^{i}}{\max_{\forall\hat{s}^{i}_{j}\in
s_{i}}\\{\kappa_{j}^{i}\\}}\right]\right)$ (9)

, where $\Delta$ is a small fractional constant, used to avoid divide by zero
situation. The terms in the denominator are normalised to the range between 0
and 1. This is primarily to handle the different units of function size
($K_{j}^{i}$), input size ($\kappa_{j}^{i}$), and user-specified critical
value ($\hat{\mathbbm{q}}^{i}_{j}$) parameters. Higher the priority value,
higher preference for the fog node. It is to be noted that the functions
having the same characteristics and are from different SSRs may have different
priority value, as it depends on the characteristics of other functions in the
same SSR.

####  3.3.3 Communication latency of SSR

Similar to the user’s latency, each serverless function posses a communication
latency denoted by $\hat{l}^{i}_{j}$, which is defined by the summing up the
user’s latency and distance, as given in Equation 4 and the communication
latency between fog and cloud environment (in case the function is deployed in
cloud). Mathematically,

$\hat{l}^{i}_{j}=P_{i}+c(\hat{s}^{i}_{j})*l^{f}$ (10)

Further, by summing up the communication latency of all serverless, we can
calculate the communication latency of a SSR $s_{i}$, as given in Equation 11.

$\mathbbm{L}_{i}=\sum_{\forall\hat{s}^{i}_{j}\in s_{i}}\hat{l}^{i}_{j}$ (11)

####  3.3.4 Computation latency of SSR

Similar to the communication latency, each serverless function also posses a
computation latency, which depends on the amount of total resource demand, the
upper limit of the resource allocation set by the serverless function and the
communication latency between fog and cloud environment. Mathematically,

$\displaystyle\mathbbm{c}^{i}_{j}=\left[\sum_{\forall
x\in\\{CPU,RAM,Storage\\}}\frac{r^{i}_{j}(x)}{\hat{r}^{i}_{j}(x)}*r^{\omega}(x)\right]+$
$\displaystyle\left[\frac{r^{i}_{j}(I/O)}{\hat{r}^{i}_{j}(I/O)}*r^{\omega}(I/O)*\left[f(\hat{s}^{i}_{j})*l_{i}+c(\hat{s}^{i}_{j})*l^{f}\right]\right]$
(12)

The average computation latency of a SSR $s_{i}$ can be calculated by taking
the computation latency of all serverless functions, as given in Equation 13.

$\mathbbm{C}_{i}=\sum_{\forall\hat{s}^{i}_{j}\in
s_{i}}\mathbbm{c}^{i}_{j}*\frac{\hat{\mathbbm{P}}^{i}_{j}}{\max_{\forall\hat{s}^{i}_{j}\in
s_{i}}\\{\hat{\mathbbm{P}}^{i}_{j}\\}}$ (13)

###  3.4 Objective function

Considering the motivation and system model of deployment environments, users,
and serverless applications, this research work aims to fulfil the purpose of
the fog computing environment by providing the service to a maximum number of
users. The primary purpose of fog computing’s introduction is to provide real-
time, geographic data-sensitive service, and many more, as discussed in
Section 1. For this, it is necessary to accommodate a maximum number of
connected user, which can be achieved by deploying only the required service
and offload the unnecessary services to the cloud. Such a problem can be
mathematically formulated as follows.

Objective function for SSR $s_{i}$:

$Minimize~{}z_{i}=\mathbbm{L}_{i}+\mathbbm{C}_{i}$ (14)

The Equation 14 indicates the objective while processing the request $s_{i}$
from the user $u_{i}$. The objective here is to minimize both the
communication latency, as defined in Equation 11 and computation latency, as
defined in Equation 13. However, at any particular time, fog environment
receives a set of requests from the connected users, which is denoted as the
SSR bucket $S$. Hence it is essential to formulate such a multi-objective
optimization (MOO) problem for the whole SSR bucket $S$ and accordingly can be
written in following manner ([33]) in Equation 15.

Objective function for the SSR bucket $S$:

$\displaystyle\min z_{i},1\leq i\leq n$ (15)

Constraints:

$|s_{i}|>0,\quad 1\leq i\leq n$ (16) $if\text{
}f(\hat{s}^{i}_{j})=1,K_{j}^{i}\leq\hat{K}^{f},\forall\hat{s}^{i}_{j}\in
s_{i}\text{ and }\forall s_{i}\in S$ (17) $if\text{
}c(\hat{s}^{i}_{j})=1,K_{j}^{i}\leq\hat{K}^{c},\forall\hat{s}^{i}_{j}\in
s_{i}\text{ and }\forall s_{i}\in S$ (18) $if\text{
}f(\hat{s}^{i}_{j})=1,\kappa_{j}^{i}\leq\hat{\kappa}^{f},\forall\hat{s}^{i}_{j}\in
s_{i}\text{ and }\forall s_{i}\in S$ (19) $if\text{
}c(\hat{s}^{i}_{j})=1,\kappa_{j}^{i}\leq\hat{\kappa}^{c},\forall\hat{s}^{i}_{j}\in
s_{i}\text{ and }\forall s_{i}\in S$ (20) $if\text{
}f(\hat{s}^{i}_{j})=1,r_{j}^{i}(x)\leq\hat{R}^{f}(x),\forall\hat{s}^{i}_{j}\in
s_{i}\text{ and }\forall s_{i}\in S$ (21) $if\text{
}c(\hat{s}^{i}_{j})=1,r_{j}^{i}(x)\leq\hat{R}^{c}(x),\forall\hat{s}^{i}_{j}\in
s_{i}\text{ and }\forall s_{i}\in S$ (22) $\sum_{\forall x}r^{\omega}(x)=1$
(23)

The constraints for above objectives function can be summarized as below:

  * 1.

Constraint 16 ensures that the SSR bucket contains the SSRs that are composed
of at least one serverless function.

  * 2.

Constraint 17 and 18 ensure the size of all the serverless functions are not
more than the function size limit of the corresponding hosting serverless
platform.

  * 3.

Similarly, Constraint 19 and 20 ensure the size of input to all the serverless
functions are not more than the limit of the corresponding hosting serverless
platform.

  * 4.

Constraint 21 and 22 ensure that the resource demand of the serverless
functions are less than the resource limit imposed by the fog and cloud
environments, respectively.

  * 5.

Constraint 23 ensures that the sum of resource importance factor for all
different types of resource is 1.

##  4 Proposed solution: DeF-DReL

The previous section discusses the serverless function distribution research
challenge. In a nutshell, when a user sends a serverless-based application to
the nearby FN, what percentage of the total workload (or the total number of
serverless function) should be handled by the fog node and cloud node. In
other words, which serverless functions should be deployed on FN and should be
deployed on the cloud environment. As discussed, it is essential to deploy a
minimum number of functions on FN that minimizes the communication and
computation latency without compromising the QoS parameters. Keeping this
objective in mind, we have proposed a Deep Reinforcement Learning-based
serverless function deployment strategy for fog and cloud computing
environments, named as DeF-DReL.

In this section, we will be mainly focusing on the working principle of the
proposed deployment strategy that uses Deep Reinforcement Learning (DRL) as a
tool in the assessment of which deployment environment would be best suited
for each serverless function. A similar approach is used in our previous work
[29] that focuses on an efficient service dispersal mechanism for both fog and
cloud environment using DRL. In the DRL approach, a branch of ML and based on
Artificial Neural Network (ANN), the features are extracted in a layer-wise
fashion unlike the large matrix of state-action pairs, in conventional
Q-learning algorithm. This allows us the obtain finer result in a
comparatively large-scale simulation/experimental environment [29]. As
discussed in Section 1, the fundamental components are the state space of the
agent, action space that the agent will follow to move forward and all
possible rewards for each possible action.

###  4.1 State space

The state-space consists of all possible combinations of the states of all
serverless functions. The state of a serverless function indicates the
environment where it is hosted. Mathematically,

$\dot{s}_{j}^{i}=<f(\hat{s}^{i}_{j}),c(\hat{s}^{i}_{j})>$ (24)

, where $\dot{s}_{j}^{i}$ is the state of a serverless function. For instance,
$\dot{s}_{5}^{2}=<1,0>$ indicates that the serverless function
$\hat{s}^{5}_{2}$ is hosted on fog computing environment, which is the state
of the this serverless function. However, the state of a serverless function
can be $\dot{s}_{j}^{i}=<0,0>$, which represents that the function is not
assigned to any of the computing environment. By combining the states of all
the serverless functions, the state of a SSR $s_{i}$, denoted by $\dot{s}_{i}$
can be formulated as
$\dot{s}_{i}=<\dot{s}_{1}^{i},\dot{s}_{2}^{i},\dot{s}_{3}^{i},\dots>$.
Further, the state of the whole SSR bucket $S$ can be represented as
$\dot{S}=<\dot{s}_{1},\dot{s}_{2},\dots\dot{s}_{n}>$.

###  4.2 Action space

The action of an agent represents its movement from one state to the other
state of the SSR bucket. In other words, an action represents the assignment
of a serverless function either onto the fog environment or onto the cloud
environment. For the agent and SSR bucket $S$, the total action space, denoted
as $\dot{A}$ is composed of all the possible movements. The possible actions
for an agent are (a) $\dot{a}^{f}$ representing the actions where the function
is assigned to FN and (b) $\dot{a}^{c}$, representing the actions where the
function is assigned to cloud computing environment.

###  4.3 Rewards

When the agent transitions from one state to another by taking a particular
action, its main objective is to optimize the reward. The one-step reward can
be broken down to reward on assigning each serverless function present in the
SSR bucket. It is to be noted that the reward gained depends on the action
taken by the agent. For each action applied to the whole SSR bucket by the
agent, the reward gained can be calculated as the average reward gained by
applying the same action to all the SSRs. Mathematically,

$\dot{S}=\left(\frac{1}{|S|}\right)\sum_{\forall s_{i}\in S}\dot{r}(s_{i})$
(25)

, where $\dot{r}(s_{i})$ represents the total reward gained by imposing an
action to all the serverless functions present in the SSR $s_{i}$. This can be
calculated as below,

$\dot{r}(s_{i})=\sum_{\forall\hat{s}^{i}_{j}\in
s_{i}}\left[\\{f(\hat{s}^{i}_{j})*\dot{r}^{f}(\hat{s}^{i}_{j})\\}+\\{c(\hat{s}^{i}_{j})*\dot{r}^{c}(\hat{s}^{i}_{j})\\}\right]$
(26)

, where $\dot{r}^{f}(\hat{s}^{i}_{j})$ and $\dot{r}^{c}(\hat{s}^{i}_{j})$ are
the rewards gained by the agent by imposing the action $\dot{a}^{f}$ and
$\dot{a}^{c}$ on the serverless function $\hat{s}^{i}_{j}$, as calculated in
Equation 27 and 28, respectively. When a serverless function is assigned to
fog node, i.e. if the action $\dot{a}^{f}$ is imposed on a function
$\hat{s}^{i}_{j}$, the reward value is in the form of both computation and
communication latency. All together the total reward gained by applying action
$\dot{a}^{f}$ on the serverless function $\hat{s}^{i}_{j}$ is calculated as

$\begin{split}\dot{r}^{f}(\hat{s}^{i}_{j})&=\left[\sum_{\forall
x\in\\{CPU,RAM,Storage\\}}\frac{r_{j}^{i}(x)}{\hat{R}^{f}(x)}*r^{\omega}(x)\right]\\\
&+\left[\frac{r_{j}^{i}(I/O)}{\hat{R}^{f}(I/O)}*r^{\omega}(I/O)*l_{i}\right]+P_{i}\end{split}$
(27)

, where $P_{i}$ is the rewards in the form of communication latency and the
rest of the terms in above Equation 27 are the rewards in the form of
computation latency. Similarly, the total reward that can be gained by
imposing the action on $\dot{a}^{c}$ on the serverless function
$\hat{s}^{i}_{j}$ is calculated as

$\begin{split}\dot{r}^{c}(\hat{s}^{i}_{j})&=\left[\sum_{\forall
x\in\\{CPU,RAM,Storage\\}}\frac{r_{j}^{i}(x)}{\hat{R}^{c}(x)}*r^{\omega}(x)\right]\\\
&+\left[\frac{r_{j}^{i}(I/O)}{\hat{R}^{c}(I/O)}*r^{\omega}(I/O)*\\{l_{i}+l^{f}\\}\right]+P_{i}+l^{f}\end{split}$
(28)

##  5 Performance evaluation

The proposed DeF-DReL algorithm is evaluated with small scale simulation. This
section mainly devoted to providing detailed information on the simulation
setup/environment, performance evaluation matrix, simulation results and
discussions. The DeF-DReL algorithm deploys the serverless applications on
both fog and cloud’s serverless platform by assessing several properties and
requirements of the serverless applications. For comparison purpose,
Computation Offloading Game (COG) [11] and Computation Offloading and Resource
Allocation (CORA) [10] algorithms are used. Shah-Mansouri et al. [11] study
the problem of resource allocation of fog and cloud computing to the connected
IoT users and proposed a computation offloading game (COG) algorithm. The IoT
devices offload the tasks to the connected fog nodes and the cloud node with
the goal to maximize the quality of experience. Similarly, Jianbo et al. in
[10] investigated the computation offloading problem and proposed an
algorithm, named CORA, that considers a similar environment where user
equipment offloads the request with the tasks’s metadata. The tasks can be
executed on the local device along with the fog and cloud node maintaining the
energy consumption and execution delay balanced. For the comparison, we have
used the function distribution as one of the matrices that indicate what
percentage of functions in the serverless application are assigned to fog and
cloud serverless platforms. This paper also provides a more in-depth
standalone analysis of the DeF-DReL algorithm’s performance in subsequent
subsections.

###  5.1 Environment setup

For the implementation of the DeF-DReL algorithm, which is basically a RL
based algorithm, Tensorflow 444https://www.tensorflow.org/ version 1.5.0 open-
source machine learning platform with Python version 3.6.8 is used as the
programming language through Jupyter Notebook version 4.4.0
555https://jupyter.org/. Keras version 2.3.0 API is used atop Tensorflow
framework. For the implementation of the proposed DeF-DReL algorithm, the
total number of serverless applications ranges from $4$ through $10$ composing
of $10$ serverless functions in each application. The size of a serverless
function ranges from $10$ MB through $500$ MB, whereas the input size ranges
from $100$ MB through $2500$ MB. Summarizing the resource requirement of
serverless functions, the CPU, memory (RAM), storage, and network I/O resource
requirements range from between $1-4$, $100MB-2048MB$, $10MB-2048MB$ and
$10KBps-4096Kbps$, respectively. It is to be noted that the critical value of
the serverless function in the simulation is randomly distributed between the
value $1$ and $5$.

To align with the system model, there are two computing environments: Fog and
Cloud computing environment. Both the environments have their computing,
storage and network capabilities. With the available resource, the serverless
platform possesses various limits, such as CPU limit, memory/RAM limits, etc.
The list of parameters on resource limit, availability, requirements is given
in Table 2.

Table 2: List of Parameters Parameters | Value in ranges
---|---
|  Min value |  Max Value
SSR Bucket
Serverless functions/SSR |  4 |  10
# of SSRs |  4 |  10
SERVERLESS FUNCTION
Size |  10 |  500
Input size |  100 |  2500
RAM demand |  100 |  2048
CPU demand |  1 |  4
Storage Demand |  10 |  2048
NW I/O demand |  10 |  4096
Critical value |  1 |  5
COMPUTING ENVIRONMENT
|  CLOUD Environment |  Fog Environment
Input Limit |  2500 |  1500
CPU Limit |  6 |  2
RAM Limit |  5120 |  1024
Storage Limit |  10240 |  1024
NW I/O Limit |  10240 |  2048

###  5.2 Simulation Results

(a) Percentage of serverless functions in fog.

(b) Percentage of serverless functions in cloud.

Figure 4: Distribution of all serverless functions in fog and cloud.

The performance in terms of the distribution of the serverless functions among
fog and cloud serverless platforms of the proposed algorithm is evaluated by
comparing with the performance COG[11] and CORA[10] algorithms. Figure 4 shows
the distribution of serverless functions indicating what percentage of
functions is assigned to Fog serverless platform, as shown in Figure 4(a), and
cloud serverless platform, as shown in Figure 4(b), in case of the proposed
and the other related algorithms. The X-axis and Y-axis in those figures
represent the total number of serverless functions ranging from $10$ to $100$
and the percentage of serverless functions, respectively. It is to be noted
that the size of each SSR ranges between $1-10$ number of serverless
functions, and the number of SSRs is kept constant at $10$. The primary goal
of DeF-DReL is to assign a minimum workload to fog as compared to the cloud
environment without compromising the performance of the serverless application
or violating the deadline of the user’s application. On the other hand, the
goals of COG and CORA algorithms is to assign more work to the fog instead of
the cloud computing environment.

(a) Distribution of CPU resource demand between fog and cloud.

(b) Distribution of memory resource demand between fog and cloud.

(c) Distribution of storage resource demand between fog and cloud.

(d) Distribution of Network I/O resource demand between fog and cloud.

Figure 5: Distribution of different types of resource demand between fog and
cloud.

With the DeF-DReL algorithm, the total percentage of the serverless functions
that are assigned to fog serverless platform ranges from approximately $10\%$
to less than $30\%$, as in Figure 4(a). Whereas approximately $70\%-90\%$ of
the functions are assigned to cloud environment, as shown in Figure 4(b). A
very opposite trend is observed when COG[11] and CORA[10] algorithms are
implemented. In the case of both the related algorithms, as the number of
serverless functions increases, a steady growth in assigning functions to the
fog environment is observed. Approximately $79\%$ and $72\%$ of the total
$100$ serverless functions are assigned to fog when COG and CORA algorithms
are implemented, respectively. Similarly, using COG algorithm, approximately
$91\%$ of serverless functions are assigned to the cloud environment when a
total of $10$ functions are distributed. This percentage declined to $21\%$
when the number of functions increases to $100$, as in Figure 4(b). A similar
trend is also observed when the CORA algorithm is implemented, where
approximately only $28\%$ of the total functions are assigned to the cloud and
approximately $72\%$ of the total $100$ functions are assigned to the fog
environment, as in Figure 4(a). It is to be noted that no serverless functions
are kept unassigned. This is due to the implementation design, where the
learning agent needs to reach a state where all the functions are assigned to
either fog or cloud computing environment.

To further analyse the SSR workload distribution, we have investigated the
resource demand distribution from the simulation result shown in, as shown in
Figure 5. The implemented resource types are CPU, RAM or memory, storage and
the Network I/O. The X-axis in all the sub-figures represents the total number
of serverless functions, whereas the Y-axis represents the corresponding
distribution of resource types in percentage. The CPU resource demand of all
the functions allocated to the fog serverless platform is increasing from
approximately $3\%$ to $18\%$ when the total number of functions increases
from $10$ to $60$. However, the percentage of memory resource demand fulfilled
by the fog environment remains less than $20\%$ even when the number of
serverless functions is increased to $100$, as shown in Figure 5(a). A similar
pattern is also observed in the other types of resources. Approximately $14\%$
and $86\%$ of RAM resources are fulfilled by the fog and cloud environments,
respectively, when a total of $100$ serverless functions are deployed on both
the computing environments, as shown in Figure 5(b). With the same number of
serverless functions, i.e. $100$ functions, $16\%$ and $13\%$ of total storage
resource demand and the network resource demands are fulfilled by the fog
environment, as shown in Figure 5(c) and 5(d), respectively. In a nutshell, it
is observed that approximately $20\%$ of the total resource demands (including
all types of resources) are fulfilled by the fog environment. This indicates
that a larger number of serverless functions are mainly offloaded to the cloud
to keep the fog environment available enough to accommodate more number of
serverless functions from different users. One of the reasons behind such
resource demand distribution is due to the limitations imposed on the
serverless platform available in the fog environment, as shown in Table 2.
This is mainly due to the fact that the fog environments are setup with
limited resources with the intention to provide the necessary services to a
maximum number of users.

(a) Average size of the functions.

(b) Average input size to the functions.

Figure 6: Characteristics (function size and input size) of serverless
functions in fog and cloud.

(a) Average critical values of the functions.

(b) # of functions with critical value in fog and cloud.

(c) Critical values of the serverless functions in fog and cloud.

Figure 7: Characteristics (critical values) of Serverless functions in fog and
cloud.

As discussed above that most of the resource demands are mainly fulfilled by
the cloud environments, which allows the service providers to carefully use
the fog resources. However, analysing the characteristics of the deployed
serverless functions in both computing environments is an essential dimension
to investigate. Accordingly, the Figure 6, 7, and 8, we have observed the
size, input size, critical values and priority of the functions that are
deployed in FN and in cloud environment. The sub-figures 6(a) and 6(b) gives
an overall picture of the size and the input size of the functions. The size
of the function indicates the total size of the serverless function code,
which is in MegaByte (MB). The range of function code ranges from $10MB$ to
$500MB$. Figure 6(a) shows that the average function size deployed in the fog
environment is $250MB$, which is less than that in the cloud environment. This
average function size (deployed in FN) increases from less than $50~{}MB$ to
$250~{}MB$ when the number of serverless function increases from $10$ to $70$.
However, the value further decreases to approximately $210~{}MB$ when the
number of functions further increased to $100$. On the other hand, it is
observed that all the functions deployed in the cloud have a larger size
ranging between $270~{}MB$ and $350~{}MB$. A similar trend observed while
investigating the input size of those functions. The average size of the input
to the functions in FN lies below $750~{}MB$, whereas it is larger than
$1150~{}MB$ in the case of the functions deployed in cloud computing. This
confirms that the algorithm is distributing the function as per requirement,
keeping the load on the fog serverless platform minimal and offloading larger
functions to the cloud.

(a) Average priority values of the functions.

(b) priority values of the serverless functions in fog and cloud.

Figure 8: Characteristics (priority values) of Serverless functions in fog and
cloud.

To extend the simulation results’ analysis and make sure that Def-DReL does
not violate the constraints mentioned in Section 3.4, we have presented the
critical values of the functions deployed in fog and cloud environments, as in
Figure 7. Figure 7(a) shows the average critical value of the serverless
function both in the fig and in cloud environments. The average critical
values of the functions in a specific serverless platform are calculated by
calculating the ratio of the sum of critical values of all the functions and
the total number of serverless functions deployed in that serverless platform.
From the figure, it is concluded that in the case of the fog serverless
platform, the average critical values of the functions is less than the
functions in the cloud environment. To complement this result, we have also
analysed the total number of functions and percentage of all functions
deployed in fog and cloud, as in Figure 7(b). It is observed (also from Figure
7(c)) that a less number of functions are in the fog and a larger percentage
of the functions are deployed in the cloud environment. For instance, there
are a total of $15$ functions (out of $100$) are having the critical value
$5$, out of which $3$ functions (accounted for $20\%$) are deployed in fog
environment and the rest $12$ functions are deployed in the cloud environment.
Similarly, out of $100$ serverless functions, a total of $28$ functions have
the critical values $1$. Out of those $28$ functions, $12$ functions (i.e.
$42\%$) are deployed in fog and $16$ functions (i.e. $57\%$) are deployed in
cloud. Limit on the resource availability and the limit imposed by the fog
serverless platform are some of the main reasons behind deploying higher
critical valued functions on the cloud.

Based on the function size, input size and the critical values, the priority
value of a serverless function is calculated, as defined in Equation 3.3.2. To
analyse the priorities of the serverless functions, we have depicted the
simulation result in Figure 8. Figure 8(a) represents the average priority of
the serverless functions deployed in fog and cloud serverless platforms. The
X-axis represents the average priority value and the Y-axis represents the
total number of serverless functions. It is observed that when the number of
functions increases beyond $40$, the average priority of the functions in fog
become less than the functions in the cloud. This is due to the larger number
of serverless functions are in the cloud affecting the final average value. In
the case of $100$ functions, it is found that the average priority of the
functions in fog ($27$ functions) is $0.77$, whereas the average priority
value of functions in the cloud ($73$ functions) is $0.88$. From Figure 8(b),
it can be observed that most of the functions that are having the higher
priority values are deployed in fog environment. It is to be noted that within
an SSR, the functions with maximum resource demand or the highest critical
value is assigned with the priority value $5$ instead of the value calculated
using Equation 3.3.2.

##  6 Conclusions and future works

Efficient deployment of serverless applications, composed of several
serverless functions, on fog and cloud computing environments is investigated
in this paper. The underlined intention of introducing a fog computing
environment is to provide services to the maximum number of connected users
without compromising the service quality. To achieve this, this paper took
advantage of offloading the optimum percentage of the user’s request to cloud
computing. This paper took advantage of deep reinforcement learning and
proposed DeF-DReL, a systematic serverless functions deployment strategy for
fog and cloud environments. The DeF-DReL strategy uses several parameters,
such as distance and latency of the users from nearby fog node, user’s
priority in accessing the fog service, priority of the fog services, and
resource demand of the user’s request, to make it more realistic and
applicable to a real-life scenario. The performance of the proposed deployment
strategy is compared with the other existing strategies to observe its
superiority and ability to achieve the concerned goal.

To further make it more realistic and applicable to the real-life scenario, it
is necessary to incorporate a multi-fog and multi-cloud scenario, where a user
is connected to multiple fog nodes and each fog node is connected to other
multiple fog nodes, which is a part of our future works. Besides, to
understand the real behaviour and performance, the proposed DeF-DReL
deployment strategy will be implemented in a controlled lab environment in
future.

## Acknowledgment

We acknowledge financial support to UoH-IoE by MHRD, India (F11/9/2019-U3(A)).

## References

  * [1] M. Xu, W. Wang, M. Zhang, K. Cumanan, G. Zhang, Z. Ding, Joint optimization of energy consumption and time delay in energy-constrained fog computing networks, in: 2019 IEEE Global Communications Conference (GLOBECOM), 2019, pp. 1–6.
  * [2] F. Jalali, K. Hinton, R. Ayre, T. Alpcan, R. S. Tucker, Fog computing may help to save energy in cloud computing, IEEE Journal on Selected Areas in Communications 34 (5) (2016) 1728–1739.
  * [3] Z. M. Nayeri, T. Ghafarian, B. Javadi, Application placement in fog computing with ai approach: Taxonomy and a state of the art survey, Journal of Network and Computer Applications 185 (2021) 103078.
  * [4] T. T. Nguyen, V. N. Ha, L. B. Le, R. Schober, Joint data compression and computation offloading in hierarchical fog-cloud systems, IEEE Transactions on Wireless Communications 19 (1) (2020) 293–309.
  * [5] U. Majeed, L. U. Khan, I. Yaqoob, S. A. Kazmi, K. Salah, C. S. Hong, Blockchain for iot-based smart cities: Recent advances, requirements, and future challenges, Journal of Network and Computer Applications 181 (2021) 103007.
  * [6] I. Baldini, P. Castro, K. Chang, P. Cheng, S. Fink, V. Ishakian, N. Mitchell, V. Muthusamy, R. Rabbah, A. Slominski, et al., Serverless computing: Current trends and open problems, in: Research Advances in Cloud Computing, Springer, 2017, pp. 1–20.
  * [7] S. K. Mohanty, G. Premsankar, M. Di Francesco, et al., An evaluation of open source serverless computing frameworks., in: CloudCom, 2018, pp. 115–120.
  * [8] R. O. Aburukba, T. Landolsi, D. Omer, A heuristic scheduling approach for fog-cloud computing environment with stationary iot devices, Journal of Network and Computer Applications 180 (2021) 102994.
  * [9] J. Shuja, K. Bilal, W. Alasmary, H. Sinky, E. Alanazi, Applying machine learning techniques for caching in next-generation edge networks: A comprehensive survey, Journal of Network and Computer Applications 181 (2021) 103005\.
  * [10] J. Du, L. Zhao, J. Feng, X. Chu, Computation offloading and resource allocation in mixed fog/cloud computing systems with min-max fairness guarantee, IEEE Transactions on Communications 66 (4) (2018) 1594–1608.
  * [11] H. Shah-Mansouri, V. W. S. Wong, Hierarchical fog-cloud computing for iot systems: A computation offloading game, IEEE Internet of Things Journal 5 (4) (2018) 3246–3257.
  * [12] H. Sami, A. Mourad, H. Otrok, J. Bentahar, Demand-driven deep reinforcement learning for scalable fog and service placement, IEEE Transactions on Services Computing (2021) 1–1.
  * [13] S. Sarkar, R. Wankar, S. N. Srirama, N. K. Suryadevara, Serverless management of sensing systems for fog computing framework, IEEE Sensors Journal 20 (3) (2019) 1564–1572.
  * [14] M. Großmann, C. Ioannidis, D. T. Le, Applicability of serverless computing in fog computing environments for iot scenarios, in: Proceedings of the 12th IEEE/ACM international conference on utility and cloud computing companion, 2019, pp. 29–34.
  * [15] L. Baresi, D. F. Mendonça, Towards a serverless platform for edge computing, in: 2019 IEEE International Conference on Fog Computing (ICFC), IEEE, 2019, pp. 1–10.
  * [16] D. Bermbach, S. Maghsudi, J. Hasenburg, T. Pfandzelter, Towards auction-based function placement in serverless fog platforms, in: 2020 IEEE International Conference on Fog Computing (ICFC), IEEE, 2020, pp. 25–31.
  * [17] T. Rausch, A. Rashed, S. Dustdar, Optimized container scheduling for data-intensive serverless edge computing, Future Generation Computer Systems 114 (2021) 259–271.
  * [18] A. Das, S. Imai, S. Patterson, M. P. Wittie, Performance optimization for edge-cloud serverless platforms via dynamic task placement, in: 2020 20th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGRID), IEEE, 2020, pp. 41–50.
  * [19] Z. M. Nayeri, T. Ghafarian, B. Javadi, Application placement in fog computing with ai approach: Taxonomy and a state of the art survey, Journal of Network and Computer Applications (2021) 103078.
  * [20] S. Guo, Y. Dai, S. Xu, X. Qiu, F. Qi, Trusted cloud-edge network resource management: Drl-driven service function chain orchestration for iot, IEEE Internet of Things Journal 7 (7) (2019) 6010–6022.
  * [21] G. Qu, H. Wu, Dmro: A deep meta reinforcement learning-based task offloading framework for edge-cloud computing, arXiv preprint arXiv:2008.09930 (2020).
  * [22] S. Guo, Y. Qi, Y. Jin, W. Li, X. Qiu, L. Meng, Endogenous trusted drl-based service function chain orchestration for iot, IEEE Transactions on Computers (2021).
  * [23] L. Chen, J. Xu, Seek common while shelving differences: Orchestrating deep neural networks for edge service provisioning, IEEE Journal on Selected Areas in Communications 39 (1) (2020) 251–264.
  * [24] W. Liang, W. Huang, J. Long, K. Zhang, K.-C. Li, D. Zhang, Deep reinforcement learning for resource protection and real-time detection in iot environment, IEEE Internet of Things Journal 7 (7) (2020) 6392–6401.
  * [25] L. Yang, M. Li, Y. Zhang, P. Si, Z. Wang, R. Yang, Resource management for energy-efficient and blockchain-enabled industrial iot: A drl approach, in: 2020 IEEE 6th International Conference on Computer and Communications (ICCC), IEEE, 2020, pp. 910–915.
  * [26] Y. Liu, C. Yang, L. Jiang, S. Xie, Y. Zhang, Intelligent edge computing for iot-based energy management in smart cities, IEEE Network 33 (2) (2019) 111–117.
  * [27] D. Bermbach, S. Maghsudi, J. Hasenburg, T. Pfandzelter, Towards auction-based function placement in serverless fog platforms, in: 2020 IEEE International Conference on Fog Computing (ICFC), 2020, pp. 25–31.
  * [28] B. Cheng, J. Fuerst, G. Solmaz, T. Sanada, Fog function: Serverless fog computing for data intensive iot services, in: 2019 IEEE International Conference on Services Computing (SCC), IEEE, 2019, pp. 28–35.
  * [29] C. Kumar Dehury, S. N. Srirama, An efficient service dispersal mechanism for fog and cloud computing using deep reinforcement learning, in: 2020 20th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGRID), 2020, pp. 589–598.
  * [30] M. Chen, T. Wang, S. Zhang, A. Liu, Deep reinforcement learning for computation offloading in mobile edge computing environment, Computer Communications 175 (2021) 1–12.
  * [31] J. Wang, J. Hu, G. Min, W. Zhan, Q. Ni, N. Georgalas, Computation offloading in multi-access edge computing using a deep sequential model based on reinforcement learning, IEEE Communications Magazine 57 (5) (2019) 64–69.
  * [32] D. Tse, P. Viswanath, Fundamentals of wireless communication, Cambridge university press, 2005.
  * [33] N. Gunantara, A review of multi-objective optimization: Methods and its applications, Cogent Engineering 5 (1) (2018) 1502242.

Chinmaya Kumar Dehury received bachelor degree from Sambalpur University,
India, in June 2009 and MCA degree from Biju Pattnaik University of
Technology, India, in June 2013. He received the PhD Degree in the department
of Computer Science and Information Engineering, Chang Gung University,
Taiwan. Currently, he is a postdoctoral research fellow in the Mobile & Cloud
Lab, Institute of Computer Science, University of Tartu, Estonia. His research
interests include scheduling, resource management and fault tolerance problems
of Cloud and fog Computing, and the application of artificial intelligence in
cloud management. He is an reviewer to several journals and conferences, such
as IEEE TPDS, IEEE JSAC, Wiley Software: Practice and Experience, etc.

Shivananda R Poojara is a Ph.D student at Mobile and Cloud Computing
Laboratory, University of Tartu, Estonia. He is also working as a Junior
Research Fellow in IT Academy program. His research interests are serverless
computing, edge analytics, fog and cloud computing. He is a member of IEEE,
IET and ISTE. He has a 6+ years of teaching experience and worked at Nokia R&D
Labs as an intern.

Satish Narayana Srirama is an Associate Professor at the School of Computer
and Information Sciences, University of Hyderabad, India. He is also a
Visiting Professor and the honorary head of the Mobile & Cloud Lab at the
Institute of Computer Science, University of Tartu, Estonia, which he led as a
Research Professor until June 2020. He received his PhD in computer science
from RWTH Aachen University, Germany in 2008. His current research focuses on
cloud computing, mobile web services, mobile cloud, Internet of Things, fog
computing, migrating scientific computing and enterprise applications to the
cloud and large-scale data analytics on the cloud. He is an IEEE Senior
Member, an Editor of Wiley Software: Practice and Experience, a 51 year old
Journal, was an Associate Editor of IEEE Transactions in Cloud Computing and a
program committee member of several international conferences and workshops.
Dr. Srirama has co-authored over 165 refereed scientific publications in
international conferences and journals.